# Course - Discrete Stochastic Processes

Course Level: Master

Discrete stochastic processes are essentially probabilistic systems that evolve in time via random changes occurring at discrete fixed or random intervals. This course aims to help students acquire both the mathematical principles and the intuition necessary to create, analyze, and understand insightful models for a broad range of these processes. The range of areas for which discrete stochastic-process models are useful is constantly expanding, and includes many applications in engineering, physics, biology, operations research and finance.

Prerequisites
• Lecture 1 - Introduction and Probability Review
• Lecture 2 - More Review; The Bernoulli Process
• Lecture 3 - Law of Large Numbers, Convergence
• Lecture 4 - Poisson (the Perfect Arrival Process)
• Lecture 5 - Poisson Combining and Splitting
• Lecture 6 - From Poisson to Markov
• Lecture 7 - Finite-state Markov Chains; The Matrix Approach
• Lecture 8 - Markov Eigenvalues and Eigenvectors
• Lecture 9 - Markov Rewards and Dynamic Programming
• Lecture 10 - Renewals and the Strong Law of Large Numbers
• Lecture 11 - Renewals Strong Law and Rewards
• Lecture 12 - Renewal Rewards, Stopping Trials, and Wald's Inequality
• Lecture 13 - Little, MG1, Ensemble Averages
• Lecture 14 - Review
• Lecture 15 - The Last Renewal
• Lecture 16 - Renewals and Countable-state Markov
• Lecture 17 - Countable-state Markov Chains
• Lecture 18 - Countable-state Markov Chains and Processes
• Lecture 19 - Countable-state Markov Processes
• Lecture 20 - Markov Processes and Random Walks
• Lecture 21 - Hypothesis Testing and Random Walks
• Lecture 22 - Random Walks and Thresholds
• Lecture 23 - Martingales (Plain, Sub, and Super)
• Lecture 24 - Martingales Stopping and Converging
• Lecture 25 - Putting It All Together