Markov property, transition probabilities, transition matrix, Chapman Kolmogorov equation, irreducibility of a chain. Classification of states, recurrent states, transient states, null recurrent states and positive recurrent states, periodicity. Existence of stationary distribution, ergodic distribution, necessary and sufficient conditions for ergodicity. Random walks, branching processes, absorbtion probabilities, mean time to absorbtion. Markov reward chains algorithms and Markov Chain Monte Carlo.

Type of methodology: Combination of lecture and hands-on

Participants receive the certificate of attendance: Yes

Paid training activity for participants: Yes, for all

Participants prerequisite knowledge: Numerical methods (linear algebra, statistics); Domain-specific background knowledge