Just signed up for this. It looks like a cool course, you should take a look.
https://www.coursera.org/course/modelthinkingLooking forward to section 10:
Markov Processes
In this section, we cover Markov Processes. Markov Processes capture dynamic processes between a fixed set of states. For example, we will consider a process in which countries transition between democratic and dictatorial. To be a Markov Process, it must be possible to get from any one state to any other and the probabilities of moving between states must remain fixed over time. If those assumptions hold, then the process will have a unique equilibrium. In other words, history will not matter. Formally, this result is called the Markov Convergence Theorem. In addition to covering Markov Processes, we will also see how the basic framework can be used in other applications such as determining authorship of a text and the efficacy of a drug protocol.