Markov chain

A Markov chain is a stateless statistical process. A Markov chain is fully defined by the state at the start of each step and does not need to "remember" any previous state. In a Markov chain, the Probability that a Random variable $X$ will take all the values in a sequence $(x_{1},\ldots,x_{N})$ in $N$ steps is

where $p_{1}(x_{1})$ is the probability of starting at state $x_{1}$ and $W$ is called the transition probability function, which changes a random variable's state. Note how $W$ determines how the chain runs and how it is only dependent on the current state: there is no "memory". A stateless process is generally said to be Markovian. A Markovian time series is commonly also said to have short memory.

Properties#

• The quantity $P_{N}$ can be interpreted as the Joint distribution function of $N$ random variables $X_{1},\ldots,X_{N}$. These variables are not independent since the value of one is needed to determine the value of the next.
• In a Markov chain, the variables of the sequence are conditionally independent. This is because each variable is only correlated with the previous one. The Conditional distribution function of the $i$-th step is $$$$