Theorem 5.6.6 — the convergence theorem

Let $p$ be a transition probability on a countable state space that is

1. irreducible — $\rho_{xy}>0$ for every pair of states,
2. aperiodic — every state has period one, the greatest common divisor of $I_x=\{n\ge1:p^n(x,x)>0\}$ being $1$, and
3. equipped with a stationary distribution $\pi$.

Then for all states $x$ and $y$,

The chain forgets its starting point: the distribution after $n$ steps converges to $\pi$ whatever it was at time zero. This is what makes a stationary distribution a long-run frequency rather than merely a fixed point of a linear map, and it is the theorem underlying Markov chain Monte Carlo, card shuffling, and the equilibrium analysis of queueing and inventory models.

Both hypotheses are needed and neither can be dropped. Without irreducibility the limit depends on which part of the state space the chain starts in. Without aperiodicity, $p^n(x,y)$ oscillates: a chain alternating between two halves of its state space never forgets the parity of the clock, and Durrett's Theorem 5.7.2 gives the corrected statement for that case. Periodicity, as the book puts it, is the only thing that can prevent convergence.

The standard proof is a coupling on $S\times S$: run two independent copies, one from $x$ and one from $\pi$, under $\bar p\bigl((x_1,x_2),(y_1,y_2)\bigr)=p(x_1,y_1)p(x_2,y_2)$. Aperiodicity and irreducibility make the product chain irreducible, $\pi\times\pi$ is stationary for it, so the two copies meet with probability one, and after they meet they may be exchanged.

Formalization Note The conclusion is pointwise convergence of the transition probabilities, not convergence in total variation and not with a rate; those are strengthenings and are listed under contributions. The transition probability is a function on a countable type with non-negative entries, summable rows and rows summing to one, and every quantity involved is defined by a recursion on the matrix rather than through a measure on path space.