Convergence theorem on countable state spaces

Let $P$ be an irreducible and aperiodic Markov chain on a countable state space $V$ (nonnegative entries, rows summing to one as convergent series; every state reaches every other; the possible return times to each state have greatest common divisor one), and suppose some state is positive recurrent: its expected return time $\mathbb E_x(\tau^+_x)=\sum_{t\ge0}\mathbb P_x\{\tau^+_x>t\}$ is finite. Distances are measured in total variation via the $\ell^1$ formula $\|\mu-\nu\|_{TV}=\tfrac12\sum_y|\mu(y)-\nu(y)|$.

The theorem (Theorem 21.14 of Levin–Peres–Wilmer) asserts the existence of a distribution $\pi$ on $V$ such that:

1. $\pi$ is stationary: nonnegative, summing to one, with $\sum_x\pi(x)P(x,y)=\pi(y)$ for all $y$;
2. $\pi$ is the unique stationary distribution of the chain;
3. from every starting state $x$, $\;\bigl\|P^t(x,\cdot)-\pi\bigr\|_{TV}\to0$ as $t\to\infty$.

This is the Convergence Theorem of Mission II transplanted to countable state spaces, with positive recurrence supplying what finiteness gave for free. It is the fundamental theorem of applied Markov chain theory — queues, birth-and-death chains, random walks with drift — and the proof couples two copies of the chain on the product space, using aperiodicity and positive recurrence to force the copies to meet.