Certainty equivalence / separation theorem (§5.2)

This theorem (whose proof in the file is sorry, i.e., not supplied) asserts the following. Fix any dimensions $n, m, q \in \mathbb{N}$, any finite types $\Omega_0, \Omega_W, \Omega_V$, any model $M$ of type BertsekasLQGModel (horizon $N > 0$; matrices $A_k, B_k, C_k$ and cost matrices $Q_k$ positive semidefinite, $R_k$ positive definite for all $k \in \mathbb{N}$; finitely supported zero-mean disturbance distributions as detailed in the structure's read-back), and any policy $\pi^\star$ — an arbitrary function from finite lists of measurement vectors in $\mathbb{R}^q$ to controls in $\mathbb{R}^m$. Assume the hypothesis: for every time $k < N$ and every sample $\omega$ (triple of initial outcome and $N$-tuples of disturbance outcomes) whose product probability $\mathbb{P}_M(\omega) = p_0(\omega_0)\prod_{j<N}p_W(j,\omega_W(j))\prod_{j<N}p_V(j,\omega_V(j))$ is nonzero,

where: $Z_k^{\pi^\star}(\omega) = [z_0,\dots,z_k]$ is the measurement list produced by running the closed-loop trajectory recursion under $\pi^\star$ itself (states $x_0 = x_0(\omega_0)$, $x_{j+1} = A_j x_j + B_j \pi^\star([z_0..z_j]) + w(j,\omega_W(j))$ for $j<N$; measurements $z_j = C_j x_j + v(j,\omega_V(j))$ for $j<N$ and $z_N = C_N x_N$ noiselessly); $\hat{x}_k^{\pi^\star}(\omega)$ is the conditional-mean estimate of BertsekasLQGEstimate — the $\mathbb{P}_M$-weighted average of $x_k(\omega')$ over all samples $\omega'$ with $Z_k^{\pi^\star}(\omega') = Z_k^{\pi^\star}(\omega)$, equal to the zero vector by the $0^{-1}=0$ convention if all such samples have probability zero (which cannot occur here since $\mathbb{P}_M(\omega) \ne 0$ and $\omega$ matches itself); and $L_k$ is the gain matrix of BertsekasLQGGain, i.e. $L_k = -\big(B_k^\top P B_k + R_k\big)^{-1} B_k^\top P A_k$ with $P = \mathcal{K}_{N \dotminus (k+1)}$ from the backward Riccati recursion $\mathcal{K}_0 = Q_N$, $\mathcal{K}_{j+1} = A_{k'}^\top\big(\mathcal{K}_j - \mathcal{K}_j B_{k'}(B_{k'}^\top \mathcal{K}_j B_{k'} + R_{k'})^{-1} B_{k'}^\top \mathcal{K}_j\big)A_{k'} + Q_{k'}$ at $k' = N \dotminus (j+1)$, where the matrix inverses are the zero matrix whenever the matrix to be inverted is singular.

Under that hypothesis, the conclusion is: for every policy $\pi$ (again an arbitrary function from measurement lists to controls — the quantification imposes no structure whatsoever on the competitor),

where $J$ is the expected quadratic cost of BertsekasLQGCost: $J(\pi) = \sum_\omega \mathbb{P}_M(\omega)\big[x_N^\top Q_N x_N + \sum_{k<N}\big(x_k^\top Q_k x_k + u_k^\top R_k u_k\big)\big]$ with $x_k, u_k$ the closed-loop states and controls under $\pi$ for sample $\omega$. Points to note about the literal strength: the theorem does not assert that a policy satisfying the hypothesis exists — it only says that if $\pi^\star$ satisfies the displayed fixed-point identity on all positive-probability samples at all times $k < N$, then it is cost-minimal among all policies; the inequality is non-strict; the hypothesis constrains $\pi^\star$ only on measurement lists actually realized by positive-probability samples under $\pi^\star$ itself (its values elsewhere are unconstrained, though they do not affect the cost); and the hypothesis is stated only for $k < N$, matching exactly the stages whose controls enter the cost.