Lemma 4: entropy decomposition of an arbitrary discrete mixture

Let $I$ be any countable type, and let $e$ consist of weights $w_k\in\mathbb R$ and complex kernels $X_k:I\times I\to\mathbb C$, indexed by $k\in\mathbb N$, such that $w_k\ge0$, the weights are summable, and $\sum_k w_k=1$. Whenever $w_k>0$, assume $X_k$ is a density kernel: it admits an expansion $X_k(i,j)=\sum_r u_r(i)\overline{v_r(j)}$ with $u_r,v_r\in\ell^2(I;\mathbb C)$ and $\sum_r\|u_r\|\|v_r\|<\infty$; every finite quadratic form $\sum_{i,j\in s}\overline{z_i}X_k(i,j)z_j$ is real and nonnegative; and its real diagonal is summable with sum $1$. Components with zero weight need not satisfy these conditions. Define the mixture entrywise by $X(i,j)=\sum_k w_kX_k(i,j)$. For any kernel $Y$ on a label set $L$, define $H(Y)\in[0,\infty]$ as the infimum of $\sum_r[-a_r\log a_r]_+$ over all representations $Y(i,j)=\sum_r a_r q_r(i)\overline{q_r(j)}$, where $a_r\ge0$ are summable with sum $1$, and the vectors $q_r\in\ell^2(L;\mathbb C)$ having positive weight are unit vectors and pairwise orthogonal; zero-weight vectors are unrestricted. Here $[t]_+=\max(t,0)$, regarded as an extended nonnegative real, and an empty infimum is $+\infty$. Define $D(Y\|Z)$ as the supremum over every $m\in\mathbb N$, including $m=0$, and every orthonormal family $v_0,\ldots,v_{m-1}\in\ell^2(L;\mathbb C)$, of the following quantity. Form matrices

The quantity is $+\infty$ unless $\ker Z^v\subseteq\ker Y^v$; when this inclusion holds, it is

with the matrix functions defined by continuous functional calculus. If $H(X)\ne+\infty$, then

This equality is in $[0,\infty]$, where $0\cdot(+\infty)=0$; no separate finiteness assumption is imposed on individual component entropies or divergences. The quantification permits an empty $I$, although no ensemble satisfying the stated density and mass requirements exists then.