Theorem 10.8.4 (Refined Monotonicity of Relative Entropy) —
ProvedWildeQIT.PinskerRefined.theorem_10_8_4entropy-inequalitiesinformation-theoryp2o-gran-per-chapterp2o-plan-bookp2o-v1recoverabilityrelative-entropy
Let be a probability distribution on a finite alphabet , let satisfy , and let be a classical channel from to a finite alphabet , with and . Let be a recovery channel from to defined by
and let . Then
If the loss of relative entropy under is small, then the recovery channel (Bayes' rule when is a probability distribution) nearly restores from . It gives the saturation clause of Corollary 10.7.2.
Formalization Note is relEntropy (bits, extended reals); under the three relative entropies are finite, so the difference is the real difference. The theorem holds for every channel satisfying (10.165).
Differences from the source:
- reading: "the recovery channel R(x|y) … is defined by" (10.165) → every channel R satisfying (10.165); reason: (10.165) does not determine R(·|y) where (Nq)(y) = 0; the statement is asserted for each such R, which is the stronger reading
Preamble
import Definitions.Def_WildeQIT_relEntropy import Definitions.Def_WildeQIT_Channel
Formal statement
namespace WildeQIT.PinskerRefined
open WildeQIT
/-- **Theorem 10.8.4 (Refined Monotonicity of Relative Entropy)**, Wilde, *From Classical to Quantum Shannon Theory*, arXiv:1106.1445v8, p. 315: let `p` be a
probability distribution on a finite alphabet `𝒳`, `q : 𝒳 → [0, ∞)` with `supp(p) ⊆ supp(q)`, and
`N(y|x)` a classical channel. For the recovery channel `R(x|y)` defined by (10.165),
`R(x|y) (Nq)(y) = N(y|x) q(x)`, `D(p‖q) − D(Np‖Nq) ≥ D(p‖RNp)` (10.164).
**Formalization Note** `D` is the published `relEntropy` (bits, `EReal`, `+∞` off support,
Definition 10.5.2); `Np` is `N.apply p`, `Nq` is `N.applyFun q`, and `RNp` is `R.apply (N.apply p)`,
with mass `∑_{y,x'} R(x|y) N(y|x') p(x')` at `x`. Equations (10.165) do not determine `R(·|y)` where
`(Nq)(y) = 0`, so the theorem is stated for every channel `R` satisfying them. Under
`supp(p) ⊆ supp(q)` all three relative entropies are finite, so the `EReal` difference is the real
difference. -/
theorem theorem_10_8_4 {α β : Type} [Fintype α] [Fintype β]
(p : FinDist α) (q : α → ℝ) (hq : ∀ x, 0 ≤ q x)
(hsupp : Function.support p.prob ⊆ Function.support q)
(N : Channel α β) (R : Channel β α)
(hR : ∀ x y, (R y).prob x * N.applyFun q y = (N x).prob y * q x) :
relEntropy p (R.apply (N.apply p)).prob ≤
relEntropy p q - relEntropy (N.apply p) (N.applyFun q) := by sorry
end WildeQIT.PinskerRefined
Source
Wilde, From Classical to Quantum Shannon Theory, arXiv:1106.1445v8, Theorem 10.8.4, p. 315, Eqs. (10.164)–(10.165)
Human review
Confirmed by the mission captain (proposal self-audit).
Confirmed by the moderator at approval.