Tensorization of KL divergence over independent products
ProvedTensorization.klDiv_prod_eq_addTensorization (sub-additivity) of Kullback–Leibler divergence over independent products. For probability measures on and on , the relative entropy of the product measures splits additively:
This is the tensorization identity at the heart of the entropy method (relative entropy of a product of independent coordinates is the sum of the per-coordinate relative entropies). It is proved from the Kullback–Leibler chain rule together with product commutativity. Boucheron–Lugosi–Massart, Concentration Inequalities (OUP 2013), Theorem 4.10 (sub-additivity / tensorization of entropy, derived from Han's inequality = the KL chain rule).
Preamble
import Mathlib.InformationTheory.KullbackLeibler.ChainRule import Mathlib.Probability.Kernel.Composition.MeasureCompProd import Mathlib.MeasureTheory.Measure.Decomposition.RadonNikodym open MeasureTheory InformationTheory ProbabilityTheory open scoped ENNReal
Formal statement
theorem Tensorization.klDiv_prod_eq_add {α β : Type*} {mα : MeasurableSpace α} {mβ : MeasurableSpace β} (μ ν : Measure α) (π ρ : Measure β) [IsProbabilityMeasure μ] [IsProbabilityMeasure ν] [IsProbabilityMeasure π] [IsProbabilityMeasure ρ] : klDiv (μ.prod π) (ν.prod ρ) = klDiv μ ν + klDiv π ρ := by sorry