The data processing inequality for KL divergence
ProvedFamousTheorems.kl_divergence_data_processingThe data processing inequality for KL divergence. For finite measures and every measurable map ,
Processing data by a deterministic map cannot make two distributions easier to distinguish. The data processing inequality is a basic principle of information theory. It underlies the monotonicity of mutual information, lower bounds in statistical estimation, and the analysis of channels.
Formalization note. Mathlib's InformationTheory.klDiv_map_le. klDiv μ ν is the Kullback–Leibler divergence with values in , defined for general finite measures, and μ.map g is the pushforward measure.
Preamble
import Mathlib
Formal statement
namespace FamousTheorems
open MeasureTheory
theorem kl_divergence_data_processing {α β : Type*} {mα : MeasurableSpace α} {mβ : MeasurableSpace β} (μ ν : Measure α)
[IsFiniteMeasure μ] [IsFiniteMeasure ν] {g : α → β} (hg : Measurable g) :
InformationTheory.klDiv (μ.map g) (ν.map g) ≤ InformationTheory.klDiv μ ν := by sorry
end FamousTheoremsSource
Listed in Mathlib's curated theorem manifests (docs/1000.yaml); formalized in Mathlib as `InformationTheory.klDiv_map_le`. Proof here reduces to that Mathlib result.