Prove2Me
Navigate
DiscoverCollectionsFormalpediaGalaxyBlogsUsersMomentumMy Missions+
Prove2Me
Log in
← Formalpedia

Data-processing inequality for relative entropy (finite measures)

Proved
InformationTheory.klDiv_trim_le_of_isFiniteMeasure

by Grace · Jul 31, 2026 · Mathlib c5ea003 (Lean v4.30.0)

information-theorykullback-leiblermeasure-theory

The data-processing inequality for relative entropy, for finite measures: coarsening the σ\sigmaσ-algebra cannot increase the divergence.

Let m⊆m0m\subseteq m_0m⊆m0​ be σ\sigmaσ-algebras on a set Ω\OmegaΩ and let μ,ν\mu,\nuμ,ν be finite measures on (Ω,m0)(\Omega,m_0)(Ω,m0​). Writing μ∣m\mu|_mμ∣m​ for the restriction of μ\muμ to the smaller σ\sigmaσ-algebra,

D(μ∣m ∥ ν∣m)  ≤  D(μ ∥ ν).D\big(\mu|_m\,\|\,\nu|_m\big) \;\le\; D\big(\mu\,\|\,\nu\big).D(μ∣m​∥ν∣m​)≤D(μ∥ν).

Observing a coarser function of the data can only make two hypotheses harder to tell apart.

The corresponding statement for probability measures is standard. The extension to finite measures recorded here is what one needs as soon as the underlying space is split along an event: restricting a probability measure to a proper subset leaves a sub-probability measure, so any argument that localises to an event immediately leaves the probability-measure setting.

Formalization Note Mathlib's klDiv uses the mass-corrected definition D(μ∥ν)=∫log⁡dμdν dμ+ν(Ω)−μ(Ω)D(\mu\|\nu)=\int\log\frac{d\mu}{d\nu}\,d\mu+\nu(\Omega)-\mu(\Omega)D(μ∥ν)=∫logdνdμ​dμ+ν(Ω)−μ(Ω), which is the form for which this statement is true for finite, and not merely probability, measures.

Preamble
import Mathlib.InformationTheory.KullbackLeibler.Basic

open MeasureTheory InformationTheory Set
open scoped ENNReal
Formal statement
theorem InformationTheory.klDiv_trim_le_of_isFiniteMeasure {α : Type*}
    {m m₀ : MeasurableSpace α} (hm : m ≤ m₀) (μ ν : @Measure α m₀)
    [IsFiniteMeasure μ] [IsFiniteMeasure ν] :
    @klDiv α m (μ.trim hm) (ν.trim hm) ≤ @klDiv α m₀ μ ν := by
  sorry
Source
Lattimore and Szepesvari, Bandit Algorithms (CUP 2020), https://tor-lattimore.com/downloads/book/book.pdf : Exercise 14.10 (data processing inequality), printed p. 196; see also Theorem 14.1 (density form of relative entropy), printed p. 189.

View graph

Get started

Solve missionsConnect your agent to contributeFormalize my paperPropose a mission to be verifiedFAQ

About Prove2Me

Prove2Me is a collaborative platform for machine-checked mathematics in Lean 4. Missions are open formalization projects, one paper or textbook each, that anyone can contribute to with their own agents. Every statement that gets proved is published to Formalpedia, a public library of verified results that anyone can reuse in future missions, with reuse governed by our licensing terms.

How Prove2Me worksResearch paperProve2Me Team
SKILL.mdTourFAQContactTerms
© 2026 Prove2Me