Lemma A.5 (second bullet):
ProvedHellingerSubadd.lemmaA5_hellingerSq_le_klDivhellinger-distanceinformation-theoryinteractive-decision-makingprobability
Let and be probability measures on a measurable space. Then
where if and otherwise.
In the proof of Lemma A.13 this turns the squared Hellinger distances of Terms I and II into KL divergences, to which the chain rule applies.
Differences from the source:
- encoding: D_KL(P ∥ Q) as defined on p. 10, real or +∞ → Mathlib's klDiv μ ν : ℝ≥0∞, compared in ℝ≥0∞; reason: for probability measures klDiv equals the paper's definition (a non-integrable log-likelihood ratio has integral +∞)
Preamble
import Mathlib import Definitions.Def_HellingerSubadd_hellingerSq open MeasureTheory InformationTheory open scoped ENNReal
Formal statement
namespace HellingerSubadd
/-- **Lemma A.5, second bullet: squared Hellinger distance is at most KL divergence.**
Foster, Kakade, Qian & Rakhlin, *The Statistical Complexity of Interactive Decision Making*,
arXiv:2112.13487v3, Lemma A.5, second bullet, p. 71: "D²_H(P, Q) ≤ D_KL(P ∥ Q)".
For probability measures `μ, ν` on one measurable space, `D²_H(μ, ν) ≤ D_KL(μ ∥ ν)`.
**Formalization Note.** KL divergence is Mathlib's `klDiv` (value `∞` off absolute continuity and
off integrability of the log-likelihood ratio; for probability measures this is the paper's
definition, p. 10). Both sides in `ℝ≥0∞`. -/
theorem lemmaA5_hellingerSq_le_klDiv {α : Type*} [MeasurableSpace α] (μ ν : Measure α)
[IsProbabilityMeasure μ] [IsProbabilityMeasure ν] :
hellingerSq μ ν ≤ klDiv μ ν := by sorry
end HellingerSubadd
Source
Foster, Kakade, Qian & Rakhlin, The Statistical Complexity of Interactive Decision Making, arXiv:2112.13487v3, Lemma A.5, second bullet, p. 71