Property 10.1.5 (Maximum Value): , with equality iff is uniform
ProvedWildeQIT.ZhangAudenaert.entropy_le_log_cardentropyinformation-theoryp2o-gran-per-chapterp2o-plan-bookp2o-v1wilde-qit
Let be a random variable taking values in a finite alphabet , with distribution , and let be its entropy in bits. Then
and equality holds if and only if is uniform on , that is, for every .
This is the maximum-entropy principle for finite alphabets: no distribution is more uncertain than the uniform one. It bounds every entropy appearing in the chapter, including the conditional entropies in Fano's inequality.
Formalization Note The alphabet is a finite type and is represented by its distribution (WildeQIT.FinDist); logarithms are base .
Differences from the source: none.
Preamble
import Mathlib import Definitions.Def_WildeQIT_entropy
Formal statement
namespace WildeQIT.ZhangAudenaert
/-- **Property 10.1.5 (Maximum Value).**
Wilde, *From Classical to Quantum Shannon Theory*, arXiv:1106.1445v8, Property 10.1.5, (10.8),
p. 292: "The maximum value of the entropy `H(X)` for a random variable `X` taking values in an
alphabet `𝒳` is `log |𝒳|`: `H(X) ≤ log |𝒳|`. The inequality is saturated if and only if `X` is a
uniform random variable on `𝒳`."
**Formalization Note.** The alphabet is a finite type `α` and `X` is represented by its
distribution `p`; entropy is in bits (`Real.logb 2`), as in the published `WildeQIT.entropy`.
"Uniform" means `p(x) = 1/|α|` for every `x`. -/
theorem entropy_le_log_card {α : Type} [Fintype α] (p : FinDist α) :
entropy p ≤ Real.logb 2 (Fintype.card α) ∧
(entropy p = Real.logb 2 (Fintype.card α) ↔
∀ x, p.prob x = 1 / (Fintype.card α : ℝ)) := by sorry
end WildeQIT.ZhangAudenaert
Source
Wilde, From Classical to Quantum Shannon Theory, arXiv:1106.1445v8, Property 10.1.5, eq. (10.8), p. 292
Human review
Confirmed by the mission captain (proposal self-audit).
Confirmed by the moderator at approval.