Shannon entropy (base D)
DefinitionSourceCoding_entropyThe base- Shannon entropy of a finite distribution , .
import Mathlib
namespace SourceCoding
/-- The Shannon entropy, in base `D`, of a finite probability distribution `p : ι → ℝ`,
`H_D(p) = -∑ i, p i * log_D (p i)`. -/
noncomputable def entropy {ι : Type*} [Fintype ι] (p : ι → ℝ) (D : ℕ) : ℝ :=
-∑ i, p i * Real.logb D (p i)
end SourceCoding
Read-back
What the Lean code literally says, in plain math · claude-sonnet-5
For an implicit finite index type (supplied via a Fintype instance, i.e. any type known to have finitely many elements, with no constraint on what those elements represent), an arbitrary function (no hypothesis of nonnegativity, boundedness, or normalization is imposed — need not sum to , need not be nonnegative, and individual values may be zero, negative, or any real number), and a natural number (used where a real number is expected, via the standard coercion, with no constraint such as ), the term entropy p D denotes the real number , where the sum ranges over every element of the finite type and is Mathlib's Real.logb, built from the natural logarithm Real.log under the stated total-function conventions: , and for , (so Real.log is defined on all of , never undefined or infinite). Because Real.logb is literally the quotient of two Real.log values, and Mathlib's real division is a total function with for every , the two degenerate bases can be read off directly from the given definition: at , by the stated convention, so for every real , making every summand and hence entropy p 0 = 0 for every ; at , (the ordinary value of the natural logarithm at ), so likewise for every , giving entropy p 1 = 0 for every as well — in both degenerate cases the entire expression collapses to regardless of , with no case split, exception, or ill-defined term anywhere in the computation. For , is the genuine natural logarithm, and behaves as the ordinary base- logarithm on the positive reals; but since no positivity is assumed of , for any index with the summand is irrespective of (via the same convention, not a separately-imposed "" rule), and for any index with the summand is — the logarithm silently substitutes the absolute value of the negative input, yielding an ordinary (possibly negative, possibly positive) real number with no error raised; consequently entropy p D need not be nonnegative, need not be bounded, and carries no guarantee of matching any conventional notion of entropy unless further hypotheses on (e.g. nonnegativity, or normalization ) are supplied elsewhere — this declaration supplies none. The definition carries the noncomputable marker, a Lean computability annotation that does not affect the mathematical value described above.
Confirmed by the mission captain (proposal self-audit).