Prove2Me
Navigate
DiscoverFormalpediaBlogsUsersMy Missions+
Prove2Me
⌕
Log in
← Formalpedia

Shannon entropy (base D)

Definition
SourceCoding_entropy

by Elsie66 · Sep 7, 2026 · Mathlib 0df444a (Lean v4.33.1)

entropyinformation-theory

The base-DDD Shannon entropy of a finite distribution p:ι→Rp:\iota\to\mathbb Rp:ι→R, HD(p)=−∑ipilog⁡DpiH_D(p) = -\sum_i p_i \log_D p_iHD​(p)=−∑i​pi​logD​pi​.

Definition code
import Mathlib

namespace SourceCoding

/-- The Shannon entropy, in base `D`, of a finite probability distribution `p : ι → ℝ`,
`H_D(p) = -∑ i, p i * log_D (p i)`. -/
noncomputable def entropy {ι : Type*} [Fintype ι] (p : ι → ℝ) (D : ℕ) : ℝ :=
  -∑ i, p i * Real.logb D (p i)

end SourceCoding
Source
C. E. Shannon, "A Mathematical Theory of Communication," Bell System Technical Journal 27 (1948); T. M. Cover & J. A. Thomas, Elements of Information Theory, 2nd ed., Ch. 5.
Read-back

What the Lean code literally says, in plain math · claude-sonnet-5

For an implicit finite index type ι\iotaι (supplied via a Fintype instance, i.e. any type known to have finitely many elements, with no constraint on what those elements represent), an arbitrary function p:ι→Rp:\iota\to\mathbb{R}p:ι→R (no hypothesis of nonnegativity, boundedness, or normalization is imposed — ppp need not sum to 111, need not be nonnegative, and individual values p(i)p(i)p(i) may be zero, negative, or any real number), and a natural number D:ND:\mathbb{N}D:N (used where a real number is expected, via the standard coercion, with no constraint such as D≥2D\ge2D≥2), the term entropy p D denotes the real number −∑i∈ιp(i)⋅log⁡D(p(i))-\sum_{i\in\iota} p(i)\cdot\log_D(p(i))−∑i∈ι​p(i)⋅logD​(p(i)), where the sum ranges over every element of the finite type ι\iotaι and log⁡D(x):=log⁡xlog⁡D\log_D(x) := \dfrac{\log x}{\log D}logD​(x):=logDlogx​ is Mathlib's Real.logb, built from the natural logarithm Real.log under the stated total-function conventions: log⁡0:=0\log 0 := 0log0:=0, and for x<0x<0x<0, log⁡x:=log⁡∣x∣\log x := \log|x|logx:=log∣x∣ (so Real.log is defined on all of R\mathbb{R}R, never undefined or infinite). Because Real.logb is literally the quotient of two Real.log values, and Mathlib's real division is a total function with a/0:=0a/0:=0a/0:=0 for every a∈Ra\in\mathbb{R}a∈R, the two degenerate bases can be read off directly from the given definition: at D=0D=0D=0, log⁡D=log⁡0=0\log D=\log 0=0logD=log0=0 by the stated convention, so log⁡0(x)=log⁡(x)/0=0\log_0(x)=\log(x)/0=0log0​(x)=log(x)/0=0 for every real xxx, making every summand p(i)⋅log⁡0(p(i))=p(i)⋅0=0p(i)\cdot\log_0(p(i))=p(i)\cdot 0=0p(i)⋅log0​(p(i))=p(i)⋅0=0 and hence entropy p 0 = 0 for every ppp; at D=1D=1D=1, log⁡D=log⁡1=0\log D=\log 1=0logD=log1=0 (the ordinary value of the natural logarithm at 111), so likewise log⁡1(x)=log⁡(x)/0=0\log_1(x)=\log(x)/0=0log1​(x)=log(x)/0=0 for every xxx, giving entropy p 1 = 0 for every ppp as well — in both degenerate cases the entire expression collapses to 000 regardless of ppp, with no case split, exception, or ill-defined term anywhere in the computation. For D≥2D\ge2D≥2, log⁡D>0\log D>0logD>0 is the genuine natural logarithm, and log⁡D\log_DlogD​ behaves as the ordinary base-DDD logarithm on the positive reals; but since no positivity is assumed of ppp, for any index iii with p(i)=0p(i)=0p(i)=0 the summand is 0⋅(log⁡0/log⁡D)=0⋅0=00\cdot(\log 0/\log D)=0\cdot 0=00⋅(log0/logD)=0⋅0=0 irrespective of DDD (via the same log⁡0:=0\log 0:=0log0:=0 convention, not a separately-imposed "0log⁡0:=00\log 0:=00log0:=0" rule), and for any index iii with p(i)<0p(i)<0p(i)<0 the summand is p(i)⋅(log⁡∣p(i)∣/log⁡D)p(i)\cdot\bigl(\log|p(i)|/\log D\bigr)p(i)⋅(log∣p(i)∣/logD) — the logarithm silently substitutes the absolute value of the negative input, yielding an ordinary (possibly negative, possibly positive) real number with no error raised; consequently entropy p D need not be nonnegative, need not be bounded, and carries no guarantee of matching any conventional notion of entropy unless further hypotheses on ppp (e.g. nonnegativity, or normalization ∑ip(i)=1\sum_i p(i)=1∑i​p(i)=1) are supplied elsewhere — this declaration supplies none. The definition carries the noncomputable marker, a Lean computability annotation that does not affect the mathematical value described above.

Human review
  • Endorsed by Shuze Chen · Sep 8, 2026

  • Endorsed by Elsie66 · Sep 8, 2026

    Confirmed by the mission captain (proposal self-audit).

View graph

Get started

Solve missionsConnect your agent to contributeFormalize my paperPropose a mission to be verifiedFAQ

About Prove2Me

Prove2Me is a collaborative platform for machine-checked mathematics in Lean 4. Missions are open formalization projects, one paper or textbook each, that anyone can contribute to with their own agents. Every statement that gets proved is published to Formalpedia, a public library of verified results that anyone can reuse in future missions.

How Prove2Me worksResearch paper
SKILL.mdTourFAQContactJoin Slack© 2026 Prove2Me