Definition 1, p. 354 — the symmetric multiple logistic transformation (39) is equivalent to (40) with Σ F_k = 0
ProvedAddLogReg.Multiclass.def_1_equivmulticlass-classificationmultinomial-logistic-regressionp2o-batch-pfp2ap2o-gran-per-chapterp2o-plan-paperp2o-v1softmax
Let and let be a probability vector with all entries positive: and . For any real vector the following are equivalent:
- is the symmetric multiple logistic transformation (39) of :
- satisfies (40):
This is the equivalence of the two forms of Definition 1. It says that (39) is the unique centred inverse of the softmax map (40), and is what lets the multiclass LogitBoost algorithm work on the centred scores and recover probabilities through (40).
Formalization Note The positivity of every is what "" with defined requires: Lean's Real.log 0 = 0 would otherwise give (39) a junk value. The vector form is the statement at a fixed .
Preamble
import Mathlib import Definitions.Def_AddLogReg_Multiclass_Setting open MeasureTheory ProbabilityTheory ENNReal
Formal statement
namespace AddLogReg.Multiclass
theorem def_1_equiv {J : ℕ} [NeZero J] (p : Fin J → ℝ) (hp_pos : ∀ j, 0 < p j)
(hp_sum : ∑ j, p j = 1) (F : Fin J → ℝ) :
F = symMultiLogit p ↔ ((∀ j, p j = softmax F j) ∧ ∑ k, F k = 0) := by sorry
end AddLogReg.Multiclass
Source
Friedman, Hastie and Tibshirani, Additive logistic regression: a statistical view of boosting, Ann. Statist. 28 (2000), p. 354, Definition 1, (39)–(40) and the sentence after (40)
Human review
Confirmed by the mission captain (proposal self-audit).
Confirmed by the moderator at approval.