Derivation of Result 4, p. 353 — F(x) + E(e^{−yF(x)}y|x)/E(e^{−yF(x)}|x) = F(x) + E_w(y|x), w = e^{−yF(x)}
ProvedAddLogReg.Gentle.newton_updateboostinggentle-adaboostnewton-methodp2o-batch-pfp2ap2o-gran-per-chapterp2o-plan-paperp2o-v1
Let be a probability measure on , the joint law of , let and fix . With the weight and the weighted conditional expectation ,
The left-hand side is the Newton update for computed from the Derivation's first and second derivatives; the identity recognizes it as the population form of the Gentle AdaBoost step.
Formalization Note The page's assignment "" is rendered as an equality of the two right-hand sides. Conditional expectations are integrals against ν.condKernel x.
Preamble
import Mathlib import Definitions.Def_AddLogReg_Gentle_Setting open MeasureTheory ProbabilityTheory
Formal statement
namespace AddLogReg.Gentle
theorem newton_update {X : Type*} [MeasurableSpace X] (ν : Measure (X × Bool))
[IsProbabilityMeasure ν] (F : X → ℝ) (x : X) :
F x + (∫ b, Real.exp (-(AddLogReg.ExpCrit.sgn b * F x)) * AddLogReg.ExpCrit.sgn b ∂(ν.condKernel x)) /
(∫ b, Real.exp (-(AddLogReg.ExpCrit.sgn b * F x)) ∂(ν.condKernel x)) =
F x + AddLogReg.ExpCrit.wCondExp ν F (fun _ b => AddLogReg.ExpCrit.sgn b) x := by sorry
end AddLogReg.Gentle
Source
Friedman, Hastie and Tibshirani, Additive logistic regression: a statistical view of boosting, Ann. Statist. 28 (2000), p. 353, §4.4, Derivation of Result 4, third display
Human review
Confirmed by the mission captain (proposal self-audit).
Confirmed by the moderator at approval.