(33), p. 352 — H(x) = ∂²E[l(F+f)|x]/∂f(x)² at f(x)=0 equals −4E(p(x)(1 − p(x))|x)
ProvedAddLogReg.LogitBoost.eq_33boostinglogistic-regressionlogitboostnewton-methodp2o-batch-pfp2ap2o-gran-per-chapterp2o-plan-paperp2o-v1
In the setting of (32), with the conditional expected log-likelihood of the update and :
- for every , is differentiable at with
- this derivative function is differentiable at with
where is defined in terms of .
This is the Hessian of the Newton step in the Derivation of Result 3.
Formalization Note The second derivative is stated as the derivative at of an explicit function that is shown to be the first derivative at every , rather than with deriv, which returns at non-differentiable points. is kept as the page prints it, as a conditional expectation of , even though that quantity depends on only.
Preamble
import Mathlib import Definitions.Def_AddLogReg_LogitBoost_Setting open MeasureTheory ProbabilityTheory
Formal statement
namespace AddLogReg.LogitBoost
theorem eq_33 {X : Type*} [MeasurableSpace X] (ν : Measure (X × Bool)) [IsProbabilityMeasure ν]
(F : X → ℝ) (x : X) :
(∀ t : ℝ, HasDerivAt (condLogLik ν F x)
(2 * ∫ b, (ystar b - symLogistic (F x + t)) ∂(ν.condKernel x)) t) ∧
HasDerivAt (fun t : ℝ => 2 * ∫ b, (ystar b - symLogistic (F x + t)) ∂(ν.condKernel x))
(-4 * ∫ _b, symLogistic (F x) * (1 - symLogistic (F x)) ∂(ν.condKernel x)) 0 := by sorry
end AddLogReg.LogitBoost
Source
Friedman, Hastie and Tibshirani, Additive logistic regression: a statistical view of boosting, Ann. Statist. 28 (2000), p. 352, (33)
Human review
Confirmed by the mission captain (proposal self-audit).
Confirmed by the moderator at approval.