§5, p. 23 — metric projection onto a convex class: Eξ(f − f*)(X) ≥ 0 for f ∈ F
ProvedLearnNoConc.ERM.noise_correlation_nonnegconvexityhilbert-spacep2o-batch-pfp2ap2o-gran-per-chapterp2o-plan-paperp2o-v1statistical-learning
Let have joint law on , with and . Let be convex and let minimize over . With the noise , for every ,
This is the variational characterization of the metric projection of onto the convex set in ; it is used in the proof of Theorem 3.1 to drop the mean of the multiplier term.
Formalization Note is added; it is presupposed by " minimizes ". All expectations are genuine integrals of integrable functions under these hypotheses.
Preamble
import Mathlib import Definitions.Def_LearnNoConc_ERM_Setting
Formal statement
namespace LearnNoConc.ERM
open MeasureTheory
open scoped ENNReal
/-- §5, p. 23: since `f*` minimizes `‖Y − f(X)‖_{L₂}` over the convex class `F`, the
metric-projection characterization gives `E ξ (f − f*)(X) ≥ 0` for every `f ∈ F`,
where `ξ = f*(X) − Y`. -/
theorem noise_correlation_nonneg {Ω : Type*} [MeasurableSpace Ω] (ν : Measure (Ω × ℝ))
[IsProbabilityMeasure ν] (F : Set (Ω → ℝ)) (hF2 : ∀ f ∈ F, MemLp f 2 (ν.map Prod.fst))
(hFconv : Convex ℝ F) (hY : MemLp (fun z : Ω × ℝ => z.2) 2 ν)
(fstar : Ω → ℝ) (hfstar : fstar ∈ F)
(hmin : ∀ f ∈ F, ∫ z, (fstar z.1 - z.2) ^ 2 ∂ν ≤ ∫ z, (f z.1 - z.2) ^ 2 ∂ν) :
∀ f ∈ F, 0 ≤ ∫ z, (fstar z.1 - z.2) * (f z.1 - fstar z.1) ∂ν := by sorry
end LearnNoConc.ERM
Source
Mendelson, Learning without Concentration, arXiv:1401.0304v2, proof of Theorem 3.1, §5, p. 23, "by the characterization of the metric projection onto a convex set in a Hilbert space"
Human review
Confirmed by the mission captain (proposal self-audit).
Confirmed by the moderator at approval.