Proof of Theorem 2, p. 30 — the inner supremum for the logistic loss
ProvedRWPI.Classif.logistic_inner_supdistributionally-robust-optimizationdualitylogistic-regressionp2o-batch-pfp2bp2o-gran-per-chapterp2o-plan-paperp2o-v1
Let satisfy . Let , let and let . Then
This is the value of the dual integrand of Proposition 1 for the logistic loss under the label-preserving cost , once the response is held at . It is the step of the proof of Theorem 2 that produces the penalty .
Formalization Note The display in the paper writes the transport norm as and the threshold as ; since and are conjugate the display holds either way, and the statement here uses Theorem 2's convention (transport norm , regularizer ). The supremum is taken in with truncated subtraction, which does not change its value because the term at is already at least every truncated term.
Preamble
import Mathlib import Definitions.Def_RWPI_Classif_losses open scoped ENNReal
Formal statement
namespace RWPI.Classif
/-- App. A.1, proof of Theorem 2, p. 30 ("following Lemma 1 in [42]"): for a label
`y₀ ∈ {−1, +1}`, a point `x₀ ∈ ℝ^d` and `λ ≥ 0`,
`sup_x { log(1 + exp(−y₀ βᵀx)) − λ ‖x − x₀‖_q } = log(1 + exp(−y₀ βᵀx₀))` if `‖β‖_p ≤ λ`
and `+∞` otherwise, where `1/p + 1/q = 1`. Stated in Theorem 2's convention (transport norm
`ℓ_q`, regularizer `ℓ_p`); the proof's display has the two exponents swapped. -/
theorem logistic_inner_sup {d : ℕ} (p q : ℝ≥0∞) (hpq : p.HolderConjugate q)
(β x₀ : Fin d → ℝ) (y₀ : ℝ) (hy₀ : y₀ = 1 ∨ y₀ = -1) (lam : ℝ) (hlam : 0 ≤ lam) :
(⨆ x : Fin d → ℝ, (ENNReal.ofReal (logLoss β (x, y₀)) -
ENNReal.ofReal lam * ENNReal.ofReal ‖WithLp.toLp q (x - x₀)‖)) =
if ‖WithLp.toLp p β‖ ≤ lam then ENNReal.ofReal (logLoss β (x₀, y₀)) else ⊤ := by sorry
end RWPI.Classif
Source
Blanchet, Kang & Murthy, Robust Wasserstein Profile Inference and Applications to Machine Learning, arXiv:1610.05627v4, App. A.1, proof of Theorem 2, p. 30 (second display, following Lemma 1 of Shafieezadeh-Abadeh et al. 2015)
Human review
Confirmed by the mission captain (proposal self-audit).
Confirmed by the moderator at approval.