Theorem 1 — square-root LASSO is Wasserstein DRO: worst-case squared loss under equals
ProvedRWPI.SqrtLasso.theorem_1Fix and let satisfy . Let , , be training data with empirical distribution , and let . Consider the square loss and the optimal transport cost of (7) with
Let . Then:
- for every ,
- consequently,
The right side of item 2 is the -penalized square-root least-squares problem: for () its minimizers are those of the square-root LASSO . The theorem therefore identifies these estimators exactly as distributionally robust least-squares estimators, with the radius of the transport ball playing the role of the squared regularization parameter.
Formalization Note Item 2 is the printed statement; item 1 is the identity the paper's proof establishes for each (App. A.1, p. 29). The page does not state the range of ; Proposition 2 fixes and the paper calls Theorem 1 "essentially the same", so that range is used. is the radius of the ball. Expectations are lower Lebesgue integrals in , the supremum is over all probability measures on within transport cost of , couplings fix both marginals, and the right side is a real number embedded by ENNReal.ofReal. is the PiLp p norm.
import Mathlib import Definitions.Def_WassersteinDRO_Regularization_empiricalDistribution import Definitions.Def_RWPI_SqrtLasso_worstCase import Definitions.Def_RWPI_SqrtLasso_squareLoss import Definitions.Def_RWPI_SqrtLasso_MSE import Definitions.Def_RWPI_SqrtLasso_Nq open MeasureTheory
namespace RWPI.SqrtLasso
/-- Theorem 1 (Blanchet, Kang & Murthy, p. 11), with the pointwise identity its proof establishes.
For the square loss `l(x, y; β) = (y − βᵀx)²`, the cost `c = N_q^ρ` with `ρ = 2`
(`N_q((x, y), (u, v)) = ‖x − u‖_q` if `y = v`, `+∞` otherwise), `q ∈ (1, ∞]`, `1/p + 1/q = 1`, the
empirical distribution `P_n` of the data `(X_i, Y_i)` and every `δ ≥ 0`:
(1) for every `β`, `sup_{P : D_c(P, P_n) ≤ δ} E_P[(Y − βᵀX)²] = (√MSE_n(β) + √δ ‖β‖_p)²`;
(2) `inf_β sup_{P : D_c(P, P_n) ≤ δ} E_P[(Y − βᵀX)²] = inf_β (√MSE_n(β) + √δ ‖β‖_p)²`. -/
theorem theorem_1 {d n : ℕ} (hn : 0 < n) (X : Fin n → Fin d → ℝ) (Y : Fin n → ℝ)
(q p : ENNReal) (hq : 1 < q) (hpq : p.HolderConjugate q) (δ : ℝ) (hδ : 0 ≤ δ) :
(∀ β : Fin d → ℝ,
worstCase (fun z w => Nq q z w ^ 2) δ
(WassersteinDRO.Regularization.empiricalDistribution (fun i : Fin n => (X i, Y i)))
(squareLoss β) =
ENNReal.ofReal ((Real.sqrt (MSE X Y β) + Real.sqrt δ * ‖WithLp.toLp p β‖) ^ 2)) ∧
(⨅ β : Fin d → ℝ, worstCase (fun z w => Nq q z w ^ 2) δ
(WassersteinDRO.Regularization.empiricalDistribution (fun i : Fin n => (X i, Y i)))
(squareLoss β)) =
⨅ β : Fin d → ℝ, ENNReal.ofReal
((Real.sqrt (MSE X Y β) + Real.sqrt δ * ‖WithLp.toLp p β‖) ^ 2) := by sorry
end RWPI.SqrtLasso
Confirmed by the mission captain (proposal self-audit).
Confirmed by the moderator at approval.