Residuals-based distributionally robust optimization with covariate information 1: Wasserstein ER-DRO with the Enlarged Radius (10) Certifies Its Out-of-Sample Cost with Probability 1−αResearch Paper
Decisions with covariate information
Many operational decisions are made after observing covariates that carry information about the uncertainty: a retailer sets inventory after seeing weather and calendar features that predict demand, a grid operator commits generation after seeing a load forecast. The decision problem is a conditional stochastic program: choose to minimize , where is the uncertain quantity, the covariates and their observed value. The conditional distribution of given is unknown; only joint data are available (Bertsimas and Kallus, 2020).
Kannan, Bayraksan and Luedtke (arXiv:2207.13554) proposed fitting a regression model and adding its empirical residuals to the point prediction at , producing scenarios for a sample average approximation (the ER-SAA). With few data, any such sample average is optimistically biased: the optimal value underestimates the true cost of the chosen decision. Distributionally robust optimization (DRO) over a Wasserstein ball corrects this bias in the covariate-free case, where Mohajerin Esfahani and Kuhn (2018) showed that a suitably sized ball turns the DRO optimal value into an upper confidence bound on the out-of-sample cost. The paper formalized here (Kannan, Bayraksan and Luedtke, 2023) extends that guarantee to the residuals-based setting. Its main result, Theorem 7, is this mission's goal.
Setting
Let , and give and the Euclidean norm . The model is
with regression function and zero-mean errors independent of . The conditional distribution is then the law of . Let be a cost and a nonempty compact feasible set. The true objective is .
A known nonempty closed convex set range of is given, with orthogonal projection (the nearest-point map). The data are covariates and errors , with responses for . A regression estimate built from the data has empirical residuals . Two empirical distributions are compared: the true empirical distribution , which needs , and the estimated empirical distribution , which is computable:
Their scenarios differ by , the prediction error at plus the estimation error at .
For the -Wasserstein distance is , with the infimum over couplings of and . The ER-DRO problem (8) with radius is
with optimal solution .
The radius (10) combines two allowances:
- Assumption 2 bounds the regression error: with probability at least , and are each at most .
- Lemma 2, quoted from Fournier and Guillin (2015), is an exponential concentration bound for with constants . It holds under the light-tail Assumption 1, for some .
The radius is . Its second term inverts Lemma 2 at level .
Formalization targets
Goal: Theorem 7 (finite sample certificate guarantee)
Under Assumptions 1 and 2 at , , i.i.d. errors, Lemma 2's bound with constants and , the ER-DRO problem with radius satisfies
Milestones, in the order the paper's proof uses them
- (6): .
- Lemma 3: .
- Lemma 4: the power mean of is at most the prediction error plus the power-mean estimation error.
- Lemma 5: .
- Lemma 6: .
- Second display of the proof of Theorem 7: .
- Third display: .
Significance
Theorem 7 gives the ER-DRO optimal value a meaning a decision maker can use. The computed number bounds the true expected cost of the computed decision, with confidence , for every finite . The scenarios are built from an estimated regression function, not from samples of the conditional distribution. The price is an enlarged radius: the covariate-free radius plus a term for the regression error. The same radius drives the paper's asymptotic optimality and rate results (Theorems 9–11).
The theorem is proved in the paper. No part of it is formalized. The covariate-free analogue, Theorem 3.5 of Mohajerin Esfahani and Kuhn, is formalized and proved on the platform (WassDDRO.Consistency.theorem_3_5). This mission adds three things:
- the regression layer: the projection inequality (6), the decomposition of Lemma 3 and the power-mean bounds;
- a precise account of the radius;
- a correction to it. The paper prints in , which sets Lemma 2's right-hand side to , not to the the text states. The proof as printed then gives only . The mission states the theorem with , the derivation the paper describes.
Difficulty
The obvious argument copies the covariate-free proof: treat as an empirical distribution of and apply a concentration inequality. This fails. The scenarios are not samples from , and they are not independent: all of them share the data-dependent estimate . No concentration inequality applies to directly. The regression error has to be separated from the sampling error, and each controlled by its own hypothesis, at a fixed covariate .
On the formal side, Lemma 3 needs the triangle inequality for between arbitrary probability measures, which needs a gluing construction. Mathlib does not have it, and on the platform it is an open statement. Lemma 4 needs Minkowski's inequality for a normalized finite power mean. The probabilistic steps act on events whose measurability is not assumed.
Formalization scope
Conventions:
- is
EuclideanSpace ℝ (Fin d)with its Borel σ-algebra, and . - The data, the regression estimate and the solution are arbitrary functions on a probability space . Each probability is the outer measure of the displayed set, so no measurability of these objects is assumed.
- The responses are . The projection is a given map with the nearest-point property on a nonempty closed convex , and is a hypothesis.
- is the pushforward of under .
- The paper's "for a.e. " is handled pointwise. Assumption 2 (a function ) and Lemma 2's conclusion (constants ) are hypotheses at the given and sample size .
- Lemma 2 itself is not posed; it is Fournier and Guillin's Theorem 2, already open on the platform. That theorem bounds , while the paper transcribes the exponents for . For and the hypothesis may therefore be unsatisfiable. The mission keeps the paper's Lemma 2 as written.
- Distances are -valued and the DRO value is an extended real. The worst case counts the distributions in the ball under which is integrable. The conclusion of Theorem 7 is in failure form, .
The integrability hypothesis on is necessary: without it the true law would not be counted in the worst case and the certificate would be vacuous. Centring the ball at instead of would trivialize the mission by removing the regression error, and is excluded.
Needed infrastructure, reusable beyond this mission: the gluing lemma and triangle inequality for ; Wasserstein bounds between empirical measures with matched atoms; nonexpansiveness of the metric projection onto a closed convex set. The general Wasserstein objects are the published WassersteinDRO.Duality definitions. Proofs of any milestone are welcome, as is a proof of Theorem 7 from the milestones.
Selected references
- R. Kannan, G. Bayraksan, J. R. Luedtke, Residuals-based distributionally robust optimization with covariate information, Mathematical Programming, 2023. https://doi.org/10.1007/s10107-023-02014-7
- R. Kannan, G. Bayraksan, J. R. Luedtke, Data-driven sample average approximation with covariate information, arXiv preprint, 2022. https://arxiv.org/abs/2207.13554
- P. Mohajerin Esfahani, D. Kuhn, Data-driven distributionally robust optimization using the Wasserstein metric: performance guarantees and tractable reformulations, Mathematical Programming 171, 2018. https://doi.org/10.1007/s10107-017-1172-1
- N. Fournier, A. Guillin, On the rate of convergence in Wasserstein distance of the empirical measure, Probability Theory and Related Fields 162, 2015. https://doi.org/10.1007/s00440-014-0583-7
- D. Bertsimas, N. Kallus, From predictive to prescriptive analytics, Management Science 66, 2020. https://doi.org/10.1287/mnsc.2018.3253