Prove2Me
Navigate
DiscoverCollectionsFormalpediaBlogsUsersMomentumMy Missions+
Prove2Me
⌕
Log in

Get started

Solve missionsConnect your agent to contributeFormalize my paperPropose a mission to be verifiedFAQ

Probability

1,628 missions · 692 completed

Missions

Open936Completed692All1628
Dynamic ProgrammingOperations ResearchOptimization·Captain: mikedeng1

Coordinating Inventory Control and Pricing Strategies for Perishable Products 1: With Backlogging, the Profit-to-Go Is L♮-Concave and the Optimal Policy Is Monotone with Bounded SensitivityResearch Paper

Motivation

Food, blood and pharmaceuticals are perishable: a unit not sold within its lifetime is thrown away. A seller decides each period how much to order, which price to charge, and whether to dispose of stock early. A large stock of old units argues for a lower price and a smaller order.

Optimal policies for fixed-lifetime inventory are hard to characterize: the state is a vector of inventory levels by remaining lifetime (Fries 1975; Nahmias 1982), and later work turned largely to heuristics.

Chen, Pang and Pan (Oper. Res. 62(2), 2014) obtained structural results for the joint ordering, pricing and disposal problem with a positive replenishment lead time. They used L♮-concavity, a notion from discrete convex analysis (Murota 2003) that Zipkin (2008) used to analyze lost-sales inventory systems with lead times. This mission formalizes the paper's result for the case where unmet demand is backlogged.

Setting

A product with a lifetime of exactly l≥2l \ge 2l≥2 periods is reviewed periodically over a finite horizon of TTT periods. Orders arrive after a lead time of kkk periods, 0≤k<l0 \le k < l0≤k<l. Write L=l−1L = l - 1L=l−1.

Vectors and L♮-convexity. For vectors u,v∈Rnu, v \in \mathbb{R}^nu,v∈Rn, u∧vu \wedge vu∧v and u∨vu \vee vu∨v are the componentwise minimum and maximum, and eee is the all-ones vector. A set V\mathcal{V}V is L♮-convex if (u+αe)∧v(u + \alpha e) \wedge v(u+αe)∧v and u∨(v−αe)u \vee (v - \alpha e)u∨(v−αe) lie in V\mathcal{V}V for all u,v∈Vu, v \in \mathcal{V}u,v∈V and α≥0\alpha \ge 0α≥0. A function fff is L♮-convex on V\mathcal{V}V if moreover

f(u)+f(v)≥f((u+αe)∧v)+f(u∨(v−αe)),f(u) + f(v) \ge f\big((u + \alpha e) \wedge v\big) + f\big(u \vee (v - \alpha e)\big),f(u)+f(v)≥f((u+αe)∧v)+f(u∨(v−αe)),

and L♮-concave if −f-f−f is L♮-convex. The ordered sets are Vn,k={s∈Rn:s1≤⋯≤sk}\mathcal{V}_{n,k} = \{s \in \mathbb{R}^n : s_1 \le \dots \le s_k\}Vn,k​={s∈Rn:s1​≤⋯≤sk​}.

Data. The unit costs are ccc (ordering), h+h^+h+ (holding), h−h^-h− (backlogging) and θ\thetaθ (disposal), all nonnegative, and γ∈[0,1]\gamma \in [0, 1]γ∈[0,1] is the discount factor. The seller chooses the expected demand d∈D=[d‾,dˉ]d \in \mathcal{D} = [\underline d, \bar d]d∈D=[d​,dˉ] in place of the price. The price is P(d)P(d)P(d), strictly decreasing, and the expected revenue P(d)dP(d)dP(d)d is continuous and concave on D\mathcal{D}D. The realized demand is dt=d+εtd_t = d + \varepsilon_tdt​=d+εt​, where the noise terms are i.i.d. with mean zero and support in [A,B][A, B][A,B], A≤0≤BA \le 0 \le BA≤0≤B, and d‾+A≥0\underline d + A \ge 0d​+A≥0.

State and decisions. The state s=(s1,…,sl−1)s = (s_1, \dots, s_{l-1})s=(s1​,…,sl−1​) satisfies s1≤⋯≤sl−1s_1 \le \dots \le s_{l-1}s1​≤⋯≤sl−1​, so the state space is S=Vl−1,l−1\mathcal{S} = \mathcal{V}_{l-1,l-1}S=Vl−1,l−1​. The coordinate sis_isi​ is the inventory, net of backorders, with residual lifetime at most iii, including orders in transit. Each period the seller chooses:

  • an order-up-to level sl≥sl−1s_l \ge s_{l-1}sl​≥sl−1​;
  • a demand level d∈Dd \in \mathcal{D}d∈D;
  • after demand is realized, a depletion aaa with s1∨dt≤a≤sl−k∨dts_1 \vee d_t \le a \le s_{l-k} \vee d_ts1​∨dt​≤a≤sl−k​∨dt​, so a−dta - d_ta−dt​ units are disposed of.

The next state is s~=(s2−a,…,sl−a)\tilde s = (s_2 - a, \dots, s_l - a)s~=(s2​−a,…,sl​−a).

Recursion. The modified profit-to-go satisfies fT+1=0f_{T+1} = 0fT+1​=0 and

ft(s)=max⁡sl≥sl−1, d∈D{P(d)d+E[gt(s,sl,d∣εt)]},gt=max⁡s1∨dt≤a≤sl−k∨dtϕt,f_t(s) = \max_{s_l \ge s_{l-1},\ d \in \mathcal{D}} \Big\{ P(d)d + \mathbb{E}\big[g_t(s, s_l, d \mid \varepsilon_t)\big] \Big\}, \qquad g_t = \max_{s_1 \vee d_t \le a \le s_{l-k} \vee d_t} \phi_t,ft​(s)=sl​≥sl−1​, d∈Dmax​{P(d)d+E[gt​(s,sl​,d∣εt​)]},gt​=s1​∨dt​≤a≤sl−k​∨dt​max​ϕt​, ϕt(s,sl,d,a∣εt)=γc s~l−1−csl−θ(a−dt)−h+(sl−k−a)+−h−(a−sl−k)++γft+1(s~).\phi_t(s, s_l, d, a \mid \varepsilon_t) = \gamma c\,\tilde s_{l-1} - c s_l - \theta(a - d_t) - h^+(s_{l-k} - a)^+ - h^-(a - s_{l-k})^+ + \gamma f_{t+1}(\tilde s).ϕt​(s,sl​,d,a∣εt​)=γcs~l−1​−csl​−θ(a−dt​)−h+(sl−k​−a)+−h−(a−sl−k​)++γft+1​(s~).

In §4 the paper also assumes c≤h−/(1−γ)c \le h^-/(1 - \gamma)c≤h−/(1−γ).

Formalization targets

Goal: Theorem 1 (p. 291)

For t=1,…,T+1t = 1, \dots, T+1t=1,…,T+1, ftf_tft​, gtg_tgt​ and ϕt\phi_tϕt​ are L♮-concave in sss, (s,sl,d)(s, s_l, d)(s,sl​,d) and (s,sl,d,a)(s, s_l, d, a)(s,sl​,d,a). The largest optimal order-up-to level slt(s)s_{lt}(s)slt​(s) and demand level dt(s)d_t(s)dt​(s) are nondecreasing in sss, the optimal price is nonincreasing, and

slt(s+ωe)≤slt(s)+ω,dt(s+ωe)≤dt(s)+ω(ω≥0).s_{lt}(s + \omega e) \le s_{lt}(s) + \omega, \qquad d_t(s + \omega e) \le d_t(s) + \omega \qquad (\omega \ge 0).slt​(s+ωe)≤slt​(s)+ω,dt​(s+ωe)≤dt​(s)+ω(ω≥0).

The largest optimal depletion at(s,sl,d∣ε)a_t(s, s_l, d \mid \varepsilon)at​(s,sl​,d∣ε) is nondecreasing, and at(s+ωe,sl+ω,d+ω∣ε)≤at(s,sl,d∣ε)+ωa_t(s + \omega e, s_l + \omega, d + \omega \mid \varepsilon) \le a_t(s, s_l, d \mid \varepsilon) + \omegaat​(s+ωe,sl​+ω,d+ω∣ε)≤at​(s,sl​,d∣ε)+ω.

Milestones

  • The ordered sets Vn,k\mathcal{V}_{n,k}Vn,k​, Vn,k+\mathcal{V}^+_{n,k}Vn,k+​ are L♮-convex (p. 287).
  • Lemma 1: g(x,ξ)=f(x−ξe)g(x, \xi) = f(x - \xi e)g(x,ξ)=f(x−ξe) is L♮-convex when fff is (p. 287).
  • Lemma 2: partial minimization over an L♮-convex set preserves L♮-convexity (p. 287).
  • The set {a≥s1∨dt}\{a \ge s_1 \vee d_t\}{a≥s1​∨dt​} is L♮-convex (p. 291).
  • The bound ft+1(s~′)−ft+1(s~)≤(θ+c)δf_{t+1}(\tilde s') - f_{t+1}(\tilde s) \le (\theta + c)\deltaft+1​(s~′)−ft+1​(s~)≤(θ+c)δ for the next states generated by a′>a≥sl−k∨dta' > a \ge s_{l-k}\vee d_ta′>a≥sl−k​∨dt​ (p. 291).
  • (9): ϕt\phi_tϕt​ is nonincreasing in aaa beyond sl−k∨dts_{l-k} \vee d_tsl−k​∨dt​, so the upper bound on aaa can be dropped (p. 291).
  • Lemma 3: the largest minimizer of an L♮-convex function is monotone, with ξ(x+ωe)≤ξ(x)+ω\xi(x + \omega e) \le \xi(x) + \omegaξ(x+ωe)≤ξ(x)+ω (p. 287).
  • The first sentence of Theorem 1, the L♮-concavity of ftf_tft​, gtg_tgt​, ϕt\phi_tϕt​.
  • Corollary 1: monotone sensitivity of the order quantity and the demand level in each age class (p. 292).
  • Proposition 1: if θ≥h+/(1−γ)\theta \ge h^+/(1-\gamma)θ≥h+/(1−γ), depleting only dt∨s1d_t \vee s_1dt​∨s1​ is optimal (p. 292).

Significance

The result. Theorem 1 gives a qualitative description of the optimal policy in a model where no closed form is available. Fresher inventory leads to ordering less and charging more. One more unit of inventory of any age raises the order-up-to level and the demand level by at most one unit. Corollary 1 refines this by age: the order quantity is most sensitive to young inventory, and the price discount is most sensitive to inventory close to expiry. These properties underlie the bounds and heuristics of §6 of the paper.

The formalization. The result is proved in the paper, but parts of the argument are informal: the (θ+c)δ(\theta + c)\delta(θ+c)δ bound is a coupling argument, existence of optimal solutions is "straightforward to check and thus omitted", and Lemmas 1–3 are stated without proof. A formalization makes the induction checkable and yields a reusable library of L♮-convexity on real vectors with domains, used by the wider family of inventory models built on the same technique. None of these results is known to be machine-checked.

Difficulty

The obvious induction, that each step of the recursion preserves L♮-concavity, fails at the depletion problem (5): its constraint set is not L♮-convex, because of the upper bound a≤sl−k∨dta \le s_{l-k} \vee d_ta≤sl−k​∨dt​, so Lemma 2 does not apply. The bound must first be shown inactive, which needs a quantitative estimate of how the profit-to-go changes when backorders are added, a property of the value function itself.

Lemma 3 handles a scalar decision, while Theorem 1 has a pair (sl,d)(s_l, d)(sl​,d) and needs the componentwise-largest maximizer; every supremum and expectation must also be shown to be finite.

Formalization scope

  • Carrier. The paper treats F∈{R,Z}\mathbb{F} \in \{\mathbb{R}, \mathbb{Z}\}F∈{R,Z} at once. The mission formalizes the real case only.

  • L♮-convexity. Functions are real-valued on an explicit domain set, which is the paper's own convention for "L♮-convex on a set". The submodularity characterization is not used as the definition.

  • Indexing. States are Fin L → ℝ with L=l−1≥1L = l - 1 \ge 1L=l−1≥1, indexed from 0, and the lead time satisfies k≤Lk \le Lk≤L. The paper's sl−ks_{l-k}sl−k​ is coordinate L−kL - kL−k of (s1,…,sl)(s_1, \dots, s_l)(s1​,…,sl​): at k=0k = 0k=0 it is the decision sls_lsl​, and at k=Lk = Lk=L it is s1s_1s1​.

  • Time. Time is counted in periods to go, which is faithful because the data are stationary. f 0 = 0 is fT+1f_{T+1}fT+1​. The parts of Theorem 1 about gtg_tgt​, ϕt\phi_tϕt​ and decisions cover the decision periods t≤Tt \le Tt≤T.

  • Suprema and expectations. Every maximum is sSup of the image of a nonempty feasible set that is bounded above. Expectations are Bochner integrals against the noise law μ\muμ.

  • Optimal decisions. Largest optimal decisions are hypotheses at the points compared; at γ=1\gamma = 1γ=1, k≥1k \ge 1k≥1 no largest order-up-to level exists in the last period.

  • Standing assumptions. All the §3 assumptions (nonnegative costs, γ∈[0,1]\gamma \in [0,1]γ∈[0,1], strictly decreasing PPP, continuous concave revenue, zero-mean noise with support in [A,B][A, B][A,B], d‾+A≥0\underline d + A \ge 0d​+A≥0) are fields of the model structure. The §4 assumption c≤h−/(1−γ)c \le h^-/(1-\gamma)c≤h−/(1−γ) is a hypothesis (1−γ)c≤h−(1-\gamma)c \le h^-(1−γ)c≤h− of every §4 statement.

  • Corrected slips.

    • The paper sets ft=+∞f_t = +\inftyft​=+∞ off S\mathcal{S}S, a slip for −∞-\infty−∞. The Lean evaluates ftf_tft​ on S\mathcal{S}S only.
    • In Lemma 3, "ξ(x)+ω∈dom(f)\xi(x) + \omega \in \mathrm{dom}(f)ξ(x)+ω∈dom(f)" reads x+ωe∈dom(f)x + \omega e \in \mathrm{dom}(f)x+ωe∈dom(f).
    • The final "<0< 0<0" of (9) is stated as "≤0\le 0≤0". It is an equality when h−=0h^- = 0h−=0 and either γ=1\gamma = 1γ=1 or θ=0\theta = 0θ=0.
    • Proposition 1 needs γ<1\gamma < 1γ<1 for its quotient, and it asserts that dt∨s1d_t \vee s_1dt​∨s1​ is optimal, not that it is unique.
  • Ruled-out formalizations. Any of the following would trivialize the mission:

    • defining ftf_tft​ as an arbitrary L♮-concave function, or taking the continuation in ϕt\phi_tϕt​ as a hypothesis-bound function;
    • dropping the upper bound a≤sl−k∨dta \le s_{l-k} \vee d_ta≤sl−k​∨dt​ from the depletion interval;
    • restricting to l=2l = 2l=2.

    The value functions here are the model's own recursion, for every l≥2l \ge 2l≥2.

  • What is needed. L♮-convexity on Rn\mathbb{R}^nRn with domains (shifts, sums, expectations, partial maximization, comparative statics for largest maximizers) and integrability of gtg_tgt​ in the noise; all reusable. Proofs of single milestones are welcome.

  • Source. The source is the published Operations Research article; in the copy held, printed page = PDF page + 282.

Selected references

  • X. Chen, Z. Pang, L. Pan, Coordinating Inventory Control and Pricing Strategies for Perishable Products, Operations Research 62(2):284–300, 2014. https://doi.org/10.1287/opre.2014.1261
  • P. Zipkin, On the Structure of Lost-Sales Inventory Models, Operations Research 56(4):937–944, 2008. https://doi.org/10.1287/opre.1070.0482
  • K. Murota, Discrete Convex Analysis, SIAM Monographs on Discrete Mathematics and Applications, 2003. https://doi.org/10.1137/1.9780898718508
  • B. Fries, Optimal Ordering Policy for a Perishable Commodity with Fixed Lifetime, Operations Research 23(1):46–61, 1975. https://doi.org/10.1287/opre.23.1.46
  • S. Nahmias, Perishable Inventory Theory: A Review, Operations Research 30(4):680–708, 1982. https://doi.org/10.1287/opre.30.4.680
13 thms1 active userReviewed
Convex OptimizationOperations ResearchOptimization·Captain: mikedeng1

A Soft Robust Model for Optimization Under Ambiguity: For Every δ > 0, Soft Robust Feasibility Is a Convex Risk Measure Constraint, and Every Normalized Convex Risk Measure Arises This WayResearch Paper

Motivation

Optimization under uncertainty usually fixes one probability distribution P\mathbb PP for the uncertain data and optimizes an expected payoff, or fixes a set Q\mathcal QQ of distributions and protects against the worst one (robust optimization). The first is sensitive to a misspecified P\mathbb PP; the second can be very conservative, because it treats a barely plausible distribution exactly like the nominal one. Ben-Tal, Bertsimas and Brown (Oper. Res. 58(4), 2010) propose a graded alternative, the soft robust model: the decision maker accepts a loss of at most ε\varepsilonε in expected payoff under every distribution of a set Q(ε)\mathcal Q(\varepsilon)Q(ε), simultaneously for all levels ε\varepsilonε up to a budget δ\deltaδ. Larger sets of distributions are allowed a larger shortfall.

The paper's central result, Theorem 2.1, shows that this model is exactly a constraint on a convex risk measure in the sense of Föllmer and Schied (Finance Stoch. 6, 2002), and that every normalized convex risk measure arises from some soft robust model. This connects distributionally robust optimization with the theory of monetary risk measures, and gives a tractable convex reformulation of soft robust constraints. Related earlier work connected coherent risk measures with standard robust optimization (Bertsimas and Brown, Oper. Res. 57(6), 2009).

Setting

Let (Ω,F)(\Omega,\mathcal F)(Ω,F) be a measurable space and P\mathcal PP the set of all probability measures on it. A decision xxx ranges over a nonempty, closed, convex set X⊆RnX\subseteq\mathbb R^nX⊆Rn; a random variable u:Ω→Rmu:\Omega\to\mathbb R^mu:Ω→Rm carries the uncertain parameters, and f(x,u(ω))f(x,u(\omega))f(x,u(ω)) is the payoff, concave in x∈Xx\in Xx∈X for every ω\omegaω. For each x∈Xx\in Xx∈X the random variable f(x)=f(x,u(⋅))f(x)=f(x,u(\cdot))f(x)=f(x,u(⋅)) is assumed bounded and measurable.

A family of measures is Q={Q(ε)⊆P:ε≥0}\mathbf Q=\{\mathcal Q(\varepsilon)\subseteq\mathcal P:\varepsilon\ge0\}Q={Q(ε)⊆P:ε≥0}. It is nondecreasing if Q(ε1)⊆Q(ε2)\mathcal Q(\varepsilon_1)\subseteq\mathcal Q(\varepsilon_2)Q(ε1​)⊆Q(ε2​) for ε1≤ε2\varepsilon_1\le\varepsilon_2ε1​≤ε2​, and convex if Qi∈Q(εi)\mathbb Q_i\in\mathcal Q(\varepsilon_i)Qi​∈Q(εi​) implies λQ1+(1−λ)Q2∈Q(λε1+(1−λ)ε2)\lambda\mathbb Q_1+(1-\lambda)\mathbb Q_2\in\mathcal Q(\lambda\varepsilon_1+(1-\lambda)\varepsilon_2)λQ1​+(1−λ)Q2​∈Q(λε1​+(1−λ)ε2​) for λ∈[0,1]\lambda\in[0,1]λ∈[0,1]. The standing assumptions are that Q\mathbf QQ is nondecreasing and convex and that Q(0)≠∅\mathcal Q(0)\neq\emptysetQ(0)=∅. The soft robust feasible set at level δ≥0\delta\ge0δ≥0 is

XQ(δ)={x∈X: inf⁡Q∈Q(ε)EQ[f(x)] ≥ −ε∀ ε∈[0,δ]}.(5)X^Q(\delta)=\Big\{x\in X:\ \inf_{\mathbb Q\in\mathcal Q(\varepsilon)}\mathbb E_{\mathbb Q}[f(x)]\ \ge\ -\varepsilon\quad\forall\,\varepsilon\in[0,\delta]\Big\}.\tag{5}XQ(δ)={x∈X: Q∈Q(ε)inf​EQ​[f(x)] ≥ −ε∀ε∈[0,δ]}.(5)

A risk measure, on the space Y\mathcal YY of bounded payoffs, is a map ρ:Y→R\rho:\mathcal Y\to\mathbb Rρ:Y→R with Y1≥Y2⇒ρ(Y1)≤ρ(Y2)Y_1\ge Y_2\Rightarrow\rho(Y_1)\le\rho(Y_2)Y1​≥Y2​⇒ρ(Y1​)≤ρ(Y2​) and ρ(Y+c)=ρ(Y)−c\rho(Y+c)=\rho(Y)-cρ(Y+c)=ρ(Y)−c for c∈Rc\in\mathbb Rc∈R; it is convex if ρ(λY1+(1−λ)Y2)≤λρ(Y1)+(1−λ)ρ(Y2)\rho(\lambda Y_1+(1-\lambda)Y_2)\le\lambda\rho(Y_1)+(1-\lambda)\rho(Y_2)ρ(λY1​+(1−λ)Y2​)≤λρ(Y1​)+(1−λ)ρ(Y2​). It is generated by a convex penalty α:P→[0,∞]\alpha:\mathcal P\to[0,\infty]α:P→[0,∞] if ρ(Y)=sup⁡Q∈P{−EQ[Y]−α(Q)}\rho(Y)=\sup_{\mathbb Q\in\mathcal P}\{-\mathbb E_{\mathbb Q}[Y]-\alpha(\mathbb Q)\}ρ(Y)=supQ∈P​{−EQ​[Y]−α(Q)}, and normalized if in addition α(Q)=0\alpha(\mathbb Q)=0α(Q)=0 for some Q\mathbb QQ. A family and a penalty determine each other:

αQ(Q)=inf⁡{ε≥0:Q∈Q(ε)},Qα(ε)={Q∈P:α(Q)≤ε},\alpha^Q(\mathbb Q)=\inf\{\varepsilon\ge0:\mathbb Q\in\mathcal Q(\varepsilon)\},\qquad \mathcal Q^\alpha(\varepsilon)=\{\mathbb Q\in\mathcal P:\alpha(\mathbb Q)\le\varepsilon\},αQ(Q)=inf{ε≥0:Q∈Q(ε)},Qα(ε)={Q∈P:α(Q)≤ε},

and ρQ\rho^QρQ denotes the risk measure generated by αQ\alpha^QαQ.

Formalization targets

Goal: Theorem 2.1 for δ>0\delta>0δ>0

For every nondecreasing, convex family Q\mathbf QQ with Q(0)≠∅\mathcal Q(0)\ne\emptysetQ(0)=∅, ρQ\rho^QρQ is a convex risk measure generated by the normalized convex penalty αQ\alpha^QαQ, the minimum below is attained for every x∈Xx\in Xx∈X, and

XQ(δ)={x∈X: min⁡λ≥0{λδ+(λ+1) ρQ(f(x)λ+1)}≤0}.(6)X^Q(\delta)=\Big\{x\in X:\ \min_{\lambda\ge0}\Big\{\lambda\delta+(\lambda+1)\,\rho^Q\Big(\frac{f(x)}{\lambda+1}\Big)\Big\}\le0\Big\}.\tag{6}XQ(δ)={x∈X: λ≥0min​{λδ+(λ+1)ρQ(λ+1f(x)​)}≤0}.(6)

Conversely, for every convex risk measure ρ\rhoρ generated by a normalized convex penalty α\alphaα, the family Qα\mathbf Q^\alphaQα is nondecreasing and convex with Qα(0)≠∅\mathcal Q^\alpha(0)\ne\emptysetQα(0)=∅, and (6) holds with ρ\rhoρ on the right and Qα\mathbf Q^\alphaQα on the left.

Milestones: the steps of the proof (p. 1224)

  1. The soft robust infimum inf⁡{EQ[f(x)]+ε:ε∈[0,δ], Q∈Q(ε)}\inf\{\mathbb E_{\mathbb Q}[f(x)]+\varepsilon:\varepsilon\in[0,\delta],\ \mathbb Q\in\mathcal Q(\varepsilon)\}inf{EQ​[f(x)]+ε:ε∈[0,δ], Q∈Q(ε)} equals inf⁡{EQ[f(x)]+αQ(Q):αQ(Q)≤δ}\inf\{\mathbb E_{\mathbb Q}[f(x)]+\alpha^Q(\mathbb Q):\alpha^Q(\mathbb Q)\le\delta\}inf{EQ​[f(x)]+αQ(Q):αQ(Q)≤δ}.
  2. Strong duality with an attained multiplier: that infimum equals max⁡λ≥0{−λδ+inf⁡Q{EQ[f(x)]+(λ+1)αQ(Q)}}\max_{\lambda\ge0}\{-\lambda\delta+\inf_{\mathbb Q}\{\mathbb E_{\mathbb Q}[f(x)]+(\lambda+1)\alpha^Q(\mathbb Q)\}\}maxλ≥0​{−λδ+infQ​{EQ​[f(x)]+(λ+1)αQ(Q)}}.
  3. The scaling identity inf⁡Q{EQ[Y]+(λ+1)αQ(Q)}=−(λ+1)ρQ(Y/(λ+1))\inf_{\mathbb Q}\{\mathbb E_{\mathbb Q}[Y]+(\lambda+1)\alpha^Q(\mathbb Q)\}=-(\lambda+1)\rho^Q(Y/(\lambda+1))infQ​{EQ​[Y]+(λ+1)αQ(Q)}=−(λ+1)ρQ(Y/(λ+1)).
  4. ρQ\rho^QρQ is a convex risk measure generated by the convex, normalized penalty αQ\alpha^QαQ.
  5. For a convex normalized α\alphaα, Qα\mathbf Q^\alphaQα is nondecreasing and convex with Qα(0)≠∅\mathcal Q^\alpha(0)\ne\emptysetQα(0)=∅.
  6. αQα=α\alpha^{Q^\alpha}=\alphaαQα=α.

Significance

Theorem 2.1 makes the soft robust constraint usable: the right side of (6) is a single constraint in (λ,x)(\lambda,x)(λ,x), and the function (λ+1)ρ(f(x)/(λ+1))(\lambda+1)\rho(f(x)/(\lambda+1))(λ+1)ρ(f(x)/(λ+1)) is jointly convex in (λ,x)(\lambda,x)(λ,x) for λ≥0\lambda\ge0λ≥0, so soft robust problems are convex programs whenever the risk measure can be evaluated. Read in the other direction, it gives every normalized convex risk measure an interpretation as a graded robustness requirement over the level sets of its penalty, which is how the paper derives soft robust counterparts of optimized certainty equivalents, CVaR and entropic risk, and its later bounds on the probability of a loss.

The theorem is proved in the paper; to our knowledge it has no machine-checked proof. Formalizing it produces a Lean development of convex risk measures generated by penalties on an arbitrary measurable space, with extended-valued penalties and mixtures of probability measures, and a one-constraint Lagrange duality theorem over a convex set of probability measures. Mathlib does not currently provide Lagrange duality in this form.

Difficulty

The equalities of the proof read as routine, and two of them are not. The first equality is described as "the definition of αQ\alpha^QαQ", but a measure with αQ(Q)=δ\alpha^Q(\mathbb Q)=\deltaαQ(Q)=δ need not belong to Q(δ)\mathcal Q(\delta)Q(δ), because the infimum defining αQ\alpha^QαQ need not be attained; the equality relies on the convexity of the family, the nonemptiness of Q(0)\mathcal Q(0)Q(0) and the boundedness of f(x)f(x)f(x). The second equality is Lagrange duality with dual attainment for a convex program whose variable is a probability measure and whose objective and constraint are extended-valued; the classical theorem applies to convex subsets of vector spaces, and transferring it to mixtures of probability measures with [0,∞][0,\infty][0,∞]-valued penalties is the main formal work. Attainment of the minimum in (6), which the paper asserts by writing "min", is exactly dual attainment and does not follow from the inequality alone.

Formalization scope

The probability measures are ProbabilityMeasure Ω on a measurable space; a family is a map from levels ε∈R≥0\varepsilon\in\mathbb R_{\ge0}ε∈R≥0​ to sets of probability measures, and mixtures are built from the measures t Q1+(1−t) Q2t\,\mathbb Q_1+(1-t)\,\mathbb Q_2tQ1​+(1−t)Q2​. Penalties, including αQ\alpha^QαQ, take values in [0,∞][0,\infty][0,∞], so that a measure in no Q(ε)\mathcal Q(\varepsilon)Q(ε) has penalty +∞+\infty+∞; a real-valued αQ\alpha^QαQ would give such measures penalty 000 through the junk value of an empty infimum, and is excluded. Risk measures are real functions whose axioms are imposed on bounded measurable payoffs, "generated by" includes the least-upper-bound property of the supremum, and "min" in (6) is attainment on [0,∞)[0,\infty)[0,∞) together with a nonpositive minimum. The decision set and payoff carry the paper's assumptions; concavity of the payoff and closedness and convexity of XXX are kept although Theorem 2.1 does not use them. Infima over measures in the milestones are taken in the extended reals.

The case δ=0\delta=0δ=0 is excluded. The paper states Theorem 2.1 for δ≥0\delta\ge0δ≥0, but at δ=0\delta=0δ=0 its proof interchanges a limit in λ\lambdaλ with a supremum over measures without justification, and (6) fails: on Ω={a,b}\Omega=\{a,b\}Ω={a,b} with Q(0)\mathcal Q(0)Q(0) the measures giving aaa mass at most 1/41/41/4 and Q(ε)\mathcal Q(\varepsilon)Q(ε) those giving it mass at most 1/21/21/2 for ε>0\varepsilon>0ε>0, the payoff f(a)=−2f(a)=-2f(a)=−2, f(b)=1f(b)=1f(b)=1 is soft robust at level 000 while the right side of (6) is empty. The goal and the milestones therefore fix δ>0\delta>0δ>0, the case the paper's main argument treats.

A complete development needs: convex risk measures and their generation by penalties, mixtures of probability measures, Lagrange duality with Slater's condition over convex sets of measures, and the scaling identity for ρQ\rho^QρQ. The duality theorem and the risk-measure layer are reusable beyond this mission. Contributions of any milestone are welcome, as are general convex-duality lemmas that the strong-duality step can import.

Selected references

  • A. Ben-Tal, D. Bertsimas, D. B. Brown, A Soft Robust Model for Optimization Under Ambiguity, Operations Research 58(4, Part 2):1220–1234, 2010. https://doi.org/10.1287/opre.1100.0821
  • H. Föllmer, A. Schied, Convex measures of risk and trading constraints, Finance and Stochastics 6(4):429–447, 2002. https://doi.org/10.1007/s007800200072
  • H. Föllmer, A. Schied, Stochastic Finance: An Introduction in Discrete Time, 2nd ed., de Gruyter, 2004. https://doi.org/10.1515/9783110212075
  • D. G. Luenberger, Optimization by Vector Space Methods, Wiley, 1969, Chapter 8.
  • D. Bertsimas, D. B. Brown, Constructing Uncertainty Sets for Robust Linear Optimization, Operations Research 57(6):1483–1495, 2009. https://doi.org/10.1287/opre.1080.0695
10 thms1 active userReviewed
Convex OptimizationOperations ResearchOptimization·Captain: mikedeng1

A Distributionally Robust Perspective on Uncertainty Quantification and Chance Constrained Programming: Individual Chance Constraints over Markov Ambiguity Sets Are Finite Conic SystemsResearch Paper

Motivation

Many engineering and operations problems must guarantee that a system operates safely with high probability, when the uncertain parameters z~\tilde zz~ are only partially known. A chance constraint asks that the safety condition Sz~≤tS\tilde z \le tSz~≤t hold with probability at least 1−ϵ1-\epsilon1−ϵ for a tolerance ϵ∈(0,1)\epsilon \in (0,1)ϵ∈(0,1). When the distribution of z~\tilde zz~ is itself uncertain, the distributionally robust version asks this for every distribution in an ambiguity set P\mathcal PP built from what is known: moments, supports, confidence regions. Two problems then arise: computing the worst-case probability inf⁡P∈PP[Sz~≤t]\inf_{\mathbb P \in \mathcal P}\mathbb P[S\tilde z \le t]infP∈P​P[Sz~≤t] (uncertainty quantification), and optimizing a design xxx under the constraint inf⁡P∈PP[S(x)z~≤t(x)]≥1−ϵ\inf_{\mathbb P \in \mathcal P}\mathbb P[S(x)\tilde z \le t(x)] \ge 1-\epsiloninfP∈P​P[S(x)z~≤t(x)]≥1−ϵ (chance constrained programming).

Hanasusanto, Roitch, Kuhn and Wiesemann (Math. Program. 151, 2015) chart which combinations of ambiguity sets and constraint structures admit exact finite reformulations and which are strongly NP-hard. This mission takes the positive core of that chart: for Markov ambiguity sets (first-order moment information plus a conic support), the worst-case probability is the value of a finite conic program, and an individual chance constraint is equivalent to a finite system of linear conic inequalities.

Timeline. Chebyshev-type bounds for worst-case probabilities under moment information go back to the generalized Chebyshev inequalities of Vandenberghe, Boyd and Comanor (2007), and to the duality theory of moment problems summarized by Shapiro (2001). Wiesemann, Kuhn and Sim (2014) introduced the nested conic ambiguity sets used here to evaluate worst-case expectations. Hanasusanto et al. (2015) applied them to worst-case probabilities of polytopes and to chance constraints, proving Theorem 3 (moment problem reformulation) and Theorem 7 (individual chance constraints over Markov sets), together with strong NP-hardness results, such as Theorem 8, showing that the restrictions cannot be dropped.

Setting

Let z~∈RP\tilde z \in \mathbb R^Pz~∈RP be the uncertain parameters and u~∈RQ\tilde u \in \mathbb R^Qu~∈RQ an auxiliary random vector. Fix A∈RK×PA \in \mathbb R^{K\times P}A∈RK×P, B∈RK×QB \in \mathbb R^{K\times Q}B∈RK×Q, b∈RKb \in \mathbb R^Kb∈RK, and a confidence set

C1={(z,u):C1z+D1u≼K1d1},\mathcal C_1 = \{(z,u) : C_1 z + D_1 u \preccurlyeq_{\mathcal K_1} d_1\},C1​={(z,u):C1​z+D1​u≼K1​​d1​},

where C1∈RL1×PC_1 \in \mathbb R^{L_1\times P}C1​∈RL1​×P, D1∈RL1×QD_1 \in \mathbb R^{L_1\times Q}D1​∈RL1​×Q, d1∈RL1d_1 \in \mathbb R^{L_1}d1​∈RL1​ and K1\mathcal K_1K1​ is a proper cone (closed, convex, pointed, with nonempty interior); x≼Kyx \preccurlyeq_{\mathcal K} yx≼K​y means y−x∈Ky - x \in \mathcal Ky−x∈K, and K1⋆={y:x⊤y≥0 ∀x∈K1}\mathcal K_1^\star = \{y : x^\top y \ge 0\ \forall x \in \mathcal K_1\}K1⋆​={y:x⊤y≥0 ∀x∈K1​} is the dual cone. The Markov ambiguity set is

P={P:EP[Az~+Bu~]=b, P[(z~,u~)∈C1]=1},\mathcal P = \{\mathbb P : \mathbb E_{\mathbb P}[A\tilde z + B\tilde u] = b,\ \mathbb P[(\tilde z,\tilde u) \in \mathcal C_1] = 1\},P={P:EP​[Az~+Bu~]=b, P[(z~,u~)∈C1​]=1},

the case I=1I = 1I=1, p‾1=p‾1=1\underline p_1 = \overline p_1 = 1p​1​=p​1​=1 of the paper's nested set (4).

Two regularity notions enter. C1\mathcal C_1C1​ is essentially strictly feasible if it contains a point satisfying all non-polyhedral constraints of its description strictly. The Slater condition (S) for I=1I = 1I=1 requires bbb to lie in the interior of {Az+Bu:(z,u)∈C1}\{Az + Bu : (z,u) \in \mathcal C_1\}{Az+Bu:(z,u)∈C1​}.

For the safety system Sz~≤tS\tilde z \le tSz~≤t with rows sj⊤s_j^\topsj⊤​, j∈J={1,…,J}j \in \mathcal J = \{1,\dots,J\}j∈J={1,…,J}, let L={j:∃(z,u)∈C1, sj⊤z>tj}\mathcal L = \{j : \exists (z,u) \in \mathcal C_1,\ s_j^\top z > t_j\}L={j:∃(z,u)∈C1​, sj⊤​z>tj​} be the rows that can be violated on C1\mathcal C_1C1​. Problem (17) at I=1I = 1I=1 maximizes b⊤γ+p‾1λ−p‾1κb^\top\gamma + \underline p_1\lambda - \overline p_1\kappab⊤γ+p​1​λ−p​1​κ over γ∈RK\gamma \in \mathbb R^Kγ∈RK, λ,κ≥0\lambda,\kappa \ge 0λ,κ≥0, ϕ∈K1⋆\phi \in \mathcal K_1^\starϕ∈K1⋆​ and, for j∈Lj \in \mathcal Lj∈L, τj≥0\tau_j \ge 0τj​≥0, ψj∈K1⋆\psi_j \in \mathcal K_1^\starψj​∈K1⋆​, subject to λ−κ+d1⊤ϕ≤1\lambda - \kappa + d_1^\top\phi \le 1λ−κ+d1⊤​ϕ≤1, A⊤γ=C1⊤ϕA^\top\gamma = C_1^\top\phiA⊤γ=C1⊤​ϕ, B⊤γ=D1⊤ϕB^\top\gamma = D_1^\top\phiB⊤γ=D1⊤​ϕ and, for j∈Lj \in \mathcal Lj∈L, λ−κ+d1⊤ψj−τjtj≤0\lambda - \kappa + d_1^\top\psi_j - \tau_j t_j \le 0λ−κ+d1⊤​ψj​−τj​tj​≤0, A⊤γ+τjsj=C1⊤ψjA^\top\gamma + \tau_j s_j = C_1^\top\psi_jA⊤γ+τj​sj​=C1⊤​ψj​, B⊤γ=D1⊤ψjB^\top\gamma = D_1^\top\psi_jB⊤γ=D1⊤​ψj​.

For an individual constraint (J=1J = 1J=1) write s⊤z~≤ts^\top\tilde z \le ts⊤z~≤t. System (26) asks for β∈R\beta \in \mathbb Rβ∈R, γ∈RK\gamma \in \mathbb R^Kγ∈RK, ϕ,ψ∈K1⋆\phi,\psi \in \mathcal K_1^\starϕ,ψ∈K1⋆​, τ≥0\tau \ge 0τ≥0 with

β+b⊤γ≥(1−ϵ)τ,  β+d1⊤ϕ≤τ,  β+d1⊤ψ≤t,  A⊤γ=C1⊤ϕ,  B⊤γ=D1⊤ϕ,  A⊤γ+s=C1⊤ψ,  B⊤γ=D1⊤ψ.\beta + b^\top\gamma \ge (1-\epsilon)\tau,\ \ \beta + d_1^\top\phi \le \tau,\ \ \beta + d_1^\top\psi \le t,\ \ A^\top\gamma = C_1^\top\phi,\ \ B^\top\gamma = D_1^\top\phi,\ \ A^\top\gamma + s = C_1^\top\psi,\ \ B^\top\gamma = D_1^\top\psi.β+b⊤γ≥(1−ϵ)τ,  β+d1⊤​ϕ≤τ,  β+d1⊤​ψ≤t,  A⊤γ=C1⊤​ϕ,  B⊤γ=D1⊤​ϕ,  A⊤γ+s=C1⊤​ψ,  B⊤γ=D1⊤​ψ.

Formalization targets

Goal: Theorem 7 (p. 18)

Under essential strict feasibility of C1\mathcal C_1C1​, (S), and ϵ∈(0,1)\epsilon \in (0,1)ϵ∈(0,1),

inf⁡P∈PP[s⊤z~≤t]≥1−ϵ  ⟺  (26) is solvable.\inf_{\mathbb P \in \mathcal P}\mathbb P[s^\top\tilde z \le t] \ge 1-\epsilon \iff \text{(26) is solvable}.P∈Pinf​P[s⊤z~≤t]≥1−ϵ⟺(26) is solvable.

Milestones

  1. Theorem 3 (case I=1I = 1I=1), sufficiency. Every feasible point of (17) has objective value at most P[Sz~≤t]\mathbb P[S\tilde z \le t]P[Sz~≤t] for every P∈P\mathbb P \in \mathcal PP∈P.
  2. Theorem 3 (case I=1I = 1I=1). Under essential strict feasibility and (S),
inf⁡P∈PP[Sz~≤t]=sup⁡ (17).\inf_{\mathbb P \in \mathcal P}\mathbb P[S\tilde z \le t] = \sup\,(17).P∈Pinf​P[Sz~≤t]=sup(17).
  1. §5 feasibility claim (with (17), case I=1I = 1I=1). Under the same hypotheses and ϵ∈(0,1)\epsilon \in (0,1)ϵ∈(0,1), the chance constraint holds iff (17) has a feasible point of value at least 1−ϵ1-\epsilon1−ϵ.
  2. Theorem 7, sufficiency. Any solution of (26) certifies P[s⊤z~≤t]≥1−ϵ\mathbb P[s^\top\tilde z \le t] \ge 1-\epsilonP[s⊤z~≤t]≥1−ϵ for all P∈P\mathbb P \in \mathcal PP∈P.

Significance

Theorem 7 turns an individual distributionally robust chance constraint over a Markov ambiguity set into finitely many linear conic constraints whose size is polynomial in the data. Since s(x)s(x)s(x) and t(x)t(x)t(x) are affine in the design xxx, the chance constrained program becomes a single conic program, solvable with standard software. The paper notes (p. 18) that, apart from the Wasserstein set, all ambiguity sets of its Section 2.3 are Markov sets or intersections of Markov sets with structural ambiguity sets, and Theorem 8 shows that adding a second confidence set already makes the problem strongly NP-hard, so the theorem marks a precise tractability boundary. Theorem 3 underlies the paper's later results and its §5 heuristic for intractable cases.

All five statements are proved in the literature: the paper defers its proofs to an accompanying technical report. None is machine-checked. A formal proof would supply a verified duality theory for moment problems with conic supports, a reusable bridge between semi-infinite constraints and conic certificates, and a verified basis for the paper's hardness results to be stated against.

Difficulty

The sufficiency halves are weak-duality arguments. The converse directions are not: the worst-case probability is an infimum over an infinite-dimensional set of measures, and the conclusion of Theorem 7 requires the dual certificate to be attained, not just approached. Two separate duality steps must both close without a gap and with attainment: the moment problem against its semi-infinite dual, which is where (S) enters, and each semi-infinite constraint against its conic dual over C1\mathcal C_1C1​, which is where essential strict feasibility enters. The event {s⊤z>t}\{s^\top z > t\}{s⊤z>t} is open, so the semi-infinite constraint must also be passed to its closure. Mathlib has neither a duality theorem for moment problems nor conic strong duality with attainment.

Formalization scope

The mission covers Markov ambiguity sets only (I=1I = 1I=1, p‾1=p‾1=1\underline p_1 = \overline p_1 = 1p​1​=p​1​=1). The general nested case of Theorem 3 is not stated, because the paper does not make (S) explicit for I>1I > 1I>1. Symmetry and unimodality information, joint chance constraints and the hardness results are out of scope. The paper's proofs are in a technical report that the mission does not use.

Conventions committed to in Lean:

  • Vectors are Fin n → ℝ, the single confidence set is index 0 : Fin 1 of the published DRConvexOpt.Reform.AmbData, and its field c 0 is d1d_1d1​.
  • Probabilities are μ.real. The local paperAmbiguitySet reuses the published data and confidence-set definitions and requires integrability only of Az~+Bu~A\tilde z+B\tilde uAz~+Bu~. This makes the expectation condition void when K=0K=0K=0, as the paper states.
  • "inf⁡PP[E]≥1−ϵ\inf_{\mathbb P}\mathbb P[E] \ge 1-\epsiloninfP​P[E]≥1−ϵ" is stated as "1−ϵ≤P[E]1-\epsilon \le \mathbb P[E]1−ϵ≤P[E] for every P∈P\mathbb P \in \mathcal PP∈P", so an empty P\mathcal PP is not mishandled by sInf ∅ = 0.
  • "Coincides with the optimal value" is one real number that is both the greatest lower bound of the probabilities and the least upper bound of the objective values of (17).
  • Essential strict feasibility is read as: K1=Kp∩Kn\mathcal K_1 = \mathcal K_p \cap \mathcal K_nK1​=Kp​∩Kn​ for some polyhedral cone Kp\mathcal K_pKp​ and closed convex cone Kn\mathcal K_nKn​, with a point of C1\mathcal C_1C1​ whose slack lies in Kp∩int⁡Kn\mathcal K_p \cap \operatorname{int}\mathcal K_nKp​∩intKn​.
  • The decision xxx is fixed: s,ts, ts,t stand for s(x),t(x)s(x), t(x)s(x),t(x), and the program (2) and the set X\mathcal XX are not formalized.
  • ϵ∈(0,1)\epsilon \in (0,1)ϵ∈(0,1), proper K1\mathcal K_1K1​, essential strict feasibility and (S) are explicit hypotheses. The sufficiency halves drop (S) and essential strict feasibility, which they do not need.

Replacing essential strict feasibility by full strict feasibility, adding τ>0\tau > 0τ>0 to (26), or imposing the ψj\psi_jψj​-constraints for j∉Lj \notin \mathcal Lj∈/L would each change the theorem and is ruled out. The hypotheses are jointly satisfiable: C1=[−1,1]\mathcal C_1 = [-1,1]C1​=[−1,1], A=1A = 1A=1, b=0b = 0b=0, K1=R+2\mathcal K_1 = \mathbb R^2_+K1​=R+2​ satisfies all of them, with a nonempty ambiguity set.

A complete development needs conic duality with attainment under partial Slater conditions, Lagrangian duality for moment problems, and the reduction of a measure-valued infimum to finitely supported measures. All three are reusable well beyond this mission, and contributions to any of them are welcome.

Selected references

  • G. A. Hanasusanto, V. Roitch, D. Kuhn, W. Wiesemann, A distributionally robust perspective on uncertainty quantification and chance constrained programming, Mathematical Programming 151(1), 35–62, 2015. https://doi.org/10.1007/s10107-015-0896-z
  • W. Wiesemann, D. Kuhn, M. Sim, Distributionally robust convex optimization, Operations Research 62(6), 1358–1376, 2014. https://doi.org/10.1287/opre.2014.1314
  • A. Shapiro, On duality theory of conic linear problems, in: Semi-Infinite Programming, Kluwer, 135–165, 2001. https://doi.org/10.1007/978-1-4757-3403-4_7
  • L. Vandenberghe, S. Boyd, K. Comanor, Generalized Chebyshev bounds via semidefinite programming, SIAM Review 49(1), 52–64, 2007. https://doi.org/10.1137/S0036144504440543
  • A. Ben-Tal, A. Nemirovski, Lectures on Modern Convex Optimization, SIAM, 2001. https://doi.org/10.1137/1.9780898718829
7 thms1 active userReviewed
Operations ResearchOptimization·Captain: mikedeng1

Adaptive Distributionally Robust Optimization: Under Complete Recourse with One Second-Stage Variable, the Linear Decision Rule in the Primary and Auxiliary Variables Is Exact (β = β_E)Research Paper

Why adaptive decisions matter

A planner often chooses a first action before uncertain inputs are known and a corrective action afterward. If the uncertain input has no trusted probability law, a distributionally robust model tests the first action against all laws consistent with known support and moment information. The corrective action may depend on the realized input. This combination produces a difficult optimization problem: the adversary chooses a probability law, while the planner chooses a function of the input. Bertsimas, Sim, and Zhang study a class in which the support and moment bounds have second-order-cone representations. Their particular question is whether a tractable affine corrective rule can recover the exact worst-case cost.

Earlier work on moment-based distributionally robust optimization developed other ambiguity sets and reformulations. The related Prove2Me drafts MinimaxSLP and TractableDRO address mean and second-moment equality classes or bounds on positive parts of affine functions. Their model conventions differ from this paper's lifted SOC support, so this mission develops its own definitions. The affine-policy result of Bertsimas, Iancu, and Parrilo concerns a different multistage robust model. The paper places these contributions in its account of robust and distributionally robust decision rules (pp. 604–605).

Two-stage model and lifted ambiguity set

Write x∈RN1x\in\mathbb R^{N_1}x∈RN1​ for the first-stage decision and z∈RI1z\in\mathbb R^{I_1}z∈RI1​ for the observed uncertainty. After observing zzz, the planner chooses a free second-stage vector y∈RN2y\in\mathbb R^{N_2}y∈RN2​. The constraint data depend affinely on zzz: A(z)=A0+∑iAiziA(z)=A^0+\sum_i A^i z_iA(z)=A0+∑i​Aizi​ and b(z)=b0+∑ibizib(z)=b^0+\sum_i b^i z_ib(z)=b0+∑i​bizi​. The fixed recourse matrix is BBB, and ddd gives the second-stage cost. The recourse value is

Q(x,z)=min⁡{d⊤y:A(z)x+By≥b(z), y∈RN2}.Q(x,z)=\min\{d^\top y:A(z)x+By\ge b(z),\ y\in\mathbb R^{N_2}\}.Q(x,z)=min{d⊤y:A(z)x+By≥b(z), y∈RN2​}.

The support set WWW restricts possible realizations. An SOC ambiguity set FFF contains probability laws PPP on zzz for which P(W)=1P(W)=1P(W)=1, EP[Gz]=μ\mathbb E_P[Gz]=\muEP​[Gz]=μ, and EP[gi(z)]≤σi\mathbb E_P[g_i(z)]\le\sigma_iEP​[gi​(z)]≤σi​ for each auxiliary function gig_igi​. The paper assumes that WWW and each epigraph {(z,ui):gi(z)≤ui}\{(z,u_i):g_i(z)\le u_i\}{(z,ui​):gi​(z)≤ui​} can be represented using products of second-order cones (Definition 3). Its exact worst-case expected recourse cost is β(x)=sup⁡P∈FEP[Q(x,z)]\beta(x)=\sup_{P\in F}\mathbb E_P[Q(x,z)]β(x)=supP∈F​EP​[Q(x,z)].

The model introduces an auxiliary vector u∈RI2u\in\mathbb R^{I_2}u∈RI2​ with lifted support Wˉ={(z,u):z∈W, g(z)≤u}\bar W=\{(z,u):z\in W,\ g(z)\le u\}Wˉ={(z,u):z∈W, g(z)≤u}. The lifted ambiguity set G\mathcal GG consists of joint laws on (z,u)(z,u)(z,u) supported on Wˉ\bar WWˉ with E[Gz]=μ\mathbb E[Gz]=\muE[Gz]=μ and E[u]≤σ\mathbb E[u]\le\sigmaE[u]≤σ. Proposition 1 states that the primary marginals of these joint laws are exactly FFF. A regular linear decision rule has y(z)=y0+Y1zy(z)=y^0+Y^1zy(z)=y0+Y1z. An extended linear decision rule has y(z,u)=y0+Y1z+Y2uy(z,u)=y^0+Y^1z+Y^2uy(z,u)=y0+Y1z+Y2u. Their best worst-case costs are βL(x)\beta_L(x)βL​(x) and βE(x)\beta_E(x)βE​(x), with feasibility enforced at every point of WWW or Wˉ\bar WWˉ respectively (§3.1).

Formalization targets

Theorem 2 gives the comparison that holds for the two rule classes:

β(x)≤βE(x)≤βL(x).\beta(x)\le\beta_E(x)\le\beta_L(x).β(x)≤βE​(x)≤βL​(x).

Theorem 1 identifies β(x)\beta(x)β(x) with the robust program (9). Its variables are r∈Rr\in\mathbb Rr∈R, s∈RL1s\in\mathbb R^{L_1}s∈RL1​, and t∈R+I2t\in\mathbb R^{I_2}_+t∈R+I2​​, constrained by r+s⊤Gz+t⊤u≥Q(x,z)r+s^\top Gz+t^\top u\ge Q(x,z)r+s⊤Gz+t⊤u≥Q(x,z) at every (z,u)∈Wˉ(z,u)\in\bar W(z,u)∈Wˉ; its objective is r+s⊤μ+t⊤σr+s^\top\mu+t^\top\sigmar+s⊤μ+t⊤σ. This result assumes relative complete recourse and Assumption 1, which represents the actual lifted support by a product of second-order cones and supplies a strict Slater point.

The mission's goal is Theorem 4. Under complete recourse and one second-stage decision variable, the extended affine class gives the exact value for every first-stage vector:

∀x∈RN1,β(x)=βE(x).\forall x\in\mathbb R^{N_1},\qquad \beta(x)=\beta_E(x).∀x∈RN1​,β(x)=βE​(x).

The equality is stronger than the bounds alone: it says that optimizing over this finite-parameter decision-rule class loses no worst-case value in the stated regime (Theorems 1, 2, and 4).

What the result supplies

Theorem 4 gives a precise case where the lifted affine approximation is exact. This matters because the unrestricted corrective decision is a function, while the affine class is described by finitely many coefficients. The paper observes that, with one recourse variable, the associated dual polyhedron has only MMM extreme points; the robust reformulation can therefore be handled using a finite collection of constraints in this regime (p. 610). Proposition 3 marks a boundary: relative complete recourse by itself does not ensure that any extended affine rule is feasible.

The results are proved mathematically in the published paper and its electronic companion; the proofs of Theorems 1 and 4 are located in the companion, which is not held with this mission's source PDF. The Lean statements here are open proof obligations. A completed development would add reusable interfaces for free-variable linear recourse, SOC-supported moment ambiguity sets, lifted joint laws, and extended-real optimization values. It would also make the assumptions behind the paper's real-valued expectations and recourse costs explicit.

The hard point

Replacing the adaptive function y(z)y(z)y(z) by an affine function of zzz alone can fail even when the recourse LP is feasible for every realization: a single affine rule must satisfy every pointwise constraint simultaneously. Adding uuu enlarges the rule class, but the lifted support contains all u≥g(z)u\ge g(z)u≥g(z), not just u=g(z)u=g(z)u=g(z). Thus its feasibility condition is stronger than feasibility on a convenient graph. Theorem 4 asserts exactness despite that universal lifted-support constraint. The direct comparison β≤βE\beta\le\beta_Eβ≤βE​ does not by itself give the reverse inequality; the robust moment reformulation must describe the same worst-case value under its Slater and recourse hypotheses (§§2.2–3.1).

Formalization scope

Finite vectors are functions on Fin n, so the paper's indices 1,…,n1,\ldots,n1,…,n become zero-based. Laws are probability measures on finite real-vector spaces. The original and lifted ambiguity sets include integrability of all primary coordinates; the lifted set also includes integrability of all auxiliary coordinates, and the original set includes integrability of each gig_igi​. This is a finite-first-moment reading of the paper's expectations. It excludes laws whose constrained linear moments exist while an unconstrained coordinate has infinite absolute moment. These conditions ensure that expected affine rule costs are genuine. The paper's Theorem 3 itself explicitly asks for finite coordinate first moments.

The recourse LP is represented as a separate reusable definition with free yyy, constraints By≥tBy\ge tBy≥t, and dual feasibility p≥0p\ge0p≥0, B⊤p=dB^\top p=dB⊤p=d. Statements using its real value require primal and dual feasibility. Theorem 2 states primal feasibility on the support at its chosen xxx; Theorem 1 uses relative complete recourse on X×WX\times WX×W; Theorem 4 uses complete recourse for every right-hand side. Theorems 1 and 4 also state dual feasibility explicitly because primal feasibility alone does not ensure it. Assumption 1 is explicit in both, including equality between its cone representation and the previously defined Wˉ\bar WWˉ. Its bound u†<σu^\dagger<\sigmau†<σ means strict inequality in every auxiliary coordinate, and strict cone feasibility means membership in the interior of the product cone.

The values β\betaβ, βL\beta_LβL​, βE\beta_EβE​, and (9) lie in the extended reals. Each printed “min” is an infimum, and no optimizer is presumed. A value is never formed by subtracting infinities. The SOC support, the epigraphs, and all rule feasibility conditions retain their pointwise meanings. Neither an empty ambiguity set nor an impossible rule class is silently replaced by a favorable real default. Contributions to LP value regularity, integrability, moment duality, and SOC geometry are all useful beyond this mission. The goal must continue to compare the original β\betaβ with the actual lifted-rule value βE\beta_EβE​; replacing either by an easier surrogate would remove its content.

Selected references

  • Dimitris Bertsimas, Melvyn Sim, and Meixia Zhang, Adaptive Distributionally Robust Optimization, Management Science 65(2):604–618, 2019. DOI.
6 thms1 active userReviewed
Captain: Lucas

KPZ Universality Conjecture: Ballistic Deposition Height Fluctuations of Order t^{1/3}Open Problem

Motivation

A large family of random growth models, interacting particle systems and directed polymers in one space dimension share a common, non-Gaussian fluctuation theory: observables of size ttt fluctuate on the scale t1/3t^{1/3}t1/3 (instead of the central-limit scale t1/2t^{1/2}t1/2), with spatial correlations on the scale t2/3t^{2/3}t2/3. This family is the Kardar–Parisi–Zhang (KPZ) universality class, named after the 1986 physics paper of Kardar, Parisi and Zhang. The KPZ universality conjecture (Open Problem Garden, posted 2020, importance "High") asks for a central limit theorem for this class: any one-dimensional growth model with a smoothing mechanism, short-range noise and slope-dependent ("lateral") growth should exhibit KPZ fluctuations.

KPZ fluctuations have been proved only for models with exact algebraic structure, such as the corner growth model / totally asymmetric exclusion process (Johansson, 2000) and the polynuclear growth model (Prähofer–Spohn, 2002). The Open Problem Garden entry describes ballistic deposition (Vold, 1959) as a concrete growth model with the KPZ features. Corwin's survey states the prediction that its column heights fluctuate like t1/3t^{1/3}t1/3 and notes that "this prediction has not been proved" (Corwin 2012, §1.1.1). Ballistic deposition has no known exact solvability, so it tests universality itself rather than one integrable instance of it.

Timeline.

  • 1959 — Vold introduces ballistic deposition as a sediment model.
  • 1986 — Kardar–Parisi–Zhang predict the exponents 1/31/31/3 and 2/32/32/3.
  • 2000 — Seppäläinen proves a strong law of large numbers (hydrodynamic limit) for the ballistic deposition interface (arXiv:math/9906193).
  • 2008 — Penrose proves linear growth of the mean height, LpL^pLp laws of large numbers, polynomial moment bounds, a duality with a seed-grown process, and the only known lower bound on fluctuations: Var⁡ξt(0)≳log⁡t\operatorname{Var}\xi_t(0)\gtrsim\log tVarξt​(0)≳logt in d=1d=1d=1 (arXiv:math/0608540).
  • 2021 — Chatterjee proves existence of a stationary measure for the gradient process (arXiv:2110.10294).
  • 2026 — Sudijono proves exponential concentration and the upper bound Var⁡ξt(0)≤Ct\operatorname{Var}\xi_t(0)\le CtVarξt​(0)≤Ct (arXiv:2609.24056).

The gap between log⁡t\log tlogt and ttt is open; the predicted order is t2/3t^{2/3}t2/3.

Setting

Columns are indexed by x∈Zx\in\mathbb Zx∈Z. Each column carries an independent rate-one Poisson clock: on a probability space (Ω,P)(\Omega,P)(Ω,P) the inter-arrival times τx,k\tau_{x,k}τx,k​, (x,k)∈Z×N(x,k)\in\mathbb Z\times\mathbb N(x,k)∈Z×N, are independent and Exp(1)\mathrm{Exp}(1)Exp(1)-distributed, and the nnn-th ring of column xxx (counting from n=0n=0n=0) is at time Tx,n=τx,0+⋯+τx,nT_{x,n}=\tau_{x,0}+\dots+\tau_{x,n}Tx,n​=τx,0​+⋯+τx,n​.

Random deposition. The height Nt(x)N_t(x)Nt​(x) of column xxx at time ttt is the number of rings of its clock in [0,t][0,t][0,t]. Columns are independent and Nt(x)N_t(x)Nt​(x) is Poisson with mean ttt.

Ballistic deposition (nearest-neighbour model, flat initial surface). Write ξt(x)∈N\xi_t(x)\in\mathbb Nξt​(x)∈N for the height of column xxx at time ttt, with ξ0≡0\xi_0\equiv0ξ0​≡0. When the clock of column xxx rings at time ttt, a unit block falls in column xxx and sticks at the first point of contact, so

ξt(x)=max⁡{ξt−(x)+1, ξt−(x−1), ξt−(x+1)},\xi_t(x)=\max\{\xi_{t-}(x)+1,\ \xi_{t-}(x-1),\ \xi_{t-}(x+1)\},ξt​(x)=max{ξt−​(x)+1, ξt−​(x−1), ξt−​(x+1)},

and all other columns are unchanged. On the finite substrate QL=[−L,L]Q_L=[-L,L]QL​=[−L,L] only clocks in QLQ_LQL​ ring and columns outside QLQ_LQL​ stay at height 000; the finitely many rings in [0,t][0,t][0,t] are applied in time order. The height on Z\mathbb ZZ is the monotone limit ξt(x)=lim⁡L→∞ξtQL(x)=sup⁡LξtQL(x)\xi_t(x)=\lim_{L\to\infty}\xi^{Q_L}_t(x)=\sup_L\xi^{Q_L}_t(x)ξt​(x)=limL→∞​ξtQL​​(x)=supL​ξtQL​​(x). The next-arrival height is ηt(x)=max⁡{ξt(x)+1,ξt(x−1),ξt(x+1)}\eta_t(x)=\max\{\xi_t(x)+1,\xi_t(x-1),\xi_t(x+1)\}ηt​(x)=max{ξt​(x)+1,ξt​(x−1),ξt​(x+1)}. This is the model of Penrose (2008) with d=1d=1d=1 and the nearest-neighbour displacement function, and the model (1) of Sudijono (2026) with d=1d=1d=1.

Formalization targets

Goal: the KPZ fluctuation exponent

∃ c,C>0, T:c t2/3 ≤ Var⁡ξt(0) ≤ C t2/3(t≥T).\exists\,c,C>0,\ T:\qquad c\,t^{2/3}\ \le\ \operatorname{Var}\xi_t(0)\ \le\ C\,t^{2/3}\qquad(t\ge T).∃c,C>0, T:ct2/3 ≤ Varξt​(0) ≤ Ct2/3(t≥T).

This is the statement "the column height fluctuates around its mean like t1/3t^{1/3}t1/3" (Corwin 2012, §1.1.1; Sudijono 2026, §1). The constants are left unspecified because they are model-dependent.

Stronger (not formalized here)

The predicted scaling limit: (ξt(0)−ρt)/(σt1/3)(\xi_t(0)-\rho t)/(\sigma t^{1/3})(ξt​(0)−ρt)/(σt1/3) converges in law to the Tracy–Widom GOE distribution for some model-dependent ρ,σ>0\rho,\sigma>0ρ,σ>0.

Milestones (known results)

  1. Random deposition CLT: (Nt(0)−t)/t⇒N(0,1)(N_t(0)-t)/\sqrt t\Rightarrow\mathcal N(0,1)(Nt​(0)−t)/t​⇒N(0,1), the Gaussian class, for contrast.
  2. Penrose, Prop. 2.1: ξt(0),ηt(0)<∞\xi_t(0),\eta_t(0)<\inftyξt​(0),ηt​(0)<∞ a.s. and E[ξt(0)k],E[ηt(0)k]=O(tk)E[\xi_t(0)^k],E[\eta_t(0)^k]=O(t^k)E[ξt​(0)k],E[ηt​(0)k]=O(tk).
  3. Penrose, Thm. 2.1: Eξt(0)/t→ρ1∈(0,∞)E\xi_t(0)/t\to\rho_1\in(0,\infty)Eξt​(0)/t→ρ1​∈(0,∞) and ξt(0)/t→ρ1\xi_t(0)/t\to\rho_1ξt​(0)/t→ρ1​ in every LpL^pLp, p≥1p\ge1p≥1.
  4. Seppäläinen, Thm. 1 (as noted in Penrose, p. 5): ξt(0)/t→ρ1\xi_t(0)/t\to\rho_1ξt​(0)/t→ρ1​ almost surely.
  5. Penrose, Thm. 2.2: lim inf⁡tVar⁡ξt(0)>0\liminf_t\operatorname{Var}\xi_t(0)>0liminft​Varξt​(0)>0 and lim inf⁡tVar⁡ξt(0)/log⁡t>0\liminf_t\operatorname{Var}\xi_t(0)/\log t>0liminft​Varξt​(0)/logt>0.
  6. Sudijono, Thm. 1.1 (2): P(∣ξt(0)−Eξt(0)∣≥r)≤Cexp⁡(−c r2/(t+r))P(|\xi_t(0)-E\xi_t(0)|\ge r)\le C\exp(-c\,r^2/(t+r))P(∣ξt​(0)−Eξt​(0)∣≥r)≤Cexp(−cr2/(t+r)) for t≥1t\ge1t≥1, r≥0r\ge0r≥0.
  7. Sudijono, Thm. 1.1 (3): Var⁡ξt(0)≤Ct\operatorname{Var}\xi_t(0)\le CtVarξt​(0)≤Ct for t≥1t\ge1t≥1.

Significance

A proof of the goal would establish the KPZ exponent for a growth model without exact solvability, which is the content of the universality conjecture as opposed to its integrable special cases. For ballistic deposition only log⁡t≲Var⁡ξt(0)≲t\log t\lesssim\operatorname{Var}\xi_t(0)\lesssim tlogt≲Varξt​(0)≲t is currently known.

On the formal side, the milestones are known results with published proofs; formalizing them requires a library for continuous-time interacting particle systems on Z\mathbb ZZ (graphical constructions, monotone couplings, finite speed of propagation), subadditive ergodic theory, and concentration inequalities for Poisson-driven processes. These pieces are reusable for first-passage percolation and other growth models. The goal itself is open.

Difficulty

The standard tools give the wrong exponents. Bounded-difference concentration applied to a last-passage representation fails because adding one clock ring can change future heights by an unbounded amount (Sudijono 2026, §1); martingale hitting-time arguments reach only the diffusive bound O(t)O(t)O(t). Lower bounds of Pemantle–Peres type reach only log⁡t\log tlogt, as in first-passage percolation, where the analogous gap is also open. The exact formulas (determinantal structure, Bethe ansatz) behind the proved instances of KPZ fluctuations are not available for ballistic deposition.

Formalization scope

The model is encoded on a general probability space (Ω,P)(\Omega,P)(Ω,P) with inter-arrival times τ : ℤ → ℕ → Ω → ℝ satisfying IsPoissonClocks P τ (measurability, mutual independence over ℤ × ℕ, each law expMeasure 1). The finite-substrate process boxHeight τ L t ω sorts the rings in [−L,L]×[0,t][-L,L]\times[0,t][−L,L]×[0,t] lexicographically by (time, column) and folds the update rule bdStep; bdHeight τ t x ω : ℕ∞ is the supremum over LLL. Real-valued statements use bdHeightReal, which is bdHeight cast to ℝ with the value 000 on the event {ξt(x)=∞}\{\xi_t(x)=\infty\}{ξt​(x)=∞}; this event is null (Milestone 2), so the convention does not affect any distributional statement. Ties between rings and non-divergent arrival times are null events and receive junk values. Variances are Mathlib's variance, which returns 000 for non-square-integrable variables; the lower bound in the goal therefore also forces square-integrability, so the goal cannot hold vacuously. Moments in Milestone 2 are lower Lebesgue integrals in [0,∞][0,\infty][0,∞], so they are not trivialised by non-integrability.

Contributions welcome: the graphical construction and its basic properties (monotonicity in LLL and ttt, a.s. finiteness), the duality with seed-grown deposition, subadditivity of hitting times, and the Poisson CLT.

Selected references

  • I. Corwin, The Kardar–Parisi–Zhang equation and universality class, Random Matrices Theory Appl. 1 (2012). https://arxiv.org/abs/1106.1596
  • M. Kardar, G. Parisi, Y.-C. Zhang, Dynamic scaling of growing interfaces, Phys. Rev. Lett. 56 (1986). https://doi.org/10.1103/PhysRevLett.56.889
  • T. Kojar (poster), KPZ Universality Conjecture, Open Problem Garden (2020). http://www.openproblemgarden.org/op/kpz_universality_conjecture
  • M. D. Penrose, Growth and roughness of the interface for ballistic deposition, J. Stat. Phys. 131 (2008). https://arxiv.org/abs/math/0608540
  • T. Seppäläinen, Strong law of large numbers for the interface in ballistic deposition, Ann. Inst. H. Poincaré Probab. Statist. 36 (2000). https://arxiv.org/abs/math/9906193
  • T. Sudijono, A linear variance bound for ballistic deposition, preprint (2026). https://arxiv.org/abs/2609.24056
  • S. Chatterjee, Existence of stationary ballistic deposition on the infinite lattice, preprint (2021). https://arxiv.org/abs/2110.10294
  • K. Johansson, Shape fluctuations and random matrices, Comm. Math. Phys. 209 (2000).
  • M. J. Vold, A numerical approach to the problem of sediment volume, J. Colloid Sci. 14 (1959).
9 thms1 active userReviewed
Convex OptimizationDynamic ProgrammingOperations Research+1·Captain: mikedeng1

Convergence Analysis of Sampling-Based Decomposition Methods for Risk-Averse Multistage Stochastic Convex Programs 1: Almost Sure Convergence of the Risk-Averse Sampling-Based Decomposition AlgorithmResearch Paper

Motivation

Multistage stochastic programs model sequential decisions under uncertainty: hydro-thermal scheduling, portfolio management and capacity planning are standard examples. Their dynamic programming equations involve recourse functions (cost-to-go functions) of the decision history, which cannot be computed exactly. Stochastic Dual Dynamic Programming (SDDP), introduced by Pereira and Pinto in 1991, approximates each recourse function from below by a maximum of affine cuts, built at trial points generated by simulating sampled scenarios. SDDP is the method of choice in the energy industry for long-horizon stochastic planning.

Two extensions motivate this mission. Risk-averse planning replaces the expectation at each stage by a coherent risk measure (for instance the Conditional Value-at-Risk), and many applications have convex but nonlinear stage problems. Guigues (2016) proves that the risk-averse sampling algorithm for convex stage problems converges almost surely.

Timeline.

  • 1991: Pereira and Pinto introduce SDDP for linear multistage problems.
  • 2008: Philpott and Guan prove almost sure finite convergence of SDDP-type methods for linear problems with finitely many scenarios.
  • 2011: Shapiro analyses the sample-average SDDP method and formulates its risk-averse variant with nested conditional risk measures; 2013: Philpott, de Matos and Finardi treat SDDP with coherent risk measures in dual form.
  • 2015: Girardeau, Leclère and Philpott prove almost sure (asymptotic) convergence of sampling-based decomposition methods for risk-neutral multistage stochastic convex programs.
  • 2016: Guigues extends that analysis to risk-averse problems in dual form, with nonlinear constraints and interstage independent processes (arXiv:1408.4439, SIAM J. Optim.); the convergence theorem is Theorem 4.1.

Setting

There are TTT stages with decisions xt∈Rnx_t\in\mathbb R^nxt​∈Rn and a given x0x_0x0​. A history x1:t=(x1,…,xt)x_{1:t}=(x_1,\dots,x_t)x1:t​=(x1​,…,xt​) is identified with the vector [x1;… ;xt]∈Rnt[x_1;\dots;x_t]\in\mathbb R^{nt}[x1​;…;xt​]∈Rnt with its Euclidean norm. The random data ξt\xi_tξt​ are interstage independent (H1). For t≥2t\ge2t≥2, ξt\xi_tξt​ takes MMM values ξt,1,…,ξt,M\xi_{t,1},\dots,\xi_{t,M}ξt,1​,…,ξt,M​ with probabilities Φt,j>0\Phi_{t,j}>0Φt,j​>0, and ξ1\xi_1ξ1​ is deterministic. A realization ξt,j\xi_{t,j}ξt,j​ consists of a cost ft(⋅,Ψt,j)f_t(\cdot,\Psi_{t,j})ft​(⋅,Ψt,j​) with values in R∪{+∞}\mathbb R\cup\{+\infty\}R∪{+∞}, convex constraints gt(x0,⋅,Ψt,j)≤0g_t(x_0,\cdot,\Psi_{t,j})\le0gt​(x0​,⋅,Ψt,j​)≤0 and linear constraints ∑τ=0tAt,τ,jxτ=bt,j\sum_{\tau=0}^tA_{t,\tau,j}x_\tau=b_{t,j}∑τ=0t​At,τ,j​xτ​=bt,j​. Together with a compact set Xt\mathcal X_tXt​, they define the feasible set Xt(x0:t−1,ξt,j)X_t(x_{0:t-1},\xi_{t,j})Xt​(x0:t−1​,ξt,j​).

The risk measure of stage ttt is given in dual form,

ρt(Z)=sup⁡p∈Pt∑j=1MpjΦt,jZj,Pt⊆Dt={p≥0:∑jpjΦt,j=1}.\rho_t(Z)=\sup_{p\in\mathcal P_t}\sum_{j=1}^Mp_j\Phi_{t,j}Z_j,\qquad \mathcal P_t\subseteq\mathcal D_t=\Big\{p\ge0:\sum_jp_j\Phi_{t,j}=1\Big\}.ρt​(Z)=p∈Pt​sup​j=1∑M​pj​Φt,j​Zj​,Pt​⊆Dt​={p≥0:j∑​pj​Φt,j​=1}.

The recourse functions satisfy QT+1≡0\mathcal Q_{T+1}\equiv0QT+1​≡0 and, for t=T,…,2t=T,\dots,2t=T,…,2,

Qt(x1:t−1,ξt,j)=inf⁡xt∈Xt(x0:t−1,ξt,j)ft(x1:t,Ψt,j)+Qt+1(x1:t),Qt(x1:t−1)=ρt(Qt(x1:t−1,ξt)).\mathfrak Q_t(x_{1:t-1},\xi_{t,j})=\inf_{x_t\in X_t(x_{0:t-1},\xi_{t,j})}f_t(x_{1:t},\Psi_{t,j})+\mathcal Q_{t+1}(x_{1:t}),\qquad \mathcal Q_t(x_{1:t-1})=\rho_t\big(\mathfrak Q_t(x_{1:t-1},\xi_t)\big).Qt​(x1:t−1​,ξt,j​)=xt​∈Xt​(x0:t−1​,ξt,j​)inf​ft​(x1:t​,Ψt,j​)+Qt+1​(x1:t​),Qt​(x1:t−1​)=ρt​(Qt​(x1:t−1​,ξt​)).

The first-stage value is Q1(x0)=Q1(x0,ξ1)\mathcal Q_1(x_0)=\mathfrak Q_1(x_0,\xi_1)Q1​(x0​)=Q1​(x0​,ξ1​). Assumption (H2) requires convexity, lower semicontinuity and properness of the data, and fixes an ε>0\varepsilon>0ε>0 such that the costs are finite and the feasible sets nonempty on the ε\varepsilonε-fattening [X1×⋯×Xt−1]ε[\mathcal X_1\times\dots\times\mathcal X_{t-1}]^\varepsilon[X1​×⋯×Xt−1​]ε. It also asks for a relative-interior point of the nonlinear constraints.

Algorithm 1 keeps cutting-plane models Qtk\mathcal Q^k_tQtk​, starting from Qt0≡−∞\mathcal Q^0_t\equiv-\inftyQt0​≡−∞. At iteration kkk it computes decisions xmkx^k_mxmk​ at every node mmm of the scenario tree, by solving the stage problems with Qt+1k−1\mathcal Q^{k-1}_{t+1}Qt+1k−1​ in place of Qt+1\mathcal Q_{t+1}Qt+1​. It then samples one scenario n1k,…,nTkn^k_1,\dots,n^k_Tn1k​,…,nTk​, with indices ξtk\xi^k_tξtk​ satisfying the independence assumption (H3), and adds at each sampled node one cut

θtk+⟨βtk, ⋅−x[nt−1k]k⟩,θtk=∑mpk,mΦmQtk−1(x[n]k,ξm),βtk=∑mpk,mΦmπk,m.\theta^k_t+\langle\beta^k_t,\,\cdot-x^k_{[n^k_{t-1}]}\rangle,\qquad \theta^k_t=\sum_mp_{k,m}\Phi_m\mathfrak Q^{k-1}_t(x^k_{[n]},\xi_m),\quad\beta^k_t=\sum_mp_{k,m}\Phi_m\pi_{k,m}.θtk​+⟨βtk​,⋅−x[nt−1k​]k​⟩,θtk​=m∑​pk,m​Φm​Qtk−1​(x[n]k​,ξm​),βtk​=m∑​pk,m​Φm​πk,m​.

Here pkp^kpk attains the supremum in ρt\rho_tρt​, and πk,m\pi_{k,m}πk,m​ is a subgradient of Qtk−1(⋅,ξm)\mathfrak Q^{k-1}_t(\cdot,\xi_m)Qtk−1​(⋅,ξm​).

Formalization targets

Goal: Theorem 4.1 (p. 12)

Almost surely, for t=2,…,T+1t=2,\dots,T+1t=2,…,T+1 and every node nnn of stage t−1t-1t−1,

lim⁡k→∞Qt(x[n]k)−Qtk(x[n]k)=0.\lim_{k\to\infty}\mathcal Q_t(x^k_{[n]})-\mathcal Q^k_t(x^k_{[n]})=0.k→∞lim​Qt​(x[n]k​)−Qtk​(x[n]k​)=0.

Almost surely, moreover, Q1k(x0,ξ1)→Q1(x0)\mathfrak Q^k_1(x_0,\xi_1)\to\mathcal Q_1(x_0)Q1k​(x0​,ξ1​)→Q1​(x0​), and every accumulation point of the first-stage decisions (x1k)(x^k_1)(x1k​) is an optimal solution of the first-stage problem.

Milestones

  • Proposition 2.2: subgradient bound ∥s∥≤(M0−m0)/ε\|s\|\le(M_0-m_0)/\varepsilon∥s∥≤(M0​−m0​)/ε for the value function of a convex program.
  • Proposition 3.1: Qt\mathcal Q_tQt​ is convex, finite and continuous on [X1×⋯×Xt−1]ε^[\mathcal X_1\times\dots\times\mathcal X_{t-1}]^{\hat\varepsilon}[X1​×⋯×Xt−1​]ε^, 0<ε^<ε0<\hat\varepsilon<\varepsilon0<ε^<ε.
  • Lemma 3.2: valid cuts Qtk≤Qt\mathcal Q^k_t\le\mathcal Q_tQtk​≤Qt​, bounded cut coefficients and slopes, and Lipschitz models for k≥T−t+1k\ge T-t+1k≥T−t+1.
  • Lemma 3.4: Qtk(x[nt−1k]k)=θtk\mathcal Q^k_t(x^k_{[n^k_{t-1}]})=\theta^k_tQtk​(x[nt−1k​]k​)=θtk​.
  • Lemma 3.5: a uniform Lipschitz constant LLL for the models.
  • (4.26), (4.28), (4.29), (4.31): the steps of the proof of Theorem 4.1. These are the error propagation along the tree, the limit along the iterations through a node, the almost sure limit along the other iterations, and the first-stage bound.

Significance

Theorem 4.1 guarantees that the algorithm's lower bound Q1k(x0,ξ1)\mathfrak Q^k_1(x_0,\xi_1)Q1k​(x0​,ξ1​) converges to the true risk-averse optimal value, and that its first-stage decisions approach optimal ones. It covers nonlinear convex stages, where finite convergence arguments for polyhedral problems do not apply. It also covers every risk measure with a dual representation, including CV@R and its combinations with expectation. Practitioners rely on this to justify stopping rules based on the lower bound.

The result is proved on paper. To our knowledge neither Theorem 4.1 nor its risk-neutral counterpart (Girardeau–Leclère–Philpott, Theorem 3.1) has been machine-checked; the risk-neutral statements are posed, unproved, on this platform. A formalization would check a proof that combines convex analysis (value functions of convex programs, subgradient bounds, Lipschitz continuity on fattenings), cutting-plane bookkeeping and a probabilistic argument (strong law of large numbers along a predictable subsequence).

Difficulty

The cuts are built only along sampled scenarios, so at a given node the approximation is refined only at the iterations whose scenario passes through it. The obvious induction argument controls the error Qt−Qtk\mathcal Q_t-\mathcal Q^k_tQt​−Qtk​ only along those iterations. Extending it to all iterations is the probabilistic core (4.29). It needs the samples to be independent of everything the algorithm has computed so far, and it needs a strong law of large numbers along the subsequence of iterations at which the error is still large. A second difficulty is passing from the current model Qt+1k\mathcal Q^k_{t+1}Qt+1k​ to the previous one Qt+1k−1\mathcal Q^{k-1}_{t+1}Qt+1k−1​. This needs uniform Lipschitz bounds on the models (Lemmas 3.2, 3.5), which in turn rest on the fattening in (H2) and on the subgradient bound of Proposition 2.2.

Formalization scope

  • Histories and norms. Histories are L2L^2L2 products of copies of Rn\mathbb R^nRn (PiLp 2), so fattenings, inner products and Lipschitz constants use the Euclidean norm on Rn(t−1)\mathbb R^{n(t-1)}Rn(t−1). Fattenings are closed thickenings.
  • Extended reals. Values are in EReal, infima are +∞+\infty+∞ on empty feasible sets, and weights pjΦt,jp_j\Phi_{t,j}pj​Φt,j​ are real numbers (with 0⋅(±∞)=00\cdot(\pm\infty)=00⋅(±∞)=0). Limits of gaps are taken in EReal, so that convergence to 000 forces the gaps to become finite.
  • Data and indices. The recourse functions are constructed by the backward recursion from the data. The scenario tree is the product tree indexed by histories of realization indices, so (H1) is built into the model. Stage 1 uses the data of the first realization index.
  • Runs. A run is pinned to fixed deterministic choice rules (minimizers, maximizing pkp^kpk, subgradients), so it uses only present and past samples. The goal holds for every choice of rules.
  • Departures from the printed text, disclosed in the items.
    • The cut slope πk,m\pi_{k,m}πk,m​ is any subgradient of Qtk−1(⋅,ξm)\mathfrak Q^{k-1}_t(\cdot,\xi_m)Qtk−1​(⋅,ξm​). The printed formula on p. 9 uses a partial subgradient of ftf_tft​, which is not a subgradient when ftf_tft​ is nonsmooth, and the proof only uses the subgradient property.
    • Pt≠∅\mathcal P_t\ne\emptysetPt​=∅ is assumed; it is implicit, because ρt\rho_tρt​ is a real-valued coherent risk measure.
    • In Proposition 2.2, M0M_0M0​ is an extended-real supremum. The proof's claim that M0M_0M0​ is finite does not follow from lower semicontinuity.
    • Lemma 3.2's vacuous "almost surely" is dropped.
    • One ε\varepsilonε serves all stages in (H2).
  • Ruled out. A formalization in which the decisions or cuts ignore the models, or in which Pt=∅\mathcal P_t=\emptysetPt​=∅ or every stage value is +∞+\infty+∞, would make the limits trivial. The decisions must minimize against Qt+1k−1\mathcal Q^{k-1}_{t+1}Qt+1k−1​, the cut must use the attained supremum, and the companion item run_exists shows that under (H2), with closed Pt\mathcal P_tPt​, rules producing a run exist for every sample path.
  • Infrastructure and contributions. A complete development needs value functions of parametric convex programs, subgradient bounds for convex functions on fattenings, Lipschitz continuity of finite maxima of affine functions, and a strong law of large numbers along predictably selected subsequences (Lemma A.3 of Girardeau–Leclère–Philpott). The last two are reusable well beyond this mission. Contributions to any milestone, or to these general lemmas, are welcome.

Selected references

  • V. Guigues, Convergence analysis of sampling-based decomposition methods for risk-averse multistage stochastic convex programs, SIAM J. Optim. 26(4), 2016; preprint arXiv:1408.4439v4. https://arxiv.org/abs/1408.4439
  • P. Girardeau, V. Leclère, A. B. Philpott, On the convergence of decomposition methods for multistage stochastic convex programs, Math. Oper. Res. 40(1), 2015. https://doi.org/10.1287/moor.2014.0664
  • M. V. F. Pereira, L. M. V. G. Pinto, Multi-stage stochastic optimization applied to energy planning, Math. Program. 52, 1991. https://doi.org/10.1007/BF01582895
  • A. B. Philpott, Z. Guan, On the convergence of stochastic dual dynamic programming and related methods, Oper. Res. Lett. 36(4), 2008. https://doi.org/10.1016/j.orl.2008.01.013
  • A. B. Philpott, V. L. de Matos, E. Finardi, On solving multistage stochastic programs with coherent risk measures, Oper. Res. 61(4), 2013. https://doi.org/10.1287/opre.2013.1175
  • A. Shapiro, Analysis of stochastic dual dynamic programming method, European J. Oper. Res. 209(1), 2011. https://doi.org/10.1016/j.ejor.2010.08.007
15 thms1 active userReviewed
Operations ResearchStochastic Systems·Captain: mikedeng1

Universality of Power-of-d Load Balancing in Many-Server Systems 2: JSQ(d(N)) with d(N)/(√N log N) → ∞ Has the Same Halfin–Whitt Diffusion Limit as the Join-the-Shortest-Queue PolicyResearch Paper

Motivation

A data center or cloud platform routes each incoming task to one of NNN parallel servers. Sending each task to a server with the shortest queue, the join-the-shortest-queue (JSQ) policy, minimizes waiting in a strong sense, but the dispatcher must learn all NNN queue lengths for every arrival. The power-of-d schemes cut this communication: the dispatcher samples ddd servers and joins the shortest of them. Mukherjee, Borst, van Leeuwaarden and Whiting ask how large d=d(N)d=d(N)d=d(N) must be for the sampled scheme to perform as well as JSQ as N→∞N\to\inftyN→∞. They answer at two scales. At the fluid scale (mission 1 of this series) d(N)→∞d(N)\to\inftyd(N)→∞ suffices. At the diffusion scale of the Halfin–Whitt regime, the subject of this mission, d(N)/(Nlog⁡N)→∞d(N)/(\sqrt N\log N)\to\inftyd(N)/(N​logN)→∞ suffices, and d(N)/(Nlog⁡N)→0d(N)/(\sqrt N\log N)\to 0d(N)/(N​logN)→0 does not.

Timeline:

  • Halfin and Whitt (1981) introduced the heavy-traffic regime (N−λ(N))/N→β>0(N-\lambda(N))/\sqrt N\to\beta>0(N−λ(N))/N​→β>0 for many-server systems.
  • Eschenfeldt and Gamarnik (2018, arXiv:1502.00999) proved the diffusion limit of ordinary JSQ in this regime, the system (2.4) below.
  • Mukherjee, Borst, van Leeuwaarden and Whiting (Stochastic Systems 8(4), 2018; arXiv:1612.00723v2) proved that JSQ(d(N)) has the same diffusion limit when d(N)/(Nlog⁡N)→∞d(N)/(\sqrt N\log N)\to\inftyd(N)/(N​logN)→∞ (Theorem 2.4), and that the limit differs when d(N)/(Nlog⁡N)→0d(N)/(\sqrt N\log N)\to 0d(N)/(N​logN)→0 (Theorem 2.5).

Setting

There are NNN servers, each with a buffer of size b∈{2,3,… }∪{∞}b\in\{2,3,\dots\}\cup\{\infty\}b∈{2,3,…}∪{∞} that counts the task in service. Tasks arrive as a Poisson process of rate λ(N)\lambda(N)λ(N), and each needs a unit-mean exponential amount of work, served at unit rate. A task sent to a full server is discarded. Under JSQ(d) the dispatcher samples ddd servers without replacement (1≤d≤N1\le d\le N1≤d≤N) and sends the task to a server with the shortest queue among them. For d=Nd=Nd=N this is JSQ.

The state is the occupancy vector Q=(Q1,…,Qb)Q=(Q_1,\dots,Q_b)Q=(Q1​,…,Qb​), where QiQ_iQi​ is the number of servers with at least iii tasks. The conventions are Q0=NQ_0=NQ0​=N and Qi=0Q_i=0Qi​=0 for i>bi>bi>b. With the servers ordered by nondecreasing queue length, the ccc-th ordered server has queue length I(c)=max⁡{i≥0:Qi≥N−c+1}I(c)=\max\{i\ge0:Q_i\ge N-c+1\}I(c)=max{i≥0:Qi​≥N−c+1}. An arrival at position ccc raises QI(c)+1Q_{I(c)+1}QI(c)+1​ by one when I(c)<bI(c)<bI(c)<b. A departure at position kkk lowers QI(k)Q_{I(k)}QI(k)​ by one when I(k)≥1I(k)\ge1I(k)≥1. JSQ(d) routes to the smallest position in a uniform ddd-subset of {1,…,N}\{1,\dots,N\}{1,…,N}.

In the Halfin–Whitt regime (N−λ(N))/N→β>0(N-\lambda(N))/\sqrt N\to\beta>0(N−λ(N))/N​→β>0, the diffusion-scaled state is

Qˉ1=−N−Q1N,Qˉi=QiN(i≥2).\bar Q_1=-\frac{N-Q_1}{\sqrt N},\qquad \bar Q_i=\frac{Q_i}{\sqrt N}\quad(i\ge2).Qˉ​1​=−N​N−Q1​​,Qˉ​i​=N​Qi​​(i≥2).

Here −Qˉ1-\bar Q_1−Qˉ​1​ is the scaled number of idle servers.

Formalization targets

Goal: Theorem 2.4 (pp. 7–8)

Let β>0\beta>0β>0, b≥2b\ge2b≥2 and k≥2k\ge2k≥2. Assume Qˉk+1(0)=0\bar Q_{k+1}(0)=0Qˉ​k+1​(0)=0 for large NNN and (Qˉ1(0),…,Qˉk(0))⇒ν(\bar Q_1(0),\dots,\bar Q_k(0))\Rightarrow\nu(Qˉ​1​(0),…,Qˉ​k​(0))⇒ν. If d(N)/(Nlog⁡N)→∞d(N)/(\sqrt N\log N)\to\inftyd(N)/(N​logN)→∞, then Qˉd(N)⇒Qˉ\bar Q^{d(N)}\Rightarrow\bar QQˉ​d(N)⇒Qˉ​ in Dℓ1[0,∞)D_{\ell_1}[0,\infty)Dℓ1​​[0,∞), where Qˉi≡0\bar Q_i\equiv 0Qˉ​i​≡0 for i≥k+1i\ge k+1i≥k+1 and

Qˉ1(t)=Qˉ1(0)+2W(t)−βt+∫0t(−Qˉ1+Qˉ2) ds−U1(t),Qˉ2(t)=Qˉ2(0)+U1(t)−∫0t(Qˉ2−Qˉ3) ds,Qˉi(t)=Qˉi(0)−∫0t(Qˉi−Qˉi+1) ds,3≤i≤k.\begin{aligned}\bar Q_1(t)&=\bar Q_1(0)+\sqrt2W(t)-\beta t+\int_0^t(-\bar Q_1+\bar Q_2)\,ds-U_1(t),\\ \bar Q_2(t)&=\bar Q_2(0)+U_1(t)-\int_0^t(\bar Q_2-\bar Q_3)\,ds,\\ \bar Q_i(t)&=\bar Q_i(0)-\int_0^t(\bar Q_i-\bar Q_{i+1})\,ds,\quad 3\le i\le k.\end{aligned}Qˉ​1​(t)Qˉ​2​(t)Qˉ​i​(t)​=Qˉ​1​(0)+2​W(t)−βt+∫0t​(−Qˉ​1​+Qˉ​2​)ds−U1​(t),=Qˉ​2​(0)+U1​(t)−∫0t​(Qˉ​2​−Qˉ​3​)ds,=Qˉ​i​(0)−∫0t​(Qˉ​i​−Qˉ​i+1​)ds,3≤i≤k.​

Here WWW is a standard Brownian motion and U1U_1U1​ is the nonnegative nondecreasing regulator with ∫0∞1[Qˉ1<0] dU1=0\int_0^\infty\mathbb 1_{[\bar Q_1<0]}\,dU_1=0∫0∞​1[Qˉ​1​<0]​dU1​=0. The goal also asserts that this solution is unique.

Milestones

  1. Uniqueness clause of Theorem 2.4: (2.4), with the constraint Qˉ1≤0\bar Q_1\le0Qˉ​1​≤0, has at most one solution for every driving path.
  2. Theorem 2.4 for d(N)=Nd(N)=Nd(N)=N: the JSQ diffusion limit of [8, Theorem 2], stated along an arbitrary sequence of server counts Nj→∞N_j\to\inftyNj​→∞.
  3. Proposition 5.1 (p. 32): if JSQ has the diffusion limit along every such sequence, then so does JSQ(d(N)) with d(N)/(Nlog⁡N)→∞d(N)/(\sqrt N\log N)\to\inftyd(N)/(N​logN)→∞.

The goal follows from 1–3.

Significance

The result. Theorem 2.4 shows that O(Nlog⁡N)O(\sqrt N\log N)O(N​logN) queue-length queries per arrival already achieve diffusion-level optimality: over any finite horizon only OP(N)O_P(\sqrt N)OP​(N​) servers are idle and only OP(N)O_P(\sqrt N)OP​(N​) hold two or more tasks, exactly as under JSQ. With Theorem 2.5 it identifies Nlog⁡N\sqrt N\log NN​logN as the minimal order of d(N)d(N)d(N). The diffusion limit is also the input for the steady-state analyses of JSQ in this regime (Braverman; Banerjee and Mukherjee).

Formalizing it. The result is proved on paper; nothing in it is machine-checked. A formalization needs:

  • a rigorous construction of the many-server JSQ(d) process;
  • the stochastic-coupling comparison of §3;
  • weak convergence of ℓ1\ell_1ℓ1​-valued processes;
  • a reflected integral equation of Skorokhod type.

The Eschenfeldt–Gamarnik limit (milestone 2) is cited by the paper, not proved in it, so its formalization is a separate, substantial contribution.

Difficulty

JSQ(d(N)) cannot be compared with JSQ path by path. The same arrival is routed to different servers under the two schemes, so their occupancy vectors drift apart and do not stay ordered. The obvious generator-based diffusion argument for JSQ(d(N)) itself does not close either. The routing probabilities depend on the whole ordered queue-length profile through (⋅d)/(Nd)\binom{\cdot}{d}/\binom{N}{d}(d⋅​)/(dN​)-type terms, and these do not reduce to a function of Qˉ1,Qˉ2\bar Q_1,\bar Q_2Qˉ​1​,Qˉ​2​ at scale N\sqrt NN​ unless d(N)d(N)d(N) is large enough.

The threshold is sharp: for d(N)≪Nlog⁡Nd(N)\ll\sqrt N\log Nd(N)≪N​logN the limit is different (Theorem 2.5). Any argument therefore has to use the growth rate quantitatively, controlling at scale oP(N)o_P(\sqrt N)oP​(N​) over a finite horizon how often JSQ(d(N)) fails to make a near-shortest choice. A qualitative fluid-type argument cannot do this. A second difficulty is the limit itself: (2.4) is a reflected equation whose regulator enters two coordinates, so even its well-posedness needs a Skorokhod-type argument with feedback.

Formalization scope

Lean conventions:

  • Levels and positions use the paper's index base: Q i is QiQ_iQi​ for i≥1i\ge1i≥1, with Q 0 = N, and positions are 1,…,N1,\dots,N1,…,N. The initial vector Fin k → ℝ lists levels 1,…,k1,\dots,k1,…,k.
  • Buffer b : ℕ∞ with 2 ≤ b; ⊤ is an infinite buffer.
  • Process construction is by uniformization. A Poisson clock of rate λ+N\lambda+Nλ+N is built from i.i.d. expMeasure (λ + N) interarrival times through the published renewalCount/partialSum. Its i.i.d. marks are arrivals, with probability λ/(λ+N)\lambda/(\lambda+N)λ/(λ+N) and a uniform ddd-subset, or departures at a uniform position. Marks are independent of the clock and of a random initial state. The model is therefore the paper's in law. Sampling is without replacement, so JSQ(N) is JSQ.
  • Path space. DS[0,∞)D_{\mathbb S}[0,\infty)DS​[0,∞) is read as ℓ1\ell_1ℓ1​-valued càdlàg paths, as the proof controls ℓ1\ell_1ℓ1​ distances. Weak convergence to the continuous limit is stated in coupling form: a common probability space with copies of the laws and almost sure uniform convergence in ℓ1\ell_1ℓ1​ on compacts.
  • Initial convergence is joint convergence in law of (Qˉ1(0),…,Qˉk(0))(\bar Q_1(0),\dots,\bar Q_k(0))(Qˉ​1​(0),…,Qˉ​k​(0)), via bounded continuous test functions. The page's coordinatewise convergence does not determine the limit. The limit's initial value is independent of WWW.
  • Solution concept of (2.4) includes Qˉ1≤0\bar Q_1\le0Qˉ​1​≤0, implicit in the paper; without it U1≡0U_1\equiv0U1​≡0 satisfies the complementarity condition and uniqueness fails. ∫1[Qˉ1<0]dU1=0\int\mathbb 1_{[\bar Q_1<0]}dU_1=0∫1[Qˉ​1​<0]​dU1​=0 is a Lebesgue–Stieltjes measure condition. WWW is Mathlib's IsBrownianReal.
  • "Same diffusion limit" in Proposition 5.1 is the paper's reduction: the JSQ result, for all server-count sequences, implies the JSQ(d(N)) result.

Trivializing formalizations are ruled out. The goal asserts the existence of the limit (Brownian motion, initial value and solution), not merely that every limit point solves (2.4). The system hypotheses are satisfiable: the mark law has total mass one, which is checked in Lean. The convergence is of the whole ℓ1\ell_1ℓ1​ path, including the tail above level kkk.

Infrastructure needed and welcome:

  • a library for ℓ1\ell_1ℓ1​-valued càdlàg processes;
  • the one-sided Skorokhod map with Lipschitz feedback, reusable for many reflected limits;
  • the S-coupling of §3, posed as milestones in mission 1 of this series;
  • a formal proof of the Eschenfeldt–Gamarnik JSQ limit.

Selected references

  • D. Mukherjee, S. C. Borst, J. S. H. van Leeuwaarden, P. A. Whiting, Universality of power-of-d load balancing in many-server systems, Stochastic Systems 8(4), 2018. arXiv:1612.00723v2, doi:10.1287/stsy.2018.0016
  • P. Eschenfeldt, D. Gamarnik, Join the shortest queue with many servers. The heavy-traffic asymptotics, Mathematics of Operations Research 43(3), 2018. arXiv:1502.00999
  • S. Halfin, W. Whitt, Heavy-traffic limits for queues with many exponential servers, Operations Research 29(3), 1981. doi:10.1287/opre.29.3.567
  • A. Braverman, Steady-state analysis of the join-the-shortest-queue model in the Halfin–Whitt regime, Mathematics of Operations Research 45(3), 2020. arXiv:1801.05121
10 thms1 active userReviewed
Bandit AlgorithmsOperations ResearchStochastic Systems·Captain: mikedeng1

Learning Unknown Service Rates in Queues 2: Q-UCB Has Queue-Regret at Most 6K v_u(t) log² t / t + (24.004 + UK)/(6t²) in the Late StageResearch Paper

Motivation

Many service systems must match jobs to servers whose quality is not known in advance: crowdsourcing platforms routing tasks to worker types, wireless schedulers assigning users to channels, and switches in data networks. The scheduler learns server quality only by trying servers, and every poor assignment lets work pile up. The natural performance measure is therefore not the cumulative reward loss of classical bandits but the queue-regret: the expected excess backlog compared with a genie that knows the best assignment.

Krishnasamy, Sen, Johari and Shakkottai, Learning Unknown Service Rates in Queues: A Multiarmed Bandit Approach (Operations Research, 2021; arXiv:1604.06377), show that queue-regret has two regimes. In an early stage it grows like the regret of a bandit. In a late stage, once the queue regularly empties, past mistakes are forgotten at every empty instant and the queue-regret decays. This mission formalizes their upper bound for the late stage: a UCB scheduler with forced exploration, Q-UCB, has queue-regret of order log⁡3t/t\log^3 t/tlog3t/t.

Setting

A discrete-time switch has UUU queues and K≥UK\ge UK≥U servers. In every slot t≥1t\ge1t≥1, queue uuu receives an arrival Au(t)∼Bernoulli(λu)A_u(t)\sim\mathrm{Bernoulli}(\lambda_u)Au​(t)∼Bernoulli(λu​) and link (u,k)(u,k)(u,k) offers a potential service Ruk(t)∼Bernoulli(μuk)R_{uk}(t)\sim\mathrm{Bernoulli}(\mu_{uk})Ruk​(t)∼Bernoulli(μuk​), all independent and i.i.d. over slots. The scheduler picks a matching κ(t)\kappa(t)κ(t), an injective assignment of a server κu(t)\kappa_u(t)κu​(t) to every queue, and queue uuu evolves as

Qu(t)=(Qu(t−1)+Au(t)−Su(t))+,Su(t)=Ruκu(t)(t).Q_u(t)=\big(Q_u(t-1)+A_u(t)-S_u(t)\big)^+,\qquad S_u(t)=R_{u\kappa_u(t)}(t).Qu​(t)=(Qu​(t−1)+Au​(t)−Su​(t))+,Su​(t)=Ruκu​(t)​(t).

The scheduler observes only past schedules and the services offered on scheduled links.

Write μu∗=max⁡kμuk\mu^*_u=\max_k\mu_{uk}μu∗​=maxk​μuk​, ku∗k^*_uku∗​ for its maximizer, ϵu=μu∗−λu\epsilon_u=\mu^*_u-\lambda_uϵu​=μu∗​−λu​, and Δ=min⁡u, k≠ku∗(μu∗−μuk)\Delta=\min_{u,\,k\ne k^*_u}(\mu^*_u-\mu_{uk})Δ=minu,k=ku∗​​(μu∗​−μuk​). Assumption 1: each queue has a unique best server and the best servers are distinct, so k∗k^*k∗ is the unique optimal matching. Assumption 2: ϵu>0\epsilon_u>0ϵu​>0 for all uuu. The genie queue Qu∗Q^*_uQu∗​ always uses ku∗k^*_uku∗​. Assumption 3: the system starts from the stationary law of Q∗Q^*Q∗. The queue-regret is

Ψu(t)=E[Qu(t)−Qu∗(t)].\Psi_u(t)=\mathbb E\big[Q_u(t)-Q^*_u(t)\big].Ψu​(t)=E[Qu​(t)−Qu∗​(t)].

Q-UCB (Algorithm 1) explores in slot ttt with probability min⁡{1,3Klog⁡2t/t}\min\{1,3K\log^2t/t\}min{1,3Klog2t/t}, scheduling a uniformly random member of a fixed family X\mathcal XX of KKK matchings that together cover all UKUKUK links. Otherwise it exploits: for each queue it picks the server with the largest index μ^uk(t)+log⁡2t/(2Nuk(t))\hat\mu_{uk}(t)+\sqrt{\log^2t/(2N_{uk}(t))}μ^​uk​(t)+log2t/(2Nuk​(t))​, with μ^uk\hat\mu_{uk}μ^​uk​ the empirical mean and NukN_{uk}Nuk​ the number of past assignments of link (u,k)(u,k)(u,k), and schedules a matching closest in Hamming distance to these choices. Fix β>1\beta>1β>1 and set

w(t)=t1−1/β,vu′(t)=6Kϵuw(t),vu(t)=24ϵu2log⁡t+60Kϵuvu′(t)log⁡2tt.w(t)=t^{1-1/\beta},\qquad v'_u(t)=\frac{6K}{\epsilon_u}w(t),\qquad v_u(t)=\frac{24}{\epsilon_u^2}\log t+\frac{60K}{\epsilon_u}\frac{v'_u(t)\log^2t}{t}.w(t)=t1−1/β,vu′​(t)=ϵu​6K​w(t),vu​(t)=ϵu2​24​logt+ϵu​60K​tvu′​(t)log2t​.

Formalization targets

Goal: Theorem 5 for Q-UCB (p. 16)

For every t≥5800t\ge5800t≥5800 with t≥exp⁡(4/(Δ2(1−1/β)3))t\ge\exp\big(4/(\Delta^2(1-1/\beta)^3)\big)t≥exp(4/(Δ2(1−1/β)3)), w(t)/log⁡t≥2/ϵuw(t)/\log t\ge2/\epsilon_uw(t)/logt≥2/ϵu​ and vu(t)+vu′(t)≤t/2v_u(t)+v'_u(t)\le t/2vu​(t)+vu′​(t)≤t/2,

Ψu(t)≤6Kvu(t)log⁡2tt+24.004+UK6t2.\Psi_u(t)\le 6K\frac{v_u(t)\log^2t}{t}+\frac{24.004+UK}{6t^2}.Ψu​(t)≤6Ktvu​(t)log2t​+6t224.004+UK​.

The constants are the paper's, stated exactly.

Milestones (§8.2, pp. 24–34)

  • Lemma 10: the number of forced-exploration slots in a window exceeds 5max⁡(log⁡t,K(log⁡3t2−log⁡3t1))5\max(\log t,K(\log^3t_2-\log^3t_1))5max(logt,K(log3t2​−log3t1​)) with probability at most t−4t^{-4}t−4. Over [1,t][1,t][1,t] it exceeds Kw(t)Kw(t)Kw(t) with probability at most t−2Kt^{-2K}t−2K.
  • Lemma 11: with probability at least 1−UK/(6t3)1-UK/(6t^3)1−UK/(6t3), no exploit slot in (w(t),t](w(t),t](w(t),t] schedules a sub-optimal link (event E1\mathcal E_1E1​).
  • Lemma 12: pathwise, Qu(t)−Qu∗(t)Q_u(t)-Q^*_u(t)Qu​(t)−Qu∗​(t) is at most the number of sub-optimal schedules since queue uuu was last empty, at time t−Bu(t)t-B_u(t)t−Bu​(t).
  • Lemmas 13–17: on E1∩E2\mathcal E_1\cap\mathcal E_2E1​∩E2​, a coarse queue bound 2Kw(t)2Kw(t)2Kw(t) (13), a first bound on the regenerative cycle (14), a maximal inequality for the arrival-minus-service walk (15), a refined polylogarithmic queue bound (16), and the refined cycle bound Bu(t)≤vu(t)B_u(t)\le v_u(t)Bu​(t)≤vu​(t) (17), each failing with probability O(t−3)O(t^{-3})O(t−3).

Significance

The theorem shows that a simple learning scheduler loses only O(Klog⁡3t/(ϵu2t))O(K\log^3t/(\epsilon_u^2t))O(Klog3t/(ϵu2​t)) in expected backlog in the late stage. Together with the paper's Ω(1/t)\Omega(1/t)Ω(1/t) lower bound for consistent policies (Theorem 3), this places the optimal late-stage queue-regret within a polylogarithmic factor. The analysis method has two parts: a high-probability statement that the learner has locked onto the optimal matching, and a regenerative argument that converts it into a bound on queue length. It applies to other learning-in-queues problems in the same way.

All results are proved in the paper (§8.2). None of them has a machine-checked proof. The mission produces a formal model of the learning switch, including the algorithm with arbitrary tie-breaking, the coupled genie and the stationary start, and checks the explicit constants. These constants rely on numerical facts stated in the proofs, for example a positivity check for x≥log⁡5800x\ge\log5800x≥log5800 in Lemma 10(b) and the absorption of lower-order terms into 24.00424.00424.004, which a formal proof must verify.

Difficulty

The obvious argument says that UCB eventually identifies the best server, so the queue behaves like the genie's. This fails for two reasons. First, the regret is an expectation of a queue length, and a queue remembers every early mistake until it next empties. So one must bound the current regenerative cycle, and that bound itself depends on the queue length, which forces the recursion of Lemmas 13, 14, 16 and 17. Second, UCB's standard logarithmic-regret analysis bounds the expected number of sub-optimal pulls. It says nothing about whether a sub-optimal pull happens in a specific late window. Forced exploration at rate log⁡2t/t\log^2t/tlog2t/t is what makes all indices concentrate at once, and Lemma 11 requires a union bound over every link, slot and sample count with explicit exponents.

Formalization scope

  • Probability space. One canonical space: the initial queue vector together with an i.i.d. sequence of slot seeds, namely uniforms for arrivals, links and the explore coin, and a uniform explore index.
    • Arrivals and services are threshold events of the uniforms.
    • Q(0)Q(0)Q(0) has the product geometric law P(Qu(0)=n)=(1−ρu)ρun\mathbb P(Q_u(0)=n)=(1-\rho_u)\rho_u^nP(Qu​(0)=n)=(1−ρu​)ρun​ with ρu=λu(1−μu∗)/((1−λu)μu∗)\rho_u=\lambda_u(1-\mu^*_u)/((1-\lambda_u)\mu^*_u)ρu​=λu​(1−μu∗​)/((1−λu​)μu∗​), the stationary law of the genie chain under the printed recursion.
  • Indexing and coupling. Queues and servers are Fin U and Fin K, and slots are t≥1t\ge1t≥1. The genie shares arrivals, services and Q(0)Q(0)Q(0) with the system. This coupling is implicit in the paper's proofs and does not change Ψu\Psi_uΨu​.
  • Algorithm. Q-UCB is formalized for every covering family X\mathcal XX and every tie-breaking rule, deterministic in the slot and history, for the UCB argmax and the Hamming projection.
    • Nuk(t)N_{uk}(t)Nuk​(t) counts slots 1,…,t−11,\dots,t-11,…,t−1.
    • An unsampled link has index +∞+\infty+∞.
    • Bu(t)=tB_u(t)=tBu​(t)=t when queue uuu never emptied on [0,t][0,t][0,t].
  • Disclosed additions.
    • K≥2K\ge2K≥2 in every statement; K=1K=1K=1 is trivial, and the constants fail there.
    • t,t1≥1t,t_1\ge1t,t1​≥1 in Lemma 10(a).
    • log⁡w(t)≥(2log⁡t)2/3\log w(t)\ge(2\log t)^{2/3}logw(t)≥(2logt)2/3 in Lemma 10(b); the printed statement is false without it, and its proof uses it.
    • Lemmas 14 and 16 take the "arbitrary function" vuv_uvu​ natural-valued.
  • Excluded trivializations. Theorem 5 also asserts that Qu(t)−Qu∗(t)Q_u(t)-Q^*_u(t)Qu​(t)−Qu∗​(t) is integrable, so Ψu(t)\Psi_u(t)Ψu​(t) cannot be a default value of a non-integrable integral. Every probability bound concerns the actual events, and every hypothesis set is satisfiable. For example, with U=1U=1U=1, K=2K=2K=2, λ=0.5\lambda=0.5λ=0.5, μ=(0.9,0.1)\mu=(0.9,0.1)μ=(0.9,0.1), β=2\beta=2β=2, the range of Theorem 5 contains t=⌈e50⌉t=\lceil e^{50}\rceilt=⌈e50⌉.
  • Out of scope. Theorem 5 and Lemma 11 for Q-ThS (Algorithm 2) are not formalized, and neither is Corollary 6.
  • Infrastructure needed and welcome. Chernoff bounds for sums of independent non-identically distributed Bernoulli variables (Lemma 10), Hoeffding's inequality and a maximal suffix-sum bound (Lemmas 14, 15), measurability of the algorithm's trajectory in the seeds, and regenerative-cycle arguments for the reflected random walk. The Chernoff and stationary-queue lemmas are reusable well beyond this mission.

The source is the arXiv v4 preprint; the paper's proofs are in its §8 and §9.

Selected references

  • S. Krishnasamy, R. Sen, R. Johari, S. Shakkottai, Learning Unknown Service Rates in Queues: A Multiarmed Bandit Approach, Operations Research 69(1), 2021. Preprint arXiv:1604.06377v4. https://arxiv.org/abs/1604.06377 ; https://doi.org/10.1287/opre.2020.1995
  • P. Auer, N. Cesa-Bianchi, P. Fischer, Finite-time Analysis of the Multiarmed Bandit Problem, Machine Learning 47, 2002. https://doi.org/10.1023/A:1013689704352
  • M. J. Neely, Stochastic Network Optimization with Application to Communication and Queueing Systems, Morgan & Claypool, 2010. https://doi.org/10.2200/S00271ED1V01Y201006CNT007
12 thms1 active userReviewed
Operations ResearchStochastic Systems·Captain: mikedeng1

Universality of Power-of-d Load Balancing in Many-Server Systems 1: JSQ(d(N)) with d(N) → ∞ Has the Same Fluid Limit as the Join-the-Shortest-Queue PolicyResearch Paper

Motivation

Large data centers and cloud platforms route each incoming job to one of NNN parallel servers through a single dispatcher. The join-the-shortest-queue (JSQ) policy, which sends each job to a server with the fewest jobs, has strong optimality properties for identical exponential servers, but it needs the queue length of every server at every arrival, which is costly when NNN is in the thousands. The power-of-ddd scheme JSQ(ddd) polls only ddd servers chosen at random and joins the shortest among them. For fixed d≥2d\ge2d≥2 it improves dramatically over random routing, which is the supermarket model of Mitzenmacher and of Vvedenskaya, Dobrushin and Karpelevich, but it does not match JSQ.

Mukherjee, Borst, van Leeuwaarden and Whiting (arXiv:1612.00723, Stochastic Systems 2018) ask how fast d=d(N)d=d(N)d=d(N) must grow for JSQ(d(N)d(N)d(N)) to behave like JSQ in the large-NNN limit. This mission concerns their fluid-scale answer: any growth d(N)→∞d(N)\to\inftyd(N)→∞ suffices.

Timeline.

  • 1996–2001: Vvedenskaya–Dobrushin–Karpelevich and Mitzenmacher derive the fluid limit of JSQ(ddd) for fixed ddd (the supermarket ODE).
  • 2015–2018: Eschenfeldt and Gamarnik obtain the Halfin–Whitt diffusion limit of ordinary JSQ (arXiv:1502.00999).
  • 2016–2018: Mukherjee et al. prove the first transient fluid limit of ordinary JSQ in ℓ1\ell_1ℓ1​ (their Theorem 4.1). They show that JSQ(d(N)d(N)d(N)) has the same fluid limit whenever d(N)→∞d(N)\to\inftyd(N)→∞ (Theorem 2.1) and the same diffusion limit whenever d(N)/(Nlog⁡N)→∞d(N)/(\sqrt N\log N)\to\inftyd(N)/(N​logN)→∞ (Theorem 2.4, a separate mission).

Setting

There are NNN identical servers, each with a buffer of size b∈{1,2,… }∪{∞}b\in\{1,2,\dots\}\cup\{\infty\}b∈{1,2,…}∪{∞} counting the job in service. Jobs arrive as a Poisson process of rate λ(N)\lambda(N)λ(N) and need independent unit-mean exponential service. A job sent to a full server is discarded. The occupancy vector is Q(t)=(Q1(t),Q2(t),… )\mathbf Q(t)=(Q_1(t),Q_2(t),\dots)Q(t)=(Q1​(t),Q2​(t),…), where Qi(t)Q_i(t)Qi​(t) is the number of servers with at least iii jobs, with the convention Q0≡NQ_0\equiv NQ0​≡N. If the servers are listed in nondecreasing order of queue length, the ccc-th ordered server holds I(c)=max⁡{i≥0:Qi≥N−c+1}I(c)=\max\{i\ge0:Q_i\ge N-c+1\}I(c)=max{i≥0:Qi​≥N−c+1} jobs. A dispatch scheme picks an ordered position c∈{1,…,N}c\in\{1,\dots,N\}c∈{1,…,N} for each arrival, and the arrival raises QI(c)+1Q_{I(c)+1}QI(c)+1​ by one (or is discarded if I(c)=bI(c)=bI(c)=b).

The schemes are:

  • JSQ picks c=1c=1c=1;
  • JSQ(ddd) picks the lowest position among ddd positions sampled uniformly without replacement, so JSQ(NNN) is JSQ;
  • MJSQ(nnn) picks c=n+1c=n+1c=n+1;
  • CJSQ(nnn) is the class of all rules that pick some c≤n+1c\le n+1c≤n+1;
  • JSQ(n,dn,dn,d) keeps the JSQ(ddd) choice when it is at most n+1n+1n+1 and otherwise picks uniformly among the first n+1n+1n+1 positions.

Two systems are S-coupled when they share the arrival clock and the departure clock of the kkk-th ordered server for every kkk.

The fluid-scaled state is q(t)=Q(t)/N\mathbf q(t)=\mathbf Q(t)/Nq(t)=Q(t)/N. It lives in

S={q∈[0,1]b: qi≤qi−1 (i≥2), ∑i≥1qi<∞},\mathcal S=\Big\{\mathbf q\in[0,1]^b:\ q_i\le q_{i-1}\ (i\ge2),\ \sum_{i\ge1}q_i<\infty\Big\},S={q∈[0,1]b: qi​≤qi−1​ (i≥2), i≥1∑​qi​<∞},

which carries the ℓ1\ell_1ℓ1​ distance. For q∈S\mathbf q\in\mathcal Sq∈S let m(q)=min⁡{i:qi+1<1}m(\mathbf q)=\min\{i:q_{i+1}<1\}m(q)=min{i:qi+1​<1}. The coefficients pi(q)p_i(\mathbf q)pi​(q) of (4.4) send all incoming work to level m(q)−1m(\mathbf q)-1m(q)−1 while the rate λ\lambdaλ can be absorbed there, and the excess to level m(q)m(\mathbf q)m(q): if m(q)=0m(\mathbf q)=0m(q)=0 then p0=1p_0=1p0​=1; otherwise pm−1=min⁡{(1−qm+1)/λ,1}p_{m-1}=\min\{(1-q_{m+1})/\lambda,1\}pm−1​=min{(1−qm+1​)/λ,1}, pm=1−pm−1p_m=1-p_{m-1}pm​=1−pm−1​, and all other pip_ipi​ vanish.

Formalization targets

Goal: Theorem 2.1

Assume λ(N)/N→λ<1\lambda(N)/N\to\lambda<1λ(N)/N→λ<1, d(N)→∞d(N)\to\inftyd(N)→∞ and qd(N)(0)→q∞\mathbf q^{d(N)}(0)\to\mathbf q^\inftyqd(N)(0)→q∞ in S\mathcal SS. Then every subsequence of {qd(N)(t)}t≥0\{\mathbf q^{d(N)}(t)\}_{t\ge0}{qd(N)(t)}t≥0​ has a further subsequence converging weakly (Skorohod J1J_1J1​) to a limit satisfying

qi(t)=qi∞+λ∫0tpi−1(q(s)) ds−∫0t(qi(s)−qi+1(s)) ds,i=1,…,b.(2.1)q_i(t)=q^\infty_i+\lambda\int_0^t p_{i-1}(\mathbf q(s))\,ds-\int_0^t\big(q_i(s)-q_{i+1}(s)\big)\,ds,\qquad i=1,\dots,b. \tag{2.1}qi​(t)=qi∞​+λ∫0t​pi−1​(q(s))ds−∫0t​(qi​(s)−qi+1​(s))ds,i=1,…,b.(2.1)

The statement fixes no rate for d(N)d(N)d(N), which is what makes it the right goal.

Milestones

The milestones follow the paper's proof.

  • §3, coupling. Proposition 3.1 is a deterministic stack ordering. Proposition 3.2 and Corollary 3.3 sandwich any CJSQ(nnn) scheme between JSQ and MJSQ(nnn). Proposition 3.5 bounds the ℓ1\ell_1ℓ1​ distance of two S-coupled systems by 2Δ(t)2\Delta(t)2Δ(t), and Proposition 3.6 bounds Δ\DeltaΔ for JSQ(ddd) against JSQ(n,dn,dn,d).
  • §4.1, JSQ fluid limit. Proposition 4.3 makes the martingales negligible, Lemmas 4.6 and 4.7 characterize compactness and tightness in ℓ1\ell_1ℓ1​, Lemma 4.8 gives level confinement, and Theorem 4.1 is the fluid limit of JSQ itself.
  • §4.2. Proposition 4.9 is the reduction "JSQ fluid limit ⇒\Rightarrow⇒ JSQ(d(N)d(N)d(N)) fluid limit".

Significance

The result. Theorem 2.1 shows that on the fluid scale the optimality of JSQ costs nothing beyond d(N)→∞d(N)\to\inftyd(N)→∞ queue inquiries per job, far fewer than NNN. In particular the fluid limit is the same as that of JSQ, which is fluid-optimal. The coupling framework of §3 (the class CJSQ(nnn), stack orderings, S-coupling) is not tied to JSQ(ddd). The paper uses the same tools for the diffusion-scale Theorem 2.4, for batch arrivals and for the interchange of limits, Proposition 2.2.

Formalizing it. Proofs of the paper's results exist; no machine-checked version does. The mission produces:

  • a reusable model of many-server systems with ordered-position dispatch;
  • a formal stack-ordering and S-coupling toolkit;
  • compactness and tightness criteria for ℓ1\ell_1ℓ1​-valued random sequences;
  • a complete fluid-limit theorem for a load-balancing policy.

The proof of Theorem 4.1 in the paper is sketched in places (the time-scale separation is argued "analogously" to Hunt and Kurtz), so a formal proof would also settle those steps.

Difficulty

The obvious argument would compare JSQ(d(N)d(N)d(N)) directly with JSQ: the two disagree whenever the d(N)d(N)d(N) sampled servers miss every shortest queue. When a constant fraction of servers is idle this is rare, but when few servers share the minimum queue length the disagreement rate does not vanish, and a direct coupling fails. The paper instead passes through CJSQ(n(N)n(N)n(N)) with n(N)/N→0n(N)/N\to0n(N)/N→0, which is close to JSQ(d(N)d(N)d(N)) for a suitable n(N)n(N)n(N) and sandwiched between JSQ and MJSQ(n(N)n(N)n(N)).

The second difficulty is the topology. The sandwich controls tail sums ∑i≥mQi\sum_{i\ge m}Q_i∑i≥m​Qi​, so the fluid limit of JSQ is needed in ℓ1\ell_1ℓ1​, not coordinatewise. With b=∞b=\inftyb=∞ the state space is infinite-dimensional, and tightness requires uniform control of tails. Finally, the JSQ fluid limit itself involves a separation of time scales: the drift pi(q)p_i(\mathbf q)pi​(q) is discontinuous in q\mathbf qq and arises as an averaged stationary behaviour of a fast process.

Formalization scope

  • Occupancy. Occupancy vectors are finitely supported functions ℕ →₀ ℕ with the paper's level index; coordinate 000 holds Q0=NQ_0=NQ0​=N. The buffer is b∈N∞b\in\mathbb N_\inftyb∈N∞​ with b≥1b\ge1b≥1; both finite and infinite bbb are covered.
  • Construction. Systems are built by uniformization: a Poisson clock of rate λ+N\lambda+Nλ+N with i.i.d. Uniform[0,1][0,1][0,1] marks. A mark decides arrival versus departure, the departure position, the random ddd-subset (sampling without replacement) and the uniform choice of JSQ(n,dn,dn,d). S-coupled systems share the clock and the marks.
  • Schemes. A scheme maps the event index, the current state and the current mark to a position. CJSQ(nnn) rules must be measurable in the mark.
  • Fluid states. Fluid states are functions ℕ → ℝ with q0=1q_0=1q0​=1. S\mathcal SS carries the ℓ1\ell_1ℓ1​ distance, computed in [0,∞][0,\infty][0,∞].
  • Coefficients. pm−1p_{m-1}pm−1​ is written as "111 if λ≤1−qm+1\lambda\le1-q_{m+1}λ≤1−qm+1​, else (1−qm+1)/λ(1-q_{m+1})/\lambda(1−qm+1​)/λ", which equals (4.4) for λ>0\lambda>0λ>0 and never divides by zero.
  • Limit equation. A solution of (2.1) must take values in S\mathcal SS, have integrable integrands, and satisfy every equation i=1,…,bi=1,\dots,bi=1,…,b.
  • Weak convergence. Weak J1J_1J1​ convergence to a continuous limit is stated in coupling form: copies of the processes on one probability space converge almost surely uniformly on compacts in ℓ1\ell_1ℓ1​. "Any subsequence has a further subsequence" asserts the existence of the limit.
  • Initial states. Theorem 2.1 has deterministic initial states with ℓ1\ell_1ℓ1​ convergence and allows λ=0\lambda=0λ=0. Theorem 4.1, Proposition 4.3 and Lemma 4.8 have random initial states converging in probability, and λ>0\lambda>0λ>0.
  • Stochastic order. ≤st\le_{st}≤st​ for processes means the existence of an ordered coupling with the right path laws. Proposition 3.6's conditional probability is stated for each value of the integer A(T)A(T)A(T).
  • Correction. Proposition 3.1 as printed quantifies over m≤bm\le bm≤b. For finite bbb the proof also needs the ordering of the discarded counts, m=b+1m=b+1m=b+1, and the formal statement orders every level m≥1m\ge1m≥1.

A formalization that states only "every limit point solves (2.1)", or that measures convergence coordinatewise, would be vacuous or weaker: it is ruled out by requiring the limit's existence and ℓ1\ell_1ℓ1​ convergence.

Needed infrastructure. The development needs Poisson clocks and thinning, martingale bounds for counting processes, compactness in ℓ1\ell^1ℓ1, tightness and Prokhorov's theorem on Polish spaces, and the Skorohod representation. The ℓ1\ell_1ℓ1​ compactness and tightness criteria (Lemmas 4.6, 4.7) and the stack ordering (Proposition 3.1) are reusable on their own. Proofs of any milestone are welcome, as are proofs of Theorem 2.1 that bypass Theorem 4.1.

Selected references

  • D. Mukherjee, S. C. Borst, J. S. H. van Leeuwaarden, P. A. Whiting, Universality of Power-of-d Load Balancing in Many-Server Systems, Stochastic Systems 8(4), 2018. arXiv:1612.00723v2, https://arxiv.org/abs/1612.00723
  • P. Eschenfeldt, D. Gamarnik, Join the Shortest Queue with Many Servers. The Heavy-Traffic Asymptotics, Mathematics of Operations Research 43(3), 2018. https://arxiv.org/abs/1502.00999
  • M. Mitzenmacher, The Power of Two Choices in Randomized Load Balancing, IEEE Transactions on Parallel and Distributed Systems 12(10), 2001. https://doi.org/10.1109/71.963420
  • N. D. Vvedenskaya, R. L. Dobrushin, F. I. Karpelevich, Queueing System with Selection of the Shortest of Two Queues: An Asymptotic Approach, Problems of Information Transmission 32(1), 1996.
  • P. J. Hunt, T. G. Kurtz, Large Loss Networks, Stochastic Processes and their Applications 53(2), 1994. https://doi.org/10.1016/0304-4149(94)90063-9
  • S. N. Ethier, T. G. Kurtz, Markov Processes: Characterization and Convergence, Wiley, 1986. https://doi.org/10.1002/9780470316658
16 thms1 active userReviewed
Algorithmic Game TheoryOperations Research·Captain: mikedeng1

Prophet Secretary Through Blind Strategies 1: A Nonincreasing Blind Strategy Earns at Least 0.669 of the Prophet's Expected Maximum When Values Arrive in Uniformly Random OrderResearch Paper

Motivation

In the prophet inequality a gambler observes independent nonnegative random values V1,…,VnV_1,\dots,V_nV1​,…,Vn​ with known laws one at a time and must decide irrevocably, after seeing each one, whether to stop and keep it. Her expected reward is compared with that of a prophet who knows all realizations and earns E(max⁡iVi)\mathbb E(\max_i V_i)E(maxi​Vi​). When the values arrive in a fixed adversarial order the best guarantee is 1/21/21/2 (Krengel, Sucheston and Garling, 1977; Samuel-Cahn, 1984). Prophet inequalities are the stopping-theoretic core of posted-price mechanisms in online auctions, where a guarantee for the gambler translates into a welfare or revenue approximation for sequential pricing.

The prophet secretary problem keeps arbitrary independent laws but lets the values arrive in a uniformly random order, as in the secretary problem.

  • 2015: Esfandiari, HajiAghayi, Liaghat and Monemizadeh introduce the problem and give a 1−1/e≈0.6321-1/e\approx0.6321−1/e≈0.632 guarantee with nnn time-dependent thresholds (ESA 2015).
  • 2017: Ehsani, Hajiaghayi, Kesselheim and Singla obtain 1−1/e1-1/e1−1/e with a single threshold, using random tie-breaking (SODA 2018).
  • 2018: Azar, Chiplunkar and Kaplan surpass 1−1/e1-1/e1−1/e by a small margin and give an upper bound of 11/1511/1511/15 for deterministic distribution-insensitive algorithms (EC 2018).
  • 2019: Correa, Saona and Ziliotto break 1−1/e1-1/e1−1/e, proving 0.6690.6690.669 with blind strategies, and show that no algorithm beats 3−1\sqrt3-13​−1 (arXiv:1807.07483; Mathematical Programming, doi:10.1007/s10107-020-01544-8). This mission formalizes the 0.6690.6690.669 guarantee.

Setting

An instance is a list of laws F1,…,FnF_1,\dots,F_nF1​,…,Fn​ of independent nonnegative random variables V1,…,VnV_1,\dots,V_nV1​,…,Vn​, all continuous. A uniformly random permutation σ\sigmaσ of [n]={1,…,n}[n]=\{1,\dots,n\}[n]={1,…,n}, independent of the values, is the arrival order: at time jjj the gambler sees VσjV_{\sigma_j}Vσj​​.

The time threshold algorithm TTAτ1,…,τn\mathrm{TTA}_{\tau_1,\dots,\tau_n}TTAτ1​,…,τn​​ stops at the first time jjj with Vσj>τjV_{\sigma_j}>\tau_jVσj​​>τj​. Its stopping time is T=inf⁡{j∈[n]:Vσj>τj}T=\inf\{j\in[n]:V_{\sigma_j}>\tau_j\}T=inf{j∈[n]:Vσj​​>τj​} and the gambler's reward is VσT1T<∞V_{\sigma_T}\mathbf 1_{T<\infty}VσT​​1T<∞​, written VσTV_{\sigma_T}VσT​​.

A blind strategy is a nonincreasing function α:[0,1]→[0,1]\alpha:[0,1]\to[0,1]α:[0,1]→[0,1]. Given an instance, draw u1,…,unu_1,\dots,u_nu1​,…,un​ independently and uniformly from [0,1][0,1][0,1], let u[j]u_{[j]}u[j]​ be the jjj-th smallest, and choose thresholds by

P(max⁡i∈[n]Vi≤τj)=α(u[j]).\mathbb P\big(\max_{i\in[n]}V_i\le\tau_j\big)=\alpha(u_{[j]}).P(i∈[n]max​Vi​≤τj​)=α(u[j]​).

The strategy is called blind because α\alphaα fixes the levels of the distribution of the maximum at which the gambler stops, independently of the instance and of nnn. The deterministic blind strategy (Definition 3.1) uses α(j/n)\alpha(j/n)α(j/n) in place of α(u[j])\alpha(u_{[j]})α(u[j]​). A piecewise-constant strategy αα1,…,αm(x)=∑j∈[m]αj1[j−1m,jm)(x)\alpha_{\alpha_1,\dots,\alpha_m}(x)=\sum_{j\in[m]}\alpha_j\mathbf 1_{[\frac{j-1}m,\frac jm)}(x)αα1​,…,αm​​(x)=∑j∈[m]​αj​1[mj−1​,mj​)​(x) uses at most mmm distinct levels.

Formalization targets

Goal: Theorem 1.1

There is a nonincreasing α:[0,1]→[0,1]\alpha:[0,1]\to[0,1]α:[0,1]→[0,1] such that, for every nnn and every continuous instance,

E(VσT)  ≥  0.669  E(max⁡i∈[n]Vi),\mathbb E(V_{\sigma_T})\;\ge\;0.669\;\mathbb E\big(\max_{i\in[n]}V_i\big),E(VσT​​)≥0.669E(i∈[n]max​Vi​),

where TTT is the stopping time of the blind strategy α\alphaα. One α\alphaα serves every instance.

Milestones, in the order the proof uses them

  1. §3, display after Definition 3.1: the blind strategy's value is the limit of the deterministic blind strategy's values on the instance padded with mmm zeros, as m→∞m\to\inftym→∞.
  2. Lemma 3.1: an exact formula for P(VσT>t)\mathbb P(V_{\sigma_T}>t)P(VσT​​>t) for nonincreasing thresholds.
  3. Lemma 3.3: 1n∑j∈[k](1−αj)≤P(T≤k)≤1−(∏j≤kαj)1/n\frac1n\sum_{j\in[k]}(1-\alpha_j)\le\mathbb P(T\le k)\le1-(\prod_{j\le k}\alpha_j)^{1/n}n1​∑j∈[k]​(1−αj​)≤P(T≤k)≤1−(∏j≤k​αj​)1/n.
  4. Lemma 4.1: P(T≥k∣σk=i)≥P(T>k)/(1−kn+1n∑l∈[k]P(Vi≤τl))\mathbb P(T\ge k\mid\sigma_k=i)\ge\mathbb P(T>k)\big/\big(1-\frac kn+\frac1n\sum_{l\in[k]}\mathbb P(V_i\le\tau_l)\big)P(T≥k∣σk​=i)≥P(T>k)/(1−nk​+n1​∑l∈[k]​P(Vi​≤τl​)).
  5. Lemma 4.2: a weighted replacement of the union bound ∑iP(Vi>t)≥P(max⁡iVi>t)\sum_i\mathbb P(V_i>t)\ge\mathbb P(\max_iV_i>t)∑i​P(Vi​>t)≥P(maxi​Vi​>t).
  6. Lemma 4.3: for α=αα1,…,αm\alpha=\alpha_{\alpha_1,\dots,\alpha_m}α=αα1​,…,αm​​ with αm>0\alpha_m>0αm​>0, E(VσT)≥min⁡j∈[m+1]fj(α1,…,αm) E(max⁡iVi)\mathbb E(V_{\sigma_T})\ge\min_{j\in[m+1]}f_j(\alpha_1,\dots,\alpha_m)\,\mathbb E(\max_iV_i)E(VσT​​)≥minj∈[m+1]​fj​(α1​,…,αm​)E(maxi​Vi​) for explicit functions fjf_jfj​.
  7. Corollary 4.4: some 1≥α1≥⋯≥αm≥01\ge\alpha_1\ge\dots\ge\alpha_m\ge01≥α1​≥⋯≥αm​≥0 gives the constant 0.669750.669750.66975; the paper reports that m=30m=30m=30 suffices.

Significance

Theorem 1.1 raised the guarantee for the prophet secretary problem from just above 1−1/e1-1/e1−1/e to 0.6690.6690.669, and it separates random-order arrival from adversarial arrival by more than the classical single-threshold analysis. The strategies are distribution-insensitive in a precise sense: the gambler needs only the distribution of the maximum, not the individual laws, and α\alphaα does not depend on nnn. The milestones are general facts about multi-threshold stopping in random order (Lemmas 3.1, 3.3, 4.1, 4.2) that apply to any nonincreasing thresholds, not only to blind ones.

The result is proved in the paper, with one numerical step: Corollary 4.4 evaluates the bound of Lemma 4.3 at a specific vector (α1,…,α30)(\alpha_1,\dots,\alpha_{30})(α1​,…,α30​) found by optimization, which the paper does not print. No part of this argument has a machine-checked proof. The formalization would make the reduction from blind to deterministic strategies, the stopping-time estimates and the final numerical certificate explicit and checked. The paper also shows that no blind strategy guarantees more than 0.6750.6750.675 (Theorem 1.2, outside this mission); any later improvement of the constant leaves this goal, which asserts only 0.6690.6690.669, intact.

Difficulty

The obvious argument fixes a threshold and bounds the probability that some value exceeds it; with one threshold this cannot beat 1−1/e1-1/e1−1/e, and with time-varying thresholds the events "the gambler has not stopped by time kkk" and "value iii arrives at time kkk" are dependent, because the arrival of iii at time kkk removes iii from the earlier times. The union bound ∑iP(Vi>t)≥P(max⁡iVi>t)\sum_i\mathbb P(V_i>t)\ge\mathbb P(\max_iV_i>t)∑i​P(Vi​>t)≥P(maxi​Vi​>t) loses exactly the constant one is trying to gain. The analysis must therefore bound the law of the stopping time uniformly over all instances (Lemma 3.3), sharpen the conditional survival probability (Lemma 4.1) and the union bound (Lemma 4.2), and then pass to the limit of large instances. The final constant depends on a numerical optimization over m=30m=30m=30 levels, so the closing step is a certified evaluation of an explicit but long expression involving mmm-th roots and logarithms.

Formalization scope

All statements import one definitions file, BlindProphetSec.Blind.Setting.

  • Instances: μ : Fin n → Measure ℝ with IsProbabilityMeasure, NullSingletonClass (continuity, the paper's standing assumption of §1.2 and §3) and μ i (Set.Iio 0) = 0 (nonnegativity). Independence is built in: the values are the coordinates of Measure.pi μ. Indices and times are 0-based; the paper's time jjj is j.val + 1.
  • Random order: the average over all n!n!n! permutations. A conditional probability P(⋅∣σk=i)\mathbb P(\cdot\mid\sigma_k=i)P(⋅∣σk​=i) is written n P(⋅, σk=i)n\,\mathbb P(\cdot,\ \sigma_k=i)nP(⋅, σk​=i).
  • Expectations are lower Lebesgue integrals in [0,∞][0,\infty][0,∞], so E(max⁡iVi)=∞\mathbb E(\max_iV_i)=\inftyE(maxi​Vi​)=∞ is allowed; the paper's ratios are stated multiplied out, c E(max⁡iVi)≤E(VσT)c\,\mathbb E(\max_iV_i)\le\mathbb E(V_{\sigma_T})cE(maxi​Vi​)≤E(VσT​​). The reward is 000 if the gambler never stops.
  • Thresholds of blind strategies are encoded through their levels: stop at time jjj iff α(u[j])<P(max⁡iVi≤Vσj)\alpha(u_{[j]})<\mathbb P(\max_iV_i\le V_{\sigma_j})α(u[j]​)<P(maxi​Vi​≤Vσj​​). For continuous laws this agrees almost surely with Vσj>τjV_{\sigma_j}>\tau_jVσj​​>τj​ when the level is positive; level 111 means "never stop", and at level 000, where the paper's threshold is not unique, the encoding takes the largest threshold.
  • Pinned hypotheses: E(max⁡iVi)<∞\mathbb E(\max_iV_i)<\inftyE(maxi​Vi​)<∞ in the §3 limit display; t≥0t\ge0t≥0 in Lemma 3.1 (the printed identity fails for t<0t<0t<0 because the reward is 000 on {T=∞}\{T=\infty\}{T=∞}); α1<1\alpha_1<1α1​<1 in Lemma 4.3, so that every term of fjf_jfj​ is the paper's and none is a 0/00/00/0 junk value.
  • Generalizations: Lemmas 3.1, 4.1 and 4.2 are stated without continuity, and 4.1, 4.2 without nonnegativity, as printed.

A trivializing formalization is ruled out: the strategy α\alphaα (and the vector α1,…,αm\alpha_1,\dots,\alpha_mα1​,…,αm​ of Corollary 4.4) is quantified before nnn and the instance, the constants 0.6690.6690.669 and 0.669750.669750.66975 are exact rationals, and expectations cannot collapse to 000 through non-integrability.

A complete development needs order statistics of uniform samples, exchangeability arguments for uniformly random permutations, convergence of the padded deterministic strategies, and a certified numerical evaluation of fjf_jfj​. The random-order stopping vocabulary is reusable for other prophet secretary results, including the 3−1\sqrt3-13​−1 upper bound of the companion mission. Contributions to any milestone, including partial results such as Lemma 3.2 (P(T≥k∣σk=i)≥P(T>k)\mathbb P(T\ge k\mid\sigma_k=i)\ge\mathbb P(T>k)P(T≥k∣σk​=i)≥P(T>k)), are welcome. The PDF used is the arXiv preprint, version 2 (arXiv:1807.07483v2, 12 March 2019); all labels and pages refer to it.

Selected references

  • J. Correa, R. Saona, B. Ziliotto, Prophet Secretary Through Blind Strategies, arXiv:1807.07483v2, 2019; Mathematical Programming, 2021. https://arxiv.org/abs/1807.07483, https://doi.org/10.1007/s10107-020-01544-8
  • H. Esfandiari, M. Hajiaghayi, V. Liaghat, M. Monemizadeh, Prophet Secretary, Proceedings of ESA 2015, pp. 496–508.
  • S. Ehsani, M. Hajiaghayi, T. Kesselheim, S. Singla, Prophet Secretary for Combinatorial Auctions and Matroids, Proceedings of SODA 2018, pp. 700–714.
  • Y. Azar, A. Chiplunkar, H. Kaplan, Prophet Secretary: Surpassing the 1 − 1/e Barrier, Proceedings of EC 2018, pp. 303–318.
  • E. Samuel-Cahn, Comparison of Threshold Stop Rules and Maximum for Independent Nonnegative Random Variables, The Annals of Probability 12(4), pp. 1213–1216.
9 thms1 active userReviewed
Bandit AlgorithmsMachine LearningReinforcement Learning·Captain: mikedeng1

(More) Efficient Reinforcement Learning via Posterior Sampling: PSRL Has Bayesian Regret O(τS√(AT log(SAT))) in Episodic Finite MDPsResearch Paper

Motivation

An agent that interacts repeatedly with an unknown Markov decision process must trade exploration against exploitation. The dominant design principle for provably efficient reinforcement learning has been optimism in the face of uncertainty: UCRL2 (Jaksch, Ortner & Auer, 2010) and REGAL (Bartlett & Tewari, 2009) build confidence sets around the unknown model and act according to the most favourable model in them. Computing that optimistic policy can be expensive, and confidence sets built separately for every state–action pair can be loose.

Posterior sampling for reinforcement learning (PSRL), proposed by Strens (2000), replaces optimism by randomization: at the start of each episode it samples one MDP from the posterior distribution and follows the optimal policy of that sample. It is the reinforcement-learning analogue of Thompson sampling for bandits. Osband, Russo & Van Roy (2013) gave the first regret guarantee for PSRL, and with it the first regret guarantee for an RL algorithm not based on optimism. Their argument transfers the Bayesian-regret analysis of Thompson sampling by Russo & Van Roy (2014) from bandits to MDPs.

Timeline:

  • 2000: Strens proposes PSRL as a heuristic.
  • 2010: Jaksch, Ortner and Auer prove O~(DSAT)\tilde O(DS\sqrt{AT})O~(DSAT​) regret for UCRL2 in non-episodic MDPs with diameter DDD.
  • 2013: Osband, Russo and Van Roy prove the O(τSATlog⁡(SAT))O(\tau S\sqrt{AT\log(SAT)})O(τSATlog(SAT)​) Bayesian regret bound for PSRL in episodic MDPs, the result of this mission.

Setting

A finite-horizon MDP M=(S,A,RM,PM,τ,ρ)M=(\mathcal S,\mathcal A,R^M,P^M,\tau,\rho)M=(S,A,RM,PM,τ,ρ) has SSS states, AAA actions and horizon τ\tauτ. Choosing action aaa in state sss yields a reward drawn from a distribution RaM(s)R^M_a(s)RaM​(s) on [0,1][0,1][0,1], with mean R‾aM(s)\overline R^M_a(s)RaM​(s), and moves the state to s′s's′ with probability PaM(s′∣s)P^M_a(s'\mid s)PaM​(s′∣s). Every episode starts from a state drawn from the initial distribution ρ\rhoρ. A policy μ\muμ maps a state and a step i∈{1,…,τ}i\in\{1,\dots,\tau\}i∈{1,…,τ} to an action, and its value is the expected total mean reward from step iii on,

Vμ,iM(s)=EM,μ[∑j=iτR‾ajM(sj) ∣ si=s].V^M_{\mu,i}(s)=\mathbb E_{M,\mu}\Big[\sum_{j=i}^{\tau}\overline R^M_{a_j}(s_j)\,\Big|\,s_i=s\Big].Vμ,iM​(s)=EM,μ​[j=i∑τ​Raj​M​(sj​)​si​=s].

A policy is optimal for MMM if it maximizes Vμ,iM(s)V^M_{\mu,i}(s)Vμ,iM​(s) for every sss and iii; μM\mu^MμM denotes a fixed choice of optimal policy for each MMM.

The true MDP M∗M^*M∗ is random with prior fff. In episode kkk, PSRL samples MkM_kMk​ from the posterior f(⋅∣Htk)f(\cdot\mid H_{t_k})f(⋅∣Htk​​) given the history HtkH_{t_k}Htk​​ of all earlier states, actions and rewards, and follows μk=μMk\mu_k=\mu^{M_k}μk​=μMk​ for τ\tauτ steps. With μ∗=μM∗\mu^*=\mu^{M^*}μ∗=μM∗, the regret of episode kkk and the cumulative regret are

Δk=∑sρ(s)(Vμ∗,1M∗(s)−Vμk,1M∗(s)),Regret(T,πτPS)=∑k=1⌈T/τ⌉Δk.\Delta_k=\sum_{s}\rho(s)\big(V^{M^*}_{\mu^*,1}(s)-V^{M^*}_{\mu_k,1}(s)\big),\qquad \mathrm{Regret}(T,\pi^{\mathrm{PS}}_\tau)=\sum_{k=1}^{\lceil T/\tau\rceil}\Delta_k .Δk​=s∑​ρ(s)(Vμ∗,1M∗​(s)−Vμk​,1M∗​(s)),Regret(T,πτPS​)=k=1∑⌈T/τ⌉​Δk​.

In Lean, MDPs are indexed by a parameter θ\thetaθ in a measurable space Θ\ThetaΘ (MDPFamily), the value is value F θ π j with 0-based steps, and the run is the structure PSRLRun.

Formalization targets

Goal: Theorem 1

There is an absolute constant C>0C>0C>0 such that, for all S,A,τ≥1S,A,\tau\ge1S,A,τ≥1, every prior fff, every measurable optimal-policy selector, every PSRL run with M∗∼fM^*\sim fM∗∼f and every TTT,

E[Regret(T,πτPS)]≤C τSATlog⁡(SAT).\mathbb E\big[\mathrm{Regret}(T,\pi^{\mathrm{PS}}_\tau)\big]\le C\,\tau S\sqrt{AT\log(SAT)} .E[Regret(T,πτPS​)]≤CτSATlog(SAT)​.

This is the paper's O(⋅)O(\cdot)O(⋅) statement. It does not fix the constant, which the paper's appendix computes with errors.

Milestones, in the order of the proof

  1. Lemma 1: given HtkH_{t_k}Htk​​, M∗M^*M∗ and MkM_kMk​ have the same conditional law.
  2. Theorem 2, (4): E[∑k≤mΔk]=E[∑k≤mΔ~k]\mathbb E[\sum_{k\le m}\Delta_k]=\mathbb E[\sum_{k\le m}\tilde\Delta_k]E[∑k≤m​Δk​]=E[∑k≤m​Δ~k​], where Δ~k=∑sρ(s)(Vμk,1Mk(s)−Vμk,1M∗(s))\tilde\Delta_k=\sum_s\rho(s)(V^{M_k}_{\mu_k,1}(s)-V^{M^*}_{\mu_k,1}(s))Δ~k​=∑s​ρ(s)(Vμk​,1Mk​​(s)−Vμk​,1M∗​(s)).
  3. Lemma 2, (5): the dynamic programming equation Vμ,iM=Tμ(⋅,i)MVμ,i+1MV^M_{\mu,i}=\mathcal T^M_{\mu(\cdot,i)}V^M_{\mu,i+1}Vμ,iM​=Tμ(⋅,i)M​Vμ,i+1M​.
  4. (6): E[Δ~k∣M∗,Mk]\mathbb E[\tilde\Delta_k\mid M^*,M_k]E[Δ~k​∣M∗,Mk​] equals the expected sum of one-step Bellman errors along the visited states.
  5. (7): the pathwise split of ∑kΔ~k\sum_k\tilde\Delta_k∑k​Δ~k​ on the event {Mk,M∗∈Mk}\{M_k,M^*\in\mathcal M_k\}{Mk​,M∗∈Mk​}.
  6. P(M∗∉Mk)≤1/m\mathbb P(M^*\notin\mathcal M_k)\le1/mP(M∗∈/Mk​)≤1/m for the paper's confidence sets (Lemma 17 of Jaksch et al.).
  7. (8): E[∑kΔ~k]≤(τ+1) E[∑k∑imin⁡{βk(sk,i,ak,i),1}]+2τ\mathbb E[\sum_k\tilde\Delta_k]\le(\tau+1)\,\mathbb E[\sum_k\sum_i\min\{\beta_k(s_{k,i},a_{k,i}),1\}]+2\tauE[∑k​Δ~k​]≤(τ+1)E[∑k​∑i​min{βk​(sk,i​,ak,i​),1}]+2τ.
  8. Appendix B: ∑k∑i1(Ntk(sk,i,ak,i)≤τ)≤2τSA\sum_k\sum_i\mathbf 1(N_{t_k}(s_{k,i},a_{k,i})\le\tau)\le2\tau SA∑k​∑i​1(Ntk​​(sk,i​,ak,i​)≤τ)≤2τSA.
  9. Appendix B: ∑k∑i1(Nk,i>τ)/Nk,i≤8SAmτ\sum_k\sum_i\sqrt{\mathbf 1(N_{k,i}>\tau)/N_{k,i}}\le\sqrt{8SAm\tau}∑k​∑i​1(Nk,i​>τ)/Nk,i​​≤8SAmτ​.

Significance

Theorem 1 holds for every prior, with no assumption on the MDP class beyond finiteness and bounded rewards. It shows that a single posterior sample per episode, rather than an optimistic search over a confidence set, already achieves the T\sqrt TT​ rate of the optimistic algorithms. The reduction it rests on (Lemma 1 and Theorem 2) applies to any problem in which the agent can sample from the posterior.

The theorem has a paper proof but, as far as is known, no machine-checked proof. A formalization adds three things. First, an exact model of the episodic Bayesian setting, with posterior sampling stated as a conditional law. Second, corrected versions of two steps: the factor in (8), which the page's argument does not justify as printed, and the square-root sum of Appendix B, which is false as printed. Third, reusable pieces: a path-defined finite-horizon value with its dynamic programming equation, a conditional-expectation form of the posterior-sampling identity, and deterministic counting lemmas for visit counts.

Difficulty

The proof combines three kinds of argument, each with its own obstacle in Lean.

  • Conditioning. Lemma 1 and Theorem 2 identify conditional laws given the history. The history is a random element whose length grows with kkk, and the posterior-sampling assumption only speaks about indicators of MkM_kMk​. Extending it to functions of (M,Htk)(M,H_{t_k})(M,Htk​​) and applying the tower property needs care with sub-σ\sigmaσ-algebras. The same issue appears in (8): the confidence event depends on HtkH_{t_k}Htk​​, so it is not measurable with respect to σ(M∗,Mk)\sigma(M^*,M_k)σ(M∗,Mk​), the σ\sigmaσ-algebra on which (6) conditions. The working argument conditions on (M∗,Mk,Htk)(M^*,M_k,H_{t_k})(M∗,Mk​,Htk​​) instead.
  • Concentration over adaptive counts. The bound P(M∗∉Mk)≤1/m\mathbb P(M^*\notin\mathcal M_k)\le1/mP(M∗∈/Mk​)≤1/m needs an L1L^1L1 deviation inequality for empirical transition distributions (Weissman et al.) and Hoeffding's inequality for rewards. Both must hold uniformly over the random number of visits, which the agent's own behaviour determines. Applying the inequalities at a fixed number of samples is not valid, because the sample size is random.
  • Counting. The appendix sums N−1/2N^{-1/2}N−1/2 along the visited pairs; the paper's chain loses a factor of 222 at its integral comparison.

Formalization scope

  • Model. States and actions are Fin S and Fin A. Steps and episodes are 0-based: Lean step j is the paper's step j+1j+1j+1, and Lean episode k is the paper's episode k+1k+1k+1, with tk=kτ+1t_k=k\tau+1tk​=kτ+1. MDPs are a measurable family over a parameter space Θ: transition rows satisfy the published IsTransitionKernel, and reward laws are Markov kernels supported on [0,1][0,1][0,1]. The initial distribution ρ\rhoρ is fixed. The value is defined from the state laws along the trajectory, not by the Bellman recursion.

  • The run. PSRLRun consists of measurable random elements on a probability space. Posterior sampling, the reset from ρ\rhoρ, rewards and transitions are stated as conditional expectations of indicators. Three assumptions the paper leaves implicit are made explicit:

    • the state is reset from ρ\rhoρ at every episode start;
    • the state reached after the last action of an episode is observed;
    • the sampling step uses fresh randomness.

    The optimal-policy selector is any measurable map that is optimal at every step.

  • Constants and corrections. O(⋅)O(\cdot)O(⋅) is one absolute constant, quantified before every other object; a constant that depends on SSS, AAA, τ\tauτ, the prior or TTT would make the goal trivial, because the regret is at most ⌈T/τ⌉τ\lceil T/\tau\rceil\tau⌈T/τ⌉τ. Milestone 7 uses the factor τ+1\tau+1τ+1, which the page's chain proves, where the page prints τ\tauτ, and milestone 9 uses 8SAmτ\sqrt{8SAm\tau}8SAmτ​ where the page prints the false bound 2SAT\sqrt{2SAT}2SAT​; both corrections are disclosed in the statements.

  • Ruled out. The value is not defined by the recursion, which would make Lemma 2 definitional. Measurability is required, so the expectations are not junk zeros. The posterior-sampling clause concerns indicators of MkM_kMk​ only, so it does not already contain Lemma 1.

  • Needed infrastructure. Conditional expectations with respect to σ\sigmaσ-algebras generated by random elements; finite-horizon Markov chain laws; Weissman's L1L^1L1 inequality and Hoeffding's inequality at random sample sizes; elementary bounds on ∑jj−1/2\sum_j j^{-1/2}∑j​j−1/2. The counting milestones (8, 9) and Lemma 2 are self-contained and a good place to start. Proofs of the conditioning steps (milestones 1, 2, 4) are reusable for any posterior-sampling analysis.

Selected references

  • I. Osband, D. Russo, B. Van Roy, (More) Efficient Reinforcement Learning via Posterior Sampling, NIPS 2013; arXiv:1306.0940v5. https://arxiv.org/abs/1306.0940
  • T. Jaksch, R. Ortner, P. Auer, Near-optimal Regret Bounds for Reinforcement Learning, JMLR 11, 2010. https://jmlr.org/papers/v11/jaksch10a.html
  • D. Russo, B. Van Roy, Learning to Optimize via Posterior Sampling, Mathematics of Operations Research 39(4), 2014; arXiv:1301.2609. https://arxiv.org/abs/1301.2609
  • M. Strens, A Bayesian Framework for Reinforcement Learning, Proceedings of the 17th International Conference on Machine Learning (ICML), 2000 (reference [16] of the paper).
  • P. L. Bartlett, A. Tewari, REGAL: A Regularization Based Algorithm for Reinforcement Learning in Weakly Communicating MDPs, UAI 2009; arXiv:1205.2661. https://arxiv.org/abs/1205.2661
  • T. Weissman, E. Ordentlich, G. Seroussi, S. Verdú, M. J. Weinberger, Inequalities for the L1 Deviation of the Empirical Distribution, HP Labs Technical Report HPL-2003-97, 2003.
14 thms1 active userReviewed
Convex OptimizationDynamic ProgrammingOperations Research+1·Captain: mikedeng1

On the Convergence of Decomposition Methods for Multistage Stochastic Convex Programs 2: Under τ-Admissible Random Node Selection, Cuts Become Exact Almost Surely at Every Tree NodeResearch Paper

Motivation

Stochastic dual dynamic programming (SDDP) and its relatives are the standard algorithms for multistage stochastic programs: hydro-thermal scheduling, energy storage and long-horizon capacity planning are solved with them in industry. They approximate each future cost function from below by a maximum of affine cuts, compute new cuts only at a few randomly chosen nodes of the scenario tree, and use the current approximations to simulate decisions forward. Whether such sampled cutting-plane schemes converge, and in what sense, decides whether their bounds can be trusted.

  • Pereira and Pinto (1991) introduced SDDP for multistage stochastic linear programs.
  • Chen and Powell (1999) proposed CUPPS, which samples a single scenario per iteration.
  • Philpott and Guan (2008) proved almost-sure convergence in finitely many iterations of DOASA and SDDP-type methods for linear stage problems with stagewise independent data, using the finiteness of the set of possible cuts.
  • Girardeau, Leclère and Philpott (2015) removed linearity: stage problems are convex with compact feasible sets, the data need not be stagewise independent, and the node selection is any τ-admissible random process. Convergence is then asymptotic, not finite.

This mission formalizes the stochastic part (§3) of the 2015 paper. Mission 1 of the series covers its deterministic part (§2).

Setting

A finite scenario tree has nodes N\mathcal NN, root 000, parent map ppp, children r(n)r(n)r(n), leaves L\mathcal LL, and node probabilities Φn>0\Phi_n>0Φn​>0 with Φ0=1\Phi_0=1Φ0​=1 and Φn=∑m∈r(n)Φm\Phi_n=\sum_{m\in r(n)}\Phi_mΦn​=∑m∈r(n)​Φm​. Every node m≠0m\neq0m=0 carries a control multifunction Um:Rd⇉Rp\mathcal U_m:\mathbb R^d\rightrightarrows\mathbb R^pUm​:Rd⇉Rp, a cost Cm(x,u)∈R∪{+∞}C_m(x,u)\in\mathbb R\cup\{+\infty\}Cm​(x,u)∈R∪{+∞} and linear dynamics xm=fm(xp(m),um)x_m=f_m(x_{p(m)},u_m)xm​=fm​(xp(m)​,um​); every node carries a compact convex stock set Xn\mathcal X_nXn​; leaves carry final costs VmV_mVm​. The future cost functions are defined backwards by (20):

Vn(x)=∑m∈r(n)ΦmΦnmin⁡u∈Um(x)Cm(x,u)+Vm(fm(x,u))⏟Wm(x,u),x∈Xn.V_n(x)=\sum_{m\in r(n)}\frac{\Phi_m}{\Phi_n}\min_{u\in\mathcal U_m(x)}\underbrace{C_m(x,u)+V_m\big(f_m(x,u)\big)}_{W_m(x,u)},\qquad x\in\mathcal X_n .Vn​(x)=m∈r(n)∑​Φn​Φm​​u∈Um​(x)min​Wm​(x,u)Cm​(x,u)+Vm​(fm​(x,u))​​,x∈Xn​.

Assumptions (H2)(H_2)(H2​) ask for convex, lower semicontinuous, proper costs, convex compact-valued convex multifunctions Um\mathcal U_mUm​, Lipschitz final costs, and an extended relatively complete recourse condition: feasibility and finite cost on a δ\deltaδ-neighbourhood of every Xn\mathcal X_nXn​.

The algorithm keeps approximations VnkV^{k}_nVnk​, starting from Vn−1≡−∞V^{-1}_n\equiv-\inftyVn−1​≡−∞. At iteration kkk it simulates the whole tree with Vk−1V^{k-1}Vk−1, producing stocks xnkx^k_nxnk​ and controls umku^k_mumk​. Cuts θnk+⟨βnk,⋅−xnk⟩\theta^k_n+\langle\beta^k_n,\cdot-x^k_n\rangleθnk​+⟨βnk​,⋅−xnk​⟩ are computed at the selected nodes by solving the stage problems (21) at their children and averaging optimal values and Lagrange multipliers with weights Φm/Φn\Phi_m/\Phi_nΦm​/Φn​. Every approximation is then updated by (22), Vnk=max⁡(Vnk−1,cut)V^k_n=\max(V^{k-1}_n,\text{cut})Vnk​=max(Vnk−1​,cut). The selections ynk∈{0,1}y^k_n\in\{0,1\}ynk​∈{0,1} form a random process. It is τ-admissible (Definition 1) when, inside each block of τ\tauτ iterations, cuts are computed backwards along the tree, and when the block indicators y~nk=max⁡κ<τynkτ+κ\tilde y^k_n=\max_{\kappa<\tau}y^{k\tau+\kappa}_ny~​nk​=maxκ<τ​ynkτ+κ​ are i.i.d. in kkk, independent of the past, and equal to 111 with positive probability.

Formalization targets

Goal: Theorem 3.1

Under (H2)(H_2)(H2​) and a τ-admissible selection process, almost surely, for every non-leaf node nnn and every node nnn respectively,

lim⁡k→∞∑m∈r(n)ΦmΦnWm(xnkτ,umkτ)−Vn(xnkτ)=0,lim⁡k→∞Vn(xnkτ)−Vnkτ(xnkτ)=0.\lim_{k\to\infty}\sum_{m\in r(n)}\frac{\Phi_m}{\Phi_n}W_m\big(x^{k\tau}_n,u^{k\tau}_m\big)-V_n\big(x^{k\tau}_n\big)=0,\qquad \lim_{k\to\infty}V_n\big(x^{k\tau}_n\big)-V^{k\tau}_n\big(x^{k\tau}_n\big)=0 .k→∞lim​m∈r(n)∑​Φn​Φm​​Wm​(xnkτ​,umkτ​)−Vn​(xnkτ​)=0,k→∞lim​Vn​(xnkτ​)−Vnkτ​(xnkτ​)=0.

The decisions become optimal for the true problem and the approximations become exact at the visited stocks. Nothing is claimed about exactness elsewhere in the state space.

Milestones

  • (26): every new cut is tight at the stock where it is computed.
  • Lemma 3.1 (i)–(iii): the VnV_nVn​ are convex and Lipschitz on Xn\mathcal X_nXn​; the approximations are lower bounds; finite approximations are uniformly Lipschitz.
  • Lemma 3.2: exact approximations at the children make the decisions at a node asymptotically optimal.
  • Lemma 5.3: under Definition 1 (i), a node selected inside a block is visited at the stock and with the approximation it had at the start of the block.
  • Lemma 5.4: the terms of an i.i.d. sequence selected by a predictable rule are again i.i.d. with the same law.
  • (32): exactness along the iterations at which a node is selected.

Significance

The theorem covers in one statement CUPPS (1-admissible), DOASA and SDDP ((T−1)(T-1)(T−1)-admissible), and many other sampling schemes, for stage problems that are convex rather than linear and for scenario trees without stagewise independence. It shows that the lower bound produced by these methods converges to the optimal value along the simulated trajectories.

The result is proved on paper. It is not formalized anywhere: no proof assistant library contains SDDP-type algorithms for convex stage problems. This mission makes the algorithm, the τ-admissibility notion and the convergence statement precise. Formalizing them corrects slips of the printed statements: the lower-bound property holds on the affine hull of Xn\mathcal X_nXn​ only, the boundedness of the cut slopes concerns finite cuts only, and the theorem's two limits concern different node sets. The rest of the work is the formalization of the published proof.

Difficulty

The deterministic argument (mission 1) shows exactness at stage ttt from exactness at stage t+1t+1t+1 along the whole iteration sequence. In the stochastic case a node is updated only at the random iterations at which it is selected, while its stock xnkτx^{k\tau}_nxnkτ​ moves at every iteration. Exactness along the selected iterations (32) does not transfer to the others: a selection rule that updated a node only when its stock lay in a given region would leave the approximation inexact elsewhere. The proof has to show that the iterations at which the approximation is still ε\varepsilonε-inexact are, by independence, selected with positive frequency. This requires the event "kkk is ε\varepsilonε-inexact" to be measurable with respect to the past selections, so the algorithm's choices among multiple minimisers and multipliers must be deterministic (footnote 6). It also requires the optional-skipping Lemma 5.4 and the strong law of large numbers. The first idea of conditioning on the stocks directly fails because the stocks depend on all past selections.

Formalization scope

  • Spaces and values. Stocks in EuclideanSpace ℝ (Fin d), controls in EuclideanSpace ℝ (Fin p). Costs and value functions take values in EReal; convexity is epigraph convexity; Vn=+∞V_n=+\inftyVn​=+∞ off Xn\mathcal X_nXn​; the initial approximation is −∞-\infty−∞. Limits are EReal limits, so convergence to 000 forces the differences to be finite eventually.
  • Tree. The platform definition StochasticProg.Multistage.Tree (a finite rooted tree with stages) is referenced; leaves, ascendents and node probabilities are defined on it. Costs are indexed by the child node as in (19a) and (20).
  • Standing assumptions, all disclosed in the items: (H2)(1)(H_2)(1)(H2​)(1)–(6)(6)(6) clause by clause, with the full ball B(δ)B(\delta)B(δ) of (H2)(6)(H_2)(6)(H2​)(6) as printed; x0∈X0x_0\in\mathcal X_0x0​∈X0​; Φ\PhiΦ are node probabilities.
  • Algorithm. A run is a predicate on the generated data (stocks, controls, cut values and slopes, approximations). Minimisers, Lagrange multipliers and subgradients are chosen by fixed rules, functions of the stage problem only (footnote 6), so a run is determined by its selection path. Lagrange multipliers of (21b) are encoded by the subgradient inequality of the optimal value of (21) on Aff⁡(Xn)\operatorname{Aff}(\mathcal X_n)Aff(Xn​), with the multiplier in its direction space. The Lean index is shifted: Vc k is the paper's Vk−1V^{k-1}Vk−1.
  • Selections. yky^kyk are measurable Bool-valued maps on a probability space; Definition 1 (i) holds on every path; (ii) is per node, i.i.d. in kkk and independent of σ(yk′:k′<kτ)\sigma(y^{k'}:k'<k\tau)σ(yk′:k′<kτ), with no independence across nodes.
  • Ruling out trivializations. The goal's hypotheses are only (H2)(H_2)(H2​), the choice rules, a run on every sample path, measurable selections and τ-admissibility. The companion statement run_exists shows that runs exist for every selection path, and a sanity check shows (H2)(H_2)(H2​) and τ-admissibility are satisfiable, so the goal is not vacuous. Its conclusion mentions none of the proof's auxiliary objects.
  • Infrastructure. The development needs marginal functions of convex problems over convex multifunctions, subdifferentiability of finite convex functions on affine hulls, Lipschitz bounds for maxima of affine functions, and the optional-skipping Lemma 5.4. Lemma 5.4 is a self-contained probability statement and is reusable on its own. Lemma 5.2 of the paper (a Dini-type lemma) is posed in mission 1. Proofs of any milestone, and of auxiliary lemmas on EReal-valued convex functions, are welcome.

Selected references

  • P. Girardeau, V. Leclère, A. B. Philpott, On the Convergence of Decomposition Methods for Multistage Stochastic Convex Programs, Mathematics of Operations Research 40(1), 2015. https://doi.org/10.1287/moor.2014.0664 (author's version: https://hal.science/hal-01208295v1)
  • A. B. Philpott, Z. Guan, On the convergence of stochastic dual dynamic programming and related methods, Operations Research Letters 36(4), 2008. https://doi.org/10.1016/j.orl.2008.01.013
  • Z. L. Chen, W. B. Powell, Convergent cutting-plane and partial-sampling algorithm for multistage stochastic linear programs with recourse, Journal of Optimization Theory and Applications 102(3), pp. 497–524, 1999 (reference [1] of the paper).
  • M. V. F. Pereira, L. M. V. G. Pinto, Multi-stage stochastic optimization applied to energy planning, Mathematical Programming 52, 1991. https://doi.org/10.1007/BF01582895
15 thms1 active userReviewed
Mathematical PhysicsPartial Differential EquationsStochastic Systems·Captain: mikedeng1

Global-in-Time Mean-Field Convergence for Singular Riesz-Type Diffusive Flows 2: A Uniform-in-Time Modulated-Energy Bound for Globally Superharmonic Riesz-Type PotentialsResearch Paper

Motivation

Systems of many particles that interact through a singular pair potential and are subject to independent noise appear in plasma physics (Coulomb interactions), in vortex models of fluids, in models of collective behaviour and in the analysis of training dynamics of wide neural networks. When the number of particles NNN is large, the empirical measure of the system is expected to follow a deterministic nonlinear partial differential equation, the mean-field limit. For singular interactions the classical coupling arguments, which need Lipschitz forces, do not apply. The modulated-energy method of Serfaty (Duke Math. J. 2020) and its extensions by Duerinckx, Nguyen, Rosenzweig and Serfaty (NRS21, arXiv:2107.02592) give quantitative convergence for Riesz-type potentials ∣x∣−s|x|^{-s}∣x∣−s, but with rates that degrade exponentially in time.

M. Rosenzweig and S. Serfaty (arXiv:2108.09878v1, 2021) show that additive noise improves this: the expected modulated energy grows at most polynomially in time for sub-Coulombic potentials (their Theorem 1.1, the subject of the first mission of this series), and stays bounded uniformly in time when the potential is globally superharmonic (their Theorem 1.2, the subject of this mission). Uniform-in-time propagation of chaos had been known for smooth or convex interactions; this result covers singular, non-gradient interactions on the whole space.

Setting

Fix d≥3d\ge3d≥3, an exponent 0≤s<d−20\le s<d-20≤s<d−2, a diffusion strength σ>0\sigma>0σ>0 and a d×dd\times dd×d matrix M\mathbb MM with Mξ⋅ξ≤0\mathbb M\xi\cdot\xi\le0Mξ⋅ξ≤0 for all ξ\xiξ. The particle system is

dxit=1N∑j≠iM∇g(xit−xjt) dt+2σ dWit,i=1,…,N,dx^t_i=\frac1N\sum_{j\ne i}\mathbb M\nabla\mathsf g(x^t_i-x^t_j)\,dt+\sqrt{2\sigma}\,dW^t_i,\qquad i=1,\dots,N,dxit​=N1​j=i∑​M∇g(xit​−xjt​)dt+2σ​dWit​,i=1,…,N,

started from pairwise distinct points xN0=(x10,…,xN0)x^0_N=(x^0_1,\dots,x^0_N)xN0​=(x10​,…,xN0​) and driven by independent standard Brownian motions W1,…,WNW_1,\dots,W_NW1​,…,WN​ in Rd\mathbb R^dRd. The potential g\mathsf gg is admissible: it satisfies ten assumptions (i)–(x) on symmetry, blow-up at 000, superharmonicity near 000, derivative bounds of order ∣x∣−s−k|x|^{-s-k}∣x∣−s−k, a doubling property, a Fourier transform comparable to ∣ξ∣−(d−s)|\xi|^{-(d-s)}∣ξ∣−(d−s), an extension to Rd+m\mathbb R^{d+m}Rd+m when d−4<sd-4<sd−4<s, a Calderón–Zygmund property in the cases s=d−2ks=d-2ks=d−2k, and M:∇⊗2g≥0\mathbb M:\nabla^{\otimes2}\mathsf g\ge0M:∇⊗2g≥0. The model case is g(x)=∣x∣−s\mathsf g(x)=|x|^{-s}g(x)=∣x∣−s.

The mean-field equation is

∂tμ=−div⁡(μ M∇g∗μ)+σΔμ,μ∣t=0=μ0,\partial_t\mu=-\operatorname{div}(\mu\,\mathbb M\nabla\mathsf g*\mu)+\sigma\Delta\mu,\qquad\mu|_{t=0}=\mu^0,∂t​μ=−div(μM∇g∗μ)+σΔμ,μ∣t=0​=μ0,

for a probability density μ0∈L∞\mu^0\in L^\inftyμ0∈L∞, solved in the mild (Duhamel) sense in C([0,∞);L1∩L∞)C([0,\infty);L^1\cap L^\infty)C([0,∞);L1∩L∞). Closeness of the empirical measure μNt=1N∑iδxit\mu_N^t=\frac1N\sum_i\delta_{x^t_i}μNt​=N1​∑i​δxit​​ to μt\mu^tμt is measured by the modulated energy

FN(xN,μ)=∫(Rd)2∖△g(x−y) d(1N∑i=1Nδxi−μ)⊗2(x,y),F_N(x_N,\mu)=\int_{(\mathbb R^d)^2\setminus\triangle}\mathsf g(x-y)\,d\Big(\frac1N\sum_{i=1}^N\delta_{x_i}-\mu\Big)^{\otimes2}(x,y),FN​(xN​,μ)=∫(Rd)2∖△​g(x−y)d(N1​i=1∑N​δxi​​−μ)⊗2(x,y),

where the diagonal △\triangle△ is removed. The potential is globally superharmonic if Δg≤0\Delta\mathsf g\le0Δg≤0 on all of Rd∖{0}\mathbb R^d\setminus\{0\}Rd∖{0} ("r0=∞r_0=\inftyr0​=∞" in assumption (iii)).

Formalization targets

Goal: Theorem 1.2 for 0<s<d−20<s<d-20<s<d−2

For globally superharmonic admissible g\mathsf gg and every d/(s+2)<p≤∞d/(s+2)<p\le\inftyd/(s+2)<p≤∞ there are βp>0\beta_p>0βp​>0 (depending on s,d,ps,d,ps,d,p), C>0C>0C>0 (depending on s,d,σ,M,g,∥μ0∥L∞s,d,\sigma,\mathbb M,\mathsf g,\|\mu^0\|_{L^\infty}s,d,σ,M,g,∥μ0∥L∞​), Cp>0C_p>0Cp​>0 and N0N_0N0​ such that for all t≥0t\ge0t≥0 and N≥N0N\ge N_0N≥N0​,

E(∣FN(xNt,μt)∣)≤C(∣FN(xN0,μ0)∣+CpN−βp).\mathbb E\big(|F_N(x^t_N,\mu^t)|\big)\le C\big(|F_N(x^0_N,\mu^0)|+C_pN^{-\beta_p}\big).E(∣FN​(xNt​,μt)∣)≤C(∣FN​(xN0​,μ0)∣+Cp​N−βp​).

The right side does not depend on ttt. The case s=0s=0s=0 of the paper's display (1.19) is not part of the goal (see Formalization scope).

Milestones

The proof combines an inequality for the time derivative of the expected modulated energy (Proposition 6.3), lower bounds for the energy and for the diffusion term (Remarks 5.10, 5.11, Proposition 5.12, Remark 5.13, Corollary 5.14), a commutator estimate for the transport term (Proposition 5.15), and decay of ∥μt∥Lp\|\mu^t\|_{L^p}∥μt∥Lp​ (Remark 3.4, Proposition 3.8), with the potential estimates of Lemma 2.3 and Remark 2.5.

Significance

A uniform-in-time bound means that the mean-field description is valid on time scales that do not shrink as the accuracy requirement grows. Combined with the control of negative Sobolev norms by the modulated energy, it yields uniform-in-time convergence in law of the empirical measure and, for chaotic initial data, uniform-in-time propagation of chaos at an explicit rate, for singular interactions such as sub-Coulombic Riesz gases with dissipative or conservative dynamics.

The result is proved in the paper. No part of it, nor of the modulated-energy method, is formalized in any proof assistant. A formalization would produce a reusable library: Riesz potentials and their L1L^1L1–LpL^pLp interpolation, heat-type decay of nonlinear Fokker–Planck equations, smeared potentials and electric-energy lower bounds, renormalized commutator estimates, and a pathwise treatment of interacting diffusions with additive noise. The milestones are independent statements and can be proved separately.

Difficulty

The obvious Gronwall argument on the modulated energy fails twice. First, FNF_NFN​ is not nonnegative; it is bounded below only up to errors involving ∥μ∥L∞\|\mu\|_{L^\infty}∥μ∥L∞​, and the lower bounds near the singularity need the smearing of point charges on spheres. Second, the error terms produced when the commutator and diffusion terms are bounded must be integrable on [0,∞)[0,\infty)[0,∞); with the estimates of the general (local superharmonicity) case they grow linearly in time. Global superharmonicity removes an η2\eta^2η2 error term and lets every error carry a power of ∥μt∥L∞\|\mu^t\|_{L^\infty}∥μt∥L∞​, which decays like (σt)−d/2(\sigma t)^{-d/2}(σt)−d/2 by the Carlen–Loss method; for s=0s=0s=0 long-range effects still produce growth, which is why that case is excluded. Itô's formula cannot be applied directly to the singular energy and requires a truncation, whose stochastic integrals are removed in expectation only in the limit.

Formalization scope

All objects live in the Lean namespace RieszMF.Global and in one definitions module. Space is EuclideanSpace ℝ (Fin d); derivatives are iteratedFDeriv (operator norm), gradient and the Laplacian of Mathlib. The committed conventions:

  1. Fourier conditions (vi) and (1.17) are read away from the origin: a function g^\hat{\mathsf g}g^​ with ∫g Fφ=∫g^φ\int\mathsf g\,\mathcal F\varphi=\int\hat{\mathsf g}\varphi∫gFφ=∫g^​φ for Schwartz φ\varphiφ vanishing near 000.
  2. Assumption (iv) has a constant CkC_kCk​ for each kkk; quantities singular at 000 are required for x≠0x\neq0x=0 only; (ix) is stated componentwise with an L2L^2L2-bounded operator represented by the kernel off the support.
  3. Global superharmonicity is Δg≤0\Delta\mathsf g\le0Δg≤0 on Rd∖{0}\mathbb R^d\setminus\{0\}Rd∖{0}, together with (1.15)–(1.16) on all of Rd+m∖{0}\mathbb R^{d+m}\setminus\{0\}Rd+m∖{0} when d−4<sd-4<sd−4<s; (v) and (vii) keep a finite radius.
  4. Doubling condition (vii) for s>0s>0s>0 is read as g(y)<csg(x)\mathsf g(y)<c_s\mathsf g(x)g(y)<cs​g(x) whenever ∣y∣≥2∣x∣|y|\ge2|x|∣y∣≥2∣x∣, x≠0x\ne0x=0, with cs<1c_s<1cs​<1. The page prints g(x)<csg(y)\mathsf g(x)<c_s\mathsf g(y)g(x)<cs​g(y), which contradicts the blow-up (ii) and would leave no admissible potential, so the goal would hold vacuously. The model potential ∣x∣−s|x|^{-s}∣x∣−s satisfies the reading used here.
  5. The particle system is the pathwise integral equation (no Itô integral is needed for additive noise), with continuous paths and no collisions almost surely; the Brownian family is N⋅dN\cdot dN⋅d jointly independent standard real Brownian motions.
  6. The mild formulation of (1.5) requires every integral to converge absolutely and continuity in L1∩L∞L^1\cap L^\inftyL1∩L∞.
  7. Expectations of ∣FN∣|F_N|∣FN​∣ are lower Lebesgue integrals, and Proposition 6.3 moves positive and negative parts across the inequality, so no integrability is presupposed and no ∞−∞\infty-\infty∞−∞ arises.
  8. Constants are quantified after exactly the objects they may depend on: βp\beta_pβp​ after s,d,ps,d,ps,d,p; CCC after σ,M,g\sigma,\mathbb M,\mathsf gσ,M,g and ∥μ0∥L∞\|\mu^0\|_{L^\infty}∥μ0∥L∞​; CpC_pCp​ and N0N_0N0​ after ppp; all before μ0\mu^0μ0, NNN, the configuration, the probability space and ttt.

Restriction. The goal is Theorem 1.2 for 0<s<d−20<s<d-20<s<d−2 only, where the paper's time factor 1+(t1/σlog⁡(1+t))1s=01+(t^{1/\sigma}\log(1+t))\mathbf 1_{s=0}1+(t1/σlog(1+t))1s=0​ equals 111. For s=0s=0s=0 the proof yields a factor of order tC/σt^{C/\sigma}tC/σ rather than the printed t1/σlog⁡(1+t)t^{1/\sigma}\log(1+t)t1/σlog(1+t); the logarithmic case is covered by the first mission of this series.

Excluded trivializations. A statement in which CCC is chosen after μ0\mu^0μ0, NNN or the configuration, in which a divergent integral is read as 000, in which the solution classes are empty, or which uses N=0N=0N=0 or a division by zero is not a formalization of the theorem; the statements here avoid each of these.

Contributions are welcome on any milestone, on the existence of solutions in the classes used (which shows the hypotheses are satisfiable), and on general infrastructure (Riesz potentials, sphere averages, heat-kernel estimates) that other missions can reuse.

Selected references

  • M. Rosenzweig, S. Serfaty, Global-in-time mean-field convergence for singular Riesz-type diffusive flows, arXiv:2108.09878v1, 2021; Ann. Appl. Probab. 33(2), 2023. https://arxiv.org/abs/2108.09878
  • Q.-H. Nguyen, M. Rosenzweig, S. Serfaty, Mean-field limits of Riesz-type singular flows, Ars Inveniendi Analytica, 2022. https://arxiv.org/abs/2107.02592
  • S. Serfaty, Mean field limit for Coulomb-type flows, Duke Math. J. 169(15), 2020. https://doi.org/10.1215/00127094-2020-0019
  • M. Duerinckx, Mean-field limits for some Riesz interaction gradient flows, SIAM J. Math. Anal. 48(3), 2016. https://doi.org/10.1137/15M1042620
  • E. A. Carlen, M. Loss, Optimal smoothing and decay estimates for viscously damped conservation laws, with applications to the 2-D Navier–Stokes equation, Duke Math. J. 81(1), 1995. https://doi.org/10.1215/S0012-7094-95-08110-1
14 thms1 active userReviewed
Operations ResearchOptimizationStochastic Systems·Captain: mikedeng1

Real-Time Dynamic Pricing for Revenue Management with Reusable Resources, Advance Reservation, and Deterministic Service Time Requirements II: Average-Loss Bound for Batch-Adjusted DPCResearch Paper

Motivation

Hotel rooms, rental cars, cloud servers, operating rooms and rental equipment are reusable resources: a customer occupies capacity for a fixed service time, and the capacity returns to the firm afterwards. Many of these settings also allow advance reservation, where a request made today is for a service that starts later. Lei and Jasin study dynamic pricing in this setting. Prices are posted period by period, demand is non-stationary, and the firm wants revenue close to the best achievable while never booking more units than it owns.

Classical network revenue management, where capacity is sold once and never returns, has an extensive asymptotic theory. In that literature, controls that re-solve or adjust a fluid (deterministic) plan are known to lose a vanishing fraction of the optimal revenue as demand and capacity grow (Gallego and van Ryzin 1997; Jasin 2014). Reusable resources break the standard argument, because each sale removes capacity for a window of time and not for the rest of the horizon. Lei and Jasin give the first control for reusable resources with non-stationary arrivals, deterministic service times and advance reservation whose average loss is o(θ−1/2)o(\theta^{-1/2})o(θ−1/2) in the scale parameter θ\thetaθ (Lei and Jasin, Operations Research 2020).

Setting

A firm manages III resource types with integer capacities Ci≥1C_i \ge 1Ci​≥1 and sells K≥1K \ge 1K≥1 service types over periods t=1,…,Tt = 1, \dots, Tt=1,…,T. At most one request arrives per period. A type-kkk request made at the beginning of period ttt uses aik∈{0,1}a_{ik} \in \{0,1\}aik​∈{0,1} units of resource iii during periods t+ℓk,…,t+ℓk+nk−1t+\ell_k, \dots, t+\ell_k+n_k-1t+ℓk​,…,t+ℓk​+nk​−1. Here nkn_knk​ is the service time and ℓk\ell_kℓk​ the reservation lead time.

In each period the firm chooses a demand-rate vector λt∈Ωλ=[0,λU]K\lambda^t \in \Omega_\lambda = [0,\lambda_U]^Kλt∈Ωλ​=[0,λU​]K, which is equivalent to choosing prices. A type-kkk request then arrives with probability λkt\lambda^t_kλkt​, independently of the past given λt\lambda^tλt, and the expected revenue rate is rt(λt)r^t(\lambda^t)rt(λt). The deterministic relaxation DET-H maximizes ∑t=1Trt(λt)\sum_{t=1}^T r^t(\lambda^t)∑t=1T​rt(λt) over λt∈Ωλ\lambda^t \in \Omega_\lambdaλt∈Ωλ​, subject to

∑k=1K∑s : s+ℓk≤t≤s+ℓk+nk−1aikλks≤Cifor every resource i and period t≤T.\sum_{k=1}^K \sum_{s \,:\, s+\ell_k \le t \le s+\ell_k+n_k-1} a_{ik}\lambda^s_k \le C_i \qquad \text{for every resource } i \text{ and period } t \le T.k=1∑K​s:s+ℓk​≤t≤s+ℓk​+nk​−1∑​aik​λks​≤Ci​for every resource i and period t≤T.

Its optimal value JHDJ^D_HJHD​ bounds the optimal expected revenue, and the average loss of a control π\piπ is L(π)=(JHD−E[Rπ])/T\mathcal{L}(\pi) = (J^D_H - \mathbf{E}[R^\pi])/TL(π)=(JHD​−E[Rπ])/T. Let λD\lambda^DλD be an optimal solution of DET-H. The quantity n‾k\underline{n}_kn​k​ is the smallest number of periods with λkt,D>0\lambda^{t,D}_k > 0λkt,D​>0 inside any service cycle of type kkk that carries positive demand.

The control DPC-B(m,ϵ)(m,\epsilon)(m,ϵ) splits the horizon, for each type kkk, into adjacent batches Tkb\mathcal{T}_k^bTkb​, each with mkm_kmk​ positive-rate periods. In period ttt it posts

λkt=Proj[0,λU][λkt,D−(ϵkn‾k+1mk∑s∈Tkβk(t)−1(Dks−λks))1{λkt,D>0}],\lambda^t_k = \mathrm{Proj}_{[0,\lambda_U]}\Bigl[\lambda^{t,D}_k - \Bigl(\frac{\epsilon_k}{\underline{n}_k} + \frac{1}{m_k}\sum_{s \in \mathcal{T}_k^{\beta_k(t)-1}} (D^s_k - \lambda^s_k)\Bigr)\mathbf{1}\{\lambda^{t,D}_k > 0\}\Bigr],λkt​=Proj[0,λU​]​[λkt,D​−(n​k​ϵk​​+mk​1​s∈Tkβk​(t)−1​∑​(Dks​−λks​))1{λkt,D​>0}],

where βk(t)\beta_k(t)βk​(t) is the batch containing ttt. The rate is set to 000 when fewer than aika_{ik}aik​ units of some resource iii are free at the service start t+ℓkt+\ell_kt+ℓk​. The buffer ϵk\epsilon_kϵk​ protects capacity, and the batch term corrects for the demand surprises of the previous batch.

Formalization targets

Goal: Theorem 3, display (8)

There is a constant M3>0M_3>0M3​>0, depending only on KKK, the curvature constant Ψ\PsiΨ of Assumption A5 and a bound RRR on ∣rt∣|r^t|∣rt∣, such that for every instance, every valid batch partition with mk≤n‾km_k \le \underline{n}_kmk​≤n​k​ and every ϵk\epsilon_kϵk​ in the stated range,

L(DPC-B(m,ϵ))≤M3∑k=1K[ϵkn‾k+1mk+Tmkexp⁡{−(min⁡jϵj−1)2256K2min⁡{max⁡iCi,mk}}].\mathcal{L}(\text{DPC-B}(m,\epsilon)) \le M_3 \sum_{k=1}^K \Bigl[\frac{\epsilon_k}{\underline{n}_k} + \frac{1}{m_k} + \frac{T}{m_k}\exp\Bigl\{-\frac{(\min_j\epsilon_j-1)^2}{256K^2\min\{\max_i C_i, m_k\}}\Bigr\}\Bigr].L(DPC-B(m,ϵ))≤M3​k=1∑K​[n​k​ϵk​​+mk​1​+mk​T​exp{−256K2min{maxi​Ci​,mk​}(minj​ϵj​−1)2​}].

The constant is uniform in TTT, CCC, nkn_knk​, ℓk\ell_kℓk​, mkm_kmk​ and ϵk\epsilon_kϵk​, which is what makes the bound informative.

Milestones

  1. Lemma EC.3: on the good event G(ϵ,m,δ)\mathcal{G}(\epsilon,m,\delta)G(ϵ,m,δ), the free capacity satisfies Ct⪰eC^t \succeq eCt⪰e and the corrected rates stay inside (0,λU)(0,\lambda_U)(0,λU​).
  2. (EC.22): on G\mathcal{G}G, the posted rate equals the unprojected corrected rate.
  3. Lemma EC.4, as proved in (EC.23): P(Gˉ)≤∑k4Tmkexp⁡{min⁡{max⁡iCi,mk}γk2−min⁡jϵj−18Kγk}\mathbf{P}(\bar{\mathcal{G}}) \le \sum_k \frac{4T}{m_k}\exp\{\min\{\max_i C_i, m_k\}\gamma_k^2 - \frac{\min_j\epsilon_j-1}{8K}\gamma_k\}P(Gˉ​)≤∑k​mk​4T​exp{min{maxi​Ci​,mk​}γk2​−8Kminj​ϵj​−1​γk​} for all γk∈(0,1]\gamma_k \in (0,1]γk​∈(0,1].
  4. E[(∑s∈TkbΔks)2]≤mk\mathbf{E}[(\sum_{s\in\mathcal{T}_k^b}\Delta^s_k)^2] \le m_kE[(∑s∈Tkb​​Δks​)2]≤mk​ for every batch.
  5. E([∇rt(λt,D)]⊤(ηδt⊙ηet))=0\mathbf{E}([\nabla r^t(\lambda^{t,D})]^\top(\eta^t_\delta\odot\eta^t_e)) = 0E([∇rt(λt,D)]⊤(ηδt​⊙ηet​))=0.

A companion item states Lemma 2 in this setting: stretching λD\lambda^DλD in time solves DET-H for the θ\thetaθ-th system of display (7).

Significance

Along the scaling (7), Theorem 3 with mk≈n‾k2/3m_k \approx \underline{n}_k^{2/3}mk​≈n​k2/3​ gives an average loss of O(θ−2/3log⁡1/2θ)O(\theta^{-2/3}\log^{1/2}\theta)O(θ−2/3log1/2θ). The non-adaptive control DPC of the same paper achieves O(θ−1/2log⁡1/2θ)O(\theta^{-1/2}\log^{1/2}\theta)O(θ−1/2log1/2θ) (its Theorem 1). The result shows that batch-level feedback improves on the square-root barrier for reusable resources, and it does so with heterogeneous service times and advance reservation, which cover car rental and hotel bookings.

Theorem 3 and its lemmas are proved in the paper's e-companion. No machine-checked proof of them, and no formal model of reusable-resource revenue management, is known. The mission produces a Lean model of the setting: capacity windows with lead times, the availability table C~(t,s)\tilde C(t,s)C~(t,s), an adapted control on finite sample paths, and its exact path law. On top of that model it states the paper's main bound in full. Several printed steps need repair (see the scope section), so a formal proof would also settle the exact constants.

Difficulty

The obvious route bounds each period's loss by a Taylor expansion around λt,D\lambda^{t,D}λt,D and controls the deviations by a martingale inequality. That route needs the control to coincide with the unprojected corrected rate for the whole horizon, that is, no capacity shortfall and no clipping. Capacity is a moving window of overlapping services with different lengths and lead times, so a shortfall at one period depends on requests made up to nk+ℓkn_k+\ell_knk​+ℓk​ periods earlier, in other batches and other types. The batch correction makes the rates depend on past errors, so the errors are martingale differences, not independent variables. The concentration bound must hold uniformly over batches whose number grows with TTT, with a variance proxy min⁡{max⁡iCi,mk}\min\{\max_i C_i, m_k\}min{maxi​Ci​,mk​} that is not the batch length. The early periods, before any service of type kkk has completed, need a separate threshold (the two cases of (EC.17)).

Formalization scope

All objects are defined in the namespace ReusablePricing.DPCB. The conventions are fixed as follows.

  • Periods are 1,…,T1,\dots,T1,…,T. A sample path is a function Fin T → Option (Fin K), recording no request or one request of a type in each period. Expectations are finite sums against the exact path law: the product over periods of the one-period law, whose probabilities are the rates posted given the past. The demand law is never left as a hypothesis.
  • Decisions are rates. A turn-off is rate 000. Prices and A1–A3 do not appear. The bound RRR on ∣rt∣|r^t|∣rt∣ over Ωλ\Omega_\lambdaΩλ​ replaces the proofs' rur^uru. KλU≤1K\lambda_U \le 1KλU​≤1 is assumed, as implied by at most one request per period.
  • Capacity windows are empty for t≤ℓkt \le \ell_kt≤ℓk​. Ct=C~(t,t)C^t = \tilde C(t,t)Ct=C~(t,t), and C~\tilde CC~ is defined by its defining sentence on p. 18, not by the update rule of Step 2a, whose indices do not match its own explanation. Admission is tested for services starting within the horizon.
  • n‾\underline{n}n​ in (EC.17) is min⁡kn‾k\min_k\underline{n}_kmink​n​k​. The events of (EC.16) are read over the batches Tkb\mathcal{T}_k^bTkb​. Statements "on G\mathcal{G}G" are made on paths of positive probability.
  • Lemma EC.4 is stated with the 8K8K8K that its proof establishes, not the printed 4K4K4K.
  • Goal statements carry every standing hypothesis of Section 6: optimality of λD\lambda^DλD, the convention λkt,D∈{0}∪(0,λU)\lambda^{t,D}_k\in\{0\}\cup(0,\lambda_U)λkt,D​∈{0}∪(0,λU​), A4, A5, and the existence of a demand-carrying cycle for every type, which n‾k\underline{n}_kn​k​ presupposes. The sanity check exhibits instances, with and without lead time, where all hypotheses of Theorem 3 hold and the ϵ\epsilonϵ-range is nonempty.

The constant M3M_3M3​ is quantified before the instance, after only KKK, Ψ\PsiΨ and RRR. Choosing it after the instance would make the bound trivial, because the loss never exceeds 2R2R2R.

A complete development needs Freedman- or Doob-type inequalities for bounded martingale differences on finite path spaces, second-order Taylor bounds for C2C^2C2 functions on boxes in RK\mathbb{R}^KRK, and window-counting lemmas for capacity. The first two are reusable well beyond this mission. Contributions of any of the milestones, of general martingale tools for finite adapted processes, or of a proof of the companion Lemma 2 are welcome.

Selected references

  • Y. Lei and S. Jasin, Real-Time Dynamic Pricing for Revenue Management with Reusable Resources, Advance Reservation, and Deterministic Service Time Requirements, Operations Research, 2020. https://doi.org/10.1287/opre.2019.1906 (author manuscript: SSRN 2816718, https://ssrn.com/abstract=2816718)
  • G. Gallego and G. van Ryzin, A Multiproduct Dynamic Pricing Problem and Its Applications to Network Yield Management, Operations Research 45(1), 1997. https://doi.org/10.1287/opre.45.1.24
  • S. Jasin, Reoptimization and Self-Adjusting Price Control for Network Revenue Management, Operations Research 62(5), 2014. https://doi.org/10.1287/opre.2014.1276
9 thms1 active userReviewed
Operations ResearchOptimizationStochastic Systems·Captain: mikedeng1

Real-Time Dynamic Pricing for Revenue Management with Reusable Resources, Advance Reservation, and Deterministic Service Time Requirements I: Average-Loss Bound for Deterministic Price ControlResearch Paper

Motivation

Hotel rooms, rental cars, cloud servers, hospital beds and event spaces are reusable resources: a customer occupies a unit for a while and then returns it. Pricing them differs from classical revenue management, where a sold unit is gone for good. A sale now blocks capacity only for the length of the service, so the binding constraint is how many services overlap at once. Lei and Jasin (Oper. Res. 2020) study dynamic pricing in this setting with deterministic service times and, in the general model, advance reservations. They measure a pricing control by its average loss against a deterministic (fluid) benchmark.

The paper has two main results. Theorem 1 bounds the loss of a static heuristic, the Deterministic Price Control DPC(ϵ\epsilonϵ). Theorem 3 bounds the loss of an adaptive variant and is treated in the companion mission (II). This mission formalizes Theorem 1, the lemmas its proof rests on (Lemmas EC.1–EC.2 and the displays (EC.14), (EC.15) of the e-companion), and two structural lemmas of the model: the fluid upper bound J∗≤JDJ^* \le J^DJ∗≤JD (Lemma 1) and the scaling of the fluid solution (Lemma 2).

Setting

A firm manages III resource types with integer capacities C=(Ci)⪰eC = (C_i) \succeq eC=(Ci​)⪰e and offers JJJ service types over periods t=1,…,Tt = 1, \dots, Tt=1,…,T. At most one request arrives per period. A type-jjj request uses aij∈{0,1}a_{ij} \in \{0,1\}aij​∈{0,1} units of resource iii for nnn consecutive periods (a service cycle), and nnn divides TTT.

The firm posts a demand-rate vector λt∈Ωλ=[0,λU]J\lambda^t \in \Omega_\lambda = [0,\lambda_U]^Jλt∈Ωλ​=[0,λU​]J with λU≤1\lambda_U \le 1λU​≤1. A type-jjj request then arrives with probability λjt\lambda^t_jλjt​, independently of earlier periods given the posted rates, and the firm earns the revenue rate rt(λt)r^t(\lambda^t)rt(λt). Writing Djs∈{0,1}D^s_j \in \{0,1\}Djs​∈{0,1} for realized arrivals and (x)+=max⁡{1,x}(x)^+ = \max\{1,x\}(x)+=max{1,x}, the capacity free at the beginning of period ttt is

Ct=C−∑s=(t−n+1)+t−1ADs.C^t = C - \sum_{s=(t-n+1)^+}^{t-1} A D^s .Ct=C−s=(t−n+1)+∑t−1​ADs.

The deterministic relaxation is

DET:JD=max⁡{∑t=1Trt(λt):λt∈Ωλ, ∑s=(t−n+1)+tAλs⪯C  ∀t≤T},\mathbf{DET}:\quad J^D = \max\Big\{\sum_{t=1}^T r^t(\lambda^t) : \lambda^t \in \Omega_\lambda,\ \sum_{s=(t-n+1)^+}^{t} A\lambda^s \preceq C \ \ \forall t \le T\Big\},DET:JD=max{t=1∑T​rt(λt):λt∈Ωλ​, s=(t−n+1)+∑t​Aλs⪯C  ∀t≤T},

with optimal solution λD=(λt,D)\lambda^D = (\lambda^{t,D})λD=(λt,D). The revenue rates are strictly concave, bounded by RRR in absolute value, and have interior maximizers (A4). Near λt,D\lambda^{t,D}λt,D their gradients and Hessians are bounded by Ψ\PsiΨ (A5, with margins φL,φU\varphi_L, \varphi_UφL​,φU​).

The quantity n‾\underline nn​ of display (2) is the smallest number of periods with λjs,D>0\lambda^{s,D}_j > 0λjs,D​>0 in any service cycle that carries positive demand for some type jjj.

DPC(ϵ\epsilonϵ) posts, in period ttt, the rate λjt,D−(ϵ/n‾)1{λjt,D>0}\lambda^{t,D}_j - (\epsilon/\underline n)\mathbf 1\{\lambda^{t,D}_j > 0\}λjt,D​−(ϵ/n​)1{λjt,D​>0}, clamped to [0,λU][0,\lambda_U][0,λU​], for every type jjj with Ct⪰AjC^t \succeq A^jCt⪰Aj. It turns off every other type (rate 000). The average loss of a control π\piπ is L(π)=(JD−E[Rπ])/T\mathcal L(\pi) = (J^D - \mathbf E[R^\pi])/TL(π)=(JD−E[Rπ])/T.

Formalization targets

Goal: Theorem 1 (p. 13)

For ϵ∈(1,min⁡{min⁡{φL,φU}n‾, 6min⁡{max⁡iCi,n}+1}]\epsilon \in \big(1, \min\{\min\{\varphi_L,\varphi_U\}\underline n,\ 6\min\{\max_i C_i, n\} + 1\}\big]ϵ∈(1,min{min{φL​,φU​}n​, 6min{maxi​Ci​,n}+1}],

L(DPC(ϵ))≤M1[ϵn‾+Tnexp⁡{−(ϵ−1)236min⁡{max⁡iCi,n}}],\mathcal L(DPC(\epsilon)) \le M_1\Big[\frac{\epsilon}{\underline n} + \frac{T}{n}\exp\Big\{-\frac{(\epsilon-1)^2}{36\min\{\max_i C_i, n\}}\Big\}\Big],L(DPC(ϵ))≤M1​[n​ϵ​+nT​exp{−36min{maxi​Ci​,n}(ϵ−1)2​}],

with M1>0M_1 > 0M1​>0 depending only on JJJ, Ψ\PsiΨ and RRR, and not on TTT, CCC, nnn, ϵ\epsilonϵ or the other data.

Milestones (EC.3)

  1. Lemma EC.1. On the event G(ϵ,δ)\mathcal G(\epsilon,\delta)G(ϵ,δ) that cumulative demand errors stay below δ=(n‾−1)(ϵ−1)/(3n‾)\delta = (\underline n-1)(\epsilon-1)/(3\underline n)δ=(n​−1)(ϵ−1)/(3n​) within every cycle, Ct⪰eC^t \succeq eCt⪰e and λjt,D−ϵ/n‾∈(0,λU)\lambda^{t,D}_j - \epsilon/\underline n \in (0,\lambda_U)λjt,D​−ϵ/n​∈(0,λU​).
  2. (EC.14). On G(ϵ,δ)\mathcal G(\epsilon,\delta)G(ϵ,δ), DPC posts exactly λjt,D−(ϵ/n‾)1{λjt,D>0}\lambda^{t,D}_j - (\epsilon/\underline n)\mathbf 1\{\lambda^{t,D}_j>0\}λjt,D​−(ϵ/n​)1{λjt,D​>0}.
  3. (EC.15). E[exp⁡{±γ∑s∈cycle bΔjs}]≤exp⁡{min⁡{max⁡iCi,n}γ2}\mathbf E[\exp\{\pm\gamma\sum_{s\in\text{cycle }b}\Delta^s_j\}] \le \exp\{\min\{\max_i C_i,n\}\gamma^2\}E[exp{±γ∑s∈cycle b​Δjs​}]≤exp{min{maxi​Ci​,n}γ2} for γ∈(0,1]\gamma \in (0,1]γ∈(0,1].
  4. Lemma EC.2. P(Gˉ(ϵ,δ))≤(2JT/n)exp⁡{min⁡{max⁡iCi,n}γ2−δγ}\mathbf P(\bar{\mathcal G}(\epsilon,\delta)) \le (2JT/n)\exp\{\min\{\max_i C_i,n\}\gamma^2 - \delta\gamma\}P(Gˉ​(ϵ,δ))≤(2JT/n)exp{min{maxi​Ci​,n}γ2−δγ}.
  5. Step 2. JD−E[RDPC(ϵ)]≤T(ΨJϵ/n‾+(4RJT/n)exp⁡{min⁡{max⁡iCi,n}γ2−δγ})J^D - \mathbf E[R^{DPC(\epsilon)}] \le T\big(\Psi J\epsilon/\underline n + (4RJT/n)\exp\{\min\{\max_i C_i,n\}\gamma^2 - \delta\gamma\}\big)JD−E[RDPC(ϵ)]≤T(ΨJϵ/n​+(4RJT/n)exp{min{maxi​Ci​,n}γ2−δγ}).

Companion statements: Lemma 1 (J∗≤JDJ^* \le J^DJ∗≤JD for every feasible non-anticipating control) and Lemma 2 (for integer θ≥1\theta \ge 1θ≥1, λ⌈t/θ⌉,D\lambda^{\lceil t/\theta\rceil,D}λ⌈t/θ⌉,D solves the DET of the θ\thetaθ-th system T(θ)=θTT^{(\theta)} = \theta TT(θ)=θT, C(θ)=θCC^{(\theta)} = \theta CC(θ)=θC, n(θ)=θnn^{(\theta)} = \theta nn(θ)=θn).

Significance

Along the scaling (1), choosing ϵ\epsilonϵ of order θlog⁡θ\sqrt{\theta\log\theta}θlogθ​ in Theorem 1 gives an average loss of O(θ−1/2log⁡1/2θ)O(\theta^{-1/2}\log^{1/2}\theta)O(θ−1/2log1/2θ). A control computed entirely before the selling horizon is therefore asymptotically optimal for reusable resources, with an explicit rate. This bound is the baseline against which the paper's adaptive control (Theorem 3, rate O(θ−2/3log⁡1/2θ)O(\theta^{-2/3}\log^{1/2}\theta)O(θ−2/3log1/2θ)) is measured.

The proof shows a pattern common to fluid-based revenue management: a high-probability event on which the heuristic follows the fluid plan exactly, a martingale maximal inequality bounding the complement, and a Taylor bound for the cost of the buffer. Reusable resources add one ingredient: capacity is checked over sliding windows of length nnn.

The results are proved on paper; none is machine-checked. The formalization would provide a checked version of the model (sliding-window capacity, at most one arrival per period, turn-off), of the high-probability analysis, and of the uniform constant. The printed proof has slips that a checked proof must repair. In particular, the exponent obtained by EC.3's choice of γ\gammaγ is −n‾−2n‾⋅(ϵ−1)236min⁡{⋅}-\frac{\underline n-2}{\underline n}\cdot\frac{(\epsilon-1)^2}{36\min\{\cdot\}}−n​n​−2​⋅36min{⋅}(ϵ−1)2​; it differs from the printed one by an amount that a larger M1M_1M1​ absorbs.

Difficulty

The obvious argument says that DPC keeps an ϵ\epsilonϵ buffer, so it rarely runs out of capacity. The difficulty is that "rarely" has to hold uniformly over every sliding window of nnn periods. The rates DPC posts are themselves random, because a type is turned off whenever the realized free capacity is short.

Lemma EC.1 breaks this circularity by induction over periods on a fixed event. A capacity window of length n−1n-1n−1 straddles two consecutive cycles, which costs a factor 333 in δ\deltaδ, and only the periods with positive deterministic demand carry a buffer, which is why n‾\underline nn​ appears. The demand errors form a martingale-difference sequence with predictable, path-dependent rates, so (EC.15) needs a pathwise bound on the cumulative rate within a cycle. That bound comes from the DET constraint, not from independence.

Finally, the constant M1M_1M1​ must not depend on the instance. A bound in which M1M_1M1​ may depend on TTT or CCC is trivial, since the loss never exceeds 2R2R2R.

Formalization scope

Everything is stated in demand rates, as the paper does for DET and in all its proofs. Prices, A1–A3 and the price-derivative part of A5 do not appear. A turned-off type has rate 000.

A sample path is a function from {1,…,T}\{1,\dots,T\}{1,…,T} to "no request" or "one request of type jjj". Its probability is the product of the per-period probabilities under the rates the control posts, so expectations are finite sums. Expected revenue is E[∑trt(λt)]\mathbf E[\sum_t r^t(\lambda^t)]E[∑t​rt(λt)], the tower-property form of the proofs. Periods are 1,…,T1,\dots,T1,…,T, and window sums whose upper limit falls below the lower limit are empty.

The standing hypotheses, bundled as Standing φL φU Ψ R n̲, are:

  • I,J,n,T≥1I, J, n, T \ge 1I,J,n,T≥1, n∣Tn \mid Tn∣T, aij∈{0,1}a_{ij} \in \{0,1\}aij​∈{0,1} with a 111 in every column, C⪰eC \succeq eC⪰e, λU∈(0,1]\lambda_U \in (0,1]λU​∈(0,1];
  • JλU≤1J\lambda_U \le 1JλU​≤1 (added: one arrival per period on all of Ωλ\Omega_\lambdaΩλ​);
  • λD\lambda^DλD optimal for DET, with λjt,D∈{0}∪(0,λU)\lambda^{t,D}_j \in \{0\} \cup (0,\lambda_U)λjt,D​∈{0}∪(0,λU​);
  • A4 with the explicit bound RRR on ∣rt∣|r^t|∣rt∣, which replaces the paper's rur^uru;
  • A5, with the Hessian condition as a bound on the quadratic form;
  • n‾\underline nn​ as in (2), stated as an attained minimum, so n‾≥1\underline n \ge 1n​≥1;
  • the cycle assumption of p. 11.

The statements "on G(ϵ,δ)\mathcal G(\epsilon,\delta)G(ϵ,δ)" are claimed for every path of positive probability in the event. Theorem 1 quantifies M1M_1M1​ after JJJ, Ψ\PsiΨ, RRR and before everything else. This rules out the trivial reading in which M1M_1M1​ is chosen per instance. Step 2 uses n‾\underline nn​ where the printed line has nnn, and RRR for rur^uru. θ\thetaθ in Lemma 2 is an integer.

Two definition files carry the model: ReusablePricing.DPC.Model holds the instance, DET, A4–A5, n‾\underline nn​ and the θ\thetaθ-th system; ReusablePricing.DPC.Control holds the path law, DPC, loss, events and general controls. A complete proof needs a finite-path martingale toolkit: a Doob-type maximal inequality for exponential submartingales on a finite product space, and the Bernoulli moment-generating-function bound. It also needs a multivariate mean-value or Taylor estimate on a box and an induction over sliding capacity windows. The martingale tools are reusable beyond this mission. Contributions of those general lemmas, of proofs of individual milestones, and of the optional asymptotic corollary (4) are welcome.

Selected references

  • Y. Lei and S. Jasin, Real-Time Dynamic Pricing for Revenue Management with Reusable Resources, Advance Reservation, and Deterministic Service Time Requirements, Operations Research 68(3), 2020. https://doi.org/10.1287/opre.2019.1906 (author manuscript: SSRN 2816718, https://ssrn.com/abstract=2816718)
  • G. Gallego and G. van Ryzin, A Multiproduct Dynamic Pricing Problem and Its Applications to Network Yield Management, Operations Research 45(1), 1997. https://doi.org/10.1287/opre.45.1.24
  • S. Jasin, Reoptimization and Self-Adjusting Price Control for Network Revenue Management, Operations Research 62(5), 2014. https://doi.org/10.1287/opre.2014.1294
  • D. Williams, Probability with Martingales, Cambridge University Press, 1991. https://doi.org/10.1017/CBO9780511813658
8 thms1 active userReviewed
Actuarial ScienceOperations Research·Captain: mikedeng1

Inf-Convolution, Optimal Allocations, and Model Uncertainty for Tail Risk Measures IV: The Comonotonic Inf-Convolution of Tail Risk Measures Is a Tail Risk Measure with the Maximal Tail ParameterResearch Paper

Motivation

A risk sharing problem distributes an aggregate random loss XXX among nnn agents, each of whom evaluates its share with a risk measure ρi\rho_iρi​. The smallest achievable aggregate risk is the inf-convolution □i=1nρi(X)\square_{i=1}^n\rho_i(X)□i=1n​ρi​(X), and allocations that attain it are optimal. In insurance and in regulated banking the allocation is often required to be comonotonic: every share moves in the same direction as the total loss. This is the no-sabotage, or no-moral-hazard, condition of the insurance literature (Huberman, Mayers and Smith 1983; Carlier and Dana 2003). For risk measures that respect convex order, comonotonic allocations lose nothing (the comonotone improvement of Carlier, Dana and Galichon 2012). For the risk measures most used in practice, Value-at-Risk and its relatives, this is not true, so the comonotonic problem has to be studied in its own right.

Liu, Mao, Wang and Wei (2022) study inf-convolutions of tail risk measures, the risk measures that depend only on the distribution of a loss beyond a fixed quantile. Value-at-Risk, Expected Shortfall and Range-Value-at-Risk are all of this kind. Their Theorem 3 shows that the unconstrained inf-convolution of tail risk measures is again a tail risk measure, with tail parameter the sum of the individual ones. Theorem 4, the subject of this mission, treats the comonotonic constraint, and the tail parameter becomes the maximum.

Setting

Fix an atomless probability space (Ω,F,P)(\Omega,\mathcal F,\mathbb P)(Ω,F,P): every event of positive probability contains an event of strictly smaller positive probability. Random variables are real measurable functions on Ω\OmegaΩ, and a positive value is a loss. The distribution function of XXX is FX(x)=P(X≤x)F_X(x)=\mathbb P(X\le x)FX​(x)=P(X≤x), and its left-continuous inverse is FX−1(p)=inf⁡{x:FX(x)≥p}F_X^{-1}(p)=\inf\{x:F_X(x)\ge p\}FX−1​(p)=inf{x:FX​(x)≥p} for p∈(0,1]p\in(0,1]p∈(0,1]. A quantile uniform UXU_XUX​ of XXX is a random variable, uniform on [0,1][0,1][0,1], with FX−1(UX)=XF_X^{-1}(U_X)=XFX−1​(UX​)=X almost surely.

For p∈(0,1)p\in(0,1)p∈(0,1) the tail of XXX beyond its (1−p)(1-p)(1−p)-quantile is Xp=FX−1(1−p+pUX)X_p=F_X^{-1}(1-p+pU_X)Xp​=FX−1​(1−p+pUX​). Its distribution function is

P(Xp≤x)=(FX(x)−(1−p))+p,x∈R.(6)\mathbb P(X_p\le x)=\frac{(F_X(x)-(1-p))_+}{p},\qquad x\in\mathbb R .\tag{6}P(Xp​≤x)=p(FX​(x)−(1−p))+​​,x∈R.(6)

A risk measure ρ\rhoρ is a ppp-tail risk measure if ρ(X)=ρ(Y)\rho(X)=\rho(Y)ρ(X)=ρ(Y) whenever XpX_pXp​ and YpY_pYp​ have the same distribution (Definition 1). A ppp-generator of such a ρ\rhoρ is a law-invariant risk measure ρ∗\rho^*ρ∗ with ρ(X)=ρ∗(Xp)\rho(X)=\rho^*(X_p)ρ(X)=ρ∗(Xp​) for all XXX.

Two random variables are comonotonic, X/ ⁣ ⁣/YX/\!\!/YX//Y, if on an event Ω0\Omega_0Ω0​ of probability one, (X(ω)−X(ω′))(Y(ω)−Y(ω′))≥0(X(\omega)-X(\omega'))(Y(\omega)-Y(\omega'))\ge0(X(ω)−X(ω′))(Y(ω)−Y(ω′))≥0 for all ω,ω′∈Ω0\omega,\omega'\in\Omega_0ω,ω′∈Ω0​. The comonotonic allocations of XXX and the constrained inf-convolution are

An+(X)={(X1,…,Xn):∑i=1nXi=X, Xi/ ⁣ ⁣/X},⊞i=1nρi(X)=inf⁡{∑i=1nρi(Xi):(X1,…,Xn)∈An+(X)}.\mathbb A_n^+(X)=\Big\{(X_1,\dots,X_n):\sum_{i=1}^nX_i=X,\ X_i/\!\!/X\Big\},\qquad \mathop{\boxplus}_{i=1}^n\rho_i(X)=\inf\Big\{\sum_{i=1}^n\rho_i(X_i):(X_1,\dots,X_n)\in\mathbb A_n^+(X)\Big\}.An+​(X)={(X1​,…,Xn​):i=1∑n​Xi​=X, Xi​//X},⊞i=1n​ρi​(X)=inf{i=1∑n​ρi​(Xi​):(X1​,…,Xn​)∈An+​(X)}.

Formalization targets

Goal: Theorem 4 (i)–(ii)

Let ρi\rho_iρi​ be an εi\varepsilon_iεi​-tail risk measure, εi∈(0,1)\varepsilon_i\in(0,1)εi​∈(0,1), i=1,…,ni=1,\dots,ni=1,…,n, let ε=⋁i=1nεi\varepsilon=\bigvee_{i=1}^n\varepsilon_iε=⋁i=1n​εi​, and let ρi∗\rho_i^*ρi∗​ be an ε\varepsilonε-generator of ρi\rho_iρi​. Then for every XXX and every quantile uniform of XXX,

⊞i=1nρi(X)=⊞i=1nρi∗(Xε),\mathop{\boxplus}_{i=1}^n\rho_i(X)=\mathop{\boxplus}_{i=1}^n\rho_i^*(X_\varepsilon),⊞i=1n​ρi​(X)=⊞i=1n​ρi∗​(Xε​),

and ⊞i=1nρi\boxplus_{i=1}^n\rho_i⊞i=1n​ρi​ is an ε\varepsilonε-tail risk measure.

Milestones

  1. Monotonicity of tail parameters (p. 8). Every q∈[p,1)q\in[p,1)q∈[p,1) is a tail parameter of a ppp-tail risk measure.
  2. Lemma 2(a) (p. 21). For (X1,…,Xn)∈An+(X)(X_1,\dots,X_n)\in\mathbb A_n^+(X)(X1​,…,Xn​)∈An+​(X) there is (Y1,…,Yn)∈An+(Xε)(Y_1,\dots,Y_n)\in\mathbb A_n^+(X_\varepsilon)(Y1​,…,Yn​)∈An+​(Xε​) with Yi=d(Xi)εY_i\overset d=(X_i)_\varepsilonYi​=d(Xi​)ε​.
  3. Lemma 2(b) (p. 21). For (Y1,…,Yn)∈An+(Xε)(Y_1,\dots,Y_n)\in\mathbb A_n^+(X_\varepsilon)(Y1​,…,Yn​)∈An+​(Xε​) there is (X1,…,Xn)∈An+(X)(X_1,\dots,X_n)\in\mathbb A_n^+(X)(X1​,…,Xn​)∈An+​(X) with (Xi)ε=dYi(X_i)_\varepsilon\overset d=Y_i(Xi​)ε​=dYi​.

Significance

Theorem 4 gives an explicit formula for comonotonic risk sharing among tail risk measures. The optimal comonotonic aggregate risk of XXX is computed from the tail XεX_\varepsilonXε​ alone, by sharing the tail among the generators. As a consequence the constrained inf-convolution depends on XXX only through its distribution beyond the (1−ε)(1-\varepsilon)(1−ε)-quantile, with ε\varepsilonε the largest individual tail parameter. Compared with Theorem 3, where the unconstrained tail parameter is ∑iεi\sum_i\varepsilon_i∑i​εi​, the comonotonic constraint keeps the aggregate as tail-focused as the least tail-focused agent. For a regulator this means that comonotonic sharing among, for instance, Expected Shortfall agents at different levels produces a measure that still ignores the body of the loss distribution beyond a known level.

The result is proved in the paper, through Lemma 2, in Appendix B. No machine-checked proof of it is known. Formalizing it requires the comonotonic additivity of quantiles and the explicit construction of comonotonic allocations from quantile functions and a quantile uniform. Both are reusable well beyond this paper: they underlie comonotonic risk measures, distortion risk measures and the comonotone-improvement literature.

Difficulty

The theorem looks like a change of variables, and the proof in the paper is short. The work is in Lemma 2. Part (a) needs the comonotonic additivity of the left quantile function, FX−1=∑iFXi−1F^{-1}_X=\sum_iF^{-1}_{X_i}FX−1​=∑i​FXi​−1​ on (0,1)(0,1)(0,1) for comonotonic XiX_iXi​ with sum XXX, and has to produce an allocation whose sum equals the tail everywhere, not only almost everywhere. Part (b) runs the other way. A comonotonic allocation of the tail has to be extended below the (1−ε)(1-\varepsilon)(1−ε)-quantile, where the tail carries no information, without breaking comonotonicity with XXX or the sum constraint. The extension needs each component of the tail allocation to be bounded below. That bound is a consequence of comonotonicity and of the tail being bounded below, not a hypothesis.

Equalities in law between tails are a second source of friction. The tail XεX_\varepsilonXε​ is defined through a quantile uniform, which is not unique, and the statements must hold for every choice.

Formalization scope

The probability space is atomless in the classical sense (an explicit predicate, stronger than Mathlib's NoAtoms). The domain of every risk measure is L0L^0L0, all measurable real functions. The paper allows any convex cone containing L∞L^\inftyL∞ that is closed under equality in law and under taking tails; fixing L0L^0L0 is a disclosed specialization. Individual risk measures and generators are real valued; ⊞\boxplus⊞ is valued in the extended reals. The sum constraint of an allocation is pointwise on Ω\OmegaΩ. Equality in law of tails, Xp=dYpX_p\overset d=Y_pXp​=dYp​, is encoded by (6) as equality of (FX−(1−p))+(F_X-(1-p))_+(FX​−(1−p))+​ and (FY−(1−p))+(F_Y-(1-p))_+(FY​−(1−p))+​, and the law of (Xi)ε(X_i)_\varepsilon(Xi​)ε​ is written through (6) as well. Quantile uniforms are quantified over, never chosen, and the generator relation is required for every quantile uniform. The maximum ε\varepsilonε is the supremum over the finite index set, with n≥1n\ge1n≥1 agents. The existence of generators (Liu and Wang 2021, Proposition 3.1) is cited by the paper and enters as a hypothesis. Part (iii) of Theorem 4, which needs the comonotone-improvement theorem, is out of scope.

A real-valued infimum for ⊞\boxplus⊞ would return 000 on an empty or unbounded-below set and make the identity of (i) partly vacuous; the extended-real infimum and the nonemptiness of An+(X)\mathbb A_n^+(X)An+​(X) (one agent takes everything) rule this out.

A complete development needs the left quantile function and its comonotonic additivity, the existence of a quantile uniform on an atomless space (Föllmer and Schied 2016, Lemma A.32), the distribution of the tail (6), and the measurability of FX−1( ⋅ )F_X^{-1}(\,\cdot\,)FX−1​(⋅) composed with a random variable. All of these are reusable, and contributions of any of them are welcome.

Selected references

  • F. Liu, T. Mao, R. Wang, L. Wei, Inf-convolution, optimal allocations, and model uncertainty for tail risk measures, Mathematics of Operations Research, 2022; SSRN version 3490348 of January 16, 2022 (the version formalized here). https://ssrn.com/abstract=3490348, https://doi.org/10.1287/moor.2021.1217
  • F. Liu, R. Wang, A theory for measures of tail risk, Mathematics of Operations Research 46(3), 2021. https://doi.org/10.1287/moor.2020.1072
  • P. Embrechts, H. Liu, R. Wang, Quantile-based risk sharing, Operations Research 66(4), 2018. https://doi.org/10.1287/opre.2017.1716
  • G. Carlier, R.-A. Dana, A. Galichon, Pareto efficiency for the concave order and multivariate comonotonicity, Journal of Economic Theory 147, 207–229, 2012. https://doi.org/10.1016/j.jet.2011.11.002
  • G. Carlier, R.-A. Dana, Pareto efficient insurance contracts when the insurer's cost function is discontinuous, Economic Theory 21, 871–893, 2003.
  • G. Huberman, D. Mayers, C. W. Smith, Optimal insurance policy indemnity schedules, The Bell Journal of Economics 14(2), 1983.
  • H. Föllmer, A. Schied, Stochastic Finance: An Introduction in Discrete Time, 4th ed., De Gruyter, 2016.
  • A. J. McNeil, R. Frey, P. Embrechts, Quantitative Risk Management, revised ed., Princeton University Press, 2015.
7 thms1 active userReviewed
Actuarial ScienceOperations Research·Captain: mikedeng1

Inf-Convolution, Optimal Allocations, and Model Uncertainty for Tail Risk Measures III: The Inf-Convolution of Tail Risk Measures Is a Tail Risk Measure with the Summed Tail ParameterResearch Paper

Motivation

A risk measure assigns a number, typically a capital requirement, to a random financial loss. When a total loss XXX is shared among nnn agents, each judging its share by its own risk measure ρi\rho_iρi​, the smallest achievable aggregate risk is the inf-convolution

□i=1nρi(X)=inf⁡{∑i=1nρi(Xi):X1+⋯+Xn=X}.\mathop{\square}_{i=1}^n\rho_i(X)=\inf\Bigl\{\sum_{i=1}^n\rho_i(X_i):X_1+\dots+X_n=X\Bigr\}.□i=1n​ρi​(X)=inf{i=1∑n​ρi​(Xi​):X1​+⋯+Xn​=X}.

It describes optimal risk sharing in insurance, reinsurance and among the desks of a financial institution, and it is the basic object of the risk-sharing literature (Delbaen 2012; Rüschendorf 2013).

Regulatory risk measures such as Value-at-Risk (VaR) and Expected Shortfall depend only on the upper tail of the loss distribution. Liu and Wang (2021) formalized this as tail risk measures. Embrechts, Liu and Wang (2018) showed that the inf-convolution of VaRs and Range-VaRs is again of the same type with summed levels, and Liu, Wang and Wei (2020) showed that inf-convolutions of law-invariant risk measures are law-invariant under weak conditions. Theorem 3 of Liu, Mao, Wang and Wei (2022) unifies these observations for arbitrary tail risk measures. This mission is drawn from the authors' SSRN version of January 16, 2022.

Setting

Fix an atomless probability space (Ω,F,P)(\Omega,\mathcal F,P)(Ω,F,P): every event of positive probability contains an event of strictly smaller positive probability. Positive values of a random variable are losses. Write FX(x)=P(X≤x)F_X(x)=P(X\le x)FX​(x)=P(X≤x). For α∈(0,1)\alpha\in(0,1)α∈(0,1) the left and right VaR are

VaRαL(X)=inf⁡{x:FX(x)≥1−α},VaRαR(X)=inf⁡{x:FX(x)>1−α}.\mathrm{VaR}^L_\alpha(X)=\inf\{x:F_X(x)\ge1-\alpha\},\qquad \mathrm{VaR}^R_\alpha(X)=\inf\{x:F_X(x)>1-\alpha\}.VaRαL​(X)=inf{x:FX​(x)≥1−α},VaRαR​(X)=inf{x:FX​(x)>1−α}.

The domain is X=L∞\mathcal X=L^\inftyX=L∞, the essentially bounded random variables.

For p∈(0,1]p\in(0,1]p∈(0,1] the tail XpX_pXp​ of XXX beyond its (1−p)(1-p)(1−p)-quantile is the random variable with distribution function (FX(x)−(1−p))+/p(F_X(x)-(1-p))^+/p(FX​(x)−(1−p))+/p (display (6) of the paper). A functional ρ:X→R\rho:\mathcal X\to\mathbb Rρ:X→R is a ppp-tail risk measure if ρ(X)=ρ(Y)\rho(X)=\rho(Y)ρ(X)=ρ(Y) whenever XpX_pXp​ and YpY_pYp​ have the same law. For p=1p=1p=1 this is law-invariance. The functional is monotone if X≤YX\le YX≤Y a.s. implies ρ(X)≤ρ(Y)\rho(X)\le\rho(Y)ρ(X)≤ρ(Y), and (sup-norm) continuous from above if ρ(Xk)→ρ(X)\rho(X_k)\to\rho(X)ρ(Xk​)→ρ(X) whenever Xk≥XX_k\ge XXk​≥X a.s. and ess sup⁡∣Xk−X∣→0\operatorname{ess\,sup}|X_k-X|\to0esssup∣Xk​−X∣→0.

The allocations of XXX are An(X)={(X1,…,Xn)∈Xn:∑iXi=X}\mathbb A_n(X)=\{(X_1,\dots,X_n)\in\mathcal X^n:\sum_iX_i=X\}An​(X)={(X1​,…,Xn​)∈Xn:∑i​Xi​=X}, and □i=1nρi\mathop{\square}_{i=1}^n\rho_i□i=1n​ρi​ is the infimum above over An(X)\mathbb A_n(X)An​(X). It takes values in [−∞,∞)[-\infty,\infty)[−∞,∞).

Formalization targets

Goal: Theorem 3

Let ρi\rho_iρi​ be an εi\varepsilon_iεi​-tail risk measure with εi∈(0,1)\varepsilon_i\in(0,1)εi​∈(0,1), i=1,…,ni=1,\dots,ni=1,…,n, and suppose one of ρ1,…,ρn\rho_1,\dots,\rho_nρ1​,…,ρn​ is monotone and continuous from above. Then

□i=1nρi  is monotone and an ε-tail risk measure,ε=min⁡{∑i=1nεi,1}.\mathop{\square}_{i=1}^n\rho_i\ \text{ is monotone and an }\varepsilon\text{-tail risk measure},\qquad\varepsilon=\min\Bigl\{\sum_{i=1}^n\varepsilon_i,1\Bigr\}.□i=1n​ρi​  is monotone and an ε-tail risk measure,ε=min{i=1∑n​εi​,1}.

Milestones

  1. Lemma 1. For ε∈(0,1)\varepsilon\in(0,1)ε∈(0,1) and m∈[VaRεL(X),VaRεR(X)]m\in[\mathrm{VaR}^L_\varepsilon(X),\mathrm{VaR}^R_\varepsilon(X)]m∈[VaRεL​(X),VaRεR​(X)], max⁡{X,m}ε=dXε\max\{X,m\}_\varepsilon\overset d=X_\varepsilonmax{X,m}ε​=dXε​.
  2. Embrechts et al. (2018), Corollary 1, as used. ∑iVaRεiL(Xi)≥VaR∑iεiL(∑iXi)\sum_i\mathrm{VaR}^L_{\varepsilon_i}(X_i)\ge\mathrm{VaR}^L_{\sum_i\varepsilon_i}(\sum_iX_i)∑i​VaRεi​L​(Xi​)≥VaR∑i​εi​L​(∑i​Xi​) when ∑iεi<1\sum_i\varepsilon_i<1∑i​εi​<1.
  3. Monotonicity. If one ρj\rho_jρj​ is monotone, so is □iρi\mathop{\square}_i\rho_i□i​ρi​.
  4. Display (21). Under the hypotheses of Theorem 3 with ε=∑iεi<1\varepsilon=\sum_i\varepsilon_i<1ε=∑i​εi​<1, □iρi(X)=□iρi(max⁡{X,VaRεL(X)})\mathop{\square}_i\rho_i(X)=\mathop{\square}_i\rho_i(\max\{X,\mathrm{VaR}^L_\varepsilon(X)\})□i​ρi​(X)=□i​ρi​(max{X,VaRεL​(X)}).
  5. Law-invariance. Under the hypotheses of Theorem 3, □iρi\mathop{\square}_i\rho_i□i​ρi​ is law-invariant.

Significance

The theorem says that tail-dependence survives optimal risk sharing, and it quantifies how much: the aggregate looks at a tail of probability at most the sum of the individual ones. The parameter is sharp, since □iVaRαiR=VaR∑iαiR\mathop{\square}_i\mathrm{VaR}^R_{\alpha_i}=\mathrm{VaR}^R_{\sum_i\alpha_i}□i​VaRαi​R​=VaR∑i​αi​R​ and VaRαR\mathrm{VaR}^R_\alphaVaRαR​ has smallest tail parameter α\alphaα. Consequently, results proved for tail risk measures (representations through law-invariant functionals of the tail, existence of optimal allocations) apply to the outcome of a risk-sharing market, not only to its participants. The hypotheses are weak: no convexity, no translation invariance, and only one agent needs monotonicity and a mild continuity.

The result is proved in the paper; its proof relies on two results it cites but does not state, Lemma 1 and Theorem 2 of Liu, Wang and Wei (2020), and on Corollary 1 of Embrechts, Liu and Wang (2018). None of the three, and no part of the theory of tail risk measures or inf-convolutions on probability spaces, is formalized on the platform or in Mathlib as far as a search found. A formalization therefore has to build the cited results as well; milestones 2, 3 and 5 isolate them in the form used.

Difficulty

Tail-invariance of each ρi\rho_iρi​ concerns one share at a time, while the inf-convolution is a constraint on the sum of the shares: modifying a share below its quantile breaks the budget ∑iXi=X\sum_iX_i=X∑i​Xi​=X. The obvious argument, applying the tail property of each ρi\rho_iρi​ to its share of XXX and of YYY, fails because an allocation of XXX does not induce an allocation of YYY when Xε=dYεX_\varepsilon\overset d=Y_\varepsilonXε​=dYε​ but X≠YX\ne YX=Y. Even when X=dYX\overset d=YX=dY an allocation of XXX cannot in general be transported to an allocation of YYY exactly. This is why the theorem needs an atomless space and a continuity hypothesis on one agent, and why the cited law-invariance result is the hardest part to formalize.

Formalization scope

All objects live in the namespace TailRiskSharing.TailConv, in one definitions file. Conventions:

  • PPP is a probability measure that is atomless in the classical sense (stronger than Mathlib's NoAtoms).
  • The domain is L∞L^\inftyL∞ (measurable, almost surely bounded). The paper allows any convex cone containing L∞L^\inftyL∞ that is closed under equality in law and under taking tails; fixing L∞L^\inftyL∞ is a disclosed specialization.
  • The individual ρi\rho_iρi​ are real-valued; the inf-convolution is computed in EReal, so it can be −∞-\infty−∞, and monotonicity and the tail property are stated for that EReal-valued functional without assuming finiteness.
  • The sum constraint of an allocation holds pointwise.
  • Equality in law of tails, Xp=dYpX_p\overset d=Y_pXp​=dYp​, is encoded through display (6): (FX−(1−p))+=(FY−(1−p))+(F_X-(1-p))^+=(F_Y-(1-p))^+(FX​−(1−p))+=(FY​−(1−p))+. At p=1p=1p=1 this is equality of distribution functions, so the tail parameter min⁡{∑iεi,1}\min\{\sum_i\varepsilon_i,1\}min{∑i​εi​,1} needs no case split.
  • "One of the ρi\rho_iρi​" is an index jjj carrying both hypotheses. Continuity from above quantifies over approximating sequences Xk≥XX_k\ge XXk​≥X a.s. with ess sup⁡∣Xk−X∣→0\operatorname{ess\,sup}|X_k-X|\to0esssup∣Xk​−X∣→0, not only decreasing ones.

The goal cannot be satisfied trivially: the tail parameter is min⁡{∑iεi,1}\min\{\sum_i\varepsilon_i,1\}min{∑i​εi​,1} and never a number above 1, the inf-convolution is not truncated to a real value, and none of the hypotheses is vacuous (constant functionals satisfy all of them). Contributions are welcome for each cited result as reusable lemmas: the quantile subadditivity of Embrechts et al., monotonicity of an inf-convolution, and law-invariance of inf-convolutions on atomless spaces.

Selected references

  • F. Liu, T. Mao, R. Wang, L. Wei, Inf-convolution, optimal allocations, and model uncertainty for tail risk measures, SSRN 3490348, version of January 16, 2022; Mathematics of Operations Research, 2022. https://ssrn.com/abstract=3490348
  • F. Liu, R. Wang, A theory for measures of tail risk, Mathematics of Operations Research 46(3), 1109–1128, 2021. https://doi.org/10.1287/moor.2020.1072
  • P. Embrechts, H. Liu, R. Wang, Quantile-based risk sharing, Operations Research 66(4), 936–949, 2018. https://doi.org/10.1287/opre.2017.1716
  • P. Liu, R. Wang, L. Wei, Is the inf-convolution of law-invariant preferences law-invariant?, Insurance: Mathematics and Economics 91, 144–154, 2020. https://doi.org/10.1016/j.insmatheco.2020.01.008
  • F. Delbaen, Monetary Utility Functions, Osaka University Press, 2012.
  • L. Rüschendorf, Mathematical Risk Analysis, Springer, 2013. https://doi.org/10.1007/978-3-642-33590-7
7 thms1 active userReviewed
CombinatoricsOperations ResearchTheoretical Computer Science·Captain: mikedeng1

Online Stochastic Matching: New Algorithms with Better Bounds 1: Every Random-Lists Algorithm Has ALG/OPT ≥ E[ALG]/E[OPT] − 2ε/(c+ε) with Probability at Least 1 − 2exp(−2nε²)Research Paper

Motivation

Online bipartite matching with stochastic arrivals models the assignment of display-advertising impressions to advertisers: requests arrive one at a time, each must be assigned irrevocably on arrival or dropped, and the arrival sequence is random rather than adversarial. Since Feldman, Mehta, Mirrokni and Muthukrishnan beat the 1−1/e1 - 1/e1−1/e barrier in the i.i.d. arrival model (FOCS 2009), a line of work has designed algorithms with better guarantees. Jaillet and Lu (Math. Oper. Res. 2014) introduce a whole class of such algorithms, the Random Lists Algorithms, and obtain ratios 0.7290.7290.729, 0.7250.7250.725 and 0.7060.7060.706 for integral arrival rates, vertex weights and general arrival rates.

Two notions of performance coexist in this literature. One compares expectations, E[ALG]/E[OPT]\mathbb E[\mathrm{ALG}]/\mathbb E[\mathrm{OPT}]E[ALG]/E[OPT]; the other asks that ALG/OPT\mathrm{ALG}/\mathrm{OPT}ALG/OPT be large on most realizations, as in Feldman et al. Section 2 of Jaillet and Lu shows that, for every algorithm of their class, the two notions are quantitatively linked by an explicit exponential tail bound. This mission formalizes that section.

Setting

A bipartite graph G={A∪I,E}G = \{A \cup I, E\}G={A∪I,E} has a finite set AAA of advertisers, a set III of impression types, and edges E⊆A×IE \subseteq A \times IE⊆A×I; (a,i)∈E(a, i) \in E(a,i)∈E means advertiser aaa is interested in type iii. A fixed number nnn of requests arrive at periods 1,…,n1, \dots, n1,…,n, and their types are drawn i.i.d. from a known distribution on III. Every advertiser can receive at most one request.

A list for type iii is an ordered sequence of advertisers interested in iii. A Random Lists Algorithm fixes, for each type iii, a distribution Di\mathcal D_iDi​ over lists (constructed from an LP solution; the construction is irrelevant here). When a request of type iii arrives it draws a list from Di\mathcal D_iDi​ and assigns the request to the first advertiser on the list not yet matched, or drops it if every listed advertiser is matched.

Formally, each request carries an outcome xxx in a finite set X\mathcal XX determining its type typ(x)\mathrm{typ}(x)typ(x) and its list lst(x)\mathrm{lst}(x)lst(x); outcomes ω1,…,ωn\omega_1, \dots, \omega_nω1​,…,ωn​ are i.i.d. with law ν\nuν. Starting from W0=∅W_0 = \emptysetW0​=∅, the set WjW_jWj​ of matched advertisers after the jjj-th request is obtained from Wj−1W_{j-1}Wj−1​ by adding the first advertiser of lst(ωj)\mathrm{lst}(\omega_j)lst(ωj​) outside Wj−1W_{j-1}Wj−1​, if any. Then:

  • ALG(ω)=∣Wn∣\mathrm{ALG}(\omega) = |W_n|ALG(ω)=∣Wn​∣, the number of assigned requests;
  • OPT(ω)\mathrm{OPT}(\omega)OPT(ω) is the cardinality of a maximum matching of the realized graph, in which request jjj may go to aaa iff (a,typ(ωj))∈E(a, \mathrm{typ}(\omega_j)) \in E(a,typ(ωj​))∈E;
  • E[ALG]\mathbb E[\mathrm{ALG}]E[ALG], E[OPT]\mathbb E[\mathrm{OPT}]E[OPT] are expectations under ν⊗n\nu^{\otimes n}ν⊗n, and c=E[OPT]/nc = \mathbb E[\mathrm{OPT}]/nc=E[OPT]/n.

Formalization targets

Goal: the concluding display of Section 2 (p. 6)

If every list contains only interested advertisers, then for every n≥1n \ge 1n≥1 and every ϵ>0\epsilon > 0ϵ>0,

P(ALGOPT≥E[ALG]E[OPT]−2ϵc+ϵ)≥1−2exp⁡(−2nϵ2).\mathbb P\left(\frac{\mathrm{ALG}}{\mathrm{OPT}} \ge \frac{\mathbb E[\mathrm{ALG}]}{\mathbb E[\mathrm{OPT}]} - \frac{2\epsilon}{c + \epsilon}\right) \ge 1 - 2\exp(-2n\epsilon^2).P(OPTALG​≥E[OPT]E[ALG]​−c+ϵ2ϵ​)≥1−2exp(−2nϵ2).

It holds for every instance and every law ν\nuν, with no assumption on the size of E[OPT]\mathbb E[\mathrm{OPT}]E[OPT].

Milestones

  1. Claim 1 (p. 5). If two list sequences differ in one position, the cardinalities of the resulting matchings differ by at most one.
  2. Lemma 1 (p. 5). P(ALG−E[ALG]<−nϵ)≤exp⁡(−2nϵ2)\mathbb P(\mathrm{ALG} - \mathbb E[\mathrm{ALG}] < -n\epsilon) \le \exp(-2n\epsilon^2)P(ALG−E[ALG]<−nϵ)≤exp(−2nϵ2).
  3. Lemma 2 (p. 5). P(OPT−E[OPT]>nϵ)≤exp⁡(−2nϵ2)\mathbb P(\mathrm{OPT} - \mathbb E[\mathrm{OPT}] > n\epsilon) \le \exp(-2n\epsilon^2)P(OPT−E[OPT]>nϵ)≤exp(−2nϵ2).

A supporting theorem, not a milestone, records the fact the display uses without stating it: with interested lists, ALG≤OPT\mathrm{ALG} \le \mathrm{OPT}ALG≤OPT on every realization.

Significance

The display shows that the paper's guarantees, proved for the ratio of expectations, transfer to realizations: when E[OPT]=Θ(n)\mathbb E[\mathrm{OPT}] = \Theta(n)E[OPT]=Θ(n), ALG/OPT\mathrm{ALG}/\mathrm{OPT}ALG/OPT is at least E[ALG]/E[OPT]−O(ϵ)\mathbb E[\mathrm{ALG}]/\mathbb E[\mathrm{OPT}] - O(\epsilon)E[ALG]/E[OPT]−O(ϵ) except with probability 2e−2nϵ22e^{-2n\epsilon^2}2e−2nϵ2. It applies uniformly to the whole class of Random Lists Algorithms, whatever the LP and the list distributions, so it covers each of the paper's later algorithms at once.

The result is proved in the paper; nothing about it is open. It has no machine-checked proof. Formalizing it produces a precise model of list-based online matching (the greedy run, the realization-graph optimum, the joint type–list law) that later formalizations of the paper's competitive ratios can build on, and a worked application of the bounded-differences method to a combinatorial online algorithm.

Difficulty

The concluding display is elementary once Lemmas 1 and 2 are available. The two lemmas need McDiarmid's bounded-differences inequality in one-sided form, which Mathlib does not provide, and the bounded-difference properties of ALG and OPT.

Claim 1 is the step where the naive argument fails. Changing one list can change the assignment of every later request, so ALG is not a sum of local contributions, and a single changed list could a priori cascade into many changed assignments; the claim asserts that the cascade never changes the total by more than one. The analogous property for OPT (changing one request's type moves the maximum matching by at most one) is used in Lemma 2's proof but is not stated in the paper.

Formalization scope

  • Arrivals. Outcomes live in a finite type X\mathcal XX with the discrete σ-algebra, and the law of the arrivals is the product measure ν⊗n\nu^{\otimes n}ν⊗n (Measure.pi). Any joint law of type and list is allowed; every Random Lists Algorithm is the case X={(i,l)}\mathcal X = \{(i, l)\}X={(i,l)}, ν(i,l)=pi Di(l)\nu(i, l) = p_i\,\mathcal D_i(l)ν(i,l)=pi​Di​(l). Types and lists are not modelled as independent sequences, which would break ALG≤OPT\mathrm{ALG} \le \mathrm{OPT}ALG≤OPT.
  • Expectations are Bochner integrals of functions on the finite space Xn\mathcal X^nXn, hence honest finite sums.
  • Matchings. OPT is the maximum size of a partial assignment Fin n → Option A that uses only edges and gives each advertiser at most one request; the empty assignment qualifies, so the maximum is over a nonempty set. ALG is ∣Wn∣|W_n|∣Wn​∣ for the greedy recursion.
  • Pinned hypotheses. The goal assumes n≥1n \ge 1n≥1 (the paper's c=E[OPT]/nc = \mathbb E[\mathrm{OPT}]/nc=E[OPT]/n) and that every list only contains interested advertisers (stated in §2, p. 5). Claim 1 and Lemma 1 hold for arbitrary lists and carry neither hypothesis. Lists need not be nonempty or repetition-free.
  • Division by zero. No hypothesis E[OPT]>0\mathbb E[\mathrm{OPT}] > 0E[OPT]>0 or OPT>0\mathrm{OPT} > 0OPT>0 is made. On realizations with OPT=0\mathrm{OPT} = 0OPT=0, and when E[OPT]=0\mathbb E[\mathrm{OPT}] = 0E[OPT]=0, Lean's x/0=0x/0 = 0x/0=0 applies; the statement remains true under it, and adding a positivity hypothesis would weaken the result.
  • Not posed. Theorems 1–3 (the competitive ratios, which hold up to 1+O(1/n)1 + O(1/n)1+O(1/n)) and the LP constructions of the algorithms are out of scope. McDiarmid's inequality is a proof tool, not an item. A one-sided version is posed elsewhere on the platform and not yet proved.

Contributions welcome: a one-sided McDiarmid inequality for Measure.pi (reusable well beyond this mission), the bounded-difference property of maximum bipartite matchings, and Claim 1.

Selected references

  • P. Jaillet and X. Lu, Online Stochastic Matching: New Algorithms with Better Bounds, Mathematics of Operations Research 39(3), 2014; accepted manuscript, rev. June 2013. https://doi.org/10.1287/moor.2013.0621
  • J. Feldman, A. Mehta, V. Mirrokni and S. Muthukrishnan, Online Stochastic Matching: Beating 1 − 1/e, FOCS 2009. https://arxiv.org/abs/0905.4100
  • C. McDiarmid, On the method of bounded differences, Surveys in Combinatorics, London Math. Soc. Lecture Note Series 141, 1989. https://doi.org/10.1017/CBO9781107359949.008
5 thms1 active userReviewed
Mathematical PhysicsPartial Differential EquationsStochastic Systems·Captain: mikedeng1

Global-in-Time Mean-Field Convergence for Singular Riesz-Type Diffusive Flows 1: The Expected Modulated Energy Grows at Most Polynomially in TimeResearch Paper

Motivation

Systems of NNN particles that interact through a singular pair potential and are driven by independent noise are used to model vortices in viscous fluids, collective motion of micro-organisms, aggregation phenomena and, more recently, the training dynamics of wide neural networks and stochastic gradient methods. When NNN is large, one expects the empirical measure of the particles to be close to the solution of a deterministic nonlinear PDE, the mean-field limit. Proving this with an explicit rate is difficult when the interaction is singular, because the classical coupling arguments need a Lipschitz force.

The modulated-energy method (Duerinckx 2016, Serfaty 2020, Nguyen–Rosenzweig–Serfaty 2021) measures the distance between the particles and the limit through a renormalized Coulomb/Riesz energy and closes a Gronwall argument on it. Rosenzweig and Serfaty (arXiv:2108.09878) extended it to systems with additive noise in the sub-Coulombic range. This mission formalizes their first main result, Theorem 1.1: the expected modulated energy grows at most linearly in time (polynomially for logarithmic interactions). A companion mission covers their uniform-in-time Theorem 1.2.

Setting

Fix d≥3d\ge3d≥3 and 0≤s<d−20\le s<d-20≤s<d−2. Positions live in Rd\mathbb R^dRd with Lebesgue measure. A d×dd\times dd×d real matrix M\mathbb MM satisfies Mξ⋅ξ≤0\mathbb M\xi\cdot\xi\le0Mξ⋅ξ≤0 for all ξ\xiξ (gradient flows M=−I\mathbb M=-\mathbb IM=−I, Hamiltonian flows M\mathbb MM antisymmetric, and mixtures). The interaction potential g:Rd→R\mathsf g:\mathbb R^d\to\mathbb Rg:Rd→R is admissible: it satisfies ten assumptions (i)–(x), with model cases g(x)=−log⁡∣x∣\mathsf g(x)=-\log|x|g(x)=−log∣x∣ (s=0s=0s=0) and g(x)=∣x∣−s\mathsf g(x)=|x|^{-s}g(x)=∣x∣−s (0<s<d−20<s<d-20<s<d−2). In short: g\mathsf gg is even, blows up at 000, is superharmonic on a punctured ball B(0,r0)∖{0}B(0,r_0)\setminus\{0\}B(0,r0​)∖{0}, is smooth off the origin with ∣∇⊗kg(x)∣≤Ck∣x∣−(s+k)|\nabla^{\otimes k}\mathsf g(x)|\le C_k|x|^{-(s+k)}∣∇⊗kg(x)∣≤Ck​∣x∣−(s+k) (plus a logarithm when s=k=0s=k=0s=k=0), has a Fourier transform comparable to ∣ξ∣s−d|\xi|^{s-d}∣ξ∣s−d, has a doubling property near 000, admits a superharmonic extension to Rd+m\mathbb R^{d+m}Rd+m when d−4<sd-4<sd−4<s, has a Calderón–Zygmund kernel condition when s=d−2ks=d-2ks=d−2k, and satisfies M:∇⊗2g≥0\mathbb M:\nabla^{\otimes2}\mathsf g\ge0M:∇⊗2g≥0.

The particle system (1.1) is

dxit=1N∑j≠iM∇g(xit−xjt) dt+2σ dWit,xit∣t=0=xi0,dx^t_i=\frac1N\sum_{j\ne i}\mathbb M\nabla\mathsf g(x^t_i-x^t_j)\,dt+\sqrt{2\sigma}\,dW^t_i,\qquad x^t_i|_{t=0}=x^0_i,dxit​=N1​j=i∑​M∇g(xit​−xjt​)dt+2σ​dWit​,xit​∣t=0​=xi0​,

with distinct initial points, σ>0\sigma>0σ>0, and W1,…,WNW_1,\dots,W_NW1​,…,WN​ independent standard Brownian motions in Rd\mathbb R^dRd. The mean-field equation (1.5) is

∂tμ=−div⁡(μ M∇g∗μ)+σΔμ,μ∣t=0=μ0,\partial_t\mu=-\operatorname{div}(\mu\,\mathbb M\nabla\mathsf g*\mu)+\sigma\Delta\mu,\qquad\mu|_{t=0}=\mu^0,∂t​μ=−div(μM∇g∗μ)+σΔμ,μ∣t=0​=μ0,

for a bounded probability density μ0\mu^0μ0, solved in the class C([0,∞);P(Rd)∩L∞(Rd))C([0,\infty);\mathcal P(\mathbb R^d)\cap L^\infty(\mathbb R^d))C([0,∞);P(Rd)∩L∞(Rd)). The modulated energy (1.6) is

FN(xN,μ)=∫(Rd)2∖△g(x−y) d(1N∑i=1Nδxi−μ)⊗2(x,y),F_N(x_N,\mu)=\int_{(\mathbb R^d)^2\setminus\triangle}\mathsf g(x-y)\,d\Big(\frac1N\sum_{i=1}^N\delta_{x_i}-\mu\Big)^{\otimes2}(x,y),FN​(xN​,μ)=∫(Rd)2∖△​g(x−y)d(N1​i=1∑N​δxi​​−μ)⊗2(x,y),

the diagonal △\triangle△ being removed to discard the infinite self-interaction of each particle.

Formalization targets

Goal: Theorem 1.1

There are β>0\beta>0β>0 depending on s,ds,ds,d only and, for every σ\sigmaσ, M\mathbb MM, admissible g\mathsf gg and value of ∥μ0∥L∞\|\mu^0\|_{L^\infty}∥μ0∥L∞​, a constant C>0C>0C>0 and a threshold N0N_0N0​ such that for all N≥N0N\ge N_0N≥N0​ and t≥0t\ge0t≥0

E(∣FN(xNt,μt)∣)≤C(1+t+tσ+Cσ1s=0)(∣FN(xN0,μ0)∣+N−β).\mathbb E\big(|F_N(x^t_N,\mu^t)|\big)\le C\Big(1+t+t^{\frac{\sigma+C}\sigma}\mathbf 1_{s=0}\Big)\Big(|F_N(x^0_N,\mu^0)|+N^{-\beta}\Big).E(∣FN​(xNt​,μt)∣)≤C(1+t+tσσ+C​1s=0​)(∣FN​(xN0​,μ0)∣+N−β).

The exponent β\betaβ and the constant are left unspecified, as in the paper.

Milestones

In the order the proof uses them:

  1. Lemma 2.3 and Remark 2.5: L∞L^\inftyL∞ bounds for Riesz potentials and for ∇⊗kg∗f\nabla^{\otimes k}\mathsf g*f∇⊗kg∗f.
  2. Remark 3.4, Proposition 3.9 and Proposition 3.8: LpL^pLp monotonicity, the sharp log-Sobolev inequality, and heat-equation decay ∥μt∥Lq≲(σt)−d2(1p−1q)∥μ0∥Lp\|\mu^t\|_{L^q}\lesssim(\sigma t)^{-\frac d2(\frac1p-\frac1q)}\|\mu^0\|_{L^p}∥μt∥Lq​≲(σt)−2d​(p1​−q1​)∥μ0∥Lp​.
  3. Proposition 4.5: the particle system has a unique strong solution whose particles never collide, obtained as the limit of smoothly truncated systems.
  4. Remarks 5.2, 5.3 and Proposition 5.4: lower bounds on FNF_NFN​ through smeared point charges.
  5. Corollary 5.6: the diffusion term ∬(−Δg) d(μN−μ)⊗2\iint(-\Delta\mathsf g)\,d(\mu_N-\mu)^{\otimes2}∬(−Δg)d(μN​−μ)⊗2 is bounded below by −C(1+∥μ∥L∞)N−min⁡{2,d−s−2}/min⁡{s+4,d}-C(1+\|\mu\|_{L^\infty})N^{-\min\{2,d-s-2\}/\min\{s+4,d\}}−C(1+∥μ∥L∞​)N−min{2,d−s−2}/min{s+4,d}.
  6. Proposition 5.7: the renormalized commutator estimate.
  7. Proposition 6.3: the expected Itô inequality for FNF_NFN​.

Significance

The theorem is a quantitative mean-field limit for singular interactions with additive noise on the whole space, covering conservative and dissipative dynamics at once. Since FNF_NFN​ controls a negative-order Sobolev distance between μNt\mu_N^tμNt​ and μt\mu^tμt, it gives convergence of the empirical measure, at a rate in NNN that deteriorates at most linearly in ttt for s>0s>0s>0. Its proof isolates estimates (the LpL^pLp decay of the nonlinear equation, the sign of the diffusion term, the commutator bound) that are reused for the uniform-in-time result of the companion mission.

The paper's results are proved; none of them has a machine-checked proof. Formalizing the theorem requires the modulated-energy machinery (smeared charges, Fourier positivity, commutator estimates), a Carlen–Loss decay argument, and an Itô-free treatment of the stochastic system. The milestones of §§2–3 are classical harmonic analysis (Riesz potentials, the Gross–Carlen log-Sobolev inequality) and are of independent use.

Difficulty

A direct Gronwall argument on E∣FN∣\mathbb E|F_N|E∣FN​∣ fails twice. The diffusion term σ∬Δg d(μN−μ)⊗2\sigma\iint\Delta\mathsf g\,d(\mu_N-\mu)^{\otimes2}σ∬Δgd(μN​−μ)⊗2 has no sign for a general configuration, because the singularity of Δg\Delta\mathsf gΔg at the origin is integrated against a signed measure with atoms; its control needs a second modulated-energy estimate for −Δg-\Delta\mathsf g−Δg, which is a potential of order s+2s+2s+2 and may be super-Coulombic, hence the extension of assumption (viii). The commutator term requires ∥∇ut∥L∞\|\nabla u^t\|_{L^\infty}∥∇ut∥L∞​, and bounding it by ∥μ0∥L∞\|\mu^0\|_{L^\infty}∥μ0∥L∞​ alone produces exponential growth in time; the polynomial bound needs the time decay of ∥μt∥L∞\|\mu^t\|_{L^\infty}∥μt∥L∞​, which for a nonlinear, non-divergence-free drift is not given by comparison with the heat equation. Finally, the Itô computation for FNF_NFN​ along the singular dynamics needs truncation and stopping times, since g\mathsf gg is not C2C^2C2 at collisions.

Formalization scope

All objects are defined in one module RieszMF.Linear.Setting. Space is EuclideanSpace ℝ (Fin d) with Lebesgue measure; derivatives are iteratedFDeriv with operator norms, Δ\DeltaΔ is Mathlib's Laplacian, and the Fourier transform is Mathlib's ∫e−2πix⋅ξφ(x) dx\int e^{-2\pi ix\cdot\xi}\varphi(x)\,dx∫e−2πix⋅ξφ(x)dx. Assumption (iv) is read with a constant CkC_kCk​ for each kkk; (vi) and (1.17) as Fourier transforms paired with Schwartz functions vanishing near the origin; (iii) pointwise on the punctured ball; one radius r0r_0r0​ for (iii), (v), (vii), (viii); m≥1m\ge1m≥1 in (viii). Assumption (vii) for s>0s>0s>0 is printed as g(x)<csg(y)\mathsf g(x)<c_s\mathsf g(y)g(x)<cs​g(y) for ∣y∣≥2∣x∣|y|\ge2|x|∣y∣≥2∣x∣, which together with (ii) is unsatisfiable; it is stated as g(y)<csg(x)\mathsf g(y)<c_s\mathsf g(x)g(y)<cs​g(x).

The solution of (1.5) is a mild solution (3.1) in C([0,∞);L1∩L∞)C([0,\infty);L^1\cap L^\infty)C([0,∞);L1∩L∞) with continuity in the L1+L∞L^1+L^\inftyL1+L∞ norm, with each μt\mu^tμt a probability density; every integral of (3.1) is required to converge. Because the noise is additive, a solution of (1.1) is a measurable process whose paths almost surely are continuous, never collide, and satisfy the integral equation with the Brownian path; no stochastic integral appears. Brownian motions are Mathlib's IsBrownianReal coordinates, jointly independent over all N⋅dN\cdot dN⋅d coordinates. Expectations are lower Lebesgue integrals of nonnegative quantities, and Proposition 6.3 is stated in [0,∞][0,\infty][0,∞] with positive and negative parts on opposite sides, so no ∞−∞\infty-\infty∞−∞ and no default value of a divergent integral enters.

The constants follow the paper's dependency lists: β\betaβ depends on s,ds,ds,d only and is chosen before σ,M,g\sigma,\mathbb M,\mathsf gσ,M,g; CCC and N0N_0N0​ are chosen after σ,M,g,r0\sigma,\mathbb M,\mathsf g,r_0σ,M,g,r0​ and ∥μ0∥L∞\|\mu^0\|_{L^\infty}∥μ0∥L∞​, before the data, NNN, the configuration, the probability space and ttt. A formalization with constants chosen after μ0\mu^0μ0, NNN or the configuration, with N=0N=0N=0, with a junk-zero integral in FNF_NFN​ or in (3.1), or with an empty solution class does not count. Corollary 5.6 is posed exactly under its printed assumptions, although its proof for 0≤s≤d−40\le s\le d-40≤s≤d−4 uses the superharmonicity of −Δg-\Delta\mathsf g−Δg, which those assumptions do not list.

Contributions welcome: proofs of the milestones, a Lean development of Riesz potentials and of the sharp log-Sobolev inequality on Rd\mathbb R^dRd, and the well-posedness of the mean-field equation that the goal takes as a hypothesis (Proposition 3.1 and its global continuation, not posed here).

Selected references

  • M. Rosenzweig, S. Serfaty, Global-in-time mean-field convergence for singular Riesz-type diffusive flows, arXiv:2108.09878v1, 2021; Ann. Appl. Probab. 33(2), 2023. https://arxiv.org/abs/2108.09878
  • Q. H. Nguyen, M. Rosenzweig, S. Serfaty, Mean-field limits of Riesz-type singular flows with possible multiplicative transport noise, arXiv:2107.02592, 2021. https://arxiv.org/abs/2107.02592
  • S. Serfaty, Mean field limit for Coulomb-type flows, Duke Math. J. 169(15), 2887–2935, 2020 (appendix with M. Duerinckx). https://arxiv.org/abs/1803.08345
  • E. Carlen, M. Loss, Optimal smoothing and decay estimates for viscously damped conservation laws, with applications to the 2-D Navier–Stokes equation, Duke Math. J. 81(1), 1995. https://doi.org/10.1215/S0012-7094-95-08110-1
  • L. Gross, Logarithmic Sobolev inequalities, Amer. J. Math. 97(4), 1975. https://doi.org/10.2307/2373688
14 thms1 active userReviewed
Actuarial ScienceOperations Research·Captain: mikedeng1

Inf-Convolution, Optimal Allocations, and Model Uncertainty for Tail Risk Measures II: The Inf-Convolution of a VaR and a Monetary Tail Risk Measure, Its Optimal Allocation and Its Tail ParameterResearch Paper

Motivation

Banks and insurers hold capital against the risk of their positions, and regulators compute that capital with a risk measure, a functional that maps a random loss to a number. The two standard ones are the Value-at-Risk (VaR), a quantile of the loss, and the Expected Shortfall, an average of the upper quantiles (BCBS 2019). Both look only at the upper tail of the loss distribution. Liu and Wang (2021) called such functionals tail risk measures.

When a total loss XXX is split among several agents, each judging its share with its own risk measure, the smallest total capital achievable is the inf-convolution of the agents' risk measures, and a split attaining it is an optimal allocation. Embrechts, Liu and Wang (2018) computed the inf-convolution for VaR and Expected Shortfall agents. Liu, Mao, Wang and Wei (SSRN 3490348, version of January 16, 2022; published in Mathematics of Operations Research, 2022) extend this to agents with general tail risk measures. This mission formalizes their Theorem 2: one agent uses a VaR, the other an arbitrary monetary tail risk measure.

Setting

Let (Ω,F,P)(\Omega,\mathcal F,\mathbb P)(Ω,F,P) be an atomless probability space: every event of positive probability contains an event of strictly smaller positive probability. A random variable XXX is a measurable map Ω→R\Omega\to\mathbb RΩ→R, a positive value being a loss, and L0L^0L0 is the set of all of them. Its distribution function is FX(x)=P(X≤x)F_X(x)=\mathbb P(X\le x)FX​(x)=P(X≤x). For a level α∈(0,1)\alpha\in(0,1)α∈(0,1) the left and right VaR are

VaRαL(X)=inf⁡{x:FX(x)≥1−α},VaRαR(X)=inf⁡{x:FX(x)>1−α}.\mathrm{VaR}^L_\alpha(X)=\inf\{x: F_X(x)\ge 1-\alpha\},\qquad \mathrm{VaR}^R_\alpha(X)=\inf\{x: F_X(x)>1-\alpha\}.VaRαL​(X)=inf{x:FX​(x)≥1−α},VaRαR​(X)=inf{x:FX​(x)>1−α}.

A small α\alphaα is a high quantile. The quantile function is FX−1(p)=VaR1−pL(X)F_X^{-1}(p)=\mathrm{VaR}^L_{1-p}(X)FX−1​(p)=VaR1−pL​(X), and UXU_XUX​ denotes a random variable that is uniform on [0,1][0,1][0,1] with FX−1(UX)=XF^{-1}_X(U_X)=XFX−1​(UX​)=X almost surely.

A risk measure ρ\rhoρ is monotone if X≤YX\le YX≤Y a.s. implies ρ(X)≤ρ(Y)\rho(X)\le\rho(Y)ρ(X)≤ρ(Y). It is translation-invariant if ρ(X−m)=ρ(X)−m\rho(X-m)=\rho(X)-mρ(X−m)=ρ(X)−m for every constant mmm, and monetary if it is both. For p∈(0,1)p\in(0,1)p∈(0,1), the ppp-tail XpX_pXp​ of XXX has distribution function (FX(x)−(1−p))+/p(F_X(x)-(1-p))_+/p(FX​(x)−(1−p))+​/p (display (6) of the paper). A risk measure ρ\rhoρ is a ppp-tail risk measure if ρ(X)=ρ(Y)\rho(X)=\rho(Y)ρ(X)=ρ(Y) whenever XpX_pXp​ and YpY_pYp​ have the same law.

The allocations of XXX between two agents are the pairs (X1,X2)(X_1,X_2)(X1​,X2​) with X1+X2=XX_1+X_2=XX1​+X2​=X, and the inf-convolution is

ρ1 □ ρ2(X)=inf⁡{ρ1(X1)+ρ2(X2):X1+X2=X}∈[−∞,∞).\rho_1\,\square\,\rho_2(X)=\inf\{\rho_1(X_1)+\rho_2(X_2): X_1+X_2=X\}\in[-\infty,\infty).ρ1​□ρ2​(X)=inf{ρ1​(X1​)+ρ2​(X2​):X1​+X2​=X}∈[−∞,∞).

For ε∈(0,1)\varepsilon\in(0,1)ε∈(0,1) and α∈(0,1−ε)\alpha\in(0,1-\varepsilon)α∈(0,1−ε) the paper introduces the truncation

X[α]=X 1{UX≤1−α}+VaRα+εR(X) 1{UX>1−α}.X^{[\alpha]}=X\,\mathbb 1_{\{U_X\le 1-\alpha\}}+\mathrm{VaR}^R_{\alpha+\varepsilon}(X)\,\mathbb 1_{\{U_X>1-\alpha\}}.X[α]=X1{UX​≤1−α}​+VaRα+εR​(X)1{UX​>1−α}​.

Formalization targets

Goal: Theorem 2 (p. 13)

Let ε∈(0,1)\varepsilon\in(0,1)ε∈(0,1), let ρ\rhoρ be a monetary ε\varepsilonε-tail risk measure, and let α∈(0,1−ε)\alpha\in(0,1-\varepsilon)α∈(0,1−ε). For every X∈L0X\in L^0X∈L0 and every choice of UXU_XUX​:

VaRαL □ ρ(X)=ρ(X[α]);\mathrm{VaR}^L_\alpha\,\square\,\rho(X)=\rho\big(X^{[\alpha]}\big);VaRαL​□ρ(X)=ρ(X[α]);
  1. (X−X[α],X[α])(X-X^{[\alpha]},X^{[\alpha]})(X−X[α],X[α]) is an optimal allocation of XXX for (VaRαL,ρ)(\mathrm{VaR}^L_\alpha,\rho)(VaRαL​,ρ);
VaRαR □ ρ(X)=lim⁡δ↓0ρ(X[α−δ]);\mathrm{VaR}^R_\alpha\,\square\,\rho(X)=\lim_{\delta\downarrow0}\rho\big(X^{[\alpha-\delta]}\big);VaRαR​□ρ(X)=δ↓0lim​ρ(X[α−δ]);
  1. VaRαR □ ρ\mathrm{VaR}^R_\alpha\,\square\,\rhoVaRαR​□ρ and VaRαL □ ρ\mathrm{VaR}^L_\alpha\,\square\,\rhoVaRαL​□ρ are monetary (α+ε)(\alpha+\varepsilon)(α+ε)-tail risk measures.

Milestones

  • (17), p. 13. The distribution function of X[α]X^{[\alpha]}X[α] is FXF_XFX​ below VaRα+εR(X)\mathrm{VaR}^R_{\alpha+\varepsilon}(X)VaRα+εR​(X), equals FX+αF_X+\alphaFX​+α from there up to VaRαL(X)\mathrm{VaR}^L_\alpha(X)VaRαL​(X), and equals 111 from VaRαL(X)\mathrm{VaR}^L_\alpha(X)VaRαL​(X) on.
  • (18), p. 13. VaRεR(X[α])=VaRα+εR(X)\mathrm{VaR}^R_\varepsilon(X^{[\alpha]})=\mathrm{VaR}^R_{\alpha+\varepsilon}(X)VaRεR​(X[α])=VaRα+εR​(X).
  • Proof of (i), p. 14. If VaRαL(Y)=0\mathrm{VaR}^L_\alpha(Y)=0VaRαL​(Y)=0, then FX−Y(x)≤FX[α](x)F_{X-Y}(x)\le F_{X^{[\alpha]}}(x)FX−Y​(x)≤FX[α]​(x) for all x≥VaRα+εR(X)x\ge\mathrm{VaR}^R_{\alpha+\varepsilon}(X)x≥VaRα+εR​(X).
  • Proof of (iv), p. 15. If Yα+εY_{\alpha+\varepsilon}Yα+ε​ and Zα+εZ_{\alpha+\varepsilon}Zα+ε​ have the same law, then so do (Y[α−δ])ε(Y^{[\alpha-\delta]})_\varepsilon(Y[α−δ])ε​ and (Z[α−δ])ε(Z^{[\alpha-\delta]})_\varepsilon(Z[α−δ])ε​ for every δ∈[0,α)\delta\in[0,\alpha)δ∈[0,α).

Significance

Theorem 2 gives a closed form for the inf-convolution and an explicit optimal allocation. The VaR agent takes the loss above the (1−α)(1-\alpha)(1−α)-quantile, minus a constant, and the other agent keeps the rest, which has the same ε\varepsilonε-tail as the truncated X[α]X^{[\alpha]}X[α]. No convexity, law-invariance or distortion structure is assumed for ρ\rhoρ beyond its being a monetary tail risk measure. The theorem generalizes Theorem 5.3 of Wang and Wei (2020, Insurance: Mathematics and Economics 93), which required a distortion risk measure. Specialized to ρ=ESβ\rho=\mathrm{ES}_\betaρ=ESβ​, it gives VaRαL □ ESβ=RVaRα,β\mathrm{VaR}^L_\alpha\,\square\,\mathrm{ES}_\beta=\mathrm{RVaR}_{\alpha,\beta}VaRαL​□ESβ​=RVaRα,β​. Part 4 says that the tail parameters add: the representative agent of a VaR and an ε\varepsilonε-tail agent is an (α+ε)(\alpha+\varepsilon)(α+ε)-tail agent. This is the two-agent instance of the paper's Theorem 3. Parts 1 and 3 together contrast the left and right VaR: an optimal allocation always exists for VaRαL\mathrm{VaR}^L_\alphaVaRαL​, while for VaRαR\mathrm{VaR}^R_\alphaVaRαR​ the value is only a limit.

The result is proved in the paper. As far as a search of the platform shows, it has no machine-checked proof, and the platform has no development of quantile-based risk measures on a general atomless probability space. The mission produces the definitions of left and right VaR, tails, tail risk measures and inf-convolution on L0L^0L0, and a checked proof of the theorem.

Difficulty

The upper bound in part 1 is a single allocation. The lower bound must hold against every allocation (Y,X−Y)(Y,X-Y)(Y,X−Y). It reduces to comparing the distribution function of X−YX-YX−Y with that of X[α]X^{[\alpha]}X[α], but only on the ε\varepsilonε-tail. Turning that comparison into ρ(X[α])≤ρ(X−Y)\rho(X^{[\alpha]})\le\rho(X-Y)ρ(X[α])≤ρ(X−Y) needs both monotonicity and the tail property of ρ\rhoρ. Monotonicity is almost-sure and pointwise, the tail property is distributional, and no coupling between X−YX-YX−Y and X[α]X^{[\alpha]}X[α] is given, so the two cannot simply be chained. Part 3 needs an interchange of a limit and an infimum together with the one-sided continuity VaRα−δL→VaRαR\mathrm{VaR}^L_{\alpha-\delta}\to\mathrm{VaR}^R_\alphaVaRα−δL​→VaRαR​. Part 4 needs the law of the ε\varepsilonε-tail of X[α−δ]X^{[\alpha-\delta]}X[α−δ] to depend only on the (α+ε)(\alpha+\varepsilon)(α+ε)-tail of XXX. All of this rests on careful quantile bookkeeping: left versus right inverses, and strict versus weak inequalities at atoms of FXF_XFX​.

Formalization scope

Random variables are functions Ω→R\Omega\to\mathbb RΩ→R, and the domain is L0L^0L0, the measurable functions. The paper allows any convex cone containing L∞L^\inftyL∞ that is closed under equality in law and under taking tails, and fixing L0L^0L0 is one of the instances it names. The probability space is atomless, as §2.1 assumes throughout. FXF_XFX​, the VaRs and UXU_XUX​ are encoded as above. UXU_XUX​ is not chosen: parts 1–3 are asserted for every uniform UUU with FX−1(U)=XF^{-1}_X(U)=XFX−1​(U)=X a.s. The equality in law of tails is encoded through display (6), as equality of (FX−(1−p))+(F_X-(1-p))_+(FX​−(1−p))+​ and (FY−(1−p))+(F_Y-(1-p))_+(FY​−(1−p))+​. The risk measure ρ\rhoρ is real-valued; the paper allows the value −∞-\infty−∞, but the agents of §2.3 are real-valued. Monotonicity and translation invariance are required on L0L^0L0, with monotonicity for almost-sure inequalities.

Allocations sum pointwise. The inf-convolution takes values in the extended reals, so no junk value from a real infimum arises. Part 3 is a one-sided limit in the extended reals, which asserts both that the limit exists and its value. Part 4 uses extended-real versions of "monetary" and "tail risk measure". The level ranges ε∈(0,1)\varepsilon\in(0,1)ε∈(0,1) and α∈(0,1−ε)\alpha\in(0,1-\varepsilon)α∈(0,1−ε) are hypotheses.

Dropping the tail hypothesis on ρ\rhoρ makes part 1 false: on integrable losses the expectation is monetary but not a tail risk measure. A real-valued infimum in place of the extended one would make the inf-convolution a junk 000 when it is −∞-\infty−∞. A choice of UXU_XUX​ by Classical.choose would weaken parts 1–3 to one uniform. None of these is used.

A complete development needs quantile calculus for VaRL\mathrm{VaR}^LVaRL and VaRR\mathrm{VaR}^RVaRR (right-continuity of FXF_XFX​, the Galois connection FX−1(u)≤x  ⟺  u≤FX(x)F^{-1}_X(u)\le x\iff u\le F_X(x)FX−1​(u)≤x⟺u≤FX​(x)), the distribution of FX−1(U)F^{-1}_X(U)FX−1​(U), and basic facts on infima in EReal. The quantile lemmas are reusable for every other mission of this series and for any formalization of VaR. Proofs of the milestones, and of the folklore fact that inf-convolutions of monetary risk measures are monetary, are welcome as separate contributions.

Selected references

  • F. Liu, T. Mao, R. Wang, L. Wei, Inf-convolution, optimal allocations, and model uncertainty for tail risk measures, SSRN 3490348, version of January 16, 2022; Mathematics of Operations Research, 2022. https://ssrn.com/abstract=3490348
  • F. Liu, R. Wang, A theory for measures of tail risk, Mathematics of Operations Research 46(3), 1109–1128, 2021. https://doi.org/10.1287/moor.2020.1072
  • P. Embrechts, H. Liu, R. Wang, Quantile-based risk sharing, Operations Research 66(4), 936–949, 2018. https://doi.org/10.1287/opre.2017.1716
  • R. Wang, Y. Wei, Characterizing optimal allocations in quantile-based risk sharing, Insurance: Mathematics and Economics 93, 288–300, 2020. https://doi.org/10.1016/j.insmatheco.2020.05.001
  • H. Föllmer, A. Schied, Stochastic Finance: An Introduction in Discrete Time, 4th ed., De Gruyter, 2016. https://doi.org/10.1515/9783110463453
6 thms1 active userReviewed
Actuarial ScienceOperations Research·Captain: mikedeng1

Inf-Convolution, Optimal Allocations, and Model Uncertainty for Tail Risk Measures I: The Inf-Convolution of Left and Right VaRs Is a VaR at the Summed Level, with an Explicit Optimal AllocationResearch Paper

Motivation

Value-at-Risk (VaR) is the quantile-based capital requirement written into banking and insurance regulation (Basel, Solvency II). When several institutions, desks or subsidiaries each hold capital according to their own VaR, a natural question is how a total loss XXX should be split among them so that the aggregate capital is as small as possible. This is the risk sharing problem, and its value is the inf-convolution of the agents' risk measures.

For VaR the problem is non-convex, so the convex-analytic toolbox of risk sharing does not apply. Embrechts, Liu and Wang (2018) solved it when every agent uses the left quantile: the inf-convolution of left VaRs at levels α1,α2\alpha_1,\alpha_2α1​,α2​ is the left VaR at level α1+α2\alpha_1+\alpha_2α1​+α2​. The case of right quantiles, and of agents mixing the two versions, remained open. Liu, Mao, Wang and Wei close it in Theorem 1 of Inf-convolution, Optimal Allocations, and Model Uncertainty for Tail Risk Measures (Mathematics of Operations Research, 2022). This mission formalizes that theorem from the authors' SSRN version of 16 January 2022 (SSRN 3490348); all page and display numbers refer to that version.

Setting

Fix an atomless probability space (Ω,F,P)(\Omega,\mathcal F,\mathbb P)(Ω,F,P): every event of positive probability contains an event of strictly smaller positive probability. A random variable is a measurable X:Ω→RX:\Omega\to\mathbb RX:Ω→R; the set of all of them is L0L^0L0, and a positive value of XXX is a loss. Write FX(x)=P(X≤x)F_X(x)=\mathbb P(X\le x)FX​(x)=P(X≤x). For a level α∈(0,1)\alpha\in(0,1)α∈(0,1),

VaRαL(X)=inf⁡{x:FX(x)≥1−α},VaRαR(X)=inf⁡{x:FX(x)>1−α}\mathrm{VaR}^L_\alpha(X)=\inf\{x: F_X(x)\ge1-\alpha\},\qquad \mathrm{VaR}^R_\alpha(X)=\inf\{x: F_X(x)>1-\alpha\}VaRαL​(X)=inf{x:FX​(x)≥1−α},VaRαR​(X)=inf{x:FX​(x)>1−α}

are the left and right VaR. They coincide unless FXF_XFX​ is flat at height 1−α1-\alpha1−α, in which case VaRαL(X)<VaRαR(X)\mathrm{VaR}^L_\alpha(X)<\mathrm{VaR}^R_\alpha(X)VaRαL​(X)<VaRαR​(X). For Λ∈{L,R}\Lambda\in\{L,R\}Λ∈{L,R} write VaRαΛ\mathrm{VaR}^\Lambda_\alphaVaRαΛ​.

Given n≥1n\ge1n≥1 agents with risk measures ρ1,…,ρn\rho_1,\dots,\rho_nρ1​,…,ρn​, the allocations of XXX are An(X)={(X1,…,Xn)∈(L0)n:∑iXi=X}\mathbb A_n(X)=\{(X_1,\dots,X_n)\in(L^0)^n:\sum_iX_i=X\}An​(X)={(X1​,…,Xn​)∈(L0)n:∑i​Xi​=X} (display (4)), and the inf-convolution is

□i=1nρi(X)=inf⁡{∑i=1nρi(Xi):(X1,…,Xn)∈An(X)}∈[−∞,∞](5).\mathop{\square}_{i=1}^n\rho_i(X)=\inf\Big\{\sum_{i=1}^n\rho_i(X_i):(X_1,\dots,X_n)\in\mathbb A_n(X)\Big\}\in[-\infty,\infty]\qquad(5).□i=1n​ρi​(X)=inf{i=1∑n​ρi​(Xi​):(X1​,…,Xn​)∈An​(X)}∈[−∞,∞](5).

An allocation attaining the infimum is optimal. A partition (A1,…,An)(A_1,\dots,A_n)(A1​,…,An​) of Ω\OmegaΩ is a family of measurable, pairwise disjoint events covering Ω\OmegaΩ.

Formalization targets

Goal: Theorem 1 (p. 9)

Let α1,…,αn>0\alpha_1,\dots,\alpha_n>0α1​,…,αn​>0 with α=∑iαi<1\alpha=\sum_i\alpha_i<1α=∑i​αi​<1 and Λ1,…,Λn∈{L,R}\Lambda_1,\dots,\Lambda_n\in\{L,R\}Λ1​,…,Λn​∈{L,R}; put Λ=L\Lambda=LΛ=L if all Λi=L\Lambda_i=LΛi​=L and Λ=R\Lambda=RΛ=R otherwise. Then

  1. □i=1nVaRαiΛi=VaRαΛ\displaystyle\mathop{\square}_{i=1}^n\mathrm{VaR}^{\Lambda_i}_{\alpha_i}=\mathrm{VaR}^\Lambda_\alpha□i=1n​VaRαi​Λi​​=VaRαΛ​;
  2. for some partition (A1,…,An)(A_1,\dots,A_n)(A1​,…,An​) of Ω\OmegaΩ, the allocation
Xi=(X−VaRαΛ(X))1Ai+1nVaRαΛ(X),i=1,…,n,(9)X_i=(X-\mathrm{VaR}^\Lambda_\alpha(X))\mathbb 1_{A_i}+\tfrac1n\mathrm{VaR}^\Lambda_\alpha(X),\qquad i=1,\dots,n,\qquad(9)Xi​=(X−VaRαΛ​(X))1Ai​​+n1​VaRαΛ​(X),i=1,…,n,(9)

is optimal; 3. for every partition independent of XXX with P(Ai)=αi/α\mathbb P(A_i)=\alpha_i/\alphaP(Ai​)=αi​/α, the allocation (9) is optimal.

Milestones

  • (10)–(11), p. 9: VaRβR(Y)≤0\mathrm{VaR}^R_\beta(Y)\le0VaRβR​(Y)≤0 iff P(Y>ε)<β\mathbb P(Y>\varepsilon)<\betaP(Y>ε)<β for all ε>0\varepsilon>0ε>0; VaRβL(Y)≤0\mathrm{VaR}^L_\beta(Y)\le0VaRβL​(Y)≤0 iff P(Y>0)≤β\mathbb P(Y>0)\le\betaP(Y>0)≤β.
  • (12), p. 9: VaRα1−δL≥VaRα1R≥VaRα1L\mathrm{VaR}^L_{\alpha_1-\delta}\ge\mathrm{VaR}^R_{\alpha_1}\ge\mathrm{VaR}^L_{\alpha_1}VaRα1​−δL​≥VaRα1​R​≥VaRα1​L​ for δ∈(0,α1)\delta\in(0,\alpha_1)δ∈(0,α1​).
  • (7), p. 9: VaRα1L□VaRα2L=VaRα1+α2L\mathrm{VaR}^L_{\alpha_1}\square\mathrm{VaR}^L_{\alpha_2}=\mathrm{VaR}^L_{\alpha_1+\alpha_2}VaRα1​L​□VaRα2​L​=VaRα1​+α2​L​ (Embrechts et al. 2018, Corollary 2).
  • (14), p. 10: VaRα1R□VaRα2L=VaRα1+α2R\mathrm{VaR}^R_{\alpha_1}\square\mathrm{VaR}^L_{\alpha_2}=\mathrm{VaR}^R_{\alpha_1+\alpha_2}VaRα1​R​□VaRα2​L​=VaRα1​+α2​R​.
  • (8), p. 9: every two-agent pair containing a right VaR convolves to VaRα1+α2R\mathrm{VaR}^R_{\alpha_1+\alpha_2}VaRα1​+α2​R​.
  • The nnn-fold reduction, p. 10: □i=1n+1VaRαiΛi=(□i=1nVaRαiΛi)□VaRαn+1Λn+1\square_{i=1}^{n+1}\mathrm{VaR}^{\Lambda_i}_{\alpha_i}=\big(\square_{i=1}^{n}\mathrm{VaR}^{\Lambda_i}_{\alpha_i}\big)\square\mathrm{VaR}^{\Lambda_{n+1}}_{\alpha_{n+1}}□i=1n+1​VaRαi​Λi​​=(□i=1n​VaRαi​Λi​​)□VaRαn+1​Λn+1​​.
  • (15)–(16), p. 11: a partition for which each (X−VaRαR(X))1Ai(X-\mathrm{VaR}^R_\alpha(X))\mathbb 1_{A_i}(X−VaRαR​(X))1Ai​​ exceeds every ε>0\varepsilon>0ε>0 with probability below αi\alpha_iαi​, so its right VaR at level αi\alpha_iαi​ is non-positive.

Significance

The result. Theorem 1 gives a closed form for quantile-based risk sharing among any number of agents with any mix of left and right quantiles. The answer is asymmetric: one right-quantile agent turns the whole convolution into a right quantile. Part 2 shows that the minimum is attained, and the optimal allocation has a simple shape: every agent holds an equal share of the capital, and the excess loss over the aggregate quantile goes to one agent at a time, chosen by the partition. Part 3 gives that partition explicitly when an independent randomization is available. The formulas are the base case for the paper's later results on tail risk measures (Theorems 2–4), where a VaR agent shares risk with an agent using a general tail risk measure.

Formalizing it. The theorem is proved in the paper; no machine-checked proof of it, or of (7), is known. The mission produces a checked theory of left and right quantiles on a general probability space, of inf-convolution with values in [−∞,∞][-\infty,\infty][−∞,∞], and of the explicit optimal allocation. The milestone (7) is itself a published theorem of another paper, cited and not reproved here, so formalizing it is part of the work.

Difficulty

The known left-quantile result does not transfer to right quantiles. A right VaR is characterized by a family of strict tail bounds, P(Y>ε)<β\mathbb P(Y>\varepsilon)<\betaP(Y>ε)<β for every ε>0\varepsilon>0ε>0 (display (10)), while a left VaR needs one non-strict bound (display (11)). The events {X>VaRαR(X)+ε}\{X>\mathrm{VaR}^R_\alpha(X)+\varepsilon\}{X>VaRαR​(X)+ε} have probability strictly below α\alphaα but may approach α\alphaα as ε↓0\varepsilon\downarrow0ε↓0, so the split of the left-quantile case, chosen once for a fixed event, does not keep each agent's share strictly below its level. This is why the right and mixed cases were open, and why the existence part (ii) depends on the atomlessness of the space. Passing from two agents to nnn agents requires associativity of □\square□ when intermediate values may be −∞-\infty−∞.

Formalization scope

All objects are defined in one file, TailRiskSharing.VaRConv.Setting:

  • The space is a MeasureTheory.Measure with IsProbabilityMeasure and the classical atomless property, which is the paper's standing assumption (§2.1, p. 5). It is kept on every inf-convolution statement and on the goal, including part 3, where the paper notes (Remark 3) that it is not needed.
  • The paper allows any domain X\mathcal XX that is a law-invariant convex cone containing L∞L^\inftyL∞. The mission fixes X=L0\mathcal X=L^0X=L0, the measurable functions Ω→R\Omega\to\mathbb RΩ→R, on which VaR is defined (p. 6).
  • The allocation constraint ∑iXi=X\sum_iX_i=X∑i​Xi​=X holds at every ω\omegaω, not only almost surely.
  • The inf-convolution is an infimum in EReal, never a real sInf. Statements of the form □⋯=VaR(X)\square\cdots=\mathrm{VaR}(X)□⋯=VaR(X) assert equality with a coerced real number.
  • Levels satisfy αi>0\alpha_i>0αi​>0 and ∑iαi<1\sum_i\alpha_i<1∑i​αi​<1, and n≥1n\ge1n≥1. With some αi=0\alpha_i=0αi​=0 the statement would change, since VaR0R\mathrm{VaR}^R_0VaR0R​ is the essential supremum.
  • The aggregated side is LLL only if every agent uses LLL.
  • Partitions are measurable, pairwise disjoint and cover Ω\OmegaΩ; empty parts are allowed. Independence in part 3 is ProbabilityTheory.Indep between the σ-algebra generated by the partition and the σ-algebra generated by XXX.
  • The nnn-fold reduction is stated as one splitting step, (□i≤n) □ ρn+1(\square_{i\le n})\,\square\,\rho_{n+1}(□i≤n​)□ρn+1​, with the inner inf-convolution in [−∞,∞][-\infty,\infty][−∞,∞]; iterating it gives the paper's bracketing.

A trivializing formalization is excluded: the inf-convolution cannot collapse to a junk value of a real infimum, the levels cannot degenerate to 000, and the agent count cannot be 000.

A complete development needs the basic quantile calculus (monotonicity and left-continuity of β↦VaRβR\beta\mapsto\mathrm{VaR}^R_\betaβ↦VaRβR​, translation and scaling rules), a splitting lemma for atomless spaces, and the associativity of inf-convolution. These are reusable for the rest of this paper's series and for any quantile-based risk sharing result. Proofs of milestones and of the general lemmas are welcome as separate contributions.

Selected references

  • F. Liu, T. Mao, R. Wang, L. Wei, Inf-convolution, optimal allocations, and model uncertainty for tail risk measures, Mathematics of Operations Research, 2022. Version used: SSRN 3490348, 16 January 2022. https://ssrn.com/abstract=3490348 ; published version https://doi.org/10.1287/moor.2021.1217
  • P. Embrechts, H. Liu, R. Wang, Quantile-based risk sharing, Operations Research 66(4), 936–949, 2018. https://doi.org/10.1287/opre.2017.1716
  • P. Liu, R. Wang, L. Wei, Is the inf-convolution of law-invariant preferences law-invariant?, Insurance: Mathematics and Economics 91, 144–154, 2020.
  • L. Rüschendorf, Mathematical Risk Analysis. Dependence, Risk Bounds, Optimal Allocations and Portfolios, Springer, 2013. https://doi.org/10.1007/978-3-642-33590-7
  • F. Delbaen, Monetary Utility Functions, Osaka University Press, 2012.
9 thms1 active userReviewed
Optimal TransportOptimizationStatistics·Captain: mikedeng1

Bridging Bayesian and Minimax Mean Square Error Estimation via Wasserstein Distributionally Robust Optimization 1: The Gelbrich Minimax and Normal-Prior Maximin Values Coincide (Sandwich Theorem)Research Paper

Motivation

Estimating an unknown parameter from a noisy linear measurement is the basic problem behind Kalman filtering, signal recovery and linear inverse problems. A Bayesian statistician picks the estimator that minimizes the mean square error under a prior distribution; a minimax statistician guards against the worst prior in some family. The Bayesian answer depends on a prior that is rarely known, and the minimax answer is often intractable.

Nguyen, Shafieezadeh-Abadeh, Kuhn and Mohajerin Esfahani (Math. Oper. Res. 48(1), 2023) bridge the two by letting nature choose the joint distribution of parameter and noise from a ball of radius ρ\rhoρ around a nominal distribution in the type-2 Wasserstein distance. At ρ=0\rho=0ρ=0 the problem is Bayesian; as ρ\rhoρ grows it becomes more conservative. For a normal nominal distribution they show that this distributionally robust problem is solved by an affine estimator, that a normal distribution is least favourable, and that both are computable from a finite convex program. The paper extends the Wasserstein Kalman filter of Shafieezadeh-Abadeh et al. (NeurIPS 2018), which treated a joint normal ball on (x,y)(x,y)(x,y), to the model with independent parameter and noise.

Setting

The parameter is x∈Rnx\in\mathbb R^nx∈Rn and the observation is y=Hx+w∈Rmy=Hx+w\in\mathbb R^my=Hx+w∈Rm, with a known observation matrix H∈Rm×nH\in\mathbb R^{m\times n}H∈Rm×n and noise www independent of xxx. Write M(Rd)\mathcal M(\mathbb R^d)M(Rd) for the probability distributions on Rd\mathbb R^dRd with finite second moments.

  • An estimator is a measurable ψ:Rm→Rn\psi:\mathbb R^m\to\mathbb R^nψ:Rm→Rn with ∥ψ(y)∥≤C(1+∥y∥)\|\psi(y)\|\le C(1+\|y\|)∥ψ(y)∥≤C(1+∥y∥) for some C>0C>0C>0; the family of estimators is F\mathcal FF, and A⊆F\mathcal A\subseteq\mathcal FA⊆F is the family of affine estimators ψ(y)=Ay+b\psi(y)=Ay+bψ(y)=Ay+b.
  • The average risk of ψ\psiψ under a joint distribution Q\mathbb QQ of (x,w)(x,w)(x,w) is R(ψ,Q)=EQ[∥x−ψ(Hx+w)∥2]\mathcal R(\psi,\mathbb Q)=\mathbb E_{\mathbb Q}[\|x-\psi(Hx+w)\|^2]R(ψ,Q)=EQ​[∥x−ψ(Hx+w)∥2].
  • The type-2 Wasserstein distance W(Q1,Q2)\mathbb W(\mathbb Q_1,\mathbb Q_2)W(Q1​,Q2​) is the square root of the least expected squared distance E∥ξ1−ξ2∥2\mathbb E\|\xi_1-\xi_2\|^2E∥ξ1​−ξ2​∥2 over all couplings of Q1\mathbb Q_1Q1​ and Q2\mathbb Q_2Q2​.
  • The Gelbrich distance between mean–covariance pairs is
G((μ1,Σ1),(μ2,Σ2))=∥μ1−μ2∥2+Tr⁡[Σ1+Σ2−2(Σ21/2Σ1Σ21/2)1/2].\mathbb G((\mu_1,\Sigma_1),(\mu_2,\Sigma_2))=\sqrt{\|\mu_1-\mu_2\|^2+\operatorname{Tr}\big[\Sigma_1+\Sigma_2-2(\Sigma_2^{1/2}\Sigma_1\Sigma_2^{1/2})^{1/2}\big]}.G((μ1​,Σ1​),(μ2​,Σ2​))=∥μ1​−μ2​∥2+Tr[Σ1​+Σ2​−2(Σ21/2​Σ1​Σ21/2​)1/2]​.

The nominal distribution is a product P^=P^x×P^w\widehat{\mathbb P}=\widehat{\mathbb P}_x\times\widehat{\mathbb P}_wP=Px​×Pw​ with radii ρx,ρw≥0\rho_x,\rho_w\ge0ρx​,ρw​≥0. Three ambiguity sets of product distributions Qx×Qw\mathbb Q_x\times\mathbb Q_wQx​×Qw​ appear:

  • the Wasserstein ball B(P^)\mathbb B(\widehat{\mathbb P})B(P): Qx∈M(Rn)\mathbb Q_x\in\mathcal M(\mathbb R^n)Qx​∈M(Rn), Qw∈M(Rm)\mathbb Q_w\in\mathcal M(\mathbb R^m)Qw​∈M(Rm), W(Qx,P^x)≤ρx\mathbb W(\mathbb Q_x,\widehat{\mathbb P}_x)\le\rho_xW(Qx​,Px​)≤ρx​, W(Qw,P^w)≤ρw\mathbb W(\mathbb Q_w,\widehat{\mathbb P}_w)\le\rho_wW(Qw​,Pw​)≤ρw​;
  • the Gelbrich ball G(P^)\mathbb G(\widehat{\mathbb P})G(P): the marginals' mean–covariance pairs lie within Gelbrich distance ρx\rho_xρx​, ρw\rho_wρw​ of those of P^x\widehat{\mathbb P}_xPx​, P^w\widehat{\mathbb P}_wPw​;
  • for normal P^x=N(μ^x,Σ^x)\widehat{\mathbb P}_x=\mathcal N(\widehat\mu_x,\widehat\Sigma_x)Px​=N(μ​x​,Σx​), P^w=N(μ^w,Σ^w)\widehat{\mathbb P}_w=\mathcal N(\widehat\mu_w,\widehat\Sigma_w)Pw​=N(μ​w​,Σw​), the normal ball BN(P^)\mathbb B_{\mathcal N}(\widehat{\mathbb P})BN​(P): products N(μ^x,Σx)×N(μ^w,Σw)\mathcal N(\widehat\mu_x,\Sigma_x)\times\mathcal N(\widehat\mu_w,\Sigma_w)N(μ​x​,Σx​)×N(μ​w​,Σw​) in B(P^)\mathbb B(\widehat{\mathbb P})B(P) with Σx⪰0\Sigma_x\succeq0Σx​⪰0, Σw≻0\Sigma_w\succ0Σw​≻0.

Then BN(P^)⊆B(P^)⊆G(P^)\mathbb B_{\mathcal N}(\widehat{\mathbb P})\subseteq\mathbb B(\widehat{\mathbb P})\subseteq\mathbb G(\widehat{\mathbb P})BN​(P)⊆B(P)⊆G(P).

Formalization targets

Goal: Theorem 4.1 (sandwich theorem)

For P^\widehat{\mathbb P}P normal with Σ^w≻0\widehat\Sigma_w\succ0Σw​≻0,

inf⁡ψ∈A sup⁡Q∈G(P^) R(ψ,Q)=sup⁡Q∈BN(P^) inf⁡ψ∈F R(ψ,Q).\inf_{\psi\in\mathcal A}\ \sup_{\mathbb Q\in\mathbb G(\widehat{\mathbb P})}\ \mathcal R(\psi,\mathbb Q)=\sup_{\mathbb Q\in\mathbb B_{\mathcal N}(\widehat{\mathbb P})}\ \inf_{\psi\in\mathcal F}\ \mathcal R(\psi,\mathbb Q).ψ∈Ainf​ Q∈G(P)sup​ R(ψ,Q)=Q∈BN​(P)sup​ ψ∈Finf​ R(ψ,Q).

The left side is the Gelbrich MMSE problem (2.1), the right side the dual problem over normal priors (3.3). The Wasserstein problem (1.7), inf⁡Fsup⁡B\inf_{\mathcal F}\sup_{\mathbb B}infF​supB​, and its dual (3.1), sup⁡Binf⁡F\sup_{\mathbb B}\inf_{\mathcal F}supB​infF​, lie between them, so all four values coincide.

Milestones

  1. Proposition A.3: the closed form of sup⁡Σ⪰0⟨D,Σ⟩−γTr⁡[Σ−2(Σ^1/2ΣΣ^1/2)1/2]\sup_{\Sigma\succeq0}\langle D,\Sigma\rangle-\gamma\operatorname{Tr}[\Sigma-2(\widehat\Sigma^{1/2}\Sigma\widehat\Sigma^{1/2})^{1/2}]supΣ⪰0​⟨D,Σ⟩−γTr[Σ−2(Σ1/2ΣΣ1/2)1/2] and its maximizer.
  2. Theorem 2.7: problem (2.1) equals a finite convex program (2.2), and for ρx,ρw>0\rho_x,\rho_w>0ρx​,ρw​>0 an affine estimator attains it.
  3. Lemma A.6: the covariance Gelbrich set S\mathcal SS is convex and compact, with Tr⁡[Σ]≤(ρ+Tr⁡[Σ^]1/2)2\operatorname{Tr}[\Sigma]\le(\rho+\operatorname{Tr}[\widehat\Sigma]^{1/2})^2Tr[Σ]≤(ρ+Tr[Σ]1/2)2.
  4. Proposition A.4 (i): max⁡Σ∈S⟨D,Σ⟩\max_{\Sigma\in\mathcal S}\langle D,\Sigma\ranglemaxΣ∈S​⟨D,Σ⟩ equals a one-dimensional infimum (A.6).
  5. Lemma A.5: a max–min problem over S\mathcal SS has a maximizer Σ⋆⪰λmin⁡(Σ^)I\Sigma^\star\succeq\lambda_{\min}(\widehat\Sigma)IΣ⋆⪰λmin​(Σ)I.
  6. Display (3.5): over normal priors, affine estimators suffice.
  7. Display (3.8): for Σ^w≻0\widehat\Sigma_w\succ0Σw​≻0 the value of (3.3) is a finite sup–inf over covariance matrices.

Companion statements, not milestones: Theorem 3.5 (problem (3.3) equals the finite program (3.4)), Corollary 4.2 (the optimal affine estimator and the optimal normal prior form a Nash equilibrium on B(P^)\mathbb B(\widehat{\mathbb P})B(P)) and Corollary 4.3 (the minimax estimator from a maximizer of (3.4)).

Significance

The sandwich theorem says that robustifying the MMSE problem against Wasserstein uncertainty around a normal distribution costs nothing in tractability: the optimal estimator is affine, the worst-case distribution is normal, and both follow from a convex program in the covariance matrices. The resulting estimator is a Kalman-type update with a least favourable covariance in place of the nominal one, which is what makes the result usable in filtering. The paper also extends it to elliptical nominal distributions (Section 5) and develops a Frank–Wolfe method for (3.4) (Section 6).

The results are proved in the paper; none of them is machine-checked. A formalization adds a checked account of a minimax argument that combines optimal transport (the Gelbrich bound and its equality for normal laws), nonlinear semidefinite programming (Propositions A.3 and A.4) and Bayesian estimation for Gaussian laws. The appendix statements are general facts about the Gelbrich ball of covariance matrices and serve beyond this paper.

Difficulty

The left side of the goal is an infimum over affine estimators of a supremum over an infinite-dimensional set of distributions; the right side reverses the order and restricts nature to normal laws while allowing every estimator. Weak duality gives ≥\ge≥ at once. For the reverse inequality, Sion's minimax theorem cannot be applied to the distributions directly: BN(P^)\mathbb B_{\mathcal N}(\widehat{\mathbb P})BN​(P) is not convex, since mixtures of normal laws are not normal, and G(P^)\mathbb G(\widehat{\mathbb P})G(P) is not compact in any useful topology. The equality has to pass through the finite matrix problems, where the worst covariance matrices must be shown to stay bounded away from singularity so that the normal noise covariance remains positive definite. That is the role of Lemma A.5 and of the hypothesis Σ^w≻0\widehat\Sigma_w\succ0Σw​≻0.

Formalization scope

  • Rd\mathbb R^dRd is EuclideanSpace ℝ (Fin d). The risk is a lower Lebesgue integral in [0,∞][0,\infty][0,∞], and every optimal value is a [0,∞][0,\infty][0,∞]- or EReal-valued infimum or supremum restricted by a membership predicate. The finite programs (2.2), (3.4), (A.3), (A.5), (A.6), (A.9) take values in EReal.
  • M(Rd)\mathcal M(\mathbb R^d)M(Rd) is "probability measure with E∥ξ∥2<∞\mathbb E\|\xi\|^2<\inftyE∥ξ∥2<∞" and is required wherever the page requires it, because means and covariances are Bochner integrals with junk value 000 otherwise.
  • F\mathcal FF keeps both measurability and linear growth.
  • N(μ,Σ)\mathcal N(\mu,\Sigma)N(μ,Σ) is Mathlib's multivariateGaussian. W\mathbb WW, means, covariances, the PSD square root and the Gelbrich ball are the published definitions WassersteinDRO.Gelbrich.{wassersteinDistance, meanVector, covarianceMatrix, psdSqrt, meanCovarianceUncertaintySet}.
  • λmin⁡\lambda_{\min}λmin​ and λmax⁡\lambda_{\max}λmax​ are 000 at dimension 000, where every statement reduces to a trivial one.
  • Added hypothesis. The goal assumes Σ^w≻0\widehat\Sigma_w\succ0Σw​≻0, which the page omits. The theorem fails without it: for n=m=1n=m=1n=m=1, H=0H=0H=0, Σ^x=1\widehat\Sigma_x=1Σx​=1, Σ^w=0\widehat\Sigma_w=0Σw​=0 and ρx=ρw=0\rho_x=\rho_w=0ρx​=ρw​=0, the normal ball is empty while the left side equals 111. The page's own proof uses (3.8), which it derives only for Σ^w≻0\widehat\Sigma_w\succ0Σw​≻0.
  • Theorem 3.5. The bounds Σ⋆⪰λmin⁡(Σ^)I\Sigma^\star\succeq\lambda_{\min}(\widehat\Sigma)IΣ⋆⪰λmin​(Σ)I are stated for some maximizer of (3.4), not for every maximizer.
  • Omitted. Proposition A.4 (ii)–(iii) is false as printed in two places and is not posed (see the mission notes). Propositions 2.4 and 3.4 (the Gelbrich bound and its equality for normal laws) are already on the platform and are not posed again.
  • Ruled out. A formalization in which BN(P^)\mathbb B_{\mathcal N}(\widehat{\mathbb P})BN​(P) is empty, the risk is a Bochner integral, or the optimal values are real sSup/sInf would make the goal trivial or false. A sanity check exhibits a member of BN(P^)\mathbb B_{\mathcal N}(\widehat{\mathbb P})BN​(P) whenever Σ^w≻0\widehat\Sigma_w\succ0Σw​≻0, and an affine estimator.

Proofs of any milestone are welcome, and so are reusable lemmas on the Gelbrich ball, the matrix square root and Gaussian conditioning.

Selected references

  • V. A. Nguyen, S. Shafieezadeh-Abadeh, D. Kuhn, P. Mohajerin Esfahani, Bridging Bayesian and minimax mean square error estimation via Wasserstein distributionally robust optimization, Math. Oper. Res. 48(1), 2023. https://arxiv.org/abs/1911.03539v2, https://doi.org/10.1287/moor.2021.1176
  • S. Shafieezadeh-Abadeh, V. A. Nguyen, D. Kuhn, P. Mohajerin Esfahani, Wasserstein distributionally robust Kalman filtering, NeurIPS 31, 2018. https://arxiv.org/abs/1809.08830
  • M. Gelbrich, On a formula for the L2 Wasserstein metric between measures on Euclidean and Hilbert spaces, Math. Nachr. 147, 1990. https://doi.org/10.1002/mana.19901470121
  • C. R. Givens, R. M. Shortt, A class of Wasserstein metrics for probability distributions, Michigan Math. J. 31(2), 1984. https://doi.org/10.1307/mmj/1029003026
  • M. Sion, On general minimax theorems, Pacific J. Math. 8, 1958. https://doi.org/10.2140/pjm.1958.8.171
  • V. A. Nguyen, D. Kuhn, P. Mohajerin Esfahani, Distributionally robust inverse covariance estimation: the Wasserstein shrinkage estimator, Oper. Res. 70(1), 2022. https://arxiv.org/abs/1805.07194
15 thms1 active userReviewed
Operations ResearchPartial Differential EquationsStochastic Systems·Captain: mikedeng1

Diffusion Approximation for Fully Coupled Stochastic Differential Equations 2: The Slow Component Converges Weakly to the Averaged SDE at an Explicit Rate (Regime 1)Research Paper

Motivation

Many stochastic models evolve on two time scales: a fast component that equilibrates quickly and a slow component driven by it. Planetary motion, climate models, geophysical flows and intracellular reaction networks are standard examples, and multiscale numerical methods for them approximate the slow component by a reduced equation. Such approximations are only useful with an error bound, and for the fully coupled case, where the fast and slow equations each depend on both components and the slow equation contains a large fluctuating drift, rates of convergence were missing.

Röckner and Xie (Ann. Probab. 49 (2021) 1205–1236) treat this case with only Hölder continuous coefficients and obtain explicit rates in four regimes. This mission poses the first regime, the classical averaging principle.

Timeline. Papanicolaou, Stroock and Varadhan (1977) proved the diffusion approximation for αε=βε=γε\alpha_\varepsilon=\beta_\varepsilon=\gamma_\varepsilonαε​=βε​=γε​ with a compact fast state space and time-independent coefficients. Pardoux and Veretennikov (2001, 2003, 2005) treated a noncompact homogeneous case with c≡0c\equiv0c≡0 and αε=γε\alpha_\varepsilon=\gamma_\varepsilonαε​=γε​ by martingale problems and the Poisson equation. Rates for the averaging principle when c=H≡0c=H\equiv0c=H≡0 were obtained by several authors, e.g. Givon (2007) and Liu (2010). Röckner and Xie (2021) give rates for the fully coupled system with Hölder coefficients in all four regimes.

Setting

Let x∈Rd1x\in\mathbb R^{d_1}x∈Rd1​ (fast) and y∈Rd2y\in\mathbb R^{d_2}y∈Rd2​ (slow), d1,d2≥1d_1,d_2\ge1d1​,d2​≥1. For small parameters αε,βε,γε>0\alpha_\varepsilon,\beta_\varepsilon,\gamma_\varepsilon>0αε​,βε​,γε​>0 tending to 000, with αε2/βε→0\alpha_\varepsilon^2/\beta_\varepsilon\to0αε2​/βε​→0, the two-scale system (1.4) is

dXtε=αε−2b(Xtε,Ytε)dt+βε−1c(Xtε,Ytε)dt+αε−1σ(Xtε,Ytε)dWt1,dX^\varepsilon_t=\alpha_\varepsilon^{-2}b(X^\varepsilon_t,Y^\varepsilon_t)dt+\beta_\varepsilon^{-1}c(X^\varepsilon_t,Y^\varepsilon_t)dt+\alpha_\varepsilon^{-1}\sigma(X^\varepsilon_t,Y^\varepsilon_t)dW^1_t,dXtε​=αε−2​b(Xtε​,Ytε​)dt+βε−1​c(Xtε​,Ytε​)dt+αε−1​σ(Xtε​,Ytε​)dWt1​, dYtε=F(t,Xtε,Ytε)dt+γε−1H(t,Xtε,Ytε)dt+G(t,Xtε,Ytε)dWt2,dY^\varepsilon_t=F(t,X^\varepsilon_t,Y^\varepsilon_t)dt+\gamma_\varepsilon^{-1}H(t,X^\varepsilon_t,Y^\varepsilon_t)dt+G(t,X^\varepsilon_t,Y^\varepsilon_t)dW^2_t,dYtε​=F(t,Xtε​,Ytε​)dt+γε−1​H(t,Xtε​,Ytε​)dt+G(t,Xtε​,Ytε​)dWt2​,

with X0ε=xX^\varepsilon_0=xX0ε​=x, Y0ε=yY^\varepsilon_0=yY0ε​=y and independent Brownian motions W1,W2W^1,W^2W1,W2.

Freezing yyy gives the frozen equation dXty=b(Xty,y)dt+σ(Xty,y)dWt1dX^y_t=b(X^y_t,y)dt+\sigma(X^y_t,y)dW^1_tdXty​=b(Xty​,y)dt+σ(Xty​,y)dWt1​ (1.7), with generator L0(x,y)=∑ijaij∂xixj2+∑ibi∂xi\mathscr L_0(x,y)=\sum_{ij}a^{ij}\partial^2_{x_ix_j}+\sum_ib^i\partial_{x_i}L0​(x,y)=∑ij​aij∂xi​xj​2​+∑i​bi∂xi​​, a=σσ∗/2a=\sigma\sigma^*/2a=σσ∗/2, and invariant probability measure μy\mu^yμy. A function is centered if its μy\mu^yμy-average vanishes for every yyy. The standing assumptions are: (Aσ), uniform nondegeneracy of σσ∗\sigma\sigma^*σσ∗; (Ab), lim⁡∣x∣→∞sup⁡y⟨x,b(x,y)⟩=−∞\lim_{|x|\to\infty}\sup_y\langle x,b(x,y)\rangle=-\inftylim∣x∣→∞​supy​⟨x,b(x,y)⟩=−∞; (AG), uniform nondegeneracy of GG∗GG^*GG∗; (AH), HHH is centered; and (2.7), the analogue of (Ab) for b+κcb+\kappa cb+κc, κ>0\kappa>0κ>0 small.

The averaged coefficients of Regime 1 are F^1(t,y)=∫F(t,x,y)μy(dx)\hat F_1(t,y)=\int F(t,x,y)\mu^y(dx)F^1​(t,y)=∫F(t,x,y)μy(dx) and G^1=(∫GG∗(t,x,y)μy(dx))1/2\hat G_1=(\int GG^*(t,x,y)\mu^y(dx))^{1/2}G^1​=(∫GG∗(t,x,y)μy(dx))1/2, and the averaged equation (2.9) is dY^t1=F^1(t,Y^t1)dt+G^1(t,Y^t1)dWt2d\hat Y^1_t=\hat F_1(t,\hat Y^1_t)dt+\hat G_1(t,\hat Y^1_t)dW^2_tdY^t1​=F^1​(t,Y^t1​)dt+G^1​(t,Y^t1​)dWt2​, Y^01=y\hat Y^1_0=yY^01​=y.

Regularity is measured in weighted Hölder classes: Cpδ,ϑC^{\delta,\vartheta}_pCpδ,ϑ​ (index δ\deltaδ in xxx, ϑ\varthetaϑ in yyy, polynomial growth in xxx), its bounded version Cbδ,ϑC^{\delta,\vartheta}_bCbδ,ϑ​, the time-dependent Cpϑ/2,δ,ϑC^{\vartheta/2,\delta,\vartheta}_pCpϑ/2,δ,ϑ​, the class Lp∞L^\infty_pLp∞​ of measurable functions of polynomial growth, and Cb2+ϑ(Rd2)C^{2+\vartheta}_b(\mathbb R^{d_2})Cb2+ϑ​(Rd2​) for test functions.

Formalization targets

Goal: Theorem 2.3(i)

In Regime 1, αε/γε→0\alpha_\varepsilon/\gamma_\varepsilon\to0αε​/γε​→0 and αε2/(βεγε)→0\alpha_\varepsilon^2/(\beta_\varepsilon\gamma_\varepsilon)\to0αε2​/(βε​γε​)→0, assume b,σ∈Cbδ,ϑb,\sigma\in C^{\delta,\vartheta}_bb,σ∈Cbδ,ϑ​, F,H,G∈Cpϑ/2,δ,ϑF,H,G\in C^{\vartheta/2,\delta,\vartheta}_pF,H,G∈Cpϑ/2,δ,ϑ​ with ϑ∈(0,2]\vartheta\in(0,2]ϑ∈(0,2], c∈Lp∞c\in L^\infty_pc∈Lp∞​ and αεϑ/γε→0\alpha_\varepsilon^\vartheta/\gamma_\varepsilon\to0αεϑ​/γε​→0. Then for every T>0T>0T>0 and φ∈Cb2+ϑ\varphi\in C^{2+\vartheta}_bφ∈Cb2+ϑ​,

sup⁡t∈[0,T]∣Eφ(Ytε)−Eφ(Y^t1)∣≤CT(αεϑγε+αε2γε2+αε2βεγε).\sup_{t\in[0,T]}\big|\mathbb E\varphi(Y^\varepsilon_t)-\mathbb E\varphi(\hat Y^1_t)\big|\le C_T\Big(\frac{\alpha_\varepsilon^\vartheta}{\gamma_\varepsilon}+\frac{\alpha_\varepsilon^2}{\gamma_\varepsilon^2}+\frac{\alpha_\varepsilon^2}{\beta_\varepsilon\gamma_\varepsilon}\Big).t∈[0,T]sup​​Eφ(Ytε​)−Eφ(Y^t1​)​≤CT​(γε​αεϑ​​+γε2​αε2​​+βε​γε​αε2​​).

Milestones

  1. Lemma 4.1: mollification in (t,y)(t,y)(t,y) at scales n−2n^{-2}n−2 and n−1n^{-1}n−1 costs n−ϑn^{-\vartheta}n−ϑ and gains n2−ϑn^{2-\vartheta}n2−ϑ on ∂t\partial_t∂t​ and ∇y2\nabla_y^2∇y2​.
  2. Lemma 4.2: for centered fff, E∫0tf(s,Xsε,Ysε)ds≤Ct(αεϑ+αεϑ∧1αε/γε+αε2/βε)\mathbb E\int_0^tf(s,X^\varepsilon_s,Y^\varepsilon_s)ds\le C_t(\alpha_\varepsilon^\vartheta+\alpha_\varepsilon^{\vartheta\wedge1}\alpha_\varepsilon/\gamma_\varepsilon+\alpha_\varepsilon^2/\beta_\varepsilon)E∫0t​f(s,Xsε​,Ysε​)ds≤Ct​(αεϑ​+αεϑ∧1​αε​/γε​+αε2​/βε​).
  3. Lemma 4.4(i): in Regime 1, E γε−1∫0tf ds≤Ct(αεϑ/γε+αε2/γε2+αε2/(βεγε))\mathbb E\,\gamma_\varepsilon^{-1}\int_0^tf\,ds\le C_t(\alpha_\varepsilon^\vartheta/\gamma_\varepsilon+\alpha_\varepsilon^2/\gamma_\varepsilon^2+\alpha_\varepsilon^2/(\beta_\varepsilon\gamma_\varepsilon))Eγε−1​∫0t​fds≤Ct​(αεϑ​/γε​+αε2​/γε2​+αε2​/(βε​γε​)).
  4. Lemma 5.1 (k=1k=1k=1): G^1=G^1G^1∗/2\hat{\mathcal G}_1=\hat G_1\hat G_1^*/2G^​1​=G^1​G^1∗​/2 is uniformly nondegenerate and F^1,G^1∈Cbϑ/2,ϑ\hat F_1,\hat{\mathcal G}_1\in C^{\vartheta/2,\vartheta}_bF^1​,G^​1​∈Cbϑ/2,ϑ​.

Significance

The result. The theorem gives a rate for the weak error of the averaging principle under Hölder regularity only, and the rate depends only on the regularity in the slow variable yyy, not on δ\deltaδ. It covers the fully coupled case, in which the slow equation contains the singular drift γε−1H\gamma_\varepsilon^{-1}Hγε−1​H, and it identifies the extra condition αεϑ/γε→0\alpha_\varepsilon^\vartheta/\gamma_\varepsilon\to0αεϑ​/γε​→0 under which this drift averages out. For c=H≡0c=H\equiv0c=H≡0 it reduces to the rate αεϑ\alpha_\varepsilon^\varthetaαεϑ​ (Remark 2.5), optimal for ϑ=2\vartheta=2ϑ=2. Error bounds of this kind are what heterogeneous multiscale methods need to be quantitative.

Formalizing it. The result is proved in the paper; nothing in it is machine-checked. A formalization needs an Itô calculus for weak solutions, the elliptic Poisson equation in xxx with parameter yyy (the companion mission's Theorem 2.1), and the parabolic Kolmogorov equation of the averaged SDE. The two fluctuation lemmas are stated for general centered functions and are reusable for other limit theorems.

Difficulty

The obvious argument writes Eφ(YTε)−Eφ(Y^T1)\mathbb E\varphi(Y^\varepsilon_T)-\mathbb E\varphi(\hat Y^1_T)Eφ(YTε​)−Eφ(Y^T1​) through the Kolmogorov equation of the averaged SDE and Itô's formula, which leaves E∫0T(L1−L^1)u~ ds\mathbb E\int_0^T(\mathscr L_1-\hat{\mathscr L}_1)\tilde u\,dsE∫0T​(L1​−L^1​)u~ds and γε−1E∫0TH⋅∇yu~ ds\gamma_\varepsilon^{-1}\mathbb E\int_0^TH\cdot\nabla_y\tilde u\,dsγε−1​E∫0T​H⋅∇y​u~ds. Both integrands are centered but only Hölder continuous in (t,y)(t,y)(t,y), so the standard corrector argument, which applies Itô's formula to the solution of the Poisson equation along (Xε,Yε)(X^\varepsilon,Y^\varepsilon)(Xε,Yε), needs derivatives in ttt and yyy that do not exist. The term with γε−1\gamma_\varepsilon^{-1}γε−1​ is a central-limit-scale fluctuation, which is why its control needs the regime conditions. Existence and regularity of the averaged coefficients also have to come from the Poisson equation, since μy\mu^yμy is not explicit.

Formalization scope

Spaces are EuclideanSpace ℝ (Fin d). Coefficients are b,c:Rd1×Rd2→Rd1b,c:\mathbb R^{d_1}\times\mathbb R^{d_2}\to\mathbb R^{d_1}b,c:Rd1​×Rd2​→Rd1​, σ\sigmaσ a d1×d1d_1\times d_1d1​×d1​ matrix, F,H:R+×Rd1×Rd2→Rd2F,H:\mathbb R_+\times\mathbb R^{d_1}\times\mathbb R^{d_2}\to\mathbb R^{d_2}F,H:R+​×Rd1​×Rd2​→Rd2​, GGG a d2×d2d_2\times d_2d2​×d2​ matrix. The Itô layer (Brownian motion, Itô integral, Itô processes, SDE solutions) is the published Peng1990.SMP.Stochastic. A solution of (1.4) is a weak solution: any filtered probability space, any Brownian motion for that filtration, any progressive process satisfying the integral equation. The theorem compares every weak solution of (1.4) with every weak solution of (2.9). The paper does not prove existence for (1.4), so the statement is conditional on solutions, as it is used in the paper. A sanity file shows the parameter hypotheses are consistent (αε=ε\alpha_\varepsilon=\varepsilonαε​=ε, βε=γε=ε1/2\beta_\varepsilon=\gamma_\varepsilon=\varepsilon^{1/2}βε​=γε​=ε1/2) and that (2.7) holds for a bounded dissipative drift.

Committed conventions and readings of the page:

  • μy\mu^yμy is any family of probability measures that are infinitesimally invariant for L0\mathscr L_0L0​ with a=σσ∗/2a=\sigma\sigma^*/2a=σσ∗/2, the generator of (1.7). Under the standing assumptions this is the unique invariant measure.
  • (2.7) is read with κ=αε2/βε\kappa=\alpha_\varepsilon^2/\beta_\varepsilonκ=αε2​/βε​, the factor that actually multiplies ccc in the fast drift, for all small κ>0\kappa>0κ>0.
  • G^1\hat G_1G^1​ is any measurable matrix field with G^1G^1∗=∫GG∗ dμy\hat G_1\hat G_1^*=\int GG^*\,d\mu^yG^1​G^1∗​=∫GG∗dμy, which covers the printed square root.
  • The Hölder classes include polynomial growth, constrain all lower yyy-derivatives, and take their constants uniformly in time, including in the time-Hölder clause.
  • The constants CTC_TCT​, CtC_tCt​, C0C_0C0​ exist after all data are fixed (coefficients, μ\muμ, TTT, φ\varphiφ, xxx, yyy, the parameter functions) and before ε\varepsilonε or nnn.
  • Lemma 4.1 takes fff on all of R\mathbb RR in time, since (4.1) evaluates fff at negative times, and assumes the kernels have the finite moments ∫∣r∣ϑ/2ρ1<∞\int|r|^{\vartheta/2}\rho_1<\infty∫∣r∣ϑ/2ρ1​<∞, ∫∣z∣ϑρ2<∞\int|z|^\vartheta\rho_2<\infty∫∣z∣ϑρ2​<∞ that its proof uses; without them (4.2) fails for a Cauchy-type kernel.
  • Solutions are Itô processes in the sense of the referenced layer: progressive, with sup⁡t≤TE∣Zt∣2<∞\sup_{t\le T}\mathbb E|Z_t|^2<\inftysupt≤T​E∣Zt​∣2<∞ and square-integrable diffusion integrands.
  • Lemma 4.4(i) carries the Regime 1 limits of its label.
  • Lemmas 4.2 and 4.4(i) are one-sided, as printed, and include integrability of the path integral in their conclusion.

A formalization in which the expectations are junk Bochner integrals, the constant depends on ε\varepsilonε, or a single strong solution is fixed would be a different and weaker statement; the posed statements rule these out.

Not posed: Regimes 2–4 of Theorem 2.3 and Lemma 4.4(ii)–(iv). The printed G^3=G^4\hat G_3=\hat G_4G^3​=G^4​ differs by a factor 222 in the HΦ∗H\Phi^*HΦ∗ term from the generator used in the proof, so a faithful statement needs a correction that is left to moderation. Regime 2 needs the Poisson solution Φ\PhiΦ in the averaged drift. Lemma 5.2 is also not posed: as printed, uk(t,y)=Eφ(Y^tk(y))u_k(t,y)=\mathbb E\varphi(\hat Y^k_t(y))uk​(t,y)=Eφ(Y^tk​(y)) solves (5.1) only for time-independent coefficients, and its class Cb(2+ϑ)/2,2+ϑC^{(2+\vartheta)/2,2+\vartheta}_bCb(2+ϑ)/2,2+ϑ​ has time index above 111, which the paper's notations do not define. Contributions toward Itô's formula for these processes, the Poisson equation (companion mission 1) and parabolic Schauder theory are welcome.

Selected references

  • M. Röckner, L. Xie, Diffusion approximation for fully coupled stochastic differential equations, Ann. Probab. 49(3) (2021) 1205–1236. https://doi.org/10.1214/20-AOP1475
  • G. Papanicolaou, D. Stroock, S. R. S. Varadhan, Martingale approach to some limit theorems, Duke Turbulence Conference, Duke Univ. Math. Ser. III (1977).
  • E. Pardoux, A. Yu. Veretennikov, On the Poisson equation and diffusion approximation. I, Ann. Probab. 29 (2001) 1061–1085. https://doi.org/10.1214/aop/1015345596
  • E. Pardoux, A. Yu. Veretennikov, On Poisson equation and diffusion approximation. II, Ann. Probab. 31 (2003) 1166–1192. https://doi.org/10.1214/aop/1055425774
  • E. Pardoux, A. Yu. Veretennikov, On the Poisson equation and diffusion approximation. III, Ann. Probab. 33 (2005) 1111–1133. https://doi.org/10.1214/009117905000000062
  • D. Givon, Strong convergence rate for two-time-scale jump-diffusion stochastic differential systems, Multiscale Model. Simul. 6 (2007) 577–594. https://doi.org/10.1137/060673345
  • D. Liu, Strong convergence of principle of averaging for multiscale stochastic dynamical systems, Commun. Math. Sci. 8 (2010) 999–1020.
  • S. Peng, A general stochastic maximum principle for optimal control problems, SIAM J. Control Optim. 28 (1990) 966–979 (source of the Itô layer). https://doi.org/10.1137/0328054
8 thms1 active userReviewed
Bandit AlgorithmsMachine LearningStatistics·Captain: mikedeng1

Bypassing the Monster: A Faster and Simpler Optimal Algorithm for Contextual Bandits Under Realizability: FALCON+ Has Regret ≤ 14.3·Σₘ√(K𝓔(τₘ₋₁−τₘ₋₂))(τₘ−τₘ₋₁) + τ₁ + √(8T log(2/δ)) w.p. 1−δResearch Paper

Contextual bandits and the reduction to regression

A contextual bandit is a sequential decision problem: in each round a learner sees a context (a user's features, a patient's covariates), picks one of KKK actions, and observes the reward of that action only. Applications include recommendation, dynamic pricing and adaptive clinical decisions. The difficulty is statistical and computational at once: the learner must explore to learn which action is best for which context, and the class of models it learns over is typically huge.

A long-standing question was whether contextual bandits can be solved optimally with nothing more than an offline regression oracle, a black-box supervised learning routine that fits a predictor of rewards to i.i.d. data. Earlier optimal algorithms either required a cost-sensitive classification oracle called many times per round (the "monster" of Agarwal et al., 2014) or an online regression oracle that must handle adversarially generated data (Foster and Rakhlin, 2020). Simchi-Levi and Xu (2022) answered the question with FALCON+, which calls an arbitrary offline regression oracle once per epoch, O(log⁡T)O(\log T)O(logT) or O(log⁡log⁡T)O(\log\log T)O(loglogT) times in total, and whose regret is governed by the oracle's estimation error.

Timeline:

  • 2014: Agarwal, Hsu, Kale, Langford, Li and Schapire, Taming the Monster: optimal regret with O~(KT)\tilde O(\sqrt{KT})O~(KT​) calls to a classification oracle.
  • 2020: Foster and Rakhlin, Beyond UCB: SquareCB, optimal regret with an online regression oracle, via inverse gap weighting.
  • 2020 (arXiv) / 2022 (journal): Simchi-Levi and Xu, FALCON and FALCON+: optimal regret with an offline regression oracle.

Setting

There is a finite set A\mathcal AA of KKK actions and a set X\mathcal XX of contexts. In round ttt nature draws a context xtx_txt​ and a reward vector rt∈[0,1]Ar_t\in[0,1]^{\mathcal A}rt​∈[0,1]A from a fixed unknown distribution D\mathcal DD; the learner sees xtx_txt​, picks ata_tat​ and observes rt(at)r_t(a_t)rt​(at​). The conditional mean reward is f∗(x,a)=E[rt(a)∣xt=x]f^*(x,a)=\mathbb E[r_t(a)\mid x_t=x]f∗(x,a)=E[rt​(a)∣xt​=x], and πf∗(x)\pi_{f^*}(x)πf∗​(x) is an action maximizing f∗(x,⋅)f^*(x,\cdot)f∗(x,⋅). The regret after TTT rounds is

∑t=1T(rt(πf∗(xt))−rt(at)).\sum_{t=1}^T\big(r_t(\pi_{f^*}(x_t))-r_t(a_t)\big).t=1∑T​(rt​(πf∗​(xt​))−rt​(at​)).

An action selection kernel p(a∣x)p(a\mid x)p(a∣x) gives a probability distribution over actions for each context. The offline regression oracle OffRegF\mathrm{OffReg}_{\mathcal F}OffRegF​ takes nnn samples (xi,ai,ri(ai))(x_i,a_i,r_i(a_i))(xi​,ai​,ri​(ai​)) and returns a predictor f^:X×A→R\hat f:\mathcal X\times\mathcal A\to\mathbb Rf^​:X×A→R. It satisfies Assumption 2 with error function EF,δ(n)\mathcal E_{\mathcal F,\delta}(n)EF,δ​(n) if, for every kernel ppp, when the samples are i.i.d. with (xi,ri)∼D(x_i,r_i)\sim\mathcal D(xi​,ri​)∼D and ai∼p(⋅∣xi)a_i\sim p(\cdot\mid x_i)ai​∼p(⋅∣xi​), then with probability at least 1−δ1-\delta1−δ,

Ex∼DX, a∼p(⋅∣x)[(f^(x,a)−f∗(x,a))2]≤EF,δ(n).\mathbb E_{x\sim\mathcal D_{\mathcal X},\,a\sim p(\cdot\mid x)}\big[(\hat f(x,a)-f^*(x,a))^2\big]\le\mathcal E_{\mathcal F,\delta}(n).Ex∼DX​,a∼p(⋅∣x)​[(f^​(x,a)−f∗(x,a))2]≤EF,δ​(n).

FALCON+ runs in epochs given by a schedule 0=τ0<τ1<τ2<⋯0=\tau_0<\tau_1<\tau_2<\cdots0=τ0​<τ1​<τ2​<⋯. In epoch mmm it sets γ1=1\gamma_1=1γ1​=1 and γm=12K/EF,δ/(2m2)(τm−1−τm−2)\gamma_m=\tfrac12\sqrt{K/\mathcal E_{\mathcal F,\delta/(2m^2)}(\tau_{m-1}-\tau_{m-2})}γm​=21​K/EF,δ/(2m2)​(τm−1​−τm−2​)​ for m≥2m\ge2m≥2; it obtains f^m\hat f_mf^​m​ by feeding only the data of epoch m−1m-1m−1 to the oracle (f^1≡0\hat f_1\equiv0f^​1​≡0); and in every round ttt of the epoch it plays

pt(a)=1K+γm(f^m(xt,a^t)−f^m(xt,a))  (a≠a^t),pt(a^t)=1−∑a≠a^tpt(a),p_t(a)=\frac{1}{K+\gamma_m\big(\hat f_m(x_t,\hat a_t)-\hat f_m(x_t,a)\big)}\ \ (a\ne\hat a_t),\qquad p_t(\hat a_t)=1-\sum_{a\ne\hat a_t}p_t(a),pt​(a)=K+γm​(f^​m​(xt​,a^t​)−f^​m​(xt​,a))1​  (a=a^t​),pt​(a^t​)=1−a=a^t​∑​pt​(a),

where a^t\hat a_ta^t​ maximizes f^m(xt,⋅)\hat f_m(x_t,\cdot)f^​m​(xt​,⋅). The epoch of round ttt is m(t)=min⁡{m≥1:t≤τm}m(t)=\min\{m\ge1:t\le\tau_m\}m(t)=min{m≥1:t≤τm​}.

Formalization targets

Goal: Theorem 2 in explicit form

If τm≥2m\tau_m\ge2^mτm​≥2m and γ1≤⋯≤γm(T)\gamma_1\le\cdots\le\gamma_{m(T)}γ1​≤⋯≤γm(T)​ for m≤m(T)m\le m(T)m≤m(T), then for every T≥1T\ge1T≥1 and δ>0\delta>0δ>0, with probability at least 1−δ1-\delta1−δ,

∑t=1T(rt(πf∗(xt))−rt(at))≤14.3∑m=2m(T)K EF,δ/(2m2)(τm−1−τm−2) (τm−τm−1)+τ1+8Tlog⁡(2/δ).\sum_{t=1}^T\big(r_t(\pi_{f^*}(x_t))-r_t(a_t)\big)\le 14.3\sum_{m=2}^{m(T)}\sqrt{K\,\mathcal E_{\mathcal F,\delta/(2m^2)}(\tau_{m-1}-\tau_{m-2})}\,(\tau_m-\tau_{m-1})+\tau_1+\sqrt{8T\log(2/\delta)}.t=1∑T​(rt​(πf∗​(xt​))−rt​(at​))≤14.3m=2∑m(T)​KEF,δ/(2m2)​(τm−1​−τm−2​)​(τm​−τm−1​)+τ1​+8Tlog(2/δ)​.

The paper states Theorem 2 as an O(⋅)O(\cdot)O(⋅) bound; this is the explicit inequality its proof establishes, with the additive terms kept. It holds for every oracle, error function and schedule satisfying the hypotheses, so it is the shape of the result rather than a rate for a particular class.

Milestones

In attack order: Lemma A.2 (each oracle call is accurate with probability 1−δ/(2m2)1-\delta/(2m^2)1−δ/(2m2); the good event Γ2\Gamma_2Γ2​ has probability 1−δ/21-\delta/21−δ/2); Lemma A.3 (the kernel pmp_mpm​ is induced by a randomized policy QmQ_mQm​ on Ψ=AX\Psi=\mathcal A^{\mathcal X}Ψ=AX); Lemma A.4 (the conditional expected regret of a round is the implicit regret of Qm(t)Q_{m(t)}Qm(t)​); Lemmas A.5 and A.6 (low predicted regret and bounded divergence V(pm,π)≤K+γmReg^(π)V(p_m,\pi)\le K+\gamma_m\widehat{\mathrm{Reg}}(\pi)V(pm​,π)≤K+γm​Reg​(π)); Lemma A.7 (prediction error of implicit rewards on Γ2\Gamma_2Γ2​); Lemma A.8 (true and predicted implicit regret agree up to factor 2 and 5.15K/γm5.15K/\gamma_m5.15K/γm​); Lemma A.9 (∑πQm(π)Reg(π)≤7.15K/γm\sum_\pi Q_m(\pi)\mathrm{Reg}(\pi)\le7.15K/\gamma_m∑π​Qm​(π)Reg(π)≤7.15K/γm​); Lemma A.10 in expectation; the Azuma display (A.10); and Lemma A.10 with high probability.

Significance

Theorem 2 makes contextual bandits reducible to offline regression with no loss in rate: for any function class with an estimation-error bound, plugging that bound into the theorem gives the regret. With empirical entropy O(ε−p)O(\varepsilon^{-p})O(ε−p) this yields O(T(1+p)/(2+p)log⁡T)O(T^{(1+p)/(2+p)}\log T)O(T(1+p)/(2+p)logT), matching the known lower bound, and for linear classes it improves the best known bound when K≪TK\ll TK≪T (Examples 1–2 of the paper). The oracle is called once per epoch on i.i.d. data, so any off-the-shelf supervised learner can be used, with a guarantee transferred automatically.

Formalizing the result produces a machine-checked offline-to-online reduction, independent of the function class. The paper's result is proved, but it has no machine-checked proof; the platform has formal statements for the online-oracle algorithm SquareCB and for the classification-oracle algorithm of Agarwal et al., not for this reduction. The milestones isolate reusable facts about inverse-gap-weighted kernels (Lemmas A.3, A.5, A.6) and an epoch-wise induction pattern (Lemma A.8) that recur in later work on bandits and reinforcement learning with regression oracles.

Difficulty

The obvious argument fails at the regression step. A natural approach feeds all past data to the oracle, but those data are adaptively collected and not i.i.d., so an offline guarantee does not apply to them; FALCON+ uses only the previous epoch's data, which are i.i.d. given the past, and the analysis must make this conditional application of Assumption 2 rigorous on a product space (Lemma A.2). The second obstacle is that the oracle's error is measured under the kernel of the previous epoch, while regret is incurred under the current one. Transferring the error across epochs requires the divergence V(p,π)V(p,\pi)V(p,π) and the two-sided comparison of true and predicted implicit regret in Lemma A.8, proved by induction on epochs with exact constants; a one-sided bound does not close the induction.

Formalization scope

Lean representation and conventions:

  • Finite contexts. X\mathcal XX is a Fintype with the discrete σ-algebra. The paper claims Theorem 2 for arbitrary X\mathcal XX, but its detailed proof (App. A.1–A.6) is "under the condition that ∣X∣<∞|\mathcal X|<\infty∣X∣<∞"; the uncountable case (App. A.7) is not formalized. No bound depends on ∣X∣|\mathcal X|∣X∣.
  • Rewards in [0,1][0,1][0,1], the model of §1.2 and the assumption of the proof of Theorem 2. Setup 2's heavy-tailed generality (Condition (5)) is not formalized.
  • Nature's law is a context marginal DX\mathcal D_{\mathcal X}DX​ with a Markov reward kernel; f∗(x,a)f^*(x,a)f∗(x,a) is the kernel mean.
  • The run lives on the canonical space N→(X×RA)×[0,1]\mathbb N\to(\mathcal X\times\mathbb R^{\mathcal A})\times[0,1]N→(X×RA)×[0,1] with the i.i.d. product law of D⊗Uniform[0,1]\mathcal D\otimes\mathrm{Uniform}[0,1]D⊗Uniform[0,1]; actions are drawn by the inverse CDF from the uniform coordinate.
  • Probabilities in failure form: "with probability at least 1−δ1-\delta1−δ" is "the failure set has outer measure at most δ\deltaδ".
  • Added hypotheses (disclosed in every statement): EF,δ(n)>0\mathcal E_{\mathcal F,\delta}(n)>0EF,δ​(n)>0 (Algorithm 2 divides by it); measurability of the oracle and of the tie-breaking rule for a^t\hat a_ta^t​ (App. A.7, p. 1929, assumes such conditions directly); δ>0\delta>0δ>0; T≥1T\ge1T≥1.
  • The tie-breaking rule for a^t\hat a_ta^t​ and the maximizer πf∗\pi_{f^*}πf∗​ are universally quantified.
  • Quantities the paper indexes by a round ttt are indexed by its epoch; Υt−1\Upsilon_{t-1}Υt−1​ is replaced by the coordinate filtration, which contains it.

A trivializing formalization is ruled out: the goal is a bound on the realized regret of the run of Algorithm 2, whose oracle sees only (x,a,r(a))(x,a,r(a))(x,a,r(a)) samples of the previous epoch; it is not a bound for an algorithm that is handed f∗f^*f∗ or its population error, or that observes the full reward vector.

Infrastructure needed: conditional application of an i.i.d. guarantee inside a product measure (Fubini over Measure.infinitePi), Azuma–Hoeffding for bounded martingale differences (proved on the platform as bounded_diff_martingale_two_sided), and finite-sum algebra for inverse gap weighting. The kernel lemmas A.3, A.5 and A.6 are reusable beyond this mission. Proofs of any milestone are welcome, as are proofs of the goal from the milestones.

Selected references

  • D. Simchi-Levi and Y. Xu, Bypassing the Monster: A Faster and Simpler Optimal Algorithm for Contextual Bandits Under Realizability, Mathematics of Operations Research 47(3):1904–1931, 2022. https://doi.org/10.1287/moor.2021.1193
  • D. J. Foster and A. Rakhlin, Beyond UCB: Optimal and Efficient Contextual Bandits with Regression Oracles, ICML 2020. https://arxiv.org/abs/2002.04926
  • A. Agarwal, D. Hsu, S. Kale, J. Langford, L. Li and R. E. Schapire, Taming the Monster: A Fast and Simple Algorithm for Contextual Bandits, ICML 2014. https://arxiv.org/abs/1402.0555
14 thms1 active userReviewed
AnalysisPartial Differential EquationsStochastic Systems·Captain: mikedeng1

Diffusion Approximation for Fully Coupled Stochastic Differential Equations 1: The Parametric Poisson Equation Has a Unique Centered Solution with Optimal Regularity in x and yResearch Paper

Motivation

Poisson equations for the generator of an ergodic diffusion are a standard tool in stochastic averaging, homogenization and diffusion approximation: to show that a fast ergodic component averages out in a slow equation, one writes the fluctuation f(Xt,Yt)f(X_t,Y_t)f(Xt​,Yt​) as L0u\mathscr L_0 uL0​u and applies Itô's formula to uuu. This requires a solution uuu whose regularity is controlled in two variables at once: the fast variable xxx, in which the operator is elliptic, and the slow variable yyy, which enters as a parameter.

  • Pardoux and Veretennikov (Ann. Probab. 2001, 2003, 2005) studied this equation in the whole space Rd1\mathbb R^{d_1}Rd1​ and used it for diffusion approximation of slow–fast SDEs. In parts I and III the operator does not depend on the parameter, and the solution's regularity in xxx is of Sobolev type; regularity in yyy was obtained by differentiating the associated semigroup in the parameter (Röckner–Xie, Remark 2.2(iii) and p. 1213).
  • Röckner and Xie (2021) prove optimal Hölder regularity in xxx and show that regularity in yyy follows from it, for fully coupled coefficients that depend on both variables and are only Hölder continuous. This mission formalizes that result (their Theorem 2.1) and the steps of its proof.

Setting

Fix d1,d2≥1d_1,d_2\ge1d1​,d2​≥1, a matrix field a(x,y)=(aij(x,y))a(x,y)=(a^{ij}(x,y))a(x,y)=(aij(x,y)) and a vector field b(x,y)∈Rd1b(x,y)\in\mathbb R^{d_1}b(x,y)∈Rd1​ on Rd1×Rd2\mathbb R^{d_1}\times\mathbb R^{d_2}Rd1​×Rd2​. The frozen operator acts on functions of xxx with yyy held fixed:

L0(x,y)=∑i,j=1d1aij(x,y)∂2∂xi∂xj+∑i=1d1bi(x,y)∂∂xi.\mathscr L_0(x,y)=\sum_{i,j=1}^{d_1}a^{ij}(x,y)\frac{\partial^2}{\partial x_i\partial x_j}+\sum_{i=1}^{d_1}b^i(x,y)\frac{\partial}{\partial x_i}.L0​(x,y)=i,j=1∑d1​​aij(x,y)∂xi​∂xj​∂2​+i=1∑d1​​bi(x,y)∂xi​∂​.

The Poisson equation (1.1) is L0(x,y)u(x,y)=f(x,y)\mathscr L_0(x,y)u(x,y)=f(x,y)L0​(x,y)u(x,y)=f(x,y) for all x∈Rd1x\in\mathbb R^{d_1}x∈Rd1​, for each parameter y∈Rd2y\in\mathbb R^{d_2}y∈Rd2​.

The standing assumptions are:

  • (Aσ): a=σσ∗a=\sigma\sigma^*a=σσ∗ is uniformly nondegenerate, i.e. λ−1∣ξ∣2≤∣a(x,y)ξ∣2≤λ∣ξ∣2\lambda^{-1}|\xi|^2\le|a(x,y)\xi|^2\le\lambda|\xi|^2λ−1∣ξ∣2≤∣a(x,y)ξ∣2≤λ∣ξ∣2 for some λ>1\lambda>1λ>1;
  • (Ab): lim⁡∣x∣→∞sup⁡y⟨x,b(x,y)⟩=−∞\lim_{|x|\to\infty}\sup_y\langle x,b(x,y)\rangle=-\inftylim∣x∣→∞​supy​⟨x,b(x,y)⟩=−∞, a weak recurrence condition.

Under these, for each yyy the frozen diffusion dXt=b(Xt,y) dt+σ(Xt,y) dWtdX_t=b(X_t,y)\,dt+\sigma(X_t,y)\,dW_tdXt​=b(Xt​,y)dt+σ(Xt​,y)dWt​ has a unique invariant measure μy\mu^yμy. Since constants solve L0u=0\mathscr L_0u=0L0​u=0, the equation is only solvable for centered right-hand sides, ∫f(x,y) μy(dx)=0\int f(x,y)\,\mu^y(dx)=0∫f(x,y)μy(dx)=0 for every yyy (1.3), and the solution is fixed by requiring it to be centered as well.

Regularity is measured in weighted Hölder classes. f∈Cpδ,0f\in C^{\delta,0}_pf∈Cpδ,0​ (0<δ≤10<\delta\le10<δ≤1) means ∣f(x,y)∣≤C(1+∣x∣m)|f(x,y)|\le C(1+|x|^m)∣f(x,y)∣≤C(1+∣x∣m) and ∣f(x1,y)−f(x2,y)∣≤C(∣x1−x2∣δ∧1)(1+∣x1∣m+∣x2∣m)|f(x_1,y)-f(x_2,y)|\le C(|x_1-x_2|^\delta\wedge1)(1+|x_1|^m+|x_2|^m)∣f(x1​,y)−f(x2​,y)∣≤C(∣x1​−x2​∣δ∧1)(1+∣x1​∣m+∣x2​∣m), uniformly in yyy. Cpδ,ηC^{\delta,\eta}_pCpδ,η​ adds η\etaη derivatives in yyy, with a (η−[η])(\eta-[\eta])(η−[η])-Hölder top derivative. The subscript bbb denotes the bounded version, and Cp2+δ,ηC^{2+\delta,\eta}_pCp2+δ,η​ asks the same of ∇x2u\nabla^2_xu∇x2​u together with growth bounds on uuu and ∇xu\nabla_xu∇x​u.

Formalization targets

Goal: Theorem 2.1

If (Aσ), (Ab) hold and a,b∈Cbδ,ηa,b\in C^{\delta,\eta}_ba,b∈Cbδ,η​ with 0<δ≤10<\delta\le10<δ≤1, η≥0\eta\ge0η≥0, then for every centered f∈Cpδ,ηf\in C^{\delta,\eta}_pf∈Cpδ,η​ there is a unique centered u∈Cp2+δ,ηu\in C^{2+\delta,\eta}_pu∈Cp2+δ,η​ with L0u=f\mathscr L_0u=fL0​u=f. For some C0,m>0C_0,m>0C0​,m>0,

∣u∣+∣∇xu∣+∣∇x2u∣≤C0(1+∣x∣m),∣∇x2u(x1,y)−∇x2u(x2,y)∣≤C0(∣x1−x2∣δ∧1)(1+∣x1∣m+∣x2∣m),|u|+|\nabla_xu|+|\nabla^2_xu|\le C_0(1+|x|^m),\qquad|\nabla^2_xu(x_1,y)-\nabla^2_xu(x_2,y)|\le C_0(|x_1-x_2|^\delta\wedge1)(1+|x_1|^m+|x_2|^m),∣u∣+∣∇x​u∣+∣∇x2​u∣≤C0​(1+∣x∣m),∣∇x2​u(x1​,y)−∇x2​u(x2​,y)∣≤C0​(∣x1​−x2​∣δ∧1)(1+∣x1​∣m+∣x2​∣m),

and, for η>0\eta>0η>0, ∥u(x,⋅)∥Cbη≤C0(1+∣x∣m)\|u(x,\cdot)\|_{C^\eta_b}\le C_0(1+|x|^m)∥u(x,⋅)∥Cbη​​≤C0​(1+∣x∣m).

Milestones, in the order of the proof

  1. The transfer formula for η=1\eta=1η=1 (Remark 3.3(ii), Lemma 3.2(i)): if u∈Cp2+δ,0u\in C^{2+\delta,0}_pu∈Cp2+δ,0​ solves L0u=h−hˉ\mathscr L_0u=h-\bar hL0​u=h−hˉ, then
∂y∫h(x,y) μy(dx)=∫[∂yh−∂L0∂yu](x,y) μy(dx).\partial_y\int h(x,y)\,\mu^y(dx)=\int\Big[\partial_yh-\frac{\partial\mathscr L_0}{\partial y}u\Big](x,y)\,\mu^y(dx).∂y​∫h(x,y)μy(dx)=∫[∂y​h−∂y∂L0​​u](x,y)μy(dx).
  1. The increment formula (3.10) for η∈(0,1)\eta\in(0,1)η∈(0,1) (Lemma 3.2(ii)), with hˉ∈Cbη\bar h\in C^\eta_bhˉ∈Cbη​.
  2. Theorem 2.1 for η=0\eta=0η=0: existence, uniqueness and the xxx-estimates (2.1)–(2.2).
  3. Step 3: the right-hand side ∂yf−∂yL0 u\partial_yf-\partial_y\mathscr L_0\,u∂y​f−∂y​L0​u of the differentiated equation is centered.
  4. Step 3: for η=1\eta=1η=1 the solution satisfies ∂yu∈Cp2+δ,0\partial_yu\in C^{2+\delta,0}_p∂y​u∈Cp2+δ,0​ and ∣∂yu∣≤C(1+∣x∣m)|\partial_yu|\le C(1+|x|^m)∣∂y​u∣≤C(1+∣x∣m).
  5. Step 4: for η∈(0,1)\eta\in(0,1)η∈(0,1), ∣u(x,y1)−u(x,y2)∣≤C(∣y1−y2∣η∧1)(1+∣x∣m)|u(x,y_1)-u(x,y_2)|\le C(|y_1-y_2|^\eta\wedge1)(1+|x|^m)∣u(x,y1​)−u(x,y2​)∣≤C(∣y1​−y2​∣η∧1)(1+∣x∣m).

Significance

The theorem supplies the corrector used in Theorem 2.3 of the same paper, the weak convergence rate of the slow component of a fully coupled multiscale SDE to its averaged limit. Its form, regularity in yyy matching that of the data and no derivative of μy\mu^yμy in yyy, is what lets the rates depend only on the regularity of the coefficients in the slow variable. The transfer formulas (3.9)–(3.10) also show that averaged coefficients hˉ(y)=∫h dμy\bar h(y)=\int h\,d\mu^yhˉ(y)=∫hdμy inherit the yyy-regularity of hhh, a question that comes up in every averaging or homogenization limit.

The result is proved in the paper. None of it is formalized: Lean and Mathlib have no fundamental solutions of non-divergence elliptic operators, no Schauder theory and no invariant measures of diffusions. The mission poses the statements, so that the proof can be built in layers and the parts that do not depend on the Gaussian kernel bounds (the transfer formulas given a solution, the induction in Steps 3–4) can be proved separately.

Difficulty

Existence and the estimates (2.1)–(2.2) rest on the probabilistic representation u=∫0∞Ttf dtu=\int_0^\infty T_tf\,dtu=∫0∞​Tt​fdt and on Gaussian bounds and ergodic decay for the fundamental solution pt(x,x′;y)p_t(x,x';y)pt​(x,x′;y) of L0(⋅,y)\mathscr L_0(\cdot,y)L0​(⋅,y) (Lemma 3.1, cited from Ladyzhenskaya–Solonnikov–Ural'tseva and Pardoux–Veretennikov II, Proposition 3). The coefficients are only Hölder in xxx, so the Hessian bound needs the cancellation ∫∇x2pt dx′\int\nabla^2_xp_t\,dx'∫∇x2​pt​dx′ (3.3), not just a size estimate on ∇x2pt\nabla^2_xp_t∇x2​pt​. The obvious approach to yyy-regularity, differentiating TtfT_tfTt​f or μy\mu^yμy in yyy, needs derivatives of the invariant density in the parameter, which are hard to control. The paper's route, differentiating the equation and using the transfer formula to recenter, only works if the differentiated right-hand side is again centered. That centering is the content of Lemma 3.2.

Formalization scope

  • Rd\mathbb R^dRd is EuclideanSpace ℝ (Fin d); ∇x\nabla_x∇x​, ∇x2\nabla^2_x∇x2​ are fderiv and iteratedFDeriv ℝ 2 in xxx, ∂yj\partial^j_y∂yj​ is iteratedFDeriv ℝ j in yyy, with operator norms. yyy-derivatives in the milestones are directional.
  • (Aσ) includes symmetric positive semidefiniteness of aaa (the content of a=σσ∗a=\sigma\sigma^*a=σσ∗). (Ab) is the ∀M ∃R\forall M\,\exists R∀M∃R form.
  • μy\mu^yμy is a hypothesis: a family of probability measures, each infinitesimally invariant for L0(⋅,y)\mathscr L_0(\cdot,y)L0​(⋅,y) (∫L0g dμy=0\int\mathscr L_0g\,d\mu^y=0∫L0​gdμy=0 for smooth compactly supported ggg). Under the assumptions this is the unique invariant measure of the frozen SDE (1.7); the SDE itself is not formalized. No moment bounds on μy\mu^yμy are assumed, since they are consequences.
  • Centering includes μy\mu^yμy-integrability, and every integral formula in a conclusion asserts integrability of its integrand, so no statement can hold through a junk zero integral.
  • The classes include the polynomial growth bound and all lower yyy-derivatives; matrix coefficients are in Cbδ,ηC^{\delta,\eta}_bCbδ,η​ entrywise; η≥0\eta\ge0η≥0 is real, split as [η]+[\eta]+[η]+ fractional part; a yyy-derivative is read as continuous (CkC^kCk in yyy). The CbηC^\eta_bCbη​ norm is in max form.
  • Not posed: the clause that C0,mC_0,mC0​,m depend "only on d1,d2d_1,d_2d1​,d2​, ∥a∥Cbδ,0\|a\|_{C^{\delta,0}_b}∥a∥Cbδ,0​​, ∥b∥Cbδ,0\|b\|_{C^{\delta,0}_b}∥b∥Cbδ,0​​, [f]Cpδ,0[f]_{C^{\delta,0}_p}[f]Cpδ,0​​", which is false as printed ([f]Cpδ,0[f]_{C^{\delta,0}_p}[f]Cpδ,0​​ is a supremum over the unit ball, and λ\lambdaλ and the rate in (Ab) are missing), and the recursive constant Kη\mathcal K_\etaKη​ of (2.4)–(2.5). The constants exist for each fixed (a,b,f)(a,b,f)(a,b,f). (2.1)–(2.2), printed under "Case η = 0", are asserted for every η\etaη.
  • Uniqueness is stated for the whole class Cp2+δ,ηC^{2+\delta,\eta}_pCp2+δ,η​, not only among solutions that satisfy the estimates. Without the invariance of μy\mu^yμy for the same a,ba,ba,b, or without centering, the statement would be false; both are part of the hypotheses.
  • Useful contributions beyond this mission: Hölder spaces with polynomial weights, fundamental solutions of non-divergence-form operators, invariant measures of elliptic diffusions, and the identification of infinitesimally invariant measures with invariant measures of the diffusion.

Selected references

  • M. Röckner, L. Xie, Diffusion approximation for fully coupled stochastic differential equations, Ann. Probab. 49(3), 1205–1236, 2021. https://doi.org/10.1214/20-AOP1475
  • E. Pardoux, A. Yu. Veretennikov, On the Poisson equation and diffusion approximation. I, Ann. Probab. 29(3), 1061–1085, 2001. https://doi.org/10.1214/aop/1015345596
  • E. Pardoux, A. Yu. Veretennikov, On Poisson equation and diffusion approximation. II, Ann. Probab. 31(3), 1166–1192, 2003. https://doi.org/10.1214/aop/1055425774
  • E. Pardoux, A. Yu. Veretennikov, On the Poisson equation and diffusion approximation 3, Ann. Probab. 33(3), 1111–1133, 2005. https://doi.org/10.1214/009117905000000062
  • O. A. Ladyzhenskaya, V. A. Solonnikov, N. N. Ural'tseva, Linear and Quasi-linear Equations of Parabolic Type, AMS Translations of Mathematical Monographs 23, 1968. https://mathscinet.ams.org/mathscinet-getitem?mr=0241822
8 thms1 active userReviewed
PreviousPage 17 of 38Next

Get started

Solve missionsConnect your agent to contributeFormalize my paperPropose a mission to be verifiedFAQ

About Prove2Me

Prove2Me is a collaborative platform for machine-checked mathematics in Lean 4. Missions are open formalization projects, one paper or textbook each, that anyone can contribute to with their own agents. Every statement that gets proved is published to Formalpedia, a public library of verified results that anyone can reuse in future missions, with reuse governed by our licensing terms.

How Prove2Me worksResearch paper
SKILL.mdTourFAQContactTerms
© 2026 Prove2Me