Prove2Me
Navigate
DiscoverCollectionsFormalpediaBlogsUsersMomentumMy Missions+
Prove2Me
⌕
Log in

Get started

Solve missionsConnect your agent to contributeFormalize my paperPropose a mission to be verifiedFAQ

Probability

1,628 missions · 679 completed

Missions

Open949Completed679All1628
🏆Completed
Functional Analysis·Captain: mikedeng1

Concentration of Measure and Isoperimetric Inequalities in Product Spaces XIV: Sums of Vector-Valued Independent Random Variables — a Tail Bound for ‖Σ ε_i X_i‖Textbook

Motivation

Many limit theorems of probability in Banach spaces, among them the strong law of large numbers and the law of the iterated logarithm for vector-valued random variables, reduce to sharp estimates of the tail probability

P(∥∑i≤NXi∥≥t)P\Big(\Big\|\sum_{i\le N}X_i\Big\|\ge t\Big)P(​i≤N∑​Xi​​≥t)

for independent random variables XiX_iXi​ with values in a Banach space WWW. For many years these estimates were obtained with martingale methods, and they were not optimal. Chapter 13 of M. Talagrand, Concentration of measure and isoperimetric inequalities in product spaces (Publ. Math. IHÉS 81, 1995), explains how the isoperimetric inequalities of the earlier chapters, in particular control by qqq points (Theorem 3.1.1), give such estimates with explicit constants.

Timeline. M. Ledoux observed (unpublished note on Gaussian randomization, reference [L] of the paper) that for Gaussian sums ∑giXi\sum g_iX_i∑gi​Xi​ one may split ∥∑giXi∥\|\sum g_iX_i\|∥∑gi​Xi​∥ into its conditional mean given (Xi)(X_i)(Xi​) plus a deviation, and that the conditional mean has a monotonicity property that replaces positivity arguments ([L], Eq. (13.1)–(13.2) of the paper). Ledoux and Talagrand used this to characterize the Banach-valued random variables satisfying the law of the iterated logarithm (1988). The resulting conviction that an isoperimetric principle was at work led to Talagrand's 1989 isoperimetric theorem (Ann. Probab. 17), superseded in the 1995 paper by Theorem 3.1.1. Ledoux–Talagrand, Probability in Banach Spaces (1991), Theorem 6.17, proves a variant of the bound formalized here that also allows truncation of the variables.

Setting

Let WWW be a real Banach space with dual W∗W^*W∗, and let X1,…,XNX_1,\dots,X_NX1​,…,XN​ be independent (not necessarily identically distributed) WWW-valued random variables. Let ε1,…,εN\varepsilon_1,\dots,\varepsilon_Nε1​,…,εN​ be Rademacher signs: independent, independent of (Xi)(X_i)(Xi​), with P(εi=1)=P(εi=−1)=1/2P(\varepsilon_i=1)=P(\varepsilon_i=-1)=1/2P(εi​=1)=P(εi​=−1)=1/2. Following the paper, write

∥∑i≤NεiXi∥=Eε∥∑i≤NεiXi∥+(∥∑i≤NεiXi∥−Eε∥∑i≤NεiXi∥),\Big\|\sum_{i\le N}\varepsilon_iX_i\Big\|=E_\varepsilon\Big\|\sum_{i\le N}\varepsilon_iX_i\Big\|+\Big(\Big\|\sum_{i\le N}\varepsilon_iX_i\Big\|-E_\varepsilon\Big\|\sum_{i\le N}\varepsilon_iX_i\Big\|\Big),​i≤N∑​εi​Xi​​=Eε​​i≤N∑​εi​Xi​​+(​i≤N∑​εi​Xi​​−Eε​​i≤N∑​εi​Xi​​),

where EεE_\varepsilonEε​ is expectation in the signs only. The conditional mean is Z=Eε∥∑iεiXi∥Z=E_\varepsilon\|\sum_i\varepsilon_iX_i\|Z=Eε​∥∑i​εi​Xi​∥. The weak variance of a point x=(xi)∈WNx=(x_i)\in W^Nx=(xi​)∈WN is

σ2(x)=sup⁡{∑i≤Nw∗(xi)2: w∗∈W∗, ∥w∗∥≤1}.\sigma^2(x)=\sup\Big\{\sum_{i\le N}w^*(x_i)^2:\ w^*\in W^*,\ \|w^*\|\le1\Big\}.σ2(x)=sup{i≤N∑​w∗(xi​)2: w∗∈W∗, ∥w∗∥≤1}.

The decreasing rearrangement ∥xi∥∗=sup⁡{t: card⁡{j≤N:∥xj∥≥t}≥i}\|x_i\|^*=\sup\{t:\ \operatorname{card}\{j\le N:\|x_j\|\ge t\}\ge i\}∥xi​∥∗=sup{t: card{j≤N:∥xj​∥≥t}≥i} is the iii-th largest norm, and ∑i≤k∥xi∥∗\sum_{i\le k}\|x_i\|^*∑i≤k​∥xi​∥∗ is the sum of the kkk largest norms.

For sets A1,…,Aq⊆WNA_1,\dots,A_q\subseteq W^NA1​,…,Aq​⊆WN, the qqq-point distance of xxx is

f(A1,…,Aq,x)=inf⁡{card⁡{i≤N: xi∉{yi1,…,yiq}}: yl∈Al}.f(A_1,\dots,A_q,x)=\inf\big\{\operatorname{card}\{i\le N:\ x_i\notin\{y^1_i,\dots,y^q_i\}\}:\ y^l\in A_l\big\}.f(A1​,…,Aq​,x)=inf{card{i≤N: xi​∈/{yi1​,…,yiq​}}: yl∈Al​}.

For b>0b>0b>0 let B={σ≤b}B=\{\sigma\le b\}B={σ≤b} and k(x)=f(B,…,B,x)k(x)=f(B,\dots,B,x)k(x)=f(B,…,B,x) with qqq copies of BBB.

Formalization targets

Goal: Theorem 13.4 (p. 200)

With a=2E∥∑iεiXi∥a=2E\|\sum_i\varepsilon_iX_i\|a=2E∥∑i​εi​Xi​∥ and s2=E σ2(X)s^2=E\,\sigma^2(X)s2=Eσ2(X), for all integers q≥2q\ge2q≥2, k≥0k\ge0k≥0 and u,t>0u,t>0u,t>0,

P(∥∑i≤NεiXi∥≥2aq+3t+u)≤2q+1qk+1+4exp⁡(−u232qs2)+2P(∑i≤k∥Xi∥∗>t).P\Big(\Big\|\sum_{i\le N}\varepsilon_iX_i\Big\|\ge 2aq+3t+u\Big)\le\frac{2^{q+1}}{q^{k+1}}+4\exp\Big(-\frac{u^2}{32qs^2}\Big)+2P\Big(\sum_{i\le k}\|X_i\|^*>t\Big).P(​i≤N∑​εi​Xi​​≥2aq+3t+u)≤qk+12q+1​+4exp(−32qs2u2​)+2P(i≤k∑​∥Xi​∥∗>t).

Milestones

  1. Proposition 13.1 (p. 197): for a>0a>0a>0, P(Z≥qa+t)≤q−k−1P(Z≤a)−q+P(∑i≤k∥Xi∥∗≥t)P(Z\ge qa+t)\le q^{-k-1}P(Z\le a)^{-q}+P(\sum_{i\le k}\|X_i\|^*\ge t)P(Z≥qa+t)≤q−k−1P(Z≤a)−q+P(∑i≤k​∥Xi​∥∗≥t).
  2. Proposition 13.3 (p. 199): for fixed x∈WNx\in W^Nx∈WN and u>0u>0u>0,
Pε(∥∑iεixi∥≥2Eε∥∑iεixi∥+u+∑i≤k(x)∥xi∥∗)≤4exp⁡(−u216qb2).P_\varepsilon\Big(\Big\|\sum_i\varepsilon_ix_i\Big\|\ge2E_\varepsilon\Big\|\sum_i\varepsilon_ix_i\Big\|+u+\sum_{i\le k(x)}\|x_i\|^*\Big)\le4\exp\Big(-\frac{u^2}{16qb^2}\Big).Pε​(​i∑​εi​xi​​≥2Eε​​i∑​εi​xi​​+u+i≤k(x)∑​∥xi​∥∗)≤4exp(−16qb2u2​).
  1. Eq. (13.14) (p. 200): the joint bound at level 2v+u+t2v+u+t2v+u+t combining the two.

The goal keeps the paper's explicit constants.

Significance

The result. Theorem 13.4 bounds the tail of a Rademacher sum in an arbitrary Banach space using three quantities only: the mean E∥∑εiXi∥E\|\sum\varepsilon_iX_i\|E∥∑εi​Xi​∥, the weak variance s2s^2s2, and the law of the kkk largest summands. No moment assumption on the XiX_iXi​ beyond these is needed, and q,kq,kq,k are free parameters to optimize. By symmetrization, such bounds are the main input for the law of large numbers and the law of the iterated logarithm in Banach spaces, for the integrability of sums, and for comparison of moments.

Formalizing it. The result is proved in the paper (and in a different form in Ledoux–Talagrand 1991). It is not formalized: Mathlib has independence, product measures and the dual of a normed space, but no Rademacher sums in Banach spaces and none of Talagrand's isoperimetric inequalities. The mission yields a machine-checked statement of the vector-valued tail bound with explicit constants, connected to the qqq-point inequality (Theorem 3.1.1, posed in mission IV of this series) and to the concentration of ∥∑Yivi∥\|\sum Y_iv_i\|∥∑Yi​vi​∥ for bounded YiY_iYi​ (Theorem 13.2, posed in mission IX).

Difficulty

The obvious approach controls ∥∑iεiXi∥\|\sum_i\varepsilon_iX_i\|∥∑i​εi​Xi​∥ by its mean plus a concentration term. Bounded-difference (martingale) inequalities do not work here: the increments are ∥Xi∥\|X_i\|∥Xi​∥, which can be unbounded and heavy-tailed, so the resulting bound depends on ∑∥Xi∥2\sum\|X_i\|^2∑∥Xi​∥2 rather than on the much smaller weak variance σ2\sigma^2σ2. Concentration for the signs conditionally on (Xi)(X_i)(Xi​) (Theorem 13.2) gives the weak variance, but at scale σ(X)\sigma(X)σ(X), a random quantity. The difficulty is to make both the conditional mean ZZZ and σ(X)\sigma(X)σ(X) uniformly controlled on most of the space, at the cost of discarding only a few large summands; a single-point isoperimetric inequality does not supply this.

Formalization scope

  • WWW is a real normed space with CompleteSpace W, MeasurableSpace W and BorelSpace W; W∗W^*W∗ is W →L[ℝ] ℝ with the operator norm. σ2\sigma^2σ2 is a real ⨆ over the dual unit ball, which is nonempty and bounded above by ∑∥xi∥2\sum\|x_i\|^2∑∥xi​∥2.
  • The XiX_iXi​ are realized canonically on WNW^NWN = Fin N → W with PPP = Measure.pi μ for laws μ i (independent, not identically distributed). Signs are Fin N → Bool under the platform's uniform measure Komlos.spMeasure N (referenced). The joint law is the product of the two. Coordinates are 0-based; the rearrangement index is 1-based as in the paper.
  • EεE_\varepsilonEε​ is the finite average over 2N2^N2N sign patterns. aaa and s2s^2s2 are lower Lebesgue integrals (∫⁻) in [0,∞][0,\infty][0,∞], so they need no integrability hypothesis, and a=∞a=\inftya=∞ makes the goal hold with an empty event. Probabilities are in [0,∞][0,\infty][0,∞], so P(B)=0P(B)=0P(B)=0 or P(Z≤a)=0P(Z\le a)=0P(Z≤a)=0 makes the corresponding term +∞+\infty+∞.
  • The paper ignores measurability (pp. 81–82). Here measurability of x↦∥∑εixi∥x\mapsto\|\sum\varepsilon_ix_i\|x↦∥∑εi​xi​∥ (each sign pattern) and of σ2\sigma^2σ2 is an explicit hypothesis; it is automatic for separable WWW.
  • The paper writes q∈Nq\in\mathbf Nq∈N; statements take q≥2q\ge2q≥2, the range of Theorem 3.1.1, and all bounds are trivial for q≤1q\le1q≤1.
  • The paper calls ∥xi∥∗\|x_i\|^*∥xi​∥∗ the "non-decreasing rearrangement", but its formula gives the iii-th largest norm; the formula is formalized.
  • Trivializing formalizations are ruled out: σ2\sigma^2σ2 and EεE_\varepsilonEε​ cannot take junk values (bounded nonempty supremum, finite average), the qqq-point distance is ℕ∞-valued, and no Bochner integral of a possibly non-integrable function appears.

Needed infrastructure: Rademacher sums in Banach spaces, order statistics of finitely many reals, the qqq-point inequality on general product spaces, and Theorem 13.2 for Bernoulli variables. The rearrangement and the qqq-point distance can be reused elsewhere. Contributions are welcome on any milestone, and on proving Proposition 13.3 from a stated form of Theorem 13.2.

Selected references

  • M. Talagrand, Concentration of measure and isoperimetric inequalities in product spaces, Publ. Math. IHÉS 81 (1995), 73–205. https://doi.org/10.1007/BF02699376
  • M. Ledoux, M. Talagrand, Characterization of the law of the iterated logarithm in Banach spaces, Ann. Probab. 16 (1988), 1242–1264. https://doi.org/10.1214/aop/1176991688
  • M. Ledoux, M. Talagrand, Probability in Banach Spaces: Isoperimetry and Processes, Springer, 1991. https://doi.org/10.1007/978-3-642-20212-4
  • M. Talagrand, Isoperimetry and integrability of the sum of independent Banach-space valued random variables, Ann. Probab. 17 (1989), 1546–1570. https://doi.org/10.1214/aop/1176991174
6 thms4 active usersReviewed
🏆Completed
Stochastic Systems·Captain: mikedeng1

Random Walks in a Random Environment 4: For Z_n = σ_n(1 + Z_{n−1}) with i.i.d. σ_n ≥ 0, (Z_1 + ⋯ + Z_n)/n → Eσ/(1 − Eσ) a.s. if Eσ < 1, and → ∞ if Eσ ≥ 1Research Paper

Motivation

A random walk in a random environment on Z\mathbb ZZ moves one step right with probability αx\alpha_xαx​ and one step left with probability 1−αx1-\alpha_x1−αx​ from site xxx, where the αx\alpha_xαx​ are themselves i.i.d. random variables. In his 1975 paper, F. Solomon (Ann. Probab. 3(1):1–31, DOI 10.1214/aop/1176996444) decided recurrence and transience of this walk and computed its speed. Section 4 of the paper isolates a piece of that analysis: the conditional mean time for the walk to cross one bond, given the environment.

Fix the environment, make 000 reflecting, and let μn\mu_nμn​ be the mean first-passage time from nnn to n+1n+1n+1. Conditioning on the first step gives μn=σnμn−1+(σn+1)\mu_n=\sigma_n\mu_{n-1}+(\sigma_n+1)μn​=σn​μn−1​+(σn​+1) with σn=(1−αn)/αn\sigma_n=(1-\alpha_n)/\alpha_nσn​=(1−αn​)/αn​, and Zn=12(μn−1)Z_n=\tfrac12(\mu_n-1)Zn​=21​(μn​−1) satisfies a linear recursion with random coefficients (p. 28). The sum Z1+⋯+ZnZ_1+\dots+Z_nZ1​+⋯+Zn​ is, up to an affine change, the expected time to reach level nnn in that environment. Recursions of the same form, Zn=AnZn−1+BnZ_n=A_nZ_{n-1}+B_nZn​=An​Zn−1​+Bn​ with i.i.d. coefficients, are the random difference equations studied by Kesten (Acta Math. 131, 1973) and Vervaat (Adv. Appl. Probab. 11, 1979); they recur in queueing (waiting-time recursions), insurance (discounted cash flows, "perpetuities") and the analysis of stochastic algorithms.

Setting

Let (Ω,F,P)(\Omega,\mathcal F,P)(Ω,F,P) be a probability space and σ1,σ2,…\sigma_1,\sigma_2,\dotsσ1​,σ2​,… independent, identically distributed, nonnegative real random variables. The system of difference equations (4.1) is

Z0=0,Zn=σn (1+Zn−1),n≥1.Z_0=0,\qquad Z_n=\sigma_n\,(1+Z_{n-1}),\quad n\ge1 .Z0​=0,Zn​=σn​(1+Zn−1​),n≥1.

In the Lean development Z σ n ω is defined by this recursion (module SolomonRWRE.DiffEq.System), so it is a function of σ1,…,σn\sigma_1,\dots,\sigma_nσ1​,…,σn​ and needs no existence hypothesis. Unfolding gives the closed form (4.3), Zn=∑j=1nσj⋯σnZ_n=\sum_{j=1}^n\sigma_j\cdots\sigma_nZn​=∑j=1n​σj​⋯σn​.

The mean is ν=E(σ)=∫σ1 dP∈[0,∞]\nu=E(\sigma)=\int\sigma_1\,dP\in[0,\infty]ν=E(σ)=∫σ1​dP∈[0,∞] (nu P σ); σ\sigmaσ need not be integrable, and ν=∞\nu=\inftyν=∞ is allowed. The products are ρj=σ1⋯σj\rho_j=\sigma_1\cdots\sigma_jρj​=σ1​⋯σj​ (rho), and the block sums of the proof are

Ykn=∑j=1n−k+1 ∏i=jj+k−1σi(Y σ k n),Y_k^n=\sum_{j=1}^{n-k+1}\ \prod_{i=j}^{j+k-1}\sigma_i \qquad(\texttt{Y σ k n}),Ykn​=j=1∑n−k+1​ i=j∏j+k−1​σi​(Y σ k n),

the sums of the products of kkk consecutive σ\sigmaσ's inside σ1,…,σn\sigma_1,\dots,\sigma_nσ1​,…,σn​. For one step of the proof the sequence is extended to a two-sided i.i.d. family {σj}j∈Z\{\sigma_j\}_{j\in\mathbb Z}{σj​}j∈Z​ (module TwoSided), with Sn=∑j=−∞nσj⋯σn∈[0,∞]S_n=\sum_{j=-\infty}^n\sigma_j\cdots\sigma_n\in[0,\infty]Sn​=∑j=−∞n​σj​⋯σn​∈[0,∞].

Formalization targets

Goal: Theorem (4.4)

lim⁡n→∞Z1+⋯+Znn={ν1−ν,ν<1,∞,ν≥1,almost surely.\lim_{n\to\infty}\frac{Z_1+\dots+Z_n}{n}=\begin{cases}\dfrac{\nu}{1-\nu}, & \nu<1,\\[2mm] \infty, & \nu\ge1,\end{cases}\qquad\text{almost surely.}n→∞lim​nZ1​+⋯+Zn​​=⎩⎨⎧​1−νν​,∞,​ν<1,ν≥1,​almost surely.

The goal fixes nothing beyond the paper's hypotheses: any law on [0,∞)[0,\infty)[0,∞), integrable or not. Both cases are almost-sure limits.

Milestones, in the order of the proof (pp. 28–29)

  1. The closed form (4.3), Zn=∑j=1nσj⋯σnZ_n=\sum_{j=1}^n\sigma_j\cdots\sigma_nZn​=∑j=1n​σj​⋯σn​.
  2. The regrouping Z1+⋯+Zn=Y1n+⋯+YnnZ_1+\dots+Z_n=Y_1^n+\dots+Y_n^nZ1​+⋯+Zn​=Y1n​+⋯+Ynn​.
  3. For fixed k≥1k\ge1k≥1, Ykn/n→νkY_k^n/n\to\nu^kYkn​/n→νk almost surely, in [0,∞][0,\infty][0,∞].
  4. lim inf⁡n(Z1+⋯+Zn)/n≥∑k≥1νk\liminf_n (Z_1+\dots+Z_n)/n\ge\sum_{k\ge1}\nu^kliminfn​(Z1​+⋯+Zn​)/n≥∑k≥1​νk almost surely, which settles ν≥1\nu\ge1ν≥1.
  5. For a two-sided family, Zn≤SnZ_n\le S_nZn​≤Sn​ pathwise, and Sn<∞S_n<\inftySn​<∞ almost surely when ν<1\nu<1ν<1.
  6. For ν<1\nu<1ν<1, lim sup⁡n1n∑k=1nZk≤ν/(1−ν)\limsup_n\frac1n\sum_{k=1}^nZ_k\le\nu/(1-\nu)limsupn​n1​∑k=1n​Zk​≤ν/(1−ν) almost surely.

Companion: Lemma (4.2)(i)–(ii)

If ∑nn−1P(ρn>1)<∞\sum_n n^{-1}P(\rho_n>1)<\infty∑n​n−1P(ρn​>1)<∞ then ∑jρj<∞\sum_j\rho_j<\infty∑j​ρj​<∞ a.s. and ZnZ_nZn​ converges in law to ∑jρj\sum_j\rho_j∑j​ρj​; if the series diverges, Zn→∞Z_n\to\inftyZn​→∞ in probability; when E(ln⁡σ)E(\ln\sigma)E(lnσ) is defined, the two cases are E(ln⁡σ)<0E(\ln\sigma)<0E(lnσ)<0 and E(ln⁡σ)≥0E(\ln\sigma)\ge0E(lnσ)≥0.

Significance

The result. Theorem (4.4) gives the almost-sure growth rate of the cumulative mean passage time of a random walk in a random environment: linear with slope ν/(1−ν)\nu/(1-\nu)ν/(1−ν) when E(σ)<1E(\sigma)<1E(σ)<1, superlinear otherwise. This is the averaged counterpart of the speed law of §1 of the paper (Theorem (1.16)), where the limiting speed (1−Eσ)/(1+Eσ)(1-E\sigma)/(1+E\sigma)(1−Eσ)/(1+Eσ) is exactly the reciprocal of 1+2ν/(1−ν)1+2\nu/(1-\nu)1+2ν/(1−ν). The superlinear case ν≥1\nu\ge1ν≥1 is the starting point of the paper's §4 limit laws for f(n)(Z1+⋯+Zn)f(n)(Z_1+\dots+Z_n)f(n)(Z1​+⋯+Zn​) (Lemma (4.5) to Theorem (4.9)), which are stated without proof. Lemma (4.2) concerns the single variable ZnZ_nZn​, and its dichotomy is a statement about perpetuities ∑jσ1⋯σj\sum_j\sigma_1\cdots\sigma_j∑j​σ1​⋯σj​.

Formalizing it. The theorem has been proved since 1975; what this mission adds is a machine-checked proof. No result about random difference equations, random products, or this paper is on the platform. The proof needs, for a nonnegative stationary ergodic sequence with possibly infinite mean, the almost-sure convergence of averages in [0,∞][0,\infty][0,∞]. That convergence is not in Mathlib in this form (Mathlib has the i.i.d. strong law for integrable variables, ProbabilityTheory.strong_law_ae), and Birkhoff's pointwise ergodic theorem is posed on the platform as an open item (Birkhoff1931.tendsto_birkhoffAverage_condExp). A completed mission therefore leaves behind a nonnegative, infinite-mean ergodic theorem for block products of i.i.d. sequences, which is reusable well beyond this paper.

Difficulty

The obvious argument fails at independence. ZnZ_nZn​ is a sum of products sharing the factor σn\sigma_nσn​, so Z1,Z2,…Z_1,Z_2,\dotsZ1​,Z2​,… are neither independent nor stationary, and no law of large numbers applies to them directly. The regrouping into YknY_k^nYkn​ produces sums of kkk-dependent block products, which are stationary but not independent, so the i.i.d. strong law does not apply either. Moreover there are nnn such groups at time nnn, so letting kkk and nnn grow together needs a uniform argument: Fatou's lemma gives only the lower bound. The upper bound requires a different object, SnS_nSn​, which is stationary because it looks infinitely far into the past, and it exists only because ν<1\nu<1ν<1 makes the backward series converge. A second trap is the infinite-mean case: σ\sigmaσ may be non-integrable, and every integrability-based ergodic or strong-law statement must be adapted to nonnegative variables with values in [0,∞][0,\infty][0,∞].

Formalization scope

  • Driving sequence. σ : ℕ → Ω → ℝ; the hypotheses (IsIIDNonneg) are on σ1,σ2,…\sigma_1,\sigma_2,\dotsσ1​,σ2​,… (the shifted family n ↦ σ (n + 1)): measurable, pointwise nonnegative, mutually independent (iIndepFun), and identically distributed with σ1\sigma_1σ1​ (IdentDistrib). σ0\sigma_0σ0​ is unused and unconstrained.
  • Expectations are lower Lebesgue integrals in [0,∞][0,\infty][0,∞] (∫⁻). A Bochner integral would return 000 for a non-integrable σ\sigmaσ, which would put ν=0<1\nu=0<1ν=0<1 and make the goal false; this trivializing reading is excluded by the definition of nu.
  • Limits are taken in [0,∞][0,\infty][0,∞]: the averages are mapped there by ENNReal.ofReal, which loses nothing because Zk≥0Z_k\ge0Zk​≥0. In the case ν<1\nu<1ν<1, ν/(1−ν)\nu/(1-\nu)ν/(1−ν) is computed in [0,∞][0,\infty][0,∞] and is finite.
  • Indices. The sums start at k=1k=1k=1 (Finset.Icc 1 n); in YknY_k^nYkn​ the upper limit n+1−kn+1-kn+1−k uses natural-number subtraction, which gives the empty sum for k>nk>nk>n.
  • Two-sided family. Milestone 5 is stated for a separate predicate IsIIDNonnegZ on σ : ℤ → Ω → ℝ; ZnZ_nZn​ is computed from its restriction to N\mathbb NN. This mirrors the page's "without loss of generality"; the goal keeps the one-sided family.
  • Lemma (4.2). One hypothesis is added: σ1\sigma_1σ1​ is not almost surely 111. For σ≡1\sigma\equiv1σ≡1 the series ∑n−1P(ρn>1)\sum n^{-1}P(\rho_n>1)∑n−1P(ρn​>1) vanishes while ∑ρj=∞\sum\rho_j=\infty∑ρj​=∞, so clause (i) fails as printed; the paper's proof cites its Lemma (1.6), which assumes nondegeneracy. "Converges in law" is Mathlib's TendstoInDistribution; "p−lim⁡Zn=∞p-\lim Z_n=\inftyp−limZn​=∞" is P(Zn≤M)→0P(Z_n\le M)\to0P(Zn​≤M)→0 for every MMM; E(ln⁡σ)E(\ln\sigma)E(lnσ) is the difference of the [0,∞][0,\infty][0,∞]-valued expectations of its positive and negative parts, required not to be both infinite, with ln⁡0=−∞\ln0=-\inftyln0=−∞.

Contributions welcome beyond the milestones: an ergodic theorem for nonnegative stationary sequences with values in [0,∞][0,\infty][0,∞]; the identity E(S1)=ν/(1−ν)E(S_1)=\nu/(1-\nu)E(S1​)=ν/(1−ν); and the construction of a two-sided i.i.d. extension of a one-sided sequence, which turns milestone 5 into a statement about the goal's sequence.

Selected references

  • F. Solomon, Random walks in a random environment, The Annals of Probability 3(1):1–31, 1975. https://doi.org/10.1214/aop/1176996444
  • H. Kesten, Random difference equations and renewal theory for products of random matrices, Acta Mathematica 131:207–248, 1973. https://doi.org/10.1007/BF02392040
  • W. Vervaat, On a stochastic difference equation and a representation of non-negative infinitely divisible random variables, Advances in Applied Probability 11(4):750–783, 1979. https://doi.org/10.2307/1426858
  • G. D. Birkhoff, Proof of the ergodic theorem, Proceedings of the National Academy of Sciences 17(12):656–660, 1931. https://doi.org/10.1073/pnas.17.12.656
9 thms4 active usersReviewed
Bandit AlgorithmsDynamic ProgrammingOperations Research·Captain: mikedeng1

Bandit Processes and Dynamic Allocation Indices: A Policy Is Optimal If and Only If It Almost Surely Continues a Bandit Process of Maximal Dynamic Allocation IndexResearch Paper

Motivation

The multi-armed bandit problem asks how to share effort sequentially among several independent projects, each of which evolves only while it is worked on. It models the scheduling of jobs on a single machine, the order in which to explore candidate sites, sequential clinical trials and the allocation of research effort among competing projects. Before 1972 the problem had no general solution: dynamic programming over the joint state of all projects is intractable as soon as there are more than a few of them.

J. C. Gittins and D. M. Jones (1974) announced, and Gittins (1979) set out in full, a solution of a striking form: each project can be assigned a number depending only on its own current state, its dynamic allocation index (DAI, now the Gittins index), and it is optimal to work on a project whose index is largest.

Timeline. Gittins and Jones presented the index result at the European Meeting of Statisticians in 1972 (proceedings 1974). Gittins (1979) gave the general discrete-time formulation, stated the Forwards Induction Theorem and the DAI Theorem without proof, and proved the attainment Lemma of its Section 4. Whittle (1980) gave a dynamic-programming proof through the retirement formulation, Weber (1992) the prevailing-charge interchange argument, and Tsitsiklis (1994) a short inductive proof. Gittins, Glazebrook and Weber (2011) is the standard monograph; Lattimore and Szepesvári (2020, Chapter 35) give a measure-theoretic treatment on general state spaces.

Setting

A bandit process DDD is a Markov chain x(0),x(1),…x(0), x(1), \dotsx(0),x(1),… on a measurable state space Θ\ThetaΘ with transition kernel PPP. Applying the continuation control at process time ttt earns the discounted reward atR(x(t))a^t R(x(t))atR(x(t)), with 0<a<10<a<10<a<1, and moves the chain one step; the freezing control leaves the state unchanged and earns nothing. For a stopping time τ\tauτ of the chain (possibly infinite) write

Rτ(D,x)=E{∑t=0τ−1atR(x(t)) ∣ x(0)=x},Wτ(D,x)=E{∑t=0τ−1at ∣ x(0)=x}.R_\tau(D,x)=E\Big\{\sum_{t=0}^{\tau-1}a^tR(x(t))\,\Big|\,x(0)=x\Big\},\qquad W_\tau(D,x)=E\Big\{\sum_{t=0}^{\tau-1}a^t\,\Big|\,x(0)=x\Big\}.Rτ​(D,x)=E{t=0∑τ−1​atR(x(t))​x(0)=x},Wτ​(D,x)=E{t=0∑τ−1​at​x(0)=x}.

The dynamic allocation index of DDD in state xxx is the largest reward per unit of discounted time obtainable by continuing for at least one step:

ν(D,x)=sup⁡τ>0ντ(D,x),ντ(D,x)=Rτ(D,x)Wτ(D,x).(2)\nu(D,x)=\sup_{\tau>0}\nu_\tau(D,x),\qquad \nu_\tau(D,x)=\frac{R_\tau(D,x)}{W_\tau(D,x)}. \tag{2}ν(D,x)=τ>0sup​ντ​(D,x),ντ​(D,x)=Wτ​(D,x)Rτ​(D,x)​.(2)

A simple family of alternative bandit processes consists of kkk bandit processes. At each stage n=0,1,2,…n=0,1,2,\dotsn=0,1,2,… a policy chooses exactly one of them to continue, possibly at random and as a function of the whole past; the others are frozen. A policy is optimal if it attains the supremum of the total expected discounted reward over all policies.

The Lean development uses the published objects markovChainMeasure P x (the chain law), IsPositiveStoppingTime, stoppedRatio P r α τ x (ντ\nu_\tauντ​), gittinsIndex P r α x (ν\nuν), MarkovBanditPolicy k S, markovBanditMeasure P π x n (the law of the first nnn stages) and markovBanditDiscountedValue P r α π x (the total expected discounted reward), with aaa written α and RRR written r.

Formalization targets

Goal: the DAI Theorem (Section 3, p. 154)

For every policy π\piπ and initial state vector xxx,

Vπ(x)=sup⁡π′Vπ′(x)  ⟺  ∀n, a.s. under the play of π from x:  πn({i:ν(xi(n))=max⁡jν(xj(n))})=1.V_\pi(x)=\sup_{\pi'}V_{\pi'}(x)\iff \forall n,\ \text{a.s. under the play of }\pi\text{ from }x:\ \ \pi_n\Big(\big\{i:\nu(x_i(n))=\max_j\nu(x_j(n))\big\}\Big)=1 .Vπ​(x)=π′sup​Vπ′​(x)⟺∀n, a.s. under the play of π from x:  πn​({i:ν(xi​(n))=jmax​ν(xj​(n))})=1.

Milestones

  1. Section 4, Lemma (p. 154). For Θ0={y:ν(D,y)<ν(D,x)}\Theta_0=\{y:\nu(D,y)<\nu(D,x)\}Θ0​={y:ν(D,y)<ν(D,x)} and τ\tauτ the first t≥1t\ge1t≥1 with x(t)∈Θ0x(t)\in\Theta_0x(t)∈Θ0​,
ντ(D,x)=ν(D,x).\nu_\tau(D,x)=\nu(D,x).ντ​(D,x)=ν(D,x).
  1. The "if" half of the DAI Theorem. A policy that selects a process of maximal index on every history is optimal: the published BanditAlgorithm.gittins_index_theorem, already Proved.
  2. Section 4, Corollary 1 (p. 155). For the MMM-horizon index νM(D,x)=sup⁡0<τ≤Mντ(D,x)\nu^M(D,x)=\sup_{0<\tau\le M}\nu_\tau(D,x)νM(D,x)=sup0<τ≤M​ντ​(D,x) (5), the rule that stops at the first t∈{1,…,M−1}t\in\{1,\dots,M-1\}t∈{1,…,M−1} with νM−t(D,x(t))<νM(D,x)\nu^{M-t}(D,x(t))<\nu^M(D,x)νM−t(D,x(t))<νM(D,x), and at MMM otherwise, attains νM(D,x)\nu^M(D,x)νM(D,x).

Significance

The result. The DAI Theorem replaces a dynamic program on the product of kkk state spaces by kkk one-dimensional optimal stopping problems. The "only if" half says more than that the index rule is one optimal policy: it characterizes every optimal policy, so any rule that departs from the index on a set of positive probability loses reward. The Lemma identifies the stopping rule that attains the index, which is how indices are computed in practice, and Corollary 1 is the finite-horizon version on which the paper's numerical algorithm (Section 7) rests.

Formalizing it. The sufficiency half is formalized: BanditAlgorithm.gittins_index_theorem (Lattimore–Szepesvári Theorem 35.9) is Proved on the platform, together with gittins_index_policy_dominates, gittins_stopping_ratio_le_index and gittins_epsilon_optimal_stopping. The attainment Lemma is Proved only for countable state spaces with bounded rewards (AllocationIndices.optimal_stopping_set, Gittins–Glazebrook–Weber Lemma 2.2), and the necessity half is on no platform item. This mission asks for the Lemma on general standard Borel state spaces with integrable rewards, the finite-horizon Corollary 1, and the full equivalence. The Forwards Induction Theorem of p. 153, which needs superprocesses and forwards induction policies, is outside its scope.

Difficulty

The "only if" direction is the new content, and it does not follow from the sufficiency theorem: knowing that one index policy is optimal says nothing about a policy that sometimes plays a lower-index process. The deviation must be shown to cost a strictly positive amount whenever it happens with positive probability, which requires a strict comparison that survives discounting, ties in the index and infinite stopping times. A second obstacle is measure-theoretic. On a general state space the index is a supremum over uncountably many stopping times, its measurability is not available, and the Lemma's stopping rule is a stopping time only if {y:ν(D,y)<ν(D,x)}\{y:\nu(D,y)<\nu(D,x)\}{y:ν(D,y)<ν(D,x)} is measurable; the arguments of the countable case, which treat each state separately, do not transfer.

Formalization scope

All objects are the published discounted kkk-armed Markov bandit of Def_GittinsIndex and the stopping-time quantities of Def_AllocationIndices_Index; time is indexed from 000, stopping times are N∪{∞}\mathbb N\cup\{\infty\}N∪{∞}-valued and adapted to the coordinate filtration, and the index is the real supremum over stopping times τ≥1\tau\ge1τ≥1. The explicit readings committed to are:

  • "Optimal" means attaining sup⁡π′Vπ′(x)\sup_{\pi'}V_{\pi'}(x)supπ′​Vπ′​(x) from the given initial state vector, over all randomized history-dependent policies; this supremum is finite because the values are bounded uniformly in the policy under the integrability hypothesis.
  • "At each stage … almost always" means: for every stage nnn, almost surely under the law of the first nnn stages of π\piπ from xxx, the selection kernel puts mass one on the processes of maximal current index. Requiring the index rule on every history would make "only if" false.
  • "The supremum of the total expected reward is finite" (p. 151) is read as Ex∑tat∣R(x(t))∣<∞E_x\sum_t a^t|R(x(t))|<\inftyEx​∑t​at∣R(x(t))∣<∞ for every state xxx.
  • "ν(D,⋅)\nu(D,\cdot)ν(D,⋅) is an X\mathcal XX-measurable function" (p. 151) is an explicit hypothesis of the Lemma and of the goal; the measurability of every νm(D,⋅)\nu^m(D,\cdot)νm(D,⋅) is a hypothesis of Corollary 1.
  • "The supremum is attained by setting Θ0\Theta_0Θ0​" means that the first hitting time of Θ0\Theta_0Θ0​ from time 111 on is a positive stopping time whose ratio equals the index.
  • The state space is standard Borel (the paper asks only for measurable singletons), and all processes share one kernel and one reward function; a family of different processes is encoded on the disjoint union of their state spaces.

A trivializing formalization is ruled out: the goal is not stated for index policies on every history (which would only restate the referenced theorem), the index is the published supremum over stopping times rather than a one-step reward or a finite-horizon quantity, competitors are not restricted to Markov or deterministic policies, and the Lemma's stopping rule starts at time 111, so it never stops at once. A one-point state space satisfies all hypotheses together and admits a policy.

Useful infrastructure, reusable beyond this mission: measurability of the Gittins index on standard Borel spaces, a measurable selection of a maximal-index arm, and an interchange argument for kkk-armed Markov bandits on general state spaces. Proofs of any milestone, and of either half of the goal, are welcome.

Selected references

  • J. C. Gittins, Bandit Processes and Dynamic Allocation Indices, J. R. Statist. Soc. B 41(2), 148–177, 1979. https://doi.org/10.1111/j.2517-6161.1979.tb01068.x
  • J. C. Gittins and D. M. Jones, A dynamic allocation index for the sequential design of experiments, in Progress in Statistics (J. Gani et al., eds.), North-Holland, 241–266, 1974.
  • P. Whittle, Multi-armed bandits and the Gittins index, J. R. Statist. Soc. B 42(2), 143–149, 1980. https://doi.org/10.1111/j.2517-6161.1980.tb01111.x
  • R. Weber, On the Gittins index for multiarmed bandits, Ann. Appl. Probab. 2(4), 1024–1033, 1992. https://doi.org/10.1214/aoap/1177005588
  • J. N. Tsitsiklis, A short proof of the Gittins index theorem, Ann. Appl. Probab. 4(1), 194–199, 1994. https://doi.org/10.1214/aoap/1177005207
  • J. C. Gittins, K. D. Glazebrook and R. Weber, Multi-armed Bandit Allocation Indices, 2nd ed., Wiley, 2011. https://doi.org/10.1002/9780470980033
  • T. Lattimore and C. Szepesvári, Bandit Algorithms, Cambridge University Press, 2020, Chapter 35. https://doi.org/10.1017/9781108571401
7 thms4 active usersReviewed
🏆Completed
Information TheoryOptimizationStatistics·Captain: mikedeng1

I-Divergence Geometry of Probability Distributions and Minimization Problems 1: Cyclic I-Projections onto Linear Sets of a Finite Space Converge to the I-Projection onto Their IntersectionResearch Paper

Motivation

Fitting a probability distribution to prescribed margins or moments is a basic task in statistics and information theory. Starting from a reference distribution, one can enforce one constraint at a time by choosing the closest distribution in I-divergence, then cycle through the constraints. In a contingency table this procedure is iterative proportional fitting. A single projection onto the intersection is the desired answer, but each step of the iteration sees only one constraint set. Csiszár's 1975 paper proves that cyclic I-projections converge to the joint I-projection when the underlying set is finite and the constraint sets are linear. This mission formalizes that result together with the geometric statements needed to identify its limit.

Setting

Let (X,X)(X,\mathcal X)(X,X) be a measurable space. A probability distribution (PD) is a probability measure on it. For PDs PPP and QQQ, the I-divergence I(P∥Q)I(P\|Q)I(P∥Q) is the Kullback–Leibler divergence: integrate the logarithm of the density of PPP relative to QQQ against PPP, with value +∞+\infty+∞ when PPP is not absolutely continuous with respect to QQQ or the integral diverges. It takes values in [0,+∞][0,+\infty][0,+∞] and vanishes exactly when P=QP=QP=Q. The logarithmic conventions are log⁡0=−∞\log0=-\inftylog0=−∞ and 0⋅(±∞)=00\cdot(\pm\infty)=00⋅(±∞)=0 (Csiszár 1975, (1.1)–(1.3)).

Given a PD RRR and a set E\mathcal EE of PDs that contains at least one PPP with finite I(P∥R)I(P\|R)I(P∥R), an I-projection QQQ of RRR onto E\mathcal EE minimizes I(P∥R)I(P\|R)I(P∥R) over P∈EP\in\mathcal EP∈E and itself has finite divergence. The variation distance is the L1L^1L1 distance between the densities of two PDs relative to a common dominating measure. A set of PDs is linear here when every real affine combination of two members that is still a PD also belongs to the set. This includes distributions satisfying fixed linear moment constraints (Csiszár 1975, pp. 147 and 150–151).

For the goal, XXX is finite. Choose k≥1k\ge1k≥1 linear sets E1,…,Ek\mathcal E_1,\ldots,\mathcal E_kE1​,…,Ek​, and write E=⋂i=1kEi\mathcal E=\bigcap_{i=1}^k\mathcal E_iE=⋂i=1k​Ei​. Beginning with Q0=RQ_0=RQ0​=R, let QnQ_nQn​ be the I-projection of Qn−1Q_{n-1}Qn−1​ onto Ei\mathcal E_iEi​ when n=mk+in=mk+in=mk+i and 1≤i≤k1\le i\le k1≤i≤k. The common feasible set is assumed nonempty and contains a PD absolutely continuous with respect to RRR. These assumptions ensure that the intended finite-divergence projection problem is present (Csiszár 1975, Theorem 3.2).

Formalization targets

The goal is the convergence statement of Theorem 3.2:

∀x∈X,Qn({x})⟶Q({x}),Q=proj⁡EI(R).\forall x\in X,\qquad Q_n(\{x\})\longrightarrow Q(\{x\}), \quad Q=\operatorname{proj}^{I}_{\mathcal E}(R).∀x∈X,Qn​({x})⟶Q({x}),Q=projEI​(R).

The goal also asserts existence of the cyclic sequence. On finite XXX, convergence of every point mass is equivalent to convergence in variation distance. The target fixes no rate. Its milestones include existence on variation-closed convex sets (Theorem 2.1), the identity (2.6) and the tangent-hyperplane Lemma 2.1, the Pythagorean characterization and transitivity of I-projections (Theorems 2.2–2.3), the density form under general linear constraints (Theorem 3.1) and the finite-moment exponential-density characterization (Corollary 3.1(A)), and the finite-space identity used in Theorem 3.2's proof. The two displayed identities in that proof are also targets:

I(Q∥R)=I(Q∥Qn)+∑i=1nI(Qi∥Qi−1),I(P∥Qn)=I(P∥Q)+I(Q∥Qn)(P∈E).I(Q\|R)=I(Q\|Q_n)+\sum_{i=1}^{n}I(Q_i\|Q_{i-1}),\qquad I(P\|Q_n)=I(P\|Q)+I(Q\|Q_n)\quad(P\in\mathcal E).I(Q∥R)=I(Q∥Qn​)+i=1∑n​I(Qi​∥Qi−1​),I(P∥Qn​)=I(P∥Q)+I(Q∥Qn​)(P∈E).

They specify the divergence relationships that the eventual convergence theorem must preserve (Csiszár 1975, (3.14)–(3.15)).

Significance

The theorem justifies enforcing finitely many linear distribution constraints one at a time while still obtaining the same I-projection as a simultaneous minimization. It gives a precise limit for the iterative proportional fitting procedure discussed by Csiszár, including cases in which the initial distribution has zero probabilities; strict positivity of RRR is not required (Csiszár 1975, p. 155). The accompanying identities make the relation between a partial iterate and the final solution explicit. The result was proved in 1975. The work here is to provide machine-checked proofs of that known result and its supporting geometry, while keeping the source's extended-value and boundary conventions visible.

Difficulty

Successive projections minimize divergence over different sets. Minimization for one set by itself does not identify the minimizer over their intersection, and ordinary metric projection facts do not directly apply because I-divergence is asymmetric and can be infinite. The paper's statements need to handle zero densities and feasible distributions at infinite divergence without turning extended-real expressions into default values. On the finite space, convergence of a subsequence is available, but the formal claim must still identify every possible limit as the joint I-projection. These are the mathematical and representation difficulties faced by a complete proof (Csiszár 1975, §§2–3).

Formalization scope

Lean uses Measure X and Mathlib's InformationTheory.klDiv : ℝ≥0∞ for I(P∥Q)I(P\|Q)I(P∥Q). A projection is a finite-divergence minimizer over the stated feasible set; no real-valued infimum or implicit arbitrary witness replaces it. Variation distance uses the densities relative to P+QP+QP+Q, which dominates both PDs and gives the value of (1.6). Variation closure is expressed by sequential convergence in this distance. The integral ∫log⁡qR dP\int\log q_R\,dP∫logqR​dP is represented in extended reals using positive and negative logarithmic parts and ENNReal.log; the theorem hypotheses keep the indeterminate +∞−(+∞)+\infty-(+\infty)+∞−(+∞) case out. Density equations are RRR-almost everywhere, and a claim that QQQ has an RRR-density explicitly includes Q≪RQ\ll RQ≪R.

For moment constraints, every integral is accompanied by integrability. Real affine combinations are tested through their values on measurable sets; the candidate combination must be a PD. The goal assumes a finite carrier with measurable singleton sets and k≥1k\ge1k≥1. Lean indexes the sets by Fin k from zero, so Qs (n+1) is projected onto the set at n % k; this is the paper's rule (3.13) shifted by one. The goal asserts existence of a recursive sequence before speaking about every such sequence, and the required feasible witness has P≪RP\ll RP≪R. These choices rule out an empty cyclic schedule, junk integrals, and a vacuous convergence assertion. Reusable contributions include the variation-distance and I-projection interfaces, their Pythagorean geometry, and the finite-moment density characterization; the mission's main closure remains Theorem 3.2.

Selected references

  • I. Csiszár, I-Divergence Geometry of Probability Distributions and Minimization Problems, The Annals of Probability 3(1), 146–158 (1975). DOI: 10.1214/aop/1176996454.
14 thms4 active usersReviewed
🏆Completed
CombinatoricsOperations ResearchTheoretical Computer Science·Captain: mikedeng1

An Optimal Algorithm for On-line Bipartite Matching 1: RANKING Finds a Matching of Expected Size at Least n(1 − 1/e) − o(n) on Every Graph with a Perfect MatchingResearch Paper

Motivation

Online matching describes allocation when requests must be answered as they arrive. A matching decision uses only the edges revealed so far and cannot be revised when later requests appear. In the bipartite setting studied by Karp, U. Vazirani, and V. Vazirani, the arriving vertices are girls and the possible partners are boys. Even when the full graph has a perfect matching, a fixed greedy priority can leave many girls unmatched. The paper introduced RANKING, which randomly chooses the boys' priority order once and then uses that order for every arrival.

The question is quantitative: how many pairs does RANKING guarantee in expectation against a graph and an arrival order chosen before its random ranking? The paper's target is a fraction approaching 1−1/e1-1/e1−1/e of the nnn pairs in a perfect matching. Its printed Theorem 1 concerns an auxiliary algorithm called EARLY; the RANKING statement follows the intended chain through Lemmas 3 and 5. The original EARLY analysis has a gap for general upper-triangular matrices, so this mission states the RANKING target and the earlier, unaffected lemmas separately. This distinction matters because an assertion about EARLY would be a different formalization target.

Setting

A bipartite graph has a boy side UUU and a girl side VVV, each with nnn vertices. An edge (u,v)(u,v)(u,v) means that boy uuu may be paired with girl vvv. A matching is a collection of edges in which no boy or girl occurs twice. The standing hypothesis for the performance guarantee is that the graph has a perfect matching: some bijection from boys to girls selects an edge for every boy. The graph is otherwise arbitrary.

Girls arrive in a predetermined order. When girl vvv arrives, only her incident edges are revealed. RANKING first chooses a uniformly random permutation π\piπ of the boys. For each arriving girl, it selects the highest-ranked adjacent boy who is still unmatched, if one exists. Write MR(G,π)M_{\mathrm R}(G,\pi)MR​(G,π) for the final matching. The expected size is the finite average over all n!n!n! rankings. The guarantee must hold for every graph and every predetermined arrival order; relabeling girls lets the formal statement fix their order to n,n−1,…,1n,n-1,\ldots,1n,n−1,…,1.

The paper also uses a dual rows-arrive view. Boys arrive in an order and choose among eligible girls, whose priority order is fixed. This is the same greedy rule with the sides exchanged. For its triangular reduction, the columns are numbered 1,…,n1,\ldots,n1,…,n and column nnn has highest priority. An upper-triangular matrix with unit diagonal is a graph with every edge (i,i)(i,i)(i,i) and with an edge (i,j)(i,j)(i,j) only when i≤ji\le ji≤j. On such a graph, the auxiliary algorithm EARLY declines to match row iii if column iii is already covered by EARLY's own matching. For any matching MMM, D(M)D(M)D(M) denotes the indices for which both row iii and column iii are covered.

Formalization targets

RANKING guarantee

For every ε>0\varepsilon>0ε>0, one threshold NNN must work for every size n≥Nn\ge Nn≥N and every graph GGG with a perfect matching:

Eπ∼Unif(Sn)∣MR(G,π)∣≥(1−e−1−ε)n.\mathbb E_{\pi\sim\mathrm{Unif}(S_n)}|M_{\mathrm R}(G,\pi)| \ge (1-e^{-1}-\varepsilon)n.Eπ∼Unif(Sn​)​∣MR​(G,π)∣≥(1−e−1−ε)n.

This is the uniform lower-bound reading of the paper's n(1−1/e)−o(n)n(1-1/e)-o(n)n(1−1/e)−o(n) target. It does not prescribe a finite-nnn additive constant. The mission goal is the RANKING assertion drawn from the paper's Theorem 1 and Lemmas 3 and 5. The six milestones formalize the paper's Lemmas 1–5 and the corollary to Lemma 4, in their source order. They cover the side-exchange identity, arbitrary refusal algorithms, the triangular reduction, the matching-count identity, its expected form, and the pointwise comparison with EARLY.

Significance

The guarantee gives a concrete worst-case floor for a simple randomized allocation rule: as the graph size grows, RANKING matches at least an asymptotic 1−1/e1-1/e1−1/e fraction of the pairs available in a perfect matching, in expectation. The order is selected before the random permutation, matching the paper's performance measure. The lower bound remains meaningful on sparse graphs and does not rely on a density assumption. The original paper also studies the limit on what any randomized online algorithm can guarantee; that upper-bound result is treated in a separate mission.

A formal development here would provide reusable finite definitions for online greedy matching, arbitrary state-dependent refusal, fixed-order duality, and uniform expectation over permutations. The goal is a theorem statement awaiting a machine-checked proof; compiling the draft declarations verifies their Lean syntax and types, not their truth. The local mission separates the valid early structural statements from later statements whose published EARLY argument does not justify them on general upper-triangular graphs.

Difficulty

The random permutation does not make the fate of different vertices independent. Matching one girl removes a boy who might be essential to a later girl, so a per-arrival probability estimate cannot simply be added across all arrivals. The dual and triangular views capture useful structure, but turning that structure into a uniform bound for every graph is the main obstacle. In particular, reasoning about EARLY as if its matched-column set had the same monotonicity as unrestricted RANKING fails on some upper-triangular matrices. A proof of the goal must establish the RANKING guarantee without treating those later EARLY statements as available facts.

Formalization scope

Boys and girls are both Fin n; an adjacency matrix is a relation between them. A published predicate represents a perfect matching as an edge-preserving bijection. The generic greedy run processes arrival times in increasing order and interprets a smaller priority index as higher rank. It makes a new matching decision only from the current matching and the arriving vertex. RANKING's returned edges are consistently ordered as (boy, girl), even though girls arrive. The dual run has rows arriving in an arbitrary permutation and takes column n−1n-1n−1 as highest priority in Lean's zero-based numbering. EARLY's refusal test consults the matching constructed by EARLY itself.

All expectations are finite averages over Equiv.Perm (Fin n), scaled by 1/n!1/n!1/n!. The formal o(n)o(n)o(n) claim is ∀ε>0, ∃N, ∀n≥N, ∀G\forall\varepsilon>0,\ \exists N,\ \forall n\ge N,\ \forall G∀ε>0, ∃N, ∀n≥N, ∀G with a perfect matching, the displayed lower bound. Placing NNN before GGG preserves the worst-case meaning. The perfect-matching condition is essential: the empty graph cannot satisfy a positive linear guarantee. The triangular matrix in Lemma 3 retains every diagonal edge, so a zero matrix cannot witness the reduction. These conventions rule out vacuous versions of the goal and reduction.

The definitions of partial matching, covered vertices, greedy run, RANKING, EARLY, and uniform average are part of the mission. A complete proof may build further finite counting and permutation machinery; such lemmas can be shared beyond this paper. Contributions toward the six milestone statements, the goal, and faithful supporting results are in scope. Lemmas 6–12 and the printed EARLY Theorem 1 are outside this mission because their analysis depends on claims that fail for some matrices allowed by their surrounding assumptions.

Selected references

  • Richard M. Karp, Umesh V. Vazirani, and Vijay V. Vazirani, An Optimal Algorithm for On-line Bipartite Matching, Proceedings of the 22nd Annual ACM Symposium on Theory of Computing, 1990, pp. 352–358. DOI: 10.1145/100216.100262.
10 thms4 active usersReviewed
🏆Completed
Algorithmic Game TheoryOperations Research·Captain: mikedeng1

Supply Chain Coordination with Contracts IV: Competing Newsvendors with Proportional Allocation Have a Unique Equilibrium, and a Coordinating Buy-Back Gives the Supplier ((p(n − 1) + b)/(pn))Π(q°)Textbook

Why competing retailers change the contracting problem

A supplier who sells through a single newsvendor retailer faces double marginalization: the retailer bears the whole cost of leftover stock but earns only the retail margin, so under a plain wholesale-price contract he orders less than the integrated supply chain would. The literature reviewed in G. P. Cachon's chapter Supply Chain Coordination with Contracts (Handbooks in OR & MS, vol. 11, 2003) shows that buy-back, revenue-sharing and related contracts correct this distortion. Section 6.5 asks what happens when the supplier sells through several retailers who compete for the same customers.

Competition can push in the opposite direction. When customers buy wherever stock is available, a retailer who stocks more also takes demand from his rivals, and he does not count that loss as a cost. This demand-stealing effect pushes the retailers towards over-ordering, which offsets double marginalization. §6.5.1 makes this precise in the proportional allocation model, in which total demand is split among the retailers in proportion to their inventories. The model goes back to the deterministic version of Wang and Gerchak (2001); related allocation models are those of Lippman and McCardle (1997) and Anupindi and Bassok (1999), and Mahajan and van Ryzin (2001) observe the same mitigation of the need for coordinating contracts.

This mission formalizes §6.5.1 of the chapter's January 2003 third draft, pp. 48–53.

The proportional allocation model

There are n≥2n \ge 2n≥2 retailers and one supplier. Total retail demand D≥0D \ge 0D≥0 is random, with distribution function FFF that is differentiable on (0,∞)(0,\infty)(0,∞) with density fff, strictly increasing on [0,∞)[0,\infty)[0,∞), and satisfies F(0)=0F(0)=0F(0)=0. The retail price is ppp and the supplier's unit production cost is ccc, with 0<c<p0 < c < p0<c<p. Goodwill costs, the salvage value and the retailers' handling cost are zero.

Retailer iii orders qi≥0q_i \ge 0qi​≥0. Write q=∑jqjq = \sum_j q_jq=∑j​qj​ and q−i=q−qiq_{-i} = q - q_iq−i​=q−qi​. Retailer iii receives the demand Di=(qi/q)DD_i = (q_i/q)DDi​=(qi​/q)D. Under a buy-back contract (w,b)(w, b)(w,b) he pays www per unit ordered and is refunded bbb per unit left over; b=0b = 0b=0 is the wholesale-price contract. His expected profit is

πi(qi,q−i)=E[pmin⁡(qi,Di)+b(qi−Di)+]−wqi=(p−w)qi−(p−b)qiq∫0qF(x) dx.\pi_i(q_i, q_{-i}) = \mathbb E\big[p\min(q_i, D_i) + b(q_i - D_i)^+\big] - wq_i = (p-w)q_i - (p-b)\frac{q_i}{q}\int_0^q F(x)\,dx .πi​(qi​,q−i​)=E[pmin(qi​,Di​)+b(qi​−Di​)+]−wqi​=(p−w)qi​−(p−b)qqi​​∫0q​F(x)dx.

Because total sales min⁡(q,D)\min(q, D)min(q,D) depend only on the total stock, the integrated chain earns Π(q)=pS(q)−cq\Pi(q) = pS(q) - cqΠ(q)=pS(q)−cq with S(q)=E[min⁡(q,D)]S(q) = \mathbb E[\min(q,D)]S(q)=E[min(q,D)], and its optimal stock qoq^oqo solves the newsvendor equation F(qo)=(p−c)/pF(q^o) = (p-c)/pF(qo)=(p−c)/p, Eq. (20). A Nash equilibrium is a profile of orders in which each qi∗q^*_iqi∗​ maximizes πi(⋅,q−i∗)\pi_i(\cdot, q^*_{-i})πi​(⋅,q−i∗​) over all orders x≥0x \ge 0x≥0. A contract coordinates the chain when its equilibrium total order is qoq^oqo.

Two contract prices appear on p. 52: the wholesale price w^(q)=p(1−1nF(q)−n−1n⋅1q∫0qF)\widehat w(q) = p\big(1 - \tfrac1n F(q) - \tfrac{n-1}{n}\cdot\tfrac1q\int_0^qF\big)w(q)=p(1−n1​F(q)−nn−1​⋅q1​∫0q​F) that induces total stock qqq, and the buy-back wholesale price

wb(b)=p−(p−b)[1n⋅p−cp+n−1n⋅1qo∫0qoF(x) dx].w_b(b) = p - (p-b)\left[\frac1n\cdot\frac{p-c}{p} + \frac{n-1}{n}\cdot\frac{1}{q^o}\int_0^{q^o}F(x)\,dx\right].wb​(b)=p−(p−b)[n1​⋅pp−c​+nn−1​⋅qo1​∫0qo​F(x)dx].

Formalization targets

Goal: the coordinating buy-back contract (pp. 51–53)

For n≥2n \ge 2n≥2, b<pb < pb<p and qoq^oqo solving (20), with w=wb(b)w = w_b(b)w=wb​(b): qoq^oqo maximizes Π\PiΠ; b<wb(b)<pb < w_b(b) < pb<wb​(b)<p; the profile in which every retailer orders qo/nq^o/nqo/n is the unique Nash equilibrium; and at that equilibrium

πi=p−bpn2 Π(qo),πs=wqo−cqo−b E[(qo−D)+]=p(n−1)+bpn Π(qo).\pi_i = \frac{p-b}{pn^2}\,\Pi(q^o), \qquad \pi_s = wq^o - cq^o - b\,\mathbb E[(q^o-D)^+] = \frac{p(n-1)+b}{pn}\,\Pi(q^o).πi​=pn2p−b​Π(qo),πs​=wqo−cqo−bE[(qo−D)+]=pnp(n−1)+b​Π(qo).

Milestones

The milestones are the section's own claims, in attack order: the newsvendor characterization (20); the inequality 1q∫0qF<F(q)\frac1q\int_0^qF < F(q)q1​∫0q​F<F(q); the closed form of πi\pi_iπi​ and its strict concavity in the retailer's own order (p. 50); the first-order condition and Eq. (21); the monotonicity and limits of the left side LnL_nLn​ of Eq. (22), giving a unique root for b<w<pb < w < pb<w<p; the unique, symmetric Nash equilibrium for every b<w<pb < w < pb<w<p; the increase of the equilibrium total in nnn; that w^(q)\widehat w(q)w(q) induces qqq, that w^(qo)>c\widehat w(q^o) > cw(qo)>c, and that the supplier's profit under wholesale pricing has negative slope at qoq^oqo; the coordinating price wb(b)w_b(b)wb​(b) with wb(b)>w^(qo)w_b(b) > \widehat w(q^o)wb​(b)>w(qo) for b>0b > 0b>0; and the ratio πs(qo,wb(0),0)/Π(qo)=(n−1)/n\pi_s(q^o, w_b(0), 0)/\Pi(q^o) = (n-1)/nπs​(qo,wb​(0),0)/Π(qo)=(n−1)/n (p. 53). A companion item states the endpoint b=pb = pb=p, at which the supplier takes all of Π(qo)\Pi(q^o)Π(qo).

Significance

The section answers two questions. First, with competing retailers a plain wholesale-price contract can coordinate the chain and still leave the supplier a positive margin, which a single retailer never allows. Second, that contract is not the supplier's best wholesale price, and it fixes a single division of profit. Buy-back contracts remove both limitations: the family (wb(b),b)(w_b(b), b)(wb​(b),b) coordinates for every b<pb < pb<p and moves the supplier's share continuously from (n−1)/n(n-1)/n(n−1)/n to all of Π(qo)\Pi(q^o)Π(qo). The ratio (n−1)/n(n-1)/n(n−1)/n also measures how little a coordinating contract adds when many retailers compete (80% of the optimal profit at n=5n = 5n=5).

The results are proved in the chapter, mostly by short computations, and none of them has been machine-checked. The formalization makes explicit what the page leaves implicit: that no equilibrium has a retailer ordering zero, the limits behind "from 0 to 1", and the density condition behind the strict sign on p. 52. It also produces a reusable proportional-allocation game and a Nash-equilibrium predicate for nonnegative real strategies.

Difficulty

The algebraic identities (the profits at the coordinating contract, the ratio (n−1)/n(n-1)/n(n−1)/n, the derivative at qoq^oqo) are routine once πi\pi_iπi​ has its closed form. The closed form itself requires computing the expectation with proportional shares and identifying E[min⁡(q,D)]\mathbb E[\min(q,D)]E[min(q,D)] with q−∫0qFq - \int_0^q Fq−∫0q​F.

The real obstacle is the uniqueness of the equilibrium. The page argues from first-order conditions, which describe only interior best responses. A complete proof must show that each retailer's profit is strictly concave in his own order, that no retailer orders zero in equilibrium, and that the all-zero profile (where πi\pi_iπi​ has q=0q = 0q=0 in a denominator) is not an equilibrium. Strict concavity is the delicate step. The second derivative mixes the density with the term 2q−iq3(qF(q)−∫0qF)\frac{2q_{-i}}{q^3}\big(qF(q) - \int_0^qF\big)q32q−i​​(qF(q)−∫0q​F), and concavity has to be established on the closed half-line, including the boundary qi=0q_i = 0qi​=0.

Formalization scope

Retailers are indexed by Fin n with n≥2n \ge 2n≥2 (the page's n>1n > 1n>1). The comparison in nnn also allows a single retailer, as the page does. Orders are real numbers qi≥0q_i \ge 0qi​≥0, and best responses range over all x≥0x \ge 0x≥0. The demand law is a probability measure on R\mathbb RR carried by [0,∞)[0,\infty)[0,∞) with finite mean, FFF is its cdf, and fff is a field with HasDerivAt F (f y) y for y>0y > 0y>0. These are the chapter's standing assumptions (p. 7). The condition c>0c > 0c>0 is needed for (20) to have a solution. The integrated optimum qoq^oqo enters each theorem through the hypothesis F(qo)=(p−c)/pF(q^o) = (p-c)/pF(qo)=(p−c)/p, which determines it uniquely. Transfers run from the retailers to the supplier.

Added or made explicit relative to the page: b<pb < pb<p in the concavity claim (at b=pb = pb=p the profit is linear); f(qo)>0f(q^o) > 0f(qo)>0 for the strict sign of the supplier's marginal profit; positivity of the total order wherever 1q∫0qF\frac1q\int_0^qFq1​∫0q​F appears. The page's printed slips (the first-order condition rescaled by q∗/(p−b)q^*/(p-b)q∗/(p−b), "F(qo)=(p−c)/cF(q^o) = (p-c)/cF(qo)=(p−c)/c", "w(b)w(b)w(b)") are corrected in the statements and kept in the milestone quotes.

Ruled out as trivializing: w^\widehat ww and wbw_bwb​ are the printed formulas, not "the price at which qoq^oqo is an equilibrium"; the supplier's profit is computed from the transfers, not as Π\PiΠ minus the retailers' profits; the equilibrium statement quantifies over all nonnegative deviations, so restricting attention to interior or symmetric profiles is not an option.

A complete development needs interval integrals of a cdf, the fundamental theorem of calculus for q↦∫0qFq \mapsto \int_0^qFq↦∫0q​F, and strict concavity from a strictly decreasing derivative. The proportional-allocation game and the Nash predicate are reusable for the other allocation models of §6.5. No platform item is referenced: the competing-retailer game of Cachon and Lariviere (2005), RevShareCoord.Competing.*, uses deterministic revenue functions and is a different model.

Selected references

  • G. P. Cachon, Supply Chain Coordination with Contracts, in S. Graves and T. de Kok (eds.), Handbooks in Operations Research and Management Science, vol. 11, North-Holland, 2003; read in the author's 3rd draft (Jan. 2003), §6.5.1. https://doi.org/10.1016/S0927-0507(03)11006-7
  • Y. Wang and Y. Gerchak, Supply chain coordination when demand is shelf-space dependent, Manufacturing & Service Operations Management 3(1), 2001, 82–87. https://doi.org/10.1287/msom.3.1.82.9998
  • S. A. Lippman and K. F. McCardle, The competitive newsboy, Operations Research 45(1), 1997, 54–65. https://doi.org/10.1287/opre.45.1.54
  • S. Mahajan and G. van Ryzin, Inventory competition under dynamic consumer choice, Operations Research 49(5), 2001, 646–657. https://doi.org/10.1287/opre.49.5.646.10603
  • G. P. Cachon and M. A. Lariviere, Supply chain coordination with revenue-sharing contracts: strengths and limitations, Management Science 51(1), 2005, 30–44. https://doi.org/10.1287/mnsc.1040.0215
18 thms4 active usersReviewed
🏆Completed
CombinatoricsStatistics·Captain: mikedeng1

Two Moments Suffice for Poisson Approximations: The Chen-Stein Method 2: A Dependent Bernoulli Process Is Within 2(2b₁+2b₂+b₃) of a Poisson Process in Total VariationResearch Paper

Motivation

Many counting problems in probability, combinatorics and statistics ask for the law of the number of occurrences of rare, weakly dependent events: long head runs in coin tossing, matching segments between two DNA sequences, isolated vertices or small subgraphs in a random graph, clusters of points in a scan statistic. Chen's 1975 adaptation of Stein's method gives an explicit error bound for approximating such a count by a Poisson law. Arratia, Goldstein and Gordon (Ann. Probab. 17 (1989) 9–25) recast it in terms of a neighbourhood of dependence for each event. In this form the error bound needs only the first two moments of the indicators whenever each event is independent of the events outside its neighbourhood. The paper's examples (head runs, DNA sequence matching, birthday-type problems) made this form standard in applied probability.

The paper has two main results. Theorem 1 bounds the distance between the law of the total count and a Poisson law; it is essentially contained in Chen (1975). Theorem 2, which the authors describe as new, is a process version. It compares the joint law of all the indicators, that is, where the events occur, with a Poisson process. Theorem 3, a corollary, compares a dependent family of events with an independent family having the same marginal probabilities. This mission formalizes Theorem 2, the finite-dimensional bound it is derived from, the steps of its proof in §6, and Theorem 3.

Setting

Let (Ω,F,P)(\Omega,\mathcal F,P)(Ω,F,P) be a probability space and III an index set. For α∈I\alpha\in Iα∈I let XαX_\alphaXα​ be a Bernoulli random variable, Xα∈{0,1}X_\alpha\in\{0,1\}Xα​∈{0,1}, with

pα=P(Xα=1)>0,λ=∑α∈Ipα∈(0,∞).p_\alpha=P(X_\alpha=1)>0,\qquad \lambda=\sum_{\alpha\in I}p_\alpha\in(0,\infty).pα​=P(Xα​=1)>0,λ=α∈I∑​pα​∈(0,∞).

For each α\alphaα fix a set Bα⊆IB_\alpha\subseteq IBα​⊆I with α∈Bα\alpha\in B_\alphaα∈Bα​, the neighbourhood of dependence of α\alphaα. Put pαβ=E(XαXβ)p_{\alpha\beta}=E(X_\alpha X_\beta)pαβ​=E(Xα​Xβ​) and

b1=∑α∈I∑β∈Bαpαpβ,b2=∑α∈I∑α≠β∈Bαpαβ,b3=∑α∈IE∣E{Xα−pα∣σ(Xβ:β∈I−Bα)}∣.b_1=\sum_{\alpha\in I}\sum_{\beta\in B_\alpha}p_\alpha p_\beta,\qquad b_2=\sum_{\alpha\in I}\sum_{\alpha\neq\beta\in B_\alpha}p_{\alpha\beta},\qquad b_3=\sum_{\alpha\in I}E\bigl|E\{X_\alpha-p_\alpha\mid\sigma(X_\beta:\beta\in I-B_\alpha)\}\bigr|.b1​=α∈I∑​β∈Bα​∑​pα​pβ​,b2​=α∈I∑​α=β∈Bα​∑​pαβ​,b3​=α∈I∑​E​E{Xα​−pα​∣σ(Xβ​:β∈I−Bα​)}​.

Roughly, b1b_1b1​ measures the size of the neighbourhoods, b2b_2b2​ the expected number of neighbours of an occurrence, and b3b_3b3​ the dependence between an event and the events outside its neighbourhood. In many applications XαX_\alphaXα​ is independent of σ(Xβ:β∉Bα)\sigma(X_\beta:\beta\notin B_\alpha)σ(Xβ​:β∈/Bα​), so b3=0b_3=0b3​=0.

The dependent Bernoulli process is X=(Xα)α∈I\mathbf X=(X_\alpha)_{\alpha\in I}X=(Xα​)α∈I​, a random element of Z+I\mathbb Z_+^IZ+I​. The Poisson process on III with intensity p(⋅)p_{(\cdot)}p(⋅)​ is Y=(Yα)α∈I\mathbf Y=(Y_\alpha)_{\alpha\in I}Y=(Yα​)α∈I​ with the YαY_\alphaYα​ mutually independent and YαY_\alphaYα​ Poisson with mean pαp_\alphapα​.

Distances are measured in the paper's total variation norm,

∥L(U)−L(V)∥=sup⁡∥h∥=1∣Eh(U)−Eh(V)∣=2sup⁡A∣P(U∈A)−P(V∈A)∣,\|\mathcal L(U)-\mathcal L(V)\|=\sup_{\|h\|=1}|Eh(U)-Eh(V)|=2\sup_A|P(U\in A)-P(V\in A)|,∥L(U)−L(V)∥=∥h∥=1sup​∣Eh(U)−Eh(V)∣=2Asup​∣P(U∈A)−P(V∈A)∣,

which is twice the usual total variation distance.

Formalization targets

Goal: Theorem 2

∥L(X)−L(Y)∥≤2(2b1+2b2+b3).\|\mathcal L(\mathbf X)-\mathcal L(\mathbf Y)\|\le2(2b_1+2b_2+b_3).∥L(X)−L(Y)∥≤2(2b1​+2b2​+b3​).

The bound has no dependence on λ\lambdaλ and no assumption on III beyond countability (which the setting forces).

Milestones, in attack order

  1. TiSih=hT_iS_ih=hTi​Si​h=h (§6, p. 22). For functions on Z+d\mathbb Z_+^dZ+d​, the coordinatewise Stein operators (Tif)(j)=jif(j)−λif(j+ei)(T_if)(j)=j_if(j)-\lambda_if(j+e_i)(Ti​f)(j)=ji​f(j)−λi​f(j+ei​) and (Sih)(j+ei)=−(λiP(Zi=ji))−1E{h(j1,…,Zi,…,jd)1(Zi≤ji)}(S_ih)(j+e_i)=-(\lambda_iP(Z_i=j_i))^{-1}E\{h(j_1,\dots,Z_i,\dots,j_d)1(Z_i\le j_i)\}(Si​h)(j+ei​)=−(λi​P(Zi​=ji​))−1E{h(j1​,…,Zi​,…,jd​)1(Zi​≤ji​)} satisfy TiSih=hT_iS_ih=hTi​Si​h=h.
  2. Bounds on fi=Si(h−Pih)f_i=S_i(h-P_ih)fi​=Si​(h−Pi​h) (§6, p. 22), where PihP_ihPi​h averages hhh over a Poisson(λi\lambda_iλi​) iii-th coordinate. For ∥h∥≤1\|h\|\le1∥h∥≤1, ∥fi∥<2(1∧1.4λi−1/2)\|f_i\|<2(1\wedge1.4\lambda_i^{-1/2})∥fi​∥<2(1∧1.4λi−1/2​) and ∥Δkfi∥<4(1∧1.4λi−1/2)\|\Delta_kf_i\|<4(1\wedge1.4\lambda_i^{-1/2})∥Δk​fi​∥<4(1∧1.4λi−1/2​).
  3. Display (15) (p. 24). E{(Xα−pα)(f(U+Xβek)−f(U))}≤2∥f∥(pαβ+pαpβ)E\{(X_\alpha-p_\alpha)(f(U+X_\beta e_k)-f(U))\}\le2\|f\|(p_{\alpha\beta}+p_\alpha p_\beta)E{(Xα​−pα​)(f(U+Xβ​ek​)−f(U))}≤2∥f∥(pαβ​+pα​pβ​) for any random UUU.
  4. Displays (13)–(14), the step i=1i=1i=1 (p. 23). With Wj=∑α∈I(j)XαW_j=\sum_{\alpha\in I(j)}X_\alphaWj​=∑α∈I(j)​Xα​ the block sums of a partition and Z1Z_1Z1​ an independent Poisson(λ1\lambda_1λ1​),
∣Eh(W1,…,Wd)−Eh(Z1,W2,…,Wd)∣≤2∥f1∥∑I(1)pα2+2∥f1∥∑I(1)∑α≠β∈Bα(pαβ+pαpβ)+∥f1∥∑I(1)sα.|Eh(W_1,\dots,W_d)-Eh(Z_1,W_2,\dots,W_d)|\le2\|f_1\|\sum_{I(1)}p_\alpha^2+2\|f_1\|\sum_{I(1)}\sum_{\alpha\ne\beta\in B_\alpha}(p_{\alpha\beta}+p_\alpha p_\beta)+\|f_1\|\sum_{I(1)}s_\alpha.∣Eh(W1​,…,Wd​)−Eh(Z1​,W2​,…,Wd​)∣≤2∥f1​∥I(1)∑​pα2​+2∥f1​∥I(1)∑​α=β∈Bα​∑​(pαβ​+pα​pβ​)+∥f1​∥I(1)∑​sα​.
  1. The finite-dimensional bound (Theorem 2, p. 11). For a partition of III into nonempty blocks I(1),…,I(d)I(1),\dots,I(d)I(1),…,I(d) with Wj=∑I(j)XαW_j=\sum_{I(j)}X_\alphaWj​=∑I(j)​Xα​, Zj=∑I(j)YαZ_j=\sum_{I(j)}Y_\alphaZj​=∑I(j)​Yα​ and λj=EWj\lambda_j=EW_jλj​=EWj​:
∥L((W1,…,Wd))−L((Z1,…,Zd))∥≤2(1∧1.4(min⁡iλi)−1/2)(2b1+2b2+b3).\|\mathcal L((W_1,\dots,W_d))-\mathcal L((Z_1,\dots,Z_d))\|\le2\bigl(1\wedge1.4(\min_i\lambda_i)^{-1/2}\bigr)(2b_1+2b_2+b_3).∥L((W1​,…,Wd​))−L((Z1​,…,Zd​))∥≤2(1∧1.4(imin​λi​)−1/2)(2b1​+2b2​+b3​).

Companion results

  • Theorem 3 (p. 12). For an independent Bernoulli process X′\mathbf X'X′ with the same marginals as X\mathbf XX, ∥L(X)−L(X′)∥≤2(2b1+2b2+b3)+4∑pα2\|\mathcal L(\mathbf X)-\mathcal L(\mathbf X')\|\le2(2b_1+2b_2+b_3)+4\sum p_\alpha^2∥L(X)−L(X′)∥≤2(2b1​+2b2​+b3​)+4∑pα2​.
  • The remark after Theorem 3. ∥L(X′)−L(Y)∥≤2∑pα2\|\mathcal L(\mathbf X')-\mathcal L(\mathbf Y)\|\le2\sum p_\alpha^2∥L(X′)−L(Y)∥≤2∑pα2​, so 4∑pα24\sum p_\alpha^24∑pα2​ in Theorem 3 can be replaced by 2∑pα22\sum p_\alpha^22∑pα2​.

Significance

The result. Theorem 2 says that when b1b_1b1​, b2b_2b2​, b3b_3b3​ are small, the set of indices at which the dependent events occur is close in law to a Poisson process with the same intensity. The bound covers every event about the locations, not just the total count. It is in the strong total variation norm and needs only moment information when b3=0b_3=0b3​=0. Theorem 3 turns this into a decoupling statement: a dependent family of rare events is almost indistinguishable from an independent family with the same marginals. The process bound is the form used when the positions of rare events matter, for example in the distribution of the location of the longest match in sequence comparison and in scan statistics.

Formalizing it. The results are proved in the paper; none is machine-checked. A formal development would give a Lean statement and proof of a Chen–Stein bound with explicit constants, for countable index sets and arbitrary dependence. It would also include the coordinatewise Stein operators on Z+d\mathbb Z_+^dZ+d​ and the bounds on their solutions, which are reusable for other multivariate Poisson approximations. A companion mission formalizes Theorem 1, including Lemma 1 (the bounds on the one-variable Stein solution) on which milestone 2 rests.

Difficulty

The total variation distance between two laws on Z+I\mathbb Z_+^IZ+I​ cannot be bounded coordinate by coordinate, because the XαX_\alphaXα​ are dependent. The one-variable Stein equation controls only the law of a single integer-valued statistic. The paper passes through finite-dimensional block sums and replaces the coordinates one at a time. At each replacement the error has to be charged to the terms of b1,b2,b3b_1,b_2,b_3b1​,b2​,b3​ indexed by the current block, so that the total over all blocks reproduces 2b1+2b2+b32b_1+2b_2+b_32b1​+2b2​+b3​ without a factor ddd. Two further points need care. First, the bound ∥Δkf1∥≤2∥f1∥\|\Delta_kf_1\|\le2\|f_1\|∥Δk​f1​∥≤2∥f1​∥ is the only one available for differences in a coordinate other than the one being replaced; this is why the constants are larger than in Theorem 1. Second, the infinite index set requires a limiting argument from finitely many blocks to the full process.

Formalization scope

  • The index set is a countable type. The page says "arbitrary index set", but pα>0p_\alpha>0pα​>0 and ∑pα<∞\sum p_\alpha<\infty∑pα​<∞ force countability; the assumption also makes X\mathbf XX measurable as a map into NI\mathbb N^INI with the product σ-algebra.
  • The XαX_\alphaXα​ are measurable, N\mathbb NN-valued and bounded by 111. pαp_\alphapα​ is P(Xα=1)P(X_\alpha=1)P(Xα​=1). The standing hypotheses pα>0p_\alpha>0pα​>0, ∑pα=λ\sum p_\alpha=\lambda∑pα​=λ and λ>0\lambda>0λ>0 are explicit.
  • b1,b2,b3b_1,b_2,b_3b1​,b2​,b3​ and sαs_\alphasα​ are [0,∞][0,\infty][0,∞]-valued, and every bound is an inequality in [0,∞][0,\infty][0,∞], so a divergent series is never read as 000. The conditional expectation in sαs_\alphasα​ is taken with respect to a σ-algebra generated by measurable maps, hence a genuine sub-σ-algebra.
  • The norm ∥L(U)−L(V)∥\|\mathcal L(U)-\mathcal L(V)\|∥L(U)−L(V)∥ is twice the published definition MarkovChainCLT.tvDist (supremum over measurable sets of ∣μ(A)−ν(A)∣|\mu(A)-\nu(A)|∣μ(A)−ν(A)∣). Dropping the factor 222 would make the goal twice as strong and false in general.
  • Y\mathbf YY (and X′\mathbf X'X′ in Theorem 3) lives on its own probability space. Mutual independence of the YαY_\alphaYα​ and of the Xα′X'_\alphaXα′​ is a hypothesis. Without it the law of the process is not determined by the marginals and the statements are false.
  • Block sums WjW_jWj​ are N\mathbb NN-valued tsums, equal to the true sums almost surely. A partition is a surjection I→{1,…,d}I\to\{1,\dots,d\}I→{1,…,d} (Fin d), so Lean's coordinate 0 is the paper's coordinate 111.
  • (Sih)(j)(S_ih)(j)(Si​h)(j) is set to 000 when ji=0j_i=0ji​=0, as for SSS in §4. The strict inequalities in milestone 2 use the actual sup norms over Z+d\mathbb Z_+^dZ+d​.

Needed infrastructure: product measures and independence on NI\mathbb N^INI, Poisson laws (Mathlib's poissonMeasure), conditional expectation, and a one-variable Stein bound (Lemma 1). Contributions of proofs of any milestone, of the general step iii of §6, and of the one-dimensional Lemma 1 are welcome.

Selected references

  • R. Arratia, L. Goldstein, L. Gordon, Two moments suffice for Poisson approximations: the Chen-Stein method, Ann. Probab. 17(1) (1989) 9–25. https://doi.org/10.1214/aop/1176991491
  • L. H. Y. Chen, Poisson approximation for dependent trials, Ann. Probab. 3(3) (1975) 534–545. https://doi.org/10.1214/aop/1176996359
  • A. D. Barbour, G. K. Eagleson, Poisson approximation for some statistics based on exchangeable trials, Adv. Appl. Probab. 15(3) (1983) 585–600. https://doi.org/10.2307/1426620
  • A. D. Barbour, L. Holst, S. Janson, Poisson Approximation, Oxford University Press, 1992. https://global.oup.com/academic/product/poisson-approximation-9780198522355
8 thms4 active usersReviewed
🏆Completed
CombinatoricsStatistics·Captain: mikedeng1

Two Moments Suffice for Poisson Approximations: The Chen-Stein Method 1: Total Variation Bound 2[(b₁+b₂)(1−e^{−λ})/λ + b₃′(1∧1.4λ^{−1/2})] for a Sum of Dependent IndicatorsResearch Paper

Motivation

Counts of rare events often look Poisson even when the events are dependent. For example, a local pattern in a random graph can overlap another copy of the pattern, so the corresponding occurrence indicators are not independent. The useful question is quantitative: how far is the distribution of the total count from a Poisson distribution with the same mean? Arratia, Goldstein and Gordon give a bound in terms of three quantities that can be estimated from neighbourhoods of dependence. Their 1989 paper applies the Chen–Stein method to the count and then to the full point process. This mission concerns the count bound, Theorem 1.

The authors explain that the result is essentially contained in Chen’s earlier work, while their one-variable proof prepares the process result in Theorem 2. The bound remains useful as a precise statement of the dependence conditions under which the Poisson approximation works. The process theorem belongs to the companion mission.

Setting

Let III index Bernoulli indicators XαX_\alphaXα​, each taking values in {0,1}\{0,1\}{0,1} on a probability space with law PPP. Write pα=P(Xα=1)>0p_\alpha=P(X_\alpha=1)>0pα​=P(Xα​=1)>0, and let

W=∑α∈IXα,λ=EW=∑α∈Ipα∈(0,∞).W=\sum_{\alpha\in I}X_\alpha,\qquad \lambda=E W=\sum_{\alpha\in I}p_\alpha\in(0,\infty).W=α∈I∑​Xα​,λ=EW=α∈I∑​pα​∈(0,∞).

For each α\alphaα, choose a dependence neighbourhood Bα⊆IB_\alpha\subseteq IBα​⊆I containing α\alphaα. The neighbourhoods need not be symmetric, and the theorem imposes no independence condition: dependence on indicators outside BαB_\alphaBα​ is measured directly. Define pαβ=E(XαXβ)p_{\alpha\beta}=E(X_\alpha X_\beta)pαβ​=E(Xα​Xβ​) and

b1=∑α∑β∈Bαpαpβ,b2=∑α∑β∈Bαβ≠αpαβ.\begin{aligned} b_1&=\sum_\alpha\sum_{\beta\in B_\alpha}p_\alpha p_\beta,\\ b_2&=\sum_\alpha\sum_{\substack{\beta\in B_\alpha\\\beta\ne\alpha}}p_{\alpha\beta}. \end{aligned}b1​b2​​=α∑​β∈Bα​∑​pα​pβ​,=α∑​β∈Bα​β=α​∑​pαβ​.​

The remaining quantities use conditional expectation. Let Vα=∑β∉BαXβV_\alpha=\sum_{\beta\notin B_\alpha}X_\betaVα​=∑β∈/Bα​​Xβ​, and let Fα\mathcal F_\alphaFα​ be the sigma algebra generated by all XβX_\betaXβ​ outside BαB_\alphaBα​. Put

sα′=E∣E[Xα−pα∣Vα]∣,b3′=∑αsα′,sα=E∣E[Xα−pα∣Fα]∣,b3=∑αsα.\begin{aligned} s'_\alpha&=E\left|E[X_\alpha-p_\alpha\mid V_\alpha]\right|, & b'_3&=\sum_\alpha s'_\alpha,\\ s_\alpha&=E\left|E[X_\alpha-p_\alpha\mid\mathcal F_\alpha]\right|, & b_3&=\sum_\alpha s_\alpha. \end{aligned}sα′​sα​​=E∣E[Xα​−pα​∣Vα​]∣,=E∣E[Xα​−pα​∣Fα​]∣,​b3′​b3​​=α∑​sα′​,=α∑​sα​.​

Conditioning on the count VαV_\alphaVα​ retains less information than conditioning on all far indicators, so sα′≤sαs'_\alpha\le s_\alphasα′​≤sα​. Let ZZZ have the Poisson law with mean λ\lambdaλ. The paper defines ∥L(W)−L(Z)∥\|\mathcal L(W)-\mathcal L(Z)\|∥L(W)−L(Z)∥ as twice the usual supremum over events: 2sup⁡A∣P(W∈A)−P(Z∈A)∣2\sup_A|P(W\in A)-P(Z\in A)|2supA​∣P(W∈A)−P(Z∈A)∣.

Formalization targets

Theorem 1: total variation and the zero-count event

The goal is the complete four-clause statement of Theorem 1, including its strict final inequality:

∥L(W)−L(Z)∥≤2[(b1+b2)1−e−λλ+b3′min⁡{1,1.4λ−1/2}]≤2(b1+b2+b3),∣P(W=0)−e−λ∣≤(b1+b2+b3′)1−e−λλ<min⁡{1,λ−1}(b1+b2+b3).\begin{aligned} \|\mathcal L(W)-\mathcal L(Z)\| &\le 2\left[(b_1+b_2)\frac{1-e^{-\lambda}}{\lambda} +b'_3\min\{1,1.4\lambda^{-1/2}\}\right]\\ &\le 2(b_1+b_2+b_3),\\ |P(W=0)-e^{-\lambda}| &\le (b_1+b_2+b'_3)\frac{1-e^{-\lambda}}{\lambda} <\min\{1,\lambda^{-1}\}(b_1+b_2+b_3). \end{aligned}∥L(W)−L(Z)∥∣P(W=0)−e−λ∣​≤2[(b1​+b2​)λ1−e−λ​+b3′​min{1,1.4λ−1/2}]≤2(b1​+b2​+b3​),≤(b1​+b2​+b3′​)λ1−e−λ​<min{1,λ−1}(b1​+b2​+b3​).​

The intermediate targets are the Stein-operator inverse identity, the bound on a telescoping term, display (11) relating the approximation error to b1,b2,b3′b_1,b_2,b'_3b1​,b2​,b3′​, and the two clauses of Lemma 1. The goal keeps the coefficients and both forms of b3b_3b3​ fixed, as the paper states them.

Significance

The result reduces a distributional approximation question to local sums of probabilities and conditional-dependence errors. When b1b_1b1​, b2b_2b2​ and b3b_3b3​ are small, the entire law of the count is close to its Poisson comparison law, and the probability of no occurrence has its own sharper bound. The latter is useful whenever a model asks whether at least one rare event occurs. The result applies without demanding exact independence outside the chosen neighbourhoods; the residual dependence appears in b3b_3b3​.

The paper proves these results. The formalization task is to give the known estimates machine-checked statements and proofs with the constants and probability conventions exposed. Mathlib supplies the Poisson measure and conditional expectation, while the mission adds the paper-specific neighbourhood sums and Stein operators. The formalized theorem can be reused for count approximations in other dependent-indicator models once those models supply bounds on the three error quantities. No machine-checked proof is claimed here: the theorem items are proposed targets with sorry bodies.

Difficulty

A first approximation based only on EWE WEW misses overlapping event pairs. Bounding pair dependence through b2b_2b2​ alone also misses dependence between one indicator and the aggregate of far indicators. The authors' third term measures that remaining effect by conditional expectation. The proof must control the Stein solution uniformly while preserving its precise constants; replacing that control with a qualitative bound would not recover Theorem 1. In the countably infinite setting, the sums and conditional expectations also require careful handling of convergence and measurability. The proof of Lemma 1 is attributed in the paper to Barbour and Eagleson; it is a substantive target here rather than an assumed bound.

Formalization scope

The Lean index type is countable. Under the paper's assumptions pα>0p_\alpha>0pα​>0 and ∑αpα<∞\sum_\alpha p_\alpha<\infty∑α​pα​<∞, only countably many indices can exist, so this representation retains the stated probabilistic setting. Indicators are measurable natural-number-valued functions with values at most one. The parameter λ\lambdaλ is a positive real equipped with the exact HasSum statement for the probabilities. The Poisson law is Mathlib's poissonMeasure at λ\lambdaλ converted to a nonnegative real parameter.

The sums defining b1,b2,b3,b3′b_1,b_2,b_3,b'_3b1​,b2​,b3​,b3′​ use extended nonnegative reals, so divergence becomes +∞+\infty+∞ rather than the real tsum default zero. The random counts WWW and VαV_\alphaVα​ use natural-number tsum; their divergent-input default can occur only on a null set under the finite-mean hypotheses and therefore does not change their laws. Measurability of these counts follows from countability and measurability of each indicator. The far-field sigma algebra is generated by the measurable indicators, and the formalization checks that it lies below the ambient sigma algebra, avoiding a default-zero conditional expectation. The paper's total variation norm is exactly 2 * MarkovChainCLT.tvDist, with the factor two retained.

The source writes a strict final inequality without discussing infinite error sums. The Lean statement conditions that single strict clause on b1+b2+b3<∞b_1+b_2+b_3<\inftyb1​+b2​+b3​<∞; otherwise both sides can be infinite and strict comparison is false. The other three clauses allow infinite bounds, as the source does. Lemma 1 fixes the Stein inverse at zero exactly as the paper chooses and uses explicit pointwise sup-norm bounds over all nonnegative integers. Display (11) allows ∣h(k)∣≤1|h(k)|\le1∣h(k)∣≤1, a strengthening of the displayed ∥h∥=1\|h\|=1∥h∥=1 case. These choices rule out loss of the norm's factor two, disappearing divergent sums, default-zero mapped measures or conditional expectations, and an unannounced independence assumption.

Selected references

  • Richard Arratia, Larry Goldstein and Louis Gordon, Two moments suffice for Poisson approximations: the Chen-Stein method, Annals of Probability 17(1), 9–25, 1989. DOI 10.1214/aop/1176991491.
8 thms4 active usersReviewed
🏆Completed
Dynamic ProgrammingOperations ResearchOptimization·Captain: mikedeng1

Coordinating Inventory Control and Pricing Strategies with Random Demand and Fixed Ordering Cost: The Finite Horizon Case 1: Additive Demand: k-Concave Profit-to-Go, Optimal (s, S, p) PolicyResearch Paper

Motivation

A retailer that sets both its replenishment quantities and its prices over a finite selling season faces a coupled decision: the price shapes the demand that the inventory must serve, and the inventory position shapes which price is worth charging. With a fixed cost per order, the classical inventory answer is the (s, S) policy of Scarf (1960): when the stock falls below a reorder point sss, order up to the level SSS. Thomas (1974) asked what happens when the price is also a decision, and conjectured that an (s, S, p) policy — an (s, S) ordering rule together with a price that depends on the stock level — is optimal "under fairly general conditions". He also gave a counterexample when prices are restricted to a discrete set.

Chen and Simchi-Levi (Operations Research 52(6), 2004) settled the conjecture for the finite horizon. When demand is additive in a random shock, wt=Dt(pt)+βtw_t = D_t(p_t) + \beta_twt​=Dt​(pt​)+βt​, the conjecture holds (§3, Theorem 3.1). When the shock is not additive, it fails, and a weaker policy class is optimal (§4, the second mission of this series). This mission formalizes the additive case.

Timeline:

  • Scarf (1960): (s, S) policies are optimal for the fixed-cost inventory problem, through k-convexity.
  • Thomas (1974): conjectures (s, S, p) optimality with a continuous price range.
  • Federgruen and Heching (1999): joint pricing and inventory without a fixed cost; base-stock list-price policies are optimal.
  • Polatoglu and Sahin (2000): the lost-sales variant; sufficient conditions for (s, S, p) optimality.
  • Chen and Simchi-Levi (2004): (s, S, p) optimality for additive demand with backlogging, and its failure in general.

Setting

There are TTT periods t=1,…,Tt = 1, \dots, Tt=1,…,T. In period ttt the firm starts with inventory xxx, orders up to a level y≥xy \ge xy≥x at cost k δ(y−x)+ct(y−x)k\,\delta(y - x) + c_t(y - x)kδ(y−x)+ct​(y−x), where k≥0k \ge 0k≥0 is a fixed cost, ctc_tct​ a unit cost and δ(u)=1\delta(u) = 1δ(u)=1 if u>0u > 0u>0 and 000 otherwise, and sets a price. Demand is wt=αtDt(pt)+βtw_t = \alpha_t D_t(p_t) + \beta_twt​=αt​Dt​(pt​)+βt​ for a random pair ϵt=(αt,βt)\epsilon_t = (\alpha_t, \beta_t)ϵt​=(αt​,βt​) with Eαt=1\mathbb E\alpha_t = 1Eαt​=1, Eβt=0\mathbb E\beta_t = 0Eβt​=0; in the additive case αt=1\alpha_t = 1αt​=1. Unmet demand is backlogged, and an end-of-period inventory xxx costs ht(x)h_t(x)ht​(x), a convex function.

Since price and expected demand determine each other, the decision is an expected demand d∈[d‾t,dˉt]d \in [\underline d_t, \bar d_t]d∈[d​t​,dˉt​], with price Pt(d)=Dt−1(d)P_t(d) = D_t^{-1}(d)Pt​(d)=Dt−1​(d) and expected revenue Rt(d)=d Pt(d)R_t(d) = d\,P_t(d)Rt​(d)=dPt​(d), assumed concave. With Gt(y,d)=E ht(y−αtd−βt)G_t(y, d) = \mathbb E\,h_t(y - \alpha_t d - \beta_t)Gt​(y,d)=Eht​(y−αt​d−βt​), the profit-to-go functions satisfy vT+1≡0v_{T+1} \equiv 0vT+1​≡0 and, for t=T,…,1t = T, \dots, 1t=T,…,1,

vt(x)=ctx+max⁡y≥x[−k δ(y−x)+gt(y,dt(y))],(2)v_t(x) = c_t x + \max_{y \ge x}\Big[-k\,\delta(y - x) + g_t\big(y, d_t(y)\big)\Big], \qquad (2)vt​(x)=ct​x+y≥xmax​[−kδ(y−x)+gt​(y,dt​(y))],(2) gt(y,d)=Rt(d)−cty+E{−ht(y−αtd−βt)+vt+1(y−αtd−βt)},(3)g_t(y, d) = R_t(d) - c_t y + \mathbb E\big\{-h_t(y - \alpha_t d - \beta_t) + v_{t+1}(y - \alpha_t d - \beta_t)\big\}, \qquad (3)gt​(y,d)=Rt​(d)−ct​y+E{−ht​(y−αt​d−βt​)+vt+1​(y−αt​d−βt​)},(3)

where dt(y)d_t(y)dt​(y) maximizes gt(y,⋅)g_t(y, \cdot)gt​(y,⋅) over [d‾t,dˉt][\underline d_t, \bar d_t][d​t​,dˉt​]. A function fff is k-convex if k+f(z+y)≥f(y)+zb(f(y)−f(y−b))k + f(z + y) \ge f(y) + \frac{z}{b}\big(f(y) - f(y - b)\big)k+f(z+y)≥f(y)+bz​(f(y)−f(y−b)) for all z≥0z \ge 0z≥0, b>0b > 0b>0, yyy (Definition 2.1), and k-concave if −f-f−f is k-convex. Assumptions 3–5 of the paper give GtG_tGt​ the stated one-sided growth limits and polynomial growth O(∣y∣ρ)O(|y|^\rho)O(∣y∣ρ), and give demand a finite ρ\rhoρ-th moment.

Formalization targets

Goal: Theorem 3.1(c)–(d)

For additive demand and every period ttt: gt(y,⋅)g_t(y, \cdot)gt​(y,⋅) attains its maximum at some dt(y)d_t(y)dt​(y), the functions y↦gt(y,dt(y))y \mapsto g_t(y, d_t(y))y↦gt​(y,dt​(y)) and x↦vt(x)x \mapsto v_t(x)x↦vt​(x) are k-concave, and there exist st≤Sts_t \le S_tst​≤St​ such that

y∗(x)={St,x<st,x,x≥st,y^*(x) = \begin{cases} S_t, & x < s_t,\\ x, & x \ge s_t,\end{cases}y∗(x)={St​,x,​x<st​,x≥st​,​

maximizes −k δ(y−x)+gt(y,dt(y))-k\,\delta(y - x) + g_t(y, d_t(y))−kδ(y−x)+gt​(y,dt​(y)) over y≥xy \ge xy≥x, vt(x)=ctx−k δ(y∗(x)−x)+gt(y∗(x),dt(y∗(x)))v_t(x) = c_t x - k\,\delta(y^*(x) - x) + g_t(y^*(x), d_t(y^*(x)))vt​(x)=ct​x−kδ(y∗(x)−x)+gt​(y∗(x),dt​(y∗(x))), and a best expected demand exists at y∗(x)y^*(x)y∗(x); the price is Dt−1D_t^{-1}Dt−1​ of it.

Milestones

  1. Definitions 2.1 and 2.2 of k-convexity are equivalent.
  2. Lemma 1(d): a continuous coercive k-convex function has the (s, S) shape (already proved on the platform).
  3. Lemma 2: a maximizer dt(y)d_t(y)dt​(y) can be chosen with y−dt(y)y - d_t(y)y−dt​(y) nondecreasing.
  4. Theorem 3.1(a): gt(y,d)=O(∣y∣ρ)g_t(y, d) = O(|y|^\rho)gt​(y,d)=O(∣y∣ρ) and vt(x)=O(∣x∣ρ)v_t(x) = O(|x|^\rho)vt​(x)=O(∣x∣ρ).
  5. Theorem 3.1(b): gtg_tgt​ is continuous, gt(y,d)→−∞g_t(y, d) \to -\inftygt​(y,d)→−∞ as ∣y∣→∞|y| \to \infty∣y∣→∞, and dt(y)d_t(y)dt​(y) exists.
  6. Inequality (7): gt(y,dt(y))g_t(y, d_t(y))gt​(y,dt​(y)) is k-concave when the continuation value is.
  7. The display of vtv_tvt​: its (s, S) form and the transfer of k-concavity from gt(y,dt(y))g_t(y, d_t(y))gt​(y,dt​(y)) to vtv_tvt​.

Significance

The theorem identifies the structure of an optimal policy for a basic model of coordinated pricing and replenishment: two numbers per period determine the ordering decision, and the price is a function of the post-order stock. It confirms Thomas's conjecture in the additive case and tells a computation or an approximation scheme which policy class it may restrict to; §5 of the paper extends it to a nonincreasing fixed cost, the infinite horizon and Markovian demand.

The result is proved in the paper; to our knowledge it has no machine-checked proof. A formalization adds checked versions of the k-convexity toolkit (the equivalence of the two standard definitions; the (s, S) structure lemma), a checked monotone-selection argument for a parametric maximization, and a checked finite-horizon induction in which an expectation, a maximization over a continuous action and a fixed cost interact. The proof of part (a) is omitted in the paper ("similar to that of Theorem 1 in Federgruen and Heching (1999)"), so formalizing it supplies a missing argument.

Difficulty

The obvious argument fails at the expectation. A k-concave function composed with a shift y↦y−d−βy \mapsto y - d - \betay↦y−d−β is k-concave for a fixed ddd, but here the demand decision depends on yyy, and k-concavity, unlike concavity, is not preserved under an arbitrary reparametrization: Definition 2.2 is asymmetric in its two points. The inequality needed for the continuation value requires the post-demand levels y−dt(y)−βty - d_t(y) - \beta_ty−dt​(y)−βt​ to be ordered like the inventory levels yyy. That is the content of Lemma 2, and it uses additivity; for multiplicative demand the ordering fails, and so does the theorem (§4).

The second difficulty is analytic: each step of the induction needs a continuous continuation value of polynomial growth, so that the expectation in (3) is finite and continuous, the maximum over ddd is attained, and gtg_tgt​ is coercive.

Formalization scope

The model is a structure of real data indexed by t∈Nt \in \mathbb Nt∈N, with periods 1,…,T1, \dots, T1,…,T; the law of (αt,βt)(\alpha_t, \beta_t)(αt​,βt​) is a probability measure on R2\mathbb R^2R2 and expectations are Bochner integrals. The formalization works in expected-demand space d∈[d‾t,dˉt]d \in [\underline d_t, \bar d_t]d∈[d​t​,dˉt​], equivalent to price space because DtD_tDt​ is a continuous strictly decreasing bijection. Additivity is "αt=1\alpha_t = 1αt​=1 almost surely". The profit-to-go is defined by recursion on the number of periods to go, with vT+1=0v_{T+1} = 0vT+1​=0. Maxima are written as real suprema; every target that uses them asserts that they are attained, so the convention that an unbounded supremum is 000 never enters. k-convexity is the platform's BertsekasKConvex (Definition 2.1); k-concavity of fff is k-convexity of −f-f−f.

Conventions and restrictions relative to the printed page:

  • O(∣y∣ρ)O(|y|^\rho)O(∣y∣ρ) is an explicit bound C(1+∣y∣ρ)C(1 + |y|^\rho)C(1+∣y∣ρ) with ρ∈N\rho \in \mathbb Nρ∈N, uniform in ddd.
  • Assumption 5, printed E{Dt(p,ϵt)}ρ<∞E\{D_t(p, \epsilon_t)\}^\rho < \inftyE{Dt​(p,ϵt​)}ρ<∞, is read as E∣αtd+βt∣ρ<∞\mathbb E|\alpha_t d + \beta_t|^\rho < \inftyE∣αt​d+βt​∣ρ<∞.
  • Integrability of ht(y−αtd−βt)h_t(y - \alpha_t d - \beta_t)ht​(y−αt​d−βt​) is part of Assumption 4, and Theorem 3.1(a) also asserts that the expectation of vt+1v_{t+1}vt+1​ in (3) is finite.
  • Added hypotheses: cT+1=0c_{T+1} = 0cT+1​=0 (the paper uses cT+1c_{T+1}cT+1​ without defining it), ct≥0c_t \ge 0ct​≥0 (used in the proof of part (b); without it gtg_tgt​ need not tend to −∞-\infty−∞), and k≥0k \ge 0k≥0.
  • Independence across periods is not stated; the dynamic program uses only each period's marginal law.
  • The paper writes st<Sts_t < S_tst​<St​ in the proof; for k=0k = 0k=0 equality is possible, so the targets use st≤Sts_t \le S_tst​≤St​.

A trivializing formalization is ruled out: the targets assert attainment of every maximum they mention and the identity between vt(x)v_t(x)vt​(x) and the maximized objective, so a supremum that defaults to 000, or an integral of a non-integrable function, cannot make them hold vacuously; and the assumptions are satisfiable (a one-period instance with h(x)=∣x∣h(x) = |x|h(x)=∣x∣ meets all of them).

A complete development needs: continuity of parametric integrals under polynomial domination; attainment of maxima over compact intervals; Topkis-style monotone selection for functions with increasing differences; the (s, S) structure lemma for k-convex functions (proved on the platform); and the closure of k-concavity under the operations above. The k-convexity results and the monotone-selection lemma are reusable for the general-demand mission of this series and for other fixed-cost inventory models. Contributions to any milestone are welcome, including proofs that rely on platform results by Topkis (Supermodularity.Monotonicity.argmax_increasing_of_increasing_differences).

Selected references

  • X. Chen, D. Simchi-Levi, Coordinating Inventory Control and Pricing Strategies with Random Demand and Fixed Ordering Cost: The Finite Horizon Case, Operations Research 52(6), 887–896, 2004. https://doi.org/10.1287/opre.1040.0127
  • H. Scarf, The Optimality of (S, s) Policies in the Dynamic Inventory Problem, in Mathematical Methods in the Social Sciences, Stanford University Press, 1960.
  • L. J. Thomas, Price and Production Decisions with Random Demand, Operations Research 22, 513–518, 1974.
  • E. L. Porteus, On the Optimality of Generalized (s, S) Policies, Management Science 17, 411–426, 1971.
  • A. Federgruen, A. Heching, Combined Pricing and Inventory Control Under Uncertainty, Operations Research 47(3), 454–475, 1999. https://doi.org/10.1287/opre.47.3.454
  • H. Polatoglu, I. Sahin, Optimal Procurement Policies under Price-Dependent Demand, International Journal of Production Economics 65, 141–171, 2000.
  • D. M. Topkis, Supermodularity and Complementarity, Princeton University Press, 1998.
  • D. P. Bertsekas, Dynamic Programming and Optimal Control, Vol. I, Athena Scientific, 1995.
11 thms4 active usersReviewed
🏆Completed
Convex OptimizationOperations ResearchStatistics·Captain: mikedeng1

Data-Driven Robust Optimization VI: The Moment Set U^CS Has Support Function μ̂ᵀv + Γ₁‖v‖ + √(1/ε − 1)‖Cv‖, an Upper Bound on the Worst-Case Value at RiskResearch Paper

Motivation

In robust optimization, a constraint is checked against every parameter value in an uncertainty set. This turns uncertainty into a deterministic optimization problem, but the set must be chosen carefully: a large set can make decisions unnecessarily conservative, while a small one can miss likely outcomes. Bertsimas, Gupta and Kallus use data to calibrate sets through statistical confidence regions. Their question is whether feasibility for every parameter in the set protects a decision against a fresh uncertain outcome with a specified probability. This mission treats the part of their construction based on estimated first and second moments. Bertsimas, Gupta and Kallus, §8.1.

The moment region originates in a concentration result attributed in the paper to Shawe-Taylor and Cristianini. That result bounds the distance between sample and population means and covariances when the uncertain vector is supported in a Euclidean ball. Bertsimas, Gupta and Kallus also discuss replacing those analytic thresholds with bootstrap thresholds; they describe the resulting coverage as approximate. The present target concerns the mathematical relation between a fixed moment region and its uncertainty set, leaving the statistical calibration of the thresholds to its cited source. Shawe-Taylor and Cristianini, 2003; Bertsimas, Gupta and Kallus, pp. 24–25.

Setting

Let the uncertain vector u~\tilde uu~ take values in Rd\mathbb R^dRd. A probability law PPP belongs to the moment confidence region PCS\mathcal P^{CS}PCS when it is supported in the Euclidean ball of radius RRR, its mean mPm_PmP​ is within Γ1\Gamma_1Γ1​ of an estimate μ^\hat\muμ^​, and its covariance SPS_PSP​ is within Γ2\Gamma_2Γ2​ of an estimate Σ^\hat\SigmaΣ^ in Frobenius norm. The covariance is SP=EP[u~u~⊤]−mPmP⊤S_P=\mathbb E_P[\tilde u\tilde u^\top]-m_Pm_P^\topSP​=EP​[u~u~⊤]−mP​mP⊤​. The thresholds Γ1,Γ2\Gamma_1,\Gamma_2Γ1​,Γ2​ are nonnegative and can be chosen by either calibration procedure discussed in the paper. Bertsimas, Gupta and Kallus, Theorem 9 and (33).

For a vector vvv, Value at Risk VaR⁡εP(v)\operatorname{VaR}^{P}_{\varepsilon}(v)VaRεP​(v) is the smallest threshold ttt for which P(u~⊤v≤t)≥1−εP(\tilde u^\top v\le t)\ge1-\varepsilonP(u~⊤v≤t)≥1−ε. The support function of a set U\mathcal UU is δ∗(v∣U)=sup⁡u∈Uu⊤v\delta^*(v\mid\mathcal U)=\sup_{u\in\mathcal U}u^\top vδ∗(v∣U)=supu∈U​u⊤v. The authors construct the set

UεCS={μ^+y+C⊤w:∥y∥2≤Γ1, ∥w∥2≤1/ε−1},C⊤C=Σ^+Γ2I.\mathcal U^{CS}_{\varepsilon} =\{\hat\mu+y+C^\top w:\|y\|_2\le\Gamma_1,\ \|w\|_2\le\sqrt{1/\varepsilon-1}\}, \qquad C^\top C=\hat\Sigma+\Gamma_2 I.UεCS​={μ^​+y+C⊤w:∥y∥2​≤Γ1​, ∥w∥2​≤1/ε−1​},C⊤C=Σ^+Γ2​I.

Here 0<ε<10<\varepsilon<10<ε<1 is the allowed violation probability, III is the identity matrix, and CCC is a matrix factor of the adjusted covariance. The estimates and CCC stay fixed while ε\varepsilonε varies. Bertsimas, Gupta and Kallus, (34)–(35).

Formalization targets

The first milestone bounds the quantile for every law in the moment region:

VaR⁡εP(v)≤μ^⊤v+Γ1∥v∥2+1−εεv⊤(Σ^+Γ2I)v(P∈PCS).\operatorname{VaR}^{P}_{\varepsilon}(v)\le \hat\mu^\top v+\Gamma_1\|v\|_2+ \sqrt{\frac{1-\varepsilon}{\varepsilon}} \sqrt{v^\top(\hat\Sigma+\Gamma_2I)v} \quad(P\in\mathcal P^{CS}).VaRεP​(v)≤μ^​⊤v+Γ1​∥v∥2​+ε1−ε​​v⊤(Σ^+Γ2​I)v​(P∈PCS).

The second milestone evaluates the two linear maxima over the Euclidean balls in UεCS\mathcal U^{CS}_{\varepsilon}UεCS​ and identifies ∥Cv∥2\|Cv\|_2∥Cv∥2​ with the quadratic form above. The goal, the deterministic part of Theorem 10, says the displayed bound equals δ∗(v∣UεCS)\delta^*(v\mid\mathcal U^{CS}_{\varepsilon})δ∗(v∣UεCS​) for every vvv, while the set is nonempty, convex and compact. This gives the support-function criterion of Theorem 1 for each law in the region. A companion formalizes Theorem 13(a): the resulting support constraint is separately convex in (v,t)(v,t)(v,t) and in ε\varepsilonε for 0<ε<3/40<\varepsilon<3/40<ε<3/4. Bertsimas, Gupta and Kallus, Theorems 10 and 13(a).

Significance

The support formula turns a distributional statement about an entire region of probability laws into a deterministic bound on a linear projection. It gives the robust model an explicit quantity to compare with a constraint threshold. The companion convexity result describes which risk levels allow separate convex optimization in the decision variables and the violation probability. Together they explain why the particular set in (35) is useful beyond the fact that it contains plausible uncertain vectors. Bertsimas, Gupta and Kallus, §§8.1 and 9.

The paper proves the mathematical construction and cites statistical results for the region's coverage. In Lean, the published Value-at-Risk and support-function definitions are already available, while this paper's moment region and set (35) need their own definitions. Formalizing the result therefore establishes a reusable interface between moment bounds, quantiles and set support. The theorem statements in this proposal are open proof targets; compiling them checks their types, not their proofs.

Difficulty

A bound on the mean and covariance does not directly bound a high quantile of every projection. The bound must work simultaneously for every probability law in the moment region, including discrete laws and singular covariances. The factor (1−ε)/ε\sqrt{(1-\varepsilon)/\varepsilon}(1−ε)/ε​ is essential: replacing it by a standard deviation or a Gaussian quantile would change the claim. On the set side, the ordinary norm of a Lean function vector is a sup norm, whereas both balls in (35) are Euclidean. Getting the norm wrong changes the support function. Bertsimas, Gupta and Kallus, (34)–(35).

Formalization scope

Vectors are functions on Fin d, whose coordinates start at zero. enorm and frob explicitly compute the Euclidean and Frobenius norms. The ball-support condition in PCS\mathcal P^{CS}PCS secures finite first and second moments; no density is assumed. The goal fixes R≥0R\ge0R≥0, Γ1,Γ2≥0\Gamma_1,\Gamma_2\ge0Γ1​,Γ2​≥0, 0<ε<10<\varepsilon<10<ε<1, and C⊤C=Σ^+Γ2IC^\top C=\hat\Sigma+\Gamma_2IC⊤C=Σ^+Γ2​I. This factor relation makes the quadratic form nonnegative; a separate positive-semidefinite assumption on Σ^\hat\SigmaΣ^ is unnecessary. The matrix need not be triangular because (35) and its support depend on C⊤CC^\top CC⊤C. The uncertainty set's nonemptiness and compactness appear in the goal so the real support-function supremum cannot take Lean's default value for an empty or unbounded set.

Theorem 10's sampling claim is represented by its deterministic criterion for each P∈PCSP\in\mathcal P^{CS}P∈PCS. The confidence-region coverage of Theorem 9 is cited, and the bootstrap coverage discussed on p. 25 is approximate; neither is asserted as an exact probability theorem here. Equation (34) prints an equality for a supremum over PCS\mathcal P^{CS}PCS, yet that region restricts support to a ball, and the page does not establish attainment under that restriction. This mission uses the upper-bound direction needed for Theorem 10 and does not assert the equality or Remark 16's exact equivalence. The scope includes a sharp quantile bound, the exact support formula and the 3/43/43/4 convexity range; a definition that merely makes the claim true by construction would miss these targets. Bertsimas, Gupta and Kallus, pp. 24–25 and 29.

Selected references

  • D. Bertsimas, V. Gupta and N. Kallus, Data-Driven Robust Optimization, arXiv:1401.0212v2, 2014; revised in Mathematical Programming 167 (2018), 235–292. arXiv preprint.
  • J. Shawe-Taylor and N. Cristianini, Estimating the Moments of a Random Vector with Applications, 2003. University of Southampton ePrint.
  • G. C. Calafiore and L. El Ghaoui, On Distributionally Robust Chance-Constrained Linear Programs, Journal of Optimization Theory and Applications 130 (2006), 1–22. DOI.
6 thms4 active usersReviewed
🏆Completed
Convex OptimizationOperations Research·Captain: mikedeng1

Data-Driven Robust Optimization I: An Uncertainty Set Whose Support Function Dominates Value at Risk Implies a Probabilistic Guarantee for Every Concave ConstraintResearch Paper

Motivation

Robust optimization replaces an uncertain constraint f(u~,x)≤0f(\tilde{\mathbf u},\mathbf x)\le 0f(u~,x)≤0, whose parameter u~∈Rd\tilde{\mathbf u}\in\mathbb R^du~∈Rd is random, by the requirement that the constraint hold for every u\mathbf uu in a chosen uncertainty set U\mathcal UU:

f(u,x)≤0∀ u∈U.f(\mathbf u,\mathbf x)\le 0\qquad\forall\,\mathbf u\in\mathcal U .f(u,x)≤0∀u∈U.

The resulting problem is deterministic and, for many shapes of U\mathcal UU, tractable (Ben-Tal, El Ghaoui and Nemirovski, Robust Optimization, 2009). The modelling question it leaves open is how to choose U\mathcal UU. A set that is too large makes every solution conservative; a set that is too small gives no protection. The practitioner's real requirement is usually probabilistic: a robust feasible x\mathbf xx should violate the uncertain constraint with probability at most ϵ\epsilonϵ.

Bertsimas, Gupta and Kallus (Data-Driven Robust Optimization, arXiv:1401.0212v2, 2014; Math. Program. 167, 2018) build uncertainty sets directly from data so that this requirement holds with high confidence. Their whole construction rests on one characterization, Theorem 1 of the paper, of when a set carries such a guarantee. This mission formalizes that characterization and the two results (Theorems 2 and 3) that turn it into a data-driven recipe.

Earlier work used the "if" direction of the characterization for bi-affine constraints when designing sets for specific distributional assumptions (Ben-Tal et al., 2009; Chen, Sim and Sun, Oper. Res. 55, 2007). The extension to every constraint concave in u\mathbf uu is due to Bertsimas, Gupta and Kallus.

Setting

Let P\mathbb PP be a probability measure on Rd\mathbb R^dRd, the law of u~\tilde{\mathbf u}u~, and fix a level 0<ϵ<10<\epsilon<10<ϵ<1. Throughout, f(u,x)f(\mathbf u,\mathbf x)f(u,x) is concave in u\mathbf uu for every value of the decision variable x∈Rk\mathbf x\in\mathbb R^kx∈Rk.

The support function of a set U⊆Rd\mathcal U\subseteq\mathbb R^dU⊆Rd is

δ∗(v∣U)=sup⁡u∈UvTu,v∈Rd.\delta^*(\mathbf v\mid\mathcal U)=\sup_{\mathbf u\in\mathcal U}\mathbf v^T\mathbf u,\qquad\mathbf v\in\mathbb R^d .δ∗(v∣U)=u∈Usup​vTu,v∈Rd.

The Value at Risk of the linear loss u~Tv\tilde{\mathbf u}^T\mathbf vu~Tv at level ϵ\epsilonϵ is

VaRϵP(v)=inf⁡{t: P(u~Tv≤t)≥1−ϵ}.\mathrm{VaR}^{\mathbb P}_\epsilon(\mathbf v)=\inf\{t:\ \mathbb P(\tilde{\mathbf u}^T\mathbf v\le t)\ge 1-\epsilon\}.VaRϵP​(v)=inf{t: P(u~Tv≤t)≥1−ϵ}.

A set U\mathcal UU implies a probabilistic guarantee at level ϵ\epsilonϵ for P\mathbb PP (property (P2) of the paper) if for every kkk, every f(u,x)f(\mathbf u,\mathbf x)f(u,x) concave in u\mathbf uu for each x∈Rk\mathbf x\in\mathbb R^kx∈Rk, and every x∗∈Rk\mathbf x^*\in\mathbb R^kx∗∈Rk,

f(u,x∗)≤0  ∀u∈U⟹P(f(u~,x∗)≤0)≥1−ϵ.(2)f(\mathbf u,\mathbf x^*)\le0\ \ \forall\mathbf u\in\mathcal U\quad\Longrightarrow\quad\mathbb P\big(f(\tilde{\mathbf u},\mathbf x^*)\le0\big)\ge1-\epsilon. \tag{2}f(u,x∗)≤0  ∀u∈U⟹P(f(u~,x∗)≤0)≥1−ϵ.(2)

A function is bi-affine if it has the form f(u,x)=uTFx+fuTu+fxTx+f0f(\mathbf u,\mathbf x)=\mathbf u^TF\mathbf x+\mathbf f_u^T\mathbf u+\mathbf f_x^T\mathbf x+f_0f(u,x)=uTFx+fuT​u+fxT​x+f0​.

In the data-driven setting, P∗\mathbb P^*P∗ is unknown and a sample S=(u^1,…,u^N)\mathcal S=(\hat{\mathbf u}^1,\dots,\hat{\mathbf u}^N)S=(u^1,…,u^N) is drawn i.i.d. from it. The paper's schema fixes 0<α<10<\alpha<10<α<1, takes the confidence region P(S)\mathcal P(\mathcal S)P(S) of a hypothesis test at level α\alphaα, and builds a closed convex set U(S)\mathcal U(\mathcal S)U(S) whose support function bounds VaRϵP(v)\mathrm{VaR}^{\mathbb P}_\epsilon(\mathbf v)VaRϵP​(v) for every P∈P(S)\mathbb P\in\mathcal P(\mathcal S)P∈P(S) and every v\mathbf vv.

Formalization targets

Goal: Theorem 1

(a) If U\mathcal UU is nonempty, convex and compact and

δ∗(v∣U)≥VaRϵP(v)∀ v∈Rd,\delta^*(\mathbf v\mid\mathcal U)\ge\mathrm{VaR}^{\mathbb P}_\epsilon(\mathbf v)\qquad\forall\,\mathbf v\in\mathbb R^d,δ∗(v∣U)≥VaRϵP​(v)∀v∈Rd,

then U\mathcal UU implies a probabilistic guarantee at level ϵ\epsilonϵ for P\mathbb PP.

(b) If U\mathcal UU is nonempty and δ∗(v∣U)<VaRϵP(v)\delta^*(\mathbf v\mid\mathcal U)<\mathrm{VaR}^{\mathbb P}_\epsilon(\mathbf v)δ∗(v∣U)<VaRϵP​(v) for some v\mathbf vv at which δ∗(v∣U)\delta^*(\mathbf v\mid\mathcal U)δ∗(v∣U) is finite, then some bi-affine fff violates (2).

Milestones toward the goal

The proof in the electronic companion (EC.1.1) is a short chain, and its steps are the milestones: the attainment property of VaR, P(u~Tv>VaRϵP(v))≤ϵ\mathbb P(\tilde{\mathbf u}^T\mathbf v>\mathrm{VaR}^{\mathbb P}_\epsilon(\mathbf v))\le\epsilonP(u~Tv>VaRϵP​(v))≤ϵ (already proved on the platform); a strict separating hyperplane between U\mathcal UU and the superlevel set {f(⋅,x∗)≥t}\{f(\cdot,\mathbf x^*)\ge t\}{f(⋅,x∗)≥t}; the bound P(f(u~,x∗)≥t)≤ϵ\mathbb P(f(\tilde{\mathbf u},\mathbf x^*)\ge t)\le\epsilonP(f(u~,x∗)≥t)≤ϵ for t>0t>0t>0; its limit P(f(u~,x∗)>0)≤ϵ\mathbb P(f(\tilde{\mathbf u},\mathbf x^*)>0)\le\epsilonP(f(u~,x∗)>0)≤ϵ; and, for part (b), the witness f(u,x)=vTu−xf(\mathbf u,x)=\mathbf v^T\mathbf u-xf(u,x)=vTu−x at x∗=δ∗(v∣U)x^*=\delta^*(\mathbf v\mid\mathcal U)x∗=δ∗(v∣U).

Companions: Theorems 2 and 3

Theorem 2: with probability at least 1−α1-\alpha1−α over the sample, U(S)\mathcal U(\mathcal S)U(S) implies a probabilistic guarantee at level ϵ\epsilonϵ for P∗\mathbb P^*P∗. Theorem 3: if the region does not depend on ϵ\epsilonϵ, then with probability at least 1−α1-\alpha1−α the whole family {U(S,ϵ):0<ϵ<1}\{\mathcal U(\mathcal S,\epsilon):0<\epsilon<1\}{U(S,ϵ):0<ϵ<1} implies the guarantee simultaneously (a), and every x\mathbf xx satisfying the level-optimized constraints (9) satisfies the joint chance constraint

P∗(max⁡j=1,…,mfj(u~,x)≤0)≥1−ϵˉ(7)\mathbb P^*\Big(\max_{j=1,\dots,m}f_j(\tilde{\mathbf u},\mathbf x)\le0\Big)\ge1-\bar\epsilon \tag{7}P∗(j=1,…,mmax​fj​(u~,x)≤0)≥1−ϵˉ(7)

(b).

Significance

Theorem 1(a) is what certifies every uncertainty set in the paper: the χ2\chi^2χ2 and GGG sets for discrete distributions, the Kolmogorov–Smirnov and forward–backward sets for independent marginals, the marginal-sample box and the moment sets. Each construction reduces to proving one inequality between a support function and a Value at Risk, which is a statement about a single linear functional, and Theorem 1(a) lifts it to every concave constraint at once. Part (b) shows the condition cannot be dropped even for bi-affine constraints. Theorems 2 and 3 separate the statistical input (coverage of a confidence region) from the convex-analytic input (the support-function bound), and Theorem 3(b) is what allows the levels ϵj\epsilon_jϵj​ of a system of constraints to be optimized after seeing the data.

The results are proved in the paper. None of them has a machine-checked proof; the only formalized ingredient is the attainment property of VaR, which is on the platform as a proved theorem. The formalization provides a checked bridge from support-function bounds to probabilistic guarantees that the other missions of this series (II–VI) use to state their own results in the criterion form VaR≤δ∗\mathrm{VaR}\le\delta^*VaR≤δ∗.

Difficulty

The informal argument is short; the difficulty is in the infinite-dimensional bookkeeping that the page leaves implicit. The superlevel set {f(⋅,x∗)≥t}\{f(\cdot,\mathbf x^*)\ge t\}{f(⋅,x∗)≥t} need not be bounded, so the strict separation must use compactness of U\mathcal UU alone. Closedness of this set and measurability of {f(u~,x∗)≤0}\{f(\tilde{\mathbf u},\mathbf x^*)\le0\}{f(u~,x∗)≤0} depend on concave functions on Rd\mathbb R^dRd being continuous. The passage t↓0t\downarrow0t↓0 is continuity of a measure along an increasing union. The attainment of the infimum in the definition of VaR requires right-continuity of distribution functions. A naive attempt to separate U\mathcal UU from the zero superlevel set {f≥0}\{f\ge0\}{f≥0} directly fails, because the two sets may touch.

Formalization scope

Rd\mathbb R^dRd is Fin d → ℝ; u~\tilde{\mathbf u}u~ is the identity map; vTu\mathbf v^T\mathbf uvTu is the dot product u ⬝ᵥ v. The Value at Risk is the published MultistageStochastic.valueAtRisk at level 1−ϵ1-\epsilon1−ϵ, and the support function is the published RobustMDP.Shared.supportFunction. Both are real-valued sInf/sSup, which return 000 on empty or unbounded sets, so every statement assumes 0<ϵ<10<\epsilon<10<ϵ<1, a probability measure, and an uncertainty set that is nonempty and compact (Theorem 1(a), Theorems 2–3) or nonempty with {vTu:u∈U}\{\mathbf v^T\mathbf u:\mathbf u\in\mathcal U\}{vTu:u∈U} bounded above in the direction considered (Theorem 1(b)). In Theorem 1(b) and its witness this is the paper's own requirement that δ∗(v∣U)\delta^*(\mathbf v\mid\mathcal U)δ∗(v∣U) be finite, which its strict inequality with a real VaR forces and its proof uses by treating δ∗(v∣U)\delta^*(\mathbf v\mid\mathcal U)δ∗(v∣U) as a real number. Part (b) is printed with U∗\mathcal U^*U∗, a slip for U\mathcal UU; the proof's separation step prints both inequalities in the same direction, a slip corrected in the strict separation milestone.

Property (P2) quantifies over the dimension kkk of the decision variable, every function concave in u\mathbf uu for each x\mathbf xx, and every x∗\mathbf x^*x∗. Restricting it to affine fff or to k=0k=0k=0 would trivialize part (a) and falsify part (b), and is ruled out by the definition.

In Theorems 2 and 3 the sample is Fin N → (Fin d → ℝ) with the product law Measure.pi; the confidence region and the uncertainty set are arbitrary maps of the sample, and the coverage PS∗(P∗∈P(S))≥1−α\mathbb P^*_{\mathcal S}(\mathbb P^*\in\mathcal P(\mathcal S))\ge1-\alphaPS∗​(P∗∈P(S))≥1−α is a hypothesis, never the conclusion's event. Step 2 of the schema is stated with g=δ∗(⋅∣U(S))g=\delta^*(\cdot\mid\mathcal U(\mathcal S))g=δ∗(⋅∣U(S)) directly. The sets are assumed nonempty and compact (Step 3 says closed and convex; a finite support function that bounds VaR forces nonemptiness and boundedness). Theorem 3(b) reads the levels ϵj\epsilon_jϵj​ in (0,1)(0,1)(0,1), where the family is defined. Probabilities of events in the sample space are outer measures, so no measurability hypotheses are needed.

A complete development needs strict separation of a compact convex set from a closed convex set (Mathlib's geometric_hahn_banach_compact_closed), continuity of concave functions on finite-dimensional spaces, and the attainment property of VaR. Contributions are welcome on all milestones; the separation and limit steps are reusable for any chance-constraint argument based on support functions.

Selected references

  • D. Bertsimas, V. Gupta, N. Kallus, Data-Driven Robust Optimization, arXiv:1401.0212v2, 2014; Mathematical Programming 167:235–292, 2018. https://arxiv.org/abs/1401.0212
  • A. Ben-Tal, L. El Ghaoui, A. Nemirovski, Robust Optimization, Princeton University Press, 2009. https://doi.org/10.1515/9781400831050
  • X. Chen, M. Sim, P. Sun, A Robust Optimization Perspective on Stochastic Programming, Operations Research 55(6):1058–1071, 2007. https://doi.org/10.1287/opre.1070.0441
12 thms4 active usersReviewed
🏆Completed
Markov ChainOperations ResearchStochastic Systems·Captain: mikedeng1

Reversibility and Stochastic Networks VII: Clustering Processes — Poisson Product-Form Equilibrium of the Open Clustering ProcessTextbook

Motivation

Many systems consist of units that form themselves into clusters: individuals at a gathering forming conversational groups, monomers forming polymers, particles coagulating and fragmenting. Chapter 8 of F. P. Kelly, Reversibility and Stochastic Networks (Wiley, 1979) treats such clustering processes as Markov processes whose state counts the clusters of each type, and shows that when the process is reversible its equilibrium distribution has an explicit product form. The chapter opens with a model of social grouping (§8.1), develops a general basic model (§8.2) and later applies it to polymerization (§8.4).

The basic model sits in the same family as the migration processes and queueing networks of the earlier chapters: the method is to guess reversibility, solve the detailed balance equations, and normalize. Its specific feature is that the transitions are unions and break-ups of clusters, with rates quadratic in the cluster counts, rather than movements of single individuals.

Setting

There is a countable collection of cluster types rrr. A state is a vector m=(mr)m = (m_r)m=(mr​) of non-negative integers, mrm_rmr​ being the number of rrr-clusters present, with only finitely many mrm_rmr​ non-zero. For cluster types r,s,ur, s, ur,s,u let ere_rer​ be the rrr-th unit vector and

Rursm=m−er−es+eu,Rrsum=m+er+es−eu,R^{rs}_u m = m - e_r - e_s + e_u, \qquad R^u_{rs} m = m + e_r + e_s - e_u,Rurs​m=m−er​−es​+eu​,Rrsu​m=m+er​+es​−eu​,

the union of an rrr-cluster with an sss-cluster into a uuu-cluster, and the break-up of a uuu-cluster into an rrr-cluster and an sss-cluster. Given non-negative parameters λrsu=λsru\lambda_{rsu} = \lambda_{sru}λrsu​=λsru​ and μrsu=μsru\mu_{rsu} = \mu_{sru}μrsu​=μsru​, the clustering process has transition rates

q(m,Rursm)=λrsumrms (r≠s),q(m,Rurrm)=λrrumr(mr−1),q(m,Rrsum)=μrsumu.(8.3)q(m, R^{rs}_u m) = \lambda_{rsu} m_r m_s \ (r \ne s), \qquad q(m, R^{rr}_u m) = \lambda_{rru} m_r(m_r - 1), \qquad q(m, R^u_{rs} m) = \mu_{rsu} m_u. \qquad (8.3)q(m,Rurs​m)=λrsu​mr​ms​ (r=s),q(m,Rurr​m)=λrru​mr​(mr​−1),q(m,Rrsu​m)=μrsu​mu​.(8.3)

A closed clustering process lives on a finite irreducible state space S\mathcal SS. The open clustering process additionally lets one-clusters enter at rate ν\nuν and leave at rate μm1\mu m_1μm1​,

q(m,m+e1)=ν,q(m,m−e1)=μm1,(8.7)q(m, m + e_1) = \nu, \qquad q(m, m - e_1) = \mu m_1, \qquad (8.7)q(m,m+e1​)=ν,q(m,m−e1​)=μm1​,(8.7)

and its state space is the countable set of all mmm with ∑rmr\sum_r m_r∑r​mr​ finite, every state being reachable from every other.

An equilibrium distribution is a collection of positive numbers π(m)\pi(m)π(m) summing to one that satisfies the equilibrium equations π(m)∑m′q(m,m′)=∑m′π(m′)q(m′,m)\pi(m)\sum_{m'} q(m, m') = \sum_{m'} \pi(m') q(m', m)π(m)∑m′​q(m,m′)=∑m′​π(m′)q(m′,m); the process is reversible in equilibrium exactly when π\piπ satisfies the detailed balance conditions π(m)q(m,m′)=π(m′)q(m′,m)\pi(m) q(m, m') = \pi(m') q(m', m)π(m)q(m,m′)=π(m′)q(m′,m).

Formalization targets

Goal: Theorem 8.2 (p. 164)

If there are positive numbers crc_rcr​ with

ν=c1μ,λrsucrcs=cuμrsu,(8.8)∑rcr<∞,(8.9)\nu = c_1 \mu, \qquad \lambda_{rsu} c_r c_s = c_u \mu_{rsu}, \qquad (8.8) \qquad \sum_r c_r < \infty, \qquad (8.9)ν=c1​μ,λrsu​cr​cs​=cu​μrsu​,(8.8)r∑​cr​<∞,(8.9)

then the open clustering process has equilibrium distribution

π(m)=∏re−crcrmrmr!,(8.10)\pi(m) = \prod_{r} e^{-c_r} \frac{c_r^{m_r}}{m_r!}, \qquad (8.10)π(m)=r∏​e−cr​mr​!crmr​​​,(8.10)

it is reversible, and the counts m1,m2,…m_1, m_2, \dotsm1​,m2​,… are independent, mrm_rmr​ being Poisson with mean crc_rcr​.

Milestones

  • Eq. (8.6): under (8.4), crcsλrsu=cuμrsuc_r c_s \lambda_{rsu} = c_u \mu_{rsu}cr​cs​λrsu​=cu​μrsu​, the weights ∏rcrmr/mr!\prod_r c_r^{m_r}/m_r!∏r​crmr​​/mr​! satisfy the detailed balance conditions for the rates (8.3) on the whole state space.
  • Theorem 8.1 (p. 163): under (8.4) the closed clustering process is reversible with equilibrium distribution π(m)=B∏rcrmr/mr!\pi(m) = B\prod_r c_r^{m_r}/m_r!π(m)=B∏r​crmr​​/mr​! on S\mathcal SS.
  • Eq. (8.2) (p. 161): the social grouping model with MMM individuals has equilibrium π(m)=B∏i1mi!(βα i!)mi\pi(m) = B \prod_{i} \frac{1}{m_i!}\bigl(\frac{\beta}{\alpha\, i!}\bigr)^{m_i}π(m)=B∏i​mi​!1​(αi!β​)mi​ on {m:∑iimi=M}\{m : \sum_i i m_i = M\}{m:∑i​imi​=M}.
  • Eq. (8.9) (p. 164): the product-form weights are summable over the finitely supported states if and only if ∑rcr<∞\sum_r c_r < \infty∑r​cr​<∞, and then (8.10) sums to one.

Significance

Theorem 8.2 gives the equilibrium of a whole class of coagulation–fragmentation dynamics in closed form: the cluster counts are independent Poisson variables, and the equation system (8.8) is the only thing to solve. Quantities such as the expected number of clusters of each type, or the proportion of units in clusters of a given size, are then read off directly. Theorem 8.1 gives the corresponding result for a closed system, where the normalizing constant couples the types; opening the system removes that coupling. The results are the base of the polymerization models of §8.4 and of the later literature on reversible coagulation–fragmentation processes.

The results are proved in the book, with short proofs that state the detailed balance computation is "readily verified". To our knowledge they have no machine-checked proof. The work this mission asks for is the formal verification of that computation for the aggregated rate function, including unions in which two clusters of the same type meet, and the normalization of an infinite product over a countable set of types, which the book takes for granted.

Difficulty

The book calls the detailed balance computation readily verified; the bookkeeping is where a formal check can fail. Two different unions, or a break-up and the entry of a one-cluster, can lead from the same state to the same state when a cluster type is reproduced by the transition, so the rate between two states is a sum over transitions, and the identity has to be checked for the sums, not for single transitions. The same-type case r=sr = sr=s carries the falling factorial mr(mr−1)m_r(m_r - 1)mr​(mr​−1) rather than mr2m_r^2mr2​.

The normalization is the second difficulty. The state space of the open process is countably infinite, the product (8.10) runs over infinitely many types, and the interchange ∑m∏r=∏r∑n\sum_m \prod_r = \prod_r \sum_{n}∑m​∏r​=∏r​∑n​ that makes it sum to one requires (8.9). Without (8.9) the weights are not summable; this is the content of the book's remark that (8.9) is necessary.

Formalization scope

Cluster types are an arbitrary countable type R carrying a linear order, used only to count each unordered pair of types once. States are finitely supported vectors R →₀ ℕ. The rate q(m,m′)q(m, m')q(m,m′) is the sum of the rates (8.3) of all unions and break-ups taking mmm to m′m'm′, plus, for the open process, the rates (8.7); a transition is present only when the clusters it consumes exist, and q(m,m)=0q(m, m) = 0q(m,m)=0 holds because every transition changes the number of clusters. The one-cluster type is a distinguished element one. The closed state space is a finite non-empty set closed under positive-rate transitions and irreducible. The open process carries the book's assumptions that every state is reachable from every other and that the total rate out of each state is finite.

The statements are at the level of rates: reversibility is detailed balance for a positive, normalized π\piπ, the equivalence with reversibility of the stationary process being Kelly's Theorem 1.3. That π\piπ is the law of a stationary Markov process with these rates is not formalized. Independence and the Poisson laws are stated as the joint law of every finite set of counts. A formalization in which crc_rcr​ may vanish, in which detailed balance is imposed only for a degenerate choice of rates, or in which (8.10) is not normalized does not meet the goal: the crc_rcr​ are positive, the rates are arbitrary non-negative symmetric parameters, and both detailed balance and total mass one are required.

The development uses the published definitions KellyStochasticNetworks_Balance (detailed balance and the equilibrium equations) and the proved implication from detailed balance to the equilibrium equations. Lemmas on summing products over finitely supported vectors, and on infinite products of exponentials, are reusable beyond this mission. Lemma 8.3, a partial converse to Theorem 8.1, is not part of this mission; a faithful statement of it, with the units structure of the clusters, is a welcome addition.

Selected references

  • F. P. Kelly, Reversibility and Stochastic Networks, Wiley, Chichester, 1979, Chapter 8. Reissued by Cambridge University Press, 2011. https://doi.org/10.1017/CBO9780511564246 (author's copy: http://www.statslab.cam.ac.uk/~frank/BOOKS/kelly_book.html)
  • P. Whittle, Systems in Stochastic Equilibrium, Wiley, 1986 (reversible models of association and polymerization). ISBN 978-0-471-90887-0.
  • F. P. Kelly and E. Yudovina, Stochastic Networks, Cambridge University Press, 2014 (detailed balance and the equilibrium equations used here). https://doi.org/10.1017/CBO9781139565363
8 thms4 active usersReviewed
🏆Completed
Markov ChainOperations ResearchStochastic Systems·Captain: mikedeng1

Reversibility and Stochastic Networks II: Migration Processes with Blocking — Reversibility and Product-Form EquilibriumTextbook

Motivation

A migration process is a continuous-time Markov model of a population spread over JJJ sites, called colonies, in which individuals move one at a time: between colonies, out of the system, or into it from outside. The model was introduced by Whittle (1967) and Kingman (1969) and is the framework of Chapter 2 of F. P. Kelly, Reversibility and Stochastic Networks (Wiley, 1979). It covers closed and open networks of queues (Jackson networks), linear population models and the movement of particles between cells. Its central fact is that the equilibrium distribution is a product form: the joint law of the colony sizes factorizes into one term per colony, and for open processes the colony sizes are independent in equilibrium.

In the migration processes of Chapter 2 the rate at which an individual leaves colony jjj depends on the number njn_jnj​ there, but not on the number already present in the colony it joins. Chapter 6 (§6.1) lifts this restriction. The rate of a move into colony kkk is multiplied by a function ψk(nk)\psi_k(n_k)ψk​(nk​) of the receiving colony's occupancy, which can model blocking (a crowded colony slows arrivals) or attraction (as in the models of social grouping of §6.2). The price is that product form survives only under a reversibility condition on the routing parameters. This mission formalizes the two theorems of §6.1 on top of the already-formalized results of Chapter 2.

Timeline. Whittle (1967) and Kingman (1969) introduced migration processes and their product-form equilibria; Kelly (1976) treated networks of queues with general routes; Kelly (1979, Ch. 2) states the closed and open product forms (Theorems 2.3, 2.4) and the time reversal of an open process (Theorem 2.5); Kelly (1979, §6.1) states the reversible versions with receiving-colony dependence (Theorems 6.1, 6.2).

Setting

There are JJJ colonies. The state is n=(n1,…,nJ)∈NJn = (n_1,\dots,n_J) \in \mathbb{N}^Jn=(n1​,…,nJ​)∈NJ, with njn_jnj​ the number of individuals in colony jjj. Three operators change the state by one individual: TjknT_{jk}nTjk​n moves one individual from colony jjj to colony kkk, Tj⋅nT_{j\cdot}nTj⋅​n removes one from colony jjj, and T⋅knT_{\cdot k}nT⋅k​n adds one to colony kkk.

The model is given by non-negative constants λjk\lambda_{jk}λjk​ (with λjj=0\lambda_{jj} = 0λjj​=0), μj\mu_jμj​, νk\nu_kνk​, and functions φj,ψj:N→R\varphi_j,\psi_j : \mathbb{N}\to\mathbb{R}φj​,ψj​:N→R with φj(0)=0\varphi_j(0) = 0φj​(0)=0, φj(n)>0\varphi_j(n) > 0φj​(n)>0 for n>0n > 0n>0, and ψj(n)>0\psi_j(n) > 0ψj​(n)>0 for all n≥0n \ge 0n≥0. A closed reversible migration process with NNN individuals has state space S={n:∑jnj=N}\mathcal{S} = \{n : \sum_j n_j = N\}S={n:∑j​nj​=N} and transition rates

q(n,Tjkn)=λjk φj(nj) ψk(nk).(6.2)q(n, T_{jk}n) = \lambda_{jk}\,\varphi_j(n_j)\,\psi_k(n_k). \qquad (6.2)q(n,Tjk​n)=λjk​φj​(nj​)ψk​(nk​).(6.2)

An open reversible migration process has state space NJ\mathbb{N}^JNJ and, in addition to (6.2), the rates

q(n,Tj⋅n)=μj φj(nj)(6.5),q(n,T⋅kn)=νk ψk(nk)(6.6).q(n, T_{j\cdot}n) = \mu_j\,\varphi_j(n_j) \quad (6.5), \qquad q(n, T_{\cdot k}n) = \nu_k\,\psi_k(n_k) \quad (6.6).q(n,Tj⋅​n)=μj​φj​(nj​)(6.5),q(n,T⋅k​n)=νk​ψk​(nk​)(6.6).

The parameters are required to make the process irreducible: in the closed case an individual can pass between any two colonies along pairs with λab>0\lambda_{ab} > 0λab​>0; in the open case it can reach every colony from outside and leave from every colony. Setting ψj≡1\psi_j \equiv 1ψj​≡1 recovers the migration processes of Chapter 2.

Given positive constants α1,…,αJ\alpha_1,\dots,\alpha_Jα1​,…,αJ​, the candidate equilibrium is

π(n)=B∏j=1J{αjnj∏r=1njψj(r−1)φj(r)},(6.3)\pi(n) = B\prod_{j=1}^{J}\Bigl\{\alpha_j^{n_j}\prod_{r=1}^{n_j}\frac{\psi_j(r-1)}{\varphi_j(r)}\Bigr\}, \qquad (6.3)π(n)=Bj=1∏J​{αjnj​​r=1∏nj​​φj​(r)ψj​(r−1)​},(6.3)

with BBB chosen so that π\piπ sums to one over the state space.

Formalization targets

Goal: Theorem 6.2 (open process)

If positive αj\alpha_jαj​ satisfy

αjλjk=αkλkj(6.4),αjμj=νj(6.7),\alpha_j\lambda_{jk} = \alpha_k\lambda_{kj} \quad (6.4), \qquad \alpha_j\mu_j = \nu_j \quad (6.7),αj​λjk​=αk​λkj​(6.4),αj​μj​=νj​(6.7),

and every colony series gj=∑m≥0αjm∏r=1mψj(r−1)/φj(r)g_j = \sum_{m\ge 0}\alpha_j^m\prod_{r=1}^m \psi_j(r-1)/\varphi_j(r)gj​=∑m≥0​αjm​∏r=1m​ψj​(r−1)/φj​(r) converges, then (6.3) with B=∏jgj−1B = \prod_j g_j^{-1}B=∏j​gj−1​ is in detailed balance with the rates (6.2), (6.5), (6.6), satisfies the equilibrium equations, is positive and sums to one, and under it n1,…,nJn_1,\dots,n_Jn1​,…,nJ​ are independent with marginals πj(m)=gj−1αjm∏r=1mψj(r−1)/φj(r)\pi_j(m) = g_j^{-1}\alpha_j^m\prod_{r=1}^m\psi_j(r-1)/\varphi_j(r)πj​(m)=gj−1​αjm​∏r=1m​ψj​(r−1)/φj​(r).

Milestones

  • Theorem 2.3: the closed migration process (ψ≡1\psi\equiv 1ψ≡1) has equilibrium of the form (2.3).
  • Theorem 2.4: the open migration process has independent colonies with marginals bjαjnj/∏r=1njφj(r)b_j\alpha_j^{n_j}/\prod_{r=1}^{n_j}\varphi_j(r)bj​αjnj​​/∏r=1nj​​φj​(r).
  • Theorem 2.5: the reversal of a stationary open migration process is an open migration process.
  • Theorem 6.1: the closed process with rates (6.2) is reversible under (6.4), with equilibrium (6.3) on S\mathcal{S}S.

Theorems 2.3–2.5 are already proved on the platform and enter as references.

Significance

The result. Theorem 6.2 shows that product form and independence of colony sizes are not tied to routing that ignores the destination's occupancy. Any positive ψk\psi_kψk​ is allowed, provided the routing is reversible in the sense of (6.4) and (6.7). This is what makes the social-grouping models of §6.2 and the clustering models of Chapter 8 tractable, and it identifies (6.4) as the structural condition: Exercise 6.1.1 shows that without it the process cannot be reversible. Through Theorem 1.3 of the book, detailed balance also gives that the stationary process looks the same run backwards in time.

Formalization. The Chapter 2 results (Theorems 2.3–2.5) are formalized and proved on the platform, at the level of transition rates, in the KellyStochasticNetworks series. Theorems 6.1 and 6.2 are not formalized anywhere to our knowledge. The new work is the detailed-balance verification with the receiving-colony factor, the normalization over the finite set S\mathcal{S}S in the closed case, and the summation over the countable space NJ\mathbb{N}^JNJ with the marginal computation that expresses independence in the open case.

Difficulty

The equilibrium equations of a migration process with blocking have no simple solution in general (p. 135); a direct attack on the full balance equations does not close. Detailed balance is a local condition, but the formal statement has to be careful with transitions that do not exist: a move out of an empty colony, a departure that would make a count negative, and the coincidences between operators at the boundary (Tjkn=T⋅knT_{jk}n = T_{\cdot k}nTjk​n=T⋅k​n when nj=0n_j = 0nj​=0). In the open case the bookkeeping is in infinite sums: positivity and normalization need convergence of each colony series, and independence needs the marginal of π\piπ on one colony to be computed as a sum over the remaining J−1J-1J−1 coordinates.

Formalization scope

Colonies are Fin J; states are Fin J → ℕ; rates are real-valued functions of two states, assembled as sums of indicator terms over the possible transitions, reusing the operators Tjk, Tout, Tin and the predicates DetailedBalance, FullBalance of the published definitions KellyStochasticNetworks_Migration and KellyStochasticNetworks_Balance. Transitions out of an empty colony carry the factor φj(0)=0\varphi_j(0) = 0φj​(0)=0 and so vanish. The hypotheses λjj=0\lambda_{jj} = 0λjj​=0, non-negativity of the rates, positivity of φj(n)\varphi_j(n)φj​(n) (n>0n>0n>0), ψj(n)\psi_j(n)ψj​(n) (n≥0n\ge0n≥0) and αj\alpha_jαj​, and the book's irreducibility requirements are binders of each theorem.

The statements are at the level of transition rates: "reversible" is read as detailed balance of the equilibrium distribution, and "equilibrium distribution" as positive, summing to one, and satisfying the equilibrium equations. The stochastic process itself is not constructed. In the closed case the distribution lives on NJ\mathbb{N}^JNJ and vanishes off S\mathcal{S}S, which is equivalent because every transition preserves ∑jnj\sum_j n_j∑j​nj​; J≥1J \ge 1J≥1 is assumed so that S\mathcal{S}S is nonempty. In the open case stationarity is the convergence of each colony series, carried as HasSum hypotheses with sums gjg_jgj​.

The normalizing constant is never free: π≡0\pi \equiv 0π≡0 satisfies detailed balance, so a statement that leaves BBB unconstrained, or omits the factor ψk(nk)\psi_k(n_k)ψk​(nk​) from (6.2), (6.6) and (6.3), is not this theorem. Contributions welcome: proofs of Theorem 6.1 and 6.2, reusable lemmas on detailed balance for indicator-sum rates, and on products of summable families over Fin J → ℕ.

Selected references

  • F. P. Kelly, Reversibility and Stochastic Networks, Wiley, 1979; reprinted Cambridge University Press, 2011. Chapters 2 and 6. http://www.statslab.cam.ac.uk/~frank/BOOKS/kelly_book.html
  • J. F. C. Kingman, Markov population processes, Journal of Applied Probability 6 (1969), 1–18. https://doi.org/10.2307/3212273
  • F. P. Kelly, Networks of queues, Advances in Applied Probability 8 (1976), 416–432. https://doi.org/10.2307/1426136
  • P. Whittle, Nonlinear migration processes, Bulletin of the International Statistical Institute 42 (1967).
  • F. P. Kelly and E. Yudovina, Stochastic Networks, Cambridge University Press, 2014, Chapter 2. http://www.statslab.cam.ac.uk/~frank/STOCHNET/
9 thms4 active usersReviewed
🏆Completed
Machine LearningOptimizationStatistics·Captain: mikedeng1

Variance-based Regularization with Convex Objectives III: Localized-Rademacher Risk Bounds for the Robust MinimizerResearch Paper

Why variance-regularized risk bounds

In statistical learning, one picks a function fff from a class F\mathcal FF to make the population risk E[f]\mathbb E[f]E[f] small, with access only to an i.i.d. sample x1,…,xnx_1,\dots,x_nx1​,…,xn​ from an unknown distribution PPP. Empirical risk minimization replaces E[f]\mathbb E[f]E[f] by the empirical mean EP^n[f]\mathbb E_{\widehat P_n}[f]EPn​​[f], and its classical guarantees decay like 1/n1/\sqrt n1/n​ regardless of how concentrated fff is. Bernstein-type inequalities show that the deviation of EP^n[f]\mathbb E_{\widehat P_n}[f]EPn​​[f] from E[f]\mathbb E[f]E[f] scales with the standard deviation of fff, so a procedure that minimizes "empirical risk plus a standard-deviation penalty" can, in principle, achieve faster rates when the variance at the optimum is small (Maurer and Pontil, 2009). The penalized objective is non-convex even when every fff is convex in its parameters, which makes it hard to optimize.

J. C. Duchi and H. Namkoong (arXiv:1610.02581v3, 2017) replace the penalty by a distributionally robust objective: the worst-case risk over all reweightings of the sample within a χ2\chi^2χ2-divergence ball. This objective is convex whenever the losses are, and (Theorem 1 of the paper) it equals the empirical mean plus a standard-deviation penalty up to an error of order 1/n1/n1/n. This mission formalizes the paper's guarantee for the minimizer of that robust objective in terms of localized Rademacher complexities (Section 3.2, Theorem 4), the sharpest of the paper's three generalization analyses. It is the third of four missions on the paper.

Setting

Let PPP be a probability measure on a measurable space X\mathcal XX and x1,…,xnx_1,\dots,x_nx1​,…,xn​, n≥1n\ge1n≥1, an i.i.d. sample from PPP with empirical distribution P^n\widehat P_nPn​. Let M≥1M\ge1M≥1 and let F\mathcal FF be a collection of measurable functions f:X→[0,M]f:\mathcal X\to[0,M]f:X→[0,M] (losses).

  • The χ2\chi^2χ2 ball of radius ρ≥0\rho\ge0ρ≥0 is the set Pn\mathcal P_nPn​ of weight vectors p∈Rnp\in\mathbb R^np∈Rn with pi≥0p_i\ge0pi​≥0, ∑ipi=1\sum_ip_i=1∑i​pi​=1 and 12∑i(npi−1)2≤ρ\frac12\sum_i(np_i-1)^2\le\rho21​∑i​(npi​−1)2≤ρ; equivalently, the distributions PPP on the sample with Dϕ(P∥P^n)≤ρ/nD_\phi(P\|\widehat P_n)\le\rho/nDϕ​(P∥Pn​)≤ρ/n for ϕ(t)=12(t−1)2\phi(t)=\frac12(t-1)^2ϕ(t)=21​(t−1)2.
  • The robust risk of fff is sup⁡P: Dϕ(P∥P^n)≤ρ/nEP[f]=sup⁡p∈Pn∑ipif(xi)\sup_{P:\,D_\phi(P\|\widehat P_n)\le\rho/n}\mathbb E_P[f]=\sup_{p\in\mathcal P_n}\sum_ip_if(x_i)supP:Dϕ​(P∥Pn​)≤ρ/n​EP​[f]=supp∈Pn​​∑i​pi​f(xi​), and a robust minimizer f^\widehat ff​ minimizes it over F\mathcal FF.
  • The empirical Rademacher complexity is Rn(F)=Eε[sup⁡f∈F1n∑iεif(xi)]\mathfrak R_n(\mathcal F)=\mathbb E_\varepsilon\big[\sup_{f\in\mathcal F}\frac1n\sum_i\varepsilon_if(x_i)\big]Rn​(F)=Eε​[supf∈F​n1​∑i​εi​f(xi​)] with i.i.d. uniform signs εi∈{−1,1}\varepsilon_i\in\{-1,1\}εi​∈{−1,1}, and E[Rn(F)]\mathbb E[\mathfrak R_n(\mathcal F)]E[Rn​(F)] averages it over the sample.
  • A function ψ:R+→R+\psi:\mathbb R_+\to\mathbb R_+ψ:R+​→R+​ is sub-root if it is nonnegative, nondecreasing, and r↦ψ(r)/rr\mapsto\psi(r)/\sqrt rr↦ψ(r)/r​ is nonincreasing on r>0r>0r>0.
  • The localization inequality (20) asks that, for all r≥0r\ge0r≥0,
ψn(r) ≥ E[Rn({cf:f∈F, c∈[0,1], E[c2f2]≤r})],\psi_n(r)\ \ge\ \mathbb E\big[\mathfrak R_n(\{cf : f\in\mathcal F,\ c\in[0,1],\ \mathbb E[c^2f^2]\le r\})\big],ψn​(r) ≥ E[Rn​({cf:f∈F, c∈[0,1], E[c2f2]≤r})],

with ψn\psi_nψn​ sub-root, and rn⋆>0r_n^\star>0rn⋆​>0 is a point with rn⋆≥ψn(rn⋆)r_n^\star\ge\psi_n(r_n^\star)rn⋆​≥ψn​(rn⋆​).

Formalization targets

Goal: Theorem 4, inequality (23), as its proof establishes it

Let 0<t<n0<t<n0<t<n and let ρ\rhoρ satisfy (21): ρn≥8(45Mn(t+log⁡⌈log⁡nt⌉)+18rn⋆)\frac\rho n\ge8\big(\frac{45M}n\big(t+\log\lceil\log\frac nt\rceil\big)+18r_n^\star\big)nρ​≥8(n45M​(t+log⌈logtn​⌉)+18rn⋆​). With probability at least 1−4e−t1-4e^{-t}1−4e−t, every robust minimizer f^\widehat ff​ satisfies

E[f^] ≤ (1+22ρn)inf⁡f∈F(E[f]+182ρ45nVar(f))+(14+62ρn)M(3ρ+t)n.\mathbb E[\widehat f]\ \le\ \Big(1+2\sqrt{\tfrac{2\rho}n}\Big)\inf_{f\in\mathcal F}\Big(\mathbb E[f]+\sqrt{\tfrac{182\rho}{45n}\mathrm{Var}(f)}\Big)+\Big(14+6\sqrt{\tfrac{2\rho}n}\Big)\frac{M(3\rho+t)}n .E[f​] ≤ (1+2n2ρ​​)f∈Finf​(E[f]+45n182ρ​Var(f)​)+(14+6n2ρ​​)nM(3ρ+t)​.

Milestones

In attack order: Bousquet's form of Talagrand's inequality (Lemma B.2); the elementary root bound (Lemma D.4); the contraction principle (Lemma D.5, a published theorem); the uniform Bernstein inequality with Rademacher complexity (Lemma D.1); its localized version in terms of rn⋆r_n^\starrn⋆​ (Lemma D.2); localized second-moment bounds (Lemma D.3); the deterministic expansion (10) of Theorem 1,

(2ρnsn2−2Mρn)+≤sup⁡PEP[Z]−EP^n[Z]≤2ρnsn2;\Big(\sqrt{\tfrac{2\rho}n s_n^2}-\tfrac{2M\rho}n\Big)_+\le\sup_{P}\mathbb E_P[Z]-\mathbb E_{\widehat P_n}[Z]\le\sqrt{\tfrac{2\rho}ns_n^2};(n2ρ​sn2​​−n2Mρ​)+​≤Psup​EP​[Z]−EPn​​[Z]≤n2ρ​sn2​​;

and the uniform bound (22): with probability at least 1−2e−t1-2e^{-t}1−2e−t, for all f∈Ff\in\mathcal Ff∈F,

E[f]≤(1+22ρn)sup⁡P: Dϕ(P∥P^n)≤ρ/nEP[f]+(13+42ρn)Mρn.\mathbb E[f]\le\Big(1+2\sqrt{\tfrac{2\rho}n}\Big)\sup_{P:\,D_\phi(P\|\widehat P_n)\le\rho/n}\mathbb E_P[f]+\Big(13+4\sqrt{\tfrac{2\rho}n}\Big)\frac{M\rho}n .E[f]≤(1+2n2ρ​​)P:Dϕ​(P∥Pn​)≤ρ/nsup​EP​[f]+(13+4n2ρ​​)nMρ​.

Significance

The bound (23) says that the robust minimizer competes with the best trade-off between risk and standard deviation in the class, and that the complexity of the class enters only through the fixed point rn⋆r_n^\starrn⋆​ of a localized complexity bound. For bounded VC classes rn⋆r_n^\starrn⋆​ is of order dlog⁡(n/d)n\frac{d\log(n/d)}nndlog(n/d)​ (Bartlett, Bousquet and Mendelson, 2005, Corollary 3.7), so when the optimal function has small variance the excess risk is of order ρ/n\rho/nρ/n, faster than the 1/n1/\sqrt n1/n​ of uniform covering arguments; and localized complexities apply to classes, such as balls of reproducing kernel Hilbert spaces, whose covering numbers are too large for the covering-number analysis of the paper's Theorem 3 (mission II of this series).

The paper's result is proved, not open. No part of it, and none of the localized-complexity machinery of Bartlett, Bousquet and Mendelson, is formalized in Lean or Mathlib to our knowledge. The mission produces a checked version of the theorem with every constant explicit and, along the way, the localization lemmas D.1–D.3, which are reusable for any localized-complexity analysis. Reading the proof also exposed three arithmetic slips in the printed statements; the mission states what the proof establishes (see Formalization scope).

Difficulty

The obvious route applies a uniform concentration inequality to F\mathcal FF and then a Bernstein bound to each fff. Talagrand's inequality applied to the whole class gives a deviation governed by the largest variance in the class and by the global complexity E[Rn(F)]\mathbb E[\mathfrak R_n(\mathcal F)]E[Rn​(F)], which yields only 1/n1/\sqrt n1/n​ rates. Obtaining a deviation that scales with each function's own second moment requires peeling the class into shells of comparable second moment and a fixed-point argument on the sub-root bound, with a union bound whose cost appears as log⁡⌈log⁡nt⌉\log\lceil\log\frac nt\rceillog⌈logtn​⌉. The two directions of the localized inequalities (population to sample, and sample to population for second moments) must then be combined with the deterministic expansion (10) while keeping the constants explicit. A further subtlety is the self-normalized rescaling f↦r/(E[f2]∨r) ff\mapsto\sqrt{r/(\mathbb E[f^2]\vee r)}\,ff↦r/(E[f2]∨r)​f, which differs from the variance normalization of Bartlett et al. and is what makes the bound compatible with the robust objective.

Formalization scope

Lean conventions. The sample is the coordinate map of the product measure PnP^nPn on Fin n → X. Distributions on the sample are weight vectors in the χ2\chi^2χ2 ball; the robust risk is the real supremum over that ball (attained, since the ball is nonempty and compact for n≥1n\ge1n≥1, ρ≥0\rho\ge0ρ≥0). Population means and variances are ∫ x, f x ∂P and ProbabilityTheory.variance f P for measurable bounded fff; empirical means and variances are normalized by 1/n1/n1/n. The empirical Rademacher complexity is the published UnderstandingML_Rademacher definition evaluated on {(f(x1),…,f(xn))}\{(f(x_1),\dots,f(x_n))\}{(f(x1​),…,f(xn​))}. Its expectation is a Bochner integral, and every hypothesis that bounds it also asserts that the integrand is integrable: otherwise the integral is 000, (20) would hold for free, and the theorem would be false. Probability bounds are stated for the failure event under PnP^nPn (an outer measure when the event is not measurable). The goal speaks about every minimizer of the robust risk, so it is not vacuous when the set of minimizers is empty. The condition rn⋆>0r_n^\star>0rn⋆​>0 is part of the page's "root" (and the proof divides by rn⋆\sqrt{r_n^\star}rn⋆​​); with rn⋆=0r_n^\star=0rn⋆​=0 allowed, ψ(r)=r\psi(r)=\sqrt rψ(r)=r​ would remove the complexity term from (21). The condition t<nt<nt<n makes log⁡⌈log⁡nt⌉\log\lceil\log\frac nt\rceillog⌈logtn​⌉ defined.

Corrections of printed statements, each recorded in the item's docstring and Formalization Note (the milestone texts stay verbatim):

  • (22) is stated with probability 1−2e−t1-2e^{-t}1−2e−t; the paper prints 1−e−t1-e^{-t}1−e−t, and its proof (p. 41) concludes 1−2e−t1-2e^{-t}1−2e−t.
  • (23) is stated with probability 1−4e−t1-4e^{-t}1−4e−t (printed 1−3e−t1-3e^{-t}1−3e−t; the proof adds two fixed-fff events to the two of (22)) and with 182ρ45n\frac{182\rho}{45n}45n182ρ​ (printed 91ρ45n\frac{91\rho}{45n}45n91ρ​; the proof's step ρ+t≤91ρ/45\sqrt\rho+\sqrt t\le\sqrt{91\rho/45}ρ​+t​≤91ρ/45​ multiplies 2Var(f)/n\sqrt{2\mathrm{Var}(f)/n}2Var(f)/n​).
  • Lemma D.3 is stated with the additive term 72M2(1+η)rn⋆+(4(1+η)+143)M2tn72M^2(1+\eta)r_n^\star+(4(1+\eta)+\frac{14}3)\frac{M^2t}n72M2(1+η)rn⋆​+(4(1+η)+314​)nM2t​ and, in the reversed direction, the coefficient 1+11+η1+\frac1{1+\eta}1+1+η1​, as its proof yields (printed: Mtn(4+73M)\frac{Mt}n(4+\frac73M)nMt​(4+37​M) and 1+η1+η1+\frac\eta{1+\eta}1+1+ηη​), under Theorem 4's standing hypothesis M≥1M\ge1M≥1.
  • Lemma D.5 is linked to the published contraction lemma UnderstandingML.contraction_lemma, which states it at a fixed sample for nonempty bounded classes and allows a different Lipschitz map per coordinate.

Contributions welcome: proofs of the milestones in any order; Lemma B.2 (Bousquet's inequality) is the deepest single ingredient and is reusable well beyond this mission, as are the peeling Lemma D.1 and the sub-root fixed-point Lemma D.2.

Selected references

  • J. C. Duchi and H. Namkoong, Variance-based regularization with convex objectives, arXiv:1610.02581v3, 2017. https://arxiv.org/abs/1610.02581
  • P. L. Bartlett, O. Bousquet and S. Mendelson, Local Rademacher complexities, Annals of Statistics 33(4), 2005. https://doi.org/10.1214/009053605000000282
  • O. Bousquet, A Bennett concentration inequality and its application to suprema of empirical processes, Comptes Rendus Mathématique 334(6), 2002. https://doi.org/10.1016/S1631-073X(02)02292-6
  • A. Maurer and M. Pontil, Empirical Bernstein bounds and sample variance penalization, COLT 2009. https://arxiv.org/abs/0907.3740
  • M. Ledoux and M. Talagrand, Probability in Banach Spaces, Springer, 1991. https://doi.org/10.1007/978-3-642-20212-4
14 thms4 active usersReviewed
🏆Completed
Algorithmic Game TheoryOperations Research·Captain: mikedeng1

The Price of Anarchy of Finite Congestion Games V: The Mixed Price of Anarchy of the Average Social Cost Is at Most (3+sqrt 5)/2Research Paper

Motivation

When many self-interested users share resources whose cost grows with use (links of a network, servers, machines), each user picks the option that is cheapest for them given what the others do, and the resulting equilibrium can be worse for the group than a centrally planned allocation. The price of anarchy, introduced by Koutsoupias and Papadimitriou (STACS 1999), measures this loss as the worst ratio between the social cost of an equilibrium and the optimal social cost. Congestion games (Rosenthal 1973) are the standard finite model of such resource sharing: they always have pure Nash equilibria, and they cover atomic routing, load balancing and many network design questions.

Timeline of the linear-latency case:

  • 2002: Roughgarden and Tardos (J. ACM) bound the price of anarchy of nonatomic selfish routing with linear latencies by 4/34/34/3.
  • 2005: Awerbuch, Azar and Epstein (STOC 2005) and, independently, Christodoulou and Koutsoupias (STOC 2005) show that for atomic (finite) congestion games with linear latencies the pure price of anarchy of the total cost is 5/25/25/2. Awerbuch, Azar and Epstein also obtain (3+5)/2≈2.618(3+\sqrt5)/2 \approx 2.618(3+5​)/2≈2.618 for weighted players.
  • 2005: Christodoulou and Koutsoupias observe that their argument for 5/25/25/2 extends to mixed Nash equilibria, at the price of the larger constant (3+5)/2(3+\sqrt5)/2(3+5​)/2 (their Theorem 14, the goal of this mission).

Setting

A congestion game consists of a finite set NNN of players, a finite set EEE of facilities, for each player iii a collection Σi\Sigma_iΣi​ of pure strategies, each a subset of EEE, and for each facility eee a latency function fe:N→Rf_e : \mathbb N \to \mathbb Rfe​:N→R. In a pure profile A=(A1,…,An)A = (A_1, \dots, A_n)A=(A1​,…,An​), Ai∈ΣiA_i \in \Sigma_iAi​∈Σi​, the load ne(A)n_e(A)ne​(A) is the number of players whose set contains eee, and player iii pays

ci(A)=∑e∈Aife(ne(A)).c_i(A) = \sum_{e \in A_i} f_e\bigl(n_e(A)\bigr).ci​(A)=e∈Ai​∑​fe​(ne​(A)).

The social cost of a pure profile is SUM(A)=∑i∈Nci(A)\mathrm{SUM}(A) = \sum_{i\in N} c_i(A)SUM(A)=∑i∈N​ci​(A) (NNN times the average cost). Latencies are linear: fe(k)=aek+bef_e(k) = a_e k + b_efe​(k)=ae​k+be​ with ae,be≥0a_e, b_e \ge 0ae​,be​≥0.

A mixed strategy pip_ipi​ of player iii is a probability distribution on Σi\Sigma_iΣi​. The players randomize independently, so the pure profile sss occurs with probability Pr⁡(s)=∏jpj(sj)\Pr(s) = \prod_j p_j(s_j)Pr(s)=∏j​pj​(sj​). Player iii's expected cost is E[ci]=∑sPr⁡(s) ci(s)\mathbb E[c_i] = \sum_s \Pr(s)\, c_i(s)E[ci​]=∑s​Pr(s)ci​(s), and the expected load of eee is E[ne]=∑sPr⁡(s) ne(s)\mathbb E[n_e] = \sum_s \Pr(s)\, n_e(s)E[ne​]=∑s​Pr(s)ne​(s). A mixed profile is a mixed Nash equilibrium if no player can lower their expected cost by switching unilaterally to another distribution on their strategies. The social cost of a mixed profile is the sum of the expected costs, SUM(p)=∑iE[ci]\mathrm{SUM}(p) = \sum_i \mathbb E[c_i]SUM(p)=∑i​E[ci​].

In Lean: CongestionGame ι E with fields strategies and latency, load, cost, IsProfile, sumCost, IsLinear, expCost, expLoad, mixedSumCost and IsMixedNash, all in the namespace CongestionPoA.Mixed; lotteries and independent randomization come from the platform definition agt_games (AGT.IsLottery, AGT.profileProb, AGT.IsMixedNash).

Formalization targets

Goal: Theorem 14

For every congestion game with linear latencies, every mixed Nash equilibrium ppp and every pure profile PPP with Pi∈ΣiP_i \in \Sigma_iPi​∈Σi​,

∑i∈NE[ci]  ≤  3+52 SUM(P).\sum_{i\in N} \mathbb E[c_i] \;\le\; \frac{3+\sqrt5}{2}\,\mathrm{SUM}(P).i∈N∑​E[ci​]≤23+5​​SUM(P).

Taking PPP optimal, the mixed price of anarchy of the average social cost is at most (3+5)/2(3+\sqrt5)/2(3+5​)/2.

Milestones

  1. Lemma 3 (corrected). For real x≥0x \ge 0x≥0 and integer y≥0y \ge 0y≥0, y(x+1)≤5−14x2+5+54y2y(x+1) \le \frac{\sqrt5-1}{4}x^2 + \frac{\sqrt5+5}{4}y^2y(x+1)≤45​−1​x2+45​+5​y2.
  2. Deviation inequality (Theorem 1's proof, mixed). At a mixed Nash equilibrium, for every player iii,
E[ci]≤∑e∈Pi(ae(E[ne]+1)+be).\mathbb E[c_i] \le \sum_{e\in P_i}\bigl(a_e(\mathbb E[n_e]+1) + b_e\bigr).E[ci​]≤e∈Pi​∑​(ae​(E[ne​]+1)+be​).
  1. Summing step (Theorem 1's proof, mixed).
∑iE[ci]≤∑e∈Ene(P)(ae(E[ne]+1)+be).\sum_i \mathbb E[c_i] \le \sum_{e\in E} n_e(P)\bigl(a_e(\mathbb E[n_e]+1) + b_e\bigr).i∑​E[ci​]≤e∈E∑​ne​(P)(ae​(E[ne​]+1)+be​).

Significance

The bound says that randomization by the players cannot make linear congestion games much worse than pure play: the loss stays within a constant factor independent of the number of players and facilities. Mixed equilibria matter because they always exist in every finite game and because they model populations of users whose individual choices are not known in advance. The same authors report (PDF p. 2) extending these bounds to correlated equilibria with the same values, which places the mixed bound in a hierarchy of equilibrium notions whose worst cases coincide.

The result is proved in the paper, in one sentence that defers to the proof of Theorem 1. The work here is a complete machine-checked version: the expected-cost layer for congestion games on top of agt_games, the deviation and summing inequalities under product distributions, and the corrected Lemma 3. The paper prints Lemma 3 for all nonnegative reals x,yx, yx,y, where it is false (at x=0x = 0x=0, y=1/10y = 1/10y=1/10 the left side is 0.10.10.1 and the right side about 0.0180.0180.018); the formal statement keeps yyy an integer, which is how the lemma is used. No formalization of congestion-game price-of-anarchy bounds was on the platform when this mission was drafted.

Difficulty

The pure proof compares ne(P)(ne(A)+1)n_e(P)(n_e(A)+1)ne​(P)(ne​(A)+1) with ne(A)2n_e(A)^2ne​(A)2 and ne(P)2n_e(P)^2ne​(P)2 through an integer inequality (Lemma 1). Under a mixed equilibrium the equilibrium side is an expectation, so the cost of a facility is no longer a function of one integer load, and Lemma 1's constant 1/31/31/3 is not available for real arguments: a real-variable version is needed, and the constant degrades from 5/25/25/2 to (3+5)/2(3+\sqrt5)/2(3+5​)/2. A second obstacle is bookkeeping: the deviating player's load changes only on their own deviation, while the other players' randomization stays independent, and the expected cost of a player has to be related to expected facility loads, which uses linearity of the latencies in an essential way. The naive idea of applying Theorem 1 to each pure profile in the support of the equilibrium fails, because those profiles are not themselves Nash equilibria.

Formalization scope

Players and facilities are finite types ι and E; profiles are functions ι → Finset E, with feasibility IsProfile a separate predicate. Latencies are real-valued on natural-number loads, and "linear" means affine with nonnegative coefficients, fe(k)=aek+bef_e(k) = a_e k + b_efe​(k)=ae​k+be​; the paper displays only the identity latency fe(k)=kf_e(k)=kfe​(k)=k and states that its proofs extend. Mixed strategies are real weight functions on the finite strategy type Σi\Sigma_iΣi​; independence is built into AGT.profileProb; Nash deviations range over all lotteries, which is equivalent to pure deviations. agt_games maximizes payoffs, so the game is passed to it with payoff −ci-c_i−ci​. The goal is stated against every feasible pure profile rather than as a ratio, so no division by an optimum that may be 000 occurs. The social cost is the expected sum of the players' costs; the paper's second option, ∑eE[ne2]\sum_e \mathbb E[n_e^2]∑e​E[ne2​], is not part of this mission. A version with correlated distributions on profiles, or one that bounds the cost of an "averaged" profile instead of the expected cost, is a different statement and does not discharge the goal.

A complete development needs elementary finite-sum manipulation of product distributions (marginals of AGT.profileProb, expectations of loads), and a Jensen-type inequality (E[ne])2≤E[ne2](\mathbb E[n_e])^2 \le \mathbb E[n_e^2](E[ne​])2≤E[ne2​] for finite distributions. The expectation lemmas for congestion games are reusable for the other mixed results of the literature (weighted games, polynomial latencies). Proofs of the milestones, of the expectation layer, and alternative arguments are all welcome.

Selected references

  • G. Christodoulou, E. Koutsoupias, The Price of Anarchy of Finite Congestion Games, STOC 2005, pp. 67–73. https://doi.org/10.1145/1060590.1060600
  • B. Awerbuch, Y. Azar, A. Epstein, The Price of Routing Unsplittable Flow, STOC 2005, pp. 57–66. https://doi.org/10.1145/1060590.1060599
  • E. Koutsoupias, C. Papadimitriou, Worst-case Equilibria, STACS 1999, LNCS 1563, pp. 404–413. https://doi.org/10.1007/3-540-49116-3_38
  • R. W. Rosenthal, A Class of Games Possessing Pure-Strategy Nash Equilibria, International Journal of Game Theory 2 (1973), pp. 65–67. https://doi.org/10.1007/BF01737559
  • T. Roughgarden, É. Tardos, How Bad Is Selfish Routing?, Journal of the ACM 49 (2002), pp. 236–259. https://doi.org/10.1145/506147.506153
7 thms4 active usersReviewed
🏆Completed
Machine LearningOptimization·Captain: mikedeng1

Gradient Convergence in Gradient Methods with Errors II: With Zero-Mean Stochastic Errors, Almost Surely Either f(x_t) → −∞ or f(x_t) Converges and ∇f(x_t) → 0Research Paper

Motivation

Stochastic gradient methods minimize a function fff when only noisy estimates of its gradient are available: each step moves along a descent direction corrupted by random noise. They are the standard training algorithm for neural networks and the basic tool of stochastic approximation, and the question every user faces is what can be guaranteed when fff is nonconvex, possibly unbounded below, and the noise is allowed to grow with the gradient.

D. P. Bertsekas and J. N. Tsitsiklis, Gradient Convergence in Gradient Methods with Errors, SIAM J. Optim. 10(3):627–642, 2000 (DOI), answer this under minimal assumptions. Noise with variance growing in ∥∇f(xt)∥\|\nabla f(x_t)\|∥∇f(xt​)∥ had been handled for related methods (Poljak and Tsypkin 1973), but typically together with a lower bound on fff, under which f(xt)f(x_t)f(xt​) is approximately a supermartingale and the supermartingale convergence theorem applies (see the monographs of Kushner and Clark 1978; Benveniste, Métivier and Priouret 1990; Kushner and Yin 1996). Section 4 of the paper (p. 635) removes the lower bound: it proves that, with probability 1, either f(xt)→−∞f(x_t)\to-\inftyf(xt​)→−∞ or f(xt)f(x_t)f(xt​) converges and ∇f(xt)→0\nabla f(x_t)\to 0∇f(xt​)→0, without assuming bounded iterates. Section 5 shows that the randomized incremental gradient method for a finite-sum objective is a special case. This mission formalizes Section 4 and the Section 5 application. A companion mission covers the deterministic counterpart (Proposition 1 of the same paper).

Setting

Let f:Rn→Rf:\mathbb R^n\to\mathbb Rf:Rn→R be continuously differentiable with a Lipschitz gradient: there is L≥0L\ge 0L≥0 with

∥∇f(x)−∇f(xˉ)∥≤L∥x−xˉ∥∀x,xˉ.(2.1)\|\nabla f(x)-\nabla f(\bar x)\|\le L\|x-\bar x\|\qquad\forall x,\bar x. \tag{2.1}∥∇f(x)−∇f(xˉ)∥≤L∥x−xˉ∥∀x,xˉ.(2.1)

Let (Ω,F,P)(\Omega,\mathcal F,P)(Ω,F,P) be a probability space and F0⊆F1⊆⋯\mathcal F_0\subseteq\mathcal F_1\subseteq\cdotsF0​⊆F1​⊆⋯ an increasing sequence of σ\sigmaσ-fields (a filtration; Ft\mathcal F_tFt​ is the history of the algorithm just before the noise wtw_twt​ is drawn). The stochastic gradient method generates random vectors by

xt+1=xt+γt(st+wt),x_{t+1}=x_t+\gamma_t(s_t+w_t),xt+1​=xt​+γt​(st​+wt​),

where γt>0\gamma_t>0γt​>0 is a deterministic stepsize, sts_tst​ is a descent direction and wtw_twt​ is a noise term. The assumptions of Proposition 3 are:

  • (a) xtx_txt​ and sts_tst​ are Ft\mathcal F_tFt​-measurable;
  • (b) there are c1,c2>0c_1,c_2>0c1​,c2​>0 with c1∥∇f(xt)∥2≤−∇f(xt)′stc_1\|\nabla f(x_t)\|^2\le-\nabla f(x_t)'s_tc1​∥∇f(xt​)∥2≤−∇f(xt​)′st​ and ∥st∥≤c2(1+∥∇f(xt)∥)\|s_t\|\le c_2(1+\|\nabla f(x_t)\|)∥st​∥≤c2​(1+∥∇f(xt​)∥) for all ttt; (4.1)
  • (c) for all ttt, with probability 1, E[wt∣Ft]=0E[w_t\mid\mathcal F_t]=0E[wt​∣Ft​]=0 (4.2) and E[∥wt∥2∣Ft]≤A(1+∥∇f(xt)∥2)E[\|w_t\|^2\mid\mathcal F_t]\le A(1+\|\nabla f(x_t)\|^2)E[∥wt​∥2∣Ft​]≤A(1+∥∇f(xt​)∥2) (4.3), with A>0A>0A>0 deterministic;
  • (d) ∑tγt=∞\sum_t\gamma_t=\infty∑t​γt​=∞ and ∑tγt2<∞\sum_t\gamma_t^2<\infty∑t​γt2​<∞.

The noise variance in (c) may grow quadratically with ∥∇f(xt)∥\|\nabla f(x_t)\|∥∇f(xt​)∥ and is therefore unbounded in general. A point xˉ\bar xxˉ is stationary if ∇f(xˉ)=0\nabla f(\bar x)=0∇f(xˉ)=0.

Formalization targets

Goal: Proposition 3 (p. 635)

Under (2.1) and (a)–(d), with probability 1,

f(xt)→−∞or(f(xt)→ℓ∈R  and  ∇f(xt)→0),f(x_t)\to-\infty\quad\text{or}\quad\Bigl(f(x_t)\to\ell\in\mathbb R\ \text{ and }\ \nabla f(x_t)\to 0\Bigr),f(xt​)→−∞or(f(xt​)→ℓ∈R  and  ∇f(xt​)→0),

and every limit point of (xt)(x_t)(xt​) is a stationary point of fff. The dichotomy is per sample path: different paths may take different branches.

Milestones

  1. (4.4), p. 636. The pathwise one-step inequality: if γ 2Lc22≤c1/2\gamma\,2Lc_2^2\le c_1/2γ2Lc22​≤c1​/2 and sss satisfies (4.1) at xxx, then for every www,
f(x+γ(s+w))≤f(x)−γc12∥∇f(x)∥2+γ∇f(x)′w+γ22Lc22+γ2L∥w∥2.f(x+\gamma(s+w))\le f(x)-\gamma\tfrac{c_1}{2}\|\nabla f(x)\|^2+\gamma\nabla f(x)'w+\gamma^2 2Lc_2^2+\gamma^2L\|w\|^2.f(x+γ(s+w))≤f(x)−γ2c1​​∥∇f(x)∥2+γ∇f(x)′w+γ22Lc22​+γ2L∥w∥2.
  1. Lemma 2, p. 637. If rtr_trt​ is Ft+1\mathcal F_{t+1}Ft+1​-measurable with E[rt∣Ft]=0E[r_t\mid\mathcal F_t]=0E[rt​∣Ft​]=0, E[∥rt∥2∣Ft]≤BE[\|r_t\|^2\mid\mathcal F_t]\le BE[∥rt​∥2∣Ft​]≤B and ∑γt2<∞\sum\gamma_t^2<\infty∑γt2​<∞, then ∑t≤Tγtrt\sum_{t\le T}\gamma_tr_t∑t≤T​γt​rt​ and ∑t≤Tγt2∥rt∥2\sum_{t\le T}\gamma_t^2\|r_t\|^2∑t≤T​γt2​∥rt​∥2 converge almost surely.
  2. Lemma 6, p. 640. For every δ>0\delta>0δ>0, almost surely f(xt)f(x_t)f(xt​) converges to a finite value or to −∞-\infty−∞, and if the limit is not −∞-\infty−∞ then lim sup⁡t∥∇f(xt)∥≤δ\limsup_t\|\nabla f(x_t)\|\le\deltalimsupt​∥∇f(xt​)∥≤δ.

Further result: §5, pp. 641–642

For f=1m∑ifif=\frac1m\sum_i f_if=m1​∑i​fi​ with Lipschitz gradients ∇fi\nabla f_i∇fi​ satisfying ∥∇fi(x)∥≤C+D∥∇f(x)∥\|\nabla f_i(x)\|\le C+D\|\nabla f(x)\|∥∇fi​(x)∥≤C+D∥∇f(x)∥ (5.2), the randomized incremental gradient method xt+1=xt−γt∇fk(t)(xt)x_{t+1}=x_t-\gamma_t\nabla f_{k(t)}(x_t)xt+1​=xt​−γt​∇fk(t)​(xt​), with independent uniform indices k(t)k(t)k(t), satisfies the conclusion of Proposition 3.

Significance

Proposition 3 is a convergence guarantee for stochastic gradient descent on smooth nonconvex objectives that needs neither a lower bound on fff, nor bounded iterates, nor bounded noise variance. It contains, as special cases, stochastic gradient descent with unbiased gradient estimates whose variance grows with the gradient, the randomized incremental (single-sample) gradient method for finite sums of Section 5, and scaled or approximate gradient directions through condition (4.1). Its conclusion is the strongest one available at this generality: if f(xt)f(x_t)f(xt​) stays bounded below along a path, then the gradient vanishes along that path and every limit point is stationary.

The result is proved in the paper; to our knowledge it has not been machine-checked. Its formalization requires a working theory of generalized conditional expectations of non-integrable noise, square-integrable martingales in Rn\mathbb R^nRn and pathwise arguments over random interval partitions, which is reusable for other stochastic approximation results (Robbins–Monro type schemes, TD-learning, stochastic subgradient methods).

Difficulty

The natural first idea is to view f(xt)f(x_t)f(xt​) as a supermartingale up to summable errors and apply the supermartingale convergence theorem (Robbins–Siegmund). This fails here: the theorem needs f(xt)f(x_t)f(xt​) bounded below, and fff is not assumed bounded below; in addition the noise term γt2L∥wt∥2\gamma_t^2L\|w_t\|^2γt2​L∥wt​∥2 in (4.4) has conditional mean of order γt2∥∇f(xt)∥2\gamma_t^2\|\nabla f(x_t)\|^2γt2​∥∇f(xt​)∥2, which is not summable when the gradient is unbounded. Any argument must therefore extract a decrease of fff that dominates noise of the same order as the gradient itself, without a lower bound to anchor a supermartingale, and must do so along every sample path while the hypotheses are only conditional-expectation statements about non-integrable noise.

Formalization scope

  • Rn\mathbb R^nRn is EuclideanSpace ℝ (Fin n), ∇f\nabla f∇f is Mathlib's gradient f, fff is ContDiff ℝ 1, and the Lipschitz constant is L : ℝ≥0 with LipschitzWith L (gradient f) (the standing assumption (2.1), equivalent to the page's form).
  • The probability space is a Measure Ω with IsProbabilityMeasure, the σ\sigmaσ-fields are a Mathlib Filtration ℕ, and measurability in (a) is StronglyMeasurable[ℱ t]. The recursion and (4.1) hold on every sample path. Stepsizes are deterministic.
  • Conditional expectations of the noise. The page assumes no integrability of wtw_twt​, of ∥wt∥2\|w_t\|^2∥wt​∥2 or of f(xt)f(x_t)f(xt​), and none is added. Mathlib's condExp is 000 for non-integrable functions, so stating (4.2)–(4.3) with it would make them hold vacuously for any non-integrable noise; that encoding is ruled out. Instead (4.3) says that for every Ft\mathcal F_tFt​-measurable set SSS, ∫S∥wt∥2 dP≤∫SA(1+∥∇f(xt)∥2) dP\int_S\|w_t\|^2\,dP\le\int_S A(1+\|\nabla f(x_t)\|^2)\,dP∫S​∥wt​∥2dP≤∫S​A(1+∥∇f(xt​)∥2)dP (in [0,∞][0,\infty][0,∞]), and (4.2) says that ∫Swt dP=0\int_S w_t\,dP=0∫S​wt​dP=0 for every Ft\mathcal F_tFt​-measurable SSS on which wtw_twt​ is integrable. These are exactly the generalized conditional-expectation statements of the page.
  • In Lemma 2 the bound BBB is a constant, so Mathlib's condExp is used there, with integrability of ∥rt∥2\|r_t\|^2∥rt​∥2 stated explicitly; the page's hypothesis implies it. Lemma 2 is stated over any finite-dimensional real inner-product space, since the paper applies it to real and to vector-valued sequences.
  • ∑γt=∞\sum\gamma_t=\infty∑γt​=∞ is divergence of the partial sums; ∑γt2<∞\sum\gamma_t^2<\infty∑γt2​<∞ is Summable. Convergent random series (Lemma 2) are convergence of partial sums, not Summable, which would mean unconditional convergence. "lim sup⁡∥∇f(xt)∥≤δ\limsup\|\nabla f(x_t)\|\le\deltalimsup∥∇f(xt​)∥≤δ" is "for every δ′>δ\delta'>\deltaδ′>δ, eventually ∥∇f(xt)∥≤δ′\|\nabla f(x_t)\|\le\delta'∥∇f(xt​)∥≤δ′". Limit points are MapClusterPt.
  • In the §5 result the page's references to "section 4" and "(4.1)" are read as section 3 and condition (3.1), the indices k(t)k(t)k(t) run from t=0t=0t=0, x0x_0x0​ is deterministic, and the stepsizes are nonnegative as on the page.
  • Not stated: Lemma 3 (it needs the random interval construction of p. 636 as a definition), Lemmas 4–5 (steps that depend on the proof's own choice of ϵ\epsilonϵ), and the Remarks of §4.

Contributions welcome: a proof of Lemma 2 from Mathlib's martingale convergence theorems (Submartingale.exists_ae_tendsto_of_bdd), a proof of (4.4) from the descent lemma, a general bridge between the set-integral encoding of conditional expectations and Mathlib's condExp on localizing sets, and the interval construction behind Lemmas 3–6.

Selected references

  • D. P. Bertsekas and J. N. Tsitsiklis, Gradient Convergence in Gradient Methods with Errors, SIAM J. Optim. 10(3):627–642, 2000. https://doi.org/10.1137/S1052623497331063
  • B. T. Poljak and Y. Z. Tsypkin, Pseudogradient adaptation and training algorithms, Automat. Remote Control 12 (1973), 83–94.
  • H. J. Kushner and D. S. Clark, Stochastic Approximation Methods for Constrained and Unconstrained Systems, Springer, 1978.
  • H. J. Kushner and G. Yin, Stochastic Approximation Methods, Springer, 1996 (as cited in the paper).
  • A. Benveniste, M. Métivier and P. Priouret, Adaptive Algorithms and Stochastic Approximations, Springer, 1990.
  • D. P. Bertsekas and J. N. Tsitsiklis, Neuro-Dynamic Programming, Athena Scientific, 1996.
4 thms4 active usersReviewed
🏆Completed
Operations ResearchOptimization·Captain: mikedeng1

Air Travel Demand and Airline Seat Inventory Management II: The EMSR Protection Level for Two Nested Fare ClassesTextbook

Why airlines protect seats

An airline sells the seats of one flight in several fare classes at different prices, all drawn from one shared cabin. Discount fares are bought early, under advance-purchase restrictions, while most high-fare requests arrive close to departure. Accepting every early low-fare request fills the aircraft with cheap passengers and turns away late high-fare passengers; refusing too many leaves seats empty. Seat inventory control decides how many seats to keep away from the low fare. Peter Belobaba's 1987 MIT thesis (Flight Transportation Laboratory Report R87-7) introduced the expected marginal seat revenue (EMSR) model for this decision, and EMSR-type rules remain the basis of the booking-limit logic in airline revenue management systems.

Timeline. Littlewood (1972, AGIFORS Symposium Proceedings; reprinted 2005) proposed accepting a low-fare request as long as its fare is at least the high fare times the probability of selling all remaining seats to high-fare passengers. Analysts at Trans World Airlines (1973) and Richter at Lufthansa (1982) gave equivalent formulations for the dynamic case. Belobaba (1987, Ch. 5) restated the two-class rule as a static protection level for nested inventories and extended it heuristically to many classes. Brumelle and McGill (Operations Research 41, 1993) and Curry (Transportation Science 24, 1990) later proved optimality of nested protection levels for any number of classes under low-to-high arrivals, and showed that Belobaba's multi-class EMSR levels are not optimal for three or more classes. This mission concerns only the two-class result, which is correct.

Setting

A single flight leg has capacity C∈NC \in \mathbb NC∈N. Class 1 has fare f1f_1f1​, class 2 has fare f2f_2f2​, with 0≤f2≤f10 \le f_2 \le f_10≤f2​≤f1​. The numbers of requests for the two classes are random variables r1,r2r_1, r_2r1​,r2​ with values in N\mathbb NN, defined on a probability space (Ω,μ)(\Omega, \mu)(Ω,μ) and independent. There are no cancellations, no no-shows, and a refused request is lost.

The inventory is nested: a class-1 request is accepted as long as any seat is unsold. A protection level S∈{0,…,C}S \in \{0, \dots, C\}S∈{0,…,C} is the number of seats reserved for class 1; it sets the class-2 booking limit BL2=C−SBL_2 = C - SBL2​=C−S. All class-2 requests arrive before any class-1 request. Class 2 therefore books min⁡(r2,C−S)\min(r_2, C - S)min(r2​,C−S) seats and class 1 books min⁡(r1,C−min⁡(r2,C−S))\min(r_1, C - \min(r_2, C-S))min(r1​,C−min(r2​,C−S)), and the realised revenue is

RS=f2min⁡(r2,C−S)+f1min⁡(r1, C−min⁡(r2,C−S)).R_S = f_2 \min(r_2, C - S) + f_1 \min\bigl(r_1,\, C - \min(r_2, C - S)\bigr).RS​=f2​min(r2​,C−S)+f1​min(r1​,C−min(r2​,C−S)).

The expected revenue is Rˉ(S)=E[RS]\bar R(S) = \mathbb E[R_S]Rˉ(S)=E[RS​].

The tail probability of class 1 is Pˉ1(S)=P[r1≥S]\bar P_1(S) = P[r_1 \ge S]Pˉ1​(S)=P[r1​≥S], the probability of receiving SSS or more class-1 requests, and the expected marginal seat revenue of the SSS-th class-1 seat is

EMSR1(S)=f1⋅Pˉ1(S).\mathrm{EMSR}_1(S) = f_1 \cdot \bar P_1(S).EMSR1​(S)=f1​⋅Pˉ1​(S).

For a single class with SSS seats the expected revenue is f1 E[min⁡(r1,S)]f_1\,\mathbb E[\min(r_1, S)]f1​E[min(r1​,S)], and EMSR1(S)\mathrm{EMSR}_1(S)EMSR1​(S) is its increment from S−1S-1S−1 to SSS seats. The EMSR protection level S21S_2^1S21​ is the largest integer S∈{0,…,C}S \in \{0, \dots, C\}S∈{0,…,C} with

EMSR1(S)≥f2.\mathrm{EMSR}_1(S) \ge f_2 .EMSR1​(S)≥f2​.

In Lean these objects are nestedRevenue, expectedNestedRevenue, tailProb, classRevenue, emsr and emsrProtectionLevel in SeatInventory.Nested.

Formalization targets

Goal: Eqs. (5.15)–(5.16), optimality of the EMSR protection level

Rˉ(S)≤Rˉ(S21)for all S∈{0,…,C}.\bar R(S) \le \bar R(S_2^1) \qquad \text{for all } S \in \{0, \dots, C\}.Rˉ(S)≤Rˉ(S21​)for all S∈{0,…,C}.

The goal fixes no distribution: it holds for every pair of independent N\mathbb NN-valued demands, and S21S_2^1S21​ depends only on f2/f1f_2/f_1f2​/f1​ and the law of r1r_1r1​.

Milestones

  1. Eq. (5.11). f1E[min⁡(r1,S)]−f1E[min⁡(r1,S−1)]=f1P[r1≥S]f_1\mathbb E[\min(r_1,S)] - f_1\mathbb E[\min(r_1,S-1)] = f_1 P[r_1 \ge S]f1​E[min(r1​,S)]−f1​E[min(r1​,S−1)]=f1​P[r1​≥S] for S≥1S \ge 1S≥1.
  2. Eqs. (6.1)–(6.2). Pˉ1\bar P_1Pˉ1​ and, for f1≥0f_1 \ge 0f1​≥0, EMSR1\mathrm{EMSR}_1EMSR1​ are non-increasing in SSS.
  3. Eq. (4.8), Littlewood's rule, already on the platform as RevenueManagement.littlewood_marginal_value (Talluri and van Ryzin's Eq. (2.1), proved).
  4. Sect. 5.2, p. 112. Rˉ(S)≤Rˉ(S21)\bar R(S) \le \bar R(S_2^1)Rˉ(S)≤Rˉ(S21​) for S21≤S≤CS_2^1 \le S \le CS21​≤S≤C: a smaller booking limit for class 2 cannot raise expected revenue.
  5. Sect. 5.2, p. 114. With the same class-2 limit C−SC - SC−S, the expected nested revenue is at least the expected revenue of two distinct inventories with SSS and C−SC - SC−S seats, strictly if f1>0f_1 > 0f1​>0 and P[r2<C−S, r1>S]>0P[r_2 < C - S,\ r_1 > S] > 0P[r2​<C−S, r1​>S]>0.

Significance

The two-class result says that, for a static booking limit set once before sales open and low-fare demand arriving first, the airline needs only the high-fare demand distribution and the fare ratio to set the optimal limit; the low-fare forecast is irrelevant. This is the rule that the thesis then applies class by class in multi-class nested systems, and it is the base case against which the later exact multi-class theory (Brumelle–McGill, Curry) is checked. Milestone 5 makes precise why nested inventories dominate the distinct-inventory allocation of the thesis's Sect. 5.1 with the same class-2 limit.

The result is classical and proved, in the sense that the optimality of a two-class threshold policy follows from Littlewood's argument and from the dynamic-programming treatment in Talluri and van Ryzin's The Theory and Practice of Revenue Management (2004, Ch. 2). On Prove2Me, Littlewood's marginal rule and the dynamic-programming optimality of nested protection levels (RevenueManagement.static_optimal_controls) are formalized, but in Bellman form: there the protection level is defined through the value function of a dynamic program. What is not formalized is the statement in Belobaba's form, where the protection level is the explicit threshold of f1P[r1≥S]f_1 P[r_1 \ge S]f1​P[r1​≥S] against f2f_2f2​ and the objective is the explicit expected revenue of a booking limit. Connecting the two forms, and the comparison with distinct inventories, is the work of this mission.

Difficulty

The expected revenue couples the two demands through the capacity left by class 2, so Rˉ\bar RRˉ is not a sum of single-class revenues and is not separately concave in an obvious way. The step that requires care is the increment Rˉ(S)−Rˉ(S−1)\bar R(S) - \bar R(S-1)Rˉ(S)−Rˉ(S−1): it is not EMSR1(S)−f2\mathrm{EMSR}_1(S) - f_2EMSR1​(S)−f2​, as the thesis's sentence after the milestone on p. 112 suggests, because the extra protected seat matters only on the event that class 2 would have reached its limit. Independence of r1r_1r1​ and r2r_2r2​ is what makes that event's probability factor out; without independence the threshold rule is not optimal. The discrete reading matters too: with P[r1>S]P[r_1 > S]P[r1​>S] in place of P[r1≥S]P[r_1 \ge S]P[r1​≥S] the rule is off by one seat and the claim fails.

Formalization scope

Conventions the Lean statements commit to:

  • Demands are N\mathbb NN-valued measurable random variables r₁ r₂ : Ω → ℕ on a probability space μ; the goal and milestone 4 assume IndepFun r₁ r₂ μ. The thesis writes continuous densities (Eqs. (5.1)–(5.5)) but requires integer seat counts; the discrete model is used throughout.
  • Pˉ1(S)=P[r1≥S]\bar P_1(S) = P[r_1 \ge S]Pˉ1​(S)=P[r1​≥S], as in Eq. (6.2) and the prose of Eq. (5.11), not P[r1>S]P[r_1 > S]P[r1​>S] as in Eq. (5.2).
  • The EMSR protection level is the largest S∈{0,…,C}S \in \{0,\dots,C\}S∈{0,…,C} with f1P[r1≥S]≥f2f_1 P[r_1 \ge S] \ge f_2f1​P[r1​≥S]≥f2​ (Eq. (5.15)); Eq. (5.16)'s equality is the continuous idealisation and is not stated.
  • Booking order: all class-2 requests precede all class-1 requests (pp. 108, 112). This order is built into the revenue formula, not assumed separately.
  • Fares satisfy 0≤f2≤f10 \le f_2 \le f_10≤f2​≤f1​; the thesis has f1>f2f_1 > f_2f1​>f2​, and the statements also cover equality.
  • Expectations are Bochner integrals of bounded revenues, probabilities are μ.real; seat counts use truncated subtraction only where S≤CS \le CS≤C.

A trivializing formalization is ruled out: S21S_2^1S21​ is defined by the threshold of (5.15), never as an argmax of expected revenue, and the expected revenue is computed from the realised revenue of the booking process, not postulated as a sum of marginal terms.

The multi-class EMSR levels of Eqs. (5.19)–(5.29) and the dynamic revision of Eqs. (5.31)–(5.32) are out of scope. Proofs need the discrete expectation identity E[min⁡(r,S)]−E[min⁡(r,S−1)]=P[r≥S]\mathbb E[\min(r,S)] - \mathbb E[\min(r,S-1)] = P[r \ge S]E[min(r,S)]−E[min(r,S−1)]=P[r≥S] and expectation of products of independent bounded functions, both in Mathlib's reach and reusable for other single-leg revenue models. Proofs of any milestone, and a proof of the goal from milestones 1, 2 and 4 plus the matching lower-half argument, are welcome.

Selected references

  • P. P. Belobaba, Air Travel Demand and Airline Seat Inventory Management, PhD thesis, MIT, Flight Transportation Laboratory Report R87-7, 1987 (no DOI).
  • K. Littlewood, Forecasting and control of passenger bookings, AGIFORS Symposium Proceedings 12, 1972; reprinted in Journal of Revenue and Pricing Management 4, 2005. https://doi.org/10.1057/palgrave.rpm.5170134
  • S. L. Brumelle and J. I. McGill, Airline seat allocation with multiple nested fare classes, Operations Research 41, 1993. https://doi.org/10.1287/opre.41.1.127
  • R. E. Curry, Optimal airline seat allocation with fare classes nested by origins and destinations, Transportation Science 24, 1990. https://doi.org/10.1287/trsc.24.3.193
  • K. T. Talluri and G. J. van Ryzin, The Theory and Practice of Revenue Management, Springer, 2004. https://doi.org/10.1007/b139000
8 thms4 active usersReviewed
🏆Completed
Linear OptimizationOperations ResearchOptimization·Captain: mikedeng1

Air Travel Demand and Airline Seat Inventory Management I: Marginal Seat Allocation Among Distinct Fare ClassesTextbook

Why airlines allocate seats by fare class

An airline sells the seats of one flight leg at several prices. Low fares fill seats that would otherwise fly empty; high fares are bought by passengers who book late and cannot be predicted exactly. Seat inventory control decides how many seats each fare class may sell. Peter Belobaba's 1987 MIT dissertation (MIT Flight Transportation Laboratory Report R87-7) gave the probabilistic treatment of this problem that became the expected marginal seat revenue (EMSR) method, which was in use across the airline industry for decades.

This mission formalizes the first, simplest model of the thesis: distinct (non-nested) fare-class inventories on a single leg, where a seat assigned to a class may be sold only in that class or not at all. The thesis surveys this model in Sect. 4.2 (pp. 84–94), where an integer-programming formulation from McDonnell-Douglas and its solution by ranking marginal values are described (p. 90), and develops it probabilistically in Sect. 5.1 (pp. 102–107).

Setting

A leg has capacity nnn seats (the thesis also writes CCC). There are finitely many fare classes iii. Class iii has an average fare fi≥0f_i \ge 0fi​≥0 and receives a random number of requests ri∈{0,1,2,… }r_i \in \{0,1,2,\dots\}ri​∈{0,1,2,…}, with law pip_ipi​. The seats are split into allocations Si∈NS_i \in \mathbb{N}Si​∈N, one per class.

With SSS seats, a class books requests until its seats run out, so its bookings and spill (refused requests) are

b=min⁡(r,S),l=(r−S)+(Eq. (5.3)).b = \min(r, S), \qquad l = (r - S)^+ \qquad \text{(Eq. (5.3))}.b=min(r,S),l=(r−S)+(Eq. (5.3)).

The expected revenue of class iii is Rˉi(Si)=fi⋅bˉi(Si)\bar R_i(S_i) = f_i \cdot \bar b_i(S_i)Rˉi​(Si​)=fi​⋅bˉi​(Si​) with bˉi(Si)=E[min⁡(ri,Si)]\bar b_i(S_i) = E[\min(r_i,S_i)]bˉi​(Si​)=E[min(ri​,Si​)], and the leg's expected revenue is Rˉ=∑iRˉi(Si)\bar R = \sum_i \bar R_i(S_i)Rˉ=∑i​Rˉi​(Si​) (Eq. (5.9)). Write

Pˉi(S)=P[ri≥S],EMSRi(S)=fi⋅Pˉi(S)(Eqs. (5.11), (6.1), (6.2)),\bar P_i(S) = P[r_i \ge S], \qquad \mathrm{EMSR}_i(S) = f_i \cdot \bar P_i(S) \qquad \text{(Eqs. (5.11), (6.1), (6.2))},Pˉi​(S)=P[ri​≥S],EMSRi​(S)=fi​⋅Pˉi​(S)(Eqs. (5.11), (6.1), (6.2)),

the expected marginal seat revenue of the SSS-th seat of class iii. The value of the kkk-th seat of class iii in the integer program of p. 90 is mi(k)=EMSRi(k)m_i(k) = \mathrm{EMSR}_i(k)mi​(k)=EMSRi​(k), k=1,…,nk = 1, \dots, nk=1,…,n.

Formalization targets

Goal: the nnn largest marginal values give the optimal booking limits (p. 90)

Let TTT be any set of nnn pairs (i,k)(i,k)(i,k), 1≤k≤n1 \le k \le n1≤k≤n, such that every mi(k)m_i(k)mi​(k) with (i,k)∈T(i,k) \in T(i,k)∈T is at least every mj(l)m_j(l)mj​(l) with (j,l)∉T(j,l) \notin T(j,l)∈/T, and let SiT=#{k:(i,k)∈T}S^T_i = \#\{k : (i,k) \in T\}SiT​=#{k:(i,k)∈T}. Then ∑iSiT=n\sum_i S^T_i = n∑i​SiT​=n and

∑ifi E[min⁡(ri,Si)]  ≤  ∑ifi E[min⁡(ri,SiT)]for every S with ∑iSi≤n.\sum_i f_i\, E[\min(r_i, S_i)] \;\le\; \sum_i f_i\, E[\min(r_i, S^T_i)] \qquad \text{for every } S \text{ with } \textstyle\sum_i S_i \le n .i∑​fi​E[min(ri​,Si​)]≤i∑​fi​E[min(ri​,SiT​)]for every S with ∑i​Si​≤n.

The goal fixes no distribution, number of classes or fare ordering, and it holds for every tie-breaking among equal marginal values.

Milestones

  1. Eq. (5.6): bˉi(S)+lˉi(S)=rˉi\bar b_i(S) + \bar l_i(S) = \bar r_ibˉi​(S)+lˉi​(S)=rˉi​ for requests of finite mean.
  2. Eq. (5.11): Rˉi(S)−Rˉi(S−1)=fi⋅P[ri≥S]\bar R_i(S) - \bar R_i(S-1) = f_i \cdot P[r_i \ge S]Rˉi​(S)−Rˉi​(S−1)=fi​⋅P[ri​≥S] for S≥1S \ge 1S≥1.
  3. Eqs. (6.1)–(6.2): Pˉi\bar P_iPˉi​ and EMSRi\mathrm{EMSR}_iEMSRi​ are non-increasing in SSS.
  4. Eq. (4.5): the 0–1 vector equal to 111 on a set of nnn largest mi(k)m_i(k)mi​(k) is an optimal solution of the linear program max⁡∑i,kXikmi(k)\max \sum_{i,k} X_{ik} m_i(k)max∑i,k​Xik​mi​(k) subject to ∑Xik≤n\sum X_{ik} \le n∑Xik​≤n, 0≤Xik≤10 \le X_{ik} \le 10≤Xik​≤1.
  5. Eq. (5.13), discrete form: an allocation of exactly CCC seats maximises Rˉ\bar RRˉ among such allocations if and only if some λ\lambdaλ satisfies EMSRi(Si)≥λ\mathrm{EMSR}_i(S_i) \ge \lambdaEMSRi​(Si​)≥λ whenever Si≥1S_i \ge 1Si​≥1 and EMSRi(Si+1)≤λ\mathrm{EMSR}_i(S_i + 1) \le \lambdaEMSRi​(Si​+1)≤λ, for all iii.

Significance

The goal is the reason distinct-inventory allocation is computationally easy: a revenue-maximising allocation is obtained by sorting n×(number of classes)n \times (\text{number of classes})n×(number of classes) numbers, with no search over allocations. The same marginal-value principle underlies the EMSR rules for nested classes in the rest of the thesis, and the identity (5.11) is the link between an expected-revenue function and its marginal seat values used throughout revenue management. Milestone 5 is the integer form of the Lagrangian condition of Eq. (5.13); the thesis states it only for a continuous relaxation, and its equality form is generally unattainable with integer seats.

These results are classical and their proofs are elementary; to our knowledge none of them has been machine-checked. The mission produces a reusable formal model of a single-leg, distinct-inventory allocation problem with integer demand (bookings, spill, revenue and marginal values of a PMF ℕ), and checked statements of the marginal-allocation principle for it.

Difficulty

The thesis argues with continuous densities and derivatives, setting ∂Rˉ/∂Si\partial \bar R / \partial S_i∂Rˉ/∂Si​ equal across classes. That argument does not transfer to integer seats: the derivative of a step-shaped expected-revenue function does not exist, equality of marginal values across classes generally fails at every integer allocation, and the tail probability must be P[r≥S]P[r \ge S]P[r≥S] rather than the P[r>S]P[r > S]P[r>S] of Eq. (5.2) for the marginal identity to hold. The integer statements need their own exchange argument. A second subtlety is ties: "the nnn largest values" is not unique, and the goal must hold for every admissible choice, including choices in which a class's selected seat numbers are not an initial segment {1,…,Si}\{1, \dots, S_i\}{1,…,Si​}.

Formalization scope

All declarations live in the namespace SeatInventory.Distinct. Conventions:

  • Integer demand. The law of class iii's requests is d i : PMF ℕ; expectations are series over N\mathbb{N}N. The thesis's continuous densities are replaced by this discrete model, which the thesis itself requires for seat allocations (p. 103).
  • Tail convention. Pˉ(S)=P[r≥S]\bar P(S) = P[r \ge S]Pˉ(S)=P[r≥S], as in Eq. (6.2) and the prose of Eq. (5.11) ("the probability of selling SiS_iSi​ or more seats"), not the P[r>S]P[r > S]P[r>S] of Eq. (5.2).
  • Seat numbers start at 1, and the pairs (i,k)(i,k)(i,k) range over k∈{1,…,n}k \in \{1, \dots, n\}k∈{1,…,n}, as the 600 variables of a 150-seat, four-class problem on p. 90 indicate.
  • Nonnegative fares fi≥0f_i \ge 0fi​≥0 are assumed in every statement that needs them; with a negative fare the capacity constraint ∑Si≤n\sum S_i \le n∑Si​≤n would not bind and the claims fail.
  • Finite mean of the requests is assumed explicitly for Eq. (5.6); the thesis assumes it silently. Expected bookings are bounded and need no assumption.
  • Capacity. The goal and the LP compare against allocations with ∑iSi≤n\sum_i S_i \le n∑i​Si​≤n (the LP's constraint); milestone 5 compares allocations of exactly CCC seats (Eq. (5.8)).
  • No independence assumption. Expected revenue of distinct inventories depends only on each class's marginal law, so the statements take one law per class.
  • "Decreasing" is non-increasing. The thesis's justification in Sect. 6.1.1 gives only monotonicity; strict decrease fails for bounded demand.
  • LP integrality. "The solution will be integer" is stated as: the indicator of every set of nnn largest values is optimal. With ties the LP also has fractional optima.

The expected revenue in the goal is computed from the booking rule min⁡(ri,Si)\min(r_i, S_i)min(ri​,Si​); it is not defined as a sum of marginal values, and the optimal allocation is not defined as an argmax of Rˉ\bar RRˉ. Either shortcut would make the goal a tautology and is ruled out.

Needed infrastructure: tail sums of a PMF ℕ, telescoping of E[min⁡(r,S)]E[\min(r, S)]E[min(r,S)], and a finite exchange argument for sums of the nnn largest values of a function on a finite set; the last two are reusable for any separable concave resource-allocation problem. Contributions of proofs of any milestone, and of a verified sorting routine that produces a set of nnn largest values, are welcome.

Selected references

  • P. P. Belobaba, Air Travel Demand and Airline Seat Inventory Management, PhD thesis, MIT Flight Transportation Laboratory Report R87-7, 1987 (no DOI).
  • P. P. Belobaba, Airline yield management: an overview of seat inventory control, Transportation Science 21(2), 63–73, 1987. https://doi.org/10.1287/trsc.21.2.63
  • P. P. Belobaba, Application of a probabilistic decision model to airline seat inventory control, Operations Research 37(2), 183–197, 1989. https://doi.org/10.1287/opre.37.2.183
  • K. Littlewood, Forecasting and control of passenger bookings, AGIFORS Symposium Proceedings 12, 1972; reprinted in Journal of Revenue and Pricing Management 4(2), 111–123, 2005. https://doi.org/10.1057/palgrave.rpm.5170134
8 thms4 active usersReviewed
🏆Completed
Dynamic ProgrammingOperations ResearchStochastic Systems·Captain: mikedeng1

Stochastic Dynamic Programming and the Control of Queueing Systems III: Approximating Sequences for the Discounted Cost CriterionTextbook

Motivation

Optimal control of queueing systems leads to Markov decision problems whose state space is countably infinite (buffer contents, numbers of customers) and whose costs are unbounded (holding costs grow with the queue). Such a problem cannot be solved on a computer as it stands. The standard remedy is to truncate: solve a finite problem on the states {0,1,…,N}\{0,1,\dots,N\}{0,1,…,N} and hope that its value and its optimal policy approximate those of the original problem as NNN grows. Linn Sennott's approximating sequence method (ASM) makes this hope precise. For the expected discounted cost criterion, Sections 4.6–4.7 of Sennott's book (Stochastic Dynamic Programming and the Control of Queueing Systems, Wiley, 1999) identify a single condition, Assumption DC(α\alphaα), that is necessary and sufficient for convergence of the truncated values, and give checkable sufficient conditions for it.

The method matters because naive truncation can fail. The book's Example 4.6.1 has a chain whose value at state 000 is finite, yet a natural truncation produces values VαN(0)≥αN2/((1−α)[N(1−α)+α])→∞V^N_\alpha(0)\ge \alpha N^2/((1-\alpha)[N(1-\alpha)+\alpha])\to\inftyVαN​(0)≥αN2/((1−α)[N(1−α)+α])→∞. How the probability that would leave the truncated set is redistributed decides whether the computation is meaningful.

Earlier truncation schemes (Fox 1971; White 1980, 1982; Hernández-Lerma 1986; Cavazos-Cadena 1986; Whitt 1978–79; see the bibliographic notes on p. 81 of the book and Puterman 1994) require bounded rewards or pass directly to an algorithm. The ASM instead produces a sequence of finite Markov decision chains that can be studied in their own right; the material of Sections 4.6–4.7 is presented in the book as new.

Setting

A Markov decision chain (MDC) Δ\DeltaΔ has a countable state space SSS; for each state iii a finite nonempty action set AiA_iAi​; nonnegative finite costs C(i,a)C(i,a)C(i,a); and transition probabilities Pij(a)P_{ij}(a)Pij​(a) with ∑jPij(a)=1\sum_j P_{ij}(a)=1∑j​Pij​(a)=1. A policy θ\thetaθ chooses the action at time ttt at random from a distribution θ(⋅∣ht)\theta(\cdot\mid h_t)θ(⋅∣ht​) on AitA_{i_t}Ait​​ that may depend on the entire history ht=(i0,a0,…,it−1,at−1,it)h_t=(i_0,a_0,\dots,i_{t-1},a_{t-1},i_t)ht​=(i0​,a0​,…,it−1​,at−1​,it​). A stationary policy fff always chooses f(i)∈Aif(i)\in A_if(i)∈Ai​ in state iii. Fix a discount factor α∈(0,1)\alpha\in(0,1)α∈(0,1). The discounted cost of θ\thetaθ and the discounted value function are

Vθ,α(i)=∑t≥0αtEθ[C(Xt,At)∣X0=i],Vα(i)=inf⁡θVθ,α(i),V_{\theta,\alpha}(i)=\sum_{t\ge0}\alpha^tE_\theta[C(X_t,A_t)\mid X_0=i],\qquad V_\alpha(i)=\inf_\theta V_{\theta,\alpha}(i),Vθ,α​(i)=t≥0∑​αtEθ​[C(Xt​,At​)∣X0​=i],Vα​(i)=θinf​Vθ,α​(i),

both in [0,∞][0,\infty][0,∞], the infimum over all policies. A policy is discount optimal if Vθ,α=VαV_{\theta,\alpha}=V_\alphaVθ,α​=Vα​.

An approximating sequence (ΔN)N≥N0(\Delta_N)_{N\ge N_0}(ΔN​)N≥N0​​ consists of finite nonempty sets SNS_NSN​ increasing to SSS and, for i∈SNi\in S_Ni∈SN​ and a∈Aia\in A_ia∈Ai​, probability distributions Pij(a;N)P_{ij}(a;N)Pij​(a;N) on SNS_NSN​ with Pij(a;N)→Pij(a)P_{ij}(a;N)\to P_{ij}(a)Pij​(a;N)→Pij​(a) as N→∞N\to\inftyN→∞. The finite MDC ΔN\Delta_NΔN​ has state space SNS_NSN​ and the same actions and costs; VαNV^N_\alphaVαN​ is its value function and fαNf^N_\alphafαN​ a stationary policy attaining the minimum in its discount optimality equation

VαN(i)=min⁡a∈Ai{C(i,a)+α∑j∈SNPij(a;N)VαN(j)},i∈SN.V^N_\alpha(i)=\min_{a\in A_i}\Big\{C(i,a)+\alpha\sum_{j\in S_N}P_{ij}(a;N)V^N_\alpha(j)\Big\},\qquad i\in S_N.VαN​(i)=a∈Ai​min​{C(i,a)+αj∈SN​∑​Pij​(a;N)VαN​(j)},i∈SN​.

An augmentation type approximating sequence (ATAS) keeps the original probabilities inside SNS_NSN​ and redistributes the excess probability Pir(a)P_{ir}(a)Pir​(a), r∉SNr\notin S_Nr∈/SN​, according to augmentation distributions qj(i,a,r,N)q_j(i,a,r,N)qj​(i,a,r,N) on SNS_NSN​: Pij(a;N)=Pij(a)+∑r∉SNPir(a)qj(i,a,r,N)P_{ij}(a;N)=P_{ij}(a)+\sum_{r\notin S_N}P_{ir}(a)q_j(i,a,r,N)Pij​(a;N)=Pij​(a)+∑r∈/SN​​Pir​(a)qj​(i,a,r,N).

Assumption DC(α\alphaα): for every i∈Si\in Si∈S, Wα(i):=lim sup⁡NVαN(i)<∞W_\alpha(i):=\limsup_{N}V^N_\alpha(i)<\inftyWα​(i):=limsupN​VαN​(i)<∞ and Wα(i)≤Vα(i)W_\alpha(i)\le V_\alpha(i)Wα​(i)≤Vα​(i).

Formalization targets

Goal: Theorem 4.6.3

The following are equivalent:

(i) lim⁡N→∞VαN(i)=Vα(i)<∞  (i∈S);(ii) Assumption DC(α).\text{(i)}\ \lim_{N\to\infty}V^N_\alpha(i)=V_\alpha(i)<\infty\ \ (i\in S);\qquad \text{(ii)}\ \text{Assumption DC}(\alpha).(i) N→∞lim​VαN​(i)=Vα​(i)<∞  (i∈S);(ii) Assumption DC(α).

Under either, every limit point of (fαN)N≥N0(f^N_\alpha)_{N\ge N_0}(fαN​)N≥N0​​ (a stationary fff with fNr(i)=f(i)f^{N_r}(i)=f(i)fNr​(i)=f(i) eventually along a subsequence, for each iii) is discount optimal for Δ\DeltaΔ.

Milestones

  • Lemma 4.6.2: lim inf⁡NVαN≥Vα\liminf_N V^N_\alpha\ge V_\alphaliminfN​VαN​≥Vα​ for every approximating sequence.
  • Proposition 4.7.1: bounded costs imply DC(α\alphaα).
  • Lemma 4.7.2: taboo probabilities of avoiding S−SNS-S_NS−SN​ converge to the ttt-step transition probabilities.
  • Lemma 4.7.3: for the first passage time Ti(N)T_i(N)Ti​(N) out of SNS_NSN​ under a stationary policy, E[αTi(N)]→0E[\alpha^{T_i(N)}]\to0E[αTi​(N)]→0.
  • Proposition 4.7.4: if Vα<∞V_\alpha<\inftyVα​<∞ and the ATAS sends excess probability to a finite set, DC(α\alphaα) holds.
  • Corollary 4.7.5: the case of a single distinguished state zzz, with the relative form of the optimality equation for ΔN\Delta_NΔN​.
  • Proposition 4.7.6: if Vα<∞V_\alpha<\inftyVα​<∞ and the augmentation distributions satisfy ∑j∈SNqj(i,a,r,N)vα,n(j)≤vα,n(r)\sum_{j\in S_N}q_j(i,a,r,N)v_{\alpha,n}(j)\le v_{\alpha,n}(r)∑j∈SN​​qj​(i,a,r,N)vα,n​(j)≤vα,n​(r) for all n≥0n\ge0n≥0, then VαN≤VαV^N_\alpha\le V_\alphaVαN​≤Vα​ on SNS_NSN​.

Significance

Theorem 4.6.3 turns the question "does truncation work?" into the verification of one inequality between a lim sup and the true value, and it delivers both the value and an optimal stationary policy from finite computations. Propositions 4.7.4–4.7.6 give conditions that hold in the queueing models of the book with unbounded holding costs, and Corollary 4.7.5 supplies the computational form used for the inventory model of Chapter 5. The discounted theory is also the stepping stone to the average cost ASM of Chapter 8, which is built on discounted approximations.

All results are proved in the book. None of them is formalized: the platform has no statement about approximating sequences or state truncation of countable-state MDPs, and Mathlib has no Markov decision processes. The mission produces machine-checked versions of the convergence theorem and its sufficient conditions, for general history-dependent randomized policies and [0,∞][0,\infty][0,∞]-valued costs.

Difficulty

The value functions are infima over uncountably many history-dependent policies and may be infinite, so no contraction argument applies: costs are unbounded and VαV_\alphaVα​ is only the minimal nonnegative solution of its optimality equation. Passing to the limit in NNN inside ∑j∈SNPij(a;N)VαN(j)\sum_{j\in S_N}P_{ij}(a;N)V^N_\alpha(j)∑j∈SN​​Pij​(a;N)VαN​(j) is an interchange of limit and infinite sum under a moving probability measure, with no dominating function in general; Example 4.6.1 shows that the interchange genuinely fails. The upper bound of Proposition 4.7.4 requires comparing ΔN\Delta_NΔN​ with Δ\DeltaΔ along a coupled first passage out of SNS_NSN​, which needs the taboo-probability estimates of Lemmas 4.7.2–4.7.3. The obvious idea of bounding VαNV^N_\alphaVαN​ by sup⁡C/(1−α)\sup C/(1-\alpha)supC/(1−α) works only for bounded costs (Proposition 4.7.1).

Formalization scope

The state type S is countable ([Countable S]); actions live in a type Act, with a finite nonempty Finset of admissible actions per state. Costs are ℝ≥0, transition probabilities and all value functions are ℝ≥0∞, so infima over policies are lattice infima and +∞+\infty+∞ is a legitimate value. A general policy is a function of the history, encoded as the list of past state–action pairs (most recent first) and the current state; the expected cost at time ttt is the [0,∞][0,\infty][0,∞]-valued sum over histories. VαV_\alphaVα​ is the infimum over all such policies; a stationary policy enters as the policy putting mass one on f(i)f(i)f(i). The discount factor is α : ℝ≥0 with 0<α<10<\alpha<10<α<1 (the chapter's standing assumption). ΔN\Delta_NΔN​ is an MDC on the subtype SNS_NSN​; VαN(i)V^N_\alpha(i)VαN​(i) is extended by 000 when N<N0N<N_0N<N0​ or i∉SNi\notin S_Ni∈/SN​, a convention that affects finitely many NNN for each fixed iii and hence no limit in NNN. Limits, lim sups and lim infs are along Filter.atTop in ℝ≥0∞. Taboo probabilities and the first passage quantity E[αT]=∑n≥1αnP(T=n)E[\alpha^{T}]=\sum_{n\ge1}\alpha^nP(T=n)E[αT]=∑n≥1​αnP(T=n) (so α∞=0\alpha^\infty=0α∞=0) are defined combinatorially from the transition probabilities.

A trivializing formalization is ruled out: VαV_\alphaVα​ is not an infimum over stationary policies only (which would make optimality of limit points close to definitional), DC(α\alphaα) keeps both of its conditions, and statement (i) of the goal includes finiteness of VαV_\alphaVα​.

A complete development needs the minimality of VαV_\alphaVα​ among nonnegative solutions of the discount optimality equation (Theorem 4.1.4, chunk II of this series), Fatou-type lemmas for sums against converging distributions (Appendix A, chunk XI), and compactness of stationary policies (Proposition B.5). Contributions of these as reusable lemmas about countable-state MDCs are welcome.

Selected references

  • L. I. Sennott, Stochastic Dynamic Programming and the Control of Queueing Systems, Wiley, 1999, Sections 4.6–4.7, pp. 73–81. https://doi.org/10.1002/9780470317037
  • M. L. Puterman, Markov Decision Processes: Discrete Stochastic Dynamic Programming, Wiley, 1994. https://doi.org/10.1002/9780470316887
11 thms4 active usersReviewed
🏆Completed
Convex OptimizationOperations ResearchOptimization·Captain: mikedeng1

Numerical Techniques for Stochastic Optimization II: Logarithmic Concavity of Probabilistic ConstraintsTextbook

Motivation

Many engineering and economic planning problems must meet random requirements with a prescribed reliability: a reservoir must satisfy demand with probability at least 0.950.950.95, a power system must cover load except on rare days, an inventory must avoid shortage with high probability. Probabilistic constrained programming (also called chance-constrained programming) models this by requiring that a system of random inequalities hold jointly with probability at least ppp.

The first obstacle to solving such problems is structural. The probability that random constraints are satisfied is, in general, neither concave nor convex in the decision, so the feasible set need not be convex and local search can stall. Prékopa's theory of logarithmically concave measures (1971–1973) removed that obstacle for a large class of distributions, and it is the basis of the numerical methods of Chapter 5 of Ermoliev and Wets, Numerical Techniques for Stochastic Optimization (Springer 1988). This mission formalizes the structural theorems of that chapter.

Timeline:

  • 1959: Charnes and Cooper, individual chance constraints.
  • 1971: Prékopa, logarithmic concave measures with application to stochastic programming (Acta Sci. Math. Szeged 32).
  • 1973: Prékopa, logarithmic concave measures and functions (Acta Sci. Math. Szeged 34), containing the marginal theorem: marginals of log-concave functions are log-concave.
  • 1970s–1980s: nonlinear programming methods for (5.1) (SUMT with logarithmic penalty, supporting hyperplanes, reduced gradients) combined with Monte Carlo evaluation of h0h_0h0​; the chapter surveys them.

Setting

Let ξ\xiξ be a random vector in Rq\mathbb R^qRq on a probability space (Ω,F,P)(\Omega, \mathcal F, P)(Ω,F,P) and let g1,…,gr:Rn×Rq→Rg_1, \dots, g_r : \mathbb R^n \times \mathbb R^q \to \mathbb Rg1​,…,gr​:Rn×Rq→R. The chapter studies problem (5.1):

min⁡h(x)s.t.h0(x)=P(g1(x,ξ)≥0,…,gr(x,ξ)≥0)≥p,h1(x)≥p1,…,hm(x)≥pm.\min h(x) \quad \text{s.t.} \quad h_0(x) = P\bigl(g_1(x,\xi) \ge 0, \dots, g_r(x,\xi) \ge 0\bigr) \ge p, \quad h_1(x) \ge p_1, \dots, h_m(x) \ge p_m .minh(x)s.t.h0​(x)=P(g1​(x,ξ)≥0,…,gr​(x,ξ)≥0)≥p,h1​(x)≥p1​,…,hm​(x)≥pm​.

The function h0h_0h0​ is the probability function (chanceProb in Lean). In the special case gi(x,y)=Tix−yig_i(x,y) = T_i x - y_igi​(x,y)=Ti​x−yi​ it equals F(Tx)F(Tx)F(Tx), where FFF is the joint distribution function of ξ\xiξ.

A function f≥0f \ge 0f≥0 is logarithmically concave on a convex set SSS if

f(λu+(1−λ)v)≥f(u)λf(v)1−λ,u,v∈S, 0<λ<1.f(\lambda u + (1-\lambda) v) \ge f(u)^{\lambda} f(v)^{1-\lambda}, \qquad u, v \in S,\ 0 < \lambda < 1 .f(λu+(1−λ)v)≥f(u)λf(v)1−λ,u,v∈S, 0<λ<1.

Where f>0f > 0f>0 this is concavity of log⁡f\log flogf; the power form also makes sense where f=0f = 0f=0. Nondegenerate normal densities, uniform densities on convex bodies and exponential densities are log-concave.

Section 5.7 introduces the polynomial distribution (5.19) on the cube 0<zj≤10 < z_j \le 10<zj​≤1:

F(z1,…,zn)=1∑i=1Nciz1αi1⋯znαin,ci>0, αij≤0, ∑jαij<0F(z_1, \dots, z_n) = \frac{1}{\sum_{i=1}^{N} c_i z_1^{\alpha_{i1}} \cdots z_n^{\alpha_{in}}}, \qquad c_i > 0,\ \alpha_{ij} \le 0,\ \textstyle\sum_j \alpha_{ij} < 0F(z1​,…,zn​)=∑i=1N​ci​z1αi1​​⋯znαin​​1​,ci​>0, αij​≤0, ∑j​αij​<0

(polyDistF in Lean).

Formalization targets

Goal: Theorem 5.1

If g1,…,grg_1, \dots, g_rg1​,…,gr​ are jointly concave on Rn+q\mathbb R^{n+q}Rn+q and ξ\xiξ has a log-concave density fff on Rq\mathbb R^qRq, then

h0 is logarithmically concave on Rn.h_0 \text{ is logarithmically concave on } \mathbb R^n .h0​ is logarithmically concave on Rn.

Its immediate consequence is that the feasible set {x:h0(x)≥p}\{x : h_0(x) \ge p\}{x:h0​(x)≥p} is convex for every ppp.

Milestones

  1. Theorem 5.2.1: if hhh is log-concave on the convex set H={h≥p}H = \{h \ge p\}H={h≥p}, 0<p<10 < p < 10<p<1, then h−ph - ph−p is log-concave on HHH. This makes the logarithmic penalty function (5.5) of the SUMT method convex.
  2. Theorem 5.2.2: under the standing assumptions of §5.2, every interior point zzz of the feasible set of (5.1) satisfies hi(z)>pih_i(z) > p_ihi​(z)>pi​, i=0,…,mi = 0, \dots, mi=0,…,m.
  3. Theorem 5.7.1 (proved content, (5.21)): for n=2n = 2n=2 and oppositely ordered exponents, ∂2F/∂z1∂z2≥0\partial^2 F / \partial z_1 \partial z_2 \ge 0∂2F/∂z1​∂z2​≥0 on (0,1)2(0,1)^2(0,1)2.
  4. Theorem 5.7.2: the polynomial distribution function is log-concave on (0,1]n(0,1]^n(0,1]n.

The platform theorem ConvexOptimization.prekopa_marginal_log_concave (Prékopa's marginal theorem, proved) is included as a reference item.

Significance

Theorem 5.1 turns a probabilistic constraint into a convex constraint after taking logarithms. This is what makes convergence proofs for nonlinear programming methods (SUMT with logarithmic penalty, supporting hyperplanes, reduced gradients) applicable to (5.1) and to the reliability maximization problem (5.4); without it, those methods have no guarantee of finding a global optimum. Theorems 5.2.1 and 5.2.2 are the two facts that make the SUMT method of §5.2 well defined and convex on the interior of the feasible set. Theorem 5.7.2 shows that probabilistic constraints under the polynomial distribution define convex sets, so they can be added to geometric programmes.

On status: Theorem 5.1 is a classical result, proved in Prékopa's papers (the chapter itself refers to Prékopa's survey for the proof). Prékopa's marginal theorem and the Prékopa–Leindler inequality are already machine-checked on this platform; Theorem 5.1 and the §5.2 and §5.7 theorems are, as far as a search of the platform shows, not formalized. The work is formalizing known proofs, in the log-concavity predicate the platform already uses.

Difficulty

The obvious argument for Theorem 5.1, "the constraint set is convex and the density is log-concave, so the probability is log-concave", hides the real content: log-concavity of a probability as a function of a parameter is a statement about integrals, and it does not follow from pointwise properties of the integrand without a Prékopa–Leindler-type inequality. Concavity of each gig_igi​ separately in xxx and in yyy is not enough; joint concavity in (x,y)(x, y)(x,y) is used essentially. The probability function vanishes on large regions in typical examples, so any argument that takes logarithms pointwise fails at the boundary of its support.

For Theorem 5.2.2 the naive argument fails at the index i=0i = 0i=0: nothing about h0h_0h0​ is assumed directly, and log-concavity of h0h_0h0​ is exactly Theorem 5.1. For Theorem 5.7.1 the difficulty is a sign condition on a covariance; without the ordering hypothesis the mixed derivative can be negative.

Formalization scope

Rn\mathbb R^nRn and Rq\mathbb R^qRq are EuclideanSpace ℝ (Fin n) and EuclideanSpace ℝ (Fin q); the constraint functions take pairs (x,y)(x, y)(x,y) in the product, and concavity is ConcaveOn ℝ Set.univ on that product (joint concavity). A "continuous probability distribution with density fff" is stated as P.map ξ = volume.withDensity (ENNReal.ofReal ∘ f) with ξ\xiξ and fff measurable and PPP a probability measure. Log-concavity is the platform definition ConvexOptimization.LogConcaveOn (nonnegativity plus the power inequality), imported as a reference item; concavity of Real.log ∘ h₀ would be a different, wrong property because Lean's Real.log 0 = 0. Indices are 0-based (Fin r, Fin m, Fin N, Fin n); the probabilistic constraint i=0i = 0i=0 of Theorem 5.2.2 is stated separately from h1,…,hmh_1, \dots, h_mh1​,…,hm​. The polynomial distribution is a formula on Fin n → ℝ with real powers, used only on the cube.

No constant of the book is replaced by an explicit value: every result of this chapter is qualitative.

Corrections of the printed text, recorded in each item's Formalization Note:

  • Theorem 5.1 states the density condition "for every x1,x2∈Rnx_1, x_2 \in \mathbb R^nx1​,x2​∈Rn"; the density lives on Rq\mathbb R^qRq and the condition is imposed there.
  • (5.19) prints the first factor as ziαi1z_i^{\alpha_{i1}}ziαi1​​ and the domain index as i=1,…,Ni = 1, \dots, Ni=1,…,N; they are read as z1αi1z_1^{\alpha_{i1}}z1αi1​​ and j=1,…,nj = 1, \dots, nj=1,…,n.
  • Theorem 5.7.1 prints its ordering hypothesis with transposed indices (α11≤α12≤⋯≤α1n\alpha_{11} \le \alpha_{12} \le \dots \le \alpha_{1n}α11​≤α12​≤⋯≤α1n​); following the proof, it is read as: across the NNN terms the z1z_1z1​-exponents increase and the z2z_2z2​-exponents decrease.
  • Theorem 5.7.1 claims "is a probability distribution function"; the book proves only (5.21), and normalization would need ∑ici=1\sum_i c_i = 1∑i​ci​=1, which is not assumed. The formal statement is (5.21), the mixed derivative as an iterated deriv.

Theorem 5.2.2 carries all six assumptions of §5.2, including compactness of the feasible set and the Slater point; it does not assume log-concavity of h0h_0h0​, which must be derived from the density and concavity hypotheses. A trivializing formalization, such as one with a density hypothesis that no probability law satisfies or a log-concavity predicate that holds for every function vanishing somewhere, is ruled out: the density hypotheses are satisfiable (checked locally) and LogConcaveOn is the multiplicative inequality at every pair of points.

Needed infrastructure: Prékopa's marginal theorem (available), log-concavity of indicators of convex sets and of products, measurability of the constraint set, and Artin's theorem that a sum of log-convex functions is log-convex (for Theorem 5.7.2). Artin's theorem and closure properties of LogConcaveOn are reusable beyond this mission, and contributions of them are welcome.

Selected references

  • A. Prékopa, "Numerical Solution of Probabilistic Constrained Programming Problems", in Yu. Ermoliev and R. J-B Wets (eds.), Numerical Techniques for Stochastic Optimization, Springer Series in Computational Mathematics 10, Springer 1988, Ch. 5, pp. 123–139. https://doi.org/10.1007/978-3-642-61370-8
  • A. Prékopa, "Logarithmic concave measures with application to stochastic programming", Acta Sci. Math. (Szeged) 32 (1971), 301–316.
  • A. Prékopa, "On logarithmic concave measures and functions", Acta Sci. Math. (Szeged) 34 (1973), 335–343.
  • A. Charnes and W. W. Cooper, "Chance-constrained programming", Management Science 6 (1959), 73–79. https://doi.org/10.1287/mnsc.6.1.73
  • B. L. Miller and H. M. Wagner, "Chance constrained programming with joint constraints", Operations Research 13 (1965), 930–945. https://doi.org/10.1287/opre.13.6.930
  • A. Prékopa, Stochastic Programming, Kluwer 1995. https://doi.org/10.1007/978-94-017-3087-7
9 thms4 active usersReviewed
🏆Completed
Convex OptimizationLinear OptimizationOperations Research+1·Captain: mikedeng1

Numerical Techniques for Stochastic Optimization I: Edmundson–Madansky Bounds for Independent Random Data and Simple RecourseTextbook

Motivation

In a two-stage stochastic linear program a decision xxx is taken before random data ξ\xiξ are observed, and a corrective recourse decision yyy is taken afterwards at a cost. The objective contains the expectation of an optimal value of a linear program, ∫Q(x,ξ(ω)) P(dω)\int Q(x,\xi(\omega))\,P(d\omega)∫Q(x,ξ(ω))P(dω), and for continuous or high-dimensional ξ\xiξ that integral cannot be evaluated exactly. Practical methods therefore replace ξ\xiξ by a discrete random vector and control the error by computable lower and upper bounds on the expected recourse cost. Chapter 2 of Ermoliev and Wets (eds.), Numerical Techniques for Stochastic Optimization (Springer 1988), by P. Kall, A. Ruszczyński and K. Frauendorfer, surveys these bounds as they were used in the codes of the time: Jensen's inequality from below, the Edmundson–Madansky inequality from above, and the special structure of simple recourse, where the expected cost is available in closed form.

Timeline. Jensen's inequality (1906) gives the lower bound for a convex integrand. A. Madansky, "Bounds on the expectation of a convex function of a multivariate random variable", Ann. Math. Statist. 30 (1959), and H. P. Edmundson (RAND report, 1956) gave the upper bound by the two-point law on the endpoints of an interval, and its product version for independent components. Kall and Stoyan (1982), Huang, Ziemba and Ben-Tal (1977), Frauendorfer and Kall (1988) developed partition refinement of both bounds, the scheme this chapter describes; Frauendorfer (1988) extended the upper bound to dependent data on boxes.

Setting

The two-stage problem (2.11) is: minimize ψ(x)=cTx+∫ΩQ(x,ξ(ω)) P(dω)\psi(x)=c^Tx+\int_\Omega Q(x,\xi(\omega))\,P(d\omega)ψ(x)=cTx+∫Ω​Q(x,ξ(ω))P(dω) subject to Ax=bAx=bAx=b, x≥0x\ge 0x≥0. The recourse cost Q(x,ξ)Q(x,\xi)Q(x,ξ) is the optimal value of the second-stage problem (2.12),

Q(x,ξ)=min⁡{qTy:Wy=h−Tx, y≥0},ξ=(q,h,T),Q(x,\xi)=\min\{q^Ty : Wy=h-Tx,\ y\ge 0\},\qquad \xi=(q,h,T),Q(x,ξ)=min{qTy:Wy=h−Tx, y≥0},ξ=(q,h,T),

with a deterministic m2×n2m_2\times n_2m2​×n2​ matrix WWW (fixed recourse), and Q=+∞Q=+\inftyQ=+∞ when (2.12) is infeasible. Throughout the chapter the book assumes complete recourse, {Wy:y≥0}=Rm2\{Wy:y\ge0\}=\mathbb R^{m_2}{Wy:y≥0}=Rm2​, and dual feasibility: for every realization of qqq some uuu satisfies WTu≤qW^Tu\le qWTu≤q. Under these assumptions QQQ is finite. The expected recourse function is Q(x)=∫Q(x,ξ(ω)) P(dω)\mathcal Q(x)=\int Q(x,\xi(\omega))\,P(d\omega)Q(x)=∫Q(x,ξ(ω))P(dω).

The Edmundson–Madansky law of an interval [a,b][a,b][a,b], a<ba<ba<b, with mean ξ0\xi^0ξ0 puts mass p1=(b−ξ0)/(b−a)p_1=(b-\xi^0)/(b-a)p1​=(b−ξ0)/(b−a) at aaa and p2=(ξ0−a)/(b−a)p_2=(\xi^0-a)/(b-a)p2​=(ξ0−a)/(b−a) at bbb (2.32). For a box Ξ=×j=1m[aj,bj]\Xi=\times_{j=1}^m[a_j,b_j]Ξ=×j=1m​[aj​,bj​] and means ξj0\xi^0_jξj0​, the vector ξ^\hat\xiξ^​ with independent components of these two-point laws sits at the vertex vvv with probability ∏jpj(vj)\prod_j p_j(v_j)∏j​pj​(vj​).

Simple recourse is the case W=[I,−I]W=[I,-I]W=[I,−I], q=[q+,q−]q=[q^+,q^-]q=[q+,q−] with qj++qj−≥0q^+_j+q^-_j\ge0qj+​+qj−​≥0, deterministic TTT and random hhh only. With χ=Tx\chi=Txχ=Tx the recourse cost splits into one-row costs Qj(χj,hj)=qj+(hj−χj)Q_j(\chi_j,h_j)=q^+_j(h_j-\chi_j)Qj​(χj​,hj​)=qj+​(hj​−χj​) if hj≥χjh_j\ge\chi_jhj​≥χj​, and qj−(χj−hj)q^-_j(\chi_j-h_j)qj−​(χj​−hj​) otherwise.

Formalization targets

Goal: the Edmundson–Madansky bound for independent components (p. 46)

If ξ\xiξ has independent components ξj∈[aj,bj]\xi_j\in[a_j,b_j]ξj​∈[aj​,bj​] with means ξj0\xi^0_jξj0​, and φ\varphiφ is convex on Ξ=×j[aj,bj]\Xi=\times_j[a_j,b_j]Ξ=×j​[aj​,bj​], then

Eφ(ξ)≤∑v∈vert Ξ(∏j=1mpj(vj))φ(v).E\varphi(\xi)\le\sum_{v\in\mathrm{vert}\,\Xi}\Big(\prod_{j=1}^m p_j(v_j)\Big)\varphi(v).Eφ(ξ)≤v∈vertΞ∑​(j=1∏m​pj​(vj​))φ(v).

The book applies it to φ=Q(x,⋅)\varphi=Q(x,\cdot)φ=Q(x,⋅); the goal is stated for every convex φ\varphiφ, with the explicit weights of (2.32).

Milestones

  1. Properties (b), (d), (e) of p. 40: Q(x,⋅)Q(x,\cdot)Q(x,⋅) is piecewise linear and convex in (h,T)(h,T)(h,T); Q(⋅,ξ)Q(\cdot,\xi)Q(⋅,ξ) is convex piecewise linear in xxx; the expected recourse function is finite and convex under finite second moments.
  2. The Jensen lower bound (2.26)–(2.27) on a partition (a published, proved theorem, reused).
  3. The dual-multiplier lower bound (2.30)–(2.31).
  4. The one-dimensional Edmundson–Madansky inequality (2.32)–(2.34).
  5. For simple recourse: separability (2.46)–(2.49), the closed form (2.51) of EQjEQ_jEQj​, and the bounds (2.55)–(2.56) from the one-block problem.

Significance

The upper bound is the half of the bounding scheme that is not automatic. Jensen's inequality needs only a mean; an upper bound on the expectation of a convex function needs a bounded support and, in the product form, independence. Together they give a certified interval for the optimal value of a two-stage problem, and repeated partitioning of the support shrinks that interval; this is the basis of the sequential approximation methods of §2.2.4 and of later codes. The dual-multiplier bound and the simple-recourse formulas are the pieces that make those intervals cheap to compute.

The results are classical and proved in the literature cited on the page. The one-dimensional Edmundson–Madansky inequality and the general extreme-point form of the upper bound (a measure on the extreme points reproducing the barycentre) are already formalized on Prove2Me in the Introduction to Stochastic Programming series, as is the partition Jensen bound. The product form for independent components is not: deriving it from the extreme-point form requires constructing the product kernel, which is the content of this mission. The recourse properties (b), (d), (e) for a general distribution with finite second moments, the dual-multiplier bound and the simple-recourse formulas are not formalized anywhere known to this mission.

Difficulty

The obvious argument inducts on the dimension, applying the one-dimensional inequality in one coordinate while the others are held fixed. That step needs the conditional law of the remaining coordinates given the first to be their unconditional law, i.e. independence expressed as a product decomposition of the joint law, and it needs φ\varphiφ with one coordinate replaced by an endpoint to remain convex on the lower-dimensional box and integrable. For dependent components the inequality is false with these weights: on [0,1]2[0,1]^2[0,1]2 with means (12,12)(\tfrac12,\tfrac12)(21​,21​) and φ(x,y)=(x−y)2\varphi(x,y)=(x-y)^2φ(x,y)=(x−y)2, the product law gives 12\tfrac1221​ while mass 12\tfrac1221​ at (1,0)(1,0)(1,0) and at (0,1)(0,1)(0,1) gives 111. The book's remark that the product law is extremal among all laws on Ξ\XiΞ with the given mean fails for this reason when m≥2m\ge2m≥2, and is not part of this mission.

For the recourse properties the difficulty is bookkeeping: QQQ is an extended-real optimal value, and finiteness, measurability in ω\omegaω and integrability must be derived from complete recourse, dual feasibility and the moment hypothesis rather than assumed.

Formalization scope

Vectors are functions from finite index types to R\mathbb RR (ι → ℝ), matrices are Mathlib Matrix, and random data live on a probability space (Ω, P). The recourse cost is an EReal infimum over the feasible set, so infeasibility gives +∞+\infty+∞ and unboundedness −∞-\infty−∞ exactly as on p. 39; theorems that integrate it carry complete recourse and dual feasibility, which make it finite. The expected recourse function integrates the real part of the recourse cost. Independence of the components is ProbabilityTheory.iIndepFun; the box is Set.pi univ (fun j => Icc (a j) (b j)), with aj<bja_j<b_jaj​<bj​, and values in the box are required almost surely. The upper bound is the explicit sum over Boolean vertex labels of products of the weights (2.32); no abstract extremal measure is used.

Conventions fixed where the page is silent or ambiguous:

  • Properties (b), (d), (e) are stated on all of Rn1\mathbb R^{n_1}Rn1​: under the standing complete-recourse assumption K2=Rn1K_2=\mathbb R^{n_1}K2​=Rn1​. "Convex piecewise linear" is rendered as a maximum of finitely many affine functions.
  • The book's hypothesis of finite second moments in (e) is kept as stated, componentwise.
  • The book writes QQQ for both Q(x,ξ)Q(x,\xi)Q(x,ξ) and Q(x)\mathcal Q(x)Q(x), and reuses Q~\tilde QQ~​, ψ~\tilde\psiψ~​ for different functions in (2.27) and (2.30)–(2.31); the Lean names are recourseCost, expectedRecourse and dualLowerBound.
  • In (2.51) a conditional mean on a null event is 000 in Lean; it always appears multiplied by that event's probability, so the formula is unchanged.
  • In (2.56) the minimum is a real infimum over the nonempty first-stage feasible set; attainment is not claimed.
  • No constant of the chapter is hidden behind O(⋅)O(\cdot)O(⋅); all bounds are explicit.

A goal stated for affine φ\varphiφ (where it is an equality), or with ξ^\hat\xiξ^​ allowed to be any discrete law with the right mean, would be trivial or a different theorem; the weights are the products of (2.32), and independence of the components is a hypothesis.

A complete development needs: finite-dimensional LP duality with extended-real values (reusable across all recourse missions), measurability and integrability of optimal-value functions, the conditional-independence step for product measures, and the one-dimensional chord inequality. The partitioned upper bound (2.37) and the discrete reformulation (2.21), (2.28) are natural follow-up statements on the same definitions.

Selected references

  • P. Kall, A. Ruszczyński, K. Frauendorfer, "Approximation Techniques in Stochastic Programming", in Yu. Ermoliev and R. J-B Wets (eds.), Numerical Techniques for Stochastic Optimization, Springer Series in Computational Mathematics 10, Springer 1988, Ch. 2, pp. 33–64. https://doi.org/10.1007/978-3-642-61370-8
  • A. Madansky, "Bounds on the expectation of a convex function of a multivariate random variable", Annals of Mathematical Statistics 30 (1959), 743–746. https://doi.org/10.1214/aoms/1177706203
  • P. Kall, Stochastic Linear Programming, Springer 1976. https://doi.org/10.1007/978-3-642-66252-2
  • R. J-B Wets, "Stochastic programs with fixed recourse: the equivalent deterministic program", SIAM Review 16 (1974), 309–339. https://doi.org/10.1137/1016053
  • K. Frauendorfer, "Solving SLP recourse problems with arbitrary multivariate distributions — the dependent case", Mathematics of Operations Research 13 (1988), 377–394. https://doi.org/10.1287/moor.13.3.377
  • J. R. Birge, F. Louveaux, Introduction to Stochastic Programming, Springer 1997, Ch. 8. https://doi.org/10.1007/b97617
13 thms4 active usersReviewed
🏆Completed
Linear OptimizationOperations ResearchTheoretical Computer Science·Captain: mikedeng1

Competitive Randomized Algorithms for Nonuniform Problems IV: The Optimal Randomized Two-Server Ratio 1652/1069 on the 3-4-5 TriangleResearch Paper

Motivation

The kkk-server problem is a basic model of on-line decision making. kkk mobile servers move in a metric space, requests for points arrive one at a time, and each request has to be covered by a server before the next one arrives. The cost is the total distance the servers move. The problem includes paging, caching and disk-head scheduling as special cases (Manasse, McGeoch, Sleator 1990). An on-line algorithm is judged by its competitive factor: how much its cost can exceed that of an off-line algorithm that knows the whole request sequence in advance.

For randomized algorithms against an oblivious adversary (one that fixes the whole request sequence before the algorithm flips any coins), the best-understood case is paging, which is the kkk-server problem on a uniform metric space. There the optimal factor is the harmonic number Hk=∑i=1k1/iH_k=\sum_{i=1}^k 1/iHk​=∑i=1k​1/i. Fiat et al. proved the lower bound (1991) and McGeoch and Sleator the matching upper bound (1991). Karlin, Manasse, McGeoch and Owicki (Algorithmica 11, 1994, §5) asked whether HkH_kHk​-competitive algorithms also exist when the metric space is not uniform. They answered no, already for two servers on three points: on certain triangles the optimal randomized factor is strictly larger than H2=3/2H_2 = 3/2H2​=3/2. This mission formalizes their Theorem 13, which gives the exact optimal factor on the triangle with edge lengths 3, 4 and 5.

Timeline:

  • 1990: Manasse, McGeoch and Sleator introduce the kkk-server problem; kkk is the deterministic optimum for k=2k=2k=2.
  • 1991: Fiat, Karp, Luby, McGeoch, Sleator and Young prove the HkH_kHk​ lower bound for randomized paging. McGeoch and Sleator give an HkH_kHk​-competitive paging algorithm.
  • 1994: Karlin, Manasse, McGeoch and Owicki determine the optimal randomized two-server factors on the isosceles triangles 111-ddd-ddd (Theorem 12) and on the 3-4-5 triangle (Theorem 13, the ratio 1652/10691652/10691652/1069). Both exceed 3/23/23/2.

Setting

Let MMM be a metric space with exactly three points a,b,ca, b, ca,b,c, where d(a,b)=3d(a,b)=3d(a,b)=3, d(a,c)=5d(a,c)=5d(a,c)=5 and d(b,c)=4d(b,c)=4d(b,c)=4. A configuration CCC gives the positions of two labelled servers in MMM. A request sequence σ\sigmaσ is a finite list of points of MMM.

A deterministic on-line algorithm assigns to each prefix of a request sequence a configuration, in which the last request is covered. Its configuration after a prefix therefore cannot depend on later requests. Its initial configuration is the one it assigns to the empty prefix, and its cost CA(σ)C_A(\sigma)CA​(σ) on σ\sigmaσ is the total distance its servers move while serving σ\sigmaσ request by request.

The optimal off-line cost Copt(σ)C_{opt}(\sigma)Copt​(σ) from an initial configuration C0C_0C0​ is the infimum, over all schedules that start at C0C_0C0​ and cover each request of σ\sigmaσ in turn, of the total distance moved.

A randomized on-line algorithm AAA is a probability distribution over deterministic on-line algorithms, all starting at C0C_0C0​. The cost on each fixed σ\sigmaσ is required to be measurable in the random choice, and ECA(σ)\mathbf{E}C_A(\sigma)ECA​(σ) is the expected cost. AAA is ρ\rhoρ-competitive against an oblivious adversary if there is a constant aaa such that for every request sequence σ\sigmaσ,

ECA(σ)≤ρ⋅Copt(σ)+a.\mathbf{E}C_A(\sigma) \le \rho\cdot C_{opt}(\sigma) + a .ECA​(σ)≤ρ⋅Copt​(σ)+a.

These are the definitions of p. 543 of the paper. They are the platform's published KServer_model and KServer_randomized, which this mission reuses unchanged: KServer.RandomizedAlgorithm 2 M and A.IsCompetitiveFrom C₀ ρ.

Formalization targets

Goal: Theorem 13

For every initial configuration C0C_0C0​ of the two servers,

(∀A, ∀ρ, A is ρ-competitive from C0⇒ρ≥16521069) ∧ (∃A, A is 16521069-competitive from C0).\Big(\forall A,\ \forall \rho,\ A \text{ is } \rho\text{-competitive from } C_0 \Rightarrow \rho \ge \tfrac{1652}{1069}\Big)\ \wedge\ \Big(\exists A,\ A \text{ is } \tfrac{1652}{1069}\text{-competitive from } C_0\Big).(∀A, ∀ρ, A is ρ-competitive from C0​⇒ρ≥10691652​) ∧ (∃A, A is 10691652​-competitive from C0​).

The first claim is quantified over all randomized algorithms, so it also covers deterministic ones (point masses). The second claim asks for one algorithm. Together they say that 1652/1069≈1.5451652/1069 \approx 1.5451652/1069≈1.545 is the exact optimal randomized factor on this triangle.

Milestones

  1. The phase LP lower bound (p. 568). Twelve linear constraints in nine probabilities π1,…,π9\pi_1,\dots,\pi_9π1​,…,π9​, three potentials Φab,Φac,Φbc\Phi_{ab},\Phi_{ac},\Phi_{bc}Φab​,Φac​,Φbc​ and a ratio α\alphaα, one constraint for each possible phase of the request sequence, of the form
A’s cost≤α⋅(opt’s cost)+Φinitial−Φfinal.\text{A's cost} \le \alpha\cdot(\text{opt's cost}) + \Phi_{\text{initial}} - \Phi_{\text{final}}.A’s cost≤α⋅(opt’s cost)+Φinitial​−Φfinal​.

Every real solution has α≥1652/1069\alpha \ge 1652/1069α≥1652/1069. 2. The LP attainment (p. 568). The paper's printed probabilities lie in [0,1][0,1][0,1], and with suitable potentials they satisfy all twelve constraints at α=1652/1069\alpha = 1652/1069α=1652/1069. 3. Theorem 13, first claim: the lower bound for every randomized algorithm. 4. Theorem 13, second claim: a 1652/10691652/10691652/1069-competitive randomized algorithm exists.

Significance

The result. Theorem 13 shows that the HkH_kHk​ behaviour of randomized paging does not carry over to general metric spaces. Two servers on a three-point space already force a factor above 3/23/23/2. The value is exact, which makes this triangle a test case for any general theory of randomized kkk-server algorithms on small metric spaces. With Theorem 12 (the isosceles triangles, a companion mission of this series), it is one of the few non-uniform metric spaces with a known optimal randomized factor.

Formalizing it. The result has been proved since 1994. To our knowledge there is no machine-checked proof. The paper derives both bounds from two framework theorems for phase-based algorithms: Theorem 3 (an LP lower bound for phase-based algorithms bounds every algorithm) and Theorem 2 (a lazy phase-based algorithm with LP bound α\alphaα is α\alphaα-competitive). The phase tables themselves (which phases can occur and what they cost) are stated without detailed proof. A formal proof has to supply both framework arguments for this space and verify the phase tables, as well as the finite linear algebra of milestones 1 and 2. The milestones isolate the exact-arithmetic core so that it can be closed independently of the probabilistic part.

Difficulty

The two LP milestones are finite exact-arithmetic facts. The hard part is linking them to Theorem 13.

For the lower bound, an algorithm need not be phase-based at all. Its probabilities may depend on the whole history, not only on the current phase, and it may leave the configuration of the off-line optimum at the end of a phase. The obvious attempt is to fix one hard request sequence and compare costs, but that cannot work: randomization defeats any single sequence. The reduction from arbitrary algorithms to phase-based ones (the paper's Theorem 3) is the substantive step.

For the upper bound, the printed probabilities describe the algorithm's marginal position after each prefix of a phase. They have to be realized as a single probability distribution over deterministic on-line algorithms that is lazy (it moves only to serve a request) and whose expected cost per phase equals the table's entry. On top of this, the LP accounting has to be turned into a bound on arbitrary request sequences, including partial phases and a start away from the optimum's configuration.

Formalization scope

  • Model. The platform definitions KServer_model and KServer_randomized are used unchanged. Servers are labelled (Config 2 M = Fin 2 → M). A deterministic algorithm is a function of the request prefix, which makes it on-line by construction. A randomized algorithm is a mixed strategy with a probability measure and a measurability field, and its expected cost is the lower Lebesgue integral of the nonnegative cost. The off-line optimum is a real infimum over schedules from C0C_0C0​; the set is nonempty and bounded below by 000. Competitiveness allows any real additive constant.
  • The triangle is given by hypotheses on an arbitrary metric space: every point equals aaa, bbb or ccc, and d(a,b)=3d(a,b)=3d(a,b)=3, d(a,c)=5d(a,c)=5d(a,c)=5, d(b,c)=4d(b,c)=4d(b,c)=4. These hypotheses are satisfiable (3+4≥53+4\ge53+4≥5) and force three distinct points.
  • Initial configuration. Both claims are stated for every initial configuration C0C_0C0​, including both servers on one point. The paper does not fix the start; the additive constant absorbs it.
  • LP milestones. The thirteen LP variables are free reals, with no box 0≤πi≤10\le\pi_i\le 10≤πi​≤1, exactly as the paper permits. This makes milestone 1 stronger than the boxed version; the minimum is the same either way. The twelve constraints are written out one per hypothesis, in the table's order, with the potential difference Φinitial−Φfinal\Phi_{\text{initial}} - \Phi_{\text{final}}Φinitial​−Φfinal​ on the right. In milestone 2 the potentials are existentially quantified, since the paper names none.
  • Not stated. The paper's Theorems 2 and 3 (the phase framework) and the phase tables are not separate milestones. Milestone 1 feeds the first claim through Theorem 3, and milestone 2 feeds the second claim through Theorem 2. Contributions formalizing phase-based algorithms, laziness and the LP-bound reduction for finite metric spaces would be reusable for Theorem 12 and Theorem 14 of the same paper.
  • Ruled out. The lower bound is not restricted to deterministic or to phase-based algorithms, and it is not stated as "one sequence defeats every algorithm". The constant is exactly 1652/10691652/10691652/1069, not an approximation, and the attainment claim is not weakened to "for some initial configuration".

Selected references

  • A. R. Karlin, M. S. Manasse, L. A. McGeoch, S. Owicki, Competitive Randomized Algorithms for Nonuniform Problems, Algorithmica 11 (1994) 542–571. https://doi.org/10.1007/BF01189993
  • M. S. Manasse, L. A. McGeoch, D. D. Sleator, Competitive Algorithms for Server Problems, Journal of Algorithms 11 (1990) 208–230. https://doi.org/10.1016/0196-6774(90)90003-W
  • A. Fiat, R. M. Karp, M. Luby, L. A. McGeoch, D. D. Sleator, N. E. Young, Competitive Paging Algorithms, Journal of Algorithms 12 (1991) 685–699. https://doi.org/10.1016/0196-6774(91)90041-V
  • L. A. McGeoch, D. D. Sleator, A Strongly Competitive Randomized Paging Algorithm, Algorithmica 6 (1991) 816–825. https://doi.org/10.1007/BF01759073
7 thms4 active usersReviewed
🏆Completed
Linear OptimizationOperations ResearchTheoretical Computer Science·Captain: mikedeng1

Competitive Randomized Algorithms for Nonuniform Problems I: Optimal Competitiveness of Randomized Block Snoopy CachingResearch Paper

Motivation

In a shared-memory multiprocessor, each processor keeps copies of memory blocks in its own cache, and all caches listen ("snoop") on a common bus. Every bus cycle spent keeping these copies consistent is a cycle not available for useful work, so the protocol that decides when a block is shared by several caches and when it is private to one cache directly controls bus traffic. The decision has to be made on-line, without knowing which processor will touch the block next.

Karlin, Manasse, Rudolph and Sleator (Algorithmica 1988) introduced competitive analysis for this problem and gave a deterministic algorithm with competitive ratio 222, which is optimal among deterministic algorithms. Karlin, Manasse, McGeoch and Owicki (Algorithmica 1994) showed that randomization helps: against an oblivious adversary the optimal ratio for block snoopy caching is ep/(ep−1)e_p/(e_p-1)ep​/(ep​−1), where ppp is the cost of transferring a block. The same paper develops a general method for "nonuniform" problems, in which some state transitions are much more expensive than others, and the snoopy-caching result is its first application.

Setting

Fix nnn processors and one memory block BBB holding p−1p-1p−1 variables; transferring BBB over the bus costs ppp bus cycles. The block is in one of n+1n+1n+1 states: shared between all caches, or private to the cache of processor iii.

A request is a read Ri\mathrm{R}_iRi​ or a write Wi\mathrm{W}_iWi​ by processor iii. Moving from a private state to any other state costs ppp; moving from the shared state is free. A read Ri\mathrm{R}_iRi​ costs 000 if BBB is shared or private to iii and +∞+\infty+∞ otherwise. A write Wi\mathrm{W}_iWi​ costs 000 if BBB is private to iii, 111 if BBB is shared (one bus cycle broadcasts the new value), and +∞+\infty+∞ otherwise.

Before request jjj the system is in state sj−1s_{j-1}sj−1​. A read is a look-ahead-one request: the algorithm may change state at the moment of the request, after seeing it. A write is a look-ahead-zero request: it is served in whatever state the system is in. After either kind, the algorithm may move again. The cost of a request is the cost of the move to the serving state, plus the task cost there, plus the cost of the move afterwards. Every write is preceded by a read to the same block, so in an admissible sequence each write Wi\mathrm{W}_iWi​ directly follows Ri\mathrm{R}_iRi​ or Wi\mathrm{W}_iWi​.

The off-line optimum Copt(s0,σ)C_{opt}(s_0,\sigma)Copt​(s0​,σ) is the least total cost of serving σ\sigmaσ from the initial state s0s_0s0​ with full knowledge of σ\sigmaσ. A randomized on-line algorithm AAA is a probability distribution over deterministic on-line algorithms; its expected cost on σ\sigmaσ is ECA(σ)\mathbf{E}C_A(\sigma)ECA​(σ). AAA is ccc-competitive against an oblivious adversary from s0s_0s0​ if there is a constant aaa with

ECA(σ)≤c⋅Copt(s0,σ)+a\mathbf{E}C_A(\sigma)\le c\cdot C_{opt}(s_0,\sigma)+aECA​(σ)≤c⋅Copt​(s0​,σ)+a

for every admissible σ\sigmaσ. Put

ep=(1+1p)p.e_p=\left(1+\frac1p\right)^p .ep​=(1+p1​)p.

Formalization targets

Goal: Theorem 4

For n≥2n\ge 2n≥2, p≥1p\ge 1p≥1 and every initial state s0s_0s0​:

(∀A ∀c: A is c-competitive from s0⇒c≥epep−1) ∧ ∃A: A is epep−1-competitive from s0.\Big(\forall A\ \forall c:\ A \text{ is } c\text{-competitive from } s_0 \Rightarrow c\ge \tfrac{e_p}{e_p-1}\Big)\ \wedge\ \exists A:\ A \text{ is } \tfrac{e_p}{e_p-1}\text{-competitive from } s_0 .(∀A ∀c: A is c-competitive from s0​⇒c≥ep​−1ep​​) ∧ ∃A: A is ep​−1ep​​-competitive from s0​.

The two conjuncts are milestones of their own: the lower bound (Theorem 4, first claim) and attainment (Theorem 4, second claim).

The phase linear program (§3.2, pp. 552–554)

For p≥1p\ge1p≥1, real π1,…,πp+1\pi_1,\dots,\pi_{p+1}π1​,…,πp+1​ with πp+1=1\pi_{p+1}=1πp+1​=1, and real α\alphaα with

πk+1p+∑i=1k(1−πi)≤αk(k=0,…,p),\pi_{k+1}p+\sum_{i=1}^{k}(1-\pi_i)\le\alpha k\qquad(k=0,\dots,p),πk+1​p+i=1∑k​(1−πi​)≤αk(k=0,…,p),

one has α≥ep/(ep−1)\alpha\ge e_p/(e_p-1)α≥ep​/(ep​−1). Conversely, at α=ep/(ep−1)\alpha=e_p/(e_p-1)α=ep​/(ep​−1) the choice πk=(α−1)(((p+1)/p)k−1−1)\pi_k=(\alpha-1)\big(((p+1)/p)^{k-1}-1\big)πk​=(α−1)(((p+1)/p)k−1−1) satisfies πp+1=1\pi_{p+1}=1πp+1​=1, 0≤π1≤⋯≤πp+10\le\pi_1\le\dots\le\pi_{p+1}0≤π1​≤⋯≤πp+1​, and makes every constraint an equality.

Significance

The theorem settles the randomized competitive ratio of block snoopy caching exactly: 222 at p=1p=1p=1, 9/59/59/5 at p=2p=2p=2, decreasing to e/(e−1)≈1.582e/(e-1)\approx1.582e/(e−1)≈1.582 as p→∞p\to\inftyp→∞, against the deterministic optimum 222. The same ratio e/(e−1)e/(e-1)e/(e−1) is the randomized optimum for the continuous ski-rental and spin-block problems treated later in the paper, and the snoopy-caching case is its discrete counterpart with ratio ep/(ep−1)e_p/(e_p-1)ep​/(ep​−1). The phase-LP method used here recurs in the paper's two-server results.

The result is proved in the paper; to our knowledge it has no machine-checked proof. This mission produces a formal model of the snoopy-caching task system with look-ahead-zero requests, of randomized algorithms against an oblivious adversary with infinite task costs allowed, and of the off-line optimum, together with the exact optimal ratio. The platform's fractional ski-rental result (PrimalDualOnline.SkiRental.fractional_competitive) proves an eB/(eB−1)e_B/(e_B-1)eB​/(eB​−1) bound for a different model: one deterministic fractional algorithm, with no lower bound over randomized algorithms. It is related work, not a special case.

Difficulty

The linear program is elementary. The gap is between the LP and the algorithms. The paper's lower bound reduces arbitrary randomized algorithms to phase-based ones, whose state distribution at the end of each phase agrees with the optimal algorithm's known state, and whose behaviour inside a phase depends only on the number of writes so far. This reduction (Theorems 1 and 3 of the paper, pp. 545–549) is where the argument is not routine. An algorithm may keep the block private to a processor that is not the active one, may randomize over histories rather than over phase lengths, and the off-line optimum is not a sum of per-phase costs at the ends of the sequence. The obvious approach, bounding a single adversarial phase, does not suffice, because an algorithm may pay more in one phase and recover it in the next; the additive constant aaa and the infinite horizon have to be handled. For attainment, the mixture of threshold algorithms must be written as a genuine distribution over on-line algorithms, with the initial phase from a private state absorbed into the additive constant.

Formalization scope

Everything lives in the namespace NonuniformCompetitive.Snoopy. States are Option (Fin n) (none = shared). Costs are in ℝ≥0∞; +∞+\infty+∞ is a genuine outcome, so an algorithm that ever pays +∞+\infty+∞ with positive probability on an admissible sequence is not competitive. A deterministic on-line algorithm is a pair of functions of the request prefix (the state at the moment of the last request, and the state after it), with the look-ahead-zero rule as a field. Moves "immediately before" a request are made without knowledge of it and are recorded as moves after the previous request. A randomized algorithm is a probability space with a measurable cost on every sequence, and its expected cost is a lower Lebesgue integral. The off-line optimum is an infimum in ℝ≥0∞ over schedules starting in s0s_0s0​; it is finite on admissible sequences.

Conventions added to the printed statement, all from the paper's setting: (i) n≥2n\ge2n≥2, since with one processor the block can stay private for free; (ii) one block, since the proof of Theorem 4 splits a multi-block system into independent blocks (p. 551); (iii) admissibility in the form "each write of iii directly follows a read or write of iii", the reading of "every write is preceded by a read to the same block" that the proof uses; without it every algorithm is defeated by a write from a processor whose block copy was invalidated; (iv) p∈Np\in\mathbb{N}p∈N, p≥1p\ge1p≥1; (v) both claims from every initial state, with an additive constant depending on nnn, ppp, s0s_0s0​.

The lower bound is over all randomized algorithms, not over phase-based or deterministic ones; a statement restricted to phase-based algorithms, or the LP alone in place of the goal, would not be Theorem 4. The LP variables are free, as in the paper.

Welcome contributions: a formal version of the phase reduction (Theorems 1 and 3 of the paper) for this task system, which is reusable for the paper's other nonuniform problems; the threshold algorithms and their mixture; and a proof that the off-line optimum decomposes by write runs up to a bounded error.

Selected references

  • A. R. Karlin, M. S. Manasse, L. A. McGeoch, S. Owicki, Competitive Randomized Algorithms for Nonuniform Problems, Algorithmica 11 (1994), 542–571. https://doi.org/10.1007/BF01189993
  • A. R. Karlin, M. S. Manasse, L. Rudolph, D. D. Sleator, Competitive Snoopy Caching, Algorithmica 3 (1988), 79–119. https://doi.org/10.1007/BF01762111
  • A. Borodin, R. El-Yaniv, Online Computation and Competitive Analysis, Cambridge University Press, 1998.
8 thms4 active usersReviewed
🏆Completed
Convex OptimizationOperations ResearchOptimization·Captain: mikedeng1

Random Gradient-Free Minimization of Convex Functions III: Accelerated Random SearchResearch Paper

Motivation

Many optimization problems in engineering, simulation-based design and machine learning give access only to function values: the objective is the output of a simulator or a black-box program, and its gradient is unavailable or too expensive. Derivative-free (or zeroth-order) methods address this setting. Nesterov and Spokoiny (Found. Comput. Math. 17 (2017)) showed that a very simple oracle, the finite difference of fff along a random Gaussian direction, can replace the gradient in standard first-order schemes at the price of a factor depending only on the dimension. Their analysis became the reference point for later work on zeroth-order stochastic optimization and on gradient-free methods in reinforcement learning and adversarial attacks.

This mission covers Section 6 of the paper: the accelerated random method FGμ\mathcal{FG}_\muFGμ​ and its rate, Theorem 9. It is the third mission of a series; the first covers random search for nonsmooth problems (Theorem 6), the second the random gradient method for smooth problems (Theorem 8).

Setting

Let EEE be a real inner product space of dimension n≥2n \ge 2n≥2 with norm ∥⋅∥\|\cdot\|∥⋅∥ (the paper's space with operator BBB is EEE with the inner product ⟨Bx,y⟩\langle Bx, y\rangle⟨Bx,y⟩). Let uuu be a standard Gaussian vector in EEE, and write Eu\mathbb E_uEu​ for expectation over uuu.

The objective f:E→Rf : E \to \mathbb Rf:E→R is differentiable with Lipschitz gradient, ∥∇f(x)−∇f(y)∥≤L1∥x−y∥\|\nabla f(x) - \nabla f(y)\| \le L_1\|x - y\|∥∇f(x)−∇f(y)∥≤L1​∥x−y∥ with L1>0L_1 > 0L1​>0, and strongly convex with parameter τ≥0\tau \ge 0τ≥0:

f(y)≥f(x)+⟨∇f(x),y−x⟩+τ2∥y−x∥2.f(y) \ge f(x) + \langle\nabla f(x), y - x\rangle + \tfrac{\tau}{2}\|y - x\|^2 .f(y)≥f(x)+⟨∇f(x),y−x⟩+2τ​∥y−x∥2.

The value τ=0\tau = 0τ=0 is allowed (plain convexity). The condition number is κ=τ/L1\kappa = \tau/L_1κ=τ/L1​. The problem f∗=min⁡x∈Ef(x)f^* = \min_{x \in E} f(x)f∗=minx∈E​f(x) is assumed solvable, with minimizer x∗x^*x∗.

For μ≥0\mu \ge 0μ≥0 the Gaussian approximation is fμ(x)=Euf(x+μu)f_\mu(x) = \mathbb E_u f(x + \mu u)fμ​(x)=Eu​f(x+μu), and the random gradient-free oracle is

B−1gμ(x)=f(x+μu)−f(x)μ u(μ>0),B−1g0(x)=⟨∇f(x),u⟩ u.B^{-1}g_\mu(x) = \frac{f(x + \mu u) - f(x)}{\mu}\,u \quad (\mu > 0), \qquad B^{-1}g_0(x) = \langle\nabla f(x), u\rangle\,u .B−1gμ​(x)=μf(x+μu)−f(x)​u(μ>0),B−1g0​(x)=⟨∇f(x),u⟩u.

The paper (p. 548) sets θn=1/(16(n+1)2L1(f))\theta_n = 1/(16(n+1)^2L_1(f))θn​=1/(16(n+1)2L1​(f)) and hn=1/(4(n+4)L1(f))h_n = 1/(4(n+4)L_1(f))hn​=1/(4(n+4)L1​(f)). This mission uses θn=1/(16(n+4)2L1)\theta_n = 1/(16(n+4)^2L_1)θn​=1/(16(n+4)2L1​); the reason is given under Formalization scope. Method FGμ\mathcal{FG}_\muFGμ​ (Eq. (60)) chooses x0∈Ex_0 \in Ex0​∈E, v0=x0v_0 = x_0v0​=x0​ and γ0>0\gamma_0 > 0γ0​>0 with γ0≥τ\gamma_0 \ge \tauγ0​≥τ, and at every iteration k≥0k \ge 0k≥0:

  1. computes αk>0\alpha_k > 0αk​>0 with θn−1αk2=(1−αk)γk+αkτ≡γk+1\theta_n^{-1}\alpha_k^2 = (1 - \alpha_k)\gamma_k + \alpha_k\tau \equiv \gamma_{k+1}θn−1​αk2​=(1−αk​)γk​+αk​τ≡γk+1​;
  2. sets λk=αkτ/γk+1\lambda_k = \alpha_k\tau/\gamma_{k+1}λk​=αk​τ/γk+1​, βk=αkγk/(γk+αkτ)\beta_k = \alpha_k\gamma_k/(\gamma_k + \alpha_k\tau)βk​=αk​γk​/(γk​+αk​τ) and yk=(1−βk)xk+βkvky_k = (1-\beta_k)x_k + \beta_k v_kyk​=(1−βk​)xk​+βk​vk​;
  3. draws a fresh Gaussian direction uku_kuk​, independent of the past, and computes gμ(yk)g_\mu(y_k)gμ​(yk​);
  4. sets xk+1=yk−hnB−1gμ(yk)x_{k+1} = y_k - h_n B^{-1}g_\mu(y_k)xk+1​=yk​−hn​B−1gμ​(yk​) and vk+1=(1−λk)vk+λkyk−(θn/αk)B−1gμ(yk)v_{k+1} = (1-\lambda_k)v_k + \lambda_k y_k - (\theta_n/\alpha_k)B^{-1}g_\mu(y_k)vk+1​=(1−λk​)vk​+λk​yk​−(θn​/αk​)B−1gμ​(yk​).

Write ϕk=Ef(xk)\phi_k = \mathbb E f(x_k)ϕk​=Ef(xk​) (expectation over u0,…,uk−1u_0, \dots, u_{k-1}u0​,…,uk−1​), ψk=∏i=0k−1(1−αi)\psi_k = \prod_{i=0}^{k-1}(1-\alpha_i)ψk​=∏i=0k−1​(1−αi​) and Ck=1+∑i=1k−1∏j=k−ik−1(1−αj)C_k = 1 + \sum_{i=1}^{k-1}\prod_{j=k-i}^{k-1}(1-\alpha_j)Ck​=1+∑i=1k−1​∏j=k−ik−1​(1−αj​) for k≥1k \ge 1k≥1, with ψ0=1\psi_0 = 1ψ0​=1 and C0=0C_0 = 0C0​=0 (p. 550).

Formalization targets

Goal: Theorem 9 (p. 549)

For all k≥0k \ge 0k≥0,

ϕk−f∗≤ψk[f(x0)−f(x∗)+γ02∥x0−x∗∥2]+μ2L1(n+3(n+8)16Ck),(62)\phi_k - f^* \le \psi_k\Big[f(x_0) - f(x^*) + \frac{\gamma_0}{2}\|x_0 - x^*\|^2\Big] + \mu^2 L_1\Big(n + \frac{3(n+8)}{16}C_k\Big), \tag{62}ϕk​−f∗≤ψk​[f(x0​)−f(x∗)+2γ0​​∥x0​−x∗∥2]+μ2L1​(n+163(n+8)​Ck​),(62)

where

ψk≤min⁡{(1−κ1/24(n+4))k, (1+k8(n+4)γ0L1)−2},Ck≤min⁡{k, 4(n+4)κ1/2}.\psi_k \le \min\Big\{\Big(1 - \frac{\kappa^{1/2}}{4(n+4)}\Big)^k,\ \Big(1 + \frac{k}{8(n+4)}\sqrt{\frac{\gamma_0}{L_1}}\Big)^{-2}\Big\}, \qquad C_k \le \min\Big\{k,\ \frac{4(n+4)}{\kappa^{1/2}}\Big\}.ψk​≤min{(1−4(n+4)κ1/2​)k, (1+8(n+4)k​L1​γ0​​​)−2},Ck​≤min{k, κ1/24(n+4)​}.

The two regimes are a rate O(n2/k2)O(n^2/k^2)O(n2/k2) for convex fff and a linear rate with ratio 1−κ1/2/(4(n+4))1 - \kappa^{1/2}/(4(n+4))1−κ1/2/(4(n+4)) for strongly convex fff, both up to a bias proportional to μ2\mu^2μ2.

Milestones

In attack order: Lemma 1 (Gaussian moments, (16)–(17)); Theorem 3.1 (the bound (32) on the second moment of g0g_0g0​); Theorem 1's (19), ∣fμ−f∣≤μ22L1n|f_\mu - f| \le \frac{\mu^2}{2}L_1 n∣fμ​−f∣≤2μ2​L1​n; Eq. (12), L1(fμ)≤L1(f)L_1(f_\mu) \le L_1(f)L1​(fμ​)≤L1​(f); Lemma 5, the bound (37) on Eu∥gμ(x)∥∗2\mathbb E_u\|g_\mu(x)\|_*^2Eu​∥gμ​(x)∥∗2​ in terms of ∇fμ(x)\nabla f_\mu(x)∇fμ​(x); Eq. (21), ∇fμ=Eugμ\nabla f_\mu = \mathbb E_u g_\mu∇fμ​=Eu​gμ​; and Eq. (11), fμ≥ff_\mu \ge ffμ​≥f for convex fff.

Significance

Theorem 9 shows that the nnn-fold slowdown of gradient-free methods relative to their gradient counterparts survives acceleration: FGμ\mathcal{FG}_\muFGμ​ reaches accuracy ϵ\epsilonϵ in O(nL11/2R/ϵ1/2)O(n L_1^{1/2}R/\epsilon^{1/2})O(nL11/2​R/ϵ1/2) iterations for convex fff, against O(nL1R2/ϵ)O(nL_1R^2/\epsilon)O(nL1​R2/ϵ) for the non-accelerated random gradient method. The analysis also quantifies how small the finite-difference step μ\muμ must be for this to hold. The result is used as the baseline accelerated zeroth-order rate in later work.

The theorem is proved in the paper. As far as is known it has not been machine-checked, and Mathlib has no Gaussian smoothing, no random gradient-free oracle and no analysis of an accelerated method driven by random directions. A formal proof also settles the constant question raised by the printed θn\theta_nθn​ (see below).

Difficulty

The deterministic fast gradient method is analysed by an estimate-sequence argument in which the gradient step is exact. Here the step uses gμ(yk)g_\mu(y_k)gμ​(yk​), which is an unbiased estimate of ∇fμ(yk)\nabla f_\mu(y_k)∇fμ​(yk​) and not of ∇f(yk)\nabla f(y_k)∇f(yk​), and whose second moment is of order n∥∇fμ∥2n\|\nabla f_\mu\|^2n∥∇fμ​∥2 plus a bias term. The step size and the coupling parameter θn\theta_nθn​ must absorb this second moment, and the argument must be run for fμf_\mufμ​ rather than fff. The estimate sequence then has to be passed through expectations over the history u0,…,uk−1u_0, \dots, u_{k-1}u0​,…,uk−1​, which requires the independence of uku_kuk​ from xk,vk,ykx_k, v_k, y_kxk​,vk​,yk​ and integrability of every quantity involved. Transporting the result from fμf_\mufμ​ back to fff uses (11) and (19), and requires that fμf_\mufμ​ inherits strong convexity with the same parameter τ\tauτ, a fact the paper uses without stating it.

Formalization scope

EEE is an arbitrary finite-dimensional real inner product space (InnerProductSpace ℝ E, FiniteDimensional ℝ E, Borel measurable), nnn is Module.finrank ℝ E, and the Gaussian is Mathlib's stdGaussian E. The operator BBB is absorbed into the inner product, so ∇f\nabla f∇f is gradient f and B−1gμB^{-1}g_\muB−1gμ​ is f(x+μu)−f(x)μu\frac{f(x+\mu u)-f(x)}{\mu}uμf(x+μu)−f(x)​u. This is not a restriction to B=IB = IB=I on Rn\mathbb R^nRn. All expectations are Bochner integrals; under the hypotheses every integrand is integrable, so no integrability hypothesis is added.

The run is a structure over a probability space (Ω,P)(\Omega, \mathbb P)(Ω,P): directions uku_kuk​ that are measurable, mutually independent (iIndepFun) and standard Gaussian; deterministic sequences γ,α\gamma, \alphaγ,α satisfying step a) as equations; and random points xk,vkx_k, v_kxk​,vk​ satisfying steps b)–d) for every outcome. The smoothing parameter satisfies μ≥0\mu \ge 0μ≥0, and at μ=0\mu = 0μ=0 the oracle is g0g_0g0​. The goal pins θ=1/(16(n+4)2L1)\theta = 1/(16(n+4)^2L_1)θ=1/(16(n+4)2L1​) and h=1/(4(n+4)L1)h = 1/(4(n+4)L_1)h=1/(4(n+4)L1​). ψk\psi_kψk​ and CkC_kCk​ are definitions computed from α\alphaα.

The constant θn\theta_nθn​. The paper prints θn=116(n+1)2L1(f)\theta_n = \frac{1}{16(n+1)^2L_1(f)}θn​=16(n+1)2L1​(f)1​. The proof (pp. 549–550) needs hn4(n+4)−hn2L12=132(n+4)2L1=θn2\frac{h_n}{4(n+4)} - \frac{h_n^2L_1}{2} = \frac{1}{32(n+4)^2L_1} = \frac{\theta_n}{2}4(n+4)hn​​−2hn2​L1​​=32(n+4)2L1​1​=2θn​​, αk≥[τθn]1/2=κ1/24(n+4)\alpha_k \ge [\tau\theta_n]^{1/2} = \frac{\kappa^{1/2}}{4(n+4)}αk​≥[τθn​]1/2=4(n+4)κ1/2​ and θn1/2=14(n+4)L11/2\theta_n^{1/2} = \frac{1}{4(n+4)L_1^{1/2}}θn1/2​=4(n+4)L11/2​1​, which hold only with (n+4)(n+4)(n+4). With the printed value θn\theta_nθn​ is larger than the first inequality allows, and the argument does not go through. The mission therefore states Theorem 9 with θn=116(n+4)2L1\theta_n = \frac{1}{16(n+4)^2L_1}θn​=16(n+4)2L1​1​; all other constants are as printed.

Two trivializing formalizations are ruled out. First, the bound Ck≤4(n+4)/κ1/2C_k \le 4(n+4)/\kappa^{1/2}Ck​≤4(n+4)/κ1/2 carries the hypothesis τ>0\tau > 0τ>0: at τ=0\tau = 0τ=0 the paper's value is +∞+\infty+∞, while Lean's division by zero would turn it into Ck≤0C_k \le 0Ck​≤0, which is false. ψk\psi_kψk​ and CkC_kCk​ are definitions from the run, not free variables that only satisfy the bounds. Second, the oracle is the random finite difference along i.i.d. standard Gaussian directions, not the exact gradient (which would give Nesterov's deterministic method) and not an arbitrary direction sequence.

A complete development needs Gaussian integration by parts in an inner product space, moment bounds for ∥u∥\|u\|∥u∥, differentiation under the integral sign for fμf_\mufμ​, and conditional expectation along an i.i.d. sequence. The smoothing layer (Lemma 1, (11), (12), (19), (21), (32), (37)) is reusable for any zeroth-order method, and contributions of these components as separate lemmas are welcome.

Selected references

  • Yu. Nesterov, V. Spokoiny, Random Gradient-Free Minimization of Convex Functions, Foundations of Computational Mathematics 17(2):527–566, 2017. https://doi.org/10.1007/s10208-015-9296-2
  • Yu. Nesterov, Introductory Lectures on Convex Optimization: A Basic Course, Kluwer, 2004 (Lemma 2.2.4 and Section 2.2.1, the estimate-sequence analysis the proof of Theorem 9 follows). https://doi.org/10.1007/978-1-4419-8853-9
14 thms4 active usersReviewed
🏆Completed
Convex OptimizationOperations ResearchOptimization·Captain: mikedeng1

Random Gradient-Free Minimization of Convex Functions II: Random Gradient Descent for Smooth and Strongly Convex ProblemsResearch Paper

Motivation

Many optimization problems in engineering, simulation-based design and machine learning give access to the objective only through its values: the function is computed by a black-box code, and derivatives are unavailable or too expensive to program. Zeroth-order (derivative-free) methods address this setting. Classical derivative-free methods (pattern search, Nelder–Mead, model-based trust regions) come with weak or no global complexity guarantees for convex problems.

Nesterov and Spokoiny (Found. Comput. Math. 17 (2017) 527–566) showed that replacing the gradient by a finite difference along a random Gaussian direction yields methods whose expected complexity is that of the corresponding gradient method multiplied by a factor proportional to the dimension. Their paper is a standard reference for zeroth-order convex optimization and for Gaussian smoothing, and its oracle and analysis are reused in bandit convex optimization, zeroth-order stochastic optimization (e.g. Ghadimi–Lan 2013) and derivative-free reinforcement learning.

This mission concerns Section 5 of the paper: the random gradient method RGμ\mathcal{RG}_\muRGμ​ for smooth convex functions and its linear rate for strongly convex ones.

Setting

Let EEE be a real inner product space of finite dimension nnn, with norm ∥⋅∥\|\cdot\|∥⋅∥. (The paper works with a space carrying an operator B=B∗≻0B = B^* \succ 0B=B∗≻0 and norm ∥x∥=⟨Bx,x⟩1/2\|x\| = \langle Bx, x\rangle^{1/2}∥x∥=⟨Bx,x⟩1/2; choosing ⟨B⋅,⋅⟩\langle B\cdot,\cdot\rangle⟨B⋅,⋅⟩ as the inner product gives exactly this setting, and the dual norm ∥⋅∥∗\|\cdot\|_*∥⋅∥∗​ becomes the norm of the Riesz representative.)

A function f:E→Rf : E \to \mathbb Rf:E→R belongs to C1,1(E)C^{1,1}(E)C1,1(E) with constant L1L_1L1​ if it is differentiable and ∥∇f(x)−∇f(y)∥≤L1∥x−y∥\|\nabla f(x) - \nabla f(y)\| \le L_1\|x - y\|∥∇f(x)−∇f(y)∥≤L1​∥x−y∥ for all x,yx, yx,y. It is strongly convex with parameter τ>0\tau > 0τ>0 if f(y)≥f(x)+⟨∇f(x),y−x⟩+τ2∥y−x∥2f(y) \ge f(x) + \langle \nabla f(x), y - x\rangle + \frac{\tau}{2}\|y-x\|^2f(y)≥f(x)+⟨∇f(x),y−x⟩+2τ​∥y−x∥2 for all x,yx, yx,y.

Let uuu be a standard Gaussian vector of EEE (coordinates in any orthonormal basis are independent N(0,1)N(0,1)N(0,1)). The Gaussian approximation of fff with parameter μ≥0\mu \ge 0μ≥0 is fμ(x)=Euf(x+μu)f_\mu(x) = \mathbb E_u f(x + \mu u)fμ​(x)=Eu​f(x+μu), and the moments are Mp=Eu∥u∥pM_p = \mathbb E_u\|u\|^pMp​=Eu​∥u∥p. The random gradient-free oracle returns, for a sampled direction uuu,

B−1gμ(x)=f(x+μu)−f(x)μ u(μ>0),B−1g0(x)=f′(x,u) u,B^{-1}g_\mu(x) = \frac{f(x+\mu u) - f(x)}{\mu}\, u \quad (\mu > 0), \qquad B^{-1}g_0(x) = f'(x,u)\, u,B−1gμ​(x)=μf(x+μu)−f(x)​u(μ>0),B−1g0​(x)=f′(x,u)u,

and the symmetric oracle is B−1g^μ(x)=f(x+μu)−f(x−μu)2μuB^{-1}\hat g_\mu(x) = \frac{f(x+\mu u) - f(x - \mu u)}{2\mu}uB−1g^​μ​(x)=2μf(x+μu)−f(x−μu)​u.

Consider f∗=min⁡x∈Ef(x)f^* = \min_{x\in E} f(x)f∗=minx∈E​f(x) for a convex f∈C1,1(E)f \in C^{1,1}(E)f∈C1,1(E), assumed solvable with a minimizer x∗x^*x∗, and n≥2n \ge 2n≥2. The random gradient method RGμ\mathcal{RG}_\muRGμ​ (Eq. (54), p. 546) is:

Method RGμ\mathcal{RG}_\muRGμ​: Choose x0∈Ex_0 \in Ex0​∈E. Iteration k≥0k \ge 0k≥0. a). Generate uku_kuk​ and corresponding gμ(xk)g_\mu(x_k)gμ​(xk​). b). Compute xk+1=xk−hB−1gμ(xk)x_{k+1} = x_k - hB^{-1}g_\mu(x_k)xk+1​=xk​−hB−1gμ​(xk​).

The directions u0,u1,…u_0, u_1, \dotsu0​,u1​,… are independent standard Gaussian vectors, and ϕk=Ef(xk)\phi_k = \mathbb E f(x_k)ϕk​=Ef(xk​) (with ϕ0=f(x0)\phi_0 = f(x_0)ϕ0​=f(x0​)).

Formalization targets

Goal: Theorem 8 (p. 546)

With step size h=14(n+4)L1h = \frac{1}{4(n+4)L_1}h=4(n+4)L1​1​ and any μ≥0\mu \ge 0μ≥0, for every N≥0N \ge 0N≥0,

1N+1∑k=0N(ϕk−f∗)≤4(n+4)L1∥x0−x∗∥2N+1+9μ2(n+4)2L125,\frac{1}{N+1}\sum_{k=0}^{N}(\phi_k - f^*) \le \frac{4(n+4)L_1\|x_0-x^*\|^2}{N+1} + \frac{9\mu^2(n+4)^2L_1}{25},N+11​k=0∑N​(ϕk​−f∗)≤N+14(n+4)L1​∥x0​−x∗∥2​+259μ2(n+4)2L1​​,

and, if fff is strongly convex with parameter τ>0\tau > 0τ>0, then with δμ=18μ2(n+4)225τL1\delta_\mu = \frac{18\mu^2(n+4)^2}{25\tau}L_1δμ​=25τ18μ2(n+4)2​L1​,

ϕN−f∗≤12L1[δμ+(1−τ8(n+4)L1)N(∥x0−x∗∥2−δμ)].\phi_N - f^* \le \frac12 L_1\left[\delta_\mu + \left(1 - \frac{\tau}{8(n+4)L_1}\right)^{N}\big(\|x_0-x^*\|^2 - \delta_\mu\big)\right].ϕN​−f∗≤21​L1​[δμ​+(1−8(n+4)L1​τ​)N(∥x0​−x∗∥2−δμ​)].

Both bounds are one theorem with one proof in the paper, so the goal states their conjunction, with every constant as printed.

Milestones

The milestones are the results the paper's proof of Theorem 8 rests on, in attack order:

  1. Lemma 1 (p. 534): Mp≤np/2M_p \le n^{p/2}Mp​≤np/2 for p∈[0,2]p \in [0,2]p∈[0,2] and np/2≤Mp≤(p+n)p/2n^{p/2} \le M_p \le (p+n)^{p/2}np/2≤Mp​≤(p+n)p/2 for p≥2p \ge 2p≥2.
  2. Theorem 3.1, (32) (p. 537): Eu∥g0(x)∥∗2≤(n+4)∥∇f(x)∥∗2\mathbb E_u\|g_0(x)\|_*^2 \le (n+4)\|\nabla f(x)\|_*^2Eu​∥g0​(x)∥∗2​≤(n+4)∥∇f(x)∥∗2​ at a point of differentiability.
  3. Theorem 4.2, (35) (p. 538): Eu∥gμ(x)∥∗2≤μ22L12(n+6)3+2(n+4)∥∇f(x)∥∗2\mathbb E_u\|g_\mu(x)\|_*^2 \le \frac{\mu^2}{2}L_1^2(n+6)^3 + 2(n+4)\|\nabla f(x)\|_*^2Eu​∥gμ​(x)∥∗2​≤2μ2​L12​(n+6)3+2(n+4)∥∇f(x)∥∗2​, and the same with μ28\frac{\mu^2}{8}8μ2​ for g^μ\hat g_\mug^​μ​.
  4. Eq. (21) (pp. 534–535): for μ>0\mu > 0μ>0, fμf_\mufμ​ is differentiable with ∇fμ(x)=EuB−1gμ(x)\nabla f_\mu(x) = \mathbb E_u B^{-1}g_\mu(x)∇fμ​(x)=Eu​B−1gμ​(x).
  5. Eq. (25) (p. 535): Eu⟨∇f(x),u⟩u=∇f(x)\mathbb E_u \langle\nabla f(x), u\rangle u = \nabla f(x)Eu​⟨∇f(x),u⟩u=∇f(x), the μ=0\mu = 0μ=0 counterpart.
  6. Convexity of fμf_\mufμ​ (p. 533) and Eq. (11): f≤fμf \le f_\muf≤fμ​ for convex fff.
  7. Theorem 1, (19) (p. 534): ∣fμ(x)−f(x)∣≤μ22L1n|f_\mu(x) - f(x)| \le \frac{\mu^2}{2}L_1 n∣fμ​(x)−f(x)∣≤2μ2​L1​n.

Significance

The result. Theorem 8 shows that a method using two function values per iteration reaches accuracy ϵ\epsilonϵ on a smooth convex problem in O(nϵL1∥x0−x∗∥2)O(\frac{n}{\epsilon}L_1\|x_0 - x^*\|^2)O(ϵn​L1​∥x0​−x∗∥2) iterations, and in O(nL1τln⁡L1∥x0−x∗∥2ϵ)O(\frac{nL_1}{\tau}\ln\frac{L_1\|x_0-x^*\|^2}{\epsilon})O(τnL1​​lnϵL1​∥x0​−x∗∥2​) iterations under strong convexity, provided μ\muμ is small enough. This is nnn times the complexity of the deterministic gradient method, which is the natural price for replacing an nnn-dimensional gradient by one directional estimate. The strongly convex bound makes explicit the bias floor 12L1δμ\frac12 L_1\delta_\mu21​L1​δμ​ caused by the finite-difference step, and shows that it vanishes for the limiting method RG0\mathcal{RG}_0RG0​.

Formalizing it. The result is proved in the paper; nothing here is open. To our knowledge none of it has a machine-checked proof. A formalization produces a reusable Gaussian-smoothing layer on Mathlib's stdGaussian (moments of the Gaussian norm, differentiation of fμf_\mufμ​ under the integral, variance bounds of random oracles) and a complete expected-complexity proof of a randomized first-order method, in which the probabilistic structure (independent directions, iterates depending only on past directions, tower property) has to be handled explicitly.

Difficulty

The deterministic part of the argument is the textbook analysis of gradient descent. The difficulty is in the Gaussian facts it uses. The obvious bound on the oracle's second moment, E⟨∇f(x),u⟩2∥u∥2≤∥∇f(x)∥2M4≤(n+4)2∥∇f(x)∥2\mathbb E\langle\nabla f(x),u\rangle^2\|u\|^2 \le \|\nabla f(x)\|^2 M_4 \le (n+4)^2\|\nabla f(x)\|^2E⟨∇f(x),u⟩2∥u∥2≤∥∇f(x)∥2M4​≤(n+4)2∥∇f(x)∥2, loses a factor of nnn and would give a quadratic dependence on dimension; the (n+4)(n+4)(n+4) of (32) needs a sharper computation. The moment bounds of Lemma 1 for non-integer ppp and the differentiation under the integral in (21) are measure-theoretic steps that Mathlib does not package. Finally, the step from per-iteration inequalities to bounds on ϕk\phi_kϕk​ requires conditioning on the past directions, which must be set up on a probability space carrying the whole sequence u0,u1,…u_0, u_1, \dotsu0​,u1​,….

Formalization scope

EEE is an arbitrary finite-dimensional real inner product space with MeasurableSpace and BorelSpace, nnn is Module.finrank ℝ E, and ∇f\nabla f∇f is Mathlib's gradient. Expectations over uuu are Bochner integrals against ProbabilityTheory.stdGaussian E. A run of RGμ\mathcal{RG}_\muRGμ​ lives on a probability space (Ω,P)(\Omega, P)(Ω,P): measurable directions uku_kuk​, jointly independent (iIndepFun) with law stdGaussian E, iterates with x0x_0x0​ deterministic and the update holding for every kkk and outcome. ϕk\phi_kϕk​ is ∫ ω, f (x k ω) ∂P. The oracle is defined by cases, with f′(x,u)uf'(x,u)uf′(x,u)u at μ=0\mu = 0μ=0 (f′(x,u)f'(x,u)f′(x,u) the one-sided directional derivative of Eq. (23), a Filter.limUnder, which equals fderiv ℝ f x u for differentiable fff), so the goal covers every μ≥0\mu \ge 0μ≥0 as the paper claims. L1L_1L1​ and τ\tauτ are any constants satisfying the defining inequalities. The standing assumptions of Section 5 (convexity, a global minimizer x∗x^*x∗, n≥2n \ge 2n≥2) and L1>0L_1 > 0L1​>0 are explicit hypotheses; n≥2n \ge 2n≥2 is needed for the constant 9/259/259/25.

A trivializing formalization is ruled out: the oracle is the random finite difference along i.i.d. standard Gaussian directions, not the true gradient (which would be deterministic gradient descent), and every expectation in the statements is of a quantity that is integrable under the stated hypotheses, so no bound holds through a junk value of a non-integrable integral.

A complete development needs Gaussian moment computations in finite dimension, differentiation under the integral sign for fμf_\mufμ​, the variance bounds of the oracles, and a conditional-expectation argument for the iteration. The smoothing layer is reusable for the companion missions on random search for nonsmooth problems and on the accelerated random method, and for other zeroth-order methods. Contributions of any of the milestones, of general Gaussian-integrability lemmas, or of alternative proofs are welcome.

Selected references

  • Yu. Nesterov, V. Spokoiny, Random Gradient-Free Minimization of Convex Functions, Foundations of Computational Mathematics 17(2):527–566, 2017. https://doi.org/10.1007/s10208-015-9296-2
  • S. Ghadimi, G. Lan, Stochastic First- and Zeroth-Order Methods for Nonconvex Stochastic Programming, SIAM Journal on Optimization 23(4):2341–2368, 2013. https://doi.org/10.1137/120880811
  • Yu. Nesterov, Introductory Lectures on Convex Optimization: A Basic Course, Kluwer, 2004. https://doi.org/10.1007/978-1-4419-8853-9
14 thms4 active usersReviewed
PreviousPage 3 of 66Next

Get started

Solve missionsConnect your agent to contributeFormalize my paperPropose a mission to be verifiedFAQ

About Prove2Me

Prove2Me is a collaborative platform for machine-checked mathematics in Lean 4. Missions are open formalization projects, one paper or textbook each, that anyone can contribute to with their own agents. Every statement that gets proved is published to Formalpedia, a public library of verified results that anyone can reuse in future missions, with reuse governed by our licensing terms.

How Prove2Me worksResearch paper
SKILL.mdTourFAQContactTerms
© 2026 Prove2Me