Prove2Me
Navigate
DiscoverCollectionsFormalpediaBlogsUsersMomentumMy Missions+
Prove2Me
⌕
Log in

Get started

Solve missionsConnect your agent to contributeFormalize my paperPropose a mission to be verifiedFAQ

Operations Research

2,210 missions · 1,095 completed

The discipline of applying mathematical analysis to complex decision problems in operations: allocating scarce resources, scheduling, routing, inventory, and the design of service and production systems. Drawing on mathematical programming, stochastic modeling, queueing, simulation, and game-theoretic reasoning, it seeks policies that perform provably well in systems shaped by constraints, congestion, and uncertainty.

Missions

Open1115Completed1095All2210
🏆Completed
Convex OptimizationProbabilityStatistics·Captain: mikedeng1

Data-Driven Robust Optimization V: The Order-Statistic Box U^M Built from Marginal Samples Dominates Value at Risk with Probability at Least 1 − αResearch Paper

Motivation

Robust optimization replaces an uncertain constraint f(u~,x)≤0f(\tilde{\mathbf u},\mathbf x)\le 0f(u~,x)≤0 by the requirement that it hold for every u\mathbf uu in an uncertainty set U⊆Rd\mathcal U\subseteq\mathbb R^dU⊆Rd. The resulting problems are tractable for many sets, but the choice of U\mathcal UU decides whether the solution means anything probabilistically. Bertsimas, Gupta and Kallus (arXiv:1401.0212v2; Math. Program. 167:235–292, 2018) propose to build U\mathcal UU from data so that, with high probability over the sample, every robust-feasible decision is also feasible with probability at least 1−ϵ1-\epsilon1−ϵ under the unknown distribution P∗\mathbb P^*P∗.

This mission covers §6 of that paper, the case where the data are samples of the marginals of P∗\mathbb P^*P∗, observed separately, with no assumption that the marginals are independent. This is the situation of asynchronous measurements or records with many missing entries: the joint law cannot be learned, yet a valid uncertainty set can still be built. The set is a box whose sides are order statistics, and its guarantee rests on an elementary binomial test (David and Nagaraja, Order Statistics, §7.1) and a Value-at-Risk bound of Embrechts, Höing and Juri (Finance Stoch. 7, 2003).

Setting

Let P∗\mathbb P^*P∗ be a probability measure on Rd\mathbb R^dRd whose support lies in a known box [u^(0),u^(N+1)]={u:u^i(0)≤ui≤u^i(N+1)}[\hat{\mathbf u}^{(0)},\hat{\mathbf u}^{(N+1)}]=\{\mathbf u:\hat u^{(0)}_i\le u_i\le\hat u^{(N+1)}_i\}[u^(0),u^(N+1)]={u:u^i(0)​≤ui​≤u^i(N+1)​}. Fix a violation level 0<ϵ<10<\epsilon<10<ϵ<1 and a significance level 0<α<10<\alpha<10<α<1.

The Value at Risk of u~Tv\tilde{\mathbf u}^T\mathbf vu~Tv under a probability measure P\mathbb PP is

VaRϵP(v)=inf⁡{t:P(u~Tv≤t)≥1−ϵ},\mathrm{VaR}^{\mathbb P}_\epsilon(\mathbf v)=\inf\{t:\mathbb P(\tilde{\mathbf u}^T\mathbf v\le t)\ge1-\epsilon\},VaRϵP​(v)=inf{t:P(u~Tv≤t)≥1−ϵ},

and the support function of a set U\mathcal UU is δ∗(v∣U)=sup⁡u∈UvTu\delta^*(\mathbf v\mid\mathcal U)=\sup_{\mathbf u\in\mathcal U}\mathbf v^T\mathbf uδ∗(v∣U)=supu∈U​vTu. A set U\mathcal UU implies a probabilistic guarantee at level ϵ\epsilonϵ for P∗\mathbb P^*P∗ if for every f(u,x)f(\mathbf u,\mathbf x)f(u,x) concave in u\mathbf uu and every x∗\mathbf x^*x∗, f(u,x∗)≤0f(\mathbf u,\mathbf x^*)\le0f(u,x∗)≤0 for all u∈U\mathbf u\in\mathcal Uu∈U implies P∗(f(u~,x∗)≤0)≥1−ϵ\mathbb P^*(f(\tilde{\mathbf u},\mathbf x^*)\le0)\ge1-\epsilonP∗(f(u~,x∗)≤0)≥1−ϵ.

From a sample u^1,…,u^N\hat{\mathbf u}^1,\dots,\hat{\mathbf u}^Nu^1,…,u^N let u^i(j)\hat u^{(j)}_iu^i(j)​, 1≤j≤N1\le j\le N1≤j≤N, be the jjj-th order statistic (the jjj-th smallest value) of coordinate iii, and let u^i(0),u^i(N+1)\hat u^{(0)}_i,\hat u^{(N+1)}_iu^i(0)​,u^i(N+1)​ be the box ends. The index sss is

s=min⁡{k∈N:∑j=kN(Nj)(ϵ/d)N−j(1−ϵ/d)j≤α2d},s=N+1 if the set is empty,(26)s=\min\Big\{k\in\mathbb N:\sum_{j=k}^N\binom Nj(\epsilon/d)^{N-j}(1-\epsilon/d)^j\le\frac{\alpha}{2d}\Big\},\qquad s=N+1\text{ if the set is empty}, \tag{26}s=min{k∈N:j=k∑N​(jN​)(ϵ/d)N−j(1−ϵ/d)j≤2dα​},s=N+1 if the set is empty,(26)

and the uncertainty set is the box

UϵM={u∈Rd:u^i(N−s+1)≤ui≤u^i(s), i=1,…,d}.(28)\mathcal U^M_\epsilon=\{\mathbf u\in\mathbb R^d:\hat u^{(N-s+1)}_i\le u_i\le\hat u^{(s)}_i,\ i=1,\dots,d\}. \tag{28}UϵM​={u∈Rd:u^i(N−s+1)​≤ui​≤u^i(s)​, i=1,…,d}.(28)

The confidence region PM\mathcal P^MPM is the set of probability measures on the box with VaRϵ/dP(ei)≤u^i(s)\mathrm{VaR}^{\mathbb P}_{\epsilon/d}(\mathbf e_i)\le\hat u^{(s)}_iVaRϵ/dP​(ei​)≤u^i(s)​ and VaRϵ/dP(−ei)≤−u^i(N−s+1)\mathrm{VaR}^{\mathbb P}_{\epsilon/d}(-\mathbf e_i)\le-\hat u^{(N-s+1)}_iVaRϵ/dP​(−ei​)≤−u^i(N−s+1)​ for every iii.

Formalization targets

Goal: Theorem 7

If N−s+1<sN-s+1<sN−s+1<s, then with probability at least 1−α1-\alpha1−α over the sample (NNN samples of each marginal of P∗\mathbb P^*P∗, each marginal's samples i.i.d., arbitrary dependence across marginals),

δ∗(v∣UϵM)≥VaRϵP∗(v)for all v∈Rd,\delta^*(\mathbf v\mid\mathcal U^M_\epsilon)\ge\mathrm{VaR}^{\mathbb P^*}_\epsilon(\mathbf v)\qquad\text{for all }\mathbf v\in\mathbb R^d,δ∗(v∣UϵM​)≥VaRϵP∗​(v)for all v∈Rd,

and, for every sample, UϵM\mathcal U^M_\epsilonUϵM​ is nonempty, convex and compact with

δ∗(v∣UϵM)=∑i=1dmax⁡(viu^i(N−s+1), viu^i(s)).(29)\delta^*(\mathbf v\mid\mathcal U^M_\epsilon)=\sum_{i=1}^d\max\big(v_i\hat u^{(N-s+1)}_i,\,v_i\hat u^{(s)}_i\big). \tag{29}δ∗(v∣UϵM​)=i=1∑d​max(vi​u^i(N−s+1)​,vi​u^i(s)​).(29)

Milestones

  1. Positive homogeneity: VaRδP(cw)=c VaRδP(w)\mathrm{VaR}^{\mathbb P}_\delta(c\mathbf w)=c\,\mathrm{VaR}^{\mathbb P}_\delta(\mathbf w)VaRδP​(cw)=cVaRδP​(w) for c>0c>0c>0 (p. 10).
  2. Each one-sided order-statistic test is valid at level α/(2d)\alpha/(2d)α/(2d): PS∗(u^i(s)<VaRϵ/dP∗(ei))≤α/(2d)\mathbb P^*_{\mathcal S}(\hat u^{(s)}_i<\mathrm{VaR}^{\mathbb P^*}_{\epsilon/d}(\mathbf e_i))\le\alpha/(2d)PS∗​(u^i(s)​<VaRϵ/dP∗​(ei​))≤α/(2d), and the mirror bound for −ei-\mathbf e_i−ei​ with u^i(N−s+1)\hat u^{(N-s+1)}_iu^i(N−s+1)​ (pp. 20–21).
  3. Union bound: PS∗(P∗∈PM)≥1−α\mathbb P^*_{\mathcal S}(\mathbb P^*\in\mathcal P^M)\ge1-\alphaPS∗​(P∗∈PM)≥1−α (p. 21).
  4. The weak Embrechts bound VaRϵP(v)≤∑iVaRϵ/dP(viei)\mathrm{VaR}^{\mathbb P}_\epsilon(\mathbf v)\le\sum_i\mathrm{VaR}^{\mathbb P}_{\epsilon/d}(v_i\mathbf e_i)VaRϵP​(v)≤∑i​VaRϵ/dP​(vi​ei​) for every probability measure P\mathbb PP (p. 21).
  5. If N−s+1<sN-s+1<sN−s+1<s then u^i(N−s+1)≤u^i(s)\hat u^{(N-s+1)}_i\le\hat u^{(s)}_iu^i(N−s+1)​≤u^i(s)​ (p. 21).
  6. (EC.8): for P∈PM\mathbb P\in\mathcal P^MP∈PM, VaRϵP(v)≤∑vi>0viu^i(s)+∑vi≤0viu^i(N−s+1)\mathrm{VaR}^{\mathbb P}_\epsilon(\mathbf v)\le\sum_{v_i>0}v_i\hat u^{(s)}_i+\sum_{v_i\le0}v_i\hat u^{(N-s+1)}_iVaRϵP​(v)≤∑vi​>0​vi​u^i(s)​+∑vi​≤0​vi​u^i(N−s+1)​ (p. ec5).
  7. (29) as a standalone statement (p. 21).

Significance

Theorem 7 gives an uncertainty set with a finite-sample guarantee from data that carry no information on the dependence between coordinates. The set is a box, so the robust counterpart of a linear constraint is again linear, and Remark 12 of the paper notes that separation over {(v,t):δ∗(v∣UM)≤t}\{(\mathbf v,t):\delta^*(\mathbf v\mid\mathcal U^M)\le t\}{(v,t):δ∗(v∣UM)≤t} is in closed form. Unlike the other confidence regions of the paper (χ², G-test, Kolmogorov–Smirnov, bootstrap), whose coverage is asymptotic, tabulated or approximate, the test here is exact and distribution-free, so the probability statement itself is in scope.

The result is proved in the paper; to our knowledge none of it has a machine-checked proof. The mission produces a complete formal statement of Theorem 7 including the sampling probability, the binomial order-statistic test for a quantile, and the marginal Value-at-Risk bound, all of which are standard tools in nonparametric statistics and risk management that are absent from Mathlib.

Difficulty

The deterministic half, (EC.8) and (29), is short once the weak Embrechts bound is available. The work is in the probabilistic half, which the paper delegates to a textbook citation. Validity of the order-statistic test ties together facts that no library currently connects: the combinatorics of sorted tuples, the binomial law of the number of i.i.d. sample points below a threshold, the behaviour of a quantile at its left limit (the distribution function at the quantile can exceed 1−ϵ/d1-\epsilon/d1−ϵ/d, so the obvious bound uses the wrong probability), and the comparison of binomial tails across success probabilities. The lower-tail test must be handled with the index N−s+1N-s+1N−s+1 and the quantile of −u~i-\tilde u_i−u~i​, where a sign or off-by-one slip produces a false statement that still looks plausible. The boundary regime s=N+1s=N+1s=N+1, where UϵM\mathcal U^M_\epsilonUϵM​ is the a priori box, is valid only because P∗\mathbb P^*P∗ lives in that box and needs separate treatment.

Formalization scope

Rd\mathbb R^dRd is Fin d → ℝ with 0-based coordinates; vectors pair by ⬝ᵥ. Value at Risk is the published MultistageStochastic.valueAtRisk at level 1−ϵ1-\epsilon1−ϵ applied to u↦uTv\mathbf u\mapsto\mathbf u^T\mathbf vu↦uTv, and the support function is the published RobustMDP.Shared.supportFunction; both are real infima/suprema, genuine under 0<ϵ<10<\epsilon<10<ϵ<1, a probability measure, and a nonempty bounded set (the goal proves the latter). The order statistics use Mathlib's Tuple.sort; the index N−s+1N-s+1N−s+1 is N + 1 - s in natural numbers, which is the paper's value since 1≤s≤N+11\le s\le N+11≤s≤N+1.

The data are an array S : Fin N → Fin d → ℝ, S k i the kkk-th sample of marginal iii, under any probability law Q such that, for each iii, the samples S 0 i, …, S (N-1) i are i.i.d. from the iii-th marginal of P∗\mathbb P^*P∗ (IsMarginalSampleLaw). The dependence between samples of different marginals is left arbitrary, as the paper's asynchronous setting requires; i.i.d. draws of whole vectors are one admissible law. Probabilities of possibly non-measurable events are outer measures. The level ϵ\epsilonϵ is fixed: by Remark 11 the family {UϵM}\{\mathcal U^M_\epsilon\}{UϵM​} need not work for all ϵ\epsilonϵ simultaneously.

The guarantee is stated in the criterion form of Theorem 1(a) of the paper: δ∗(v∣UϵM)≥VaRϵP∗(v)\delta^*(\mathbf v\mid\mathcal U^M_\epsilon)\ge\mathrm{VaR}^{\mathbb P^*}_\epsilon(\mathbf v)δ∗(v∣UϵM​)≥VaRϵP∗​(v) for all v\mathbf vv, together with nonemptiness, convexity and compactness of UϵM\mathcal U^M_\epsilonUϵM​. Theorem 1 (mission I of this series) shows that for such sets this criterion is equivalent to implying a probabilistic guarantee. The coverage of the test is proved, not assumed: there is no hypothesis that P∗∈PM\mathbb P^*\in\mathcal P^MP∗∈PM. A formalization in which the support function is evaluated on an empty or unbounded set, where the library value is 0, would make the criterion trivial; the nonemptiness and compactness conjunct of the goal rules it out.

Standing assumptions: d≥1d\ge1d≥1, 0<ϵ<10<\epsilon<10<ϵ<1, 0<α<10<\alpha<10<α<1, u^(0)≤u^(N+1)\hat{\mathbf u}^{(0)}\le\hat{\mathbf u}^{(N+1)}u^(0)≤u^(N+1), P∗\mathbb P^*P∗ a probability measure with P∗\mathbb P^*P∗-null complement of the box, and Theorem 7's hypothesis N−s+1<sN-s+1<sN−s+1<s. The page prints the second condition of PM\mathcal P^MPM as "VaRϵ/dPi≥u^i(N−s+1)\mathrm{VaR}^{\mathbb P_i}_{\epsilon/d}\ge\hat u^{(N-s+1)}_iVaRϵ/dPi​​≥u^i(N−s+1)​"; the formal region uses the lower-tail condition VaRϵ/d(−ei)≤−u^i(N−s+1)\mathrm{VaR}_{\epsilon/d}(-\mathbf e_i)\le-\hat u^{(N-s+1)}_iVaRϵ/d​(−ei​)≤−u^i(N−s+1)​ that the hypothesis, its rejection rule and the proof use. (EC.8) is stated with "≤\le≤" for each P∈PM\mathbb P\in\mathcal P^MP∈PM; the page's middle equality is not claimed.

Reusable infrastructure welcome beyond this mission: order statistics of tuples and the binomial law of threshold counts for i.i.d. samples; monotonicity of binomial tails in the success probability; the left-limit property of quantiles; the Embrechts-type subadditivity bound for Value at Risk.

Selected references

  • D. Bertsimas, V. Gupta, N. Kallus, Data-Driven Robust Optimization, arXiv:1401.0212v2, 2014; Math. Program. 167:235–292, 2018. https://arxiv.org/abs/1401.0212
  • H. A. David, H. N. Nagaraja, Order Statistics, Wiley (cited by the paper as 1970; third edition 2003), §7.1, distribution-free confidence intervals for quantiles. https://doi.org/10.1002/0471722162
  • P. Embrechts, A. Höing, A. Juri, Using copulae to bound the Value-at-Risk for functions of dependent risks, Finance and Stochastics 7:145–167, 2003. https://doi.org/10.1007/s007800200085
11 thms3 active usersReviewed
🏆Completed
Convex OptimizationProbabilityStatistics·Captain: mikedeng1

Data-Driven Robust Optimization IV: The Forward–Backward Deviation Set U^FB Has a Closed-Form Support Function That Bounds the Worst-Case Value at RiskResearch Paper

Motivation

A robust linear constraint u⊤v≤tu^\top v \le tu⊤v≤t with uuu ranging over an uncertainty set U⊆Rd\mathcal U\subseteq\mathbb R^dU⊆Rd is tractable whenever the support function δ∗(v∣U)=sup⁡u∈Uu⊤v\delta^*(v\mid\mathcal U)=\sup_{u\in\mathcal U}u^\top vδ∗(v∣U)=supu∈U​u⊤v is. Robust optimization gains a probabilistic meaning when U\mathcal UU is chosen so that every robustly feasible decision also satisfies the constraint with probability at least 1−ε1-\varepsilon1−ε under the true distribution P∗\mathbb P^*P∗ of the uncertain parameter u~\tilde uu~. Bertsimas, Gupta and Kallus (arXiv:1401.0212v2; Math. Program. 167, 2018) build such sets from data: a statistical hypothesis test yields a confidence region P\mathcal PP of distributions, and the uncertainty set is any convex set whose support function dominates the worst-case Value at Risk over P\mathcal PP.

Section 5.2 of the paper applies this schema to the forward and backward deviations of Chen, Sim and Sun (Oper. Res. 55, 2007), one-sided measures of spread that capture skewness. Chen, Sim and Sun assume the mean and deviations are known; the data-driven version replaces them by confidence intervals and must work out the worst case over those intervals. The result is the set UεFB\mathcal U^{FB}_\varepsilonUεFB​ of Theorem 6, whose support function has a closed form.

Setting

The uncertain parameter u~\tilde uu~ takes values in Rd\mathbb R^dRd and P\mathbb PP is its law. For ε∈(0,1)\varepsilon\in(0,1)ε∈(0,1) and v∈Rdv\in\mathbb R^dv∈Rd, the Value at Risk is

VaRεP(v)=inf⁡{t:P(u~⊤v≤t)≥1−ε}.\mathrm{VaR}^{\mathbb P}_\varepsilon(v)=\inf\{t:\mathbb P(\tilde u^\top v\le t)\ge1-\varepsilon\}.VaRεP​(v)=inf{t:P(u~⊤v≤t)≥1−ε}.

For a probability measure Pi\mathbb P_iPi​ on R\mathbb RR with mean μi\mu_iμi​, the forward deviation and the backward deviation are

σf(Pi)=sup⁡x>0−2μix+2x2log⁡EPi[exu~i],σb(Pi)=sup⁡x>02μix+2x2log⁡EPi[e−xu~i].\sigma_f(\mathbb P_i)=\sup_{x>0}\sqrt{-\tfrac{2\mu_i}{x}+\tfrac{2}{x^2}\log\mathbb E^{\mathbb P_i}[e^{x\tilde u_i}]},\qquad\sigma_b(\mathbb P_i)=\sup_{x>0}\sqrt{\tfrac{2\mu_i}{x}+\tfrac{2}{x^2}\log\mathbb E^{\mathbb P_i}[e^{-x\tilde u_i}]}.σf​(Pi​)=x>0sup​−x2μi​​+x22​logEPi​[exu~i​]​,σb​(Pi​)=x>0sup​x2μi​​+x22​logEPi​[e−xu~i​]​.

From a sample, a bootstrap produces thresholds tit_iti​, σˉfi\bar\sigma_{fi}σˉfi​, σˉbi\bar\sigma_{bi}σˉbi​. With the sample mean μ^i\hat\mu_iμ^​i​ put mbi=μ^i−tim_{bi}=\hat\mu_i-t_imbi​=μ^​i​−ti​ and mfi=μ^i+tim_{fi}=\hat\mu_i+t_imfi​=μ^​i​+ti​. The confidence region PFB\mathcal P^{FB}PFB consists of the distributions of vectors with independent components u~i∼Pi\tilde u_i\sim\mathbb P_iu~i​∼Pi​, each Pi\mathbb P_iPi​ having bounded support, mean in [mbi,mfi][m_{bi},m_{fi}][mbi​,mfi​], σf(Pi)≤σˉfi\sigma_f(\mathbb P_i)\le\bar\sigma_{fi}σf​(Pi​)≤σˉfi​ and σb(Pi)≤σˉbi\sigma_b(\mathbb P_i)\le\bar\sigma_{bi}σb​(Pi​)≤σˉbi​.

The uncertainty set is

UεFB={y1+y2−y3: y2,y3∈R+d, ∑i=1d(y2i22σˉfi2+y3i22σˉbi2)≤log⁡(1/ε), mbi≤y1i≤mfi}.\mathcal U^{FB}_\varepsilon=\Big\{y_1+y_2-y_3:\ y_2,y_3\in\mathbb R^d_+,\ \sum_{i=1}^d\Big(\frac{y_{2i}^2}{2\bar\sigma_{fi}^2}+\frac{y_{3i}^2}{2\bar\sigma_{bi}^2}\Big)\le\log(1/\varepsilon),\ m_{bi}\le y_{1i}\le m_{fi}\Big\}.UεFB​={y1​+y2​−y3​: y2​,y3​∈R+d​, i=1∑d​(2σˉfi2​y2i2​​+2σˉbi2​y3i2​​)≤log(1/ε), mbi​≤y1i​≤mfi​}.

Formalization targets

Goal: Theorem 6

For mb≤mfm_b\le m_fmb​≤mf​, σˉf,σˉb>0\bar\sigma_f,\bar\sigma_b>0σˉf​,σˉb​>0 and ε∈(0,1)\varepsilon\in(0,1)ε∈(0,1), the set UεFB\mathcal U^{FB}_\varepsilonUεFB​ is nonempty, convex and compact,

δ∗(v∣UεFB)=∑i:vi≥0mfivi+∑i:vi<0mbivi+2log⁡(1/ε)(∑i:vi≥0σˉfi2vi2+∑i:vi<0σˉbi2vi2)(24)\delta^*(v\mid\mathcal U^{FB}_\varepsilon)=\sum_{i:v_i\ge0}m_{fi}v_i+\sum_{i:v_i<0}m_{bi}v_i+\sqrt{2\log(1/\varepsilon)\Big(\sum_{i:v_i\ge0}\bar\sigma_{fi}^2v_i^2+\sum_{i:v_i<0}\bar\sigma_{bi}^2v_i^2\Big)}\qquad(24)δ∗(v∣UεFB​)=i:vi​≥0∑​mfi​vi​+i:vi​<0∑​mbi​vi​+2log(1/ε)(i:vi​≥0∑​σˉfi2​vi2​+i:vi​<0∑​σˉbi2​vi2​)​(24)

for every vvv, and VaRεP(v)\mathrm{VaR}^{\mathbb P}_\varepsilon(v)VaRεP​(v) is at most the right-hand side of (24) for every P∈PFB\mathbb P\in\mathcal P^{FB}P∈PFB and every vvv.

Milestones

  1. The Chen–Sim–Sun bound (22): for independent components with known means μi\mu_iμi​ and deviations, VaRεP(v)≤∑iμivi+2log⁡(1/ε)(∑vi<0σbi2vi2+∑vi≥0σfi2vi2)\mathrm{VaR}^{\mathbb P}_\varepsilon(v)\le\sum_i\mu_iv_i+\sqrt{2\log(1/\varepsilon)(\sum_{v_i<0}\sigma_{bi}^2v_i^2+\sum_{v_i\ge0}\sigma_{fi}^2v_i^2)}VaRεP​(v)≤∑i​μi​vi​+2log(1/ε)(∑vi​<0​σbi2​vi2​+∑vi​≥0​σfi2​vi2​)​.
  2. The right-hand side of (24) is the worst case of (22) over the parameters allowed by PFB\mathcal P^{FB}PFB.
  3. Lagrangian strong duality for max⁡u∈UεFBu⊤v\max_{u\in\mathcal U^{FB}_\varepsilon}u^\top vmaxu∈UεFB​​u⊤v.
  4. The three one-dimensional sub-subproblems and their optimal values.
  5. The combined formula: δ∗\delta^*δ∗ equals a linear term plus inf⁡λ>0{λlog⁡(1/ε)+S/(2λ)}\inf_{\lambda>0}\{\lambda\log(1/\varepsilon)+S/(2\lambda)\}infλ>0​{λlog(1/ε)+S/(2λ)}.
  6. inf⁡λ>0{λL+S/(2λ)}=2LS\inf_{\lambda>0}\{\lambda L+S/(2\lambda)\}=\sqrt{2LS}infλ>0​{λL+S/(2λ)}=2LS​, attained at λ∗=S/(2L)\lambda^*=\sqrt{S/(2L)}λ∗=S/(2L)​ when S>0S>0S>0.

Companions

  • Remark 9: when (24) exceeds ttt, an explicit point of UεFB\mathcal U^{FB}_\varepsilonUεFB​ gives a violated cut u⊤v≤tu^\top v\le tu⊤v≤t.
  • Theorem 13(b): the constraint δ∗(v∣UεFB)≤t\delta^*(v\mid\mathcal U^{FB}_\varepsilon)\le tδ∗(v∣UεFB​)≤t is convex in (v,t)(v,t)(v,t) and convex in ε\varepsilonε for 0<ε<1/e0<\varepsilon<1/\sqrt e0<ε<1/e​.

Significance

Theorem 6 gives a data-driven uncertainty set for which a robust linear constraint is a second-order cone constraint, (24) being an explicit norm expression. By Theorem 1 of the paper, the domination of the worst-case Value at Risk over the region by the support function means that every robustly feasible solution satisfies a chance constraint at level ε\varepsilonε for every distribution in the region. Unlike the Chen–Sim–Sun set, which requires the true mean and deviations, UεFB\mathcal U^{FB}_\varepsilonUεFB​ needs only data and allows the mean and the support to be unknown. Theorem 13(b) supports the alternating heuristic of §9 for choosing the levels εj\varepsilon_jεj​ across several constraints.

The paper's proof is short and leans on "by inspection" and "by Lagrangian strong duality". A formal development makes each of these steps explicit, including the case where the multiplier is not attained, and corrects two printed slips (the optimal values viσˉ2/(2λ)v_i\bar\sigma^2/(2\lambda)vi​σˉ2/(2λ), which should be vi2σˉ2/(2λ)v_i^2\bar\sigma^2/(2\lambda)vi2​σˉ2/(2λ), and the bound mb≤y1≤mbm_b\le y_1\le m_bmb​≤y1​≤mb​). The Chen–Sim–Sun bound itself, a Chernoff-type tail bound under one-sided moment-generating conditions, is cited by the paper without proof. None of these results has a machine-checked proof that this mission is aware of.

Difficulty

The support function of (23) is a maximisation over a set defined by a box, two nonnegative orthants and one coupled quadratic constraint. A coordinate-wise argument does not apply directly because the quadratic budget is shared. The worst case over PFB\mathcal P^{FB}PFB is not a single distribution: the extreme mean and the extreme deviations are chosen coordinate by coordinate according to the sign of viv_ivi​. The Value at Risk bound needs independence of the components; without it (22) fails. When v=0v=0v=0 or the sign pattern makes the quadratic term vanish, the dual multiplier escapes to zero and the dual minimum is only an infimum.

Formalization scope

Vectors are Fin d → ℝ, with 0-based coordinates; u~⊤v\tilde u^\top vu~⊤v is u ⬝ᵥ v. The Value at Risk is the published MultistageStochastic.valueAtRisk at level 1−ε1-\varepsilon1−ε, and the support function is the published RobustMDP.Shared.supportFunction, a real supremum; the goal includes nonemptiness and compactness of UεFB\mathcal U^{FB}_\varepsilonUεFB​, so the supremum is a true maximum and cannot hold through the value 000 of an empty or unbounded set.

The statement is formalized in the criterion form: VaRεP(v)≤δ∗(v∣UεFB)\mathrm{VaR}^{\mathbb P}_\varepsilon(v)\le\delta^*(v\mid\mathcal U^{FB}_\varepsilon)VaRεP​(v)≤δ∗(v∣UεFB​) for all vvv and every P\mathbb PP in the region. By Theorem 1 (mission I of this series), for a nonempty convex compact set this criterion is equivalent to the probabilistic guarantee. The page's "with probability 1−α1-\alpha1−α with respect to the sample" is the coverage of the bootstrap confidence region, which the paper itself treats as approximate; it is not formalized.

Conventions and added hypotheses:

  • σˉfi,σˉbi>0\bar\sigma_{fi},\bar\sigma_{bi}>0σˉfi​,σˉbi​>0, so that the denominators of (23) are genuine; with σˉ=0\bar\sigma=0σˉ=0 Lean's x/0=0x/0=0x/0=0 would leave y2y_2y2​ unconstrained instead of forcing y2=0y_2=0y2​=0.
  • mb≤mfm_b\le m_fmb​≤mf​, which holds because ti≥0t_i\ge0ti​≥0.
  • The region is built from a product measure (independence) of probability measures with bounded support. These are the hypotheses of Theorem 6 on P∗\mathbb P^*P∗ and the section's standing assumption; the page's set-builder for PFB\mathcal P^{FB}PFB omits independence. Without bounded support, the Bochner integral of a non-integrable exponential is 000 in Lean and the deviation conditions would lose their meaning.
  • "σf(Pi)≤σˉ\sigma_f(\mathbb P_i)\le\bar\sigmaσf​(Pi​)≤σˉ" is the predicate "the expression under the root is at most σˉ2\bar\sigma^2σˉ2 for every x>0x>0x>0", which is equivalent and avoids an unbounded supremum.
  • Dual minimisations over λ≥0\lambda\ge0λ≥0 are infima over λ>0\lambda>0λ>0, stated with IsGLB.

A trivializing formalization is ruled out: a region without independence would make the goal false, a region without the probability and bounded-support conditions would let junk integrals satisfy the deviation predicates, and a support function of an empty set would make (24) a statement about 000.

The development needs a Chernoff argument for products of measures, finite-dimensional Lagrangian duality for one convex quadratic constraint (or a direct Cauchy–Schwarz argument), and compactness of the set (23). The definitions file is self-contained and reusable for other forward/backward-deviation sets. Proofs of any milestone, and of the Chen–Sim–Sun bound as a standalone tail inequality, are welcome.

Selected references

  • D. Bertsimas, V. Gupta, N. Kallus, Data-Driven Robust Optimization, arXiv:1401.0212v2, 2014; Math. Program. 167:235–292, 2018. https://arxiv.org/abs/1401.0212
  • X. Chen, M. Sim, P. Sun, A Robust Optimization Perspective on Stochastic Programming, Operations Research 55(6):1058–1071, 2007. https://doi.org/10.1287/opre.1070.0441
12 thms3 active usersReviewed
🏆Completed
Convex OptimizationProbabilityStatistics·Captain: mikedeng1

Data-Driven Robust Optimization III: For Independent Marginals, the Kolmogorov–Smirnov Set U^I Has Support Function (19) and Bounds the Worst-Case Value at RiskResearch Paper

Motivation

A robust linear constraint f(u,x)≤0f(\mathbf u,\mathbf x)\le 0f(u,x)≤0 for all u∈U\mathbf u\in\mathcal Uu∈U replaces an uncertain parameter u~\tilde{\mathbf u}u~ by a deterministic uncertainty set U⊆Rd\mathcal U\subseteq\mathbb R^dU⊆Rd. Bertsimas, Gupta and Kallus (arXiv:1401.0212v2; Math. Program. 167, 2018) build such sets directly from data. Their requirement is a probabilistic guarantee: every robust-feasible decision should satisfy the constraint with probability at least 1−ϵ1-\epsilon1−ϵ under the true distribution P∗\mathbb P^*P∗, and this should hold with probability at least 1−α1-\alpha1−α over the sample. The construction runs a statistical hypothesis test, takes its confidence region of distributions, and turns the worst-case Value at Risk over that region into a set.

This mission covers the case where P∗\mathbb P^*P∗ may be continuous but its ddd coordinates are known to be independent and supported in a known box (§5.1 of the paper). The test is the classical Kolmogorov–Smirnov (KS) goodness-of-fit test, applied separately to each marginal. The result is a convex set UϵI\mathcal U^I_\epsilonUϵI​ whose support function has a one-dimensional closed form, (19). The set is representable with exponential cones, and a line search over a single multiplier separates over it (Remarks 6–7).

Setting

Let d≥0d\ge 0d≥0 and N≥1N\ge 1N≥1 (the sample size). For each coordinate iii we are given points u^i(0)<u^i(1)<⋯<u^i(N)<u^i(N+1)\hat u^{(0)}_i<\hat u^{(1)}_i<\cdots<\hat u^{(N)}_i<\hat u^{(N+1)}_iu^i(0)​<u^i(1)​<⋯<u^i(N)​<u^i(N+1)​. The interval [u^i(0),u^i(N+1)][\hat u^{(0)}_i,\hat u^{(N+1)}_i][u^i(0)​,u^i(N+1)​] is the known box containing the support, and u^i(1),…,u^i(N)\hat u^{(1)}_i,\dots,\hat u^{(N)}_iu^i(1)​,…,u^i(N)​ are the order statistics of the iii-th coordinates of the data. Let Γ=ΓKS∈(0,1)\Gamma=\Gamma^{KS}\in(0,1)Γ=ΓKS∈(0,1) be the KS threshold and 0<ϵ<10<\epsilon<10<ϵ<1.

  • The Value at Risk of P\mathbb PP in direction v\mathbf vv is VaRϵP(v)=inf⁡{t:P(u~Tv≤t)≥1−ϵ}\mathrm{VaR}^{\mathbb P}_\epsilon(\mathbf v)=\inf\{t:\mathbb P(\tilde{\mathbf u}^{\mathsf T}\mathbf v\le t)\ge 1-\epsilon\}VaRϵP​(v)=inf{t:P(u~Tv≤t)≥1−ϵ}.
  • The support function of a set is δ∗(v∣U)=sup⁡u∈UvTu\delta^*(\mathbf v\mid\mathcal U)=\sup_{\mathbf u\in\mathcal U}\mathbf v^{\mathsf T}\mathbf uδ∗(v∣U)=supu∈U​vTu.
  • The KS region PiKS\mathcal P^{KS}_iPiKS​ is the set of Borel probability measures Pi\mathbb P_iPi​ on [u^i(0),u^i(N+1)][\hat u^{(0)}_i,\hat u^{(N+1)}_i][u^i(0)​,u^i(N+1)​] with Pi(u~i≤u^i(j))≥j/N−Γ\mathbb P_i(\tilde u_i\le\hat u^{(j)}_i)\ge j/N-\GammaPi​(u~i​≤u^i(j)​)≥j/N−Γ and Pi(u~i<u^i(j))≤(j−1)/N+Γ\mathbb P_i(\tilde u_i<\hat u^{(j)}_i)\le (j-1)/N+\GammaPi​(u~i​<u^i(j)​)≤(j−1)/N+Γ for j=1,…,Nj=1,\dots,Nj=1,…,N.
  • The independent region PI\mathcal P^IPI is the set of product measures ∏iPi\prod_i\mathbb P_i∏i​Pi​ with Pi∈PiKS\mathbb P_i\in\mathcal P^{KS}_iPi​∈PiKS​.
  • The vectors qL(Γ),qR(Γ)∈ΔN+2q^L(\Gamma),q^R(\Gamma)\in\Delta_{N+2}qL(Γ),qR(Γ)∈ΔN+2​ of (17) are the two boundary distributions of the KS band. With k=⌊N(1−Γ)⌋k=\lfloor N(1-\Gamma)\rfloork=⌊N(1−Γ)⌋, qLq^LqL puts mass Γ\GammaΓ at j=0j=0j=0, mass 1/N1/N1/N at j=1,…,kj=1,\dots,kj=1,…,k, and mass 1−Γ−k/N1-\Gamma-k/N1−Γ−k/N at j=k+1j=k+1j=k+1. Its mirror image is qjR=qN+1−jLq^R_j=q^L_{N+1-j}qjR​=qN+1−jL​.
  • The relative entropy is D(q,p)=∑jqjlog⁡(qj/pj)D(\mathbf q,\mathbf p)=\sum_jq_j\log(q_j/p_j)D(q,p)=∑j​qj​log(qj​/pj​).
  • The uncertainty set (18) is
UϵI={u:∃ θi∈[0,1], qi∈ΔN+2, ∑j=0N+1u^i(j)qji=ui, ∑i=1dD(qi,θiqL+(1−θi)qR)≤log⁡(1/ϵ)}.\mathcal U^I_\epsilon=\Big\{\mathbf u:\exists\,\theta_i\in[0,1],\ \mathbf q^i\in\Delta_{N+2},\ \sum_{j=0}^{N+1}\hat u^{(j)}_iq^i_j=u_i,\ \sum_{i=1}^dD\big(\mathbf q^i,\theta_i\mathbf q^L+(1-\theta_i)\mathbf q^R\big)\le\log(1/\epsilon)\Big\}.UϵI​={u:∃θi​∈[0,1], qi∈ΔN+2​, j=0∑N+1​u^i(j)​qji​=ui​, i=1∑d​D(qi,θi​qL+(1−θi​)qR)≤log(1/ϵ)}.

Formalization targets

Goal: Theorem 5 (deterministic content)

For every v∈Rd\mathbf v\in\mathbb R^dv∈Rd:

UϵI is nonempty, convex and compact,VaRϵP(v)≤δ∗(v∣UϵI)  ∀ P∈PI,\mathcal U^I_\epsilon\ \text{is nonempty, convex and compact},\qquad \mathrm{VaR}^{\mathbb P}_\epsilon(\mathbf v)\le\delta^*(\mathbf v\mid\mathcal U^I_\epsilon)\ \ \forall\,\mathbb P\in\mathcal P^I,UϵI​ is nonempty, convex and compact,VaRϵP​(v)≤δ∗(v∣UϵI​)  ∀P∈PI, δ∗(v∣UϵI)=inf⁡λ>0{λlog⁡(1/ϵ)+λ∑i=1dlog⁡[max⁡(∑jqjLeviu^i(j)/λ,∑jqjReviu^i(j)/λ)]}.(19)\delta^*(\mathbf v\mid\mathcal U^I_\epsilon)=\inf_{\lambda>0}\Big\{\lambda\log(1/\epsilon)+\lambda\sum_{i=1}^d\log\Big[\max\Big(\sum_{j}q^L_je^{v_i\hat u^{(j)}_i/\lambda},\sum_jq^R_je^{v_i\hat u^{(j)}_i/\lambda}\Big)\Big]\Big\}.\tag{19}δ∗(v∣UϵI​)=λ>0inf​{λlog(1/ϵ)+λi=1∑d​log[max(j∑​qjL​evi​u^i(j)​/λ,j∑​qjR​evi​u^i(j)​/λ)]}.(19)

Milestones, in attack order

  1. The Nemirovski–Shapiro bound VaRϵP(v)≤λlog⁡(1/ϵ)+λ∑ilog⁡EPi[eviu~i/λ]\mathrm{VaR}^{\mathbb P}_\epsilon(\mathbf v)\le\lambda\log(1/\epsilon)+\lambda\sum_i\log\mathbb E^{\mathbb P_i}[e^{v_i\tilde u_i/\lambda}]VaRϵP​(v)≤λlog(1/ϵ)+λ∑i​logEPi​[evi​u~i​/λ] for independent, compactly supported marginals.
  2. The boundary laws qLq^LqL, qRq^RqR belong to PiKS\mathcal P^{KS}_iPiKS​.
  3. Theorem EC.2: for monotone ggg, sup⁡PiKSE[g(u~i)]=max⁡(∑jqjLg(u^i(j)),∑jqjRg(u^i(j)))\sup_{\mathcal P^{KS}_i}\mathbb E[g(\tilde u_i)]=\max(\sum_jq^L_jg(\hat u^{(j)}_i),\sum_jq^R_jg(\hat u^{(j)}_i))supPiKS​​E[g(u~i​)]=max(∑j​qjL​g(u^i(j)​),∑j​qjR​g(u^i(j)​)).
  4. (16) combined with EC.2: the Value at Risk over PI\mathcal P^IPI is at most the expression in (19), for every λ>0\lambda>0λ>0.
  5. The Lagrangian dual of max⁡{vTu:u∈UϵI}\max\{\mathbf v^{\mathsf T}\mathbf u:\mathbf u\in\mathcal U^I_\epsilon\}max{vTu:u∈UϵI​}.
  6. (EC.6): max⁡q∈Δ{cTq−D(q,p)}=log⁡∑jpjecj\max_{\mathbf q\in\Delta}\{\mathbf c^{\mathsf T}\mathbf q-D(\mathbf q,\mathbf p)\}=\log\sum_jp_je^{c_j}maxq∈Δ​{cTq−D(q,p)}=log∑j​pj​ecj​.
  7. (EC.7): the linear optimization over θi∈[0,1]\theta_i\in[0,1]θi​∈[0,1] is solved at an endpoint.

Significance

Theorem 5 gives a data-driven uncertainty set for continuous distributions with independent components. Its guarantee is finite-sample, not asymptotic, and its support function costs one line search over λ\lambdaλ to evaluate. Theorem 1 of the paper shows that VaR≤δ∗\mathrm{VaR}\le\delta^*VaR≤δ∗ for all v\mathbf vv is equivalent to the probabilistic guarantee for nonempty convex compact sets. So the goal certifies that every robust-feasible solution of a constraint concave in u\mathbf uu satisfies the chance constraint for every distribution the KS tests cannot reject. Theorem EC.2 is a reusable fact about KS bands: monotone expectations are extremized at the band's two boundary distributions.

The result is proved in the paper, but none of it has been formalized. A formal development would supply:

  • worst-case expectations over a KS confidence band;
  • the finite Gibbs variational identity with possibly vanishing reference masses;
  • a Chernoff-type Value-at-Risk bound for product measures;
  • a strong-duality statement for an entropy-constrained convex program.

Difficulty

The obvious route to the VaR bound is a union bound over coordinates. It loses a factor of ddd in ϵ\epsilonϵ, which is why the paper uses exponential moments and independence instead. The KS region is infinite dimensional, so the inner supremum of (16) is not a finite linear program. Reducing it to the boundary distributions needs the monotonicity of u↦eviu/λu\mapsto e^{v_iu/\lambda}u↦evi​u/λ, and a measure-level comparison of distribution functions against the band. The support-function identity needs strong duality for a jointly convex divergence constraint. The duality holds because θi↦θiqL+(1−θi)qR\theta_i\mapsto\theta_i\mathbf q^L+(1-\theta_i)\mathbf q^Rθi​↦θi​qL+(1−θi​)qR is affine and DDD is jointly convex. The reference vector can have zero entries (when N(1−Γ)N(1-\Gamma)N(1−Γ) is an integer, or in the middle of the band), so the Gibbs step must handle vanishing masses.

Formalization scope

  • Data and conventions. Coordinates are Fin d. The points are uhat : Fin d → Fin (N + 2) → ℝ with the page's indices j=0,…,N+1j=0,\dots,N+1j=0,…,N+1, and the KS constraints run over j : Fin N, which is the page's j−1j-1j−1. The order statistics are data. They are ordered, u^i(0)≤u^i(1)≤⋯≤u^i(N+1)\hat u^{(0)}_i\le\hat u^{(1)}_i\le\dots\le\hat u^{(N+1)}_iu^i(0)​≤u^i(1)​≤⋯≤u^i(N+1)​ (Monotone (uhat i)), as order statistics of a sample in the box are; ties are allowed.
  • Standing assumptions. N≥1N\ge1N≥1, 0<Γ<10<\Gamma<10<Γ<1 and 0<ϵ<10<\epsilon<10<ϵ<1.
  • Regions. Measures in PiKS\mathcal P^{KS}_iPiKS​ are probability measures on R\mathbb RR carried by the box, and PI\mathcal P^IPI consists of the Measure.pi products of such measures, so independence is built in.
  • Relative entropy. DDD carries an explicit finiteness predicate (qj>0⇒pj>0q_j>0\Rightarrow p_j>0qj​>0⇒pj​>0), so Lean's log 0 = 0 cannot make an infinite divergence finite.
  • Published definitions. Value at Risk is the published MultistageStochastic.valueAtRisk at level 1−ϵ1-\epsilon1−ϵ, and δ∗\delta^*δ∗ is the published RobustMDP.Shared.supportFunction, a real sSup. The goal proves UϵI\mathcal U^I_\epsilonUϵI​ nonempty and compact, so δ∗\delta^*δ∗ is never the junk value 000 of an empty or unbounded set.
  • Infima and the multiplier. Infima over λ\lambdaλ are over λ>0\lambda>0λ>0 and stated with IsGLB. The page's λ≥0\lambda\ge0λ≥0 gives the same value.
  • Criterion form of the guarantee. The guarantee is stated as VaRϵP(v)≤δ∗(v∣UϵI)\mathrm{VaR}^{\mathbb P}_\epsilon(\mathbf v)\le\delta^*(\mathbf v\mid\mathcal U^I_\epsilon)VaRϵP​(v)≤δ∗(v∣UϵI​) for all P∈PI\mathbb P\in\mathcal P^IP∈PI. By Theorem 1 (mission I of this series), this criterion is equivalent to the probabilistic guarantee for nonempty convex compact sets.
  • Coverage of the test. The statement "with probability at least 1−α1-\alpha1−α over the sample" is the coverage of PI\mathcal P^IPI. It rests on the distribution-free law of the KS statistic (tables) and on combining ddd tests at level 1−1−αd1-\sqrt[d]{1-\alpha}1−d1−α​. This part is cited, not formalized.
  • Ruled out. The VaR inequality is never checked against a δ∗\delta^*δ∗ that sSup collapses to 000, and the divergence budget is never relaxed by unguarded logarithms.

Contributions are welcome on any milestone. Milestones 1, 3 and 6 are independent of each other and of the rest; the goal follows from milestones 1–7 together with the convex-analytic facts about UϵI\mathcal U^I_\epsilonUϵI​.

Selected references

  • D. Bertsimas, V. Gupta, N. Kallus, Data-Driven Robust Optimization, arXiv:1401.0212v2, 2014; Math. Program. 167:235–292, 2018. https://arxiv.org/abs/1401.0212
  • A. Nemirovski, A. Shapiro, Convex approximations of chance constrained programs, SIAM J. Optim. 17(4):969–996, 2006. https://doi.org/10.1137/050622328
  • M. A. Stephens, EDF statistics for goodness of fit and some comparisons, J. Amer. Statist. Assoc. 69(347):730–737, 1974. https://doi.org/10.1080/01621459.1974.10480196
  • S. Boyd, L. Vandenberghe, Convex Optimization, Cambridge University Press, 2004. https://web.stanford.edu/~boyd/cvxbook/
11 thms3 active usersReviewed
🏆Completed
CombinatoricsLinear algebra·Captain: mikedeng1

On the Abstract Properties of Linear Dependence 4: Orthogonal Subspaces Have Dual MatroidsResearch Paper

Motivation

Hassler Whitney introduced matroids in On the Abstract Properties of Linear Dependence (Amer. J. Math. 57, 1935, doi:10.2307/2371182) to capture what linear dependence among the columns of a matrix and the circuit structure of a graph have in common. One of the central constructions of the paper is duality. For graphs, duality exists only for planar graphs (Whitney, Non-separable and planar graphs, Trans. AMS 34, 1932); for matroids Whitney showed that every matroid has a dual, and that duality has a direct linear-algebra model: the matroid of a subspace of Rn\mathbb R^nRn and the matroid of its orthogonal complement are duals.

Matroid duality underlies much of combinatorial optimization: the duality between cycles and cuts in graphs, the relation between a linear code and its dual code, the exchange of rank and corank in matroid intersection and partition, and the treatment of network flows as duals of potential problems. Whitney's §§11–13 are where this structure is first defined and where its linear-algebra meaning (Theorem 28) is established.

Setting

A matroid MMM is a finite set of elements e1,…,ene_1,\dots,e_ne1​,…,en​ with a rank function rrr on its subsets; equivalently, a family of independent sets, or of bases (maximal independent sets). For a subset NNN write ρ(N)\rho(N)ρ(N) for its number of elements and

n(N)=ρ(N)−r(N)n(N) = \rho(N) - r(N)n(N)=ρ(N)−r(N)

for its nullity; r(M)r(M)r(M) and n(M)n(M)n(M) are the rank and nullity of the whole set of elements.

Duality (§11). Let MMM and M′M'M′ be matroids and σ\sigmaσ a one-to-one correspondence between their elements. M′M'M′ is a dual of MMM (via σ\sigmaσ) if, for every subset NNN of MMM, with N′N'N′ the complement in M′M'M′ of σ(N)\sigma(N)σ(N),

r(N′)=r(M′)−n(N).(11.1)r(N') = r(M') - n(N). \tag{11.1}r(N′)=r(M′)−n(N).(11.1)

The matroid of a subspace (§12). Let EnE_nEn​ be nnn-dimensional Euclidean space with coordinates x1,…,xnx_1,\dots,x_nx1​,…,xn​, and let HHH be a hyperplane through the origin, which in Whitney's usage is a linear subspace of any dimension. For a set SSS of coordinates, project HHH onto the coordinate subspace ES′E'_SES′​ spanned by the axes xix_ixi​, i∈Si\in Si∈S. The matroid associated with HHH has elements e1,…,ene_1,\dots,e_ne1​,…,en​, one per coordinate, and gives the set {ei:i∈S}\{e_i: i\in S\}{ei​:i∈S} the rank

rH(S)=dim⁡πS(H).r_H(S) = \dim \pi_S(H).rH​(S)=dimπS​(H).

If HHH is the row space of a matrix M\mathbf MM, then rH(S)r_H(S)rH​(S) is the rank of the columns of M\mathbf MM indexed by SSS, so the associated matroid is the column matroid of M\mathbf MM.

In the Lean development these objects are WhitneyMatroid.Components.nullity (shared), IsDualVia M M' σ, IsDual M M', subspaceRank H S and IsAssociated M H.

Formalization targets

Goal: Theorem 28

Let HHH be a subspace of EnE_nEn​ and H′=H⊥H' = H^\perpH′=H⊥ its orthogonal complement. If MMM and M′M'M′ are the matroids associated with HHH and H′H'H′, then MMM and M′M'M′ are duals under the correspondence of equal coordinates:

∀N⊆{e1,…,en}:rM′(N‾)=r(M′)−nM(N).\forall N\subseteq\{e_1,\dots,e_n\}:\qquad r_{M'}(\overline N) = r(M') - n_M(N).∀N⊆{e1​,…,en​}:rM′​(N)=r(M′)−nM​(N).

Milestones

  1. Theorem 27. For every subspace HHH there is exactly one matroid associated with HHH.
  2. Theorem 6 (already proved on the platform). All bases have the same number of elements.
  3. Theorem 7. BBB is a base iff r(B)=r(M)r(B)=r(M)r(B)=r(M) and n(B)=0n(B)=0n(B)=0.
  4. Theorem 8. If BBB is a base and NNN is independent, then N∪N′N\cup N'N∪N′ is a base for some N′⊆BN'\subseteq BN′⊆B.
  5. Theorem 20. If M′M'M′ is a dual of MMM, then r(M′)=n(M)r(M') = n(M)r(M′)=n(M) and n(M′)=r(M)n(M') = r(M)n(M′)=r(M).
  6. Theorem 23. M′M'M′ is a dual of MMM via σ\sigmaσ iff, for every BBB, BBB is a base of MMM exactly when the complement of σ(B)\sigma(B)σ(B) is a base of M′M'M′.
  7. Theorem 21. Duality is symmetric.
  8. Theorem 22. Every matroid has a dual.

Significance

The result. Theorem 28 identifies abstract duality with orthogonal complementation. With Theorem 23 it says that the column matroid of a real matrix whose rows span HHH and the column matroid of a matrix whose rows span H⊥H^\perpH⊥ have complementary bases. This is the basis of the standard representation of the dual of a represented matroid ([Ir∣A][I_r\mid A][Ir​∣A] and [−AT∣In−r][-A^{\mathsf T}\mid I_{n-r}][−AT∣In−r​]), of the cycle/cocycle duality of graphs viewed through incidence matrices, and of the fact that a matroid representable over a field has a dual representable over the same field. Theorems 20–23 are the basic facts every later treatment of matroid duality starts from: duality exchanges rank and nullity, is symmetric, always exists, and is characterized by base complements.

Formalizing it. All of these results are classical and proved in Whitney's paper; what this mission adds is a machine-checked development of Whitney's own definition of duality, the rank identity (11.1), rather than the base-complement definition used in modern libraries. Mathlib defines the dual matroid M∗M^*M∗ by base complements and proves that it is a matroid; it does not, at the pinned revision, contain the rank formula (11.1) for M∗M^*M∗, the matroid of a real subspace, or Theorem 28. Theorem 6 is already proved on the platform and enters as a reference. Theorems 7 and 8 are close to Mathlib lemmas and are footholds rather than new content.

Difficulty

The difficulty of the goal is not in the combinatorics but in the passage between the two descriptions of a subspace. The natural first idea is to compare the ranks rH(S)r_H(S)rH​(S) and rH⊥(S‾)r_{H^\perp}(\overline S)rH⊥​(S) directly by counting dimensions of HHH and H⊥H^\perpH⊥. This does not close: rH(S)r_H(S)rH​(S) is the dimension of a projection of HHH onto coordinates, while dim⁡H⊥=n−dim⁡H\dim H^\perp = n - \dim HdimH⊥=n−dimH only controls H⊥H^\perpH⊥ as a whole, and the rank of a complementary coordinate set in M′M'M′ is not determined by the dimensions of HHH and H⊥H^\perpH⊥ alone. Some relation between coordinate projections of HHH and the structure of H⊥H^\perpH⊥ on the complementary coordinates has to be established. Theorem 27 is itself nontrivial in Lean: the rank function rHr_HrH​ has to be shown to satisfy the matroid axioms, which needs a careful treatment of coordinate projections and of submodularity of dimension. On the abstract side, Theorem 23 needs both directions of the passage between the rank identity and base complements, including the counting step r(M)+r(M′)=ρ(M)r(M) + r(M') = \rho(M)r(M)+r(M′)=ρ(M).

Formalization scope

  • Matroids are Mathlib Matroid α on a finite type α whose ground set is all of α (Whitney's matroid is its finite set of elements). This finiteness and the ground set convention are part of the definitions; all theorems are stated under them.
  • Ranks and nullities. Ranks are Mathlib's eRk, finite on a finite type and converted to integers; the nullity n(N)=ρ(N)−r(N)n(N)=\rho(N)-r(N)n(N)=ρ(N)−r(N) and the identity (11.1) are computed in Z\mathbb ZZ. No truncated subtraction appears.
  • Duality is Whitney's rank identity (11.1) for every subset, for a fixed bijection σ : α ≃ β (IsDualVia) or some bijection (IsDual). It is deliberately not defined as Mathlib's M✶: with that definition Theorem 23 would be nearly definitional and Theorem 28 would lose the rank identity. A formalization that replaces (11.1) by the base-complement condition, or states Theorem 28 only for bases, is not this mission's goal.
  • Euclidean space is EuclideanSpace ℝ (Fin n) with its standard inner product; the orthogonal hyperplane is Hᗮ. The field is R\mathbb RR, as in the paper; the analogous statement over other fields with the dot-product annihilator is a generalization and not part of this mission.
  • The associated matroid is a predicate (IsAssociated M H): ground set Fin n, and the rank of every set SSS of coordinates equals the finrank of the image of HHH under the coordinate projection onto SSS (not of H∩ES′H\cap E'_SH∩ES′​, a different set function). Existence and uniqueness is Theorem 27.
  • Implicit hypotheses made explicit: the ground sets are finite and equal to the whole element type; Whitney's "dimension rrr" and "dimension n−rn-rn−r" in Theorem 28 are consequences of H′=H⊥H'=H^\perpH′=H⊥ and are not hypotheses. The goal covers n=0n=0n=0, H={0}H=\{0\}H={0} and H=EnH=E_nH=En​.
  • Infrastructure needed and reusable: the matroid of a real subspace (equivalently the column matroid of a real matrix), with its rank function in terms of coordinate projections; the rank formula of the dual matroid; the dimension identity relating projections of HHH and sections of H⊥H^\perpH⊥. All are reusable for the other missions of this series (the Fano matroid and binary matroids) and for any later work on represented matroids. Proofs of the milestones, alternative proofs of Theorem 28 through Mathlib's M✶, and supporting lemmas on coordinate projections are welcome.

Selected references

  • H. Whitney, On the Abstract Properties of Linear Dependence, American Journal of Mathematics 57 (1935), 509–533. https://doi.org/10.2307/2371182
  • H. Whitney, Non-separable and planar graphs, Transactions of the American Mathematical Society 34 (1932), 339–362. https://doi.org/10.1090/S0002-9947-1932-1501641-2
  • J. Oxley, Matroid Theory, 2nd ed., Oxford Graduate Texts in Mathematics 21, Oxford University Press, 2011, Chapter 2 (duality). https://doi.org/10.1093/acprof:oso/9780198566946.001.0001
  • Mathlib, Mathlib.Combinatorics.Matroid (Dual, Rank). https://github.com/leanprover-community/mathlib4
12 thms3 active usersReviewed
🏆Completed
Probability·Captain: mikedeng1

Computational Issues in an Infinite-Horizon, Multiechelon Inventory Model 3: A Closed Form for the Induced Penalty Cost under Normal DemandResearch Paper

Motivation

Multiechelon inventory theory studies supply systems in which stock is held at several levels: here, a depot that orders from an outside supplier and a retail outlet that is replenished from the depot and faces random customer demand. The question is how much to order and ship in each period so as to minimize expected holding, shortage and ordering costs over an infinite horizon. Clark and Scarf (Management Science, 1960) showed that the finite-horizon problem of a serial system decomposes into single-location problems linked by an induced penalty cost. Federgruen and Zipkin (Operations Research 32(4), 1984) extended the decomposition to the infinite horizon under discounted and average costs, and then asked what it costs to compute an optimal policy.

In the reduced single-location problem the only nonlinear part of the one-period cost is the expected induced penalty PLP^LPL. Any algorithm for the reduced problem (Veinott–Wagner type policy computations, for instance) evaluates PLP^LPL many times. In general each evaluation is a numerical integral. Section 4 of the paper shows that when demand is normal, PLP^LPL has a closed form in the univariate and bivariate standard normal distribution functions. This mission formalizes that closed form, eq. (13) on p. 830.

Setting

Time is discrete. One-period demand is u∼N(μ,σ2)u\sim N(\mu,\sigma^2)u∼N(μ,σ2) with σ>0\sigma>0σ>0, independent across periods. For i≥1i\ge1i≥1, u(i)u^{(i)}u(i) is the total demand over iii periods. It is normal with mean μ(i)=iμ\mu^{(i)}=i\muμ(i)=iμ and standard deviation σ(i)=i1/2σ\sigma^{(i)}=i^{1/2}\sigmaσ(i)=i1/2σ, density f(i)f^{(i)}f(i) and cdf F(i)F^{(i)}F(i). The shipment lead time from depot to outlet is l≥0l\ge0l≥0 and the order lead time from the supplier is L≥1L\ge1L≥1. The cost factors are a system-wide holding cost hd>0h^d>0hd>0, a retailer holding cost hr>0h^r>0hr>0 and a retailer shortage penalty pr>0p^r>0pr>0. Write ps=hd+prp^s=h^d+p^rps=hd+pr. Costs are average costs, so the discount factor is α=1\alpha=1α=1 throughout.

The retailer's one-period cost (p. 822) is

R(x)=−hd(x−μ(l))+prE[u(l+1)−x]++(hd+hr)E[x−u(l+1)]+.R(x)=-h^d(x-\mu^{(l)})+p^rE[u^{(l+1)}-x]^++(h^d+h^r)E[x-u^{(l+1)}]^+ .R(x)=−hd(x−μ(l))+prE[u(l+1)−x]++(hd+hr)E[x−u(l+1)]+.

The critical number xr∗x^{r*}xr∗ is a global minimizer of RRR (property (b), p. 824). The induced penalty cost is P(x)=0P(x)=0P(x)=0 for x≥xr∗x\ge x^{r*}x≥xr∗ and P(x)=R(x)−R(xr∗)P(x)=R(x)-R(x^{r*})P(x)=R(x)−R(xr∗) for x<xr∗x<x^{r*}x<xr∗. Its expectation over the order lead time is

PL(x)=E P[x−u(L)](eq. (10)).P^L(x)=E\,P[x-u^{(L)}]\qquad\text{(eq. (10))}.PL(x)=EP[x−u(L)](eq. (10)).

Let Φ\PhiΦ and ϕ\phiϕ be the standard normal cdf and density, Θ(z)=zΦ(z)+ϕ(z)\Theta(z)=z\Phi(z)+\phi(z)Θ(z)=zΦ(z)+ϕ(z), and Φ(ξ1,ξ2;ρ)\Phi(\xi_1,\xi_2;\rho)Φ(ξ1​,ξ2​;ρ) the cdf of a bivariate normal pair with standard normal marginals and correlation ρ\rhoρ. The paper defines (p. 830)

τ1(x)=−x−(xr∗+μ(L))σ(L),τ2(x)=x−μ(L+l+1)σ(L+l+1),νr∗=xr∗−μ(l+1)σ(l+1),\tau_1(x)=-\frac{x-(x^{r*}+\mu^{(L)})}{\sigma^{(L)}},\quad \tau_2(x)=\frac{x-\mu^{(L+l+1)}}{\sigma^{(L+l+1)}},\quad \nu^{r*}=\frac{x^{r*}-\mu^{(l+1)}}{\sigma^{(l+1)}},τ1​(x)=−σ(L)x−(xr∗+μ(L))​,τ2​(x)=σ(L+l+1)x−μ(L+l+1)​,νr∗=σ(l+1)xr∗−μ(l+1)​, τ3(x)=−x−(xr∗+μ(L))−[σ(L)/σ(l+1)]2[xr∗−μ(l+1)]σ(L)σ(L+l+1)/σ(l+1),\tau_3(x)=-\frac{x-(x^{r*}+\mu^{(L)})-[\sigma^{(L)}/\sigma^{(l+1)}]^2[x^{r*}-\mu^{(l+1)}]}{\sigma^{(L)}\sigma^{(L+l+1)}/\sigma^{(l+1)}},τ3​(x)=−σ(L)σ(L+l+1)/σ(l+1)x−(xr∗+μ(L))−[σ(L)/σ(l+1)]2[xr∗−μ(l+1)]​, ϵ1(x)=Φ[τ3(x)]ϕ[τ2(x)]σ(L+l+1),ϵ2(x)=Φ(νr∗)ϕ[τ1(x)]σ(L),ι(x)=σ(L)Θ[τ1(x)],\epsilon_1(x)=\frac{\Phi[\tau_3(x)]\phi[\tau_2(x)]}{\sigma^{(L+l+1)}},\qquad \epsilon_2(x)=\frac{\Phi(\nu^{r*})\phi[\tau_1(x)]}{\sigma^{(L)}},\qquad \iota(x)=\sigma^{(L)}\Theta[\tau_1(x)],ϵ1​(x)=σ(L+l+1)Φ[τ3​(x)]ϕ[τ2​(x)]​,ϵ2​(x)=σ(L)Φ(νr∗)ϕ[τ1​(x)]​,ι(x)=σ(L)Θ[τ1​(x)], κ(x)=σ(l+1)Θ(νr∗)Φ[τ1(x)]−{[σ(L+l+1)]2ϵ1(x)−[σ(L)]2ϵ2(x)}−[x−μ(L+l+1)] Φ[τ1(x),τ2(x);ρ],\kappa(x)=\sigma^{(l+1)}\Theta(\nu^{r*})\Phi[\tau_1(x)]-\{[\sigma^{(L+l+1)}]^2\epsilon_1(x)-[\sigma^{(L)}]^2\epsilon_2(x)\}-[x-\mu^{(L+l+1)}]\,\Phi[\tau_1(x),\tau_2(x);\rho],κ(x)=σ(l+1)Θ(νr∗)Φ[τ1​(x)]−{[σ(L+l+1)]2ϵ1​(x)−[σ(L)]2ϵ2​(x)}−[x−μ(L+l+1)]Φ[τ1​(x),τ2​(x);ρ],

with ρ=−σ(L)/σ(L+l+1)\rho=-\sigma^{(L)}/\sigma^{(L+l+1)}ρ=−σ(L)/σ(L+l+1).

Formalization targets

Goal: eq. (13)

For every real xxx,

PL(x)=ps ι(x)−(ps+hr) κ(x).P^L(x)=p^s\,\iota(x)-(p^s+h^r)\,\kappa(x).PL(x)=psι(x)−(ps+hr)κ(x).

This is an exact identity for every admissible parameter value. It holds with no constants left free.

Milestones

The milestones follow the paper's outline of the derivation on pp. 829–831:

  1. eq. (11), R(x)=ps[μ(l+1)−x]+(ps+hr)∫−∞xF(l+1)(t) dt−hdμR(x)=p^s[\mu^{(l+1)}-x]+(p^s+h^r)\int_{-\infty}^xF^{(l+1)}(t)\,dt-h^d\muR(x)=ps[μ(l+1)−x]+(ps+hr)∫−∞x​F(l+1)(t)dt−hdμ;
  2. eq. (12), PL(x)=∫x−xr∗∞[R(x−t)−R(xr∗)]f(L)(t) dtP^L(x)=\int_{x-x^{r*}}^\infty[R(x-t)-R(x^{r*})]f^{(L)}(t)\,dtPL(x)=∫x−xr∗∞​[R(x−t)−R(xr∗)]f(L)(t)dt;
  3. eq. (14), PL′(x)=−psΦ[τ1(x)]+(ps+hr)∫x−xr∗∞F(l+1)(x−t)f(L)(t) dtP^{L\prime}(x)=-p^s\Phi[\tau_1(x)]+(p^s+h^r)\int_{x-x^{r*}}^\infty F^{(l+1)}(x-t)f^{(L)}(t)\,dtPL′(x)=−psΦ[τ1​(x)]+(ps+hr)∫x−xr∗∞​F(l+1)(x−t)f(L)(t)dt;
  4. eq. (15), the same derivative with the integral replaced by Φ[τ1(x),τ2(x);ρ]\Phi[\tau_1(x),\tau_2(x);\rho]Φ[τ1​(x),τ2​(x);ρ];
  5. PL(x)→0P^L(x)\to0PL(x)→0 as x→∞x\to\inftyx→∞, hence PL(x)=−∫x∞PL′(t) dtP^L(x)=-\int_x^\infty P^{L\prime}(t)\,dtPL(x)=−∫x∞​PL′(t)dt;
  6. ι′(x)=−Φ[τ1(x)]\iota'(x)=-\Phi[\tau_1(x)]ι′(x)=−Φ[τ1​(x)] and ι(x)→0\iota(x)\to0ι(x)→0;
  7. the two conditional-normal identities, which give Φ[τ3(x)]\Phi[\tau_3(x)]Φ[τ3​(x)] and Φ(νr∗)\Phi(\nu^{r*})Φ(νr∗);
  8. ddxΦ[τ1(x),τ2(x);ρ]=ϵ1(x)−ϵ2(x)\frac{d}{dx}\Phi[\tau_1(x),\tau_2(x);\rho]=\epsilon_1(x)-\epsilon_2(x)dxd​Φ[τ1​(x),τ2​(x);ρ]=ϵ1​(x)−ϵ2​(x);
  9. and 10. the formulas for ϵ1′\epsilon_1'ϵ1′​ and ϵ2′\epsilon_2'ϵ2′​;
  10. κ′(x)=−Φ[τ1(x),τ2(x);ρ]\kappa'(x)=-\Phi[\tau_1(x),\tau_2(x);\rho]κ′(x)=−Φ[τ1​(x),τ2​(x);ρ] and κ(x)→0\kappa(x)\to0κ(x)→0.

Two side remarks of p. 830 are also included: Θ′=Φ\Theta'=\PhiΘ′=Φ, and the simplified form of τ3\tau_3τ3​.

Significance

The result. Under normal demand, (13) replaces the numerical integral (12) with a few evaluations of Φ\PhiΦ, ϕ\phiϕ and the bivariate normal cdf, all available in standard numerical libraries. Together with the decomposition results of Sections 1–3 of the paper, it makes the policy computation for the two-echelon system with normal demand no harder than a single-location computation with an explicit cost function. The same functions reappear in the paper's Section 5 for several retail outlets, after a reinterpretation of σ(l+1)\sigma^{(l+1)}σ(l+1).

Formalizing it. The paper proves (13) only in outline: it calls the derivation "an elementary integration problem, but … sufficiently involved to warrant an outline" and leaves "tedious algebra" and "more algebra" to the reader. A machine-checked proof turns that outline into a complete argument, including the analytic steps the outline passes over: differentiation under the integral sign, the limits at +∞+\infty+∞, and the identification of an integral of normal densities with a bivariate normal probability. To our knowledge no formal proof of (13) exists, and no bivariate normal distribution function is on the platform yet.

Difficulty

The obvious approach is to substitute (11) into (12) and integrate. The result is a double integral of normal densities over a region bounded by a line, and it does not reduce to univariate functions. The paper's route is to differentiate first, identify the derivative (14) as a probability for the correlated pair (u(L),u(L)+u(l+1))(u^{(L)},u^{(L)}+u^{(l+1)})(u(L),u(L)+u(l+1)), and then recover PLP^LPL by integrating from +∞+\infty+∞. That route needs three things: justification for differentiating under the integral in (12), whose integrand has a kink at t=x−xr∗t=x-x^{r*}t=x−xr∗; control of the limits at +∞+\infty+∞; and the conditional-normal identities, which involve conditioning on a null event and so must be handled through densities. The verification of κ′\kappa'κ′ is a long computation with Θ\ThetaΘ, ϵ1\epsilon_1ϵ1​ and ϵ2\epsilon_2ϵ2​, in which every constant matters.

Formalization scope

Everything lives in the namespace FZEchelon.NormalDemand. The model data form the structure Data (fields μ,σ,hd,hr,pr,l,L,xr∗\mu,\sigma,h^d,h^r,p^r,l,L,x^{r*}μ,σ,hd,hr,pr,l,L,xr∗). The law of u(i)u^{(i)}u(i) is the platform's normal demand law InventoryControl.newsboyDemand with mean iμi\muiμ and standard deviation i σ\sqrt i\,\sigmai​σ. Expectations are Bochner integrals, and improper integrals are set integrals over Set.Ioi/Set.Iic. Φ\PhiΦ is cdf (gaussianReal 0 1). The bivariate cdf is the iterated integral of the explicit bivariate density over a lower-left quadrant (meaningful for ∣ρ∣<1|\rho|<1∣ρ∣<1; here ρ∈(−1,0)\rho\in(-1,0)ρ∈(−1,0)). Derivatives are HasDerivAt and limits are Tendsto … atTop (𝓝 0).

Standing hypotheses of every theorem: σ>0\sigma>0σ>0; hd,hr,pr>0h^d,h^r,p^r>0hd,hr,pr>0 (p. 821); L≥1L\ge1L≥1; and xr∗x^{r*}xr∗ minimizes RRR. Two of these are added to the page and disclosed. σ>0\sigma>0σ>0 is needed because every τ\tauτ divides by some σ(i)\sigma^{(i)}σ(i). L≥1L\ge1L≥1 is needed because σ(0)=0\sigma^{(0)}=0σ(0)=0, and the paper treats zero order lead time separately (p. 819). Demand is exactly normal, as in §4, which acknowledges that this violates u≥0u\ge0u≥0 and ignores the objection. No nonnegativity, truncation or approximation enters. The goal fixes α=1\alpha=1α=1, the average-cost case the section restricts to. The discounted analogue is not part of this mission.

A trivializing formalization is ruled out: every Bochner integral in the statements has an integrable integrand, since integrands grow at most linearly and the normal law has all moments. No division by a zero standard deviation can occur under the hypotheses. A sorry-free check in the workspace shows that all hypotheses hold together, with a minimizer xr∗x^{r*}xr∗ of RRR proved to exist.

Needed infrastructure: properties of gaussianReal (moments, convolution of independent normals), differentiation of parametric integrals, and a bivariate normal distribution function with its partial derivatives. A reusable treatment of the bivariate normal cdf, linking the density form used here to Mathlib's multivariateGaussian, would be a contribution of independent value. Proofs of individual milestones are welcome in any order.

Selected references

  • A. Federgruen and P. Zipkin, Computational Issues in an Infinite-Horizon, Multiechelon Inventory Model, Operations Research 32(4):818–836, 1984. https://doi.org/10.1287/opre.32.4.818
  • A. J. Clark and H. Scarf, Optimal Policies for a Multi-Echelon Inventory Problem, Management Science 6(4):475–490, 1960. https://doi.org/10.1287/mnsc.6.4.475
  • A. F. Veinott Jr. and H. M. Wagner, Computing Optimal (s, S) Inventory Policies, Management Science 11(5):525–552, 1965. https://doi.org/10.1287/mnsc.11.5.525
17 thms3 active usersReviewed
🏆Completed
Dynamic ProgrammingOptimization·Captain: mikedeng1

Contraction Mappings in the Theory Underlying Dynamic Programming 1: Under the Contraction and Monotonicity Assumptions the Solution of the Functional Equation Is the Optimal ReturnResearch Paper

Why this problem arose

Dynamic programming often replaces a search over whole plans with an equation for a value function. That replacement is useful only when solving the equation recovers the value actually available through policies. In his 1967 paper, Eric Denardo isolated conditions on an abstract return function that make this identification possible, without committing to a particular transition law or a finite state space. The framework covers discounted decision models and other examples discussed in the paper, while allowing different decisions at different states. Denardo, 1967.

The mission concerns the paper's first regime: a one-step contraction together with monotonicity. It focuses on the question raised just after equation (4): does the solution of the functional equation equal the supremum of the return functions generated by policies? The paper proves that it does under these assumptions. Its later NNN-stage regime is a separate mission. Denardo, 1967, pp. 167–169.

States, decisions, and returns

Let Ω\OmegaΩ be a set of states. At each x∈Ωx\in\Omegax∈Ω there is a decision set DxD_xDx​. A policy δ\deltaδ selects a decision δx∈Dx\delta_x\in D_xδx​∈Dx​ at every state, so the policy space Δ\DeltaΔ is the full Cartesian product ∏x∈ΩDx\prod_{x\in\Omega}D_x∏x∈Ω​Dx​. Let VVV be the bounded real functions on Ω\OmegaΩ, with the uniform metric ρ(u,v)=sup⁡x∈Ω∣u(x)−v(x)∣\rho(u,v)=\sup_{x\in\Omega}|u(x)-v(x)|ρ(u,v)=supx∈Ω​∣u(x)−v(x)∣. A return function h(x,d,v)h(x,d,v)h(x,d,v) assigns a real return to a state, an available decision, and a continuation value v∈Vv\in Vv∈V. Denardo, 1967, p. 166.

For a policy δ\deltaδ, its policy operator HδH_\deltaHδ​ acts by (Hδv)(x)=h(x,δx,v)(H_\delta v)(x)=h(x,\delta_x,v)(Hδ​v)(x)=h(x,δx​,v). Under the contraction assumption, some ccc satisfies 0≤c<10\le c<10≤c<1 and ∣h(x,d,u)−h(x,d,v)∣≤cρ(u,v)|h(x,d,u)-h(x,d,v)|\le c\rho(u,v)∣h(x,d,u)−h(x,d,v)∣≤cρ(u,v) for all eligible x,d,u,vx,d,u,vx,d,u,v. Each HδH_\deltaHδ​ then has a unique fixed point vδ∈Vv_\delta\in Vvδ​∈V, called that policy's return function. The optimal return function is defined pointwise by f(x)=sup⁡δ∈Δvδ(x)f(x)=\sup_{\delta\in\Delta}v_\delta(x)f(x)=supδ∈Δ​vδ​(x). These are the paper's definitions, rather than a presumed equality between fff and a Bellman fixed point. Denardo, 1967, pp. 166–167.

The maximization operator AAA instead chooses the best decision at each state for a supplied continuation value: (Av)(x)=sup⁡d∈Dxh(x,d,v)(Av)(x)=\sup_{d\in D_x}h(x,d,v)(Av)(x)=supd∈Dx​​h(x,d,v). Denardo assumes that HδvH_\delta vHδ​v and AvAvAv remain bounded, so both operators act on VVV. The monotonicity assumption says that u≥vu\ge vu≥v pointwise implies Hδu≥HδvH_\delta u\ge H_\delta vHδ​u≥Hδ​v for every policy. These are separate conditions: contraction controls distance between value functions, while monotonicity controls their order. Denardo, 1967, pp. 166–168.

Formalization targets

The central target is Theorem 3. Under contraction and monotonicity, the unique bounded solution v∗v^*v∗ of equation (4) is the optimal return:

Av∗=v∗,v∗(x)=f(x)=sup⁡δ∈Δvδ(x)(x∈Ω).Av^*=v^*,\qquad v^*(x)=f(x)=\sup_{\delta\in\Delta}v_\delta(x)\quad(x\in\Omega).Av∗=v∗,v∗(x)=f(x)=δ∈Δsup​vδ​(x)(x∈Ω).

The milestones follow the paper's route to that assertion. Theorem 1 bounds a policy's return by its fixed-point residual, ρ(vδ,w)≤ρ(Hδw,w)/(1−c)\rho(v_\delta,w)\le\rho(H_\delta w,w)/(1-c)ρ(vδ​,w)≤ρ(Hδ​w,w)/(1−c). Theorem 2 says that AAA itself has modulus at most ccc. The text accompanying equation (4) establishes that AAA has exactly one fixed point. Corollary 1 then gives, for each ε>0\varepsilon>0ε>0, a policy with residual at most ε(1−c)\varepsilon(1-c)ε(1−c) at that fixed point and return within ε\varepsilonε of it, together with an exact equality criterion for zero residual. Denardo, 1967, pp. 167–168.

What the result gives

Theorem 3 justifies reading the functional equation as an optimization equation: its solution is the pointwise supremum of returns realized by the paper's policy model. Corollary 1 supplies policy returns arbitrarily close to that solution in the uniform metric. With the compactness and continuity assumptions of Corollary 2, the paper also obtains a policy that attains it, though that optional result is outside this mission's milestone list. Denardo, 1967, pp. 167–168.

The mathematical result was proved in the cited paper. This mission asks for machine-checked proofs of its abstract statements in Lean, including the residual estimate, the maximization operator's contraction, fixed-point existence, and the identification with optimal policy return. The definitions are reusable for decision models whose returns fit Denardo's assumptions; they do not prescribe a probability kernel or a finite action set.

Where the difficulty lies

The fixed-point theorem alone identifies a unique solution of Av=vAv=vAv=v; it does not identify that solution with fff, because fff is defined through the separate fixed points of all HδH_\deltaHδ​. The distinction matters: the paper observes that contraction without monotonicity can leave v∗v^*v∗ different from fff, and gives h(x,d,v)=−v(x)/2h(x,d,v)=-v(x)/2h(x,d,v)=−v(x)/2 as a nonmonotone example. The formal goal therefore retains both definitions and requires monotonicity. Denardo, 1967, pp. 167–168.

There is also a choice issue in Corollary 1. A near-maximizing decision is available separately at every state, while its conclusion asks for one policy choosing all those decisions at once. The full Cartesian policy space is essential to the statement. On an infinite state set, the value functions must remain bounded even after applying AAA or a policy operator; contraction of differences by itself does not provide that range condition. Denardo, 1967, pp. 166–168.

Formalization scope

Lean represents VVV by bounded real functions lp (fun _ : Ω => ℝ) ⊤; its metric is ρ\rhoρ. The state type may be infinite or empty, and decision sets depend on the state. Policies are precisely dependent functions δ:(x:Ω)→Dx\delta:(x:\Omega)\to D_xδ:(x:Ω)→Dx​. The operators HδH_\deltaHδ​ and AAA are maps into VVV supplied with equations fixing their values from hhh. This expresses the bounded-range assumptions on pages 166–167. Pointwise order is explicit because the bounded-function type has no order instance. Denardo, 1967, pp. 166–168.

Every supremum in the mission is a genuine real least upper bound. IsMaxOperator requires it for the decision returns at each state; IsOptimalReturn requires it for the policy returns. These predicates avoid assigning a default value to an empty or unbounded supremum. Where states exist, IsMaxOperator also entails a nonempty decision set. The family vδv_\deltavδ​ is supplied with Hδvδ=vδH_\delta v_\delta=v_\deltaHδ​vδ​=vδ​; contraction ensures each such return exists uniquely. Equation (4) and the main goal explicitly assert existence of the fixed point of AAA, so the goal cannot hold merely because there is no solution to identify. Defining fff from the fixed point of AAA, or defining v∗v^*v∗ from the policy supremum, would erase the paper's substantive claim and is excluded.

The definition layer needs bounded functions, uniform distance, real least upper bounds, and dependent policies. The theorem layer needs fixed-point and order reasoning on these objects. Contributions toward those general operator facts and the paper's four milestone statements are within scope.

Selected references

  • Eric V. Denardo, Contraction Mappings in the Theory Underlying Dynamic Programming, SIAM Review 9(2), 165–177, 1967. DOI: 10.1137/1009030.
6 thms3 active usersReviewed
🏆Completed
CombinatoricsOptimization·Captain: mikedeng1

Assortment Optimization under Variants of the Nested Logit Model 4: With Dissimilarity Parameters at Most One, the Knapsack-Relaxation and Singleton LP Optimum Scaled by 2 Is Feasible for the Full LPResearch Paper

Motivation

Assortment optimization asks which set of products a firm should offer when customers choose among the offered products according to a discrete choice model; it underlies shelf-space planning in retail and fare-class control in airline revenue management (Talluri and van Ryzin, 2004). Under the nested logit model products are grouped into nests, and a customer first picks a nest and then a product inside it. Davis, Gallego and Topaloglu (Operations Research, 2014; DOI 10.1287/opre.2014.1256) map out how hard this problem is across variants of the model.

When every nest dissimilarity parameter is at most one and a customer who chose a nest always buys there, offering the top-revenue products of each nest is optimal (Theorem 4 of the paper, the subject of an earlier mission of this series). Once a customer may leave a nest without buying — a partially-captured nest — that structure breaks and the problem becomes NP-hard (Theorem 8). This mission targets the paper's response: a small, explicitly constructed family of candidate assortments per nest from which a linear program recovers a solution within a factor of two of optimal.

Setting

There are mmm nests MMM and, in each nest, nnn products N={1,…,n}N = \{1, \dots, n\}N={1,…,n}. Product jjj of nest iii has a revenue rij≥0r_{ij} \ge 0rij​≥0 and a preference weight vij>0v_{ij} > 0vij​>0, with ri1≥⋯≥rinr_{i1} \ge \dots \ge r_{in}ri1​≥⋯≥rin​. Nest iii has a dissimilarity parameter γi>0\gamma_i > 0γi​>0 and a within-nest no-purchase weight vi0≥0v_{i0} \ge 0vi0​≥0; v0≥0v_0 \ge 0v0​≥0 is the weight of choosing no nest. For an assortment Si⊆NS_i \subseteq NSi​⊆N,

Vi(Si)=vi0+∑j∈Sivij,Ri(Si)=∑j∈SirijvijVi(Si),Ri(∅)=0,V_i(S_i) = v_{i0} + \sum_{j \in S_i} v_{ij}, \qquad R_i(S_i) = \frac{\sum_{j \in S_i} r_{ij} v_{ij}}{V_i(S_i)},\quad R_i(\emptyset)=0,Vi​(Si​)=vi0​+j∈Si​∑​vij​,Ri​(Si​)=Vi​(Si​)∑j∈Si​​rij​vij​​,Ri​(∅)=0,

and the expected revenue of (S1,…,Sm)(S_1, \dots, S_m)(S1​,…,Sm​) is Π=∑iVi(Si)γiRi(Si)/(v0+∑iVi(Si)γi)\Pi = \sum_i V_i(S_i)^{\gamma_i} R_i(S_i) / (v_0 + \sum_i V_i(S_i)^{\gamma_i})Π=∑i​Vi​(Si​)γi​Ri​(Si​)/(v0​+∑i​Vi​(Si​)γi​). The optimal expected revenue Z∗Z^*Z∗ is the optimal value of the linear program

(3)min⁡ xs.t.v0x≥∑i∈Myi,yi≥Vi(Si)γi(Ri(Si)−x)  ∀Si⊆N, i∈M,\text{(3)}\quad \min\ x \quad\text{s.t.}\quad v_0 x \ge \sum_{i \in M} y_i,\qquad y_i \ge V_i(S_i)^{\gamma_i}\big(R_i(S_i) - x\big)\ \ \forall S_i \subseteq N,\ i \in M,(3)min xs.t.v0​x≥i∈M∑​yi​,yi​≥Vi​(Si​)γi​(Ri​(Si​)−x)  ∀Si​⊆N, i∈M,

and problem (4) is the same program with the second family of constraints imposed only for a chosen collection of candidate assortments in each nest.

Throughout, γi≤1\gamma_i \le 1γi​≤1 for every nest and the vi0v_{i0}vi0​ are arbitrary. For a capacity ϵi≥0\epsilon_i \ge 0ϵi​≥0, the knapsack value Ki(ϵi)K_i(\epsilon_i)Ki​(ϵi​) is the largest ∑j∈Srijvij\sum_{j \in S} r_{ij} v_{ij}∑j∈S​rij​vij​ over assortments SSS with ∑j∈Svij≤ϵi\sum_{j \in S} v_{ij} \le \epsilon_i∑j∈S​vij​≤ϵi​ (display (9)). Its continuous relaxation (11) allows fractional zij∈[0,1(vij≤ϵi)]z_{ij} \in [0, \mathbf 1(v_{ij} \le \epsilon_i)]zij​∈[0,1(vij​≤ϵi​)] under the same capacity. The greedy solution z^i(ϵi)\hat z_i(\epsilon_i)z^i​(ϵi​) of (11) fills the capacity with the products of weight at most ϵi\epsilon_iϵi​ in revenue order, each fully while it fits and the next one fractionally, and

S^i(ϵi)={j∈N:z^ij(ϵi)=1}.\hat S_i(\epsilon_i) = \{ j \in N : \hat z_{ij}(\epsilon_i) = 1 \}.S^i​(ϵi​)={j∈N:z^ij​(ϵi​)=1}.

Problem (10) replaces the per-assortment constraints of (3) by yi≥max⁡ϵi≥0(vi0+ϵi)γi[Ki(ϵi)/(vi0+ϵi)−x]y_i \ge \max_{\epsilon_i \ge 0} (v_{i0}+\epsilon_i)^{\gamma_i}[K_i(\epsilon_i)/(v_{i0}+\epsilon_i) - x]yi​≥maxϵi​≥0​(vi0​+ϵi​)γi​[Ki​(ϵi​)/(vi0​+ϵi​)−x].

Formalization targets

Goal: Theorem 10 (p. 24)

Let (x^,y^)(\hat x, \hat y)(x^,y^​) be an optimal solution of (4) with candidate collections {S^i(ϵi):ϵi∈[0,∞]}∪{{j}:j∈N}\{\hat S_i(\epsilon_i) : \epsilon_i \in [0,\infty]\} \cup \{\{j\} : j \in N\}{S^i​(ϵi​):ϵi​∈[0,∞]}∪{{j}:j∈N}. Then

(2x^, 2y^)  is feasible for (3).(2\hat x,\ 2\hat y) \ \text{ is feasible for (3).}(2x^, 2y^​)  is feasible for (3).

Milestones

  1. Per-nest identity (proof of Lemma 9, p. 23). For x≥0x \ge 0x≥0, max⁡SiVi(Si)γi(Ri(Si)−x)=max⁡ϵi≥0(vi0+ϵi)γi[Ki(ϵi)/(vi0+ϵi)−x]\max_{S_i} V_i(S_i)^{\gamma_i}(R_i(S_i) - x) = \max_{\epsilon_i \ge 0}(v_{i0}+\epsilon_i)^{\gamma_i}[K_i(\epsilon_i)/(v_{i0}+\epsilon_i) - x]maxSi​​Vi​(Si​)γi​(Ri​(Si​)−x)=maxϵi​≥0​(vi0​+ϵi​)γi​[Ki​(ϵi​)/(vi0​+ϵi​)−x].
  2. Lemma 9 (p. 23). Problems (3) and (10) have the same optimal solutions.
  3. Relaxation (p. 23). Every feasible point of (9) is feasible for (11), so K^i(ϵi)≥Ki(ϵi)\hat K_i(\epsilon_i) \ge K_i(\epsilon_i)K^i​(ϵi​)≥Ki​(ϵi​).
  4. Greedy solution (pp. 23–24). z^i(ϵi)\hat z_i(\epsilon_i)z^i​(ϵi​) is optimal for (11) and has at most one fractional component.
  5. Sign (A.3, p. 45). x^≥0\hat x \ge 0x^≥0.
  6. Inequalities (28) and (29) (A.3, pp. 45–46). In both cases — z^i(ϵ)\hat z_i(\epsilon)z^i​(ϵ) with and without a fractional component — 2y^i≥(vi0+ϵ)γi[Ki(ϵ)/(vi0+ϵ)−2x^]2\hat y_i \ge (v_{i0}+\epsilon)^{\gamma_i}[K_i(\epsilon)/(v_{i0}+\epsilon) - 2\hat x]2y^​i​≥(vi0​+ϵ)γi​[Ki​(ϵ)/(vi0​+ϵ)−2x^].

Two further statements accompany the goal: the factor-two revenue guarantee obtained from Theorem 10 and Theorem 1 of the paper, and the fact that every S^i(ϵi)\hat S_i(\epsilon_i)S^i​(ϵi​) is one of the at most 1+n21 + n^21+n2 assortments NijkN^k_{ij}Nijk​, the first jjj products by revenue among the kkk lightest.

Significance

Theorem 10 turns an NP-hard assortment problem into a linear program with 1+m1 + m1+m variables and 1+m(1+n+n2)1 + m(1 + n + n^2)1+m(1+n+n2) constraints whose solution is within a factor of two of optimal. The construction is explicit: the candidates are defined by a greedy rule, not by an optimization oracle. The same template, a restricted linear program whose doubled optimum is feasible for the full one, is reused in §6 of the paper for the most general instances, and Lemma 9's knapsack reformulation is the link to the classical approximation theory of knapsack problems (Williamson and Shmoys, 2011).

The theorem is proved in the paper. No machine-checked proof of it, of Lemma 9, or of greedy optimality for the continuous knapsack with an eligibility bound exists on the platform. Formalizing it yields a checked factor-two guarantee and a reusable fractional-knapsack development.

Difficulty

The obvious argument would compare the restricted program (4) with (3) constraint by constraint. That fails: (3) has one constraint per subset of products, and most subsets are not candidates. The comparison has to pass through the knapsack reformulation (10), which requires showing that a maximum over all subsets equals a maximum over a one-dimensional capacity parameter, using γi≤1\gamma_i \le 1γi​≤1 and x≥0x \ge 0x≥0 in an essential way. The second obstacle is that the greedy assortment S^i(ϵi)\hat S_i(\epsilon_i)S^i​(ϵi​) keeps only the fully taken products, so its value can fall short of the continuous knapsack value, and no single candidate assortment need attain the knapsack bound. With dissimilarity parameters above one the monotonicity behind the reformulation is lost, and §6 of the paper needs a different factor.

Formalization scope

Products are Fin n (indices 0,…,n−10, \dots, n-10,…,n−1), nests a finite type, and every quantity is real. Powers are Real.rpow; x/0=0x/0 = 0x/0=0, which gives Ri(∅)=0R_i(\emptyset) = 0Ri​(∅)=0. An optimal solution of a linear program is a feasible pair whose xxx is minimal among feasible pairs. The constraint "yi≥max⁡ϵi≥0(… )y_i \ge \max_{\epsilon_i \ge 0}(\dots)yi​≥maxϵi​≥0​(…)" of (10) is stated in constraint form, for every ϵi≥0\epsilon_i \ge 0ϵi​≥0, so no real supremum is taken. Ki(ϵ)K_i(\epsilon)Ki​(ϵ) is defined for ϵ≥0\epsilon \ge 0ϵ≥0 only; its placeholder value for ϵ<0\epsilon < 0ϵ<0 is never used. Ties in revenue (and, for NijkN^k_{ij}Nijk​, in weight) are broken by index. The candidate collection is taken over real ϵi≥0\epsilon_i \ge 0ϵi​≥0; ϵi=∞\epsilon_i = \inftyϵi​=∞ adds nothing, since every capacity of at least ∑jvij\sum_j v_{ij}∑j​vij​ already gives S^i=N\hat S_i = NS^i​=N.

Standing assumptions and added hypotheses: γi≤1\gamma_i \le 1γi​≤1 for every nest (the section's assumption) on the goal and on every model milestone; the pins vij>0v_{ij} > 0vij​>0, rij≥0r_{ij} \ge 0rij​≥0, γi>0\gamma_i > 0γi​>0 and the revenue ordering, shared by the series; n≥1n \ge 1n≥1 on Lemma 9, on x^≥0\hat x \ge 0x^≥0 and on (28)/(29), the paper's nonempty NNN; and v0>0v_0 > 0v0​>0 on the factor-two revenue guarantee, where Theorem 1 of the paper fails without it.

The greedy assortments S^i(ϵi)\hat S_i(\epsilon_i)S^i​(ϵi​) are defined explicitly. Quantifying over arbitrary optimal solutions of (11) instead would change the candidate collection and is not the paper's theorem. The goal states feasibility for the full program (3) and does not mention knapsack values, the greedy solution or the case split. A formalization that weakens the conclusion to feasibility for (10), or that drops the singletons from the candidate collection, is not a solution.

Needed infrastructure: fractional knapsack optimality of the greedy rule with an eligibility bound, monotonicity of t↦tγt \mapsto t^{\gamma}t↦tγ and t↦tγ−1t \mapsto t^{\gamma - 1}t↦tγ−1 for γ≤1\gamma \le 1γ≤1, and finite maximization over subsets. The fractional-knapsack lemmas are reusable beyond this mission. Proofs of any milestone, and alternative decompositions of the goal, are welcome.

Selected references

  • J. M. Davis, G. Gallego, H. Topaloglu, Assortment Optimization under Variants of the Nested Logit Model, Operations Research 62(2), 2014 (revised manuscript of June 18, 2013, cited here). DOI 10.1287/opre.2014.1256
  • K. T. Talluri, G. J. van Ryzin, Revenue Management Under a General Discrete Choice Model of Consumer Behavior, Management Science 50(1), 15–33, 2004. DOI 10.1287/mnsc.1030.0147
  • D. P. Williamson, D. B. Shmoys, The Design of Approximation Algorithms, Cambridge University Press, 2011. DOI 10.1017/CBO9780511921735
11 thms3 active usersReviewed
🏆Completed
Optimization·Captain: mikedeng1

Assortment Optimization under Variants of the Nested Logit Model 2: With Dissimilarity Parameters at Most One and Fully-Captured Nests, a Nested-by-Revenue Assortment in Every Nest Is OptimalResearch Paper

Motivation

A retailer choosing which products to display, or an airline choosing which fare classes to open, solves an assortment optimization problem: pick the set of offered products that maximizes expected revenue when customers choose among what is offered according to a discrete choice model. Under the multinomial logit model the answer has a simple form: an optimal assortment consists of the few highest-revenue products (Talluri and van Ryzin, 2004). The multinomial logit model, however, forces every pair of products to compete in the same way. The nested logit model relaxes this by grouping products into nests (brands, store sections, departure times) and letting a customer first choose a nest and then a product inside it.

Davis, Gallego and Topaloglu (Operations Research, 2014; DOI 10.1287/opre.2014.1256) study how much of the multinomial logit structure survives under the nested logit model. Their first answer is the theorem this mission targets: when the nest dissimilarity parameters are at most one and no customer who chose a nest leaves it without buying, offering the top products of each nest is still optimal. The other missions of this series treat the cases where this fails: dissimilarity parameters above one (the problem becomes NP-hard) and nests with their own no-purchase option.

Setting

There are mmm nests M={1,…,m}M = \{1, \dots, m\}M={1,…,m} and, in each nest, nnn products N={1,…,n}N = \{1, \dots, n\}N={1,…,n}. Product jjj of nest iii has a revenue rij≥0r_{ij} \ge 0rij​≥0 and a preference weight vij>0v_{ij} > 0vij​>0; products are ordered so that ri1≥ri2≥⋯≥rinr_{i1} \ge r_{i2} \ge \dots \ge r_{in}ri1​≥ri2​≥⋯≥rin​. Nest iii carries a dissimilarity parameter γi>0\gamma_i > 0γi​>0 and a within-nest no-purchase weight vi0≥0v_{i0} \ge 0vi0​≥0, and v0≥0v_0 \ge 0v0​≥0 is the weight of choosing no nest at all.

An assortment is a tuple (S1,…,Sm)(S_1, \dots, S_m)(S1​,…,Sm​) of subsets Si⊆NS_i \subseteq NSi​⊆N. Write

Vi(Si)=vi0+∑j∈Sivij,Ri(Si)=∑j∈SirijvijVi(Si),Ri(∅)=0.V_i(S_i) = v_{i0} + \sum_{j \in S_i} v_{ij}, \qquad R_i(S_i) = \frac{\sum_{j \in S_i} r_{ij} v_{ij}}{V_i(S_i)}, \quad R_i(\emptyset) = 0.Vi​(Si​)=vi0​+j∈Si​∑​vij​,Ri​(Si​)=Vi​(Si​)∑j∈Si​​rij​vij​​,Ri​(∅)=0.

A customer picks nest iii with probability Qi=Vi(Si)γi/(v0+∑l∈MVl(Sl)γl)Q_i = V_i(S_i)^{\gamma_i} / (v_0 + \sum_{l \in M} V_l(S_l)^{\gamma_l})Qi​=Vi​(Si​)γi​/(v0​+∑l∈M​Vl​(Sl​)γl​) and then, inside the nest, product jjj with probability vij/Vi(Si)v_{ij}/V_i(S_i)vij​/Vi​(Si​). The expected revenue is

Π(S1,…,Sm)=∑i∈MQi Ri(Si)=∑i∈MVi(Si)γiRi(Si)v0+∑i∈MVi(Si)γi,\Pi(S_1, \dots, S_m) = \sum_{i \in M} Q_i\, R_i(S_i) = \frac{\sum_{i \in M} V_i(S_i)^{\gamma_i} R_i(S_i)}{v_0 + \sum_{i \in M} V_i(S_i)^{\gamma_i}},Π(S1​,…,Sm​)=i∈M∑​Qi​Ri​(Si​)=v0​+∑i∈M​Vi​(Si​)γi​∑i∈M​Vi​(Si​)γi​Ri​(Si​)​,

and problem (2) asks for Z∗=max⁡Π(S1,…,Sm)Z^* = \max \Pi(S_1, \dots, S_m)Z∗=maxΠ(S1​,…,Sm​) over all assortments. The nested-by-revenue assortment Nij={1,…,j}N_{ij} = \{1, \dots, j\}Nij​={1,…,j} collects the jjj highest-revenue products of nest iii, with Ni0=∅N_{i0} = \emptysetNi0​=∅ and N+={0,1,…,n}N_+ = \{0, 1, \dots, n\}N+​={0,1,…,n}.

This mission works under the standing assumptions of §3 of the paper: competitive products, γi≤1\gamma_i \le 1γi​≤1, and fully-captured nests, vi0=0v_{i0} = 0vi0​=0, for every nest iii.

Formalization targets

Goal: Theorem 4 (p. 15)

If γi≤1\gamma_i \le 1γi​≤1 and vi0=0v_{i0} = 0vi0​=0 for all i∈Mi \in Mi∈M, there exists an optimal solution (S1∗,…,Sm∗)(S^*_1, \dots, S^*_m)(S1∗​,…,Sm∗​) of problem (2) such that

Si∗=Nij  for some j∈N+,for all i∈M.S^*_i = N_{ij} \ \text{ for some } j \in N_+, \qquad \text{for all } i \in M.Si∗​=Nij​  for some j∈N+​,for all i∈M.

Milestones

  1. The case v0=0v_0 = 0v0​=0 (p. 14). Offering only the single product with the largest revenue max⁡iri1\max_{i} r_{i1}maxi​ri1​ is optimal.
  2. Proposition 2 (p. 14). If S∗S^*S∗ is optimal and Si∗≠∅S^*_i \ne \emptysetSi∗​=∅, then Ri(Si∗)≥Z∗R_i(S^*_i) \ge Z^*Ri​(Si∗​)≥Z∗.
  3. Lemma 3 (p. 14). If Z=Π(S)Z = \Pi(S)Z=Π(S), Ri(Si)≥ZR_i(S_i) \ge ZRi​(Si​)≥Z and some j∈Sij \in S_ij∈Si​ has rij<γiZ+(1−γi)Ri(Si)r_{ij} < \gamma_i Z + (1-\gamma_i) R_i(S_i)rij​<γi​Z+(1−γi​)Ri​(Si​), removing jjj strictly increases the expected revenue.
  4. g(α)≤γg(\alpha) \le \gammag(α)≤γ (p. 15). For 0<γ≤10 < \gamma \le 10<γ≤1 and 0<α<10 < \alpha < 10<α<1: (1−αγ)/(αγ−1−αγ)≤γ(1 - \alpha^{\gamma})/(\alpha^{\gamma-1} - \alpha^{\gamma}) \le \gamma(1−αγ)/(αγ−1−αγ)≤γ.
  5. Revenue threshold (p. 15). Every j∈Si∗j \in S^*_ij∈Si∗​ of an optimal S∗S^*S∗ has rij≥γiZ∗+(1−γi)Ri(Si∗)r_{ij} \ge \gamma_i Z^* + (1-\gamma_i) R_i(S^*_i)rij​≥γi​Z∗+(1−γi​)Ri​(Si∗​).
  6. h(α)≥γh(\alpha) \ge \gammah(α)≥γ (p. 16). For 0<γ≤10 < \gamma \le 10<γ≤1 and 0<α<10 < \alpha < 10<α<1: (1−αγ)/(1−α)≥γ(1 - \alpha^{\gamma})/(1 - \alpha) \ge \gamma(1−αγ)/(1−α)≥γ.
  7. Exchange step (p. 15). If S∗S^*S∗ is optimal, j∈Si∗j \in S^*_ij∈Si∗​, k∉Si∗k \notin S^*_ik∈/Si∗​ and k<jk < jk<j, then adding kkk to Si∗S^*_iSi∗​ keeps the assortment optimal.

A companion item (not a milestone) states the algorithmic consequence at the end of §3: solving the linear program (4) over the candidates {Nij:j∈N+}\{N_{ij} : j \in N_+\}{Nij​:j∈N+​} and choosing in each nest a maximizer of problem (5) gives an optimal solution of (2).

Significance

Theorem 4 reduces problem (2), a search over 2mn2^{mn}2mn assortments, to (n+1)m(n+1)^m(n+1)m nested-by-revenue combinations, and the paper then finds the best one with a linear program with 1+m1 + m1+m variables and 1+m(1+n)1 + m(1+n)1+m(1+n) constraints. It marks the exact boundary of the classical multinomial logit structure inside the nested logit model: the paper's §4 shows that a single nest with γi>1\gamma_i > 1γi​>1 already breaks it, and that the general problem is NP-hard. The structural statement is also the base case for the approximation guarantees of §§5–6, which compare against nested-by-revenue assortments.

The theorem is proved in the paper; to our knowledge it has no machine-checked proof. A formal proof would supply a verified reduction from a combinatorial revenue maximization over the nested logit model to a polynomial-size search, with every boundary case (empty nests, v0=0v_0 = 0v0​=0, ties in revenues) handled explicitly.

Difficulty

The obvious argument copies the multinomial logit proof: take an optimal assortment and swap a low-revenue product for a missing higher-revenue one. Under the nested logit model this exchange changes the nest's attraction Vi(Si)γiV_i(S_i)^{\gamma_i}Vi​(Si​)γi​ non-linearly, so the revenue of the modified assortment is not an affine function of the change, and a simple swap can lower the expected revenue. The argument must instead control how adding or removing one product moves the nest weight relative to the nest revenue, and this is exactly where γi≤1\gamma_i \le 1γi​≤1 enters, through two scalar inequalities in the ratio α\alphaα of nest weights. With γi>1\gamma_i > 1γi​>1 these inequalities fail and so does the theorem.

A second subtlety is ties: several optimal assortments may exist, and only some of them are nested by revenue. The statement asserts existence, not that every optimum has this form.

Formalization scope

All statements live in the namespace NestedLogitVariants.Competitive and share one definition file. Nests form a finite type ι with decidable equality; products are Fin n, indexed 0,…,n−10, \dots, n-10,…,n−1, so NijN_{ij}Nij​ is nbr n j ={k:k<j}= \{k : k < j\}={k:k<j} with j≤nj \le nj≤n, and j=0j = 0j=0 gives ∅\emptyset∅. Powers are Real.rpow, and x/0=0x / 0 = 0x/0=0, which gives Ri(∅)=0R_i(\emptyset) = 0Ri​(∅)=0. Optimality of an assortment means its revenue is at least that of every assortment.

Standing assumptions carried as hypotheses: v0≥0v_0 \ge 0v0​≥0, vi0≥0v_{i0} \ge 0vi0​≥0, revenues ordered within each nest, and §3's γi≤1\gamma_i \le 1γi​≤1 and vi0=0v_{i0} = 0vi0​=0. Three pins are disclosed: vij>0v_{ij} > 0vij​>0 (the paper allows zero-weight padding products, under which Proposition 2 fails), rij≥0r_{ij} \ge 0rij​≥0, and γi>0\gamma_i > 0γi​>0 (the paper's γi≥0\gamma_i \ge 0γi​≥0; its convention Vi(∅)γi=0V_i(\emptyset)^{\gamma_i} = 0Vi​(∅)γi​=0 fails at γi=0\gamma_i = 0γi​=0). The section's "without loss of generality v0>0v_0 > 0v0​>0" is a hypothesis of Proposition 2, Lemma 3, the threshold, the exchange step and the LP item; the goal itself only assumes v0≥0v_0 \ge 0v0​≥0, and the case v0=0v_0 = 0v0​=0 is milestone 1. The two scalar inequalities are stated as inequalities, not as monotonicity claims.

The goal is not trivialized by any hypothesis: it assumes none of the milestones, and stating "some nested-by-revenue assortment exists" (always true) or "every optimal assortment is nested by revenue" (false under ties) would be a different theorem.

Needed infrastructure is light: finite sums, real powers, and concavity of x↦xγx \mapsto x^{\gamma}x↦xγ for γ≤1\gamma \le 1γ≤1. The scalar lemmas are reusable for other nested logit results. Proofs of any milestone are welcome independently.

Selected references

  • J. M. Davis, G. Gallego, H. Topaloglu, Assortment optimization under variants of the nested logit model, Operations Research 62(2), 250–273, 2014. https://doi.org/10.1287/opre.2014.1256 (cited from the authors' revised manuscript of June 18, 2013)
  • K. Talluri, G. van Ryzin, Revenue management under a general discrete choice model of consumer behavior, Management Science 50(1), 15–33, 2004. https://doi.org/10.1287/mnsc.1030.0147
  • D. McFadden, Modelling the choice of residential location, in A. Karlqvist et al. (eds.), Spatial Interaction Theory and Planning Models, North-Holland, 75–96, 1978.
9 thms3 active usersReviewed
🏆Completed
Machine LearningStatistics·Captain: mikedeng1

The Big Data Newsvendor: Practical Insights from Machine Learning: The L2-Regularized Feature-Based Newsvendor Rule Generalizes with a Bound Free of the Number of FeaturesResearch Paper

Motivation

The newsvendor problem is the basic model of inventory under uncertain demand. A decision maker orders qqq units before demand DDD is observed, and pays a unit backordering cost bbb for each unit of unmet demand and a unit holding cost hhh for each unit left over. When the demand distribution is known, the optimal order is a quantile of it. In practice the distribution is unknown and the decision maker holds historical data, often including features: observable covariates such as the day of the week, the weather or a recent sales trend, recorded alongside each past demand.

Rudin and Vahn (MIT Sloan Working Paper 5036-13, version of February 6, 2014, from MIT DSpace; published as Ban and Rudin, Operations Research 67(1), 2019, doi:10.1287/opre.2018.1757) propose to learn the order quantity directly as a linear function of the features, by minimizing the empirical newsvendor cost over the training data, with or without a regularization penalty. Their question is the one any data-driven decision rule must answer: how much worse can the learned rule do on new data than it did on the data it was fitted to? When the number of features ppp is comparable to the sample size nnn ("big data"), a bound that grows with ppp says nothing, and the paper's Theorem 2 gives one for the regularized rule that does not depend on ppp.

This mission formalizes that bound, Theorem 2 of the working paper, together with the results its proof is assembled from. All page and result numbers refer to the 2014 working paper, not to the published article.

Setting

Cost. For an order qqq and a demand ddd, the newsvendor cost is

C(q;d)=b (d−q)++h (q−d)+,b,h>0.C(q;d)=b\,(d-q)^+ + h\,(q-d)^+ ,\qquad b,h>0 .C(q;d)=b(d−q)++h(q−d)+,b,h>0.

Write b∨h=max⁡(b,h)b\vee h=\max(b,h)b∨h=max(b,h).

Data. A data point is a pair z=(x,d)z=(x,d)z=(x,d) of a feature vector x∈Rpx\in\mathbb R^px∈Rp and a demand d∈Rd\in\mathbb Rd∈R. Features lie in a domain X\mathcal XX inside the ball ∥x∥22≤Xmax⁡2\|x\|_2^2\le X_{\max}^2∥x∥22​≤Xmax2​, and demands lie in D=[0,Dˉ]\mathcal D=[0,\bar D]D=[0,Dˉ]. A sample Sn={(xi,di)}i=1nS_n=\{(x_i,d_i)\}_{i=1}^nSn​={(xi​,di​)}i=1n​ consists of nnn independent draws from an unknown probability distribution μ\muμ concentrated on X×D\mathcal X\times\mathcal DX×D.

Rules and risks. A vector q∈Rpq\in\mathbb R^pq∈Rp defines the linear decision rule q(x)=q⊤xq(x)=q^\top xq(x)=q⊤x. Its true risk and empirical risk are

Rtrue(q)=E(x,d)∼μ[C(q⊤x;d)],R^(q;Sn)=1n∑i=1nC(q⊤xi;di).R_{true}(q)=\mathbb E_{(x,d)\sim\mu}\bigl[C(q^\top x;d)\bigr],\qquad \hat R(q;S_n)=\frac1n\sum_{i=1}^n C(q^\top x_i;d_i).Rtrue​(q)=E(x,d)∼μ​[C(q⊤x;d)],R^(q;Sn​)=n1​i=1∑n​C(q⊤xi​;di​).

The regularized algorithm (NV-reg). For a parameter λ>0\lambda>0λ>0, the rule q^=q^(Sn)\hat q=\hat q(S_n)q^​=q^​(Sn​) minimizes

R^(q;Sn)+λ∥q∥22over q∈Rp.\hat R(q;S_n)+\lambda\|q\|_2^2\qquad\text{over } q\in\mathbb R^p .R^(q;Sn​)+λ∥q∥22​over q∈Rp.

The objective is strictly convex, so the minimizer is unique. Following Appendix B of the paper, the rules the algorithm outputs on samples from X×D\mathcal X\times\mathcal DX×D are assumed to map X\mathcal XX into D\mathcal DD (the paper's Q⊂DX\mathcal Q\subset\mathcal D^{\mathcal X}Q⊂DX): the learned order quantity is never negative and never exceeds the demand cap.

Uniform stability. An algorithm is uniformly stable with parameter αn\alpha_nαn​ if removing one observation from any sample changes the loss of its output at any test point by at most αn\alpha_nαn​ (Bousquet and Elisseeff's Definition 6, the paper's Definition 1).

Formalization targets

Goal: Theorem 2 (p. 9)

For every δ∈(0,1)\delta\in(0,1)δ∈(0,1) and n≥1n\ge1n≥1, with probability at least 1−δ1-\delta1−δ over SnS_nSn​,

∣Rtrue(q^)−R^(q^;Sn)∣≤(b∨h)2Xmax⁡2nλ+(2(b∨h)2Xmax⁡2λ+(b∨h)Dˉ)ln⁡(2/δ)2n.|R_{true}(\hat q)-\hat R(\hat q;S_n)|\le\frac{(b\vee h)^2X_{\max}^2}{n\lambda}+\Bigl(\frac{2(b\vee h)^2X_{\max}^2}{\lambda}+(b\vee h)\bar D\Bigr)\sqrt{\frac{\ln(2/\delta)}{2n}} .∣Rtrue​(q^​)−R^(q^​;Sn​)∣≤nλ(b∨h)2Xmax2​​+(λ2(b∨h)2Xmax2​​+(b∨h)Dˉ)2nln(2/δ)​​.

The dimension ppp appears nowhere in the bound.

Milestones, in the order of the proof (p. 32)

  1. Lemma 5 (p. 28). For q,d∈[0,Dˉ]q,d\in[0,\bar D]q,d∈[0,Dˉ], ∣C(q;d)∣≤(b∨h)Dˉ|C(q;d)|\le(b\vee h)\bar D∣C(q;d)∣≤(b∨h)Dˉ, and the bound is attained.
  2. Display (31) (p. 31). CCC is convex in its first argument and (b∨h)(b\vee h)(b∨h)-Lipschitz in it, i.e. (b∨h)(b\vee h)(b∨h)-admissible in the sense of Definition 2.
  3. Theorem 5 (p. 31), Bousquet and Elisseeff's Theorem 22: regularization in a reproducing kernel Hilbert space with a σ\sigmaσ-admissible loss and kernel bound κ2\kappa^2κ2 has uniform stability σ2κ2/(2λn)\sigma^2\kappa^2/(2\lambda n)σ2κ2/(2λn). This is an existing platform statement, referenced rather than restated.
  4. Theorem 4 (p. 30). (NV-reg) is uniformly stable with parameter αnr=(b∨h)2Xmax⁡2/(2nλ)\alpha_n^r=(b\vee h)^2X_{\max}^2/(2n\lambda)αnr​=(b∨h)2Xmax2​/(2nλ).
  5. Theorem 6 (p. 31). Any algorithm with uniform stability αn\alpha_nαn​ and loss in [0,M][0,M][0,M] satisfies, with probability at least 1−δ1-\delta1−δ,
∣Rtrue(A,Sn)−R^(A,Sn)∣≤2αn+(4nαn+M)ln⁡(2/δ)2n.|R_{true}(A,S_n)-\hat R(A,S_n)|\le2\alpha_n+(4n\alpha_n+M)\sqrt{\frac{\ln(2/\delta)}{2n}} .∣Rtrue​(A,Sn​)−R^(A,Sn​)∣≤2αn​+(4nαn​+M)2nln(2/δ)​​.

Significance

The result. Theorem 2 bounds the generalization gap of the regularized feature-based newsvendor rule at rate O(1/n)O(1/\sqrt n)O(1/n​) with constants that depend on the costs, the demand cap, the feature radius and λ\lambdaλ, but not on the number of features. It gives a theoretical basis for regularizing when p/np/np/n is not small, and it indicates how to scale λ\lambdaλ with the feature radius. The companion bound for the unregularized rule (Theorem 1) grows linearly in ppp.

Formalizing it. The theorem is proved in the paper, but its proof is short and relies on cited results: Theorem 5 and Theorem 6 are stated with references to Bousquet and Elisseeff and no proof of their own, and the paper's Theorem 6 is a two-sided variant of Bousquet and Elisseeff's Theorem 12 that is only sketched. None of these results has a machine-checked proof. A formalization produces a checked two-sided stability-to-generalization theorem for general algorithms (reusable for any stable learner), a checked stability bound for a regularized piecewise-linear loss, and the newsvendor bound itself, with the constants the proof actually supports.

Difficulty

The bound is not a uniform-convergence argument: the class of linear rules on Rp\mathbb R^pRp has complexity growing with ppp, so any bound that holds simultaneously for all rules in the class depends on ppp. The bound must exploit the specific rule produced by the algorithm. The two analytic steps that carry this are the stability of the regularized minimizer, which needs a strong-convexity comparison between the full and the leave-one-out objective, and a concentration inequality of bounded-differences type for a function of the whole sample whose differences are controlled only through stability.

The leave-one-out comparison is a known pitfall. If the leave-one-out problem is run literally on n−1n-1n−1 points it carries the weight 1/(n−1)1/(n-1)1/(n−1), and the comparison argument then gives twice the constant of Theorem 4. The stated constant holds for the leave-one-out objective that keeps the weight 1/n1/n1/n, which is the form of Theorem 5.

Formalization scope

Representation. Features are EuclideanSpace ℝ (Fin p), rules are vectors acting by the inner product, and data points live in EuclideanSpace ℝ (Fin p) × ℝ. The cost is the published newsboy loss with overage cost hhh and underage cost bbb; the empirical and true risks are the published empirical and generalization errors, and the (NV-reg) objective and its 1/n1/n1/n-weighted leave-one-out version are the published regularized objectives of Bousquet and Elisseeff. The regularizer is λ∥q∥22\lambda\|q\|_2^2λ∥q∥22​ (the display prints both λ∥q∥22\lambda\|q\|_2^2λ∥q∥22​ and λ∥q∥2\lambda\|q\|_2λ∥q∥2​; the text calls the problem a quadratic program). "With probability at least 1−δ1-\delta1−δ" is stated as: the event on which the gap exceeds the bound has measure at most δ\deltaδ under the product measure μn\mu^nμn.

Standing assumptions and pinned hypotheses.

  1. b,h,λ>0b,h,\lambda>0b,h,λ>0, Xmax⁡,Dˉ≥0X_{\max},\bar D\ge0Xmax​,Dˉ≥0, n≥1n\ge1n≥1, δ∈(0,1)\delta\in(0,1)δ∈(0,1).
  2. μ\muμ is a probability measure giving full mass to X×[0,Dˉ]\mathcal X\times[0,\bar D]X×[0,Dˉ], with ∥x∥22≤Xmax⁡2\|x\|_2^2\le X_{\max}^2∥x∥22​≤Xmax2​ on X\mathcal XX (§3, p. 9; the page writes the ball as ∥x∥22≤Xmax⁡\|x\|_2^2\le X_{\max}∥x∥22​≤Xmax​, while Theorem 2's "Xmax⁡2X_{\max}^2Xmax2​ as the largest possible value of ∥x∥22\|x\|_2^2∥x∥22​" and Theorem 5's note fix the reading).
  3. The algorithm is any map Sn↦q^(Sn)S_n\mapsto\hat q(S_n)Sn​↦q^​(Sn​) whose value minimizes the (NV-reg) objective, measurable in SnS_nSn​ (Appendix B: "all functions are measurable").
  4. The range assumption of Appendix B: for samples from X×D\mathcal X\times\mathcal DX×D, q^⊤x∈[0,Dˉ]\hat q^\top x\in[0,\bar D]q^​⊤x∈[0,Dˉ] for x∈Xx\in\mathcal Xx∈X.
  5. The last constant is (b∨h)Dˉ(b\vee h)\bar D(b∨h)Dˉ, the loss bound MMM from Lemma 5 that the proof feeds into Theorem 6; display (6) prints Dˉ\bar DDˉ there.
  6. The convention that all sets are countable, and the intercept convention x1=1x^1=1x1=1, are not imposed.

Trivializing formalization ruled out. The range assumption is quantified only over feature vectors in X\mathcal XX and over samples drawn from X×D\mathcal X\times\mathcal DX×D; stated over the whole ball ∥x∥2≤Xmax⁡\|x\|_2\le X_{\max}∥x∥2​≤Xmax​, it would force q^=0\hat q=0q^​=0 (both q^⊤x\hat q^\top xq^​⊤x and q^⊤(−x)\hat q^\top(-x)q^​⊤(−x) would lie in [0,Dˉ][0,\bar D][0,Dˉ]) and the goal would be nearly empty.

What is needed and reusable. McDiarmid's two-sided bounded-differences inequality under product measures; integrability of bounded measurable losses; existence and properties of minimizers of strongly convex objectives on Rp\mathbb R^pRp; the comparison argument behind Theorem 5. Theorem 6 is stated for an arbitrary data space and hypothesis space and is reusable for any uniformly stable algorithm. Contributions to the Theorem 5 reference and to McDiarmid's inequality benefit other missions as well.

Selected references

  • C. Rudin and G.-Y. Vahn, The Big Data Newsvendor: Practical Insights from Machine Learning, MIT Sloan School Working Paper 5036-13, version of February 6, 2014 (MIT DSpace). The version formalized here.
  • G.-Y. Ban and C. Rudin, The Big Data Newsvendor: Practical Insights from Machine Learning, Operations Research 67(1):90–108, 2019. https://doi.org/10.1287/opre.2018.1757
  • O. Bousquet and A. Elisseeff, Stability and Generalization, Journal of Machine Learning Research 2:499–526, 2002. https://jmlr.org/papers/v2/bousquet02a.html
  • C. McDiarmid, On the method of bounded differences, Surveys in Combinatorics, London Math. Soc. Lecture Note Series 141, 148–188, 1989. https://doi.org/10.1017/CBO9781107359949.008
13 thms3 active usersReviewed
🏆Completed
Convex OptimizationStatistics·Captain: mikedeng1

Conditional Logit Analysis of Qualitative Choice Behavior 3: The Conditional Logit Likelihood Has a Maximum Exactly When No Direction Makes Every Observed Choice Weakly BestResearch Paper

Motivation

The conditional logit model is the workhorse of discrete choice analysis in transportation, marketing, labour and industrial organization. McFadden's 1974 chapter derived it from a theory of population choice behaviour and showed how to estimate it by maximum likelihood; this line of work was recognized by his 2000 Nobel Prize in Economic Sciences, awarded for theory and methods of discrete choice analysis. Every applied logit estimation rests on a basic question: does the maximum likelihood estimate exist for the sample at hand? In small samples it may not. When one alternative is always chosen whenever it is available, the likelihood keeps increasing as a parameter tends to infinity, and numerical optimizers report diverging coefficients. This failure is known in the binary case as complete or quasi-complete separation. McFadden's Lemma 3 gives the exact condition, for the multinomial conditional logit model with general alternative sets, under which a maximizer exists.

Timeline. Berkson (1951, 1955) popularized binomial logit; multinomial versions were developed by Gurland (1960), Bloch (1967), Rassam (1971), McFadden (1968) and Theil (1969, 1970). McFadden (1974) stated the existence criterion for the conditional logit likelihood (Lemma 3) together with a quadratic-programming test for it (Lemma 4). Albert and Anderson (1984) later classified separation patterns for binary and multinomial logistic regression, and Haberman (1974) treated existence for log-linear models.

Setting

A choice experiment has N≥1N \ge 1N≥1 trials. Trial nnn offers an alternative set of JnJ_nJn​ alternatives, indexed i=1,…,Jni = 1,\dots,J_ni=1,…,Jn​, each described by an attribute vector zin∈RKz_{in} \in \mathbb{R}^Kzin​∈RK (the values of KKK specified functions of the individual's and the alternative's characteristics). Trial nnn is repeated Rn≥1R_n \ge 1Rn​≥1 times, and alternative iii is chosen SinS_{in}Sin​ times, so Rn=∑jSjnR_n = \sum_{j} S_{jn}Rn​=∑j​Sjn​.

For a parameter θ∈RK\theta \in \mathbb{R}^Kθ∈RK, with zinθz_{in}\thetazin​θ the inner product, the selection probabilities are

Pin(θ)=ezinθ∑j=1Jnezjnθ(16)P_{in}(\theta) = \frac{e^{z_{in}\theta}}{\sum_{j=1}^{J_n} e^{z_{jn}\theta}} \qquad (16)Pin​(θ)=∑j=1Jn​​ezjn​θezin​θ​(16)

and the log-likelihood of the sample is

L(θ)=C−∑n=1N∑i=1JnSinlog⁡∑j=1Jne(zjn−zin)θ,C=∑n=1N[log⁡Rn!−∑j=1Jnlog⁡Sjn!].(18)L(\theta) = C - \sum_{n=1}^N \sum_{i=1}^{J_n} S_{in} \log \sum_{j=1}^{J_n} e^{(z_{jn} - z_{in})\theta}, \qquad C = \sum_{n=1}^N \Big[\log R_n! - \sum_{j=1}^{J_n}\log S_{jn}!\Big]. \qquad (18)L(θ)=C−n=1∑N​i=1∑Jn​​Sin​logj=1∑Jn​​e(zjn​−zin​)θ,C=n=1∑N​[logRn​!−j=1∑Jn​​logSjn​!].(18)

Write zˉn(θ)=∑izinPin(θ)\bar z_n(\theta) = \sum_i z_{in}P_{in}(\theta)zˉn​(θ)=∑i​zin​Pin​(θ) for the probability-weighted mean attribute vector of trial nnn.

Axiom 5 (Full Rank). The (∑nJn)×K\big(\sum_n J_n\big)\times K(∑n​Jn​)×K matrix with rows zin−zˉnz_{in} - \bar z_nzin​−zˉn​ has rank KKK.

Axiom 6. There is no nonzero γ∈RK\gamma \in \mathbb{R}^Kγ∈RK with Sin(zjn−zin)γ≤0S_{in}(z_{jn} - z_{in})\gamma \le 0Sin​(zjn​−zin​)γ≤0 for all i,j=1,…,Jni, j = 1,\dots,J_ni,j=1,…,Jn​ and n=1,…,Nn = 1,\dots,Nn=1,…,N. Equivalently, no nonzero direction makes every observed choice weakly best in its alternative set.

Formalization targets

Goal: Lemma 3

Under Axiom 5,

(∃ θ^∈RK, ∀θ, L(θ)≤L(θ^))  ⟺  Axiom 6.\big(\exists\, \hat\theta \in \mathbb{R}^K,\ \forall \theta,\ L(\theta) \le L(\hat\theta)\big) \iff \text{Axiom 6}.(∃θ^∈RK, ∀θ, L(θ)≤L(θ^))⟺Axiom 6.

Milestones

  1. Equation (19): the gradient ∂L/∂θ=∑n∑j(Sjn−RnPjn)zjn\partial L/\partial\theta = \sum_n \sum_j (S_{jn} - R_nP_{jn}) z_{jn}∂L/∂θ=∑n​∑j​(Sjn​−Rn​Pjn​)zjn​.
  2. Equation (20): the Hessian ∂2L/∂θ ∂θ′=−∑nRn∑j(zjn−zˉn)′Pjn(zjn−zˉn)\partial^2L/\partial\theta\,\partial\theta' = -\sum_n R_n \sum_j (z_{jn} - \bar z_n)'P_{jn}(z_{jn} - \bar z_n)∂2L/∂θ∂θ′=−∑n​Rn​∑j​(zjn​−zˉn​)′Pjn​(zjn​−zˉn​).
  3. LLL is concave, and every critical point is a global maximizer.
  4. A Hessian that is nonsingular everywhere makes LLL strictly concave with at most one maximizer.
  5. Axiom 5 holds at θ\thetaθ if and only if the Hessian at θ\thetaθ is negative definite.
  6. Necessity: under Axiom 5, a maximizer forces Axiom 6.
  7. Equation (21): under Axiom 6, b(γ)=max⁡nmax⁡i,jSin(zjn−zin)γb(\gamma) = \max_n \max_{i,j} S_{in}(z_{jn}-z_{in})\gammab(γ)=maxn​maxi,j​Sin​(zjn​−zin​)γ has a positive lower bound b∗b^*b∗ on the unit sphere.
  8. The bound L(θ)−C≤−b∗∣θ∣L(\theta) - C \le -b^*|\theta|L(θ)−C≤−b∗∣θ∣ for all θ\thetaθ.
  9. Sufficiency: Axiom 6 gives a maximizer.

Significance

Lemma 3 tells the practitioner when the conditional logit maximum likelihood estimate exists, before any numerical optimization is attempted. It is a linear-inequality condition on the data alone, so it can be checked by linear or quadratic programming (Lemma 4 of the same paper). The existence of the estimator is also the first step of McFadden's asymptotic theory: Lemma 5 shows that Axiom 6 holds with probability tending to one, and Lemma 6, consistency and asymptotic normality, concerns the estimator whose existence Lemma 3 characterizes. The concavity and Hessian formulas (19)–(20) are the basis of the Newton–Raphson computation of the estimator and of its asymptotic covariance matrix.

The result has been proved since 1974 and is classical. To our knowledge it has no machine-checked proof; Mathlib has no statement about the existence of logit or softmax-regression maximum likelihood estimates. Formalizing it produces a verified existence criterion for the multinomial logit likelihood, verified gradient and Hessian formulas for log-sum-exp likelihoods with repeated observations, and a verified link between full column rank and strict concavity.

Difficulty

The likelihood is concave, and concave functions on RK\mathbb{R}^KRK need not attain their supremum. Concavity alone therefore gives nothing, and existence must come from a growth condition. The obvious approach, "the likelihood is bounded above by CCC, hence attains its maximum", fails: LLL is bounded but can approach its supremum only at infinity, which is exactly the separation case. Sufficiency needs a quantitative rate at which LLL decreases, uniform over all directions; a direction-by-direction argument does not suffice. Necessity requires strict concavity, which is where Axiom 5 and the requirement that every trial be observed enter. A trial with Rn=0R_n = 0Rn​=0 can supply the rank of Axiom 5 while contributing nothing to LLL, so with such a trial necessity fails. The calculus part, (19)–(20), involves differentiating sums of log-sum-exp terms over dependent index types and identifying the result with a weighted covariance operator.

Formalization scope

  • Representation. RK\mathbb{R}^KRK is EuclideanSpace ℝ (Fin K), so ∣θ∣=(θ′θ)1/2|\theta| = (\theta'\theta)^{1/2}∣θ∣=(θ′θ)1/2 is the Euclidean norm and zθz\thetazθ is the inner product ⟪z, θ⟫. Trials are Fin N, alternatives of trial nnn are Fin (J n), and the counts SinS_{in}Sin​ are natural numbers.
  • Data structure. The structure Data K bundles NNN, JJJ, zzz, SSS and the standing assumptions N≥1N \ge 1N≥1 and Rn=∑iSin≥1R_n = \sum_i S_{in} \ge 1Rn​=∑i​Sin​≥1 for every trial; these make the trial and alternative index sets nonempty.
  • Axioms 1–4 are built in. The model is the logit form (16) with vvv linear in θ\thetaθ (Axiom 4), so "Suppose Axioms 1–5 hold" becomes "Data plus Axiom 5".
  • Axiom 5 is read at every θ\thetaθ. The row space of the matrix does not depend on θ\thetaθ.
  • Hessian. The Hessian is the Fréchet derivative of the gradient vector field (19), as a continuous linear map.
  • The maximizer is global over all of RK\mathbb{R}^KRK. Neither a local maximizer nor "L(θ^)≥L(0)L(\hat\theta) \ge L(0)L(θ^)≥L(0)" is acceptable as the goal; that would make it trivial.
  • Infrastructure. Gradients and Hessians of log-sum-exp with dependent finite index types; positive definiteness from full column rank; attainment of the maximum of a coercive continuous function on a finite-dimensional space. The calculus lemmas are reusable for any multinomial logit or softmax likelihood. Missions 4 and 5 of this series reuse the same model. Contributions of general log-sum-exp lemmas, independent of this mission's definitions, are welcome.

Selected references

  • D. McFadden, Conditional logit analysis of qualitative choice behavior, in P. Zarembka (ed.), Frontiers in Econometrics, Academic Press, New York, 1974, pp. 105–142. https://eml.berkeley.edu/reprints/mcfadden/zarembka.pdf
  • A. Albert and J. A. Anderson, On the existence of maximum likelihood estimates in logistic regression models, Biometrika 71(1), 1984, pp. 1–10. https://doi.org/10.1093/biomet/71.1.1
  • S. J. Haberman, The Analysis of Frequency Data, University of Chicago Press, 1974.
  • J. Berkson, Maximum likelihood and minimum χ² estimates of the logistic function, Journal of the American Statistical Association 50, 1955, pp. 130–162. https://doi.org/10.1080/01621459.1955.10501255
12 thms3 active usersReviewed
🏆Completed
Markov ChainProbabilityStochastic Systems·Captain: mikedeng1

Reversibility and Stochastic Networks IX: Partial Balance — Equivalent Characterizations via Truncation, Rate Changes and Time ReversalTextbook

Why partial balance

Equilibrium distributions of Markov models of networks are rarely computed by solving the full equilibrium equations directly. In the classical product-form results (migration processes, Jackson and Kelly networks, loss networks, clustering processes) the equilibrium distribution satisfies a stronger, local family of equations, and that is what makes it computable. The strongest such family is detailed balance, which characterizes reversibility. Many models that are not reversible still satisfy an intermediate family, partial balance: the probability flux balances not pair by pair but across a chosen set of transitions. F. P. Kelly's Reversibility and Stochastic Networks (Wiley, 1979) uses partial balance throughout: quasi-reversibility in Chapter 3 is a form of it, and the models that display it tend to be insensitive, meaning their equilibrium distribution does not change when exponential holding times are replaced by general ones with the same mean.

Section 9.4 of the book collects what partial balance means in a single statement. Theorem 9.5 summarizes Exercises 1.6.2–1.6.4 and 1.7.7–1.7.8 and gives five operational characterizations of partial balance. Corollaries 9.6–9.8 translate them into the language of spatial processes, and Theorem 9.9 turns partial balance of a coarse description into a product-form equilibrium for a finer one. This last step is the mechanism behind insensitivity.

Setting

A Markov process on a state space S\mathcal SS has transition rates q(j,k)≥0q(j,k)\ge0q(j,k)≥0 for j≠kj\ne kj=k, with q(j,j)=0q(j,j)=0q(j,j)=0. It is irreducible: every state can be reached from every other through transitions of positive rate. An equilibrium distribution is a collection of positive numbers π(j)\pi(j)π(j) summing to one that satisfies the equilibrium equations

π(j)∑k∈Sq(j,k)=∑k∈Sπ(k)q(k,j),j∈S.\pi(j)\sum_{k\in\mathcal S}q(j,k)=\sum_{k\in\mathcal S}\pi(k)q(k,j),\qquad j\in\mathcal S.π(j)k∈S∑​q(j,k)=k∈S∑​π(k)q(k,j),j∈S.

For an irreducible process on a finite state space it exists and is unique.

Given a set A⊆S\mathcal A\subseteq\mathcal SA⊆S, π\piπ satisfies partial balance with respect to A\mathcal AA if

π(j)∑k∈Aq(j,k)=∑k∈Aπ(k)q(k,j),j∈A.\pi(j)\sum_{k\in\mathcal A}q(j,k)=\sum_{k\in\mathcal A}\pi(k)q(k,j),\qquad j\in\mathcal A.π(j)k∈A∑​q(j,k)=k∈A∑​π(k)q(k,j),j∈A.

Truncating the process to A\mathcal AA deletes every transition out of A\mathcal AA, and the result is required to be irreducible within A\mathcal AA. The time-reversed process of a process with equilibrium distribution π\piπ has rates π(k)q(k,j)/π(j)\pi(k)q(k,j)/\pi(j)π(k)q(k,j)/π(j).

A spatial process has JJJ sites, the vertices of a graph GGG. Site jjj carries an attribute njn_jnj​ from a finite set Nj\mathcal N_jNj​, and the state space is S=N1×⋯×NJ\mathcal S=\mathcal N_1\times\cdots\times\mathcal N_JS=N1​×⋯×NJ​. Write TjmnT_j^m\mathbf nTjm​n for the state n\mathbf nn with the attribute of site jjj changed to mmm. The process must satisfy three conditions: only one site changes at a time; the rate q(n,Tjmn)q(\mathbf n,T_j^m\mathbf n)q(n,Tjm​n) depends on the other sites only through the neighbours of jjj; and any TjmnT_j^m\mathbf nTjm​n can be reached from n\mathbf nn without changing the other sites. The conditional distribution of site jjj given the rest is P(nj∣nG−j)=π(n)/∑mπ(Tjmn)P(n_j\mid\mathbf n_{G-j})=\pi(\mathbf n)/\sum_m\pi(T_j^m\mathbf n)P(nj​∣nG−j​)=π(n)/∑m​π(Tjm​n). A Markov field is a positive distribution whose conditional distributions depend only on the neighbours of jjj.

Formalization targets

Goal: Theorem 9.5

For an irreducible process on a finite S\mathcal SS with equilibrium distribution π\piπ, a nonempty A\mathcal AA within which the truncated process is irreducible, and a constant c>0c>0c>0, c≠1c\ne1c=1, the following are equivalent:

  1. partial balance with respect to A\mathcal AA;
  2. the equilibrium distribution of the truncated process is π(j)/∑k∈Aπ(k)\pi(j)/\sum_{k\in\mathcal A}\pi(k)π(j)/∑k∈A​π(k);
  3. multiplying the rates q(j,k)q(j,k)q(j,k), j,k∈Aj,k\in\mathcal Aj,k∈A, by ccc leaves the equilibrium distribution unchanged;
  4. multiplying the rates q(j,k)q(j,k)q(j,k), j∈Aj\in\mathcal Aj∈A, k∉Ak\notin\mathcal Ak∈/A, by ccc changes the equilibrium distribution to
Bπ(j) (j∈A),Bcπ(j) (j∉A),B−1=∑j∈Aπ(j)+c∑j∉Aπ(j);B\pi(j)\ (j\in\mathcal A),\qquad Bc\pi(j)\ (j\notin\mathcal A),\qquad B^{-1}=\sum_{j\in\mathcal A}\pi(j)+c\sum_{j\notin\mathcal A}\pi(j);Bπ(j) (j∈A),Bcπ(j) (j∈/A),B−1=j∈A∑​π(j)+cj∈/A∑​π(j);
  1. time reversal and truncation to A\mathcal AA commute.

When S−A\mathcal S-\mathcal AS−A is nonempty, these are also equivalent to:

  1. the chain observed just before each exit from A\mathcal AA and the chain observed just after each entry into A\mathcal AA have the same equilibrium distribution.

Milestones

  • Corollary 9.6: the equivalences for the sets on which all sites but jjj are frozen, i.e. partial balance (9.26) at a site.
  • Corollary 9.7: on a state space with at least two states, partial balance at every site makes π\piπ a Markov field, with 0<π(n)<10<\pi(\mathbf n)<10<π(n)<1 as in the book's definition.
  • Corollary 9.8: the equivalences for the set on which site jjj is frozen at one attribute (9.27).
  • Theorem 9.9: if a reduced description n=f(x)\mathbf n=f(\mathbf x)n=f(x) has a distribution π(n)\pi(\mathbf n)π(n) in partial balance for the rates (9.31), then the finer process has equilibrium distribution π(x)=π(n)∏jPj(xj∣nj)\pi(\mathbf x)=\pi(\mathbf n)\prod_jP_j(x_j\mid n_j)π(x)=π(n)∏j​Pj​(xj​∣nj​).

What the results give

Theorem 9.5 makes partial balance testable by operations on the process itself: truncation, speeding up or slowing down transitions, and time reversal. The book points to close relationships between statement (iv) and the product form of Section 2.3, and between statement (ii) and part (iii) of Theorem 3.12. Corollary 9.6 (iv) explains why the reversed migration process has such a simple form. Corollary 9.7 strengthens Theorem 9.3 by replacing reversibility with partial balance at each site. Theorem 9.9 is the step from partial balance to insensitivity: it is what the book uses to show that a spatial process keeps its equilibrium distribution when the lifetimes of attributes are mixtures of gamma distributions.

All of these results were proved in 1979. None has a machine-checked proof. The platform has the reversible special case of statement (ii) (KellyStochasticNetworks.truncated_reversible) and the rate-level objects for time reversal and truncation, which this mission reuses. Formalizing Theorem 9.5 also produces a reusable account of embedded exit and entry chains of a finite Markov process.

Difficulty

Most of the equivalences (i)–(v) are short manipulations of the equilibrium equations, but each direction from a property of an altered process back to partial balance needs uniqueness of equilibrium distributions for irreducible finite processes, and (v) ⇒ (i) also needs their existence. Mathlib has neither in the form needed here. Statement (vi) is a different kind of claim. The equilibrium distribution of the exit chain is proportional to the exit flux π(j)∑k∉Aq(j,k)\pi(j)\sum_{k\notin\mathcal A}q(j,k)π(j)∑k∈/A​q(j,k), and that of the entry chain to the entry flux. Proving this requires the hitting distributions and the Green's function of the jump chain killed on leaving a set, and the convergence of the series that define them. Corollary 9.8 (v) inherits that work. Theorem 9.9 requires uniqueness for the reduced frozen processes and careful bookkeeping of the fibres {xj:fj(xj)=nj}\{x_j:f_j(x_j)=n_j\}{xj​:fj​(xj​)=nj​}.

Formalization scope

State spaces are finite types, and every sum is an unconditional sum over a finite type. Because the state space is finite, the book's extra condition for (vi), a finite flux out of A\mathcal AA, holds automatically. Rates are real functions with q(j,j)=0q(j,j)=0q(j,j)=0 built into the hypotheses; the reduced rates of (9.31) also have zero self-rates. "The equilibrium distribution of a process is XXX" means: XXX is positive, sums to one, satisfies the equilibrium equations, and every distribution with these properties equals XXX. The book's "c≠0c\ne0c=0 or 111" is read as c>0c>0c>0, c≠1c\ne1c=1, so that altered rates remain rates. Truncation to A\mathcal AA is the published truncatedRates, a process on the subtype A\mathcal AA, and the reversed rates are the published reversedRates. The exit and entry chains are defined from the jump chain through series of restricted matrix powers. Spatial processes live on ∏jNj\prod_j\mathcal N_j∏j​Nj​ with TjmT_j^mTjm​ given by Function.update. In Theorem 9.9 the graph is complete, as the book assumes from p. 202 on.

The equilibrium distribution of the truncated process must be the unique positive normalized solution of the truncated equilibrium equations. It must not be defined as the conditional distribution, which would make (ii) a tautology. For the same reason (iii) and (iv) are stated through the equilibrium equations of the altered rates, not by assumption.

Theorem 9.10 (p. 207) is not a target. Its hypothesis, that a nominal lifetime "can have any distribution with unit mean", is not defined on the page, and the point-process and lifetime description it needs lies outside this rate-level development.

Contributions are welcome at every level: uniqueness and existence of equilibrium distributions for irreducible finite rate matrices (reusable across the series), convergence of the killed Green's function, and proofs of the corollaries from the goal.

Selected references

  • F. P. Kelly, Reversibility and Stochastic Networks, Wiley, Chichester, 1979; reissued Cambridge University Press, 2011. https://doi.org/10.1017/CBO9781139171724 (§9.4, pp. 200–208; §1.6, pp. 25–27)
  • F. P. Kelly and E. Yudovina, Stochastic Networks, Cambridge University Press, 2014. https://doi.org/10.1017/CBO9781139565363
11 thms3 active usersReviewed
🏆Completed
ProbabilityStatistics·Captain: mikedeng1

Conditional Logit Analysis of Qualitative Choice Behavior 2: Random Utility Maximizers Choose by Logit Exactly When Taste Shocks Are Extreme-Value DistributedResearch Paper

Motivation

The conditional logit model assigns to an alternative iii in a finite choice set the probability eVi/∑jeVje^{V_i}/\sum_j e^{V_j}eVi​/∑j​eVj​, where VjV_jVj​ is a "representative utility" built from observed attributes of the alternative and the decision maker. It is the workhorse of discrete choice econometrics, transportation demand forecasting, marketing and revenue management, where it underlies multinomial logit assortment and pricing models. Its appeal for applied work is computational; its appeal for economics is that it can be read as the aggregate behaviour of a population of utility maximizers. This mission formalizes the result that makes that reading exact: Lemmas 1 and 2 of D. McFadden, Conditional logit analysis of qualitative choice behavior (1974), which show that, under a mild regularity condition, logit choice probabilities arise from random utility maximization exactly when the idiosyncratic taste shocks follow the extreme value (Gumbel) distribution.

Timeline:

  • 1959. J. Marschak gives a nonconstructive proof that i.i.d. extreme value shocks yield logit probabilities; R. D. Luce's choice axiom appears the same year.
  • 1965. Luce and Suppes publish the constructive argument, attributed to E. Holman and A. Marley, that is reproduced as the proof of Lemma 1.
  • 1974. McFadden proves the converse (Lemma 2): if i.i.d. shocks with a translation complete distribution produce logit probabilities, the distribution is extreme value.
  • Later. The random utility characterization was extended to correlated shocks (generalized extreme value models, McFadden 1978), which are not part of this mission.

Setting

An individual faces J≥1J \ge 1J≥1 alternatives with representative utilities V1,…,VJ∈RV_1, \dots, V_J \in \mathbb{R}V1​,…,VJ​∈R. The utility of alternative jjj is Uj=Vj+εjU_j = V_j + \varepsilon_jUj​=Vj​+εj​, where the taste shocks ε1,…,εJ\varepsilon_1, \dots, \varepsilon_Jε1​,…,εJ​ are independent and identically distributed with a common law μ\muμ on R\mathbb{R}R and distribution function G(t)=μ((−∞,t])G(t) = \mu((-\infty, t])G(t)=μ((−∞,t]). The individual chooses the alternative of highest utility, so the selection probability of iii is (Equation (2) of the paper)

Pi(V)=Pr⁡[εj−εi<Vi−Vj  for all j≠i],P_i(V) = \Pr\big[\varepsilon_j - \varepsilon_i < V_i - V_j \ \text{ for all } j \ne i\big],Pi​(V)=Pr[εj​−εi​<Vi​−Vj​  for all j=i],

computed under the product law of the shocks. The logit formula (Equation (12)) is Li(V)=eVi/∑j=1JeVjL_i(V) = e^{V_i}/\sum_{j=1}^J e^{V_j}Li​(V)=eVi​/∑j=1J​eVj​. The extreme value law (Equation (13)) is G(ε)=e−e−εG(\varepsilon) = e^{-e^{-\varepsilon}}G(ε)=e−e−ε.

A law μ\muμ is translation complete if for every function hhh of bounded total variation on R\mathbb{R}R with h(±∞)=0h(\pm\infty) = 0h(±∞)=0, the condition ∫h(e+a) dμ(e)=0\int h(e + a)\, d\mu(e) = 0∫h(e+a)dμ(e)=0 for every real aaa forces h=0h = 0h=0 outside a Lebesgue-null set. Laws whose characteristic function never vanishes, the extreme value law among them, are translation complete (footnote 5 of the paper).

In Lean the law is μ : Measure ℝ with [IsProbabilityMeasure μ], GGG is ProbabilityTheory.cdf μ, the selection probability is selProb μ V i for V : Fin J → ℝ, and the logit formula is logitProb V i.

Formalization targets

Goal: the characterization

Fix a universe XXX of alternatives with a representative utility map u:X→Ru:X\to\mathbb Ru:X→R onto the real line. For a translation complete law μ\muμ normalized by G(0)=e−1G(0) = e^{-1}G(0)=e−1,

(for every finite B⊆X, i∈B: Pi(B)=eu(i)∑j∈Beu(j))  ⟺  (∀ε∈R: G(ε)=e−e−ε).\Big(\text{for every finite }B\subseteq X,\ i\in B:\ P_i(B) = \frac{e^{u(i)}}{\sum_{j\in B} e^{u(j)}}\Big) \iff \Big(\forall \varepsilon \in \mathbb{R}:\ G(\varepsilon) = e^{-e^{-\varepsilon}}\Big).(for every finite B⊆X, i∈B: Pi​(B)=∑j∈B​eu(j)eu(i)​)⟺(∀ε∈R: G(ε)=e−e−ε).

The normalization only fixes the location of the shocks: without it the conclusion is the one-parameter family of Lemma 2 below.

Milestones

  1. Equation (3) for i.i.d. shocks without atoms: Pi(V)=∫∏j≠iG(ε+Vi−Vj) dG(ε)P_i(V) = \int \prod_{j \ne i} G(\varepsilon + V_i - V_j)\, dG(\varepsilon)Pi​(V)=∫∏j=i​G(ε+Vi​−Vj​)dG(ε).
  2. The integrand of Lemma 1's proof: under (13), the density times the other distribution functions equals e−εexp⁡(−e−ε∑jeVj−Vi)e^{-\varepsilon} \exp\big(-e^{-\varepsilon} \sum_j e^{V_j - V_i}\big)e−εexp(−e−ε∑j​eVj​−Vi​).
  3. Lemma 1: extreme value shocks give Pi(V)=Li(V)P_i(V) = L_i(V)Pi​(V)=Li​(V) for every JJJ and VVV.
  4. The functional equation of Lemma 2's proof: G(v−log⁡K)=G(v)KG(v - \log K) = G(v)^KG(v−logK)=G(v)K for every positive integer KKK and real vvv.
  5. Values at logarithms of rationals: with α=−log⁡G(0)\alpha = -\log G(0)α=−logG(0), α>0\alpha > 0α>0 and G(log⁡(K/L))=e−αL/KG(\log(K/L)) = e^{-\alpha L / K}G(log(K/L))=e−αL/K for positive integers K,LK, LK,L.
  6. Lemma 2: G(ε)=e−αe−εG(\varepsilon) = e^{-\alpha e^{-\varepsilon}}G(ε)=e−αe−ε for some α>0\alpha > 0α>0, and G(0)=e−1G(0) = e^{-1}G(0)=e−1 gives (13).

Significance

The result separates two readings of the logit formula. Lemma 1 shows that it is consistent with utility maximization; Lemma 2 shows that, within the class of i.i.d. additive random utility models with translation complete shocks, the extreme value law is the only one consistent with it. Consequences drawn from the random utility reading, such as the log-sum formula for expected maximum utility used in welfare analysis, therefore apply to logit models without further distributional assumptions inside that class. The same reading supports the interpretation of multinomial logit demand in assortment optimization and revenue management.

Both lemmas are proved in the paper and in later textbooks; neither is open. As far as a search of the Prove2Me catalog shows, neither has a machine-checked proof. The mission provides Lean statements of the random utility model with i.i.d. shocks, of translation completeness and of the extreme value law that later discrete choice formalizations can reuse.

Difficulty

Lemma 1 is a computation with the extreme value density; its formal cost lies in passing from the product-measure probability (2) to the iterated integral (3) and evaluating an improper integral. Lemma 2 is harder. The natural first idea is to differentiate the logit identity in the utilities and solve a differential equation for GGG; this requires a density, which Lemma 2 does not assume. Without a density, the only handle on GGG is the logit identity itself, an equality of integrals against dGdGdG that holds for every utility vector; turning such integral identities into pointwise information about GGG is where the hypothesis of translation completeness enters, and it yields statements only outside a Lebesgue-null set, so one-sided continuity of distribution functions is needed to recover identities at every point. A second subtlety is that the paper's (14) is written with GGG while the event (2) is strict, so with a general law the integrals involve left limits of GGG.

Formalization scope

Alternatives are indexed by Fin J; the model is indexed by the utility vector, so the individual attributes sss and alternative attributes xjx_jxj​ enter only through VVV. The shocks have joint law Measure.pi (fun _ => μ), which is what "independently identically distributed" means; a general joint law is not allowed. The event in (2) uses strict inequalities, and the selection probability is defined for every law, with no density. Translation completeness quantifies over BoundedVariationOn h Set.univ with limits 000 at both ends; such hhh are bounded and measurable, so the integrals are genuine, and "measure zero" is Lebesgue measure.

Lemma 2's hypothesis is stated on every finite subset of the paper's alternative universe, with a surjective utility map. Distinct alternatives may have the same utility. The printed proof uses KKK equal-utility alternatives, which surjectivity alone need not supply; proving the stated theorem requires an additional continuity argument. A trivializing formalization is ruled out: the selection probability is a genuine product-measure probability, and the hypotheses of the goal are met by the extreme value law, which is translation complete with G(0)=e−1G(0) = e^{-1}G(0)=e−1.

A complete development needs: Fubini for Measure.pi over Fin J split at one coordinate; the Gumbel density and the improper integral ∫e−εe−ce−εdε=1/c\int e^{-\varepsilon} e^{-c e^{-\varepsilon}} d\varepsilon = 1/c∫e−εe−ce−εdε=1/c; the facts that bounded-variation functions are bounded and measurable and that distribution functions are right-continuous with left limits. The integral representation (milestone 1) and the Gumbel computations are reusable in any random utility formalization. Proofs of the milestones, of the footnote-5 fact that the extreme value law is translation complete, and alternative proofs of Lemma 2 are welcome.

Selected references

  • D. McFadden, Conditional logit analysis of qualitative choice behavior, in P. Zarembka (ed.), Frontiers in Econometrics, Academic Press, New York, 1974, pp. 105–142.
  • J. Marschak, Binary choice constraints and random utility indicators, in K. Arrow, S. Karlin, P. Suppes (eds.), Mathematical Methods in the Social Sciences, Stanford University Press, 1960 (Stanford Symposium, 1959).
  • R. D. Luce and P. Suppes, Preference, utility, and subjective probability, in R. D. Luce, R. Bush, E. Galanter (eds.), Handbook of Mathematical Psychology, Vol. III, Wiley, 1965.
  • R. D. Luce, Individual Choice Behavior: A Theoretical Analysis, Wiley, 1959.
  • W. Feller, An Introduction to Probability Theory and Its Applications, Vol. II, Wiley, 1966, p. 479.
  • D. McFadden, Modelling the choice of residential location, in A. Karlqvist et al. (eds.), Spatial Interaction Theory and Planning Models, North-Holland, 1978, pp. 75–96.
8 thms3 active usersReviewed
🏆Completed
Graph TheoryMarkov ChainProbability+2·Captain: mikedeng1

Reversibility and Stochastic Networks VIII: Markov Fields — A Positive Random Field Is Markov iff It Factorizes over the Simplices of the GraphTextbook

Motivation

Many systems consist of a finite number of sites whose states influence one another only locally: fruit trees in an orchard that are diseased or healthy, power sources that are working or broken, individuals holding one of several views. Chapter 9 of F. P. Kelly, Reversibility and Stochastic Networks (Wiley, 1979) asks which joint distributions such systems have in equilibrium. Earlier chapters of the book produce product-form distributions in which the components are independent; spatial models instead give a limited dependence, and §9.1 makes that notion precise through Markov fields.

The central characterization, that a positive random field is Markov with respect to a graph exactly when it factorizes over the cliques of that graph, is the theorem of Hammersley and Clifford (1971, unpublished manuscript), with published proofs by Besag (1974), Grimmett (1973) and Preston (1973). It underlies Gibbs random fields in statistical mechanics, spatial statistics, image analysis and graphical models. Kelly's §§9.2–9.3 then use it to identify the equilibrium distributions of interacting-particle Markov processes ("spatial processes"), connecting it to reversibility and partial balance.

Setting

There are JJJ sites, the vertices of a finite graph GGG; ∂j\partial j∂j is the set of neighbours of site jjj and G−jG-jG−j the set of sites other than jjj. Site jjj carries an attribute njn_jnj​ from a finite set Nj\mathcal N_jNj​, and a state is n=(n1,…,nJ)\mathbf n=(n_1,\dots,n_J)n=(n1​,…,nJ​) in S=N1×⋯×NJ\mathcal S=\mathcal N_1\times\cdots\times\mathcal N_JS=N1​×⋯×NJ​. For a set of sites HHH, nH\mathbf n_HnH​ is the vector of attributes of the sites in HHH. The operator TjmT_j^mTjm​ changes the attribute of site jjj to mmm.

A random field is a function π\piπ on S\mathcal SS with π(n)>0\pi(\mathbf n)>0π(n)>0 for every state and ∑nπ(n)=1\sum_{\mathbf n}\pi(\mathbf n)=1∑n​π(n)=1. The conditional probability that site jjj has attribute njn_jnj​ given all other sites is

P(nj∣nG−j)=π(n)∑m∈Njπ(Tjmn).(9.1)P(n_j\mid\mathbf n_{G-j}) = \frac{\pi(\mathbf n)}{\sum_{m\in\mathcal N_j}\pi(T_j^m\mathbf n)}. \qquad (9.1)P(nj​∣nG−j​)=∑m∈Nj​​π(Tjm​n)π(n)​.(9.1)

π\piπ is a Markov field if P(nj∣nG−j)=P(nj∣n∂j)P(n_j\mid\mathbf n_{G-j}) = P(n_j\mid\mathbf n_{\partial j})P(nj​∣nG−j​)=P(nj​∣n∂j​) for every jjj and n\mathbf nn (9.2): the attribute of a site depends on the rest of the system only through its neighbours. A simplex is a single site or a set of sites any two of which are neighbours; C\mathcal CC is the set of simplices of GGG.

A spatial process is a Markov process n(t)\mathbf n(t)n(t) on S\mathcal SS with rates qqq such that (i) only one component changes at a time, (ii) q(n,Tjmn)q(\mathbf n,T_j^m\mathbf n)q(n,Tjm​n) depends on n\mathbf nn only through njn_jnj​ and n∂j\mathbf n_{\partial j}n∂j​, and (iii) TjmnT_j^m\mathbf nTjm​n can be reached from n\mathbf nn by transitions that do not alter nG−j\mathbf n_{G-j}nG−j​. The general spatial process of §9.3 has rates

q(n,Tjmn)=λj(nj,m) Φ(n)ΦG−j(nG−j)(9.15)q(\mathbf n,T_j^m\mathbf n)=\lambda_j(n_j,m)\,\frac{\Phi(\mathbf n)}{\Phi_{G-j}(\mathbf n_{G-j})} \qquad (9.15)q(n,Tjm​n)=λj​(nj​,m)ΦG−j​(nG−j​)Φ(n)​(9.15)

for positive functions Φ\PhiΦ, ΦG−j\Phi_{G-j}ΦG−j​.

Formalization targets

Goal: Theorem 9.2 (p. 186)

A random field π\piπ is a Markov field if and only if

π(n)=B∏C∈CϕC(nC),n∈S,(9.5)\pi(\mathbf n) = B\prod_{C\in\mathcal C}\phi_C(\mathbf n_C), \qquad \mathbf n\in\mathcal S, \qquad (9.5)π(n)=BC∈C∏​ϕC​(nC​),n∈S,(9.5)

for some constant BBB and functions ϕC\phi_CϕC​.

Milestones

  • Lemma 9.1 (p. 185): the conditional probabilities P(nj∣nG−j)P(n_j\mid\mathbf n_{G-j})P(nj​∣nG−j​), j∈Gj\in Gj∈G, n∈S\mathbf n\in\mathcal Sn∈S, determine the random field uniquely.
  • Theorem 9.3 (p. 189): the equilibrium distribution of a reversible spatial process is a Markov field.
  • Theorem 9.4 (p. 193): for the rates (9.15), with αj>0\alpha_j>0αj​>0 solving αj(n)∑mλj(n,m)=∑mαj(m)λj(m,n)\alpha_j(n)\sum_m\lambda_j(n,m)=\sum_m\alpha_j(m)\lambda_j(m,n)αj​(n)∑m​λj​(n,m)=∑m​αj​(m)λj​(m,n) (9.16), the equilibrium distribution is
π(n)=B ∏j=1Jαj(nj)Φ(n),(9.17)\pi(\mathbf n) = B\,\frac{\prod_{j=1}^J\alpha_j(n_j)}{\Phi(\mathbf n)}, \qquad (9.17)π(n)=BΦ(n)∏j=1J​αj​(nj​)​,(9.17)

and it satisfies the partial balance equations (9.18) site by site.

Significance

Theorem 9.2 turns a statement about conditional laws, which is how local interaction is usually specified, into an explicit parametrization of the joint law by clique potentials. On a lattice with binary attributes it reduces a Markov field to one parameter per site and one per pair of adjacent sites, giving the form π(n)=BαMβR\pi(\mathbf n)=B\alpha^M\beta^Rπ(n)=BαMβR (9.9). Theorem 9.3 shows that local, reversible dynamics produce Markov-field equilibria, and Theorem 9.4 gives a family of non-reversible processes, containing the closed migration process of Chapter 2, whose equilibria are still explicit; its partial balance equations are the bridge to §9.4.

All four results are classical and proved in the book. They are not, to our knowledge, machine-checked in this discrete form. The platform has an open statement of Hammersley–Clifford in a different setting, HighDimStat.GraphicalModels.thm11_8_hammersley_clifford (Wainwright, High-Dimensional Statistics, Theorem 11.8): a random vector in RV\mathbb R^VRV with a strictly positive Lebesgue density and the global (separation) Markov property. Neither statement implies the other as formalized, so this mission poses Kelly's finite, local version separately. A formal proof here also gives reusable infrastructure: conditional probabilities of a distribution on a finite product space, and factorizations over the cliques of a graph.

Difficulty

The "if" direction is routine. The "only if" direction is where the content lies: the functions ϕC\phi_CϕC​ must be produced from π\piπ alone, and a product over the cliques of GGG must reproduce π\piπ at every state, not only at the states whose nonzero attributes sit on a single clique. The natural first idea, one factor per site read off from the conditional laws, fails as soon as two sites interact. The Markov property must also be brought from its explicit form (9.2), which involves a marginal over the non-neighbours, into a usable statement about π\piπ itself. Strict positivity is essential: without it the "only if" direction is false (Exercise 9.2.2). For Theorem 9.3, condition (iii) cannot be dropped: Exercise 9.2.2 gives a reversible process satisfying (i) and (ii) whose equilibrium is not a Markov field, so any argument that uses only the local form of the rates fails.

Formalization scope

  • Sites form an arbitrary finite type V with decidable equality; attributes at site j form a finite type N j, which may differ between sites. States are dependent functions (j : V) → N j, and TjmnT_j^m\mathbf nTjm​n is Function.update n j m. The graph is a Mathlib SimpleGraph V, so ∂j\partial j∂j is G.neighborSet j.
  • A random field is a real function, positive at every state, with finite sum 111. P(nj∣n∂j)P(n_j\mid\mathbf n_{\partial j})P(nj​∣n∂j​) is the conditional probability computed from π\piπ (a ratio of finite sums), so (9.2) is stated literally.
  • Simplices are the nonempty cliques of the given graph GGG, including single sites. The factorization ranges over exactly these sets; a product over all subsets of sites, or over the cliques of the complete graph, would make the goal trivially true and is ruled out.
  • Theorems 9.3 and 9.4 are read at the level of rates: "equilibrium distribution of a reversible process" is a positive distribution summing to one in detailed balance with qqq (the published KellyStochasticNetworks.DetailedBalance), and "equilibrium distribution" in 9.4 is a positive distribution summing to one satisfying the equilibrium equations (KellyStochasticNetworks.FullBalance), together with its uniqueness under irreducibility. The Markov process itself is not constructed. The state space is always finite, so all sums are finite.
  • Contributions welcome: proofs of any item, general lemmas on conditional laws over finite product spaces, and a formal account of the general Hammersley–Clifford theorem that both this mission and the Wainwright statement could use.

Selected references

  • F. P. Kelly, Reversibility and Stochastic Networks, Wiley, 1979, Chapter 9. https://www.statslab.cam.ac.uk/~frank/BOOKS/kelly_book.html
  • J. Besag, Spatial interaction and the statistical analysis of lattice systems, J. Roy. Statist. Soc. B 36 (1974), 192–236. https://doi.org/10.1111/j.2517-6161.1974.tb00999.x
  • G. R. Grimmett, A theorem about random fields, Bull. London Math. Soc. 5 (1973), 81–84. https://doi.org/10.1112/blms/5.1.81
  • C. J. Preston, Generalized Gibbs states and Markov random fields, Adv. Appl. Probab. 5 (1973), 242–261. https://doi.org/10.2307/1426035
  • M. J. Wainwright, High-Dimensional Statistics, Cambridge University Press, 2019, Theorem 11.8. https://doi.org/10.1017/9781108627771
8 thms3 active usersReviewed
🏆Completed
Markov ChainOptimizationProbability+1·Captain: mikedeng1

Reversibility and Stochastic Networks V: Optimal Capacity Allocation in a Network of QueuesTextbook

Motivation

Chapter 4 of F. P. Kelly's Reversibility and Stochastic Networks (Wiley, 1979) applies the product-form theory of Chapter 3 to concrete systems. Two of its results have numbered statements, and they answer two practical questions.

The first comes from the design of store-and-forward communication networks (telegraph and packet-switched data networks). Messages queue at channels; the designer chooses each channel's capacity subject to a budget, and wants to minimize the delay messages suffer. Because the equilibrium law at every channel is geometric under several different modelling assumptions (§4.1, pp. 95–96), the mean delay has a closed form, and the budget allocation problem becomes a small convex program with an explicit solution. This square-root capacity assignment goes back to L. Kleinrock's work on communication nets (Communication Nets, McGraw-Hill, 1964) and remains the textbook example of optimal design for a network of queues.

The second comes from compartmental models in biology, birth–illness–death processes and manpower planning (§4.5, pp. 113–115). Individuals enter a system as a Poisson stream and move through it independently. Equilibrium results follow from Chapter 3; Theorem 4.2 describes the transient behaviour exactly, starting from an empty system.

Setting

Capacity allocation (§4.1). A network has J≥1J \ge 1J≥1 channels. Channel jjj receives traffic at average rate aj>0a_j > 0aj​>0 and is given capacity ϕj\phi_jϕj​. In equilibrium the number njn_jnj​ of messages at channel jjj has the geometric law (4.1),

P(nj=n)=(1−ajϕj)(ajϕj)n,n=0,1,2,…,P(n_j = n) = \Big(1 - \frac{a_j}{\phi_j}\Big)\Big(\frac{a_j}{\phi_j}\Big)^n, \qquad n = 0, 1, 2, \dots,P(nj​=n)=(1−ϕj​aj​​)(ϕj​aj​​)n,n=0,1,2,…,

which requires ϕj>aj\phi_j > a_jϕj​>aj​. Its mean is aj/(ϕj−aj)a_j/(\phi_j - a_j)aj​/(ϕj​−aj​). Capacity on channel jjj costs fj>0f_j > 0fj​>0 per unit and the total budget is FFF, giving the cost constraint (4.2)

∑jfjϕj=F.\sum_j f_j\phi_j = F.j∑​fj​ϕj​=F.

The mean number of customers in the network is

N(ϕ)=∑jajϕj−aj,N(\phi) = \sum_j \frac{a_j}{\phi_j - a_j},N(ϕ)=j∑​ϕj​−aj​aj​​,

and the feasible set is the set of ϕ∈RJ\phi \in \mathbb{R}^Jϕ∈RJ with ϕj>aj\phi_j > a_jϕj​>aj​ for every jjj that satisfy (4.2). In Lean these are meanNumberInNetwork a φ and FeasibleCapacities a f F. The proof works with the Lagrangian lagrangian a f F y φ =N(ϕ)+y(∑jfjϕj−F)= N(\phi) + y(\sum_j f_j\phi_j - F)=N(ϕ)+y(∑j​fj​ϕj​−F).

Compartmental model (§4.5). Individuals arrive in a Poisson stream of rate ν>0\nu > 0ν>0 at a system of JJJ compartments that is empty at time 000. Let pj(s)p_j(s)pj​(s) be the probability that an individual is in compartment jjj a time sss after its arrival; pj(s)≥0p_j(s) \ge 0pj​(s)≥0 and ∑jpj(s)≤1\sum_j p_j(s) \le 1∑j​pj​(s)≤1, since individuals may leave. Let nj(t)n_j(t)nj​(t) be the number of individuals in compartment jjj at time t>0t > 0t>0, and

αj(t)=∫0tpj(u) du.\alpha_j(t) = \int_0^t p_j(u)\,du.αj​(t)=∫0t​pj​(u)du.

In Lean the model is the predicate IsCompartmentModel P ν t p M T Loc, the counts are compartmentCount M Loc j, and αj(t)\alpha_j(t)αj​(t) is alpha p j t.

Formalization targets

Goal: Theorem 4.1 (p. 97)

If J≥1J \ge 1J≥1, aj>0a_j > 0aj​>0, fj>0f_j > 0fj​>0 and F>∑kakfkF > \sum_k a_k f_kF>∑k​ak​fk​, then

ϕj∗=aj+ajfj∑kakfk⋅F−∑kakfkfj\phi^*_j = a_j + \frac{\sqrt{a_j f_j}}{\sum_k \sqrt{a_k f_k}}\cdot\frac{F - \sum_k a_k f_k}{f_j}ϕj∗​=aj​+∑k​ak​fk​​aj​fj​​​⋅fj​F−∑k​ak​fk​​

is feasible and minimizes NNN over the feasible set, and every other feasible ϕ\phiϕ has N(ϕ)>N(ϕ∗)N(\phi) > N(\phi^*)N(ϕ)>N(ϕ∗).

Milestones toward the goal

  1. The mean of the geometric law (4.1) is aj/(ϕj−aj)a_j/(\phi_j - a_j)aj​/(ϕj​−aj​) (p. 97).
  2. For y>0y > 0y>0 the Lagrangian is minimized over {ϕj>aj}\{\phi_j > a_j\}{ϕj​>aj​} by ϕj=aj+aj/(yfj)\phi_j = a_j + \sqrt{a_j/(y f_j)}ϕj​=aj​+aj​/(yfj​)​ (proof of Theorem 4.1).
  3. The choice 1/y=(F−∑kakfk)/∑kakfk1/\sqrt y = (F - \sum_k a_k f_k)/\sum_k\sqrt{a_k f_k}1/y​=(F−∑k​ak​fk​)/∑k​ak​fk​​ makes that minimizer equal to ϕ∗\phi^*ϕ∗ and feasible for (4.2).

Second result: Theorem 4.2 (pp. 114–115)

The proof's generating-function identity, for zj∈[0,1]z_j \in [0,1]zj​∈[0,1],

E(z1n1(t)⋯zJnJ(t))=∏j=1Jexp⁡[−(1−zj)ναj(t)],E\big(z_1^{n_1(t)}\cdots z_J^{n_J(t)}\big) = \prod_{j=1}^J \exp\big[-(1 - z_j)\nu\alpha_j(t)\big],E(z1n1​(t)​⋯zJnJ​(t)​)=j=1∏J​exp[−(1−zj​)ναj​(t)],

and the theorem itself: n1(t),…,nJ(t)n_1(t), \dots, n_J(t)n1​(t),…,nJ​(t) are independent and nj(t)n_j(t)nj​(t) is Poisson with mean ναj(t)\nu\alpha_j(t)ναj​(t).

Significance

Theorem 4.1 is a closed-form design rule. Every channel first receives the capacity aja_jaj​ needed to carry its traffic; the remaining budget is shared in proportion to ajfj\sqrt{a_j f_j}aj​fj​​, not to the traffic aja_jaj​. Sizing capacity in proportion to the load, which is the obvious rule, is therefore not optimal. The same calculation applies to any network whose stations have the geometric law (4.1), for example a manufacturing job shop (p. 97). The mean number in the network and the mean time a customer spends in it are minimized together.

Theorem 4.2 is the exact transient law of a network of infinite-server queues started empty. It holds however complicated the motion of an individual is, provided individuals move independently, and letting t→∞t \to \inftyt→∞ it recovers the equilibrium Poisson law of §4.5.

Both results are classical and proved in the book. Neither is formalized on Prove2Me. The platform has the geometric equilibrium law of the M/M/1 queue (KellyStochasticNetworks.mm1_equilibrium) but not its mean, and no result on Poisson thinning or marking by independent random locations. The formal work for Theorem 4.1 is a strict-convexity and Lagrangian-sufficiency argument in RJ\mathbb{R}^JRJ. For Theorem 4.2 it is a Poisson marking theorem in measure-theoretic probability. Both pieces are reusable.

Difficulty

For Theorem 4.1, the book's proof sets the partial derivatives of the Lagrangian to zero. A stationary point is not a global minimizer in general, so the formal proof must show that LLL is (strictly) convex on the open region ϕj>aj\phi_j > a_jϕj​>aj​, and it must use Lagrangian sufficiency, not first-order conditions alone. The region is open and the objective is unbounded near its boundary. Feasibility of ϕ∗\phi^*ϕ∗ needs F>∑kakfkF > \sum_k a_k f_kF>∑k​ak​fk​, which the book leaves implicit.

For Theorem 4.2, the steps of the proof that read "conditional on MMM" have to be carried out with measure-theoretic independence. One step averages a product over MMM independent uniform instants. Another sums the Poisson mixture into an exponential. The last turns a factorized generating function into mutual independence of JJJ counts with Poisson marginals. Mathlib has the Poisson distribution (ProbabilityTheory.poissonMeasure) but no marking or thinning theorem, and no uniqueness theorem for multivariate probability generating functions.

Formalization scope

Channels and compartments are indexed by Fin J; all rates, costs and capacities are real numbers.

Theorem 4.1. The statement carries the book's implicit hypotheses explicitly: J≥1J \ge 1J≥1, aj>0a_j > 0aj​>0, fj>0f_j > 0fj​>0, F>∑kakfkF > \sum_k a_k f_kF>∑k​ak​fk​. Stability ϕj>aj\phi_j > a_jϕj​>aj​ is part of the feasible set. The conclusion is global optimality over the feasible set (IsMinOn) together with feasibility of ϕ∗\phi^*ϕ∗, plus strict optimality against every other feasible point. Uniqueness is a slight strengthening of the book's "the optimal allocation is", and it holds by strict convexity. A statement that ϕ∗\phi^*ϕ∗ satisfies (4.2), or that it is a stationary point of the Lagrangian, is not the theorem: those are one-line computations or the proof method, and the goal is stated as global optimality to rule them out.

Theorem 4.2. The model is pinned down as in the proof on p. 115:

  • the number MMM of arrivals in (0,t)(0,t)(0,t) is Poisson with mean νt\nu tνt;
  • an i.i.d. sequence of (arrival instant, location at time ttt) pairs is independent of MMM, and only its first MMM entries are used;
  • each instant is uniform on (0,t)(0,t)(0,t), and an individual arriving at uuu is in compartment jjj at time ttt with probability pj(t−u)p_j(t-u)pj​(t−u), or has left.

"Individuals move independently" is formalized as this conditional independence. The pjp_jpj​ are measurable sub-probabilities, not assumed to sum to one. The conclusion is mutual independence of the JJJ counts (iIndepFun) together with the Poisson probability mass function of each. Infinite time horizons and the point-process description of the system are out of scope.

Useful contributions: a reusable Lagrangian-sufficiency lemma for separable convex objectives under one linear constraint; a Poisson marking (colouring) theorem for finitely many colours; the multivariate generating-function uniqueness lemma for NJ\mathbb{N}^JNJ-valued random vectors.

Selected references

  • F. P. Kelly, Reversibility and Stochastic Networks, John Wiley & Sons, 1979, Chapter 4 (§4.1, pp. 95–97; §4.5, pp. 113–115).
  • L. Kleinrock, Communication Nets: Stochastic Message Flow and Delay, McGraw-Hill, 1964 (reprinted Dover, 1972).
  • J. F. C. Kingman, Poisson Processes, Oxford University Press, 1993 (colouring and marking theorems).
8 thms3 active usersReviewed
🏆Completed
Markov ChainProbabilityStochastic Systems·Captain: mikedeng1

Reversibility and Stochastic Networks III: Open Networks of Queues with General Customer Routes Have Product-Form EquilibriumTextbook

Motivation

Networks of queues model systems in which jobs visit a sequence of service stations: items in a manufacturing job-shop, packets in a communication network, patients moving between hospital departments. The open migration process of Chapter 2 of F. P. Kelly, Reversibility and Stochastic Networks (Wiley, 1979), and the job-shop networks of Jackson (Jackson 1963) route a customer leaving a queue at random, independently of where he has been. That rules out the most common situation in practice: an item that has passed machines 1 and 3 must next go to machine 4, while an item that has passed machines 2 and 3 must go to machine 5.

Section 3.1 of the book removes this restriction. Customers are divided into types, a type fixes a deterministic route through the queues, and a stochastic routing rule is recovered by using one type per possible route. Within each queue, the order of service is described by two position-dependent functions, which cover first-come first-served KKK-server queues, last-come first-served, processor sharing and service in random order. Theorem 3.1 states that, for every such network, the equilibrium distribution is a product of explicit single-queue factors. This is the result behind the "Kelly network" and "Kelly-type queue" terminology of later work (Kelly 1975; Baskett, Chandy, Muntz, Palacios 1975).

Setting

There are III customer types and JJJ queues. Customers of type iii enter the system in a Poisson stream of rate ν(i)>0\nu(i)>0ν(i)>0 and visit the queues r(i,1),r(i,2),…,r(i,S(i))r(i,1),r(i,2),\dots,r(i,S(i))r(i,1),r(i,2),…,r(i,S(i)) in that order before leaving; two successive stages of a route are at different queues.

Queue jjj holds its njn_jnj​ customers in positions 1,…,nj1,\dots,n_j1,…,nj​. Each customer needs an exponentially distributed amount of service with unit mean. The queue supplies total service effort at rate ϕj(nj)\phi_j(n_j)ϕj​(nj​), with ϕj(n)>0\phi_j(n)>0ϕj​(n)>0 for n>0n>0n>0; a proportion γj(l,nj)\gamma_j(l,n_j)γj​(l,nj​) goes to the customer in position lll. An arriving customer takes position lll with probability δj(l,nj+1)\delta_j(l,n_j+1)δj​(l,nj​+1). For each n≥1n\ge1n≥1, γj(⋅,n)\gamma_j(\cdot,n)γj​(⋅,n) and δj(⋅,n)\delta_j(\cdot,n)δj​(⋅,n) are probability vectors on {1,…,n}\{1,\dots,n\}{1,…,n}.

The class of the customer in position lll of queue jjj is cj(l)=(tj(l),sj(l))c_j(l)=(t_j(l),s_j(l))cj​(l)=(tj​(l),sj​(l)), his type and the stage of his route. The state of queue jjj is cj=(cj(1),…,cj(nj))\mathbf c_j=(c_j(1),\dots,c_j(n_j))cj​=(cj​(1),…,cj​(nj​)) and the state of the network is C=(c1,…,cJ)\mathbf C=(\mathbf c_1,\dots,\mathbf c_J)C=(c1​,…,cJ​). Its transition rates q(C,D)q(\mathbf C,\mathbf D)q(C,D), displays (3.1)–(3.6), are the sums of the intensities of all events taking C\mathbf CC to D\mathbf DD: a departure from the system (intensity ϕj(nj)γj(l,nj)\phi_j(n_j)\gamma_j(l,n_j)ϕj​(nj​)γj​(l,nj​)), a move from position lll of queue jjj to position mmm of the next queue kkk (intensity ϕj(nj)γj(l,nj)δk(m,nk+1)\phi_j(n_j)\gamma_j(l,n_j)\delta_k(m,n_k+1)ϕj​(nj​)γj​(l,nj​)δk​(m,nk​+1)), and an arrival into position mmm of the first queue kkk of a route (intensity ν(i)δk(m,nk+1)\nu(i)\delta_k(m,n_k+1)ν(i)δk​(m,nk​+1)).

With αj(i,s)=ν(i)\alpha_j(i,s)=\nu(i)αj​(i,s)=ν(i) if r(i,s)=jr(i,s)=jr(i,s)=j and 000 otherwise, set

aj=∑i,sαj(i,s),bj−1=∑n=0∞ajn∏l=1nϕj(l),πj(cj)=bj∏l=1njαj(tj(l),sj(l))ϕj(l).a_j=\sum_{i,s}\alpha_j(i,s),\qquad b_j^{-1}=\sum_{n=0}^{\infty}\frac{a_j^n}{\prod_{l=1}^{n}\phi_j(l)},\qquad \pi_j(\mathbf c_j)=b_j\prod_{l=1}^{n_j}\frac{\alpha_j(t_j(l),s_j(l))}{\phi_j(l)}.aj​=i,s∑​αj​(i,s),bj−1​=n=0∑∞​∏l=1n​ϕj​(l)ajn​​,πj​(cj​)=bj​l=1∏nj​​ϕj​(l)αj​(tj​(l),sj​(l))​.

Formalization targets

Goal: Theorem 3.1 (p. 61)

If every series defining bj−1b_j^{-1}bj−1​ converges, then

π(C)=∏j=1Jπj(cj)\pi(\mathbf C)=\prod_{j=1}^{J}\pi_j(\mathbf c_j)π(C)=j=1∏J​πj​(cj​)

is positive, sums to 111 over all network states, and satisfies the equilibrium equations

π(C)∑Dq(C,D)=∑Dπ(D) q(D,C)for every C.\pi(\mathbf C)\sum_{\mathbf D}q(\mathbf C,\mathbf D)=\sum_{\mathbf D}\pi(\mathbf D)\,q(\mathbf D,\mathbf C)\quad\text{for every }\mathbf C.π(C)D∑​q(C,D)=D∑​π(D)q(D,C)for every C.

Milestones

  • Theorem 3.2 (p. 62). The time-reversed rates π(D)q(D,C)/π(C)\pi(\mathbf D)q(\mathbf D,\mathbf C)/\pi(\mathbf C)π(D)q(D,C)/π(C) are the rates of the reversed network: routes traversed backwards, γj\gamma_jγj​ and δj\delta_jδj​ interchanged.
  • Corollary 3.4 (p. 63). Queue jjj is independent of the rest of the network, is in state cj\mathbf c_jcj​ with probability πj(cj)\pi_j(\mathbf c_j)πj​(cj​), holds nnn customers with probability bjajn/∏l=1nϕj(l)b_ja_j^n/\prod_{l=1}^n\phi_j(l)bj​ajn​/∏l=1n​ϕj​(l) (3.7), and a customer in position lll is of class (i,s)(i,s)(i,s) with probability αj(i,s)/aj\alpha_j(i,s)/a_jαj​(i,s)/aj​.
  • Corollary 3.5 (p. 63). A type-iii customer reaching queue jjj at stage sss finds it in state cj\mathbf c_jcj​ with probability πj(cj)\pi_j(\mathbf c_j)πj​(cj​).
  • Lemma 3.13 (p. 89). For a multiclass queue with Poisson arrivals of rate ν(c)\nu(c)ν(c) and departure intensities ν(c)ϕc(n)\nu(c)\phi_c(\mathbf n)ν(c)ϕc​(n): reversible ⇔\Leftrightarrow⇔ quasi-reversible ⇔\Leftrightarrow⇔ Φ(n)=ϕc(n)Φ(n−ec)\Phi(\mathbf n)=\phi_c(\mathbf n)\Phi(\mathbf n-\mathbf e_c)Φ(n)=ϕc​(n)Φ(n−ec​) for some positive Φ\PhiΦ (3.26).

Significance

Theorem 3.1 gives the full joint law of a network in which routes carry memory, and its corollaries turn it into usable performance formulas: each queue behaves, in its marginal law and as seen by arriving customers, like an isolated queue fed by a Poisson stream of rate aja_jaj​, even though the actual arrival stream at queue jjj is not Poisson. Mean sojourn times along a route then follow from Little's result. Theorem 3.2 identifies the reversed process as a network of the same kind; it is the source of the departure-stream results (Corollary 3.3) and of the arrival theorem (Corollary 3.5). Lemma 3.13 isolates the condition (3.26) under which state-dependent arrival rates preserve the product form (Theorem 3.14).

The results are classical and proved in the book. None of them has a machine-checked proof: the Prove2Me catalogue holds the rate-level theorems for migration processes (Chapter 2 of Kelly–Yudovina), and open targets for the BCMP and Jackson models, which have different state descriptions. This mission adds a formal model of the position-structured multiclass network itself, with the summation over coinciding transitions that (3.2), (3.4) and (3.6) require, and product-form, reversal and arrival-theorem statements over it.

Difficulty

The obvious first attempt, detailed balance, fails: π(C)q(C,D)\pi(\mathbf C)q(\mathbf C,\mathbf D)π(C)q(C,D) and π(D)q(D,C)\pi(\mathbf D)q(\mathbf D,\mathbf C)π(D)q(D,C) differ in general, because a customer's route cannot be run backwards inside the same network (q(D,C)q(\mathbf D,\mathbf C)q(D,C) is usually 000 when q(C,D)>0q(\mathbf C,\mathbf D)>0q(C,D)>0). The equilibrium equations therefore involve, for each state, all its predecessors at once. The rates are themselves sums over coinciding transitions, so a statement about individual events does not transfer to the rates without accounting for which positions lead to the same successor state. In Lean this brings in insertion into and deletion from position lists, the relabelling of stages, and the normalization of a product over a countable space of JJJ-tuples of lists, reorganized by queue length together with the identity ∑classes at jαj=aj\sum_{\text{classes at } j}\alpha_j=a_j∑classes at j​αj​=aj​.

Formalization scope

  • Finite types and queues. Types are Fin I, queues Fin J; the book allows countably many types with ∑iν(i)<∞\sum_i\nu(i)<\infty∑i​ν(i)<∞. A network state is a function assigning to each queue a list of classes (i,s)(i,s)(i,s) with r(i,s)=jr(i,s)=jr(i,s)=j; the state space is countable and all sums over it are tsum/HasSum.
  • Indexing. Stages and list positions are 000-based in Lean; γj(l,n)\gamma_j(l,n)γj​(l,n) and δj(l,n)\delta_j(l,n)δj​(l,n) keep the book's 111-based position argument.
  • Rate level. Equilibrium means: positive, summing to 111, and satisfying the equilibrium equations (the published KellyStochasticNetworks.FullBalance). The existence of the Markov process, irreducibility and non-explosion are not formalized. "The reversed process" (Theorem 3.2) is read through the reversed rates π(D)q(D,C)/π(C)\pi(\mathbf D)q(\mathbf D,\mathbf C)/\pi(\mathbf C)π(D)q(D,C)/π(C); "the probability he finds" (Corollary 3.5) is read as a ratio of equilibrium arrival fluxes; quasi-reversibility is its rate characterization (3.8), (3.10).
  • Normalizing constants. bjb_jbj​ is defined through a tsum, which Lean sets to 000 for a divergent series; every theorem assumes the series converges, the book's "none of b1,…,bJb_1,\dots,b_Jb1​,…,bJ​ is zero".
  • No trivial instance. The goal holds for arbitrary III, JJJ, ν\nuν, routes, ϕj\phi_jϕj​, γj\gamma_jγj​, δj\delta_jδj​ subject only to the book's constraints; a proof for a single queue, or for fixed γ=δ\gamma=\deltaγ=δ disciplines, does not prove it. In Lemma 3.13 the function Φ\PhiΦ is required to be positive, since Φ≡0\Phi\equiv0Φ≡0 satisfies (3.26) for every queue.

Infrastructure that a complete development needs: list insertion/deletion lemmas for position bookkeeping, sums of products over ∏jList(⋅)\prod_j \mathrm{List}(\cdot)∏j​List(⋅), and a bijection-of-events argument for summed rates. The quasi-reversibility predicate and the reversed-rate apparatus are reusable for the closed networks of §3.4 and the symmetric queues of §3.3. Contributions are welcome on any milestone, in any order.

Selected references

  • F. P. Kelly, Reversibility and Stochastic Networks, John Wiley & Sons, 1979. https://www.statslab.cam.ac.uk/~frank/BOOKS/kelly_book.html
  • F. P. Kelly, Networks of queues with customers of different types, Journal of Applied Probability 12 (1975), 542–554. https://doi.org/10.2307/3212785
  • F. Baskett, K. M. Chandy, R. R. Muntz, F. G. Palacios, Open, closed, and mixed networks of queues with different classes of customers, Journal of the ACM 22 (1975), 248–260. https://doi.org/10.1145/321879.321887
  • J. R. Jackson, Jobshop-like queueing systems, Management Science 10 (1963), 131–142. https://doi.org/10.1287/mnsc.10.1.131
  • F. P. Kelly, E. Yudovina, Stochastic Networks, Cambridge University Press, 2014. https://doi.org/10.1017/CBO9781139565363
10 thms3 active usersReviewed
🏆Completed
Markov ChainProbabilityStochastic Systems·Captain: mikedeng1

Reversibility and Stochastic Networks IV: Symmetric Queues with Gamma-Mixture Service RequirementsTextbook

Why symmetric queues

The classical product-form results for queueing networks (Jackson, Kelly, Baskett–Chandy–Muntz–Palacios) assume exponentially distributed service requirements, because then the state of a queue need not record how much service each customer has received. Real service times are rarely exponential: telephone call lengths, job sizes in time-shared computers and web transfers are far from it. Symmetric queues, introduced in §3.3 of F. P. Kelly, Reversibility and Stochastic Networks (Wiley, 1979), form the class of single queues for which the stationary distribution of the number of customers, and of their classes, depends on the service requirement distribution only through its mean. This property, insensitivity, is what makes Erlang's loss formula valid for arbitrarily distributed call lengths (Kelly, p. 79), and it is the reason processor-sharing, last-come-first-served preemptive and infinite-server stations may appear with general service in product-form networks.

Timeline. Sevastyanov (1957) proved that Erlang's loss formula holds for arbitrarily distributed call lengths. Kelly (1975, 1976) introduced queues with customers of different types whose effort and arrival-position functions coincide, and showed product form for networks of them with non-exponential service built from exponential stages; Barbour (1976) extended the method of stages; Baskett, Chandy, Muntz and Palacios (1975) gave product form for networks containing processor-sharing, LCFS-preemptive and infinite-server stations with phase-type service. Kelly's 1979 book presents the symmetric queue in the form formalized here.

Setting

A symmetric queue holds customers in positions 1,2,…,n1, 2, \dots, n1,2,…,n, where nnn is the number present. It operates as follows (Kelly, p. 72):

  1. the service requirement of a customer is a random variable whose distribution may depend on the class of the customer;
  2. a total service effort is supplied at rate ϕ(n)\phi(n)ϕ(n), with ϕ(n)>0\phi(n) > 0ϕ(n)>0 for n>0n > 0n>0;
  3. a proportion γ(l,n)\gamma(l, n)γ(l,n) of this effort, ∑l=1nγ(l,n)=1\sum_{l=1}^n \gamma(l, n) = 1∑l=1n​γ(l,n)=1, goes to the customer in position lll; when he leaves, the customers in positions l+1,…,nl+1, \dots, nl+1,…,n move down by one;
  4. an arriving customer moves into position l∈{1,…,n+1}l \in \{1, \dots, n+1\}l∈{1,…,n+1} with probability γ(l,n+1)\gamma(l, n+1)γ(l,n+1) — the same function — and the customers in positions l,…,nl, \dots, nl,…,n move up by one.

Server-sharing (γ(l,n)=1/n\gamma(l, n) = 1/nγ(l,n)=1/n), the stack (γ(n,n)=1\gamma(n, n) = 1γ(n,n)=1, last come first served preemptive), the queue with no waiting room and the infinite-server queue are examples (pp. 73–74).

Customers of class ccc arrive in a Poisson stream of rate ν(c)\nu(c)ν(c). On arrival a class-ccc customer receives a refined class (c,z)(c, z)(c,z) with probability p(c,z)p(c, z)p(c,z), ∑zp(c,z)=1\sum_z p(c, z) = 1∑z​p(c,z)=1, and then needs w(c,z)≥1w(c, z) \ge 1w(c,z)≥1 independent stages of service, each exponentially distributed with mean d(c,z)>0d(c, z) > 0d(c,z)>0. The class-ccc service requirement is therefore a mixture of gamma distributions with mean

a(c)=∑zp(c,z) w(c,z) d(c,z),a(c) = \sum_z p(c, z)\, w(c, z)\, d(c, z),a(c)=z∑​p(c,z)w(c,z)d(c,z),

and the work arriving per unit time is a=∑cν(c)a(c)a = \sum_c \nu(c) a(c)a=∑c​ν(c)a(c). The record of the customer in position lll is c(l)=(c(l),z(l),u(l))\mathbf c(l) = (c(l), z(l), u(l))c(l)=(c(l),z(l),u(l)), u(l)u(l)u(l) the stage in progress, and c=(c(1),…,c(n))\mathbf c = (\mathbf c(1), \dots, \mathbf c(n))c=(c(1),…,c(n)) is a Markov process. Its transitions are: an arrival of class (c,z)(c,z)(c,z) into position lll at stage 111, at rate ν(c)p(c,z)γ(l,n+1)\nu(c)p(c,z)\gamma(l, n+1)ν(c)p(c,z)γ(l,n+1); and, at rate ϕ(n)γ(l,n)/d(c(l),z(l))\phi(n)\gamma(l,n)/d(c(l),z(l))ϕ(n)γ(l,n)/d(c(l),z(l)), completion of the current stage of the customer in position lll, which moves him to the next stage or, after stage w(c(l),z(l))w(c(l), z(l))w(c(l),z(l)), out of the queue. The normalizing constant is

b−1=∑n=0∞an∏l=1nϕ(l).(3.15)b^{-1} = \sum_{n=0}^{\infty} \frac{a^n}{\prod_{l=1}^n \phi(l)}. \tag{3.15}b−1=n=0∑∞​∏l=1n​ϕ(l)an​.(3.15)

Formalization targets

Goal: Theorem 3.8

When (3.15) converges, the distribution

π(c)=b∏l=1nν(c(l)) p(c(l),z(l)) d(c(l),z(l))ϕ(l)(3.18)\pi(\mathbf c) = b \prod_{l=1}^n \frac{\nu(c(l))\, p(c(l), z(l))\, d(c(l), z(l))}{\phi(l)} \tag{3.18}π(c)=bl=1∏n​ϕ(l)ν(c(l))p(c(l),z(l))d(c(l),z(l))​(3.18)

is the equilibrium distribution of c\mathbf cc, and under it

P(n customers)=b an∏l=1nϕ(l),P(classes c1,…,cn∣n)=∏l=1nν(cl) a(cl)a,\mathbb P(n \text{ customers}) = \frac{b\,a^n}{\prod_{l=1}^n \phi(l)}, \qquad \mathbb P(\text{classes } c_1, \dots, c_n \mid n) = \prod_{l=1}^n \frac{\nu(c_l)\, a(c_l)}{a},P(n customers)=∏l=1n​ϕ(l)ban​,P(classes c1​,…,cn​∣n)=l=1∏n​aν(cl​)a(cl​)​,

and the queue is quasi-reversible with respect to the classification ccc and to (c,z)(c, z)(c,z): from every state, the rate of arrivals of each class does not depend on the state, both for the process and its time reversal (relations (3.8) and (3.10) of p. 67).

Milestones

  • Eqs. (3.14)–(3.15): the case of one refinement per class, π(c)=b∏lν(c(l))d(c(l))/ϕ(l)\pi(\mathbf c) = b\prod_l \nu(c(l))d(c(l))/\phi(l)π(c)=b∏l​ν(c(l))d(c(l))/ϕ(l) with a=∑cν(c)d(c)w(c)a = \sum_c \nu(c)d(c)w(c)a=∑c​ν(c)d(c)w(c).
  • Eqs. (3.16)–(3.17): in that case, the law of nnn, and given nnn independent positions of class ccc with probability ν(c)d(c)w(c)/a\nu(c)d(c)w(c)/aν(c)d(c)w(c)/a and uniform stage.
  • Eq. (3.18): the equilibrium distribution under gamma-mixture service.
  • Lemma 3.9: mixtures of gamma distributions approximate, at continuity points, the distribution function of any positive random variable.

Significance

Theorem 3.8 gives the stationary law of a symmetric queue in closed form and shows that it depends on the service requirement distributions only through their means a(c)a(c)a(c). Quasi-reversibility (part (iii)) is the property that lets symmetric queues be placed in networks: Kelly's §3.2 shows that a network of quasi-reversible queues has a product-form equilibrium, so Theorem 3.8 is the single-queue input to product-form networks with processor-sharing, LCFS-preemptive and infinite-server stations and class-dependent, non-exponential service. Lemma 3.9 is the approximation step behind the extension to arbitrary service distributions (Theorem 3.10).

These results are classical and proved in the book. To our knowledge none of them is machine-checked; Mathlib has the gamma distribution and distribution functions but no queueing theory. The mission produces a formal model of the symmetric queue as a countable-state Markov process, its equilibrium distribution, the insensitive marginals and the rate characterization of quasi-reversibility, all reusable by the chapter on networks of quasi-reversible queues.

Difficulty

The obvious first attempt, detailed balance, fails: the stage process is not reversible in general, since an intermediate stage completion has no transition back. The equilibrium equations must be verified in full, over a countable state space in which a single state is reached from infinitely many others. The symmetry condition γ≡δ\gamma \equiv \deltaγ≡δ is essential and must enter the argument: without it (for first-come-first-served, say) the distribution (3.18) is false for non-exponential service. Positions shift on every arrival and departure, so the bookkeeping of which list results from which event, including coincidences when neighbouring customers have identical records, is the main formal burden. The marginal computations sum (3.18) over lists of records, where convergence must be tracked, and Lemma 3.9 needs an explicit construction of approximating gamma mixtures.

Formalization scope

  • The state is a List of customer records (class, refined class, stage); list index iii is position l=i+1l = i + 1l=i+1, and γ(l,n)\gamma(l, n)γ(l,n), ϕ(n)\phi(n)ϕ(n) keep the book's 111-based indexing. The state space consists of the lists whose records have ν(c)p(c,z)>0\nu(c)p(c,z) > 0ν(c)p(c,z)>0 and 1≤u≤w(c,z)1 \le u \le w(c, z)1≤u≤w(c,z): records of a refined class arriving at rate zero are unreachable and excluded.
  • Classes C\mathcal CC and refinements Z\mathcal ZZ are arbitrary countable types. All infinite sums are HasSum or tsum with explicit convergence hypotheses: ∑cν(c)<∞\sum_c \nu(c) < \infty∑c​ν(c)<∞ (finite exit rates, the book's standing assumption of §1.1), convergence of a(c)a(c)a(c), of aaa, and of (3.15). "Equilibrium distribution" means positive, summing to one, and satisfying the equilibrium equations with convergent series.
  • The statement is rate level: (3.18) is shown to satisfy the equilibrium equations of the stage process, and quasi-reversibility is its rate characterization (3.8), (3.10). That these describe the stationary process and its time reversal is the book's Chapter 1 and is not reformalized.
  • Trivializing readings ruled out: the goal keeps ppp, www and ddd general (not w≡1w \equiv 1w≡1, which would make the result Section 3.1's exponential case); the arrival position uses the same γ\gammaγ as the service split; and in Lemma 3.9 the approximants must be genuine gamma mixtures with integer shapes, which a point mass is not.
  • Not planned: Theorem 3.10 (arbitrary service distributions; only an outline of proof and a continuous state space the book does not construct) and Theorem 3.11 (reversibility of the number in queue, a non-Markov process).

Welcome contributions: proofs of the milestones, lemmas about List.insertIdx/List.eraseIdx bookkeeping for queues with positions, and summation over lists of records.

Selected references

  • F. P. Kelly, Reversibility and Stochastic Networks, Wiley, Chichester, 1979, §3.3, pp. 72–82. https://www.statslab.cam.ac.uk/~frank/BOOKS/kelly_book.html
  • F. P. Kelly, Networks of queues with customers of different types, Journal of Applied Probability 12 (1975), 542–554. https://www.jstor.org/journal/japplprob
  • F. P. Kelly, Networks of queues, Advances in Applied Probability 8 (1976), 416–432. https://www.jstor.org/journal/advaapplprob
  • A. D. Barbour, Networks of queues and the method of stages, Advances in Applied Probability 8 (1976), 584–591. https://www.jstor.org/journal/advaapplprob
  • B. A. Sevastyanov, An ergodic theorem for Markov processes and its application to telephone systems with refusals, Theory of Probability and its Applications 2 (1957), 104–112.
  • F. Baskett, K. M. Chandy, R. R. Muntz, F. G. Palacios, Open, closed, and mixed networks of queues with different classes of customers, Journal of the ACM 22 (1975), 248–260. https://doi.org/10.1145/321879.321887
9 thms3 active usersReviewed
🏆Completed
Algorithmic Game TheoryOptimization·Captain: mikedeng1

Supply Chain Coordination with Revenue-Sharing Contracts: Strengths and Limitations 2: With Competing Retailers, Revenue Sharing Supports the System-Optimal Quantities as a Nash EquilibriumResearch Paper

Motivation

A supplier that sells through independent retailers usually loses part of the profit an integrated firm would earn: each retailer orders to maximize its own profit, not the channel's. Supply chain coordination asks which contracts make the decentralized choices coincide with the integrated optimum. Cachon and Lariviere study revenue-sharing contracts, under which a retailer pays a per-unit wholesale price and keeps only a fraction ϕ\phiϕ of its revenue, the rest going to the supplier. The contracts were made prominent by the video rental industry around 1998, where studios lowered tape prices in exchange for a share of rental income.

With a single retailer, revenue sharing at the wholesale price ϕc\phi cϕc coordinates the channel and splits its profit in the proportion ϕ\phiϕ. This mission formalizes the extension in Section 3.2 of the paper to competing retailers: several locations whose revenues depend on each other's stock, so that one retailer's order lowers the others' revenue. Competition creates externalities the single-retailer argument does not have, and the question is whether revenue sharing still coordinates, and at what prices. Section 4.1.2 then works out a Cournot example in closed form, measuring how far the supplier's own optimal wholesale price leaves the channel from the integrated profit.

The source is the authors' working paper of June 2000; the published version (Management Science 51(1), 2005) renumbers and revises the results. The working paper numbers no theorem, so results are cited by section, displayed equation and page.

Setting

A single supplier sells one product through nnn locations i=1,…,ni = 1,\dots,ni=1,…,n, each run by an independent retailer. A stocking profile is qˉ=(q1,…,qn)\bar q = (q_1,\dots,q_n)qˉ​=(q1​,…,qn​), and the revenue at location iii is Ri(qˉ)R_i(\bar q)Ri​(qˉ​), which may depend on every location's quantity. The system revenue is R(qˉ)=∑iRi(qˉ)R(\bar q) = \sum_i R_i(\bar q)R(qˉ​)=∑i​Ri​(qˉ​), every unit costs the supplier c>0c > 0c>0, and the integrated system profit is

Π(qˉ)=R(qˉ)−c∑i=1nqi.\Pi(\bar q) = R(\bar q) - c\sum_{i=1}^n q_i .Π(qˉ​)=R(qˉ​)−ci=1∑n​qi​.

Write Rji(qˉ)=∂Rj(qˉ)/∂qiR_j^i(\bar q) = \partial R_j(\bar q)/\partial q_iRji​(qˉ​)=∂Rj​(qˉ​)/∂qi​: the superscript is the variable differentiated, the subscript the revenue function. The paper assumes that each RiR_iRi​ is continuous, that ∂2Ri/∂qi∂qj≤0\partial^2 R_i/\partial q_i\partial q_j \le 0∂2Ri​/∂qi​∂qj​≤0 for j≠ij \ne ij=i (locations are substitutes), and that RiR_iRi​ is unimodal in qiq_iqi​. The system-optimal profile qˉI\bar q^Iqˉ​I has positive entries and solves the first-order system

Rii(qˉI)+∑j≠iRji(qˉI)=c,i=1,…,n.(6)R_i^i(\bar q^I) + \sum_{j\ne i} R_j^i(\bar q^I) = c, \qquad i = 1,\dots,n. \tag{6}Rii​(qˉ​I)+j=i∑​Rji​(qˉ​I)=c,i=1,…,n.(6)

Under a revenue-sharing contract (ϕ,wi)(\phi, w_i)(ϕ,wi​) retailer iii earns πri(qˉ,ϕ,wˉ)=ϕRi(qˉ)−wiqi\pi_{r_i}(\bar q,\phi,\bar w) = \phi R_i(\bar q) - w_i q_iπri​​(qˉ​,ϕ,wˉ)=ϕRi​(qˉ​)−wi​qi​ and the supplier earns πs(qˉ,ϕ,wˉ)=∑i((1−ϕ)Ri(qˉ)+wiqi)−c∑iqi\pi_s(\bar q,\phi,\bar w) = \sum_i\big((1-\phi)R_i(\bar q) + w_i q_i\big) - c\sum_i q_iπs​(qˉ​,ϕ,wˉ)=∑i​((1−ϕ)Ri​(qˉ​)+wi​qi​)−c∑i​qi​; the wholesale-price contract is ϕ=1\phi = 1ϕ=1, with profits written πri(qˉ,wˉ)\pi_{r_i}(\bar q,\bar w)πri​​(qˉ​,wˉ) and πs(qˉ,wˉ)\pi_s(\bar q,\bar w)πs​(qˉ​,wˉ). A Nash equilibrium in order quantities is a profile qˉ≥0\bar q \ge 0qˉ​≥0 from which no retailer gains by changing its own quantity to any x≥0x \ge 0x≥0. The coordinating wholesale prices are

wiI=c−∑j≠iRji(qˉI).w_i^I = c - \sum_{j\ne i} R_j^i(\bar q^I).wiI​=c−j=i∑​Rji​(qˉ​I).

The Cournot example (7) is Ri(qˉ)=qi(1−qi−β∑j≠iqj)R_i(\bar q) = q_i\big(1 - q_i - \beta\sum_{j\ne i} q_j\big)Ri​(qˉ​)=qi​(1−qi​−β∑j=i​qj​) with 0≤β<10 \le \beta < 10≤β<1.

Formalization targets

Goal: revenue sharing supports qˉI\bar q^Iqˉ​I (Sec. 3.2, p. 14)

For ϕ∈[0,1]\phi\in[0,1]ϕ∈[0,1] and wi(ϕ)=ϕwiIw_i(\phi) = \phi w_i^Iwi​(ϕ)=ϕwiI​:

qˉI is a Nash equilibrium,πri(qˉI,ϕ,ϕwˉI)=ϕ πri(qˉI,wˉI),πs(qˉI,ϕ,ϕwˉI)=(1−ϕ)Π(qˉI)+ϕ πs(qˉI,wˉI).\bar q^I \text{ is a Nash equilibrium},\quad \pi_{r_i}(\bar q^I,\phi,\phi\bar w^I) = \phi\,\pi_{r_i}(\bar q^I,\bar w^I),\quad \pi_s(\bar q^I,\phi,\phi\bar w^I) = (1-\phi)\Pi(\bar q^I) + \phi\,\pi_s(\bar q^I,\bar w^I).qˉ​I is a Nash equilibrium,πri​​(qˉ​I,ϕ,ϕwˉI)=ϕπri​​(qˉ​I,wˉI),πs​(qˉ​I,ϕ,ϕwˉI)=(1−ϕ)Π(qˉ​I)+ϕπs​(qˉ​I,wˉI).

Wholesale-price contracts (Sec. 3.2, pp. 13–14)

An interior equilibrium satisfies Rii(qˉN)=wiR_i^i(\bar q^N) = w_iRii​(qˉ​N)=wi​ (Eq. (8)), so marginal-cost pricing does not support qˉI\bar q^Iqˉ​I when a location imposes a negative externality; the prices wˉI\bar w^IwˉI make qˉI\bar q^Iqˉ​I an equilibrium; wiI≥cw_i^I \ge cwiI​≥c when cross-effects are nonpositive; and wˉI\bar w^IwˉI supports exactly the split πs(qˉI,wˉI)=∑iqiI∑j≠i(−Rji(qˉI))\pi_s(\bar q^I,\bar w^I) = \sum_i q_i^I\sum_{j\ne i}(-R_j^i(\bar q^I))πs​(qˉ​I,wˉI)=∑i​qiI​∑j=i​(−Rji​(qˉ​I)).

Revenue sharing (Sec. 3.2, p. 14)

An interior equilibrium satisfies ϕRii(qˉN)=wi(ϕ)\phi R_i^i(\bar q^N) = w_i(\phi)ϕRii​(qˉ​N)=wi​(ϕ); the two profit identities hold for every ϕ\phiϕ; and πri(qˉI,wˉI)≥0\pi_{r_i}(\bar q^I,\bar w^I) \ge 0πri​​(qˉ​I,wˉI)≥0.

The Cournot example (Sec. 4.1.2, pp. 19–20)

At a common price w<1w<1w<1 the unique equilibrium is qiN=(1−w)/(2+β(n−1))q_i^N = (1-w)/(2+\beta(n-1))qiN​=(1−w)/(2+β(n−1)); the integrated optimum is qiI=(1−c)/(2+2β(n−1))q_i^I = (1-c)/(2+2\beta(n-1))qiI​=(1−c)/(2+2β(n−1)); the coordinating price wI=c+β(n−1)(1−c)/(2+2β(n−1))w^I = c + \beta(n-1)(1-c)/(2+2\beta(n-1))wI=c+β(n−1)(1−c)/(2+2β(n−1)) increases in β\betaβ and nnn; the supplier's optimal price is w∗=(1+c)/2w^* = (1+c)/2w∗=(1+c)/2; and the efficiency at w∗w^*w∗ is

Π(qˉN(w∗))Π(qˉI)=1−1(2+β(n−1))2.\frac{\Pi(\bar q^N(w^*))}{\Pi(\bar q^I)} = 1 - \frac{1}{(2+\beta(n-1))^2}.Π(qˉ​I)Π(qˉ​N(w∗))​=1−(2+β(n−1))21​.

Significance

The goal shows that the single-retailer coordination result survives competition, with one change: the coordinating price must charge each retailer for the externality it imposes on the others, so it depends on every location's revenue function, and wiIw^I_iwiI​ exceeds the production cost. The supplier's profit then moves along a line between what wholesale prices alone give her and the whole system profit, which is how revenue sharing provides a profit split that linear prices cannot. The Cournot results make the comparison quantitative: when retailers compete intensely, the supplier's own optimal wholesale price already achieves most of the integrated profit, so revenue sharing, which has administrative costs, is less attractive.

No machine-checked proof of these results is known. The mission produces a reusable formal description of an nnn-player quantity game under per-retailer linear contracts, equilibrium conditions for it, and a fully worked Cournot instance, including a uniqueness claim for equilibria among all (not only symmetric) profiles.

Difficulty

The equilibrium claims are global: a retailer must not gain from any nonnegative deviation, not only from small ones. A first-order condition at qˉI\bar q^Iqˉ​I does not give this by itself. The page assumes RiR_iRi​ unimodal in qiq_iqi​, but unimodality does not survive subtracting the linear purchase cost, so the first-order condition is not sufficient under that assumption alone; the formalization uses concavity in the own quantity, under which it is. The participation claim πri(qˉI,wˉI)≥0\pi_{r_i}(\bar q^I,\bar w^I) \ge 0πri​​(qˉ​I,wˉI)≥0 is stated on the page without proof and needs a bound on the revenue of a location that stocks nothing.

In the Cournot example, uniqueness of the equilibrium must exclude asymmetric profiles and profiles where some retailers stock nothing, and the supplier's optimal price must be compared against every equilibrium at every price, including prices at which the retailers order nothing.

Formalization scope

Locations are Fin n; a profile is Fin n → ℝ; revenues are R : Fin n → (Fin n → ℝ) → ℝ, and dR i j q is Rji(qˉ)=∂Rj/∂qiR_j^i(\bar q) = \partial R_j/\partial q_iRji​(qˉ​)=∂Rj​/∂qi​, given as a partial derivative at every profile with all entries positive. A deviation of retailer iii to xxx is Function.update q i x, and Nash equilibria quantify over all x≥0x \ge 0x≥0. The standing assumptions of Section 3.2 are fields of the structure Model: c>0c > 0c>0; continuity of RiR_iRi​ on the nonnegative orthant; the partial derivatives; ∂2Ri/∂qi∂qj≤0\partial^2 R_i/\partial q_i\partial q_j \le 0∂2Ri​/∂qi​∂qj​≤0, encoded as "RiiR_i^iRii​ does not increase in qjq_jqj​"; and concavity of RiR_iRi​ in qiq_iqi​, which is the formalization's reading of "unimodal in qiq_iqi​". The paper's assumption that marginal revenue eventually falls below every δ>0\delta > 0δ>0 is used only for existence of an equilibrium, which is not formalized, and is omitted.

Deviations from the page, each disclosed in the item concerned:

  • qˉI\bar q^Iqˉ​I is taken as any positive solution of (6); its optimality for Π\PiΠ is not used.
  • "qˉ∗\bar q^*qˉ​∗ is a Nash equilibrium" (p. 13) is read as qˉI\bar q^Iqˉ​I.
  • In ϕ(Ri(qˉI)−qiIwi)\phi(R_i(\bar q^I) - q_i^I w_i)ϕ(Ri​(qˉ​I)−qiI​wi​) (p. 14), wiw_iwi​ is read as wiIw_i^IwiI​.
  • "Rii(qˉI)>cR_i^i(\bar q^I) > cRii​(qˉ​I)>c" needs a negative externality ∑j≠iRji(qˉI)<0\sum_{j\ne i}R_j^i(\bar q^I) < 0∑j=i​Rji​(qˉ​I)<0, which is assumed.
  • "Rji(qˉ)≤0R_j^i(\bar q) \le 0Rji​(qˉ​)≤0" is not a standing assumption, so it is a hypothesis of wiI≥cw_i^I \ge cwiI​≥c, and strictness needs some strictly negative cross-effect.
  • πri(qˉI,wˉI)≥0\pi_{r_i}(\bar q^I,\bar w^I) \ge 0πri​​(qˉ​I,wˉI)≥0 assumes nonnegative revenue at a location that stocks nothing.
  • In the Cournot example the implicit w<1w < 1w<1 and 0<c<10 < c < 10<c<1 are hypotheses; all retailers pay a common price; "increasing" is strict exactly where it holds (n≥2n \ge 2n≥2 for β\betaβ, β>0\beta > 0β>0 for nnn).

A formalization that defines "the prices coordinate" as "the prices satisfy the first-order condition at qˉI\bar q^Iqˉ​I" restates (6) and is ruled out: every equilibrium claim here is the game-theoretic statement about unilateral deviations. Contributions welcome: proofs of the equilibrium lemmas from concavity and the derivative, the Cournot uniqueness argument, and a general existence theorem for the quantity game.

Selected references

  • G. P. Cachon, M. A. Lariviere, Supply Chain Coordination with Revenue-Sharing Contracts: Strengths and Limitations, working paper, June 2000. Published version: Management Science 51(1):30–44, 2005. https://doi.org/10.1287/mnsc.1040.0215
  • D. Fudenberg, J. Tirole, Game Theory, MIT Press, 1991 (Theorem 1.2, existence of pure-strategy equilibria).
  • F. Bernstein, A. Federgruen, Pricing and Replenishment Strategies in a Distribution System with Competing Retailers, Operations Research 51(3):409–426, 2003. https://doi.org/10.1287/opre.51.3.409.14957
  • J. Tirole, The Theory of Industrial Organization, MIT Press, 1988.
16 thms3 active usersReviewed
🏆Completed
Control TheoryDynamic ProgrammingOptimization·Captain: mikedeng1

Stochastic Optimal Control: The Discrete-Time Case II: Contraction Models — the Optimal Cost Is the Unique Fixed Point of T in the Closed Set B̄Textbook

Motivation

Discounted dynamic programming with bounded cost per stage is the standard setting in which infinite-horizon sequential decision problems are well posed: the optimal cost exists, satisfies Bellman's equation, and can be computed by iterating the DP operator. Shapley proved this for stochastic games in 1953 (Shapley 1953), Blackwell for discounted Markov decision processes in 1965 (Blackwell 1965), and Denardo observed in 1967 that the arguments use only two properties of the DP operator: monotonicity and contraction in the supremum norm (Denardo 1967). Bertsekas (1975, 1977) and Bertsekas and Shreve (1978) turned this observation into an abstract dynamic programming framework, in which a single mapping HHH encodes stochastic, deterministic, minimax and multiplicative-cost problems at once (Bertsekas 1977).

Chapter 4 of Bertsekas and Shreve, Stochastic Optimal Control: The Discrete-Time Case, is the contraction part of that framework. Its results are the abstract form of what every course on Markov decision processes proves for the discounted case, and they are what later work on abstract DP (Bertsekas, Abstract Dynamic Programming, 2022) and on robust and regularized MDPs builds on.

Setting

A model consists of a state space SSS, a control space CCC, a nonempty constraint set U(x)⊆CU(x)\subseteq CU(x)⊆C for each x∈Sx\in Sx∈S, a mapping H:S×C×F→[−∞,∞]H:S\times C\times F\to[-\infty,\infty]H:S×C×F→[−∞,∞], where FFF is the set of functions S→[−∞,∞]S\to[-\infty,\infty]S→[−∞,∞], and a function J0∈FJ_0\in FJ0​∈F with J0>−∞J_0>-\inftyJ0​>−∞. HHH is monotone: J≤J′J\le J'J≤J′ implies H(x,u,J)≤H(x,u,J′)H(x,u,J)\le H(x,u,J')H(x,u,J)≤H(x,u,J′).

A selector is a function μ:S→C\mu:S\to Cμ:S→C with μ(x)∈U(x)\mu(x)\in U(x)μ(x)∈U(x); MMM is the set of selectors, and a policy is a sequence π=(μ0,μ1,… )\pi=(\mu_0,\mu_1,\dots)π=(μ0​,μ1​,…) in MMM. The operators are

Tμ(J)(x)=H(x,μ(x),J),T(J)(x)=inf⁡u∈U(x)H(x,u,J).T_\mu(J)(x)=H(x,\mu(x),J),\qquad T(J)(x)=\inf_{u\in U(x)}H(x,u,J).Tμ​(J)(x)=H(x,μ(x),J),T(J)(x)=u∈U(x)inf​H(x,u,J).

The cost of π\piπ is Jπ(x)=lim⁡N→∞(Tμ0⋯TμN−1)(J0)(x)J_\pi(x)=\lim_{N\to\infty}(T_{\mu_0}\cdots T_{\mu_{N-1}})(J_0)(x)Jπ​(x)=limN→∞​(Tμ0​​⋯TμN−1​​)(J0​)(x), the optimal cost is J∗(x)=inf⁡πJπ(x)J^*(x)=\inf_{\pi}J_\pi(x)J∗(x)=infπ​Jπ​(x), and JμJ_\muJμ​ is the cost of the stationary policy (μ,μ,… )(\mu,\mu,\dots)(μ,μ,…).

BBB is the Banach space of bounded real functions on SSS with ∥J∥=sup⁡x∣J(x)∣\|J\|=\sup_x|J(x)|∥J∥=supx​∣J(x)∣. Assumption C asks for a closed set Bˉ⊆B\bar B\subseteq BBˉ⊆B containing J0J_0J0​ and invariant under TTT and every TμT_\muTμ​; that every limit defining JπJ_\piJπ​ exist and be real; and that for some integer m≥1m\ge1m≥1 and scalars 0<ρ<10<\rho<10<ρ<1, α>0\alpha>0α>0,

∥Tμ(J)−Tμ(J′)∥≤α∥J−J′∥(J,J′∈B),∥(Tμ0⋯Tμm−1)(J)−(Tμ0⋯Tμm−1)(J′)∥≤ρ∥J−J′∥(J,J′∈Bˉ).\|T_\mu(J)-T_\mu(J')\|\le\alpha\|J-J'\|\quad(J,J'\in B),\qquad \|(T_{\mu_0}\cdots T_{\mu_{m-1}})(J)-(T_{\mu_0}\cdots T_{\mu_{m-1}})(J')\|\le\rho\|J-J'\|\quad(J,J'\in\bar B).∥Tμ​(J)−Tμ​(J′)∥≤α∥J−J′∥(J,J′∈B),∥(Tμ0​​⋯Tμm−1​​)(J)−(Tμ0​​⋯Tμm−1​​)(J′)∥≤ρ∥J−J′∥(J,J′∈Bˉ).

Formalization targets

Goal: Proposition 4.2

Under Assumption C,

J∗∈Bˉ,J∗=T(J∗),J′∈Bˉ, J′=T(J′) ⇒ J′=J∗,J^*\in\bar B,\qquad J^*=T(J^*),\qquad J'\in\bar B,\ J'=T(J')\ \Rightarrow\ J'=J^*,J∗∈Bˉ,J∗=T(J∗),J′∈Bˉ, J′=T(J′) ⇒ J′=J∗,

T(J′)≤J′T(J')\le J'T(J′)≤J′ implies J∗≤J′J^*\le J'J∗≤J′ and J′≤T(J′)J'\le T(J')J′≤T(J′) implies J′≤J∗J'\le J^*J′≤J∗ for J′∈BˉJ'\in\bar BJ′∈Bˉ; each JμJ_\muJμ​ is the unique fixed point of TμT_\muTμ​ in Bˉ\bar BBˉ; and for every J∈BˉJ\in\bar BJ∈Bˉ

lim⁡N→∞∥TN(J)−J∗∥=0,lim⁡N→∞∥TμN(J)−Jμ∥=0.\lim_{N\to\infty}\|T^N(J)-J^*\|=0,\qquad\lim_{N\to\infty}\|T_\mu^N(J)-J_\mu\|=0.N→∞lim​∥TN(J)−J∗∥=0,N→∞lim​∥TμN​(J)−Jμ​∥=0.

The statement carries no constants beyond those of Assumption C.

Milestones

  • Fixed Point Theorem (p. 55): an mmm-step contraction of a nonempty closed subset of a Banach space has a unique fixed point, which attracts every orbit.
  • Proposition 4.1 (p. 53): JπJ_\piJπ​ does not depend on the terminal function in Bˉ\bar BBˉ; inf⁡π(Tμ0⋯TμN−1)(J)=TN(J)\inf_\pi(T_{\mu_0}\cdots T_{\mu_{N-1}})(J)=T^N(J)infπ​(Tμ0​​⋯TμN−1​​)(J)=TN(J); TmT^mTm and TμmT_\mu^mTμm​ are ρ\rhoρ-contractions on Bˉ\bar BBˉ.
  • Proposition 4.3 (p. 56): (μ∗,μ∗,… )(\mu^*,\mu^*,\dots)(μ∗,μ∗,…) is optimal iff Tμ∗(J∗)=T(J∗)T_{\mu^*}(J^*)=T(J^*)Tμ∗​(J∗)=T(J∗); pointwise optimal policies yield a stationary optimal one; stationary ε\varepsilonε-optimal policies exist.
  • Proposition 4.4 (p. 57): compactness of the sets {u∈U(x)∣H[x,u,Tk(Jˉ)]≤λ}\{u\in U(x)\mid H[x,u,T^k(\bar J)]\le\lambda\}{u∈U(x)∣H[x,u,Tk(Jˉ)]≤λ} gives policies attaining the DP infimum, and their accumulation points are optimal stationary policies.
  • Proposition 4.11 (p. 69): the discounted minimax model with 0≤g≤b0\le g\le b0≤g≤b and α<1\alpha<1α<1 satisfies Assumption C with Bˉ=B\bar B=BBˉ=B, m=1m=1m=1, ρ=α\rho=\alphaρ=α.

Further result

  • Proposition 4.5 (p. 59), a draft theorem of this mission that is not a milestone: the error bound J∗≤Jμ≤J∗+(2αε1+ε2)(1+α+⋯+αm−1)/(1−ρ)J^*\le J_\mu\le J^*+(2\alpha\varepsilon_1+\varepsilon_2)(1+\alpha+\cdots+\alpha^{m-1})/(1-\rho)J∗≤Jμ​≤J∗+(2αε1​+ε2​)(1+α+⋯+αm−1)/(1−ρ).

Significance

Proposition 4.2 is the existence-and-uniqueness theorem for Bellman's equation in the contraction regime, together with the convergence of value iteration from an arbitrary start in Bˉ\bar BBˉ. Propositions 4.3 to 4.5 turn it into statements about policies: when a stationary optimal policy exists, how one is found from the DP algorithm, and how much is lost when Bellman's equation is solved only approximately. Proposition 4.11 shows the assumption is met by a concrete class of problems, discounted minimax control, and so certifies that the abstract theorems are not vacuous.

The results are classical and have been proved in print since 1978; none of them is open. What this mission adds is a machine-checked version of the abstract theory itself, rather than of a single model. Mathlib has the Banach fixed point theorem for a contracting map of a complete space (ContractingWith) and a lemma for contracting iterates, but not the version on a closed subset with norm convergence of every orbit, and nothing on abstract DP. The platform has proved the finite-state discounted case for a concrete model (BertsekasDP.discounted_main_theorem); the abstract statements here cover infinite state spaces, minimax problems and mmm-step contractions, and are reused by the later missions of this series (generalized models, Chapter 6) and by papers that cite the book.

Difficulty

The first idea is to apply the contraction mapping principle to TTT and read off J∗J^*J∗ as its fixed point. That gives a fixed point of TTT but says nothing about J∗J^*J∗, which is defined as an infimum over all, generally nonstationary, policies of limits of compositions. The identification of the fixed point with J∗J^*J∗ is the content of the proposition, and it is where the Lipschitz condition (2) on all of BBB, not only on Bˉ\bar BBˉ, enters.

Two further features block a direct appeal to Mathlib. The contraction is only mmm-step, so neither TTT nor TμT_\muTμ​ need be a contraction. And HHH takes extended-real values, so every passage between FFF and the Banach space BBB must be justified by the invariance of Bˉ\bar BBˉ.

Formalization scope

The state and control spaces are arbitrary types. FFF is S → EReal; BBB is Mathlib's lp (fun _ : S => ℝ) ⊤, whose norm is the supremum norm, and toF embeds BBB into FFF. Bˉ\bar BBˉ is an arbitrary closed subset of BBB, not BBB itself, and uniqueness of fixed points is asserted within Bˉ\bar BBˉ. Policies are sequences ℕ → M; (Tμ0⋯TμN−1)(J)(T_{\mu_0}\cdots T_{\mu_{N-1}})(J)(Tμ0​​⋯TμN−1​​)(J) applies TμN−1T_{\mu_{N-1}}TμN−1​​ first. JπJ_\piJπ​ is the pointwise limit (limUnder), which exists and is real under Assumption C; J∗J^*J∗ is the infimum over all policies.

The book computes in [−∞,∞][-\infty,\infty][−∞,∞] with ∞−∞=∞\infty-\infty=\infty∞−∞=∞, whereas Mathlib's EReal has ⊥+⊤=⊥\bot+\top=\bot⊥+⊤=⊥. No statement adds infinities of opposite sign. A norm bound ∥J−J′∥≤c\|J-J'\|\le c∥J−J′∥≤c between functions of FFF is the predicate SupDistLe: both functions are real at every point and differ by at most ccc, which is what the bound means under the book's arithmetic. Condition (2) is imposed on all of BBB, as on p. 53. The scalars m,ρ,αm,\rho,\alpham,ρ,α of Assumption C are explicit parameters, so the constant of Proposition 4.5 is the book's exact expression. The Fixed Point Theorem assumes Bˉ\bar BBˉ nonempty, which the page leaves implicit and without which the statement is false.

Defining J∗J^*J∗ as the fixed point of TTT, or replacing it by the infimum over stationary policies, would make the goal trivial. Neither is done here: J∗J^*J∗ is the infimum of the policy costs, exactly as in Eq. (8) of Chapter 2.

A complete development needs the mmm-step fixed point theorem on closed subsets of a Banach space, which can be reused well beyond dynamic programming; the elementary calculus of SupDistLe and of the embedding of BBB into S → EReal; and the monotone-operator inequalities of Section 2.1. Contributions of any of these, or alternative proofs of the milestones, are welcome.

Selected references

  • D. P. Bertsekas and S. E. Shreve, Stochastic Optimal Control: The Discrete-Time Case, Academic Press 1978; Athena Scientific reprint 1996, Chapter 4. http://web.mit.edu/dimitrib/www/soc.html
  • D. P. Bertsekas, Monotone mappings with application in dynamic programming, SIAM J. Control Optim. 15 (1977) 438–464. https://doi.org/10.1137/0315031
  • E. V. Denardo, Contraction mappings in the theory underlying dynamic programming, SIAM Review 9 (1967) 165–177. https://doi.org/10.1137/1009030
  • D. Blackwell, Discounted dynamic programming, Ann. Math. Statist. 36 (1965) 226–235. https://doi.org/10.1214/aoms/1177700285
  • L. S. Shapley, Stochastic games, Proc. Natl. Acad. Sci. USA 39 (1953) 1095–1100. https://doi.org/10.1073/pnas.39.10.1095
  • D. P. Bertsekas, Abstract Dynamic Programming, 3rd ed., Athena Scientific 2022. https://web.mit.edu/dimitrib/www/abstractdp_MIT.html
8 thms3 active usersReviewed
🏆Completed
CombinatoricsOptimizationProbability+1·Captain: mikedeng1

A Tight Linear Time (1/2)-Approximation for Unconstrained Submodular Maximization 2: Randomized Double Greedy Achieves 1/2 of the Optimum in ExpectationResearch Paper

Motivation

Many selection problems assign a value to each subset of a finite collection: the coverage supplied by chosen facilities, the influence reached by chosen seeds, or the value of a coalition. A submodular set function has diminishing returns in the precise sense that the combined value of two sets, counting their overlap once, does not exceed the sum of their separate values. When the function is also monotone, taking more elements never hurts. The unconstrained problem studied here permits nonmonotone functions, so both accepting and rejecting an element can matter. The question is what a single pass through the elements can guarantee when the function is available through value queries. Buchbinder et al., FOCS 2012

The randomized algorithm in this mission attains an expected one-half approximation for every nonnegative submodular function. The paper presents this as tight in the value-oracle setting: it recalls the earlier result of Feige, Mirrokni and Vondrák that a fixed improvement beyond one-half requires exponentially many queries. The contribution here is therefore both the guarantee and a short adaptive rule that attains it in a linear number of iterations. The local proposal follows the FOCS 2012 version of the paper; its theorem numbering differs from the later SIAM Journal on Computing article. Buchbinder et al., §I.A and Theorem I.2

Setting

Let N\mathcal NN be a finite ground set, and let f:2N→R≥0f:2^{\mathcal N}\to\mathbb R_{\ge0}f:2N→R≥0​ assign a nonnegative real value to every subset. The unconstrained submodular maximization problem asks for the largest value f(S)f(S)f(S) among all S⊆NS\subseteq\mathcal NS⊆N. Write OPTOPTOPT for that value when no confusion arises, and OOO for a set attaining it. Submodularity means

f(A∪B)+f(A∩B)≤f(A)+f(B)(A,B⊆N).f(A\cup B)+f(A\cap B)\le f(A)+f(B)\qquad(A,B\subseteq\mathcal N).f(A∪B)+f(A∩B)≤f(A)+f(B)(A,B⊆N).

There is no monotonicity or normalization assumption: f(∅)f(\varnothing)f(∅) and f(N)f(\mathcal N)f(N) may both be positive. A value oracle returns f(S)f(S)f(S) for a requested subset SSS. The paper's complexity claim counts such queries, assuming a query takes constant time. Buchbinder et al., §I and footnotes 1–2

Algorithm 2 visits the elements once in an arbitrary order u1,…,unu_1,\ldots,u_nu1​,…,un​. It keeps two sets, starting at X0=∅X_0=\varnothingX0​=∅ and Y0=NY_0=\mathcal NY0​=N. At step iii, it measures the gain aia_iai​ from adding uiu_iui​ to Xi−1X_{i-1}Xi−1​ and the gain bib_ibi​ from removing uiu_iui​ from Yi−1Y_{i-1}Yi−1​. It clips each gain at zero, giving ai′=max⁡(ai,0)a'_i=\max(a_i,0)ai′​=max(ai​,0) and bi′=max⁡(bi,0)b'_i=\max(b_i,0)bi′​=max(bi​,0). It adds uiu_iui​ to XXX with probability ai′/(ai′+bi′)a'_i/(a'_i+b'_i)ai′​/(ai′​+bi′​) and otherwise removes it from YYY. When both clipped gains vanish, the paper defines the add probability as one. After all elements have been processed, the two sets coincide, and the algorithm returns their common value. The state law is adaptive: its probability at step iii depends on the actual pair of sets produced by earlier choices. Buchbinder et al., Algorithm 2

Formalization targets

The main target is Theorem I.2 for this exact algorithm and for every enumeration of the ground set:

max⁡S⊆Nf(S)≤2 E[f(Xn)].\max_{S\subseteq\mathcal N}f(S)\le 2\,\mathbb E[f(X_n)].S⊆Nmax​f(S)≤2E[f(Xn​)].

The milestone statements retain the paper's key local quantities. For a comparison optimum OOO, set OPTi=(O∪Xi)∩YiOPT_i=(O\cup X_i)\cap Y_iOPTi​=(O∪Xi​)∩Yi​. Lemma II.1 asserts ai+bi≥0a_i+b_i\ge0ai​+bi​≥0. The endpoint statement identifies OPT0=OOPT_0=OOPT0​=O and OPTn=Xn=YnOPT_n=X_n=Y_nOPTn​=Xn​=Yn​. Inequality (3) bounds the conditional loss in the positive-gain case; Lemma III.1 compares the expected change of OPTiOPT_iOPTi​ with the expected combined change of XiX_iXi​ and YiY_iYi​. The telescoped display keeps the initial endpoint values f(∅)f(\varnothing)f(∅) and f(N)f(\mathcal N)f(N) before using nonnegativity. Buchbinder et al., Lemmas II.1 and III.1, inequality (3), proof of Theorem I.2

A companion target is Theorem I.4 via its second proof. For two normalized monotone submodular utilities f1,f2f_1,f_2f1​,f2​, let g(S)=f1(S)+f2(N∖S)g(S)=f_1(S)+f_2(\mathcal N\setminus S)g(S)=f1​(S)+f2​(N∖S). The maximum of ggg is exactly the optimal welfare of a two-player partition. Algorithm 2 on ggg is asked to satisfy

3max⁡S⊆Ng(S)≤4 E[g(Xn)].3\max_{S\subseteq\mathcal N}g(S)\le4\,\mathbb E[g(X_n)].3S⊆Nmax​g(S)≤4E[g(Xn​)].

This is the paper's three-quarter guarantee in its welfare application. Buchbinder et al., Theorem I.4 and Proof (2)

Significance

The main theorem gives a specific randomized rule whose expected value is at least half the best subset value, even when accepting an element can lower the objective. It applies without restricting the cardinality or shape of the chosen subset. The welfare corollary shows that keeping the initial endpoint values in the analysis yields a stronger guarantee for the objective formed from two monotone players. Buchbinder et al., Theorems I.2 and I.4

This mission formalizes the statement of the algorithm, its intermediate state laws, its comparison set, and the paper's numbered proof targets. The algorithmic guarantee is proved in the source paper; the local Lean theorem files are open statements with sorry and do not yet give machine-checked proofs of these results. A completed development would supply a reusable formal model of an adaptive finite random process over pairs of subsets, as well as the specific submodular inequalities. The published Submodular and OPT definitions from the earlier Feige–Mirrokni–Vondrák formalization are reused here.

Difficulty

The two possible updates cannot be assessed independently. The probability of each choice depends on the current state, and the comparison set OPTiOPT_iOPTi​ can gain or lose the processed element in a way that differs from the two algorithm sets. A bound on the expected value of XiX_iXi​ alone does not control the movement of OPTiOPT_iOPTi​. The proof must handle the clipped gains, including the case when both are zero, while preserving the exact joint law of (Xi,Yi)(X_i,Y_i)(Xi​,Yi​). Buchbinder et al., proof of Lemma III.1

Formalization scope

The ground set is a finite Lean type; subsets are Finset X, and values are real numbers. An order is a list with no repeated elements that covers the type, including the empty type. The run is an explicit finite mass function on pairs of subsets after every prefix of the list. Expectation is a finite weighted sum, so it has no integrability exception. The transition clips the two real marginal gains and handles 0/00/00/0 by assigning probability one to the add branch, exactly as Algorithm 2 specifies. The optimum is the published maximum over all subsets. No ratio divides by a possibly zero optimum.

The theorem fixes Algorithm 2 itself; an arbitrary process with nested sets or a process defined by its desired approximation property does not satisfy this scope. The Lean goal states the value bound and leaves the paper's linear-time claim outside the formal theorem. The algorithm uses four value evaluations per processed element in its printed rule; the Lean development represents those evaluations, not an implementation cost model. The statement that its two final sets coincide is a separate milestone.

The source's main-text decreasing-returns definition has an overbroad quantifier on the added element. This development uses the equivalent lattice inequality given in the paper's footnote, which permits nonmonotone functions. The proof of Lemma II.1 also has a set-index slip, and the proof of Theorem I.2 prints FFF for fff in one display; neither slip is copied into a formal statement. The one-step inequality (3) is stated for any nested pair with the processed element in Y∖XY\setminus XY∖X, a generalization of the conditioned reachable states in the paper. Contributions proving the endpoint invariant, conditional inequality, one-step expected estimate, and final bound are all within scope.

Selected references

  • Niv Buchbinder, Moran Feldman, Joseph Naor and Roy Schwartz, A Tight Linear Time (1/2)-Approximation for Unconstrained Submodular Maximization, Proceedings of the 53rd IEEE Symposium on Foundations of Computer Science, 2012. FOCS version used here.
  • Niv Buchbinder, Moran Feldman, Joseph Naor and Roy Schwartz, A Tight Linear Time (1/2)-Approximation for Unconstrained Submodular Maximization, SIAM Journal on Computing 44(5), 2015. DOI: 10.1137/130929205. The cited statement indices above refer to the FOCS version.
11 thms3 active usersReviewed
🏆Completed
Dynamical SystemsProbabilityStochastic Systems·Captain: mikedeng1

Dynamics of Stochastic Approximation Algorithms 4: Subgaussian Martingale Noise with Σ exp(−c/γ_n) < ∞ for Every c > 0 Satisfies Assumption A1 Almost SurelyResearch Paper

Motivation

A stochastic approximation algorithm is a recursion

xn+1−xn=γn+1(F(xn)+Un+1)x_{n+1}-x_n=\gamma_{n+1}\big(F(x_n)+U_{n+1}\big)xn+1​−xn​=γn+1​(F(xn​)+Un+1​)

in Rd\mathbb R^dRd, where FFF is a vector field, γn\gamma_nγn​ are small step sizes and Un+1U_{n+1}Un+1​ is noise. Such recursions go back to Robbins and Monro's root-finding scheme (Robbins–Monro 1951) and underlie stochastic gradient descent, temporal-difference learning, adaptive control and learning in games. The ODE method studies them by comparing the iterates with the trajectories of x˙=F(x)\dot x=F(x)x˙=F(x).

Benaïm's lecture notes (Benaïm 1999) organize the ODE method in two steps. A deterministic step, Proposition 4.1, shows that whenever the noise satisfies a condition called A1 (together with a boundedness condition on the iterates), the interpolated process is an asymptotic pseudotrajectory of the flow of FFF. A probabilistic step then verifies A1 for concrete noise models. Proposition 4.2 does this for martingale difference noise with bounded qqq-th moments, at the price of step sizes with ∑nγn1+q/2<∞\sum_n\gamma_n^{1+q/2}<\infty∑n​γn1+q/2​<∞. This mission formalizes the second verification, Proposition 4.4: when the noise is subgaussian, A1 holds almost surely under the much weaker requirement that ∑ne−c/γn<∞\sum_ne^{-c/\gamma_n}<\infty∑n​e−c/γn​<∞ for every c>0c>0c>0, which allows step sizes decaying only slightly faster than 1/log⁡n1/\log n1/logn. The notes attribute the result to Duflo (1997), see also Kushner and Yin (1997) and Benaïm and Hirsch (1996).

Setting

Let {γn}n≥1\{\gamma_n\}_{n\ge1}{γn​}n≥1​ be a deterministic sequence with γn≥0\gamma_n\ge0γn​≥0, ∑nγn=∞\sum_n\gamma_n=\infty∑n​γn​=∞ and γn→0\gamma_n\to0γn​→0 (a step sequence). Put τ0=0\tau_0=0τ0​=0, τn=∑i=1nγi\tau_n=\sum_{i=1}^n\gamma_iτn​=∑i=1n​γi​, and let

m(t)=sup⁡{k≥0: t≥τk}m(t)=\sup\{k\ge0:\ t\ge\tau_k\}m(t)=sup{k≥0: t≥τk​}

be the index of the step that contains time t≥0t\ge0t≥0. For a sequence {Un}n≥1\{U_n\}_{n\ge1}{Un​}n≥1​ define the piecewise constant processes Uˉ(t)=Um(t)+1\bar U(t)=U_{m(t)+1}Uˉ(t)=Um(t)+1​ and γˉ(t)=γm(t)+1\bar\gamma(t)=\gamma_{m(t)+1}γˉ​(t)=γm(t)+1​, so that step n+1n+1n+1 occupies the time interval [τn,τn+1)[\tau_n,\tau_{n+1})[τn​,τn+1​) of length γn+1\gamma_{n+1}γn+1​.

Assumption A1 asks that for every T>0T>0T>0

lim⁡n→∞sup⁡{∥∑i=nk−1γi+1Ui+1∥: k=n+1,…,m(τn+T)}=0,\lim_{n\to\infty}\sup\Big\{\Big\|\sum_{i=n}^{k-1}\gamma_{i+1}U_{i+1}\Big\|:\ k=n+1,\dots,m(\tau_n+T)\Big\}=0,n→∞lim​sup{​i=n∑k−1​γi+1​Ui+1​​: k=n+1,…,m(τn​+T)}=0,

or, in the form the notes call equivalent, lim⁡t→∞Δ(t,T)=0\lim_{t\to\infty}\Delta(t,T)=0limt→∞​Δ(t,T)=0 for every T>0T>0T>0, where

Δ(t,T)=sup⁡0≤h≤T∥∫tt+hUˉ(s) ds∥.\Delta(t,T)=\sup_{0\le h\le T}\Big\|\int_t^{t+h}\bar U(s)\,ds\Big\|.Δ(t,T)=0≤h≤Tsup​​∫tt+h​Uˉ(s)ds​.

Let (Ω,F,P)(\Omega,\mathcal F,P)(Ω,F,P) be a probability space with a nondecreasing sequence {Fn}\{\mathcal F_n\}{Fn​} of sub-σ\sigmaσ-algebras, and F:Rd→RdF:\mathbb R^d\to\mathbb R^dF:Rd→Rd continuous. A sequence {xn}\{x_n\}{xn​} given by the recursion above is a Robbins–Monro algorithm if γ\gammaγ is deterministic, UnU_nUn​ is Fn\mathcal F_nFn​-measurable, and E(Un+1∣Fn)=0E(U_{n+1}\mid\mathcal F_n)=0E(Un+1​∣Fn​)=0. The noise is subgaussian if there is a number Γ>0\Gamma>0Γ>0 such that for all nnn and all θ∈Rd\theta\in\mathbb R^dθ∈Rd

E(exp⁡⟨θ,Un+1⟩ ∣ Fn)≤exp⁡(Γ2∥θ∥2).E\big(\exp\langle\theta,U_{n+1}\rangle\,\big|\,\mathcal F_n\big)\le\exp\Big(\frac\Gamma2\|\theta\|^2\Big).E(exp⟨θ,Un+1​⟩​Fn​)≤exp(2Γ​∥θ∥2).

Bounded noise, ∥Un∥≤Γ\|U_n\|\le\sqrt\Gamma∥Un​∥≤Γ​, is an example.

Formalization targets

Goal: Proposition 4.4

For a Robbins–Monro algorithm with subgaussian noise and a deterministic step sequence such that

∑ne−c/γn<∞for each c>0,\sum_ne^{-c/\gamma_n}<\infty\qquad\text{for each }c>0,n∑​e−c/γn​<∞for each c>0,

with probability one the realised noise sequence satisfies A1, in both of its forms, simultaneously for all T>0T>0T>0.

Milestones

  1. The exponential supermartingale. For every θ∈Rd\theta\in\mathbb R^dθ∈Rd,
Zn(θ)=exp⁡[∑i=1n⟨θ,γiUi⟩−Γ2∑i=1nγi2∥θ∥2]Z_n(\theta)=\exp\Big[\sum_{i=1}^n\langle\theta,\gamma_iU_i\rangle-\frac\Gamma2\sum_{i=1}^n\gamma_i^2\|\theta\|^2\Big]Zn​(θ)=exp[i=1∑n​⟨θ,γi​Ui​⟩−2Γ​i=1∑n​γi2​∥θ∥2]

is a supermartingale. 2. Directional maximal tail bound. For every unit vector eee, α>0\alpha>0α>0, nnn and T>0T>0T>0,

P(sup⁡n<k≤m(τn+T)⟨e,∑i=nk−1γi+1Ui+1⟩≥α)≤exp⁡(−α22Γ∑i=nm(τn+T)−1γi+12).P\Big(\sup_{n<k\le m(\tau_n+T)}\Big\langle e,\sum_{i=n}^{k-1}\gamma_{i+1}U_{i+1}\Big\rangle\ge\alpha\Big)\le\exp\Big(\frac{-\alpha^2}{2\Gamma\sum_{i=n}^{m(\tau_n+T)-1}\gamma_{i+1}^2}\Big).P(n<k≤m(τn​+T)sup​⟨e,i=n∑k−1​γi+1​Ui+1​⟩≥α)≤exp(2Γ∑i=nm(τn​+T)−1​γi+12​−α2​).
  1. Eq. (18). There are C,C′>0C,C'>0C,C′>0 depending only on ddd and Γ\GammaΓ with
P(Δ(t,T)≥α)≤Cexp⁡(−α2C′∫tt+Tγˉ(s) ds)(t≥0, T>0, α>0).P(\Delta(t,T)\ge\alpha)\le C\exp\Big(\frac{-\alpha^2}{C'\int_t^{t+T}\bar\gamma(s)\,ds}\Big)\qquad(t\ge0,\ T>0,\ \alpha>0).P(Δ(t,T)≥α)≤Cexp(C′∫tt+T​γˉ​(s)ds−α2​)(t≥0, T>0, α>0).
  1. Block comparison. Δ(t,T)≤2Δ(kT,T)+Δ((k+1)T,T)\Delta(t,T)\le2\Delta(kT,T)+\Delta((k+1)T,T)Δ(t,T)≤2Δ(kT,T)+Δ((k+1)T,T) for kT≤t<(k+1)TkT\le t<(k+1)TkT≤t<(k+1)T.

Significance

Proposition 4.4 is the sufficient condition for the ODE method when the noise has Gaussian-type tails. Its step-size condition holds whenever γnlog⁡n→0\gamma_n\log n\to0γn​logn→0, so it admits steps that decrease far more slowly than the ∑γn2<∞\sum\gamma_n^2<\infty∑γn2​<∞ of the classical L2L^2L2 theory; slowly decreasing steps are what practitioners use to keep algorithms responsive. Combined with Proposition 4.1 it shows that the interpolated process of such an algorithm, with bounded iterates, is almost surely an asymptotic pseudotrajectory of the flow of FFF, and the limit set theorems of the notes then locate the limit points of the algorithm.

The result is proved in the notes and in the cited literature; it has not, to our knowledge, been machine-checked. A formal proof would add reusable pieces: an exponential supermartingale and maximal inequality for vector-valued martingale differences with a conditional subgaussian bound (Mathlib's conditional subgaussian notion is scalar), a Borel–Cantelli argument along the grid kTkTkT, and the continuous-time bookkeeping of Uˉ\bar UUˉ, γˉ\bar\gammaγˉ​ and Δ\DeltaΔ shared with the other missions of this series.

Difficulty

The moment method of Proposition 4.2 does not reach this regime: any fixed polynomial moment of the window sums decays only polynomially in the window's step sizes, and under ∑e−c/γn<∞\sum e^{-c/\gamma_n}<\infty∑e−c/γn​<∞ alone polynomial bounds are not summable over windows. Exponential tail bounds are needed, and they must be maximal (uniform over the window) and must hold for the norm of a vector, not only for a scalar. The continuous-time deviation Δ(t,T)\Delta(t,T)Δ(t,T) involves partial steps at both ends of [t,t+h][t,t+h][t,t+h], so the bound must be stated in terms of ∫tt+Tγˉ\int_t^{t+T}\bar\gamma∫tt+T​γˉ​ rather than a sum over whole steps, with constants that do not depend on ttt, TTT or α\alphaα. Finally, A1 quantifies over all T>0T>0T>0: the almost-sure statement must hold on a single event of full probability for every TTT.

Formalization scope

The space is Rd\mathbb R^dRd as EuclideanSpace ℝ (Fin d) (the paper writes Rm\mathbb R^mRm); time is real. The sequences γ\gammaγ and UUU are indexed by N\mathbb NN, and their values at 000 are unused, as the paper indexes them from 111. The filtration is a Mathlib Filtration ℕ; Un+1U_{n+1}Un+1​ is Fn+1\mathcal F_{n+1}Fn+1​-strongly measurable and integrable, and E(Un+1∣Fn)=0E(U_{n+1}\mid\mathcal F_n)=0E(Un+1​∣Fn​)=0 almost surely. The subgaussian condition requires exp⁡⟨θ,Un+1⟩\exp\langle\theta,U_{n+1}\rangleexp⟨θ,Un+1​⟩ to be integrable for every θ\thetaθ and nnn. The summand e−c/γne^{-c/\gamma_n}e−c/γn​ is taken to be 000 when γn=0\gamma_n=0γn​=0, its limiting value. The suprema in A1 and Δ\DeltaΔ are taken in [0,∞][0,\infty][0,∞]; the supremum over an empty range of kkk is 000. In Eq. (18) the constants are chosen before the probability space, the algorithm and t,T,αt,T,\alphat,T,α.

The following readings are excluded and are not acceptable formalizations: a subgaussian condition that holds vacuously because the exponential is not integrable (Lean's conditional expectation of a non-integrable function is 000); a summability condition made trivial or false by the convention c/0=0c/0=0c/0=0; and the conclusion "for each TTT, A1 holds almost surely" in place of "almost surely, A1 holds for all TTT". The second sentence of Proposition 4.4 (the asymptotic pseudotrajectory conclusion) is outside this mission.

All hypotheses are satisfiable: U=0U=0U=0, x=0x=0x=0, F=0F=0F=0, Γ=1\Gamma=1Γ=1 and γn=1/n\gamma_n=1/nγn​=1/n satisfy every one of them.

Contributions welcome: a maximal inequality for nonnegative supermartingales in the form needed here, vector subgaussian tail bounds for martingale transforms with deterministic weights (reusable well beyond this mission), lemmas on the step processes and Δ\DeltaΔ (measurability, local integrability, additivity), and the proofs of the milestones.

Selected references

  • M. Benaïm, Dynamics of Stochastic Approximation Algorithms, Séminaire de Probabilités XXXIII, Lecture Notes in Mathematics 1709, Springer, 1999, pp. 1–68. https://doi.org/10.1007/BFb0096509
  • M. Duflo, Random Iterative Models, Applications of Mathematics 34, Springer, 1997.
  • H. J. Kushner and G. G. Yin, Stochastic Approximation Algorithms and Applications, Springer, 1997.
  • M. Benaïm and M. W. Hirsch, Asymptotic pseudotrajectories and chain recurrent flows, with applications, Journal of Dynamics and Differential Equations 8 (1996), 141–176. https://doi.org/10.1007/BF02218617
  • H. Robbins and S. Monro, A stochastic approximation method, Annals of Mathematical Statistics 22 (1951), 400–407. https://doi.org/10.1214/aoms/1177729586
10 thms3 active usersReviewed
🏆Completed
AnalysisDynamic ProgrammingOptimization+2·Captain: mikedeng1

Stochastic Optimal Control: The Discrete-Time Case VI: Lower Semianalytic Functions — Analytically Measurable ε-Optimal Selectors (Jankov–von Neumann)Textbook

Motivation

Dynamic programming over uncountable state and control spaces needs two things at every stage: the optimal cost-to-go, obtained by minimizing over the control, must be a function that can be integrated against the next stage's transition probabilities, and a policy that nearly attains the minimum must be measurable, so that it defines a stochastic process. With Borel-measurable costs and Borel-measurable policies both requirements fail. Minimizing a Borel function of (x,y)(x,y)(x,y) over yyy produces a function whose level sets are projections of Borel sets, and such projections need not be Borel (Suslin, 1917). The repair, developed by Blackwell, Freedman and Orkin (1974), Shreve and Bertsekas, and set out in Chapter 7 of Bertsekas and Shreve's Stochastic Optimal Control: The Discrete-Time Case (1978), is to enlarge the class of costs to the lower semianalytic functions and the class of policies to the analytically or universally measurable ones. Sections 7.6–7.7 of the book establish that this class is closed under partial minimization and admits measurable ε-optimal selectors. Chapters 8–10 of the book, and much of the later literature on Borel-space Markov decision processes (Hernández-Lerma and Lasserre; Feinberg and coauthors), build on these results.

Timeline:

  • 1917: Suslin shows that projections of Borel sets need not be Borel and introduces analytic sets; Lusin proves that analytic sets are universally measurable.
  • 1941–1949: Jankov and von Neumann independently prove that an analytic subset of a product admits a selector measurable with respect to the σ-algebra generated by analytic sets.
  • 1974: Blackwell, Freedman and Orkin use analytic sets to construct ε-optimal policies in Borel dynamic programming.
  • 1978: Bertsekas and Shreve give the treatment used here (§7.6–7.7), including the selection theorem for lower semianalytic functions, Proposition 7.50.

Setting

A Borel space is a topological space homeomorphic to a Borel subset of a complete separable metric space (Definition 7.7); its Borel σ-algebra is BX\mathscr B_XBX​. The Baire space is N=NN\mathscr N=\mathbb N^{\mathbb N}N=NN with the product topology. A set A⊆XA\subseteq XA⊆X is analytic if it is empty or the image of N\mathscr NN under a continuous map; by Proposition 7.41 this is the book's Definition 7.16 (the Suslin operation applied to closed sets). Every Borel set is analytic, and the converse fails when XXX is uncountable.

Three σ-algebras on XXX are in play. The analytic σ-algebra AX\mathscr A_XAX​ is generated by the analytic sets (Definition 7.19). The universal σ-algebra is UX=⋂pBX(p)\mathscr U_X=\bigcap_{p}\mathscr B_X(p)UX​=⋂p​BX​(p), the intersection over all probability measures ppp on (X,BX)(X,\mathscr B_X)(X,BX​) of the ppp-completions of BX\mathscr B_XBX​ (Definition 7.18). For a function fff from D⊆XD\subseteq XD⊆X into a Borel space YYY, fff is analytically measurable if D∈AXD\in\mathscr A_XD∈AX​ and f−1(B)∈AXf^{-1}(B)\in\mathscr A_Xf−1(B)∈AX​ for every B∈BYB\in\mathscr B_YB∈BY​, and universally measurable if the same holds with UX\mathscr U_XUX​ (Definition 7.20).

Let R∗=[−∞,∞]R^*=[-\infty,\infty]R∗=[−∞,∞]. A function f:D→R∗f:D\to R^*f:D→R∗ is lower semianalytic if DDD is analytic and {x∈D∣f(x)<c}\{x\in D\mid f(x)<c\}{x∈D∣f(x)<c} is analytic for every real ccc (Definition 7.21). For D⊆X×YD\subseteq X\times YD⊆X×Y write Dx={y∣(x,y)∈D}D_x=\{y\mid (x,y)\in D\}Dx​={y∣(x,y)∈D}, projX(D)={x∣Dx≠∅}\mathrm{proj}_X(D)=\{x\mid D_x\neq\emptyset\}projX​(D)={x∣Dx​=∅}, and define the partial infimum

f∗(x)=inf⁡y∈Dxf(x,y),x∈projX(D).f^*(x)=\inf_{y\in D_x}f(x,y),\qquad x\in\mathrm{proj}_X(D).f∗(x)=y∈Dx​inf​f(x,y),x∈projX​(D).

A selector is a function φ:projX(D)→Y\varphi:\mathrm{proj}_X(D)\to Yφ:projX​(D)→Y whose graph Gr(φ)\mathrm{Gr}(\varphi)Gr(φ) lies in DDD.

Formalization targets

Goal: Proposition 7.50

Let X,YX,YX,Y be Borel spaces, D⊆X×YD\subseteq X\times YD⊆X×Y analytic, and f:D→R∗f:D\to R^*f:D→R∗ lower semianalytic.

(a) For every ε>0\varepsilon>0ε>0 there is an analytically measurable selector φ\varphiφ with

f[x,φ(x)]≤{f∗(x)+εif f∗(x)>−∞,−1/εif f∗(x)=−∞.f[x,\varphi(x)]\le\begin{cases}f^*(x)+\varepsilon&\text{if }f^*(x)>-\infty,\\-1/\varepsilon&\text{if }f^*(x)=-\infty.\end{cases}f[x,φ(x)]≤{f∗(x)+ε−1/ε​if f∗(x)>−∞,if f∗(x)=−∞.​

(b) The set III of points where the infimum is attained is universally measurable, and for every ε>0\varepsilon>0ε>0 there is a universally measurable selector φ\varphiφ with f[x,φ(x)]=f∗(x)f[x,\varphi(x)]=f^*(x)f[x,φ(x)]=f∗(x) on III and the bounds of (a) off III.

The goal fixes no constant beyond the book's ε\varepsilonε and −1/ε-1/\varepsilon−1/ε.

Milestones

In attack order: Proposition 7.40 (Borel images and preimages of analytic sets are analytic), Corollary 7.42.1 (AX⊆UX\mathscr A_X\subseteq\mathscr U_XAX​⊆UX​), Corollary 7.44.2 (composites of analytically measurable maps are universally measurable), and Proposition 7.49, the Jankov–von Neumann theorem:

A⊆X×Y analytic ⟹ ∃ φ:projX(A)→Y analytically measurable, Gr(φ)⊆A.A\subseteq X\times Y\text{ analytic}\ \Longrightarrow\ \exists\,\varphi:\mathrm{proj}_X(A)\to Y\ \text{analytically measurable},\ \mathrm{Gr}(\varphi)\subseteq A.A⊆X×Y analytic ⟹ ∃φ:projX​(A)→Y analytically measurable, Gr(φ)⊆A.

Further items of the mission, on the same definitions: Proposition 7.39 (projections of analytic sets are analytic, and every analytic set is a projection of a Borel set), Lemma 7.30(1) (strict and non-strict, real and extended level sets give the same class) and Proposition 7.47 (lower semianalytic functions are exactly partial infima of Borel functions).

Significance

Proposition 7.50 is the selection theorem behind the existence of ε-optimal policies in Borel-space dynamic programming. In the finite-horizon model of Chapter 8 the optimal cost-to-go at each stage is lower semianalytic, by Propositions 7.47 and 7.48. Proposition 7.50 then turns the one-stage minimization into a measurable policy, analytically measurable when only ε-optimality is required and universally measurable when the minimum is attained. Chapters 8–9 of the book (the finite-horizon recursion JK∗=TK(J0)J^*_K=T^K(J_0)JK∗​=TK(J0​) and the optimality equation under (P), (N), (D)) use it at every step. Downstream catalog papers on average-cost and stochastic shortest-path problems over Borel spaces cite these results.

All results here are proved in the book and in the descriptive set theory literature (Kechris, Classical Descriptive Set Theory, §18 and §29). None is formalized on Prove2Me. Mathlib has analytic sets in Polish-type settings, the Lusin separation theorem and Suslin's theorem, but it has no universal σ-algebra, no analytic σ-algebra, no lower semianalytic functions and no Jankov–von Neumann uniformization. The definitions in this mission are reusable by the later missions of the series (Chapters 8–10), which restate them locally until these are published.

Difficulty

The obvious route to a selector is to choose, for each xxx, a minimizing or near-minimizing yyy. The axiom of choice provides such a function, but nothing makes it measurable, and the conclusion of the theorem is exactly that measurability. The Borel route fails too: the set {x∣f∗(x)<c}\{x\mid f^*(x)<c\}{x∣f∗(x)<c} is a projection of a Borel set, which is analytic but in general not Borel, so no Borel-measurable selector exists in general. The Jankov–von Neumann theorem needs a lexicographically least branch of a continuous parametrization of AAA by N\mathscr NN, and an argument that the resulting map is measurable with respect to AX\mathscr A_XAX​, which is generated by sets that are not closed under complementation. Part (b) adds a further obstacle: the composite of two analytically measurable maps need not be analytically measurable, so the exact selector is only universally measurable. Proving that requires Lusin's theorem that analytic sets are measurable for every completed probability measure.

Formalization scope

  • A Borel space is a type with a topology satisfying the class IsBorelSpace (Definition 7.7, the ambient complete separable metric space taken in the same universe), together with Mathlib's [MeasurableSpace X] [BorelSpace X], so measurable sets are exactly the Borel sets. On X×YX\times YX×Y the product σ-algebra is used; it coincides with BX×Y\mathscr B_{X\times Y}BX×Y​ for separable metrizable spaces (Proposition 7.13).
  • Analytic sets are Mathlib's MeasureTheory.AnalyticSet (empty or a continuous image of ℕ → ℕ).
  • R∗R^*R∗ is EReal. The book uses ∞−∞=∞\infty-\infty=\infty∞−∞=∞, and Mathlib's EReal uses ⊥+⊤=⊥\bot+\top=\bot⊥+⊤=⊥. No statement of this mission adds infinities of opposite sign; f∗(x)+εf^*(x)+\varepsilonf∗(x)+ε adds a real number.
  • Functions on DDD and on projX(D)\mathrm{proj}_X(D)projX​(D) are functions on subtypes. The graph condition Gr(φ)⊆D\mathrm{Gr}(\varphi)\subseteq DGr(φ)⊆D is part of every selector statement.
  • Universally measurable means NullMeasurableSet E p for every probability measure p.
  • "Analytically measurable" refers to the σ-algebra generated by analytic sets. Replacing it by the power set, dropping the graph condition, or dropping the −1/ε-1/\varepsilon−1/ε case would make the selection theorems a consequence of the axiom of choice. The statements rule all three out.

Not included: Lusin's theorem in Suslin-scheme form (Proposition 7.42, which needs the Suslin operation as a definition), Proposition 7.43 on P(X)P(X)P(X), the integration results of Propositions 7.46 and 7.48, and Lemma 7.30(2)–(4). None is used in the proof of the goal. Contributions welcome: the bridge between IsBorelSpace and Mathlib's StandardBorelSpace, the universal σ-algebra API, and the Jankov–von Neumann theorem itself.

Selected references

  • D. P. Bertsekas and S. E. Shreve, Stochastic Optimal Control: The Discrete-Time Case, Academic Press 1978; Athena Scientific 1996, §7.6–7.7. https://web.mit.edu/dimitrib/www/soc.html
  • D. Blackwell, D. Freedman and M. Orkin, The optimal reward operator in dynamic programming, Annals of Probability 2 (1974) 926–941. https://doi.org/10.1214/aop/1176996558
  • A. S. Kechris, Classical Descriptive Set Theory, Graduate Texts in Mathematics 156, Springer 1995, §18 (Jankov–von Neumann uniformization), §29 (measurability of analytic sets). https://doi.org/10.1007/978-1-4612-4190-4
  • S. E. Shreve and D. P. Bertsekas, Universally measurable policies in dynamic programming, Mathematics of Operations Research 4 (1979) 15–30. https://doi.org/10.1287/moor.4.1.15
8 thms3 active usersReviewed
🏆Completed
Optimization·Captain: mikedeng1

Supply Chain Coordination with Revenue-Sharing Contracts: Strengths and Limitations 1: Revenue Sharing at w = φc Coordinates the Channel and Gives the Retailer the Share φ of Its Optimal ProfitResearch Paper

Why revenue sharing

A supplier who sells to an independent retailer through a plain per-unit wholesale price faces double marginalization: the retailer orders less than the quantity that maximizes the profit of the supply chain as a whole, because each unit costs him the wholesale price rather than the production cost. Supply chain contracting studies payment schemes under which the retailer's own optimum coincides with the system optimum. Such a scheme is said to coordinate the channel. The usual examples are buy-back contracts (Pasternack, 1985), quantity-flexibility contracts (Tsay and Lovejoy, 1999) and quantity discounts (Jeuland and Shugan, 1983; Moorthy, 1987).

Cachon and Lariviere study revenue sharing, in which the retailer pays a low wholesale price and also hands over a fixed fraction of his revenue. The scheme was common in video-cassette rental in the late 1990s, where it let rental chains stock far more copies of new releases. This mission formalizes the paper's single-retailer result: revenue sharing coordinates the channel, and the supplier can choose any split of the channel's maximal profit. It also includes the three further results of the paper that use the same argument.

The source is the authors' working paper of June 2000. Its results are unnumbered, so every item cites a section, a displayed equation and a printed page. The 2005 Management Science version renumbers and revises the material.

Setting

A supplier sells to one retailer, who orders q≥0q \ge 0q≥0 units before a selling season. The retailer's expected revenue is a function R(q)R(q)R(q) of the quantity alone. Leftover units have zero salvage value, and the supplier produces each unit at cost c>0c > 0c>0. The paper's standing assumptions (Sec. 1, p. 5) are:

  • RRR is strictly concave and differentiable for q≥0q \ge 0q≥0, with marginal revenue R′(q)R'(q)R′(q);
  • the product is viable: R′(0)>cR'(0) > cR′(0)>c;
  • a finite quantity is optimal: R′(∞)<cR'(\infty) < cR′(∞)<c.

A revenue-sharing contract {ϕ,w}\{\phi, w\}{ϕ,w} has two terms. The retailer pays the wholesale price w≥0w \ge 0w≥0 per unit, and he keeps the share ϕ\phiϕ of the revenue and transfers (1−ϕ)R(q)(1-\phi)R(q)(1−ϕ)R(q) to the supplier. The case ϕ=1\phi = 1ϕ=1 is the plain wholesale-price contract. The profits of the supply chain, the retailer and the supplier are

Π(q)=R(q)−qc,πr(q)=ϕR(q)−qw,πs(q)=(1−ϕ)R(q)+qw−qc.\Pi(q) = R(q) - qc,\qquad \pi_r(q) = \phi R(q) - qw,\qquad \pi_s(q) = (1-\phi)R(q) + qw - qc .Π(q)=R(q)−qc,πr​(q)=ϕR(q)−qw,πs​(q)=(1−ϕ)R(q)+qw−qc.

The integrated channel quantity qIq_IqI​ is the maximizer of Π\PiΠ over q≥0q \ge 0q≥0. In Lean these objects are RevShareCoord.Single.Model (fields R, R', c and the three assumptions) and its functions Pi, retailerProfit and supplierProfit.

Formalization targets

Goal: revenue sharing coordinates the channel (Sec. 2.2, p. 6)

Let ϕ∈(0,1]\phi \in (0,1]ϕ∈(0,1] and w(ϕ)=ϕcw(\phi) = \phi cw(ϕ)=ϕc. Then

qI=arg max⁡q≥0 πr(q) (uniquely),w(ϕ)≤c,πr(qI)=ϕ Π(qI),πs(qI)=(1−ϕ) Π(qI).q_I = \operatorname*{arg\,max}_{q\ge 0}\ \pi_r(q) \ \text{(uniquely)},\qquad w(\phi)\le c,\qquad \pi_r(q_I) = \phi\,\Pi(q_I),\qquad \pi_s(q_I) = (1-\phi)\,\Pi(q_I).qI​=q≥0argmax​ πr​(q) (uniquely),w(ϕ)≤c,πr​(qI​)=ϕΠ(qI​),πs​(qI​)=(1−ϕ)Π(qI​).

The statement fixes no revenue function and no share. It holds for every model and every ϕ∈(0,1]\phi \in (0, 1]ϕ∈(0,1], which is what "the supplier can take any share of the channel profit" means.

Milestones on the way

  1. Eq. (1), p. 6. qIq_IqI​ exists, is unique and positive, and is the only positive root of R′(qI)=cR'(q_I) = cR′(qI​)=c.
  2. Retailer's first-order condition, p. 6. If R′(0)>w/ϕR'(0) > w/\phiR′(0)>w/ϕ, an order q^≥0\hat q \ge 0q^​≥0 is optimal for the retailer exactly when q^>0\hat q > 0q^​>0 and ϕR′(q^)=w\phi R'(\hat q) = wϕR′(q^​)=w. The retailer has at most one optimal order.
  3. Profit identities, p. 6. Under {ϕ,ϕc}\{\phi, \phi c\}{ϕ,ϕc}, πr(q)=ϕΠ(q)\pi_r(q) = \phi\Pi(q)πr​(q)=ϕΠ(q) and πs(q)=(1−ϕ)Π(q)\pi_s(q) = (1-\phi)\Pi(q)πs​(q)=(1−ϕ)Π(q) at every qqq.
  4. Heterogeneous retailers, p. 7. Given ccc and ϕ\phiϕ, a single wholesale price, chosen before the revenue function, coordinates every retailer of the model.

Further results on the same argument

  1. Buy-back equivalence, Sec. 2.3, p. 9. Take the fixed-price newsvendor and the buy-back contract b∗=p(1−ϕ)b^* = p(1-\phi)b∗=p(1−ϕ), wb∗=p(1−ϕ)+ϕcw_b^* = p(1-\phi)+\phi cwb∗​=p(1−ϕ)+ϕc. It gives the retailer and the supplier the same realized profits as {ϕ,ϕc}\{\phi, \phi c\}{ϕ,ϕc}, for every order and every demand realization.
  2. Endogenous price, Sec. 3.1 and footnote 3, p. 11. Let revenue Rev(q,p)\mathrm{Rev}(q,p)Rev(q,p) be any function of quantity and price, with costs linear in quantity. Then πr(q,p)=ϕ Π(q,p)\pi_r(q,p) = \phi\,\Pi(q,p)πr​(q,p)=ϕΠ(q,p) under {ϕ,ϕc}\{\phi,\phi c\}{ϕ,ϕc}, and the integrated optimum (qI,pI)(q_I,p_I)(qI​,pI​), assumed unique, is the retailer's unique optimum.

Significance

The result separates coordination from profit division. A contract family coordinates for every value of a parameter, and that parameter then moves profit between the firms without changing the quantity, so the contract terms can be settled by bargaining power alone. The heterogeneous-retailer milestone gives the practical advantage over quantity discounts: the coordinating terms do not depend on the retailer's demand, so one price list serves retailers who face different markets. The Sec. 2.3 equivalence shows that, in the fixed-price newsvendor, buy-backs are a special case of revenue sharing. The Sec. 3.1 statement shows that revenue sharing still coordinates when the retailer also sets the price, a setting in which Emmons and Gilbert (1998) showed buy-backs fail.

All of these results are proved in the paper, and none is open. The mission adds a machine-checked version of the single-retailer theory for a general strictly concave revenue function. A related newsvendor version is already formalized on the platform: SupplyChainTheory.revenue_sharing_coordinates, from Snyder and Shen, Fundamentals of Supply Chain Theory, Thm 14.6. That version has a newsvendor revenue with salvage values and goodwill costs, and it concludes the optimality of three profits, not the ϕ\phiϕ-split of this paper. It is a different statement, so it is not reused here.

Difficulty

The algebra is short. The identity πr=ϕΠ\pi_r = \phi\Piπr​=ϕΠ under w=ϕcw = \phi cw=ϕc is a single line, and it is a milestone, not the goal. The work lies in the optimization claims over a half-line with only one-sided information at 000. The integrated optimum must be shown to exist. R′(∞)<cR'(\infty) < cR′(∞)<c gives only an eventual bound on the derivative, so the existence argument needs the continuity of a concave function and its supergradient inequality. It must also be shown positive, which uses R′(0)>cR'(0) > cR′(0)>c as a one-sided derivative. Its uniqueness rests on strict concavity. The retailer's first-order condition needs the same machinery for ϕR−wq\phi R - wqϕR−wq, including the observation that the boundary point 000 is never optimal. A stationary point of πr\pi_rπr​ is not enough. The goal asserts that qIq_IqI​ is the unique maximizer over all of [0,∞)[0,\infty)[0,∞).

Formalization scope

  • Quantities, prices and shares are real numbers. RRR and R′R'R′ are functions R→R\mathbb R \to \mathbb RR→R, constrained only on [0,∞)[0,\infty)[0,∞). Differentiability is HasDerivWithinAt R (R' q) (Set.Ici 0) q for q≥0q \ge 0q≥0, so it is one-sided at 000. Strict concavity is StrictConcaveOn ℝ (Set.Ici 0) R.
  • R′(∞)<cR'(\infty) < cR′(∞)<c is encoded as "R′(Q)<cR'(Q) < cR′(Q)<c for some Q≥0Q \ge 0Q≥0". For a decreasing R′R'R′ this is equivalent, and it allows R′→−∞R' \to -\inftyR′→−∞.
  • "Optimal" means IsMaxOn over [0,∞)[0,\infty)[0,∞) (over [0,∞)×P[0,\infty)\times P[0,∞)×P in Sec. 3.1), and "unique" means every other maximizer equals it.
  • The supplier's profit πs\pi_sπs​ is not displayed in the paper. It is read off the sequence of events of Sec. 1.
  • The goal and the first-order condition take ϕ∈(0,1]\phi \in (0,1]ϕ∈(0,1]. At ϕ=0\phi = 0ϕ=0 the retailer's profit is identically zero and qIq_IqI​ is not the unique optimum. The profit identities and the buy-back identities hold for all real parameters and are stated that way.
  • Corrected slips. (a) Eq. (1) is introduced with "R′(0)≥cR'(0) \ge cR′(0)≥c". This contradicts the standing assumption R′(0)>cR'(0) > cR′(0)>c: with equality, qI=0q_I = 0qI​=0 is not positive. The statement uses R′(0)>cR'(0) > cR′(0)>c. (b) The display πr(qI)=ϕR(qI)−qIc=ϕΠ(qI)\pi_r(q_I) = \phi R(q_I) - q_I c = \phi\Pi(q_I)πr​(qI​)=ϕR(qI​)−qI​c=ϕΠ(qI​) has a wrong middle term, which should read ϕR(qI)−qIϕc\phi R(q_I) - q_I\phi cϕR(qI​)−qI​ϕc. The outer equality is stated.
  • The first-order-condition milestone adds the converse direction and uniqueness to the paper's "must satisfy". It does not claim that an optimum exists, which may fail when w/ϕ<cw/\phi < cw/ϕ<c.
  • Sec. 3.1 is stated in the generality of footnote 3: an arbitrary revenue function Rev(q,p)\mathrm{Rev}(q,p)Rev(q,p) and a set PPP of admissible prices, with the integrated optimum's uniqueness as a hypothesis, as the paper assumes it. The paper's monotonicity of F(x,p)F(x,p)F(x,p) in ppp is unused and omitted.
  • Sec. 2.3 is formalized pathwise. The expected-profit equations (2)–(4) are not part of the mission.
  • A goal that only asserts πr(ϕ,ϕc,q)=ϕ Π(q)\pi_r(\phi, \phi c, q) = \phi\,\Pi(q)πr​(ϕ,ϕc,q)=ϕΠ(q) would be an unfolding of definitions. The goal therefore carries the argmax-and-uniqueness claim, which needs strict concavity and the model's assumptions.
  • Needed infrastructure: first-order conditions for concave functions on a closed half-line with one-sided derivatives, and existence of maximizers from an eventual derivative bound. Both are reusable beyond this mission. Contributions of that general kind are welcome.

Selected references

  • G. P. Cachon, M. A. Lariviere, Supply Chain Coordination with Revenue-Sharing Contracts: Strengths and Limitations, working paper, June 2000. Published version: Management Science 51(1):30–44, 2005. https://doi.org/10.1287/mnsc.1040.0215
  • B. A. Pasternack, Optimal pricing and return policies for perishable commodities, Marketing Science 4(2):166–176, 1985. https://doi.org/10.1287/mksc.4.2.166
  • K. S. Moorthy, Managing channel profits: Comment, Marketing Science 6(4):375–379, 1987. https://doi.org/10.1287/mksc.6.4.375
  • A. A. Tsay, W. S. Lovejoy, Quantity flexibility contracts and supply chain performance, Manufacturing & Service Operations Management 1(2):89–111, 1999. https://doi.org/10.1287/msom.1.2.89
  • H. Emmons, S. M. Gilbert, Note: The role of returns policies in pricing and inventory decisions for catalogue goods, Management Science 44(2):276–283, 1998. https://doi.org/10.1287/mnsc.44.2.276
  • L. V. Snyder, Z.-J. M. Shen, Fundamentals of Supply Chain Theory, 2nd ed., Wiley, 2019, Ch. 14. https://doi.org/10.1002/9781119584445
10 thms3 active usersReviewed
🏆Completed
AnalysisDynamic ProgrammingOptimization+2·Captain: mikedeng1

Stochastic Optimal Control: The Discrete-Time Case V: Semicontinuous Functions — a Borel-Measurable Minimizing Selector for Lower Semicontinuous CostsTextbook

Motivation

Every step of the dynamic programming algorithm on a general state space does three things: it takes a conditional expectation of the cost-to-go under a transition kernel, it minimizes the resulting function of state and control over the control, and, if a policy is to be produced, it picks a control for each state that attains or nearly attains that minimum. On a finite or countable state space all three are harmless. On an uncountable state space each can destroy the measurability needed to take the next expectation: the infimum over an uncountable family of measurable functions need not be measurable, and a minimizer chosen state by state need not be a measurable function of the state, so it does not define a policy at all.

Section 7.5 of Bertsekas and Shreve, Stochastic Optimal Control: The Discrete-Time Case (1978; Athena Scientific reprint 1996), settles the three operations for semicontinuous costs and continuous kernels. The results are the topological half of the book's measurability theory; the descriptive set theory half (lower semianalytic functions and analytically measurable selectors, §7.6–7.7) is a separate mission in this series. The semicontinuous results are what Propositions 8.6–8.7 and Corollaries 9.17.2–9.17.3 of the book use to obtain Borel-measurable optimal policies for finite-horizon and infinite-horizon models with lower semicontinuous costs and compact control sets.

Timeline. The exact selection theorem for lower semicontinuous functions (Proposition 7.33 below) is credited by the book's notes to Dubins and Savage, How to Gamble If You Must (1965). The Hausdorff metric on closed sets goes back to Hausdorff's Set Theory. Measurable selection in the closed-valued setting was later systematized by Kuratowski and Ryll-Nardzewski (1965), whose theorem gives a different route to results of this kind.

Setting

Throughout, R∗=[−∞,+∞]R^*=[-\infty,+\infty]R∗=[−∞,+∞] is the extended real line. A function f:X→R∗f:X\to R^*f:X→R∗ on a metrizable space XXX is lower semicontinuous if every sublevel set {x∣f(x)≤c}\{x\mid f(x)\le c\}{x∣f(x)≤c}, c∈Rc\in\mathbb Rc∈R, is closed, and upper semicontinuous if every superlevel set {x∣f(x)≥c}\{x\mid f(x)\ge c\}{x∣f(x)≥c} is closed (Definition 7.13). C(X)C(X)C(X) is the space of bounded continuous real-valued functions on XXX.

For a separable metrizable space YYY, P(Y)P(Y)P(Y) is the set of Borel probability measures on YYY with the weak topology (convergence of integrals of functions in C(Y)C(Y)C(Y)). A stochastic kernel q(dy∣x)q(dy\mid x)q(dy∣x) on YYY given XXX is a map x↦q(dy∣x)x\mapsto q(dy\mid x)x↦q(dy∣x) from XXX to P(Y)P(Y)P(Y), and it is continuous if this map is continuous (Definition 7.12). The integral of a Borel-measurable f:Y→R∗f:Y\to R^*f:Y→R∗ is ∫f dp=∫f+dp−∫f−dp\int f\,dp=\int f^+dp-\int f^-dp∫fdp=∫f+dp−∫f−dp with the convention −∞+∞=+∞−∞=+∞-\infty+\infty=+\infty-\infty=+\infty−∞+∞=+∞−∞=+∞ (Eq. (43) of Chapter 7).

For a compact metric space YYY, 2Y2^Y2Y is the collection of closed subsets of YYY with the topology of the Hausdorff metric (Appendix C). For D⊆X×YD\subseteq X\times YD⊆X×Y, the section at xxx is Dx={y∣(x,y)∈D}D_x=\{y\mid (x,y)\in D\}Dx​={y∣(x,y)∈D}, the projection is projX(D)={x∣Dx≠∅}\mathrm{proj}_X(D)=\{x\mid D_x\neq\emptyset\}projX​(D)={x∣Dx​=∅}, and a function φ:projX(D)→Y\varphi:\mathrm{proj}_X(D)\to Yφ:projX​(D)→Y has its graph in DDD if (x,φ(x))∈D(x,\varphi(x))\in D(x,φ(x))∈D for every x∈projX(D)x\in\mathrm{proj}_X(D)x∈projX​(D). "Borel-measurable" refers to the Borel σ-algebras of the topologies in question; on projX(D)\mathrm{proj}_X(D)projX​(D) this is the Borel σ-algebra of the subspace topology.

Formalization targets

Goal: Proposition 7.33

Let XXX be metrizable, YYY compact metrizable, D⊆X×YD\subseteq X\times YD⊆X×Y closed, and f:D→R∗f:D\to R^*f:D→R∗ lower semicontinuous. Put

f∗(x)=min⁡y∈Dxf(x,y),x∈projX(D).f^*(x)=\min_{y\in D_x}f(x,y),\qquad x\in\mathrm{proj}_X(D).f∗(x)=y∈Dx​min​f(x,y),x∈projX​(D).

Then projX(D)\mathrm{proj}_X(D)projX​(D) is closed, f∗f^*f∗ is lower semicontinuous, and there is a Borel-measurable φ:projX(D)→Y\varphi:\mathrm{proj}_X(D)\to Yφ:projX​(D)→Y with graph in DDD and

f(x,φ(x))=f∗(x)∀x∈projX(D).f\bigl(x,\varphi(x)\bigr)=f^*(x)\qquad\forall x\in\mathrm{proj}_X(D).f(x,φ(x))=f∗(x)∀x∈projX​(D).

Milestones

  • Proposition 7.32: for f∗(x)=inf⁡y∈Yf(x,y)f^*(x)=\inf_{y\in Y}f(x,y)f∗(x)=infy∈Y​f(x,y), lower semicontinuity of fff and compactness of YYY give lower semicontinuity of f∗f^*f∗ and attainment; upper semicontinuity of fff gives upper semicontinuity of f∗f^*f∗.
  • Lemma 7.18: there is a Borel-measurable σ:2Y−{∅}→Y\sigma:2^Y-\{\emptyset\}\to Yσ:2Y−{∅}→Y with σ(A)∈A\sigma(A)\in Aσ(A)∈A.
  • Lemma 7.20: for lower semicontinuous fff on a nonempty compact YYY, the argmin map x↦{y∣f(x,y)≤f∗(x)}x\mapsto\{y\mid f(x,y)\le f^*(x)\}x↦{y∣f(x,y)≤f∗(x)} is Borel-measurable into 2Y2^Y2Y.
  • Lemma 7.14: fff is lower semicontinuous and bounded below iff fn↑ff_n\uparrow ffn​↑f for some fn∈C(X)f_n\in C(X)fn​∈C(X) (and dually).
  • Proposition 7.30: x↦∫f(x,y) q(dy∣x)x\mapsto\int f(x,y)\,q(dy\mid x)x↦∫f(x,y)q(dy∣x) is continuous for f∈C(X×Y)f\in C(X\times Y)f∈C(X×Y) and continuous qqq.
  • Proposition 7.31: the same map is lower (upper) semicontinuous and bounded below (above) when fff is.
  • Lemma 7.21: an open G⊆X×YG\subseteq X\times YG⊆X×Y, YYY separable, has open projection and a Borel-measurable selector with graph in GGG.
  • Proposition 7.34: for open DDD and upper semicontinuous fff, projX(D)\mathrm{proj}_X(D)projX​(D) is open, f∗=inf⁡Dxff^*=\inf_{D_x}ff∗=infDx​​f is upper semicontinuous, and for each ε>0\varepsilon>0ε>0 there is a Borel-measurable φε\varphi_\varepsilonφε​ with graph in DDD and
f(x,φε(x))≤{f∗(x)+εif f∗(x)>−∞,−1/εif f∗(x)=−∞.f\bigl(x,\varphi_\varepsilon(x)\bigr)\le\begin{cases}f^*(x)+\varepsilon&\text{if }f^*(x)>-\infty,\\-1/\varepsilon&\text{if }f^*(x)=-\infty.\end{cases}f(x,φε​(x))≤{f∗(x)+ε−1/ε​if f∗(x)>−∞,if f∗(x)=−∞.​

Significance

The results. Propositions 7.31–7.33 are the closure properties that make the dynamic programming recursion stay inside the class of lower semicontinuous functions bounded below: the expectation step preserves the class (7.31), the minimization step preserves it (7.32, 7.33), and the minimization admits a Borel-measurable exact minimizer (7.33). This is why, in semicontinuous models, the optimal cost functions are lower semicontinuous and optimal policies can be taken Borel-measurable and nonrandomized. Proposition 7.34 gives the weaker, ε\varepsilonε-optimal counterpart for upper semicontinuous costs, where the infimum need not be attained.

Formalizing them. All of these results are proved in the book; none is open. As far as is known, none has a machine-checked proof: Mathlib has semicontinuity, the Hausdorff extended metric on closed and on nonempty compact sets, and the weak topology on probability measures, but no theorem combining them into a measurable selection result of this kind. A formal development would supply measurable selectors for semicontinuous minimization in Lean and the Borel-measurability of set-valued maps into the hyperspace of closed sets, both reusable well beyond dynamic programming.

Difficulty

The obvious attempt at the goal is to pick, for each xxx, some minimizer yyy of f(x,⋅)f(x,\cdot)f(x,⋅) over the compact section DxD_xDx​. The minimizer exists by compactness and lower semicontinuity, but the choice is made pointwise and gives no control on measurability: a minimizer chosen by the axiom of choice need not be Borel-measurable. The argmin sets F∗(x)F^*(x)F∗(x) vary with xxx only semicontinuously: they can jump from a single point to a large set, so a continuous selection generally does not exist, and continuity arguments cannot replace measurability. Lemma 7.18 isolates the hardest part: a choice of a point of each nonempty closed set that is measurable as a function of the set itself.

A second difficulty is bookkeeping at infinity. Values ±∞\pm\infty±∞ are allowed throughout, so sublevel sets, minima, integrals and ε\varepsilonε-bounds must all be handled in R∗R^*R∗; the integral in Proposition 7.31 uses the convention ∞−∞=+∞\infty-\infty=+\infty∞−∞=+∞, which is not Mathlib's.

Formalization scope

  • Extended reals. Values are in EReal. The only place where values of opposite infinite sign are combined is the integral, which is the published definition DupacovaWets.Consistency.expect (reused, not restated): ∫f+−∫f−\int f^+-\int f^-∫f+−∫f− with an explicit case returning +∞+\infty+∞ when ∫f+=∞\int f^+=\infty∫f+=∞, exactly the book's convention (42). The ε\varepsilonε-bound of Proposition 7.34 adds a real ε\varepsilonε to a value different from −∞-\infty−∞, which is safe in EReal.
  • Semicontinuity is Mathlib's LowerSemicontinuous/UpperSemicontinuous, equivalent to Definition 7.13 for EReal-valued functions. Lemma 7.13 of the book (the sequential characterization) is Mathlib's lowerSemicontinuous_iff_le_liminf together with first countability of metrizable spaces, and is not restated here.
  • Functions on DDD. Functions "on DDD" are functions on X×YX\times YX×Y with LowerSemicontinuousOn f D (resp. UpperSemicontinuousOn); values off DDD play no role. projX(D)\mathrm{proj}_X(D)projX​(D) is Prod.fst '' D, selectors are functions on that subtype, and its σ-algebra is the Borel σ-algebra of the subspace topology.
  • Hyperspace. 2Y2^Y2Y is Closeds Y, and 2Y−{∅}2^Y-\{\emptyset\}2Y−{∅} for compact YYY is NonemptyCompacts Y, each with the Hausdorff extended metric and the Borel σ-algebra of its topology. This topology agrees with the book's (the exponential topology of Appendix C, independent of the metric).
  • Boundedness. "Bounded below/above" is by a real constant. BddBelow in EReal would be vacuous and is not used.
  • Edge cases. Proposition 7.32(a)'s attainment clause is stated for nonempty YYY, since for Y=∅Y=\emptysetY=∅ the infimum is +∞+\infty+∞ and nothing attains it.
  • Argmin minimum. Lemma 7.20 assumes nonempty YYY because its defining formula uses a minimum; for empty YYY there is no minimizer.
  • Ruling out trivial readings. The graph condition (x,φ(x))∈D(x,\varphi(x))\in D(x,φ(x))∈D is part of every selection statement; without it the goal would follow from the unconstrained case. The selector must be Borel-measurable on projX(D)\mathrm{proj}_X(D)projX​(D) and must attain the minimum exactly, not up to ε\varepsilonε.

A complete development needs the Borel structure of the hyperspace (measurability of maps into Closeds Y from upper semicontinuity in the sense of Kuratowski, Proposition C.4 of the book), the construction of a measurable choice function on NonemptyCompacts Y, and approximation of semicontinuous functions by monotone sequences in C(X)C(X)C(X). Each of these is reusable on its own; proofs of individual milestones by any route are welcome.

Selected references

  • D. P. Bertsekas and S. E. Shreve, Stochastic Optimal Control: The Discrete-Time Case, Academic Press, 1978; Athena Scientific reprint, 1996, Section 7.5 and Appendix C. https://web.mit.edu/dimitrib/www/soc.html
  • L. E. Dubins and L. J. Savage, How to Gamble If You Must: Inequalities for Stochastic Processes, McGraw-Hill, 1965.
  • K. Kuratowski and C. Ryll-Nardzewski, "A general theorem on selectors," Bull. Acad. Polon. Sci. 13 (1965), 397–403.
  • F. Hausdorff, Set Theory, Chelsea, New York, 1957.
10 thms3 active usersReviewed
🏆Completed
Algorithmic Game Theory·Captain: mikedeng1

The Price of Stability for Network Design with Fair Cost Allocation I: The Price of Stability of Fair Cost Sharing Is at Most H(k), and This Is TightResearch Paper

Motivation

Many networks are built and paid for by the users they serve: multicast trees, virtual overlays, shared subnetworks of the Internet. A protocol proposes a design and a rule for splitting its cost, and each participant is free to accept the proposal or defect to a cheaper alternative. The designer therefore cannot impose the global optimum; the best it can do is propose the cheapest outcome that no participant wants to leave, a Nash equilibrium. The ratio between the cost of the best equilibrium and the optimal cost, the price of stability, measures the loss caused by requiring stability. It contrasts with the price of anarchy, which compares the worst equilibrium with the optimum and suits settings with no coordinating protocol at all.

Anshelevich, Dasgupta, Kleinberg, Tardos, Wexler and Roughgarden (SIAM J. Comput. 2008; preliminary version FOCS 2004) studied this question for the most common cost-sharing rule, the Shapley (equal-split) rule, under which every user of an edge pays the same share of its cost. Their first result, the subject of this mission, is that the price of stability is at most the harmonic number H(k)H(k)H(k), where kkk is the number of players, and that this bound is attained in the limit.

Setting

There are kkk players and a finite set EEE of edges. Each player iii has a family Σi\Sigma_iΣi​ of feasible strategies, each a set of edges; in the network setting Σi\Sigma_iΣi​ consists of the edge sets connecting player iii's terminals in a directed graph. Each edge eee has a nonnegative cost cec_ece​. In a strategy vector S=(S1,…,Sk)S=(S_1,\dots,S_k)S=(S1​,…,Sk​), Si∈ΣiS_i\in\Sigma_iSi​∈Σi​, let xex_exe​ be the number of players whose strategy contains eee. Under Shapley cost sharing player iii pays

Ci(S)=∑e∈Sicexe.C_i(S)=\sum_{e\in S_i}\frac{c_e}{x_e}.Ci​(S)=e∈Si​∑​xe​ce​​.

The cost of the designed network is

cost⁡(S)=∑e∈⋃iSice,\operatorname{cost}(S)=\sum_{e\in\bigcup_i S_i}c_e ,cost(S)=e∈⋃i​Si​∑​ce​,

and the payments add up to exactly this amount. The profile SSS is a (pure) Nash equilibrium if no player iii has a strategy Si′∈ΣiS_i'\in\Sigma_iSi′​∈Σi​ with Ci(S−i,Si′)<Ci(S)C_i(S_{-i},S_i')<C_i(S)Ci​(S−i​,Si′​)<Ci​(S). Finally,

H(k)=1+12+⋯+1k.H(k)=1+\tfrac12+\dots+\tfrac1k .H(k)=1+21​+⋯+k1​.

The game is a congestion game: the per-user cost of an edge, fe(x)=ce/xf_e(x)=c_e/xfe​(x)=ce​/x, depends only on the edge and its number of users. Rosenthal's potential

Φ(S)=∑e∈E∑x=1xefe(x)\Phi(S)=\sum_{e\in E}\sum_{x=1}^{x_e}f_e(x)Φ(S)=e∈E∑​x=1∑xe​​fe​(x)

changes by exactly the deviating player's change in cost when a single player changes strategy. The same objects are used with load-dependent edge costs ce(x)c_e(x)ce​(x), in which case fe(x)=ce(x)/xf_e(x)=c_e(x)/xfe​(x)=ce​(x)/x.

Formalization targets

Goal: Theorem 2.1 and its tightness

For every game with ce≥0c_e\ge0ce​≥0 in which each player has a feasible strategy there is a Nash equilibrium SSS with

cost⁡(S)≤H(k)⋅cost⁡(P)for every profile P,\operatorname{cost}(S)\le H(k)\cdot\operatorname{cost}(P)\quad\text{for every profile }P,cost(S)≤H(k)⋅cost(P)for every profile P,

and for every k≥1k\ge1k≥1, ε>0\varepsilon>0ε>0 the instance of Fig. 1.1 (player iii has its own path of cost 1/i1/i1/i, and all players can share a path of cost 1+ε1+\varepsilon1+ε) has a Nash equilibrium, every Nash equilibrium of it costs H(k)H(k)H(k), and some profile costs 1+ε1+\varepsilon1+ε. The ratio H(k)/(1+ε)H(k)/(1+\varepsilon)H(k)/(1+ε) tends to H(k)H(k)H(k) as ε→0\varepsilon\to0ε→0.

Milestones

  1. Budget balance: ∑iCi(S)=cost⁡(S)\sum_iC_i(S)=\operatorname{cost}(S)∑i​Ci​(S)=cost(S) for every SSS (Sect. 1).
  2. Rosenthal's potential is exact (Theorem 2.1, proof, (2.1)).
  3. From every profile some Nash equilibrium of no larger potential is reached (Theorem 2.1, proof).
  4. Theorem 3.1: if cost⁡(S)≤A Φ(S)\operatorname{cost}(S)\le A\,\Phi(S)cost(S)≤AΦ(S) and Φ(S)≤Bcost⁡(S)\Phi(S)\le B\operatorname{cost}(S)Φ(S)≤Bcost(S) for all SSS, the price of stability is at most ABABAB.
  5. For nondecreasing concave edge costs ce(x)c_e(x)ce​(x): cost⁡(S)≤Φ(S)≤H(k)cost⁡(S)\operatorname{cost}(S)\le\Phi(S)\le H(k)\operatorname{cost}(S)cost(S)≤Φ(S)≤H(k)cost(S) (Theorem 2.3, proof).
  6. Theorem 2.3: the H(k)H(k)H(k) bound for nondecreasing concave edge costs.
  7. The Fig. 1.1 instance: its Nash equilibrium is unique and costs H(k)H(k)H(k); a profile costs 1+ε1+\varepsilon1+ε.

Significance

The bound shows that requiring stability under the Shapley rule costs at most a logarithmic factor, H(k)=Θ(log⁡k)H(k)=\Theta(\log k)H(k)=Θ(logk), whereas the price of anarchy of the same game is kkk (two parallel edges of costs 111 and kkk already show this). It was among the first price-of-stability results and is the starting point for a line of work on network design games: the undirected case, where the H(k)H(k)H(k) bound is not tight and the correct value remained open for years, weighted players, and other cost-sharing rules. The argument (an exact potential that over- and under-estimates the social cost by bounded factors) is the standard tool for price-of-stability bounds, and Theorem 3.1 isolates it in a reusable form.

The theorem is proved in the paper; to our knowledge no machine-checked proof exists. This mission produces a formal account of Shapley cost-sharing games as congestion games, of Rosenthal's potential and the finite improvement property, of the potential-sandwich argument of Theorem 3.1, and of the matching lower-bound instance, for constant and for nondecreasing concave edge costs.

Difficulty

The obvious approach, bounding the cost of an arbitrary equilibrium, fails: some equilibria cost kkk times the optimum, so any proof must select a particular equilibrium. The selection uses the finiteness of the strategy space together with the exact potential, and the bound needs the inequality Φ≤H(k)⋅cost⁡\Phi\le H(k)\cdot\operatorname{cost}Φ≤H(k)⋅cost, which for concave costs requires the per-user cost ce(x)/xc_e(x)/xce​(x)/x to be nonincreasing. On the lower-bound side, the claim that every equilibrium of Fig. 1.1 costs H(k)H(k)H(k) requires excluding all equilibria in which some players share the common path, for every kkk at once, not only checking that the all-own profile is stable.

Formalization scope

Games are encoded as congestion games over arbitrary finite families of edge sets, reusing the published CongestionPoA.AsymSum.Model (congestion game, loads, player costs, cost-form pure Nash equilibrium, total cost). The paper notes that its proofs do not use the graph structure; the directed-graph game is the instance in which Σi\Sigma_iΣi​ is the family of edge sets connecting player iii's terminals. Players and edges form finite types; kkk is the number of players and H(k)H(k)H(k) is Mathlib's harmonic k cast to R\mathbb RR. The price of stability is stated as the existence of a Nash equilibrium whose cost is at most the constant times the cost of every profile, with no division by the optimum, and with the hypothesis that some profile exists. Concave costs are functions on N\mathbb NN with nonincreasing increments, nondecreasing, with ce(0)≥0c_e(0)\ge0ce​(0)≥0; this last condition is implicit in the paper and needed for ce(x)/xc_e(x)/xce​(x)/x to be nonincreasing. Theorem 3.1 carries the implicit hypothesis A≥0A\ge0A≥0.

A statement that bounds every equilibrium is false, and one that asserts a cheap profile without the Nash condition is trivial; both are excluded, and the tightness part quantifies over every Nash equilibrium of the instance and asserts that one exists.

Needed infrastructure: finite improvement paths in potential games, sum manipulations over loads, and bounds on harmonic sums. The potential lemmas apply to every finite congestion game and are reusable beyond this mission. Proofs of any milestone are welcome independently.

Selected references

  • E. Anshelevich, A. Dasgupta, J. Kleinberg, É. Tardos, T. Wexler, T. Roughgarden, The Price of Stability for Network Design with Fair Cost Allocation, SIAM J. Comput. 38(4):1602–1623, 2008. https://doi.org/10.1137/070680096
  • R. W. Rosenthal, A class of games possessing pure-strategy Nash equilibria, Int. J. Game Theory 2:65–67, 1973. https://doi.org/10.1007/BF01737559
  • D. Monderer, L. S. Shapley, Potential games, Games Econ. Behav. 14:124–143, 1996. https://doi.org/10.1006/game.1996.0044
11 thms3 active usersReviewed
🏆Completed
Algorithmic Game TheoryComplexity Theory·Captain: mikedeng1

The Complexity of Computing a Nash Equilibrium 2: The Three-Colorable Addition-Multiplication Gadget Has Error Amplification 81Research Paper

Motivation

Nash's theorem guarantees that every finite game has a mixed equilibrium, but says nothing about how hard one is to find. Daskalakis, Goldberg and Papadimitriou (SIAM J. Comput. 39(1), 2009) showed that computing a Nash equilibrium is complete for the class PPAD of total search problems whose solutions are guaranteed by a parity argument on a directed graph (Papadimitriou 1994). Their proof simulates an arithmetic circuit by a graphical game: a multiplayer game in which each player's payoff depends on a few neighbours only, built from small game gadgets whose equilibria compute x↦αxx \mapsto \alpha xx↦αx, x+yx + yx+y, xyxyxy, constants and comparisons on mixed-strategy probabilities.

These gadgets are the reusable core of the reduction. The PPAD-completeness results for three-player games, and the later results for two-player games and for approximate equilibria, all rest on gadget analyses of this kind.

Timeline.

  • 1994: Papadimitriou defines PPAD and shows that Nash equilibrium computation lies in it (JCSS 48).
  • 2001: Kearns, Littman and Singh introduce graphical games (UAI 2001).
  • 2005–2006: Goldberg and Papadimitriou reduce rrr-player games to four-player games, and Daskalakis, Goldberg and Papadimitriou prove PPAD-completeness for four players (STOC 2006). The three-player case is obtained independently by Daskalakis and Papadimitriou and by Chen and Deng (ECCC reports, 2005); the two-player case by Chen and Deng (FOCS 2006; J. ACM 56(3), 2009, with Teng).
  • 2009: the journal version unifies the three-player argument; its Proposition 4.18 is the gadget G+,∗\mathcal G_{+,*}G+,∗​ that makes the reduction to three players work.

Setting

A game has finitely many players; player ppp has a finite set SpS_pSp​ of pure strategies and a nonnegative payoff uspu^p_susp​ for each pure profile sss. A mixed profile xxx gives each player a probability distribution (xjp)j∈Sp(x^p_j)_{j\in S_p}(xjp​)j∈Sp​​, and xsx_sxs​ denotes the product of the probabilities of the components of a partial profile sss. The expected payoff of a pure strategy jjj of ppp is ∑s∈S−pujspxs\sum_{s\in S_{-p}} u^p_{js}x_s∑s∈S−p​​ujsp​xs​.

An ϵ\epsilonϵ-Nash equilibrium (the paper's notion, an ϵ\epsilonϵ-approximately well-supported equilibrium, Eq. (2)) is a mixed profile in which no player puts weight on a pure strategy that another of its pure strategies beats by more than ϵ\epsilonϵ:

∀p, j,j′∈Sp:∑s∈S−pujspxs>∑s∈S−puj′spxs+ϵ  ⟹  xj′p=0.\forall p,\ j, j' \in S_p:\quad \sum_{s\in S_{-p}} u^p_{js}x_s > \sum_{s\in S_{-p}} u^p_{j's}x_s + \epsilon \implies x^p_{j'} = 0 .∀p, j,j′∈Sp​:s∈S−p​∑​ujsp​xs​>s∈S−p​∑​uj′sp​xs​+ϵ⟹xj′p​=0.

For ϵ=0\epsilon = 0ϵ=0 this characterizes Nash equilibria.

A player with strategies {0,1}\{0,1\}{0,1} is binary; p[v]\mathbf p[v]p[v] is the probability that it plays 111, and p[v:s]\mathbf p[v : s]p[v:s] is the probability that vvv plays sss. The affects graph (Definition 2.2) has an edge (v1,v2)(v_1, v_2)(v1​,v2​) between distinct players when the payoff of v2v_2v2​ is a nonconstant function of the action of v1v_1v1​. A legal kkk-coloring (Definition 4.8) gives each player one of kkk colors so that the two ends of an edge differ and two distinct players with a common successor differ.

The game G+,∗\mathcal G_{+,*}G+,∗​ (Fig. 11) has inputs v1,v2v_1, v_2v1​,v2​, output v3v_3v3​ and intermediate players w1,v1′,w2,v2′,w3,w,uw_1, v_1', w_2, v_2', w_3, w, uw1​,v1′​,w2​,v2′​,w3​,w,u, all binary except v2′v_2'v2′​, which has strategies {0,1,∗}\{0, 1, *\}{0,1,∗}. Its payoff tables are fixed by nonnegative integers α,β,γ\alpha, \beta, \gammaα,β,γ; the inputs' payoffs are unconstrained.

Formalization targets

Goal: Proposition 4.18

For nonnegative integers α,β,γ\alpha, \beta, \gammaα,β,γ with α+β+γ≤3\alpha + \beta + \gamma \le 3α+β+γ≤3, the affects graph of G+,∗\mathcal G_{+,*}G+,∗​ has a legal 3-coloring, and for every ϵ∈[0,0.01]\epsilon \in [0, 0.01]ϵ∈[0,0.01], at every ϵ\epsilonϵ-Nash equilibrium,

∣ p[v3]−min⁡{1, αp[v1]+βp[v2]+γp[v1]p[v2]}∣≤81 ϵ.\Big|\,\mathbf p[v_3] - \min\{1,\ \alpha\mathbf p[v_1] + \beta\mathbf p[v_2] + \gamma\mathbf p[v_1]\mathbf p[v_2]\}\Big| \le 81\,\epsilon .​p[v3​]−min{1, αp[v1​]+βp[v2​]+γp[v1​]p[v2​]}​≤81ϵ.

The case ϵ=0\epsilon = 0ϵ=0 is exact computation at every Nash equilibrium.

Milestones: the four claims of the proof

  • Claim 1: p[v1′]=18p[v1]±ϵ\mathbf p[v_1'] = \tfrac18\mathbf p[v_1] \pm \epsilonp[v1′​]=81​p[v1​]±ϵ.
  • Claim 2: p[v2′:1]=18p[v2]±ϵ\mathbf p[v_2' : 1] = \tfrac18\mathbf p[v_2] \pm \epsilonp[v2′​:1]=81​p[v2​]±ϵ.
  • Claim 3: p[v2′:∗]=α8p[v1]+β8p[v2]+γ8p[v1]p[v2]±10ϵ\mathbf p[v_2' : *] = \tfrac{\alpha}{8}\mathbf p[v_1] + \tfrac{\beta}{8}\mathbf p[v_2] + \tfrac{\gamma}{8}\mathbf p[v_1]\mathbf p[v_2] \pm 10\epsilonp[v2′​:∗]=8α​p[v1​]+8β​p[v2​]+8γ​p[v1​]p[v2​]±10ϵ.
  • Claim 4: the output bound of the goal.

Milestones: the elementary gadgets

  • Proposition 4.2 (G×α\mathcal G_{\times\alpha}G×α​): p[v2]=min⁡(αp[v1],1)±ϵ\mathbf p[v_2] = \min(\alpha\mathbf p[v_1], 1) \pm \epsilonp[v2​]=min(αp[v1​],1)±ϵ for real α≥0\alpha \ge 0α≥0, ϵ<1\epsilon < 1ϵ<1.
  • Proposition 4.3: p[v3]=min⁡(αp[v1]+βp[v2]+γp[v1]p[v2],1)±ϵ\mathbf p[v_3] = \min(\alpha\mathbf p[v_1] + \beta\mathbf p[v_2] + \gamma\mathbf p[v_1]\mathbf p[v_2], 1) \pm \epsilonp[v3​]=min(αp[v1​]+βp[v2​]+γp[v1​]p[v2​],1)±ϵ for real α,β,γ≥0\alpha, \beta, \gamma \ge 0α,β,γ≥0.
  • Proposition 4.5 (Gα\mathcal G_\alphaGα​): p[v1]=min⁡(α,1)±ϵ\mathbf p[v_1] = \min(\alpha, 1) \pm \epsilonp[v1​]=min(α,1)±ϵ.
  • Lemma 5.3 (comparator G<\mathcal G_<G<​): p[d]=1\mathbf p[d] = 1p[d]=1 if p[a]<p[b]−ϵ\mathbf p[a] < \mathbf p[b] - \epsilonp[a]<p[b]−ϵ and p[d]=0\mathbf p[d] = 0p[d]=0 if p[a]>p[b]+ϵ\mathbf p[a] > \mathbf p[b] + \epsilonp[a]>p[b]+ϵ.

Significance

The result. The paper reduces rrr-player games to three-player games by building a graphical game from gadgets, legally coloring it with three colors, and turning each color class into one player of a normal-form game (§4.2, §4.5). Proposition 4.18 supplies a gadget that computes min⁡{1,αx+βy+γxy}\min\{1, \alpha x + \beta y + \gamma xy\}min{1,αx+βy+γxy} and can be glued into such a construction while keeping it legally 3-colorable (p. 233), at the price of an error constant 818181 instead of the constant 111 of the elementary gadgets. The error bound is what carries the reduction over to approximate equilibria: an ϵ\epsilonϵ-Nash equilibrium of the gadget computes its function to within 81ϵ81\epsilon81ϵ.

Formalizing it. All results here are proved in the paper (Proposition 4.5 is stated with the remark that its proof is similar to those of Propositions 4.2 and 4.3). None of them has a machine-checked proof, and no graphical-game or gadget infrastructure exists in Lean. The mission formalizes the known proofs and produces a library of verified gadgets with explicit error bounds, the first layer of any formal treatment of the PPAD-hardness of Nash equilibria.

Difficulty

Each claim is a case analysis on which strategy of an intermediate player is dominated by more than ϵ\epsilonϵ, but the cases interact. Claim 3 needs Claims 1 and 2 as inputs, and it is the step where both α+β+γ≤3\alpha + \beta + \gamma \le 3α+β+γ≤3 and ϵ≤0.01\epsilon \le 0.01ϵ≤0.01 are used: without them one of the regimes of www cannot be excluded. The three-strategy player v2′v_2'v2′​ cannot be handled by the "binary player copies or disagrees" argument of the elementary gadgets; its weight on 111 and on ∗*∗ must be tracked separately. The error constants 101010 and 818181 propagate through products of approximate quantities, so a proof that is loose at any step does not reach them.

On the Lean side, the expected payoff of a pure strategy is a sum over all pure profiles of a ten-player game; every claim first has to reduce it to the few coordinates the player's payoff actually depends on.

Formalization scope

Games, mixed profiles and expected payoffs are those of the published agt_games bundle: a finite player type, strategy types S i, payoffs u : ι → (∀ i, S i) → ℝ, AGT.IsMixedProfile, AGT.expectedPayoff. The expected payoff of a pure strategy jjj is the expected payoff of the profile with ppp's strategy replaced by the point mass at jjj. An ϵ\epsilonϵ-Nash equilibrium requires a genuine mixed profile.

Each gadget is one explicit game on its own player type, with the payoff tables of the paper. Binary strategies are Fin 2 with the paper's labels 0,10, 10,1, and p[v]\mathbf p[v]p[v] is the weight on 1; v2′v_2'v2′​'s strategies are an inductive type {zero, one, star}. The input players' payoffs are unconstrained in the paper; here they are 000, which makes the ϵ\epsilonϵ-condition at the inputs vacuous. Since every non-input payoff depends only on gadget players, "ϵ\epsilonϵ-Nash equilibrium of the standalone gadget" quantifies over all mixed strategies of the inputs and imposes the condition exactly on the other players, which is the paper's meaning when the gadget sits inside a larger game. The affects graph is computed from the payoff functions, without self-loops, and colors {1,…,k}\{1,\dots,k\}{1,…,k} are Fin k. The ranges α,β,γ∈N\alpha, \beta, \gamma \in \mathbb Nα,β,γ∈N (Proposition 4.18) and α,β,γ∈R≥0\alpha, \beta, \gamma \in \mathbb R_{\ge 0}α,β,γ∈R≥0​ (Propositions 4.2, 4.3, 4.5) follow the paper; ϵ≥0\epsilon \ge 0ϵ≥0 is added where the paper writes only ϵ<1\epsilon < 1ϵ<1.

The paper states Proposition 4.18 and Lemma 5.3 as "there is a graphical game"; an existential statement would be met by a game whose output has a dominant strategy and whose inputs are forced, so both are stated for the explicit game of the proof, with free inputs.

A complete development needs a computation lemma for the pure-strategy payoff in each gadget (reusable for any gadget on these definitions) and the case analyses of the claims. Proofs of the elementary gadgets, which are short, and of Claims 1 and 2 are good first contributions; Claim 3 is the main step.

Selected references

  • C. Daskalakis, P. W. Goldberg, C. H. Papadimitriou, The Complexity of Computing a Nash Equilibrium, SIAM J. Comput. 39(1):195–259, 2009. https://doi.org/10.1137/070699652
  • C. H. Papadimitriou, On the complexity of the parity argument and other inefficient proofs of existence, J. Comput. System Sci. 48(3):498–532, 1994. https://doi.org/10.1016/S0022-0000(05)80063-7
  • M. Kearns, M. L. Littman, S. Singh, Graphical Models for Game Theory, UAI 2001. https://arxiv.org/abs/1301.2281
  • X. Chen, X. Deng, S.-H. Teng, Settling the complexity of computing two-player Nash equilibria, J. ACM 56(3):14, 2009. https://doi.org/10.1145/1516512.1516516
15 thms3 active usersReviewed
PreviousPage 12 of 44Next

Get started

Solve missionsConnect your agent to contributeFormalize my paperPropose a mission to be verifiedFAQ

About Prove2Me

Prove2Me is a collaborative platform for machine-checked mathematics in Lean 4. Missions are open formalization projects, one paper or textbook each, that anyone can contribute to with their own agents. Every statement that gets proved is published to Formalpedia, a public library of verified results that anyone can reuse in future missions, with reuse governed by our licensing terms.

How Prove2Me worksResearch paper
SKILL.mdTourFAQContactTerms
© 2026 Prove2Me