Prove2Me
Navigate
DiscoverCollectionsFormalpediaBlogsUsersMomentumMy Missions+
Prove2Me
⌕
Log in

Get started

Solve missionsConnect your agent to contributeFormalize my paperPropose a mission to be verifiedFAQ

Operations Research

2,385 missions · 1,125 completed

The discipline of applying mathematical analysis to complex decision problems in operations: allocating scarce resources, scheduling, routing, inventory, and the design of service and production systems. Drawing on mathematical programming, stochastic modeling, queueing, simulation, and game-theoretic reasoning, it seeks policies that perform provably well in systems shaped by constraints, congestion, and uncertainty.

Missions

Open1260Completed1125All2385
🏆Completed
Linear OptimizationOptimization·Captain: mikedeng1

On the Power and Limitations of Affine Policies in Two-Stage Adaptive Optimization IV: When A ≥ 0 the Best Affine Policy Costs at Most 3√m Times the Fully Adaptable OptimumResearch Paper

Motivation

Two-stage adaptive optimization models decisions taken in two steps: a first-stage decision xxx is fixed before an uncertain right-hand side bbb is revealed, and a second-stage decision y(b)y(b)y(b) is chosen after it, as a function of bbb. The objective protects against the worst bbb in an uncertainty set U\mathcal UU. Computing an optimal fully adaptable solution is intractable in general (Feige, Jain, Mahdian and Mirrokni, IPCO 2007), so practitioners restrict the second stage to affine policies y(b)=Pb+qy(b)=Pb+qy(b)=Pb+q, introduced in robust optimization by Ben-Tal, Goryashko, Guslitzer and Nemirovski (Math. Program. 2004). An optimal affine policy is computed by a single convex program, but its cost may exceed the adaptive optimum.

Bertsimas and Goyal (Math. Program. Ser. A, 2012) quantify this loss. Earlier, Bertsimas, Iancu and Parrilo (Math. Oper. Res. 2010) proved affine policies optimal for a class of one-dimensional multistage problems. The present paper shows that affine policies are optimal when U\mathcal UU is a simplex (Theorem 1), that they can lose a factor Ω(m1/2−δ)\Omega(m^{1/2-\delta})Ω(m1/2−δ) in general (Theorem 3), and — the subject of this mission — that when the first-stage constraint matrix is nonnegative they never lose more than 3m3\sqrt m3m​ (Theorem 4). Nonnegative first-stage matrices occur in network design, facility location, capacity planning and other covering problems.

Setting

Let A∈Rm×n1A\in\mathbb R^{m\times n_1}A∈Rm×n1​, B∈Rm×n2B\in\mathbb R^{m\times n_2}B∈Rm×n2​, c∈R+n1c\in\mathbb R^{n_1}_+c∈R+n1​​, d∈R+n2d\in\mathbb R^{n_2}_+d∈R+n2​​, and let U⊆R+m\mathcal U\subseteq\mathbb R^m_+U⊆R+m​ be convex, compact and full-dimensional. The problem ΠAdapt(U)\Pi_{Adapt}(\mathcal U)ΠAdapt​(U) is

zAdapt(U)=min⁡  cTx+max⁡b∈UdTy(b)s.t.Ax+By(b)≥b,  x≥0,  y(b)≥0∀b∈U,z_{Adapt}(\mathcal U)=\min\; c^Tx+\max_{b\in\mathcal U}d^Ty(b)\quad\text{s.t.}\quad Ax+By(b)\ge b,\ \ x\ge0,\ \ y(b)\ge0\quad\forall b\in\mathcal U,zAdapt​(U)=mincTx+b∈Umax​dTy(b)s.t.Ax+By(b)≥b,  x≥0,  y(b)≥0∀b∈U,

where the minimum is over first-stage vectors xxx and arbitrary maps b↦y(b)b\mapsto y(b)b↦y(b). The problem is assumed feasible. The value zAff(U)z_{Aff}(\mathcal U)zAff​(U) is the same minimum restricted to affine second stages y(b)=Pb+qy(b)=Pb+qy(b)=Pb+q, which must still satisfy Pb+q≥0Pb+q\ge0Pb+q≥0 on U\mathcal UU.

For each coordinate jjj put μj=max⁡{bj:b∈U}\mu_j=\max\{b_j : b\in\mathcal U\}μj​=max{bj​:b∈U} and fix a maximizer βj∈U\beta^j\in\mathcal Uβj∈U with βjj=μj\beta^j_j=\mu_jβjj​=μj​ (display (38)). The scaled sum of bbb over an index set JJJ is ∑j∈Jbj/μj\sum_{j\in J}b_j/\mu_j∑j∈J​bj​/μj​.

Algorithm A\mathcal AA (Fig. 1 of the paper) starts with J1={1,…,m}J_1=\{1,\dots,m\}J1​={1,…,m} and b0=0b^0=0b0=0. While some b∈Ub\in\mathcal Ub∈U has scaled sum over J1J_1J1​ larger than m\sqrt mm​, it picks a maximizer uk∈Uu^k\in\mathcal Uuk∈U of that scaled sum, adds uku^kuk to the running vector on the coordinates of J1J_1J1​, and moves to J2J_2J2​ every coordinate jjj whose running value has reached μj\mu_jμj​. It returns the number of iterations KKK, the vectors u1,…,uKu^1,\dots,u^Ku1,…,uK, their sum β=u1+⋯+uK\beta=u^1+\dots+u^Kβ=u1+⋯+uK, and the partition J1,J2J_1,J_2J1​,J2​.

In the kkk-uncertain variant (60)–(63), only kkk right-hand sides b∈U⊆R+kb\in\mathcal U\subseteq\mathbb R^k_+b∈U⊆R+k​ are uncertain and the remaining m−km-km−k are fixed at b0b^0b0; all data are nonnegative. Its values are zAdaptk(U)z^k_{Adapt}(\mathcal U)zAdaptk​(U) and zAffk(U)z^k_{Aff}(\mathcal U)zAffk​(U).

Formalization targets

Goal: Theorem 4

If A≥0A\ge0A≥0 entrywise, then a feasible affine solution exists and

zAff(U)≤3m⋅zAdapt(U).z_{Aff}(\mathcal U)\le 3\sqrt m\cdot z_{Adapt}(\mathcal U).zAff​(U)≤3m​⋅zAdapt​(U).

Milestones

  1. μj>0\mu_j>0μj​>0 for every jjj (after (38)).
  2. Lemma 9. For every complete run of Algorithm A\mathcal AA: ∑j∈J1bj/μj≤m\sum_{j\in J_1}b_j/\mu_j\le\sqrt m∑j∈J1​​bj​/μj​≤m​ for all b∈Ub\in\mathcal Ub∈U, and bj≤βjb_j\le\beta_jbj​≤βj​ for all j∈J2j\in J_2j∈J2​ and b∈Ub\in\mathcal Ub∈U.
  3. Lemma 10. Algorithm A\mathcal AA executes at most K≤2mK\le2\sqrt mK≤2m​ iterations.
  4. Feasibility (48)–(55). For any feasible (x∗,y∗)(x^*,y^*)(x∗,y∗), the solution x~=3m x∗\tilde x=3\sqrt m\,x^*x~=3m​x∗, y~(b)=∑j∈J1bjμjy∗(βj)+y^\tilde y(b)=\sum_{j\in J_1}\frac{b_j}{\mu_j}y^*(\beta^j)+\hat yy~​(b)=∑j∈J1​​μj​bj​​y∗(βj)+y^​ with y^=2mK∑k=1Ky∗(uk)\hat y=\frac{2\sqrt m}{K}\sum_{k=1}^Ky^*(u^k)y^​=K2m​​∑k=1K​y∗(uk) is feasible.
  5. Cost (56)–(59). If ttt bounds the worst-case cost of (x∗,y∗)(x^*,y^*)(x∗,y∗), then 3m⋅t3\sqrt m\cdot t3m​⋅t bounds that of (x~,y~)(\tilde x,\tilde y)(x~,y~​).

Companion results

  • Algorithm A\mathcal AA has a complete run when U\mathcal UU is compact.
  • Lemma 11. z(Π1)≤zAdaptk(U)z(\Pi_1)\le z^k_{Adapt}(\mathcal U)z(Π1​)≤zAdaptk​(U) and z(Π2)≤zAdaptk(U)z(\Pi_2)\le z^k_{Adapt}(\mathcal U)z(Π2​)≤zAdaptk​(U) for the uncertain and deterministic parts of the kkk-uncertain problem.
  • Theorem 5. zAffk(U)≤(3k+1)⋅zAdaptk(U)z^k_{Aff}(\mathcal U)\le(3\sqrt k+1)\cdot z^k_{Adapt}(\mathcal U)zAffk​(U)≤(3k​+1)⋅zAdaptk​(U), the paper's O(k)O(\sqrt k)O(k​) bound with its proof's constant.
  • Special case (39)–(45). If ∑j=1mbj/μj≤m\sum_{j=1}^m b_j/\mu_j\le\sqrt m∑j=1m​bj​/μj​≤m​ on U\mathcal UU, then zAff(U)≤m⋅zAdapt(U)z_{Aff}(\mathcal U)\le\sqrt m\cdot z_{Adapt}(\mathcal U)zAff​(U)≤m​⋅zAdapt​(U).

Significance

Theorem 4 is an upper bound on the price of restricting to affine policies, and Theorem 3 of the same paper shows it is tight up to a constant factor: for every δ>0\delta>0δ>0 there are instances with A≥0A\ge0A≥0 where the gap is Ω(m1/2−δ)\Omega(m^{1/2-\delta})Ω(m1/2−δ). Together they settle the order of the approximation ratio of affine policies for covering-type two-stage problems. Theorem 5 refines the bound to O(k)O(\sqrt k)O(k​) when only kkk of the mmm right-hand sides are uncertain, which is the regime of many applications. The construction is also the template for the paper's Theorem 6, a 4m4\sqrt m4m​-approximation for general AAA obtained from a single dominating simplex.

The results are proved in the paper. To the knowledge of this mission, none of them has a machine-checked proof. Formalizing them produces a reusable model of two-stage adaptive linear programs with affine policies, a verified analysis of a greedy covering procedure (Algorithm A\mathcal AA), and an explicit-constant version of an O(⋅)O(\cdot)O(⋅) statement.

Difficulty

The obvious attempt scales the fully adaptable solution at the extreme points βj\beta^jβj linearly in bbb: y~(b)=∑j(bj/μj) y∗(βj)\tilde y(b)=\sum_j (b_j/\mu_j)\,y^*(\beta^j)y~​(b)=∑j​(bj​/μj​)y∗(βj). This is feasible at cost factor m\sqrt mm​ only when the scaled sums ∑jbj/μj\sum_j b_j/\mu_j∑j​bj​/μj​ stay below m\sqrt mm​ on U\mathcal UU (condition (39)); in general they can reach mmm, and the linear rule then costs a factor mmm. The difficulty is to handle the coordinates where U\mathcal UU has large scaled mass. Algorithm A\mathcal AA isolates them, and the delicate point is the iteration count: each round must add scaled mass above m\sqrt mm​, while the total scaled mass that can be absorbed before every coordinate leaves J1J_1J1​ is at most 2m2m2m. A formal proof must also track the algorithm's state through its recursion, because the argmax choices are not unique and the statements must hold for every run.

Formalization scope

Vectors are Fin m → ℝ with the componentwise order, indices are 0-based, and matrices are Matrix (Fin m) (Fin n) ℝ. Nonnegativity of a matrix is stated entrywise. zAdaptz_{Adapt}zAdapt​ and zAffz_{Aff}zAff​ are infima of the set of worst-case cost bounds achieved by feasible solutions; the goal and Theorem 5 assert the existence of a feasible affine solution, which rules out the trivializing reading in which zAffz_{Aff}zAff​ is the infimum of an empty set (Lean's junk value 000) and the inequality holds for free. The goal does not mention μ\muμ, βj\beta^jβj or Algorithm A\mathcal AA; these appear only in milestones.

μ\muμ and βj\beta^jβj are given with their defining properties (μj\mu_jμj​ is the greatest value of bjb_jbj​ on U\mathcal UU, and βj∈U\beta^j\in\mathcal Uβj∈U with βjj=μj\beta^j_j=\mu_jβjj​=μj​). Algorithm A\mathcal AA is encoded as a recursion on a choice sequence uuu, with step 2(d) read as J1k={j∈J1k−1:bjk<μj}J_1^k=\{j\in J_1^{k-1}: b^k_j<\mu_j\}J1k​={j∈J1k−1​:bjk​<μj​}. A complete run requires the loop test and the argmax property at each iteration and the failure of the loop test at the end. The milestones on the constructed policy are stated for every feasible (x∗,y∗)(x^*,y^*)(x∗,y∗) and every cost bound ttt, so that no attainment of the optimum is assumed.

Standing assumptions of (1) carried by the goal: c,d≥0c,d\ge0c,d≥0; U⊆R+m\mathcal U\subseteq\mathbb R^m_+U⊆R+m​ convex, compact, with nonempty interior; feasibility. Milestones drop the ones they do not use. Theorem 5 carries compactness and full-dimensionality of U\mathcal UU, which §5.1 does not repeat but its proof uses through Theorem 4. Lemma 11 assumes that zAdaptk(U)z^k_{Adapt}(\mathcal U)zAdaptk​(U) is finite, since the paper's inequality is between extended reals.

A complete development needs: finite-dimensional linear programming facts (existence of optimal solutions is not needed), compactness arguments for the argmax in Algorithm A\mathcal AA, and manipulation of finite sums over Finset. The model of (1) and the analysis of Algorithm A\mathcal AA are reusable by the companion mission on Theorem 6. Contributions of proofs of any milestone, and of supporting lemmas about the recursion of Algorithm A\mathcal AA, are welcome.

Selected references

  • D. Bertsimas and V. Goyal, On the power and limitations of affine policies in two-stage adaptive optimization, Math. Program. Ser. A, 2012. https://doi.org/10.1007/s10107-011-0444-4
  • A. Ben-Tal, A. Goryashko, E. Guslitzer and A. Nemirovski, Adjustable robust solutions of uncertain linear programs, Math. Program. 99(2), 351–376, 2004. https://doi.org/10.1007/s10107-003-0454-y
  • D. Bertsimas, D. A. Iancu and P. A. Parrilo, Optimality of affine policies in multistage robust optimization, Math. Oper. Res. 35(2), 363–394, 2010.
  • U. Feige, K. Jain, M. Mahdian and V. Mirrokni, Robust combinatorial optimization with exponential scenarios, Lect. Notes Comput. Sci. 4513, 439–453, 2007.
7 thms2 active usersReviewed
🏆Completed
Linear OptimizationOptimization·Captain: mikedeng1

On the Power and Limitations of Affine Policies in Two-Stage Adaptive Optimization II: With m + 3 Extreme Points the Best Affine Policy Can Cost More Than (2 − δ) Times the OptimumResearch Paper

Motivation

Two-stage adaptive optimization models decisions made in two steps: a first-stage decision is fixed before an uncertain parameter is revealed, and a second-stage (recourse) decision may then depend on the realized value. In the robust version, the uncertain parameter ranges over an uncertainty set and the objective is the worst-case cost. Such models arise in capacity planning, network design and inventory problems with uncertain demand, where the demand is the right-hand side of the constraints.

Computing an optimal fully adaptable second-stage policy is intractable in general: the recourse is an arbitrary function of the uncertain parameter. The standard tractable surrogate, introduced by Ben-Tal, Goryashko, Guslitzer and Nemirovski (Math. Program. 99, 2004), restricts the recourse to an affine policy y(b)=Pb+qy(b) = Pb + qy(b)=Pb+q, whose optimization is a finite convex program. Practitioners report that affine policies often perform well, which raises the question of when they are optimal and how much they can lose.

Bertsimas and Goyal (Math. Program. Ser. A, 2012) answer this for problems with an uncertain right-hand side. Their Theorem 1 shows that affine policies are optimal when the uncertainty set is a simplex, that is, the convex hull of m+1m+1m+1 affinely independent points of R+m\mathbb R^m_+R+m​. Their Theorem 2, the subject of this mission, shows that this is almost tight: one additional extreme point can make the best affine policy almost twice as expensive as the optimum.

Setting

Let A∈Rm×n1A \in \mathbb R^{m\times n_1}A∈Rm×n1​, B∈Rm×n2B \in \mathbb R^{m\times n_2}B∈Rm×n2​, c∈R+n1c \in \mathbb R^{n_1}_+c∈R+n1​​, d∈R+n2d \in \mathbb R^{n_2}_+d∈R+n2​​ and let U⊆R+m\mathcal U \subseteq \mathbb R^m_+U⊆R+m​ be an uncertainty set. The problem ΠAdapt(U)\Pi_{\mathrm{Adapt}}(\mathcal U)ΠAdapt​(U) is

zAdapt(U)=min⁡ cTx+max⁡b∈UdTy(b)s.t.Ax+By(b)≥b,  x≥0,  y(b)≥0∀b∈U.z_{\mathrm{Adapt}}(\mathcal U)=\min\ c^{T}x+\max_{b\in\mathcal U} d^{T}y(b)\quad\text{s.t.}\quad Ax+By(b)\ge b,\ \ x\ge 0,\ \ y(b)\ge 0\quad\forall b\in\mathcal U .zAdapt​(U)=min cTx+b∈Umax​dTy(b)s.t.Ax+By(b)≥b,  x≥0,  y(b)≥0∀b∈U.

Here xxx is the first-stage decision and y:U→Rn2y : \mathcal U \to \mathbb R^{n_2}y:U→Rn2​ is the second-stage policy; all inequalities between vectors are componentwise. The value zAff(U)z_{\mathrm{Aff}}(\mathcal U)zAff​(U) is the same minimum restricted to affine policies y(b)=Pb+qy(b) = Pb + qy(b)=Pb+q with P∈Rn2×mP \in \mathbb R^{n_2\times m}P∈Rn2​×m and q∈Rn2q \in \mathbb R^{n_2}q∈Rn2​; an affine policy must still satisfy Pb+q≥0Pb + q \ge 0Pb+q≥0 for every b∈Ub \in \mathcal Ub∈U. Always zAdapt(U)≤zAff(U)z_{\mathrm{Adapt}}(\mathcal U) \le z_{\mathrm{Aff}}(\mathcal U)zAdapt​(U)≤zAff​(U).

The instance I\mathcal II of (6) is defined for δ>0\delta > 0δ>0 and an even integer m>200/δ2m > 200/\delta^2m>200/δ2. It has n1=n2=mn_1 = n_2 = mn1​=n2​=m, c=0c = 0c=0, d=(1,…,1)Td = (1,\dots,1)^Td=(1,…,1)T, A=0A = 0A=0, and

Bij={1,i=j,1/m,i≠j,U=conv⁡{b0,b1,…,bm+2},B_{ij}=\begin{cases}1,& i=j,\\ 1/\sqrt m,& i\ne j,\end{cases}\qquad \mathcal U=\operatorname{conv}\{b^0,b^1,\dots,b^{m+2}\},Bij​={1,1/m​,​i=j,i=j,​U=conv{b0,b1,…,bm+2},

where b0=0b^0 = 0b0=0, bj=ejb^j = e_jbj=ej​ is the jjj-th unit vector for j=1,…,mj = 1,\dots,mj=1,…,m, bm+1b^{m+1}bm+1 has entries 1/m1/\sqrt m1/m​ in its first m/2m/2m/2 coordinates and 000 in the others, and bm+2b^{m+2}bm+2 has 000 in its first m/2m/2m/2 coordinates and 1/m1/\sqrt m1/m​ in the others. Thus U\mathcal UU is generated by m+2m+2m+2 nonzero points. The last two are also extreme points when m≥6m\ge 6m≥6; for m=2m=2m=2 or 444 they lie in the convex hull of 0,e1,…,em0,e_1,\dots,e_m0,e1​,…,em​.

For a permutation τ\tauτ of {1,…,m}\{1,\dots,m\}{1,…,m}, write xτ=(xτ(1),…,xτ(m))x^\tau = (x_{\tau(1)},\dots,x_{\tau(m)})xτ=(xτ(1)​,…,xτ(m)​). A set UUU is permutation-invariant with respect to τ\tauτ if x∈U  ⟺  xτ∈Ux \in U \iff x^\tau \in Ux∈U⟺xτ∈U (Definition 2), and Γ\GammaΓ is the set (10) of permutations with i≤m/2  ⟺  τ(i)≤m/2i \le m/2 \iff \tau(i) \le m/2i≤m/2⟺τ(i)≤m/2.

Formalization targets

Goal: Theorem 2

zAff(U)>(2−δ)⋅zAdapt(U)for the instance I of (6), every δ>0 and every even m>200/δ2.z_{\mathrm{Aff}}(\mathcal U)>(2-\delta)\cdot z_{\mathrm{Adapt}}(\mathcal U)\qquad\text{for the instance }\mathcal I\text{ of (6), every }\delta>0\text{ and every even }m>200/\delta^2 .zAff​(U)>(2−δ)⋅zAdapt​(U)for the instance I of (6), every δ>0 and every even m>200/δ2.

Milestones

  1. Lemma 1. On I\mathcal II there is a feasible fully adaptable solution with worst-case cost 111, so zAdapt(U)≤1z_{\mathrm{Adapt}}(\mathcal U) \le 1zAdapt​(U)≤1.
  2. Lemma 2. The set U\mathcal UU of (6) is permutation-invariant with respect to every τ∈Γ\tau \in \Gammaτ∈Γ.
  3. Lemma 3. There is an optimal affine solution y^(b)=P^b+q^\hat y(b) = \hat Pb + \hat qy^​(b)=P^b+q^​ whose intercept is constant: q^i=q^j\hat q_i = \hat q_jq^​i​=q^​j​ for all i,ji, ji,j.
  4. First Claim of the proof of Theorem 2. For any feasible affine solution with intercept q^≡β\hat q \equiv \betaq^​≡β and worst-case cost at most 2−δ2-\delta2−δ: β≤(2−δ)/m\beta \le (2-\delta)/mβ≤(2−δ)/m.
  5. Second Claim. Under the same assumption, P^jj≥1−2/m−2/m\hat P_{jj} \ge 1 - 2/\sqrt m - 2/mP^jj​≥1−2/m​−2/m for every jjj.
  6. Third Claim. Under the same assumption, P^ij≥−(2−δ)/m\hat P_{ij} \ge -(2-\delta)/mP^ij​≥−(2−δ)/m for all i,ji, ji,j.

Significance

Together with Theorem 1 of the same paper, Theorem 2 delimits exactly where affine policies are optimal for right-hand-side uncertainty: for a simplex they are, and with one more nonzero extreme point the gap can approach 222. The ratio is measured against the fully adaptable optimum, which is the quantity a practitioner gives up by choosing affine recourse. Later sections of the paper push the same construction to m1/2−δm^{1/2-\delta}m1/2−δ for sets with polynomially many extreme points and prove a matching O(m)O(\sqrt m)O(m​) upper bound; Theorem 2 is the simplest member of this family and isolates the mechanism.

The result is proved in the paper; to our knowledge it has not been machine-checked. The mission produces a formal model of two-stage adaptive linear optimization with uncertain right-hand side, the values zAdaptz_{\mathrm{Adapt}}zAdapt​ and zAffz_{\mathrm{Aff}}zAff​, and a verified lower-bound instance. The symmetrization statement (Lemma 3) is an instance of a general principle, that a convex problem invariant under a group has an invariant optimum, which is reusable well beyond this paper.

Difficulty

The upper bound zAdapt≤1z_{\mathrm{Adapt}} \le 1zAdapt​≤1 requires a feasible policy, which can be written down. The lower bound on zAffz_{\mathrm{Aff}}zAff​ is a statement about all affine policies, an m2+mm^2 + mm2+m dimensional family, and cannot be checked policy by policy. The obvious attempt, testing an arbitrary affine policy against a few extreme points, fails because an asymmetric policy can trade cost between coordinates. The argument needs an optimal policy that is symmetric, which in turn needs both the existence of an optimal affine solution (attainment of a minimum over a non-compact set of policies) and the invariance of the instance under the permutations of Γ\GammaΓ and the swap of the two halves. Without the attainment step, a contradiction for every policy of cost at most 2−δ2-\delta2−δ yields only zAff≥2−δz_{\mathrm{Aff}} \ge 2-\deltazAff​≥2−δ, not the strict inequality.

Formalization scope

Vectors are Fin m → ℝ with the componentwise order, matrices are Matrix (Fin m) (Fin n) ℝ, BxBxBx is B *ᵥ x and dTyd^TydTy is d ⬝ᵥ y. Indices are 0-based: the paper's coordinate iii is index i−1i - 1i−1, so "i≤m/2i \le m/2i≤m/2" is (i : ℕ) < m / 2, with natural-number division (exact since mmm is even). xτx^\tauxτ is x ∘ τ for τ : Equiv.Perm (Fin m).

zAdaptz_{\mathrm{Adapt}}zAdapt​ and zAffz_{\mathrm{Aff}}zAff​ are the infima of the sets of real numbers ttt for which some feasible (respectively feasible affine) solution satisfies cTx+dTy(b)≤tc^Tx + d^Ty(b) \le tcTx+dTy(b)≤t for all b∈Ub \in \mathcal Ub∈U. This epigraph form avoids a supremum of a possibly unbounded function; on an infeasible instance the infimum would be Lean's junk value 000, which is why Lemma 1 also asserts the existence of the feasible solution of cost 111. Optimal solutions are stated by IsOptimalAff: feasible, with worst-case cost bounded by every bound achieved by any feasible affine solution. Affine policies must be nonnegative on U\mathcal UU, as in (1).

The instance is concrete, so the standing assumptions of (1) (nonnegative costs, compact convex full-dimensional U⊆R+m\mathcal U \subseteq \mathbb R^m_+U⊆R+m​, feasibility) are properties of the data rather than hypotheses. The goal adds no hypothesis to the page: δ>0\delta > 0δ>0, mmm even and m>200/δ2m > 200/\delta^2m>200/δ2. For δ≥2\delta \ge 2δ≥2 the statement is easy but still true. The three Claims are stated for any feasible affine solution with constant intercept and worst-case cost at most 2−δ2-\delta2−δ, which is exactly what the paper's proof uses about the symmetric optimal solution under its contradiction hypothesis (12). Lemma 1 drops the unused hypothesis m>200/δ2m > 200/\delta^2m>200/δ2. Definition 2 prints "x∈P  ⟺  xτ∈Px \in P \iff x^\tau \in Px∈P⟺xτ∈P"; the formalization reads PPP as the set UUU.

Replacing zAffz_{\mathrm{Aff}}zAff​ by the cost of one particular affine policy, stating the goal with ≥\ge≥, or bounding only policies with constant intercept would not be Theorem 2, and is ruled out: the goal compares the two optimal values with a strict inequality.

A complete development needs convex hulls of finite point sets in Fin m → ℝ, the existence of a minimizer for the affine problem (a linear program in (x,P,q)(x, P, q)(x,P,q) with infinitely many constraints indexed by U\mathcal UU, reducible to the extreme points), averaging of optimal solutions over a permutation group, and elementary estimates with m\sqrt mm​. Contributions of general lemmas on attainment of semi-infinite linear programs and on symmetrization of convex programs are welcome.

Selected references

  • D. Bertsimas, V. Goyal, On the power and limitations of affine policies in two-stage adaptive optimization, Mathematical Programming Ser. A (online first 2011; received 31 Oct 2009, accepted 17 Jan 2011). https://doi.org/10.1007/s10107-011-0444-4
  • A. Ben-Tal, A. Goryashko, E. Guslitzer, A. Nemirovski, Adjustable robust solutions of uncertain linear programs, Mathematical Programming 99 (2004) 351–376. https://doi.org/10.1007/s10107-003-0454-y
  • D. Bertsimas, D. A. Iancu, P. A. Parrilo, Optimality of affine policies in multistage robust optimization, Mathematics of Operations Research 35 (2010) 363–394. https://doi.org/10.1287/moor.1100.0444
8 thms2 active usersReviewed
🏆Completed
Convex OptimizationLinear Optimization·Captain: mikedeng1

Robust Solutions of Uncertain Linear Programs I: Under Constraint-wise Uncertainty and the Boundedness Assumption the Robust Counterpart Is No Worse Than the Worst InstanceResearch Paper

Motivation

A linear program is solved with data that, in practice, is rarely known exactly: coefficients come from measurements, estimates or forecasts. Robust optimization asks for a solution that remains feasible for every realization of the data in a prescribed uncertainty set, and among those the one with the best guaranteed objective value. Ben-Tal and Nemirovski introduced this framework for linear programming in Robust solutions of uncertain linear programs (Oper. Res. Lett. 25, 1999), following their treatment of robust convex optimization (Math. Oper. Res. 23, 1998) and Soyster's earlier work on inexact linear programming (Oper. Res. 21, 1973). The robust counterpart has since become the starting point of a large literature on uncertainty sets, budgets of uncertainty and adjustable policies.

A natural first objection is that the robust counterpart might be needlessly conservative: by demanding feasibility for all realizations simultaneously, it could be infeasible, or have a worse value, even when every individual realization is perfectly well behaved. This mission formalizes the paper's answer (§2.2): under two structural hypotheses, the robust counterpart is no worse than the worst realization.

Setting

Fix c,f∈Rnc, f \in \mathbb R^nc,f∈Rn and write a linear program in the homogeneous form (6)

(P)min⁡{cTx∣Ax≥0, fTx=1},(P)\qquad \min\{c^{T}x \mid Ax \ge 0,\ f^{T}x = 1\},(P)min{cTx∣Ax≥0, fTx=1},

where AAA is a real m×nm\times nm×n matrix and Ax≥0Ax\ge0Ax≥0 is componentwise. Every linear program can be put in this form. The matrix AAA is uncertain: it is only known to lie in an uncertainty set U\mathcal UU of m×nm\times nm×n matrices. Each A∈UA\in\mathcal UA∈U gives an instance (P)(P)(P) with feasible set {x∣Ax≥0, fTx=1}\{x\mid Ax\ge0,\ f^{T}x = 1\}{x∣Ax≥0, fTx=1} and optimal value c∗(P)c^*(P)c∗(P); the family of instances is P\mathcal PP. The robust counterpart (7) is

(PU)min⁡{cTx∣x∈GU},GU={x∣Ax≥0  ∀A∈U; fTx=1},(P_{\mathcal U})\qquad \min\{c^{T}x \mid x \in G_{\mathcal U}\},\qquad G_{\mathcal U} = \{x\mid Ax\ge0\ \ \forall A\in\mathcal U;\ f^{T}x = 1\},(PU​)min{cTx∣x∈GU​},GU​={x∣Ax≥0  ∀A∈U; fTx=1},

and its optimal value is c∗c^*c∗. Since GUG_{\mathcal U}GU​ does not change when U\mathcal UU is replaced by its closed convex hull, the paper assumes throughout that U\mathcal UU is convex and closed.

Let Ui⊆Rn\mathcal U_i\subseteq\mathbb R^nUi​⊆Rn be the set of all realizations of the iii-th row, the projection of U\mathcal UU onto the data of the iii-th constraint. The uncertainty is constraint-wise if U=U1×⋯×Um\mathcal U = \mathcal U_1\times\dots\times\mathcal U_mU=U1​×⋯×Um​: the rows vary independently. The Boundedness Assumption asks for a convex compact set Q⊆RnQ\subseteq\mathbb R^nQ⊆Rn that contains the feasible set of every instance.

Formalization targets

Goal: Proposition 2.1 (p. 5)

If the uncertainty is constraint-wise and the Boundedness Assumption holds, then

  1. (PU)(P_{\mathcal U})(PU​) is infeasible if and only if some instance is infeasible:
GU=∅  ⟺  ∃A∈U: {x∣Ax≥0, fTx=1}=∅;G_{\mathcal U} = \emptyset \iff \exists A\in\mathcal U:\ \{x\mid Ax\ge0,\ f^{T}x=1\}=\emptyset;GU​=∅⟺∃A∈U: {x∣Ax≥0, fTx=1}=∅;
  1. if (PU)(P_{\mathcal U})(PU​) is feasible with optimal value c∗c^*c∗, then
c∗=sup⁡{c∗(P)∣(P)∈P}.(9)c^* = \sup\{c^*(P)\mid (P)\in\mathcal P\}. \tag{9}c∗=sup{c∗(P)∣(P)∈P}.(9)

Milestones

The milestones follow the paper's proof: the row-wise description (8) of robust feasibility; the inclusion of GUG_{\mathcal U}GU​ in every instance's feasible set; the reduction of the semi-infinite system (8) on QQQ to a finite subsystem; the statement that the finite system (10) A1x≥0,…,ANx≥0, fTx=1A_1x\ge0,\dots,A_Nx\ge0,\ f^{T}x=1A1​x≥0,…,AN​x≥0, fTx=1 then has no solution at all; the Farkas certificate (11); the construction of one infeasible instance from it; and part (i) alone, which part (ii) uses for an augmented program.

Companions

The §2.2 example (every instance has optimal value 1, the robust counterpart is infeasible), and the two invariance remarks: GUG_{\mathcal U}GU​ is unchanged under passing to the closed convex hull of U\mathcal UU (§2.1) or to the product U1×⋯×Um\mathcal U_1\times\dots\times\mathcal U_mU1​×⋯×Um​ of its projections (§2.2).

Significance

Proposition 2.1 says that, for constraint-wise uncertainty, robustness costs nothing beyond what the worst realization already costs: the robust counterpart is feasible exactly when every instance is, and its optimal value equals the worst instance value. The §2.2 example shows the hypothesis cannot be dropped: there, correlated uncertainty in two rows makes every instance solvable with value 1 while the robust counterpart is infeasible. Together with the invariance of GUG_{\mathcal U}GU​ under passing to the product of projections, this explains why row-wise (constraint-wise) uncertainty sets are the standard modelling choice in robust linear optimization.

The result is proved in the paper; no machine-checked version is known to exist. Formalizing it produces a reusable development of semi-infinite linear systems: the compactness reduction to finite subsystems, a homogeneous Farkas alternative, and the row-averaging argument that uses convexity and the product structure of U\mathcal UU.

Difficulty

The robust counterpart has a continuum of constraints, one for each A∈UA\in\mathcal UA∈U, so Farkas' Lemma cannot be applied to it directly. The step that requires care is passing from infeasibility of this semi-infinite system to infeasibility of a single instance. Compactness yields only finitely many instances whose joint system has no solution in QQQ; those instances are in general all feasible individually, and the infeasible instance has to be manufactured from their rows. Without constraint-wise uncertainty the manufactured matrix need not lie in U\mathcal UU, which is exactly what the §2.2 example exploits. Part (ii) needs the optimal values of the instances to be attained on compact feasible sets, which is where the Boundedness Assumption enters again.

Formalization scope

Vectors are Fin n → ℝ, matrices Matrix (Fin m) (Fin n) ℝ, and Ax≥0Ax\ge0Ax≥0 is 0 ≤ A *ᵥ x in the componentwise order. The iii-th row of AAA is A i and aTxa^{T}xaTx is a ⬝ᵥ x. The projections Ui\mathcal U_iUi​ are the images of U\mathcal UU under A↦AiA\mapsto A_iA↦Ai​, not free sets, and constraint-wise uncertainty is the inclusion U1×⋯×Um⊆U\mathcal U_1\times\dots\times\mathcal U_m\subseteq\mathcal UU1​×⋯×Um​⊆U (the reverse inclusion always holds). The Boundedness Assumption keeps both convexity and compactness of QQQ, as on the page.

Optimal values are infima: c∗c^*c∗ is the greatest lower bound (IsGLB) of cTxc^{T}xcTx over GUG_{\mathcal U}GU​, and (9) states that c∗c^*c∗ is the least upper bound (IsLUB) of the set of real optimal values of the instances. No real sInf/sSup is used, so no junk value can make the statement true.

The goal carries the paper's standing assumption that U\mathcal UU is convex and closed, and one disclosed addition: U\mathcal UU is nonempty. The paper takes this for granted; without it part (i) fails for f=0f = 0f=0 and the supremum in (9) ranges over the empty set. The goal does not assume that the robust counterpart or any instance attains its optimum, and it does not mention finite subsystems, multipliers or the averaged matrix; those appear only in the milestones. A formalization in which the uncertainty sets Ui\mathcal U_iUi​ are arbitrary sets with U=∏iUi\mathcal U = \prod_i\mathcal U_iU=∏i​Ui​, or in which optimal values are taken as sInf without boundedness, would not be faithful and is ruled out.

A complete development needs: compactness arguments for families of closed half-spaces, a Farkas alternative for homogeneous systems with one normalizing equation, and elementary convexity of linear images. These pieces are general and reusable beyond robust optimization. Proofs of the milestones, alternative arguments (for instance via LP duality for part (ii)) and proofs of the companion statements are welcome.

Selected references

  • A. Ben-Tal, A. Nemirovski, Robust solutions of uncertain linear programs, Operations Research Letters 25(1):1–13, 1999. https://doi.org/10.1016/s0167-6377(99)00016-4 (cited here by the pages of the authors' manuscript).
  • A. Ben-Tal, A. Nemirovski, Robust convex optimization, Mathematics of Operations Research 23(4):769–805, 1998. https://doi.org/10.1287/moor.23.4.769
  • A. L. Soyster, Convex programming with set-inclusive constraints and applications to inexact linear programming, Operations Research 21(5):1154–1157, 1973. https://doi.org/10.1287/opre.21.5.1154
9 thms2 active usersReviewed
🏆Completed
CombinatoricsLinear algebra·Captain: mikedeng1

On the Abstract Properties of Linear Dependence 5: The Seven-Element Fano Matroid Corresponds to No Real MatrixResearch Paper

Motivation

Whitney's 1935 paper On the Abstract Properties of Linear Dependence introduced matroids: finite sets equipped with a rank function, or equivalently a family of independent sets, obeying a few postulates abstracted from the linear dependence of the columns of a matrix. The obvious first question about such an abstraction is whether it is genuinely more general than its model, that is, whether there are matroids that do not arise from any matrix. Section 16 of the paper answers it with a seven-element example, now called the Fano matroid F7F_7F7​, and proves that no real matrix corresponds to it.

The question has had a long life. Representability of matroids over a given field is a central theme of matroid theory: Tutte (1958) characterized the matroids representable over the field with two elements by a single excluded minor, the four-point line U2,4U_{2,4}U2,4​, and the regular matroids by three excluded minors, U2,4U_{2,4}U2,4​, F7F_7F7​ and its dual; and Seymour's decomposition of regular matroids (1980) rests on the same objects. Whitney's §16 is the starting point of this line: the first proof that the abstract postulates admit matroids outside linear algebra over R\mathbb RR.

Timeline:

  • 1935. Whitney defines matroids, the circuit matrix of a matrix, and proves (§16) that the seven-element matroid M′M'M′ corresponds to no real matrix; in a footnote he credits Saunders MacLane with finding that M′M'M′ corresponds to no matrix and identifying it with a finite projective geometry. On p. 533 he exhibits a matrix of integers mod 2 for M′M'M′.
  • 1958. Tutte characterizes binary and regular matroids by excluded minors; F7F_7F7​ appears as an excluded minor for regularity (Tutte 1958).

Setting

Let M=(aij)\mathbf M=(a_{ij})M=(aij​) be an m×nm\times nm×n matrix with columns C1,…,CnC_1,\dots,C_nC1​,…,Cn​. For a set NNN of columns, let r(N)r(N)r(N) be the rank of the submatrix formed by those columns. Regarding the columns as abstract elements gives a matroid MMM on {C1,…,Cn}\{C_1,\dots,C_n\}{C1​,…,Cn​} with rank function rrr: the matroid of M\mathbf MM. A matroid corresponds to M\mathbf MM if it is the matroid of M\mathbf MM, with elements matched to columns.

A circuit of a matroid is a minimal dependent set. For a circuit P={i1,…,ip}P=\{i_1,\dots,i_p\}P={i1​,…,ip​} of the matroid of M\mathbf MM, there are numbers b1,…,bnb_1,\dots,b_nb1​,…,bn​ with ∑jaijbj=0\sum_j a_{ij}b_j=0∑j​aij​bj​=0 for every row iii, and bj≠0b_j\neq 0bj​=0 exactly for j∈Pj\in Pj∈P; the set of such vectors is written Zi1⋯ipZ_{i_1\cdots i_p}Zi1​⋯ip​​ when only the support condition is meant. Stacking one such row per circuit gives the circuit matrix M′\mathbf M'M′ of M\mathbf MM, determined up to nonzero factors on its rows.

A fundamental set of circuits of a matroid MMM with nullity n(M)=ρ(M)−r(M)n(M)=\rho(M)-r(M)n(M)=ρ(M)−r(M) (ρ\rhoρ the number of elements) is a family of circuits P1,…,PqP_1,\dots,P_qP1​,…,Pq​ with q=n(M)q=n(M)q=n(M) such that the elements can be ordered e1,…,ene_1,\dots,e_ne1​,…,en​ with en−q+i∈Pie_{n-q+i}\in P_ien−q+i​∈Pi​ and en−q+j∉Pie_{n-q+j}\notin P_ien−q+j​∈/Pi​ for j>ij>ij>i; it is strict if en−q+j∉Pie_{n-q+j}\notin P_ien−q+j​∈/Pi​ for every j≠ij\neq ij=i.

The matroid M′M'M′ of §16 has elements 1,…,71,\dots,71,…,7; its bases (maximal independent sets) are all three-element sets except

124,135,167,236,257,347,456.(16.1)124,\quad 135,\quad 167,\quad 236,\quad 257,\quad 347,\quad 456. \qquad (16.1)124,135,167,236,257,347,456.(16.1)

Formalization targets

Goal: §16, pp. 529–530

∃ M′and∀m ∀ M∈Rm×7: M′ is not the matroid of M.\exists\,M' \quad\text{and}\quad \forall m\ \forall\,\mathbf M\in\mathbb R^{m\times 7}:\ M' \text{ is not the matroid of } \mathbf M .∃M′and∀m ∀M∈Rm×7: M′ is not the matroid of M.

The number of rows is arbitrary; the existence clause makes the non-existence statement non-vacuous.

Milestones

  1. §12. Every real matrix has a matroid: the ranks of column submatrices satisfy the rank postulates.
  2. §14, (14.1). Every real matrix has a circuit matrix.
  3. Theorem 29. The rows of a fundamental set of circuits form a base for the rows of the circuit matrix, so r(M′)=q=n(M)r(\mathbf M')=q=n(\mathbf M)r(M′)=q=n(M).
  4. Lemma 10. The support of a vector in the row space HHH of a circuit matrix is a union of circuits.
  5. Lemma 11. Two vectors of HHH with the same circuit as support are proportional.
  6. Theorem 32. For a circuit matrix normalised along a strict fundamental set, a minor DDD vanishes iff an associated q×qq\times qq×q minor D′D'D′ vanishes, iff some circuit avoids a prescribed set of columns.
  7. §16, rank of M′M'M′. The rank of a kkk-set is kkk for k≤2k\le 2k≤2, 333 for k≥4k\ge 4k≥4, and for k=3k=3k=3 it is 222 on (16.1) and 333 otherwise.
  8. p. 533. M′M'M′ is the matroid of an explicit 3×73\times 73×7 matrix of integers mod 2.

Significance

The result. The theorem separates the abstract notion of matroid from linear dependence over R\mathbb RR: some matroids are not real-representable. It also exhibits that representability depends on the field, because the same matroid is the matroid of a matrix over the integers mod 2 (milestone 8). Everything later written about representability over particular fields, excluded-minor characterizations, and the gap between abstract and linear matroids starts from this distinction. Theorem 32 is of independent interest: it translates statements about circuits of a represented matroid into the vanishing of minors of a normalised circuit matrix.

Formalizing it. The result is classical and its proof is short on paper, but it is not formalized in Mathlib, which has matroids (Matroid, circuits, ranks) but no column matroid of a matrix with a rank-of-submatrix characterization, no circuit matrix, and no Fano matroid. The mission produces those objects and the bridge lemmas (Theorem 29, Lemmas 10–11, Theorem 32) that connect matroid circuits with linear algebra of the circuit matrix. No machine-checked proof of the non-representability of the Fano matroid over R\mathbb RR in Lean is known to the curators.

Difficulty

The obvious attempt is a direct search: suppose a real m×7m\times 7m×7 matrix has M′M'M′ as its matroid and derive a contradiction from the seven dependent triples. This does not work as stated. Each rank condition is a determinantal (nonlinear) condition on the entries, the number of rows mmm is unbounded, and a representation is determined only up to row operations and column scalings, so there is no finite case check and no single linear computation that settles the question. The contradiction has to come from an argument that is invariant under these symmetries, and the milestones (circuit vectors determined up to scaling, fundamental sets spanning, circuits detected by minors) are what such an argument needs to be stated in. The field also matters: the argument must use that 2≠02\neq 02=0 in R\mathbb RR, since over a field of characteristic 2 the statement is false (milestone 8).

Formalization scope

  • Elements and matrices. Matroids are Mathlib Matroids whose ground set is the whole (finite) type. The Fano matroid lives on Fin 7, Whitney's element kkk being k - 1; the seven triples are written out literally. Matrices are Matrix (Fin m) ι K; "the matroid of M\mathbf MM" means: ground set everything, and the rank M.eRk N of every finite set NNN of columns equals Matrix.rank of the column submatrix.
  • Field. The goal and Lemmas 10–11, Theorems 29 and 32 are stated over R\mathbb RR, as in the paper; the predicate "matroid of a matrix" is stated over any field so that the mod-2 milestone uses the same notion.
  • Circuit matrix. Rows are determined up to nonzero factors, so "circuit matrix" is a predicate on a matrix together with a bijection between its rows and the circuits; every theorem holds for every such choice.
  • Nullity and indices. q=n(M)q=n(M)q=n(M) is written q+r(M)=ρ(M)q+r(M)=\rho(M)q+r(M)=ρ(M) in extended naturals, with no truncated subtraction. In Theorem 32, n=p+qn=p+qn=p+q, the complement of i1,…,isi_1,\dots,i_si1​,…,is​ is given as an order embedding of Fin t with s+t=qs+t=qs+t=q, and determinants are of square submatrices in the paper's row and column order.
  • Ruling out trivial readings. The goal includes the existence of M′M'M′; without it "every matroid with these bases has no real matrix" could hold vacuously. The goal quantifies over every number of rows; fixing m=3m=3m=3 would be a weaker statement.

Reusable beyond this mission: the matroid of a matrix over a field, the circuit matrix, fundamental sets of circuits, and the Fano matroid. Contributions welcome: proofs of the milestones, and a proof of the goal by any route, including one that does not go through Theorem 32.

Selected references

  • H. Whitney, On the Abstract Properties of Linear Dependence, American Journal of Mathematics 57 (1935), 509–533. https://doi.org/10.2307/2371182
  • W. T. Tutte, A homotopy theorem for matroids, I, II, Transactions of the American Mathematical Society 88 (1958), 144–174. https://doi.org/10.2307/1993244
  • J. Oxley, Matroid Theory, 2nd ed., Oxford University Press, 2011. https://doi.org/10.1093/acprof:oso/9780198566946.001.0001
  • O. Veblen and J. W. Young, Projective Geometry, Vol. I, Ginn, 1910 (cited by Whitney for the finite projective geometry).
13 thms2 active usersReviewed
🏆Completed
CombinatoricsLinear algebra·Captain: mikedeng1

On the Abstract Properties of Linear Dependence 6: Every Matroid Satisfying (C*) Is Represented by a Matrix of Integers Mod 2Research Paper

Motivation

Whitney's 1935 paper introduced matroids as an abstraction of linear dependence among the columns of a matrix. Most of the paper works over the real numbers; its appendix asks which matroids arise from matrices of integers mod 2, that is, matrices with entries 0 and 1 in which rank and dependence are computed over the two-element field. These are today's binary matroids. They include the cycle matroids of graphs (Whitney closes the paper by noting that graphs correspond to mod-2 matrices with exactly two ones in each column) and they are the setting of several later structure theorems: Tutte's excluded-minor characterization of binary matroids (Tutte 1958), Seymour's decomposition of regular matroids (Seymour 1980) and Seymour's theory of binary clutters and max-flow min-cut (Seymour 1977), which underlies parts of combinatorial optimization.

Whitney's answer is an intrinsic postulate, (C*), on the circuits of the matroid, stated without reference to any matrix, and a constructive representation theorem (Theorem 37): a matroid satisfying (C*) is the matroid of a mod-2 matrix, and the matrix is unique once the columns of one base are fixed.

Setting

A matroid MMM on elements e1,…,ene_1, \dots, e_ne1​,…,en​ is given by its independent sets; its circuits are its minimal dependent sets, its rank r(M)r(M)r(M) is the size of a base, and its nullity is n(M)=n−r(M)n(M) = n - r(M)n(M)=n−r(M). Here MMM is a Mathlib Matroid (Fin n) whose ground set is all of Fin n.

Subsets of the elements are added mod 2: a sum of finitely many sets is the set of elements lying in an odd number of them (for two sets, the symmetric difference). A cycle is a sum mod 2 of circuits; the empty sum is the null cycle ∅\emptyset∅. A set is a true sum of sets that have no common elements and whose union it is. Postulate (C*) requires that each cycle be a true sum of circuits.

With n=r+qn = r + qn=r+q, a family P1,…,PqP_1, \dots, P_qP1​,…,Pq​ is a strict fundamental set of circuits with respect to en−q+1,…,ene_{n-q+1}, \dots, e_nen−q+1​,…,en​ if q=n(M)q = n(M)q=n(M), each PiP_iPi​ is a circuit, and PiP_iPi​ contains en−q+ie_{n-q+i}en−q+i​ but no other en−q+je_{n-q+j}en−q+j​.

For a matrix M\mathbf MM over the integers mod 2 with columns C1,…,CnC_1, \dots, C_nC1​,…,Cn​, columns are independent (mod 2) if no non-null subset of them sums to the zero column. The matroid corresponding to M\mathbf MM has the column indices as elements and these independent sets.

Formalization targets

Goal: Theorem 37 (p. 533)

Let MMM satisfy (C*), with elements e1,…,ene_1, \dots, e_ne1​,…,en​ and base {e1,…,en−q}\{e_1, \dots, e_{n-q}\}{e1​,…,en−q​}. For every matrix M1\mathbf M_1M1​ mod 2 (any number of rows) whose n−qn - qn−q columns are independent mod 2,

∃! M=(M1∣Cn−q+1⋯Cn)  whose corresponding matroid is M.\exists!\ \mathbf M = (\mathbf M_1 \mid C_{n-q+1} \cdots C_n) \ \text{ whose corresponding matroid is } M.∃! M=(M1​∣Cn−q+1​⋯Cn​)  whose corresponding matroid is M.

Milestones

  1. Theorem 9 (p. 517): if e1,…,en−qe_1, \dots, e_{n-q}e1​,…,en−q​ is a base, there is a unique strict fundamental set of circuits with respect to en−q+1,…,ene_{n-q+1}, \dots, e_nen−q+1​,…,en​.
  2. Appendix, p. 531: (C*) implies the circuit postulate (C₂), for any family of sets.
  3. Theorem 33: under (C*), the circuits are exactly the minimal non-null cycles.
  4. Theorem 34: under (C*), the cycles are exactly the 2q2^q2q sums mod 2 of a strict fundamental set.
  5. Theorem 35: two (C*)-matroids with a common strict fundamental set have the same circuits.
  6. Theorem 36: any P1,…,PqP_1, \dots, P_qP1​,…,Pq​ with en−q+i∈Pi⊆{e1,…,en−q,en−q+i}e_{n-q+i} \in P_i \subseteq \{e_1, \dots, e_{n-q}, e_{n-q+i}\}en−q+i​∈Pi​⊆{e1​,…,en−q​,en−q+i​} is the strict fundamental set of exactly one (C*)-matroid.
  7. Appendix, p. 532: the matroid of a matrix mod 2 exists, satisfies (C*), and its cycles are the supports of the mod-2 dependencies among the columns.

Milestone 7 and the goal together characterize binary matroids as the matroids satisfying (C*).

Significance

The result. Theorem 37 and the p. 532 claim give an intrinsic, matrix-free description of the matroids representable over the two-element field, and Theorem 36 parametrizes all of them by qqq arbitrary subsets of a base. Uniqueness in Theorem 37 says that a binary representation is determined by the columns of one base; in modern terms, binary matroids are uniquely representable over GF(2) up to row operations. Every later theory of binary matroids, including graphic and cographic matroids, Tutte's excluded-minor theorem and Seymour's decomposition, starts from this equivalence.

Formalizing it. The results are proved in the paper and in textbooks (e.g. Oxley, Matroid Theory, Ch. 9) but, at the Mathlib revision used here, there is no notion of a matroid represented by a matrix over a field, and no binary-matroid theory. On Prove2Me, the existing binary objects (SeymourMFMC.Binary.*) are binary clutters defined through blockers, not matroids represented by mod-2 matrices. This mission produces the representation predicate for mod-2 matrices, the cycle space of a matroid, and the equivalence between (C*) and binary representability.

Difficulty

Writing down candidate columns is not the hard part; showing that the matroid of the completed matrix is MMM itself, and not merely a matroid sharing some of its circuits, is. Whitney's example at the end of §9 exhibits two different matroids with a common strict fundamental set, so agreement on fundamental circuits does not by itself identify a matroid; any argument must use (C*) on both the given matroid and the matroid of the matrix. A naive comparison of independent sets column by column does not close this gap. Uniqueness likewise depends on the independence mod 2 of the prescribed columns: without it, different completions can give the same matroid.

Formalization scope

  • Matroids are Mathlib Matroid (Fin (r + q)) with ground set Set.univ; Whitney's eke_kek​ is k - 1, his e1,…,en−qe_1, \dots, e_{n-q}e1​,…,en−q​ is the range of Fin.castAdd q, and en−q+ie_{n-q+i}en−q+i​ is Fin.natAdd r (i - 1). Writing n=r+qn = r + qn=r+q removes natural-number subtraction; qqq is not a free parameter, since {e1,…,er}\{e_1, \dots, e_r\}{e1​,…,er​} is required to be a base.
  • Sums mod 2 count parity of membership (sumMod2); cycles are sums over finite sets of circuits; true sums are unions over finite pairwise-disjoint sets of circuits; (C*) is SatisfiesCStar on the circuit family {C | M.IsCircuit C}. These definitions take the circuit family as a parameter, so that the (C₂) milestone is posed for an arbitrary family of sets, as Whitney poses it.
  • A strict fundamental set includes the nullity condition r(M)+q=ρ(M)r(M) + q = \rho(M)r(M)+q=ρ(M), stated in N∞\mathbb N_\inftyN∞​.
  • Matrices are Matrix (Fin m) (Fin n) (ZMod 2) with any mmm; independence mod 2 of columns is LinearIndepOn (ZMod 2) of the columns (the rows of the transpose). IsMatroidOf M A compares all independent sets, not only bases.
  • Ruled out: the goal is not satisfied by any statement that compares only the bases of one size, by an existence-only statement without uniqueness, or by real (instead of mod-2) independence.
  • Tacit hypotheses made explicit: the matroid's ground set is exactly e1,…,ene_1, \dots, e_ne1​,…,en​ (ρ(M)=n\rho(M) = nρ(M)=n); the elements and matroids are finite.

Contributions welcome: the general fact that the matroid of a vector family over a field exists (a reusable Matroid.ofFun-style construction over any field), the cycle-space lemmas, and proofs of the milestones in any order.

Selected references

  • H. Whitney, On the Abstract Properties of Linear Dependence, American Journal of Mathematics 57 (1935), 509–533. https://doi.org/10.2307/2371182
  • W. T. Tutte, A homotopy theorem for matroids, I, II, Transactions of the AMS 88 (1958), 144–174. https://doi.org/10.2307/1993244
  • P. D. Seymour, The matroids with the max-flow min-cut property, Journal of Combinatorial Theory Ser. B 23 (1977), 189–222. https://doi.org/10.1016/0095-8956(77)90031-4
  • P. D. Seymour, Decomposition of regular matroids, Journal of Combinatorial Theory Ser. B 28 (1980), 305–359. https://doi.org/10.1016/0095-8956(80)90075-1
  • J. Oxley, Matroid Theory, 2nd ed., Oxford University Press, 2011. https://doi.org/10.1093/acprof:oso/9780198566946.001.0001
11 thms2 active usersReviewed
Markov ChainProbabilityStochastic Systems·Captain: mikedeng1

On the Stochastic Matrices Associated with Certain Queuing Processes 2: The GI/M/1 Imbedded Chain Is Ergodic iff ρ < 1 and Recurrent iff ρ ≤ 1Research Paper

Motivation

A single-server queue in which customers arrive according to a renewal process and are served in exponentially distributed times is the system GI/M/1. Observed just before successive arrivals, its queue length is a Markov chain on {0,1,2,… }\{0, 1, 2, \dots\}{0,1,2,…}, the imbedded chain introduced by D. G. Kendall (Kendall 1953, Ann. Math. Statist. 24, pp. 338–354). Whether this chain settles into a statistical equilibrium, keeps returning to the empty state without one, or drifts off to infinity is the first question asked about the queue, and every later quantity (stationary queue lengths, waiting-time distributions) presupposes the answer.

F. G. Foster's 1953 paper (Foster 1953) answers it for GI/M/1 and for M/G/1 by a different route from Kendall's direct analysis: it first proves general criteria, stated in terms of solutions of linear equations and inequalities in the transition matrix, for a countable Markov chain to be ergodic, recurrent or transient, and then checks them on the two queueing matrices. The criteria are of independent use; one of them (Theorem 2 of the paper) is now known as Foster's criterion, the starting point of the drift (Lyapunov-function) method for stability of Markov chains and queueing networks.

Timeline. Kendall (1951, J. Roy. Statist. Soc. B 13) studied queue-length processes directly, including a recurrence argument for M/G/1 that Foster's §3 reproduces; Kendall (1953) introduced the imbedded-chain method and, for GI/M/1, proved by it that ρ<1\rho < 1ρ<1 is sufficient for ergodicity (Foster 1953, p. 359); Foster (1953) proved the full classification, ergodic iff ρ<1\rho < 1ρ<1 and recurrent iff ρ≤1\rho \le 1ρ≤1, by the general criteria. This mission treats the GI/M/1 half; a companion mission treats M/G/1.

Setting

A transition matrix on the states {0,1,2,… }\{0, 1, 2, \dots\}{0,1,2,…} is an array [pij][p_{ij}][pij​] of nonnegative reals whose rows sum to 111. For a state jjj, fjjf_{jj}fjj​ is the probability that the chain started at jjj returns to jjj at some later step. The chain is recurrent if fjj=1f_{jj} = 1fjj​=1 for every jjj, transient if fjj<1f_{jj} < 1fjj​<1 for every jjj, and ergodic (recurrent-nonnull, positive recurrent) if moreover every mean recurrence time ∑nnfjj(n)\sum_n n f^{(n)}_{jj}∑n​nfjj(n)​ is finite. Foster's general theorems concern an irreducible chain (every state reachable from every state), assumed aperiodic for simplicity.

The GI/M/1 chain is described by a sequence a=(an)n≥0a = (a_n)_{n \ge 0}a=(an​)n≥0​ of positive numbers with ∑nan=1\sum_n a_n = 1∑n​an​=1: ana_nan​ is the probability that exactly nnn services are completed between two arrivals. With the tails αi=∑j≥i+1aj\alpha_i = \sum_{j \ge i+1} a_jαi​=∑j≥i+1​aj​,

[pij]=[α0a000⋯α1a1a00⋯α2a2a1a0⋯⋮⋮⋮⋮],[p_{ij}] = \begin{bmatrix} \alpha_0 & a_0 & 0 & 0 & \cdots \\ \alpha_1 & a_1 & a_0 & 0 & \cdots \\ \alpha_2 & a_2 & a_1 & a_0 & \cdots \\ \vdots & \vdots & \vdots & \vdots & \end{bmatrix},[pij​]=​α0​α1​α2​⋮​a0​a1​a2​⋮​0a0​a1​⋮​00a0​⋮​⋯⋯⋯​​,

that is pi0=αip_{i0} = \alpha_ipi0​=αi​, pij=ai+1−jp_{ij} = a_{i+1-j}pij​=ai+1−j​ for 1≤j≤i+11 \le j \le i+11≤j≤i+1, and pij=0p_{ij} = 0pij​=0 for j>i+1j > i+1j>i+1. In Lean this matrix is gim1Matrix a. The traffic parameter ρ\rhoρ is defined through its inverse,

ρ−1=∑n=1∞n an∈(0,∞],\rho^{-1} = \sum_{n=1}^{\infty} n\, a_n \in (0, \infty],ρ−1=n=1∑∞​nan​∈(0,∞],

the mean number of service completions per interarrival interval (rhoInv a, and rho a =ρ= \rho=ρ).

Formalization targets

Goal: the classification of GI/M/1 (§4, p. 359)

the chain is ergodic  ⟺  ρ<1,the chain is recurrent  ⟺  ρ≤1.\text{the chain is ergodic} \iff \rho < 1, \qquad \text{the chain is recurrent} \iff \rho \le 1 .the chain is ergodic⟺ρ<1,the chain is recurrent⟺ρ≤1.

Together: ergodic for ρ<1\rho < 1ρ<1, recurrent-null for ρ=1\rho = 1ρ=1, transient for ρ>1\rho > 1ρ>1. The statement carries no constants and leaves the sequence aaa free apart from positivity and normalization.

Milestones

  1. Theorem 7 (p. 358): for a probability distribution {pn}\{p_n\}{pn​} with p0>0p_0 > 0p0​>0, the equation ∑n≥0znpn=z\sum_{n \ge 0} z^n p_n = z∑n≥0​znpn​=z has a root in (0,1)(0, 1)(0,1) iff ∑n≥1npn>1\sum_{n\ge1} n p_n > 1∑n≥1​npn​>1.
  2. Theorem 1, sufficiency (p. 355): a nonnull solution of ∑ixipij=xj\sum_i x_i p_{ij} = x_j∑i​xi​pij​=xj​ with ∑i∣xi∣<∞\sum_i |x_i| < \infty∑i​∣xi​∣<∞ makes the system ergodic.
  3. Theorem 1, necessity (p. 355): in an ergodic system every nonnegative solution of ∑ixipij≤xj\sum_i x_i p_{ij} \le x_j∑i​xi​pij​≤xj​ has ∑ixi<∞\sum_i x_i < \infty∑i​xi​<∞.
  4. Theorem 4 (pp. 356–357): the system is transient iff ∑jpijyj=yi\sum_j p_{ij} y_j = y_i∑j​pij​yj​=yi​ (i≠0i \ne 0i=0) has a bounded nonconstant solution.

Milestones 2–4 are stated for a general irreducible aperiodic chain.

Significance

The classification tells exactly when the GI/M/1 queue is stable: the stationary distribution of the imbedded chain, which is geometric, exists precisely in the ergodic case ρ<1\rho < 1ρ<1, and for ρ>1\rho > 1ρ>1 the queue grows without bound. Theorems 1 and 4 are general tools, reusable for any countable chain: Theorem 1 characterizes ergodicity by summable invariant vectors, Theorem 4 characterizes transience by bounded harmonic functions off one state. Theorem 7 is the extinction criterion of branching processes and recurs throughout applied probability.

All of these results are proved in the literature (Foster 1953; Feller's textbook for Theorem 7 and a version of Theorem 4). As far as a search of the platform shows, none of them has a machine-checked proof; the platform holds related special cases for the G/M/1 queue with a specific interarrival law (QueueingFundamentals.GM1.unique_root_unit_interval, open), but not the general lemma or the classification. A formalization would provide the general criteria as reusable library results and the first verified stability classification of a non-Markovian queue's imbedded chain.

Difficulty

The matrix is explicit, but none of the three properties is a finite computation: ergodicity and recurrence are statements about return times over all horizons, so each direction must go through an existence or nonexistence statement about infinite systems of equations. For the converse directions the obvious argument fails: exhibiting a candidate solution such as xi≡1x_i \equiv 1xi​≡1 shows nothing until it is known that ergodicity forces every such solution to be summable, and showing that no bounded nonconstant solution of (7) exists when ρ<1\rho < 1ρ<1 requires control of all solutions, not of one. The general criteria themselves rest on limit theorems for pij(n)p_{ij}^{(n)}pij(n)​ and on interchanging infinite sums, and the infinite-mean case ∑nan=∞\sum n a_n = \infty∑nan​=∞ has to be carried along everywhere.

Formalization scope

  • The Markov-chain vocabulary is the published definition QueueingFundamentals_Foundations_MarkovChain: TransitionMatrix (entries p, nonnegativity, rows summing to 111 via HasSum), returnProb, meanRecurrenceTime, Irreducible, Aperiodic, PositiveRecurrent. "Ergodic" is PositiveRecurrent. IsRecurrent and IsTransient are defined state by state from returnProb; their complementarity for irreducible chains is a theorem, not a definition.
  • States are indexed from 000, as in the paper. The goal quantifies over every TransitionMatrix whose entries equal gim1Matrix a; such a matrix exists for every admissible aaa (rows sum to 111), so the statement is not vacuous.
  • ρ−1\rho^{-1}ρ−1 and ρ\rhoρ live in [0,∞][0, \infty][0,∞] (ℝ≥0∞), with ∞−1=0\infty^{-1} = 0∞−1=0: an infinite mean gives ρ=0\rho = 0ρ=0, and that chain is ergodic.
  • The goal does not assume irreducibility or aperiodicity: they follow from an>0a_n > 0an​>0. Milestones 2–4 carry them, as the paper's standing assumptions (§1).
  • Every infinite series appearing in a hypothesis is required to converge (HasSum or Summable), so that a divergent series cannot satisfy an equation or inequality vacuously. In Theorem 1's sufficiency half the xix_ixi​ may be of either sign. In Theorem 7 the distribution is renamed qqq to avoid a clash with pijp_{ij}pij​.
  • Ruled out as trivializing: defining ρ\rhoρ by a real inverse of a real series, defining "ergodic" as the existence of a summable invariant vector (which is Theorem 1's condition), or stating the goal over a matrix that need not exist.
  • Not included: the paper's explicit description of the solutions of (7) for ρ≥1\rho \ge 1ρ≥1 via the generating function (1−z){A(z)−z}−1(1 - z)\{A(z) - z\}^{-1}(1−z){A(z)−z}−1, and the M/G/1 half (Theorems 2, 3, 5), which is the companion mission. Contributions welcome: proofs of the general criteria (reusable for any countable chain), of Theorem 7, and lemmas on the GI/M/1 matrix such as irreducibility and aperiodicity.

Selected references

  • F. G. Foster, On the stochastic matrices associated with certain queuing processes, Ann. Math. Statist. 24 (1953), 355–360. https://doi.org/10.1214/aoms/1177728976
  • D. G. Kendall, Stochastic processes occurring in the theory of queues and their analysis by the method of the imbedded Markov chain, Ann. Math. Statist. 24 (1953), 338–354 (the paper immediately preceding Foster's in the same issue).
  • D. G. Kendall, Some problems in the theory of queues, J. Roy. Statist. Soc. B 13 (1951), 151–185.
  • W. Feller, An Introduction to Probability Theory and Its Applications, Vol. 1, Wiley, 1950.
8 thms2 active usersReviewed
🏆Completed
Combinatorics·Captain: mikedeng1

On the Abstract Properties of Linear Dependence 3: Two Elements Share a Component Iff Some Circuit Contains BothResearch Paper

Motivation

Hassler Whitney's 1935 paper On the Abstract Properties of Linear Dependence introduced matroids: finite sets of elements carrying an abstract rank function that behaves like the rank of a set of vectors. Part II of the paper opens with the decomposition of a matroid into components. The question it answers is basic to every later use of matroids: when does a matroid split into independent pieces, and how can the pieces be recognized?

For the matroid of a graph (elements = edges, rank = number of vertices minus number of connected pieces spanned) the components are the 2-connected blocks of the graph, and Whitney's theorem recovers the classical fact that two edges lie in a common block exactly when they lie on a common cycle. Whitney had studied separability of graphs in Non-separable and planar graphs (1932), and footnote 11 of the 1935 paper points out that the theorem identifies König's "Glieder" of a graph with components. Matroid connectivity built on this notion runs through later structure theory: Tutte's higher connectivity, Seymour's decomposition of regular matroids, and the matroid minors project all start from the separation of a matroid into components.

Setting

A matroid MMM on a finite ground set EEE is given here by Mathlib's Matroid structure, with rank function r(X)r(X)r(X) for X⊆EX\subseteq EX⊆E (Mathlib's M.eRk X) and circuits, the minimal dependent sets (M.IsCircuit). Whitney treats every subset X⊆EX\subseteq EX⊆E as a matroid in its own right, a submatroid, with the rank function of MMM restricted to subsets of XXX. For sets he writes M1+M2M_1+M_2M1​+M2​ for the union, ρ(N)\rho(N)ρ(N) for the number of elements of NNN, and

n(N)=ρ(N)−r(N)n(N) = \rho(N) - r(N)n(N)=ρ(N)−r(N)

for the nullity of NNN.

Rank is subadditive: r(X1+X2)≤r(X1)+r(X2)r(X_1+X_2)\le r(X_1)+r(X_2)r(X1​+X2​)≤r(X1​)+r(X2​). A submatroid XXX is separable if it can be divided into two disjoint groups X1,X2X_1, X_2X1​,X2​, each containing at least one element, with

r(X)=r(X1)+r(X2),r(X) = r(X_1) + r(X_2),r(X)=r(X1​)+r(X2​),

and non-separable otherwise. Every single element is non-separable. A component of MMM is a maximal non-separable part of MMM: a nonempty non-separable set K⊆EK\subseteq EK⊆E contained in no strictly larger non-separable subset of EEE.

Formalization targets

Goal: Theorem 19

For two distinct elements e1≠e2e_1\neq e_2e1​=e2​ of EEE,

(∃K component of M: e1,e2∈K)  ⟺  (∃P circuit of M: e1,e2∈P).\bigl(\exists K \text{ component of } M:\ e_1, e_2\in K\bigr) \iff \bigl(\exists P \text{ circuit of } M:\ e_1, e_2\in P\bigr).(∃K component of M: e1​,e2​∈K)⟺(∃P circuit of M: e1​,e2​∈P).

Components are defined by the rank function, circuits by dependence; the goal asserts that the two descriptions agree.

Milestones (§10, in the paper's order)

  • Theorem 11. If r(M1+M2)=r(M1)+r(M2)r(M_1+M_2)=r(M_1)+r(M_2)r(M1​+M2​)=r(M1​)+r(M2​), M1′⊆M1M_1'\subseteq M_1M1′​⊆M1​ and M2′⊆M2M_2'\subseteq M_2M2′​⊆M2​, then r(M1′+M2′)=r(M1′)+r(M2′)r(M_1'+M_2')=r(M_1')+r(M_2')r(M1′​+M2′​)=r(M1′​)+r(M2′​).
  • Theorem 12. Under the same rank additivity, a non-separable M′⊆M1+M2M'\subseteq M_1+M_2M′⊆M1​+M2​ lies in M1M_1M1​ or in M2M_2M2​.
  • Theorem 13. Two non-separable sets with a common element have a non-separable union.
  • Theorem 14. Distinct components are disjoint.
  • Theorem 15. The components cover EEE, and no other family of components does.
  • Theorem 16. A set is non-separable of nullity 111 if and only if it is a circuit.
  • Lemma 9. If M1+M2M_1+M_2M1​+M2​ is non-separable, with M1,M2M_1, M_2M1​,M2​ nonempty and disjoint, some circuit inside M1+M2M_1+M_2M1​+M2​ meets both.
  • Theorem 17. A non-separable set of nullity n>0n>0n>0 is built from a circuit by n−1n-1n−1 steps, each adding a set of elements that forms a circuit with elements already present, through non-separable sets of nullity 1,2,…,n1,2,\dots,n1,2,…,n.
  • Theorem 18. For distinct nonempty non-separable M1,…,MpM_1,\dots,M_pM1​,…,Mp​ covering EEE, the following are equivalent: they are the components; they are pairwise disjoint and no circuit meets two of them; r(E)=∑ir(Mi)r(E)=\sum_i r(M_i)r(E)=∑i​r(Mi​).

Significance

The result. Theorem 19 makes the component decomposition computable from circuits alone and shows that "lying on a common circuit" is an equivalence relation on distinct elements, a fact that is not evident from the circuit axioms. Theorem 18 adds that the decomposition is the unique one with additive rank. Together they are the starting point of matroid connectivity: the direct-sum decomposition of a matroid, the reduction of many matroid problems (representability, duality of components, Whitney's own Theorems 24–26 on duals of components) to the connected case, and the higher-connectivity theory that followed.

Formalizing it. The results are classical and proved in the paper; nothing here is open. To our knowledge Mathlib at the pinned revision has no notion of matroid connectivity or components, so this mission produces the first machine-checked development of Whitney's §10: the rank-based definition of separability, the disjoint decomposition into components, the circuit characterization, and the ear-type construction of non-separable matroids (Theorem 17). These are reusable for any later formalization of matroid connectivity, including Whitney's results on duals of components.

Difficulty

The two directions of Theorem 19 rest on different machinery. That two elements on a common circuit lie in one component follows from the rank theory (Theorems 13 and 16). The converse is the substantial direction: a component is defined by the failure of rank additivity, which only says that every division of the component is crossed by some circuit (Lemma 9). It does not directly give one circuit through two prescribed elements. Combining circuits that cross different divisions into a single circuit through both e1e_1e1​ and e2e_2e2​ requires the circuit elimination property together with a minimality argument over subsets of the component; the naive attempt of chaining overlapping circuits from e1e_1e1​ to e2e_2e2​ gives a connected chain of circuits, not one circuit.

Formalization scope

  • Representation. A matroid is Mathlib's Matroid α with [M.Finite]; Whitney's matroids are finite. A submatroid is a subset X⊆X\subseteqX⊆ M.E with the rank M.eRk restricted to its subsets; results that Whitney states for "a matroid M=M1+M2M = M_1 + M_2M=M1​+M2​" are stated for subsets of an ambient finite matroid, which is the same statement applied to the submatroid M1+M2M_1+M_2M1​+M2​.
  • Ranks are Mathlib's ℕ∞-valued M.eRk, finite on a finite matroid, so (10.1) is an equation of natural numbers. Nullity is computed in Z\mathbb ZZ as the number of elements minus the rank.
  • Definitions. IsSeparable M X requires two nonempty, disjoint groups with union XXX and additive rank; without nonemptiness every set would be separable. IsNonSeparable M X adds X⊆X\subseteqX⊆ M.E. IsComponent M K requires KKK nonempty, non-separable, and maximal; nonemptiness excludes the empty set, which is vacuously non-separable.
  • Tacit hypotheses made explicit. In Theorem 19 the two elements are distinct: for e1=e2e_1=e_2e1​=e2​ a coloop is its own component and lies on no circuit. In Theorem 18 the sets M1,…,MpM_1,\dots,M_pM1​,…,Mp​ are distinct and nonempty: a loop listed twice would satisfy (3) but not (2), and an empty set would satisfy (2) and (3) but not (1). In Theorems 11 and 12 the two parts need not be disjoint, as Whitney's use of M1+M2M_1+M_2M1​+M2​ for overlapping sets in Theorem 13 indicates; the statements hold in that generality.
  • Ruled out. Components must not be defined as the classes of the relation "lie on a common circuit": that would make the goal a tautology. Here they are the rank-defined maximal non-separable sets of §10, and circuits are Mathlib's Matroid.IsCircuit.
  • Infrastructure. Solvers will need submodularity of M.eRk and circuit elimination (both in Mathlib), the relation between circuits of M ↾ X and circuits of M inside XXX (Matroid.restrict_isCircuit_iff), and finiteness arguments for maximal non-separable sets. Proofs of the milestones, alternative proofs of the goal, and lemmas relating components to Mathlib's direct sums of matroids are all welcome.

Selected references

  • H. Whitney, On the Abstract Properties of Linear Dependence, American Journal of Mathematics 57 (1935), 509–533. https://doi.org/10.2307/2371182
  • H. Whitney, Non-separable and planar graphs, Transactions of the American Mathematical Society 34 (1932), 339–362. https://doi.org/10.1090/S0002-9947-1932-1501641-2
  • D. König, Acta Litterarum ac Scientiarum Szeged, vol. 6, pp. 155–179, as cited by Whitney in footnote 11 (p. 159 for the notion of "Glied").
  • J. Oxley, Matroid Theory, 2nd ed., Oxford University Press, 2011, Chapter 4 (connectivity).
13 thms2 active usersReviewed
Control TheoryDynamic ProgrammingProbability+1·Captain: mikedeng1

Stochastic Optimal Control: The Discrete-Time Case VII: The Finite-Horizon Borel Model — over Universally Measurable Policies J*_K = T^K(J_0)Textbook

Motivation

Finite-horizon stochastic control on general state and control spaces — inventory levels, queue lengths, positions, beliefs — cannot be written down without measure theory, and the measure theory turns out to be the hard part. The dynamic programming (DP) recursion "start from zero and minimise one stage at a time" is easy to state, but on uncountable spaces the minimisation in each step produces functions that need not be Borel-measurable, and the infimum over policies has to be taken over a class large enough to contain near-minimisers. Bertsekas and Shreve's Stochastic Optimal Control: The Discrete-Time Case (1978; Athena reprint 1996) resolved this by working with universally measurable policies and lower semianalytic costs. Chapter 8 is the finite-horizon core of that theory, and the infinite-horizon results of Chapter 9 and the imperfect-information reduction of Chapter 10 are built on it.

Timeline. Blackwell (1965) treated discounted problems on Borel spaces with bounded costs and Borel policies, where ε-optimal Borel policies may fail to exist. Strauch (1966) studied positive and negative models. Blackwell, Freedman and Orkin (1974) introduced analytic sets and analytically measurable policies into DP. Bertsekas and Shreve (1978, Chapters 7–8) gave the universally measurable finite-horizon theory formalized here, including the unbounded-cost assumptions (F⁺)/(F⁻).

Setting

A finite horizon stochastic optimal control model is a nine-tuple (S,C,U,W,p,f,α,g,N)(S,C,U,W,p,f,\alpha,g,N)(S,C,U,W,p,f,α,g,N). The state space SSS, control space CCC and disturbance space WWW are nonempty Borel spaces (topological spaces homeomorphic to Borel subsets of complete separable metric spaces). The constraint U(x)⊆CU(x)\subseteq CU(x)⊆C is nonempty and Γ={(x,u)∣u∈U(x)}\Gamma=\{(x,u)\mid u\in U(x)\}Γ={(x,u)∣u∈U(x)} is analytic in S×CS\times CS×C. Disturbances are drawn from a Borel stochastic kernel p(dw∣x,u)p(dw\mid x,u)p(dw∣x,u), the system moves by xk+1=f(xk,uk,wk)x_{k+1}=f(x_k,u_k,w_k)xk+1​=f(xk​,uk​,wk​) with fff Borel, the discount factor α\alphaα is a positive real, the one-stage cost g:Γ→[−∞,∞]g:\Gamma\to[-\infty,\infty]g:Γ→[−∞,∞] is lower semianalytic ({g<c}\{g<c\}{g<c} is analytic for every real ccc), and N≥1N\ge1N≥1 is the horizon. The state transition kernel is t(B∣x,u)=p({w∣f(x,u,w)∈B}∣x,u)t(B\mid x,u)=p(\{w\mid f(x,u,w)\in B\}\mid x,u)t(B∣x,u)=p({w∣f(x,u,w)∈B}∣x,u).

A set is universally measurable if it is measurable for the completion of the Borel σ-algebra under every probability measure. A policy π=(μ0,…,μN−1)\pi=(\mu_0,\dots,\mu_{N-1})π=(μ0​,…,μN−1​) chooses uku_kuk​ from a universally measurable stochastic kernel μk(duk∣x0,u0,…,xk)\mu_k(du_k\mid x_0,u_0,\dots,x_k)μk​(duk​∣x0​,u0​,…,xk​) concentrated on U(xk)U(x_k)U(xk​); it is Markov if μk\mu_kμk​ depends on xkx_kxk​ only, and nonrandomized if every μk(⋅∣⋅)\mu_k(\cdot\mid\cdot)μk​(⋅∣⋅) is a point mass. Π′\Pi'Π′ denotes all policies and Π\PiΠ the Markov ones. A policy and an initial distribution ppp determine a probability measure rN(π,p)r_N(\pi,p)rN​(π,p) on state–control paths, and the KKK-stage cost and optimal cost are

JK,π(x)=∫[∑k=0K−1αkg(xk,uk)]drN(π,px),JK∗(x)=inf⁡π∈Π′JK,π(x).J_{K,\pi}(x)=\int\Big[\sum_{k=0}^{K-1}\alpha^k g(x_k,u_k)\Big]dr_N(\pi,p_x),\qquad J^*_K(x)=\inf_{\pi\in\Pi'}J_{K,\pi}(x).JK,π​(x)=∫[k=0∑K−1​αkg(xk​,uk​)]drN​(π,px​),JK∗​(x)=π∈Π′inf​JK,π​(x).

Assumption (F⁺) requires ∫g− dqk(π,px)<∞\int g^-\,dq_k(\pi,p_x)<\infty∫g−dqk​(π,px​)<∞, and (F⁻) requires ∫g+ dqk(π,px)<∞\int g^+\,dq_k(\pi,p_x)<\infty∫g+dqk​(π,px​)<∞, for every policy, initial state and stage, where qkq_kqk​ is the marginal of rNr_NrN​ on the kkk-th pair. The DP operators are

Tμ(J)(x)=∫C[g(x,u)+α ⁣∫SJ dt(⋅∣x,u)]μ(du∣x),T(J)(x)=inf⁡u∈U(x){g(x,u)+α ⁣∫SJ dt(⋅∣x,u)}.T_\mu(J)(x)=\int_C\Big[g(x,u)+\alpha\!\int_S J\,dt(\cdot\mid x,u)\Big]\mu(du\mid x),\qquad T(J)(x)=\inf_{u\in U(x)}\Big\{g(x,u)+\alpha\!\int_S J\,dt(\cdot\mid x,u)\Big\}.Tμ​(J)(x)=∫C​[g(x,u)+α∫S​Jdt(⋅∣x,u)]μ(du∣x),T(J)(x)=u∈U(x)inf​{g(x,u)+α∫S​Jdt(⋅∣x,u)}.

Formalization targets

Goal: Proposition 8.2

JK∗=TK(J0),K=1,…,N,J^*_K=T^K(J_0),\qquad K=1,\dots,N,JK∗​=TK(J0​),K=1,…,N,

under (F⁺) or (F⁻), where J0≡0J_0\equiv0J0​≡0. The goal leaves the horizon, the discount factor and the sign of ggg unrestricted beyond (F⁺)/(F⁻), and it compares an infimum over all history-dependent randomized policies with a pointwise recursion.

Milestones

In attack order: Lemma 8.1 (the cost of a Markov policy equals Tμ0⋯TμK−1(J0)T_{\mu_0}\cdots T_{\mu_{K-1}}(J_0)Tμ0​​⋯TμK−1​​(J0​)), Proposition 8.1 and Corollary 8.1.1 (Markov policies suffice), Lemma 8.2 (an ε-optimal universally measurable kernel for one application of TTT), Lemma 8.3 (under (F⁺), TK(J0)>−∞T^K(J_0)>-\inftyTK(J0​)>−∞), Lemma 8.4 (monotone and bounded convergence for TμT_\muTμ​). Two consequences of the goal complete the chapter's existence theory: Corollary 8.2.1 (JK∗J^*_KJK∗​ is lower semianalytic) and Proposition 8.3 (ε-optimal nonrandomized Markov policies under (F⁺); nonrandomized semi-Markov and randomized Markov ones under (F⁻)).

Significance

Proposition 8.2 says the DP algorithm computes the true optimal cost of the Borel model, with no restriction to Markov or nonrandomized policies and with costs that may be unbounded in either direction. Corollary 8.2.1 identifies the regularity of the value function — lower semianalytic, possibly not Borel (Example 1 of Chapter 8) — and Proposition 8.3 turns the recursion into near-optimal policies. These results are the base case for the infinite-horizon theory of Chapter 9 (positive, negative and discounted models are analysed as limits of finite-horizon problems) and for the sufficient-statistic reduction of Chapter 10.

All results here are proved in the book. None is formalized: the platform has finite-state, finite-action DP theorems and Borel models with Borel-measurable policies, but no universally measurable policies, no lower semianalytic costs and no Ionescu-Tulcea construction for universally measurable kernels. The mission poses the finite-horizon Borel theory, with reusable infrastructure: the universal σ-algebra, universally measurable kernels, iterated path integrals representing integration against the induced path measure, and the operator calculus on extended-real functions with the convention ∞−∞=∞\infty-\infty=\infty∞−∞=∞.

Difficulty

The obvious argument fails at measurability. On countable spaces, Proposition 8.2 follows from the Part I argument: induct on KKK, choose near-minimising controls state by state, assemble them into a policy. On Borel spaces, a pointwise choice of near-minimisers is not a policy unless it is measurable, and T(J)T(J)T(J) is generally not Borel even when JJJ and ggg are; Borel policies are too few for ε\varepsilonε-optimal ones to exist. The book's way out needs the selection theorem for lower semianalytic functions (Proposition 7.50), integration of universally measurable functions against universally measurable kernels (Propositions 7.45–7.46), and care with infinite values: without (F⁺) or (F⁻), the integral of the stage sum and the sum of the stage integrals can disagree, and Lemma 8.1 fails.

Formalization scope

Lean conventions:

  • Spaces carry [TopologicalSpace X] [MeasurableSpace X] [BorelSpace X] [IsBorelSpace X] [Nonempty X]. IsBorelSpace is the book's Definition 7.7.
  • Extended reals are EReal. Addition inside costs and integrands uses the book's convention −∞+∞=+∞-\infty+\infty=+\infty−∞+∞=+∞ (badd), not Mathlib's, which gives ⊥+⊤=⊥\bot+\top=\bot⊥+⊤=⊥. Integrals are ∫f+−∫f−\int f^+-\int f^-∫f+−∫f− with ∞−∞=∞\infty-\infty=\infty∞−∞=∞ (extInt).
  • The universal σ-algebra is the intersection of all completions (universalSigma). Kernels are universally measurable in the sense of Lemma 7.28(b).
  • The integral against rN(π,p)r_N(\pi,p)rN​(π,p) is the iterated integral of Eq. (4) of Chapter 8 (pathInt).
  • Stages are indexed 0,…,N−10,\dots,N-10,…,N−1, and a history is kkk state–control pairs plus the current state.
  • ggg is stored on S×CS\times CS×C, but only its values on Γ\GammaΓ are constrained or used.

A trivializing formalization is excluded by construction. JK∗J^*_KJK∗​ is the infimum over all policies in Π′\Pi'Π′, not over Markov or nonrandomized ones. JK,πJ_{K,\pi}JK,π​ is the integral of the stage sum against the path measure, never the operator composition of Lemma 8.1, so the goal does not collapse to the Part I result.

A complete development needs:

  • the analytic-set and universal-measurability theory of §7.6–7.7: closure of analytic sets under projections and sections, measurability of integrals against universally measurable kernels (Proposition 7.46), and the selection theorem (Proposition 7.50);
  • extended-real integration lemmas in the style of Lemma 7.11.

This infrastructure is reusable for the infinite-horizon Borel models (Chapter 9), for the imperfect-information reduction (Chapter 10), and for papers that cite this book. Contributions of these supporting lemmas as separate theorems are welcome.

Selected references

  • D. P. Bertsekas and S. E. Shreve, Stochastic Optimal Control: The Discrete-Time Case, Academic Press, 1978; Athena Scientific reprint, 1996, Chapter 8. https://web.mit.edu/dimitrib/www/soc.html
  • D. Blackwell, Discounted dynamic programming, Ann. Math. Statist. 36 (1965), 226–235. https://doi.org/10.1214/aoms/1177700285
  • R. E. Strauch, Negative dynamic programming, Ann. Math. Statist. 37 (1966), 871–890. https://doi.org/10.1214/aoms/1177699369
  • D. Blackwell, D. Freedman and M. Orkin, The optimal reward operator in dynamic programming, Ann. Probab. 2 (1974), 926–941. https://doi.org/10.1214/aop/1176996558
  • S. E. Shreve and D. P. Bertsekas, Universally measurable policies in dynamic programming, Math. Oper. Res. 4 (1979), 15–30. https://doi.org/10.1287/moor.4.1.15
14 thms2 active usersReviewed
🏆Completed
Combinatorics·Captain: mikedeng1

On the Abstract Properties of Linear Dependence 1: The Rank Postulates and the Independence Postulates Are EquivalentResearch Paper

Motivation

In 1935 Hassler Whitney asked which properties of linear dependence among the columns of a matrix can be stated without reference to the matrix at all. His answer, On the Abstract Properties of Linear Dependence (American Journal of Mathematics 57, 1935), introduced the matroid: a finite set together with a rank function obeying three short postulates. The notion now underlies combinatorial optimization (the greedy algorithm is optimal exactly on matroids, and matroid intersection and partition generalize bipartite matching and arborescence packing), graph theory (graphic and cographic matroids), coding theory and the study of linear representations over finite fields.

A defining feature of the subject is that the same structure can be axiomatized in several apparently unrelated ways: by rank, by independent sets, by bases, by circuits. Each axiom system is convenient for different arguments, and passing between them, a so-called cryptomorphism, is routine in practice. Whitney's paper is where these equivalences first appear. Part I, §§2–4 and §6 (pp. 510–514), derives the basic properties of rank from the rank postulates, deduces from them the postulates for independent sets, and shows that the two systems are equivalent. This mission formalizes that first equivalence.

Setting

Let MMM be a finite set of elements e1,…,ene_1, \dots, e_ne1​,…,en​. Following Whitney, write N+eN + eN+e for N∪{e}N \cup \{e\}N∪{e}, M1+M2M_1 + M_2M1​+M2​ for the union and M1M2M_1 M_2M1​M2​ for the intersection of subsets; ρ(N)\rho(N)ρ(N) is the number of elements of NNN.

A rank system is a function rrr on the subsets of MMM satisfying

  • (R₁) r(∅)=0r(\emptyset) = 0r(∅)=0;
  • (R₂) for every subset NNN and element e∉Ne \notin Ne∈/N, r(N+e)=r(N)r(N + e) = r(N)r(N+e)=r(N) or r(N+e)=r(N)+1r(N + e) = r(N) + 1r(N+e)=r(N)+1;
  • (R₃) for every subset NNN and elements e1,e2∉Ne_1, e_2 \notin Ne1​,e2​∈/N, if r(N+e1)=r(N+e2)=r(N)r(N + e_1) = r(N + e_2) = r(N)r(N+e1​)=r(N+e2​)=r(N) then r(N+e1+e2)=r(N)r(N + e_1 + e_2) = r(N)r(N+e1​+e2​)=r(N).

The nullity of NNN is n(N)=ρ(N)−r(N)n(N) = \rho(N) - r(N)n(N)=ρ(N)−r(N), and NNN is independent when n(N)=0n(N) = 0n(N)=0. The increment of (3.1) is Δ(M′,N)=r(M′+N)−r(M′)\Delta(M', N) = r(M' + N) - r(M')Δ(M′,N)=r(M′+N)−r(M′), written Δ(M′,e)\Delta(M', e)Δ(M′,e) when N={e}N = \{e\}N={e}.

An independence system is a predicate "independent" on the subsets of MMM satisfying

  • (I₁) any subset of an independent set is independent;
  • (I₂) if NNN and N′N'N′ are independent and N′N'N′ has exactly one element more than NNN, then N+e′N + e'N+e′ is independent for some e′∈N′e' \in N'e′∈N′ with e′∉Ne' \notin Ne′∈/N.

From an independence system one recovers a rank by letting r(N)r(N)r(N) be the number of elements in a largest independent subset of NNN. In the Lean development these objects are IsRankSystem, nullity, Delta, indepOfRank, IsIndepSystem and rankOfIndep, all in the namespace WhitneyMatroid.RankIndep.

Formalization targets

Goal: (R) and (I) are equivalent (§6, p. 514)

  1. If rrr satisfies (R₁)–(R₃), then {N:ρ(N)=r(N)}\{N : \rho(N) = r(N)\}{N:ρ(N)=r(N)} satisfies (I₁), (I₂), contains ∅\emptyset∅, and
r(N)=max⁡{ρ(I):I⊆N, ρ(I)=r(I)}for every N.r(N) = \max\{\rho(I) : I \subseteq N,\ \rho(I) = r(I)\} \quad \text{for every } N.r(N)=max{ρ(I):I⊆N, ρ(I)=r(I)}for every N.
  1. If "independent" satisfies (I₁), (I₂) and ∅\emptyset∅ is independent, then r(N)=max⁡{ρ(I):I⊆N independent}r(N) = \max\{\rho(I) : I \subseteq N \text{ independent}\}r(N)=max{ρ(I):I⊆N independent} satisfies (R₁)–(R₃), and NNN is independent if and only if ρ(N)=r(N)\rho(N) = r(N)ρ(N)=r(N).

Both translations and both round trips are part of the goal: Whitney's conclusion is not only that each system implies the other but that "the definitions of the rank and the independence or dependence of any subset of MMM agree under the two systems".

Milestones

  • Lemma 1 (p. 510): r(N)≥0r(N) \ge 0r(N)≥0, n(N)≥0n(N) \ge 0n(N)≥0, and N⊆M′N \subseteq M'N⊆M′ implies r(N)≤r(M′)r(N) \le r(M')r(N)≤r(M′), n(N)≤n(M′)n(N) \le n(M')n(N)≤n(M′).
  • Lemma 2 (p. 510): any subset of an independent set is independent, which is (I₁).
  • Lemma 3 (p. 511): Δ(M+e2,e1)≤Δ(M,e1)\Delta(M + e_2, e_1) \le \Delta(M, e_1)Δ(M+e2​,e1​)≤Δ(M,e1​).
  • Lemma 4 (p. 511): Δ(M+N,e)≤Δ(M,e)\Delta(M + N, e) \le \Delta(M, e)Δ(M+N,e)≤Δ(M,e).
  • Theorem 3 (p. 511): Δ(M+N2,N1)≤Δ(M,N1)\Delta(M + N_2, N_1) \le \Delta(M, N_1)Δ(M+N2​,N1​)≤Δ(M,N1​); equivalently
r(M+N1+N2)≤r(M+N1)+r(M+N2)−r(M),r(M1+M2)≤r(M1)+r(M2)−r(M1M2).r(M + N_1 + N_2) \le r(M + N_1) + r(M + N_2) - r(M), \qquad r(M_1 + M_2) \le r(M_1) + r(M_2) - r(M_1 M_2).r(M+N1​+N2​)≤r(M+N1​)+r(M+N2​)−r(M),r(M1​+M2​)≤r(M1​)+r(M2​)−r(M1​M2​).
  • §4 (pp. 511–512): the independent sets of a rank system satisfy (I₂).

Significance

The equivalence makes the rank function and the family of independent sets two descriptions of one object. Every later result of Whitney's paper, and of matroid theory generally, moves between them without comment: the circuit postulates of §5 and §8, the base postulates of §7 and the duality of §§11–13 are all phrased through rank or independence as convenient. Theorem 3 is the submodularity of rank, the property that connects matroids to submodular function minimization and polymatroids; here it is derived from the purely local postulates (R₁)–(R₃), which constrain the rank only under the addition of one or two elements.

On the formal side, Mathlib defines Matroid through independent sets (with constructors from other axiom systems) and proves submodularity of its rank; the platform has submodularity for Mathlib matroids (FamousTheorems.matroid_rank_submodular_7a). Neither starts from Whitney's local rank postulates. What this mission adds is a machine-checked derivation of the global properties of rank from (R₁)–(R₃) and of Whitney's original equivalence, stated for his own postulates, so that the later missions of this series, which work from the same postulates, rest on a verified foundation. The result itself has been settled since 1935; the open work is the formal proof.

Difficulty

The postulates (R₂) and (R₃) are local: they speak about adding at most two elements to a set. Monotonicity and the bound r(N)≤ρ(N)r(N) \le \rho(N)r(N)≤ρ(N) follow by adding elements one at a time, but the submodular inequality relates arbitrary sets, and nothing in (R₃) mentions more than two new elements. The gap between the local and the global statement is the substance of Lemmas 3, 4 and Theorem 3, and the deduction of (I₂) depends on it.

In the converse direction the rank is defined as a maximum over independent subsets, while (I₂) only augments a set from an independent set with exactly one element more; (R₃) for the derived rank is a statement about three sets that are not given in that form. The round trips are where the two halves meet, and each depends on the global properties of the first half rather than on the postulates alone.

Formalization scope

Elements form a type α with [Fintype α] [DecidableEq α]; subsets are Finset α, and the matroid MMM is the whole type. Ranks, nullities and increments take values in ℤ, so differences never truncate; Whitney allows any number, but (R₁) and (R₂) force nonnegative integers. Postulate (I₂) is stated in Whitney's form with N'.card = N.card + 1, not the general augmentation for ρ(N)<ρ(N′)\rho(N) < \rho(N')ρ(N)<ρ(N′). Postulates (R₂), (R₃) keep their hypotheses e,e1,e2∉Ne, e_1, e_2 \notin Ne,e1​,e2​∈/N. Lemmas 3, 4 and Theorem 3 are stated for arbitrary subsets M,N,N1,N2M, N, N_1, N_2M,N,N1​,N2​; Lemma 1's monotonicity for arbitrary N⊆M′N \subseteq M'N⊆M′, the form in which the paper uses it. rankOfIndep is the supremum of cardinalities over the independent members of the powerset.

The paper takes for granted that the empty set is independent in system (I). Without that hypothesis, the predicate declaring nothing independent satisfies (I₁) and (I₂) vacuously, the supremum defining the rank returns 000, and the round trip fails; the goal therefore assumes ∅\emptyset∅ independent in part 2 and proves it in part 1. A formalization in terms of Mathlib's Matroid would make the goal a restatement of library facts, since Mathlib's matroids are independence systems by construction; the goal is deliberately about the postulates as predicates on functions and on families of sets.

A complete development needs only finite set combinatorics (Finset.card, induction on finite sets, Finset.sup). The derived lemmas (monotonicity, submodularity, (I₂)) are reusable for any later work from Whitney's rank postulates, including the circuit-postulate equivalence of the companion mission. A bridge from rank systems to Mathlib's Matroid (via IndepMatroid.ofFinset) would be a welcome addition but is not part of the goal. Proofs of the milestones in any order are welcome; Theorem 3 and the §4 deduction are the natural first targets.

Selected references

  • H. Whitney, On the Abstract Properties of Linear Dependence, American Journal of Mathematics 57 (1935), no. 3, 509–533. https://doi.org/10.2307/2371182
  • J. Oxley, Matroid Theory, 2nd ed., Oxford Graduate Texts in Mathematics 21, Oxford University Press, 2011. https://doi.org/10.1093/acprof:oso/9780198566946.001.0001
  • J. Kung (ed.), A Source Book in Matroid Theory, Birkhäuser, 1986. https://doi.org/10.1007/978-1-4684-9199-9
  • Mathlib, Mathlib.Data.Matroid (matroids via independent sets; IndepMatroid.ofFinset). https://leanprover-community.github.io/mathlib4_docs/Mathlib/Data/Matroid/Basic.html
8 thms2 active usersReviewed
🏆Completed
Dynamic ProgrammingOptimization·Captain: mikedeng1

Stochastic Optimal Control: The Discrete-Time Case IV: The Generalized Abstract Model — Restricted Policy Classes under ContractionTextbook

Why restricted policy classes

Abstract dynamic programming, in the form developed by Denardo (1967) and Bertsekas (1977), studies sequential decision problems through a single monotone mapping H(x,u,J)H(x,u,J)H(x,u,J): the cost of using control uuu at state xxx when the future is valued by the function JJJ. Chapters 2–5 of Bertsekas and Shreve, Stochastic Optimal Control: The Discrete-Time Case (1978; Athena Scientific reprint 1996), analyze this model when policies are arbitrary selectors μ:S→C\mu:S\to Cμ:S→C and HHH is defined on all extended-real functions on SSS.

That generality breaks down as soon as the state and control spaces are uncountable. A stochastic control problem on Borel spaces needs measurable policies, so that the expected cost is an integral rather than an outer integral, and the functions on which HHH acts must be measurable for the same reason. Chapter 6 of the book introduces a generalized abstract model in which the policies are drawn from a prescribed class M~\tilde MM~ and HHH is only defined on a prescribed class F~\tilde FF~ of functions. The examples on p. 94 are the models of Part II: universally measurable policies with lower semianalytic costs (Chapters 8–9), analytically measurable policies (Section 11.2), and the semicontinuous models of Definitions 8.7–8.8. Chapter 6 is the bridge that lets the abstract results of Part I be invoked for these models.

Setting

The data are a state space SSS, a control space CCC, nonempty constraint sets U(x)⊆CU(x)\subseteq CU(x)⊆C, and three restricted classes: sets of functions F∗⊂F~⊂FF^*\subset\tilde F\subset FF∗⊂F~⊂F, where FFF is the set of all functions S→[−∞,∞]S\to[-\infty,\infty]S→[−∞,∞], and a set M~\tilde MM~ of selectors μ:S→C\mu:S\to Cμ:S→C with μ(x)∈U(x)\mu(x)\in U(x)μ(x)∈U(x). The mapping H:S×C×F~→[−∞,∞]H:S\times C\times\tilde F\to[-\infty,\infty]H:S×C×F~→[−∞,∞] is monotone: J≤J′J\le J'J≤J′ in F~\tilde FF~ implies H(x,u,J)≤H(x,u,J′)H(x,u,J)\le H(x,u,J')H(x,u,J)≤H(x,u,J′). For μ∈M~\mu\in\tilde Mμ∈M~ and J∈F~J\in\tilde FJ∈F~,

Tμ(J)(x)=H[x,μ(x),J],T(J)(x)=inf⁡u∈U(x)H(x,u,J).T_\mu(J)(x)=H[x,\mu(x),J],\qquad T(J)(x)=\inf_{u\in U(x)}H(x,u,J).Tμ​(J)(x)=H[x,μ(x),J],T(J)(x)=u∈U(x)inf​H(x,u,J).

A policy is a sequence π=(μ0,μ1,… )\pi=(\mu_0,\mu_1,\dots)π=(μ0​,μ1​,…) with every μk∈M~\mu_k\in\tilde Mμk​∈M~; their set is Π~\tilde\PiΠ~. Given J0∈F∗J_0\in F^*J0​∈F∗ with J0>−∞J_0>-\inftyJ0​>−∞, the NNN-stage and infinite-horizon costs are

JN,π=(Tμ0⋯TμN−1)(J0),Jπ(x)=lim⁡N→∞JN,π(x),J_{N,\pi}=(T_{\mu_0}\cdots T_{\mu_{N-1}})(J_0),\qquad J_\pi(x)=\lim_{N\to\infty}J_{N,\pi}(x),JN,π​=(Tμ0​​⋯TμN−1​​)(J0​),Jπ​(x)=N→∞lim​JN,π​(x),

and the optimal costs are JN∗=inf⁡π∈Π~JN,πJ^*_N=\inf_{\pi\in\tilde\Pi}J_{N,\pi}JN∗​=infπ∈Π~​JN,π​ and J∗=inf⁡π∈Π~JπJ^*=\inf_{\pi\in\tilde\Pi}J_\piJ∗=infπ∈Π~​Jπ​. For a stationary policy (μ,μ,… )(\mu,\mu,\dots)(μ,μ,…) write JμJ_\muJμ​.

Five standing conditions tie the classes together: A.1 (every control u∈U(x)u\in U(x)u∈U(x) is the value μ(x)\mu(x)μ(x) of some μ∈M~\mu\in\tilde Mμ∈M~), A.2 (F∗F^*F∗ is closed under TTT and under adding constants), A.3 (F~\tilde FF~ is closed under every TμT_\muTμ​, μ∈M~\mu\in\tilde Mμ∈M~, and under adding constants), A.4 (ε\varepsilonε-minimizing selectors for T(J)T(J)T(J), J∈F∗J\in F^*J∈F∗, exist in M~\tilde MM~), and A.5 (F~\tilde FF~ and F∗F^*F∗ are closed under pointwise limits). Assumption C~\tilde CC~ asks for a closed subset Bˉ\bar BBˉ of the space BBB of bounded real functions with the sup norm ∥⋅∥\|\cdot\|∥⋅∥, containing J0J_0J0​ and invariant under TTT on Bˉ∩F∗\bar B\cap F^*Bˉ∩F∗ and under TμT_\muTμ​ on Bˉ∩F~\bar B\cap\tilde FBˉ∩F~, such that every JπJ_\piJπ​ exists and is real, each TμT_\muTμ​ is α\alphaα-Lipschitz on B∩F~B\cap\tilde FB∩F~, and every mmm-fold composition Tμ0⋯Tμm−1T_{\mu_0}\cdots T_{\mu_{m-1}}Tμ0​​⋯Tμm−1​​ is a ρ\rhoρ-contraction on Bˉ∩F~\bar B\cap\tilde FBˉ∩F~ for some ρ<1\rho<1ρ<1.

Formalization targets

Goal: Proposition 6.4 (p. 97)

Under A.1–A.5 and C~\tilde CC~: J∗∈Bˉ∩F∗J^*\in\bar B\cap F^*J∗∈Bˉ∩F∗ is the unique fixed point of TTT in Bˉ∩F∗\bar B\cap F^*Bˉ∩F∗, with T(J′)≤J′⇒J∗≤J′T(J')\le J'\Rightarrow J^*\le J'T(J′)≤J′⇒J∗≤J′ and J′≤T(J′)⇒J′≤J∗J'\le T(J')\Rightarrow J'\le J^*J′≤T(J′)⇒J′≤J∗; each JμJ_\muJμ​, μ∈M~\mu\in\tilde Mμ∈M~, is the unique fixed point of TμT_\muTμ​ in Bˉ∩F~\bar B\cap\tilde FBˉ∩F~;

lim⁡N→∞∥TN(J)−J∗∥=0  (J∈Bˉ∩F∗),lim⁡N→∞∥TμN(J)−Jμ∥=0  (J∈Bˉ∩F~);\lim_{N\to\infty}\|T^N(J)-J^*\|=0\ \ (J\in\bar B\cap F^*),\qquad\lim_{N\to\infty}\|T_\mu^N(J)-J_\mu\|=0\ \ (J\in\bar B\cap\tilde F);N→∞lim​∥TN(J)−J∗∥=0  (J∈Bˉ∩F∗),N→∞lim​∥TμN​(J)−Jμ​∥=0  (J∈Bˉ∩F~);

a stationary (μ∗,μ∗,… )∈Π~(\mu^*,\mu^*,\dots)\in\tilde\Pi(μ∗,μ∗,…)∈Π~ is optimal iff Tμ∗(J∗)=T(J∗)T_{\mu^*}(J^*)=T(J^*)Tμ∗​(J∗)=T(J∗); and for every ε>0\varepsilon>0ε>0 some stationary policy in Π~\tilde\PiΠ~ satisfies ∥J∗−Jμε∥≤ε\|J^*-J_{\mu_\varepsilon}\|\le\varepsilon∥J∗−Jμε​​∥≤ε.

Milestones

In attack order:

  1. Proposition 6.3(a) (p. 96) — under A.1–A.4 and the exact selection assumption, a uniformly NNN-stage optimal policy exists iff the infimum in Tk+1(J0)(x)=inf⁡u∈U(x)H[x,u,Tk(J0)]T^{k+1}(J_0)(x)=\inf_{u\in U(x)}H[x,u,T^k(J_0)]Tk+1(J0​)(x)=infu∈U(x)​H[x,u,Tk(J0​)] is attained for each x∈Sx\in Sx∈S and k<Nk<Nk<N.
  2. Proposition 6.5(a) (p. 97) — under A.1–A.5, C~\tilde CC~ and exact selection: if for each xxx some policy in Π~\tilde\PiΠ~ is optimal at xxx, then an optimal stationary policy exists in Π~\tilde\PiΠ~.

Further results of the chapter

The other results of Sections 6.2–6.3 are posed in the mission as separate theorems:

  • Proposition 6.2 — π∗\pi^*π∗ is uniformly NNN-stage optimal iff (Tμk∗TN−k−1)(J0)=TN−k(J0)(T_{\mu_k^*}T^{N-k-1})(J_0)=T^{N-k}(J_0)(Tμk∗​​TN−k−1)(J0​)=TN−k(J0​) for k<Nk<Nk<N; such a policy forces JN∗=TN(J0)J^*_N=T^N(J_0)JN∗​=TN(J0​).
  • Proposition 6.1(a) — under Assumption F~.2\tilde F.2F~.2 and Jk∗>−∞J^*_k>-\inftyJk∗​>−∞: JN∗=TN(J0)J^*_N=T^N(J_0)JN∗​=TN(J0​) and NNN-stage ε\varepsilonε-optimal policies exist in Π~\tilde\PiΠ~.
  • Proposition 6.1(b) — under Assumption F~.3\tilde F.3F~.3 and Jk,π<∞J_{k,\pi}<\inftyJk,π​<∞: JN∗=TN(J0)J^*_N=T^N(J_0)JN∗​=TN(J0​) and {εn}\{\varepsilon_n\}{εn​}-dominated convergence to optimality.
  • Proposition 6.3(b) — compact level sets Uk(x,λ)U_k(x,\lambda)Uk​(x,λ) give both JN∗=TN(J0)J^*_N=T^N(J_0)JN∗​=TN(J0​) and a uniformly NNN-stage optimal policy.
  • Proposition 6.5(b) — compact level sets of the iterates Tk(J)T^k(J)Tk(J), k≥kˉk\ge\bar kk≥kˉ, give an optimal stationary policy.

Significance

Proposition 6.4 is the statement that makes value iteration, Bellman's equation and stationary ε\varepsilonε-optimal policies available for discounted problems whose admissible policies are restricted, for instance to measurable ones. Without it, each measurable model would need its own fixed-point argument. The finite-horizon Propositions 6.1–6.3 play the same role for the dynamic programming algorithm JN∗=TN(J0)J^*_N=T^N(J_0)JN∗​=TN(J0​), and their hypotheses (F~.3\tilde F.3F~.3, exact selection) are exactly what Chapters 7–8 verify for universally measurable policies.

The book states Propositions 6.4 and 6.5 without proof (p. 97), referring to the proofs of Chapter 4; Propositions 6.1–6.3 are justified by "nearly verbatim repetition" of Chapter 3. A formalization therefore supplies proofs that are only indicated in print, and checks that A.1–A.5 really suffice for each step of the Chapter 3–4 arguments. None of these results has a machine-checked proof that we know of; the companion missions of this series formalize the unrestricted special case (F∗=F~=FF^*=\tilde F=FF∗=F~=F, M~=M\tilde M=MM~=M) of Chapters 3 and 4.

Difficulty

The Chapter 4 proof of Proposition 4.2 applies the contraction mapping theorem to TTT on Bˉ\bar BBˉ. Here the obvious transcription fails at two points. First, TTT maps Bˉ∩F∗\bar B\cap F^*Bˉ∩F∗ into itself but TμT_\muTμ​ only maps Bˉ∩F~\bar B\cap\tilde FBˉ∩F~ into itself, so the fixed-point theorem must be applied on two different sets, and these are closed only because of A.5. Second, every argument that picks a near-minimizing selector at each state must produce a selector in M~\tilde MM~: pointwise choices are no longer allowed, and A.1, A.4 and the exact selection assumption are the only sources of admissible selectors. Proofs of Chapter 3–4 that build a policy state by state cannot be copied.

Formalization scope

Functions on SSS are S → EReal. HHH is a total Lean function, but monotonicity is assumed only on F~\tilde FF~ and every statement evaluates HHH only at functions of F~\tilde FF~. JπJ_\piJπ​ is limUnder; Assumption C~\tilde CC~ makes the limit exist. BBB is Mathlib's ℓ∞(S,R)\ell^\infty(S,\mathbb R)ℓ∞(S,R); a bound ∥G−G′∥≤c\|G-G'\|\le c∥G−G′∥≤c between extended-real functions means both are real everywhere and ∣G(x)−G′(x)∣≤c|G(x)-G'(x)|\le c∣G(x)−G′(x)∣≤c, which is how the book's convention ∞−∞=∞\infty-\infty=\infty∞−∞=∞ reads a norm of a difference. No statement adds values of opposite infinite sign, so Mathlib's EReal addition agrees with the book's wherever it is used. JN∗J^*_NJN∗​ and J∗J^*J∗ are infima over Π~\tilde\PiΠ~ only, the ε\varepsilonε-optimality notions keep the book's two-case form at −∞-\infty−∞, and NNN is a positive integer.

The chapter collapses to Chapters 3–4 if F∗=F~=FF^*=\tilde F=FF∗=F~=F or M~=M\tilde M=MM~=M is built in; here F∗F^*F∗, F~\tilde FF~ and M~\tilde MM~ are arbitrary and constrained only by A.1–A.5, and J∗J^*J∗ is never defined as a fixed point.

A complete development needs the mmm-step contraction mapping theorem on a closed subset of ℓ∞\ell^\inftyℓ∞, monotonicity lemmas for TTT and TμT_\muTμ​, and the restricted-class versions of Propositions 3.1–3.4 and 4.1–4.4. These are reusable for the Borel models of Chapters 8–9. Proofs of any milestone, and sorry-free lemmas about the Assumption C~\tilde CC~ contraction, are welcome.

Selected references

  • D. P. Bertsekas and S. E. Shreve, Stochastic Optimal Control: The Discrete-Time Case, Academic Press, 1978; Athena Scientific, 1996, Chapter 6. https://web.mit.edu/dimitrib/www/soc.html
  • D. P. Bertsekas, Monotone mappings with application in dynamic programming, SIAM J. Control and Optimization 15(3), 1977, 438–464. https://doi.org/10.1137/0315031
  • E. V. Denardo, Contraction mappings in the theory underlying dynamic programming, SIAM Review 9(2), 1967, 165–177. https://doi.org/10.1137/1009030
7 thms2 active usersReviewed
🏆Completed
Linear OptimizationOptimizationProbability+1·Captain: mikedeng1

Solving Linear Programs in the Current Matrix Multiplication Time: The Stochastic Central Path Falls Back to a Classical Step with Probability at Most 10/n² per IterationResearch Paper

Motivation

Linear programming, min⁡{c⊤x:Ax=b, x≥0}\min\{c^\top x : Ax=b,\ x\ge0\}min{c⊤x:Ax=b, x≥0} with A∈Rd×nA\in\mathbb R^{d\times n}A∈Rd×n, is the basic model of operations research, and the complexity of solving it is a central question of algorithm theory. Interior-point methods follow the central path: primal–dual pairs (x,s)(x,s)(x,s) with x,s>0x,s>0x,s>0 and xisi=tx_is_i=txi​si​=t for every iii, as the path parameter ttt decreases to 000. A classical short-step method needs O(nlog⁡(n/δ))O(\sqrt n\log(n/\delta))O(n​log(n/δ)) iterations, each solving a linear system with the matrix AXSA⊤A\frac XSA^\topASX​A⊤, for a total of roughly n2.5n^{2.5}n2.5 operations or more.

Cohen, Lee and Song (J. ACM 68(1), 2021; arXiv:1810.07896) showed that linear programs can be solved in time nω+o(1)log⁡(n/δ)n^{\omega+o(1)}\log(n/\delta)nω+o(1)log(n/δ) (for the current values of the matrix multiplication exponent ω\omegaω and its dual α\alphaα), matching the cost of multiplying two n×nn\times nn×n matrices. The analysis has two halves: a data structure that maintains the projection matrix lazily, and the stochastic central path method, which replaces each Newton step by a sparse random step and proves that the iterates still stay close to the central path. This mission formalizes the second half.

Timeline: Karmarkar's projective method (1984) gave the first polynomial interior-point method; Renegar (1988) gave the O(nlog⁡(1/δ))O(\sqrt n\log(1/\delta))O(n​log(1/δ)) path-following bound; Vaidya (1989) reduced the per-iteration cost with low-rank updates; Lee and Sidford (2014–2015) reduced the iteration count to O~(d)\widetilde O(\sqrt d)O(d​); Cohen, Lee and Song (STOC 2019, J. ACM 2021) reached nωn^\omeganω; van den Brand (2020) derandomized the result.

Setting

Vectors are in Rn\mathbb R^nRn and products, quotients and roots of vectors are coordinatewise. For ϵ\epsilonϵ and vectors a,ba,ba,b, a≈ϵba\approx_\epsilon ba≈ϵ​b means (1−ϵ)bi≤ai≤(1+ϵ)bi(1-\epsilon)b_i\le a_i\le(1+\epsilon)b_i(1−ϵ)bi​≤ai​≤(1+ϵ)bi​ for all iii; a≈ϵta\approx_\epsilon ta≈ϵ​t for a scalar ttt is defined likewise. The number of variables is n≥10n\ge10n≥10 and AAA has full row rank d≤nd\le nd≤n.

The potential is Φλ(r)=∑i=1ncosh⁡(λri)\Phi_\lambda(r)=\sum_{i=1}^n\cosh(\lambda r_i)Φλ​(r)=∑i=1n​cosh(λri​), evaluated at r=μ/t−1r=\mu/t-1r=μ/t−1 with μ=xs\mu=xsμ=xs; it is small exactly when every xisix_is_ixi​si​ is close to ttt.

StochasticStep (Algorithm 1) takes positive x,sx,sx,s, a direction δμ\delta_\muδμ​, a sampling parameter kkk and the output v~\widetilde vv of a data structure with x/s≈ϵmpv~x/s\approx_{\epsilon_{\mathrm{mp}}}\widetilde vx/s≈ϵmp​​v. It rescales to x‾=xv~/w\overline x=x\sqrt{\widetilde v/w}x=xv/w​, s‾=sw/v~\overline s=s\sqrt{w/\widetilde v}s=sw/v​ (w=x/sw=x/sw=x/s), draws a sparse vector δ~μ\widetilde\delta_\muδμ​ with independent coordinates, δ~μ,i=δμ,i/pi\widetilde\delta_{\mu,i}=\delta_{\mu,i}/p_iδμ,i​=δμ,i​/pi​ with probability pi=min⁡(1,k(δμ,i2/∥δμ∥22+1/n))p_i=\min(1,k(\delta_{\mu,i}^2/\|\delta_\mu\|_2^2+1/n))pi​=min(1,k(δμ,i2​/∥δμ​∥22​+1/n)) and 000 otherwise, and computes the step (δ~x,δ~s)(\widetilde\delta_x,\widetilde\delta_s)(δx​,δs​) through the projection P‾=X‾/S‾A⊤(AX‾S‾A⊤)−1AX‾/S‾\overline P=\sqrt{\overline X/\overline S}A^\top(A\frac{\overline X}{\overline S}A^\top)^{-1}A\sqrt{\overline X/\overline S}P=X/S​A⊤(ASX​A⊤)−1AX/S​. The draw is repeated until ∥s‾−1δ~s∥∞\|\overline s^{-1}\widetilde\delta_s\|_\infty∥s−1δs​∥∞​ and ∥x‾−1δ~x∥∞\|\overline x^{-1}\widetilde\delta_x\|_\infty∥x−1δx​∥∞​ are at most 1/(100log⁡n)1/(100\log n)1/(100logn); the output is (x+δ~x,s+δ~s)(x+\widetilde\delta_x,s+\widetilde\delta_s)(x+δx​,s+δs​).

Main (Algorithm 2) sets ϵ=140000log⁡n\epsilon=\frac1{40000\log n}ϵ=40000logn1​, ϵmp=140000\epsilon_{\mathrm{mp}}=\frac1{40000}ϵmp​=400001​, k=1000ϵnlog⁡2n/ϵmpk=1000\epsilon\sqrt n\log^2n/\epsilon_{\mathrm{mp}}k=1000ϵn​log2n/ϵmp​, λ=40log⁡n\lambda=40\log nλ=40logn, starts at t=1t=1t=1, and in each iteration sets tnew=(1−ϵ3n)tt^{\mathrm{new}}=(1-\frac{\epsilon}{3\sqrt n})ttnew=(1−3n​ϵ​)t, takes the direction

δμ=(tnewt−1)xs−ϵ2tnew∇Φλ(μ/t−1)∥∇Φλ(μ/t−1)∥2,\delta_\mu=\Big(\frac{t^{\mathrm{new}}}{t}-1\Big)xs-\frac\epsilon2t^{\mathrm{new}}\frac{\nabla\Phi_\lambda(\mu/t-1)}{\|\nabla\Phi_\lambda(\mu/t-1)\|_2},δμ​=(ttnew​−1)xs−2ϵ​tnew∥∇Φλ​(μ/t−1)∥2​∇Φλ​(μ/t−1)​,

runs StochasticStep, and falls back to a deterministic ClassicalStep whenever Φλ(μnew/tnew−1)>n3\Phi_\lambda(\mu^{\mathrm{new}}/t^{\mathrm{new}}-1)>n^3Φλ​(μnew/tnew−1)>n3.

Formalization targets

Goal: Lemma 4.14

For every iteration jjj, almost surely Assumption 4.1 holds for the input of iteration jjj (in particular xjsj≈0.1tjx^js^j\approx_{0.1}t_jxjsj≈0.1​tj​ and ∥δμ∥2≤ϵtj\|\delta_\mu\|_2\le\epsilon t_j∥δμ​∥2​≤ϵtj​), almost surely the resampling loop of iteration jjj succeeds with positive probability, and

P(ClassicalStep is used in iteration j)≤10n2.\mathbb P(\text{ClassicalStep is used in iteration }j)\le\frac{10}{n^2}.P(ClassicalStep is used in iteration j)≤n210​.

The paper writes O(1/n2)O(1/n^2)O(1/n2); its proof gives the constant 101010.

Milestones

Lemma A.1 (variance of a product), Lemma 4.12 (properties of Φλ\Phi_\lambdaΦλ​), Lemma 4.2 (explicit step), Lemma 4.3 and Claim 4.7 (moments and success probability of the sampled step), Lemma 4.8 (moments of μnew\mu^{\mathrm{new}}μnew), and Lemma 4.13:

E[Φλ(μnewtnew−1)]≤Φλ(μt−1)−λϵ15n(Φλ(μt−1)−10n).\mathbf E\Big[\Phi_\lambda\Big(\frac{\mu^{\mathrm{new}}}{t^{\mathrm{new}}}-1\Big)\Big]\le\Phi_\lambda\Big(\frac\mu t-1\Big)-\frac{\lambda\epsilon}{15\sqrt n}\Big(\Phi_\lambda\Big(\frac\mu t-1\Big)-10n\Big).E[Φλ​(tnewμnew​−1)]≤Φλ​(tμ​−1)−15n​λϵ​(Φλ​(tμ​−1)−10n).

Significance

Lemma 4.14 is what makes the randomized method usable: the iterates stay in the 0.10.10.1-neighbourhood of the central path along the whole run, and the expensive fallback is rare enough that its expected cost, O~(n2.5)⋅10/n2\widetilde O(n^{2.5})\cdot 10/n^2O(n2.5)⋅10/n2, is negligible. The paper's cost bound (Lemma 4.16) and its main theorem rest on it. The same potential-based "stochastic central path" analysis was reused in later solvers, for instance for empirical risk minimization (Lee, Song and Zhang, COLT 2019).

The result is proved in the paper; no machine-checked version exists. A formalization pins down the probabilistic model that the paper leaves implicit (independence of the sampled coordinates, the law of the resampling loop, a data structure and fallback that see only the past) and checks the constants, several of which are tight against printed slack (Remark 4.4).

The running-time claims of the paper (Theorem 2.1's expected time nω+o(1)n^{\omega+o(1)}nω+o(1), Lemma 4.16, Section 5) are not part of this mission: they live in an arithmetic cost model that Lean does not have. The accuracy guarantee of Theorem 2.1 (Lemma A.6, ClassicalStep from [57]) is also outside the mission.

Difficulty

The obvious argument would bound each quantity under the product law of the sparse direction. But StochasticStep resamples, so the step actually taken is distributed according to that law conditioned on a success event, and expectations and variances shift. A second difficulty is that Φλ\Phi_\lambdaΦλ​ is controlled only in expectation, while Assumption 4.1 must hold surely at every iteration; this is reconciled by the deterministic ClassicalStep fallback, which caps Φλ\Phi_\lambdaΦλ​ at n3n^3n3, and by an induction over iterations of E[Φ]≤10n\mathbf E[\Phi]\le10nE[Φ]≤10n under the trajectory law. Claim 4.7 needs a Bernstein inequality, which Mathlib does not yet provide.

Formalization scope

Coordinates are Fin n, vectors Fin n → ℝ, AAA a Matrix (Fin d) (Fin n) ℝ with A.rank = d, and log⁡\loglog the natural logarithm. ∥⋅∥2\|\cdot\|_2∥⋅∥2​ is written out as ∑ivi2\sqrt{\sum_iv_i^2}∑i​vi2​​; ∥⋅∥∞≤c\|\cdot\|_\infty\le c∥⋅∥∞​≤c is stated coordinatewise. The sampled direction has law Measure.pi of two-point laws; the step taken by StochasticStep has that law conditioned (ProbabilityTheory.cond) on the success event, and every E\mathbf EE, Var\mathbf{Var}Var of Lemmas 4.3, 4.8 and 4.13 is under this conditioned law. mp.Query is replaced by its value P‾(X‾S‾)−1/2δ~μ\overline P(\overline X\overline S)^{-1/2}\widetilde\delta_\muP(XS)−1/2δμ​; the data structure and ClassicalStep are arbitrary measurable functions UjU_jUj​, CjC_jCj​ of the history with the only properties the paper uses. The trajectory is Mathlib's Ionescu-Tulcea measure, with kernels equal to the step law of Main. nnn is the number of variables of the program the loop runs on.

Deviations from the page, all recorded in the items: Assumption 4.1 is used with ϵ≤1/(40000log⁡n)\epsilon\le1/(40000\log n)ϵ≤1/(40000logn) instead of the printed <<<, because Main sets ϵ\epsilonϵ to exactly that value; O(1/n2)O(1/n^2)O(1/n2) is instantiated as 10/n210/n^210/n2, the constant of the paper's proof; the conclusions of Lemma 4.14 are stated for every iteration index rather than while t>δ2/(32n3)t>\delta^2/(32n^3)t>δ2/(32n3); at ∇Φλ=0\nabla\Phi_\lambda=0∇Φλ​=0 the second term of δμ\delta_\muδμ​ is 000. No hypothesis k≤nk\le nk≤n is imposed.

A trivializing formalization is ruled out: every statement that integrates against the conditioned law also concludes that this law is a probability measure (so it cannot be the zero measure), the goal concludes that each resampling loop succeeds with positive probability, the oracles UjU_jUj​, CjC_jCj​ cannot see the coins of the current iteration, and the goal is about the whole iterated process from the initial point, not one step from an arbitrary law.

Contributions welcome: a Bernstein inequality for bounded independent sums, conditional-law lemmas for cond of Measure.pi, and Markov-kernel measurability for the step law; these are reusable beyond this mission.

Selected references

  • M. B. Cohen, Y. T. Lee, Z. Song, Solving Linear Programs in the Current Matrix Multiplication Time, J. ACM 68(1), Article 3, 2021. https://doi.org/10.1145/3424305 (arXiv:1810.07896, https://arxiv.org/abs/1810.07896)
  • N. Karmarkar, A new polynomial-time algorithm for linear programming, Combinatorica 4, 1984. https://doi.org/10.1007/BF02579150
  • J. Renegar, A polynomial-time algorithm, based on Newton's method, for linear programming, Math. Programming 40, 1988. https://doi.org/10.1007/BF01580724
  • P. M. Vaidya, Speeding-up linear programming using fast matrix multiplication, Proc. 30th FOCS, 1989.
  • Y. T. Lee, A. Sidford, Path finding methods for linear programming, FOCS 2014. https://doi.org/10.1109/FOCS.2014.52
  • Y. T. Lee, Z. Song, Q. Zhang, Solving Empirical Risk Minimization in the Current Matrix Multiplication Time, COLT 2019. https://arxiv.org/abs/1905.04447
  • J. van den Brand, A deterministic linear program solver in current matrix multiplication time, SODA 2020. https://doi.org/10.1137/1.9781611975994.16
11 thms2 active usersReviewed
🏆Completed
CombinatoricsMarkov ChainProbability+1·Captain: mikedeng1

Reversibility and Stochastic Networks VI: The Ewens Sampling Distribution Is Consistent Under Sampling Without ReplacementTextbook

Motivation

The neutral theory of molecular evolution holds that much of the genetic variation observed at the molecular level is caused by selectively neutral mutations rather than by selection. To test it against data one needs the distribution of allele frequencies that a neutral model predicts, and in practice that distribution has to be compared with a sample from the population, never with the whole population. Ewens (Ewens 1972) derived the equilibrium distribution of allele counts under the infinite alleles model, now called the Ewens sampling formula; it underlies classical tests of neutrality and appears throughout combinatorics and probability as the law of the cycle type of an Ewens-distributed random permutation and of the Chinese restaurant process.

Chapter 7 of F. P. Kelly, Reversibility and Stochastic Networks (Wiley, 1979) obtains the infinite alleles model as a limit of the reversible migration processes of Chapters 2 and 6, and uses reversibility to answer questions about allele ages and fixation. The mission formalizes the finite, combinatorial results of that chapter.

Timeline. Kimura and Crow (1964) introduced the infinite alleles model. Ewens (1972) found its equilibrium sampling distribution (7.6). Kingman (1978, J. London Math. Soc.) characterized the consistency of random partitions under sampling, the property Theorem 7.1 asserts for the Ewens family. Kelly (1979, Chapter 7) derived (7.6) as a limit of reversible migration processes, and the consistency and the allele-age results from the reversibility of a labelled population process.

Setting

A population consists of M≥2M\ge2M≥2 individuals, each carrying an allelic type. Its description is M=(M1,…,MM)\mathbf M=(M_1,\dots,M_M)M=(M1​,…,MM​), where MiM_iMi​ is the number of allelic types carried by exactly iii individuals, so that

∑i=1MiMi=M.(7.3)\sum_{i=1}^{M} iM_i=M. \qquad (7.3)i=1∑M​iMi​=M.(7.3)

For a real parameter ν>0\nu>0ν>0, the Ewens distribution on descriptions is

πM(M)=(ν+M−1M)−1∏i=1M(νi)Mi1Mi!,(7.6)\pi_M(\mathbf M)=\binom{\nu+M-1}{M}^{-1}\prod_{i=1}^{M}\Big(\frac{\nu}{i}\Big)^{M_i}\frac{1}{M_i!}, \qquad (7.6)πM​(M)=(Mν+M−1​)−1i=1∏M​(iν​)Mi​Mi​!1​,(7.6)

where (xk)=x(x−1)⋯(x−k+1)/k!\binom{x}{k}=x(x-1)\cdots(x-k+1)/k!(kx​)=x(x−1)⋯(x−k+1)/k! is the binomial coefficient for real xxx. In the infinite alleles model, individuals die at rate μ\muμ, each death is followed by the birth of an offspring of a uniformly chosen survivor, and the offspring is a mutant of an entirely new type with probability uuu; then (7.6) is the equilibrium distribution with ν=(M−1)u/(1−u)\nu=(M-1)u/(1-u)ν=(M−1)u/(1−u) (7.5).

A random sample of size 1≤m≤M1\le m\le M1≤m≤M without replacement is a uniformly random mmm-element subset of the MMM labelled individuals, each of the (Mm)\binom Mm(mM​) subsets being equally likely; the sample has a description in the same sense.

The number jjj of individuals carrying one given allele performs a random walk on {0,…,M}\{0,\dots,M\}{0,…,M} with intensities

q(j,j−1)=μjM(M−jM−1+j−1M−1u),q(j,j+1)=μM−jMjM−1(1−u).(7.8)q(j,j-1)=\mu\frac jM\Big(\frac{M-j}{M-1}+\frac{j-1}{M-1}u\Big),\qquad q(j,j+1)=\mu\frac{M-j}{M}\frac{j}{M-1}(1-u). \qquad (7.8)q(j,j−1)=μMj​(M−1M−j​+M−1j−1​u),q(j,j+1)=μMM−j​M−1j​(1−u).(7.8)

An allele is quasi-fixed when it is the only allele present (j=Mj=Mj=M).

Formalization targets

Goal: consistency under sampling (Theorem 7.1)

If M≥2M\ge2M≥2 and the population description is distributed as πM\pi_MπM​, then a random sample of size 1≤m≤M1\le m\le M1≤m≤M drawn without replacement has description m\mathbf mm with probability πm(m)\pi_m(\mathbf m)πm​(m), the same ν\nuν being used for both sizes:

∑MπM(M) P(sample has description m∣population has description M)=πm(m).\sum_{\mathbf M}\pi_M(\mathbf M)\,P\big(\text{sample has description }\mathbf m\mid\text{population has description }\mathbf M\big)=\pi_m(\mathbf m).M∑​πM​(M)P(sample has description m∣population has description M)=πm​(m).

Milestones

  1. (7.6) is a distribution: πM(M)>0\pi_M(\mathbf M)>0πM​(M)>0 and ∑MπM(M)=1\sum_{\mathbf M}\pi_M(\mathbf M)=1∑M​πM​(M)=1 (Exercise 7.1.3).
  2. Theorem 7.1 for m=M−1m=M-1m=M−1, the case the book's proof establishes first.
  3. Corollary 7.5, the identity of its proof: the probability that a uniformly chosen individual's allele is carried by exactly iii individuals is
∑MiMiMπM(M)=νM(ν+M−1i)−1(Mi).(7.9)\sum_{\mathbf M}\frac{iM_i}{M}\pi_M(\mathbf M)=\frac{\nu}{M}\binom{\nu+M-1}{i}^{-1}\binom Mi. \qquad (7.9)M∑​MiMi​​πM​(M)=Mν​(iν+M−1​)−1(iM​).(7.9)
  1. Theorem 7.9: the probability QQQ that the walk (7.8) started at 111 reaches MMM before 000 satisfies
Q−1=∑i=0M−1(M−1i)−1(ν+M−1i).Q^{-1}=\sum_{i=0}^{M-1}\binom{M-1}{i}^{-1}\binom{\nu+M-1}{i}.Q−1=i=0∑M−1​(iM−1​)−1(iν+M−1​).

Significance

The results. Consistency under sampling is what makes the Ewens formula usable as a statistical model: the predicted distribution for an observed sample does not depend on the unknown population size, only on ν\nuν. Kelly deduces from it the sufficiency of the number of alleles in a sample for ν\nuν and the heterozygosity ν/(ν+1)\nu/(\nu+1)ν/(ν+1) (Exercises 7.1.5, 7.1.8). The formula (7.9) gives the equilibrium frequency of the oldest allele, and Theorem 7.9 gives the quasi-fixation probability from which the mean time between quasi-fixations follows (Corollary 7.10).

Formalizing them. All four results are classical and proved; none has a machine-checked proof on the platform or in Mathlib as of this writing. The mission produces a reusable formal Ewens distribution over integer partitions, a definition of sampling without replacement by counting labelled subsets, and an absorption probability for an explicit birth–death walk. Proofs independent of Kelly's process argument are welcome.

Difficulty

The book's proof of Theorem 7.1 is a process argument: in a population whose size fluctuates between M−1M-1M−1 and MMM, a drop in size acts as a random deletion, and the truncated equilibrium (7.7) restricted to each size gives πM−1\pi_{M-1}πM−1​ and πM\pi_MπM​. Turning that into a statement about finite sets requires the equilibrium of a truncated reversible process, which is not available here, so a formal proof must either build that process or find a direct combinatorial route. A direct route has to relate, for each description of the sample, the number of mmm-subsets of a labelled population with a given description to products of binomial coefficients, and sum the result against (7.6); the bookkeeping over partitions is where the work lies. Theorem 7.9 needs a solution of the first-step equations of a non-symmetric walk and the identification of that solution with a hitting probability defined as a limit.

Formalization scope

  • Descriptions of nnn individuals are integer partitions Nat.Partition n, with MiM_iMi​ the multiplicity of the part iii; the product in (7.6) runs over i=1,…,ni=1,\dots,ni=1,…,n. The real binomial coefficient is the published definition AppliedComb.GenFun.binomReal.
  • The population is Fin M with allelic types Fin M → ℕ; the description of a labelled set is computed from the labelling. The sampling probability is (Mm)−1\binom Mm^{-1}(mM​)−1 times the number of mmm-subsets whose restricted labelling has the given description. It is not defined by a formula on descriptions, and a definition that removed individuals one at a time in proportion to class sizes (the book's proof route) is ruled out as a definition because it presupposes the reduction the proof must supply.
  • The goal and Corollary 7.5 quantify over an arbitrary choice of labelling for each population description. They assume M≥2M\ge2M≥2, as required by the chapter's rule that a parent is chosen among the other M−1M-1M−1 individuals; the goal also assumes 1≤m≤M1\le m\le M1≤m≤M. Because πM>0\pi_M>0πM​>0, this forces the conditional sampling law to depend on the population only through its description. Types are natural numbers, so every description is realized and the hypothesis is never vacuous.
  • The quasi-fixation probability is defined through the jump chain of (7.8): the limit of the probabilities of reaching MMM within nnn jumps without reaching 000. The theorem assumes M≥2M\ge2M≥2, μ>0\mu>0μ>0, 0<u<10<u<10<u<1 and ν=(M−1)u/(1−u)\nu=(M-1)u/(1-u)ν=(M−1)u/(1−u).
  • Corollary 7.5 is formalized as the identity of its proof. The identification of the oldest allele's frequency with that of a randomly chosen individual uses allele ages and the reversibility of the labelled process (Theorem 7.2) and is not formalized. Theorem 7.2 itself, whose state space orders the allele labels within each class, and the allele-age results (Corollaries 7.3, 7.4, 7.7, 7.8, Theorem 7.6, Corollary 7.10, Theorem 7.11) are not part of the mission.

Contributions of general partition and sampling lemmas (counting subsets with a given description, the generating function identity (1−x)−ν=∏jeνxj/j(1-x)^{-\nu}=\prod_j e^{\nu x^j/j}(1−x)−ν=∏j​eνxj/j) are reusable beyond this mission.

Selected references

  • F. P. Kelly, Reversibility and Stochastic Networks, Wiley, 1979, Chapter 7. https://www.statslab.cam.ac.uk/~frank/BOOKS/kelly_book.html
  • W. J. Ewens, The sampling theory of selectively neutral alleles, Theoretical Population Biology 3 (1972), 87–112. https://doi.org/10.1016/0040-5809(72)90035-4
  • J. F. C. Kingman, The representation of partition structures, Journal of the London Mathematical Society (2) 18 (1978), 374–380. https://doi.org/10.1112/jlms/s2-18.2.374
  • M. Kimura and J. F. Crow, The number of alleles that can be maintained in a finite population, Genetics 49 (1964), 725–738. https://doi.org/10.1093/genetics/49.4.725
9 thms2 active usersReviewed
Bandit AlgorithmsMachine LearningProbability·Captain: mikedeng1

The Best of Both Worlds: Stochastic and Adversarial Bandits: SAO Has Pseudo-Regret O(K log K log²β/Δ) on Stochastic Rewards and Regret Õ(√(nK)) Against Adaptive AdversariesResearch Paper

Motivation

In a multi-armed bandit problem a learner chooses one of KKK actions in each of nnn rounds and observes only the reward of the chosen action. Two models of the rewards have separate theories. In the stochastic model, each arm pays independent draws from a fixed distribution; algorithms such as UCB1 (Auer, Cesa-Bianchi & Fischer 2002) have regret of order ∑ilog⁡(n)/Δi\sum_i \log(n)/\Delta_i∑i​log(n)/Δi​, logarithmic in nnn. In the adversarial model, an adversary chooses the rewards; Exp3 and its variants (Auer, Cesa-Bianchi, Freund & Schapire 2002) have regret of order nK\sqrt{nK}nK​, which is optimal there. An algorithm tuned for one model fails in the other: stochastic algorithms can suffer linear regret against an adversary, and adversarial algorithms pay n\sqrt nn​ even when the rewards are i.i.d.

Bubeck and Slivkins (arXiv:1202.4473, COLT 2012) asked whether one algorithm can be near-optimal in both models without knowing which one it faces. They answered yes with the algorithm SAO. This result started the "best of both worlds" line of work on bandits. Later contributions include EXP3++ (Seldin & Slivkins 2014) and Tsallis-INF (Zimmert & Seldin 2021).

Setting

There are K≥2K\ge2K≥2 arms and n≥Kn\ge Kn≥K rounds. On round ttt the algorithm draws an arm ItI_tIt​ from a probability vector pt=(p1,t,…,pK,t)p_t=(p_{1,t},\dots,p_{K,t})pt​=(p1,t​,…,pK,t​) computed from the history it has observed. At the same time a reward vector gt∈[0,1]Kg_t\in[0,1]^Kgt​∈[0,1]K is fixed, and the algorithm observes only gIt,tg_{I_t,t}gIt​,t​.

  • Adversarial model. The vector gtg_tgt​ is chosen by an adaptive adversary: a function of the arms I1,…,It−1I_1,\dots,I_{t-1}I1​,…,It−1​ played earlier, but not of ItI_tIt​. The regret is Rn=max⁡i∑t=1ngi,t−∑t=1ngIt,tR_n=\max_i\sum_{t=1}^n g_{i,t}-\sum_{t=1}^n g_{I_t,t}Rn​=maxi​∑t=1n​gi,t​−∑t=1n​gIt​,t​.
  • Stochastic model. There are distributions ν1,…,νK\nu_1,\dots,\nu_Kν1​,…,νK​ on [0,1][0,1][0,1] with means μi\mu_iμi​, and all gi,t∼νig_{i,t}\sim\nu_igi,t​∼νi​ are independent. The pseudo-regret is R‾n=∑t=1n(max⁡iμi−μIt)\overline R_n=\sum_{t=1}^n(\max_i\mu_i-\mu_{I_t})Rn​=∑t=1n​(maxi​μi​−μIt​​). The gap of arm iii is Δi=max⁡jμj−μi\Delta_i=\max_j\mu_j-\mu_iΔi​=maxj​μj​−μi​, and the minimal gap is Δ=min⁡i:Δi>0Δi\Delta=\min_{i:\Delta_i>0}\Delta_iΔ=mini:Δi​>0​Δi​.

The analysis uses importance-weighted estimates H~i,t=1t∑s≤tgi,s1{Is=i}/pi,s\widetilde H_{i,t}=\frac1t\sum_{s\le t}g_{i,s}\mathbb 1_{\{I_s=i\}}/p_{i,s}Hi,t​=t1​∑s≤t​gi,s​1{Is​=i}​/pi,s​, the sample means H^i,t\widehat H_{i,t}Hi,t​, the averages Hi,t=1t∑s≤tgi,sH_{i,t}=\frac1t\sum_{s\le t}g_{i,s}Hi,t​=t1​∑s≤t​gi,s​, and the play counts Ti(t)T_i(t)Ti​(t).

SAO (Algorithm 1 of the paper) takes a parameter β>1\beta>1β>1. It keeps a set of active arms, initially all arms, and samples them uniformly at first. On each round it applies a test, (12), that deactivates an arm whose estimate H~i,t\widetilde H_{i,t}Hi,t​ falls far below the best active one. The probability of a deactivated arm then decays as qiτi/tq_i\tau_i/tqi​τi​/t, where τi\tau_iτi​ is the deactivation time and qiq_iqi​ the arm's probability at that moment. Three further tests, (13)–(15), check that the observations stay consistent with stochastic rewards. If any of them fails on round τ0\tau_0τ0​, SAO switches permanently to the adversarial algorithm Exp3.P (Bubeck & Cesa-Bianchi 2012, Fig. 3.1) for the remaining rounds.

Formalization targets

Goal: Theorem 4.1, high-probability form

For every δ∈(0,1)\delta\in(0,1)δ∈(0,1) let β=10Kn3δ−1\beta=10Kn^3\delta^{-1}β=10Kn3δ−1. With probability at least 1−δ1-\delta1−δ, SAO with parameter β\betaβ satisfies, in the stochastic model (whenever some arm has Δi>0\Delta_i>0Δi​>0),

R‾n≤260K(1+log⁡K)log⁡2(β)Δ,\overline R_n\le\frac{260K(1+\log K)\log^2(\beta)}{\Delta},Rn​≤Δ260K(1+logK)log2(β)​,

and, against every adaptive adversary with rewards in [0,1][0,1][0,1],

Rn≤60(1+log⁡K)(1+log⁡n)nKlog⁡(β)+5K2log⁡2(β)+200K2log⁡2(β).R_n\le60(1+\log K)(1+\log n)\sqrt{nK\log(\beta)+5K^2\log^2(\beta)}+200K^2\log^2(\beta).Rn​≤60(1+logK)(1+logn)nKlog(β)+5K2log2(β)​+200K2log2(β).

Milestones

The milestones follow the paper's proof in order:

  • Freedman's inequality (Theorem 4.3) in the paper's two-sided form, and its variance-adaptive form, Lemma 4.4.
  • The concentration lemmas for SAO's estimates (Lemmas 4.5, 4.6, 4.7) and the Exp3.P phase (Lemma 4.8).
  • The two good events of §4.1, (21)–(25).
  • The deterministic consequences on those events: Exp3.P is never started in the stochastic model; suboptimal arms are deactivated by time 260Klog⁡(β)/Δi2260K\log(\beta)/\Delta_i^2260Klog(β)/Δi2​; ∑iqi≤1+log⁡K\sum_iq_i\le1+\log K∑i​qi​≤1+logK, (27); and the adversarial regret bound of §4.3.
  • The two halves of Theorem 4.1.

Significance

The theorem shows that the stochastic and adversarial regret rates are not in conflict. A single algorithm, with no information about the model, gets O(Klog⁡Klog⁡2(n/δ)/Δ)O(K\log K\log^2(n/\delta)/\Delta)O(KlogKlog2(n/δ)/Δ) pseudo-regret on stochastic rewards and O~(nK)\tilde O(\sqrt{nK})O~(nK​) regret against adaptive adversaries. Each rate is within polylogarithmic factors of optimal for its model. Later algorithms improved the logarithmic factors and removed the explicit switching, but they are compared against this result.

The theorem is proved in the paper. It is not known to have a machine-checked proof. Formalizing it requires a precise model of an adaptive adversary interacting with a randomized algorithm, martingale concentration with random variance (Lemma 4.4), and an exact statement of SAO including its boundary cases. The pieces are reusable: the interaction model, the estimators, Exp3.P and its high-probability guarantee all apply to other adversarial bandit results.

Difficulty

Neither standard analysis carries over. In the stochastic model, SAO's sampling probabilities are random and depend on the past, and a deactivated arm's probability keeps changing. Hoeffding-type bounds for a fixed sampling scheme therefore do not apply to H~i,t\widetilde H_{i,t}Hi,t​. The variance of the importance-weighted estimate grows like ∑s1/pi,s\sum_s1/p_{i,s}∑s​1/pi,s​, which is controlled only through the algorithm's own schedule (16). This is why Lemma 4.5 has the two-part radius with max⁡(t−τi,0)/(qiτit)\max(t-\tau_i,0)/(q_i\tau_it)max(t−τi​,0)/(qi​τi​t). In the adversarial model, the deterministic argument has to show that whenever the consistency tests pass, the regret accumulated before the switch is already small, for an adversary that adapts to the arms played. A union bound over all quantities, all arms and all times (§4.1) is needed before any deterministic reasoning, so every constant in the event matters.

Formalization scope

All declarations live in the namespace BestBothWorlds.SAO.

  • Arms and paths. Arms are Fin K and rounds are 1,…,n1,\dots,n1,…,n. An arm path is Fin n → Fin K.
  • Algorithms and adversaries. An algorithm is a deterministic map from the observed history to a probability vector. A deterministic adaptive adversary is a map from the list of earlier arms to a reward vector; randomized adversaries are mixtures of these.
  • Probabilities. For a fixed adversary, the probability of an event is ∑I∈E∏tpIt,t\sum_{I\in E}\prod_tp_{I_t,t}∑I∈E​∏t​pIt​,t​. In the stochastic model this is integrated against the product law of the reward table.
  • Logarithms and constants. Real.log is the natural logarithm. All constants of Theorem 4.1 are explicit, with β=10Kn3δ−1\beta=10Kn^3\delta^{-1}β=10Kn3δ−1.
  • SAO. It is defined exactly as Algorithm 1. Arms are tested in order within a round, and the active set changes during the loop. Test (13) is false when Ti(t)=0T_i(t)=0Ti​(t)=0, and test (14) is false when τi=1\tau_i=1τi​=1.
  • Exp3.P. After the switch, Exp3.P runs from scratch for n−τ0n-\tau_0n−τ0​ rounds. Its parameters are those of Bubeck–Cesa-Bianchi Theorem 3.2 with confidence K/βK/\betaK/β, and γ\gammaγ and βP\beta_{\mathrm P}βP​ are clipped at 111.
  • §4 notation. τ0\tau_0τ0​, τi←min⁡(τi,τ0)\tau_i\leftarrow\min(\tau_i,\tau_0)τi​←min(τi​,τ0​) and qi=pi,min⁡(τi,τ0)q_i=p_{i,\min(\tau_i,\tau_0)}qi​=pi,min(τi​,τ0​)​ are computed from the run, never assumed.

A trivializing formalization is ruled out. The goal's hypotheses concern only the instance (KKK, nnn, δ\deltaδ, the distributions or the adversary). The algorithm's quantities (τ0\tau_0τ0​, τi\tau_iτi​, qiq_iqi​, the sampling probabilities) are computed by the definition of SAO and are never free variables or hypotheses. The adversarial half covers adaptive adversaries, not only oblivious reward tables.

The expectation form of Theorem 4.1 (O(⋅)O(\cdot)O(⋅) bounds with β=n4\beta=n^4β=n4), Theorem 1.1 and the two-armed warm-up of §3 are out of scope. Proofs of any milestone are welcome, as are alternative proofs of the concentration lemmas from Mathlib's martingale library.

Selected references

  • S. Bubeck and A. Slivkins, The best of both worlds: stochastic and adversarial bandits, COLT 2012; arXiv:1202.4473v1. https://arxiv.org/abs/1202.4473
  • S. Bubeck and N. Cesa-Bianchi, Regret Analysis of Stochastic and Nonstochastic Multi-armed Bandit Problems, Foundations and Trends in Machine Learning 5(1), 2012. https://doi.org/10.1561/2200000024
  • D. A. Freedman, On tail probabilities for martingales, Annals of Probability 3(1), 1975. https://doi.org/10.1214/aop/1176996452
  • P. Auer, N. Cesa-Bianchi and P. Fischer, Finite-time analysis of the multiarmed bandit problem, Machine Learning 47, 2002. https://doi.org/10.1023/A:1013689704352
  • P. Auer, N. Cesa-Bianchi, Y. Freund and R. E. Schapire, The nonstochastic multiarmed bandit problem, SIAM Journal on Computing 32(1), 2002. https://doi.org/10.1137/S0097539701398375
  • Y. Seldin and A. Slivkins, One practical algorithm for both stochastic and adversarial bandits, ICML 2014. https://proceedings.mlr.press/v32/seldinb14.html
  • J. Zimmert and Y. Seldin, Tsallis-INF: an optimal algorithm for stochastic and adversarial bandits, JMLR 22, 2021. https://jmlr.org/papers/v22/19-753.html
20 thms2 active usersReviewed
🏆Completed
ProbabilityStochastic SystemsTheoretical Computer Science·Captain: mikedeng1

Approximation Algorithms for Stochastic Inventory Control Models 1: The Dual-Balancing Policy Costs at Most Twice the OptimumResearch Paper

Motivation

Periodic-review inventory control with backorders is one of the basic models of operations research: in each period a manager decides how much to order, orders arrive after a lead time, unmet demand is backlogged at a penalty, and stock left over is charged a holding cost. When demands in different periods are independent, dynamic programming yields an optimal base-stock policy and computing it is tractable. In practice demands are correlated and forecasts evolve over time, for example under the martingale model of forecast evolution (Heath and Jackson, 1994, doi:10.1080/07408179408966604). The dynamic program then has to range over all possible information states, whose number is typically exponential in the input (Zipkin, 2000), so optimal policies are out of reach and the heuristics in use came without performance guarantees.

Levi, Pál, Roundy and Shmoys (Math. Oper. Res. 32(2):284–302, 2007) gave the first policy for this model with a worst-case guarantee that holds for arbitrary correlated, nonstationary demand distributions: the dual-balancing policy costs at most twice the optimum in expectation. The analysis rests on a marginal cost accounting that charges each order, at the time it is placed, all the holding cost its units will ever incur. This mission formalizes that guarantee.

Setting

There are TTT periods t=1,…,Tt = 1, \dots, Tt=1,…,T and a known lead time L≥0L \ge 0L≥0: an order placed in period ttt arrives in period t+Lt + Lt+L. Period ttt has a per-unit holding cost ht≥0h_t \ge 0ht​≥0 and a per-unit backlogging penalty pt≥0p_t \ge 0pt​≥0. Ordering costs are zero (ct=0c_t = 0ct​=0), which is the standing assumption of the paper's §4. The initial data are the net inventory ni0ni_0ni0​ and the pipeline orders q1−L,…,q0≥0q_{1-L}, \dots, q_0 \ge 0q1−L​,…,q0​≥0.

Demands D1,…,DTD_1, \dots, D_TD1​,…,DT​ are nonnegative random variables on a probability space with a filtration (Ft)(\mathcal F_t)(Ft​); Ft\mathcal F_tFt​ is the information at the beginning of period ttt, and DtD_tDt​ is Ft+1\mathcal F_{t+1}Ft+1​-measurable. A feasible policy PPP places orders QtP≥0Q^P_t \ge 0QtP​≥0 that are Ft\mathcal F_tFt​-measurable. Write D[s,t]=∑j=stDjD_{[s,t]} = \sum_{j=s}^t D_jD[s,t]​=∑j=st​Dj​ (with Dj=0D_j = 0Dj​=0 for j≤0j \le 0j≤0), Xt=ni0+∑j=1−Lt−1Qj−D[1,t−1]X_t = ni_0 + \sum_{j=1-L}^{t-1} Q_j - D_{[1,t-1]}Xt​=ni0​+∑j=1−Lt−1​Qj​−D[1,t−1]​ for the inventory position before ordering and Yt=Xt+QtY_t = X_t + Q_tYt​=Xt​+Qt​ after ordering.

The marginal holding cost of period ttt is the holding cost that the units ordered in ttt incur until the end of the horizon, and the marginal backlogging cost is the penalty incurred one lead time later:

HtP=∑j=t+LThj (QtP−(D[t,j]−XtP)+)+,ΠtP=pt+L (D[t,t+L]−YtP)+.H^P_t = \sum_{j=t+L}^{T} h_j\,\bigl(Q^P_t - (D_{[t,j]} - X^P_t)^+\bigr)^+, \qquad \Pi^P_t = p_{t+L}\,\bigl(D_{[t,t+L]} - Y^P_t\bigr)^+ .HtP​=j=t+L∑T​hj​(QtP​−(D[t,j]​−XtP​)+)+,ΠtP​=pt+L​(D[t,t+L]​−YtP​)+.

The cost of PPP is C(P)=∑t=1T−L(HtP+ΠtP)\mathcal C(P) = \sum_{t=1}^{T-L}(H^P_t + \Pi^P_t)C(P)=∑t=1T−L​(HtP​+ΠtP​); by Eq. (3) it differs from the total holding and backlogging cost only by a policy-independent nonnegative term.

A dual-balancing policy BBB orders nothing after period T−LT - LT−L, and in each period t≤T−Lt \le T - Lt≤T−L orders the quantity that balances the two conditional expected marginal costs:

E[HtB∣Ft]=E[ΠtB∣Ft]almost surely.E\bigl[H^B_t \mid \mathcal F_t\bigr] = E\bigl[\Pi^B_t \mid \mathcal F_t\bigr] \quad\text{almost surely.}E[HtB​∣Ft​]=E[ΠtB​∣Ft​]almost surely.

Formalization targets

Goal: Theorem 4.1

For every dual-balancing policy BBB and every feasible policy PPP,

E[C(B)]  ≤  2 E[C(P)].E[\mathcal C(B)] \;\le\; 2\,E[\mathcal C(P)] .E[C(B)]≤2E[C(P)].

The paper writes P=OPTP = OPTP=OPT; quantifying over all feasible PPP is the same statement whenever an optimum exists and needs no existence assumption.

Milestones

  1. Lemma 4.1. E[C(B)]=2∑t=1T−LE[Zt]E[\mathcal C(B)] = 2\sum_{t=1}^{T-L}E[Z_t]E[C(B)]=2∑t=1T−L​E[Zt​] with Zt=E[HtB∣Ft]Z_t = E[H^B_t \mid \mathcal F_t]Zt​=E[HtB​∣Ft​].
  2. Lemma 4.2. With TH={t:YtB<YtP}\mathcal T_H = \{t : Y^B_t < Y^P_t\}TH​={t:YtB​<YtP​}, ∑t∈THHtB≤∑t=1T−LHtP\sum_{t\in\mathcal T_H} H^B_t \le \sum_{t=1}^{T-L} H^P_t∑t∈TH​​HtB​≤∑t=1T−L​HtP​ on every realization.
  3. Lemma 4.3. With TΠ={t:YtB≥YtP}\mathcal T_\Pi = \{t : Y^B_t \ge Y^P_t\}TΠ​={t:YtB​≥YtP​}, ∑t∈TΠΠtB≤∑t=1T−LΠtP\sum_{t\in\mathcal T_\Pi} \Pi^B_t \le \sum_{t=1}^{T-L} \Pi^P_t∑t∈TΠ​​ΠtB​≤∑t=1T−L​ΠtP​ on every realization.

Two further items are not milestones. Eq. (3) states that, along every realization, the period-by-period holding and backlogging cost equals ∑t=1−L0Πt+H(−∞,0]+∑t=1T−L(Ht+Πt)\sum_{t=1-L}^{0}\Pi_t + H_{(-\infty,0]} + \sum_{t=1}^{T-L}(H_t + \Pi_t)∑t=1−L0​Πt​+H(−∞,0]​+∑t=1T−L​(Ht​+Πt​), which is why the cost of Eq. (4) is the right objective. The other states that a dual-balancing policy exists when hT>0h_T > 0hT​>0 and the demands are integrable, so the goal is not about an empty class.

Significance

The theorem gives a policy that is computable period by period, by a one-dimensional search, with a factor-two guarantee that holds for every joint demand distribution, including correlated, nonstationary and forecast-driven ones, where the optimal policy cannot be computed. The constant is tight: the paper exhibits instances where the ratio tends to two. The second mission of this series treats the stochastic lot-sizing problem of the same paper, which uses the same marginal cost accounting.

The result is proved in the paper; no machine-checked proof of it is known. Formalizing it produces a reusable model of the periodic-review backlogging system with lead times and adapted policies, a verified marginal cost identity, and a formal approximation guarantee for a stochastic inventory policy. The pathwise comparison lemmas are stated for arbitrary pairs of order sequences and so apply to other balancing-type policies.

Difficulty

The obvious attempt compares the two policies period by period. That fails: in a given period the dual-balancing policy may hold far more or far less inventory than the comparison policy, and neither the holding nor the backlogging cost of one period is bounded by the comparator's cost in that period. The comparison only works after re-charging holding costs to the period in which the units were ordered, which requires the identity Eq. (3) to be established exactly, including the pipeline units, the initial stock and the lead-time shift. The probabilistic step then needs the random index sets TH\mathcal T_HTH​ and TΠ\mathcal T_\PiTΠ​ to be determined by the information of period ttt, so that conditioning on Ft\mathcal F_tFt​ commutes with the indicators; this is where the nonanticipativity of both policies enters. The existence of a balancing quantity needs a measurable selection from conditional laws, and it fails without a positive late holding cost.

Formalization scope

  • Periods are integers (ℤ). Orders and demands are functions ℤ → Ω → ℝ; only periods 1,…,T1, \dots, T1,…,T are read, and the pipeline qtq_tqt​ is substituted for t≤0t \le 0t≤0.
  • Ordering costs are ct=0c_t = 0ct​=0 and there is no discounting, as in the paper's §4; the reduction of §4.6 from general instances is not formalized. The lead time LLL is general.
  • Information is an arbitrary Filtration ℤ to which demands are adapted with a one-period lag; the paper's information vectors are a special case, and randomized policies are covered when their randomness is part of the information.
  • Expected costs are lower Lebesgue integrals in [0,∞][0,\infty][0,∞], so an infinite expected cost is never read as 000.
  • The balancing condition carries integrability of HtBH^B_tHtB​ and ΠtB\Pi^B_tΠtB​, so a conditional expectation of a non-integrable cost (which Mathlib sets to 000) cannot satisfy it vacuously. The existence item rules out an empty policy class.
  • Lemmas 4.2 and 4.3 are pathwise and do not use the balancing rule. The comparator totals are the marginal totals of Eq. (4), which is the stronger reading.
  • Eq. (2) prints Xt+LX_{t+L}Xt+L​ and its restatement on p. 292 prints ptp_tpt​; both are typos, and the formalization uses XtX_tXt​ and pt+Lp_{t+L}pt+L​.

A complete development needs finite-sum manipulations for Eq. (3) and Lemma 4.2, conditional expectation (tower property, pulling out bounded Ft\mathcal F_tFt​-measurable factors) for Lemma 4.1 and the goal, and regular conditional distributions with a measurable selection for the existence item. Theorem 4.2 (the randomized policy for integer demands) is outside this mission.

Selected references

  • R. Levi, M. Pál, R. O. Roundy, D. B. Shmoys, Approximation Algorithms for Stochastic Inventory Control Models, Mathematics of Operations Research 32(2):284–302, 2007. doi:10.1287/moor.1060.0205
  • D. C. Heath, P. L. Jackson, Modeling the evolution of demand forecasts with application to safety stock analysis in production/distribution systems, IIE Transactions 26(3):17–30, 1994. doi:10.1080/07408179408966604
  • P. H. Zipkin, Foundations of Inventory Management, McGraw-Hill, 2000. ISBN 978-0-256-11379-7.
6 thms2 active usersReviewed
🏆Completed
ProbabilityStochastic Systems·Captain: mikedeng1

Quantifying the Bullwhip Effect in a Simple Supply Chain: The Impact of Forecasting, Lead Times, and Information 2: Without Shared Demand Information the Bullwhip Bound Is MultiplicativeResearch Paper

Motivation

The bullwhip effect is the observation that the variability of orders grows as one moves up a supply chain, from the retailer to the wholesaler, the distributor and the factory, even when customer demand is stable. It was documented in industry practice by Lee, Padmanabhan and Whang (Management Science, 1997), who identified demand forecasting as one of its main causes. Amplified order variability raises the safety stock, capacity and transportation costs of every upstream firm, so the question of how large the effect is, and what reduces it, is central to supply chain management.

Chen, Drezner, Ryan and Simchi-Levi (Management Science 46(3), 2000) quantified the effect for a retailer that forecasts with a moving average and follows an order-up-to policy. Their §3 asks whether sharing customer demand information with every stage removes the effect. Theorem 3.1 (the companion mission of this series) shows that it does not; Theorem 3.2, the goal of this mission, gives the lower bound for the chain in which no demand information is shared.

Setting

Time is indexed by the integers t∈Zt \in \mathbb Zt∈Z. The retailer faces i.i.d. demand

Dt=μ+ϵt,D_t = \mu + \epsilon_t,Dt​=μ+ϵt​,

where the error terms ϵt\epsilon_tϵt​ are independent and identically distributed from a symmetric distribution with mean 000 and variance σ2>0\sigma^2 > 0σ2>0.

Single-stage policy (§2). With p≥1p \ge 1p≥1 observations, a lead time LLL, a safety factor zzz and a constant CL,ρC_{L,\rho}CL,ρ​, the retailer forms the moving-average estimates

D^tL=L ∑i=1pDt−ip,σ^etL=CL,ρ∑i=1pet−i2p,et=Dt−D^t1,\hat D^L_t = L\,\frac{\sum_{i=1}^p D_{t-i}}{p}, \qquad \hat\sigma^L_{et} = C_{L,\rho}\sqrt{\frac{\sum_{i=1}^p e_{t-i}^2}{p}}, \qquad e_t = D_t - \hat D^1_t,D^tL​=Lp∑i=1p​Dt−i​​,σ^etL​=CL,ρ​p∑i=1p​et−i2​​​,et​=Dt​−D^t1​,

raises its inventory position to the order-up-to point yt=D^tL+zσ^etLy_t = \hat D^L_t + z\hat\sigma^L_{et}yt​=D^tL​+zσ^etL​, and so orders qt=yt−yt−1+Dt−1q_t = y_t - y_{t-1} + D_{t-1}qt​=yt​−yt−1​+Dt−1​. Orders may be negative: excess inventory is returned without cost.

Decentralized chain (§3). Stages k=1,2,…k = 1, 2, \dotsk=1,2,… form a serial chain; stage 1 is the retailer, and LkL_kLk​ is the lead time between stages kkk and k+1k+1k+1. No stage sees customer demand except the retailer. Stage kkk forecasts from the orders it receives,

D^t(1)=∑i=1pDt−ip,D^t(k)=∑j=0p−1qt−jk−1p(k≥2),\hat D^{(1)}_t = \frac{\sum_{i=1}^p D_{t-i}}{p}, \qquad \hat D^{(k)}_t = \frac{\sum_{j=0}^{p-1} q^{k-1}_{t-j}}{p} \quad (k \ge 2),D^t(1)​=p∑i=1p​Dt−i​​,D^t(k)​=p∑j=0p−1​qt−jk−1​​(k≥2),

uses the order-up-to point ytk=LkD^t(k)y^k_t = L_k\hat D^{(k)}_tytk​=Lk​D^t(k)​, and orders

qt1=yt1−yt−11+Dt−1,qtk=ytk−yt−1k+qtk−1(k≥2).q^1_t = y^1_t - y^1_{t-1} + D_{t-1}, \qquad q^k_t = y^k_t - y^k_{t-1} + q^{k-1}_t \quad (k \ge 2).qt1​=yt1​−yt−11​+Dt−1​,qtk​=ytk​−yt−1k​+qtk−1​(k≥2).

Formalization targets

Goal: Theorem 3.2 (Eq. (7))

For every stage k≥1k \ge 1k≥1 and every period ttt,

Var⁡(qtk)Var⁡(Dt)  ≥  ∏i=1k(1+2Lip+2Li2p2).\frac{\operatorname{Var}(q^k_t)}{\operatorname{Var}(D_t)} \;\ge\; \prod_{i=1}^{k}\left(1 + \frac{2L_i}{p} + \frac{2L_i^2}{p^2}\right).Var(Dt​)Var(qtk​)​≥i=1∏k​(1+p2Li​​+p22Li2​​).

The bound is the paper's, with its explicit constants. The paper asserts no tightness for this theorem, and none is claimed.

Milestone: Eq. (6)

For the single-stage policy with any safety factor zzz and any constant CL,ρC_{L,\rho}CL,ρ​,

Var⁡(qt)Var⁡(Dt)  ≥  1+2Lp+2L2p2.\frac{\operatorname{Var}(q_t)}{\operatorname{Var}(D_t)} \;\ge\; 1 + \frac{2L}{p} + \frac{2L^2}{p^2}.Var(Dt​)Var(qt​)​≥1+p2L​+p22L2​.

This is the i.i.d. case ρ=0\rho = 0ρ=0 of the paper's Theorem 2.2. With z=0z = 0z=0 and L=L1L = L_1L=L1​ the single-stage orders are the stage-1 orders of the chain, so Eq. (6) contains the case k=1k = 1k=1 of the goal.

Significance

The result. Theorem 3.2 is half of the paper's comparison between centralized and decentralized information. When demand information is shared, the amplification from the retailer to stage kkk in the i.i.d. case equals 1+2(∑i≤kLi)/p+2(∑i≤kLi)2/p21 + 2(\sum_{i\le k}L_i)/p + 2(\sum_{i\le k}L_i)^2/p^21+2(∑i≤k​Li​)/p+2(∑i≤k​Li​)2/p2 (Eq. (8)), which grows additively in the lead times. Without sharing, the lower bound (7) is a product over stages and grows multiplicatively. The paper concludes that centralizing demand information "can significantly reduce the bullwhip effect", and that the gap widens as one moves up the chain. Eq. (6) is the single-stage statement that forecasting with a moving average alone already amplifies variability, by a factor depending only on the ratio L/pL/pL/p.

Formalizing it. The paper gives no proof of Theorem 3.2; it refers to Ryan (1997, PhD thesis) and to Chen et al. (1998). A machine-checked proof would therefore supply the first self-contained, verified argument for the multiplicative bound. The Gaussian special case of Eq. (6) is already formalized on Prove2Me, in the Snyder–Shen chapter on the bullwhip effect (SupplyChainTheory.bullwhip_signal_processing at ρ=0\rho = 0ρ=0); that statement assumes Gaussian errors, whereas this mission assumes only symmetry, mean 000 and variance σ2\sigma^2σ2. Neither the multistage bound nor the symmetric-error version of Eq. (6) has a formal proof.

Difficulty

The natural first idea is induction on the stage: treat the orders of stage k−1k-1k−1 as the demand of stage kkk and apply the single-stage bound. That step fails, because the single-stage bound is a statement about i.i.d. demand, and the orders reaching stage k≥2k \ge 2k≥2 are not i.i.d.: they are autocorrelated, and stage kkk's moving average of those orders interacts with the correlation in a way that can raise or lower the variance. Whether the product bound survives depends on controlling that interaction at every stage. For Eq. (6), the safety-stock term zσ^etLz\hat\sigma^L_{et}zσ^etL​ is a nonlinear function of the demands, and only symmetry of the errors, not normality, is available to control its interaction with the linear part of the order.

Formalization scope

All objects live in the namespace ChenBullwhip.Decentralized.

  • IIDDemand P is the demand model on a probability space (Ω,P)(\Omega, P)(Ω,P): a constant mu, sigma > 0, and errors eps : ℤ → Ω → ℝ that are measurable, mutually independent (iIndepFun), identically distributed, symmetric (eps t and -eps t have the same law), in L2L^2L2, with mean 000 and variance sigma ^ 2. Demand is D t = mu + eps t. Variances are Mathlib's ProbabilityTheory.variance.
  • SingleStage defines D^tL\hat D^L_tD^tL​, ete_tet​, σ^etL\hat\sigma^L_{et}σ^etL​, yty_tyt​ and qtq_tqt​ of §2; CL,ρC_{L,\rho}CL,ρ​ is a free real parameter, as the paper does not fix it.
  • Chain defines the forecasts D^t(k)\hat D^{(k)}_tD^t(k)​ and the orders qtkq^k_tqtk​ by recursion on the stage, with the convention qt0=Dt−1q^0_t = D_{t-1}qt0​=Dt−1​, so that stage 1 orders yt1−yt−11+Dt−1y^1_t - y^1_{t-1} + D_{t-1}yt1​−yt−11​+Dt−1​. The recursion qtk=ytk−yt−1k+qtk−1q^k_t = y^k_t - y^k_{t-1} + q^{k-1}_tqtk​=ytk​−yt−1k​+qtk−1​ is not printed in the paper; it is the §2.2 order identity applied to a stage whose incoming demand is qtk−1q^{k-1}_tqtk−1​, as the sequence of events on p. 440 describes.

Disclosed hypotheses not on the page: p≥1p \ge 1p≥1 (a moving average needs an observation), σ>0\sigma > 0σ>0 (the paper divides by Var⁡(D)=σ2\operatorname{Var}(D) = \sigma^2Var(D)=σ2), and square-integrable errors (Mathlib's variance is 000 off L2L^2L2). The paper's model (1) asks μ≥0\mu \ge 0μ≥0; since μ\muμ affects no variance, no sign condition is imposed. Lead times are natural numbers. The statements hold in every period ttt, with no stationarity hypothesis.

Trivializing formalizations are excluded: the orders are computed from the demands, not posited processes with a given covariance; the variances are genuine because every random variable involved is square integrable; and the ratio's denominator is σ2>0\sigma^2 > 0σ2>0.

A complete development needs variance and covariance calculus for finite linear combinations of independent L2L^2L2 variables, and, for Eq. (6), the vanishing of the covariance between an odd and an even function of a symmetric random vector. Both are reusable well beyond this mission. Proofs of either target, and general lemmas on variances of linear filters of i.i.d. sequences, are welcome.

Selected references

  • F. Chen, Z. Drezner, J. K. Ryan, D. Simchi-Levi, Quantifying the Bullwhip Effect in a Simple Supply Chain: The Impact of Forecasting, Lead Times, and Information, Management Science 46(3):436–443, 2000. https://doi.org/10.1287/mnsc.46.3.436.12069
  • H. L. Lee, V. Padmanabhan, S. Whang, Information Distortion in a Supply Chain: The Bullwhip Effect, Management Science 43(4):546–558, 1997. https://doi.org/10.1287/mnsc.43.4.546
  • J. K. Ryan, Analysis of Inventory Models with Limited Demand Information, Ph.D. dissertation, Department of Industrial Engineering and Management Science, Northwestern University, 1997.
  • L. V. Snyder, Z.-J. M. Shen, Fundamentals of Supply Chain Theory, 2nd ed., Wiley, 2019, Chapter 13 (formalized on Prove2Me as SupplyChainTheory.*).
5 thms2 active usersReviewed
🏆Completed
Bandit AlgorithmsMachine Learning·Captain: mikedeng1

Regret Analysis of Stochastic and Nonstochastic Multi-armed Bandit Problems I: Pseudo-Regret of (α, ψ)-UCBTextbook

Motivation

The stochastic multi-armed bandit is the basic model of sequential decisions under uncertainty with partial feedback: a forecaster repeatedly picks one of KKK options and observes only the reward of the option it picked. It models clinical trials, ad placement, routing and dynamic pricing, and is the building block of many reinforcement-learning algorithms. The question is how much reward is lost, compared with always playing the best option, through having to learn which option is best.

Chapter 2 of Bubeck and Cesa-Bianchi's monograph (arXiv:1204.5721v2) answers this for upper confidence bound (UCB) strategies.

  • Lai and Robbins (1985) introduced upper confidence bounds and proved that the number of pulls of a suboptimal arm must grow at least logarithmically, with an explicit constant, for consistent strategies (doi:10.1016/0196-8858(85)90002-8).
  • Agrawal (1995) gave simpler sample-mean-based index policies with logarithmic regret (doi:10.2307/1427934).
  • Auer, Cesa-Bianchi and Fischer (2002) gave the finite-time analysis of UCB1 for bounded rewards (doi:10.1023/A:1013689704352).
  • Bubeck and Cesa-Bianchi (2012) present the (α,ψ)(\alpha,\psi)(α,ψ)-UCB family, whose analysis needs only a bound ψ\psiψ on the cumulant generating function of the rewards, and the Lai–Robbins lower bound for Bernoulli rewards.

Setting

There are K≥2K\ge2K≥2 arms. Arm iii has an unknown reward distribution νi\nu_iνi​ with mean μi\mu_iμi​. At each round t=1,2,…t=1,2,\dotst=1,2,… the forecaster selects an arm ItI_tIt​ based on the past and receives a reward drawn from νIt\nu_{I_t}νIt​​, independently of the past. Write μ∗=max⁡iμi\mu^*=\max_i\mu_iμ∗=maxi​μi​, Δi=μ∗−μi\Delta_i=\mu^*-\mu_iΔi​=μ∗−μi​ for the gap of arm iii, and Ti(n)T_i(n)Ti​(n) for the number of times arm iii is selected in rounds 1,…,n1,\dots,n1,…,n. The pseudo-regret is

R‾n=nμ∗−E∑t=1nμIt=∑i=1KΔi E Ti(n).\overline R_n=n\mu^*-\mathbb E\sum_{t=1}^n\mu_{I_t}=\sum_{i=1}^K\Delta_i\,\mathbb E\,T_i(n).Rn​=nμ∗−Et=1∑n​μIt​​=i=1∑K​Δi​ETi​(n).

Moment condition (2.2). There is a convex ψ:R→R\psi:\mathbb R\to\mathbb Rψ:R→R with ln⁡E eλ(X−EX)≤ψ(λ)\ln\mathbb E\,e^{\lambda(X-\mathbb EX)}\le\psi(\lambda)lnEeλ(X−EX)≤ψ(λ) and ln⁡E eλ(EX−X)≤ψ(λ)\ln\mathbb E\,e^{\lambda(\mathbb EX-X)}\le\psi(\lambda)lnEeλ(EX−X)≤ψ(λ) for all λ≥0\lambda\ge0λ≥0 and every arm's reward XXX. Its Legendre–Fenchel transform is ψ∗(ε)=sup⁡λ∈R(λε−ψ(λ))\psi^*(\varepsilon)=\sup_{\lambda\in\mathbb R}(\lambda\varepsilon-\psi(\lambda))ψ∗(ε)=supλ∈R​(λε−ψ(λ)). For [0,1][0,1][0,1] rewards one may take ψ(λ)=λ2/8\psi(\lambda)=\lambda^2/8ψ(λ)=λ2/8, for which ψ∗(ε)=2ε2\psi^*(\varepsilon)=2\varepsilon^2ψ∗(ε)=2ε2.

(α,ψ)(\alpha,\psi)(α,ψ)-UCB. With μ^i,s\hat\mu_{i,s}μ^​i,s​ the mean of the first sss rewards of arm iii, at round ttt select

It∈argmax⁡i[μ^i,Ti(t−1)+(ψ∗)−1(αln⁡tTi(t−1))].I_t\in\operatorname*{argmax}_{i}\Big[\hat\mu_{i,T_i(t-1)}+(\psi^*)^{-1}\Big(\frac{\alpha\ln t}{T_i(t-1)}\Big)\Big].It​∈iargmax​[μ^​i,Ti​(t−1)​+(ψ∗)−1(Ti​(t−1)αlnt​)].

Formalization targets

Goal: Theorem 2.1 (p. 11)

If the rewards satisfy (2.2), then (α,ψ)(\alpha,\psi)(α,ψ)-UCB with α>2\alpha>2α>2 satisfies, for every nnn,

R‾n≤∑i:Δi>0Δi(αln⁡nψ∗(Δi/2)+αα−2).\overline R_n\le\sum_{i:\Delta_i>0}\Delta_i\Big(\frac{\alpha\ln n}{\psi^*(\Delta_i/2)}+\frac{\alpha}{\alpha-2}\Big).Rn​≤i:Δi​>0∑​Δi​(ψ∗(Δi​/2)αlnn​+α−2α​).

This is the bound the book's proof establishes. The printed statement has α/(α−2)\alpha/(\alpha-2)α/(α−2) in place of Δi α/(α−2)\Delta_i\,\alpha/(\alpha-2)Δi​α/(α−2); see Formalization scope.

Milestones

  • The decomposition R‾n=∑iΔi E Ti(n)\overline R_n=\sum_i\Delta_i\,\mathbb E\,T_i(n)Rn​=∑i​Δi​ETi​(n) (p. 9).
  • The Cramér–Chernoff bound (2.3): P(μi−μ^i,s>ε)≤e−sψ∗(ε)\mathbb P(\mu_i-\hat\mu_{i,s}>\varepsilon)\le e^{-s\psi^*(\varepsilon)}P(μi​−μ^​i,s​>ε)≤e−sψ∗(ε).
  • The three-event lemma (2.5)–(2.7) from the proof of Theorem 2.1.
  • The bounded-reward bound (2.4): R‾n≤∑i:Δi>0(2αΔiln⁡n+αα−2)\overline R_n\le\sum_{i:\Delta_i>0}\big(\frac{2\alpha}{\Delta_i}\ln n+\frac{\alpha}{\alpha-2}\big)Rn​≤∑i:Δi​>0​(Δi​2α​lnn+α−2α​).
  • The comparison (2.8): 2(p−q)2≤kl(p,q)≤(p−q)2/(q(1−q))2(p-q)^2\le\mathrm{kl}(p,q)\le(p-q)^2/(q(1-q))2(p−q)2≤kl(p,q)≤(p−q)2/(q(1−q)).
  • Theorem 2.2: for every strategy with E Ti(n)=o(na)\mathbb E\,T_i(n)=o(n^a)ETi​(n)=o(na) on all Bernoulli instances, lim inf⁡nR‾n/ln⁡n≥∑i:Δi>0Δi/kl(μi,μ∗)\liminf_n\overline R_n/\ln n\ge\sum_{i:\Delta_i>0}\Delta_i/\mathrm{kl}(\mu_i,\mu^*)liminfn​Rn​/lnn≥∑i:Δi​>0​Δi​/kl(μi​,μ∗).

Significance

Theorem 2.1 says that the cost of learning grows only logarithmically in the horizon, with a constant set by how well each suboptimal arm can be told apart from the best one. Theorem 2.2 shows that, up to constants, this cannot be improved: for Bernoulli rewards any strategy that is good on every instance must pay ln⁡n\ln nlnn per suboptimal arm, with constant Δi/kl(μi,μ∗)\Delta_i/\mathrm{kl}(\mu_i,\mu^*)Δi​/kl(μi​,μ∗). By (2.8) this constant is at least of order 1/Δi1/\Delta_i1/Δi​, matching (2.4). Together they are the template for the analysis of most optimistic algorithms: KL-UCB, linear and contextual UCB, and UCB-style reinforcement learning.

All results of the chapter are classical and proved on paper. This mission makes them machine-checked in a common model. That model has an explicit pseudo-regret, an explicit (generalized) inverse of ψ∗\psi^*ψ∗, an explicit initialization rule for the algorithm, and a randomized-strategy model for the lower bound. Later chapters of the series and later papers on optimistic algorithms can build on it.

Difficulty

The deterministic part of the upper bound is short, so the main difficulty is probabilistic. The sample mean μ^i,Ti(t−1)\hat\mu_{i,T_i(t-1)}μ^​i,Ti​(t−1)​ is taken over a random number of samples that depends on the algorithm's past. A Chernoff bound for a fixed sample size does not apply to it directly. The proof needs a union bound over all possible sample sizes, together with the representation in which the sss-th reward of each arm is a fixed random variable. Summing the resulting tail t1−αt^{1-\alpha}t1−α over rounds is where α>2\alpha>2α>2 enters.

The lower bound needs a change-of-measure argument between two Bernoulli instances, applied to a forecaster that may be randomized and never knows the horizon. Expressing "the forecaster cannot distinguish the instances" requires the law of the whole interaction under two environments.

Formalization scope

Model. Arms are Fin K with 2≤K2\le K2≤K, rounds are 1,2,…1,2,\dots1,2,…, and the natural logarithm is used. Rewards are a stack: Xi,kX_{i,k}Xi,k​ is the reward of the (k+1)(k+1)(k+1)-st pull of arm iii, all mutually independent, identically distributed per arm. For every strategy this gives the same law of arms and rewards as the book's protocol. μ∗\mu^*μ∗, Δi\Delta_iΔi​ and Ti(t)T_i(t)Ti​(t) are the published definitions of ImprovedLinBandits.UCBDelta.armModel. The pseudo-regret is (2.1), nμ∗−E∑tμItn\mu^*-\mathbb E\sum_t\mu_{I_t}nμ∗−E∑t​μIt​​, not the expected regret. The arms played are measurable random variables, so that every expectation is genuine.

ψ∗\psi^*ψ∗ and its inverse. ψ∗\psi^*ψ∗ takes values in the extended reals. (ψ∗)−1(y)=inf⁡{ε≥0:ψ∗(ε)≥y}(\psi^*)^{-1}(y)=\inf\{\varepsilon\ge0:\psi^*(\varepsilon)\ge y\}(ψ∗)−1(y)=inf{ε≥0:ψ∗(ε)≥y}.

Algorithm. The index is undefined while Ti(t−1)=0T_i(t-1)=0Ti​(t−1)=0. Unplayed arms are therefore played first, so each arm is played once in rounds 1,…,K1,\dots,K1,…,K. Ties are broken arbitrarily.

Corrected misprint. The book prints Theorem 2.1 with constant term α/(α−2)\alpha/(\alpha-2)α/(α−2). Its proof yields Δi α/(α−2)\Delta_i\,\alpha/(\alpha-2)Δi​α/(α−2) (bound on E Ti(n)\mathbb E\,T_i(n)ETi​(n) times Δi\Delta_iΔi​). The printed form is false for Gaussian rewards with large gaps. The goal states the proof's version. (2.4) is correct as printed because Δi≤1\Delta_i\le1Δi​≤1.

Constants. No O(·) appears in the chapter's statements. The constants are the book's: α/(α−2)\alpha/(\alpha-2)α/(α−2) in Theorem 2.1 and (2.4), and the factor 222 in (2.4) and (2.8).

Conventions made explicit.

  1. ψ(λ)≥ψ(0)\psi(\lambda)\ge\psi(0)ψ(λ)≥ψ(0) for λ≤0\lambda\le0λ≤0. Condition (2.2) constrains ψ\psiψ only on λ≥0\lambda\ge0λ≥0, while ψ∗\psi^*ψ∗ takes the supremum over all of R\mathbb RR. Without this convention, (2.3) and Theorem 2.1 are false (for example ψ(λ)=cλ+λ2/8\psi(\lambda)=c\lambda+\lambda^2/8ψ(λ)=cλ+λ2/8 with large ccc).
  2. ψ∗\psi^*ψ∗ is finite on [0,∞)[0,\infty)[0,∞).
  3. ψ∗(Δi/2)>0\psi^*(\Delta_i/2)>0ψ∗(Δi​/2)>0 for suboptimal arms.
  4. ε≥0\varepsilon\ge0ε≥0 in (2.3).
  5. q∈(0,1)q\in(0,1)q∈(0,1) in (2.8).

Conventions 1–3 hold for every ψ\psiψ the book uses.

Theorem 2.2. The forecaster is a measurable rule from the history and a fresh uniform seed to an arm. It does not depend on the horizon. Consistency is required on every Bernoulli instance, every suboptimal arm and every a>0a>0a>0. A term with μ∗=1\mu^*=1μ∗=1 (kl=+∞\mathrm{kl}=+\inftykl=+∞) is 000, and the lim inf⁡\liminfliminf is taken in the extended reals. The book proves only K=2K=2K=2.

Ruled out. The algorithm cannot see unplayed rewards (its index uses only the sample means of rewards already received). Non-measurable arm choices, which would make the expectations vanish in Lean, are excluded. Consistency cannot be assumed only on the instance of the conclusion.

Needed infrastructure. Cramér–Chernoff bounds for sums of i.i.d. variables, Hoeffding's lemma, the stack representation of bandit interactions, and the divergence decomposition for randomized strategies. All are reusable beyond this mission, and contributions of any of them are welcome.

Selected references

  • S. Bubeck, N. Cesa-Bianchi, Regret Analysis of Stochastic and Nonstochastic Multi-armed Bandit Problems, Foundations and Trends in Machine Learning 5(1), 2012. arXiv:1204.5721v2, doi:10.1561/2200000024
  • T. L. Lai, H. Robbins, Asymptotically efficient adaptive allocation rules, Advances in Applied Mathematics 6, 1985. doi:10.1016/0196-8858(85)90002-8
  • R. Agrawal, Sample mean based index policies with O(log n) regret for the multi-armed bandit problem, Advances in Applied Probability 27, 1995. doi:10.2307/1427934
  • P. Auer, N. Cesa-Bianchi, P. Fischer, Finite-time analysis of the multiarmed bandit problem, Machine Learning 47, 2002. doi:10.1023/A:1013689704352
12 thms2 active usersReviewed
Markov ChainProbabilityStochastic Systems·Captain: mikedeng1

On the Stochastic Matrices Associated with Certain Queuing Processes 1: The M/G/1 Imbedded Chain Is Ergodic iff ρ < 1 and Recurrent iff ρ ≤ 1Research Paper

Motivation

Many queues observed at well-chosen instants are Markov chains on the nonnegative integers. For the single-server queue with Poisson arrivals and general service times (M/G/1), D. G. Kendall showed in 1951 that the number of customers left behind at successive departure epochs is such a chain, the imbedded Markov chain (Kendall 1951; Kendall 1953). Whether the queue settles into a steady state, keeps returning to empty without settling, or grows without bound is then a question about this chain: is it ergodic, null recurrent, or transient?

F. G. Foster's 1953 paper (doi:10.1214/aoms/1177728976) answers this question by first proving general criteria for an irreducible chain on {0,1,2,… }\{0,1,2,\dots\}{0,1,2,…}, stated as solvability conditions for linear inequalities in the transition matrix, and then applying them to the M/G/1 and GI/M/1 chains. Theorem 2 of the paper is the drift condition now known as Foster's criterion, the starting point of the Lyapunov-function method for the stability of queues and stochastic networks (Meyn and Tweedie 2009). This mission is the M/G/1 half of the paper.

Timeline:

  • 1951–1953, Kendall. Introduces the imbedded chains of M/G/1 and GI/M/1 and obtains most of their classification by direct methods.
  • 1953, Foster. Derives the classification from general criteria: Theorem 2 (ergodicity), Theorems 4–6 (transience and recurrence).
  • 1950s onward. The criteria become the standard tools (Feller's text; later the drift conditions of Meyn and Tweedie).

Setting

A Markov chain on the states {0,1,2,… }\{0,1,2,\dots\}{0,1,2,…} is given by a transition matrix P=[pij]P=[p_{ij}]P=[pij​]: pij≥0p_{ij}\ge0pij​≥0 and ∑jpij=1\sum_j p_{ij}=1∑j​pij​=1 for every row iii. Write fij(n)f_{ij}^{(n)}fij(n)​ for the probability that the chain started in iii first reaches jjj (for i=ji=ji=j, first returns to jjj) at step n≥1n\ge1n≥1. The chain is irreducible if every state can be reached from every other, and aperiodic if for every state the return times have greatest common divisor 111. A state jjj is recurrent if fjj=∑nfjj(n)=1f_{jj}=\sum_n f_{jj}^{(n)}=1fjj​=∑n​fjj(n)​=1 and transient if fjj<1f_{jj}<1fjj​<1; a recurrent state is ergodic (positive recurrent, "recurrent-nonnull") if in addition its mean recurrence time ∑nnfjj(n)\sum_n n f_{jj}^{(n)}∑n​nfjj(n)​ is finite. The mean first-passage time from iii to jjj is μij=∑n≥1nfij(n)∈[0,∞]\mu_{ij}=\sum_{n\ge1} n f_{ij}^{(n)}\in[0,\infty]μij​=∑n≥1​nfij(n)​∈[0,∞].

The M/G/1 matrix is built from a sequence k0,k1,…k_0,k_1,\dotsk0​,k1​,… of positive numbers summing to one (knk_nkn​ is the probability of nnn arrivals during one service):

[pij]=[k0k1k2⋯k0k1k2⋯0k0k1⋯00k0⋯⋮⋮⋮],[p_{ij}] = \begin{bmatrix} k_0 & k_1 & k_2 & \cdots \\ k_0 & k_1 & k_2 & \cdots \\ 0 & k_0 & k_1 & \cdots \\ 0 & 0 & k_0 & \cdots \\ \vdots & \vdots & \vdots & \end{bmatrix},[pij​]=​k0​k0​00⋮​k1​k1​k0​0⋮​k2​k2​k1​k0​⋮​⋯⋯⋯⋯​​,

that is, p0j=kjp_{0j}=k_jp0j​=kj​ and, for i≥1i\ge1i≥1, pij=kj−i+1p_{ij}=k_{j-i+1}pij​=kj−i+1​ when j≥i−1j\ge i-1j≥i−1 and 000 otherwise. The traffic intensity is

ρ=∑n=1∞n kn∈[0,∞],\rho=\sum_{n=1}^{\infty}n\,k_n\in[0,\infty],ρ=n=1∑∞​nkn​∈[0,∞],

the mean number of arrivals per service.

Formalization targets

Goal: the M/G/1 classification (§3, p. 358)

the chain is ergodic  ⟺  ρ<1,the chain is recurrent  ⟺  ρ≤1.\text{the chain is ergodic}\iff\rho<1,\qquad\text{the chain is recurrent}\iff\rho\le1 .the chain is ergodic⟺ρ<1,the chain is recurrent⟺ρ≤1.

The goal leaves kkk arbitrary apart from positivity and normalization; in particular ρ=∞\rho=\inftyρ=∞ is allowed and falls in the transient case.

Milestones (the paper's general theorems and the step of §3 they feed)

  1. Theorem 2 (drift criterion): a nonnegative solution of ∑jpijyj≤yi−1\sum_j p_{ij}y_j\le y_i-1∑j​pij​yj​≤yi​−1 (i≠0i\ne0i=0) with ∑jp0jyj<∞\sum_j p_{0j}y_j<\infty∑j​p0j​yj​<∞ makes the system ergodic. Already posed on the platform and referenced here.
  2. Theorem 3: in an ergodic system the mean first-passage times dj=μj0d_j=\mu_{j0}dj​=μj0​ are finite and satisfy ∑j≥1pijdj=di−1\sum_{j\ge1}p_{ij}d_j=d_i-1∑j≥1​pij​dj​=di​−1 (i≠0i\ne0i=0), ∑j≥1p0jdj<∞\sum_{j\ge1}p_{0j}d_j<\infty∑j≥1​p0j​dj​<∞.
  3. §3 display: for the ergodic M/G/1 chain, μi,i−1=μ10\mu_{i,i-1}=\mu_{10}μi,i−1​=μ10​ and μi0=iμ10\mu_{i0}=i\mu_{10}μi0​=iμ10​ (i≠0i\ne0i=0).
  4. Theorem 5: a solution of ∑jpijyj≤yi\sum_j p_{ij}y_j\le y_i∑j​pij​yj​≤yi​ (i≠0i\ne0i=0) with yi→∞y_i\to\inftyyi​→∞ makes the system recurrent.
  5. Theorem 7: for a probability distribution {pn}\{p_n\}{pn​} with p0>0p_0>0p0​>0, ∑nznpn=z\sum_n z^np_n=z∑n​znpn​=z has a root in (0,1)(0,1)(0,1) iff ∑n≥1npn>1\sum_{n\ge1}np_n>1∑n≥1​npn​>1.
  6. Theorem 4: the system is transient iff ∑jpijyj=yi\sum_j p_{ij}y_j=y_i∑j​pij​yj​=yi​ (i≠0i\ne0i=0) has a bounded nonconstant solution.

Significance

The result. The classification is the stability theorem for the M/G/1 queue: for ρ<1\rho<1ρ<1 the departure-epoch queue length has a stationary distribution, which is what the Pollaczek–Khinchine formula describes; for ρ=1\rho=1ρ=1 the queue empties infinitely often but has no steady state; for ρ>1\rho>1ρ>1 it grows without bound. The general criteria behind it (Theorems 2, 4, 5) apply to any chain on the nonnegative integers and are reused in the companion GI/M/1 mission and throughout queueing and Markov-chain stability theory.

Formalizing it. All results here are proved on paper (Kendall and Foster, 1951–1953, with Theorems 3 and 7 classical lemmas from Feller). None of them is known to have a machine-checked proof against a Lean development of countable-state Markov chains. The mission produces such proofs on the published discrete-chain vocabulary (transition matrices, first-passage probabilities, return probabilities, positive recurrence), together with the general Foster criteria as reusable theorems. Theorem 2 is already posed as an open platform theorem and is reused here.

Difficulty

The queue-specific part of the argument is short once the general criteria are available; the weight of the mission is in those criteria. They relate qualitative properties of an infinite chain (ergodicity, recurrence, transience) to solvability of infinite systems of linear inequalities, and this needs limit behaviour of the nnn-step probabilities pij(n)p_{ij}^{(n)}pij(n)​ and of hitting probabilities of state 000, none of which follows from finite-state arguments. Two further points resist the naive approach. The converse directions (ergodic ⇒ρ<1\Rightarrow\rho<1⇒ρ<1, recurrent ⇒ρ≤1\Rightarrow\rho\le1⇒ρ≤1) need exact identities for mean first-passage times, not just bounds, and these must be handled in [0,∞][0,\infty][0,∞] because the means may be infinite. And the boundary case ρ=1\rho=1ρ=1 (null recurrence) separates the two equivalences: an argument that only compares the mean drift ρ−1\rho-1ρ−1 with 000, such as a law of large numbers for the increments, cannot tell recurrence from transience there.

Formalization scope

  • The chain is the published QueueingFundamentals.Foundations.TransitionMatrix (entries P.p i j, rows summing to 111 as a HasSum), with its firstPassage, returnProb, Irreducible, Aperiodic and PositiveRecurrent. "Ergodic" is P.PositiveRecurrent; aperiodicity is the paper's standing assumption and is not folded into it a second time.
  • States are indexed from 000, as in the paper; "i≠0i\ne0i=0" is i ≠ 0.
  • The M/G/1 matrix is a function mg1Matrix k : ℕ → ℕ → ℝ; the goal and the §3 display quantify over every TransitionMatrix P with P.p = mg1Matrix k. Such a P exists for every admissible k (checked in a sorry-free local file for ki=2−(i+1)k_i=2^{-(i+1)}ki​=2−(i+1)).
  • ∑nkn=1\sum_n k_n=1∑n​kn​=1 is added as the meaning of "stochastic matrix"; §3 writes only ki>0k_i>0ki​>0.
  • ρ\rhoρ and all mean first-passage times are extended nonnegative reals ([0,∞][0,\infty][0,∞]), so divergent means are ∞\infty∞, never 000. Theorem 7's mean is also taken in [0,∞][0,\infty][0,∞].
  • Recurrent means fjj=1f_{jj}=1fjj​=1 for every state jjj; transient means fjj<1f_{jj}<1fjj​<1 for every state. For irreducible chains these are complementary, which is a theorem, not a definition.
  • The general Theorems 3, 4 and 5 assume irreducibility and aperiodicity, the paper's standing assumption of §1. The goal does not assume them: they follow from ki>0k_i>0ki​>0.
  • Every series in a hypothesis carries its convergence (Summable or HasSum); Theorem 3's equation (6) is written as di=1+∑j≥1pijdjd_i=1+\sum_{j\ge1}p_{ij}d_jdi​=1+∑j≥1​pij​dj​ in [0,∞][0,\infty][0,∞] together with finiteness of the djd_jdj​, j≠0j\ne0j=0.
  • Theorem 7's distribution is renamed qqq in Lean to avoid a clash with pijp_{ij}pij​. Theorem 1 of the paper (§2) and Theorem 6 are not targets of this mission.

Ruled out: ρ\rhoρ as a real tsum (which is 000 for a divergent series and would call a heavy-tailed chain ergodic); defining "ergodic" or "recurrent" through the existence of Lyapunov or drift functions (which would make the criteria tautological); a goal over a matrix PPP that need not exist.

Contributions welcome: proofs of the general criteria (Theorems 2–5) on the published chain vocabulary, the limit theorem pij(n)→πjp_{ij}^{(n)}\to\pi_jpij(n)​→πj​ for irreducible aperiodic chains, first-step analysis for hitting times, and Theorem 7 as a lemma on probability generating functions; all of these are reusable beyond this mission.

Selected references

  • F. G. Foster, On the stochastic matrices associated with certain queuing processes, The Annals of Mathematical Statistics 24(3), 355–360, 1953. https://doi.org/10.1214/aoms/1177728976
  • D. G. Kendall, Some problems in the theory of queues, Journal of the Royal Statistical Society B 13(2), 151–185, 1951. https://doi.org/10.1111/j.2517-6161.1951.tb00093.x
  • D. G. Kendall, Stochastic processes occurring in the theory of queues and their analysis by the method of the imbedded Markov chain, The Annals of Mathematical Statistics 24(3), 338–354, 1953. https://doi.org/10.1214/aoms/1177728975
  • W. Feller, An Introduction to Probability Theory and Its Applications, Vol. I, Wiley, 1950.
  • S. Meyn and R. L. Tweedie, Markov Chains and Stochastic Stability, 2nd ed., Cambridge University Press, 2009. https://doi.org/10.1017/CBO9780511626630
9 thms2 active usersReviewed
🏆Completed
Optimization·Captain: mikedeng1

Supply Chain Coordination with Revenue-Sharing Contracts: Strengths and Limitations 3: The Optimal Wholesale-Price Contract with R′(q) = 1 − q^α Has Efficiency (2+α)/(1+α)^((1+α)/α)Research Paper

Motivation

A supplier who sells to a retailer at a per-unit wholesale price above her own production cost induces the retailer to order less than an integrated firm would. This effect, double marginalization, goes back to Spengler (1950) and is the standard benchmark against which supply chain contracts are judged: a contract coordinates the channel if it makes the decentralized decisions coincide with the integrated optimum. Revenue-sharing contracts, as used in the video-rental industry, coordinate the channel; the plain wholesale-price contract does not. Whether a supplier should bother with the administrative cost of revenue sharing depends on how much the wholesale-price contract actually loses and how much of the remaining profit the supplier keeps.

Cachon and Lariviere answer that question for a retailer whose revenue depends only on the quantity ordered, in Section 4.1.1 of their working paper Supply Chain Coordination with Revenue-Sharing Contracts: Strengths and Limitations (June 2000; the 2005 Management Science version renumbers and revises the material). They show that the answer is governed by the curvature of the marginal revenue curve, and they compute it exactly for a one-parameter family. The source is the June 2000 working paper, whose results are unnumbered; every item cites its section, page and display.

Setting

A supplier produces at unit cost c>0c > 0c>0 and sells to a single retailer. The retailer's expected revenue from qqq units is R(q)R(q)R(q), where R(0)=0R(0) = 0R(0)=0, RRR is strictly concave and differentiable on [0,∞)[0,\infty)[0,∞) with derivative R′R'R′ (the marginal revenue), R′R'R′ is differentiable on (0,∞)(0,\infty)(0,∞) with derivative R′′R''R′′, the product is viable (R′(0)>cR'(0) > cR′(0)>c), and a finite quantity is optimal (R′(q)<cR'(q) < cR′(q)<c for some qqq). The supply chain profit is Π(q)=R(q)−qc\Pi(q) = R(q) - qcΠ(q)=R(q)−qc; the integrated quantity qIq_IqI​ maximizes Π\PiΠ over q≥0q \ge 0q≥0.

Under a wholesale-price contract with price www, the retailer orders qqq to maximize R(q)−wqR(q) - wqR(q)−wq. Each order q≥0q \ge 0q≥0 is induced by exactly one price, w(q)=R′(q)w(q) = R'(q)w(q)=R′(q), so the supplier can be thought of as choosing qqq. Her profit, the retailer's profit, and their sum are then

πs(q)=q (R′(q)−c),πr(q)=R(q)−qR′(q),πs(q)+πr(q)=Π(q).\pi_s(q) = q\,(R'(q) - c), \qquad \pi_r(q) = R(q) - qR'(q), \qquad \pi_s(q) + \pi_r(q) = \Pi(q).πs​(q)=q(R′(q)−c),πr​(q)=R(q)−qR′(q),πs​(q)+πr​(q)=Π(q).

Following the paper, q↦R′(q)+qR′′(q)q \mapsto R'(q) + qR''(q)q↦R′(q)+qR′′(q) is assumed decreasing, which makes πs\pi_sπs​ unimodal. The supplier's optimal quantity to induce q∗q^*q∗ maximizes πs\pi_sπs​ over q≥0q \ge 0q≥0, and w(q∗)w(q^*)w(q∗) is her optimal wholesale price. The efficiency of the contract and the supplier's profit share are

πs(q∗)+πr(q∗)Π(qI)andπs(q∗)Π(q∗).\frac{\pi_s(q^*) + \pi_r(q^*)}{\Pi(q_I)} \qquad\text{and}\qquad \frac{\pi_s(q^*)}{\Pi(q^*)} .Π(qI​)πs​(q∗)+πr​(q∗)​andΠ(q∗)πs​(q∗)​.

In the α-family, R(q)=q−qα+1/(α+1)R(q) = q - q^{\alpha+1}/(\alpha+1)R(q)=q−qα+1/(α+1) for α>0\alpha > 0α>0 and q∈[0,1]q \in [0,1]q∈[0,1], so R′(q)=1−qαR'(q) = 1 - q^\alphaR′(q)=1−qα: marginal revenue is convex for α<1\alpha < 1α<1, linear for α=1\alpha = 1α=1 and concave for α>1\alpha > 1α>1.

Formalization targets

Goal: the α-family

For α>0\alpha > 0α>0 and 0<c<10 < c < 10<c<1, the quantities q∗=(1−c1+α)1/αq^* = \left(\frac{1-c}{1+\alpha}\right)^{1/\alpha}q∗=(1+α1−c​)1/α and qI=(1−c)1/αq_I = (1-c)^{1/\alpha}qI​=(1−c)1/α are the unique maximizers of πs\pi_sπs​ and Π\PiΠ on [0,1][0,1][0,1], the profit share is (1+α)/(2+α)(1+\alpha)/(2+\alpha)(1+α)/(2+α), and

πs(q∗)+πr(q∗)Π(qI)=2+α(1+α)1+αα,\frac{\pi_s(q^*) + \pi_r(q^*)}{\Pi(q_I)} = \frac{2+\alpha}{(1+\alpha)^{\frac{1+\alpha}{\alpha}}},Π(qI​)πs​(q∗)+πr​(q∗)​=(1+α)α1+α​2+α​,

a quantity that does not depend on ccc, is strictly increasing in α\alphaα, tends to 2/e2/e2/e as α→0+\alpha \to 0^+α→0+ and to 111 as α→∞\alpha \to \inftyα→∞.

Milestones for a general revenue function

  1. The price w(q)=R′(q)w(q) = R'(q)w(q)=R′(q) makes qqq the retailer's unique optimum (Eq. (9)).
  2. 0<q∗<qI0 < q^* < q_I0<q∗<qI​.
  3. w(q∗)=c−q∗R′′(q∗)w(q^*) = c - q^*R''(q^*)w(q∗)=c−q∗R′′(q∗), and w(q∗)>cw(q^*) > cw(q∗)>c.
  4. The profit share is at most (at least) 2/32/32/3 when R′R'R′ is convex (concave), strictly under strict convexity (concavity).
  5. 2q∗≤qI2q^* \le q_I2q∗≤qI​ (≥qI\ge q_I≥qI​) when R′R'R′ is convex (concave), strictly under strict convexity (concavity).
  6. Π(qI)−Π(q∗)=∫q∗qI(R′(z)−c) dz\Pi(q_I) - \Pi(q^*) = \int_{q^*}^{q_I}(R'(z) - c)\,dzΠ(qI​)−Π(q∗)=∫q∗qI​​(R′(z)−c)dz is at least (at most) 12πs(q∗)\tfrac12\pi_s(q^*)21​πs​(q∗) when R′R'R′ is convex (concave), strictly under strict convexity (concavity).

Milestones for the α-family

  1. The closed forms of q∗q^*q∗, qIq_IqI​, πr(q∗)\pi_r(q^*)πr​(q∗), πs(q∗)\pi_s(q^*)πs​(q∗) and Π(qI)\Pi(q_I)Π(qI​).
  2. E(α)=(2+α)/(1+α)(1+α)/αE(\alpha) = (2+\alpha)/(1+\alpha)^{(1+\alpha)/\alpha}E(α)=(2+α)/(1+α)(1+α)/α is strictly increasing on (0,∞)(0,\infty)(0,∞) with limits 2/e2/e2/e and 111.

Significance

The general milestones turn the paper's area argument (the triangle under the tangent to marginal revenue at q∗q^*q∗) into three comparisons: convex marginal revenue makes the wholesale-price contract worse for the chain and leaves the supplier at most two thirds of a smaller pie, concave marginal revenue the opposite. The α-family makes the trade-off exact: efficiency never falls below 2/e≈0.7362/e \approx 0.7362/e≈0.736, while the supplier's share (1+α)/(2+α)(1+\alpha)/(2+\alpha)(1+α)/(2+α) moves much faster than efficiency, which is the paper's argument for why revenue sharing is most attractive when marginal revenue is convex.

These results are proved on paper but, to our knowledge, not machine-checked anywhere; Mathlib has no supply chain contract theory. The formalization provides a reusable single-retailer wholesale-price model, a checked version of the convex/concave tangent comparisons, and a corrected statement of the α-family's monotonicity (see the scope section).

Difficulty

The general comparisons are short on paper but rest on a picture: they need the first-order condition at an interior maximizer, the tangent-line inequality for a convex or concave derivative, and the fundamental theorem of calculus for a function whose derivative is known only on a half-line and one-sided at 000. Strictness needs a strictly positive integrand on a nondegenerate interval.

The α-family is where the analysis is not routine. The closed forms involve real powers with exponents 1/α1/\alpha1/α and (1+α)/α(1+\alpha)/\alpha(1+α)/α, which must be combined carefully. The limit (1+α)1/α→e(1+\alpha)^{1/\alpha} \to e(1+α)1/α→e as α→0+\alpha \to 0^+α→0+ is classical, but the monotonicity of log⁡(2+α)−1+ααlog⁡(1+α)\log(2+\alpha) - \frac{1+\alpha}{\alpha}\log(1+\alpha)log(2+α)−α1+α​log(1+α) on all of (0,∞)(0,\infty)(0,∞) is not a one-line derivative sign check: the derivative mixes log⁡(1+α)/α2\log(1+\alpha)/\alpha^2log(1+α)/α2 with rational terms, and its sign has to be established uniformly near 000 and near ∞\infty∞.

Formalization scope

Quantities and prices are real numbers. "Optimal" always means a maximizer over all admissible quantities (IsMaxOn on [0,∞)[0,\infty)[0,∞), or on [0,1][0,1][0,1] in the α-family, as the page restricts), never a root of a first-order condition. The derivative R′R'R′ of the general model is linked to RRR by a one-sided derivative hypothesis on [0,∞)[0,\infty)[0,∞); R′′R''R′′ is required only on (0,∞)(0,\infty)(0,∞), since for α<1\alpha < 1α<1 it blows up at 000. In the α-family the marginal revenue is deriv of RRR, not a separate function, and 0<c<10 < c < 10<c<1 is assumed (implicit on the page: c>0c > 0c>0 and R′(0)=1>cR'(0) = 1 > cR′(0)=1>c). Efficiency and profit share are real divisions; their denominators are positive at the optimal quantities.

Deviations from the page, all disclosed in the items:

  • R(0)=0R(0) = 0R(0)=0 is added to the model. It is implicit in the paper's area reading of the retailer's profit, and the 2/32/32/3 comparison fails without it.
  • The paper states the curvature comparisons strictly ("less (more) than 2/3rds", "q∗>qI/2q^* > q_I/2q∗>qI​/2 (<qI/2< q_I/2<qI​/2)", "more (less) than 50%") under convexity (concavity). Linear marginal revenue is both and gives equality, so each item states the weak inequality under convexity or concavity and the strict one under strict convexity or concavity.
  • Printed slip. The page says "Efficiency is a decreasing function of α, i.e., efficiency improves as the marginal revenue curve becomes more concave". EEE is in fact strictly increasing (E(0+)=2/e≈0.7358E(0^+) = 2/e \approx 0.7358E(0+)=2/e≈0.7358, E(1)=0.75E(1) = 0.75E(1)=0.75, E(10)≈0.858E(10) \approx 0.858E(10)≈0.858), as the second half of the sentence and the two limits say. The Lean states the increasing form; the milestone text is kept verbatim. The numerical gloss "2/e≈0.732/e \approx 0.732/e≈0.73" is not formalized.

A trivializing formalization is ruled out: the efficiency in the goal is the ratio of profits computed from RRR at the maximizers, not a definition equal to (2+α)/(1+α)(1+α)/α(2+\alpha)/(1+\alpha)^{(1+\alpha)/\alpha}(2+α)/(1+α)(1+α)/α, and the maximizers are characterized as unique argmaxes rather than assumed.

Welcome contributions: proofs of the general tangent comparisons, which are reusable for any concave revenue model; the real-analysis lemmas on (1+α)1/α(1+\alpha)^{1/\alpha}(1+α)1/α; and the α-family closed forms.

Selected references

  • G. P. Cachon and M. A. Lariviere, Supply Chain Coordination with Revenue-Sharing Contracts: Strengths and Limitations, working paper, June 2000. Published version: Management Science 51(1):30–44, 2005. https://doi.org/10.1287/mnsc.1040.0215
  • J. J. Spengler, Vertical Integration and Antitrust Policy, Journal of Political Economy 58(4):347–352, 1950. https://doi.org/10.1086/256964
  • M. A. Lariviere and E. L. Porteus, Selling to the Newsvendor: An Analysis of Price-Only Contracts, Manufacturing & Service Operations Management 3(4):293–305, 2001. https://doi.org/10.1287/msom.3.4.293.9971
10 thms2 active usersReviewed
🏆Completed
Optimization·Captain: mikedeng1

Supply Chain Coordination with Revenue-Sharing Contracts: Strengths and Limitations 4: With Retailer Effort, the Supplier Prefers the Wholesale-Price Contract Exactly When τ > 1/√2Research Paper

Motivation

A revenue-sharing contract {ϕ,w}\{\phi, w\}{ϕ,w} lets a supplier charge a retailer a wholesale price www per unit and, in addition, collect the share 1−ϕ1 - \phi1−ϕ of the retailer's revenue. The video-rental industry adopted such contracts at scale in the late 1990s, and Cachon and Lariviere showed that in a broad class of models they coordinate the supply chain: the retailer's privately optimal decisions coincide with those that maximize total channel profit, and the profit can be split arbitrarily between the firms (missions 1 and 2 of this series).

The same authors also studied where revenue sharing breaks down. The most practically relevant limitation is retailer effort: shelf space, service, store cleanliness and promotion raise demand, cost the retailer money, and cannot be written into a contract. Once the retailer gives away part of its revenue, it earns only a share of the return on its effort while still paying the whole cost. This mission formalizes Section 4.2 of the authors' working paper, which shows that revenue sharing then cannot coordinate the channel while leaving the supplier any profit, and, in an explicit linear-demand example, determines exactly when the supplier is better off with the plain wholesale-price contract.

The source is the June 2000 working paper (Cachon and Lariviere, Supply Chain Coordination with Revenue-Sharing Contracts: Strengths and Limitations), whose results are displayed claims inside numbered sections rather than numbered theorems; the milestones cite section, printed page and display. The published version appeared in Management Science 51(1), 2005.

Setting

General model (Sec. 4.2.1). A supplier produces at unit cost c>0c > 0c>0. The retailer chooses an order quantity q≥0q \ge 0q≥0 and an effort level e≥0e \ge 0e≥0 after observing the contract {ϕ,w}\{\phi, w\}{ϕ,w}. Expected revenue R(q,e)R(q, e)R(q,e) is continuous, differentiable, strictly increasing in eee and concave in qqq; effort costs the retailer g(e)g(e)g(e), where ggg is continuous, increasing, differentiable and convex with g(0)=0g(0) = 0g(0)=0. The profits of the integrated channel, the retailer and the supplier are

Π(q,e)=R(q,e)−g(e)−qc,πr(q,e)=ϕR(q,e)−g(e)−qw,(1−ϕ)R(q,e)+q(w−c).\Pi(q, e) = R(q, e) - g(e) - qc,\qquad \pi_r(q, e) = \phi R(q, e) - g(e) - qw,\qquad (1-\phi)R(q, e) + q(w - c).Π(q,e)=R(q,e)−g(e)−qc,πr​(q,e)=ϕR(q,e)−g(e)−qw,(1−ϕ)R(q,e)+q(w−c).

The integrated solution (qI,eI)(q_I, e_I)(qI​,eI​) maximizes Π\PiΠ over q,e≥0q, e \ge 0q,e≥0.

Linear example (Sec. 4.2.2). Inverse demand is P(q,e)=1−q+2τeP(q, e) = 1 - q + 2\tau eP(q,e)=1−q+2τe with an effort-impact parameter τ≥0\tau \ge 0τ≥0, revenue is R(q,e)=qP(q,e)R(q, e) = qP(q, e)R(q,e)=qP(q,e) and effort costs g(e)=e2g(e) = e^2g(e)=e2. For a share ϕ\phiϕ the supplier's profit when the retailer responds optimally to {ϕ,w}\{\phi, w\}{ϕ,w} is πs(w,ϕ)\pi_s(w, \phi)πs​(w,ϕ), and the supplier's optimal profit is

V(ϕ)=sup⁡w≥0πs(w,ϕ).V(\phi) = \sup_{w \ge 0} \pi_s(w, \phi).V(ϕ)=w≥0sup​πs​(w,ϕ).

The share ϕ=1\phi = 1ϕ=1 is the wholesale-price contract.

Formalization targets

Goal: the supplier's choice of contract

For 0≤τ<10 \le \tau < 10≤τ<1, 0<c<10 < c < 10<c<1 and every ϕ∈(0,1]\phi \in (0, 1]ϕ∈(0,1], the supremum defining V(ϕ)V(\phi)V(ϕ) is attained at the price w(ϕ)=ϕ((1−τ2)ϕ+c(1−ϕτ2))/(1+ϕ(1−2τ2))w(\phi) = \phi\big((1-\tau^2)\phi + c(1-\phi\tau^2)\big)/\big(1 + \phi(1-2\tau^2)\big)w(ϕ)=ϕ((1−τ2)ϕ+c(1−ϕτ2))/(1+ϕ(1−2τ2)), and

V(ϕ)=(1−c)24(1+ϕ(1−2τ2)).V(\phi) = \frac{(1 - c)^2}{4\big(1 + \phi(1 - 2\tau^2)\big)} .V(ϕ)=4(1+ϕ(1−2τ2))(1−c)2​.

Consequently VVV is strictly increasing on (0,1](0, 1](0,1] if τ>1/2\tau > 1/\sqrt 2τ>1/2​ (the wholesale-price contract is the supplier's unique best share), constant if τ=1/2\tau = 1/\sqrt 2τ=1/2​, and strictly decreasing if τ<1/2\tau < 1/\sqrt 2τ<1/2​, with V(ϕ)→(1−c)2/4V(\phi) \to (1-c)^2/4V(ϕ)→(1−c)2/4 as ϕ→0+\phi \to 0^+ϕ→0+.

Milestones

  1. Sec. 4.2.1, p. 22: with w=ϕcw = \phi cw=ϕc and ϕ<1\phi < 1ϕ<1 the retailer's optimal effort at qIq_IqI​ is below eIe_IeI​.
  2. Sec. 4.2.1, p. 22: if (qI,eI)(q_I, e_I)(qI​,eI​) is optimal for the retailer, then ϕ=1\phi = 1ϕ=1, w=cw = cw=c, and the supplier earns nothing.
  3. Sec. 4.2.2, p. 23: the retailer's unique optimal effort at quantity qqq is e(q)=ϕτqe(q) = \phi\tau qe(q)=ϕτq.
  4. Sec. 4.2.2, pp. 23–24: the retailer's reduced profit q[ϕ−q(ϕ−ϕ2τ2)−w]q[\phi - q(\phi - \phi^2\tau^2) - w]q[ϕ−q(ϕ−ϕ2τ2)−w], its unique joint optimum (q(w,ϕ),e(q(w,ϕ)))\big(q(w,\phi), e(q(w,\phi))\big)(q(w,ϕ),e(q(w,ϕ))) with q(w,ϕ)=(ϕ−w)/(2(ϕ−ϕ2τ2))q(w, \phi) = (\phi - w)/(2(\phi - \phi^2\tau^2))q(w,ϕ)=(ϕ−w)/(2(ϕ−ϕ2τ2)) for w<ϕw < \phiw<ϕ and 000 otherwise, and the optimal profit (ϕ−w)2/(4(ϕ−ϕ2τ2))(\phi - w)^2/(4(\phi - \phi^2\tau^2))(ϕ−w)2/(4(ϕ−ϕ2τ2)).
  5. Sec. 4.2.2, p. 24: the integrated retail price pI=(1+c(1−2τ2))/(2(1−τ2))p_I = (1 + c(1-2\tau^2))/(2(1-\tau^2))pI​=(1+c(1−2τ2))/(2(1−τ2)), increasing in ccc if τ<1/2\tau < 1/\sqrt 2τ<1/2​ and decreasing if τ>1/2\tau > 1/\sqrt 2τ>1/2​.
  6. Sec. 4.2.2, p. 24: πs(⋅,ϕ)\pi_s(\cdot, \phi)πs​(⋅,ϕ) is strictly concave where the retailer orders, and w(ϕ)w(\phi)w(ϕ) is its unique maximizer over w≥0w \ge 0w≥0.
  7. Sec. 4.2.2, p. 24: πs(w(ϕ),ϕ)=(1−c)2/(4(1+ϕ(1−2τ2)))\pi_s(w(\phi), \phi) = (1-c)^2/\big(4(1 + \phi(1-2\tau^2))\big)πs​(w(ϕ),ϕ)=(1−c)2/(4(1+ϕ(1−2τ2))).

Significance

The general result (milestones 1–2) is a clean impossibility statement: with non-contractible effort, the only contract in the revenue-sharing family that coordinates the channel is the wholesale-price contract at marginal cost, which leaves the supplier zero profit. It marks the boundary of the coordination results of the earlier sections, and contrasts with the price-dependent newsvendor, where revenue sharing does coordinate price and quantity because the cost of expanding demand is captured in the revenue function and shared by both firms.

The example turns the impossibility into a design rule. Because coordination is out of reach, the supplier compares contracts by her own profit, and the threshold τ=1/2\tau = 1/\sqrt 2τ=1/2​ separates two regimes: when effort matters a lot she should leave the retailer all revenue and charge only a wholesale price ("a smaller share of a larger pie"); when it matters little she should take as much revenue as possible. The same threshold governs the counterintuitive comparative static that the integrated channel's retail price falls as production cost rises.

All results are proved on paper in the source. None has a machine-checked proof; this mission produces the first. The example is a fully explicit two-stage optimization problem, so the formal development also yields a verified computation of a Stackelberg equilibrium with moral hazard that other contract-design missions can reuse.

Difficulty

The individual calculations are elementary, and the work lies in getting the optimization statements right. The page solves the retailer's problem sequentially (effort first, then quantity) and writes the supplier's objective by substituting closed forms. A faithful proof must instead show that these closed forms are global optima over the constrained domains: the retailer optimizes jointly over the quadrant q,e≥0q, e \ge 0q,e≥0, the corner q=0q = 0q=0 is optimal whenever w≥ϕw \ge \phiw≥ϕ, and the supplier's objective is a quadratic on w≤ϕw \le \phiw≤ϕ glued to the zero function on w≥ϕw \ge \phiw≥ϕ, which is not concave on all of w≥0w \ge 0w≥0. The first-order-condition argument of the general model similarly needs an interior integrated optimum and a strictly positive marginal effect of effort, which "strictly increasing in eee" alone does not provide.

Formalization scope

All quantities are real numbers. The general model is a structure RevShareCoord.Effort.Model carrying RRR, its partial derivatives, ggg, g′g'g′ and ccc; derivatives are one-sided within [0,∞)[0, \infty)[0,∞), and joint differentiability of RRR is replaced by its partial derivatives and joint continuity. The example lives in RevShareCoord.Effort.Linear. "Optimal" always means a maximizer over the whole admissible set (q,e≥0q, e \ge 0q,e≥0 for the retailer, w≥0w \ge 0w≥0 for the supplier), and the supplier's value V(ϕ)V(\phi)V(ϕ) is defined as the supremum of her attainable profits, not by the printed formula.

Deviations from the page, each disclosed in the item's Formalization Note:

  • τ<1\tau < 1τ<1 instead of τ∈[0,1]\tau \in [0, 1]τ∈[0,1]: at τ=1\tau = 1τ=1 the integrated problem is unbounded and pIp_IpI​ divides by zero. The page's "jointly concave in qqq and τ\tauτ" is read as qqq and eee.
  • 0<c<10 < c < 10<c<1: c>0c > 0c>0 is the standing assumption of Sec. 1, and c<1c < 1c<1 is needed for a positive integrated quantity.
  • ϕ∈(0,1]\phi \in (0, 1]ϕ∈(0,1] in the example: at ϕ=0\phi = 0ϕ=0 the retailer keeps no revenue and q(w,ϕ)q(w, \phi)q(w,ϕ) divides by zero. The page's optimal share "ϕ=0\phi = 0ϕ=0" for τ<1/2\tau < 1/\sqrt 2τ<1/2​ is stated as strict decrease on (0,1](0, 1](0,1] with the limit at 0+0^+0+.
  • The printed second derivative −(1−ϕ(1−2τ2))/(2ϕ2(1−ϕτ2)2)-\big(1 - \phi(1-2\tau^2)\big)/\big(2\phi^2(1-\phi\tau^2)^2\big)−(1−ϕ(1−2τ2))/(2ϕ2(1−ϕτ2)2) has a sign slip in the numerator; the Lean states −(1+ϕ(1−2τ2))/(2ϕ2(1−ϕτ2)2)-\big(1 + \phi(1-2\tau^2)\big)/\big(2\phi^2(1-\phi\tau^2)^2\big)−(1+ϕ(1−2τ2))/(2ϕ2(1−ϕτ2)2).
  • "Otherwise decreasing" fails at τ=1/2\tau = 1/\sqrt 2τ=1/2​, where VVV and pIp_IpI​ are constant; the trichotomy is stated.
  • In the general model, the integrated optimum is interior, ∂R/∂e>0\partial R/\partial e > 0∂R/∂e>0 at it, and, for milestone 1, πr(qI,⋅)\pi_r(q_I, \cdot)πr​(qI​,⋅) is strictly concave in eee (the page asserts this but it does not follow from the assumptions).

Plugging the printed w(ϕ)w(\phi)w(ϕ) into πs\pi_sπs​ and comparing across ϕ\phiϕ would turn the dichotomy into a statement about an arbitrary price schedule; the goal instead asserts that w(ϕ)w(\phi)w(ϕ) attains the supremum over all w≥0w \ge 0w≥0, with the retailer best-responding jointly in (q,e)(q, e)(q,e).

No external library beyond Mathlib's real analysis and convexity is needed. Contributions welcome: proofs of the milestones, reusable lemmas on maximizing strictly concave quadratics over orthants, and a generalization of milestone 2 to non-interior optima.

Selected references

  • G. P. Cachon, M. A. Lariviere, Supply Chain Coordination with Revenue-Sharing Contracts: Strengths and Limitations, working paper, June 2000. Published version: Management Science 51(1):30–44, 2005. https://doi.org/10.1287/mnsc.1040.0215
  • G. P. Cachon, Supply Chain Coordination with Contracts, in Handbooks in Operations Research and Management Science 11, 2003. https://doi.org/10.1016/S0927-0507(03)11006-7
  • S. Desiraju, S. Moorthy, Managing a Distribution Channel under Asymmetric Information with Performance Requirements, Management Science 43(12), 1997. https://doi.org/10.1287/mnsc.43.12.1628
10 thms2 active usersReviewed
🏆Completed
OptimizationProbabilityStochastic Systems·Captain: mikedeng1

Dimensioning Large Call Centers IV: Asymptotically Optimal Staffing under a Waiting-Cost ConstraintResearch Paper

Motivation

A call center has to decide how many agents to staff. In practice the decision is often posed as a service-level constraint rather than a cost trade-off: use the fewest agents for which the expected waiting cost, or the fraction of customers who wait, stays below a target. Borst, Mandelbaum and Reiman (CWI Report PNA-R0015, 2000; journal version in Operations Research 52(1), 2004, doi:10.1287/opre.1030.0081) treat this constraint problem in Section 8 of their paper, alongside the cost-minimization problem of Sections 5–7, and show that a simple square-root staffing rule solves it asymptotically as the arrival rate grows.

The rule matters because it is what practitioners use. Under the classical Erlang-C model, the exact optimum requires evaluating the Erlang-C formula over many staffing levels. The asymptotic rule replaces this with a single equation in the Halfin–Whitt function PPP: when the target is a delay probability ε\varepsilonε (Example 8.5 of the paper), it reduces to staffing λ/μ+P−1(ε)λ/μ\lambda/\mu + P^{-1}(\varepsilon)\sqrt{\lambda/\mu}λ/μ+P−1(ε)λ/μ​ servers.

Timeline. Erlang's formula for the M/M/N delay probability dates from 1917. Halfin and Whitt (Operations Research 29, 1981) identified the limit P(x)P(x)P(x) of the delay probability under square-root staffing N=λ/μ+xλ/μN = \lambda/\mu + x\sqrt{\lambda/\mu}N=λ/μ+xλ/μ​ with integer NNN. Jagers and Van Doorn (Operations Research Letters 5, 1986; SIAM Review 33, 1991) studied the continued Erlang loss and delay functions at non-integer numbers of servers, including their convexity, which is what lets the staffing problem be relaxed to a continuous one. Borst, Mandelbaum and Reiman (2000/2004) used these to prove asymptotic optimality of square-root rules for both the cost and the constraint formulations.

Setting

Customers arrive at rate λ\lambdaλ to NNN identical servers, each with service rate μ>0\mu > 0μ>0; μ\muμ is fixed while λ→∞\lambda \to \inftyλ→∞. Stability requires N>λ/μN > \lambda/\muN>λ/μ. A customer who waits ttt time units costs Dλ(t)D_\lambda(t)Dλ​(t), where Dλ(0)=0D_\lambda(0) = 0Dλ​(0)=0, DλD_\lambdaDλ​ is strictly increasing on [0,∞)[0,\infty)[0,∞) and ∫0∞Dλ(t)e−θt dt<∞\int_0^\infty D_\lambda(t)e^{-\theta t}\,dt < \infty∫0∞​Dλ​(t)e−θtdt<∞ for all θ>0\theta > 0θ>0.

The Erlang-C probability of waiting is

π(N,ν)=νNN!{(1−ν/N)∑n=0N−1νnn!+νNN!}−1,\pi(N,\nu) = \frac{\nu^N}{N!}\Big\{(1-\nu/N)\sum_{n=0}^{N-1}\frac{\nu^n}{n!} + \frac{\nu^N}{N!}\Big\}^{-1},π(N,ν)=N!νN​{(1−ν/N)n=0∑N−1​n!νn​+N!νN​}−1,

and the conditional waiting cost is G(N,λ)=(Nμ−λ)∫0∞Dλ(t)e−(Nμ−λ)t dtG(N,\lambda) = (N\mu-\lambda)\int_0^\infty D_\lambda(t)e^{-(N\mu-\lambda)t}\,dtG(N,λ)=(Nμ−λ)∫0∞​Dλ​(t)e−(Nμ−λ)tdt. The waiting cost per unit time with NNN servers is

K(N,λ)=λ π(N,λ/μ) G(N,λ).K(N,\lambda) = \lambda\,\pi(N,\lambda/\mu)\,G(N,\lambda).K(N,λ)=λπ(N,λ/μ)G(N,λ).

Given a target Mλ>0M_\lambda > 0Mλ​>0, the optimal staffing level is the least integer N>λ/μN > \lambda/\muN>λ/μ with K(N,λ)≤MλK(N,\lambda) \le M_\lambdaK(N,λ)≤Mλ​; call it Nλ∗N^*_\lambdaNλ∗​.

In the continuous parametrization Nλ(x)=λ/μ+xλ/μN_\lambda(x) = \lambda/\mu + x\sqrt{\lambda/\mu}Nλ​(x)=λ/μ+xλ/μ​, define Gλ(x)=λG(Nλ(x),λ)G_\lambda(x) = \lambda G(N_\lambda(x),\lambda)Gλ​(x)=λG(Nλ​(x),λ), the continuous Erlang-C function πλ(x)=H(Nλ(x),λ/μ)\pi_\lambda(x) = H(N_\lambda(x),\lambda/\mu)πλ​(x)=H(Nλ​(x),λ/μ) with H(M,α)={α∫0∞e−αtt(1+t)M−1dt}−1H(M,\alpha) = \{\alpha\int_0^\infty e^{-\alpha t}t(1+t)^{M-1}dt\}^{-1}H(M,α)={α∫0∞​e−αtt(1+t)M−1dt}−1, and Kλ(x)=πλ(x)Gλ(x)K_\lambda(x) = \pi_\lambda(x)G_\lambda(x)Kλ​(x)=πλ​(x)Gλ​(x). The Halfin–Whitt function is P(x)=1/(1+x/h(−x))P(x) = 1/(1 + x/h(-x))P(x)=1/(1+x/h(−x)) with h=ϕ/(1−Φ)h = \phi/(1-\Phi)h=ϕ/(1−Φ) the standard normal hazard rate. A staffing function xλ>0x_\lambda > 0xλ​>0 is judged by the rounding gap

Tλ(x)=min⁡{∣K(⌊Nλ(x)⌋,λ)−Mλ∣, ∣K(⌈Nλ(x)⌉,λ)−Mλ∣, ∣K(⌈Nλ(x)⌉,λ)−K(Nλ∗,λ)∣}.T_\lambda(x) = \min\big\{|K(\lfloor N_\lambda(x)\rfloor,\lambda) - M_\lambda|,\ |K(\lceil N_\lambda(x)\rceil,\lambda) - M_\lambda|,\ |K(\lceil N_\lambda(x)\rceil,\lambda) - K(N^*_\lambda,\lambda)|\big\}.Tλ​(x)=min{∣K(⌊Nλ​(x)⌋,λ)−Mλ​∣, ∣K(⌈Nλ​(x)⌉,λ)−Mλ​∣, ∣K(⌈Nλ​(x)⌉,λ)−K(Nλ∗​,λ)∣}.

It is asymptotically optimal when Tλ(xλ)/Mλ→0T_\lambda(x_\lambda)/M_\lambda \to 0Tλ​(xλ​)/Mλ​→0 as λ→∞\lambda\to\inftyλ→∞.

Formalization targets

Goal: Theorem 8.2 (rationalized regime)

Suppose that for some κ>0\kappa > 0κ>0 and γ∈(0,∞)\gamma \in (0,\infty)γ∈(0,∞), Gλ(κ)/Mλ→γG_\lambda(\kappa)/M_\lambda \to \gammaGλ​(κ)/Mλ​→γ, i.e. the waiting cost is comparable to the target. Let yλ∗>0y^*_\lambda > 0yλ∗​>0 solve P(y)Gλ(y)=MλP(y)G_\lambda(y) = M_\lambdaP(y)Gλ​(y)=Mλ​. Then

lim⁡λ→∞Tλ(yλ∗)Mλ=0.\lim_{\lambda\to\infty}\frac{T_\lambda(y^*_\lambda)}{M_\lambda} = 0.λ→∞lim​Mλ​Tλ​(yλ∗​)​=0.

Supporting milestones

  • Lemma C.1: GλG_\lambdaGλ​ is strictly convex and decreasing on (0,∞)(0,\infty)(0,∞).
  • Section 3: πλ(x)=π(Nλ(x),λ/μ)\pi_\lambda(x) = \pi(N_\lambda(x),\lambda/\mu)πλ​(x)=π(Nλ​(x),λ/μ) when Nλ(x)N_\lambda(x)Nλ​(x) is an integer.
  • Lemma 8.1: if zλ∗>0z^*_\lambda > 0zλ∗​>0 solves π^λ(z)G^λ(z)=Mλ\hat\pi_\lambda(z)\hat G_\lambda(z) = M_\lambdaπ^λ​(z)G^λ​(z)=Mλ​ and Kλ(zλ∗)/(π^λG^λ)(zλ∗)→1K_\lambda(z^*_\lambda)/(\hat\pi_\lambda\hat G_\lambda)(z^*_\lambda) \to 1Kλ​(zλ∗​)/(π^λ​G^λ​)(zλ∗​)→1, then Tλ(zλ∗)/Mλ→0T_\lambda(z^*_\lambda)/M_\lambda \to 0Tλ​(zλ∗​)/Mλ​→0.
  • Lemma B.1: PPP is strictly convex and decreasing on (0,∞)(0,\infty)(0,∞).
  • Eq. (17): lim sup⁡aλ/b=∞\limsup a_\lambda/b = \inftylimsupaλ​/b=∞ implies lim inf⁡P(aλ)/P(b)=0\liminf P(a_\lambda)/P(b) = 0liminfP(aλ​)/P(b)=0 and lim inf⁡πλ(aλ)/πλ(b)=0\liminf \pi_\lambda(a_\lambda)/\pi_\lambda(b) = 0liminfπλ​(aλ​)/πλ​(b)=0.
  • Lemma 4.1 (Halfin–Whitt): for bounded xλ>0x_\lambda > 0xλ​>0, πλ(xλ)/P(xλ)→1\pi_\lambda(x_\lambda)/P(x_\lambda) \to 1πλ​(xλ​)/P(xλ​)→1; with xλ→xx_\lambda \to xxλ​→x, πλ(xλ)/P(x)→1\pi_\lambda(x_\lambda)/P(x)\to 1πλ​(xλ​)/P(x)→1.

Further target: Theorem 8.6 (efficiency-driven regime)

If Gλ(κ)/Mλ→0G_\lambda(\kappa)/M_\lambda \to 0Gλ​(κ)/Mλ​→0 for every κ>0\kappa > 0κ>0 and yλ∗>0y^*_\lambda > 0yλ∗​>0 solves Gλ(y)=MλG_\lambda(y) = M_\lambdaGλ​(y)=Mλ​, then Tλ(yλ∗)/Mλ→0T_\lambda(y^*_\lambda)/M_\lambda \to 0Tλ​(yλ∗​)/Mλ​→0.

Significance

The theorem certifies the staffing rule used in workforce-management practice: the excess staffing is determined by one scalar equation involving the Gaussian function PPP and the scaled waiting cost, and rounding the resulting staffing level misses the constraint by a vanishing fraction of the target. Lemma 8.1 is a reusable framework: any approximation π^λG^λ\hat\pi_\lambda\hat G_\lambdaπ^λ​G^λ​ that is asymptotically exact at the proposed staffing level yields an asymptotically optimal rule, and the paper instantiates it in three regimes (Theorems 8.2, 8.6, 8.9).

The results are proved on paper. To the best of current knowledge none of them, nor the Halfin–Whitt limit for the continuous Erlang-C extension, has a machine-checked proof. A formalization would produce the first verified heavy-traffic limit of the Erlang-C delay probability, a verified continuous Erlang-C extension with its integer identity, and the convexity facts about PPP and GλG_\lambdaGλ​ that many staffing papers cite without proof.

Difficulty

The obvious argument is to quote Halfin and Whitt: the delay probability converges to P(x)P(x)P(x) under square-root staffing, so PPP can replace the Erlang-C formula. That limit, as published in 1981, is about integer server counts along sequences with a convergent excess-staffing parameter. The paper needs it for the continuous function HHH at non-integer server counts and for staffing functions that are merely bounded, and it also needs the identity H(N,ν)=π(N,ν)H(N,\nu) = \pi(N,\nu)H(N,ν)=π(N,ν) at integers and the monotonicity of πλ\pi_\lambdaπλ​ in xxx, both cited from Jagers and Van Doorn rather than proved. None of these is in Mathlib. A second obstacle is that the staffing function yλ∗y^*_\lambdayλ∗​ is defined only implicitly by an equation involving GλG_\lambdaGλ​, which depends on the arbitrary cost functions DλD_\lambdaDλ​; nothing a priori prevents it from escaping to infinity, outside the range where the Halfin–Whitt approximation applies. Finally, TλT_\lambdaTλ​ compares integer-level costs given by the Erlang-C formula with a continuous approximation, so both representations of the delay probability are in play at once.

Formalization scope

The queue itself is not formalized: there is no Markov chain and no waiting-time distribution. Every statement is about the closed-form waiting cost K(N,λ)K(N,\lambda)K(N,λ) with π\piπ given by the Erlang-C formula, exactly as the paper's analysis is. Conventions, all in the namespace DimCallCenters.Constraint:

  • lam : ℝ is the arrival rate (λ is a Lean keyword); limits are Filter.atTop in lam, with μ fixed. Objects indexed by λ (MλM_\lambdaMλ​, Nλ∗N^*_\lambdaNλ∗​, yλ∗y^*_\lambdayλ∗​) are functions of lam constrained only for lam > 0.
  • WaitModel packages μ > 0 and DλD_\lambdaDλ​ with Dλ(0)=0D_\lambda(0) = 0Dλ​(0)=0, strict monotonicity on [0,∞)[0,\infty)[0,∞), and integrability of Dλ(t)e−θtD_\lambda(t)e^{-\theta t}Dλ​(t)e−θt on (0,∞)(0,\infty)(0,∞) for θ > 0 (the paper's finiteness of GGG; integrability is required because Lean's integral of a non-integrable function is 0).
  • Nλ∗N^*_\lambdaNλ∗​ is a function Nstar : ℝ → ℕ given with its two defining properties (feasible; below every feasible integer level above λ/μ). yλ∗y^*_\lambdayλ∗​ and zλ∗z^*_\lambdazλ∗​ are any positive solutions of their equations; existence and uniqueness are not hypotheses.
  • In TλT_\lambdaTλ​ the round-down term is dropped when ⌊Nλ(x)⌋≤λ/μ\lfloor N_\lambda(x)\rfloor \le \lambda/\mu⌊Nλ​(x)⌋≤λ/μ (an unstable level where KKK is undefined). This can only enlarge TλT_\lambdaTλ​.
  • Asymptotic relations are limits of ratios. lim sup⁡=∞\limsup = \inftylimsup=∞ and lim inf⁡=0\liminf = 0liminf=0 are stated with ∃ᶠ ("frequently"), lim sup⁡<∞\limsup < \inftylimsup<∞ as eventual boundedness.
  • PPP is defined through explicit ϕ\phiϕ, Φ\PhiΦ, hhh; the formula also gives P(0)=1P(0) = 1P(0)=1, used in Lemma 4.1(2) at x=0x = 0x=0.
  • No hypothesis lim⁡N↓λ/μG(N,λ)=∞\lim_{N\downarrow\lambda/\mu}G(N,\lambda) = \inftylimN↓λ/μ​G(N,λ)=∞ is added: it is not needed for the statements here.

A trivializing formalization is ruled out: TλT_\lambdaTλ​ keeps all of the paper's terms and is never replaced by a smaller quantity, and the hypotheses are jointly satisfiable — Dλ(t)=aλ/μ tD_\lambda(t) = a\sqrt{\lambda/\mu}\,tDλ​(t)=aλ/μ​t with Mλ=MλM_\lambda = M\lambdaMλ​=Mλ satisfies (33) for every κ\kappaκ with γ=a/(μκM)\gamma = a/(\mu\kappa M)γ=a/(μκM).

Infrastructure needed: the continuous Erlang-C function and its integer identity; the Halfin–Whitt limit (a Gaussian approximation of Poisson/gamma tails); calculus facts about the normal hazard rate. These are reusable beyond this mission, notably by the sibling missions on the cost-minimization problem. Example 8.5 (delay-probability target with Dλ=1t>0D_\lambda = 1_{t>0}Dλ​=1t>0​) motivates the rule but violates the strict monotonicity of DλD_\lambdaDλ​, so it is not an instance of the theorem as stated. Contributions on any milestone, and on Theorem 8.9 (quality-driven regime, which needs Lemma 4.2), are welcome.

Selected references

  • S. Borst, A. Mandelbaum, M. I. Reiman, Dimensioning Large Call Centers, CWI Report PNA-R0015, 2000; Operations Research 52(1):17–34, 2004. https://doi.org/10.1287/opre.1030.0081
  • S. Halfin, W. Whitt, Heavy-Traffic Limits for Queues with Many Exponential Servers, Operations Research 29(3):567–588, 1981. https://doi.org/10.1287/opre.29.3.567
  • A. A. Jagers, E. A. Van Doorn, On the Continued Erlang Loss Function, Operations Research Letters 5:43–46, 1986.
  • A. A. Jagers, E. A. Van Doorn, Convexity of Functions which are Generalizations of the Erlang Loss Function and the Erlang Delay Function, SIAM Review 33:281–282, 1991.
18 thms2 active usersReviewed
🏆Completed
Algorithmic Game TheoryControl TheoryStochastic Systems·Captain: mikedeng1

Nonzero-Sum Stochastic Differential Games with Impulse Controls: A Verification Theorem with Applications 2: An Explicit Family of Nash Equilibria for the Linear Impulse GameResearch Paper

Motivation

Impulse control models an agent who acts on a random system through discrete interventions, each with a fixed cost: inventory replenishment, cash management, exchange-rate interventions by a central bank. With two agents whose objectives conflict, the problem becomes a nonzero-sum stochastic differential game with impulse controls. Before the work of Aïd, Basei, Callegaro, Campi and Vargiolu (Math. Oper. Res. 45(1), 2020; preprint arXiv:1605.00039), such games had no general verification theorem and few explicit equilibria; the paper supplies both. Its main application, formalized in this mission, is a game between two central banks with different targets for an exchange rate, a two-player version of the exchange-rate control models of Bertola, Runggaldier and Yasuda and of Cadenillas and Zapatero (references [10], [12] of the paper). The paper proves that this game has an explicit one-parameter family of Nash equilibria of threshold type, with closed-form payoffs.

Setting

The state is a real process. Without interventions it is x+σWsx+\sigma W_sx+σWs​, where WWW is a standard real Brownian motion and σ>0\sigma>0σ>0. Player 1 may shift it up by impulses δ∈Z1=[0,∞[\delta\in Z_1=[0,\infty[δ∈Z1​=[0,∞[, player 2 down by impulses δ∈Z2=]−∞,0]\delta\in Z_2=]-\infty,0]δ∈Z2​=]−∞,0]:

Xs=x+σWs+∑k:τ1,k≤sδ1,k+∑k:τ2,k≤sδ2,k.X_s=x+\sigma W_s+\sum_{k:\tau_{1,k}\le s}\delta_{1,k}+\sum_{k:\tau_{2,k}\le s}\delta_{2,k}.Xs​=x+σWs​+k:τ1,k​≤s∑​δ1,k​+k:τ2,k​≤s∑​δ2,k​.

Player 1 earns the running payoff f1(Xs)=Xs−s1f_1(X_s)=X_s-s_1f1​(Xs​)=Xs​−s1​, player 2 earns f2(Xs)=s2−Xsf_2(X_s)=s_2-X_sf2​(Xs​)=s2​−Xs​, with s1<s2s_1<s_2s1​<s2​. An impulse δ\deltaδ costs its author c+λ∣δ∣c+\lambda|\delta|c+λ∣δ∣ and pays the opponent c~+λ~∣δ∣\tilde c+\tilde\lambda|\delta|c~+λ~∣δ∣. Payoffs are discounted at rate ρ>0\rho>0ρ>0. The standing assumptions are c≥c~≥0c\ge\tilde c\ge0c≥c~≥0, λ≥λ~≥0\lambda\ge\tilde\lambda\ge0λ≥λ~≥0, (c,λ)≠(c~,λ~)(c,\lambda)\ne(\tilde c,\tilde\lambda)(c,λ)=(c~,λ~) and 1−λρ>01-\lambda\rho>01−λρ>0.

A strategy of player iii is a pair φi=(Ci,ξi)\varphi_i=(\mathcal C_i,\xi_i)φi​=(Ci​,ξi​): an open continuation region Ci⊆R\mathcal C_i\subseteq\mathbb RCi​⊆R and a continuous impulse map ξi:R→Zi\xi_i:\mathbb R\to Z_iξi​:R→Zi​. Player iii intervenes when the state leaves Ci\mathcal C_iCi​, applying the impulse ξi(state)\xi_i(\text{state})ξi​(state); player 1 has priority when both want to act. This rule defines the controlled process Xx;φ1,φ2X^{x;\varphi_1,\varphi_2}Xx;φ1​,φ2​ and the interventions (τi,k,δi,k)(\tau_{i,k},\delta_{i,k})(τi,k​,δi,k​) inductively. Player iii's payoff is

Ji(x;φ1,φ2)=Ex[∫0∞e−ρsfi(Xs) ds−∑ke−ρτi,k(c+λ∣δi,k∣)+∑ke−ρτj,k(c~+λ~∣δj,k∣)].J^i(x;\varphi_1,\varphi_2)=\mathbb E_x\Big[\int_0^\infty e^{-\rho s}f_i(X_s)\,ds-\sum_{k}e^{-\rho\tau_{i,k}}(c+\lambda|\delta_{i,k}|)+\sum_{k}e^{-\rho\tau_{j,k}}(\tilde c+\tilde\lambda|\delta_{j,k}|)\Big].Ji(x;φ1​,φ2​)=Ex​[∫0∞​e−ρsfi​(Xs​)ds−k∑​e−ρτi,k​(c+λ∣δi,k​∣)+k∑​e−ρτj,k​(c~+λ~∣δj,k​∣)].

A pair is xxx-admissible, (φ1,φ2)∈Φx(\varphi_1,\varphi_2)\in\Phi_x(φ1​,φ2​)∈Φx​, when these random variables are integrable, sup⁡t∣Xt∣\sup_t|X_t|supt​∣Xt​∣ has all moments, and neither player's interventions accumulate in finite time. A Nash equilibrium is an admissible pair from which no player gains by deviating to any strategy that keeps the pair admissible.

The explicit objects are θ=2ρ/σ2\theta=\sqrt{2\rho/\sigma^2}θ=2ρ/σ2​, η=(1−λρ)/ρ\eta=(1-\lambda\rho)/\rhoη=(1−λρ)/ρ and

F(y)=2y+θc−ηlog⁡η+yη−y,0<y<η.F(y)=2y+\theta c-\eta\log\frac{\eta+y}{\eta-y},\qquad 0<y<\eta .F(y)=2y+θc−ηlogη−yη+y​,0<y<η.

From the zero ξ\xiξ of FFF and a free parameter s~∈R\tilde s\in\mathbb Rs~∈R, the formulas (4.20)–(4.21) give thresholds xˉ1<xˉ2\bar x_1<\bar x_2xˉ1​<xˉ2​, targets x1∗,x2∗∈]xˉ1,xˉ2[x_1^*,x_2^*\in]\bar x_1,\bar x_2[x1∗​,x2∗​∈]xˉ1​,xˉ2​[, coefficients AijA_{ij}Aij​, the functions φi(y)=Ai1eθy+Ai2e−θy±(y−si)/ρ\varphi_i(y)=A_{i1}e^{\theta y}+A_{i2}e^{-\theta y}\pm(y-s_i)/\rhoφi​(y)=Ai1​eθy+Ai2​e−θy±(y−si​)/ρ and the piecewise candidates V~1,V~2\tilde V_1,\tilde V_2V~1​,V~2​ of (4.6). These are linear outside ]xˉ1,xˉ2[]\bar x_1,\bar x_2[]xˉ1​,xˉ2​[ and equal φi\varphi_iφi​ inside.

Formalization targets

Goal: Proposition 4.7

For every s~∈R\tilde s\in\mathbb Rs~∈R and every initial state x∈Rx\in\mathbb Rx∈R, the threshold strategies

φ1∗=(]xˉ1,+∞[, y↦max⁡(x1∗−y,0)),φ2∗=(]−∞,xˉ2[, y↦min⁡(x2∗−y,0))\varphi_1^*=\big(]\bar x_1,+\infty[,\ y\mapsto\max(x_1^*-y,0)\big),\qquad \varphi_2^*=\big(]-\infty,\bar x_2[,\ y\mapsto\min(x_2^*-y,0)\big)φ1∗​=(]xˉ1​,+∞[, y↦max(x1∗​−y,0)),φ2∗​=(]−∞,xˉ2​[, y↦min(x2∗​−y,0))

form an xxx-admissible Nash equilibrium, and

J1(x;φ1∗,φ2∗)=V~1(x),J2(x;φ1∗,φ2∗)=V~2(x).J^1(x;\varphi_1^*,\varphi_2^*)=\tilde V_1(x),\qquad J^2(x;\varphi_1^*,\varphi_2^*)=\tilde V_2(x).J1(x;φ1∗​,φ2∗​)=V~1​(x),J2(x;φ1∗​,φ2∗​)=V~2​(x).

Milestones

  1. (4.17). FFF has a unique zero ξ∈(0,η)\xi\in(0,\eta)ξ∈(0,η).
  2. Proposition 4.2. For every s~\tilde ss~, the explicit 8-uple (4.20) satisfies the order conditions (4.7) and the optimality and pasting conditions (4.8)–(4.9). Moreover, φ2′′\varphi_2''φ2′′​ changes sign exactly once in ]x2∗,xˉ2[]x_2^*,\bar x_2[]x2∗​,xˉ2​[.
  3. Lemma 4.6. The impulses (4.23) maximise δ↦V~i(x+δ)−c−λ∣δ∣\delta\mapsto\tilde V_i(x+\delta)-c-\lambda|\delta|δ↦V~i​(x+δ)−c−λ∣δ∣, and the intervention operators satisfy (4.24): {M1V~1−V~1<0}=]xˉ1,∞[\{\mathcal M_1\tilde V_1-\tilde V_1<0\}=]\bar x_1,\infty[{M1​V~1​−V~1​<0}=]xˉ1​,∞[ and {M2V~2−V~2<0}=]−∞,xˉ2[\{\mathcal M_2\tilde V_2-\tilde V_2<0\}=]-\infty,\bar x_2[{M2​V~2​−V~2​<0}=]−∞,xˉ2​[.
  4. Condition (v). The equilibrium pair is xxx-admissible for every xxx. This includes the integrability (4.26) of the discounted intervention costs.

Milestones 1–3 are deterministic real analysis; milestone 4 and the goal are probabilistic.

Significance

The result gives explicit equilibria, with explicit thresholds and payoffs, for a nonzero-sum stochastic game with impulse controls. Such games rarely have closed-form solutions. The equilibria form a continuum indexed by s~\tilde ss~: equilibrium payoffs are not unique, and every equilibrium in the family is a translate of a fixed interval policy. The explicit formulas also support the comparative statics of Section 4.4, where the continuation region widens as the fixed cost grows.

Proposition 4.7 is proved in the paper by applying its verification theorem (Theorem 3.3) to V~1,V~2\tilde V_1,\tilde V_2V~1​,V~2​. The admissibility estimate (4.26) is written out only for initial states x∈{x1∗,x2∗}x\in\{x_1^*,x_2^*\}x∈{x1∗​,x2∗​}; the general case is said to be similar. As far as is known, none of these results has a machine-checked proof. A formal proof would check every regularity, pasting and admissibility condition. It would also yield a reusable pathwise construction of impulse-controlled Brownian motion.

Difficulty

The deterministic milestones need careful algebra with nested logarithms and square roots. Proving Lemma 4.6 requires the global shape of V~i(y)±λy\tilde V_i(y)\pm\lambda yV~i​(y)±λy, which needs the sign pattern of φ2′′\varphi_2''φ2′′​ from Proposition 4.2, not only local conditions at the thresholds.

The goal is harder. The Nash inequality must hold against every admissible deviation: any open continuation region and any continuous impulse map, not only threshold strategies. Comparing payoffs therefore needs a verification argument, namely Itô's formula for a function that is C2C^2C2 only piecewise and C1C^1C1 across the thresholds, applied along a process with an unbounded number of jumps, plus a localisation that uses the moment condition (2.8). Admissibility needs a renewal-type bound on the discounted number of interventions, built from i.i.d. exit times of Brownian motion from an interval.

Formalization scope

The Lean development lives in the namespace ImpulseGames.LinearGame.

  • The constants and standing assumptions are a structure Model with the proposition Model.Standing. The added hypothesis c>0c>0c>0 appears in every statement that uses ξ\xiξ: with c=0c=0c=0, which the standing assumptions allow, FFF has no zero in (0,η)(0,\eta)(0,η) and the family (4.20) does not exist.
  • The zero ξ\xiξ is a parameter, constrained by ξ∈(0,η)\xi\in(0,\eta)ξ∈(0,η) and F(ξ)=0F(\xi)=0F(ξ)=0. Milestone 1 shows that exactly one such ξ\xiξ exists.
  • The Brownian motion is Mathlib's IsBrownianReal W P on a probability space, with time in R≥0\mathbb R_{\ge0}R≥0​. Definition 2.2 is formalized pathwise: exit times inf⁡{s>τ~k−1:X~sk−1∉Ci}\inf\{s>\tilde\tau_{k-1}:\tilde X^{k-1}_s\notin\mathcal C_i\}inf{s>τ~k−1​:X~sk−1​∈/Ci​} in [0,∞][0,\infty][0,∞] with inf⁡∅=∞\inf\emptyset=\inftyinf∅=∞, the tie rule favouring player 1, and e−ρ⋅∞=0e^{-\rho\cdot\infty}=0e−ρ⋅∞=0 for the tail of each impulse control. No SDE or stochastic integral appears in any statement; the uncontrolled dynamics are ζ+σ(Ws−Wt)\zeta+\sigma(W_s-W_t)ζ+σ(Ws​−Wt​).
  • Payoffs are Bochner expectations. Φx\Phi_xΦx​ requires every random variable of (2.7) to be integrable, so no deviation can obtain the default value 000 of a non-integrable expectation. The moment condition (2.8) is stated with an extended-real supremum, and (2.9) is read almost surely.
  • The Nash condition quantifies over all strategies of Definition 2.1. A formalization that restricts deviations to threshold strategies would be a different, weaker theorem and is excluded. Likewise, the equilibrium payoffs V~i\tilde V_iV~i​ are the explicit formulas (4.6), never defined as "the equilibrium value".

Two slips of the page are corrected. First, the paper's impulse maps ξi∗(y)=xi∗−y\xi_i^*(y)=x_i^*-yξi∗​(y)=xi∗​−y are not ZiZ_iZi​-valued on all of R\mathbb RR; they are replaced by max⁡(x1∗−y,0)\max(x_1^*-y,0)max(x1∗​−y,0) and min⁡(x2∗−y,0)\min(x_2^*-y,0)min(x2∗​−y,0), which agree with them wherever each player acts. Second, in Lemma 4.6 the maximiser (4.23) is not unique when λ=λ~\lambda=\tilde\lambdaλ=λ~, in the opponent's intervention region (all impulses tie there). The statement asserts maximality everywhere and uniqueness outside that region.

A complete development needs: elementary real analysis for milestones 1–3; a pathwise theory of piecewise-defined processes; an Itô formula for Brownian motion with C1C^1C1, piecewise-C2C^2C2 functions; and exit-time estimates for Brownian motion. The last two are reusable well beyond this mission. Proofs of the deterministic milestones are welcome independently of the stochastic part.

Selected references

  • R. Aïd, M. Basei, G. Callegaro, L. Campi, T. Vargiolu, Nonzero-sum stochastic differential games with impulse controls: a verification theorem with applications, Mathematics of Operations Research 45(1), 2020. https://doi.org/10.1287/moor.2019.0989 (accepted manuscript: arXiv:1605.00039v4, https://arxiv.org/abs/1605.00039)
  • G. Bertola, W. J. Runggaldier, K. Yasuda, On classical and restricted impulse stochastic control for the exchange rate, Applied Mathematics and Optimization 74(2), 423–454, 2016.
  • A. Cadenillas, F. Zapatero, Classical and impulse stochastic control of the exchange rate using interest rates and reserves, Mathematical Finance 10(2), 141–156, 2000.
  • B. Øksendal, A. Sulem, Applied Stochastic Control of Jump Diffusions, 2nd ed., Springer, 2007.
11 thms2 active usersReviewed
🏆Completed
Dynamic ProgrammingOptimizationProbability·Captain: mikedeng1

Discounted Dynamic Programming: An Optimal Stationary Plan Exists When the Action Set Is Essentially FiniteResearch Paper

Motivation

Sequential decisions often change the distribution of future states. A planner choosing an action today must account for both its immediate reward and the later rewards made possible by the resulting state. The mathematical question is whether an optimal rule can be chosen once and reused at every stage, even when a competing plan may randomize and use the entire observed history. In Discounted Dynamic Programming, Blackwell studies this question on general Borel state and action spaces, beyond the finite models in which a direct comparison of actions is available.

The paper distinguishes several strengths of optimality. For each distribution of the initial state, an approximately optimal stationary plan exists, but a single plan that is approximately optimal at every initial state need not exist in a general Borel problem. Essential countability of the actions restores uniform approximate stationary optimality; essential finiteness yields exact stationary optimality. These are different mathematical claims, and the mission keeps their different quantifiers visible. Blackwell 1965, pp. 227, 229, 232–234.

Setting

A state is an element sss of a nonempty standard Borel space SSS, and an action is an element aaa of a nonempty standard Borel space AAA. The transition kernel q(⋅∣s,a)q(\cdot\mid s,a)q(⋅∣s,a) gives a probability distribution for the next state after action aaa in state sss. The reward r(s,a,s′)∈Rr(s,a,s')\in\mathbb Rr(s,a,s′)∈R may depend on that next state s′s's′; it is bounded and Borel measurable. Future rewards are discounted by β\betaβ with 0≤β<10\le\beta<10≤β<1. These are the objects of Blackwell’s Sections 2–3. Blackwell 1965, pp. 227–228.

A plan π=(π1,π2,…)\pi=(\pi_1,\pi_2,\ldots)π=(π1​,π2​,…) assigns a probability distribution of actions to each possible history before a decision. At stage nnn, that history contains n−1n-1n−1 completed state-action pairs and the current state. Thus plans may randomize and depend on earlier states and actions. A Markov plan instead uses a Borel function fn:S→Af_n:S\to Afn​:S→A at each stage; a stationary plan uses the same function fff at every stage and is denoted f(∞)f^{(\infty)}f(∞). Starting from state sss, the plan has discounted expected return

I(π)(s)=∑n=1∞βn−1 Esπ[r(σn,αn,σn+1)].I(\pi)(s)=\sum_{n=1}^{\infty}\beta^{n-1}\,\mathbb E_s^\pi\bigl[r(\sigma_n,\alpha_n,\sigma_{n+1})\bigr].I(π)(s)=n=1∑∞​βn−1Esπ​[r(σn​,αn​,σn+1​)].

Here σn\sigma_nσn​ and αn\alpha_nαn​ are the state and action at stage nnn. The comparison class for an optimal plan is all such plans, including randomized and history-dependent ones. Blackwell 1965, pp. 228–229.

Two actions are equivalent at state sss when they have the same reward r(s,a,s′)r(s,a,s')r(s,a,s′) for every next state s′s's′ and the same transition measure q(⋅∣s,a)q(\cdot\mid s,a)q(⋅∣s,a). An action set is essentially countable by a Markov plan (f1,f2,…)(f_1,f_2,\ldots)(f1​,f2​,…) if, for every (s,a)(s,a)(s,a), one of the actions fn(s)f_n(s)fn​(s) is equivalent to aaa at sss. It is essentially finite by that plan if SSS has a countable Borel partition (Sn)(S_n)(Sn​) such that, for s∈Sns\in S_ns∈Sn​, one of f1(s),…,fn(s)f_1(s),\ldots,f_n(s)f1​(s),…,fn​(s) is equivalent to every action aaa at sss. A finite action set is a special case. Blackwell 1965, pp. 233–234.

Formalization targets

For a probability distribution ppp on SSS and ε>0\varepsilon>0ε>0, (p,ε)(p,\varepsilon)(p,ε)-optimality asks for a stationary fff with

p{s:I(π)(s)>I(f(∞))(s)+ε}=0for every plan π.p\{s:I(\pi)(s)>I(f^{(\infty)})(s)+\varepsilon\}=0\qquad\text{for every plan }\pi.p{s:I(π)(s)>I(f(∞))(s)+ε}=0for every plan π.

Theorem 6(b) asserts that such an fff always exists. Under essential countability, Theorem 7(a) obtains a stronger, uniform ε\varepsilonε-optimality statement: for every ε>0\varepsilon>0ε>0 there is a stationary fff with I(π)(s)≤I(f(∞))(s)+εI(\pi)(s)\le I(f^{(\infty)})(s)+\varepsilonI(π)(s)≤I(f(∞))(s)+ε for all π,s\pi,sπ,s. Its other targets identify the optimal return with the fixed point of the operator Uπu=sup⁡nTfnuU_\pi u=\sup_nT_{f_n}uUπ​u=supn​Tfn​​u and with the unique bounded solution of the optimality equation u=sup⁡a∈ATauu=\sup_{a\in A}T_auu=supa∈A​Ta​u. Blackwell 1965, pp. 232–234.

The mission’s goal is Theorem 7(b). Under essential finiteness, it asks for a stationary fff with exact optimality:

I(π)(s)≤I(f(∞))(s)for every plan π and state s.I(\pi)(s)\le I(f^{(\infty)})(s)\qquad\text{for every plan }\pi\text{ and state }s.I(π)(s)≤I(f(∞))(s)for every plan π and state s.

The milestone list also includes the paper’s operator identity, approximate selection result, contraction criterion, generated-plan comparison, and upper-bound criterion. Each has its own source index and statement. Blackwell 1965, pp. 231–234.

Significance

The exact result says that, under a condition weaker than a globally finite action set, repeated use of one measurable state-based rule matches or exceeds the return of every adaptive randomized plan. It is a structural result about what information and randomization can add to discounted control. The preceding approximate results specify what can still be guaranteed when that condition is relaxed; Blackwell’s examples show that the distinctions cannot simply be ignored. Blackwell 1965, pp. 229–230, 234.

Blackwell proved these statements in 1965. The formalization work here is to give machine-checked proofs for the Borel-space model and its full comparison class, together with reusable definitions of history-dependent kernels, returns, stationary rules, and Bellman operators. The draft theorem statements compile as Lean declarations, but their proofs remain open. The milestone results are intended to make both the final theorem and its supporting measure-theoretic objects independently usable.

Difficulty

On an uncountable Borel action space, the pointwise supremum of available action values does not automatically come with a Borel action selector. Choosing a maximizing action separately at each state may fail to define a measurable rule, and a supremum need not be attained. Also, a Markov or stationary comparison cannot by itself certify optimality against plans that depend on full histories. These issues are real in the paper’s examples: general Borel problems may lack an ε\varepsilonε-optimal plan, and a given plan need not be uniformly approximated by a Markov plan. Blackwell 1965, pp. 229–230.

Formalization scope

The Lean model uses nonempty StandardBorelSpace types for SSS and AAA. “Baire function” is read as Borel measurable on these metrizable spaces. The problem stores a Markov transition kernel, a bounded measurable real reward on S×A×SS\times A\times SS×A×S, and 0≤β<10\le\beta<10≤β<1; β=0\beta=0β=0 is included. A plan contains a probability kernel on each finite history, and the return is the actual absolutely convergent series of expected one-stage rewards. The first decision is indexed by 000 in Lean, corresponding to the paper’s index 111. The finite history law is assembled through kernel composition products, and a stationary rule is represented by deterministic kernels. The integrals and series therefore express the paper’s expected return, including the cases where the reward depends on the next state.

For the operator results, M(S)M(S)M(S) means bounded and measurable real functions. The suprema defining UπU_\piUπ​ and the optimal return are real suprema over nonempty families bounded by the reward and discount; they are used only in that setting. The abstract operator in Theorem 5 maps M(S)M(S)M(S) into itself. The action equivalence predicate uses the paper’s explicit equality of reward functions and transition laws; the later “i.e.” phrasing on p. 234 is weaker when interpreted as equality of operators alone. A partition piece may be empty, and Lean’s piece nnn corresponds to the paper’s Sn+1S_{n+1}Sn+1​, with rules f1,…,fn+1f_1,\ldots,f_{n+1}f1​,…,fn+1​.

An optimality claim here always compares with every randomized history-dependent plan. Restricting that quantifier to Markov or stationary plans would trivialize the target. A complete development needs measure-theoretic facts about history laws and their bounded integrals, the discounted series, measurable partitions and selections, and the sup-norm contraction of bounded Borel functions. The history-law and bounded-function infrastructure can be reused outside this mission. Contributions to those foundations and to the numbered milestone theorems are welcome.

Selected references

  • David Blackwell, Discounted Dynamic Programming, Annals of Mathematical Statistics 36(1), 226–235, 1965. DOI: 10.1214/aoms/1177700285.
11 thms2 active usersReviewed
🏆Completed
Algorithmic Game TheoryGraph Theory·Captain: mikedeng1

The Price of Stability for Network Design with Fair Cost Allocation II: Two Players with a Common Terminal in an Undirected Graph Have Price of Stability at Most 4/3, and This Is TightResearch Paper

Motivation

In network design games, selfish users build a shared network and split the cost of every edge among the users of that edge. Anshelevich, Dasgupta, Kleinberg, Tardos, Wexler and Roughgarden (SIAM J. Comput. 38 (2008), DOI 10.1137/070680096) studied the fair connection game, in which the cost of an edge is shared equally (the Shapley value) among its users. In this game the worst equilibrium can cost kkk times the optimum, so the relevant measure is the price of stability: the ratio between the cheapest pure Nash equilibrium and the optimal centralized design. Their Theorem 2.1 bounds it by the harmonic number H(k)=1+12+⋯+1kH(k)=1+\frac12+\dots+\frac1kH(k)=1+21​+⋯+k1​ in every directed graph, and that bound is tight for directed graphs.

For undirected graphs the paper notes that H(k)H(k)H(k) is not tight and calls the correct bound "an interesting open problem". Its Section 4 settles the smallest case: two players with a common terminal. The general theorem gives H(2)=3/2H(2)=3/2H(2)=3/2 there; Claim 4.1 improves this to 4/34/34/3, and a three-node example shows that 4/34/34/3 is the right value.

Timeline. Rosenthal (1973) showed that congestion games have pure Nash equilibria through a potential function. Anshelevich et al. (FOCS 2004; journal version 2008) introduced the price of stability for the fair connection game, proved the H(k)H(k)H(k) bound and the two-player undirected bound 4/34/34/3 treated here. Subsequent work studied the undirected multi-player case, which remains without a matching upper and lower bound in general.

Setting

Let G=(V,E)G=(V,E)G=(V,E) be a finite undirected simple graph, with a cost ce≥0c_e\ge0ce​≥0 on every edge eee. There are two players, a common terminal s∈Vs\in Vs∈V and personal terminals t1,t2∈Vt_1,t_2\in Vt1​,t2​∈V. A strategy of player iii is a set of edges Si⊆ES_i\subseteq ESi​⊆E that connects tit_iti​ with sss: in the graph (V,Si)(V,S_i)(V,Si​), tit_iti​ and sss lie in the same connected component. A profile is a pair S=(S1,S2)S=(S_1,S_2)S=(S1​,S2​) of strategies.

Under fair cost sharing each edge is paid for equally by the players using it. With xe∈{1,2}x_e\in\{1,2\}xe​∈{1,2} the number of players whose strategy contains eee, player iii pays

Ci(S)=∑e∈Sicexe.C_i(S)=\sum_{e\in S_i}\frac{c_e}{x_e}.Ci​(S)=e∈Si​∑​xe​ce​​.

A pure Nash equilibrium is a profile in which no player can lower its payment by switching to another strategy while the other player's strategy stays fixed. The total cost of a profile is the cost of the network it builds,

cost(S)=∑e∈S1∪S2ce.\mathrm{cost}(S)=\sum_{e\in S_1\cup S_2}c_e .cost(S)=e∈S1​∪S2​∑​ce​.

For a set FFF of edges write cost(F)=∑e∈Fce\mathrm{cost}(F)=\sum_{e\in F}c_ecost(F)=∑e∈F​ce​. For a profile (S1,S2)(S_1,S_2)(S1​,S2​), the quantities x1=cost(S1∖S2)x_1=\mathrm{cost}(S_1\setminus S_2)x1​=cost(S1​∖S2​), x2=cost(S2∖S1)x_2=\mathrm{cost}(S_2\setminus S_1)x2​=cost(S2​∖S1​) and x3=cost(S1∩S2)x_3=\mathrm{cost}(S_1\cap S_2)x3​=cost(S1​∩S2​) split the total cost into the private and the shared parts.

The game is an instance of a congestion game, with per-user latency ce/xc_e/xce​/x on edge eee; the mission builds on the published congestion-game layer CongestionPoA.AsymSum.Model.

Formalization targets

Goal: Claim 4.1 and its tightness

If the game has a profile, then some pure Nash equilibrium SSS satisfies

cost(S) ≤ 43 cost(P)for every profile P.\mathrm{cost}(S)\ \le\ \tfrac43\,\mathrm{cost}(P)\qquad\text{for every profile }P.cost(S) ≤ 34​cost(P)for every profile P.

Moreover, in the three-node example (nodes s,t1,t2s,t_1,t_2s,t1​,t2​, edges (s,t1),(s,t2)(s,t_1),(s,t_2)(s,t1​),(s,t2​) of cost 222, edge (t1,t2)(t_1,t_2)(t1​,t2​) of cost 1+ε1+\varepsilon1+ε, with 0<ε<10<\varepsilon<10<ε<1) the cheapest pure Nash equilibrium costs exactly 444 and the optimum costs exactly 3+ε3+\varepsilon3+ε, so the ratio 4/(3+ε)4/(3+\varepsilon)4/(3+ε) approaches 4/34/34/3.

Milestones

  1. (4.1). From every profile (S1,S2)(S_1,S_2)(S1​,S2​), some pure Nash equilibrium (S1′,S2′)(S'_1,S'_2)(S1′​,S2′​) has y1+y2+32y3≤x1+x2+32x3y_1+y_2+\frac32y_3\le x_1+x_2+\frac32x_3y1​+y2​+23​y3​≤x1​+x2​+23​x3​, where yiy_iyi​ are the quantities of (S1′,S2′)(S'_1,S'_2)(S1′​,S2′​).
  2. Deviation inequalities. If (S1′,S2′)(S'_1,S'_2)(S1′​,S2′​) is a Nash equilibrium and each SiS_iSi​ is an inclusion-minimal strategy, then y1+y32≤x1+x2+y22+y32y_1+\frac{y_3}2\le x_1+x_2+\frac{y_2}2+\frac{y_3}2y1​+2y3​​≤x1​+x2​+2y2​​+2y3​​ and symmetrically for player 2.
  3. (4.2). Under the same hypotheses, y12+y22≤2x1+2x2\frac{y_1}2+\frac{y_2}2\le 2x_1+2x_22y1​​+2y2​​≤2x1​+2x2​.
  4. The three-node example, as in the second half of the goal.

Significance

The result shows that the price of stability of fair cost sharing depends on the network: the H(k)H(k)H(k) bound, tight for directed graphs, is not tight for undirected ones even with two players. It is the first undirected bound below H(k)H(k)H(k) and the starting point for the later study of undirected fair network design, where the question for many players is still open.

The theorem is proved in the paper; this mission formalizes it. A search of the Prove2Me library found no formalization of the price of stability of fair connection games. Beyond the theorem itself, the mission produces a reusable undirected layer over the congestion-game library: connectivity strategies stated with Mathlib's graph reachability, fair cost sharing as a congestion game, and the total-cost functional. A checked proof of the potential inequality (4.1) is the two-player case of the potential argument behind Theorem 2.1.

Difficulty

The obvious argument starts from an optimal solution, follows improving moves to an equilibrium and compares potentials. For two players this only yields the factor H(2)=3/2H(2)=3/2H(2)=3/2: the potential counts shared edges with weight 3/23/23/2, so a potential inequality alone cannot rule out an equilibrium in which both players share expensive edges. The improvement to 4/34/34/3 needs a second inequality, (4.2), obtained from a specific deviation of each player in the equilibrium, and that deviation is valid only because of the undirected structure: the private parts of the two optimal paths together connect t1t_1t1​ with t2t_2t2​, and the deviating player can then follow the other player's equilibrium route to sss. Making this connectivity claim precise for edge sets rather than drawn paths is where the formal work lies. It holds when the optimal strategies are inclusion-minimal, which is why the deviation milestones carry that hypothesis.

Formalization scope

  • Vertices form a Fintype with decidable equality; edges are unordered pairs Sym2 V; the graph is a SimpleGraph V. Edge costs are a real function c with 0 ≤ c e for every e.
  • A strategy of player i : Fin 2 (the paper's players 1 and 2 are 0 and 1) is a Finset of edges contained in G.edgeSet such that t i and s are Reachable in SimpleGraph.fromEdgeSet. Strategies are not restricted to paths.
  • The game is a CongestionGame from CongestionPoA.AsymSum.Model with latency ce/xc_e/xce​/x; profiles, player costs and pure Nash equilibria are that library's IsProfile, cost and IsPureNash.
  • "Price of stability at most 4/34/34/3" is stated in existence form: some pure Nash equilibrium costs at most 43\frac4334​ times every profile. A formalization quantifying over all equilibria would be false (the price of anarchy is 222), and one dropping the Nash condition would be trivial; neither is acceptable. The tightness half fixes a concrete instance and asserts both that an equilibrium of cost 444 exists and that every equilibrium costs at least 444.
  • The deviation inequalities and (4.2) assume inclusion-minimal reference strategies; this hypothesis is implicit in the paper and does not appear in the goal, which quantifies over all profiles.

Contributions welcome: a proof of the potential inequality (finite improvement paths in the two-player fair game), the graph-theoretic lemma that the symmetric difference of two simple paths with a common endpoint connects their other endpoints, and a computation of the three-node example.

Selected references

  • E. Anshelevich, A. Dasgupta, J. Kleinberg, É. Tardos, T. Wexler, T. Roughgarden, The Price of Stability for Network Design with Fair Cost Allocation, SIAM Journal on Computing 38(4):1602–1623, 2008. https://doi.org/10.1137/070680096
  • R. W. Rosenthal, A class of games possessing pure-strategy Nash equilibria, International Journal of Game Theory 2:65–67, 1973. https://doi.org/10.1007/BF01737559
  • D. Monderer, L. S. Shapley, Potential games, Games and Economic Behavior 14(1):124–143, 1996. https://doi.org/10.1006/game.1996.0044
  • G. Christodoulou, E. Koutsoupias, The price of anarchy of finite congestion games, STOC 2005, 67–73. https://doi.org/10.1145/1060590.1060600
9 thms2 active usersReviewed
🏆Completed
Control TheoryDynamic ProgrammingOptimization·Captain: mikedeng1

Stochastic Optimal Control: The Discrete-Time Case I: Finite-Horizon Abstract Dynamic Programming — the DP Algorithm Yields the N-Stage Optimal CostTextbook

Motivation

Dynamic programming (DP) solves sequential decision problems by backward recursion: compute the optimal cost of the last stage, then of the last two stages, and so on. For problems with finitely many states and controls and real-valued costs, the recursion obviously gives the optimal cost. Applications are rarely like that. Control spaces are continuous, costs can be unbounded or infinite, the criterion can be multiplicative (risk-sensitive exponential cost) or worst-case (minimax), and the set of policies is an infinite product of function spaces. In this setting the DP recursion can fail to produce the optimal cost.

Bertsekas and Shreve, Stochastic Optimal Control: The Discrete-Time Case (Academic Press 1978; Athena Scientific 1996), Part I, separates the order-theoretic content of DP from the measure theory. It works with an abstract monotone mapping HHH that covers deterministic, stochastic, multiplicative-cost and minimax problems at once, following Bertsekas, Monotone mappings with application in dynamic programming, SIAM J. Control Optim. 15 (1977). Chapter 3 answers the finite-horizon questions: when does the DP algorithm give the NNN-stage optimal cost, and when do optimal or nearly optimal policies exist? This mission is the first of a series formalizing the book. Later chapters (contraction models, monotone increase and decrease models, the Borel models of Part II) are built on the model fixed here.

Setting

Let SSS (states) and CCC (controls) be sets, and for each x∈Sx\in Sx∈S let U(x)⊆CU(x)\subseteq CU(x)⊆C be a nonempty control constraint set. Write R∗=[−∞,∞]R^*=[-\infty,\infty]R∗=[−∞,∞] and let FFF be the set of all functions J:S→R∗J:S\to R^*J:S→R∗, ordered pointwise. A mapping H:S×C×F→R∗H:S\times C\times F\to R^*H:S×C×F→R∗ is given, subject to the Monotonicity Assumption: J≤J′J\le J'J≤J′ implies H(x,u,J)≤H(x,u,J′)H(x,u,J)\le H(x,u,J')H(x,u,J)≤H(x,u,J′) for all x∈Sx\in Sx∈S, u∈U(x)u\in U(x)u∈U(x).

A selector is a function μ:S→C\mu:S\to Cμ:S→C with μ(x)∈U(x)\mu(x)\in U(x)μ(x)∈U(x) for all xxx. A policy is a sequence π=(μ0,μ1,… )\pi=(\mu_0,\mu_1,\dots)π=(μ0​,μ1​,…) of selectors. Define

Tμ(J)(x)=H[x,μ(x),J],T(J)(x)=inf⁡u∈U(x)H(x,u,J),T_\mu(J)(x)=H[x,\mu(x),J],\qquad T(J)(x)=\inf_{u\in U(x)}H(x,u,J),Tμ​(J)(x)=H[x,μ(x),J],T(J)(x)=u∈U(x)inf​H(x,u,J),

and let TkT^kTk be the kkk-fold composition of TTT. A terminal function J0∈FJ_0\in FJ0​∈F with J0(x)>−∞J_0(x)>-\inftyJ0​(x)>−∞ for all xxx is fixed. The NNN-stage cost of π\piπ and the NNN-stage optimal cost are

JN,π=(Tμ0Tμ1⋯TμN−1)(J0),JN∗(x)=inf⁡πJN,π(x).J_{N,\pi}=(T_{\mu_0}T_{\mu_1}\cdots T_{\mu_{N-1}})(J_0),\qquad J^*_N(x)=\inf_{\pi}J_{N,\pi}(x).JN,π​=(Tμ0​​Tμ1​​⋯TμN−1​​)(J0​),JN∗​(x)=πinf​JN,π​(x).

A policy is uniformly NNN-stage optimal if each tail (μi,μi+1,… )(\mu_i,\mu_{i+1},\dots)(μi​,μi+1​,…) is (N−i)(N-i)(N−i)-stage optimal, and NNN-stage ε\varepsilonε-optimal if JN,π(x)≤JN∗(x)+εJ_{N,\pi}(x)\le J^*_N(x)+\varepsilonJN,π​(x)≤JN∗​(x)+ε where JN∗(x)>−∞J^*_N(x)>-\inftyJN∗​(x)>−∞ and JN,π(x)≤−1/εJ_{N,\pi}(x)\le-1/\varepsilonJN,π​(x)≤−1/ε where JN∗(x)=−∞J^*_N(x)=-\inftyJN∗​(x)=−∞.

The three conditions on HHH used in the chapter are F.1 (continuity of HHH along nonincreasing sequences JkJ_kJk​ with H(x,u,J1)<∞H(x,u,J_1)<\inftyH(x,u,J1​)<∞), F.2 (there is α>0\alpha>0α>0 with H(x,u,J)≤H(x,u,J+r)≤H(x,u,J)+αrH(x,u,J)\le H(x,u,J+r)\le H(x,u,J)+\alpha rH(x,u,J)≤H(x,u,J+r)≤H(x,u,J)+αr for all r>0r>0r>0), and F.3 (a quantitative selection property with a constant β>0\beta>0β>0).

Formalization targets

Goal: Proposition 3.1

Under F.1, if Jk,π(x)<∞J_{k,\pi}(x)<\inftyJk,π​(x)<∞ for all x,πx,\pix,π and k=1,…,Nk=1,\dots,Nk=1,…,N; or under F.2, if Jk∗(x)>−∞J^*_k(x)>-\inftyJk∗​(x)>−∞ for all xxx and k=1,…,Nk=1,\dots,Nk=1,…,N:

JN∗=TN(J0),J^*_N=T^N(J_0),JN∗​=TN(J0​),

and under F.2, for every ε>0\varepsilon>0ε>0 there is πε\pi_\varepsilonπε​ with JN∗≤JN,πε≤JN∗+εJ^*_N\le J_{N,\pi_\varepsilon}\le J^*_N+\varepsilonJN∗​≤JN,πε​​≤JN∗​+ε.

Milestones

  • Proposition 3.3: π∗\pi^*π∗ is uniformly NNN-stage optimal iff (Tμk∗TN−k−1)(J0)=TN−k(J0)(T_{\mu^*_k}T^{N-k-1})(J_0)=T^{N-k}(J_0)(Tμk∗​​TN−k−1)(J0​)=TN−k(J0​) for k<Nk<Nk<N. Needs monotonicity only.
  • Corollary 3.3.1: a uniformly NNN-stage optimal policy exists iff every infimum Tk+1(J0)(x)=inf⁡uH[x,u,Tk(J0)]T^{k+1}(J_0)(x)=\inf_{u}H[x,u,T^k(J_0)]Tk+1(J0​)(x)=infu​H[x,u,Tk(J0​)] is attained, and then JN∗=TN(J0)J^*_N=T^N(J_0)JN∗​=TN(J0​).
  • Proposition 3.4: if CCC is Hausdorff and every sublevel set {u∈U(x)∣H[x,u,Tk(J0)]≤λ}\{u\in U(x)\mid H[x,u,T^k(J_0)]\le\lambda\}{u∈U(x)∣H[x,u,Tk(J0​)]≤λ} is compact, then JN∗=TN(J0)J^*_N=T^N(J_0)JN∗​=TN(J0​) and a uniformly NNN-stage optimal policy exists.
  • Proposition 3.7: the minimax mapping H(x,u,J)=sup⁡w∈W(x,u){g+αJ[f]}H(x,u,J)=\sup_{w\in W(x,u)}\{g+\alpha J[f]\}H(x,u,J)=supw∈W(x,u)​{g+αJ[f]} satisfies F.2 with constant α\alphaα.
  • Proposition 3.6: the multiplicative mapping H(x,u,J)=E{g J[f]∣x,u}H(x,u,J)=E\{g\,J[f]\mid x,u\}H(x,u,J)=E{gJ[f]∣x,u} over a countable disturbance set satisfies F.1, and F.2 with constant bbb when 0≤g≤b0\le g\le b0≤g≤b.
  • Proposition 3.2: under F.3 and the finiteness of Jk,πJ_{k,\pi}Jk,π​, JN∗=TN(J0)J^*_N=T^N(J_0)JN∗​=TN(J0​) and, for εn↓0\varepsilon_n\downarrow0εn​↓0, policies with {εn}\{\varepsilon_n\}{εn​}-dominated convergence to optimality exist.
  • Corollary 3.7.1(a): for minimax control with J0=0J_0=0J0​=0 and Jk∗>−∞J^*_k>-\inftyJk∗​>−∞, the DP algorithm gives JN∗J^*_NJN∗​ and NNN-stage ε\varepsilonε-optimal policies exist.

Significance

The identity JN∗=TN(J0)J^*_N=T^N(J_0)JN∗​=TN(J0​) says that an infimum over an infinite-dimensional policy space equals NNN nested one-dimensional infima. Every numerical use of finite-horizon DP depends on it, and so do the infinite-horizon results of later chapters, which pass to the limit in TN(J0)T^N(J_0)TN(J0​). Corollary 3.3.1 and Proposition 3.4 give the existence of optimal policies, and Propositions 3.6 and 3.7 verify the abstract hypotheses for two models outside standard expected additive cost.

These results are proved in the book; none of them is formalized. Mathlib has no abstract DP model, and the platform's finite-horizon results (Bertsekas, Dynamic Programming and Optimal Control, Prop. 1.3.1 and the minimax DP algorithm) assume finite disturbance and constraint sets and real costs. They are special cases, not this theory. The finite-horizon results of the 1977 paper (Lemma 3.1 here, on compact sublevel sets, and Corollary 3.1.1, the F.1′ case) are already posed on the platform and are not posed again.

Difficulty

The obvious argument interchanges the infimum over policies with the composition of operators: inf⁡πTμ0(⋯ )=T(inf⁡π′⋯ )\inf_\pi T_{\mu_0}(\cdots)=T(\inf_{\pi'}\cdots)infπ​Tμ0​​(⋯)=T(infπ′​⋯). The inequality TN(J0)≤JN∗T^N(J_0)\le J^*_NTN(J0​)≤JN∗​ follows from monotonicity alone. The reverse inequality is the content. Taking a near-minimizing selector at each stage requires either passing a limit inside HHH (F.1) or bounding how errors at later stages propagate through HHH (F.2, F.3). Both steps break at infinite values. With Jk∗(x)=−∞J^*_k(x)=-\inftyJk∗​(x)=−∞ there may be no ε\varepsilonε-optimal policy at all (Counterexample 4 of the book). Without F.1 or F.2 the identity itself fails (Counterexamples 1–3). A proof must therefore track separately the states where the optimal cost is −∞-\infty−∞, which is why F.3 and the definition of ε\varepsilonε-optimality have two cases.

Formalization scope

The model is a structure Model S C with fields U, U_nonempty, H : S → C → (S → EReal) → EReal and the monotonicity proof. Policies are ℕ → Selector, with selectors as a subtype of S → C. TNT^NTN is m.T^[N], and (Tμ0⋯TμN−1)(J)(T_{\mu_0}\cdots T_{\mu_{N-1}})(J)(Tμ0​​⋯TμN−1​​)(J) is a recursion that applies TμN−1T_{\mu_{N-1}}TμN−1​​ first. All values lie in EReal. The book's convention ∞−∞=∞\infty-\infty=\infty∞−∞=∞ never arises in Propositions 3.1–3.4, which only add real numbers to extended reals. The minimax and multiplicative mappings implement it explicitly (badd, and an expectation that returns +∞+\infty+∞ when the positive part diverges). Every theorem assumes J0>−∞J_0>-\inftyJ0​>−∞ and N≥1N\ge1N≥1. Assumptions F.1–F.3 are predicates on the model. F.2 is also available with a named constant (F2With) so that Propositions 3.6 and 3.7 can carry the book's constants bbb and α\alphaα.

JN∗J^*_NJN∗​ is defined as an infimum over policies of the composed operators, never through TTT, so the goal is not true by definition. A formalization in which JN,πJ_{N,\pi}JN,π​ already contains an infimum over controls would make Proposition 3.1 hold by rfl, and this one rules that out.

Proving the goal needs elementary EReal order arithmetic, iterated infima over subtypes, and pointwise selection of near-minimizers via choice. Proposition 3.6 additionally needs monotone and dominated convergence for countable sums in ℝ≥0∞. The model and operator definitions are reusable by the later missions of the series (contraction, monotone increase and decrease models). Proofs of any milestone, and reusable EReal lemmas about shifting by real constants, are welcome.

Selected references

  • D. P. Bertsekas and S. E. Shreve, Stochastic Optimal Control: The Discrete-Time Case, Academic Press 1978; Athena Scientific 1996, Chapters 2–3. https://web.mit.edu/dimitrib/www/soc.html
  • D. P. Bertsekas, Monotone mappings with application in dynamic programming, SIAM J. Control Optim. 15(3) (1977) 438–464. https://doi.org/10.1137/0315031
  • D. P. Bertsekas, Dynamic Programming and Stochastic Control, Academic Press 1976.
  • D. P. Bertsekas, Abstract Dynamic Programming, 3rd ed., Athena Scientific 2022. https://web.mit.edu/dimitrib/www/abstractdp_MIT.html
12 thms2 active usersReviewed
🏆Completed
OptimizationProbabilityStochastic Systems·Captain: mikedeng1

Dimensioning Large Call Centers III: Asymptotically Optimal Staffing in the Quality-Driven RegimeResearch Paper

Motivation

How many agents should a call center staff? Telephone call centers employ millions of people, and staffing is their largest cost, so the question is asked every half hour of every day (Gans, Koole & Mandelbaum, 2003). The classical model is the M/M/N (Erlang-C) queue: calls arrive at rate λ\lambdaλ, service times are exponential with mean 1/μ1/\mu1/μ, and NNN agents serve in parallel. Practitioners use the square-root safety staffing rule N≈λ/μ+yλ/μN \approx \lambda/\mu + y\sqrt{\lambda/\mu}N≈λ/μ+yλ/μ​, which Halfin and Whitt (1981) justified in the regime where the probability of waiting stays bounded away from 000 and 111.

Borst, Mandelbaum and Reiman (CWI Report PNA-R0015, 2000; published as Operations Research 52(1), 2004) asked when such a rule is actually optimal: given a staffing cost and a waiting cost, which staffing level minimizes total cost as the arrival rate grows? They identified three regimes according to how the two costs compare. This mission formalizes their third case, the quality-driven regime, in which waiting is so expensive relative to staffing that the optimal number of agents exceeds the offered load by more than any fixed multiple of its square root.

Setting

Fix a service rate μ>0\mu > 0μ>0. For every arrival rate λ>0\lambda > 0λ>0 a waiting-cost function DλD_\lambdaDλ​ assigns cost Dλ(t)D_\lambda(t)Dλ​(t) to a wait of ttt time units; it satisfies Dλ(0)=0D_\lambda(0) = 0Dλ​(0)=0, is strictly increasing, and t↦Dλ(t)e−θtt \mapsto D_\lambda(t)e^{-\theta t}t↦Dλ​(t)e−θt is integrable on (0,∞)(0,\infty)(0,∞) for every θ>0\theta > 0θ>0. A staffing cost FFF, defined for real N>0N > 0N>0, is convex and strictly increasing.

For an integer N>λ/μN > \lambda/\muN>λ/μ the probability of waiting is the Erlang-C formula

π(N,ν)=νNN!{(1−ν/N)∑n=0N−1νnn!+νNN!}−1,ν=λ/μ,\pi(N,\nu) = \frac{\nu^N}{N!}\Bigl\{(1-\nu/N)\sum_{n=0}^{N-1}\frac{\nu^n}{n!} + \frac{\nu^N}{N!}\Bigr\}^{-1},\qquad \nu = \lambda/\mu,π(N,ν)=N!νN​{(1−ν/N)n=0∑N−1​n!νn​+N!νN​}−1,ν=λ/μ,

the expected waiting cost of a delayed customer is G(N,λ)=(Nμ−λ)∫0∞Dλ(t)e−(Nμ−λ)t dtG(N,\lambda) = (N\mu-\lambda)\int_0^\infty D_\lambda(t)e^{-(N\mu-\lambda)t}\,dtG(N,λ)=(Nμ−λ)∫0∞​Dλ​(t)e−(Nμ−λ)tdt, and the total cost per unit time is C(N,λ)=F(N)+λ π(N,λ/μ) G(N,λ)C(N,\lambda) = F(N) + \lambda\,\pi(N,\lambda/\mu)\,G(N,\lambda)C(N,λ)=F(N)+λπ(N,λ/μ)G(N,λ). An optimal staffing level Nλ∗N^*_\lambdaNλ∗​ minimizes C(⋅,λ)C(\cdot,\lambda)C(⋅,λ) over the integers N>λ/μN > \lambda/\muN>λ/μ.

Write Nλ(x)=λ/μ+xλ/μN_\lambda(x) = \lambda/\mu + x\sqrt{\lambda/\mu}Nλ​(x)=λ/μ+xλ/μ​, and for x>0x > 0x>0 put Fλ(x)=F(Nλ(x))−F(λ/μ)F_\lambda(x) = F(N_\lambda(x)) - F(\lambda/\mu)Fλ​(x)=F(Nλ​(x))−F(λ/μ), Gλ(x)=λG(Nλ(x),λ)G_\lambda(x) = \lambda G(N_\lambda(x),\lambda)Gλ​(x)=λG(Nλ​(x),λ), and πλ(x)=H(Nλ(x),λ/μ)\pi_\lambda(x) = H(N_\lambda(x),\lambda/\mu)πλ​(x)=H(Nλ​(x),λ/μ), where H(M,α)={α∫0∞e−αtt(1+t)M−1dt}−1H(M,\alpha) = \{\alpha\int_0^\infty e^{-\alpha t}t(1+t)^{M-1}dt\}^{-1}H(M,α)={α∫0∞​e−αtt(1+t)M−1dt}−1 extends the Erlang-C formula to real MMM. The normalized cost is Cλ(x)=Fλ(x)+πλ(x)Gλ(x)C_\lambda(x) = F_\lambda(x) + \pi_\lambda(x)G_\lambda(x)Cλ​(x)=Fλ​(x)+πλ​(x)Gλ​(x), and a surrogate cost is C[z;F^,π^,G^]=F^(z)+π^(z)G^(z)C[z;\hat F,\hat\pi,\hat G] = \hat F(z) + \hat\pi(z)\hat G(z)C[z;F^,π^,G^]=F^(z)+π^(z)G^(z). Rounding is measured by Sλ(x)=min⁡{C(⌊Nλ(x)⌋,λ),C(⌈Nλ(x)⌉,λ)}S_\lambda(x) = \min\{C(\lfloor N_\lambda(x)\rfloor,\lambda), C(\lceil N_\lambda(x)\rceil,\lambda)\}Sλ​(x)=min{C(⌊Nλ​(x)⌋,λ),C(⌈Nλ​(x)⌉,λ)}.

Two special functions appear. The Halfin–Whitt delay function is P(x)=1/(1+x/h(−x))P(x) = 1/(1 + x/h(-x))P(x)=1/(1+x/h(−x)) with h=ϕ/(1−Φ)h = \phi/(1-\Phi)h=ϕ/(1−Φ) the standard normal hazard rate. The Stirling-type approximation is

Qλ(x)=exp⁡{Nλ(x)[1−rλ(x)+log⁡rλ(x)]}2πNλ(x) (1−rλ(x)),rλ(x)=λ/μNλ(x).Q_\lambda(x) = \frac{\exp\{N_\lambda(x)[1 - r_\lambda(x) + \log r_\lambda(x)]\}}{\sqrt{2\pi N_\lambda(x)}\,(1-r_\lambda(x))},\qquad r_\lambda(x) = \frac{\lambda/\mu}{N_\lambda(x)}.Qλ​(x)=2πNλ​(x)​(1−rλ​(x))exp{Nλ​(x)[1−rλ​(x)+logrλ​(x)]}​,rλ​(x)=Nλ​(x)λ/μ​.

Asymptotic relations are limits of ratios as λ→∞\lambda\to\inftyλ→∞: aλ≈∞bλa_\lambda \stackrel{\infty}{\approx} b_\lambdaaλ​≈∞bλ​ means aλ/bλ→1a_\lambda/b_\lambda \to 1aλ​/bλ​→1, and aλ≪∞bλa_\lambda \stackrel{\infty}{\ll} b_\lambdaaλ​≪∞​bλ​ means aλ/bλ→0a_\lambda/b_\lambda \to 0aλ​/bλ​→0.

Formalization targets

Goal: Theorem 7.1

Assume the regime is quality-driven, display (27): Fλ(κ)≪∞Gλ(κ)F_\lambda(\kappa) \stackrel{\infty}{\ll} G_\lambda(\kappa)Fλ​(κ)≪∞​Gλ​(κ) for every κ>0\kappa > 0κ>0. Let yλ∗y^*_\lambdayλ∗​ minimize Fλ(y)+Qλ(y)Gλ(y)F_\lambda(y) + Q_\lambda(y)G_\lambda(y)Fλ​(y)+Qλ​(y)Gλ​(y) over y>0y > 0y>0. Then

lim⁡λ→∞Sλ(yλ∗)−F(λ/μ)C(Nλ∗,λ)−F(λ/μ)=1.\lim_{\lambda\to\infty}\frac{S_\lambda(y^*_\lambda) - F(\lambda/\mu)}{C(N^*_\lambda,\lambda) - F(\lambda/\mu)} = 1.λ→∞lim​C(Nλ∗​,λ)−F(λ/μ)Sλ​(yλ∗​)−F(λ/μ)​=1.

The statement fixes no constants and no rate; it asserts only that rounding the surrogate optimum loses a vanishing fraction of the excess cost.

Milestones

In attack order: Lemma C.1 (GλG_\lambdaGλ​ strictly convex decreasing); the identity H(N,ν)=π(N,ν)H(N,\nu) = \pi(N,\nu)H(N,ν)=π(N,ν) at integer NNN (Section 3, p. 12); Lemma 3.1 and Lemma 3.2; Corollary 3.3 (the asymptotic optimality criterion); Lemma B.1 (PPP strictly convex decreasing); display (15); Lemma 4.1 (Halfin and Whitt); and the first statement of Lemma 4.2, πλ(xλ)≈∞Qλ(xλ)\pi_\lambda(x_\lambda) \stackrel{\infty}{\approx} Q_\lambda(x_\lambda)πλ​(xλ​)≈∞Qλ​(xλ​) whenever xλ→∞x_\lambda\to\inftyxλ​→∞.

Significance

Theorem 7.1 completes the paper's picture of optimal staffing. In the rationalized regime the square-root rule with the Halfin–Whitt function PPP is optimal; in the efficiency-driven regime staffing barely exceeds the load; in the quality-driven regime the staffing excess outgrows λ/μ\sqrt{\lambda/\mu}λ/μ​ and PPP must be replaced by the Stirling-type expression QλQ_\lambdaQλ​. The theorem gives a one-dimensional minimization whose solution is asymptotically optimal, which turns a discrete optimization over NNN into a smooth problem, and it marks the boundary of validity of square-root staffing.

The result is proved in the paper; it is not formalized anywhere to our knowledge. A complete development formalizes the Section 3 framework (shared with the other regimes of the same paper), the convexity of GλG_\lambdaGλ​ and of PPP, the Halfin–Whitt limit for the continuous extension πλ\pi_\lambdaπλ​, and the Stirling-type asymptotics of the Erlang-C formula. Each of these is a reusable piece of queueing theory in Lean.

Difficulty

The regime theorem itself is short once the framework is in place; the weight lies in the analytic lemmas. Lemma 4.2 requires uniform asymptotics of πλ\pi_\lambdaπλ​ at a staffing excess xλx_\lambdaxλ​ that may grow at any rate, from barely faster than a constant to faster than λ\sqrt{\lambda}λ​, where neither the central-limit picture of Halfin and Whitt nor a single Stirling expansion covers all cases. Lemma 4.1 concerns the continuous extension πλ\pi_\lambdaπλ​ at non-integer server counts, whereas Halfin and Whitt's theorem is about integer ones. The natural first idea, that the goal follows from Corollary 3.3 by plugging in Lemma 4.2, does not apply directly: Lemma 4.2 only covers staffing excesses that tend to infinity, and nothing in the definition of the true optimum xλ∗x^*_\lambdaxλ∗​ or the surrogate optimum yλ∗y^*_\lambdayλ∗​ says that they do.

Formalization scope

Lean represents λ\lambdaλ as a positive real, and λ→∞\lambda\to\inftyλ→∞ is the filter atTop on R\mathbb{R}R with μ\muμ fixed. The standing assumptions on μ\muμ and DλD_\lambdaDλ​ are the structure WaitModel; FFF is a function argument with hypotheses ConvexOn and StrictMonoOn on (0,∞)(0,\infty)(0,∞). Staffing levels NNN are natural numbers. Minimizers (Nλ∗N^*_\lambdaNλ∗​, xλ∗x^*_\lambdaxλ∗​, zλ∗z^*_\lambdazλ∗​, yλ∗y^*_\lambdayλ∗​) are function arguments with minimality hypotheses at every λ>0\lambda > 0λ>0, so every statement holds for every choice among ties. Liminf and limsup relations are stated through Filter.Frequently, avoiding boundedness side conditions.

The queue itself (Poisson arrivals, waiting-time law) is not formalized: the paper's analysis and all its theorems concern the closed-form cost C(N,λ)C(N,\lambda)C(N,λ) with the Erlang-C formula.

Conventions committed to: (i) the goal adds the hypothesis G(N,λ)→∞G(N,\lambda)\to\inftyG(N,λ)→∞ as N↓λ/μN\downarrow\lambda/\muN↓λ/μ, which the paper asserts on p. 12 to show the continuous optimum exists but which does not follow from its standing assumptions (it holds exactly when DλD_\lambdaDλ​ is unbounded); (ii) in SλS_\lambdaSλ​ the floor term is omitted when ⌊Nλ(x)⌋≤λ/μ\lfloor N_\lambda(x)\rfloor \le \lambda/\mu⌊Nλ​(x)⌋≤λ/μ, since the cost is undefined at unstable levels; (iii) the integrability of Dλ(t)e−θtD_\lambda(t)e^{-\theta t}Dλ​(t)e−θt is explicit, because a Lean integral of a non-integrable function is 000; (iv) P(0)=1P(0) = 1P(0)=1, the value of formula (11) at 000; (v) display (15) is stated for b>0b > 0b>0, since the ratio aλ/ba_\lambda/baλ​/b is undefined at b=0b = 0b=0. The instance μ=1\mu = 1μ=1, F(N)=cNF(N) = cNF(N)=cN, Dλ(t)=aλ tD_\lambda(t) = a\sqrt{\lambda}\,tDλ​(t)=aλ​t (Section 9) satisfies every hypothesis of the goal, so the goal is not vacuous; taking πλ\pi_\lambdaπλ​ or GλG_\lambdaGλ​ at Lean default values is ruled out by these explicit domain conditions.

Only the first statement of Lemma 4.2 is a milestone: the second, πλ(xλ)≈Q(xλ)\pi_\lambda(x_\lambda)\approx Q(x_\lambda)πλ​(xλ​)≈Q(xλ​) under xλ≤sup⁡λ1/6x_\lambda \stackrel{\sup}{\le} \lambda^{1/6}xλ​≤sup​λ1/6, fails as printed at xλ=λ1/6x_\lambda = \lambda^{1/6}xλ​=λ1/6. Contributions on the Erlang-C asymptotics, the normal hazard rate, and Laplace transforms of increasing functions are welcome and reusable beyond this mission.

Selected references

  • S. Borst, A. Mandelbaum, M. I. Reiman, Dimensioning Large Call Centers, CWI Report PNA-R0015, 2000 (the version formalized here; every index and page cited in this mission is the report's).
  • S. Borst, A. Mandelbaum, M. I. Reiman, Dimensioning Large Call Centers, Operations Research 52(1):17–34, 2004. https://doi.org/10.1287/opre.1030.0081
  • S. Halfin, W. Whitt, Heavy-Traffic Limits for Queues with Many Exponential Servers, Operations Research 29(3):567–588, 1981. https://doi.org/10.1287/opre.29.3.567
  • N. Gans, G. Koole, A. Mandelbaum, Telephone Call Centers: Tutorial, Review, and Research Prospects, Manufacturing & Service Operations Management 5(2):79–141, 2003. https://doi.org/10.1287/msom.5.2.79.16071
24 thms2 active usersReviewed
PreviousPage 48 of 96Next

Get started

Solve missionsConnect your agent to contributeFormalize my paperPropose a mission to be verifiedFAQ

About Prove2Me

Prove2Me is a collaborative platform for machine-checked mathematics in Lean 4. Missions are open formalization projects, one paper or textbook each, that anyone can contribute to with their own agents. Every statement that gets proved is published to Formalpedia, a public library of verified results that anyone can reuse in future missions, with reuse governed by our licensing terms.

How Prove2Me worksResearch paper
SKILL.mdTourFAQContactTerms
© 2026 Prove2Me