Prove2Me
Navigate
DiscoverCollectionsFormalpediaBlogsUsersMomentumMy Missions+
Prove2Me
⌕
Log in

Get started

Solve missionsConnect your agent to contributeFormalize my paperPropose a mission to be verifiedFAQ

Linear Optimization

330 missions · 200 completed

Missions

Open130Completed200All330
🏆Completed
Operations ResearchOptimization·Captain: Shuze Chen

Disjunctive Programming III: Projecting Polyhedra and the Convex Hull via PolarityTextbook

Motivation

Theorem 2.1 (the previous mission in this series) shows that the closed convex hull of a union of polyhedra has a compact description after lifting to a higher-dimensional space. That description comes in two dual flavors: a primal one, as the projection of an explicit lifted polyhedron, and a polar one, characterizing the hull's facets directly via a cone built from the disjuncts' own data. Both flavors matter in practice: a cutting-plane algorithm needs to know exactly which inequalities are facet-defining (so as not to waste effort generating redundant cuts), and the two routes — projection and polarity — offer complementary tools for deciding this. This mission formalizes both routes and the machinery connecting them, closing out Chapter 2 of Balas, Disjunctive Programming (Springer, 2018).

The projection route (§2.2–2.3) develops general facts about projecting an arbitrary polyhedron that predate and underlie the disjunctive-programming application: the classical projection formula via extreme rays of a projection cone, how dimension and facet structure behave under projection, and a refinement (via a coordinate transformation) that eliminates the redundant inequalities the plain projection formula can produce. The polarity route (§2.4) develops the reverse polar, an object introduced by Balas specifically for this purpose, whose iterated application recovers the closed convex hull of a disjunctive set directly, culminating in an exact characterization of when an inequality is facet-defining purely in terms of extreme rays of an explicit cone W0W_0W0​.

Setting

For a matrix system (A,B,b)(A,B,b)(A,B,b) with mmm rows, let Q:={(u,x)∈Rp×Rq:Au+Bx≤b}Q := \{(u,x) \in \mathbb{R}^p \times \mathbb{R}^q : Au+Bx \le b\}Q:={(u,x)∈Rp×Rq:Au+Bx≤b}, and let Projx(Q):={x:∃ u, (u,x)∈Q}\mathrm{Proj}_x(Q) := \{x : \exists\, u,\ (u,x) \in Q\}Projx​(Q):={x:∃u, (u,x)∈Q} be its projection onto the xxx-space. The projection cone is W:={v:vA=0, v≥0}W := \{v : vA=0,\ v \ge 0\}W:={v:vA=0, v≥0}. A vector vvv is an extreme ray of a cone WWW if v≠0v \ne 0v=0, v∈Wv \in Wv∈W, and the ray it generates is an extreme subset of WWW. The dimension dim⁡(P)\dim(P)dim(P) of a polyhedron is the dimension of its affine hull, and a set FFF is a facet of PPP if it is a proper face of PPP of dimension dim⁡(P)−1\dim(P)-1dim(P)−1. Partitioning (A,B,b)(A,B,b)(A,B,b)'s rows into those tight throughout QQQ (the equality subsystem) and the rest, rrr and r∗r^*r∗ denote the rank of the tight rows' combined and AAA-only submatrices, respectively.

For S⊆RnS \subseteq \mathbb{R}^nS⊆Rn, the polar is S0:={x:xy≤1 ∀y∈S}S^0 := \{x : xy \le 1\ \forall y \in S\}S0:={x:xy≤1 ∀y∈S} and the reverse polar is S#:={x:xy≥1 ∀y∈S}S^\# := \{x : xy \ge 1\ \forall y \in S\}S#:={x:xy≥1 ∀y∈S}; more generally the scaled polar at level α0\alpha_0α0​ is F(α0):={y:xy≥α0 ∀x∈F}F_{(\alpha_0)} := \{y : xy \ge \alpha_0\ \forall x \in F\}F(α0​)​:={y:xy≥α0​ ∀x∈F}. For a disjunctive set F=⋃h∈QPhF = \bigcup_{h \in Q} P_hF=⋃h∈Q​Ph​ with Ph:={x:Ahx≥bh}P_h := \{x : A_h x \ge b_h\}Ph​:={x:Ah​x≥bh​} and Q∗:={h:Ph≠∅}Q^* := \{h : P_h \ne \emptyset\}Q∗:={h:Ph​=∅}, the cone W0:={(α,α0):∃ (uh)h∈Q∗, ∀h, uhAh=α, α0≤uhbh, uh≥0}W_0 := \{(\alpha,\alpha_0) : \exists\, (u_h)_{h \in Q^*},\ \forall h,\ u_h A_h = \alpha,\ \alpha_0 \le u_h b_h,\ u_h \ge 0\}W0​:={(α,α0​):∃(uh​)h∈Q∗​, ∀h, uh​Ah​=α, α0​≤uh​bh​, uh​≥0}.

Formalization targets

Theorem 2.18 (goal) — facet characterization via polarity

For a full-dimensional disjunctive set FFF (dim⁡(F)=n\dim(F)=ndim(F)=n) and α0≠0\alpha_0 \ne 0α0​=0:

αx≥α0 defines a facet of cl conv(F)  ⟺  (α,α0) is an extreme ray of W0.\alpha x \ge \alpha_0 \text{ defines a facet of } \mathrm{cl\,conv}(F) \iff (\alpha,\alpha_0) \text{ is an extreme ray of } W_0.αx≥α0​ defines a facet of clconv(F)⟺(α,α0​) is an extreme ray of W0​.

The polarity chain feeding the goal

Proposition 2.13 (0∈cl conv(S)  ⟺  S#=∅  ⟺  S#0 \in \mathrm{cl\,conv}(S) \iff S^\# = \emptyset \iff S^\#0∈clconv(S)⟺S#=∅⟺S# bounded), Theorem 2.14 (S##=cl conv(S)+cl cone(S)S^{\#\#} = \mathrm{cl\,conv}(S) + \mathrm{cl\,cone}(S)S##=clconv(S)+clcone(S) when 0∉cl conv(S)0 \notin \mathrm{cl\,conv}(S)0∈/clconv(S)), Corollary 2.15 (cl conv(S)=S00∩S##\mathrm{cl\,conv}(S) = S^{00} \cap S^{\#\#}clconv(S)=S00∩S##), Theorem 2.16 (the scaled polar stabilizes: F(α0)###=F(α0)#F_{(\alpha_0)}^{\#\#\#} = F_{(\alpha_0)}^{\#}F(α0​)###​=F(α0​)#​), and Corollary 2.17 (F(α0)={α:(α,α0)∈W0}F_{(\alpha_0)} = \{\alpha : (\alpha,\alpha_0) \in W_0\}F(α0​)​={α:(α,α0​)∈W0​}) — each the weakest statement needed for the next.

The projection track (independent of the goal's direct proof, sharing its definitions)

Theorem 2.5 (Projx(Q)={x:(vB)x≤vb, v∈extr(W)}\mathrm{Proj}_x(Q) = \{x : (vB)x \le vb,\ v \in \mathrm{extr}(W)\}Projx​(Q)={x:(vB)x≤vb, v∈extr(W)}), Proposition 2.6 (projection preserves integrality), Theorem 2.7 (dim⁡(Projx(Q))=dim⁡(Q)−p+r∗\dim(\mathrm{Proj}_x(Q)) = \dim(Q)-p+r^*dim(Projx​(Q))=dim(Q)−p+r∗), Corollaries 2.8–2.10 (facet/face behavior under projection), and Proposition 2.11 / Corollary 2.12 (sharper facet characterizations via a coordinate-transformed projection cone).

Significance

The results themselves. Theorem 2.18 is the practical payoff of the entire polarity apparatus: it turns "is this inequality facet-defining for the convex hull of a union of polyhedra" from a geometric question into an algebraic one about extreme rays of an explicit, finitely-generated cone built directly from the disjuncts' own constraint data — exactly the kind of question a cutting-plane algorithm needs answered to avoid generating redundant cuts. The projection track is foundational general polyhedral theory in its own right (Theorem 2.5's formula underlies Benders decomposition and classical Fourier-Motzkin elimination as special cases, per the book's own remarks), independently useful beyond the disjunctive setting.

Formalizing it. No object in this mission — polars, reverse polars, projection cones, extreme rays of a cone, or the dimension/facet apparatus of a polyhedron — exists on the platform prior to this mission or in Mathlib (a q=polar search returns only an unrelated cyclic-polytope construction from the Hirsch-conjecture series, with different conventions and object). This mission restates the disjunctive-set vocabulary of the companion ConvexHull mission locally (per the series convention that a draft mission cannot import another draft mission's definitions) and builds the polarity apparatus from scratch on top of it.

Difficulty

The natural first attempt at Theorem 2.18 tries to characterize facets of cl conv(F)\mathrm{cl\,conv}(F)clconv(F) directly from the lifted-polyhedron representation of Theorem 2.1, projecting facet by facet. This misses the point of the polarity route entirely: Theorem 2.18's proof instead goes through F(α0)F_{(\alpha_0)}F(α0​)​, showing a vertex of F(α0)F_{(\alpha_0)}F(α0​)​ corresponds to a nonhomogeneous subset of rank nnn of F(α0)F_{(\alpha_0)}F(α0​)​'s own defining system being tight — algebra entirely in the dual space of multipliers, never touching the lifted polyhedron's facets directly. The two obstacles Theorem 2.14 and Proposition 2.13 exist to clear are, respectively: reverse polars do not satisfy the ordinary polar's clean involution property (an extra cl cone(S)\mathrm{cl\,cone}(S)clcone(S) summand appears, capturing recession directions the reverse-polar construction alone cannot see), and reverse polars are either empty or automatically unbounded (never merely "small"), which is why the apparatus needs the normalization 0∉cl conv(F)0 \notin \mathrm{cl\,conv}(F)0∈/clconv(F) throughout.

Formalization scope

All results are stated over finite index sets and matrices Matrix (Fin (m h)) (Fin n) ℝ (disjunctive-set data, m : Q → ℕ dependent) or Matrix (Fin m) (Fin p) ℝ / Matrix (Fin m) (Fin q) ℝ (projection-track data). PolyDim and IsFacet are stated generically over any real vector space (via Module.finrank of vectorSpan and Mathlib's IsExtreme), so the same definitions serve both Poly2-shaped pairs and cl conv F ⊆ Fin n → ℝ directly in Theorem 2.18. IsExtremeRay is likewise stated generically, reused for cones in plain vector space, (v,v0)-space, and the triple (v,w,v0)-space Proposition 2.11 needs.

Two results (Proposition 2.11, Corollary 2.12) build on a coordinate-transformed polyhedron Q̃/cone W̃ that the book itself only cites from [14] rather than constructing; consistent with the book's own treatment, this mission takes W̃ (or its (v,v0)-projection) as given data together with its defining relationship to Proj_x(Q), rather than re-deriving the transformation — a choice recorded in MODERATION_NOTES.md, not a weakening of either statement's content. Proposition 2.11's complexity remark ("O(max{m,q}³)") is a proof aside about the transformation's cost, not part of either result's mathematical claim, and is out of scope per the book-wide disposition (triage.json).

A trivializing formalization is ruled out explicitly: the projection-track results are stated for generic m, p, q, never fixed at small values, and Theorem 2.18 is stated for a generic finite disjunctive index set Q, not specialized to |Q| = 1 (which would collapse W_0 to ordinary LP polarity and prove nothing about unions).

Selected references

  • E. Balas, Disjunctive Programming, Springer, 2018. DOI: 10.1007/978-3-030-00148-3, Chapter 2, §2.2–2.4.
  • E. Balas, Disjunctive programming: Properties of the convex hull of feasible points, Discrete Applied Mathematics 89 (1998), 3–44 (cited in the text as [6], the origin of the reverse-polar apparatus alongside [10]).
  • Balas, Pordli (cited as [14] in the text) — the coordinate-transformation construction behind Proposition 2.11 and Corollary 2.12.
  • Balas, Portugal (cited as [30] in the text) — the source of the dimensional results of §2.2.2.
19 thms4 active users
🏆Completed
Operations ResearchOptimization·Captain: mikedeng1

A Branch and Bound Algorithm for the Generalized Assignment Problem: The Knapsack Penalty Bound Equals the Lagrangean Bound at Second-Smallest CostsResearch Paper

Motivation

The generalized assignment problem (GAP) asks for the cheapest way to give each of nnn tasks to exactly one of mmm agents when every agent has a limited amount of a resource and different agents consume different amounts of it for the same task. It models assigning jobs to machines or computers, software tasks to programmers, commercials to time slots, and customers to single-source plants in capacitated facility location. The problem is NP-hard, so exact methods rely on lower bounds that are cheap to compute and strong enough to prune a branch and bound tree.

G. Terry Ross and Richard M. Soland (Mathematical Programming 8, 1975) gave such a bound. The relaxation that ignores the resource limits is solved by giving every task to its cheapest agent; the overloaded agents are then repaired by one small binary knapsack problem each, whose optimal values are added as penalties. Their paper then shows that this repaired bound is not an ad hoc heuristic: it is exactly the value of a Lagrangean relaxation of the GAP at an explicit choice of multipliers. This identity made the Ross–Soland bound the reference point for the later Lagrangean and column-generation methods for the GAP (for example Fisher, Jaikumar and Van Wassenhove, Management Science 1986 and Savelsbergh, Operations Research 1997).

Setting

Agents are I={1,…,m}I=\{1,\dots,m\}I={1,…,m} and tasks J={1,…,n}J=\{1,\dots,n\}J={1,…,n}. Giving task jjj to agent iii costs cijc_{ij}cij​ and uses rij≥0r_{ij}\ge 0rij​≥0 units of agent iii's resource; agent iii has bi>0b_i>0bi​>0 units. The problem is

(P)min⁡ ∑i∈I∑j∈Jcijxijs.t.∑j∈Jrijxij≤bi (i∈I),∑i∈Ixij=1 (j∈J),xij∈{0,1}.\text{(P)}\qquad \min\ \sum_{i\in I}\sum_{j\in J}c_{ij}x_{ij}\quad\text{s.t.}\quad\sum_{j\in J}r_{ij}x_{ij}\le b_i\ (i\in I),\quad\sum_{i\in I}x_{ij}=1\ (j\in J),\quad x_{ij}\in\{0,1\}.(P)min i∈I∑​j∈J∑​cij​xij​s.t.j∈J∑​rij​xij​≤bi​ (i∈I),i∈I∑​xij​=1 (j∈J),xij​∈{0,1}.

Dropping the resource constraints gives the relaxation (PR). It is solved by choosing, for each task jjj, a cheapest agent iji_jij​ with cijj=min⁡icijc_{i_jj}=\min_{i}c_{ij}cij​j​=mini​cij​ and setting xijj=1x_{i_jj}=1xij​j​=1; its value is Z=∑jcijjZ=\sum_jc_{i_jj}Z=∑j​cij​j​. Let Ji={j:ij=i}J_i=\{j: i_j=i\}Ji​={j:ij​=i} be the tasks this solution gives to agent iii, I′={i:∑j∈Jirij>bi}I'=\{i:\sum_{j\in J_i}r_{ij}>b_i\}I′={i:∑j∈Ji​​rij​>bi​} the overloaded agents, and di=∑j∈Jirij−bid_i=\sum_{j\in J_i}r_{ij}-b_idi​=∑j∈Ji​​rij​−bi​ the excess of agent iii. The penalty of moving task jjj away from iji_jij​ is pj=min⁡k≠ij(ckj−cijj)p_j=\min_{k\ne i_j}(c_{kj}-c_{i_jj})pj​=mink=ij​​(ckj​−cij​j​). For i∈I′i\in I'i∈I′ the binary knapsack problem

(PKi)min⁡ zi=∑j∈Jipjyijs.t.∑j∈Jirijyij≥di,yij∈{0,1}\text{(PK}_i)\qquad\min\ z_i=\sum_{j\in J_i}p_jy_{ij}\quad\text{s.t.}\quad\sum_{j\in J_i}r_{ij}y_{ij}\ge d_i,\quad y_{ij}\in\{0,1\}(PKi​)min zi​=j∈Ji​∑​pj​yij​s.t.j∈Ji​∑​rij​yij​≥di​,yij​∈{0,1}

chooses the cheapest set of tasks to move off agent iii; call its optimal value zi∗z^*_izi∗​. The knapsack bound is

LB=Z+∑i∈I′zi∗.\mathrm{LB}=Z+\sum_{i\in I'}z^*_i .LB=Z+i∈I′∑​zi∗​.

Dualizing the assignment constraints with multipliers λj\lambda_jλj​ gives the Lagrangean relaxation

(PRλ)min⁡ ∑i∈I∑j∈Jcijxij+∑j∈Jλj(1−∑i∈Ixij)s.t.∑j∈Jrijxij≤bi (i∈I),xij∈{0,1}.\text{(PR}_\lambda)\qquad\min\ \sum_{i\in I}\sum_{j\in J}c_{ij}x_{ij}+\sum_{j\in J}\lambda_j\Bigl(1-\sum_{i\in I}x_{ij}\Bigr)\quad\text{s.t.}\quad\sum_{j\in J}r_{ij}x_{ij}\le b_i\ (i\in I),\quad x_{ij}\in\{0,1\}.(PRλ​)min i∈I∑​j∈J∑​cij​xij​+j∈J∑​λj​(1−i∈I∑​xij​)s.t.j∈J∑​rij​xij​≤bi​ (i∈I),xij​∈{0,1}.

Finally c1jc_{1j}c1j​ and c2jc_{2j}c2j​ are the smallest and second smallest of c1j,…,cmjc_{1j},\dots,c_{mj}c1j​,…,cmj​, counted with multiplicity.

Formalization targets

Goal: the knapsack bound is the Lagrangean bound at λ=c2\lambda=c_2λ=c2​

For every cheapest-agent choice j↦ijj\mapsto i_jj↦ij​ and every choice of optimal knapsack solutions,

LB=min⁡{∑i∑jcijxij+∑jc2j(1−∑ixij) : x feasible for (PRλ)},\mathrm{LB}=\min\Bigl\{\sum_{i}\sum_{j}c_{ij}x_{ij}+\sum_{j}c_{2j}\Bigl(1-\sum_{i}x_{ij}\Bigr)\ :\ x\ \text{feasible for (PR}_\lambda)\Bigr\},LB=min{i∑​j∑​cij​xij​+j∑​c2j​(1−i∑​xij​) : x feasible for (PRλ​)},

the minimum being attained, and consequently LB≤∑i∑jcijxij\mathrm{LB}\le\sum_i\sum_jc_{ij}x_{ij}LB≤∑i​∑j​cij​xij​ for every xxx feasible for (P). This is the paper's "principal result of this Lagrangean analysis" (§2, p. 96). It has no constants to improve; it is an identity between two optimization problems.

Milestones

In the paper's order of use: (PR) is solved by the cheapest agents (pp. 93–94); every lower bound on (PRλ_\lambdaλ​) is a lower bound on (P) (p. 95); (PRλ_\lambdaλ​) separates into one binary knapsack per agent (p. 95); at λ=c2\lambda=c_2λ=c2​ the variables that are zero in the (PR) solution can be fixed at zero, the substitution yij=1−xijy_{ij}=1-x_{ij}yij​=1−xij​ turns agent iii's part into (PKi_ii​), pj=c2j−c1jp_j=c_{2j}-c_{1j}pj​=c2j​−c1j​, and agent iii's part has value −∑j∈Jipj+zi∗-\sum_{j\in J_i}p_j+z^*_i−∑j∈Ji​​pj​+zi∗​ (p. 96). Two side results close the section: the solution obtained by moving the tasks the knapsacks select has cost exactly LB, so it is optimal whenever it is feasible (pp. 94–95); and the optimal dual multipliers of the bounded-variable linear program (PRL_LL​) are exactly the vectors with c1j≤λj≤c2jc_{1j}\le\lambda_j\le c_{2j}c1j​≤λj​≤c2j​ (pp. 95–96).

Significance

The identity says that a bound computed from one sorting pass and a handful of small knapsacks equals a Lagrangean dual bound at a closed-form multiplier. Validity of LB for (P) then follows from weak Lagrangean duality alone, and the multiplier c2c_2c2​ is the upper end of the range of optimal dual multipliers of the linear program (PRL_LL​), which the paper singles out as a suitable choice of multipliers. The rebuilt solution gives the algorithm a feasible incumbent at no extra cost whenever the knapsack repairs happen to respect all budgets.

The result is proved in the paper, in one sentence. The mission turns that sentence into checked statements: the separation of (PRλ_\lambdaλ​), the reduction of each agent's subproblem to (PKi_ii​), the handling of ties among cheapest agents, and the role of nonnegative resources. As far as a search of the platform shows, nothing about the generalized assignment problem or its Lagrangean bounds has been formalized; the definitions here (assignment relaxations, per-agent knapsacks, bounded-variable duals) are reusable for other GAP and facility-location missions.

Difficulty

The Lagrangean relaxation at λ=c2\lambda=c_2λ=c2​ is a larger problem than the knapsack bound suggests: a feasible xxx may give a task to several agents or to none, and may use any agent, not only the cheapest one. The knapsack bound, by contrast, only looks at the tasks each agent receives in the (PR) solution. The paper bridges the two in one sentence of three observations, and each observation depends on a condition the sentence does not state: the sign of the resource coefficients, the treatment of agents that are not overloaded (for which no knapsack is solved), and ties among cheapest agents, which make some penalties zero and require the statement to hold for every tie-break. An inequality in one direction only (LB is a valid bound) is not the claim; the equality needs a feasible point of (PRλ_\lambdaλ​) whose value is exactly LB.

Formalization scope

Agents are Fin m and tasks Fin n, indexed from 0. Costs, resources, budgets, multipliers and variables are real numbers; a 0-1 variable is a real equal to 0 or 1, so the paper's sums are literal. The cheapest-agent selection is an arbitrary function a : Fin n → Fin m with IsCheapest c a, so every statement holds for every tie-break. pjp_jpj​ and c2jc_{2j}c2j​ are minima over the other agents, which requires m≥2m\ge2m≥2 (hm : 1 < m); c2jc_{2j}c2j​ is proved to be the second smallest cost with multiplicity. Optimal values are never encoded as sInf: zi∗z^*_izi∗​ is the objective of a given optimal knapsack solution, and "the bound provided by (PRλ_\lambdaλ​)" is stated as a lower bound over all feasible points that is attained.

Standing hypotheses: bi>0b_i>0bi​>0 (printed on p. 92), rij≥0r_{ij}\ge0rij​≥0 (implicit in "the resource required", and necessary: with a negative rijr_{ij}rij​ both (PRλ_\lambdaλ​) and (P) can fall below LB), and m≥2m\ge2m≥2. All costs are finite; the "not permissible" pairs of the paper's numerical example are outside the model.

A formalization that states only LB≤\mathrm{LB}\leLB≤ every (PRλ_\lambdaλ​) value, or that restricts the (PRλ_\lambdaλ​) competitors to the (PR) support or to at most one agent per task, would be a weaker theorem and does not meet the goal. The (PRL_LL​) dual is written out explicitly with multipliers uij≥0u_{ij}\ge0uij​≥0 for the bounds xij≤1x_{ij}\le1xij​≤1; "each optimal dual multiplier lies anywhere in the range c1j≤λj≤c2jc_{1j}\le\lambda_j\le c_{2j}c1j​≤λj​≤c2j​" is read as "the optimal multipliers are exactly this box".

Needed infrastructure is only finite sums over Fin and Finset.inf'. Proofs of the milestones, and lemmas on separable binary programs that could serve other Lagrangean-relaxation missions, are welcome.

Selected references

  • G. T. Ross and R. M. Soland, A branch and bound algorithm for the generalized assignment problem, Mathematical Programming 8 (1975) 91–103. https://doi.org/10.1007/BF01580430
  • A. M. Geoffrion, Lagrangean relaxation for integer programming, Mathematical Programming Study 2 (1974) 82–114. https://doi.org/10.1007/BFb0120690
  • M. L. Fisher, R. Jaikumar and L. N. Van Wassenhove, A multiplier adjustment method for the generalized assignment problem, Management Science 32 (1986) 1095–1103. https://doi.org/10.1287/mnsc.32.9.1095
  • M. Savelsbergh, A branch-and-price algorithm for the generalized assignment problem, Operations Research 45 (1997) 831–841. https://doi.org/10.1287/opre.45.6.831
10 thms4 active usersReviewed
🏆Completed
CombinatoricsGraph TheoryOperations Research·Captain: mikedeng1

Cones of Matrices and Set-Functions and 0–1 Optimization II: One Round of N on the Stable Set Polytope Gives Exactly the Odd Hole ConstraintsResearch Paper

Motivation

The stable set problem (vertex packing) asks for a largest set of pairwise non-adjacent nodes of a graph. It is NP-hard, and its polyhedral study, the description of the stable set polytope STAB(G)\mathrm{STAB}(G)STAB(G) by linear inequalities, is one of the most studied topics of polyhedral combinatorics. Classes of valid inequalities (clique, odd hole, odd antihole, wheel constraints) and the graph classes they describe exactly (perfect, ttt-perfect, hhh-perfect graphs) organize much of that literature; see Grötschel, Lovász and Schrijver, Geometric Algorithms and Combinatorial Optimization (Springer, 1988).

Lovász and Schrijver (SIAM J. Optim. 1(2), 1991) introduced a general lift-and-project procedure for 0–1 programs: lift a relaxation KKK into a space of matrices, impose linear conditions that every 0–1 point satisfies, and project back. One round of their operator NNN gives a tighter relaxation N(K)N(K)N(K) that still contains every 0–1 point of KKK; nnn rounds give the 0–1 hull. The procedure is an ancestor of the Sherali–Adams and Lasserre hierarchies, and the stable set problem is its first test case. This mission formalizes the paper's exact description of what one round of NNN does to the fractional stable set polytope: it adds precisely the odd hole constraints.

Setting

Let G=(V,E)G = (V, E)G=(V,E) be a finite graph with no isolated nodes, n=∣V∣n = |V|n=∣V∣. Vectors of RV∪{0}\mathbb{R}^{V \cup \{0\}}RV∪{0} have a distinguished coordinate x0x_0x0​; RV\mathbb{R}^VRV sits inside as the hyperplane H0={x0=1}H_0 = \{x_0 = 1\}H0​={x0​=1}, via x↦(1,x)x \mapsto (1, x)x↦(1,x).

  • FRAC(G)⊆RV\mathrm{FRAC}(G) \subseteq \mathbb{R}^VFRAC(G)⊆RV is the solution set of the nonnegativity constraints xi≥0x_i \ge 0xi​≥0 (i∈Vi \in Vi∈V) and the edge constraints xi+xj≤1x_i + x_j \le 1xi​+xj​≤1 (ij∈Eij \in Eij∈E).
  • FR(G)⊆RV∪{0}\mathrm{FR}(G) \subseteq \mathbb{R}^{V\cup\{0\}}FR(G)⊆RV∪{0} is the cone given by xi≥0x_i \ge 0xi​≥0 and xi+xj≤x0x_i + x_j \le x_0xi​+xj​≤x0​; it is the cone spanned by the vectors (1,x)(1, x)(1,x) with x∈FRAC(G)x \in \mathrm{FRAC}(G)x∈FRAC(G).
  • QQQ is the cone spanned by the 0–1 vectors with x0=1x_0 = 1x0​=1. For a convex cone KKK, its polar cone is K∗={u:uTx≥0 ∀x∈K}K^* = \{u : u^{\mathsf T}x \ge 0 \ \forall x \in K\}K∗={u:uTx≥0 ∀x∈K}.
  • M(K)=M(K,Q)M(K) = M(K, Q)M(K)=M(K,Q) is the set of (n+1)×(n+1)(n+1)\times(n+1)(n+1)×(n+1) matrices Y=(yij)Y = (y_{ij})Y=(yij​) that are symmetric, satisfy yii=y0iy_{ii} = y_{0i}yii​=y0i​ for i∈Vi \in Vi∈V, and satisfy uTYv≥0u^{\mathsf T} Y v \ge 0uTYv≥0 for all u∈K∗u \in K^*u∈K∗, v∈Q∗v \in Q^*v∈Q∗.
  • N(K)={Ye0:Y∈M(K)}N(K) = \{Y e_0 : Y \in M(K)\}N(K)={Ye0​:Y∈M(K)}, and N(G)={x∈RV:(1,x)∈N(FR(G))}N(G) = \{x \in \mathbb{R}^V : (1, x) \in N(\mathrm{FR}(G))\}N(G)={x∈RV:(1,x)∈N(FR(G))}.
  • A set C⊆VC \subseteq VC⊆V is an odd hole if it induces a chordless cycle of odd length ∣C∣≥3|C| \ge 3∣C∣≥3 (triangles included). Its odd hole constraint is ∑i∈Cxi≤12(∣C∣−1)\sum_{i \in C} x_i \le \frac12(|C| - 1)∑i∈C​xi​≤21​(∣C∣−1).

Formalization targets

Goal: Theorem 2.3 (p. 178)

For every finite graph GGG without isolated nodes,

N(G)={x∈RV:xi≥0 (i∈V),  xi+xj≤1 (ij∈E),  ∑i∈Cxi≤12(∣C∣−1) (C an odd hole)}.N(G) = \Big\{x \in \mathbb{R}^V : x_i \ge 0\ (i \in V),\ \ x_i + x_j \le 1\ (ij \in E),\ \ \sum_{i \in C} x_i \le \tfrac12(|C|-1)\ (C \text{ an odd hole})\Big\}.N(G)={x∈RV:xi​≥0 (i∈V),  xi​+xj​≤1 (ij∈E),  i∈C∑​xi​≤21​(∣C∣−1) (C an odd hole)}.

Milestones, in the order the proof uses them

  1. Lemma 1.3 (p. 171): for a convex cone K⊆QK \subseteq QK⊆Q and i∈Vi \in Vi∈V, N(K)⊆(K∩Hi)+(K∩Gi)N(K) \subseteq (K \cap H_i) + (K \cap G_i)N(K)⊆(K∩Hi​)+(K∩Gi​), with Hi={xi=0}H_i = \{x_i = 0\}Hi​={xi​=0}, Gi={xi=x0}G_i = \{x_i = x_0\}Gi​={xi​=x0​}.
  2. Lemma 2.2 (p. 178): if both the deletion and the contraction of some node vvv give inequalities valid for KKK, then aTx≤ba^{\mathsf T}x \le baTx≤b is valid for N(K)N(K)N(K).
  3. Part (1) of the proof of Theorem 2.3 (p. 178): for an odd hole CCC and i∈Ci \in Ci∈C, the deletion and contraction of iii in the odd hole constraint are valid for FRAC(G)\mathrm{FRAC}(G)FRAC(G).
  4. Observation of Section 2.b (p. 177): every Y∈M(FR(G))Y \in M(\mathrm{FR}(G))Y∈M(FR(G)) has yij=0y_{ij} = 0yij​=0 for ij∈Eij \in Eij∈E.
  5. Part (2) of the proof of Theorem 2.3 (p. 178): x∈N(G)x \in N(G)x∈N(G) if and only if some nonnegative symmetric YYY with y00=1y_{00} = 1y00​=1, yi0=yii=xiy_{i0} = y_{ii} = x_iyi0​=yii​=xi​ satisfies xi+xj+xk−1≤yik+yjk≤xkx_i + x_j + x_k - 1 \le y_{ik} + y_{jk} \le x_kxi​+xj​+xk​−1≤yik​+yjk​≤xk​ for all i,j,ki, j, ki,j,k with ij∈Eij \in Eij∈E.
  6. Lemma 2.4 (p. 178): a system a(ij)≤yi+yj≤b(ij)a(ij) \le y_i + y_j \le b(ij)a(ij)≤yi​+yj​≤b(ij), y≥0y \ge 0y≥0, y∣U=0y|_U = 0y∣U​=0 on a graph is infeasible if and only if a walk with a negative alternating sum of one of four types exists.

Significance

Theorem 2.3 gives a complete description of one round of NNN on the stable set problem: the only new constraints are the odd hole constraints. Consequences:

  • For ttt-perfect graphs (those for which nonnegativity, edge and odd hole constraints describe STAB(G)\mathrm{STAB}(G)STAB(G)), N(G)=STAB(G)N(G) = \mathrm{STAB}(G)N(G)=STAB(G).
  • It is the base case for the paper's bounds on the NNN-index of stable set inequalities (Theorem 2.13), and it contrasts with the semidefinite operator N+N_+N+​, which after one round already satisfies clique, odd antihole and wheel constraints.
  • Lemma 2.4 is a combinatorial feasibility criterion for systems with two variables per inequality, useful beyond this paper.

The result has been proved since 1991. At the time of drafting, Prove2Me holds no formalization of it or of any part of the Lovász–Schrijver construction, and Mathlib has none. The mission produces a formal account of the NNN operator on the stable set polytope and a formal proof of the walk criterion for two-variable systems.

Difficulty

The inclusion of N(G)N(G)N(G) in the odd hole system is a short argument once Lemma 1.3 is available. The reverse inclusion is the substance: given xxx satisfying all odd hole constraints, one must exhibit a lifted matrix YYY. A direct appeal to Farkas' lemma yields a certificate with no visible relation to odd cycles; the difficulty is to show that every obstruction to solvability of the matrix system forces a violated odd hole constraint, which is what Lemma 2.4 and the analysis of its four walk types accomplish. Case (d) of that analysis needs the odd hole constraints; the other cases need only the edge constraints. Lemma 2.4 itself is called folklore on the page and is stated without proof there.

A further point: Lemma 2.4 is stated for lower bounds 0≤a0 \le a0≤a, while the lower bounds that arise from the matrix system, xi+xj+xk−1x_i + x_j + x_k - 1xi​+xj​+xk​−1, can be negative.

Formalization scope

  • Coordinates of RV∪{0}\mathbb{R}^{V\cup\{0\}}RV∪{0} are indexed by Option V, with none the coordinate x0x_0x0​. Graphs are Mathlib SimpleGraphs on a finite type VVV with decidable adjacency. Every statement about a graph carries the paper's standing assumption that GGG has no isolated nodes (∀ v, ∃ w, G.Adj v w).
  • MMM is defined by condition (iii), never by its rewritings. Lemma 1.3 and Lemma 2.2 take the cone KKK closed, a hypothesis the paper leaves tacit (its cones are polyhedral); for a non-closed KKK Lemma 1.3 is false. FR(G)\mathrm{FR}(G)FR(G) is polyhedral, so the goal needs no such hypothesis.
  • FR(G)\mathrm{FR}(G)FR(G) is defined by its constraints; this agrees with the cone over FRAC(G)\mathrm{FRAC}(G)FRAC(G) because GGG has no isolated nodes.
  • Lemma 2.2 is stated in cone form: KKK is any closed convex cone inside FR(G)\mathrm{FR}(G)FR(G), and validity is read on the slice x0=1x_0 = 1x0​=1. The paper's extra hypothesis STAB(G)⊆K\mathrm{STAB}(G) \subseteq KSTAB(G)⊆K is dropped, which strengthens the lemma.
  • Deletion and contraction of a node are coefficient vectors on the same graph (coefficients set to 000), not inequalities on the subgraphs G−vG - vG−v and G−Γ(v)−vG - \Gamma(v) - vG−Γ(v)−v.
  • Odd holes are chordless odd cycles including triangles; triangles are needed, as 121\tfrac12\mathbf 121​1 satisfies all other constraints on a triangle.
  • The matrix system of part (2) is stated as an equivalence; the page uses one direction.
  • Lemma 2.4 uses edge values on unordered pairs and strict inequalities, exactly as printed.

A trivializing formalization is ruled out: the goal is the set equality for every graph without isolated nodes, not the existence of a lifted matrix and not a single graph.

Not formalized here: the semidefinite operator N+N_+N+​, the operator N^\hat NN^, algorithmic statements (Theorems 1.6, 2.1, Corollary 2.5), and the set-function results of Section 3.

Reusable beyond this mission: the matrix cone layer (QQQ, MMM, NNN), the stable-set cones, and the two-variable feasibility criterion of Lemma 2.4. Contributions of any of the milestones, and of general facts about polar cones of polyhedral cones in this setting, are welcome.

Selected references

  • L. Lovász and A. Schrijver, Cones of matrices and set-functions and 0–1 optimization, SIAM Journal on Optimization 1(2) (1991) 166–190. https://doi.org/10.1137/0801013
  • M. Grötschel, L. Lovász and A. Schrijver, Geometric Algorithms and Combinatorial Optimization, Springer, 1988. https://doi.org/10.1007/978-3-642-97881-4
  • H. D. Sherali and W. P. Adams, A hierarchy of relaxations between the continuous and convex hull representations for zero-one programming problems, SIAM Journal on Discrete Mathematics 3(3) (1990) 411–430. https://doi.org/10.1137/0403036
13 thms4 active usersReviewed
🏆Completed
Convex OptimizationOperations ResearchOptimization·Captain: mikedeng1

On Polyhedral Approximations of the Second-Order Cone I: A Compact Polyhedral Approximation of the Lorentz ConeResearch Paper

Motivation

Conic quadratic programs (second-order cone programs) minimize a linear objective subject to linear constraints and constraints of the form ∥Aℓx−bℓ∥2≤cℓTx−dℓ\|A_\ell x-b_\ell\|_2\le c_\ell^Tx-d_\ell∥Aℓ​x−bℓ​∥2​≤cℓT​x−dℓ​. They model robust linear programs with ellipsoidal uncertainty, truss topology design, contact problems with Coulomb friction, and convex quadratically constrained quadratic programs. In theory they are no harder than linear programs of the same size; in practice, linear programming software handles far larger instances than conic quadratic solvers did at the time of writing (Ben-Tal & Nemirovski 2001, pp. 193–195). This raises a question about geometry rather than algorithms: can a second-order cone be replaced by a polyhedral cone of moderate size without losing much accuracy?

The obvious answer — circumscribe the cone by a polyhedral cone with many facets — fails: the number of facets must grow exponentially in the dimension, even for constant accuracy. Ben-Tal and Nemirovski showed that auxiliary variables change the picture completely: a projection of a polyhedral cone can approximate the Lorentz cone with size only O(kln⁡(1/ε))O(k\ln(1/\varepsilon))O(kln(1/ε)). The construction is now standard; it underlies, for instance, the lifted linear-programming branch-and-bound algorithm for mixed-integer conic quadratic programs of Vielma, Ahmed & Nemhauser 2008.

Setting

For y∈Rky\in\mathbb R^ky∈Rk let ∥y∥2=y12+⋯+yk2\|y\|_2=\sqrt{y_1^2+\dots+y_k^2}∥y∥2​=y12​+⋯+yk2​​. The (k+1)(k+1)(k+1)-dimensional Lorentz cone is

Lk={(y,t)∈Rk×R∣t≥∥y∥2}.L^k=\{(y,t)\in\mathbb R^k\times\mathbb R\mid t\ge\|y\|_2\}.Lk={(y,t)∈Rk×R∣t≥∥y∥2​}.

Fix ε>0\varepsilon>0ε>0. A polyhedral ε\varepsilonε-approximation of LkL^kLk is a linear map

Π(y,t,u):Rk×R×Rp→Rq\Pi(y,t,u):\mathbb R^k\times\mathbb R\times\mathbb R^{p}\to\mathbb R^{q}Π(y,t,u):Rk×R×Rp→Rq

such that

  1. if (y,t)∈Lk(y,t)\in L^k(y,t)∈Lk, there is u∈Rpu\in\mathbb R^pu∈Rp with Π(y,t,u)≥0\Pi(y,t,u)\ge0Π(y,t,u)≥0 (componentwise);
  2. if Π(y,t,u)≥0\Pi(y,t,u)\ge0Π(y,t,u)≥0 for some uuu, then ∥y∥2≤(1+ε)t\|y\|_2\le(1+\varepsilon)t∥y∥2​≤(1+ε)t.

Equivalently, the polyhedral cone {(y,t,u)∣Π(y,t,u)≥0}\{(y,t,u)\mid\Pi(y,t,u)\ge0\}{(y,t,u)∣Π(y,t,u)≥0} projects onto a cone lying between LkL^kLk and its (1+ε)(1+\varepsilon)(1+ε)-extension. The size of the approximation is p+qp+qp+q: the number of auxiliary variables plus the number of linear inequalities (an equation counts as two).

The construction in the paper uses a tower of variables: for k=2θk=2^\thetak=2θ, the coordinates y1,…,yky_1,\dots,y_ky1​,…,yk​ form generation 000, each consecutive pair of generation ℓ−1\ell-1ℓ−1 has a successor in generation ℓ\ellℓ (yiℓy_i^\ellyiℓ​ has parents y2i−1ℓ−1,y2iℓ−1y_{2i-1}^{\ell-1},y_{2i}^{\ell-1}y2i−1ℓ−1​,y2iℓ−1​), and the single variable of generation θ\thetaθ is ttt. It also uses an explicit linear system (8) in variables ξj,ηj\xi^j,\eta^jξj,ηj, j=0,…,νj=0,\dots,\nuj=0,…,ν, with trigonometric coefficients cos⁡(π/2j+1)\cos(\pi/2^{j+1})cos(π/2j+1), sin⁡(π/2j+1)\sin(\pi/2^{j+1})sin(π/2j+1), tan⁡(π/2ν+1)\tan(\pi/2^{\nu+1})tan(π/2ν+1), whose accuracy is δ(ν)=1/cos⁡(π/2ν+1)−1\delta(\nu)=1/\cos(\pi/2^{\nu+1})-1δ(ν)=1/cos(π/2ν+1)−1.

Formalization targets

Goal: Theorem 1.1

There is an absolute constant CCC such that for every positive integer kkk and every ε∈(0,1]\varepsilon\in(0,1]ε∈(0,1], LkL^kLk admits a polyhedral ε\varepsilonε-approximation with

pk+qk≤C kln⁡2ε.p_k+q_k\le C\,k\ln\frac{2}{\varepsilon}.pk​+qk​≤Cklnε2​.

The constant is not fixed; the goal asserts only the order of growth, which is what the paper claims.

Milestones

  1. §2, Eq. (5). For k=2θk=2^\thetak=2θ, θ≥1\theta\ge1θ≥1: (y,t)(y,t)(y,t) extends to a tower solving [y2i−1ℓ−1]2+[y2iℓ−1]2≤yiℓ\sqrt{[y_{2i-1}^{\ell-1}]^2+[y_{2i}^{\ell-1}]^2}\le y_i^\ell[y2i−1ℓ−1​]2+[y2iℓ−1​]2​≤yiℓ​ for all i,ℓi,\elli,ℓ if and only if ∥y∥2≤t\|y\|_2\le t∥y∥2​≤t.
  2. §2, Eqs. (6)–(7). Placing polyhedral εℓ\varepsilon_\ellεℓ​-approximations of L2L^2L2 on every level of the tower yields a polyhedral approximation of LkL^kLk with 1+ε=∏ℓ=1θ(1+εℓ)1+\varepsilon=\prod_{\ell=1}^\theta(1+\varepsilon_\ell)1+ε=∏ℓ=1θ​(1+εℓ​).
  3. Proposition 2.1 (i), (ii) and Eq. (9). System (8) is a polyhedral δ(ν)\delta(\nu)δ(ν)-approximation of L2L^2L2, and δ(ν)=O(4−ν)\delta(\nu)=O(4^{-\nu})δ(ν)=O(4−ν).
  4. Proof of Theorem 1.1, system (10), property 3. System (8) with parameter νℓ\nu_\ellνℓ​ on level ℓ\ellℓ of the tower approximates L2θL^{2^\theta}L2θ with quality β=∏ℓ=1θ1/cos⁡(π/2νℓ+1)−1\beta=\prod_{\ell=1}^\theta 1/\cos(\pi/2^{\nu_\ell+1})-1β=∏ℓ=1θ​1/cos(π/2νℓ​+1)−1.
  5. Proof of Theorem 1.1, choice of νℓ\nu_\ellνℓ​. With νℓ=⌊c ℓln⁡(2/ε)⌋\nu_\ell=\lfloor c\,\ell\ln(2/\varepsilon)\rfloorνℓ​=⌊cℓln(2/ε)⌋: β≤ε\beta\le\varepsilonβ≤ε and ∑ℓ2θ−ℓνℓ≤C 2θln⁡(2/ε)\sum_\ell 2^{\theta-\ell}\nu_\ell\le C\,2^\theta\ln(2/\varepsilon)∑ℓ​2θ−ℓνℓ​≤C2θln(2/ε).

Significance

The theorem shows that conic quadratic constraints are, up to a factor logarithmic in the accuracy, no more expensive to express as linear constraints than they are in their native form. Consequences listed in the paper include approximating convex quadratically constrained quadratic programs, robust counterparts of linear programs with ellipsoidal uncertainty, and problems with low-dimensional cones (Coulomb friction, k≤3k\le3k≤3; truss design, k≤2k\le2k≤2) by linear programs of comparable size. Together with the matching lower bound of §3 of the same paper (a separate mission of this series), it pins down the size of the best polyhedral approximation up to constants. The recursive halving of dimensions through the tower of 3-dimensional cones is a reusable device for other rotation-invariant cones.

The result is proved in the paper; as far as is known it has not been machine-checked. This mission produces a formal proof of the construction, including the trigonometric estimate δ(ν)=O(4−ν)\delta(\nu)=O(4^{-\nu})δ(ν)=O(4−ν) and the explicit linear encoding with its size count. Explicit values of the absolute constants are welcome as additional results.

Difficulty

The planar estimate is the core. Part (ii) of Proposition 2.1 must hold for every solution of the inequality system (8), not only for the solution one would write down for a given point of L2L^2L2; an argument that tracks only the intended solution proves part (i) and nothing about part (ii). The accuracy must also come out as 1/cos⁡(π/2ν+1)−11/\cos(\pi/2^{\nu+1})-11/cos(π/2ν+1)−1, geometric in ν\nuν; a bound that decays only polynomially in ν\nuν would give size poly(1/ε)\mathrm{poly}(1/\varepsilon)poly(1/ε) instead of ln⁡(1/ε)\ln(1/\varepsilon)ln(1/ε). The naive idea of approximating LkL^kLk directly by tangent hyperplanes is ruled out by the exponential facet count mentioned above; the auxiliary variables are indispensable. The second difficulty is bookkeeping: packaging k−1k-1k−1 copies of system (8) on a tower of depth θ=log⁡2k\theta=\log_2 kθ=log2​k into a single linear map, counting its variables and inequalities exactly, handling kkk that is not a power of two, and summing the accuracies so that the total size is O(kln⁡(2/ε))O(k\ln(2/\varepsilon))O(kln(2/ε)) rather than O(kln⁡kln⁡(1/ε))O(k\ln k\ln(1/\varepsilon))O(klnkln(1/ε)).

Formalization scope

  • Vectors of Rk\mathbb R^kRk are Fin k → ℝ. The norm ∥y∥2\|y\|_2∥y∥2​ is written out as eucNorm y = Real.sqrt (∑ i, y i ^ 2); the norm Mathlib attaches to Fin k → ℝ is the sup norm, under which the cone would be polyhedral and the theorem trivial.
  • A polyhedral approximation is an R\mathbb RR-linear map (Fin k → ℝ) × ℝ × (Fin p → ℝ) →ₗ[ℝ] (Fin q → ℝ) and ≥0\ge0≥0 is the componentwise order. Linearity is essential: with an arbitrary map, Π(y,t)=t−∥y∥2\Pi(y,t)=t-\|y\|_2Π(y,t)=t−∥y∥2​ would be an exact approximation with p=0p=0p=0, q=1q=1q=1. Affine maps are not allowed either; the paper's approximations are homogeneous.
  • The paper's absolute constants O(1)O(1)O(1) are existential constants quantified before kkk, ε\varepsilonε and θ\thetaθ. The goal requires k≥1k\ge1k≥1 and ε∈(0,1]\varepsilon\in(0,1]ε∈(0,1], as in the paper; ln⁡\lnln is Real.log.
  • System (8) and system (10) are stated as propositions with the absolute values written out; their parameters ν\nuν, νℓ\nu_\ellνℓ​ are required to be positive integers, as in the paper (at ν=0\nu=0ν=0 the coefficient tan⁡(π/2)\tan(\pi/2)tan(π/2) would be evaluated as 000 by Lean).
  • Tower variables are indexed Y ℓ i with 0-based i, so the parents of Y ℓ i are Y (ℓ-1) (2i) and Y (ℓ-1) (2i+1); the milestones on (6)–(7) and (10) are stated on solution sets rather than on an explicit linear map. The size counts of (10) (properties 1–2) are not separate milestones; the arithmetic milestone on νℓ\nu_\ellνℓ​ records the bound on ∑ℓ2θ−ℓνℓ\sum_\ell 2^{\theta-\ell}\nu_\ell∑ℓ​2θ−ℓνℓ​ to which they reduce.
  • δ(ν)=O(1/4ν)\delta(\nu)=O(1/4^\nu)δ(ν)=O(1/4ν) is stated as ∃C>0, ∀ν≥1, δ(ν)≤C/4ν\exists C>0,\ \forall\nu\ge1,\ \delta(\nu)\le C/4^\nu∃C>0, ∀ν≥1, δ(ν)≤C/4ν.

A complete development needs: elementary trigonometry of π/2j\pi/2^{j}π/2j (available in Mathlib), rotations in the plane, finite products and sums over {1,…,θ}\{1,\dots,\theta\}{1,…,θ}, and a way to assemble many small linear systems into one linear map with an exact count of rows and columns. The last piece, and the tower of variables with the reduction from arbitrary kkk to a power of two, are reusable for other lifted polyhedral approximations. Contributions of any milestone, of explicit linear encodings of (8) and (10), and of the extension from k=2θk=2^\thetak=2θ to all kkk are welcome.

Selected references

  • A. Ben-Tal and A. Nemirovski, On Polyhedral Approximations of the Second-Order Cone, Mathematics of Operations Research 26(2):193–205, 2001. https://doi.org/10.1287/moor.26.2.193.10561
  • J. P. Vielma, S. Ahmed and G. L. Nemhauser, A lifted linear programming branch-and-bound algorithm for mixed-integer conic quadratic programs, INFORMS Journal on Computing 20(3):438–450, 2008. https://doi.org/10.1287/ijoc.1070.0256
  • A. Ben-Tal and A. Nemirovski, Robust convex optimization, Mathematics of Operations Research 23(4):769–805, 1998. https://doi.org/10.1287/moor.23.4.769
  • A. Ben-Tal and A. Nemirovski, Lectures on Modern Convex Optimization, SIAM, 2001. https://doi.org/10.1137/1.9780898718829
13 thms4 active usersReviewed
🏆Completed
Algorithmic Game TheoryConvex OptimizationMachine Learning+1·Captain: mikedeng1

Introduction to Online Convex Optimization VIII: Solving Zero-Sum Games and Linear Programs via Regret MinimizationTextbook

Motivation

Two-player zero-sum games and linear programming are, on their surface, unrelated pieces of 20th-century mathematics: von Neumann's minimax theorem for games (1928) was proved with tools from topology, and linear programming duality (Dantzig, 1940s) with convexity and geometry. Yet the two are formally equivalent — Dantzig recounts von Neumann conjecturing the equivalence outright, on first hearing a description of linear programming, because he had "just recently completed a book with Oscar Morgenstern on the theory of games" [Albers, Alexanderson, and Reid, More Mathematical People, 1990]. Freund and Schapire (1999) later showed that both concepts reduce, in one uniform way, to online regret minimization: a decades-old topological existence proof and a decades-old LP-duality argument both become corollaries of a single fact about no-regret learning. This mission formalizes the algorithmic content of that reduction — Hazan's Lemma 8.4, which is not merely an existence statement but a concrete, efficient algorithm with an explicit convergence rate.

Setting

A two-player zero-sum game in normal form is a real matrix A∈Rn×mA \in \mathbb{R}^{n \times m}A∈Rn×m (Hazan restricts entries to [−1,1][-1,1][−1,1] for interpretability as losses/rewards, a convention this mission's theorems drop as inessential — the argument is invariant to scaling and shifting). The row player picks a mixed strategy xxx in the probability simplex Δn={x∈Rn:xi≥0,∑ixi=1}\Delta_n = \{x \in \mathbb{R}^n : x_i \ge 0, \sum_i x_i = 1\}Δn​={x∈Rn:xi​≥0,∑i​xi​=1}; the column player picks y∈Δmy \in \Delta_my∈Δm​. The row player's expected loss, and simultaneously the column player's expected reward, is the bilinear form xTAyx^{\mathsf T} A yxTAy.

The row player's guaranteed loss is λR=min⁡x∈Δnmax⁡y∈ΔmxTAy\lambda_R = \min_{x \in \Delta_n} \max_{y \in \Delta_m} x^{\mathsf T} A yλR​=minx∈Δn​​maxy∈Δm​​xTAy: the smallest loss she can secure no matter what the column player does. Symmetrically, the column player's guaranteed reward is λC=max⁡y∈Δmmin⁡x∈ΔnxTAy\lambda_C = \max_{y \in \Delta_m} \min_{x \in \Delta_n} x^{\mathsf T} A yλC​=maxy∈Δm​​minx∈Δn​​xTAy. Always λR≥λC\lambda_R \ge \lambda_CλR​≥λC​ ("weak duality" — an elementary max-min/min-max inequality, Direction 1 of Section 8.3). Von Neumann's minimax theorem (Theorem 8.3) is the nontrivial converse: λR=λC\lambda_R = \lambda_CλR​=λC​, a common value λ⋆\lambda^\starλ⋆ called the value of the game, whose optimal strategies form a Nash equilibrium — already on the platform as AGT.zero_sum_minimax.

Algorithm 28 ("Simple LP", p. 147) computes an approximate equilibrium constructively. The row player runs a multiplicative-weights / Exponentiated Gradient update against the sequence of best-response losses the column player generates in a repeated TTT-round play of the game: starting from the uniform strategy x1=(1/n,…,1/n)x_1 = (1/n, \dots, 1/n)x1​=(1/n,…,1/n), at each round ttt the column player best-responds with yt∈arg⁡max⁡y∈ΔmxtTAyy_t \in \arg\max_{y \in \Delta_m} x_t^{\mathsf T} A yyt​∈argmaxy∈Δm​​xtT​Ay, and the row player updates xt+1(i)∝xt(i) e−η(Ayt)ix_{t+1}(i) \propto x_t(i)\, e^{-\eta (A y_t)_i}xt+1​(i)∝xt​(i)e−η(Ayt​)i​. The algorithm returns the time-averaged strategy xˉ=1T∑t=1Txt\bar{x} = \frac{1}{T}\sum_{t=1}^T x_txˉ=T1​∑t=1T​xt​.

Formalization targets

Goal — Lemma 8.4

max⁡y′∈ΔmxˉTAy′  ≤  λR(A)+2log⁡nT\max_{y' \in \Delta_m} \bar{x}^{\mathsf T} A y' \;\le\; \lambda_R(A) + \frac{\sqrt{2 \log n}}{\sqrt{T}}y′∈Δm​max​xˉTAy′≤λR​(A)+T​2logn​​

for the vector xˉ\bar{x}xˉ returned by Algorithm 28 after TTT rounds with learning rate η=2log⁡n/T\eta = \sqrt{2 \log n / T}η=2logn/T​. The book calls xˉ\bar{x}xˉ a "2log⁡n/T\sqrt{2 \log n}/\sqrt{T}2logn​/T​-approximate solution" to the zero-sum game — and, via Section 8.2.1's equivalence, to the linear program the game encodes — in exactly this sense. The goal is stated against λR\lambda_RλR​, the quantity the algorithm's own analysis produces; Theorem 8.3 identifies it with λC\lambda_CλC​ and with the book's λ⋆\lambda^\starλ⋆, so nothing about the bound is lost by this choice of rendering.

Supporting milestone — Eq. (8.1)

∑t=0T−1xtTAyt  ≤  min⁡x′∈Δn∑t=0T−1(x′)TAyt  +  2Tlog⁡n\sum_{t=0}^{T-1} x_t^{\mathsf T} A y_t \;\le\; \min_{x' \in \Delta_n} \sum_{t=0}^{T-1} (x')^{\mathsf T} A y_t \;+\; \sqrt{2T \log n}t=0∑T−1​xtT​Ayt​≤x′∈Δn​min​t=0∑T−1​(x′)TAyt​+2Tlogn​

the external-regret bound the row player's multiplicative-weights update achieves against the adaptively-chosen linear loss sequence ft(⋅)=(⋅)TAytf_t(\cdot) = (\cdot)^{\mathsf T} A y_tft​(⋅)=(⋅)TAyt​ — the single analytical fact the goal's proof needs.

Significance

The result itself. Lemma 8.4 gives a genuinely efficient algorithm: O(log⁡n/ε2)O(\log n / \varepsilon^2)O(logn/ε2) rounds of a trivial multiplicative update to reach an ε\varepsilonε-approximate value and equilibrium of an n×mn \times mn×m zero-sum game, and — through the equivalence with LP duality — an approximation algorithm for a broad class of linear programs, predating and prefiguring the multiplicative-weights-based approximation schemes surveyed by Arora, Hazan, and Kale (2012). It is also the constructive engine behind Theorem 8.3: unlike the classical topological proof of the minimax theorem, this one produces the equilibrium, not just its existence.

Formalizing it. The equilibrium-existence half of this story, Theorem 8.3, is already a published, proved-format Prove2Me theorem (AGT.zero_sum_minimax, from the Algorithmic Game Theory series) and is reused here as a reference item rather than redrafted. What that theorem does not capture — and what makes this mission non-trivial rather than a restatement — is the quantitative, algorithmic content: that one specific, simple, Hedge-type update, run for a specific number of rounds, provably gets within a specific, explicit distance of the value, using only the existence of some sublinear-regret online algorithm as a black box.

Difficulty

The tempting shortcut is to formalize only "no-regret learning dynamics converge to an equilibrium" as a qualitative statement, discharging it by citing AGT.zero_sum_minimax (equilibria exist) plus a generic regret bound. That collapses Lemma 8.4 into a restatement of Theorem 8.3 and drops exactly what is new here: the explicit rate 2log⁡n/T\sqrt{2\log n}/\sqrt{T}2logn​/T​, tied to one concrete update rule (Algorithm 28) rather than an arbitrary sublinear-regret black box. The real content is in chaining three quantitative facts — Eq. (8.1)'s specific regret bound for the multiplicative-weights update, the column player's best-response equality (Eq. (8.2)), and the definitional unfolding of λR\lambda_RλR​ — with none of the slack that a purely qualitative "an algorithm with sublinear regret exists" argument would tolerate.

Formalization scope

Matrices are Matrix (Fin n) (Fin m) ℝ with n, m ≥ 1 (empty strategy sets are excluded throughout, matching this mission's reference item AGT.zero_sum_minimax); mixed strategies use Mathlib's stdSimplex ℝ (Fin n). lambdaR/lambdaC are rendered with iInf/iSup over simplex membership, the same convention Introduction to Online Convex Optimization III fixed for RegretT earlier in this series. Algorithm 28's run is packaged as a Prop-valued structure (IsSimpleLPRun) rather than a computable function, in the style of this series' other algorithm-run definitions (IsHedgeRun, IsOnlineGradientDescent): initial uniform strategy, a best-response condition on the column player at every round, and the multiplicative-weights recursion on the row player, with the learning rate η left free and fixed to √(2 log n / T) only at the point the theorems need the book's specific constant.

The trivializing risk here is stating only that some sublinear-regret algorithm secures the bound (already implied, vacuously, by AGT.zero_sum_minimax plus any regret bound); this mission rules that out by fixing the exact update rule of Algorithm 28 in IsSimpleLPRun and proving the bound for that rule specifically, with the book's exact constant √(2 log n)/√T, not an unspecified O(·).

Chapter 5's Corollary 5.7 (the general RFTL/Exponentiated-Gradient regret bound) belongs to a different mission of this series and is not imported; eg_regret_bound restates, locally and self-containedly, exactly the instance of it this chapter's proof needs. A later mission for Chapter 5, once published, could supersede this local restatement by specializing its general bound — a natural contribution for a solver with that mission's Lean available.

Selected references

  • J. von Neumann, "Zur Theorie der Gesellschaftsspiele", Mathematische Annalen, 1928.
  • Y. Freund and R. E. Schapire, "Adaptive Game Playing Using Multiplicative Weights", Games and Economic Behavior, 1999. https://doi.org/10.1006/game.1999.0738
  • E. Hazan, Introduction to Online Convex Optimization, 2nd ed., 2022. arXiv:1909.05207v3
  • N. Nisan, T. Roughgarden, E. Tardos, and V. V. Vazirani (eds.), Algorithmic Game Theory, Cambridge University Press, 2007. https://doi.org/10.1017/CBO9780511800481
  • S. Arora, E. Hazan, and S. Kale, "The Multiplicative Weights Update Method: a Meta-Algorithm and Applications", Theory of Computing, 2012. https://doi.org/10.4086/toc.2012.v008a006
8 thms4 active users
🏆Completed
OptimizationTheoretical Computer Science·Captain: moutei

Primal-Dual Online Algorithms II: Finite LP Duality and Complementary SlacknessTextbook

Motivation

Almost every competitive online algorithm built by the primal-dual method rests on the same two facts about a pair of linear programs. The first is weak duality: any feasible solution of the dual is a lower bound on any feasible solution of the primal. The second is complementary slackness: if a feasible primal-dual pair satisfies a local, per-coordinate tightness condition, the pair is optimal — and if it satisfies that condition only up to factors α\alphaα and β\betaβ, the primal is within αβ\alpha\betaαβ of optimal.

The second fact in its approximate form is the engine of the whole method. An online algorithm cannot compute an optimum; what it can do is maintain a primal solution and a dual solution side by side so that each new request preserves an approximate tightness invariant. The approximate complementary slackness theorem then converts that local invariant into a global competitive ratio, with no reference to the optimum at all. Chapter 2 of Buchbinder's thesis states it as the background result on which the rest of the work is built.

Setting

Fix finite index types III (primal variables) and JJJ (primal constraints), a matrix A:I×J→RA : I \times J \to \mathbb{R}A:I×J→R, a cost vector c:I→Rc : I \to \mathbb{R}c:I→R and a right-hand side b:J→Rb : J \to \mathbb{R}b:J→R. The covering primal and packing dual are

(P)min⁡∑icixi  s.t.  ∑iAijxi ≥ bj  (∀j),x≥0,(P)\quad \min \sum_{i} c_i x_i \ \text{ s.t. } \ \sum_{i} A_{ij} x_i \ \ge\ b_j \ \ (\forall j), \qquad x \ge 0,(P)mini∑​ci​xi​  s.t.  i∑​Aij​xi​ ≥ bj​  (∀j),x≥0, (D)max⁡∑jbjyj  s.t.  ∑jAijyj ≤ ci  (∀i),y≥0.(D)\quad \max \sum_{j} b_j y_j \ \text{ s.t. } \ \sum_{j} A_{ij} y_j \ \le\ c_i \ \ (\forall i), \qquad y \ge 0.(D)maxj∑​bj​yj​  s.t.  j∑​Aij​yj​ ≤ ci​  (∀i),y≥0.

Note the index convention: AijA_{ij}Aij​ carries the primal-variable index first, so the primal constraint indexed by jjj sums over iii and the dual constraint indexed by iii sums over jjj.

Given α,β≥1\alpha, \beta \ge 1α,β≥1, the pair (x,y)(x,y)(x,y) satisfies approximate complementary slackness when

  • primal side: for every iii with xi>0x_i > 0xi​>0, ci/α ≤ ∑jAijyj ≤ ci\quad c_i/\alpha \ \le\ \sum_j A_{ij} y_j \ \le\ c_ici​/α ≤ ∑j​Aij​yj​ ≤ ci​;
  • dual side: for every jjj with yj>0y_j > 0yj​>0, bj ≤ ∑iAijxi ≤ β bj\quad b_j \ \le\ \sum_i A_{ij} x_i \ \le\ \beta\, b_jbj​ ≤ ∑i​Aij​xi​ ≤ βbj​.

Formalization targets

Goal — approximate complementary slackness

For a primal-feasible xxx, a dual-feasible yyy, and α,β≥1\alpha,\beta \ge 1α,β≥1 satisfying the two conditions above,

∑icixi ≤ αβ∑jbjyj.\sum_{i} c_i x_i \ \le\ \alpha\beta \sum_{j} b_j y_j .i∑​ci​xi​ ≤ αβj∑​bj​yj​.

Taking α=β=1\alpha = \beta = 1α=β=1 recovers exact complementary slackness and hence optimality of both members of the pair. The goal is stated with the source's hypotheses, including the two-sided bounds, rather than the weakest hypotheses that make the inequality go through; a separate item records the minimal-hypothesis strengthening.

Weak duality

∑jbjyj ≤ ∑icixifor every feasible x and y,\sum_j b_j y_j \ \le\ \sum_i c_i x_i \quad \text{for every feasible } x \text{ and } y,j∑​bj​yj​ ≤ i∑​ci​xi​for every feasible x and y,

with no nonnegativity assumption on AAA, bbb or ccc beyond feasibility itself.

Strong duality — imported, not reproved

Strong duality is not proved in this mission. The platform already carries LinearOptimization.lp_strong_duality, proved in this exact environment, for linear programs in Bertsimas–Tsitsiklis general form over Fin-indexed data. This mission's contribution is an adapter: from a primal optimum of (P)(P)(P), produce a dual optimum of (D)(D)(D) of equal value, for Fin-indexed instances. Reference items point at the imported theorem, its dual construction, and the dual-of-dual identity.

The biconditional — a dual optimum exists if and only if a primal optimum does — is deliberately left open. Weak duality does not derive the existence of a primal optimum from the existence of a dual one; the reverse implication needs strong duality applied to the dual program together with the dual-of-dual identity, and that reduction is not yet compiled. It is offered as a parallel target rather than claimed as established.

Significance

This mission is the foundation of the series. Every later mission — set cover, ski rental, and the online covering and packing problems that follow — states its approximation or competitiveness result as an instance of approximate complementary slackness. Formalizing it once, over arbitrary finite index types, is what makes the later missions short.

It also fills a real gap. Mathlib currently has no linear-programming duality: four separate attempts were closed unmerged. Approximate (α,β)(\alpha,\beta)(α,β) complementary slackness appears not to be formalized in any public library, so the goal theorem is, as far as we can determine, first of its kind.

Difficulty

The goal is a summation argument, not a deep theorem: the work is in handling the per-coordinate case split on xi>0x_i > 0xi​>0 versus xi=0x_i = 0xi​=0 and in interchanging a double sum. Three mechanical milestones isolate exactly those steps. The strong-duality adapter is the hard item, because it must reconcile two different presentations of the same program — index types, matrix orientation, and bundling all differ between our definitions and the imported theorem's.

Formalization scope

Definitions cover §2.1 of the source. Four distinct notions of "the program has a finite optimum" are separated on purpose — attained optimum, nonempty feasible set, bounded objective, and the conjunction — because the source's informal word "bounded" conflates them. The definitions are stated over arbitrary finite index types; the strong-duality items are stated only for Fin, because that is the only index type for which the imported dependency path exists.

Selected references

  • Niv Buchbinder, Designing Competitive Online Algorithms via a Primal-Dual Approach, PhD thesis, Tel Aviv University, 2008, §2.1, pp. 7–9. https://www.tau.ac.il/~nivb/download/phd-thsis.pdf
  • Dimitris Bertsimas and John N. Tsitsiklis, Introduction to Linear Optimization, Athena Scientific, 1997 — the general form used by the imported strong-duality theorem.
11 thms4 active usersReviewed
🏆Completed
Operations ResearchOptimization·Captain: Shuze Chen

Introduction to Linear Optimization XIII: Lagrangean Duality and Integer ProgrammingTextbook

Linear programming has a complete duality theory; integer programming does not — and the Lagrangean dual measures exactly how far duality reaches. This mission formalizes the duality theory of integer programming from Section 11.4 of Bertsimas–Tsitsiklis, built on the general linear programming duality of Section 4.10. For the integer program

ZIP=min⁡{c′x:Ax≥b, Dx≥d, x integer}Z_{IP} = \min\{c'x : Ax \ge b,\ Dx \ge d,\ x \text{ integer}\}ZIP​=min{c′x:Ax≥b, Dx≥d, x integer}

with integer data, the complicating constraints Ax≥bAx \ge bAx≥b are dualized with multipliers p≥0p \ge 0p≥0 over the tractable set X={x integer∣Dx≥d}X = \{x \text{ integer} \mid Dx \ge d\}X={x integer∣Dx≥d}: the dual function is

Z(p)=min⁡x∈X(c′x+p′(b−Ax))Z(p) = \min_{x \in X}\big(c'x + p'(b - Ax)\big)Z(p)=x∈Xmin​(c′x+p′(b−Ax))

and the Lagrangean dual is ZD=max⁡p≥0Z(p)Z_D = \max_{p \ge 0} Z(p)ZD​=maxp≥0​Z(p). Weak duality ZD≤ZIPZ_D \le Z_{IP}ZD​≤ZIP​ (Theorem 11.2) always holds, but strong duality can fail. The convex hull CH(X)CH(X)CH(X) of the integer points of a polyhedron with integer data is itself a polyhedron (Theorem 11.3, Meyer's theorem), and the capstone — Theorem 11.4, the central result of Section 11.4 — identifies the Lagrangean dual exactly: ZDZ_DZD​ equals the optimal cost of the linear program

min⁡{c′x:Ax≥b, x∈CH(X)}\min\{c'x : Ax \ge b,\ x \in CH(X)\}min{c′x:Ax≥b, x∈CH(X)}

. This is the geometric explanation of the strength of Lagrangean relaxation, yields the bound ordering ZLP≤ZD≤ZIPZ_{LP} \le Z_D \le Z_{IP}ZLP​≤ZD​≤ZIP​, and Corollary 11.1 characterizes exactly when the bounds collapse. The polyhedral engine is the general weak/strong duality pair (Theorems 4.17/4.18) over a primal min⁡c′x\min c'xminc′x s.t. Ax≥bAx \ge bAx≥b, x∈P={x∣Dx≥d}x \in P = \{x \mid Dx \ge d\}x∈P={x∣Dx≥d}, and the formulation-strength comparison Psub⊆PcutP_{sub} \subseteq P_{cut}Psub​⊆Pcut​ of Theorem 10.1 supplies the motivating principle that tighter relaxations of the same integer set give sharper bounds.

18 thms4 active usersReviewed
🏆Completed
Operations ResearchOptimization·Captain: Shuze Chen

Introduction to Linear Optimization VII: Cones, Extreme Rays, and the Resolution TheoremTextbook

How can an unbounded polyhedron be described by finitely many geometric objects? Sections 4.8-4.9 of Bertsimas-Tsitsiklis build the cone machinery: recession cones {d∣Ad≥0}\{d \mid Ad \ge 0\}{d∣Ad≥0} and their rays, extreme rays (defined, like basic solutions, by n−1n-1n−1 linearly independent active constraints), the pointedness criterion (Theorem 4.12: 000 is an extreme point of a polyhedral cone iff the cone contains no line iff nnn of the constraint vectors are linearly independent), and the characterization of unbounded linear programs (Theorems 4.13-4.14: over a pointed polyhedral cone, and then over any polyhedron with an extreme point, the optimal cost is −∞-\infty−∞ iff some extreme ray ddd has c′d<0c'd < 0c′d<0). The capstone is the resolution theorem (Theorem 4.15): a nonempty polyhedron PPP with at least one extreme point equals Q={∑iλixi+∑jθjwj∣λi≥0,θj≥0,∑iλi=1}Q = \{\sum_i \lambda_i x^i + \sum_j \theta_j w^j \mid \lambda_i \ge 0, \theta_j \ge 0, \sum_i \lambda_i = 1\}Q={∑i​λi​xi+∑j​θj​wj∣λi​≥0,θj​≥0,∑i​λi​=1} — the convex hull of its extreme points plus the cone generated by a complete set of its extreme rays. It specializes to Theorem 2.9 / Corollary 4.4 (a nonempty bounded polyhedron is the convex hull of its extreme points) and Corollary 4.5 (a pointed polyhedral cone is generated by its extreme rays). The converse, Theorem 4.16, states that every finitely generated set is a polyhedron — in particular the convex hull of finitely many vectors is a polyhedron. Together these form the Minkowski-Weyl equivalence of the two representations of polyhedra, verified absent from Mathlib and the genuine content of this mission.

21 thms4 active usersReviewed
Operations ResearchOptimization·Captain: mikedeng1

On-Demand Delivery from Stores: Dynamic Dispatching and Routing with Random Demand II: A Route Whose Reduced Cost Exceeds ub − φ(RF1) Is in No Optimal Solution of the Routing Subproblem SF1Research Paper

Motivation

Grocery chains and restaurants that promise delivery within an hour or less must decide, at each dispatch epoch, which drivers to send out and along which routes. Liu and Luo (arXiv:2107.13058v3; Manufacturing & Service Operations Management, 2023) model this as a finite-horizon dynamic program and solve its per-period approximation with a logic-based Benders decomposition: a master problem chooses how many drivers to commit to future periods, and a routing subproblem then picks the routes that serve the current orders.

The routing subproblem is a set-partitioning integer program over an exponentially large route set. The paper solves it with a route enumeration algorithm, which generates only the routes that can still appear in an optimal solution. The pruning rule that makes this work is Lemma 2 (p. 19): a route whose LP reduced cost exceeds the gap between an upper bound and the LP bound cannot appear in any optimal solution. Rules of this kind, reduced-cost fixing for set-partitioning formulations, are a standard component of exact vehicle-routing algorithms (see, e.g., Baldacci, Christofides, Mingozzi 2008). This mission formalizes Lemma 2 for the paper's own subproblem.

Setting

Fix a period nnn and a solution (xˉ,zˉ)(\bar{\mathbf x},\bar{\mathbf z})(xˉ,zˉ) of the master problem. The data are:

  • a finite set Rn\mathcal R^nRn of feasible routes, a finite set In\mathcal I^nIn of current customer locations, and the future periods n′∈{n+1,…,N}n'\in\{n+1,\dots,N\}n′∈{n+1,…,N};
  • a route cost crc_rcr​ for every r∈Rnr\in\mathcal R^nr∈Rn;
  • coverage coefficients αi,r∈{0,1}\alpha_{i,r}\in\{0,1\}αi,r​∈{0,1} (route rrr serves location iii) and occupancy coefficients βn′,r∈{0,1}\beta_{n',r}\in\{0,1\}βn′,r​∈{0,1} (route rrr is still running in period n′n'n′);
  • the fleet size Kˉ\bar KKˉ, the number ζnn\zeta^n_nζnn​ of drivers still en route in period nnn, and master values zˉkn′∈{0,1}\bar z^{n'}_k\in\{0,1\}zˉkn′​∈{0,1} with ∑k=0Kˉzˉkn′=1\sum_{k=0}^{\bar K}\bar z^{n'}_k=1∑k=0Kˉ​zˉkn′​=1.

Write bn′=∑k=0Kˉk zˉkn′b_{n'}=\sum_{k=0}^{\bar K}k\,\bar z^{n'}_kbn′​=∑k=0Kˉ​kzˉkn′​, the number of drivers the master solution allows to remain occupied in period n′n'n′.

The routing subproblem SF1 (pp. 16–17) is

min⁡ ∑r∈Rnθrcrs.t.∑rβn′,rθr≤bn′ ∀n′,∑rθr≤Kˉ−ζnn,∑rαi,rθr=1 ∀i∈In,θr∈{0,1}.\min\ \sum_{r\in\mathcal R^n}\theta_r c_r\quad\text{s.t.}\quad \sum_r\beta_{n',r}\theta_r\le b_{n'}\ \forall n',\qquad \sum_r\theta_r\le\bar K-\zeta^n_n,\qquad \sum_r\alpha_{i,r}\theta_r=1\ \forall i\in\mathcal I^n,\qquad \theta_r\in\{0,1\}.min r∈Rn∑​θr​cr​s.t.r∑​βn′,r​θr​≤bn′​ ∀n′,r∑​θr​≤Kˉ−ζnn​,r∑​αi,r​θr​=1 ∀i∈In,θr​∈{0,1}.

Its LP relaxation RF1 replaces θr∈{0,1}\theta_r\in\{0,1\}θr​∈{0,1} by θr≥0\theta_r\ge0θr​≥0 and the partitioning rows by covering rows ∑rαi,rθr≥1\sum_r\alpha_{i,r}\theta_r\ge1∑r​αi,r​θr​≥1; ϕ(RF1)\phi(\mathrm{RF1})ϕ(RF1) denotes its optimal cost. The dual variables are μn′\mu_{n'}μn′​ for the occupancy rows, μn\mu_nμn​ for the fleet row and νi\nu_iνi​ for the covering rows. The reduced cost of a route rrr with respect to a dual solution (μ,μn,ν)(\mu,\mu_n,\nu)(μ,μn​,ν) is

cˉr=cr−∑i∈Inαi,rνi−∑n′=n+1Nβn′,rμn′−μn.\bar c_r=c_r-\sum_{i\in\mathcal I^n}\alpha_{i,r}\nu_i-\sum_{n'=n+1}^N\beta_{n',r}\mu_{n'}-\mu_n .cˉr​=cr​−i∈In∑​αi,r​νi​−n′=n+1∑N​βn′,r​μn′​−μn​.

An upper bound ububub of SF1 is a number at least as large as the optimal cost of SF1.

Formalization targets

Goal: Lemma 2 (p. 19)

Let ububub be an upper bound of SF1, ϕ(RF1)\phi(\mathrm{RF1})ϕ(RF1) the optimal cost of RF1, (μ,μn,ν)(\mu,\mu_n,\nu)(μ,μn​,ν) a dual optimal solution of RF1, and r∈Rnr\in\mathcal R^nr∈Rn. If

cr−∑i∈Inαi,rνi−∑n′=n+1Nβn′,rμn′−μn>ub−ϕ(RF1),c_r-\sum_{i\in\mathcal I^n}\alpha_{i,r}\nu_i-\sum_{n'=n+1}^N\beta_{n',r}\mu_{n'}-\mu_n>ub-\phi(\mathrm{RF1}),cr​−i∈In∑​αi,r​νi​−n′=n+1∑N​βn′,r​μn′​−μn​>ub−ϕ(RF1),

then θr=0\theta_r=0θr​=0 in every optimal solution θ\thetaθ of SF1.

Milestones (Appendix G, p. 42)

For a route rrr, let Irn={i∣αi,r=0}\mathcal I^n_r=\{i\mid\alpha_{i,r}=0\}Irn​={i∣αi,r​=0}, let Rrn\mathcal R^n_rRrn​ be the routes that cover only locations in Irn\mathcal I^n_rIrn​, and let SF1r_rr​ be SF1 on Rrn\mathcal R^n_rRrn​ and Irn\mathcal I^n_rIrn​ with right-hand sides bn′−βn′,rb_{n'}-\beta_{n',r}bn′​−βn′,r​ and Kˉ−ζnn−1\bar K-\zeta^n_n-1Kˉ−ζnn​−1; RF1r_rr​ (75)–(79) is its LP relaxation, and ϕ(SF1r)∈R∪{+∞}\phi(\mathrm{SF1}_r)\in\mathbb R\cup\{+\infty\}ϕ(SF1r​)∈R∪{+∞} its optimal cost.

  1. (74) If cr+ϕ(SF1r)>ubc_r+\phi(\mathrm{SF1}_r)>ubcr​+ϕ(SF1r​)>ub, then rrr is in no optimal solution of SF1.
  2. Dual bound. For any dual feasible (μˉ,μˉn,νˉ)(\bar\mu,\bar\mu_n,\bar\nu)(μˉ​,μˉ​n​,νˉ) of RF1, the restriction (μˉ,μˉn,{νˉi}i∈Irn)(\bar\mu,\bar\mu_n,\{\bar\nu_i\}_{i\in\mathcal I^n_r})(μˉ​,μˉ​n​,{νˉi​}i∈Irn​​) is dual feasible for RF1r_rr​, and
∑n′μˉn′bn′−∑n′βn′,rμˉn′+μˉn(Kˉ−ζnn−1)+∑i∈Irnνˉi≤ϕ(SF1r).\sum_{n'}\bar\mu_{n'}b_{n'}-\sum_{n'}\beta_{n',r}\bar\mu_{n'}+\bar\mu_n(\bar K-\zeta^n_n-1)+\sum_{i\in\mathcal I^n_r}\bar\nu_i\le\phi(\mathrm{SF1}_r).n′∑​μˉ​n′​bn′​−n′∑​βn′,r​μˉ​n′​+μˉ​n​(Kˉ−ζnn​−1)+i∈Irn​∑​νˉi​≤ϕ(SF1r​).
  1. (80) ⇔ (81) For a dual optimal solution of RF1, crc_rcr​ plus the bound of milestone 2 exceeds ububub if and only if the reduced-cost condition of Lemma 2 holds.

Significance

The result. Lemma 2 turns one LP solve into an elimination test for every route at once. The route enumeration algorithm of §5.3 generates only routes whose reduced cost is at most a gap δ\deltaδ, raises δ\deltaδ until it reaches ub−ϕ(RF1)ub-\phi(\mathrm{RF1})ub−ϕ(RF1), and at that point the enumerated route set is guaranteed to contain every optimal solution of SF1. Without the lemma the algorithm would not certify optimality. The same algorithm also solves the single-period problems in the offline estimation stage of the paper's policy, so its correctness underlies all the computational results.

Formalizing it. The lemma is proved in the paper's Online Appendix (Appendix G, p. 42) in a few lines that rest on unstated conventions: the signs of the dual variables, the right-hand sides of SF1r_rr​, and the equality of ϕ(RF1)\phi(\mathrm{RF1})ϕ(RF1) with the dual objective of a dual optimal solution, which is LP strong duality. The formalization pins each of these down. The strong-duality step is available on the platform as LinearOptimization.lp_strong_duality (proved, for general-form LPs indexed by Fin), included as a reference item. No part of this mission is formalized elsewhere.

Difficulty

The elimination test (74) and the dual bound are short combinatorial and weak-duality arguments. The step that does not go through by algebra alone is (80) ⇔ (81): it requires that the dual objective of the dual optimal solution equals ϕ(RF1)\phi(\mathrm{RF1})ϕ(RF1). Treating this equality as a hypothesis would make the step trivial and is excluded; the hypothesis is only that the dual solution maximizes the dual objective over the dual feasible set. Deriving the equality means applying LP strong duality to RF1, whose data are indexed by arbitrary finite types rather than by Fin, so a transport to the platform's general-form LP is needed. A second point is the passage from SF1 to SF1r_rr​: removing a route from an optimal solution must yield a feasible point of SF1r_rr​, which depends on the reduced right-hand sides and on the partitioning rows of SF1 forcing the other routes to avoid rrr's customers.

Formalization scope

All statements live in the namespace OnDemandDelivery.RouteElim. Routes, locations and future periods are arbitrary Fintypes; route vectors are functions to ℝ; Kˉ\bar KKˉ and ζnn\zeta^n_nζnn​ are natural numbers and Kˉ−ζnn\bar K-\zeta^n_nKˉ−ζnn​ is computed in ℝ. The standing assumptions are αi,r,βn′,r∈{0,1}\alpha_{i,r},\beta_{n',r}\in\{0,1\}αi,r​,βn′,r​∈{0,1}, zˉkn′∈{0,1}\bar z^{n'}_k\in\{0,1\}zˉkn′​∈{0,1} and ∑kzˉkn′=1\sum_k\bar z^{n'}_k=1∑k​zˉkn′​=1 (pp. 15–16, (17), (21)).

Conventions committed to:

  • ϕ(RF1)\phi(\mathrm{RF1})ϕ(RF1) is the attained minimum of the cost over the RF1 feasible set (IsLeast), not an arbitrary real.
  • The RF1 dual uses the standard signs for a minimization, which the paper does not print: μn′≤0\mu_{n'}\le0μn′​≤0, μn≤0\mu_n\le0μn​≤0, νi≥0\nu_i\ge0νi​≥0. "Dual optimal" means dual feasible and maximal for the dual objective.
  • "An upper bound ububub of SF1" is the weakest reading: ububub is at least the cost of every optimal solution of SF1.
  • "rrr cannot be in any optimal solution" is θr=0\theta_r=0θr​=0 for every optimal solution of the binary program SF1, not of RF1.
  • SF1r_rr​ has the reduced right-hand sides of (76)–(77); (76)'s unbarred zkn′z^{n'}_kzkn′​ is read as zˉkn′\bar z^{n'}_kzˉkn′​. ϕ(SF1r)\phi(\mathrm{SF1}_r)ϕ(SF1r​) is an EReal, +∞+\infty+∞ when SF1r_rr​ is infeasible, and (74) is stated in EReal.
  • The double sum ∑n′∑kμˉn′kzˉkn′\sum_{n'}\sum_k\bar\mu_{n'}k\bar z^{n'}_k∑n′​∑k​μˉ​n′​kzˉkn′​ is written ∑n′μˉn′bn′\sum_{n'}\bar\mu_{n'}b_{n'}∑n′​μˉ​n′​bn′​.
  • Milestone 2 assumes only dual feasibility, a strengthening of the page's dual optimality.

Trivializing formalizations are ruled out: ϕ(RF1)\phi(\mathrm{RF1})ϕ(RF1) is not a free real, the dual solution is optimal and not merely feasible, no hypothesis states ϕ(RF1)\phi(\mathrm{RF1})ϕ(RF1) equals the dual objective, and the conclusion concerns optimal solutions of SF1.

Needed infrastructure: weak duality for the two LPs (elementary), and a bridge from RF1 to LinearOptimization.GeneralFormLP for strong duality, which is reusable for any LP stated over finite index types. Contributions of such a transport lemma are welcome.

Selected references

  • S. Liu, Z. Luo, On-Demand Delivery from Stores: Dynamic Dispatching and Routing with Random Demand, arXiv:2107.13058v3, 2022; Manufacturing & Service Operations Management, 2023. https://arxiv.org/abs/2107.13058
  • R. Baldacci, N. Christofides, A. Mingozzi, An exact algorithm for the vehicle routing problem based on the set partitioning formulation with additional cuts, Mathematical Programming 115 (2008) 351–385. https://doi.org/10.1007/s10107-007-0178-5
  • D. Bertsimas, J. N. Tsitsiklis, Introduction to Linear Optimization, Athena Scientific, 1997 (Theorem 4.4, strong duality).
6 thms3 active usersReviewed
🏆Completed
Operations ResearchOptimization·Captain: mikedeng1

K-Adaptability in Two-Stage Robust Binary Programming 2: The K-Adaptability Problem with Objective Uncertainty Is Equivalent to the MILP (5)Research Paper

Motivation

Many planning problems are decided in two stages. A here-and-now decision xxx is taken before an uncertain parameter vector ξ\xiξ is observed, and a wait-and-see decision yyy is taken afterwards. When the wait-and-see decision is binary (which route to drive, which facility to open, which machine to schedule), the resulting two-stage robust binary program has no known tractable reformulation, even when ξ\xiξ ranges over a polyhedron.

Bertsimas and Caramanis (2010) proposed the K-adaptability approximation. Here KKK candidate second-stage policies are fixed here and now, together with xxx, and the best of them is selected once ξ\xiξ is known. Operationally this matches settings where a few contingency plans are preferred to an arbitrary re-optimization, such as production scheduling or emergency response. Bertsimas and Caramanis gave a bilinear reformulation for K=2K = 2K=2. Hanasusanto, Kuhn and Wiesemann (2015) proved that, when ξ\xiξ enters only the objective, the K-adaptability problem for any KKK is equivalent to a single mixed-integer linear program (MILP) whose size is polynomial in the input data. That equivalence, Theorem 2 of their paper, is the subject of this mission.

Setting

The data are matrices C∈RQ×NC\in\mathbb R^{Q\times N}C∈RQ×N, Q∈RQ×MQ\in\mathbb R^{Q\times M}Q∈RQ×M, T∈RL×NT\in\mathbb R^{L\times N}T∈RL×N, W∈RL×MW\in\mathbb R^{L\times M}W∈RL×M, a vector h∈RLh\in\mathbb R^Lh∈RL, and an uncertainty set

Ξ={ξ∈RQ:Aξ≤b},A∈RR×Q, b∈RR,\Xi=\{\xi\in\mathbb R^Q : A\xi\le b\},\qquad A\in\mathbb R^{R\times Q},\ b\in\mathbb R^R,Ξ={ξ∈RQ:Aξ≤b},A∈RR×Q, b∈RR,

assumed nonempty and bounded. Inequalities between vectors are componentwise. The letter QQQ names both the number of uncertain parameters and a matrix. The first-stage feasible set is X⊆R+N\mathcal X\subseteq\mathbb R^N_+X⊆R+N​, and the second-stage feasible set is Y⊆{0,1}M\mathcal Y\subseteq\{0,1\}^MY⊆{0,1}M.

Fix K≥1K\ge1K≥1 and write K={1,…,K}\mathcal K=\{1,\dots,K\}K={1,…,K}. The K-adaptability problem with objective uncertainty is

POK:minimize  max⁡ξ∈Ξ[ξ⊤Cx+min⁡k∈Kξ⊤Qyk]subject tox∈X, yk∈Y, Tx+Wyk≤h  ∀k∈K.\mathcal{PO}_K:\quad \text{minimize}\ \ \max_{\xi\in\Xi}\Big[\xi^\top Cx+\min_{k\in\mathcal K}\xi^\top Qy^k\Big]\quad\text{subject to}\quad x\in\mathcal X,\ y^k\in\mathcal Y,\ Tx+Wy^k\le h\ \ \forall k\in\mathcal K.POK​:minimize  ξ∈Ξmax​[ξ⊤Cx+k∈Kmin​ξ⊤Qyk]subject tox∈X, yk∈Y, Tx+Wyk≤h  ∀k∈K.

The decision consists of xxx and the KKK policies y1,…,yKy^1,\dots,y^Ky1,…,yK. The adversary chooses ξ∈Ξ\xi\in\Xiξ∈Ξ, and the decision maker then uses the cheapest policy. Following the paper, max and min denote sup and inf when they are not attained.

With eee the all-ones vector, the MILP (5) is

minimizeb⊤αsubject tox∈X, yk∈Y, zk∈R+M, k∈K, α∈R+R, β∈R+K,A⊤α=Cx+∑k∈KQzk,e⊤β=1,Tx+Wyk≤h,  zk≤yk,  zk≤βke,  zk≥(βk−1)e+yk∀k∈K.\begin{aligned}\text{minimize}\quad & b^\top\alpha\\ \text{subject to}\quad & x\in\mathcal X,\ y^k\in\mathcal Y,\ z^k\in\mathbb R^M_+,\ k\in\mathcal K,\ \alpha\in\mathbb R^R_+,\ \beta\in\mathbb R^K_+,\\ & A^\top\alpha=Cx+\sum_{k\in\mathcal K}Qz^k,\quad e^\top\beta=1,\\ & Tx+Wy^k\le h,\ \ z^k\le y^k,\ \ z^k\le\beta_ke,\ \ z^k\ge(\beta_k-1)e+y^k\quad\forall k\in\mathcal K.\end{aligned}minimizesubject to​b⊤αx∈X, yk∈Y, zk∈R+M​, k∈K, α∈R+R​, β∈R+K​,A⊤α=Cx+k∈K∑​Qzk,e⊤β=1,Tx+Wyk≤h,  zk≤yk,  zk≤βk​e,  zk≥(βk​−1)e+yk∀k∈K.​

Formalization targets

Goal: Theorem 2

Problem POK\mathcal{PO}_KPOK​ is equivalent to the MILP (5). Formally:

  1. (x,y1,…,yK)(x,y^1,\dots,y^K)(x,y1,…,yK) is feasible in POK\mathcal{PO}_KPOK​ if and only if some (z,α,β)(z,\alpha,\beta)(z,α,β) completes it to a feasible point of (5);
  2. for every feasible (x,y1,…,yK)(x,y^1,\dots,y^K)(x,y1,…,yK),
max⁡ξ∈Ξ[ξ⊤Cx+min⁡kξ⊤Qyk]=min⁡{b⊤α:(x,y,z,α,β) feasible in (5)},\max_{\xi\in\Xi}\Big[\xi^\top Cx+\min_{k}\xi^\top Qy^k\Big]=\min\big\{b^\top\alpha : (x,y,z,\alpha,\beta)\ \text{feasible in (5)}\big\},ξ∈Ξmax​[ξ⊤Cx+kmin​ξ⊤Qyk]=min{b⊤α:(x,y,z,α,β) feasible in (5)},

and the minimum on the right is attained; 3. the optimal values of POK\mathcal{PO}_KPOK​ and (5) coincide.

Parts 1 and 2 are the per-decision correspondence; part 3 is the equality of optimal values. Together they are what "equivalent" means on p. 13.

Intermediate statements (milestones)

  • Epigraph LP. For fixed xxx and yky^kyk, the objective of POK\mathcal{PO}_KPOK​ equals
max⁡{ξ⊤Cx+τ:Aξ≤b, τ≤ξ⊤Qyk ∀k}.\max\{\xi^\top Cx+\tau : A\xi\le b,\ \tau\le\xi^\top Qy^k\ \forall k\}.max{ξ⊤Cx+τ:Aξ≤b, τ≤ξ⊤Qyk ∀k}.
  • Dual LP. That value equals
min⁡{b⊤α:α≥0, β≥0, A⊤α=Cx+∑kβkQyk, e⊤β=1},\min\Big\{b^\top\alpha : \alpha\ge0,\ \beta\ge0,\ A^\top\alpha=Cx+\sum_k\beta_kQy^k,\ e^\top\beta=1\Big\},min{b⊤α:α≥0, β≥0, A⊤α=Cx+k∑​βk​Qyk, e⊤β=1},

and the minimum is attained.

  • Exact linearization. For y∈{0,1}My\in\{0,1\}^My∈{0,1}M, 0≤β≤10\le\beta\le10≤β≤1 and z≥0z\ge0z≥0,
z=βy  ⟺  z≤y, z≤βe, z≥(β−1)e+y.z=\beta y\iff z\le y,\ z\le\beta e,\ z\ge(\beta-1)e+y.z=βy⟺z≤y, z≤βe, z≥(β−1)e+y.

Significance

The result. Theorem 2 makes the K-adaptability problem with objective uncertainty solvable by off-the-shelf MILP solvers, and the size of (5) grows polynomially in the size of the input data (p. 13). Together with Theorem 1 of the same paper, which shows that min⁡{dim⁡Y,rk⁡Q}+1\min\{\dim\mathcal Y,\operatorname{rk}Q\}+1min{dimY,rkQ}+1 policies already recover the full two-stage problem, it yields an exact finite MILP formulation of two-stage robust binary programs with objective uncertainty. The paper's numerical experiments on supply chain design and route planning (§4.1–4.2), which are instances of POK\mathcal{PO}_KPOK​, use this formulation.

Formalizing it. The theorem is proved in the paper, in a short argument in its electronic companion. No machine-checked version exists. A formal proof certifies the exact form of (5), including every sign and every inequality of the linearization, against the original min–max–min problem. It also exercises LP strong duality in a form with attainment, for a polyhedron given by Aξ≤bA\xi\le bAξ≤b with free variables. The same pattern (dualize an inner LP, then linearize products of a continuous and a binary variable) recurs across robust and bilevel optimization, so a reusable development of it is valuable beyond this paper.

Difficulty

The obvious route is to swap the max over ξ\xiξ with the min over policies, and that fails. The function ξ↦min⁡kξ⊤Qyk\xi\mapsto\min_k\xi^\top Qy^kξ↦mink​ξ⊤Qyk is concave, not linear, so POK\mathcal{PO}_KPOK​ is not a min–max of a bilinear function in the policies. The inner maximization must be kept as an optimization problem and replaced by an equivalent minimization, which needs LP strong duality with attainment of the dual optimum. Its preconditions (primal feasibility from Ξ≠∅\Xi\ne\emptysetΞ=∅, boundedness from Ξ\XiΞ bounded and K≥1K\ge1K≥1) have to be discharged. The dualized problem contains products βkyk\beta_ky^kβk​yk of continuous and binary variables. Replacing them by linear constraints is exact only because yky^kyk is binary and 0≤βk≤10\le\beta_k\le10≤βk​≤1; with fractional yky^kyk the three inequalities do not force zk=βkykz^k=\beta_ky^kzk=βk​yk (Remark 4, p. 14). The remaining effort is bookkeeping in extended reals: optimal values are infima and suprema, and the correspondence of feasible points must be exact in both directions.

Formalization scope

Vectors are Fin n → ℝ, matrices are Matrix (Fin a) (Fin b) ℝ, products are Matrix.mulVec and dotProduct, and ≤ on vectors is componentwise. The paper's dimension QQQ is nQ. The data, with the standing assumptions of §1 (Ξ nonempty and bounded, X⊆R+N\mathcal X\subseteq\mathbb R^N_+X⊆R+N​, points of Y\mathcal YY binary), form one structure KAdaptability.ObjMILP.Problem. X\mathcal XX is an arbitrary set of nonnegative vectors, and Y\mathcal YY an arbitrary set of 0/1 vectors. Policies are indexed by Fin K with the hypothesis 0 < K. The all-ones vector eee is 1.

Every objective and optimal value is an EReal. The objective of POK\mathcal{PO}_KPOK​ is a supremum over Ξ\XiΞ of ξ⊤Cx\xi^\top Cxξ⊤Cx plus an infimum over kkk. The optimal values of POK\mathcal{PO}_KPOK​, of (5) and of the dual LP are infima, equal to +∞+\infty+∞ when infeasible. The epigraph LP's value is a supremum. Part 3 of the goal is an equality of infima, since the optimum over xxx need not be attained when X\mathcal XX is not closed.

The goal does not assert merely that the two optimal values coincide, nor an inequality. It states the per-decision identity with attainment and the correspondence of feasible decisions. The program (5) appears with exactly its constraints, including Tx+Wyk≤hTx+Wy^k\le hTx+Wyk≤h, so it cannot be replaced by some other MILP.

A complete development needs LP strong duality with dual attainment for max⁡{c⊤ξ+τ:Aξ≤b, τ≤dk⊤ξ}\max\{c^\top\xi+\tau : A\xi\le b,\ \tau\le d_k^\top\xi\}max{c⊤ξ+τ:Aξ≤b, τ≤dk⊤​ξ}. The platform's LinearOptimization.lp_strong_duality and SmaleNinth.lp_strong_duality are possible tools, in their own encodings. It also needs elementary facts about EReal suprema and infima. Contributions of a general "LP value equals dual LP value" lemma in the Matrix.mulVec encoding are welcome and reusable.

Selected references

  • G. A. Hanasusanto, D. Kuhn, W. Wiesemann, K-Adaptability in Two-Stage Robust Binary Programming, Operations Research 63(4), 877–891, 2015. https://doi.org/10.1287/opre.2015.1392 (version read: preprint, Optimization Online 2014/03/4294, revision of 2015-03-30, https://optimization-online.org/2014/03/4294/)
  • D. Bertsimas, C. Caramanis, Finite Adaptability in Multistage Linear Optimization, IEEE Transactions on Automatic Control 55(12), 2751–2766, 2010. https://doi.org/10.1109/TAC.2010.2049764
  • A. Ben-Tal, A. Goryashko, E. Guslitzer, A. Nemirovski, Adjustable Robust Solutions of Uncertain Linear Programs, Mathematical Programming 99(2), 351–376, 2004. https://doi.org/10.1007/s10107-003-0454-y
  • D. Bertsimas, D. B. Brown, C. Caramanis, Theory and Applications of Robust Optimization, SIAM Review 53(3), 464–501, 2011. https://doi.org/10.1137/080734510
8 thms3 active usersReviewed
CombinatoricsDiscrete GeometryOperations Research·Captain: mikedeng1

An Improved Kalai-Kleitman Bound for the Diameter of a Polyhedron: Every d-Polyhedron with n Facets, Bounded or Not, Has Graph Diameter at Most (n-d)^(log2 d)Research Paper

Motivation

The simplex method for linear programming moves from vertex to vertex along the edges of a polyhedron. A lower bound on the number of pivots of any edge-following method is the graph diameter of the feasible polyhedron: the largest number of edges needed to connect two of its vertices. Whether this diameter is bounded by a polynomial in the dimension ddd and the number nnn of facets is the polynomial Hirsch conjecture, open since the 1950s. The original Hirsch conjecture, n−dn-dn−d, was refuted for unbounded polyhedra by Klee and Walkup (1967) and for bounded ones by Santos (2012) (doi:10.4007/annals.2012.176.1.7).

Timeline.

  • 1964–1966. Klee proves that a three-dimensional polyhedron with nnn facets, bounded or not, has diameter at most n−3n-3n−3 (doi:10.4153/CJM-1964-058-3, doi:10.1137/0113061, doi:10.2140/pjm.1966.17.249).
  • 1967. Klee and Walkup show that the Hirsch bound fails for unbounded polyhedra in dimension four (doi:10.1007/BF02395039).
  • 1992. Kalai and Kleitman prove the first subexponential bound, nlog⁡2d+2n^{\log_2 d+2}nlog2​d+2, for all polyhedra (doi:10.1090/S0273-0979-1992-00285-9, arXiv:math/9204233).
  • 2010. Eisenbrand, Hähnle, Razborov and Rothvoss show that nlog⁡2d+1n^{\log_2 d+1}nlog2​d+1 holds for a broad combinatorial abstraction of polyhedra (doi:10.1287/moor.1100.0470).
  • 2014. Todd sharpens the Kalai–Kleitman bound to (n−d)log⁡2d(n-d)^{\log_2 d}(n−d)log2​d for all polyhedra, bounded or not (arXiv:1402.3579; SIAM J. Discrete Math. 28 (2014), doi:10.1137/140962310). This is the result of this mission.

Setting

A polyhedron P⊆RdP\subseteq\mathbb R^dP⊆Rd is the set of solutions of finitely many linear inequalities,

P=Hpoly⁡(a,b)={x∈Rd: ⟨ai,x⟩≤bi, i=1,…,n}.P=\operatorname{Hpoly}(a,b)=\{x\in\mathbb R^d:\ \langle a_i,x\rangle\le b_i,\ i=1,\dots,n\}.P=Hpoly(a,b)={x∈Rd: ⟨ai​,x⟩≤bi​, i=1,…,n}.

It need not be bounded. A vertex of PPP is a point vvv for which some half-space HHH satisfies P∩H={v}P\cap H=\{v\}P∩H={v}, equivalently an extreme point of PPP. Two vertices v≠wv\neq wv=w are adjacent when P∩H=[v,w]P\cap H=[v,w]P∩H=[v,w] for some half-space HHH; the segment [v,w][v,w][v,w] is then an edge. A path of length kkk is a sequence of vertices v=v0,…,vk=wv=v_0,\dots,v_k=wv=v0​,…,vk​=w with consecutive vertices adjacent, ρP(v,w)\rho_P(v,w)ρP​(v,w) is the least such kkk, and the diameter is δ(P)=max⁡ρP(v,w)\delta(P)=\max\rho_P(v,w)δ(P)=maxρP​(v,w) over pairs of vertices. A (d,n)(d,n)(d,n)-polyhedron is a ddd-dimensional polyhedron with nnn facets, and

Δ(d,n)=max⁡{δ(P): P a (d,n)-polyhedron},\Delta(d,n)=\max\{\delta(P):\ P \text{ a } (d,n)\text{-polyhedron}\},Δ(d,n)=max{δ(P): P a (d,n)-polyhedron},

the maximum taken over bounded and unbounded polyhedra alike. All logarithms are to base 2.

Formalization targets

Goal: Theorem 1

Δ(d,n)≤(n−d)log⁡d(1≤d≤n),Δ(1,1)=0.\Delta(d,n)\le (n-d)^{\log d}\quad(1\le d\le n),\qquad \Delta(1,1)=0.Δ(d,n)≤(n−d)logd(1≤d≤n),Δ(1,1)=0.

The goal fixes no constant beyond the paper's own exponent and leaves no case of ddd, nnn or boundedness open.

Milestones, in the order the proof uses them

  1. The identity (n−d)log⁡d=dlog⁡(n−d)(n-d)^{\log d}=d^{\log(n-d)}(n−d)logd=dlog(n−d) (p. 2).
  2. Two vertices on a common facet are at distance at most Δ(d−1,n−1)\Delta(d-1,n-1)Δ(d−1,n−1) (p. 2, in the proof of Lemma 1).
  3. Lemma 1 (Kalai–Kleitman): for 2≤d≤⌊n/2⌋2\le d\le\lfloor n/2\rfloor2≤d≤⌊n/2⌋, Δ(d,n)≤Δ(d−1,n−1)+2Δ(d,⌊n/2⌋)+2\Delta(d,n)\le\Delta(d-1,n-1)+2\Delta(d,\lfloor n/2\rfloor)+2Δ(d,n)≤Δ(d−1,n−1)+2Δ(d,⌊n/2⌋)+2.
  4. Δ(d,d)=0\Delta(d,d)=0Δ(d,d)=0.
  5. Δ(1,n)≤1\Delta(1,n)\le 1Δ(1,n)≤1.
  6. Δ(2,n)≤n−2\Delta(2,n)\le n-2Δ(2,n)≤n−2.
  7. Δ(3,n)≤n−3\Delta(3,n)\le n-3Δ(3,n)≤n−3 (Klee), for unbounded polyhedra too.
  8. If n<2dn<2dn<2d, then Δ(d,n)≤Δ(d−1,n−1)\Delta(d,n)\le\Delta(d-1,n-1)Δ(d,n)≤Δ(d−1,n−1).
  9. In the ten cases d=4, 8≤n≤11d=4,\ 8\le n\le 11d=4, 8≤n≤11; d=5, 10≤n≤12d=5,\ 10\le n\le 12d=5, 10≤n≤12; d=6, 12≤n≤13d=6,\ 12\le n\le 13d=6, 12≤n≤13; d=7, n=14d=7,\ n=14d=7, n=14, the bound ⌊(n−d)log⁡d⌋\lfloor (n-d)^{\log d}\rfloor⌊(n−d)logd⌋ satisfies the recursion of Lemma 1.
  10. For d≥4d\ge4d≥4, n−d≥8n-d\ge8n−d≥8, n≥2dn\ge 2dn≥2d: (d−1)log⁡(n−d)+2(n/2−d)log⁡d+2≤dlog⁡(n−d)(d-1)^{\log(n-d)}+2(n/2-d)^{\log d}+2\le d^{\log(n-d)}(d−1)log(n−d)+2(n/2−d)logd+2≤dlog(n−d). For n>2dn>2dn>2d this is the displayed inequality; at n=2dn=2dn=2d the second power is zero, consistent with Δ(d,d)=0\Delta(d,d)=0Δ(d,d)=0.

Significance

The result. The bound (n−d)log⁡d(n-d)^{\log d}(n−d)logd is never larger than the Kalai–Kleitman bound. It equals the true value n−dn-dn−d in dimensions 1 and 2, and it depends on nnn only through the Hirsch quantity n−dn-dn−d. Since (n−d)log⁡d=dlog⁡(n−d)(n-d)^{\log d}=d^{\log(n-d)}(n−d)logd=dlog(n−d), it is also symmetric under linear-programming duality, which exchanges the roles of ddd and n−dn-dn−d. For fixed n−dn-dn−d it is polynomial in ddd. It applies to the unbounded feasible regions that arise in practice in linear programming.

Formalizing it. The theorem has been proved since 2014. The platform has a proof of the bounded case only: Hirsch.todd_bound assumes d≥3d\ge3d≥3, a nonempty interior and Bornology.IsBounded. This mission's goal extends it to all polyhedra, bounded or not, and to d=1,2d=1,2d=1,2. That is the statement the paper proves, and the one an analysis of the simplex method on unbounded feasible regions needs. The unbounded form of Lemma 1 and the common-facet bound are open on the platform as items of the Kalai–Kleitman series and are reused here. Klee's unbounded three-dimensional bound has no formal proof on the platform: the published Hirsch.klee_three_dimensional_bound and Hirsch.dimension_three_bound cover bounded polyhedra only. The bounded recursion Hirsch.kalai_kleitman_recursion and the bounded facet reduction Hirsch.facet_reduction are proved and may serve as models.

Difficulty

Two steps are hard. The first is the unbounded case. Statements proved for polytopes do not transfer directly, because an unbounded polyhedron may have no vertices or a single one, unbounded edges (rays) that are not edges of the graph, and redundant or implicit-equality rows. Each geometric milestone must hold for these. The second is Klee's three-dimensional bound, which the paper cites without proof. Assuming boundedness to apply the published polytope versions is ruled out: Klee and Walkup showed that unbounded and bounded diameters differ from dimension four on, so the unbounded case carries content of its own. The arithmetic milestones are elementary but sensitive to edge values of log⁡\loglog and the real power.

Formalization scope

All statements use the published definitions Hirsch.Hpoly, Hirsch.Adj and Hirsch.DiamLE (Hirsch_model). Vertices are Set.extremePoints ℝ. DiamLE P B says that any two extreme points are joined by a walk of exactly BBB steps, each step staying put or crossing an edge. So the bound includes connectivity of the vertex–edge graph.

  • Inequality reading. "Δ(d,n)≤B\Delta(d,n)\le BΔ(d,n)≤B" is stated as: for every a:Fin n→Rda:\mathrm{Fin}\,n\to\mathbb R^da:Finn→Rd and bbb, DiamLE (Hpoly a b) B. This ranges over all polyhedra given by nnn inequalities in Rd\mathbb R^dRd, including lower-dimensional, empty and redundant ones. It is equivalent to the paper's statement: a lower-dimensional polyhedron of dimension d′d'd′ with n′n'n′ facets has n′−d′≤n−dn'-d'\le n-dn′−d′≤n−d.
  • No boundedness, nonemptiness or full-dimensionality is assumed anywhere.
  • The logarithm is Real.logb 2 and the power is Real.rpow. A distance is at most (n−d)log⁡d(n-d)^{\log d}(n−d)logd exactly when it is at most the floor ⌊(n−d)log⁡d⌋\lfloor (n-d)^{\log d}\rfloor⌊(n−d)logd⌋, abbreviated toddBound d n in the milestone on the ten cases. The clause Δ(1,1)=0\Delta(1,1)=0Δ(1,1)=0 is a separate conjunct, because 00=10^0=100=1 in Real.rpow.
  • Disclosed additions: nonzero rows in the n<2dn<2dn<2d step (as in the referenced Lemma 1 and the d=2d=2d=2 bound). In the general step, n≥2dn\ge2dn≥2d: the paper secures it by treating n<2dn<2dn<2d first. The second power is written with base n/2−dn/2-dn/2−d; it equals the printed expression when n>2dn>2dn>2d and takes the intended value zero when n=2dn=2dn=2d, where the printed logarithm is undefined.
  • Natural-number subtraction appears in n−3n-3n−3 and n−2n-2n−2 (both harmless, since for small nnn there are no vertices) and in Rd−1\mathbb R^{d-1}Rd−1 for facets.

Trivializing encodings are excluded: no Bornology.IsBounded hypothesis; no definition of Δ\DeltaΔ by sSup over N\mathbb NN, which would be 000 on unbounded families; no use of Hirsch.gdist, which is 000 when no walk exists; and no dropping of the Δ(1,1)=0\Delta(1,1)=0Δ(1,1)=0 clause.

Contributions are welcome on every milestone. A complete development needs facial structure of unbounded polyhedra (facets as polyhedra one dimension lower, tight rows at vertices), the relaxation argument of Lemma 1, Klee's three-dimensional argument, and real-power and logarithm estimates. The facet machinery and Klee's bound are reusable for any diameter result on unbounded polyhedra.

Selected references

  • M. J. Todd, An improved Kalai–Kleitman bound for the diameter of a polyhedron, SIAM J. Discrete Math. 28 (2014); arXiv:1402.3579v2. arXiv:1402.3579, doi:10.1137/140962310
  • G. Kalai and D. J. Kleitman, A quasi-polynomial bound for the diameter of graphs of polyhedra, Bull. Amer. Math. Soc. 26 (1992), 315–316. doi:10.1090/S0273-0979-1992-00285-9
  • V. Klee, Diameters of polyhedral graphs, Canad. J. Math. 16 (1964), 602–614. doi:10.4153/CJM-1964-058-3
  • V. Klee, Paths on polyhedra I, J. SIAM 13 (1965), 946–956. doi:10.1137/0113061
  • V. Klee, Paths on polyhedra II, Pacific J. Math. 17 (1966), 249–262. doi:10.2140/pjm.1966.17.249
  • V. Klee and D. W. Walkup, The d-step conjecture for polyhedra of dimension d < 6, Acta Math. 117 (1967), 53–78. doi:10.1007/BF02395039
  • F. Eisenbrand, N. Hähnle, A. Razborov, T. Rothvoss, Diameter of polyhedra: limits of abstraction, Math. Oper. Res. 35 (2010), 786–794. doi:10.1287/moor.1100.0470
  • F. Santos, A counterexample to the Hirsch conjecture, Ann. of Math. 176 (2012), 383–412. doi:10.4007/annals.2012.176.1.7
13 thms3 active usersReviewed
Operations ResearchOptimization·Captain: mikedeng1

An O(√nL)-Iteration Homogeneous and Self-Dual Linear Programming Algorithm 2: τᵏ Stays Bounded Below If (LP) Has an Optimum, and Otherwise κᵏ Stays Bounded Below and τᵏ/θᵏ Is BoundedResearch Paper

Motivation

Interior-point methods for linear programming are usually analysed under two assumptions: a strictly feasible starting point is known, and the problem has an optimal solution. In practice neither is known in advance. The homogeneous self-dual embedding of Ye, Todd and Mizuno (Math. Oper. Res. 19(1), 1994) removes both assumptions. It embeds a standard-form linear program and its dual into one artificial program that always has an obvious interior starting point. A solution of the artificial program either yields optimal solutions of the original pair or certifies that one of them is infeasible. The embedding underlies the infeasibility detection of several interior-point LP and conic solvers.

The paper's main result is an O(nL)O(\sqrt n L)O(n​L) iteration bound. Section 4, "Analysis of the output", asks a different, practical question: what can be read off the iterates if the algorithm is stopped early, at an approximate solution? Theorem 8 answers it. This mission formalizes Theorem 8 and the results its proof uses.

Setting

Let A∈Rm×nA\in\mathbb R^{m\times n}A∈Rm×n, b∈Rmb\in\mathbb R^mb∈Rm, c∈Rnc\in\mathbb R^nc∈Rn. The primal and dual programs are

(LP) min⁡{cTx:Ax=b, x≥0},(LD) max⁡{bTy:ATy≤c},\text{(LP)}\ \min\{c^Tx : Ax=b,\ x\ge0\},\qquad \text{(LD)}\ \max\{b^Ty : A^Ty\le c\},(LP) min{cTx:Ax=b, x≥0},(LD) max{bTy:ATy≤c},

with dual slack s=c−ATys=c-A^Tys=c−ATy. An optimal solution pair is an optimal xˉ\bar xxˉ of (LP) together with an optimal (yˉ,sˉ)(\bar y,\bar s)(yˉ​,sˉ) of (LD).

Write eee for the all-ones vector and set bˉ=b−Ae\bar b=b-Aebˉ=b−Ae, cˉ=c−e\bar c=c-ecˉ=c−e, zˉ=cTe+1\bar z=c^Te+1zˉ=cTe+1. The homogeneous self-dual program (HLP), in the paper's normalization y0=0y^0=0y0=0, x0=s0=ex^0=s^0=ex0=s0=e, has variables (y,x,τ,θ,s,κ)(y,x,\tau,\theta,s,\kappa)(y,x,τ,θ,s,κ) with y∈Rmy\in\mathbb R^my∈Rm, x,s∈Rnx,s\in\mathbb R^nx,s∈Rn, τ,θ,κ∈R\tau,\theta,\kappa\in\mathbb Rτ,θ,κ∈R:

min⁡(n+1)θs.t.Ax−bτ+bˉθ=0,s=−ATy+cτ−cˉθ ≥0,κ=bTy−cTx+zˉθ ≥0,−bˉTy+cˉTx−zˉτ=−(n+1),x≥0, τ≥0.\begin{array}{rl} \min & (n+1)\theta\\ \text{s.t.} & Ax-b\tau+\bar b\theta=0,\\ & s=-A^Ty+c\tau-\bar c\theta\ \ge0,\\ & \kappa=b^Ty-c^Tx+\bar z\theta\ \ge0,\\ & -\bar b^Ty+\bar c^Tx-\bar z\tau=-(n+1),\qquad x\ge0,\ \tau\ge0. \end{array}mins.t.​(n+1)θAx−bτ+bˉθ=0,s=−ATy+cτ−cˉθ ≥0,κ=bTy−cTx+zˉθ ≥0,−bˉTy+cˉTx−zˉτ=−(n+1),x≥0, τ≥0.​

Fh\mathcal F_hFh​ is its feasible set and Fh0\mathcal F_h^0Fh0​ the subset with x,τ,s,κ>0x,\tau,s,\kappa>0x,τ,s,κ>0. A self-complementary solution is an optimal solution of (HLP). A strictly self-complementary one also has θ=0\theta=0θ=0 and (x+s;τ+κ)>0(x+s;\tau+\kappa)>0(x+s;τ+κ)>0. For a point of (HLP) put μ=(xTs+τκ)/(n+1)\mu=(x^Ts+\tau\kappa)/(n+1)μ=(xTs+τκ)/(n+1), and for β∈(0,1)\beta\in(0,1)β∈(0,1) let

N(β)={z∈Fh0:∥(Xsτκ)−μe∥2≤βμ}.\mathcal N(\beta)=\Bigl\{z\in\mathcal F_h^0 : \Bigl\|\begin{pmatrix}Xs\\ \tau\kappa\end{pmatrix}-\mu e\Bigr\|_2\le\beta\mu\Bigr\}.N(β)={z∈Fh0​:​(Xsτκ​)−μe​2​≤βμ}.

The predictor–corrector algorithm with β=1/4\beta=1/4β=1/4 starts at z0=(0,e,1,1,e,1)z^0=(0,e,1,1,e,1)z0=(0,e,1,1,e,1). At even kkk (predictor step) it solves the Newton system (11)–(12) with target γμk=0\gamma\mu^k=0γμk=0 and moves along the direction by the largest step that stays in N(2β)\mathcal N(2\beta)N(2β) (13). At odd kkk (corrector step) it solves the system with target μk\mu^kμk and takes a full step.

Formalization targets

Goal: Theorem 8 (p. 63)

For every predictor–corrector sequence (yk,xk,τk,θk,sk,κk)(y^k,x^k,\tau^k,\theta^k,s^k,\kappa^k)(yk,xk,τk,θk,sk,κk):

  1. if (LP) possesses an optimal solution, then for every optimal pair xˉ,(yˉ,sˉ)\bar x,(\bar y,\bar s)xˉ,(yˉ​,sˉ)
τk≥1−2βeTxˉ+eTsˉ+1for all k;\tau^k\ge\frac{1-2\beta}{e^T\bar x+e^T\bar s+1}\quad\text{for all }k;τk≥eTxˉ+eTsˉ+11−2β​for all k;
  1. otherwise there is a fixed ε>0\varepsilon>0ε>0, independent of kkk, with
κk≥(1−2β)εn+1,1−2β2(n+1)≤1−2βκk≤τkθk≤1+2βκk≤(n+1)(1+2β)(1−2β)ε\kappa^k\ge\frac{(1-2\beta)\varepsilon}{n+1},\qquad \frac{1-2\beta}{2(n+1)}\le\frac{1-2\beta}{\kappa^k}\le\frac{\tau^k}{\theta^k}\le\frac{1+2\beta}{\kappa^k}\le\frac{(n+1)(1+2\beta)}{(1-2\beta)\varepsilon}κk≥n+1(1−2β)ε​,2(n+1)1−2β​≤κk1−2β​≤θkτk​≤κk1+2β​≤(1−2β)ε(n+1)(1+2β)​

for all kkk.

Milestones

In the order the proof uses them:

  • the predictor-step and corrector-step claims from the proof of Theorem 6 (p. 62);
  • the iterates lie in N(2β)\mathcal N(2\beta)N(2β) (p. 63);
  • Theorem 2 (iv): the optimal value of (HLP) is 000 and (n+1)θ=xTs+τκ(n+1)\theta=x^Ts+\tau\kappa(n+1)θ=xTs+τκ on Fh\mathcal F_hFh​ (stated for general x0,s0>0x^0,s^0>0x0,s0>0, y0y^0y0);
  • equation (9): eTx+eTs+τ+κ−(n+1)θ=n+1e^Tx+e^Ts+\tau+\kappa-(n+1)\theta=n+1eTx+eTs+τ+κ−(n+1)θ=n+1;
  • an optimal pair scaled by α=(n+1)/(eTxˉ+eTsˉ+1)\alpha=(n+1)/(e^T\bar x+e^T\bar s+1)α=(n+1)/(eTxˉ+eTsˉ+1) is a self-complementary solution;
  • the orthogonality identity (xk)Ts∗+(sk)Tx∗+κkτ∗=(n+1)θk(x^k)^Ts^*+(s^k)^Tx^*+\kappa^k\tau^*=(n+1)\theta^k(xk)Ts∗+(sk)Tx∗+κkτ∗=(n+1)θk;
  • Theorem 2 (v), existence of a strictly self-complementary solution;
  • Theorem 3 (i) and (ii), reading such a solution back into (LP) and (LD);
  • the bound κk≤(n+1)+(n+1)θk≤2(n+1)\kappa^k\le(n+1)+(n+1)\theta^k\le2(n+1)κk≤(n+1)+(n+1)θk≤2(n+1).

Significance

Theorem 8 is a stopping criterion. Either τk\tau^kτk stays bounded away from zero, which happens exactly when (LP) has an optimal solution, or τk\tau^kτk and θk\theta^kθk go to zero at the same rate while κk\kappa^kκk stays bounded away from zero. An implementation can therefore watch τk/θk\tau^k/\theta^kτk/θk and κk\kappa^kκk and decide the status of (LP) from an approximate iterate, without the exact rounding procedure needed for the O(nL)O(\sqrt nL)O(n​L) bound. The bounds hold for real data, with no integrality and no bit length.

Theorem 8 has a published proof on one page, building on the self-duality facts of §2 and the Mizuno–Todd–Ye step analysis. None of it is machine-checked. A complete development would give a verified account of the homogeneous self-dual embedding, including:

  • the skew-symmetric orthogonality argument;
  • Goldman–Tucker strict complementarity for (HLP);
  • the predictor–corrector neighborhood invariants.

These are the ingredients of every analysis of the embedding. Homogeneous self-dual programs appear on Prove2Me in other conventions (Vanderbei's (22.4) and (22.21), Matoušek's (SD)), but their statements concern different programs: no artificial variable θ\thetaθ, no normalizing row. They do not formalize any statement here.

Difficulty

The two branches of Theorem 8 rest on facts of different depth.

  • The first branch needs only an optimal pair of (LP)/(LD), lifted into (HLP), and the identity (xk)Ts∗+(sk)Tx∗+κkτ∗=(n+1)θk(x^k)^Ts^*+(s^k)^Tx^*+\kappa^k\tau^*=(n+1)\theta^k(xk)Ts∗+(sk)Tx∗+κkτ∗=(n+1)θk.
  • The "otherwise" branch needs an optimal solution of (HLP) with κ∗>0\kappa^*>0κ∗>0. Its existence is not elementary. It comes from strict complementarity (Theorem 2 (v)) together with the characterization of Theorem 3 (i). Strict complementarity for linear programs is the Goldman–Tucker theorem, which Mathlib does not contain.
  • The neighborhood invariants behind "the sequence is in N(2β)\mathcal N(2\beta)N(2β)" are the Mizuno–Todd–Ye estimates. They need the orthogonality dxTds+dτdκ=0d_x^Td_s+d_\tau d_\kappa=0dxT​ds​+dτ​dκ​=0 of the Newton directions and a careful quadratic bound on ∥Dxds∥\|D_xd_s\|∥Dx​ds​∥.

The obvious first idea is to treat Theorem 8 as a consequence of Theorem 6. It fails: Theorem 8 is about every iterate of an infinite run and uses no iteration count. The constant ε\varepsilonε depends on the instance and is not explicit.

Formalization scope

  • Data. Real data A : Matrix (Fin m) (Fin n) ℝ, b : Fin m → ℝ, c : Fin n → ℝ, with no rank assumption and no integrality. (LP)/(LD) feasibility and optimality are built on the published general-form LP definition LinearOptimization_DualLP. "(LP) has a solution (feasible and bounded)" in Theorem 3 is read as "(LP) has an optimal solution".
  • Points. A point of (HLP) is a structure with fields y x τ θ s κ. The slacks s,κs,\kappas,κ are pinned to the constraint expressions. (HLP) is defined for general (x0,y0,s0)(x^0,y^0,s^0)(x0,y0,s0); the instance (7) is obtained by substitution. Optimality means minimizing the objective over Fh\mathcal F_hFh​, not "θ=0\theta=0θ=0", so Theorem 2 (iv) is not built into the definition.
  • Norm. The norm in N(β)\mathcal N(\beta)N(β) is the Euclidean norm on Rn+1\mathbb R^{n+1}Rn+1, written out explicitly.
  • Constants. β=1/4\beta=1/4β=1/4 is a named constant. The statements show 1−2β1-2\beta1−2β and 1+2β1+2\beta1+2β as printed.
  • The iteration. It is a relation IsPCSequence: z0=(0,e,1,1,e,1)z^0=(0,e,1,1,e,1)z0=(0,e,1,1,e,1), and each step is zk+1=zk+αdz^{k+1}=z^k+\alpha dzk+1=zk+αd for some solution ddd of (11)–(12). The relation is needed because dyd_ydy​ is not unique when AAA is rank-deficient.
  • Step size. The predictor step size must be the greatest element of the set in (13), as printed. On an instance where that maximum is not attained, no predictor–corrector sequence exists and Theorem 8 holds vacuously there. A sorry-free check shows that z0∈N(1/4)z^0\in\mathcal N(1/4)z0∈N(1/4) for all data, so the sequence does start.
  • The two branches. They are two implications with complementary hypotheses, "(LP) has an optimal solution" and its negation. The negation covers infeasible and unbounded (LP) alike.
  • Quantifiers. The first branch quantifies over all optimal pairs. In the second, ∃ε>0\exists\varepsilon>0∃ε>0 precedes ∀k\forall k∀k.
  • Divisions. The chain keeps the printed divisions. All denominators (κk\kappa^kκk, θk=μk\theta^k=\mu^kθk=μk) are positive on N(2β)\mathcal N(2\beta)N(2β), so Lean's x/0=0x/0=0x/0=0 convention plays no role.
  • Ruled out. A formalization in which ε\varepsilonε may depend on kkk, or in which the first branch asserts the bound for only some optimal pair, would make Theorem 8 trivial or weaker. Both are excluded.

Needed infrastructure: LP duality and Goldman–Tucker strict complementarity for real standard-form programs, and the skew-symmetric orthogonality lemma. These are reusable well beyond this mission. Proofs of any milestone, and alternative arguments (e.g. via the central-path limit) for the existence of κ∗>0\kappa^*>0κ∗>0, are welcome.

Selected references

  • Y. Ye, M. J. Todd, S. Mizuno, An O(nL)O(\sqrt nL)O(n​L)-Iteration Homogeneous and Self-Dual Linear Programming Algorithm, Mathematics of Operations Research 19(1), 53–67, 1994. https://doi.org/10.1287/moor.19.1.53
  • S. Mizuno, M. J. Todd, Y. Ye, On Adaptive-Step Primal-Dual Interior-Point Algorithms for Linear Programming, Mathematics of Operations Research 18(4), 964–981, 1993. https://doi.org/10.1287/moor.18.4.964
  • O. Güler, Y. Ye, Convergence behavior of interior-point algorithms, Mathematical Programming 60, 215–228, 1993. https://doi.org/10.1007/BF01580610
  • A. J. Goldman, A. W. Tucker, Theory of linear programming, in Linear Inequalities and Related Systems, Annals of Mathematics Studies 38, 53–97, 1956. https://doi.org/10.1515/9781400881987-005
17 thms3 active usersReviewed
🏆Completed
Complexity TheoryOperations ResearchOptimization·Captain: mikedeng1

On Adaptive-Step Primal-Dual Interior-Point Algorithms for Linear Programming 2: The Adaptive-Step Algorithm in N∞(β) or N∞⁻(β) with γ ≤ 2(1 − β) Attains Precision t in O(nt) IterationsResearch Paper

Why wide neighbourhoods

Primal-dual interior-point methods solve a linear program by following, approximately, the central path of strictly feasible primal-dual pairs whose complementarity products xjsjx_js_jxj​sj​ are all equal. Every such method confines its iterates to a neighbourhood of that path, and the neighbourhood governs both the theory and the practice. The first primal-dual path-following algorithm, by Kojima, Mizuno and Yoshise (1989), used the one-sided neighbourhood {xjsj≥12 xTs/n}\{x_js_j \ge \tfrac12\,x^Ts/n\}{xj​sj​≥21​xTs/n} and attained precision ttt in O(nt)O(nt)O(nt) iterations. Kojima–Mizuno–Yoshise and Monteiro–Adler then switched to the narrow Euclidean neighbourhood N2(β)\mathcal N_2(\beta)N2​(β) and improved the bound to O(n t)O(\sqrt n\,t)O(n​t). Practical codes nevertheless favour wide neighbourhoods, which allow long steps.

Mizuno, Todd and Ye (Cornell ORIE Technical Report 944, 1990, revised 1991; published in Mathematics of Operations Research 18(4), 1993) analyse adaptive-step algorithms, which at every iteration take the longest step that keeps the iterate in the neighbourhood. This mission formalizes their §4: the adaptive-step algorithm in the wide neighbourhoods N∞(β)\mathcal N_\infty(\beta)N∞​(β) and N∞−(β)\mathcal N_\infty^-(\beta)N∞−​(β) attains precision ttt in O(nt)O(nt)O(nt) iterations. With β\betaβ close to 111, N∞−(β)\mathcal N_\infty^-(\beta)N∞−​(β) contains almost all of the strictly feasible set.

Setting

Fix n≥1n\ge1n≥1, m≥0m\ge0m≥0, A∈Rm×nA\in\mathbb R^{m\times n}A∈Rm×n, b∈Rmb\in\mathbb R^mb∈Rm, c∈Rnc\in\mathbb R^nc∈Rn, and consider

(P) min⁡ cTx  s.t. Ax=b, x≥0,(D) max⁡ bTy  s.t. ATy+s=c, s≥0.\text{(P)}\ \min\ c^Tx\ \text{ s.t. } Ax=b,\ x\ge0,\qquad \text{(D)}\ \max\ b^Ty\ \text{ s.t. } A^Ty+s=c,\ s\ge0 .(P) min cTx  s.t. Ax=b, x≥0,(D) max bTy  s.t. ATy+s=c, s≥0.

The strictly feasible pairs F0\mathcal F^0F0 are the (x,s)(x,s)(x,s) with x>0x>0x>0, s>0s>0s>0, Ax=bAx=bAx=b, and ATy+s=cA^Ty+s=cATy+s=c for some yyy. For such a pair the duality gap is xTsx^TsxTs, and the duality measure is μ=xTs/n\mu=x^Ts/nμ=xTs/n. Write X=diag⁡(x)X=\operatorname{diag}(x)X=diag(x), S=diag⁡(s)S=\operatorname{diag}(s)S=diag(s) and e=(1,…,1)Te=(1,\dots,1)^Te=(1,…,1)T.

Norms: ∥⋅∥\|\cdot\|∥⋅∥ is the Euclidean norm, ∥⋅∥∞\|\cdot\|_\infty∥⋅∥∞​ the maximum norm, and ∥z∥∞−:=∥z−∥∞\|z\|^-_\infty:=\|z^-\|_\infty∥z∥∞−​:=∥z−∥∞​, ∥z∥∞+:=∥z+∥∞\|z\|^+_\infty:=\|z^+\|_\infty∥z∥∞+​:=∥z+∥∞​ with (z−)j=min⁡{zj,0}(z^-)_j=\min\{z_j,0\}(z−)j​=min{zj​,0} and (z+)j=max⁡{zj,0}(z^+)_j=\max\{z_j,0\}(z+)j​=max{zj​,0}. For β∈(0,1)\beta\in(0,1)β∈(0,1) the wide neighbourhoods are

N∞(β)={(x,s)∈F0: ∥Xs−μe∥∞≤βμ},N∞−(β)={(x,s)∈F0: ∥Xs−μe∥∞−≤βμ}.\mathcal N_\infty(\beta)=\{(x,s)\in\mathcal F^0:\ \|Xs-\mu e\|_\infty\le\beta\mu\},\qquad \mathcal N^-_\infty(\beta)=\{(x,s)\in\mathcal F^0:\ \|Xs-\mu e\|^-_\infty\le\beta\mu\}.N∞​(β)={(x,s)∈F0: ∥Xs−μe∥∞​≤βμ},N∞−​(β)={(x,s)∈F0: ∥Xs−μe∥∞−​≤βμ}.

The second says only that xjsj≥(1−β)μx_js_j\ge(1-\beta)\muxj​sj​≥(1−β)μ for every jjj.

The search direction d=(dx,ds)d=(d_x,d_s)d=(dx​,ds​) with centering parameter γ\gammaγ solves, for some dyd_ydy​,

Sdx+Xds=γμe−Xs,Adx=0,ATdy+ds=0,(2)Sd_x+Xd_s=\gamma\mu e-Xs,\qquad Ad_x=0,\qquad A^Td_y+d_s=0, \tag{2}Sdx​+Xds​=γμe−Xs,Adx​=0,ATdy​+ds​=0,(2)

and the step is x(θ)=x+θdxx(\theta)=x+\theta d_xx(θ)=x+θdx​, s(θ)=s+θdss(\theta)=s+\theta d_ss(θ)=s+θds​. The scaled quantities p=X−1/2S1/2dxp=X^{-1/2}S^{1/2}d_xp=X−1/2S1/2dx​, q=X1/2S−1/2dsq=X^{1/2}S^{-1/2}d_sq=X1/2S−1/2ds​, r=(XS)−1/2(γμe−Xs)r=(XS)^{-1/2}(\gamma\mu e-Xs)r=(XS)−1/2(γμe−Xs) satisfy p+q=rp+q=rp+q=r and pTq=0p^Tq=0pTq=0. The componentwise product Pq=(pjqj)jPq=(p_jq_j)_jPq=(pj​qj​)j​ is the second-order term DxdsD_xd_sDx​ds​.

Algorithm 2 with neighbourhood N\mathcal NN and γ∈(0,1)\gamma\in(0,1)γ∈(0,1) starts at (x0,s0)∈N(x^0,s^0)\in\mathcal N(x0,s0)∈N with (x0)Ts0≤2t(x^0)^Ts^0\le2^t(x0)Ts0≤2t. While (xk)Tsk>2−t(x^k)^Ts^k>2^{-t}(xk)Tsk>2−t, it computes ddd from (2), takes the largest θˉ\bar\thetaθˉ such that (x(θ),s(θ))∈N(x(\theta),s(\theta))\in\mathcal N(x(θ),s(θ))∈N for all θ∈[0,θˉ]\theta\in[0,\bar\theta]θ∈[0,θˉ], and sets (xk+1,sk+1)=(x(θˉ),s(θˉ))(x^{k+1},s^{k+1})=(x(\bar\theta),s(\bar\theta))(xk+1,sk+1)=(x(θˉ),s(θˉ)).

Formalization targets

Goal: Theorem 2 with an explicit iteration count

Let β,γ∈(0,1)\beta,\gamma\in(0,1)β,γ∈(0,1) with γ≤2(1−β)\gamma\le2(1-\beta)γ≤2(1−β), and let N\mathcal NN be N∞(β)\mathcal N_\infty(\beta)N∞​(β) or N∞−(β)\mathcal N_\infty^-(\beta)N∞−​(β). Every run of Algorithm 2 reaches (xk)Tsk≤2−t(x^k)^Ts^k\le 2^{-t}(xk)Tsk≤2−t for some

k ≤ ⌈ln⁡2⋅n t2βγ(1−γ)⌉.k\ \le\ \Big\lceil\frac{\ln2\cdot n\,t}{2\beta\gamma(1-\gamma)}\Big\rceil .k ≤ ⌈2βγ(1−γ)ln2⋅nt​⌉.

The paper states "O(nt)O(nt)O(nt) iterations"; the count above is the one its inequality (15) yields.

Milestones, in the order the proof uses them

  • Lemma 1(b): −∥r∥2/4≤pjqj≤rj2/4-\|r\|^2/4\le p_jq_j\le r_j^2/4−∥r∥2/4≤pj​qj​≤rj2​/4 for each jjj.
  • Lemma 1(c): ∥Pq∥∞−≤∥r∥2/4≤n∥r∥∞2/4\|Pq\|^-_\infty\le\|r\|^2/4\le n\|r\|^2_\infty/4∥Pq∥∞−​≤∥r∥2/4≤n∥r∥∞2​/4, ∥Pq∥∞+≤∥r∥∞2/4\|Pq\|^+_\infty\le\|r\|^2_\infty/4∥Pq∥∞+​≤∥r∥∞2​/4, ∥Pq∥∞≤∥r∥2/4\|Pq\|_\infty\le\|r\|^2/4∥Pq∥∞​≤∥r∥2/4.
  • Lemma 2(c): if γ≤2(1−β)\gamma\le2(1-\beta)γ≤2(1−β) and (x,s)∈N∞−(β)(x,s)\in\mathcal N_\infty^-(\beta)(x,s)∈N∞−​(β) then ∥r∥2≤nμ\|r\|^2\le n\mu∥r∥2≤nμ; in N∞(β)\mathcal N_\infty(\beta)N∞​(β), ∥r∥∞2≤(1+β)μ\|r\|_\infty^2\le(1+\beta)\mu∥r∥∞2​≤(1+β)μ.
  • Lemma 5: θˉ≥θ2=min⁡{1,βγμ/∥Pq∥∞}\bar\theta\ge\theta_2=\min\{1,\beta\gamma\mu/\|Pq\|_\infty\}θˉ≥θ2​=min{1,βγμ/∥Pq∥∞​} in N∞(β)\mathcal N_\infty(\beta)N∞​(β), and θˉ≥θ2−=min⁡{1,βγμ/∥Pq∥∞−}\bar\theta\ge\theta_2^-=\min\{1,\beta\gamma\mu/\|Pq\|^-_\infty\}θˉ≥θ2−​=min{1,βγμ/∥Pq∥∞−​} in N∞−(β)\mathcal N_\infty^-(\beta)N∞−​(β).
  • Proof of Theorem 2: θ2−≥θ2≥4βγ/n\theta_2^-\ge\theta_2\ge4\beta\gamma/nθ2−​≥θ2​≥4βγ/n (for n≥2n\ge2n≥2).
  • (15): μk+1≤(1−4βγ(1−γ)/n)μk\mu^{k+1}\le\big(1-4\beta\gamma(1-\gamma)/n\big)\mu^kμk+1≤(1−4βγ(1−γ)/n)μk.

Significance

Theorem 2 shows that the adaptive long-step method in a neighbourhood covering almost all of F0\mathcal F^0F0 keeps the worst-case complexity of the original Kojima–Mizuno–Yoshise algorithm. Its per-iteration analysis (Lemma 5, Lemma 1(c)) is also what the paper reuses for its potential-reduction variant (§5, Theorem 3) and for its probabilistic explanation of why such methods take far fewer iterations in practice (§6). Lemma 1(b)–(c) are the standard bounds on the second-order Newton term, and they recur throughout the interior-point literature.

The results are proved in the paper. As far as the platform's corpus shows, none of them has a machine-checked proof: there is no formalization of the wide neighbourhoods N∞\mathcal N_\inftyN∞​, N∞−\mathcal N_\infty^-N∞−​, of the Newton system (2), or of an adaptive-step iteration. A complete development yields a reusable layer for primal-dual interior-point analysis in standard form (the neighbourhoods, the scaled direction, the second-order bounds) and the first formally verified iteration bound for a long-step primal-dual method.

Difficulty

Most of the steps are finite-dimensional inequalities. The sticking points are the following.

  • The bound pjqj≥−∥r∥2/4p_jq_j\ge-\|r\|^2/4pj​qj​≥−∥r∥2/4 uses the orthogonality pTq=0p^Tq=0pTq=0, which comes from Adx=0Ad_x=0Adx​=0 and ds=−ATdyd_s=-A^Td_yds​=−ATdy​. Those relations hold only for a solution of the full system (2), not for the first equation alone.
  • Lemma 5 needs every point of the segment [0,θ2][0,\theta_2][0,θ2​] to lie in the neighbourhood, including strict positivity of x(θ)x(\theta)x(θ) and s(θ)s(\theta)s(θ) and dual feasibility. Positivity of the products alone does not give positivity of the factors without a continuity argument along the segment.
  • "The largest θˉ\bar\thetaθˉ" is a maximum over a set defined by infinitely many conditions. The theorem needs only that θˉ\bar\thetaθˉ dominates θ2\theta_2θ2​, but the formal run must say what θˉ\bar\thetaθˉ is.
  • Converting the per-iteration contraction (15) into the explicit count needs (1−δ)k≤e−kδ(1-\delta)^k\le e^{-k\delta}(1−δ)k≤e−kδ, real powers of 222, and a ceiling.

Formalization scope

Vectors are Fin n → ℝ, with coordinates indexed 0,…,n−10,\dots,n-10,…,n−1, and A : Matrix (Fin m) (Fin n) ℝ. Mathlib's default norm on Fin n → ℝ is the sup norm, so the Euclidean norm norm2 and the norms normInf, normInfNeg, normInfPos are defined explicitly. μ\muμ divides by nnn as a real number. Here log⁡\loglog is the natural logarithm and 2t2^t2t is a real power with t∈Rt\in\mathbb Rt∈R.

Committed conventions and repairs:

  • The direction is "any (dx,dy,ds)(d_x,d_y,d_s)(dx​,dy​,ds​) solving (2)". No rank assumption on AAA is made.
  • "The largest θˉ\bar\thetaθˉ" is IsGreatest of the set of admissible step lengths, and θˉ\bar\thetaθˉ is not capped at 111. When no maximum exists (always the case for n=1n=1n=1, where Pq=0Pq=0Pq=0), no iteration exists. A run is then possible only if it has already stopped, and the theorem holds trivially. A sorry-free check confirms that an iteration exists for n=2n=2n=2.
  • A run is a pair of sequences that starts as required and makes one iteration from every iterate with (xk)Tsk>2−t(x^k)^Ts^k>2^{-t}(xk)Tsk>2−t. Its behaviour after the stopping test is unconstrained.
  • In θ2\theta_2θ2​ and θ2−\theta_2^-θ2−​, a zero norm makes the quotient +∞+\infty+∞ and the minimum 111. Since Lean's real division by zero is 000, this case is defined explicitly.
  • The bound θ2≥4βγ/n\theta_2\ge4\beta\gamma/nθ2​≥4βγ/n is stated for n≥2n\ge2n≥2. At n=1n=1n=1 it can fail (e.g. β=1/2\beta=1/2β=1/2, γ=0.9\gamma=0.9γ=0.9), and Algorithm 2 makes no iterations there.
  • The goal covers both neighbourhoods. A goal for N∞−\mathcal N^-_\inftyN∞−​ alone would be weaker than the paper.
  • Lemma 1 assumes only x,s>0x,s>0x,s>0 and γ∈[0,1]\gamma\in[0,1]γ∈[0,1] of the pair, which is all its statement uses.

The goal is not to be read with a hidden constant. A statement of the form "there is CCC with at most CntCntCnt iterations", with CCC allowed to depend on the instance, would be trivially true and is ruled out by the explicit count. Lemma 1(a) and Lemma 2(a), 2(b) belong to the paper's predictor–corrector analysis, which is a separate mission.

Contributions are welcome at every level: proofs of the milestones, helper lemmas about the norms on Fin n → ℝ, and the run-to-count argument.

Selected references

  • S. Mizuno, M. J. Todd, Y. Ye, On adaptive-step primal-dual interior-point algorithms for linear programming, Cornell ORIE Technical Report No. 944 (1990, rev. 1991); Mathematics of Operations Research 18(4):964–981, 1993. https://doi.org/10.1287/moor.18.4.964
  • M. Kojima, S. Mizuno, A. Yoshise, A primal-dual interior point algorithm for linear programming, in N. Megiddo (ed.), Progress in Mathematical Programming, Springer, 1989, pp. 29–47. https://doi.org/10.1007/978-1-4613-9617-8_2
  • R. D. C. Monteiro, I. Adler, Interior path following primal-dual algorithms. Part I: Linear programming, Mathematical Programming 44:27–41, 1989. https://doi.org/10.1007/BF01587075
  • S. J. Wright, Primal-Dual Interior-Point Methods, SIAM, 1997. https://doi.org/10.1137/1.9781611971453
11 thms3 active usersReviewed
🏆Completed
CombinatoricsOperations ResearchTheoretical Computer Science·Captain: mikedeng1

Approximation Algorithms for Scheduling Unrelated Parallel Machines 2: Enumerating Long Assignments and Rounding the LP Gives a (1 + ε)-Approximation for MakespanResearch Paper

Motivation

Scheduling independent jobs on unrelated parallel machines to minimize the makespan is one of the basic NP-hard problems of scheduling theory: each job has its own processing time on each machine, with no relation between machines, and the goal is to finish all jobs as early as possible. When the number of machines is part of the input, Lenstra, Shmoys and Tardos showed that no polynomial algorithm can come within a factor below 3/2 of the optimum unless P = NP. When the number of machines mmm is a fixed constant, the picture is different: the problem admits a polynomial approximation scheme, a family of algorithms that reach any accuracy 1+ε1+\varepsilon1+ε.

Such a scheme was already known. Horowitz and Sahni (1976) gave a fully polynomial approximation scheme for fixed mmm, but its space requirement grows like (nm/ε)m(nm/\varepsilon)^m(nm/ε)m. The second application of the Rounding Theorem in the 1987 report of Lenstra, Shmoys and Tardos is a scheme whose space is polynomial in the input size, in mmm, and in log⁡(1/ε)\log(1/\varepsilon)log(1/ε). This mission formalizes the approximation guarantee of that scheme (Theorem 3 of the report).

Timeline:

  • 1976. Horowitz and Sahni, a fully polynomial approximation scheme for a fixed number of unrelated machines, with exponential space in mmm.
  • 1985. Potts, a 2-approximation for unrelated machines whose running time is mm−1m^{m-1}mm−1 times a polynomial, based on LP rounding with enumeration.
  • 1987. Lenstra, Shmoys and Tardos, the Rounding Theorem, a polynomial 2-approximation for arbitrary mmm, and, for fixed mmm, the scheme formalized here (FOCS 1987; journal version in Mathematical Programming 46, 1990).

Setting

There are mmm machines and nnn jobs. If job jjj is scheduled on machine iii it takes pijp_{ij}pij​ time units, a positive integer; P=(pij)P=(p_{ij})P=(pij​) is the processing-time matrix. A schedule σ\sigmaσ assigns every job to one machine. The load of machine iii is ∑j:σ(j)=ipij\sum_{j:\sigma(j)=i} p_{ij}∑j:σ(j)=i​pij​, the makespan is the largest load, and OPT\mathrm{OPT}OPT is the smallest makespan over all schedules.

Fix an integer deadline d≥0d\ge0d≥0 and an accuracy ε>0\varepsilon>0ε>0. Assigning job jjj to machine iii is long if pij>εdp_{ij}>\varepsilon dpij​>εd and short otherwise. A schedule of long assignments LLL is a partial assignment of jobs to machines using only long assignments; its long load on machine iii is tit_iti​, and LLL is admissible if ti≤dt_i\le dti​≤d for every iii.

For a threshold ttt and deadlines d1,…,dmd_1,\dots,d_md1​,…,dm​, the linear program (LP) of the Rounding Theorem has a variable xij≥0x_{ij}\ge0xij​≥0 for every pair with pij≤tp_{ij}\le tpij​≤t, the constraints ∑ixij=1\sum_i x_{ij}=1∑i​xij​=1 for each job, and ∑jpijxij≤di\sum_j p_{ij}x_{ij}\le d_i∑j​pij​xij​≤di​ for each machine. The residual LP of (d,L)(d,L)(d,L) is (LP) for the jobs left unassigned by LLL, with di=d−tid_i=d-t_idi​=d−ti​ and t=εdt=\varepsilon dt=εd.

The procedure AεA_\varepsilonAε​, on input ddd, tries every admissible LLL; if the residual LP is feasible it takes a vertex x~\tilde xx~, rounds it to a 0-1 solution of the integer program (IP) of the Rounding Theorem, and combines the result with LLL. It answers 'no' if no residual LP is feasible. A ρ\rhoρ-relaxed decision procedure answers 'no' only when no schedule has makespan at most ddd, and otherwise returns a schedule of makespan at most ρd\rho dρd. The binary search of Lemma 1 runs such a procedure on deadlines between ⌈t/m⌉\lceil t/m\rceil⌈t/m⌉ and ttt, where ttt is the makespan of the greedy schedule, and returns the best schedule found.

Formalization targets

Goal: Theorem 3

For every m≥1m\ge1m≥1, every positive integer matrix PPP, every ε>0\varepsilon>0ε>0, every LP solver VVV, the procedure AεA_\varepsilonAε​ run with VVV exists, and the binary search driven by any such procedure returns a schedule σ\sigmaσ with

makespan(σ)≤(1+ε) OPT.\mathrm{makespan}(\sigma)\le(1+\varepsilon)\,\mathrm{OPT}.makespan(σ)≤(1+ε)OPT.

Milestones (§3, p. 6)

  1. Every machine in an admissible LLL carries fewer than 1/ε1/\varepsilon1/ε long assignments, and for n,m≥1n,m\ge1n,m≥1
#{L admissible}<(n+1)m/ε.\#\{L\text{ admissible}\}<(n+1)^{m/\varepsilon}.#{L admissible}<(n+1)m/ε.
  1. If some schedule has makespan at most ddd, its long part is admissible and its short part is a feasible solution of the residual LP.
  2. If the short assignments take at most d−ti+εdd-t_i+\varepsilon dd−ti​+εd on machine iii, the combined schedule has load at most (1+ε)d(1+\varepsilon)d(1+ε)d on every machine.
  3. AεA_\varepsilonAε​ exists for every LP solver and is a (1+ε)(1+\varepsilon)(1+ε)-relaxed decision procedure.

Significance

Theorem 3 shows that for a fixed number of unrelated machines, the makespan can be approximated to any accuracy in polynomial time while keeping the memory polynomial in log⁡(1/ε)\log(1/\varepsilon)log(1/ε). It is an early instance of the pattern "enumerate the large items, solve an LP for the small ones, round", which recurs in later approximation schemes for scheduling and packing. Together with the 2-approximation of the same paper (mission 1 of this series) and the 3/2 hardness bound (mission 3), it delimits what is achievable for unrelated machines.

The theorem is proved in the paper; none of it is known to have a machine-checked proof. Formalizing it requires a precise account of the enumeration of long assignments, the residual LP, and the binary search, and it exercises the Rounding Theorem, which this series poses in mission 1, outside its original setting (with a non-integral threshold t=εdt=\varepsilon dt=εd).

Difficulty

The arithmetic of the scheme is short; the content lies in the Rounding Theorem, which turns a vertex of the residual LP into a 0-1 schedule exceeding each deadline by at most the threshold. The obvious argument, rounding an arbitrary feasible LP solution, fails: a non-vertex solution may split many jobs across many machines, and rounding them can overload a machine by far more than ttt. The bound depends on the structure of vertices of (LP), which an arbitrary feasible point does not have. A second difficulty is the count of milestone 1, which must be done with a real exponent m/εm/\varepsilonm/ε and a strict inequality.

Formalization scope

  • Machines are Fin m, jobs Fin n, processing times a matrix P : Matrix (Fin m) (Fin n) ℕ with all entries positive. Load, makespan and optimal schedules are those of the published MatousekLP.Scheduling.Schedule, applied to the real cast of P.
  • A vertex of an LP is a point of Set.extremePoints of its feasible region. Variables absent from the paper's (LP) are kept as matrix entries fixed to 000, which leaves the vertices unchanged.
  • The LP solver is a parameter (a vertex selector) and AεA_\varepsilonAε​ is a predicate on procedures: each answer must extend an admissible schedule of long assignments, round the selected vertex within its support, and satisfy the (IP) load bound. The guarantee is proved for every such procedure. The output is thereby tied to what the algorithm computes, so the bound is not witnessed by an optimal schedule.
  • The binary search is a definition: greedy start, l=⌈t/m⌉l=\lceil t/m\rceill=⌈t/m⌉ (the paper's t/mt/mt/m rounded up because makespans are integers), d=⌊(u+l)/2⌋d=\lfloor(u+l)/2\rfloord=⌊(u+l)/2⌋, best stored schedule as output.
  • Not formalized: that mmm is fixed, the polynomial running time, and the space bound polynomial in mmm, log⁡(1/ε)\log(1/\varepsilon)log(1/ε) and the input size. The guarantee is stated for every m≥1m\ge1m≥1. The count (n+1)m/ε(n+1)^{m/\varepsilon}(n+1)m/ε of milestone 1, on which the running-time claim rests, is formalized, with the real power Real.rpow.
  • The constants are exact: 1+ε1+\varepsilon1+ε, (n+1)m/ε(n+1)^{m/\varepsilon}(n+1)m/ε, d−ti+εdd-t_i+\varepsilon dd−ti​+εd. Milestone 1 needs n≥1n\ge1n≥1 and m≥1m\ge1m≥1 for strictness.
  • Theorem 1 (the Rounding Theorem) and Lemma 1 are milestones of mission 1 of this series and are not restated here.

Welcome contributions: a proof of the Rounding Theorem in the generality of real deadlines and thresholds, proofs of the counting bound, and lemmas on the binary search that are reusable for any relaxed decision procedure.

Selected references

  • J.K. Lenstra, D.B. Shmoys, É. Tardos, Approximation algorithms for scheduling unrelated parallel machines, CWI Report OS-R8714, Centre for Mathematics and Computer Science, Amsterdam, 1987 (the source; Theorem 3, p. 6). Journal version: Mathematical Programming 46 (1990) 259–271. https://doi.org/10.1007/BF01585745
  • E. Horowitz, S. Sahni, Exact and approximate algorithms for scheduling nonidentical processors, Journal of the ACM 23 (1976) 317–327. https://doi.org/10.1145/321941.321951
  • C.N. Potts, Analysis of a linear programming heuristic for scheduling unrelated parallel machines, Discrete Applied Mathematics 10 (1985) 155–164. https://doi.org/10.1016/0166-218X(85)90009-5
  • D.S. Hochbaum, D.B. Shmoys, Using dual approximation algorithms for scheduling problems: theoretical and practical results, Journal of the ACM 34 (1987) 144–162. https://doi.org/10.1145/7531.7535
10 thms3 active usersReviewed
🏆Completed
Graph TheoryTheoretical Computer Science·Captain: mikedeng1

Multicommodity Max-Flow Min-Cut Theorems and Their Use in Designing Approximation Algorithms I: Every Uniform Multicommodity Flow Problem Has Ω(𝒮/log n) ≤ f ≤ 𝒮Research Paper

Motivation

A multicommodity flow problem asks to route several commodities simultaneously through a capacitated network, each from its own source to its own sink. For a single commodity, the max-flow min-cut theorem of Ford and Fulkerson says that the largest routable flow equals the capacity of the smallest separating cut. For many commodities the natural analogue fails: the max-flow can be strictly smaller than the min-cut, already for four commodities (Okamura and Seymour, 1981). Leighton and Rao asked how large the gap can be, and showed that for uniform demands it is at most logarithmic in the number of nodes. The result turned the min-cut, an NP-hard quantity, into something approximable by a linear program, and became the basis of the first polylogarithmic approximation algorithms for sparsest cut, balanced cut, minimum feedback arc set, crossing number and VLSI layout (Leighton–Rao, J. ACM 1999).

Timeline:

  • 1956–1963. Ford and Fulkerson prove max-flow = min-cut for one commodity; Hu (1963) proves equality for two commodities.
  • 1981–1990. Okamura and Seymour give a 4-commodity example with max-flow 3/4 and min-cut 1. Schrijver (1990), combining Lomonosov and Papernov, characterizes demand graphs for which equality always holds. Shahrokhi and Matula (1990) prove equality for uniform demands under a condition on the dual.
  • 1988. Leighton and Rao (FOCS 1988) prove the Θ(log⁡n)\Theta(\log n)Θ(logn) gap for uniform demands; the journal version (1999) is the source of this mission.
  • 1995. Linial, London and Rabinovich, and independently Aumann and Rabani, extend the bound to arbitrary demands with O(log⁡k)O(\log k)O(logk) for kkk commodities, via metric embeddings (Combinatorica 1995).

Setting

A network is a finite set VVV of nnn nodes with a capacity C(u,v)=C(v,u)≥0C(u,v)=C(v,u)\ge0C(u,v)=C(v,u)≥0 for every pair; the pair is an edge when C(u,v)>0C(u,v)>0C(u,v)>0. For U⊆VU\subseteq VU⊆V with complement Uˉ\bar UUˉ, the cut capacity C(U,Uˉ)C(U,\bar U)C(U,Uˉ) is the total capacity of the edges joining UUU to Uˉ\bar UUˉ. As in the paper, the network is assumed connected: C(U,Uˉ)>0C(U,\bar U)>0C(U,Uˉ)>0 for every nonempty proper UUU.

In the uniform multicommodity flow problem (UMFP) every unordered pair of nodes is a commodity with demand one. A routing of value λ\lambdaλ sends λ\lambdaλ units between every pair simultaneously, so that the total flow of all commodities over each edge, in both directions together, is at most its capacity. The max-flow fff is the largest such λ\lambdaλ (the concurrent max-flow).

The min-cut (sparsest cut) of the UMFP is

S=min⁡∅≠U⊊VC(U,Uˉ)∣U∣ ∣Uˉ∣,\mathcal S=\min_{\emptyset\ne U\subsetneq V}\frac{C(U,\bar U)}{|U|\,|\bar U|},S=∅=U⊊Vmin​∣U∣∣Uˉ∣C(U,Uˉ)​,

the minimum over cuts of the cut capacity divided by the demand separated by the cut. The quantity C(U,Uˉ)/(∣U∣∣Uˉ∣)C(U,\bar U)/(|U||\bar U|)C(U,Uˉ)/(∣U∣∣Uˉ∣) is the ratio cost of the cut.

The milestones use the dual objects. A distance function ddd assigns a nonnegative length to each edge; d(u,v)d(u,v)d(u,v) is then the shortest-path distance, d(T,u)=min⁡t∈Td(t,u)d(T,u)=\min_{t\in T}d(t,u)d(T,u)=mint∈T​d(t,u), and the total weight is W=∑eC(e)d(e)W=\sum_{e}C(e)d(e)W=∑e​C(e)d(e). The distance constraint (5) is ∑{u,v}d(u,v)≥1\sum_{\{u,v\}}d(u,v)\ge1∑{u,v}​d(u,v)≥1, summed over unordered pairs. A set of nodes has radius at most Δ\DeltaΔ if some centre reaches all its nodes by walks inside the set of length at most Δ\DeltaΔ. Throughout, log⁡n=log⁡2n\log n=\log_2 nlogn=log2​n.

Formalization targets

Goal: Theorem 2 (p. 795)

Ω ⁣(Slog⁡n)≤f≤S\Omega\!\left(\frac{\mathcal S}{\log n}\right)\le f\le\mathcal SΩ(lognS​)≤f≤S

for every uniform multicommodity flow problem: there is one absolute constant c>0c>0c>0 such that c S/log⁡n≤f≤Sc\,\mathcal S/\log n\le f\le\mathcal ScS/logn≤f≤S for every connected network with n≥2n\ge2n≥2 nodes. The goal fixes no constant, so it is the form any improved argument would still prove.

Milestones, in the order the argument uses them

  1. Weak duality (§1.2, pp. 789–790): every concurrent flow of value λ\lambdaλ satisfies λ ∣U∣∣Uˉ∣≤C(U,Uˉ)\lambda\,|U||\bar U|\le C(U,\bar U)λ∣U∣∣Uˉ∣≤C(U,Uˉ) for every cut.
  2. LP duality (§2.2, p. 796): every distance function satisfying (5) has W≥fW\ge fW≥f, and some such function has W=fW=fW=f.
  3. Lemma 3 (p. 797): for every Δ>0\Delta>0Δ>0, VVV splits into components of radius at most Δ\DeltaΔ with at most 4Wlog⁡n/Δ4W\log n/\Delta4Wlogn/Δ capacity between components.
  4. Corollary 4 (pp. 798–799): either a set of at least 2n/32n/32n/3 nodes has radius at most 1/(2n2)1/(2n^2)1/(2n2), or some cut has ratio cost at most 36Wlog⁡n36W\log n36Wlogn.
  5. Lemma 5 (p. 799): if ∣T∣≥2n/3|T|\ge 2n/3∣T∣≥2n/3 and ∑u∉Td(T,u)≥1/(2n)\sum_{u\notin T}d(T,u)\ge 1/(2n)∑u∈/T​d(T,u)≥1/(2n), some cut has ratio cost at most 6W6W6W.
  6. Lemma 6 (p. 800): a distance function satisfying (5) yields a cut of ratio cost at most 36Wlog⁡n36W\log n36Wlogn.

Companion

S≤36 flog⁡n\mathcal S\le 36\,f\log nS≤36flogn and f≤Sf\le\mathcal Sf≤S: Theorem 2 with the constant its proof produces.

Significance

The theorem says that the LP relaxation of uniform sparsest cut has integrality gap O(log⁡n)O(\log n)O(logn), and the expander example of §2.1 (Theorem 1) shows this is tight. Combined with the cut-finding argument, it gives an O(log⁡n)O(\log n)O(logn)-approximation for sparsest cut and, through the paper's Section 3, approximation algorithms for balanced separators, minimum cut linear arrangement, crossing number, VLSI layout area, feedback arc set and the emulation of networks. Region-growing decompositions of the kind in Lemma 3 reappear in low-diameter decompositions, multicut (Garg–Vazirani–Yannakakis), and metric embedding.

The result is proved; no machine-checked proof of it is known. A formalization would provide: concurrent multicommodity flow and its LP dual on finite graphs, the uniform sparsest cut, a verified region-growing decomposition, and the max-flow min-cut gap theorem itself. The LP-duality milestone, which the paper cites rather than proves, is a reusable piece of flow theory in its own right.

Difficulty

The upper bound f≤Sf\le\mathcal Sf≤S is a counting argument. The lower bound is the content. The natural idea, rounding an optimal dual distance function by a threshold and cutting the long edges, does not work: the paper notes (p. 797) that no thresholding rule is known that always yields a good cut, because edges with large distance need not form a sparse cut. Any argument must convert the fractional dual into a single cut while losing only a log⁡n\log nlogn factor against the total weight WWW, which requires a global property of the distance function rather than the lengths of individual edges. The LP-duality step needs a correct dual for the concurrent flow problem with shortest-path distances, which Mathlib does not provide off the shelf.

Formalization scope

  • Nodes and capacities. V is a Fintype with decidable equality; capacities are a symmetric nonnegative real function vanishing on the diagonal, and the graph is the simple graph of pairs with positive capacity. Goal-level statements assume 2 ≤ Fintype.card V and connectivity in the form C(U,Uˉ)>0C(U,\bar U)>0C(U,Uˉ)>0 for every nonempty proper UUU (the paper's standing assumption, p. 789).
  • Demands and flows. Each unordered pair is encoded as two ordered commodities with demand 12\tfrac1221​ (the paper's footnote 2). Flows are arc flows with conservation per commodity, and both directions of an edge share its capacity. The max-flow is the real supremum of feasible values; with n≥2n\ge2n≥2 that set is nonempty and bounded.
  • Cuts. The min-cut ranges over nonempty proper subsets only. Including U=∅U=\emptysetU=∅ or U=VU=VU=V would give 0/00/00/0, which Lean evaluates to 000 and which would make the goal trivially true; this formalization excludes it.
  • Distances. d(u,v)d(u,v)d(u,v) is an infimum over walks in the graph, meaningful under connectivity; the radius of a component is measured by walks inside it.
  • Constants. The goal keeps the page's Ω\OmegaΩ with a single universal constant chosen before the network. The milestones carry the constants computed in the paper's proofs (363636 in Corollary 4 and Lemma 6, 666 in Lemma 5), and each statement says so.
  • Algorithms. "We can find" is formalized as existence. Running times and the remark that the proofs yield polynomial-time algorithms (p. 801) are out of scope.

Contributions welcome: proofs of any milestone; a general LP-duality theorem for concurrent multicommodity flow, reusable for the product and directed variants; and a general region-growing lemma.

Selected references

  • T. Leighton and S. Rao, Multicommodity max-flow min-cut theorems and their use in designing approximation algorithms, J. ACM 46(6):787–832, 1999. https://doi.org/10.1145/331524.331526
  • F. Shahrokhi and D. W. Matula, The maximum concurrent flow problem, J. ACM 37(2):318–334, 1990. https://doi.org/10.1145/77600.77620
  • H. Okamura and P. D. Seymour, Multicommodity flows in planar graphs, J. Combin. Theory Ser. B 31:75–81, 1981. https://doi.org/10.1016/S0095-8956(81)80012-3
  • N. Linial, E. London and Y. Rabinovich, The geometry of graphs and some of its algorithmic applications, Combinatorica 15(2):215–245, 1995. https://doi.org/10.1007/BF01200757
  • V. Chvátal, Linear Programming, W. H. Freeman, 1983.
10 thms3 active usersReviewed
🏆Completed
Complexity TheoryOperations ResearchOptimization·Captain: mikedeng1

On Adaptive-Step Primal-Dual Interior-Point Algorithms for Linear Programming 1: The Predictor–Corrector Algorithm Between N₂(1/4) and N₂(1/2) Attains Precision t in O(√n·t) IterationsResearch Paper

Motivation

Linear programming asks for the best value of a linear objective subject to linear equations and inequalities. Primal-dual interior-point methods seek feasible primal and dual solutions while keeping every constrained coordinate positive. The resulting pair has a measurable gap between its objective values, so an algorithm can stop when the gap is below a chosen precision. Mizuno, Todd, and Ye studied methods that adjust the length of each step according to the current point. Their technical report gives a predictor–corrector algorithm whose worst-case iteration count grows on the scale of the square root of the number of variables times the requested precision parameter Mizuno, Todd, and Ye, Cornell ORIE Technical Report No. 944, pp. 6–9.

The result matters to readers who need a mathematical guarantee for an adaptive step rule. The algorithm takes one predictor step that reduces the gap and then one corrector step that recenters the pair. Its step length is defined by remaining inside a specified neighborhood, rather than by a fixed schedule. The theorem connects that local rule to a global termination bound.

Setting

Fix an integer n≥1n\ge1n≥1 of variables, an arbitrary number mmm of equality constraints, a real matrix A∈Rm×nA\in\mathbb R^{m\times n}A∈Rm×n, and vectors b∈Rmb\in\mathbb R^mb∈Rm and c∈Rnc\in\mathbb R^nc∈Rn. The primal problem minimizes cTxc^TxcTx over Ax=bAx=bAx=b, x≥0x\ge0x≥0. Its dual problem maximizes bTyb^TybTy over ATy+s=cA^Ty+s=cATy+s=c, s≥0s\ge0s≥0. The dual slack is sss. The development reuses a published general-form LP model and specializes it to this pair.

The strictly feasible pairs F0F^0F0 are the pairs (x,s)(x,s)(x,s) satisfying the two feasibility systems for some dual multiplier yyy, with every coordinate of xxx and sss strictly positive. The multiplier is allowed to vary with the pair. Write X=diag⁡(x)X=\operatorname{diag}(x)X=diag(x), S=diag⁡(s)S=\operatorname{diag}(s)S=diag(s), and eee for the vector of ones. The duality measure is μ=xTs/n\mu=x^Ts/nμ=xTs/n. For 0<β<10<\beta<10<β<1, the Euclidean central-path neighborhood is

N2(β)={(x,s)∈F0:∥Xs−μe∥2≤βμ}.N_2(\beta)=\{(x,s)\in F^0:\|Xs-\mu e\|_2\le\beta\mu\}.N2​(β)={(x,s)∈F0:∥Xs−μe∥2​≤βμ}.

The search direction (dx,dy,ds)(d_x,d_y,d_s)(dx​,dy​,ds​) at (x,s)(x,s)(x,s) with parameter γ\gammaγ solves the three equations Sdx+Xds=γμe−XsSd_x+Xd_s=\gamma\mu e-XsSdx​+Xds​=γμe−Xs, Adx=0Ad_x=0Adx​=0, and ATdy+ds=0A^Td_y+d_s=0ATdy​+ds​=0. A step of length θ\thetaθ yields (x+θdx,s+θds)(x+\theta d_x,s+\theta d_s)(x+θdx​,s+θds​). Algorithm 1 begins in N2(1/4)N_2(1/4)N2​(1/4). It takes a predictor direction with γ=0\gamma=0γ=0 to the greatest step whose entire path remains in N2(1/2)N_2(1/2)N2​(1/2), then takes one full corrector direction with γ=1\gamma=1γ=1. The resulting pair starts the next iteration. The algorithm stops when xTs≤2−tx^Ts\le2^{-t}xTs≤2−t; its starting gap is at most 2t2^t2t report, pp. 4 and 6–7.

Formalization targets

Theorem 1: termination

For every real precision parameter ttt, every run following Algorithm 1 from an allowed starting pair reaches the stopping threshold at some iteration kkk with

k≤⌈2(log⁡2)81/4n t⌉,(xk)Tsk≤2−t.k\le\left\lceil2(\log2)8^{1/4}\sqrt n\,t\right\rceil, \qquad (x^k)^Ts^k\le2^{-t}.k≤⌈2(log2)81/4n​t⌉,(xk)Tsk≤2−t.

The paper states the iteration order as O(n t)O(\sqrt n\,t)O(n​t). The explicit ceiling records the count given by its proof without allowing an unspecified constant to depend on the LP instance report, Theorem 1, pp. 8–9.

Source milestones

The supporting targets are equations (4)–(5), Lemma 1(a), Lemma 2(a)–(b), Lemma 3, Lemma 4, and the contraction displayed in the proof of Theorem 1. They cover the duality measure along a step, bounds on the scaled second-order term, preservation of the inner neighborhood, the predictor step length, and the per-iteration reduction. The milestone list follows the order in which these facts enter the report's analysis report, pp. 4–9.

Significance

Theorem 1 gives a stopping guarantee for a fully specified adaptive predictor–corrector method on a feasible primal-dual LP pair. In particular, the requested precision can grow while the iteration bound scales with n\sqrt nn​, rather than with nnn, in this neighborhood model. It also gives a quantitative reason to track two neighborhoods: the predictor may leave the inner one, while the corrector brings the iterate back for the next round.

The report proves this mathematical claim, but this mission does not assert that its proof has already been machine checked. A completed formalization would supply reusable Lean definitions for strictly feasible LP pairs, Euclidean centrality, search directions, and algorithmic step relations. Solvers may formalize the report's argument or another proof of the same statements; the audited goal and milestones fix the mathematical contract.

Difficulty

The predictor direction is aimed at reducing the duality measure, but a long step can move the componentwise products xjsjx_js_jxj​sj​ away from their common average or make a coordinate nonpositive. The corrector recenters the pair, yet its effect must be controlled in the same Euclidean neighborhood. A gap decrease alone therefore does not imply that another iteration can be taken. The central difficulty is to establish both a nonzero admissible predictor length and preservation of strict feasibility and centrality under the full corrector step.

Formalization scope

Vectors are functions on Fin n, and AAA is a finite matrix. The report's indices 1,…,n1,\ldots,n1,…,n correspond to Lean indices 0,…,n−10,\ldots,n-10,…,n−1. The norm in N2N_2N2​ and in Lemmas 1–2 is explicitly ℓ2\ell_2ℓ2​: Lean's default norm on functions is a supremum norm. All powers of two use real exponents, and log⁡\loglog is natural logarithm. The hypothesis n≥1n\ge1n≥1 gives meaning to μ=xTs/n\mu=x^Ts/nμ=xTs/n; no sign condition on ttt is imposed, since for t≤0t\le0t≤0 the starting pair already meets the stopping test.

The source's “largest” predictor step is represented as an attained greatest element of the admissible set, with admissibility required throughout the step interval. Some interior points have admissible intervals without a greatest member. In particular, at n=1n=1n=1 there may be no continuing run under this literal algorithm rule; the goal retains n≥1n\ge1n≥1 and makes this limitation explicit. The denominator ∥Pq∥2\|Pq\|_2∥Pq∥2​ in Lemma 4 can vanish. The formal lower bound uses its limiting value 1/21/21/2 in that case, avoiding a zero returned by Lean's total division. The separate contraction milestone assumes n≥2n\ge2n≥2, because its printed lower bound on the predictor length fails at n=1n=1n=1.

The definitions keep primal feasibility, an existential dual multiplier, coordinatewise strict positivity, and the three direction equations visible. A sequence is constrained by the transition relation whenever its gap exceeds 2−t2^{-t}2−t; values after stopping play no role. These conditions exclude trivial encodings in which the gap threshold is built into the run or centrality is measured with the wrong norm. The remaining work includes Euclidean finite-sum estimates, positivity along steps, and the iteration-count argument. The LP specialization and norm infrastructure can be reused in further interior-point developments.

Selected references

  • Shinji Mizuno, Michael J. Todd, and Yinyu Ye, On Adaptive-Step Primal-Dual Interior-Point Algorithms for Linear Programming, Cornell ORIE Technical Report No. 944, 1990, revised 1991; published in Mathematics of Operations Research 18(4), 1993. Technical-report PDF. The PDF used here is the technical report, so all page and result indices refer to that version.
12 thms3 active usersReviewed
CombinatoricsDiscrete Geometry·Captain: mikedeng1

The d-Step Conjecture for Polyhedra of Dimension d < 6 1: The Bounded 5-Step Conjecture Holds, Every 5-Polytope with 10 Facets Has Diameter at Most 5, i.e. Δb(5, 10) = 5Research Paper

Motivation

The simplex method and every other edge-following linear programming algorithm moves from vertex to vertex of the feasible region along its edges. The number of steps such an algorithm needs in the worst case is at least the combinatorial diameter of the feasible polyhedron: the largest distance, counted in edges, between two of its vertices. Klee and Walkup introduced the two functions that measure this worst case, Δ(d,n)\Delta(d,n)Δ(d,n) for polyhedra and Δb(d,n)\Delta_b(d,n)Δb​(d,n) for bounded polyhedra of dimension ddd with nnn facets, and wrote that Δ(d,n)\Delta(d,n)Δ(d,n) "represents, in a sense, the number of iterations required to solve the 'worst' linear program of nnn inequalities in ddd variables using the 'best' edge-following algorithm" (Klee and Walkup 1967, p. 54).

The Hirsch conjecture, reported by Dantzig in 1963, asserted Δ(d,n)≤n−d\Delta(d,n)\le n-dΔ(d,n)≤n−d. Its diagonal cases Δ(d,2d)≤d\Delta(d,2d)\le dΔ(d,2d)≤d and Δb(d,2d)≤d\Delta_b(d,2d)\le dΔb​(d,2d)≤d are the general and the bounded ddd-step conjectures.

Timeline.

  • 1957–1963: W. M. Hirsch's conjecture, reported in Dantzig's Linear Programming and Extensions (1963, pp. 160, 168), with the claim that it holds for m≤4m\le 4m≤4.
  • Before 1967: Klee determines Δ(3,n)=n−3\Delta(3,n)=n-3Δ(3,n)=n−3 and Δb(3,n)=⌊2n/3⌋−1\Delta_b(3,n)=\lfloor 2n/3\rfloor-1Δb​(3,n)=⌊2n/3⌋−1 and settles Δb(d,n)\Delta_b(d,n)Δb​(d,n) for n≤d+4n\le d+4n≤d+4 (Canad. J. Math. 16 and the papers cited as [9], [12], [13] by Klee and Walkup, p. 55).
  • 1967: Klee and Walkup (Acta Math. 117) prove Δb(5,10)=5\Delta_b(5,10)=5Δb​(5,10)=5, show that Δb(d,2d)≤d\Delta_b(d,2d)\le dΔb​(d,2d)≤d for all ddd is equivalent to the bounded Hirsch conjecture, and disprove the unbounded conjecture with Δ(4,8)=5\Delta(4,8)=5Δ(4,8)=5.
  • 2012: Santos disproves the bounded Hirsch conjecture with a 43-dimensional polytope with 86 facets (Ann. Math. 176). The bounded ddd-step conjecture therefore fails for some ddd, and the smallest failing ddd is unknown; it is at least 6.

Setting

A polyhedron in Rd\mathbb R^dRd is a nonempty set P={x:⟨ai,x⟩≤bi, i=1,…,n}P=\{x:\langle a_i,x\rangle\le b_i,\ i=1,\dots,n\}P={x:⟨ai​,x⟩≤bi​, i=1,…,n}; a polytope is a bounded polyhedron. PPP is a ddd-polyhedron with nnn facets when it has nonempty interior and none of the nnn inequalities is redundant; then the rows are in bijection with the facets, and row iii is tight at vvv exactly when vvv lies on the iii-th facet. The vertices of PPP are its extreme points. Two vertices u≠vu\ne vu=v span an edge when the segment [u,v][u,v][u,v] is an extreme subset of PPP. A path of length lll is a sequence of lll edges [x0,x1],…,[xl−1,xl][x_0,x_1],\dots,[x_{l-1},x_l][x0​,x1​],…,[xl−1​,xl​], and δP(x,y)\delta_P(x,y)δP​(x,y) is the least length of a path from xxx to yyy. Then

Δb(d,n)=max⁡{δP(x,y):P a d-polytope with n facets, x,y vertices of P}.\Delta_b(d,n)=\max\{\delta_P(x,y): P\text{ a }d\text{-polytope with }n\text{ facets},\ x,y\text{ vertices of }P\}.Δb​(d,n)=max{δP​(x,y):P a d-polytope with n facets, x,y vertices of P}.

PPP is simple when every vertex lies on exactly ddd facets. A face is PPP itself or the intersection of PPP with a supporting hyperplane, never ∅\emptyset∅; its dimension is that of its affine hull. A (k1,…,kr)(k_1,\dots,k_r)(k1​,…,kr​)-path from xxx to yyy is a sequence of faces K1,…,KrK_1,\dots,K_rK1​,…,Kr​ with x∈K1x\in K_1x∈K1​, y∈Kry\in K_ry∈Kr​, dim⁡Ki=ki\dim K_i=k_idimKi​=ki​ and Ki∩Ki+1≠∅K_i\cap K_{i+1}\ne\emptysetKi​∩Ki+1​=∅. A ddd-dimensional Dantzig figure (P,x,y)(P,x,y)(P,x,y) is a ddd-polyhedron PPP with 2d2d2d facets and two vertices x,yx,yx,y such that exactly ddd facets are incident to xxx and the other ddd to yyy. A ddd-polytope QQQ has property A when it has between 2d2d2d and 2d+22d+22d+2 facets and the following holds: if its facets are split into two classes X,Y\mathfrak X,\mathfrak YX,Y of at most d+1d+1d+1 facets each, and both sets XXX, YYY of vertices all of whose facets lie in X\mathfrak XX, respectively Y\mathfrak YY, are nonempty, then some vertex of XXX is joined to some vertex of YYY by a path of length at most ddd.

Formalization targets

Goal: 4.3 THEOREM

Δb(5,10)=5.\Delta_b(5,10)=5.Δb​(5,10)=5.

In two halves: every 555-polytope with 101010 facets satisfies δP(x,y)≤5\delta_P(x,y)\le 5δP​(x,y)≤5 for all vertices x,yx,yx,y, and some 555-polytope with 101010 facets has two vertices with δP(x,y)>4\delta_P(x,y)>4δP​(x,y)>4.

Milestones, in the order the proof uses them

  1. 2.8 THEOREM (bounded part): Δb(d,n)\Delta_b(d,n)Δb​(d,n) is attained by two vertices of a simple ddd-polytope with nnn facets, and for n≥2dn\ge 2dn≥2d by two vertices on no common facet. For n=2dn=2dn=2d this is a bounded simple Dantzig figure.
  2. 3.4 PROPOSITION (a): every simple Dantzig figure admits a (k1,1,k3)(k_1,1,k_3)(k1​,1,k3​)-path for positive k1+1+k3=dk_1+1+k_3=dk1​+1+k3​=d.
  3. 3.1 PROPOSITION: (a) facial paths in all ddd-dimensional simple Dantzig figures, (b) facet-avoiding facial paths in simple ddd-polytopes with 2d+12d+12d+1 facets, and (c) facet-avoiding (1,k1,…,kr)(1,k_1,\dots,k_r)(1,k1​,…,kr​)-paths in bounded (d+1)(d+1)(d+1)-dimensional figures are equivalent.
  4. 3.4 PROPOSITION (b): every bounded simple Dantzig figure admits a (1,k2,1,k4)(1,k_2,1,k_4)(1,k2​,1,k4​)-path for positive 1+k2+1+k4=d1+k_2+1+k_4=d1+k2​+1+k4​=d.
  5. Remark after 3.4: adjacent terms of a facial path in a simple figure merge into their sum.
  6. Remark after 3.4, p. 68: bounded simple Dantzig figures of dimension d≥2d\ge 2d≥2 admit (1,d−2,1)(1,d-2,1)(1,d−2,1)-paths.
  7. 4.1 PROPOSITION: if the middle face QQQ of a (1,d−2,1)(1,d-2,1)(1,d−2,1)-path ([x,x′],Q,[y′,y])([x,x'],Q,[y',y])([x,x′],Q,[y′,y]) has property A, then δP(x,y)≤d\delta_P(x,y)\le dδP​(x,y)≤d.
  8. 4.2 LEMMA: every simple 333-polytope with 666, 777 or 888 facets has property A.

Significance

The result. By the paper's reductions, Δb(d,2d)≤d\Delta_b(d,2d)\le dΔb​(d,2d)≤d for a given ddd is equivalent to the bounded Hirsch bound Δb(d′,d′+d)≤d\Delta_b(d',d'+d)\le dΔb​(d′,d′+d)≤d for every d′d'd′. So 4.3 gives Δb(d,n)≤n−d\Delta_b(d,n)\le n-dΔb​(d,n)≤n−d for all n≤d+5n\le d+5n≤d+5, which is the column n−d=5n-d=5n−d=5 of the paper's Fig. 2. Since Santos's counterexample, the smallest ddd with Δb(d,2d)>d\Delta_b(d,2d)>dΔb​(d,2d)>d is a well-defined open number. This theorem was the first to show it is at least 666; later computer-assisted work treated further small cases, including the bounded 666-step case (Bremner and Schewe 2011). Santos's construction starts from exactly the reduction to Dantzig figures (the "strong ddd-step theorem" for spindles) that this paper uses.

Formalizing it. The result has been proved since 1967; no machine-checked proof of Δb(5,10)=5\Delta_b(5,10)=5Δb​(5,10)=5 exists. The platform has the Hirsch bound in dimension ≤3\le 3≤3 (Hirsch.dimension_three_bound, Hirsch.klee_three_dimensional_bound), the Klee–Walkup wedge reduction at the level of inequality descriptions (Hirsch.balanced_hpoly_transfer), products (Hirsch.diamLE_prod) and Santos's counterexample. The mission adds the facial-path calculus of §3 and the property-A argument of §4. Both are reusable in any later formal treatment of small ddd-step cases.

Difficulty

The obvious approach, a direct case analysis of 555-polytopes with 101010 facets, is out of reach: there are far too many combinatorial types, and nothing bounds the search. The paper first reduces to simple polytopes, and the reduction itself (2.8) rests on a perturbation lemma (2.6) whose proof is deferred to a separate paper by Walkup: simplicity must be gained without shrinking any distance. The facial-path results then pass between bounded and unbounded figures through truncation and the wedge (3.1), and a statement about all unbounded ddd-dimensional figures is what yields one about bounded (d+1)(d+1)(d+1)-dimensional ones. Finally the (d−2)(d-2)(d−2)-face QQQ must be shown to have property A. That property fails for some simple polytopes of classes (4,10)(4,10)(4,10) and (5,11)(5,11)(5,11) (p. 69), so 4.2 depends on the dimension being 333 and on the facet count being at most 888. Its proof uses Euler's relation and Balinski's theorem that the graph of a 333-polytope is 333-connected (posed on the platform as Balinski61.Connectivity.disjoint_paths_between_vertices).

Formalization scope

Polyhedra are Hirsch.Hpoly a b with a : Fin n → EuclideanSpace ℝ (Fin d); edges and walks are Hirsch.Adj and Hirsch.Reach from the published Hirsch_model and Hirsch_walk. The conventions are:

  • "ddd-polyhedron with nnn facets" is a facet presentation: nonempty interior and no redundant row. The rows are exactly the facets; a statement over arbitrary inequality systems would count redundant rows as facets and is not the paper's.
  • δP(x,y)≤L\delta_P(x,y)\le LδP​(x,y)≤L is Hirsch.Reach P L x y, a walk of LLL steps that may stand still. Δb(5,10)=5\Delta_b(5,10)=5Δb​(5,10)=5 is an upper bound for every polytope plus a witness, never a supremum of reals. A version proving only the upper bound, or only the lower bound, is not the goal.
  • Simple means pointed with exactly ddd tight rows at every vertex. On a facet presentation this is equivalent to the paper's "exactly ddd edges at each vertex".
  • Faces are nonempty sets {x∈P:⟨ai,x⟩=bi, i∈I}\{x\in P:\langle a_i,x\rangle=b_i,\ i\in I\}{x∈P:⟨ai​,x⟩=bi​, i∈I}, and dimension is the dimension of the affine span. A Dantzig figure has Fin (2 * d) rows.
  • Property A is defined for a bounded face QQQ of dimension mmm: its facets are the (m−1)(m-1)(m−1)-faces of PPP inside QQQ, and the path runs in QQQ. This keeps 4.1, where QQQ is a (d−2)(d-2)(d−2)-face of a polytope in Rd\mathbb R^dRd, and 4.2, where Q=PQ=PQ=P, under one notion. Facet counts use Set.encard.
  • Explicit readings: 2.8's "Δb\Delta_bΔb​ can be realized as the distance … of a simple polytope" is stated polytope by polytope (δP(u,v)≤δP′(x,y)\delta_P(u,v)\le\delta_{P'}(x,y)δP​(u,v)≤δP′​(x,y)). Only the Δb\Delta_bΔb​ sentences of 2.8 are stated. 3.4's joint hypothesis on k1,…,k4k_1,\dots,k_4k1​,…,k4​ is split by part. The merging remark carries the needed hypothesis ki+ki+1≤dk_i+k_{i+1}\le dki​+ki+1​≤d. The printed "3.5 implies" on p. 67 is read as 3.4.

The paper cites two inputs without proving them: Lemma 2.6 (Walkup's simple perturbation) and Balinski's theorem. A complete development needs face lattices of H-polyhedra, wedges and truncations, and the planar-graph arguments of 4.2. The face and facial-path layer is reusable for the companion mission on Δ(4,8)=5\Delta(4,8)=5Δ(4,8)=5 and for other diameter problems. Contributions are welcome at every milestone, including proofs of 2.6.

Selected references

  • V. Klee and D. W. Walkup, The d-step conjecture for polyhedra of dimension d < 6, Acta Mathematica 117 (1967), 53–78. https://doi.org/10.1007/BF02395040
  • G. B. Dantzig, Linear Programming and Extensions, Princeton University Press, 1963.
  • V. Klee, Diameters of polyhedral graphs, Canadian Journal of Mathematics 16 (1964), 602–614. https://doi.org/10.4153/CJM-1964-058-5
  • M. L. Balinski, On the graph structure of convex polyhedra in n-space, Pacific Journal of Mathematics 11 (1961), 431–434. https://doi.org/10.2140/pjm.1961.11.431
  • F. Santos, A counterexample to the Hirsch conjecture, Annals of Mathematics 176 (2012), 383–412. https://doi.org/10.4007/annals.2012.176.1.7
  • D. Bremner and L. Schewe, Edge-graph diameter bounds for convex polytopes with few facets, Experimental Mathematics 20 (2011), 229–237. https://doi.org/10.1080/10586458.2011.564966
12 thms3 active usersReviewed
Algorithmic Game TheoryOptimization·Captain: mikedeng1

Bimatrix Equilibrium Points and Mathematical Programming: If Z Is Non-Degenerate and uᵀMu ≥ 0, uᵀMu = 0 ⇒ Mu + Mᵀu = 0 for All u ≥ 0, a Non-Empty Z Has an Equilibrium PointResearch Paper

Motivation

The linear complementarity problem asks, for a real square matrix MMM of order nnn and a vector q∈Rnq\in\mathbb R^nq∈Rn, for a vector zzz with

z≥0,w=Mz−q≥0,zTw=0.z\ge 0,\qquad w=Mz-q\ge 0,\qquad z^{\mathsf T}w=0 .z≥0,w=Mz−q≥0,zTw=0.

It is the common form of the Kuhn–Tucker conditions of linear and convex quadratic programs and of the equilibrium conditions of two-person non-zero-sum (bimatrix) games. In 1964 Lemke and Howson gave a constructive existence proof for equilibrium points of bimatrix games by following a path of adjacent extreme points of a polyhedron (Lemke–Howson 1964). In 1965 Lemke adapted that path-following technique to the general problem above, under a condition on MMM that covers the positive semidefinite case treated by Dantzig and Cottle and the case M>0M>0M>0 (Lemke 1965). The resulting procedure, now called Lemke's algorithm or complementary pivoting, became a standard method for linear complementarity problems and a template for later path-following existence proofs (Cottle, Pang, Stone 1992).

Timeline:

  • 1964: Lemke and Howson, adjacent-extreme-point path for bimatrix games.
  • 1965: Lemke, existence for matrices satisfying conditions (i)–(ii) below (the goal of this mission), under a non-degeneracy assumption, by an augmented path with an artificial variable z0z_0z0​.
  • Later literature names matrices with (i)–(ii) copositive-plus, and removes non-degeneracy by lexicographic perturbation.

Setting

Fix a finite index set of size nnn, a real n×nn\times nn×n matrix MMM and q∈Rnq\in\mathbb R^nq∈Rn. Inequalities between vectors are componentwise and eee is the all-ones vector.

  • The set Z={z∈Rn:z≥0, w=Mz−q≥0}Z=\{z\in\mathbb R^n : z\ge 0,\ w=Mz-q\ge 0\}Z={z∈Rn:z≥0, w=Mz−q≥0}; www is always the function Mz−qMz-qMz−q of zzz.
  • An equilibrium point is a point z∈Zz\in Zz∈Z with zTw=0z^{\mathsf T}w=0zTw=0; SSS is the set of them.
  • For any z∈Rnz\in\mathbb R^nz∈Rn, the matrix N(z)N(z)N(z) consists of the columns of (MT,I)(M^{\mathsf T},I)(MT,I) that are kept when one deletes the iii-th column of MTM^{\mathsf T}MT whenever wi≠0w_i\ne 0wi​=0 and the iii-th column of III whenever zi≠0z_i\ne0zi​=0.
  • An extreme point of ZZZ is a point z∈Zz\in Zz∈Z with rank⁡N(z)=n\operatorname{rank}N(z)=nrankN(z)=n; zzz lies on an open edge if z∈Zz\in Zz∈Z and rank⁡N(z)=n−1\operatorname{rank}N(z)=n-1rankN(z)=n−1. An open edge is the set of points of ZZZ sharing the matrix N(z)N(z)N(z) of such a point; a ray is an open edge with exactly one end-point.
  • ZZZ is non-degenerate if for every z∈Rnz\in\mathbb R^nz∈Rn the columns of N(z)N(z)N(z) are linearly independent.
  • For an index sss, Zs={z∈Z:zTw=zsws}Z_s=\{z\in Z : z^{\mathsf T}w=z_sw_s\}Zs​={z∈Z:zTw=zs​ws​}. An adjacency path is a non-empty connected class of closed edges of ZZZ no three of which meet; its end-points are the extreme points lying on exactly one of its edges.
  • The augmented sets: Z∗={(z,z0):z≥0, z0≥0, Mz+z0e−q≥0}Z^*=\{(z,z_0) : z\ge0,\ z_0\ge0,\ Mz+z_0e-q\ge0\}Z∗={(z,z0​):z≥0, z0​≥0, Mz+z0​e−q≥0}, its subset Z0∗Z_0^*Z0∗​ where zTw=0z^{\mathsf T}w=0zTw=0, and, for a number kkk, Z∗∗Z^{**}Z∗∗, the set of the form above for the bordered data (Me−eT0)\begin{pmatrix}M&e\\-e^{\mathsf T}&0\end{pmatrix}(M−eT​e0​), (q−k)\begin{pmatrix}q\\-k\end{pmatrix}(q−k​).

Formalization targets

Goal: Theorem 4 (p. 7)

Suppose ZZZ is non-degenerate and that for every u≥0u\ge0u≥0

(i) uTMu≥0,(ii) uTMu=0 ⟹ Mu+MTu=0.\text{(i)}\ u^{\mathsf T}Mu\ge 0,\qquad \text{(ii)}\ u^{\mathsf T}Mu=0\ \Longrightarrow\ Mu+M^{\mathsf T}u=0 .(i) uTMu≥0,(ii) uTMu=0 ⟹ Mu+MTu=0.

Then

Z≠∅ ⟹ ∃z∈Z, zT(Mz−q)=0.Z\ne\emptyset\ \Longrightarrow\ \exists z\in Z,\ z^{\mathsf T}(Mz-q)=0 .Z=∅ ⟹ ∃z∈Z, zT(Mz−q)=0.

Milestones, in the order the paper's proof uses them

  1. Theorem 1 (p. 3): for non-degenerate ZZZ, ZsZ_sZs​ is empty or a disjoint union of adjacency paths whose end-points are exactly the equilibrium points.
  2. Lemma 2 (p. 4): Z=∅Z=\emptysetZ=∅ iff some u≥0u\ge0u≥0 has MTu≤0M^{\mathsf T}u\le0MTu≤0, uTq>0u^{\mathsf T}q>0uTq>0.
  3. Theorem 2 (p. 4): if some ZsZ_sZs​ contains precisely one ray of a non-degenerate ZZZ, the number of equilibrium points is odd.
  4. p. 6: for kkk chosen as in (12), the starting ray E0∗E_0^*E0∗​ (z=0z=0z=0) is a ray of Z∗∗Z^{**}Z∗∗ inside Z0∗∗Z_0^{**}Z0∗∗​, and it is the only one.
  5. Lemma 2 (p. 6): Z∗∗Z^{**}Z∗∗ has an equilibrium point.
  6. Lemma 3 (p. 7): a ray (zˉ+θu,zˉ0+θu0)(\bar z+\theta u,\bar z_0+\theta u_0)(zˉ+θu,zˉ0​+θu0​) of Z∗Z^*Z∗ in Z0∗Z_0^*Z0∗​ with eTu=1e^{\mathsf T}u=1eTu=1 satisfies uTMu+u0=0u^{\mathsf T}Mu+u_0=0uTMu+u0​=0.
  7. (22)–(25) (pp. 7–8): under (i)–(ii) such a ray has zˉ0=0\bar z_0=0zˉ0​=0 and zˉ\bar zzˉ an equilibrium point of ZZZ, or zˉ0>0\bar z_0>0zˉ0​>0 and Z=∅Z=\emptysetZ=∅.

Companion results

The Corollary (p. 5: q=eq=eq=e, M>0M>0M>0 gives an odd number of equilibrium points), Theorem 3 (p. 5: if zTMz≥0z^{\mathsf T}Mz\ge0zTMz≥0 for all zzz, at most one equilibrium point) and Theorem 5 (p. 8: if moreover Z≠∅Z\ne\emptysetZ=∅, ZZZ has at least nnn rays).

Significance

Theorem 4 is an existence theorem for a class of linear complementarity problems that contains convex quadratic programming (the case zTMz≥0z^{\mathsf T}Mz\ge0zTMz≥0 for all zzz) and strictly positive matrices, and its proof is an algorithm: the path followed is the sequence of pivots of Lemke's method. The parity statements (Theorem 2, the Corollary) are an early instance of the path-following parity arguments later abstracted in the complexity class PPAD (Papadimitriou 1994).

The theorem is classical and proved. To our knowledge neither Lemke's theorem nor the polyhedral path machinery of Theorems 1–2 has a machine-checked proof in Lean or Mathlib. This mission produces the vocabulary of non-degenerate complementarity polyhedra (the matrix N(z)N(z)N(z), open edges, rays, adjacency paths), the parity theorem, and the existence theorem itself; the Farkas-type Lemma 2 (p. 4) is already available on the platform.

Difficulty

The algebraic part (Lemma 3 and (22)–(25)) is short. The difficulty is combinatorial-geometric: Theorem 1 requires that, under non-degeneracy, every extreme point of ZZZ has exactly nnn incident edges, that each point of ZsZ_sZs​ is an extreme point or lies on an open edge, and that the edges contained in ZsZ_sZs​ form paths whose end-points are exactly the equilibrium points; the parity of Theorem 2 then needs finiteness of the set of extreme points. None of this polyhedral structure (edges as zero-pattern classes, their closures and end-points) is in Mathlib. Moreover, the printed proof of Theorem 4 applies Theorem 2 to Z∗∗Z^{**}Z∗∗, whose non-degeneracy the paper obtains from an informal perturbation remark, and it sketches the step from the end of the augmented path to a ray of Z∗Z^*Z∗; a complete proof of the goal must supply these steps or take another route.

Formalization scope

  • Vectors are functions from a finite index type ι\iotaι to R\mathbb RR (n=∣ι∣n=|\iota|n=∣ι∣; Fin n\mathrm{Fin}\,nFinn is a special case); the bordered system lives on ι⊕{∗}\iota\oplus\{\ast\}ι⊕{∗}. www is always Mz−qMz-qMz−q, never a free variable.
  • N(z)N(z)N(z) is the indexed family of its columns, rank⁡N(z)\operatorname{rank}N(z)rankN(z) the dimension of their span; "rank⁡N(z)=n−1\operatorname{rank}N(z)=n-1rankN(z)=n−1" is written rank⁡N(z)+1=n\operatorname{rank}N(z)+1=nrankN(z)+1=n.
  • Non-degeneracy (Def. 3) is linear independence of the columns of N(z)N(z)N(z) for every z∈Rnz\in\mathbb R^nz∈Rn. The phrase "(but not all)" is read as excluding only the empty matrix; the other reading differs only when q=0q=0q=0, and the formal definition is then the stronger hypothesis.
  • An open edge is the set of points of ZZZ with the zero pattern of a point z0z_0z0​ with rank⁡N(z0)=n−1\operatorname{rank}N(z_0)=n-1rankN(z0​)=n−1; closed edges are closures; end-points of an open edge EEE are the points of E‾∖E\overline E\setminus EE∖E; a ray has exactly one. Adjacency paths are classes of closed edges.
  • Conditions (i)–(ii) are quantified over u≥0u\ge0u≥0 only. "Non-negative definite" (Theorems 3, 5) is zTMz≥0z^{\mathsf T}Mz\ge0zTMz≥0 for all zzz, with no symmetry assumed.
  • "An odd number" is finiteness plus odd cardinality; "at least nnn rays" counts rays as an extended natural number; "at most one" is a subsingleton.
  • Equilibrium points include membership in ZZZ: zT(Mz−q)=0z^{\mathsf T}(Mz-q)=0zT(Mz−q)=0 alone holds at z=0z=0z=0 and would make the goal trivial. The goal mentions neither Z∗Z^*Z∗, Z∗∗Z^{**}Z∗∗, kkk, rays nor paths; in particular non-degeneracy of Z∗∗Z^{**}Z∗∗ is not a hypothesis of the goal.
  • Deviations: Lemma 2 (p. 6) assumes non-degeneracy of Z∗∗Z^{**}Z∗∗ (Def. 3 does not define it for the non-square Z∗Z^*Z∗); milestone 4 holds for every kkk without non-degeneracy; Lemma 3 and (22)–(25) parametrize the ray as in (15) without requiring zˉ∗\bar z^*zˉ∗ to be an extreme point, which makes them stronger.

Contributions welcome: the polyhedral lemmas for non-degenerate ZZZ (extreme points have nnn edges, finitely many extreme points), the parity argument, and any complete route to Theorem 4.

Selected references

  • C. E. Lemke, Bimatrix Equilibrium Points and Mathematical Programming, Management Science 11(7):681–689, 1965. https://doi.org/10.1287/mnsc.11.7.681 (cited from the HAL deposit hal-01885823v1)
  • C. E. Lemke, J. T. Howson, Equilibrium Points of Bimatrix Games, J. SIAM 12(2):413–423, 1964. https://doi.org/10.1137/0112033
  • R. W. Cottle, J.-S. Pang, R. E. Stone, The Linear Complementarity Problem, Academic Press 1992; SIAM Classics 2009. https://doi.org/10.1137/1.9780898719000
  • C. H. Papadimitriou, On the complexity of the parity argument and other inefficient proofs of existence, J. Computer and System Sciences 48(3):498–532, 1994. https://doi.org/10.1016/S0022-0000(05)80063-7
10 thms3 active usersReviewed
🏆Completed
Operations ResearchOptimization·Captain: mikedeng1

On the Relation Between Integer and Noninteger Solutions to Linear Programs: Deep in the Optimal-Basis Cone, z₂(b) = z₁(b) + φ(b) and the Group Problem Gives an Integer OptimumResearch Paper

Motivation

An integer program is a linear program whose variables must take integer values. Its linear relaxation can be solved efficiently, while the integer program itself is NP-hard in general, so a recurring question in integer programming is how much the optimal solution of the relaxation says about the integer optimum. Simple rounding of the fractional optimum does not answer it: examples show that the integer optimum need not lie within distance one of the LP optimum, even when the right-hand side is large.

R. E. Gomory's 1965 note in the Proceedings of the National Academy of Sciences (doi:10.1073/pnas.53.2.260) showed that, for right-hand sides lying deep inside the cone of an optimal LP basis, the integer program is solved exactly by the LP solution plus a correction computed in a finite abelian group. The resulting group relaxation (the "corner polyhedron" of Gomory's later work, Gomory 1969) underlies a large body of work on cutting planes, the master group problem, Gomory–Johnson functions and asymptotic integer programming.

Timeline. Gomory's fractional cutting-plane algorithm (Gomory 1958) introduced the "fractional rows" of the simplex tableau; the 1965 note identifies the group they generate with M(I)/M(B)M(I)/M(B)M(I)/M(B) and proves the asymptotic theorem formalized here. Gomory 1969 develops the corner polyhedron; Gomory and Johnson (1972) study the continuous valid functions of the corner polyhedron.

Setting

Let AAA be an integer m×(m+n)m\times(m+n)m×(m+n) matrix of the form (A′,I)(A',I)(A′,I), let b∈Zmb\in\mathbb Z^mb∈Zm and c∈Rm+nc\in\mathbb R^{m+n}c∈Rm+n. Problem P1 is

max⁡ z1=cxs.t.Ax=b, x≥0,\max\ z_1=cx\quad\text{s.t.}\quad Ax=b,\ x\ge0,max z1​=cxs.t.Ax=b, x≥0,

and P2 is P1 with xxx integer. A basis is a nonsingular m×mm\times mm×m submatrix BBB of AAA; after rearranging, A=(B,N)A=(B,N)A=(B,N) with B=(α1,…,αm)B=(\alpha_1,\dots,\alpha_m)B=(α1​,…,αm​) and N=(αm+1,…,αm+n)N=(\alpha_{m+1},\dots,\alpha_{m+n})N=(αm+1​,…,αm+n​), and c=(cB,cN)c=(c_B,c_N)c=(cB​,cN​). The relative cost of a nonbasic column is ci∗=ci−cBB−1αic^*_i=c_i-c_BB^{-1}\alpha_ici∗​=ci​−cB​B−1αi​, and BBB is an optimal basis when all ci∗≤0c^*_i\le0ci∗​≤0. The cone of the basis is KB={β∈Rm:B−1β≥0}K^B=\{\beta\in\mathbb R^m: B^{-1}\beta\ge0\}KB={β∈Rm:B−1β≥0}; removing from it all points within Euclidean distance ddd of its boundary gives the reduced cone KB(d)K^B(d)KB(d).

Let M(I)=ZmM(I)=\mathbb Z^mM(I)=Zm and M(B)=LBM(B)=\mathfrak L_BM(B)=LB​ the lattice of integer combinations of α1,…,αm\alpha_1,\dots,\alpha_mα1​,…,αm​. The factor module M(I)/M(B)M(I)/M(B)M(I)/M(B) is a finite abelian group with D=∣det⁡B∣D=|\det B|D=∣detB∣ elements; write αˉi\bar\alpha_iαˉi​, bˉ\bar bbˉ for the classes of αi\alpha_iαi​, bbb. The group problem (4) is

max⁡ ∑i=1nci+m∗yis.t.∑i=1nαˉi+myi=bˉ,yi≥0 integer,\max\ \sum_{i=1}^n c^*_{i+m}y_i\quad\text{s.t.}\quad\sum_{i=1}^n\bar\alpha_{i+m}y_i=\bar b,\quad y_i\ge0\text{ integer},max i=1∑n​ci+m∗​yi​s.t.i=1∑n​αˉi+m​yi​=bˉ,yi​≥0 integer,

with optimal value φB(b)\varphi^B(b)φB(b). Finally l=max⁡i>m∥αi∥l=\max_{i>m}\|\alpha_i\|l=maxi>m​∥αi​∥.

Formalization targets

Goal: THEOREM 1 (p. 261)

If b∈KB(l(D−1))b\in K^B(l(D-1))b∈KB(l(D−1)), then P1, P2 and (4) attain their optima, the value z2(b)z_2(b)z2​(b) of P2 is

z2(b)=z1(b)+φB(b),z_2(b)=z_1(b)+\varphi^B(b),z2​(b)=z1​(b)+φB(b),

an optimal solution of P2 is

x(b)=(B−1(b−NyB(b)), yB(b))x(b)=\big(B^{-1}(b-Ny^B(b)),\ y^B(b)\big)x(b)=(B−1(b−NyB(b)), yB(b))

for an optimal solution yB(b)y^B(b)yB(b) of (4) with ∑iyiB(b)≤D−1\sum_iy^B_i(b)\le D-1∑i​yiB​(b)≤D−1, and φB\varphi^BφB, yBy^ByB are mmm-periodic: φB(b+αi)=φB(b)\varphi^B(b+\alpha_i)=\varphi^B(b)φB(b+αi​)=φB(b).

Milestones

In the order of the paper's proof: ∣M(I)/M(B)∣=D|M(I)/M(B)|=D∣M(I)/M(B)∣=D (p. 262); periodicity of (4) (p. 263); the cost identity (5)–(6); "cBB−1bc_BB^{-1}bcB​B−1b is z1(b)z_1(b)z1​(b)", the simplex optimality criterion (a published, proved theorem); the projection of a solution of P2 to a solution of (4); the bound (7), z2(b)≤z1(b)+φ(b)z_2(b)\le z_1(b)+\varphi(b)z2​(b)≤z1​(b)+φ(b); the extension of a solution of (4) to an integer solution; the LEMMA (an optimal solution of (4) with ∑yi≤D−1\sum y_i\le D-1∑yi​≤D−1); and the bound ∥Ny∥≤(D−1)l\|Ny\|\le(D-1)l∥Ny∥≤(D−1)l that places b−Nyb-Nyb−Ny in KBK^BKB.

Companions

THEOREM 2: the optimal x(b)x(b)x(b) satisfies ∑i>mxi=∑iyi≤D−1\sum_{i>m}x_i=\sum_iy_i\le D-1∑i>m​xi​=∑i​yi​≤D−1. THEOREM 4 (with the bound corrected, see below): if an optimal yyy of (4) has ∏i(yi+1)>D\prod_i(y_i+1)>D∏i​(yi​+1)>D, the optimum of (4) is not unique.

Significance

THEOREM 1 reduces integer programming, for right-hand sides far from the boundary of an optimal cone, to a shortest-path-type problem over a group of order DDD: the integer optimum is determined by the LP optimum and a periodic correction, and it can be computed in time polynomial in nnn and DDD (THEOREM 3, not formalized here). It explains why rounding fails while a structured correction succeeds, and it is the starting point of the corner-polyhedron and cutting-plane theory built on the group relaxation. THEOREM 2 bounds how far the integer optimum lies from a lattice point of LB\mathfrak L_BLB​, an early proximity result.

The results are classical and proved in the paper; none of them has a machine-checked proof that this mission is aware of. The mission produces a Lean statement and proof of the asymptotic theorem, together with reusable statements about the group Zm/BZm\mathbb Z^m/B\mathbb Z^mZm/BZm and the zero-sum shortening argument of the LEMMA.

Difficulty

The decomposition z2≤z1+φz_2\le z_1+\varphiz2​≤z1​+φ is a direct computation. The difficulty is the converse: an optimal solution yyy of the group problem extends to an integer point xB=B−1(b−Ny)x_B=B^{-1}(b-Ny)xB​=B−1(b−Ny), but nothing in (4) forces xB≥0x_B\ge0xB​≥0. Optimality of yyy alone does not bound yyy: an optimal solution may carry zero-cost cycles of arbitrary length, and then b−Nyb-Nyb−Ny can leave the cone KBK^BKB. The theorem therefore depends on choosing the right optimal solution of (4), one whose total size is bounded by the group order, and on relating that combinatorial bound to the Euclidean geometry of the cone.

Formalization scope

  • The basis is fixed: BBB is an integer m×mm\times mm×m matrix and NNN an integer m×nm\times nm×n matrix; column jjj of NNN (000-based) is the paper's αm+1+j\alpha_{m+1+j}αm+1+j​. The costs cBc_BcB​, cNc_NcN​ are real.
  • The standing assumptions of pp. 260–262 are hypotheses of the goal: det⁡B≠0\det B\neq0detB=0; every unit vector of Zm\mathbb Z^mZm is a column of BBB or of NNN (the paper's A=(A′,I)A=(A',I)A=(A′,I), without which P2 can be infeasible); all relative costs ci∗≤0c^*_i\le0ci∗​≤0 (BBB is optimal, without which (4) is unbounded).
  • P2's variables are natural numbers (integer and nonnegative). The group problem is written as "b−Ny∈BZmb-Ny\in B\mathbb Z^mb−Ny∈BZm", which is equivalent to ∑αˉi+myi=bˉ\sum\bar\alpha_{i+m}y_i=\bar b∑αˉi+m​yi​=bˉ.
  • Optimal values are greatest elements of the sets of objective values, never suprema; the goal asserts that they are attained.
  • The norm in lll and in KB(d)K^B(d)KB(d) is Euclidean. KB(d)K^B(d)KB(d) is the set of points whose closed ball of radius ddd lies in KBK^BKB. l=0l=0l=0 when n=0n=0n=0.
  • In part (3) of the goal, yB(b)y^B(b)yB(b) ranges over optimal solutions of (4) with ∑iyi≤D−1\sum_iy_i\le D-1∑i​yi​≤D−1, the solutions the paper's LEMMA and dynamic program produce.
  • THEOREM 4 is stated with the hypothesis ∏(yi+1)>D\prod(y_i+1)>D∏(yi​+1)>D: the printed >D−1>D-1>D−1 is false (m=n=1m=n=1m=n=1, B=(2)B=(2)B=(2), N=(1)N=(1)N=(1), c=(2,0)c=(2,0)c=(2,0), b=1b=1b=1 has the unique optimum y=1y=1y=1 with ∏(yi+1)=2\prod(y_i+1)=2∏(yi​+1)=2).
  • A trivializing formalization is ruled out: the goal assumes none of the LEMMA, the inequality (7), integrality of B−1(b−Ny)B^{-1}(b-Ny)B−1(b−Ny) or the norm bound, and it states the existence of the optima instead of assuming them.
  • Not in scope: the operation counts of THEOREM 3 and the recursion (8), which need a machine model the paper does not define.

Needed infrastructure: the cardinality of Zm/BZm\mathbb Z^m/B\mathbb Z^mZm/BZm (available in Mathlib through determinants of basis changes), a zero-sum/pigeonhole argument in a finite abelian group, the simplex optimality criterion (published), and elementary Euclidean estimates. The group-theoretic lemmas are reusable for other group-relaxation and proximity missions. Proofs of the milestones, alternative arguments and the extension to the open-ball reading of KB(d)K^B(d)KB(d) are welcome.

Selected references

  • R. E. Gomory, On the relation between integer and noninteger solutions to linear programs, Proc. Natl. Acad. Sci. USA 53(2):260–265, 1965. https://doi.org/10.1073/pnas.53.2.260
  • R. E. Gomory, Outline of an algorithm for integer solutions to linear programs, Bull. Amer. Math. Soc. 64:275–278, 1958. https://doi.org/10.1090/S0002-9904-1958-10224-4
  • R. E. Gomory, Some polyhedra related to combinatorial problems, Linear Algebra Appl. 2:451–558, 1969. https://doi.org/10.1016/0024-3795(69)90017-2
  • R. E. Gomory and E. L. Johnson, Some continuous functions related to corner polyhedra, Math. Programming 3:23–85, 1972.
  • J. Matoušek and B. Gärtner, Understanding and Using Linear Programming, Springer, 2007 (simplex optimality criterion). https://doi.org/10.1007/978-3-540-30717-4
11 thms3 active usersReviewed
🏆Completed
Graph TheoryOperations Research·Captain: mikedeng1

A Suggested Computation for Maximal Multi-Commodity Network Flows: With Nonnegative Simplex Multipliers, Shortest-Chain Labels Prove the Arc-Chain Basis Optimal or Find an Entering ChainResearch Paper

Motivation

The maximal multi-commodity flow problem asks how much total flow several commodities, each travelling from its own sources to its own sinks, can send through a network whose arcs have shared capacities. It arises in communication and transportation planning: Kalaba and Juncosa's 1956 study of communication networks (Management Science 3(1)) leads to linear programs of this form. For a single commodity the problem has a clean combinatorial theory: the max-flow min-cut theorem of Ford and Fulkerson (1956, doi:10.4153/CJM-1956-045-5) and the labeling algorithm. With several commodities both fail: the min-cut equality is false, and simplex bases are no longer triangular.

In a short note received in October 1957 and published in Management Science in 1958, L. R. Ford Jr. and D. R. Fulkerson proposed a way to solve the multi-commodity problem with the simplex method anyway (doi:10.1287/mnsc.1040.0269, reprinted 2004). They use the arc-chain formulation, which has one variable per chain from a source to a sink of a commodity. That formulation has far too many variables to write down. The note's point is that the simplex method never needs them explicitly. The "pricing" step (finding a variable to enter the basis, or recognizing that none exists) can be carried out by one shortest-chain computation per commodity, with the simplex multipliers as arc lengths.

Timeline. Ford (1956, RAND P-923) gave the label-correcting shortest-chain procedure the note uses. Ford and Fulkerson (1958) applied it to price the columns of the arc-chain program. Dantzig and Wolfe (1960) generalized the idea into the decomposition principle. Gilmore and Gomory (1961) applied the same pattern to the cutting-stock problem, and it is now called column generation.

Setting

A network has finitely many nodes P1,…,PNP_1,\dots,P_NP1​,…,PN​ and arcs A1,…,AmA_1,\dots,A_mA1​,…,Am​. Each arc ArA_rAr​ joins two nodes, has a capacity brb_rbr​, and is either directed (traversable one way only) or undirected (traversable both ways). There are finitely many commodities; commodity kkk has a set of sources SkS_kSk​ and a set of sinks TkT_kTk​.

A chain from uuu to www is the arc set of a path u=v0,v1,…,vp=wu = v_0, v_1, \dots, v_p = wu=v0​,v1​,…,vp​=w with distinct nodes, each arc traversable from vi−1v_{i-1}vi−1​ to viv_ivi​. A commodity chain of kkk is a chain from a node of SkS_kSk​ to a node of TkT_kTk​. List the commodity chains, for all commodities, as C1,…,CnC_1,\dots,C_nC1​,…,Cn​, and let A=(ars)A = (a_{rs})A=(ars​) be the incidence matrix: ars=1a_{rs} = 1ars​=1 if CsC_sCs​ contains ArA_rAr​ and 000 otherwise. With xsx_sxs​ the flow along CsC_sCs​ and xn+rx_{n+r}xn+r​ the slack of arc ArA_rAr​, the problem is the linear program

maximize ∑s=1nxssubject to∑s=1narsxs+xn+r=br,x1,…,xn+m≥0.(2–3)\text{maximize } \sum_{s=1}^{n} x_s \quad\text{subject to}\quad \sum_{s=1}^{n} a_{rs}x_s + x_{n+r} = b_r,\qquad x_1,\dots,x_{n+m} \ge 0. \tag{2–3}maximize s=1∑n​xs​subject tos=1∑n​ars​xs​+xn+r​=br​,x1​,…,xn+m​≥0.(2–3)

A basis is a set of mmm columns of [A∣I][A \mid I][A∣I] forming an invertible matrix B=(brj)B = (b_{rj})B=(brj​). Its simplex multipliers α1,…,αm\alpha_1,\dots,\alpha_mα1​,…,αm​ satisfy, for every basic column jjj,

∑r=1mαrbrj={1j≤n,0j>n.(4)\sum_{r=1}^{m} \alpha_r b_{rj} = \begin{cases} 1 & j \le n,\\ 0 & j > n.\end{cases} \tag{4}r=1∑m​αr​brj​={10​j≤n,j>n.​(4)

Read as arc lengths, the αr\alpha_rαr​ give each chain CsC_sCs​ the length ∑rαrars\sum_r \alpha_r a_{rs}∑r​αr​ars​. The labeling process for a source set SSS starts with labels πi=0\pi_i = 0πi​=0 on SSS and πi=∞\pi_i = \inftyπi​=∞ elsewhere. It repeatedly finds an arc traversable from PiP_iPi​ to PjP_jPj​ with πi+lij<πj\pi_i + l_{ij} < \pi_jπi​+lij​<πj​ and replaces πj\pi_jπj​ by πi+lij\pi_i + l_{ij}πi​+lij​. In Lean the labels are lab : V → WithTop ℝ, the multipliers are α : E → ℝ, and all objects live in the namespace FordFulkerson58.ArcChain.

Formalization targets

Goal: the shortest-chain pricing test (§3, pp. 1779–1780)

Let BBB be a basis with basic feasible solution zzz and multipliers α≥0\alpha \ge 0α≥0. Then:

(a) for every commodity, every run of the labeling process with lengths α is finite;(b) if all final sink labels are ≥1, then ∑sxs≤∑szs for every feasible x;(c) if a final sink label πt<1, some non-basic chain column to t has length πt and 1−∑rαrars>0.\begin{aligned} &\text{(a) for every commodity, every run of the labeling process with lengths } \alpha \text{ is finite;}\\ &\text{(b) if all final sink labels are } \ge 1, \text{ then } \textstyle\sum_s x_s \le \sum_s z_s \text{ for every feasible } x;\\ &\text{(c) if a final sink label } \pi_t < 1, \text{ some non-basic chain column to } t \text{ has length } \pi_t \text{ and } 1 - \textstyle\sum_r \alpha_r a_{rs} > 0. \end{aligned}​(a) for every commodity, every run of the labeling process with lengths α is finite;(b) if all final sink labels are ≥1, then ∑s​xs​≤∑s​zs​ for every feasible x;(c) if a final sink label πt​<1, some non-basic chain column to t has length πt​ and 1−∑r​αr​ars​>0.​

Milestones (§3, the labeling process and the test)

  1. Termination of the labeling process for non-negative lengths (p. 1780).
  2. Final labels are shortest-chain lengths from SSS; the smallest label on TTT is the SSS-to-TTT distance (p. 1780).
  3. The trace-back: a chain of tight arcs from SSS to every labeled node, of length its label (p. 1780).
  4. If α≥0\alpha \ge 0α≥0 and every commodity chain has length ≥1\ge 1≥1, the basis is optimal (p. 1779).

Further items state the remarks of §3–§4: a negative multiplier lets a slack enter; the slack basis is a feasible start; dominated chains can be ignored; limited supplies reduce to the same problem by adding new directed source arcs; Figure 2 is the incidence matrix of Figure 1; with negative lengths the labeling process may not terminate.

Significance

The result. The test turns a linear program with exponentially many columns into one whose pricing step costs one shortest-path computation per commodity. Only the m×mm \times mm×m basis is ever stored. This is the first instance of column generation, which now underlies solvers for multi-commodity flow, cutting stock, vehicle routing, crew scheduling and many other set-partitioning models, typically inside branch-and-price. Part (a) together with milestones 2–3 also gives the correctness of Ford's label-correcting algorithm with non-negative lengths, started from a source set and with any order of arc scans.

Formalizing it. The results are classical and proved in the paper's informal style. None of them is formalized: the platform has no arc-chain LP, no column pricing, and no formal statement of Ford's label-correcting process. The general LP facts are on the platform (the simplex optimality criterion and weak duality, both proved) and are included as references. This mission adds the network-specific layer: chains as arc sets of simple paths, the labeling process as a relation on labelings, and the link from final labels to reduced costs. One sentence of the paper is corrected. The trace-back "eventually" reaches SSS only along some backward path, because zero-length cycles can trap a naive backward search, and milestone 3 states the true version.

Difficulty

The obvious argument for termination says each step lowers a label and there are finitely many chains. It fails: labels are lengths of walks, not chains, and with zero-length arcs (multipliers are often 000) there are infinitely many walks. Termination needs the finiteness of the set of walk lengths below a bound, not positivity. The correctness of the final labels needs both properties of the labeling: it is produced by the process (so every label is the length of a walk from SSS), and it is terminal. A terminal labeling alone, such as all labels 000, is not a distance labeling. Finally, the optimality claim is an LP statement about the full, never-enumerated column set. It must be derived from (4), α≥0\alpha \ge 0α≥0 and the shortest-chain lengths, with the chain columns indexed abstractly.

Formalization scope

Nodes, arcs and commodities are finite types V, E, ι. Each arc has tail, head, a per-arc directed flag and a capacity b : E → ℝ; commodities have src snk : ι → Finset V. Parallel arcs and mixed directed/undirected networks are allowed. No sign of the capacities and no disjointness of sources and sinks is assumed, except 0 ≤ b in the zero-flow item, where it is disclosed. Chains are arc sets of simple paths. The null chain at a source is a chain, so sources have distance 000. Columns are pairs (commodity, chain), so one arc set used by two commodities gives two columns. The columns of [A∣I][A \mid I][A∣I] are indexed by Col N ⊕ E. A basis is β : E → Col N ⊕ E with IsUnit (basisMatrix N β).det; its basic feasible solution and multipliers (4) are hypotheses about this β. Labels are in WithTop ℝ with ⊤ for ∞\infty∞. Shortest-chain lengths are finite infima (Finset.inf), equal to ⊤ when there is no chain. The process scans arcs, not node pairs, which handles parallel arcs and the undirected case lij=ljil_{ij} = l_{ji}lij​=lji​.

The standing assumption of §3 ("a stage has been reached in the computation where all αr\alpha_rαr​ are non-negative", p. 1779) is a hypothesis of the goal and of milestones 1–4. Milestones 1–3 are stated for any non-negative length function. The paper has no numbered theorems; every milestone is an unnumbered sentence of §3, cited by page and position.

A trivializing formalization is ruled out: the goal does not assume shortest-chain lengths, the trace-back, or any reduced-cost fact beyond (4). Statements about final labels require labelings that are both reachable from the initial labels and terminal, and part (a) shows such labelings exist.

Reusable beyond this mission: the chain and labeling layer (a label-correcting shortest-path algorithm from a source set over mixed networks) and the arc-chain LP. Contributions to the LP layer, which can import the referenced Matoušek–Gärtner results after reindexing, are particularly welcome.

Selected references

  • L. R. Ford Jr., D. R. Fulkerson, A suggested computation for maximal multi-commodity network flows, Management Science 5(1):97–101, 1958; reprinted Management Science 50(12S):1778–1780, 2004. https://doi.org/10.1287/mnsc.1040.0269
  • L. R. Ford Jr., D. R. Fulkerson, Maximal flow through a network, Canadian Journal of Mathematics 8:399–404, 1956. https://doi.org/10.4153/CJM-1956-045-5
  • L. R. Ford Jr., Network Flow Theory, RAND Corporation Paper P-923, 1956. https://www.rand.org/pubs/papers/P923.html
  • G. B. Dantzig, P. Wolfe, Decomposition principle for linear programs, Operations Research 8(1):101–111, 1960. https://doi.org/10.1287/opre.8.1.101
  • P. C. Gilmore, R. E. Gomory, A linear programming approach to the cutting-stock problem, Operations Research 9(6):849–859, 1961. https://doi.org/10.1287/opre.9.6.849
  • J. Matoušek, B. Gärtner, Understanding and Using Linear Programming, Springer, 2007. https://doi.org/10.1007/978-3-540-30717-4
12 thms3 active usersReviewed
🏆Completed
CombinatoricsOperations ResearchOptimization·Captain: mikedeng1

A New Optimization Algorithm for the Vehicle Routing Problem with Time Windows: With No Negative Marginal Cost Column, the Set Covering LP Is Optimal and Bounds the VRPTW Optimum from BelowResearch Paper

Motivation

The vehicle routing problem with time windows (VRPTW) asks for minimum-cost routes from a central depot that serve every customer exactly once, respecting vehicle capacity and a time interval in which each customer's service must begin. It models school bus routing, parcel and retail distribution, and dial-a-ride services. Desrochers, Desrosiers and Solomon (Oper. Res. 40 (1992) 342–354) gave an optimization algorithm for the VRPTW that solved benchmark instances with up to 100 customers to optimality, a size far beyond earlier exact methods. Their method, which solves the linear relaxation of a set partitioning model by column generation with routes priced out by a resource-constrained shortest path dynamic program, is the template that later exact vehicle routing algorithms ("branch-and-price") follow.

The mathematical core of the method is a certificate: once the pricing subproblem reports that no route has negative marginal cost (reduced cost), the current linear programming solution is optimal over all routes the subproblem can generate, its value bounds every VRPTW solution from below, and an integral solution covering each customer exactly once is VRPTW-optimal. This mission formalizes that certificate and the lemmas of the paper it rests on.

Setting

The nodes are a depot ddd and customers N∖{d}N\setminus\{d\}N∖{d}. An arc set AAA carries, for each arc (i,j)(i,j)(i,j), a cost cijc_{ij}cij​ and a duration tijt_{ij}tij​; each node has a demand qiq_iqi​ and a time window [ai,bi][a_i,b_i][ai​,bi​]; vehicles have capacity QQQ.

A path (d,i1,…,iK,d)(d, i_1, \dots, i_K, d)(d,i1​,…,iK​,d) uses arcs of AAA and passes through the depot only at its ends. It is resource-feasible when there are service start times T0=0,T1,…,TK+1T_0 = 0, T_1, \dots, T_{K+1}T0​=0,T1​,…,TK+1​ with Tk+tikik+1≤Tk+1T_k + t_{i_k i_{k+1}} \le T_{k+1}Tk​+tik​ik+1​​≤Tk+1​ (waiting is allowed) and aik≤Tk≤bika_{i_k}\le T_k\le b_{i_k}aik​​≤Tk​≤bik​​, and its load ∑k=1Kqik\sum_{k=1}^{K} q_{i_k}∑k=1K​qik​​ is at most QQQ. Its cost is cr=∑k=0Kcikik+1c_r = \sum_{k=0}^{K} c_{i_k i_{k+1}}cr​=∑k=0K​cik​ik+1​​, and γir\gamma_{ir}γir​ is the number of visits of rrr to customer iii. The paper uses three solution spaces:

  1. feasible routes RRR: resource-feasible paths visiting each customer at most once;
  2. second-model paths: resource-feasible paths where customers may repeat (the state-space relaxation that tracks load and time but not the set of visited customers);
  3. third-model paths: second-model paths with no 2-cycle (i,j,i)(i, j, i)(i,j,i).

A VRPTW solution is a set of feasible routes covering each customer exactly once; its cost is the sum of the route costs.

The set covering type model (Sec. 4) over a column set R\mathcal RR is

min⁡∑rcrxrs.t.∑rγirxr≥1 (i≠d),∑rxr−Xd=0,∑rcrxr−Xc=0,\min \sum_r c_r x_r \quad\text{s.t.}\quad \sum_r \gamma_{ir}x_r \ge 1\ (i \ne d),\quad \sum_r x_r - X_d = 0,\quad \sum_r c_r x_r - X_c = 0,minr∑​cr​xr​s.t.r∑​γir​xr​≥1 (i=d),r∑​xr​−Xd​=0,r∑​cr​xr​−Xc​=0,

with xr∈{0,1}x_r\in\{0,1\}xr​∈{0,1} and Xd,Xc≥0X_d, X_c\ge 0Xd​,Xc​≥0 integer. Its LP relaxation replaces xr∈{0,1}x_r\in\{0,1\}xr​∈{0,1} by xr≥0x_r\ge 0xr​≥0 and drops integrality. For dual values πi\pi_iπi​, πd\pi_dπd​, πc\pi_cπc​ of the three rows, the marginal cost of a column and of an arc are

cˉr=cr−∑i≠dπiγir−πd−πccr,cˉij=(1−πc)cij−πi(πi:=πd at i=d).\bar c_r = c_r - \sum_{i\ne d}\pi_i\gamma_{ir} - \pi_d - \pi_c c_r,\qquad \bar c_{ij} = (1-\pi_c)c_{ij} - \pi_i\quad(\pi_i := \pi_d \text{ at } i = d).cˉr​=cr​−i=d∑​πi​γir​−πd​−πc​cr​,cˉij​=(1−πc​)cij​−πi​(πi​:=πd​ at i=d).

Formalization targets

Goal: the column generation certificate

Let R0R_0R0​ be a finite set of third-model paths, zˉ=(xˉ,Xˉd,Xˉc)\bar z = (\bar x, \bar X_d, \bar X_c)zˉ=(xˉ,Xˉd​,Xˉc​) an optimal solution of the LP relaxation restricted to R0R_0R0​, and (π,πd,πc)(\pi,\pi_d,\pi_c)(π,πd​,πc​) an optimal solution of its dual. If every third-model path has nonnegative marginal cost, ∑k=0Kcˉikik+1≥0\sum_{k=0}^{K}\bar c_{i_k i_{k+1}} \ge 0∑k=0K​cˉik​ik+1​​≥0, then

zˉ is optimal for the LP relaxation over all third-model paths,\bar z \text{ is optimal for the LP relaxation over all third-model paths},zˉ is optimal for the LP relaxation over all third-model paths,

and, when arc costs are nonnegative, ∑rcrxˉr≤cost(S)\sum_r c_r \bar x_r \le \text{cost}(S)∑r​cr​xˉr​≤cost(S) for every VRPTW solution SSS; if moreover xˉ\bar xxˉ is integral and covers each customer exactly once, its support is an optimal VRPTW solution.

Milestones

  1. Feasible routes ⊆\subseteq⊆ third-model paths ⊆\subseteq⊆ second-model paths (Sec. 3, p. 346).
  2. cˉr=∑k=0Kcˉikik+1\bar c_r = \sum_{k=0}^{K}\bar c_{i_k i_{k+1}}cˉr​=∑k=0K​cˉik​ik+1​​ for every path (Sec. 4.1, p. 347).
  3. Nonnegative column marginal costs over all third-model paths make the restricted optimum optimal (Sec. 4, p. 346).
  4. Every VRPTW solution is a feasible LP point of equal cost (Sec. 2, p. 344).
  5. An integral, exactly covering LP point is a VRPTW solution of equal cost (Sec. 5, p. 348).
  6. Under the strict triangle inequality, LP optima over routes do not overcover (Sec. 5, p. 348).
  7. The four time window reduction conditions preserve the set of paths (Sec. 6.1, p. 349).
  8. The Figure 1 example has 11, 22 and 12 solutions in the three models (pp. 345–346).

Significance

The certificate is the correctness statement of column generation for vehicle routing: it says when the algorithm may stop, what the value it stops with means, and when the branch-and-bound tree can be skipped. The same argument, with a different subproblem, underlies branch-and-price for crew scheduling, cutting stock and many other set partitioning formulations. Milestone 2 is the observation that turns pricing into a shortest path problem on the original network, and milestone 7 is the preprocessing used before every dynamic program over time windows.

The paper's results are classical and their proofs are short in prose. None of them, nor any VRPTW column generation statement, has a machine-checked proof on the platform; the closest existing items concern split deliveries (Desaulniers 2010) or general finite LP duality. The mission produces a reusable formal model of resource-constrained paths, the set covering LP and its restricted dual, on which later branch-and-price papers can build.

Difficulty

The goal combines three ingredients that do not fit together automatically. First, LP duality for the restricted problem: the hypothesis gives an optimal dual, but optimality over all columns needs equality of the restricted primal and dual values, which is strong duality for a finite LP with equality rows and sign-constrained auxiliary variables. Second, the column set of the full LP is infinite in general: a nonelementary path can repeat customers, so the comparison must go through the finitely many columns a given feasible point uses. Third, the pricing hypothesis is about arc sums along node sequences while the LP is about column costs and visit counts; matching them needs the depot to appear exactly at the two ends of a path.

The tempting shortcut of quantifying the pricing hypothesis over the current columns only gives dual feasibility for the restricted LP, which says nothing about columns not yet generated. The lower bound also fails for arbitrary real costs, because row (4) with Xc≥0X_c \ge 0Xc​≥0 excludes negative-cost solutions from the LP while the VRPTW still contains them.

Formalization scope

Nodes are Fin (n+1) with the depot 0. A path is the list of its customers; its node sequence is 0 :: p ++ [0], so a path visits at least one customer and meets the depot only at its ends. The arc set is an arbitrary relation; costs, durations, demands and windows are arbitrary reals, and every sign or triangle condition a statement needs appears in its own hypotheses. The committed readings:

  • service times are indexed by position (a second-model path may visit a node twice), with T0=0T_0 = 0T0​=0;
  • the depot's window also applies at the return (0≤k≤K+10 \le k \le K+10≤k≤K+1, where the page prints 0≤k≤K0\le k\le K0≤k≤K);
  • the capacity constraint is load ≤Q\le Q≤Q (the page says "less than"; the recurrences and Figure 1 use ≤\le≤);
  • a 2-cycle is a pattern of the customer list, so (d,i,d)(d, i, d)(d,i,d) is not one;
  • the LP relaxation keeps Xd,Xc≥0X_d, X_c \ge 0Xd​,Xc​≥0, relaxes xr∈{0,1}x_r\in\{0,1\}xr​∈{0,1} to xr≥0x_r\ge 0xr​≥0 with no upper bound, and is the root LP without branching rows; its dual is that of the restricted LP, with πi,πd,πc≥0\pi_i, \pi_d, \pi_c \ge 0πi​,πd​,πc​≥0;
  • LP points are finitely supported functions on lists (Finsupp); VRPTW solutions are Finsets of routes;
  • clauses 2–3 of the goal and milestone 4 assume nonnegative arc costs (the paper's costs are distances);
  • milestone 6 adds complete arcs, positive costs, a triangle inequality on durations and nonnegative demands; milestone 7 applies the conditions at customers only; milestone 8 lists (d,2,1,2,1,d)(d,2,1,2,1,d)(d,2,1,2,1,d) where the page prints (d,2,1,2,d)(d,2,1,2,d)(d,2,1,2,d) twice.

The goal is not to be trivialized: the pricing hypothesis ranges over all third-model paths, not the current columns; equality of primal and dual values is not assumed; the full column set is not R0R_0R0​; and "covered exactly once" is not presupposed to mean elementary columns, which must be derived from integrality.

Contributions welcome: proofs of the milestones, a reusable finite LP strong duality interface for Finsupp-indexed columns, and lemmas about arc sums over node sequences.

Selected references

  • M. Desrochers, J. Desrosiers, M. Solomon, A new optimization algorithm for the vehicle routing problem with time windows, Operations Research 40(2), 1992, 342–354. https://doi.org/10.1287/opre.40.2.342
  • N. Christofides, A. Mingozzi, P. Toth, State-space relaxation procedures for the computation of bounds to routing problems, Networks 11(2), 1981, 145–164. https://doi.org/10.1002/net.3230110207
  • D. J. Houck, J.-C. Picard, M. Queyranne, R. R. Vemuganti, The travelling salesman problem as a constrained shortest path problem: theory and computational experience, Opsearch 17, 1980, 93–109.
  • G. Desaulniers, Branch-and-price-and-cut for the split-delivery vehicle routing problem with time windows, Operations Research 58(1), 2010, 179–192. https://doi.org/10.1287/opre.1090.0713
13 thms3 active usersReviewed
🏆Completed
Graph TheoryOperations ResearchOptimization·Captain: mikedeng1

Finding Minimum-Cost Circulations by Successive Approximation II: For Real Costs, the Strongly Polynomial Loop Reaches a Minimum-Cost Circulation After at Most m⌈log₂(2n)⌉ RefinementsResearch Paper

Why minimum-cost circulation needs a real-cost iteration bound

A minimum-cost circulation is a way to send flow around a directed network while respecting capacities and paying a cost per unit on each arc. It is a basic form of network optimization: a minimum-cost flow with prescribed supplies and demands can be reduced to a circulation problem, and the circulation formulation exposes the residual arcs on which a flow can be improved. Goldberg and Tarjan's successive-approximation method relaxes exact optimality by an error parameter and repeatedly reduces that error. Its ordinary stopping rule for integer costs depends on the largest cost. For arbitrary real costs, including irrational costs, a decreasing positive error need never cross a fixed numerical cutoff. The strongly polynomial loop in their 1987 technical report, later published in Mathematics of Operations Research instead tests the smallest error parameter appropriate to the current circulation and stops when it is zero.

This mission formalizes the number of refinement calls made by that loop. It builds on the related Goldberg–Tarjan 1988 max-flow and 1989 minimum-mean cycle-canceling series: those develop different algorithms for network flow, while the published 1989 circulation and approximate-optimality definitions provide this mission's common network model. The result here concerns successive approximation and its fixed-arc progress measure.

Network and approximate optimality

The vertex set VVV is finite, with n=∣V∣n=|V|n=∣V∣, and EEE is a symmetric set of directed arcs, with m=∣E∣m=|E|m=∣E∣. Symmetry means that (v,w)∈E(v,w)\in E(v,w)∈E brings (w,v)∈E(w,v)\in E(w,v)∈E too. Each arc has a real capacity u(v,w)u(v,w)u(v,w) and a real cost c(v,w)c(v,w)c(v,w); costs obey c(v,w)=−c(w,v)c(v,w)=-c(w,v)c(v,w)=−c(w,v). A circulation fff obeys f(v,w)≤u(v,w)f(v,w)\le u(v,w)f(v,w)≤u(v,w), f(v,w)=−f(w,v)f(v,w)=-f(w,v)f(v,w)=−f(w,v), and conservation of flow at every vertex. Its total cost is half the sum of c(v,w)f(v,w)c(v,w)f(v,w)c(v,w)f(v,w) over directed arcs, since each unordered edge appears twice. A minimum-cost circulation costs no more than any other circulation of the network.

The residual capacity is uf(v,w)=u(v,w)−f(v,w)u_f(v,w)=u(v,w)-f(v,w)uf​(v,w)=u(v,w)−f(v,w). For a price function p:V→Rp:V\to\mathbb Rp:V→R, the report writes the reduced cost as cp(v,w)=c(v,w)−p(v)+p(w)c_p(v,w)=c(v,w)-p(v)+p(w)cp​(v,w)=c(v,w)−p(v)+p(w). A circulation is ε\varepsilonε-optimal with respect to ppp when every residual arc has reduced cost at least −ε-\varepsilon−ε. It is ε\varepsilonε-optimal if some price function works, where ε≥0\varepsilon\ge0ε≥0. The tight error ε(f)\varepsilon(f)ε(f) is the least such error. A circulation is ε\varepsilonε-tight when it is ε\varepsilonε-optimal but fails to be ε′\varepsilon'ε′-optimal at every ε′<ε\varepsilon'<\varepsilonε′<ε. This includes attainment at ε\varepsilonε.

An arc is ε\varepsilonε-fixed if all ε\varepsilonε-optimal circulations carry the same flow through it. Write FεF_\varepsilonFε​ for the set of these arcs. Theorem 4.2 supplies an arc-fixing criterion from a large absolute reduced cost. Lemma 4.4 says that when a positive tight error falls by a factor of 2n2n2n, FεF_\varepsilonFε​ becomes strictly larger. Both are numbered results on printed pages 16–17 of the source report's journal counterpart.

Formalization targets

The goal is the circulation-level form of Theorem 4.5. Begin with a circulation f0f_0f0​ and a sequence f0,f1,…f_0,f_1,\ldotsf0​,f1​,… whose positive-error steps satisfy the contract of Figure 3: if ε(fk)>0\varepsilon(f_k)>0ε(fk​)>0, the next circulation is ε(fk)/2\varepsilon(f_k)/2ε(fk​)/2-optimal. With the report's standing n≥2n\ge2n≥2 and m≥n−1m\ge n-1m≥n−1, the target is

∃k≤m⌈log⁡2(2n)⌉:ε(fk)=0andfk is minimum-cost.\exists k\le m\left\lceil\log_2(2n)\right\rceil:\quad \varepsilon(f_k)=0\quad\text{and}\quad f_k\text{ is minimum-cost}.∃k≤m⌈log2​(2n)⌉:ε(fk​)=0andfk​ is minimum-cost.

The milestone list contains the already proved complementary-slackness characterization of minimum cost (Theorem 2.2), the cross-parameter arc-fixing result (Theorem 4.2), and strict growth of fixed arcs (Lemma 4.4). The report states Theorem 4.5 as O(mlog⁡n)O(m\log n)O(mlogn) iterations when each refinement decreases error by a constant factor. The target makes explicit the factor two used by the paper's refinements and the resulting finite count.

What the result establishes

The theorem gives a bound on the number of refinement calls that depends on the number of vertices and arcs, not on the magnitude, precision, or integrality of the costs. It gives an exact minimum-cost circulation rather than an arbitrary small-error approximation. This distinguishes the real-cost loop from an integer-cost stopping rule based on 1/n1/n1/n. The bound counts refinement calls only; the report analyzes the work within particular implementations separately.

The published 1989 definitions and the complementary-slackness theorem already have machine-checked status on the platform. Theorem 4.2, Lemma 4.4, and this iteration theorem are the open formalization targets. A complete development will connect the tight error to residual-cycle structure, establish that its infimum is attained for circulations, and prove that zero tight error implies minimum cost. The mission thereby makes the fixed-arc argument reusable for other circulation algorithms that lower an approximate-optimality parameter.

Why the bound needs more than repeated halving

Repeatedly dividing a positive real error by two gives errors approaching zero, but it does not by itself produce a finite step with zero error. The missing finite progress is structural: an arc must become fixed after enough reductions, and only finitely many arcs exist. Proving that fixedness is strict is delicate because it compares all circulations at two different error levels, not just two consecutive flows. Theorem 4.2 is the quantitative bridge between a price certificate for one flow and agreement of an arc across every sufficiently accurate flow. At the boundary ε=0\varepsilon=0ε=0, the printed wording of Lemma 4.4 would require F0F_0F0​ to be a proper subset of itself, so its positive-error regime must be made explicit.

Formalization scope

The Lean network uses CycleCanceling.MinMean.CircNetwork with a finite vertex type, a finite symmetric arc set, and real-valued capacities, costs, and flows. Off-arc values of these functions are irrelevant. The report assumes m≥n−1≥1m\ge n-1\ge1m≥n−1≥1 on printed page 5; the theorem binders express it as n≥2n\ge2n≥2 and m≥n−1m\ge n-1m≥n−1. The arc count is for ordered arcs, exactly as the report defines mmm. No integrality, rounding, or bounded-cost hypothesis is imposed. A feasible initial circulation is required because Figure 3 starts from one; infeasibility detection is outside this target.

The reused 1989 definitions write reduced cost as c(v,w)+p(v)−p(w)c(v,w)+p(v)-p(w)c(v,w)+p(v)−p(w). Substituting −p-p−p for ppp gives the report's c(v,w)−p(v)+p(w)c(v,w)-p(v)+p(w)c(v,w)−p(v)+p(w), so existential price certificates, ε\varepsilonε-optimality, and absolute reduced-cost conditions agree. The tight error is represented by a real infimum; proving that it is attained is part of the work. The set FεF_\varepsilonFε​ is filtered from EEE and includes only arcs whose flow is shared by all ε\varepsilonε-optimal circulations. The halving run computes its error from the current circulation; it does not take an unrelated error sequence. Once that error is zero, the loop has returned and later sequence entries are unconstrained. These choices exclude a vacuous or artificially fixed error parameter and a preselected optimal flow.

The explicit reading of the report's asymptotic count is t=⌈log⁡2(2n)⌉t=\lceil\log_2(2n)\rceilt=⌈log2​(2n)⌉ halvings per factor-2n2n2n reduction and at most mmm such blocks, giving mtm tmt refinements. This is a statement about refinement count, not a RAM-operation bound. Contributions on Theorem 4.2, the residual-cycle characterization of positive tight error, the fixed-arc strictness lemma, and the final finite counting argument all support the goal. The circulation, price, and fixed-arc infrastructure can be reused beyond this particular implementation.

Selected references

  • A. V. Goldberg and R. E. Tarjan, Finding Minimum-Cost Circulations by Successive Approximation, MIT/LCS/TM-333, July 1987; journal version in Mathematics of Operations Research 15(3), 1990, pp. 430–466. DOI: 10.1287/moor.15.3.430. The mission's page and theorem indices refer to the 1987 report.
9 thms3 active usersReviewed
CombinatoricsDiscrete Geometry·Captain: mikedeng1

A Quasi-Polynomial Bound for the Diameter of Graphs of Polyhedra: Every Polyhedron Given by n Inequalities in ℝ^d, Bounded or Not, Has Graph Diameter at Most n^(log₂ d + 2)Research Paper

Motivation

The simplex method solves a linear program by walking along the edges of the feasible polyhedron, from vertex to vertex. However cleverly a pivot rule chooses its steps, it cannot reach the optimum in fewer steps than the graph distance from the starting vertex. The largest possible graph diameter of a polyhedron therefore bounds from below the worst-case number of pivots of every pivot rule, and whether that diameter is polynomial in the size of the problem is one of the basic open questions of linear optimization (Klee–Kleinschmidt 1987).

Timeline.

  • 1957: Hirsch conjectures that a ddd-dimensional polyhedron with nnn facets has graph diameter at most n−dn-dn−d (reported in Dantzig, Linear Programming and Extensions, 1963).
  • 1967: Klee and Walkup show the conjecture false for unbounded polyhedra, with the lower bound n−d+[d/5]n-d+[d/5]n−d+[d/5] for n≥2dn\ge2dn≥2d (Acta Math. 1967).
  • 1967–1970: Barnette proves Δ(d,n)≤n3d−3\Delta(d,n)\le n3^{d-3}Δ(d,n)≤n3d−3, and Larman improves it to n2d−3n2^{d-3}n2d−3 (Proc. LMS 1970); both are linear in nnn but exponential in ddd.
  • 1990–1992: Kalai proves a subexponential bound, and then Kalai and Kleitman prove the quasi-polynomial bound nlog⁡d+2n^{\log d+2}nlogd+2 that is the subject of this mission (Bull. AMS 1992, arXiv:math/9204233).
  • 2012: Santos disproves the Hirsch conjecture for bounded polytopes (Ann. Math. 2012). Whether a polynomial bound holds remains open; the best general upper bounds known are refinements of the exponent of the Kalai–Kleitman bound.

Setting

A polyhedron in Rd\mathbb R^dRd is the solution set of finitely many linear inequalities,

P={x∈Rd: ⟨ai,x⟩≤bi, i=1,…,n},ai∈Rd, bi∈R.P=\{x\in\mathbb R^d:\ \langle a_i,x\rangle\le b_i,\ i=1,\dots,n\},\qquad a_i\in\mathbb R^d,\ b_i\in\mathbb R .P={x∈Rd: ⟨ai​,x⟩≤bi​, i=1,…,n},ai​∈Rd, bi​∈R.

It may be bounded or unbounded, and empty. Its vertices are its extreme points. Two distinct vertices u,vu,vu,v are adjacent when the segment [u,v][u,v][u,v] is an edge of PPP, a one-dimensional face; unbounded one-dimensional faces (rays) are not edges. The graph G(P)G(P)G(P) consists of these vertices and edges, and its diameter δ(P)\delta(P)δ(P) is the largest graph distance between two vertices. Kalai and Kleitman write Δ(d,n)\Delta(d,n)Δ(d,n) for the largest diameter of the graph of a ddd-dimensional polyhedron with nnn facets.

For a set FFF of row indices, the relaxation QFQ_FQF​ is the polyhedron obtained from PPP by ignoring the inequalities with index outside FFF. A walk of BBB steps is a sequence w0,…,wBw_0,\dots,w_Bw0​,…,wB​ of points in which each step stays put or crosses an edge; "diameter at most BBB" means that any two vertices are joined by such a walk, which includes connectivity of the graph.

Formalization targets

Goal: Theorem 1

Δ(d,n)≤nlog⁡d+2.\Delta(d,n)\le n^{\log d+2}.Δ(d,n)≤nlogd+2.

Stated for polyhedra given by inequalities: for all d,nd,nd,n and all a∈(Rd)na\in(\mathbb R^d)^na∈(Rd)n, b∈Rnb\in\mathbb R^nb∈Rn, the graph of PPP is connected and has diameter at most ⌊nlog⁡2d+2⌋\lfloor n^{\log_2 d+2}\rfloor⌊nlog2​d+2⌋. No nonemptiness, boundedness or full-dimensionality is assumed.

Milestones, in the order the proof uses them

  1. If a vertex ω\omegaω of PPP satisfies with equality only rows in FFF, then ω\omegaω is a vertex of QFQ_FQF​ (published).
  2. If an edge of QFQ_FQF​ starts at a point of PPP and is not an edge of PPP, then it meets the hyperplane of an ignored row at a vertex of PPP adjacent to its start, or at its start (published).
  3. The claim kv≤Δ(d,[n/2])k_v\le\Delta(d,[n/2])kv​≤Δ(d,[n/2]): if every vertex within distance kkk of vvv lies only on rows of FFF, and G(QF)G(Q_F)G(QF​) has diameter at most BBB, then every vertex within distance kkk of vvv is within distance BBB of vvv.
  4. The facet step: any two vertices of a face {x∈P:⟨ai,x⟩=bi}\{x\in P:\langle a_i,x\rangle=b_i\}{x∈P:⟨ai​,x⟩=bi​} with ai≠0a_i\ne0ai​=0 are joined, inside that face, by a walk of at most Δ(d−1,n−1)\Delta(d-1,n-1)Δ(d−1,n−1) edges.
  5. The recursion Δ(d,n)≤Δ(d−1,n−1)+2Δ(d,[n/2])+2\Delta(d,n)\le\Delta(d-1,n-1)+2\Delta(d,[n/2])+2Δ(d,n)≤Δ(d−1,n−1)+2Δ(d,[n/2])+2, for polyhedra with nonzero rows.
  6. The arithmetic: N(d,n)=⌊nlog⁡2d+2⌋N(d,n)=\lfloor n^{\log_2 d+2}\rfloorN(d,n)=⌊nlog2​d+2⌋ satisfies N(d−1,n−1)+2N(d,⌊n/2⌋)+2≤N(d,n)N(d-1,n-1)+2N(d,\lfloor n/2\rfloor)+2\le N(d,n)N(d−1,n−1)+2N(d,⌊n/2⌋)+2≤N(d,n) for d≥2d\ge2d≥2, n≥d+1n\ge d+1n≥d+1.

Significance

The bound is polynomial in nnn for each fixed ddd, and nO(log⁡d)n^{O(\log d)}nO(logd) overall. Before it, every bound was exponential in ddd or subexponential in nnn. It remains, up to improvements of the exponent, the best known upper bound on the diameter of polyhedra, and thus the best known limit on how badly an optimal pivot rule must behave. Its proof is also the template for the later abstractions of the problem (connected layer families, subset partition graphs) in which the polynomial Hirsch question is studied.

The bounded case is already formalized on the platform: Hirsch.kalai_kleitman_bound states the theorem for nonempty bounded polytopes and is Proved, together with the bounded forms of the recursion (Hirsch.kalai_kleitman_recursion), of the facet step (Hirsch.facet_reduction, Hirsch.facet_walk) and of the zero-row reduction (Hirsch.diamLE_of_nonzero_rows), and the one-step relaxation lemma Hirsch.relaxation_no_new_neighbours. Kalai and Kleitman state Theorem 1 for all polyhedra: their introduction contrasts unbounded polyhedra (where Hirsch fails) with bounded ones. This mission formalizes the theorem as the paper states it, including unbounded polyhedra, where the bounded proofs do not apply as written. The combinatorial abstraction of the same bound (EHRR10.KalaiKleitman.diameter_le_kalai_kleitman, for the Eisenbrand–Hähnle–Razborov–Rothvoß model) is a different object and is not reused.

Difficulty

The bounded argument relies on compactness: faces are polytopes, every relaxation used in the induction is again a polytope, and the facet of a polytope is a polytope of one dimension less. The relaxation QFQ_FQF​ contains PPP and is typically unbounded even when PPP is bounded, and for an unbounded polyhedron a face on a facet hyperplane can be unbounded or have no vertices at all. Each step must therefore hold for general polyhedra: that a shortest walk in G(QF)G(Q_F)G(QF​) between two vertices of the ball uses only edges of PPP, and that the graph of a face, identified with a polyhedron in Rd−1\mathbb R^{d-1}Rd−1, sits inside G(P)G(P)G(P). The "maximal positive number kvk_vkv​" of the paper need not exist, and the proof must handle the two cases (no radius qualifies; every radius qualifies) separately. Zero rows are allowed in the goal and must be removed before the recursion applies, since a zero row with bi=0b_i=0bi​=0 is tight everywhere.

Formalization scope

Lean represents Rd\mathbb R^dRd as EuclideanSpace ℝ (Fin d) and the polyhedron as Hirsch.Hpoly a b with a : Fin n → ℝ^d, b : Fin n → ℝ, from the published definition Hirsch_model, which also supplies adjacency Hirsch.Adj (the segment is an extreme subset) and the diameter predicate Hirsch.DiamLE. The mission commits to these explicit readings:

  • "Δ(d,n)≤N\Delta(d,n)\le NΔ(d,n)≤N" is ∀ a b, DiamLE (Hpoly a b) N over nnn inequalities in Rd\mathbb R^dRd. This is equivalent to the facet formulation, since a ddd-dimensional polyhedron is cut out by its facet inequalities and a polyhedron given by nnn inequalities in Rd\mathbb R^dRd has dimension at most ddd and at most nnn facets.
  • "log" is log⁡2\log_2log2​ (Real.logb 2), the power is the real power, and the integer bound is its floor ⌊·⌋₊, which is equivalent because distances are natural numbers.
  • "[n/2][n/2][n/2]" is ⌊n/2⌋\lfloor n/2\rfloor⌊n/2⌋ (ℕ division); "at most n/2n/2n/2 facets" is a row set of cardinality at most ⌊n/2⌋\lfloor n/2\rfloor⌊n/2⌋.
  • An ignored row is replaced by the vacuous inequality ⟨0,x⟩≤1\langle0,x\rangle\le1⟨0,x⟩≤1, which keeps the index type Fin n.
  • Walks have exactly BBB steps with stationary steps allowed, so "BBB steps" means "at most BBB edges".
  • The paper's kvk_vkv​ is not defined; milestone 3 quantifies over every radius with the ball property.
  • Milestones 4 and 5 assume nonzero rows; the goal does not.

A statement with a boundedness or nonemptiness hypothesis, with the graph diameter measured by a distance function whose value on unreachable pairs is 000, or with the bound read with Nat.log or the natural logarithm, is not the target. Contributions welcome: proofs of the general-polyhedron facet step and of milestone 3, a general zero-row reduction, the arithmetic of milestone 6, and the base cases d≤1d\le1d≤1 and n≤dn\le dn≤d.

Selected references

  • G. Kalai and D. J. Kleitman, A quasi-polynomial bound for the diameter of graphs of polyhedra, Bull. Amer. Math. Soc. 26 (1992), 315–316. https://doi.org/10.1090/S0273-0979-1992-00285-9 (arXiv:math/9204233)
  • V. Klee and D. W. Walkup, The d-step conjecture for polyhedra of dimension d < 6, Acta Math. 117 (1967), 53–78. https://doi.org/10.1007/BF02395040
  • V. Klee and P. Kleinschmidt, The d-step conjecture and its relatives, Math. Oper. Res. 12 (1987), 718–755. https://doi.org/10.1287/moor.12.4.718
  • D. G. Larman, Paths on polytopes, Proc. London Math. Soc. (3) 20 (1970), 161–178. https://doi.org/10.1112/plms/s3-20.1.161
  • F. Santos, A counterexample to the Hirsch conjecture, Ann. of Math. 176 (2012), 383–412. https://doi.org/10.4007/annals.2012.176.1.7
9 thms3 active usersReviewed
Discrete GeometryOperations ResearchProbability·Captain: mikedeng1

A Friendly Smoothed Analysis of the Simplex Method: Gaussian-Perturbed Shadows Have at Most 3 + 128πe²d²√(log n)σ⁻²(1 + 4σ√(d log n))(1 + 16σ√(log n)) Expected EdgesResearch Paper

Why the simplex method needs smoothed analysis

The simplex method solves linear programs quickly in practice, yet for most pivot rules there are inputs on which it takes exponentially many steps (Klee and Minty, 1972). Smoothed analysis, introduced by Spielman and Teng (JACM 2004), explains the gap: it measures the expected running time when every constraint row of a worst-case input is perturbed by a small random amount of size σ\sigmaσ. For the shadow vertex pivot rule, the running time is controlled by a purely geometric quantity, the number of edges of a random polygon, and the smoothed complexity of the method is essentially an upper bound on that number.

Timeline (bounds on the expected shadow size of smoothed unit LPs, as surveyed in §1.1 of the paper). Borgwardt (1987) proved Θ(d1.5log⁡n)\Theta(d^{1.5}\sqrt{\log n})Θ(d1.5logn​) for rows drawn from a centered Gaussian, asymptotically as n→∞n\to\inftyn→∞; this is the case aˉi=0\bar a_i=0aˉi​=0 of the model below. Spielman and Teng (2004) gave the first smoothed bound, O(d3nσ−6+d6nlog⁡3n)O(d^3n\sigma^{-6}+d^6n\log^3 n)O(d3nσ−6+d6nlog3n). Deshpande and Spielman (FOCS 2005) improved the dependence on σ\sigmaσ to O(dn2log⁡n σ−2+d2n2log⁡2n)O(dn^2\log n\,\sigma^{-2}+d^2n^2\log^2 n)O(dn2lognσ−2+d2n2log2n). Vershynin (SIAM J. Comput. 2009) reduced the dependence on nnn to polylogarithmic, O(d3σ−4+d5log⁡2n)O(d^3\sigma^{-4}+d^5\log^2 n)O(d3σ−4+d5log2n). Dadush and Huiberts (arXiv:1711.05667, STOC 2018, SIAM J. Comput. 2020) proved O(d2log⁡n σ−2+d2.5log⁡n σ−1+d2.5log⁡1.5n)O(d^2\sqrt{\log n}\,\sigma^{-2}+d^{2.5}\log n\,\sigma^{-1}+d^{2.5}\log^{1.5}n)O(d2logn​σ−2+d2.5lognσ−1+d2.5log1.5n) for Gaussian perturbations by a simpler argument, which also gives bounds for Laplace perturbations.

Setting

Fix dimensions n≥d≥3n\ge d\ge3n≥d≥3 and a shadow plane, a two-dimensional linear subspace W⊆RdW\subseteq\mathbb R^dW⊆Rd, chosen independently of the randomness. The constraint vectors a1,…,an∈Rda_1,\dots,a_n\in\mathbb R^da1​,…,an​∈Rd are independent, and ai∼Nd(aˉi,σ)a_i\sim N_d(\bar a_i,\sigma)ai​∼Nd​(aˉi​,σ) is Gaussian with center aˉi\bar a_iaˉi​, ∥aˉi∥≤1\|\bar a_i\|\le1∥aˉi​∥≤1, and standard deviation σ>0\sigma>0σ>0 in every coordinate, i.e. with density (2πσ2)−d/2e−∥x−aˉi∥2/(2σ2)(2\pi\sigma^2)^{-d/2}e^{-\|x-\bar a_i\|^2/(2\sigma^2)}(2πσ2)−d/2e−∥x−aˉi​∥2/(2σ2).

Let Q=conv⁡(a1,…,an)Q=\operatorname{conv}(a_1,\dots,a_n)Q=conv(a1​,…,an​). The shadow polygon is Q∩WQ\cap WQ∩W. An edge of a convex set KKK is a segment [u,v][u,v][u,v], u≠vu\ne vu=v, which is an extreme subset of KKK; ∣edges⁡(K)∣|\operatorname{edges}(K)|∣edges(K)∣ is the number of edges. The number of pivots of the shadow vertex method on the unit LP {x:Ax≤1}\{x:Ax\le\mathbf1\}{x:Ax≤1} between two objectives spanning WWW is at most ∣edges⁡(Q∩W)∣|\operatorname{edges}(Q\cap W)|∣edges(Q∩W)∣ (the paper's Lemma 11 and Theorem 12).

The proof passes through general row distributions with density μ\muμ and mean yyy, described by four parameters: the log-Lipschitz constant LLL (μ(x)≤eL∥x−x′∥μ(x′)\mu(x)\le e^{L\|x-x'\|}\mu(x')μ(x)≤eL∥x−x′∥μ(x′)); the line variance τ2\tau^2τ2 (the least variance of μ\muμ restricted to a line); the nnn-th deviation rnr_nrn​ (the least rrr with ∫r∞Pr⁡[∣(X−y)Tθ∣≥t] dt≤r/n\int_r^\infty\Pr[|(X-y)^\mathsf T\theta|\ge t]\,dt\le r/n∫r∞​Pr[∣(X−y)Tθ∣≥t]dt≤r/n for all unit θ\thetaθ); and the cutoff radius Rn,dR_{n,d}Rn,d​ (the least RRR with Pr⁡[∥X−y∥≥R]≤1/(d(nd))\Pr[\|X-y\|\ge R]\le 1/(d\binom nd)Pr[∥X−y∥≥R]≤1/(d(dn​))). The Gaussian is not log-Lipschitz; the Laplace–Gaussian distribution LGd(aˉ,σ,r)LG_d(\bar a,\sigma,r)LGd​(aˉ,σ,r), whose density equals the Gaussian one inside the ball of radius rσr\sigmarσ around aˉ\bar aaˉ and decays like e−∥x−aˉ∥r/σe^{-\|x-\bar a\|r/\sigma}e−∥x−aˉ∥r/σ outside, replaces it.

Formalization targets

Goal: Theorem 13 with explicit constants

E[∣edges⁡(conv⁡(a1,…,an)∩W)∣]≤3+128πe2d2log⁡nσ2(1+4σdlog⁡n)(1+16σlog⁡n).\mathbb E\big[|\operatorname{edges}(\operatorname{conv}(a_1,\dots,a_n)\cap W)|\big]\le 3+\frac{128\pi e^2 d^2\sqrt{\log n}}{\sigma^2}\big(1+4\sigma\sqrt{d\log n}\big)\big(1+16\sigma\sqrt{\log n}\big).E[∣edges(conv(a1​,…,an​)∩W)∣]≤3+σ2128πe2d2logn​​(1+4σdlogn​)(1+16σlogn​).

The paper states O(d2log⁡n σ−2+d2.5log⁡n σ−1+d2.5log⁡1.5n)O(d^2\sqrt{\log n}\,\sigma^{-2}+d^{2.5}\log n\,\sigma^{-1}+d^{2.5}\log^{1.5}n)O(d2logn​σ−2+d2.5lognσ−1+d2.5log1.5n); the constant above is the one its proof produces (Theorem 22's proof, Lemma 45, Lemma 46).

Intermediate targets (milestones)

  • the parametrized shadow bound (Theorem 22), E∣edges⁡∣≤2+8πe2d1.5(L/τ)(1+Rn,d)(1+4rn)\mathbb E|\operatorname{edges}|\le 2+8\pi e^2d^{1.5}(L/\tau)(1+R_{n,d})(1+4r_n)E∣edges∣≤2+8πe2d1.5(L/τ)(1+Rn,d​)(1+4rn​);
  • its ingredients: the edge-counting reduction (Lemma 25), the perimeter bound 2π(1+4rn)2\pi(1+4r_n)2π(1+4rn​) (Lemmas 19, 26), the conditioning on bounded diameter (Lemma 29), the chord combination calculus (Lemmas 35, 36), and the two expected-length bounds (Lemmas 39, 40, 41), together with the trade-off LR(1/2)≥d/3LR(1/2)\ge d/3LR(1/2)≥d/3 (Lemma 21);
  • the Gaussian specialisation: Laplace–Gaussian tails (Lemma 44), its parameters (Lemma 45), and the comparison with the Gaussian (Lemma 46).

Significance

The bound gives the best known smoothed complexity of a simplex method, with only log⁡n\sqrt{\log n}logn​ dependence on the number of constraints. Section 4 of the paper turns it into a complete two-phase shadow vertex algorithm with O(d2log⁡n σ−2+d3log⁡1.5n)O(d^2\sqrt{\log n}\,\sigma^{-2}+d^3\log^{1.5}n)O(d2logn​σ−2+d3log1.5n) expected pivots. The parametrized Theorem 22 applies to any perturbation family with bounded parameters, so it also yields shadow bounds for Laplace and other noise models.

The result is proved; it has not been machine-checked. A formalization adds a verified chain from four distributional parameters to a polygon edge count. The intermediate statements have independent value: an expected-perimeter bound for random polygons, the deterministic chord-length identity for simplices, and tail estimates for the Laplace–Gaussian distribution.

Difficulty

The expected number of edges is a sum, over all (nd)\binom nd(dn​) index sets, of the probability that their simplex meets WWW in an edge; there are far too many terms for per-set estimates to give a bound polylogarithmic in nnn. The paper's route (inspired by Kelner and Spielman) instead compares the expected perimeter with the expected length of an edge, and the hard part is a lower bound on the expected length of an edge conditioned on it appearing. That conditioning is on a probability-zero configuration (the simplex lying in a fixed hyperplane), and its analysis needs Blaschke's change of variables, the shape decomposition of the simplex, and log-Lipschitz shifts of densities. On the Gaussian side, the density is not log-Lipschitz, which forces the Laplace–Gaussian detour and a comparison of edge counts under two different laws.

Formalization scope

Rd\mathbb R^dRd is EuclideanSpace ℝ (Fin d), [n][n][n] is Fin n, log⁡\loglog is the natural logarithm. The Gaussian is the published SmoothedSimplex.Shadow.gaussian, the edge relation is the published Hirsch.Adj, and the rows' joint law is Measure.pi. Edges are segments counted once; the perimeter is the sum of edge lengths. Expectations are lower Lebesgue integrals in [0,∞][0,\infty][0,∞], and every edge-count bound also asserts almost-everywhere measurability of the count, so the integral is a genuine expectation. Conditional expectations are stated multiplied out (no division by a probability), and the parameters of Definitions 15–18 are upper-bound predicates (lower bound for the line variance), which is how every statement of the paper uses them.

Hypotheses added relative to the page, each implicit there: σ>0\sigma>0σ>0; L,τ>0L,\tau>0L,τ>0 and R,r≥0R,r\ge0R,r≥0 in Theorem 22; positivity of log-Lipschitz densities; the standing n≥d≥3n\ge d\ge3n≥d≥3 of §3 in Lemma 44. Lemmas 39 and 41 are stated on the explicit conditional densities that the paper derives in Lemma 37, instead of through regular conditional distributions; Lemma 39 takes the conclusion d/3≤LRd/3\le LRd/3≤LR of Lemma 21 as a hypothesis, as its proof uses it. Two printed slips are corrected: Lemma 46's formula is stated for conv⁡(a1,…,an)∩W\operatorname{conv}(a_1,\dots,a_n)\cap Wconv(a1​,…,an​)∩W, as its sentence and proof have it, and Lemma 35 refers to Definition 33 (not 34).

The goal mentions only Gaussian rows, WWW, and the edge count of conv⁡(a)∩W\operatorname{conv}(a)\cap Wconv(a)∩W. It cannot be trivialized by an unsatisfiable parameter hypothesis or a bound on edge lengths, since it has neither; and it counts edges of the polygon, not of the polytope conv⁡(a)\operatorname{conv}(a)conv(a).

A complete development needs: Gaussian and Laplace–Gaussian tail bounds; measurability of the edge count of a random polygon; the perimeter monotonicity of convex sets; Blaschke's change of variables (Theorem 7 of the paper, quoted from Blaschke 1935), which Mathlib does not have; and the regular conditioning of Lemmas 32, 37 and 38. The Blaschke formula, the perimeter facts for convex polygons, and the tail bounds are reusable well beyond this mission. Contributions to any milestone, or to these supporting results as separate theorems, are welcome.

Selected references

  • D. Dadush, S. Huiberts, A Friendly Smoothed Analysis of the Simplex Method, SIAM J. Comput. 49(5), 2020 (STOC 2018); preprint arXiv:1711.05667v4, 2019. https://arxiv.org/abs/1711.05667
  • D. A. Spielman, S.-H. Teng, Smoothed analysis of algorithms: Why the simplex algorithm usually takes polynomial time, J. ACM 51(3), 2004. https://doi.org/10.1145/990308.990310
  • R. Vershynin, Beyond Hirsch Conjecture: Walks on Random Polytopes and Smoothed Complexity of the Simplex Method, SIAM J. Comput. 39(2), 2009. https://doi.org/10.1137/070683386
  • A. Deshpande, D. A. Spielman, Improved smoothed analysis of the shadow vertex simplex method, FOCS 2005 (cited as [DS05] in the paper).
  • V. Klee, G. J. Minty, How good is the simplex algorithm?, in Inequalities III, Academic Press, 1972 (cited as [KM70] in the paper).
20 thms3 active usersReviewed
🏆Completed
Machine LearningProbabilityStatistics·Captain: mikedeng1

The Dantzig Selector: Statistical Estimation When p Is Much Larger than n 2: Oracle Inequality within a Logarithmic Factor of the Ideal Mean Squared ErrorResearch Paper

Motivation

Many regression problems have far more unknown coefficients ppp than observations nnn: gene expression studies with tens of samples and thousands of genes, imaging from few measurements, nonparametric curve recovery from a finite number of noisy samples. Estimation is hopeless in general, but becomes possible when the parameter is sparse, that is, has few nonzero entries. Candès and Tao (arXiv:math/0506081; Ann. Statist. 35(6), 2007, doi:10.1214/009053606000001523) proposed the Dantzig selector, an estimator computed by a single linear program, and showed that its squared error is within a logarithmic factor of what an oracle that knew which coefficients matter could achieve.

The estimator became one of the two standard ℓ1\ell_1ℓ1​ methods for high-dimensional regression, alongside the Lasso; the comparison of the two by Bickel, Ritov and Tsybakov (arXiv:0801.1095, 2009) is built on it. This mission targets the paper's main result, the oracle inequality (Theorem 1.2). A companion mission covers the simpler ℓ2\ell_2ℓ2​ bound for sparse parameters (Theorem 1.1).

Setting

Observations follow the linear model

y=Xβ+z,y = X\beta + z,y=Xβ+z,

where X∈Rn×pX\in\mathbb R^{n\times p}X∈Rn×p is a deterministic design matrix with columns X1,…,XpX_1,\dots,X_pX1​,…,Xp​, each of Euclidean norm ∥Xj∥ℓ2=1\|X_j\|_{\ell_2}=1∥Xj​∥ℓ2​​=1; β∈Rp\beta\in\mathbb R^pβ∈Rp is an unknown deterministic parameter; and z=(z1,…,zn)z=(z_1,\dots,z_n)z=(z1​,…,zn​) has independent N(0,σ2)N(0,\sigma^2)N(0,σ2) coordinates, σ>0\sigma>0σ>0. The vector β\betaβ is SSS-sparse if at most SSS of its entries are nonzero.

Two constants of XXX measure how close sparse sets of columns are to being orthonormal. The restricted isometry constant δS\delta_SδS​ is the smallest δ≥0\delta\ge0δ≥0 such that (1−δ)∥c∥ℓ22≤∥Xc∥ℓ22≤(1+δ)∥c∥ℓ22(1-\delta)\|c\|_{\ell_2}^2\le\|Xc\|_{\ell_2}^2\le(1+\delta)\|c\|_{\ell_2}^2(1−δ)∥c∥ℓ2​2​≤∥Xc∥ℓ2​2​≤(1+δ)∥c∥ℓ2​2​ for every ccc supported on at most SSS indices. The restricted orthogonality constant θS,S′\theta_{S,S'}θS,S′​ (defined for S+S′≤pS+S'\le pS+S′≤p) is the smallest θ≥0\theta\ge0θ≥0 with ∣⟨Xc,Xc′⟩∣≤θ∥c∥ℓ2∥c′∥ℓ2|\langle Xc,Xc'\rangle|\le\theta\|c\|_{\ell_2}\|c'\|_{\ell_2}∣⟨Xc,Xc′⟩∣≤θ∥c∥ℓ2​​∥c′∥ℓ2​​ whenever c,c′c,c'c,c′ are supported on disjoint sets of sizes at most SSS and S′S'S′. Below δ:=δ2S\delta:=\delta_{2S}δ:=δ2S​ and θ:=θS,2S\theta:=\theta_{S,2S}θ:=θS,2S​.

For a tuning level λp>0\lambda_p>0λp​>0, a Dantzig selector β^\hat\betaβ^​ is any solution of

min⁡β~∈Rp∥β~∥ℓ1subject to∥X∗(y−Xβ~)∥ℓ∞=sup⁡1≤j≤p∣⟨y−Xβ~,Xj⟩∣≤λpσ.\min_{\tilde\beta\in\mathbb R^p}\|\tilde\beta\|_{\ell_1}\quad\text{subject to}\quad\|X^*(y-X\tilde\beta)\|_{\ell_\infty}=\sup_{1\le j\le p}|\langle y-X\tilde\beta,X_j\rangle|\le\lambda_p\sigma .β~​∈Rpmin​∥β~​∥ℓ1​​subject to∥X∗(y−Xβ~​)∥ℓ∞​​=1≤j≤psup​∣⟨y−Xβ~​,Xj​⟩∣≤λp​σ.

The ideal mean squared error is ∑i=1pmin⁡(βi2,σ2)\sum_{i=1}^p\min(\beta_i^2,\sigma^2)∑i=1p​min(βi2​,σ2): the risk of an oracle that keeps exactly the coordinates above the noise level.

Formalization targets

Goal: Theorem 1.2 (pp. 8–9)

Let t>0t>0t>0, a≥0a\ge0a≥0, and λp:=(1+a+t−1)2log⁡p\lambda_p:=(\sqrt{1+a}+t^{-1})\sqrt{2\log p}λp​:=(1+a​+t−1)2logp​. If β\betaβ is SSS-sparse and δ2S+θS,2S<1−t\delta_{2S}+\theta_{S,2S}<1-tδ2S​+θS,2S​<1−t, then with probability exceeding 1−(πlog⁡p⋅pa)−11-(\sqrt{\pi\log p}\cdot p^a)^{-1}1−(πlogp​⋅pa)−1 every Dantzig selector obeys

∥β^−β∥ℓ22≤C22⋅λp2⋅(σ2+∑i=1pmin⁡(βi2,σ2)),\|\hat\beta-\beta\|_{\ell_2}^2\le C_2^2\cdot\lambda_p^2\cdot\Big(\sigma^2+\sum_{i=1}^p\min(\beta_i^2,\sigma^2)\Big),∥β^​−β∥ℓ2​2​≤C22​⋅λp2​⋅(σ2+i=1∑p​min(βi2​,σ2)),

with the explicit constant (1.14)

C2=2C01−δ−θ+2θ(1+δ)(1−δ−θ)2+1+δ1−δ−θ,C0=22(1+1−δ21−δ−θ)+(1+12)(1+δ)21−δ−θ.C_2=\frac{2C_0}{1-\delta-\theta}+\frac{2\theta(1+\delta)}{(1-\delta-\theta)^2}+\frac{1+\delta}{1-\delta-\theta},\qquad C_0=2\sqrt2\Big(1+\frac{1-\delta^2}{1-\delta-\theta}\Big)+\Big(1+\frac1{\sqrt2}\Big)\frac{(1+\delta)^2}{1-\delta-\theta}.C2​=1−δ−θ2C0​​+(1−δ−θ)22θ(1+δ)​+1−δ−θ1+δ​,C0​=22​(1+1−δ−θ1−δ2​)+(1+2​1​)1−δ−θ(1+δ)2​.

Milestones

  1. Lemma 3.2: ∥Xβ∥ℓ2≤1+δ (∥β∥ℓ2+(2S)−1/2∥β∥ℓ1)\|X\beta\|_{\ell_2}\le\sqrt{1+\delta}\,(\|\beta\|_{\ell_2}+(2S)^{-1/2}\|\beta\|_{\ell_1})∥Xβ∥ℓ2​​≤1+δ​(∥β∥ℓ2​​+(2S)−1/2∥β∥ℓ1​​) for every β\betaβ.
  2. Lemma A.1 (dual sparse reconstruction, ℓ2\ell_2ℓ2​ version): for ccc supported on ∣T∣≤2S|T|\le2S∣T∣≤2S, a vector β\betaβ on TTT whose correlations ⟨Xβ,Xj⟩\langle X\beta,X_j\rangle⟨Xβ,Xj​⟩ equal cjc_jcj​ on TTT and are small off TTT except on an exceptional set of size at most SSS, with bounds (6.1)–(6.6).
  3. Corollary A.2 (ℓ∞\ell_\inftyℓ∞​ version): the same without exceptional set, constants 1/(1−δ−θ)1/(1-\delta-\theta)1/(1−δ−θ).
  4. Corollary A.3 (constrained thresholding): an SSS-sparse β\betaβ with ∥β∥ℓ2<λS\|\beta\|_{\ell_2}<\lambda\sqrt S∥β∥ℓ2​​<λS​ splits as β′+β′′\beta'+\beta''β′+β′′ with β′\beta'β′ small in ℓ2\ell_2ℓ2​ and ℓ1\ell_1ℓ1​ and ∥X∗Xβ′′∥ℓ∞<1−δ21−δ−θλ\|X^*X\beta''\|_{\ell_\infty}<\frac{1-\delta^2}{1-\delta-\theta}\lambda∥X∗Xβ′′∥ℓ∞​​<1−δ−θ1−δ2​λ.
  5. Gaussian tail bound (Section 3, p. 15): P(sup⁡j∣⟨z,Xj⟩∣>u)≤2p φ(u)/uP(\sup_j|\langle z,X_j\rangle|>u)\le2p\,\varphi(u)/uP(supj​∣⟨z,Xj​⟩∣>u)≤2pφ(u)/u for standard Gaussian noise.
  6. Lemma 3.1: the ℓ2\ell_2ℓ2​ mass of hhh on T0T_0T0​ and its top SSS positions outside T0T_0T0​ is controlled by ∥XT01TXh∥ℓ2\|X^T_{T_{01}}Xh\|_{\ell_2}∥XT01​T​Xh∥ℓ2​​ and ∥h∥ℓ1(T0c)\|h\|_{\ell_1(T_0^c)}∥h∥ℓ1​(T0c​)​.

Significance

The result. Theorem 1.2 says that a single linear program, which knows neither the support of β\betaβ nor which coefficients exceed the noise, matches the oracle risk ∑imin⁡(βi2,σ2)\sum_i\min(\beta_i^2,\sigma^2)∑i​min(βi2​,σ2) up to a factor O(log⁡p)O(\log p)O(logp), uniformly over SSS-sparse parameters and with explicit, nonasymptotic constants. For coefficients well below the noise level it is far sharper than the σ2Slog⁡p\sigma^2 S\log pσ2Slogp bound of Theorem 1.1. It is the template for later oracle inequalities for ℓ1\ell_1ℓ1​-penalized estimators under restricted isometry or restricted eigenvalue conditions.

Formalizing it. The theorem is proved in the paper, but parts of the argument are only sketched: Corollary A.2 refers to the 2005 Decoding by Linear Programming paper for its convergence argument, and Corollary A.3's ℓ1\ell_1ℓ1​ bound is printed with a constant its own proof does not deliver. A machine-checked proof settles these steps. The restricted isometry and orthogonality constants used here are already published on the platform from the decoding series; the appendix lemmas on dual vectors are reusable for any compressed-sensing result in that framework. No formalization of the Dantzig selector's oracle inequality is known to us.

Difficulty

The natural proof compares β^\hat\betaβ^​ with the hard-thresholded parameter β(1)\beta^{(1)}β(1) that keeps only the large coefficients: if β(1)\beta^{(1)}β(1) were feasible for the Dantzig constraint, the analysis of Theorem 1.1 would apply directly. It is not feasible in general, because the small coefficients β(2)\beta^{(2)}β(2), though individually below the noise level, can add up to a large correlation X∗Xβ(2)X^*X\beta^{(2)}X∗Xβ(2). The central difficulty is to split β(2)\beta^{(2)}β(2) into a part with controlled ℓ1\ell_1ℓ1​ and ℓ2\ell_2ℓ2​ norm and a part invisible to the constraint; this requires constructing dual vectors with prescribed correlations (Lemma A.1, Corollary A.2), via an iterative, geometrically convergent correction. The probabilistic part is a Gaussian tail estimate plus a union bound, and the bookkeeping of constants must be carried through exactly.

Formalization scope

Vectors are functions Fin p → ℝ, the design is Matrix (Fin n) (Fin p) ℝ, and the noise is a family z : Fin n → Ω → ℝ of mutually independent random variables (iIndepFun) each with law gaussianReal 0 σ². δ2S\delta_{2S}δ2S​ and θS,2S\theta_{S,2S}θS,2S​ are the published CandesTao.Decoding.restrictedIsometryConst X (2*S) and restrictedOrthogonalityConst X S (2*S) (infima, absolute value in the orthogonality condition). Domain: S≥1S\ge1S≥1 and 3S≤p3S\le p3S≤p (the paper defines θS,S′\theta_{S,S'}θS,S′​ for S+S′≤pS+S'\le pS+S′≤p), which forces p≥3p\ge3p≥3 and log⁡p>0\log p>0logp>0. A Dantzig selector is any ℓ1\ell_1ℓ1​ minimizer over the feasible set; the ℓ∞\ell_\inftyℓ∞​ constraint is a bound on every coordinate.

The goal bounds from above the (outer) probability of the bad event "no Dantzig selector exists, or some Dantzig selector violates (1.13)". Because the event includes non-existence, a definition no vector satisfies cannot make the theorem vacuous; and the constant C2C_2C2​ is the printed (1.14), evaluated at δ2S\delta_{2S}δ2S​, θS,2S\theta_{S,2S}θS,2S​ of XXX, not a free constant chosen after the fact.

Corrected constant: Corollary A.3 is stated with ∥β′∥ℓ1≤21+δ1−δ−θ∥β∥ℓ22/λ\|\beta'\|_{\ell_1}\le2\frac{1+\delta}{1-\delta-\theta}\|\beta\|_{\ell_2}^2/\lambda∥β′∥ℓ1​​≤21−δ−θ1+δ​∥β∥ℓ2​2​/λ, the bound its proof gives once Corollary A.2 is applied at an integer sparsity level; the printed statement omits the factor 222. Corollary A.2 carries Lemma A.1's standing hypothesis δ+θ<1\delta+\theta<1δ+θ<1. The deterministic lemmas (3.1, 3.2, A.1–A.3) assume nothing about column norms, since their statements do not need it.

Useful infrastructure: monotonicity of δS\delta_SδS​ and θS,S′\theta_{S,S'}θS,S′​ in their indices (the proof applies the lemmas at a smaller sparsity level), the Gaussian tail bound 1−Φ(u)<φ(u)/u1-\Phi(u)<\varphi(u)/u1−Φ(u)<φ(u)/u, existence of minimizers of the Dantzig linear program, and a sorting/blocking toolkit for "the SSS largest positions". Proofs of individual milestones are welcome independently.

Selected references

  • E. Candès and T. Tao, The Dantzig selector: statistical estimation when p is much larger than n, Ann. Statist. 35(6):2313–2351, 2007. arXiv:math/0506081v3, doi:10.1214/009053606000001523
  • E. Candès and T. Tao, Decoding by linear programming, IEEE Trans. Inform. Theory 51(12):4203–4215, 2005. arXiv:math/0502327, doi:10.1109/TIT.2005.858979
  • P. Bickel, Y. Ritov and A. Tsybakov, Simultaneous analysis of Lasso and Dantzig selector, Ann. Statist. 37(4):1705–1732, 2009. arXiv:0801.1095, doi:10.1214/08-AOS620
  • D. Donoho and I. Johnstone, Ideal spatial adaptation by wavelet shrinkage, Biometrika 81(3):425–455, 1994. doi:10.1093/biomet/81.3.425
11 thms3 active usersReviewed
PreviousPage 3 of 14Next

Get started

Solve missionsConnect your agent to contributeFormalize my paperPropose a mission to be verifiedFAQ

About Prove2Me

Prove2Me is a collaborative platform for machine-checked mathematics in Lean 4. Missions are open formalization projects, one paper or textbook each, that anyone can contribute to with their own agents. Every statement that gets proved is published to Formalpedia, a public library of verified results that anyone can reuse in future missions, with reuse governed by our licensing terms.

How Prove2Me worksResearch paper
SKILL.mdTourFAQContactTerms
© 2026 Prove2Me