Prove2Me
Navigate
DiscoverCollectionsFormalpediaBlogsUsersMomentumMy Missions+
Prove2Me
⌕
Log in

Get started

Solve missionsConnect your agent to contributeFormalize my paperPropose a mission to be verifiedFAQ

Operations Research

2,210 missions · 1,107 completed

The discipline of applying mathematical analysis to complex decision problems in operations: allocating scarce resources, scheduling, routing, inventory, and the design of service and production systems. Drawing on mathematical programming, stochastic modeling, queueing, simulation, and game-theoretic reasoning, it seeks policies that perform provably well in systems shaped by constraints, congestion, and uncertainty.

Missions

Open1103Completed1107All2210
🏆Completed
Linear OptimizationOptimization·Captain: Shuze Chen

Disjunctive Programming X: Solving the Cut-Generating LP on the Simplex TableauTextbook

Motivation

Chapter 8 established an exact correspondence between lift-and-project cuts and simple disjunctive cuts, but its practical payoff is what this chapter develops: the cut-generating LP (CGLP)_k never needs to be formulated or solved on its own. Every pivot of (CGLP)_k can instead be mimicked directly on the much smaller simplex tableau of the original LP relaxation — replacing a large auxiliary linear program with bookkeeping on a tableau the solver already has. This chapter works out that correspondence at the level of individual pivots: which tableau pivot improves the resulting cut, and by how much, answered entirely in terms of ordinary tableau coefficients and two closed-form evaluation functions.

Setting

S:={1,…,m+p} and N:={m+p+1,…,m+p+n} index the surplus and structural variables of (LP) respectively — giving a direct correspondence between (LP)'s own variables and the surplus variables of Ãx≥b̃. For a basic solution with nonbasic set J (row set M1∪M2 from Chapter 8), Â:=Ã_J is the resulting nonsingular submatrix, and row k of the tableau reads x_k+ Σ_{j∈J}ā_{kj}s_j=ā_{k0}. Adding γ times row i to row k gives the composite row (9.10), x_k+γx_i+Σ_{j∈J}(ā_{kj}+γā_{ij})s_j=ā_{k0}+γā_{i0}, from which a new simple disjunctive cut can be read off whenever 0<ā_{k0}+γā_{i0}<1.

Formalization targets

Theorem 9.3 (goal) — the most-improving pivot column

The pivot column in row i most improving the cut from row k is indexed by l*∈J minimizing f⁺(γ_l) (if ā_{kl}ā_{il}<0) or f⁻(γ_l) (if ā_{kl}ā_{il}>0), over all l∈J with -ā_{k0}/ā_{i0}<γ_l<(1-ā_{k0})/ā_{i0}, γ_l:=-ā_{kl}/ā_{il}.

The chain of results building toward it

Lemma 9.1 (the tableau coefficients' closed form, eq. (9.4)-(9.5)) and Theorem 9.2 (the reduced costs of the CGLP columns u_i,v_i in terms of tableau coefficients, eq. (9.6)) are the two milestones the goal's own machinery is built from. Proposition 9.4, a bridge to Chapters 10-11's general split disjunctions, is included as a genuine milestone despite its payoff lying mostly outside this chapter.

Significance

The results themselves. This chapter is what makes lift-and-project cuts practical: instead of solving an (m+p+n)-row auxiliary LP from scratch for every candidate cut, a single pivot on the (LP)'s own tableau — guided by reduced costs that are themselves closed-form functions of tableau entries — identifies whether an improving cut exists and which one it is. Theorem 9.3's evaluation functions f⁺,f⁻ are exactly the tool a cutting-plane implementation would compute at every candidate pivot.

Formalizing it. No object in this mission exists on the platform prior to it or in Mathlib. This mission restates 08-cut-correspondence's (CGLP)_k apparatus locally, per the series convention and BRIEF.md's explicit instruction, and extends it with this chapter's own generalization to an arbitrary tableau row (needed since Lemma 9.1/Theorem 9.2 concern every basic variable's row, not only the disjunction row k).

Difficulty

Lemma 9.1's book proof is a four-case block-matrix verification (structural/surplus, basic/nonbasic); this mission instead states its content as the identity it is actually for — that the closed-form coefficients express every row's slack as an affine function of the nonbasic rows' slacks, for every point x — which follows tautologically from x=Â⁻¹b̂+Â⁻¹s_J's own definition once stated this way, without needing to reconstruct the block-matrix case analysis. Theorem 9.2's difficulty is that "reduced cost" is not already available as a formalized LP concept in this mission's apparatus; rather than build a generic LP reduced-cost theory, this mission follows the book's own derivation directly — explicitly constructing the pivoted-out extension of a basic solution (eq. (9.7)-(9.9)) and asserting that its objective value decomposes with r_{u_i},r_{v_i} as coefficients, which is genuine, non-circular content matching the proof's own final step ("we can then read the reduced costs... as the coefficients").

Formalization scope

This chapter makes the row/variable identification of Chapters 6-8 fully explicit (N directly indexes the structural variables), but no theorem's own displayed formula in this chunk needs that correspondence beyond what SurplusM's row-general treatment (this chunk's own generalization of 08-cut-correspondence's Surplus) already provides — see MODERATION_NOTES.md for why the S/N/B/R/P/Q block structure is proof machinery, not part of the stated content, throughout.

Theorem 9.3's range condition on γ_l, truncated in BRIEF.md's own excerpt, was completed by reading the PDF directly (confirmed identical to the range derived earlier in the same section): -ā_{k0}/ā_{i0}<γ_l<(1-ā_{k0})/ā_{i0}.

Selected references

  • E. Balas, Disjunctive Programming, Springer, 2018. DOI: 10.1007/978-3-030-00148-3, Chapter 9.
  • E. Balas, M. Perregaard, A precise correspondence between lift-and-project cuts, simple disjunctive cuts, and mixed integer Gomory cuts for 0-1 programming, Mathematical Programming B 94 (2003), 221–245 (cited in the text as [33], the origin of the tableau-pivoting procedure this chapter derives Lemma 9.1 and Theorem 9.2 from).
9 thms3 active usersReviewed
🏆Completed
CombinatoricsLinear OptimizationOptimization·Captain: Shuze Chen

Disjunctive Programming VI: Extended Formulations for Perfectly Matchable Subgraph PolytopesTextbook

Motivation

Many polytopes that arise from combinatorial optimization problems have no small facet description in their natural variable space, yet become describable by a compact linear system once lifted to a higher-dimensional space of auxiliary variables and projected back down — Chapter 2's own extended formulation of the convex hull of a disjunctive set is one instance of this phenomenon. This chapter turns the idea around: rather than using projection to build a compact formulation, it uses projection to prove integrality of a formulation that is already compact but whose integrality is not obvious from any standard sufficient condition (total unimodularity, balancedness, etc.). The technique is illustrated on three closely related combinatorial polytopes built from perfectly matchable, assignable, and path-decomposable vertex subsets of a graph or digraph — each proved integral by lifting to an edge- or arc-variable space where total unimodularity is easy to check, then projecting.

Setting

For a finite vertex set VVV, the incidence vector of W⊆VW \subseteq VW⊆V is 111 on WWW, 000 elsewhere, and x(S):=∑i∈Sxix(S) := \sum_{i \in S} x_ix(S):=∑i∈S​xi​. A graph G(W)G(W)G(W) has a perfect matching if there is a fixed-point-free involution on WWW respecting adjacency. The PMS (Perfectly Matchable Subgraph) polytope of GGG is conv(X)\mathrm{conv}(X)conv(X) where XXX is the set of incidence vectors of such WWW; N(S):={j∉S:(i,j)∈E for some i∈S}N(S) := \{j \notin S : (i,j) \in E \text{ for some } i \in S\}N(S):={j∈/S:(i,j)∈E for some i∈S}.

For a digraph (V,A)(V,A)(V,A): G(W)G(W)G(W) is assignable if it admits a cycle decomposition (a permutation of WWW respecting arcs), giving the Assignable Subgraph Polytope. For an acyclic digraph with distinguished nodes s,ts,ts,t: G(W∪{s,t})G(W \cup \{s,t\})G(W∪{s,t}) admits an sss-ttt path decomposition if a collection of interior-node-disjoint sss-ttt paths covers it, giving the sss-ttt Path Decomposable Subgraph Polytope over W⊆V∖{s,t}W \subseteq V \setminus \{s,t\}W⊆V∖{s,t}. Γ(S)\Gamma(S)Γ(S) and Γ∗(S)\Gamma^*(S)Γ∗(S) are the corresponding out-neighborhood operators. For an arbitrary graph, c(S)c(S)c(S) counts the connected components of the induced subgraph G(S)G(S)G(S).

Formalization targets

Theorem 5.1 (goal) — the PMS polytope of a bipartite graph

0≤xi≤1 (i∈V),x(V1)−x(V2)=0,x(S)−x(N(S))≤0  (S⊆V1).0 \le x_i \le 1\ (i \in V), \qquad x(V_1) - x(V_2) = 0, \qquad x(S) - x(N(S)) \le 0\ \ (S \subseteq V_1).0≤xi​≤1 (i∈V),x(V1​)−x(V2​)=0,x(S)−x(N(S))≤0  (S⊆V1​).

Theorem 5.2 — the Assignable Subgraph Polytope

0≤xi≤1 (i∈V),x(S∖Γ(S))−x(Γ(S)∖S)≤0(S⊆V).0 \le x_i \le 1\ (i \in V), \qquad x(S \setminus \Gamma(S)) - x(\Gamma(S) \setminus S) \le 0 \quad (S \subseteq V).0≤xi​≤1 (i∈V),x(S∖Γ(S))−x(Γ(S)∖S)≤0(S⊆V).

Theorem 5.3 — the sss-ttt Path Decomposable Subgraph Polytope

0≤xi≤1 (i∈V),x(S∖Γ∗(S))−x(Γ∗(S)∖S)≤0(S⊆V∖{s,t}).0 \le x_i \le 1\ (i \in V), \qquad x(S \setminus \Gamma^*(S)) - x(\Gamma^*(S) \setminus S) \le 0 \quad (S \subseteq V \setminus \{s,t\}).0≤xi​≤1 (i∈V),x(S∖Γ∗(S))−x(Γ∗(S)∖S)≤0(S⊆V∖{s,t}).

Theorem 5.4 — the PMS polytope of an arbitrary graph

0≤xi≤1 (i∈V),x(S)−x(N(S))≤∣S∣−c(S)0 \le x_i \le 1\ (i \in V), \qquad x(S) - x(N(S)) \le |S| - c(S)0≤xi​≤1 (i∈V),x(S)−x(N(S))≤∣S∣−c(S)

for every SSS all of whose components are single nodes or nonbipartite with odd order — the weakest faithful statement, since dropping the side condition would assert the inequality for subsets it does not hold for.

Significance

The results themselves. Each theorem gives an explicit, checkable linear system defining a polytope that arises naturally from a combinatorial covering/decomposition property, turning "does G(W)G(W)G(W) have property XXX" into a linear-programming feasibility question. Theorem 5.1 is the one the book proves in full and the template for the other three: bipartite matching, digraph assignment, and acyclic-digraph path decomposition are structurally parallel problems (all reduce to checking a König–Hall-type combinatorial condition), and the same lift-and-project technique handles all three uniformly. Theorem 5.4 extends the idea to arbitrary (non-bipartite) graphs at the cost of a sharper right-hand side and a component-based side condition, connecting to Edmonds' classical matching-polytope theory while remaining a genuinely different object (a polytope of coverable vertex sets, not of matchings themselves).

Formalizing it. No object in this mission — the PMS, Assignable, or Path Decomposable Subgraph polytopes, or their defining neighbor operators — exists on the platform prior to this mission. The closest platform result, MetricTSP.pm_polytope_decomposition (Edmonds' perfect matching polytope theorem, in edge-variable space over a fixed vertex set requiring every vertex matched), is a genuinely different object from Theorem 5.4's PMS polytope (vertex-variable space, vertices may be left unmatched by design) and is not reused as a kind: reference item; it is noted here as related, not equivalent.

Difficulty

The natural first attempt tries to verify each polytope's integrality directly, by checking a known sufficient condition (total unimodularity, balancedness) on the displayed vertex-space system itself. This fails: the book states explicitly that (5.5)'s coefficient matrix is not totally unimodular, which is exactly why the lift-to-edge-variables step is necessary at all. The real content of each theorem is the two-part argument: (1) the lifted system in edge/arc variables is totally unimodular (checkable directly), so its polyhedron is integral; and (2) the vertex- space system is exactly the projection of the lifted one — a nontrivial fact requiring Chapter 2's projection machinery, not merely an unfolding of definitions. Theorem 5.4's extra difficulty, flagged explicitly in the text, is that its projection cone is not pointed, so the proof must work with a finite generating set rather than extreme rays, and it suffices to find a subset of generators producing every facet rather than a complete generating set — a genuinely harder argument the book itself outsources to a citation.

Formalization scope

Undirected graphs use Mathlib's SimpleGraph; digraphs use a bare relation A : V → V → Prop (not required symmetric or irreflexive, matching the book's unrestricted notion). Bipartition is recorded via part : V → Bool (decidable by construction) rather than two Set V halves, keeping the sums x(V_1), x(V_2) computable over Finsets throughout. IsAssignable uses Equiv.Perm on the vertex-set subtype, since a cycle decomposition is exactly a permutation. IsComponentOf and IsBipartiteOn (Theorem 5.4) are built directly from reachability and 2-colorability rather than Mathlib's induced-subgraph/ConnectedComponent API, matching the "maximal connected subset" reading of "component" the book's own prose intends.

IsPathDecomposable (Theorem 5.3) encodes "admits an sss-ttt path decomposition" via a degree-constrained arc set (every interior node has exactly one incoming and one outgoing chosen arc, none entering sss or leaving ttt, at least one leaving sss) rather than an explicit list of vertex-disjoint paths — provably equivalent by the standard fact that an acyclic arc set with this degree pattern always decomposes into such a path family, and considerably lighter to state and reason about than constructing Path objects directly.

A trivializing formalization is ruled out explicitly: every theorem keeps the fractional box constraint 0≤xi≤10 \le x_i \le 10≤xi​≤1 rather than the integral xi∈{0,1}x_i \in \{0,1\}xi​∈{0,1} (per BRIEF.md's own warning, dropping the relaxation collapses the claim to a restatement of the combinatorial definition), and Theorem 5.1 is stated only for bipartite graphs — never generalized to subsume Theorem 5.4's genuinely different inequality system and side condition.

Selected references

  • E. Balas, Disjunctive Programming, Springer, 2018. DOI: 10.1007/978-3-030-00148-3, Chapter 5, §5.2.
  • M. O. Ball, U. Derigs, An analysis of alternate strategies for implementing matching algorithms, Networks 13 (1983) (cited in the text as [13], the origin of Theorems 5.2 and 5.3).
  • W. R. Pulleyblank, J. Edmonds, Facets of 1-matching polyhedra, in Hypergraph Seminar, Springer Lecture Notes in Mathematics 411 (1974) — the origin of the perfectly matchable subgraph polytope literature (cited in the text as [34], the origin of Theorem 5.1).
  • L. Lovász, M. D. Plummer, Matching Theory, Elsevier, 1986 (cited in the text as [35], the origin of Theorem 5.4).
7 thms3 active usersReviewed
🏆Completed
Linear OptimizationOptimization·Captain: Shuze Chen

Disjunctive Programming II: The Convex Hull of a Disjunctive Set via Lifting and ProjectionTextbook

Motivation

Convexity is what makes optimization tractable: a linear program's feasible region is convex, and this single fact underwrites the simplex method, LP duality, and everything built on top of them. Integer and disjunctive programs have no such luck — their feasible regions are unions of polyhedra, and a union of convex sets is generally not convex. If the convex hull of such a union could always be described compactly, integer programming would reduce to linear programming: optimize the same linear objective over the hull instead of the union, and any optimal vertex of the hull is automatically integral. The obstacle has always been that the convex hull of a union of polyhedra in Rn\mathbb{R}^nRn, described directly by its facets in Rn\mathbb{R}^nRn, typically needs exponentially many inequalities.

Balas's Theorem 2.1, proved in the 1970s and presented here as Chapter 2 of Disjunctive Programming (Balas, Springer 2018), breaks this exponential barrier by changing where the description lives. Rather than writing down the hull's facets in Rn\mathbb{R}^nRn, Theorem 2.1 lifts the problem to a higher-dimensional space — one auxiliary copy of Rn\mathbb{R}^nRn per polyhedron in the union — where the hull becomes the projection of a single, explicitly given polyhedron whose size grows only linearly with the number of polyhedra. This "extended formulation" technique, born here, became one of the central tools of modern integer programming and combinatorial optimization: representing a hard polytope as the projection of an easy one in higher dimension underlies, for instance, the polynomial-size extended formulations known for many combinatorial polytopes.

Setting

Fix a finite index set QQQ. For h∈Qh \in Qh∈Q, let AhA_hAh​ be a real matrix and bhb_hbh​ a vector of matching row dimension, and set Ph:={x∈Rn:Ahx≥bh}P_h := \{x \in \mathbb{R}^n : A_h x \ge b_h\}Ph​:={x∈Rn:Ah​x≥bh​}. The union F:=⋃h∈QPhF := \bigcup_{h \in Q} P_hF:=⋃h∈Q​Ph​ is the disjunctive set. Write Q∗:={h∈Q:Ph≠∅}Q^* := \{h \in Q : P_h \ne \emptyset\}Q∗:={h∈Q:Ph​=∅} for the feasible disjuncts.

The recession cone of a nonempty polyhedron PhP_hPh​ is Ch:={y:Ahy≥0}C_h := \{y : A_h y \ge 0\}Ch​:={y:Ah​y≥0}: the set of directions along which one can travel indefinitely from any point of PhP_hPh​ while remaining in PhP_hPh​. For a subset M⊆QM \subseteq QM⊆Q and sets ShS_hSh​ (h∈Mh \in Mh∈M), the (finite) Minkowski sum ∑h∈MSh\sum_{h \in M} S_h∑h∈M​Sh​ is {x:x=∑h∈Myh for some yh∈Sh}\{x : x = \sum_{h \in M} y^h \text{ for some } y^h \in S_h\}{x:x=∑h∈M​yh for some yh∈Sh​}. The maximal indices Q∗∗⊆Q∗Q^{**} \subseteq Q^*Q∗∗⊆Q∗ are the feasible disjuncts whose polyhedron is not contained in any other feasible disjunct's polyhedron.

Given a set S⊆Rn×βS \subseteq \mathbb{R}^n \times \betaS⊆Rn×β, its projection onto xxx is Projx(S):={x:∃ y∈β, (x,y)∈S}\mathrm{Proj}_x(S) := \{x : \exists\, y \in \beta,\ (x,y) \in S\}Projx​(S):={x:∃y∈β, (x,y)∈S}.

Formalization targets

Theorem 2.1 (goal) — the convex hull of a disjunctive set

cl conv(F)=Projx(P),P:={(x,{yh}h∈Q∗,{y0h}h∈Q∗):x= ⁣ ⁣∑h∈Q∗ ⁣ ⁣yh, Ahyh−bhy0h≥0, y0h≥0,  ⁣ ⁣∑h∈Q∗ ⁣ ⁣y0h=1}.\mathrm{cl}\,\mathrm{conv}(F) = \mathrm{Proj}_x(P), \qquad P := \Big\{(x, \{y^h\}_{h \in Q^*}, \{y^h_0\}_{h \in Q^*}) : x = \!\!\sum_{h \in Q^*}\!\! y^h,\ A_h y^h - b_h y^h_0 \ge 0,\ y^h_0 \ge 0,\ \!\!\sum_{h \in Q^*}\!\! y^h_0 = 1 \Big\}.clconv(F)=Projx​(P),P:={(x,{yh}h∈Q∗​,{y0h​}h∈Q∗​):x=h∈Q∗∑​yh, Ah​yh−bh​y0h​≥0, y0h​≥0, h∈Q∗∑​y0h​=1}.

This is the weakest correct statement: it claims only that the closed convex hull equals the projection of this specific lifted polyhedron PPP, not any stronger uniqueness or minimality claim about lifted representations in general (that refinement is Theorem 2.1's own follow-up discussion, not part of the theorem itself).

Corollary 2.2 — the extreme-point correspondence

Extreme points of cl conv(F)\mathrm{cl}\,\mathrm{conv}(F)clconv(F) correspond bijectively to the extreme points of PPP that place all of their mass on a single disjunct's coordinates.

Theorem 2.3 — tightness of the lifted representation

PQ=P  ⟺  Ck⊆∑h∈Q∗Ch∀ k∈Q∖Q∗,P_Q = P \iff C_k \subseteq \sum_{h \in Q^*} C_h \quad \forall\, k \in Q \setminus Q^*,PQ​=P⟺Ck​⊆h∈Q∗∑​Ch​∀k∈Q∖Q∗,

where PQP_QPQ​ is the variant of PPP indexed by all of QQQ rather than only Q∗Q^*Q∗.

Theorem 2.4 — from the convex hull to the union itself

Under two recession-cone conditions on Q∗∗Q^{**}Q∗∗, restricting PQP_QPQ​'s y0hy^h_0y0h​ variables to {0,1}\{0,1\}{0,1} makes its xxx-projection recover FFF itself, not merely cl conv(F)\mathrm{cl}\,\mathrm{conv}(F)clconv(F).

Significance

The result itself. Theorem 2.1 is the founding extended-formulation result of integer programming: it shows that every union of finitely many polyhedra — hence every mixed-integer program's feasible region, once expressed in disjunctive normal form — has a lifted description of size linear in the number of disjuncts, in stark contrast to the union's own facet description, which is generally exponential. Corollary 2.2 shows this lifting is not merely an upper bound with extraneous points: its extreme points correspond exactly, one-to-one, with the extreme points of the object it represents. Theorems 2.3 and 2.4 sharpen the picture: 2.3 tells you exactly when you can avoid knowing in advance which disjuncts are nonempty, and 2.4 tells you exactly when the same family of lifted systems, restricted to integral y0hy^h_0y0h​, describes the union FFF exactly rather than only its convex hull — this is Jeroslow and Lowe's characterization of when a disjunctive set is representable as the feasible region of an integer program at all.

Formalizing it. No object in this mission — the disjunctive set FFF, its lifted polyhedron PPP, recession cones of a union's components, or the extreme-point correspondence between a polytope and its lift — exists on the platform prior to this mission or anywhere in Mathlib (substrate.md records zero LP/polyhedron modules in Mathlib as of this writing). This mission is a from-scratch formalization of the book's central construction, restating (rather than importing) the disjunctive-set vocabulary introduced by the companion IntroDuality mission, per the series' convention that a draft mission cannot import another draft mission's definitions.

Difficulty

The natural first attempt at Theorem 2.1 tries to prove the two inclusions cl conv(F)⊆Projx(P)\mathrm{cl}\,\mathrm{conv}(F) \subseteq \mathrm{Proj}_x(P)clconv(F)⊆Projx​(P) and Projx(P)⊆cl conv(F)\mathrm{Proj}_x(P) \subseteq \mathrm{cl}\,\mathrm{conv}(F)Projx​(P)⊆clconv(F) by a direct facet-by-facet or vertex-by-vertex argument in Rn\mathbb{R}^nRn — exactly the exponential-size approach the theorem exists to avoid. The book's own first proof instead works entirely with convex combinations: an arbitrary point of cl conv(F)\mathrm{cl}\,\mathrm{conv}(F)clconv(F) is a combination of at most ∣Q∗∣|Q^*|∣Q∗∣ points, one from each polyhedron in the union (Carathéodory-style), which converts directly into a point of PPP by splitting the combination's weight across the lifted coordinates — and conversely, a point of PPP decomposes, disjunct by disjunct, into a convex combination of that disjunct's own vertices and extreme rays. Neither direction ever needs to enumerate facets of cl conv(F)\mathrm{cl}\,\mathrm{conv}(F)clconv(F) in Rn\mathbb{R}^nRn. The second proof (via projection and the polar cone WWW of the lifted system) shows the projected inequalities coincide with exactly the valid-inequality characterization of Theorem 1.2 (disjunctive Farkas), which is a different, complementary way of seeing why no facet of cl conv(F)\mathrm{cl}\,\mathrm{conv}(F)clconv(F) is missed.

Formalization scope

All theorems are stated over a finite index set Q : Type* with [Fintype Q], matrices Matrix (Fin (m h)) (Fin n) ℝ with m : Q → ℕ allowed to depend on h, and vectors in Fin n → ℝ. DisjunctiveSet, FeasibleIndices, MaximalIndices, RecessionCone, and MinkowskiSumOver fix the chapter's vocabulary; ProjX, LiftedPolyhedron, and IntegerRestricted fix the lifted system and its variants. cl conv F is Mathlib's closure (convexHull ℝ ·); extreme points use Mathlib's Set.extremePoints.

LiftedPolyhedron ranges its auxiliary vectors {yh}\{y^h\}{yh}, {y0h}\{y^h_0\}{y0h​} over all of QQQ rather than only the index subset Qidx the book restricts to, forcing the components outside Qidx to zero. This is an equivalent, Finset/decidability-free encoding — appending zero terms changes neither the defining sums nor the constraints — documented as a convention, not a weakening, in MODERATION_NOTES.md; the same definition instantiates both the (2.1)(2.1)(2.1) system (Qidx = Q^*) and the (2.1)Q(2.1)_Q(2.1)Q​ variant (Qidx = Q) that Theorem 2.3 compares.

A trivializing formalization is ruled out explicitly: taking ∣Q∗∣=1|Q^*| = 1∣Q∗∣=1 collapses the lifted system to x=y1x = y^1x=y1, y01=1y^1_0 = 1y01​=1, a vacuous restatement of x∈P1x \in P_1x∈P1​ that proves nothing about unions. Every theorem here is stated for a generic finite Q, never specialized to a fixed small size. Contributions beyond this mission's statements would need genuine polyhedral machinery (vertex/extreme-ray decomposition of a polyhedron, Carathéodory's theorem for cones) that is itself absent from Mathlib and would be welcome as a separate, reusable definitions layer.

Selected references

  • E. Balas, Disjunctive Programming, Springer, 2018. DOI: 10.1007/978-3-030-00148-3, Chapter 2, §2.1.
  • E. Balas, Disjunctive programming: Properties of the convex hull of feasible points, Discrete Applied Mathematics 89 (1998), 3–44 (reprint of a 1974 MSRR, cited in the text as [6], the origin of Theorem 2.1).
  • M. Conforti, M. Di Summa, Y. Faenza, On the size of extended formulations for polytopes associated with unions of polyhedra, SIAM Journal on Discrete Mathematics, cited in the text as [59] — establishes the tightness (minimum additional-variable count) of Theorem 2.1's lifted representation.
  • R. G. Jeroslow, J. K. Lowe, Modelling with integer variables, Mathematical Programming Study 22 (1984), 167–184 (cited in the text as [86]; the characterization behind Theorem 2.4's significance).
8 thms3 active usersReviewed
🏆Completed
Linear OptimizationOptimization·Captain: Shuze Chen

Disjunctive Programming I: Intersection Cuts and Duality for Disjunctive ProgramsTextbook

Motivation

Linear programming duality is one of the load-bearing facts of optimization: every feasible linear program has a dual whose value matches the primal's, and this correspondence drives the simplex method's stopping criterion, sensitivity analysis, and most complexity results for polyhedral problems. Integer and mixed-integer programs have no such duality theorem in general — the feasible region of a mixed-integer program is not convex, and the entire apparatus of linear programming duality is built on convexity.

Disjunctive programming, introduced by Egon Balas in the early 1970s, closes part of this gap. A disjunctive set is a union of finitely many polyhedra rather than a single polyhedron — the natural convex-analytic shadow of the "either/or" logical structure that integer variables encode (an integer variable's feasible region is a finite union of half-open pieces, hence a disjunction of the linear constraints that pin it to each value). Balas's insight was that disjunctive programs — linear programs whose feasible region is such a union — admit a strong duality theorem of their own, generalizing the linear-programming case rather than replacing it. This mission formalizes that theorem (Theorem 1.5 of Balas, Disjunctive Programming, Springer 2018) together with the two results the same chapter builds around it: the founding construction of the field, the intersection cut (Theorem 1.1, circa 1970), and the disjunctive generalization of Farkas' Lemma (Theorem 1.2), which characterizes every valid inequality — hence every cutting plane — for a disjunctive set.

Setting

Fix a finite index set QQQ. For each h∈Qh \in Qh∈Q, let AhA_hAh​ be a real mh×nm_h \times nmh​×n matrix and bh∈Rmhb_h \in \mathbb{R}^{m_h}bh​∈Rmh​, and set Ph:={x∈Rn:Ahx≥bh}P_h := \{x \in \mathbb{R}^n : A_h x \ge b_h\}Ph​:={x∈Rn:Ah​x≥bh​}. The union F:=⋃h∈QPhF := \bigcup_{h \in Q} P_hF:=⋃h∈Q​Ph​ is a disjunctive set: any (linear) system of inequalities combined with the logical connectives "and", "or", "not" reduces, via its disjunctive normal form, to a set of exactly this shape. Because a union of convex sets need not be convex, FFF is generally nonconvex even though each PhP_hPh​ is a polyhedron.

A disjunctive program minimizes a linear objective over such a union:

(DP)z0=min⁡{cx:x∈⋃h∈QXh},Xh:={x:Ahx≥bh, x≥0}.(DP)\qquad z_0 = \min\Big\{ c x : x \in \textstyle\bigcup_{h \in Q} X_h \Big\}, \qquad X_h := \{x : A_h x \ge b_h,\ x \ge 0\}.(DP)z0​=min{cx:x∈⋃h∈Q​Xh​},Xh​:={x:Ah​x≥bh​, x≥0}.

Its dual (DD)(DD)(DD) pairs a scalar www with one dual multiplier vector uhu_huh​ per disjunct, requiring w≤uhbhw \le u_h b_hw≤uh​bh​ and uhAh≤cu_h A_h \le cuh​Ah​≤c, uh≥0u_h \ge 0uh​≥0, simultaneously for every h∈Qh \in Qh∈Q, and maximizes www. Write Q∗:={h∈Q:Xh≠∅}Q^* := \{h \in Q : X_h \ne \emptyset\}Q∗:={h∈Q:Xh​=∅} for the disjuncts whose primal system is feasible, and Q∗∗:={h∈Q:Uh≠∅}Q^{**} := \{h \in Q : U_h \ne \emptyset\}Q∗∗:={h∈Q:Uh​=∅} (with Uh:={uh≥0:uhAh≤c}U_h := \{u_h \ge 0 : u_h A_h \le c\}Uh​:={uh​≥0:uh​Ah​≤c}) for those whose dual system is feasible.

The theorems below also use two objects from the origin of the subject (§1.2): given a basic solution xˉ\bar xxˉ of a linear program's optimal simplex tableau, with basic index set III and nonbasic index set JJJ, the tableau's coefficients aˉij\bar a_{ij}aˉij​ (i∈Ii \in Ii∈I, j∈Jj \in Jj∈J) determine, for each nonbasic jjj, an extreme ray direction rjr^jrj of the associated LP cone. A convex set SSS is PIP_IPI​-free at xˉ\bar xxˉ if xˉ\bar xxˉ lies in the interior of SSS and that interior contains no point of the mixed-integer feasible set PIP_IPI​.

Formalization targets

Theorem 1.1 — the intersection cut

λj∗:=max⁡{λj≥0:xˉ+λjrj∈S},∑j∈J1λj∗ xj≥1.\lambda^*_j := \max\{\lambda_j \ge 0 : \bar x + \lambda_j r^j \in S\}, \qquad \sum_{j \in J} \frac{1}{\lambda^*_j}\, x_j \ge 1.λj∗​:=max{λj​≥0:xˉ+λj​rj∈S},j∈J∑​λj∗​1​xj​≥1.

The displayed inequality cuts off xˉ\bar xxˉ but excludes no point of PIP_IPI​, for any PIP_IPI​-free convex set SSS containing xˉ\bar xxˉ in its interior.

Theorem 1.2 — Farkas' Lemma for Disjunctive Sets

(∀x∈F, αx≥α0)  ⟺  (∀h∈Q∗, ∃ uh≥0, uhAh=α, α0≤uhbh).\big(\forall x \in F,\ \alpha x \ge \alpha_0\big) \iff \big(\forall h \in Q^*,\ \exists\, u_h \ge 0,\ u_h A_h = \alpha,\ \alpha_0 \le u_h b_h\big).(∀x∈F, αx≥α0​)⟺(∀h∈Q∗, ∃uh​≥0, uh​Ah​=α, α0​≤uh​bh​).

Theorem 1.5 (goal) — duality for disjunctive programs

Under the Regularity Condition — (Q∗≠∅(Q^* \ne \emptyset(Q∗=∅ and Q∖Q∗∗≠∅)⇒Q∗∖Q∗∗≠∅Q \setminus Q^{**} \ne \emptyset) \Rightarrow Q^* \setminus Q^{**} \ne \emptysetQ∖Q∗∗=∅)⇒Q∗∖Q∗∗=∅ — exactly one of:

  1. both (DP)(DP)(DP) and (DD)(DD)(DD) are feasible, each attains an optimum, and z0=w0z_0 = w_0z0​=w0​; or
  2. one of the two is infeasible, and the other is infeasible or has no finite optimum.

This is the weakest faithful statement of the theorem: it asserts only the shape of the dichotomy established by Balas, not any strengthened or specialized form of it.

Corollary 1.6 — necessity of the Regularity Condition

If the Regularity Condition fails, (DP)(DP)(DP) is feasible, and (DD)(DD)(DD) is infeasible, then (DP)(DP)(DP) still has a finite minimum — exhibiting the duality gap that opens up once the condition is dropped.

Significance

The results themselves. Theorem 1.5 is the mission-critical fact that makes disjunctive programming a genuine extension of linear programming rather than an unrelated combinatorial device: every LP-duality-based algorithmic tool (bounding, sensitivity, complementary-slackness optimality certificates) has a disjunctive-programming counterpart because of this theorem. Theorem 1.1's intersection cut is the historical seed of an entire branch of integer-programming algorithms — lift-and-project cuts, mixed-integer Gomory cuts, and the split closure (later missions of this series) all specialize or generalize it. Theorem 1.2 is the structural fact that makes cutting-plane generation for disjunctive sets tractable at all: every valid inequality decomposes into per-disjunct Farkas certificates.

Formalizing it. None of these results, nor the union-of-polyhedra machinery they are stated over, exist on the platform prior to this mission: the platform's existing Farkas' Lemma and linear-programming strong duality theorems (SmaleNinth.farkas_lemma, SmaleNinth.lp_strong_duality) are the ordinary single-polyhedron statements, which is exactly the special case ∣Q∣=1|Q|=1∣Q∣=1 of the theorems formalized here — genuinely different statements, not restatements. This mission is a from-scratch formalization of the disjunctive generalization, including the vocabulary (disjunctive sets, the paired primal/dual index sets Q∗,Q∗∗Q^*, Q^{**}Q∗,Q∗∗, the Regularity Condition) that the rest of the fifteen-mission Balas series builds on.

Difficulty

The obvious first attempt collapses the disjunctive dual (DD)(DD)(DD) to ∣Q∣|Q|∣Q∣ separate ordinary LP duals, one per disjunct, and tries to combine their individual strong-duality statements. This fails: (DD)(DD)(DD) couples all disjuncts through the single shared scalar www, which must simultaneously satisfy w≤uhbhw \le u_h b_hw≤uh​bh​ for every h∈Qh \in Qh∈Q at once, not disjunct-by-disjunct. The Regularity Condition exists precisely because this coupling can break down — Balas's own example (a two-term disjunctive program with an infeasible dual but a feasible, bounded primal) shows that without the condition, situation (2) of the dichotomy can fail: the primal can have a finite optimum with no matching dual optimum. Any formalization that omits the Regularity Condition, or weakens it to an informal restriction like "nondegenerate", either proves a false statement or proves nothing (a vacuous hypothesis), which Corollary 1.6 exists specifically to rule out.

Formalization scope

All three theorems are stated over a finite index set Q : Type* with [Fintype Q], real matrices Matrix (Fin (m h)) (Fin n) ℝ with row-dimension m : Q → ℕ allowed to depend on h (the book never assumes a common row count across disjuncts), and vectors in Fin n → ℝ. Poly, PolyNonneg, and DualPoly are the plain, nonnegative-orthant, and dual polyhedral systems respectively; FeasibleIndices and RegularityCondition pin Q∗Q^*Q∗/Q∗∗Q^{**}Q∗∗ and the Regularity Condition exactly as stated on p. 13. "No finite optimum" is formalized via UnboundedBelowOn / UnboundedAboveOn: nonempty (feasible) together with no finite bound on the objective, matching Balas's case (2), which explicitly distinguishes infeasibility from unboundedness.

A trivializing formalization is ruled out explicitly: fixing ∣Q∣=1|Q| = 1∣Q∣=1 collapses Theorem 1.5 to ordinary LP duality (already on the platform) and Theorem 1.2 to ordinary Farkas' Lemma, so both theorems are stated for a generic finite Q, never specialized. Theorem 1.5's "exactly one of" dichotomy is formalized as a logical Xor of the two situations, not a weaker Or, since the book asserts mutual exclusivity, not merely that one holds.

The intersection-cut theorem (1.1) is formalized over a generic finite index type ι standing for the full set of structural and surplus variables, with I J : Finset ι the basic/nonbasic partition; a complete development would additionally need the simplex-tableau apparatus connecting ι, I, J, and abar to an actual linear program, which lies outside this mission and belongs instead to the tableau-focused later missions of the series (SimplexTableau, RayCGLP). The extremeRay and PIFree definitions introduced here are local to this mission and are restated, not imported, by later missions that need related vocabulary — per the series' convention that a draft mission cannot import another draft mission's definitions.

Selected references

  • E. Balas, Disjunctive Programming, Springer, 2018. DOI: 10.1007/978-3-030-00148-3, Chapter 1.
  • E. Balas, Intersection cuts — a new type of cutting planes for integer programming, Operations Research 19 (1971), 19–39. (Theorem 1.1's origin, cited in the text as [4].)
  • E. Balas, Disjunctive programming, Annals of Discrete Mathematics 5 (1979), 3–51. (Cited in the text as [9], the origin of Theorem 1.5.)
9 thms3 active usersReviewed
🏆Completed
Convex OptimizationLinear OptimizationOptimization·Captain: mikedeng1

Cones of Matrices and Set-Functions and 0–1 Optimization I: n Rounds of the Lovász–Schrijver N Operator Give the 0–1 HullResearch Paper

Motivation

A 0–1 integer program asks for the best 0–1 vector satisfying a system of linear inequalities. Its linear relaxation is easy to optimize over, but the relaxation is usually much larger than the convex hull of the 0–1 solutions. Lift-and-project methods close this gap systematically: they lift the relaxation to a higher-dimensional space, add constraints that every 0–1 point satisfies there, and project back, obtaining a tighter relaxation that still contains every 0–1 solution.

L. Lovász and A. Schrijver introduced one of the two standard lift-and-project hierarchies in Cones of matrices and set-functions and 0–1 optimization (SIAM J. Optim., 1991). Their operators NNN and N+N_+N+​ represent a 0–1 point xxx by the matrix xxTxx^{\mathsf T}xxT, impose linear (and for N+N_+N+​ semidefinite) constraints on such matrices, and project back to Rn+1\mathbb R^{n+1}Rn+1. The same paper applies the operators to the stable set polytope, where one round already produces the odd hole, odd wheel, clique and odd antihole constraints. The Lovász–Schrijver hierarchy, the Sherali–Adams hierarchy (1990) and Lasserre's semidefinite hierarchy (2001) are the three reference lift-and-project methods; their rank lower bounds are a standard tool for proving that a relaxation cannot solve a combinatorial problem in few rounds.

This mission formalizes the first structural fact about the operator NNN: iterating it nnn times on any relaxation in nnn variables yields exactly the 0–1 hull (Theorem 1.4 of the paper).

Setting

Vectors live in Rn+1\mathbb R^{n+1}Rn+1 with coordinates x0,x1,…,xnx_0, x_1, \dots, x_nx0​,x1​,…,xn​; the space Rn\mathbb R^nRn of the original problem is the hyperplane x0=1x_0 = 1x0​=1, and polytopes are replaced by the convex cones they generate.

  • A convex cone is a nonempty set closed under addition and nonnegative scaling. For a set SSS, cone⁡(S)\operatorname{cone}(S)cone(S) is the set of nonnegative combinations of finitely many vectors of SSS.
  • The polar cone of KKK is K∗={u:uTx≥0 for all x∈K}K^* = \{u : u^{\mathsf T}x \ge 0 \text{ for all } x \in K\}K∗={u:uTx≥0 for all x∈K}.
  • A 0–1 vector has every coordinate, x0x_0x0​ included, equal to 000 or 111. The cube cone QQQ is the cone spanned by the 0–1 vectors with x0=1x_0 = 1x0​=1; it is the cone over the unit cube.
  • For a convex cone KKK, K∘K^\circK∘ is the cone spanned by the 0–1 vectors in KKK. For K⊆QK \subseteq QK⊆Q this is the cone over the convex hull of the 0–1 points of the relaxation.

For convex cones K1,K2⊆QK_1, K_2 \subseteq QK1​,K2​⊆Q, the matrix cone M(K1,K2)M(K_1, K_2)M(K1​,K2​) consists of the (n+1)×(n+1)(n+1)\times(n+1)(n+1)×(n+1) real matrices Y=(yij)Y = (y_{ij})Y=(yij​) such that

  1. YYY is symmetric;
  2. yii=y0iy_{ii} = y_{0i}yii​=y0i​ for 1≤i≤n1 \le i \le n1≤i≤n (the diagonal equals the 0th column);
  3. uTYv≥0u^{\mathsf T} Y v \ge 0uTYv≥0 for every u∈K1∗u \in K_1^*u∈K1∗​ and v∈K2∗v \in K_2^*v∈K2∗​.

M+(K1,K2)M_+(K_1, K_2)M+​(K1​,K2​) adds the condition that YYY is positive semidefinite. The projections are N(K1,K2)={Ye0:Y∈M(K1,K2)}N(K_1, K_2) = \{Ye_0 : Y \in M(K_1, K_2)\}N(K1​,K2​)={Ye0​:Y∈M(K1​,K2​)} and N+(K1,K2)={Ye0:Y∈M+(K1,K2)}N_+(K_1, K_2) = \{Ye_0 : Y \in M_+(K_1, K_2)\}N+​(K1​,K2​)={Ye0​:Y∈M+​(K1​,K2​)}, where e0e_0e0​ is the 0th unit vector. The cut operator is N(K)=N(K,Q)N(K) = N(K, Q)N(K)=N(K,Q), and its iterates are N0(K)=KN^0(K) = KN0(K)=K, Nt(K)=N(Nt−1(K))N^t(K) = N(N^{t-1}(K))Nt(K)=N(Nt−1(K)).

Two families of hyperplanes appear in the proofs: Hi={x:xi=0}H_i = \{x : x_i = 0\}Hi​={x:xi​=0} and Gi={x:xi=x0}G_i = \{x : x_i = x_0\}Gi​={x:xi​=x0​}, the hyperplanes through the two opposite facets of QQQ in direction iii.

Formalization targets

Goal: Theorem 1.4

For every closed convex cone K⊆QK \subseteq QK⊆Q,

Nn(K)=K∘.N^n(K) = K^\circ .Nn(K)=K∘.

The statement is uniform in nnn and in KKK: no polyhedrality, no bound on the number of constraints, and no assumption that KKK contains a 0–1 point.

Milestones

  1. Condition (iii″). For a closed convex cone K⊆QK \subseteq QK⊆Q and a symmetric YYY with yii=y0iy_{ii} = y_{0i}yii​=y0i​: Y∈M(K,Q)Y \in M(K, Q)Y∈M(K,Q) if and only if every column of YYY is in KKK and the difference of the first column and any other column is in KKK.
  2. Lemma 1.1. For closed convex cones K1,K2⊆QK_1, K_2 \subseteq QK1​,K2​⊆Q,
(K1∩K2)∘⊆N+(K1,K2)⊆N(K1,K2)⊆K1∩K2.(K_1 \cap K_2)^\circ \subseteq N_+(K_1, K_2) \subseteq N(K_1, K_2) \subseteq K_1 \cap K_2 .(K1​∩K2​)∘⊆N+​(K1​,K2​)⊆N(K1​,K2​)⊆K1​∩K2​.
  1. Lemma 1.3. For a closed convex cone K⊆QK \subseteq QK⊆Q and every 1≤i≤n1 \le i \le n1≤i≤n,
N(K)⊆(K∩Hi)+(K∩Gi).N(K) \subseteq (K \cap H_i) + (K \cap G_i).N(K)⊆(K∩Hi​)+(K∩Gi​).
  1. Claim (4) in the proof of Theorem 1.4. For every set TTT of t≥1t \ge 1t≥1 coordinates, with Fˉ\bar FFˉ the union of the faces of the unit cube that fix the coordinates in TTT to 000 or 111,
Nt(K)⊆cone⁡(K∩Fˉ).N^t(K) \subseteq \operatorname{cone}(K \cap \bar F).Nt(K)⊆cone(K∩Fˉ).
  1. The remark after Lemma 1.1. N(K1∩K2,K1∩K2)⊆N(K1,K2)⊆N(K1∩K2,Q)N(K_1 \cap K_2, K_1 \cap K_2) \subseteq N(K_1, K_2) \subseteq N(K_1 \cap K_2, Q)N(K1​∩K2​,K1​∩K2​)⊆N(K1​,K2​)⊆N(K1​∩K2​,Q).

Significance

Theorem 1.4 is what makes NNN a hierarchy rather than a single cut: the relaxations K⊇N(K)⊇N2(K)⊇…K \supseteq N(K) \supseteq N^2(K) \supseteq \dotsK⊇N(K)⊇N2(K)⊇… reach the 0–1 hull after at most nnn rounds, so the NNN-rank of a valid inequality (the least ttt with the inequality valid for Nt(K)N^t(K)Nt(K)) is a well-defined number between 000 and nnn. The rest of the paper measures combinatorial constraints by this rank: odd hole constraints have rank one on the stable set polytope, and the rank of a stable set inequality is bounded by its defect. Rank lower bounds for lift-and-project hierarchies, in the literature that followed, all presuppose this finite convergence.

The theorem is proved in the paper; to the best of available knowledge none of the Lovász–Schrijver operators has been formalized in a proof assistant. A formalization provides machine-checked definitions of the matrix cones and the cut operators that later missions in this series (odd holes, the defect bound, the N+N_+N+​ constraints) state their results against, and a checked proof of the column characterization (iii″) that all of those proofs use.

Difficulty

The inclusion K∘⊆Nn(K)K^\circ \subseteq N^n(K)K∘⊆Nn(K) follows from Lemma 1.1 once each Nt(K)N^t(K)Nt(K) is known to be a convex cone. The reverse inclusion is the content. A first attempt shows that one round of NNN forces one coordinate to be integral, and then iterates; but N(K)N(K)N(K) is not contained in the union of K∩HiK \cap H_iK∩Hi​ and K∩GiK \cap G_iK∩Gi​, only in their Minkowski sum (Lemma 1.3), so a point of N(K)N(K)N(K) is not itself integral in any coordinate. The induction must carry a statement about cones spanned by intersections with unions of cube faces, and it needs each iterate Nt(K)N^t(K)Nt(K) to again be a closed convex cone inside QQQ so that Lemma 1.3 can be reapplied. Closedness of the projection N(K)N(K)N(K) is not automatic: a linear image of a closed cone need not be closed.

Formalization scope

  • Coordinates of Rn+1\mathbb R^{n+1}Rn+1 are indexed by Option ι for a finite type ι; none is x0x_0x0​ and some i is xix_ixi​, and nnn is the cardinality of ι, which may be 000.
  • cone⁡(S)\operatorname{cone}(S)cone(S) is Mathlib's PointedCone.hull ℝ S; QQQ and K∘K^\circK∘ are defined as spans of 0–1 vectors, as on the page, not by the inequality description 0≤xi≤x00 \le x_i \le x_00≤xi​≤x0​.
  • M(K1,K2)M(K_1, K_2)M(K1​,K2​) is defined by condition (iii) through the polar cones; the column form (iii″) is a milestone, not the definition.
  • The operators NNN, N+N_+N+​ and the iterates are defined on arbitrary sets; the hypotheses (convex cone, contained in QQQ, closed) are carried by the theorems.
  • Closedness. The paper tacitly takes its cones closed (they are polyhedral in all its applications), and the rewriting (iii′) on p. 169 needs it. Every statement here assumes the cones closed. Without this the goal is false: for K={x:0<x1<x0}∪{0}K = \{x : 0 < x_1 < x_0\} \cup \{0\}K={x:0<x1​<x0​}∪{0} in R2\mathbb R^2R2, K∘={0}K^\circ = \{0\}K∘={0} while N(K)=QN(K) = QN(K)=Q.
  • In the proof of Theorem 1.4 the page places the cube Q′Q'Q′ in the hyperplane "x0=0x_0 = 0x0​=0"; this is a misprint for x0=1x_0 = 1x0​=1, and claim (4) is formalized with x0=1x_0 = 1x0​=1.
  • Not formalized in this mission: Lemma 1.2 (the dual description of N(K)∗N(K)^*N(K)∗), Lemma 1.5 (the N+N_+N+​ analogue of Lemma 1.3, part of a later mission), and the algorithmic results of Section 1.c.

Contributions welcome: proofs that N(K)N(K)N(K) is a closed convex cone contained in QQQ whenever KKK is, a proof of Q∗=cone⁡{ei,e0−ei}Q^* = \operatorname{cone}\{e_i, e_0 - e_i\}Q∗=cone{ei​,e0​−ei​}, and lemmas on cones spanned by the intersection of a generating set with a supporting hyperplane; these are reusable by the other missions of the series.

Selected references

  • L. Lovász and A. Schrijver, Cones of matrices and set-functions and 0–1 optimization, SIAM Journal on Optimization 1(2) (1991) 166–190. https://doi.org/10.1137/0801013
  • H. D. Sherali and W. P. Adams, A hierarchy of relaxations between the continuous and convex hull representations for zero-one programming problems, SIAM Journal on Discrete Mathematics 3(3) (1990) 411–430. https://doi.org/10.1137/0403036
  • J. B. Lasserre, Global optimization with polynomials and the problem of moments, SIAM Journal on Optimization 11(3) (2001) 796–817. https://doi.org/10.1137/S1052623400366802
  • M. Laurent, A comparison of the Sherali–Adams, Lovász–Schrijver, and Lasserre relaxations for 0–1 programming, Mathematics of Operations Research 28(3) (2003) 470–496. https://doi.org/10.1287/moor.28.3.470.16391
8 thms3 active usersReviewed
🏆Completed
Control TheoryTheoretical Computer Science·Captain: mikedeng1

Supervisory Control of a Class of Discrete Event Processes I: Minimally Restrictive Supervisors Exist iff the Supremal Controllable Legal Sublanguage Contains the Minimal Acceptable LanguageResearch Paper

Motivation

Manufacturing cells, communication protocols, traffic systems and database transaction managers are naturally described not by differential equations but by sequences of discrete events: a machine starts, a part arrives, a message is lost. Ramadge and Wonham's 1987 paper (SIAM J. Control Optim. 25(1)) set up a control theory for such systems in which the plant is an automaton, the controller is another automaton that may disable some events, and specifications are formal languages. The framework, now called supervisory control theory or the Ramadge–Wonham framework, is the standard model for the logical control of discrete event systems and underlies the textbook treatment in Cassandras and Lafortune (Introduction to Discrete Event Systems, 2008) and Wonham and Cai (Supervisory Control of Discrete-Event Systems, 2019).

The question the paper answers is the basic synthesis question of the theory: given a plant, a set of legal behaviours and a set of minimally acceptable behaviours, when does a controller exist that keeps the plant legal, achieves at least the acceptable behaviour, and never deadlocks, and what is the least restrictive such controller?

Setting

A generator is G=(Q,Σ,δ,q0,Qm)\mathcal G = (Q, \Sigma, \delta, q_0, Q_m)G=(Q,Σ,δ,q0​,Qm​) with a state set QQQ, a finite alphabet Σ\SigmaΣ of events, a partial transition function δ:Σ×Q→Q\delta : \Sigma \times Q \to Qδ:Σ×Q→Q, an initial state q0q_0q0​ and marker states Qm⊆QQ_m \subseteq QQm​⊆Q. Extending δ\deltaδ to strings, the generated language L(G)L(\mathcal G)L(G) is the set of strings www for which δ(w,q0)\delta(w, q_0)δ(w,q0​) is defined, and the marked language Lm(G)L_m(\mathcal G)Lm​(G) is the subset of those that end in QmQ_mQm​. The closure Kˉ\bar KKˉ of a language KKK is its set of prefixes; KKK is closed if K=KˉK = \bar KK=Kˉ. Throughout, G\mathcal GG is assumed trim in the sense L(G)=Lˉm(G)L(\mathcal G) = \bar L_m(\mathcal G)L(G)=Lˉm​(G): every generated string can be completed to a marked one.

The alphabet is split into controllable events Σc\Sigma_cΣc​ and uncontrollable events Σu=Σ−Σc\Sigma_u = \Sigma - \Sigma_cΣu​=Σ−Σc​. A supervisor S=(S,ϕ)\mathcal S = (S, \phi)S=(S,ϕ) consists of a deterministic, accessible automaton S=(X,Σ,ξ,x0,Xm)S = (X, \Sigma, \xi, x_0, X_m)S=(X,Σ,ξ,x0​,Xm​), whose state set XXX may be infinite, and a map ϕ\phiϕ assigning to each state xxx the set of controllable events it enables; uncontrollable events are always enabled. The closed loop S/G\mathcal S/\mathcal GS/G runs SSS and G\mathcal GG in lockstep: an event occurs when the plant can execute it, the supervisor enables it, and the supervisor's automaton can follow it. This defines the languages L(S/G)L(\mathcal S/\mathcal G)L(S/G), Lm(S/G)L_m(\mathcal S/\mathcal G)Lm​(S/G) and the controlled language Lc(S/G)=L(S/G)∩Lm(G)L_c(\mathcal S/\mathcal G) = L(\mathcal S/\mathcal G) \cap L_m(\mathcal G)Lc​(S/G)=L(S/G)∩Lm​(G).

S\mathcal SS is complete if its automaton never refuses an event that the plant can execute and ϕ\phiϕ enables; it is proper if it is complete and

Lˉm(S/G)=Lˉc(S/G)=L(S/G),\bar L_m(\mathcal S/\mathcal G) = \bar L_c(\mathcal S/\mathcal G) = L(\mathcal S/\mathcal G),Lˉm​(S/G)=Lˉc​(S/G)=L(S/G),

that is, every closed-loop string can be completed to a marked task. A language KKK is controllable if K⊆L(G)K \subseteq L(\mathcal G)K⊆L(G) and KˉΣu∩L(G)⊆Kˉ\bar K \Sigma_u \cap L(\mathcal G) \subseteq \bar KKˉΣu​∩L(G)⊆Kˉ. For L⊆L(G)L \subseteq L(\mathcal G)L⊆L(G), CG(L)\mathbf C_{\mathcal G}(L)CG​(L) is the class of controllable sublanguages of LLL and FG(L)\mathbf F_{\mathcal G}(L)FG​(L) the class of sublanguages KKK of LLL with K=Kˉ∩Lm(G)K = \bar K \cap L_m(\mathcal G)K=Kˉ∩Lm​(G).

Given ∅≠La⊆Lg⊆Lm(G)\emptyset \neq L_a \subseteq L_g \subseteq L_m(\mathcal G)∅=La​⊆Lg​⊆Lm​(G), the Supervisory Marking Problem (SMP) asks for a proper S\mathcal SS with La⊆Lm(S/G)⊆LgL_a \subseteq L_m(\mathcal S/\mathcal G) \subseteq L_gLa​⊆Lm​(S/G)⊆Lg​, and the Supervisory Control Problem (SCP) for a proper S\mathcal SS with La⊆Lc(S/G)⊆LgL_a \subseteq L_c(\mathcal S/\mathcal G) \subseteq L_gLa​⊆Lc​(S/G)⊆Lg​.

Formalization targets

Goal: Theorem 7.1 (pp. 218–219)

SMP solvable  ⟺  sup⁡CG(Lg)⊇La,SCP solvable  ⟺  sup⁡{CG(Lg)∩FG(Lg)}⊇La,\text{SMP solvable} \iff \sup \mathbf C_{\mathcal G}(L_g) \supseteq L_a, \qquad \text{SCP solvable} \iff \sup\{\mathbf C_{\mathcal G}(L_g) \cap \mathbf F_{\mathcal G}(L_g)\} \supseteq L_a,SMP solvable⟺supCG​(Lg​)⊇La​,SCP solvable⟺sup{CG​(Lg​)∩FG​(Lg​)}⊇La​,

and in each case the solving supervisor can be taken minimally restrictive: its marked (respectively controlled) language equals the supremal element and contains that of every proper supervisor whose language lies in LgL_gLg​.

Milestones

  1. Proposition 4.1 (i), (ii): marking is independent of control; every K⊆Lm(G)K \subseteq L_m(\mathcal G)K⊆Lm​(G), or K⊆L∩Lm(G)K \subseteq L \cap L_m(\mathcal G)K⊆L∩Lm​(G) for an achievable closed LLL, is the marked language of a complete supervisor.
  2. Proposition 5.1: a complete supervisor realizes (Lm,Lc,L)=(K1,K2,K3)(L_m, L_c, L) = (K_1, K_2, K_3)(Lm​,Lc​,L)=(K1​,K2​,K3​) iff K1⊆K2K_1 \subseteq K_2K1​⊆K2​, K2=K3∩Lm(G)K_2 = K_3 \cap L_m(\mathcal G)K2​=K3​∩Lm​(G) and K3K_3K3​ is closed and controllable.
  3. Theorem 6.1 (i), (ii): a proper supervisor with Lm(S/G)=KL_m(\mathcal S/\mathcal G) = KLm​(S/G)=K exists iff KKK is controllable; one with Lc(S/G)=KL_c(\mathcal S/\mathcal G) = KLc​(S/G)=K exists iff KKK is controllable and Lm(G)L_m(\mathcal G)Lm​(G)-closed.
  4. Proposition 7.1: CG(L)\mathbf C_{\mathcal G}(L)CG​(L) and FG(L)\mathbf F_{\mathcal G}(L)FG​(L) contain ∅\emptyset∅ and are closed under arbitrary unions.
  5. The supremal elements (p. 218): sup⁡CG(L)\sup \mathbf C_{\mathcal G}(L)supCG​(L), sup⁡FG(L)\sup \mathbf F_{\mathcal G}(L)supFG​(L) and sup⁡{CG(L)∩FG(L)}\sup\{\mathbf C_{\mathcal G}(L) \cap \mathbf F_{\mathcal G}(L)\}sup{CG​(L)∩FG​(L)} belong to their classes.

Significance

Theorem 7.1 reduces the existence of a correct, non-blocking controller to a single language inclusion, and identifies the supremal controllable sublanguage as the behaviour of the least restrictive solution. That object is the backbone of the later theory: modular and decentralized control, control under partial observation, and the computational results that sup⁡CG(L)\sup \mathbf C_{\mathcal G}(L)supCG​(L) is regular and computable when G\mathcal GG is finite and LLL regular all start from it.

The result is proved in the paper and has been taught for decades; it is not open. Mathlib has no model of generators with partial transitions, supervisors or controllability (its DFA has a total transition function and a single accepted language), and no formalization of this theory exists on the platform. This mission provides one: a reusable Lean model of generators with partial transitions, supervisors with possibly infinite state, closed loops and controllability, together with the paper's existence theorems stated against it. A second mission on the same paper (quotients of supervisors, Theorem 10.1) builds on the same objects.

Difficulty

The combinatorial core of Proposition 7.1 is a short prefix-closure computation. The work lies in the constructions: to show existence, a supervisor must be built for an arbitrary controllable language, which in general is not regular, so the supervisor needs an infinite state set (for example strings of the target language) together with a proof that the closed loop generates exactly the intended language, is complete, and is proper. The converse directions require relating the closed-loop run to separate runs of the plant and of the supervisor's automaton. The naive shortcut of reading the "sup" as an arbitrary member of CG(Lg)\mathbf C_{\mathcal G}(L_g)CG​(Lg​) containing LaL_aLa​ skips the content of the supremal-element milestone; the goal is stated with the actual union.

Formalization scope

  • The alphabet is a type α with [Fintype α]; strings are List α, the empty string is [], and sσs\sigmasσ is s ++ [σ]. Languages are Set (List α); the closure is pre K = {s | ∃ t, s ++ t ∈ K}.
  • A generator is a structure with a state type Q : Type, a partial transition δ : α → Q → Option Q, an initial state and a marker set. No finiteness is assumed on states, of the plant or of the supervisor. Supervisors and generators live in Type 1.
  • ϕ\phiϕ maps states to Ec → Bool; an event outside Σc\Sigma_cΣc​ is enabled by definition, so uncontrollable events cannot be disabled.
  • The closed loop is the product run from (x0,q0)(x_0, q_0)(x0​,q0​); the accessible part in the paper's display (2.1) changes no language and is not built.
  • Standing assumptions carried by every theorem: Σ\SigmaΣ finite, L(G)=Lˉm(G)L(\mathcal G) = \bar L_m(\mathcal G)L(G)=Lˉm​(G), supervisor automata accessible (as a hypothesis on every input supervisor and a conjunct of every "there exists a supervisor").
  • sup⁡\supsup is sSup in the complete lattice Set (List α).
  • A formalization in which the closed loop ignores ϕ\phiϕ or the plant, in which completeness is dropped from properness, or in which the supervisor is restricted to finitely many states, is not the paper's theorem and is ruled out by the definitions above.

Welcome contributions: the basic run lemmas (closed-loop runs project to plant and supervisor runs), the string-state supervisor construction, and proofs of the milestones in the given order.

Selected references

  • P. J. Ramadge and W. M. Wonham, Supervisory Control of a Class of Discrete Event Processes, SIAM J. Control Optim. 25(1):206–230, 1987. https://doi.org/10.1137/0325013
  • W. M. Wonham and P. J. Ramadge, On the Supremal Controllable Sublanguage of a Given Language, SIAM J. Control Optim. 25(3):637–659, 1987. https://doi.org/10.1137/0325036
  • C. G. Cassandras and S. Lafortune, Introduction to Discrete Event Systems, 2nd ed., Springer, 2008. https://doi.org/10.1007/978-0-387-68612-7
  • W. M. Wonham and K. Cai, Supervisory Control of Discrete-Event Systems, Springer, 2019. https://doi.org/10.1007/978-3-319-77452-7
12 thms3 active usersReviewed
🏆Completed
CombinatoricsGraph TheoryLinear algebra+1·Captain: mikedeng1

Approximating Clique-Width and Branch-Width: Well-Linked Sets Certify Clique-WidthResearch Paper

Motivation

Clique-width is a graph parameter introduced by Courcelle and Olariu (Discrete Appl. Math. 101 (2000)) that measures how far a graph is from being built by a few labelled operations. Every problem expressible in monadic second-order logic with quantification over vertices and vertex sets (MSO1_11​) can be solved in linear time on graphs given together with a decomposition of bounded clique-width (Courcelle, Makowsky and Rotics, Theory Comput. Syst. 33 (2000)). Bounded clique-width is more general than bounded tree-width: complete graphs have unbounded tree-width but clique-width 222.

For fixed kkk there was, before this paper, no polynomial-time algorithm that either decides that a graph has clique-width at least k+1k+1k+1 or outputs a decomposition of clique-width bounded by a function of kkk; the best known algorithm, by Johansson (2001), gave width 2klog⁡n2k\log n2klogn. Oum and Seymour (J. Combin. Theory Ser. B 96 (2006)) closed this gap with approximation 23k+2−12^{3k+2}-123k+2−1, through rank-width and a factor-3 approximation for the branch-width of symmetric submodular functions.

Timeline:

  • 1991: Robertson and Seymour introduce branch-width of graphs and hypergraphs (J. Combin. Theory Ser. B 52).
  • 2000: Courcelle and Olariu define clique-width; Courcelle, Makowsky and Rotics solve MSO1_11​ problems on graphs given with a kkk-expression.
  • 2001: Johansson gives a 2klog⁡n2k\log n2klogn approximation.
  • 2006: Oum and Seymour define rank-width, prove rwd(G)≤cwd(G)≤2rwd(G)+1−1\mathrm{rwd}(G) \le \mathrm{cwd}(G) \le 2^{\mathrm{rwd}(G)+1}-1rwd(G)≤cwd(G)≤2rwd(G)+1−1, and give an O(n9log⁡n)O(n^9 \log n)O(n9logn) algorithm that outputs a (23k+2−1)(2^{3k+2}-1)(23k+2−1)-expression or certifies clique-width above kkk.

Setting

All graphs are finite and simple. For a finite set VVV, a function f:2V→Zf : 2^V \to \mathbb{Z}f:2V→Z is submodular if f(X)+f(Y)≥f(X∩Y)+f(X∪Y)f(X)+f(Y) \ge f(X\cap Y)+f(X\cup Y)f(X)+f(Y)≥f(X∩Y)+f(X∪Y) and symmetric if f(X)=f(V∖X)f(X) = f(V\setminus X)f(X)=f(V∖X).

A branch-decomposition of fff is a pair (T,L)(T, L)(T,L) where TTT is a tree with at least two vertices and all degrees at most 333, and LLL is a bijection from VVV onto the leaves of TTT. Removing an edge eee of TTT splits the leaves in two; the width of eee is fff of the set of elements of VVV on one side. The width of (T,L)(T, L)(T,L) is the largest edge width, and the branch-width bw(f)\mathrm{bw}(f)bw(f) is the least width of a branch-decomposition, with bw(f)=f(∅)\mathrm{bw}(f) = f(\emptyset)bw(f)=f(∅) when ∣V∣≤1|V| \le 1∣V∣≤1.

A set W⊆VW \subseteq VW⊆V is well-linked with respect to fff if for every partition (X,Y)(X, Y)(X,Y) of WWW and every ZZZ with X⊆Z⊆V∖YX \subseteq Z \subseteq V\setminus YX⊆Z⊆V∖Y, f(Z)≥min⁡(∣X∣,∣Y∣)f(Z) \ge \min(|X|, |Y|)f(Z)≥min(∣X∣,∣Y∣).

Let A(G)A(G)A(G) be the adjacency matrix of GGG over GF(2)\mathrm{GF}(2)GF(2). For disjoint X,Y⊆V(G)X, Y \subseteq V(G)X,Y⊆V(G), cutrkG∗(X,Y)\mathrm{cutrk}^*_G(X, Y)cutrkG∗​(X,Y) is the rank of the submatrix of A(G)A(G)A(G) with rows XXX and columns YYY, and the cut-rank function is cutrkG(X)=cutrkG∗(X,V(G)∖X)\mathrm{cutrk}_G(X) = \mathrm{cutrk}^*_G(X, V(G)\setminus X)cutrkG​(X)=cutrkG∗​(X,V(G)∖X). The rank-width rwd(G)\mathrm{rwd}(G)rwd(G) is bw(cutrkG)\mathrm{bw}(\mathrm{cutrk}_G)bw(cutrkG​).

A kkk-expression is a term built from constants ⋅i\cdot_i⋅i​ (a vertex with label i∈{1,…,k}i \in \{1,\dots,k\}i∈{1,…,k}), the operators ηi,j\eta_{i,j}ηi,j​ (i≠ji \ne ji=j; add all edges between labels iii and jjj), ρi→j\rho_{i\to j}ρi→j​ (relabel iii into jjj) and disjoint union ⊕\oplus⊕. Its value is the labelled graph it produces; GGG has clique-width cwd(G)≤k\mathrm{cwd}(G) \le kcwd(G)≤k if some kkk-expression has value isomorphic to GGG.

An interpolation of fff is a function f∗f^*f∗ on disjoint pairs (X,Y)(X, Y)(X,Y) that agrees with fff on (X,V∖X)(X, V\setminus X)(X,V∖X), is monotone, submodular in the sense f∗(A,B)+f∗(C,D)≥f∗(A∩C,B∪D)+f∗(A∪C,B∩D)f^*(A,B)+f^*(C,D) \ge f^*(A\cap C, B\cup D) + f^*(A\cup C, B\cap D)f∗(A,B)+f∗(C,D)≥f∗(A∩C,B∪D)+f∗(A∪C,B∩D), and has f∗(∅,∅)=f(∅)f^*(\emptyset,\emptyset)=f(\emptyset)f∗(∅,∅)=f(∅).

Formalization targets

Goal: Theorem 1.1, certificate form

For a graph GGG with at least one vertex and an integer k≥1k \ge 1k≥1:

∃ W, ∣W∣=3k+1, W well-linked for cutrkG  ⟹  cwd(G)≥k+1,\exists\, W,\ |W| = 3k+1,\ W \text{ well-linked for } \mathrm{cutrk}_G \;\Longrightarrow\; \mathrm{cwd}(G) \ge k+1,∃W, ∣W∣=3k+1, W well-linked for cutrkG​⟹cwd(G)≥k+1, ∄ W, ∣W∣=3k+1, W well-linked for cutrkG  ⟹  cwd(G)≤23k+2−1.\nexists\, W,\ |W| = 3k+1,\ W \text{ well-linked for } \mathrm{cutrk}_G \;\Longrightarrow\; \mathrm{cwd}(G) \le 2^{3k+2}-1.∄W, ∣W∣=3k+1, W well-linked for cutrkG​⟹cwd(G)≤23k+2−1.

The same explicit condition decides which side of the approximation holds; this is what the paper's algorithm certifies.

Milestones

  1. Proposition 4.1: properties of an interpolation, including that X↦f∗(X,B)−f(∅)X \mapsto f^*(X, B) - f(\emptyset)X↦f∗(X,B)−f(∅) is a matroid rank function on V∖BV\setminus BV∖B when f({v})−f(∅)≤1f(\{v\}) - f(\emptyset) \le 1f({v})−f(∅)≤1.
  2. Proposition 4.2: fmin⁡(X,Y)=min⁡X⊆Z⊆V∖Yf(Z)f_{\min}(X,Y) = \min_{X\subseteq Z\subseteq V\setminus Y} f(Z)fmin​(X,Y)=minX⊆Z⊆V∖Y​f(Z) is an interpolation.
  3. Theorem 5.1: a well-linked set of size kkk forces bw(f)≥k/3\mathrm{bw}(f) \ge k/3bw(f)≥k/3 (for k≠1k \ne 1k=1).
  4. Theorem 5.2: no well-linked set of size kkk implies bw(f)≤k\mathrm{bw}(f) \le kbw(f)≤k, when f({v})≤1f(\{v\}) \le 1f({v})≤1.
  5. Proposition 6.1: rk M[X1,Y1]+rk M[X2,Y2]≥rk M[X1∪X2,Y1∩Y2]+rk M[X1∩X2,Y1∪Y2]\mathrm{rk}\,M[X_1,Y_1] + \mathrm{rk}\,M[X_2,Y_2] \ge \mathrm{rk}\,M[X_1\cup X_2, Y_1\cap Y_2] + \mathrm{rk}\,M[X_1\cap X_2, Y_1\cup Y_2]rkM[X1​,Y1​]+rkM[X2​,Y2​]≥rkM[X1​∪X2​,Y1​∩Y2​]+rkM[X1​∩X2​,Y1​∪Y2​].
  6. Corollary 6.2: submodularity of cutrkG∗\mathrm{cutrk}^*_GcutrkG∗​ and cutrkG\mathrm{cutrk}_GcutrkG​.
  7. Section 6 claim: cutrkG\mathrm{cutrk}_GcutrkG​ is symmetric submodular and cutrkG∗\mathrm{cutrk}^*_GcutrkG∗​ interpolates it.
  8. Proposition 6.3: rwd(G)≤cwd(G)≤2rwd(G)+1−1\mathrm{rwd}(G) \le \mathrm{cwd}(G) \le 2^{\mathrm{rwd}(G)+1}-1rwd(G)≤cwd(G)≤2rwd(G)+1−1.

Significance

The dichotomy turns clique-width, for which no exact polynomial algorithm is known even for fixed kkk, into a parameter that can be approximated with an explicit witness in each direction. Downstream, every algorithm for graphs of bounded clique-width that needs a kkk-expression as input becomes applicable to graphs given without one, at the cost of an exponential blow-up of the width.

The result is proved in the literature; this mission formalizes it. To our knowledge none of the objects involved — branch-width of set functions, rank-width, cut-rank, kkk-expressions, clique-width — has been formalized in Mathlib, and the submodularity of submatrix rank (Proposition 6.1) is absent from Mathlib's Matrix.rank API. The formal development would give reusable definitions of branch-decompositions of arbitrary integer set functions, of cut-rank, and of clique-width, and a machine-checked link between the combinatorial and the linear-algebraic width parameters.

Difficulty

The upper bound in Theorem 5.2 is the core. The natural approach, growing a branch-decomposition one leaf split at a time while keeping the width at most kkk, gets stuck at a leaf carrying a set BBB with f(B)=kf(B) = kf(B)=k: a split of BBB into two parts of fff-value below kkk has to be found, and it must be found from the failure of well-linkedness of a set that is not obviously related to BBB. The paper's device is the interpolation f∗f^*f∗, which attaches a matroid to BBB whose base has exactly f(B)f(B)f(B) elements. Formalizing this requires handling partial branch-decompositions, their extensions, and a maximality argument over trees, none of which exists in Mathlib.

Proposition 6.3's upper bound is a second, independent difficulty: a rank-decomposition must be converted into a kkk-expression by an induction over a rooted binary tree, with a relabelling argument bounding the number of labels by the number of distinct nonzero rows of a GF(2)\mathrm{GF}(2)GF(2) matrix of rank kkk. Its lower bound needs the tree structure of a kkk-expression to be read as a branch-decomposition.

Formalization scope

The ground set is a Fintype V with DecidableEq V; subsets are Finset V; set functions are Finset V → ℤ, as in the paper. A branch-decomposition is a tree T : SimpleGraph (Fin n) with n≥2n \ge 2n≥2, all neighbour sets of size at most 333, and an injective map LLL from VVV onto the vertices of degree 111; the side of an edge uwuwuw is found by reachability from uuu after deleting uwuwuw. Branch-width, rank-width and clique-width are never computed as minima: "bw(f)≤k\mathrm{bw}(f) \le kbw(f)≤k" is the predicate "∣V∣≤1|V| \le 1∣V∣≤1 and f(∅)≤kf(\emptyset) \le kf(∅)≤k, or a branch-decomposition of width at most kkk exists", lower bounds say that every branch-decomposition has a wide edge, and "cwd(G)≤k\mathrm{cwd}(G) \le kcwd(G)≤k" is "GGG has a kkk-expression". Labels {1,…,k}\{1,\dots,k\}{1,…,k} are Fin k. The value of a kkk-expression has as vertex type the occurrences of constants (a nested sum type), and ηi,j\eta_{i,j}ηi,j​ requires i≠ji \ne ji=j. Cut-rank uses Matrix.rank over ZMod 2 of submatrices of SimpleGraph.adjMatrix. An interpolation is a function on all pairs of subsets whose axioms are imposed on disjoint pairs only.

Running time is not formalized. The paper's Theorem 1.1 asserts an O(n9log⁡n)O(n^9\log n)O(n9logn) algorithm; there is no cost model on the page, and the goal states the certificate the algorithm returns instead. Without the running time, "cwd(G)≥k+1\mathrm{cwd}(G) \ge k+1cwd(G)≥k+1 or cwd(G)≤23k+2−1\mathrm{cwd}(G) \le 2^{3k+2}-1cwd(G)≤23k+2−1" holds for every graph, so that reading is ruled out as a formalization of the goal; so are well-linkedness with respect to anything other than cutrkG\mathrm{cutrk}_GcutrkG​, widths defined by an unguarded infimum (which is 000 on an empty family), kkk-expressions whose value is not the graph up to isomorphism or whose η\etaη may join equal labels, and Theorem 5.1 stated for k=1k = 1k=1.

Correction of Theorem 5.1. As printed, Theorem 5.1 fails for k=1k = 1k=1: a singleton is always well-linked, but the edgeless graph on two vertices has cut-rank identically 000 and branch-width 0<1/30 < 1/30<1/3. The milestone carries the hypothesis k≠1k \ne 1k=1; the goal uses the theorem only at size 3k+1≥43k+1 \ge 43k+1≥4.

The graph with no vertex is excluded from the goal and from the upper bound of Proposition 6.3, since it has no kkk-expression for any kkk. Contributions welcome: proofs of the milestones, lemmas on branch-decompositions (suppressing degree-2 vertices, extending partial decompositions), and submatrix-rank submodularity, which is reusable beyond this mission.

Selected references

  • S. Oum and P. Seymour, Approximating clique-width and branch-width, J. Combin. Theory Ser. B 96 (2006) 514–528. https://doi.org/10.1016/j.jctb.2005.10.006
  • B. Courcelle and S. Olariu, Upper bounds to the clique width of graphs, Discrete Appl. Math. 101 (2000) 77–114. https://doi.org/10.1016/S0166-218X(99)00184-5
  • B. Courcelle, J. A. Makowsky and U. Rotics, Linear time solvable optimization problems on graphs of bounded clique-width, Theory Comput. Syst. 33 (2000) 125–150. https://doi.org/10.1007/s002249910009
  • N. Robertson and P. D. Seymour, Graph minors. X. Obstructions to tree-decomposition, J. Combin. Theory Ser. B 52 (1991) 153–190. https://doi.org/10.1016/0095-8956(91)90061-N
14 thms3 active usersReviewed
🏆Completed
CombinatoricsOptimizationTheoretical Computer Science·Captain: mikedeng1

Approximation Algorithms for Combinatorial Problems V: The Overlap-Ratio Greedy C2 Is Within 1 + ln k of the Least-Overlap Cover on EC(k)Research Paper

Motivation

David S. Johnson's 1974 paper Approximation Algorithms for Combinatorial Problems (J. Comput. System Sci. 9 (1974) 256–278) was one of the first systematic worst-case analyses of polynomial-time heuristics for NP-complete optimization problems. Its Section 5 proves the harmonic bound ∑j=1k1/j\sum_{j=1}^k 1/j∑j=1k​1/j for the greedy algorithm on minimum-cardinality set cover, a result that still underlies the standard ln⁡n\ln nlnn approximation guarantee.

Section 6, the subject of this mission, asks what happens when the cost of a cover is its total size rather than its number of sets. This problem, SET COVERING II (EC), is the optimization version of the EXACT COVER recognition problem of Karp's list (Karp 1972): a family has a disjoint subcover exactly when the optimum equals the number of covered points. Johnson shows that the change of measure breaks the cardinality greedy but that a greedy rule based on an overlap ratio recovers essentially the same guarantee. The same accounting (paying for each newly covered point) later became the standard analysis of greedy weighted set cover (Chvátal 1979).

Setting

An input is a finite family F={S1,…,Sp}F = \{S_1, \dots, S_p\}F={S1​,…,Sp​} of finite sets. Its covered set is T=⋃S∈FST = \bigcup_{S \in F} ST=⋃S∈F​S. A subcover is a subfamily F′⊆FF' \subseteq FF′⊆F with ⋃S∈F′S=T\bigcup_{S\in F'} S = T⋃S∈F′​S=T, and its measure is

mEC(F′)=∑S∈F′∣S∣.m_{EC}(F') = \sum_{S \in F'} |S|.mEC​(F′)=S∈F′∑​∣S∣.

The optimum F∗F^*F∗ is the least measure of a subcover; since every subcover has measure at least ∣T∣|T|∣T∣, an optimal subcover is one with the least possible overlapping. The subproblem EC(k)(k)(k) admits only families in which every set has at most kkk points.

Algorithm C2 keeps a subfamily SUB (initially empty), the unused sets LEFT (initially FFF) and the uncovered points UNCOV (initially TTT). While UNCOV is nonempty it chooses S′∈S' \inS′∈ LEFT minimizing

Ratio(S)=∣S−UNCOV∣∣S∩UNCOV∣,\mathrm{Ratio}(S) = \frac{|S - \mathrm{UNCOV}|}{|S \cap \mathrm{UNCOV}|},Ratio(S)=∣S∩UNCOV∣∣S−UNCOV∣​,

the number of already-covered points of SSS per newly covered point, and moves S′S'S′ from LEFT to SUB, removing its points from UNCOV. When several sets tie, any of them may be chosen; a subcover is choosable by C2 if some sequence of admissible choices returns it.

The overlap of a chosen set is ∣S′−UNCOV∣|S' - \mathrm{UNCOV}|∣S′−UNCOV∣ at the moment it is chosen, and the cumulative overlap OV(F1)\mathrm{OV}(F_1)OV(F1​) of a run returning F1F_1F1​ is the sum of these overlaps.

Formalization targets

Goal: Theorem 6 (p. 271)

For all k≥1k \ge 1k≥1 and n>0n > 0n>0,

R[C2,EC(k)](n)≤1+ln⁡(k)≤∑j=1k1j+12,R[C2, EC(k)](n) \le 1 + \ln(k) \le \sum_{j=1}^k \frac1j + \frac12,R[C2,EC(k)](n)≤1+ln(k)≤j=1∑k​j1​+21​,

and for all sufficiently large nnn, R[C2,EC(k)](n)≥∑j=1k(1/j)R[C2, EC(k)](n) \ge \sum_{j=1}^k (1/j)R[C2,EC(k)](n)≥∑j=1k​(1/j). In the size-free form used here, for every k≥1k \ge 1k≥1:

  1. every subcover MMM choosable by C2 on an input of EC(k)(k)(k) satisfies mEC(M)≤(1+ln⁡k) F∗m_{EC}(M) \le (1 + \ln k)\,F^*mEC​(M)≤(1+lnk)F∗;
  2. 1+ln⁡k≤∑j=1k1/j+1/21 + \ln k \le \sum_{j=1}^k 1/j + 1/21+lnk≤∑j=1k​1/j+1/2;
  3. some input of EC(k)(k)(k) with F∗>0F^* > 0F∗>0 has a choosable subcover with mEC(M)≥(∑j=1k1/j)F∗m_{EC}(M) \ge \big(\sum_{j=1}^k 1/j\big) F^*mEC​(M)≥(∑j=1k​1/j)F∗.

Milestones (proof of Theorem 6, pp. 271–272)

  • the measure of the output is ∣T∣+OV(F1)|T| + \mathrm{OV}(F_1)∣T∣+OV(F1​);
  • if C2 may choose a set with Ratio(S′)≥y\mathrm{Ratio}(S') \ge yRatio(S′)≥y, then (y+1) ∣UNCOV∣≤F∗(y+1)\,|\mathrm{UNCOV}| \le F^*(y+1)∣UNCOV∣≤F∗;
  • with a=F∗/∣T∣a = F^*/|T|a=F∗/∣T∣ and x=∣T−UNCOV∣/∣T∣x = |T - \mathrm{UNCOV}|/|T|x=∣T−UNCOV∣/∣T∣, the next chosen set has Ratio(S′)≤a/(1−x)−1\mathrm{Ratio}(S') \le a/(1-x) - 1Ratio(S′)≤a/(1−x)−1;
  • on EC(k)(k)(k), OV(F1)≤∣T∣ (a[ln⁡(k)+1]−1)\mathrm{OV}(F_1) \le |T|\,(a[\ln(k) + 1] - 1)OV(F1​)≤∣T∣(a[ln(k)+1]−1);
  • the analytic inequality 1+ln⁡(k)≤∑j=1k1/j+1/21 + \ln(k) \le \sum_{j=1}^k 1/j + 1/21+ln(k)≤∑j=1k​1/j+1/2;
  • the lower-bound input (Fig. 1 of the paper with every set of F1F_1F1​ filled out to exactly kkk points) on which C2 may pay ∑j=1k1/j\sum_{j=1}^k 1/j∑j=1k​1/j times the optimum.

Significance

The result. The measure ∑∣S∣\sum|S|∑∣S∣ penalizes overlap, and the paper notes (without proof, p. 270) that an algorithm returning an optimal cover for the cardinality measure can be a factor kkk from optimal for this one. Theorem 6 shows that the ratio rule C2 is within 1+ln⁡k1 + \ln k1+lnk of the least-overlap cover, and the lower bound shows that no analysis of C2 can beat ∑j=1k1/j\sum_{j=1}^k 1/j∑j=1k​1/j. The two bounds differ by less than 1/21/21/2 for every kkk. The theorem was an early instance of a logarithmic guarantee for a weighted covering problem, where each set's cost is its size.

Formalizing it. The theorem has been proved since 1974; no machine-checked proof is known to exist. The mission produces a formal model of the EC problem and of C2 as a nondeterministic process, the overlap identity, and the discrete form of the paper's area-under-a-curve estimate. The last is the part the paper argues informally, through a step function and an integral.

Difficulty

The cardinality argument for C1 counts the sets chosen; here the sets have different sizes, so it does not apply. The overlap C2 pays per newly covered point is not bounded by a constant: early choices can be free and late ones cost up to k−1k-1k−1 per point, and the bound on the cumulative overlap must hold against the whole run, for every sequence of tie-breaks.

In a formal proof the integral must be replaced by a sum. Covered points arrive in blocks (one block per chosen set), the charge is constant on a block but the bound depends on the covered fraction at the start of the block, and the sum has to be compared with a logarithm. The terms aln⁡aa\ln aalna and a/ka/ka/k that the paper drops using 1≤a≤k1 \le a \le k1≤a≤k must be controlled as well, and the relation 1≤a≤k1 \le a \le k1≤a≤k must itself be proved from optimality. The lower bound needs an explicit run of C2 through ties on an input with k⋅k!k \cdot k!k⋅k! points, checking at every stage that the intended set is a ratio minimizer.

Formalization scope

  • Inputs. A family is p : ℕ with S : Fin p → Finset α (0-based, repetitions allowed; a repeated set counts twice in the measure if both copies are chosen, which C2 never does). Subcovers and SUB, LEFT are index sets. F∗F^*F∗ is a minimum over the finite, nonempty set of subcovers (Finset.inf'), never a junk value.
  • Algorithm. C2 is a step relation on states (SUB, LEFT, UNCOV). The choice at Step 3 is existential over all minimizers, so every result quantifies over every choosable output (the paper's WORST). Ratio(S)\mathrm{Ratio}(S)Ratio(S) is +∞+\infty+∞ when S∩UNCOV=∅S \cap \mathrm{UNCOV} = \emptysetS∩UNCOV=∅; the formal rule requires the chosen set to meet UNCOV and compares ratios by cross-multiplication, with no division.
  • Overlap. The cumulative overlap depends on the run, not on the output alone, so it is carried by an inductive run relation RunOV.
  • No problem size. The paper's R[A,P](n)R[A, P](n)R[A,P](n) maximizes over inputs of size at most nnn in an unspecified encoding. Upper bounds are stated for every input and every choosable output; the lower bound exhibits one input and one choosable output. Given monotonicity of RRR in nnn, these are equivalent to the paper's claims. Ratios are stated multiplicatively, so F∗=0F^* = 0F∗=0 does not create a vacuous bound.
  • Numbers. Measures are natural numbers cast to R\mathbb RR; ln⁡\lnln is Real.log; ∑j=1k1/j\sum_{j=1}^k 1/j∑j=1k​1/j is Mathlib's harmonic k.
  • Ruled out. A deterministic tie-break would prove a weaker upper bound and could not realize the lower-bound run, and a ratio with x/0=0x/0 = 0x/0=0 would make disjoint-from-UNCOV sets the most attractive choice. The formalization uses neither.

The overlap identity and the discrete integral comparison are reusable for any greedy covering analysis that charges cost per newly covered point. Contributions welcome include proofs of the milestones, the invariants of the C2 run relation (SUB and LEFT partition the indices; UNCOV =T−⋃= T - \bigcup=T−⋃ SUB), and the explicit lower-bound run.

Selected references

  • D. S. Johnson, Approximation algorithms for combinatorial problems, J. Comput. System Sci. 9 (1974), 256–278. https://doi.org/10.1016/S0022-0000(74)80044-9
  • R. M. Karp, Reducibility among combinatorial problems, in Complexity of Computer Computations, Plenum, 1972, 85–103. https://doi.org/10.1007/978-1-4684-2001-2_9
  • V. Chvátal, A greedy heuristic for the set-covering problem, Math. Oper. Res. 4 (1979), 233–235. https://doi.org/10.1287/moor.4.3.233
10 thms3 active usersReviewed
🏆Completed
CombinatoricsOptimization·Captain: mikedeng1

Scenario Reduction Algorithms in Stochastic Programming III: The Minimal Reduction Distance of a Regular Ternary Scenario TreeResearch Paper

Motivation

Multistage stochastic programs are solved on a finite scenario tree: a discrete probability distribution whose support points are paths of a random process. Realistic trees have far too many scenarios for the resulting optimization problem, so practitioners reduce the tree, keeping nnn of its NNN scenarios and redistributing the probability of the deleted ones. The reduction should keep the reduced distribution as close as possible to the original one in a probability metric that controls the optimal value of the stochastic program (Dupačová, Gröwe-Kuska, Römisch, Math. Program. 95 (2003)).

Choosing the best nnn scenarios is a set-covering problem and NP-hard, and the algorithms used in practice (backward reduction, fast forward selection) are heuristics without error guarantees. Heitsch and Römisch (2003) therefore derived test instances with an exactly known optimum: regular binary and ternary scenario trees, for which the minimal reduction distance has a closed form once nnn is not too small. This mission formalizes the ternary case, Proposition 3.2 of that paper. The binary case (Proposition 3.1) is a separate mission of the same series.

Setting

Fix a depth K∈NK \in \mathbb{N}K∈N and branch widths δ1,…,δK≥0\delta^1, \dots, \delta^K \ge 0δ1,…,δK≥0, with δ0=0\delta^0 = 0δ0=0. A regular ternary scenario tree has N=3KN = 3^KN=3K scenarios, one for each index tuple (i1,…,iK)∈{1,2,3}K(i_1, \dots, i_K) \in \{1, 2, 3\}^K(i1​,…,iK​)∈{1,2,3}K, where iki_kik​ is the successor chosen at level kkk. Choosing successor iki_kik​ adds the increment δikk=(ik−2) δk∈{−δk,0,δk}\delta^k_{i_k} = (i_k - 2)\,\delta^k \in \{-\delta^k, 0, \delta^k\}δik​k​=(ik​−2)δk∈{−δk,0,δk}, and scenario iii is the vector ωi=(ωi0,…,ωiK)∈RK+1\omega_i = (\omega_i^0, \dots, \omega_i^K) \in \mathbb{R}^{K+1}ωi​=(ωi0​,…,ωiK​)∈RK+1 with

ωik=∑j=0kδijj,k=0,…,K(eq. (19)).\omega_i^k = \sum_{j=0}^{k} \delta^j_{i_j}, \qquad k = 0, \dots, K \quad \text{(eq. (19))}.ωik​=j=0∑k​δij​j​,k=0,…,K(eq. (19)).

All scenarios have probability pi=1/Np_i = 1/Npi​=1/N. The distance between scenarios is the maximum norm c(ωi,ωj)=∥ωi−ωj∥∞=max⁡0≤k≤K∣ωik−ωjk∣c(\omega_i, \omega_j) = \|\omega_i - \omega_j\|_\infty = \max_{0 \le k \le K} |\omega_i^k - \omega_j^k|c(ωi​,ωj​)=∥ωi​−ωj​∥∞​=max0≤k≤K​∣ωik​−ωjk​∣.

Deleting the scenarios of an index set JJJ and moving each deleted scenario's probability to a nearest kept scenario costs the reduction distance

DJ=∑i∈Jpimin⁡j∉J∥ωi−ωj∥∞(eq. (8)),D_J = \sum_{i \in J} p_i \min_{j \notin J} \|\omega_i - \omega_j\|_\infty \quad \text{(eq. (8))},DJ​=i∈J∑​pi​j∈/Jmin​∥ωi​−ωj​∥∞​(eq. (8)),

which by Theorem 2.1 of the paper is the optimal transport-type distance between the original distribution and the best distribution supported on the kept scenarios. The minimal reduction distance to nnn scenarios is Dnmin=min⁡{DJ:#J=N−n}D^{min}_n = \min\{D_J : \#J = N - n\}Dnmin​=min{DJ​:#J=N−n}.

Formalization targets

Goal: Proposition 3.2 (7/9-solution)

Let K≥3K \ge 3K≥3 and let k0∈arg⁡min⁡1≤k≤Kδkk_0 \in \arg\min_{1 \le k \le K} \delta^kk0​∈argmin1≤k≤K​δk with k0≤K−2k_0 \le K - 2k0​≤K−2 and max⁡{δk0+1,δk0+2}≤2δk0\max\{\delta^{k_0+1}, \delta^{k_0+2}\} \le 2\delta^{k_0}max{δk0​+1,δk0​+2}≤2δk0​. Then any two distinct scenarios are at distance at least δk0\delta^{k_0}δk0​; there is a set of 79N\tfrac79 N97​N scenarios each paired with a scenario outside it at distance exactly δk0\delta^{k_0}δk0​; and for each n∈Nn \in \mathbb{N}n∈N with 29N≤n<N\tfrac29 N \le n < N92​N≤n<N,

Dnmin=min⁡{DJ:#J=N−n}=N−nN δk0(eq. (21)),D^{min}_n = \min\{D_J : \#J = N - n\} = \frac{N - n}{N}\,\delta^{k_0} \quad \text{(eq. (21))},Dnmin​=min{DJ​:#J=N−n}=NN−n​δk0​(eq. (21)),

with the minimum attained.

Milestones

  1. Distinct scenarios satisfy ∥ωi−ωj∥∞≥δk0\|\omega_i - \omega_j\|_\infty \ge \delta^{k_0}∥ωi​−ωj​∥∞​≥δk0​.
  2. Every JJJ with #J=N−n\#J = N - n#J=N−n has DJ≥N−nNδk0D_J \ge \frac{N-n}{N}\delta^{k_0}DJ​≥NN−n​δk0​.
  3. The index set I∗∗I_{**}I∗∗​ of the proof has #I∗∗=29N\#I_{**} = \tfrac29 N#I∗∗​=92​N, and its complement J∗∗J_{**}J∗∗​ has 79N\tfrac79 N97​N elements.
  4. Every j∈J∗∗j \in J_{**}j∈J∗∗​ has a partner i∈I∗∗i \in I_{**}i∈I∗∗​ with ∥ωi−ωj∥∞=δk0\|\omega_i - \omega_j\|_\infty = \delta^{k_0}∥ωi​−ωj​∥∞​=δk0​.
  5. Example 4.2: for K=6K = 6K=6 and (δ1,…,δ6)=(0.7,0.9,1.2,1.5,2.6,3.3)(\delta^1, \dots, \delta^6) = (0.7, 0.9, 1.2, 1.5, 2.6, 3.3)(δ1,…,δ6)=(0.7,0.9,1.2,1.5,2.6,3.3), Dnmin=0.7 N−nND^{min}_n = 0.7\,\frac{N-n}{N}Dnmin​=0.7NN−n​ for 162≤n<729162 \le n < 729162≤n<729.

Significance

The result gives an exact optimal value for an NP-hard reduction problem on an infinite family of instances. Heitsch and Römisch use it in their numerical section to measure how far the heuristics' reduced trees are from optimal (Examples 4.1 and 4.2 are the binary and ternary test trees of that study). A closed form of this kind is also the only way to certify that a heuristic is exactly optimal on some instances rather than only competitive with other heuristics.

The proposition is proved in the paper, but the published proof of its central step is one sentence: "Similarly as in Proposition 3.1 it can be shown that there exists an index i∈I∗∗i \in I_{**}i∈I∗∗​ for each j∈J∗∗j \in J_{**}j∈J∗∗​ …". A formal proof supplies that case analysis, which is absent from the literature. The formalization also settles two points the printed statement leaves loose (see Formalization scope): the count of pairs at distance δk0\delta^{k_0}δk0​, and the definition of I∗∗I_{**}I∗∗​ when some widths vanish. No machine-checked version of this result or of the reduction distance DJD_JDJ​ is known to exist.

Difficulty

The lower bound is routine: two distinct scenarios first differ at some level lll, where their coordinates differ by δl\delta^lδl or 2δl2\delta^l2δl. The substance is attainment: one must exhibit, for every n≥29Nn \ge \tfrac29 Nn≥92​N, a kept set of size nnn whose every deleted scenario lies at distance exactly δk0\delta^{k_0}δk0​ from some kept one. The obvious candidate, keeping the scenarios that take the middle branch at level k0k_0k0​, has every other scenario at distance exactly δk0\delta^{k_0}δk0​ from a kept one, but it keeps 13N\tfrac13 N31​N scenarios and so covers only n≥13Nn \ge \tfrac13 Nn≥31​N. Going down to 29N\tfrac29 N92​N kept scenarios forces a deleted scenario and its partner to differ at more than one level, and since coordinates are running sums the differences at levels k0+1k_0+1k0​+1 and k0+2k_0+2k0​+2 accumulate on top of the one at level k0k_0k0​. The paper's proof of this step is not written out, and it depends on the widths of the two levels below k0k_0k0​: the hypothesis max⁡{δk0+1,δk0+2}≤2δk0\max\{\delta^{k_0+1}, \delta^{k_0+2}\} \le 2\delta^{k_0}max{δk0​+1,δk0​+2}≤2δk0​ is essential, and the result is false without it (for K=3K = 3K=3 and (δ1,δ2,δ3)=(1,3,3)(\delta^1, \delta^2, \delta^3) = (1, 3, 3)(δ1,δ2,δ3)=(1,3,3) one has D6min=31/27D^{min}_6 = 31/27D6min​=31/27, not 7/97/97/9).

Formalization scope

  • A scenario is an index tuple σ:Fin K→Fin 3\sigma : \mathrm{Fin}\,K \to \mathrm{Fin}\,3σ:FinK→Fin3; σ(r)\sigma(r)σ(r) is the successor at paper level r+1r + 1r+1, with Fin 3\mathrm{Fin}\,3Fin3 values 0,1,20, 1, 20,1,2 standing for the paper's i=1,2,3i = 1, 2, 3i=1,2,3. The widths are δ:N→R\delta : \mathbb{N} \to \mathbb{R}δ:N→R, of which only δ(1),…,δ(K)\delta(1), \dots, \delta(K)δ(1),…,δ(K) are used; the standing assumption δk∈R+\delta^k \in \mathbb{R}_+δk∈R+​ (p. 196) is the hypothesis δ(k)≥0\delta(k) \ge 0δ(k)≥0 for 1≤k≤K1 \le k \le K1≤k≤K. Scenarios live in Fin(K+1)→R\mathrm{Fin}(K+1) \to \mathbb{R}Fin(K+1)→R, whose Mathlib norm is the maximum norm. Probabilities are uniform, 1/3K1/3^K1/3K.
  • DJD_JDJ​ is defined for a general finite index set, probabilities and cost, with the inner minimum a Finset.inf' over the complement of JJJ; the complement must be nonempty, so no default value arises. DnminD^{min}_nDnmin​ is stated as IsLeast of the set of all values DJD_JDJ​ with #J=N−n\#J = N - n#J=N−n: the goal asserts both the lower bound for every JJJ and attainment by some JJJ. A formalization that exhibits a single JJJ with DJ=N−nNδk0D_J = \frac{N-n}{N}\delta^{k_0}DJ​=NN−n​δk0​, or states an infimum without attainment, or drops any hypothesis on k0k_0k0​, is a different (and in the last case false) statement.
  • 29N≤n\tfrac29 N \le n92​N≤n is written 2⋅3K≤9n2 \cdot 3^K \le 9n2⋅3K≤9n, and 79N\tfrac79 N97​N as 7⋅3K−27 \cdot 3^{K-2}7⋅3K−2.
  • Pairs. As printed, "there are 79N\tfrac79 N97​N distinct pairs of scenarios such that the distance between the members of each pair is exactly δk0\delta^{k_0}δk0​" is false as an exact count (for K=3K = 3K=3 and δ=(1,1,1)\delta = (1,1,1)δ=(1,1,1) there are 130 such pairs, not 21). The goal states what the proof constructs: a set J∗∗J_{**}J∗∗​ of 79N\tfrac79 N97​N scenarios, each paired with a scenario outside J∗∗J_{**}J∗∗​ at distance exactly δk0\delta^{k_0}δk0​.
  • I∗∗I_{**}I∗∗​. The paper defines I∗∗I_{**}I∗∗​ by testing whether the increments δikk\delta^k_{i_k}δik​k​ vanish. When one of δk0,δk0+1,δk0+2\delta^{k_0}, \delta^{k_0+1}, \delta^{k_0+2}δk0​,δk0​+1,δk0​+2 is 000 this no longer identifies the middle branch and the count 29N\tfrac29 N92​N fails, although the proposition remains true. The formalization defines I∗∗I_{**}I∗∗​ by branch indices (middle branch versus outer branches), which agrees with the paper whenever these three widths are positive. No positivity hypothesis is added to the goal.
  • Welcome contributions: a reusable library for regular scenario trees (first differing level, distance of paths), and the case analysis of milestone 4. The binary mission of this series needs the same lower-bound argument with the constant 2δk02\delta^{k_0}2δk0​.

Selected references

  • H. Heitsch, W. Römisch, Scenario Reduction Algorithms in Stochastic Programming, Computational Optimization and Applications 24 (2003), 187–206. https://doi.org/10.1023/A:1021805924152
  • J. Dupačová, N. Gröwe-Kuska, W. Römisch, Scenario reduction in stochastic programming: An approach using probability metrics, Mathematical Programming 95 (2003), 493–511. https://doi.org/10.1007/s10107-002-0331-0
9 thms3 active usersReviewed
🏆Completed
CombinatoricsOptimization·Captain: mikedeng1

Scenario Reduction Algorithms in Stochastic Programming II: The Minimal Reduction Distance of a Regular Binary Scenario TreeResearch Paper

Why exact reduction distances matter

Multistage stochastic programs are solved on a finite scenario tree, a discrete probability measure whose atoms are paths of a stochastic process. The size of the deterministic equivalent grows with the number of scenarios, so practitioners replace the original measure P=∑i=1NpiδωiP=\sum_{i=1}^N p_i\delta_{\omega_i}P=∑i=1N​pi​δωi​​ by a measure supported on n<Nn<Nn<N of its scenarios. Stability theory for stochastic programs (Dupačová, Gröwe-Kuska and Römisch, Math. Program. 95 (2003), doi:10.1007/s10107-002-0331-0) bounds the change of the optimal value by a probability metric between the two measures, which leads to the optimal scenario reduction problem: choose which N−nN-nN−n scenarios to delete so that this distance is smallest.

That problem is a set-covering problem and is NP-hard, and the algorithms that Heitsch and Römisch study in the same paper (backward reduction, fast forward selection) are heuristics without error guarantees. To test them one needs original measures whose optimal reduction distance is known exactly. Section 3 of Heitsch and Römisch, Scenario Reduction Algorithms in Stochastic Programming, Comput. Optim. Appl. 24 (2003) (doi:10.1023/A:1021805924152) supplies such instances: regular binary and ternary scenario trees, for which the minimal distance to any reduced tree with at least a fixed fraction of the scenarios is an explicit formula. This mission formalizes the binary case, Proposition 3.1.

Setting

Fix a horizon K∈NK\in\mathbb NK∈N and level parameters δ1,…,δK≥0\delta^1,\dots,\delta^K\ge0δ1,…,δK≥0, with δ0=0\delta^0=0δ0=0. A regular binary scenario tree has N=2KN=2^KN=2K scenarios. A scenario is determined by a branch ik∈{1,2}i_k\in\{1,2\}ik​∈{1,2} at every level k=1,…,Kk=1,\dots,Kk=1,…,K, and is the vector ωi=(ωi0,…,ωiK)∈RK+1\omega_i=(\omega_i^0,\dots,\omega_i^K)\in\mathbb R^{K+1}ωi​=(ωi0​,…,ωiK​)∈RK+1 with

ωik=∑j=0kδijj,δijj=(2ij−3) δj∈{−δj,+δj}.(19)\omega_i^k=\sum_{j=0}^k\delta^j_{i_j},\qquad \delta^j_{i_j}=(2i_j-3)\,\delta^j\in\{-\delta^j,+\delta^j\}.\tag{19}ωik​=j=0∑k​δij​j​,δij​j​=(2ij​−3)δj∈{−δj,+δj}.(19)

All scenarios start at the root ωi0=0\omega_i^0=0ωi0​=0 and carry probability pi=1/Np_i=1/Npi​=1/N. Scenarios are compared in the maximum norm ∥ω−ω~∥∞=max⁡k=0,…,K∣ωk−ω~k∣\|\omega-\tilde\omega\|_\infty=\max_{k=0,\dots,K}|\omega^k-\tilde\omega^k|∥ω−ω~∥∞​=maxk=0,…,K​∣ωk−ω~k∣.

Deleting the scenarios with indices in J⊂{1,…,N}J\subset\{1,\dots,N\}J⊂{1,…,N} and moving each deleted scenario's probability to a nearest kept scenario costs the reduction cost

DJ=∑i∈Jpimin⁡j∉J∥ωi−ωj∥∞,(8)D_J=\sum_{i\in J}p_i\min_{j\notin J}\|\omega_i-\omega_j\|_\infty,\tag{8}DJ​=i∈J∑​pi​j∈/Jmin​∥ωi​−ωj​∥∞​,(8)

which by Theorem 2.1 of the paper is the minimal Kantorovich-type distance between PPP and a measure supported on the kept scenarios. The minimal reduction distance for nnn kept scenarios is Dnmin=min⁡{DJ:#J=N−n}D^{min}_n=\min\{D_J:\#J=N-n\}Dnmin​=min{DJ​:#J=N−n}.

In the Lean development, scenarios are indexed by σ : Fin K → Fin 2 (Fin-index rrr is tree level r+1r+1r+1, value 000 is the branch −δ-\delta−δ, value 111 is +δ+\delta+δ), lev σ k is the branch at level kkk, scenario δ σ : Fin (K+1) → ℝ is ωσ\omega_\sigmaωσ​, and redCost δ J hJ is DJD_JDJ​.

Formalization targets

Goal: Proposition 3.1 (3/4-solution)

Let K≥3K\ge3K≥3, k0∈arg⁡min⁡1≤k≤Kδkk_0\in\arg\min_{1\le k\le K}\delta^kk0​∈argmin1≤k≤K​δk, k0≤K−2k_0\le K-2k0​≤K−2 and max⁡{δk0+1,δk0+2}≤2δk0\max\{\delta^{k_0+1},\delta^{k_0+2}\}\le2\delta^{k_0}max{δk0​+1,δk0​+2}≤2δk0​. Then any two distinct scenarios are at distance at least 2δk02\delta^{k_0}2δk0​; there is a set J∗J_*J∗​ of 34N\frac34N43​N scenarios each of which has a partner outside J∗J_*J∗​ at distance exactly 2δk02\delta^{k_0}2δk0​; and for every n∈Nn\in\mathbb Nn∈N with N4≤n<N\frac N4\le n<N4N​≤n<N

Dnmin=min⁡{DJ:#J=N−n}=N−nN 2δk0.(20)D^{min}_n=\min\{D_J:\#J=N-n\}=\frac{N-n}{N}\,2\delta^{k_0}.\tag{20}Dnmin​=min{DJ​:#J=N−n}=NN−n​2δk0​.(20)

Milestones

  1. Two scenarios that first differ at level lll are at distance ≥2δl≥2δk0\ge2\delta^l\ge2\delta^{k_0}≥2δl≥2δk0​.
  2. DJ≥N−nN2δk0D_J\ge\frac{N-n}{N}2\delta^{k_0}DJ​≥NN−n​2δk0​ for every JJJ with #J=N−n\#J=N-n#J=N−n.
  3. The index set I∗I_*I∗​ (branch at k0k_0k0​ opposite to the common branch at k0+1,k0+2k_0+1,k_0+2k0​+1,k0​+2) has #I∗=N/4\#I_*=N/4#I∗​=N/4, so #J∗=34N\#J_*=\frac34N#J∗​=43​N.
  4. Every j∈J∗j\in J_*j∈J∗​ has a partner i∈I∗i\in I_*i∈I∗​ with ∥ωi−ωj∥∞=2δk0\|\omega_i-\omega_j\|_\infty=2\delta^{k_0}∥ωi​−ωj​∥∞​=2δk0​.
  5. Example 4.1: for K=10K=10K=10 and the paper's parameters, Proposition 3.1 applies with k0=1k_0=1k0​=1 and Dnmin=N−nND^{min}_n=\frac{N-n}{N}Dnmin​=NN−n​ for 256≤n<1024256\le n<1024256≤n<1024.

Significance

The result. Proposition 3.1 gives the exact optimum of an NP-hard combinatorial problem on an explicit, parametrized family of instances of every size N=2KN=2^KN=2K. Section 4 of the paper uses it (Example 4.1, N=1024N=1024N=1024) as ground truth for the relative accuracy of backward reduction and fast forward selection. Without it, the quality of a heuristic reduction on a large tree could only be compared with other heuristics or with lower bounds.

The formalization. The proposition is proved in the paper; to our knowledge no machine-checked version exists. A formal proof certifies the benchmark values, and the statement also corrects the printed text in two places. First, "there are 34N\frac34N43​N distinct pairs … at distance exactly 2δk02\delta^{k_0}2δk0​" is false as an exact count (for K=3K=3K=3, δ=(1,1,1)\delta=(1,1,1)δ=(1,1,1) there are twenty such pairs, not six), so the goal states "at least", in the form the proof exhibits. Second, the paper's sign-based definition of I∗I_*I∗​ degenerates when some δk=0\delta^k=0δk=0, although the proposition still holds; the mission defines I∗I_*I∗​ by branch indices.

Difficulty

The lower bound is a direct computation. The content is the matching upper bound: a set JJJ of the prescribed size for which every deleted scenario has a kept scenario at the minimal possible distance. A natural first attempt pairs scenarios that differ only at level k0k_0k0​. That handles only half of the scenarios with a single partner each, and it cannot reach 34N\frac34N43​N deleted scenarios. Once scenarios differ at more than one level, their maximum-norm distance is a maximum of several partial sums, and keeping all of them at most 2δk02\delta^{k_0}2δk0​ is exactly where the hypothesis max⁡{δk0+1,δk0+2}≤2δk0\max\{\delta^{k_0+1},\delta^{k_0+2}\}\le2\delta^{k_0}max{δk0​+1,δk0​+2}≤2δk0​ enters. Without it, eq. (20) fails: for K=3K=3K=3, δ=(1,3,3)\delta=(1,3,3)δ=(1,3,3) and n=2n=2n=2 the true minimum is 52\frac5225​, not 32\frac3223​. The passage from n=N/4n=N/4n=N/4 to general n≥N/4n\ge N/4n≥N/4 also needs care: the deleted set must shrink while each remaining deleted scenario keeps its partner among the kept ones.

Formalization scope

  • The index type is Fin K → Fin 2, which has exactly 2K2^K2K elements. The paper's (K+1)(K+1)(K+1)-tuple has a level-0 entry with no choice, so it is dropped, and the vector ω\omegaω keeps its K+1K+1K+1 coordinates with ω0=0\omega^0=0ω0=0.
  • The parameters are δ : ℕ → ℝ, with δk≥0\delta^k\ge0δk≥0 and the arg min stated for k=1,…,Kk=1,\dots,Kk=1,…,K only. δk>0\delta^k>0δk>0 is not assumed, because the paper allows δk∈R+\delta^k\in\mathbb R_+δk∈R+​ and the proposition holds with zeros.
  • The cost is Mathlib's norm on Fin (K+1) → ℝ, which is the maximum norm, and pi=1/2Kp_i=1/2^Kpi​=1/2K is written out.
  • DJD_JDJ​ requires a nonempty set of kept scenarios, and its inner minimum is a finite Finset.inf'.
  • DnminD^{min}_nDnmin​ is stated with IsLeast over the set of attained values DJD_JDJ​, #J=N−n\#J=N-n#J=N−n, so both the lower bound and attainment are part of the goal. A proof of DJ≤N−nN2δk0D_{J}\le\frac{N-n}{N}2\delta^{k_0}DJ​≤NN−n​2δk0​ for a single exhibited JJJ, or a real infimum without attainment, does not prove the goal.
  • "N4≤n\frac N4\le n4N​≤n" is written 2K≤4n2^K\le4n2K≤4n, and 34N\frac34N43​N is 3⋅2K−23\cdot2^{K-2}3⋅2K−2.
  • The pairs claim is "at least 34N\frac34N43​N pairs", expressed as a set J∗J_*J∗​ of that size with a partner outside J∗J_*J∗​ for every member.
  • All hypotheses on k0k_0k0​ appear in the goal; dropping any of them makes (20) false.

Useful infrastructure: sup-norm lemmas for Fin n → ℝ (pi_norm_le_iff_of_nonneg, norm_le_pi_norm), Finset.inf' lemmas, and counting functions Fin K → Fin 2 with prescribed values (Fintype.card_fun, Fintype.card_pi). The tree and reduction-cost definitions are shared in spirit with the ternary-tree mission of this series (Proposition 3.2), and a proof whose structure transfers to d=3d=3d=3 is welcome. Contributions of proofs of the milestones individually, in any order, are welcome.

Selected references

  • H. Heitsch, W. Römisch, Scenario Reduction Algorithms in Stochastic Programming, Computational Optimization and Applications 24 (2003), 187–206. doi:10.1023/A:1021805924152
  • J. Dupačová, N. Gröwe-Kuska, W. Römisch, Scenario reduction in stochastic programming: An approach using probability metrics, Mathematical Programming 95 (2003), 493–511. doi:10.1007/s10107-002-0331-0
9 thms3 active usersReviewed
🏆Completed
Optimal TransportOptimization·Captain: mikedeng1

Scenario Reduction Algorithms in Stochastic Programming I: Fast Forward Selection Realizes the Forward Selection PrincipleResearch Paper

Why reduce scenarios

Multistage and two-stage stochastic programs are solved numerically on a discrete probability distribution: a finite set of scenarios ω1,…,ωN\omega_1,\dots,\omega_Nω1​,…,ωN​ with probabilities p1,…,pNp_1,\dots,p_Np1​,…,pN​. The size of the resulting optimization problem grows with NNN, and scenario sets produced by sampling or by historical data are often far too large to be solved directly. Scenario reduction replaces the original distribution by one supported on a small subset of the scenarios, chosen so that the optimal value and solutions of the stochastic program change as little as possible.

Stability theory for stochastic programs (Rachev and Römisch, 2002) shows that this change is controlled by a probability distance of Fortet–Mourier type, which for discrete measures is bounded by the value of a transportation problem. Dupačová, Gröwe-Kuska and Römisch (2003) turned this into a combinatorial problem and proposed greedy backward and forward heuristics. Heitsch and Römisch (2003) gave faster versions of both heuristics; the forward version, fast forward selection, is the subject of this mission. Implementations of these reduction heuristics are distributed with the GAMS modelling system (SCENRED) and are used in energy and finance applications of stochastic programming.

Setting

Let EEE be a finite-dimensional real vector space with a norm ∥⋅∥\|\cdot\|∥⋅∥, let ω0∈E\omega_0\in Eω0​∈E, and let h:[0,∞)→[0,∞)h:[0,\infty)\to[0,\infty)h:[0,∞)→[0,∞) be continuous and nondecreasing with h(0)=0h(0)=0h(0)=0. The cost between two points of EEE is

c(ω,ω~)=max⁡{1, h(∥ω−ω0∥), h(∥ω~−ω0∥)} ∥ω−ω~∥.c(\omega,\tilde\omega)=\max\bigl\{1,\,h(\|\omega-\omega_0\|),\,h(\|\tilde\omega-\omega_0\|)\bigr\}\,\|\omega-\tilde\omega\| .c(ω,ω~)=max{1,h(∥ω−ω0​∥),h(∥ω~−ω0​∥)}∥ω−ω~∥.

It is nonnegative, symmetric, and zero on the diagonal.

The original distribution is P=∑i=1NpiδωiP=\sum_{i=1}^N p_i\delta_{\omega_i}P=∑i=1N​pi​δωi​​ with pi>0p_i>0pi​>0 and ∑ipi=1\sum_i p_i=1∑i​pi​=1. Deleting the scenarios in a set J⊂{1,…,N}J\subset\{1,\dots,N\}J⊂{1,…,N} and assigning new weights qj≥0q_j\ge 0qj​≥0, ∑j∉Jqj=1\sum_{j\notin J}q_j=1∑j∈/J​qj​=1, to the kept ones gives Q=∑j∉JqjδωjQ=\sum_{j\notin J}q_j\delta_{\omega_j}Q=∑j∈/J​qj​δωj​​. The distance D(J;q)D(J;q)D(J;q) between PPP and QQQ is the optimal value of the transportation problem

D(J;q)=min⁡{∑i=1N∑j∉Jc(ωi,ωj)ηij: ηij≥0, ∑iηij=qj, ∑j∉Jηij=pi}.D(J;q)=\min\Bigl\{\sum_{i=1}^N\sum_{j\notin J}c(\omega_i,\omega_j)\eta_{ij}:\ \eta_{ij}\ge 0,\ \sum_{i}\eta_{ij}=q_j,\ \sum_{j\notin J}\eta_{ij}=p_i\Bigr\}.D(J;q)=min{i=1∑N​j∈/J∑​c(ωi​,ωj​)ηij​: ηij​≥0, i∑​ηij​=qj​, j∈/J∑​ηij​=pi​}.

The reduction cost of deleting JJJ is

DJ=∑i∈Jpimin⁡j∉Jc(ωi,ωj),D_J=\sum_{i\in J}p_i\min_{j\notin J}c(\omega_i,\omega_j),DJ​=i∈J∑​pi​j∈/Jmin​c(ωi​,ωj​),

and the optimal reduction problem (8) minimizes DJD_JDJ​ over all JJJ with #J=N−n\#J=N-n#J=N−n, where nnn is the number of scenarios to keep.

Forward selection builds the kept set greedily. With J[0]={1,…,N}J^{[0]}=\{1,\dots,N\}J[0]={1,…,N} and J[i]={1,…,N}∖{u1,…,ui}J^{[i]}=\{1,\dots,N\}\setminus\{u_1,\dots,u_i\}J[i]={1,…,N}∖{u1​,…,ui​}, it chooses

ui∈arg⁡min⁡u∈J[i−1]DJ[i−1]∖{u},i=1,…,n.(16)u_i\in\arg\min_{u\in J^{[i-1]}}D_{J^{[i-1]}\setminus\{u\}},\qquad i=1,\dots,n. \tag{16}ui​∈argu∈J[i−1]min​DJ[i−1]∖{u}​,i=1,…,n.(16)

Fast forward selection (Algorithm 2.4) computes the same choices through an updated cost matrix: cku[1]=c(ωk,ωu)c^{[1]}_{ku}=c(\omega_k,\omega_u)cku[1]​=c(ωk​,ωu​), cku[i]=min⁡{cku[i−1],ckui−1[i−1]}c^{[i]}_{ku}=\min\{c^{[i-1]}_{ku},c^{[i-1]}_{ku_{i-1}}\}cku[i]​=min{cku[i−1]​,ckui−1​[i−1]​}, zu[i]=∑k∈J[i−1]∖{u}pkcku[i]z^{[i]}_u=\sum_{k\in J^{[i-1]}\setminus\{u\}}p_kc^{[i]}_{ku}zu[i]​=∑k∈J[i−1]∖{u}​pk​cku[i]​, and ui∈arg⁡min⁡u∈J[i−1]zu[i]u_i\in\arg\min_{u\in J^{[i-1]}}z^{[i]}_uui​∈argminu∈J[i−1]​zu[i]​.

Formalization targets

Goal: Theorem 2.5

For 1≤n≤N1\le n\le N1≤n≤N and every run u1,…,unu_1,\dots,u_nu1​,…,un​ of Algorithm 2.4, with any tie-breaking in the arg min,

ui satisfies (16)andzui[i]=DJ[i](i=1,…,n).u_i\ \text{satisfies (16)}\quad\text{and}\quad z^{[i]}_{u_i}=D_{J^{[i]}}\qquad(i=1,\dots,n).ui​ satisfies (16)andzui​[i]​=DJ[i]​(i=1,…,n).

Milestones

  1. Theorem 2.1 (redistribution). For JJJ with at least one kept scenario, DJ=min⁡qD(J;q)D_J=\min_q D(J;q)DJ​=minq​D(J;q), and the minimum is attained at qˉj=pj+∑i∈J, j(i)=jpi\bar q_j=p_j+\sum_{i\in J,\,j(i)=j}p_iqˉ​j​=pj​+∑i∈J,j(i)=j​pi​ for every choice of nearest kept scenarios j(i)j(i)j(i).
  2. Eq. (10). D{1,…,N}∖{u}=∑i=1Npic(ωi,ωu)D_{\{1,\dots,N\}\setminus\{u\}}=\sum_{i=1}^Np_ic(\omega_i,\omega_u)D{1,…,N}∖{u}​=∑i=1N​pi​c(ωi​,ωu​), so (8) with #J=N−1\#J=N-1#J=N−1 is problem (10).
  3. Eq. (12). The sum lblblb of the N−nN-nN−n smallest single-deletion costs plmin⁡j≠lc(ωl,ωj)p_l\min_{j\neq l}c(\omega_l,\omega_j)pl​minj=l​c(ωl​,ωj​), taken in the greedy order (11), is at most DJD_JDJ​ for every JJJ with #J=N−n\#J=N-n#J=N−n.
  4. Optimality condition (p. 191). If each lil_ili​ has a nearest other scenario outside {l1,…,lN−n}∖{li}\{l_1,\dots,l_{N-n}\}\setminus\{l_i\}{l1​,…,lN−n​}∖{li​}, then {l1,…,lN−n}\{l_1,\dots,l_{N-n}\}{l1​,…,lN−n​} solves (8).
  5. Eq. (17), unrolled recursion. For any index sequence, cku[i]=min⁡j∉J[i−1]∖{u}c(ωk,ωj)c^{[i]}_{ku}=\min_{j\notin J^{[i-1]}\setminus\{u\}}c(\omega_k,\omega_j)cku[i]​=minj∈/J[i−1]∖{u}​c(ωk​,ωj​) for u∈J[i−1]u\in J^{[i-1]}u∈J[i−1].
  6. Eq. (17), conclusion. For any index sequence, zu[i]=DJ[i−1]∖{u}z^{[i]}_u=D_{J^{[i-1]}\setminus\{u\}}zu[i]​=DJ[i−1]∖{u}​ for u∈J[i−1]u\in J^{[i-1]}u∈J[i−1].

Significance

Theorem 2.5 certifies that the cheap update of Algorithm 2.4 (one pairwise minimum per matrix entry and step) produces exactly the greedy forward selection defined through the reduction costs, and that the running objective zui[i]z^{[i]}_{u_i}zui​[i]​ is the reduction cost of the scenarios deleted so far. Combined with Theorem 2.1, zui[i]z^{[i]}_{u_i}zui​[i]​ is the optimal transportation distance between PPP and the best measure on the kept scenarios, which is the quantity practitioners monitor to decide how many scenarios to keep. The lower bound (12) and the optimality condition give a posteriori quality certificates for any reduced set.

All results of this mission are proved in the paper or in the works it cites (Dupačová et al., 2003); none is open. To the best of a platform search, none has been machine-checked. The mission provides a verified specification of a widely deployed algorithm, a formal link between a combinatorial set-covering objective and a finite transportation problem, and definitions (reduction cost, transportation plans with a partially free target marginal, greedy runs with arbitrary tie-breaking) reusable by the regular-tree missions of this series and by later scenario-tree construction papers.

Difficulty

The mathematics is elementary; the difficulty is bookkeeping. The recursion for c[i]c^{[i]}c[i] refers to the previous step's column ui−1u_{i-1}ui−1​, which itself was updated, so unrolling it to a minimum over {u,u1,…,ui−1}\{u,u_1,\dots,u_{i-1}\}{u,u1​,…,ui−1​} is an induction on iii in which the index sets J[i]J^{[i]}J[i], the 1-based step counter and the complement structure all move together. The natural first attempt, identifying cku[i]c^{[i]}_{ku}cku[i]​ with the minimum over the complement of J[i]J^{[i]}J[i], is off by one step: the correct set is the complement of J[i−1]∖{u}J^{[i-1]}\setminus\{u\}J[i−1]∖{u}, which contains uuu itself. For Theorem 2.1 the lower bound requires using that c(ωi,ωi)=0c(\omega_i,\omega_i)=0c(ωi​,ωi​)=0 for kept scenarios and that every plan ships all of pip_ipi​ somewhere outside JJJ; the attainment part requires constructing the plan explicitly from the choice j(⋅)j(\cdot)j(⋅), including scenarios for which several kept scenarios are equally near.

Formalization scope

  • Scenarios are ω : Fin N → E with E a finite-dimensional real normed space; the paper's closed set Ω⊂Rs\Omega\subset\mathbb R^sΩ⊂Rs plays no role beyond containing the scenarios and is omitted. Scenarios need not be distinct.
  • hhh is a function ℝ → ℝ with the paper's assumptions imposed on [0,∞)[0,\infty)[0,∞) (IsGrowthFunction); every theorem carries them, together with pi>0p_i>0pi​>0 and ∑ipi=1\sum_ip_i=1∑i​pi​=1.
  • The functions f0f_0f0​, ggg and the stochastic program (1)–(2) that motivate ccc appear in no statement.
  • D(J;q)D(J;q)D(J;q) is the paper's finite transportation problem (p. 188), not the Kantorovich functional on measures. Weights qqq and plans η\etaη are indexed by all of {1,…,N}\{1,\dots,N\}{1,…,N} with entries at deleted indices fixed to 000.
  • DJD_JDJ​ requires a proof that the complement of JJJ is nonempty; minima are Finset.inf', never a real infimum with a default value.
  • Algorithm 2.4 is a relation on sequences u : ℕ → Fin N with 1-based steps. c[i]c^{[i]}c[i] is the printed recursion, extended to all indices; runs are any sequences satisfying the arg-min conditions, so every tie-breaking rule is covered.
  • The paper's standing restriction n<Nn<Nn<N is relaxed to n≤Nn\le Nn≤N in Theorem 2.5; the statement remains true at n=Nn=Nn=N.
  • A trivializing formalization is ruled out: defining c[i]c^{[i]}c[i] or z[i]z^{[i]}z[i] directly as the minimum over the selected set or as DJ[i−1]∖{u}D_{J^{[i-1]}\setminus\{u\}}DJ[i−1]∖{u}​ would make Theorem 2.5 hold by definition, and proving it for one fixed tie-breaking rule would prove less than the paper; neither is done.
  • Proofs of the milestones, alternative proofs of Theorem 2.1 via LP duality, and a verified executable implementation of Algorithm 2.4 are all welcome.

Selected references

  • H. Heitsch, W. Römisch, Scenario Reduction Algorithms in Stochastic Programming, Computational Optimization and Applications 24 (2003), 187–206. https://doi.org/10.1023/A:1021805924152
  • J. Dupačová, N. Gröwe-Kuska, W. Römisch, Scenario reduction in stochastic programming: an approach using probability metrics, Mathematical Programming 95 (2003), 493–511. https://doi.org/10.1007/s10107-002-0331-0
  • S. T. Rachev, W. Römisch, Quantitative stability in stochastic programming: the method of probability metrics, Mathematics of Operations Research 27 (2002), 792–818. https://doi.org/10.1287/moor.27.4.792.304
12 thms3 active usersReviewed
🏆Completed
Convex OptimizationNumerical AnalysisOptimization·Captain: mikedeng1

Golden Ratio Algorithms for Variational Inequalities I: The Golden Ratio Algorithm with a Fixed Step Converges to a Solution of a Monotone Variational InequalityResearch Paper

Motivation

A monotone variational inequality asks for a point at which a monotone operator and a convex function are in equilibrium. It unifies convex minimization (where FFF is a gradient), convex–concave saddle-point problems (where FFF is the skew gradient of a Lagrangian), Nash equilibria of monotone games, and complementarity problems in economics and traffic assignment. In operations research, first-order methods for such problems are the workhorse behind large-scale saddle-point formulations of linear and conic programs, where only one operator evaluation and one projection or proximal step per iteration are affordable.

The classical method for Lipschitz monotone operators is Korpelevich's extragradient method (1976) and its proximal variant, Tseng's forward–backward–forward method (2000); both need two evaluations of FFF per iteration. The reflected projected gradient method of Malitsky (SIAM J. Optim., 2015) uses one evaluation of FFF but evaluates it at 2zk−zk−12z^k-z^{k-1}2zk−zk−1, a point that may lie outside the domain of ggg. Malitsky's Golden Ratio Algorithm (GRAAL), introduced in Golden Ratio Algorithms for Variational Inequalities (preprint 2018; published in Mathematical Programming, doi:10.1007/s10107-019-01416-w), uses one evaluation of FFF, always at a feasible point, and one proximal step per iteration. Its fixed-step version, Theorem 1 of that paper, is the subject of this mission; the explicit, adaptive-step version (Theorem 2) is a separate mission of this series.

Setting

Let E\mathcal EE be a finite-dimensional real inner product space with inner product ⟨⋅,⋅⟩\langle\cdot,\cdot\rangle⟨⋅,⋅⟩ and norm ∥⋅∥=⟨⋅,⋅⟩\|\cdot\| = \sqrt{\langle\cdot,\cdot\rangle}∥⋅∥=⟨⋅,⋅⟩​. Let g:E→(−∞,+∞]g:\mathcal E\to(-\infty,+\infty]g:E→(−∞,+∞] and write dom⁡g={x:g(x)<+∞}\operatorname{dom} g = \{x : g(x)<+\infty\}domg={x:g(x)<+∞}. Let F:dom⁡g→EF:\operatorname{dom} g\to\mathcal EF:domg→E. The variational inequality is

find z∗∈Esuch that⟨F(z∗),z−z∗⟩+g(z)−g(z∗) ≥ 0∀z∈E.(1)\text{find } z^*\in\mathcal E \quad\text{such that}\quad \langle F(z^*), z-z^*\rangle + g(z)-g(z^*)\ \ge\ 0\qquad \forall z\in\mathcal E. \tag{1}find z∗∈Esuch that⟨F(z∗),z−z∗⟩+g(z)−g(z∗) ≥ 0∀z∈E.(1)

The standing assumptions are:

  • (C1) the solution set SSS of (1) is nonempty;
  • (C2) ggg is proper (never −∞-\infty−∞, finite somewhere), convex, and lower semicontinuous;
  • (C3) FFF is monotone: ⟨F(u)−F(v),u−v⟩≥0\langle F(u)-F(v),u-v\rangle\ge0⟨F(u)−F(v),u−v⟩≥0 for all u,v∈dom⁡gu,v\in\operatorname{dom} gu,v∈domg.

The proximal operator of ggg is prox⁡g(z)=argmin⁡x{g(x)+12∥x−z∥2}\operatorname{prox}_g(z) = \operatorname{argmin}_x\{g(x)+\tfrac12\|x-z\|^2\}proxg​(z)=argminx​{g(x)+21​∥x−z∥2}. Let φ=5+12\varphi = \frac{\sqrt5+1}{2}φ=25​+1​ be the golden ratio, so that φ2=1+φ\varphi^2 = 1+\varphiφ2=1+φ. For a step λ>0\lambda>0λ>0 and arbitrary starting points z1,zˉ0∈Ez^1,\bar z^0\in\mathcal Ez1,zˉ0∈E, the Golden Ratio Algorithm generates, for k≥1k\ge1k≥1,

zˉk=(φ−1)zk+zˉk−1φ,zk+1=prox⁡λg(zˉk−λF(zk)).(6)\bar z^k = \frac{(\varphi-1)z^k + \bar z^{k-1}}{\varphi},\qquad z^{k+1} = \operatorname{prox}_{\lambda g}\big(\bar z^k - \lambda F(z^k)\big). \tag{6}zˉk=φ(φ−1)zk+zˉk−1​,zk+1=proxλg​(zˉk−λF(zk)).(6)

The first line is a convex combination of the newest iterate and the previous average; the second is a forward–backward step taken from the average rather than from zkz^kzk.

Formalization targets

Goal: Theorem 1

If FFF is LLL-Lipschitz on dom⁡g\operatorname{dom} gdomg (L>0L>0L>0), (C1)–(C3) hold, and λ∈(0,φ2L]\lambda\in\big(0,\frac{\varphi}{2L}\big]λ∈(0,2Lφ​], then there is z∗∈Sz^*\in Sz∗∈S with

zk→z∗andzˉk→z∗(k→∞).z^k\to z^*\qquad\text{and}\qquad \bar z^k\to z^*\qquad(k\to\infty).zk→z∗andzˉk→z∗(k→∞).

Both sequences converge, to one and the same solution. The goal is stated with the paper's exact step range; no rate is claimed, as the paper claims none.

Milestones

  1. Eq. (4), the prox-inequality: for proper convex lsc ggg,
xˉ=prox⁡gz  ⟺  ⟨xˉ−z,x−xˉ⟩≥g(xˉ)−g(x)∀x∈E.\bar x = \operatorname{prox}_g z \iff \langle\bar x - z, x-\bar x\rangle\ge g(\bar x)-g(x)\quad\forall x\in\mathcal E.xˉ=proxg​z⟺⟨xˉ−z,x−xˉ⟩≥g(xˉ)−g(x)∀x∈E.
  1. Eq. (12), an identity using only the averaging step of (6): for every point z∗z^*z∗,
∥zk+1−z∗∥2=(1+φ)∥zˉk+1−z∗∥2−φ∥zˉk−z∗∥2+1φ∥zk+1−zˉk∥2.\|z^{k+1}-z^*\|^2 = (1+\varphi)\|\bar z^{k+1}-z^*\|^2-\varphi\|\bar z^k-z^*\|^2+\tfrac1\varphi\|z^{k+1}-\bar z^k\|^2 .∥zk+1−z∗∥2=(1+φ)∥zˉk+1−z∗∥2−φ∥zˉk−z∗∥2+φ1​∥zk+1−zˉk∥2.
  1. Eq. (14), the energy inequality: for z∗∈Sz^*\in Sz∗∈S and k≥2k\ge2k≥2,
(1+φ)∥zˉk+1−z∗∥2+φ2∥zk+1−zk∥2≤(1+φ)∥zˉk−z∗∥2+φ2∥zk−zk−1∥2−φ∥zk−zˉk∥2.(1+\varphi)\|\bar z^{k+1}-z^*\|^2+\tfrac\varphi2\|z^{k+1}-z^k\|^2\le(1+\varphi)\|\bar z^k-z^*\|^2+\tfrac\varphi2\|z^k-z^{k-1}\|^2-\varphi\|z^k-\bar z^k\|^2 .(1+φ)∥zˉk+1−z∗∥2+2φ​∥zk+1−zk∥2≤(1+φ)∥zˉk−z∗∥2+2φ​∥zk−zk−1∥2−φ∥zk−zˉk∥2.
  1. Lemma 1 (Bauschke–Combettes, Theorem 5.5): a sequence that is Fejér monotone with respect to a nonempty set CCC and whose cluster points all lie in CCC converges to a point of CCC.

Significance

The result. Theorem 1 shows that monotone variational inequalities with a Lipschitz operator can be solved with one operator evaluation and one proximal step per iteration, with FFF evaluated only at points of dom⁡g\operatorname{dom} gdomg, where it is defined. This matters when FFF is expensive (a large matrix–vector product, a simulation) or undefined outside the feasible set (for instance an operator involving log⁡x\log xlogx on the positive orthant). The analysis also explains the constant: the averaging weight φ\varphiφ is the largest ccc with 1/c≥c−11/c\ge c-11/c≥c−1, and the step bound φ/(2L)\varphi/(2L)φ/(2L) follows from it. The fixed-step analysis is the template for the explicit, adaptive-step EGRAAL of the same paper (Theorem 2), which needs only local Lipschitz continuity of FFF.

The formalization. The theorem has a published proof, and no machine-checked version of it or of GRAAL is known. Mathlib contains the golden ratio, Lipschitz conditions, lower semicontinuity and cluster points, but no proximal operator of an extended-valued function, no prox-inequality and no Fejér-monotonicity convergence lemma. This mission produces those pieces and a complete convergence proof for a first-order VI method, which are reusable for projected gradient, forward–backward, extragradient and reflected-gradient analyses.

Difficulty

The naive approach, to show that ∥zk−z∗∥\|z^k-z^*\|∥zk−z∗∥ decreases, fails: GRAAL is not Fejér monotone in zkz^kzk, because the forward step is taken from the average zˉk\bar z^kzˉk and uses F(zk)F(z^k)F(zk) rather than FFF at the new point. The quantity that decreases is an energy mixing ∥zˉk−z∗∥2\|\bar z^k-z^*\|^2∥zˉk−z∗∥2 with the successive difference ∥zk−zk−1∥2\|z^k-z^{k-1}\|^2∥zk−zk−1∥2, and both the averaging identity and the Lipschitz estimate must produce matching coefficients for the cross terms to cancel. The energy inequality alone gives only boundedness and vanishing successive differences; convergence of the whole sequence, and the fact that the limit solves (1) when ggg is merely lower semicontinuous and extended-valued, is a separate step. On the formal side, ggg takes the value +∞+\infty+∞, so the prox-inequality and the variational inequality must be handled in extended arithmetic without letting ∞−∞\infty-\infty∞−∞ decide anything.

Formalization scope

  • E\mathcal EE is a type E with [NormedAddCommGroup E] [InnerProductSpace ℝ E] [FiniteDimensional ℝ E].
  • ggg is E → EReal. (C2) is IsProperConvexLSC g: never ⊥\bot⊥, somewhere finite, convex epigraph {(x,t)∈E×R:g(x)≤t}\{(x,t)\in E\times\mathbb R: g(x)\le t\}{(x,t)∈E×R:g(x)≤t}, and LowerSemicontinuous g on all of E. dom⁡g\operatorname{dom} gdomg is effDom g = {x | g x ≠ ⊤}.
  • FFF is a total function E → E; monotonicity and the Lipschitz bound ∥F(u)−F(v)∥≤L∥u−v∥\|F(u)-F(v)\|\le L\|u-v\|∥F(u)−F(v)∥≤L∥u−v∥ are required on effDom g only. The step range is 0 < λ, λ ≤ φ / (2 * L) with 0 < L and φ = Real.goldenRatio.
  • SSS is solutionSet g F: points of effDom g satisfying (1) for every z∈Ez\in Ez∈E, evaluated in EReal.
  • The proximal step is the argmin predicate IsProxPoint (fun x => λ * g x) w z⁺, not a choice function, so no junk value is involved. A run of (6) is IsGRAALRun g F λ z zbar on sequences ℕ → E indexed as in the paper: z1z^1z1 and zˉ0\bar z^0zˉ0 are free and the entry z0z^0z0 is unused.
  • The conclusion is ∃ zs ∈ solutionSet g F, Tendsto z atTop (𝓝 zs) ∧ Tendsto zbar atTop (𝓝 zs).

The hypotheses of the goal are jointly satisfiable, so the theorem is not vacuous: for g≡0g\equiv0g≡0 and F≡0F\equiv0F≡0 every point is a solution and constant sequences form a run of (6); a formalization under which IsGRAALRun has no instances, or in which SSS may be empty, is ruled out. Two hypotheses are added to printed statements and flagged in their notes: C≠∅C\neq\emptysetC=∅ in Lemma 1, which is false without it, and z1∈dom⁡gz^1\in\operatorname{dom} gz1∈domg in Eq. (14), needed at k=2k=2k=2 because the paper's FFF is only defined on dom⁡g\operatorname{dom} gdomg.

Welcome contributions: existence and uniqueness of the proximal point of a proper convex lsc function in finite dimensions; the prox-inequality; Fejér-monotonicity lemmas; the energy inequality; and the final convergence argument. The prox and Fejér infrastructure is independent of the golden ratio and is shared with the second mission of this series.

Selected references

  • Y. Malitsky, Golden Ratio Algorithms for Variational Inequalities, preprint, Optimization Online 6598, 2018. https://optimization-online.org/wp-content/uploads/2018/05/6598.pdf ; published in Mathematical Programming. https://doi.org/10.1007/s10107-019-01416-w
  • H. H. Bauschke, P. L. Combettes, Convex Analysis and Monotone Operator Theory in Hilbert Spaces, Springer, 2011 (2nd ed. 2017). https://doi.org/10.1007/978-3-319-48311-5
  • G. M. Korpelevich, The extragradient method for finding saddle points and other problems, Ekonomika i Matematicheskie Metody 12 (1976) 747–756.
  • P. Tseng, A modified forward–backward splitting method for maximal monotone mappings, SIAM J. Control Optim. 38 (2000) 431–446. https://doi.org/10.1137/S0363012998338806
  • Y. Malitsky, Projected reflected gradient methods for monotone variational inequalities, SIAM J. Optim. 25 (2015) 502–520. https://doi.org/10.1137/14097238X
8 thms3 active usersReviewed
🏆Completed
Numerical AnalysisOptimization·Captain: mikedeng1

Pattern Search Algorithms for Bound Constrained Minimization: Generalized Pattern Search Drives the Projected Stationarity Measure to ZeroResearch Paper

Motivation

Pattern search methods minimize a function f:Rn→Rf:\mathbb R^n\to\mathbb Rf:Rn→R by comparing values of fff at points of a structured set of trial points, without evaluating or approximating derivatives. Coordinate search and the method of Hooke and Jeeves (Hooke–Jeeves 1961) are the classical members of the family. Such methods remain in use when derivatives are unavailable, unreliable or expensive, for instance when fff is the output of a simulation, and practical problems of this kind usually carry simple bounds on the variables.

Torczon (SIAM J. Optim. 1997) gave a global convergence theory for pattern search on unconstrained problems: under compactness of the level set and continuous differentiability of fff, lim inf⁡k∥∇f(xk)∥=0\liminf_k\|\nabla f(x_k)\|=0liminfk​∥∇f(xk​)∥=0, and under stronger hypotheses lim⁡k∥∇f(xk)∥=0\lim_k\|\nabla f(x_k)\|=0limk​∥∇f(xk​)∥=0. Lewis and Torczon extended this theory to bound constrained problems (ICASE Report 96-20, 1996; SIAM J. Optim. 1999). The extension is not automatic: the paper exhibits a pattern search method for unconstrained problems (Box's evolutionary operation with factorial designs) that fails on bound constrained ones, and identifies the structural condition on the pattern that restores convergence.

Timeline.

  • 1961: Hooke and Jeeves introduce "direct search" pattern methods.
  • 1987–1988: Calamai and Moré (Math. Program. 1987) and Conn, Gould and Toint (SIAM J. Numer. Anal. 1988) develop the projected-gradient stationarity theory for bound and linear constraints, for methods that use derivatives.
  • 1997: Torczon proves global convergence of generalized pattern search for unconstrained problems.
  • 1996/1999: Lewis and Torczon prove the bound constrained theory formalized here.

Setting

The problem is

min⁡f(x)subject toℓ≤x≤u,\min f(x)\quad\text{subject to}\quad \ell\le x\le u,minf(x)subject toℓ≤x≤u,

with ℓ,u\ell,uℓ,u vectors of extended reals and ℓj<uj\ell_j<u_jℓj​<uj​ for every jjj; ℓj=−∞\ell_j=-\inftyℓj​=−∞ or uj=+∞u_j=+\inftyuj​=+∞ is allowed. The feasible region is Ω={x:ℓ≤x≤u}\Omega=\{x:\ell\le x\le u\}Ω={x:ℓ≤x≤u}, PPP is the coordinatewise projection onto Ω\OmegaΩ, g=∇fg=\nabla fg=∇f, and LΩ(y)={x∈Ω:f(x)≤f(y)}L_\Omega(y)=\{x\in\Omega:f(x)\le f(y)\}LΩ​(y)={x∈Ω:f(x)≤f(y)} is the feasible level set. A stationary point is an x∈Ωx\in\Omegax∈Ω with ⟨g(x),z−x⟩≥0\langle g(x),z-x\rangle\ge0⟨g(x),z−x⟩≥0 for all z∈Ωz\in\Omegaz∈Ω. The stationarity measure is

q(x)=P(x−g(x))−x,q(x)=P\bigl(x-g(x)\bigr)-x,q(x)=P(x−g(x))−x,

which vanishes exactly at stationary points.

A generalized pattern search method is fixed by a nonsingular basis matrix B∈Rn×nB\in\mathbb R^{n\times n}B∈Rn×n, a finite set M\mathcal MM of nonsingular integer matrices, a rational τ>1\tau>1τ>1, an integer w0<0w_0<0w0​<0 and nonnegative integers w1,…,wLw_1,\dots,w_Lw1​,…,wL​. At iteration kkk the generating matrix is Ck=[Mk  −Mk  Lk]=[Γk  Lk]C_k=[M_k\ \ {-M_k}\ \ L_k]=[\Gamma_k\ \ L_k]Ck​=[Mk​  −Mk​  Lk​]=[Γk​  Lk​] with Mk∈MM_k\in\mathcal MMk​∈M, LkL_kLk​ an integer matrix containing a zero column, and BMkBM_kBMk​ diagonal. A trial step is ΔkBc\Delta_kBcΔk​Bc for a column ccc of CkC_kCk​. The step sks_ksk​ is a trial step with xk+sk∈Ωx_k+s_k\in\Omegaxk​+sk​∈Ω, and it must decrease fff whenever some feasible trial step from the core ΔkBΓk\Delta_kB\Gamma_kΔk​BΓk​ does. The iterate moves, xk+1=xk+skx_{k+1}=x_k+s_kxk+1​=xk​+sk​, exactly when f(xk+sk)<f(xk)f(x_k+s_k)<f(x_k)f(xk​+sk​)<f(xk​). The step length Δk\Delta_kΔk​ is multiplied by θ=τw0<1\theta=\tau^{w_0}<1θ=τw0​<1 after an unsuccessful iteration and by some τwi≥1\tau^{w_i}\ge1τwi​≥1 after a successful one. The Strong Hypotheses additionally require f(xk+sk)f(x_k+s_k)f(xk​+sk​) to be no larger than the best feasible core trial value whenever that value is below f(xk)f(x_k)f(xk​).

Formalization targets

Goal: Theorem 3.3

If LΩ(x0)L_\Omega(x_0)LΩ​(x0​) is compact, fff is continuously differentiable, the columns of the CkC_kCk​ are uniformly bounded, Δk→0\Delta_k\to0Δk​→0, and the Strong Hypotheses hold, then

lim⁡k→∞∥q(xk)∥=0.\lim_{k\to\infty}\|q(x_k)\|=0 .k→∞lim​∥q(xk​)∥=0.

Milestones

  • Lemma 2.1, Theorem 2.2, Lemma 2.3: the unconstrained results the paper recalls from Torczon (1997): nonzero steps have length at least ζ∗Δk\zeta_*\Delta_kζ∗​Δk​; the iterates lie on the translated lattice x0+βrLBα−rUBΔ0B Znx_0+\beta^{r_{LB}}\alpha^{-r_{UB}}\Delta_0B\,\mathbb Z^nx0​+βrLB​α−rUB​Δ0​BZn (with τ=β/α\tau=\beta/\alphaτ=β/α); bounded columns give Δk≥ψ∗∥ski∥\Delta_k\ge\psi_*\|s_k^i\|Δk​≥ψ∗​∥ski​∥.
  • Proposition 3.1 (6), (8): ∥q(x)∥≤∥g(x)∥\|q(x)\|\le\|g(x)\|∥q(x)∥≤∥g(x)∥, and xxx is stationary iff q(x)=0q(x)=0q(x)=0.
  • All iterates lie in LΩ(x0)L_\Omega(x_0)LΩ​(x0​) (§4, p. 10).
  • Propositions 4.1–4.3: a descent estimate along short steep directions; a feasible core step with gkTs≤−n−1/2∥qk∥∥s∥g_k^Ts\le-n^{-1/2}\|q_k\|\|s\|gkT​s≤−n−1/2∥qk​∥∥s∥ whenever qk≠0q_k\ne0qk​=0 and the step length is small; a uniform δ\deltaδ (and, under the Strong Hypotheses, a σ\sigmaσ) with f(xk+1)≤f(xk)−σ∥q(xk)∥∥sk∥f(x_{k+1})\le f(x_k)-\sigma\|q(x_k)\|\|s_k\|f(xk+1​)≤f(xk​)−σ∥q(xk​)∥∥sk​∥ when Δk<δ\Delta_k<\deltaΔk​<δ and ∥q(xk)∥>η\|q(x_k)\|>\eta∥q(xk​)∥>η.
  • Corollary 4.4 and Theorem 4.5: lim inf⁡∥q(xk)∥≠0\liminf\|q(x_k)\|\ne0liminf∥q(xk​)∥=0 keeps Δk\Delta_kΔk​ bounded away from zero, whereas compactness alone forces lim inf⁡Δk=0\liminf\Delta_k=0liminfΔk​=0.
  • Theorem 3.2: lim inf⁡k∥q(xk)∥=0\liminf_k\|q(x_k)\|=0liminfk​∥q(xk​)∥=0.

Significance

Theorem 3.2 shows that a method which never computes a gradient still has a subsequence approaching first-order stationarity for the bound constrained problem, even though it cannot enforce a sufficient decrease condition measured by the projected gradient. Theorem 3.3 upgrades this to the whole sequence, so every limit point of the iterates is a KKT point. These results justify the bound constrained variants of coordinate search and Hooke–Jeeves discussed in §5 of the paper, and they are the template for the later theory of pattern search under linear constraints and generating set search.

The results are proved in the paper, and three of the milestones are proved in Torczon (1997). None of them has a machine-checked proof. Formalizing them produces a Lean model of generalized pattern search (patterns, exploratory moves, step-length updates) that later missions on direct search, mesh adaptive direct search or linearly constrained pattern search can reuse, and checks the details the paper handles briefly: the lattice argument, the feasibility of the chosen coordinate step, and uniform constants.

Difficulty

The obvious argument copies the unconstrained proof with ∇f\nabla f∇f replaced by qqq. The step that fails is the existence of a good trial step: in the unconstrained case some pattern direction makes an acute angle with −∇f(xk)-\nabla f(x_k)−∇f(xk​), but near the boundary of Ω\OmegaΩ that direction may leave the feasible region, and a feasible direction may not be a descent direction. For a general pattern no uniform choice exists, and the paper's counterexample in §5.2 shows convergence can fail. The diagonality of BMkBM_kBMk​ is what makes the pattern contain coordinate directions, one of which is both feasible and a descent direction of quality n−1/2∥qk∥n^{-1/2}\|q_k\|n−1/2∥qk​∥ (Proposition 4.2). The second difficulty is Theorem 4.5, which uses no derivatives: it rests on the rationality of τ\tauτ and the integrality of the CkC_kCk​, which confine the iterates to a lattice that meets the compact set LΩ(x0)L_\Omega(x_0)LΩ​(x0​) in finitely many points.

Formalization scope

Points are EuclideanSpace ℝ (Fin n) with the Euclidean norm; the bounds are Fin n → EReal with the hypothesis ℓj<uj\ell_j<u_jℓj​<uj​ for all jjj, so infinite bounds are allowed as in the paper. Paper coordinates 1,…,n1,\dots,n1,…,n are Lean's Fin n. The gradient is Mathlib's gradient f. A run of the method is a structure of sequences (xk,Δk,sk,Mk,Lk)(x_k,\Delta_k,s_k,M_k,L_k)(xk​,Δk​,sk​,Mk​,Lk​) together with a predicate IsGPSRun that encodes §2.1–§2.4 clause by clause; the parameter m≥1m\ge1m≥1 is the number of columns of LkL_kLk​ (the paper's p−2np-2np−2n). τ\tauτ is rational and the CkC_kCk​ are integer matrices, as the lattice argument requires. "min⁡{f(xk+y):… }<f(xk)\min\{f(x_k+y):\dots\}<f(x_k)min{f(xk​+y):…}<f(xk​)" over the finite set of feasible core trial points is encoded as "some feasible core trial step strictly decreases fff". lim inf⁡\liminfliminf statements are encoded with ∃ᶠ, not Filter.liminf. The modulus of continuity ω\omegaω is not formed as a real supremum; Proposition 4.1 takes an explicit radius δ>∥d∥\delta>\|d\|δ>∥d∥.

Standing assumptions and every departure from the page:

  1. Smoothness. The page assumes fff continuously differentiable on LΩ(x0)L_\Omega(x_0)LΩ​(x0​). The mission assumes fff is C1C^1C1 on an open set U⊇ΩU\supseteq\OmegaU⊇Ω. The proofs evaluate ∇f\nabla f∇f along segments to trial points that lie in Ω\OmegaΩ but generally outside LΩ(x0)L_\Omega(x_0)LΩ​(x0​), and LΩ(x0)L_\Omega(x_0)LΩ​(x0​) may have empty interior, so the page's hypothesis does not define what the proofs use.
  2. Strong Hypothesis 3. The page prints f(xk+sk)<min⁡{⋯ }f(x_k+s_k)<\min\{\cdots\}f(xk​+sk​)<min{⋯}. No core step can satisfy the strict form, which would exclude coordinate search, which the paper says satisfies it. The mission uses ≤\le≤, the form of Torczon (1997) and the one the proof of Proposition 4.3 uses. The theorem with ≤\le≤ implies the one with <<<.
  3. The nonemptiness of {w1,…,wL}\{w_1,\dots,w_L\}{w1​,…,wL​} is made explicit.
  4. Proposition 3.1 (7) is omitted: its P(g(x))P(g(x))P(g(x)) is a projected gradient the paper does not define.
  5. Proposition 4.2 quantifies over every step length below νk\nu_kνk​, because νk\nu_kνk​ does not depend on Δk\Delta_kΔk​.

The run predicate is not vacuous: an explicit run of coordinate search on f(x)=xf(x)=xf(x)=x over [0,∞)[0,\infty)[0,∞) satisfies IsGPSRun, the Strong Hypotheses, bounded columns, Δk→0\Delta_k\to0Δk​→0 and compactness of LΩ(x0)L_\Omega(x_0)LΩ​(x0​). That check is proved in Lean without sorry, so the goal cannot be closed by exhibiting an unsatisfiable hypothesis.

A complete development needs the mean value theorem along segments, uniform continuity of ∇f\nabla f∇f near the compact set LΩ(x0)L_\Omega(x_0)LΩ​(x0​), finiteness of a discrete lattice inside a compact set, and elementary facts about the coordinatewise projection. The projection and lattice lemmas are reusable beyond this mission. Proofs of any milestone, alternative proofs, and a general statement of Proposition 3.1 for closed convex Ω\OmegaΩ are welcome.

Selected references

  • R. M. Lewis and V. Torczon, Pattern Search Algorithms for Bound Constrained Minimization, ICASE Report No. 96-20 (NASA CR-198306), 1996; SIAM J. Optim. 9(4):1082–1099, 1999. https://doi.org/10.1137/S1052623496300507
  • V. Torczon, On the Convergence of Pattern Search Algorithms, SIAM J. Optim. 7(1):1–25, 1997. https://doi.org/10.1137/S1052623493250780
  • P. H. Calamai and J. J. Moré, Projected Gradient Methods for Linearly Constrained Problems, Math. Program. 39:93–116, 1987. https://doi.org/10.1007/BF02592073
  • A. R. Conn, N. I. M. Gould and P. L. Toint, Global Convergence of a Class of Trust Region Algorithms for Optimization with Simple Bounds, SIAM J. Numer. Anal. 25(2):433–460, 1988. https://doi.org/10.1137/0725029
  • R. Hooke and T. A. Jeeves, "Direct Search" Solution of Numerical and Statistical Problems, J. ACM 8(2):212–229, 1961. https://doi.org/10.1145/321062.321069
16 thms3 active usersReviewed
🏆Completed
Graph TheoryOptimizationTheoretical Computer Science·Captain: mikedeng1

A New Approach to the Maximum-Flow Problem 2: The Nonsaturating-Push Bound for FIFO Push-RelabelResearch Paper

Motivation

The maximum-flow problem asks how much of a commodity can be sent from a source to a sink through a network whose edges carry capacities. It is a basic model of operations research. Transportation, scheduling, bipartite matching and image segmentation reduce to it, and it is the inner step of many combinatorial algorithms.

Goldberg and Tarjan introduced the push–relabel (preflow) method in A New Approach to the Maximum-Flow Problem (J. ACM 35(4), 1988). Ford–Fulkerson-type algorithms augment along whole source–sink paths. The push–relabel method instead moves excess flow across single edges, guided by integer distance labels on the vertices. Whatever order its local operations are applied in, it is correct and performs O(n2m)O(n^2 m)O(n2m) of them (§3 of the paper). Section 4 shows that one particular order, processing the active vertices first-in, first-out, cuts the dominant term, the number of nonsaturating pushes, to O(n3)O(n^3)O(n3). The method and its FIFO and highest-label variants are the standard practical maximum-flow codes.

Timeline:

  • 1956: Ford and Fulkerson, augmenting paths and max-flow min-cut.
  • 1970–72: Dinic, and Edmonds and Karp, give polynomial augmenting-path bounds.
  • 1974: Karzanov introduces preflows and obtains O(n3)O(n^3)O(n3).
  • 1982: Shiloach and Vishkin give a parallel O(n2log⁡n)O(n^2 \log n)O(n2logn) preflow algorithm with a first-in, first-out flavour.
  • 1988: Goldberg and Tarjan, the generic push–relabel method, the FIFO bound of this mission, and O(nmlog⁡(n2/m))O(nm \log(n^2/m))O(nmlog(n2/m)) with dynamic trees.

Setting

A flow network has a finite vertex set VVV with n=∣V∣n = |V|n=∣V∣, a capacity c(v,w)≥0c(v,w) \ge 0c(v,w)≥0 on every ordered pair, a source sss and a sink t≠st \neq st=s. The edges are the pairs with c(v,w)>0c(v,w) > 0c(v,w)>0, and there are no loops. A preflow is a function fff on vertex pairs with f(v,w)≤c(v,w)f(v,w) \le c(v,w)f(v,w)≤c(v,w) and f(v,w)=−f(w,v)f(v,w) = -f(w,v)f(v,w)=−f(w,v). Its excess e(v)=∑uf(u,v)e(v) = \sum_u f(u,v)e(v)=∑u​f(u,v) must be nonnegative at every v≠sv \neq sv=s. The residual capacity is rf(v,w)=c(v,w)−f(v,w)r_f(v,w) = c(v,w) - f(v,w)rf​(v,w)=c(v,w)−f(v,w). A labeling ddd assigns each vertex a value in N∪{∞}\mathbb{N} \cup \{\infty\}N∪{∞}. A vertex v∉{s,t}v \notin \{s,t\}v∈/{s,t} is active if d(v)<∞d(v) < \inftyd(v)<∞ and e(v)>0e(v) > 0e(v)>0.

The two basic operations (Fig. 1 of the paper) are:

  • push(v,w)(v,w)(v,w), applicable when vvv is active, rf(v,w)>0r_f(v,w) > 0rf​(v,w)>0 and d(v)=d(w)+1d(v) = d(w)+1d(v)=d(w)+1. It sends δ=min⁡(e(v),rf(v,w))\delta = \min(e(v), r_f(v,w))δ=min(e(v),rf​(v,w)) from vvv to www. The push is saturating if rf(v,w)=0r_f(v,w) = 0rf​(v,w)=0 afterwards and nonsaturating otherwise.
  • relabel(v)(v)(v), applicable when vvv is active and d(v)≤d(w)d(v) \le d(w)d(v)≤d(w) for every residual edge (v,w)(v,w)(v,w). It sets d(v)←min⁡{d(w)+1:rf(v,w)>0}d(v) \leftarrow \min\{d(w)+1 : r_f(v,w) > 0\}d(v)←min{d(w)+1:rf​(v,w)>0}.

The algorithm starts by saturating every edge leaving sss, with d(s)=nd(s) = nd(s)=n and d(v)=0d(v) = 0d(v)=0 for v≠sv \neq sv=s.

In the first-in, first-out algorithm (§4), each vertex vvv scans a fixed list L(v)L(v)L(v) of its neighbours through a current edge. The push/relabel(v)(v)(v) operation pushes through the current edge if possible. Otherwise it advances the current edge, or, at the end of the list, returns to the first edge and relabels vvv. Active vertices wait in a queue QQQ, initially {v∈V−{s,t}:c(s,v)>0}\{v \in V - \{s,t\} : c(s,v) > 0\}{v∈V−{s,t}:c(s,v)>0}. The discharge operation removes the front vertex vvv and repeats push/relabel(v)(v)(v) until e(v)=0e(v) = 0e(v)=0 or d(v)d(v)d(v) increases. Every vertex that becomes active meanwhile is appended to QQQ, and vvv is appended too if it is still active. Passes over the queue are defined inductively. Pass 1 consists of the discharges of the initially queued vertices. Pass i+1i+1i+1 consists of the discharges of vertices added during pass iii.

Formalization targets

Goal: Corollary 4.4 (p. 931)

For every network, every edge-list order, every initial queue order, and every run of the FIFO algorithm,

#{nonsaturating pushes}≤4n3.\#\{\text{nonsaturating pushes}\} \le 4n^3 .#{nonsaturating pushes}≤4n3.

The constant is the printed one.

Milestones

  • Lemma 4.1 (p. 929): the push/relabel operation relabels only when relabeling is applicable.
  • Lemma 3.5 (p. 926): from any vertex with positive excess, the source is reachable in the residual graph.
  • Lemma 3.7 (p. 927): at any time, d(v)≤2n−1d(v) \le 2n-1d(v)≤2n−1 for every vertex.
  • Lemma 3.8 (p. 927): at most 2n−12n-12n−1 relabelings per vertex and at most (2n−1)(n−2)<2n2(2n-1)(n-2) < 2n^2(2n−1)(n−2)<2n2 in total.
  • Lemma 4.3 (p. 930): at most 4n24n^24n2 passes over the queue.

Significance

Corollary 4.4 is the combinatorial core of Theorem 4.5, which states that the FIFO algorithm runs in O(n3)O(n^3)O(n3) time. Theorem 4.2 shows that the remaining work of the implementation is O(nm)O(nm)O(nm) plus constant time per nonsaturating push. The bound of Corollary 4.4 is therefore what separates the O(n3)O(n^3)O(n3) FIFO method from the O(n2m)O(n^2 m)O(n2m) bound of the generic method, which matters on dense networks. The same pass-counting argument is reused for the parallel algorithm of §6 and underlies later analyses of highest-label and wave variants.

The results are proved in the paper. Formalizing them adds an analysis of a push–relabel algorithm, which the platform does not yet have. Its existing network-flow material states max-flow min-cut and Ford–Fulkerson termination in an arc-based model with nonnegative flows (the Introduction to Linear Optimization missions). The mission builds a precise operational model of the FIFO implementation, with edge lists, current edges and a queue carrying pass numbers, and states an explicit operation count for it. A companion mission in this series treats the generic algorithm's correctness and its (2n−1)(n−2)+2nm+4n2m(2n-1)(n-2) + 2nm + 4n^2m(2n−1)(n−2)+2nm+4n2m operation bound.

Difficulty

The obvious argument is the potential-function count of §3, over the sum of the labels of active vertices. It yields only 4n2m4n^2 m4n2m and does not use the queue discipline at all. The 4n34n^34n3 bound has to charge nonsaturating pushes to passes over the queue, and then bound the number of passes by the total growth of the labels. Neither step is visible in the generic algorithm, because both depend on the order in which vertices are processed.

Making this rigorous requires invariants of the implementation that the paper uses silently:

  • a vertex is in the queue exactly when it is active, and at most once;
  • pass numbers are nondecreasing along the queue;
  • current edges only move forward between relabelings.

Lemma 4.1 in particular depends on the current-edge scan: an edge passed over earlier stays inadmissible until vvv is relabeled.

Formalization scope

The Lean development works in namespace GoldbergTarjan.FIFO. Vertices form a type V with [Fintype V] [DecidableEq V], and nnn is Fintype.card V. Capacities are c : V → V → ℝ with c ≥ 0 and c v v = 0. Flows are antisymmetric real functions on all ordered pairs, not nonnegative arc flows. Excess is computed from the flow. Labels are in ℕ∞, and the empty minimum in relabel is ⊤.

The state of the algorithm consists of the flow, the labels, the current-edge index cur v into the edge list L v, and the queue Q : List (V × ℕ), each entry tagged with its pass number. Push/relabel (Fig. 3) is a total function, and a discharge (Fig. 4) is a relation carrying the number of push/relabel operations it performs. A run consists of the states S 0, …, S K with S 0 the initial state and consecutive states related by one discharge. The printed variant of Fig. 4, which stops as soon as vvv is relabeled, is the one formalized. Counts are natural numbers over all push/relabel operations of all discharges. The number of passes is the largest pass tag of a discharged entry.

All constants are explicit, exactly as printed:

  • 2n−12n-12n−1 (Lemmas 3.7, 3.8);
  • (2n−1)(n−2)(2n-1)(n-2)(2n−1)(n−2) and 2n22n^22n2 (Lemma 3.8);
  • 4n24n^24n2 (Lemma 4.3);
  • 4n34n^34n3 (Corollary 4.4).

No asymptotic notation is used, and no m≥n−1m \ge n-1m≥n−1 assumption is made.

A model without current edges, where relabeling happens whenever no push applies, would make Lemma 4.1 vacuous and change the algorithm. Pass numbers that are not propagated by the "added during pass iii" rule would make the pass count arbitrary. Both are ruled out by the definitions. A sorry-free check, outside the proposal, exhibits a three-vertex network with two legal discharges, two passes and no nonsaturating push, so the run hypotheses are satisfiable.

Reusable beyond this mission are the network, preflow, push and relabel definitions and Lemma 3.5, which is about an arbitrary preflow. Contributions welcome: invariants of FIFO runs (preflow, valid labeling, queue = active set, cur within bounds), proofs of the milestones, and the reduction of Corollary 4.4 to Lemma 4.3.

Selected references

  • A. V. Goldberg, R. E. Tarjan, A New Approach to the Maximum-Flow Problem, Journal of the ACM 35(4):921–940, 1988. https://doi.org/10.1145/48014.61051
  • A. V. Karzanov, Determining the maximal flow in a network by the method of preflows, Soviet Math. Doklady 15:434–437, 1974.
  • Y. Shiloach, U. Vishkin, An O(n² log n) parallel max-flow algorithm, Journal of Algorithms 3(2):128–146, 1982. https://doi.org/10.1016/0196-6774(82)90013-X
  • L. R. Ford, D. R. Fulkerson, Maximal flow through a network, Canadian Journal of Mathematics 8:399–404, 1956. https://doi.org/10.4153/CJM-1956-045-5
  • J. Edmonds, R. M. Karp, Theoretical improvements in algorithmic efficiency for network flow problems, Journal of the ACM 19(2):248–264, 1972. https://doi.org/10.1145/321694.321699
10 thms3 active usersReviewed
🏆Completed
Graph TheoryOptimizationTheoretical Computer Science·Captain: mikedeng1

A New Approach to the Maximum-Flow Problem 1: The Generic Push-Relabel Algorithm and Its Operation BoundResearch Paper

Motivation

The maximum-flow problem asks how much of a commodity can be sent from a source to a sink through a network whose edges have capacities. It is a basic model in operations research (transportation, scheduling, bipartite matching) and a standard subroutine in combinatorial optimization.

Classical algorithms, from Ford and Fulkerson (1956) through Edmonds–Karp and Dinic (1970–1972) and Karzanov (1974), increase a feasible flow along augmenting paths or blocking flows. Goldberg and Tarjan, A New Approach to the Maximum-Flow Problem (J. ACM 35(4), 1988, doi:10.1145/48014.61051), replaced this global view by a local one: the push-relabel method maintains a preflow, which may violate conservation at intermediate vertices, and moves excess along edges toward vertices with smaller distance labels. The generic method, with the basic operations applied in any order, is the starting point of the FIFO, highest-label and dynamic-tree implementations analysed later in the same paper, and push-relabel codes remain among the fastest practical maximum-flow solvers.

This mission formalizes §2–§3 of the paper: the generic algorithm is correct, and it stops after a number of basic operations bounded by an explicit polynomial in the numbers of vertices and edges, whatever order of operations is chosen.

Setting

A flow network has a finite vertex set VVV with n=∣V∣n = |V|n=∣V∣, a source sss and a sink t≠st \ne st=s, and a capacity c(v,w)≥0c(v,w) \ge 0c(v,w)≥0 for every ordered pair of vertices, positive exactly on the edges E={(v,w):c(v,w)>0}E = \{(v,w) : c(v,w) > 0\}E={(v,w):c(v,w)>0}; m=∣E∣m = |E|m=∣E∣, and there are no loops, c(v,v)=0c(v,v) = 0c(v,v)=0.

Flows are real functions on all vertex pairs. A function fff satisfies the capacity constraint if f(v,w)≤c(v,w)f(v,w) \le c(v,w)f(v,w)≤c(v,w) and antisymmetry if f(v,w)=−f(w,v)f(v,w) = -f(w,v)f(v,w)=−f(w,v) for all pairs. The excess of vvv is e(v)=∑uf(u,v)e(v) = \sum_{u} f(u,v)e(v)=∑u​f(u,v). A flow also has e(v)=0e(v) = 0e(v)=0 for v∉{s,t}v \notin \{s,t\}v∈/{s,t}; a preflow only e(v)≥0e(v) \ge 0e(v)≥0 for v≠sv \ne sv=s. The value of a flow is ∣f∣=∑vf(v,t)|f| = \sum_v f(v,t)∣f∣=∑v​f(v,t), and a maximum flow is a flow of maximum value.

The residual capacity is rf(v,w)=c(v,w)−f(v,w)r_f(v,w) = c(v,w) - f(v,w)rf​(v,w)=c(v,w)−f(v,w); pairs with rf(v,w)>0r_f(v,w) > 0rf​(v,w)>0 are the edges of the residual graph GfG_fGf​. A valid labeling is d:V→N∪{∞}d : V \to \mathbb{N} \cup \{\infty\}d:V→N∪{∞} with d(s)=nd(s) = nd(s)=n, d(t)=0d(t) = 0d(t)=0 and d(v)≤d(w)+1d(v) \le d(w) + 1d(v)≤d(w)+1 on every residual edge. A vertex vvv is active if v∉{s,t}v \notin \{s,t\}v∈/{s,t}, d(v)<∞d(v) < \inftyd(v)<∞ and e(v)>0e(v) > 0e(v)>0.

The two basic operations (Fig. 1 of the paper) are:

  • Push(v,w)(v,w)(v,w), applicable when vvv is active, rf(v,w)>0r_f(v,w) > 0rf​(v,w)>0 and d(v)=d(w)+1d(v) = d(w)+1d(v)=d(w)+1: send δ=min⁡(e(v),rf(v,w))\delta = \min(e(v), r_f(v,w))δ=min(e(v),rf​(v,w)), i.e. f(v,w)+=δf(v,w) \mathrel{+}= \deltaf(v,w)+=δ, f(w,v)−=δf(w,v) \mathrel{-}= \deltaf(w,v)−=δ. It is saturating if rf(v,w)=0r_f(v,w) = 0rf​(v,w)=0 afterwards and nonsaturating otherwise.
  • Relabel(v)(v)(v), applicable when vvv is active and d(v)≤d(w)d(v) \le d(w)d(v)≤d(w) for every residual edge (v,w)(v,w)(v,w): set d(v)←min⁡{d(w)+1:(v,w)∈Ef}d(v) \leftarrow \min\{d(w)+1 : (v,w) \in E_f\}d(v)←min{d(w)+1:(v,w)∈Ef​} (∞\infty∞ if there is none).

The generic algorithm (Fig. 2) starts from the preflow that saturates every edge leaving sss and is zero elsewhere, with the simple labeling d(s)=nd(s) = nd(s)=n, d(v)=0d(v) = 0d(v)=0 otherwise, and applies applicable basic operations in any order while one exists. An execution with KKK basic operations is a sequence of states (f0,d0),…,(fK,dK)(f_0,d_0),\dots,(f_K,d_K)(f0​,d0​),…,(fK​,dK​) from the initial state, each obtained from the previous one by one applicable operation.

Formalization targets

Goal: Theorems 3.11 and 3.4

Assume the paper's standing assumption m≥n−1m \ge n-1m≥n−1. For every execution with KKK basic operations,

K≤(2n−1)(n−2)+2nm+4n2m,K \le (2n-1)(n-2) + 2nm + 4n^2 m,K≤(2n−1)(n−2)+2nm+4n2m,

and if no basic operation applies in the final state, then fKf_KfK​ is a maximum flow. The paper states the bound as O(n2m)O(n^2m)O(n2m) and proves it as "immediate from Lemmas 3.8, 3.9, and 3.10"; the goal states the sum of those three printed bounds. Since every execution is this short, no order of operations runs forever.

Milestones

In the order the proof uses them: Lemma 2.1 (at an active vertex a push or a relabel applies); Lemma 3.1 (the labeling stays valid); Theorem 3.2 (Ford–Fulkerson: a flow is maximum iff ttt is unreachable from sss in GfG_fGf​); Lemma 3.3 (under a valid labeling ttt is unreachable from sss); Lemma 3.5 (from any vertex with positive excess, sss is reachable); Lemma 3.6 (labels never decrease; a relabeling increases the label); Lemma 3.7 (d(v)≤2n−1d(v) \le 2n-1d(v)≤2n−1 throughout); Theorem 3.4 (termination with finite labels gives a maximum flow); Lemma 3.8 (≤2n−1\le 2n-1≤2n−1 relabelings per vertex, ≤(2n−1)(n−2)<2n2\le (2n-1)(n-2) < 2n^2≤(2n−1)(n−2)<2n2 in total); Lemma 3.9 (≤2nm\le 2nm≤2nm saturating pushes); Lemma 3.10 (≤4n2m\le 4n^2m≤4n2m nonsaturating pushes, under m≥n−1m \ge n-1m≥n−1). A further, non-milestone item states the unnumbered invariant that every fkf_kfk​ is a preflow.

Significance

The generic bound shows that push-relabel terminates in a polynomial number of steps without any rule for choosing the next operation; the specific orderings of §4–§5 of the paper (first-in first-out, O(n3)O(n^3)O(n3); dynamic trees, O(nmlog⁡(n2/m))O(nm\log(n^2/m))O(nmlog(n2/m))) refine only the count of nonsaturating pushes, and reuse Lemmas 3.1–3.9 unchanged. The correctness argument, a valid labeling excludes augmenting paths, is the template for the push-relabel minimum-cost flow and assignment algorithms that followed.

These results are proved in the paper and are textbook material. Their machine-checked counterparts are, as far as is known here, not on the Prove2Me platform: the platform's network-flow statements (from Introduction to Linear Optimization, e.g. LinearOptimization.max_flow_min_cut) use a different model, with arc-indexed nonnegative flows and extended-real capacities, and contain nothing about preflows, labels or operation counts. This mission produces a formal account of the antisymmetric-flow model, of Ford–Fulkerson in that model, and of the amortized counting arguments, with the constants the paper prints.

Difficulty

The correctness half is short once the invariants are in place; the difficulty is in the counting. The label bound (Lemma 3.7) is a statement about the whole execution, and it depends on a structural fact about preflows (Lemma 3.5) whose truth rests on antisymmetry and on the nonnegativity of excesses. The obvious first idea for the push counts, bounding pushes per edge or per vertex locally, fails for nonsaturating pushes: flow pushed across a pair can be pushed back later, and nothing local limits how often this happens, so Lemma 3.10 holds only as an amortized statement over the entire execution and depends on both earlier counts. Saturating pushes on a pair can also recur, in both directions, and Lemma 3.9 has to control the interaction between the two directions.

Formally, all of this is reasoning about arbitrary interleavings of operations, with labels in N∪{∞}\mathbb{N} \cup \{\infty\}N∪{∞} and real-valued flows.

Formalization scope

  • Vertices form a finite type with decidable equality; nnn is its cardinality, s≠ts \ne ts=t, so n≥2n \ge 2n≥2 and the natural-number subtractions 2n−12n-12n−1 and n−2n-2n−2 are exact. Capacities are a real function on all pairs, nonnegative, zero on the diagonal; EEE is its support and mmm its cardinality.
  • Flows and preflows are antisymmetric real functions on all pairs (not nonnegative arc flows); the excess is computed from fff, never stored. A maximum flow is a flow whose value is at least that of every flow.
  • Labels live in ℕ∞, with ∞+1=∞\infty + 1 = \infty∞+1=∞; the relabel value is an infimum, which is ∞\infty∞ on the empty set.
  • An execution is a sequence of states σ : ℕ → State V with a length KKK, starting at the Fig. 2 state with the simple labeling (the paper's own assumption for its proofs), each step an applicable push or relabel. "Terminates" means that no basic operation applies, the loop guard of Fig. 2. The three counts are cardinalities of the sets of step indices of each kind.
  • Explicit constants: 2n−12n-12n−1 per-vertex relabelings, (2n−1)(n−2)<2n2(2n-1)(n-2) < 2n^2(2n−1)(n−2)<2n2 total relabelings, 2nm2nm2nm saturating pushes, 4n2m4n^2m4n2m nonsaturating pushes, label bound 2n−12n-12n−1, and the total (2n−1)(n−2)+2nm+4n2m(2n-1)(n-2)+2nm+4n^2m(2n−1)(n−2)+2nm+4n2m. The standing assumption m≥n−1m \ge n-1m≥n−1 appears only on Lemma 3.10 and the goal.
  • A trivializing formalization is ruled out: the step relation fixes the pushed amount δ=min⁡(e(v),rf(v,w))\delta = \min(e(v), r_f(v,w))δ=min(e(v),rf​(v,w)) and the new label exactly as in Fig. 1, termination is the loop guard rather than "the result is a flow", and a sorry-free check exhibits a concrete network s→a→ts \to a \to ts→a→t with a two-step execution (relabel aaa, then push (a,t)(a,t)(a,t)), so the run hypotheses are satisfiable.

Welcome contributions: proofs of the invariants (preflow, valid labeling, label monotonicity), of Ford–Fulkerson for antisymmetric flows (reusable beyond this mission), and of the counting lemmas. The FIFO bound of §4 is the subject of a companion mission.

Selected references

  • A. V. Goldberg, R. E. Tarjan, A New Approach to the Maximum-Flow Problem, Journal of the ACM 35(4):921–940, 1988. doi:10.1145/48014.61051
  • L. R. Ford, D. R. Fulkerson, Flows in Networks, Princeton University Press, 1962.
  • J. Edmonds, R. M. Karp, Theoretical improvements in algorithmic efficiency for network flow problems, Journal of the ACM 19(2):248–264, 1972. doi:10.1145/321694.321699
  • R. K. Ahuja, T. L. Magnanti, J. B. Orlin, Network Flows: Theory, Algorithms, and Applications, Prentice Hall, 1993.
17 thms3 active usersReviewed
🏆Completed
Algorithmic Game TheoryOptimizationProbability·Captain: mikedeng1

A Supply Chain Theory of Factoring and Reverse Factoring 2: The Retailer's Optimal Reverse Factoring Payment ExtensionResearch Paper

Motivation

Large retailers pay their suppliers weeks or months after delivery, and small suppliers fill the gap with short-term finance. In factoring the supplier sells the receivable to a factor for immediate cash; in reverse factoring the retailer arranges the program with a bank, which pays the supplier early at a rate priced on the retailer's credit rating. Retailers commonly attach a condition: the supplier must accept a longer payment term. Wuttke et al. (Journal of Operations Management, 2019) report that buyers extended payment terms by 54 days on average on adopting reverse factoring and that many suppliers delayed adoption; Corsten (2010) reports suppliers resisting a program because of the demanded payment delay (both as cited by Kouvelis and Xu, pp. 6082–6083). How long an extension a retailer can demand, and what it gains by demanding it, is therefore a practical design question.

Kouvelis and Xu (Management Science 67(10), 2021) answer it inside a Stackelberg supply chain model with credit and liquidity risk. This mission formalizes their answer, Proposition 6 of §5.3: the retailer's optimal payment extension when she keeps the existing wholesale price.

Setting

Demand D≥0D\ge0D≥0 has density fff, distribution function FFF and Fˉ=1−F\bar F=1-FFˉ=1−F; f>0f>0f>0 on [0,Z][0,\mathbb Z][0,Z] with Z≤+∞\mathbb Z\le+\inftyZ≤+∞ the upper end of the support, fff is continuous there, the mean is finite, and the failure rate z(ξ)=f(ξ)/Fˉ(ξ)z(\xi)=f(\xi)/\bar F(\xi)z(ξ)=f(ξ)/Fˉ(ξ) is strictly increasing. Write S(q)=∫0qFˉ(ξ) dξS(q)=\int_0^q\bar F(\xi)\,d\xiS(q)=∫0q​Fˉ(ξ)dξ for expected sales and k(q)=S(q)/Fˉ(q)k(q)=S(q)/\bar F(q)k(q)=S(q)/Fˉ(q).

A retailer (the leader) sets a wholesale price www, and a capital-constrained supplier (the follower) chooses a production quantity q≥0q\ge0q≥0; the retail price ppp exceeds the unit cost ccc. Each firm j∈{s,r}j\in\{s,r\}j∈{s,r} has a credit rating Cj∈(Cmin⁡,Cmax⁡)C_j\in(C_{\min},C_{\max})Cj​∈(Cmin​,Cmax​), a default probability ρj=ρ(Cj)∈[0,1]\rho_j=\rho(C_j)\in[0,1]ρj​=ρ(Cj​)∈[0,1] with ρ\rhoρ strictly decreasing, and an interest premium ηj=η(Cj)>0\eta_j=\eta(C_j)>0ηj​=η(Cj​)>0 with η\etaη decreasing. The lead time is t1t_1t1​, the payment term t2t_2t2​, and λs,λr≥0\lambda_s,\lambda_r\ge0λs​,λr​≥0 are the liquidity risks.

Under a post-shipment scheme with coefficient Λ\LambdaΛ the supplier earns

π(q;w)=(1−ρs)(Λe−λst1wS(q)−c q eηst1),\pi(q;w)=(1-\rho_s)\bigl(\Lambda e^{-\lambda_s t_1}wS(q)-c\,q\,e^{\eta_s t_1}\bigr),π(q;w)=(1−ρs​)(Λe−λs​t1​wS(q)−cqeηs​t1​),

with ΛF=(1−ρr)+(1−ρs)−eηst2\Lambda_{\mathcal F}=(1-\rho_r)+(1-\rho_s)-e^{\eta_s t_2}ΛF​=(1−ρr​)+(1−ρs​)−eηs​t2​ (recourse factoring), ΛN=e−ηrt2(1−ρr)\Lambda_{\mathcal N}=e^{-\eta_r t_2}(1-\rho_r)ΛN​=e−ηr​t2​(1−ρr​) (non-recourse factoring) and ΛR=e−ηr(t2+τ)\Lambda_{\mathcal R}=e^{-\eta_r(t_2+\tau)}ΛR​=e−ηr​(t2​+τ) (reverse factoring with payment extension τ≥0\tau\ge0τ≥0). The retailer earns Π=e−λst1(1−ρr)(p−w)S(q)\Pi=e^{-\lambda_s t_1}(1-\rho_r)(p-w)S(q)Π=e−λs​t1​(1−ρr​)(p−w)S(q) under factoring and

ΠR(w,τ)=e−λst1(1−ρr)(2−e−λrτ)(p−w)S(qR)\Pi_{\mathcal R}(w,\tau)=e^{-\lambda_s t_1}(1-\rho_r)(2-e^{-\lambda_r\tau})(p-w)S(q_{\mathcal R})ΠR​(w,τ)=e−λs​t1​(1−ρr​)(2−e−λr​τ)(p−w)S(qR​)

under reverse factoring, where qRq_{\mathcal R}qR​ is the supplier's best response. Its first-order condition is wFˉ(qR)=cR(τ)=c e(ηs+λs)t1+ηr(t2+τ)w\bar F(q_{\mathcal R})=c_{\mathcal R}(\tau)=c\,e^{(\eta_s+\lambda_s)t_1+\eta_r(t_2+\tau)}wFˉ(qR​)=cR​(τ)=ce(ηs​+λs​)t1​+ηr​(t2​+τ) (Eq. (12)).

Before reverse factoring, the supplier uses the better of the two factoring schemes. By Proposition 4 this is non-recourse, with equilibrium (wN∗,qN∗)(w^*_{\mathcal N},q^*_{\mathcal N})(wN∗​,qN∗​), when CN<Cs≤C1\mathbb C_{\mathcal N}<C_s\le\mathbb C_1CN​<Cs​≤C1​, and recourse, with (wF∗,qF∗)(w^*_{\mathcal F},q^*_{\mathcal F})(wF∗​,qF∗​), when Cs>CF∨C1C_s>\mathbb C_{\mathcal F}\vee\mathbb C_1Cs​>CF​∨C1​. The retailer keeps the existing wholesale price wsw_sws​ and solves problem (13): maximize ΠR(ws,τ)\Pi_{\mathcal R}(w_s,\tau)ΠR​(ws​,τ) over τ≥0\tau\ge0τ≥0, subject to the supplier's acceptance (his reverse factoring profit is at least his existing one). CRmax⁡\mathbb C^{\max}_{\mathcal R}CRmax​ is the rating at which ΛF=e−ηrt2\Lambda_{\mathcal F}=e^{-\eta_r t_2}ΛF​=e−ηr​t2​, and Ξ[0,z](x)=max⁡{0,min⁡{z,x}}\Xi_{[0,z]}(x)=\max\{0,\min\{z,x\}\}Ξ[0,z]​(x)=max{0,min{z,x}}.

Formalization targets

Goal: Proposition 6

(i) If Cs≥CRmax⁡C_s\ge\mathbb C^{\max}_{\mathcal R}Cs​≥CRmax​, reverse factoring is dominated by recourse factoring. (ii) If CN<Cs<CRmax⁡\mathbb C_{\mathcal N}<C_s<\mathbb C^{\max}_{\mathcal R}CN​<Cs​<CRmax​, reverse factoring should be offered with

τR∗=Ξ[0,τs](τ0∗),λrk(q)z(q)+ηr=2ηreλrτ0∗,wsFˉ(q)=cR(τ0∗),\tau^*_{\mathcal R}=\Xi_{[0,\tau_s]}(\tau^*_0),\qquad \lambda_r k(q)z(q)+\eta_r=2\eta_r e^{\lambda_r\tau^*_0},\quad w_s\bar F(q)=c_{\mathcal R}(\tau^*_0),τR∗​=Ξ[0,τs​]​(τ0∗​),λr​k(q)z(q)+ηr​=2ηr​eλr​τ0∗​,ws​Fˉ(q)=cR​(τ0∗​),

where τs=−ηr−1ln⁡(1−ρr)\tau_s=-\eta_r^{-1}\ln(1-\rho_r)τs​=−ηr−1​ln(1−ρr​) with ws=wN∗w_s=w^*_{\mathcal N}ws​=wN∗​ in the non-recourse case, and τs=−ηr−1ln⁡[(1−ρr)+(1−ρs)−eηst2]−t2\tau_s=-\eta_r^{-1}\ln[(1-\rho_r)+(1-\rho_s)-e^{\eta_s t_2}]-t_2τs​=−ηr−1​ln[(1−ρr​)+(1−ρs​)−eηs​t2​]−t2​ with ws=wF∗w_s=w^*_{\mathcal F}ws​=wF∗​ in the recourse case.

Milestones, in attack order

  1. Eq. (12): the supplier's best response under reverse factoring.
  2. Proposition 4: which factoring scheme is in force before reverse factoring.
  3. §5.3, τs\tau_sτs​: acceptance holds exactly on [0,τs][0,\tau_s][0,τs​].
  4. §5.3, τ0∗\tau^*_0τ0∗​: the retailer's unconstrained profit is unimodal around τ0∗\tau^*_0τ0∗​.

A follow-on item states Corollary 3(ii): the retailer's profit strictly increases, and the supplier's profit is unchanged when τ0∗≥τs\tau^*_0\ge\tau_sτ0∗​≥τs​.

Significance

Proposition 6 is the paper's prescription for program design. It says which suppliers should be offered reverse factoring: every supplier below the indifference rating CRmax⁡\mathbb C^{\max}_{\mathcal R}CRmax​ and above the non-recourse feasibility threshold. It also gives the extension in closed form, the unconstrained optimum clipped to the supplier's acceptance limit. Two consequences are drawn in the paper: non-recourse factoring is dominated once the extension is optimized, and reverse factoring may leave the supplier exactly as well off as before, so it is not necessarily a win-win (Corollary 3).

The proofs are in the paper's Online Appendix B and have not been machine-checked. A formal proof here produces a checked derivation of the projection formula from the model's primitives. It covers the strict-IFR analysis of the follower's response, the reduction of the acceptance constraint to an interval, and the unimodality of the retailer's objective. The same analysis of the pull game with an effective unit cost recurs across the supply chain finance literature.

Difficulty

The retailer's objective depends on τ\tauτ through two opposing channels: the liquidity factor 2−e−λrτ2-e^{-\lambda_r\tau}2−e−λr​τ increases, while expected sales S(qR(τ))S(q_{\mathcal R}(\tau))S(qR​(τ)) decrease because the supplier's effective cost rises. Neither factor is concave in τ\tauτ, and the objective need not be concave. The natural move, to set the derivative to zero and call the root a maximum, proves nothing without a sign analysis. That analysis needs the monotonicity of k⋅zk\cdot zk⋅z along the implicitly defined response qR(τ)q_{\mathcal R}(\tau)qR​(τ), which is where strict IFR enters. The acceptance constraint compares the supplier's profits in two different games (reverse factoring at τ\tauτ against the existing equilibrium). Reducing it to τ≤τs\tau\le\tau_sτ≤τs​ requires the supplier's best-response profit as an explicit increasing function of his quantity. Identifying the existing equilibrium requires Proposition 4, whose "adopted" compares equilibrium profits of two Stackelberg games.

Formalization scope

The model is a single Lean structure SupplyChainFactoring.Extension.Model. Demand is a probability measure on R\mathbb RR with a density fff, and Z\mathbb ZZ is an extended real. "Continuous p.d.f. with f>0f>0f>0 in [0,Z][0,\mathbb Z][0,Z]" is read as continuity on [0,Z][0,\mathbb Z][0,Z], with f=0f=0f=0 outside the support. Credit functions ρ,η\rho,\etaρ,η are real functions constrained on (Cmin⁡,Cmax⁡)(C_{\min},C_{\max})(Cmin​,Cmax​). The finance derivations behind the profit functions (Eqs. (1), (5), (6), Lemma 1) are not formalized: the profit functions are the model.

Readings of informal words, each also recorded in the item's Formalization Note:

  • Best response: a maximizer of the supplier's profit over q≥0q\ge0q≥0; equilibrium: a best response pair from which no nonnegative wholesale price with a best response gives the retailer more. Neither is defined through first-order conditions.
  • Feasible: some w≥0w\ge0w≥0 with a best response gives the retailer positive profit; adopted (Proposition 4): feasible, with equilibrium supplier profit at least (non-recourse) or strictly above (recourse) the other feasible scheme's.
  • Thresholds "the unique value of CsC_sCs​ that satisfies …" are hypotheses in exactly that form; cN=pc_{\mathcal N}=pcN​=p and cF=pc_{\mathcal F}=pcF​=p are cross-multiplied because ΛF\Lambda_{\mathcal F}ΛF​ can be ≤0\le0≤0.
  • In (13) www is fixed at wsw_sws​ (§5.3's first sentence, footnote 23). πR∗\pi^*_{\mathcal R}πR∗​ is the supplier's best-response profit under reverse factoring at (ws,τ)(w_s,\tau)(ws​,τ), and max⁡{πF∗,πN∗}\max\{\pi^*_{\mathcal F},\pi^*_{\mathcal N}\}max{πF∗​,πN∗​} is his profit in the existing equilibrium.
  • Dominated (Proposition 6(i)): at every τ≥0\tau\ge0τ≥0 and every www, the supplier's reverse factoring best-response profit is at most his recourse one. Should be offered (6(ii)): τR∗\tau^*_{\mathcal R}τR∗​ solves (13) and the retailer's profit is at least her existing equilibrium profit.
  • τ0∗\tau^*_0τ0∗​ is a hypothesis: it and some q∈(0,Z)q\in(0,\mathbb Z)q∈(0,Z) solve the paper's two equations (the paper does not argue existence). Its optimality "without the nonnegativity constraint" is stated as unimodality of ΠR\Pi_{\mathcal R}ΠR​ on the set of real τ\tauτ with cR(τ)<wc_{\mathcal R}(\tau)<wcR​(τ)<w.
  • Always increases (Corollary 3(ii)) is strict; may remain unchanged when τ0∗≥τs\tau^*_0\ge\tau_sτ0∗​≥τs​ is read as "is unchanged whenever τ0∗≥τs\tau^*_0\ge\tau_sτ0∗​≥τs​".

Three misprints of the paper are corrected: Ξ[0,z](x)=0\Xi_{[0,z]}(x)=0Ξ[0,z]​(x)=0 "if x<zx<zx<z" is read as "if x<0x<0x<0"; "the retailer's maximization problem in (16)" refers to (13); the middle line of the ΠR\Pi_{\mathcal R}ΠR​ display on p. 6082 carries a stray factor www, and the last line is used.

The hypotheses on τ0∗\tau^*_0τ0∗​ cannot be met when λr=0\lambda_r=0λr​=0, and the goal then says nothing about the case, as in the paper. The existing equilibrium, τs\tau_sτs​ and τ0∗\tau^*_0τ0∗​ are never free parameters: τs\tau_sτs​ is the paper's explicit formula, and the reduction of acceptance to τ≤τs\tau\le\tau_sτ≤τs​ is a milestone to be proved, not an assumption. Every logarithm is applied to a quantity the hypotheses force positive. A formalization that assumed acceptance equivalent to τ≤τs\tau\le\tau_sτ≤τs​ or assumed unimodality would be trivial and is excluded.

The pull game with an effective cost has the same structure as Cachon's pull contract without salvage value (platform items CachonPushPull.*), but those items assume IGFR demand with a salvage value, so they are not reused. Reusable infrastructure welcome: the strict-IFR lemmas (kkk, k⋅zk\cdot zk⋅z and k(q)−qk(q)-qk(q)−q increasing) and the explicit best response of a newsvendor-type follower.

Selected references

  • P. Kouvelis, F. Xu, A Supply Chain Theory of Factoring and Reverse Factoring, Management Science 67(10):6071–6088, 2021. https://doi.org/10.1287/mnsc.2020.3788
  • G. P. Cachon, The Allocation of Inventory Risk in a Supply Chain: Push, Pull, and Advance-Purchase Discount Contracts, Management Science 50(2):222–238, 2004. https://doi.org/10.1287/mnsc.1030.0190
  • D. A. Wuttke, E. S. Rosenzweig, H. S. Heese, An Empirical Analysis of Supply Chain Finance Adoption, Journal of Operations Management 65(3):242–261, 2019. https://doi.org/10.1002/joom.1023
6 thms3 active usersReviewed
🏆Completed
ProbabilityStatistics·Captain: mikedeng1

Are Call Center and Hospital Arrivals Well Modeled by Nonhomogeneous Poisson Processes?: Combining k Equal Subintervals of a Linear Arrival Rate Bounds the Degree of Nonhomogeneity by C/kResearch Paper

Motivation

Arrival processes to call centers and hospital emergency departments are routinely modeled as nonhomogeneous Poisson processes (NHPPs): Poisson processes whose arrival rate varies over the day. Staffing and queueing models built on this assumption are only as good as the assumption itself, so practitioners test it on data. The standard test, going back to Brown et al. (2005, doi:10.1198/016214504000001808), divides the day into short subintervals, treats the rate as constant on each, rescales the arrival times within each subinterval to [0,1][0,1][0,1], combines all the rescaled data, and applies a Kolmogorov–Smirnov (KS) test of uniformity.

Kim and Whitt (2014, doi:10.1287/msom.2014.0490) ask when this piecewise-constant approximation is justified. If the true rate is not constant on a subinterval, the rescaled arrival times are not uniform, and with enough data the KS test rejects the Poisson hypothesis even when the process really is an NHPP. Section 3 of the paper quantifies this effect through a single number, the degree of nonhomogeneity, and shows how it behaves when the interval is cut into kkk equal pieces. This mission formalizes that section's exact computations for a linear arrival rate.

Setting

An arrival rate function λ\lambdaλ on an interval [0,T][0,T][0,T], T>0T > 0T>0, is nonnegative, integrable, and strictly positive except at finitely many points. Its cumulative arrival rate is

Λ(t)=∫0tλ(s) ds.\Lambda(t) = \int_0^t \lambda(s)\,ds .Λ(t)=∫0t​λ(s)ds.

Conditionally on nnn arrivals in [0,T][0,T][0,T], the arrival times of an NHPP with rate λ\lambdaλ, divided by TTT, are distributed as the order statistics of nnn independent random variables on [0,1][0,1][0,1] with the conditional cdf

F(t)=Λ(tT)Λ(T),0≤t≤1.F(t) = \frac{\Lambda(tT)}{\Lambda(T)}, \qquad 0 \le t \le 1 .F(t)=Λ(T)Λ(tT)​,0≤t≤1.

The degree of nonhomogeneity is the Kolmogorov distance of FFF from the uniform cdf,

D=sup⁡0≤t≤1∣F(t)−t∣.D = \sup_{0 \le t \le 1} |F(t) - t| .D=0≤t≤1sup​∣F(t)−t∣.

It is zero exactly when λ\lambdaλ is constant, and it is the limit of the KS test statistic as the amount of data grows.

For k≥1k \ge 1k≥1, divide [0,T][0,T][0,T] into kkk subintervals of length T/kT/kT/k. For 1≤j≤k1 \le j \le k1≤j≤k the jjj-th subinterval has cumulative rate Λj(t)=Λ((j−1)T/k+t)−Λ((j−1)T/k)\Lambda_j(t) = \Lambda((j-1)T/k + t) - \Lambda((j-1)T/k)Λj​(t)=Λ((j−1)T/k+t)−Λ((j−1)T/k), conditional cdf Fj(t)=Λj(tT/k)/Λj(T/k)F_j(t) = \Lambda_j(tT/k)/\Lambda_j(T/k)Fj​(t)=Λj​(tT/k)/Λj​(T/k), and share of arrivals pj=(Λ(jT/k)−Λ((j−1)T/k))/Λ(T)p_j = (\Lambda(jT/k) - \Lambda((j-1)T/k))/\Lambda(T)pj​=(Λ(jT/k)−Λ((j−1)T/k))/Λ(T). The data of all subintervals, each rescaled to [0,1][0,1][0,1] and combined, have the conditional cdf F=∑j=1kpjFjF = \sum_{j=1}^k p_j F_jF=∑j=1k​pj​Fj​ (LEMMA 1).

The linear arrival rate is λ(t)=a+bt\lambda(t) = a + btλ(t)=a+bt with b≥0b \ge 0b≥0 and a≥0a \ge 0a≥0, not identically zero. When a>0a > 0a>0 its relative slope is r=b/ar = b/ar=b/a; on the jjj-th subinterval the relative slope is rj=b/λ((j−1)T/k)r_j = b/\lambda((j-1)T/k)rj​=b/λ((j−1)T/k).

In the Lean development these are cumRate, condCdf, degree, subCum, subCdf, weight, mixCdf, linRate and subSlope, in the namespace NHPPArrivals.LinearRate.

Formalization targets

Goal: THEOREM 5, combining equally spaced subintervals

For the linear rate, there is a constant CCC such that for every k≥1k \ge 1k≥1

D=sup⁡0≤t≤1∣F(t)−t∣=∑j=1kpjDj=∑j=1kpjsup⁡0≤t≤1∣Fj(t)−t∣,(20)D = \sup_{0 \le t \le 1}|F(t) - t| = \sum_{j=1}^k p_j D_j = \sum_{j=1}^k p_j \sup_{0 \le t \le 1}|F_j(t) - t|, \tag{20}D=0≤t≤1sup​∣F(t)−t∣=j=1∑k​pj​Dj​=j=1∑k​pj​0≤t≤1sup​∣Fj​(t)−t∣,(20)

with, if a>0a > 0a>0,

D=∑j=1kpj rjT/k8+4rjT/k,(21)D = \sum_{j=1}^k \frac{p_j\, r_j T/k}{8 + 4 r_j T/k}, \tag{21}D=j=1∑k​8+4rj​T/kpj​rj​T/k​,(21)

and, if a=0a = 0a=0,

D=p14+∑j=2kpj/(j−1)8+4/(j−1),(22)D = \frac{p_1}{4} + \sum_{j=2}^k \frac{p_j/(j-1)}{8 + 4/(j-1)}, \tag{22}D=4p1​​+j=2∑k​8+4/(j−1)pj​/(j−1)​,(22)

and in both cases D≤C/kD \le C/kD≤C/k. The constant CCC may depend on aaa, bbb and TTT, but not on kkk; its value is left open, as in the paper.

Milestones

  1. LEMMA 1, (17): for a general rate, the rescaled combined data have cdf ∑jpjFj\sum_j p_j F_j∑j​pj​Fj​, and the pjp_jpj​ form a probability vector.
  2. THEOREM 4, a>0a > 0a>0, (14), (16): F(t)=(tT+r(tT)2/2)/(T+rT2/2)F(t) = (tT + r(tT)^2/2)/(T + rT^2/2)F(t)=(tT+r(tT)2/2)/(T+rT2/2) and D=∣F(1/2)−1/2∣=rT/(8+4rT)D = |F(1/2) - 1/2| = rT/(8 + 4rT)D=∣F(1/2)−1/2∣=rT/(8+4rT).
  3. THEOREM 4, a=0a = 0a=0, (15): F(t)=t2F(t) = t^2F(t)=t2 and D=1/4D = 1/4D=1/4.
  4. LEMMA 1, (18): closed forms of Λj\Lambda_jΛj​, FjF_jFj​, pjp_jpj​, rjr_jrj​ when a>0a > 0a>0.
  5. LEMMA 1, (19): closed forms of Λj\Lambda_jΛj​, FjF_jFj​, pjp_jpj​, rjr_jrj​ when a=0a = 0a=0.
  6. THEOREM 5, (20): D=∑jpjDjD = \sum_j p_j D_jD=∑j​pj​Dj​ for one fixed kkk.

Significance

The result gives a quantitative criterion for the piecewise-constant approximation: for a linear rate, cutting the interval into kkk equal pieces reduces the degree of nonhomogeneity of the combined data by a factor of order 1/k1/k1/k. Since the KS critical value at sample size nnn is of order 1/n1/\sqrt n1/n​, this tells a practitioner how fine the subintervals must be, relative to the amount of data, before a KS test of the Poisson hypothesis stops rejecting merely because the rate varies within subintervals. The paper's later THEOREM 6 and its practical guidelines (§3.4, §3.6) rest on these formulas.

The results are proved in the paper by direct calculation; none of them has a machine-checked proof. The mission produces a verified library of the conditional-cdf calculus for NHPPs on an interval (the conditional cdf, its degree of nonhomogeneity, the subinterval decomposition) and the exact linear-rate formulas that the testing literature cites.

Difficulty

The computations are elementary, but two steps are not immediate. First, the supremum of ∣F(t)−t∣|F(t) - t|∣F(t)−t∣ over [0,1][0,1][0,1] is a supremum of a nonsmooth function; showing that it is attained at t=1/2t = 1/2t=1/2 requires knowing the sign of F(t)−tF(t) - tF(t)−t on the whole interval, and for the combined cdf it requires that all the pieces FjF_jFj​ attain their maximal deviation at the same point, which is special to linear rates. For a general rate the naive identity D=∑jpjDjD = \sum_j p_j D_jD=∑j​pj​Dj​ fails: the sup of a sum is at most the sum of the sups, with equality only when the maximizers coincide. Second, LEMMA 1 is a statement about the law of a rescaled random variable (the fractional part of kX/TkX/TkX/T), which requires splitting a measure along the kkk subintervals and handling their boundary points.

Formalization scope

Rates are real functions λ:R→R\lambda : \mathbb R \to \mathbb Rλ:R→R; only their values on [0,T][0,T][0,T] enter. Λ\LambdaΛ is an interval integral, subintervals are indexed by j∈{1,…,k}j \in \{1, \dots, k\}j∈{1,…,k} with k,jk, jk,j natural numbers cast to reals, and (j−1)(j-1)(j−1) is computed in R\mathbb RR. All quotients are real divisions; the hypotheses of every statement (T>0T > 0T>0, k≥1k \ge 1k≥1, b≥0b \ge 0b≥0, and a>0a > 0a>0 or b>0b > 0b>0 for the linear rate; integrability, nonnegativity and a finite zero set for a general rate) make every denominator Λ(T)\Lambda(T)Λ(T) and Λj(T/k)\Lambda_j(T/k)Λj​(T/k) positive. b≥0b \ge 0b≥0 is the paper's standing assumption of §3.3; excluding a=b=0a = b = 0a=b=0 is §3.2's requirement that the rate be positive except at finitely many points. The degree of nonhomogeneity is sSup of the image of [0,1][0,1][0,1], and every statement that uses it also asserts that the supremum is attained, so no default value of sSup can make a statement true. The constant CCC of THEOREM 5 is quantified before kkk; choosing it after kkk would make the bound empty. The statements are about the general definitions of (17) applied to λ(t)=a+bt\lambda(t) = a + btλ(t)=a+bt, not about the closed forms (18)–(19), which are separate milestones. The formula for rjr_jrj​ in (19) is stated for 2≤j≤k2 \le j \le k2≤j≤k only: r1=b/λ(0)r_1 = b/\lambda(0)r1​=b/λ(0) is undefined when a=0a = 0a=0.

The Poisson process itself is not formalized. LEMMA 1's "i.i.d. random variables" is the paper's THEOREM 1 (the conditioning property) applied to each arrival; LEMMA 1 is stated for the law of one arrival time, the probability measure with density λ/Λ(T)\lambda/\Lambda(T)λ/Λ(T) on [0,T][0,T][0,T]. THEOREM 1, THEOREMS 2–3 and COROLLARY 1 (limits of the empirical cdf and of the KS statistic) are out of scope: they need a point-process layer, the Glivenko–Cantelli theorem and KS critical values, none of which exists in Mathlib. THEOREM 6 is out of scope because the paper gives only a sketch comparing DDD with the KS critical value.

Contributions welcome: proofs of the milestones, general lemmas on sups of ∣F(t)−t∣|F(t) - t|∣F(t)−t∣ for convex cdfs, and the measure-splitting argument of LEMMA 1, which is reusable for any subinterval-based test of the Poisson hypothesis.

Selected references

  • S.-H. Kim and W. Whitt, Are call center and hospital arrivals well modeled by nonhomogeneous Poisson processes?, Manufacturing & Service Operations Management 16(3):464–480, 2014. doi:10.1287/msom.2014.0490
  • L. Brown, N. Gans, A. Mandelbaum, A. Sakov, H. Shen, S. Zeltyn, L. Zhao, Statistical analysis of a telephone call center: a queueing-science perspective, Journal of the American Statistical Association 100(469):36–50, 2005. doi:10.1198/016214504000001808
  • F. J. Massey, The Kolmogorov–Smirnov test for goodness of fit, Journal of the American Statistical Association 46(253):68–78, 1951. doi:10.1080/01621459.1951.10500769
8 thms3 active usersReviewed
🏆Completed
Dynamic ProgrammingOptimizationTopology·Captain: mikedeng1

The Structure of Dynamic Programing Models: A Solution of the Principle of Optimality with Vanishing Tail Is the Optimal ReturnResearch Paper

Motivation

Dynamic programming, as introduced by Bellman in the early 1950s, solves sequential decision problems through a functional equation: the value of a problem started in a given state equals the best one-stage return plus the value of the problem started in the state that decision leads to. In practice the argument usually runs backwards. One writes down the functional equation, finds or characterizes a solution, and reads off the structure of optimal decisions from that solution. This is legitimate only if two things hold: an optimal policy exists at all, and the solution of the functional equation that was found is the optimal value, not some other solution of the same equation.

Samuel Karlin's 1955 paper The Structure of Dynamic Programing Models (Naval Research Logistics Quarterly 2(4):285–294) gives an abstract deterministic model in which both questions can be posed precisely. It proves existence of optimal strategies by a compactness argument (Theorem 1), derives the functional equation, which it calls the Principle of Optimality, and identifies the condition under which a solution of that equation is the optimal return: a tail term must vanish. Later treatments of dynamic programming on general state spaces, such as Blackwell's discounted and positive programming (1965–1967) and the monographs of Bertsekas and Shreve, state their verification theorems in the same form, with a solution of the optimality equation plus a condition at infinity.

Setting

The model has a state space Ω\OmegaΩ, a Hausdorff topological space, and a decision space DDD, a nonempty compact Hausdorff space. A strategy is a sequence s=(δ1,δ2,… )s = (\delta_1, \delta_2, \dots)s=(δ1​,δ2​,…) of decisions, one per stage. The strategy space S=D×D×⋯S = D \times D \times \cdotsS=D×D×⋯ carries the product topology and is compact by Tychonoff's theorem.

The data are:

  • a return function L:Ω×D→RL : \Omega \times D \to \mathbb{R}L:Ω×D→R, continuous and non-negative, where L(ω,δ)L(\omega, \delta)L(ω,δ) is the return for taking decision δ\deltaδ in state ω\omegaω;
  • a transition (δ,ω)↦Tδ ω∈Ω(\delta, \omega) \mapsto T_\delta\,\omega \in \Omega(δ,ω)↦Tδ​ω∈Ω, the state faced at the next stage after decision δ\deltaδ in state ω\omegaω;
  • a normalization factor P:D→RP : D \to \mathbb{R}P:D→R, continuous and positive.

From an initial state ω\omegaω, a strategy sss generates the trajectory ω1=ω\omega_1 = \omegaω1​=ω, ωn=Tδn−1 ωn−1\omega_n = T_{\delta_{n-1}}\,\omega_{n-1}ωn​=Tδn−1​​ωn−1​, and the weights Pn(s)=∏i=1n−1P(δi)P_n(s) = \prod_{i=1}^{n-1} P(\delta_i)Pn​(s)=∏i=1n−1​P(δi​) with P1(s)=1P_1(s) = 1P1​(s)=1. The total yield is

Φ(ω,s)=∑n=1∞L(ωn,δn) Pn(s),\Phi(\omega, s) = \sum_{n=1}^{\infty} L(\omega_n, \delta_n)\, P_n(s),Φ(ω,s)=n=1∑∞​L(ωn​,δn​)Pn​(s),

and the optimal return is K(ω)=max⁡s∈SΦ(ω,s)K(\omega) = \max_{s \in S} \Phi(\omega, s)K(ω)=maxs∈S​Φ(ω,s). The standing assumption of the paper, display (1), is that the partial sums ∑n=1kL(ωn,δn)Pn(s)\sum_{n=1}^{k} L(\omega_n, \delta_n) P_n(s)∑n=1k​L(ωn​,δn​)Pn​(s) converge uniformly in s∈Ss \in Ss∈S for each ω\omegaω.

Formalization targets

Goal: uniqueness of solutions with vanishing tail

Let M:Ω→RM : \Omega \to \mathbb{R}M:Ω→R solve the functional equation

M(ω)=max⁡δ∈D{L(ω,δ)+P(δ) M(Tδ ω)}for all ω,M(\omega) = \max_{\delta \in D} \bigl\{ L(\omega, \delta) + P(\delta)\, M(T_\delta\,\omega) \bigr\} \quad \text{for all } \omega,M(ω)=δ∈Dmax​{L(ω,δ)+P(δ)M(Tδ​ω)}for all ω,

with the maximum attained, and suppose that for every ω\omegaω

lim⁡n→∞ sup⁡δ1,…,δn∣M(ωn)∣∏i=1n−1P(δi)=0.\lim_{n \to \infty} \ \sup_{\delta_1, \dots, \delta_n} |M(\omega_n)| \prod_{i=1}^{n-1} P(\delta_i) = 0.n→∞lim​ δ1​,…,δn​sup​∣M(ωn​)∣i=1∏n−1​P(δi​)=0.

Then M(ω)=max⁡s∈SΦ(ω,s)M(\omega) = \max_{s \in S} \Phi(\omega, s)M(ω)=maxs∈S​Φ(ω,s) for every ω\omegaω, and the maximum is attained (pp. 290–291, §Uniqueness).

Milestones

  1. Theorem 1 (p. 287). If the series (1) converges uniformly in SSS, an optimal strategy s∗s^*s∗ exists: Φ(ω,s∗)=max⁡SΦ(ω,s)\Phi(\omega, s^*) = \max_S \Phi(\omega, s)Φ(ω,s∗)=maxS​Φ(ω,s).
  2. Shift identity (p. 290, first display). For a strategy sss with convergent yield series and the shift s′=(δ2,δ3,… )s' = (\delta_2, \delta_3, \dots)s′=(δ2​,δ3​,…),
Φ(ω,s)=L(ω,δ1)+P(δ1) Φ(Tδ1 ω,s′).\Phi(\omega, s) = L(\omega, \delta_1) + P(\delta_1)\, \Phi(T_{\delta_1}\,\omega, s').Φ(ω,s)=L(ω,δ1​)+P(δ1​)Φ(Tδ1​​ω,s′).
  1. Principle of Optimality, eq. (2) (p. 290). K(ω)=max⁡δ1{L(ω,δ1)+P(δ1)K(Tδ1 ω)}K(\omega) = \max_{\delta_1} \{ L(\omega, \delta_1) + P(\delta_1) K(T_{\delta_1}\,\omega) \}K(ω)=maxδ1​​{L(ω,δ1​)+P(δ1​)K(Tδ1​​ω)}.
  2. n-step expansion (p. 291, first display). A solution MMM of (2) satisfies, for every nnn,
M(ω)=max⁡δ1,…,δn{∑m=1nL(ωm,δm)Pm(s)+M(ωn+1)Pn+1(s)}.M(\omega) = \max_{\delta_1, \dots, \delta_n} \Bigl\{ \sum_{m=1}^{n} L(\omega_m, \delta_m) P_m(s) + M(\omega_{n+1}) P_{n+1}(s) \Bigr\}.M(ω)=δ1​,…,δn​max​{m=1∑n​L(ωm​,δm​)Pm​(s)+M(ωn+1​)Pn+1​(s)}.

Significance

Milestone 3 says that the optimal return solves the functional equation. The goal gives the converse on a class of candidate solutions: any solution with a vanishing tail is the optimal return. Together they justify solving a dynamic program by solving its functional equation. The paper's two examples, a two-operation allocation problem with discounting and a resource allocation model driving the state to the origin, obtain uniqueness among bounded solutions and among continuous solutions vanishing at the origin respectively, by checking the tail condition. Without the tail condition the conclusion fails; the paper notes that the limit term "need not be true in general for any solution to the functional equation".

All four milestones and the goal are classical results with published proofs. None of them has a machine-checked proof on this platform: its existing Bellman-equation theorems concern finite-state stochastic models with a constant discount factor, and this model has neither restriction. The mission produces a formal version of the general deterministic model on topological state spaces, with optimality characterized by a verification theorem, which later missions on the paper's examples can reuse.

Difficulty

Existence rests on continuity of s↦Φ(ω,s)s \mapsto \Phi(\omega, s)s↦Φ(ω,s) on the product space. Each term L(ωn,δn)Pn(s)L(\omega_n, \delta_n) P_n(s)L(ωn​,δn​)Pn​(s) depends on the first nnn decisions through the composite map ωn=Tδn−1∘⋯∘Tδ1 ω\omega_n = T_{\delta_{n-1}} \circ \cdots \circ T_{\delta_1}\,\omegaωn​=Tδn−1​​∘⋯∘Tδ1​​ω, and continuity of that composite in all decisions at once does not follow from separate continuity of Tδ ωT_\delta\,\omegaTδ​ω in δ\deltaδ and in ω\omegaω. The limit of the series is continuous only because the convergence is uniform.

For uniqueness, the paper's display "M(ω)=max⁡SΦ(ω,s)+lim⁡nmax⁡M(ωn)∏P(δi)M(\omega) = \max_S \Phi(\omega, s) + \lim_n \max M(\omega_n) \prod P(\delta_i)M(ω)=maxS​Φ(ω,s)+limn​maxM(ωn​)∏P(δi​)" is not an identity: a maximum of a sum is not the sum of the maxima. A proof has to bound MMM from above by the yield of every strategy and from below by the yield of one particular strategy, and the lower bound fails if the tail term is controlled only from above. Attainment of the maximum in the conclusion needs a strategy to be exhibited, not only a supremum computed.

Formalization scope

A strategy is a function s : ℕ → D, with the product topology. Lean's 0-based index kkk is the paper's stage k+1k+1k+1: s 0 is δ1\delta_1δ1​, trajectory T ω s 0 is ω1=ω\omega_1 = \omegaω1​=ω, and weight P s k is Pk+1(s)P_{k+1}(s)Pk+1​(s), so weight P s 0 = 1. The total yield is a tsum and the optimal return a supremum over all strategies. "Maximum" is encoded as IsGreatest of a range, so every stated maximum is attained. Uniform convergence of (1) is TendstoUniformly of the partial sums to Φ(ω,⋅)\Phi(\omega, \cdot)Φ(ω,⋅) along atTop.

The formalization commits to the following, relative to the page:

  • The model's assumptions (1)–(3) of p. 286, non-negativity of LLL, and positivity and continuity of PPP appear as hypotheses of every statement.
  • A single return function LLL is used, not stage-dependent LnL_nLn​, as in display (1) and as the functional equation (2) requires. PnP_nPn​ has the product form, which the paper adopts "unless stated to the contrary".
  • Assumption (4), separate continuity of Tδ ωT_\delta\,\omegaTδ​ω in δ\deltaδ and in ω\omegaω, is strengthened to joint continuity of (δ,ω)↦Tδ ω(\delta, \omega) \mapsto T_\delta\,\omega(δ,ω)↦Tδ​ω. This supports the paper's assertion (p. 287) that each term is a continuous function of sss, which separate continuity does not give.
  • DDD is assumed nonempty. With DDD empty there is no strategy and Theorem 1 is false.
  • The paper leaves the class of admissible solutions open ("an appropriate class of M's for which the lim = 0"). The goal fixes it as the two-sided condition: for every ω\omegaω and ε>0\varepsilon > 0ε>0 there is NNN with ∣M(ωn)∣ Pn(s)≤ε|M(\omega_n)|\,P_n(s) \le \varepsilon∣M(ωn​)∣Pn​(s)≤ε for all n≥Nn \ge Nn≥N and all sss. Both of the paper's examples verify this form.

Lean's tsum of a non-summable series is 000, and a supremum of an unbounded family is 000. Neither default can make a statement trivially true. The uniform-convergence hypothesis forces the series to converge, and under Theorem 1's hypotheses Φ(ω,⋅)\Phi(\omega, \cdot)Φ(ω,⋅) is continuous on a compact space, so the supremum is a maximum. The goal's conclusion is stated without a supremum. A sorry-free check confirms that all hypotheses of the goal hold on a concrete instance with non-zero return: two decisions, constant return 111, P≡1/2P \equiv 1/2P≡1/2, and M≡2M \equiv 2M≡2.

Needed infrastructure: Tychonoff's theorem, continuity of uniform limits, and attainment of maxima on compact spaces, all in Mathlib. The mission's own definitions are the trajectory, weights, partial and total yield, and the optimal return. Contributions of intermediate lemmas are welcome, for example continuity of s↦ωns \mapsto \omega_ns↦ωn​, summability from uniform convergence, and the upper and lower tail estimates for MMM. So are formalizations of the paper's Remarks 1 and 3 (Dini's theorem and convergence of the kkk-stage optimal returns).

Selected references

  • S. Karlin, The Structure of Dynamic Programing Models, Naval Research Logistics Quarterly 2(4):285–294, 1955. https://doi.org/10.1002/nav.3800020408
  • R. Bellman, Dynamic Programming, Princeton University Press, 1957. https://press.princeton.edu/books/paperback/9780691146683/dynamic-programming
  • D. Blackwell, Discounted Dynamic Programming, Annals of Mathematical Statistics 36(1):226–235, 1965. https://doi.org/10.1214/aoms/1177700285
  • D. P. Bertsekas and S. E. Shreve, Stochastic Optimal Control: The Discrete-Time Case, Academic Press, 1978. https://web.mit.edu/dimitrib/www/soc.html
6 thms3 active usersReviewed
Discrete GeometryNumber Theory·Captain: mikedeng1

Minkowski's Convex Body Theorem and Integer Programming: Lattice-Free Convex Bodies Meet Few Translates of an Integral SubspaceResearch Paper

Motivation

Integer programming asks whether a system of linear inequalities Ax≤bAx\le bAx≤b has a solution x∈Znx\in\mathbb Z^nx∈Zn. In fixed dimension nnn it is solvable in polynomial time: Lenstra (1983) proved this by showing that a convex body without integer points is "flat" in some integral direction, so that the search splits into few lower-dimensional subproblems. Kannan's 1987 paper in Mathematics of Operations Research sharpened this approach. It computes a Korkine–Zolotarev ("reduced") basis of a lattice, solves the shortest and closest vector problems exactly in nO(n)n^{O(n)}nO(n) operations, and runs integer programming in O(n9n/2s)O(n^{9n/2}s)O(n9n/2s) arithmetic operations. Underneath the algorithm sits a purely geometric statement, Theorem (5.5): a lattice-free convex body meets only boundedly many integer translates of some integral subspace.

Timeline.

  • Korkine and Zolotareff (1873): the reduced bases used here.
  • Minkowski (1896): a symmetric convex body of volume greater than 2n2^n2n contains a nonzero integer point.
  • Khinchine (1948): lattice-free convex bodies have lattice width bounded by a function of nnn alone (the flatness theorem).
  • Lenstra (1983): integer programming in fixed dimension is polynomial, via a flat direction.
  • Kannan (1987, this paper): Theorem (5.5), with subspaces VVV of any dimension between 111 and n−1n-1n−1 and an explicit bound n2(n−dim⁡V)n^{2(n-\dim V)}n2(n−dimV).
  • Kannan and Lovász (1988), Banaszczyk et al. (1999), and later work: polynomial bounds on the flatness constant.

Setting

Rn\mathcal R^nRn is Euclidean space with dot product (a,b)(a,b)(a,b) and length ∣a∣|a|∣a∣, and Zn\mathbb Z^nZn is the set of integer vectors. For linearly independent b1,…,bm∈Rkb_1,\dots,b_m\in\mathcal R^kb1​,…,bm​∈Rk, the lattice L(b1,…,bm)L(b_1,\dots,b_m)L(b1​,…,bm​) is the set of integer combinations ∑jλjbj\sum_j\lambda_jb_j∑j​λj​bj​, λj∈Z\lambda_j\in\mathbb Zλj​∈Z, and b1,…,bmb_1,\dots,b_mb1​,…,bm​ is a basis. Gram–Schmidt orthogonalisation gives b1∗,…,bm∗b_1^*,\dots,b_m^*b1∗​,…,bm∗​ and unit vectors uj=bj∗/∣bj∗∣u_j=b_j^*/|b_j^*|uj​=bj∗​/∣bj∗​∣, and bi(j)=(bi,uj)b_i(j)=(b_i,u_j)bi​(j)=(bi​,uj​), so bi=∑jbi(j)ujb_i=\sum_jb_i(j)u_jbi​=∑j​bi​(j)uj​ and bj(j)=∣bj∗∣b_j(j)=|b_j^*|bj​(j)=∣bj∗​∣. The determinant is d(L)=∏j∣bj∗∣d(L)=\prod_j|b_j^*|d(L)=∏j​∣bj∗​∣. Λ1(L)\Lambda_1(L)Λ1​(L) is the length of a shortest nonzero vector of LLL. The projected lattice Lj(b1,…,bm)L_j(b_1,\dots,b_m)Lj​(b1​,…,bm​) is the image of LLL under orthogonal projection onto the complement of span⁡(b1,…,bj−1)\operatorname{span}(b_1,\dots,b_{j-1})span(b1​,…,bj−1​). A basis is reduced (Definition 2.6) if bj(j)=Λ1(Lj)b_j(j)=\Lambda_1(L_j)bj​(j)=Λ1​(Lj​) for every jjj and ∣bi(j)∣≤bj(j)/2|b_i(j)|\le b_j(j)/2∣bi​(j)∣≤bj​(j)/2 for i>ji>ji>j.

A convex body is a convex set of positive volume, which for a convex set means nonempty interior. A subspace VVV has a basis of integer vectors if it is the real span of integer vectors. Its integer translates are the sets z+Vz+Vz+V with z∈Znz\in\mathbb Z^nz∈Zn.

In Lean, Rk\mathcal R^kRk is EuclideanSpace ℝ (Fin k), a basis is b : Fin m → EuclideanSpace ℝ (Fin k), the lattice is lattice b = Submodule.span ℤ (Set.range b), ∣bj∗∣|b_j^*|∣bj∗​∣ is gsLen b j, bi(j)b_i(j)bi​(j) is gsCoeff b i j, d(L)d(L)d(L) is latticeDet b, Λ1\Lambda_1Λ1​ is lambdaOne, Lj+1L_{j+1}Lj+1​ is projLattice b j, and a reduced basis is IsReduced b.

Formalization targets

Goal: Theorem (5.5), corrected reading

For n≥2n\ge2n≥2 and every bounded convex set K⊆RnK\subseteq\mathcal R^nK⊆Rn with nonempty interior and K∩Zn=∅K\cap\mathbb Z^n=\emptysetK∩Zn=∅ there is a subspace VVV spanned by integer vectors with 1≤dim⁡V≤n−11\le\dim V\le n-11≤dimV≤n−1 and

#{ z+V:z∈Zn, (z+V)∩K≠∅ } ≤ n2(n−dim⁡V).\#\{\,z+V : z\in\mathbb Z^n,\ (z+V)\cap K\ne\emptyset\,\}\ \le\ n^{2(n-\dim V)}.#{z+V:z∈Zn, (z+V)∩K=∅} ≤ n2(n−dimV).

The printed theorem allows "an iii dimensional space VVV" with 1≤i≤n1\le i\le n1≤i≤n and bound n2(n−i+1)n^{2(n-i+1)}n2(n−i+1). Taken literally that is trivial (V=RnV=\mathcal R^nV=Rn, one translate). The proof on the same page takes V=span⁡(b1,…,bi−1)V=\operatorname{span}(b_1,\dots,b_{i-1})V=span(b1​,…,bi−1​), of dimension i−1i-1i−1, and remarks that this "ensures that the subspace VVV is always of dimension at least 1". The goal states that reading.

Milestones

  1. Theorem (1.11), Minkowski's convex body theorem (referenced from the platform, in Mathlib's general form).
  2. Theorem (1.12): every mmm-dimensional lattice has a nonzero vector with ∣v∣≤m d(L)1/m|v|\le\sqrt m\,d(L)^{1/m}∣v∣≤m​d(L)1/m.
  3. Proposition 1.9: a primitive lattice vector belongs to some basis.
  4. Proposition 2.16, existence form: every lattice has a reduced basis.
  5. Proposition 4.2: for any b0b_0b0​ with projection bˉ0\bar b_0bˉ0​ onto the span, some b∈Lb\in Lb∈L has ∣b−bˉ0∣≤12(∑jbj(j)2)1/2≤m2max⁡jbj(j)|b-\bar b_0|\le\frac12(\sum_jb_j(j)^2)^{1/2}\le\frac{\sqrt m}2\max_jb_j(j)∣b−bˉ0​∣≤21​(∑j​bj​(j)2)1/2≤2m​​maxj​bj​(j).
  6. Proposition 4.3: for a reduced basis and iii maximising bi(i)b_i(i)bi​(i), the tail (λi,…,λm)(\lambda_i,\dots,\lambda_m)(λi​,…,λm​) of every closest lattice point to b0b_0b0​ lies in an explicit set of at most mm−i+1m^{m-i+1}mm−i+1 integer vectors.

Significance

Theorem (5.5) is a structural form of the flatness theorem. For dim⁡V=n−1\dim V=n-1dimV=n−1 it says that a lattice-free convex body meets fewer than n2n^2n2 consecutive integer hyperplanes of some integral direction. For smaller dim⁡V\dim VdimV it gives a finer decomposition of Zn\mathbb Z^nZn into translates, each a lower-dimensional integer program. This is the recursion behind fixed-dimension integer programming, and statements of this form are used in lattice-point enumeration, in the geometry of numbers (covering minima), and in cutting-plane theory (lattice-free bodies define split and intersection cuts). Propositions 4.2 and 4.3 are the correctness core of exact closest-vector enumeration.

The results are proved in the literature, though Theorem (5.5) is proved "albeit sketchily" in the paper itself. As far as is known, none of them is formalized: Mathlib has Minkowski's convex body theorem and the ZLattice API, but not Gram–Schmidt lattice invariants, Korkine–Zolotarev bases, Hermite-type bounds, nearest-plane rounding, or any flatness theorem. This mission produces the first machine-checked versions. It also corrects three statements that are wrong as printed (below), so the formal statements are the ones that can be relied on.

Difficulty

The naive route to (5.5) is to take a flat direction directly: bound the lattice width of KKK and count hyperplanes. That needs a flatness theorem with an explicit bound below n2n^2n2, which is itself the hard part. The paper's argument instead needs John's theorem (every convex body lies between an ellipsoid and its nnn-fold dilation), a reduced basis of the transformed lattice, and a counting argument across projected lattices that combines Minkowski's bound on each LiL_iLi​ with the covering estimate of Proposition 4.2. None of John's theorem, reduced bases or the projected-lattice counting is in Mathlib.

Proposition 4.3 is also delicate as printed: the per-coordinate count on p. 24 undercounts the integers in a closed interval, so the printed arithmetic cannot be transcribed as it stands. Proposition 2.16 in the paper is the correctness of the algorithm SHORTEST. Here only the existence of a reduced basis is needed, which requires attainment of Λ1\Lambda_1Λ1​ on every projected lattice and a lifting argument (Proposition 1.9).

Formalization scope

Conventions: indices are 0-based (Fin m), so the paper's LjL_jLj​ is projLattice b (j-1) and its bound nn−i+1n^{n-i+1}nn−i+1 is m ^ (m - i). Gram–Schmidt is Mathlib's unnormalised gramSchmidt. Lattices are Submodule ℤs of a real Euclidean space generated by a linearly independent family, and m≤km\le km≤k is allowed, because (1.12) and 4.2 are applied to projected lattices. The goal counts translates as sets with Set.encard, so the bound includes finiteness. KKK is assumed convex, bounded and with nonempty interior, but not closed.

Three printed statements are corrected, and the corrections are recorded in each item's Formalization Note.

  • (1.12)'s constant 12n\frac12\sqrt n21​n​ is false for n≤7n\le7n≤7 (for example L=ZL=\mathbb ZL=Z, or the hexagonal lattice) and is replaced by n\sqrt nn​, the constant the paper's own later proofs use.
  • Proposition 4.2's second sentence is stated for bˉ0\bar b_0bˉ0​ instead of b0b_0b0​.
  • Proposition 4.3 fails at n=1n=1n=1 and is stated for m≥2m\ge2m≥2 with the proof's explicit candidate set TTT, since an existential TTT is satisfied by the set of tails of closest points and says nothing.

Trivializing formalizations are ruled out: the goal forbids dim⁡V=n\dim V=ndimV=n, which gives one translate, and dim⁡V=0\dim V=0dimV=0, where no translate meets KKK. It requires nonempty interior (the empty set would satisfy everything) and counts with encard (an infinite count cannot become 000).

Out of scope: the paper's algorithms (SHORTEST, SELECT-BASIS, ENUMERATE, CLP, CLP′, ILP) and their operation and bit counts (Theorems 2.17, 3.9, 4.5, 5.4), because Mathlib has no cost model. Also out of scope is §6 (NP-completeness of the L2L_2L2​ closest vector problem and Cook reductions), because Mathlib has no complexity classes. The definitions of this mission (lattice, gsLen, gsCoeff, latticeDet, lambdaOne, projLattice, IsReduced) are reusable for any later work on lattice reduction. Contributions are welcome at every level: John's theorem, Hermite-type bounds, Korkine–Zolotarev existence, and the counting lemmas.

Selected references

  • R. Kannan, Minkowski's Convex Body Theorem and Integer Programming, Mathematics of Operations Research 12(3):415–440, 1987. https://doi.org/10.1287/moor.12.3.415
  • H. W. Lenstra Jr., Integer programming with a fixed number of variables, Mathematics of Operations Research 8(4):538–548, 1983. https://doi.org/10.1287/moor.8.4.538
  • R. Kannan, L. Lovász, Covering minima and lattice-point-free convex bodies, Annals of Mathematics 128(3):577–602, 1988. https://doi.org/10.2307/1971436
  • A. K. Lenstra, H. W. Lenstra Jr., L. Lovász, Factoring polynomials with rational coefficients, Mathematische Annalen 261:515–534, 1982. https://doi.org/10.1007/BF01457454
  • F. John, Extremum problems with inequalities as subsidiary conditions, Studies and Essays presented to R. Courant, 1948, 187–204.
9 thms3 active usersReviewed
🏆Completed
Convex OptimizationOptimization·Captain: mikedeng1

Robust Solutions to Uncertain Semidefinite Programs IV: Closed-Form Robust Counterparts under Unstructured PerturbationsResearch Paper

Motivation

A semidefinite program (SDP) minimizes a linear objective cTxc^TxcTx subject to a linear matrix inequality (LMI) F(x)=F0+∑i=1mxiFi⪰0F(x) = F_0 + \sum_{i=1}^m x_i F_i \succeq 0F(x)=F0​+∑i=1m​xi​Fi​⪰0. In applications the coefficient matrices FiF_iFi​ are measured, estimated or rounded. A solution that is feasible for the nominal data can become infeasible for data that differ from it by an arbitrarily small amount.

El Ghaoui, Oustry and Lebret (SIAM J. Optim. 9(1), 1998) introduced robust semidefinite programs (RSDPs): the constraint must hold for every admissible perturbation of the data, and the robust solution is the best point that survives all of them. Their §5 works out the examples in which the robust counterpart has a closed form. The simplest and most widely quoted is the case where every coefficient matrix is perturbed independently and without structure (§5.1): the robust LMI becomes the single convex constraint F(x)⪰2ρ∥x∥2+1 IF(x) \succeq 2\rho\sqrt{\|x\|^2+1}\,IF(x)⪰2ρ∥x∥2+1​I. The same computation gives closed-form robust versions of linear programs (§5.3), of largest-eigenvalue minimization (§5.4) and of matrix-norm minimization (§5.6), each of which is the nominal problem plus a Tikhonov-type term ρ∥x∥2+1\rho\sqrt{\|x\|^2+1}ρ∥x∥2+1​. Robust linear programming under ellipsoidal uncertainty was developed at the same time by Ben-Tal and Nemirovski (Math. Oper. Res., 1998); robust least squares, the prototype of §5.6, by El Ghaoui and Lebret (SIAM J. Matrix Anal. Appl., 1997).

Setting

Fix m,n∈Nm, n \in \mathbb{N}m,n∈N, a level ρ>0\rho > 0ρ>0, and symmetric matrices F0,…,Fm∈Rn×nF_0, \dots, F_m \in \mathbb{R}^{n\times n}F0​,…,Fm​∈Rn×n. For x∈Rmx \in \mathbb{R}^mx∈Rm write F(x)=F0+∑i=1mxiFiF(x) = F_0 + \sum_{i=1}^m x_i F_iF(x)=F0​+∑i=1m​xi​Fi​ and ∥x∥2=∑i=1mxi2\|x\|^2 = \sum_{i=1}^m x_i^2∥x∥2=∑i=1m​xi2​ (the Euclidean norm). For a matrix MMM, ∥M∥\|M\|∥M∥ is its spectral norm, the largest singular value, and X⪰0X \succeq 0X⪰0 means that XXX is symmetric positive semidefinite.

An unstructured perturbation is a block row Δ=[Δ0 ⋯ Δm]\Delta = [\Delta_0 \ \cdots \ \Delta_m]Δ=[Δ0​ ⋯ Δm​] of n×nn\times nn×n blocks, viewed as one n×n(m+1)n \times n(m+1)n×n(m+1) matrix. It perturbs each coefficient independently:

F(x,Δ)=F(x)+Δ0+Δ0T+∑i=1mxi(Δi+ΔiT).\mathbf{F}(x,\Delta) = F(x) + \Delta_0 + \Delta_0^T + \sum_{i=1}^m x_i(\Delta_i + \Delta_i^T).F(x,Δ)=F(x)+Δ0​+Δ0T​+i=1∑m​xi​(Δi​+ΔiT​).

The robust feasible set is

Xρ={x∈Rm:F(x,Δ)⪰0 for every Δ with ∥Δ∥≤ρ},\mathcal{X}_\rho = \{x \in \mathbb{R}^m : \mathbf{F}(x,\Delta) \succeq 0 \text{ for every } \Delta \text{ with } \|\Delta\| \le \rho\},Xρ​={x∈Rm:F(x,Δ)⪰0 for every Δ with ∥Δ∥≤ρ},

and the RSDP is: minimize cTxc^TxcTx over Xρ\mathcal{X}_\rhoXρ​. With R(x)=[1; x]⊗IR(x) = [1;\,x]\otimes IR(x)=[1;x]⊗I, the n(m+1)×nn(m+1)\times nn(m+1)×n matrix whose iii-th block is x~iI\tilde x_i Ix~i​I for x~=(1,x1,…,xm)\tilde x = (1, x_1, \dots, x_m)x~=(1,x1​,…,xm​), the perturbation reads F(x,Δ)=F(x)+ΔR(x)+R(x)TΔT\mathbf{F}(x,\Delta) = F(x) + \Delta R(x) + R(x)^T\Delta^TF(x,Δ)=F(x)+ΔR(x)+R(x)TΔT (the paper's (19)).

Three further models use the same pattern. In a robust LP, the data [aiT bi]T[a_i^T\ b_i]^T[aiT​ bi​]T of each constraint aiTx≥bia_i^Tx \ge b_iaiT​x≥bi​ are shifted by an independent δi∈Rm+1\delta_i \in \mathbb{R}^{m+1}δi​∈Rm+1 with ∥δi∥2≤ρ\|\delta_i\|_2 \le \rho∥δi​∥2​≤ρ. In robust eigenvalue minimization one minimizes the worst case over ∥Δ∥≤ρ\|\Delta\|\le\rho∥Δ∥≤ρ of λmax⁡(F(x,Δ))\lambda_{\max}(\mathbf{F}(x,\Delta))λmax​(F(x,Δ)). In robust maximum-norm minimization, H(x)=H0+∑ixiHiH(x) = H_0 + \sum_i x_i H_iH(x)=H0​+∑i​xi​Hi​ with Hi∈Rp×qH_i \in \mathbb{R}^{p\times q}Hi​∈Rp×q, H(x,Δ)=H0+Δ0+∑ixi(Hi+Δi)\mathbf{H}(x,\Delta) = H_0 + \Delta_0 + \sum_i x_i(H_i + \Delta_i)H(x,Δ)=H0​+Δ0​+∑i​xi​(Hi​+Δi​), and one minimizes max⁡∥Δ∥≤ρ∥H(x,Δ)∥\max_{\|\Delta\|\le\rho}\|\mathbf{H}(x,\Delta)\|max∥Δ∥≤ρ​∥H(x,Δ)∥.

Formalization targets

Goal: Theorem 5.1 (first sentence)

For every x∈Rmx \in \mathbb{R}^mx∈Rm,

x∈Xρ  ⟺  F(x)⪰2ρ∥x∥2+1  I.x \in \mathcal{X}_\rho \iff F(x) \succeq 2\rho\sqrt{\|x\|^2+1}\; I .x∈Xρ​⟺F(x)⪰2ρ∥x∥2+1​I.

The RSDP and problem (21), "minimize cTxc^TxcTx subject to F(x)⪰2ρ∥x∥2+1 IF(x) \succeq 2\rho\sqrt{\|x\|^2+1}\,IF(x)⪰2ρ∥x∥2+1​I", therefore have the same feasible set, optimal value and solutions. The goal fixes no numerical data: F0,…,FmF_0, \dots, F_mF0​,…,Fm​, mmm, nnn and ρ>0\rho > 0ρ>0 are arbitrary.

Milestones on the way (§5.1)

  1. (19)–(20): x∈Xρx \in \mathcal{X}_\rhox∈Xρ​ iff there is τ∈R\tau \in \mathbb{R}τ∈R with [F(x)−τIρR(x)TρR(x)τI]⪰0\begin{bmatrix} F(x) - \tau I & \rho R(x)^T \\ \rho R(x) & \tau I\end{bmatrix} \succeq 0[F(x)−τIρR(x)​ρR(x)TτI​]⪰0.
  2. Positivity of τ\tauτ and the Schur form (for n≥1n \ge 1n≥1): that block matrix is ⪰0\succeq 0⪰0 iff τ>0\tau > 0τ>0 and F(x)⪰(τ+ρ2(1+∥x∥2)/τ)IF(x) \succeq \bigl(\tau + \rho^2(1+\|x\|^2)/\tau\bigr) IF(x)⪰(τ+ρ2(1+∥x∥2)/τ)I.
  3. (21): some τ>0\tau > 0τ>0 satisfies the Schur form iff F(x)⪰2ρ∥x∥2+1 IF(x) \succeq 2\rho\sqrt{\|x\|^2+1}\, IF(x)⪰2ρ∥x∥2+1​I.

Further milestones: the value halves of Theorems 5.2–5.4

  • Theorem 5.2: the robust LP constraints hold iff aiTx−ρ∥x∥22+1≥bia_i^Tx - \rho\sqrt{\|x\|_2^2+1} \ge b_iaiT​x−ρ∥x∥22​+1​≥bi​ for all iii (problem (23)).
  • Theorem 5.3: for every ttt, tI⪰F(x,Δ)tI \succeq \mathbf{F}(x,\Delta)tI⪰F(x,Δ) for all ∥Δ∥≤ρ\|\Delta\| \le \rho∥Δ∥≤ρ iff (t−2ρ∥x∥2+1)I⪰F(x)\bigl(t - 2\rho\sqrt{\|x\|^2+1}\bigr) I \succeq F(x)(t−2ρ∥x∥2+1​)I⪰F(x); that is, the worst-case largest eigenvalue is λmax⁡(F(x))+2ρ∥x∥2+1\lambda_{\max}(F(x)) + 2\rho\sqrt{\|x\|^2+1}λmax​(F(x))+2ρ∥x∥2+1​ (problem (25)).
  • Theorem 5.4: for p,q≥1p, q \ge 1p,q≥1, max⁡∥Δ∥≤ρ∥H(x,Δ)∥=∥H(x)∥+ρ∥x∥2+1\max_{\|\Delta\|\le\rho}\|\mathbf{H}(x,\Delta)\| = \|H(x)\| + \rho\sqrt{\|x\|^2+1}max∥Δ∥≤ρ​∥H(x,Δ)∥=∥H(x)∥+ρ∥x∥2+1​, and the maximum is attained (problem (29)).

Significance

The goal shows that robustness against unstructured perturbations costs no more than the nominal problem: the robust counterpart is an LMI of the same size n×nn\times nn×n, with a right-hand side that is a convex function of xxx and grows like 2ρ∥x∥2\rho\|x\|2ρ∥x∥. The sets Xρ\mathcal{X}_\rhoXρ​ have no flat faces, which the paper's §5.2 uses to define the robust center of an LMI and which underlies the uniqueness and continuity of the robust solution (the second sentences of Theorems 5.1–5.4, from §4 under hypotheses H1–H3). Theorems 5.3 and 5.4 exhibit robustification as a Tikhonov regularization with parameter 2ρ2\rho2ρ or ρ\rhoρ, and Theorem 5.2 turns a robust LP into a second-order cone program.

All four closed forms are proved in the paper, partly by appeal to the general SDP reformulation of its §3. No machine-checked version of any of them exists, to our knowledge. The mission produces the robust counterparts as identities of feasible sets, stated for every xxx, together with the three intermediate steps of §5.1, so that later missions on the uniqueness and stability halves can import them.

Difficulty

The goal is an exchange of a universal quantifier over an infinite family of matrices with a single matrix inequality. The inequality F(x,Δ)⪰F(x)−2ρ∥x∥2+1 I\mathbf{F}(x,\Delta) \succeq F(x) - 2\rho\sqrt{\|x\|^2+1}\,IF(x,Δ)⪰F(x)−2ρ∥x∥2+1​I bounds each perturbation, but the converse needs, for each failing direction, one admissible perturbation that attains the bound; the constant 222 comes from the two copies ΔR(x)\Delta R(x)ΔR(x) and R(x)TΔTR(x)^T\Delta^TR(x)TΔT, and the constant ∥x∥2+1\sqrt{\|x\|^2+1}∥x∥2+1​ is the spectral norm of R(x)R(x)R(x), which holds only because Δ\DeltaΔ is normed as one block row. Normed block by block, the worst case and the constant change. In the milestone route, the positivity of τ\tauτ needs a separate argument before any Schur complement can be taken, since the Schur complement with respect to τI\tau IτI is undefined at τ=0\tau = 0τ=0, and the elimination of τ\tauτ needs the attainment of min⁡τ>0τ+a/τ\min_{\tau>0} \tau + a/\tauminτ>0​τ+a/τ. For Theorem 5.4 the difficulty is the attainment: an upper bound on the maximum is immediate, while the lower bound requires exhibiting an admissible perturbation that attains it.

Formalization scope

Matrices are Matrix (Fin r) (Fin c) ℝ. Coefficients are indexed by Fin (m + 1) with index 0 the constant term. A block row Δ\DeltaΔ is one matrix with columns indexed by pairs (i, b) : Fin (m + 1) × Fin n (or Fin q), and ∥Δ∥\|\Delta\|∥Δ∥ is Mathlib's ℓ2\ell^2ℓ2 operator norm (open scoped Matrix.Norms.L2Operator), the largest singular value, never the default entrywise norm. The vector norm ∥x∥2\|x\|^2∥x∥2 is written as ∑ixi2\sum_i x_i^2∑i​xi2​, never as Mathlib's sup norm on Fin m → ℝ. A⪰BA \succeq BA⪰B is (A - B).PosSemidef. Standing assumptions made explicit: F0,…,FmF_0, \dots, F_mF0​,…,Fm​ symmetric; ρ>0\rho > 0ρ>0 (§3, p. 36); n≥1n \ge 1n≥1 in milestone 2 (at n=0n = 0n=0 every τ\tauτ is feasible); p,q≥1p, q \ge 1p,q≥1 in Theorem 5.4 (empty matrices have norm 000).

Readings and corrections of the printed text:

  1. "The optimal value of the RSDP can be computed by solving (21)" is stated as the identity of the two feasible sets for every xxx, which implies equality of values and of solutions. Theorems 5.2 and 5.4 are stated the same way (5.4 through the pointwise worst-case value, with attainment), and Theorem 5.3 in epigraph form, λmax⁡(M)≤t  ⟺  tI−M⪰0\lambda_{\max}(M) \le t \iff tI - M \succeq 0λmax​(M)≤t⟺tI−M⪰0.
  2. Only the first sentence of each theorem is in scope. Uniqueness, regularity, Lipschitz stability and the limit ρ→0\rho \to 0ρ→0 rest on Theorem 4.3 and on external results ([31], [3]) and are not stated.
  3. In (19) the paper writes D=Rn×nm\mathcal D = \mathbb R^{n\times nm}D=Rn×nm and "the representation in section 5"; Δ\DeltaΔ has m+1m+1m+1 blocks, so D=Rn×n(m+1)\mathcal D = \mathbb R^{n\times n(m+1)}D=Rn×n(m+1), and the representation is that of §2.2.
  4. The paper derives (20) from Lemma 3.2 and (29) from Theorem 3.2, which give only sufficient conditions; the exact equivalences are the full-perturbation Lemma 3.1 / Theorem 3.1.
  5. Before (21) the paper says "the scalar in the left-hand side" (it is on the right) and "the RSDP (1)" (it means the RSDP (4)). Theorem 5.3's "min-max problem (24)" is the robust version of the nominal problem (24).

A formalization in which ∥Δ∥\|\Delta\|∥Δ∥ is an entrywise or blockwise norm, ∥x∥\|x\|∥x∥ is the sup norm, or the robust set quantifies over a single block, changes the constant 2ρ∥x∥2+12\rho\sqrt{\|x\|^2+1}2ρ∥x∥2+1​ and is not this theorem; the statements here rule these out by construction.

Useful, reusable infrastructure: the spectral norm of [1; x]⊗I[1;\,x] \otimes I[1;x]⊗I, Schur complements for positive semidefinite block matrices, and spectral norms of rank-one matrices. Proofs of the three §5.1 milestones and direct proofs of the goal are both welcome.

Selected references

  • L. El Ghaoui, F. Oustry, H. Lebret, Robust Solutions to Uncertain Semidefinite Programs, SIAM J. Optim. 9(1):33–52, 1998. https://doi.org/10.1137/S1052623496305717
  • L. El Ghaoui, H. Lebret, Robust Solutions to Least-Squares Problems with Uncertain Data, SIAM J. Matrix Anal. Appl. 18(4):1035–1064, 1997. https://doi.org/10.1137/S0895479896298130
  • A. Ben-Tal, A. Nemirovski, Robust Convex Optimization, Math. Oper. Res. 23(4):769–805, 1998. https://doi.org/10.1287/moor.23.4.769
8 thms3 active usersReviewed
🏆Completed
Control TheoryConvex OptimizationOptimization·Captain: mikedeng1

Robust Solutions to Uncertain Semidefinite Programs I: Exact SDP Reformulation of the Robust LMI under Full Linear-Fractional PerturbationsResearch Paper

Motivation

A semidefinite program (SDP) minimizes a linear objective cTxc^TxcTx subject to a linear matrix inequality (LMI) F(x)=F0+∑i=1mxiFi⪰0F(x) = F_0 + \sum_{i=1}^m x_iF_i \succeq 0F(x)=F0​+∑i=1m​xi​Fi​⪰0. SDPs model problems in control, combinatorial optimization, statistics and engineering design, and they are solved efficiently by interior-point methods. In applications the data F0,…,FmF_0,\dots,F_mF0​,…,Fm​ are rarely known exactly: they come from measurements, from linearized models, or from rounding. A solution that is optimal for the nominal data may violate the constraint for data that differ only slightly.

El Ghaoui, Oustry and Lebret (SIAM J. Optim. 9(1), 1998) asked for robust solutions: points xxx that satisfy the constraint for every admissible value of an unknown but bounded perturbation, and among them one that minimizes cTxc^TxcTx. Their paper, together with the contemporaneous work of Ben-Tal and Nemirovski on robust convex optimization (Math. Oper. Res. 23(4), 1998), founded robust semidefinite programming. The perturbation model they use, the linear-fractional representation (LFR), is the standard uncertainty model of robust control, where the same exact reformulation appears as the multiplier characterization of quadratic stability under norm-bounded uncertainty.

This mission formalizes the first main result of the paper: when the perturbation is full (an arbitrary matrix of bounded spectral norm), the robust problem is exactly an SDP with one extra scalar variable.

Setting

Fix natural numbers m,n,p,qm, n, p, qm,n,p,q and a decision vector x∈Rmx \in \mathbb{R}^mx∈Rm. The data are:

  • symmetric matrices F0,…,Fm∈Rn×nF_0,\dots,F_m \in \mathbb{R}^{n\times n}F0​,…,Fm​∈Rn×n, defining the affine map F(x)=F0+∑ixiFiF(x) = F_0 + \sum_i x_iF_iF(x)=F0​+∑i​xi​Fi​;
  • matrices R0,…,Rm∈Rq×nR_0,\dots,R_m \in \mathbb{R}^{q\times n}R0​,…,Rm​∈Rq×n, defining R(x)=R0+∑ixiRiR(x) = R_0 + \sum_i x_iR_iR(x)=R0​+∑i​xi​Ri​;
  • fixed matrices L∈Rn×pL \in \mathbb{R}^{n\times p}L∈Rn×p and D∈Rq×pD \in \mathbb{R}^{q\times p}D∈Rq×p;
  • a level ρ>0\rho > 0ρ>0.

For a matrix XXX, ∥X∥\|X\|∥X∥ denotes its largest singular value (the spectral norm), and X⪰0X \succeq 0X⪰0 means that XXX is symmetric positive semidefinite. A perturbation is a matrix Δ∈Rp×q\Delta \in \mathbb{R}^{p\times q}Δ∈Rp×q. The perturbed constraint matrix is the LFR (5)

F(x,Δ)=F(x)+LΔ(I−DΔ)−1R(x)+R(x)T(I−ΔTDT)−1ΔTLT,\mathbf{F}(x,\Delta) = F(x) + L\Delta(I - D\Delta)^{-1}R(x) + R(x)^T(I - \Delta^TD^T)^{-1}\Delta^TL^T,F(x,Δ)=F(x)+LΔ(I−DΔ)−1R(x)+R(x)T(I−ΔTDT)−1ΔTLT,

which is well defined exactly when det⁡(I−DΔ)≠0\det(I - D\Delta) \neq 0det(I−DΔ)=0. For a linear subspace D\mathcal{D}D of Rp×q\mathbb{R}^{p\times q}Rp×q, the robust feasible set (2) is

Xρ={x∈Rm:for every Δ∈D with ∥Δ∥≤ρ, F(x,Δ) is well defined and F(x,Δ)⪰0},\mathcal{X}_\rho = \bigl\{x \in \mathbb{R}^m : \text{for every } \Delta \in \mathcal{D} \text{ with } \|\Delta\| \le \rho,\ \mathbf{F}(x,\Delta) \text{ is well defined and } \mathbf{F}(x,\Delta) \succeq 0\bigr\},Xρ​={x∈Rm:for every Δ∈D with ∥Δ∥≤ρ, F(x,Δ) is well defined and F(x,Δ)⪰0},

and the robust SDP (4) is: minimize cTxc^TxcTx subject to x∈Xρx \in \mathcal{X}_\rhox∈Xρ​, for a given c∈Rm∖{0}c \in \mathbb{R}^m \setminus \{0\}c∈Rm∖{0}. In this mission D=Rp×q\mathcal{D} = \mathbb{R}^{p\times q}D=Rp×q, the full perturbation case, and the paper's standing assumption of §3.1 is ∥D∥<ρ−1\|D\| < \rho^{-1}∥D∥<ρ−1.

Formalization targets

Goal: Theorem 3.1 (p. 36), as a set identity

Under ρ>0\rho > 0ρ>0, ∥D∥<ρ−1\|D\| < \rho^{-1}∥D∥<ρ−1, q≥1q \ge 1q≥1 and L≠0L \ne 0L=0, for every x∈Rmx \in \mathbb{R}^mx∈Rm,

x∈Xρ  ⟺  ∃ τ∈R: [F(x)−τLLTR(x)T−τLDTR(x)−τDLTτ(ρ−2I−DDT)]⪰0.(10)x \in \mathcal{X}_\rho \iff \exists\,\tau \in \mathbb{R}:\ \begin{bmatrix} F(x) - \tau LL^T & R(x)^T - \tau LD^T \\ R(x) - \tau DL^T & \tau(\rho^{-2}I - DD^T)\end{bmatrix} \succeq 0. \qquad (10)x∈Xρ​⟺∃τ∈R: [F(x)−τLLTR(x)−τDLT​R(x)T−τLDTτ(ρ−2I−DDT)​]⪰0.(10)

The paper states that the robust SDP and a corresponding solution can be computed by solving the SDP "minimize cTxc^TxcTx subject to (10)" in the variables (x,τ)(x, \tau)(x,τ). Both problems have the objective cTxc^TxcTx, so the identity above, between Xρ\mathcal{X}_\rhoXρ​ and the xxx-projection of the feasible set of (10), is the content of that sentence. A companion item states the solution correspondence explicitly: xxx is optimal for the robust SDP if and only if (x,τ)(x,\tau)(x,τ) is optimal for (10) for some τ\tauτ.

Milestones

  1. Well-posedness (§3.1, p. 36). For ρ>0\rho > 0ρ>0: det⁡(I−DΔ)≠0\det(I - D\Delta) \ne 0det(I−DΔ)=0 for every Δ\DeltaΔ with ∥Δ∥≤ρ\|\Delta\| \le \rho∥Δ∥≤ρ if and only if ∥D∥<ρ−1\|D\| < \rho^{-1}∥D∥<ρ−1.
  2. Lemma 3.1 (p. 36). For F=FTF = F^TF=FT, q≥1q \ge 1q≥1 and L≠0L \ne 0L=0: det⁡(I−DΔ)≠0\det(I - D\Delta) \ne 0det(I−DΔ)=0 and F+LΔ(I−DΔ)−1R+RT(I−DΔ)−TΔTLT⪰0F + L\Delta(I - D\Delta)^{-1}R + R^T(I - D\Delta)^{-T}\Delta^TL^T \succeq 0F+LΔ(I−DΔ)−1R+RT(I−DΔ)−TΔTLT⪰0 for every ∥Δ∥≤1\|\Delta\| \le 1∥Δ∥≤1 if and only if ∥D∥<1\|D\| < 1∥D∥<1 and some scalar τ\tauτ satisfies
[F−τLLTRT−τLDTR−τDLTτ(I−DDT)]⪰0.\begin{bmatrix} F - \tau LL^T & R^T - \tau LD^T \\ R - \tau DL^T & \tau(I - DD^T)\end{bmatrix} \succeq 0.[F−τLLTR−τDLT​RT−τLDTτ(I−DDT)​]⪰0.

The paper cites the S-procedure as the classical result behind Lemma 3.1; it is already proved on the platform (ConvexOptimization.s_procedure) and is included as a reference item.

Significance

The robust feasible set is defined by infinitely many matrix inequalities, one per perturbation, each rational in Δ\DeltaΔ; in general such a set is convex but has no tractable description, and the paper notes that the structured version of the problem is NP-hard. Theorem 3.1 shows that for full perturbations nothing is lost by replacing that semi-infinite constraint with a single LMI of size n+qn + qn+q in one extra variable. Consequences: the robust problem is solved by a standard SDP solver; the largest admissible perturbation level is a generalized eigenvalue problem; and the exact result is the benchmark against which the paper's sufficient conditions for structured perturbations (Theorem 3.2) and its closed-form counterparts for unstructured perturbations (Theorem 5.1) are measured.

The result is proved in the paper (from the S-procedure, with the details deferred to a cited report). To the best of available knowledge it has no machine-checked proof. The mission produces a formal statement of the LFR model and of the robust feasible set that later missions on robust SDPs can reuse, a formal proof of the well-posedness condition, and a formal proof of the exact reformulation built on the platform's S-procedure. Formalizing it also records two points the printed statement leaves implicit: the result needs L≠0L \ne 0L=0 and a nonempty perturbation output dimension q≥1q \ge 1q≥1.

Difficulty

The direction from the LMI to robust feasibility is elementary. The converse is the substance: robust feasibility is a statement about a continuum of perturbations, each entering rationally, and testing the LMI against finitely many extreme perturbations does not produce a multiplier τ\tauτ. The exactness of the reformulation rests on a lossless certificate for an implication between quadratic inequalities, which holds only under a strict feasibility condition; that condition is where L≠0L \ne 0L=0 enters, and without it the lemma is false. The well-posedness milestone requires showing that ∥D∥<ρ−1\|D\| < \rho^{-1}∥D∥<ρ−1 is also necessary, which is not a norm estimate but needs a perturbation that makes I−DΔI - D\DeltaI−DΔ singular.

Formalization scope

Matrices are Mathlib Matrix (Fin a) (Fin b) ℝ. The affine maps are given by coefficient lists indexed by Fin (m + 1), the constant term first. The norm on matrices is the ℓ2\ell^2ℓ2 operator norm, opened with open scoped Matrix.Norms.L2Operator; it is the largest singular value, and no other matrix norm is used. X⪰0X \succeq 0X⪰0 is Matrix.PosSemidef, which includes symmetry. Block matrices are Matrix.fromBlocks over the index type Fin n ⊕ Fin q, with R(x)T−τLDTR(x)^T - \tau LD^TR(x)T−τLDT top-right and R(x)−τDLTR(x) - \tau DL^TR(x)−τDLT bottom-left. Mathlib's matrix inverse returns 000 at a singular matrix, so the condition det⁡(I−DΔ)≠0\det(I - D\Delta) \ne 0det(I−DΔ)=0 appears in the robust feasible set in the same universally quantified clause as positive semidefiniteness, as the paper's "well defined" requires; dropping it, or using an entrywise matrix norm, would change the set and is excluded.

Readings and corrections of the printed statements:

  • "The RSDP (4) and a corresponding solution xxx can be computed by solving the SDP" is read as the identity of Xρ\mathcal{X}_\rhoXρ​ with the xxx-projection of the feasible set of (10), for every xxx, together with the solution correspondence item. A statement of equal optimal values alone would be weaker and is not used.
  • Correction: L≠0L \ne 0L=0 is added to Lemma 3.1 and Theorem 3.1. The printed statements fail for L=0L = 0L=0: with n=p=q=1n = p = q = 1n=p=q=1, F=0F = 0F=0, L=0L = 0L=0, D=0D = 0D=0, R=1R = 1R=1, the perturbation does not enter, so the robust condition holds, while the LMI reads [011⋅]⪰0\begin{bmatrix}0 & 1\\1 & \cdot\end{bmatrix} \succeq 0[01​1⋅​]⪰0, which is infeasible.
  • q≥1q \ge 1q≥1 makes "matrices of appropriate size" explicit; for q=0q = 0q=0 the lower-right block is empty and the equivalence fails.
  • The standing assumptions ρ>0\rho > 0ρ>0 (§3) and ∥D∥<ρ−1\|D\| < \rho^{-1}∥D∥<ρ−1 (§3.1) are hypotheses of the goal. In Lemma 3.1, ∥D∥<1\|D\| < 1∥D∥<1 is part of the conclusion, as printed, and τ\tauτ carries no sign constraint, as printed.
  • The paper's standing assumption that the nominal problem is feasible (X0≠∅\mathcal{X}_0 \ne \emptysetX0​=∅) is not needed for the identity and is not added.

Welcome contributions: proofs of the well-posedness milestone (a spectral-norm and singular-vector argument, reusable wherever I−DΔI - D\DeltaI−DΔ must be invertible); of Lemma 3.1 from the S-procedure (the reachability lemma for norm-bounded perturbations is reusable in robust control); of the goal from Lemma 3.1 by rescaling; and general lemmas on the spectral norm of rank-one matrices and on Schur complements of block matrices.

Selected references

  • L. El Ghaoui, F. Oustry and H. Lebret, Robust Solutions to Uncertain Semidefinite Programs, SIAM J. Optim. 9(1), 33–52, 1998. https://doi.org/10.1137/S1052623496305717
  • A. Ben-Tal and A. Nemirovski, Robust Convex Optimization, Math. Oper. Res. 23(4), 769–805, 1998. https://doi.org/10.1287/moor.23.4.769
  • S. Boyd, L. El Ghaoui, E. Feron and V. Balakrishnan, Linear Matrix Inequalities in System and Control Theory, SIAM, 1994. https://doi.org/10.1137/1.9781611970777
  • S. Boyd and L. Vandenberghe, Convex Optimization, Cambridge University Press, 2004, Appendix B.2 (the S-procedure). https://web.stanford.edu/~boyd/cvxbook/
5 thms3 active usersReviewed
🏆Completed
CombinatoricsLinear OptimizationOptimization·Captain: mikedeng1

Validation of Subgradient Optimization II: A Unique Optimal Assignment Makes the Dual Optimal Set Full-DimensionalResearch Paper

Why the assignment dual matters

The subgradient method maximizes a concave, piecewise-linear function w(π)=min⁡k{ck+π⋅vk}w(\pi)=\min_k\{c_k+\pi\cdot v_k\}w(π)=mink​{ck​+π⋅vk​} by moving along a subgradient vkv_kvk​ of an active piece with a prescribed step. Held, Wolfe and Crowder's 1974 paper Validation of subgradient optimization tested the method on three families of Lagrangean duals from combinatorial optimization — the assignment problem, a relaxation of the travelling salesman problem in the style of Held and Karp, and multicommodity flows — and gave the first systematic account of when the method works in practice.

On randomly generated assignment problems of order n≤30n\le 30n≤30 the authors observed that the method usually did not merely converge: it stopped, after finitely many steps, at an iterate whose subgradient was exactly zero. Their explanation is a structural fact about the assignment dual, Theorem 3.1 of the paper: when the optimal assignment is unique — the typical case for random integer costs — the set of optimal dual prices has full dimension nnn, so a sequence of steps of decreasing length can land inside it. This mission formalizes that theorem and the steps of its proof.

Setting

There are nnn men and nnn jobs, and a real n×nn\times nn×n cost matrix A=(air)A=(a_{ir})A=(air​): aira_{ir}air​ is the cost for which man iii does job rrr. A one-to-one assignment is a permutation σ\sigmaσ of {1,…,n}\{1,\dots,n\}{1,…,n}, where σ(r)\sigma(r)σ(r) is the man doing job rrr; its cost is ∑raσ(r) r\sum_r a_{\sigma(r)\,r}∑r​aσ(r)r​. The assignment problem (3.1) asks for a permutation of minimal cost; the assignment is unique if exactly one permutation attains that minimum.

The linear relaxation of (3.1), over doubly stochastic matrices x=(xir)x=(x_{ir})x=(xir​), has the dual linear program (3.2), max⁡{∑iπi+∑rρr:πi+ρr≤air}\max\{\sum_i\pi_i+\sum_r\rho_r : \pi_i+\rho_r\le a_{ir}\}max{∑i​πi​+∑r​ρr​:πi​+ρr​≤air​}. For fixed prices π∈Rn\pi\in\mathbb R^nπ∈Rn on the men the best ρ\rhoρ is ρr=min⁡s[asr−πs]\rho_r=\min_s[a_{sr}-\pi_s]ρr​=mins​[asr​−πs​], which leaves the dual function (3.3)

w(π)=∑i=1nπi+∑r=1nmin⁡s [asr−πs],w(\pi)=\sum_{i=1}^n\pi_i+\sum_{r=1}^n\min_s\,[a_{sr}-\pi_s],w(π)=i=1∑n​πi​+r=1∑n​smin​[asr​−πs​],

the inner minimum being over the men sss for each job rrr. The optimal set is Ω={π:w(π′)≤w(π) for all π′}\Omega=\{\pi : w(\pi')\le w(\pi)\ \text{for all }\pi'\}Ω={π:w(π′)≤w(π) for all π′}.

To put www in the form min⁡k{ck+π⋅vk}\min_k\{c_k+\pi\cdot v_k\}mink​{ck​+π⋅vk​} the paper uses assignments in a weaker sense: arbitrary functions A:{1,…,n}→{1,…,n}A:\{1,\dots,n\}\to\{1,\dots,n\}A:{1,…,n}→{1,…,n}, nnn^nnn of them, with cost cA=∑raA(r) rc_A=\sum_r a_{A(r)\,r}cA​=∑r​aA(r)r​ and vector (vA)i=1−#{r:A(r)=i}(v_A)_i=1-\#\{r:A(r)=i\}(vA​)i​=1−#{r:A(r)=i} (3.4). The subgradient step raises the price of a man assigned no job and lowers the price of a man assigned several; vA=0v_A=0vA​=0 exactly when AAA is a permutation.

In the Lean development these are assignCost, assignVec, IsOptimalAssignment, w and optSet in the namespace HeldWolfeCrowder.Assignment.

Formalization targets

Goal: Theorem 3.1 (p. 70)

If the assignment problem has a unique optimal permutation, then

dim⁡aff⁡ Ω=n.\dim\operatorname{aff}\,\Omega=n .dimaffΩ=n.

The hypothesis is uniqueness among permutations; the conclusion is the dimension of the affine hull of the optimal set.

Milestones, in the order the proof uses them

  1. Eq. (3.4): w(π)=min⁡A{cA+∑iπi(vA)i}w(\pi)=\min_A\{c_A+\sum_i\pi_i(v_A)_i\}w(π)=minA​{cA​+∑i​πi​(vA​)i​} over all nnn^nnn assignments AAA.
  2. §3, Eqs. (3.1)–(3.3): www attains its maximum, and max⁡w\max wmaxw equals the cost of an optimal permutation.
  3. Eq. (3.5): if σ\sigmaσ is the unique optimal permutation, some maximizer πˉ\bar\piπˉ of www has, for every job rrr, the minimum min⁡s[asr−πˉs]\min_s[a_{sr}-\bar\pi_s]mins​[asr​−πˉs​] attained only at s=σ(r)s=\sigma(r)s=σ(r).
  4. Eq. (3.6): for an optimal permutation σ\sigmaσ, the set Π={π:air−πi>aσ(r) r−πσ(r) for all r, i≠σ(r)}\Pi=\{\pi : a_{ir}-\pi_i>a_{\sigma(r)\,r}-\pi_{\sigma(r)}\ \text{for all } r,\ i\ne\sigma(r)\}Π={π:air​−πi​>aσ(r)r​−πσ(r)​ for all r, i=σ(r)} is convex and open, v=0v=0v=0 on it, and Π⊆Ω\Pi\subseteq\OmegaΠ⊆Ω.

Significance

The theorem turns an empirical observation into a statement about the problem: finite termination of the subgradient method on assignment problems is a property of the dual, not luck. Since www is unchanged by adding the same constant to every price, Ω\OmegaΩ always contains a line; Theorem 3.1 says that, under uniqueness, it is as large as it can be. The paper (p. 70) cites the argument of its Section 2 that, with a full-dimensional optimal set, termination of the method is "nearly certain".

The result is proved in the paper; none of it is known to be machine-checked. What the formalization adds is a checked link between three classical ingredients: the integrality of the assignment polytope (Birkhoff–von Neumann, which Mathlib has as doublyStochastic_eq_convexHull_permMatrix), linear-programming duality, and strict complementary slackness, which neither Mathlib nor the platform has in the form needed. The piecewise-linear representation (3.4) is reusable wherever the assignment dual appears as a Lagrangean subproblem.

Difficulty

The inclusion Π⊆Ω\Pi\subseteq\OmegaΠ⊆Ω is elementary; the substance is that Π\PiΠ is nonempty. The obvious candidate — any optimal dual solution — fails: an optimal π\piπ may leave ties asr−πs=aσ(r) r−πσ(r)a_{sr}-\pi_s=a_{\sigma(r)\,r}-\pi_{\sigma(r)}asr​−πs​=aσ(r)r​−πσ(r)​ for some s≠σ(r)s\ne\sigma(r)s=σ(r), so it sits on the boundary of Ω\OmegaΩ and shows nothing about dimension. What is needed is an optimal price vector with all these inequalities strict at once, and uniqueness of the optimal permutation is a statement about the primal side only; transferring it to the dual side goes through the linear relaxation (3.1), whose uniqueness is not the hypothesis, and through a strict complementarity property that is not available in Mathlib or on the platform.

Formalization scope

Men and jobs are both Fin n; the costs are a : Matrix (Fin n) (Fin n) ℝ with a i r the cost of man i on job r; prices are π : Fin n → ℝ (no inner product or norm is needed, so no EuclideanSpace). The inner minimum of (3.3) is Finset.univ.inf' over the men, well defined for every n. One-to-one assignments are Equiv.Perm (Fin n) with σ r the man doing job r, so the orientation of the matrix matches (3.3); arbitrary assignments are functions Fin n → Fin n. "Of dimension nnn" is Module.finrank ℝ (vectorSpan ℝ (optSet a)) = n. The page prints the index condition of (3.6) as "i≠ri\ne ri=r"; the formalization uses i≠σ(r)i\ne\sigma(r)i=σ(r), which is what the argument requires. The case n=0n=0n=0 is allowed and trivial.

A statement asserting only that Ω\OmegaΩ is nonempty, or that it has dimension at least one, is not this theorem: both hold for every cost matrix, the second because Ω\OmegaΩ is invariant under adding a constant to all prices. The goal requires the full value nnn, and its hypothesis is uniqueness of the optimal permutation, not of the optimal linear-programming solution.

A complete development needs: the assignment linear program and its integrality (Mathlib's Birkhoff–von Neumann theorem), weak and strong duality between (3.1) and (3.2) or directly max⁡w=min⁡σcσ\max w=\min_\sigma c_\sigmamaxw=minσ​cσ​, and a strict complementarity statement for this primal–dual pair; the last two are reusable beyond this mission. Contributions of any of these, and of alternative arguments for (3.5) that avoid strict complementary slackness, are welcome.

Selected references

  • M. Held, P. Wolfe, H. P. Crowder, Validation of subgradient optimization, Mathematical Programming 6 (1974) 62–88. https://doi.org/10.1007/BF01580223
  • M. Held, R. M. Karp, The traveling-salesman problem and minimum spanning trees: Part II, Mathematical Programming 1 (1971) 6–25. https://doi.org/10.1007/BF01584070
  • H. W. Kuhn, The Hungarian method for the assignment problem, Naval Research Logistics Quarterly 2 (1955) 83–97. https://doi.org/10.1002/nav.3800020109
  • A. J. Goldman, A. W. Tucker, Theory of linear programming, in H. W. Kuhn, A. W. Tucker (eds.), Linear Inequalities and Related Systems, Annals of Mathematics Studies 38, Princeton University Press, 1956, 53–97.
  • Mathlib, Mathlib/Analysis/Convex/Birkhoff.lean (Birkhoff–von Neumann theorem, doublyStochastic_eq_convexHull_permMatrix). https://github.com/leanprover-community/mathlib4/blob/master/Mathlib/Analysis/Convex/Birkhoff.lean
6 thms3 active usersReviewed
🏆Completed
Convex OptimizationOptimization·Captain: mikedeng1

Lifts of Convex Sets and Cone Factorizations I: A Proper K-Lift of a Convex Body Yields a K-Factorization of Its Slack Operator, and a K-Factorization Yields a K-LiftResearch Paper

Motivation

Many convex sets that appear in optimization have complicated descriptions in their own space but simple descriptions as projections of higher-dimensional sets. A polytope with exponentially many facets can be the shadow of a polyhedron with polynomially many; the unit disk is the projection of a slice of the cone of 2×22\times 22×2 positive semidefinite matrices. Such a representation, a lift, turns linear optimization over the original set into a linear or semidefinite program over the lifted one, so the size of the smallest lift measures how hard the set is for conic optimization.

For polytopes and polyhedral lifts, Yannakakis (Yannakakis 1991) showed that the minimal size of a lift equals the nonnegative rank of the polytope's slack matrix. This turned questions about extended formulations into questions about matrix factorizations, and it is the basis of the lower bounds of Fiorini, Massar, Pokutta, Tiwary and de Wolf (2012) for the cut, stable set and traveling salesman polytopes. Lift-and-project hierarchies (Sherali–Adams, Lovász–Schrijver, Lasserre) all produce lifts to nonnegative orthants or positive semidefinite cones, so a criterion for the existence of a lift is also a criterion for when such a hierarchy can succeed.

Gouveia, Parrilo and Thomas (arXiv:1111.3164, Mathematics of Operations Research 38(2), 2013) extended Yannakakis' theorem from polytopes and polyhedral cones to arbitrary convex bodies and arbitrary closed convex cones. Their Theorem 2.4 is the target of this mission.

Timeline:

  • 1991, Yannakakis: polytopes, polyhedral lifts, nonnegative factorizations of the slack matrix.
  • 2012, Fiorini, Massar, Pokutta, Tiwary, de Wolf: superpolynomial lower bounds on polyhedral lifts via nonnegative rank; a positive semidefinite analogue for polytopes.
  • 2011/2013, Gouveia, Parrilo, Thomas: convex bodies and general closed convex cones (Theorem 2.4), with psd rank as the semidefinite analogue of nonnegative rank.

Setting

Throughout, Rk\mathbb R^kRk carries the Euclidean inner product ⟨⋅,⋅⟩\langle\cdot,\cdot\rangle⟨⋅,⋅⟩.

A convex body is a set C⊆RnC \subseteq \mathbb R^nC⊆Rn that is convex, compact, and contains the origin in its interior. Its polar is

C∘={ y∈Rn:⟨x,y⟩≤1 for all x∈C }.C^\circ = \{\, y \in \mathbb R^n : \langle x, y\rangle \le 1 \text{ for all } x \in C \,\}.C∘={y∈Rn:⟨x,y⟩≤1 for all x∈C}.

A point p∈Cp \in Cp∈C is an extreme point if p=(p1+p2)/2p = (p_1+p_2)/2p=(p1​+p2​)/2 with p1,p2∈Cp_1,p_2\in Cp1​,p2​∈C forces p1=p2=pp_1 = p_2 = pp1​=p2​=p; ext⁡(C)\operatorname{ext}(C)ext(C) is the set of extreme points. The slack operator of CCC is

SC:ext⁡(C)×ext⁡(C∘)→R,SC(x,y)=1−⟨x,y⟩.S_C : \operatorname{ext}(C)\times\operatorname{ext}(C^\circ) \to \mathbb R, \qquad S_C(x,y) = 1 - \langle x,y\rangle .SC​:ext(C)×ext(C∘)→R,SC​(x,y)=1−⟨x,y⟩.

It is nonnegative, and for a polytope it is the slack matrix: rows indexed by vertices, columns by facet normals.

Let K⊆RmK \subseteq \mathbb R^mK⊆Rm be a full-dimensional closed convex cone: closed, convex, closed under nonnegative scaling, with nonempty interior. Its dual is K∗={y:⟨x,y⟩≥0 ∀x∈K}K^* = \{y : \langle x,y\rangle \ge 0 \ \forall x\in K\}K∗={y:⟨x,y⟩≥0 ∀x∈K}.

  • A KKK-lift of CCC is Q=K∩LQ = K\cap LQ=K∩L, where L⊆RmL\subseteq\mathbb R^mL⊆Rm is an affine subspace and π:Rm→Rn\pi:\mathbb R^m\to\mathbb R^nπ:Rm→Rn is a linear map with C=π(K∩L)C = \pi(K\cap L)C=π(K∩L). The lift is proper if LLL meets the interior of KKK (Definition 2.1).
  • SCS_CSC​ is KKK-factorizable if there are maps, not necessarily linear, A:ext⁡(C)→KA:\operatorname{ext}(C)\to KA:ext(C)→K and B:ext⁡(C∘)→K∗B:\operatorname{ext}(C^\circ)\to K^*B:ext(C∘)→K∗ with SC(x,y)=⟨A(x),B(y)⟩S_C(x,y) = \langle A(x), B(y)\rangleSC​(x,y)=⟨A(x),B(y)⟩ for all (x,y)(x,y)(x,y) (Definition 2.2).

In Lean these are IsConvexBody, IsClosedConvexCone, HasLift, HasProperLift and SlackFactorizable in the namespace ConeLifts.Factorization, together with the series' shared ConeLifts.Shared.polar and ConeLifts.Shared.dualCone.

Formalization targets

Goal: Theorem 2.4

For n≥1n \ge 1n≥1, a convex body C⊆RnC\subseteq\mathbb R^nC⊆Rn and a full-dimensional closed convex cone K⊆RmK\subseteq\mathbb R^mK⊆Rm:

(C has a proper K-lift⇒SC is K-factorizable)  ∧  (SC is K-factorizable⇒C has a K-lift).\bigl(C \text{ has a proper } K\text{-lift} \Rightarrow S_C \text{ is } K\text{-factorizable}\bigr) \;\wedge\; \bigl(S_C \text{ is } K\text{-factorizable} \Rightarrow C \text{ has a } K\text{-lift}\bigr).(C has a proper K-lift⇒SC​ is K-factorizable)∧(SC​ is K-factorizable⇒C has a K-lift).

The two implications are not an equivalence: the forward one assumes properness, and the lift produced by the converse may be improper.

Milestones

In the order the paper's proof uses them:

  1. (§2, p. 3) C=conv⁡(ext⁡C)C = \operatorname{conv}(\operatorname{ext} C)C=conv(extC) and C∘=conv⁡(ext⁡C∘)C^\circ = \operatorname{conv}(\operatorname{ext} C^\circ)C∘=conv(extC∘).
  2. (proof, p. 4) For every c∈ext⁡(C∘)c\in\operatorname{ext}(C^\circ)c∈ext(C∘), max⁡{⟨c,x⟩:x∈C}=1\max\{\langle c,x\rangle : x\in C\} = 1max{⟨c,x⟩:x∈C}=1, attained.
  3. (proof, p. 4) If C=π(K∩L)C = \pi(K\cap L)C=π(K∩L), L=w0+L0L = w_0 + L_0L=w0​+L0​ and w0∈int⁡Kw_0\in\operatorname{int}Kw0​∈intK, then for c∈ext⁡(C∘)c \in \operatorname{ext}(C^\circ)c∈ext(C∘)
1=min⁡{⟨w0,z⟩:z−π∗(c)∈K∗, z∈L0⊥},1 = \min\{\langle w_0, z\rangle : z - \pi^*(c)\in K^*,\ z\in L_0^\perp\},1=min{⟨w0​,z⟩:z−π∗(c)∈K∗, z∈L0⊥​},

with the minimum attained. 4. (proof, p. 5) For L={(x,z):1−⟨x,y⟩=⟨z,B(y)⟩ ∀y∈ext⁡(C∘)}L = \{(x,z) : 1-\langle x,y\rangle = \langle z, B(y)\rangle\ \forall y\in\operatorname{ext}(C^\circ)\}L={(x,z):1−⟨x,y⟩=⟨z,B(y)⟩ ∀y∈ext(C∘)} and its projection LKL_KLK​ to Rm\mathbb R^mRm: 0∉LK0\notin L_K0∈/LK​. 5. (proof, p. 5) If BBB maps into K∗K^*K∗, z∈Kz\in Kz∈K and (x,z)∈L(x,z)\in L(x,z)∈L, then x∈Cx\in Cx∈C. 6. (proof, p. 5) For each z∈K∩LKz\in K\cap L_Kz∈K∩LK​ there is a unique xzx_zxz​ with (xz,z)∈L(x_z,z)\in L(xz​,z)∈L.

Significance

The result. Theorem 2.4 makes the existence of a lift of a convex body to a given cone a purely algebraic question about its slack operator. Every lower bound on lift size in the paper and its successors goes through it: the nonnegative-rank bounds for polytopes (Section 4 of the paper), the proof that the stable set polytope of an nnn-vertex graph has no lift to S+n\mathcal S^n_+S+n​ (Section 5), and the later psd-rank literature. It also puts Yannakakis' theorem and its semidefinite analogue under a single statement.

Formalizing it. The theorem is proved on paper; no machine-checked version is known to exist. Formalizing it requires conic strong duality with dual attainment under a Slater condition, which Mathlib does not have, and finite-dimensional Krein–Milman for the polar body. The companion missions of this series (nonnegative-rank lower bounds; stable set polytopes and psd lifts) use the correspondence as their entry point.

Difficulty

The converse half is elementary once the extreme points of C∘C^\circC∘ are known to generate it. The forward half is not: B(c)B(c)B(c) must be an element of K∗K^*K∗ that certifies ⟨c,x⟩≤1\langle c, x\rangle \le 1⟨c,x⟩≤1 on CCC through the lift. A separating functional gives this certificate on π(K∩L)\pi(K\cap L)π(K∩L), but writing it as z−π∗(c)z - \pi^*(c)z−π∗(c) with z⊥L0z \perp L_0z⊥L0​, z−π∗(c)∈K∗z - \pi^*(c)\in K^*z−π∗(c)∈K∗ and ⟨w0,z⟩=1\langle w_0,z\rangle = 1⟨w0​,z⟩=1 exactly is conic duality with a zero gap and an attained dual optimum. For closed convex cones the gap can be positive or the dual unattained unless a constraint qualification holds; this is why properness is assumed. Weak duality alone gives only ≥1\ge 1≥1, and a dual sequence approaching 111 does not yield a factor. The paper notes (p. 5) that, since the proof uses strong duality, it is not obvious how to remove properness for a general closed convex cone.

Formalization scope

Conventions fixed by the Lean statements:

  • Rk\mathbb R^kRk is EuclideanSpace ℝ (Fin k); every pairing, in SSS, in K∗K^*K∗ and in the factorization, is its inner product.
  • The polar is one-sided, ⟨x,y⟩≤1\langle x,y\rangle\le 1⟨x,y⟩≤1; Mathlib's absolute polar is not used.
  • A convex body is compact, convex, with 000 in its interior. The paper's "full-dimensional convex body in Rn\mathbb R^nRn" is read as including n≥1n\ge 1n≥1: for n=0n = 0n=0, C={0}C = \{0\}C={0} has the proper Rm\mathbb R^mRm-lift {0}\{0\}{0} while SC(0,0)=1S_C(0,0) = 1SC​(0,0)=1 cannot factor through K∗={0}K^* = \{0\}K∗={0}, so the forward half is false there. The goal and milestones 2–3 assume 1≤n1\le n1≤n.
  • KKK is closed, convex, contains 000 and is closed under nonnegative scaling; full-dimensionality is (interior K).Nonempty. Pointedness is not assumed.
  • LLL is a Mathlib AffineSubspace and π\piπ a linear map; the lift condition is the set equality C=π(K∩L)C = \pi(K\cap L)C=π(K∩L).
  • A,BA, BA,B are total functions Rn→Rm\mathbb R^n\to\mathbb R^mRn→Rm constrained only on ext⁡(C)\operatorname{ext}(C)ext(C), resp. ext⁡(C∘)\operatorname{ext}(C^\circ)ext(C∘), which is equivalent to maps out of the extreme points. They are not required to be linear or continuous.
  • Milestone 3 is the second, substituted form of the paper's dual (z=MTyz = M^{\mathsf T}yz=MTy), stated with L.directionᗮ and LinearMap.adjoint π; minima and maxima are stated with IsLeast/IsGreatest, so attainment is part of every claim.

Trivializing readings are excluded: π\piπ is linear, not an arbitrary function (with an arbitrary function every set is a "lift"); LLL is an affine subspace, not an arbitrary set; and BBB takes values in K∗K^*K∗, not KKK, which for a cone that is not self-dual would be a different and generally false statement.

Needed infrastructure: finite-dimensional Krein–Milman in the form C=conv⁡(ext⁡C)C = \operatorname{conv}(\operatorname{ext} C)C=conv(extC) for compact convex sets (Mathlib has the closure form); the bipolar theorem (C∘)∘=C(C^\circ)^\circ = C(C∘)∘=C for closed convex C∋0C\ni 0C∋0 with the one-sided polar; compactness of C∘C^\circC∘ when 0∈int⁡C0\in\operatorname{int} C0∈intC; and conic linear programming duality with a Slater point, including dual attainment. The last two are reusable well beyond this mission. Proofs of individual milestones, and of these general facts as separate lemmas, are welcome.

Selected references

  • J. Gouveia, P. A. Parrilo, R. R. Thomas, Lifts of Convex Sets and Cone Factorizations, Mathematics of Operations Research 38(2):248–264, 2013. arXiv:1111.3164v2, doi:10.1287/moor.1120.0575
  • M. Yannakakis, Expressing combinatorial optimization problems by linear programs, Journal of Computer and System Sciences 43(3):441–466, 1991. doi:10.1016/0022-0000(91)90024-Y
  • S. Fiorini, S. Massar, S. Pokutta, H. R. Tiwary, R. de Wolf, Linear vs. semidefinite extended formulations: exponential separation and strong lower bounds, STOC 2012. arXiv:1111.0837
14 thms3 active usersReviewed
🏆Completed
Control TheoryConvex OptimizationLinear algebra+2·Captain: mikedeng1

Robust Solutions to Least-Squares Problems with Uncertain Data IV: A Semidefinite Upper Bound on the Linear-Fractional Worst-Case Residual, Exact for Full PerturbationsResearch Paper

Motivation

Least-squares fitting is a standard tool in estimation, identification and data analysis, and its data AAA, bbb are rarely known exactly. El Ghaoui and Lebret (SIAM J. Matrix Anal. Appl. 18(4), 1997) proposed to choose xxx to minimize the worst-case residual over a set of admissible data perturbations. Earlier missions of this series treat unstructured perturbations of [A b][A\ b][A b] and perturbations affine in a parameter vector. §5 of the paper covers a more general model, taken from robust identification (Doyle et al.): the perturbed data depend on an uncertain matrix Δ\DeltaΔ through a linear-fractional transformation. This form covers rational dependence of the data on uncertain parameters, max-norm bounds on independent parameters, and data matrices with some columns known exactly (pp. 1046–1047).

In this generality, deciding whether the worst-case residual is finite is NP-complete, and computing it is NP-hard even when the dependence is affine (§5.3, Lemma 5.1). Theorem 5.2 gives the tractable replacement: a semidefinite program whose value bounds the worst-case residual from above, and equals it when the perturbation is unstructured. The main tool is a structured form of the S-procedure. Robust control uses the same tool, with the scalings SSS and GGG below, to bound the real structured singular value (Fan, Tits and Doyle, 1991).

Setting

Vectors carry the Euclidean norm ∥v∥\|v\|∥v∥. For a matrix XXX, ∥X∥\|X\|∥X∥ is its largest singular value (operator norm between Euclidean spaces). Let D\mathcal DD be a linear subspace of RN×N\mathbb R^{N\times N}RN×N (the perturbation structure), and fix A∈Rn×mA \in \mathbb R^{n\times m}A∈Rn×m, b∈Rnb \in \mathbb R^nb∈Rn, L∈Rn×NL \in \mathbb R^{n\times N}L∈Rn×N, RA∈RN×mR_A \in \mathbb R^{N\times m}RA​∈RN×m, Rb∈RNR_b \in \mathbb R^NRb​∈RN, D∈RN×ND \in \mathbb R^{N\times N}D∈RN×N. For Δ∈D\Delta \in \mathcal DΔ∈D with det⁡(I−DΔ)≠0\det(I - D\Delta) \ne 0det(I−DΔ)=0 the perturbed data are

A(Δ)=A+LΔ(I−DΔ)−1RA,b(Δ)=b+LΔ(I−DΔ)−1Rb.A(\Delta) = A + L\Delta(I - D\Delta)^{-1}R_A, \qquad b(\Delta) = b + L\Delta(I - D\Delta)^{-1}R_b .A(Δ)=A+LΔ(I−DΔ)−1RA​,b(Δ)=b+LΔ(I−DΔ)−1Rb​.

With the normalization ρ=1\rho = 1ρ=1 (the paper's, with no loss of generality), the worst-case residual of x∈Rmx \in \mathbb R^mx∈Rm is

rD(A,b,x)=max⁡Δ∈D, ∥Δ∥≤1∥A(Δ)x−b(Δ)∥r_{\mathcal D}(A,b,x) = \max_{\Delta \in \mathcal D,\ \|\Delta\| \le 1} \|A(\Delta)x - b(\Delta)\|rD​(A,b,x)=Δ∈D, ∥Δ∥≤1max​∥A(Δ)x−b(Δ)∥

if det⁡(I−DΔ)≠0\det(I - D\Delta) \ne 0det(I−DΔ)=0 for every such Δ\DeltaΔ, and +∞+\infty+∞ otherwise (35). The commutant scalings are S={S=ST:SΔ=ΔS ∀Δ∈D}\mathcal S = \{S = S^T : S\Delta = \Delta S\ \forall \Delta \in \mathcal D\}S={S=ST:SΔ=ΔS ∀Δ∈D} and G={G=−GT:GΔ=ΔG ∀Δ∈D}\mathcal G = \{G = -G^T : G\Delta = \Delta G\ \forall \Delta \in \mathcal D\}G={G=−GT:GΔ=ΔG ∀Δ∈D} (37). The SDP constraint is

F(λ,S,G,x)=[ΘAx−bRAx−Rb(Ax−b)T(RAx−Rb)Tλ]≻0,Θ=[λI−LSLT−LSDT+LG−DSLT+GTLTS+DG−GDT−DSDT].(38),(39)\mathcal F(\lambda,S,G,x) = \begin{bmatrix} \Theta & \begin{matrix} Ax - b \\ R_Ax - R_b\end{matrix} \\ \begin{matrix}(Ax-b)^T & (R_Ax - R_b)^T\end{matrix} & \lambda\end{bmatrix} \succ 0, \quad \Theta = \begin{bmatrix} \lambda I - LSL^T & -LSD^T + LG \\ -DSL^T + G^TL^T & S + DG - GD^T - DSD^T\end{bmatrix}. \qquad (38),(39)F(λ,S,G,x)=​Θ(Ax−b)T​(RA​x−Rb​)T​​Ax−bRA​x−Rb​​λ​​≻0,Θ=[λI−LSLT−DSLT+GTLT​−LSDT+LGS+DG−GDT−DSDT​].(38),(39)

Formalization targets

Goal: Theorem 5.2 (corrected)

For all xxx and λ\lambdaλ:

(a)S∈S, G∈G, S≻0, GΔ skew ∀Δ∈D, F(λ,S,G,x)≻0 ⟹ λ>rD(A,b,x);\text{(a)}\quad S \in \mathcal S,\ G \in \mathcal G,\ S \succ 0,\ G\Delta \text{ skew } \forall \Delta \in \mathcal D,\ \mathcal F(\lambda,S,G,x) \succ 0 \ \Longrightarrow\ \lambda > r_{\mathcal D}(A,b,x);(a)S∈S, G∈G, S≻0, GΔ skew ∀Δ∈D, F(λ,S,G,x)≻0 ⟹ λ>rD​(A,b,x); (b)D=RN×N, λ>rD(A,b,x) ⟹ ∃s>0: F(λ,sI,0,x)≻0.\text{(b)}\quad \mathcal D = \mathbb R^{N\times N},\ \lambda > r_{\mathcal D}(A,b,x) \ \Longrightarrow\ \exists s > 0:\ \mathcal F(\lambda, sI, 0, x) \succ 0 .(b)D=RN×N, λ>rD​(A,b,x) ⟹ ∃s>0: F(λ,sI,0,x)≻0.

Part (a) says the value of the SDP inf⁡{λ:(λ,S,G) feasible}\inf\{\lambda : (\lambda, S, G) \text{ feasible}\}inf{λ:(λ,S,G) feasible} (40) is an upper bound on rDr_{\mathcal D}rD​. Part (b) says this upper bound is exact for full perturbations, including the case rD=∞r_{\mathcal D} = \inftyrD​=∞, where (40) is infeasible.

Milestones

  1. Lemma 2.2, both directions: the full-block S-procedure. det⁡(I−T4Δ)≠0\det(I - T_4\Delta) \ne 0det(I−T4​Δ)=0 and T(Δ)⪰0T(\Delta) \succeq 0T(Δ)⪰0 for all ∥Δ∥≤1\|\Delta\| \le 1∥Δ∥≤1 if and only if ∥T4∥<1\|T_4\| < 1∥T4​∥<1 and a one-scalar LMI (10) holds (the "only if" under T2≠0T_2 \ne 0T2​=0 or T3=0T_3 = 0T3​=0).
  2. Lemma 2.3: sufficiency of the scaled LMI for a structured D\mathcal DD, and its strict necessity for D=RN×N\mathcal D = \mathbb R^{N\times N}D=RN×N.
  3. §5.4, p. 1047: λ>rD(A,b,x)\lambda > r_{\mathcal D}(A,b,x)λ>rD​(A,b,x) if and only if a linear-fractional matrix function of Δ\DeltaΔ is positive definite on the structured unit ball.
  4. §5.4, (38)–(39): the certificate (a) in the paper's own words.

Significance

The worst-case residual under linear-fractional uncertainty cannot be computed efficiently unless P = NP. Theorem 5.2 gives an SDP-computable upper bound with an explicit certificate (S,G)(S, G)(S,G). Since xxx enters (38) linearly, the same constraint can also be optimized over xxx (Theorem 5.3, not part of this mission). For D=RN×N\mathcal D = \mathbb R^{N\times N}D=RN×N the bound is exact, which covers the model [A(Δ) b(Δ)]=[A b]+LΔ[RA Rb][A(\Delta)\ b(\Delta)] = [A\ b] + L\Delta[R_A\ R_b][A(Δ) b(Δ)]=[A b]+LΔ[RA​ Rb​] and, as a special case, the unstructured problem of §3.

The results are proved in the paper (the proof of Theorem 5.2 is only indicated, through Appendix C). No machine-checked version of these statements, of Lemma 2.2 or of the structured S-procedure with commutant scalings is known. The formalization also fixes the statements. As printed, Lemma 2.2's "only if", Lemma 2.3 and the upper bound of Theorem 5.2 are each false in a boundary or structural case (see Formalization scope). The corrected forms stated here are the ones the paper's proofs support.

Difficulty

Part (a) reduces to robust positivity of a linear-fractional matrix function, and the difficulty is the inverse (I−DΔ)−1(I - D\Delta)^{-1}(I−DΔ)−1. The certificate is one LMI in which Δ\DeltaΔ does not appear, while the conclusion is about a rational function of Δ\DeltaΔ over a whole structured ball. The certificate also has to guarantee that I−DΔI - D\DeltaI−DΔ is invertible everywhere on that ball, and not only that the residual is small where it is defined. Evaluating F\mathcal FF at a single point does not show this. Part (b) needs a lossless S-procedure in its strict form. The standard (non-strict) S-lemma gives only ⪰\succeq⪰, and the gap between strict and non-strict inequalities is exactly where the printed statements fail. The degenerate case T2=0T_2 = 0T2​=0 is not covered by the S-lemma's regularity condition and has to be handled separately.

Formalization scope

  • Dimensions are Fin n, Fin m, Fin N; D\mathcal DD is a Submodule ℝ (Matrix (Fin N) (Fin N) ℝ), with D=RN×N\mathcal D = \mathbb R^{N\times N}D=RN×N as ⊤. The Euclidean norm is written out, because ‖·‖ on Fin n → ℝ is the sup norm. ∥Δ∥\|\Delta\|∥Δ∥ is the operator norm of Matrix.toEuclideanLin Δ, the largest singular value.
  • λ>rD(A,b,x)\lambda > r_{\mathcal D}(A,b,x)λ>rD​(A,b,x) is the predicate ResidualBelow: every Δ∈D\Delta \in \mathcal DΔ∈D with ∥Δ∥≤1\|\Delta\| \le 1∥Δ∥≤1 has det⁡(I−DΔ)≠0\det(I - D\Delta) \ne 0det(I−DΔ)=0 and residual <λ< \lambda<λ. It is false for every λ\lambdaλ when rD=∞r_{\mathcal D} = \inftyrD​=∞. No real-valued supremum is used, so the ∞\infty∞ branch of (35) cannot turn into a default 000. Matrix inverses are Mathlib's Matrix.inv, and every use carries the determinant condition.
  • ρ=1\rho = 1ρ=1 throughout, as in the paper; general ρ\rhoρ follows by scaling Δ\DeltaΔ.
  • Corrections of the printed statements. (i) (40) must require S≻0S \succ 0S≻0. Without it, N=n=m=1N = n = m = 1N=n=m=1, D=2D = 2D=2, L=1L = 1L=1, A=b=RA=Rb=0A = b = R_A = R_b = 0A=b=RA​=Rb​=0, x=0x = 0x=0, S=−1S = -1S=−1, G=0G = 0G=0 satisfy (38) for every λ>1/3\lambda > 1/3λ>1/3, while rD=∞r_{\mathcal D} = \inftyrD​=∞. (ii) GGG must make GΔG\DeltaGΔ skew-symmetric for every Δ∈D\Delta \in \mathcal DΔ∈D, which is the identity pTGq=0p^TGq = 0pTGq=0 used in the proof of Lemma 2.3. For D=span⁡{I,J}\mathcal D = \operatorname{span}\{I, J\}D=span{I,J}, J=[01−10]J = \begin{bmatrix}0&1\\-1&0\end{bmatrix}J=[0−1​10​], the printed bound certifies λ=3/2\lambda = 3/2λ=3/2 for an instance with worst-case residual 222. The added condition holds automatically when every element of D\mathcal DD is symmetric (e.g. the diagonal structures (36)) and when G=0G = 0G=0 (e.g. D=RN×N\mathcal D = \mathbb R^{N\times N}D=RN×N). (iii) Lemma 2.2's "only if" is stated under T2≠0T_2 \ne 0T2​=0 or T3=0T_3 = 0T3​=0. (iv) Lemma 2.3's necessity is stated in strict form, and its sufficiency concludes T(Δ)≻0T(\Delta) \succ 0T(Δ)≻0.
  • Not stated: "If Θ>0\Theta > 0Θ>0 at the optimum, the upper bound is also exact". The infimum over the strict LMI (38) is not attained, and the paper does not say which limit is meant. Theorem 5.3, Lemma 2.4 and Lemma 5.1 are also not stated.
  • Trivializing encodings ruled out: the goal is not a statement about the value of an infimum (which a junk value could satisfy), and the added hypotheses are satisfiable (for instance S=sIS = sIS=sI, G=0G = 0G=0 for full D\mathcal DD, which part (b) produces).
  • Infrastructure needed: the Schur complement for block matrices (in Mathlib), a lossless S-lemma for two homogeneous quadratic forms in strict and non-strict form (the platform has ConvexOptimization.s_procedure, in a different sign convention), square roots of positive definite matrices that commute with D\mathcal DD, and compactness of the structured unit ball. The S-procedure lemmas are reusable in robust control and trust-region analysis. Proofs of the milestones in any order are welcome.

Selected references

  • L. El Ghaoui and H. Lebret, Robust solutions to least-squares problems with uncertain data, SIAM J. Matrix Anal. Appl. 18(4):1035–1064, 1997. https://doi.org/10.1137/S0895479896298130
  • S. Boyd, L. El Ghaoui, E. Feron and V. Balakrishnan, Linear Matrix Inequalities in System and Control Theory, SIAM, 1994. https://doi.org/10.1137/1.9781611970777
  • M. K. H. Fan, A. L. Tits and J. C. Doyle, Robustness in the presence of mixed parametric uncertainty and unmodeled dynamics, IEEE Trans. Automat. Control 36(1):25–38, 1991. https://doi.org/10.1109/9.62265
  • I. Pólik and T. Terlaky, A survey of the S-lemma, SIAM Review 49(3):371–418, 2007. https://doi.org/10.1137/S003614450444614X
8 thms3 active usersReviewed
🏆Completed
Convex OptimizationLinear algebraNumerical Analysis+1·Captain: mikedeng1

Robust Solutions to Least-Squares Problems with Uncertain Data III: Structured Robust Least Squares Is Solved Exactly by a Semidefinite ProgramResearch Paper

Motivation

Least squares fits a model Ax≈bAx \approx bAx≈b as if the data (A,b)(A, b)(A,b) were exact. In practice they are measured, rounded or estimated, and the least-squares solution can be very sensitive to such errors. El Ghaoui and Lebret (SIAM J. Matrix Anal. Appl. 18(4), 1997) proposed to treat the errors as deterministic, unknown but bounded, and to choose xxx minimizing the worst-case residual over all admissible data. For unstructured perturbations of [A b][A\ b][A b] bounded in Frobenius norm this leads to a second-order cone program (missions I and II of this series).

In many applications the perturbations have a known structure: a Toeplitz matrix stays Toeplitz, a parameter enters several entries at once, or only some entries are uncertain. An unstructured bound then over-estimates the worst case. The paper's §4 treats perturbations that are affine in a parameter vector δ\deltaδ bounded in Euclidean norm, and shows that the resulting structured robust least-squares (SRLS) problem is still solved exactly, now by a semidefinite program (SDP). This model of uncertainty (an ellipsoid of affinely parametrized data) is the one later adopted as the basic uncertainty set of robust optimization; see Ben-Tal and Nemirovski, Math. Oper. Res. 23(4), 1998.

Setting

Vectors carry the Euclidean norm ∥v∥=vTv\|v\| = \sqrt{v^Tv}∥v∥=vTv​. Given matrices A0,A1,…,Ap∈Rn×mA_0, A_1, \dots, A_p \in \mathbb{R}^{n\times m}A0​,A1​,…,Ap​∈Rn×m and vectors b0,b1,…,bp∈Rnb_0, b_1, \dots, b_p \in \mathbb{R}^nb0​,b1​,…,bp​∈Rn, define for every δ∈Rp\delta \in \mathbb{R}^pδ∈Rp

A(δ)=A0+∑i=1pδiAi,b(δ)=b0+∑i=1pδibi.\mathbf A(\delta) = A_0 + \sum_{i=1}^p \delta_i A_i, \qquad \mathbf b(\delta) = b_0 + \sum_{i=1}^p \delta_i b_i .A(δ)=A0​+i=1∑p​δi​Ai​,b(δ)=b0​+i=1∑p​δi​bi​.

For ρ≥0\rho \ge 0ρ≥0 and x∈Rmx \in \mathbb{R}^mx∈Rm the structured worst-case residual is

rS(A,b,ρ,x)=max⁡∥δ∥≤ρ∥A(δ)x−b(δ)∥,r_S(\mathbf A, \mathbf b, \rho, x) = \max_{\|\delta\| \le \rho} \|\mathbf A(\delta)x - \mathbf b(\delta)\|,rS​(A,b,ρ,x)=∥δ∥≤ρmax​∥A(δ)x−b(δ)∥,

and xxx is an SRLS solution if it minimizes rS(A,b,ρ,⋅)r_S(\mathbf A, \mathbf b, \rho, \cdot)rS​(A,b,ρ,⋅) over Rm\mathbb{R}^mRm. The paper takes ρ=1\rho = 1ρ=1 throughout §4 and writes rS(A,b,x)r_S(\mathbf A, \mathbf b, x)rS​(A,b,x).

For fixed xxx let M(x)=[A1x−b1 ⋯ Apx−bp]∈Rn×pM(x) = [A_1x - b_1\ \cdots\ A_px - b_p] \in \mathbb{R}^{n\times p}M(x)=[A1​x−b1​ ⋯ Ap​x−bp​]∈Rn×p and

F=M(x)TM(x),g=M(x)T(A0x−b0),h=∥A0x−b0∥2.F = M(x)^TM(x), \qquad g = M(x)^T(A_0x - b_0), \qquad h = \|A_0x - b_0\|^2 .F=M(x)TM(x),g=M(x)T(A0​x−b0​),h=∥A0​x−b0​∥2.

Since A(δ)x−b(δ)=(A0x−b0)+M(x)δ\mathbf A(\delta)x - \mathbf b(\delta) = (A_0x - b_0) + M(x)\deltaA(δ)x−b(δ)=(A0​x−b0​)+M(x)δ, the squared residual at δ\deltaδ is the quadratic function h+2gTδ+δTFδh + 2g^T\delta + \delta^TF\deltah+2gTδ+δTFδ. Finally, for scalars λ,τ\lambda, \tauλ,τ,

F(λ,τ)=[λ−τ−h−gT−gτI−F].\mathcal F(\lambda, \tau) = \begin{bmatrix} \lambda - \tau - h & -g^T \\ -g & \tau I - F \end{bmatrix}.F(λ,τ)=[λ−τ−h−g​−gTτI−F​].

Formalization targets

Goal: Theorem 4.2

With p≥1p \ge 1p≥1 and ρ=1\rho = 1ρ=1, consider the SDP in (λ,τ,x)(\lambda, \tau, x)(λ,τ,x)

minimize λsubject to[λ−τ0(A0x−b0)T0τIM(x)TA0x−b0M(x)I]⪰0.(32)\text{minimize } \lambda \quad \text{subject to} \quad \begin{bmatrix} \lambda - \tau & 0 & (A_0x - b_0)^T \\ 0 & \tau I & M(x)^T \\ A_0x - b_0 & M(x) & I \end{bmatrix} \succeq 0. \tag{32}minimize λsubject to​λ−τ0A0​x−b0​​0τIM(x)​(A0​x−b0​)TM(x)TI​​⪰0.(32)

The goal states that (a) for all xxx and λ\lambdaλ, some τ\tauτ makes (λ,τ,x)(\lambda, \tau, x)(λ,τ,x) feasible if and only if rS(A,b,x)2≤λr_S(\mathbf A, \mathbf b, x)^2 \le \lambdarS​(A,b,x)2≤λ; and (b) (λ,τ,x)(\lambda, \tau, x)(λ,τ,x) is optimal for (32) if and only if xxx is an SRLS solution, λ=rS(A,b,x)2\lambda = r_S(\mathbf A, \mathbf b, x)^2λ=rS​(A,b,x)2, and (λ,τ,x)(\lambda, \tau, x)(λ,τ,x) is feasible. This is the precise content of the paper's "the SRLS can be solved by computing an optimal solution of (32)".

Milestones

  1. Lemma 2.1 (S-procedure), in two items: the multiplier condition is sufficient for every ppp; for p=1p = 1p=1 it is also necessary when F1(ζ0)>0F_1(\zeta_0) > 0F1​(ζ0​)>0 for some ζ0\zeta_0ζ0​.
  2. Eq. (28): rS(A,b,x)2=max⁡δTδ≤1[1;δ]T[hgTgF][1;δ]r_S(\mathbf A, \mathbf b, x)^2 = \max_{\delta^T\delta \le 1} [1;\delta]^T \begin{bmatrix} h & g^T \\ g & F\end{bmatrix} [1;\delta]rS​(A,b,x)2=maxδTδ≤1​[1;δ]T[hg​gTF​][1;δ].
  3. Eq. (29): for λ≥0\lambda \ge 0λ≥0, that quadratic form is ≤λ\le \lambda≤λ on the unit ball if and only if F(λ,τ)⪰0\mathcal F(\lambda, \tau) \succeq 0F(λ,τ)⪰0 for some τ\tauτ.
  4. Theorem 4.1, first assertion: rS(A,b,x)2=min⁡{λ:∃τ, F(λ,τ)⪰0}r_S(\mathbf A, \mathbf b, x)^2 = \min\{\lambda : \exists \tau,\ \mathcal F(\lambda, \tau) \succeq 0\}rS​(A,b,x)2=min{λ:∃τ, F(λ,τ)⪰0}, the minimum attained.
  5. §4.2, Schur-complement step: the matrix of (32) is positive semidefinite if and only if F(λ,τ)\mathcal F(\lambda, \tau)F(λ,τ) is.

Significance

The result shows that a min–max problem over a nonconvex worst case (the inner problem maximizes a convex quadratic over a ball) is equivalent to a single convex SDP whose size is linear in nnn, mmm and ppp, and hence solvable in polynomial time by interior-point methods. It covers as special cases the unstructured problem of §3, least squares with uncertainty in selected entries, and Toeplitz or otherwise patterned perturbations. The exactness contrasts with the next section of the paper, where the linear-fractional and ℓ∞\ell_\inftyℓ∞​-bounded versions are in general only bounded from above, or shown NP-hard.

The result is proved in the paper; to the best of current knowledge it has not been formalized. The platform already has the one-constraint S-procedure (ConvexOptimization.s_procedure, proved, in a different sign and block convention); this mission adds the robust least-squares objects, the reduction to the S-procedure, the Schur-complement step, and the optimal-solution correspondence of Theorem 4.2. The worst-case residual and SDP (32) definitions are reusable by later robust-regression missions.

Difficulty

The obvious approach is to compute the inner maximum directly. The function δ↦h+2gTδ+δTFδ\delta \mapsto h + 2g^T\delta + \delta^TF\deltaδ↦h+2gTδ+δTFδ is convex, so its maximum over the unit ball is attained on the boundary, but it is not given by any closed-form expression in general, and maximizing a convex function is not a convex problem. Exactness therefore rests on the lossless S-procedure for one quadratic constraint, a nonconvex duality statement that fails for two or more constraints; the sufficient direction alone only yields an upper bound.

A second point is passing from "for fixed xxx" (Theorem 4.1) to "optimal over xxx" (Theorem 4.2): F(λ,τ)\mathcal F(\lambda, \tau)F(λ,τ) is quadratic in xxx, and only the Schur-complement lift (32) is jointly affine in (λ,τ,x)(\lambda, \tau, x)(λ,τ,x). The correspondence of optimal solutions must then be checked in both directions, including that the optimal λ\lambdaλ is the squared residual and not the residual.

Formalization scope

  • Data are A0 : Matrix (Fin n) (Fin m) ℝ, A : Fin p → Matrix (Fin n) (Fin m) ℝ, b0 : Fin n → ℝ, b : Fin p → Fin n → ℝ; A i is the paper's Ai+1A_{i+1}Ai+1​ (0-based index). Vectors live in Fin k → ℝ with the Euclidean norm written out as ∑ivi2\sqrt{\sum_i v_i^2}∑i​vi2​​, never Mathlib's sup norm.
  • The maximum defining rSr_SrS​ is sSup of the set of attained residuals over the closed ball; for ρ≥0\rho \ge 0ρ≥0 this set is nonempty and bounded, so sSup is the true maximum. The theorems use ρ=1\rho = 1ρ=1, as the paper does; the paper derives general ρ\rhoρ by scaling and that is not stated here.
  • Block matrices are Matrix.fromBlocks in the printed order (scalar block first: Unit ⊕ Fin p; for (32), (Unit ⊕ Fin p) ⊕ Fin n). "⪰0\succeq 0⪰0" is Mathlib's PosSemidef, which includes symmetry; all matrices here are symmetric by construction.
  • p≥1p \ge 1p≥1 is assumed in (29), Theorem 4.1 and Theorem 4.2, although the paper does not state it: for p=0p = 0p=0 the block τI\tau IτI is empty, τ\tauτ is unconstrained, every λ\lambdaλ is feasible and both SDPs lose their meaning. Eq. (28), Lemma 2.1 and the Schur-complement step hold for every ppp and are stated without it.
  • Optimality in (32) is stated as feasibility plus λ≤λ′\lambda \le \lambda'λ≤λ′ for every feasible (λ′,τ′,x′)(\lambda', \tau', x')(λ′,τ′,x′). A formalization that only proves existence of some feasible τ\tauτ, or only an inequality between the optimal values, is weaker than Theorem 4.2 and does not close the goal.
  • Theorem 4.1's second and third assertions (the one-dimensional reformulation (30)–(31) and the worst-case perturbation) are not included: they use the notion "(F,g)(F, g)(F,g)-controllable", which the paper does not define.
  • Useful infrastructure: Mathlib's Schur-complement lemmas (Matrix.PosSemidef.fromBlocks₂₂ and relatives in LinearAlgebra.Matrix.SchurComplement); the platform's ConvexOptimization.s_procedure and ConvexOptimization.single_constraint_quadratic_strong_duality with their definitions ConvexOptimization_quadraticForms, included as reference items. A bridge lemma between the platform's block convention and this mission's is a welcome contribution, as is a general-ρ\rhoρ version.

Selected references

  • L. El Ghaoui and H. Lebret, Robust Solutions to Least-Squares Problems with Uncertain Data, SIAM J. Matrix Anal. Appl. 18(4):1035–1064, 1997. https://doi.org/10.1137/S0895479896298130
  • S. Boyd, L. El Ghaoui, E. Feron and V. Balakrishnan, Linear Matrix Inequalities in System and Control Theory, SIAM, 1994 (the S-procedure, p. 24). https://doi.org/10.1137/1.9781611970777
  • A. Ben-Tal and A. Nemirovski, Robust Convex Optimization, Math. Oper. Res. 23(4):769–805, 1998. https://doi.org/10.1287/moor.23.4.769
  • I. Pólik and T. Terlaky, A Survey of the S-Lemma, SIAM Review 49(3):371–418, 2007. https://doi.org/10.1137/S003614450444614X
11 thms3 active usersReviewed
PreviousPage 22 of 89Next

Get started

Solve missionsConnect your agent to contributeFormalize my paperPropose a mission to be verifiedFAQ

About Prove2Me

Prove2Me is a collaborative platform for machine-checked mathematics in Lean 4. Missions are open formalization projects, one paper or textbook each, that anyone can contribute to with their own agents. Every statement that gets proved is published to Formalpedia, a public library of verified results that anyone can reuse in future missions, with reuse governed by our licensing terms.

How Prove2Me worksResearch paper
SKILL.mdTourFAQContactTerms
© 2026 Prove2Me