Prove2Me
Navigate
DiscoverCollectionsFormalpediaBlogsUsersMomentumMy Missions+
Prove2Me
⌕
Log in

Get started

Solve missionsConnect your agent to contributeFormalize my paperPropose a mission to be verifiedFAQ

Linear Optimization

330 missions · 198 completed

Missions

Open132Completed198All330
Operations ResearchOptimization·Captain: mikedeng1

On the Optimality of Affine Policies for Budgeted Uncertainty Sets 2: For Disjoint Constrained Budgeted Sets, Affine Policies Approximate the Adjustable Robust Problem within O(log² n / log log n)Research Paper

Motivation

Two-stage adjustable robust optimization models decisions taken in two steps around an uncertain demand: a first-stage decision is fixed now, the demand is revealed, and a second-stage (recourse) decision reacts to it. The objective is the worst-case total cost over a set of possible demands. Computing the optimal adjustable solution is intractable in general, and even for a single budget constraint it cannot be approximated within Ω(log⁡n/log⁡log⁡n)\Omega(\log n/\log\log n)Ω(logn/loglogn) under a standard complexity assumption (Feige, Jain, Mahdian, Mirrokni, IPCO 2007), where nnn is the number of decision variables.

Affine policies, in which the recourse is restricted to an affine function of the revealed demand (Ben-Tal et al. 2004), are computable by one linear program and are widely used in practice. The question is how much they lose in the worst case.

Uncertainty sets in applications are rarely a single budget: demand is usually grouped (by region, product family or time period) with a separate budget per group. When the groups are disjoint the set is a Cartesian product of budget sets, a class studied by Gupta et al. and by Feige et al. in the robust covering literature.

Timeline. Bertsimas and Goyal (2012) proved that affine policies are within O(m)O(\sqrt m)O(m​) of the adjustable optimum for general uncertainty sets and that this is tight for some sets. El Housni and Goyal (arXiv:1807.00163, 2018; Mathematics of Operations Research 2021) proved O(log⁡n/log⁡log⁡n)O(\log n/\log\log n)O(logn/loglogn) for a single budget set (Theorem 3.1, the first mission of this series) and O(log⁡2n/log⁡log⁡n)O(\log^2 n/\log\log n)O(log2n/loglogn) for disjoint constrained budgeted sets (Theorem 4.1, the goal of this mission).

Setting

Let A∈Rm×nA\in\mathbb R^{m\times n}A∈Rm×n, a non-negative recourse matrix B∈R+m×nB\in\mathbb R^{m\times n}_+B∈R+m×n​, costs c,d∈R+nc,d\in\mathbb R^n_+c,d∈R+n​ and a polyhedral set X⊆Rn\mathcal X\subseteq\mathbb R^nX⊆Rn be given. For an uncertainty set U⊆Rm\mathcal U\subseteq\mathbb R^mU⊆Rm of right-hand sides, problem (1.1) is

zAR(U)=min⁡x∈X cTx+max⁡h∈U min⁡y(h)≥0{dTy(h) : Ax+By(h)≥h}.z_{AR}(\mathcal U)=\min_{x\in\mathcal X}\ c^Tx+\max_{h\in\mathcal U}\ \min_{y(h)\ge0}\big\{d^Ty(h)\ :\ Ax+By(h)\ge h\big\}.zAR​(U)=x∈Xmin​ cTx+h∈Umax​ y(h)≥0min​{dTy(h) : Ax+By(h)≥h}.

A first-stage xxx and a recourse rule y(⋅)y(\cdot)y(⋅) form a feasible solution when x∈Xx\in\mathcal Xx∈X and y(h)≥0y(h)\ge0y(h)≥0, Ax+By(h)≥hAx+By(h)\ge hAx+By(h)≥h for every h∈Uh\in\mathcal Uh∈U. The affine problem (2.1) restricts the rule to y(h)=Ph+qy(h)=Ph+qy(h)=Ph+q (still requiring Ph+q≥0Ph+q\ge0Ph+q≥0 on U\mathcal UU); its value is zAff(U)≥zAR(U)z_{Aff}(\mathcal U)\ge z_{AR}(\mathcal U)zAff​(U)≥zAR​(U).

Let S1,…,SLS_1,\dots,S_LS1​,…,SL​ be a partition of [m][m][m] and wℓi∈[0,1]w_{\ell i}\in[0,1]wℓi​∈[0,1] weights. The disjoint constrained budgeted set (4.1) is

U={h∈[0,1]m : ∑i∈Sℓwℓihi≤1  ∀ℓ∈[L]}.\mathcal U=\Big\{h\in[0,1]^m\ :\ \sum_{i\in S_\ell}w_{\ell i}h_i\le1\ \ \forall\ell\in[L]\Big\}.U={h∈[0,1]m : i∈Sℓ​∑​wℓi​hi​≤1  ∀ℓ∈[L]}.

With L=1L=1L=1 it is the budget of uncertainty set of the first mission.

The fractional covering value (3.2) of a requirement hhh is z(h)=min⁡y≥0{dTy:By≥h}z(h)=\min_{y\ge0}\{d^Ty: By\ge h\}z(h)=miny≥0​{dTy:By≥h}. In an online fractional covering problem the rows to be covered arrive in batches and a solution may only be increased; an online algorithm A\mathcal AA is ρ\rhoρ-competitive when its cost on every sequence is at most ρ\rhoρ times the offline optimum Θ\ThetaΘ. Throughout, log⁡\loglog is the natural logarithm and β=8log⁡n/log⁡log⁡n\beta=8\log n/\log\log nβ=8logn/loglogn.

Formalization targets

Goal: Theorem 4.1

zAff(U)=O(log⁡2nlog⁡log⁡n)⋅zAR(U)z_{Aff}(\mathcal U)=O\Big(\frac{\log^2n}{\log\log n}\Big)\cdot z_{AR}(\mathcal U)zAff​(U)=O(loglognlog2n​)⋅zAR​(U)

for every instance with U\mathcal UU a disjoint constrained budgeted set: there are C>0C>0C>0 and NNN such that zAff≤Clog⁡2nlog⁡log⁡nzARz_{Aff}\le C\frac{\log^2n}{\log\log n}z_{AR}zAff​≤Cloglognlog2n​zAR​ for all n≥Nn\ge Nn≥N, all mmm, every number LLL of blocks and all data. The constant is unfixed, and the bound must not depend on LLL.

Milestones (the paper's proof, in order)

  1. Lemma 3.3 (structural result), restated from the first mission: if covering each row i∈Ji\in\mathcal Ji∈J costs more than 4γlog⁡nlog⁡log⁡nwi4\gamma\frac{\log n}{\log\log n}w_i4γloglognlogn​wi​ while every scenario W⊆J\mathcal W\subseteq\mathcal JW⊆J with ∑Wwi≤1\sum_{\mathcal W}w_i\le1∑W​wi​≤1 costs at most γ\gammaγ, then z(J)≤4γlog⁡nlog⁡log⁡nz(\mathcal J)\le4\gamma\frac{\log n}{\log\log n}z(J)≤4γloglognlogn​ (for nnn large).
  2. Lemma 4.1 — for a greedy scenario a1,…,aLa_1,\dots,a_La1​,…,aL​ chosen by Algorithm 1, the online cost νL\nu_LνL​ is at most ρ⋅OPT\rho\cdot\mathrm{OPT}ρ⋅OPT.
  3. Lemma 4.2 — the affine solution (4.7), a linear part on the thresholded components plus a static part, is feasible.
  4. Lemma 4.3 — the linear part costs at most βρ⋅OPT\beta\rho\cdot\mathrm{OPT}βρ⋅OPT for every h∈Uh\in\mathcal Uh∈U.
  5. Lemma 4.4 — the static part costs at most (1+2ρ+βρ)⋅OPT(1+2\rho+\beta\rho)\cdot\mathrm{OPT}(1+2ρ+βρ)⋅OPT.

With ρ=O(log⁡n)\rho=O(\log n)ρ=O(logn), Lemmas 4.3 and 4.4 give the goal.

Significance

The result. For disjoint budgets the bound is within a log⁡n\log nlogn factor of the hardness bound for a single budget, and it does not depend on the number of groups LLL. It shows that affine policies stay near-optimal beyond the single-budget case, for a class that models grouped demand. The paper notes that it is open whether the extra log⁡n\log nlogn factor is necessary. Gupta et al. give an O(log⁡n)O(\log n)O(logn)-approximation for a special case with 0/1 matrices and equal weights, but their algorithm is not an affine policy.

Formalizing it. The theorem is proved on paper; no machine-checked proof is known. The formal development makes explicit what the paper uses about the online covering algorithm (monotone, covering, competitive), handles the "without loss of generality" relabelling of the blocks in the greedy scenario, and records two gaps in the proof of Lemma 4.4 (zero weights, and a zero increment νℓ−νℓ−1\nu_\ell-\nu_{\ell-1}νℓ​−νℓ−1​, for which Lemma 3.3 is stated with γ>0\gamma>0γ>0).

Difficulty

The thresholds that split cheap from expensive components cannot use the single quantity OPT\mathrm{OPT}OPT as in the single-budget case: a scenario may activate one budget in every block at once, and summing per-block bounds gives a factor LLL. That is the O(Llog⁡n/log⁡log⁡n)O(L\log n/\log\log n)O(Llogn/loglogn) bound of the paper's Section 5. Removing the dependence on LLL requires per-block thresholds that add up to O(log⁡n)⋅OPTO(\log n)\cdot\mathrm{OPT}O(logn)⋅OPT, and the paper obtains them from the increments of an online covering algorithm along a greedily chosen worst sequence of scenarios. A proof of the goal therefore needs an O(log⁡n)O(\log n)O(logn)-competitive online fractional covering algorithm, Buchbinder and Naor's Theorem 4.1, posed on this platform as BNCovPack.Covering.theorem_4_1 (b57de23d-5953-424f-a729-5a018aa74b37, open). That theorem requires positive costs, so zero-cost columns, which this paper allows, need separate handling. The static part also needs Lemma 3.3, the core of the first mission.

Formalization scope

  • Data. Vectors are functions on Fin m, Fin n with the componentwise order; matrices are Matrix (Fin m) (Fin n) ℝ; Lean indices and blocks are 0-based. First stage and recourse share the dimension nnn, as on p. 1. The partition is S : Fin L → Finset (Fin m), pairwise disjoint and covering Fin m (empty blocks allowed).
  • Values. zARz_{AR}zAR​, zAffz_{Aff}zAff​, z(h)z(h)z(h), Θ\ThetaΘ and zStaz_{Sta}zSta​ are infima in the extended reals, +∞+\infty+∞ when infeasible. A formalization with real infima (equal to 000 on infeasible problems) or without Ph+q≥0Ph+q\ge0Ph+q≥0 would trivialize the goal and is excluded, as is a constant CCC that depends on the instance or on LLL.
  • Standing assumptions of p. 1: B≥0B\ge0B≥0, c,d≥0c,d\ge0c,d≥0, X\mathcal XX polyhedral, wℓi∈[0,1]w_{\ell i}\in[0,1]wℓi​∈[0,1], SSS a partition. "U⊆[0,1]m\mathcal U\subseteq[0,1]^mU⊆[0,1]m compact convex with ei∈Ue_i\in\mathcal Uei​∈U" holds for the set (4.1) and is not added.
  • Added hypotheses and corrections. (i) cTx≥0c^Tx\ge0cTx≥0 for x∈Xx\in\mathcal Xx∈X: Theorem 4.1 is false without it (already for L=1L=1L=1), and it holds whenever X⊆R+n\mathcal X\subseteq\mathbb R^n_+X⊆R+n​. (ii) n≥3n\ge3n≥3 where β≥0\beta\ge0β≥0 is needed, and a threshold N0N_0N0​ in Lemma 3.3, which as printed fails at n=2n=2n=2. (iii) Covering solutions viv_ivi​ are required only for i∈Ii\in\mathcal Ii∈I.
  • The online algorithm is an abstract structure OnlineCover with ratio ρ\rhoρ: starting at 000, non-negative, monotone, covering every revealed coverable row, and ρ\rhoρ-competitive on disjoint batches. The milestones hold for every such algorithm, with the paper's O(log⁡n)O(\log n)O(logn) written as ρ\rhoρ. The goal does not mention the algorithm. Algorithm 1 is a predicate on a run (a bijection of blocks and the chosen scenarios) with its maximality clause, so ties are arbitrary.
  • Thresholds are in multiplied form αiz(ei)≤wβ(νℓ−νℓ−1)\alpha_iz(e_i)\le w\beta(\nu_\ell-\nu_{\ell-1})αi​z(ei​)≤wβ(νℓ​−νℓ−1​); no division by a weight that may be 000.
  • Infrastructure. Lemma 3.3 is shared with the first mission and needs LP duality and randomized rounding. Buchbinder–Naor's theorem, or any O(log⁡n)O(\log n)O(logn)-competitive online fractional covering algorithm, is needed for the goal. Proofs of any milestone are welcome.

The source is arXiv:1807.00163v2; its printed page numbers equal the PDF page numbers.

Selected references

  • O. El Housni, V. Goyal, On the Optimality of Affine Policies for Budgeted Uncertainty Sets, arXiv:1807.00163v2, 2019; Mathematics of Operations Research 46(2), 2021. https://arxiv.org/abs/1807.00163
  • N. Buchbinder, J. Naor, Online primal-dual algorithms for covering and packing, Mathematics of Operations Research 34(2), 2009. https://doi.org/10.1287/moor.1080.0363
  • D. Bertsimas, V. Goyal, On the power and limitations of affine policies in two-stage adaptive optimization, Mathematical Programming 134(2), 2012. https://doi.org/10.1007/s10107-011-0444-4
  • U. Feige, K. Jain, M. Mahdian, V. Mirrokni, Robust combinatorial optimization with exponential scenarios, IPCO 2007, LNCS 4513.
  • A. Ben-Tal, A. Goryashko, E. Guslitzer, A. Nemirovski, Adjustable robust solutions of uncertain linear programs, Mathematical Programming 99(2), 2004. https://doi.org/10.1007/s10107-003-0454-y
11 thms1 active userReviewed
Operations ResearchOptimizationProbability·Captain: mikedeng1

On the Optimality of Affine Policies for Budgeted Uncertainty Sets 1: For Budget of Uncertainty Sets, Affine Policies Approximate the Two-Stage Adjustable Robust Problem within O(log n / log log n)Research Paper

Motivation

Two-stage adjustable robust optimization models decisions taken in two steps around an uncertain demand: a first-stage decision is fixed now, the demand is revealed, and a second-stage (recourse) decision reacts to it. The objective is the worst-case total cost over a set of possible demands. The model covers set cover, capacity planning and network design under uncertain demand (Bertsimas, Goyal 2012; Gupta, Nagarajan, Ravi 2014). Computing the optimal adjustable solution is intractable in general: Feige, Jain, Mahdian and Mirrokni (2007) showed that, for the budget of uncertainty sets below, the problem cannot be approximated within a factor better than Ω(log⁡n/log⁡log⁡n)\Omega(\log n/\log\log n)Ω(logn/loglogn) under a standard complexity assumption, where nnn is the number of decision variables.

Affine policies, in which the recourse is restricted to an affine function of the revealed demand (Ben-Tal et al. 2004), are computable by a single linear program and perform well empirically. The question is how much they can lose in the worst case.

Timeline. Bertsimas and Goyal (2012) proved that affine policies are within O(m)O(\sqrt m)O(m​) of the adjustable optimum for general uncertainty sets (mmm = number of uncertain parameters) and that this is tight for some sets. El Housni and Goyal (arXiv:1807.00163, 2018; Mathematics of Operations Research 2021) proved that for budget of uncertainty sets the factor is O(log⁡n/log⁡log⁡n)O(\log n/\log\log n)O(logn/loglogn), matching the hardness bound. That result is the goal of this mission.

Setting

Let A∈Rm×nA\in\mathbb R^{m\times n}A∈Rm×n, a non-negative recourse matrix B∈R+m×nB\in\mathbb R^{m\times n}_+B∈R+m×n​, costs c,d∈R+nc,d\in\mathbb R^n_+c,d∈R+n​ and a polyhedral set X⊆Rn\mathcal X\subseteq\mathbb R^nX⊆Rn be given. For an uncertainty set U⊆Rm\mathcal U\subseteq\mathbb R^mU⊆Rm of right-hand sides, problem (1.1) is

zAR(U)=min⁡x∈X cTx+max⁡h∈U min⁡y(h)≥0{dTy(h) : Ax+By(h)≥h}.z_{AR}(\mathcal U)=\min_{x\in\mathcal X}\ c^Tx+\max_{h\in\mathcal U}\ \min_{y(h)\ge0}\big\{d^Ty(h)\ :\ Ax+By(h)\ge h\big\}.zAR​(U)=x∈Xmin​ cTx+h∈Umax​ y(h)≥0min​{dTy(h) : Ax+By(h)≥h}.

A first-stage xxx and a recourse rule y(⋅)y(\cdot)y(⋅) form a feasible solution when x∈Xx\in\mathcal Xx∈X and y(h)≥0y(h)\ge0y(h)≥0, Ax+By(h)≥hAx+By(h)\ge hAx+By(h)≥h for every h∈Uh\in\mathcal Uh∈U. The affine problem (2.1) restricts the rule to y(h)=Ph+qy(h)=Ph+qy(h)=Ph+q with P∈Rn×mP\in\mathbb R^{n\times m}P∈Rn×m, q∈Rnq\in\mathbb R^nq∈Rn (still requiring Ph+q≥0Ph+q\ge0Ph+q≥0 on U\mathcal UU); its value is zAff(U)≥zAR(U)z_{Aff}(\mathcal U)\ge z_{AR}(\mathcal U)zAff​(U)≥zAR​(U).

The budget of uncertainty set (1.2) with weights wi∈[0,1]w_i\in[0,1]wi​∈[0,1] is

U={h∈[0,1]m : ∑i=1mwihi≤1}.\mathcal U=\Big\{h\in[0,1]^m\ :\ \sum_{i=1}^mw_ih_i\le1\Big\}.U={h∈[0,1]m : i=1∑m​wi​hi​≤1}.

The fractional covering value (3.2) of a requirement hhh is z(h)=min⁡y≥0{dTy:By≥h}z(h)=\min_{y\ge0}\{d^Ty: By\ge h\}z(h)=miny≥0​{dTy:By≥h}, and z(W)=z(1(W))z(\mathcal W)=z(\mathbb 1(\mathcal W))z(W)=z(1(W)) for W⊆[m]\mathcal W\subseteq[m]W⊆[m]. Throughout, log⁡\loglog is the natural logarithm and β=4log⁡n/log⁡log⁡n\beta=4\log n/\log\log nβ=4logn/loglogn.

Formalization targets

Goal: Theorem 3.1

zAff(U)=O(log⁡nlog⁡log⁡n)⋅zAR(U)z_{Aff}(\mathcal U)=O\Big(\frac{\log n}{\log\log n}\Big)\cdot z_{AR}(\mathcal U)zAff​(U)=O(loglognlogn​)⋅zAR​(U)

for every instance with U\mathcal UU a budget of uncertainty set: there are C>0C>0C>0 and NNN such that zAff≤Clog⁡nlog⁡log⁡nzARz_{Aff}\le C\frac{\log n}{\log\log n}z_{AR}zAff​≤Cloglognlogn​zAR​ for all n≥Nn\ge Nn≥N, all mmm and all data. The constant is left unfixed, so the goal survives any improvement of the constant.

Milestones (the paper's proof, in order)

  1. Lemma 3.1 — the explicit affine solution (3.5), a linear part yLin(h)=∑i∈Iαihiviy_{Lin}(h)=\sum_{i\in\mathcal I}\alpha_ih_iv_iyLin​(h)=∑i∈I​αi​hi​vi​ plus a static part from (3.4), is feasible.
  2. Lemma 3.2 — the linear part costs at most β⋅OPT\beta\cdot\mathrm{OPT}β⋅OPT for every h∈Uh\in\mathcal Uh∈U.
  3. Lemmas B.1, B.2 — two Chernoff bounds for weighted sums of independent Bernoulli variables.
  4. Claims 3.1–3.3 — rescaling of the covering problem, and two probability bounds for a randomized rounding of a dual (packing) solution.
  5. Lemma 3.3 (structural result) — if covering each single row i∈Ji\in\mathcal Ji∈J costs more than 4γlog⁡nlog⁡log⁡nwi4\gamma\frac{\log n}{\log\log n}w_i4γloglognlogn​wi​, while every scenario W⊆J\mathcal W\subseteq\mathcal JW⊆J with ∑Wwi≤1\sum_{\mathcal W}w_i\le1∑W​wi​≤1 costs at most γ\gammaγ, then z(J)≤4γlog⁡nlog⁡log⁡nz(\mathcal J)\le4\gamma\frac{\log n}{\log\log n}z(J)≤4γloglognlogn​.
  6. Lemma 3.4 — the static part costs O(β)⋅OPTO(\beta)\cdot\mathrm{OPT}O(β)⋅OPT.

Significance

The result. The bound shows that affine policies are an optimal approximation for this class, up to a constant factor: no polynomial-time algorithm can beat them by more than a constant in the worst case (under the hardness assumption). It improves the earlier O(m)O(\sqrt m)O(m​) bound for budget sets exponentially, and it is the base case from which the paper derives bounds for intersections of budget constraints (Sections 4–5). Lemma 3.3 is a statement about fractional covering problems alone; it generalizes a set-cover result of Gupta, Nagarajan and Ravi (2014) to arbitrary non-negative matrices and weights, and improves its bound from O(log⁡n)O(\log n)O(logn) to O(log⁡n/log⁡log⁡n)O(\log n/\log\log n)O(logn/loglogn).

Formalizing it. The theorem is proved on paper; no machine-checked proof of it or of the structural lemma is known. A formal development would also settle the paper's slips: Claim 3.2's 1−O(1/n)1-O(1/n)1−O(1/n) does not follow from its proof (stated here in the explicit form the proof gives), Lemma B.2's strict inequality fails when μ=0\mu=0μ=0, and Lemma 3.3 needs nnn large.

Difficulty

Feasibility and the cost of the linear part follow directly from the construction. The difficulty is the static part, that is, Lemma 3.3. The obvious argument covers J\mathcal JJ by a union of cheap scenarios, and it fails: each scenario covers rows of total weight at most 111, there can be many disjoint scenarios, and adding up their costs gives a bound proportional to ∑i∈Jwi\sum_{i\in\mathcal J}w_i∑i∈J​wi​, which may be as large as mmm. The lemma asserts a bound that depends on nnn alone, at the rate log⁡n/log⁡log⁡n\log n/\log\log nlogn/loglogn of the hardness result. Any argument therefore has to exploit condition 1 (every single row is expensive) together with condition 2 across all 2∣J∣2^{|\mathcal J|}2∣J∣ scenarios at once. As printed, the lemma holds only for nnn large; for n=2n=2n=2 it is false.

Formalization scope

  • Data. Vectors are functions on Fin m, Fin n with the componentwise order; matrices are Matrix (Fin m) (Fin n) ℝ; Lean index iii is the paper's i+1i+1i+1. First stage and recourse share the same dimension nnn, as on p. 1.
  • Values. zARz_{AR}zAR​, zAffz_{Aff}zAff​, z(h)z(h)z(h) and the static value zStaz_{Sta}zSta​ are infima in the extended reals, +∞+\infty+∞ when infeasible. The recourse is a function of hhh, which gives the same value as the paper's inner minimum. A formalization with real infima (equal to 000 on infeasible problems) or without Ph+q≥0Ph+q\ge0Ph+q≥0 would trivialize the goal and is excluded.
  • Standing assumptions of p. 1: B≥0B\ge0B≥0, c,d≥0c,d\ge0c,d≥0, X\mathcal XX polyhedral, wi∈[0,1]w_i\in[0,1]wi​∈[0,1]. The assumption "U⊆[0,1]m\mathcal U\subseteq[0,1]^mU⊆[0,1]m compact convex with ei∈Ue_i\in\mathcal Uei​∈U" holds for the budget set and is not added.
  • Added hypotheses and corrections. (i) cTx≥0c^Tx\ge0cTx≥0 for x∈Xx\in\mathcal Xx∈X: Theorem 3.1 is false without it, and it holds whenever X⊆R+n\mathcal X\subseteq\mathbb R^n_+X⊆R+n​. (ii) n≥3n\ge3n≥3 where β>0\beta>0β>0 is needed, and a threshold N0N_0N0​ in Lemma 3.3. (iii) Lemma B.2 in non-strict form. (iv) Claim 3.2 with the explicit bound 1−n eδ/(1+δ)1+δ1-n\,e^\delta/(1+\delta)^{1+\delta}1−neδ/(1+δ)1+δ, δ=η/2−2\delta=\eta/2-2δ=η/2−2. (v) Covering solutions viv_ivi​ are required only for i∈Ii\in\mathcal Ii∈I.
  • Quantifiers. In the goal and in Lemmas 3.3, 3.4 the constants CCC, NNN, N0N_0N0​ are chosen before mmm and the instance; a constant allowed to depend on the instance would make the goal trivial.
  • Randomness is finite: independent Bernoulli selection with probabilities and expectations as finite sums, reusing the published definition SpectralSparsify.Sampling.BernoulliSubset. No measure theory is needed.
  • Infrastructure. LP strong duality (LinearOptimization.lp_strong_duality, proved on the platform) is a tool for Lemma 3.3. The Chernoff bounds and the structural lemma are reusable beyond this mission. Proofs of any milestone, and alternative arguments for Lemma 3.3, are welcome.

The source is arXiv:1807.00163v2; its printed page numbers equal the PDF page numbers.

Selected references

  • O. El Housni, V. Goyal, On the Optimality of Affine Policies for Budgeted Uncertainty Sets, arXiv:1807.00163v2, 2019; Mathematics of Operations Research 46(2), 2021. https://arxiv.org/abs/1807.00163
  • D. Bertsimas, V. Goyal, On the power and limitations of affine policies in two-stage adaptive optimization, Mathematical Programming 134(2), 2012. https://doi.org/10.1007/s10107-011-0444-4
  • U. Feige, K. Jain, M. Mahdian, V. Mirrokni, Robust combinatorial optimization with exponential scenarios, IPCO 2007, LNCS 4513 (cited as [19] in the source).
  • A. Gupta, V. Nagarajan, R. Ravi, Thresholded covering algorithms for robust and max–min optimization, Mathematical Programming 146(1–2), 2014 (cited as [21] in the source).
  • A. Ben-Tal, A. Goryashko, E. Guslitzer, A. Nemirovski, Adjustable robust solutions of uncertain linear programs, Mathematical Programming 99(2), 2004. https://doi.org/10.1007/s10107-003-0454-y
  • P. Raghavan, Probabilistic construction of deterministic algorithms: approximating packing integer programs, FOCS 1986 (cited as [26] in the source).
14 thms1 active userReviewed
Mechanism DesignOperations Research·Captain: mikedeng1

Optimal Design for Multi-Item Auctions: A Robust Optimization Approach: The Robust Optimal Mechanism Is Incentive Compatible, Budget Feasible and Individually RationalResearch Paper

Motivation

Designing a revenue-maximizing auction for several items is hard once buyers have budgets. Myerson's optimal auction (Myerson 1981) solves the single-item problem without budgets, under a known prior on the buyers' values. With several items, correlated values or budget constraints, no comparable characterization is known, and the Bayesian formulation requires a prior the seller rarely has.

Bandi and Bertsimas (2014) replace the prior by an uncertainty set: a set U\mathcal UU of valuation matrices that the seller considers possible. The seller then maximizes worst-case revenue over U\mathcal UU, subject to incentive compatibility, budget feasibility and individual rationality. This is the approach of robust optimization (Ben-Tal and Nemirovski 1998; Bertsimas and Sim 2004) applied to mechanism design. The paper's central object is a mechanism built from linear optimization problems, the robust optimal mechanism ROM. It looks like a Vickrey–Clarke–Groves mechanism with reservation prices that depend on both the buyer and the item.

Setting

There are nnn buyers N={1,…,n}\mathcal N=\{1,\dots,n\}N={1,…,n} and mmm divisible items M={1,…,m}\mathcal M=\{1,\dots,m\}M={1,…,m}. A valuation vector is an n×mn\times mn×m real matrix v=(vij)\mathbf v=(v_{ij})v=(vij​), with vijv_{ij}vij​ buyer iii's value for one unit of item jjj. The uncertainty set U\mathcal UU is a set of such matrices, and B1,…,BnB_1,\dots,B_nB1​,…,Bn​ are public budgets. Writing (ui,v−i)(\mathbf u_i,\mathbf v_{-i})(ui​,v−i​) for v\mathbf vv with row iii replaced by ui\mathbf u_iui​, a mechanism assigns to each bid vector v∈U\mathbf v\in\mathcal Uv∈U an allocation xv≥0x^{\mathbf v}\ge 0xv≥0 and payments pvp^{\mathbf v}pv. It is feasible for the robust auction LP (5) if, for all v∈U\mathbf v\in\mathcal Uv∈U and i∈Ni\in\mathcal Ni∈N:

  • (7) ∑ixijv≤1\sum_i x^{\mathbf v}_{ij}\le 1∑i​xijv​≤1 for every item jjj;
  • (8) ∑jvijxij(ui,v−i)−pi(ui,v−i)≤∑jvijxijv−piv\sum_j v_{ij}x^{(\mathbf u_i,\mathbf v_{-i})}_{ij}-p^{(\mathbf u_i,\mathbf v_{-i})}_i\le\sum_j v_{ij}x^{\mathbf v}_{ij}-p^{\mathbf v}_i∑j​vij​xij(ui​,v−i​)​−pi(ui​,v−i​)​≤∑j​vij​xijv​−piv​ whenever (ui,v−i)∈U(\mathbf u_i,\mathbf v_{-i})\in\mathcal U(ui​,v−i​)∈U (incentive compatibility);
  • (9) piv≤Bip^{\mathbf v}_i\le B_ipiv​≤Bi​ (budget feasibility);
  • (10) piv≤∑jvijxijvp^{\mathbf v}_i\le\sum_j v_{ij}x^{\mathbf v}_{ij}piv​≤∑j​vij​xijv​ (individual rationality).

ROM has two stages. ROM.a runs before any bid. It picks a valuation z∈U\mathbf z\in\mathcal Uz∈U and a nominal allocation x∗\mathbf x^*x∗ solving the robust optimization problem (12). For each buyer it picks a minimizer u~i∈U\tilde{\mathbf u}^i\in\mathcal Uu~i∈U of ∑jxij∗uij\sum_j x^*_{ij}u_{ij}∑j​xij∗​uij​ (14), and an optimal solution (ξ∗,η∗,θ∗)(\xi^*,\eta^*,\theta^*)(ξ∗,η∗,θ∗) of a linear program (13). It then sets reservation prices rij∗=ξj∗+ηi∗zij+θi∗zijr^*_{ij}=\xi^*_j+\eta^*_iz_{ij}+\theta^*_iz_{ij}rij∗​=ξj∗​+ηi∗​zij​+θi∗​zij​ (15). ROM.b runs on the bids v\mathbf vv:

  • it maximizes ∑i,jyij(vij−rij∗)\sum_{i,j}y_{ij}(v_{ij}-r^*_{ij})∑i,j​yij​(vij​−rij∗​) over a polyhedron P\mathcal PP (16), (18), and for each buyer kkk the same objective without buyer kkk over a polyhedron Qk\mathcal Q_kQk​ (17), (19);
  • it allocates av=x∗+yva^{\mathbf v}=\mathbf x^*+\mathbf y^{\mathbf v}av=x∗+yv (20) and charges VCG-type payments pvp^{\mathbf v}pv (21).

The Lean development uses the same names: ROMData for the chosen optimizers, IsROMa, IsROM, rStar, PSet, QSet, alloc, pay, and IsFeasibleMechanism for (7)–(10).

Formalization targets

Goal: ROM is feasible for (5)

For every uncertainty set of nonnegative valuation matrices, every nonnegative budget vector and every choice of optimal solutions of (12), (13), (14), (16), (17):

(a,p) satisfies (7), a≥0, (8), (9), (10) on U.(a,p)\ \text{satisfies (7)},\ a\ge 0,\ \text{(8)},\ \text{(9)},\ \text{(10) on } \mathcal U.(a,p) satisfies (7), a≥0, (8), (9), (10) on U.

This is the feasibility half of the paper's Theorem 1; its proof (p. 1025) says "the feasibility follows from Propositions 3–5".

Milestones

  • Proposition 2 (p. 1020): the relations (22)–(27) among z,x∗,ξ∗,η∗,θ∗,r∗\mathbf z,\mathbf x^*,\xi^*,\eta^*,\theta^*,r^*z,x∗,ξ∗,η∗,θ∗,r∗, among them xij∗rij∗=xij∗zijx^*_{ij}r^*_{ij}=x^*_{ij}z_{ij}xij∗​rij∗​=xij∗​zij​ and ∑jxij∗rij∗≤Bi\sum_jx^*_{ij}r^*_{ij}\le B_i∑j​xij∗​rij∗​≤Bi​.
  • Proposition 3 (p. 1021): ROM satisfies (8).
  • Proposition 4 (p. 1022): ROM satisfies (9).
  • Proposition 5 (p. 1023): ROM satisfies (10).

Significance

The result shows that a mechanism specified by the paper's optimization problems is incentive compatible, budget feasible and individually rational over an arbitrary uncertainty set. No distributional assumption is made, and multiple items and budgets are allowed. Proposition 2 also records the identity between the total reserved revenue and the optimal value of the auxiliary program (13).

Status. The paper's main claim, Theorem 1, says that ROM is optimal for (5), i.e. that it maximizes worst-case revenue. That claim is false as printed, and so is Proposition 6 (the worst case of ROM is attained at z\mathbf zz). Small counterexamples refute both:

  • with one buyer, one item, U={1,2}\mathcal U=\{1,2\}U={1,2} and a large budget, some choice of optimal dual solution in (13) makes ROM's worst-case revenue 000, while a posted price of 111 earns 111;
  • with two buyers, one item and U={(1,2),(2,1)}\mathcal U=\{(1,2),(2,1)\}U={(1,2),(2,1)}, full surplus extraction earns 222, while ROM earns at most 111 for every choice of optimizers.

The feasibility half is true, and this mission formalizes exactly that half. No result of this paper is on Prove2Me yet, and no machine-checked proof of it is known.

Difficulty

The difficulty lies in the definitions and in how many problems interlock, not in any single deep step. ROM is defined through five optimization problems, none with a unique solution. The payments (21) of a buyer depend on the optimizer chosen in (17), which may differ between a truthful and a deviating bid vector. Incentive compatibility must therefore rest on properties of optimal values, not of particular optimizers. The robust constraints of (12) and (18) range over the whole of U\mathcal UU, which may be infinite. Relating x∗\mathbf x^*x∗ to the multipliers of (13) requires replacing the robust constraint by a finite one before linear programming duality applies. Budget feasibility requires comparing two polyhedra whose budget rows differ by a term involving θ∗\theta^*θ∗ and u~i\tilde{\mathbf u}^iu~i. Neighbouring statements in the same section (Theorem 1, Proposition 6) are false, so no step can be taken on the paper's word.

Formalization scope

Buyers are Fin n and items Fin m. A valuation matrix is Fin n → Fin m → ℝ, the uncertainty set an arbitrary Set of them, and (ui,v−i)(\mathbf u_i,\mathbf v_{-i})(ui​,v−i​) is Function.update v i u_i. Every optimizer is a field of ROMData. IsROMa/IsROM assert optimality for exactly the page's problems, built from the same z\mathbf zz, x∗\mathbf x^*x∗, θ∗\theta^*θ∗, u~i\tilde{\mathbf u}^iu~i. Every statement holds for every such choice, never for one canonical selection. Allocation and payments are 000 for bids outside U\mathcal UU (step 1 of ROM.b); constraints (7)–(10) are imposed only on U\mathcal UU.

Committed conventions and additions to the page:

  • Nonnegative valuations (uij≥0u_{ij}\ge 0uij​≥0 for u∈U\mathbf u\in\mathcal Uu∈U) and nonnegative budgets are standing assumptions the paper uses without stating. The proof of Proposition 5 invokes "u~kj\tilde u_{kj}u~kj​ ... nonnegative", and (12) is infeasible for negative budgets. Both are explicit hypotheses.
  • No finiteness, boundedness or convexity of U\mathcal UU is assumed. Existence of the optimizers is part of the hypothesis IsROM, and its satisfiability has been checked on a one-buyer, one-item instance.
  • Payments may be negative; no p≥0p\ge 0p≥0 is imposed, as (5) has none.
  • In Qk\mathcal Q_kQk​ row kkk is only nonnegative; it enters neither constraint nor objective.
  • Printed slip corrected: in (18) the term ∑jxkj∗θi∗u~ji\sum_jx^*_{kj}\theta^*_i\tilde u^i_j∑j​xkj∗​θi∗​u~ji​ has an unbound kkk and is read as ∑jxij∗θi∗u~ji\sum_jx^*_{ij}\theta^*_i\tilde u^i_j∑j​xij∗​θi∗​u~ji​, matching (13) and (34).
  • The paper's optimality claims (Theorem 1, Proposition 6), the objective (5)–(6) and the dual (11) are deliberately not part of the mission.

A trivializing formalization is ruled out: the optimizers are not fixed to degenerate values (such as θ∗=0\theta^*=0θ∗=0 or a canonical y\mathbf yy), constraint (8) ranges over all deviations that stay in U\mathcal UU with the true values multiplying both allocations, and the hypotheses are jointly satisfiable.

A complete development needs finite-dimensional LP duality and complementary slackness (for Proposition 2), and finite-sum manipulation over Fin n with one index removed. LP duality lemmas in matrix form are reusable well beyond this mission. Contributions are welcome on any milestone, and on general LP duality statements stated in a form that applies directly to (33)–(34).

Selected references

  • S. Bandi, D. Bertsimas, Optimal Design for Multi-Item Auctions: A Robust Optimization Approach, Mathematics of Operations Research 39(4):1012–1038, 2014. https://doi.org/10.1287/moor.2014.0645
  • R. B. Myerson, Optimal Auction Design, Mathematics of Operations Research 6(1):58–73, 1981. https://doi.org/10.1287/moor.6.1.58
  • A. Ben-Tal, A. Nemirovski, Robust Convex Optimization, Mathematics of Operations Research 23(4):769–805, 1998. https://doi.org/10.1287/moor.23.4.769
  • D. Bertsimas, M. Sim, The Price of Robustness, Operations Research 52(1):35–53, 2004. https://doi.org/10.1287/opre.1030.0065
7 thms1 active userReviewed
CombinatoricsOptimization·Captain: mikedeng1

The Running Intersection Relaxation of the Multilinear Polytope 2: If a Hypergraph Is Not β-Acyclic, Its Multilinear Polytope Is Strictly Contained in the Running Intersection RelaxationResearch Paper

Motivation

Many nonconvex optimization problems in binary variables have a polynomial objective or polynomial constraints. The standard way to bring them into the reach of linear and mixed-integer programming solvers is to introduce one new variable per monomial and to study the convex hull of the resulting set of feasible 0/1 vectors, the multilinear polytope. Strong linear descriptions of this polytope translate directly into tighter relaxations in global solvers such as BARON, which is where the inequalities of this mission were tested (Del Pia and Khajavirad 2021, §2.5).

The multilinear polytope of a general hypergraph has no compact description, so the question is for which hypergraphs a given family of inequalities is enough. The quadratic case is classical: the Boolean quadric polytope of a graph (Padberg 1989) is described by its standard linearization exactly when the graph is a forest, and the odd-cycle inequalities are the first facets that appear on a cycle. For higher-degree monomials, Del Pia and Khajavirad showed in 2018 that their flower relaxation is exact if and only if the hypergraph is γ-acyclic. The 2021 paper introduces the stronger running intersection relaxation and asks the same question for it. This mission formalizes the necessary condition the paper proves: the relaxation cannot be exact unless the hypergraph is β-acyclic.

Setting

A hypergraph G=(V,E)G=(V,E)G=(V,E) is a finite node set VVV with a set EEE of subsets of VVV of size at least two (no loops, no parallel edges). Its multilinear set is

SG={z∈{0,1}V+E:ze=∏v∈ezv  ∀e∈E},\mathcal S_G=\Big\{z\in\{0,1\}^{V+E}: z_e=\prod_{v\in e}z_v\ \ \forall e\in E\Big\},SG​={z∈{0,1}V+E:ze​=v∈e∏​zv​  ∀e∈E},

and the multilinear polytope is MPG=conv⁡SG\mathrm{MP}_G=\operatorname{conv}\mathcal S_GMPG​=convSG​. The standard linearization is

MPGLP={z:zv≤1 ∀v; ze≥0, ze≥∑v∈ezv−∣e∣+1 ∀e; ze≤zv ∀e, v∈e}.\mathrm{MP}^{\mathrm{LP}}_G=\Big\{z: z_v\le 1\ \forall v;\ z_e\ge 0,\ z_e\ge\sum_{v\in e}z_v-|e|+1\ \forall e;\ z_e\le z_v\ \forall e,\ v\in e\Big\}.MPGLP​={z:zv​≤1 ∀v; ze​≥0, ze​≥v∈e∑​zv​−∣e∣+1 ∀e; ze​≤zv​ ∀e, v∈e}.

An ordering p1,…,pmp_1,\dots,p_mp1​,…,pm​ of a multiset of sets is a running intersection ordering if each pkp_kpk​, k≥2k\ge2k≥2, meets the union of its predecessors inside a single predecessor pjp_jpj​; it induces N(p1)=∅N(p_1)=\emptysetN(p1​)=∅ and N(pk)=pk∩⋃i<kpiN(p_k)=p_k\cap\bigcup_{i<k}p_iN(pk​)=pk​∩⋃i<k​pi​. A running intersection inequality has a center e0∈Ee_0\in Ee0​∈E, neighbors eke_kek​ (k∈Kk\in Kk∈K) meeting e0e_0e0​ such that E~={e0∩ek}\tilde E=\{e_0\cap e_k\}E~={e0​∩ek​} admits a running intersection ordering, and nodes uk∈N(e0∩ek)u_k\in N(e_0\cap e_k)uk​∈N(e0​∩ek​) when these sets are nonempty:

−∑k: N(e0∩ek)≠∅zuk+∑v∈e0∖⋃kekzv+∑k∈Kzek−ze0≤ω−1,-\sum_{k:\,N(e_0\cap e_k)\ne\emptyset}z_{u_k}+\sum_{v\in e_0\setminus\bigcup_k e_k}z_v+\sum_{k\in K}z_{e_k}-z_{e_0}\le\omega-1,−k:N(e0​∩ek​)=∅∑​zuk​​+v∈e0​∖⋃k​ek​∑​zv​+k∈K∑​zek​​−ze0​​≤ω−1,

with ω\omegaω the number of connected components of (e0,E~)(e_0,\tilde E)(e0​,E~). The running intersection relaxation MPGRI\mathrm{MP}^{\mathrm{RI}}_GMPGRI​ is MPGLP\mathrm{MP}^{\mathrm{LP}}_GMPGLP​ together with all these inequalities.

A β-cycle of length t≥3t\ge3t≥3 is a sequence v1,e1,…,vt,et,v1v_1,e_1,\dots,v_t,e_t,v_1v1​,e1​,…,vt​,et​,v1​ of distinct nodes and distinct edges in which each viv_ivi​ lies in ei−1e_{i-1}ei−1​ and eie_iei​ and in no other eje_jej​; GGG is β-acyclic if it has none. For W⊆VW\subseteq VW⊆V the subhypergraph GWG_WGW​ has node set WWW and edges {e∩W:∣e∩W∣≥2}\{e\cap W: |e\cap W|\ge2\}{e∩W:∣e∩W∣≥2}.

Formalization targets

Goal: Proposition 5

G not β-acyclic ⟹ MPG⊊MPGRI.G\ \text{not β-acyclic}\ \Longrightarrow\ \mathrm{MP}_G\subsetneq\mathrm{MP}^{\mathrm{RI}}_G .G not β-acyclic ⟹ MPG​⊊MPGRI​.

The milestones follow the paper's proof, in order:

  1. MPG⊆MPGLP\mathrm{MP}_G\subseteq\mathrm{MP}^{\mathrm{LP}}_GMPG​⊆MPGLP​ (p. 1014) and validity of every running intersection inequality (Proposition 1), giving MPG⊆MPGRI\mathrm{MP}_G\subseteq\mathrm{MP}^{\mathrm{RI}}_GMPG​⊆MPGRI​.
  2. Lemma 3: β-cycles of GV′G_{V'}GV′​ lift to β-cycles of GGG of the same length.
  3. Lemma 5: MPGVˉ=projGVˉ(MPG∩LVˉ)\mathrm{MP}_{G_{\bar V}}=\mathrm{proj}_{G_{\bar V}}(\mathrm{MP}_G\cap L_{\bar V})MPGVˉ​​=projGVˉ​​(MPG​∩LVˉ​), with LVˉ={z:zv=1 ∀v∉Vˉ}L_{\bar V}=\{z: z_v=1\ \forall v\notin\bar V\}LVˉ​={z:zv​=1 ∀v∈/Vˉ}.
  4. Lemma 6: MPGVˉRI⊆projGVˉ(MPGRI∩LVˉ)\mathrm{MP}^{\mathrm{RI}}_{G_{\bar V}}\subseteq\mathrm{proj}_{G_{\bar V}}(\mathrm{MP}^{\mathrm{RI}}_G\cap L_{\bar V})MPGVˉ​RI​⊆projGVˉ​​(MPGRI​∩LVˉ​).
  5. The reduction: strictness for GWG_WGW​ implies strictness for GGG.
  6. For a shortest β-cycle CCC, GV(C)G_{V(C)}GV(C)​ is a chordless cycle, possibly enclosed by the edge V(C)V(C)V(C).
  7. Strictness for a chordless cycle.
  8. Strictness for a chordless cycle enclosed by the edge V(C)V(C)V(C).

Significance

The result is one half of the paper's picture of when the running intersection relaxation is exact. Proposition 5 shows β-acyclicity is necessary; Theorem 3 of the paper shows that kite-free β-acyclicity is sufficient; Example 2 shows β-acyclicity alone is not. Combined with the 2018 characterization of the flower relaxation by γ-acyclicity, it places the running intersection relaxation strictly between two acyclicity notions and tells a solver designer that on hypergraphs with β-cycles some other family of cuts (for instance odd-cycle-type inequalities lifted from Padberg's) is unavoidable.

The intermediate results are reusable on their own. Lemma 5 is a general statement that multilinear polytopes of induced subhypergraphs are projections of faces, and Lemma 3 with the shortest-cycle structure step is pure hypergraph combinatorics.

The result is proved in the paper. No part of it, nor the multilinear polytope, nor the Boolean quadric polytope, is formalized in Lean or Mathlib as far as a search of the platform showed. Lemma 6 is stated in the paper without proof, and the two strictness cases are justified there by "it can be checked" and by a citation of Padberg; a formal proof closes these gaps.

Difficulty

The obvious argument exhibits a point of MPGRI∖MPG\mathrm{MP}^{\mathrm{RI}}_G\setminus\mathrm{MP}_GMPGRI​∖MPG​ directly from a β-cycle of GGG. This fails because a β-cycle of GGG is surrounded by arbitrary other edges, which contribute running intersection inequalities that can cut off any candidate point. The paper's route is to pass to the subhypergraph induced by the cycle's nodes, and that step requires two facts with real content: the projection lemmas (Lemma 6 is an inclusion between relaxations defined by different families of inequalities), and the structure of GV(C)G_{V(C)}GV(C)​, which holds only for a shortest β-cycle. A long β-cycle can have chords in GV(C)G_{V(C)}GV(C)​.

Strictness on the two small families requires a point, or an inequality, separating MP\mathrm{MP}MP from MPRI\mathrm{MP}^{\mathrm{RI}}MPRI. That means controlling every running intersection inequality of the cycle, over all centers, neighbor sets, orderings and choices of uku_kuk​, and also checking that the separating inequality is valid for MP\mathrm{MP}MP.

Formalization scope

Lean conventions:

  • Nodes are elements of a finite type α with decidable equality. Hypergraphs carry their node set and their edge set as Finsets, with every edge of size at least two.
  • Points of RV+E\mathbb R^{V+E}RV+E are functions α ⊕ Finset α → ℝ, and every polytope of GGG is pinned to zero off GGG's own coordinates, so polytopes of different hypergraphs, and the projection projGVˉ\mathrm{proj}_{G_{\bar V}}projGVˉ​​, live in one space.
  • MPGLP\mathrm{MP}^{\mathrm{LP}}_GMPGLP​ is exactly (8), with no row zv≥0z_v\ge0zv​≥0.
  • MPGRI\mathrm{MP}^{\mathrm{RI}}_GMPGRI​ quantifies over all centers, all sets of distinct neighbors other than e0e_0e0​, all running intersection orderings and all choices of uku_kuk​.
  • ω\omegaω is the number of connected components of the 2-section graph on e0e_0e0​.
  • Orderings and cycles are 0-indexed.
  • The projection takes the choice e′(f)e'(f)e′(f) of an edge of GGG with trace fff as an argument, and Lemmas 5 and 6 hold for every choice.
  • The strict inclusion ⊂\subset⊂ of the paper is Lean's ⊂ (⊆\subseteq⊆ and ≠\ne=).

There is one disclosed addition. Lemma 6 and the reduction step assume that GVˉG_{\bar V}GVˉ​ has no isolated node. Without it both statements are false: since (8) has no lower bound on zvz_vzv​, a node isolated in GVˉG_{\bar V}GVˉ​ is unbounded below in MPGVˉRI\mathrm{MP}^{\mathrm{RI}}_{G_{\bar V}}MPGVˉ​RI​. The paper uses them only for Vˉ=V(C)\bar V=V(C)Vˉ=V(C), where the hypothesis holds. The goal itself carries no added hypothesis.

A trivializing formalization is ruled out. A relaxation missing some running intersection inequalities (a fixed ordering, a fixed uku_kuk​, a fixed neighbor set) would make strictness easier, and an empty or mis-pinned relaxation would make the inclusion false. The definitions here take the full family.

A complete development needs:

  • convex hulls of finite point sets and separation of a point from them;
  • connected components of the 2-section graph;
  • running intersection orderings;
  • the combinatorics of β-cycles.

The β-cycle and subhypergraph material and Lemma 5 are reusable beyond this mission. Contributions are welcome on any milestone, on alternative proofs of the two strictness cases (for instance by an explicit fractional point), and on a proof of Lemma 6.

Selected references

  • A. Del Pia and A. Khajavirad, The running intersection relaxation of the multilinear polytope, Mathematics of Operations Research 46(3):1008–1037, 2021. https://doi.org/10.1287/moor.2021.1121
  • A. Del Pia and A. Khajavirad, A polyhedral study of binary polynomial programs, Mathematics of Operations Research 42(2):389–410, 2017. https://doi.org/10.1287/moor.2016.0804
  • A. Del Pia and A. Khajavirad, The multilinear polytope for acyclic hypergraphs, SIAM Journal on Optimization 28(2):1049–1076, 2018. https://doi.org/10.1137/16M1095998
  • M. Padberg, The Boolean quadric polytope: some characteristics, facets and relatives, Mathematical Programming 45:139–172, 1989. https://doi.org/10.1007/BF01589101
  • C. Beeri, R. Fagin, D. Maier and M. Yannakakis, On the desirability of acyclic database schemes, Journal of the ACM 30(3):479–513, 1983. https://doi.org/10.1145/2402.322389
11 thms1 active userReviewed
CombinatoricsOptimization·Captain: mikedeng1

The Running Intersection Relaxation of the Multilinear Polytope 1: For Kite-Free β-Acyclic Hypergraphs the Running Intersection Relaxation Equals the Multilinear PolytopeResearch Paper

Motivation

Binary polynomial optimization asks to minimize a polynomial over {0,1}n\{0,1\}^n{0,1}n. The standard way to attack it with linear programming is to introduce one variable zez_eze​ for every monomial ∏v∈ezv\prod_{v\in e} z_v∏v∈e​zv​ and to work with the convex hull of the resulting set of binary points. This convex hull, the multilinear polytope, is the object behind the cutting planes that global solvers such as BARON add at every node of a branch-and-bound tree, and in general its facets are as hard to describe as the original problem is to solve. The useful theory is therefore structural: which hypergraphs of monomials give multilinear polytopes with an explicit, combinatorial description.

A line of work by Del Pia and Khajavirad answers this for increasingly general classes of acyclic hypergraphs: Berge-acyclic and γ-acyclic hypergraphs via flower inequalities (SIAM J. Optim. 2018), sufficient conditions for decomposing multilinear sets (Math. Program. 2018), and, in the paper this mission formalizes (Math. Oper. Res. 2021), the running intersection inequalities, a generalization of flower inequalities that does not need γ-acyclicity. The paper reports that adding these cuts reduces BARON's root-node relaxation gap by more than 95% on most of 35 computer-vision instances.

Setting

A hypergraph G=(V,E)G=(V,E)G=(V,E) has a finite node set VVV and a set EEE of subsets of VVV, each with at least two nodes. Points live in RV+E\mathbb R^{V+E}RV+E, one coordinate per node and per edge. The multilinear set and multilinear polytope are

SG={z∈{0,1}V+E:ze=∏v∈ezv ∀e∈E},MPG=conv⁡SG.\mathcal S_G=\Big\{z\in\{0,1\}^{V+E}: z_e=\prod_{v\in e}z_v\ \forall e\in E\Big\},\qquad \mathrm{MP}_G=\operatorname{conv}\mathcal S_G .SG​={z∈{0,1}V+E:ze​=v∈e∏​zv​ ∀e∈E},MPG​=convSG​.

The standard linearization MPGLP\mathrm{MP}^{\mathrm{LP}}_GMPGLP​ replaces each equation ze=∏v∈ezvz_e=\prod_{v\in e}z_vze​=∏v∈e​zv​ by its convex hull over the unit cube: zv≤1z_v\le 1zv​≤1, ze≥0z_e\ge 0ze​≥0, ze≥∑v∈ezv−∣e∣+1z_e\ge\sum_{v\in e}z_v-|e|+1ze​≥∑v∈e​zv​−∣e∣+1 and ze≤zvz_e\le z_vze​≤zv​ for v∈ev\in ev∈e.

An ordering p1,…,pmp_1,\dots,p_mp1​,…,pm​ of a family of sets is a running intersection ordering if each pkp_kpk​ meets the union of its predecessors inside a single predecessor pjp_jpj​; write N(pk)=pk∩⋃i<kpiN(p_k)=p_k\cap\bigcup_{i<k}p_iN(pk​)=pk​∩⋃i<k​pi​ and N(p1)=∅N(p_1)=\emptysetN(p1​)=∅. Given a center edge e0e_0e0​ and neighbor edges eke_kek​, k∈Kk\in Kk∈K, meeting e0e_0e0​ such that the sets e0∩eke_0\cap e_ke0​∩ek​ admit a running intersection ordering, and nodes uk∈N(e0∩ek)u_k\in N(e_0\cap e_k)uk​∈N(e0​∩ek​) wherever this set is nonempty, the running intersection inequality is

−∑k: N(e0∩ek)≠∅zuk+∑v∈e0∖⋃kekzv+∑k∈Kzek−ze0≤ω−1,-\sum_{k:\,N(e_0\cap e_k)\neq\emptyset}z_{u_k}+\sum_{v\in e_0\setminus\bigcup_k e_k}z_v+\sum_{k\in K}z_{e_k}-z_{e_0}\le\omega-1,−k:N(e0​∩ek​)=∅∑​zuk​​+v∈e0​∖⋃k​ek​∑​zv​+k∈K∑​zek​​−ze0​​≤ω−1,

with ω\omegaω the number of connected components of the hypergraph on e0e_0e0​ with edges e0∩eke_0\cap e_ke0​∩ek​. The running intersection relaxation MPGRI\mathrm{MP}^{\mathrm{RI}}_GMPGRI​ is MPGLP\mathrm{MP}^{\mathrm{LP}}_GMPGLP​ together with all these inequalities.

A β-cycle is a cyclic sequence of t≥3t\ge 3t≥3 distinct nodes and distinct edges v1,e1,…,vt,etv_1,e_1,\dots,v_t,e_tv1​,e1​,…,vt​,et​ in which viv_ivi​ lies in ei−1e_{i-1}ei−1​ and eie_iei​ and in no other edge of the cycle; GGG is β-acyclic if it has none. A kite is three edges e0,e1,e2e_0,e_1,e_2e0​,e1​,e2​ with ∣e0∩e1∩e2∣≥2|e_0\cap e_1\cap e_2|\ge 2∣e0​∩e1​∩e2​∣≥2, (e0∩e1)∖e2≠∅(e_0\cap e_1)\setminus e_2\ne\emptyset(e0​∩e1​)∖e2​=∅ and (e0∩e2)∖e1≠∅(e_0\cap e_2)\setminus e_1\ne\emptyset(e0​∩e2​)∖e1​=∅; GGG is kite-free if it has none. GGG is two-laminar if any two distinct edges sharing at least two nodes are strictly nested.

Formalization targets

Goal: Theorem 3 (p. 1023)

For every kite-free β-acyclic hypergraph GGG without isolated nodes,

MPG=MPGRI.\mathrm{MP}_G=\mathrm{MP}^{\mathrm{RI}}_G .MPG​=MPGRI​.

Milestones

In attack order: validity of the inequalities (Proposition 1); heredity of β-acyclicity (Lemma 3); kite-free implies edge-induced subhypergraphs are two-laminar (Lemma 7); β-acyclicity as the running intersection property of every edge subset (Lemma 4); the explicit facet description of MPG\mathrm{MP}_GMPG​ for two-laminar β-acyclic GGG by system (20) (Proposition 6) and its consequence MPG=MPGRI\mathrm{MP}_G=\mathrm{MP}^{\mathrm{RI}}_GMPG​=MPGRI​ there (Corollary 3); the decomposition theorem for multilinear sets (Theorem 1); and five steps of the inductive proof of Theorem 3 in §5.3: GωG_\omegaGω​ is kite-free β-acyclic with one maximal edge fewer, the inductive step MPG+=MPG+RI\mathrm{MP}_{G^+}=\mathrm{MP}^{\mathrm{RI}}_{G^+}MPG+​=MPG+RI​, Claim 14 on partial hypergraphs, the projection MPG=proj MPG+\mathrm{MP}_G=\mathrm{proj}\,\mathrm{MP}_{G^+}MPG​=projMPG+​, and the Fourier–Motzkin step MPGRI⊆proj MPG+RI\mathrm{MP}^{\mathrm{RI}}_G\subseteq\mathrm{proj}\,\mathrm{MP}^{\mathrm{RI}}_{G^+}MPGRI​⊆projMPG+RI​.

Companions

Theorem 2 (p. 1022), the polynomial-size extended formulation MPG+=⋂iMPGi+\mathrm{MP}_{G^+}=\bigcap_{i}\mathrm{MP}_{G^+_i}MPG+​=⋂i​MPGi+​​ over the maximal edges, and the form (19) taken by running intersection inequalities in kite-free β-acyclic hypergraphs.

Significance

Theorem 3 gives a description of the multilinear polytope in the original space, by inequalities defined combinatorially from the hypergraph, for a class that strictly contains the γ-acyclic hypergraphs covered by flower inequalities. With Theorem 2 it implies that linear optimization and separation over MPG\mathrm{MP}_GMPG​ are polynomial for kite-free β-acyclic hypergraphs, and that separation can be done without the ellipsoid method. The paper also shows (Proposition 5, a separate mission of this series) that β-acyclicity is necessary for MPG=MPGRI\mathrm{MP}_G=\mathrm{MP}^{\mathrm{RI}}_GMPG​=MPGRI​, so the characterization sits close to the boundary of what such relaxations can achieve.

All results of this mission are proved in the paper; none is machine-checked. A complete development would be the first formal treatment of hypergraph acyclicity degrees and of multilinear polytopes, and would include a formal Fourier–Motzkin elimination argument on a structured polytope.

Difficulty

The inclusion MPG⊆MPGRI\mathrm{MP}_G\subseteq\mathrm{MP}^{\mathrm{RI}}_GMPG​⊆MPGRI​ is a validity check. The reverse inclusion is the content. A direct argument through integrality fails: for two-laminar β-acyclic hypergraphs, facets have general integer coefficients, so total unimodularity and balancedness arguments used for laminar hypergraphs do not apply. The decomposition step is also harder than in earlier work: the shared part of the two pieces may be an edge whose nodes are only partly covered by edges of one side, so a point cannot be written by pairing one vertex of each side. Finally, the induction lifts to G+G^+G+, which has an extra variable, and the projection back must be shown to produce only running intersection inequalities, which requires recognizing every Fourier–Motzkin combination as one.

Formalization scope

Nodes form a type α with Fintype and DecidableEq; a point is a function z : α ⊕ Finset α → ℝ, and every polytope of a hypergraph vanishes off that hypergraph's coordinates, so that polytopes of different hypergraphs, lifts and projections are literal sets in one space. Hypergraphs have no loops or parallel edges, as the paper assumes. Orderings are lists, 0-based. The relaxation quantifies over every center, every neighbor set, every running intersection ordering and every choice of the nodes uku_kuk​; the number of components counts isolated nodes and ignores loops. The standard linearization has no row zv≥0z_v\ge 0zv​≥0, exactly as printed.

One hypothesis is added to the paper: no isolated nodes, on Theorem 3, Corollary 3, Theorem 2 and the inductive step. It is necessary: for V={a,b,c}V=\{a,b,c\}V={a,b,c}, E={{a,b}}E=\{\{a,b\}\}E={{a,b}} the point with zc=−1z_c=-1zc​=−1 and all other coordinates 000 lies in MPGRI\mathrm{MP}^{\mathrm{RI}}_GMPGRI​ but not in MPG\mathrm{MP}_GMPG​. No other result carries it.

A trivializing formalization is ruled out by construction: restricting the relaxation to one fixed ordering or choice of uku_kuk​ would make the goal false rather than easy, and the goal is an equality of sets, not an inclusion.

A complete development needs: convex hulls of finite sets of 0/1 points, hypergraph connectivity, β-acyclicity and the running intersection property (Lemma 4 is a classical database-theory theorem of Beeri, Fagin, Maier and Yannakakis), and Fourier–Motzkin elimination. The hypergraph acyclicity layer is reusable beyond this mission, including by the two other missions of this series. Proofs of any milestone, and of reusable lemmas on hypergraph acyclicity, are welcome.

Selected references

  • A. Del Pia, A. Khajavirad, The running intersection relaxation of the multilinear polytope, Math. Oper. Res. 46(3):1008–1037, 2021. https://doi.org/10.1287/moor.2021.1121
  • A. Del Pia, A. Khajavirad, The multilinear polytope for acyclic hypergraphs, SIAM J. Optim. 28(2):1049–1076, 2018. https://doi.org/10.1137/16M1095998
  • A. Del Pia, A. Khajavirad, On decomposability of multilinear sets, Math. Program. 170(2):387–415, 2018. https://doi.org/10.1007/s10107-017-1158-y
  • C. Beeri, R. Fagin, D. Maier, M. Yannakakis, On the desirability of acyclic database schemes, J. ACM 30(3):479–513, 1983. https://doi.org/10.1145/2402.322389
  • R. Fagin, Degrees of acyclicity for hypergraphs and relational database schemes, J. ACM 30(3):514–550, 1983. https://doi.org/10.1145/2402.322390
14 thms1 active userReviewed
CombinatoricsOptimization·Captain: mikedeng1

The Running Intersection Relaxation of the Multilinear Polytope 3: Running Intersection Inequalities Meeting Three Support Conditions Define Facets of the Multilinear PolytopeResearch Paper

Motivation

Binary polynomial optimization asks for the maximum of a polynomial over {0,1}n\{0,1\}^n{0,1}n. Linearizing each monomial ∏v∈ezv\prod_{v\in e}z_v∏v∈e​zv​ by a new variable zez_eze​ turns the problem into a linear program over the convex hull of a set of 0/1 points, the multilinear polytope of the hypergraph whose edges are the monomials. Global solvers such as BARON build polyhedral relaxations of this polytope at every node of their branch-and-bound trees, so valid inequalities that are as strong as possible, that is, facet-defining, translate directly into smaller root gaps.

Del Pia and Khajavirad introduced running intersection inequalities in The running intersection relaxation of the multilinear polytope (Math. Oper. Res. 46 (2021) 1008–1037, doi:10.1287/moor.2021.1121). They generalize the flower inequalities of the same authors (SIAM J. Optim. 2018), which are facet-defining when the support hypergraph is γ\gammaγ-acyclic, to support hypergraphs that are not. This mission formalizes the paper's sufficient condition for a running intersection inequality to define a facet (§2.4, Proposition 4) together with the structural results it rests on.

Timeline. Crama (1993) studied the standard linearization; Del Pia and Khajavirad (2017, Math. Oper. Res.) proved that the multilinear polytope is full dimensional and gave lifting theorems for facets; flower inequalities followed in 2018; running intersection inequalities and the facet conditions formalized here appeared in 2021.

Setting

A hypergraph G=(V,E)G=(V,E)G=(V,E) has a finite node set VVV and a set EEE of subsets of VVV of cardinality at least two. Its multilinear set is

SG={z∈{0,1}V+E: ze=∏v∈ezv  ∀e∈E},\mathcal S_G=\Big\{z\in\{0,1\}^{V+E}:\ z_e=\prod_{v\in e}z_v\ \ \forall e\in E\Big\},SG​={z∈{0,1}V+E: ze​=v∈e∏​zv​  ∀e∈E},

and its multilinear polytope is MPG=conv⁡SG\mathrm{MP}_G=\operatorname{conv}\mathcal S_GMPG​=convSG​.

A multiset FFF of sets has the running intersection property if its elements can be ordered p1,…,pmp_1,\dots,p_mp1​,…,pm​ so that each pk∩(p1∪⋯∪pk−1)p_k\cap(p_1\cup\dots\cup p_{k-1})pk​∩(p1​∪⋯∪pk−1​), k≥2k\ge 2k≥2, is contained in a single earlier pjp_jpj​. Such an ordering defines N(p1)=∅N(p_1)=\emptysetN(p1​)=∅ and N(pk)=pk∩⋃i<kpiN(p_k)=p_k\cap\bigcup_{i<k}p_iN(pk​)=pk​∩⋃i<k​pi​. A leaf of FFF is an element that comes last in some such ordering.

Fix an edge e0e_0e0​ (the center) and edges eke_kek​, k∈Kk\in Kk∈K, adjacent to e0e_0e0​ (the neighbors), such that the multiset E~={e0∩ek:k∈K}\tilde E=\{e_0\cap e_k:k\in K\}E~={e0​∩ek​:k∈K} has the running intersection property. Choose an ordering of E~\tilde EE~, and for each kkk with N(e0∩ek)≠∅N(e_0\cap e_k)\neq\emptysetN(e0​∩ek​)=∅ a node uk∈N(e0∩ek)u_k\in N(e_0\cap e_k)uk​∈N(e0​∩ek​). The running intersection inequality is

−∑k∈K: N(e0∩ek)≠∅zuk+∑v∈e0∖⋃kekzv+∑k∈Kzek−ze0 ≤ ω−1,-\sum_{k\in K:\,N(e_0\cap e_k)\neq\emptyset} z_{u_k}+\sum_{v\in e_0\setminus\bigcup_{k}e_k} z_v+\sum_{k\in K}z_{e_k}-z_{e_0}\ \le\ \omega-1,−k∈K:N(e0​∩ek​)=∅∑​zuk​​+v∈e0​∖⋃k​ek​∑​zv​+k∈K∑​zek​​−ze0​​ ≤ ω−1,

where ω\omegaω is the number of connected components of the hypergraph G~=(e0,E~)\tilde G=(e_0,\tilde E)G~=(e0​,E~). Its support hypergraph GGG has node set e0∪⋃keke_0\cup\bigcup_k e_ke0​∪⋃k​ek​ and edge set {e0}∪{ek:k∈K}\{e_0\}\cup\{e_k:k\in K\}{e0​}∪{ek​:k∈K}. An inequality defines a facet of MPG\mathrm{MP}_GMPG​ if it is valid and the face it cuts out is nonempty of dimension dim⁡MPG−1\dim\mathrm{MP}_G-1dimMPG​−1.

Formalization targets

Goal: Proposition 4 (p. 1016)

Assume

  1. ∣e0∩ek∣≥2|e_0\cap e_k|\ge 2∣e0​∩ek​∣≥2 for every k∈Kk\in Kk∈K;
  2. for every K′⊆KK'\subseteq KK′⊆K with e0∩⋂k∈K′ek≠∅e_0\cap\bigcap_{k\in K'}e_k\neq\emptysete0​∩⋂k∈K′​ek​=∅:  e0∩(ei∖⋃k∈K′∖{i}ek)≠∅\ e_0\cap\big(e_i\setminus\bigcup_{k\in K'\setminus\{i\}}e_k\big)\neq\emptyset e0​∩(ei​∖⋃k∈K′∖{i}​ek​)=∅ for all i∈K′i\in K'i∈K′;
  3. each nonempty N(e0∩ek)N(e_0\cap e_k)N(e0​∩ek​) meets U={uk:N(e0∩ek)≠∅}U=\{u_k: N(e_0\cap e_k)\neq\emptyset\}U={uk​:N(e0​∩ek​)=∅} in exactly one node.

Then the running intersection inequality defines a facet of MPG\mathrm{MP}_GMPG​ for its support hypergraph GGG.

Milestones

Lemma 1 (p. 1011, leaves and reorderings), Lemma 2 (p. 1011, components =n0+∣{e:N(e)=∅}∣=n_0+|\{e:N(e)=\emptyset\}|=n0​+∣{e:N(e)=∅}∣), Proposition 1 (p. 1012, validity), and the full dimensionality of MPG\mathrm{MP}_GMPG​ (cited on p. 1016 from [12]):

dim⁡MPG=∣V∣+∣E∣.\dim\mathrm{MP}_G=|V|+|E|.dimMPG​=∣V∣+∣E∣.

Companions

Corollary 2 (p. 1017): ∣e0∩ek∣≥2|e_0\cap e_k|\ge 2∣e0​∩ek​∣≥2 and ∣e0∩ek∩ek′∣≤1|e_0\cap e_k\cap e_{k'}|\le 1∣e0​∩ek​∩ek′​∣≤1 for k≠k′k\neq k'k=k′ suffice. Proposition 2 and Corollary 1 (pp. 1013–1014): the multiset {N(e)}\{N(e)\}{N(e)}, hence the system of inequalities, does not depend on the ordering. Proposition 3 (p. 1014): necessary conditions for a facet of the running intersection relaxation MPGRI\mathrm{MP}^{\mathrm{RI}}_GMPGRI​.

Significance

Proposition 4 certifies a large family of facets of multilinear polytopes whose support hypergraphs contain cycles that flower inequalities cannot handle. Together with the lifting theorems of Del Pia and Khajavirad (2017), it yields sufficient conditions under which these inequalities define facets of the multilinear polytope of general hypergraphs (p. 1015); §2.5 of the paper reports that running intersection cuts reduce BARON's root-node relaxation gap on image-restoration instances. Proposition 3 and Example 1 show that condition 1 is necessary while conditions 2–3 are not, so the result is a sufficient condition and not a characterization.

The results are proved in the paper; none is formalized. A Lean development would provide the first machine-checked library for the running intersection property (orderings, leaves, the multiset of NNN-sets), the connected-component count of Lemma 2, and the multilinear polytope as a convex hull with an exact dimension count, all reusable for the companion missions on the tightness of MPGRI\mathrm{MP}^{\mathrm{RI}}_GMPGRI​ for kite-free β\betaβ-acyclic hypergraphs and its strictness otherwise.

Difficulty

Validity (Proposition 1) decomposes over the connected components of G~\tilde GG~; the facet claim does not, because a face of codimension one is a global object and the neighbors of different components, and the nodes uku_kuk​ shared between neighbors, interact in it. Conditions 2 and 3 are exactly where this interaction is controlled, and Example 1 of the paper shows that dropping condition 3 does not destroy facetness, so no simple equivalence is available to fall back on. Formally, three nontrivial ingredients have no Mathlib counterpart: the combinatorics of running intersection orderings (Lemma 1, Proposition 2), the count of connected components of a hypergraph with loops and parallel edges (Lemma 2), and the affine dimension of a convex hull of 0/1 points in a space indexed by nodes and edges.

Formalization scope

Nodes form a finite type α; every hypergraph's space RV+E\mathbb R^{V+E}RV+E lives inside α ⊕ Finset α → ℝ, with the coordinates outside V+EV+EV+E pinned to 000 in the definition of SG\mathcal S_GSG​, so dim⁡MPG=∣V∣+∣E∣\dim\mathrm{MP}_G=|V|+|E|dimMPG​=∣V∣+∣E∣ is meaningful. Orderings are lists indexed from 000. The number of components is that of the 2-section graph on the node set. An inequality is given by its data: center, a set KKK of distinct neighbors other than e0e_0e0​, an ordering of KKK whose traces form a running intersection ordering of E~\tilde EE~, and the choice k↦ukk\mapsto u_kk↦uk​; the first sum runs over neighbors, so a node chosen for two neighbors has coefficient −2-2−2. "Defines a facet" requires a nonempty face and compares dimensions as dim⁡F+1=dim⁡P\dim F+1=\dim PdimF+1=dimP, so the empty face can never qualify and no truncated subtraction occurs. The facet statement is about MPG\mathrm{MP}_GMPG​ for the support hypergraph only, not for a larger host hypergraph, which would require the lifting theorems. The reading k≠k′k\neq k'k=k′ is used where the page's "for all k,k′k,k'k,k′" would otherwise be contradictory (Corollary 2, Proposition 3(i)); Lemma 2 is stated for lists of nonempty sets, which covers its use on G~\tilde GG~.

Contributions welcome: proofs of the milestones, a reusable library for the running intersection property and α\alphaα-acyclic hypergraphs, and the full-dimensionality proof.

Selected references

  • A. Del Pia, A. Khajavirad, The running intersection relaxation of the multilinear polytope, Math. Oper. Res. 46(3):1008–1037, 2021. https://doi.org/10.1287/moor.2021.1121
  • A. Del Pia, A. Khajavirad, A polyhedral study of binary polynomial programs, Math. Oper. Res. 42(2):389–410, 2017. https://doi.org/10.1287/moor.2016.0804
  • A. Del Pia, A. Khajavirad, The multilinear polytope for acyclic hypergraphs, SIAM J. Optim. 28(2):1049–1076, 2018. https://doi.org/10.1137/16M1095998
  • C. Beeri, R. Fagin, D. Maier, M. Yannakakis, On the desirability of acyclic database schemes, J. ACM 30(3):479–513, 1983. https://doi.org/10.1145/2402.322389
  • Y. Crama, Concave extensions for nonlinear 0–1 maximization problems, Math. Programming 61:53–60, 1993. https://doi.org/10.1007/BF01582138
6 thms1 active userReviewed
Operations ResearchProbabilityTheoretical Computer Science·Captain: mikedeng1

Algorithms for Online Matching, Assortment, and Pricing with Tight Weight-Dependent Competitive Ratios II: In the Deterministic Case Multi-price Ranking Is minᵢ F(𝒫ᵢ)-CompetitiveResearch Paper

Motivation

Online matching models a platform that must commit resources to requests as they arrive, without knowing the future. In revenue management the resources are items in limited supply and the requests are customers, and the firm also chooses the price. Ma and Simchi-Levi (arXiv:1905.04770, Oper. Res. 2020, DOI 10.1287/opre.2019.1957) give algorithms whose worst-case guarantee depends on the set of prices each item can be sold at, and show that the guarantee is tight.

Timeline of the deterministic (matching) case:

  • 1990. Karp, Vazirani and Vazirani introduce RANKING for unweighted online bipartite matching: each item gets a random rank, and each arriving customer takes the best-ranked available neighbour. Its competitive ratio is 1−1/e1-1/e1−1/e, which is optimal.
  • 2011. Aggarwal, Goel, Karande and Mehta extend RANKING to vertex-weighted matching, where each item has a single fixed price, with the same ratio 1−1/e1-1/e1−1/e.
  • 2013. Devanur, Jain and Kleinberg give a randomized primal–dual proof of these results with continuous seeds Wi∼U[0,1]W_i\sim U[0,1]Wi​∼U[0,1].
  • 2019. Ma and Simchi-Levi allow each item a set of prices. Their Multi-price Ranking ranks items by a price-dependent value function, and it is min⁡iF(Pi)\min_iF(\mathcal P_i)mini​F(Pi​)-competitive. F(P)F(\mathcal P)F(P) depends only on the ratios of the prices; it equals 1−1/e1-1/e1−1/e for one price and lies in [1−e−1/m,1−1/e][1-e^{-1/m},1-1/e][1−e−1/m,1−1/e] for mmm prices (p. 14).

Setting

A firm sells nnn items, one unit of each. Item iii has a price set Pi={ri(1)<⋯<ri(mi)}\mathcal P_i=\{r_i^{(1)}<\dots<r_i^{(m_i)}\}Pi​={ri(1)​<⋯<ri(mi​)​} with ri(1)>0r_i^{(1)}>0ri(1)​>0, and ri(0):=0r_i^{(0)}:=0ri(0)​:=0. Customers t=1,…,Tt=1,\dots,Tt=1,…,T arrive one at a time. In the deterministic case, customer ttt either would or would not buy item iii at price jjj: pt,i(j)∈{0,1}p^{(j)}_{t,i}\in\{0,1\}pt,i(j)​∈{0,1}. Let jt,ij_{t,i}jt,i​ be the largest jjj with pt,i(j)=1p^{(j)}_{t,i}=1pt,i(j)​=1, or 000 if there is none. Assigning item iii to customer ttt earns ri(jt,i)r_i^{(j_{t,i})}ri(jt,i​)​. An online algorithm sees ptp_{t}pt​ only when customer ttt arrives.

The benchmark is the LP (5): maximize ∑t,i,jpt,i(j)ri(j)xt,i(j)\sum_{t,i,j}p^{(j)}_{t,i}r_i^{(j)}x^{(j)}_{t,i}∑t,i,j​pt,i(j)​ri(j)​xt,i(j)​ subject to ∑t,jpt,i(j)xt,i(j)≤ki\sum_{t,j}p^{(j)}_{t,i}x^{(j)}_{t,i}\le k_i∑t,j​pt,i(j)​xt,i(j)​≤ki​, ∑i,jxt,i(j)≤1\sum_{i,j}x^{(j)}_{t,i}\le1∑i,j​xt,i(j)​≤1 and x≥0x\ge0x≥0, with optimal value OPT\mathrm{OPT}OPT. Its dual (19) minimizes ∑ikiyi+∑tzt\sum_ik_iy_i+\sum_tz_t∑i​ki​yi​+∑t​zt​ subject to pt,i(j)yi+zt≥pt,i(j)ri(j)p^{(j)}_{t,i}y_i+z_t\ge p^{(j)}_{t,i}r_i^{(j)}pt,i(j)​yi​+zt​≥pt,i(j)​ri(j)​ and y,z≥0y,z\ge0y,z≥0. An algorithm is ccc-competitive if E[ALG]≥c⋅OPT\mathbb E[\mathrm{ALG}]\ge c\cdot\mathrm{OPT}E[ALG]≥c⋅OPT for every arrival sequence.

Booking limits. By Proposition 1 there are unique positive α(1),…,α(m)\alpha^{(1)},\dots,\alpha^{(m)}α(1),…,α(m) that sum to 111 and satisfy

1−e−α(1)=1−e−α(j)1−r(j−1)/r(j)(j=2,…,m).1-e^{-\alpha^{(1)}}=\frac{1-e^{-\alpha^{(j)}}}{1-r^{(j-1)}/r^{(j)}}\quad (j=2,\dots,m).1−e−α(1)=1−r(j−1)/r(j)1−e−α(j)​(j=2,…,m).

Set F(P)=1−e−α(1)F(\mathcal P)=1-e^{-\alpha^{(1)}}F(P)=1−e−α(1) and L(j)=α(1)+⋯+α(j)L^{(j)}=\alpha^{(1)}+\dots+\alpha^{(j)}L(j)=α(1)+⋯+α(j). For w∈[0,1]w\in[0,1]w∈[0,1] let ℓ(w)\ell(w)ℓ(w) be the segment containing www. The value function is

ΦP(w)=r(ℓ−1)+(r(ℓ)−r(ℓ−1))e w−L(ℓ−1)−1eα(ℓ)−1,ℓ=ℓ(w).\Phi_{\mathcal P}(w)=r^{(\ell-1)}+\bigl(r^{(\ell)}-r^{(\ell-1)}\bigr)\frac{e^{\,w-L^{(\ell-1)}}-1}{e^{\alpha^{(\ell)}}-1},\qquad \ell=\ell(w).ΦP​(w)=r(ℓ−1)+(r(ℓ)−r(ℓ−1))eα(ℓ)−1ew−L(ℓ−1)−1​,ℓ=ℓ(w).

It increases continuously from 000 to r(m)r^{(m)}r(m) and equals r(j)r^{(j)}r(j) at L(j)L^{(j)}L(j).

Multi-price Ranking (Algorithm 2). Each item draws an independent seed Wi∼U[0,1]W_i\sim U[0,1]Wi​∼U[0,1]. Customer ttt is assigned an available item maximizing ri(jt,i)−ΦPi(Wi)r_i^{(j_{t,i})}-\Phi_{\mathcal P_i}(W_i)ri(jt,i​)​−ΦPi​​(Wi​), provided the maximum is strictly positive. Ties are broken arbitrarily. The analysis uses dual variables: when iii is assigned to ttt, Zt=ri(jt,i)−ΦPi(Wi)Z_t=r_i^{(j_{t,i})}-\Phi_{\mathcal P_i}(W_i)Zt​=ri(jt,i​)​−ΦPi​​(Wi​) and Yi=ΦPi′(Wi)Y_i=\Phi'_{\mathcal P_i}(W_i)Yi​=ΦPi​′​(Wi​); all other dual variables are 000.

Formalization targets

Goal: Theorem 2 (p. 16)

For every setup in the deterministic case with ki=1k_i=1ki​=1, every arrival sequence, every tie-breaking rule and every feasible xxx of (5),

min⁡iF(Pi)⋅∑t,i,jpt,i(j)ri(j)xt,i(j) ≤ EW[ALG].\min_iF(\mathcal P_i)\cdot\sum_{t,i,j}p^{(j)}_{t,i}r_i^{(j)}x^{(j)}_{t,i}\ \le\ \mathbb E_{W}\bigl[\mathrm{ALG}\bigr].imin​F(Pi​)⋅t,i,j∑​pt,i(j)​ri(j)​xt,i(j)​ ≤ EW​[ALG].

Milestones

  1. Weak duality for (5)/(19) (p. 19).
  2. Lemma 2 (p. 22): if item iii is assigned to customer ttt, then (1−e−αi(1))(Yi+Zt)≤ri(jt,i)(1-e^{-\alpha_i^{(1)}})(Y_i+Z_t)\le r_i^{(j_{t,i})}(1−e−αi(1)​)(Yi​+Zt​)≤ri(jt,i​)​.
  3. Dominance claim (App. C, pp. 45–46): let ZcritZ^{\mathrm{crit}}Zcrit be the pseudorevenue at ttt of the run without item iii. If Wi∈[0,Wcrit)W_i\in[0,W^{\mathrm{crit}})Wi​∈[0,Wcrit), then item iii is matched by time ttt in the run with item iii.
  4. Monotonicity claim (App. C, p. 46): Zt≥ZcritZ_t\ge Z^{\mathrm{crit}}Zt​≥Zcrit for every value of WiW_iWi​.
  5. Lemma 3 (p. 22): yi=E[Yi]y_i=\mathbb E[Y_i]yi​=E[Yi​], zt=E[Zt]z_t=\mathbb E[Z_t]zt​=E[Zt​] is feasible for (19).

Significance

The theorem shows that one randomized rule achieves the optimal guarantee for every price set, and that the guarantee depends only on the price ratios. For one price, F=1−1/eF=1-1/eF=1−1/e and the classical vertex-weighted result is recovered. With several prices the benchmark (5) is stronger, because the offline optimum may also choose prices, and the guarantee degrades gracefully: F(P)F(\mathcal P)F(P) lies in [1−e−1/m,1−1/e][1-e^{-1/m},1-1/e][1−e−1/m,1−1/e] and depends only on how far apart the prices are. Theorem 3 of the same paper shows that no online algorithm does better than F(P)F(\mathcal P)F(P) when every item has price set P\mathcal PP, so the ratio is tight.

The results are proved in the paper, in §4 and App. C. As far as the platform's records show, they have not been formalized. A formalization adds three things on top of the paper. It machine-checks a randomized primal–dual argument whose expectations are integrals over [0,1]n[0,1]^n[0,1]n. It makes precise the measure-zero conventions the paper uses: ties in (23), and Φ′\Phi'Φ′ at segment borders. It also gives a Lean model of a ranking algorithm with continuous seeds that sees only the present customer, which could be reused for related online algorithms.

Difficulty

The pointwise part (Lemma 2) is a direct computation from the booking-limit identity (7). The difficulty is Lemma 3. The constraint for a pair (t,i)(t,i)(t,i) needs E[Yi+Zt]≥ri(jt,i)\mathbb E[Y_i+Z_t]\ge r_i^{(j_{t,i})}E[Yi​+Zt​]≥ri(jt,i​)​, but on a single sample path Yi+ZtY_i+Z_tYi​+Zt​ can be far below this: item iii may be taken by an earlier customer, or customer ttt may take another item. The bound only holds after conditioning on the other seeds and comparing two coupled runs, one with item iii and one without. That comparison needs an invariant that holds at every step: the set of items still available in the run with iii contains the set in the run without iii. The invariant fails when ties are broken differently in the two runs, so the measure-zero set of ties must be handled explicitly. The conditional expectation of YiY_iYi​ is then an integral of Φ′\Phi'Φ′, which is discontinuous at the segment borders.

Formalization scope

  • Indexing. Items are Fin n and customers Fin T, both 0-based. Prices are 1-based within an item, with pr r 0 = 0 for r(0)=0r^{(0)}=0r(0)=0. Data at indices outside [1,mi][1,m_i][1,mi​] are never read.
  • Standing assumptions (p. 11), passed as hypotheses: every price set has mi≥1m_i\ge1mi​≥1 and 0<ri(1)<⋯<ri(mi)0<r_i^{(1)}<\dots<r_i^{(m_i)}0<ri(1)​<⋯<ri(mi​)​; the booking limits are positive, sum to 111 and satisfy (7) (IsBookingLimits, unique by Proposition 1); p∈{0,1}p\in\{0,1\}p∈{0,1} on [1,mi][1,m_i][1,mi​]; ki=1k_i=1ki​=1. Weak duality is stated for any p∈[0,1]p\in[0,1]p∈[0,1] and any inventories.
  • Algorithm. The step at customer ttt reads only ptp_tpt​, the seeds and the set of available items. Tie-breaking is any rule returning a maximizer of (23), universally quantified. A run can start from a subset of the items, which gives the run "with item iii removed".
  • Expectations are Lebesgue integrals over the product of nnn uniform laws on [0,1][0,1][0,1]. No integrability hypothesis is added: the integrands are bounded, and measurable outside the null set of ties.
  • Competitive ratio. It is stated multiplied out, for every feasible xxx of (5), and needs n≥1n\ge1n≥1 for the minimum (with no items both sides are 000). This rules out the trivializing readings E[ALG]/OPT\mathbb E[\mathrm{ALG}]/\mathrm{OPT}E[ALG]/OPT with x/0=0x/0=0x/0=0, and an infimum over arrival sequences. The algorithm is fixed as Algorithm 2; it is not "any algorithm with a property".
  • Added or changed hypotheses.
    • Lemma 2 is claimed for every seed vector in [0,1]n[0,1]^n[0,1]n, with Φ′\Phi'Φ′ the right derivative at a border. This implies the paper's "w.p.1".
    • The two claims assume no ties, the paper's "we ignore measure-zero events".
    • Dominance is written as ΦPi(Wi)<ri(j)−Zcrit\Phi_{\mathcal P_i}(W_i)<r_i^{(j)}-Z^{\mathrm{crit}}ΦPi​​(Wi​)<ri(j)​−Zcrit. This is equivalent to Wi<WcritW_i<W^{\mathrm{crit}}Wi​<Wcrit because Φ\PhiΦ is a strictly increasing bijection [0,Li(j)]→[0,ri(j)][0,L_i^{(j)}]\to[0,r_i^{(j)}][0,Li(j)​]→[0,ri(j)​].
  • Infrastructure. A solver needs: monotonicity and continuity of ΦP\Phi_{\mathcal P}ΦP​ and the integral ∫0wΦ′=Φ(w)\int_0^{w}\Phi'=\Phi(w)∫0w​Φ′=Φ(w); measurability of the run in WWW; the fact that ties are null under the product of uniform laws; and conditioning via Fubini on [0,1]n[0,1]^n[0,1]n. These pieces are reusable for other randomized primal–dual analyses. The paper's proofs are in §4 (p. 23) and App. C (pp. 45–46). Proofs of any milestone, and lemmas on ΦP\Phi_{\mathcal P}ΦP​, are welcome.

Selected references

  • W. Ma, D. Simchi-Levi, Algorithms for Online Matching, Assortment, and Pricing with Tight Weight-dependent Competitive Ratios, arXiv:1905.04770v1, 2019; Operations Research 68(6), 2020. https://arxiv.org/abs/1905.04770 · https://doi.org/10.1287/opre.2019.1957
  • R. M. Karp, U. V. Vazirani, V. V. Vazirani, An optimal algorithm for on-line bipartite matching, STOC 1990. https://doi.org/10.1145/100216.100262
  • G. Aggarwal, G. Goel, C. Karande, A. Mehta, Online vertex-weighted bipartite matching and single-bid budgeted allocations, SODA 2011. https://arxiv.org/abs/1007.1271
  • N. R. Devanur, K. Jain, R. D. Kleinberg, Randomized primal-dual analysis of RANKING for online bipartite matching, SODA 2013. https://doi.org/10.1137/1.9781611973105.7
9 thms1 active userReviewed
Operations ResearchProbabilityTheoretical Computer Science·Captain: mikedeng1

Algorithms for Online Matching, Assortment, and Pricing with Tight Weight-Dependent Competitive Ratios I: Multi-price Balance Is minᵢ F̃ᵢ-Competitive, F̃ᵢ Bounded via F(𝒫ᵢ), G(𝒫ᵢ)/2 and kᵢResearch Paper

Motivation

Online matching and its revenue-management variants model a firm that allocates limited inventory to customers who arrive one at a time, without knowing who comes next. Classical results in this area assume each item has a single price, or weight. For a single price and large inventories, the BALANCE algorithm of Kalyanasundaram and Pruhs (2000) and its generalizations by Mehta, Saberi, Vazirani and Vazirani (2007), Buchbinder, Jain and Naor (2007) and Golrezaei, Nazerzadeh and Rusmevichientong (2014) achieve the competitive ratio 1−1/e1-1/e1−1/e, which is best possible.

Firms that can also choose a price for each sale face a different problem. Selling an item cheaply now uses inventory that a later customer might have paid more for. Ma and Simchi-Levi (2020) study this setting. Each item can be offered at one of finitely many prices, and the attainable competitive ratio depends on those prices. Their Multi-price Balance algorithm attains, as inventories grow, a ratio that their Theorem 3 shows no online algorithm can beat. This mission formalizes their Theorem 1, the guarantee of Multi-price Balance for finite inventories.

Setting

A firm sells items iii from a finite set. Item iii starts with ki≥1k_i\ge1ki​≥1 units and has a price set Pi\mathcal P_iPi​ of mi≥1m_i\ge1mi​≥1 prices 0<ri(1)<⋯<ri(mi)0<r_i^{(1)}<\dots<r_i^{(m_i)}0<ri(1)​<⋯<ri(mi​)​, with ri(0):=0r_i^{(0)}:=0ri(0)​:=0. Customers t=1,…,Tt=1,\dots,Tt=1,…,T arrive one by one. On arrival of customer ttt the firm learns pt,i(j)∈[0,1]p^{(j)}_{t,i}\in[0,1]pt,i(j)​∈[0,1], the probability that ttt buys item iii at price jjj. It offers at most one item, at one price. The customer accepts with that probability, and a sale earns ri(j)r_i^{(j)}ri(j)​ and uses one unit. An online algorithm decides using only the setup, the past, and the present customer's probabilities.

The benchmark is the linear program (5):

OPT=max⁡∑t,i,jpt,i(j)ri(j)xt,i(j)  s.t.  ∑t,jpt,i(j)xt,i(j)≤ki,  ∑i,jxt,i(j)≤1,  x≥0.\mathrm{OPT}=\max\sum_{t,i,j}p^{(j)}_{t,i}r_i^{(j)}x^{(j)}_{t,i}\ \ \text{s.t.}\ \ \sum_{t,j}p^{(j)}_{t,i}x^{(j)}_{t,i}\le k_i,\ \ \sum_{i,j}x^{(j)}_{t,i}\le1,\ \ x\ge0 .OPT=maxt,i,j∑​pt,i(j)​ri(j)​xt,i(j)​  s.t.  t,j∑​pt,i(j)​xt,i(j)​≤ki​,  i,j∑​xt,i(j)​≤1,  x≥0.

An algorithm is ccc-competitive on a setup if E[ALG]≥c⋅OPT\mathbb E[\mathrm{ALG}]\ge c\cdot\mathrm{OPT}E[ALG]≥c⋅OPT for every arrival sequence (6).

For a price set P\mathcal PP, the booking limits α(1),…,α(m)\alpha^{(1)},\dots,\alpha^{(m)}α(1),…,α(m) are the unique positive numbers summing to 111 with

1−e−α(1)=1−e−α(j)1−r(j−1)/r(j)(j=2,…,m),1-e^{-\alpha^{(1)}}=\frac{1-e^{-\alpha^{(j)}}}{1-r^{(j-1)}/r^{(j)}}\qquad(j=2,\dots,m),1−e−α(1)=1−r(j−1)/r(j)1−e−α(j)​(j=2,…,m),

and the values σ(j)\sigma^{(j)}σ(j) are the unique positive numbers summing to 111 with σ(1)=σ(j)/(1−r(j−1)/r(j))\sigma^{(1)}=\sigma^{(j)}/(1-r^{(j-1)}/r^{(j)})σ(1)=σ(j)/(1−r(j−1)/r(j)) (Proposition 1). Write F(P)=1−e−α(1)F(\mathcal P)=1-e^{-\alpha^{(1)}}F(P)=1−e−α(1) and G(P)=σ(1)G(\mathcal P)=\sigma^{(1)}G(P)=σ(1), and L(j)=α(1)+⋯+α(j)L^{(j)}=\alpha^{(1)}+\dots+\alpha^{(j)}L(j)=α(1)+⋯+α(j).

Multi-price Balance (Algorithm 1) gives each item random segment borders 0=L~i(0)≤⋯≤L~i(mi)=10=\tilde L_i^{(0)}\le\dots\le\tilde L_i^{(m_i)}=10=L~i(0)​≤⋯≤L~i(mi​)​=1 in {0,1/ki,…,1}\{0,1/k_i,\dots,1\}{0,1/ki​,…,1} and a nondecreasing value function Φ~i\tilde\Phi_iΦ~i​ on that grid with Φ~i(0)=0\tilde\Phi_i(0)=0Φ~i​(0)=0. Both are drawn from a randomized procedure, independently across items. With NiN_iNi​ units of item iii sold, customer ttt is offered a pair (i,j)(i,j)(i,j) maximizing

pt,i(j)(Φ~i(L~i(j))−Φ~i(Ni/ki))p^{(j)}_{t,i}\big(\tilde\Phi_i(\tilde L_i^{(j)})-\tilde\Phi_i(N_i/k_i)\big)pt,i(j)​(Φ~i​(L~i(j)​)−Φ~i​(Ni​/ki​))

when this maximum is positive. Two procedures are used. Definition 3 draws one uniform seed WWW and rounds every L(j)kL^{(j)}kL(j)k up or down with it, then sets Φ~\tilde\PhiΦ~ by formula (20). The procedure (45), for k=1k=1k=1, draws configuration ddd with probability σ(d)\sigma^{(d)}σ(d) and sets Φ~(1)=r(d)/σ(1)\tilde\Phi(1)=r^{(d)}/\sigma^{(1)}Φ~(1)=r(d)/σ(1).

Formalization targets

Goal: Theorem 1 (p. 16)

Give each item a mode. A perturbed item uses Definition 3. A split item is replaced by kik_iki​ single-unit copies, each using (45). With

bi=F(Pi)(1+ki)(e1/ki−1) (i),bi=G(Pi)2 (ii, split),bi=1−1/e(1+ki)(1−e−1/ki) (iii, if ∣Pi∣=1),b_i=\frac{F(\mathcal P_i)}{(1+k_i)(e^{1/k_i}-1)}\ \text{(i)},\qquad b_i=\frac{G(\mathcal P_i)}{2}\ \text{(ii, split)},\qquad b_i=\frac{1-1/e}{(1+k_i)(1-e^{-1/k_i})}\ \text{(iii, if }|\mathcal P_i|=1),bi​=(1+ki​)(e1/ki​−1)F(Pi​)​ (i),bi​=2G(Pi​)​ (ii, split),bi​=(1+ki​)(1−e−1/ki​)1−1/e​ (iii, if ∣Pi​∣=1),

Multi-price Balance with those modes satisfies, for every arrival sequence, every tie-breaking rule and every feasible xxx of (5),

(min⁡ibi)∑t,i,jpt,i(j)ri(j)xt,i(j)≤E[ALG].\Big(\min_i b_i\Big)\sum_{t,i,j}p^{(j)}_{t,i}r_i^{(j)}x^{(j)}_{t,i}\le\mathbb E[\mathrm{ALG}].(imin​bi​)t,i,j∑​pt,i(j)​ri(j)​xt,i(j)​≤E[ALG].

Milestones

  1. Weak duality for (5) and its dual (19) (p. 19).
  2. Theorem 4 (p. 19). If every item's procedure satisfies
ki(Φ~i(N+1ki)−Φ~i(Nki))+Φ~i(L~i(j))−Φ~i(Nki)≤ri(j)c  (17),E[Φ~i(L~i(j))]≥ri(j)  (18),k_i\big(\tilde\Phi_i(\tfrac{N+1}{k_i})-\tilde\Phi_i(\tfrac N{k_i})\big)+\tilde\Phi_i(\tilde L_i^{(j)})-\tilde\Phi_i(\tfrac N{k_i})\le\frac{r_i^{(j)}}c\ \ (17),\qquad \mathbb E[\tilde\Phi_i(\tilde L_i^{(j)})]\ge r_i^{(j)}\ \ (18),ki​(Φ~i​(ki​N+1​)−Φ~i​(ki​N​))+Φ~i​(L~i(j)​)−Φ~i​(ki​N​)≤cri(j)​​  (17),E[Φ~i​(L~i(j)​)]≥ri(j)​  (18),

then Algorithm 1 is ccc-competitive. 3. Proposition 3 (p. 21). E[L~(j)]=L(j)\mathbb E[\tilde L^{(j)}]=L^{(j)}E[L~(j)]=L(j), and ∣(L~(j)−L~(j′))−(L(j)−L(j′))∣≤1/k|(\tilde L^{(j)}-\tilde L^{(j')})-(L^{(j)}-L^{(j')})|\le1/k∣(L~(j)−L~(j′))−(L(j)−L(j′))∣≤1/k. 4. (35) (p. 39). r(j−1)/r(j)≤e−α(j)r^{(j-1)}/r^{(j)}\le e^{-\alpha^{(j)}}r(j−1)/r(j)≤e−α(j). 5. Theorem 5 (p. 22). Definition 3 satisfies (17)–(18) with c=F(P)/((1+k)(e1/k−1))c=F(\mathcal P)/((1+k)(e^{1/k}-1))c=F(P)/((1+k)(e1/k−1)), and with c=F(P)/((1+k)(1−e−1/k))c=F(\mathcal P)/((1+k)(1-e^{-1/k}))c=F(P)/((1+k)(1−e−1/k)) when m=1m=1m=1. 6. (45) (p. 44). For k=1k=1k=1, the procedure (45) satisfies (17)–(18) with c=σ(1)/2c=\sigma^{(1)}/2c=σ(1)/2.

Significance

For a single price, bound (iii) improves the dependence on kik_iki​ of the guarantee of Golrezaei et al. (2014). For several prices, bound (i) tends to F(Pi)F(\mathcal P_i)F(Pi​) as ki→∞k_i\to\inftyki​→∞ (Corollary 1). Mission III of this series formalizes Theorem 3, by which no online algorithm beats F(P)F(\mathcal P)F(P), so Theorem 1 makes that ratio tight. Bound (ii) uses a different procedure and gives a guarantee for small inventories that does not rely on perturbing the ideal value function. The paper's Section 6 applies the same framework to the Adwords problem and to assortment problems with multiple prices.

The paper's proofs are in its Appendix B: Theorem 4 by a randomized primal–dual argument, Proposition 3 and Theorem 5 by analysis of the comonotone rounding, (45) by solving a small system, and Theorem 1 by assembling them. None of these results has a machine-checked proof. A complete formalization would yield a reusable, checked primal–dual framework for randomized online allocation: Theorem 4 holds for any family of procedures satisfying (17)–(18).

Difficulty

The primal–dual proof of Theorem 4 sets the dual variables yi=E[Φ~i(NT,i/ki)]y_i=\mathbb E[\tilde\Phi_i(N_{T,i}/k_i)]yi​=E[Φ~i​(NT,i​/ki​)] and zt=E[Zt]z_t=\mathbb E[Z_t]zt​=E[Zt​], where ZtZ_tZt​ is the pseudorevenue. These are expectations over both the initialization and the purchase decisions, and the dual constraint holds only after conditioning on the history and taking expectations. A deterministic value function does not suffice for multiple prices with finite inventory, since the borders L(j)kiL^{(j)}k_iL(j)ki​ are generally not integers. This is why the value functions are random. Rounding each border independently would break the segment-length control. Theorem 5 needs the comonotone bound (22), and an induction over segments with careful bookkeeping of the rounding errors.

Formalization scope

  • Indices. Items form an arbitrary nonempty finite type. Customers are indexed 0,…,T−10,\dots,T-10,…,T−1. Prices are 111-based with r(0)=0r^{(0)}=0r(0)=0. Probabilities pt,i(j)∈[0,1]p^{(j)}_{t,i}\in[0,1]pt,i(j)​∈[0,1], ki≥1k_i\ge1ki​≥1, mi≥1m_i\ge1mi​≥1 and 0<ri(1)<⋯<ri(mi)0<r_i^{(1)}<\dots<r_i^{(m_i)}0<ri(1)​<⋯<ri(mi​)​ are hypotheses.
  • Booking limits. α\alphaα and σ\sigmaσ enter as functions satisfying the defining systems (7) and (8) with positivity and unit sum. By Proposition 1 they are the paper's values.
  • Unit indices. Borders are stored as kL~(j)∈{0,…,k}k\tilde L^{(j)}\in\{0,\dots,k\}kL~(j)∈{0,…,k} and value functions as N↦Φ~(N/k)N\mapsto\tilde\Phi(N/k)N↦Φ~(N/k). In (17) the range of NNN is N<kL~(j)N<k\tilde L^{(j)}N<kL~(j).
  • The algorithm. The offer at customer ttt is computed from that customer's probabilities and the sales counts. The count in (16) is read as min⁡(Ni,ki)\min(N_i,k_i)min(Ni​,ki​), which equals NiN_iNi​ because the algorithm never sells a stocked-out item. Ties are broken by an arbitrary rule returning a maximizer, and every statement holds for every such rule.
  • Expectations. Expected revenue is a finite sum over configurations, weighted by product probabilities, combined with a backward recursion over accept/reject outcomes. Definition 3's configuration probabilities are Lebesgue measures of seed sets.
  • Competitive ratio. The ratio is always stated as c⋅(5a)(x)≤E[ALG]c\cdot(5\mathrm a)(x)\le\mathbb E[\mathrm{ALG}]c⋅(5a)(x)≤E[ALG] for every feasible xxx of the original LP (5). This rules out the trivializations through x/0=0x/0=0x/0=0 when OPT=0\mathrm{OPT}=0OPT=0, through an infimum over arrival sequences, and through a benchmark taken on the split setup instead of the original one.
  • F~i\tilde F_iF~i​ is not used. The printed theorem is phrased through F~i\tilde F_iF~i​, the optimum of the problem (44). The proof never shows that this optimum is attained. Its bound (ii) also comes from splitting the item, not from a procedure feasible for (44). The goal is therefore stated with explicit bounds and modes; choosing the better mode per item gives the printed min⁡imax⁡{… }\min_i\max\{\dots\}mini​max{…}.
  • Not posed. Corollaries 1–2 and the optimality and uniqueness claims about (45).

Weak duality is shared with mission II and (35) with mission III; each mission defines its own copy of the model. Contributions welcome: proofs of the milestones, and of the goal from them.

Selected references

  • W. Ma, D. Simchi-Levi, Algorithms for Online Matching, Assortment, and Pricing with Tight Weight-dependent Competitive Ratios, arXiv:1905.04770v1 (2019); Operations Research 68(6), 2020. https://arxiv.org/abs/1905.04770, https://doi.org/10.1287/opre.2019.1957
  • B. Kalyanasundaram, K. Pruhs, An optimal deterministic algorithm for online b-matching, Theoretical Computer Science 233, 2000. https://doi.org/10.1016/S0304-3975(99)00140-1
  • A. Mehta, A. Saberi, U. Vazirani, V. Vazirani, AdWords and generalized online matching, Journal of the ACM 54(5), 2007. https://doi.org/10.1145/1284320.1284321
  • N. Buchbinder, K. Jain, J. Naor, Online primal-dual algorithms for maximizing ad-auctions revenue, ESA 2007. https://doi.org/10.1007/978-3-540-75520-3_24
  • N. Golrezaei, H. Nazerzadeh, P. Rusmevichientong, Real-time optimization of personalized assortments, Management Science 60(6), 2014. https://doi.org/10.1287/mnsc.2014.1939
11 thms1 active userReviewed
CombinatoricsGraph TheoryOperations Research·Captain: mikedeng1

Shorter Tours by Nicer Ears 3: Every 2-Edge-Connected Graph Has a 2-Edge-Connected Spanning Subgraph with at Most 4/3 LP(G) EdgesResearch Paper

Motivation

Designing a network that survives the failure of any single link is a basic problem of network design: given a graph GGG whose links can be bought, choose as few links as possible so that the chosen subgraph still connects all vertices after any one of its links is removed. This is the minimum 2-edge-connected spanning subgraph problem (2ECSS). It is NP-hard, since the 2-edge-connected spanning subgraphs with ∣V(G)∣|V(G)|∣V(G)∣ edges are exactly the Hamiltonian circuits, and it is a relaxation of the graphic travelling salesman problem: every tour of a 2-edge-connected graph yields a 2ECSS with at most as many edges.

Timeline of approximation ratios for unweighted 2ECSS:

  • taking a spanning tree and doubling it gives ratio 2;
  • Khuller and Vishkin (1994) gave a 3/23/23/2-approximation;
  • Cheriyan, Sebő and Szigeti (2001) improved the ratio to 17/1217/1217/12, using ear-decompositions with few even ears;
  • Sebő and Vygen (2012, Combinatorica 2014) obtained 4/34/34/3, measured against the linear programming relaxation, as part of the same paper that gives 7/57/57/5 for graphic TSP and 3/23/23/2 for connected-TTT-joins (arXiv:1201.1870).

Setting

A graph GGG has finite vertex and edge sets; every edge has two distinct ends, and parallel edges are allowed. For W⊆V(G)W\subseteq V(G)W⊆V(G), δ(W)\delta(W)δ(W) is the set of edges with exactly one end in WWW, and for x∈RE(G)x\in\mathbb R^{E(G)}x∈RE(G) we write x(S)=∑e∈Sxex(S)=\sum_{e\in S}x_ex(S)=∑e∈S​xe​. An edge set F⊆E(G)F\subseteq E(G)F⊆E(G) is 2-edge-connected spanning (a 2ECSS of GGG) if (V(G),F)(V(G),F)(V(G),F) is connected and stays connected after the deletion of any single edge of FFF; GGG is 2-edge-connected if E(G)E(G)E(G) is.

The cut relaxation of 2ECSS is

LP(G)=min⁡{x(E(G)):x∈R≥0E(G), x(δ(W))≥2 for all ∅≠W⊊V(G)}.\mathrm{LP}(G)=\min\Bigl\{x(E(G)) : x\in\mathbb R^{E(G)}_{\ge0},\ x(\delta(W))\ge2\ \text{for all}\ \emptyset\ne W\subsetneq V(G)\Bigr\}.LP(G)=min{x(E(G)):x∈R≥0E(G)​, x(δ(W))≥2 for all ∅=W⊊V(G)}.

Every 2ECSS crosses every cut at least twice, so LP(G)≤OPT2EC(G)\mathrm{LP}(G)\le\mathrm{OPT}_{2EC}(G)LP(G)≤OPT2EC​(G), the minimum size of a 2-edge-connected spanning multi-subgraph.

An ear-decomposition of GGG starts from a single vertex and adds ears P1,…,PkP_1,\dots,P_kP1​,…,Pk​: paths whose two ends, or circuits whose one vertex, lie on the earlier ears, and whose other (internal) vertices in(Pi)\mathrm{in}(P_i)in(Pi​) are new. The ears partition E(G)E(G)E(G). An ear of length lll is even if lll is even, nontrivial if l≥2l\ge2l≥2, and short if l∈{2,3}l\in\{2,3\}l∈{2,3}. A nontrivial ear is pendant if no nontrivial ear has an end among its internal vertices; π\piπ denotes the number of pendant ears. φ(G)\varphi(G)φ(G) is the minimum number of even ears over all ear-decompositions, and Lφ(G)=∣V(G)∣+φ(G)−1L_\varphi(G)=|V(G)|+\varphi(G)-1Lφ​(G)=∣V(G)∣+φ(G)−1.

An ear-decomposition is nice if it has φ(G)\varphi(G)φ(G) even ears, all its short ears are pendant, and internal vertices of different short ears are non-adjacent. The internal vertex sets of its short ears avoiding a set TTT form an eardrum MMM. An earmuff chooses, for some F⊆MF\subseteq MF⊆M, a path PfP_fPf​ with in(Pf)=f\mathrm{in}(P_f)=fin(Pf​)=f for each f∈Ff\in Ff∈F so that the union of their edges is a forest. μ(G,M)\mu(G,M)μ(G,M) is the largest such ∣F∣|F|∣F∣, and Lμ(G,M)=∣V(G)∣−1+∣M∣−μ(G,M)L_\mu(G,M)=|V(G)|-1+|M|-\mu(G,M)Lμ​(G,M)=∣V(G)∣−1+∣M∣−μ(G,M).

Formalization targets

Goal (Theorem 30, p. 22)

For every 2-edge-connected graph GGG there is a 2ECSS F⊆E(G)F\subseteq E(G)F⊆E(G) with

∣F∣ ≤ 43 LP(G).|F|\ \le\ \tfrac43\,\mathrm{LP}(G).∣F∣ ≤ 34​LP(G).

Milestones, in the order of the paper's proof

  1. Proposition 2: OPT(G)≥OPT2EC(G)≥LP(G)≥∣V(G)∣\mathrm{OPT}(G)\ge\mathrm{OPT}_{2EC}(G)\ge\mathrm{LP}(G)\ge|V(G)|OPT(G)≥OPT2EC​(G)≥LP(G)≥∣V(G)∣.
  2. Proposition 4: OPT(G,T)\mathrm{OPT}(G,T)OPT(G,T), OPT2EC(G)\mathrm{OPT}_{2EC}(G)OPT2EC​(G) and LP(G,T)\mathrm{LP}(G,T)LP(G,T) split additively over a cut vertex, so it suffices to treat 2-vertex-connected graphs.
  3. §1, p. 3: a tour in a 2-edge-connected graph gives a 2ECSS with at most as many edges.
  4. Theorem 7 (Frank): some even TTT has τ(G,T)=12Lφ(G)\tau(G,T)=\tfrac12L_\varphi(G)τ(G,T)=21​Lφ​(G).
  5. Theorem 19 (Cheriyan, Sebő, Szigeti): Lφ(G)≤LP(G)L_\varphi(G)\le\mathrm{LP}(G)Lφ​(G)≤LP(G).
  6. Corollary 21: Lμ(G,M)≤LP(G)L_\mu(G,M)\le\mathrm{LP}(G)Lμ​(G,M)≤LP(G).
  7. Lemma 23: a 2-vertex-connected graph has a nice ear-decomposition containing a maximum earmuff.
  8. Theorem 24: a connected-TTT-join, and for T=∅T=\emptysetT=∅ a tour, with at most Lμ(G,M)+12Lφ(G)−πL_\mu(G,M)+\tfrac12L_\varphi(G)-\piLμ​(G,M)+21​Lφ​(G)−π edges.
  9. Claim, p. 23: the nontrivial ears form a 2ECSS with at most 54Lφ(G)+12π\tfrac54L_\varphi(G)+\tfrac12\pi45​Lφ​(G)+21​π edges.

Significance

The theorem gives a polynomial-time 4/34/34/3-approximation for unweighted 2ECSS. Because it compares with LP(G)\mathrm{LP}(G)LP(G) rather than with the optimum, it also bounds the integrality ratio of the cut relaxation on unweighted instances by 4/34/34/3 (the paper's Corollary 31). The two lower bounds LφL_\varphiLφ​ and LμL_\muLμ​ and the construction of Theorem 24 are shared with the paper's 7/57/57/5 result for graphic TSP and its 3/23/23/2 result for connected-TTT-joins, so the milestones of this mission are also steps toward those results.

The theorem is proved in the paper, and to the best of current knowledge none of it is machine-checked: there is no formal development of ear-decompositions, of Frank's even-ear parameter φ(G)\varphi(G)φ(G), or of 2ECSS approximation in Mathlib or on Prove2Me. The work here is formalizing the known proofs. Theorems 7 and 19 rest on classical results that are absent from Mathlib: Frank's theorem on ear-decompositions and the Edmonds–Johnson/Lovász packing of TTT-cuts.

Difficulty

The arithmetic of the final step is short: 54+12⋅16=43\tfrac54+\tfrac12\cdot\tfrac16=\tfrac4345​+21​⋅61​=34​ and 32−16=43\tfrac32-\tfrac16=\tfrac4323​−61​=34​. The difficulty lies in the lower bounds and in the decomposition. A natural first idea is to bound every nontrivial ear by its internal vertices. This fails for 3-ears, whose three edges cover only two new vertices, and the paper handles them by making all short ears pendant and pairwise non-adjacent and then re-choosing them as a maximum earmuff. That step needs a matroid-intersection or forest-representative min-max theorem (Theorem 18), which in turn yields Lμ(G,M)≤LP(G)L_\mu(G,M)\le\mathrm{LP}(G)Lμ​(G,M)≤LP(G). The bound Lφ(G)≤LP(G)L_\varphi(G)\le\mathrm{LP}(G)Lφ​(G)≤LP(G) needs Frank's characterization of φ(G)\varphi(G)φ(G) through minimum TTT-joins together with the half-integral packing of TTT-cuts. None of this machinery exists formally yet.

Formalization scope

  • Graphs. Graphs are multigraphs: an edge type EEE with a map to unordered pairs of distinct vertices of VVV. 2G2G2G has edge type E×{0,1}E\times\{0,1\}E×{0,1}. Connectivity is that of the simple graph of pairs joined by at least one chosen edge. 2-edge-connectivity deletes one edge at a time, so two parallel edges are 2-edge-connected and a single edge is not.
  • Algorithms. All algorithmic statements (Theorems 7, 24, 30 and Lemma 23) are formalized as existence statements for every input; running times are not formalized.
  • LP bounds. "≤c⋅LP(G)\le c\cdot\mathrm{LP}(G)≤c⋅LP(G)" is stated as "≤c⋅x(E(G))\le c\cdot x(E(G))≤c⋅x(E(G)) for every feasible xxx", with the solution chosen before xxx. This is equivalent because LP(G)\mathrm{LP}(G)LP(G) is a minimum over the feasible points. Propositions 2 and 4, which compare optimum values, use the LP value as an infimum over the feasible set; that set is nonempty for connected graphs.
  • Graph parameters. φ(G)\varphi(G)φ(G) and μ(G,M)\mu(G,M)μ(G,M) are an infimum over ear-decompositions and a supremum over earmuffs. Every statement using them assumes 2-edge-connectivity or a given decomposition.
  • 2-vertex-connectivity means 2-edge-connected with no cut vertex. This includes the one-vertex graph and two vertices joined by parallel edges.
  • Added and corrected hypotheses. Proposition 2 adds ∣V(G)∣≥2|V(G)|\ge2∣V(G)∣≥2, since LP(G)=0\mathrm{LP}(G)=0LP(G)=0 for one vertex. Proposition 4 states LP(G,T)=LP(G1,T1)+LP(G2,T2)\mathrm{LP}(G,T)=\mathrm{LP}(G_1,T_1)+\mathrm{LP}(G_2,T_2)LP(G,T)=LP(G1​,T1​)+LP(G2​,T2​), as its proof does; the page prints LP(G,Ti)\mathrm{LP}(G,T_i)LP(G,Ti​).
  • Ruled-out trivializations. The goal is not about simple graphs only, and its subgraph uses each edge of GGG at most once (it is not a multi-subgraph of 2G2G2G). Its feasibility condition is satisfiable: x≡2x\equiv2x≡2 is feasible for every connected GGG. φ\varphiφ and μ\muμ are never evaluated where they take junk values.

Every definition is local to the namespace NicerEars.TwoEC. Reusable beyond this mission:

  • the multigraph setting, TTT-joins and the cut LP;
  • ear-decompositions with φ(G)\varphi(G)φ(G) and pendant ears;
  • eardrums and earmuffs.

Contributions are welcome on any milestone. Especially valuable are Frank's Theorem 7, the TTT-cut packing behind Theorem 19, and the basic fact that a graph with an ear-decomposition is 2-edge-connected.

Selected references

  • A. Sebő, J. Vygen, Shorter tours by nicer ears: 7/5-approximation for graphic TSP, 3/2 for the path version, and 4/3 for two-edge-connected subgraphs, Combinatorica 34 (2014); preprint arXiv:1201.1870v3, 2012. https://arxiv.org/abs/1201.1870v3, https://doi.org/10.1007/s00493-014-2960-3
  • J. Cheriyan, A. Sebő, Z. Szigeti, Improving on the 1.5-approximation of a smallest 2-edge connected spanning subgraph, SIAM J. Discrete Math. 14 (2001) 170–180. https://doi.org/10.1137/S0895480199362071
  • S. Khuller, U. Vishkin, Biconnectivity approximations and graph carvings, J. ACM 41 (1994) 214–235. https://doi.org/10.1145/174652.174654
  • A. Frank, Conservative weightings and ear-decompositions of graphs, Combinatorica 13 (1993) 65–81. https://doi.org/10.1007/BF01202790
14 thms1 active userReviewed
CombinatoricsGraph TheoryOperations Research·Captain: mikedeng1

Shorter Tours by Nicer Ears 2: Every Connected Graph Has a Graphic TSP Tour with at Most 7/5 LP(G) EdgesResearch Paper

Motivation

The graphic TSP is the travelling salesman problem in the shortest-path metric of an unweighted connected graph: visit every vertex of a graph GGG by a closed walk using as few edge traversals as possible. It is the unweighted special case of the metric TSP, and it is the setting in which the 3/2 barrier of Christofides' algorithm (1976) was first broken for general graphs. It is also a natural test bed for the subtour LP: for graphic instances the LP relaxation is the 2-edge-connected subgraph relaxation LP(G)\mathrm{LP}(G)LP(G) below, and the conjectured integrality ratio of 4/3 for metric TSP is open even here.

Timeline of approximation ratios for graphic TSP on general graphs:

  • 1976: Christofides, 3/2 for metric TSP, hence for graphic TSP.
  • 2011: Oveis Gharan, Saberi and Singh, 3/2−ε3/2-\varepsilon3/2−ε for a tiny ε>0\varepsilon>0ε>0, by a probabilistic analysis (FOCS 2011).
  • 2011: Mömke and Svensson, 1.461, with removable pairings (arXiv:1104.3090).
  • 2012: Mucha refined the Mömke–Svensson analysis to 13/9≈1.44413/9\approx1.44413/9≈1.444 (arXiv:1108.1130).
  • 2012: Sebő and Vygen, 7/5, measured against LP(G)\mathrm{LP}(G)LP(G) (arXiv:1201.1870; Combinatorica 34 (2014), doi:10.1007/s00493-014-2960-3).

Setting

A graph GGG is finite and undirected, with vertex set V(G)V(G)V(G) and edge set E(G)E(G)E(G); it may have parallel edges but no loops. 2G2G2G is the graph obtained by doubling every edge. For W⊆V(G)W\subseteq V(G)W⊆V(G), δ(W)\delta(W)δ(W) is the set of edges with exactly one endpoint in WWW.

For T⊆V(G)T\subseteq V(G)T⊆V(G), a TTT-join is an edge set FFF in which exactly the vertices of TTT have odd degree. A tour of GGG (Definition 1) is a ∅\emptyset∅-join FFF in 2G2G2G such that (V(G),F)(V(G),F)(V(G),F) is connected: a connected multi-subgraph using each edge at most twice in which every degree is even. By Euler's theorem its edges form a closed walk through every vertex, and conversely. OPT(G)\mathrm{OPT}(G)OPT(G) is the minimum size of a tour.

The relaxation is

LP(G):=min⁡{x(E(G)):x∈R≥0E(G), x(δ(W))≥2 for all ∅≠W⊊V(G)},\mathrm{LP}(G):=\min\Big\{x(E(G)) : x\in\mathbb R^{E(G)}_{\ge0},\ x(\delta(W))\ge2 \text{ for all } \emptyset\ne W\subsetneq V(G)\Big\},LP(G):=min{x(E(G)):x∈R≥0E(G)​, x(δ(W))≥2 for all ∅=W⊊V(G)},

with x(S)=∑e∈Sxex(S)=\sum_{e\in S}x_ex(S)=∑e∈S​xe​. Every tour gives a feasible point, so LP(G)≤OPT(G)\mathrm{LP}(G)\le\mathrm{OPT}(G)LP(G)≤OPT(G) (Proposition 2).

The analysis uses ear-decompositions: GGG is built from one vertex by adding ears, each a path joining two existing vertices through new ones, or a circuit through one existing vertex. φ(G)\varphi(G)φ(G) is the minimum number of ears of even length; π\piπ counts pendant ears (nontrivial ears to which no nontrivial ear is attached). Short (2- and 3-) ears in a nice ear-decomposition (Definition 9) form an eardrum MMM; an earmuff (Definition 12) re-routes some of them so that their edges form a forest, and μ(G,M)\mu(G,M)μ(G,M) is its maximum size. The two lower bounds are

Lφ(G)=∣V(G)∣+φ(G)−1,Lμ(G,M)=∣V(G)∣−1+∣M∣−μ(G,M),Λ=23Lμ+13Lφ.L_\varphi(G)=|V(G)|+\varphi(G)-1,\qquad L_\mu(G,M)=|V(G)|-1+|M|-\mu(G,M),\qquad \Lambda=\tfrac23L_\mu+\tfrac13L_\varphi .Lφ​(G)=∣V(G)∣+φ(G)−1,Lμ​(G,M)=∣V(G)∣−1+∣M∣−μ(G,M),Λ=32​Lμ​+31​Lφ​.

Formalization targets

Goal: Theorem 29

for every connected graph G there is a tour F with ∣F∣≤75 LP(G).\text{for every connected graph } G \text{ there is a tour } F \text{ with } |F|\le\tfrac75\,\mathrm{LP}(G).for every connected graph G there is a tour F with ∣F∣≤57​LP(G).

Milestones, in the order of the proof

  1. Proposition 2: OPT(G)≥OPT2EC(G)≥LP(G)≥∣V(G)∣\mathrm{OPT}(G)\ge\mathrm{OPT}_{2EC}(G)\ge\mathrm{LP}(G)\ge|V(G)|OPT(G)≥OPT2EC​(G)≥LP(G)≥∣V(G)∣ (for ∣V(G)∣≥2|V(G)|\ge2∣V(G)∣≥2).
  2. Theorem 7 (Frank): for 2-edge-connected GGG some even TTT has τ(G,T)=12(∣V(G)∣+φ(G)−1)\tau(G,T)=\frac12(|V(G)|+\varphi(G)-1)τ(G,T)=21​(∣V(G)∣+φ(G)−1).
  3. Theorem 19: Lφ(G)≤LP(G)L_\varphi(G)\le\mathrm{LP}(G)Lφ​(G)≤LP(G).
  4. Corollary 21: Lμ(G,M)≤LP(G)L_\mu(G,M)\le\mathrm{LP}(G)Lμ​(G,M)≤LP(G) for an eardrum with every Pf≠∅\mathcal P_f\ne\emptysetPf​=∅.
  5. Lemma 23: a 2-vertex-connected graph has a nice ear-decomposition containing a maximum earmuff.
  6. Theorem 24: a connected-TTT-join with at most Lμ(G,M)+12Lφ(G)−πL_\mu(G,M)+\frac12L_\varphi(G)-\piLμ​(G,M)+21​Lφ​(G)−π edges.
  7. Theorem 27 (Mömke–Svensson): a tour with at most 43∣E(G)∣−23∣R∣\frac43|E(G)|-\frac23|R|34​∣E(G)∣−32​∣R∣ edges for a removable pairing (R,P)(R,\mathcal P)(R,P).
  8. Lemma 28: a tour with at most 43(∣V(G)∣−1)+23π\frac43(|V(G)|-1)+\frac23\pi34​(∣V(G)∣−1)+32​π edges when all ears are nontrivial.
  9. Claim (proof of Theorem 29, p. 22): a tour with at most 75Λ(G′,M)\frac75\Lambda(G',M)57​Λ(G′,M) edges.

Significance

The result. Theorem 29 gives a polynomial-time 7/5-approximation for graphic TSP and shows that the integrality ratio of LP(G)\mathrm{LP}(G)LP(G) on graphic instances is at most 7/5. The same paper uses the shared machinery (nice ear-decompositions, earmuffs, the bound Lμ≤LPL_\mu\le\mathrm{LP}Lμ​≤LP) for a 3/2 bound for connected TTT-joins, including graphic sss-ttt-path TSP, and a 4/3 bound for 2-edge-connected spanning subgraphs; those are separate missions of this series.

Formalizing it. The theorem is proved in the paper; none of it is machine-checked. The platform has metric-TSP results over Hamiltonian permutations (Christofides-type and Wolsey-type bounds), but no graphic TSP, no multigraph tours, no ear-decompositions and no TTT-join polyhedra. The formalization would add a reusable multigraph layer (doubled graphs, TTT-joins, ear-decompositions, Frank's φ\varphiφ), two classical cited theorems (Frank's Theorem 7 and the Mömke–Svensson Lemma) and the paper's own earmuff analysis.

Difficulty

The two tour constructions are each worse than 7/5 on their own: Theorem 24 alone gives 32Λ−π\frac32\Lambda-\pi23​Λ−π, and Lemma 28 alone gives 43Λ+23π\frac43\Lambda+\frac23\pi34​Λ+32​π. The bound comes only from taking the better of the two, balanced at π=110Λ\pi=\frac1{10}\Lambdaπ=101​Λ, and only after the short ears have been re-chosen as a maximum earmuff, which the lower bound LμL_\muLμ​ requires.

The cited inputs are deep. Theorem 19 relies on Frank's ear theorem and the Edmonds–Johnson–Lovász packing of TTT-cuts; Theorem 27 relies on the odd-join polytope of an auxiliary 2-edge-connected graph; Lemma 23 relies on Lemma 10 (nice ear-decompositions exist) and a min-max theorem for earmuffs (Theorem 18, via matroid intersection or forest representatives). Mathlib has none of these. A Christofides-style argument (spanning tree plus matching) cannot get below 3/2 and is not a route to the goal.

Formalization scope

  • Graphs are a structure Graph V E with ends : E → Sym2 V and no loops, over finite types, so parallel edges are allowed as in the paper. Restricting to simple graphs would be a weaker theorem and is ruled out. 2G2G2G has edge type E × Fin 2, so every multiplicity is at most 2. Connectivity of an edge set is connectivity of the simple graph of pairs joined by one of its edges.
  • Existence instead of algorithms. Every "there is an algorithm … in O(∣V(G)∣3)O(|V(G)|^3)O(∣V(G)∣3) time" is formalized as "for every input there exists a solution with the stated bound". Running times are not formalized.
  • LP bounds through feasible points. "∣F∣≤75LP(G)|F|\le\frac75\mathrm{LP}(G)∣F∣≤57​LP(G)" is stated as: there is a tour FFF with ∣F∣≤75x(E(G))|F|\le\frac75x(E(G))∣F∣≤57​x(E(G)) for every LP-feasible xxx. This is equivalent to the paper's statement and needs no infimum. The constant vector x≡2x\equiv2x≡2 is feasible for every connected graph, so the hypothesis is never vacuous.
  • Minima as infima of natural numbers. φ(G)\varphi(G)φ(G) is an infimum over ear-decompositions, and every statement using it assumes 2-edge-connectivity or supplies an ear-decomposition, so the junk value is never reached. μ(G,M)\mu(G,M)μ(G,M) is a supremum of a nonempty set bounded by ∣M∣|M|∣M∣.
  • "2-vertex-connected" means 2-edge-connected with no cut vertex. It includes the one-vertex graph and two vertices joined by parallel edges, the blocks the paper's reduction (Proposition 4) needs.
  • Eardrums read "degree at most 1" as adjacency in the underlying simple graph. The eardrum associated with a nice ear-decomposition is the family of internal vertex sets of its clean ears; condition (iii) of Definition 9 makes these the components the paper describes.
  • Proposition 2 carries the added hypothesis ∣V(G)∣≥2|V(G)|\ge2∣V(G)∣≥2: on one vertex LP(G)=0<∣V(G)∣\mathrm{LP}(G)=0<|V(G)|LP(G)=0<∣V(G)∣.
  • Not included: Proposition 4 (the block reduction), Lemmas 5 and 10, Theorems 18 and 20, and the tightness examples. The proof of the goal needs Proposition 4 and solvers may add it.

Welcome contributions: proofs of any milestone; the multigraph and ear-decomposition layer (Whitney's theorem that 2-edge-connected graphs have ear-decompositions); TTT-join existence and parity lemmas; the block decomposition at a cut vertex.

Selected references

  • A. Sebő and J. Vygen, Shorter tours by nicer ears: 7/5-approximation for graphic TSP, 3/2 for the path version, and 4/3 for two-edge-connected subgraphs, arXiv:1201.1870v3, 2012; Combinatorica 34 (2014) 597–629. https://arxiv.org/abs/1201.1870, https://doi.org/10.1007/s00493-014-2960-3
  • T. Mömke and O. Svensson, Approximating graphic TSP by matchings, FOCS 2011. https://arxiv.org/abs/1104.3090
  • M. Mucha, 13/9-approximation for graphic TSP, STACS 2012; Theory Comput. Syst. 55 (2014). https://arxiv.org/abs/1108.1130
  • S. Oveis Gharan, A. Saberi and M. Singh, A randomized rounding approach to the traveling salesman problem, FOCS 2011. https://doi.org/10.1109/FOCS.2011.80
  • A. Frank, Conservative weightings and ear-decompositions of graphs, Combinatorica 13 (1993) 65–81. https://doi.org/10.1007/BF01202790
  • J. Cheriyan, A. Sebő and Z. Szigeti, Improving on the 1.5-approximation of a smallest 2-edge connected spanning subgraph, SIAM J. Discrete Math. 14 (2001) 170–180. https://doi.org/10.1137/S0895480199362071
  • N. Christofides, Worst-case analysis of a new heuristic for the travelling salesman problem, Report 388, GSIA, Carnegie Mellon University, 1976.
14 thms1 active userReviewed
CombinatoricsGraph TheoryOperations Research·Captain: mikedeng1

Shorter Tours by Nicer Ears 1: Every Connected Graph Has a Connected T-Join with at Most 3/2 LP(G,T) EdgesResearch Paper

Motivation

In graphic TSP the cities are the vertices of a connected graph GGG and the distance between two of them is the number of edges on a shortest path. A closed walk visiting every vertex is the same thing as a connected multi-subgraph of GGG in which every degree is even. Dropping "closed" and asking for a walk from sss to ttt gives graphic sss-ttt-path TSP, where exactly sss and ttt must have odd degree. Both are special cases of the connected-TTT-join problem: given GGG and a vertex set TTT of even size, find a smallest edge multiset in which exactly the vertices of TTT have odd degree and which connects all vertices.

Graphic TSP is MAXSNP-hard (Papadimitriou and Yannakakis 1993), and until 2011 no algorithm beat the ratio 32\tfrac3223​ of Christofides (1976) on it.

Timeline.

  • Christofides (1976) gave a 32\tfrac3223​-approximation for metric TSP. His idea, adapted to connected-TTT-joins, gives 53\tfrac5335​; Hoogeveen (1991) noted this for ∣T∣=2|T|=2∣T∣=2.
  • In 2011, Oveis Gharan, Saberi and Singh gave a (32−ϵ)(\tfrac32-\epsilon)(23​−ϵ)-approximation for graphic TSP, and Mömke and Svensson a 1.4611.4611.461-approximation; Mucha (2012) refined the latter analysis to 139\tfrac{13}{9}913​.
  • An, Kleinberg and Shmoys (2012) obtained 1.5781.5781.578 for graphic sss-ttt-path TSP.
  • Sebő and Vygen (arXiv 2012; Combinatorica 2014) proved 75\tfrac7557​ for graphic TSP, 43\tfrac4334​ for 2-edge-connected spanning subgraphs, and 32\tfrac3223​ for connected-TTT-joins (hence for graphic sss-ttt-path TSP), each against an LP lower bound. This mission formalizes the third.

Setting

All graphs are finite and undirected, with parallel edges allowed and no loops. For a graph GGG, 2G2G2G is the graph with every edge doubled. A TTT-join is an edge set FFF such that TTT is exactly the set of vertices of odd degree in FFF. A connected-TTT-join of GGG is a TTT-join FFF in 2G2G2G such that (V(G),F)(V(G),F)(V(G),F) is connected. Each edge of GGG is used at most twice. A tour is a connected-∅\emptyset∅-join. OPT(G,T)\mathrm{OPT}(G,T)OPT(G,T) is the minimum size of a connected-TTT-join.

For W⊆V(G)W\subseteq V(G)W⊆V(G), δ(W)\delta(W)δ(W) is the set of edges with exactly one end in WWW. For a partition W\mathcal WW of V(G)V(G)V(G), δ(W)\delta(\mathcal W)δ(W) is the set of edges joining different classes. The LP relaxation is

LP(G,T)=min⁡{x(E(G)):x≥0, x(δ(W))≥2 for ∅≠W⊊V(G) with ∣W∩T∣ even, x(δ(W))≥∣W∣−1 for all partitions W}.\mathrm{LP}(G,T)=\min\Bigl\{x(E(G)) : x\ge0,\ x(\delta(W))\ge2\ \text{for } \emptyset\ne W\subsetneq V(G) \text{ with } |W\cap T| \text{ even},\ x(\delta(\mathcal W))\ge|\mathcal W|-1\ \text{for all partitions } \mathcal W\Bigr\}.LP(G,T)=min{x(E(G)):x≥0, x(δ(W))≥2 for ∅=W⊊V(G) with ∣W∩T∣ even, x(δ(W))≥∣W∣−1 for all partitions W}.

The proof works with ear-decompositions: a vertex followed by ears, each a path attached at its two ends or a circuit attached at one vertex. φ(G)\varphi(G)φ(G) is Frank's minimum number of even-length ears. A nice ear-decomposition has φ(G)\varphi(G)φ(G) even ears, all of its short (2- and 3-) ears are pendant, and the internal vertices of different short ears are non-adjacent. An eardrum MMM is a family of single vertices and adjacent pairs that are pairwise non-adjacent. An earmuff chooses, for some of its members fff, a path PfP_fPf​ whose internal vertex set is fff, so that the chosen paths together form a forest. μ(G,M)\mu(G,M)μ(G,M) is the maximum number of paths in an earmuff, and

Lμ(G,M)=∣V(G)∣−1+∣M∣−μ(G,M),Lφ(G)=∣V(G)∣+φ(G)−1.L_\mu(G,M)=|V(G)|-1+|M|-\mu(G,M),\qquad L_\varphi(G)=|V(G)|+\varphi(G)-1.Lμ​(G,M)=∣V(G)∣−1+∣M∣−μ(G,M),Lφ​(G)=∣V(G)∣+φ(G)−1.

Formalization targets

Goal: Theorem 25 (p. 19)

For every connected graph GGG and every T⊆V(G)T\subseteq V(G)T⊆V(G) with ∣T∣|T|∣T∣ even there is a connected-TTT-join FFF with

∣F∣ ≤ 32 LP(G,T).|F|\ \le\ \tfrac32\,\mathrm{LP}(G,T).∣F∣ ≤ 23​LP(G,T).

Milestones, in attack order

  1. Proposition 3: OPT(G,T)≥LP(G,T)≥∣V(G)∣−1\mathrm{OPT}(G,T)\ge\mathrm{LP}(G,T)\ge|V(G)|-1OPT(G,T)≥LP(G,T)≥∣V(G)∣−1.
  2. Proposition 4: OPT(G,T)\mathrm{OPT}(G,T)OPT(G,T), OPT2EC(G)\mathrm{OPT}_{2EC}(G)OPT2EC​(G) and LP(G,T)\mathrm{LP}(G,T)LP(G,T) are additive over a cut vertex.
  3. Lemma 5: local bounds for a pendant ear.
  4. Proposition 6 (Frank): τ(G,T)≤12(∣V(G)∣+φ(G)−1)\tau(G,T)\le\tfrac12(|V(G)|+\varphi(G)-1)τ(G,T)≤21​(∣V(G)∣+φ(G)−1).
  5. Proposition 8: a connected-TTT-join with at most 32(∣V(G)∣−1)+π2−12φ(G)\tfrac32(|V(G)|-1)+\pi_2-\tfrac12\varphi(G)23​(∣V(G)∣−1)+π2​−21​φ(G) edges.
  6. Lemma 10: nice ear-decompositions exist.
  7. Corollary 14 (Lovász): the min-max formula for forest representative systems.
  8. Theorem 18: the earmuff min-max formula for μ(G,M)\mu(G,M)μ(G,M).
  9. Lemma 23: a nice ear-decomposition containing a maximum earmuff.
  10. Theorem 20: Lμ(G,M)≤LP(G,T)L_\mu(G,M)\le\mathrm{LP}(G,T)Lμ​(G,M)≤LP(G,T).
  11. Theorem 24: a connected-TTT-join with at most Lμ(G,M)+12Lφ(G)−πL_\mu(G,M)+\tfrac12L_\varphi(G)-\piLμ​(G,M)+21​Lφ​(G)−π edges, where π\piπ is the number of pendant ears.

Significance

The result. Theorem 25 gives a 32\tfrac3223​-approximation for connected-TTT-joins, and therefore for graphic sss-ttt-path TSP. It also bounds the integrality ratio of LP(G,T)\mathrm{LP}(G,T)LP(G,T) by 32\tfrac3223​, and the paper shows this is tight (Figure 3). Lemma 23, Theorem 20 and Theorem 24 are shared with the 7/5 bound for graphic TSP and the 4/3 bound for 2-edge-connected spanning subgraphs: the three results differ only in the second construction used when there are few pendant ears. The earmuff lower bound LμL_\muLμ​ is a combinatorial dual certificate for LP(G,T)\mathrm{LP}(G,T)LP(G,T) in its own right.

Formalizing it. The theorem is proved on paper; none of it is machine-checked. A formal development would contribute:

  • multigraph ear-decompositions with parity bookkeeping;
  • Frank's bound on TTT-joins;
  • Lovász's forest-representative min-max, a matroid-intersection-type theorem;
  • a fully checked LP lower bound via a combinatorial dual.

None of these is in Mathlib.

Difficulty

The natural first attempt is Christofides' construction: a spanning tree plus a parity-correcting join. The tree costs at most OPT(G,T)\mathrm{OPT}(G,T)OPT(G,T) and the join at most 23OPT(G,T)\tfrac23\mathrm{OPT}(G,T)32​OPT(G,T), which gives only 53\tfrac5335​ (paper, p. 3). It gives nothing better against LP(G,T)\mathrm{LP}(G,T)LP(G,T).

Ear-decompositions with few even ears control the parity part (Proposition 6). Then short pendant ears become the bottleneck. Proposition 8 is good only when there are few 2-ears. When there are many pendant ears, the matching lower bound must account for them, and it is not visible from ∣V(G)∣−1|V(G)|-1∣V(G)∣−1. It comes from the earmuff min-max theorem (Theorem 18), which rests on Lovász's matroidal generalization of Kőnig's theorem (Corollary 14). The ear-decomposition must also be re-chosen so that its clean short ears form a maximum earmuff while it stays nice (Lemma 23).

Formalization scope

  • Graphs. A graph is a loopless multigraph Graph V E with ends : E → Sym2 V over finite types. 2G2G2G has edge type E × Fin 2, so a connected-TTT-join is a Finset (E × Fin 2) and uses each edge at most twice.
  • Restriction to simple graphs would be a weaker theorem and is ruled out: parallel edges arise in the ear arguments.
  • Algorithms. Every algorithmic statement of the paper ("can be computed in O(⋅)O(\cdot)O(⋅) time") is formalized as existence for every input. Algorithms and running times are not formalized.
  • LP bounds. "∣F∣≤c⋅LP(G,T)|F|\le c\cdot\mathrm{LP}(G,T)∣F∣≤c⋅LP(G,T)" is formalized as ∣F∣≤c⋅x(E(G))|F|\le c\cdot x(E(G))∣F∣≤c⋅x(E(G)) for every feasible xxx, with FFF chosen before xxx. This is equivalent to the bound on the minimum and needs no infimum. Equalities of optimum values (Proposition 4) and min-max identities (Corollary 14, Theorem 18) are stated as the two inequalities between solutions, with the minimum attained.
  • Non-vacuity. A trivializing reading of the LP is ruled out: for connected GGG the constant vector ∣V(G)∣|V(G)|∣V(G)∣ is feasible. φ(G)\varphi(G)φ(G) is an infimum over ear-decompositions and is used only under 2-edge-connectivity or with an ear-decomposition in hand. μ(G,M)\mu(G,M)μ(G,M) is a maximum over a nonempty set bounded by ∣M∣|M|∣M∣.
  • 2-vertex-connected means 2-edge-connected without a cut vertex. This includes two vertices joined by parallel edges and excludes a single bridge.
  • Eardrums (Definition 9) read "degree at most 1" as adjacency, so a parallel copy of an edge does not disqualify a 3-ear. The eardrum associated with a nice ear-decomposition is the family of internal vertex sets of its clean ears.
  • Integer quantities. Surplus, LμL_\muLμ​ and LφL_\varphiLφ​ are integers, with no truncated subtraction.
  • Corrected misprint. Proposition 4 is stated with the corrected identity LP(G1,T1)+LP(G2,T2)\mathrm{LP}(G_1,T_1)+\mathrm{LP}(G_2,T_2)LP(G1​,T1​)+LP(G2​,T2​) (the page prints LP(G,T1)+LP(G,T2)\mathrm{LP}(G,T_1)+\mathrm{LP}(G,T_2)LP(G,T1​)+LP(G,T2​)). Its OPT2EC\mathrm{OPT}_{2EC}OPT2EC​ identity is included, with 2-edge-connected spanning multi-subgraphs taken in 2G2G2G.

All definitions are local to the namespace NicerEars.TJoin. Mathlib has neither an ear-decomposition theory nor a multigraph connectivity API. A reusable library of multigraph ears, TTT-joins and forest representative systems would serve the two sibling missions (graphic TSP, 2ECSS) as well. Contributions to any milestone, and to general lemmas such as the existence of TTT-joins in connected graphs, are welcome.

Selected references

  • A. Sebő and J. Vygen, Shorter tours by nicer ears: 7/5-approximation for graphic TSP, 3/2 for the path version, and 4/3 for two-edge-connected subgraphs, Combinatorica 34 (2014). arXiv:1201.1870v3. https://arxiv.org/abs/1201.1870 , https://doi.org/10.1007/s00493-014-2960-3
  • A. Frank, Conservative weightings and ear-decompositions of graphs, Combinatorica 13 (1993) 65–81. https://doi.org/10.1007/BF01202790
  • J. Cheriyan, A. Sebő and Z. Szigeti, Improving on the 1.5-approximation of a smallest 2-edge connected spanning subgraph, SIAM J. Discrete Math. 14 (2001) 170–180. https://doi.org/10.1137/S0895480199362071
  • L. Lovász, A generalization of Kőnig's theorem, Acta Math. Acad. Sci. Hungar. 21 (1970) 443–446. https://doi.org/10.1007/BF01894789
  • T. Mömke and O. Svensson, Approximating graphic TSP by matchings, FOCS 2011, 560–569. arXiv:1104.3090. https://arxiv.org/abs/1104.3090
  • M. Mucha, 13/9-approximation for graphic TSP, STACS 2012, 30–41. https://doi.org/10.4230/LIPIcs.STACS.2012.30
  • J. A. Hoogeveen, Analysis of Christofides' heuristic: some paths are more difficult than cycles, Oper. Res. Lett. 10 (1991) 291–295. https://doi.org/10.1016/0167-6377(91)90016-I
16 thms1 active userReviewed
Machine LearningStatistics·Captain: mikedeng1

Fast Rates for Contextual Linear Optimization 1: Under the Margin Condition, ERM over a Policy Class of Natarajan Dimension η Has Regret O(B(η log(|Z∠|+1) log(n+1)/n)^((1+α)/(2+α)))Research Paper

Motivation

Many operational decisions are linear programs whose cost vector is uncertain but partly predictable: shortest-path routing with uncertain travel times, portfolio choice with uncertain returns, shipment planning with uncertain costs. Before deciding, one observes features XXX (traffic, market signals) correlated with the costs YYY. Contextual linear optimization asks for a policy mapping features to decisions that minimizes the conditional expected cost, learned from past data (X1,Y1),…,(Xn,Yn)(X_1, Y_1), \dots, (X_n, Y_n)(X1​,Y1​),…,(Xn​,Yn​).

Two families of methods compete. Estimate-then-optimize fits a regression of YYY on XXX and plugs it into the optimizer. Empirical risk minimization (ERM) and its "integrated" variants choose a policy directly to minimize the empirical decision cost. A widespread view holds that the integrated approach should win because it targets the decision cost. Hu, Kallus and Mao (arXiv:2011.03030, Management Science 2022) compare the two through their regret rates. Slow, noise-independent rates of order n−1/2n^{-1/2}n−1/2 do not separate them. The separation appears under a margin (noise) condition controlling how often the problem is nearly dual-degenerate. This mission formalizes the paper's fast rate for ERM, Theorem 6.

Timeline. Margin conditions for classification go back to Mammen–Tsybakov (1999) and Tsybakov (2004); Massart–Nédélec (2006) obtained the corresponding fast rates for ERM over VC classes. El Balghiti, Elmachtoub, Grigas and Tewari (2019) gave slow generalization bounds for decision-focused losses in terms of the Natarajan dimension. Hu, Kallus and Mao (2020, revised 2021) proved the fast rate for ERM over policy classes of finite Natarajan dimension, a matching lower bound, and a faster rate for estimate-then-optimize.

Setting

The decision set is a polytope Z={z∈Rd:Az≤b}\mathcal Z = \{z \in \mathbb R^d : Az \le b\}Z={z∈Rd:Az≤b}, nonempty, with ∥z∥≤B\|z\| \le B∥z∥≤B on Z\mathcal ZZ for some B>0B > 0B>0; Z∠\mathcal Z^\angleZ∠ is its finite set of extreme points. The pair (X,Y)∈Rp×Rd(X, Y) \in \mathbb R^p \times \mathbb R^d(X,Y)∈Rp×Rd has ∥Y∥≤1\|Y\| \le 1∥Y∥≤1, and f∗(x)=E[Y∣X=x]f^*(x) = \mathbb E[Y \mid X = x]f∗(x)=E[Y∣X=x]. The optimal set at xxx is Z∗(x)=arg⁡min⁡z∈Zf∗(x)⊤z\mathcal Z^*(x) = \arg\min_{z \in \mathcal Z} f^*(x)^\top zZ∗(x)=argminz∈Z​f∗(x)⊤z.

A policy is a map π:Rp→Z∠\pi : \mathbb R^p \to \mathcal Z^\angleπ:Rp→Z∠. A data-driven policy π^\hat\piπ^ maps a sample D\mathcal DD of nnn i.i.d. draws to a policy, and its regret is

Regret(π^)=EDEX[f∗(X)⊤π^(X)−min⁡z∈Zf∗(X)⊤z].\mathrm{Regret}(\hat\pi) = \mathbb E_{\mathcal D}\mathbb E_X\bigl[f^*(X)^\top\hat\pi(X) - \min_{z \in \mathcal Z} f^*(X)^\top z\bigr].Regret(π^)=ED​EX​[f∗(X)⊤π^(X)−z∈Zmin​f∗(X)⊤z].

Given a class Π\PiΠ of policies, π^ΠERM\hat\pi^{\mathrm{ERM}}_\Piπ^ΠERM​ returns a member of Π\PiΠ minimizing 1n∑iYi⊤π(Xi)\frac1n\sum_i Y_i^\top\pi(X_i)n1​∑i​Yi⊤​π(Xi​).

The gap Δ(x)\Delta(x)Δ(x) is the difference between the best value over suboptimal extreme points and the optimal value, set to 000 when every point of Z\mathcal ZZ is optimal. Assumption 2 (noise condition) with parameters α,γ≥0\alpha, \gamma \ge 0α,γ≥0 requires

PX(0<Δ(X)≤δ)≤(γδ/B)αfor all δ>0.\mathbb P_X\bigl(0 < \Delta(X) \le \delta\bigr) \le (\gamma\delta/B)^\alpha \qquad \text{for all } \delta > 0.PX​(0<Δ(X)≤δ)≤(γδ/B)αfor all δ>0.

The Natarajan dimension of Π\PiΠ is the largest kkk for which there are points x1,…,xkx_1, \dots, x_kx1​,…,xk​ and labels si≠si′s_i \ne s'_isi​=si′​ such that every pattern of choices between sis_isi​ and si′s'_isi′​ is realized by some π∈Π\pi \in \Piπ∈Π.

Formalization targets

Goal: Theorem 6

If Assumption 2 holds, P(∣Z∗(X)∣>1)=0\mathbb P(|\mathcal Z^*(X)| > 1) = 0P(∣Z∗(X)∣>1)=0, Π\PiΠ has Natarajan dimension at most η\etaη, and Π\PiΠ contains an optimal policy π∗\pi^*π∗, then for a constant C(α,γ)C(\alpha, \gamma)C(α,γ) depending only on α,γ\alpha, \gammaα,γ,

Regret(π^ΠERM)≤C(α,γ) B(ηlog⁡(∣Z∠∣+1)log⁡(n+1)n)1+α2+α.\mathrm{Regret}(\hat\pi^{\mathrm{ERM}}_\Pi) \le C(\alpha, \gamma)\,B\left(\frac{\eta\log(|\mathcal Z^\angle| + 1)\log(n + 1)}{n}\right)^{\frac{1+\alpha}{2+\alpha}}.Regret(π^ΠERM​)≤C(α,γ)B(nηlog(∣Z∠∣+1)log(n+1)​)2+α1+α​.

The constant is existential and uniform over all dimensions, polytopes, distributions, classes, η\etaη and nnn; the goal fixes no numerical value.

Milestones

With d(π,π′)=1BEX[f∗(X)⊤(π′(X)−π(X))]d(\pi, \pi') = \frac1B\mathbb E_X[f^*(X)^\top(\pi'(X) - \pi(X))]d(π,π′)=B1​EX​[f∗(X)⊤(π′(X)−π(X))] and dΔ(π,π′)=PX(π(X)≠π′(X))d_\Delta(\pi, \pi') = \mathbb P_X(\pi(X) \ne \pi'(X))dΔ​(π,π′)=PX​(π(X)=π′(X)):

  1. Lemma 1: d(π∗,π)≤2dΔ(π∗,π)d(\pi^*, \pi) \le 2d_\Delta(\pi^*, \pi)d(π∗,π)≤2dΔ​(π∗,π) and dΔ(π∗,π)≤c1d(π∗,π)α/(α+1)d_\Delta(\pi^*, \pi) \le c_1 d(\pi^*, \pi)^{\alpha/(\alpha+1)}dΔ​(π∗,π)≤c1​d(π∗,π)α/(α+1) with c1=(αγα)−α/(α+1)(α+1)γαc_1 = (\alpha\gamma^\alpha)^{-\alpha/(\alpha+1)}(\alpha+1)\gamma^\alphac1​=(αγα)−α/(α+1)(α+1)γα.
  2. Lemma 2 (Bousquet's inequality): for a countable class bounded by Hˉ\bar HHˉ with second moments at most δ2\delta^2δ2, S−ESS - \mathbb E SS−ES exceeds 2(δ2+4HˉES)t/n+2Hˉt/(3n)\sqrt{2(\delta^2 + 4\bar H\mathbb E S)t/n} + 2\bar Ht/(3n)2(δ2+4HˉES)t/n​+2Hˉt/(3n) with probability at most e−te^{-t}e−t.
  3. Lemma 3: the localized excess-loss process satisfies ED[sup⁡h∈Hδ(Enh−EPh)]≤(1+2)C05ηlog⁡(∣Z∠∣2+1)log⁡(n+1)/n δ\mathbb E_{\mathcal D}[\sup_{h \in \mathcal H_\delta}(\mathbb E_n h - \mathbb E_P h)] \le (1+\sqrt2)C_0\sqrt{5\eta\log(|\mathcal Z^\angle|^2+1)\log(n+1)/n}\,\deltaED​[suph∈Hδ​​(En​h−EP​h)]≤(1+2​)C0​5ηlog(∣Z∠∣2+1)log(n+1)/n​δ for nnn above a threshold.
  4. Tail bound (proof of Theorem 6, p. 27): for t≥1t \ge 1t≥1, P(d(π∗,π^ΠERM)≥Kt(Vˉlog⁡(n+1)/n)(1+α)/(2+α))≤e−t\mathbb P(d(\pi^*, \hat\pi^{\mathrm{ERM}}_\Pi) \ge K t(\bar V\log(n+1)/n)^{(1+\alpha)/(2+\alpha)}) \le e^{-t}P(d(π∗,π^ΠERM​)≥Kt(Vˉlog(n+1)/n)(1+α)/(2+α))≤e−t with Vˉ=5ηlog⁡(∣Z∠∣2+1)\bar V = 5\eta\log(|\mathcal Z^\angle|^2+1)Vˉ=5ηlog(∣Z∠∣2+1) and n≥cVˉlog⁡(n+1)n \ge c\bar V\log(n+1)n≥cVˉlog(n+1).

Significance

Theorem 6 shows that the regret of ERM improves from n−1/2n^{-1/2}n−1/2 to n−(1+α)/(2+α)n^{-(1+\alpha)/(2+\alpha)}n−(1+α)/(2+α) when the instance is rarely near dual degeneracy. With α=1\alpha = 1α=1, the generic case when XXX has a bounded density and f∗f^*f∗ is linear, the rate is n−2/3n^{-2/3}n−2/3. Theorem 7 of the same paper (mission 2 of this series) shows that the exponent is optimal for any method that uses only the policy class. Theorem 8 (mission 3) shows that estimate-then-optimize can reach n−(1+α)/2n^{-(1+\alpha)/2}n−(1+α)/2, which is the paper's case against the conventional preference for integrated methods.

The result is proved in the paper; no machine-checked proof is known. Formalizing it adds a checked fast-rate theorem for a decision-focused learning method, a Lean statement of Bousquet's inequality in the form used throughout statistical learning theory, and a localized empirical-process bound for classes of finite Natarajan dimension.

Difficulty

The slow rate follows from uniform convergence of the empirical cost over Π\PiΠ. This argument cannot give more than n−1/2n^{-1/2}n−1/2: it bounds the deviation of every policy by the same worst-case amount, ignoring that near-optimal policies have small variance. The fast rate needs a variance–mean relation (Lemma 1, which is where the noise condition and the uniqueness assumption P(∣Z∗(X)∣>1)=0\mathbb P(|\mathcal Z^*(X)| > 1) = 0P(∣Z∗(X)∣>1)=0 enter), a concentration inequality that exploits small variance (Lemma 2), and a localized complexity bound (Lemma 3), combined over shells of the class. Lemma 2 is itself a deep result whose known proofs go through the entropy method. Lemma 3 rests on chaining and on bounding the pseudo-dimension of the excess-loss class by the Natarajan dimension, neither of which is in Mathlib.

Formalization scope

All declarations live in the namespace FastCLO.ERM. Conventions:

  • Spaces are EuclideanSpace ℝ (Fin k). The polytope is a structure carrying AAA, bbb, B>0B > 0B>0, nonemptiness and the bound ∥z∥≤B\|z\| \le B∥z∥≤B; ∣Z∠∣|\mathcal Z^\angle|∣Z∠∣ is the cardinality of Set.extremePoints, whose finiteness is a proof obligation, not a hypothesis.
  • The distribution is given by the law μ\muμ of XXX and a Markov kernel κ\kappaκ for YYY given XXX, with ∥Y∥≤1\|Y\| \le 1∥Y∥≤1 κ(x)\kappa(x)κ(x)-almost surely for every xxx; f∗f^*f∗ is the kernel mean; data are Measure.pi of μ⊗κ\mu \otimes \kappaμ⊗κ.
  • Regret is measured against the optimal value; this equals the paper's f∗(X)⊤(π^(X)−π∗(X))f^*(X)^\top(\hat\pi(X) - \pi^*(X))f∗(X)⊤(π^(X)−π∗(X)).
  • ERM is any selection from the empirical argmin over the whole class; the paper's tie-breaking order is not encoded. The ERM rule is assumed jointly measurable in the data and the feature (an added hypothesis, so that the regret is a genuine integral).
  • "π∗∈Π\pi^* \in \Piπ∗∈Π" means some member of Π\PiΠ is optimal at almost every xxx; "Natarajan dimension at most η\etaη" means Π\PiΠ does not shatter η+1\eta + 1η+1 points.
  • Added hypotheses: n≥1n \ge 1n≥1 wherever the paper divides by nnn; Hˉ>0\bar H > 0Hˉ>0 in Lemma 2; measurability of the supremum in Lemma 3; η≥1\eta \ge 1η≥1 in the tail bound (the paper's ϵn>0\epsilon_n > 0ϵn​>0). Constants (CCC, C0C_0C0​, KKK, ccc) are quantified before everything they are uniform over.

A statement in which the regret integrated the empirical cost instead of f∗f^*f∗, or in which ERM minimized over a subset of the class, or in which the constant could depend on the instance, would be a different theorem; the Lean statements rule these out by construction.

Needed infrastructure: Bousquet's (or Talagrand's) inequality, symmetrization and Dudley chaining for empirical processes, packing bounds for classes of finite pseudo-dimension, and the Natarajan-to-pseudo-dimension reduction. These are reusable far beyond this mission; contributions of any of them are welcome.

Selected references

  • Y. Hu, N. Kallus, X. Mao, Fast Rates for Contextual Linear Optimization, Management Science, 2022; arXiv:2011.03030v3, 2021. https://arxiv.org/abs/2011.03030
  • O. Bousquet, A Bennett concentration inequality and its application to suprema of empirical processes, Comptes Rendus Mathématique 334(6), 2002. https://doi.org/10.1016/S1631-073X(02)02292-6
  • P. Massart, É. Nédélec, Risk bounds for statistical learning, Annals of Statistics 34(5), 2006. https://doi.org/10.1214/009053606000000786
  • A. B. Tsybakov, Optimal aggregation of classifiers in statistical learning, Annals of Statistics 32(1), 2004. https://doi.org/10.1214/aos/1079120131
  • O. El Balghiti, A. N. Elmachtoub, P. Grigas, A. Tewari, Generalization Bounds in the Predict-then-Optimize Framework, NeurIPS 2019; arXiv:1905.11488. https://arxiv.org/abs/1905.11488
  • B. K. Natarajan, On learning sets and functions, Machine Learning 4, 1989. https://doi.org/10.1007/BF00114804
10 thms1 active userReviewed
Machine LearningStatistics·Captain: mikedeng1

Fast Rates for Contextual Linear Optimization 2: Any Algorithm Choosing from a Policy Class of Natarajan Dimension η Has Regret ≥ (ρ(Z)/2e⁴)((η−1)/n)^((1+α)/(2+α)) on Some Margin-α InstanceResearch Paper

Motivation

In contextual linear optimization (CLO) a decision maker observes a covariate XXX (features of a day, a customer, a network state) and then chooses a decision zzz from a polytope Z\mathcal ZZ to minimise an uncertain linear cost Y⊤zY^\top zY⊤z. Shortest-path routing with predicted travel times and inventory or portfolio decisions driven by forecasts are standard instances. Since YYY is unknown at decision time, the relevant cost is f∗(X)⊤zf^*(X)^\top zf∗(X)⊤z with f∗(x)=E[Y∣X=x]f^*(x) = \mathbb E[Y \mid X = x]f∗(x)=E[Y∣X=x], and a policy is learned from nnn past observations (Xi,Yi)(X_i, Y_i)(Xi​,Yi​).

Hu, Kallus and Mao (arXiv:2011.03030, Management Science 2022) compare two families of methods: estimate-then-optimize (fit f∗f^*f∗, then optimise) and induced empirical risk minimisation (search a policy class directly for low empirical cost). Their main finding is that under a noise condition that controls how often two vertices of Z\mathcal ZZ are nearly tied, the regret decays faster than the usual n−1/2n^{-1/2}n−1/2. For ERM over a policy class of Natarajan dimension η\etaη they prove the rate (η/n)(1+α)/(2+α)(\eta/n)^{(1+\alpha)/(2+\alpha)}(η/n)(1+α)/(2+α) up to logarithms (their Theorem 6). This mission formalizes the matching lower bound, their Theorem 7: no algorithm that only knows the true optimal policy lies in the class can do better.

Setting

A polytope Z={z∈Rd:Az≤b}\mathcal Z = \{z \in \mathbb R^d : Az \le b\}Z={z∈Rd:Az≤b} is nonempty with sup⁡z∈Z∥z∥≤B\sup_{z \in \mathcal Z}\|z\| \le Bsupz∈Z​∥z∥≤B, B>0B > 0B>0. Its set of extreme points is Z∠\mathcal Z^\angleZ∠. An instance is a law PX\mathbb P_XPX​ of X∈RpX \in \mathbb R^pX∈Rp together with a conditional law of Y∈RdY \in \mathbb R^dY∈Rd given X=xX = xX=x, with ∥Y∥≤1\|Y\| \le 1∥Y∥≤1; f∗(x)f^*(x)f∗(x) is the conditional mean. The optimal set is Z∗(x)=arg⁡min⁡z∈Zf∗(x)⊤z\mathcal Z^*(x) = \arg\min_{z\in\mathcal Z} f^*(x)^\top zZ∗(x)=argminz∈Z​f∗(x)⊤z and the gap is

Δ(x)=inf⁡z∈Z∠∖Z∗(x)f∗(x)⊤z−min⁡z∈Zf∗(x)⊤z\Delta(x) = \inf_{z \in \mathcal Z^\angle \setminus \mathcal Z^*(x)} f^*(x)^\top z - \min_{z \in \mathcal Z} f^*(x)^\top zΔ(x)=z∈Z∠∖Z∗(x)inf​f∗(x)⊤z−z∈Zmin​f∗(x)⊤z

when Z∗(x)≠Z\mathcal Z^*(x) \ne \mathcal ZZ∗(x)=Z, and Δ(x)=0\Delta(x) = 0Δ(x)=0 otherwise. The noise condition (Assumption 2) with parameters α,γ≥0\alpha, \gamma \ge 0α,γ≥0 is

PX(0<Δ(X)≤δ)≤(γδ/B)α∀δ>0.\mathbb P_X\big(0 < \Delta(X) \le \delta\big) \le (\gamma\delta/B)^\alpha \qquad \forall \delta > 0.PX​(0<Δ(X)≤δ)≤(γδ/B)α∀δ>0.

A policy is a map π:Rp→Z∠\pi : \mathbb R^p \to \mathcal Z^\angleπ:Rp→Z∠. A class Π\PiΠ of policies Natarajan-shatters η\etaη points if there are x1,…,xηx_1, \dots, x_\etax1​,…,xη​ and values si≠si′s_i \ne s_i'si​=si′​ such that every pattern in {0,1}η\{0,1\}^\eta{0,1}η is realised by some π∈Π\pi \in \Piπ∈Π choosing sis_isi​ or si′s_i'si′​ at xix_ixi​; the Natarajan dimension is the largest such η\etaη. An algorithm maps a sample D\mathcal DD of nnn i.i.d. draws of (X,Y)(X, Y)(X,Y) to a policy π^D∈Π\hat\pi_{\mathcal D} \in \Piπ^D​∈Π, and its regret is

Regret(π^)=ED EX[f∗(X)⊤π^D(X)−min⁡z∈Zf∗(X)⊤z].\mathrm{Regret}(\hat\pi) = \mathbb E_{\mathcal D}\,\mathbb E_X\Big[f^*(X)^\top \hat\pi_{\mathcal D}(X) - \min_{z\in\mathcal Z} f^*(X)^\top z\Big].Regret(π^)=ED​EX​[f∗(X)⊤π^D​(X)−z∈Zmin​f∗(X)⊤z].

The scale of the bound is the vertex separation

ρ(Z)=inf⁡z∈Z∠, z′∈conv(Z∠∖{z})∥z−z′∥.\rho(\mathcal Z) = \inf_{z \in \mathcal Z^\angle,\ z' \in \mathrm{conv}(\mathcal Z^\angle \setminus \{z\})} \|z - z'\|.ρ(Z)=z∈Z∠, z′∈conv(Z∠∖{z})inf​∥z−z′∥.

Formalization targets

Goal: Theorem 7

Fix α≥0\alpha \ge 0α≥0, a polytope Z\mathcal ZZ, a class Π\PiΠ of Natarajan dimension at least η≥1\eta \ge 1η≥1, and n≥22+α(η−1)n \ge 2^{2+\alpha}(\eta - 1)n≥22+α(η−1). For every algorithm D↦π^D∈Π\mathcal D \mapsto \hat\pi_{\mathcal D} \in \PiD↦π^D​∈Π there is an instance with ∥Y∥≤1\|Y\| \le 1∥Y∥≤1 on which some π∗∈Π\pi^* \in \Piπ∗∈Π is optimal at every xxx, the noise condition holds with the given α\alphaα and γ=B/ρ(Z)\gamma = B/\rho(\mathcal Z)γ=B/ρ(Z), and

Regret(π^)  ≥  ρ(Z)2e4(η−1n)1+α2+α.\mathrm{Regret}(\hat\pi) \;\ge\; \frac{\rho(\mathcal Z)}{2e^4}\Big(\frac{\eta - 1}{n}\Big)^{\frac{1+\alpha}{2+\alpha}}.Regret(π^)≥2e4ρ(Z)​(nη−1​)2+α1+α​.

Milestones

  1. ρ(Z)>0\rho(\mathcal Z) > 0ρ(Z)>0 when Z\mathcal ZZ has two extreme points (Theorem 3, p. 7).
  2. Eq. (25): the vertex-separating directions w(z)w(z)w(z) have norm at most one and separate zzz from the other vertices by exactly ρ(Z)\rho(\mathcal Z)ρ(Z).
  3. Eq. (14): the error of any estimate of a hidden bit is at least the minimum Bayes risk.
  4. The posterior identity of p. 21.
  5. The Jensen step of p. 21.
  6. The binomial bound E∣N1−N0∣≤nq\mathbb E|N^1 - N^0| \le \sqrt{nq}E∣N1−N0∣≤nq​ of pp. 21–22.
  7. The exponential bound in Eq. (15).
  8. The closing calculation of the proof of Theorem 7 (p. 30).

Significance

Theorem 7 shows that the fast rate of Theorem 6 is optimal in both nnn and η\etaη, up to logarithmic factors, among methods that exploit only the well-specification of the policy class. The paper uses this to explain why estimate-then-optimize, which also exploits the structure of the regression class, can beat ERM: it is not subject to this lower bound. At α=0\alpha = 0α=0 the bound recovers the order of the paper's slow-rate lower bound (Theorem 3), and as α→∞\alpha \to \inftyα→∞ it approaches η/n\eta/nη/n.

The result is proved in the paper; no machine-checked proof of it or of its milestones is known. Formalizing it produces a checked minimax lower bound for a decision problem over an arbitrary polytope, together with a reusable toolkit for Assouad-type arguments over hypercubes of hard instances: the binary Bayes-risk bound, the closed-form posterior for a biased coin, the Jensen step, and the binomial mean absolute deviation bound.

Difficulty

The statement quantifies over every algorithm, including non-measurable and randomisation-free ones, so the argument cannot analyse a specific estimator. The standard route averages the regret over a family of 2η−12^{\eta-1}2η−1 instances and reduces it to hidden-bit estimation, but two features of CLO make the construction delicate. First, the instances must keep ∥Y∥≤1\|Y\| \le 1∥Y∥≤1 while giving each shattered point a prescribed gap between its optimal vertex and every other vertex of an arbitrary polytope; this is where ρ(Z)\rho(\mathcal Z)ρ(Z) and the directions of Eq. (25) enter. Second, each instance must satisfy the noise condition with the given α\alphaα and γ=B/ρ(Z)\gamma = B/\rho(\mathcal Z)γ=B/ρ(Z), which fixes how much probability mass the near-tied points may carry; the mass cannot simply be spread uniformly as in the slow-rate argument. The constants have to be tracked exactly, since the goal states the explicit constant 1/(2e4)1/(2e^4)1/(2e4).

Formalization scope

Everything lives in the namespace FastCLO.LowerBound. Vectors are EuclideanSpace ℝ (Fin k). A polytope bundles its constraint rows, right-hand sides, a radius B>0B > 0B>0, nonemptiness and boundedness. An instance is a probability measure μ\muμ on Rp\mathbb R^pRp and a Markov kernel κ\kappaκ with κ(x)({∥y∥>1})=0\kappa(x)(\{\|y\| > 1\}) = 0κ(x)({∥y∥>1})=0; f∗f^*f∗ is the kernel mean, and the sample law is the nnn-fold product of μ⊗κ\mu \otimes \kappaμ⊗κ. Every joint law disintegrates this way. The regret is measured against the optimal value min⁡z∈Zf∗(x)⊤z\min_{z\in\mathcal Z} f^*(x)^\top zminz∈Z​f∗(x)⊤z, which equals the paper's comparison with π∗\pi^*π∗. Natarajan dimension "at least η\etaη" is the existence of a shattered η\etaη-tuple. Infima are real infima over sets that are nonempty and bounded under the hypotheses. Probabilities in the noise condition are in [0,∞][0,\infty][0,∞], and (γδ/B)α(\gamma\delta/B)^\alpha(γδ/B)α is a real power.

The paper prints Theorem 7 with the distribution chosen before nnn; its proof builds the distribution from nnn, and the formal goal is stated in the proved order: for every nnn and every algorithm there is an instance. The hypothesis η≥1\eta \ge 1η≥1 excludes the empty case. No measurability is assumed of the algorithm. The algorithm must return a member of Π\PiΠ for every sample, and the instance must satisfy ∥Y∥≤1\|Y\| \le 1∥Y∥≤1, the optimality of some member of Π\PiΠ everywhere, and the noise condition. Dropping any of these would make the bound trivial or false, and none of them is relaxed.

A complete development needs finiteness of the extreme points of a polytope, projections onto convex hulls, products of finitely supported measures, and elementary binomial identities. The generic milestones (3–7) are reusable for other hypercube lower bounds. Proofs of individual milestones are welcome independently of the goal.

The companion missions Fast Rates for Contextual Linear Optimization 1 (the ERM upper bound, Theorem 6) and 3 (the estimate-then-optimize bound, Theorem 8) use the same model.

Selected references

  • Y. Hu, N. Kallus, X. Mao, Fast Rates for Contextual Linear Optimization, arXiv:2011.03030v3, 2021; Management Science 68(6), 2022. https://arxiv.org/abs/2011.03030
  • L. Devroye, G. Lugosi, Lower bounds in pattern recognition and learning, Pattern Recognition 28(7), 1995. https://doi.org/10.1016/0031-3203(95)00002-Z
  • A. B. Tsybakov, Optimal aggregation of classifiers in statistical learning, Annals of Statistics 32(1), 2004. https://doi.org/10.1214/aos/1079120131
  • B. K. Natarajan, On learning sets and functions, Machine Learning 4, 1989. https://doi.org/10.1007/BF00114804
  • O. El Balghiti, A. N. Elmachtoub, P. Grigas, A. Tewari, Generalization bounds in the predict-then-optimize framework, NeurIPS 2019. https://arxiv.org/abs/1905.11488
12 thms1 active userReviewed
Operations ResearchProbabilityTheoretical Computer Science·Captain: mikedeng1

Greed Works – Online Algorithms For Unrelated Machine Stochastic Scheduling 2: Greedy Assignment With Nominal Start Times Is (7.216 + 3.608Δ)h(Δ)-Competitive in the Online-Time ModelResearch Paper

Motivation

Scheduling jobs on parallel machines to minimize the total weighted completion time ∑jwjCj\sum_j w_jC_j∑j​wj​Cj​ is a basic model of service and computing systems. Two features of practice are hard to combine with performance guarantees: jobs are not known in advance but arrive over time, and their processing times are random, with only their distributions known when they arrive. Machines are moreover often unrelated: a job's processing time distribution depends arbitrarily on the machine, and some jobs cannot run on some machines at all.

Gupta, Moseley, Uetz and Xie (arXiv:1703.01634, Math. Oper. Res. 2020) give the first performance guarantee for this combination: a simple greedy policy whose expected cost is within a factor depending only on the variability of the processing times of the best nonanticipatory policy, which knows all jobs in advance.

Timeline. For deterministic jobs arriving over time, Hall, Schulz, Shmoys and Wein (1997) gave an 8-competitive algorithm on unrelated machines. Megow, Uetz and Vredeveld (2006) introduced the stochastic online model and analysed greedy policies on identical machines with modified release dates. Skutella, Sviridenko and Uetz (2016) gave LP-based approximation algorithms for offline stochastic scheduling on unrelated machines, with guarantees linear in the squared coefficient of variation. The present paper (conference version IPCO 2017) treats the online, stochastic, unrelated case with release dates.

Setting

There is a finite set MMM of machines and jobs j=1,…,nj=1,\dots,nj=1,…,n. Job jjj is released at an integer time rj≥0r_j\ge0rj​≥0, jobs are indexed so that r1≤⋯≤rnr_1\le\dots\le r_nr1​≤⋯≤rn​, and job jjj has weight wj≥0w_j\ge0wj​≥0. If job jjj runs on machine iii its processing time is a random variable PijP_{ij}Pij​ with values in {0,1,2,… }\{0,1,2,\dots\}{0,1,2,…}; on forbidden pairs the page writes E[Pij]=∞\mathbb E[P_{ij}]=\inftyE[Pij​]=∞, and every job has an allowed machine. On allowed pairs E[Pij]≥1\mathbb E[P_{ij}]\ge1E[Pij​]≥1 and the variance is finite. Different jobs are independent; one job's times on different machines need not be. The squared coefficient of variation is CV[Pij]2=Var⁡[Pij]/E[Pij]2\mathbb{CV}[P_{ij}]^2=\operatorname{Var}[P_{ij}]/\mathbb E[P_{ij}]^2CV[Pij​]2=Var[Pij​]/E[Pij​]2, and Δ\DeltaΔ bounds it on every allowed pair.

A policy decides, for each job, a machine and a start time; jobs are nonpreemptive and a machine runs one job at a time. It is nonanticipatory if its decisions up to time ttt use only what is known at ttt: the processing time of a job is learned when the job completes. OPT\mathsf{OPT}OPT is the least expected cost E[∑jwjCj]\mathbb E[\sum_jw_jC_j]E[∑j​wj​Cj​] of a nonanticipatory policy that knows all jobs, release dates and distributions in advance.

The greedy algorithm (§6.1) is first defined for deterministic times pijp_{ij}pij​. On machine iii job jjj gets the modified release date rij=max⁡{rj,c pij}r_{ij}=\max\{r_j,c\,p_{ij}\}rij​=max{rj​,cpij​}. On arrival, job jjj is assigned to a machine m(j)m(j)m(j) minimizing an explicit upper bound cost⁡(j→i)\operatorname{cost}(j\to i)cost(j→i) on the increase of the objective (Definition 4), and each machine runs WSPT (largest wk/pikw_k/p_{ik}wk​/pik​ first) among its jobs whose modified release dates have passed. Run on pij=E[Pij]p_{ij}=\mathbb E[P_{ij}]pij​=E[Pij​] with c=23c=\frac23c=32​ it yields the nominal schedule (m,s)(m,s)(m,s). The stochastic greedy policy (§6.3) keeps mmm and the nominal order on each machine, and starts the ℓ\ellℓth job on machine iii at Si,ℓ=max⁡{si,ℓ,Si,ℓ−1+Pi,ℓ−1}S_{i,\ell}=\max\{s_{i,\ell},S_{i,\ell-1}+P_{i,\ell-1}\}Si,ℓ​=max{si,ℓ​,Si,ℓ−1​+Pi,ℓ−1​}, never before its nominal start. Its expected cost is ALG\mathsf{ALG}ALG, and

h(Δ)=1+Δ2 (Δ≤1),h(Δ)=1+ΔΔ+1 (Δ≥1).h(\Delta)=1+\tfrac{\sqrt\Delta}2\ (\Delta\le1),\qquad h(\Delta)=1+\tfrac{\Delta}{\Delta+1}\ (\Delta\ge1).h(Δ)=1+2Δ​​ (Δ≤1),h(Δ)=1+Δ+1Δ​ (Δ≥1).

Formalization targets

Goal: Theorem 4

ALG  ≤  (7.216+3.608 Δ) h(Δ) OPT.\mathsf{ALG}\;\le\;(7.216+3.608\,\Delta)\,h(\Delta)\,\mathsf{OPT}.ALG≤(7.216+3.608Δ)h(Δ)OPT.

The goal is stated against every admissible nonanticipatory policy, for every tie-breaking of the greedy algorithm.

Milestones

  1. Lemma 6: the deterministic greedy schedule has ALG≤∑jcost⁡(j→m(j))\mathsf{ALG}\le\sum_j\operatorname{cost}(j\to m(j))ALG≤∑j​cost(j→m(j)).
  2. Lemma 7, feasibility: with αj=cost⁡(j→m(j))\alpha_j=\operatorname{cost}(j\to m(j))αj​=cost(j→m(j)), βi,t\beta_{i,t}βi,t​ the weight of jobs on machine iii released by ttt and unfinished at ttt, and the speed-scaled values (12), (αf/a,βf/b)(\alpha^f/a,\beta^f/b)(αf/a,βf/b) is feasible for the dual (Dr)(\mathrm D_r)(Dr​) when af≥2(2+c)af\ge2(2+c)af≥2(2+c), 1/c≤f(a−1)1/c\le f(a-1)1/c≤f(a−1), af≥baf\ge baf≥b.
  3. (13): zPr≤(1+Δ2)zSr≤(1+Δ2)OPTz^{P_r}\le(1+\frac\Delta2)z^{S_r}\le(1+\frac\Delta2)\mathsf{OPT}zPr​≤(1+2Δ​)zSr​≤(1+2Δ​)OPT for the time-indexed relaxations with release dates.
  4. Lemma 10: E[(X−μ)+]≤μΔ/2\mathbb E[(X-\mu)^+]\le\mu\sqrt\Delta/2E[(X−μ)+]≤μΔ​/2 for Δ≤1\Delta\le1Δ≤1 and ≤μΔ/(Δ+1)\le\mu\Delta/(\Delta+1)≤μΔ/(Δ+1) for Δ≥1\Delta\ge1Δ≥1.
  5. Lemma 8: E[Sj]≤h(Δ) sj\mathbb E[S_j]\le h(\Delta)\,s_jE[Sj​]≤h(Δ)sj​ for every job.

Significance

The result. Theorem 4 is the first constant-factor guarantee, for fixed Δ\DeltaΔ, for stochastic online scheduling on unrelated machines with release dates, and it is achieved by a combinatorial policy that needs no LP solution. For deterministic jobs (Δ=0\Delta=0Δ=0, h=1h=1h=1) it gives a deterministic 7.216-competitive algorithm (Theorem 3), improving the ratio 8 of Hall et al.; for exponential processing times (Δ=1\Delta=1Δ=1) the factor is (7.216+3.608)⋅32(7.216+3.608)\cdot\frac32(7.216+3.608)⋅23​. The order O(Δ)O(\Delta)O(Δ) matches earlier offline results on unrelated machines and online results on identical machines.

Formalizing it. The result is proved on paper and has no machine-checked proof. A formalization needs nonanticipatory policies with machine choice, a time-indexed LP relaxation with countably many variables and its weak duality, and a dual-fitting argument. It also exposes a gap: the second sentence of Lemma 7, the bound zDr(αf/a,βf/b)≥ALG/(7+1151)z^{D_r}(\alpha^f/a,\beta^f/b)\ge\mathsf{ALG}/(7+\frac{11}{51})zDr​(αf/a,βf/b)≥ALG/(7+5111​), fails as printed. On one machine with jobs r=(0,0,0,5)r=(0,0,0,5)r=(0,0,0,5), p=(54,54,1,3)p=(\frac54,\frac54,1,3)p=(45​,45​,1,3), w=(2,1,11000,1100)w=(2,1,\frac1{1000},\frac1{100})w=(2,1,10001​,1001​) the greedy schedule has ALG=6049600\mathsf{ALG}=\frac{6049}{600}ALG=6006049​ while zDr=88796400<ALG/(7+1151)=44713200z^{D_r}=\frac{8879}{6400}<\mathsf{ALG}/(7+\frac{11}{51})=\frac{4471}{3200}zDr​=64008879​<ALG/(7+5111​)=32004471​. The step ∑i,sβisf=ALG/f\sum_{i,s}\beta^f_{is}=\mathsf{ALG}/f∑i,s​βisf​=ALG/f counts a job at ⌊Ck/f⌋−⌈rk/f⌉+1\lfloor C_k/f\rfloor-\lceil r_k/f\rceil+1⌊Ck​/f⌋−⌈rk​/f⌉+1 integer slots, up to one more than Ck/fC_k/fCk​/f. A complete proof of the goal has to close this gap.

Difficulty

Three parts are hard. First, the comparison is with an optimal adaptive policy, so the lower bound has to come from a relaxation that every nonanticipatory policy satisfies in expectation. Writing the time-indexed LP for policies that choose machines at start time, idle deliberately and start at real times, and showing it is a relaxation, requires nonanticipation and job independence to decouple a job's machine from its own processing time. Second, the dual fitting runs through a speed-scaled instance with a non-integer speed f=236f=\frac{23}6f=623​, so the dual's integer slots and the schedule's real times do not align; this is where the printed objective bound breaks. Third, the obvious stochastic policy, WSEPT list scheduling without forced idleness, fails: Remark 2 of the paper gives an instance where it has E[C]=Ω(n)\mathbb E[C]=\Omega(n)E[C]=Ω(n) against O(1)O(1)O(1). The analysis therefore has to work with deliberately delayed starts.

Formalization scope

Machines are a Fintype, Nonempty type; jobs are Fin n in release order. Processing times are ℕ-valued measurable functions on a probability space, with finite second moment and mean at least one on eligible pairs, and with jobs independent (iIndepFun over jobs of the machine-indexed vectors). Forbidden pairs are an eligibility relation, never a zero or junk mean. Release dates are natural numbers: the relaxation of §6.2 has integer slots s≥rjs\ge r_js≥rj​, and (13) is false for a fractional release date. Weights are nonnegative; Δ\DeltaΔ is any upper bound on the squared coefficients of variation, which is at least as strong as the page's maximum.

OPT\mathsf{OPT}OPT is never written as an infimum. The goal and (13) quantify over every admissible policy: a map from realizations to machines and real start times that is feasible (eligible machine, start at or after release, disjoint half-open processing intervals per machine), nonanticipatory, and has integrable completion times. A non-integrable comparator would have Bochner integral 000 and would make the bound trivial or false; the integrability requirement rules this out. The greedy algorithm is a predicate on an outcome (assignment and start times): cost-minimal assignment, no start before the modified release date, no unnecessary idleness, and the WSPT priority. Every tie-breaking is covered, and the goal fixes c=23c=\frac23c=32​. The stochastic policy is the recursion Si,ℓ=max⁡{si,ℓ,Si,ℓ−1+Pi,ℓ−1}S_{i,\ell}=\max\{s_{i,\ell},S_{i,\ell-1}+P_{i,\ell-1}\}Si,ℓ​=max{si,ℓ​,Si,ℓ−1​+Pi,ℓ−1​} itself. LP feasibility includes summability of every series, and LP optima appear only through "for every feasible point there is a feasible point" statements.

A complete development needs: weak duality for (Pr)(\mathrm P_r)(Pr​)/(Dr)(\mathrm D_r)(Dr​) with countably many variables; the identity E[Cj]=CjS\mathbb E[C_j]=C^S_jE[Cj​]=CjS​ for the expected occupation variables of a nonanticipatory policy (reusable for any time-indexed stochastic scheduling relaxation); Lemma 10, a self-contained probability inequality; and the dual fitting. Contributions are welcome on each milestone and on a repaired objective bound for Lemma 7.

Selected references

  • V. Gupta, B. Moseley, M. Uetz, Q. Xie, Greed Works – Online Algorithms For Unrelated Machine Stochastic Scheduling, Mathematics of Operations Research, 2020; arXiv:1703.01634v4. https://arxiv.org/abs/1703.01634, https://doi.org/10.1287/moor.2019.0999
  • L. A. Hall, A. S. Schulz, D. B. Shmoys, J. Wein, Scheduling to minimize average completion time: off-line and on-line approximation algorithms, Mathematics of Operations Research 22:513–544, 1997. https://doi.org/10.1287/moor.22.3.513
  • N. Megow, M. Uetz, T. Vredeveld, Models and algorithms for stochastic online scheduling, Mathematics of Operations Research 31(3):513–525, 2006. https://doi.org/10.1287/moor.1060.0201
  • M. Skutella, M. Sviridenko, M. Uetz, Unrelated machine scheduling with stochastic processing times, Mathematics of Operations Research 41(3):851–864, 2016. https://doi.org/10.1287/moor.2015.0757
11 thms1 active userReviewed
Operations ResearchProbabilityTheoretical Computer Science·Captain: mikedeng1

Greed Works – Online Algorithms For Unrelated Machine Stochastic Scheduling 1: Greedy Assignment With WSEPT Sequencing Is (4 + 2Δ)-Competitive in the Online-List ModelResearch Paper

Motivation

Online scheduling asks for decisions before the full job set is known. In the online-list model, jobs are revealed one at a time and each must be assigned to a machine before the next job is revealed; the scheduler may choose the processing order after the list is complete. When processing durations are random and machines are unrelated, the same job may have different duration distributions on different machines. An assignment decision therefore affects both its own completion time and the waiting time of other jobs. Gupta, Moseley, Uetz and Xie study a greedy assignment rule for this setting and prove a bound against a scheduler that knows all distributions and all jobs in advance but cannot see future processing-time realizations (arXiv:1703.01634v4, §§2–5). This comparison is relevant to systems that must commit work to heterogeneous servers as requests arrive, while service times remain uncertain.

Setting

Let MMM be a nonempty finite set of machines and J={0,…,n−1}J=\{0,\ldots,n-1\}J={0,…,n−1} a finite set of jobs indexed by their presentation order. Job jjj has a nonnegative weight wjw_jwj​. An eligibility relation E⊆M×JE\subseteq M\times JE⊆M×J records which machines can process each job; every job has at least one eligible machine. For an eligible pair (i,j)(i,j)(i,j), PijP_{ij}Pij​ is a nonnegative integer-valued random processing time with mean μij=E[Pij]≥1\mu_{ij}=\mathbb E[P_{ij}]\ge1μij​=E[Pij​]≥1 and a finite second moment. The vector (Pij)i∈M(P_{ij})_{i\in M}(Pij​)i∈M​ for one job may have dependence across machines, but the vectors for different jobs are independent. A parameter Δ≥0\Delta\ge0Δ≥0 bounds Var⁡(Pij)/μij2\operatorname{Var}(P_{ij})/\mu_{ij}^{2}Var(Pij​)/μij2​ for all eligible pairs. These conventions follow the standing model and coefficient-of-variation bound in the paper (§2, pp. 4–6 and Definition 3, p. 7).

The greedy assignment sends each newly presented job to an eligible machine that minimizes the instantaneous increase in expected total weighted completion time. For a fixed machine, jobs are sequenced by decreasing ratio wj/μijw_j/\mu_{ij}wj​/μij​, with job index breaking equal-ratio ties. This is the weighted shortest expected processing time, or WSEPT, order. The algorithm runs each machine continuously from time zero in that order. Its expected objective, ALG\mathrm{ALG}ALG, is the expectation of ∑jwjCj\sum_j w_j C_j∑j​wj​Cj​ for the resulting realized completion times, not a value assigned by a closed formula (§4, pp. 8–9).

A comparator policy can choose machines and start times as earlier jobs finish. It must schedule nonpreemptively, use only information available when a decision is made, and have integrable completion times. It may know the full job set and every processing-time distribution before time zero, but not the future realizations. Write CjΠC_j^\PiCjΠ​ for its realized completion time and OPT\mathrm{OPT}OPT for the least expected weighted completion cost among such policies when an optimum exists (§2.1, pp. 5–6).

Formalization targets

The goal is the paper's Theorem 1 (p. 11). It compares every eligible-minimum greedy assignment, including every tie choice, with every admissible nonanticipatory comparator Π\PiΠ:

ALG≤(4+2Δ) E ⁣[∑j∈JwjCjΠ].\mathrm{ALG}\le(4+2\Delta)\,\mathbb E\!\left[\sum_{j\in J}w_j C_j^\Pi\right].ALG≤(4+2Δ)E​j∈J∑​wj​CjΠ​​.

This universal comparison expresses ALG≤(4+2Δ)OPT\mathrm{ALG}\le(4+2\Delta)\mathrm{OPT}ALG≤(4+2Δ)OPT without requiring an optimizer to exist. The milestone list contains the paper's integer tail identities (Lemma 1), the completion-time expression for one job (Lemma 9), the comparison of two time-indexed relaxations and its policy consequence (Lemma 2 and Corollary 1), and the three dual-variable claims used around the greedy algorithm (Lemmas 3–5). Their printed indices and formulas are attached to the proposal in proof order.

Significance

The result supplies a performance guarantee for a combinatorial online algorithm on heterogeneous machines with random durations. The factor varies explicitly with the processing-time variability: for deterministic eligible processing times, Δ=0\Delta=0Δ=0 and the bound is 444; greater permitted variability increases it linearly. Forbidden machine-job pairs are included, so the claim applies when some jobs have a restricted set of possible servers. The comparator may postpone machine selection until processing begins and adapt to completed work, so the bound is not limited to fixed-assignment schedules (Theorem 1 and §2.1).

The theorem is proved in the cited paper. In this proposal, the Lean statements and definitions have compiled, while the theorem modules still contain sorry; the remaining formalization work is to supply machine-checked proofs. A completed development would contribute reusable definitions for nonanticipatory unrelated-machine policies, time-indexed stochastic scheduling relaxations, and comparisons with countably many LP variables. Those interfaces can support later scheduling results that use the same information structure.

Difficulty

The most immediate comparison treats the greedy assignment costs as dual variables. For this paper's linear program, the direct halved dual point is feasible, but the resulting dual objective is zero, so feasibility alone gives no useful upper bound on ALG\mathrm{ALG}ALG (Lemmas 3–4 and the beginning of §5). The random processing times also require care when translating occupancy probabilities into completion costs: the time-indexed expression uses both first and second moments, and a policy's start decision must not depend on the job's unobserved duration. The proof must reconcile the greedy order, those probability identities, and a comparison with policies that may change machines and start times adaptively.

Formalization scope

The Lean model uses Fin n in arrival order and permits n=0n=0n=0; the empty instance gives a trivial inequality. The machine type is finite and nonempty. Processing times are natural-valued random variables, while policy start times are real. Processing intervals are half-open, so a zero-duration job occupies no machine time. An ineligible pair is represented by the eligibility relation, never by a zero mean. Each eligible processing time has finite second moment and mean at least one; Δ\DeltaΔ is any nonnegative upper bound on squared coefficients of variation. The paper leaves wj≥0w_j\ge0wj​≥0 implicit; it is included because negative weights invalidate the comparison. Dependence across machines for the same job is allowed, and the entire time vector is independent across jobs.

Nonanticipation is stated by comparing two realizations that agree on the durations observed by time ttt and on the fact that running jobs have not yet completed. Admissible comparator policies must work for every realization and have integrable completion times, preventing a default zero value for a nonintegrable integral. The goal quantifies over all such policies, so it cannot be made true by choosing a restricted benchmark. The greedy predicate reads only earlier assignments and admits every eligible minimizing machine; it does not fix a favorable tie choice.

The LP variables yijsy_{ijs}yijs​ are real and nonnegative, supported on eligible pairs, and their countable sums and time-weighted sums must converge. LP comparisons use feasible witnesses instead of a real infimum, whose value would be misleading for an empty feasible set. A source-level discretization issue is isolated in Lemma 4: ∑sβis\sum_s\beta_{is}∑s​βis​ counts whole slots up to a nominal completion time, so the printed second equality requires integer means. Its first equality and the goal do not take that condition. A complete proof needs measurable-policy reasoning, tail-sum identities, summability arguments, WSEPT order properties and dual weak duality. Definitions and lemmas that isolate those components are welcome.

Selected references

  • V. Gupta, B. Moseley, M. Uetz and Q. Xie, Greed Works – Online Algorithms For Unrelated Machine Stochastic Scheduling, accepted version of Mathematics of Operations Research, 2020. arXiv:1703.01634v4; DOI:10.1287/moor.2019.0999.
11 thms1 active userReviewed
Graph TheoryOperations ResearchOptimization·Captain: mikedeng1

Minimum Cost Flows in Graphs with Unit Capacities: When the Second Phase of a Cost Scale Starts, at Most 3λ Excess Is Left for λ = √m and 36λ for λ = n^(2/3)Research Paper

Motivation

Minimum-cost flow is a standard model for routing a quantity through a network while paying a cost on each arc. Unit capacities arise when each input arc can carry at most one unit; they include flow formulations of several matching and assignment problems. Goldberg, Kaplan, Hed and Tarjan analyze a cost-scaling algorithm for this setting. Their second-phase argument depends on a bound on the excess left in a pseudoflow when every excess vertex has undergone enough potential increases. This mission formalizes that bound, Lemma 5 of their STACS 2015 paper. The published paper gives an O(min⁡{m1/2,n2/3} mlog⁡(nC))O(\min\{m^{1/2},n^{2/3}\}\,m\log(nC))O(min{m1/2,n2/3}mlog(nC)) running-time bound for its unit-capacity algorithm; the running-time statement itself is outside this mission.

Setting

A network has a finite vertex set VVV and an arc set EEE that contains both orientations of each input arc. Write n=∣V∣n=|V|n=∣V∣. Every input arc has capacity one; its reverse has capacity zero and opposite cost. Write mmm for the number of input arcs, so ∣E∣=2m|E|=2m∣E∣=2m. The paper assumes no parallel or antiparallel input arcs, allowing an arc to be identified by its ordered endpoints. Costs are antisymmetric between orientations.

A pseudoflow fff assigns an antisymmetric quantity to each arc and respects its capacity. Its excess at vvv is the sum ef(v)e_f(v)ef​(v) of incoming flow. Positive excess means that vvv has flow to discharge; negative excess is a deficit. A circulation is a pseudoflow with zero excess at every vertex. An arc is residual for fff when its residual capacity uf(v,w)=u(v,w)−f(v,w)u_f(v,w)=u(v,w)-f(v,w)uf​(v,w)=u(v,w)−f(v,w) is positive. For vertex prices ppp, its reduced cost is cp(v,w)=c(v,w)−p(v)+p(w)c_p(v,w)=c(v,w)-p(v)+p(w)cp​(v,w)=c(v,w)−p(v)+p(w). A pseudoflow is ε\varepsilonε-optimal at ppp when every residual arc has reduced cost at least −ε-\varepsilon−ε. These are the paper's conventions on p. 408.

At the start of a cost scale, let f′f'f′ be a circulation that is 2ε2\varepsilon2ε-optimal at prices p′p'p′. During the scale, fff is a pseudoflow that is ε\varepsilonε-optimal at prices ppp. The potential increase in scale units is d(v)=(p(v)−p′(v))/εd(v)=(p(v)-p'(v))/\varepsilond(v)=(p(v)−p′(v))/ε. Put λ=min⁡{m,n2/3}\lambda=\min\{\sqrt m,n^{2/3}\}λ=min{m​,n2/3}. The second phase begins when every excess vertex has d(v)≥λd(v)\ge\lambdad(v)≥λ; the paper also states that every deficit then has d(v)=0d(v)=0d(v)=0 and that the levels d(v)d(v)d(v) are integral. The remaining excess is X(f)=∑v:ef(v)>0ef(v)X(f)=\sum_{v:e_f(v)>0}e_f(v)X(f)=∑v:ef​(v)>0​ef​(v).

The comparison graph has arcs E+={(v,w)∈E:f′(v,w)>f(v,w)}E^+=\{(v,w)\in E:f'(v,w)>f(v,w)\}E+={(v,w)∈E:f′(v,w)>f(v,w)} and vertices VVV. An excess–deficit cut Y⊆E+Y\subseteq E^+Y⊆E+ meets every path in this graph from an excess vertex to a deficit vertex. Thus a cut is measured among comparison arcs, not among arbitrary arcs of the original network. Lemmas 3 and 4 of the paper establish the level and cut bounds on which the excess estimate rests.

Formalization targets

The goal is Lemma 5 on p. 410, stated with both constants obtained in its proof. Under the cost-scale conditions above, and the divisibility conventions in footnotes 5–6, its two cases are

λ=m⟹X(f)≤3λ,\lambda=\sqrt m\quad\Longrightarrow\quad X(f)\le 3\lambda,λ=m​⟹X(f)≤3λ, λ=n2/3⟹X(f)≤36λ.\lambda=n^{2/3}\quad\Longrightarrow\quad X(f)\le36\lambda.λ=n2/3⟹X(f)≤36λ.

The second case also assumes that six divides the integer λ\lambdaλ; the first uses divisibility by three. The goal keeps the two conclusions separate so that the stronger 3λ3\lambda3λ bound is retained when m\sqrt mm​ determines the scale. Its milestone list follows the paper: Lemma 3 bounds d(v)−d(w)d(v)-d(w)d(v)−d(w) by three on an arc of E+E^+E+; Lemma 4 bounds total excess by a cut; and the proof of Lemma 5 supplies the cut bands and the two size estimates. The bands use the exact three-level and six-level ranges printed on p. 410.

Significance

The result connects a condition on price increases to a numerical bound on the amount of flow still needing discharge. Combined with the paper's time per unit of remaining excess, this is the bound needed for its second-phase running-time estimate. The two regimes show how the same cost-scale state can be controlled by either the number of input arcs or the number of vertices. Without the excess bound, the paper's argument gives no comparable limit on the remaining work at the phase transition.

The paper proves Lemma 5, but this particular unit-capacity excess argument has no matching theorem in the platform catalog checked for this mission. The network, circulation and pseudoflow interfaces are already published as reusable definitions from earlier cost-scaling work. A completed machine-checked development would add the cut argument and band counts to those interfaces. Those facts can also support other analyses that compare an entry circulation with an intermediate pseudoflow; this mission does not claim to formalize the algorithm's entire execution or its running-time model.

Difficulty

The main obstacle is linking local price inequalities to a global bound on excess. A reduced-cost inequality controls a single residual arc, while excess may move through a path and is measured over many vertices. The comparison graph G+G^+G+ is essential: on a residual arc outside E+E^+E+, the level difference used by Lemma 3 is not available. Counting all residual arcs as if they were comparison arcs would therefore not justify the cut bound. There is a separate counting issue: the represented network has both directions of each input arc, so using ∣E∣|E|∣E∣ for mmm changes the constant in the square-root case.

Formalization scope

Lean represents the network by a finite symmetric arc set with real capacities and antisymmetric real costs. A local unit-capacity predicate marks each reverse pair as capacities (1,0)(1,0)(1,0) or (0,1)(0,1)(0,1); mmm counts the capacity-one orientations. Flows are real valued as on p. 408. Integer costs on that page are generalized to real costs because Lemmas 3–5 use only antisymmetry and reduced-cost inequalities. No result here depends on cost integrality.

The formalization takes the cost-scale state as hypotheses: ε>0\varepsilon>0ε>0, entry circulation and 2ε2\varepsilon2ε-optimality, current ε\varepsilonε-optimality, integral levels, level at least λ\lambdaλ at excess vertices, and zero level at deficits. It takes λ\lambdaλ as a natural number equal to min⁡{m,n2/3}\min\{\sqrt m,n^{2/3}\}min{m​,n2/3} and records the paper's divisibility simplifications. The indexed pigeonhole milestones also require λ>0\lambda>0λ>0 so their index ranges are nonempty; the goal includes the zero-arc boundary without requiring an index. Cuts use reachability after deleting a subset of E+E^+E+, equivalent to meeting every excess-to-deficit path. The proof of Lemma 5 prints “cut in GfG_fGf​”; the formalized claim uses G+G^+G+, the graph for which Lemma 3 gives the required level control and Lemma 4 states its bound.

The definition layer, path-cut lemma and finite counting arguments are the reusable infrastructure. Contributions that establish these exact statements, their necessary elementary flow identities, or a machine-checked goal proof are in scope. Counting mmm with reverse arcs, allowing cuts outside E+E^+E+, using arbitrary real levels in place of integral ones, or replacing both cases by the weaker 36λ36\lambda36λ bound does not express this target.

Selected references

  • A. V. Goldberg, H. Kaplan, S. Hed and R. E. Tarjan, Minimum Cost Flows in Graphs with Unit Capacities, 32nd International Symposium on Theoretical Aspects of Computer Science, LIPIcs 30, pp. 406–419, 2015. doi:10.4230/LIPIcs.STACS.2015.406.
9 thms1 active userReviewed
Convex OptimizationOptimization·Captain: mikedeng1

Faster First-Order Primal-Dual Methods for Linear Programming using Restarts and Sharpness I: The LP Saddle-Point Problem Is 1/(H(K)√(1+4R²))-Sharp on Every Ball W_R(0)Research Paper

Motivation

First-order primal-dual methods (PDHG, extragradient, ADMM) are the engine of a new generation of large-scale linear programming solvers, such as PDLP, because they need only matrix–vector products with the constraint matrix. Their textbook guarantees for convex–concave saddle-point problems are sublinear, O(1/t)O(1/t)O(1/t) for the averaged iterate, while in practice restarted variants converge linearly on linear programs. Applegate, Hinder, Lu and Lubin (arXiv:2105.12715v4, Math. Program. 2022) explain this gap through a sharpness condition on the normalized duality gap, and show that restarts turn sharpness into linear convergence.

This mission formalizes the part of that paper which shows that linear programming is sharp: Lemma 5 (p. 15), together with the bound of the KKT residual by the normalized duality gap (Lemma 4, p. 14) and the bilinear results of §3.2. The restart theorems that consume this property are the subject of the companion missions II and III of this series.

Setting

Let A∈Rm×nA\in\mathbb R^{m\times n}A∈Rm×n, b∈Rmb\in\mathbb R^mb∈Rm, c∈Rnc\in\mathbb R^nc∈Rn. The standard-form LP and its dual are

min⁡x∈Rnc⊤x  s.t. Ax=b, x≥0(17),max⁡y∈Rmb⊤y  s.t. A⊤y≤c(18),\min_{x\in\mathbb R^n} c^\top x\ \text{ s.t. } Ax=b,\ x\ge0 \qquad (17), \qquad\quad \max_{y\in\mathbb R^m} b^\top y\ \text{ s.t. } A^\top y\le c \qquad (18),x∈Rnmin​c⊤x  s.t. Ax=b, x≥0(17),y∈Rmmax​b⊤y  s.t. A⊤y≤c(18),

and their Lagrangian form is the saddle-point problem

min⁡x≥0 max⁡y∈Rm L(x,y)=c⊤x+y⊤b−y⊤Ax.(19)\min_{x\ge0}\ \max_{y\in\mathbb R^m}\ L(x,y)=c^\top x+y^\top b-y^\top Ax. \qquad (19)x≥0min​ y∈Rmmax​ L(x,y)=c⊤x+y⊤b−y⊤Ax.(19)

A primal-dual point is z=(x,y)z=(x,y)z=(x,y), the feasible region is Z={x≥0}×RmZ=\{x\ge0\}\times\mathbb R^mZ={x≥0}×Rm, and Z⋆Z^\starZ⋆ is the set of saddle points of LLL on ZZZ. On Rn+m\mathbb R^{n+m}Rn+m the norm is the Euclidean norm ∥(x,y)∥=∥x∥22+∥y∥22\|(x,y)\|=\sqrt{\|x\|_2^2+\|y\|_2^2}∥(x,y)∥=∥x∥22​+∥y∥22​​, and dist⁡(z,Z⋆)=inf⁡z^∈Z⋆∥z−z^∥\operatorname{dist}(z,Z^\star)=\inf_{\hat z\in Z^\star}\|z-\hat z\|dist(z,Z⋆)=infz^∈Z⋆​∥z−z^∥.

For r>0r>0r>0 let Wr(z)={z^∈Z:∥z−z^∥≤r}W_r(z)=\{\hat z\in Z:\|z-\hat z\|\le r\}Wr​(z)={z^∈Z:∥z−z^∥≤r}. The normalized duality gap (4a) is

ρr(z)=sup⁡z^∈Wr(z)L(x,y^)−L(x^,y)r.\rho_r(z)=\sup_{\hat z\in W_r(z)}\frac{L(x,\hat y)-L(\hat x,y)}{r}.ρr​(z)=z^∈Wr​(z)sup​rL(x,y^​)−L(x^,y)​.

The problem is α\alphaα-sharp on a set S⊆ZS\subseteq ZS⊆Z (Definition 1) if α dist⁡(z,Z⋆)≤ρr(z)\alpha\,\operatorname{dist}(z,Z^\star)\le\rho_r(z)αdist(z,Z⋆)≤ρr​(z) for all z∈Sz\in Sz∈S and all r∈(0,diam⁡(S)]r\in(0,\operatorname{diam}(S)]r∈(0,diam(S)].

The KKT system (20) collects primal feasibility, dual feasibility and a nonpositive duality gap as Kz≥hKz\ge hKz≥h, with

K=(I0−A0A00−A⊤−c⊤b⊤),h=(0−bb−c0).K=\begin{pmatrix} I & 0\\ -A & 0\\ A & 0\\ 0 & -A^\top\\ -c^\top & b^\top\end{pmatrix},\qquad h=\begin{pmatrix}0\\-b\\b\\-c\\0\end{pmatrix}.K=​I−AA0−c⊤​000−A⊤b⊤​​,h=​0−bb−c0​​.

The KKT residual ∥(h−Kz)+∥\|(h-Kz)^+\|∥(h−Kz)+∥ is the Euclidean norm of the componentwise positive part, a standard stopping criterion of LP solvers. A Hoffman constant H(K)H(K)H(K) (Hoffman 1952) is a constant with

dist⁡(z,Z⋆)≤H(K) ∥(h−Kz)+∥for all z∈Rn+m.(26)\operatorname{dist}(z,Z^\star)\le H(K)\,\|(h-Kz)^+\|\quad\text{for all } z\in\mathbb R^{n+m}. \qquad (26)dist(z,Z⋆)≤H(K)∥(h−Kz)+∥for all z∈Rn+m.(26)

Formalization targets

Goal: Lemma 5 (Sharpness of linear programming), p. 15

If (19) has a solution and H(K)>0H(K)>0H(K)>0 satisfies (26), then for every R∈(0,∞)R\in(0,\infty)R∈(0,∞) the problem (19) is α\alphaα-sharp on WR(0)={z∈Z:∥z∥≤R}W_R(0)=\{z\in Z:\|z\|\le R\}WR​(0)={z∈Z:∥z∥≤R} with

α=1H(K)1+4R2.\alpha=\frac{1}{H(K)\sqrt{1+4R^2}}.α=H(K)1+4R2​1​.

Milestones (proof of Lemma 4, p. 14)

With v=((−c+A⊤y)+, b−Ax)v=\big((-c+A^\top y)^+,\ b-Ax\big)v=((−c+A⊤y)+, b−Ax):

(22)ρr(z)≥∥v∥(z∈Z, r>0),(25)ρr(z)≥1R (c⊤x−b⊤y)+(z∈WR(0), 0<r≤R),\text{(22)}\quad \rho_r(z)\ge\|v\|\quad (z\in Z,\ r>0),\qquad\qquad \text{(25)}\quad \rho_r(z)\ge\tfrac1R\,(c^\top x-b^\top y)^+\quad (z\in W_R(0),\ 0<r\le R),(22)ρr​(z)≥∥v∥(z∈Z, r>0),(25)ρr​(z)≥R1​(c⊤x−b⊤y)+(z∈WR​(0), 0<r≤R),

and their combination, Lemma 4:

∥(h−Kz)+∥≤ρr(z)1+R2(R>0, r∈(0,R], z∈WR(0)).\|(h-Kz)^+\|\le\rho_r(z)\sqrt{1+R^2}\qquad (R>0,\ r\in(0,R],\ z\in W_R(0)).∥(h−Kz)+∥≤ρr​(z)1+R2​(R>0, r∈(0,R], z∈WR​(0)).

Companion statements

  • §3.3, p. 13: if (17) and (18) are feasible, Z⋆={z:Kz≥h}Z^\star=\{z: Kz\ge h\}Z⋆={z:Kz≥h}, so that (26) is Hoffman's bound for a polyhedron.
  • Lemma 2, p. 12: for S={z:Hz=h}≠∅S=\{z:Hz=h\}\neq\emptysetS={z:Hz=h}=∅, dist⁡(S,z)≤∥Hz−h∥/σmin⁡+(H)\operatorname{dist}(S,z)\le\|Hz-h\|/\sigma^+_{\min}(H)dist(S,z)≤∥Hz−h∥/σmin+​(H).
  • Lemma 3, p. 12: the unconstrained bilinear problem L(x,y)=c⊤x−y⊤Ax+b⊤yL(x,y)=c^\top x-y^\top Ax+b^\top yL(x,y)=c⊤x−y⊤Ax+b⊤y on Rn+m\mathbb R^{n+m}Rn+m is σmin⁡+(A)\sigma^+_{\min}(A)σmin+​(A)-sharp on all of Rn+m\mathbb R^{n+m}Rn+m; with the identities (14) and (15) from its proof.

Significance

Lemma 5 is the input that makes the paper's restart theorems apply to linear programming: combined with Theorems 1 and 2 (missions II and III), it yields linear convergence of restarted PDHG, extragradient and ADMM on any feasible LP, with a rate governed by the Hoffman constant of the KKT system. Lemma 4 is of independent use: it shows that the normalized duality gap, which restart schemes can monitor, controls the KKT residual that solvers report.

The results are proved in the paper. As far as the platform and Mathlib show, none of them is formalized: there is no normalized duality gap, no primal-dual sharpness and no Hoffman bound for KKT systems in Mathlib. The remaining work is to formalize the paper's proofs. Strong duality for standard-form LP, needed for the §3.3 companion, exists on the platform (Polyhedral.lp_strong_duality, LinearOptimization.lp_strong_duality) in a different form.

Difficulty

The sharpness inequality relates two quantities of different nature: dist⁡(z,Z⋆)\operatorname{dist}(z,Z^\star)dist(z,Z⋆) is a geometric distance to an implicitly defined polyhedron, while ρr(z)\rho_r(z)ρr​(z) is a supremum of a bilinear gap over a truncated ball Wr(z)W_r(z)Wr​(z) that is cut by the constraint x≥0x\ge0x≥0. The unconstrained argument of Lemma 3, which identifies ρr(z)\rho_r(z)ρr​(z) with ∥F(z)∥\|F(z)\|∥F(z)∥, fails once x≥0x\ge0x≥0 is imposed, because the ball is no longer symmetric around zzz and the maximizing direction may leave ZZZ. Control of the duality-gap row of the KKT system also depends on the size of zzz, which is why the constant degrades with RRR and sharpness is claimed only on bounded sets WR(0)W_R(0)WR​(0). In Lean the normalized duality gap is an extended-real supremum over a set defined by a non-default norm, so each lower bound requires exhibiting a feasible point of Wr(z)W_r(z)Wr​(z) and computing its gap.

Formalization scope

Vectors live in EuclideanSpace ℝ (Fin n) and EuclideanSpace ℝ (Fin m), and a primal-dual point is a pair. The norm on pairs is the L2L^2L2 norm of WithLp 2, eucl. The norm instance of the bare product type is the sup norm and is not used: with the sup norm the constants 1+R2\sqrt{1+R^2}1+R2​ and 1+4R2\sqrt{1+4R^2}1+4R2​ would describe different statements. ρr\rho_rρr​ and diam⁡\operatorname{diam}diam are suprema in EReal. dist⁡(z,Z⋆)\operatorname{dist}(z,Z^\star)dist(z,Z⋆) is a real infimum, and every statement that uses it assumes Z⋆≠∅Z^\star\neq\emptysetZ⋆=∅ ("(19) has a solution"). Z⋆Z^\starZ⋆ is defined as the set of saddle points of LLL on ZZZ. KKK and hhh are encoded row block by row block exactly as (20) prints them; the page's "A⊤y≥cA^\top y\ge cA⊤y≥c" in (18) is a misprint for A⊤y≤cA^\top y\le cA⊤y≤c, which is what (20) encodes.

The Hoffman constant enters only through its defining property (26), quantified over all z∈Rn+mz\in\mathbb R^{n+m}z∈Rn+m, together with H(K)>0H(K)>0H(K)>0. Lemma 5 is stated for every such constant. Sharpness follows Definition 1 literally, with r∈(0,diam⁡(WR(0))]r\in(0,\operatorname{diam}(W_R(0))]r∈(0,diam(WR​(0))]; replacing that range by (0,2R](0,2R](0,2R] would be a different statement. A formalization that restricts (26) to z∈Zz\in Zz∈Z, uses the sup norm, or takes a real-valued supremum for ρr\rho_rρr​ would trivialize or change the goal and is ruled out. σmin⁡+(H)\sigma^+_{\min}(H)σmin+​(H) is the least σ>0\sigma>0σ>0 with σ2\sigma^2σ2 an eigenvalue of H⊤HH^\top HH⊤H; its Lean value for H=0H=0H=0 is 000.

Contributions welcome: proofs of (22) and (25), whose combination gives Lemma 4 and then Lemma 5; general lemmas on suprema over Euclidean balls in WithLp 2 products; and a Hoffman-type existence theorem for {Kz≥h}\{Kz\ge h\}{Kz≥h}, which this mission does not pose. The bilinear statements (Lemmas 2–3) need a minimum-nonzero-singular-value API and are reusable beyond this mission.

Selected references

  • D. Applegate, O. Hinder, H. Lu, M. Lubin, Faster First-Order Primal-Dual Methods for Linear Programming using Restarts and Sharpness, Math. Program. (2022); cited from arXiv:2105.12715v4. https://arxiv.org/abs/2105.12715 · https://doi.org/10.1007/s10107-022-01901-9
  • A. J. Hoffman, On approximate solutions of systems of linear inequalities, J. Res. Natl. Bur. Stand. 49 (1952) 263–265. https://doi.org/10.6028/jres.049.027
  • D. Applegate, M. Díaz, O. Hinder, H. Lu, M. Lubin, B. O'Donoghue, W. Schudy, Practical Large-Scale Linear Programming using Primal-Dual Hybrid Gradient, NeurIPS 2021. https://arxiv.org/abs/2106.04756
  • A. Chambolle, T. Pock, A first-order primal-dual algorithm for convex problems with applications to imaging, J. Math. Imaging Vis. 40 (2011) 120–145. https://doi.org/10.1007/s10851-010-0251-1
7 thms1 active userReviewed
Operations ResearchTheoretical Computer Science·Captain: mikedeng1

Online Primal-Dual Algorithms for Covering and Packing 2: The Phased Online Fractional Covering Scheme Covers Every Constraint to 1/B at Cost at Most 8 log(2n)/B Times the OptimumResearch Paper

Motivation

Online covering models decisions that must be made as requirements arrive. A planner knows the cost of each available resource, but learns the requirements one at a time and cannot undo an allocation already made. This occurs, for example, when requests for network service or elements needing coverage appear over time. The central question is how much more an online fractional allocation can cost than a best allocation chosen after all requirements are known. Buchbinder and Naor study this question through a paired covering and packing linear program and give an online scheme for arbitrary nonnegative constraint coefficients, not only incidence coefficients in {0,1}\{0,1\}{0,1} (Buchbinder and Naor, 2009, §§2–4).

The result targeted here is Theorem 4.1 of that paper. It concerns the fractional covering scheme of §4, before the separate extension to box constraints in §4.1. The theorem is a statement about the particular phased scheme, so the scheme's state and update rule form part of the mathematical setting. The online requirement also matters: after each new constraint, earlier allocations remain in force (Buchbinder and Naor, 2009, pp. 4, 8).

Setting

Let III be a finite nonempty set of nnn primal variables and let k=0,…,m−1k=0,\ldots,m-1k=0,…,m−1 index constraints in arrival order. The known cost of variable iii is ci>0c_i>0ci​>0. Constraint kkk has coefficient aik≥0a_{ik}\ge0aik​≥0 for variable iii; when it arrives, the online algorithm learns its coefficients. The normalized offline covering problem minimizes ∑icizi\sum_i c_i z_i∑i​ci​zi​ over zi≥0z_i\ge0zi​≥0 subject to ∑iaikzi≥1\sum_i a_{ik}z_i\ge1∑i​aik​zi​≥1 for every constraint considered. Its packing dual maximizes ∑kyk\sum_k y_k∑k​yk​ over yk≥0y_k\ge0yk​≥0 subject to ∑kaikyk≤ci\sum_k a_{ik}y_k\le c_i∑k​aik​yk​≤ci​ for each iii. The finite data and sign conventions are those of Figure 1 (Buchbinder and Naor, 2009, pp. 3–4).

The scheme accepts a parameter B>0B>0B>0 and asks for coverage only to 1/B1/B1/B. It maintains a sequence of phases. The first phase bound α1\alpha_1α1​ is B−1B^{-1}B−1 times the least ratio ci/ai0c_i/a_{i0}ci​/ai0​ among the positive coefficients of the first constraint. The bound doubles at a restart. Within a phase, the vector yyy begins at zero, the initial primal allocation is xi=α/(2nci)x_i=\alpha/(2nc_i)xi​=α/(2nci​), and the allocation subsequently follows the exponential expression on page 8. The current phase may need to process constraints already seen in earlier phases. Old phase vectors are retained: the actual online allocation is the coordinatewise maximum of the current and all finished phase vectors. Thus resetting a phase does not retract an earlier allocation (Buchbinder and Naor, 2009, p. 8).

Formalization targets

Theorem 4.1: coverage and competitive cost

For every nonempty arrival prefix of length JJJ, a run of the phased scheme exists. Every completed run has output xxx satisfying

∑iaikxi≥1B(k<J),\sum_i a_{ik}x_i\ge\frac1B\qquad(k<J),i∑​aik​xi​≥B1​(k<J),

and, for every nonnegative offline comparison vector zzz satisfying ∑iaikzi≥1\sum_i a_{ik}z_i\ge1∑i​aik​zi​≥1 for each k<Jk<Jk<J,

∑icixi≤8log⁡(2n)B∑icizi.\sum_i c_i x_i\le\frac{8\log(2n)}{B}\sum_i c_i z_i.i∑​ci​xi​≤B8log(2n)​i∑​ci​zi​.

The paper states the ratio as O(log⁡n/B)O(\log n/B)O(logn/B). Its closing display on page 9 yields the explicit factor 8log⁡(2n)/B8\log(2n)/B8log(2n)/B used here. The benchmark uses the unscaled covering constraints of Figure 1; replacing them with constraints at 1/B1/B1/B would change the guarantee (Buchbinder and Naor, 2009, Theorem 4.1 and proof, pp. 8–9).

The four claims within its proof

The milestone list follows the four claims printed under Theorem 4.1. A finished phase has dual objective at least Bαr/(2log⁡(2n))B\alpha_r/(2\log(2n))Bαr​/(2log(2n)); every phase dual vector satisfies the packing constraints; the sum of phase primal costs through the current phase is less than 2αr2\alpha_r2αr​; and the actual allocation covers every arrived constraint to 1/B1/B1/B. These are statements about the phase records of the scheme, including a partly processed round that ends in a restart (Buchbinder and Naor, 2009, pp. 8–9).

Significance

The theorem gives a cost guarantee for a general online covering problem while allowing arbitrary nonnegative coefficients. It separates the cost paid by an allocation from the level of coverage requested through BBB. Its four claims identify the quantities that make the guarantee meaningful: the dual vector certifies a lower bound for an offline comparison, the phase costs control the online expenditure, and the output remains feasible as constraints arrive. The same primal–dual setting is used elsewhere in the paper for online packing and for applications of covering and packing methods (Buchbinder and Naor, 2009, §§3–5).

The paper proves Theorem 4.1. This mission asks for a machine-checked account of the known result and of the scheme to which it applies. A published formal restatement, OnlinePrimalDual.GeneralPacking.theorem14_3, already gives a ratio conclusion under an assumed ratio bound and an assumed weak-duality inequality; those hypotheses contain the main work needed here. The present goal instead quantifies over runs generated by the update rule and asks both that a run exists and that all such runs satisfy the guarantees. The published general instance definition is reused, so the finite covering and packing data have a shared interface.

Difficulty

An online allocation cannot be evaluated solely from its final vector. A phase restart resets the working dual vector and restarts the constraint scan, while allocations from completed phases still affect the actual output. A statement about arbitrary vectors with a presumed cost bound would leave out this behavior. The continuous update also requires a precise stopping event when coverage or the phase cost reaches its threshold. The cost guarantee is sensitive to which condition wins if both occur together, and to whether a phase can restart indefinitely. Those details must be represented before any claim about completed phases has the paper's meaning (Buchbinder and Naor, 2009, pp. 8–9).

Formalization scope

Lean uses OnlinePrimalDual.GeneralPacking.GeneralInstance I (Fin m) for the coefficient matrix and costs. A nonempty finite type III represents the nnn primal variables; Fin m supplies the ordered constraints. The theorem assumes m≥1m\ge1m≥1, B>0B>0B>0, and a positive coefficient in every constraint. The last condition makes each arriving constraint coverable and makes the first ratio minimum nonempty; zero coefficients are omitted from that minimum. The paper treats infeasible all-zero rows implicitly. The minimum is over a finite nonempty set, so the real infimum in the Lean definition equals the ordinary minimum. All sums are finite and all logarithms are natural.

The run is an event relation with three events: arrival of one new constraint after all known constraints have been processed, completion of a covered round at its first stopping time, and restart at the cost cap while coverage remains deficient. A restart doubles α\alphaα, records the old phase, resets the dual variables, and resumes at the first known constraint. The state records the current phase and all finished phases; the output takes their coordinatewise maximum. A simultaneous coverage and cost event counts as covered. Reaching the cost cap models the continuous stopping time behind the paper's wording that the cost “exceeds” the bound. The theorem asserts run existence, so a ratio claim cannot hold merely because the run relation has no witness.

The explicit O(⋅)O(\cdot)O(⋅) instantiation in this mission is 8log⁡(2n)/B8\log(2n)/B8log(2n)/B, from the page 9 closing display. The first-phase case is bounded there by 1/B1/B1/B times the optimum and is included in the same stated coefficient. The comparison vector must satisfy the original level-one constraints. Theorem 4.2, which adds upper bounds on variables and appears in §4.1, is outside this mission. The run definition, phase-cost and coverage functions, and the four claims are reusable infrastructure for later work on continuous online primal–dual schemes.

Selected references

  • Niv Buchbinder and Joseph Naor, Online Primal-Dual Algorithms for Covering and Packing, Mathematics of Operations Research, 2009. DOI: 10.1287/moor.1080.0363.
7 thms1 active userReviewed
Operations ResearchOptimization·Captain: mikedeng1

Online Vehicle Routing: The Edge of Optimization in Large-Scale Applications: With Fixed Pick-up Times, the Offline Taxi Routing MIO Formulation (5)-(14) Is IntegralResearch Paper

Motivation

Taxi fleets, ride-hailing services and dial-a-ride systems assign incoming customer requests to vehicles under time constraints. Bertsimas, Jaillet and Martin (Online Vehicle Routing: The Edge of Optimization in Large-Scale Applications, Operations Research 67(1), 2019) model the online taxi routing problem, a special case of the online dial-a-ride problem with time windows in which a vehicle serves one customer at a time. They show that mixed-integer optimization over a sparsified graph can dispatch thousands of New York City taxis in real time. The offline problem, in which all requests are known in advance, is the building block of their online algorithms, and its structure determines which offline methods are fast.

The structural fact this mission formalizes is the paper's Theorem 1. When every customer has a fixed pick-up time, the mixed-integer formulation of the offline problem is integral: its linear relaxation has only binary extreme points. So the problem is a linear program in disguise, which the paper uses for its maxflow heuristic.

The source is the authors' accepted manuscript (March 2018, 46 pages); every page and display number below refers to that manuscript, not to the typeset article.

Setting

Let C\mathcal CC be a finite set of customers and K\mathcal KK a finite set of taxis. Customer ccc has a pick-up time window [tcmin⁡,tcmax⁡][t^{\min}_c, t^{\max}_c][tcmin​,tcmax​], and taxi kkk becomes available at time tkinitt^{\mathrm{init}}_ktkinit​. For customers c′,cc', cc′,c, the number Tc′,cT_{c',c}Tc′,c​ is the travel time from serving c′c'c′ to picking up ccc, and Rc′,cR_{c',c}Rc′,c​ is the profit earned when ccc is served right after c′c'c′. The numbers Tk,cT_{k,c}Tk,c​ and Rk,cR_{k,c}Rk,c​ are the travel time and profit when ccc is the first customer of taxi kkk.

The graph G\mathcal GG has the customers and taxis as nodes. Between customers there is an arc c→c′c \to c'c→c′ exactly when

tcmin⁡+Tc,c′≤tc′max⁡.(1)t^{\min}_c + T_{c,c'} \le t^{\max}_{c'}. \qquad (1)tcmin​+Tc,c′​≤tc′max​.(1)

Taxi nodes have outgoing arcs only. Throughout the paper G\mathcal GG is assumed acyclic (§2.1, p. 7).

The mixed-integer formulation (5)–(14) (pp. 11–12) has binary variables xc′,cx_{c',c}xc′,c​ (customer ccc is picked up right after c′c'c′), yk,cy_{k,c}yk,c​ (ccc is the first customer of taxi kkk) and pcp_cpc​ (ccc is served), and continuous pick-up times tct_ctc​. It maximizes ∑k,cRk,cyk,c+∑c′,cRc′,cxc′,c\sum_{k,c} R_{k,c} y_{k,c} + \sum_{c',c} R_{c',c} x_{c',c}∑k,c​Rk,c​yk,c​+∑c′,c​Rc′,c​xc′,c​ subject to the flow constraints

pc=∑kyk,c+∑c′xc′,c,∑cxc′,c≤pc′,∑cyk,c≤1,(6)–(8)p_c = \sum_k y_{k,c} + \sum_{c'} x_{c',c}, \qquad \sum_c x_{c',c} \le p_{c'}, \qquad \sum_c y_{k,c} \le 1, \qquad (6)\text{–}(8)pc​=k∑​yk,c​+c′∑​xc′,c​,c∑​xc′,c​≤pc′​,c∑​yk,c​≤1,(6)–(8)

the binary constraints (9)–(11), and the time constraints

tcmin⁡≤tc≤tcmax⁡,(12)tc−tc′≥(tcmin⁡−tc′max⁡)+(Tc′,c−(tcmin⁡−tc′max⁡))xc′,c,(13)tc≥tcmin⁡+(tkinit+Tk,c−tcmin⁡) yk,c.(14)\begin{aligned} t^{\min}_c &\le t_c \le t^{\max}_c, &(12)\\ t_c - t_{c'} &\ge (t^{\min}_c - t^{\max}_{c'}) + \big(T_{c',c} - (t^{\min}_c - t^{\max}_{c'})\big) x_{c',c}, &(13)\\ t_c &\ge t^{\min}_c + (t^{\mathrm{init}}_k + T_{k,c} - t^{\min}_c)\, y_{k,c}. &(14) \end{aligned}tcmin​tc​−tc′​tc​​≤tc​≤tcmax​,≥(tcmin​−tc′max​)+(Tc′,c​−(tcmin​−tc′max​))xc′,c​,≥tcmin​+(tkinit​+Tk,c​−tcmin​)yk,c​.​(12)(13)(14)​

Constraints (13) and (14) are strengthened Big-M constraints: with xc′,c=1x_{c',c} = 1xc′,c​=1, (13) reads tc−tc′≥Tc′,ct_c - t_{c'} \ge T_{c',c}tc​−tc′​≥Tc′,c​.

The LP relaxation replaces (9)–(11) by 0≤x,y,p≤10 \le x, y, p \le 10≤x,y,p≤1. A formulation is integral when every extreme point of its LP relaxation has x,y,p∈{0,1}x, y, p \in \{0,1\}x,y,p∈{0,1}.

Formalization targets

Goal: Theorem 1 (p. 13)

If tcmin⁡=tcmax⁡=tc∗t^{\min}_c = t^{\max}_c = t^*_ctcmin​=tcmax​=tc∗​ for every customer ccc (and G\mathcal GG is acyclic, the standing assumption), then

every extreme point (x,y,p,t) of the LP relaxation of (5)–(14) satisfies xc′,c, yk,c, pc∈{0,1}.\text{every extreme point } (x,y,p,t) \text{ of the LP relaxation of (5)–(14) satisfies } x_{c',c},\, y_{k,c},\, p_c \in \{0,1\}.every extreme point (x,y,p,t) of the LP relaxation of (5)–(14) satisfies xc′,c​,yk,c​,pc​∈{0,1}.

Milestones (proof of Theorem 1 and the remark before it)

  1. Display (15), p. 13: with fixed windows, (12) gives t=t∗t = t^*t=t∗, and (13) becomes (Tc′,c−(tc∗−tc′∗))xc′,c≤0\big(T_{c',c} - (t^*_c - t^*_{c'})\big) x_{c',c} \le 0(Tc′,c​−(tc∗​−tc′∗​))xc′,c​≤0.
  2. p. 13: with fixed times the relaxation is exactly {t=t∗}\{t = t^*\}{t=t∗} times the relaxed flow system (6)–(11). In that system xc′,cx_{c',c}xc′,c​ is removed when Tc′,c>tc∗−tc′∗T_{c',c} > t^*_c - t^*_{c'}Tc′,c​>tc∗​−tc′∗​, and yk,cy_{k,c}yk,c​ is removed when tkinit+Tk,c>tc∗t^{\mathrm{init}}_k + T_{k,c} > t^*_ctkinit​+Tk,c​>tc∗​.
  3. pp. 12–13: the relaxed flow system (6)–(11), on any subset of arcs, has 0/10/10/1 extreme points.

Three further statements from the same pages are included as plain theorems. The first is the paragraph after Theorem 1: a solution with fixed times tc∗∈[tcmin⁡,tcmax⁡]t^*_c \in [t^{\min}_c, t^{\max}_c]tc∗​∈[tcmin​,tcmax​] is feasible for the formulation with windows. The other two are the conditions (3) and (4) of §2.1, which exclude 2-cycles and all cycles of G\mathcal GG.

Significance

Theorem 1 turns a mixed-integer program into a linear program when the time windows shrink to points. The fixed-time problem can then be solved by the simplex method or by a max-flow algorithm. By the paragraph after the theorem, any choice of times inside the windows then gives a feasible solution of the original problem. This is the maxflow heuristic, and the paper reports it to be near-optimal when windows are small. The theorem also locates the source of the integrality gap: the time constraints (12)–(14), not the flow structure.

The result is proved on the page, briefly. No machine-checked version exists. Formalizing it means making the page's "equivalent to a formulation in which variable xc′,cx_{c',c}xc′,c​ is removed" precise. The extreme points of the relaxation in (x,y,p,t)(x,y,p,t)(x,y,p,t)-space must correspond to those of a flow polytope in (x,y,p)(x,y,p)(x,y,p)-space. The integrality of that flow polytope must then be proved, which the page settles by appeal to max-flow.

Difficulty

The reduction to the flow system is elementary linear arithmetic. The substance is milestone 3. The system (6)–(8) is not literally in the standard form of a network-flow problem: it mixes an equation defining pcp_cpc​ with inequalities, has unit upper bounds, and allows arbitrary arc sets, including cycles and loops (c,c)(c,c)(c,c). Neither the integrality theorem for standard-form network flows nor total unimodularity can be applied without first exhibiting a network and checking that the polytope corresponds to its flow polytope. Mathlib has the definition of total unimodularity but not the Hoffman–Kruskal theorem.

The page's phrase "the formulations are equivalent" also hides a step: the goal speaks of extreme points in (x,y,p,t)(x,y,p,t)(x,y,p,t)-space, while the flow system lives in (x,y,p)(x,y,p)(x,y,p)-space, and the two notions of extreme point have to be related explicitly.

Formalization scope

  • Customers and taxis are arbitrary finite types; empty sets of customers or taxis are allowed, and the statements remain the paper's there.
  • A point (x,y,p,t)(x,y,p,t)(x,y,p,t) is an element of the real vector space (C×C→R)×(K×C→R)×(C→R)×(C→R)(\mathcal C \times \mathcal C \to \mathbb R) \times (\mathcal K \times \mathcal C \to \mathbb R) \times (\mathcal C \to \mathbb R) \times (\mathcal C \to \mathbb R)(C×C→R)×(K×C→R)×(C→R)×(C→R). Extreme points are Mathlib's Set.extremePoints ℝ.
  • All constraints are indexed by all pairs, the diagonal c′=cc' = cc′=c included, exactly as printed. The variables are not restricted to the arcs of G\mathcal GG.
  • No sign conditions are imposed on the data TTT, RRR, tinitt^{\mathrm{init}}tinit.
  • Fixed times are the hypothesis tcmin⁡=tcmax⁡t^{\min}_c = t^{\max}_ctcmin​=tcmax​ for all ccc. The acyclicity of G\mathcal GG is kept on the goal as the paper's standing assumption, although the conclusion does not need it; milestones and companions omit it, which makes them stronger.
  • Milestone 3 is stated for an arbitrary arc subset, which covers both the system (6)–(11) as printed and the system with variables removed that the proof uses.

Two trivializing readings are ruled out. "Integral" is the extreme-point property of the relaxation, not the existence of an integral optimum for a given objective, and not the integrality of the mixed-integer feasible set, which holds by definition. Integrality is never demanded of the continuous times ttt.

A complete development needs an integrality theorem for flow polytopes with integer bounds, which Mathlib does not have. Such a result is reusable well beyond this mission. Contributions of general network-flow integrality lemmas are welcome.

Selected references

  • D. Bertsimas, P. Jaillet, S. Martin, Online Vehicle Routing: The Edge of Optimization in Large-Scale Applications, Operations Research 67(1):143–162, 2019; authors' accepted manuscript, March 2018. https://doi.org/10.1287/opre.2018.1763
  • A. J. Hoffman, J. B. Kruskal, Integral boundary points of convex polyhedra, in Linear Inequalities and Related Systems, Annals of Mathematics Studies 38, 1956. https://doi.org/10.1515/9781400881987-013
  • J.-F. Cordeau, G. Laporte, The dial-a-ride problem: models and algorithms, Annals of Operations Research 153:29–46, 2007. https://doi.org/10.1007/s10479-007-0170-8
5 thms1 active userReviewed
Machine LearningOperations ResearchOptimization·Captain: mikedeng1

Strong Mixed-Integer Programming Formulations for Trained Neural Networks 2: Under Strict Activity Every Inequality of the Exponential Family (6b) Is Facet-DefiningResearch Paper

Motivation

Trained neural networks are increasingly embedded inside optimization models: to verify that a classifier is robust to small input perturbations, to optimize over a learned surrogate of an expensive system, or to choose decisions whose outcome is predicted by a network. When the network uses rectified linear units (ReLU), each neuron y=max⁡{0, w⋅x+b}y=\max\{0,\,w\cdot x+b\}y=max{0,w⋅x+b} is piecewise linear, and the whole network can be written exactly as a mixed-integer program (MIP) with one binary variable per neuron. How fast a branch-and-bound solver closes such a model depends on how tight the linear-programming relaxation of each neuron's formulation is.

Anderson, Huchette, Tjandraatmadja and Vielma (arXiv:1811.08359v2, the IPCO 2019 extended abstract) gave a formulation (6) of a single ReLU neuron over a box that uses only the original variables and one binary variable, and is ideal: its LP relaxation has integral extreme points (Proposition 1, p. 6). The price is an exponential family of inequalities (6b), one for every subset III of the support of www. The present mission formalizes their Proposition 2: each of these inequalities is facet-defining, so no member of the family can be dropped without weakening the relaxation. A longer journal version of the work, with different numbering, appeared later (arXiv:1811.01988); this mission follows the extended abstract.

Setting

Fix η∈N\eta\in\mathbb Nη∈N, a weight vector w∈Rηw\in\mathbb R^\etaw∈Rη, a bias b∈Rb\in\mathbb Rb∈R, and bounds L,U∈RηL,U\in\mathbb R^\etaL,U∈Rη with Li<UiL_i<U_iLi​<Ui​ for every iii (§1.3, p. 4). Write f(x)=w⋅x+bf(x)=w\cdot x+bf(x)=w⋅x+b and [L,U]={x:L≤x≤U}[L,U]=\{x : L\le x\le U\}[L,U]={x:L≤x≤U}. The sign-adjusted bounds are

L˘i={Liwi≥0Uiwi<0,U˘i={Uiwi≥0Liwi<0,\breve L_i=\begin{cases}L_i & w_i\ge 0\\ U_i & w_i<0\end{cases},\qquad \breve U_i=\begin{cases}U_i & w_i\ge 0\\ L_i & w_i<0\end{cases},L˘i​={Li​Ui​​wi​≥0wi​<0​,U˘i​={Ui​Li​​wi​≥0wi​<0​,

so that M+(f)=w⋅U˘+bM^+(f)=w\cdot\breve U+bM+(f)=w⋅U˘+b and M−(f)=w⋅L˘+bM^-(f)=w\cdot\breve L+bM−(f)=w⋅L˘+b are the maximum and minimum of fff over [L,U][L,U][L,U]. The support is supp⁡(w)={i:wi≠0}\operatorname{supp}(w)=\{i : w_i\neq 0\}supp(w)={i:wi​=0}. Strict activity means M−(f)<0<M+(f)M^-(f)<0<M^+(f)M−(f)<0<M+(f): the neuron is neither always off nor always on over the box. The paper assumes it throughout (§1.3).

Formulation (6) (p. 6) consists of the points (x,y,z)(x,y,z)(x,y,z) with

y≥w⋅x+b,(6a)y≤∑i∈Iwi(xi−L˘i(1−z))+(b+∑i∉IwiU˘i)z∀I⊆supp⁡(w),(6b)(x,y,z)∈[L,U]×R≥0×{0,1}.(6c)\begin{aligned} &y\ge w\cdot x+b, &&\text{(6a)}\\ &y\le\sum_{i\in I}w_i\bigl(x_i-\breve L_i(1-z)\bigr)+\Bigl(b+\sum_{i\notin I}w_i\breve U_i\Bigr)z\quad\forall I\subseteq\operatorname{supp}(w), &&\text{(6b)}\\ &(x,y,z)\in[L,U]\times\mathbb R_{\ge0}\times\{0,1\}. &&\text{(6c)} \end{aligned}​y≥w⋅x+b,y≤i∈I∑​wi​(xi​−L˘i​(1−z))+(b+i∈/I∑​wi​U˘i​)z∀I⊆supp(w),(x,y,z)∈[L,U]×R≥0​×{0,1}.​​(6a)(6b)(6c)​

In Lean, form6 w b L U is this set, and relax6 w b L U is its LP relaxation (0≤z≤10\le z\le10≤z≤1 in place of z∈{0,1}z\in\{0,1\}z∈{0,1}). The right-hand side of (6b) for the subset III is rhs6b w b L U I x z.

An inequality g≤0g\le0g≤0 is facet-defining for a set PPP when it holds on PPP, its face F=P∩{g=0}F=P\cap\{g=0\}F=P∩{g=0} is nonempty, and dim⁡F=dim⁡P−1\dim F=\dim P-1dimF=dimP−1, the dimension of a set being that of its affine hull. The paper uses this standard notion without defining it.

Formalization targets

Goal: Proposition 2 (p. 6)

Under L<UL<UL<U and strict activity, for every I⊆supp⁡(w)I\subseteq\operatorname{supp}(w)I⊆supp(w), the inequality (6b) for III is facet-defining for

P=conv⁡{(x,y,z):(x,y,z) satisfies (6a)–(6c)}.P=\operatorname{conv}\{(x,y,z) : (x,y,z)\text{ satisfies (6a)–(6c)}\}.P=conv{(x,y,z):(x,y,z) satisfies (6a)–(6c)}.

The page states the result as "Each inequality in (6b) is facet-defining", and adds right after the proof line: "We require the assumption of strict activity above, as introduced in Section 1.3."

Milestones (App. A.2, p. 15)

  1. For some ε>0\varepsilon>0ε>0, the η+2\eta+2η+2 points p0=(L˘,0,0)p^0=(\breve L,0,0)p0=(L˘,0,0), p1=(U˘,f(U˘),1)p^1=(\breve U,f(\breve U),1)p1=(U˘,f(U˘),1), p~i=(L˘+εσiei,0,0)\tilde p^i=(\breve L+\varepsilon\sigma_ie^i,0,0)p~​i=(L˘+εσi​ei,0,0) for i∉Ii\notin Ii∈/I, and p~i=(U˘−εσiei,f(U˘−εσiei),1)\tilde p^i=(\breve U-\varepsilon\sigma_ie^i,f(\breve U-\varepsilon\sigma_ie^i),1)p~​i=(U˘−εσi​ei,f(U˘−εσi​ei),1) for i∈Ii\in Ii∈I are feasible with respect to (6) and satisfy (6b) for III at equality; here σi=±1\sigma_i=\pm1σi​=±1 is the sign of wiw_iwi​ (with σi=1\sigma_i=1σi​=1 when wi=0w_i=0wi​=0).
  2. For every ε>0\varepsilon>0ε>0 these η+2\eta+2η+2 points are affinely independent.

Significance

The result. Proposition 1 shows that (6) is ideal; Proposition 2 shows it cannot be made smaller: removing any single inequality (6b) produces a strictly weaker relaxation. Since the family has 2∣supp⁡(w)∣2^{|\operatorname{supp}(w)|}2∣supp(w)∣ members, this is what justifies the paper's practical recommendation to start from the big-MMM formulation and separate inequalities of (6b) on demand (Proposition 3, p. 7) rather than to search for a smaller ideal description in the same variables. The facet structure also gives the geometric picture the paper describes after Proposition 2: each facet is the convex combination of an (η−∣I∣)(\eta-|I|)(η−∣I∣)-dimensional face at z=0z=0z=0 and an ∣I∣|I|∣I∣-dimensional face at z=1z=1z=1.

Formalizing it. The result is proved in the paper, in a half-page appendix; it has not been machine-checked. The formalization adds two things. First, the proof is written for w≥0w\ge0w≥0 "without loss of generality by appropriately interchanging +++ and −-−"; the Lean statements are for every sign pattern, including zero weights. Second, the appendix exhibits η+2\eta+2η+2 affinely independent points on the face, which bounds the face dimension from below; the statement that the face has dimension exactly one less than the polyhedron also needs the polyhedron to be full-dimensional and the face to lie in a proper hyperplane, steps the extended abstract leaves implicit and a complete proof must supply.

Difficulty

The arithmetic in each step is elementary. The work lies in the bookkeeping: choosing a single ε\varepsilonε that keeps every perturbed point inside the box and on the correct side of f=0f=0f=0 (this is exactly where strict activity enters), checking the perturbed points against all 2∣supp⁡(w)∣2^{|\operatorname{supp}(w)|}2∣supp(w)∣ inequalities of (6b) and not only the one for III, and turning a row-reduction argument on an (η+1)×(η+2)(\eta+1)\times(\eta+2)(η+1)×(η+2) matrix into a statement about AffineIndependent and finrank of a vectorSpan in Lean. The natural shortcut, proving only that η+2\eta+2η+2 affinely independent tight points exist, is not Proposition 2: it says nothing about the dimension of the polyhedron itself.

Formalization scope

Inputs are Fin η → ℝ (indices 0,…,η−10,\dots,\eta-10,…,η−1 for the paper's 1,…,η1,\dots,\eta1,…,η); a point (x,y,z)(x,y,z)(x,y,z) is p : (Fin η → ℝ) × ℝ × ℝ with p.1 = x, p.2.1 = y, p.2.2 = z. M±(f)M^\pm(f)M±(f) are given by their closed forms w⋅U˘+bw\cdot\breve U+bw⋅U˘+b and w⋅L˘+bw\cdot\breve L+bw⋅L˘+b. In (6b), "i∉Ii\notin Ii∈/I" ranges over all indices outside III, zero weights included; III ranges over subsets of supp⁡(w)\operatorname{supp}(w)supp(w), as on the page.

Every goal and milestone keeps the standing assumptions of §1.3, Li<UiL_i<U_iLi​<Ui​ for all iii and strict activity, except the affine-independence milestone, which holds without them and is stated without them (a stronger statement). No other hypothesis is added. Strict activity excludes η=0\eta=0η=0, so no nonemptiness assumption on the index set is needed.

IsFacetDefining P g is the standard notion: validity, a nonempty face, and dim⁡F+1=dim⁡P\dim F+1=\dim PdimF+1=dimP with dimensions the finrank of the vectorSpan. The equation is written with +1+1+1 on the left so that no natural-number subtraction can make the empty set or a point a facet. The polyhedron is the convex hull of the points of (6), not the set form6 itself (which is not convex, since z∈{0,1}z\in\{0,1\}z∈{0,1}); by Proposition 1 it equals the LP relaxation relax6, but the statement does not depend on that.

The shared objects of this paper (the ReLU graph, L˘\breve LL˘, U˘\breve UU˘, M±M^\pmM±, support, strict activity, formulation (6), ideality) come from the shared definitions module ReluMIP.Ideal.Setting, common to the companion mission on Proposition 1; this mission's own definitions module adds form6, IsFacetDefining, the sign inward, and the indexed family facetPts of the η+2\eta+2η+2 points. The facet notion and the affine-independence argument are reusable for other facet proofs of polyhedra in product spaces. Proofs of the milestones, of the goal, and of the full-dimensionality step are all welcome.

Selected references

  • R. Anderson, J. Huchette, C. Tjandraatmadja, J. P. Vielma, Strong mixed-integer programming formulations for trained neural networks, IPCO 2019, LNCS 11480, pp. 27–42; preprint arXiv:1811.08359v2, 2019. https://arxiv.org/abs/1811.08359v2
  • R. Anderson, J. Huchette, W. Ma, C. Tjandraatmadja, J. P. Vielma, Strong mixed-integer programming formulations for trained neural networks, Mathematical Programming 183 (2020), 3–39. https://doi.org/10.1007/s10107-020-01474-5
  • G. L. Nemhauser, L. A. Wolsey, Integer and Combinatorial Optimization, Wiley, 1988. https://doi.org/10.1002/9781118627372
  • M. Conforti, G. Cornuéjols, G. Zambelli, Integer Programming, Springer, 2014. https://doi.org/10.1007/978-3-319-11008-0
  • J. P. Vielma, Mixed integer linear programming formulation techniques, SIAM Review 57 (2015), 3–57. https://doi.org/10.1137/130915303
5 thms1 active userReviewed
CombinatoricsComplexity TheoryTheoretical Computer Science·Captain: mikedeng1

Promise Constraint Satisfaction: Algebraic Structure and a Symmetric Boolean Dichotomy 2: The LP Algorithm Decides PCSPs with Majority or Alternating-Threshold Polymorphisms of All Odd AritiesResearch Paper

Motivation

A constraint satisfaction problem (CSP) asks whether variables can be assigned values so that every constraint of a given instance holds. Feder and Vardi conjectured, and Bulatov and Zhuk proved in 2017, that every CSP over a finite set of relations is either polynomial-time solvable or NP-complete, and that the answer is governed by the polymorphisms of the relations: the functions that map tuples of satisfying assignments, coordinate by coordinate, to a satisfying assignment.

A promise CSP (PCSP) relaxes the question. Each constraint comes as a pair of relations P⊆QP \subseteq QP⊆Q; the input is promised to be satisfiable with every constraint read as PPP, or unsatisfiable even with every constraint read as QQQ, and the task is to tell which. Approximate graph colouring ("is this 3-colourable graph 100-colourable?") and (2+ε)(2+\varepsilon)(2+ε)-SAT are of this kind. Austrin, Guruswami and Håstad (2017, doi:10.1137/15M1006507) showed that (2+ε)(2+\varepsilon)(2+ε)-SAT is NP-hard via polymorphisms; Brakensiek and Guruswami extended the polymorphism view to general Boolean PCSPs and proved a dichotomy for symmetric Boolean families (arXiv:1704.01937v2, SODA 2018, SIAM J. Comput. 2021). Later work by Barto, Bulín, Krokhin and Opršal (arXiv:1811.00970) built the general algebraic theory of PCSPs on this foundation.

This mission takes one ingredient of the dichotomy: the tractable side for two polymorphism families that have no counterpart among tractable CSPs, the Majority and the Alternating-Threshold functions. The paper solves both with the same linear programming test.

Setting

The domain is {0,1}\{0,1\}{0,1}. A finite family of promise relations is Γ={(PR,QR):R∈τ}\Gamma = \{(P_R, Q_R) : R \in \tau\}Γ={(PR​,QR​):R∈τ}, with τ\tauτ finite and PR⊆QR⊆{0,1}kRP_R \subseteq Q_R \subseteq \{0,1\}^{k_R}PR​⊆QR​⊆{0,1}kR​. An instance Ψ=(ΨP,ΨQ)\Psi = (\Psi_P, \Psi_Q)Ψ=(ΨP​,ΨQ​) has variables x1,…,xnx_1, \dots, x_nx1​,…,xn​ and clauses Rj(xj1,…,xjk)R_j(x_{j_1}, \dots, x_{j_k})Rj​(xj1​​,…,xjk​​); a variable may occur several times in a clause. ΨP\Psi_PΨP​ reads each clause with PRjP_{R_j}PRj​​, and ΨQ\Psi_QΨQ​ with QRjQ_{R_j}QRj​​. Since P⊆QP \subseteq QP⊆Q, satisfiability of ΨP\Psi_PΨP​ implies that of ΨQ\Psi_QΨQ​.

A function f:{0,1}L→{0,1}f : \{0,1\}^L \to \{0,1\}f:{0,1}L→{0,1} is a polymorphism of Γ\GammaΓ if for every RRR and all x(1),…,x(L)∈PRx^{(1)}, \dots, x^{(L)} \in P_Rx(1),…,x(L)∈PR​ the tuple (f(x1(1),…,x1(L)),…,f(xk(1),…,xk(L)))\big(f(x^{(1)}_1, \dots, x^{(L)}_1), \dots, f(x^{(1)}_k, \dots, x^{(L)}_k)\big)(f(x1(1)​,…,x1(L)​),…,f(xk(1)​,…,xk(L)​)) lies in QRQ_RQR​. For odd LLL,

MajL(x)=1  ⟺  ∑i=1Lxi>L2,ATL(x)=1  ⟺  ∑i=1L(−1)i−1xi>0.\mathrm{Maj}_L(x) = 1 \iff \sum_{i=1}^L x_i > \frac L2, \qquad \mathrm{AT}_L(x) = 1 \iff \sum_{i=1}^L (-1)^{i-1} x_i > 0 .MajL​(x)=1⟺i=1∑L​xi​>2L​,ATL​(x)=1⟺i=1∑L​(−1)i−1xi​>0.

The LP relaxation of Ψ\PsiΨ has one unknown vj∈[0,1]v_j \in [0,1]vj​∈[0,1] per variable, and for every clause Rj(xj1,…,xjk)R_j(x_{j_1}, \dots, x_{j_k})Rj​(xj1​​,…,xjk​​) of ΨP\Psi_PΨP​ it requires (vj1,…,vjk)(v_{j_1}, \dots, v_{j_k})(vj1​​,…,vjk​​) to lie in the convex hull of PRjP_{R_j}PRj​​. The §3.2 algorithm loops over the variables: it fixes vj=0v_j = 0vj​=0 and re-solves the LP, then, if that is infeasible, fixes vj=1v_j = 1vj​=1 and re-solves, and outputs "unsatisfiable" if both are infeasible. If every variable passes, it outputs "satisfiable". In Lean the answer is the predicate LPAlgAccepts 𝔸 X: the LP is feasible and, for each jjj, it has a solution with vj=0v_j = 0vj​=0 or one with vj=1v_j = 1vj​=1.

Formalization targets

Goal: correctness of the §3.2 algorithm

If MajL∈Pol(Γ)\mathrm{Maj}_L \in \mathrm{Pol}(\Gamma)MajL​∈Pol(Γ) for every odd LLL, or ATL∈Pol(Γ)\mathrm{AT}_L \in \mathrm{Pol}(\Gamma)ATL​∈Pol(Γ) for every odd LLL, then for every instance Ψ\PsiΨ

ΨP satisfiable  ⟹  LPAlgAccepts(Ψ)  ⟹  ΨQ satisfiable.\Psi_P \text{ satisfiable} \;\Longrightarrow\; \texttt{LPAlgAccepts}(\Psi) \;\Longrightarrow\; \Psi_Q \text{ satisfiable}.ΨP​ satisfiable⟹LPAlgAccepts(Ψ)⟹ΨQ​ satisfiable.

No symmetry of the relations is assumed.

Milestones, in the order of the proof (pp. 14–15)

  1. Completeness. If ΨP\Psi_PΨP​ is satisfiable, the algorithm accepts; no polymorphism is needed.
  2. Convexity. A convex combination of LP solutions is an LP solution.
  3. Case 1 claim. If M∈([0,1]∩Q)n×nM \in ([0,1]\cap\mathbb Q)^{n\times n}M∈([0,1]∩Q)n×n has Mii∈{0,1}M_{ii} \in \{0,1\}Mii​∈{0,1}, some rational probability vector vvv has (Mv)i≠1/2(Mv)_i \ne 1/2(Mv)i​=1/2 for all iii.
  4. Case 1 rounding. With MajL\mathrm{Maj}_LMajL​ for all odd LLL, an LP solution www with no coordinate 1/21/21/2 rounds to a satisfying assignment xi∗=⌊wi⌉x^*_i = \lfloor w_i \rceilxi∗​=⌊wi​⌉ of ΨQ\Psi_QΨQ​.
  5. Case 2 perturbation. For such MMM and any rational w^\hat ww^, some vvv has (Mv)i≠w^i(Mv)_i \ne \hat w_i(Mv)i​=w^i​ wherever w^i∉{0,1}\hat w_i \notin \{0,1\}w^i​∈/{0,1}.
  6. Case 2 rounding. With ATL\mathrm{AT}_LATL​ for all odd LLL, two LP solutions w,w^w, \hat ww,w^ that agree only where w^i∈{0,1}\hat w_i \in \{0,1\}w^i​∈{0,1} give the satisfying assignment xi∗=[wi>w^i or wi=w^i=1]x^*_i = [w_i > \hat w_i \text{ or } w_i = \hat w_i = 1]xi∗​=[wi​>w^i​ or wi​=w^i​=1] of ΨQ\Psi_QΨQ​.

Significance

The theorem shows that the Majority and Alternating-Threshold cases of Theorem 3.2 are tractable. Since Γ\GammaΓ is fixed, the LP has size linear in the instance, so linear programming decides PCSP(Γ)\mathrm{PCSP}(\Gamma)PCSP(Γ) in polynomial time. Unlike the Zero, One, AND, OR and Parity cases of §3.1, these families cannot be handled by sandwiching a tractable CSP between PPP and QQQ: footnote 12 of the paper observes that the closure of {MajL}\{\mathrm{Maj}_L\}{MajL​} or {ATL}\{\mathrm{AT}_L\}{ATL​} under identification of variables is not a clone. The tractable half of the paper's symmetric Boolean dichotomy (Theorem 2.16) depends on this result. Later work replaced this LP by the Basic LP and BLP+Affine relaxations (Brakensiek–Guruswami 2019, Barto et al. 2021).

The result is proved in the paper; to our knowledge it has no machine-checked proof. The formal content is a finite-dimensional rational convexity argument plus a counting argument on columns of polymorphism inputs. Related platform work: the published PCSPBLPAff.Symmetric.theorem_2 and theorem_3 concern the BLP+Affine algorithm with symmetric or block-symmetric polymorphisms. That is a different relaxation, and ATL\mathrm{AT}_LATL​ is not symmetric.

Difficulty

Completeness is immediate. Soundness is where the difficulty lies. The obvious approach, rounding an arbitrary LP solution, fails. For Majority, a coordinate equal to 1/21/21/2 has no nearest integer, and the LP may admit no solution without such coordinates unless one uses the per-variable solutions the algorithm found. For Alternating-Threshold, there is no fixed threshold at all, and a single LP solution gives no rule for rounding a fractional coordinate. Two further points need care. Getting from a fractional point back to a polymorphism application needs a polymorphism of an arity that depends on the denominators of the hull weights, so a single fixed arity does not suffice. In the Alternating-Threshold case, as printed, the paper pads the inputs with an arbitrary point of PPP, and that step fails at coordinates where both solutions equal 000 or 111; the padding point has to come from the support of the hull weights.

Formalization scope

The domain {0,1}\{0,1\}{0,1} is Bool. Γ\GammaΓ is a pair 𝔸 𝔹 : RelStruct τ ar Bool with [Fintype τ] and 𝔸.rel R ⊆ 𝔹.rel R, and instances, satisfiability and polymorphisms come from the published PCSPBLPAff_Symmetric_Setting. Coordinates are 0-based (Fin L), so the sign of ATL\mathrm{AT}_LATL​ is (−1)i(-1)^i(−1)i. The LP is stated over Q\mathbb QQ; an LP with rational data is feasible over R\mathbb RR exactly when it is feasible over Q\mathbb QQ. The hull condition uses PRP_RPR​, never QRQ_RQR​. The acceptance predicate also requires the LP itself to be feasible, which matters only for instances without variables. The hypothesis is a disjunction of two universal statements ("MajL\mathrm{Maj}_LMajL​ for all odd LLL, or ATL\mathrm{AT}_LATL​ for all odd LLL"), not "for every odd LLL, one of the two". A formalization that drops the hull constraint, replaces PPP by QQQ in the LP, or lets the acceptance predicate be satisfied without solving the LP would make the theorem trivial or different, and is ruled out by the definitions here.

Out of scope: polynomial running time and the printed conclusion "PCSP(Γ) is polynomial-time tractable" of Theorem 3.2; the polynomial-time solvability of linear programs; the Zero/One/AND/OR/Parity cases (Lemma 3.1, which invokes Schaefer's theorem); and the non-idempotent ("anti-") cases, which go through the reduction of Lemma 2.13(2). The Remark on p. 15 says the algorithm decides but does not find a solution. Only the decision is formalized.

Infrastructure needed: finite convex combinations over Q\mathbb QQ, a perturbation lemma for avoiding finitely many hyperplanes in the rational simplex, and counting identities for MajL\mathrm{Maj}_LMajL​ and ATL\mathrm{AT}_LATL​ on block inputs. The perturbation lemma and the counting identities are reusable beyond this mission. Proofs of any milestone, and alternative arguments (e.g. via the affine-hull algorithm in the Remark on p. 14), are welcome.

Selected references

  • J. Brakensiek, V. Guruswami, Promise Constraint Satisfaction: Algebraic Structure and a Symmetric Boolean Dichotomy, arXiv:1704.01937v2, 2021; SIAM J. Comput. 50(6), 2021. https://arxiv.org/abs/1704.01937v2
  • P. Austrin, V. Guruswami, J. Håstad, (2+ε)-Sat Is NP-Hard, SIAM J. Comput. 46(5), 2017. https://doi.org/10.1137/15M1006507
  • L. Barto, J. Bulín, A. Krokhin, J. Opršal, Algebraic Approach to Promise Constraint Satisfaction, J. ACM 68(4), 2021. https://arxiv.org/abs/1811.00970
  • J. Brakensiek, V. Guruswami, An Algorithmic Blend of LPs and Ring Equations for Promise CSPs, SODA 2019. https://arxiv.org/abs/1807.05194
  • T. J. Schaefer, The Complexity of Satisfiability Problems, STOC 1978. https://doi.org/10.1145/800133.804350
10 thms1 active userReviewed
Machine LearningOperations ResearchOptimization·Captain: mikedeng1

Strong Mixed-Integer Programming Formulations for Trained Neural Networks 1: A ReLU Neuron over a Box Has an Ideal Formulation with One Binary Variable and the Exponential Family (6b)Research Paper

Why optimize over a trained ReLU neuron

A trained feed-forward neural network with ReLU activations, ReLU(v)=max⁡{0,v}\mathrm{ReLU}(v)=\max\{0,v\}ReLU(v)=max{0,v}, is a piecewise linear function of its input. Many tasks ask for an optimization over such a network with its weights held fixed: verifying that no small perturbation of an image changes its classification, finding adversarial examples, or embedding a learned model of demand or cost inside a decision problem ("predict, then optimize"). The standard way to solve such problems exactly is mixed-integer programming (MIP): each neuron is written as a small set of linear constraints with one binary variable, and the network is the composition of these neuron formulations. How strong each neuron formulation is decides how fast branch-and-bound can close the gap.

The formulation used in the literature up to 2018 is the big-M formulation. It is valid but weak. This mission formalizes the main result of Anderson, Huchette, Tjandraatmadja and Vielma, IPCO 2019 extended abstract (arXiv:1811.08359v2), which gives the strongest possible formulation of a single ReLU neuron that uses the original variables and one binary variable only.

Timeline. Big-M formulations of ReLU networks were used by several groups in 2017–2018 for verification and adversarial analysis (see §1.2 of the paper). Balas's disjunctive programming (1985, 1998) and the multiple choice formulation of piecewise linear functions (Vielma and Nemhauser, Math. Program. 2011) give an ideal formulation of the neuron that needs a copy of the input variables. Anderson et al. (2019) project out that copy and obtain the formulation (6) below; a longer journal version (Math. Program. 2020, with W. Ma) develops the analysis further, including interactions between neurons.

Setting

Fix η∈N\eta\in\mathbb Nη∈N, a weight vector w∈Rηw\in\mathbb R^\etaw∈Rη, a bias b∈Rb\in\mathbb Rb∈R, and bounds L,U∈RηL,U\in\mathbb R^\etaL,U∈Rη with Li<UiL_i<U_iLi​<Ui​ for every iii. The neuron computes ReLU(f(x))\mathrm{ReLU}(f(x))ReLU(f(x)) for the affine function f(x)=w⋅x+bf(x)=w\cdot x+bf(x)=w⋅x+b on the box [L,U]={x:L≤x≤U}[L,U]=\{x: L\le x\le U\}[L,U]={x:L≤x≤U}. Its graph is

gr⁡(ReLU∘f;[L,U])={(x,ReLU(f(x))):L≤x≤U}.\operatorname{gr}(\mathrm{ReLU}\circ f;[L,U])=\{(x,\mathrm{ReLU}(f(x))) : L\le x\le U\}.gr(ReLU∘f;[L,U])={(x,ReLU(f(x))):L≤x≤U}.

The sign-adjusted bounds are L˘i=Li\breve L_i=L_iL˘i​=Li​, U˘i=Ui\breve U_i=U_iU˘i​=Ui​ when wi≥0w_i\ge 0wi​≥0, and L˘i=Ui\breve L_i=U_iL˘i​=Ui​, U˘i=Li\breve U_i=L_iU˘i​=Li​ when wi<0w_i<0wi​<0. Then M+(f)=w⋅U˘+bM^+(f)=w\cdot\breve U+bM+(f)=w⋅U˘+b and M−(f)=w⋅L˘+bM^-(f)=w\cdot\breve L+bM−(f)=w⋅L˘+b are the maximum and minimum of fff on [L,U][L,U][L,U], and supp⁡(w)={i:wi≠0}\operatorname{supp}(w)=\{i: w_i\ne 0\}supp(w)={i:wi​=0}. Strict activity means M−(f)<0<M+(f)M^-(f)<0<M^+(f)M−(f)<0<M+(f): the neuron is neither always off nor always on. The paper assumes throughout that Li<UiL_i<U_iLi​<Ui​ and that strict activity holds.

A set RRR of points (x,y,z)(x,y,z)(x,y,z) with z∈[0,1]z\in[0,1]z∈[0,1], read together with the constraint z∈{0,1}z\in\{0,1\}z∈{0,1}, is a formulation of the graph if (x,y)(x,y)(x,y) lies on the graph exactly when (x,y,z)∈R(x,y,z)\in R(x,y,z)∈R for some z∈{0,1}z\in\{0,1\}z∈{0,1}; RRR is then its LP relaxation. The formulation is ideal if every extreme point of RRR has z∈{0,1}z\in\{0,1\}z∈{0,1}.

The big-M formulation (3) is y≥f(x)y\ge f(x)y≥f(x), y≤f(x)−M−(f)(1−z)y\le f(x)-M^-(f)(1-z)y≤f(x)−M−(f)(1−z), y≤M+(f)zy\le M^+(f)zy≤M+(f)z, (x,y,z)∈[L,U]×R≥0×{0,1}(x,y,z)\in[L,U]\times\mathbb R_{\ge0}\times\{0,1\}(x,y,z)∈[L,U]×R≥0​×{0,1}. The formulation of Proposition 1 is

y≥w⋅x+b(6a)y≤∑i∈Iwi(xi−L˘i(1−z))+(b+∑i∉IwiU˘i)z∀I⊆supp⁡(w)(6b)(x,y,z)∈[L,U]×R≥0×{0,1}.(6c)\begin{aligned} &y\ge w\cdot x+b &&(6a)\\ &y\le\sum_{i\in I}w_i\bigl(x_i-\breve L_i(1-z)\bigr)+\Bigl(b+\sum_{i\notin I}w_i\breve U_i\Bigr)z\qquad\forall I\subseteq\operatorname{supp}(w) &&(6b)\\ &(x,y,z)\in[L,U]\times\mathbb R_{\ge0}\times\{0,1\}. &&(6c) \end{aligned}​y≥w⋅x+by≤i∈I∑​wi​(xi​−L˘i​(1−z))+(b+i∈/I∑​wi​U˘i​)z∀I⊆supp(w)(x,y,z)∈[L,U]×R≥0​×{0,1}.​​(6a)(6b)(6c)​

Formalization targets

Goal: Proposition 1 (p. 6)

(6a)–(6c) is a formulation of gr⁡(ReLU∘f;[L,U]), and every extreme point of its LP relaxation has z∈{0,1}.\text{(6a)–(6c) is a formulation of }\operatorname{gr}(\mathrm{ReLU}\circ f;[L,U]),\ \text{and every extreme point of its LP relaxation has } z\in\{0,1\}.(6a)–(6c) is a formulation of gr(ReLU∘f;[L,U]), and every extreme point of its LP relaxation has z∈{0,1}.

Milestones (Appendix A.1 and §2.2)

  1. The multiple choice formulation (5), with copies x0,x1,y0,y1x^0,x^1,y^0,y^1x0,x1,y0,y1, has only integral zzz at extreme points in its lifted space and formulates the graph; projected to (x,y,z)(x,y,z)(x,y,z), its LP relaxation is the convex hull of its points with z∈{0,1}z\in\{0,1\}z∈{0,1} (§2.2, pp. 5–6).
  2. Projecting the copies out of the LP relaxation of (5) gives the linear system (7): (6a), (6b), a second exponential family (7c), and the bounds (7d) (App. A.1, p. 14).
  3. The family (7c) is implied by the other constraints (App. A.1, pp. 14–15).
  4. The LP relaxation of (6) is the convex hull of its points with z∈{0,1}z\in\{0,1\}z∈{0,1} (App. A.1, p. 14).

Companion results

  • M±(f)M^\pm(f)M±(f) are the maximum and minimum of fff on [L,U][L,U][L,U] (§1.3, p. 4).
  • Proposition 3 (p. 7): for (x^,y^,z^)∈[L,U]×R≥0×[0,1](\hat x,\hat y,\hat z)\in[L,U]\times\mathbb R_{\ge0}\times[0,1](x^,y^​,z^)∈[L,U]×R≥0​×[0,1], if some inequality of (6b) is violated, the one for I^={i∈supp⁡(w):wix^i<wi(L˘i(1−z^)+U˘iz^)}\hat I=\{i\in\operatorname{supp}(w): w_i\hat x_i<w_i(\breve L_i(1-\hat z)+\breve U_i\hat z)\}I^={i∈supp(w):wi​x^i​<wi​(L˘i​(1−z^)+U˘i​z^)} is the most violated.
  • The big-M formulation (3) is a formulation of the graph (p. 4); Examples 1 and 2 (p. 5) show that it is not ideal and that its gap grows like 12γη\tfrac12\gamma\eta21​γη; its inequalities (3b), (3c) are (6b) for I=supp⁡(w)I=\operatorname{supp}(w)I=supp(w) and I=∅I=\emptysetI=∅ (p. 7).

Significance

Proposition 1 says that the convex hull of the graph, lifted with one binary variable, is described by (6a), (6b) and the bounds, with no auxiliary continuous variables. Optimizing a linear function over the LP relaxation of (6) therefore gives the tightest convex relaxation available for a single neuron, and Proposition 3 gives a separation routine linear in η\etaη, so the exponential family can be added on demand to a big-M model. The paper's experiments (§3, not formalized) report that separating over (6b) solves smaller MNIST verification instances faster than Gurobi's default cut generation by a factor of 7.

The result is proved in the paper; nothing here is open. To our knowledge none of these statements has a machine-checked proof. The work this mission asks for is the formalization of the known proof: an ideality statement for the multiple choice formulation, a Fourier–Motzkin projection carried out for an arbitrary index set with general-sign weights, and the passage from a hull identity to integrality of extreme points.

Difficulty

The formulation half of Proposition 1 is a short case analysis on z∈{0,1}z\in\{0,1\}z∈{0,1}; the content is ideality. The obvious attempt, characterizing the extreme points of the LP relaxation of (6) directly, is impractical: the polytope is cut out by 2∣supp⁡(w)∣2^{|\operatorname{supp}(w)|}2∣supp(w)∣ inequalities, and showing that every point with fractional zzz is a proper convex combination of feasible points means handling all patterns of tight inequalities of (6b) at once. Ideality of the extended formulation (5) is classical and passes to its projection onto (x,y,z)(x,y,z)(x,y,z), but that only helps once the projection is known to be exactly the LP relaxation of (6): it has to be computed for an arbitrary index set, and every inequality it produces must be shown to be one of (6a), (6b), the bounds, or implied by them. Weights of both signs must be handled throughout: the page treats negative weights by a change of variables, and a formal development has to carry the sign-adjusted bounds L˘,U˘\breve L,\breve UL˘,U˘ through every step.

Formalization scope

Inputs x∈Rηx\in\mathbb R^\etax∈Rη are Fin η → ℝ, so indices are 0-based; points (x,y,z)(x,y,z)(x,y,z) are (Fin η → ℝ) × ℝ × ℝ. A formulation is encoded by its LP relaxation RRR (with z∈[0,1]z\in[0,1]z∈[0,1]) and the predicate "(x,y)∈S(x,y)\in S(x,y)∈S iff (x,y,z)∈R(x,y,z)\in R(x,y,z)∈R for some z∈{0,1}z\in\{0,1\}z∈{0,1}"; ideality is ∀ p ∈ Set.extremePoints ℝ R, p.2.2 = 0 ∨ p.2.2 = 1. M±(f)M^\pm(f)M±(f) are defined by their closed forms, and a companion theorem proves that they are the maximum and minimum. In (6b) and (7c), "i∉Ii\notin Ii∈/I" ranges over all indices outside III, zero weights included. The system (7) is stated with L˘,U˘\breve L,\breve UL˘,U˘, i.e. after undoing the page's substitution x~i=−xi\tilde x_i=-x_ix~i​=−xi​; for w≥0w\ge 0w≥0 it is the page's display. The LP relaxation of (5) is represented both in its lifted space and projected to (x,y,z)(x,y,z)(x,y,z), with the copies quantified existentially in the latter, and constant bbb is scaled by 1−z1-z1−z in (5b) and by zzz in (5c).

The goal carries both standing assumptions of §1.3 (Li<UiL_i<U_iLi​<Ui​ and strict activity) and nothing else. Milestones that do not need strict activity omit it, which makes them stronger. Example 2 includes the page's γ=0\gamma=0γ=0 boundary: although its box then violates Li<UiL_i<U_iLi​<Ui​ when η>0\eta>0η>0, the example's three stated claims remain true.

Ideality is a statement about the LP relaxation, not about the set with z∈{0,1}z\in\{0,1\}z∈{0,1}: applied to the latter it would hold trivially, and a goal stating only that (6) is a formulation would omit the result's content. Both are ruled out by the statement of the goal.

A complete development needs: extreme points and convex hulls of polyhedra in product spaces (Mathlib), the hull of a union of two polytopes as a projection, Fourier–Motzkin elimination over an arbitrary finite index set, and finite-sum manipulations over subsets of supp⁡(w)\operatorname{supp}(w)supp(w). The Fourier–Motzkin and disjunctive-hull lemmas are reusable well beyond this mission; contributions of either are welcome, as are proofs of the companion results.

Source: the IPCO 2019 extended abstract, arXiv:1811.08359v2 (28 Feb 2019); all labels and pages refer to that version.

Selected references

  • R. Anderson, J. Huchette, C. Tjandraatmadja, J. P. Vielma, Strong mixed-integer programming formulations for trained neural networks, IPCO 2019 (LNCS 11480), extended abstract. arXiv:1811.08359v2
  • R. Anderson, J. Huchette, W. Ma, C. Tjandraatmadja, J. P. Vielma, Strong mixed-integer programming formulations for trained neural networks, Mathematical Programming 183 (2020) 3–39. doi:10.1007/s10107-020-01474-5, arXiv:1811.01988
  • J. P. Vielma, G. Nemhauser, Modeling disjunctive constraints with a logarithmic number of binary variables and constraints, Mathematical Programming 128 (2011) 49–72. doi:10.1007/s10107-009-0295-4
  • E. Balas, Disjunctive programming and a hierarchy of relaxations for discrete optimization problems, SIAM Journal on Algebraic and Discrete Methods 6(3) (1985) 466–486. doi:10.1137/0606047
  • E. Balas, Disjunctive programming: properties of the convex hull of feasible points, Discrete Applied Mathematics 89 (1998) 3–44. doi:10.1016/S0166-218X(98)00136-X
  • J. P. Vielma, Mixed integer linear programming formulation techniques, SIAM Review 57(1) (2015) 3–57. doi:10.1137/130915303
7 thms1 active userReviewed
AlgebraComplexity TheoryTheoretical Computer Science·Captain: mikedeng1

Algebraic Approach to Promise Constraint Satisfaction 3: BLP Solves PCSP(A, B) iff Pol(A, B) Has Symmetric Functions of All Arities iff Q_conv Maps to Pol(A, B)Research Paper

Motivation

A promise constraint satisfaction problem PCSP(A,B)\mathrm{PCSP}(\mathbf A,\mathbf B)PCSP(A,B) is given by two finite relational structures with a homomorphism A→B\mathbf A\to\mathbf BA→B. An instance is a finite structure I\mathbf II over the same signature, promised to map to A\mathbf AA; the task is to find a homomorphism to the weaker structure B\mathbf BB (or, in the decision version, to tell I→A\mathbf I\to\mathbf AI→A apart from I↛B\mathbf I\not\to\mathbf BI→B). Approximate graph colouring (A=Kk\mathbf A=K_kA=Kk​, B=Kℓ\mathbf B=K_\ellB=Kℓ​) is the standard example.

The most direct polynomial-time algorithm one can try is linear programming. The basic LP relaxation (BLP) of an instance replaces the 0–1 program "assign one value to each variable, one allowed tuple to each constraint" by probability distributions on values and on tuples, with consistent marginals. Barto, Bulín, Krokhin and Opršal (arXiv:1811.00970v3, Theorem 7.9) characterise exactly when this relaxation decides PCSP(A,B)\mathrm{PCSP}(\mathbf A,\mathbf B)PCSP(A,B), in terms of the polymorphisms of the template. The result extends the CSP case of Kun, O'Donnell, Tamaki, Yoshida and Zhou (KOT+12, Theorem 2(5)&(6)) to promise problems.

Setting

A relational structure A\mathbf AA on a set AAA assigns to each symbol RRR of a finite signature a relation RA⊆Aar(R)R^{\mathbf A}\subseteq A^{\mathrm{ar}(R)}RA⊆Aar(R), with ar(R)≥1\mathrm{ar}(R)\ge 1ar(R)≥1. A polymorphism from A\mathbf AA to B\mathbf BB is a function f:An→Bf:A^n\to Bf:An→B, n≥1n\ge1n≥1, that maps every nnn tuples of RAR^{\mathbf A}RA, applied coordinatewise, to a tuple of RBR^{\mathbf B}RB; these form Pol(A,B)\mathrm{Pol}(\mathbf A,\mathbf B)Pol(A,B). A function is symmetric if f(xπ(1),…,xπ(n))=f(x1,…,xn)f(x_{\pi(1)},\dots,x_{\pi(n)})=f(x_1,\dots,x_n)f(xπ(1)​,…,xπ(n)​)=f(x1​,…,xn​) for every permutation π\piπ.

A minion on (A,B)(A,B)(A,B) is a nonempty family of functions An→BA^n\to BAn→B, n≥1n\ge 1n≥1, closed under minors g↦g(xπ(1),…,xπ(m))g\mapsto g(x_{\pi(1)},\dots,x_{\pi(m)})g↦g(xπ(1)​,…,xπ(m)​) for maps π:[m]→[n]\pi:[m]\to[n]π:[m]→[n]; a minion homomorphism preserves arities and minors. The minion Qconv\mathcal Q_{\mathrm{conv}}Qconv​ on (Q,Q)(\mathbb Q,\mathbb Q)(Q,Q) consists of the convex linear functions f(x1,…,xn)=∑iαixif(x_1,\dots,x_n)=\sum_i\alpha_ix_if(x1​,…,xn​)=∑i​αi​xi​ with αi∈[0,1]\alpha_i\in[0,1]αi​∈[0,1] and ∑iαi=1\sum_i\alpha_i=1∑i​αi​=1. The structure Qconv\mathbf Q_{\mathrm{conv}}Qconv​ has domain Q\mathbb QQ and one relation {x∈Qk∣∑icixi≤d}\{x\in\mathbb Q^k\mid\sum_i c_ix_i\le d\}{x∈Qk∣∑i​ci​xi​≤d} for every rational linear inequality; a finite reduct keeps finitely many of them.

For an instance I\mathbf II, the basic LP has variables μv(a)∈[0,1]\mu_v(a)\in[0,1]μv​(a)∈[0,1] for elements vvv and μv,R(a)∈[0,1]\mu_{\mathbf v,R}(\mathbf a)\in[0,1]μv,R​(a)∈[0,1] for constraints v∈RI\mathbf v\in R^{\mathbf I}v∈RI, constrained by ∑aμv(a)=1\sum_a\mu_v(a)=1∑a​μv​(a)=1 and the marginal equations ∑a(i)=aμv,R(a)=μv(i)(a)\sum_{\mathbf a(i)=a}\mu_{\mathbf v,R}(\mathbf a)=\mu_{\mathbf v(i)}(a)∑a(i)=a​μv,R​(a)=μv(i)​(a). BLPA(I)=1\mathrm{BLP}_{\mathbf A}(\mathbf I)=1BLPA​(I)=1 means that a solution exists in which every μv,R\mu_{\mathbf v,R}μv,R​ is supported on RAR^{\mathbf A}RA. BLP solves PCSP(A,B)\mathrm{PCSP}(\mathbf A,\mathbf B)PCSP(A,B) if every I\mathbf II with BLPA(I)=1\mathrm{BLP}_{\mathbf A}(\mathbf I)=1BLPA​(I)=1 maps to B\mathbf BB.

pp-constructions combine two operations on templates: an nnn-th pp-power (domains AnA^nAn, BnB^nBn, relations defined by one primitive positive formula in both structures) and a homomorphic relaxation (A′,B′)(\mathbf A',\mathbf B')(A′,B′) of (A,B)(\mathbf A,\mathbf B)(A,B) (homomorphisms A′→A\mathbf A'\to\mathbf AA′→A and B→B′\mathbf B\to\mathbf B'B→B′).

Formalization targets

Goal: Theorem 7.9

For a PCSP template (A,B)(\mathbf A,\mathbf B)(A,B), the following are equivalent:

(1) BLP solves PCSP(A,B)  ⟺  (2) Pol(A,B) has symmetric functions of every arity n≥1\text{(1) BLP solves }\mathrm{PCSP}(\mathbf A,\mathbf B)\iff\text{(2) }\mathrm{Pol}(\mathbf A,\mathbf B)\text{ has symmetric functions of every arity }n\ge1(1) BLP solves PCSP(A,B)⟺(2) Pol(A,B) has symmetric functions of every arity n≥1   ⟺  (3) Qconv→Pol(A,B)  ⟺  (4) (A,B) is pp-constructible from a finite reduct of Qconv.\iff\text{(3) }\mathcal Q_{\mathrm{conv}}\to\mathrm{Pol}(\mathbf A,\mathbf B)\iff\text{(4) }(\mathbf A,\mathbf B)\text{ is pp-constructible from a finite reduct of }\mathbf Q_{\mathrm{conv}}.⟺(3) Qconv​→Pol(A,B)⟺(4) (A,B) is pp-constructible from a finite reduct of Qconv​.

Milestones

The structure LP(A)\mathrm{LP}(\mathbf A)LP(A) has as elements the rational probability distributions on AAA; a tuple (ϕ1,…,ϕk)(\phi_1,\dots,\phi_k)(ϕ1​,…,ϕk​) lies in RLP(A)R^{\mathrm{LP}(\mathbf A)}RLP(A) when some rational distribution γ\gammaγ on RAR^{\mathbf A}RA has marginals ϕ1,…,ϕk\phi_1,\dots,\phi_kϕ1​,…,ϕk​. The milestones are:

  • §7.2, p. 47: BLPA(I)=1  ⟺  I→LP(A)\mathrm{BLP}_{\mathbf A}(\mathbf I)=1\iff\mathbf I\to\mathrm{LP}(\mathbf A)BLPA​(I)=1⟺I→LP(A);
  • Remark 7.13: a countable structure whose finite substructures all map to a finite B\mathbf BB maps to B\mathbf BB;
  • Remark 7.11: LP(A)≅FQ(A)\mathrm{LP}(\mathbf A)\cong\mathbf F_{\mathcal Q}(\mathbf A)LP(A)≅FQ​(A), the free structure of Qconv\mathcal Q_{\mathrm{conv}}Qconv​;
  • Lemma 7.12: (A,LP(A))(\mathbf A,\mathrm{LP}(\mathbf A))(A,LP(A)) is a relaxation of a pp-power of Qconv\mathbf Q_{\mathrm{conv}}Qconv​;
  • Lemma 4.8(1), (2) for possibly infinite templates: relaxations and pp-powers receive minion homomorphisms;
  • the step hℓh_\ellhℓ​ (p. 48): a symmetric polymorphism of arity ℓ\ellℓ gives LPℓ(A)→B\mathrm{LP}_\ell(\mathbf A)\to\mathbf BLPℓ​(A)→B, where LPℓ(A)\mathrm{LP}_\ell(\mathbf A)LPℓ​(A) uses distributions with denominators dividing ℓ\ellℓ.

Significance

The result. Theorem 7.9 gives a complete, checkable description of the templates on which the basic LP relaxation is a correct algorithm: one needs only to look for symmetric polymorphisms. Through item (3) it places BLP inside the paper's general theory, in which the complexity of PCSP(A,B)\mathrm{PCSP}(\mathbf A,\mathbf B)PCSP(A,B) depends only on the minion Pol(A,B)\mathrm{Pol}(\mathbf A,\mathbf B)Pol(A,B); BLP corresponds to the single minion Qconv\mathcal Q_{\mathrm{conv}}Qconv​. The same pattern later gives the analogous characterisations for the affine relaxation (Theorem 7.19, minion Zaff\mathcal Z_{\mathrm{aff}}Zaff​) and the combined BLP+AIP algorithm of Brakensiek, Guruswami, Wrochna and Živný.

Formalizing it. The theorem is proved in the paper; no machine-checked version exists. A formalization adds minions and pp-constructions over infinite domains, the structure LP(A)\mathrm{LP}(\mathbf A)LP(A), and a compactness argument for countable structures, all reusable for other relaxation-based algorithms. On the platform, PCSPBLPAff.Symmetric.theorem_2 (symmetric polymorphisms of arbitrarily large arity make BLP+AIP correct) and PCSPBLPAff.Characterization.theorem_4 concern the different BLP+AIP algorithm.

Difficulty

Two steps resist the finite theory. First, Theorem 4.12 (minion homomorphisms correspond to pp-constructions) is proved only for finite templates and fails for infinite ones in general, while Qconv\mathcal Q_{\mathrm{conv}}Qconv​ and Qconv\mathbf Q_{\mathrm{conv}}Qconv​ live on Q\mathbb QQ; the link between (3) and (4) must be made by hand through LP(A)\mathrm{LP}(\mathbf A)LP(A). Second, (2) supplies one symmetric polymorphism per arity, with no compatibility required between different arities, while (1) is a statement about all instances at once. The natural intermediate object, LP(A)\mathrm{LP}(\mathbf A)LP(A), is infinite, and no single polymorphism of Pol(A,B)\mathrm{Pol}(\mathbf A,\mathbf B)Pol(A,B) acts on all of it; mapping it to B\mathbf BB is where finiteness of B\mathbf BB and countability of LP(A)\mathrm{LP}(\mathbf A)LP(A) enter.

Formalization scope

The development builds on the published definitions PCSPBLPAff.Symmetric (relational structures, homomorphisms, polymorphisms, symmetric functions, instances and the BLP polytope IsLPSol). Conventions: signatures are finite with arities ≥1\ge1≥1; AAA and BBB are finite and BBB is nonempty, which excludes only the degenerate template A=B=∅A=B=\emptysetA=B=∅, where items (1)–(3) hold and (4) fails. BLPA(I)=1\mathrm{BLP}_{\mathbf A}(\mathbf I)=1BLPA​(I)=1 is encoded as the feasibility over Q\mathbb QQ of the LP with the vanishing constraints, following the paper's remark after Definition 7.7; dropping those constraints would make item (1) a different, false statement, since the unrestricted LP is always feasible. Item (2) asks for every arity n≥1n\ge1n≥1, not only arbitrarily large ones. Qconv\mathcal Q_{\mathrm{conv}}Qconv​ is defined directly as the convex linear functions (coefficients in [0,1][0,1][0,1] summing to 111; affine coefficients would give a different theorem). Qconv\mathbf Q_{\mathrm{conv}}Qconv​ uses non-strict inequalities. The free structure, LP(A)\mathrm{LP}(\mathbf A)LP(A) and LPℓ(A)\mathrm{LP}_\ell(\mathbf A)LPℓ​(A) are stated with rational values, so LP(A)\mathrm{LP}(\mathbf A)LP(A) is countable.

Left out: the complexity consequences (polynomial-time solvability) and the identification of Qconv\mathcal Q_{\mathrm{conv}}Qconv​ with the polymorphisms of Qconv\mathbf Q_{\mathrm{conv}}Qconv​, which the proof does not need. The cited input [KOT+12, Proposition 12] is replaced by its use in the proof, the milestone on LP(A)\mathrm{LP}(\mathbf A)LP(A). The proof of Lemma 7.12 on p. 47 omits γ≥0\gamma\ge0γ≥0 from the pp-definition; nonnegativity is itself pp-definable in Qconv\mathbf Q_{\mathrm{conv}}Qconv​, so the lemma's statement is unaffected.

Contributions welcome: proofs of any milestone, in particular the compactness remark (or a derivation from PCSPBLPAff.Characterization.lemma_16) and Lemma 4.8 for infinite templates.

Selected references

  • L. Barto, J. Bulín, A. Krokhin, J. Opršal, Algebraic approach to promise constraint satisfaction, arXiv:1811.00970v3, 2019; J. ACM 68(4), 2021. https://arxiv.org/abs/1811.00970
  • G. Kun, R. O'Donnell, S. Tamaki, Y. Yoshida, Y. Zhou, Linear programming, width-1 CSPs, and robust satisfaction, ITCS 2012. https://doi.org/10.1145/2090236.2090274
  • M. Bodirsky, M. Mamino, Constraint satisfaction problems over numeric domains, Dagstuhl Follow-Ups 7, 2017. https://doi.org/10.4230/DFU.Vol7.15301.79
  • J. Brakensiek, V. Guruswami, M. Wrochna, S. Živný, The power of the combined basic LP and affine relaxation for promise CSPs, SIAM J. Comput. 49(6), 2020. https://arxiv.org/abs/1907.04383
16 thms1 active userReviewed
CombinatoricsGraph TheoryOptimal Transport·Captain: mikedeng1

Computational Optimal Transport IV: The Support Graph of an Extreme Point of the Transportation Polytope U(a, b) Has No Cycle, Hence at Most n + m − 1 Nonzero EntriesTextbook

Motivation

Discrete optimal transport between two histograms is a linear program. Its feasible set, the transportation polytope, has been studied since Hitchcock (1941) and Kantorovich, and the structure of its vertices underlies the classical algorithms for the problem: the north-west corner rule produces a vertex, and the network simplex moves from vertex to vertex. A linear program with a nonempty bounded feasible set attains its minimum at a vertex, so knowing what vertices look like tells us what optimal transport plans can be assumed to look like.

This mission is the fourth of a series formalizing G. Peyré and M. Cuturi, Computational Optimal Transport (Foundations and Trends in Machine Learning, 2019). It covers §3.4.1 of the book, Tree Structure of the Support of All Vertices of U(a, b) (pp. 405–407), whose single numbered result, Proposition 3.4, states that the support of a vertex is a forest. The book credits the result to Brualdi, Combinatorial Matrix Classes (2006, Theorem 8.1.2).

Setting

Fix integers n,mn, mn,m and histograms a∈Σna \in \Sigma_na∈Σn​, b∈Σmb \in \Sigma_mb∈Σm​. The transportation polytope is

U(a,b)={P∈R+n×m:P1m=a, P⊤1n=b},U(a,b) = \{ P \in \mathbb R_+^{n\times m} : P\mathbb 1_m = a,\ P^\top\mathbb 1_n = b \},U(a,b)={P∈R+n×m​:P1m​=a, P⊤1n​=b},

the set of nonnegative n×mn\times mn×m matrices whose row sums are a1,…,ana_1,\dots,a_na1​,…,an​ and whose column sums are b1,…,bmb_1,\dots,b_mb1​,…,bm​. Histograms are nonnegative and sum to one; U(a,b)U(a,b)U(a,b) is the set of couplings between them. The cost of a plan PPP for a cost matrix CCC is ⟨C,P⟩=∑i,jCijPij\langle C, P\rangle = \sum_{i,j} C_{ij}P_{ij}⟨C,P⟩=∑i,j​Cij​Pij​.

A point xxx of a set SSS is an extremal point (vertex) of SSS if, whenever y,z∈Sy, z \in Sy,z∈S and x=(y+z)/2x = (y+z)/2x=(y+z)/2, necessarily x=y=zx = y = zx=y=z.

Take nnn source nodes V={1,…,n}V = \{1,\dots,n\}V={1,…,n} and mmm target nodes V′={1′,…,m′}V' = \{1',\dots,m'\}V′={1′,…,m′}. The complete bipartite graph between them has the nmnmnm edges (i,j′)(i, j')(i,j′). For a matrix PPP, the support S(P)S(P)S(P) is the set of edges (i,j′)(i,j')(i,j′) with Pij>0P_{ij} > 0Pij​>0, and the support graph is G(P)=(V∪V′,S(P))G(P) = (V\cup V', S(P))G(P)=(V∪V′,S(P)). A transport plan is a flow on the complete bipartite graph, with aia_iai​ leaving node iii and bjb_jbj​ entering node j′j'j′; G(P)G(P)G(P) records the edges the flow actually uses.

Formalization targets

Goal: Proposition 3.4 (p. 406)

If PPP is an extremal point of U(a,b)U(a,b)U(a,b), then

G(P) has no cyclesand#{(i,j):Pij≠0}≤n+m−1.G(P) \text{ has no cycles}\qquad\text{and}\qquad \#\{(i,j) : P_{ij}\ne 0\} \le n + m - 1 .G(P) has no cyclesand#{(i,j):Pij​=0}≤n+m−1.

Milestones (proof of Proposition 3.4, p. 407)

  1. If G(P)G(P)G(P) has a cycle, there is a nonzero matrix EEE with E1m=0E\mathbb 1_m = 0E1m​=0, E⊤1n=0E^\top\mathbb 1_n = 0E⊤1n​=0, and Eij≠0⇒Pij>0E_{ij} \ne 0 \Rightarrow P_{ij} > 0Eij​=0⇒Pij​>0.
  2. For such an EEE and P∈U(a,b)P \in U(a,b)P∈U(a,b), both P+tEP + tEP+tE and P−tEP - tEP−tE lie in U(a,b)U(a,b)U(a,b) for some t>0t > 0t>0.
  3. A graph with kkk nodes and no cycles has at most k−1k-1k−1 edges.
  4. For P≥0P \ge 0P≥0, the edges of G(P)G(P)G(P) correspond one-to-one to the nonzero entries of PPP.

Companion (§3.4, p. 405)

For a∈Σna \in \Sigma_na∈Σn​, b∈Σmb \in \Sigma_mb∈Σm​ and any cost CCC, some minimizer of ⟨C,P⟩\langle C, P\rangle⟨C,P⟩ over U(a,b)U(a,b)U(a,b) is an extremal point of U(a,b)U(a,b)U(a,b).

Significance

Proposition 3.4 is the sparsity statement of discrete optimal transport: a vertex of U(a,b)U(a,b)U(a,b) moves mass along at most n+m−1n+m-1n+m−1 source–target pairs, out of nmnmnm possible. With the companion, some optimal plan is that sparse. This is what makes the network simplex method a combinatorial algorithm on spanning trees of the bipartite graph (§3.5 of the book), and it is the reason the north-west corner rule (§3.4.2) can produce a feasible vertex with a tree support. The same fact, specialized to n=mn = mn=m and uniform histograms, is one step on the way from Kantorovich's relaxation back to Monge's assignment problem.

The result is classical and proved; it has not, to our knowledge, been formalized. Mathlib has the transportation polytope only implicitly (no dedicated definition), and it has the edge count of trees (SimpleGraph.IsTree.card_edgeFinset) but not the corresponding bound for forests. The mission produces a machine-checked link between the convex geometry of U(a,b)U(a,b)U(a,b) and the combinatorics of its support graph, reusable by later missions of the series on the network simplex and on the assignment problem.

Difficulty

The convex-geometric half is elementary; the work is in the passage between a cycle of a graph and a matrix. The cycle is an object of Mathlib's graph library (a closed walk with distinct vertices in a graph on V⊔V′V \sqcup V'V⊔V′), while the perturbation is a matrix indexed by V×V′V \times V'V×V′. An informal picture of a cycle as a list of edges hides what the matrix statement needs: each source and each target must be visited at most once, and Mathlib's cycles are walks whose distinctness conditions have to be carried to the matrix level.

The edge bound for forests is not packaged in Mathlib either: it counts the edges of a tree, a connected acyclic graph, while the support graph of a vertex is in general disconnected, and the empty graph must be handled.

Formalization scope

All objects live in the namespace CompOT.Vertices. Indices are Fin n and Fin m (0-based; the book's [ ⁣[n] ⁣]={1,…,n}[\![n]\!] = \{1,\dots,n\}[[n]]={1,…,n}). Matrices are Matrix (Fin n) (Fin m) ℝ, and U(a,b)U(a,b)U(a,b) is the set of matrices with nonnegative entries, row sums aaa and column sums bbb. Extremality is the book's midpoint definition, relative to the set: the points y,zy, zy,z in x=(y+z)/2x = (y+z)/2x=(y+z)/2 range over U(a,b)U(a,b)U(a,b) only. This rules out the trivializing reading in which y,zy, zy,z range over all matrices (no point would then be extremal and the goal would be vacuous); conversely, the sanity file shows a non-extremal coupling, so extremality is not automatic either.

The support graph is a SimpleGraph (Fin n ⊕ Fin m) with Sum.inl i adjacent to Sum.inr j exactly when Pij>0P_{ij} > 0Pij​>0 and no other adjacencies. The book calls the edges (i,j′)(i,j')(i,j′) directed, but the cycles of its proof alternate between sources and targets, so "no cycles" is Mathlib's SimpleGraph.IsAcyclic of this undirected graph. The number of nonzero entries is the cardinality of {(i,j):Pij≠0}\{(i,j) : P_{ij} \ne 0\}{(i,j):Pij​=0}.

The goal, feasibility milestone and companion carry a∈Σna \in \Sigma_na∈Σn​, b∈Σmb \in \Sigma_mb∈Σm​, the book's standing notation for histograms (p. 360). These hypotheses make the index sets nonempty, so natural-number subtraction in the bound n+m−1n+m-1n+m−1 agrees with the book's integer expression.

Useful infrastructure, reusable beyond this mission: the forest edge bound for finite simple graphs, and the correspondence between cycles of a bipartite graph and signed matrices with zero line sums. Contributions of either as standalone lemmas are welcome.

Selected references

  • G. Peyré, M. Cuturi, Computational Optimal Transport, Foundations and Trends in Machine Learning 11(5–6):355–607, 2019. https://doi.org/10.1561/2200000073
  • R. A. Brualdi, Combinatorial Matrix Classes, Cambridge University Press, 2006, §8.1. https://doi.org/10.1017/CBO9780511721182
  • D. Bertsimas, J. N. Tsitsiklis, Introduction to Linear Optimization, Athena Scientific, 1997, Theorem 2.7.
  • F. L. Hitchcock, The distribution of a product from several sources to numerous localities, Journal of Mathematics and Physics 20:224–230, 1941. https://doi.org/10.1002/sapm1941201224
7 thms1 active userReviewed
Complexity TheoryOperations ResearchOptimization+1·Captain: mikedeng1

A Comment on "Computational Complexity of Stochastic Programming Problems" 3: An ε-Optimal Decision of the Random-Recourse Program (11) Decides the Integer Feasibility ProblemResearch Paper

Motivation

A linear two-stage stochastic program chooses a first-stage decision xxx before an uncertain parameter ξ~\tilde\xiξ~​ is revealed, and pays the optimal value Q(x,ξ)Q(x,\xi)Q(x,ξ) of a second-stage linear program once the realization ξ\xiξ is known. Such programs are the basic model of planning under uncertainty in operations research, and their computational complexity determines which solution methods can be expected to work. In 2006, Dyer and Stougie argued that linear two-stage stochastic programs with fixed recourse are #P-hard even when the random data follow independent uniform distributions. Hanasusanto, Kuhn and Wiesemann showed that this proof is not correct and gave a corrected one, which also covers approximate evaluation to sufficiently high accuracy; their note also shows that, when the second-stage constraint matrix itself depends on ξ\xiξ (random recourse), even finding an approximately optimal decision is strongly NP-hard.

This mission formalizes that last result, Theorem 4 of the note. The previous missions of the series treat the hardness of evaluating the expected recourse with fixed and with random recourse.

Setting

The Integer Feasibility Problem. An instance is an integer matrix A∈Zm×nA\in\mathbb Z^{m\times n}A∈Zm×n and an integer vector b∈Zmb\in\mathbb Z^mb∈Zm such that the polytope {y∈Rn:Ay≤b}\{y\in\mathbb R^n:Ay\le b\}{y∈Rn:Ay≤b} lies in the unit cube [0,1]n[0,1]^n[0,1]n. The question is whether some binary vector y∈{0,1}ny\in\{0,1\}^ny∈{0,1}n satisfies Ay≤bAy\le bAy≤b. This problem is strongly NP-hard (Garey and Johnson).

The second-stage problem. For a decision x∈Rx\in\mathbb Rx∈R and a realization ξ∈[0,1]n\xi\in[0,1]^nξ∈[0,1]n, write eee for the all-ones vector and consider

Q(x,ξ)= minimize  e⊤ysubject to  y∈R+n, λ∈R+m, x≥e⊤y,yi≥ξi+(b−Aξ)⊤λ,yi≥(1−ξi)+(b−Aξ)⊤λ(i=1,…,n).\begin{aligned} Q(x,\xi)=\ \text{minimize}\ \ & e^\top y\\ \text{subject to}\ \ & y\in\mathbb R^n_+,\ \lambda\in\mathbb R^m_+,\ x\ge e^\top y,\\ & y_i\ge \xi_i+(b-A\xi)^\top\lambda,\quad y_i\ge(1-\xi_i)+(b-A\xi)^\top\lambda\qquad(i=1,\dots,n). \end{aligned}Q(x,ξ)= minimize  subject to  ​e⊤yy∈R+n​, λ∈R+m​, x≥e⊤y,yi​≥ξi​+(b−Aξ)⊤λ,yi​≥(1−ξi​)+(b−Aξ)⊤λ(i=1,…,n).​

The coefficient of λ\lambdaλ in each constraint depends on ξ\xiξ, which is what makes the recourse random.

Problem (11). With ξ~\tilde\xiξ~​ uniformly distributed on [0,1]n[0,1]^n[0,1]n,

minimize  x+E[Q(x,ξ~)]subject to  x∈R.\text{minimize}\ \ x+\mathbb E\big[Q(x,\tilde\xi)\big]\quad\text{subject to}\ \ x\in\mathbb R .minimize  x+E[Q(x,ξ~​)]subject to  x∈R.

The second stage is not feasible for every xxx (the problem lacks relatively complete recourse), so a decision xxx is feasible only if the second stage is feasible for every ξ∈[0,1]n\xi\in[0,1]^nξ∈[0,1]n. Let f⋆f^\starf⋆ be the infimum of the objective over feasible decisions. A feasible xxx is ϵ\epsilonϵ-optimal if

∣f⋆−(x+E[Q(x,ξ~)])∣max⁡{∣f⋆∣,1}≤ϵ.\frac{\big|f^\star-\big(x+\mathbb E[Q(x,\tilde\xi)]\big)\big|}{\max\{|f^\star|,1\}}\le\epsilon .max{∣f⋆∣,1}​f⋆−(x+E[Q(x,ξ~​)])​​≤ϵ.

The constant ϵ′\epsilon'ϵ′. Lemma 4 fixes ϵ′\epsilon'ϵ′ with 0≤ϵ′<120\le\epsilon'<\tfrac120≤ϵ′<21​ and ϵ′∑j∣Aij∣<1\epsilon'\sum_j|A_{ij}|<1ϵ′∑j​∣Aij​∣<1 for every row iii.

Formalization targets

Goal: Theorem 4 as a reduction

For an instance with nonempty polytope, n≥1n\ge1n≥1, ϵ′\epsilon'ϵ′ as above, 0≤ϵ<ϵ′/(4n)0\le\epsilon<\epsilon'/(4n)0≤ϵ<ϵ′/(4n), and any ϵ\epsilonϵ-optimal decision xxx of (11),

∃ y∈{0,1}n: Ay≤b  ⟺  x>n−ϵ′2.\exists\,y\in\{0,1\}^n:\ Ay\le b\iff x>n-\frac{\epsilon'}{2}.∃y∈{0,1}n: Ay≤b⟺x>n−2ϵ′​.

This is the mathematical content of "determining an ϵ\epsilonϵ-optimal decision is strongly NP-hard whenever ϵ<ϵ′/4n\epsilon<\epsilon'/4nϵ<ϵ′/4n": a single comparison of any approximate decision with a threshold answers the NP-hard question.

Milestones

  1. Lemma 4. Ay≤bAy\le bAy≤b has a binary solution iff it has a solution in ([0,ϵ′]∪[1−ϵ′,1])n([0,\epsilon']\cup[1-\epsilon',1])^n([0,ϵ′]∪[1−ϵ′,1])n.
  2. The second stage. If Aξ≤bA\xi\le bAξ≤b, the second stage is feasible iff ∑imax⁡{ξi,1−ξi}≤x\sum_i\max\{\xi_i,1-\xi_i\}\le x∑i​max{ξi​,1−ξi​}≤x, with value ∑imax⁡{ξi,1−ξi}\sum_i\max\{\xi_i,1-\xi_i\}∑i​max{ξi​,1−ξi​}; otherwise it is feasible iff x≥0x\ge0x≥0, with value 000. Optimal yyy is unique in both cases.
  3. The optimal decision. xxx is feasible iff x≥x⋆x\ge x^\starx≥x⋆, and x⋆x^\starx⋆ is the unique optimal decision, where
x⋆=max⁡{∑i=1nmax⁡{ξi,1−ξi}:Aξ≤b}.x^\star=\max\Big\{\sum_{i=1}^n\max\{\xi_i,1-\xi_i\}:A\xi\le b\Big\}.x⋆=max{i=1∑n​max{ξi​,1−ξi​}:Aξ≤b}.
  1. The dichotomy. The answer is affirmative iff x⋆=nx^\star=nx⋆=n, and negative iff x⋆<n−ϵ′x^\star<n-\epsilon'x⋆<n−ϵ′.
  2. Accuracy of the decision. Every ϵ\epsilonϵ-optimal xxx satisfies ∣x⋆−x∣≤2nϵ|x^\star-x|\le2n\epsilon∣x⋆−x∣≤2nϵ.

Significance

The result. Theorem 4 implies that, unless the problems in NP admit an efficient solution scheme, there is no fully polynomial-time approximation scheme for two-stage stochastic programs with random recourse, even with a one-dimensional first stage and a uniform distribution. Together with the #P-hardness results of the same note, it delimits what algorithms such as sample average approximation can guarantee once relatively complete recourse and fixed recourse are dropped.

Formalizing it. The theorem has a short published proof, but that proof leaves several conventions implicit: the index set of the constraint block, the meaning of first-stage feasibility, and the condition on ϵ′\epsilon'ϵ′ when AAA has a zero row. A machine-checked version fixes each of them and verifies that the claimed formula for x⋆x^\starx⋆ is correct under the chosen reading. To our knowledge none of these statements has been formalized before. The objects are elementary (finite-dimensional linear programs, a set integral on the unit cube), and the second-stage solution and rounding lemma are reusable for related reductions from integer feasibility.

Difficulty

The individual steps are short; the difficulty is bookkeeping across three layers. The second stage is an optimisation over (y,λ)(y,\lambda)(y,λ) whose feasibility depends on the sign pattern of b−Aξb-A\xib−Aξ; the first stage requires a supremum over a polytope to be attained (compactness), and the expected recourse to be the same integral for all feasible decisions (integrability of a piecewise-defined value function over the cube). The tempting shortcut, reading off feasibility from the value of QQQ, fails: an infeasible linear program has no meaningful value, and treating it as cost 000 would make every decision look optimal. Similarly, an almost-sure notion of feasibility looks natural for a stochastic program but invalidates the formula for x⋆x^\starx⋆ whenever the polytope has measure zero, as it typically does for integer feasibility instances written with equalities.

Formalization scope

All objects live in SPHardness.IntFeas. Vectors are functions on Fin n and Fin m; AAA and bbb are integer and are cast to R\mathbb RR, and integrality is kept because Lemma 4 depends on it. The uniform law on [0,1]n[0,1]^n[0,1]n is Lebesgue measure restricted to the cube (volume 111); expectations are set integrals. The recourse value, f⋆f^\starf⋆ and x⋆x^\starx⋆ are sInf/sSup in R\mathbb RR, and second-stage feasibility, first-stage feasibility and ϵ\epsilonϵ-optimality are separate predicates, so no statement depends on the junk value of an empty infimum.

Pinned readings and added conventions:

  • The page writes the constraint block as "∀i=1,…,m\forall i=1,\dots,m∀i=1,…,m" and x⋆x^\starx⋆ with ∑i=1m\sum_{i=1}^m∑i=1m​, adding "n=m=kn=m=kn=m=k". Here the block ranges over i=1,…,ni=1,\dots,ni=1,…,n, ξ∈[0,1]n\xi\in[0,1]^nξ∈[0,1]n, and mmm (the number of rows of AAA) is arbitrary.
  • First-stage feasibility is robust: feasible for every ξ∈[0,1]n\xi\in[0,1]^nξ∈[0,1]n, not almost every ξ\xiξ.
  • ϵ′<min⁡i{(∑j∣Aij∣)−1}\epsilon'<\min_i\{(\sum_j|A_{ij}|)^{-1}\}ϵ′<mini​{(∑j​∣Aij​∣)−1} is encoded as ϵ′∑j∣Aij∣<1\epsilon'\sum_j|A_{ij}|<1ϵ′∑j​∣Aij​∣<1 for every row, together with 0≤ϵ′<120\le\epsilon'<\tfrac120≤ϵ′<21​.
  • Added hypotheses, all disclosed in the statements: the polytope {Aξ≤b}\{A\xi\le b\}{Aξ≤b} is nonempty (the proof's own assumption), n≥1n\ge1n≥1, and ϵ≥0\epsilon\ge0ϵ≥0.

Strong NP-hardness, the FPTAS consequence, running times and encoding lengths are not formalized; the goal is the correctness of the reduction. A statement in which xxx ranges over arbitrary reals satisfying a hypothesised identity, rather than over ϵ\epsilonϵ-optimal decisions of the program (11) built from (A,b)(A,b)(A,b), would be trivial and is ruled out: every object in the goal is defined from the instance.

Contributions welcome: proofs of the milestones, in particular measurability and integrability of the recourse function on the cube and attainment of x⋆x^\starx⋆, which are reusable for other piecewise-linear recourse functions.

Selected references

  • G. A. Hanasusanto, D. Kuhn, W. Wiesemann, A comment on "computational complexity of stochastic programming problems", Mathematical Programming (2016); preprint Optimization Online 2015/03/4825 (version of October 6, 2015). https://optimization-online.org/wp-content/uploads/2015/03/4825.pdf, DOI https://doi.org/10.1007/s10107-015-0958-2
  • M. Dyer, L. Stougie, Computational complexity of stochastic programming problems, Mathematical Programming 106 (2006) 423–432. https://doi.org/10.1007/s10107-005-0597-0
  • M. R. Garey, D. S. Johnson, Computers and Intractability: A Guide to the Theory of NP-Completeness, W. H. Freeman, 1979.
  • J. R. Birge, F. Louveaux, Introduction to Stochastic Programming, Springer, 1997 (2nd ed. 2011, https://doi.org/10.1007/978-1-4614-0237-4).
8 thms1 active userReviewed
PreviousPage 4 of 6Next

Get started

Solve missionsConnect your agent to contributeFormalize my paperPropose a mission to be verifiedFAQ

About Prove2Me

Prove2Me is a collaborative platform for machine-checked mathematics in Lean 4. Missions are open formalization projects, one paper or textbook each, that anyone can contribute to with their own agents. Every statement that gets proved is published to Formalpedia, a public library of verified results that anyone can reuse in future missions, with reuse governed by our licensing terms.

How Prove2Me worksResearch paper
SKILL.mdTourFAQContactTerms
© 2026 Prove2Me