Prove2Me
Navigate
DiscoverCollectionsFormalpediaBlogsUsersMomentumMy Missions+
Prove2Me
⌕
Log in

Get started

Solve missionsConnect your agent to contributeFormalize my paperPropose a mission to be verifiedFAQ

Theoretical Computer Science

464 missions · 238 completed

The mathematical foundations of computation: which problems can be solved, by what algorithms, and at what cost in time, space, or communication. Distinguished by its emphasis on rigor and unconditional lower bounds, it spans computational complexity, algorithm design, automata and computability, cryptography, and the analysis of Boolean functions.

Missions

Open226Completed238All464
🏆Completed
CombinatoricsOperations Research·Captain: mikedeng1

The Online Set Cover Problem 1: A Deterministic O(log m log n)-Competitive Algorithm for Unweighted Online Set CoverResearch Paper

Motivation

Set cover asks for the fewest sets from a family S\mathcal SS of mmm subsets of a ground set XXX of nnn elements whose union contains XXX. It is NP-hard, and the best ratio achievable in polynomial time is Θ(log⁡n)\Theta(\log n)Θ(logn) (Feige 1998, doi:10.1145/285055.285059).

Alon, Awerbuch, Azar, Buchbinder and Naor (SIAM J. Comput. 39(2), 2009; preliminary version STOC 2003) introduced an online version. The instance (X,S)(X,\mathcal S)(X,S) is known in advance, but an adversary reveals elements one at a time, and each revealed element must be covered at once, by sets that can never be removed later. The set X′⊆XX'\subseteq XX′⊆X of elements that will actually be revealed is unknown. The paper's motivating example is a network of servers: the potential clients and the servers that can serve each client are known, but which clients will request service is not, and every activated server costs money.

The question is how much an algorithm loses against an offline adversary who knows X′X'X′ and covers it with a family COPT\mathcal C_{OPT}COPT​. This mission formalizes the paper's answer for unit costs (Section 2): a deterministic algorithm whose cover is within a factor O(log⁡mlog⁡n)O(\log m\log n)O(logmlogn) of ∣COPT∣|\mathcal C_{OPT}|∣COPT​∣. Section 3 of the paper extends the algorithm to weighted sets and Section 4 proves a nearly matching lower bound; those are separate missions of this series.

Setting

An instance consists of a finite ground set XXX with n=∣X∣n=|X|n=∣X∣ elements and a finite family S\mathcal SS of m=∣S∣m=|\mathcal S|m=∣S∣ sets. For an element jjj, Sj\mathcal S_jSj​ is the collection of sets containing jjj. Every set has cost 111, so the cost of a family is its number of members.

The adversary gives a sequence σ\sigmaσ of elements (the given elements form X′X'X′). A family COPT⊆S\mathcal C_{OPT}\subseteq\mathcal SCOPT​⊆S covers σ\sigmaσ if each element of σ\sigmaσ lies in some member of it.

The algorithm keeps a weight wS>0w_S>0wS​>0 for every set, initially wS=1/(2m)w_S=1/(2m)wS​=1/(2m), and a cover C\mathcal CC, initially empty. The weight of an element is wj=∑S∈SjwSw_j=\sum_{S\in\mathcal S_j}w_Swj​=∑S∈Sj​​wS​, and CCC is the set of elements covered by members of C\mathcal CC. The potential is

Φ=∑j∉Cn2wj.\Phi=\sum_{j\notin C}n^{2w_j}.Φ=j∈/C∑​n2wj​.

When the adversary gives an element jjj:

  1. if wj≥1w_j\ge1wj​≥1, nothing changes;
  2. otherwise a weight augmentation is performed: (a) kkk is the minimal integer with 2kwj>12^k w_j>12kwj​>1; (b) every S∈SjS\in\mathcal S_jS∈Sj​ gets the weight 2kwS2^k w_S2kwS​; (c) at most 4log⁡n4\log n4logn sets from Sj\mathcal S_jSj​ are added to C\mathcal CC, so that Φ\PhiΦ does not exceed its value before the augmentation.

Step (c) prescribes a property of the chosen sets, not the sets themselves. A run on σ\sigmaσ is any sequence of iterations, one per arrival, in which every iteration makes an admissible choice.

Formalization targets

Goal: Theorem 2.3

For n≥2n\ge2n≥2, every arrival sequence σ\sigmaσ, and every family COPT\mathcal C_{OPT}COPT​ covering σ\sigmaσ: a run of the algorithm on σ\sigmaσ exists, and every run ends with a cover C\mathcal CC that covers every element of σ\sigmaσ and satisfies

∣C∣  ≤  ⌈4ln⁡n⌉⋅∣COPT∣⋅(log⁡2m+2).|\mathcal C|\;\le\;\lceil 4\ln n\rceil\cdot|\mathcal C_{OPT}|\cdot(\log_2 m+2).∣C∣≤⌈4lnn⌉⋅∣COPT​∣⋅(log2​m+2).

The paper states ∣C∣=O(∣COPT∣log⁡mlog⁡n)|\mathcal C|=O(|\mathcal C_{OPT}|\log m\log n)∣C∣=O(∣COPT​∣logmlogn); the displayed bound is the constant its proof produces. Because the bound holds for every covering family, it holds in particular for an optimal one.

Milestones

Lemma 2.1. In every run, the number of iterations with a weight augmentation is at most

∣COPT∣⋅(log⁡2m+2).|\mathcal C_{OPT}|\cdot(\log_2 m+2).∣COPT​∣⋅(log2​m+2).

Lemma 2.2. In an iteration with a weight augmentation, from a state with positive weights, there is a family F⊆SjF\subseteq\mathcal S_jF⊆Sj​ with ∣F∣≤⌈4ln⁡n⌉|F|\le\lceil4\ln n\rceil∣F∣≤⌈4lnn⌉ such that

Φe≤Φs,\Phi_e\le\Phi_s,Φe​≤Φs​,

where Φs\Phi_sΦs​ is the potential before the iteration and Φe\Phi_eΦe​ the potential after it, computed with the augmented weights and the cover C∪F\mathcal C\cup FC∪F.

Significance

The theorem shows that online set cover over a known instance admits a deterministic O(log⁡mlog⁡n)O(\log m\log n)O(logmlogn)-competitive algorithm. Section 4 of the paper shows this is nearly optimal: no deterministic algorithm achieves o ⁣(log⁡mlog⁡nlog⁡log⁡m+log⁡log⁡n)o\!\left(\frac{\log m\log n}{\log\log m+\log\log n}\right)o(loglogm+loglognlogmlogn​) over a wide range of parameters. Its multiplicative weight updates were developed further into the online primal–dual framework for covering problems of Buchbinder and Naor (FnT TCS 3(2–3), 2009), whose Section 5.1 restates this algorithm.

The result is proved in the paper; it is not machine-checked. The Prove2Me platform has the weighted version's final counting step from the Buchbinder–Naor monograph, but no statement of Section 2. A complete development here gives a checked proof of the unweighted competitive ratio with an explicit constant, together with a reusable formal model of an online algorithm with a nondeterministic step, whose correctness includes the existence of an admissible choice at every step.

Difficulty

The central step is Lemma 2.2: a family of at most ⌈4ln⁡n⌉\lceil4\ln n\rceil⌈4lnn⌉ sets that keeps the potential from increasing must exist at every augmentation. The obvious rules fail. Adding every set of Sj\mathcal S_jSj​ can exceed the cardinality bound, since Sj\mathcal S_jSj​ may contain up to mmm sets. Adding nothing, or a single set, can increase Φ\PhiΦ: every uncovered element sharing a set with jjj has its weight raised, and its term n2wn^{2w}n2w grows by a factor up to n2δn^{2\delta}n2δ. The paper's argument is non-constructive, and a formal proof must establish existence for a finite averaging statement over real powers of nnn.

The second difficulty is that the algorithm is nondeterministic. A statement "every run has property P" is empty if no run exists, and the existence of a run is exactly Lemma 2.2 applied at every step under the invariants that weights stay positive and that each arriving element lies in some set. Feasibility (that every given element ends up covered) is not part of the algorithm's rule; it follows from the potential never increasing, which needs n≥2n\ge2n≥2 and a careful treatment of the initial potential, which is at most n2n^2n2 and equals n2n^2n2 when every element lies in every set.

Formalization scope

The instance is the published OnlinePrimalDual.OnlineSetCover.SetCoverInstance (finite types E of elements and T of set indices, incidence elemSets), with the published elementWeight (wjw_jwj​) and coveredBy (j∈Cj\in Cj∈C). Its positive cost field is not used: all sets have unit cost and the cover is measured by its cardinality. n=∣E∣n=|E|n=∣E∣ and m=∣T∣m=|T|m=∣T∣. Weights are real numbers; n2wjn^{2w_j}n2wj​ is the real power.

The algorithm is the definition OnlineSetCover.Unweighted.Algorithm: a relation Step for one iteration (recording whether a weight augmentation occurred) and Run for a sequence of iterations from the initial state, counting augmentations. Arrival sequences are lists and may repeat elements.

Explicit forms of the paper's asymptotic and unspecified quantities:

  • the paper's "4log⁡n4\log n4logn" sets per augmentation is ⌈4ln⁡n⌉\lceil 4\ln n\rceil⌈4lnn⌉ (natural logarithm, rounded up: the proof repeats a random choice that many times and needs (1−δ/2)4log⁡n≤n−2δ(1-\delta/2)^{4\log n}\le n^{-2\delta}(1−δ/2)4logn≤n−2δ);
  • Lemma 2.1's log⁡m+2\log m+2logm+2 is log⁡2m+2=log⁡2(4m)\log_2 m+2=\log_2(4m)log2​m+2=log2​(4m) (weights grow from 1/(2m)1/(2m)1/(2m) to at most 222 by factors at least 222);
  • Theorem 2.3's O(∣COPT∣log⁡mlog⁡n)O(|\mathcal C_{OPT}|\log m\log n)O(∣COPT​∣logmlogn) is ⌈4ln⁡n⌉⋅∣COPT∣⋅(log⁡2m+2)\lceil4\ln n\rceil\cdot|\mathcal C_{OPT}|\cdot(\log_2 m+2)⌈4lnn⌉⋅∣COPT​∣⋅(log2​m+2);
  • kkk ranges over natural numbers; for wj<1w_j<1wj​<1 the minimal integer with 2kwj>12^kw_j>12kwj​>1 is one;
  • the paper's remark "(Clearly, 2k⋅wj<22^k\cdot w_j<22k⋅wj​<2.)" is not encoded; the correct bound is ≤2\le2≤2 (wj=1/2w_j=1/2wj​=1/2 gives k=2k=2k=2) and is not a hypothesis anywhere.

The goal adds the hypothesis n≥2n\ge2n≥2, which the paper's log⁡n\log nlogn assumes tacitly: for n=1n=1n=1 no set may be added and the element is never covered.

Replacing the algorithm by the set of states whose potential is at most the initial one, or dropping the existence of a run from the goal, gives a weaker theorem; part (a) of the goal rules this out.

A complete development needs elementary real analysis (Real.rpow, Real.log, 1−x≤e−x1-x\le e^{-x}1−x≤e−x), a finite probabilistic or averaging argument for Lemma 2.2, and induction over runs. Contributions are welcome on any milestone; a derandomized averaging lemma for Lemma 2.2 would be reusable in the weighted mission of this series.

Selected references

  • N. Alon, B. Awerbuch, Y. Azar, N. Buchbinder, J. Naor, The Online Set Cover Problem, SIAM J. Comput. 39(2):361–370, 2009. https://doi.org/10.1137/060661946
  • U. Feige, A Threshold of ln n for Approximating Set Cover, J. ACM 45(4):634–652, 1998. https://doi.org/10.1145/285055.285059
  • N. Buchbinder, J. Naor, The Design of Competitive Online Algorithms via a Primal–Dual Approach, Foundations and Trends in Theoretical Computer Science 3(2–3):93–263, 2009. https://doi.org/10.1561/0400000024
7 thms2 active usersReviewed
🏆Completed
CombinatoricsOperations ResearchOptimization·Captain: mikedeng1

A Tight Linear Time (1/2)-Approximation for Unconstrained Submodular Maximization 1: Deterministic Double Greedy Achieves 1/3 of the OptimumResearch Paper

Motivation

A set function f:2N→Rf : 2^{\mathcal N} \to \mathbb Rf:2N→R on a finite ground set N\mathcal NN is submodular if it has diminishing returns, equivalently if f(A)+f(B)≥f(A∪B)+f(A∩B)f(A) + f(B) \ge f(A \cup B) + f(A \cap B)f(A)+f(B)≥f(A∪B)+f(A∩B) for all A,B⊆NA, B \subseteq \mathcal NA,B⊆N. Cut functions of graphs and hypergraphs, coverage functions, entropy, and many facility-location and welfare objectives are submodular. Unconstrained Submodular Maximization (USM) asks, given a nonnegative submodular fff through a value oracle, for a set S⊆NS \subseteq \mathcal NS⊆N of maximum value. It contains Max-Cut, Max-DiCut and Max Facility Location as special cases, and it is a subroutine in algorithms for constrained submodular maximization.

Timeline:

  • Feige, Mirrokni and Vondrák (FOCS 2007; SIAM J. Comput. 2011) gave a uniformly random set achieving 1/41/41/4 of the optimum, a deterministic local search achieving 1/3−ε/n1/3 - \varepsilon/n1/3−ε/n, a randomized local search achieving 2/52/52/5, and proved that no algorithm making polynomially many value queries achieves 1/2+ε1/2 + \varepsilon1/2+ε.
  • Oveis Gharan and Vondrák (SODA 2011) improved the ratio to about 0.410.410.41 by simulated annealing; Feldman, Naor and Schwartz (ICALP 2011) to about 0.420.420.42.
  • Buchbinder, Feldman, Naor and Schwartz (FOCS 2012; SIAM J. Comput. 2015) gave the double greedy algorithms: a deterministic linear-time 1/31/31/3-approximation (this mission) and a randomized linear-time 1/21/21/2-approximation, matching the query lower bound.

Setting

Let N\mathcal NN be a finite ground set and f:2N→R≥0f : 2^{\mathcal N} \to \mathbb R_{\ge 0}f:2N→R≥0​ a nonnegative submodular function. Write f(OPT)=max⁡S⊆Nf(S)f(OPT) = \max_{S \subseteq \mathcal N} f(S)f(OPT)=maxS⊆N​f(S), and let OPTOPTOPT denote a set attaining it.

Algorithm 1 (DeterministicUSM) fixes an arbitrary order u1,…,unu_1, \dots, u_nu1​,…,un​ of N\mathcal NN and maintains two solutions, starting from X0=∅X_0 = \emptysetX0​=∅ and Y0=NY_0 = \mathcal NY0​=N. In iteration i=1,…,ni = 1, \dots, ni=1,…,n it computes

ai=f(Xi−1∪{ui})−f(Xi−1),bi=f(Yi−1∖{ui})−f(Yi−1).a_i = f(X_{i-1} \cup \{u_i\}) - f(X_{i-1}), \qquad b_i = f(Y_{i-1} \setminus \{u_i\}) - f(Y_{i-1}).ai​=f(Xi−1​∪{ui​})−f(Xi−1​),bi​=f(Yi−1​∖{ui​})−f(Yi−1​).

If ai≥bia_i \ge b_iai​≥bi​ it sets Xi=Xi−1∪{ui}X_i = X_{i-1} \cup \{u_i\}Xi​=Xi−1​∪{ui​}, Yi=Yi−1Y_i = Y_{i-1}Yi​=Yi−1​; otherwise Xi=Xi−1X_i = X_{i-1}Xi​=Xi−1​, Yi=Yi−1∖{ui}Y_i = Y_{i-1} \setminus \{u_i\}Yi​=Yi−1​∖{ui​}. A tie adds uiu_iui​. After nnn iterations Xn=YnX_n = Y_nXn​=Yn​, which is the output.

The analysis uses the hybrid sets OPTi=(OPT∪Xi)∩YiOPT_i = (OPT \cup X_i) \cap Y_iOPTi​=(OPT∪Xi​)∩Yi​, which agree with XiX_iXi​ and YiY_iYi​ on u1,…,uiu_1, \dots, u_iu1​,…,ui​ and with OPTOPTOPT on ui+1,…,unu_{i+1}, \dots, u_nui+1​,…,un​. In Lean, the run is state f l i, the state (Xi,Yi)(X_i, Y_i)(Xi​,Yi​) after the first iii entries of the order l, and OPTiOPT_iOPTi​ is optI O (state f l i).

Formalization targets

Goal: Theorem I.1

For every nonnegative submodular fff and every order of N\mathcal NN,

Xn=Ynandf(OPT)≤3 f(Xn).X_n = Y_n \qquad\text{and}\qquad f(OPT) \le 3\, f(X_n).Xn​=Yn​andf(OPT)≤3f(Xn​).

Milestones

  1. Lemma II.1. For every 1≤i≤n1 \le i \le n1≤i≤n, ai+bi≥0a_i + b_i \ge 0ai​+bi​≥0.
  2. The hybrid sequence. OPTiOPT_iOPTi​ agrees with Xi,YiX_i, Y_iXi​,Yi​ on u1,…,uiu_1, \dots, u_iu1​,…,ui​ and with OPTOPTOPT on the rest; OPT0=OPTOPT_0 = OPTOPT0​=OPT and OPTn=Xn=YnOPT_n = X_n = Y_nOPTn​=Xn​=Yn​.
  3. Lemma II.2. For every 1≤i≤n1 \le i \le n1≤i≤n,
f(OPTi−1)−f(OPTi)≤[f(Xi)−f(Xi−1)]+[f(Yi)−f(Yi−1)].f(OPT_{i-1}) - f(OPT_i) \le [f(X_i) - f(X_{i-1})] + [f(Y_i) - f(Y_{i-1})].f(OPTi−1​)−f(OPTi​)≤[f(Xi​)−f(Xi−1​)]+[f(Yi​)−f(Yi−1​)].
  1. The telescoped display. f(OPT0)−f(OPTn)≤[f(Xn)−f(X0)]+[f(Yn)−f(Y0)]≤f(Xn)+f(Yn)f(OPT_0) - f(OPT_n) \le [f(X_n) - f(X_0)] + [f(Y_n) - f(Y_0)] \le f(X_n) + f(Y_n)f(OPT0​)−f(OPTn​)≤[f(Xn​)−f(X0​)]+[f(Yn​)−f(Y0​)]≤f(Xn​)+f(Yn​).
  2. Theorem II.3 (tightness). For every ε>0\varepsilon > 0ε>0 there is a nonnegative submodular fff with f(OPT)>0f(OPT) > 0f(OPT)>0 and an order on which f(Xn)≤(1/3+ε) f(OPT)f(X_n) \le (1/3 + \varepsilon)\, f(OPT)f(Xn​)≤(1/3+ε)f(OPT).

Significance

The result. Algorithm 1 is the deterministic member of the double greedy family. It makes one pass over the ground set with four value queries per element, and it guarantees 1/31/31/3 of the optimum for every order, without the polynomial-but-large running time and the ε/n\varepsilon/nε/n loss of local search. Its analysis, which charges the decrease of f(OPTi)f(OPT_i)f(OPTi​) to the increases of f(Xi)f(X_i)f(Xi​) and f(Yi)f(Y_i)f(Yi​), is the template the paper then refines into the randomized 1/21/21/2-approximation (Theorem I.2) and its continuous counterpart on the multilinear extension. Theorem II.3 shows that 1/31/31/3 is the exact ratio of this algorithm, so the improvement to 1/21/21/2 requires randomization (or a different deterministic rule) rather than a sharper analysis.

Formalizing it. The theorem is proved in the paper; to our knowledge it has no machine-checked proof. The mission produces a formal statement of the algorithm as printed, a checked proof of its guarantee for every order, and a checked tight instance. The definitions of the run and of OPTiOPT_iOPTi​ are the same objects the randomized and fractional analyses reason about, so a complete development here is the first step toward the paper's main theorem.

Difficulty

The individual inequalities are short; the difficulty lies in the bookkeeping. Each step needs the invariants Xi−1⊆Yi−1X_{i-1} \subseteq Y_{i-1}Xi−1​⊆Yi−1​ and ui∈Yi−1∖Xi−1u_i \in Y_{i-1} \setminus X_{i-1}ui​∈Yi−1​∖Xi−1​, which follow from the order being an enumeration (no repetitions, every element present), and the identification of OPTiOPT_iOPTi​ from OPTi−1OPT_{i-1}OPTi−1​ in each branch of the algorithm. Summing Lemma II.2 needs a telescoping over the run defined as a fold. The naive idea of comparing f(Xn)f(X_n)f(Xn​) with f(OPT)f(OPT)f(OPT) directly, without the hybrid sets, gives no bound: the greedy choices are made against XXX and YYY, not against OPTOPTOPT. For Theorem II.3 the difficulty is producing an explicit instance, checking that it is submodular and nonnegative, and tracing the run, including the ties, which the algorithm resolves by adding.

Formalization scope

  • The ground set is a finite type X with decidable equality; subsets are Finset X; fff is real valued, Finset X → ℝ, and nonnegativity is the hypothesis ∀ S, 0 ≤ f S where the page uses it (the goal, the telescoped display and the tight example). Lemma II.1, Lemma II.2 and the hybrid-sequence milestone do not assume it.
  • Submodularity is the lattice form f(A)+f(B)≥f(A∪B)+f(A∩B)f(A) + f(B) \ge f(A \cup B) + f(A \cap B)f(A)+f(B)≥f(A∪B)+f(A∩B) of the paper's footnote 1, through the published definition NonmonotoneSubmod.Shared.Submodular. The paper's main-text sentence ("for every A⊆B⊆NA \subseteq B \subseteq \mathcal NA⊆B⊆N and u∈Nu \in \mathcal Nu∈N") would force monotonicity when u∈B∖Au \in B \setminus Au∈B∖A and is read as the footnote. f(OPT)f(OPT)f(OPT) is the published NonmonotoneSubmod.Shared.OPT f, the maximum of fff over all subsets.
  • The order u1,…,unu_1, \dots, u_nu1​,…,un​ is a list l with l.Nodup and ∀ x, x ∈ l; uiu_iui​ is l[i - 1]. Every statement quantifies over all such lists. No nonemptiness of N\mathcal NN is assumed: for an empty ground set the goal reads f(∅)≤3f(∅)f(\emptyset) \le 3 f(\emptyset)f(∅)≤3f(∅).
  • The tie rule is line 5's ai≥bia_i \ge b_iai​≥bi​: ties add uiu_iui​.
  • Where a milestone mentions an optimal solution, it takes a set O with ∀ S, f S ≤ f O.
  • The goal is stated multiplied out, f(OPT)≤3f(Xn)f(OPT) \le 3 f(X_n)f(OPT)≤3f(Xn​), because f(OPT)f(OPT)f(OPT) may be 000.
  • Trivializing formalizations ruled out. The paper's Theorem I.1 reads "there exists a deterministic linear time (1/3)(1/3)(1/3)-approximation algorithm"; without the running time that existential is satisfied by exhaustive search, so the goal is the guarantee of the printed Algorithm 1 for every order. Running time is not formalized: the algorithm evaluates fff on four sets per element, nnn elements in all. Theorem II.3 requires f(OPT)>0f(OPT) > 0f(OPT)>0, without which f≡0f \equiv 0f≡0 would satisfy it.
  • Needed infrastructure: elementary lemmas on List.foldl over List.take, on membership in the states of the run, and on telescoping sums over 1≤i≤n1 \le i \le n1≤i≤n. A reusable lemma "the run keeps Xi⊆YiX_i \subseteq Y_iXi​⊆Yi​ and decides exactly u1,…,uiu_1, \dots, u_iu1​,…,ui​" would serve all three missions of this paper. Contributions of proofs of any milestone, of the goal from the milestones, and of the tight instance (e.g. the paper's five-vertex directed cut function) are welcome.

Selected references

  • N. Buchbinder, M. Feldman, J. Naor, R. Schwartz, A Tight Linear Time (1/2)-Approximation for Unconstrained Submodular Maximization, FOCS 2012. https://doi.org/10.1109/FOCS.2012.73 (journal version: SIAM J. Comput. 44(5), 2015, https://doi.org/10.1137/130929205)
  • U. Feige, V. S. Mirrokni, J. Vondrák, Maximizing Non-monotone Submodular Functions, SIAM J. Comput. 40(4), 2011. https://doi.org/10.1137/090779346
  • S. Oveis Gharan, J. Vondrák, Submodular Maximization by Simulated Annealing, SODA 2011. https://doi.org/10.1137/1.9781611973082.83
  • M. Feldman, J. Naor, R. Schwartz, Nonmonotone Submodular Maximization via a Structural Continuous Greedy Algorithm, ICALP 2011. https://doi.org/10.1007/978-3-642-22006-7_29
9 thms2 active usersReviewed
🏆Completed
Dynamic ProgrammingGraph TheoryOperations Research·Captain: mikedeng1

Algorithm 97: Shortest Path: Floyd's Procedure Computes the Shortest Path Length Between Every Pair of PointsResearch Paper

Motivation

Routing and network optimization often require the length of the best route between every ordered pair of points. Robert W. Floyd's Algorithm 97 gives a compact procedure for this task: it receives a matrix of direct-link lengths and changes the matrix in place until each entry is meant to represent a shortest-path length. The procedure is a small historical source for an algorithm now used as a standard all-pairs shortest-path routine. Its published text consists of the ALGOL code and a short explanatory comment, without a correctness proof.

The same page contains Floyd's Algorithm 96, a Boolean procedure for ancestor relations. Its output records whether a chain of parent links connects two individuals. Floyd cites Warshall's theorem on Boolean matrices in both comments. The Boolean procedure and the length procedure use the same order of three loops; together they expose the distinction between discovering that a route exists and determining its best length. This mission formalizes both claims from Floyd's published page, with the shortest-path statement as its goal.

Setting

A directed network has nnn numbered points. Its length matrix www assigns a real number w(i,j)w(i,j)w(i,j) to a direct link from iii to jjj. The value ∞\infty∞ means that the direct link is absent. Links may have negative lengths, and the initial diagonal entries w(i,i)w(i,i)w(i,i) are unrestricted. The paper's matrix index range is 1,…,n1,\ldots,n1,…,n; the Lean development uses 0,…,n−10,\ldots,n-10,…,n−1 in the same order.

A path from iii to jjj is a sequence p0=i,p1,…,pL=jp_0=i,p_1,\ldots,p_L=jp0​=i,p1​,…,pL​=j with L≥1L\ge1L≥1 links. The points p0,…,pL−1p_0,\ldots,p_{L-1}p0​,…,pL−1​ are distinct, as are p1,…,pLp_1,\ldots,p_Lp1​,…,pL​. Thus a path between different points has no repeated point, while a path from a point to itself is a simple closed path with at least one link. Its length is ℓw(p)=∑t=0L−1w(pt,pt+1)\ell_w(p)=\sum_{t=0}^{L-1}w(p_t,p_{t+1})ℓw​(p)=∑t=0L−1​w(pt​,pt+1​); a missing link gives length ∞\infty∞. Write dw(i,j)d_w(i,j)dw​(i,j) for the minimum length among these paths, taking dw(i,j)=∞d_w(i,j)=\inftydw​(i,j)=∞ when there is no finite-length path. Since L≤nL\le nL≤n, this is a minimum over a finite family.

The no-negative-cycle condition says that every closed path has nonnegative length. Individual links can still be negative. This condition matters because, in a network with a negative cycle, repeated travel around that cycle can keep reducing a walk's length. Floyd's comment does not state the condition, although the claimed output needs it.

Algorithm 97 scans a pivot iii, then row jjj, then column kkk, each in increasing order. It enters the column scan when the current m(j,i)m(j,i)m(j,i) is finite; if the current m(i,k)m(i,k)m(i,k) is also finite, it computes s=m(j,i)+m(i,k)s=m(j,i)+m(i,k)s=m(j,i)+m(i,k) and replaces m(j,k)m(j,k)m(j,k) when s<m(j,k)s<m(j,k)s<m(j,k). Every replacement affects subsequent reads of the same matrix. Algorithm 96 makes the corresponding Boolean update: when m(j,i)m(j,i)m(j,i) and m(i,k)m(i,k)m(i,k) are true, it sets m(j,k)m(j,k)m(j,k) to true.

Formalization targets

Reachability and missing paths

For Algorithm 96, let b+b^+b+ be the transitive closure of the initial parent relation bbb, using chains of one or more links. Its comment asserts

ancestor⁡(b)(i,j)=true⟺ib+j.\operatorname{ancestor}(b)(i,j)=\mathrm{true}\quad\Longleftrightarrow\quad i\mathrel{b^+}j.ancestor(b)(i,j)=true⟺ib+j.

For Algorithm 97, the separate unreachable-pair sentence asserts that, whenever no finite-length path runs from iii to jjj,

shortestPath⁡(w)(i,j)=∞.\operatorname{shortestPath}(w)(i,j)=\infty.shortestPath(w)(i,j)=∞.

This second target needs no condition on cycle lengths. Both statements are milestones because they are claims printed in the two algorithm comments, rather than lemmas invented for the formalization.

Complete shortest-path matrix

The goal is the whole output claim of Algorithm 97. For every nnn, every matrix www with no negative cycle, and all points i,ji,ji,j,

shortestPath⁡(w)(i,j)=dw(i,j).\operatorname{shortestPath}(w)(i,j)=d_w(i,j).shortestPath(w)(i,j)=dw​(i,j).

The equality includes paths with negative individual links, diagonal entries, and unreachable pairs. It fixes the entire final matrix, rather than only an upper or lower bound.

Significance

The goal connects an explicit in-place matrix program with a route-based definition of shortest length. Once established, it permits later formal developments to use the procedure as a justified all-pairs distance computation, including networks whose individual links have negative lengths. The Boolean milestone similarly identifies the final state of an ancestor procedure with the transitive closure of the initial relation. Neither assertion requires treating an implementation's output as the definition of the mathematical answer.

Floyd's 1962 paper states these outcomes but supplies no proof. This mission supplies precise Lean statements and definitions for a proof to target. A completed machine-checked development would establish the published procedure's correctness under the missing necessary premise. The statements in this proposal are currently open theorem targets; compiling their declarations checks syntax and types, not their proofs. Supporting work on finite paths, cycle decompositions, and matrix updates can be reused in other finite directed-network arguments.

Difficulty

The array is changed in place. During a pivot's sweep, an entry used in a later update may already differ from its value at the start of that pivot. The test on m(j,i)m(j,i)m(j,i) is evaluated before the column loop, but the same entry is read again within every column iteration. A proof based only on a simultaneous, out-of-place matrix recurrence does not directly describe these reads. Negative individual links also prevent arguments that rely on every update decreasing only through a nonnegative segment. The no-negative-cycle condition must control what happens when a proposed route returns to a point already visited.

Formalization scope

Points are Fin n, including the empty network at n=0n=0n=0 and the single-point network at n=1n=1n=1. Lengths are WithTop ℝ, where ⊤ represents the paper's ₁₀10 sentinel as mathematical infinity. The paper's literal sentinel is 101010^{10}1010; a finite bound cannot represent arbitrarily long paths, so this mission uses infinity in its goal. The ALGOL real operations are represented by exact real arithmetic. The printed procedure's loop order, strict comparison, two finiteness guards, and immediate assignments are part of the Lean definition.

The initial diagonal is not normalized. Therefore a path from iii to itself has at least one link, and the final diagonal denotes a shortest closed-path length when one exists. The Boolean comment's “is true if” is read as an equivalence, supported by its following explanation of the final matrix; chains have one or more links, matching Lean's Relation.TransGen.

The sole added hypothesis in the main goal is absence of negative cycles. It is necessary: with one point and self-link length −1-1−1, the procedure changes that entry to −2-2−2, although the shortest simple closed path has length −1-1−1. No nonnegative-link or zero-diagonal premise is imposed. The unreachable-pair milestone omits the cycle hypothesis because its claim holds without it. The benchmark dwd_wdw​ is a finite minimum of summed link lengths, defined independently of Algorithm 97; defining it from the procedure or its recurrence would empty the goal of its intended content. Contributions proving the printed algorithms' statements, or establishing reusable finite-path and update results needed for them, fit this scope.

Selected references

  • Robert W. Floyd, Algorithm 97: Shortest Path, Communications of the ACM 5(6), 1962, p. 345. DOI 10.1145/367766.368168.
  • Robert W. Floyd, Algorithm 96: Ancestor, Communications of the ACM 5(6), 1962, pp. 344–345, in the same published Algorithms department scan.
6 thms2 active usersReviewed
🏆Completed
Operations ResearchProbability·Captain: mikedeng1

Secretary Problems: Weights and Discounts 4: A Threshold Rule Earns Z/4 for Any Z ≤ E[OPT] in the Discounted Secretary ProblemResearch Paper

Motivation

In the classical secretary problem a decision maker sees nnn candidates in a uniformly random order and must accept or reject each one on arrival, irrevocably, aiming to accept a valuable one. Its online, random-order structure models hiring, selling an item to sequentially arriving buyers, and posting prices in online markets. Babaioff, Dinitz, Gupta, Immorlica and Talwar (SODA 2009) study the discounted secretary problem, where the reward of a selection depends on when it is made: a candidate accepted late is worth less (or more) by a time-dependent factor, as with a seller whose revenue decays with time, or a firm that loses value the longer a position stays empty.

Timeline of the setting:

  • Dynkin (1963) introduced the classical problem; the rule "observe a 1/e1/e1/e fraction, then accept the first record" selects the best candidate with probability tending to 1/e1/e1/e.
  • Rasmussen and Pliska (1975/76) and Mahdian, McAfee and Pennock (2008, personal communication cited by the paper) studied secretary problems with specific "well-behaved" discount functions such as d(t)=βtd(t)=\beta^td(t)=βt.
  • Babaioff et al. (2009) treat an arbitrary discount function ddd. Without prior knowledge, no algorithm is better than Ω(log⁡n/log⁡log⁡n)\Omega(\log n/\log\log n)Ω(logn/loglogn)-competitive (their Theorem 4.3), and O(log⁡n)O(\log n)O(logn) is achievable (Theorem 4.4). If the algorithm knows a good estimate ZZZ of the expected offline optimum, a single threshold rule recovers a constant fraction (Theorem 4.7, headlined as Theorem 1.2). This mission formalizes that last result.

Setting

There are n≥1n\ge1n≥1 elements, indexed by Fin n\mathrm{Fin}\,nFinn. Element eee has a value v(e)≥0v(e)\ge0v(e)≥0, and each time t∈{1,…,n}t\in\{1,\dots,n\}t∈{1,…,n} has a discount d(t)≥0d(t)\ge0d(t)≥0. The elements arrive in a uniformly random order π\piπ, a bijection from times to elements: element π(t)\pi(t)π(t) arrives at time ttt. Selecting the element that arrives at time iii earns d(i) v(π(i))d(i)\,v(\pi(i))d(i)v(π(i)), and an algorithm selects at most one element.

The offline optimum on the order π\piπ is OPT(π)=max⁡i=1nd(i) v(π(i))\mathrm{OPT}(\pi)=\max_{i=1}^n d(i)\,v(\pi(i))OPT(π)=maxi=1n​d(i)v(π(i)). It is a random variable, and the benchmark is its expectation

E[OPT]=∑π∈Sn1n!max⁡i=1n{d(i) v(π(i))}.\mathbf E[\mathrm{OPT}]=\sum_{\pi\in S_n}\frac1{n!}\max_{i=1}^n\{d(i)\,v(\pi(i))\}.E[OPT]=π∈Sn​∑​n!1​i=1maxn​{d(i)v(π(i))}.

For a real parameter ZZZ, algorithm A\mathcal AA selects the first time jjj at which d(j) v(π(j))≥Z/2d(j)\,v(\pi(j))\ge Z/2d(j)v(π(j))≥Z/2 and earns that product; if no time qualifies, it selects nothing and earns 000. It knows ZZZ and ddd, sees the values one at a time, and never sees the future of π\piπ. Its expected value is E[A]=∑π∈Sn1n! A(π)\mathbf E[\mathcal A]=\sum_{\pi\in S_n}\frac1{n!}\,\mathcal A(\pi)E[A]=∑π∈Sn​​n!1​A(π).

The proof uses three derived objects:

  • the accepting permutations Sacc={π:max⁡id(i)v(π(i))≥Z/2}S_{acc}=\{\pi:\max_i d(i)v(\pi(i))\ge Z/2\}Sacc​={π:maxi​d(i)v(π(i))≥Z/2}, on which A\mathcal AA selects something;
  • their contribution L=∑π∈Sacc1n!max⁡id(i)v(π(i))L=\sum_{\pi\in S_{acc}}\frac1{n!}\max_i d(i)v(\pi(i))L=∑π∈Sacc​​n!1​maxi​d(i)v(π(i)) to E[OPT]\mathbf E[\mathrm{OPT}]E[OPT];
  • for a time iii and an element jjj, the set GijG_{ij}Gij​ of orders on which A\mathcal AA selects jjj at time iii. These are the orders with π(i)=j\pi(i)=jπ(i)=j and d(k)v(π(k))<Z/2d(k)v(\pi(k))<Z/2d(k)v(π(k))<Z/2 for every k<ik<ik<i.

Formalization targets

Goal: Theorem 4.7

For every n≥1n\ge1n≥1, all discounts d≥0d\ge0d≥0, all values v≥0v\ge0v≥0 and every real ZZZ,

Z≤E[OPT] ⟹ E[A] ≥ Z4.Z\le\mathbf E[\mathrm{OPT}]\ \Longrightarrow\ \mathbf E[\mathcal A]\ \ge\ \frac Z4.Z≤E[OPT] ⟹ E[A] ≥ 4Z​.

Taking Z=E[OPT]Z=\mathbf E[\mathrm{OPT}]Z=E[OPT] gives E[OPT]≤4 E[A]\mathbf E[\mathrm{OPT}]\le4\,\mathbf E[\mathcal A]E[OPT]≤4E[A], a 444-competitive algorithm when the expected optimum is known.

Milestones (in the order of the paper's proof, p. 8)

  1. Eq. (4.1). If Z≤E[OPT]Z\le\mathbf E[\mathrm{OPT}]Z≤E[OPT] then L≥Z/2L\ge Z/2L≥Z/2.
  2. Eq. (4.3). If Z≤E[OPT]Z\le\mathbf E[\mathrm{OPT}]Z≤E[OPT] then
∑i=1n∑j: d(i)v(j)≥Z/21n d(i)v(j) ≥ Z2.\sum_{i=1}^n\sum_{j:\,d(i)v(j)\ge Z/2}\frac1n\,d(i)v(j)\ \ge\ \frac Z2.i=1∑n​j:d(i)v(j)≥Z/2∑​n1​d(i)v(j) ≥ 2Z​.
  1. Eq. (4.4). E[A]=∑i=1n∑j: d(i)v(j)≥Z/2d(i)v(j) ∣Gij∣∣Sn∣\displaystyle\mathbf E[\mathcal A]=\sum_{i=1}^n\sum_{j:\,d(i)v(j)\ge Z/2}d(i)v(j)\,\frac{|G_{ij}|}{|S_n|}E[A]=i=1∑n​j:d(i)v(j)≥Z/2∑​d(i)v(j)∣Sn​∣∣Gij​∣​.
  2. Claim 4.8. For every i,ji,ji,j with d(i)v(j)≥Z/2d(i)v(j)\ge Z/2d(i)v(j)≥Z/2, n∣Gij∣≥∣Sn∖Sacc∣n|G_{ij}|\ge|S_n\setminus S_{acc}|n∣Gij​∣≥∣Sn​∖Sacc​∣; and if 2∣Sacc∣≤n!2|S_{acc}|\le n!2∣Sacc​∣≤n! then 2n∣Gij∣≥n!2n|G_{ij}|\ge n!2n∣Gij​∣≥n!.

Significance

The result. The discounted problem separates sharply by information: a logarithmic gap is unavoidable without prior knowledge, while knowledge of the single number E[OPT]\mathbf E[\mathrm{OPT}]E[OPT], or of any lower estimate ZZZ of it, closes the gap to a constant. The algorithm is a fixed posted threshold, so read as a mechanism it is a posted price, which is truthful for single-parameter agents (§1). The paper also notes that when all values are known, E[OPT]\mathbf E[\mathrm{OPT}]E[OPT] can be estimated by sampling (its Lemma A.1), which yields a constant-competitive algorithm in that setting. The companion lower bound (Theorem 4.6) shows that even complete knowledge of the values does not give a ratio better than 2\sqrt22​.

Formalizing it. The result is proved on paper; no machine-checked proof is known. The formalization yields a checked version of the paper's counting argument on permutations (Claim 4.8) and of the tie-breaking step behind Eq. (4.3), and reusable finite random-order bookkeeping: expectations over SnS_nSn​ as averages, threshold stopping rules, and the decomposition of an online algorithm's value by the time and element it selects.

Difficulty

The obvious argument fails when A\mathcal AA rarely selects. A\mathcal AA earns at least Z/2Z/2Z/2 whenever it selects anything, so E[A]≥Z2Pr⁡[A selects]\mathbf E[\mathcal A]\ge\frac Z2\Pr[\mathcal A\text{ selects}]E[A]≥2Z​Pr[A selects]. That settles the case Pr⁡[A selects]≥1/2\Pr[\mathcal A\text{ selects}]\ge1/2Pr[A selects]≥1/2 and nothing else: the probability of selecting can be tiny while E[OPT]\mathbf E[\mathrm{OPT}]E[OPT] is still large, because the optimum may be concentrated on a few orders with a large product. In that case the bound must come from comparing the algorithm with the optimum pair by pair: every time–element pair (i,j)(i,j)(i,j) with d(i)v(j)≥Z/2d(i)v(j)\ge Z/2d(i)v(j)≥Z/2 must be realized by A\mathcal AA on a positive fraction of the orders.

Two points need care in a formal proof:

  • Eq. (4.2) rewrites LLL as a sum over pairs weighted by the conditional probability that d(i)v(j)d(i)v(j)d(i)v(j) is the highest product. It relies on a consistent tie-breaking rule, which the paper leaves implicit.
  • Claim 4.8 is a counting argument on SnS_nSn​. A map from the rejecting orders into GijG_{ij}Gij​ swaps element jjj into position iii, and must be shown to be at most nnn-to-111 and to land in GijG_{ij}Gij​.

Neither (4.2) nor the map appears in the statements, so solvers may replace either with any argument they like.

Formalization scope

  • Types. Times and elements are Fin n; the paper's time ttt is the index t−1t-1t−1. An order is π : Equiv.Perm (Fin n), read as time ↦ element, as on p. 3. The instance [NeZero n] encodes n≥1n\ge1n≥1, so the maximum over times is a genuine maximum (Finset.sup').
  • Expectations. Expectations over the uniform order are finite averages 1n!∑π\frac1{n!}\sum_\pin!1​∑π​. No measure theory is used.
  • Values and constants. Values, discounts and ZZZ are real numbers, and the hypotheses d≥0d\ge0d≥0, v≥0v\ge0v≥0 are explicit. The constant 1/41/41/4 is the paper's. The bound is stated multiplicatively, Z/4≤E[A]Z/4\le\mathbf E[\mathcal A]Z/4≤E[A], never as a ratio.
  • Thresholds and ties. Every threshold is non-strict (≥Z/2\ge Z/2≥Z/2), exactly as on pp. 7–8. A\mathcal AA selects the first qualifying time, so it needs no tie-breaking. The tie-breaking remark at Eq. (4.2) concerns only the paper's intermediate identity (4.2), which is not a milestone.
  • Claim 4.8. Both inequalities are stated with cleared denominators. The second carries the proof's case hypothesis 2∣Sacc∣≤n!2|S_{acc}|\le n!2∣Sacc​∣≤n!, which the paper uses in the same place ("at most half the permutations are in SaccS_{acc}Sacc​").
  • What is not this theorem. A\mathcal AA is the online threshold rule with threshold Z/2Z/2Z/2 applied to π\piπ as it unfolds. An algorithm that inspects the whole order, or that chooses its threshold after seeing the values, would make the bound trivial and is not this theorem.
  • Contributions welcome. Proofs of each milestone, including the counting argument of Claim 4.8. Lemmas on averages over Equiv.Perm (Fin n) and on first-hitting times are reusable beyond this mission.

Selected references

  • M. Babaioff, M. Dinitz, A. Gupta, N. Immorlica, K. Talwar, Secretary Problems: Weights and Discounts, Proceedings of the 20th ACM-SIAM Symposium on Discrete Algorithms (SODA), 2009.
  • E. B. Dynkin, Optimal choice of the stopping moment of a Markov process, Doklady Akademii Nauk SSSR 150:238–240, 1963.
  • W. T. Rasmussen, S. R. Pliska, Choosing the maximum from a sequence with a discount function, Applied Mathematics and Optimization 2(3):279–289, 1975/76.
  • M. Mahdian, P. McAfee, D. Pennock, The secretary problem with durable employment, personal communication, 2008 (cited as [MMP08]).
  • M. Babaioff, N. Immorlica, R. Kleinberg, Matroids, secretary problems, and online mechanisms, SODA 2007, pp. 434–443.
6 thms2 active usersReviewed
🏆Completed
Operations ResearchProbability·Captain: mikedeng1

Secretary Problems: Weights and Discounts 2: An Ω(log n / log log n) Lower Bound on the Competitive Ratio of the Discounted Secretary ProblemResearch Paper

Motivation

In the classical secretary problem a decision maker sees nnn candidates in uniformly random order, learns each candidate's value on arrival, and must accept or reject it on the spot; the goal is to pick a valuable one. A simple sample-then-select rule picks the best candidate with probability at least 1/e1/e1/e, so the problem is constant-competitive. The secretary problem is also a model of online mechanism design: a rule that accepts the first agent above a threshold computed from earlier agents is a truthful posted-price mechanism (as the paper notes in §1).

Babaioff, Dinitz, Gupta, Immorlica and Talwar (SODA 2009; authors' version) study the discounted secretary problem, where accepting at time ttt is worth d(t) v(e)d(t)\,v(e)d(t)v(e) for a known discount function ddd. Discounts model settings where a sale is worth more at some times than at others. The case d(t)=βtd(t)=\beta^td(t)=βt had been studied before (Rasmussen and Pliska 1976); the paper asks what happens for arbitrary ddd. Its answer has two sides: an O(log⁡n)O(\log n)O(logn)-competitive algorithm, and the result of this mission, a lower bound showing that no online algorithm is better than Ω(log⁡n/log⁡log⁡n)\Omega(\log n/\log\log n)Ω(logn/loglogn)-competitive. So, unlike the classical problem, the discounted problem with a general discount is not constant-competitive.

Setting

There are nnn elements e∈{0,…,n−1}e\in\{0,\dots,n-1\}e∈{0,…,n−1} with values v(e)≥0v(e)\ge 0v(e)≥0, and a discount function ddd on the times. The elements arrive in a uniformly random order π\piπ: element π(t)\pi(t)π(t) arrives at time ttt. A randomized online stopping rule AAA specifies, for each time ttt and each sequence of values seen so far h=(v(π(0)),…,v(π(t)))h=(v(\pi(0)),\dots,v(\pi(t)))h=(v(π(0)),…,v(π(t))), a probability pt(h)∈[0,1]p_t(h)\in[0,1]pt​(h)∈[0,1] of stopping at ttt if it has not stopped yet. Stopping at ttt selects π(t)\pi(t)π(t) and earns d(t) v(π(t))d(t)\,v(\pi(t))d(t)v(π(t)); the rule selects at most one element and may select none. The rule knows nnn and ddd, but it sees only values, only as they arrive, and it is not told which instance it is facing.

The expected value of AAA is

E[A]=Eπ[∑td(t) v(π(t)) pt(ht)∏s<t(1−ps(hs))],\mathbb E[A]=\mathbb E_\pi\Bigl[\sum_t d(t)\,v(\pi(t))\,p_t(h_t)\prod_{s<t}\bigl(1-p_s(h_s)\bigr)\Bigr],E[A]=Eπ​[t∑​d(t)v(π(t))pt​(ht​)s<t∏​(1−ps​(hs​))],

and the benchmark is the expected offline optimum

E[OPT]=Eπ[max⁡td(t) v(π(t))],\mathbb E[\mathrm{OPT}]=\mathbb E_\pi\Bigl[\max_t d(t)\,v(\pi(t))\Bigr],E[OPT]=Eπ​[tmax​d(t)v(π(t))],

which is itself a random variable averaged over the order. AAA is α\alphaα-competitive on an instance when E[OPT]≤α E[A]\mathbb E[\mathrm{OPT}]\le\alpha\,\mathbb E[A]E[OPT]≤αE[A].

The hard family (§4.1.1 of the paper): fix an integer c≥1c\ge1c≥1 and put L=cL=cL=c, n=L4cn=L^{4c}n=L4c, nt=L2tn_t=L^{2t}nt​=L2t for t≤2ct\le 2ct≤2c, and K=n2K=n^2K=n2. The step discount is d(j)=L−1d(j)=L^{-1}d(j)=L−1 on the times 1≤j≤n11\le j\le n_11≤j≤n1​ and d(j)=L−td(j)=L^{-t}d(j)=L−t on nt−1<j≤ntn_{t-1}<j\le n_tnt−1​<j≤nt​. The instance I1\mathcal I_1I1​ has n/n1n/n_1n/n1​ elements of value KKK and the rest 000; It+1\mathcal I_{t+1}It+1​ is obtained from It\mathcal I_tIt​ by raising n/nt+1n/n_{t+1}n/nt+1​ of its values KtK^tKt to Kt+1K^{t+1}Kt+1, so It\mathcal I_tIt​ has n/ntn/n_tn/nt​ elements of value KtK^tKt.

Formalization targets

Goal: Theorem 4.3 in the form its proof establishes

For every integer c≥1c\ge1c≥1 and every randomized online stopping rule AAA for horizon n=c4cn=c^{4c}n=c4c and the step discount,

∃ t∈{1,…,2c}:c⋅E[A(It)] < 10⋅E[OPT(It)].\exists\,t\in\{1,\dots,2c\}:\qquad c\cdot\mathbb E[A(\mathcal I_t)]\ <\ 10\cdot\mathbb E[\mathrm{OPT}(\mathcal I_t)].∃t∈{1,…,2c}:c⋅E[A(It​)] < 10⋅E[OPT(It​)].

That is, no online rule is c/10c/10c/10-competitive on all of I1,…,I2c\mathcal I_1,\dots,\mathcal I_{2c}I1​,…,I2c​.

Milestones

  1. Lemma 4.1: E[OPT(It)]≥(1−1/e)KtL−t\mathbb E[\mathrm{OPT}(\mathcal I_t)]\ge(1-1/e)K^tL^{-t}E[OPT(It​)]≥(1−1/e)KtL−t for 1≤t≤2c1\le t\le 2c1≤t≤2c.
  2. Coupling step of Lemma 4.2's proof: for every rule and 1≤t<2c1\le t<2c1≤t<2c, the probability of stopping among the first ntn_tnt​ arrivals drops by at most 1/L21/L^21/L2 from It\mathcal I_tIt​ to It+1\mathcal I_{t+1}It+1​.
  3. Lemma 4.2: a rule that is c/10c/10c/10-competitive on I1,…,I2c\mathcal I_1,\dots,\mathcal I_{2c}I1​,…,I2c​ stops among the first ntn_tnt​ arrivals of It\mathcal I_tIt​ with probability at least t/ct/ct/c.
  4. Theorem 4.3, asymptotic form: for c≥2c\ge2c≥2 and n=c4cn=c^{4c}n=c4c, every rule has some It\mathcal I_tIt​ with
140⋅log⁡nlog⁡log⁡n⋅E[A(It)]<E[OPT(It)].\frac1{40}\cdot\frac{\log n}{\log\log n}\cdot\mathbb E[A(\mathcal I_t)]<\mathbb E[\mathrm{OPT}(\mathcal I_t)].401​⋅loglognlogn​⋅E[A(It​)]<E[OPT(It​)].

Significance

The result separates the discounted secretary problem from its classical and weighted relatives, which admit constant-competitive algorithms (the paper's Theorem 3.4 and the eee-competitive classical rule). Together with the paper's O(log⁡n)O(\log n)O(logn) upper bound (Theorem 4.4) it pins the competitive ratio for general discounts between log⁡n/log⁡log⁡n\log n/\log\log nlogn/loglogn and log⁡n\log nlogn up to constants, and it motivates the paper's known-OPT\mathrm{OPT}OPT model (§4.2), where an estimate of E[OPT]\mathbb E[\mathrm{OPT}]E[OPT] restores a constant ratio. The construction is a template for lower bounds against randomized online algorithms in random-order models: geometrically nested instances that a rule cannot tell apart early, played against a discount that punishes waiting.

The theorem is proved in the paper, in about a page. To our knowledge no part of it has a machine-checked proof. This mission produces the formal model of randomized online stopping rules in the random-order discounted setting, a reusable object for the paper's other discounted results (the O(log⁡n)O(\log n)O(logn) upper bound, and the 2\sqrt22​ lower bound with known values of Theorem 4.6), and a checked version of the lower bound with explicit constants.

Difficulty

The obvious attempt is to fix one instance and show that every rule loses on it. That fails: for any single instance there is a rule tuned to it (a rule that waits exactly as long as that instance warrants). The lower bound has to play the 2c2c2c instances against each other. A rule that does well on It\mathcal I_tIt​ must commit early, within the first ntn_tnt​ steps, yet the rule cannot distinguish It\mathcal I_tIt​ from It+1\mathcal I_{t+1}It+1​ during those steps except with probability L−2L^{-2}L−2. Making "cannot distinguish" precise is the central step: it needs a coupling of the two runs over the same random order and the same internal randomness, which works only because the rule's decision at time ttt depends on the values observed so far and nothing else. The accounting then has to show that the rule's early earnings on It+1\mathcal I_{t+1}It+1​ and its late earnings are both small compared with E[OPT(It+1)]\mathbb E[\mathrm{OPT}(\mathcal I_{t+1})]E[OPT(It+1​)], which uses L≥2L\ge 2L≥2 and that K=n2K=n^2K=n2 dwarfs L2cL^{2c}L2c.

Formalization scope

  • Elements and times are Fin n, 0-based: index jjj is the paper's time j+1j+1j+1, so the paper's block (nt−1,nt](n_{t-1},n_t](nt−1​,nt​] is the index range [nt−1,nt)[n_{t-1},n_t)[nt−1​,nt​). The random order is π : Equiv.Perm (Fin n) read as time ↦\mapsto↦ element, and every expectation over it is the finite average 1n!∑π\frac1{n!}\sum_\pin!1​∑π​. Values and discounts are real.
  • Algorithms are the structure StoppingRule n: stopping probabilities pt(h)∈[0,1]p_t(h)\in[0,1]pt​(h)∈[0,1] indexed by time and the arrival-ordered value sequence, with the non-anticipation condition that pt(h)p_t(h)pt​(h) depends only on h0,…,hth_0,\dots,h_th0​,…,ht​. The theorem quantifies over all such rules, so it covers deterministic and randomized online algorithms that observe values only. A rule may depend on nnn and ddd but not on the instance index.
  • OPT is Eπ[max⁡td(t)v(π(t))]\mathbb E_\pi[\max_t d(t)v(\pi(t))]Eπ​[maxt​d(t)v(π(t))] (a supremum over the finite type Fin n), and competitiveness is multiplicative, E[OPT]≤α E[A]\mathbb E[\mathrm{OPT}]\le\alpha\,\mathbb E[A]E[OPT]≤αE[A], never a quotient.
  • Constants. The goal uses the paper's constant 101010 (from "if AAA is c/10c/10c/10-competitive"); the asymptotic form uses 1/401/401/40, from log⁡n/log⁡log⁡n≤4c\log n/\log\log n\le 4clogn/loglogn≤4c for c≥2c\ge2c≥2, with the natural logarithm. K=n2K=n^2K=n2, the value the paper suggests.
  • The construction (nnn, ntn_tnt​, ddd, KKK, It\mathcal I_tIt​) is fixed by explicit formulas in the definition file. A solver cannot choose the discount or the instances, and the goal is not stated for a restricted class of algorithms; a formalization that let the rule see the instance index or future values, or quantified only over threshold rules, would be a different and trivial or weaker theorem. For c<10c<10c<10 the goal is immediate, since E[A]≤E[OPT]\mathbb E[A]\le\mathbb E[\mathrm{OPT}]E[A]≤E[OPT] and E[OPT(It)]>0\mathbb E[\mathrm{OPT}(\mathcal I_t)]>0E[OPT(It​)]>0; the content lies in c≥10c\ge10c≥10. The bound is stated only for the horizons n=c4cn=c^{4c}n=c4c the paper constructs.
  • Needed infrastructure: counting arguments over permutations of Fin n (the probability that a set of mmm elements misses the first kkk positions), the coupling of two value sequences that agree on a prefix, and elementary estimates on geometric sums. The rule model and the permutation-counting lemmas are reusable for the paper's other discounted results. Contributions of these supporting lemmas, as well as proofs of the milestones, are welcome.

Selected references

  • M. Babaioff, M. Dinitz, A. Gupta, N. Immorlica, K. Talwar, Secretary Problems: Weights and Discounts, Proceedings of the 20th ACM-SIAM Symposium on Discrete Algorithms (SODA), 2009. https://doi.org/10.1137/1.9781611973068.135 (authors' full version, the one cited here: https://www.cs.jhu.edu/~mdinitz/papers/secretary.pdf)
  • E. B. Dynkin, Optimal choice of the stopping moment of a Markov process, Doklady Akademii Nauk SSSR, 1963.
  • W. T. Rasmussen, S. R. Pliska, Choosing the maximum from a sequence with a discount function, Applied Mathematics and Optimization 2(3), 1976.
  • T. S. Ferguson, Who solved the secretary problem?, Statistical Science 4(3), 1989. https://doi.org/10.1214/ss/1177012493
7 thms2 active usersReviewed
🏆Completed
Linear OptimizationOperations ResearchOptimization·Captain: mikedeng1

Understanding and Using Linear Programming III: The Simplex Method with Bland's Rule Never CyclesTextbook

Motivation

The simplex method, introduced by G. B. Dantzig in 1947, is the standard algorithm for linear programming and remains the core of commercial solvers. It moves from one basic feasible solution to another by pivot steps, and at each step a pivot rule chooses which variable enters and which leaves the basis. For several natural rules, including Dantzig's original largest-coefficient rule, the method can cycle: on a degenerate linear program it can return to a basis it has already visited and repeat forever without improving the objective. Hoffman (1953) and Beale (1955) gave cycling examples.

R. G. Bland (New finite pivoting rules for the simplex method, Mathematics of Operations Research 2(2), 1977) showed that a simple combinatorial rule, choosing the smallest eligible index for both the entering and the leaving variable, never cycles. This makes the simplex method a finite algorithm on every linear program in equational form, and it gives an algorithmic proof of the duality theorem. Chapter 5 of J. Matoušek and B. Gärtner, Understanding and Using Linear Programming (Springer, 2007, DOI 10.1007/978-3-540-30717-4), develops the general theory of simplex tableaus and proves Bland's theorem as Theorem 5.8.1. This mission is the third of a series formalizing that book.

Setting

A linear program in equational form is

maximize cTxsubject toAx=b, x≥0,\text{maximize } c^{T}x \quad\text{subject to}\quad Ax=b,\ x\ge 0,maximize cTxsubject toAx=b, x≥0,

with AAA a real m×nm\times nm×n matrix, b∈Rmb\in\mathbb{R}^mb∈Rm, c∈Rnc\in\mathbb{R}^nc∈Rn. Following §4.2 of the book, AAA has n≥mn\ge mn≥m columns and rank mmm. For an mmm-element set B={k1<⋯<km}⊆{1,…,n}B=\{k_1<\dots<k_m\}\subseteq\{1,\dots,n\}B={k1​<⋯<km​}⊆{1,…,n} let N={ℓ1<⋯<ℓn−m}N=\{\ell_1<\dots<\ell_{n-m}\}N={ℓ1​<⋯<ℓn−m​} be its complement, and ABA_BAB​, ANA_NAN​ the matrices of the columns of AAA indexed by BBB and NNN. BBB is a feasible basis if ABA_BAB​ is nonsingular and AB−1b≥0A_B^{-1}b\ge0AB−1​b≥0; its basic feasible solution is the unique xxx with Ax=bAx=bAx=b and xj=0x_j=0xj​=0 for j∉Bj\notin Bj∈/B.

A simplex tableau T(B)T(B)T(B) is a system

xB=p+Q xN,z=z0+rTxNx_B=p+Q\,x_N,\qquad z=z_0+r^{T}x_NxB​=p+QxN​,z=z0​+rTxN​

in the variables x1,…,xn,zx_1,\dots,x_n,zx1​,…,xn​,z with the same solutions as Ax=bAx=bAx=b, z=cTxz=c^Txz=cTx. A nonbasic variable xvx_vxv​, v=ℓβv=\ell_\betav=ℓβ​, may enter if rβ>0r_\beta>0rβ​>0; a basic variable xux_uxu​, u=kαu=k_\alphau=kα​, may then leave if

qαβ<0and−pαqαβ=min⁡{−piqiβ:qiβ<0}.(5.3)q_{\alpha\beta}<0\quad\text{and}\quad-\frac{p_\alpha}{q_{\alpha\beta}}=\min\Bigl\{-\frac{p_i}{q_{i\beta}}: q_{i\beta}<0\Bigr\}.\tag{5.3}qαβ​<0and−qαβ​pα​​=min{−qiβ​pi​​:qiβ​<0}.(5.3)

The pivot step replaces BBB by B′=(B∖{u})∪{v}B'=(B\setminus\{u\})\cup\{v\}B′=(B∖{u})∪{v}. Bland's rule takes the entering variable of smallest index among those with rβ>0r_\beta>0rβ​>0, and the leaving variable of smallest index among those satisfying (5.3).

Formalization targets

Goal: Theorem 5.8.1 (p. 73)

There is no infinite sequence of bases

B0→B1→B2→⋯B_0\to B_1\to B_2\to\cdotsB0​→B1​→B2​→⋯

in which each Bt+1B_{t+1}Bt+1​ is obtained from the feasible basis BtB_tBt​ by a pivot step obeying Bland's rule. Since there are finitely many bases and a Bland step is determined by its starting basis, this is the book's "always finite; i.e., cycling is impossible".

Milestones

  1. Lemma 5.5.1 (p. 66): a feasible basis has exactly one simplex tableau, with Q=−AB−1ANQ=-A_B^{-1}A_NQ=−AB−1​AN​, p=AB−1bp=A_B^{-1}bp=AB−1​b, z0=cBTAB−1bz_0=c_B^TA_B^{-1}bz0​=cBT​AB−1​b, r=cN−(cBTAB−1AN)Tr=c_N-(c_B^TA_B^{-1}A_N)^Tr=cN​−(cBT​AB−1​AN​)T.
  2. Optimality criterion (§5.6, p. 67): if r≤0r\le0r≤0, the basic feasible solution of BBB is optimal.
  3. Lemma 5.6.1 (p. 68): a pivot step leads to a feasible basis; if no leaving variable exists, the program is unbounded along an explicit ray.
  4. Claim in the proof of Theorem 5.8.1 (p. 73): for any pivot rule, all bases of a cycle have the same basic feasible solution, and every variable that enters during the cycle is 000 in it.

Significance

With the optimality criterion and Lemma 5.6.1, Theorem 5.8.1 turns the simplex method into an algorithm: started from any feasible basis, it stops after finitely many pivot steps at an optimal basic feasible solution or with a ray certifying unboundedness. Combined with the auxiliary program of §5.6 for finding a first feasible basis, this yields a constructive proof that every feasible, bounded linear program has an optimal basic feasible solution, and the book remarks that the duality theorem follows easily. Bland's rule is also the model for later combinatorial anticycling rules in oriented matroid programming.

The theorem is classical and fully proved in the book. What the mission adds is a machine-checked version stated in the book's own tableau notation. On Prove2Me the simplex method is formalized in the Bertsimas–Tsitsiklis series (Introduction to Linear Optimization IV), for minimization with reduced costs, with termination proved under nondegeneracy and for the lexicographic rule; Bland's rule is not formalized there.

Difficulty

The obvious termination argument is that the objective value strictly increases at each step, so no basis repeats. That argument fails exactly at degenerate pivot steps, where the minimum in (5.3) is 000: the basis changes, the basic feasible solution and the objective value do not. Along a degenerate stretch the objective gives no progress measure, and for general pivot rules the method does cycle there. Any proof must therefore use the specific tie-breaking of Bland's rule, which is a statement about indices, not about values, and relate the tableaus of two different bases in the cycle to each other. Counting bases or tracking the objective value alone does not suffice.

Formalization scope

Vectors are Fin n → ℝ and the book's indices 1,…,n1,\dots,n1,…,n become 0,…,n−10,\dots,n-10,…,n−1. Bases are Finset (Fin n); the sorted enumerations k1<⋯<kmk_1<\dots<k_mk1​<⋯<km​ and ℓ1<⋯<ℓn−m\ell_1<\dots<\ell_{n-m}ℓ1​<⋯<ℓn−m​ are Finset.orderEmbOfFin, tableau rows are indexed by Fin m and nonbasic columns by Fin (n - m). Nonsingularity of ABA_BAB​ is IsUnit A_B.det, so the Mathlib inverse is the true inverse wherever it appears. The tableau parameters used by the pivot rules are the explicit formulas of Lemma 5.5.1; the tableau itself is also defined as in the book (same solution set) so that Lemma 5.5.1 is a genuine statement. "Smallest index" compares variable indices, not row positions. Optimality and unboundedness are stated against feasible points, not through a supremum. Every theorem assumes n≥mn\ge mn≥m and rank⁡A=m\operatorname{rank}A=mrankA=m, the standing assumption of §4.2.

A formalization in which any improving variable may enter proves a different, false statement, since cycling examples exist for such rules; the step relation here fixes both choices by Bland's rule. The step relation is not empty: it holds whenever the current tableau has a positive last-row coefficient and a negative entry in the entering column, so the goal is not vacuous.

The development needs basic linear algebra over Matrix, the uniqueness of basic feasible solutions, and bookkeeping for sorted index enumerations. Lemma 5.5.1 and Lemma 5.6.1 are reusable for any later formalization of the simplex method in this notation. Contributions are welcome for each milestone, for the cycle-form corollary, and for a sorry-free proof of the goal.

Selected references

  • J. Matoušek, B. Gärtner, Understanding and Using Linear Programming, Springer Universitext, 2007, Chapter 5. https://doi.org/10.1007/978-3-540-30717-4
  • R. G. Bland, New finite pivoting rules for the simplex method, Mathematics of Operations Research 2(2):103–107, 1977. https://doi.org/10.1287/moor.2.2.103
  • E. M. L. Beale, Cycling in the dual simplex algorithm, Naval Research Logistics Quarterly 2(4):269–275, 1955. https://doi.org/10.1002/nav.3800020407
  • D. Bertsimas, J. N. Tsitsiklis, Introduction to Linear Optimization, Athena Scientific, 1997, Chapter 3.
7 thms2 active usersReviewed
🏆Completed
Captain: wurtle

WordRAM and Turing machines: two-way halting equivalenceResearch Paper

We formalize the equivalence between Turing machines and the WordRAM model of computation. Turing machines are the standard model for studying computability, but algorithms are rarely described in terms of tape operations. WordRAM is much closer to assembly, with memory accesses, arithmetic instructions, branches, and loops. The goal is to connect this familiar way of expressing algorithms to the foundations of computability. This will also bring us closer to one day formalizing Fine Grained Complexity.

The formal target is halting equivalence through simulations in both directions, with the stated memory bounds and a family of word widths, each fixed during a run.

References:

  • Stephen A. Cook and Robert A. Reckhow. Time Bounded Random Access Machines. Journal of Computer and System Sciences 7(4), 354–375, 1973.
  • Torben Hagerup. Sorting and Searching on the Word RAM. STACS 1998, 366–398.
9 thms2 active usersReviewed
🏆Completed
CombinatoricsLinear OptimizationOperations Research+1·Captain: mikedeng1

Understanding and Using Linear Programming VII: LP Rounding Schedules Unrelated Machines Within Twice the Optimal MakespanTextbook

Motivation

Scheduling indivisible jobs on parallel machines to finish all of them as early as possible is a basic problem in operations research and in the theory of algorithms. In the unrelated machines model each job may take a different time on each machine, with no relation between the rows of the time table, as when machines of different types (black-and-white, duplex, colour copiers in the book's example) handle jobs of different kinds. Minimizing the makespan in this model is NP-hard, so the question is how close to the optimum a polynomial-time algorithm can get.

  • 1990. Lenstra, Shmoys and Tardos (Math. Programming 46, 259–271) give a polynomial-time algorithm that rounds a basic optimal solution of a linear programming relaxation and returns a schedule of makespan at most 2 topt2\,t_{\mathrm{opt}}2topt​. The same paper shows that approximating the optimum makespan within a factor less than 3/23/23/2 is NP-hard.
  • 2007. Matoušek and Gärtner present the algorithm in §8.3 of Understanding and Using Linear Programming in a simplified, somewhat less efficient form: minimize t∗(T)+Tt^*(T) + Tt∗(T)+T over the thresholds TTT rather than binary-searching for the smallest TTT with t∗(T)≤Tt^*(T) \le Tt∗(T)≤T. This mission follows the book's presentation.

The gap between 3/23/23/2 and 222 for the general unrelated-machines problem has remained open since 1990; it is the standard example of LP rounding driven by the sparsity of basic solutions.

Setting

There are mmm machines MMM and nnn jobs JJJ; dij>0d_{ij} > 0dij​>0 is the running time of job jjj on machine iii. A schedule is a map σ:J→M\sigma : J \to Mσ:J→M assigning each job to one machine. The load of machine iii is ∑j:σ(j)=idij\sum_{j:\sigma(j)=i} d_{ij}∑j:σ(j)=i​dij​, the makespan of σ\sigmaσ is the largest load, and toptt_{\mathrm{opt}}topt​ is the makespan of an optimal schedule, one whose makespan is at most that of every schedule.

For a real threshold TTT, the linear program LPR(T)\mathrm{LPR}(T)LPR(T) in the variables ttt and xijx_{ij}xij​ is

minimize  tsubject to  ∑i∈Mxij=1  (j∈J),∑j∈Jdijxij≤t  (i∈M),xij≥0,xij=0  whenever dij>T.\begin{aligned} \text{minimize } \ & t \\ \text{subject to } \ & \textstyle\sum_{i \in M} x_{ij} = 1 \ \ (j \in J), \qquad \textstyle\sum_{j \in J} d_{ij} x_{ij} \le t \ \ (i \in M),\\ & x_{ij} \ge 0, \qquad x_{ij} = 0 \ \text{ whenever } d_{ij} > T . \end{aligned}minimize  subject to  ​t∑i∈M​xij​=1  (j∈J),∑j∈J​dij​xij​≤t  (i∈M),xij​≥0,xij​=0  whenever dij​>T.​

Its optimal value is t∗(T)t^*(T)t∗(T), with t∗(T)=∞t^*(T) = \inftyt∗(T)=∞ when LPR(T)\mathrm{LPR}(T)LPR(T) is infeasible. The constraint matrix AAA has one row per machine, one per job and one per pair with dij>Td_{ij} > Tdij​>T; the column of xijx_{ij}xij​ carries dijd_{ij}dij​ in the row of machine iii, 111 in the row of job jjj, and 111 in the row of the constraint xij=0x_{ij} = 0xij​=0 if present. Assumption 8.3.1 on a solution x∗x^*x∗ is that the columns of AAA belonging to its nonzero variables are linearly independent; basic feasible solutions satisfy it. The support graph of x∗x^*x∗ is the bipartite graph G=(M∪J,E)G = (M \cup J, E)G=(M∪J,E) with E={{i,j}:xij∗>0}E = \{\{i,j\} : x^*_{ij} > 0\}E={{i,j}:xij∗​>0}.

Formalization targets

Goal: Theorem 8.3.4

Let T∗T^*T∗ minimize t∗(T)+Tt^*(T) + Tt∗(T)+T over all real TTT and let (t∗,x∗)(t^*, x^*)(t∗,x∗) be an optimal solution of LPR(T∗)\mathrm{LPR}(T^*)LPR(T∗) satisfying Assumption 8.3.1. Then there is a schedule σ\sigmaσ with xσ(j)j∗>0x^*_{\sigma(j) j} > 0xσ(j)j∗​>0 for every job jjj and

max⁡i∈M∑j:σ(j)=idij  ≤  2 topt.\max_{i \in M} \sum_{j : \sigma(j) = i} d_{ij} \;\le\; 2\, t_{\mathrm{opt}} .i∈Mmax​j:σ(j)=i∑​dij​≤2topt​.

Milestones

  1. Lemma 8.3.2. Every subgraph of the support graph GGG has at most as many edges as vertices: ∣E′∣≤∣M′∣+∣J′∣|E'| \le |M'| + |J'|∣E′∣≤∣M′∣+∣J′∣.
  2. Lemma 8.3.3. For T≥0T \ge 0T≥0 and an optimal solution (t∗,x∗)(t^*, x^*)(t∗,x∗) of LPR(T)\mathrm{LPR}(T)LPR(T) satisfying Assumption 8.3.1, some schedule along the edges of GGG has makespan at most t∗+Tt^* + Tt∗+T.
  3. Proof of Theorem 8.3.4, first step. LPR(topt)\mathrm{LPR}(t_{\mathrm{opt}})LPR(topt​) is feasible and t∗(topt)≤toptt^*(t_{\mathrm{opt}}) \le t_{\mathrm{opt}}t∗(topt​)≤topt​.
  4. Proof of Theorem 8.3.4, second step. t∗(T∗)+T∗≤2 toptt^*(T^*) + T^* \le 2\,t_{\mathrm{opt}}t∗(T∗)+T∗≤2topt​.

Significance

The theorem gives a polynomial-time 2-approximation for an NP-hard problem, and its proof isolates a reusable principle: a basic solution of an assignment-type LP has a support graph in which every subgraph has at most as many edges as vertices (a pseudoforest), so all but a matching's worth of the fractional assignment is already integral. The same sparsity argument underlies rounding results for the generalized assignment problem and for many later scheduling and allocation relaxations.

The result has been proved since 1990 and is textbook material. It is not formalized on Prove2Me or, to the maintainers' knowledge, in Mathlib. This mission produces a machine-checked version of the rounding theorem together with the counting lemma on basic solutions, the relaxation inequality t∗(topt)≤toptt^*(t_{\mathrm{opt}}) \le t_{\mathrm{opt}}t∗(topt​)≤topt​, and the bound on the chosen threshold, each stated on shared definitions of the scheduling LP.

Difficulty

The obvious approach, rounding every job to the machine carrying the largest fraction of it, can overload a machine by many jobs at once and gives no constant factor. The bound t∗+Tt^* + Tt∗+T needs two facts that are not visible from the LP value alone: that the support of a basic solution is sparse in the precise sense of Lemma 8.3.2, which has to be read off the linear independence of columns of the constraint matrix after deleting rows; and that the jobs left fractional can be matched injectively to machines, which requires a Hall-type condition derived from that sparsity. Relating linear independence of real column vectors to an edge count in a bipartite graph, and then producing a matching, is where the formal work lies.

A second subtlety is the threshold TTT: the bound t∗+T≤2toptt^* + T \le 2 t_{\mathrm{opt}}t∗+T≤2topt​ holds only because T∗T^*T∗ is chosen by minimizing over thresholds, and the relaxation at T=toptT = t_{\mathrm{opt}}T=topt​ must be compared with the one at T∗T^*T∗ through optimal solutions of different linear programs.

Formalization scope

Machines are Fin m, jobs are Fin n (0-based; the book's machines 1,…,m1,\dots,m1,…,m and jobs m+1,…,m+nm+1,\dots,m+nm+1,…,m+n are disjoint index sets), running times form d : Matrix (Fin m) (Fin n) ℝ, and the standing hypothesis dij>0d_{ij} > 0dij​>0 of §8.3 appears in every theorem. A schedule is a function Fin n → Fin m; the makespan is the supremum of the loads over the finite type Fin m, which is the maximum for m≥1m \ge 1m≥1. The optimum toptt_{\mathrm{opt}}topt​ is the makespan of a schedule assumed optimal, never an infimum.

Optimal values of LPR(T)\mathrm{LPR}(T)LPR(T) are never written as sInf: statements quantify over optimal solutions, i.e. feasible (t,x)(t, x)(t,x) with t≤t′t \le t't≤t′ for every feasible (t′,x′)(t', x')(t′,x′). The book's convention t∗(T)=∞t^*(T) = \inftyt∗(T)=∞ for infeasible LPR(T)\mathrm{LPR}(T)LPR(T) is encoded by letting thresholds without an optimal solution impose no condition in the minimality hypothesis on T∗T^*T∗, which reads t∗+T∗≤t+Tt^* + T^* \le t + Tt∗+T∗≤t+T for every real TTT and every optimal solution (t,x)(t, x)(t,x) of LPR(T)\mathrm{LPR}(T)LPR(T). The constraint matrix used in Assumption 8.3.1 has rows indexed by Fin m ⊕ Fin n ⊕ {(i, j) // T < d i j} and excludes the column of ttt, as on p. 151.

"Efficiently construct" in Lemma 8.3.3 and "computes" in Theorem 8.3.4 are formalized by the property of the constructed schedule, not by its running time: every job goes to a machine iii with xij∗>0x^*_{ij} > 0xij∗​>0. This constraint is what rules out the trivializing formalization — "some schedule has makespan at most 2topt2 t_{\mathrm{opt}}2topt​" is true of the optimal schedule itself and says nothing about the rounding.

A complete development needs: finite linear algebra (a linearly independent family of vectors supported on kkk coordinates has at most kkk members), Hall's marriage theorem (available in Mathlib as Finset.all_card_le_biUnion_card_iff_exists_injective), and the existence of an optimal solution of a feasible, bounded linear program (used to apply the minimality of T∗T^*T∗ at T=toptT = t_{\mathrm{opt}}T=topt​). The counting lemma for basic solutions and the definitions of LPR(T)\mathrm{LPR}(T)LPR(T) are reusable for other assignment relaxations. Proofs of any milestone, and alternative proofs of Lemma 8.3.3 by the direct pseudoforest argument of p. 153–154, are welcome.

Selected references

  • J. Matoušek, B. Gärtner, Understanding and Using Linear Programming, Springer Universitext, 2007, §8.3, pp. 148–156. https://doi.org/10.1007/978-3-540-30717-4
  • J. K. Lenstra, D. B. Shmoys, É. Tardos, Approximation algorithms for scheduling unrelated parallel machines, Mathematical Programming 46 (1990), 259–271. https://doi.org/10.1007/BF01585745
7 thms2 active usersReviewed
🏆Completed
Linear OptimizationNumerical Analysis·Captain: Lucas

Extended Smale's 9th Problem I: no algorithm computes K digits of LP minimisersResearch Paper

Motivation

Linear programming is usually described as "solvable in polynomial time", but that statement is about rational inputs given exactly. In Smale's list of problems for the 21st century (Smale 1998), Problem 9 asks for a polynomial-time algorithm over the reals deciding the feasibility of Ax≥yAx \ge yAx≥y, and Smale explicitly calls for "models which process approximate inputs and which permit round-off computations". Real data such as 2\sqrt 22​, entries of a discrete cosine transform, or even 1/31/31/3 in floating point can only be accessed approximately.

Bastounis, Hansen and Vlačić pose the extended Smale's 9th problem: in a model where the algorithm can only query approximations of the input to any requested accuracy, can one compute minimisers of linear programming, basis pursuit and Lasso to KKK correct digits? Their Main Theorem I (Theorem 3.4) shows that the answer depends on KKK in a sharp way: for a suitable class of well-conditioned, bounded inputs, no algorithm at all (not only no efficient one) produces KKK correct digits, while K−1K-1K−1 digits are computable (but not in bounded time) and K−2K-2K−2 digits are computable in polynomial time.

This mission targets the first, impossibility, half of Theorem 3.4(i) for linear programming.

Setting

Linear program. For A∈Rm×NA \in \mathbb R^{m\times N}A∈Rm×N, y∈Rmy\in\mathbb R^my∈Rm and c=1N=(1,…,1)c = \mathbf 1_N=(1,\dots,1)c=1N​=(1,…,1), the solution set is

Ξ(y,A)=argmin⁡x∈RN ⟨x,c⟩subject toAx=y, x≥0.\Xi(y,A) = \operatorname*{argmin}_{x\in\mathbb R^N}\ \langle x, c\rangle \quad\text{subject to}\quad Ax = y,\ x\ge 0 .Ξ(y,A)=x∈RNargmin​ ⟨x,c⟩subject toAx=y, x≥0.

It is a subset of MN=RNM_N = \mathbb R^NMN​=RN with the ℓp\ell^pℓp norm, p∈[1,∞]p\in[1,\infty]p∈[1,∞]. An input is a pair ι=(y,A)\iota = (y,A)ι=(y,A), and the evaluations of ι\iotaι are its coordinates yiy_iyi​ and entries AijA_{ij}Aij​.

Extended model (Δ1\Delta_1Δ1​-information). Let Dn={k2−n:k∈Z}D_n = \{k2^{-n} : k\in\mathbb Z\}Dn​={k2−n:k∈Z}. An oracle representation of ι\iotaι is a family ι~=(ι~j,n)\tilde\iota = (\tilde\iota_{j,n})ι~=(ι~j,n​), indexed by evaluations jjj and accuracies n=1,2,…n = 1,2,\dotsn=1,2,…, with ι~j,n∈Dn+iDn\tilde\iota_{j,n}\in D_n + iD_nι~j,n​∈Dn​+iDn​ and ∣ι~j,n−fj(ι)∣≤2−n|\tilde\iota_{j,n} - f_j(\iota)|\le 2^{-n}∣ι~j,n​−fj​(ι)∣≤2−n. An algorithm must succeed on every oracle representation of every input.

General algorithm. To make impossibility results independent of the machine model, the paper uses general algorithms (Definition 9.3): a map Γ\GammaΓ from inputs to M∪{NH}M\cup\{\mathrm{NH}\}M∪{NH} (NH\mathrm{NH}NH = no output) together with a nonempty set ΛΓ(ι)\Lambda_\Gamma(\iota)ΛΓ​(ι) of evaluations read on ι\iotaι. This set is finite whenever Γ\GammaΓ halts. The output is determined by the values read, and any input that agrees on those values reads the same set. Turing machines and BSS machines with an oracle are special cases; general algorithms can even solve the halting problem.

Error and breakdown epsilon. The error is dist⁡(Γ(ι),Ξ(ι))=inf⁡ξ∈Ξ(ι)d(Γ(ι),ξ)\operatorname{dist}(\Gamma(\iota),\Xi(\iota)) = \inf_{\xi\in\Xi(\iota)} d(\Gamma(\iota),\xi)dist(Γ(ι),Ξ(ι))=infξ∈Ξ(ι)​d(Γ(ι),ξ), with distance ∞\infty∞ from NH\mathrm{NH}NH. The strong breakdown epsilon εBs\varepsilon_B^sεBs​ is the supremum of all ε≥0\varepsilon\ge 0ε≥0 such that every general algorithm has error >ε>\varepsilon>ε on some input (Definition 9.17).

Formalization targets

Goal: Theorem 3.4(i), deterministic part, for LP

For every integer K≥1K\ge1K≥1, all dimensions 4≤m<N4\le m<N4≤m<N and every p∈[1,∞]p\in[1,\infty]p∈[1,∞] there is a nonempty class Ωm,N\Omega_{m,N}Ωm,N​ of inputs (y,A)(y,A)(y,A) with nonempty solution sets, ∥y∥∞≤2\|y\|_\infty\le 2∥y∥∞​≤2 and ∥A∥max⁡=1\|A\|_{\max}=1∥A∥max​=1, such that

¬ ∃ Γ general algorithm on oracle representations:∀ ι~,  dist⁡ℓp(Γ(ι~), Ξ(ι))≤10−K.\neg\,\exists\,\Gamma\ \text{general algorithm on oracle representations}:\quad \forall\,\tilde\iota,\ \ \operatorname{dist}_{\ell^p}\big(\Gamma(\tilde\iota),\,\Xi(\iota)\big)\le 10^{-K}.¬∃Γ general algorithm on oracle representations:∀ι~,  distℓp​(Γ(ι~),Ξ(ι))≤10−K.

Milestones

  1. Lemma 11.1: the explicit solution sets of the LP inputs (y1e1,A(α,β,m,N))(y_1e_1, A(\alpha,\beta,m,N))(y1​e1​,A(α,β,m,N)).
  2. Proposition 10.5 (ii), deterministic part: two input sequences that converge in evaluation to a common input and whose solutions stay κ\kappaκ apart force εBs≥κ/2\varepsilon_B^s\ge\kappa/2εBs​≥κ/2 for a suitable choice of Δ1\Delta_1Δ1​-information.
  3. §9.6, (i) ⇒ (ii): a lower bound on εBs\varepsilon_B^sεBs​ for one specific Δ1\Delta_1Δ1​-information transfers to the problem with all oracle representations.
  4. Proposition 9.32 (i) (deterministic consequence via Proposition 10.1): εBs>10−K\varepsilon_B^s>10^{-K}εBs​>10−K for LP on a suitable Ωm,N\Omega_{m,N}Ωm,N​.

Significance

The theorem shows that for LP with inexact input, being non-computable in Turing's sense does not rule out a finer complexity theory. The paper builds a "KKK / K−1K-1K−1 / K−2K-2K−2 digits" classification on this. It also explains why established solvers can return wrong answers with a success flag on small, well-conditioned LPs (§4 of the paper), and it bears on computer-assisted proofs that rely on inexact LP, such as the Flyspeck proof of the Kepler conjecture.

The result is proved on paper. As far as the proposer knows, it has not been machine-checked. This mission formalizes the deterministic impossibility part for LP and puts in place reusable infrastructure: general algorithms, breakdown epsilons and Δ1\Delta_1Δ1​-information. That infrastructure is the base for later missions on the randomised parts of Theorem 3.4(i)–(ii), the weak breakdown epsilon (iii), the exit-flag theorem (Theorem 5.1), and basis pursuit and Lasso.

Difficulty

The obvious objection is that LP is in P for rational inputs, so some rounding scheme ought to work. It fails because an algorithm must halt after reading finitely many approximations. Two inputs that agree to that accuracy but have minimisers far apart then receive the same output. Setting this up needs a notion of algorithm strong enough to cover every computational model, a precise Δ1\Delta_1Δ1​-information model in which the adversary controls the approximations, and explicit LP geometry in which an arbitrarily small perturbation of AAA moves the minimiser by a fixed amount.

Formalization scope

  • Inputs are (y,A)∈(Fin m→R)×Matrix(Fin m)(Fin N) R(y,A)\in(\mathrm{Fin}\,m\to\mathbb R)\times\mathrm{Matrix}(\mathrm{Fin}\,m)(\mathrm{Fin}\,N)\,\mathbb R(y,A)∈(Finm→R)×Matrix(Finm)(FinN)R. Evaluations are complex-valued, as in Definition 9.2. Outputs lie in PiLp p (Fin N → ℝ).
  • A general algorithm is a structure with an output run : Ω → Option M (none = NH) and a read set queried, satisfying the axioms (i)–(iii) of Definition 9.3.
  • Errors take values in [0,∞][0,\infty][0,∞] (ℝ≥0∞), and the error of NH is ∞\infty∞. The infimum over an empty solution set is ∞\infty∞. The goal also requires nonempty solution sets, so no junk value enters.
  • Oracle accuracies are indexed by n∈{1,2,… }n\in\{1,2,\dots\}n∈{1,2,…} (ℕ+). An oracle input is stored as a pair (input, oracle family), and algorithms can read only the oracle family.
  • Out of scope: randomised algorithms, the positive statements (iii)–(iv), runtime, and the condition-number bounds Cond(AA∗)≤3.2\mathrm{Cond}(AA^*)\le3.2Cond(AA∗)≤3.2, CFP≤4C_{FP}\le4CFP​≤4, Cond(Ξ)≤179\mathrm{Cond}(\Xi)\le179Cond(Ξ)≤179.

Selected references

  • A. Bastounis, A. C. Hansen, V. Vlačić, The extended Smale's 9th problem — On computational barriers and paradoxes in estimation, regularisation, computer-assisted proofs, and learning, preprint (2021).
  • S. Smale, Mathematical problems for the next century, Math. Intelligencer 20 (1998). https://doi.org/10.1007/BF03025291
8 thms2 active usersReviewed
🏆Completed
Number TheoryProbabilityQuantum Information·Captain: mikedeng1

Polynomial-Time Algorithms for Prime Factorization and Discrete Logarithms on a Quantum Computer 4: The Discrete Logarithm Circuit Gives a Good Output with Probability at Least 1/480Research Paper

Motivation

The discrete logarithm problem modulo a prime asks, given a prime ppp, a generator ggg of the multiplicative group modulo ppp, and a nonzero residue xxx, for the exponent rrr with gr≡x(modp)g^r\equiv x \pmod pgr≡x(modp). Its presumed classical hardness underlies Diffie–Hellman key exchange, ElGamal encryption and the Digital Signature Algorithm. The best classical algorithm known when Shor wrote, Gordon's adaptation of the number field sieve, runs in time exp⁡(O((log⁡p)1/3(log⁡log⁡p)2/3))\exp(O((\log p)^{1/3}(\log\log p)^{2/3}))exp(O((logp)1/3(loglogp)2/3)).

In §6 of Polynomial-Time Algorithms for Prime Factorization and Discrete Logarithms on a Quantum Computer (SIAM J. Comput. 26(5), 1997; doi:10.1137/S0097539795293172, arXiv:quant-ph/9508027), Shor gave a quantum algorithm that uses two modular exponentiations and two quantum Fourier transforms and outputs, with constant probability, a pair from which rrr can be computed. The quantitative core of that analysis is a single number: the circuit produces a "good" output with probability at least 1/4801/4801/480. This mission formalizes that bound and the three estimates it is assembled from.

Setting

Let ppp be a prime and ggg a generator of (Z/pZ)×(\mathbb Z/p\mathbb Z)^\times(Z/pZ)×, so that 1,g,…,gp−21,g,\dots,g^{p-2}1,g,…,gp−2 are all the nonzero residues. Fix the unknown rrr with 0≤r<p−10\le r<p-10≤r<p−1 and put x=grx=g^rx=gr. Let q=2lq=2^lq=2l be the power of 222 with p<q<2pp<q<2pp<q<2p.

The Fourier matrix AqA_qAq​ is the q×qq\times qq×q matrix with entries (Aq)a,c=q−1/2exp⁡(2πi ac/q)(A_q)_{a,c}=q^{-1/2}\exp(2\pi i\,ac/q)(Aq​)a,c​=q−1/2exp(2πiac/q) for 0≤a,c<q0\le a,c<q0≤a,c<q (§4, eq. (4.1)). Rows index input basis vectors and columns output basis vectors.

The algorithm uses three registers: two holding numbers 0≤a,b<q0\le a,b<q0≤a,b<q and one holding a nonzero residue modulo ppp. It starts from the state

1p−1∑a=0p−2∑b=0p−2∣a,b,gax−b (mod p)⟩(6.1)\frac{1}{p-1}\sum_{a=0}^{p-2}\sum_{b=0}^{p-2}|a,b,g^ax^{-b}\ (\mathrm{mod}\ p)\rangle \qquad (6.1)p−11​a=0∑p−2​b=0∑p−2​∣a,b,gax−b (mod p)⟩(6.1)

(preFourierState), applies AqA_qAq​ to each of the first two registers (finalState), and measures all three registers. The probability of observing ∣c,d,y⟩|c,d,y\rangle∣c,d,y⟩ is the squared modulus of its amplitude (outcomeProb).

For integers zzz and q>0q>0q>0, the symmetric residue {z}q\{z\}_q{z}q​ is the residue of zzz modulo qqq in (−q/2,q/2](-q/2,q/2](−q/2,q/2] (symmRes). Put

T=rc+d−rp−1{c(p−1)}q.T=rc+d-\frac{r}{p-1}\{c(p-1)\}_q .T=rc+d−p−1r​{c(p−1)}q​.

An observed state ∣c,d,y⟩|c,d,y\rangle∣c,d,y⟩ is good (IsGood) when

∣{T}q∣≤12(6.10)and∣{c(p−1)}q∣≤q/12(6.11).|\{T\}_q|\le\tfrac12 \quad (6.10) \qquad\text{and}\qquad |\{c(p-1)\}_q|\le q/12 \quad (6.11).∣{T}q​∣≤21​(6.10)and∣{c(p−1)}q​∣≤q/12(6.11).

Goodness depends only on (c,d)(c,d)(c,d).

Formalization targets

Goal: a good output with probability at least 1/4801/4801/480 (§6, p. 1504)

∑0≤c,d<q(c,d) good ∑y∈(Z/p)×Pr⁡[c,d,y] ≥ 1480.\sum_{\substack{0\le c,d<q\\ (c,d)\ \text{good}}}\ \sum_{y\in(\mathbb Z/p)^\times}\Pr[c,d,y]\ \ge\ \frac1{480}.0≤c,d<q(c,d) good​∑​ y∈(Z/p)×∑​Pr[c,d,y] ≥ 4801​.

The constant is the one the page carries forward. The goal fixes no threshold on ppp: it is stated for every prime ppp that admits a power of two strictly between ppp and 2p2p2p.

Milestones

  1. The output distribution, eq. (6.4). For 0≤k<p−10\le k<p-10≤k<p−1,
Pr⁡[c,d,gk]=∣1(p−1)q∑0≤a,b≤p−2a−rb≡k (p−1)exp⁡(2πiq(ac+bd))∣2.\Pr[c,d,g^k]=\left|\frac{1}{(p-1)q}\sum_{\substack{0\le a,b\le p-2\\ a-rb\equiv k\ (p-1)}}\exp\Bigl(\frac{2\pi i}{q}(ac+bd)\Bigr)\right|^2 .Pr[c,d,gk]=​(p−1)q1​0≤a,b≤p−2a−rb≡k (p−1)​∑​exp(q2πi​(ac+bd))​2.
  1. Each good state is likely, eq. (6.17). If (c,d)(c,d)(c,d) is good, then Pr⁡[c,d,y]≥1/(20q2)\Pr[c,d,y]\ge 1/(20q^2)Pr[c,d,y]≥1/(20q2) for every yyy.
  2. Many good pairs (p. 1504). At least q/12q/12q/12 pairs (c,d)(c,d)(c,d) are good.
  3. Each good ccc is likely (p. 1504). If (c,d)(c,d)(c,d) is good for some ddd, then ∑d′,yPr⁡[c,d′,y]≥(p−1)/(20q2)≥1/(40q)\sum_{d',y}\Pr[c,d',y]\ge(p-1)/(20q^2)\ge1/(40q)∑d′,y​Pr[c,d′,y]≥(p−1)/(20q2)≥1/(40q).

Significance

The result. The bound 1/4801/4801/480 is what turns the circuit into an algorithm. Repeating the circuit O(1)O(1)O(1) times in expectation yields a good output, and from a good pair (c,d)(c,d)(c,d) one reads off an equation that determines rrr modulo divisors of p−1p-1p−1 (§6, eqs. (6.18)–(6.20)). Together with the quantum Fourier transform circuit and reversible modular exponentiation, this places the discrete logarithm modulo a prime in quantum polynomial time. Every later analysis of quantum attacks on discrete-logarithm cryptography starts from this success probability or a sharpened version of it.

Formalizing it. The result has been proved since 1994–1997 and is textbook material; it is not open. As far as is known, no machine-checked proof of Shor's discrete-logarithm analysis exists. The paper's proof of eq. (6.17) replaces a sum by an integral with an error term O(W/(pq))O(W/(pq))O(W/(pq)) whose constant is not given, yet states 1/(20q2)1/(20q^2)1/(20q2) for every prime. A formal proof must therefore either control that error explicitly or find another argument, and so settles a point the paper leaves informal. Numerically, the smallest value of q2Pr⁡[c,d,y]q^2\Pr[c,d,y]q2Pr[c,d,y] over good states is about 0.490.490.49 for all primes p<90p<90p<90, so the unconditional claim is not in doubt for small ppp. The page also contains two small slips, recorded under Formalization scope; a complete development pins down exactly what is true.

Difficulty

The exponential sum (6.4) runs over pairs (a,b)(a,b)(a,b) satisfying a congruence modulo p−1p-1p−1, while the phases are taken modulo qqq. The two moduli are unrelated: qqq is a power of two and p−1p-1p−1 is arbitrary. Eliminating aaa through the congruence introduces a floor function ⌊(br+k)/(p−1)⌋\lfloor(br+k)/(p-1)\rfloor⌊(br+k)/(p−1)⌋, and the resulting phase is not linear in bbb. The obvious estimate treats the sum as a geometric series in bbb and bounds it by its first-order phase; this fails because the floor term perturbs every phase by an amount of size up to ∣{c(p−1)}q∣|\{c(p-1)\}_q|∣{c(p−1)}q​∣. Condition (6.11) only keeps this perturbation within π/6\pi/6π/6 of the main phase; it does not remove it. The per-state bound must survive this perturbation uniformly in ppp, rrr and kkk, including small primes where the paper's integral approximation gives no explicit control.

The count of good pairs needs a separate argument about how often a multiple c(p−1)c(p-1)c(p−1) lies within q/12q/12q/12 of a multiple of qqq when gcd⁡(p−1,q)\gcd(p-1,q)gcd(p−1,q) is large.

Formalization scope

  • States are functions Fin q × Fin q × (ZMod p)ˣ → ℂ. The first two registers range over {0,…,q−1}\{0,\dots,q-1\}{0,…,q−1}; the third over the units modulo ppp.
  • Matrix convention. Following §2, rows are inputs, so the amplitude of ∣c,d,y⟩|c,d,y\rangle∣c,d,y⟩ after the transforms is ∑a,bψ(a,b,y)(Aq)a,c(Aq)b,d\sum_{a,b}\psi(a,b,y)(A_q)_{a,c}(A_q)_{b,d}∑a,b​ψ(a,b,y)(Aq​)a,c​(Aq​)b,d​. finalState is defined this way from (6.1) and AqA_qAq​. It is not typed in as the closed form (6.3) or (6.4). A formalization that defined the final state by (6.4) directly would make milestone 1 trivial, and is ruled out.
  • Probability of a basis state is the squared norm of its amplitude, with no normalization hypothesis.
  • Parameters. ppp is prime (Fact p.Prime). The generator is encoded as orderOf g = p - 1. r<p−1r<p-1r<p−1 is a parameter, with x=grx=g^rx=gr. qqq is given by q = 2 ^ l together with p<q<2pp<q<2pp<q<2p. No large-ppp threshold is added anywhere.
  • Arithmetic. x−bx^{-b}x−b is x⁻¹ ^ b in the unit group. p−1p-1p−1 is computed in Z\mathbb ZZ and R\mathbb RR inside TTT and the congruences, and as natural-number subtraction only where p≥2p\ge2p≥2 makes it exact. TTT is real.
  • Condition (6.10) is stated as "some integer jjj has ∣T−jq∣≤12|T-jq|\le\frac12∣T−jq∣≤21​". Because q≥4q\ge4q≥4, this is equivalent to the page's form with jjj the closest integer to T/qT/qT/q.
  • Not formalized. The preparation of (6.1) by testing and restarting is not formalized; the state (6.1) is taken as displayed. The printed test "whether the number is less than ppp" should read p−1p-1p−1, as the sums in (6.1) show. Also out of scope: the recovery of rrr (eqs. (6.18)–(6.20)), the repetition count "480t480t480t", and all running-time claims.
  • Printed slips.
    • The page asserts that for each ccc there is exactly one ddd satisfying (6.10). At a tie {T}q=±12\{T\}_q=\pm\frac12{T}q​=±21​ there can be two such ddd. Milestone 3 states only the count, which needs at least one.
    • The page's intermediate bound "at least p/(240q)p/(240q)p/(240q)" should be (p−1)/(240q)(p-1)/(240q)(p−1)/(240q). The conclusion 1/4801/4801/480 is unaffected, since qqq and 2p2p2p are both even and so q≤2(p−1)q\le 2(p-1)q≤2(p−1). Only 1/4801/4801/480 is stated.

Needed infrastructure: finite exponential sums and their modulus, the symmetric residue and its basic properties, and counting multiples in residue classes of Z/q\mathbb Z/qZ/q. The exponential-sum estimates of milestones 1 and 2 are reusable in the order-finding analysis of §5 of the same paper. Proofs of any milestone, of the normalization ∑Pr⁡=1\sum\Pr=1∑Pr=1, and of auxiliary lemmas about symmRes are welcome.

Selected references

  • P. W. Shor, Polynomial-Time Algorithms for Prime Factorization and Discrete Logarithms on a Quantum Computer, SIAM J. Comput. 26(5):1484–1509, 1997. https://doi.org/10.1137/S0097539795293172 (preprint: https://arxiv.org/abs/quant-ph/9508027)
  • D. M. Gordon, Discrete logarithms in GF(p) using the number field sieve, SIAM J. Discrete Math. 6(1):124–138, 1993. https://doi.org/10.1137/0406010
  • W. Diffie and M. E. Hellman, New directions in cryptography, IEEE Trans. Inform. Theory 22(6):644–654, 1976. https://doi.org/10.1109/TIT.1976.1055638
  • M. A. Nielsen and I. L. Chuang, Quantum Computation and Quantum Information, Cambridge University Press, 2000. https://doi.org/10.1017/CBO9780511976667
11 thms2 active usersReviewed
🏆Completed
Number TheoryProbability·Captain: mikedeng1

Polynomial-Time Algorithms for Prime Factorization and Discrete Logarithms on a Quantum Computer 2: Factoring from a Random ResidueResearch Paper

Motivation

Shor's 1997 paper (SIAM J. Comput. 26(5), arXiv:quant-ph/9508027) gives a polynomial-time quantum algorithm for factoring integers. The quantum computer does not factor directly: it finds the multiplicative order of an element modulo nnn. The step from order finding to factoring is classical and randomized, and goes back to Miller's 1976 work on primality testing (G. L. Miller, Riemann's hypothesis and tests for primality, J. Comput. System Sci. 13 (1976)). Every account of Shor's algorithm, and every resource estimate for breaking RSA with a quantum computer, depends on this reduction succeeding with a constant probability per trial. This mission formalizes that probability bound as Shor states it on p. 1498 of the published paper.

Setting

Let n>1n > 1n>1 be an odd integer with prime factorization

n=∏i=1kpiαi,n = \prod_{i=1}^{k} p_i^{\alpha_i},n=i=1∏k​piαi​​,

so kkk is the number of distinct prime factors of nnn, all odd. The unit group (Z/nZ)×(\mathbb{Z}/n\mathbb{Z})^\times(Z/nZ)× consists of the residues coprime to nnn; it has φ(n)\varphi(n)φ(n) elements, where φ\varphiφ is Euler's totient function.

For a unit xxx the order r=ord⁡n(x)r = \operatorname{ord}_n(x)r=ordn​(x) is the least positive integer with xr≡1(modn)x^r \equiv 1 \pmod nxr≡1(modn). For each iii the local order rir_iri​ is the order of x mod piαix \bmod p_i^{\alpha_i}xmodpiαi​​, taken modulo the full prime power, not modulo pip_ipi​. For a positive integer mmm, ν2(m)\nu_2(m)ν2​(m) denotes the exponent of the largest power of 222 dividing mmm.

The reduction is: choose xxx uniformly at random from (Z/nZ)×(\mathbb{Z}/n\mathbb{Z})^\times(Z/nZ)×, obtain its order rrr (from the quantum subroutine), and compute

g(x)=gcd⁡(xr/2−1, n).g(x) = \gcd\bigl(x^{r/2} - 1,\ n\bigr).g(x)=gcd(xr/2−1, n).

The procedure yields a nontrivial factor at xxx when rrr is even and 1<g(x)<n1 < g(x) < n1<g(x)<n. In Lean this event is ShorAlgorithms.Reduction.successEvent n u for u : (ZMod n)ˣ, and rir_iri​ is localOrder n u p for p ∈ n.primeFactors.

Formalization targets

Goal: the success probability

Pr⁡x∈(Z/n)×[r even and 1<gcd⁡(xr/2−1,n)<n]  ≥  1−12k−1.\Pr_{x \in (\mathbb{Z}/n)^\times}\bigl[r \text{ even and } 1 < \gcd(x^{r/2}-1, n) < n\bigr] \;\ge\; 1 - \frac{1}{2^{k-1}}.x∈(Z/n)×Pr​[r even and 1<gcd(xr/2−1,n)<n]≥1−2k−11​.

It is stated for every odd n>1n > 1n>1. For a prime power (k=1k = 1k=1) the bound is 000, so the statement says nothing there; it is informative exactly when nnn is not a prime power, as the paper remarks. The constant is sharp: for n=21n = 21n=21 exactly 666 of the 121212 units succeed, so 1−1/2k1 - 1/2^{k}1−1/2k in place of 1−1/2k−11 - 1/2^{k-1}1−1/2k−1 would be false.

Milestones, in the order the page uses them

  1. Success criterion. If rrr is even and xr/2≢−1(modn)x^{r/2} \not\equiv -1 \pmod nxr/2≡−1(modn), then 1<gcd⁡(xr/2−1,n)<n1 < \gcd(x^{r/2}-1, n) < n1<gcd(xr/2−1,n)<n.
  2. Order is the lcm. r=lcm⁡(r1,…,rk)r = \operatorname{lcm}(r_1, \dots, r_k)r=lcm(r1​,…,rk​).
  3. Failure forces agreement. For odd nnn, if the procedure fails at xxx, then ν2(r1)=⋯=ν2(rk)\nu_2(r_1) = \cdots = \nu_2(r_k)ν2​(r1​)=⋯=ν2​(rk​).
  4. At most half per odd prime power. For an odd prime ppp and α≥1\alpha \ge 1α≥1, at most φ(pα)/2\varphi(p^\alpha)/2φ(pα)/2 units modulo pαp^\alphapα have order with a prescribed 2-adic valuation.
  5. All agree rarely. The units for which ν2(r1)=⋯=ν2(rk)\nu_2(r_1) = \cdots = \nu_2(r_k)ν2​(r1​)=⋯=ν2​(rk​) number at most φ(n)/2k−1\varphi(n)/2^{k-1}φ(n)/2k−1.

Significance

The result. The bound turns an order-finding oracle into a factoring algorithm: when nnn is odd and not a prime power, each trial succeeds with probability at least 1/21/21/2, so ttt independent trials all fail with probability at most 2−t2^{-t}2−t. Even numbers and prime powers are split classically, as the paper notes, so the bound completes the reduction from factoring to order finding. The same criterion — a square root of 111 other than ±1\pm 1±1 splits nnn — underlies the Miller–Rabin test and several classical factoring methods.

Formalizing it. The mathematics is classical and proved; the paper gives a sketch of one paragraph. This mission writes out the sketch as machine-checked statements over Mathlib's ZMod, including the probabilistic step, which in the paper is an informal appeal to the Chinese remainder theorem and "50% probability of agreeing with the previous ones". Mathlib already has the needed ingredients (cyclicity of (Z/pα)×(\mathbb{Z}/p^\alpha)^\times(Z/pα)× for odd ppp, ZMod.chineseRemainder, ZMod.card_units_eq_totient), but not the reduction or its probability bound.

Difficulty

The success criterion (milestone 1) is elementary. The substance is the counting. The obvious route — treating the ν2(ri)\nu_2(r_i)ν2​(ri​) as independent and each "equal to the previous one with probability 1/21/21/2" — needs both a precise product decomposition of the unit group modulo nnn into the unit groups modulo piαip_i^{\alpha_i}piαi​​, compatible with the local orders, and the count in a cyclic group of even order of the elements whose order has a given 2-adic valuation. The informal phrase "at most a 50% probability of agreeing with the previous ones" hides a conditioning argument over k−1k - 1k−1 coordinates that has to be done by an explicit cardinality bound. A second pitfall is milestone 3: its converse direction and its forward direction use oddness of nnn in different places, and modulo a power of 222 the argument breaks because −1≡1(mod2)-1 \equiv 1 \pmod 2−1≡1(mod2).

Formalization scope

  • Sample space. Uniform on (ZMod n)ˣ; probabilities are stated in cleared-denominator form, (1−2−(k−1)) φ(n)≤#{successes}(1 - 2^{-(k-1)})\,\varphi(n) \le \#\{\text{successes}\}(1−2−(k−1))φ(n)≤#{successes} in R\mathbb{R}R, with the count as Nat.card of a subtype. Non-units have no multiplicative order and are not sampled.
  • The gcd. xr/2x^{r/2}xr/2 is represented by its least nonnegative residue .val, which is at least 111 for a unit when n>1n > 1n>1, so the natural-number subtraction in val - 1 never truncates. r/2r/2r/2 is natural-number division, used only under Even r.
  • kkk. n.primeFactors.card, at least 111 for n>1n > 1n>1, so k - 1 does not truncate. Since nnn is odd this equals the page's "number of distinct odd prime factors".
  • Local orders. The order of the image of xxx in ZMod (p ^ n.factorization p) under the reduction homomorphism.
  • Hypotheses. The goal assumes exactly nnn odd and n>1n > 1n>1. It does not assume "not a prime power": that clause in the paper describes when the bound is useful. Milestones 1 and 2 do not assume nnn odd, because they do not need it; milestones 3 and 5 do.
  • No trivialization. Counting over all of ZMod n instead of the units would put non-units (with junk order 000) into the denominator; the goal counts over (ZMod n)ˣ and divides by φ(n)\varphi(n)φ(n). The goal's constant is the paper's 1−1/2k−11 - 1/2^{k-1}1−1/2k−1, which is attained, so it cannot be weakened into a triviality without changing the theorem.
  • Welcome contributions. A reusable counting lemma for elements of prescribed 2-adic order in a finite cyclic group; the transfer of ZMod.chineseRemainder to unit groups and to local orders; and proofs of the milestones in any order.

Selected references

  • P. W. Shor, Polynomial-Time Algorithms for Prime Factorization and Discrete Logarithms on a Quantum Computer, SIAM J. Comput. 26(5):1484–1509, 1997. https://doi.org/10.1137/S0097539795293172 (preprint arXiv:quant-ph/9508027, https://arxiv.org/abs/quant-ph/9508027)
  • G. L. Miller, Riemann's hypothesis and tests for primality, J. Comput. System Sci. 13(3):300–317, 1976. https://doi.org/10.1016/S0022-0000(76)80043-8
  • D. E. Knuth, The Art of Computer Programming, Vol. 2: Seminumerical Algorithms, 2nd ed., Addison-Wesley, 1981.
  • G. H. Hardy and E. M. Wright, An Introduction to the Theory of Numbers, 5th ed., Oxford University Press, 1979 (Theorem 121, Chinese remainder theorem).
8 thms2 active usersReviewed
🏆Completed
Harmonic AnalysisQuantum Information·Captain: mikedeng1

Polynomial-Time Algorithms for Prime Factorization and Discrete Logarithms on a Quantum Computer 1: The Quantum Fourier Transform Circuit Computes A_q up to Bit ReversalResearch Paper

Motivation

Shor's factoring and discrete logarithm algorithms (Shor 1997) reduce both problems to sampling from the output of a quantum Fourier transform: a register holding a superposition with a hidden period is transformed, then measured, and the measured value carries information about the period. The whole speed-up rests on one engineering fact: for q=2lq = 2^lq=2l the q×qq \times qq×q Fourier matrix, which has q2=4lq^2 = 4^lq2=4l entries, can be applied by a quantum circuit of only O(l2)O(l^2)O(l2) elementary gates, each acting on one or two bits.

That circuit was found independently by Coppersmith (IBM RC 19642, 1994) and Deutsch, and Shor's §4 presents it following Ekert and Jozsa (Rev. Mod. Phys. 68, 1996). It is the quantum analogue of the radix-2 fast Fourier transform, and it reappears in phase estimation, in the hidden subgroup algorithms for abelian groups, and in every textbook account of quantum computation. This mission formalizes Shor's statement that the circuit computes the Fourier matrix, up to a reversal of the output bits, together with the two displayed steps of its verification.

Setting

A register of lll bits has one basis state ∣a⟩=∣al−1al−2…a0⟩|a\rangle = |a_{l-1} a_{l-2} \dots a_0\rangle∣a⟩=∣al−1​al−2​…a0​⟩ for every bit string, with a0a_0a0​ the least significant bit; the string encodes the integer

a=∑j=0l−12jaj,0≤a<q=2l.a = \sum_{j=0}^{l-1} 2^j a_j, \qquad 0 \le a < q = 2^l .a=j=0∑l−1​2jaj​,0≤a<q=2l.

A state is a vector of complex amplitudes, one per basis state. A gate is a matrix whose rows are indexed by input basis vectors and whose columns are indexed by output basis vectors (§2, p. 1489). A gate on some of the bits acts on those bits through its matrix and leaves the others alone (§2, p. 1490): the output amplitude at a string bbb is the sum, over the possible input values uuu of the acted-on bits, of the input amplitude at bbb with those bits replaced by uuu, times the matrix entry from uuu to the corresponding bits of bbb.

The Fourier matrix AqA_qAq​ (eq. (4.1)) is the q×qq \times qq×q matrix with (a,c)(a, c)(a,c) entry q−1/2exp⁡(2πi ac/q)q^{-1/2}\exp(2\pi i\,ac/q)q−1/2exp(2πiac/q); it takes ∣a⟩|a\rangle∣a⟩ to q−1/2∑c=0q−1exp⁡(2πi ac/q) ∣c⟩q^{-1/2}\sum_{c=0}^{q-1}\exp(2\pi i\,ac/q)\,|c\rangleq−1/2∑c=0q−1​exp(2πiac/q)∣c⟩.

The circuit uses two gates (eqs. (4.2), (4.3)):

  • RjR_jRj​ acts on bit jjj with matrix 12(111−1)\frac{1}{\sqrt 2}\begin{pmatrix} 1 & 1 \\ 1 & -1\end{pmatrix}2​1​(11​1−1​);
  • Sj,kS_{j,k}Sj,k​, for j<kj < kj<k, acts on bits jjj and kkk with matrix diag(1,1,1,eiθk−j)\mathrm{diag}(1, 1, 1, e^{i\theta_{k-j}})diag(1,1,1,eiθk−j​), where θk−j=π/2k−j\theta_{k-j} = \pi / 2^{k-j}θk−j​=π/2k−j; it multiplies the amplitude of a basis state by eiθk−je^{i\theta_{k-j}}eiθk−j​ when bits jjj and kkk are both 111.

The circuit is the gate sequence (4.4), applied from left to right:

Rl−1 Sl−2,l−1 Rl−2 Sl−3,l−1 Sl−3,l−2 Rl−3⋯R1 S0,l−1 S0,l−2⋯S0,2 S0,1 R0,R_{l-1}\, S_{l-2,l-1}\, R_{l-2}\, S_{l-3,l-1}\, S_{l-3,l-2}\, R_{l-3} \cdots R_1\, S_{0,l-1}\, S_{0,l-2} \cdots S_{0,2}\, S_{0,1}\, R_0 ,Rl−1​Sl−2,l−1​Rl−2​Sl−3,l−1​Sl−3,l−2​Rl−3​⋯R1​S0,l−1​S0,l−2​⋯S0,2​S0,1​R0​,

that is, for j=l−1,…,0j = l-1, \dots, 0j=l−1,…,0 it applies Sj,l−1,…,Sj,j+1S_{j,l-1}, \dots, S_{j,j+1}Sj,l−1​,…,Sj,j+1​ and then RjR_jRj​. On three bits it is R2S1,2R1S0,2S0,1R0R_2 S_{1,2} R_1 S_{0,2} S_{0,1} R_0R2​S1,2​R1​S0,2​S0,1​R0​.

The bit reversal of a string bbb is the string ccc with ck=bl−1−kc_k = b_{l-1-k}ck​=bl−1−k​.

Formalization targets

Goal: the circuit computes AqA_qAq​ up to bit reversal (§4, p. 1496)

For every l≥0l \ge 0l≥0 and every basis state ∣a⟩|a\rangle∣a⟩, with q=2lq = 2^lq=2l,

circuit ∣a⟩=1q1/2∑bexp⁡(2πi ac/q) ∣b⟩,c=bit reversal of b.\text{circuit}\,|a\rangle = \frac{1}{q^{1/2}}\sum_{b}\exp(2\pi i\,ac/q)\,|b\rangle, \qquad c = \text{bit reversal of } b .circuit∣a⟩=q1/21​b∑​exp(2πiac/q)∣b⟩,c=bit reversal of b.

The sum runs over all lll-bit strings bbb. Reading the output register in reverse order therefore yields Aq∣a⟩A_q|a\rangleAq​∣a⟩.

Milestone 1: the amplitude along the circuit (§4, eq. (4.5), p. 1496)

The amplitude of ∣b⟩|b\rangle∣b⟩ in circuit ∣a⟩\text{circuit}\,|a\ranglecircuit∣a⟩ is

2−l/2exp⁡(i(∑0≤j<lπajbj+∑0≤j<k<lπ2k−jajbk)).2^{-l/2}\exp\Big(i\Big(\sum_{0\le j<l}\pi a_jb_j + \sum_{0\le j<k<l}\frac{\pi}{2^{k-j}}a_jb_k\Big)\Big).2−l/2exp(i(0≤j<l∑​πaj​bj​+0≤j<k<l∑​2k−jπ​aj​bk​)).

Milestone 2: the phase identity (§4, eqs. (4.6)–(4.10), pp. 1496–1497)

For bit strings a,ba, ba,b, with ccc the bit reversal of bbb and a,ca, ca,c their values,

exp⁡(i(∑0≤j<lπajbj+∑0≤j<k<lπ2k−jajbk))=exp⁡(2πi ac/q).\exp\Big(i\Big(\sum_{0\le j<l}\pi a_jb_j + \sum_{0\le j<k<l}\frac{\pi}{2^{k-j}}a_jb_k\Big)\Big) = \exp(2\pi i\,ac/q).exp(i(0≤j<l∑​πaj​bj​+0≤j<k<l∑​2k−jπ​aj​bk​))=exp(2πiac/q).

Significance

The result. The goal says that AqA_qAq​, a dense unitary on 2l2^l2l amplitudes, is realized by lll one-bit gates and l(l−1)/2l(l-1)/2l(l−1)/2 two-bit gates, followed by a relabelling of the output. This is what makes the Fourier sampling step of the factoring algorithm (§5) and of the discrete logarithm algorithm (§6) polynomial in the number of bits. Without it, the analyses of those sections describe measurements of states that no efficient circuit is known to prepare. The bit reversal is a real part of the statement: the circuit does not compute AqA_qAq​ itself for l≥2l \ge 2l≥2, and an implementation must either permute the output bits or read them in reverse order.

Formalizing it. The identity is classical and its proof is short on paper; its content lies in bookkeeping that is easy to get wrong: which bit a gate touches, which of the input or output value a bit holds when a phase gate acts, and the order in which the gates are applied. A machine-checked version fixes all of these conventions explicitly and yields a reusable model of gate-level circuits on bit strings. To the drafter's knowledge there is no Lean formalization of this circuit on Prove2Me; the platform's FastFourierTransform rows state the classical Cooley–Tukey recursion for the unnormalized transform with the opposite sign, which is a different object.

Difficulty

Multiplying out the gate matrices is not an argument beyond tiny lll: the difficulty is to control the whole product of l(l+1)/2l(l+1)/2l(l+1)/2 gates symbolically. The paper's argument (§4, p. 1496) is informal at exactly the points a formal proof must make precise: that only one sequence of intermediate basis states from ∣a⟩|a\rangle∣a⟩ to ∣b⟩|b\rangle∣b⟩ carries nonzero amplitude, and that at the moment Sj,kS_{j,k}Sj,k​ acts, bit kkk already holds its output value bkb_kbk​ while bit jjj still holds its input value aja_jaj​. Both facts depend on the order (4.4); applying the same gates in the reverse order gives a different unitary, whose phase pairs bjb_jbj​ with aka_kak​. The phase identity then holds only modulo 2π2\pi2π, not as an equality of real numbers, so it cannot be closed by rearranging sums alone.

Formalization scope

  • States. A bit string on lll bits is Fin l → Fin 2, with bit j equal to aja_jaj​ and value ∑j2jaj\sum_j 2^j a_j∑j​2jaj​ (least significant bit first). A state is a function from bit strings to ℂ. The case l=0l = 0l=0 (q=1q = 1q=1, empty circuit) is included.
  • Gate application follows the row = input convention: a one-bit gate MMM on bit jjj sends ψ\psiψ to b↦∑uψ(b[j↦u]) Mu,bjb \mapsto \sum_u \psi(b[j\mapsto u])\,M_{u, b_j}b↦∑u​ψ(b[j↦u])Mu,bj​​, and a two-bit gate acts analogously on the pair (bit jjj, bit kkk).
  • The circuit is the gate list (4.4) — an explicit list of constructors R j and S j k — run by a left fold, so the leftmost gate acts first. It is not defined as the matrix AqA_qAq​ or by its entries; a definition of that kind would make the goal true by unfolding and is ruled out.
  • Angles. θk−j=π/2k−j\theta_{k-j} = \pi/2^{k-j}θk−j​=π/2k−j uses natural-number subtraction, which is exact because the list only contains Sj,kS_{j,k}Sj,k​ with j<kj < kj<k.
  • Normalization. The prefactor q−1/2q^{-1/2}q−1/2 is written (2l)−1(\sqrt{2^l})^{-1}(2l​)−1.
  • Basis form. The goal is stated on basis states, as on the page; by linearity it determines the circuit on every state.
  • Not stated. The gate count: the paper's sentence "we thus need to use l(l−1)/2l(l-1)/2l(l−1)/2 quantum gates" (p. 1496) counts only the gates Sj,kS_{j,k}Sj,k​; the sequence (4.4) also contains the lll gates RjR_jRj​, l(l+1)/2l(l+1)/2l(l+1)/2 gates in all. The approximate transform of Coppersmith and the one-bit construction of Griffiths and Niu (p. 1497) are out of scope, as is the polynomial-time claim.

Welcome contributions: proofs of the two milestones and the goal; general lemmas about one- and two-bit gate actions on Fin l → Fin 2 states (commutation of gates on disjoint bits, linearity, action on basis states), which are reusable for any gate-level circuit; and a proof that the circuit is unitary.

Selected references

  • P. W. Shor, Polynomial-Time Algorithms for Prime Factorization and Discrete Logarithms on a Quantum Computer, SIAM J. Comput. 26(5):1484–1509, 1997. https://doi.org/10.1137/S0097539795293172
  • D. Coppersmith, An approximate Fourier transform useful in quantum factoring, IBM Research Report RC 19642, 1994. https://arxiv.org/abs/quant-ph/0201067
  • A. Ekert and R. Jozsa, Quantum computation and Shor's factoring algorithm, Rev. Mod. Phys. 68:733–753, 1996. https://doi.org/10.1103/RevModPhys.68.733
  • R. B. Griffiths and C.-S. Niu, Semiclassical Fourier transform for quantum computation, Phys. Rev. Lett. 76:3228–3231, 1996. https://doi.org/10.1103/PhysRevLett.76.3228
5 thms2 active usersReviewed
🏆Completed
Operations ResearchProbability·Captain: mikedeng1

Competitive Randomized Algorithms for Nonuniform Problems II: Optimal Competitiveness for the Spin-Block ProblemResearch Paper

Motivation

A process on a shared-memory multiprocessor that finds a lock held must decide what to do while it waits. It can spin, repeatedly testing the lock and occupying its processor, or it can block, giving the processor to another process and paying a fixed context-switch cost CCC to be descheduled and later restored. Spinning is cheap when the lock is released soon; blocking is cheap when the wait is long. The waiting time is not known in advance, so the choice has to be made on-line. This is the spin-block problem, studied by Karlin, Manasse, McGeoch and Owicki in Competitive Randomized Algorithms for Nonuniform Problems (Algorithmica 11, 1994, doi:10.1007/BF01189993), §4. Mathematically it is the continuous form of the ski-rental problem, and the same rent-or-buy structure recurs in power-down policies and in TCP acknowledgement (Karlin, Kenyon, Randall, STOC 2001).

Timeline:

  • Karlin, Manasse, Rudolph and Sleator (1988, doi:10.1007/BF01762111) introduced competitive analysis of snoopy caching and showed that 222 is the optimal deterministic factor there. The spin-block analogue is recorded in the 1994 paper (pp. 558–559): spinning for time CCC and then blocking is 222-competitive, and no deterministic algorithm does better.
  • Karlin, Manasse, McGeoch and Owicki (SODA 1990; Algorithmica 1994) found the optimal randomized factors for snoopy caching and spin-block. For spin-block, Theorem 10 (p. 559) gives e/(e−1)≈1.582e/(e-1)\approx1.582e/(e−1)≈1.582 against an oblivious adversary. Theorem 9 (p. 559) shows that against an adaptive on-line adversary randomization does not help: the factor stays 222.

Setting

Fix a context-switch cost C>0C>0C>0. A lock wait is described by its release time τ≥0\tau\ge0τ≥0. An algorithm handling the wait chooses a blocking time b∈[0,∞]b\in[0,\infty]b∈[0,∞]: it spins until time bbb and then blocks (b=∞b=\inftyb=∞: never block). The cost of the wait is

waitCostC(b,τ)={τ,τ≤b,b+C,b<τ,\mathrm{waitCost}_C(b,\tau)=\begin{cases}\tau,&\tau\le b,\\ b+C,&b<\tau,\end{cases}waitCostC​(b,τ)={τ,b+C,​τ≤b,b<τ,​

so a lock released exactly at the blocking time costs τ\tauτ. The optimal off-line algorithm, which knows τ\tauτ, pays min⁡(τ,C)\min(\tau,C)min(τ,C).

An input is a finite sequence σ=(τ0,…,τn−1)\sigma=(\tau_0,\dots,\tau_{n-1})σ=(τ0​,…,τn−1​) of lock waits. A deterministic on-line algorithm chooses the blocking time of wait jjj as a function of τ0,…,τj−1\tau_0,\dots,\tau_{j-1}τ0​,…,τj−1​, the release times it has already observed. Its cost CA(σ)C_A(\sigma)CA​(σ) is the sum of the wait costs, and the off-line cost is Copt(σ)=∑jmin⁡(τj,C)C_{opt}(\sigma)=\sum_j\min(\tau_j,C)Copt​(σ)=∑j​min(τj​,C).

A randomized on-line algorithm is a probability distribution over deterministic on-line algorithms: a probability space (I,μ)(I,\mu)(I,μ) and a deterministic algorithm AiA_iAi​ for each i∈Ii\in Ii∈I, with i↦CAi(σ)i\mapsto C_{A_i}(\sigma)i↦CAi​​(σ) measurable for every σ\sigmaσ. Its expected cost is ECA(σ)=∫CAi(σ) dμ(i)\mathbf{E}C_A(\sigma)=\int C_{A_i}(\sigma)\,d\mu(i)ECA​(σ)=∫CAi​​(σ)dμ(i). Following §1 of the paper, AAA is ccc-competitive against an oblivious adversary if there is a constant aaa with

ECA(σ)≤c⋅Copt(σ)+afor every input σ.\mathbf{E}C_A(\sigma)\le c\cdot C_{opt}(\sigma)+a\qquad\text{for every input }\sigma .ECA​(σ)≤c⋅Copt​(σ)+afor every input σ.

The adversary is oblivious: it fixes σ\sigmaσ before the algorithm's random choices are made.

The paper's algorithm blocks at a random time with cumulative distribution

π(t)={et/C−1e−1,0≤t≤C,1,t>C,\pi(t)=\begin{cases}\dfrac{e^{t/C}-1}{e-1},&0\le t\le C,\\[1ex] 1,&t>C,\end{cases}π(t)=⎩⎨⎧​e−1et/C−1​,1,​0≤t≤C,t>C,​

where π(t)\pi(t)π(t) is the probability of blocking before time ttt.

Formalization targets

Goal: Theorem 10 (p. 559)

For every C>0C>0C>0:

(∀A ∀c, A is c-competitive ⇒ c≥ee−1)and∃A, A is ee−1-competitive.\Big(\forall A\ \forall c,\ A\ \text{is } c\text{-competitive}\ \Rightarrow\ c\ge\tfrac{e}{e-1}\Big)\quad\text{and}\quad\exists A,\ A\ \text{is } \tfrac{e}{e-1}\text{-competitive}.(∀A ∀c, A is c-competitive ⇒ c≥e−1e​)and∃A, A is e−1e​-competitive.

Milestones

  1. Expected cost of one wait (§4.1, p. 560). For a blocking time with law ν\nuν and release time τ\tauτ,
E waitCostC(b,τ)=π(τ) C+∫0τ(1−π(t)) dt,π(t)=ν{b<t}.\mathbf{E}\,\mathrm{waitCost}_C(b,\tau)=\pi(\tau)\,C+\int_0^\tau(1-\pi(t))\,dt,\qquad \pi(t)=\nu\{b<t\}.EwaitCostC​(b,τ)=π(τ)C+∫0τ​(1−π(t))dt,π(t)=ν{b<t}.
  1. The ratio of the paper's distribution (§4.1, p. 560). With the π\piπ above, for all τ≥0\tau\ge0τ≥0,
π(τ) C+∫0τ(1−π(t)) dt≤ee−1min⁡(τ,C).\pi(\tau)\,C+\int_0^\tau(1-\pi(t))\,dt\le\tfrac{e}{e-1}\min(\tau,C).π(τ)C+∫0τ​(1−π(t))dt≤e−1e​min(τ,C).
  1. Theorem 10, first claim: the lower bound c≥e/(e−1)c\ge e/(e-1)c≥e/(e−1) for every ccc-competitive randomized algorithm.
  2. Theorem 10, second claim: existence of an e/(e−1)e/(e-1)e/(e−1)-competitive randomized algorithm.

Significance

Theorem 10 settles the randomized competitive ratio of the continuous ski-rental problem: e/(e−1)e/(e-1)e/(e−1) is achievable and cannot be improved by any on-line algorithm, randomized or not, against an oblivious adversary. The same constant is the limit of the paper's snoopy-caching ratios ep/(ep−1)e_p/(e_p-1)ep​/(ep​−1) as the block size grows, and it recurs in randomized rent-or-buy problems and in on-line primal-dual analyses.

The result is proved in the literature, and the mission's work is to formalize it. The platform has related material but not this statement: Primal-Dual Online Algorithms I: Fractional Ski Rental proves a deterministic, fractional, discrete-day bound (PrimalDualOnline.SkiRental.fractional_competitive), which is a different model and contains no lower bound. A complete formalization produces a reusable model of randomized on-line algorithms with history-dependent decisions and an exact lower bound for them.

Difficulty

The upper bound per lock wait is an explicit computation. The difficulty lies elsewhere. First, the on-line algorithm is allowed to adapt to the release times of all previous waits, so a bound for a single wait does not by itself bound a sequence: the randomized algorithm must be assembled so that each wait is handled with the right blocking law, whatever happened before. Second, the lower bound is a statement about every randomized algorithm and must survive the additive constant aaa: a single hard lock wait proves nothing, because aaa absorbs any bounded loss. It has to be shown that on long sequences of waits every algorithm loses a factor e/(e−1)e/(e-1)e/(e−1) on average. The natural first idea, to exhibit one bad release time for each algorithm, fails for randomized algorithms facing an oblivious adversary.

Formalization scope

All objects live in the namespace NonuniformCompetitive.SpinBlock. Release times are ℝ≥0, blocking times ℝ≥0∞, wait costs ℝ≥0∞, and the off-line cost is real. Committed conventions:

  • C>0C>0C>0 is a hypothesis of every theorem (the paper's "some large cost CCC"); at C=0C=0C=0 blocking at once is free and the lower bound fails.
  • A tie b=τb=\taub=τ costs τ\tauτ; this matches "π(t)\pi(t)π(t) is the probability that the algorithm blocks sometime before time ttt".
  • Inputs are finite sequences of lock waits and competitiveness carries the additive constant aaa of §1 (p. 543). A formalization with a single wait and no additive constant would be a different, easier lower bound and is ruled out.
  • The on-line algorithm sees the release times of earlier waits (the information used by the paper's adaptive algorithms, p. 561). This enlarges the class of algorithms: it strengthens the lower bound and does not affect the upper bound.
  • A randomized algorithm is a mixed strategy whose cost on each fixed input is measurable in the random outcome; the expected cost is a lower Lebesgue integral. Without measurability the lower integral would not be the expectation and the upper bound would become easier than the paper's.
  • Milestone 1 is stated for every blocking law, not only for the paper's π\piπ. Milestone 2 keeps the paper's inequality, although equality holds.

Infrastructure a complete development needs: Lebesgue integrals of functions of a random variable (the layer-cake formula), interval integrals of the exponential, the construction of a probability measure on [0,∞][0,\infty][0,∞] with a prescribed continuous distribution function, and a Yao-type averaging argument over finitely supported input distributions for the lower bound. The model of randomized on-line algorithms with history-dependent decisions is reusable for other rent-or-buy problems. Contributions welcome: proofs of the milestones, and intermediate lemmas such as the discretised lower bound for a fixed step C/pC/pC/p.

Selected references

  • A. R. Karlin, M. S. Manasse, L. A. McGeoch, S. Owicki, Competitive Randomized Algorithms for Nonuniform Problems, Algorithmica 11 (1994), 542–571. doi:10.1007/BF01189993
  • A. R. Karlin, M. S. Manasse, L. Rudolph, D. D. Sleator, Competitive Snoopy Caching, Algorithmica 3 (1988), 79–119. doi:10.1007/BF01762111
  • A. Borodin, R. El-Yaniv, Online Computation and Competitive Analysis, Cambridge University Press, 1998.
  • A. R. Karlin, C. Kenyon, D. Randall, Dynamic TCP Acknowledgement and Other Stories about e/(e−1), STOC 2001, 502–509. doi:10.1145/380752.380845
  • N. Buchbinder, J. Naor, The Design of Competitive Online Algorithms via a Primal–Dual Approach, Foundations and Trends in Theoretical Computer Science 3 (2009). doi:10.1561/0400000024
8 thms2 active usersReviewed
🏆Completed
Operations ResearchOptimizationProbability·Captain: mikedeng1

Single Machine Scheduling with Release Dates III: The Random Per-Job α-Schedule with a Truncated Exponential DensityResearch Paper

Motivation

Scheduling jobs with release dates on one machine to minimize the total weighted completion time, written 1∣rj∣∑wjCj1|r_j|\sum w_jC_j1∣rj​∣∑wj​Cj​, is strongly NP-hard even with unit weights (Lenstra, Rinnooy Kan & Brucker, 1977). It is a basic model of scheduling theory and a standard testbed for approximation algorithms built on linear programming relaxations. Several LP relaxations give lower bounds for it (Dyer & Wolsey, 1990; Queyranne, 1993), and the question how far these bounds can be from the optimum is also a question about the quality of branch-and-bound methods that use them.

Timeline, as surveyed in Table 1 of Goemans, Queyranne, Schulz, Skutella & Wang (2002):

  • Phillips, Stein & Wein (Math. Programming, 1998) introduce converting a preemptive schedule into a nonpreemptive one by list scheduling, and the notion of α\alphaα-points; for the weighted problem their bound is 16+ϵ16+\epsilon16+ϵ.
  • Hall, Shmoys & Wein (SODA 1996) obtain 4; Schulz (IPCO 1996) and Hall, Schulz, Shmoys & Wein (Math. Oper. Res., 1997) obtain 3; Chakrabarti et al. obtain 2.8854+ϵ2.8854+\epsilon2.8854+ϵ, and a combination of methods gives 2.4427+ϵ2.4427+\epsilon2.4427+ϵ.
  • Goemans (SODA 1997) orders jobs by α\alphaα-points of the LP schedule: α=1/2\alpha=1/\sqrt2α=1/2​ gives 1+2≈2.41431+\sqrt2\approx2.41431+2​≈2.4143, a uniformly random α\alphaα gives 2.
  • Chekuri, Motwani, Natarajan & Stein (SIAM J. Comput., 2001) use random α\alphaα-points of an arbitrary preemptive schedule and obtain e/(e−1)e/(e-1)e/(e−1) for unit weights, relative to the preemptive optimum rather than an LP value.
  • Goemans, Queyranne, Schulz, Skutella & Wang (2002) prove 1.74511.74511.7451 for the best common α\alphaα and 1.68531.68531.6853 for job-dependent random αj\alpha_jαj​, both relative to the LP value ZRZ_RZR​. Afrati et al. (FOCS 1999) later give a polynomial-time approximation scheme, which does not bound the LP relaxations.

This mission formalizes the 1.68531.68531.6853 result, the paper's main theorem.

Setting

There are nnn jobs N={0,…,n−1}N=\{0,\dots,n-1\}N={0,…,n−1}. Job jjj has an integral processing time pj>0p_j>0pj​>0, an integral release date rj≥0r_j\ge0rj​≥0 and a weight wj>0w_j>0wj​>0. The jobs are indexed so that w0/p0≥w1/p1≥⋯≥wn−1/pn−1w_0/p_0\ge w_1/p_1\ge\dots\ge w_{n-1}/p_{n-1}w0​/p0​≥w1​/p1​≥⋯≥wn−1​/pn−1​.

The LP schedule is the preemptive schedule that at every moment processes the available (released, unfinished) job of smallest index. Since the data are integral, it is determined slot by slot: in [τ,τ+1)[\tau,\tau+1)[τ,τ+1) it runs the smallest-index job jjj with rj≤τr_j\le\taurj​≤τ and work left, or idles. Let AjLP⊆RA^{LP}_j\subseteq\mathbb RAjLP​⊆R be the set of times at which it processes jjj. The mean busy time of jjj is

MjLP=1pj∫AjLPt dt.M^{LP}_j=\frac1{p_j}\int_{A^{LP}_j}t\,dt .MjLP​=pj​1​∫AjLP​​tdt.

The mean busy time relaxation (R) has a variable MjM_jMj​ per job:

ZR=min⁡{∑jwj(Mj+12pj) : ∑j∈SpjMj≥p(S)(rmin⁡(S)+12p(S)) for all nonempty S⊆N},Z_R=\min\Bigl\{\sum_j w_j\bigl(M_j+\tfrac12p_j\bigr)\ :\ \sum_{j\in S}p_jM_j\ge p(S)\bigl(r_{\min}(S)+\tfrac12p(S)\bigr)\ \text{for all nonempty }S\subseteq N\Bigr\},ZR​=min{j∑​wj​(Mj​+21​pj​) : j∈S∑​pj​Mj​≥p(S)(rmin​(S)+21​p(S)) for all nonempty S⊆N},

with p(S)=∑j∈Spjp(S)=\sum_{j\in S}p_jp(S)=∑j∈S​pj​ and rmin⁡(S)=min⁡j∈Srjr_{\min}(S)=\min_{j\in S}r_jrmin​(S)=minj∈S​rj​. ZRZ_RZR​ is a lower bound on the optimum of 1∣rj∣∑wjCj1|r_j|\sum w_jC_j1∣rj​∣∑wj​Cj​.

For 0<α≤10<\alpha\le10<α≤1 the α\alphaα-point tj(α)t_j(\alpha)tj​(α) is the first time at which jjj has been processed for αpj\alpha p_jαpj​ units in the LP schedule; tj(0+)t_j(0^+)tj​(0+) is the start time of jjj. For a vector α∈(0,1]n\boldsymbol\alpha\in(0,1]^nα∈(0,1]n, the (αj)(\alpha_j)(αj​)-schedule processes the jobs nonpreemptively, as early as possible, in nondecreasing order of tj(αj)t_j(\alpha_j)tj​(αj​); CjαC^{\boldsymbol\alpha}_jCjα​ is the completion time of jjj in it.

Let γ≈0.4835\gamma\approx0.4835γ≈0.4835 be the solution in (0,1)(0,1)(0,1) of γ+ln⁡(2−γ)=e−γ((2−γ)eγ−1)\gamma+\ln(2-\gamma)=e^{-\gamma}\bigl((2-\gamma)e^{\gamma}-1\bigr)γ+ln(2−γ)=e−γ((2−γ)eγ−1), and set

δ=γ+ln⁡(2−γ)≈0.8999,c=1+e−γδ,g(α)={(c−1)eα0<α≤δ,0otherwise.\delta=\gamma+\ln(2-\gamma)\approx0.8999,\qquad c=1+\frac{e^{-\gamma}}{\delta},\qquad g(\alpha)=\begin{cases}(c-1)e^\alpha&0<\alpha\le\delta,\\0&\text{otherwise.}\end{cases}δ=γ+ln(2−γ)≈0.8999,c=1+δe−γ​,g(α)={(c−1)eα0​0<α≤δ,otherwise.​

Formalization targets

Goal: Theorem 3.9 (p. 185)

c<1.6853c<1.6853c<1.6853, and if α1,…,αn\alpha_1,\dots,\alpha_nα1​,…,αn​ each have density ggg and are pairwise independent, then ∑jwjCjα\sum_j w_jC^{\boldsymbol\alpha}_j∑j​wj​Cjα​ is integrable and

E[∑jwjCjα]≤c⋅ZR.\mathbb E\Bigl[\sum_j w_jC^{\boldsymbol\alpha}_j\Bigr]\le c\cdot Z_R .E[j∑​wj​Cjα​]≤c⋅ZR​.

The bound is against the constant ccc defined by the formula; the decimal 1.68531.68531.6853 appears only in the separate inequality c<1.6853c<1.6853c<1.6853.

Milestones

  1. Theorem 2.5 (p. 173): MLPM^{LP}MLP is an optimal solution of (R), so ZR=∑jwj(MjLP+12pj)Z_R=\sum_jw_j(M^{LP}_j+\tfrac12p_j)ZR​=∑j​wj​(MjLP​+21​pj​).
  2. Eq. (3.1) (p. 176): MjLP=∫01tj(α) dαM^{LP}_j=\int_0^1t_j(\alpha)\,d\alphaMjLP​=∫01​tj​(α)dα.
  3. Corollary 3.2 (p. 179): Cjα≤tj(αj)+∑k:αk≤ηk(αj)(1+αk−ηk(αj))pkC^{\boldsymbol\alpha}_j\le t_j(\alpha_j)+\sum_{k:\alpha_k\le\eta_k(\alpha_j)}(1+\alpha_k-\eta_k(\alpha_j))p_kCjα​≤tj​(αj​)+∑k:αk​≤ηk​(αj​)​(1+αk​−ηk​(αj​))pk​, where ηk(αj)\eta_k(\alpha_j)ηk​(αj​) is the fraction of kkk processed by tj(αj)t_j(\alpha_j)tj​(αj​).
  4. Eq. (3.10) (p. 182): MjLP=tj(0+)+∑k∈N2(1−μk)pk+12pjM^{LP}_j=t_j(0^+)+\sum_{k\in N_2}(1-\mu_k)p_k+\tfrac12p_jMjLP​=tj​(0+)+∑k∈N2​​(1−μk​)pk​+21​pj​, where N2N_2N2​ is the set of jobs processed between the start and completion of jjj and μk\mu_kμk​ is the fraction of jjj processed before kkk starts.
  5. Eq. (3.11) (p. 183): the bound of Corollary 3.2 split over N1=N∖(N2∪{j})N_1=N\setminus(N_2\cup\{j\})N1​=N∖(N2​∪{j}) and N2N_2N2​.
  6. Lemma 3.11 (p. 185): ggg is a probability density on (0,1](0,1](0,1], and (i) ∫0ηg(α)(1+α−η) dα≤(c−1)η\int_0^\eta g(\alpha)(1+\alpha-\eta)\,d\alpha\le(c-1)\eta∫0η​g(α)(1+α−η)dα≤(c−1)η, (ii) (1+Eg[α])∫μ1g(α) dα≤c(1−μ)(1+E_g[\alpha])\int_\mu^1g(\alpha)\,d\alpha\le c(1-\mu)(1+Eg​[α])∫μ1​g(α)dα≤c(1−μ) for η,μ∈[0,1]\eta,\mu\in[0,1]η,μ∈[0,1].

Significance

Theorem 3.9 gives a randomized algorithm for 1∣rj∣∑wjCj1|r_j|\sum w_jC_j1∣rj​∣∑wj​Cj​ whose expected cost is at most 1.68531.68531.6853 times the optimum, and a variant of it runs on-line (Theorem 3.14). Since ZRZ_RZR​ is a lower bound, it also shows that the relaxation (R), and the preemptive time-indexed relaxation (D), which has the same value, is within a factor 1.68531.68531.6853 of the optimum (Corollary 3.10). The paper's Section 3.6 shows the gap of these relaxations can approach e/(e−1)≈1.5819e/(e-1)\approx1.5819e/(e−1)≈1.5819, so the bound is not far from what this relaxation can give.

The result is proved on paper; no machine-checked proof exists, and no part of this theory (LP schedule, α\alphaα-points, list scheduling from a preemptive schedule) is on the platform. The mission produces, besides the goal, a reusable formal account of α\alphaα-point scheduling: the LP schedule as a concrete object, the identity between mean busy times and average α\alphaα-points, and the deterministic completion-time bound of Corollary 3.2, which underlies many later α\alphaα-point analyses.

Difficulty

The obvious argument, bounding CjαC^{\boldsymbol\alpha}_jCjα​ by Corollary 3.2 and integrating each αk\alpha_kαk​ independently against ggg, does not work directly: the set of jobs kkk with αk≤ηk(αj)\alpha_k\le\eta_k(\alpha_j)αk​≤ηk​(αj​) depends on αj\alpha_jαj​, and the two effects of the random αk\alpha_kαk​ pull in opposite directions (a small αk\alpha_kαk​ shrinks the terms 1+αk−ηk1+\alpha_k-\eta_k1+αk​−ηk​, a large one removes terms from the sum). The analysis needs the structure of the LP schedule around job jjj (equations (3.9)–(3.11)) to separate the jobs whose ηk\eta_kηk​ is constant in αj\alpha_jαj​ from those for which it jumps from 000 to 111, and then a density tuned to both at once. The claim is made under pairwise independence only, so no product structure of the random vector is available. On the formal side, the LP schedule, α\alphaα-points and list scheduling are defined from scratch, and the measure-theoretic content (integrability of a piecewise-constant function of α\boldsymbol\alphaα, conditioning under pairwise independence) is real work.

Formalization scope

Jobs are Fin n (0-based), ppp and rrr are natural numbers and www is real. The ordering by wj/pjw_j/p_jwj​/pj​ is a hypothesis of every statement about the LP schedule, which is defined by the smallest-index rule; under that hypothesis the two coincide. The LP schedule is defined slot by slot, which is exact for integral data. Processing is represented by sets of times, not indicator functions. tj(α)t_j(\alpha)tj​(α) is an infimum over times, tj(0+)=inf⁡AjLPt_j(0^+)=\inf A^{LP}_jtj​(0+)=infAjLP​, and the (αj)(\alpha_j)(αj​)-schedule is the closed form Cjα=max⁡k⪯j(rk+∑k⪯i⪯jpi)C^{\boldsymbol\alpha}_j=\max_{k\preceq j}\bigl(r_k+\sum_{k\preceq i\preceq j}p_i\bigr)Cjα​=maxk⪯j​(rk​+∑k⪯i⪯j​pi​) of list scheduling in lexicographic (α-point, index) order. ZRZ_RZR​ is the real infimum of the objective over the feasible set of (R), which is nonempty and bounded below for w≥0w\ge0w≥0. The random vector is any probability measure on Rn\mathbb R^nRn whose coordinate laws all equal the law with density ggg and whose coordinates are pairwise independent; the product measure is one example, but the theorem is for all of them. γ\gammaγ is any solution in (0,1)(0,1)(0,1) of its equation.

A trivializing formalization is ruled out: the expectation is asserted together with integrability (a non-integrable integrand would have Bochner integral 000), ggg is supported on (0,δ](0,\delta](0,δ] so every αj\alpha_jαj​ lies in (0,1](0,1](0,1] almost surely, and the bound is against ZRZ_RZR​ defined from (R), not against an expression that already contains Theorem 2.5.

Running times (O(nlog⁡n)O(n\log n)O(nlogn), O(n2)O(n^2)O(n2)), derandomization, the counting results (Proposition 3.8, Lemma 3.12) and the on-line variant are not formalized. Lemma 3.1 on the auxiliary (αj)(\alpha_j)(αj​)-Conversion schedule is not a milestone; Corollary 3.2 is stated directly for the (αj)(\alpha_j)(αj​)-schedule.

Contributions welcome: proofs of the milestones in any order; general lemmas on list scheduling and α\alphaα-points of preemptive schedules, which are reusable beyond this mission; and the measure-theoretic step from pairwise independence to the conditional bound on E[Cjα∣αj]\mathbb E[C^{\boldsymbol\alpha}_j\mid\alpha_j]E[Cjα​∣αj​].

Selected references

  • M. X. Goemans, M. Queyranne, A. S. Schulz, M. Skutella, Y. Wang, Single machine scheduling with release dates, SIAM J. Discrete Math. 15(2):165–192, 2002. https://doi.org/10.1137/S089548019936223X
  • C. Phillips, C. Stein, J. Wein, Minimizing average completion time in the presence of release dates, Math. Programming 82:199–223, 1998.
  • L. A. Hall, A. S. Schulz, D. B. Shmoys, J. Wein, Scheduling to minimize average completion time: off-line and on-line approximation algorithms, Math. Oper. Res. 22:513–544, 1997. https://doi.org/10.1287/moor.22.3.513
  • M. X. Goemans, Improved approximation algorithms for scheduling with release dates, Proc. 8th ACM–SIAM SODA, 591–598, 1997.
  • C. Chekuri, R. Motwani, B. Natarajan, C. Stein, Approximation techniques for average completion time scheduling, SIAM J. Comput. 31:146–166, 2001. https://doi.org/10.1137/S0097539797327180
  • M. E. Dyer, L. A. Wolsey, Formulating the single machine sequencing problem with release dates as a mixed integer program, Discrete Appl. Math. 26:255–270, 1990.
  • J. K. Lenstra, A. H. G. Rinnooy Kan, P. Brucker, Complexity of machine scheduling problems, Ann. Discrete Math. 1:343–362, 1977.
13 thms2 active usersReviewed
🏆Completed
Operations ResearchOptimizationProbability·Captain: mikedeng1

Single Machine Scheduling with Release Dates II: The Random α-Schedule with a Truncated Exponential DensityResearch Paper

Motivation

Minimizing the total weighted completion time ∑jwjCj\sum_j w_j C_j∑j​wj​Cj​ of jobs with release dates on a single machine, written 1 ∣ rj ∣ ∑wjCj1\,|\,r_j\,|\,\sum w_j C_j1∣rj​∣∑wj​Cj​ in scheduling notation, is strongly NP-hard. It is one of the basic models of machine scheduling, and it has served as a test case for a general technique in approximation algorithms: solve a linear programming relaxation, read off a preemptive schedule, and convert it into a nonpreemptive one using α\alphaα-points, the times at which a given fraction of each job has been processed. Phillips, Stein and Wein (doi:10.1007/BF01585872) introduced ordering jobs by points of a preemptive schedule; Goemans (SODA 1997, reference [11] of the paper below) and Chekuri, Motwani, Natarajan and Stein (doi:10.1137/S0097539797327180) showed that choosing α\alphaα at random improves the guarantee.

Goemans, Queyranne, Schulz, Skutella and Wang (doi:10.1137/S089548019936223X) combine two LP relaxations, shown to have equal value, with carefully chosen random α\alphaα. This mission formalizes their Theorem 3.5: when a single α\alphaα is drawn from a truncated exponential density, the resulting schedule costs in expectation at most c<1.7451c < 1.7451c<1.7451 times the LP lower bound.

Timeline:

  • 1998: Phillips, Stein and Wein give the first constant-factor approximation (ratio 222) for the unit-weight problem 1 ∣ rj ∣ ∑Cj1\,|\,r_j\,|\,\sum C_j1∣rj​∣∑Cj​ by converting a preemptive schedule.
  • 1997: Goemans (SODA) introduces randomly chosen α\alphaα-points for the weighted problem, the conference precursor of the paper formalized here.
  • 2001: Chekuri, Motwani, Natarajan and Stein give a randomized e/(e−1)≈1.58e/(e-1) \approx 1.58e/(e−1)≈1.58-approximation for the unit-weight problem.
  • 2002: Goemans, Queyranne, Schulz, Skutella and Wang give 1.74511.74511.7451 for a single random α\alphaα (Theorem 3.5) and 1.68531.68531.6853 for job-dependent random αj\alpha_jαj​ (Theorem 3.9), both against the same LP bound.
  • 1999: Afrati et al. (doi:10.1109/SFFCS.1999.814574) give a polynomial-time approximation scheme for the problem. It is not LP-based, so the LP-relative factors of Theorems 3.5 and 3.9 remain of interest as bounds on the relaxations' integrality gaps.

Setting

There are nnn jobs N={1,…,n}N = \{1, \dots, n\}N={1,…,n}. Job jjj has an integral processing time pj>0p_j > 0pj​>0, an integral release date rj≥0r_j \ge 0rj​≥0 and a weight wj>0w_j > 0wj​>0. The jobs are indexed so that w1/p1≥w2/p2≥⋯≥wn/pnw_1/p_1 \ge w_2/p_2 \ge \cdots \ge w_n/p_nw1​/p1​≥w2​/p2​≥⋯≥wn​/pn​.

A preemptive schedule gives each job a set Aj⊆[rj,∞)A_j \subseteq [r_j, \infty)Aj​⊆[rj​,∞) of processing times of measure pjp_jpj​, the sets pairwise disjoint. The mean busy time of jjj is Mj=1pj∫Ajt dtM_j = \frac{1}{p_j}\int_{A_j} t\,dtMj​=pj​1​∫Aj​​tdt.

The LP schedule is the preemptive schedule that always processes the available job of smallest index, which under the indexing above is the available job of largest ratio wj/pjw_j/p_jwj​/pj​. Its mean busy times are MjLPM^{LP}_jMjLP​.

The mean busy time relaxation (R) minimizes ∑jwj(Mj+12pj)\sum_j w_j (M_j + \tfrac12 p_j)∑j​wj​(Mj​+21​pj​) subject to ∑j∈SpjMj≥p(S)(rmin⁡(S)+12p(S))\sum_{j \in S} p_j M_j \ge p(S)\big(r_{\min}(S) + \tfrac12 p(S)\big)∑j∈S​pj​Mj​≥p(S)(rmin​(S)+21​p(S)) for every nonempty S⊆NS \subseteq NS⊆N, where p(S)=∑j∈Spjp(S) = \sum_{j\in S} p_jp(S)=∑j∈S​pj​ and rmin⁡(S)=min⁡j∈Srjr_{\min}(S) = \min_{j\in S} r_jrmin​(S)=minj∈S​rj​. Its optimal value is ZRZ_RZR​, a lower bound on the optimum of 1 ∣ rj ∣ ∑wjCj1\,|\,r_j\,|\,\sum w_j C_j1∣rj​∣∑wj​Cj​.

For 0<α≤10 < \alpha \le 10<α≤1, the α\alphaα-point tj(α)t_j(\alpha)tj​(α) is the first time at which jjj has received αpj\alpha p_jαpj​ units of processing in the LP schedule, and tj(0+)t_j(0^+)tj​(0+) is its start time. The α\alphaα-schedule processes the jobs nonpreemptively, each as early as possible, in nondecreasing order of tj(α)t_j(\alpha)tj​(α); CjαC^\alpha_jCjα​ is the completion time of jjj in it. More generally, the (αj)(\alpha_j)(αj​)-schedule orders the jobs by tj(αj)t_j(\alpha_j)tj​(αj​) for a vector α=(αj)\boldsymbol\alpha = (\alpha_j)α=(αj​).

Let 0<γ<10 < \gamma < 10<γ<1 solve 1−γ21+γ=γ+ln⁡(1+γ)1 - \frac{\gamma^2}{1+\gamma} = \gamma + \ln(1+\gamma)1−1+γγ2​=γ+ln(1+γ) (γ≈0.4675\gamma \approx 0.4675γ≈0.4675), and set

c=1+γ1+γ−e−γ,δ=1−γ21+γ,f(α)={(c−1)eα0<α≤δ,0otherwise.c = \frac{1+\gamma}{1+\gamma-e^{-\gamma}}, \qquad \delta = 1 - \frac{\gamma^2}{1+\gamma}, \qquad f(\alpha) = \begin{cases}(c-1)e^{\alpha} & 0 < \alpha \le \delta,\\ 0 & \text{otherwise.}\end{cases}c=1+γ−e−γ1+γ​,δ=1−1+γγ2​,f(α)={(c−1)eα0​0<α≤δ,otherwise.​

Formalization targets

Goal: Theorem 3.5

If α\alphaα is drawn with density fff, then c<1.7451c < 1.7451c<1.7451, the expectation below is finite, and

Ef[∑jwjCjα]≤c⋅ZR.\mathbb E_f\Big[\sum_{j} w_j C^\alpha_j\Big] \le c \cdot Z_R .Ef​[j∑​wj​Cjα​]≤c⋅ZR​.

Milestones

  1. Theorem 2.5: MLPM^{LP}MLP is an optimal solution to (R), so ZR=∑jwj(MjLP+12pj)Z_R = \sum_j w_j (M^{LP}_j + \tfrac12 p_j)ZR​=∑j​wj​(MjLP​+21​pj​).
  2. Eq. (3.1): MjLP=∫01tj(α) dαM^{LP}_j = \int_0^1 t_j(\alpha)\,d\alphaMjLP​=∫01​tj​(α)dα.
  3. Corollary 3.2: Cjα≤tj(αj)+∑k: αk≤ηk(αj)(1+αk−ηk(αj)) pkC^{\boldsymbol\alpha}_j \le t_j(\alpha_j) + \sum_{k:\,\alpha_k\le\eta_k(\alpha_j)} (1+\alpha_k-\eta_k(\alpha_j))\,p_kCjα​≤tj​(αj​)+∑k:αk​≤ηk​(αj​)​(1+αk​−ηk​(αj​))pk​, where ηk(αj)\eta_k(\alpha_j)ηk​(αj​) is the fraction of kkk processed by tj(αj)t_j(\alpha_j)tj​(αj​) in the LP schedule.
  4. Eq. (3.10): MjLP=tj(0+)+∑k∈N2(1−μk)pk+12pjM^{LP}_j = t_j(0^+) + \sum_{k\in N_2}(1-\mu_k)p_k + \tfrac12 p_jMjLP​=tj​(0+)+∑k∈N2​​(1−μk​)pk​+21​pj​, where N2N_2N2​ is the set of jobs processed between the start and the completion of jjj and μk\mu_kμk​ the fraction of jjj done before kkk starts.
  5. Eq. (3.11): the bound of Corollary 3.2 rewritten in terms of N1N_1N1​, N2N_2N2​ and μk\mu_kμk​.
  6. Lemma 3.6: fff is a density on [0,1][0,1][0,1] with ∫0ηf(α)(1+α−η) dα≤(c−1)η\int_0^\eta f(\alpha)(1+\alpha-\eta)\,d\alpha \le (c-1)\eta∫0η​f(α)(1+α−η)dα≤(c−1)η and ∫μ1f(α)(1+α) dα≤c(1−μ)\int_\mu^1 f(\alpha)(1+\alpha)\,d\alpha \le c(1-\mu)∫μ1​f(α)(1+α)dα≤c(1−μ) for η,μ∈[0,1]\eta, \mu \in [0,1]η,μ∈[0,1].

Significance

Theorem 3.5 is a randomized 1.74511.74511.7451-approximation for 1 ∣ rj ∣ ∑wjCj1\,|\,r_j\,|\,\sum w_j C_j1∣rj​∣∑wj​Cj​, and simultaneously a bound on the integrality gap of (R): every instance has OPT≤1.7451 ZR\mathrm{OPT} \le 1.7451\, Z_ROPT≤1.7451ZR​. The paper also derandomizes it: there are at most nnn distinct α\alphaα-schedules (its Proposition 3.8), and the best of them can be found in O(n2)O(n^2)O(n2) time. The paper remarks that the truncated exponential density is optimal for its analysis and that the factor is tight for it.

On the formal side, the mission produces a reusable model of single-machine preemptive schedules, the LP schedule, α\alphaα-points and list scheduling, and the first machine-checked instance of the α\alphaα-point rounding technique that recurs across scheduling approximation. The result itself is proved in the paper; no machine-checked proof of it or of the LP-schedule identities (3.1), (3.10) is known to exist.

Difficulty

The obvious approach, bounding each CjαC^\alpha_jCjα​ separately by a multiple of tj(α)t_j(\alpha)tj​(α), gives only the factor max⁡{1+1/α,1+2α}\max\{1 + 1/\alpha, 1 + 2\alpha\}max{1+1/α,1+2α} for fixed α\alphaα (Theorem 3.3 of the paper), which is at least 1+21+\sqrt21+2​. The improvement needs the precise structure of the LP schedule around each job: which jobs interrupt jjj (N2N_2N2​) and which do not (N1N_1N1​), and how the fraction ηk(α)\eta_k(\alpha)ηk​(α) of each other job depends on α\alphaα. Turning this structure into exact identities for piecewise-constant, piecewise-linear functions of α\alphaα defined through a recursively built schedule is the bulk of the formal work. The analytic part (Lemma 3.6) is elementary but depends on the specific equation that defines γ\gammaγ.

Formalization scope

Jobs are Fin n (0-based) with p,r:Fin n→Np, r : \texttt{Fin } n \to \mathbb Np,r:Fin n→N and real weights. The sortedness wj/pj≥wk/pkw_j/p_j \ge w_k/p_kwj​/pj​≥wk​/pk​ for j≤kj \le kj≤k, positivity pj>0p_j > 0pj​>0 and wj>0w_j > 0wj​>0 are hypotheses of every theorem about the LP schedule. Because the data are integral, the LP schedule is defined slot by slot on unit intervals [τ,τ+1)[\tau, \tau+1)[τ,τ+1); a job's processing set is a finite union of such intervals. α\alphaα-points are infima over reals, and every statement restricts α\alphaα to (0,1](0,1](0,1]. The (αj)(\alpha_j)(αj​)-schedule's completion times are given by the closed form of list scheduling, Cj=max⁡k⪯j(rk+∑k⪯i⪯jpi)C_j = \max_{k \preceq j}(r_k + \sum_{k \preceq i \preceq j} p_i)Cj​=maxk⪯j​(rk​+∑k⪯i⪯j​pi​), with ties of α\alphaα-points broken by index (they do not occur for α∈(0,1]\alpha \in (0,1]α∈(0,1]). ZRZ_RZR​ is the infimum of the objective of (R) over its feasible set. The random α\alphaα has law f(α) dαf(\alpha)\,d\alphaf(α)dα on R\mathbb RR, and the expectation is a Lebesgue integral.

The goal asserts integrability of α↦∑jwjCjα\alpha \mapsto \sum_j w_j C^\alpha_jα↦∑j​wj​Cjα​ together with the bound, so the inequality cannot hold through the convention that a non-integrable function has integral 000. The bound is against ZRZ_RZR​ defined from (R), not against ∑jwj(MjLP+12pj)\sum_j w_j (M^{LP}_j + \tfrac12 p_j)∑j​wj​(MjLP​+21​pj​), which would build Theorem 2.5 into the goal; and the density's support (0,δ](0,\delta](0,δ] is written out, so no mass is placed outside (0,1](0,1](0,1]. The running-time claims are not formalized, and the uniqueness of γ\gammaγ is not asserted: the statements hold for every solution in (0,1)(0,1)(0,1).

Useful infrastructure: finite unions of intervals and their measures, monotone piecewise-linear functions and their integrals, and list-scheduling identities. Contributions to any milestone, and general lemmas about the LP schedule (it is a preemptive schedule, has no idle time inside a job's span, processes interrupting jobs completely), are welcome.

Selected references

  • M. X. Goemans, M. Queyranne, A. S. Schulz, M. Skutella, Y. Wang, Single Machine Scheduling with Release Dates, SIAM J. Discrete Math. 15(2):165–192, 2002. doi:10.1137/S089548019936223X
  • C. Phillips, C. Stein, J. Wein, Minimizing average completion time in the presence of release dates, Math. Programming 82:199–223, 1998. doi:10.1007/BF01585872
  • M. X. Goemans, Improved approximation algorithms for scheduling with release dates, Proc. 8th ACM-SIAM SODA, 591–598, 1997 (no DOI; reference [11] of Goemans et al. 2002).
  • C. Chekuri, R. Motwani, B. Natarajan, C. Stein, Approximation techniques for average completion time scheduling, SIAM J. Comput. 31(1):146–166, 2001. doi:10.1137/S0097539797327180
  • F. Afrati et al., Approximation schemes for minimizing average weighted completion time with release dates, Proc. 40th IEEE FOCS, 32–43, 1999. doi:10.1109/SFFCS.1999.814574
13 thms2 active usersReviewed
🏆Completed
Linear OptimizationOperations ResearchOptimization·Captain: mikedeng1

Single Machine Scheduling with Release Dates I: The Preemptive Time-Indexed and Mean Busy Time LP Relaxations Have the Same Optimal ValueResearch Paper

Motivation

Minimizing the total weighted completion time ∑jwjCj\sum_j w_j C_j∑j​wj​Cj​ of jobs with release dates on a single machine, written 1 ∣ rj ∣ ∑wjCj1\,|\,r_j\,|\,\sum w_j C_j1∣rj​∣∑wj​Cj​ in scheduling notation, is strongly NP-hard. Most constant-factor approximation algorithms for it, and for many related scheduling problems, follow one pattern: solve a linear programming relaxation, which gives a lower bound on the optimum, and round its solution into a schedule whose cost is compared with that bound. The quality of the algorithm is therefore limited by the quality of the relaxation, and the question of which relaxations are equivalent is a basic one for the method.

Two relaxations of 1 ∣ rj ∣ ∑wjCj1\,|\,r_j\,|\,\sum w_j C_j1∣rj​∣∑wj​Cj​ are central. The time-indexed relaxation of Dyer and Wolsey (doi:10.1016/0166-218X(90)90104-K) has one variable per job and unit time slot, a pseudopolynomial number of variables. The mean busy time relaxation has one variable per job but one constraint per subset of jobs, the shifted parallel inequalities studied by Queyranne and others. Goemans, Queyranne, Schulz, Skutella and Wang (doi:10.1137/S089548019936223X, Section 2) show that both have the same optimal value, and that both are solved by one simple preemptive schedule. This mission formalizes that result, Corollary 2.6, together with the lemmas and theorems its proof uses.

Timeline:

  • 1990: Dyer and Wolsey formulate several time-indexed relaxations of 1 ∣ rj ∣ ∑wjCj1\,|\,r_j\,|\,\sum w_j C_j1∣rj​∣∑wj​Cj​, among them the formulation (D) used here.
  • 1993: Queyranne (doi:10.1007/BF01581271) describes the polyhedron of completion-time vectors on one machine without release dates by the parallel inequalities.
  • 1994–1995: Queyranne and Schulz use shifted parallel inequalities in polyhedral approaches to machine scheduling with release dates.
  • 1996: Goemans (IPCO, LNCS 1084) gives a supermodular relaxation for scheduling with release dates, the source of the canonical decompositions used for (R).
  • 2002: Goemans, Queyranne, Schulz, Skutella and Wang prove that (D) and the mean busy time relaxation (R) have equal value, attained by the preemptive "LP schedule", and use this common bound for randomized approximation algorithms with ratios 1.74511.74511.7451 and 1.68531.68531.6853.

Setting

There are nnn jobs N={1,…,n}N = \{1, \dots, n\}N={1,…,n}. Job jjj has an integral processing time pj>0p_j > 0pj​>0, an integral release date rj≥0r_j \ge 0rj​≥0 and a weight wj≥0w_j \ge 0wj​≥0. For a nonempty set SSS of jobs, p(S)=∑j∈Spjp(S) = \sum_{j\in S} p_jp(S)=∑j∈S​pj​ and rmin⁡(S)=min⁡j∈Srjr_{\min}(S) = \min_{j \in S} r_jrmin​(S)=minj∈S​rj​.

A preemptive schedule gives each job a bounded measurable set Aj⊆[rj,∞)A_j \subseteq [r_j, \infty)Aj​⊆[rj​,∞) of processing times of Lebesgue measure pjp_jpj​, the sets pairwise disjoint. The mean busy time of jjj is Mj=1pj∫Ajt dtM_j = \frac{1}{p_j}\int_{A_j} t\,dtMj​=pj​1​∫Aj​​tdt.

The LP schedule processes, at every moment, the available (released, unfinished) job of largest ratio wj/pjw_j/p_jwj​/pj​, ties broken by index. With the jobs indexed so that w1/p1≥⋯≥wn/pnw_1/p_1 \ge \cdots \ge w_n/p_nw1​/p1​≥⋯≥wn​/pn​, this is the available job of smallest index. As the data are integral, it runs one job or none in each unit slot [τ,τ+1)[\tau, \tau+1)[τ,τ+1); yjτLP∈{0,1}y^{LP}_{j\tau} \in \{0,1\}yjτLP​∈{0,1} records whether it runs jjj there, and MjLPM^{LP}_jMjLP​ is its mean busy time of jjj.

The preemptive time-indexed relaxation (D) with horizon TTT has real variables yjτ≥0y_{j\tau} \ge 0yjτ​≥0 for τ=rj,…,T−1\tau = r_j, \dots, T-1τ=rj​,…,T−1 and reads

ZD=min⁡∑jwjCjs.t.∑j:rj≤τyjτ≤1 (τ<T),∑τ=rjT−1yjτ=pj,Cj=12pj+1pj∑τ=rjT−1(τ+12)yjτ.Z_D = \min \sum_j w_j C_j \quad\text{s.t.}\quad \sum_{j : r_j \le \tau} y_{j\tau} \le 1\ (\tau < T), \qquad \sum_{\tau=r_j}^{T-1} y_{j\tau} = p_j, \qquad C_j = \tfrac12 p_j + \tfrac{1}{p_j}\sum_{\tau=r_j}^{T-1}\big(\tau + \tfrac12\big) y_{j\tau}.ZD​=minj∑​wj​Cj​s.t.j:rj​≤τ∑​yjτ​≤1 (τ<T),τ=rj​∑T−1​yjτ​=pj​,Cj​=21​pj​+pj​1​τ=rj​∑T−1​(τ+21​)yjτ​.

The mean busy time relaxation (R) reads

ZR=min⁡∑jwj(Mj+12pj)s.t.∑j∈SpjMj≥p(S)(rmin⁡(S)+12p(S))(∅≠S⊆N).Z_R = \min \sum_j w_j\big(M_j + \tfrac12 p_j\big) \quad\text{s.t.}\quad \sum_{j\in S} p_j M_j \ge p(S)\big(r_{\min}(S) + \tfrac12 p(S)\big) \quad (\emptyset \ne S \subseteq N).ZR​=minj∑​wj​(Mj​+21​pj​)s.t.j∈S∑​pj​Mj​≥p(S)(rmin​(S)+21​p(S))(∅=S⊆N).

The horizon TTT is required to bound the makespan of some feasible nonpreemptive schedule, for instance T=max⁡jrj+∑jpjT = \max_j r_j + \sum_j p_jT=maxj​rj​+∑j​pj​.

Formalization targets

Goal: Corollary 2.6

For every instance, every weight vector w≥0w \ge 0w≥0 and every admissible horizon TTT,

ZD=ZR.Z_D = Z_R .ZD​=ZR​.

No ordering of the jobs and no reference to the LP schedule appear in the goal.

Milestones

  1. Lemma 2.1. (D) has an optimal solution with yjτ∈{0,1}y_{j\tau} \in \{0,1\}yjτ​∈{0,1}.
  2. Theorem 2.2. With the jobs sorted by wj/pjw_j/p_jwj​/pj​, yLPy^{LP}yLP is an optimal solution to (D).
  3. Lemma 2.4. For every preemptive schedule and nonempty SSS, ∑j∈SpjMj≥p(S)(rmin⁡(S)+12p(S))\sum_{j\in S} p_j M_j \ge p(S)\big(r_{\min}(S)+\tfrac12 p(S)\big)∑j∈S​pj​Mj​≥p(S)(rmin​(S)+21​p(S)), with equality if and only if SSS occupies [rmin⁡(S),rmin⁡(S)+p(S))[r_{\min}(S), r_{\min}(S)+p(S))[rmin​(S),rmin​(S)+p(S)) without interruption.
  4. Theorem 2.5. With the jobs sorted by wj/pjw_j/p_jwj​/pj​, MLPM^{LP}MLP is an optimal solution to (R).
  5. Eq. (2.6). MjLP=1pj∑τ=rjT−1yjτLP(τ+12)M^{LP}_j = \frac{1}{p_j}\sum_{\tau=r_j}^{T-1} y^{LP}_{j\tau}\big(\tau + \frac12\big)MjLP​=pj​1​∑τ=rj​T−1​yjτLP​(τ+21​).

Significance

The equality ZD=ZRZ_D = Z_RZD​=ZR​ lets one choose, for each purpose, the more convenient of the two relaxations. (D) is intuitive and a transportation problem, but has pseudopolynomially many variables; (R) has nnn variables and a supermodular right-hand side, which the paper uses to describe its polyhedron. Theorems 2.2 and 2.5 show that the common optimum is attained by the LP schedule, which is computable greedily; the approximation guarantees of Section 3 of the paper, and of later work on α\alphaα-point scheduling, are all measured against this value.

A formal development adds three things. First, a machine-checked model of preemptive single-machine schedules with release dates, of mean busy times, and of the LP schedule as a concrete recursive object, reusable by any formalization of α\alphaα-point methods (two companion missions of this series use the same objects). Second, a formal statement of the two relaxations with honest optimal values. Third, verified proofs of results that are proved in the paper by short interchange and averaging arguments, whose measure-theoretic details (integrals over processing sets, null sets in the equality case) the paper leaves implicit. The results are proved in the literature; to our knowledge none of them has been machine-checked.

Difficulty

The paper's arguments are short, and each rests on a step that is informal on the page. Lemma 2.1 cites the integrality of transportation problems, a statement about the vertices of a polytope rather than a one-line fact. Theorem 2.2 ends with the claim that a 0/1 solution admitting no improving exchange "must correspond to the LP schedule", which is a property of the greedy rule that has to be derived from its definition. Theorem 2.5 depends on how the LP schedule arranges the jobs of each prefix {1,…,i}\{1, \dots, i\}{1,…,i} of the sorted order in time; this is the only place sortedness enters, and it is again a property of the concrete schedule. Lemma 2.4 is an extremal statement about integrals over sets of prescribed measure, and its equality case holds only up to null sets. Finally, the goal concerns arbitrary, unsorted weights, while the two theorems it combines are about sorted indices, so the goal is not a direct conjunction of the milestones.

Formalization scope

Jobs are Fin n, numbered from 000; pjp_jpj​, rjr_jrj​ and the horizon TTT are natural numbers and weights are real. A preemptive schedule is a family of processing sets Aj⊆RA_j \subseteq \mathbb RAj​⊆R, not indicator functions. The LP schedule is defined by recursion on unit slots with the smallest-index rule; the sortedness of wj/pjw_j/p_jwj​/pj​ is a hypothesis of Theorems 2.2 and 2.5, not part of the definition. Variables of (D) are functions y:jobs×N→Ry : \text{jobs} \times \mathbb N \to \mathbb Ry:jobs×N→R required to vanish outside rj≤τ<Tr_j \le \tau < Trj​≤τ<T. ZDZ_DZD​ and ZRZ_RZR​ are infima of the objective over the feasible sets; under the stated hypotheses the feasible sets are nonempty and the objectives bounded below, so these are the LP values. Optimality in the milestones is stated as attaining the minimum, not through these infima.

The horizon hypothesis rules out the trivializing case in which (D) is infeasible and its infimum takes the junk value 000; (D) is kept a linear program over real yyy, since restricting to {0,1}\{0,1\}{0,1} would make Lemma 2.1 vacuous. The running-time claim of Corollary 2.6 (O(nlog⁡n)O(n\log n)O(nlogn)) is not formalized.

A complete development needs: integrals of the identity over finite unions of intervals; a rearrangement lemma for sets of given measure; basic properties of the LP schedule (it is a preemptive schedule, it is work-conserving and finishes by any admissible TTT, its blocks are canonical); and a relabelling argument. The schedule model and the LP-schedule lemmas are reusable for the companion missions on α\alphaα-point scheduling. Contributions of any of these lemmas as separate theorems are welcome.

Selected references

  • M. X. Goemans, M. Queyranne, A. S. Schulz, M. Skutella, Y. Wang, Single machine scheduling with release dates, SIAM Journal on Discrete Mathematics 15(2):165–192, 2002. doi:10.1137/S089548019936223X
  • M. E. Dyer, L. A. Wolsey, Formulating the single machine sequencing problem with release dates as a mixed integer program, Discrete Applied Mathematics 26(2–3):255–270, 1990. doi:10.1016/0166-218X(90)90104-K
  • M. Queyranne, Structure of a simple scheduling polyhedron, Mathematical Programming 58:263–285, 1993. doi:10.1007/BF01581271
  • M. X. Goemans, Improved approximation algorithms for scheduling with release dates, Proceedings of the 8th ACM-SIAM Symposium on Discrete Algorithms (SODA), 591–598, 1997.
10 thms2 active usersReviewed
🏆Completed
CombinatoricsGraph TheoryOperations Research+1·Captain: mikedeng1

Applied Combinatorics VII: Minimum Spanning Trees and Dijkstra's AlgorithmTextbook

Motivation

Two optimization problems on weighted networks sit at the base of operations research and algorithm design. The first asks for the cheapest way to connect every node of a network, such as a cable, pipeline or communication network. The answer is a minimum weight spanning tree. The second asks for the shortest route from a depot to every other node of a road or data network, the single-source shortest path problem. Chapter 12 of Keller and Trotter's Applied Combinatorics (appliedcombinatorics.org, CC BY-SA 4.0) treats both. It proves the structural lemmas behind the greedy spanning tree algorithms of Kruskal (1956) and Prim (1957), and the correctness of the shortest path algorithm of Dijkstra (1959).

The minimum spanning tree problem goes back to Borůvka (1926), who designed an electrical network for Moravia. Kruskal and Prim gave the two greedy algorithms taught today, and Dijkstra's 1959 note treated both problems. Dijkstra's shortest path method, with heap-based refinements such as Fredman and Tarjan (1987), remains the standard solver for non-negative lengths and a building block of routing, scheduling and network flow codes.

Setting

A graph G=(V,E)G = (V, E)G=(V,E) has a finite vertex set VVV and a set EEE of 2-element subsets of VVV. A weight w(e)∈N0w(e) \in \mathbb N_0w(e)∈N0​ is attached to each edge, and a set SSS of edges has weight w(S)=∑e∈Sw(e)w(S) = \sum_{e \in S} w(e)w(S)=∑e∈S​w(e). A spanning forest of GGG is an acyclic graph H=(V,S)H = (V, S)H=(V,S) with S⊆ES \subseteq ES⊆E. A spanning tree is a spanning forest that is connected. The weight of a spanning tree is the weight of its edge set. In Lean these are SimpleGraph V with [Fintype V], IsSpanningForest G H (H≤GH \le GH≤G and acyclic), IsSpanningTree G T (T≤GT \le GT≤G and a tree), and weight w T for a weight w : Sym2 V → ℕ.

A digraph G=(V,E)G = (V, E)G=(V,E) has E⊆V×VE \subseteq V \times VE⊆V×V with x≠yx \ne yx=y for every directed edge (x,y)(x, y)(x,y). Each directed edge has a length w(x,y)∈N0w(x, y) \in \mathbb N_0w(x,y)∈N0​. The length is extended by w(x,y)=∞w(x, y) = \inftyw(x,y)=∞ for non-edges. A directed path from aaa to bbb is a sequence (a=u0,…,ut=b)(a = u_0, \dots, u_t = b)(a=u0​,…,ut​=b) of distinct vertices in which consecutive pairs are directed edges. Its length is ∑i<tw(ui,ui+1)\sum_{i<t} w(u_i, u_{i+1})∑i<t​w(ui​,ui+1​). The distance dist⁡(a,b)∈N0∪{∞}\operatorname{dist}(a, b) \in \mathbb N_0 \cup \{\infty\}dist(a,b)∈N0​∪{∞} is the minimum length of a directed path from aaa to bbb, and is ∞\infty∞ when no such path exists. A shortest path is a directed path attaining it. In Lean this is WeightedDigraph V with ext, IsDirPath, pathLength, dist and IsShortestPath.

Dijkstra's algorithm (Algorithm 12.14) with root rrr and n=∣V∣n = |V|n=∣V∣ keeps a sequence σ\sigmaσ of permanent vertices, a value δ(x)∈N0∪{∞}\delta(x) \in \mathbb N_0 \cup \{\infty\}δ(x)∈N0​∪{∞} and a sequence P(x)P(x)P(x) for each vertex. Step 1 sets δ(r)=0\delta(r) = 0δ(r)=0, P(r)=(r)P(r) = (r)P(r)=(r), σ=(r)\sigma = (r)σ=(r), and δ(x)=w(r,x)\delta(x) = w(r, x)δ(x)=w(r,x), P(x)=(r,x)P(x) = (r, x)P(x)=(r,x) for x≠rx \ne rx=r. Step iii with 1<i<n1 < i < n1<i<n scans from the last permanent vertex viv_ivi​. For every temporary xxx it sets δ(x)←min⁡{δ(x),δ(vi)+w(vi,x)}\delta(x) \leftarrow \min\{\delta(x), \delta(v_i) + w(v_i, x)\}δ(x)←min{δ(x),δ(vi​)+w(vi​,x)}, and on a strict decrease it replaces P(x)P(x)P(x) by P(vi)P(v_i)P(vi​) followed by xxx. Each step ends by appending to σ\sigmaσ a temporary vertex of minimum δ\deltaδ, chosen arbitrarily among ties. The algorithm halts at Step nnn. DijkstraRun G r i s holds when some sequence of admissible choices leads to state s at the start of Step iii.

Formalization targets

Goal: correctness of Dijkstra's algorithm (Theorem 12.18)

For every halted state of every run, and every vertex xxx,

δ(x)=dist⁡(r,x),dist⁡(r,x)<∞  ⟹  P(x) is a shortest path from r to x.\delta(x) = \operatorname{dist}(r, x), \qquad \operatorname{dist}(r, x) < \infty \implies P(x) \text{ is a shortest path from } r \text{ to } x.δ(x)=dist(r,x),dist(r,x)<∞⟹P(x) is a shortest path from r to x.

Milestones

  1. Proposition 12.3. A spanning forest H=(V,S)H = (V, S)H=(V,S) of a graph on n≥1n \ge 1n≥1 vertices has ∣S∣≤n−1|S| \le n - 1∣S∣≤n−1 and exactly n−∣S∣n - |S|n−∣S∣ components. It is a spanning tree if and only if ∣S∣=n−1|S| = n - 1∣S∣=n−1.
  2. Proposition 12.4 (Exchange Principle). Let TTT be a spanning tree and xy∈E∖Txy \in E \setminus Txy∈E∖T. Then TTT contains a unique path x=x0,…,xt=yx = x_0, \dots, x_t = yx=x0​,…,xt​=y, and replacing any edge xixi+1x_i x_{i+1}xi​xi+1​ of it by xyxyxy gives a spanning tree.
  3. Lemma 12.6. In a connected weighted graph, let FFF be a spanning forest and CCC a component of FFF. A minimum weight edge leaving CCC lies in some spanning tree that has minimum weight among the spanning trees containing FFF.
  4. Proposition 12.16. Every prefix and every suffix of a shortest path is a shortest path.
  5. Proposition 12.17. When the algorithm halts, δ(v1)≤δ(v2)≤⋯≤δ(vn)\delta(v_1) \le \delta(v_2) \le \cdots \le \delta(v_n)δ(v1​)≤δ(v2​)≤⋯≤δ(vn​).

Milestones 4 and 5 are the two statements the book's proof of the goal rests on. Milestones 1–3 are the spanning tree half of the chapter. Lemma 12.6 is the result from which the book derives the correctness of Kruskal's and Prim's algorithms.

Significance

Theorem 12.18 certifies that one pass of nnn steps computes all distances from rrr and a shortest path tree, with no condition on the digraph beyond non-negative lengths. Lemma 12.6 is the cut property. Every greedy minimum spanning tree method (Kruskal, Prim, Borůvka) is an instance of it, and the exchange principle is the matroid basis-exchange axiom specialised to the graphic matroid.

All of these results are classical and proved. None is formalized in this form on the platform. Mathlib has spanning trees of connected graphs, uniqueness of paths in acyclic graphs, and the edge count n−1n - 1n−1 of a tree. It has no edge–component count for forests, no exchange principle, no weighted spanning trees, and no Dijkstra. On Prove2Me, FamousTheorems.tree_card_edges_6b and ClassicalGaps.isAcyclic_edges_eq_card_sub_one_imp_connected cover only the tree case of Proposition 12.3. KServer.mst_cut_property is a cut property for complete graphs encoded by parent maps, a different statement. The label-correcting algorithm of Dynamic Programming and Optimal Control II (BertsekasDP.label_correcting_*) is a different algorithm: it keeps an open list and scans in arbitrary order, not by minimum label.

Difficulty

The goal is a statement about the final state of a run, but the facts it depends on only become visible across steps: a permanent vertex's δ\deltaδ and PPP never change again, and δ(x)\delta(x)δ(x) is always the length of the current P(x)P(x)P(x). None of this is recorded in the final state itself. An argument over the steps of the run has to show that each P(x)P(x)P(x) remains a path with distinct vertices, including when edges of length 000 allow ties. It also has to handle the value ∞\infty∞, where ∞+a=∞\infty + a = \infty∞+a=∞ and a comparison between two infinite values never counts as a decrease. Tie-breaking is arbitrary, so no argument may depend on which minimum is chosen. For Lemma 12.6 the difficulty is the exchange step: removing an edge of a tree path and adding a crossing edge must again give a tree that still contains the forest FFF, and this is a statement about cycles and components, not about counts.

Formalization scope

  • Graphs are SimpleGraph V over a Fintype V. Weights are Sym2 V → ℕ (the book's w:E→N0w : E \to \mathbb N_0w:E→N0​; values off EEE are never used). Acyclic, tree and connected components are Mathlib's. In Proposition 12.3, ∣S∣=n−k|S| = n - k∣S∣=n−k is written ∣S∣+k=n|S| + k = n∣S∣+k=n and n≥1n \ge 1n≥1 is assumed, which the bound n−1n - 1n−1 presupposes.
  • Lemma 12.6 assumes GGG connected, the section's standing assumption (p. 239). The page's "to avoid trivialities, we assume n≥3n \ge 3n≥3" is not imposed, because the statement holds for every nnn. The crossing edge may have either endpoint in CCC.
  • Lengths in the digraph are ℕ, and δ\deltaδ and distances are ℕ∞, where ∞\infty∞ is ⊤, never a large finite number. A version with real or ℝ≥0 lengths would be a generalization and is not what is asked.
  • Dijkstra's algorithm is defined step by step exactly as on pp. 246–247, including δ(x)=w(r,x)=∞\delta(x) = w(r, x) = \inftyδ(x)=w(r,x)=∞ and P(x)=(r,x)P(x) = (r, x)P(x)=(r,x) for non-neighbours at Step 1. The goal quantifies over every halted state, so it holds for every tie-breaking. A halted state always exists; a sorry-free check of this is in the workspace. For a vertex not reachable from rrr the book is silent. The distance there is read as ∞\infty∞, and the shortest-path conclusion is asserted only at finite distance.
  • A trivializing formalization is ruled out: δ\deltaδ is computed by the update rule of Algorithm 12.14, not defined as the distance, and the theorem is not stated for an arbitrary procedure satisfying its own conclusion.
  • The book uses no O(⋅)O(\cdot)O(⋅) bounds or approximate constants in these statements, so there are no constants to instantiate.
  • Reusable infrastructure: a list-based theory of directed paths and distances in ℕ∞, the invariants of Dijkstra's algorithm, and forest edge counting. Contributions of general lemmas (walks shortcut to paths without increasing length, component counts under edge insertion) are welcome.

Selected references

  • M. T. Keller and W. T. Trotter, Applied Combinatorics, 2017 Edition, Chapter 12. https://www.appliedcombinatorics.org/
  • E. W. Dijkstra, "A note on two problems in connexion with graphs", Numerische Mathematik 1 (1959) 269–271. https://doi.org/10.1007/BF01386390
  • J. B. Kruskal, "On the shortest spanning subtree of a graph and the traveling salesman problem", Proc. AMS 7 (1956) 48–50. https://doi.org/10.1090/S0002-9939-1956-0078686-7
  • R. C. Prim, "Shortest connection networks and some generalizations", Bell System Technical Journal 36 (1957) 1389–1401. https://doi.org/10.1002/j.1538-7305.1957.tb01515.x
  • M. L. Fredman and R. E. Tarjan, "Fibonacci heaps and their uses in improved network optimization algorithms", J. ACM 34 (1987) 596–615. https://doi.org/10.1145/28869.28874
  • O. Borůvka, "O jistém problému minimálním", Práce Moravské přírodovědecké společnosti 3 (1926) 37–58. https://dml.cz/handle/10338.dmlcz/500114
8 thms2 active usersReviewed
🏆Completed
CombinatoricsGraph TheoryOperations Research+1·Captain: mikedeng1

An Analysis of Several Heuristics for the Traveling Salesman Problem IV: Insertion Heuristics Can Return Poor k-Optimal ToursResearch Paper

Motivation

Insertion heuristics build a traveling salesman tour one city at a time: start from a single city, and at each step choose a city not yet on the subtour and splice it into the subtour where it lengthens the subtour least. Local search heuristics start from a tour and repeatedly replace a few of its edges by others while this shortens the tour. Both families are standard in practice, and a natural engineering idea is to combine them: run an insertion heuristic, then polish the result by local search. The question this mission formalizes is whether local optimality of the insertion tour certifies anything about its quality.

Rosenkrantz, Stearns and Lewis (SIAM J. Comput. 6(3), 1977) answered this for graphs satisfying the triangle inequality. Their §4 proves that nearest and cheapest insertion always return a tour of length at most 2(1−1/n)2(1-1/n)2(1−1/n) times the optimal length, and their Theorem 5 shows this bound is attained. Their §7 then shows that the very tour attaining the bound is kkk-optimal for every k≤n/4k\le n/4k≤n/4: no exchange of kkk edges shortens it. So the insertion bound is tight even for tours that local search with kkk-changes cannot improve.

Timeline, as far as this mission is concerned:

  • 1965: Lin (Bell System Tech. J. 44) defines kkk-optimal tours and uses 3-optimal local search.
  • 1973: Lin and Kernighan (Oper. Res. 21) generalize the edge-exchange neighbourhoods.
  • 1977: Rosenkrantz, Stearns and Lewis prove the 2(1−1/n)2(1-1/n)2(1−1/n) upper bound for nearest and cheapest insertion (Theorem 4 and its corollary), its tightness for n≥6n\ge 6n≥6 (Theorem 5), the existence of kkk-optimal tours with the same ratio (Theorem 6, stated for n≥8n\ge 8n≥8), and the Corollary combining the two.

Setting

A traveling salesman graph on nnn nodes is the node set N={1,…,n}N=\{1,\dots,n\}N={1,…,n} with a distance d(i,j)≥0d(i,j)\ge 0d(i,j)≥0 that is symmetric and satisfies the triangle inequality d(i,k)≤d(i,j)+d(j,k)d(i,k)\le d(i,j)+d(j,k)d(i,k)≤d(i,j)+d(j,k). A tour is a Hamiltonian circuit; its length is the sum of its edge lengths; OPTIMAL is the least length of a tour. As in the paper, the identically zero distance is excluded, so OPTIMAL >0>0>0.

A subtour is a circuit on a subset of the nodes (a single node is a subtour without edges). For a subtour TTT and a node k∉Tk\notin Tk∈/T, TOUR(T,k)(T,k)(T,k) is obtained by deleting an edge (x,y)(x,y)(x,y) of TTT minimizing d(x,k)+d(k,y)−d(x,y)d(x,k)+d(k,y)-d(x,y)d(x,k)+d(k,y)−d(x,y) and adding (x,k)(x,k)(x,k) and (k,y)(k,y)(k,y); COST(T,k)(T,k)(T,k) is the resulting increase in length. An insertion method chooses nodes a0,a1,…,an−1a_0,a_1,\dots,a_{n-1}a0​,a1​,…,an−1​, starts from T1={a0}T_1=\{a_0\}T1​={a0​} and sets Ti+1=TOUR(Ti,ai)T_{i+1}=\mathrm{TOUR}(T_i,a_i)Ti+1​=TOUR(Ti​,ai​); INSERT is the length of TnT_nTn​. Nearest insertion chooses aia_iai​ minimizing d(Ti,x)=min⁡y∈Tid(y,x)d(T_i,x)=\min_{y\in T_i}d(y,x)d(Ti​,x)=miny∈Ti​​d(y,x) over x∉Tix\notin T_ix∈/Ti​; cheapest insertion chooses aia_iai​ minimizing COST(Ti,x)(T_i,x)(Ti​,x). Ties are broken arbitrarily.

A kkk-change of a tour deletes kkk of its edges and adds kkk other edges so that another tour is obtained. A tour is kkk-optimal if no kkk-change produces a strictly shorter tour.

The extremal instance is the circle (Nn,dn)(N_n,d_n)(Nn​,dn​): nnn cities equally spaced on a circular road, with dn(i,j)d_n(i,j)dn​(i,j) the smallest m≥0m\ge 0m≥0 with i−j≡mi-j\equiv mi−j≡m or j−i≡m(modn)j-i\equiv m \pmod nj−i≡m(modn). The insertion run of Theorem 5 inserts the cities in the order 1,2,…,n1,2,\dots,n1,2,…,n and produces the zig-zag tour TnT_nTn​: city 1, then the even cities in increasing order, then the odd cities in decreasing order.

Formalization targets

Goal: the Corollary to Theorem 6

For n≥6n\ge 6n≥6 and 4k≤n4k\le n4k≤n there is a traveling salesman graph with OPTIMAL >0>0>0 on which some run of nearest insertion, and some run of cheapest insertion, return a kkk-optimal tour with

INSERTOPTIMAL=2(1−1n).\frac{\mathrm{INSERT}}{\mathrm{OPTIMAL}}=2\left(1-\frac1n\right).OPTIMALINSERT​=2(1−n1​).

Milestones

  1. The insertion run on the circle: the subtours TiT_iTi​ and nodes ai=i+1a_i=i+1ai​=i+1 form an insertion run that obeys both the nearest and the cheapest rule (proof of Theorem 5).
  2. On the circle, TnT_nTn​ has length 2(n−1)2(n-1)2(n−1) and OPTIMAL =n=n=n (proof of Theorem 5).
  3. Theorem 5: for n≥6n\ge 6n≥6 there is a graph with INSERT/OPTIMAL =2(1−1/n)=2(1-1/n)=2(1−1/n) for both methods.
  4. Equation (7.4): the length of a tour of the circle is the sum over unit edges eee of COUNT(e,T)(e,T)(e,T), the number of times eee is traversed when each tour edge is replaced by a shortest arc.
  5. Every tour of the circle is odd or even (all counts of one parity), eq. (7.5).
  6. TnT_nTn​ is the shortest even tour, so every tour shorter than TnT_nTn​ is odd.
  7. TnT_nTn​ is kkk-optimal for every k≤n/4k\le n/4k≤n/4.
  8. Theorem 6: for n≥8n\ge 8n≥8 there is a graph with a tour that is kkk-optimal for all k≤n/4k\le n/4k≤n/4 and has LOCALOPT/OPTIMAL =2(1−1/n)=2(1-1/n)=2(1−1/n).

Significance

The result. The Corollary shows that the 2(1−1/n)2(1-1/n)2(1−1/n) worst-case guarantee of nearest and cheapest insertion cannot be improved by requiring that the returned tour survive kkk-change local search, for kkk up to a quarter of the number of cities. Theorem 6 says more generally that kkk-optimality with k≤n/4k\le n/4k≤n/4 does not bound the ratio to the optimum below 2(1−1/n)2(1-1/n)2(1−1/n). Together with the paper's upper bound, the insertion guarantee is exact, and it stays exact after local polishing with small neighbourhoods.

Formalizing it. All statements are proved in the paper; none, to our knowledge, has been machine-checked. The platform has an upper bound for nearest insertion (SupplyChainTheory, Theorem 10.7, ratio at most 2) but no tightness example and no notion of kkk-optimality. This mission produces a reusable definition of kkk-changes against arbitrary tours, the circle metric, and the parity-counting argument on a cycle, and it supplies the calculations the paper omits ("We omit these calculations but note that they require the assumption n≥6n\ge 6n≥6").

Difficulty

Two steps carry the weight. First, the omitted calculations for the insertion run: at each stage one must show that inserting aia_iai​ between i−1i-1i−1 and iii minimizes the insertion increase over every edge of the zig-zag subtour, and that no other outside city can be inserted for less than 2; the claim fails for n=4n=4n=4 and n=5n=5n=5, so the verification must use n≥6n\ge 6n≥6 in an essential way. Second, kkk-optimality is a statement about every tour at edge difference kkk, not about 2-opt segment reversals or any specific move family. A search over moves of a special form does not establish it; the argument must bound the length of an arbitrary tour at edge difference kkk from below.

Formalization scope

Nodes are Fin n, so the paper's node mmm is index m−1m-1m−1 and ai=i+1a_i=i+1ai​=i+1 is index iii; subtour indices stay 1-based (T1=[a0]T_1=[a_0]T1​=[a0​], approximation TnT_nTn​). A distance is d : Fin n → Fin n → ℝ with the structure IsTSPDist (symmetric, nonnegative, triangle inequality, and d(i,i)=0d(i,i)=0d(i,i)=0; the last is a normalization absent from the paper that changes no length). A tour is an Equiv.Perm (Fin n), a subtour a list read cyclically; OPTIMAL is Finset.univ.inf' over permutations, the true minimum. TOUR(T,k)(T,k)(T,k) is insertion at a position minimizing the new length; COST is a minimum over positions; the nearest-insertion distance (4.1) takes values in WithTop ℝ, so no junk value arises. kkk-optimality compares the tour with every permutation whose edge set (unordered pairs) misses exactly kkk of the tour's edges. Ratios are multiplied out.

COUNT(e,T)(e,T)(e,T) needs a choice of shortest arc for antipodal pairs when nnn is even; the formalization takes the arc through min⁡(x,y),…,max⁡(x,y)\min(x,y),\dots,\max(x,y)min(x,y),…,max(x,y). The paper's argument does not depend on this choice.

Deviations from the printed text: the Corollary is stated for n≥6n\ge 6n≥6 (the printed statement says only 4k≤n4k\le n4k≤n, but its proof uses the example of Theorem 5, which exists for n≥6n\ge 6n≥6; for k=0k=0k=0, n=3n=3n=3 the printed statement is false). Theorem 6 keeps its printed n≥8n\ge 8n≥8. In the proof of Theorem 5 the paper writes "(4.2) holds" where the cheapest-insertion condition (4.3) is meant; the formal statement uses (4.3).

The existence statements carry OPTIMAL >0>0>0, the paper's standing assumption (1.1). Without it, the zero distance would make every length zero and every tour kkk-optimal, which would satisfy the ratio equations trivially; that formalization is ruled out.

Contributions welcome: proofs of any milestone, in particular the omitted insertion calculations and the parity lemma, and reusable lemmas on cyclic lists and edge sets of permutations.

Selected references

  • D. J. Rosenkrantz, R. E. Stearns, P. M. Lewis II, An Analysis of Several Heuristics for the Traveling Salesman Problem, SIAM J. Comput. 6(3):563–581, 1977. https://doi.org/10.1137/0206041
  • S. Lin, Computer solutions of the traveling salesman problem, Bell System Tech. J. 44:2245–2269, 1965. https://doi.org/10.1002/j.1538-7305.1965.tb04146.x
  • S. Lin, B. W. Kernighan, An effective heuristic algorithm for the traveling-salesman problem, Oper. Res. 21(2):498–516, 1973. https://doi.org/10.1287/opre.21.2.498
14 thms2 active usersReviewed
🏆Completed
CombinatoricsOperations ResearchOptimization·Captain: mikedeng1

Approximation Algorithms for Combinatorial Problems III: The Worst-Case Ratio of the Weighted Algorithm B2Research Paper

Motivation

MAXIMUM SATISFIABILITY asks for a truth assignment satisfying as many clauses of a given set as possible. It was among the first NP-hard optimization problems for which an approximation algorithm came with a proven worst-case guarantee. In Approximation Algorithms for Combinatorial Problems (J. Comput. System Sci. 9, 1974, doi:10.1016/S0022-0000(74)80044-9), David S. Johnson set up a framework for measuring such guarantees and analyzed two heuristics for the problem. The second of these, algorithm B2, weights each clause by 2−∣C∣2^{-|C|}2−∣C∣ and repeatedly sets a literal so that the heavier side is satisfied. On inputs whose clauses all have at least kkk literals, it satisfies at least a 1−2−k1 - 2^{-k}1−2−k fraction of the optimum.

B2 is the ancestor of a line of work on MAX-SAT approximation:

  • 1974. Johnson proves the 2k/(2k−1)2^k/(2^k-1)2k/(2k−1) bound for B2 on MS(k) and the (k+1)/k(k+1)/k(k+1)/k bound for the unweighted greedy algorithm B1.
  • 1994. Goemans and Williamson (SIAM J. Discrete Math. 7) present Johnson's algorithm as the derandomization, by conditional expectations, of the uniformly random assignment, and combine it with LP rounding to obtain a 3/43/43/4-approximation.
  • 1999. Chen, Friesen and Zheng (JCSS 58) show that B2 is a 2/32/32/3-approximation on general inputs, sharper than the 1/21/21/2 that Johnson's bound gives at k=1k = 1k=1.

Setting

A literal is a variable xix_ixi​ or its negation xˉi\bar x_ixˉi​. A clause is a finite set of literals. A truth assignment is a set TTT of literals containing no pair {xi,xˉi}\{x_i, \bar x_i\}{xi​,xˉi​}; it satisfies a clause CCC when C∩T≠∅C \cap T \ne \emptysetC∩T=∅. An input is a finite set SSS of clauses, and S∗S^*S∗ is the largest ∣S′∣|S'|∣S′∣ over subsets S′⊆SS' \subseteq SS′⊆S satisfied by one truth assignment. MS(k) is the restriction to inputs in which every clause contains at least kkk distinct literals.

Algorithm B2 keeps a set SUB of satisfied clauses, a set LEFT of unsettled clauses, the literals LIT still available, and a weight w(C)w(C)w(C) per clause, starting from w(C)=2−∣C∣w(C) = 2^{-|C|}w(C)=2−∣C∣, SUB=∅\mathrm{SUB} = \emptysetSUB=∅, LEFT=S\mathrm{LEFT} = SLEFT=S. While some literal of LIT occurs in a clause of LEFT, it picks any such literal yyy. Let YT be the clauses of LEFT containing yyy and YF those containing yˉ\bar yyˉ​. If ∑YTw≥∑YFw\sum_{\mathrm{YT}} w \ge \sum_{\mathrm{YF}} w∑YT​w≥∑YF​w, it makes yyy true, moves YT to SUB and doubles the weight of each clause of YF; otherwise it does the symmetric move with yˉ\bar yyˉ​. Finally it removes y,yˉy, \bar yy,yˉ​ from LIT. When no literal of LIT remains in LEFT it returns SUB.

The choice of yyy is free, so several outputs may be choosable on one input. Johnson measures the algorithm by its worst choosable output, through the ratio r(B2,S)=S∗/∣SUB∣r(B2, S) = S^*/|\mathrm{SUB}|r(B2,S)=S∗/∣SUB∣ and its maximum R[B2,MS(k)](n)R[B2, \mathrm{MS}(k)](n)R[B2,MS(k)](n) over inputs of size at most nnn.

Formalization targets

Goal: Theorem 3, with the equality range corrected

For every k≥1k \ge 1k≥1, every SSS in MS(k) and every SUB choosable by B2 on SSS,

(2k−1) S∗≤2k ∣SUB∣,(2^k - 1)\,S^* \le 2^k\,|\mathrm{SUB}|,(2k−1)S∗≤2k∣SUB∣,

and for every k≥2k \ge 2k≥2 there are SSS in MS(k) and a choosable SUB with ∣SUB∣>0|\mathrm{SUB}| > 0∣SUB∣>0 and

(2k−1) S∗=2k ∣SUB∣.(2^k - 1)\,S^* = 2^k\,|\mathrm{SUB}|.(2k−1)S∗=2k∣SUB∣.

The paper states equality "for all sufficiently large nnn" for every k≥1k \ge 1k≥1. At k=1k = 1k=1 this is false: the ratio 2 would require every clause to be a unit clause and all clauses to be jointly satisfiable, and on such inputs B2 satisfies everything. The goal therefore asserts tightness for k≥2k \ge 2k≥2. A separate item states that at k=1k = 1k=1 the ratio is never attained.

Milestones

  1. Initially, the total weight of LEFT is at most ∣S∣/2k|S|/2^k∣S∣/2k.
  2. No iteration increases the total weight of LEFT, so at halting it is still at most ∣S∣/2k|S|/2^k∣S∣/2k.
  3. At halting, every clause left in LEFT has weight exactly 111.
  4. At halting, ∣LEFT∣≤∣S∣/2k|\mathrm{LEFT}| \le |S|/2^k∣LEFT∣≤∣S∣/2k and ∣SUB∣≥∣S∣(1−2−k)|\mathrm{SUB}| \ge |S|(1 - 2^{-k})∣SUB∣≥∣S∣(1−2−k).
  5. The eight-clause instance for k=3k = 3k=3 has S∗=8S^* = 8S∗=8 and admits a choosable output of size 777.

Significance

The bound 1−2−k1 - 2^{-k}1−2−k is exactly the expected fraction of clauses a uniformly random assignment satisfies on MS(k). B2 attains it deterministically, and the proof bounds ∣SUB∣|\mathrm{SUB}|∣SUB∣ against ∣S∣|S|∣S∣ rather than against S∗S^*S∗, a feature the paper points out. For k=3k = 3k=3 the constant 8/78/78/7 was later shown by Håstad (2001) to be optimal among polynomial-time algorithms unless P = NP, so the guarantee of this 1974 algorithm is best possible for MAX-E3-SAT.

Formalizing the result produces a machine-checked potential-function argument for a nondeterministic algorithm: the invariant links each clause's weight to how many of its literals have been removed. The mission also records a correction to the printed statement at k=1k = 1k=1. The paper's proof is complete; to our knowledge no machine-checked version exists.

Difficulty

The obvious argument follows the counting proof for algorithm B1 and compares clauses saved with clauses wounded in each step. It fails here: B2 can wound more clauses than it saves in a step, and the bound holds only in the weighted sense. The weight of a clause is not a static quantity. It is 2−∣C∣2^{-|C|}2−∣C∣ times 222 to the number of its literals already discarded, and this invariant must be carried through every step, including clauses that contain both yyy and yˉ\bar yyˉ​. Tightness needs an explicit input and an explicit adversarial run that exploits the tie in Step 4, and then a proof that every assignment of the remaining variables kills exactly one clause.

Formalization scope

  • Literals and clauses. A literal is a pair (variable index in N\mathbb NN, sign). Clauses are Finsets of literals, and an input is a Finset of clauses, so there are no duplicate clauses. Truth assignments are partial and consistent. S∗S^*S∗ is a maximum over the finite, nonempty family of satisfiable subsets.
  • B2 as a relation. B2 is a nondeterministic run relation, and "choosable" means reachable by finitely many steps from the initial state and halting. Step 3 allows a literal of either sign. The tie in Step 4 goes to yyy, and the comparison and doublings use the weights before the update. Weights are rationals.
  • Size-free ratios. The size-dependent R[B2,MS(k)](n)R[B2, \mathrm{MS}(k)](n)R[B2,MS(k)](n) is replaced by statements about every input (upper bound) and one attaining input (tightness). The two forms are equivalent because RRR is a maximum over finitely many inputs and nondecreasing in nnn.
  • No division. All ratios are multiplied out, so no division by zero can make a bound vacuous.
  • Trivializations ruled out. A deterministic tie-break in Step 3 or 4, or tightness at a single fixed kkk, would be a weaker theorem and is not acceptable. So is stating only the upper bound.

The definitions (the MS layer, the run relation) can be reused by other MAX-SAT approximation results, and an identical MS layer appears in the companion mission on algorithm B1. Contributions welcome: proofs of the milestones, of the k≥2k \ge 2k≥2 tightness family, and of the k=1k = 1k=1 correction.

Selected references

  • D. S. Johnson, Approximation algorithms for combinatorial problems, J. Comput. System Sci. 9 (1974) 256–278. https://doi.org/10.1016/S0022-0000(74)80044-9
  • J. Chen, D. K. Friesen, H. Zheng, Tight bound on Johnson's algorithm for maximum satisfiability, J. Comput. System Sci. 58 (1999) 622–640. https://doi.org/10.1006/jcss.1998.1610
  • M. X. Goemans, D. P. Williamson, New 3/4-approximation algorithms for the maximum satisfiability problem, SIAM J. Discrete Math. 7 (1994) 656–666. https://doi.org/10.1137/S0895480192243516
  • J. Håstad, Some optimal inapproximability results, J. ACM 48 (2001) 798–859. https://doi.org/10.1145/502090.502098
9 thms2 active usersReviewed
🏆Completed
CombinatoricsGraph TheoryOperations Research+1·Captain: mikedeng1

An Analysis of Several Heuristics for the Traveling Salesman Problem III: Nearest and Cheapest Insertion Are Within a Factor of TwoResearch Paper

Motivation

The traveling salesman problem (TSP) asks for a shortest closed route through a finite set of points. It is NP-hard, and in practice tours are built by fast constructive heuristics whose output is then improved or used as is. A central question in the analysis of algorithms, raised in this form by Rosenkrantz, Stearns and Lewis in 1977, is how far such a heuristic can be from optimal in the worst case, as a function of the number of points nnn, when the distances satisfy the triangle inequality.

The paper (SIAM J. Comput. 6(3), 1977) answers this for several heuristics. For the general class of insertion methods it proves a logarithmic bound (Theorem 3); for two specific rules, nearest insertion and cheapest insertion, it proves a bound that does not grow with nnn: the tour is less than twice the optimal (Theorem 4), and more precisely at most 2(1−1/n)2(1-1/n)2(1−1/n) times the optimal (Corollary, eq. (4.12)). Theorem 5 of the same paper shows the constant 2(1−1/n)2(1-1/n)2(1−1/n) is attained, so this is the exact worst case of both rules. These results, with Christofides' 3/2 bound of 1976, are the classical reference points for approximation ratios of TSP construction heuristics and appear in standard OR and approximation-algorithm texts.

Setting

A traveling salesman graph (N,d)(N,d)(N,d) has a finite node set NNN with ∣N∣=n|N| = n∣N∣=n and a distance d:N×N→Rd : N\times N\to\mathbb Rd:N×N→R that is symmetric, nonnegative and satisfies the triangle inequality d(i,k)≤d(i,j)+d(j,k)d(i,k)\le d(i,j)+d(j,k)d(i,k)≤d(i,j)+d(j,k). A tour is a circuit visiting every node exactly once; its length is the sum of its edge lengths, and OPTIMAL is the least tour length.

A subtour TTT is a tour on a subset of NNN (a one-node subtour has no edges). For k∉Tk\notin Tk∈/T, TOUR(T,k)\mathrm{TOUR}(T,k)TOUR(T,k) inserts kkk into TTT where it is cheapest: if TTT has at least two nodes, choose an edge (x,y)(x,y)(x,y) of TTT minimizing d(x,k)+d(k,y)−d(x,y)d(x,k)+d(k,y)-d(x,y)d(x,k)+d(k,y)−d(x,y) and replace it by (x,k),(k,y)(x,k),(k,y)(x,k),(k,y); if T={i}T=\{i\}T={i}, form the two-node tour on i,ki,ki,k. COST(T,k)\mathrm{COST}(T,k)COST(T,k) is the length of TOUR(T,k)\mathrm{TOUR}(T,k)TOUR(T,k) minus the length of TTT.

An insertion method builds subtours T1,…,TnT_1,\dots,T_nT1​,…,Tn​ with T1={a0}T_1=\{a_0\}T1​={a0​} and Ti+1=TOUR(Ti,ai)T_{i+1}=\mathrm{TOUR}(T_i,a_i)Ti+1​=TOUR(Ti​,ai​) for some ai∉Tia_i\notin T_iai​∈/Ti​, 1≤i<n1\le i<n1≤i<n; INSERT is the length of TnT_nTn​. With d(T,p)=min⁡x∈Td(x,p)d(T,p)=\min_{x\in T}d(x,p)d(T,p)=minx∈T​d(x,p):

  • nearest insertion chooses each aia_iai​ with d(Ti,ai)=min⁡{d(Ti,x):x∈N−Ti}d(T_i,a_i)=\min\{d(T_i,x): x\in N-T_i\}d(Ti​,ai​)=min{d(Ti​,x):x∈N−Ti​};
  • cheapest insertion chooses each aia_iai​ with COST(Ti,ai)=min⁡{COST(Ti,x):x∈N−Ti}\mathrm{COST}(T_i,a_i)=\min\{\mathrm{COST}(T_i,x): x\in N-T_i\}COST(Ti​,ai​)=min{COST(Ti​,x):x∈N−Ti​}.

The start node a0a_0a0​ and every tie (between candidate nodes, and between candidate edges) are arbitrary. TREE denotes the length of a minimal spanning tree of (N,d)(N,d)(N,d).

Formalization targets

Goal: Corollary to Theorem 4, eq. (4.12)

For every traveling salesman graph on n≥1n\ge1n≥1 nodes and every run of nearest insertion or of cheapest insertion,

INSERT  ≤  2(1−1n)⋅OPTIMAL.\mathrm{INSERT}\;\le\;2\Bigl(1-\frac1n\Bigr)\cdot\mathrm{OPTIMAL}.INSERT≤2(1−n1​)⋅OPTIMAL.

Milestones

  1. Lemma 2, (3.3): COST(T,k)≤2 d(k,j)\mathrm{COST}(T,k)\le 2\,d(k,j)COST(T,k)≤2d(k,j) for k∉Tk\notin Tk∈/T, j∈Tj\in Tj∈T.
  2. Eq. (3.7): for every insertion method, INSERT=∑i=1n−1COST(Ti,ai)\mathrm{INSERT}=\sum_{i=1}^{n-1}\mathrm{COST}(T_i,a_i)INSERT=∑i=1n−1​COST(Ti​,ai​).
  3. Eqs. (4.9)–(4.10): nearest insertion satisfies COST(Ti,ai)≤2 d(p,q)\mathrm{COST}(T_i,a_i)\le 2\,d(p,q)COST(Ti​,ai​)≤2d(p,q) for all p∈Tip\in T_ip∈Ti​, q∉Tiq\notin T_iq∈/Ti​ (4.5).
  4. Proof of Theorem 4: cheapest insertion satisfies (4.5) as well.
  5. Lemma 3: every insertion run satisfying (4.5) has INSERT≤2⋅TREE\mathrm{INSERT}\le 2\cdot\mathrm{TREE}INSERT≤2⋅TREE (4.6).
  6. Eq. (4.11): TREE≤(1−1/n)⋅OPTIMAL\mathrm{TREE}\le(1-1/n)\cdot\mathrm{OPTIMAL}TREE≤(1−1/n)⋅OPTIMAL.

Theorem 4 itself, INSERT<2⋅OPTIMAL\mathrm{INSERT}<2\cdot\mathrm{OPTIMAL}INSERT<2⋅OPTIMAL when ddd is not identically zero, is included as a companion statement.

Significance

The bound says that two simple O(n2)O(n^2)O(n2) and O(n2log⁡n)O(n^2\log n)O(n2logn) construction rules are never worse than a factor 2(1−1/n)2(1-1/n)2(1−1/n) from optimal on any metric instance, a guarantee independent of nnn, in contrast with nearest neighbor and with arbitrary insertion orders, whose ratios the same paper shows can grow logarithmically. Lemma 3 is reusable on its own: any insertion rule satisfying the local inequality (4.5) inherits the bound 2⋅TREE2\cdot\mathrm{TREE}2⋅TREE, and the paper notes that similar arguments apply to nearest addition and nearest merger.

The result has been proved since 1977. The Prove2Me library has a machine-checked proof of the weaker statement for nearest insertion only with constant 222 (SupplyChainTheory.nearest_insertion_bound, from Snyder–Shen, Theorem 10.7) and of TREE≤OPTIMAL\mathrm{TREE}\le\mathrm{OPTIMAL}TREE≤OPTIMAL (SupplyChainTheory.mst_lower_bound). This mission asks for the paper's full statement: both rules, the exact constant 2(1−1/n)2(1-1/n)2(1−1/n), and the general Lemma 3 via its correspondence between insertion steps and spanning-tree edges. Paired with the tightness result of the companion mission (Theorem 5), it would give a formally verified exact worst-case ratio for both heuristics.

Difficulty

Lemma 2 and inequality (4.5) are local consequences of the triangle inequality; the difficulty is global. The obvious attempt at Lemma 3 charges step iii to the tree edge joining aia_iai​ to its nearest node of TiT_iTi​, but distinct steps can then be charged to the same tree edge, and the sum of the charges no longer bounds 2⋅TREE2\cdot\mathrm{TREE}2⋅TREE. Any correct argument must control how the insertion order interacts with the structure of an arbitrary spanning tree, which in Lean means reasoning about paths in SimpleGraph together with the evolving subtours. For cheapest insertion the chosen node need not be a nearest node, so (4.5) is not immediate from the rule. Finally, the goal's constant 2(1−1/n)2(1-1/n)2(1−1/n) is sharper than the bound 2⋅OPTIMAL2\cdot\mathrm{OPTIMAL}2⋅OPTIMAL obtained from TREE≤OPTIMAL\mathrm{TREE}\le\mathrm{OPTIMAL}TREE≤OPTIMAL, so the weaker spanning-tree bound already in the library does not suffice.

Formalization scope

Nodes are Fin n with n≥1n\ge1n≥1 (the paper's nodes 1,…,n1,\dots,n1,…,n shifted to 0,…,n−10,\dots,n-10,…,n−1). The distance satisfies the paper's three axioms plus the normalization d(i,i)=0d(i,i)=0d(i,i)=0, which never affects a tour, subtour or tree length. Tours are permutations; OPTIMAL is a minimum over all of them (Finset.inf'). Subtours are lists of distinct nodes with closed length. TOUR(T,k)\mathrm{TOUR}(T,k)TOUR(T,k) is encoded as insertion of kkk at a list position whose resulting length is minimal over all ∣T∣+1|T|+1∣T∣+1 positions, which is the minimization of (3.1) over the edges of TTT; COST is the corresponding minimum increase. The subtour index is 1-based as printed (T1={a0}T_1=\{a_0\}T1​={a0​}, TnT_nTn​ final). The distance d(T,p)d(T,p)d(T,p) is taken in R∪{+∞}\mathbb R\cup\{+\infty\}R∪{+∞}, so no default value enters the nearest rule. Spanning trees are SimpleGraph (Fin n) with IsTree; statements about TREE are phrased over every spanning tree (upper bounds) or some spanning tree (bounds on TREE), which is equivalent. Ratios are multiplied out, so the goal needs no nontriviality hypothesis; Theorem 4's strict form carries the paper's exclusion of the identically zero distance (p. 564).

A formalization in which TOUR inserts at an arbitrary rather than a cheapest position, or in which the run fixes the start node or the tie-breaking, would state a different (and, for arbitrary positions, false) theorem; the statements here quantify over every run.

A complete development needs subtour-length lemmas for List.insertIdx, the telescoping identity (3.7), and a spanning-tree edge-assignment argument on Mathlib's SimpleGraph paths; the last two are reusable for other insertion rules and for the companion missions of this series. Proofs of any milestone are welcome independently.

Selected references

  • D. J. Rosenkrantz, R. E. Stearns, P. M. Lewis II, An Analysis of Several Heuristics for the Traveling Salesman Problem, SIAM Journal on Computing 6(3):563–581, 1977. https://doi.org/10.1137/0206041
  • N. Christofides, Worst-Case Analysis of a New Heuristic for the Travelling Salesman Problem, Report 388, GSIA, Carnegie Mellon University, 1976. https://doi.org/10.1007/s43069-021-00101-z (reprint in Operations Research Forum 3, 2022)
  • L. V. Snyder, Z.-J. M. Shen, Fundamentals of Supply Chain Theory, 2nd ed., Wiley, 2019, Chapter 10 (Theorem 10.7). https://doi.org/10.1002/9781119584445
10 thms2 active usersReviewed
🏆Completed
Control TheoryOperations Research·Captain: mikedeng1

Supervisory Control of a Class of Discrete Event Processes II: Every Reduced, Trim Supervisor Is the Quotient of a Supervisor Built on a Recognizer for Its Closed-Loop LanguageResearch Paper

Motivation

Supervisory control theory, introduced by Ramadge and Wonham (SIAM J. Control Optim. 25(1), 1987), models a manufacturing cell, a communication protocol or a resource-sharing system as an automaton whose transitions are events, some of which an external controller may disable. A controller, the supervisor, is itself an automaton that watches the event sequence and, in each of its states, decides which controllable events are currently allowed. The framework is the standard model for logical (untimed) control of discrete event systems and is the subject of textbooks such as Cassandras and Lafortune (Springer, 2008) and Wonham and Cai (Springer, 2019).

A practical concern is supervisor size. The synthesis procedure of the paper (§9) produces a supervisor whose automaton records exactly as much of the past as the desired closed-loop language requires, but there are many other supervisors realising the same behaviour. The paper's second main result, the quotient structure theorem (Theorem 10.1, p. 223), explains how they are related: any supervisor with two natural economy properties is obtained from a canonical one, built on a recognizer of the closed-loop language, by lumping states. This is the structural starting point of the later literature on supervisor reduction.

Setting

A generator is G=(Q,Σ,δ,q0,Qm)\mathcal G = (Q, \Sigma, \delta, q_0, Q_m)G=(Q,Σ,δ,q0​,Qm​): a state set QQQ, a finite alphabet Σ\SigmaΣ of events, a partial transition function δ:Σ×Q→Q\delta : \Sigma \times Q \to Qδ:Σ×Q→Q, an initial state q0q_0q0​ and marker states Qm⊆QQ_m \subseteq QQm​⊆Q. Extending δ\deltaδ to strings gives the generated language L(G)L(\mathcal G)L(G) (strings along which δ\deltaδ is defined from q0q_0q0​) and the marked language Lm(G)L_m(\mathcal G)Lm​(G) (those that end in QmQ_mQm​). The closure Kˉ\bar KKˉ of a language KKK is its set of prefixes. Throughout, G\mathcal GG is trim: L(G)=Lˉm(G)L(\mathcal G) = \bar L_m(\mathcal G)L(G)=Lˉm​(G). A recognizer for a language KKK is an accessible generator whose marked language is KKK.

A subset Σc⊆Σ\Sigma_c \subseteq \SigmaΣc​⊆Σ of events is controllable. A supervisor is S=(S,ϕ)\mathcal S = (S, \phi)S=(S,ϕ) where S=(X,Σ,ξ,x0,Xm)S = (X, \Sigma, \xi, x_0, X_m)S=(X,Σ,ξ,x0​,Xm​) is an accessible deterministic automaton with possibly infinite state set and ϕ:X→{0,1}Σc\phi : X \to \{0,1\}^{\Sigma_c}ϕ:X→{0,1}Σc​ is a state feedback map; events outside Σc\Sigma_cΣc​ are always enabled. The closed loop S/G\mathcal S/\mathcal GS/G runs SSS and G\mathcal GG in lockstep and allows σ\sigmaσ from (x,q)(x, q)(x,q) iff ξ(σ,x)\xi(\sigma, x)ξ(σ,x) and δ(σ,q)\delta(\sigma, q)δ(σ,q) are defined and ϕ(x)(σ)=1\phi(x)(\sigma) = 1ϕ(x)(σ)=1. Its languages are L(S/G)L(\mathcal S/\mathcal G)L(S/G), Lm(S/G)L_m(\mathcal S/\mathcal G)Lm​(S/G) (marker set Xm×QmX_m \times Q_mXm​×Qm​) and Lc(S/G)=L(S/G)∩Lm(G)L_c(\mathcal S/\mathcal G) = L(\mathcal S/\mathcal G) \cap L_m(\mathcal G)Lc​(S/G)=L(S/G)∩Lm​(G).

S\mathcal SS is complete if SSS never refuses an event that the plant can execute and ϕ\phiϕ enables: s∈L(S/G)s \in L(\mathcal S/\mathcal G)s∈L(S/G), sσ∈L(G)s\sigma \in L(\mathcal G)sσ∈L(G) and ϕ(ξ(s,x0))(σ)=1\phi(\xi(s, x_0))(\sigma) = 1ϕ(ξ(s,x0​))(σ)=1 imply sσ∈L(S/G)s\sigma \in L(\mathcal S/\mathcal G)sσ∈L(S/G). It is proper if it is complete and Lˉm(S/G)=Lˉc(S/G)=L(S/G)\bar L_m(\mathcal S/\mathcal G) = \bar L_c(\mathcal S/\mathcal G) = L(\mathcal S/\mathcal G)Lˉm​(S/G)=Lˉc​(S/G)=L(S/G).

A projection π:S→S^\pi : \mathcal S \to \hat{\mathcal S}π:S→S^ is a surjection X→X^X \to \hat XX→X^ with π(x0)=x^0\pi(x_0) = \hat x_0π(x0​)=x^0​, Xm=π−1(X^m)X_m = \pi^{-1}(\hat X_m)Xm​=π−1(X^m​), ξ^(σ,π(x))=π(ξ(σ,x))\hat\xi(\sigma, \pi(x)) = \pi(\xi(\sigma, x))ξ^​(σ,π(x))=π(ξ(σ,x)) wherever ξ(σ,x)\xi(\sigma, x)ξ(σ,x) is defined, and ϕ^∘π=ϕ\hat\phi \circ \pi = \phiϕ^​∘π=ϕ.

For a language KKK, strings s,s′s, s's,s′ are Kˉ\bar KKˉ-equivalent if st∈Kˉ  ⟺  s′t∈Kˉst \in \bar K \iff s't \in \bar Kst∈Kˉ⟺s′t∈Kˉ for all ttt. The automaton SSS is Kˉ\bar KKˉ-reduced if Kˉ\bar KKˉ-equivalent strings of Kˉ\bar KKˉ lead to the same state, and Kˉ\bar KKˉ-trim if every state is reached by a string of Kˉ\bar KKˉ.

Formalization targets

Goal: Theorem 10.1 (quotient structure theorem)

Let S=(S,ϕ)\mathcal S = (S, \phi)S=(S,ϕ) be complete, K1:=Lm(S/G)K_1 := L_m(\mathcal S/\mathcal G)K1​:=Lm​(S/G), K3:=L(S/G)K_3 := L(\mathcal S/\mathcal G)K3​:=L(S/G), with SSS K3K_3K3​-reduced and K3K_3K3​-trim, and let S^0=(X0,Σ,ξ0,x00,X0)\hat S^0 = (X^0, \Sigma, \xi^0, x^0_0, X^0)S^0=(X0,Σ,ξ0,x00​,X0) be a trim recognizer for K3K_3K3​. Then there are Xm0⊆X0X^0_m \subseteq X^0Xm0​⊆X0 and ϕ0\phi^0ϕ0 such that S0=((X0,Σ,ξ0,x00,Xm0),ϕ0)\mathcal S^0 = ((X^0, \Sigma, \xi^0, x^0_0, X^0_m), \phi^0)S0=((X0,Σ,ξ0,x00​,Xm0​),ϕ0) satisfies

S0 complete,Lm(S0/G)=K1,L(S0/G)=K3,∃ π:S0→S,S proper⇒S0 proper.\mathcal S^0 \text{ complete},\quad L_m(\mathcal S^0/\mathcal G) = K_1,\quad L(\mathcal S^0/\mathcal G) = K_3,\quad \exists\, \pi : \mathcal S^0 \to \mathcal S,\quad \mathcal S \text{ proper} \Rightarrow \mathcal S^0 \text{ proper}.S0 complete,Lm​(S0/G)=K1​,L(S0/G)=K3​,∃π:S0→S,S proper⇒S0 proper.

Milestones

Proposition 8.1 (p. 219): for complete S\mathcal SS and a projection π:S→S^\pi : \mathcal S \to \hat{\mathcal S}π:S→S^, (i) π\piπ is unique; (ii) (Lm,Lc,L)(S/G)=(Lm,Lc,L)(S^/G)(L_m, L_c, L)(\mathcal S/\mathcal G) = (L_m, L_c, L)(\hat{\mathcal S}/\mathcal G)(Lm​,Lc​,L)(S/G)=(Lm​,Lc​,L)(S^/G); (iii) S^\hat{\mathcal S}S^ is complete; (iv) nonblocking, nonrejecting and proper transfer in both directions.

The displayed steps of the proof of Theorem 10.1 (pp. 223–224): π(ξ0(s,x00)):=ξ(s,x0)\pi(\xi^0(s, x^0_0)) := \xi(s, x_0)π(ξ0(s,x00​)):=ξ(s,x0​) is well defined on X0X^0X0; it is a projection once Xm0:=π−1(Xm)X^0_m := \pi^{-1}(X_m)Xm0​:=π−1(Xm​) and ϕ0:=ϕ∘π\phi^0 := \phi \circ \piϕ0:=ϕ∘π; L(S0/G)=K3L(\mathcal S^0/\mathcal G) = K_3L(S0/G)=K3​ follows from two enablement conditions; S0\mathcal S^0S0 is complete; and Lm(S0/G)=K1L_m(\mathcal S^0/\mathcal G) = K_1Lm​(S0/G)=K1​.

Significance

Theorem 10.1 says that the reduction properties the synthesis procedure of §9 guarantees are exactly what makes a supervisor a quotient of the canonical supervisor on a recognizer of K3K_3K3​. Combined with Proposition 8.1, which shows that a projection preserves every closed-loop language, completeness and properness, it identifies supervisors with the same behaviour up to state lumping. This is the basis on which supervisor reduction and the comparison of supervisor realisations rest.

The result is proved in the paper. What this mission produces is a machine-checked version of the model (generators with partial transitions, supervisors with infinite state sets, the closed loop, completeness and properness) and of the two results above. No formalization of Ramadge–Wonham supervisory control was found on Prove2Me at the time of drafting; the definitions of this mission are reusable by any later mission on the theory, including the synthesis results of the first mission of the series.

Difficulty

The mathematics is elementary; the difficulty is bookkeeping with partial functions. Every step compares runs of three automata (the plant, SSS and the recognizer) that may be undefined at different strings, and a proof must track at each string which of them is defined. The tempting shortcut of treating the projection condition as full commutation, ξ^(σ,π(x))=π(ξ(σ,x))\hat\xi(\sigma, \pi(x)) = \pi(\xi(\sigma, x))ξ^​(σ,π(x))=π(ξ(σ,x)) for all σ,x\sigma, xσ,x, is not available: the page asks for it only where ξ(σ,x)\xi(\sigma, x)ξ(σ,x) is defined, and Proposition 8.1 (ii) holds only because completeness of S\mathcal SS compensates for transitions that ξ^\hat\xiξ^​ has and ξ\xiξ lacks. Likewise, the map π\piπ of Theorem 10.1 is defined through arbitrary representatives, and its well-definedness uses both that S^0\hat S^0S^0 recognizes K3K_3K3​ with all states marked and that SSS is K3K_3K3​-reduced.

Formalization scope

The alphabet is a Lean type α with [Fintype α] (the page's Σ\SigmaΣ, which is Lean syntax); strings are List α, sσs\sigmasσ is s ++ [σ]. A generator is a structure with its own state type in Type and a partial transition δ : α → Q → Option Q; its extended transition is the left fold. The feedback map has type X → Ec → Bool, and "σ\sigmaσ enabled at xxx" means σ∉Σc\sigma \notin \Sigma_cσ∈/Σc​ or ϕ(x)(σ)=1\phi(x)(\sigma) = 1ϕ(x)(σ)=1; the page's ϕ:X→{0,1}Σ\phi : X \to \{0,1\}^\Sigmaϕ:X→{0,1}Σ is the same object under this extension. The closed loop is run from (x0,q0)(x_0, q_0)(x0​,q0​) without forming its accessible part, which changes none of its languages. Existential statements over supervisors quantify over state types in Type.

Standing assumptions that are hypotheses of every theorem: Σ\SigmaΣ finite; G\mathcal GG trim, stated as 𝒢.L = pre 𝒢.Lm; every supervisor automaton accessible. Theorem 10.1 and its proof steps also assume S\mathcal SS complete, SSS K3K_3K3​-reduced and K3K_3K3​-trim, and a trim recognizer RRR for K3K_3K3​ with all states marked, and S0\mathcal S^0S0 is built on that given RRR rather than on a recognizer chosen by the prover. The conclusion names K1K_1K1​ and K3K_3K3​ as the languages of S\mathcal SS, not as free variables. A projection predicate that drops π(x0)=x^0\pi(x_0) = \hat x_0π(x0​)=x^0​, surjectivity or the marker equation would make the goal's clause (ii) trivially satisfiable by a constant map; the sanity check shipped with the mission rules this out on the primitive plant of §2.3.

Contributions welcome: proofs of the milestones, lemmas relating the fold-based extended transition to concatenation, and a reusable library for closed-loop runs.

Selected references

  • P. J. Ramadge and W. M. Wonham, Supervisory Control of a Class of Discrete Event Processes, SIAM J. Control Optim. 25(1):206–230, 1987. https://doi.org/10.1137/0325013
  • C. G. Cassandras and S. Lafortune, Introduction to Discrete Event Systems, 2nd ed., Springer, 2008. https://doi.org/10.1007/978-0-387-68612-7
  • W. M. Wonham and K. Cai, Supervisory Control of Discrete-Event Systems, Springer, 2019. https://doi.org/10.1007/978-3-319-77452-7
15 thms2 active usersReviewed
🏆Completed
Combinatorics·Captain: mikedeng1

A Faster Algorithm Computing String Edit Distances 2: Discrete Edit Costs Are NecessaryResearch Paper

Motivation

The edit distance between two strings is the least total cost of a sequence of single-character insertions, deletions and replacements that turns one string into the other. It underlies spelling correction, sequence alignment in computational biology, and file comparison. Wagner and Fischer (J. ACM 21, 1974) computed it for strings of length nnn in time O(n2)O(n^2)O(n2) by filling an (n+1)×(n+1)(n+1) \times (n+1)(n+1)×(n+1) matrix. Masek and Paterson (J. Comput. System Sci. 20, 1980) lowered this to O(n2/log⁡n)O(n^2/\log n)O(n2/logn) with a "Four Russians" block method: the matrix is cut into m×mm \times mm×m blocks, and the effect of every possible block is tabulated in advance.

The tabulation only pays off if the number of possible blocks is small. The paper guarantees this under two hypotheses: the alphabet is finite, and the edit costs are discrete, that is, all integer multiples of one constant. Its §4 asks whether discreteness can be dropped, and answers no with an explicit example whose costs are 000, 111, π\piπ and 555. This mission formalizes that example.

Timeline:

  • 1974, Wagner and Fischer: the O(∣A∣ ∣B∣)O(|A|\,|B|)O(∣A∣∣B∣) matrix algorithm and its recurrence, for nonnegative costs.
  • 1980, Masek and Paterson: the O(n2/log⁡n)O(n^2/\log n)O(n2/logn) algorithm for a finite alphabet and discrete costs (§2, Lemma 4), and the example of §4 showing the discreteness hypothesis cannot simply be removed (Theorem 5).
  • 2015, Backurs and Indyk (STOC 2015): no strongly subquadratic algorithm under the Strong Exponential Time Hypothesis, which places the gap the paper left open in context.

Setting

Let Σ\SigmaΣ be an alphabet and λ\lambdaλ the null string. An edit operation a→ba \to ba→b is a pair of strings of length at most one other than (λ,λ)(\lambda, \lambda)(λ,λ): a replacement when both are symbols, a deletion when b=λb = \lambdab=λ, an insertion when a=λa = \lambdaa=λ. BBB results from AAA by a→ba \to ba→b if A=σaτA = \sigma a \tauA=σaτ and B=σbτB = \sigma b \tauB=σbτ. A cost function γ\gammaγ assigns a nonnegative real to each edit operation; Ra,b=γ(a→b)R_{a,b} = \gamma(a \to b)Ra,b​=γ(a→b), Da=γ(a→λ)D_a = \gamma(a \to \lambda)Da​=γ(a→λ), Ia=γ(λ→a)I_a = \gamma(\lambda \to a)Ia​=γ(λ→a). The edit distance δ(γ,A,B)\delta(\gamma, A, B)δ(γ,A,B) is the minimum of ∑iγ(si)\sum_i \gamma(s_i)∑i​γ(si​) over sequences s1,…,sms_1, \dots, s_ms1​,…,sm​ of edit operations taking AAA to BBB. For fixed strings, δi,j=δ(γ,Ai,Bj)\delta_{i,j} = \delta(\gamma, A^i, B^j)δi,j​=δ(γ,Ai,Bj), where Ai=A1⋯AiA^i = A_1 \cdots A_iAi=A1​⋯Ai​; this is the edit matrix. A step is the difference of two horizontally or vertically adjacent entries, δi,j−δi−1,j\delta_{i,j} - \delta_{i-1,j}δi,j​−δi−1,j​ or δi,j−δi,j−1\delta_{i,j} - \delta_{i,j-1}δi,j​−δi,j−1​, and the possible steps of γ\gammaγ are the steps of all edit matrices of all pairs of strings. The cost set Ω={Da}∪{Ia}∪{Ra,b}\Omega = \{D_a\} \cup \{I_a\} \cup \{R_{a,b}\}Ω={Da​}∪{Ia​}∪{Ra,b​} is discrete if some r>0r > 0r>0 has every element of Ω\OmegaΩ as an integer multiple.

An edit path is a sequence of matrix cells (p,q)(p, q)(p,q) in which each cell increases ppp, qqq or both by one: a deletion of Ap+1A_{p+1}Ap+1​ (cost DAp+1D_{A_{p+1}}DAp+1​​), an insertion of Bq+1B_{q+1}Bq+1​ (cost IBq+1I_{B_{q+1}}IBq+1​​), or a replacement of Ap+1A_{p+1}Ap+1​ by Bq+1B_{q+1}Bq+1​ (cost RAp+1,Bq+1R_{A_{p+1},B_{q+1}}RAp+1​,Bq+1​​). The eccentricity of (i,j)(i, j)(i,j) is ∣i−j∣|i - j|∣i−j∣.

The example. Σ={a,b,c}\Sigma = \{a, b, c\}Σ={a,b,c} with

Rσ,σ=0,Ra,b=Rb,a=1,Rc,a=Rc,b=Ra,c=Rb,c=π,Iσ=Dσ=5.R_{\sigma,\sigma} = 0,\quad R_{a,b} = R_{b,a} = 1,\quad R_{c,a} = R_{c,b} = R_{a,c} = R_{b,c} = \pi,\quad I_\sigma = D_\sigma = 5.Rσ,σ​=0,Ra,b​=Rb,a​=1,Rc,a​=Rc,b​=Ra,c​=Rb,c​=π,Iσ​=Dσ​=5.

Let μ2k=μ2k+1=⌊2k/(2π+1)⌋\mu_{2k} = \mu_{2k+1} = \lfloor 2k/(2\pi+1) \rfloorμ2k​=μ2k+1​=⌊2k/(2π+1)⌋. The infinite strings AAA and BBB are baba…baba\ldotsbaba… and abab…abab\ldotsabab… with a ccc written into both, at each even position iii where μi>μi−1\mu_i > \mu_{i-1}μi​>μi−1​, so that AiA^iAi and BiB^iBi each contain exactly μi\mu_iμi​ letters ccc. The first ccc is at position 888. P∗(i,j,k)P^*(i, j, k)P∗(i,j,k) is the minimum cost of an edit path from (i,j)(i, j)(i,j) to (i+k,j+k)(i+k, j+k)(i+k,j+k) through points all of eccentricity at least ∣i−j∣|i-j|∣i−j∣.

Formalization targets

Goal (Theorem 5, as its proof establishes it)

For the example's γ\gammaγ, AAA and BBB,

k↦δk,k+1−δk,k  is injective on N,and the set of possible steps of γ is infinite.k \mapsto \delta_{k,k+1} - \delta_{k,k} \ \text{ is injective on } \mathbb{N}, \qquad \text{and the set of possible steps of } \gamma \text{ is infinite.}k↦δk,k+1​−δk,k​  is injective on N,and the set of possible steps of γ is infinite.

The first part gives at least nnn distinct steps in the edit matrix of AnA^nAn and BnB^nBn; the second is the negation of the conclusion of the paper's Lemma 4. The goal fixes no constants and no growth rate beyond "at least one new step per diagonal position".

Milestones

  1. §2.3: for every nonnegative normalized cost function and all strings, δi,j\delta_{i,j}δi,j​ equals the minimum cost of an edit path from (0,0)(0,0)(0,0) to (i,j)(i,j)(i,j).
  2. Lemma 5: P∗(i,j,k)≥k−μi+k+μiP^*(i,j,k) \ge k - \mu_{i+k} + \mu_iP∗(i,j,k)≥k−μi+k​+μi​ if i−ji - ji−j is even, and P∗(i,j,k)≥(μi+k−μi+μj+k−μj)πP^*(i,j,k) \ge (\mu_{i+k} - \mu_i + \mu_{j+k} - \mu_j)\piP∗(i,j,k)≥(μi+k​−μi​+μj+k​−μj​)π if i−ji - ji−j is odd.
  3. Lemma 6: for 0≤k′≤k0 \le k' \le k0≤k′≤k, P∗(0,0,k′)+5+P∗(k′+1,k′,k−k′)≥5+(μk+1+μk)πP^*(0,0,k') + 5 + P^*(k'+1, k', k-k') \ge 5 + (\mu_{k+1} + \mu_k)\piP∗(0,0,k′)+5+P∗(k′+1,k′,k−k′)≥5+(μk+1​+μk​)π.
  4. Lemma 7: δk,k=k−μk\delta_{k,k} = k - \mu_kδk,k​=k−μk​ and δk,k+1=δk+1,k=5+(μk+1+μk)π\delta_{k,k+1} = \delta_{k+1,k} = 5 + (\mu_{k+1} + \mu_k)\piδk,k+1​=δk+1,k​=5+(μk+1​+μk​)π.

A further item records that the example satisfies every other condition of the paper: its costs are nonnegative and normalized (γ(a→b)=δ(γ,a,b)\gamma(a \to b) = \delta(\gamma, a, b)γ(a→b)=δ(γ,a,b)), but Ω={0,1,π,5}\Omega = \{0, 1, \pi, 5\}Ω={0,1,π,5} is not discrete.

Significance

The block algorithm precomputes one table entry per block and per pair of initial step vectors, so its preprocessing is polynomial in nnn only when the number of possible steps is bounded independently of the strings. The example shows that without discreteness the steps can grow with nnn even over a three-letter alphabet with nonnegative normalized costs, so the table size becomes of order (kn)m(kn)^m(kn)m and the method gives no speedup. It explains why the finite-alphabet, non-discrete case is left open in the paper's conclusion.

The paper's result is proved on paper; no machine-checked version is known to exist, and Mathlib has no edit distance at the pinned revision. The formalization produces exact closed forms for three diagonals of a nontrivial edit matrix with irrational entries, a formal link between edit distance over arbitrary edit sequences and minimum-cost paths, and a verified counterexample to the naive generalization of the algorithm. The sibling mission of the series formalizes the algorithm and Lemma 4.

Difficulty

The central difficulty is Lemma 5: a lower bound on the cost of every path confined to a band of eccentricity, not only the straight diagonal. A path may leave its diagonal, pay 101010 for a deletion and an insertion, and travel along another diagonal whose ccc's may or may not line up. The bound must hold uniformly in iii, jjj and kkk, and it depends on the floor function μ\muμ and on precise inequalities between μr+s−μr\mu_{r+s} - \mu_rμr+s​−μr​ and s/(2π+1)s/(2\pi+1)s/(2π+1). Checking that the straight diagonals are optimal for small kkk does not suffice: the ccc-densities are chosen so that even and odd diagonals cost almost exactly the same per step, and a periodic placement of ccc's would let one diagonal eventually undercut another.

Formalization scope

Strings are Lists; the infinite strings AAA, BBB are functions N→Σ\mathbb{N} \to \SigmaN→Σ read from index 111, and AnA^nAn is the list of their first nnn symbols. An edit operation is a pair of Option values other than (none,none)(\text{none}, \text{none})(none,none). The edit distance and P∗P^*P∗ are real infima (sInf) of nonempty sets of nonnegative reals, so they coincide with the paper's minima. δi,j\delta_{i,j}δi,j​ has iii indexing AAA and jjj indexing BBB (the paper's Figure 4 prints AAA across the columns). μi\mu_iμi​ is ⌊2⌊i/2⌋/(2π+1)⌋\lfloor 2\lfloor i/2 \rfloor/(2\pi+1) \rfloor⌊2⌊i/2⌋/(2π+1)⌋. The constraint of P∗P^*P∗ applies to every point of the path, endpoints included. Costs use Real.pi itself.

Pinned statements: the paper states Theorem 5 about the running time of Algorithm Y ("Discreteness is a necessary condition for Algorithm Y to run in time O(km)O(k^m)O(km) on length mmm strings and step sequences"); its proof establishes that the number of distinct steps grows linearly with the string length, which is what the goal states. Running time is not formalized. The §2.3 milestone is stated, as in the paper, for all strings and every nonnegative cost function satisfying the §1.1 normalization γ(a→b)=δ(γ,a,b)\gamma(a \to b) = \delta(\gamma, a, b)γ(a→b)=δ(γ,a,b); both standing assumptions are hypotheses. The paper's standing assumption ∣A∣≥∣B∣|A| \ge |B|∣A∣≥∣B∣ is used only for running times and is omitted.

The example must be the paper's: replacing π\piπ by a rational, or quantifying over "some" cost function or "some" strings, makes the goal false or empty, and the edit distance must be the minimum over edit sequences, not a recurrence.

Needed infrastructure: edit sequences and their costs, the reduction of edit sequences to edit paths, and bounds on ⌊⋅⌋\lfloor \cdot \rfloor⌊⋅⌋ with π\piπ (Mathlib's irrational_pi and Real.pi_gt_d2). The edit-distance definitions are shared in shape with the sibling mission and are reusable. Proofs of any milestone are welcome.

Selected references

  • W. J. Masek, M. S. Paterson, A Faster Algorithm Computing String Edit Distances, J. Comput. System Sci. 20 (1980), 18–31. https://doi.org/10.1016/0022-0000(80)90002-1
  • R. A. Wagner, M. J. Fischer, The String-to-String Correction Problem, J. ACM 21 (1974), 168–173. https://doi.org/10.1145/321796.321811
  • A. Backurs, P. Indyk, Edit Distance Cannot Be Computed in Strongly Subquadratic Time (unless SETH is false), STOC 2015, 51–58. https://doi.org/10.1145/2746539.2746612
9 thms2 active usersReviewed
🏆Completed
CombinatoricsOperations ResearchOptimization·Captain: mikedeng1

Maximizing Non-Monotone Submodular Functions III: Guarantees of Deterministic Local SearchResearch Paper

Motivation

Many optimization problems ask for a subset of a finite ground set that maximizes a function with diminishing returns: the cut of a graph or a directed graph, the value of a facility-location configuration, the entropy of a set of random variables, or a welfare function in combinatorial auctions. These functions are submodular but typically non-monotone: adding elements can decrease the value. Max Cut and Max Directed Cut are the textbook special cases. Unconstrained maximization of a nonnegative non-monotone submodular function is NP-hard, and before Feige, Mirrokni and Vondrák (SIAM J. Comput. 40(4), 2011) no constant-factor approximation was known for general such functions in the value-oracle model.

The paper gives several algorithms. A uniformly random set achieves 1/41/41/4 of the optimum (a separate mission of this series). This mission concerns the paper's first deterministic algorithm: a local search that repeatedly adds or removes a single element while the value improves by a factor larger than 1+ϵ/n21 + \epsilon/n^21+ϵ/n2, and returns the better of the final set and its complement. The key structural fact behind it, that a local optimum of a submodular function dominates all its subsets and supersets, goes back to Cherenin (1962) and Goldengorin, Tijssen and Tso (1999).

Timeline. Cherenin (1962) and Goldengorin–Tijssen–Tso (1999): local optima dominate comparable sets. Schäffer and Yannakakis (SIAM J. Comput. 1991): finding an exact local optimum of Max Cut is PLS-complete, which is why the algorithm here uses an approximate improvement threshold. Feige–Mirrokni–Vondrák (FOCS 2007; SIAM J. Comput. 2011): the 1/31/31/3 and 1/21/21/2 guarantees of this mission, and 2/52/52/5 for a randomized "smooth" local search. Buchbinder, Feldman, Naor and Schwartz (SIAM J. Comput. 2015): a randomized double-greedy 1/21/21/2-approximation, which is optimal in the value-oracle model by the lower bound of the same 2011 paper.

Setting

Let XXX be a finite ground set with n=∣X∣n = |X|n=∣X∣ elements. A set function assigns a real number f(S)f(S)f(S) to every S⊆XS \subseteq XS⊆X. It is submodular (Definition 1.1) if

f(S∪T)+f(S∩T)≤f(S)+f(T)for all S,T⊆X,f(S \cup T) + f(S \cap T) \le f(S) + f(T) \qquad \text{for all } S, T \subseteq X,f(S∪T)+f(S∩T)≤f(S)+f(T)for all S,T⊆X,

and symmetric if f(X∖S)=f(S)f(X \setminus S) = f(S)f(X∖S)=f(S) for all SSS. Throughout the section of the paper formalized here, fff is nonnegative. The algorithm may query f(S)f(S)f(S) for any SSS (a value oracle). The optimum is OPT=max⁡S⊆Xf(S)\mathrm{OPT} = \max_{S \subseteq X} f(S)OPT=maxS⊆X​f(S).

A set SSS is a local optimum if f(S∪{a})≤f(S)f(S \cup \{a\}) \le f(S)f(S∪{a})≤f(S) for every a∉Sa \notin Sa∈/S and f(S∖{a})≤f(S)f(S \setminus \{a\}) \le f(S)f(S∖{a})≤f(S) for every a∈Sa \in Sa∈S. It is a (1+α)(1+\alpha)(1+α)-approximate local optimum (Definition 3.2) if (1+α)f(S)≥f(S∖{v})(1+\alpha)f(S) \ge f(S \setminus \{v\})(1+α)f(S)≥f(S∖{v}) for v∈Sv \in Sv∈S and (1+α)f(S)≥f(S∪{v})(1+\alpha)f(S) \ge f(S \cup \{v\})(1+α)f(S)≥f(S∪{v}) for v∉Sv \notin Sv∈/S.

Algorithm LS with parameter ϵ>0\epsilon > 0ϵ>0 and c=1+ϵ/n2c = 1 + \epsilon/n^2c=1+ϵ/n2:

  1. Let S:={v}S := \{v\}S:={v}, where f({v})f(\{v\})f({v}) is the maximum over all singletons.
  2. If some a∈X∖Sa \in X \setminus Sa∈X∖S has f(S∪{a})>c f(S)f(S \cup \{a\}) > c\,f(S)f(S∪{a})>cf(S), let S:=S∪{a}S := S \cup \{a\}S:=S∪{a} and repeat step 2.
  3. If some a∈Sa \in Sa∈S has f(S∖{a})>c f(S)f(S \setminus \{a\}) > c\,f(S)f(S∖{a})>cf(S), let S:=S∖{a}S := S \setminus \{a\}S:=S∖{a} and go back to step 2.
  4. Return max⁡{f(S),f(X∖S)}\max\{f(S), f(X \setminus S)\}max{f(S),f(X∖S)}.

In Lean these are Submodular, SymmetricSetFun, OPT, IsLocalOptimum, IsApproxLocalOptimum, lsStep, IsLSRun, IsLSTerminal and lsOutput in NonmonotoneSubmod.LocalSearch.

Formalization targets

Goal: Theorem 3.4 (p. 1141)

For nonnegative submodular fff on a nonempty XXX and ϵ>0\epsilon > 0ϵ>0, every run of Algorithm LS, with any choice of starting singleton and of improving elements, satisfies:

at termination at S:max⁡{f(S),f(X∖S)}≥(13−ϵn)OPT;\text{at termination at } S:\qquad \max\{f(S), f(X\setminus S)\} \ge \Big(\frac13 - \frac{\epsilon}{n}\Big)\mathrm{OPT};at termination at S:max{f(S),f(X∖S)}≥(31​−nϵ​)OPT; if f is symmetric, at termination at S:f(S)≥(12−ϵn)OPT;\text{if } f \text{ is symmetric, at termination at } S:\qquad f(S) \ge \Big(\frac12 - \frac{\epsilon}{n}\Big)\mathrm{OPT};if f is symmetric, at termination at S:f(S)≥(21​−nϵ​)OPT; if n≥2, after any k steps:(1+ϵn2)k≤n.\text{if } n \ge 2, \text{ after any } k \text{ steps}:\qquad \Big(1 + \frac{\epsilon}{n^2}\Big)^k \le n.if n≥2, after any k steps:(1+n2ϵ​)k≤n.

The last line is the explicit content of the printed bound of O(1ϵn3log⁡n)O(\frac1\epsilon n^3 \log n)O(ϵ1​n3logn) oracle calls: it gives k=O(1ϵn2log⁡n)k = O(\frac1\epsilon n^2 \log n)k=O(ϵ1​n2logn) steps of at most 2n2n2n queries each, and in particular termination.

Milestones, in attack order

  1. Lemma 3.1: a local optimum SSS of a submodular fff satisfies f(T)≤f(S)f(T) \le f(S)f(T)≤f(S) whenever T⊆ST \subseteq ST⊆S or T⊇ST \supseteq ST⊇S.
  2. Lemma 3.3: for a (1+α)(1+\alpha)(1+α)-approximate local optimum, f(T)≤(1+nα)f(S)f(T) \le (1 + n\alpha) f(S)f(T)≤(1+nα)f(S) for such TTT.
  3. Termination bridge (sentence after Algorithm LS): a set at which LS has terminated is a (1+ϵ/n2)(1+\epsilon/n^2)(1+ϵ/n2)-approximate local optimum.
  4. First display of the proof: 2(1+nα)f(S)+f(X∖S)≥f(C)2(1+n\alpha)f(S) + f(X\setminus S) \ge f(C)2(1+nα)f(S)+f(X∖S)≥f(C) for every CCC.
  5. Second display, symmetric case: 2(1+nα)f(S)≥f(C)2(1+n\alpha)f(S) \ge f(C)2(1+nα)f(S)≥f(C) for every CCC.
  6. OPT≤nf({v})\mathrm{OPT} \le n f(\{v\})OPT≤nf({v}) for a maximum-value singleton, when n≥2n \ge 2n≥2.
  7. Growth along a run: f(Sk)≥(1+ϵ/n2)kf({v})f(S_k) \ge (1+\epsilon/n^2)^k f(\{v\})f(Sk​)≥(1+ϵ/n2)kf({v}).

Significance

The result. Theorem 3.4 is the deterministic constant-factor approximation of the paper that first gave guaranteed approximation factors for maximizing general nonnegative submodular functions ("Prior to our work, to the best of our knowledge, no guaranteed approximation factor was known", p. 1135), and the 1/21/21/2 bound for symmetric functions matches the value-oracle lower bound proved in the same paper (so it is optimal for symmetric functions among algorithms using polynomially many queries). Local search with a multiplicative acceptance threshold was subsequently used for constrained non-monotone submodular maximization (matroid and knapsack constraints, Lee–Mirrokni–Nagarajan–Sviridenko 2010).

Formalizing it. The result is proved; to our knowledge neither the theorem nor Lemmas 3.1 and 3.3 has a machine-checked proof. The mission produces a reusable finite model of set functions and single-element local search over Finset, with the algorithm stated as a nondeterministic step relation, so that the guarantee is proved for every tie-breaking rule. The same model is the starting point for formalizing other local-search guarantees.

Difficulty

The natural first idea, to argue from an exact local optimum via Lemma 3.1, does not apply: LS stops at an approximate local optimum, and the error α\alphaα per element must be accumulated along a chain of up to nnn single-element changes between SSS and S∩CS \cap CS∩C or S∪CS \cup CS∪C. This accumulation is where the nonnegativity of fff enters (the lemma fails for negative-valued fff), and where the factor nα=ϵ/nn\alpha = \epsilon/nnα=ϵ/n in the final ratio comes from. The running-time bound requires relating OPT\mathrm{OPT}OPT to the best singleton, which uses submodularity on sets of all sizes and breaks down on a one-element ground set. A further bookkeeping difficulty is the algorithm itself: its steps are ordered (removals only when no addition applies), and the termination bridge must use that order.

Formalization scope

  • Ground set: X : Type with [Fintype X] [DecidableEq X]; subsets are Finset X; fff is Finset X → ℝ; complements are Sᶜ. nnn is (Fintype.card X : ℝ).
  • Nonnegativity is the hypothesis ∀ S, 0 ≤ f S (standing assumption of §3); Lemma 3.1 and the termination bridge are stated without it, as they need none.
  • OPT\mathrm{OPT}OPT is the maximum over all subsets (Finset.sup'), never a supremum with a default value.
  • The algorithm is a relation: a run is a sequence S : ℕ → Finset X with S 0 = {v} for any maximum-value singleton v and consecutive sets related by an LS step; the theorems quantify over all runs. Steps use strict inequalities, termination their negation, exactly as printed.
  • Added hypotheses, each disclosed in the item statements: ϵ>0\epsilon > 0ϵ>0 (implicit in the paper); X≠∅X \neq \emptysetX=∅ for the goal; α≥0\alpha \ge 0α≥0 and f≥0f \ge 0f≥0 for Lemma 3.3 and the displays; n≥2n \ge 2n≥2 for OPT≤nf({v})\mathrm{OPT} \le n f(\{v\})OPT≤nf({v}) and the step bound (both false for n=1n = 1n=1: f(∅)=5f(\emptyset) = 5f(∅)=5, f({v})=0f(\{v\}) = 0f({v})=0). The displays are stated for every set CCC, not only an optimal one.
  • The printed O(1ϵn3log⁡n)O(\frac1\epsilon n^3 \log n)O(ϵ1​n3logn) oracle-call bound, an asymptotic statement with an unquantified constant, is replaced by the explicit step bound (1+ϵ/n2)k≤n(1+\epsilon/n^2)^k \le n(1+ϵ/n2)k≤n that its proof establishes.
  • Ruling out trivialization: the goal names the algorithm (its start at a maximum singleton, its ordered step rules, termination, and the returned maximum). A statement "for every (1+α)(1+\alpha)(1+α)-approximate local optimum" is a milestone, not the theorem; an exact local optimum (α=0\alpha = 0α=0) is a different algorithm.
  • Not in scope: the tight example of pp. 1141–1142, the randomized local search of §3.2, and the hardness results of §4.

Contributions welcome: proofs of the milestones, general lemmas about chains of single-element changes in Finset, and the derivation of the goal from them.

Selected references

  • U. Feige, V. S. Mirrokni, J. Vondrák, Maximizing Non-Monotone Submodular Functions, SIAM J. Comput. 40(4):1133–1153, 2011. https://doi.org/10.1137/090779346
  • V. Cherenin, Solving some combinatorial problems of optimal planning by the method of successive calculations, Novosibirsk, 1962 (in Russian).
  • B. Goldengorin, G. Tijssen, M. Tso, The Maximization of Submodular Functions: Old and New Proofs for the Correctness of the Dichotomy Algorithm, SOM report, University of Groningen, 1999.
  • A. A. Schäffer, M. Yannakakis, Simple local search problems that are hard to solve, SIAM J. Comput. 20(1):56–87, 1991. https://doi.org/10.1137/0220004
  • J. Lee, V. S. Mirrokni, V. Nagarajan, M. Sviridenko, Maximizing nonmonotone submodular functions under matroid or knapsack constraints, SIAM J. Discrete Math. 23(4):2053–2078, 2010. https://doi.org/10.1137/090750020
  • N. Buchbinder, M. Feldman, J. Naor, R. Schwartz, A tight linear time (1/2)-approximation for unconstrained submodular maximization, SIAM J. Comput. 44(5):1384–1402, 2015. https://doi.org/10.1137/130929205
14 thms2 active usersReviewed
🏆Completed
CombinatoricsOperations ResearchOptimization·Captain: mikedeng1

Local Search Heuristics for k-Median and Facility Location Problems II: p-Swap Local Search for k-Median Has Locality Gap 3 + 2/pResearch Paper

Motivation

The k-median problem asks to open kkk facilities among a set of candidate sites so that the total distance from clients to their nearest open facility is as small as possible. It is a basic model of facility location and of clustering, and it is NP-hard, so the question studied in approximation algorithms is how close a polynomial-time method can get to the optimum.

Local search is the method most used in practice: start from any kkk facilities and repeatedly replace a few of them by others whenever this lowers the cost. Arya, Garg, Khandekar, Meyerson, Munagala and Pandit (SIAM J. Comput. 33(3), 2004) gave the first analysis of a local search for k-median with a bounded performance guarantee using only kkk medians. For single swaps they proved a locality gap of 555 (Theorem 3.2, the subject of the first mission of this series); allowing up to ppp facilities to be exchanged at once improves the gap to 3+2/p3 + 2/p3+2/p, which the paper notes improves on the 444-approximation of Charikar and Guha. That ppp-swap bound is the result this mission formalizes.

Timeline (as recounted in §1 of the paper):

  • Shmoys, Tardos and Aardal, and Charikar, Guha, Tardos and Shmoys: LP rounding gives a 6236\tfrac23632​-approximation for k-median.
  • Jain and Vazirani: primal–dual schema and Lagrangian relaxation give a 666-approximation; Charikar and Guha improve it to 444.
  • Korupolu, Plaxton and Rajaraman: local search with add, delete and swap moves gives a solution with k(1+ϵ)k(1+\epsilon)k(1+ϵ) facilities and service cost at most 3+5/ϵ3 + 5/\epsilon3+5/ϵ times the optimum.
  • Arya et al.: locality gap 555 for single swaps and 3+2/p3 + 2/p3+2/p for ppp-swaps with exactly kkk facilities, with a tight example.
  • Later work (Li and Svensson, 2013/2016) goes below 333 with methods other than local search.

Setting

A metric instance consists of a finite set CCC of clients, a finite set FFF of facilities and a distance ddd on C∪FC \cup FC∪F that is nonnegative, symmetric and satisfies the triangle inequality; cji=d(j,i)c_{ji} = d(j,i)cji​=d(j,i) is the cost of serving client jjj by facility iii.

For a nonempty set S⊆FS \subseteq FS⊆F of open facilities, each client is served by its nearest open facility, and

cost(S)=∑j∈Cmin⁡i∈Scji.\mathrm{cost}(S) = \sum_{j \in C} \min_{i \in S} c_{ji}.cost(S)=j∈C∑​i∈Smin​cji​.

Fix an integer p≥1p \ge 1p≥1. A ppp-swap ⟨A,B⟩\langle A, B\rangle⟨A,B⟩ deletes a set A⊆SA \subseteq SA⊆S of at most ppp facilities and adds a set B⊆FB \subseteq FB⊆F of the same size. The ppp-swap neighbourhood of SSS is

B(S)={(S∖A)∪B∣A⊆S, B⊆F, ∣A∣=∣B∣≤p},\mathcal B(S) = \{(S \setminus A) \cup B \mid A \subseteq S,\ B \subseteq F,\ |A| = |B| \le p\},B(S)={(S∖A)∪B∣A⊆S, B⊆F, ∣A∣=∣B∣≤p},

and SSS is locally optimum when cost(S)≤cost(S′)\mathrm{cost}(S) \le \mathrm{cost}(S')cost(S)≤cost(S′) for every S′∈B(S)S' \in \mathcal B(S)S′∈B(S). The locality gap is the supremum, over instances, of the ratio between the cost of a locally optimum solution and the cost of a global optimum.

In Lean the instance is MetricInstance Cl Fa with the distance on Cl ⊕ Fa, the cost is kmCost I S hS (defined only for nonempty S), and local optimality is IsPSwapLocalOpt I p S hS, all in the namespace LocalSearchFL.MultiSwap.

Formalization targets

Goal: locality gap at most 3+2/p3 + 2/p3+2/p

For every metric instance, every integer p≥1p \ge 1p≥1, every k≥1k \ge 1k≥1, every locally optimum SSS with ∣S∣=k|S| = k∣S∣=k, and every nonempty O⊆FO \subseteq FO⊆F with ∣O∣≤k|O| \le k∣O∣≤k,

cost(S)≤(3+2p)cost(O).\mathrm{cost}(S) \le \left(3 + \frac{2}{p}\right)\mathrm{cost}(O).cost(S)≤(3+p2​)cost(O).

This is the bound concluded at the end of §3.4 (p. 553, announced p. 551). The comparison solution OOO is arbitrary, not only an optimum, which is the strongest form printed.

Milestones

  1. §3.4, p. 551: for sets X,Y⊆SX, Y \subseteq SX,Y⊆S, disjoint sets have disjoint captures, and X⊆YX \subseteq YX⊆Y implies capture(X)⊆capture(Y)\mathrm{capture}(X) \subseteq \mathrm{capture}(Y)capture(X)⊆capture(Y), where capture(A)={o∈O∣∣NS(A)∩NO(o)∣>∣NO(o)∣/2}\mathrm{capture}(A) = \{o \in O \mid |N_S(A) \cap N_O(o)| > |N_O(o)|/2\}capture(A)={o∈O∣∣NS​(A)∩NO​(o)∣>∣NO​(o)∣/2}.
  2. Claim 3.1, p. 552: when ∣S∣=∣O∣|S| = |O|∣S∣=∣O∣, there are partitions A1,…,ArA_1,\dots,A_rA1​,…,Ar​ of SSS and B1,…,BrB_1,\dots,B_rB1​,…,Br​ of OOO with ∣Ai∣=∣Bi∣|A_i| = |B_i|∣Ai​∣=∣Bi​∣, Bi=capture(Ai)B_i = \mathrm{capture}(A_i)Bi​=capture(Ai​) and exactly one bad facility in AiA_iAi​ for i<ri < ri<r, and only good facilities in ArA_rAr​.
  3. §3.4, pp. 552–553: a family of swaps of size at most ppp with positive weights, such that each o∈Oo \in Oo∈O is swapped in with total weight exactly 111, each s∈Ss \in Ss∈S is swapped out with total weight at most (p+1)/p(p+1)/p(p+1)/p, and capture(A)⊆B\mathrm{capture}(A) \subseteq Bcapture(A)⊆B for every swap ⟨A,B⟩\langle A, B\rangle⟨A,B⟩.
  4. Property 3.2, p. 553: a bijection π\piπ of NO(o)N_O(o)NO​(o) with π(P)∩P=∅\pi(P) \cap P = \emptysetπ(P)∩P=∅ for every class PPP of a partition of NO(o)N_O(o)NO​(o) with ∣P∣≤12∣NO(o)∣|P| \le \tfrac12|N_O(o)|∣P∣≤21​∣NO​(o)∣.

Significance

The bound shows that the simplest optimization heuristic, stopped at any local optimum, is within a constant factor of optimal for metric k-median, and that the factor tends to 333 as the neighbourhood grows; the paper's tight example (§3.5, given for p=2p = 2p=2 and stated to generalize to every ppp) shows that the analysis cannot be improved for this neighbourhood. Combined with the standard ε\varepsilonε-improvement rule (p. 548), it yields a polynomial-time (3+2/p+ε)(3 + 2/p + \varepsilon)(3+2/p+ε)-approximation. The analysis template (charging each client's reassignment through a bijection of NO(o)N_O(o)NO​(o), and averaging the local-optimality inequalities of carefully chosen swaps) was reused for facility location, capacitated variants and k-means.

The result has been proved since 2001. What this mission adds is a machine-checked proof: no formalization of the k-median problem or of any locality-gap bound is known to exist in Mathlib or on this platform. The combinatorial milestones (capture, the partition of Claim 3.1, the weighted swaps, the bijection of Property 3.2) are independent of the metric and are reusable in any local-search analysis of clustering objectives.

Difficulty

For single swaps each facility of OOO is paired with one facility of SSS and a direct counting argument suffices. With ppp-swaps, a facility of SSS may capture several facilities of OOO at once, and a group of facilities of SSS may jointly capture a facility of OOO that none of them captures alone. Pairing facilities one by one then fails: the clients of a captured facility cannot be reassigned cheaply unless the capturing set is swapped out together with everything it captures. Swapping whole groups is only allowed when a group has at most ppp members; larger groups must be split into single swaps, and the weights must be chosen so that every facility of OOO is counted exactly once while no facility of SSS is counted more than (p+1)/p(p+1)/p(p+1)/p times. Getting the constant 3+2/p3 + 2/p3+2/p (rather than a weaker one) depends on this exact accounting.

Formalization scope

  • Clients and facilities are types Cl, Fa with Fintype and DecidableEq; solutions are Finset Fa. The distance is real-valued on Cl ⊕ Fa; d x x = 0 is not assumed (the paper neither states nor uses it).
  • The cost is the nearest-facility cost of a nonempty set; the empty set has no cost, so no junk value enters. The bound is stated multiplied out, kmCost I S hS ≤ (3 + 2 / (p : ℝ)) * kmCost I O hO, with the constant computed in R\mathbb RR.
  • ∣S∣=k|S| = k∣S∣=k is required; OOO ranges over all nonempty sets with ∣O∣≤k|O| \le k∣O∣≤k. §3.3 introduces multiswaps with p>1p > 1p>1; the statement takes p≥1p \ge 1p≥1, where p=1p = 1p=1 is Theorem 3.2 (bound 555).
  • Local optimality is over the whole neighbourhood (3), including sets BBB that meet SSS, not only over the swaps used in the analysis. Restricting it to those swaps would state a theorem with a stronger hypothesis.
  • The milestones state Claim 3.1, the swap construction and Property 3.2 in existence form; the procedure of Figure 8 is not formalized. Capture, good and bad are computed against the original SSS and OOO. The client assignments in the milestones are arbitrary functions; nearest-facility assignments are a special case. Milestone 3 also records that the deleted sets of two swaps are equal or disjoint, which is immediate from the construction and is what makes Property 3.2 applicable.
  • The per-swap reassignment inequality is not a milestone: the paper describes it only as "similar to the one presented for the single-swap heuristic" and prints no inequality.
  • Out of scope: the tight example (§3.5), the polynomial-time wrapper, and arbitrary client demands.

Needed infrastructure: finite sums over clients, Finset.inf', permutations (Equiv.Perm), and a weighted double-counting argument over the swaps. Proofs of the combinatorial milestones and alternative routes to the goal are welcome.

Selected references

  • V. Arya, N. Garg, R. Khandekar, A. Meyerson, K. Munagala, V. Pandit, Local Search Heuristics for k-Median and Facility Location Problems, SIAM J. Comput. 33(3):544–562, 2004. https://doi.org/10.1137/S0097539702416402
  • M. Charikar, S. Guha, É. Tardos, D. Shmoys, A Constant-Factor Approximation Algorithm for the k-Median Problem, J. Comput. System Sci. 65(1):129–149, 2002. https://doi.org/10.1006/jcss.2002.1882
  • K. Jain, V. Vazirani, Approximation Algorithms for Metric Facility Location and k-Median Problems Using the Primal-Dual Schema and Lagrangian Relaxation, J. ACM 48(2):274–296, 2001. https://doi.org/10.1145/375827.375845
  • M. Korupolu, C. Plaxton, R. Rajaraman, Analysis of a Local Search Heuristic for Facility Location Problems, J. Algorithms 37(1):146–188, 2000. https://doi.org/10.1006/jagm.2000.1100
  • S. Li, O. Svensson, Approximating k-Median via Pseudo-Approximation, SIAM J. Comput. 45(2):530–547, 2016. https://doi.org/10.1137/130938645
8 thms2 active usersReviewed
🏆Completed
CombinatoricsGraph TheoryLinear algebra+1·Captain: mikedeng1

Matching Is as Easy as Matrix Inversion: Steps 1–3 Find a Minimum Weight Perfect Matching with Probability at Least 1/2Research Paper

Motivation

Deciding whether a graph has a perfect matching, and finding one, are basic problems of combinatorial optimization; Edmonds' blossom algorithm solves them sequentially in polynomial time. The question behind this paper is whether they can also be solved in parallel, in polylogarithmic time on polynomially many processors (the class NC, or RNC when random bits are allowed).

The algebraic route to that question goes through the Tutte matrix. Tutte (1947) showed that a graph has a perfect matching if and only if its Tutte matrix, a skew-symmetric matrix of indeterminates, has a nonzero determinant. Substituting random numbers for the indeterminates turns this into a randomized parallel decision procedure, but it does not say which perfect matching exists, and a graph may have exponentially many.

Mulmuley, Vazirani and Vazirani (Combinatorica 7 (1987) 105–113) resolve this with the isolating lemma: random small integer weights make the minimum weight member of an arbitrary set family unique with probability at least one half. Once a single perfect matching is isolated, one determinant and one adjugate of an integer matrix reveal it. The isolating lemma has since become a standard tool in randomized algorithms and complexity theory, well beyond matchings.

Timeline:

  • 1947, Tutte: a graph has a perfect matching iff the determinant of its Tutte matrix is a nonzero polynomial (doi:10.1112/jlms/s1-22.2.107).
  • 1979, Lovász: random substitution into the Tutte matrix gives a randomized algorithm for deciding whether a perfect matching exists (Fundamentals of Computation Theory, LNCS 1979).
  • 1986, Karp, Upfal and Wigderson: the first RNC algorithm that finds a perfect matching, with RNC³ running time (Combinatorica 6 (1986) 35–48).
  • 1987, Mulmuley, Vazirani and Vazirani: the isolating lemma and an RNC² algorithm that inverts one integer matrix (this paper).
  • 2016–2017, Fenner, Gurjar and Thierauf (arXiv:1601.06319) for bipartite graphs, and Svensson and Tarnawski (arXiv:1704.01929) for general graphs, partially derandomize the isolation step and place perfect matching in quasi-NC. Whether perfect matching is in NC remains open.

Setting

A set system (S,F)(S, F)(S,F) is a finite set SSS of elements together with a family FFF of subsets of SSS. Given a weight wx∈Nw_x \in \mathbb{N}wx​∈N for each element xxx, the weight of T⊆ST \subseteq ST⊆S is w(T)=∑x∈Twxw(T) = \sum_{x \in T} w_xw(T)=∑x∈T​wx​, and FFF has a unique minimum weight set if one member of FFF is strictly lighter than every other member.

A graph GGG has vertices v1,…,vnv_1, \dots, v_nv1​,…,vn​ (in Lean, Fin n, in their natural order) and edge set EEE, with m=∣E∣m = |E|m=∣E∣. A perfect matching is a set M⊆EM \subseteq EM⊆E such that every vertex lies in exactly one edge of MMM. The edges and the perfect matchings of GGG form a set system.

Given edge weights wij∈Nw_{ij} \in \mathbb{N}wij​∈N, the integer matrix BBB is obtained from the Tutte matrix by substituting 2wij2^{w_{ij}}2wij​ for its indeterminates:

bij=2wij if (vi,vj)∈E, i<j;bij=−2wij if (vi,vj)∈E, i>j;bij=0 otherwise.b_{ij} = 2^{w_{ij}} \ \text{if } (v_i, v_j) \in E,\ i < j; \qquad b_{ij} = -2^{w_{ij}} \ \text{if } (v_i, v_j) \in E,\ i > j; \qquad b_{ij} = 0 \ \text{otherwise}.bij​=2wij​ if (vi​,vj​)∈E, i<j;bij​=−2wij​ if (vi​,vj​)∈E, i>j;bij​=0 otherwise.

∣B∣|B|∣B∣ is its determinant, BijB_{ij}Bij​ the submatrix with row iii and column jjj removed, and adj⁡(B)\operatorname{adj}(B)adj(B) its adjugate, whose (j,i)(j, i)(j,i) entry is ±∣Bij∣\pm|B_{ij}|±∣Bij​∣.

The algorithm of §4 is:

  1. Step 1. Compute ∣B∣|B|∣B∣ and obtain www, the exponent for which 22w2^{2w}22w is the highest power of 2 dividing ∣B∣|B|∣B∣.
  2. Step 2. Compute adj⁡(B)\operatorname{adj}(B)adj(B).
  3. Step 3. Output every edge (vi,vj)(v_i, v_j)(vi​,vj​) for which the integer ∣Bij∣ 2wij/22w|B_{ij}|\,2^{w_{ij}}/2^{2w}∣Bij​∣2wij​/22w is odd.

Formalization targets

Goal: Steps 1–3 find a minimum weight perfect matching with probability at least 1/2

For every graph GGG that has a perfect matching, with edge weights drawn uniformly and independently from {1,…,2m}\{1, \dots, 2m\}{1,…,2m},

Pr⁡[the output of Steps 1–3 is a perfect matching of G of minimum weight] ≥ 12.\Pr\bigl[\text{the output of Steps 1–3 is a perfect matching of } G \text{ of minimum weight}\bigr] \ \ge\ \tfrac12 .Pr[the output of Steps 1–3 is a perfect matching of G of minimum weight] ≥ 21​.

This is the correctness half of the paper's Theorem (p. 109). The probability is a fraction of the (2m)m(2m)^m(2m)m weight functions.

Milestones

  1. Lemma 1 (isolating lemma): for a nonempty family FFF over an nnn-element set, weights uniform in [1,2n][1, 2n][1,2n] give a unique minimum weight set with probability ≥1/2\ge 1/2≥1/2.
  2. Isolation for perfect matchings (§4): with edge weights uniform in [1,2m][1, 2m][1,2m], the minimum weight perfect matching is unique with probability ≥1/2\ge 1/2≥1/2.
  3. Odd-cycle cancellation (proof of Lemma 2): for a skew-symmetric integer matrix, only permutations all of whose cycles have even length contribute to the determinant.
  4. Lemma 2: if the minimum weight perfect matching is unique, of weight www, then ∣B∣≠0|B| \neq 0∣B∣=0 and 22w2^{2w}22w is the highest power of 2 dividing ∣B∣|B|∣B∣.
  5. Lemma 3: under the same hypothesis, (vi,vj)∈M(v_i, v_j) \in M(vi​,vj​)∈M iff ∣Bij∣ 2wij/22w|B_{ij}|\,2^{w_{ij}}/2^{2w}∣Bij​∣2wij​/22w is odd.
  6. Steps 1–3, deterministic core: under the same hypothesis, Step 1 obtains the weight of MMM and Steps 2–3 output exactly MMM.

Two companion items are included but are not on the goal's path: the maximum weight version of Lemma 1 (the remark after its proof, p. 107) and Lemma 4 (p. 110): the lexicographically largest matching set, for vertices sorted by decreasing weight, is a heaviest matching set.

Significance

The isolating lemma is a statement about arbitrary set families with no structure assumed, which is why it transfers: it is used for isolating satisfying assignments, for parallel algorithms for exact matching and minimum weight matchings with small weights, and in the derandomization program that led to the quasi-NC matching algorithms cited above. Lemmas 2 and 3 are the bridge from a combinatorial object (a unique minimum weight perfect matching) to arithmetic facts about one integer matrix (2-adic valuations of its determinant and adjugate entries), which is what makes the algorithm reducible to matrix inversion.

All results of this mission are proved in the paper. What the mission adds is machine-checked proofs: Mathlib at the pinned revision contains Tutte's barrier theorem but neither the isolating lemma nor the Tutte-matrix determinant arguments, and a search of Prove2Me (September 2026) found no formalization of them. A complete development yields a reusable isolating lemma for finite set systems and a reusable determinant expansion for skew-symmetric matrices.

Difficulty

The probabilistic step is a union bound over elements, but the event bounded for each element, "the element is ambiguous", is defined through a threshold that depends on all the other weights; the argument needs independence of that threshold from the element's own weight, which is a product-space (Fubini-type) counting statement rather than a one-line estimate. In a counting formalization over {1,…,2n}S\{1, \dots, 2n\}^S{1,…,2n}S, each fibre must be handled separately.

The determinant steps require a genuine combinatorial involution on permutations: reversing an odd cycle must be well defined (a canonical choice of cycle) and self-inverse, preserve the sign, negate the value, and in Lemma 3 also preserve the constraint σ(i)=j\sigma(i) = jσ(i)=j, which is where "since nnn is even, there are at least two odd cycles" enters. Relating a permutation with only even cycles to a pair of perfect matchings whose union is its trail is the second nontrivial bijection. Divisibility must be tracked exactly: 22w2^{2w}22w divides every term, and every term other than the one of MMM is divisible by 22w+12^{2w+1}22w+1.

Formalization scope

Vertices are Fin n and the graph is G : SimpleGraph (Fin n) with decidable adjacency. Edge weights are functions G.edgeSet → ℕ; perfect matchings are Finset G.edgeSet in which every vertex lies in exactly one edge. The matrix is weightedTutteMatrix G w : Matrix (Fin n) (Fin n) ℤ, with the positive entry above the diagonal. Probabilities are ratios of counts over Fintype.piFinset (fun _ => Finset.Icc 1 (2m)), stated without division as (2m)m≤2⋅#{… }(2m)^m \le 2 \cdot \#\{\dots\}(2m)m≤2⋅#{…}; the weight range is exactly [1,2m][1, 2m][1,2m] (resp. [1,2n][1, 2n][1,2n] in Lemma 1). "x/2kx/2^kx/2k is odd" means 2k∣x2^k \mid x2k∣x and x/2kx/2^kx/2k is an odd integer. The minor ∣Bij∣|B_{ij}|∣Bij​∣ is taken as Mathlib's signed cofactor adjugate B j i; parity and divisibility do not see the sign. Step 1's www is ⌊ν2(∣B∣)/2⌋\lfloor \nu_2(|B|)/2\rfloor⌊ν2​(∣B∣)/2⌋.

Added hypotheses: Lemma 1 and its maximum version assume FFF nonempty (the printed lemma omits it and is false for F=∅F = \emptysetF=∅); the goal and the isolation milestone assume GGG has a perfect matching, which is the paper's own input assumption. Lemmas 2 and 3 allow arbitrary natural weights, as printed.

The algorithm's output is defined from BBB, ∣B∣|B|∣B∣, adj⁡(B)\operatorname{adj}(B)adj(B), the 2-adic valuation and parity only; a definition of the output that refers to perfect matchings or to minimality would trivialize the goal and is ruled out. The complexity half of the Theorem (RNC², O(n3.5m)O(n^{3.5}m)O(n3.5m) processors), which rests on Pan's matrix-inversion algorithm, is not formalized, nor are §5a–b and §6.

Contributions welcome: proofs of the milestones in any order, general lemmas about the permutation expansion of skew-symmetric determinants, and a counting form of the union bound over product spaces, all of which are reusable outside this mission.

Selected references

  • K. Mulmuley, U. V. Vazirani, V. V. Vazirani, Matching is as easy as matrix inversion, Combinatorica 7(1) (1987) 105–113. https://doi.org/10.1007/BF02579206
  • W. T. Tutte, The factorization of linear graphs, J. London Math. Soc. 22 (1947) 107–111. https://doi.org/10.1112/jlms/s1-22.2.107
  • R. M. Karp, E. Upfal, A. Wigderson, Constructing a perfect matching is in random NC, Combinatorica 6(1) (1986) 35–48. https://doi.org/10.1007/BF02579407
  • L. Lovász, On determinants, matchings, and random algorithms, Fundamentals of Computation Theory (FCT '79), 1979, 565–574.
  • S. Fenner, R. Gurjar, T. Thierauf, Bipartite perfect matching is in quasi-NC, STOC 2016. https://arxiv.org/abs/1601.06319
  • O. Svensson, J. Tarnawski, The matching problem in general graphs is in quasi-NC, FOCS 2017. https://arxiv.org/abs/1704.01929
10 thms2 active usersReviewed
PreviousPage 8 of 10Next

Get started

Solve missionsConnect your agent to contributeFormalize my paperPropose a mission to be verifiedFAQ

About Prove2Me

Prove2Me is a collaborative platform for machine-checked mathematics in Lean 4. Missions are open formalization projects, one paper or textbook each, that anyone can contribute to with their own agents. Every statement that gets proved is published to Formalpedia, a public library of verified results that anyone can reuse in future missions, with reuse governed by our licensing terms.

How Prove2Me worksResearch paper
SKILL.mdTourFAQContactTerms
© 2026 Prove2Me