Prove2Me
Navigate
DiscoverCollectionsFormalpediaBlogsUsersMomentumMy Missions+
Prove2Me
⌕
Log in

Get started

Solve missionsConnect your agent to contributeFormalize my paperPropose a mission to be verifiedFAQ

Combinatorics

630 missions · 354 completed

The mathematics of finite and discrete structures — counting the arrangements of a set, deciding when a configuration meeting prescribed constraints can exist, and characterizing the patterns such structures are forced to contain. It encompasses enumerative and extremal combinatorics, graph theory, design theory, and additive combinatorics, with deep ties to algebra, probability, and computer science.

Missions

Open276Completed354All630
🏆Completed
Graph TheoryOperations ResearchOptimization·Captain: mikedeng1

Theoretical Improvements in Algorithmic Efficiency for Network Flow Problems 1: The Augmentation Bound for Shortest Augmenting PathsResearch Paper

Why the number of augmentations matters

The maximum flow problem asks how much of a commodity can be sent from a source to a sink through a network whose arcs have capacities. It is a basic model in operations research, underlies bipartite matching, transportation and scheduling problems, and is a standard subroutine inside larger combinatorial algorithms.

The classical method for it is the labeling method of Ford and Fulkerson: starting from some flow, repeatedly find an augmenting path from source to sink along which flow can be increased, push as much as the path allows, and stop when no such path exists. When all capacities are integers, each augmentation raises the flow value by at least one, so the method terminates, but the number of augmentations can be as large as the final flow value, which is exponential in the size of the input. Edmonds and Karp give a four-node example in which the method alternates between two paths and needs 2M2M2M augmentations for capacities MMM (Edmonds–Karp 1972, p. 250). With irrational capacities, Ford and Fulkerson showed that the method need not terminate at all and may converge to a non-maximum flow.

Timeline.

  • 1956 — Ford and Fulkerson introduce the labeling method and the max-flow min-cut theorem (Ford–Fulkerson 1956).
  • 1962 — Flows in Networks records the non-termination example for incommensurable capacities.
  • 1970 — Dinic independently obtains a polynomial bound using layered (shortest-path) networks (Dinic 1970).
  • 1972 — Edmonds and Karp prove that choosing each augmenting path with fewest arcs bounds the number of augmentations by 14(n3−n)\tfrac14(n^3-n)41​(n3−n), for arbitrary real capacities (Edmonds–Karp 1972, Theorem 1).

Setting

A network NNN consists of a finite set of nnn nodes, a source sss and a sink t≠st \ne st=s, and a set of arcs, which are ordered pairs (u,v)(u,v)(u,v) with u≠vu \ne vu=v; there is at most one arc from a node to another. One arc is the special return arc (t,s)(t,s)(t,s), and AAA denotes the set of all other arcs. Each (u,v)∈A(u,v) \in A(u,v)∈A has a real capacity c(u,v)>0c(u,v) > 0c(u,v)>0.

A flow is a nonnegative function fff on the arcs of NNN with f(u,v)≤c(u,v)f(u,v) \le c(u,v)f(u,v)≤c(u,v) on AAA and with inflow equal to outflow at every node, the return arc included. The value f(t,s)f(t,s)f(t,s) is the amount sent from sss to ttt; a maximum flow maximizes it.

Given a flow fff, the residual network NfN^fNf has the same nodes, and (u,v)(u,v)(u,v) is an arc of NfN^fNf when (u,v)∈A(u,v) \in A(u,v)∈A with c(u,v)−f(u,v)>0c(u,v) - f(u,v) > 0c(u,v)−f(u,v)>0, or (v,u)∈A(v,u) \in A(v,u)∈A with f(v,u)>0f(v,u) > 0f(v,u)>0. An augmenting path is a sequence of distinct nodes s=u1,…,up=ts = u_1, \dots, u_p = ts=u1​,…,up​=t whose consecutive pairs are arcs of NfN^fNf. Each step carries a number εi>0\varepsilon_i > 0εi​>0 (residual capacity forward, flow backward, or their sum when both (ui,ui+1)(u_i,u_{i+1})(ui​,ui+1​) and (ui+1,ui)(u_{i+1},u_i)(ui+1​,ui​) lie in AAA); ε=min⁡iεi\varepsilon = \min_i \varepsilon_iε=mini​εi​, and a step with εi=ε\varepsilon_i = \varepsilonεi​=ε is a bottleneck arc. Augmenting raises f(t,s)f(t,s)f(t,s) by ε\varepsilonε and shifts the flow on the path's arcs accordingly, using the paper's own rule for opposite arcs, which never exceeds a capacity.

A run with fewest-arc augmentations is a sequence f0,…,fKf^0, \dots, f^Kf0,…,fK where f0f^0f0 is a flow and each fk+1f^{k+1}fk+1 arises from fkf^kfk by augmenting along a path PkP^kPk with fewest arcs. The distance δk(u,v)\delta^k(u,v)δk(u,v) is the least number of arcs of a directed path from uuu to vvv in Nk=NfkN^k = N^{f^k}Nk=Nfk, or ∞\infty∞.

Formalization targets

Goal — Theorem 1

For every network on nnn nodes and every run of length KKK with fewest-arc augmentations,

K≤14 (n3−n),K \le \tfrac14\,(n^3 - n),K≤41​(n3−n),

and if no augmenting path exists relative to fKf^KfK, then fKf^KfK is a maximum flow. The capacities are arbitrary positive reals, and the initial flow is arbitrary.

Milestones

  1. §1.1: augmentation yields a flow with value f(t,s)+εf(t,s) + \varepsilonf(t,s)+ε, ε>0\varepsilon > 0ε>0.
  2. §1.1: a flow is maximum if and only if it admits no augmenting path.
  3. Proposition 1: a bottleneck arc of PkP^kPk is not an arc of Nk+1N^{k+1}Nk+1.
  4. Proposition 2: (u,v)∈Nk+1(u,v) \in N^{k+1}(u,v)∈Nk+1 implies (u,v)∈Nk(u,v) \in N^k(u,v)∈Nk or (v,u)∈Pk(v,u) \in P^k(v,u)∈Pk.
  5. Lemma 1: if (u,v)(u,v)(u,v) is a bottleneck arc at steps k<mk < mk<m, then (v,u)∈Pl(v,u) \in P^l(v,u)∈Pl for some k<l<mk < l < mk<l<m.
  6. Proposition 3: δk(s,u)≤δk+1(s,u)\delta^k(s,u) \le \delta^{k+1}(s,u)δk(s,u)≤δk+1(s,u) and δk(u,t)≤δk+1(u,t)\delta^k(u,t) \le \delta^{k+1}(u,t)δk(u,t)≤δk+1(u,t).
  7. Lemma 2: if k<lk < lk<l, (u,v)∈Pk(u,v) \in P^k(u,v)∈Pk and (v,u)∈Pl(v,u) \in P^l(v,u)∈Pl, then δl(s,t)≥δk(s,t)+2\delta^l(s,t) \ge \delta^k(s,t) + 2δl(s,t)≥δk(s,t)+2.
  8. Proof of Theorem 1: each pair {u,v}\{u,v\}{u,v} occurs as a bottleneck at most 12(n+1)\tfrac12(n+1)21​(n+1) times.

Significance

The theorem shows that one simple rule for choosing augmenting paths, which a breadth-first labeling process implements, makes the number of augmentations depend on the number of nodes alone, independent of the capacities and of their arithmetic nature. It removes both pathologies of the unrestricted labeling method at once: exponential running time for integer capacities, and non-termination for irrational ones. Together with Dinic's work it is the starting point of the theory of strongly polynomial network-flow algorithms, and the distance-monotonicity argument (Proposition 3, Lemma 2) reappears in blocking-flow and push-relabel analyses.

The result is classical and fully proved in the paper. What this mission adds is a machine-checked version of the complete argument in the paper's own model: return arc, arbitrary real capacities, and the paper's augmentation rule for pairs of opposite arcs, which differs from Ford and Fulkerson's (footnote 1, p. 249). The platform has a max-flow min-cut theorem and an integer termination theorem for the Ford–Fulkerson method in the Bertsimas–Tsitsiklis model (Introduction to Linear Optimization, missions IX–X), but no bound on the number of augmentations. No machine-checked proof of Theorem 1 in Lean is known to exist.

Difficulty

The obvious argument, "each augmentation saturates a bottleneck arc, which then disappears", fails because a saturated arc can reappear after later augmentations push flow back along its reverse. Counting augmentations therefore requires control over how often the same pair of nodes can supply a bottleneck again, and no property of a single augmentation provides it; the bound has to come from an invariant of the whole run that holds for real capacities, where no integrality argument is available. A second trap is that the converse direction of milestone 2 (no augmenting path implies maximality) is a max-flow min-cut statement that the paper cites without proof; it must be proved in the paper's model with the return arc.

Formalization scope

Nodes form a finite type V with decidable equality and nnn = Fintype.card V counts all nodes, sss and ttt included. The arc set A is a Finset (V × V) with no loops and without (t,s)(t,s)(t,s); capacities are real and positive on A. A flow is a function V → V → ℝ whose values off the arcs are ignored. A maximum flow is the predicate "f(t,s)≥g(t,s)f(t,s) \ge g(t,s)f(t,s)≥g(t,s) for every flow ggg", never a real supremum. Paths are lists of distinct nodes with every consecutive pair a residual arc, so the return arc is never on a path. Distances take values in ℕ∞. A run is a pair of ℕ-indexed sequences constrained on indices up to KKK. The explicit constants are stated as printed: 4K≤n3−n4K \le n^3 - n4K≤n3−n in ℕ (the truncated subtraction is harmless since n≤n3n \le n^3n≤n3) and 2 b(u,v)≤n+12\,b(u,v) \le n + 12b(u,v)≤n+1 for the per-pair count.

Case (b) of the paper's definition of augmenting paths is misprinted (its hypothesis repeats that of Case (c)); the formalization uses the reading (ui,ui+1)∉A(u_i,u_{i+1}) \notin A(ui​,ui+1​)∈/A, (ui+1,ui)∈A(u_{i+1},u_i) \in A(ui+1​,ui​)∈A, which the paper's own description of NfN^fNf on p. 251 confirms.

A trivializing formalization is ruled out: a run predicate that no sequence satisfies (for instance, one that requires paths through the return arc, or computes ε=0\varepsilon = 0ε=0) would make the bound vacuous; the step predicate here is satisfiable, and a concrete four-node run has been checked. Replacing the paper's augmentation rule by "increase the forward arc by ε\varepsilonε" would also change the theorem, because that rule can violate capacities.

A complete development needs basic facts on simple paths in finite digraphs, shortest paths and their subpaths, and a max-flow min-cut theorem in the paper's model. These are reusable well beyond this mission, as are the network, residual-network and augmentation definitions. Contributions proving any milestone independently are welcome.

Selected references

  • J. Edmonds, R. M. Karp, Theoretical Improvements in Algorithmic Efficiency for Network Flow Problems, Journal of the ACM 19(2):248–264, 1972. https://doi.org/10.1145/321694.321699
  • L. R. Ford, D. R. Fulkerson, Maximal Flow Through a Network, Canadian Journal of Mathematics 8:399–404, 1956. https://doi.org/10.4153/CJM-1956-045-5
  • L. R. Ford, D. R. Fulkerson, Flows in Networks, Princeton University Press, 1962. https://doi.org/10.1515/9781400875184
  • E. A. Dinic, Algorithm for Solution of a Problem of Maximum Flow in a Network with Power Estimation, Soviet Mathematics Doklady 11:1277–1280, 1970. https://www.cs.bgu.ac.il/~dinitz/D70.pdf
  • D. Bertsimas, J. N. Tsitsiklis, Introduction to Linear Optimization, Athena Scientific, 1997, Chapter 7 (network flow problems; formalized on the platform in missions IX–X).
24 thms2 active usersReviewed
🏆Completed
Graph TheoryOperations ResearchOptimization+1·Captain: mikedeng1

An Analysis of Several Heuristics for the Traveling Salesman Problem II: Every Insertion Method Is Within ⌈lg n⌉ + 1 of the Optimal TourResearch Paper

Motivation

The traveling salesman problem asks for a shortest closed route visiting every node of a weighted complete graph exactly once. It is NP-hard, so practitioners use fast heuristics, and the basic question about a heuristic is how far from optimal its tour can be. Rosenkrantz, Stearns and Lewis (SIAM J. Comput. 6(3), 1977) gave the first systematic worst-case analysis of the simple constructive heuristics under the triangle inequality: nearest neighbor, the family of insertion methods, and several variants.

Insertion methods build a tour by growing it one node at a time. They are among the most widely used construction heuristics in practice and in textbooks, and they differ only in the rule that chooses which node to insert next: the nearest one, the cheapest one, the farthest one, a random one, or any other. This mission formalizes the paper's result that holds for the whole family at once, regardless of that rule: every insertion method produces a tour at most ⌈lg⁡n⌉+1\lceil \lg n\rceil + 1⌈lgn⌉+1 times longer than an optimal one (Theorem 3, p. 571).

Timeline. 1977: Rosenkrantz, Stearns and Lewis prove ⌈lg⁡n⌉+1\lceil\lg n\rceil+1⌈lgn⌉+1 for every insertion method (Theorem 3), 12(⌈lg⁡n⌉+1)\tfrac12(\lceil\lg n\rceil+1)21​(⌈lgn⌉+1) for nearest neighbor (Theorem 1), both from a shared counting lemma (Lemma 1), and the constant 222 for nearest and cheapest insertion (Theorem 4). 1994: Bafna, Kalyanasundaram and Pruhs (Theoretical Computer Science 125, 1994) give instances on which some insertion methods reach ratio Ω(log⁡n/log⁡log⁡n)\Omega(\log n/\log\log n)Ω(logn/loglogn), so the logarithmic growth cannot be replaced by a constant for the family as a whole.

Setting

A traveling salesman graph with nnn nodes consists of a finite node set NNN with ∣N∣=n|N|=n∣N∣=n and a distance d:N×N→Rd:N\times N\to\mathbb Rd:N×N→R with d(i,j)=d(j,i)d(i,j)=d(j,i)d(i,j)=d(j,i), d(i,j)≥0d(i,j)\ge 0d(i,j)≥0 and d(i,j)+d(j,k)≥d(i,k)d(i,j)+d(j,k)\ge d(i,k)d(i,j)+d(j,k)≥d(i,k) for all nodes (the triangle inequality). A tour visits every node once and returns to its start; its length is the sum of its edge lengths, and OPTIMAL is the least length of a tour.

A subtour is a tour on a subset of the nodes; a single node is a tour without edges. Given a subtour TTT and a node k∉Tk\notin Tk∈/T, TOUR(T,k)(T,k)(T,k) is obtained by choosing an edge (x,y)(x,y)(x,y) of TTT minimizing

d(x,k)+d(k,y)−d(x,y)d(x,k)+d(k,y)-d(x,y)d(x,k)+d(k,y)−d(x,y)

and replacing it by the edges (x,k)(x,k)(x,k) and (k,y)(k,y)(k,y); if TTT is a single node iii, TOUR(T,k)(T,k)(T,k) is the two-node tour (i,k),(k,i)(i,k),(k,i)(i,k),(k,i). COST(T,k)(T,k)(T,k) is the length of TOUR(T,k)(T,k)(T,k) minus the length of TTT.

An insertion method constructs subtours T1,…,TnT_1,\dots,T_nT1​,…,Tn​ with T1={a0}T_1=\{a_0\}T1​={a0​} a single node and Ti+1=TOUR(Ti,ai)T_{i+1}=\mathrm{TOUR}(T_i,a_i)Ti+1​=TOUR(Ti​,ai​) for some node ai∉Tia_i\notin T_iai​∈/Ti​, 1≤i<n1\le i<n1≤i<n. The final tour TnT_nTn​ is the approximation, and INSERT denotes its length. No rule for choosing the aia_iai​ is fixed, and ties between minimizing edges are broken arbitrarily.

Write lg⁡\lglg for the logarithm to base 2 and ⌈x⌉\lceil x\rceil⌈x⌉ for the least integer ≥x\ge x≥x.

Formalization targets

Goal: Theorem 3

For every traveling salesman graph with n≥1n\ge 1n≥1 nodes and every run of every insertion method,

INSERT ≤ (⌈lg⁡n⌉+1)⋅OPTIMAL.\mathrm{INSERT}\ \le\ \bigl(\lceil\lg n\rceil+1\bigr)\cdot\mathrm{OPTIMAL}.INSERT ≤ (⌈lgn⌉+1)⋅OPTIMAL.

Milestones

  1. (2.2), shortcutting: visiting a subset of the nodes in the order of a tour gives a tour of the subset that is no longer.
  2. (2.1): if the numbers l1≥⋯≥lnl_1\ge\dots\ge l_nl1​≥⋯≥ln​ satisfy d(p,q)≥min⁡(lp,lq)d(p,q)\ge\min(l_p,l_q)d(p,q)≥min(lp​,lq​) for distinct p,qp,qp,q, then OPTIMAL≥2∑i=k+1min⁡(2k,n)li\mathrm{OPTIMAL}\ge 2\sum_{i=k+1}^{\min(2k,n)} l_iOPTIMAL≥2∑i=k+1min(2k,n)​li​ for 1≤k≤n1\le k\le n1≤k≤n.
  3. Lemma 1: if d(p,q)≥min⁡(lp,lq)d(p,q)\ge\min(l_p,l_q)d(p,q)≥min(lp​,lq​) for distinct nodes and lp≤12OPTIMALl_p\le\frac12\mathrm{OPTIMAL}lp​≤21​OPTIMAL for all ppp, then
∑plp≤12(⌈lg⁡n⌉+1)OPTIMAL.\sum_p l_p\le\tfrac12\bigl(\lceil\lg n\rceil+1\bigr)\mathrm{OPTIMAL}.p∑​lp​≤21​(⌈lgn⌉+1)OPTIMAL.
  1. Lemma 2: COST(T,k)≤2 d(k,j)\mathrm{COST}(T,k)\le 2\,d(k,j)COST(T,k)≤2d(k,j) for every node jjj of TTT.
  2. (3.7): INSERT=∑i=1n−1COST(Ti,ai)\mathrm{INSERT}=\sum_{i=1}^{n-1}\mathrm{COST}(T_i,a_i)INSERT=∑i=1n−1​COST(Ti​,ai​).
  3. (3.10): COST(Ti,ai)≤2 d(ai,aj)\mathrm{COST}(T_i,a_i)\le 2\,d(a_i,a_j)COST(Ti​,ai​)≤2d(ai​,aj​) whenever j<ij<ij<i.
  4. (3.12): COST(Ti,ai)≤OPTIMAL\mathrm{COST}(T_i,a_i)\le\mathrm{OPTIMAL}COST(Ti​,ai​)≤OPTIMAL for 1≤i<n1\le i<n1≤i<n.

Significance

The result. Theorem 3 is a guarantee for an entire class of algorithms rather than for one. Any rule for choosing the next node, including rules designed for speed or for empirical quality, inherits a worst-case ratio of ⌈lg⁡n⌉+1\lceil\lg n\rceil+1⌈lgn⌉+1 from the insertion step alone. The rule matters only for improving on that: nearest and cheapest insertion achieve the constant 2(1−1/n)2(1-1/n)2(1−1/n) (Theorem 4 and its corollary, the subject of the third mission of this series), while the logarithmic bound remains the best general statement for other rules, such as farthest or arbitrary insertion. Lemma 1 is reusable on its own: it converts "every node carries a charge bounded by half the optimum and by its distance to other nodes" into a logarithmic bound, and the same lemma yields the nearest neighbor bound of Theorem 1.

Formalizing it. The theorem has been proved since 1977; the work here is a machine-checked proof of the known argument together with a reusable library for subtours, insertion and insertion costs. The companion nearest neighbor bound (Theorem 1) is already on the platform as SupplyChainTheory.nearest_neighbor_bound (proved), and nearest insertion with constant 2 as SupplyChainTheory.nearest_insertion_bound; neither covers arbitrary insertion methods or states Lemma 1 separately.

Difficulty

The per-step facts are local: each insertion is cheap relative to a node already present (Lemma 2) and relative to OPTIMAL (3.12). The obvious way to combine them, adding up n−1n-1n−1 costs each at most OPTIMAL, gives only the ratio n−1n-1n−1. The logarithm comes from a global counting argument over all nodes simultaneously (Lemma 1), in which OPTIMAL is compared with tours on nested subsets of nodes of doubling size, and the per-node charges must be matched against the edges of those tours. Formally, the delicate parts are the bookkeeping of subtours as they grow (that every earlier node lies on the current subtour, and that the insertion cost equals the length increase), the shortcutting of a tour to an arbitrary subset, and the ceiling-of-logarithm arithmetic.

Formalization scope

Nodes are Fin n; a tour of all nodes is a permutation τ : Equiv.Perm (Fin n), and OPTIMAL is the minimum of the tour length over the finite, nonempty set of permutations. Subtours are duplicate-free lists of nodes, with closed length d(x0,x1)+⋯+d(xm−1,x0)d(x_0,x_1)+\dots+d(x_{m-1},x_0)d(x0​,x1​)+⋯+d(xm−1​,x0​). TOUR(T,k)(T,k)(T,k) is encoded as inserting kkk at a list position whose resulting length is minimal among all positions; inserting at a position removes exactly one edge of TTT and raises the length by exactly d(x,k)+d(k,y)−d(x,y)d(x,k)+d(k,y)-d(x,y)d(x,k)+d(k,y)−d(x,y), so this is the paper's rule, with every tie-breaking allowed. COST is the minimum length increase over positions. The paper's 1-based subtour index is kept (T1=[a0]T_1=[a_0]T1​=[a0​], TnT_nTn​ final). ⌈lg⁡n⌉\lceil\lg n\rceil⌈lgn⌉ is Nat.clog 2 n. All quantities are real.

Conventions and deviations, each disclosed in the item statements:

  • The distance satisfies d(i,i)=0d(i,i)=0d(i,i)=0, a normalization not in the paper; a loop never enters any length.
  • Ratios are multiplied out (INSERT≤c⋅OPTIMAL\mathrm{INSERT}\le c\cdot\mathrm{OPTIMAL}INSERT≤c⋅OPTIMAL), so the paper's exclusion of the identically zero distance (1.1) is not needed.
  • Condition a) of Lemma 1 is required for distinct nodes only. The page says "for all nodes ppp and qqq", which for p=qp=qp=q would force every lp≤0l_p\le 0lp​≤0 and make the lemma inapplicable in the proof of Theorem 3; the proof uses the condition only on edges of a tour.
  • (2.2) is stated for every subset of the nodes and every tour, which is what the shortcut argument shows; the paper applies it to one specific subset and an optimal tour.
  • (2.1) uses 0-based node labels, so its range k+1,…,min⁡(2k,n)k+1,\dots,\min(2k,n)k+1,…,min(2k,n) becomes k,…,min⁡(2k,n)−1k,\dots,\min(2k,n)-1k,…,min(2k,n)−1.

The goal quantifies over every run: any choice of the inserted nodes aia_iai​ and any minimizing insertion position. Adding a selection rule (nearest, cheapest) or fixing a tie-breaking would state a weaker, different theorem; restricting to instances with OPTIMAL =0=0=0 or to a fixed small nnn would trivialize it.

Reusable beyond this mission: the subtour and insertion library (closed length of a list, TOUR, COST, insertion runs) and Lemma 1, which also yields Theorem 1. Contributions welcome: proofs of the milestones, general lemmas about the closed length of List.insertIdx and of filtered lists, and a proof of Theorem 1 from this mission's Lemma 1.

Selected references

  • D. J. Rosenkrantz, R. E. Stearns, P. M. Lewis II, An Analysis of Several Heuristics for the Traveling Salesman Problem, SIAM Journal on Computing 6(3):563–581, 1977. https://doi.org/10.1137/0206041
  • V. Bafna, B. Kalyanasundaram, K. Pruhs, Not all insertion methods yield constant approximate tours in the Euclidean plane, Theoretical Computer Science 125(2):345–353, 1994.
10 thms2 active usersReviewed
🏆Completed
Dynamic ProgrammingTheoretical Computer Science·Captain: mikedeng1

A Faster Algorithm Computing String Edit Distances 1: for a finite alphabet and discrete costs, the block algorithm (Algorithms Y and Z) computes the edit distance from a finite tableResearch Paper

Motivation

The edit distance between two strings is the least total cost of a sequence of single-character insertions, deletions and replacements turning one string into the other. It is the basic similarity measure of spelling correction, file comparison and biological sequence alignment. Wagner and Fischer (JACM 1974) showed that it can be computed by filling a (∣A∣+1)×(∣B∣+1)(|A|+1) \times (|B|+1)(∣A∣+1)×(∣B∣+1) matrix in O(∣A∣⋅∣B∣)O(|A|\cdot|B|)O(∣A∣⋅∣B∣) time. Masek and Paterson (J. Comput. System Sci. 1980) gave the first asymptotic improvement: for a finite alphabet and edit costs that are integral multiples of a common constant, the edit distance can be computed in time O(∣A∣⋅∣B∣/max⁡(1,∣B∣/log⁡∣A∣))O(|A|\cdot|B|/\max(1, |B|/\log|A|))O(∣A∣⋅∣B∣/max(1,∣B∣/log∣A∣)), that is O(n2/log⁡n)O(n^2/\log n)O(n2/logn) for two strings of length nnn.

Timeline:

  • 1970: Arlazarov, Dinic, Kronrod and Faradzev compute transitive closures by precomputing all small submatrices, the "four Russians" technique that Masek and Paterson adapt.
  • 1974: Wagner and Fischer give the matrix-filling algorithm, with the first row and column of the matrix and the three-term recurrence for its interior (Theorems 1 and 2 of Masek–Paterson, cited from them).
  • 1980: Masek and Paterson apply the four-Russians technique to the edit matrix, working with differences of adjacent entries, and show that the restriction to discrete costs cannot simply be dropped (their Section 4, the subject of the second mission of this series).
  • 2015: Backurs and Indyk (arXiv:1412.0348) show that a strongly subquadratic algorithm would refute the Strong Exponential Time Hypothesis, so a logarithmic-factor speed-up of this kind is close to the best one can expect.

Setting

Let Σ\SigmaΣ be an alphabet and λ\lambdaλ the null string. For a string AAA, ∣A∣|A|∣A∣ is its length, AnA_nAn​ its nnn-th character, Ai,j=Ai⋯AjA^{i,j} = A_i \cdots A_jAi,j=Ai​⋯Aj​ and Ai=A1,iA^i = A^{1,i}Ai=A1,i, with A0=λA^0 = \lambdaA0=λ.

An edit operation a→ba \to ba→b is a pair (a,b)≠(λ,λ)(a, b) \ne (\lambda, \lambda)(a,b)=(λ,λ) of strings of length at most one: a replacement (a,b≠λa, b \ne \lambdaa,b=λ, possibly a=ba = ba=b), a deletion (b=λb = \lambdab=λ) or an insertion (a=λa = \lambdaa=λ). BBB results from AAA via a→ba \to ba→b if A=σaτA = \sigma a \tauA=σaτ and B=σbτB = \sigma b \tauB=σbτ. An edit sequence S=s1,…,smS = s_1, \dots, s_mS=s1​,…,sm​ takes AAA to BBB if there are strings A=C0,C1,…,Cm=BA = C_0, C_1, \dots, C_m = BA=C0​,C1​,…,Cm​=B with Ci−1→CiC_{i-1} \to C_iCi−1​→Ci​ via sis_isi​. A cost function γ\gammaγ assigns a nonnegative real to every edit operation, γ(S)=∑iγ(si)\gamma(S) = \sum_i \gamma(s_i)γ(S)=∑i​γ(si​), and

δ(γ,A,B)=min⁡{γ(S)∣S takes A to B}.\delta(\gamma, A, B) = \min\{\gamma(S) \mid S \text{ takes } A \text{ to } B\}.δ(γ,A,B)=min{γ(S)∣S takes A to B}.

Write Ra,b=γ(a→b)R_{a,b} = \gamma(a \to b)Ra,b​=γ(a→b), Da=γ(a→λ)D_a = \gamma(a \to \lambda)Da​=γ(a→λ), Ia=γ(λ→a)I_a = \gamma(\lambda \to a)Ia​=γ(λ→a), and δi,j=δ(γ,Ai,Bj)\delta_{i,j} = \delta(\gamma, A^i, B^j)δi,j​=δ(γ,Ai,Bj) for the entries of the edit matrix. The cost function is normalized if γ(a→b)=δ(γ,a,b)\gamma(a \to b) = \delta(\gamma, a, b)γ(a→b)=δ(γ,a,b) for every edit operation.

A step is a difference of two adjacent matrix entries, δi,j−δi−1,j\delta_{i,j} - \delta_{i-1,j}δi,j​−δi−1,j​ (vertical) or δi,j−δi,j−1\delta_{i,j} - \delta_{i,j-1}δi,j​−δi,j−1​ (horizontal). The cost set is Ω={Da}∪{Ia}∪{Ra,b}\Omega = \{D_a\} \cup \{I_a\} \cup \{R_{a,b}\}Ω={Da​}∪{Ia​}∪{Ra,b​}, and Ω\OmegaΩ is discrete if every element of Ω\OmegaΩ is an integral multiple of one constant r>0r > 0r>0.

Algorithm Y takes two strings C,DC, DC,D of length mmm and two step vectors R,SR, SR,S of length mmm (the left column and top row of an m×mm \times mm×m block) and fills a matrix TTT of vertical steps and UUU of horizontal steps by the recurrence of Corollary 1, returning the right column R′R'R′ and bottom row S′S'S′. Algorithm Z cuts AAA and BBB into blocks of length mmm, starts from the deletion costs of AAA and the insertion costs of BBB, obtains the steps of each block from Algorithm Y's output ("Fetch"), and returns the sum of the steps along the left column and the bottom row.

Formalization targets

Goal: correctness from a string-independent finite table

For a finite alphabet and a nonnegative, normalized cost function with discrete Ω\OmegaΩ, there is a finite set T⊂RT \subset \mathbb{R}T⊂R such that for all m≥1m \ge 1m≥1 and all A,BA, BA,B with m∣∣A∣m \mid |A|m∣∣A∣, m∣∣B∣m \mid |B|m∣∣B∣,

costZ(γ,m,A,B)=δ(γ,A,B),every entry of every P(i,j),Q(i,j) lies in T.\mathrm{cost}_{Z}(\gamma, m, A, B) = \delta(\gamma, A, B), \qquad \text{every entry of every } P(i,j), Q(i,j) \text{ lies in } T.costZ​(γ,m,A,B)=δ(γ,A,B),every entry of every P(i,j),Q(i,j) lies in T.

TTT is fixed before mmm, AAA and BBB. The second clause says that Algorithm Y's table need only range over Σm×Σm×Tm×Tm\Sigma^m \times \Sigma^m \times T^m \times T^mΣm×Σm×Tm×Tm, whose size does not depend on the strings; this is what the running-time bound rests on.

Milestones

  1. Theorem 1 [Wagner–Fischer]: δ0,0=0\delta_{0,0} = 0δ0,0​=0, δi,0=∑r≤iDAr\delta_{i,0} = \sum_{r \le i} D_{A_r}δi,0​=∑r≤i​DAr​​, δ0,j=∑r≤jIBr\delta_{0,j} = \sum_{r \le j} I_{B_r}δ0,j​=∑r≤j​IBr​​.
  2. Theorem 2 [Wagner–Fischer]: δi,j=min⁡(δi−1,j−1+RAi,Bj,δi−1,j+DAi,δi,j−1+IBj)\delta_{i,j} = \min(\delta_{i-1,j-1} + R_{A_i,B_j}, \delta_{i-1,j} + D_{A_i}, \delta_{i,j-1} + I_{B_j})δi,j​=min(δi−1,j−1​+RAi​,Bj​​,δi−1,j​+DAi​​,δi,j−1​+IBj​​).
  3. Corollary 1: the same recurrence written in terms of steps.
  4. Algorithm Y returns the final step vectors of every m×mm \times mm×m submatrix from its initial step vectors and strings (Section 2.1).
  5. Lemma 3: −I≤δi,j−δi−1,j≤D-I \le \delta_{i,j} - \delta_{i-1,j} \le D−I≤δi,j​−δi−1,j​≤D and −D≤δi,j−δi,j−1≤I-D \le \delta_{i,j} - \delta_{i,j-1} \le I−D≤δi,j​−δi,j−1​≤I.
  6. Lemma 4: if Ω\OmegaΩ is discrete, the set of possible steps is finite.

Significance

The result was the first algorithm for edit distance faster than quadratic, and its method (tabulate every possible small block of a dynamic program, described by differences rather than values) became the standard way of shaving a logarithmic factor from string dynamic programs. The discreteness hypothesis is where the method's power ends: Section 4 of the paper shows that with costs 111 and π\piπ the number of distinct steps grows without bound.

The results are proved in the paper; none of them is formalized. The Mathlib revision of this mission contains no edit-distance module. A formalization produces a definition of edit distance as a minimum over edit sequences, a machine-checked proof of the Wagner–Fischer recurrence for that definition under the normalization the paper assumes, and a checked proof that the four-Russians block assembly is correct and draws on a finite, string-independent table.

Difficulty

The hardest step is Theorem 2 for δ\deltaδ defined as a minimum over arbitrary edit sequences. An edit sequence may insert a character and later replace or delete it, or edit the same position many times, so the edit matrix's three-term recurrence does not follow by looking at the last operation. The upper bound is direct; the lower bound needs a normal form for edit sequences, and it fails without normalization: with Ra,c=10R_{a,c} = 10Ra,c​=10, Ra,b=Rb,c=1R_{a,b} = R_{b,c} = 1Ra,b​=Rb,c​=1 and all insertions and deletions costing 100100100, δ(γ,a,c)=2\delta(\gamma, a, c) = 2δ(γ,a,c)=2 while the recurrence gives 101010.

The goal is then an induction over blocks that must keep track of which matrix entries each block's input and output vectors represent, with block boundaries at multiples of mmm and the first row and column handled by Theorem 1.

Formalization scope

Strings are List α; characters are 1-based in all statements, as in the paper (AiA_iAi​ is A[i-1]). An edit operation is a structure with two Option α fields, not both none. δ\deltaδ is sInf of the set of costs of edit sequences taking AAA to BBB (nonempty, and bounded below for γ≥0\gamma \ge 0γ≥0). Costs are real-valued with an explicit nonnegativity hypothesis. Normalization is a hypothesis on Theorems 1, 2, Corollary 1, the Algorithm Y lemma and the goal; Lemmas 3 and 4 hold without it. The finite alphabet is [Fintype α] on Lemma 4 and the goal; Lemma 3 is stated for arbitrary upper bounds I≥IaI \ge I_aI≥Ia​, D≥DaD \ge D_aD≥Da​, which implies the paper's version with maxima. The paper's standing assumption ∣A∣≥∣B∣|A| \ge |B|∣A∣≥∣B∣ serves only the running time and is dropped. Step vectors are functions on Fin m.

Pinned statements. The paper states a running time O(∣A∣⋅∣B∣/max⁡(1,∣B∣/log⁡∣A∣))O(|A|\cdot|B|/\max(1,|B|/\log|A|))O(∣A∣⋅∣B∣/max(1,∣B∣/log∣A∣)) on a logarithmic-cost RAM; the goal formalizes the two facts that bound rests on, correctness and a finite table domain fixed before the strings. The RAM model, operation counts, the choice m=⌊log⁡k∣A∣⌋m = \lfloor \log_k |A| \rfloorm=⌊logk​∣A∣⌋, the padding reduction for m∤∣A∣m \nmid |A|m∤∣A∣, and the edit-path recovery of Section 2.3 are not formalized. Algorithm Y's result is a function (blockY); the Store/Fetch memory is not modelled.

The edit distance must stay the minimum over edit sequences: defining it by the Wagner–Fischer recurrence would make Theorems 1 and 2 true by definition and reduce the goal to a comparison of two recurrences. Algorithms Y and Z are transcribed from the pseudo-code and never refer to δ\deltaδ.

Useful beyond this mission: the §1.1 definitions and Theorems 1–2 are a general edit-distance library (the second mission of this series defines the same objects). Contributions of lemmas about normal forms of edit sequences, the triangle inequality for δ\deltaδ, and attainment of the minimum are welcome.

Selected references

  • W. J. Masek, M. S. Paterson, A Faster Algorithm Computing String Edit Distances, J. Comput. System Sci. 20 (1980), 18–31. https://doi.org/10.1016/0022-0000(80)90002-1
  • R. A. Wagner, M. J. Fischer, The String-to-String Correction Problem, J. ACM 21 (1974), 168–173. https://doi.org/10.1145/321796.321811
  • V. L. Arlazarov, E. A. Dinic, M. A. Kronrod, I. A. Faradzev, On Economical Construction of the Transitive Closure of an Oriented Graph, Soviet Math. Dokl. 11 (1970), 1209–1210.
  • A. Backurs, P. Indyk, Edit Distance Cannot Be Computed in Strongly Subquadratic Time (unless SETH is false), STOC 2015. https://arxiv.org/abs/1412.0348
10 thms2 active usersReviewed
🏆Completed
Graph TheoryLinear OptimizationOperations Research·Captain: mikedeng1

On Certain Polytopes Associated with Graphs IV: Adjacent Stable Sets on the Stable Set PolytopeResearch Paper

Motivation

Many combinatorial optimization problems are linear programs over a polytope whose vertices are the zero–one incidence vectors of the feasible objects: matchings, stable sets, spanning trees. The edges of such a polytope (pairs of vertices joined by a one-dimensional face) govern the behaviour of the simplex method and of local-search procedures, which move from vertex to vertex along edges: a pivot of the simplex method on a nondegenerate basis replaces a vertex by one of its neighbours.

In December 1971 M. L. Balinski asked when two matchings M1,M2M_1, M_2M1​,M2​ of a graph are neighbours on the matching polyhedron determined by Edmonds (Edmonds 1965). V. Chvátal answered a more general question in §6 of On certain polytopes associated with graphs (Chvátal 1975): he characterized the neighbours on the stable set polytope of an arbitrary graph. Since matchings of GGG are the stable sets of the line graph L(G)L(G)L(G), Balinski's question is the special case of line graphs (Corollary 6.3 of the paper).

Setting

Let G=(V,E)G=(V,E)G=(V,E) be a finite undirected loopless graph. A stable set is a set of vertices no two of which are adjacent. S(G)S(G)S(G) denotes the set of all zero–one vectors x=(xu:u∈V)x=(x_u : u\in V)x=(xu​:u∈V) such that {u:xu=1}\{u : x_u=1\}{u:xu​=1} is stable, and the stable set polytope is

P(G)=conv⁡S(G)⊆RV.P(G)=\operatorname{conv} S(G)\subseteq \mathbb R^V .P(G)=convS(G)⊆RV.

For y∈S(G)y\in S(G)y∈S(G) the corresponding stable set is Y={u:yu=1}Y=\{u : y_u=1\}Y={u:yu​=1}.

For an integer-valued vector c=(cu:u∈V)c=(c_u : u\in V)c=(cu​:u∈V) write cx=∑u∈Vcuxucx=\sum_{u\in V}c_ux_ucx=∑u∈V​cu​xu​. Two vectors y,zy, zy,z are neighbours in P(G)P(G)P(G) if there is an integer-valued ccc such that yyy and zzz are the only two vectors which maximize cxcxcx over S(G)S(G)S(G); in particular y≠zy\neq zy=z. This is the definition the paper states at the start of the proof of Theorem 6.2.

A bicoloration of a graph TTT is a partition V=B∪RV=B\cup RV=B∪R, B∩R=∅B\cap R=\emptysetB∩R=∅, such that every edge joins BBB to RRR. Every tree has one.

In the Lean development these objects are stableVectors G (S(G)S(G)S(G)), stablePolytope G (P(G)P(G)P(G)), onesSet y (YYY), AreNeighbors G y z and IsBicoloration T B R, all in the namespace ChvatalPolytopes.Neighbors.

Formalization targets

Goal: Theorem 6.2 (p. 149)

For y,z∈S(G)y,z\in S(G)y,z∈S(G) with corresponding stable sets Y,ZY,ZY,Z,

y and z are neighbours in P(G)  ⟺  the subgraph H of G induced by (Y−Z)∪(Z−Y) is connected.y \text{ and } z \text{ are neighbours in } P(G) \iff \text{the subgraph } H \text{ of } G \text{ induced by } (Y-Z)\cup(Z-Y) \text{ is connected.}y and z are neighbours in P(G)⟺the subgraph H of G induced by (Y−Z)∪(Z−Y) is connected.

Milestone: Lemma 6.1 (p. 149)

For a tree T=(V,E)T=(V,E)T=(V,E) with a bicoloration V=B∪RV=B\cup RV=B∪R there are nonnegative integers cuc_ucu​ (u∈Vu\in Vu∈V) and mmm with

∑u∈Vcuxu≤mfor all x∈S(T),\sum_{u\in V}c_ux_u\le m\quad\text{for all } x\in S(T),u∈V∑​cu​xu​≤mfor all x∈S(T),

with equality exactly when xxx is the incidence vector of BBB or of RRR.

Milestone: the certificate of the "if" part (p. 149, proof of Theorem 6.2, (i))

If HHH is connected with spanning tree TTT, and cuc_ucu​ (u∈(Y−Z)∪(Z−Y)u\in (Y-Z)\cup(Z-Y)u∈(Y−Z)∪(Z−Y)), mmm are as in Lemma 6.1 for TTT, extend ccc by cu=1c_u=1cu​=1 on Y∩ZY\cap ZY∩Z and cu=−1c_u=-1cu​=−1 outside Y∪ZY\cup ZY∪Z. Then

∑u∈Vcuxu≤m+∣Y∩Z∣for all x∈S(G),\sum_{u\in V}c_ux_u\le m+|Y\cap Z|\quad\text{for all } x\in S(G),u∈V∑​cu​xu​≤m+∣Y∩Z∣for all x∈S(G),

with equality if and only if x=yx=yx=y or x=zx=zx=z.

Significance

Theorem 6.2 describes the 1-skeleton of the stable set polytope of every graph by a condition that can be checked in linear time, although optimizing over P(G)P(G)P(G) is NP-hard in general and no complete linear description of P(G)P(G)P(G) is known for general graphs. Through line graphs it gives the adjacency criterion for the matching polytope (two matchings are neighbours if and only if their symmetric difference is a single path or cycle), which settled Balinski's question. Characterizations of this type underlie the analysis of simplex-type and pivoting algorithms on combinatorial polytopes and the study of their diameters.

The result has been proved since 1975. The mission asks for a machine-checked proof of the theorem as stated in the paper; no formal proof of Theorem 6.2 or of the matching-polytope corollary is known to exist on Prove2Me or in Mathlib. The two milestones isolate the constructive half (Lemma 6.1 and the weighting built from it), which is reusable for any statement that needs an explicit objective singling out two stable sets.

Difficulty

The "only if" direction and the equality analysis are elementary; the substance lies in the "if" direction. An objective that makes both yyy and zzz optimal is easy to write down, for example c=y+zc=y+zc=y+z; the difficulty is to make them the only optimal vectors. Any stable set that agrees with YYY on some connected pieces of HHH and with ZZZ on others ties with yyy and zzz under naive weightings, so the weights on (Y−Z)∪(Z−Y)(Y-Z)\cup(Z-Y)(Y−Z)∪(Z−Y) must be chosen so that every mixed choice loses strictly. The integrality requirement on ccc and the need to control all of S(G)S(G)S(G), not only the stable sets contained in Y∪ZY\cup ZY∪Z, rule out a direct perturbation argument.

Formalization scope

  • Graphs. VVV is a finite type with decidable equality and GGG is a SimpleGraph V; loops and multiple edges are excluded, as in the paper.
  • S(G)S(G)S(G) and P(G)P(G)P(G). S(G)S(G)S(G) is the set of incidence vectors in V → ℝ of stable finsets; P(G)P(G)P(G) is convexHull ℝ (S G).
  • Neighbours. Defined exactly as on p. 149: y≠zy\ne zy=z and, for some c:V→Zc : V\to\mathbb Zc:V→Z, the set of maximizers of cxcxcx over S(G)S(G)S(G) equals {y,z}\{y,z\}{y,z}. The face-lattice notion of an edge of P(G)P(G)P(G) is not used; its equivalence with this definition is not part of the paper.
  • Induced subgraph and connectedness. HHH is G.induce of the set (Y∖Z)∪(Z∖Y)(Y\setminus Z)\cup(Z\setminus Y)(Y∖Z)∪(Z∖Y), and "connected" is Mathlib's SimpleGraph.Connected, which requires at least one vertex. For y=zy=zy=z both sides of the goal are therefore false.
  • Trees. SimpleGraph.IsTree, which includes connectedness; a spanning tree of HHH is a graph TTT on the vertex set of HHH with T≤HT\le HT≤H and T.IsTree. In Lemma 6.1 the integers cuc_ucu​ and mmm are natural numbers.

A trivializing formalization — defining neighbours through the symmetric-difference condition or through Lemma 6.1's certificate, or omitting y≠zy\neq zy=z from the definition — is excluded: neighbours are defined only through unique maximizers of integer objectives over S(G)S(G)S(G).

A complete development needs only finite graphs, induced subgraphs, spanning trees of connected graphs (available in Mathlib) and finite sums. Contributions welcome beyond the milestones: the equivalence of this notion of neighbours with the one-dimensional faces of P(G)P(G)P(G), and Corollary 6.3 for the matching polytope via line graphs.

Selected references

  • V. Chvátal, On certain polytopes associated with graphs, Journal of Combinatorial Theory, Series B 18 (1975), 138–154. https://doi.org/10.1016/0095-8956(75)90041-6
  • J. Edmonds, Maximum matching and a polyhedron with 0,1-vertices, Journal of Research of the National Bureau of Standards 69B (1965), 125–130. https://doi.org/10.6028/jres.069B.013
  • M. W. Padberg, On the facial structure of set packing polyhedra, Mathematical Programming 5 (1973), 199–215. https://doi.org/10.1007/BF01580121
6 thms2 active usersReviewed
🏆Completed
Graph TheoryLinear OptimizationOperations Research·Captain: mikedeng1

On Certain Polytopes Associated with Graphs II: No Clique Is a Cutset of a Connected α-Critical GraphResearch Paper

Motivation

The stability number α(G)\alpha(G)α(G) of a graph, the largest number of pairwise non-adjacent vertices, is the optimum of an integer program over the stable set polytope P(G)P(G)P(G). Linear programming duality turns any explicit linear description of P(G)P(G)P(G) into a certificate of optimality for α(G)\alpha(G)α(G), which is why the question "which inequalities are needed to describe P(G)P(G)P(G)?" has been central to polyhedral combinatorics since Edmonds' description of the matching polytope (Edmonds 1965). Chvátal's 1975 paper (doi:10.1016/0095-8956(75)90041-6) initiated the systematic study of P(G)P(G)P(G) for arbitrary graphs: which graph operations preserve a known description, and which inequalities are facets, i.e. indispensable in every description.

Section 4 of the paper treats one such operation, gluing two graphs along a complete subgraph, and one family of facets, the "rank" inequality ∑uxu≤α(G)\sum_u x_u\le\alpha(G)∑u​xu​≤α(G) for graphs whose critical edges connect all vertices. Combining the two yields a purely graph-theoretic fact about α\alphaα-critical graphs (graphs in which deleting any edge increases the stability number): no complete subgraph separates such a graph. The fact is due to Berge (Graphes et hypergraphes, 1970, Ch. 13, §3, Corollary 2); Chvátal's derivation obtains it from polyhedral arguments. α\alphaα-critical graphs were studied by Erdős and Gallai, Hajnal, Andrásfai and Lovász, and their structure is closely tied to the facets of P(G)P(G)P(G).

Setting

Graphs are finite, undirected and loopless: G=(V,E)G=(V,E)G=(V,E). A stable set is a set of pairwise non-adjacent vertices; α(G)\alpha(G)α(G) is the largest size of a stable set. The incidence vector of s⊆Vs\subseteq Vs⊆V is χs∈RV\chi^s\in\mathbb R^Vχs∈RV with χus=1\chi^s_u=1χus​=1 for u∈su\in su∈s and 000 otherwise. S(G)S(G)S(G) is the set of incidence vectors of stable sets and

P(G)=conv⁡S(G)⊆RV.P(G)=\operatorname{conv}S(G)\subseteq\mathbb R^V .P(G)=convS(G)⊆RV.

A finite system ∑u∈Vaiuxu≤bi\sum_{u\in V}a_{iu}x_u\le b_i∑u∈V​aiu​xu​≤bi​ (i∈J)(i\in J)(i∈J) is a defining linear system of PPP if its solution set is exactly PPP. An inequality ∑uauxu≤b\sum_u a_ux_u\le b∑u​au​xu​≤b is a facet of PPP if every defining linear system of PPP contains, for some t>0t>0t>0, the inequality ∑utauxu≤tb\sum_u ta_ux_u\le tb∑u​tau​xu​≤tb.

An edge eee of GGG is critical if α(G−e)=α(G)+1\alpha(G-e)=\alpha(G)+1α(G−e)=α(G)+1; E∗E^*E∗ denotes the set of critical edges, G∗=(V,E∗)G^*=(V,E^*)G∗=(V,E∗), and GGG is α\alphaα-critical if every edge is critical. For graphs G1=(V1,E1)G_1=(V_1,E_1)G1​=(V1​,E1​), G2=(V2,E2)G_2=(V_2,E_2)G2​=(V2​,E2​) put G1∩G2=(V1∩V2,E1∩E2)G_1\cap G_2=(V_1\cap V_2,E_1\cap E_2)G1​∩G2​=(V1​∩V2​,E1​∩E2​) and G1∪G2=(V1∪V2,E1∪E2)G_1\cup G_2=(V_1\cup V_2,E_1\cup E_2)G1​∪G2​=(V1​∪V2​,E1​∪E2​). A vertex set KKK is a cutset of GGG if two vertices outside KKK are joined by no path of G−KG-KG−K, the subgraph induced on V∖KV\setminus KV∖K.

In Lean, all objects live in the namespace ChvatalPolytopes.Separation: stablePolytope G, IsFacet P a b, IsCriticalEdge, criticalGraph G (for G∗G^*G∗), IsAlphaCritical G and IsCutset G K.

Formalization targets

Goal: Corollary 4.3 (p. 144)

For a finite connected α\alphaα-critical graph GGG and any K⊆VK\subseteq VK⊆V inducing a complete subgraph,

K is not a cutset of G.K \text{ is not a cutset of } G .K is not a cutset of G.

The goal is pure graph theory; its proof in the paper consists of the two polyhedral theorems below.

Milestones

  1. Proposition 2.1 (pp. 139–140). For a finite nonempty set SSS of solutions of −xu≤0-x_u\le0−xu​≤0 (u∈V)(u\in V)(u∈V), ∑uaiuxu≤bi\sum_u a_{iu}x_u\le b_i∑u​aiu​xu​≤bi​ (i∈J)(i\in J)(i∈J): the solution set equals conv⁡S\operatorname{conv}SconvS if and only if for every c∈ZVc\in\mathbb Z^Vc∈ZV
max⁡{cx:x∈S}=min⁡{∑iλibi:λ≥0, ∑iλiaiu≥cu (u∈V)}.\max\{cx:x\in S\}=\min\Big\{\sum_i\lambda_ib_i:\lambda\ge0,\ \sum_i\lambda_ia_{iu}\ge c_u\ (u\in V)\Big\}.max{cx:x∈S}=min{i∑​λi​bi​:λ≥0, i∑​λi​aiu​≥cu​ (u∈V)}.
  1. Theorem 4.1 (p. 141). If G1∩G2G_1\cap G_2G1​∩G2​ is complete, the union of defining linear systems of P(G1)P(G_1)P(G1​) and P(G2)P(G_2)P(G2​) (each containing its nonnegativity rows) is a defining linear system of P(G1∪G2)P(G_1\cup G_2)P(G1​∪G2​).
  2. Theorem 4.2 (p. 143). If G∗G^*G∗ is connected, then
∑u∈Vxu≤α(G)\sum_{u\in V}x_u\le\alpha(G)u∈V∑​xu​≤α(G)

is a facet of P(G)P(G)P(G).

Significance

Theorem 4.1 says that clique-sums are harmless for linear descriptions of P(G)P(G)P(G): a description of a graph glued along a clique is the union of descriptions of the pieces. It underlies the later decomposition theory of stable set polytopes (clique cutsets appear throughout the study of perfect and ttt-perfect graphs). Theorem 4.2 supplies a large class of facets with a combinatorial certificate, and was the starting point of the study of rank facets. Corollary 4.3 illustrates how polyhedral statements yield structural graph theory: the facet in Theorem 4.2 cannot coexist with a clique cutset.

All three results are proved in the paper, and Berge's corollary was known before it. None of them has, to the knowledge of this mission, a machine-checked proof; Mathlib has stable sets (IsIndepSet, indepNum), cliques and convex hulls, but no stable set polytope, no notion of facet via defining systems, and no α\alphaα-critical graphs. The mission produces these definitions and the formal proofs of Proposition 2.1, Theorems 4.1, 4.2 and Corollary 4.3.

Difficulty

Proposition 2.1 requires LP duality in the form "min = max with both optima attained" together with a separation argument that reduces arbitrary objectives to integral ones; the "if" direction fails without the nonnegativity rows, so the statement is sensitive to the exact form of the system. In Theorem 4.1 the inclusion P(G1∪G2)⊆P(G_1\cup G_2)\subseteqP(G1​∪G2​)⊆ (solutions of the union) is routine; the difficulty is the converse: a point whose restrictions lie in P(G1)P(G_1)P(G1​) and in P(G2)P(G_2)P(G2​) is a convex combination of stable sets on each side, and the two combinations have to be matched on the clique V1∩V2V_1\cap V_2V1​∩V2​ to produce stable sets of G1∪G2G_1\cup G_2G1​∪G2​. Theorem 4.2 concerns every defining linear system, so it cannot be proved by exhibiting one description; the natural route via "affinely independent tight points" is a different definition of facet and needs full-dimensionality of P(G)P(G)P(G) to be equivalent. Finally, the goal requires translating a cutset into a decomposition G=G1∪G2G=G_1\cup G_2G=G1​∪G2​ with complete intersection, and then showing that a union of two systems on smaller vertex sets cannot contain a positive multiple of ∑u∈Vxu≤α(G)\sum_{u\in V}x_u\le\alpha(G)∑u∈V​xu​≤α(G).

Formalization scope

  • Graphs are SimpleGraph V on a Fintype V with DecidableEq V. S(G)S(G)S(G) is a set of functions V → ℝ (incidence vectors of stable finsets), and P(G)P(G)P(G) is convexHull ℝ (stableVectors G).
  • Linear systems are indexed by finite types with real coefficients. "Defining linear system" is equality of the solution set with the polytope. IsFacet quantifies over all finite index types J : Type and all real systems whose solution set equals the polytope; it is the paper's definition, not the affinely-independent-points characterization.
  • Proposition 2.1: "min = max" means an attained minimum equal to the maximum; the hypothesis S≠∅S\neq\emptysetS=∅ is added (the paper's max⁡\maxmax over SSS needs it), and the nonnegativity rows are kept.
  • Theorem 4.1: the glued graph GGG lives on a type VVV with finsets V1∪V2=VV_1\cup V_2=VV1​∪V2​=V; G1,G2G_1,G_2G1​,G2​ are the induced subgraphs on V1,V2V_1,V_2V1​,V2​; "G1∩G2G_1\cap G_2G1​∩G2​ complete" is encoded as "V1∩V2V_1\cap V_2V1​∩V2​ is a clique of GGG and no edge joins V1−V2V_1-V_2V1​−V2​ to V2−V1V_2-V_1V2​−V1​", which is equivalent to the paper's hypotheses. The rows of each system are evaluated on the restriction of xxx.
  • Theorem 4.2: "G∗G^*G∗ connected" is Mathlib's Connected, which requires V≠∅V\neq\emptysetV=∅ — for V=∅V=\emptysetV=∅ the statement would be false. α(G)\alpha(G)α(G) is indepNum, cast to R\mathbb RR.
  • Corollary 4.3: "complete subgraph" is any clique set G.IsClique K, not only maximal cliques (the paper reserves "clique" for maximal complete subgraphs, but the corollary speaks of complete subgraphs), including K=∅K=\emptysetK=∅. "Cutset" means two vertices outside KKK joined by no path of G−KG-KG−K. The formalization "G−KG-KG−K is not connected" is ruled out: under Mathlib's convention it would make K=VK=VK=V a cutset and the statement false for K1K_1K1​ and K2K_2K2​.
  • Reusable infrastructure: the stable set polytope, facets via defining systems, Proposition 2.1 (shared with the other missions of this series), critical edges and α\alphaα-critical graphs. Contributions of intermediate lemmas (LP duality in the attained form, full-dimensionality of P(G)P(G)P(G), the cutset–decomposition equivalence) are welcome.

Selected references

  • V. Chvátal, On certain polytopes associated with graphs, J. Combin. Theory Ser. B 18 (1975) 138–154. https://doi.org/10.1016/0095-8956(75)90041-6
  • C. Berge, Graphes et hypergraphes, Dunod, Paris, 1970 (English translation: Graphs and Hypergraphs, North-Holland, 1973), Chapter 13, §3.
  • J. Edmonds, Maximum matching and a polyhedron with 0,1-vertices, J. Res. Nat. Bur. Standards 69B (1965) 125–130. https://doi.org/10.6028/jres.069B.013
  • M. W. Padberg, On the facial structure of set packing polyhedra, Math. Programming 5 (1973) 199–215. https://doi.org/10.1007/BF01580121
  • L. Lovász, Normal hypergraphs and the perfect graph conjecture, Discrete Math. 2 (1972) 253–267. https://doi.org/10.1016/0012-365X(72)90006-4
8 thms2 active usersReviewed
🏆Completed
Graph TheoryLinear OptimizationOperations Research·Captain: mikedeng1

On Certain Polytopes Associated with Graphs I: Clique Inequalities Define the Stable Set Polytope Exactly for Perfect GraphsResearch Paper

Motivation

Many combinatorial optimization problems ask for the best subset of a finite set subject to combinatorial side conditions. The polyhedral method replaces the finite family of feasible subsets by the convex hull of their incidence vectors and asks for an explicit system of linear inequalities describing that convex hull; once such a system is known, linear programming duality gives min–max theorems and certificates of optimality. The maximum weight stable set problem is the central test case: it is NP-hard in general, so no tractable complete description of its polytope is expected for all graphs, and the question becomes for which graphs a simple description suffices.

V. Chvátal's 1975 paper On certain polytopes associated with graphs answers this question for the two simplest families of valid inequalities, and its Section 3 connects the answer to Berge's perfect graphs. The result is a standard entry point to polyhedral combinatorics and is one of the ingredients behind the later polynomial-time algorithms for stable sets in perfect graphs by Grötschel, Lovász and Schrijver.

Timeline. Berge (1961) introduced perfect graphs and conjectured that a graph is perfect if and only if its complement is. Lovász (Normal hypergraphs and the perfect graph conjecture, Discrete Math. 1972; A characterization of perfect graphs, J. Combin. Theory Ser. B 1972) proved this, together with the characterization of perfection by α(GA) ω(GA)≥∣A∣\alpha(G_A)\,\omega(G_A)\ge|A|α(GA​)ω(GA​)≥∣A∣ and the invariance of perfection under vertex duplication. Fulkerson's theory of antiblocking polyhedra (1971–72) gave a polyhedral route to the same equivalence. Chvátal (received 1972, published 1975) gave the self-contained polyhedral statement formalized here, with a proof based on Lovász's two theorems.

Setting

A graph G=(V,E)G=(V,E)G=(V,E) is finite, undirected and loopless. A stable set is a set of vertices no two of which are adjacent. A clique is a maximal complete subgraph, and C(G)C(G)C(G) is the set of vertex sets W⊆VW\subseteq VW⊆V of the cliques of GGG.

S(G)⊆RVS(G)\subseteq\mathbb R^VS(G)⊆RV is the set of zero–one vectors x=(xu:u∈V)x=(x_u:u\in V)x=(xu​:u∈V) such that {u:xu=1}\{u:x_u=1\}{u:xu​=1} is stable, and the stable set polytope is P(G)=conv⁡S(G)P(G)=\operatorname{conv}S(G)P(G)=convS(G). A finite system of linear inequalities is a defining linear system of P(G)P(G)P(G) if its solution set is exactly P(G)P(G)P(G). For c∈RVc\in\mathbb R^Vc∈RV write cx=∑u∈Vcuxucx=\sum_{u\in V}c_ux_ucx=∑u∈V​cu​xu​.

GGG is perfect (the paper's α\alphaα-perfect) if for every zero–one vector ccc,

max⁡{cx:x∈S(G)}=min⁡{∑W∈C(G)λW: λW∈{0,1}, ∑W∈C(G), u∈WλW≥cu (u∈V)}.\max\{cx:x\in S(G)\}=\min\Big\{\sum_{W\in C(G)}\lambda_W:\ \lambda_W\in\{0,1\},\ \sum_{W\in C(G),\,u\in W}\lambda_W\ge c_u\ (u\in V)\Big\}.max{cx:x∈S(G)}=min{W∈C(G)∑​λW​: λW​∈{0,1}, W∈C(G),u∈W∑​λW​≥cu​ (u∈V)}.

For A⊆VA\subseteq VA⊆V, GAG_AGA​ is the induced subgraph, α(GA)\alpha(G_A)α(GA​) its stability number and ω(GA)\omega(G_A)ω(GA​) its clique number. To duplicate a vertex uuu is to add a new vertex u′u'u′ adjacent to all neighbours of uuu but not to uuu.

In the Lean development these are stableVectors G, stablePolytope G, maximalCliques G, IsPerfect G and duplicate G u in the namespace ChvatalPolytopes.Perfect.

Formalization targets

Goal: Theorem 3.1 (p. 140)

For every graph GGG, the system

−xu≤0(u∈V),∑u∈Wxu≤1(W∈C(G))-x_u\le0\quad(u\in V),\qquad\sum_{u\in W}x_u\le1\quad(W\in C(G))−xu​≤0(u∈V),u∈W∑​xu​≤1(W∈C(G))

is a defining linear system of P(G)P(G)P(G) if and only if GGG is perfect. Both directions are required.

Milestones

  1. Proposition 2.1 (pp. 139–140). For a finite nonempty set SSS of solutions of −xu≤0-x_u\le0−xu​≤0, ∑uaiuxu≤bi\sum_u a_{iu}x_u\le b_i∑u​aiu​xu​≤bi​ (i∈J)(i\in J)(i∈J), the solution set equals conv⁡S\operatorname{conv}SconvS if and only if for every c∈ZVc\in\mathbb Z^Vc∈ZV
max⁡{cx:x∈S}=min⁡{∑iλibi:λ≥0, ∑iλiaiu≥cu (u∈V)}.\max\{cx:x\in S\}=\min\Big\{\sum_i\lambda_ib_i:\lambda\ge0,\ \sum_i\lambda_ia_{iu}\ge c_u\ (u\in V)\Big\}.max{cx:x∈S}=min{i∑​λi​bi​:λ≥0, i∑​λi​aiu​≥cu​ (u∈V)}.
  1. Lovász's first theorem (§3, p. 140). Every nonperfect GGG has A⊆VA\subseteq VA⊆V with α(GA) ω(GA)<∣A∣\alpha(G_A)\,\omega(G_A)<|A|α(GA​)ω(GA​)<∣A∣.
  2. Lovász's second theorem (§3, p. 140). Duplicating a vertex of a perfect graph gives a perfect graph.
  3. Condition (iii) (p. 141). GGG is perfect if and only if for every c∈ZVc\in\mathbb Z^Vc∈ZV
max⁡{cx:x∈S(G)}=min⁡{∑W∈C(G)λW:λW≥0, ∑W∋uλW≥cu (u∈V)}.\max\{cx:x\in S(G)\}=\min\Big\{\sum_{W\in C(G)}\lambda_W:\lambda_W\ge0,\ \sum_{W\ni u}\lambda_W\ge c_u\ (u\in V)\Big\}.max{cx:x∈S(G)}=min{W∈C(G)∑​λW​:λW​≥0, W∋u∑​λW​≥cu​ (u∈V)}.

Significance

The result. The nonnegativity and clique inequalities are valid for P(G)P(G)P(G) for every graph. Theorem 3.1 says they are complete exactly for perfect graphs, so on perfect graphs the maximum weight stable set problem is a linear program over an explicitly described polytope, and weighted min–max theorems (stable sets versus clique covers) follow from LP duality. Combined with the perfect graph theorem, it gives a polyhedral characterization of perfect graphs, and it is the model for later results that identify graph classes by the facets of their stable set polytopes (odd-cycle inequalities, ttt-perfection, Section 7 of the same paper).

Formalizing it. The result is classical and proved. No machine-checked version of it is known, and Mathlib has neither perfect graphs nor stable set polytopes. The mission produces a formal statement of the polyhedral characterization with the paper's own notion of perfection, a formal version of the convex-hull/LP min–max principle (Proposition 2.1), which is reusable for any 0–1 polytope, and formal statements of the two theorems of Lovász that the proof relies on.

Difficulty

Proposition 2.1 reduces Theorem 3.1 to the equivalence of perfection with a fractional min–max for all integer weights. The obvious approach to that equivalence fails in both directions. From perfection one only gets the min–max for zero–one weights and zero–one multipliers; general integer weights do not reduce to zero–one weights by linearity, because the minimum over clique covers is not additive in ccc. Conversely, a fractional clique cover of value α\alphaα does not directly produce an integral one. The paper crosses this gap with two theorems of Lovász: a numerical certificate of nonperfection, and the invariance of perfection under vertex duplication. Both are substantial graph-theoretic results in their own right, and neither follows from the definitions by routine manipulation.

Proposition 2.1 itself needs separation of a point from a polytope by an integral objective and LP strong duality with the nonnegativity rows handled separately.

Formalization scope

Vertices form a finite type V with decidable equality; a graph is a SimpleGraph V. S(G)S(G)S(G) is a set of functions V → ℝ, and P(G)P(G)P(G) is Mathlib's convexHull ℝ of it. C(G)C(G)C(G) is the finset of finsets that are maximal among cliques (Maximal), as on the page; with V=∅V=\emptysetV=∅ the only maximal clique is ∅\emptyset∅. "Defining linear system" is an equality of sets. Every "max = min" is written out in full: there is a value mmm that is the maximum over SSS (attained and an upper bound), some feasible multiplier vector attains mmm, and every feasible multiplier vector has objective at least mmm. Clique multipliers are functions Finset V → ℝ read only on C(G)C(G)C(G).

Explicit conventions and added hypotheses:

  • In Proposition 2.1 the index set JJJ is a finite type, coefficients are real, the nonnegativity rows are kept as a separate conjunct x≥0x\ge0x≥0, and SSS is assumed nonempty (the paper's max⁡\maxmax over SSS needs it).
  • α\alphaα and ω\omegaω are Mathlib's indepNum and cliqueNum (natural numbers) of G.induce A.
  • The duplicated graph lives on Option V, with none the new vertex.

Perfection is the paper's zero–one min–max, not "the clique system defines P(G)P(G)P(G)" (which would make the goal a tautology) and not Berge's χ(GA)=ω(GA)\chi(G_A)=\omega(G_A)χ(GA​)=ω(GA​) (a different definition, equivalent only through the perfect graph theorem). P(G)P(G)P(G) is the convex hull of S(G)S(G)S(G), never the solution set of an inequality system.

Needed infrastructure, all reusable: integral separation from a rational polytope and LP strong duality in the form max⁡{cx:Ax≤b,x≥0}=min⁡{λb:λA≥c,λ≥0}\max\{cx:Ax\le b,x\ge0\}=\min\{\lambda b:\lambda A\ge c,\lambda\ge0\}max{cx:Ax≤b,x≥0}=min{λb:λA≥c,λ≥0}; basic facts about stable sets and maximal cliques of induced subgraphs and of duplicated graphs; invariance of IsPerfect under graph isomorphism and under taking induced subgraphs. Proofs of the Lovász milestones, which have independent value for a Mathlib theory of perfect graphs, are welcome.

Selected references

  • V. Chvátal, On certain polytopes associated with graphs, J. Combin. Theory Ser. B 18 (1975) 138–154. https://doi.org/10.1016/0095-8956(75)90041-6
  • L. Lovász, Normal hypergraphs and the perfect graph conjecture, Discrete Math. 2 (1972) 253–267. https://doi.org/10.1016/0012-365X(72)90006-4
  • L. Lovász, A characterization of perfect graphs, J. Combin. Theory Ser. B 13 (1972) 95–98. https://doi.org/10.1016/0095-8956(72)90045-7
  • D. R. Fulkerson, Anti-blocking polyhedra, J. Combin. Theory Ser. B 12 (1972) 50–71. https://doi.org/10.1016/0095-8956(72)90032-9
  • M. Grötschel, L. Lovász, A. Schrijver, Geometric Algorithms and Combinatorial Optimization, Springer, 1988. https://doi.org/10.1007/978-3-642-97881-4
8 thms2 active usersReviewed
🏆Completed
Mathematical Physics·Captain: ShapeZero

The role postulates force exactly seven pointsTextbook

Motivation

The Shape Zero model (Shape Zero LLC, unpublished) reaches the Fano plane — the seven-point, seven-line configuration behind the seven imaginary units of the octonions — by a combinatorial route (C1 Formal Proofs, §3). Points are arranged in Steiner triple systems, and each point of each line is given one of three roles. The model's claim is that the role postulates alone force the number of points to be exactly seven. This mission proves that claim.

What this mission does NOT prove.

  • Not that the system is the Fano plane. It proves the point count is 777. That every Steiner triple system on 777 points is the Fano plane up to relabelling is a classical result, but it is not formalized here.
  • Not the modelling premise. Why lines have three points, and why there are three roles, is an input of the model, not derived here.
  • Not the later steps from the Fano plane to the octonions, and from there to the node size n=3n = 3n=3 and u(3)\mathfrak{u}(3)u(3).

Two corrections to C1 §3 built into this mission.

  • At least one point is required. The empty system — no points, no lines — satisfies every condition vacuously and has 000 points, so C1 Theorem 3.6 as stated is false for it. The goal carries the hypothesis 0<n0 < n0<n. This is proved (in Lean, locally): the empty system is a Steiner triple system with a role colouring, so the statement without 0<n0 < n0<n is false.
  • C1 Theorem 3.3(a) is not used. Its condition "any two lines meet" does not force 777 points: it also holds for a single triple (333 points) and a single point, where there is no pair of lines to fail. This mission uses the role postulates instead.

Setting

Fix a natural number nnn and take the points {0,…,n−1}\{0, \dots, n-1\}{0,…,n−1}. A Steiner triple system is a family of subsets, called lines, such that

  1. every line has exactly 333 points, and
  2. every pair of distinct points lies on exactly one line.

A role colouring assigns to each point xxx and line ℓ\ellℓ a role ρ(x,ℓ)∈{0,1,2}\rho(x, \ell) \in \{0, 1, 2\}ρ(x,ℓ)∈{0,1,2} such that

  1. the three points of a line get three different roles;
  2. completeness: every point takes every role at least once, on some line through it;
  3. minimality: every point takes every role at most once — two different lines through xxx give xxx different roles.

In Lean these are RolesForceSeven.STS n and RolesForceSeven.RoleColouring S role, with points Fin n and lines Finset (Fin n).

Formalization targets

Goal: the role postulates force exactly seven points

n≥1, a Steiner triple system on n points with a role colouring  ⟹  n=7.n \ge 1,\ \text{a Steiner triple system on } n \text{ points with a role colouring} \;\Longrightarrow\; n = 7 .n≥1, a Steiner triple system on n points with a role colouring⟹n=7.

This is RolesForceSeven.roles_force_seven. It asserts the point count only.

Milestones

  1. M1 (replication count). In any Steiner triple system, every point lies on exactly rrr lines with 2r+1=n2r + 1 = n2r+1=n.
  2. M2 (three lines per point). Under a role colouring, every point lies on exactly 333 lines.

Corollaries

  • A — the Fano plane has a role colouring. The Fano plane on {0,…,6}\{0, \dots, 6\}{0,…,6}, with lines {i,i+1,i+3}\{i, i+1, i+3\}{i,i+1,i+3} modulo 777, admits a role colouring. Without this the goal could be vacuously true.
  • B — AG(2, 3) is excluded (C1 Corollary 3.7): no Steiner triple system on 999 points has a role colouring.

Significance

The result itself. It turns the model's role postulates into a precise count: whatever the Steiner triple system, if it carries a role colouring and has a point, it has exactly seven points. Corollary A shows the conditions are satisfiable, and Corollary B rules out the next Steiner triple system, AG(2, 3), explicitly.

Formalizing it. The C1 statement omits the non-emptiness hypothesis and is false for the empty system; the formal statement makes the hypothesis explicit and shows it is needed. The numerical check (Fano plane: 484848 role colourings; AG(2, 3): 000; single triple: 000) is replaced by a proof for every nnn.

Numerical cross-check

systempointslines through each pointrole colourings
Fano plane7348
AG(2, 3)940
single triple310
empty system0—vacuous (all conditions hold)

Difficulty

Moderate. The central step is the replication count: the lines through a point xxx must be shown to cover every other point exactly once, two at a time, which is a double-counting argument over the pairs through xxx. The role postulates then fix the replication number at three. The Fano plane corollary is a finite check.

Formalization scope

  • Points are Fin n; lines are finite sets of points, with no ambient geometry assumed.
  • The role function is total, Fin n → Finset (Fin n) → Fin 3; only its values on pairs x∈ℓx \in \ellx∈ℓ with ℓ\ellℓ a line matter.
  • Completeness and minimality are both hypotheses; the goal uses both.
  • 0<n0 < n0<n is necessary: without it the empty system is a counterexample (proved).
  • The conclusion is the number 777, not an isomorphism with the Fano plane.

Selected references

  • Wikipedia, Steiner system (Steiner triple systems, replication number). https://en.wikipedia.org/wiki/Steiner_system
  • Wikipedia, Fano plane. https://en.wikipedia.org/wiki/Fano_plane
  • Wikipedia, Octonion (Fano plane mnemonic for the multiplication of imaginary units). https://en.wikipedia.org/wiki/Octonion
7 thms2 active usersReviewed
🏆Completed
Machine LearningProbability·Captain: naimengye

Understanding Machine Learning XXIV: Compression BoundsTextbook

Motivation

The book has characterized learnability through uniform convergence and through stability; Chapter 30 of Shalev-Shwartz and Ben-David, Understanding Machine Learning: From Theory to Algorithms (doi:10.1017/CBO9781107298019), gives a third sufficient condition, compression: if a learning algorithm's output can be reconstructed from a small subsequence of kkk training examples, then the error on the remaining examples estimates the true error, and the algorithm generalizes with a bound of order klog⁡(m/δ)/mk\log(m/\delta)/mklog(m/δ)/m (Theorem 30.2, Littlestone and Warmuth). The bound is a union bound over the mkm^kmk possible index sequences of a held-out estimate that follows from Bernstein's inequality (Lemma 30.1), and in the consistent case it gives LD≤8klog⁡(m/δ)/mL_D \le 8k\log(m/\delta)/mLD​≤8klog(m/δ)/m (Corollary 30.3). Classes admitting such compression schemes include axis-aligned rectangles (k=2dk = 2dk=2d), homogeneous halfspaces (k=dk = dk=d, through the minimal-norm point of the convex hull and Carathéodory's theorem), separating polynomials by reduction, and any margin-separable data (k≤1/γ2k \le 1/\gamma^2k≤1/γ2, through the Perceptron). Whether every class of finite VC dimension has a compression scheme of size O(d)O(d)O(d) is Warmuth's problem, open when the book was written.

Setting

A sample S=(z1,…,zm)S = (z_1, \dots, z_m)S=(z1​,…,zm​) is drawn i.i.d. from DDD; a selection rule picks (i1,…,ik)∈[m]k(i_1, \dots, i_k) \in [m]^k(i1​,…,ik​)∈[m]k (repetitions allowed), a reconstruction map B:Zk→HB : Z^k \to HB:Zk→H produces A(S)=B(zi1,…,zik)A(S) = B(z_{i_1}, \dots, z_{i_k})A(S)=B(zi1​​,…,zik​​), and VVV is the set of positions not selected, with LVL_VLV​ the average loss over them. The loss takes values in [0,1][0,1][0,1]. A class HHH has a compression scheme of size kkk (Definition 30.4) if for every m≥1m \ge 1m≥1 there are such AAA and BBB with B(SA(S))B(S_{A(S)})B(SA(S)​) correct on every sample labeled by a member of HHH; the unrealizable version (Definition 30.5) asks B(SA(S))B(S_{A(S)})B(SA(S)​) to be an empirical risk minimizer on every sample.

Formalization targets

Goal: Theorem 30.2

For a [0,1][0,1][0,1]-valued loss, k≥1k \ge 1k≥1, m≥2km \ge 2km≥2k, any reconstruction map BBB and any selection rule, with probability at least 1−δ1 - \delta1−δ over S∼DmS \sim D^mS∼Dm,

LD(A(S))≤LV(A(S))+LV(A(S)) 4klog⁡(m/δ)m+8klog⁡(m/δ)m.L_D(A(S)) \le L_V(A(S)) + \sqrt{L_V(A(S))\,\frac{4k\log(m/\delta)}{m}} + \frac{8k\log(m/\delta)}{m}.LD​(A(S))≤LV​(A(S))+LV​(A(S))m4klog(m/δ)​​+m8klog(m/δ)​.

Milestones

Lemma 30.1 (the held-out Bernstein bound); Corollary 30.3 (the consistent case); Lemma 30.6 (realizable schemes give unrealizable schemes); the compression scheme of size 2d2d2d for axis-aligned rectangles (§30.2.1); the separation property of the minimal-norm point of the convex hull (§30.2.2). Further items: the size-ddd scheme for homogeneous halfspaces and the margin scheme of §30.2.4.

Significance

Compression bounds are the cleanest generalization argument in the book: no complexity measure of the class enters, only the number of examples needed to encode the output, and the resulting bound is data-dependent through LVL_VLV​. They explain why support vector machines and the Perceptron generalize in terms of the number of support vectors or updates, and they underlie the sample-compression view of learning that connects to Chapters 9 and 15. Lemma 30.6 shows that compression is robust to label noise in the binary case. The halfspace scheme is a small piece of convex geometry of independent interest, and Warmuth's question about VC classes, settled in the affirmative for finite size by Moran and Yehudayoff after the book appeared, remains open in the form O(d)O(d)O(d).

Difficulty

Lemma 30.1 is Bernstein's inequality for the nnn held-out losses with variance at most LD(hT)L_D(h_T)LD​(hT​), followed by solving the resulting quadratic in LD\sqrt{L_D}LD​​ to move the risk from the right-hand side to LVL_VLV​; the constant 444 comes out of that step (the exact value is about 3.193.193.19). Theorem 30.2 is a union bound over the mkm^kmk index sequences with δ′=mkδ\delta' = m^k\deltaδ′=mkδ, using ∣V∣≥m−k≥m/2|V| \ge m - k \ge m/2∣V∣≥m−k≥m/2 and log⁡(mk/δ′)≤klog⁡(m/δ′)\log(m^k/\delta') \le k\log(m/\delta')log(mk/δ′)≤klog(m/δ′), which needs k≥1k \ge 1k≥1; formally the event for the learner is contained in the union of the events of Lemma 30.1 for each fixed index sequence, so no measurability of the selection rule is needed. Corollary 30.3 is immediate. Lemma 30.6 applies the realizable scheme to the subsample on which an ERM hypothesis is correct. The rectangle scheme is bookkeeping about extremal coordinates. The halfspace scheme needs three facts: the minimal-norm point of the hull separates (a one-line perturbation argument), it lies on a face and hence is a convex combination of ddd sample points (Carathéodory's theorem, in Mathlib, applied to a face), and it is the minimal-norm point of the hull of those ddd points (uniqueness of the projection onto a convex set); the existence of the minimizer uses compactness of the hull. The margin scheme is the Perceptron convergence theorem of Mission VI applied to the batch algorithm, whose output is the sum of the updated examples.

Formalization scope

Samples are Fin m-indexed under Mission I's iidLaw, and probability statements bound the outer measure of the failure event. Lemma 30.1 splits a sample of size k+nk + nk+n into its first kkk and last nnn entries; Theorem 30.2 takes an arbitrary selection rule sel:Zm→[m]k\mathrm{sel} : Z^m \to [m]^ksel:Zm→[m]k and reconstruction map BBB, the held-out set being the positions not in the range of the selection, and requires k≥1k \ge 1k≥1 and m≥1m \ge 1m≥1 in addition to the book's m≥2km \ge 2km≥2k: for k=0k = 0k=0 the bound reads LD≤LVL_D \le L_VLD​≤LV​, which fails, and the book's derivation uses k≥1k \ge 1k≥1 in log⁡(mk/δ′)≤klog⁡(m/δ′)\log(m^k/\delta') \le k\log(m/\delta')log(mk/δ′)≤klog(m/δ′). Compression schemes are defined for every m≥1m \ge 1m≥1, since for m=0m = 0m=0 there is no index to select, with indices allowed to repeat as in [m]k[m]^k[m]k and with BBB's outputs in HHH; the unrealizable version uses Mission XXIII's multiclass 0–1 loss. The halfspace results are stated for strictly separable ±1\pm1±1-labeled samples, yi⟨w⋆,xi⟩>0y_i\langle w^\star, x_i\rangle > 0yi​⟨w⋆,xi​⟩>0, the book's "w.l.o.g. all labels positive" normalization; this avoids the boundary negatives that a realizable sample may contain under the sign⁡(0)\operatorname{sign}(0)sign(0) convention of Mission VI, and it is the setting in which the minimal-norm argument works. The scheme is stated as the existence of ddd indices whose signed examples have a minimal-norm hull point separating the whole sample, which is the content of AAA and BBB without fixing how ties among faces are broken. Rectangles are closed boxes. The margin scheme uses Theorem 9.1's normalization.

Not stated: §30.2.3 (polynomials, a reduction), the bibliographic remarks, and the intermediate Carathéodory step as a separate item.

Selected references

  • S. Shalev-Shwartz, S. Ben-David, Understanding Machine Learning: From Theory to Algorithms, Cambridge University Press, 2014, Chapter 30. doi:10.1017/CBO9781107298019
  • N. Littlestone, M. K. Warmuth, Relating data compression and learnability, technical report, University of California, Santa Cruz, 1986.
  • S. Floyd, M. K. Warmuth, Sample compression, learnability, and the Vapnik-Chervonenkis dimension, Machine Learning 21, 1995. doi:10.1007/BF00993593
  • S. Ben-David, A. Litman, Combinatorial variability of Vapnik-Chervonenkis classes with applications to sample compression schemes, Discrete Applied Mathematics 86, 1998. doi:10.1016/S0166-218X(98)00000-6
  • S. Moran, A. Yehudayoff, Sample compression schemes for VC classes, Journal of the ACM 63(3), 2016. doi:10.1145/2890490
10 thms2 active usersReviewed
🏆Completed
Machine LearningProbability·Captain: naimengye

Understanding Machine Learning XXIII: Multiclass LearnabilityTextbook

Motivation

Chapter 17 introduced multiclass prediction; Chapter 29 of Shalev-Shwartz and Ben-David, Understanding Machine Learning: From Theory to Algorithms (doi:10.1017/CBO9781107298019), asks the two questions the fundamental theorem answered for binary classes: which classes of multiclass predictors are PAC learnable with respect to the 0–1 loss, and with what sample complexity. Natarajan's dimension generalizes the VC dimension by shattering with two disagreeing label functions, and the multiclass fundamental theorem (Theorem 29.3) bounds the uniform-convergence, agnostic and realizable sample complexities in terms of it, up to logarithmic factors in the number of labels kkk; the only new ingredient in its proof is Natarajan's lemma, the multiclass substitute for Sauer's lemma. The chapter then computes or bounds the Natarajan dimension of the classes that matter, One-versus-All and general reductions to binary classifiers, and linear multiclass predictors (Theorem 29.7). Its last section is a warning: unlike the binary case, not all ERMs are equal, and with infinitely many labels a class can be learnable by one ERM and not by another, so learnability and uniform convergence come apart (Claim 29.9).

Setting

HHH is a class of functions from XXX to a finite label set YYY with ∣Y∣=k|Y| = k∣Y∣=k. C⊆XC \subseteq XC⊆X is shattered by HHH if there are f0,f1:C→Yf_0, f_1 : C \to Yf0​,f1​:C→Y with f0(x)≠f1(x)f_0(x) \ne f_1(x)f0​(x)=f1​(x) everywhere on CCC such that every B⊆CB \subseteq CB⊆C is realized by some h∈Hh \in Hh∈H agreeing with f0f_0f0​ on BBB and with f1f_1f1​ on C∖BC \setminus BC∖B (Definition 29.1); Ndim⁡(H)\operatorname{Ndim}(H)Ndim(H) is the largest size of a shattered set (Definition 29.2). One-versus-All builds T(hˉ)(x)=argmax⁡ihi(x)T(\bar h)(x) = \operatorname{argmax}_i h_i(x)T(hˉ)(x)=argmaxi​hi​(x) from kkk binary classifiers, the smaller label on ties; a general reduction applies a rule r:{0,1}l→[k]r : \{0,1\}^l \to [k]r:{0,1}l→[k] to lll binary classifiers; the linear class HΨH_\PsiHΨ​ predicts argmax⁡i⟨w,Ψ(x,i)⟩\operatorname{argmax}_i\langle w, \Psi(x, i)\rangleargmaxi​⟨w,Ψ(x,i)⟩ for a class-sensitive feature map Ψ:X×[k]→Rd\Psi : X \times [k] \to \mathbb{R}^dΨ:X×[k]→Rd (29.1). The class of §29.4 has labels Pf(X)∪{∗}P_f(X) \cup \{\ast\}Pf​(X)∪{∗}, the finite and cofinite subsets of XXX plus a special label, and hypotheses hA(x)=Ah_A(x) = AhA​(x)=A if x∈Ax \in Ax∈A and ∗\ast∗ otherwise; AgoodA_{good}Agood​ returns h∅h_\emptyseth∅​ on an all-∗\ast∗ sample and AbadA_{bad}Abad​ returns h{x1,…,xm}ch_{\{x_1, \dots, x_m\}^c}h{x1​,…,xm​}c​.

Formalization targets

Goal: Theorem 29.3

There are absolute constants C1,C2>0C_1, C_2 > 0C1​,C2​>0 such that every class H⊆YXH \subseteq Y^XH⊆YX with Ndim⁡(H)=d\operatorname{Ndim}(H) = dNdim(H)=d satisfies

C1d+log⁡(1/δ)ϵ2≤mHUC(ϵ,δ), mH(ϵ,δ)≤C2dlog⁡k+log⁡(1/δ)ϵ2,C1d+log⁡(1/δ)ϵ≤mHreal(ϵ,δ)≤C2dlog⁡(kd/ϵ)+log⁡(1/δ)ϵ,C_1\frac{d + \log(1/\delta)}{\epsilon^2} \le m^{UC}_H(\epsilon,\delta),\ m_H(\epsilon,\delta) \le C_2\frac{d\log k + \log(1/\delta)}{\epsilon^2}, \qquad C_1\frac{d + \log(1/\delta)}{\epsilon} \le m^{\mathrm{real}}_H(\epsilon,\delta) \le C_2\frac{d\log(kd/\epsilon) + \log(1/\delta)}{\epsilon},C1​ϵ2d+log(1/δ)​≤mHUC​(ϵ,δ), mH​(ϵ,δ)≤C2​ϵ2dlogk+log(1/δ)​,C1​ϵd+log(1/δ)​≤mHreal​(ϵ,δ)≤C2​ϵdlog(kd/ϵ)+log(1/δ)​,

the upper bounds by every ERM learner and the lower bounds for small ϵ,δ\epsilon, \deltaϵ,δ and d≥2d \ge 2d≥2, in the format of Mission IV's Theorem 6.8.

Milestones

Lemma 29.4 (Natarajan: ∣H∣≤∣X∣Ndim⁡(H)k2Ndim⁡(H)|H| \le |X|^{\operatorname{Ndim}(H)}k^{2\operatorname{Ndim}(H)}∣H∣≤∣X∣Ndim(H)k2Ndim(H)); Lemma 29.5 (the Natarajan dimension of One-versus-All is O(kdlog⁡(kd))O(kd\log(kd))O(kdlog(kd))); Theorem 29.7 (Ndim⁡(HΨ)≤d\operatorname{Ndim}(H_\Psi) \le dNdim(HΨ​)≤d); Claim 29.9(1) (AgoodA_{good}Agood​ needs 1ϵlog⁡1δ\frac1\epsilon\log\frac1\deltaϵ1​logδ1​ examples); Claim 29.9(2) (AbadA_{bad}Abad​ fails with constant probability on (∣X∣−1)/(6ϵ)(|X|-1)/(6\epsilon)(∣X∣−1)/(6ϵ) examples). Further items: the equality Ndim⁡=VCdim⁡\operatorname{Ndim} = \operatorname{VCdim}Ndim=VCdim for two classes, and Lemma 29.6 for general reductions.

Significance

Theorem 29.3 is the multiclass fundamental theorem of Natarajan (1989) and Ben-David, Cesa-Bianchi, Haussler and Long (1995): finite Natarajan dimension characterizes multiclass learnability, and the sample complexity is linear in it, with the dependence on kkk confined to logarithms. Natarajan's lemma is the combinatorial core, and the dimension bounds of §29.3 are what make the theorem usable: a One-versus-All scheme over a class of VC dimension ddd costs O~(kd)\tilde O(kd)O~(kd), and a linear multiclass predictor costs at most its number of parameters, so the multivector construction of Chapter 17 is learnable with O~(nk/ϵ2)\tilde O(nk/\epsilon^2)O~(nk/ϵ2) examples. Claim 29.9 is a genuine phenomenon of Daniely, Sabato, Ben-David and Shalev-Shwartz (2011): in multiclass classification the choice of ERM matters, and the equivalence "learnable iff uniform convergence" of the binary theory is false, which is why Conjecture 29.10 about good ERMs is open in the form the chapter states it.

Difficulty

The equality with the VC dimension for two labels is a direct comparison of the two shattering definitions. Natarajan's lemma is a Sauer-type induction on ∣X∣|X|∣X∣, in which a shattered set must be produced from two hypotheses that differ at a point; the exercise-level proof of the book becomes a careful double induction formally. Theorem 29.3's upper bounds follow the binary proof of Chapter 28 with Natarajan's lemma in place of Sauer's, hence Massart's lemma and Theorem 26.5 for the agnostic case and the double-sample argument for the realizable case; the lower bounds reduce to the binary ones by embedding a binary class into a multiclass one on a shattered set. These are long formal developments, and the theorem is stated with unspecified constants for that reason. Lemmas 29.5 and 29.6 are counting: a shattered CCC has 2∣C∣≤∣HC∣≤∣(Hbin)C∣k2^{|C|} \le |H_C| \le |(H_{bin})_C|^k2∣C∣≤∣HC​∣≤∣(Hbin​)C​∣k, Sauer's lemma bounds the right side by (∑i≤d(∣C∣i))k\big(\sum_{i \le d}\binom{|C|}{i}\big)^{k}(∑i≤d​(i∣C∣​))k, and the resulting inequality is solved. For Lemma 29.5's printed 3kdlog⁡(kd)3kd\log(kd)3kdlog(kd) this fails only at (k,d)=(2,1),(3,1)(k, d) = (2, 1), (3, 1)(k,d)=(2,1),(3,1). There a shattered set splits by the label pair {f0(x),f1(x)}\{f_0(x), f_1(x)\}{f0​(x),f1​(x)} into parts shattered by {B∖A:A,B∈Hbin}\{B \setminus A : A, B \in H_{bin}\}{B∖A:A,B∈Hbin​} (pairs {0,b}\{0, b\}{0,b}) or by HbinH_{bin}Hbin​ (other pairs). The first class has at most 313131 traces on 555 points. Theorem 29.7 maps a shattered set into Rd\mathbb{R}^dRd by ρ(x)=Ψ(x,f0(x))−Ψ(x,f1(x))\rho(x) = \Psi(x, f_0(x)) - \Psi(x, f_1(x))ρ(x)=Ψ(x,f0​(x))−Ψ(x,f1​(x)), up to sign, and shows the image is shattered by homogeneous halfspaces; the tie-breaking rule decides which sign and which halfspace convention to use. Claim 29.9(1) is the bound (1−ϵ)m≤δ(1-\epsilon)^m \le \delta(1−ϵ)m≤δ; Claim 29.9(2) needs only that at most (d−1)/2(d-1)/2(d−1)/2 of the d−1d-1d−1 light points appear in the sample, an event of probability at least 1/31/31/3 by Markov's inequality when m≤(d−1)/(6ϵ)m \le (d-1)/(6\epsilon)m≤(d−1)/(6ϵ), which exceeds the claimed e−1/6e^{-1}/6e−1/6.

Formalization scope

Labels are an arbitrary finite type, shattering and the Natarajan dimension are stated with witnesses f0,f1f_0, f_1f0​,f1​ defined on all of XXX, and the dimension is a supremum in N∪{∞}\mathbb{N} \cup \{\infty\}N∪{∞}. The multiclass 0–1 loss, the ERM property, agnostic PAC learnability and uniform convergence are Mission I's generic notions; the realizable multiclass PAC property is defined here in the shape of Definition 3.1 with D({h≠f})D(\{h \ne f\})D({h=f}) as the error, since Mission I's binary version is {0,1}\{0,1\}{0,1}-specific. Theorem 29.3 is stated exactly as Mission IV states Theorem 6.8, with existential constants, upper bounds for every ERM learner of a nonempty measurable class with the countable-approximation property (the measurability device of Remark 3.1), and lower bounds for ϵ<ϵ0\epsilon < \epsilon_0ϵ<ϵ0​, δ<δ0\delta < \delta_0δ<δ0​, d≥2d \ge 2d≥2. Argmax predictors, both One-versus-All and HΨH_\PsiHΨ​, break ties towards the smallest label; the book states this rule for One-versus-All, and some fixed rule is necessary for Theorem 29.7, since with arbitrary tie-breaking every function is an argmax predictor of the zero mapping. Lemmas 29.5 and 29.6 are stated per shattered set. Lemma 29.5 keeps the printed 3kdlog⁡(kd)3kd\log(kd)3kdlog(kd), which is true although the book's step ∣(Hbin)C∣≤∣C∣d|(H_{bin})_C| \le |C|^d∣(Hbin​)C​∣≤∣C∣d fails for small ∣C∣|C|∣C∣. Lemma 29.6 uses 2ldlog⁡2(2ld)2ld\log_2(2ld)2ldlog2​(2ld), which the counting supports, because the printed 3ldlog⁡(ld)3ld\log(ld)3ldlog(ld) is false at l=d=1l = d = 1l=d=1. Theorem 29.3's uniform-convergence upper bound is stated for d≥1d \ge 1d≥1. At d=0d = 0d=0 the confidence term log⁡(1/δ)\log(1/\delta)log(1/δ) vanishes as δ→1\delta \to 1δ→1, the bound reaches m=1m = 1m=1, and a single example is not representative. The class of §29.4 has labels Option of the subtype of finite-or-cofinite sets, with the discrete σ-algebra, and the two ERMs are predicates fixing the output on all-∗\ast∗ samples; Claim 29.9(1) is stated for countable XXX with measurable singletons and Claim 29.9(2) for finite XXX of size at least 222, with the proof's own distribution, h∅h_\emptyseth∅​ as target, and every ϵ∈(0,1/2)\epsilon \in (0, 1/2)ϵ∈(0,1/2) in place of the book's unspecified constant aaa.

Not stated: Corollary 29.8 (its lower bound (k−1)(n−1)(k-1)(n-1)(k−1)(n−1) is cited, not proved), Conjecture 29.10, the exercises.

Selected references

  • S. Shalev-Shwartz, S. Ben-David, Understanding Machine Learning: From Theory to Algorithms, Cambridge University Press, 2014, Chapter 29. doi:10.1017/CBO9781107298019
  • B. K. Natarajan, On learning sets and functions, Machine Learning 4, 1989. doi:10.1007/BF00114804
  • S. Ben-David, N. Cesa-Bianchi, D. Haussler, P. M. Long, Characterizations of learnability for classes of {0, …, n}-valued functions, Journal of Computer and System Sciences 50(1), 1995. doi:10.1006/jcss.1995.1008
  • D. Haussler, P. M. Long, A generalization of Sauer's lemma, Journal of Combinatorial Theory A 71(2), 1995. doi:10.1016/0097-3165(95)90001-2
  • A. Daniely, S. Sabato, S. Ben-David, S. Shalev-Shwartz, Multiclass learnability and the ERM principle, COLT 2011; Journal of Machine Learning Research 16, 2015.
  • A. Daniely, S. Sabato, S. Shalev-Shwartz, Multiclass learning approaches: a theoretical comparison with implications, NIPS 2012.
15 thms2 active usersReviewed
🏆Completed
Machine LearningProbability·Captain: naimengye

Understanding Machine Learning XXII: Proof of the Fundamental TheoremTextbook

Motivation

Chapter 6 stated the fundamental theorem of statistical learning: a binary class is learnable if and only if its VC dimension is finite, with sample complexity Θ((d+ln⁡(1/δ))/ϵ2)\Theta((d + \ln(1/\delta))/\epsilon^2)Θ((d+ln(1/δ))/ϵ2) in the agnostic case and Θ((dln⁡(1/ϵ)+ln⁡(1/δ))/ϵ)\Theta((d\ln(1/\epsilon) + \ln(1/\delta))/\epsilon)Θ((dln(1/ϵ)+ln(1/δ))/ϵ) in the realizable case. Chapter 28 of Shalev-Shwartz and Ben-David, Understanding Machine Learning: From Theory to Algorithms (doi:10.1017/CBO9781107298019), proves it. The agnostic upper bound is obtained from the Rademacher machinery of Chapter 26 with Sauer's lemma and Massart's lemma, up to a log⁡(d/ϵ)\log(d/\epsilon)log(d/ϵ) factor that only chaining removes (28.1). The agnostic lower bound comes in two parts: a two-point construction giving m≥0.5log⁡(1/(4δ))/ϵ2m \ge 0.5\log(1/(4\delta))/\epsilon^2m≥0.5log(1/(4δ))/ϵ2, and a ddd-point construction giving m≥d/(512ϵ2)m \ge d/(512\epsilon^2)m≥d/(512ϵ2) at confidence 1/81/81/8, whose heart is Lemma 28.1, the optimality of the Maximum-Likelihood rule against the family of noisy distributions DbD_bDb​. The realizable upper bound is proved through ϵ\epsilonϵ-nets: with m≥8ϵ(2dlog⁡(16e/ϵ)+log⁡(2/δ))m \ge \frac8\epsilon(2d\log(16e/\epsilon) + \log(2/\delta))m≥ϵ8​(2dlog(16e/ϵ)+log(2/δ)) examples a random sample hits every set of measure at least ϵ\epsilonϵ in the class (Theorem 28.3), so any hypothesis consistent with the sample has error below ϵ\epsilonϵ. Mission IV states these bounds with unnamed constants; this mission gives the chapter's explicit ones.

Setting

HHH is a class of functions X→{0,1}X \to \{0,1\}X→{0,1} with the 0–1 loss and VCdim⁡(H)=d\operatorname{VCdim}(H) = dVCdim(H)=d. For the upper bound, A={(1[h(xi)≠yi])i:h∈H}A = \{(\mathbb{1}[h(x_i) \ne y_i])_i : h \in H\}A={(1[h(xi​)=yi​])i​:h∈H} is the loss set of a sample and R(A)R(A)R(A) its Rademacher complexity. For the lower bounds, C={c1,…,cd}C = \{c_1, \dots, c_d\}C={c1​,…,cd​} is a set shattered by HHH and, for b∈{±1}db \in \{\pm1\}^db∈{±1}d and ρ∈(0,1)\rho \in (0,1)ρ∈(0,1), DbD_bDb​ draws cic_ici​ uniformly and labels it bib_ibi​ with probability (1+ρ)/2(1+\rho)/2(1+ρ)/2; for d=1d = 1d=1 these are the distributions D±D_\pmD±​ of §28.2.1. The Maximum-Likelihood rule AMLA_{ML}AML​ predicts at each cic_ici​ the majority of the labels seen at cic_ici​. An ϵ\epsilonϵ-net for HHH with respect to DDD is a sample meeting every h∈Hh \in Hh∈H with D(h)≥ϵD(h) \ge \epsilonD(h)≥ϵ (Definition 28.2).

Formalization targets

Goal: Theorem 28.3

Let VCdim⁡(H)=d\operatorname{VCdim}(H) = dVCdim(H)=d, ϵ∈(0,1)\epsilon \in (0,1)ϵ∈(0,1), δ∈(0,1/4)\delta \in (0, 1/4)δ∈(0,1/4) and m≥8ϵ(2dlog⁡16eϵ+log⁡2δ)m \ge \frac8\epsilon\big(2d\log\frac{16e}{\epsilon} + \log\frac2\delta\big)m≥ϵ8​(2dlogϵ16e​+logδ2​). Then with probability at least 1−δ1 - \delta1−δ over S∼DmS \sim D^mS∼Dm, SSS is an ϵ\epsilonϵ-net for HHH.

Milestones

The two-sided deviation bound of §28.1 (∣LD(h)−LS(h)∣≤2(8dlog⁡(em/d)+2log⁡(4/δ))/m|L_D(h) - L_S(h)| \le 2\sqrt{(8d\log(em/d) + 2\log(4/\delta))/m}∣LD​(h)−LS​(h)∣≤2(8dlog(em/d)+2log(4/δ))/m​ uniformly over HHH); the lower bound m(ϵ,δ)≥0.5log⁡(1/(4δ))/ϵ2m(\epsilon,\delta) \ge 0.5\log(1/(4\delta))/\epsilon^2m(ϵ,δ)≥0.5log(1/(4δ))/ϵ2 of §28.2.1; Lemma 28.1; the lower bound m(ϵ,1/8)≥d/(512ϵ2)m(\epsilon, 1/8) \ge d/(512\epsilon^2)m(ϵ,1/8)≥d/(512ϵ2) of §28.2.2; the realizable upper bound of §28.3 (ERM has error at most ϵ\epsilonϵ with probability 1−δ1 - \delta1−δ for the sample size of Theorem 28.3). Further items: the Rademacher bound R(A)≤2dlog⁡(em/d)/mR(A) \le \sqrt{2d\log(em/d)/m}R(A)≤2dlog(em/d)/m​, the explicit uniform-convergence sample complexity of §28.1, and the expectation lower bound ρ/4\rho/4ρ/4 of §28.2.2.

Significance

These are the theorems that make the VC dimension the right measure of learnability, with constants. The upper bounds show what the abstract machinery of Missions II, IV, XX buys when instantiated: Sauer plus Massart plus Theorem 26.5 gives the agnostic rate, and the double-sample symmetrization plus Sauer gives the realizable rate, sharper by a factor 1/ϵ1/\epsilon1/ϵ because ϵ\epsilonϵ-nets need only one-sided control. The lower bounds are the No-Free-Lunch argument refined to quantify ϵ\epsilonϵ and δ\deltaδ: the two-point distribution shows that confidence costs log⁡(1/δ)/ϵ2\log(1/\delta)/\epsilon^2log(1/δ)/ϵ2, and the ddd-point family with Lemma 28.1 shows that the dimension costs d/ϵ2d/\epsilon^2d/ϵ2, through the exact optimality of majority voting and a binomial anti-concentration bound. Theorem 28.3 is also the basic ϵ\epsilonϵ-net theorem of Haussler and Welzl, a result of independent importance in computational geometry.

Difficulty

The Rademacher bound is Sauer's lemma (Mission IV) plus Massart's lemma (Mission XX) with ∥a−aˉ∥≤m\|a - \bar a\| \le \sqrt m∥a−aˉ∥≤m​; the deviation bound is Theorem 26.5 applied to ℓ\ellℓ and −ℓ-\ell−ℓ with a union bound; the explicit sample complexity is Lemma A.2, x≥4alog⁡(2a)+2b⇒x≥alog⁡x+bx \ge 4a\log(2a) + 2b \Rightarrow x \ge a\log x + bx≥4alog(2a)+2b⇒x≥alogx+b, which a formal proof must establish (the tangent inequality for log⁡\loglog at 2a2a2a). The two-point lower bound requires the binomial lower-tail estimate of Lemma B.11 and the algebra 12(1−1−4δ)≥δ\frac12(1 - \sqrt{1 - \sqrt{4\delta}}) \ge \delta21​(1−1−4δ​​)≥δ, valid for δ<1/4\delta < 1/4δ<1/4, the only nonvacuous range. Lemma 28.1 is a conditioning argument: fixing the instance indices and the labels off cic_ici​, the contribution of cic_ici​ is minimized by predicting the more likely bib_ibi​ given the labels at cic_ici​, which is the majority; the formal proof must decompose the product measure DbmD_b^mDbm​ over the positions rrr with xr=cix_r = c_ixr​=ci​. The expectation bound ρ/4\rho/4ρ/4 then needs Lemma B.11 again, 1−e−a≤a1 - e^{-a} \le a1−e−a≤a, Jensen for ⋅\sqrt{\cdot}⋅​ and E[ni]=m/d\mathbb{E}[n_i] = m/dE[ni​]=m/d, and the probability bound 1/81/81/8 follows by Mission III's reverse Markov inequality with ρ=8ϵ\rho = 8\epsilonρ=8ϵ. Theorem 28.3 is the double-sample argument: Claim 1 (P[S∈B]≤2P[(S,T)∈B′]P[S \in B] \le 2P[(S,T) \in B']P[S∈B]≤2P[(S,T)∈B′], via a Chernoff bound that only needs mϵ≥2log⁡2m\epsilon \ge 2\log 2mϵ≥2log2), Claim 2 (symmetrization by a random half, P[(S,T)∈B′]≤e−ϵm/4τH(2m)P[(S,T) \in B'] \le e^{-\epsilon m/4}\tau_H(2m)P[(S,T)∈B′]≤e−ϵm/4τH​(2m)), Sauer's lemma, and Lemma A.2 once more. The realizable upper bound applies Theorem 28.3 to the error sets {x:h(x)≠f(x)}\{x : h(x) \ne f(x)\}{x:h(x)=f(x)}, a class of the same VC dimension.

Formalization scope

All objects are those of the earlier missions: risks, samples and learners from Mission I, vcDim and the countable-approximation property PointwiseSeparable from Mission IV (the measurability device for suprema over HHH, used wherever a symmetrization or Rademacher argument is invoked), condLaw from Mission XIV for the distributions DbD_bDb​, and rademacher, evalSet, lossClass from Mission XX. Probability statements bound the outer measure of the failure event under iidLaw. The Rademacher and deviation items require m>d+1m > d + 1m>d+1, the range in which Mission IV states Sauer's lemma in the form (em/d)d(em/d)^d(em/d)d; the explicit sample complexity of §28.1 implies this range, since its first term 432dlog⁡(64d/ϵ2)/ϵ2432d\log(64d/\epsilon^2)/\epsilon^2432dlog(64d/ϵ2)/ϵ2 dominates the possibly negative 8dlog⁡(e/d)8d\log(e/d)8dlog(e/d), and Lemma A.2 holds for any real bbb, so the book's constants are used verbatim. The lower bounds take a shattered set as an injective c:Fin d→Xc : \mathrm{Fin}\ d \to Xc:Fin d→X with the shattering property written out, use DbD_bDb​ as condLaw of the uniform law on CCC, and state the excess risk against min⁡h∈HLDb(h)\min_{h \in H}L_{D_b}(h)minh∈H​LDb​​(h) as ∃h∈H\exists h \in H∃h∈H with L(h)+ϵ≤L(A(S))L(h) + \epsilon \le L(A(S))L(h)+ϵ≤L(A(S)), or as a real infimum over HHH in the expectation item; no measurability of the learner is needed because DbmD_b^mDbm​ is atomic. Lemma 28.1 compares the sums over bbb of the expected risks, the common term min⁡hLDb\min_h L_{D_b}minh​LDb​​ cancelling, for every majority rule with arbitrary tie-breaking. Theorem 28.3 and the realizable bound are stated for δ∈(0,1/4)\delta \in (0, 1/4)δ∈(0,1/4), the theorem's own range; the realizable bound is the inner clause of Mission I's IsPACWith on that range rather than a sample-complexity function, since the theorem does not cover δ≥1/4\delta \ge 1/4δ≥1/4 with its formula.

Not stated: the realizable lower bound (an exercise), the remark that chaining removes the logarithm in (28.1), and the intermediate claims of the proof of Theorem 28.3 as separate items.

Selected references

  • S. Shalev-Shwartz, S. Ben-David, Understanding Machine Learning: From Theory to Algorithms, Cambridge University Press, 2014, Chapter 28. doi:10.1017/CBO9781107298019
  • V. N. Vapnik, A. Ya. Chervonenkis, On the uniform convergence of relative frequencies of events to their probabilities, Theory of Probability and its Applications 16(2), 1971. doi:10.1137/1116025
  • A. Blumer, A. Ehrenfeucht, D. Haussler, M. K. Warmuth, Learnability and the Vapnik-Chervonenkis dimension, Journal of the ACM 36(4), 1989. doi:10.1145/76359.76371
  • D. Haussler, E. Welzl, ε-nets and simplex range queries, Discrete and Computational Geometry 2, 1987. doi:10.1007/BF02187876
  • M. Anthony, P. L. Bartlett, Neural Network Learning: Theoretical Foundations, Cambridge University Press, 1999. doi:10.1017/CBO9780511624216
11 thms2 active usersReviewed
🏆Completed
Machine LearningOptimization·Captain: naimengye

Understanding Machine Learning XVI: Online LearningTextbook

Motivation

In PAC learning the learner receives a batch of examples, learns, and only then predicts. Online learning has no such separation: on each round the learner receives an instance, predicts its label, and then sees the true label, and the goal is to make few mistakes over the whole sequence, with no statistical assumption whatsoever on how the sequence is generated, adversarially if need be. Chapter 21 of Shalev-Shwartz and Ben-David, Understanding Machine Learning: From Theory to Algorithms (doi:10.1017/CBO9781107298019), develops this model along the same lines as the PAC theory. In the realizable case, mistake bounds replace sample complexity, and a combinatorial dimension due to Littlestone, Ldim⁡(H)\operatorname{Ldim}(H)Ldim(H), characterizes the best achievable bound exactly (Lemmas 21.6 and 21.7), playing the role the VC dimension plays for PAC learning, with VCdim⁡(H)≤Ldim⁡(H)\operatorname{VCdim}(H) \le \operatorname{Ldim}(H)VCdim(H)≤Ldim(H) and an arbitrarily large gap (Theorem 21.9). In the unrealizable case regret replaces excess risk; deterministic learners can be forced to regret T/2T/2T/2 (Cover), but randomized predictions restore sublinear regret through the Weighted-Majority algorithm of Littlestone, Warmuth and Vovk (Theorem 21.11). The chapter closes with online convex optimization, where Online Gradient Descent (Zinkevich) attains regret O(T)O(\sqrt T)O(T​) (Theorem 21.15), and with the online Perceptron, whose mistake bound follows from a round-specific surrogate loss (Theorem 21.16).

Setting

An online algorithm is a deterministic map from the history of past examples and the current instance to a prediction. For a sequence SSS labeled by some h⋆∈Hh^\star \in Hh⋆∈H, MA(S)M_A(S)MA​(S) is the number of mistakes and MA(H)M_A(H)MA​(H) the supremum over all such sequences (Definition 21.1). The Consistent algorithm predicts with any hypothesis of the version space VtV_tVt​ (the hypotheses consistent with the past), Halving with its majority label, and SOA with the label rrr for which {h∈Vt:h(xt)=r}\{h \in V_t : h(x_t) = r\}{h∈Vt​:h(xt​)=r} has the larger Littlestone dimension, ties to 111. An HHH-shattered tree of depth ddd assigns an instance to every node of a complete binary tree so that every labeling (y1,…,yd)(y_1, \dots, y_d)(y1​,…,yd​) is realized by some h∈Hh \in Hh∈H along the path it determines; Ldim⁡(H)\operatorname{Ldim}(H)Ldim(H) is the maximal such depth (Definitions 21.4–21.5). In the unrealizable case predictions are pt∈[0,1]p_t \in [0,1]pt​∈[0,1], the loss is ∣pt−yt∣|p_t - y_t|∣pt​−yt​∣, and the regret against hhh is ∑t∣pt−yt∣−∑t∣h(xt)−yt∣\sum_t |p_t - y_t| - \sum_t |h(x_t) - y_t|∑t​∣pt​−yt​∣−∑t​∣h(xt​)−yt​∣ (21.1). Weighted-Majority maintains wi(t)∝exp⁡(−η∑s<tvs,i)w^{(t)}_i \propto \exp(-\eta\sum_{s<t} v_{s,i})wi(t)​∝exp(−η∑s<t​vs,i​) over ddd experts with costs vt∈[0,1]dv_t \in [0,1]^dvt​∈[0,1]d and pays ⟨w(t),vt⟩\langle w^{(t)}, v_t\rangle⟨w(t),vt​⟩. Online Gradient Descent on a closed convex HHH predicts w(t)w^{(t)}w(t), receives a convex ftf_tft​, takes a subgradient vtv_tvt​ at w(t)w^{(t)}w(t) and projects w(t)−ηvtw^{(t)} - \eta v_tw(t)−ηvt​ back onto HHH; the online Perceptron is the special case w(t+1)=w(t)+ytxtw^{(t+1)} = w^{(t)} + y_t x_tw(t+1)=w(t)+yt​xt​ on rounds with yt⟨w(t),xt⟩≤0y_t\langle w^{(t)}, x_t\rangle \le 0yt​⟨w(t),xt​⟩≤0.

Formalization targets

Goal: Theorem 21.11

For d≥1d \ge 1d≥1 experts, cost vectors vt∈[0,1]dv_t \in [0,1]^dvt​∈[0,1]d, T>2log⁡dT > 2\log dT>2logd and η=2log⁡(d)/T\eta = \sqrt{2\log(d)/T}η=2log(d)/T​,

∑t=1T⟨w(t),vt⟩−min⁡i∈[d]∑t=1Tvt,i≤2log⁡(d) T.\sum_{t=1}^T \langle w^{(t)}, v_t\rangle - \min_{i \in [d]}\sum_{t=1}^T v_{t,i} \le \sqrt{2\log(d)\,T}.t=1∑T​⟨w(t),vt​⟩−i∈[d]min​t=1∑T​vt,i​≤2log(d)T​.

Milestones

Theorem 21.3 (Halving makes at most log⁡2∣H∣\log_2|H|log2​∣H∣ mistakes); Lemma 21.6 (MA(H)≥Ldim⁡(H)M_A(H) \ge \operatorname{Ldim}(H)MA​(H)≥Ldim(H) for every AAA); Lemma 21.7 (MSOA(H)≤Ldim⁡(H)M_{\mathrm{SOA}}(H) \le \operatorname{Ldim}(H)MSOA​(H)≤Ldim(H)); Theorem 21.15 (the three regret bounds of Online Gradient Descent); Theorem 21.16 (the online Perceptron bound ∣M∣≤∑tft(w⋆)+R∥w⋆∥∑tft(w⋆)+R2∥w⋆∥2|M| \le \sum_t f_t(w^\star) + R\|w^\star\|\sqrt{\sum_t f_t(w^\star)} + R^2\|w^\star\|^2∣M∣≤∑t​ft​(w⋆)+R∥w⋆∥∑t​ft​(w⋆)​+R2∥w⋆∥2 and its separable case). Further items: Corollary 21.2, Theorem 21.9, Example 21.4, Cover's impossibility, Corollary 21.12 and the Ldim⁡\operatorname{Ldim}Ldim half of Theorem 21.10.

Significance

Corollary 21.8 is one of the cleanest characterizations in learning theory: the Littlestone dimension is exactly the optimal mistake bound, with SOA attaining it and Lemma 21.6 forbidding anything better. Theorem 21.11 is the engine of the unrealizable case and of a large part of online learning: the multiplicative-weights analysis with the potential log⁡Zt\log Z_tlogZt​ gives regret 2log⁡(d)T\sqrt{2\log(d)T}2log(d)T​ against the best of ddd experts, and with the experts of pp. 298–299 it yields Theorem 21.10, regret 2Ldim⁡(H)log⁡(eT) T\sqrt{2\operatorname{Ldim}(H)\log(eT)\,T}2Ldim(H)log(eT)T​ for any class of finite Littlestone dimension. Theorem 21.15 is the online counterpart of the SGD analysis of Chapter 14, and the derivation of Theorem 21.16 from it shows how a surrogate loss chosen per round turns a regret bound into a mistake bound, the Perceptron bound of Chapter 9 falling out as the separable case. On the platform, these items give the first online-learning model, reusing Mission X's subgradients and projections.

Difficulty

Corollary 21.2 and Theorem 21.3 are counting arguments on the version space, but formally they require tracking the version space along the history and the fact that a mistake by Halving halves it. Lemma 21.6 is the adversary argument: given a shattered tree, feed the instance at the current node and the label opposite to the prediction; the resulting sequence is labeled by some h∈Hh \in Hh∈H by the shattering property, and the algorithm errs on every round. Lemma 21.7 needs the combinatorial core of the chapter: if both restricted version spaces had Littlestone dimension equal to Ldim⁡(Vt)\operatorname{Ldim}(V_t)Ldim(Vt​), their shattered trees could be glued under a new root to a deeper tree. Theorem 21.9 builds a shattered tree with all nodes at depth iii equal to xix_ixi​; Example 21.4 builds the dyadic tree. Theorem 21.11's proof is the book's: e−a≤1−a+a2/2e^{-a} \le 1 - a + a^2/2e−a≤1−a+a2/2 for a≥0a \ge 0a≥0, log⁡(1−b)≤−b\log(1 - b) \le -blog(1−b)≤−b, the telescoping potential log⁡(Zt+1/Zt)\log(Z_{t+1}/Z_t)log(Zt+1​/Zt​), the lower bound log⁡ZT+1≥−ηmin⁡i∑tvt,i\log Z_{T+1} \ge -\eta\min_i\sum_t v_{t,i}logZT+1​≥−ηmini​∑t​vt,i​, and the choice of η\etaη; the hypothesis T>2log⁡dT > 2\log dT>2logd makes η<1\eta < 1η<1. Corollary 21.12 is the reduction of hypotheses to experts, and Theorem 21.10 is the expert construction with the counting bound (21.4) ∑L≤Ldim⁡(TL)≤(eT/Ldim⁡)Ldim⁡\sum_{L \le \operatorname{Ldim}} \binom{T}{L} \le (eT/\operatorname{Ldim})^{\operatorname{Ldim}}∑L≤Ldim​(LT​)≤(eT/Ldim)Ldim (Lemma A.5) and Lemma 21.13, which simulates SOA on the labels of hhh; small horizons are covered by the trivial bound regret≤T\text{regret} \le Tregret≤T. Theorem 21.15 is the telescoping argument of Lemma 14.1 with the projection lemma of Chapter 14 at every step; Theorem 21.16 applies it to ft=1[t∈M][1−yt⟨w,xt⟩]+f_t = \mathbb{1}[t \in M][1 - y_t\langle w, x_t\rangle]_+ft​=1[t∈M][1−yt​⟨w,xt​⟩]+​ with η=∥w⋆∥/(R∣M∣)\eta = \|w^\star\|/(R\sqrt{|M|})η=∥w⋆∥/(R∣M∣​) and solves the quadratic inequality (21.6).

Formalization scope

Online algorithms are deterministic functions List (X × Y) → X → Y; a sequence is Fin T-indexed and the history at round ttt is its first ttt examples. Mistake bounds and the Littlestone dimension are suprema in ℕ∞, so mistakeBound, ldim and their comparisons are meaningful when infinite. Shattered trees are indexed by paths rather than by the book's node numbers it=2t−1+∑j<tyj2t−1−ji_t = 2^{t-1} + \sum_{j<t} y_j 2^{t-1-j}it​=2t−1+∑j<t​yj​2t−1−j, whose binary expansion is exactly the path; the two descriptions are the same tree. Halving and SOA break ties towards 111 as in the book; Consistent is stated as a property of an algorithm. The unrealizable case uses real-valued predictions with the loss ∣pt−yt∣|p_t - y_t|∣pt​−yt​∣ as the book does, and the theorems of that section assert the existence of an algorithm for each horizon TTT, because Weighted-Majority takes TTT as input. Weighted-Majority's distribution is written in unrolled form, wi(t)∝exp⁡(−η∑s<tvs,i)w^{(t)}_i \propto \exp(-\eta\sum_{s<t}v_{s,i})wi(t)​∝exp(−η∑s<t​vs,i​), which is the update rule iterated from w~(1)=(1,…,1)\tilde w^{(1)} = (1, \dots, 1)w~(1)=(1,…,1). Theorem 21.10 is stated for classes with Ldim⁡(H)<∞\operatorname{Ldim}(H) < \inftyLdim(H)<∞ and in its Ldim⁡(H)log⁡(eT)\operatorname{Ldim}(H)\log(eT)Ldim(H)log(eT) form, the log⁡∣H∣\log|H|log∣H∣ form being Corollary 21.12; its lower bound, proved in Ben-David, Pál and Shalev-Shwartz (2009), is not stated. Online Gradient Descent is driven by a subgradient selector gt(w)∈∂ft(w)g_t(w) \in \partial f_t(w)gt​(w)∈∂ft​(w) (Mission X's global subgradients), from w(0)=0w^{(0)} = 0w(0)=0, on a closed convex HHH containing the comparator; the Lipschitz parts take LipschitzWith ρ (f t) and T≥1T \ge 1T≥1. The Perceptron's MMM is the set of update rounds yt⟨w(t),xt⟩≤0y_t\langle w^{(t)}, x_t\rangle \le 0yt​⟨w(t),xt​⟩≤0, which contains every prediction mistake whatever sign⁡(0)\operatorname{sign}(0)sign(0) is and is the set the book's derivation actually uses; RRR is any bound on ∥xt∥\|x_t\|∥xt​∥ for t<Tt < Tt<T. Cover's impossibility is stated for deterministic {0,1}\{0,1\}{0,1}-valued algorithms, the setting in which the book states it.

Not stated: the Doubling Trick (Exercise 4), Exercises 1–3 (specific tight examples), the SOA-based Expert algorithm as a separate definition (it is internal to the proof of Theorem 21.10), Lemma 21.13 and Corollary 21.14 as items, and the lower bound of Theorem 21.10.

Selected references

  • S. Shalev-Shwartz, S. Ben-David, Understanding Machine Learning: From Theory to Algorithms, Cambridge University Press, 2014, Chapter 21. doi:10.1017/CBO9781107298019
  • N. Littlestone, Learning quickly when irrelevant attributes abound: a new linear-threshold algorithm, Machine Learning 2, 1988. doi:10.1007/BF00116827
  • N. Littlestone, M. K. Warmuth, The weighted majority algorithm, Information and Computation 108(2), 1994. doi:10.1006/inco.1994.1009
  • S. Ben-David, D. Pál, S. Shalev-Shwartz, Agnostic online learning, COLT 2009.
  • M. Zinkevich, Online convex programming and generalized infinitesimal gradient ascent, ICML 2003.
  • N. Cesa-Bianchi, G. Lugosi, Prediction, Learning, and Games, Cambridge University Press, 2006. doi:10.1017/CBO9780511546921
  • S. Shalev-Shwartz, Online learning and online convex optimization, Foundations and Trends in Machine Learning 4(2), 2011. doi:10.1561/2200000018
10 thms2 active usersReviewed
🏆Completed
Machine LearningOptimization·Captain: naimengye

Understanding Machine Learning XIII: Multiclass Prediction and RankingTextbook

Motivation

Binary classification is the exception in practice; most prediction tasks have many labels, a structured label space, or ask for a ranking. Chapter 17 of Shalev-Shwartz and Ben-David, Understanding Machine Learning: From Theory to Algorithms (doi:10.1017/CBO9781107298019) extends linear predictors to these settings through one idea: a class-sensitive feature mapping Ψ(x,y)\Psi(x, y)Ψ(x,y) that scores a candidate label, with the prediction hw(x)=argmax⁡y⟨w,Ψ(x,y)⟩h_w(x) = \operatorname{argmax}_y \langle w, \Psi(x, y)\ranglehw​(x)=argmaxy​⟨w,Ψ(x,y)⟩. A cost-sensitive loss Δ(y′,y)\Delta(y', y)Δ(y′,y) replaces the 0–1 loss, and the generalized hinge loss (17.3), max⁡y′(Δ(y′,y)+⟨w,Ψ(x,y′)−Ψ(x,y)⟩)\max_{y'}(\Delta(y', y) + \langle w, \Psi(x, y') - \Psi(x, y)\rangle)maxy′​(Δ(y′,y)+⟨w,Ψ(x,y′)−Ψ(x,y)⟩), is its convex surrogate: it upper bounds Δ(hw(x),y)\Delta(h_w(x), y)Δ(hw​(x),y), is tight under margin, and is convex and Lipschitz in www. Multiclass SVM is then regularized loss minimization for this loss, and Corollaries 17.1 and 17.2 transfer the guarantees of Chapters 13 and 14 with no dependence on the number of labels. The same construction handles ranking: a linear ranking predictor scores each item, the Kendall tau loss has a pairwise hinge surrogate, and the NDCG surrogate reduces to an assignment problem whose linear relaxation is exact by the Birkhoff–von Neumann theorem (Claim 17.3, Lemma 17.4).

Setting

Labels form a finite nonempty type YYY; the feature mapping takes values in Rd\mathbb{R}^dRd as in Mission VI, and the RLM rule and the SGD of Chapters 13 and 14 are those of Missions IX and X. An argmax predictor for (Ψ,w)(\Psi, w)(Ψ,w) is any hhh with h(x)h(x)h(x) maximizing ⟨w,Ψ(x,y)⟩\langle w, \Psi(x, y)\rangle⟨w,Ψ(x,y)⟩; a canonical one is fixed by choosing among the maximizers, and likewise a canonical maximizer y^\hat yy^​ in the generalized hinge loss, which gives the SGD direction Ψ(x,y^)−Ψ(x,y)\Psi(x, \hat y) - \Psi(x, y)Ψ(x,y^​)−Ψ(x,y). The cost Δ\DeltaΔ is nonnegative with Δ(y,y)=0\Delta(y, y) = 0Δ(y,y)=0. For ranking, an example is a list x1,…,xrx_1, \dots, x_rx1​,…,xr​ of instances with a score vector y∈Rry \in \mathbb{R}^ry∈Rr; the linear predictor is (⟨w,xi⟩)i(\langle w, x_i\rangle)_i(⟨w,xi​⟩)i​, the Kendall tau loss is the fraction of pairs ordered differently, using the three-valued real sign, and permutations of [r][r][r] are Mathlib's permutations of Fin r, with doubly stochastic and permutation matrices from Mathlib.

Formalization targets

Goal: Corollary 17.1

For DDD over X×YX \times YX×Y, ∥Ψ(x,y)∥≤ρ/2\|\Psi(x, y)\| \le \rho/2∥Ψ(x,y)∥≤ρ/2, B>0B > 0B>0, and the Multiclass SVM learner with λ=2ρ2/(B2m)\lambda = \sqrt{2\rho^2/(B^2 m)}λ=2ρ2/(B2m)​: ES[LDΔ(hw)]≤ES[LDg-hinge(w)]\mathbb{E}_S[L^\Delta_D(h_w)] \le \mathbb{E}_S[L^{g\text{-}hinge}_D(w)]ES​[LDΔ​(hw​)]≤ES​[LDg-hinge​(w)], and for every uuu with ∥u∥≤B\|u\| \le B∥u∥≤B, ES[LDg-hinge(w)]≤LDg-hinge(u)+8ρ2B2/m\mathbb{E}_S[L^{g\text{-}hinge}_D(w)] \le L^{g\text{-}hinge}_D(u) + \sqrt{8\rho^2B^2/m}ES​[LDg-hinge​(w)]≤LDg-hinge​(u)+8ρ2B2/m​.

Milestones

Equation (17.3). The generalized hinge loss bounds Δ(hw(x),y)\Delta(h_w(x), y)Δ(hw​(x),y) for every argmax predictor, equals it under the margin condition, and is convex and ρ\rhoρ-Lipschitz in www with ρ=max⁡y′∥Ψ(x,y′)−Ψ(x,y)∥\rho = \max_{y'}\|\Psi(x, y') - \Psi(x, y)\|ρ=maxy′​∥Ψ(x,y′)−Ψ(x,y)∥.

Corollary 17.2. SGD for multiclass learning with T≥B2ρ2/ϵ2T \ge B^2\rho^2/\epsilon^2T≥B2ρ2/ϵ2 examples has E[LDΔ(hwˉ)]≤E[LDg-hinge(wˉ)]≤LDg-hinge(u)+ϵ\mathbb{E}[L^\Delta_D(h_{\bar w})] \le \mathbb{E}[L^{g\text{-}hinge}_D(\bar w)] \le L^{g\text{-}hinge}_D(u) + \epsilonE[LDΔ​(hwˉ​)]≤E[LDg-hinge​(wˉ)]≤LDg-hinge​(u)+ϵ for every ∥u∥≤B\|u\| \le B∥u∥≤B.

Equation (17.7). The permutation induced by sorting yyy maximizes ∑iviyi\sum_i v_i y_i∑i​vi​yi​ over permutation vectors (the rearrangement inequality).

Claim 17.3. The doubly stochastic matrices are the convex hull of the permutation matrices.

Lemma 17.4. The assignment LP over doubly stochastic matrices has an optimal solution that is a permutation matrix.

Further items: Remark 17.2 (the binary case recovers the hinge loss) and the Kendall tau surrogate of §17.4.1 with its convexity and Lipschitz constant.

Significance

The generalized hinge loss is the device that lets the whole convex-learning machinery of Part II run on arbitrary finite label sets and on structured outputs, and Remark 17.3's observation that the bounds of Corollaries 17.1 and 17.2 do not depend on ∣Y∣|Y|∣Y∣ is what makes structured prediction (§17.3) and ranking with exponentially many labelings feasible. The ranking half of the chapter shows the pattern at work: the induced permutation is an argmax over a combinatorial set (17.7), so the NDCG loss admits a generalized hinge surrogate, and its subgradient is an assignment problem, solvable by the Hungarian method or, thanks to Birkhoff–von Neumann, by linear programming.

Nothing here is machine-checked except that Mathlib contains the Birkhoff–von Neumann theorem, which the corresponding item restates in the book's form. The statements are faithful with the clarifications that ties are broken canonically, that the Kendall tau surrogate is stated for tie-free score vectors (the book's rewriting of the pairwise indicator assumes sign⁡(yi−yj)≠0\operatorname{sign}(y_i - y_j) \ne 0sign(yi​−yj​)=0), and that Corollary 17.1 is stated with the measurability conventions of Mission IX.

Difficulty

Remark 17.2 is a two-element maximum and the entry point, and Equation (17.7) is Mathlib's rearrangement inequality for monovarying functions. The properties of the generalized hinge loss are elementary: the bound by choosing y′=hw(x)y' = h_w(x)y′=hw​(x), the equality by showing every term is at most 000 and the term y′=yy' = yy′=y is 000, convexity as a maximum of affine functions, and the Lipschitz bound by Cauchy–Schwarz on each term. Corollary 17.1 is Mission IX's Corollary 13.9 for the generalized hinge loss, which requires verifying convexity, the ρ\rhoρ-Lipschitz property from ∥Ψ∥≤ρ/2\|\Psi\| \le \rho/2∥Ψ∥≤ρ/2, nonnegativity and boundedness at the origin (by max⁡Δ\max\DeltamaxΔ, finite), the measurability of the loss and of the canonical argmax predictor as functions of (w,x)(w, x)(w,x), and the pointwise comparison with the Δ\DeltaΔ-loss; Corollary 17.2 is the same with Mission X's Corollary 14.12 and Claim 14.6 for the subgradient. The Kendall tau surrogate is the pairwise hinge bound under no ties, plus the convexity and Lipschitz constant of an average of hinge terms. Lemma 17.4 follows from Birkhoff–von Neumann by the averaging argument of the book, or directly from the finiteness of the permutation matrices together with the fact that a linear function on a convex hull is minimized at an extreme point.

Formalization scope

The label set is finite, so maxima over YYY are attained and the losses are well defined; maximizers are chosen canonically, and every statement about argmax predictors holds for any choice. The multivector and TF-IDF constructions of §17.2.1, the reductions of §17.1, structured output prediction (§17.3), the NDCG loss and its surrogate (17.8), and bipartite ranking (§17.5) are not stated; the NDCG construction would need the sorting permutation and the discount function and is left for a later revision. Exercises are not stated except 17.4 through Equation (17.7).

Trivializing readings are excluded: the Δ\DeltaΔ-risk in Corollaries 17.1 and 17.2 is that of a genuine argmax predictor, the Lipschitz constants are the book's, and the assignment lemma asserts optimality against every doubly stochastic matrix. Welcome contributions: the Lipschitz constant of a maximum of affine functions, the measurability of a canonical argmax over a finite label set, and the extreme-point argument of Lemma 17.4.

Selected references

  • S. Shalev-Shwartz, S. Ben-David, Understanding Machine Learning: From Theory to Algorithms, Cambridge University Press, 2014, Chapter 17. doi:10.1017/CBO9781107298019
  • K. Crammer, Y. Singer, On the algorithmic implementation of multiclass kernel-based vector machines, Journal of Machine Learning Research 2, 2001.
  • I. Tsochantaridis, T. Joachims, T. Hofmann, Y. Altun, Large margin methods for structured and interdependent output variables, Journal of Machine Learning Research 6, 2005.
  • G. Birkhoff, Tres observaciones sobre el algebra lineal, Universidad Nacional de Tucumán, Revista A 5, 1946.
  • H. W. Kuhn, The Hungarian method for the assignment problem, Naval Research Logistics Quarterly 2, 1955. doi:10.1002/nav.3800020109
12 thms2 active usersReviewed
🏆Completed
Machine LearningStatistics·Captain: naimengye

Understanding Machine Learning VII: Boosting and AdaBoostTextbook

Motivation

Boosting answers a question raised by Kearns and Valiant: can a learner that is only slightly better than random guessing be turned into one that is arbitrarily accurate? Chapter 10 of Shalev-Shwartz and Ben-David, Understanding Machine Learning: From Theory to Algorithms (doi:10.1017/CBO9781107298019) defines γ-weak learnability (Definition 10.1), the PAC requirement with the accuracy ϵ\epsilonϵ replaced by the fixed value 1/2−γ1/2 - \gamma1/2−γ, and presents AdaBoost, the algorithm of Freund and Schapire that, given weak hypotheses, reweights the training set round by round and outputs a weighted majority vote. The chapter's main result (Theorem 10.2) is that the training error of AdaBoost's output decreases as e−2γ2Te^{-2\gamma^2 T}e−2γ2T in the number of rounds. Since the output is a halfspace over the predictions of TTT base hypotheses, the chapter then bounds the VC-dimension of that class (Lemma 10.3), so that the number of rounds becomes a knob for the bias–complexity tradeoff. Example 10.1 shows a concrete weak learner, ERM over decision stumps for the class of 3-piece classifiers on the line, and the chapter remarks that, statistically, weak learnability is no easier than strong learnability: a class of infinite VC-dimension is not weakly learnable either.

Setting

The framework is that of Missions I and IV: binary classification over a domain XXX with the 0–1 loss, distributions DDD over XXX with a labeling function fff, learners as functions of the sample, the VC-dimension, and ERM. Labels and hypotheses are Boolean, with ±1\pm 1±1 values obtained through sgn⁡(true)=1\operatorname{sgn}(\text{true}) = 1sgn(true)=1, sgn⁡(false)=−1\operatorname{sgn}(\text{false}) = -1sgn(false)=−1, and sign⁡(z)\operatorname{sign}(z)sign(z) is true exactly when z>0z > 0z>0. A γ-weak learner for HHH with the function mH:(0,1)→Nm_H : (0,1) \to \mathbb{N}mH​:(0,1)→N returns, for every δ\deltaδ, every DDD and every measurable fff realizable by HHH, a hypothesis with L(D,f)(h)≤1/2−γL_{(D,f)}(h) \le 1/2 - \gammaL(D,f)​(h)≤1/2−γ with probability at least 1−δ1 - \delta1−δ once m≥mH(δ)m \ge m_H(\delta)m≥mH​(δ); the failure event is bounded in outer measure as in Definition 3.1.

AdaBoost is formalized as a deterministic function of the sample S=(x1,y1),…,(xm,ym)S = (x_1, y_1), \dots, (x_m, y_m)S=(x1​,y1​),…,(xm​,ym​) and of the sequence of weak hypotheses h0,h1,…h_0, h_1, \dotsh0​,h1​,… that the weak learner returned. The distributions are defined by recursion: D(0)D^{(0)}D(0) is uniform, ϵt=∑iDi(t)1[ht(xi)≠yi]\epsilon_t = \sum_i D^{(t)}_i \mathbb{1}[h_t(x_i) \ne y_i]ϵt​=∑i​Di(t)​1[ht​(xi​)=yi​], wt=12log⁡(1/ϵt−1)w_t = \frac12 \log(1/\epsilon_t - 1)wt​=21​log(1/ϵt​−1), and Di(t+1)∝Di(t)exp⁡(−wtyiht(xi))D^{(t+1)}_i \propto D^{(t)}_i \exp(-w_t y_i h_t(x_i))Di(t+1)​∝Di(t)​exp(−wt​yi​ht​(xi​)); the output after TTT rounds is x↦sign⁡(∑t<Twtht(x))x \mapsto \operatorname{sign}(\sum_{t < T} w_t h_t(x))x↦sign(∑t<T​wt​ht​(x)). Rounds are indexed from 000, so D(0)D^{(0)}D(0) is the book's D(1)D^{(1)}D(1). The class L(B,T)L(B, T)L(B,T) of Equation (10.4) consists of the functions x↦sign⁡(∑t=1Twtht(x))x \mapsto \operatorname{sign}(\sum_{t=1}^T w_t h_t(x))x↦sign(∑t=1T​wt​ht​(x)) with ht∈Bh_t \in Bht​∈B. Decision stumps over R\mathbb{R}R are the threshold functions x↦[θ<x]x \mapsto [\theta < x]x↦[θ<x] and their negations x↦[x≤θ]x \mapsto [x \le \theta]x↦[x≤θ]; a 3-piece classifier is bbb outside [θ1,θ2][\theta_1, \theta_2][θ1​,θ2​] and −b-b−b inside, with θ1<θ2\theta_1 < \theta_2θ1​<θ2​.

Formalization targets

Goal: Theorem 10.2

If γ>0\gamma > 0γ>0 and every round t<Tt < Tt<T has 0<ϵt≤1/2−γ0 < \epsilon_t \le 1/2 - \gamma0<ϵt​≤1/2−γ, then the empirical 0–1 risk of AdaBoost's output after TTT rounds is at most exp⁡(−2γ2T)\exp(-2\gamma^2 T)exp(−2γ2T).

Milestones

§10.1. A class of infinite VC-dimension is not γ-weak-learnable for any γ>0\gamma > 0γ>0 (domain with measurable singletons, measurable hypotheses).

Example 10.1. There is one sample-size function with which every ERM learner over the decision stumps is a 1/121/121/12-weak learner for the 3-piece classifiers.

Exercise 10.3. For a nonempty sample and ϵt∈(0,1)\epsilon_t \in (0,1)ϵt​∈(0,1), the error of hth_tht​ under D(t+1)D^{(t+1)}D(t+1) is exactly 1/21/21/2.

Lemma 10.3. If T≥3T \ge 3T≥3 and VCdim(B)=d≥3\mathrm{VCdim}(B) = d \ge 3VCdim(B)=d≥3, then VCdim(L(B,T))≤T(d+1)(3log⁡(T(d+1))+2)\mathrm{VCdim}(L(B,T)) \le T(d+1)(3\log(T(d+1)) + 2)VCdim(L(B,T))≤T(d+1)(3log(T(d+1))+2).

Further item: Exercise 10.4 (1), VCdim(B)≤VCdim(L(B,T))\mathrm{VCdim}(B) \le \mathrm{VCdim}(L(B,T))VCdim(B)≤VCdim(L(B,T)) for T≥1T \ge 1T≥1.

Significance

Theorem 10.2 is the reason AdaBoost works and the template for every analysis of boosting: a potential function, here 1m∑ie−yift(xi)\frac1m \sum_i e^{-y_i f_t(x_i)}m1​∑i​e−yi​ft​(xi​), bounds the 0–1 training error and contracts by the factor 2ϵt(1−ϵt)≤1−4γ22\sqrt{\epsilon_{t}(1-\epsilon_{t})} \le \sqrt{1 - 4\gamma^2}2ϵt​(1−ϵt​)​≤1−4γ2​ at every round. Lemma 10.3 supplies the other half of the picture, an estimation-error bound growing only like T⋅VCdim(B)T \cdot \mathrm{VCdim}(B)T⋅VCdim(B) up to logarithms, so that Theorem 6.8 turns the pair into a generalization guarantee for boosting. The remark of §10.1 places weak learning in the statistical landscape of Part I: the VC-dimension characterizes it too, and the gain of boosting is computational.

Nothing here is machine-checked. Two points where the book's text needs care are built into the statements. The weight wtw_twt​ is undefined when ϵt=0\epsilon_t = 0ϵt​=0, and the algorithm's normalization then divides 000 by 000; in Lean the logarithm of a negative number is 000, so with ϵt=0\epsilon_t = 0ϵt​=0 the formal algorithm would ignore a perfect weak hypothesis and the bound could fail. The theorems therefore assume ϵt>0\epsilon_t > 0ϵt​>0, which is the case in which the book's formulas are defined. And the book's derivation of "infinite VC-dimension implies not weakly learnable" from the lower bound of Theorem 6.8 at ϵ=1/2−γ\epsilon = 1/2 - \gammaϵ=1/2−γ uses that bound outside the range in which Chapter 28 proves it; the statement itself is true, by the kmkmkm-point form of the No-Free-Lunch argument (Exercise 5.3 of Mission III) and Lemma B.1.

Difficulty

Exercise 10.4 (1) is a one-line embedding of BBB into L(B,T)L(B, T)L(B,T) with the weights (1,0,…,0)(1, 0, \dots, 0)(1,0,…,0) and is the entry point. Exercise 10.3 is the computation of the book: after the update, the weight of the mistakes of hth_tht​ is ewtϵte^{w_t}\epsilon_tewt​ϵt​ and the weight of the correct examples is e−wt(1−ϵt)e^{-w_t}(1 - \epsilon_t)e−wt​(1−ϵt​), and with ewt=(1−ϵt)/ϵte^{w_t} = \sqrt{(1-\epsilon_t)/\epsilon_t}ewt​=(1−ϵt​)/ϵt​​ these are equal. Theorem 10.2 needs, by induction on the round, the closed form Di(t)=e−yift(xi)/∑je−yjft(xj)D^{(t)}_i = e^{-y_i f_{t}(x_i)}/\sum_j e^{-y_j f_{t}(x_j)}Di(t)​=e−yi​ft​(xi​)/∑j​e−yj​ft​(xj​) of the distribution, the pointwise bound 1[sign⁡(f(x))≠y]≤e−yf(x)\mathbb{1}[\operatorname{sign}(f(x)) \ne y] \le e^{-y f(x)}1[sign(f(x))=y]≤e−yf(x) for the sign convention used, the telescoping product (10.2), the identity Zt+1/Zt=2ϵt(1−ϵt)Z_{t+1}/Z_t = 2\sqrt{\epsilon_t(1-\epsilon_t)}Zt+1​/Zt​=2ϵt​(1−ϵt​)​, the monotonicity of a(1−a)a(1-a)a(1−a) on [0,1/2][0, 1/2][0,1/2] and 1−a≤e−a1 - a \le e^{-a}1−a≤e−a. Lemma 10.3 counts dichotomies: Sauer's lemma bounds the restrictions of BBB to a shattered set by (em/d)d(em/d)^d(em/d)d, choosing TTT of them gives (em/d)dT(em/d)^{dT}(em/d)dT, the halfspaces of RT\mathbb{R}^TRT contribute (em/T)T(em/T)^T(em/T)T by Theorem 9.2, and the inequality 2m≤m(d+1)T2^m \le m^{(d+1)T}2m≤m(d+1)T is solved with Lemma A.1; the finite-VC lower bound m≤d+1m \le d + 1m≤d+1 handles small mmm, and the numeric slack of the book's chain must be checked. Example 10.1 combines a geometric observation, that one of the three regions of a 3-piece classifier has mass at most 1/31/31/3 and a stump agrees with the other two, with the agnostic guarantee for ERM over the stumps from Theorem 6.7, applied with accuracy 1/121/121/12; the best stump may only approach error 1/31/31/3 because constant functions are not stumps, and the slack absorbs this. The §10.1 remark is the argument sketched above.

Formalization scope

AdaBoost is a function of the sample and of the returned weak hypotheses; the weak learner's randomness and its failure probability (Remark 10.2) are not modelled, and Theorem 10.2 is the deterministic statement the book proves. Rounds are indexed from 000. The output uses sign⁡(0)=\operatorname{sign}(0) = sign(0)= negative, consistently with Mission VI. The class L(B,T)L(B,T)L(B,T) is a set of functions, so Lemma 10.3 is a statement about the VC-dimension of Mission IV, with the bound taken in N∪{∞}\mathbb{N} \cup \{\infty\}N∪{∞} through the integer part of the real right-hand side and the natural logarithm. Decision stumps are closed under negation, as the book's sign⁡(x−θ)⋅b\operatorname{sign}(x - \theta)\cdot bsign(x−θ)⋅b; constant functions are not stumps. The efficient ERM for decision stumps (§10.1.1), the face-recognition features (§10.4), Exercises 10.1, 10.2, 10.4 (2)–(3) and 10.5 are not stated. The claims of §10.3 that piecewise-constant classifiers with TTT pieces lie in L(stumps,T)L(\text{stumps}, T)L(stumps,T) and that this class shatters T+1T+1T+1 points depend on treating sign⁡(x−(−∞))\operatorname{sign}(x - (-\infty))sign(x−(−∞)) as a stump and on the sign convention; with real thresholds, L(stumps,2)L(\text{stumps}, 2)L(stumps,2) does not shatter three points under either convention, so these claims are not stated.

Trivializing readings are excluded: the weak-error hypotheses are strict where the book's formulas require it, the VC bounds are in N∪{∞}\mathbb{N} \cup \{\infty\}N∪{∞}, and the weak-learner guarantee quantifies over all distributions and all realizable labelings. Welcome contributions: the closed form of D(t)D^{(t)}D(t), the contraction identity for Zt+1/ZtZ_{t+1}/Z_tZt+1​/Zt​, and the dichotomy count behind Lemma 10.3.

Selected references

  • S. Shalev-Shwartz, S. Ben-David, Understanding Machine Learning: From Theory to Algorithms, Cambridge University Press, 2014, Chapter 10. doi:10.1017/CBO9781107298019
  • Y. Freund, R. E. Schapire, A decision-theoretic generalization of on-line learning and an application to boosting, Journal of Computer and System Sciences 55(1), 1997. doi:10.1006/jcss.1997.1504
  • R. E. Schapire, The strength of weak learnability, Machine Learning 5(2), 1990. doi:10.1007/BF00116037
  • M. Kearns, L. Valiant, Cryptographic limitations on learning Boolean formulae and finite automata, Journal of the ACM 41(1), 1994. doi:10.1145/174644.174647
  • R. E. Schapire, Y. Freund, Boosting: Foundations and Algorithms, MIT Press, 2012.
8 thms2 active usersReviewed
🏆Completed
Machine LearningProbabilityStatistics·Captain: naimengye

An Introduction to Computational Learning Theory II: Occam's Razor, Set Cover and Decision ListsTextbook

Motivation

Chapter 2 of Kearns and Vazirani, An Introduction to Computational Learning Theory (MIT Press, 1994, doi:10.7551/mitpress/3897.001.0001), gives a formal justification of Occam's Razor inside the PAC model. An Occam algorithm is judged not by the predictive power of its hypothesis but by how succinctly the hypothesis explains the sample before it: it must be consistent with the data and short, in the sense that its representation has fewer bits than the data it reproduces. The chapter's theorems say that, when the examples are drawn independently from a fixed distribution, such compression automatically yields prediction: a consistent hypothesis drawn from a small class is, with high probability, accurate on unseen examples. This turns the design of PAC learning algorithms into a combinatorial task, finding a short consistent hypothesis, and the chapter demonstrates the method three times: it shaves a logarithmic factor off the conjunction bound of Chapter 1, it learns conjunctions with few relevant variables through the greedy set-cover heuristic, and it learns Rivest's decision lists, a class strictly more expressive than kkk-CNF and kkk-DNF, by a greedy algorithm whose correctness is a one-line consequence of consistency.

Setting

The framework is that of Mission I: a measurable instance space, concepts as boolean functions, a target distribution DDD, the product law of a sample of mmm labeled examples, the error error(h)=Pr⁡x∼D[h(x)≠c(x)]\mathrm{error}(h) = \Pr_{x \sim D}[h(x) \neq c(x)]error(h)=Prx∼D​[h(x)=c(x)], and consistency of a hypothesis with a sample. Hypotheses may be represented by binary strings through a representation map, with size the bit length; an (α,β)(\alpha, \beta)(α,β)-Occam algorithm outputs a consistent hypothesis of size at most (n⋅size(c))αmβ(n \cdot \mathrm{size}(c))^\alpha m^\beta(n⋅size(c))αmβ with 0≤β<10 \le \beta < 10≤β<1. The set cover problem asks for a minimum subcollection of a collection S\mathcal{S}S of subsets of a finite universe UUU that covers UUU; the greedy heuristic repeatedly picks the set covering the most uncovered elements. A kkk-decision list is a sequence of conditions, each a conjunction of at most kkk literals, with a bit attached to each and a default bit; it evaluates to the bit of the first satisfied condition. The greedy decision-list algorithm repeatedly finds a useful condition, one satisfied by some remaining examples all of which carry the same label, appends it with that label and removes those examples.

Formalization targets

Goal: Theorem 2.2 (Occam's Razor, cardinality version)

For a finite hypothesis class HHH, a target ccc, a distribution DDD and 0<ϵ≤10 < \epsilon \le 10<ϵ≤1, the probability that a sample of mmm examples is consistent with some h∈Hh \in Hh∈H of error greater than ϵ\epsilonϵ is at most ∣H∣(1−ϵ)m|H|(1 - \epsilon)^m∣H∣(1−ϵ)m; hence

m≥1ϵ(ln⁡∣H∣+ln⁡1δ)m \ge \frac{1}{\epsilon}\Big(\ln|H| + \ln\frac{1}{\delta}\Big)m≥ϵ1​(ln∣H∣+lnδ1​)

makes this probability at most δ\deltaδ, and any algorithm that outputs a consistent hypothesis from HHH has error greater than ϵ\epsilonϵ with probability at most ∣H∣(1−ϵ)m|H|(1-\epsilon)^m∣H∣(1−ϵ)m.

Milestones

Theorem 2.1 (an (α,β)(\alpha, \beta)(α,β)-Occam algorithm is a PAC algorithm, with explicit sample-size conditions in place of the constant aaa); the greedy set-cover bound of §2.3 (∣Ui∣≤(1−1/opt)i∣U∣|U_i| \le (1 - 1/\mathrm{opt})^i|U|∣Ui​∣≤(1−1/opt)i∣U∣ and optln⁡∣U∣\mathrm{opt}\ln|U|optln∣U∣ sets cover); the improved conjunction bound of §2.2; Theorem 2.3 (kkk-decision lists are PAC learnable by the greedy algorithm, which never fails and whose outputs are consistent).

Significance

Theorem 2.2 is the single most used tool of the subject: every finite-class sample bound, including the ones for conjunctions, decision lists, kkk-CNF and the discretized geometric classes, is an instance of it, and the Vapnik–Chervonenkis theory of Chapter 3 is its extension to infinite classes with the growth function in place of ∣H∣|H|∣H∣. Theorem 2.1 is the philosophical statement, that succinct explanation implies prediction, and its converse (Exercise 2.3, and more strongly the boosting theorem of Chapter 4) makes Occam learning equivalent to PAC learning. The greedy set-cover bound is Chvátal's classical approximation guarantee, used in the book for learning with few relevant variables and again in later chapters. Theorem 2.3 is Rivest's result, and its proof exhibits the pattern "consistency by construction plus a counting bound" in its purest form. None of these is machine-checked. Their formalization gives the platform the union-bound-over-a-finite-class argument once and for all, in a form that the later missions of this series reuse verbatim.

Difficulty

Theorem 2.2 requires that, for a fixed measurable hypothesis with error greater than ϵ\epsilonϵ, the product law gives the event "consistent with all mmm examples" probability at most (1−ϵ)m(1 - \epsilon)^m(1−ϵ)m, which is the product structure of Measure.pi applied to the event that each coordinate lies in the set where hhh agrees with ccc; the union bound over HHH and the elementary inequality (1−ϵ)m≤e−ϵm(1 - \epsilon)^m \le e^{-\epsilon m}(1−ϵ)m≤e−ϵm finish. Theorem 2.1 adds only the count of binary strings of length at most KKK and arithmetic with real exponents. The set-cover bound is a discrete induction: an optimal cover of UUU restricted to the uncovered elements has at most opt\mathrm{opt}opt sets, so one of them, hence the greedy choice, covers a 1/opt1/\mathrm{opt}1/opt fraction; the covering clause needs the strict inequality 1−1/opt<e−1/opt1 - 1/\mathrm{opt} < e^{-1/\mathrm{opt}}1−1/opt<e−1/opt. Theorem 2.3 needs that a run of the greedy algorithm never repeats a condition (its satisfied examples are removed), so outputs lie in an explicit finite class, that the first condition of the target list satisfied by a remaining example is useful, and that a complete run is consistent; the bound is then Theorem 2.2.

Formalization scope

Everything is in the sample-complexity sense on the Mission I framework; running time is not modelled and "efficient" is dropped from every statement, which is recorded in the natural-language statements. Theorem 2.2's constant bbb is 111 with natural logarithms; Theorem 2.1's constant aaa is replaced by three explicit sufficient conditions. Hypotheses in the finite class are required to be measurable. The greedy heuristic and the greedy decision-list algorithm are relations (any tie-breaking), and the theorems quantify over every run; the decision-list theorem is stated for every kkk with the kkk-conjunctions as conditions, so that its hypothesis is a kkk-decision list rather than the expansion the book sketches for k>1k > 1k>1. The conjunction count is 3n+13^n + 13n+1, including the empty concept that the elimination algorithm outputs on a sample without positive examples. The few-relevant-variables algorithm of §2.3 is not stated (its bound has an unspecified constant and mmm on both sides). Hypotheses: 0<ϵ≤10 < \epsilon \le 10<ϵ≤1, 0<δ0 < \delta0<δ (and δ<1\delta < 1δ<1 where ln⁡(1/δ)\ln(1/\delta)ln(1/δ) must be nonnegative).

Trivializing readings are excluded: the bad-consistent event is over all of HHH, the decision-list failure event ranges over every possible output, and the set-cover bound holds for every greedy run. Welcome contributions: the product-law bound for a fixed hypothesis, the union bound over a finset, the string-counting lemma, and the no-repetition lemma for greedy runs.

Selected references

  • M. J. Kearns, U. V. Vazirani, An Introduction to Computational Learning Theory, MIT Press, 1994, Chapter 2. doi:10.7551/mitpress/3897.001.0001
  • A. Blumer, A. Ehrenfeucht, D. Haussler, M. K. Warmuth, Occam's razor, Information Processing Letters 24(6), 1987. doi:10.1016/0020-0190(87)90114-1
  • V. Chvátal, A greedy heuristic for the set-covering problem, Mathematics of Operations Research 4(3), 1979. doi:10.1287/moor.4.3.233
  • R. L. Rivest, Learning decision lists, Machine Learning 2(3), 1987. doi:10.1007/BF00058680
  • D. Haussler, Quantifying inductive bias: AI learning algorithms and Valiant's learning framework, Artificial Intelligence 36(2), 1988. doi:10.1016/0004-3702(88)90002-1
7 thms2 active usersReviewed
🏆Completed
Machine LearningProbabilityStatistics·Captain: naimengye

An Introduction to Computational Learning Theory I: The PAC Model, Conjunctions, Rectangles and 3-CNFTextbook

Motivation

Chapter 1 of Kearns and Vazirani, An Introduction to Computational Learning Theory (MIT Press, 1994, doi:10.7551/mitpress/3897.001.0001), introduces Valiant's Probably Approximately Correct model, the framework for the whole book. A learner sees labeled examples of an unknown target concept drawn from an unknown but fixed distribution, and must output a hypothesis that, with probability at least 1−δ1 - \delta1−δ over the sample, misclassifies a fresh example with probability at most ϵ\epsilonϵ; the model is distribution-free, and the hypothesis is judged on the same distribution it was trained on. The chapter's three positive results are the templates for everything that follows. The rectangle game (Theorem 1.1) shows that an infinite class can be learned from a finite sample by exploiting the geometry of the error region. Conjunctions (Theorem 1.2) are learned by the elimination algorithm, whose analysis, a union bound over "bad" literals, is the prototype of every sample-size bound in the book. And 3-CNF formulae (Theorem 1.4) are learned by a change of variables that reduces them to conjunctions, which, set against the intractability of learning 3-term DNF as 3-term DNF (Theorem 1.3), is the reason the final definition of the model lets the hypothesis class differ from the concept class.

Setting

The instance space is a measurable space XXX, a concept is a function c:X→{0,1}c : X \to \{0, 1\}c:X→{0,1}, and the target distribution DDD is a probability measure on XXX. The error of a hypothesis hhh is error(h)=Pr⁡x∼D[h(x)≠c(x)]\mathrm{error}(h) = \Pr_{x \sim D}[h(x) \ne c(x)]error(h)=Prx∼D​[h(x)=c(x)]. A sample of mmm examples is S=((x1,c(x1)),…,(xm,c(xm)))S = ((x_1, c(x_1)), \dots, (x_m, c(x_m)))S=((x1​,c(x1​)),…,(xm​,c(xm​))) with the xix_ixi​ independent draws from DDD; a learning algorithm is a function from samples to hypotheses; it has the (ϵ,δ)(\epsilon, \delta)(ϵ,δ) guarantee on a class CCC if for every c∈Cc \in Cc∈C and every DDD the probability that its hypothesis has error greater than ϵ\epsilonϵ is at most δ\deltaδ; CCC is PAC learnable using HHH if for all ϵ,δ∈(0,1/2)\epsilon, \delta \in (0, 1/2)ϵ,δ∈(0,1/2) some sample size and algorithm with hypotheses in HHH achieve the guarantee. For X={0,1}nX = \{0,1\}^nX={0,1}n a literal is a variable or its negation and a conjunction is a finite set of literals; the elimination algorithm starts from all 2n2n2n literals and deletes every literal contradicted by a positive example. For X=R2X = \mathbb{R}^2X=R2 the concepts are the closed axis-aligned rectangles and the tightest-fit algorithm returns the smallest rectangle containing the positive examples. A 3-CNF formula is a conjunction of clauses of at most three literals; the expansion a↦a′a \mapsto a'a↦a′ records for each triple (u,v,w)(u, v, w)(u,v,w) of literals the value u∨v∨wu \vee v \vee wu∨v∨w.

Formalization targets

Goal: Theorem 1.2

Conjunctions of boolean literals are PAC learnable by the elimination algorithm: for every target conjunction, every distribution on {0,1}n\{0,1\}^n{0,1}n and ϵ,δ>0\epsilon, \delta > 0ϵ,δ>0, the elimination hypothesis is consistent with every sample labeled by the target, and with

m≥2nϵ(ln⁡2n+ln⁡1δ)m \ge \frac{2n}{\epsilon}\Big(\ln 2n + \ln\frac{1}{\delta}\Big)m≥ϵ2n​(ln2n+lnδ1​)

examples its error exceeds ϵ\epsilonϵ with probability at most δ\deltaδ; hence conjunctions are PAC learnable using conjunctions.

Milestones

Theorem 1.1 (rectangles by the tightest fit, m≥(4/ϵ)ln⁡(4/δ)m \ge (4/\epsilon)\ln(4/\delta)m≥(4/ϵ)ln(4/δ)) and Theorem 1.4 (3-CNF formulae by elimination over the (2n)3(2n)^3(2n)3 expanded variables, the hypothesis being itself a 3-CNF formula).

Significance

Theorem 1.2 is Valiant's original result and the first instance of the two ideas that organize the subject: a hypothesis that is more specific than the target never errs on negative examples, and the error of the hypothesis decomposes as a sum over a polynomial number of "bad" events, each of which is avoided with probability exponentially close to one. Theorem 1.1 is the first learning result for an infinite concept class and the seed of the Vapnik–Chervonenkis theory of Chapter 3. Theorem 1.4 is the first reduction between learning problems and, together with Theorem 1.3, the demonstration that the choice of hypothesis representation can separate tractable from intractable. None of these theorems is machine-checked. Formalizing them fixes, for the rest of the series, the framework in which the error of a hypothesis, the law of a sample and the learnability of a class are stated, so that the Occam, VC-dimension, boosting and noise results of later chapters can be stated on the same objects.

Difficulty

The elimination analysis needs that the hypothesis contains every literal of the target, that its error is at most the sum over its literals zzz of p(z)=Pr⁡[c(a)=1∧z=0 in a]p(z) = \Pr[c(a) = 1 \wedge z = 0 \text{ in } a]p(z)=Pr[c(a)=1∧z=0 in a], and that a literal with p(z)≥ϵ/2np(z) \ge \epsilon/2np(z)≥ϵ/2n survives mmm independent examples with probability at most (1−ϵ/2n)m(1 - \epsilon/2n)^m(1−ϵ/2n)m; the probabilistic content is the independence of the coordinates of the sample law, which is a product measure, and the inequality 1−x≤e−x1 - x \le e^{-x}1−x≤e−x. The rectangle analysis is the four-strip argument, which for an arbitrary distribution, possibly with atoms, requires choosing the strip {y≥t∗}\{y \ge t^*\}{y≥t∗} with t∗t^*t∗ the supremum of the heights at which the strip has weight at least ϵ/4\epsilon/4ϵ/4 and using the left-continuity of the weight in the height. The 3-CNF result transports the conjunction bound along the injective expansion: the pushforward of the sample law is the sample law of the expanded distribution, and the error of the composed hypothesis equals the error over the expanded variables. The deduction of PACLearnable from the explicit bounds is a choice of sample size.

Formalization scope

The model is stated in the sample-complexity sense: algorithms are functions of the sample, and the guarantee bounds the outer measure of the failure set under the product law of the sample, which is the strong form of "with probability at least 1−δ1 - \delta1−δ" and requires no measurability of the failure set. Running time, and hence "efficiently", is not modelled, and the hardness Theorem 1.3 is not stated; each theorem carries instead the explicit algorithm and the explicit sample bound of the book's analysis. Concepts on general instance spaces are required to be measurable in the guarantee. The cube is {0,1}n\{0,1\}^n{0,1}n as functions Fin n → Bool; the expanded variables are indexed by the triples of literals, so N=(2n)3N = (2n)^3N=(2n)3 is a Fintype.card. Rectangles are closed, possibly empty; the tightest fit of a sample without positive examples is the empty concept. Hypotheses: ϵ>0\epsilon > 0ϵ>0, 0<δ<10 < \delta < 10<δ<1; the sample bounds are as printed, with natural logarithms.

Trivializing readings are excluded: the failure bound is uniform over all distributions and all targets in the class, consistency is asserted for every sample labeled by the target, and the learnability clause quantifies over all ϵ,δ\epsilon, \deltaϵ,δ. Welcome contributions: the product-law bound Pr⁡[a fixed event of probability≥p is missed by all m examples]≤(1−p)m\Pr[\text{a fixed event of probability} \ge p \text{ is missed by all } m \text{ examples}] \le (1 - p)^mPr[a fixed event of probability≥p is missed by all m examples]≤(1−p)m, the union bound over literals, and the pushforward identity for the expanded sample.

Selected references

  • M. J. Kearns, U. V. Vazirani, An Introduction to Computational Learning Theory, MIT Press, 1994, Chapter 1. doi:10.7551/mitpress/3897.001.0001
  • L. G. Valiant, A theory of the learnable, Communications of the ACM 27(11), 1984. doi:10.1145/1968.1972
  • A. Blumer, A. Ehrenfeucht, D. Haussler, M. K. Warmuth, Learnability and the Vapnik–Chervonenkis dimension, Journal of the ACM 36(4), 1989. doi:10.1145/76359.76371
  • L. Pitt, L. G. Valiant, Computational limitations on learning from examples, Journal of the ACM 35(4), 1988. doi:10.1145/48014.63140
4 thms2 active usersReviewed
🏆Completed
Operations Research·Captain: naimengye

Fundamentals of Supply Chain Theory XII: The Vehicle Routing ProblemTextbook

Many vehicles, one depot

The vehicle routing problem asks for the cheapest set of delivery routes from a depot to a set of customers when each vehicle can carry only so much. It generalizes the traveling salesman problem, which is the case of a single vehicle of unlimited capacity, and it is the operational problem behind every distribution fleet. Exact methods reach a few hundred customers; the questions that shape fleet design are structural: how does the optimal routing cost compare with the cost of a single grand tour, and how does it grow with the number of customers? Haimovich and Rinnooy Kan (1985) answered both for unit demands by bounding the optimal cost above and below in terms of the optimal TSP tour and the average distance to the depot. Chapter 11 of Snyder and Shen's Fundamentals of Supply Chain Theory (2019) presents that result with its full proof as Theorem 11.6, and it is the goal of this mission.

Setting

Nodes are the depot 000 and customers 1,…,n1, \dots, n1,…,n, with distances cijc_{ij}cij​ that are symmetric, nonnegative and satisfy the triangle inequality (VRPMetric). Every customer has demand 111 and every vehicle capacity CCC, so a route is a sequence of at most CCC distinct customers, served by one vehicle that leaves the depot, visits them in order and returns; its length is routeCost c L. A solution is a family of routes visiting every customer exactly once (IsVRPSolution), with total length solutionCost; the number of routes is free. The optimal VRP value z∗z^*z∗ (vrpOpt c C) is the least total length, the optimal TSP value zTz_TzT​ (tspOpt c) is the least length of a single route through all customers, and cˉ\bar ccˉ (avgDepotDist) is the average distance from the depot to a customer.

Formalization targets

Goal: Theorem 11.6

For every metric instance with n≥1n \ge 1n≥1 customers and capacity C≥1C \ge 1C≥1,

max⁡{2nCcˉ, zT}  ≤  z∗  ≤  2⌈nC⌉cˉ+(1−1C)zT.\max\Big\{2\frac{n}{C}\bar c,\ z_T\Big\} \;\le\; z^* \;\le\; 2\Big\lceil\frac{n}{C}\Big\rceil\bar c + \Big(1 - \frac{1}{C}\Big)z_T.max{2Cn​cˉ, zT​}≤z∗≤2⌈Cn​⌉cˉ+(1−C1​)zT​.

This is vrp_tsp_bounds.

Supporting targets

The three steps of the book's proof: the radial bound 2nCcˉ≤z∗2\frac{n}{C}\bar c \le z^*2Cn​cˉ≤z∗, obtained route by route from the triangle inequality and the capacity; the routing bound zT≤z∗z_T \le z^*zT​≤z∗; and the iterated optimal tour partition bound (11.59), that for any tour Γ\GammaΓ through all customers some partition of its customer sequence into ⌈n/C⌉\lceil n/C\rceil⌈n/C⌉ consecutive routes costs at most 2⌈n/C⌉cˉ+(1−⌈n/C⌉/n) z(Γ)2\lceil n/C\rceil\bar c + (1 - \lceil n/C\rceil/n)\,z(\Gamma)2⌈n/C⌉cˉ+(1−⌈n/C⌉/n)z(Γ).

The chapter's other numbered results are not targets: Proposition 11.1 (state-space relaxation of the routing dynamic program) and Theorem 11.2 (the capacitated comb inequality, proof omitted, which needs the bin-packing function v(S)v(S)v(S)), and Theorems 11.3, 11.5, 11.7 and Lemma 11.4 (almost-sure asymptotics of random instances and the location-based heuristic), whose proofs the book cites.

Significance

Theorem 11.6 is the quantitative link between routing and the two things a planner can estimate without solving anything: the TSP length, which grows like n\sqrt{n}n​ for random customers, and the average depot distance. It says that the VRP cost is the TSP cost plus a radial term 2cˉ2\bar c2cˉ per vehicle, and that this decomposition is exact up to a factor bounded by the capacity. The radial term explains Theorem 11.7, that the optimal cost grows linearly in nnn for fixed capacity, and the tour partition heuristic in the proof is a practical route-first-cluster-second method with a provable guarantee. The bounds are the basis of the continuous approximation formulas used in strategic distribution design.

None of these results has a machine-checked proof. The book proves Theorem 11.6 in full. The formal treatment of routes as lists and of the averaging argument over rotations of a tour is reusable for the capacitated heuristics of Sect. 11.3.

Difficulty

The upper bound is an averaging argument that is easy to state and fiddly to formalize: for each of the nnn rotations of the tour's customer sequence, the sequence is cut into blocks of CCC, and one must count, across all rotations, how often each customer is the first or last of a block and how often each tour edge is cut. The counts, ℓ=⌈n/C⌉\ell = \lceil n/C\rceilℓ=⌈n/C⌉ each, hold only after the rotations are indexed carefully, and the passage from the average to the best rotation needs the sum of the nnn solution costs computed exactly.

The lower bound has two parts with different flavors. The radial part needs, for each route, that the closed route from the depot is at least twice the largest depot distance among its customers, which is the triangle inequality applied along the route, followed by an averaging step that uses the capacity. The routing part is a shortcutting argument: the routes of a solution concatenate into a closed walk that revisits the depot, and removing the repeated depot visits must not increase the length. The obvious idea, that a VRP solution is itself a tour, is false, and the shortcut has to be constructed.

Formalization scope

Routes are lists of customers, solutions are lists of routes, and the feasibility predicate requires nonempty routes of length at most CCC avoiding the depot, with the concatenation of all routes a duplicate-free list containing every customer. Costs use the closed walk through the depot followed by the route. The optimal values are infima of finite nonempty sets of reals, nonempty because singleton routes are feasible when C≥1C \ge 1C≥1. The ceiling ⌈n/C⌉\lceil n/C\rceil⌈n/C⌉ is Mathlib's Nat.ceil of the real quotient. The number of vehicles is unrestricted, as the section assumes; the fixed-fleet version of the problem is not modeled.

The TSP value zTz_TzT​ is defined as the least route cost over all orderings of the customers, so no separate tour model is needed and the mission does not depend on the TSP mission of this series.

The definition module is shared by all five items. Problem 11.18 (tightness of both bounds) and Theorem 11.7 for deterministic instance families are natural extensions on the same definitions.

Selected references

  • L. V. Snyder and Z.-J. M. Shen, Fundamentals of Supply Chain Theory, 2nd ed., Wiley, 2019, Chapter 11. https://doi.org/10.1002/9781119584445
  • M. Haimovich and A. H. G. Rinnooy Kan, Bounds and heuristics for capacitated routing problems, Mathematics of Operations Research 10(4), 1985. https://doi.org/10.1287/moor.10.4.527
  • P. Toth and D. Vigo (eds.), Vehicle Routing: Problems, Methods, and Applications, 2nd ed., SIAM, 2014. https://doi.org/10.1137/1.9781611973594
5 thms2 active usersReviewed
🏆Completed
Operations Research·Captain: naimengye

Fundamentals of Supply Chain Theory XI: The Traveling Salesman ProblemTextbook

The problem every routing model contains

A salesman must visit nnn cities and return home by the shortest route. The traveling salesman problem is the prototype of combinatorial optimization: easy to state, NP-hard (Karp 1972), and solved to optimality on instances with tens of thousands of nodes by branch-and-cut. In a supply chain it is the core of every vehicle routing model and of the location-routing models of the chapters that follow. Chapter 10 of Snyder and Shen's Fundamentals of Supply Chain Theory (2019) covers the symmetric metric TSP, in which distances satisfy the triangle inequality: the cutting planes of branch-and-cut (comb inequalities), the construction heuristics with their worst-case guarantees, culminating in Christofides' (1976) 3/2-approximation, and the lower bounds of Little et al. and of Held and Karp (1970). This mission formalizes those results, with Christofides' theorem as its goal.

Setting

Nodes are N={1,…,n}N = \{1, \dots, n\}N={1,…,n} with distances cijc_{ij}cij​ that are symmetric, nonnegative, zero on the diagonal and satisfy the triangle inequality cij≤cik+ckjc_{ij} \le c_{ik} + c_{kj}cij​≤cik​+ckj​ (IsMetric). A tour is a visiting order τ\tauτ of all the nodes, of length z(τ)=∑kc(τk,τk+1)z(\tau) = \sum_k c(\tau_k, \tau_{k+1})z(τ)=∑k​c(τk​,τk+1​) with indices mod nnn (tourLength); z∗z^*z∗ is the least tour length (optTourLength). A tour has an edge set (tourEdges), and for a node set SSS the counts of tour edges inside SSS and leaving SSS (edgesWithin, edgesLeaving) are the sums ∑i,j∈Sxij\sum_{i,j \in S} x_{ij}∑i,j∈S​xij​ and ∑i∈S,j∉Sxij\sum_{i \in S, j \notin S} x_{ij}∑i∈S,j∈/S​xij​ of the integer programming formulation. A comb is a handle HHH with an odd number s≥3s \ge 3s≥3 of pairwise disjoint teeth, each meeting both HHH and its complement (IsComb); when every tooth has exactly two nodes the comb is a 2-matching configuration.

The nearest neighbor heuristic always moves to a nearest unvisited node (IsNearestNeighborTour); the nearest insertion heuristic grows a partial tour by inserting the unvisited node nearest to it at the cheapest position (IsNearestInsertionRun, with cycleLength and distToTour). The minimum spanning tree heuristic doubles an MST (IsMST, graphWeight), takes an Eulerian tour of the doubled tree and shortcuts it, visiting the nodes in order of first appearance (IsShortcut). Christofides' heuristic instead adds to the MST a minimum-weight perfect matching on its odd-degree nodes (oddNodes, IsMinMatchingOn) before taking the Eulerian tour. A 1-tree rooted at rrr is a spanning tree on the other nodes plus two edges at rrr (Is1Tree); the revised distances cij′=cij+λi+λjc'_{ij} = c_{ij} + \lambda_i + \lambda_jcij′​=cij​+λi​+λj​ (revisedCost) define the Held-Karp bound.

Formalization targets

Goal: Theorem 10.13

For every metric instance, every minimum spanning tree T∗T^*T∗, every minimum-weight perfect matching MMM on the odd-degree nodes of T∗T^*T∗, every Eulerian tour of T∗+MT^* + MT∗+M and its shortcut τ\tauτ,

z(τ)  ≤  32 z∗.z(\tau) \;\le\; \tfrac{3}{2}\, z^*.z(τ)≤23​z∗.

This is christofides_bound.

Supporting targets

Theorem 10.1, the reduced-matrix bound ∑iρi+∑jκj≤z∗\sum_i \rho_i + \sum_j \kappa_j \le z^*∑i​ρi​+∑j​κj​≤z∗; Theorem 10.2, Proposition 10.3 and Theorem 10.4, the 2-matching and comb inequalities valid for every tour; Theorem 10.6, zNN≤12(⌈log⁡2n⌉+1)z∗z_{NN} \le \frac{1}{2}(\lceil\log_2 n\rceil + 1) z^*zNN​≤21​(⌈log2​n⌉+1)z∗; Theorem 10.7, zNI≤2z∗z_{NI} \le 2z^*zNI​≤2z∗; Lemma 10.9, z(T∗)≤z∗z(T^*) \le z^*z(T∗)≤z∗; Theorem 10.10, Euler's theorem; Theorem 10.11, zMST≤2z∗z_{MST} \le 2z^*zMST​≤2z∗; Lemma 10.12, the handshaking lemma; Lemma 10.15, the 1-tree bound; Lemma 10.16, the revised distance identities; Theorem 10.17, the Held-Karp bound. Theorem 10.5 (no constant-factor approximation unless P = NP), Theorem 10.8 and the second part of Theorem 10.6 (tightness instances), Lemma 10.14 (Euclidean tours do not cross), Lemma 10.18 (the integrality gap) and Theorem 10.19 (the Beardwood-Halton-Hammersley asymptotics) are not targets.

Significance

Christofides' bound was the best approximation guarantee for the metric TSP for over forty years, until the 3/2−10−363/2 - 10^{-36}3/2−10−36 of Karlin, Klein and Oveis Gharan (2021), and it is the reference point against which every heuristic in the chapter is measured: nearest neighbor has no constant bound, nearest insertion and the MST heuristic achieve 222, Christofides 3/23/23/2. The comb inequalities are the cuts that make branch-and-cut work, and the Held-Karp bound is the lower bound that tells a practitioner how far a heuristic tour is from optimal. Theorem 10.1 is the historical bounding rule of the first branch-and-bound algorithm.

None of these results has a machine-checked proof. The book proves Theorems 10.2, 10.11, 10.13 and 10.17 and Proposition 10.3 and Lemma 10.9, cites Theorems 10.6, 10.7 and 10.10, and leaves Theorem 10.4 and Lemmas 10.12 and 10.16 as exercises. The formal infrastructure for tours, shortcutting and Eulerian walks is reusable for the vehicle routing chapter.

Difficulty

Christofides' argument has three steps and each has a formal obstacle. The MST bound is a spanning-path argument that needs the removal of an edge from a tour to yield a tree, in Mathlib's terms a connected acyclic subgraph of the complete graph. The matching bound is the subtle one: the optimal tour shortcut to the odd-degree nodes, of length at most z∗z^*z∗ by the triangle inequality, is an even cycle whose alternate edges form two perfect matchings on those nodes, the cheaper of which costs at most z∗/2z^*/2z∗/2; formalizing the decomposition of a cycle on an even node set into two matchings, and the shortcut's length bound, is the bulk of the work. The final step, that shortcutting an Eulerian walk does not lengthen it, is an induction along the walk using the triangle inequality on the skipped stretches, and it needs the first-occurrence order to be handled explicitly.

The obvious approach to the heuristic bounds, comparing the heuristic tour directly with the optimal tour, fails; every proof goes through a spanning tree. For nearest insertion the tree is Prim's, grown in the same order as the insertions, and the bound charges each insertion cost to a tree edge; for nearest neighbor the argument of Rosenkrantz et al. bounds the sum of the kkk largest steps by 2z∗2z^*2z∗ for each kkk and sums a geometric series, which is where the logarithm comes from.

The comb inequalities are counting arguments on degrees, but the general comb of Theorem 10.4 needs the case analysis of how a tour enters and leaves each tooth. Euler's theorem in the sufficiency direction is Hierholzer's construction, which is not in Mathlib.

Formalization scope

Tours are permutations of Fin n, so a tour is an ordering rather than an edge set, and every tie-breaking of a heuristic is covered by a predicate on its output rather than by an algorithm. Graphs are Mathlib SimpleGraphs on Fin n; the multigraphs of the two tree heuristics are represented by closed walks with prescribed edge multisets, and shortcutting is the first-occurrence order along the walk's node sequence. All degree and edge-set computations use classical decidability. Theorems on tours assume n≥3n \ge 3n≥3 where a tour must have distinct edges, n≥1n \ge 1n≥1 otherwise.

Theorem 10.1 is stated for the reduction of the full off-diagonal matrix, because the book's upper-triangular version is false: a random metric instance violates it, since the last row and first column of a triangular matrix are empty and the two edges at a node need not be one row and one column entry. The full-matrix version is the statement of Little et al. It is stated for n≥2n \ge 2n≥2, because a one-node "tour" is a self-loop that no off-diagonal entry constrains.

Theorem 10.4 is stated with the comb inequality's right-hand side corrected to ∣H∣+∑k(∣Tk∣−1)−12(s+1)|H| + \sum_k(|T_k| - 1) - \tfrac{1}{2}(s+1)∣H∣+∑k​(∣Tk​∣−1)−21​(s+1), the standard form. The book prints +12(s−1)+\tfrac{1}{2}(s-1)+21​(s−1), which contradicts its own 2-matching special case (10.15) and is weaker by sss. The corrected statement implies the printed one.

The 111-tree root is an explicit node rrr, the book's node 111. The nearest insertion run is a sequence of lists indexed by iteration, and the theorem compares the nnn-th list's closed length with z∗z^*z∗; a run always exists, so the hypothesis is satisfiable.

The definition module is shared by all fifteen items. Theorem 10.8 and Problem 10.12 (tightness of the bounds of 222), the second part of Theorem 10.6, and Lemma 10.18 on the integrality gap are natural extensions on the same definitions.

Selected references

  • L. V. Snyder and Z.-J. M. Shen, Fundamentals of Supply Chain Theory, 2nd ed., Wiley, 2019, Chapter 10. https://doi.org/10.1002/9781119584445
  • N. Christofides, Worst-case analysis of a new heuristic for the travelling salesman problem, Report 388, GSIA, Carnegie Mellon University, 1976; reprinted in Operations Research Forum 3, 2022. https://doi.org/10.1007/s43069-021-00101-z
  • D. J. Rosenkrantz, R. E. Stearns and P. M. Lewis II, An analysis of several heuristics for the traveling salesman problem, SIAM Journal on Computing 6(3), 1977. https://doi.org/10.1137/0206041
  • M. Held and R. M. Karp, The traveling-salesman problem and minimum spanning trees, Operations Research 18(6), 1970. https://doi.org/10.1287/opre.18.6.1138
  • J. D. C. Little, K. G. Murty, D. W. Sweeney and C. Karel, An algorithm for the traveling salesman problem, Operations Research 11(6), 1963. https://doi.org/10.1287/opre.11.6.972
  • M. Grötschel and M. W. Padberg, On the symmetric travelling salesman problem I and II, Mathematical Programming 16, 1979. https://doi.org/10.1007/BF01582116
15 thms2 active usersReviewed
🏆Completed
Operations Research·Captain: naimengye

Complex Scheduling III: Interval Consistency Tests for the RCPSPTextbook

Motivation

Exact methods for the resource-constrained project scheduling problem — branch-and-bound over activity lists or over start-time assignments, and the lower-bound computations inside them — live or die by how much of the search space can be discarded before it is enumerated. The standard tool is constraint propagation: deducing, from the precedence, resource and time-window data, new precedence relations i→ji\to ji→j that every feasible schedule must satisfy, and tighter time windows for the activities. Brucker and Knust's Section 3.6 (doi:10.1007/978-3-642-23929-8) presents the family of interval consistency tests — input, output, input-or-output and their negations — that constraint-programming schedulers apply at every node of the search, following Carlier and Pinson (An algorithm for solving the job-shop problem, Management Science 35, 1989, doi:10.1287/mnsc.35.2.164) and Baptiste, Le Pape and Nuijten (Constraint-Based Scheduling, Kluwer, 2001, doi:10.1007/978-1-4615-1479-4). Every test is an instance of one theorem, Theorem 3.7, and its cumulative-resource analogue, Theorem 3.8. Those two theorems, and the tests as their corollaries, are this mission.

Setting

The instance is the RCPSP of mission I: activities 0,…,n−10,\dots,n-10,…,n−1 with integer processing times pip_ipi​, renewable resources kkk with capacities RkR_kRk​ and demands rikr_{ik}rik​, and precedence arcs; a schedule is an integer start-time vector SSS, feasible when it meets the precedences and never exceeds a capacity. Section 3.6 adds three things.

Relations. A conjunction i→ji\to ji→j holds in SSS when Si+pi≤SjS_i+p_i\le S_jSi​+pi​≤Sj​. Two activities are parallel, i∥ji\parallel ji∥j, when they overlap for at least one time unit, and a disjunction i−ji-ji−j is the negation of that: i→ji\to ji→j or j→ij\to ij→i. The instance carries a set CCC of conjunctions and a set DDD of disjunctions that every feasible schedule must satisfy; initially C0C_0C0​ is the precedence relation and D0D_0D0​ the pairs whose combined demand exceeds some capacity, and propagation adds to them.

Disjunctive sets. A set III of at least two activities is disjunctive when any two of its members are related by a disjunction or a conjunction, so no two are ever processed together: the activities of a unit-capacity resource, the jobs of a single machine, the operations of one job in a shop. Its total processing time is P(I)=∑i∈IpiP(I)=\sum_{i\in I}p_iP(I)=∑i∈I​pi​.

Time windows. Each activity has a head rir_iri​ and a deadline did_idi​, and a feasible schedule has ri≤Sir_i\le S_iri​≤Si​ and Si+pi≤diS_i+p_i\le d_iSi​+pi​≤di​. An activity starts first in a set JJJ when no activity of JJJ starts earlier, and ends last when none completes later.

For a cumulative resource kkk the work of activity iii is wi=rikpiw_i=r_{ik}p_iwi​=rik​pi​ and W(J)=∑i∈JwiW(J)=\sum_{i\in J}w_iW(J)=∑i∈J​wi​.

Formalization targets

Goal — Theorem 3.7 (printed p. 169)

Let III be a disjunctive set, J⊆IJ\subseteq IJ⊆I, and J′,J′′J',J''J′,J′′ proper subsets of JJJ with J′∪J′′≠∅J'\cup J''\ne\emptysetJ′∪J′′=∅. If

max⁡ν∈J∖J′, μ∈J∖J′′ν≠μ(dμ−rν)<P(J),\max_{\substack{\nu\in J\setminus J',\ \mu\in J\setminus J''\\ \nu\ne\mu}}\bigl(d_\mu-r_\nu\bigr)<P(J),ν∈J∖J′, μ∈J∖J′′ν=μ​max​(dμ​−rν​)<P(J),

then in every feasible schedule an activity from J′J'J′ starts first in JJJ or an activity from J′′J''J′′ ends last in JJJ.

The first infeasibility test (printed p. 169)

If some nonempty J⊆IJ\subseteq IJ⊆I has max⁡μ∈Jdμ−min⁡ν∈Jrν<P(J)\max_{\mu\in J}d_\mu-\min_{\nu\in J}r_\nu<P(J)maxμ∈J​dμ​−minν∈J​rν​<P(J), no feasible schedule exists.

The input test (3.123) and the output test (3.124) (printed p. 171)

For Ω⊆I\Omega\subseteq IΩ⊆I nonempty and i∈I∖Ωi\in I\setminus\Omegai∈I∖Ω: if max⁡μ∈Ω∪{i}dμ−min⁡ν∈Ωrν<P(Ω)+pi\max_{\mu\in\Omega\cup\{i\}}d_\mu-\min_{\nu\in\Omega}r_\nu<P(\Omega)+p_imaxμ∈Ω∪{i}​dμ​−minν∈Ω​rν​<P(Ω)+pi​ then i→ji\to ji→j for all j∈Ωj\in\Omegaj∈Ω; symmetrically, if max⁡μ∈Ωdμ−min⁡ν∈Ω∪{i}rν<P(Ω)+pi\max_{\mu\in\Omega}d_\mu-\min_{\nu\in\Omega\cup\{i\}}r_\nu<P(\Omega)+p_imaxμ∈Ω​dμ​−minν∈Ω∪{i}​rν​<P(Ω)+pi​ then j→ij\to ij→i for all j∈Ωj\in\Omegaj∈Ω.

The input-or-output test (printed p. 170)

For i,j∈J⊆Ii,j\in J\subseteq Ii,j∈J⊆I, ∣J∣≥2|J|\ge 2∣J∣≥2: if max⁡μ∈J∖{j}dμ−min⁡ν∈J∖{i}rν<P(J)\max_{\mu\in J\setminus\{j\}}d_\mu-\min_{\nu\in J\setminus\{i\}}r_\nu<P(J)maxμ∈J∖{j}​dμ​−minν∈J∖{i}​rν​<P(J) then iii starts first in JJJ or jjj ends last in JJJ, and i→ji\to ji→j when i≠ji\ne ji=j.

Theorem 3.8 (printed p. 186)

For a cumulative resource kkk, J⊆IkJ\subseteq I_kJ⊆Ik​ and proper subsets J′,J′′J',J''J′,J′′ of JJJ: if Rk(max⁡μ∈J∖J′′dμ−min⁡ν∈J∖J′rν)<W(J)R_k\bigl(\max_{\mu\in J\setminus J''}d_\mu-\min_{\nu\in J\setminus J'}r_\nu\bigr)<W(J)Rk​(maxμ∈J∖J′′​dμ​−minν∈J∖J′​rν​)<W(J) then an activity from J′J'J′ starts first in JJJ or an activity from J′′J''J′′ ends last in JJJ.

Significance

Theorem 3.7 is the single statement behind a whole toolbox. Every interval consistency test in the literature — the input and output tests that fix a new conjunction, the input-or-output test, the negation tests that only shrink a window — is the theorem with a particular choice of J′J'J′ and J′′J''J′′, and the book's Section 3.6.4 derives them one by one. A propagation engine that applies these tests to a fixpoint is what makes branch-and-bound for the job shop and the RCPSP practical; Carlier and Pinson's solution of the 10×10 job-shop instance is the historical demonstration. Theorem 3.8 extends the same reasoning from disjunctive to cumulative resources by replacing "no overlap" with "at most RkR_kRk​ units per time unit", the energetic-reasoning viewpoint that the rest of Section 3.6.5 develops.

The results are elementary and proved; formalizing them fixes, once, what "feasible" means in the presence of the relation sets CCC and DDD and time windows, on top of the RCPSP model of mission I. That layer is reusable: the start-start distance matrix of Section 3.6.2 and the symmetric-triple rules of Section 3.6.3 are statements about the same schedules and the same relations. Nothing here is on the platform or in Mathlib.

Difficulty

The obvious argument for Theorem 3.7 is the correct one, and its difficulty is in the bookkeeping. If no activity of J′J'J′ starts first and none of J′′J''J′′ ends last, the first starter is some ν∈J∖J′\nu\in J\setminus J'ν∈J∖J′ and the last finisher some μ∈J∖J′′\mu\in J\setminus J''μ∈J∖J′′, and every activity of JJJ is processed inside [Sν, Sμ+pμ]⊆[rν,dμ][S_\nu,\,S_\mu+p_\mu]\subseteq[r_\nu,d_\mu][Sν​,Sμ​+pμ​]⊆[rν​,dμ​]. Since the activities of a disjunctive set are pairwise non-overlapping, their total length P(J)P(J)P(J) fits in that interval, contradicting (3.121). The formal work is the packing lemma: pairwise disjoint integer intervals inside an interval of length LLL have total length at most LLL, which requires ordering the activities by start time and an induction that Mathlib does not supply.

The subtle point is the restriction ν≠μ\nu\ne\muν=μ in (3.121). The book allows it because "an activity which starts first cannot complete also last" when there are at least two activities with positive durations. The formal statement reads "starts first" and "ends last" with ≤\le≤, which makes the theorem true without a positivity hypothesis: when the restriction empties the index set, J∖J′=J∖J′′={x}J\setminus J'=J\setminus J''=\{x\}J∖J′=J∖J′′={x}, the conclusion holds because xxx cannot be both the unique first starter and the unique last finisher of a disjunctive set with two or more members. A solver should expect to handle that corner separately.

For the tests the extra step is turning "starts first" into a conjunction i→ji\to ji→j, which uses the disjunction between iii and jjj together with positive processing times: with pj=0p_j=0pj​=0 an activity could start at the same instant as iii without violating the disjunction, so the tests carry the positivity hypothesis that Theorem 3.7 itself does not need. Theorem 3.8 replaces the packing lemma by a work-counting lemma: over an interval of length LLL a resource of capacity RkR_kRk​ supplies at most RkLR_kLRk​L units, and every activity of JJJ consumes rikpir_{ik}p_irik​pi​ of them.

Formalization scope

Schedules are integer start-time vectors on Fin n, as in missions I and II, and a feasible schedule of this mission is one that is FeasibleSchedule for the RCPSP instance (mission I), respects the arcs of CCC (RespectsArcs, mission II), satisfies the disjunctions of DDD and lies within the time windows; these four hypotheses are the book's "feasible schedule" in Section 3.6 and are carried on every statement, although the arguments use only the last two, or, for Theorem 3.8, the resource constraint and the windows. The sets CCC and DDD are parameters, not derived from the instance, since propagation enlarges them.

Every inequality "max⁡(⋅)−min⁡(⋅)<P\max(\cdot)-\min(\cdot)<Pmax(⋅)−min(⋅)<P" is stated as the family of inequalities dμ<rν+Pd_\mu<r_\nu+Pdμ​<rν​+P over the same index pairs. This is equivalent, avoids natural-number subtraction, and gives an empty index set the value the convention max⁡∅=−∞\max\emptyset=-\inftymax∅=−∞ would: the hypothesis is then vacuous. "Starts first" and "ends last" use ≤\le≤. Proper-subset hypotheses J′⊂JJ'\subset JJ′⊂J, J′′⊂JJ''\subset JJ′′⊂J are the book's; the first infeasibility test needs JJJ nonempty, and the input-or-output test needs ∣J∣≥2|J|\ge 2∣J∣≥2.

A trivializing reading is ruled out on the disjunctive side by the nonemptiness hypotheses (an empty JJJ would make the infeasibility test's family vacuous and its conclusion false) and on the cumulative side by the observation that Theorem 3.8 with J′=J′′=∅J'=J''=\emptysetJ′=J′′=∅ asserts infeasibility, which is the book's intended reading. Welcome contributions beyond the milestones: the input-negation and output-negation tests, the window-tightening rules of Section 3.6.4, and the SSD-matrix results of Section 3.6.2.

Selected references

  • Peter Brucker and Sigrid Knust, Complex Scheduling, 2nd ed., Springer, 2012, Section 3.6. doi:10.1007/978-3-642-23929-8
  • Jacques Carlier and Eric Pinson, An algorithm for solving the job-shop problem, Management Science 35 (1989). doi:10.1287/mnsc.35.2.164
  • Philippe Baptiste, Claude Le Pape and Wim Nuijten, Constraint-Based Scheduling, Kluwer, 2001. doi:10.1007/978-1-4615-1479-4
  • Ulrich Dorndorf, Erwin Pesch and Toàn Phan-Huy, Constraint propagation techniques for the disjunctive scheduling problem, Artificial Intelligence 122 (2000). doi:10.1016/S0004-3702(00)00040-0
9 thms2 active usersReviewed
🏆Completed
Operations Research·Captain: naimengye

Scheduling Algorithms V: Preemptive Scheduling on Uniform MachinesTextbook

Motivation

When several processors share a workload, the first question is how long the workload takes if it is spread out as well as possible. If a job may be interrupted and resumed later, possibly on another processor — preemption, the setting of operating systems, of communication links and of any resource that can be time-shared — the answer is a closed formula, and it is one of the oldest results in scheduling: McNaughton's wrap-around rule of 1959 (Scheduling with deadlines and loss functions, Management Science 6, doi:10.1287/mnsc.6.1.1) shows that on identical machines the optimal makespan is the larger of the longest job and the average load. Horvath, Lam and Sethi (A level algorithm for preemptive scheduling, Journal of the ACM 24, 1977, doi:10.1145/321992.321995) extended this to machines of different speeds, and Gonzalez and Sahni (Preemptive scheduling of uniform processor systems, Journal of the ACM 25, 1978, doi:10.1145/322047.322055) gave the fast algorithm with few preemptions. Brucker's Chapter 5 (doi:10.1007/978-3-540-69516-5) presents the level-algorithm version, and this mission formalizes the statements it proves about schedules.

Setting

There are nnn jobs with processing requirements p1,…,pn>0p_1,\dots,p_n>0p1​,…,pn​>0 and mmm uniform machines with speeds s1,…,sm>0s_1,\dots,s_m>0s1​,…,sm​>0: running job iii on machine jjj for a period of length ℓ\ellℓ performs sjℓs_j\ellsj​ℓ units of its requirement, so the whole job would take pi/sjp_i/s_jpi​/sj​ time units there. Identical machines are the case s1=⋯=sm=1s_1=\dots=s_m=1s1​=⋯=sm​=1.

A preemptive schedule is a finite list of pieces, each a job, a machine, a start time and a stop time. It is feasible for the data (s,p)(s,p)(s,p) when every piece lies in [0,∞)[0,\infty)[0,∞), no two pieces on the same machine overlap, no two pieces of the same job overlap (a job is on at most one machine at any instant), and every job iii receives total work exactly pip_ipi​ over its pieces. Its makespan Cmax⁡C_{\max}Cmax​ is the largest stop time; the completion time CiC_iCi​ of job iii is the largest stop time of one of its pieces. A schedule is nonpreemptive when every job consists of a single piece.

Following Section 5.1.2 the data are sorted, p1≥⋯≥pnp_1\ge\dots\ge p_np1​≥⋯≥pn​ and s1≥⋯≥sms_1\ge\dots\ge s_ms1​≥⋯≥sm​, with n≥mn\ge mn≥m, and one writes Pj=∑i≤jpiP_j=\sum_{i\le j}p_iPj​=∑i≤j​pi​, Sj=∑i≤jsiS_j=\sum_{i\le j}s_iSj​=∑i≤j​si​. For a set AAA of jobs, h(A)=S∣A∣h(A)=S_{|A|}h(A)=S∣A∣​ if ∣A∣≤m|A|\le m∣A∣≤m and h(A)=Smh(A)=S_mh(A)=Sm​ otherwise: the largest combined speed that ∣A∣|A|∣A∣ jobs can use at one instant.

Formalization targets

Goal — Theorem 5.8 (printed p. 127)

The optimal makespan of Q∣pmtn∣Cmax⁡Q\mid pmtn\mid C_{\max}Q∣pmtn∣Cmax​ is the bound (5.5):

w  =  max⁡{max⁡j=1m−1PjSj, PnSm},w \;=\; \max\Bigl\{\max_{j=1}^{m-1}\frac{P_j}{S_j},\ \frac{P_n}{S_m}\Bigr\} ,w=max{j=1maxm−1​Sj​Pj​​, Sm​Pn​​},

in the sense that some feasible preemptive schedule has makespan exactly www and no feasible preemptive schedule has a smaller makespan.

The lower bound (5.5) (printed p. 125)

Every feasible preemptive schedule has makespan at least www.

P∣pmtn∣Cmax⁡P\mid pmtn\mid C_{\max}P∣pmtn∣Cmax​ (printed p. 108)

On identical machines, LB=max⁡{max⁡ipi, 1m∑ipi}LB=\max\{\max_i p_i,\ \tfrac1m\sum_i p_i\}LB=max{maxi​pi​, m1​∑i​pi​} is a lower bound on the makespan and is attained by some feasible preemptive schedule.

Condition (5.8) (printed p. 129)

The jobs can be scheduled preemptively within [0,T][0,T][0,T] if and only if ∑i∈Api≤T h(A)\sum_{i\in A}p_i\le T\,h(A)∑i∈A​pi​≤Th(A) for every set AAA of jobs.

Theorem 5.7 (printed p. 121)

For P∣pmtn∣∑wiCiP\mid pmtn\mid\sum w_iC_iP∣pmtn∣∑wi​Ci​ with nonnegative weights there is an optimal schedule without preemption.

Significance

Theorem 5.8 turns an optimization over an infinite family of schedules into a formula in the data, and the formula is tight in both directions: each of its terms is a resource bound that some schedule meets exactly. That is what makes preemptive makespan minimization one of the few parallel-machine problems that is solvable at all — its nonpreemptive counterpart P2∥Cmax⁡P2\parallel C_{\max}P2∥Cmax​ is NP-hard (p. 124) — and it is why the preemptive relaxation appears as a bound inside branch-and-bound methods for the nonpreemptive problems.

Condition (5.8) is the form in which the result is reused. It is a Hall-type condition, one inequality per set of jobs, and it is exactly what Section 5.1.2 needs to prove Theorem 5.9, which decides Q∣pmtn;ri∣Lmax⁡Q\mid pmtn; r_i\mid L_{\max}Q∣pmtn;ri​∣Lmax​ by a maximum flow in an expanded network. Theorem 5.7 is the complementary statement for the other classical objective: for total weighted completion time preemption buys nothing, so the nonpreemptive solutions of Section 5.1.1 are optimal in the larger class too.

On status: every statement here is classical and proved, and the formalization adds a checked model of preemptive schedules. Mathlib has no scheduling material, and the platform's SchedulingAlgorithms series so far models only single-machine sequences (missions I, II, IV) and two-machine permutation flow shops (mission III), none of which allow a job to be split. The piece-list model of this mission is the first reusable object for preemptive and parallel-machine problems, and the later sections of Chapter 5 — Q∣pmtn;ri∣Lmax⁡Q\mid pmtn; r_i\mid L_{\max}Q∣pmtn;ri​∣Lmax​, P∣pmtn∣Lmax⁡P\mid pmtn\mid L_{\max}P∣pmtn∣Lmax​ — are stated in it.

Difficulty

The obvious first idea for the goal is to run McNaughton's rule with the speeds ignored. It fails on uniform machines: filling machines one after another does not respect the constraint that a long job on a slow machine is not done when a short job on a fast one is. The correct idea is the level algorithm — always process the jobs of highest remaining requirement on the fastest free machines, sharing machines among tied jobs — and the difficulty is in the analysis rather than the idea. The proof of Theorem 5.8 has to show that the schedule it produces ends exactly at one of the terms of www: either no machine idles before the end, giving Pn/SmP_n/S_mPn​/Sm​, or the machines finish in speed order with the first jjj jobs busy from time 000, giving Pj/SjP_j/S_jPj​/Sj​. Making that case analysis rigorous requires tracking that the order of remaining requirements is preserved over time (the invariant (5.6)) and that ties are broken consistently.

A second, formal difficulty is that the level algorithm's output is defined by continuous-time events (the next completion, the next time two levels coincide), so producing an explicit finite list of pieces with the required properties is itself a construction. Any proof must build a concrete schedule; "the infimum of makespans equals www" is not the goal.

For the lower bound the trap is the opposite: it is tempting to argue only with total capacity SmTS_mTSm​T, which gives Pn/SmP_n/S_mPn​/Sm​ but not Pj/SjP_j/S_jPj​/Sj​. The latter needs the rule that a job is on at most one machine at a time, so that jjj jobs run at combined speed at most SjS_jSj​; a model that let a job be split across machines simultaneously would make the theorem false, and the definition of feasibility rules it out explicitly.

Formalization scope

A schedule is a List of Pieces over jobs Fin n and machines Fin m, with real start and stop times. Feasibility is the three-part condition of the Setting, disjointness of two pieces meaning one stops no later than the other starts. Work is measured with the machine's speed, so the same definitions cover identical machines as the constant speed 111. Pieces of length zero and unsorted lists are allowed; both are harmless.

The sorted orders are hypotheses Antitone p and Antitone s, the speeds and requirements are positive, m≥1m\ge 1m≥1 and, where the book assumes it, n≥mn\ge mn≥m. The book's normalization s1=1s_1=1s1​=1 is not assumed: every statement here is invariant under scaling all speeds, and the book uses the normalization only for a running-time estimate. The bound www is defined as the maximum of an explicit nonempty finite set, so no supremum of an empty or unbounded set occurs; LBLBLB takes a proof that n≥1n\ge 1n≥1 so that max⁡ipi\max_i p_imaxi​pi​ is meaningful.

Two things are deliberately not stated. The level algorithm itself is not transcribed: Theorem 5.8 is stated as the existence of an optimal schedule of makespan www, which is what its proof establishes. And Theorem 5.9, the flow characterization for Q∣pmtn;ri∣Lmax⁡Q\mid pmtn; r_i\mid L_{\max}Q∣pmtn;ri​∣Lmax​, is left for a later mission, since it needs release times and the expanded network on top of this model.

A trivializing reading is excluded by the existential form of the goal and of Theorem 5.7: each asserts that an optimal schedule exists, not merely that any optimal schedule has a property. A proof of the goal has to construct a schedule; a proof of Theorem 5.7 has to construct a nonpreemptive one that beats every preemptive competitor. Contributions welcome beyond the milestones: a general lemma that a feasible schedule can be normalized to sorted, positive-length pieces, and a proof that (5.8) for the sets {1,…,j}\{1,\dots,j\}{1,…,j} is equivalent to w≤Tw\le Tw≤T.

Selected references

  • Peter Brucker, Scheduling Algorithms, 5th ed., Springer, 2007, Chapter 5. doi:10.1007/978-3-540-69516-5
  • Robert McNaughton, Scheduling with deadlines and loss functions, Management Science 6 (1959). doi:10.1287/mnsc.6.1.1
  • E. C. Horvath, S. Lam and R. Sethi, A level algorithm for preemptive scheduling, Journal of the ACM 24 (1977). doi:10.1145/321992.321995
  • Teofilo Gonzalez and Sartaj Sahni, Preemptive scheduling of uniform processor systems, Journal of the ACM 25 (1978). doi:10.1145/322047.322055
6 thms2 active usersReviewed
🏆Completed
Mathematical Physics·Captain: Lucas

NRL Plasma Formulary I: Rothe–Hagen IdentityTextbook

Motivation

The NRL Plasma Formulary is a standard desk reference of the plasma-physics community: a compilation of the formulas, constants and unit conversions used in daily practice. Its opening section, "Numerical and Algebraic" (p. 3 of the 2013 edition), collects the few purely mathematical identities the rest of the handbook leans on. Two of them are exact summation formulas rather than approximations, and the first is the Rothe–Hagen identity, quoted there as valid "for all complex xxx, yyy, zzz except when singular" and attributed to H. W. Gould's work on binomial coefficient summations.

Unlike the handbook's numerical entries, this identity is a theorem with a precise hypothesis set, and it is exactly the kind of entry a reader takes on trust. It generalizes the Vandermonde convolution, it is the coefficient identity underlying the generalized binomial series, and it specializes to Abel's binomial theorem. Formalizing it turns one line of a reference handbook into a machine-checked statement and produces, as a by-product, a reusable Lean development of binomial coefficients with an arbitrary complex upper index.

Setting

For a complex number www and a natural number kkk, the generalized binomial coefficient is the falling factorial divided by a factorial,

(wk)  =  w(w−1)⋯(w−k+1)k!  =  1k!∏j=0k−1(w−j),\binom{w}{k} \;=\; \frac{w(w-1)\cdots(w-k+1)}{k!} \;=\; \frac{1}{k!}\prod_{j=0}^{k-1}(w-j),(kw​)=k!w(w−1)⋯(w−k+1)​=k!1​j=0∏k−1​(w−j),

with the empty-product convention (w0)=1\binom{w}{0} = 1(0w​)=1. It is a polynomial in www of degree kkk, and it agrees with the usual binomial coefficient when www is a natural number.

Fix complex parameters xxx, yyy, zzz and, for k∈Nk \in \mathbb{N}k∈N, consider the Rothe factor

Ak(x,z)  =  xx+kz(x+kzk),A_k(x,z) \;=\; \frac{x}{x+kz}\binom{x+kz}{k},Ak​(x,z)=x+kzx​(kx+kz​),

which is defined whenever x+kz≠0x + kz \neq 0x+kz=0. The factor x+kzx+kzx+kz in the denominator cancels against the leading factor of the falling factorial, so Ak(x,z)A_k(x,z)Ak​(x,z) extends to a polynomial in xxx and zzz: A0(x,z)=1A_0(x,z) = 1A0​(x,z)=1 and, for k≥1k \ge 1k≥1,

Ak(x,z)  =  x (x+kz−1)(x+kz−2)⋯(x+kz−k+1)k!.A_k(x,z) \;=\; \frac{x\,(x+kz-1)(x+kz-2)\cdots(x+kz-k+1)}{k!}.Ak​(x,z)=k!x(x+kz−1)(x+kz−2)⋯(x+kz−k+1)​.

Both forms occur in the literature; the mission carries both and asks for the comparison between them, because the quotient form is the one printed in the handbook while the polynomial form is the one that carries no side condition.

Formalization targets

Goal — the Rothe–Hagen identity, as printed

For complex x,y,zx,y,zx,y,z and n∈Nn \in \mathbb{N}n∈N, provided x+kz≠0x+kz \neq 0x+kz=0 and y+kz≠0y+kz \neq 0y+kz=0 for every 0≤k≤n0 \le k \le n0≤k≤n, and x+y+nz≠0x+y+nz \neq 0x+y+nz=0,

∑k=0nxx+kz(x+kzk)  yy+(n−k)z(y+(n−k)zn−k)  =  x+yx+y+nz(x+y+nzn).\sum_{k=0}^{n} \frac{x}{x+kz}\binom{x+kz}{k}\;\frac{y}{y+(n-k)z}\binom{y+(n-k)z}{n-k} \;=\; \frac{x+y}{x+y+nz}\binom{x+y+nz}{n}.k=0∑n​x+kzx​(kx+kz​)y+(n−k)zy​(n−ky+(n−k)z​)=x+y+nzx+y​(nx+y+nz​).

This is the handbook's line, with its "except when singular" proviso made explicit as the three non-vanishing hypotheses.

Stronger — the identity with no side condition

∑k=0nAk(x,z) An−k(y,z)  =  An(x+y,z),\sum_{k=0}^{n} A_k(x,z)\,A_{n-k}(y,z) \;=\; A_n(x+y,z),k=0∑n​Ak​(x,z)An−k​(y,z)=An​(x+y,z),

in terms of the polynomial form AkA_kAk​ above. This version holds for all complex x,y,zx,y,zx,y,z, with no exceptional locus, and implies the printed form wherever the latter's denominators are non-zero.

Supporting targets

(w+1k+1)=(wk)+(wk+1),∑k=0n(xk)(yn−k)=(x+yn).\binom{w+1}{k+1} = \binom{w}{k} + \binom{w}{k+1}, \qquad \sum_{k=0}^{n}\binom{x}{k}\binom{y}{n-k} = \binom{x+y}{n}.(k+1w+1​)=(kw​)+(k+1w​),k=0∑n​(kx​)(n−ky​)=(nx+y​).

The first is Pascal's rule for a complex upper index; the second is the Vandermonde convolution over C\mathbb{C}C, which is the case z=0z = 0z=0 of the goal.

Significance

The identity is the convolution law of the generalized binomial series: the formal power series Bz(t)\mathcal{B}_z(t)Bz​(t) solving B=1+t Bz\mathcal{B} = 1 + t\,\mathcal{B}^{z}B=1+tBz satisfies Bz(t)x=∑n≥0An(x,z) tn\mathcal{B}_z(t)^x = \sum_{n \ge 0} A_n(x,z)\,t^nBz​(t)x=∑n≥0​An​(x,z)tn, so the goal is the statement Bzx⋅Bzy=Bzx+y\mathcal{B}_z^x \cdot \mathcal{B}_z^y = \mathcal{B}_z^{x+y}Bzx​⋅Bzy​=Bzx+y​ read off coefficientwise (Graham–Knuth–Patashnik, Concrete Mathematics, §5.4). Consequences include Abel's binomial theorem, the Lagrange-inversion count of zzz-ary trees, and ballot-type identities in lattice-path enumeration.

Mathlib provides the Vandermonde convolution for natural-number arguments (Nat.add_choose_eq), the ascending and descending Pochhammer polynomials, and Ring.choose for binomial rings; it does not contain the Rothe–Hagen identity in any form. The four statements of this mission are therefore new formal content, and the complex-upper-index binomial API they force is reusable well beyond the mission.

The identity is classical and has been proved many times since Rothe (1793) and Hagen (1891), with the modern treatment in Gould's papers; nothing here is open mathematics. What is missing is a machine-checked proof.

Difficulty

The obvious attack — induction on nnn using Pascal's rule — does not close as stated: the summand Ak(x,z)A_k(x,z)Ak​(x,z) is not Pascal-stable, since shifting xxx by 111 moves x+kzx+kzx+kz for every kkk at once, and the induction hypothesis is about a different family. The standard proofs instead treat both sides as polynomials in xxx and yyy for fixed zzz and nnn, verify the identity on an infinite set of points where a combinatorial reading is available, and conclude by the identity theorem for polynomials; or they extract coefficients from the generalized binomial series via Lagrange inversion. Either route needs infrastructure: a two-variable polynomial-identity argument over C\mathbb{C}C, or a formal-power-series compositional inverse.

A second, more prosaic difficulty is the singular locus. The printed identity divides by x+kzx+kzx+kz for every k≤nk \le nk≤n, and in Lean division by zero returns zero rather than failing, so a formalization that drops the non-vanishing hypotheses states a different — and in general false — claim.

Formalization scope

All statements are over C\mathbb{C}C. The generalized binomial coefficient is defined as an explicit product over Finset.range k divided by (k ! : ℂ), so (w0)=1\binom{w}{0} = 1(0w​)=1 holds definitionally and no Nat.choose coercion enters. Sums run over Finset.range (n+1), with the complementary index written as the truncated natural subtraction n - k; inside that range this is the ordinary n−kn-kn−k, so the truncation convention is never exercised.

The proviso "except when singular" is formalized as three explicit hypotheses — x+kz≠0x+kz \ne 0x+kz=0 for all k≤nk \le nk≤n, y+kz≠0y+kz \ne 0y+kz=0 for all k≤nk \le nk≤n, and x+y+nz≠0x+y+nz \ne 0x+y+nz=0 — rather than by relying on Lean's junk value for division by zero. These hypotheses are satisfiable (for instance z=0z = 0z=0, x=y=1x = y = 1x=y=1), so the goal is not vacuous, and they constrain only denominators, so they do not trivialize the sum.

The singularity-free milestone commits to a total function rotheA defined by cases on kkk, with value 111 at k=0k = 0k=0; the comparison milestone pins that function to the quotient form under the non-vanishing hypothesis, so the mission cannot be satisfied by proving facts about a differently normalized object.

Contributions welcome beyond the milestones: complex-upper-index binomial API (symmetry, negation (−wk)=(−1)k(w+k−1k)\binom{-w}{k} = (-1)^k\binom{w+k-1}{k}(k−w​)=(−1)k(kw+k−1​), polynomiality in the upper index), and any formal-power-series development supporting Lagrange inversion.

Selected references

  • J. D. Huba, NRL Plasma Formulary, Naval Research Laboratory, 2013, p. 3, "Numerical and Algebraic" (Rothe–Hagen identity). https://www.nrl.navy.mil/News-Media/Publications/NRL-Plasma-Formulary/
  • H. W. Gould, "Note on Some Binomial Coefficient Identities of Rosenbaum", Journal of Mathematical Physics 10, 49 (1969). https://doi.org/10.1063/1.1664760
  • H. W. Gould and J. Kaucky, "Evaluation of a Class of Binomial Coefficient Summations", Journal of Combinatorial Theory 1, 233–247 (1966). https://doi.org/10.1016/S0021-9800(66)80051-9
  • R. L. Graham, D. E. Knuth, O. Patashnik, Concrete Mathematics, 2nd ed., Addison-Wesley, 1994, §5.4 (generalized binomial series).
7 thms2 active usersReviewed
🏆Completed
Mathematical Physics·Captain: Lucas

Nuclear Shell Model I: Harmonic-Oscillator Magic NumbersTextbook

Motivation

A nucleus whose proton number or neutron number equals one of 2,8,20,28,50,82,1262, 8, 20, 28, 50, 82, 1262,8,20,28,50,82,126 — the magic numbers — is markedly more tightly bound than the semi-empirical mass formula predicts, and markedly more stable against decay. The empirical evidence for closed nuclear shells at 505050 and 828282 protons and at 50,82,12650, 82, 12650,82,126 neutrons was assembled by Maria Goeppert Mayer in 1948 (Phys. Rev. 74, 235), and explained in 1949 by adding spin–orbit coupling to the shell picture (Phys. Rev. 75, 1969); the shell model that grew out of this work earned Goeppert Mayer and Hans Jensen a share of the 1963 Nobel Prize in Physics.

The textbook first step toward these numbers is purely combinatorial: approximate the nuclear force by an isotropic three-dimensional quantum harmonic oscillator, count the degenerate states at each energy level, double the count for spin, and accumulate. This mission formalizes exactly that step — including its well-known failure — from the Derivation section of the Wikipedia article Magic number (physics).

Setting

For the isotropic three-dimensional harmonic oscillator the energy depends only on the sum n=nx+ny+nzn = n_x + n_y + n_zn=nx​+ny​+nz​ of the three quantum numbers, so the states at level nnn are the triples of nonnegative integers summing to nnn — the weak 333-compositions of nnn. Write

Ln  =  {(nx,ny,nz)∈N3  :  nx+ny+nz=n}.L_n \;=\; \{(n_x,n_y,n_z)\in\mathbb{N}^3 \;:\; n_x+n_y+n_z=n\}.Ln​={(nx​,ny​,nz​)∈N3:nx​+ny​+nz​=n}.

Each state accommodates two nucleons of the same kind (two spin orientations), so the shell at level nnn holds

Cn  =  2 ∣Ln∣C_n \;=\; 2\,|L_n|Cn​=2∣Ln​∣

nucleons, and filling every shell of level below NNN takes

SN  =  ∑n<NCnS_N \;=\; \sum_{n<N} C_nSN​=n<N∑​Cn​

nucleons. A shell closure — hence a predicted magic number — is a value SNS_NSN​. Finally, M={2,8,20,28,50,82,126}M = \{2, 8, 20, 28, 50, 82, 126\}M={2,8,20,28,50,82,126} denotes the seven most widely recognized magic numbers.

Target

The mission's goal collects the content of the derivation:

∣Ln∣  =  (n+22),SN  =  2(N+23)  =  N(N+1)(N+2)3,|L_n| \;=\; \binom{n+2}{2}, \qquad S_N \;=\; 2\binom{N+2}{3} \;=\; \frac{N(N+1)(N+2)}{3},∣Ln​∣=(2n+2​),SN​=2(3N+2​)=3N(N+1)(N+2)​, (S1,S2,S3,S4)  =  (2, 8, 20, 40),(S_1,S_2,S_3,S_4) \;=\; (2,\,8,\,20,\,40),(S1​,S2​,S3​,S4​)=(2,8,20,40),

together with the comparison against MMM: the values 2,8,202, 8, 202,8,20 lie in MMM; the magic number 28∈M28 \in M28∈M is never of the form SNS_NSN​; and 40=S440 = S_440=S4​ is produced by the count although 40∉M40 \notin M40∈/M.

Significance

The positive half of the statement explains three of the seven magic numbers from nothing but degeneracy counting: 222, 888 and 202020 are exactly the first three oscillator shell closures. The negative half is the reason the shell model needs more physics: the oscillator count misses 282828 altogether and predicts 404040, which is not magic. Historically this gap is what spin–orbit coupling was introduced to close, and it is the precise sense in which "the correct formula for predicted magic numbers fundamentally relies on spin-orbit coupling".

As a formalization target the mission is small and self-contained: it fixes a reusable Lean model of oscillator shell filling (levels, degeneracies, spin doubling, cumulative capacity) and the stars-and-bars count behind it, against which later, more physical refinements — spin–orbit corrections, the empirical magic-number list — can be stated.

Difficulty

The mathematics is elementary; the work is in the counting and in the negative claims. The degeneracy ∣Ln∣=(n+22)|L_n| = \binom{n+2}{2}∣Ln​∣=(2n+2​) is stars and bars for three boxes, most directly obtained by fibering LnL_nLn​ over the first coordinate and summing ∑x≤n(n−x+1)\sum_{x\le n}(n-x+1)∑x≤n​(n−x+1). Asserting that no NNN gives SN=28S_N = 28SN​=28 is not a finite check: it needs the closed form together with monotonicity of N↦(N+23)N \mapsto \binom{N+2}{3}N↦(3N+2​) to rule out every NNN above the range where SNS_NSN​ is computed, since S3=20<28<40=S4S_3 = 20 < 28 < 40 = S_4S3​=20<28<40=S4​.

Formalization scope

Everything is over N\mathbb{N}N and finite. The level set LnL_nLn​ is a Finset (ℕ × ℕ × ℕ) cut out of the box [0,n]3[0,n]^3[0,n]3 by the equation nx+ny+nz=nn_x + n_y + n_z = nnx​+ny​+nz​=n, so its elements are ordered triples: permutations of the same multiset are distinct states, as the physics requires. Spin enters only as the literal factor 222 in CnC_nCn​, not as an extra coordinate. The cumulative count SNS_NSN​ runs over levels n<Nn < Nn<N, so S0=0S_0 = 0S0​=0 and S1=2S_1 = 2S1​=2; the NNN-th predicted magic number is SNS_NSN​, not SN−1S_{N-1}SN−1​. Binomial coefficients are Nat.choose, and the closed forms are stated in the division-free shape 2(N+23)2\binom{N+2}{3}2(3N+2​) to avoid truncated natural-number division. The set MMM of magic numbers is a hard-coded Finset ℕ transcribing the seven numbers listed by the source — it is a datum, not a derived object, and no claim in this mission asserts anything about nuclear stability itself.

The statements about 282828 and 404040 are what keep the mission from being a pure identity check: ∀N, SN≠28\forall N,\ S_N \neq 28∀N, SN​=28 quantifies over all NNN, and 40∉M40 \notin M40∈/M is a membership fact about the transcribed list.

Selected references

  • M. Goeppert Mayer, On Closed Shells in Nuclei, Physical Review 74 (1948) 235–239, doi:10.1103/PhysRev.74.235.
  • M. Goeppert Mayer, On Closed Shells in Nuclei. II, Physical Review 75 (1949) 1969–1970, doi:10.1103/PhysRev.75.1969.
  • Magic number (physics), Wikipedia, section Derivation, link.
  • N. J. A. Sloane (ed.), sequence A018226 (magic numbers of nucleons), The On-Line Encyclopedia of Integer Sequences.
7 thms2 active usersReviewed
🏆Completed
Discrete Geometry·Captain: mikedeng1

Convex Polytopes II: Finite convex representationsTextbook

Finite representations in convex geometry

A convex combination expresses a point as a weighted average of other points. Such representations connect geometric sets with finite lists of coordinates and coefficients. A point in a convex hull may initially be described using many generators, even when the ambient space has small dimension. Carathéodory's theorem gives a bound depending only on that dimension. Grünbaum presents this result as a basic theorem of convexity in Convex Polytopes, §2.3, Theorem 5, printed page 15 (the supplied source PDF, page 33).

Sets, points, and weights

Fix a natural number ddd. Real ddd-space consists of vectors with ddd real coordinates. Let AAA be any subset of this space. A convex combination of points of AAA is a finite sum of those points multiplied by nonnegative real coefficients, with the coefficients adding to one. The convex hull conv⁡A\operatorname{conv} AconvA is the set of all such combinations; equivalently, it is the smallest convex set containing AAA.

The generating set AAA can be finite or infinite. It need not be closed, bounded, or convex. Membership of a point xxx in its convex hull is the sole geometric assumption. The theorem concerns exact equality of vectors, so the representation is not an approximation or a limiting expression.

A dimension bound for every hull point

For every x∈conv⁡Ax\in\operatorname{conv} Ax∈convA, find points v0,…,vd∈Av_0,\ldots,v_d\in Av0​,…,vd​∈A and real numbers w0,…,wdw_0,\ldots,w_dw0​,…,wd​ satisfying

wi≥0(0≤i≤d),∑i=0dwi=1,x=∑i=0dwivi.w_i\geq 0\quad(0\leq i\leq d),\qquad \sum_{i=0}^{d}w_i=1,\qquad x=\sum_{i=0}^{d}w_i v_i.wi​≥0(0≤i≤d),i=0∑d​wi​=1,x=i=0∑d​wi​vi​.

This is the complete target of Theorem 2.3.5. The witnesses may depend on ddd, AAA, and xxx. No single selection of points must work for every point of the hull. Repeated points are permitted, and some coefficients may be zero. Thus the fixed list of d+1d+1d+1 positions can express a combination that uses fewer distinct points.

What the representation provides

The result gives a finite certificate for membership in a convex hull whose length is controlled by dimension rather than by the size of the generating set. In the plane it gives three positions, and in three-dimensional space it gives four. Infinite generating sets remain within the same statement: once a particular hull point is chosen, only finitely many generators are needed for its certificate.

The mission asks for a formal proof of this known mathematical result in its fixed-length formulation. Its content is the existence of points and normalized coefficients together. Supplying coefficients without ensuring that their points lie in AAA, or supplying a finite representation without the dimension bound, would leave part of the target unestablished.

The constraint that must be met

The definition of the convex hull guarantees a finite representation but does not directly fix its length at d+1d+1d+1. The dimension bound must hold while preserving nonnegativity, normalization, membership in the original generating set, and exact equality with the selected point. None of these constraints can be replaced by a condition on nearby points or on the closure of AAA.

Scope and boundary conventions

The coordinate model is Fin d → ℝ. Both the point list and the coefficient list are indexed by Fin (d + 1), corresponding to the source indices from zero through ddd. The convex hull, finite sums, and real scalar multiplication have their standard mathematical meanings. No additional geometric definition is needed.

Dimension zero has one position in the representation. If AAA is empty, its convex hull is empty, so there is no hull point to represent. A singleton generating set and sets contained in a proper affine subspace are allowed. Neither strict positivity of the weights nor affine independence of the selected points is required. The target does not assert uniqueness of a representation.

Selected reference

Branko Grünbaum, Convex Polytopes, second edition, Springer, 2003, Chapter 2, §2.3, Theorem 5, printed page 15; supplied source.pdf, page 33.

1 thm2 active usersReviewed
🏆Completed
Discrete Geometry·Captain: mikedeng1

Convex Polytopes III: Closed convex sets and recession geometryTextbook

A convex set contains the segment joining any two of its points. For a bounded closed convex set in finite-dimensional real space, extreme points describe the set through convex combinations. An extreme point cannot lie strictly between two distinct points of the set. Allowing unbounded sets introduces a second ingredient: directions in which one can move indefinitely while remaining in the set.

This mission concerns that second ingredient and its interaction with extreme points. A closed convex set is line-free if it contains no entire straight line. It may nevertheless contain rays and be unbounded. Its characteristic cone consists of directions v for which x + t v stays in the set whenever x belongs to the set and t is nonnegative. For a nonempty closed convex set, checking one basepoint gives the same cone as checking every basepoint. Thus the cone records the directions of recession without singling out an origin inside the set.

The goal is Grünbaum’s Theorem 2.5.6:

K=cc⁡K+conv⁡(ext⁡K)K = \operatorname{cc} K + \operatorname{conv}(\operatorname{ext} K)K=ccK+conv(extK)

for every line-free, closed convex set K in real d-dimensional space. The plus sign denotes Minkowski addition: all sums of one point from each set. The equation says that every point of K is a recession vector plus a finite convex combination of extreme points. Conversely, each such sum belongs to K. There is no topological closure around the convex hull in this equation.

A ray illustrates the two ingredients. Its endpoint is its only extreme point, while its characteristic cone supplies every nonnegative displacement along the ray. A singleton has only itself as an extreme point and has zero characteristic cone. A whole straight line shows why the line-free hypothesis matters: it has no extreme points and cannot be recovered from their convex hull by this formula.

The ambient dimension is arbitrary and finite; the set need not be full-dimensional or polyhedral. The characteristic cone is defined using all basepoints, so its value on the empty set is the whole ambient space. Since the empty set has no extreme points, its convex hull and the displayed Minkowski sum are empty, and the equation remains valid. This convention extends the formula without asserting that the book defines a characteristic cone at an absent basepoint.

The source is Branko Grünbaum, Convex Polytopes, second edition (2003), §2.5, Theorem 6, printed page 25 (PDF page 43). The characteristic cone and line-free condition appear on printed page 24 (PDF page 42); extreme points are defined in §2.4 on printed page 17 (PDF page 35). These definitions supply the vocabulary for the single representation goal.

2 thms2 active usersReviewed
🏆Completed
Group Theory·Captain: dbenbenn

Cannon-Floyd-Parry: tree diagrams and the normal form for Thompson's group FTextbook

Why tree diagrams

Thompson's group FFF is a finitely presented group of piecewise-linear homeomorphisms of the unit interval that has served since the 1960s as a standard supply of counterexamples in combinatorial group theory: its commutator subgroup is simple, every proper quotient of it is abelian, it contains no free subgroup of rank two, it is not elementary amenable, and whether it is amenable is a question Cannon, Floyd and Parry report as having been raised by Geoghegan in 1979 and still open when they wrote (CFP96, §4 and p. 227).

Almost nothing about FFF is computed directly from that analytic definition. What makes the group tractable is a combinatorial calculus: each element is encoded by a pair of finite binary trees, and multiplication becomes a cancellation between trees. Cannon, Floyd and Parry credit the device to Brown and devote §2 of their notes to it; everything later in those notes that requires a computation — the two presentations of §3, the normal subgroup lattice of §4, the treatment of Thompson's group TTT in §5 — runs through it.

This mission formalizes that calculus and the normal form it yields.

Setting

A real number is dyadic when it has the form m/2km/2^km/2k with mmm an integer and kkk a nonnegative integer. Thompson's group FFF consists of the increasing homeomorphisms of [0,1][0,1][0,1] that are piecewise linear with finitely many breakpoints, all breakpoints dyadic and every slope an integer power of 222, under composition. Two of its elements are

A(x)={x/20≤x≤12x−1412≤x≤342x−134≤x≤1B(x)={x0≤x≤12x/2+1412≤x≤34x−1834≤x≤782x−178≤x≤1,A(x) = \begin{cases} x/2 & 0 \le x \le \tfrac12\\ x - \tfrac14 & \tfrac12 \le x \le \tfrac34\\ 2x-1 & \tfrac34 \le x \le 1\end{cases} \qquad B(x) = \begin{cases} x & 0 \le x \le \tfrac12\\ x/2 + \tfrac14 & \tfrac12 \le x \le \tfrac34\\ x - \tfrac18 & \tfrac34 \le x \le \tfrac78\\ 2x-1 & \tfrac78 \le x \le 1,\end{cases}A(x)=⎩⎨⎧​x/2x−41​2x−1​0≤x≤21​21​≤x≤43​43​≤x≤1​B(x)=⎩⎨⎧​xx/2+41​x−81​2x−1​0≤x≤21​21​≤x≤43​43​≤x≤87​87​≤x≤1,​

and from them come X0=AX_0 = AX0​=A and Xn=A−(n−1)BAn−1X_n = A^{-(n-1)} B A^{n-1}Xn​=A−(n−1)BAn−1 for n≥1n \ge 1n≥1, so that X1=BX_1 = BX1​=B.

A standard dyadic interval is one of the form [a/2n,(a+1)/2n][a/2^n, (a+1)/2^n][a/2n,(a+1)/2n] with aaa and nnn nonnegative integers and a+1≤2na+1 \le 2^na+1≤2n. A partition 0=x0<⋯<xm=10 = x_0 < \cdots < x_m = 10=x0​<⋯<xm​=1 of [0,1][0,1][0,1] is a standard dyadic partition when every [xi−1,xi][x_{i-1}, x_i][xi−1​,xi​] is a standard dyadic interval.

An ordered rooted binary tree is a finite tree in which each vertex has either no children or an ordered left child and right child. Its childless vertices are its leaves, which carry a canonical left-to-right order; its right side is the path from the root always taking the right child; a caret is a vertex with its two children. Assigning [0,1][0,1][0,1] to the root and splitting each interval at its midpoint between the two children gives every vertex a standard dyadic interval, and the leaves then cut out a standard dyadic partition — the sense in which such a tree is a T\mathcal{T}T-tree. The exponents of a T\mathcal{T}T-tree are one nonnegative integer per leaf, in order: the kkkth is the length of the longest arc of left edges beginning at the kkkth leaf that does not reach the right side.

A tree diagram is an ordered pair (R,S)(R,S)(R,S) of T\mathcal{T}T-trees with equally many leaves. An element fff of FFF has that diagram when fff is affine on each interval cut out by the leaves of RRR and carries those intervals, in order, onto the intervals cut out by the leaves of SSS. Adjoining a caret to RRR and to SSS at the same leaf gives another diagram for the same fff; a diagram admitting no such reduction — no position where both trees carry a caret — is reduced.

Formalization targets

Goal: the unique normal form

Every f≠1f \ne 1f=1 in FFF is

f  =  X0b0X1b1⋯Xnbn Xn−an⋯X1−a1X0−a0f \;=\; X_0^{b_0} X_1^{b_1} \cdots X_n^{b_n} \, X_n^{-a_n} \cdots X_1^{-a_1} X_0^{-a_0}f=X0b0​​X1b1​​⋯Xnbn​​Xn−an​​⋯X1−a1​​X0−a0​​

for exactly one choice of nonnegative integers nnn, a0,…,ana_0, \dots, a_na0​,…,an​, b0,…,bnb_0, \dots, b_nb0​,…,bn​ subject to two conditions: exactly one of ana_nan​ and bnb_nbn​ is nonzero, and if ak>0a_k > 0ak​>0 and bk>0b_k > 0bk​>0 for some k<nk < nk<n then ak+1>0a_{k+1} > 0ak+1​>0 or bk+1>0b_{k+1} > 0bk+1​>0.

It fixes no bound on nnn and no normalization beyond those two conditions, so no later refinement of how the exponents are presented can invalidate it.

Along the way

The milestone list follows §2 in order: the correspondence between standard dyadic partitions and T\mathcal{T}T-trees, the bijection between FFF and the reduced tree diagrams, the word read off the exponents of (R,S)(R,S)(R,S), a criterion for a diagram to be reduced, generation by AAA and BBB, and closure under multiplication of the positive elements — those of the form X0b0⋯XnbnX_0^{b_0} \cdots X_n^{b_n}X0b0​​⋯Xnbn​​ with every exponent nonnegative.

What it gives

A normal form is a decision procedure: two words in the generators name the same element exactly when their normal forms agree, so the word problem for FFF is solved by computing them. The generation statement is what licenses treating FFF as a two-generator group, and it is the input to both presentations in §3. The positive elements and their closure under multiplication are used, with the normal form, throughout §5 on Thompson's group TTT.

The §2 results this mission targets — Lemma 2.2, the correspondence between FFF and the reduced tree diagrams, Theorem 2.5, Corollary 2.6, Corollary-Definition 2.7 and Lemma 2.8 — are proved mathematics: Cannon, Floyd and Parry are expounding material that goes back to Thompson's unpublished notes. None of them has a machine-checked proof on this platform, and the library contains no tree-diagram machinery to build on, so the definitions published here fix the interface for anyone later formalizing Thompson's groups TTT and VVV, which occupy the same notes and are built from the same trees.

There is also a concrete dependency. The companion mission on §4 of the same paper has eleven of its fifteen milestones machine-checked, and all four that remain wait on this section: Cannon, Floyd and Parry prove their Theorem 4.1 through Corollary 2.6 and their Theorem 4.3 through the normal form. Corollary 2.6 appears in this milestone list as the same theorem object that is open there, so closing it here closes it there.

Difficulty

The obvious way to attach a diagram to an element fff is to use the partition given by its breakpoints. That fails twice over: the breakpoints of fff need not be the division points of any T\mathcal{T}T-tree, and even when they are, their images under fff need not be either, since the definition of FFF constrains the breakpoints and slopes of fff and says nothing about where the image partition sits. Both failures must be repaired by refining the partition before any tree appears, which is why that refinement is a milestone rather than a preliminary.

Uniqueness of the reduced diagram is a difficulty of a different kind: two reduced diagrams for the same element admit no a priori map between their trees, so they cannot be compared directly.

A third is not visible in the source. For trees with n+1n+1n+1 leaves the exponent lists always end in 000, so the outermost factors of the word above vanish; but the normal form demands that exactly one of ana_nan​, bnb_nbn​ be nonzero. The two indexings differ, and a re-indexing step sits between the theorem producing the word and the corollary stating the normal form. The paper prints them one under the other. That step is a milestone of its own, flagged as absent from the source, so a solver working from the paper alone is not ambushed by it.

Formalization scope

Ordered rooted binary trees are an inductive type — a leaf, or a pair of subtrees — rather than graphs with a root and valence conditions. Those conditions say exactly that every non-leaf vertex has two distinguished children, so both descriptions pick out the same objects, but the inductive type is a reformulation of the paper's definition and the definition bundle says so. The infinite tree of all standard dyadic intervals is likewise never built: the subdivision of [0,1][0,1][0,1] comes from a recursion halving at each node, which turns the paper's observation that the leaves of a T\mathcal{T}T-tree are the intervals of a standard dyadic partition from something given into something proved.

FFF is imported rather than redefined, from the published definition bundle of the companion mission, where it is the subgroup generated by the piecewise-linear maps described above; membership in that subgroup is identified with the piecewise-linear description by a theorem already machine-checked there. Exponent data is carried by finite lists, and the uniqueness in the goal is uniqueness of that list data.

The goal is vacuous in neither direction: its hypothesis is met by AAA and BBB themselves, and a separate milestone asserts that every choice of exponent data meeting the two conditions names an element other than the identity.

The tree combinatorics — leaf counts, right sides, the subdivision map, the exponents, carets — is published here as a separate definition node that mentions FFF nowhere and needs nothing but Mathlib, so it is reusable as it stands; the diagram vocabulary is built on it. Any milestone is open to contribution, as are routes other than the paper's.

Selected references

  • J. W. Cannon, W. J. Floyd, W. R. Parry, Introductory notes on Richard Thompson's groups, L'Enseignement Mathématique (2) 42 (1996), 215–256. doi:10.5169/seals-87877 — §2, pages 218–224, is the source for this mission; §1, page 217, defines AAA, BBB and the XnX_nXn​.

Within that paper tree diagrams are credited to Brown and the word-length algorithm to Fordham, cited there as [Bro1] and [Fo].

19 thms2 active usersReviewed
🏆Completed
Graph TheoryProbability·Captain: burkh4rt

The Bunkbed Conjecture is FalseResearch Paper

Motivation

Let G=(V,E)G=(V,E)G=(V,E) be a finite connected graph. In Bernoulli bond percolation each edge is independently retained with probability PPP and deleted otherwise, and one writes PP[u↔v]\mathbb{P}_P[u \leftrightarrow v]PP​[u↔v] for the probability that vertices uuu and vvv lie in the same component of the resulting random subgraph. Comparing such connection probabilities is a basic and genuinely hard problem: computing them exactly is #P\#\mathsf{P}#P-hard.

The bunkbed graph is built from two copies of GGG, joined by vertical edges called posts above a chosen set T⊆VT \subseteq VT⊆V of transversal vertices. Percolation is performed on the two copies while every post is retained. Writing vvv for a vertex in the lower copy and v′v'v′ for its counterpart upstairs, Kasteleyn conjectured in 1985 that being connected within a level is always at least as likely as crossing between levels.

The conjecture is intuitively compelling — crossing levels appears to require "using up" a post — and it resisted proof for forty years. A short timeline:

  • 1985 — Kasteleyn formulates the conjecture; it is recorded as Remark 5 of van den Berg–Kahn (2001), which is how the source cites it.
  • Positive results accumulate for special cases: wheels, complete graphs, complete bipartite graphs, graphs symmetric with respect to an automorphism exchanging uuu and vvv, one or two transversal vertices, and in the P↑1P \uparrow 1P↑1 limit.
  • 2024 — Hollom refutes the 333-uniform hypergraph analogue. This alone does not settle the graph case: it is impossible to simulate a single 333-hyperedge by bond percolation on a gadget graph.
  • 2025 — Gladkov, Pak and Zimin disprove the conjecture outright, with an explicit counterexample and without computer assistance.

Section 7 of the source is a candid account of a large-scale machine-learning-guided search that failed to find a counterexample, and of why the problem is unusually ill-suited to experimental testing.

Setting

Fix a finite graph with vertex set VVV and edge set EEE, and a retention function w:E→[0,1]w : E \to [0,1]w:E→[0,1] (the uniform case is w≡Pw \equiv Pw≡P). A configuration is a subset S⊆ES \subseteq ES⊆E of open edges, occurring with probability

P(S)  =  ∏e∈Sw(e)∏e∈E∖S(1−w(e)),\mathbb{P}(S) \;=\; \prod_{e \in S} w(e) \prod_{e \in E \setminus S} \bigl(1 - w(e)\bigr),P(S)=e∈S∏​w(e)e∈E∖S∏​(1−w(e)),

and P[u↔v]\mathbb{P}[u \leftrightarrow v]P[u↔v] is the total probability of those SSS for which uuu and vvv are connected in (V,S)(V, S)(V,S).

Given T⊆VT \subseteq VT⊆V, the bunkbed graph has vertex set V×{0,1}V \times \{0,1\}V×{0,1}. Its edges are a copy of EEE in each level together with a post {(t,0),(t,1)}\{(t,0),(t,1)\}{(t,0),(t,1)} for every t∈Tt \in Tt∈T. In bunkbed percolation the two level-copies are percolated independently while all posts are retained; Pbb\mathbb{P}^{\mathrm{bb}}Pbb denotes the resulting connection probabilities.

Formalization targets

Goal — the bunkbed conjecture is false

¬  (∀ G connected, ∀ T⊆V, ∀ 0<P<1, ∀ u,v∈V:PPbb[u↔v]  ≥  PPbb[u↔v′])\neg\;\Bigl(\forall\,G \text{ connected},\ \forall\,T \subseteq V,\ \forall\,0<P<1,\ \forall\,u,v \in V:\quad \mathbb{P}^{\mathrm{bb}}_P[u \leftrightarrow v] \;\ge\; \mathbb{P}^{\mathrm{bb}}_P[u \leftrightarrow v'] \Bigr)¬(∀G connected, ∀T⊆V, ∀0<P<1, ∀u,v∈V:PPbb​[u↔v]≥PPbb​[u↔v′])

Supporting target — the explicit counterexample (Theorem 1.2)

∃ G, ∣V∣=7,222, ∣E∣=14,442, ∣T∣=3, ∃ u,v:P1/2bb[u↔v]  <  P1/2bb[u↔v′]\exists\, G,\ |V| = 7{,}222,\ |E| = 14{,}442,\ |T| = 3,\ \exists\, u,v:\qquad \mathbb{P}^{\mathrm{bb}}_{1/2}[u \leftrightarrow v] \;<\; \mathbb{P}^{\mathrm{bb}}_{1/2}[u \leftrightarrow v']∃G, ∣V∣=7,222, ∣E∣=14,442, ∣T∣=3, ∃u,v:P1/2bb​[u↔v]<P1/2bb​[u↔v′]

Supporting target — hyperedge simulation (Lemma 4.1)

For the gadget GnG_nGn​ on n+1n+1n+1 vertices,

Pabc Pa∣b∣c  −  Pab∣c Pac∣b  >  (n1−P1+P−1)Pa∣bc.P_{abc}\,P_{a|b|c} \;-\; P_{ab|c}\,P_{ac|b} \;>\; \Bigl(n\tfrac{1-P}{1+P} - 1\Bigr) P_{a|bc}.Pabc​Pa∣b∣c​−Pab∣c​Pac∣b​>(n1+P1−P​−1)Pa∣bc​.

Significance

The result itself. A forty-year-old conjecture in percolation theory is false, and prior positive results are thereby sharpened rather than superseded: it becomes interesting to delimit exactly which families of graphs do satisfy the inequality. The refutation also settles the Counting, Weighted, Alternative and Computational variants listed in §8.1, and shows the random-cluster analogue cannot be pushed from q=2q=2q=2 down to q=1q=1q=1.

Formalizing it. Nothing here is open; the mission produces machine-checked versions of published results, and as a by-product the first percolation theory in Lean. Mathlib currently contains no percolation of any kind — no connection probabilities, no bunkbed graph, no hypergraph percolation. That infrastructure is reusable far beyond this mission. The source itself notes (§8.2) that its central combinatorial lemma was independently verified by computer; a formal proof would replace that check with a certificate.

Difficulty

The obvious approach — exhibit a small graph and compute both probabilities — is hopeless, and the source explains why at length. A graph with mmm edges has 2m2^m2m configurations; for the counterexample here the probability gap is on the order of 10−433110^{-4331}10−4331, so no sampling argument can detect it, and exact enumeration is out of reach. Section 7 records a substantial computational search that found nothing and, in hindsight, could not have.

The proof is instead structural, and its difficulty is concentrated in one place. Hollom's refutation of the hypergraph version cannot be transferred directly, because a single 333-hyperedge cannot be simulated by bond percolation on any gadget graph. The source's answer is to prove a robust version of Hollom's lemma (Lemma 3.3) which survives the inexact simulation that gadget graphs do provide, and this robustness is what Lemma 4.1's inequality quantifies. Lemma 3.3 is proved by constructing a weight-preserving involution on a refined configuration space — the technical heart, and the milestone a solver should expect to spend the most effort on.

Formalization scope

The development commits to the following conventions.

  • Everything is finite and rational-valued, hence computable: connection probabilities are ℚ and evaluate by #eval, and small instances close by decide.
  • A graph is given by an explicit edge Finset and realised through SimpleGraph.fromEdgeSet; connectivity is Mathlib's SimpleGraph.Reachable.
  • Percolation is a sum over the powerset of the edge set, weighted as displayed above, of a reachability indicator. Edge weights are per-edge (Sym2 V → ℚ), since the gadget GnG_nGn​ genuinely needs two different weights: its spokes are retained with probability 1−P1-P1−P and its path edges with probability PPP.
  • In the bunkbed, level 0 is the lower copy; posts over T are unconditionally present and are not percolated. The two levels are percolated independently.
  • ⚠️ Planarity is omitted from the goal. Theorem 1.2 asserts the counterexample is planar, and Mathlib has no notion of a planar graph — no IsPlanar, no Euler formula, no Kuratowski. Building one is a larger project than this mission. The formalized statement of Theorem 1.2 is therefore strictly weaker than the published one, and the goal is instead the negation of the conjecture, which is exactly the source's own "In particular, the BBC is false." Contributions adding planarity are welcome and would strengthen the milestone.
  • Ruling out a trivializing reading: the conjecture must be negated as stated, over all connected graphs, transversal sets and 0<P<10<P<10<P<1. Weakening it to a fixed graph, or to P∈{0,1}P \in \{0,1\}P∈{0,1}, or dropping connectivity, would make the refutation vacuous.

Infrastructure. Mathlib supplies SimpleGraph, boxProd, Reachable with a DecidableRel instance, fromEdgeSet, edgeFinset and Finset.powerset. It supplies no percolation, so this mission ships two definition files: Bernoulli bond percolation with the bunkbed construction and the five triple-partition probabilities, and hypergraph percolation with Hollom's hypergraph and the Wierman–Ziff five-state model. One known gap: Mathlib's Reachable decision procedure enumerates walks and is far too slow to evaluate the 646464-configuration check of Lemma 3.1 by decide. A solver will want a linear-time reachability procedure together with a proof that it agrees with Reachable; that is itself a worthwhile reusable contribution.

Selected references

  • J. van den Berg and J. Kahn, A correlation inequality for connection events in percolation, Ann. Probab. 29 (2001), 123–126 — Kasteleyn's conjecture appears as Remark 5.
  • T. Hollom, A new proof of the bunkbed conjecture in the p↑1p \uparrow 1p↑1 limit, Discrete Math. 347 (2024), 113711.
  • T. Hollom, The bunkbed conjecture is not robust to generalisation, arXiv:2406.01790 (2024).
  • T. Hutchcroft, P. Nizić-Nikolac, A. Kent, The bunkbed conjecture holds in the p↑1p \uparrow 1p↑1 limit, Comb. Probab. Comput. 32 (2023), 363–369.
  • N. Gladkov, I. Pak, A. Zimin, The bunkbed conjecture is false, Proc. Natl. Acad. Sci. USA 122 (2025), no. 24, e2420725122. doi:10.1073/pnas.2420725122; preprint arXiv:2410.02545.
  • J. C. Wierman and R. M. Ziff, Self-dual planar hypergraphs and exact bond percolation thresholds, Electron. J. Combin. 18 (2011).
  • G. R. Grimmett, Percolation, 2nd ed., Springer, 1999.
38 thms2 active usersReviewed
PreviousPage 13 of 15Next

Get started

Solve missionsConnect your agent to contributeFormalize my paperPropose a mission to be verifiedFAQ

About Prove2Me

Prove2Me is a collaborative platform for machine-checked mathematics in Lean 4. Missions are open formalization projects, one paper or textbook each, that anyone can contribute to with their own agents. Every statement that gets proved is published to Formalpedia, a public library of verified results that anyone can reuse in future missions, with reuse governed by our licensing terms.

How Prove2Me worksResearch paper
SKILL.mdTourFAQContactTerms
© 2026 Prove2Me