Approximation Algorithms for Combinatorial Problems I: The Subset-Sum Algorithms A_k Have Worst-Case Ratio (k+1)/kResearch Paper
Motivation
David S. Johnson's 1974 paper Approximation Algorithms for Combinatorial Problems (J. Comput. System Sci. 9 (1974) 256–278) is one of the founding papers of the theory of approximation algorithms. It asks, for optimization problems whose decision versions Karp had just shown to be polynomial complete, how close a fast heuristic can be guaranteed to come to the optimum in the worst case, and it measures this with a worst-case performance ratio that is still the standard yardstick.
Its first example is SUBSET-SUM, the simplest form of the knapsack problem: pack items of given sizes into a knapsack of capacity so as to fill it as much as possible. For this problem the paper gives a family of algorithms , one for each , whose guaranteed ratio tends to . It is one of the first examples of what is now called a polynomial-time approximation scheme: for every there is a polynomial-time algorithm within a factor of optimal. Sahni (1975) extended the idea to the knapsack problem with utilities, and Ibarra and Kim (1975) later obtained fully polynomial schemes for knapsack and subset-sum.
This mission formalizes Theorem 1 of the paper, the performance guarantee of together with its tightness.
Setting
An input of SUBSET-SUM is a finite set , a positive rational size for every , and a positive rational bound . An approximate solution is a subset with , where the measure is . The problem is a maximization problem with optimal measure
Fix and call big if and small otherwise. Algorithm keeps a set , its measure , and the remaining elements :
- is a subset of the big elements whose measure is as large as possible without exceeding ; and .
- If for every , return .
- Otherwise pick with as large as possible without exceeding , move it from to , add to , and return to step 2.
Steps 1 and 3 may have ties. Following the paper, a set is choosable by if some resolution of all ties produces it, and the performance on input is the smallest measure of a choosable output. The ratio is , and is its maximum over inputs of size at most .
Formalization targets
Goal: Theorem 1 (p. 260)
For and ,
Formally, for every : every choosable output of every input satisfies ; and for every some input has a choosable output with and .
Milestones
- For choosable and any approximate solution, (p. 260).
- If a small is not in a choosable , then , hence (p. 261).
- The stronger dichotomy: or (p. 260).
- The lower-bound input , , otherwise, : its optimum is , some output is choosable, and every choosable output has measure (p. 261).
Significance
Theorem 1 shows that SUBSET-SUM admits polynomial-time algorithms with any worst-case ratio above , in contrast with the other problems of the paper (set covering, graph colouring, maximum clique), whose best known ratios grow with the input. The algorithms are an early instance of the partial-enumeration schemes later used for knapsack-type problems. The tightness half shows that the analysis of itself cannot be sharpened.
The theorem has a short published proof, but no machine-checked version is known; there is no subset-sum or knapsack approximation result on the platform. The mission produces a reusable model of SUBSET-SUM, a model of nondeterministic algorithms through a run relation that captures every tie-break, and a checked proof that the worst case is exactly . The same modelling pattern (choosable outputs, worst-case ratio taken over them) is used in the sibling missions of this series for MAX-SAT, set covering and exact covering.
Difficulty
The arithmetic of the upper bound is short; the difficulty is in reasoning about the algorithm as a nondeterministic process. The natural first attempt, implementing as a function with a fixed tie-breaking rule, proves a weaker statement: the guarantee must hold for every output the algorithm may return, including adversarial ties in step 1 (several maximum-measure sets of big elements) and step 3. Facts that are obvious for a single run, such as always equalling or which elements can enter after step 1, have to be established for the run relation as a whole. The lower bound requires tracing the run on the explicit input for general : exactly unit elements are added after , and this must be shown for every choosable run, not only for one.
Formalization scope
- Numbers. Sizes and the bound are rationals (
ℚ), as in the paper; sizes are required to be positive on and . The index is a natural number with as a hypothesis; is rational division, and "big" is the strict inequality . - Optimum.
opt uisFinset.sup'of the measure over the finite set of approximate solutions, which always contains ; it is when no element fits. - Run relation.
Choosable k u T₁states that some admissible step 1 choice, followed by a finite chain of admissible iterations (Relation.ReflTransGen), reaches a halting state returning . Every "closest to, without exceeding" is an existential choice among all maximizers. - Size-free restatement. The paper's input size ("in some standard notation") is never fixed, so the goal quantifies over all inputs instead of over sizes. The upper bound for all choosable outputs is equivalent to for all ; since is nondecreasing, the limit claim is equivalent to the supremum of the ratio over all inputs being , which is the second part.
- Multiplicative ratios. No ratio is written as a division, so an output of measure cannot satisfy a bound vacuously; the lower-bound part requires . The value is not claimed to be attained: the paper's family has ratio .
- Lower-bound input. A
defonFin (k + 2)exactly as on the page, with (the page leaves the range implicit; keeps the only big element that fits when ). - Ruled out. A formalization with a deterministic tie-break, with a bound of the form in a field where , or with tightness for a single fixed would be trivial or weaker; none of these is the target.
Contributions welcome: proofs of the milestones and the goal, invariant lemmas for the run relation, and further sanity checks on small inputs. The running-time remark ( for step 1) and Sahni's knapsack extension are not part of the mission.
Selected references
- D. S. Johnson, Approximation algorithms for combinatorial problems, Journal of Computer and System Sciences 9 (1974) 256–278. https://doi.org/10.1016/S0022-0000(74)80044-9
- S. Sahni, Approximate algorithms for the 0/1 knapsack problem, Journal of the ACM 22 (1975) 115–124. https://doi.org/10.1145/321864.321873
- O. H. Ibarra, C. E. Kim, Fast approximation algorithms for the knapsack and sum of subset problems, Journal of the ACM 22 (1975) 463–468. https://doi.org/10.1145/321906.321909
- R. M. Karp, Reducibility among combinatorial problems, in Complexity of Computer Computations, Plenum (1972) 85–103. https://doi.org/10.1007/978-1-4684-2001-2_9