Sorting in c log n Parallel Steps: Sorting Networks of Logarithmic DepthResearch Paper
Motivation
A sorting network is a sorting procedure whose sequence of comparisons is fixed in advance, independently of the data. Its depth, the number of rounds of simultaneous comparisons on disjoint pairs, is the parallel running time. Sorting networks are used in parallel and hardware sorting, in switching networks, and in cryptography, where a data-independent (oblivious) sequence of operations is required. How small the depth can be as a function of the number of inputs is a basic question of parallel computation.
Timeline:
- 1968. Batcher's odd-even merge sort and bitonic sort give networks of depth and size (K. E. Batcher, Sorting networks and their applications, AFIPS Spring Joint Computer Conference, 1968). For a power of two they remain the best explicit networks in practice.
- 1973. Knuth's The Art of Computer Programming, Vol. 3, §5.3.4, surveys sorting networks. A simple counting argument gives the lower bound: every sorting network has depth at least , since each output depends on at most inputs.
- 1983. Ajtai, Komlós and Szemerédi construct networks of depth and size (Combinatorica 3 (1983) 1–19, doi:10.1007/BF02579338), matching the lower bound up to a constant. The constant is not computed in the paper and is known to be very large.
- 1990. Paterson simplifies the construction and gives the first explicit, still very large, depth constant (M. S. Paterson, Improved sorting networks with O(log N) depth, Algorithmica 5 (1990) 75–92, doi:10.1007/BF01840378).
- 2014. Goodrich gives Zig-zag sort, a simpler deterministic data-oblivious sorting algorithm with comparisons that avoids the AKS machinery but is not of logarithmic depth (arXiv:1403.2777).
Setting
There are registers holding elements of a linearly ordered set. An elementary step (a comparator) with compares the contents of and and exchanges them if the content of is larger. Afterwards holds the minimum and the maximum of the two, and every other register is unchanged. A parallel step is a set of comparators in which no register occurs twice, so it has at most comparators. A comparator network is a finite sequence of parallel steps, fixed before the input is seen. Its depth is the number of parallel steps and its size the total number of comparators. sorts if for every input the output satisfies .
The construction runs on the tree of finite - sequences, whose levels are ordered lexicographically. A chain on level assigns to every node of that level a set of registers, with the sets pairwise disjoint and of a common size . A ⟨k, ε⟩ expander on ⟨A, B⟩, for disjoint register sets and , is a bipartite graph between and of maximum degree in which every nonempty has more than neighbours, and symmetrically for . The Lean development uses the names ComparatorNetwork, compareExchange, IsChain, chainN, IsExpander, IsLowerSection for these objects.
Formalization targets
Goal: the AKS theorem (Abstract and §1, p. 1)
The constant is absolute and is not fixed. Any explicit value would be invalidated by the next improvement, and the paper gives none.
Milestones (the paper's numbered lemmas that hold as stated)
- Lemma 3 (p. 6): for and there is such that every pair of disjoint sets with carries a expander.
- Lemma 4 (p. 7): performing every comparator of such an expander once, in any order, from to leaves all but an -fraction of any lower section with in , and symmetrically for upper sections in :
- Lemma 1 (pp. 3–4): the splitting of a chain, which moves one register of each leaf set up the tree, produces chains with properties (1.1)–(1.5).
- Lemma 2 (p. 4): chains with exist under conditions (2a), (2.b).
- Lemma 12(a) (p. 14): a violation of the order relation between two nodes of a level is witnessed by two consecutive nodes.
Significance
The result. The AKS theorem settles the asymptotic depth of sorting networks at and their size at . It gives an -time sorting algorithm with processors that performs only data-independent comparisons. It is the standard reference point for oblivious sorting in parallel algorithms, circuit complexity (sorting is in via comparators) and oblivious RAM constructions. Lemma 4, the ε-halver property of expander comparisons, is the component that later constructions (Paterson) reuse.
Formalizing it. The theorem has been proved since 1983. No Lean proof of the AKS theorem is known. Mathlib has no expander graphs in the ⟨k, ε⟩ sense and no sorting networks. The mission asks for a formal proof of the headline theorem by any route (the AKS construction or Paterson's variant), and for formal proofs of the paper's verified lemmas as reusable components. The expander lemma needs either an explicit family (Margulis; Gabber–Galil) or a probabilistic existence argument, both substantial on their own.
Difficulty
Every elementary argument stalls at depth : recursive merging needs merge rounds, and merging two sorted lists by a comparator network needs depth . A depth of therefore cannot come from exact merging. It must come from constant-depth approximate operations (ε-halvers, which require bounded-degree expanders) combined with a mechanism that corrects the errors they leave. In the paper this mechanism is a movement of registers up and down a binary tree, controlled by a family of constants chosen in a fixed order ("", p. 2). The accounting that shows the misplaced elements decay geometrically is the hard part. Several intermediate lemmas of the paper are false as printed, so the paper's text is not a checklist to transcribe.
Formalization scope
Conventions committed to in Lean:
- Registers are
Fin n, and contents lie in an arbitrary linearly ordered type. A network is aListof layers, each aList (Fin n × Fin n)of comparators with distinct endpoints, and no register occurs twice in a layer. A comparator puts the minimum into , and both directions and are allowed. Sorts means the output is monotone for every linearly ordered type and every input, not only for permutations. - is
Real.logb 2 n, and the goal is stated for . The constant is quantified before . - Tree levels are
Fin (2^i), with numeric order equal to lexicographic order. A chain is aFin (2^i) → Finset R. - Definition 2.2 of the paper, read literally, requires , which fails, so no graph would be an expander. The expansion inequalities are imposed on nonempty sets only, and the strict inequality is kept.
Trivializing formalizations are ruled out. The goal is not "for every there is a network of depth " with the constant chosen after , which is true for trivial reasons. Layers without the disjointness condition would let a single layer contain a whole insertion sort. A bound on the number of comparisons alone, with unbounded depth, is a different and much older result; the goal states both the depth and the size bound.
The mission states the AKS theorem and the paper's lemmas that are correct as stated. It does not formalize the AKS algorithm itself (, , the operations CH1–CH4, IMP) or its intermediate Lemmas 5–11 and 13–15. Those depend on unspecified constants constrained only by "sufficiently small" chains, and Lemmas 5, 10 and 12(b) are false as printed. A solver may of course define the algorithm, with pinned constants, as part of a proof.
Contributions welcome: a library of comparator networks (composition, the 0-1 principle, depth of Batcher's networks), existence of bounded-degree bipartite expanders, the ε-halver lemma, and any complete proof of the goal. The network and expander definitions are independent of this paper and reusable.
Selected references
- M. Ajtai, J. Komlós, E. Szemerédi, Sorting in c log n parallel steps, Combinatorica 3(1) (1983) 1–19. doi:10.1007/BF02579338
- K. E. Batcher, Sorting networks and their applications, Proc. AFIPS Spring Joint Computer Conference 32 (1968) 307–314. doi:10.1145/1468075.1468121
- D. E. Knuth, The Art of Computer Programming, Vol. 3: Sorting and Searching, Addison-Wesley, 1973, §5.3.4.
- G. A. Margulis, Explicit constructions of concentrators, Problems of Information Transmission 9 (1973) 325–332.
- O. Gabber, Z. Galil, Explicit constructions of linear-sized superconcentrators, J. Computer and System Sciences 22(3) (1981) 407–420. doi:10.1016/0022-0000(81)90040-4
- M. S. Paterson, Improved sorting networks with O(log N) depth, Algorithmica 5 (1990) 75–92. doi:10.1007/BF01840378
- M. T. Goodrich, Zig-zag sort: a simple deterministic data-oblivious sorting algorithm running in O(n log n) time, STOC 2014. arXiv:1403.2777