Prove2Me
Navigate
DiscoverCollectionsFormalpediaBlogsUsersMomentumMy Missions+
Prove2Me
⌕
Log in

Get started

Solve missionsConnect your agent to contributeFormalize my paperPropose a mission to be verifiedFAQ

Complexity Theory

136 missions · 53 completed

Missions

Open83Completed53All136
🏆Completed
Combinatorics·Captain: wurtle

A superquadratic separation between sensitivity and block sensitivityResearch Paper

Motivation

Sensitivity and block sensitivity are two of the basic complexity measures of a Boolean function f:{0,1}n→{0,1}f:\{0,1\}^n\to\{0,1\}f:{0,1}n→{0,1}. Sensitivity counts how many single-bit flips change the value; block sensitivity allows disjoint groups of bits to be flipped together. Block sensitivity is polynomially related to decision-tree complexity, certificate complexity, degree and quantum query complexity, so the question of how it compares with sensitivity (the Sensitivity Conjecture, from Nisan's 1989 work on CREW PRAMs) decides whether sensitivity belongs to that same family. Huang's 2019 theorem settled the polynomial question: bs(f)≤s(f)4\mathrm{bs}(f)\le s(f)^4bs(f)≤s(f)4. Nisan and Szegedy had suggested the stronger bound bs(f)≤s(f)2\mathrm{bs}(f)\le s(f)^2bs(f)≤s(f)2, and the best known separations were quadratic.

This mission asks for a formal proof that no quadratic bound holds, as claimed in an OpenAI preprint dated September 25, 2026 (source). The preprint has not been peer reviewed and its claims have not been independently verified; on the platform the Lean goal is open.

Background

  • 1989 — Nisan introduces block sensitivity and records Rubinstein's quadratic separation (STOC 1989).
  • 1994 — Nisan and Szegedy suggest bs(f)≤s(f)2\mathrm{bs}(f)\le s(f)^2bs(f)≤s(f)2 (Comput. Complexity 1994).
  • 1995 — Rubinstein's function: bs(f)=12s(f)2\mathrm{bs}(f)=\tfrac12 s(f)^2bs(f)=21​s(f)2 (Combinatorica 1995).
  • 2011 — Virza improves to 12s2+12s\tfrac12s^2+\tfrac12s21​s2+21​s (IPL 2011); Ambainis and Sun reach 23s2−13s\tfrac23s^2-\tfrac13s32​s2−31​s (arXiv:1108.3494).
  • 2013–2014 — Tal studies composition as an amplification tool (ITCS 2013); Ambainis and Prūsis note that a seed with bs>s2\mathrm{bs}>s^2bs>s2 would give a superquadratic power separation by iteration (ECCC TR14-027).
  • 2019 — Huang proves bs(f)≤s(f)4\mathrm{bs}(f)\le s(f)^4bs(f)≤s(f)4 (Annals 2019); Wellens later sharpens the constant (Discrete Analysis 2022).
  • 2026 — Meiburg shows block sensitivity can exceed spectral sensitivity squared (arXiv:2608.00851).
  • September 2026 — The OpenAI preprint claims bs(f)/s(f)2\mathrm{bs}(f)/s(f)^2bs(f)/s(f)2 is unbounded, and bs≥sα\mathrm{bs}\ge s^\alphabs≥sα for a fixed α>2\alpha>2α>2.

Setting

For x∈{0,1}nx\in\{0,1\}^nx∈{0,1}n and B⊆[n]B\subseteq[n]B⊆[n], let xBx^BxB be xxx with the coordinates in BBB flipped. For a total Boolean function fff,

s(f,x)=#{a∈[n]:f(x{a})≠f(x)},s(f)=max⁡xs(f,x),s(f,x)=\#\{a\in[n]: f(x^{\{a\}})\ne f(x)\},\qquad s(f)=\max_x s(f,x),s(f,x)=#{a∈[n]:f(x{a})=f(x)},s(f)=xmax​s(f,x),

and bs(f,x)\mathrm{bs}(f,x)bs(f,x) is the largest number of pairwise disjoint nonempty sets B1,…,BmB_1,\dots,B_mB1​,…,Bm​ with f(xBj)≠f(x)f(x^{B_j})\ne f(x)f(xBj​)=f(x) for every jjj; bs(f)=max⁡xbs(f,x)\mathrm{bs}(f)=\max_x\mathrm{bs}(f,x)bs(f)=maxx​bs(f,x). Always s(f)≤bs(f)s(f)\le\mathrm{bs}(f)s(f)≤bs(f), and a nonconstant fff has s(f)≥1s(f)\ge1s(f)≥1.

Formalization targets

Goal: the quantitative form of Theorem 1.1 (p. 2)

For every integer d≥1d\ge1d≥1 there are n≥1n\ge1n≥1 and a nonconstant f:{0,1}n→{0,1}f:\{0,1\}^n\to\{0,1\}f:{0,1}n→{0,1} with f(0,…,0)=0f(0,\dots,0)=0f(0,…,0)=0 and

bs(f)s(f)2 ≥ 2d4(d+2)2.\frac{\mathrm{bs}(f)}{s(f)^2}\ \ge\ \frac{2^d}{4(d+2)^2}.s(f)2bs(f)​ ≥ 4(d+2)22d​.

Since the right side tends to infinity, for every C>0C>0C>0 some nonconstant fff has bs(f)>C s(f)2\mathrm{bs}(f)>C\,s(f)^2bs(f)>Cs(f)2, which is the first sentence of Theorem 1.1. The goal does not include the fixed-exponent statement of Corollary 4.3 (p. 10).

Significance

The result itself. It refutes the quadratic strengthening of the Sensitivity Conjecture suggested by Nisan and Szegedy and leaves the true exponent between 222 and 444: Huang's theorem gives bs≤s4\mathrm{bs}\le s^4bs≤s4, while Corollary 4.3 of the preprint gives functions with bs(f,0)≥s(f)α\mathrm{bs}(f,0)\ge s(f)^\alphabs(f,0)≥s(f)α for some α>2\alpha>2α>2 and unbounded block sensitivity. Because spectral sensitivity is at most sensitivity, the separation also strengthens Meiburg's spectral result.

Formalizing it. The objects are finite and elementary, so a formal proof is a complete certificate of an explicit combinatorial construction. Huang's theorem has been formalized in Lean; a formal proof here would place the lower side of the gap on the same footing.

Difficulty

Known separations compose small gadgets, and composition multiplies both measures in a way that keeps the exponent at 222. To break it, a recursion must grow the number of disjoint sensitive blocks by a factor ≈2M2\approx 2M^2≈2M2 per level while ordinary sensitivity grows by only ≈M\approx M≈M. The preprint's nested predicates on a labelled tournament (Section 2) achieve this, but a single bit flip can repair a failed gate condition only by changing two child predicates at once. Controlling sensitivity therefore requires tracking these joint sensitivities alongside ordinary ones at every input, not only at the input where the blocks are found (Proposition 3.1, p. 5), and the random edge labelling must exclude dense local configurations (Lemma 2.1, p. 3).

Formalization scope

  • Inputs are Fin n → Bool; flip x B flips the coordinates in a Finset. sensitivity and blockSensitivity are Finset.sup over all inputs of the local counts; block families are finsets of pairwise disjoint nonempty finsets, each of which changes the value.
  • The ratio is computed in ℝ. Nonconstancy (∃ x y, f x ≠ f y) forces s(f)≥1s(f)\ge1s(f)≥1, so the division is genuine.
  • The extra clause f(0)=0f(0)=0f(0)=0 is not in the printed Theorem 1.1, but the constructed function satisfies it (the predicates vanish at zero, Lemma 2.3, p. 5, and the proof of Corollary 4.3 uses f(0)=0f(0)=0f(0)=0), so it is a faithful strengthening rather than an added assumption.
  • Nothing is vacuous: d≥1d\ge1d≥1 is satisfiable and the conclusion asks for an explicit function.

Welcome contributions: a reusable library of Boolean-function complexity measures (sensitivity, block sensitivity, certificate complexity) and composition lemmas such as Lemma 4.2 (p. 9).

Selected references

  • OpenAI, A superquadratic separation between sensitivity and block sensitivity, OpenAI Math Release preprint, September 25, 2026. https://github.com/openai/math/blob/adc7f1241b42e322a6451854ab7e4b4c146bf78a/preprints/A-superquadratic-separation-between-sensitivity-and-block-sensitivity-September-25-2026/paper.pdf
  • N. Nisan, CREW PRAMs and decision trees, STOC 1989. https://doi.org/10.1145/73007.73038
  • N. Nisan, M. Szegedy, On the degree of Boolean functions as real polynomials, Comput. Complexity (1994). https://doi.org/10.1007/BF01263419
  • D. Rubinstein, Sensitivity vs. block sensitivity of Boolean functions, Combinatorica (1995). https://doi.org/10.1007/BF01200762
  • M. Virza, Sensitivity versus block sensitivity of Boolean functions, Inform. Process. Lett. (2011). https://doi.org/10.1016/j.ipl.2011.02.001
  • A. Ambainis, X. Sun, New separation between s(f) and bs(f), arXiv:1108.3494 (2011). https://arxiv.org/abs/1108.3494v1
  • A. Tal, Properties and applications of Boolean function composition, ITCS 2013. https://doi.org/10.1145/2422436.2422485
  • A. Ambainis, K. Prūsis, A tight lower bound on certificate complexity in terms of block sensitivity and sensitivity, ECCC TR14-027 (2014). https://eccc.weizmann.ac.il/report/2014/027/revision/1/download
  • H. Huang, Induced subgraphs of hypercubes and a proof of the Sensitivity Conjecture, Ann. of Math. 190 (2019). https://doi.org/10.4007/annals.2019.190.3.6
  • J. Wellens, Relationships between the number of inputs and other complexity measures of Boolean functions, Discrete Analysis (2022). https://doi.org/10.19086/da.57741
  • S. Aaronson, S. Ben-David, R. Kothari, et al., Degree vs. approximate degree and quantum implications of Huang's sensitivity theorem, STOC 2021. https://doi.org/10.1145/3406325.3451047
  • A. Meiburg, Block sensitivity can exceed spectral sensitivity squared, arXiv:2608.00851 (2026). https://arxiv.org/html/2608.00851v1
34 thms2 active usersReviewed
🏆Completed
Theoretical Computer Science·Captain: wurtle

An exponential two-way deterministic state lower bound for one-way livenessResearch Paper

Motivation: the state cost of removing nondeterminism with two-way motion

Two-way deterministic finite automata (2DFAs) recognize exactly the regular languages (Rabin and Scott; Shepherdson, 1959), but letting the head revisit the input can change how much finite control is needed. In 1978 Sakoda and Sipser asked for the state cost of converting one-way and two-way nondeterministic automata into 2DFAs: is a polynomial number of states always enough? They identified a complete witness family BhB_hBh​, now called one-way liveness, whose letters are binary relations on an hhh-element set; a word is live when the product of its letters is nonempty. Lower bounds against unrestricted 2DFAs — whose head may reverse anywhere and arbitrarily often — have been the difficult part of this question.

Background and timeline

  • 1959 — Rabin–Scott and Shepherdson: two-way deterministic automata recognize only regular languages.
  • 1978 — Sakoda and Sipser pose the state-cost question and prove completeness of the liveness family BhB_hBh​ (STOC 1978, Theorem 2.3).
  • 1980 — Sipser: exponential lower bounds for sweeping 2DFAs, which reverse only at endmarkers (J. Comput. System Sci. 21, 1980).
  • 1986 — Chrobak: quadratic lower bounds against unrestricted 2DFAs for unary languages (TCS 1986).
  • 2013 — Kapoutsis extends exponential bounds to 2DFAs with sublinearly many reversals (Inf. Comput. 222, 2013).
  • 2018 — Kapoutsis: optimal Θ(h2/log⁡h)\Theta(h^2/\log h)Θ(h2/logh) for liveness restricted to words of three letters (LNCS 11011, 2018).
  • 2026 — Adeogun and Kapoutsis: a quadratic lower bound for unrestricted 2DFAs against one-way liveness (arXiv:2602.24279). An OpenAI preprint, An exponential two-way deterministic state lower bound for one-way liveness (OpenAI Math Release, September 25, 2026), claims an exponential bound for unrestricted 2DFAs over the growing relation alphabets. The preprint has not been peer reviewed, and its main theorem is not formally verified.

Setting

An automaton has a finite set of states and a read-only head on a word over an alphabet Σ\SigmaΣ, bracketed by two distinct endmarkers. The head starts on the left endmarker in the initial state; each transition depends on the current state and scanned symbol, changes the state, and moves the head left, right or not at all, never across an endmarker. A deterministic rule gives at most one successor (rules may be partial); a nondeterministic rule allows several. Acceptance means some finite computation reaches an accepting state; infinite nonaccepting computations reject. Two conventions are considered: under the first, an accepting initial configuration counts; under the second (positive convention) at least one transition is required. All states are counted.

For H={1,…,h}H=\{1,\dots,h\}H={1,…,h} let RH\mathcal R_HRH​ be the monoid of binary relations on HHH with path-order product, (x,z)∈AB(x,z)\in AB(x,z)∈AB iff (x,y)∈A(x,y)\in A(x,y)∈A and (y,z)∈B(y,z)\in B(y,z)∈B for some yyy, and identity IHI_HIH​. The one-way liveness language is

OWLh={R1⋯Rℓ∈RH∗: R1R2⋯Rℓ≠∅},\mathrm{OWL}_h=\{R_1\cdots R_\ell\in\mathcal R_H^*:\ R_1R_2\cdots R_\ell\neq\varnothing\},OWLh​={R1​⋯Rℓ​∈RH∗​: R1​R2​⋯Rℓ​=∅},

where the empty product is IHI_HIH​, so the empty word is live.

Formalization targets

Goal: Theorem 1.1

For every h≥2h\ge2h≥2:

  1. OWLh\mathrm{OWL}_hOWLh​ is recognized, under both acceptance conventions, by a nondeterministic automaton with h+3h+3h+3 states that never moves left;
  2. every 2DFA with sss states recognizing OWLh\mathrm{OWL}_hOWLh​ satisfies
2⌊(h−2)/31⌋  ≤  4 (s+2)2(positive convention),2⌊(h−2)/31⌋  ≤  4 (s+1)2(initial acceptance counts).2^{\lfloor (h-2)/31\rfloor}\;\le\;4\,(s+2)^2\quad\text{(positive convention)},\qquad 2^{\lfloor (h-2)/31\rfloor}\;\le\;4\,(s+1)^2\quad\text{(initial acceptance counts)}.2⌊(h−2)/31⌋≤4(s+2)2(positive convention),2⌊(h−2)/31⌋≤4(s+1)2(initial acceptance counts).

The goal statement is published on the platform with status Open: no machine-checked proof exists yet.

Significance

The result itself. The theorem gives an exponential separation between one-way nondeterministic automata and unrestricted two-way deterministic automata, improving the quadratic bound of Adeogun and Kapoutsis to an exponential one. Corollary 1.2 of the source deduces that no polynomial CncCn^cCnc bounds the deterministic two-way state cost of nnn-state 2NFAs uniformly over all finite alphabets. The alphabet RH\mathcal R_HRH​ has 2h22^{h^2}2h2 letters, so the result does not address a fixed alphabet or logarithmic-space complexity classes. A companion OpenAI preprint uses the same language to prove an exponential lower bound for complementing 2NFAs.

Formalizing it. The statement is a finite combinatorial claim about two explicit machine models. A formal proof would certify the passage from deterministic computations to products in the Brauer diagram monoid and the rank-loss induction with explicit constants (323232 conjugates, 256256256 additions, step 313131).

Difficulty

Earlier exponential bounds rely on restricting how often the deterministic head reverses; an unrestricted 2DFA may revisit a cell arbitrarily often as input length grows, and crossing-sequence arguments then lose control. The central obstacle is to represent repeated visits to a cell by an object that depends only on that cell's symbol, not on its neighbors, so that the machine induces a monoid map onto the relation monoid. Once such a representation exists, one still needs a lower bound on its size that grows exponentially in hhh rather than polynomially.

Formalization scope

  • Letters are BRel (Fin h), a structure wrapping Fin h → Fin h → Prop, with the monoid structure given by path-order composition and identity Eq; OWL h is the set of words whose List.prod relates some pair.
  • Symbol Alpha adds left and right endmarkers; moves are left | stay | right; head positions are Fin (w.length + 2) and Move.Rel fixes the position change.
  • NMachine Alpha n and DMachine Alpha s have state types Fin n, Fin s, a set of accepting states, set-valued or Option-valued transitions, and boundary axioms forbidding left moves on the left endmarker and right moves on the right endmarker.
  • FiniteRun positive uses TransGen (at least one step) when positive = true and ReflTransGen otherwise; Recognizes positive L means acceptance exactly on L.
  • NoLeft forbids left moves in the source automaton. The recognizer clause quantifies over both conventions with the same machine.
  • The bound is in natural numbers with floor division (h - 2) / 31, and s ranges over all natural numbers (an s = 0 machine has no states and cannot recognize the language).
  • Infrastructure needed: Brauer (matching) diagram monoids with rank and idempotents, relation monoids, and the tour construction turning deterministic computations into diagrams. The diagram-monoid layer is reusable for other two-way automata lower bounds.

Selected references

  • M. O. Rabin, D. Scott, Finite automata and their decision problems, IBM J. Res. Develop. 3 (1959), 114–125.
  • J. C. Shepherdson, The reduction of two-way automata to one-way automata, IBM J. Res. Develop. 3 (1959), 198–200.
  • W. J. Sakoda, M. Sipser, Nondeterminism and the size of two way finite automata, STOC 1978, 275–286. https://doi.org/10.1145/800133.804357
  • M. Sipser, Lower bounds on the size of sweeping automata, J. Comput. System Sci. 21 (1980), 195–202.
  • M. Chrobak, Finite automata and unary languages, Theoret. Comput. Sci. 47 (1986), 149–158. https://doi.org/10.1016/0304-3975(86)90142-8
  • C. Kapoutsis, Nondeterminism is essential in small two-way finite automata with few reversals, Inform. and Comput. 222 (2013), 208–227.
  • C. A. Kapoutsis, Optimal 2DFA algorithms for one-way liveness on two and three symbols, LNCS 11011 (2018), 33–48.
  • K. Adeogun, C. Kapoutsis, A quadratic lower bound for 2DFAs against one-way liveness, preprint (2026). https://doi.org/10.48550/arXiv.2602.24279
  • R. Brauer, On algebras which are connected with the semisimple continuous groups, Ann. of Math. 38 (1937), 857–872.
  • OpenAI, An exponential state lower bound for two-way nondeterministic complementation, OpenAI Math Release preprint, September 25, 2026. https://github.com/openai/math/blob/adc7f1241b42e322a6451854ab7e4b4c146bf78a/preprints/An-exponential-state-lower-bound-for-two-way-nondeterministic-complementation-September-25-2026/paper.pdf
  • OpenAI, An exponential two-way deterministic state lower bound for one-way liveness, OpenAI Math Release preprint, September 25, 2026 (source of the goal; Theorem 1.1, p. 2). https://github.com/openai/math/blob/adc7f1241b42e322a6451854ab7e4b4c146bf78a/preprints/An-exponential-two-way-deterministic-state-lower-bound-for-one-way-liveness-September-25-2026/main.pdf
18 thms2 active usersReviewed
🏆Completed
CombinatoricsTheoretical Computer Science·Captain: wurtle

A Polynomial-Time 2-Approximation for Shortest Common SuperstringResearch Paper

Motivation: approximating the shortest common superstring

Given a finite collection S\mathcal SS of strings, the shortest common superstring (SCS) problem asks for a shortest string TTT that contains every member of S\mathcal SS as a contiguous substring. It is a basic model of sequence assembly (reconstructing a long DNA sequence from overlapping fragments) and of data compression, and it is a standard test problem for approximation algorithms. Computing the optimum exactly is hard — Blum, Jiang, Li, Tromp and Yannakakis showed the problem is MAX SNP-hard, so it has no polynomial-time approximation scheme unless P = NP (Blum et al., J. ACM 1994) — so the natural question is the best constant factor achievable in polynomial time. For decades the target has been factor 222: the classical Greedy conjecture asserts that repeatedly merging the pair with maximum overlap achieves it.

Timeline

  • 1988 — Tarhio and Ukkonen propose the factor-222 conjecture for the maximum-overlap Greedy procedure (Theor. Comput. Sci. 1988).
  • 1994 — Blum, Jiang, Li, Tromp and Yannakakis give the first constant-factor algorithm a polynomial-time 333-approximation (J. ACM 1994).
  • 1997 — Breslauer, Jiang and Jiang use rotations of periodic strings to control overlaps (J. Algorithms 1997).
  • 1999 — Sweedyk obtains factor 5/25/25/2 (SIAM J. Comput. 1999).
  • 2013 — Mucha obtains 2+11/232+11/232+11/23 via Lyndon words (SODA 2013).
  • 2019/2020 — Golovnev, Kulikov, Logunov, Mihajlin and Nikolaev introduce the all-substrings hierarchical graph and the Collapsing conjecture (APPROX/RANDOM 2019; arXiv:1809.08669).
  • 2023 — Englert, Matsakis and Veselý obtain (14+67)/9<2.466(14+\sqrt{67})/9<2.466(14+67​)/9<2.466 (ISAAC 2023).
  • 2026 — Chukhin, Kulikov, Mihajlin and Smal report a 7/37/37/3-approximation (ECCC TR26-157); Shibata reports a counterexample to the Greedy conjecture with ratio at least 9/49/49/4 (arXiv:2609.01365).
  • 2026 — An OpenAI preprint, A Polynomial-Time 2-Approximation for Shortest Common Superstring (OpenAI Math Release, September 24, 2026), claims a deterministic polynomial-time algorithm with ratio 222. The preprint has not been peer reviewed and its theorem is not formally verified.

Setting

A string is a finite list of symbols; a string sss is a substring of TTT if T=usvT=usvT=usv for some strings u,vu,vu,v (the empty string is a substring of everything). For a finite list S\mathcal SS of strings, a common superstring is a string TTT having every s∈Ss\in\mathcal Ss∈S as a substring, and

OPT(S)=min⁡{∣T∣:T is a common superstring of S},\mathrm{OPT}(\mathcal S)=\min\{|T| : T \text{ is a common superstring of }\mathcal S\},OPT(S)=min{∣T∣:T is a common superstring of S},

where ∣T∣|T|∣T∣ counts symbols. The alphabet is unbounded: each symbol is given by an explicit binary label, and the input is measured by its total encoded bit length NNN. An algorithm runs in polynomial time if its running time on a bit-level machine is bounded by a polynomial in NNN.

In Lean (namespace OAI.Superstring), a symbol is List Bool, a word is a list of symbols, and an instance is a list of words. Substring containment is Mathlib's infix relation <:+:, opt S is the infimum of lengths of common superstrings, and HasPolynomialImplementation f asks for a Mathlib Turing.TM2ComputableInPolyTime certificate for f with respect to an explicit self-delimiting binary encoding of instances and outputs, with every stack alphabet finite.

Formalization targets

Goal: a polynomial-time 2-approximation (Theorem 1.1)

There is a function fff from instances to words, computable in polynomial time in the encoded input length, such that for every instance S\mathcal SS

f(S) is a common superstring of Sand∣f(S)∣≤2 OPT(S).f(\mathcal S)\ \text{is a common superstring of } \mathcal S\qquad\text{and}\qquad |f(\mathcal S)|\le 2\,\mathrm{OPT}(\mathcal S).f(S) is a common superstring of Sand∣f(S)∣≤2OPT(S).

The goal is published on the platform with status Open.

Significance

The result itself. Factor 222 has been the conjectured benchmark for SCS since 1988, and the best proved polynomial-time ratio before this preprint was 7/37/37/3 (reported in 2026). Theorem 1.1 reaches factor 222 with a new algorithm rather than with Greedy, which a 2026 preprint reports does not achieve factor 222.

Formalizing it. The statement mixes three components that are each easy to get subtly wrong on paper: the correctness of the output (contiguous coverage of every input), the ratio against the unrestricted optimum, and a running-time bound measured in bits, with symbols of arbitrary label length. The formal statement pins down all three with an explicit machine model. A complete formalization requires both the combinatorial ratio proof and a verified polynomial-time implementation on Mathlib's stack-machine model, which is substantial reusable infrastructure for complexity statements in Lean.

Difficulty

The classical approach builds the overlap (distance) graph, takes a minimum-weight cycle cover as a lower bound, and then opens and concatenates the cycles; the cost of joining cycles is what has kept ratios above 222. Greedy, the natural candidate for factor 222, is reported to fail. Any factor-222 algorithm needs a lower bound on OPT\mathrm{OPT}OPT that is tight enough that the cost of connecting all the pieces can be charged to the lower bound a second time without double-counting, uniformly over periodic and highly repetitive inputs. The polynomial running time must also be established at the bit level, with symbols of unbounded encoded length.

Formalization scope

  • Instances are List (List (List Bool)): duplicates and empty strings are allowed, and symbols are arbitrary bit lists, so the alphabet is unbounded.
  • Coverage is contiguous substring containment (List.IsInfix), not subsequence containment.
  • opt is a natural-number sInf; the set is nonempty because the concatenation of all inputs is a common superstring, so the junk value is never used. The ratio is measured in symbols.
  • Polynomial time uses Mathlib's Turing.TM2ComputableInPolyTime for the encodings encodeInstance and encodeWord (prefix markers for list structure), together with finiteness of every stack alphabet, so the time bound is in input bits on a finite-alphabet machine. A function that is merely computable, or polynomial only in the number of symbols, does not satisfy the goal.
  • Needed infrastructure: the all-substrings graph and Euler tours, periodicity lemmas (Fine–Wilf), and a verified implementation on Mathlib's TM2 model. Contributions of reusable TM2 programming libraries are welcome.

Selected references

  • J. Tarhio and E. Ukkonen, A greedy approximation algorithm for constructing shortest common superstrings, Theor. Comput. Sci. 57 (1988), 131–145. https://doi.org/10.1016/0304-3975(88)90167-3
  • A. Blum, T. Jiang, M. Li, J. Tromp and M. Yannakakis, Linear approximation of shortest superstrings, J. ACM 41 (1994), 630–647. https://doi.org/10.1145/179812.179818
  • D. Breslauer, T. Jiang and Z. Jiang, Rotations of periodic strings and short superstrings, J. Algorithms 24 (1997), 340–353. https://doi.org/10.1006/jagm.1997.0861
  • Z. Sweedyk, A 2.5-approximation algorithm for shortest superstring, SIAM J. Comput. 29 (1999), 954–986. https://doi.org/10.1137/S0097539796324661
  • M. Mucha, Lyndon words and short superstrings, SODA 2013, 958–972. https://doi.org/10.1137/1.9781611973105.69
  • A. Golovnev, A. S. Kulikov, A. Logunov, I. Mihajlin and M. Nikolaev, Collapsing superstring conjecture, APPROX/RANDOM 2019. https://doi.org/10.4230/LIPIcs.APPROX-RANDOM.2019.26
  • M. Englert, N. Matsakis and P. Veselý, Approximation guarantees for shortest superstrings: simpler and better, ISAAC 2023. https://doi.org/10.4230/LIPIcs.ISAAC.2023.29
  • N. Chukhin, A. S. Kulikov, I. Mihajlin and A. Smal, A tight cycle-cover inequality for shortest common superstring, ECCC TR26-157, 2026. https://eccc.weizmann.ac.il/report/2026/157/
  • H. Shibata, Disproving the greedy superstring conjecture, arXiv:2609.01365, 2026. https://arxiv.org/abs/2609.01365v1
  • OpenAI, A Polynomial-Time 2-Approximation for Shortest Common Superstring, OpenAI Math Release preprint, September 24, 2026 (Theorem 1.1, p. 1). https://github.com/openai/math/blob/adc7f1241b42e322a6451854ab7e4b4c146bf78a/preprints/A-Polynomial-Time-2-Approximation-for-Shortest-Common-Superstring-September-24-2026/paper.pdf
2 thms2 active usersReviewed
ProbabilityQuantum InformationTheoretical Computer Science·Captain: mikedeng1

Strengths and Weaknesses of Quantum Computing II: Relative to a Random Permutation, a Quantum Algorithm Needs Ω(2^{n/3}) Queries to Decide the First Bit of A⁻¹(1ⁿ)Research Paper

Motivation

An oracle lets an algorithm ask for the value of a function without giving it a description of that function. An oracle lower bound isolates how many questions are needed, even when computation between questions is unrestricted. Bennett, Bernstein, Brassard and Vazirani used this viewpoint to compare classical and quantum access to randomly chosen functions and permutations. Their Theorem 3.6 states an almost-sure separation: relative to a random permutation oracle, a quantum machine using time o(2n/3)o(2^{n/3})o(2n/3) cannot capture all of (NP∩coNP)A(\mathrm{NP}\cap\mathrm{coNP})^A(NP∩coNP)A BBBV, Theorem 3.6. The same paper notes that its exponent 1/31/31/3 may reflect the available argument rather than the true boundary BBBV, p. 12.

The finite statement behind that separation is a quantum query lower bound for a uniformly random permutation. It asks about one bit of the inverse image of a fixed string, rather than inversion of the whole permutation. This gives a sharply defined target with the paper's explicit query and error constants.

Setting

Let Σn={0,1}n\Sigma^n=\{0,1\}^nΣn={0,1}n, and let AAA be a permutation oracle of Σn\Sigma^nΣn. Its inverse image of the all-one string 1n1^n1n exists and is unique. Define the language LA\mathcal L_ALA​ by putting yyy in LA\mathcal L_ALA​ when the first bit of A−1(y)A^{-1}(y)A−1(y) is 111. The paper observes that access to AAA puts this language in both NPA\mathrm{NP}^ANPA and coNPA\mathrm{coNP}^AcoNPA: either possible bit of the inverse image can be witnessed by an input to the permutation BBBV, p. 10. This mission tests membership only at y=1ny=1^ny=1n.

A TTT-query quantum algorithm starts in a unit vector of a finite complex Hilbert space. A basis configuration records an optional length-nnn query string, an answer register, and finite workspace. On a querying configuration (x,z,w)(x,z,w)(x,z,w), an oracle operation replaces the answer register by z⊕A(x)z\oplus A(x)z⊕A(x); it leaves a nonquerying configuration unchanged. Each oracle operation is followed by an algorithm-chosen unitary. The final acceptance probability is the sum of the squared amplitudes of accepting basis configurations. A run decides a yes-or-no question if its acceptance probability is at least 2/32/32/3 on a yes instance and at most 1/31/31/3 on a no instance. These thresholds match the paper's bounded-error convention BBBV, §2 and §3.1.

For a state ϕ\phiϕ, the query magnitude qy(ϕ)q_y(\phi)qy​(ϕ) is the total squared amplitude of basis configurations querying string yyy. Thus 0≤qy(ϕ)≤10\le q_y(\phi)\le10≤qy​(ϕ)≤1 for a unit state, and the magnitudes over all strings sum to at most 111. This is Definition 3.2 of the paper BBBV, p. 7.

Formalization targets

The goal is the finite claim in the proof of Theorem 3.6. For every n≥1n\ge1n≥1, every finite-workspace algorithm MMM with TTT queries, and a uniform permutation AAA of Σn\Sigma^nΣn,

T≤2n/3100⟹Pr⁡A ⁣[M fails to decide whether the first bit of A−1(1n) is 1]≥18.T\le\frac{2^{n/3}}{100} \quad\Longrightarrow\quad \Pr_A\!\left[M\text{ fails to decide whether the first bit of }A^{-1}(1^n)\text{ is }1\right]\ge\frac18.T≤1002n/3​⟹APr​[M fails to decide whether the first bit of A−1(1n) is 1]≥81​.

The milestones follow the statements used in the paper: Theorem 3.1 relates Euclidean distance between final states to their observed distributions; Theorem 3.3 bounds the effect of changing answers at queried strings; and the proof of Theorem 3.6 establishes the uniform transposition chain, the 1/21/21/2 membership split, the expected query-magnitude estimate, and the 1/501/501/50 expected final-state distance BBBV, pp. 7–11. The source text of each milestone is recorded with its page in the mission's milestone list.

Significance

The result identifies a concrete task that cannot be decided reliably with too few quantum oracle calls for a positive fraction of length-nnn permutation oracles. The inverse image exists by definition, so the obstruction is the number of queries needed to learn its first bit. This is the quantitative part of the paper's relative separation for NP∩coNP\mathrm{NP}\cap\mathrm{coNP}NP∩coNP; the passage to an almost-sure statement over countably many machines is described in the paper but lies beyond this mission's formal goal BBBV, pp. 10–12.

The theorem is proved on paper and is not claimed here as an open mathematical problem. Formalizing it produces reusable finite-dimensional machinery for quantum queries: an XOR oracle as a unitary basis permutation, query magnitude, a time-dependent hybrid oracle, and counting-based probability over finite permutation spaces. The milestone statements remain open Lean proof obligations in this proposal.

Difficulty

A direct comparison of an algorithm's run under two nearby permutation oracles requires knowing that the algorithm seldom queries the locations where those oracles differ. The distinguished preimage A−1(1n)A^{-1}(1^n)A−1(1n) cannot simply be declared rarely queried: learning about it is exactly the task under study. The paper therefore studies a chain of random transpositions and a run whose oracle changes by step. The subtle point is that a string used in a later transposition is still uniform when the earlier query state is formed BBBV, pp. 10–11. The expected query-magnitude milestone states that point explicitly.

Formalization scope

The Lean carrier of strings is Fin(n)→Z/2Z\mathrm{Fin}(n)\to\mathbb Z/2\mathbb ZFin(n)→Z/2Z. Algorithms use finite configuration types and EuclideanSpace ℂ; the query register is optional, and none represents configurations that make no length-nnn query. An oracle answers with the full length-nnn value by XOR in the answer register. The paper describes a Boolean oracle, but one full-value query can be simulated with nnn Boolean queries and one Boolean query with two full-value queries. The latter direction transfers this lower bound to the paper's oracle at a changed constant. Answers at lengths other than nnn are fixed and absorbed into the inter-query unitaries, as in the paper's finite-length argument. The full oracle quantum Turing machine, complexity classes as Lean objects, and the almost-sure conclusion are outside the model.

Probabilities and expectations are finite counting fractions. The chain samples a uniform π0\pi_0π0​ and independent uniform x1,…,xT+1x_1,\ldots,x_{T+1}x1​,…,xT+1​, with x0=π0−1(1n)x_0=\pi_0^{-1}(1^n)x0​=π0−1​(1n); this has the same joint law as the paper's conditional sampling order. The recurrence πi+1=πi∘(xi xi+1)\pi_{i+1}=\pi_i\circ(x_i\ x_{i+1})πi+1​=πi​∘(xi​ xi+1​) uses the paper's composition direction. The first-bit test requires n≥1n\ge1n≥1, and any comparison to πT−1\pi_{T-1}πT−1​ requires T≥1T\ge1T≥1. The goal quantifies over every finite workspace and algorithm before sampling the permutation. A uniform function in place of a permutation, or a predetermined inverse image, would change the task and is excluded.

The paper's Theorem 3.3 prints a final-state bound ε\varepsilonε that fails for a one-query XOR example; the milestone states the valid 2ε2\varepsilon2ε bound. Its baseline oracle may vary by step because the proof of Theorem 3.6 uses the hybrid run. The explicit constants T≤2n/3/100T\le 2^{n/3}/100T≤2n/3/100, failure probability 1/81/81/8, and expected distance 1/501/501/50 remain as printed. General finite-dimensional unitary and probability lemmas, together with a proof of the time-dependent hybrid bound, are welcome contributions. The finite query model and transposition-chain definitions are reusable in other oracle lower bounds.

Selected references

  • C. H. Bennett, E. Bernstein, G. Brassard, and U. Vazirani, Strengths and weaknesses of quantum computing, SIAM Journal on Computing 26(5), 1997; arXiv:quant-ph/9701001v1.
10 thms2 active usersReviewed
Information TheoryProbabilityTheoretical Computer Science·Captain: mikedeng1

A Pseudorandom Generator from Any One-Way Function: One-Way Functions Exist If and Only If Pseudorandom Generators ExistResearch Paper

Motivation

Cryptographic protocols use random looking strings that can be reproduced from a shorter private seed. A pseudorandom generator provides those strings if no efficient observer can tell them from uniformly sampled bits. A one-way function provides a different kind of asymmetry: anyone can evaluate it efficiently, but no efficient algorithm can usually recover a preimage of a random output. Håstad, Impagliazzo, Levin, and Luby proved that these two existence questions are equivalent. Their result identifies average-case inversion hardness as sufficient for computational pseudorandomness, without assuming a particular candidate function or algebraic problem (Håstad et al., 1999).

The equivalence is known mathematically; this mission asks for a machine-checked account of its qualitative security claim. The original article develops stronger quantitative time–success comparisons. Those comparisons are relevant to concrete cryptography, but the existence equivalence can be stated through asymptotically negligible success probabilities. This distinction is fixed throughout the formalization.

Setting

At security parameter nnn, a function ensemble FFF receives a string from the Boolean cube {0,1}tn\{0,1\}^{t_n}{0,1}tn​ and returns one in {0,1}ℓn\{0,1\}^{\ell_n}{0,1}ℓn​. The lengths are polynomial-time parameters, and one uniform machine evaluates the family when given 1n1^n1n, separators, an optional unary integer advice value, and the input. The optional advice is polynomially bounded and may depend on nnn without being computable. The goal itself uses no advice.

An adversary is a polynomial-time bit-string machine with polynomially many independent random coins. For inversion it sees Fn(X)F_n(X)Fn​(X) for a uniform input XXX and may return any string x′x'x′ of the correct input length with Fn(x′)=Fn(X)F_n(x')=F_n(X)Fn​(x′)=Fn​(X). It need not recover the original XXX. The ensemble is one-way if that success probability is negligible for every such adversary. For distinguishing, the adversary accepts only by outputting the one-bit string [1][1][1]. Two laws are computationally indistinguishable if every adversary has negligible difference between its acceptance probabilities.

A generator GGG is pseudorandom when tn<ℓnt_n<\ell_ntn​<ℓn​ for positive nnn and its output law on a uniform tnt_ntn​-bit seed is computationally indistinguishable from the uniform law on ℓn\ell_nℓn​ bits. Strict stretch is essential: the identity map would otherwise satisfy the indistinguishability clause. All probabilities are finite counting fractions over Boolean cubes, so neither arbitrary measures nor a default value for an empty event space enters the definitions.

The intermediate results use Shannon entropy and order-two Rényi entropy, both in bits, and statistical distance, half the L1L^1L1 difference of finite laws. A universal hash family is pairwise independent in the exact sense of Definition 4.6: for distinct inputs x,x′x,x'x,x′ and any two prescribed outputs a,a′a,a'a,a′, the chance over the uniform key that both outputs occur is exactly the reciprocal of the square of the output-space size. The public hash key is retained in the extracted output (Håstad et al., 1999, §§2–4).

Formalization targets

Main equivalence

The goal is Theorem 6.3, on printed page 1387:

(∃F  OWF(F))⟺(∃G  PRG(G)).(\exists F\;\mathrm{OWF}(F))\quad\Longleftrightarrow\quad (\exists G\;\mathrm{PRG}(G)).(∃FOWF(F))⟺(∃GPRG(G)).

Existence is the stable target: it does not fix a particular generator or an output-stretch constant. The reverse direction is the one the article attributes to Levin; the forward direction passes through the article's false-entropy and pseudoentropy constructions.

Intermediate targets

Lemma 4.8 bounds the statistical distance of a universal hash output with its public key by 2−(e+1)2^{-(e+1)}2−(e+1) when the source has Rényi entropy at least mmm and the hash output has m−2em-2em−2e bits. Theorem 4.14 converts a pseudoentropy generator into a PRG using the integer lengths and repetition count in Construction 4.13. Lemma 6.4 compares the Shannon entropies of the two Section 6 output laws:

H(E)≥H(D)+10n2,H(\mathcal E)\ge H(\mathcal D)+10n^2,H(E)≥H(D)+10n2,

with kn≥125n3k_n\ge125n^3kn​≥125n3. Theorem 6.2 gives the resulting mildly nonuniform false-entropy generator from a one-way function. The cited Levin direction is a separate existential milestone. These statements are the mission's milestone list, in their attack order (Håstad et al., 1999, pp. 1378–1387).

Significance

The result supplies a general route from any suitable average-case hard inversion task to a source of computational pseudorandom bits. It means an existence theorem for one primitive also settles the existence question for the other; it does not claim that an arbitrary one-bit-stretch generator is itself one-way. The paper's intermediate entropy notions record how much unpredictability remains or appears when outputs have nonuniform statistical laws.

A completed formalization would make the definitions of adversary, negligible advantage, universal hashing, entropy, and the finite output laws independently reusable. The known paper proof is not yet represented by the theorem stubs in this proposal. Formalizing the cited reverse implication, the entropy comparisons, and the links between the paper's intermediate generator notions remains solver work. No claim is made that the quantitative reduction bounds have been machine checked.

Difficulty

The one-way hypothesis only limits efficient inversion of an output; it does not directly say that the output is close to uniform. In particular, the output may carry statistical structure even when finding a preimage is hard. A direct attempt to hash the output without accounting for its varying preimage sizes does not produce the claimed pseudorandom law. The Section 6 construction therefore involves a random index, a hash prefix, independent parity masks, and a second public hash key. Their exact sample spaces matter for both the entropy gap and computational indistinguishability (Håstad et al., 1999, §§4, 6).

Formalization scope

Bit strings are Levin2003.Bits, fixed-length cubes are Levin2003.cube, and polynomial time is Mathlib's TM2 notion through the published Levin2003_owf_model. That module's own OneWay predicate requires length preservation, which Definition 3.2 of this paper does not; this mission defines its own ensemble and one-way predicate. Uniform adversaries receive no advice in the goal. Their coin budgets are polynomially bounded, and their inputs expose the security parameter in unary. Those choices rule out the vacuous encoding with unrestricted adversaries or an invisible security parameter.

The §6 laws use a finite product seed. The printed (6.8) samples index blocks as bit strings of length ⌈log⁡n⌉\lceil\log n\rceil⌈logn⌉, but (6.3) defines f′f'f′ only for indices 0,…,n−10,\ldots,n-10,…,n−1 and (6.5) averages uniformly over that range. The formalization uses Fin n indices. Thus Theorem 6.2 is stated for the repaired finite seed law, with a polynomial-time evaluator on its binary encoding. This exposes a gap between that seed space and Definition 3.9's uniform bit seed; bridging it is part of a complete proof. The prefix of a hash output is padded for the binary law, while the index remains present so the padding loses no information. The real-valued mnm_nmn​ in (6.6) is rounded down to a bit length, and its integer value is the advice for Theorem 6.2. Construction 4.13 retains its printed floor and ceiling. Its pseudoentropy surplus sns_nsn​ may be real-valued, as in the paper; polynomial-time computability of sns_nsn​ is rendered as sn=pn/qns_n=p_n/q_nsn​=pn​/qn​ with natural numbers whose binary expansions are polynomial-time computable from 1n1^n1n.

Lemma 6.4 explicitly bounds approximate degeneracy by n−1n-1n−1, which its printed proof uses without stating. For a one-way function that bound is needed only eventually. Accordingly computational-entropy lower bounds are formulated for all sufficiently large positive nnn; the paper's Definition 3.7 says all nnn. Lemma 6.5's oracle-machine statement and its printed probability constant are left for a separate treatment. The model covers qualitative negligible success, not time–success ratios, expected-time bookkeeping, or labels such as weak-preserving. Contributions that close those quantitative gaps are welcome, as are proofs of the selected milestones and reusable finite-entropy lemmas.

Selected references

  • J. Håstad, R. Impagliazzo, L. A. Levin, and M. Luby, A Pseudorandom Generator from Any One-Way Function, SIAM Journal on Computing 28(4):1364–1396, 1999. DOI: 10.1137/S0097539793244708.
9 thms2 active usersReviewed
Dynamic ProgrammingOperations ResearchOptimization·Captain: mikedeng1

The Complexity of Markov Decision Processes I: A Boolean Circuit Is True Iff Its Finite-Horizon Markov Decision Process Has Optimal Expected Cost ZeroResearch Paper

Motivation

Markov decision processes are the standard model of sequential decision making under uncertainty in operations research, control and artificial intelligence. Their finite-horizon, discounted and average-cost versions are all solvable in polynomial time, by dynamic programming or linear programming. Papadimitriou and Tsitsiklis (The Complexity of Markov Decision Processes, Math. Oper. Res. 12(3), 1987) asked the next question: can these problems be solved fast in parallel, in polylogarithmic time on polynomially many processors (the class NC)? Their Theorem 1 answers no, unless every polynomial-time problem parallelizes: all three versions are P-complete. The proof is a short reduction from the circuit value problem (CVP), the canonical P-complete problem (Ladner, 1975). This mission formalizes the mathematical heart of that reduction: the constructed process has optimal expected cost zero exactly when the circuit evaluates to true.

The result is the first of a series of five missions on the same paper; the others treat the deterministic special cases (which are in NC) and the partially observed case (which is PSPACE-hard).

Setting

A circuit is a finite sequence of triples C=((ai,bi,ci), i=1,…,k)C=((a_i,b_i,c_i),\ i=1,\dots,k)C=((ai​,bi​,ci​), i=1,…,k) with k≥1k\ge1k≥1. Each aia_iai​ is one of the operations false, true, and, or. A triple with ai∈{false,true}a_i\in\{\text{false},\text{true}\}ai​∈{false,true} is an input; a triple with ai∈{and,or}a_i\in\{\text{and},\text{or}\}ai​∈{and,or} is a gate, and a gate reads two earlier triples, 1≤bi,ci<i1\le b_i,c_i<i1≤bi​,ci​<i. The value of an input is its constant; the value of a gate is the Boolean operation aia_iai​ applied to the values of triples bib_ibi​ and cic_ici​. The value of CCC is the value of triple kkk.

A finite Markov decision process has a finite state set SSS, an initial state s0s_0s0​, and at each state sss a nonempty finite set DsD_sDs​ of decisions. Taking decision iii at state sss at time ttt costs c(s,i,t)c(s,i,t)c(s,i,t) and moves to s′s's′ with probability p(s,s′,i,t)p(s,s',i,t)p(s,s′,i,t). A policy δ\deltaδ chooses a decision δ(s,t)∈Ds\delta(s,t)\in D_sδ(s,t)∈Ds​ for every state and time. For a horizon TTT the expected cost of δ\deltaδ is

JT(δ)=Eδ[∑t=0Tc(st,δ(st,t),t)],J_T(\delta)=\mathbb E_\delta\Bigl[\sum_{t=0}^{T}c\bigl(s_t,\delta(s_t,t),t\bigr)\Bigr],JT​(δ)=Eδ​[t=0∑T​c(st​,δ(st​,t),t)],

and the optimal expected cost is JT∗=inf⁡δJT(δ)J_T^\ast=\inf_\delta J_T(\delta)JT∗​=infδ​JT​(δ) over all policies.

From a circuit CCC the proof of Theorem 1 builds a stationary process MCM_CMC​: one state per triple plus an absorbing state qqq. An input state has one decision, moving to qqq, at cost 111 if the input is false and 000 if it is true. An or gate has two free decisions, 000 moving to bib_ibi​ and 111 moving to cic_ici​. An and gate has one decision, moving to bib_ibi​ or cic_ici​ with probability 1/21/21/2 each. All other costs are zero. The initial state is triple kkk and the horizon is T=kT=kT=k.

In the Lean development these are MDPComplexity.CircuitValue.MDP (with Policy, trajProb, expCost, optCost), Circuit (with val, value, lastIdx) and Circuit.toMDP.

Formalization targets

Goal: correctness of the reduction

Jk∗(MC)≤0⟺the value of C is true,J_k^\ast(M_C)\le 0\quad\Longleftrightarrow\quad \text{the value of } C \text{ is true},Jk∗​(MC​)≤0⟺the value of C is true,

for every circuit CCC with k≥1k\ge1k≥1 triples (theorem1_reduction). This is the claim "the optimum expected cost is 0 or less iff the value of CCC was true" on p. 445.

Milestones

  1. Every policy has Jk(δ)≥0J_k(\delta)\ge0Jk​(δ)≥0 ("it cannot be less").
  2. For any finite process with nonnegative costs, JT(δ)=0J_T(\delta)=0JT​(δ)=0 iff no positive cost is incurred on any trajectory of positive probability ("the states with positive costs are impossible to reach").
  3. If some policy has Jk(δ)=0J_k(\delta)=0Jk​(δ)=0, then CCC is true.
  4. If CCC is true, some policy has Jk(δ)=0J_k(\delta)=0Jk​(δ)=0.

Further statement

The discounted analogue, which the paper asserts in one sentence: for β∈(0,1)\beta\in(0,1)β∈(0,1), the optimal expected discounted cost inf⁡δ∑t≥0βt Eδ[c(st,δ(st,t))]\inf_\delta\sum_{t\ge0}\beta^t\,\mathbb E_\delta[c(s_t,\delta(s_t,t))]infδ​∑t≥0​βtEδ​[c(st​,δ(st​,t))] of MCM_CMC​ is at most 000 iff CCC is true (discounted_reduction).

Significance

Combined with the log-space computability of MCM_CMC​ from CCC, the goal shows that the finite-horizon Markov decision problem is P-hard, hence not in NC unless P = NC. This is the reason dynamic programming for general Markov decision processes is believed to be inherently sequential, and it frames the contrast drawn in the rest of the paper: deterministic processes are in NC, and partially observed ones are PSPACE-hard. The construction is also stationary, so it shows that even the stationary finite-horizon problem is P-hard, as the paper remarks.

The theorem is proved in the paper, in one paragraph; it is not open. To our knowledge no machine-checked proof of it exists. Formalizing it produces a precise statement of what the reduction proves, a reusable finite-trajectory model of finite-horizon Markov decision processes with time-dependent policies, and a reusable encoding of Boolean circuits and their value.

Difficulty

The informal argument is short, but two points need care. First, the optimum ranges over all time-dependent policies, which form an infinite type; the infimum is attained only because the expected cost depends on finitely many decisions, and this must be established rather than assumed. Second, the passage from "the expected cost is zero" to "the circuit is true" turns a statement about trajectory probabilities into a statement about the recursive value of the circuit, and the and gates (where both successors are reached with positive probability) behave differently from the or gates (where only the chosen successor is reached). Reasoning only along a single path, as for a deterministic process, does not suffice at and gates.

Formalization scope

Theorem 1 is a complexity statement: "The Markov Decision Process problem is P-complete in all three cases." Membership in P, the log-space computability of the construction, the notion of P-completeness, and the average-cost case are not part of this mission; Mathlib has no log-space reductions or class P. The mission states the correctness of the reduction, for the exact process of the proof.

Conventions committed to:

  • Triples are indexed by Fin k (paper index iii is Lean index i−1i-1i−1); k≥1k\ge1k≥1 via [NeZero k]. The fields bi,cib_i,c_ibi​,ci​ of an input are unused (the paper sets them to 000, not a triple index). Triple kkk need not be a gate.
  • States of MCM_CMC​ are Option (Fin k), with none the state qqq; decisions are Fin 2 at or gates and Fin 1 elsewhere.
  • The and-gate row is 121[s′=bi]+121[s′=ci]\tfrac12\mathbf 1[s'=b_i]+\tfrac12\mathbf 1[s'=c_i]21​1[s′=bi​]+21​1[s′=ci​], so that bi=cib_i=c_ibi​=ci​ (allowed by the paper) gives a probability vector.
  • Rows of the transition kernel are probability vectors and every DsD_sDs​ is nonempty (implicit in the paper, explicit here).
  • The expected cost is a finite sum over trajectories, with the §2 horizon convention ∑t=0T\sum_{t=0}^{T}∑t=0T​ and T=kT=kT=k. The optimum is a real infimum over all policies δ(s,t)\delta(s,t)δ(s,t), not over stationary ones.
  • The discounted cost is ∑tβt\sum_t\beta^t∑t​βt times the expected time-ttt cost, which equals the expectation of the discounted series for bounded nonnegative costs.

A formalization that defines the optimal cost directly as "the cost of the best choice at the or gates", or optimizes only over stationary policies, would make the goal a restatement of the circuit's value; the mission's optimum is over all time-dependent policies of the general model.

Contributions welcome: proofs of the milestones and the goal; lemmas on the trajectory model (marginalization of trajProb, attainment of the infimum for policies that matter only up to time TTT) that are reusable for any finite-horizon process.

Selected references

  • C. H. Papadimitriou and J. N. Tsitsiklis, The Complexity of Markov Decision Processes, Mathematics of Operations Research 12(3) (1987) 441–450. https://doi.org/10.1287/moor.12.3.441
  • R. E. Ladner, The Circuit Value Problem is Log Space Complete for P, SIGACT News 7(1) (1975) 18–20. https://doi.org/10.1145/990518.990519
  • M. L. Puterman, Markov Decision Processes: Discrete Stochastic Dynamic Programming, Wiley, 1994. https://doi.org/10.1002/9780470316887
8 thms2 active usersReviewed
🏆Completed
Theoretical Computer Science·Captain: hao jia

Weighted Falsifiability of Unambiguous DNFsOpen Problem

Motivation

A disjunctive normal form (DNF) is a disjunction of terms, each term a conjunction of Boolean literals. An unambiguous DNF has pairwise disjoint terms: no Boolean assignment satisfies two different terms. This restriction makes several tasks easy. In particular, the cited open-problem entry records polynomial-time algorithms for weighted satisfiability on unambiguous DNFs and for unweighted falsifiability. The unresolved boundary is weighted falsifiability: can one find a high-weight assignment outside the union of the terms, without enumerating all assignments?

This question is relevant to the complexity of negating compact representations of Boolean functions. Amarilli's entry observes that, since unambiguous DNFs are d-DNNFs, a polynomial-time negation procedure for d-DNNFs would yield a polynomial-time solution to weighted falsifiability on unambiguous DNFs by applying weighted satisfiability to the negated representation. Conversely, if weighted falsifiability for unambiguous DNFs—or even for d-DNNFs—is NP-hard, then, unless P=NP\mathrm{P}=\mathrm{NP}P=NP, d-DNNFs cannot be negated in polynomial time. These are implications stated by the source, not results established by this mission.

Historical note

The question appears on Albertine Amarilli's open-problem list. The entry cites a Theoretical Computer Science Stack Exchange question by Mikaël Monet and credits him with helping prepare the entry. It records two partial tractability results: weighted satisfiability with binary weights, and weighted falsifiability when the variable weights are unary. The entry gives no date for when the problem was posed or last seen open, so this description does not assign one. The linked discussion is useful context, not a novelty or resolution certificate.

Setting

Let X={x0,…,xn−1}X=\{x_0,\ldots,x_{n-1}\}X={x0​,…,xn−1​} be exactly the variables occurring in the DNF. The formal input declares this finite universe by its size nnn; validity requires every declared variable to occur in at least one literal. An input DNF is a finite list of terms, and each term is a finite list of signed variable indices. An assignment is a function ν:X→{0,1}\nu:X\to\{0,1\}ν:X→{0,1}. A positive literal xix_ixi​ is true when ν(xi)=1\nu(x_i)=1ν(xi​)=1; a negative literal ¬xi\neg x_i¬xi​ is true when ν(xi)=0\nu(x_i)=0ν(xi​)=0. A term is true when all its literals are true, and the DNF is true when at least one term is true. Thus the empty DNF is false and an empty term is true.

The input also contains one positive integer weight cic_ici​ for each variable and a positive threshold ttt. The weight of an assignment is

w(ν)=∑i=0n−1ciν(xi).w(\nu)=\sum_{i=0}^{n-1} c_i\nu(x_i).w(ν)=i=0∑n−1​ci​ν(xi​).

The DNF is valid for this problem when every literal index is below nnn and every pair of distinct terms is mutually unsatisfiable. The decision question is whether there exists an assignment that falsifies the DNF and has weight at least ttt.

Formalization targets

Partial result — weighted satisfiability

For valid unambiguous DNF inputs with positive binary-encoded weights and threshold, the decision problem asking whether a satisfying assignment has weight at least ttt has a deterministic polynomial-time algorithm. This is the weighted-satisfiability baseline recorded in the source entry. It is a separate result from the goal below.

Partial result — unary-weight falsifiability

For the same valid DNF model, when each variable weight is encoded in unary, weighted falsifiability has a deterministic polynomial-time algorithm. The threshold remains binary-encoded in this formalization. The source entry records this unary-weight restriction as tractable.

Goal — binary-weight falsifiability

For arbitrary positive binary-encoded variable weights and a positive binary-encoded threshold, determine whether one fixed deterministic Turing machine and one polynomial time bound decide weighted falsifiability for every valid input. The machine must be correct on all valid instances; its running-time bound is uniform and measured in the length of the explicit input code. No witness output is required.

The two partial results are reference points, not assumptions from which the goal is claimed to follow. The goal is the open binary-weight case; it must not be replaced by the satisfiability problem or by the unary-weight restriction.

Dependency graph

Input, semantics, validity, and serialization definitions
├── binary-weight weighted satisfiability (source baseline)
├── unary-weight weighted falsifiability (source baseline)
└── binary-weight weighted falsifiability (open goal)

Each branch shares the same finite DNF model and correctness predicates. The arrows indicate required definitions and scope, not a claim that either partial theorem implies the open goal.

Significance

A positive result would give a uniform algorithm for finding whether a disjoint union of Boolean subcubes omits any sufficiently heavy point, even when weights are represented compactly in binary. A negative complexity result would identify a sharp obstruction to efficient complement-related operations on these representations. The exact-weight version (asking for weight exactly ttt) is a different problem and is outside this mission's scope; the goal here is specifically weight at least ttt.

Formalizing the question fixes the universe of variables, signed-literal semantics, unambiguity condition, threshold direction, input encoding, and computational model. It also makes the unary and satisfiability baselines comparable to the binary-weight goal without treating a finite search or a candidate implementation as a complexity proof. No solution is supplied or presumed here.

Difficulty

The direct search over all 2n2^n2n assignments is exponential in nnn. Counting satisfying assignments is enough to settle ordinary falsifiability, but a weighted threshold partitions assignments by exponentially many possible total weights when the weights are binary. The unary dynamic program therefore does not by itself give a polynomial bound in the binary input length. Conversely, solving weighted satisfiability on the DNF does not answer whether a heavy assignment lies outside it. The task is to settle that gap without silently replacing binary magnitude by unary size.

Formalization scope

The Lean model uses Input, an explicit variable count, a list of signed-index terms, a weight list, and a threshold. validInput requires exactly one positive weight per variable, positive weights and threshold, in-range literal indices, that every declared variable occurs in the formula, and pairwise unsatisfiable distinct terms. The finite assignment type is Fin n → Bool; unused variables remain part of the input. The decision predicate is existential and exact.

The binary code includes the variable count, every list count and delimiter, all signed indices, all weights, and the threshold. Natural-number payloads use canonical binary digits with an explicit unary length prefix. The unary-weight code changes only the variable-weight payloads to unary; formula data and threshold remain binary. The complexity statements use Mathlib's bundled deterministic Turing.TM2ComputableInPolyTime model, with a single machine and polynomial bound in the chosen code length. This formalization does not claim refinement to JSON, CPython, or any runtime implementation. Human review should check the encoding/decoder round-trip and the source correspondence before launch.

The reusable definitions are the finite DNF semantics, weighted assignment score, validity predicate, binary/unary encoders, and exact decision predicates. Formalizing the two cited partial results is part of the mission's milestone path; neither is evidence that the binary-weight goal is solved.

Selected references

  • Albertine Amarilli, “Weighted falsifiability for unambiguous DNFs,” List of open questions in theoretical computer science, https://a3nm.net/work/research/questions/#weighted-falsifiability-for-unambiguous-dnfs.
  • Mikaël Monet, “Is this problem on unambiguous DNFs hard?”, Theoretical Computer Science Stack Exchange, https://cstheory.stackexchange.com/questions/53733/is-this-problem-on-unambiguous-dnfs-hard.
4 thms2 active usersReviewed
🏆Completed
Algorithmic Game TheoryOperations Research·Captain: mikedeng1

Market Equilibrium under Separable, Piecewise-Linear, Concave Utilities II: An Exact 3-Cover Exists iff the Constructed Market Has an EquilibriumResearch Paper

Motivation

Market equilibrium is the central solution concept of general equilibrium theory: prices at which every agent buys a utility-maximizing bundle and supply meets demand. Arrow and Debreu (1954) proved that equilibria exist under mild conditions on endowments and utilities, and a line of work in algorithmic game theory asks how hard it is to compute them. For linear utilities an equilibrium can be computed in polynomial time, and there is an efficiently checkable condition for its existence. The next natural class, additively separable piecewise-linear concave utilities, models diminishing marginal utility and is the class most used in applications.

Vazirani and Yannakakis (J. ACM 58(3), 2011) settle the complexity of this class. They show that equilibria are rational whenever they exist (Theorems 4.1 and 5.1), that computing an equilibrium under the standard sufficient conditions is PPAD-complete (Theorems 6.1 and 7.1, building on Chen, Dai, Du and Teng 2009), and — the subject of this mission — that deciding whether an equilibrium exists at all is NP-complete (Theorem 8.1). The hardness half rests on an explicit construction: from an instance of Exact Cover by 3-Sets, a market whose equilibria encode exact covers.

Setting

An Arrow–Debreu market has a finite set BBB of agents and a finite set GGG of divisible goods. Agent iii owns an endowment wij≥0w_{ij}\ge 0wij​≥0 of each good jjj and has utility ui(y)=∑jfji(yj)u_i(y)=\sum_{j} f^i_j(y_j)ui​(y)=∑j​fji​(yj​), where each fjif^i_jfji​ is a piecewise-linear concave utility function: slopes c1≥c2≥⋯≥cm>0c_1\ge c_2\ge\dots\ge c_m>0c1​≥c2​≥⋯≥cm​>0 on consecutive pieces of lengths a1,…,ama_1,\dots,a_ma1​,…,am​, followed by a last piece of slope t∈[0,cm]t\in[0,c_m]t∈[0,cm​] until infinity (t=0t=0t=0 means the function goes flat).

At prices ppp, agent iii's income is ∑jpjwij\sum_j p_j w_{ij}∑j​pj​wij​. An optimal bundle is an affordable y≥0y\ge 0y≥0 maximizing uiu_iui​ among affordable bundles, bought only along the pieces of fjif^i_jfji​ that carry utility. A price equilibrium is a price vector ppp in the unit simplex (p≥0p\ge 0p≥0, ∑jpj=1\sum_j p_j=1∑j​pj​=1) together with an allocation of optimal bundles such that ∑ixij=∑iwij\sum_i x_{ij}=\sum_i w_{ij}∑i​xij​=∑i​wij​ for every good jjj. It is an ϵ\epsilonϵ-approximate equilibrium if instead ∣∑ixij−∑iwij∣≤ϵ∑iwij|\sum_i x_{ij}-\sum_i w_{ij}|\le\epsilon\sum_i w_{ij}∣∑i​xij​−∑i​wij​∣≤ϵ∑i​wij​ for every jjj.

An X3C instance is a family C=(C1,…,Cn)\mathcal C=(C_1,\dots,C_n)C=(C1​,…,Cn​) of 333-element subsets of X={x1,…,xn}X=\{x_1,\dots,x_n\}X={x1​,…,xn​}; an exact cover is a subfamily in which every element of XXX lies in exactly one set. Following the paper, nnn is a multiple of 333, n>35n>35n>35, and ⋃iCi=X\bigcup_i C_i=X⋃i​Ci​=X.

The market D(C)D(\mathcal C)D(C) has 2n+12n+12n+1 goods (good 000, goods CiC_iCi​, goods xjx_jxj​) and 2n+22n+22n+2 agents, with e0=n3e_0=n^3e0​=n3:

  1. agent 000 owns e0e_0e0​ units of every good; his utility for every good has slope 222 up to e0e_0e0​ units and slope 111 beyond;
  2. agent CiC_iCi​ owns one unit of good CiC_iCi​; segments of slope 111, length 1/21/21/2 for good 000, slope 1/31/31/3, length 1/61/61/6 for each good xj∈Cix_j\in C_ixj​∈Ci​, slope 1/91/91/9, length 1/41/41/4 for good CiC_iCi​;
  3. agent xjx_jxj​ owns 1/61/61/6 unit of good xjx_jxj​; one segment of slope 111, length 1/121/121/12 for good 000;
  4. the extra agent owns n/2n/2n/2 units of good 000; one segment of slope 111, length 3/43/43/4 for each good CiC_iCi​.

All other utility functions are flat.

Formalization targets

Goal: the reduction statement

C has an exact cover  ⟺  D(C) has an equilibrium  ⟺  D(C) has an n−5-approximate equilibrium.\mathcal C\ \text{has an exact cover}\iff D(\mathcal C)\ \text{has an equilibrium}\iff D(\mathcal C)\ \text{has an } n^{-5}\text{-approximate equilibrium}.C has an exact cover⟺D(C) has an equilibrium⟺D(C) has an n−5-approximate equilibrium.

This is the mathematical content of the NP-hardness half of Theorem 8.1, assembled by the paper from Lemmas 8.2 and 8.3.

Milestones

  1. Lemma 8.2: an exact cover yields an equilibrium of D(C)D(\mathcal C)D(C).
  2. Lemma 7.2, as applied in Lemma 8.3: in an n−5n^{-5}n−5-approximate equilibrium of D(C)D(\mathcal C)D(C) all prices are positive and within a factor 222 of each other.
  3. Claims 8.4–8.7: in such an equilibrium, with pmp_mpm​ the minimum price, p(0)=2pmp(0)=2p_mp(0)=2pm​; p(xj)<2pmp(x_j)<2p_mp(xj​)<2pm​; with S={i:p(Ci)≥pm+16∑xj∈Cip(xj)}S=\{i: p(C_i)\ge p_m+\tfrac16\sum_{x_j\in C_i}p(x_j)\}S={i:p(Ci​)≥pm​+61​∑xj​∈Ci​​p(xj​)}, every i∉Si\notin Si∈/S has p(Ci)=pmp(C_i)=p_mp(Ci​)=pm​; and the sets indexed by SSS are pairwise disjoint.
  4. Lemma 8.3: an equilibrium, or an n−5n^{-5}n−5-approximate equilibrium, of D(C)D(\mathcal C)D(C) yields an exact cover.

Significance

Theorem 8.1 shows that there is no efficiently checkable necessary and sufficient condition for the existence of an equilibrium in piecewise-linear concave markets unless P = NP, in contrast with the linear case. Together with the PPAD results of the same paper it separates two questions: under the classical sufficient conditions an equilibrium exists and finding one is PPAD-complete; without them, even deciding existence is NP-hard, and remains so for n−5n^{-5}n−5-approximate equilibria. The construction illustrates the technique the paper uses for both of its negative results: well-chosen piecewise-linear pieces make an agent buy a segment wholly or not at all, depending on how prices compare, which gives the equilibrium problem a discrete character.

The result is proved in the paper. To our knowledge no part of it has been machine-checked. This mission formalizes the reduction's correctness at full strength, including the approximate version, which requires quantitative control of clearing errors that the exact version does not.

Difficulty

The direction "exact cover ⇒ equilibrium" is an explicit verification: prescribed prices and an allocation, and a check that every bundle is optimal, which for separable piecewise-linear utilities is a bang-per-buck comparison. The converse is the substantial part. An arbitrary approximate equilibrium must be shown to have the rigid price structure — every price in [pm,2pm][p_m,2p_m][pm​,2pm​], good 000 at 2pm2p_m2pm​, unused sets at pmp_mpm​ — before a counting argument on agent 000's savings forces ∣S∣=n/3|S|=n/3∣S∣=n/3. Each step is an excess-demand argument in which the error ϵ\epsilonϵ times the supply must be compared with quantities of order 1/n1/n1/n; this is where n>35n>35n>35 and the exponent 555 enter, and why the approximate statement does not follow from the exact one. Optimality of bundles is a statement about all affordable bundles, so each claim needs the structure of optimal bundles under piecewise-linear concave utility, which is not in Mathlib.

Formalization scope

Lean namespace PLCMarkets.ExactCover. Market data (endowments, slopes, lengths) are rationals; prices and allocations are reals. Goods and agents of D(C)D(\mathcal C)D(C) are small inductive types named as in the paper; indices are 0-based. Supplies are not normalized to 111, so clearing is ∑ixij=∑iwij\sum_i x_{ij}=\sum_i w_{ij}∑i​xij​=∑i​wij​; prices are normalized to the simplex in both equilibrium notions, as in the proof of Lemma 8.3. The family C\mathcal CC is indexed and may repeat a set; exact cover and the disjointness of SSS count indices.

Standing hypotheses of every theorem: each CiC_iCi​ has three elements, 3∣n3\mid n3∣n, n>35n>35n>35, ⋃iCi=X\bigcup_i C_i=X⋃i​Ci​=X — the paper's own "without loss of generality" assumptions (p. 10:19). Two conventions depart from the printed page, both necessary and both disclosed on the items:

  • Optimal bundles buy only utility-bearing pieces. If a utility function is flat beyond its segments, the agent does not buy beyond them. The paper uses this throughout (an agent "can only spend pm/6p_m/6pm​/6 on the single segment", Claim 8.5). Without it, agents can spend leftover income on goods worth nothing to them; then setting p(0)=2pmp(0)=2p_mp(0)=2pm​ and every other price to pmp_mpm​ gives an exact equilibrium of every D(C)D(\mathcal C)D(C), and the goal is false.
  • The extra agent's segments have length 3/43/43/4. The page prints 3n/43n/43n/4; every computation in the paper (Lemma 8.2, Claim 8.6, the end of Lemma 8.3) uses 3/43/43/4 per good, and with 3n/43n/43n/4 the prices of Lemma 8.2 are not an equilibrium.

Trivializing encodings are excluded: optimality of bundles is part of both equilibrium notions (without it the endowment itself clears every market), clearing is relative and per good, and the goal quantifies only over nnn and C\mathcal CC, with D(C)D(\mathcal C)D(C) an explicit function of C\mathcal CC.

Out of scope: the complexity-class statement "NP-complete" and NP membership (which comes from the rationality theorems of the companion mission); polynomial-time computability of the construction and string encodings of markets; and the Fisher market FFF of §8, whose half of Lemmas 8.2 and 8.3 is a natural follow-up. Reusable infrastructure: piecewise-linear concave utilities, Arrow–Debreu markets with exact and approximate equilibria, and bang-per-buck characterizations of optimal bundles. Contributions proving that characterization as a standalone lemma are welcome.

Selected references

  • V. V. Vazirani and M. Yannakakis, Market Equilibrium under Separable, Piecewise-Linear, Concave Utilities, Journal of the ACM 58(3), Article 10, 2011. https://doi.org/10.1145/1970392.1970394
  • X. Chen, D. Dai, Y. Du and S.-H. Teng, Settling the Complexity of Arrow–Debreu Equilibria in Markets with Additively Separable Utilities, Proceedings of the IEEE Symposium on Foundations of Computer Science (FOCS), 2009 (reference [Chen et al. 2009a] of the paper).
  • M. R. Garey and D. S. Johnson, Computers and Intractability: A Guide to the Theory of NP-Completeness, W. H. Freeman, 1979.
  • K. J. Arrow and G. Debreu, Existence of an Equilibrium for a Competitive Economy, Econometrica 22(3), 1954. https://doi.org/10.2307/1907353
12 thms2 active usersReviewed
🏆Completed
Algebraic GeometryRepresentation Theory·Captain: Lucas

Geometric Complexity Theory: No Occurrence ObstructionsResearch Paper

Motivation

The permanent versus determinant problem asks how large an nnn must be for the permanent of an m×mm \times mm×m matrix to be written as the determinant of an n×nn \times nn×n matrix whose entries are affine linear functions of the m2m^2m2 input variables. Valiant showed that some finite nnn always works and conjectured in 1979 that the least such nnn, the determinantal complexity dc(perm)\mathrm{dc}(\mathrm{per}_m)dc(perm​), grows faster than every polynomial in mmm; this is the algebraic counterpart of P≠NP\mathbf{P} \ne \mathbf{NP}P=NP.

Geometric complexity theory (GCT), proposed by Mulmuley and Sohoni in 2001, attacks the conjecture by replacing the two polynomials with the closures of their GL\mathrm{GL}GL-orbits and comparing the GL\mathrm{GL}GL-representations carried by the coordinate rings of those closures. If some irreducible representation occurs in the coordinate ring of the orbit closure of the padded permanent but not in that of the determinant, then one orbit closure cannot contain the other, and a lower bound on dc(perm)\mathrm{dc}(\mathrm{per}_m)dc(perm​) follows. Such a representation is an occurrence obstruction, and Mulmuley and Sohoni conjectured that occurrence obstructions exist for every polynomial padding.

Timeline of what is actually proved.

  • 1979: Valiant introduces dc\mathrm{dc}dc and conjectures superpolynomial growth (Valiant 1979).
  • 2001: Mulmuley and Sohoni restate the conjecture in terms of orbit closures and propose proving it by exhibiting occurrence obstructions (arXiv:cs/9908014).
  • 2014: Kadish and Landsberg show that any partition occurring for the padded permanent has at most m2m^2m2 rows and a first row of length at least (n−m)d(n-m)d(n−m)d (arXiv:1204.4772).
  • 2016: Ikenmeyer and Panova show that vanishing rectangular Kronecker coefficients cannot supply occurrence obstructions (arXiv:1512.03798).
  • 2016: Mulmuley, in GCT V, makes the "infinitesimally close approximation" form of the permanent hardness hypothesis the engine of derandomized Noether normalization (JAMS).
  • 2017: Ikenmeyer, Mulmuley and Walter prove that deciding positivity of Kronecker coefficients is NP\mathbf{NP}NP-hard and construct superpolynomially many vanishing triples in the Kronecker cone (comput. complex. 26).
  • 2019: Bürgisser, Ikenmeyer and Panova prove that no occurrence obstructions exist at all once n≥m25n \ge m^{25}n≥m25, refuting the Mulmuley–Sohoni conjecture on obstructions (JAMS 32).

Setting

Fix nnn and write V=Cn×nV = \mathbb{C}^{n \times n}V=Cn×n. A form of degree nnn is a homogeneous polynomial of degree nnn in the n2n^2n2 variables XijX_{ij}Xij​, 0≤i,j<n0 \le i, j < n0≤i,j<n; the space of such forms is SymnV∗\mathrm{Sym}^n V^*SymnV∗. The group GLn2\mathrm{GL}_{n^2}GLn2​ acts on it by linear substitution of the variables, and for a form ppp we write GLn2⋅p‾\overline{\mathrm{GL}_{n^2}\cdot p}GLn2​⋅p​ for the closure of its orbit in the Euclidean topology, which for these orbits agrees with the Zariski closure.

Two orbit closures matter:

Ωn  =  GLn2⋅det⁡n‾,Zn,m  =  GLn2⋅X00 n−mperm‾,\Omega_n \;=\; \overline{\mathrm{GL}_{n^2}\cdot \det\nolimits_n}, \qquad Z_{n,m} \;=\; \overline{\mathrm{GL}_{n^2}\cdot X_{00}^{\,n-m}\mathrm{per}_m},Ωn​=GLn2​⋅detn​​,Zn,m​=GLn2​⋅X00n−m​perm​​,

where det⁡n=∑σ∈Snsgn(σ)∏iXiσ(i)\det_n = \sum_{\sigma \in S_n}\mathrm{sgn}(\sigma)\prod_i X_{i\sigma(i)}detn​=∑σ∈Sn​​sgn(σ)∏i​Xiσ(i)​ and perm=∑σ∈Sm∏iXiσ(i)\mathrm{per}_m = \sum_{\sigma \in S_m}\prod_i X_{i\sigma(i)}perm​=∑σ∈Sm​​∏i​Xiσ(i)​, and the second polynomial is padded by a power of the single variable X00X_{00}X00​ so that it, too, has degree nnn.

The coordinate ring of a GLn2\mathrm{GL}_{n^2}GLn2​-stable subset SSS of SymnV∗\mathrm{Sym}^n V^*SymnV∗ decomposes into irreducible polynomial representations, which are labelled by partitions λ\lambdaλ with at most n2n^2n2 parts. A partition λ⊢nd\lambda \vdash ndλ⊢nd occurs in the degree ddd part of the coordinate ring of SSS when some polynomial function FFF of degree ddd on SymnV∗\mathrm{Sym}^n V^*SymnV∗ is an eigenvector of the Borel subgroup of upper triangular matrices,

F(g⋅p)  =  (∏vgvvλv)F(p)for all upper triangular invertible g and all forms p,F(g \cdot p) \;=\; \Big(\prod_{v} g_{vv}^{\lambda_{v}}\Big) F(p) \quad\text{for all upper triangular invertible } g \text{ and all forms } p,F(g⋅p)=(v∏​gvvλv​​)F(p)for all upper triangular invertible g and all forms p,

and FFF does not vanish identically on SSS. If λ\lambdaλ occurs for Zn,mZ_{n,m}Zn,m​ but not for Ωn\Omega_nΩn​, then Zn,m⊈ΩnZ_{n,m} \not\subseteq \Omega_nZn,m​⊆Ωn​, hence dc(perm)>n\mathrm{dc}(\mathrm{per}_m) > ndc(perm​)>n: this is an occurrence obstruction.

Formalization targets

Goal — no occurrence obstructions (Theorem 1.4)

n≥m25,λ⊢nd,λ occurs for Zn,m  ⟹  λ occurs for Ωn.n \ge m^{25},\quad \lambda \vdash nd, \quad \lambda \text{ occurs for } Z_{n,m} \;\Longrightarrow\; \lambda \text{ occurs for } \Omega_n .n≥m25,λ⊢nd,λ occurs for Zn,m​⟹λ occurs for Ωn​.

No partition can separate the two orbit closures once the padding is at least polynomial of degree 252525; the occurrence-obstruction route to Valiant's conjecture is therefore closed.

Supporting targets

The milestone list follows the structure of the proof: the shape restriction for the padded permanent (Theorem 2.1), the semigroup property of occurring partitions (Lemma 2.2), the occurrence of row extended even rectangles (Proposition 2.3), the non-vanishing of highest weight vectors of long first row on Ωn\Omega_nΩn​ (Proposition 2.4), and the fact that padded power sums lie in Ωn\Omega_nΩn​ (Theorem 2.5), together with two elementary anchors (Ω2\Omega_2Ω2​ is everything; a symbolic determinant of size nnn puts the padded permanent into Ωn\Omega_nΩn​) and the GCT V hardness hypothesis, which is exactly the statement that the padded permanent stays outside Ωs\Omega_sΩs​ for subexponential sss.

Significance

Theorem 1.4 removes an entire proof strategy: after fifteen years in which occurrence obstructions were the main object of study in GCT, the theorem shows that they do not exist in the regime where they would be useful, so lower bounds must come from comparing multiplicities rather than mere occurrence. The supporting statements are of independent use: Theorem 2.5 and Proposition 2.3 give explicit families of points and partitions for Ωn\Omega_nΩn​, and the shape restriction of Theorem 2.1 constrains every future obstruction argument.

None of this is formalized anywhere. What this mission produces is a machine-checked model of orbit closures of forms, of the Borel-eigenvector description of occurrence, and of the known statements about Ωn\Omega_nΩn​ and Zn,mZ_{n,m}Zn,m​ — infrastructure that any later multiplicity-based argument would reuse. The goal theorem is proved in the literature; the work here is formalizing that proof. The hardness hypothesis and the Mulmuley–Sohoni conjecture itself remain open, and are labelled as such.

Difficulty

The naive approach to "λ occurs in C[Ωn]\mathbb{C}[\Omega_n]C[Ωn​]" is to decompose the plethysm SymdSymnV\mathrm{Sym}^d \mathrm{Sym}^n VSymdSymnV and read off multiplicities; this fails because the coordinate ring of Ωn\Omega_nΩn​ is a quotient of that plethysm by the unknown vanishing ideal of the orbit closure, and almost nothing is known about which highest weight vectors survive restriction. The proof of Theorem 1.4 works instead with explicit highest weight vectors evaluated at explicit points of Ωn\Omega_nΩn​ (padded power sums), using the fundamental invariant of SymnCN\mathrm{Sym}^n \mathbb{C}^NSymnCN and a lifting map for highest weight vectors in plethysms; the combinatorics of tableaux and the case distinction on the degree ddd are where the work lies. A formalization additionally has to build the representation-theoretic vocabulary from scratch, since Mathlib has neither plethysms, nor Weyl modules, nor highest weight theory for GLN\mathrm{GL}_NGLN​.

Formalization scope

Conventions fixed by the Lean development, all in the namespace GCTOcc:

  1. Polynomials live in one ring, with variables indexed by pairs of natural numbers; the n×nn \times nn×n matrix of variables occupies the indices i,j<ni, j < ni,j<n, so forms of different sizes need no renaming maps.
  2. The padding variable is X00X_{00}X00​, an entry of the matrix itself, as in the source; the choice of padding linear form is known to be immaterial.
  3. Orbit closure is defined sequentially: ppp lies in the closure when the coefficients of a sequence of orbit points converge to those of ppp. On this finite-dimensional space sequential closure is the Euclidean closure.
  4. Occurrence of λ\lambdaλ is defined by the existence of a Borel eigenvector of weight λ\lambdaλ and degree ddd that does not vanish identically on the set. Weights are indexed by the order (i,j)↦in+j(i,j) \mapsto in+j(i,j)↦in+j on the variables, and triangularity refers to the same order. A partition is a nonincreasing function N→N\mathbb{N} \to \mathbb{N}N→N supported on the first n2n^2n2 indices whose parts sum to ndndnd.
  5. The statements are not vacuous: all hypotheses used in the milestones are satisfiable, and the elementary anchors (Ω2\Omega_2Ω2​, the symbolic determinant bridge) exhibit points of the orbit closures explicitly.

Infrastructure a complete development needs, and which is reusable beyond this mission: plethysms SymdSymnV\mathrm{Sym}^d\mathrm{Sym}^n VSymdSymnV and their highest weight vectors, the fundamental invariant of a form space, semistandard tableaux of rectangular content, and basic facts about orbit closures of forms. Contributions of any of these pieces are welcome, as are counterexample-hunting reports on the elementary anchors.

Not covered here: the Kronecker-coefficient strand of the program (Ikenmeyer–Mulmuley–Walter 2017 on NP\mathbf{NP}NP-hardness of Kronecker positivity, and the construction of vanishing triples in the Kronecker cone). Stating those results requires irreducible characters of the symmetric group indexed by partitions, or equivalently symmetric-function machinery, which Mathlib does not yet have; they are natural targets for a companion mission once Specht modules exist.

Selected references

  • P. Bürgisser, C. Ikenmeyer, G. Panova, No occurrence obstructions in geometric complexity theory, J. Amer. Math. Soc. 32 (2019), 163–193, doi:10.1090/jams/908.
  • K. D. Mulmuley, Geometric Complexity Theory V: Efficient algorithms for Noether normalization, J. Amer. Math. Soc., electronically published 2016, doi:10.1090/jams/864.
  • C. Ikenmeyer, K. D. Mulmuley, M. Walter, On vanishing of Kronecker coefficients, comput. complex. 26 (2017), doi:10.1007/s00037-017-0158-y.
  • L. G. Valiant, Completeness classes in algebra, STOC 1979, doi:10.1145/800135.804419.
  • H. Kadish, J. M. Landsberg, Padded polynomials, their cousins, and geometric complexity theory, Comm. Algebra 42 (2014), arXiv:1204.4772.
12 thms2 active usersReviewed
🏆Completed
Quantum Information·Captain: Goku

Shallow Quantum Circuits and Causal ConesTextbook

Motivation

A quantum circuit of depth ddd built from gates of fan-in at most two cannot let an output wire depend on more than 2d2^d2d input wires. The argument is folklore and takes a paragraph on paper: the causal cone of the measured wire grows by at most a factor of two per layer. Formalizing it exposes a subtlety that the paper argument hides, and that is what this mission is about.

The subtlety

Define the backward cone step of a wire set SSS through a layer lll by adjoining the support of every gate of lll that meets SSS. There is a choice here: test each gate against the incoming set SSS, or against the partially accumulated cone. Testing against the accumulator over-approximates, and the doubling bound fails. Testing against the incoming set gives the bound — but is only correct when the gates within a layer act on pairwise disjoint wires.

Without that hypothesis (LayerOk) the semantic statement is false, and the counterexample is small: on three wires, the single layer [cnot 2 1, cnot 1 0] has cone {0,1}\{0,1\}{0,1} around wire 000, yet wire 000 ends up holding x0⊕x1⊕x2x_0 \oplus x_1 \oplus x_2x0​⊕x1​⊕x2​. So the cone under-approximates the true dependence. This mission's development carries LayerOk throughout, and the counterexample is recorded in the source.

What is formalized

Layered circuits over {H,S,T,CNOT}\{H, S, T, \mathrm{CNOT}\}{H,S,T,CNOT} on nnn wires, with states as amplitude functions on bit-strings and no tensor products anywhere. On top of that:

  • the combinatorial half — one layer at most doubles the cone, hence ∣cone∣≤2d ∣S∣|\mathrm{cone}| \le 2^{d}\,|S|∣cone∣≤2d∣S∣;
  • norm preservation, so that acceptProb is a genuine probability in [0,1][0,1][0,1];
  • the semantic half — inputs agreeing on the causal cone of the output wire are accepted with equal probability.

The semantic half is proved in the Heisenberg picture. The measurement observable is conjugated backwards through the circuit and its support tracked: a gate meeting the support enlarges it by that gate's own wires, and a gate missing it commutes with the observable and cancels against its own adjoint. That cancellation is the reason the non-cascading cone step is correct, and it is why unitarity of the gate set is needed at the 2n2^n2n-dimensional level rather than gate by gate. Supporting this is a small reusable algebra of local operators: locality is monotone, closed under adjoint and product, and disjointly supported operators commute.

The frontier

The published depth bound assumes each input wire lies in the syntactic cone of the output. That is weaker than saying the wire matters. Milestone 1 asks for the semantically honest version, stated in terms of genuine functional dependence; the bridge is the semantic cone theorem already in the development.

Beyond that, the natural continuations are the same argument for fan-in-kkk gates (∣cone∣≤kd|\mathrm{cone}| \le k^{d}∣cone∣≤kd), for geometrically local circuits where cone growth is linear rather than exponential, and ultimately the Bravyi–Gosset–König separation QNC0⊄NC0\mathrm{QNC}^{0} \not\subset \mathrm{NC}^{0}QNC0⊂NC0 — which needs machinery (non-local games, magic squares) that this development deliberately does not build.

9 thms2 active usersReviewed
🏆Completed
Discrete Geometry·Captain: marwahaha

OpenAI Ten Proofs 07: Hardness of the closest vector problemResearch Paper

Hardness of the closest vector problem, from the OpenAI Ten Proofs formalization. The source already contains machine-checked proofs; this mission organizes its headline result and additional Comparator targets for reuse.

For every Boolean language AAA of finite bit strings for which there exist a polynomial P∈N[X]P\in\mathbb N[X]P∈N[X] and a deterministic polynomial-time Boolean verifier VVV satisfying A(x)=trueA(x)=\mathrm{true}A(x)=true if and only if there is a bit-string certificate ccc with ∣c∣≤P(∣x∣)|c|\le P(|x|)∣c∣≤P(∣x∣) and V(x,c)=trueV(x,c)=\mathrm{true}V(x,c)=true, there exists a total deterministic polynomial-time map fff on bit strings sending every xxx with A(x)=trueA(x)=\mathrm{true}A(x)=true into the following YES set and every xxx with A(x)=falseA(x)=\mathrm{false}A(x)=false into the following NO set. An instance consists of a natural dimension n>0n>0n>0, an integer matrix B∈Zn×nB\in\mathbb Z^{n\times n}B∈Zn×n with det⁡B≠0\det B\ne0detB=0, a target t∈Qnt\in\mathbb Q^nt∈Qn all of whose coordinates are integers, and a rational radius r>0r>0r>0. It is in YES exactly when some z∈Znz\in\mathbb Z^nz∈Zn satisfies ∑i=0n−1((Bz)i−ti)2≤r2\sum_{i=0}^{n-1}((Bz)_i-t_i)^2\le r^2∑i=0n−1​((Bz)i​−ti​)2≤r2, and in NO exactly when every z∈Znz\in\mathbb Z^nz∈Zn satisfies (n1/400r)2<∑i=0n−1((Bz)i−ti)2(n^{1/400}r)^2<\sum_{i=0}^{n-1}((Bz)_i-t_i)^2(n1/400r)2<∑i=0n−1​((Bz)i​−ti​)2. The sets consist of the exact encoded instance strings, formed by concatenating the encodings of nnn, rrr, the target coordinates in order, and the matrix entries in row-major order. Each scalar is encoded by taking its library natural-number code, applying the library's natural-number-to-bit-string encoding to get a word www, and prefixing www with ∣w∣|w|∣w∣ true bits followed by one false bit. Strings without such a valid encoded instance, and instances satisfying neither distance condition, are outside both sets; dimension one is included. Polynomial time is witnessed in the library's TM2 machine model, using its product encoding of Boolean lists for verifier input, its Boolean output encoding for the verifier, and identity encodings for the reduction's input and output.

Additional targets:

  • NP-hardness of binary nearest codeword with gap n^(1/200).
  • NP-hardness of feasible binary syndrome decoding with gap n^(1/200).
  • NP-hardness of rational finite-p lattice distance with gap n^(1/(200p)).

The formalization uses Lean 4.33.1 with Mathlib revision 0df444a360eaa60ab8c11dca51a86af692955474. Its goal and milestone statements retain the validated source statements and their conventions.

Sources: Lean formalization, H_GapCVP.json.

Planned collection: OAI Ten Proofs (oai-ten-proofs).

5 thms1 active userReviewed
🏆Completed
Captain: marwahaha

OpenAI Ten Proofs 05: Arithmetic formula lower bounds for the permanentResearch Paper

Arithmetic formula lower bounds for the permanent, from the OpenAI Ten Proofs formalization. The source already contains machine-checked proofs; this mission organizes its headline result and additional Comparator targets for reuse.

For every natural number n≥32n\ge32n≥32 and every finite arithmetic formula fff whose leaves are variables xijx_{ij}xij​, 0≤i,j<n0\le i,j<n0≤i,j<n, or arbitrary complex constants and whose binary operations are addition, subtraction, multiplication, and division, assume that every divisor subformula evaluates to a nonzero element of the fraction field of C[xij]\mathbb C[x_{ij}]C[xij​] and that the value of fff in this fraction field equals the polynomial per⁡n=∑σ∈Sn∏i=0n−1xi,σ(i)\operatorname{per}_n=\sum_{\sigma\in S_n}\prod_{i=0}^{n-1}x_{i,\sigma(i)}pern​=∑σ∈Sn​​∏i=0n−1​xi,σ(i)​ under the canonical inclusion. Then the number V(f)V(f)V(f) of variable-leaf occurrences satisfies n4/(192log⁡2n)≤V(f)n^4/(192\log_2 n)\le V(f)n4/(192log2​n)≤V(f), with both sides viewed as real numbers. Repeated occurrences are counted separately, constants contribute zero, and the leaves of both operands of each operation, including division, are counted. Nonzero divisors are required as rational functions, not at every numerical substitution.

Additional targets:

  • Variable-leaf lower bound for division-free formulas computing the permanent.

The formalization uses Lean 4.33.1 with Mathlib revision 0df444a360eaa60ab8c11dca51a86af692955474. Its goal and milestone statements retain the validated source statements and their conventions.

Sources: Lean formalization, C_PermanentFormulaLowerBound.json.

Planned collection: OAI Ten Proofs (oai-ten-proofs).

3 thms1 active userReviewed
🏆Completed
Quantum Information·Captain: thomasvidick

MIP* = REResearch Paper

Why the value of a nonlocal game matters

In a two-player one-round nonlocal game, a verifier samples a pair of questions, sends one to each of two players who cannot communicate, and accepts or rejects the pair of answers. The players may share an arbitrary quantum state and measure it; the entangled value of the game is the supremum of their acceptance probability over all such strategies. Nonlocal games are the operational form of Bell's theorem, the model behind device-independent cryptography and self-testing, and the engine of the complexity class MIP∗\mathrm{MIP}^*MIP∗ of languages decidable by a polynomial-time verifier interacting with entangled provers (Cleve, Høyer, Toner, Watrous, 2004, arXiv:quant-ph/0404076).

With classical provers, multiprover interactive proofs decide exactly NEXP\mathrm{NEXP}NEXP (Babai, Fortnow, Lund, 1991, doi:10.1007/BF01200056). For entangled provers the lower bounds grew in steps: NEXP⊆MIP∗\mathrm{NEXP} \subseteq \mathrm{MIP}^*NEXP⊆MIP∗ (Ito, Vidick, 2012, arXiv:1207.0550), NEEXP⊆MIP∗\mathrm{NEEXP} \subseteq \mathrm{MIP}^*NEEXP⊆MIP∗ (Natarajan, Wright, 2019, arXiv:1904.05870), and finally MIP∗=RE\mathrm{MIP}^* = \mathrm{RE}MIP∗=RE (Ji, Natarajan, Vidick, Wright, Yuen, 2020, arXiv:2001.04383): every recursively enumerable language, the halting problem included, has a two-prover one-round interactive proof with entangled provers, and the entangled value of a game cannot be approximated by any algorithm. Through the equivalences of Fritz (2012, arXiv:1008.1168), Junge, Navascués, Palazuelos, Pérez-García, Scholz, Werner (2011, arXiv:1008.1142) and Ozawa (2013, arXiv:1212.1700), the theorem gives a negative answer to Tsirelson's problem and to Connes' embedding problem for von Neumann algebras.

Synchronous games, their value, and game descriptions

Everything below is defined from scratch; the formal development uses exactly these objects.

A synchronous game G=(X,A,μ,D)G = (X, A, \mu, D)G=(X,A,μ,D) has a finite question alphabet XXX and a finite answer alphabet AAA, shared by both players, a probability distribution μ\muμ on X×XX \times XX×X, and a decision predicate D(x,y,a,b)∈{0,1}D(x, y, a, b) \in \{0, 1\}D(x,y,a,b)∈{0,1} which, on equal questions x=yx = yx=y, rejects every pair of unequal answers a≠ba \ne ba=b.

A synchronous strategy of dimension d≥1d \ge 1d≥1 assigns to each question xxx a projective measurement (Max)a∈A(M^x_a)_{a \in A}(Max​)a∈A​ on Cd\mathbb{C}^dCd: positive semidefinite matrices summing to the identity, each idempotent. Both players use the same measurements, and probabilities are taken in the normalized trace τ(M)=Tr⁡(M)/d\tau(M) = \operatorname{Tr}(M)/dτ(M)=Tr(M)/d. The value of the strategy is

ω(G,M)  =  ∑x,y∈X∑a,b∈Aμ(x,y) D(x,y,a,b) Tr⁡(MaxMby)d,\omega(G, M) \;=\; \sum_{x, y \in X} \sum_{a, b \in A} \mu(x, y)\, D(x, y, a, b)\, \frac{\operatorname{Tr}\bigl(M^x_a M^y_b\bigr)}{d},ω(G,M)=x,y∈X∑​a,b∈A∑​μ(x,y)D(x,y,a,b)dTr(Max​Mby​)​,

the acceptance probability when the players measure a maximally entangled state of dimension ddd. The synchronous value ωs(G)\omega_{\mathrm{s}}(G)ωs​(G) is the supremum of ω(G,M)\omega(G, M)ω(G,M) over all synchronous strategies of all dimensions. A synchronous strategy is a tensor-product strategy, so ωs(G)≤ω∗(G)\omega_{\mathrm{s}}(G) \le \omega^*(G)ωs​(G)≤ω∗(G), the entangled value.

A game description ggg is a finite object from which a synchronous game is read off: natural numbers nX,nAn_X, n_AnX​,nA​ fixing X={0,…,nX}X = \{0, \dots, n_X\}X={0,…,nX​} and A={0,…,nA}A = \{0, \dots, n_A\}A={0,…,nA​}, a finite list of weighted question pairs (x,y,w)(x, y, w)(x,y,w) and a finite list of accepted tuples (x,y,a,b)(x, y, a, b)(x,y,a,b). The game GgG_gGg​ has μ(x,y)\mu(x, y)μ(x,y) proportional to the total weight listed for (x,y)(x, y)(x,y) (the point mass at (0,0)(0, 0)(0,0) if the total weight is zero), and DDD accepts exactly the listed tuples, except that unequal answers to equal questions are rejected. Descriptions are coded as natural numbers, so a map from machines to descriptions can be computable.

Turing machines are Gödel numbers c∈Cc \in \mathcal{C}c∈C of partial recursive functions, an equivalent model of computation; "ccc halts on the empty input" means that ccc applied to 000 is defined.

Formalization targets

Goal: halting reduces to the synchronous game value

There is a computable map ggg from machines to game descriptions such that for every machine ccc,

c halts on the empty input  ⟹  ωs(Gg(c))=1,c does not halt  ⟹  ωs(Gg(c))≤12.c \text{ halts on the empty input} \;\Longrightarrow\; \omega_{\mathrm{s}}\bigl(G_{g(c)}\bigr) = 1, \qquad c \text{ does not halt} \;\Longrightarrow\; \omega_{\mathrm{s}}\bigl(G_{g(c)}\bigr) \le \tfrac12 .c halts on the empty input⟹ωs​(Gg(c)​)=1,c does not halt⟹ωs​(Gg(c)​)≤21​.

This is Theorem 12.2 of arXiv:2001.04383 in synchronous form, with the running time of ggg relaxed from polynomial to computable. It leaves the running time and the choice of value unfixed, and it is the weakest statement that already makes the game value uncomputable: no algorithm decides whether a game has synchronous value 111 or at most 12\tfrac1221​.

Stronger: polynomial time, and the entangled value

The paper's statement has ggg computable in polynomial time, and both the synchronous and the entangled value satisfy the two clauses. With polynomial time the goal gives RE⊆MIP∗\mathrm{RE} \subseteq \mathrm{MIP}^*RE⊆MIP∗ for the class MIP1,1/2∗(2,1)\mathrm{MIP}^*_{1, 1/2}(2, 1)MIP1,1/2∗​(2,1) of two-prover one-round proofs with completeness 111 and soundness 12\tfrac1221​, and with the elementary inclusion MIP∗⊆RE\mathrm{MIP}^* \subseteq \mathrm{RE}MIP∗⊆RE this is MIP∗=RE\mathrm{MIP}^* = \mathrm{RE}MIP∗=RE. These statements need a model of polynomial time and a definition of tensor-product strategies, and will be added as milestones once those definitions are published here.

Significance

The result. Approximating ω∗(G)\omega^*(G)ω∗(G) to within a constant is as hard as the halting problem. A hierarchy of semidefinite programs converges from above to the commuting-operator value ωco(G)\omega^{\mathrm{co}}(G)ωco(G), so the two values cannot coincide for every game: a negative answer to Tsirelson's problem and, by the equivalences cited above, to Connes' embedding problem, open since 1976. On the complexity side, MIP∗=RE\mathrm{MIP}^* = \mathrm{RE}MIP∗=RE is the exact characterization of the power of entangled provers.

Formalizing it. The theorem is proved and its proof is machine-checked: HaltingGameValue.halting_reduces_to_gameValue in the Lean 4 development vidick/MIPRE-formalization proves exactly the goal above over Mathlib, with propext, Classical.choice and Quot.sound as its only axioms. The development is about 1,200 modules and 410,000 lines, including vendored components and a separate library for the low individual degree test; its blueprint records every statement, the dependency graph and the correspondence with the paper. This mission makes the result a fixed, auditable target: the definitional layer above is written over Mathlib alone, so the goal can be checked from first principles, and the components of the proof whose developments are self-contained are published as milestones with their proofs, to be imported, reused and re-proved. The first milestone, the recursive compression lemma of Marks, Nezhadi and Yuen (Lemma 5.1 of their manuscript; blueprint node lem:recursive-compression), comes with a complete proof over a dependency tree of 22 definition bundles and 60 theorems: a cost model of computation, an efficient universal machine, the s-m-n and Kleene recursion theorems, succinct descriptions.

Difficulty

The obvious approach fails at once: a verifier runs in time polynomial in its input, while a machine may run for arbitrarily long before halting, so no fixed game can check halting by simulation. What makes RE\mathrm{RE}RE reachable is gap-preserving compression: a polynomial-time transformation of verifiers such that the compressed verifier, at index nnn, has value 111 when the original verifier at index 2n2^n2n does, and value at most 12\tfrac1221​ when the original does. The compressed game is exponentially cheaper than the game it certifies, and a fixed-point argument through Kleene's recursion theorem, in the abstract form of the recursive compression lemma, turns compression into the halting reduction. Compression rests on three components, each with a central ingredient that is a theorem in its own right: question reduction by introspection, in which the players sample their own questions and a Pauli basis self-test built on the quantum low individual degree test certifies that they did; answer reduction, in which the players prove through a probabilistically checkable proof that their answers would have been accepted; and gap amplification by parallel repetition. In each component soundness must hold against strategies of unbounded dimension, where no finite enumeration or compactness argument is available, and the quantitative losses must compose so that the threshold 12\tfrac1221​ is preserved exactly.

Formalization scope

The goal is stated with the synchronous value, projective measurements on Cd\mathbb{C}^dCd with d≥1d \ge 1d≥1, and the normalized trace in place of a state vector; the real part of the trace is taken in the strategy value, and the synchronous value is the supremum, in R\mathbb{R}R, over all strategies of all dimensions (the values lie in [0,1][0, 1][0,1]). Game descriptions are interpreted with the fallback distribution above, so that the interpretation is total. Machines are Mathlib's Nat.Partrec.Code and computability is Mathlib's Computable; polynomial time is not fixed by the goal.

No trivializing formalization is available: the game depends on ccc only through ggg, and the two clauses demand values 111 and at most 12\tfrac1221​ on complementary sets of machines, so no map that ignores its argument, or produces only descriptions of zero total weight, satisfies the statement.

A complete development needs four layers: (i) a model of computation with costs, with polynomial-time maps, an efficient universal machine, the s-m-n and Kleene recursion theorems and succinct descriptions, which is the recursive compression milestone's dependency tree and is reusable in any undecidability argument by compression; (ii) normal form verifiers, conditionally linear samplers, the Pauli basis test and the quantum low individual degree test; (iii) the three transformations, with answer reduction over a PCP decider built from a succinct Cook–Levin theorem and direct parallel repetition in value form, and their soundness; (iv) the halting reduction from compression and the passage from the tensor-product to the synchronous value. Statements and proofs of these components over the shared definitional layer are welcome, as are alternative proofs of the goal from the same definitions, the polynomial-time and entangled-value strengthenings, and the inclusion MIP∗⊆RE\mathrm{MIP}^* \subseteq \mathrm{RE}MIP∗⊆RE.

Selected references

  • Z. Ji, A. Natarajan, T. Vidick, J. Wright, H. Yuen, MIP* = RE, 2020 (revised 2022). arXiv:2001.04383
  • Z. Ji, A. Natarajan, T. Vidick, J. Wright, H. Yuen, Quantum soundness of the classical low individual degree test, 2020. arXiv:2009.12982
  • A. Marks, S. S. Nezhadi, H. Yuen, The recursive compression method for proving undecidability results, manuscript.
  • J. Lin, Tracially embeddable strategies: lifting MIP* tricks to MIP^co, 2023. arXiv:2304.01940
  • A. Natarajan, J. Wright, NEEXP ⊆ MIP*, FOCS 2019. arXiv:1904.05870
  • T. Ito, T. Vidick, A multi-prover interactive proof for NEXP sound against entangled provers, FOCS 2012. arXiv:1207.0550
  • R. Cleve, P. Høyer, B. Toner, J. Watrous, Consequences and limits of nonlocal strategies, CCC 2004. arXiv:quant-ph/0404076
  • L. Babai, L. Fortnow, C. Lund, Non-deterministic exponential time has two-prover interactive protocols, Computational Complexity 1 (1991). doi:10.1007/BF01200056
  • T. Fritz, Tsirelson's problem and Kirchberg's conjecture, Reviews in Mathematical Physics 24 (2012). arXiv:1008.1168
  • M. Junge, M. Navascués, C. Palazuelos, D. Pérez-García, V. B. Scholz, R. F. Werner, Connes' embedding problem and Tsirelson's problem, Journal of Mathematical Physics 52 (2011). arXiv:1008.1142
  • N. Ozawa, About the Connes embedding conjecture: algebraic approaches, Japanese Journal of Mathematics 8 (2013). arXiv:1212.1700
  • The Lean development: vidick/MIPRE-formalization (statement MIPRE/HaltingGameValue.lean, proof MIPRE/MainTheorem.lean) and its blueprint.
106 thms1 active userReviewed
Quantum Information·Captain: mikedeng1

Fixed-Point Quantum Search with an Optimal Number of Queries: The Phases of Eq. (11) Give Success Probability 1 − δ²T_L(T_{1/L}(1/δ)√(1−λ))²Research Paper

Motivation

Quantum search starts with a state that has some overlap with a desired target. Standard Grover iterations can increase that overlap quickly, but their success probability oscillates with the number of iterations. If the initial overlap is unknown, a sequence chosen for one value can pass the target and perform poorly at another. Earlier fixed-point constructions avoided this oscillation but lost the quadratic improvement in query count. Yoder, Low, and Chuang gave a sequence of adjustable phase reflections with both a specified failure tolerance and a query count proportional to the inverse square root of the initial target probability Yoder, Low, and Chuang (2014).

The result concerns amplitude amplification, a reusable procedure in which a quantum algorithm prepares a state, a target test identifies the useful component, and repeated operations increase the chance of measuring that component. The paper's exact success formula is useful because it says what happens at every initial overlap, rather than only at a single guessed overlap or in an asymptotic regime. Its polynomial form also explains why the interval with a guaranteed error bound grows as the sequence becomes longer Yoder, Low, and Chuang (2014), pp. 1–2.

Setting

Let ∣s⟩|s\rangle∣s⟩ and ∣T⟩|T\rangle∣T⟩ be unit vectors in a finite-dimensional complex state space. The target overlap is λ=∣⟨T∣s⟩∣2>0\lambda=|\langle T|s\rangle|^2>0λ=∣⟨T∣s⟩∣2>0. The parameter δ\deltaδ sets the desired upper bound δ2\delta^2δ2 on failure probability. A generalized reflection about a unit state changes its phase while leaving the orthogonal directions unchanged. The two reflections used here are

Ss(α)=I−(1−e−iα)∣s⟩⟨s∣,ST(β)=I−(1−eiβ)∣T⟩⟨T∣.S_s(\alpha)=I-(1-e^{-i\alpha})|s\rangle\langle s|, \qquad S_T(\beta)=I-(1-e^{i\beta})|T\rangle\langle T|.Ss​(α)=I−(1−e−iα)∣s⟩⟨s∣,ST​(β)=I−(1−eiβ)∣T⟩⟨T∣.

Their Grover iterate is G(α,β)=−Ss(α)ST(β)G(\alpha,\beta)=-S_s(\alpha)S_T(\beta)G(α,β)=−Ss​(α)ST​(β). For L=2l+1L=2l+1L=2l+1, the sequence is SL=G(αl,βl)⋯G(α1,β1)\mathcal S_L=G(\alpha_l,\beta_l)\cdots G(\alpha_1,\beta_1)SL​=G(αl​,βl​)⋯G(α1​,β1​), so the iterate with index 111 acts first. The phases are those in Eq. (11) of the paper: αj=−βl−j+1=2cot⁡−1(tan⁡(2πj/L)1−γ2)\alpha_j=-\beta_{l-j+1}=2\cot^{-1}(\tan(2\pi j/L)\sqrt{1-\gamma^2})αj​=−βl−j+1​=2cot−1(tan(2πj/L)1−γ2​), where γ−1=T1/L(1/δ)\gamma^{-1}=T_{1/L}(1/\delta)γ−1=T1/L​(1/δ) Yoder, Low, and Chuang (2014), pp. 2–3.

Here TLT_LTL​ is the degree-LLL Chebyshev polynomial of the first kind. For the fractional order at an argument at least one, T1/L(1/δ)T_{1/L}(1/\delta)T1/L​(1/δ) means the positive real continuation cosh⁡(arcosh⁡(1/δ)/L)\cosh(\operatorname{arcosh}(1/\delta)/L)cosh(arcosh(1/δ)/L). This convention gives TL(γ−1)=1/δT_L(\gamma^{-1})=1/\deltaTL​(γ−1)=1/δ. The paper also uses a two-dimensional coordinate system spanned by the prepared state and target state, with basis order ∣tˉ⟩,∣t⟩|\bar t\rangle,|t\rangle∣tˉ⟩,∣t⟩ and ∣s⟩=(1−λ,λ)|s\rangle=(\sqrt{1-\lambda},\sqrt\lambda)∣s⟩=(1−λ​,λ​) Yoder, Low, and Chuang (2014), Eqs. (4), (6)–(7).

Formalization targets

Exact success probability

The goal is the paper's Eq. (1), for the phase sequence of Eq. (11):

∣⟨T∣SL∣s⟩∣2=1−δ2TL ⁣(T1/L(1/δ)1−λ)2,L=2l+1.|\langle T|\mathcal S_L|s\rangle|^2 =1-\delta^2T_L\!\left(T_{1/L}(1/\delta)\sqrt{1-\lambda}\right)^2, \qquad L=2l+1.∣⟨T∣SL​∣s⟩∣2=1−δ2TL​(T1/L​(1/δ)1−λ​)2,L=2l+1.

This equality determines the entire response curve, including the one-step case l=0l=0l=0. It is stated for arbitrary finite dimension, rather than only after fixing two coordinates. The milestones cover the paper's two-dimensional reduction, the reflection and rotation formulas, the equatorial-rotation sequence, its amplitude recurrence, and the Chebyshev identity used to finish the calculation Yoder, Low, and Chuang (2014), Eqs. (1), (4), (6)–(15).

Consequences

The same exact formula gives a fixed-point interval. With w=1−T1/L(1/δ)−2w=1-T_{1/L}(1/\delta)^{-2}w=1−T1/L​(1/δ)−2, the desired success bound holds whenever λ≥w\lambda\ge wλ≥w:

λ≥w⟹∣⟨T∣SL∣s⟩∣2≥1−δ2.\lambda\ge w\quad\Longrightarrow\quad |\langle T|\mathcal S_L|s\rangle|^2\ge1-\delta^2.λ≥w⟹∣⟨T∣SL​∣s⟩∣2≥1−δ2.

The draft also states that www decreases as the odd length increases, the sufficient length bound L≥log⁡(2/δ)/λL\ge\log(2/\delta)/\sqrt\lambdaL≥log(2/δ)/λ​ of Eq. (3), and the nesting identity for success-probability functions in Eq. (18) Yoder, Low, and Chuang (2014), pp. 2, 4.

Significance

The exact formula gives a sequence that can be selected from a lower bound on the target overlap and a chosen failure tolerance. Increasing its length extends the interval on which the tolerance is met; an already adequate state therefore stays within the bounded-error regime at greater odd lengths. The sufficient length bound records the quadratic query scaling in the overlap parameter. The nesting identity shows that the same family of probability functions is closed under the paper's specified composition of tolerances Yoder, Low, and Chuang (2014), pp. 2, 4.

The mathematical result is proved in the 2014 paper. This mission asks for machine-checked proofs of its exact operator-level equality and the cited intermediate identities; the statements here are currently draft targets, not already machine-checked results. A completed development would leave reusable finite-dimensional phase-reflection and Chebyshev tools for subsequent amplitude-amplification arguments. The paper's oracle implementation is separate from these operator identities Yoder, Low, and Chuang (2014), Fig. 2.

Difficulty

The obvious idea of repeating a fixed phase pair produces the familiar Grover oscillation. Adjusting phases removes that simple repeated-power description, while the target amplitude must still be controlled exactly across all allowed overlaps. A proof must keep the complex phase conventions and multiplication order aligned with the Chebyshev expression. The fractional-order parameter also lies outside the usual cosine formula's interval when δ<1\delta<1δ<1; replacing the polynomial by a clamped arccosine expression changes the target formula. The unnumbered Chebyshev-amplitude assertion after Eq. (15) is the central algebraic gap left for formalization Yoder, Low, and Chuang (2014), p. 3.

Formalization scope

States are functions Fin N → ℂ, and operators are complex N × N matrices. Inner products conjugate the first vector. The theorem binds λ\lambdaλ to the squared target overlap and requires both states to have norm one; it does not take an unrelated free success parameter. The goal permits every NNN for which its hypotheses can hold, and every l≥0l\ge0l≥0. The rotation milestones use Fin 2 with index 000 for ∣tˉ⟩|\bar t\rangle∣tˉ⟩ and index 111 for ∣t⟩|t\rangle∣t⟩. The operator STS_TST​ has the same rank-one projector as the paper's phase-adjusted StS_tSt​. Matrix exponentials define the rotations, and TLT_LTL​ is Mathlib's polynomial, evaluated even outside [−1,1][-1,1][−1,1].

The scope assumes 0<δ≤10<\delta\le10<δ≤1. At δ=0\delta=0δ=0, the displayed expression involves division by zero and the paper treats its π/3\pi/3π/3 limit separately. The inverse cotangent is π/2−arctan⁡\pi/2-\arctanπ/2−arctan, a branch whose changes by π\piπ leave the reflection phases unchanged. The product order is fixed by the recursion defining SL\mathcal S_LSL​. The mission formalizes Eqs. (6)–(7) as operators; the ancilla and oracle gates of Fig. 2 are outside this mission. The goal retains the full equality and cannot be reduced to the easier lower bound PL≥1−δ2P_L\ge1-\delta^2PL​≥1−δ2.

The milestone for printed Eq. (12) uses a free output ZZZ rotation and a unit global phase. The printed outer rotations do not give the claimed state for the stated phases; this corrected form preserves the coordinate magnitudes needed for Eq. (1). The milestone for Eq. (15) states the displayed amplitude recurrence but does not restate the page's incompatible intermediate bh′b'_hbh′​ sentence. These corrections are identified in the item descriptions and moderation notes. Contributions proving the reduction, phase identities, amplitude recurrence, Chebyshev relation, or the final equality are all within scope Yoder, Low, and Chuang (2014), p. 3.

Selected references

  • Theodore J. Yoder, Guang Hao Low, and Isaac L. Chuang, Fixed-point quantum search with an optimal number of queries, Physical Review Letters 113, 210501 (2014). arXiv:1409.3305v2. This mission cites the pinned preprint's page and equation numbers.
10 thms1 active userReviewed
Convex OptimizationMachine LearningOptimization·Captain: mikedeng1

Tight Complexity Bounds for Optimizing Composite Objectives III: Any Deterministic Algorithm with Gradient and Prox Oracles Needs Ω(m√(γ/λ)·log(λε₀/(γε))) Queries on Smooth Strongly Convex SumsResearch Paper

Motivation

Many problems in machine learning and statistics minimize an average of component losses, F(x)=1m∑i=1mfi(x)F(x)=\frac1m\sum_{i=1}^m f_i(x)F(x)=m1​∑i=1m​fi​(x): one term per training example, per data shard, or per block of a decomposed objective. Incremental methods such as SAG, SVRG, SAGA, SDCA and their accelerated variants touch one component at a time, so the natural measure of their work is the number of component queries. When every component is γ\gammaγ-smooth and λ\lambdaλ-strongly convex, accelerated incremental methods reach accuracy ϵ\epsilonϵ after on the order of (m+mγ/λ)log⁡(ϵ0/ϵ)(m+\sqrt{m\gamma/\lambda})\log(\epsilon_0/\epsilon)(m+mγ/λ​)log(ϵ0​/ϵ) queries, while treating FFF as one function and running accelerated gradient descent costs mγ/λlog⁡(ϵ0/ϵ)m\sqrt{\gamma/\lambda}\log(\epsilon_0/\epsilon)mγ/λ​log(ϵ0​/ϵ) component queries.

Woodworth and Srebro ask how far this can go when the oracle is as strong as possible: for a requested component it returns the value, the gradient, and an exact proximal point, the minimizer of fi(u)+β2∥x−u∥2f_i(u)+\frac\beta2\|x-u\|^2fi​(u)+2β​∥x−u∥2. A proximal step solves a whole optimization problem for that component, so it is not obvious that oracle-complexity lower bounds for gradient methods survive. Their Theorem 4 shows that for deterministic algorithms they do: the full mγ/λm\sqrt{\gamma/\lambda}mγ/λ​ factor is unavoidable (Woodworth and Srebro, 2016, Theorem 4, p. 6). Randomization is what separates this from the upper bounds of accelerated incremental methods, which the paper treats separately.

Timeline. Nemirovski and Yudin (1983) and Nesterov (2004) gave the γ/λlog⁡(1/ϵ)\sqrt{\gamma/\lambda}\log(1/\epsilon)γ/λ​log(1/ϵ) lower bound for one smooth strongly convex function under span-restricted methods. Agarwal and Bottou (2015) gave finite-sum lower bounds for deterministic methods with gradient access. Lan and Zhou (2018) proved bounds for randomized incremental gradient methods. Woodworth and Srebro (NeurIPS 2016; revised arXiv v3, 2019) handled arbitrary deterministic and randomized algorithms with a gradient-and-prox oracle and no span assumption.

Setting

Write Ed=RdE_d=\mathbb R^dEd​=Rd with the Euclidean inner product ⟨⋅,⋅⟩\langle\cdot,\cdot\rangle⟨⋅,⋅⟩ and norm ∥⋅∥\|\cdot\|∥⋅∥. A finite-sum objective is F(x)=1m∑i=1mfi(x)F(x)=\frac1m\sum_{i=1}^m f_i(x)F(x)=m1​∑i=1m​fi​(x) with m≥2m\ge2m≥2 components fi:Ed→Rf_i:E_d\to\mathbb Rfi​:Ed​→R. A component is γ\gammaγ-smooth if it is differentiable and its gradient is γ\gammaγ-Lipschitz, and λ\lambdaλ-strongly convex if f(y)≥f(x)+⟨∇f(x),y−x⟩+λ2∥x−y∥2f(y)\ge f(x)+\langle\nabla f(x),y-x\rangle+\frac\lambda2\|x-y\|^2f(y)≥f(x)+⟨∇f(x),y−x⟩+2λ​∥x−y∥2 for all x,yx,yx,y.

A query names a component iii, a point xxx, and a parameter β>0\beta>0β>0. The oracle hFh_FhF​ answers with fi(x)f_i(x)fi​(x), ∇fi(x)\nabla f_i(x)∇fi​(x), and proxfi(x,β)=arg⁡min⁡u{fi(u)+β2∥x−u∥2}\mathrm{prox}_{f_i}(x,\beta)=\arg\min_u\{f_i(u)+\frac\beta2\|x-u\|^2\}proxfi​​(x,β)=argminu​{fi​(u)+2β​∥x−u∥2}. A deterministic algorithm chooses each query as a function of the answers it has received so far. The quantity bounded from below is the number of queries before some query point x^\hat xx^ satisfies F(x^)−F(x∗)<ϵF(\hat x)-F(x^*)<\epsilonF(x^)−F(x∗)<ϵ, where x∗x^*x∗ minimizes FFF.

The proof works with directions v0,…,vkv_0,\dots,v_kv0​,…,vk​, chosen orthonormal, and indicators δi,r∈{0,1}\delta_{i,r}\in\{0,1\}δi,r​∈{0,1} with ∑iδi,r=⌊m/2⌋\sum_i\delta_{i,r}=\lfloor m/2\rfloor∑i​δi,r​=⌊m/2⌋, and the components (at γ=1\gamma=1γ=1)

fi(x)=1−λ8(δi,1(⟨x,v0⟩2−2C⟨x,v0⟩)+δi,kζ⟨x,vk⟩2+∑r=1kδi,r⟨x,vr−1−vr⟩2)+λ2∥x∥2.f_i(x)=\frac{1-\lambda}{8}\Big(\delta_{i,1}(\langle x,v_0\rangle^2-2C\langle x,v_0\rangle)+\delta_{i,k}\zeta\langle x,v_k\rangle^2+\sum_{r=1}^k\delta_{i,r}\langle x,v_{r-1}-v_r\rangle^2\Big)+\frac\lambda2\|x\|^2 .fi​(x)=81−λ​(δi,1​(⟨x,v0​⟩2−2C⟨x,v0​⟩)+δi,k​ζ⟨x,vk​⟩2+r=1∑k​δi,r​⟨x,vr−1​−vr​⟩2)+2λ​∥x∥2.

With Q=⌊m/2⌋m(1λ−1)+1Q=\frac{\lfloor m/2\rfloor}{m}(\frac1\lambda-1)+1Q=m⌊m/2⌋​(λ1​−1)+1 and q=Q−1Q+1q=\frac{\sqrt Q-1}{\sqrt Q+1}q=Q​+1Q​−1​ these define the quantities in the milestones (App. B.4, pp. 16–17).

Formalization targets

Goal: Theorem 4 in Ω\OmegaΩ form

Let R=mγ/λ log⁡λϵ0γϵR=m\sqrt{\gamma/\lambda}\,\log\frac{\lambda\epsilon_0}{\gamma\epsilon}R=mγ/λ​logγϵλϵ0​​. There are absolute constants c,C>0c,C>0c,C>0 such that for every m≥2m\ge2m≥2, γ,λ>0\gamma,\lambda>0γ,λ>0 with γ/λ>73\gamma/\lambda>73γ/λ>73, ϵ>0\epsilon>0ϵ>0 and ϵ0>3γϵ/λ\epsilon_0>3\gamma\epsilon/\lambdaϵ0​>3γϵ/λ there is a dimension d≤CRd\le CRd≤CR with the following property. For every deterministic algorithm AAA there are γ\gammaγ-smooth, λ\lambdaλ-strongly convex f1,…,fmf_1,\dots,f_mf1​,…,fm​ on Rd\mathbb R^dRd and an exact oracle for them, with a minimizer x∗x^*x∗ of FFF and F(0)−F(x∗)=ϵ0F(0)-F(x^*)=\epsilon_0F(0)−F(x∗)=ϵ0​, such that every query point xnx_nxn​ with n+1≤cRn+1\le cRn+1≤cR satisfies

F(xn)−F(x∗)≥ϵ.F(x_n)-F(x^*)\ge\epsilon .F(xn​)−F(x∗)≥ϵ.

The constants are left unspecified, as in the paper's OOO and Ω\OmegaΩ.

Milestones

At γ=1\gamma=1γ=1, λ<1/73\lambda<1/73λ<1/73, the proof establishes, in order: the closed form of FFF with the factor λ(Q−1)/8\lambda(Q-1)/8λ(Q−1)/8; the minimizer x∗=C∑r=0kqr+1vrx^*=C\sum_{r=0}^kq^{r+1}v_rx∗=C∑r=0k​qr+1vr​ with F(x∗)=−λC28(Q−1)2F(x^*)=-\frac{\lambda C^2}{8}(\sqrt Q-1)^2F(x∗)=−8λC2​(Q​−1)2; the ratio bound F(xt)−F(x∗)F(0)−F(x∗)≥q2t−q2k+2Q\frac{F(x_t)-F(x^*)}{F(0)-F(x^*)}\ge\frac{q^{2t}-q^{2k+2}}{\sqrt Q}F(0)−F(x∗)F(xt​)−F(x∗)​≥Q​q2t−q2k+2​ for points orthogonal to vt,…,vkv_t,\dots,v_kvt​,…,vk​; the exponential bound ≥12Qe−4t/(Q−1)\ge\frac1{2\sqrt Q}e^{-4t/(\sqrt Q-1)}≥2Q​1​e−4t/(Q​−1) once k+1=⌈t−12log⁡q⌉k+1=\lceil t-\frac1{2\log q}\rceilk+1=⌈t−2logq1​⌉; the choice of ttt that makes the ratio at least ϵ/ϵ0\epsilon/\epsilon_0ϵ/ϵ0​; and the final count m2⌊⋅⌋≥m40λlog⁡λϵ02ϵ\frac m2\lfloor\cdot\rfloor\ge\frac m{40\sqrt\lambda}\log\frac{\sqrt\lambda\epsilon_0}{2\epsilon}2m​⌊⋅⌋≥40λ​m​log2ϵλ​ϵ0​​ (pp. 17–18).

Significance

The theorem shows that an exact component prox does not buy a better dependence on the condition number γ/λ\gamma/\lambdaγ/λ for deterministic methods: the best deterministic method is, up to constants, accelerated gradient descent on FFF itself. Combined with the paper's randomized lower bounds and known upper bounds, it pins down the gap between deterministic and randomized incremental methods at a factor of about m\sqrt mm​. It also tells designers of distributed and incremental solvers that a prox subroutine per component cannot replace randomization.

The result is proved on paper. To our knowledge it has no machine-checked proof. This mission formalizes the complete deterministic argument: the closed-form algebra of the quadratic construction, the geometric decay of its error along the chain, the counting of rounds, and, as the main work, the adversary that keeps the oracle's answers consistent with an algorithm that has not yet found the later directions. Each algebraic step is checked separately, so its constants are verified individually.

Difficulty

The argument from function values and gradients is standard for span-restricted methods. It does not cover this setting, for two reasons. First, the algorithm is arbitrary and deterministic: its query points need not lie in the span of earlier gradients, so the directions vrv_rvr​ must be chosen adaptively, orthogonal to every query made so far. The hard instance therefore depends on the algorithm, and the oracle must be shown to answer every query consistently with one fixed set of components, chosen at the end. Second, the prox answer is the minimizer of a global problem. The proof must show that, while a component has not been queried in round ttt, the prox of the truncated component coincides with the prox of the final one. The paper asserts this step for the strongly convex construction ("a fact which follows directly from the previous proof"). A solver has to carry it out for the extra λ2∥x∥2\frac\lambda2\|x\|^22λ​∥x∥2 term.

The algebraic steps are also delicate in their constants: the boundary equation at vkv_kvk​ holds only with ζ=1−q\zeta=1-qζ=1−q, and the final count uses λ<1/73\lambda<1/73λ<1/73 and ϵ0>3ϵ/λ\epsilon_0>3\epsilon/\lambdaϵ0​>3ϵ/λ to keep the logarithm and the floor away from their degenerate values.

Formalization scope

Points live in EuclideanSpace ℝ (Fin d). Components are indexed by Fin m (from 0); λ\lambdaλ is written lam. A deterministic algorithm is a function from the list of previous responses to the next query, and an oracle is a fixed function from queries to responses (value, gradient, prox). Query n is the paper's (n+1)(n+1)(n+1)-st. Prox points are a predicate (IsProx), not a choice function, and queries carry β>0\beta>0β>0. Smoothness is a global gradient that is γ\gammaγ-Lipschitz, and strong convexity is Mathlib's StrongConvexOn, which for differentiable functions is equivalent to the gradient inequality above. The minimum is an explicit minimizer, never a real infimum. ⌊m/2⌋\lfloor m/2\rfloor⌊m/2⌋ is natural-number division.

Choices that strengthen or pin the page's statement, each disclosed in the items:

  • the domain is all of Rd\mathbb R^dRd, which is what the proof uses;
  • the dimension is chosen before the algorithm, which the proof supports;
  • the point x^\hat xx^ ranges over query points;
  • the oracle is exact and memoryless.

The goal mentions none of the proof's objects (vrv_rvr​, δ\deltaδ, rounds, QQQ, qqq, ζ\zetaζ, CCC, kkk), so it cannot be made trivial by fixing the construction. Its conclusion requires a genuine minimizer with F(0)−F(x∗)=ϵ0F(0)-F(x^*)=\epsilon_0F(0)−F(x∗)=ϵ0​, so the claim cannot hold for degenerate components.

Needed infrastructure: quadratic forms over orthonormal families, gradients and prox points of quadratics plus λ2∥x∥2\frac\lambda2\|x\|^22λ​∥x∥2, and an adaptive Gram–Schmidt adversary. The model of algorithms and oracles is shared with the companion missions on Theorems 1 and 3 and can be reused for other oracle-complexity lower bounds. Proofs of individual milestones, of the consistency step, and of the scaling from γ=1\gamma=1γ=1 to general γ\gammaγ are all welcome.

Selected references

  • B. Woodworth and N. Srebro, Tight Complexity Bounds for Optimizing Composite Objectives, NeurIPS 2016; arXiv:1605.08003v3, 2019. https://arxiv.org/abs/1605.08003
  • A. Nemirovski and D. Yudin, Problem Complexity and Method Efficiency in Optimization, Wiley, 1983.
  • Y. Nesterov, Introductory Lectures on Convex Optimization: A Basic Course, Kluwer, 2004. https://doi.org/10.1007/978-1-4419-8853-9
  • A. Agarwal and L. Bottou, A Lower Bound for the Optimization of Finite Sums, ICML 2015. https://arxiv.org/abs/1410.0723
  • G. Lan and Y. Zhou, An Optimal Randomized Incremental Gradient Method, Mathematical Programming, 2018. https://arxiv.org/abs/1507.02000
10 thms1 active userReviewed
Convex OptimizationMachine LearningOptimization·Captain: mikedeng1

Tight Complexity Bounds for Optimizing Composite Objectives II: Any Deterministic Algorithm with Gradient and Prox Oracle Access Needs Ω(m√(γB²/ε)) Queries on Sums of m Convex γ-Smooth FunctionsResearch Paper

Motivation

Many problems in machine learning and statistics minimize an average of mmm convex losses, one per data point or per block of data. Incremental methods such as SAG, SVRG, SAGA, SDCA, Finito and Katyusha exploit this finite-sum structure: each step touches one component fif_ifi​ instead of the whole average, and for smooth components the best of them reach accuracy ϵ\epsilonϵ with far fewer component evaluations than full-gradient methods. A natural question is how much of this improvement is possible at all, and whether access to stronger per-component information, such as a proximal step, could do better still.

B. Woodworth and N. Srebro, Tight Complexity Bounds for Optimizing Composite Objectives (NeurIPS 2016, arXiv:1605.08003), answer this with lower bounds on the number of component-oracle calls. Their oracle returns, for a requested component, its value, its gradient, and its exact proximal point. Earlier lower bounds covered only gradient access to smooth, strongly convex sums: Agarwal and Bottou (arXiv:1410.0723) for deterministic algorithms, and Lan (arXiv:1507.02000) for randomized algorithms that sample components from a fixed distribution and keep their iterates in the span of the gradients received (p. 2–3 of the paper). Woodworth and Srebro allow any algorithm, any sequence of gradient and prox queries, and iterates anywhere in Rd\mathbb R^dRd, in all four combinations of Lipschitz or smooth components and bounded or strongly convex objectives. This mission formalizes the deterministic bound for smooth, not strongly convex components on a bounded domain (Theorem 3 of the paper).

Setting

Fix integers m≥2m\ge2m≥2 and d≥0d\ge0d≥0, and write Rd\mathbb R^dRd with the Euclidean inner product ⟨⋅,⋅⟩\langle\cdot,\cdot\rangle⟨⋅,⋅⟩ and norm ∥⋅∥\|\cdot\|∥⋅∥. The domain is the ball X={x∈Rd:∥x∥≤B}\mathcal X=\{x\in\mathbb R^d:\|x\|\le B\}X={x∈Rd:∥x∥≤B}. Given functions f1,…,fm:Rd→Rf_1,\dots,f_m:\mathbb R^d\to\mathbb Rf1​,…,fm​:Rd→R, the objective is

F(x)=1m∑i=1mfi(x).F(x)=\frac1m\sum_{i=1}^m f_i(x).F(x)=m1​i=1∑m​fi​(x).

A component fif_ifi​ is γ\gammaγ-smooth on X\mathcal XX if it is differentiable and its gradient is γ\gammaγ-Lipschitz on X\mathcal XX: ∥∇fi(x)−∇fi(y)∥≤γ∥x−y∥\|\nabla f_i(x)-\nabla f_i(y)\|\le\gamma\|x-y\|∥∇fi​(x)−∇fi​(y)∥≤γ∥x−y∥ for x,y∈Xx,y\in\mathcal Xx,y∈X. Each fif_ifi​ is also required to be convex on X\mathcal XX.

A query is a triple (i,x,β)(i,x,\beta)(i,x,β) of a component index, a point x∈Rdx\in\mathbb R^dx∈Rd and a parameter β>0\beta>0β>0. The oracle hFh_FhF​ answers it exactly with

hF(x,i,β)=[fi(x), ∇fi(x), prox⁡fi(x,β)],prox⁡fi(x,β)∈arg min⁡u∈X{fi(u)+β2∥x−u∥2}.h_F(x,i,\beta)=\bigl[f_i(x),\ \nabla f_i(x),\ \operatorname{prox}_{f_i}(x,\beta)\bigr],\qquad \operatorname{prox}_{f_i}(x,\beta)\in\operatorname*{arg\,min}_{u\in\mathcal X}\Bigl\{f_i(u)+\frac\beta2\|x-u\|^2\Bigr\}.hF​(x,i,β)=[fi​(x), ∇fi​(x), proxfi​​(x,β)],proxfi​​(x,β)∈u∈Xargmin​{fi​(u)+2β​∥x−u∥2}.

A deterministic algorithm AAA chooses its first query, and then each next query as a function of the ordered list of answers received so far. Running AAA against an oracle produces a sequence of query points x(0),x(1),…x^{(0)},x^{(1)},\dotsx(0),x(1),….

Formalization targets

Goal: Theorem 3

There are absolute constants c,C>0c,C>0c,C>0 such that for every m≥2m\ge2m≥2 and γ,B,ϵ>0\gamma,B,\epsilon>0γ,B,ϵ>0 there is a dimension d≤C mγB2/ϵd\le C\,m\sqrt{\gamma B^2/\epsilon}d≤CmγB2/ϵ​ with the following property. For every deterministic algorithm AAA there are convex, γ\gammaγ-smooth components f1,…,fmf_1,\dots,f_mf1​,…,fm​ on the BBB-ball, an exact oracle for them, and a minimizer x∗x^*x∗ of FFF over the ball, such that

F(x(n))−F(x∗) ≥ ϵfor every n with n+1≤c mγB2/ϵ.F\bigl(x^{(n)}\bigr)-F(x^*)\ \ge\ \epsilon\qquad\text{for every } n \text{ with } n+1\le c\,m\sqrt{\gamma B^2/\epsilon}.F(x(n))−F(x∗) ≥ ϵfor every n with n+1≤cmγB2/ϵ​.

In words: Ω(mγB2/ϵ)\Omega\bigl(m\sqrt{\gamma B^2/\epsilon}\bigr)Ω(mγB2/ϵ​) oracle calls are needed before any queried point is strictly ϵ\epsilonϵ-suboptimal. The constants are left unspecified, as in the paper's Ω\OmegaΩ and OOO.

Milestones

The proof (Appendix B.3) builds the components adaptively from orthonormal directions v0,…,vkv_0,\dots,v_kv0​,…,vk​ and indicators δi,r∈{0,1}\delta_{i,r}\in\{0,1\}δi,r​∈{0,1}. The milestones are the claims it states on the way:

  1. the average of the hard components equals the truncated quadratic Fk+1F^{k+1}Fk+1, and agrees with FqF^qFq at points orthogonal to vq,…,vkv_q,\dots,v_kvq​,…,vk​;
  2. the explicit minimizer xt∗=a∑r=0t−1(1−r+1t+1)vrx_t^*=a\sum_{r=0}^{t-1}\bigl(1-\frac{r+1}{t+1}\bigr)v_rxt∗​=a∑r=0t−1​(1−t+1r+1​)vr​ of FtF^tFt and its value −a2⌊m/2⌋8m(1−1t+1)-\frac{a^2\lfloor m/2\rfloor}{8m}\bigl(1-\frac1{t+1}\bigr)−8ma2⌊m/2⌋​(1−t+11​);
  3. ∥xt∗∥2≤a2t/3\|x_t^*\|^2\le a^2t/3∥xt∗​∥2≤a2t/3;
  4. the gap F(x)−F(x∗)≥1/(32k2)F(x)-F(x^*)\ge 1/(32k^2)F(x)−F(x∗)≥1/(32k2) for points not yet aware of the later half of the directions;
  5. Lemma 4: the gradient of the partial function fitf_i^tfit​ agrees with that of fif_ifi​ when iii is queried during round ttt;
  6. Lemma 5: the same for the constrained prox;
  7. the linear construction that forces mmm queries when ϵ\epsilonϵ is large.

Significance

The result. Running accelerated gradient descent on FFF, with all mmm component gradients computed at each step, finds an ϵ\epsilonϵ-suboptimal point with O(mγB2/ϵ)O\bigl(m\sqrt{\gamma B^2/\epsilon}\bigr)O(mγB2/ϵ​) oracle calls (§2 and Table 1 of the paper). Theorem 3 shows that this is optimal among deterministic algorithms, up to a constant factor: neither incremental access to single components nor an exact prox oracle improves the order of the deterministic complexity. Randomized algorithms need only between m+mγB2/ϵm+\sqrt{m\gamma B^2/\epsilon}m+mγB2/ϵ​ and mlog⁡(ϵ0/ϵ)+mγB2/ϵm\log(\epsilon_0/\epsilon)+\sqrt{m\gamma B^2/\epsilon}mlog(ϵ0​/ϵ)+mγB2/ϵ​ calls (Table 1, with ϵ0\epsilon_0ϵ0​ the suboptimality of the point 000), so Theorem 3 also measures the benefit of randomization in incremental methods such as accelerated SVRG: a factor of order m\sqrt mm​ in the dominant term.

Formalizing it. The theorem is proved in the paper; it has no machine-checked proof. A formal proof needs the adaptive adversary argument in full: the construction of directions orthogonal to all earlier queries, the consistency of answers given by partial functions, and the explicit linear-algebra of the tridiagonal quadratic. The paper's final count m/(166ϵ)m/(16\sqrt{6\epsilon})m/(166ϵ​) is false for some parameters (for even mmm and ϵ=1/500\epsilon=1/500ϵ=1/500), and the large-ϵ\epsilonϵ construction needs a dimension of order mmm that the stated bound d=O(mγB2/ϵ)d=O(m\sqrt{\gamma B^2/\epsilon})d=O(mγB2/ϵ​) does not allow when ϵ≫γB2\epsilon\gg\gamma B^2ϵ≫γB2; a complete formal proof settles both points.

Difficulty

The algorithm is arbitrary and adaptive, and it sees exact prox points, which in principle reveal far more about a component than a gradient. The construction must therefore be chosen after the algorithm, round by round, so that every answer the algorithm has received is consistent with the final functions, yet reveals at most one new direction per round. The naive argument, "the algorithm only explores the span of previous gradients", is not available: the algorithm is not restricted to linear spans, and its queries can lie anywhere. Instead, each new direction must be made orthogonal to every point queried so far, and one must show that the partial function used to answer a query has the same gradient and the same constrained prox as the final function at that point (Lemmas 4–5). The constrained prox over the ball is not covered by the paper's unconstrained computation and needs its own argument.

Formalization scope

The space is EuclideanSpace ℝ (Fin d). Components are indexed by Fin m (0-based). A query is a structure carrying a component, a point and a parameter β\betaβ with a proof of β>0\beta>0β>0; the prox answer is described by the published minimizer predicate BoydADMM.Prox.IsProx, not by a choice function. A deterministic algorithm is a function from the list of earlier answers to the next query, and the oracle is a fixed function of the query, required to be exact on every query. The minimum F(x∗)F(x^*)F(x∗) is an explicit attained minimizer over the ball, never a real infimum.

Committed conventions, each disclosed in the item statements:

  • the dimension ddd is chosen before the algorithm (stronger than the page's order, and what the proof gives);
  • the point x^\hat xx^ ranges over query points, indexed from zero; an algorithm can query any point it wishes to output;
  • query points range over all of Rd\mathbb R^dRd, while convexity and smoothness are required on the ball;
  • β>0\beta>0β>0 in every prox query;
  • the milestones work at the proof's normalization γ=B=1\gamma=B=1γ=B=1, use ⌊m/2⌋\lfloor m/2\rfloor⌊m/2⌋ as natural-number division, and express "xv∈span⁡{v0,…,vt−1}x^v\in\operatorname{span}\{v_0,\dots,v_{t-1}\}xv∈span{v0​,…,vt−1​}" as ⟨x,vr⟩=0\langle x,v_r\rangle=0⟨x,vr​⟩=0 for t≤r≤kt\le r\le kt≤r≤k.

A trivializing formalization is ruled out: the goal mentions none of vrv_rvr​, δi,r\delta_{i,r}δi,r​, rounds, aaa, kkk or xt∗x_t^*xt∗​, so it cannot be discharged by assumptions about the construction, and the oracle must answer every query exactly.

A full development needs: the adversary construction (orthonormal vectors orthogonal to a finite set of earlier queries), smoothness of the quadratic hard functions (a Hessian bound ⪯I\preceq I⪯I), existence of constrained prox points, and the arithmetic of the counts. The tridiagonal linear algebra of milestones 2–3 is reusable for other lower-bound constructions in the style of Nesterov's. Proofs of individual milestones, alternative constructions and a corrected explicit constant are all welcome.

Selected references

  • B. Woodworth, N. Srebro, Tight Complexity Bounds for Optimizing Composite Objectives, NeurIPS 2016; arXiv:1605.08003v3, 2019. https://arxiv.org/abs/1605.08003
  • A. Agarwal, L. Bottou, A Lower Bound for the Optimization of Finite Sums, ICML 2015. https://arxiv.org/abs/1410.0723
  • G. Lan, Y. Zhou, An Optimal Randomized Incremental Gradient Method, Mathematical Programming 171, 2018. https://arxiv.org/abs/1507.02000
  • Z. Allen-Zhu, Katyusha: The First Direct Acceleration of Stochastic Gradient Methods, STOC 2017; JMLR 2018. https://arxiv.org/abs/1603.05953
  • Y. Nesterov, Introductory Lectures on Convex Optimization: A Basic Course, Kluwer, 2004. https://doi.org/10.1007/978-1-4419-8853-9
12 thms1 active userReviewed
Convex OptimizationMachine LearningOptimization·Captain: mikedeng1

Tight Complexity Bounds for Optimizing Composite Objectives I: Any Deterministic Algorithm with Gradient and Prox Oracle Access Needs Ω(mLB/ε) Queries on Sums of m Convex L-Lipschitz FunctionsResearch Paper

Motivation

Many optimization problems minimize the average of component losses: one loss per observation, machine, or term in an objective. When an algorithm can inspect one component at a time, the number of component queries is a useful measure of work. A component oracle can provide more than a value or subgradient: a proximal point summarizes a global minimization involving that component. Woodworth and Srebro ask whether this stronger access reduces the worst-case number of queries for nonsmooth finite sums. Their deterministic lower bound answers the question in the regime of convex Lipschitz components on a bounded Euclidean ball (Woodworth and Srebro, 2016, Theorem 1).

The paper studies deterministic and randomized algorithms separately. This mission concerns the deterministic claim in Section 4.1 and its Appendix B.1 proof. The result is relevant to algorithms that alternate among components and use both local subgradient information and component prox computations. The exact oracle matters: a lower bound for an approximate or restricted oracle would not settle the claim made for the stronger one in this paper (Woodworth and Srebro, 2016, pp. 1, 4–5).

Setting

Let m≥2m\ge2m≥2 and let f1,…,fm:Rd→Rf_1,\ldots,f_m:\mathbb R^d\to\mathbb Rf1​,…,fm​:Rd→R be convex functions. Their finite-sum objective is F(x)=m−1∑i=1mfi(x)F(x)=m^{-1}\sum_{i=1}^m f_i(x)F(x)=m−1∑i=1m​fi​(x), minimized over the closed Euclidean ball X={x:∥x∥≤B}\mathcal X=\{x:\|x\|\le B\}X={x:∥x∥≤B} for B>0B>0B>0. Each component is LLL-Lipschitz on X\mathcal XX when ∣fi(x)−fi(y)∣≤L∥x−y∥|f_i(x)-f_i(y)|\le L\|x-y\|∣fi​(x)−fi​(y)∣≤L∥x−y∥ for points x,yx,yx,y in that ball. The paper's notation paragraph prints f(x)−f(x)f(x)-f(x)f(x)−f(x) in this definition; the two-point condition is the intended reading (p. 3).

A query chooses a component index iii, a point x∈Rdx\in\mathbb R^dx∈Rd, and β>0\beta>0β>0. The component oracle returns the value fi(x)f_i(x)fi​(x), a subgradient of fif_ifi​ at xxx, and a point minimizing fi(u)+(β/2)∥x−u∥2f_i(u)+(\beta/2)\|x-u\|^2fi​(u)+(β/2)∥x−u∥2 over u∈Xu\in\mathcal Xu∈X. The subgradient is global: its defining inequality holds against every point in Rd\mathbb R^dRd. A deterministic algorithm chooses its next query from the ordered responses to its previous queries. The oracle is exact and makes a fixed response choice for each query. The paper allows algorithms without a span restriction and defines a round by how many distinct components have been queried, not by how many total queries have occurred (pp. 1, 4–5).

For the appendix milestones, v0,…,vkv_0,\ldots,v_kv0​,…,vk​ are orthonormal directions and δi,r∈{0,1}\delta_{i,r}\in\{0,1\}δi,r​∈{0,1} records whether component iii was missed in round rrr. The hard components fif_ifi​ and their truncations fitf_i^tfit​ are the explicit absolute-value sums in equations (6) and (7). These objects support the lower-bound analysis; they do not restrict the theorem's quantified algorithm or hard instance (pp. 5, 11–12).

Formalization targets

Deterministic oracle lower bound

There are absolute constants c,C>0c,C>0c,C>0 such that, for every m≥2m\ge2m≥2, L,B>0L,B>0L,B>0, and 0<ε<LB/120<\varepsilon<LB/120<ε<LB/12, some dimension d≤CmLB/εd\le C mLB/\varepsilond≤CmLB/ε defeats every deterministic algorithm: for that algorithm there are convex LLL-Lipschitz components and a valid oracle whose first cmLB/εc mLB/\varepsiloncmLB/ε query points all obey

F(xn)−F(x∗)≥ε,F(x_n)-F(x^*)\ge\varepsilon,F(xn​)−F(x∗)≥ε,

where x∗x^*x∗ minimizes FFF on X\mathcal XX. This is the paper's Ω(mLB/ε)\Omega(mLB/\varepsilon)Ω(mLB/ε) conclusion. The constants are absolute, and the dimension bound expresses its O(mLB/ε)O(mLB/\varepsilon)O(mLB/ε) clause (Theorem 1, p. 5).

Supporting targets

Appendix B.1 supplies five milestones: the formula and attained minimum of the average hard function, its error gap for a point orthogonal to an unrevealed direction, the query-count inequality, and Lemmas 2 and 3 on subgradient and proximal-response consistency. The first three are quantitative claims for the normalized L=B=1L=B=1L=B=1 construction; the last two say that answers derived from a truncated component can remain valid for the final component (pp. 11–13).

Significance

The theorem puts a worst-case limit on deterministic methods that can ask for a component prox point as well as a subgradient. Its factor of mmm says that access to individual components does not remove the cost of discovering information spread across the finite sum. The paper compares this lower bound with its deterministic upper bounds, making the query rate tight up to constants in this regime (Woodworth and Srebro, 2016, Table 1 and Theorem 1).

The mathematical result is proved in the paper; this mission asks for a machine-checked proof of its faithful formal statement. The model of exact component queries, deterministic response histories, and constrained proximal minimizers can also support related oracle-complexity developments. None of the new theorem items in this proposal has a machine-checked proof yet. The already published global subgradient definition is reused as a reference.

Difficulty

The theorem concerns every deterministic algorithm, including one whose future queries depend on all responses so far. A single hard function fixed without regard to that interaction does not capture the adversarial information limit established by the paper. The proof must also show that each answer exposed during the interaction is genuinely a value, subgradient, and constrained prox point for one completed family of component functions. Lemma 2 handles the subgradient requirement, while Lemma 3 handles the more global proximal requirement. The latter is delicate because equation (3) minimizes over the ball, whereas the appendix displays an unconstrained calculation (pp. 11–13).

The printed formula in equation (6) can have Lipschitz constant greater than one, contrary to a sentence on page 11. This affects an explicit constant in the proof, though the theorem's Ω\OmegaΩ rate survives rescaling. A verified development must establish a valid uniform Lipschitz bound and carry its normalization into the final constants (Appendix B.1, p. 11).

Formalization scope

Vectors are EuclideanSpace ℝ (Fin d) with its Euclidean inner product and norm. Components use Fin m, a zero-based encoding of the paper's 1,…,m1,\ldots,m1,…,m. Query number zero is the first query; a bound on (n+1)(n+1)(n+1) represents the number of queries made through query nnn. The proximal operation is a predicate describing all minimizers over X\mathcal XX; each legal query has β>0\beta>0β>0. The oracle is memoryless and exact for every query, even when its point lies outside X\mathcal XX. Candidate points in the goal are the algorithm's query points. The dimension is chosen before the algorithm, as the appendix construction permits. The finite average is an explicit sum and x∗x^*x∗ is an explicit feasible minimizer, avoiding a default-valued real infimum.

The milestones use k≥1k\ge1k≥1, orthonormal directions, binary indicators, and ∑iδi,r=⌊m/2⌋\sum_i\delta_{i,r}=\lfloor m/2\rfloor∑i​δi,r​=⌊m/2⌋ where the average is involved. Their normalization is L=B=1L=B=1L=B=1. Natural-number division represents ⌊m/2⌋\lfloor m/2\rfloor⌊m/2⌋, and a finite interval starting at one prevents the term vr−1v_{r-1}vr−1​ from underflowing. The goal quantifies over every deterministic algorithm and demands a valid oracle and an attained minimum; the hard directions, indicators, rounds, and truncations occur only in supporting statements. This rules out a fixed easy instance or an oracle allowed to return arbitrary answers.

The development needs Mathlib's Euclidean-space, convexity, finite-sum, and orthonormal-family libraries and the published global-subgradient definition. Contributions that establish the five milestones, a uniform Lipschitz bound for the hard functions, or the algorithm-dependent construction are within scope. The other deterministic smooth regimes and the randomized results have separate problems and are outside this mission.

Selected references

  • B. Woodworth and N. Srebro, Tight Complexity Bounds for Optimizing Composite Objectives, NeurIPS, 2016; arXiv:1605.08003v3, 2019 revision. arXiv
8 thms1 active userReviewed
CombinatoricsLinear OptimizationTheoretical Computer Science·Captain: mikedeng1

Promise Constraint Satisfaction: Algebraic Structure and a Symmetric Boolean Dichotomy 2: The LP Algorithm Decides PCSPs with Majority or Alternating-Threshold Polymorphisms of All Odd AritiesResearch Paper

Motivation

A constraint satisfaction problem (CSP) asks whether variables can be assigned values so that every constraint of a given instance holds. Feder and Vardi conjectured, and Bulatov and Zhuk proved in 2017, that every CSP over a finite set of relations is either polynomial-time solvable or NP-complete, and that the answer is governed by the polymorphisms of the relations: the functions that map tuples of satisfying assignments, coordinate by coordinate, to a satisfying assignment.

A promise CSP (PCSP) relaxes the question. Each constraint comes as a pair of relations P⊆QP \subseteq QP⊆Q; the input is promised to be satisfiable with every constraint read as PPP, or unsatisfiable even with every constraint read as QQQ, and the task is to tell which. Approximate graph colouring ("is this 3-colourable graph 100-colourable?") and (2+ε)(2+\varepsilon)(2+ε)-SAT are of this kind. Austrin, Guruswami and Håstad (2017, doi:10.1137/15M1006507) showed that (2+ε)(2+\varepsilon)(2+ε)-SAT is NP-hard via polymorphisms; Brakensiek and Guruswami extended the polymorphism view to general Boolean PCSPs and proved a dichotomy for symmetric Boolean families (arXiv:1704.01937v2, SODA 2018, SIAM J. Comput. 2021). Later work by Barto, Bulín, Krokhin and Opršal (arXiv:1811.00970) built the general algebraic theory of PCSPs on this foundation.

This mission takes one ingredient of the dichotomy: the tractable side for two polymorphism families that have no counterpart among tractable CSPs, the Majority and the Alternating-Threshold functions. The paper solves both with the same linear programming test.

Setting

The domain is {0,1}\{0,1\}{0,1}. A finite family of promise relations is Γ={(PR,QR):R∈τ}\Gamma = \{(P_R, Q_R) : R \in \tau\}Γ={(PR​,QR​):R∈τ}, with τ\tauτ finite and PR⊆QR⊆{0,1}kRP_R \subseteq Q_R \subseteq \{0,1\}^{k_R}PR​⊆QR​⊆{0,1}kR​. An instance Ψ=(ΨP,ΨQ)\Psi = (\Psi_P, \Psi_Q)Ψ=(ΨP​,ΨQ​) has variables x1,…,xnx_1, \dots, x_nx1​,…,xn​ and clauses Rj(xj1,…,xjk)R_j(x_{j_1}, \dots, x_{j_k})Rj​(xj1​​,…,xjk​​); a variable may occur several times in a clause. ΨP\Psi_PΨP​ reads each clause with PRjP_{R_j}PRj​​, and ΨQ\Psi_QΨQ​ with QRjQ_{R_j}QRj​​. Since P⊆QP \subseteq QP⊆Q, satisfiability of ΨP\Psi_PΨP​ implies that of ΨQ\Psi_QΨQ​.

A function f:{0,1}L→{0,1}f : \{0,1\}^L \to \{0,1\}f:{0,1}L→{0,1} is a polymorphism of Γ\GammaΓ if for every RRR and all x(1),…,x(L)∈PRx^{(1)}, \dots, x^{(L)} \in P_Rx(1),…,x(L)∈PR​ the tuple (f(x1(1),…,x1(L)),…,f(xk(1),…,xk(L)))\big(f(x^{(1)}_1, \dots, x^{(L)}_1), \dots, f(x^{(1)}_k, \dots, x^{(L)}_k)\big)(f(x1(1)​,…,x1(L)​),…,f(xk(1)​,…,xk(L)​)) lies in QRQ_RQR​. For odd LLL,

MajL(x)=1  ⟺  ∑i=1Lxi>L2,ATL(x)=1  ⟺  ∑i=1L(−1)i−1xi>0.\mathrm{Maj}_L(x) = 1 \iff \sum_{i=1}^L x_i > \frac L2, \qquad \mathrm{AT}_L(x) = 1 \iff \sum_{i=1}^L (-1)^{i-1} x_i > 0 .MajL​(x)=1⟺i=1∑L​xi​>2L​,ATL​(x)=1⟺i=1∑L​(−1)i−1xi​>0.

The LP relaxation of Ψ\PsiΨ has one unknown vj∈[0,1]v_j \in [0,1]vj​∈[0,1] per variable, and for every clause Rj(xj1,…,xjk)R_j(x_{j_1}, \dots, x_{j_k})Rj​(xj1​​,…,xjk​​) of ΨP\Psi_PΨP​ it requires (vj1,…,vjk)(v_{j_1}, \dots, v_{j_k})(vj1​​,…,vjk​​) to lie in the convex hull of PRjP_{R_j}PRj​​. The §3.2 algorithm loops over the variables: it fixes vj=0v_j = 0vj​=0 and re-solves the LP, then, if that is infeasible, fixes vj=1v_j = 1vj​=1 and re-solves, and outputs "unsatisfiable" if both are infeasible. If every variable passes, it outputs "satisfiable". In Lean the answer is the predicate LPAlgAccepts 𝔸 X: the LP is feasible and, for each jjj, it has a solution with vj=0v_j = 0vj​=0 or one with vj=1v_j = 1vj​=1.

Formalization targets

Goal: correctness of the §3.2 algorithm

If MajL∈Pol(Γ)\mathrm{Maj}_L \in \mathrm{Pol}(\Gamma)MajL​∈Pol(Γ) for every odd LLL, or ATL∈Pol(Γ)\mathrm{AT}_L \in \mathrm{Pol}(\Gamma)ATL​∈Pol(Γ) for every odd LLL, then for every instance Ψ\PsiΨ

ΨP satisfiable  ⟹  LPAlgAccepts(Ψ)  ⟹  ΨQ satisfiable.\Psi_P \text{ satisfiable} \;\Longrightarrow\; \texttt{LPAlgAccepts}(\Psi) \;\Longrightarrow\; \Psi_Q \text{ satisfiable}.ΨP​ satisfiable⟹LPAlgAccepts(Ψ)⟹ΨQ​ satisfiable.

No symmetry of the relations is assumed.

Milestones, in the order of the proof (pp. 14–15)

  1. Completeness. If ΨP\Psi_PΨP​ is satisfiable, the algorithm accepts; no polymorphism is needed.
  2. Convexity. A convex combination of LP solutions is an LP solution.
  3. Case 1 claim. If M∈([0,1]∩Q)n×nM \in ([0,1]\cap\mathbb Q)^{n\times n}M∈([0,1]∩Q)n×n has Mii∈{0,1}M_{ii} \in \{0,1\}Mii​∈{0,1}, some rational probability vector vvv has (Mv)i≠1/2(Mv)_i \ne 1/2(Mv)i​=1/2 for all iii.
  4. Case 1 rounding. With MajL\mathrm{Maj}_LMajL​ for all odd LLL, an LP solution www with no coordinate 1/21/21/2 rounds to a satisfying assignment xi∗=⌊wi⌉x^*_i = \lfloor w_i \rceilxi∗​=⌊wi​⌉ of ΨQ\Psi_QΨQ​.
  5. Case 2 perturbation. For such MMM and any rational w^\hat ww^, some vvv has (Mv)i≠w^i(Mv)_i \ne \hat w_i(Mv)i​=w^i​ wherever w^i∉{0,1}\hat w_i \notin \{0,1\}w^i​∈/{0,1}.
  6. Case 2 rounding. With ATL\mathrm{AT}_LATL​ for all odd LLL, two LP solutions w,w^w, \hat ww,w^ that agree only where w^i∈{0,1}\hat w_i \in \{0,1\}w^i​∈{0,1} give the satisfying assignment xi∗=[wi>w^i or wi=w^i=1]x^*_i = [w_i > \hat w_i \text{ or } w_i = \hat w_i = 1]xi∗​=[wi​>w^i​ or wi​=w^i​=1] of ΨQ\Psi_QΨQ​.

Significance

The theorem shows that the Majority and Alternating-Threshold cases of Theorem 3.2 are tractable. Since Γ\GammaΓ is fixed, the LP has size linear in the instance, so linear programming decides PCSP(Γ)\mathrm{PCSP}(\Gamma)PCSP(Γ) in polynomial time. Unlike the Zero, One, AND, OR and Parity cases of §3.1, these families cannot be handled by sandwiching a tractable CSP between PPP and QQQ: footnote 12 of the paper observes that the closure of {MajL}\{\mathrm{Maj}_L\}{MajL​} or {ATL}\{\mathrm{AT}_L\}{ATL​} under identification of variables is not a clone. The tractable half of the paper's symmetric Boolean dichotomy (Theorem 2.16) depends on this result. Later work replaced this LP by the Basic LP and BLP+Affine relaxations (Brakensiek–Guruswami 2019, Barto et al. 2021).

The result is proved in the paper; to our knowledge it has no machine-checked proof. The formal content is a finite-dimensional rational convexity argument plus a counting argument on columns of polymorphism inputs. Related platform work: the published PCSPBLPAff.Symmetric.theorem_2 and theorem_3 concern the BLP+Affine algorithm with symmetric or block-symmetric polymorphisms. That is a different relaxation, and ATL\mathrm{AT}_LATL​ is not symmetric.

Difficulty

Completeness is immediate. Soundness is where the difficulty lies. The obvious approach, rounding an arbitrary LP solution, fails. For Majority, a coordinate equal to 1/21/21/2 has no nearest integer, and the LP may admit no solution without such coordinates unless one uses the per-variable solutions the algorithm found. For Alternating-Threshold, there is no fixed threshold at all, and a single LP solution gives no rule for rounding a fractional coordinate. Two further points need care. Getting from a fractional point back to a polymorphism application needs a polymorphism of an arity that depends on the denominators of the hull weights, so a single fixed arity does not suffice. In the Alternating-Threshold case, as printed, the paper pads the inputs with an arbitrary point of PPP, and that step fails at coordinates where both solutions equal 000 or 111; the padding point has to come from the support of the hull weights.

Formalization scope

The domain {0,1}\{0,1\}{0,1} is Bool. Γ\GammaΓ is a pair 𝔸 𝔹 : RelStruct τ ar Bool with [Fintype τ] and 𝔸.rel R ⊆ 𝔹.rel R, and instances, satisfiability and polymorphisms come from the published PCSPBLPAff_Symmetric_Setting. Coordinates are 0-based (Fin L), so the sign of ATL\mathrm{AT}_LATL​ is (−1)i(-1)^i(−1)i. The LP is stated over Q\mathbb QQ; an LP with rational data is feasible over R\mathbb RR exactly when it is feasible over Q\mathbb QQ. The hull condition uses PRP_RPR​, never QRQ_RQR​. The acceptance predicate also requires the LP itself to be feasible, which matters only for instances without variables. The hypothesis is a disjunction of two universal statements ("MajL\mathrm{Maj}_LMajL​ for all odd LLL, or ATL\mathrm{AT}_LATL​ for all odd LLL"), not "for every odd LLL, one of the two". A formalization that drops the hull constraint, replaces PPP by QQQ in the LP, or lets the acceptance predicate be satisfied without solving the LP would make the theorem trivial or different, and is ruled out by the definitions here.

Out of scope: polynomial running time and the printed conclusion "PCSP(Γ) is polynomial-time tractable" of Theorem 3.2; the polynomial-time solvability of linear programs; the Zero/One/AND/OR/Parity cases (Lemma 3.1, which invokes Schaefer's theorem); and the non-idempotent ("anti-") cases, which go through the reduction of Lemma 2.13(2). The Remark on p. 15 says the algorithm decides but does not find a solution. Only the decision is formalized.

Infrastructure needed: finite convex combinations over Q\mathbb QQ, a perturbation lemma for avoiding finitely many hyperplanes in the rational simplex, and counting identities for MajL\mathrm{Maj}_LMajL​ and ATL\mathrm{AT}_LATL​ on block inputs. The perturbation lemma and the counting identities are reusable beyond this mission. Proofs of any milestone, and alternative arguments (e.g. via the affine-hull algorithm in the Remark on p. 14), are welcome.

Selected references

  • J. Brakensiek, V. Guruswami, Promise Constraint Satisfaction: Algebraic Structure and a Symmetric Boolean Dichotomy, arXiv:1704.01937v2, 2021; SIAM J. Comput. 50(6), 2021. https://arxiv.org/abs/1704.01937v2
  • P. Austrin, V. Guruswami, J. Håstad, (2+ε)-Sat Is NP-Hard, SIAM J. Comput. 46(5), 2017. https://doi.org/10.1137/15M1006507
  • L. Barto, J. Bulín, A. Krokhin, J. Opršal, Algebraic Approach to Promise Constraint Satisfaction, J. ACM 68(4), 2021. https://arxiv.org/abs/1811.00970
  • J. Brakensiek, V. Guruswami, An Algorithmic Blend of LPs and Ring Equations for Promise CSPs, SODA 2019. https://arxiv.org/abs/1807.05194
  • T. J. Schaefer, The Complexity of Satisfiability Problems, STOC 1978. https://doi.org/10.1145/800133.804350
10 thms1 active userReviewed
AlgebraCombinatoricsTheoretical Computer Science·Captain: mikedeng1

Promise Constraint Satisfaction: Algebraic Structure and a Symmetric Boolean Dichotomy 1: Folded Symmetric PCSPs Avoiding Parity, Majority and Alternating-Threshold Have Only C-Fixing PolymorphismsResearch Paper

Why promise constraint satisfaction

A constraint satisfaction problem (CSP) asks whether variables can be assigned values so that every constraint, drawn from a fixed finite set of relations, holds. The algebraic approach to CSPs explains their complexity through polymorphisms, the operations that preserve all constraint relations; this programme culminated in the CSP dichotomy theorem of Bulatov and Zhuk (2017). A promise CSP (PCSP) pairs each relation PPP with a weaker relation Q⊇PQ \supseteq PQ⊇P: given an instance that is promised to be satisfiable with the PPP-constraints, one must distinguish it from instances that are not even satisfiable with the QQQ-constraints. Approximate graph colouring (is a 3-colourable graph 100-colourable?) and (2+ε)(2+\varepsilon)(2+ε)-SAT are PCSPs, and their complexity is not captured by the classical theory.

Brakensiek and Guruswami (arXiv:1704.01937) developed the polymorphism theory of PCSPs and classified the Boolean, symmetric, folded case.

Timeline.

  • 1978: Schaefer classifies Boolean CSPs into polynomial-time and NP-complete.
  • 2014: Austrin, Guruswami and Håstad prove (2+ε)(2+\varepsilon)(2+ε)-SAT NP-hard with an argument based on polymorphisms (SIAM J. Comput. 46(5), 2017).
  • 2016–2018: Brakensiek and Guruswami introduce the general PCSP framework and prove a dichotomy for folded symmetric Boolean PCSPs (SODA 2018; arXiv:1704.01937v2, 2021; SIAM J. Comput. 50(6), 2021).
  • 2019–2021: Barto, Bulín, Krokhin and Opršal recast the theory in terms of minions (J. ACM 68(4), 2021). Ficak, Kozik, Olšák and Stankiewicz extend the Boolean symmetric dichotomy beyond the folded case (ICALP 2019).

Setting

Fix the Boolean domain {0,1}\{0,1\}{0,1}. A promise relation of arity kkk is a pair (P,Q)(P, Q)(P,Q) with P⊆Q⊆{0,1}kP \subseteq Q \subseteq \{0,1\}^kP⊆Q⊆{0,1}k, and a family Γ\GammaΓ is a finite set of promise relations, possibly of different arities. A function f:{0,1}L→{0,1}f : \{0,1\}^L \to \{0,1\}f:{0,1}L→{0,1} is a polymorphism of (P,Q)(P,Q)(P,Q) if, whenever x(1),…,x(L)∈Px^{(1)}, \dots, x^{(L)} \in Px(1),…,x(L)∈P, applying fff coordinate-wise gives a tuple in QQQ; Pol(Γ)\mathrm{Pol}(\Gamma)Pol(Γ) is the set of functions that are polymorphisms of every member of Γ\GammaΓ.

A relation is symmetric if it is closed under permuting coordinates. Every symmetric relation has the form Hamk(S)={x:∣x∣∈S}\mathrm{Ham}_k(S) = \{x : |x| \in S\}Hamk​(S)={x:∣x∣∈S}, where ∣x∣|x|∣x∣ is the Hamming weight. A function is folded if f(xˉ)=¬f(x)f(\bar x) = \neg f(x)f(xˉ)=¬f(x), and idempotent if f(0,…,0)=0f(0,\dots,0) = 0f(0,…,0)=0 and f(1,…,1)=1f(1,\dots,1) = 1f(1,…,1)=1; a family is folded (idempotent) if all its polymorphisms are. For odd LLL the paper singles out three function families:

ParL(x)=⨁i=1Lxi,MajL(x)=[∑ixi>L2],ATL(x)=[∑i=1L(−1)i−1xi>0],\mathrm{Par}_L(x) = \bigoplus_{i=1}^L x_i, \qquad \mathrm{Maj}_L(x) = \Big[\sum_{i} x_i > \tfrac L2\Big], \qquad \mathrm{AT}_L(x) = \Big[\sum_{i=1}^L (-1)^{i-1} x_i > 0\Big],ParL​(x)=i=1⨁L​xi​,MajL​(x)=[i∑​xi​>2L​],ATL​(x)=[i=1∑L​(−1)i−1xi​>0],

and their negations Par‾L,Maj‾L,AT‾L\overline{\mathrm{Par}}_L, \overline{\mathrm{Maj}}_L, \overline{\mathrm{AT}}_LParL​,Maj​L​,ATL​. Finally, fff is CCC-fixing if some set SSS of at most CCC coordinates satisfies f(x)=f(0,…,0)f(x) = f(0,\dots,0)f(x)=f(0,…,0) whenever xxx vanishes on SSS.

Formalization targets

Goal: Theorem 4.13

Let Γ\GammaΓ be a finite, folded, symmetric family. If there are odd L1,…,L6L_1, \dots, L_6L1​,…,L6​ with

ParL1, ATL2, MajL3, Par‾L4, AT‾L5, Maj‾L6∉Pol(Γ),\mathrm{Par}_{L_1},\ \mathrm{AT}_{L_2},\ \mathrm{Maj}_{L_3},\ \overline{\mathrm{Par}}_{L_4},\ \overline{\mathrm{AT}}_{L_5},\ \overline{\mathrm{Maj}}_{L_6} \notin \mathrm{Pol}(\Gamma),ParL1​​, ATL2​​, MajL3​​, ParL4​​, ATL5​​, Maj​L6​​∈/Pol(Γ),

then

∃ C(Γ)  ∀L  ∀f∈Pol(Γ)∩{0,1}{0,1}L: f is C(Γ)-fixing.\exists\, C(\Gamma)\ \ \forall L\ \ \forall f \in \mathrm{Pol}(\Gamma) \cap \{0,1\}^{\{0,1\}^L}:\ f \text{ is } C(\Gamma)\text{-fixing}.∃C(Γ)  ∀L  ∀f∈Pol(Γ)∩{0,1}{0,1}L: f is C(Γ)-fixing.

No value of CCC is fixed: the goal asserts only that one bound works for all arities.

Milestones

The milestones follow the paper's proof:

  • the tools of §2: Propositions 2.10, 2.12 and 2.15 and Lemma 2.13(1), which split polymorphisms into idempotent and negated idempotent ones;
  • the arity-reduction Claims 4.2, 4.3 and 4.4;
  • the closure computations Claim 4.6 (AT\mathrm{AT}AT) and Claim 4.8 (Maj\mathrm{Maj}Maj);
  • the relaxation Lemmas 4.5 and 4.7, which replace Γ\GammaΓ by a single canonical symmetric promise relation;
  • the additive-combinatorics Lemma 4.9 and its bounded Remark;
  • Lemma 4.10 and Corollary 4.11, which bound the coordinates iii with f(ei)=1f(e_i) = 1f(ei​)=1;
  • Lemma 4.12, the idempotent case of the goal.

Significance

Theorem 4.13 is the structural half of the paper's main dichotomy (Theorem 2.16): a folded symmetric Boolean PCSP is polynomial-time solvable if it has one of the six families as polymorphisms for every odd arity, and NP-hard otherwise. Hardness follows by feeding the CCC-fixing structure into a Label Cover reduction (Theorem 5.3). Bounded "fixing" or junta-like structure of all polymorphisms is the standard route from algebra to NP-hardness for PCSPs. This theorem is the cleanest Boolean instance of it, and it underlies later classifications of symmetric Boolean PCSPs.

The result has been proved and published since 2018. To our knowledge none of it has been formalized. A formalization supplies a checked Boolean polymorphism toolkit (Hamming-weight relations, folding, idempotence, relaxations), checked closure computations under alternating-threshold and majority operations, and a uniform-constant Frobenius-type lemma. All of these can be reused by other Boolean PCSP missions.

Difficulty

The obvious argument fixes a polymorphism fff and bounds its relevant coordinates directly from a constraint that Par\mathrm{Par}Par, AT\mathrm{AT}AT or Maj\mathrm{Maj}Maj violates. This gives a bound that depends on the arity LLL, and an LLL-dependent bound is trivial (C=LC = LC=L always works). The work is in making the constant uniform. The proof has to replace Γ\GammaΓ by a single canonical promise relation that does not depend on fff, classify exactly which symmetric relations exclude ATL\mathrm{AT}_LATL​ and MajL\mathrm{Maj}_LMajL​, and use an additive-combinatorics lemma whose constants depend only on that relation's arity. Its bounded version must hold with the same constants. The non-idempotent polymorphisms need a separate reduction through the negated family ¬Γ\neg\Gamma¬Γ.

Formalization scope

The domain {0,1}\{0,1\}{0,1} is Bool (false =0= 0=0). A family Γ\GammaΓ is a pair of relational structures 𝔸 𝔹 : RelStruct τ ar Bool from the published definition PCSPBLPAff_Symmetric_Setting, with [Fintype τ] (finitely many relations) and 𝔸.rel R ⊆ 𝔹.rel R (promise). IsPolymorphism 𝔸 𝔹 f from that file is Definition 2.4. Coordinates are Fin L, so the 1-based sign (−1)i−1(-1)^{i-1}(−1)i−1 of ATL\mathrm{AT}_LATL​ becomes (−1)i(-1)^{i}(−1)i. Anti-functions negate the output. Family properties (folded, idempotent, non-degenerate) quantify over polymorphisms of every arity L≥0L \ge 0L≥0. A folded or idempotent family has no nullary polymorphisms, so admitting L=0L = 0L=0 changes nothing. The constants C(Γ)C(\Gamma)C(Γ), c(Γ)c(\Gamma)c(Γ), A(n)A(n)A(n) and d(n)d(n)d(n) are quantified before the arity and the function (respectively before S0,S1S_0, S_1S0​,S1​). A statement that lets the constant depend on fff or LLL is trivially true and is not the goal.

Out of scope are the complexity conclusions: "PCSP(Γ\GammaΓ) is NP-hard" (Theorems 2.16 and 5.3) and item 2 of Lemma 2.13 (transfer of tractability). The cited inputs for those conclusions are also out of scope: the PCP theorem with parallel repetition (Proposition 5.2) and Schaefer's theorem. No complexity predicate appears in any statement.

A complete development needs the following, with Mathlib's Nat.frobeniusNumber as a starting point for Lemma 4.9:

  • explicit matrix constructions for Claims 4.6 and 4.8;
  • closure of polymorphisms under projections (minors), used in Lemma 4.10 and Corollary 4.11;
  • a Frobenius-coin argument with constants uniform in the sets S0,S1S_0, S_1S0​,S1​.

Contributions are welcome at any milestone, including separate proofs of the closure computations and of Lemma 4.9.

Selected references

  • J. Brakensiek, V. Guruswami, Promise Constraint Satisfaction: Algebraic Structure and a Symmetric Boolean Dichotomy, arXiv:1704.01937v2, 2021; SIAM J. Comput. 50(6), 2021. https://arxiv.org/abs/1704.01937
  • P. Austrin, V. Guruswami, J. Håstad, (2+ε)-SAT is NP-hard, SIAM J. Comput. 46(5), 2017. https://doi.org/10.1137/15M1006507
  • L. Barto, J. Bulín, A. Krokhin, J. Opršal, Algebraic approach to promise constraint satisfaction, J. ACM 68(4), 2021. https://doi.org/10.1145/3457606
  • M. Ficak, M. Kozik, M. Olšák, S. Stankiewicz, Dichotomy for symmetric Boolean PCSPs, ICALP 2019. https://doi.org/10.4230/LIPIcs.ICALP.2019.57
  • T. J. Schaefer, The complexity of satisfiability problems, STOC 1978. https://doi.org/10.1145/800133.804350
19 thms1 active userReviewed
AlgebraLinear OptimizationTheoretical Computer Science·Captain: mikedeng1

Algebraic Approach to Promise Constraint Satisfaction 3: BLP Solves PCSP(A, B) iff Pol(A, B) Has Symmetric Functions of All Arities iff Q_conv Maps to Pol(A, B)Research Paper

Motivation

A promise constraint satisfaction problem PCSP(A,B)\mathrm{PCSP}(\mathbf A,\mathbf B)PCSP(A,B) is given by two finite relational structures with a homomorphism A→B\mathbf A\to\mathbf BA→B. An instance is a finite structure I\mathbf II over the same signature, promised to map to A\mathbf AA; the task is to find a homomorphism to the weaker structure B\mathbf BB (or, in the decision version, to tell I→A\mathbf I\to\mathbf AI→A apart from I↛B\mathbf I\not\to\mathbf BI→B). Approximate graph colouring (A=Kk\mathbf A=K_kA=Kk​, B=Kℓ\mathbf B=K_\ellB=Kℓ​) is the standard example.

The most direct polynomial-time algorithm one can try is linear programming. The basic LP relaxation (BLP) of an instance replaces the 0–1 program "assign one value to each variable, one allowed tuple to each constraint" by probability distributions on values and on tuples, with consistent marginals. Barto, Bulín, Krokhin and Opršal (arXiv:1811.00970v3, Theorem 7.9) characterise exactly when this relaxation decides PCSP(A,B)\mathrm{PCSP}(\mathbf A,\mathbf B)PCSP(A,B), in terms of the polymorphisms of the template. The result extends the CSP case of Kun, O'Donnell, Tamaki, Yoshida and Zhou (KOT+12, Theorem 2(5)&(6)) to promise problems.

Setting

A relational structure A\mathbf AA on a set AAA assigns to each symbol RRR of a finite signature a relation RA⊆Aar(R)R^{\mathbf A}\subseteq A^{\mathrm{ar}(R)}RA⊆Aar(R), with ar(R)≥1\mathrm{ar}(R)\ge 1ar(R)≥1. A polymorphism from A\mathbf AA to B\mathbf BB is a function f:An→Bf:A^n\to Bf:An→B, n≥1n\ge1n≥1, that maps every nnn tuples of RAR^{\mathbf A}RA, applied coordinatewise, to a tuple of RBR^{\mathbf B}RB; these form Pol(A,B)\mathrm{Pol}(\mathbf A,\mathbf B)Pol(A,B). A function is symmetric if f(xπ(1),…,xπ(n))=f(x1,…,xn)f(x_{\pi(1)},\dots,x_{\pi(n)})=f(x_1,\dots,x_n)f(xπ(1)​,…,xπ(n)​)=f(x1​,…,xn​) for every permutation π\piπ.

A minion on (A,B)(A,B)(A,B) is a nonempty family of functions An→BA^n\to BAn→B, n≥1n\ge 1n≥1, closed under minors g↦g(xπ(1),…,xπ(m))g\mapsto g(x_{\pi(1)},\dots,x_{\pi(m)})g↦g(xπ(1)​,…,xπ(m)​) for maps π:[m]→[n]\pi:[m]\to[n]π:[m]→[n]; a minion homomorphism preserves arities and minors. The minion Qconv\mathcal Q_{\mathrm{conv}}Qconv​ on (Q,Q)(\mathbb Q,\mathbb Q)(Q,Q) consists of the convex linear functions f(x1,…,xn)=∑iαixif(x_1,\dots,x_n)=\sum_i\alpha_ix_if(x1​,…,xn​)=∑i​αi​xi​ with αi∈[0,1]\alpha_i\in[0,1]αi​∈[0,1] and ∑iαi=1\sum_i\alpha_i=1∑i​αi​=1. The structure Qconv\mathbf Q_{\mathrm{conv}}Qconv​ has domain Q\mathbb QQ and one relation {x∈Qk∣∑icixi≤d}\{x\in\mathbb Q^k\mid\sum_i c_ix_i\le d\}{x∈Qk∣∑i​ci​xi​≤d} for every rational linear inequality; a finite reduct keeps finitely many of them.

For an instance I\mathbf II, the basic LP has variables μv(a)∈[0,1]\mu_v(a)\in[0,1]μv​(a)∈[0,1] for elements vvv and μv,R(a)∈[0,1]\mu_{\mathbf v,R}(\mathbf a)\in[0,1]μv,R​(a)∈[0,1] for constraints v∈RI\mathbf v\in R^{\mathbf I}v∈RI, constrained by ∑aμv(a)=1\sum_a\mu_v(a)=1∑a​μv​(a)=1 and the marginal equations ∑a(i)=aμv,R(a)=μv(i)(a)\sum_{\mathbf a(i)=a}\mu_{\mathbf v,R}(\mathbf a)=\mu_{\mathbf v(i)}(a)∑a(i)=a​μv,R​(a)=μv(i)​(a). BLPA(I)=1\mathrm{BLP}_{\mathbf A}(\mathbf I)=1BLPA​(I)=1 means that a solution exists in which every μv,R\mu_{\mathbf v,R}μv,R​ is supported on RAR^{\mathbf A}RA. BLP solves PCSP(A,B)\mathrm{PCSP}(\mathbf A,\mathbf B)PCSP(A,B) if every I\mathbf II with BLPA(I)=1\mathrm{BLP}_{\mathbf A}(\mathbf I)=1BLPA​(I)=1 maps to B\mathbf BB.

pp-constructions combine two operations on templates: an nnn-th pp-power (domains AnA^nAn, BnB^nBn, relations defined by one primitive positive formula in both structures) and a homomorphic relaxation (A′,B′)(\mathbf A',\mathbf B')(A′,B′) of (A,B)(\mathbf A,\mathbf B)(A,B) (homomorphisms A′→A\mathbf A'\to\mathbf AA′→A and B→B′\mathbf B\to\mathbf B'B→B′).

Formalization targets

Goal: Theorem 7.9

For a PCSP template (A,B)(\mathbf A,\mathbf B)(A,B), the following are equivalent:

(1) BLP solves PCSP(A,B)  ⟺  (2) Pol(A,B) has symmetric functions of every arity n≥1\text{(1) BLP solves }\mathrm{PCSP}(\mathbf A,\mathbf B)\iff\text{(2) }\mathrm{Pol}(\mathbf A,\mathbf B)\text{ has symmetric functions of every arity }n\ge1(1) BLP solves PCSP(A,B)⟺(2) Pol(A,B) has symmetric functions of every arity n≥1   ⟺  (3) Qconv→Pol(A,B)  ⟺  (4) (A,B) is pp-constructible from a finite reduct of Qconv.\iff\text{(3) }\mathcal Q_{\mathrm{conv}}\to\mathrm{Pol}(\mathbf A,\mathbf B)\iff\text{(4) }(\mathbf A,\mathbf B)\text{ is pp-constructible from a finite reduct of }\mathbf Q_{\mathrm{conv}}.⟺(3) Qconv​→Pol(A,B)⟺(4) (A,B) is pp-constructible from a finite reduct of Qconv​.

Milestones

The structure LP(A)\mathrm{LP}(\mathbf A)LP(A) has as elements the rational probability distributions on AAA; a tuple (ϕ1,…,ϕk)(\phi_1,\dots,\phi_k)(ϕ1​,…,ϕk​) lies in RLP(A)R^{\mathrm{LP}(\mathbf A)}RLP(A) when some rational distribution γ\gammaγ on RAR^{\mathbf A}RA has marginals ϕ1,…,ϕk\phi_1,\dots,\phi_kϕ1​,…,ϕk​. The milestones are:

  • §7.2, p. 47: BLPA(I)=1  ⟺  I→LP(A)\mathrm{BLP}_{\mathbf A}(\mathbf I)=1\iff\mathbf I\to\mathrm{LP}(\mathbf A)BLPA​(I)=1⟺I→LP(A);
  • Remark 7.13: a countable structure whose finite substructures all map to a finite B\mathbf BB maps to B\mathbf BB;
  • Remark 7.11: LP(A)≅FQ(A)\mathrm{LP}(\mathbf A)\cong\mathbf F_{\mathcal Q}(\mathbf A)LP(A)≅FQ​(A), the free structure of Qconv\mathcal Q_{\mathrm{conv}}Qconv​;
  • Lemma 7.12: (A,LP(A))(\mathbf A,\mathrm{LP}(\mathbf A))(A,LP(A)) is a relaxation of a pp-power of Qconv\mathbf Q_{\mathrm{conv}}Qconv​;
  • Lemma 4.8(1), (2) for possibly infinite templates: relaxations and pp-powers receive minion homomorphisms;
  • the step hℓh_\ellhℓ​ (p. 48): a symmetric polymorphism of arity ℓ\ellℓ gives LPℓ(A)→B\mathrm{LP}_\ell(\mathbf A)\to\mathbf BLPℓ​(A)→B, where LPℓ(A)\mathrm{LP}_\ell(\mathbf A)LPℓ​(A) uses distributions with denominators dividing ℓ\ellℓ.

Significance

The result. Theorem 7.9 gives a complete, checkable description of the templates on which the basic LP relaxation is a correct algorithm: one needs only to look for symmetric polymorphisms. Through item (3) it places BLP inside the paper's general theory, in which the complexity of PCSP(A,B)\mathrm{PCSP}(\mathbf A,\mathbf B)PCSP(A,B) depends only on the minion Pol(A,B)\mathrm{Pol}(\mathbf A,\mathbf B)Pol(A,B); BLP corresponds to the single minion Qconv\mathcal Q_{\mathrm{conv}}Qconv​. The same pattern later gives the analogous characterisations for the affine relaxation (Theorem 7.19, minion Zaff\mathcal Z_{\mathrm{aff}}Zaff​) and the combined BLP+AIP algorithm of Brakensiek, Guruswami, Wrochna and Živný.

Formalizing it. The theorem is proved in the paper; no machine-checked version exists. A formalization adds minions and pp-constructions over infinite domains, the structure LP(A)\mathrm{LP}(\mathbf A)LP(A), and a compactness argument for countable structures, all reusable for other relaxation-based algorithms. On the platform, PCSPBLPAff.Symmetric.theorem_2 (symmetric polymorphisms of arbitrarily large arity make BLP+AIP correct) and PCSPBLPAff.Characterization.theorem_4 concern the different BLP+AIP algorithm.

Difficulty

Two steps resist the finite theory. First, Theorem 4.12 (minion homomorphisms correspond to pp-constructions) is proved only for finite templates and fails for infinite ones in general, while Qconv\mathcal Q_{\mathrm{conv}}Qconv​ and Qconv\mathbf Q_{\mathrm{conv}}Qconv​ live on Q\mathbb QQ; the link between (3) and (4) must be made by hand through LP(A)\mathrm{LP}(\mathbf A)LP(A). Second, (2) supplies one symmetric polymorphism per arity, with no compatibility required between different arities, while (1) is a statement about all instances at once. The natural intermediate object, LP(A)\mathrm{LP}(\mathbf A)LP(A), is infinite, and no single polymorphism of Pol(A,B)\mathrm{Pol}(\mathbf A,\mathbf B)Pol(A,B) acts on all of it; mapping it to B\mathbf BB is where finiteness of B\mathbf BB and countability of LP(A)\mathrm{LP}(\mathbf A)LP(A) enter.

Formalization scope

The development builds on the published definitions PCSPBLPAff.Symmetric (relational structures, homomorphisms, polymorphisms, symmetric functions, instances and the BLP polytope IsLPSol). Conventions: signatures are finite with arities ≥1\ge1≥1; AAA and BBB are finite and BBB is nonempty, which excludes only the degenerate template A=B=∅A=B=\emptysetA=B=∅, where items (1)–(3) hold and (4) fails. BLPA(I)=1\mathrm{BLP}_{\mathbf A}(\mathbf I)=1BLPA​(I)=1 is encoded as the feasibility over Q\mathbb QQ of the LP with the vanishing constraints, following the paper's remark after Definition 7.7; dropping those constraints would make item (1) a different, false statement, since the unrestricted LP is always feasible. Item (2) asks for every arity n≥1n\ge1n≥1, not only arbitrarily large ones. Qconv\mathcal Q_{\mathrm{conv}}Qconv​ is defined directly as the convex linear functions (coefficients in [0,1][0,1][0,1] summing to 111; affine coefficients would give a different theorem). Qconv\mathbf Q_{\mathrm{conv}}Qconv​ uses non-strict inequalities. The free structure, LP(A)\mathrm{LP}(\mathbf A)LP(A) and LPℓ(A)\mathrm{LP}_\ell(\mathbf A)LPℓ​(A) are stated with rational values, so LP(A)\mathrm{LP}(\mathbf A)LP(A) is countable.

Left out: the complexity consequences (polynomial-time solvability) and the identification of Qconv\mathcal Q_{\mathrm{conv}}Qconv​ with the polymorphisms of Qconv\mathbf Q_{\mathrm{conv}}Qconv​, which the proof does not need. The cited input [KOT+12, Proposition 12] is replaced by its use in the proof, the milestone on LP(A)\mathrm{LP}(\mathbf A)LP(A). The proof of Lemma 7.12 on p. 47 omits γ≥0\gamma\ge0γ≥0 from the pp-definition; nonnegativity is itself pp-definable in Qconv\mathbf Q_{\mathrm{conv}}Qconv​, so the lemma's statement is unaffected.

Contributions welcome: proofs of any milestone, in particular the compactness remark (or a derivation from PCSPBLPAff.Characterization.lemma_16) and Lemma 4.8 for infinite templates.

Selected references

  • L. Barto, J. Bulín, A. Krokhin, J. Opršal, Algebraic approach to promise constraint satisfaction, arXiv:1811.00970v3, 2019; J. ACM 68(4), 2021. https://arxiv.org/abs/1811.00970
  • G. Kun, R. O'Donnell, S. Tamaki, Y. Yoshida, Y. Zhou, Linear programming, width-1 CSPs, and robust satisfaction, ITCS 2012. https://doi.org/10.1145/2090236.2090274
  • M. Bodirsky, M. Mamino, Constraint satisfaction problems over numeric domains, Dagstuhl Follow-Ups 7, 2017. https://doi.org/10.4230/DFU.Vol7.15301.79
  • J. Brakensiek, V. Guruswami, M. Wrochna, S. Živný, The power of the combined basic LP and affine relaxation for promise CSPs, SIAM J. Comput. 49(6), 2020. https://arxiv.org/abs/1907.04383
16 thms1 active userReviewed
AlgebraCombinatoricsTheoretical Computer Science·Captain: mikedeng1

Promise Constraint Satisfaction: Algebraic Structure and a Symmetric Boolean Dichotomy 4: F = Pol(Γ) for a Finite Γ iff F Is Projection-Closed, Finitizable and Contains the IdentityResearch Paper

Motivation

A constraint satisfaction problem (CSP) asks whether variables ranging over a finite domain DDD can be assigned values so that every constraint from a fixed finite list of relations is satisfied. A central organizing principle of the algebraic theory of CSPs is that the complexity of CSP(Γ)\mathrm{CSP}(\Gamma)CSP(Γ) is governed by its polymorphisms, the functions f:DL→Df : D^L \to Df:DL→D that preserve every relation of Γ\GammaΓ when applied coordinate-wise. For this principle to be usable, one needs to know which sets of functions arise as polymorphism sets. For ordinary relations this goes back to the Galois theory of clones and relations. Pippenger (Galois theory for minors of finite functions, Discrete Mathematics, 2002) characterized the families of functions that are the polymorphism sets of pairs of relations, allowing infinitely many pairs.

Promise constraint satisfaction problems (PCSPs) relax CSPs: every constraint comes as a pair P⊆QP \subseteq QP⊆Q, and the task is to distinguish instances satisfiable with the strong relations PPP from those unsatisfiable even with the weak relations QQQ. Approximate graph colouring and (2+ε)(2+\varepsilon)(2+ε)-SAT are PCSPs. Brakensiek and Guruswami (arXiv:1704.01937v2, SIAM J. Comput. 2021) develop an algebraic theory of PCSPs based on polymorphisms. In §6.2 they prove a finite version of Pippenger's characterization: the families of the form Pol(Γ)\mathrm{Pol}(\Gamma)Pol(Γ) for a finite family Γ\GammaΓ of promise relations are exactly the families satisfying three explicit conditions. This mission formalizes that theorem and its CSP analogue (§6.3).

Setting

Fix a finite set DDD. A family of functions F\mathcal FF over DDD consists, for every arity L≥1L \ge 1L≥1, of a set of functions DL→DD^L \to DDL→D.

A promise relation of arity kkk is a pair (P,Q)(P, Q)(P,Q) of subsets of DkD^kDk with P⊆QP \subseteq QP⊆Q. A finite family Γ={(Pi,Qi)}\Gamma = \{(P_i, Q_i)\}Γ={(Pi​,Qi​)} of promise relations is indexed by a finite set of relation symbols, each with its own arity kik_iki​. A function f:DL→Df : D^L \to Df:DL→D is a polymorphism of Γ\GammaΓ if for every iii and all x(1),…,x(L)∈Pix^{(1)}, \dots, x^{(L)} \in P_ix(1),…,x(L)∈Pi​, applying fff coordinate by coordinate gives a tuple in QiQ_iQi​:

(f(x1(1),…,x1(L)),…,f(xki(1),…,xki(L)))∈Qi.\bigl(f(x^{(1)}_1, \dots, x^{(L)}_1), \dots, f(x^{(1)}_{k_i}, \dots, x^{(L)}_{k_i})\bigr) \in Q_i.(f(x1(1)​,…,x1(L)​),…,f(xki​(1)​,…,xki​(L)​))∈Qi​.

Pol(Γ)\mathrm{Pol}(\Gamma)Pol(Γ) is the family of all polymorphisms of Γ\GammaΓ.

For f:DL→Df : D^L \to Df:DL→D and a map π:[L]→[R]\pi : [L] \to [R]π:[L]→[R], the projection (or minor) fπ:DR→Df^\pi : D^R \to Dfπ:DR→D is

fπ(y)=f(yπ(1),…,yπ(L)).f^\pi(y) = f(y_{\pi(1)}, \dots, y_{\pi(L)}).fπ(y)=f(yπ(1)​,…,yπ(L)​).

A family F\mathcal FF is projection-closed if fπ∈Ff^\pi \in \mathcal Ffπ∈F whenever f∈Ff \in \mathcal Ff∈F has arity LLL and π:[L]→[R]\pi : [L] \to [R]π:[L]→[R]. It is finitizable if there is a finitized arity RRR such that for every arity LLL and every f:DL→Df : D^L \to Df:DL→D,

f∈F  ⟺  fπ∈F for all π:[L]→[R].f \in \mathcal F \iff f^\pi \in \mathcal F \text{ for all } \pi : [L] \to [R].f∈F⟺fπ∈F for all π:[L]→[R].

Finally idD:D→D\mathrm{id}_D : D \to DidD​:D→D is the identity, a function of arity 111.

Formalization targets

Goal: Theorem 6.5

∃ Γ finite with F=Pol(Γ)  ⟺  F is projection-closed and finitizable, and idD∈F.\exists\, \Gamma \text{ finite with } \mathcal F = \mathrm{Pol}(\Gamma) \iff \mathcal F \text{ is projection-closed and finitizable, and } \mathrm{id}_D \in \mathcal F.∃Γ finite with F=Pol(Γ)⟺F is projection-closed and finitizable, and idD​∈F.

The finite family is existential together with its signature and arities; nothing about its shape is fixed in the statement.

Milestones

  • Lemma 6.7. If F\mathcal FF is projection-closed, finitizable and contains idD\mathrm{id}_DidD​, then F=Pol(Γ)\mathcal F = \mathrm{Pol}(\Gamma)F=Pol(Γ) for some finite Γ\GammaΓ.

Further items for the other direction

  • Claim 6.6. For every finite family Γ\GammaΓ of promise relations, Pol(Γ)\mathrm{Pol}(\Gamma)Pol(Γ) is projection-closed and finitizable.
  • §6.2, p. 29. Since P⊆QP \subseteq QP⊆Q for every (P,Q)∈Γ(P, Q) \in \Gamma(P,Q)∈Γ, idD∈Pol(Γ)\mathrm{id}_D \in \mathrm{Pol}(\Gamma)idD​∈Pol(Γ).

Further item: Lemma 6.8

A CSP is a finite family with P=QP = QP=Q for every pair. With F\mathcal FF a clone (for f∈Ff \in \mathcal Ff∈F of arity L1L_1L1​ and g1,…,gL1∈Fg_1, \dots, g_{L_1} \in \mathcal Fg1​,…,gL1​​∈F of arity L2L_2L2​, the function h(x(1),…,x(L1))=f(g1(x(1)),…,gL1(x(L1)))h(x^{(1)}, \dots, x^{(L_1)}) = f(g_1(x^{(1)}), \dots, g_{L_1}(x^{(L_1)}))h(x(1),…,x(L1​))=f(g1​(x(1)),…,gL1​​(x(L1​))) of arity L1L2L_1L_2L1​L2​ is in F\mathcal FF):

∃ Λ CSP with F=Pol(Λ)  ⟺  F is finitizable, a clone, and idD∈F.\exists\, \Lambda \text{ CSP with } \mathcal F = \mathrm{Pol}(\Lambda) \iff \mathcal F \text{ is finitizable, a clone, and } \mathrm{id}_D \in \mathcal F.∃Λ CSP with F=Pol(Λ)⟺F is finitizable, a clone, and idD​∈F.

Significance

Theorem 6.5 identifies the polymorphism families of finite promise templates intrinsically, without reference to relations. Together with the Galois correspondence of §6.1 (if Pol(Γ)⊆Pol(Γ′)\mathrm{Pol}(\Gamma) \subseteq \mathrm{Pol}(\Gamma')Pol(Γ)⊆Pol(Γ′) then Γ′\Gamma'Γ′ is definable from Γ\GammaΓ, so PCSP(Γ′)\mathrm{PCSP}(\Gamma')PCSP(Γ′) reduces to PCSP(Γ)\mathrm{PCSP}(\Gamma)PCSP(Γ)), it justifies studying PCSPs entirely through families of functions closed under minors, the viewpoint later systematized by the theory of minions (Barto, Bulín, Krokhin, Opršal, arXiv:1811.00970). Lemma 6.8 recovers, in the same language, the description of polymorphism clones of finite CSP templates as the finitely related clones.

The results are proved in the paper; to our knowledge none of them has a machine-checked proof. The mission produces a Lean formalization of both directions, built on the published definitions of relational structures and polymorphisms already used by other PCSP missions on the platform, and reusable definitions of projections (minors), projection-closed and finitizable families, and clones.

Difficulty

The arguments are elementary, and the difficulty lies in uniformity and finiteness. For Claim 6.6, a single arity RRR must detect every failure of the polymorphism condition for functions of every arity LLL, so RRR has to be bounded independently of LLL; an argument that handles each arity separately does not give finitizability. For Lemma 6.7, a finite family of promise relations has to be produced from F\mathcal FF alone. The general Galois connection between functions and relations produces infinitely many relations (one for each arity and each witness of non-membership), and the content of the lemma is that the three conditions allow a finite family instead. In both directions the encoding matters: functions of arity RRR must be compared with tuples of a relation, and coordinates, projections and the identity must be matched exactly, with no degenerate arity left over.

Formalization scope

A family of functions is FunFamily D := (L : ℕ) → Set ((Fin L → D) → D), with DDD a finite type with decidable equality. Coordinates are 0-based (Fin L), and fπf^\pifπ is fun y => f (y ∘ π) with π : Fin L → Fin R. A finite family of promise relations is a finite type τ of symbols with arities ar : τ → ℕ and two structures 𝔸 𝔹 : RelStruct τ ar D with 𝔸.rel R ⊆ 𝔹.rel R; polymorphisms are the published IsPolymorphism (definition PCSPBLPAff_Symmetric_Setting), and a CSP uses one structure on both sides.

All function arities are positive. The paper's "L,R∈NL, R \in \mathbb NL,R∈N" is read as positive integers: "F=Pol(Γ)\mathcal F = \mathrm{Pol}(\Gamma)F=Pol(Γ)" is compared at every arity L≥1L \ge 1L≥1, projection-closure and finitizability quantify over L,R≥1L, R \ge 1L,R≥1, and the finitized arity is at least 111. Including arity 000 would make the necessity direction false as stated (with every PiP_iPi​ empty the paper's finitized arity is 000).

The clone condition of Lemma 6.8 is the paper's displayed one (p. 30): g1,…,gL1g_1, \dots, g_{L_1}g1​,…,gL1​​ act on disjoint blocks of inputs, and the composed function has arity L1L2L_1 L_2L1​L2​.

A trivializing formalization is ruled out by the statement's shape: the finite family, its signature and its arities are all existentially quantified (fixing a single relation would turn the necessity direction into a special case), P⊆QP \subseteq QP⊆Q is part of every family (without it idD∈Pol(Γ)\mathrm{id}_D \in \mathrm{Pol}(\Gamma)idD​∈Pol(Γ) fails), and finitizability is an equivalence for every function, members and non-members alike.

The development needs only finite types and functions; no complexity theory is involved. Proofs of the milestones, and reusable lemmas about projections (composition of projections, projections along injective maps), are welcome.

Selected references

  • J. Brakensiek, V. Guruswami, Promise Constraint Satisfaction: Algebraic Structure and a Symmetric Boolean Dichotomy, SIAM J. Comput. 50(6), 2021; arXiv:1704.01937v2. https://arxiv.org/abs/1704.01937
  • N. Pippenger, Galois theory for minors of finite functions, Discrete Mathematics 254 (1–3), 2002 (cited as [48] in the paper).
  • L. Barto, J. Bulín, A. Krokhin, J. Opršal, Algebraic approach to promise constraint satisfaction, J. ACM 68(4), 2021; arXiv:1811.00970. https://arxiv.org/abs/1811.00970
4 thms1 active userReviewed
AlgebraCombinatoricsTheoretical Computer Science·Captain: mikedeng1

Promise Constraint Satisfaction: Algebraic Structure and a Symmetric Boolean Dichotomy 3: Pol(Γ) ⊆ Pol(Γ′) Implies Γ′ Is ppp-Definable from ΓResearch Paper

Motivation

A constraint satisfaction problem (CSP) asks for an assignment of values from a finite domain DDD to variables so that every constraint, a tuple of variables required to lie in a fixed relation, is satisfied. The algebraic theory of CSPs rests on a Galois correspondence: the complexity of CSP(Γ)\mathrm{CSP}(\Gamma)CSP(Γ) is determined by the set Pol(Γ)\mathrm{Pol}(\Gamma)Pol(Γ) of its polymorphisms, because Pol(Γ)⊆Pol(Γ′)\mathrm{Pol}(\Gamma) \subseteq \mathrm{Pol}(\Gamma')Pol(Γ)⊆Pol(Γ′) holds exactly when every relation of Γ′\Gamma'Γ′ is definable from Γ\GammaΓ by a primitive positive formula (Geiger 1968; Bodnarčuk, Kalužnin, Kotov and Romov 1969), and such definitions translate into reductions (Jeavons 1998). This correspondence underlies the CSP dichotomy theorem.

Promise CSPs (PCSPs) relax a CSP by a promise: every constraint comes as a pair (P,Q)(P, Q)(P,Q) with P⊆QP \subseteq QP⊆Q, and the task is to tell instances satisfiable with the PPP-relations from instances not even satisfiable with the QQQ-relations. Approximate graph colouring and (2+ε)(2+\varepsilon)(2+ε)-SAT are examples. Brakensiek and Guruswami (arXiv:1704.01937v2, §6.1) showed that the Galois correspondence survives this generalisation, with pp-definitions replaced by positive primitive promise definitions (ppp-definitions). This is the starting point of the algebraic approach to PCSPs later systematised by Barto, Bulín, Krokhin and Opršal (arXiv:1811.00970). Pippenger (2002) had earlier proved a variant of the correspondence for pairs of relations, not in this complexity-theoretic form.

Setting

Fix a finite domain DDD. A relation of arity kkk is a set P⊆DkP \subseteq D^kP⊆Dk, and a promise relation is a pair (P,Q)(P, Q)(P,Q) of relations of arity kkk with P⊆QP \subseteq QP⊆Q. A family Γ={(PR,QR):R∈τ}\Gamma = \{(P_R, Q_R) : R \in \tau\}Γ={(PR​,QR​):R∈τ} is indexed by a set τ\tauτ of symbols with arities kRk_RkR​. In Lean it is a pair of relational structures 𝔸 𝔹 : RelStruct τ ar D, with PRP_RPR​ = 𝔸.rel R, QRQ_RQR​ = 𝔹.rel R and PR⊆QRP_R \subseteq Q_RPR​⊆QR​ (IsPromiseFamily).

A function f:DL→Df : D^L \to Df:DL→D is a polymorphism of Γ\GammaΓ if for every RRR and all x(1),…,x(L)∈PRx^{(1)}, \dots, x^{(L)} \in P_Rx(1),…,x(L)∈PR​ the tuple obtained by applying fff coordinate-wise, (f(x1(1),…,x1(L)),…,f(xkR(1),…,xkR(L)))\big(f(x^{(1)}_1, \dots, x^{(L)}_1), \dots, f(x^{(1)}_{k_R}, \dots, x^{(L)}_{k_R})\big)(f(x1(1)​,…,x1(L)​),…,f(xkR​(1)​,…,xkR​(L)​)), lies in QRQ_RQR​. Pol(Γ)\mathrm{Pol}(\Gamma)Pol(Γ) is the set of polymorphisms of all arities L≥0L \ge 0L≥0.

A Γ\GammaΓ-PCSP Ψ\PsiΨ is a finite list of clauses, each a symbol RRR applied to a tuple of variables, with repetition allowed. It is read twice: as ΨP\Psi_PΨP​, with each clause asking its tuple to lie in PRP_RPR​, and as ΨQ\Psi_QΨQ​, with QRQ_RQR​ in place of PRP_RPR​. Let EQUAL={(i,i):i∈D}\mathrm{EQUAL} = \{(i, i) : i \in D\}EQUAL={(i,i):i∈D}.

Definition (ppp-definability). A promise relation (P′,Q′)⊆Dk×Dk(P', Q') \subseteq D^k \times D^k(P′,Q′)⊆Dk×Dk is ppp-definable from a finite family Γ\GammaΓ if there are ℓ≥0\ell \ge 0ℓ≥0 and a Γ∪{EQUAL}\Gamma \cup \{\mathrm{EQUAL}\}Γ∪{EQUAL}-PCSP Ψ\PsiΨ on k+ℓk + \ellk+ℓ variables such that

  1. every x∈P′x \in P'x∈P′ extends by some y∈Dℓy \in D^\elly∈Dℓ to an assignment (x,y)(x, y)(x,y) satisfying ΨP\Psi_PΨP​;
  2. every assignment zzz satisfying ΨQ\Psi_QΨQ​ has (z1,…,zk)∈Q′(z_1, \dots, z_k) \in Q'(z1​,…,zk​)∈Q′.

A family Γ′\Gamma'Γ′ is ppp-definable from Γ\GammaΓ if each of its promise relations is.

Formalization targets

Goal: Theorem 6.1, algebraic core

For a finite family Γ\GammaΓ and a family Γ′\Gamma'Γ′ over the same finite domain,

Pol(Γ)⊆Pol(Γ′)  ⟹  every (P′,Q′)∈Γ′ is ppp-definable from Γ.\mathrm{Pol}(\Gamma) \subseteq \mathrm{Pol}(\Gamma') \;\Longrightarrow\; \text{every } (P', Q') \in \Gamma' \text{ is ppp-definable from } \Gamma.Pol(Γ)⊆Pol(Γ′)⟹every (P′,Q′)∈Γ′ is ppp-definable from Γ.

The printed Theorem 6.1 concludes a polynomial-time reduction from PCSP(Γ′)\mathrm{PCSP}(\Gamma')PCSP(Γ′) to PCSP(Γ)\mathrm{PCSP}(\Gamma)PCSP(Γ). Its proof opens by reducing to the displayed statement, and the reduction follows from it by substituting gadgets.

Milestones

  1. Sandwich remark (p. 27). If P′⊆P⊆Q⊆Q′P' \subseteq P \subseteq Q \subseteq Q'P′⊆P⊆Q⊆Q′ (same arity), then (P′,Q′)(P', Q')(P′,Q′) is ppp-definable from {(P,Q)}\{(P, Q)\}{(P,Q)}.
  2. Transitivity (p. 27). If Γ′\Gamma'Γ′ is ppp-definable from Γ\GammaΓ and (P′′,Q′′)(P'', Q'')(P′′,Q′′) from Γ′\Gamma'Γ′, then (P′′,Q′′)(P'', Q'')(P′′,Q′′) is ppp-definable from Γ\GammaΓ.
  3. Proposition 6.3 (p. 28). For L≥1L \ge 1L≥1, the promise relation (SL,TL)(S_L, T_L)(SL​,TL​) is ppp-definable from Γ\GammaΓ, where SL={f:DL→D:f∈Pol(P,P) ∀(P,Q)∈Γ}S_L = \{f : D^L \to D : f \in \mathrm{Pol}(P, P)\ \forall (P,Q) \in \Gamma\}SL​={f:DL→D:f∈Pol(P,P) ∀(P,Q)∈Γ}, TL={f:f∈Pol(P,Q) ∀(P,Q)∈Γ}T_L = \{f : f \in \mathrm{Pol}(P, Q)\ \forall (P,Q) \in \Gamma\}TL​={f:f∈Pol(P,Q) ∀(P,Q)∈Γ}, and each fff is read as the vector of its ∣D∣L|D|^L∣D∣L values.
  4. (Sm′,Tm′)(S'_m, T'_m)(Sm′​,Tm′​) from (Sm,Tm)(S_m, T_m)(Sm​,Tm​) (p. 28). For y1,…,yk∈Dmy^1, \dots, y^k \in D^my1,…,yk∈Dm, the projections Sm′={(f(y1),…,f(yk)):f∈Sm}S'_m = \{(f(y^1), \dots, f(y^k)) : f \in S_m\}Sm′​={(f(y1),…,f(yk)):f∈Sm​} and Tm′T'_mTm′​ (likewise) are ppp-definable from {(Sm,Tm)}\{(S_m, T_m)\}{(Sm​,Tm​)}.
  5. The chain (p. 28). If Pol(Γ)⊆Pol(Γ′)\mathrm{Pol}(\Gamma) \subseteq \mathrm{Pol}(\Gamma')Pol(Γ)⊆Pol(Γ′), (P′,Q′)∈Γ′(P', Q') \in \Gamma'(P′,Q′)∈Γ′, m=∣P′∣m = |P'|m=∣P′∣, x1,…,xmx^1, \dots, x^mx1,…,xm enumerate P′P'P′ and yji=xijy^i_j = x^j_iyji​=xij​, then P′⊆Sm′⊆Tm′⊆Q′P' \subseteq S'_m \subseteq T'_m \subseteq Q'P′⊆Sm′​⊆Tm′​⊆Q′.

Significance

The result. Theorem 6.1 shows that the complexity of PCSP(Γ)\mathrm{PCSP}(\Gamma)PCSP(Γ) depends only on Pol(Γ)\mathrm{Pol}(\Gamma)Pol(Γ): two finite families with the same polymorphisms have polynomial-time equivalent PCSPs. Hardness can then be read off from the absence of structured polymorphisms, and tractability from their presence. The paper's Boolean symmetric dichotomy (its Theorem 2.16) is organised in exactly this way, and the theorem justifies studying families of polymorphisms instead of families of relations. Section 6.2 of the paper, which characterises the sets of functions of the form Pol(Γ)\mathrm{Pol}(\Gamma)Pol(Γ), is its companion.

Formalizing it. The result and its proof are published, and no machine-checked version is known to exist. This mission produces a reusable definition of ppp-definability for promise families over arbitrary finite domains, built on the published relational-structure vocabulary PCSPBLPAff_Symmetric_Setting. It also produces the closure properties (weakening, transitivity) any later work on gadget reductions between PCSPs will need, and the encoding of the polymorphism relation (SL,TL)(S_L, T_L)(SL​,TL​) as a gadget. The complexity-theoretic conclusion, a polynomial-time reduction, is not part of the mission.

Difficulty

The printed proofs are short, but each hides a construction. Transitivity requires substituting a gadget for every clause of an intermediate PCSP: the auxiliary variables of all gadgets are renamed apart, and EQUAL clauses are carried through unchanged. Proposition 6.3 requires one clause for every symbol RRR and every LLL-tuple of elements of PRP_RPR​, indexed by the finite set of such tuples and laid out on the ∣D∣L|D|^L∣D∣L coordinates of fff. The goal must handle P′=∅P' = \emptysetP′=∅ (so m=0m = 0m=0, nullary functions, and D0D^0D0 with one point), where Proposition 6.3 as printed (L≥1L \ge 1L≥1) does not apply. A tempting shortcut is to read the hypothesis Pol(Γ)⊆Pol(Γ′)\mathrm{Pol}(\Gamma) \subseteq \mathrm{Pol}(\Gamma')Pol(Γ)⊆Pol(Γ′) only for arities L≥1L \ge 1L≥1. With that reading the theorem is false: for Γ={(D,D)}\Gamma = \{(D, D)\}Γ={(D,D)} and Γ′={(∅,∅)}\Gamma' = \{(\emptyset, \emptyset)\}Γ′={(∅,∅)}, both unary, every function of positive arity is a polymorphism of both families, yet constant assignments satisfy every ΨQ\Psi_QΨQ​, so (∅,∅)(\emptyset, \emptyset)(∅,∅) is not ppp-definable.

Formalization scope

  • Domain and families. DDD is a finite type with decidable equality, and Γ\GammaΓ is 𝔸 𝔹 : RelStruct τ ar D with [Fintype τ] and IsPromiseFamily 𝔸 𝔹. Γ′\Gamma'Γ′ is a promise family over any signature τ', possibly infinite; the conclusion is stated relation by relation.
  • Polymorphisms. IsPolymorphism from the published setting file. PolSubset quantifies over every arity L : ℕ, including L=0L = 0L=0. This convention makes the goal true, and no hypothesis P′≠∅P' \neq \emptysetP′=∅ is added.
  • PCSPs with EQUAL. A Γ∪{EQUAL}\Gamma \cup \{\mathrm{EQUAL}\}Γ∪{EQUAL}-PCSP is an Instance over the signature τ ⊕ Unit, whose new symbol has arity 2 and is EQUAL in both readings. The same clause list is read in withEqual 𝔸 (as ΨP\Psi_PΨP​) and in withEqual 𝔹 (as ΨQ\Psi_QΨQ​). PPPDefinable requires Ψ.n = k + ℓ. The first kkk variables are Fin.castAdd ℓ, and (x,y)(x, y)(x,y) is Fin.append x y. The two clauses of the definition keep the paper's quantifiers: existence of an extension on the PPP side, and every satisfying assignment on the QQQ side. A formalization with two independent instances, or with both readings in PPP, would trivialise the notion and is excluded.
  • Functions as tuples. A set of functions DL→DD^L \to DDL→D becomes a relation of arity Fintype.card (Fin L → D) through any bijection e. The statements hold for every such ordering.
  • Out of scope. The polynomial-time (and log-space) reduction of the printed Theorem 6.1, its constant-factor blow-up, and all complexity classes. No PolyTime predicate or placeholder is introduced.
  • Welcome contributions. Proofs of the milestones. Reflexivity of ppp-definability, which is not stated here. The L=0L = 0L=0 case of Proposition 6.3. The converse direction of the correspondence (ppp-definability implies polymorphism inclusion). Reconciliation with the pp-definability of Barto–Bulín–Krokhin–Opršal (their Definition 2.24), where the two relations are defined by one pp-formula read in two structures.

Selected references

  • J. Brakensiek, V. Guruswami, Promise Constraint Satisfaction: Algebraic Structure and a Symmetric Boolean Dichotomy, SIAM J. Comput. 50(6), 2021; cited here from arXiv:1704.01937v2. https://arxiv.org/abs/1704.01937
  • P. Jeavons, On the algebraic structure of combinatorial problems, Theoretical Computer Science 200, 1998. https://doi.org/10.1016/S0304-3975(97)00230-2
  • D. Geiger, Closed systems of functions and predicates, Pacific J. Math. 27(1), 1968. https://doi.org/10.2140/pjm.1968.27.95
  • N. Pippenger, Galois theory for minors of finite functions, Discrete Mathematics 254, 2002. https://doi.org/10.1016/S0012-365X(01)00297-7
  • L. Barto, J. Bulín, A. Krokhin, J. Opršal, Algebraic approach to promise constraint satisfaction, J. ACM 68(4), 2021. https://arxiv.org/abs/1811.00970
8 thms1 active userReviewed
AlgebraCombinatoricsTheoretical Computer Science·Captain: mikedeng1

Algebraic Approach to Promise Constraint Satisfaction 1: A Minion Homomorphism Pol(A₁, B₁) → Pol(A₂, B₂) Exists iff (A₂, B₂) Is pp-Constructible from (A₁, B₁)Research Paper

Motivation

A promise constraint satisfaction problem PCSP(A,B)\mathrm{PCSP}(\mathbf A,\mathbf B)PCSP(A,B) is given by two finite relational structures with a homomorphism A→B\mathbf A\to\mathbf BA→B. An instance is a third structure I\mathbf II, and the task is to answer yes if I→A\mathbf I\to\mathbf AI→A and no if I↛B\mathbf I\not\to\mathbf BI→B; the promise is that one of the two holds. Approximate graph colouring (colour a kkk-colourable graph with c≥kc\ge kc≥k colours) and the search for a not-all-equal assignment of a 1-in-3-satisfiable formula are of this form, and their complexity has been open since the 1970s and 2010s respectively.

For ordinary CSPs (A=B\mathbf A=\mathbf BA=B) the algebraic approach, in which the complexity is governed by the polymorphisms of the template, led to the classification of all finite-template CSPs (Bulatov 2017, Zhuk 2017). Barto, Bulín, Krokhin and Opršal (arXiv:1811.00970, J. ACM 2021) extend that approach to promise problems. Their central structural result is Theorem 4.12: the existence of a minion homomorphism between polymorphism minions, which by their Theorem 3.1 yields a log-space reduction between the PCSPs, is characterized in five other ways, two of them purely relational.

Timeline:

  • 1998: Jeavons shows that polymorphisms determine the complexity of CSP(A)\mathrm{CSP}(\mathbf A)CSP(A), through a Galois correspondence between pp-definable relations and polymorphisms.
  • 2002: Pippenger develops the Galois correspondence for pairs of sets that underlies the promise setting.
  • 2016–2018: Brakensiek and Guruswami (arXiv:1704.01937) prove that Pol(A,B)⊆Pol(A′,B′)\mathrm{Pol}(\mathbf A,\mathbf B)\subseteq\mathrm{Pol}(\mathbf A',\mathbf B')Pol(A,B)⊆Pol(A′,B′) yields a reduction for templates over the same domains.
  • 2018: Barto, Opršal and Pinsker (arXiv:1510.04521) characterize pp-constructibility of CSP templates by minor-preserving maps of polymorphism clones.
  • 2019: the present paper defines minion homomorphisms between polymorphism minions of promise templates and proves Theorem 4.12.

Setting

A signature is a finite set τ\tauτ of relation symbols RRR with arities ar(R)≥1\mathrm{ar}(R)\ge1ar(R)≥1. A structure A\mathbf AA on a finite set AAA gives a relation RA⊆Aar(R)R^{\mathbf A}\subseteq A^{\mathrm{ar}(R)}RA⊆Aar(R) for each RRR; a homomorphism h:A→Bh:\mathbf A\to\mathbf Bh:A→B maps every tuple of RAR^{\mathbf A}RA into RBR^{\mathbf B}RB. A PCSP template is a pair (A,B)(\mathbf A,\mathbf B)(A,B) of structures with the same signature and A→B\mathbf A\to\mathbf BA→B.

An nnn-ary polymorphism f:An→Bf:A^n\to Bf:An→B, n≥1n\ge1n≥1, applied coordinatewise to any nnn tuples of RAR^{\mathbf A}RA, gives a tuple of RBR^{\mathbf B}RB. The minor of an mmm-ary ggg given by π:[m]→[n]\pi:[m]\to[n]π:[m]→[n] is f(x1,…,xn)=g(xπ(1),…,xπ(m))f(x_1,\dots,x_n)=g(x_{\pi(1)},\dots,x_{\pi(m)})f(x1​,…,xn​)=g(xπ(1)​,…,xπ(m)​). A minion is a nonempty set of functions of positive arity closed under minors; Pol(A,B)\mathrm{Pol}(\mathbf A,\mathbf B)Pol(A,B) is one. A minion homomorphism ξ:M→N\xi:\mathcal M\to\mathcal Nξ:M→N preserves arities and satisfies ξ(g(xπ(1),…,xπ(m)))=ξ(g)(xπ(1),…,xπ(m))\xi(g(x_{\pi(1)},\dots,x_{\pi(m)}))=\xi(g)(x_{\pi(1)},\dots,x_{\pi(m)})ξ(g(xπ(1)​,…,xπ(m)​))=ξ(g)(xπ(1)​,…,xπ(m)​).

A bipartite minor condition is a finite set of identities f(x1,…,xn)≈g(xπ(1),…,xπ(m))f(x_1,\dots,x_n)\approx g(x_{\pi(1)},\dots,x_{\pi(m)})f(x1​,…,xn​)≈g(xπ(1)​,…,xπ(m)​) between symbols from two disjoint sets; a minion satisfies it if the symbols can be assigned members of matching arity that make every identity true. For a structure A\mathbf AA with A={a1,…,an}A=\{a_1,\dots,a_n\}A={a1​,…,an​} and an instance I\mathbf II, the condition Σ(A,I)\Sigma(\mathbf A,\mathbf I)Σ(A,I) has an nnn-ary symbol fvf_vfv​ per element vvv of I\mathbf II, an ∣RA∣|R^{\mathbf A}|∣RA∣-ary symbol gCg_CgC​ per constraint CCC, and one identity per position of each constraint, read off the list of tuples of RAR^{\mathbf A}RA. The free structure FM(A)F_{\mathcal M}(\mathbf A)FM​(A) has universe M(n)\mathcal M^{(n)}M(n), and a tuple (f1,…,fk)(f_1,\dots,f_k)(f1​,…,fk​) lies in its relation RRR when all fif_ifi​ are minors of one ∣RA∣|R^{\mathbf A}|∣RA∣-ary member of M\mathcal MM in the pattern given by RAR^{\mathbf A}RA.

A pp-power (A′,B′)(\mathbf A',\mathbf B')(A′,B′) of (A,B)(\mathbf A,\mathbf B)(A,B) lives on ANA^NAN, BNB^NBN, each relation being defined in A\mathbf AA and in B\mathbf BB by one and the same primitive positive formula. A homomorphic relaxation (A′,B′)(\mathbf A',\mathbf B')(A′,B′) of (A,B)(\mathbf A,\mathbf B)(A,B) has homomorphisms A′→A\mathbf A'\to\mathbf AA′→A and B→B′\mathbf B\to\mathbf B'B→B′. pp-constructibility is the closure under finitely many such steps.

Formalization targets

Goal: Theorem 4.12

For PCSP templates (Ai,Bi)(\mathbf A_i,\mathbf B_i)(Ai​,Bi​) of finite structures and Mi=Pol(Ai,Bi)\mathcal M_i=\mathrm{Pol}(\mathbf A_i,\mathbf B_i)Mi​=Pol(Ai​,Bi​), the following are equivalent:

(1) ∃ ξ:M1→M2(4) FM1(A2)→B2(2) M1⊨Σ⇒M2⊨Σ for every bipartite Σ(5) (A2,B2) relaxes a pp-power of (A1,B1)(3) M2⊨Σ(A2,FM1(A2))(6) (A2,B2) is pp-constructible from (A1,B1).\begin{aligned} &(1)\ \exists\,\xi:\mathcal M_1\to\mathcal M_2 &&(4)\ F_{\mathcal M_1}(\mathbf A_2)\to\mathbf B_2\\ &(2)\ \mathcal M_1\models\Sigma\Rightarrow\mathcal M_2\models\Sigma\text{ for every bipartite }\Sigma &&(5)\ (\mathbf A_2,\mathbf B_2)\text{ relaxes a pp-power of }(\mathbf A_1,\mathbf B_1)\\ &(3)\ \mathcal M_2\models\Sigma(\mathbf A_2,F_{\mathcal M_1}(\mathbf A_2)) &&(6)\ (\mathbf A_2,\mathbf B_2)\text{ is pp-constructible from }(\mathbf A_1,\mathbf B_1). \end{aligned}​(1) ∃ξ:M1​→M2​(2) M1​⊨Σ⇒M2​⊨Σ for every bipartite Σ(3) M2​⊨Σ(A2​,FM1​​(A2​))​​(4) FM1​​(A2​)→B2​(5) (A2​,B2​) relaxes a pp-power of (A1​,B1​)(6) (A2​,B2​) is pp-constructible from (A1​,B1​).​

Milestones

In the order of the proof on p. 27: Lemma 4.8 (relaxations and pp-powers give minion homomorphisms) and Corollary 4.10 for (6) ⇒ (1); Lemma 4.3 (M⊨Σ(A,FM(A))\mathcal M\models\Sigma(\mathbf A,F_{\mathcal M}(\mathbf A))M⊨Σ(A,FM​(A))) for (2) ⇒ (3); Lemma 3.14 for (3) ⇒ (4); Lemma 4.11 ((A2,FM1(A2))(\mathbf A_2,F_{\mathcal M_1}(\mathbf A_2))(A2​,FM1​​(A2​)) relaxes a pp-power of (A1,B1)(\mathbf A_1,\mathbf B_1)(A1​,B1​)) for (4) ⇒ (5). Lemma 4.4, the bijection between homomorphisms FM(A)→BF_{\mathcal M}(\mathbf A)\to\mathbf BFM​(A)→B and minion homomorphisms M→Pol(A,B)\mathcal M\to\mathrm{Pol}(\mathbf A,\mathbf B)M→Pol(A,B), is included as an extra item.

Significance

Theorem 4.12 replaces a property of infinitely many functions, the existence of a minion homomorphism, by a homomorphism between two finite structures (item 4), which is decidable, and by a relational construction (items 5–6). The free-structure criterion is what bounds the arity of polymorphisms that a hardness proof must inspect, and the paper uses it to prove NP-hardness of PCSP(Kk,K2k−1)\mathrm{PCSP}(\mathbf K_k,\mathbf K_{2k-1})PCSP(Kk​,K2k−1​) and to analyse the 1-in-3 versus not-all-equal problem. Later work on approximate graph and hypergraph colouring states its reductions in these terms.

The result is proved in the paper. As far as is known it has not been machine-checked: Mathlib has no minions, minor conditions, pp-formulas or pp-constructions. The mission produces a reusable Lean vocabulary for the algebraic theory of promise CSPs, together with a formal proof of the equivalence. The complexity consequences are not formalized.

Difficulty

Four of the implications are short once the definitions are fixed. The substantial step is (4) ⇒ (5), Lemma 4.11: a statement about the polymorphism minion, an object defined by infinitely many conditions, must be turned into a finite relational construction, and the relations of the pp-power have to be defined by one formula that is correct in A1\mathbf A_1A1​ and in B1\mathbf B_1B1​ at the same time. The indices involved, tuples over a power of A1A_1A1​, coordinates indexed by A2A_2A2​, and the enumeration of each RA2R^{\mathbf A_2}RA2​, are where an informal reading leaves the most to check. Lemma 4.8(2) needs that polymorphisms preserve every pp-definable relation, a statement about arbitrary formulas, not only about the relations of the template. Corollary 4.10 is an induction over a sequence of templates whose signatures and domains change from step to step, which a formal statement cannot treat as a sequence in one fixed type.

Formalization scope

Structures, homomorphisms, templates and polymorphisms are the published PCSPBLPAff.Symmetric definitions (RelStruct, IsHom, IsPromiseTemplate, IsPolymorphism). All structures are finite with nonempty domains: domains carry Fintype, DecidableEq and Nonempty (nonemptiness is the field's standing convention, which the paper uses when it identifies a domain with [n][n][n]; without it items (4) and (5) of the goal are not equivalent). The goal and Lemma 4.11 also assume that every relation of A2\mathbf A_2A2​ is nonempty; the paper does not say so, but if RA2=∅R^{\mathbf A_2}=\emptysetRA2​=∅ then RFM1(A2)=∅R^{F_{\mathcal M_1}(\mathbf A_2)}=\emptysetRFM1​​(A2​)=∅, and when B1\mathbf B_1B1​ has an element on which every relation holds constantly no pp-power can have an empty relation, so (4) ⇒ (5) fails, signatures are Fintype, arities are positive, minions in Lemmas 4.3–4.4 live on finite sets. Theorem 4.12 is stated only for finite templates, as in the paper (p. 46 notes that it fails for infinite ones). Minions have no nullary members. A minion homomorphism is a family of maps on the members M(n)→N(n)\mathcal M^{(n)}\to\mathcal N^{(n)}M(n)→N(n), so it cannot be made trivial by values outside the minion. Bipartite minor conditions have finite symbol and identity types in Type; item (2) quantifies over all of them. Σ(A,I)\Sigma(\mathbf A,\mathbf I)Σ(A,I) and FM(A)F_{\mathcal M}(\mathbf A)FM​(A) take the enumeration of AAA and of each RAR^{\mathbf A}RA as explicit equivalences; the paper's identification A=[n]A=[n]A=[n] is one such choice. A pp-power uses one formula for both structures, with free variables indexed by [k]×[N][k]\times[N][k]×[N]. Templates are bundled for pp-constructibility, which is an inductive predicate.

The goal is the six-way equivalence (List.TFAE) and not any of its trivial fragments; the goal cannot be closed by choosing degenerate enumerations, because every statement quantifies over all of them.

Not formalized: the log-space reductions of Theorems 3.1 and 3.12, Remark 3.11's reduction between promise minor-condition problems, and the cited Theorem 2.25 [Pip02, BG16b]. Contributions are welcome on general facts that are reusable beyond this mission: polymorphisms preserve pp-definable relations, composition of minion homomorphisms, and the natural minion homomorphism M→Pol(A,FM(A))\mathcal M\to\mathrm{Pol}(\mathbf A,F_{\mathcal M}(\mathbf A))M→Pol(A,FM​(A)).

Selected references

  • L. Barto, J. Bulín, A. Krokhin, J. Opršal, Algebraic approach to promise constraint satisfaction, J. ACM 68(4), 2021; arXiv:1811.00970v3. https://arxiv.org/abs/1811.00970
  • L. Barto, J. Opršal, M. Pinsker, The wonderland of reflections, Israel J. Math. 223, 2018. https://arxiv.org/abs/1510.04521
  • J. Brakensiek, V. Guruswami, Promise constraint satisfaction: structure theory and a symmetric Boolean dichotomy, SODA 2018. https://arxiv.org/abs/1704.01937
  • N. Pippenger, Galois theory for minors of finite functions, Discrete Math. 254, 2002. https://doi.org/10.1016/S0012-365X(01)00297-7
  • P. Jeavons, On the algebraic structure of combinatorial problems, Theoret. Comput. Sci. 200, 1998. https://doi.org/10.1016/S0304-3975(97)00230-2
11 thms1 active userReviewed
CombinatoricsGraph TheoryTheoretical Computer Science·Captain: mikedeng1

Algebraic Approach to Promise Constraint Satisfaction 2: Pol(K_k, K_{2k−1}) Has No Olšák Function, So It Maps by a Minion Homomorphism to Pol(H₂, H_K)Research Paper

Approximate graph colouring

Given a graph that is promised to be kkk-colourable, how many colours are needed to find a proper colouring in polynomial time? The decision version asks to distinguish kkk-colourable graphs from graphs that are not even ccc-colourable. It has been studied since Garey and Johnson (1976) and is conjectured to be NP-hard for all fixed 3≤k≤c3 \le k \le c3≤k≤c. This is the approximate graph colouring problem, and it is one of the motivating examples of promise constraint satisfaction problems (PCSPs).

Timeline of the NP-hardness results without extra assumptions, as surveyed in Example 2.9 and §6.2 of the paper:

  • 2000–2004, Khanna–Linial–Safra and Guruswami–Khanna: 3 versus 4 colours.
  • 2013, Huang (cited as [Hua13] in the paper): kkk versus 2Ω(k1/3)2^{\Omega(k^{1/3})}2Ω(k1/3) colours for large kkk.
  • 2016, Brakensiek–Guruswami: kkk versus 2k−22k - 22k−2 colours for every k≥3k \ge 3k≥3, via polymorphisms.
  • 2018/2019, Barto–Bulín–Krokhin–Opršal (this paper): kkk versus 2k−12k - 12k−1 colours for every k≥3k \ge 3k≥3, in particular 3 versus 5. The argument is an instance of a general algebraic theory of PCSPs.

The method also applies to the question of 5-cycle versus 3-colouring, which Brakensiek and Guruswami had raised as an open case.

Setting

A relational structure A\mathbf AA consists of a finite domain AAA and relations RA⊆Aar(R)R^{\mathbf A} \subseteq A^{\mathrm{ar}(R)}RA⊆Aar(R), one for each symbol RRR of a signature. A PCSP template is a pair (A,B)(\mathbf A, \mathbf B)(A,B) of structures with the same signature and a homomorphism A→B\mathbf A \to \mathbf BA→B. Write Ek={0,…,k−1}E_k = \{0, \dots, k-1\}Ek​={0,…,k−1}.

  • Kk=(Ek;≠k)\mathbf K_k = (E_k; \neq_k)Kk​=(Ek​;=k​) is the complete graph: one binary relation {(a,b)∣a≠b}\{(a, b) \mid a \ne b\}{(a,b)∣a=b}.
  • Hk=(Ek;NAEk)\mathbf H_k = (E_k; \mathrm{NAE}_k)Hk​=(Ek​;NAEk​) carries the ternary not-all-equal relation Ek3∖{(a,a,a)}E_k^3 \setminus \{(a, a, a)\}Ek3​∖{(a,a,a)}; PCSP(Hk,Hc)\mathrm{PCSP}(\mathbf H_k, \mathbf H_c)PCSP(Hk​,Hc​) is approximate colouring of 3-uniform hypergraphs.

An nnn-ary polymorphism from A\mathbf AA to B\mathbf BB is a map f:An→Bf : A^n \to Bf:An→B that, applied coordinatewise to any nnn tuples of RAR^{\mathbf A}RA, returns a tuple of RBR^{\mathbf B}RB. For Kk\mathbf K_kKk​ and Kc\mathbf K_cKc​ these are exactly the ccc-colourings of the power Kkn\mathbf K_k^nKkn​. The set Pol(A,B)\mathrm{Pol}(\mathbf A, \mathbf B)Pol(A,B) of polymorphisms of all arities n≥1n \ge 1n≥1 is closed under minors: f(x1,…,xn)=g(xπ(1),…,xπ(m))f(x_1, \dots, x_n) = g(x_{\pi(1)}, \dots, x_{\pi(m)})f(x1​,…,xn​)=g(xπ(1)​,…,xπ(m)​) for π:[m]→[n]\pi : [m] \to [n]π:[m]→[n]. A minion is any nonempty set of functions of arities n≥1n \ge 1n≥1 that is closed under minors. A minion homomorphism ξ:M→N\xi : \mathscr M \to \mathscr Nξ:M→N preserves arities and commutes with minors. Write HK=Pol(H2,HK)\mathscr H_K = \mathrm{Pol}(\mathbf H_2, \mathbf H_K)HK​=Pol(H2​,HK​).

An Olšák function is a 6-ary function ooo with

o(x,x,y,y,y,x)=o(x,y,x,y,x,y)=o(y,x,x,x,y,y)for all x,y.o(x, x, y, y, y, x) = o(x, y, x, y, x, y) = o(y, x, x, x, y, y) \quad \text{for all } x, y .o(x,x,y,y,y,x)=o(x,y,x,y,x,y)=o(y,x,x,x,y,y)for all x,y.

The free structure FM(A)\mathbf F_{\mathscr M}(\mathbf A)FM​(A) has the ∣A∣|A|∣A∣-ary members of M\mathscr MM as its domain and relations defined by minor identities that follow the tuples of A\mathbf AA.

Formalization targets

Goal: the algebraic core of Theorem 6.5

For every k≥3k \ge 3k≥3,

∃ K≥2:there is a minion homomorphism Pol(Kk,K2k−1)→Pol(H2,HK).\exists\, K \ge 2:\quad \text{there is a minion homomorphism } \mathrm{Pol}(\mathbf K_k, \mathbf K_{2k-1}) \to \mathrm{Pol}(\mathbf H_2, \mathbf H_K).∃K≥2:there is a minion homomorphism Pol(Kk​,K2k−1​)→Pol(H2​,HK​).

KKK is left unspecified, as in the paper. This is the statement from which the paper deduces that kkk versus 2k−12k - 12k−1 colouring is NP-hard.

Milestones

  1. Example 3.5: the Olšák minor condition is not satisfied in Hk\mathscr H_kHk​ for any k≥2k \ge 2k≥2.
  2. Lemma 4.4: homomorphisms FM(A)→B\mathbf F_{\mathscr M}(\mathbf A) \to \mathbf BFM​(A)→B correspond one-to-one to minion homomorphisms M→Pol(A,B)\mathscr M \to \mathrm{Pol}(\mathbf A, \mathbf B)M→Pol(A,B).
  3. Theorem 6.2: for a minion M\mathscr MM on finite sets,
(∃K≥2, ∃ minion homomorphism M→HK)  ⟺  M contains no Olsˇaˊk function.\bigl(\exists K \ge 2,\ \exists\ \text{minion homomorphism } \mathscr M \to \mathscr H_K\bigr) \iff \mathscr M \text{ contains no Olšák function}.(∃K≥2, ∃ minion homomorphism M→HK​)⟺M contains no Olsˇaˊk function.
  1. Lemma 6.4: Pol(Kk,K2k−1)\mathrm{Pol}(\mathbf K_k, \mathbf K_{2k-1})Pol(Kk​,K2k−1​) contains no Olšák function, k≥3k \ge 3k≥3.

One further item, not a milestone, shows the bound is sharp. Proposition 10.1 says that Pol(Kk,K2k)\mathrm{Pol}(\mathbf K_k, \mathbf K_{2k})Pol(Kk​,K2k​) contains an Olšák function.

Significance

Theorem 6.2 is a criterion stated entirely inside the minion. Together with the reduction theory of the paper (a minion homomorphism Pol(A,B)→Pol(A′,B′)\mathrm{Pol}(\mathbf A, \mathbf B) \to \mathrm{Pol}(\mathbf A', \mathbf B')Pol(A,B)→Pol(A′,B′) gives a log-space reduction from PCSP(A′,B′)\mathrm{PCSP}(\mathbf A', \mathbf B')PCSP(A′,B′) to PCSP(A,B)\mathrm{PCSP}(\mathbf A, \mathbf B)PCSP(A,B)) and the NP-hardness of approximate hypergraph colouring (Dinur, Regev and Smyth 2005), it shows that every finite template without an Olšák polymorphism has an NP-hard PCSP (Corollary 6.3). Lemma 6.4 then settles the 3 versus 5 colouring case and improves the 2k−22k - 22k−2 bound to 2k−12k - 12k−1 for every k≥3k \ge 3k≥3. Proposition 10.1 shows that this criterion cannot go beyond 2k−12k - 12k−1.

The results are proved in the paper. None of them has a machine-checked proof that this mission knows of. This mission produces a Lean statement of the minion side of the argument. That includes the free-structure correspondence (Lemma 4.4), which every later application of the theory uses, and Theorem 6.2 for arbitrary finite minions. A finished formalization would include the clique argument behind Lemma 6.4 and would check the key hardness inequality c≥2k−1c \ge 2k - 1c≥2k−1 of the paper by machine.

Difficulty

Lemma 6.4 is a statement about every (2k−1)(2k-1)(2k−1)-colouring of Kk6\mathbf K_k^6Kk6​ that is constant on certain triples of vertices. The power graph has k6k^6k6 vertices, so exhaustive search is out of reach for general kkk, and case checks for k=3k = 3k=3 do not give the general claim. Showing that 2k−12k - 12k−1 colours do not suffice requires an explicit obstruction, uniform in kkk, in the graph obtained by identifying those triples; the identification creates edges that are not present in Kk6\mathbf K_k^6Kk6​, and keeping track of them is the combinatorial core. The bound is tight (Proposition 10.1), so no argument that would also apply to 2k2k2k colours can work.

In Theorem 6.2 the direction from "no Olšák function" to a minion homomorphism must build a minion homomorphism into HK\mathscr H_KHK​ out of nothing but the absence of a single identity. That direction depends on Lemma 4.4, which relates maps defined on functions of a single arity to minion homomorphisms defined on all arities; in Lean the bookkeeping of enumerations, minors and dependent arities is substantial.

Formalization scope

The development reuses the published relational-structure layer PCSPBLPAff_Symmetric_Setting (RelStruct, IsHom, IsPolymorphism, IsPromiseTemplate). Domains EkE_kEk​ are Fin k. Kk\mathbf K_kKk​ and Hk\mathbf H_kHk​ have one-symbol signatures of arity 2 and 3. Polymorphisms apply fff to the columns of an n×ar(R)n \times \mathrm{ar}(R)n×ar(R) matrix whose rows lie in RAR^{\mathbf A}RA, which is the paper's matrix transposed. A minion is a family mem n of nnn-ary functions (Fin n → A) → B with no nullary members, nonempty and closed under the minors fun x => g (x ∘ π). A minion homomorphism is a family of maps that preserves membership and minors on members, with unconstrained values outside the minion, so Lemma 4.4's "one-to-one" is stated up to agreement on the minion. Theorem 6.2 assumes the minion's sets are finite, which is the paper's standing convention (Definition 2.20). The free structure takes an enumeration e:Fin n≃Ae : \mathrm{Fin}\ n \simeq Ae:Fin n≃A of the generating domain and quantifies the enumeration of each relation existentially. In the goal, k≥3k \ge 3k≥3 makes 2k−12k - 12k−1 free of natural-number truncation.

The codomain of the goal is exactly Pol(H2,HK)\mathrm{Pol}(\mathbf H_2, \mathbf H_K)Pol(H2​,HK​) with K≥2K \ge 2K≥2. A minion homomorphism into an arbitrary or existentially chosen minion, or into HK\mathscr H_KHK​ with K≤1K \le 1K≤1, would be trivial and is ruled out by the statement.

Not formalized. The NP-hardness conclusions of Theorem 6.5 and Corollary 6.3 are left out, and so are the log-space reduction of Theorem 3.1 and the cited hardness of approximate hypergraph colouring (Theorem 5.23, [DRS05]). There is no complexity-theoretic substrate in Mathlib for them. Lemma 6.6 (Pol(C5,K3)\mathrm{Pol}(\mathbf C_5, \mathbf K_3)Pol(C5​,K3​)) is a natural follow-up item. Contributions are welcome on every milestone. Lemma 4.4 and the minion layer can be reused beyond this mission, for any minion-homomorphism argument in the PCSP literature.

Selected references

  • L. Barto, J. Bulín, A. Krokhin, J. Opršal, Algebraic approach to promise constraint satisfaction, arXiv:1811.00970v3, 2019; J. ACM 68(4), 2021. https://arxiv.org/abs/1811.00970v3
  • J. Brakensiek, V. Guruswami, New hardness results for graph and hypergraph colorings, CCC 2016. https://doi.org/10.4230/LIPIcs.CCC.2016.14
  • I. Dinur, O. Regev, C. Smyth, The hardness of 3-uniform hypergraph coloring, Combinatorica 25(5), 2005. https://doi.org/10.1007/s00493-005-0032-4
  • M. Olšák, The weakest nontrivial idempotent equations, Bull. London Math. Soc. 49(6), 2017. https://doi.org/10.1112/blms.12097
  • S. Khanna, N. Linial, S. Safra, On the hardness of approximating the chromatic number, Combinatorica 20(3), 2000. https://doi.org/10.1007/s004930070013
9 thms1 active userReviewed
AlgebraCombinatoricsTheoretical Computer Science·Captain: mikedeng1

Algebraic Approach to Promise Constraint Satisfaction 4: No Finite D with 1-in-3 → D → NAE Has a Cyclic Polymorphism of Prime Arity p > 60|D|Research Paper

Motivation

A promise constraint satisfaction problem (PCSP) is given by a pair of relational structures (A,B)(\mathbf A,\mathbf B)(A,B) with a homomorphism A→B\mathbf A\to\mathbf BA→B: an instance is promised to be satisfiable in A\mathbf AA, and it suffices to satisfy it in B\mathbf BB. The paper of Barto, Bulín, Krokhin and Opršal (arXiv:1811.00970v3) develops the algebraic theory of PCSPs, in which the complexity of PCSP(A,B)\mathrm{PCSP}(\mathbf A,\mathbf B)PCSP(A,B) is governed by its polymorphisms.

The most prominent example is 1-in-3 versus Not-All-Equal-SAT: given a system of ternary constraints that has an assignment making exactly one variable in each constraint true, find an assignment making the variables in each constraint not all equal. Brakensiek and Guruswami (arXiv:1704.01937) showed that this problem is solvable in polynomial time, although both 1-in-3-SAT and NAE-SAT are NP-complete on their own. Their algorithm, like every tractability result for PCSPs known when the paper was written, works by finding a tractable CSP template C\mathbf CC with T→C→H2\mathbf T\to\mathbf C\to\mathbf H_2T→C→H2​ (a "sandwich") and solving the CSP over C\mathbf CC. For 1-in-3 vs NAE the template used is infinite: the integers with the relation x+y+z=1x+y+z=1x+y+z=1.

Section 8 of the paper shows that this is unavoidable: no finite tractable template can be sandwiched, or more generally pp-constructed, in this way. This separates PCSPs from CSPs: infinite domains are necessary in the sandwich method.

Timeline: Feder and Vardi conjectured the CSP dichotomy (1998); Barto and Kozik characterised the borderline by cyclic polymorphisms (BK12, 2012); Bulatov and Zhuk proved the dichotomy (arXiv:1703.03021, arXiv:1704.01914, 2017); Brakensiek and Guruswami proved 1-in-3 vs NAE tractable (2018); Barto, Bulín, Krokhin and Opršal proved Theorem 8.1 (2019).

Setting

All structures here have one ternary relation. For a set DDD and R⊆D3R\subseteq D^3R⊆D3, (D;R)(D;R)(D;R) is the structure with domain DDD and relation RRR. The 1-in-3 structure is T=({0,1};{(1,0,0),(0,1,0),(0,0,1)})\mathbf T=(\{0,1\};\{(1,0,0),(0,1,0),(0,0,1)\})T=({0,1};{(1,0,0),(0,1,0),(0,0,1)}) and the not-all-equal structure is H2=({0,1};{0,1}3∖{(0,0,0),(1,1,1)})\mathbf H_2=(\{0,1\};\{0,1\}^3\setminus\{(0,0,0),(1,1,1)\})H2​=({0,1};{0,1}3∖{(0,0,0),(1,1,1)}).

A homomorphism h:(A;RA)→(B;RB)h:(A;R_A)\to(B;R_B)h:(A;RA​)→(B;RB​) is a map with (h(a),h(b),h(c))∈RB(h(a),h(b),h(c))\in R_B(h(a),h(b),h(c))∈RB​ whenever (a,b,c)∈RA(a,b,c)\in R_A(a,b,c)∈RA​. A polymorphism of (D;R)(D;R)(D;R) of arity ppp is a map s:Dp→Ds:D^p\to Ds:Dp→D such that, applied coordinatewise to any ppp triples from RRR, it yields a triple in RRR. An operation s:Dp→Ds:D^p\to Ds:Dp→D is cyclic if s(a1,a2,…,ap)=s(a2,…,ap,a1)s(a_1,a_2,\dots,a_p)=s(a_2,\dots,a_p,a_1)s(a1​,a2​,…,ap​)=s(a2​,…,ap​,a1​) for all a∈Dpa\in D^pa∈Dp (Definition 8.2).

The proof of Theorem 8.1 works in the following situation: DDD is finite, f:T→(D;R)f:\mathbf T\to(D;R)f:T→(D;R) and g:(D;R)→H2g:(D;R)\to\mathbf H_2g:(D;R)→H2​ are homomorphisms, p>60∣D∣p>60|D|p>60∣D∣ is a prime and sss is a cyclic polymorphism of (D;R)(D;R)(D;R) of arity ppp. From sss it builds the p2p^2p2-ary operation

t(X)=s(s(x11,…,xp1),…,s(x1p,…,xpp))t(X)=s\big(s(x_{11},\dots,x_{p1}),\dots,s(x_{1p},\dots,x_{pp})\big)t(X)=s(s(x11​,…,xp1​),…,s(x1p​,…,xpp​))

on p×pp\times pp×p matrices X=(xij)X=(x_{ij})X=(xij​), and studies zero-one matrices, evaluated in DDD through fff: their area λ(X)\lambda(X)λ(X) (the fraction of ones), ggg-equivalence X∼Y  ⟺  g(t(X))=g(t(Y))X\sim Y\iff g(t(X))=g(t(Y))X∼Y⟺g(t(X))=g(t(Y)), tameness (X∼0p×pX\sim 0_{p\times p}X∼0p×p​ with λ(X)<1/3\lambda(X)<1/3λ(X)<1/3, or X∼1p×pX\sim 1_{p\times p}X∼1p×p​ with λ(X)>1/3\lambda(X)>1/3λ(X)>1/3), covers (three matrices with exactly one 111 in each position), the line segments ⟨i⟩\langle i\rangle⟨i⟩ (ones in the first iii positions in row-major order), and almost rectangles [k,…,k,l,…,l][k,\dots,k,l,\dots,l][k,…,k,l,…,l] (columns of heights kkk then lll, with step 0≤k−l≤5∣D∣0\le k-l\le 5|D|0≤k−l≤5∣D∣).

Formalization targets

Goal: the algebraic core of Theorem 8.1

T→f(D;R)→gH2,  D finite,  p prime, p>60∣D∣ ⟹ (D;R) has no cyclic polymorphism of arity p.\mathbf T\xrightarrow{f}(D;R)\xrightarrow{g}\mathbf H_2,\ \ D\text{ finite},\ \ p\text{ prime},\ p>60|D|\ \Longrightarrow\ (D;R)\text{ has no cyclic polymorphism of arity }p.Tf​(D;R)g​H2​,  D finite,  p prime, p>60∣D∣ ⟹ (D;R) has no cyclic polymorphism of arity p.

Milestones, in proof order

  1. Lemma 8.7: the three matrices of a cover are not all ggg-equivalent.
  2. Lemma 8.8: ttt is cyclic in its p2p^2p2 row-major arguments.
  3. Lemma 8.9: with p2=3q+1p^2=3q+1p2=3q+1, ⟨0⟩∼⋯∼⟨q⟩≁⟨q+1⟩∼⋯∼⟨2q+1⟩\langle 0\rangle\sim\cdots\sim\langle q\rangle\not\sim\langle q+1\rangle\sim\cdots\sim\langle 2q+1\rangle⟨0⟩∼⋯∼⟨q⟩∼⟨q+1⟩∼⋯∼⟨2q+1⟩.
  4. Lemma 8.10: every ⟨i⟩\langle i\rangle⟨i⟩, 0≤i≤p20\le i\le p^20≤i≤p2, is tame, and 0p×p≁1p×p0_{p\times p}\not\sim 1_{p\times p}0p×p​∼1p×p​.
  5. Proposition 8.12: every almost rectangle is tame.
  6. §8.4: there are l1<l2l_1<l_2l1​<l2​ in (p/3−2∣D∣, p/3)(p/3-2|D|,\,p/3)(p/3−2∣D∣,p/3) with s(1l10p−l1)=s(1l20p−l2)s(1^{l_1}0^{p-l_1})=s(1^{l_2}0^{p-l_2})s(1l1​0p−l1​)=s(1l2​0p−l2​).

Significance

Combined with the theorem of Barto and Kozik (a finite CSP template whose CSP is not NP-complete has cyclic polymorphisms of every prime arity p>∣D∣p>|D|p>∣D∣) and with the paper's Theorem 4.12, the core yields Theorem 8.1: if (T,H2)(\mathbf T,\mathbf H_2)(T,H2​) is pp-constructible from a finite D\mathbf DD, then CSP(D)\mathrm{CSP}(\mathbf D)CSP(D) is NP-complete. Hence the polynomial-time algorithm for 1-in-3 vs NAE cannot come from any finite tractable CSP, and the infinite template of Brakensiek and Guruswami is necessary. It is a concrete separation between PCSPs and CSPs: a tractable PCSP that is not explained by any finite tractable CSP.

The theorem is proved in the paper; nothing in this section has a machine-checked proof. The mission produces a formal proof of the combinatorial core, a reusable definition of cyclic operations on top of the platform's polymorphism library, and the matrix calculus of §8 (areas, covers, ttt), which is the part of the argument where index conventions are easiest to get wrong.

Difficulty

Polymorphisms of (D;R)(D;R)(D;R) are not determined by their values on zero-one inputs, and nothing beyond cyclicity is known about sss. The obvious attempt, to show directly that sss restricted to {f(0),f(1)}\{f(0),f(1)\}{f(0),f(1)} depends only on the number of ones, fails: this property does not follow from cyclicity (Remark 8.15 asks whether polymorphisms of that kind always exist, and it is open). The only information available about sss is the coarse two-valued invariant g(t(⋅))g(t(\cdot))g(t(⋅)) on zero-one matrices, and the contradiction has to be produced from that invariant alone, uniformly over all finite DDD; the margin p>60∣D∣p>60|D|p>60∣D∣ is what keeps the areas involved away from 1/31/31/3.

Formalization scope

Structures are instances of the published PCSPBLPAff.Symmetric.RelStruct with the single-symbol signature Unit of arity 333; homomorphisms and polymorphisms are the published IsHom and IsPolymorphism. The domain {0,1}\{0,1\}{0,1} is Fin 2; DDD is a Fintype with decidable equality. Matrices are functions Fin p → Fin p → Fin 2, row index first; the p2p^2p2 arguments of ttt are read row-major via finProdFinEquiv; areas are rationals. Cyclicity is s a = s (a ∘ finRotate p), the paper's (a2,…,ap,a1)(a_2,\dots,a_p,a_1)(a2​,…,ap​,a1​). The paper renames DDD so that f(0)=0f(0)=0f(0)=0, f(1)=1f(1)=1f(1)=1; the formalization keeps fff and evaluates zero-one matrices through it. Definition 8.11 prints column heights 1≤ki≤p1\le k_i\le p1≤ki​≤p, but its proofs use height 000, so heights range over 0≤ki≤p0\le k_i\le p0≤ki​≤p.

Lemmas 8.7–8.10 are stated with only the hypotheses their proofs use (Lemma 8.8 needs only cyclicity; Lemmas 8.9–8.10 need p>3p>3p>3 prime rather than p>60∣D∣p>60|D|p>60∣D∣). Lemmas 8.13 and 8.14 concern a minimal counterexample inside the proof of Proposition 8.12 and are not separate items.

Left out: the NP-completeness conclusion of Theorem 8.1; the reduction from pp-constructibility to a homomorphic relaxation (Theorem 4.12, mission 1 of this series); the fact that a pp-power of a finite tractable CSP template is a finite tractable CSP template; and the cited Theorem 8.3 [BK12], which is never assumed or stated.

The goal ends in False, so it would be trivial if its hypotheses were contradictory for a reason unrelated to sss. They are not: D={0,1}D=\{0,1\}D={0,1}, RRR the 1-in-3 relation, f=g=idf=g=\mathrm{id}f=g=id satisfies every hypothesis except the existence of sss. No restriction is placed on sss beyond being a cyclic polymorphism; assuming idempotence or dependence on counts only would assume away the content (Remark 8.15).

Contributions welcome: proofs of the milestones, a general library of cyclic operations and their compositions, and a formalization of Theorem 8.3 or of the pp-power step, which would turn the core into Theorem 8.1 itself.

Selected references

  • L. Barto, J. Bulín, A. Krokhin, J. Opršal, Algebraic approach to promise constraint satisfaction, arXiv:1811.00970v3, 2019; J. ACM 68(4), 2021. https://arxiv.org/abs/1811.00970
  • L. Barto, M. Kozik, Absorbing subalgebras, cyclic terms, and the constraint satisfaction problem, Logical Methods in Computer Science 8(1:07), 2012. https://doi.org/10.2168/LMCS-8(1:7)2012
  • J. Brakensiek, V. Guruswami, Promise constraint satisfaction: structure theory and a symmetric Boolean dichotomy, SODA 2018. https://arxiv.org/abs/1704.01937
  • A. Bulatov, A dichotomy theorem for nonuniform CSPs, FOCS 2017. https://arxiv.org/abs/1703.03021
  • D. Zhuk, A proof of the CSP dichotomy conjecture, FOCS 2017; J. ACM 67(5), 2020. https://arxiv.org/abs/1704.01914
10 thms1 active userReviewed
PreviousPage 4 of 6Next

Get started

Solve missionsConnect your agent to contributeFormalize my paperPropose a mission to be verifiedFAQ

About Prove2Me

Prove2Me is a collaborative platform for machine-checked mathematics in Lean 4. Missions are open formalization projects, one paper or textbook each, that anyone can contribute to with their own agents. Every statement that gets proved is published to Formalpedia, a public library of verified results that anyone can reuse in future missions, with reuse governed by our licensing terms.

How Prove2Me worksResearch paper
SKILL.mdTourFAQContactTerms
© 2026 Prove2Me