Fast Polynomial Factorization and Modular Composition 3: The Frobenius Lift φ and Projection π Recover f(α) from the Kronecker Substitution f*(Y) = f(Y, Y^h, …, Y^{h^{m−1}})Research Paper
Motivation
Multivariate multipoint evaluation asks for the values of one polynomial at points of . Kedlaya and Umans showed that this problem is equivalent to modular composition, the computation of for univariate polynomials, and that fast algorithms for it yield the fastest known algorithms for factoring univariate polynomials over finite fields (Kedlaya–Umans, Dagstuhl Seminar Proceedings 08381, 2008; journal version SIAM J. Comput. 40(6), 2011, doi:10.1137/08073408X). Modular composition is the bottleneck of the factoring algorithms of von zur Gathen and Shoup (1992) and Kaltofen and Shoup (1998), so a near-linear multipoint evaluation algorithm translates directly into a faster factoring algorithm.
The paper gives two such algorithms. Section 4 works over any ring and is not algebraic: it lifts the problem to the integers and reduces modulo many small primes. Section 6, the subject of this mission, gives an algebraic algorithm, using only ring operations, for fields of small characteristic. Its correctness rests on one identity, Lemma 6.1, which lets the evaluation of the -variate at a point be read off from the evaluation of a single univariate polynomial at a related point of an extension ring.
Setting
Let be a prime and a finite field of characteristic . Fix integers and . Let be a power of with (the paper takes the least such power), and let be a polynomial of degree that is irreducible over . Then:
- is a field with elements. Let be a primitive element of , a generator of its multiplicative group, so has multiplicative order .
- , with viewed over . need not remain irreducible over , so is a ring containing and but in general not a field.
- , , is a power of the Frobenius endomorphism; is . On it is an automorphism, with inverse .
- and . Every element of has a canonical representative, its remainder modulo the monic , of degree less than .
Three maps connect the problem to :
- the lift sends to the class of the polynomial of degree at most with for (Eq. (6.1));
- the projection sends an element with canonical representative to ;
- the Kronecker substitution sends to the univariate
In the Lean development these are K, R, h, ι (the inclusion ), sigma i (), E, S, sigmaInv i ( on ), gAlpha, phi, pi, gIter () and fStar, in the namespace KedlayaUmans.FrobeniusLift.
Formalization targets
Goal: Lemma 6.1 (p. 19)
For every with degree at most in each variable and every ,
Milestones, in the order the proof uses them
- Properties of (p. 18): has order in , and is a unit of for in .
- Eq. (6.1) (p. 19): for ; ; .
- The Frobenius congruence (p. 19): is fixed by , and for every .
- Eq. (6.2) (p. 19): with , and for .
- No reduction (pp. 19–20): the canonical representative of is , and this remainder equals itself.
Significance
The result. Lemma 6.1 is the correctness half of Theorem 6.2: to evaluate at points one computes the lifts , evaluates the one univariate polynomial at all of them by fast univariate multipoint evaluation, and projects back with . Since univariate multipoint evaluation runs in nearly linear time, the identity turns a multivariate problem into a univariate one at the cost of the degree growth . Corollary 6.3 turns this into an operation count of for every , provided and the characteristic satisfies .
Formalizing it. The lemma is proved in the paper; no machine-checked version is known to exist. This mission states it for the objects exactly as constructed in Section 6 (an explicit interpolant for , the canonical-representative projection for ) and splits its proof into the five steps above. The definitions of the rings and , the Frobenius powers on a ring that is not a field, and the Kronecker substitution into a quotient ring are reusable for formalizing other uses of the Frobenius-lift technique, such as the Parvaresh–Vardy codes and their Guruswami–Rudra instantiation, which the paper names as the inspiration for this algorithm (§1.5, p. 6).
Difficulty
The obvious attempt is to treat as evaluation at and push it through . This fails: is a unit, not zero, so evaluation at does not factor through , and is not a ring homomorphism. The identity holds only because the polynomial is already reduced modulo , which requires both the degree bound and the fact that raising to the power does not raise the degree of modulo . The second fact depends on being fixed by and on . The equality further needs that does not kill the leading coefficient in , a ring with possible zero divisors. Finally, interpolation must happen in , not a field, which is possible only because the nodes lie in the subfield .
Formalization scope
- is a type
Fwith[Field F] [Fintype F] [CharP F p], a prime ([Fact p.Prime]). is a polynomial overZMod pwith[Fact (Irreducible P)]; isP.natDegreeand isp ^ P.natDegree. isAdjoinRootof mapped toF; is never assumed to be a field and is never assumed irreducible overF. - The paper's is the least power of above ; the statements assume only
m ^ 2 * d < h, which contains the paper's case. is a hypothesis. - "Primitive element" is
IsPrimitiveRoot η (h - 1), equivalent to generating since . - "Individual degrees " means at most :
∀ i, f.degreeOf i ≤ d - 1. - Variables and points are indexed by
Fin m, starting at ; substitutes for . - is the explicit Lagrange interpolant: Mathlib's
Lagrange.basisover the field at the nodes , mapped into , with coefficients . is the remainder modulo (AdjoinRoot.modByMonicHom) evaluated at . - Only correctness is formalized. The operation counts of Theorem 6.2 and Corollary 6.3, and the cost of finding and , are out of scope.
A formalization that takes to be any element satisfying (6.2), or defines as a ring homomorphism, would make the goal trivial or false; both are ruled out because and are explicit definitions with no hypotheses attached, and (6.1) and (6.2) are milestones to be proved.
Infrastructure a full proof needs: the Frobenius on polynomial rings in characteristic (, cf. Mathlib's expand and map_frobenius_expand), Lagrange interpolation transported along a ring homomorphism from a field, degree bounds for MvPolynomial.eval₂, and reducedness of (from separability of over the perfect field ). Proofs of any milestone, and alternative arguments, are welcome.
Selected references
- K. S. Kedlaya, C. Umans, Fast polynomial factorization and modular composition, Dagstuhl Seminar Proceedings 08381, version of Aug. 31, 2008. http://drops.dagstuhl.de/opus/volltexte/2008/1777
- K. S. Kedlaya, C. Umans, Fast polynomial factorization and modular composition, SIAM J. Comput. 40(6), 2011. https://doi.org/10.1137/08073408X
- F. Parvaresh, A. Vardy, Correcting errors beyond the Guruswami–Sudan radius in polynomial time, FOCS 2005, pp. 285–294. https://doi.org/10.1109/SFCS.2005.29
- V. Guruswami, A. Rudra, Explicit capacity-achieving list-decodable codes, STOC 2006, pp. 1–10. https://doi.org/10.1145/1132516.1132518