An Introduction to Matrix Concentration Inequalities Ch 5: Matrix ChernoffTextbook
Bound the extreme eigenvalues of a sum of independent positive-semidefinite random matrices. The goal is the complete Matrix Chernoff theorem (Theorem 5.1.1): both expectation estimates, both tails, and the identities for the mean’s extreme eigenvalues.
The supporting milestone is the matrix mgf/cgf estimate (Lemma 5.4.1); Chapter 3 supplies the master bounds. The Lean statement retains the different lower- and upper-tail deviation ranges, permits zero mean eigenvalues, and handles a zero summand bound explicitly.
An Introduction to Matrix Concentration Inequalities Ch 3: Matrix Laplace TransformTextbook
Develop the reusable matrix Laplace-transform method behind Tropp’s concentration inequalities. The goal is Theorem 3.6.1: upper and lower tail and expectation bounds for sums of independent Hermitian random matrices, expressed through their matrix cumulant-generating functions.
Milestones cover the eigenvalue tail and expectation bounds (Propositions 3.2.1–2), probabilistic Lieb inequality (Corollary 3.4.2), and trace-cgf subadditivity (Lemma 3.5.1). This mission also supplies the series’ shared spectral, probability, and dilation definitions. Lieb’s underlying concavity theorem is the Chapter 8 goal.
The Lean statements give bounds for each admissible transform parameter, with explicit integrability hypotheses; optimization recovers the source’s infimum/supremum forms.
Graph Sparsification by Effective Resistances 1: Sampling O(n log n/ε²) Edges by Effective Resistance Yields a (1±ε) Spectral Sparsifier with Probability 1/2Research Paper
Motivation
Many graph algorithms run in time proportional to the number of edges. A sparsifier of a weighted graph G is a graph H on the same vertices with far fewer edges that approximates G for the purpose at hand, so that the algorithm can be run on H instead. Benczúr and Karger (1996) showed that every graph has a sparsifier with O(nlogn/ε2) edges preserving the weight of every cut up to a factor 1±ε. Spielman and Teng (2004) introduced the stronger spectral notion, in which the Laplacian quadratic form is preserved for every real vector, not only for cut indicators; their sparsifiers had O(nlogcn) edges for a large constant c and were the first step of their nearly-linear-time solvers for symmetric diagonally dominant linear systems.
Spielman and Srivastava (arXiv:0803.0929, STOC 2008, SIAM J. Comput. 2011) proved that independent sampling of O(nlogn/ε2) edges, each edge with probability proportional to its weight times its effective resistance, yields a spectral sparsifier. The result improved both earlier bounds, replaced a recursive partitioning construction by a one-line sampling rule, and made effective resistance a standard tool in graph algorithms. Batson, Spielman and Srivastava (arXiv:0808.0163) later obtained O(n/ε2) edges deterministically, by a slower method built on the same matrix Π.
Setting
Let G=(V,E,w) be a connected weighted undirected graph with n=∣V∣ vertices, m=∣E∣ edges and weights we>0. Orient every edge arbitrarily, so that it has a head and a tail (distinct vertices).
The incidence matrixB∈Rm×n has B(e,v)=1 if v is the head of e, −1 if v is its tail, and 0 otherwise; be denotes its row for e. W is the diagonal m×m matrix with W(e,e)=we.
The Laplacian is L=BTWB, with quadratic form xTLx=∑ewe(x(heade)−x(taile))2.
L+ is the Moore–Penrose pseudoinverse of L: if L=∑i=1n−1λiuiuiT over its nonzero eigenvalues, then L+=∑i=1n−1λi−1uiuiT.
The effective resistance of the edge e is Re=beL+beT: the potential difference across e when a unit current enters at one end and leaves at the other, the edges being resistors of conductance we.
Π=W1/2BL+BTW1/2 is an m×m matrix with Π(e,e)=weRe.
Sparsify(G,q) draws q edges independently with replacement, the edge e with probability pe=weRe/∑fwfRf, and gives e the weight we/(qpe) for each time it is drawn. Its output H has Laplacian L~=BTW1/2SW1/2B, where S is the random diagonal matrix with S(e,e)=#{draws of e}/(qpe).
Formalization targets
Goal: Theorem 1
There are an absolute constant C and a threshold N0 such that, for every n≥N0, every connected weighted graph on n vertices and every 1/n<ε≤1, with q=⌈9C2nlogn/ε2⌉, with probability at least 1/2
∀x∈Rn:(1−ε)xTLx≤xTL~x≤(1+ε)xTLx.
The constant C is left unfixed; the statement asserts the shape q=O(nlogn/ε2) with the paper's explicit dependence on C.
Milestones
Lemma 3 (four parts): Π is an orthogonal projection; imΠ=imW1/2B; the eigenvalues of Π are 1 with multiplicity n−1 and 0 with multiplicity m−n+1; Π(e,e)=∥Π(⋅,e)∥2.
Lemma 4: for a nonnegative diagonal S, ∥ΠSΠ−ΠΠ∥2≤ε implies (1−ε)xTLx≤xTBTW1/2SW1/2Bx≤(1+ε)xTLx for all x.
Lemma 5 (Rudelson–Vershynin): for independent samples y1,…,yq of a random vector with ∥y∥2≤M and ∥EyyT∥2≤1, and q≥2,
Eq1j=1∑qyjyjT−EyyT2≤CMqlogqwhenever the right side is <1.
Significance
Theorem 1 says that every weighted graph is spectrally approximated by a reweighted subgraph with O(nlogn/ε2) edges. A spectral approximation preserves cut weights, the eigenvalues of the Laplacian up to 1±ε, effective resistances, and the condition number of L as a preconditioner, so any algorithm whose output depends on these quantities can run on H. Combined with fast approximate computation of effective resistances (the second mission of this series), it gives a nearly-linear-time construction, and it is the sampling step used in Laplacian solvers, in sparsification of sums of rank-one matrices, and in leverage-score sampling for regression, where weRe is exactly the statistical leverage of a row.
The result is proved in the paper; to our knowledge no machine-checked proof exists. This mission formalizes the statement and the paper's proof structure: the linear algebra of Π (Lemma 3), the deterministic reduction from quadratic forms to a spectral-norm bound (Lemma 4), and the matrix concentration inequality (Lemma 5), which is the substantive analytic input and is itself a reusable result about sums of independent rank-one matrices.
Difficulty
The deterministic part is linear algebra over the pseudoinverse. The obstacle is concentration: the expected Laplacian of H equals L, but bounding the deviation uniformly over all x∈Rn is a statement about the spectral norm of a random matrix, and a union bound over a net of directions costs a factor of n in the sample count rather than logn. Scalar Chernoff bounds per cut, which suffice for cut sparsifiers, do not give the spectral statement. Lemma 5 is the matrix inequality that removes this loss, and no inequality of this kind (a concentration bound for the spectral norm of a sum of independent random matrices) is in Mathlib.
Formalization scope
All declarations sit in the namespace EffResSparsify.Sampling.
Graphs. A structure WGraph V E with orientation maps head, tail : E → V, weights w : E → ℝ, and the fields head e ≠ tail e (no loops) and 0 < w e. Parallel edges are allowed; nothing in §3 uses simplicity. Connectivity is a separate hypothesis on the underlying simple graph and includes V=∅. In Theorem 1 the vertex type is Fin n; the edge type is an arbitrary finite type.
Pseudoinverse.L+ is the matrix of the published Moore–Penrose pseudoinverse HarmonicGames.Decomposition.pinv of x↦Lx on Euclidean RV; it equals the spectral formula above and is the gauge-fixed inverse, never "some solution of Lx=y".
Sampling.pe is defined as weRe/∑fwfRf; the identity ∑fwfRf=n−1 is part of the proof. An outcome of Sparsify is a sequence in Eq with probability ∏ipsi, and probabilities and expectations are finite sums over Eq, with no measure theory.
Norms and logarithms.∥⋅∥2 is the ℓ2 operator norm on matrices (Matrix.Norms.L2Operator); vector norms in Lemma 5 are Euclidean; log is natural (the base is absorbed into C).
Constants. In Theorem 1, ∃C>0,∃N0,∀n≥N0 precede the graph and ε; the sample count is ⌈9C2nlogn/ε2⌉, rounded up because the printed value is not an integer. In Lemma 5, ∃C>0 precedes the dimension, the distribution, M and q.
Corrected statement. Lemma 5 as printed, with right side min(CMlogq/q,1), is false (at q=1 the bound is 0). The formal milestone is Rudelson and Vershynin's own Theorem 3.1: q≥2, and the bound a=CMlogq/q holds when a<1. It is stated for finitely supported distributions, which is all Theorem 1 uses. Theorem 1 applies it with a≤ε/2, so the correction does not affect the goal.
Not a trivialization. Theorem 1 does not take Lemma 5, a bound on E∥ΠSΠ−Π∥2, or a free sample count as a hypothesis; a statement with any of these, or with "∃q" in place of ⌈9C2nlogn/ε2⌉, is a different theorem.
Reusable beyond this mission: the weighted Laplacian with its pseudoinverse and effective resistances, and Lemma 5, which applies to any sampling scheme for sums of rank-one matrices (leverage-score sampling, column subset selection). Contributions toward a matrix Chernoff or Rudelson-type inequality in Mathlib are welcome.
M. Rudelson, R. Vershynin, Sampling from large matrices: an approach through geometric functional analysis, J. ACM 54(4), 2007. https://doi.org/10.1145/1255443.1255449
All-temperature pressure for orthogonally invariant Ising spin glassesOpen Problem
Motivation
An Ising spin system probes a quadratic form only on cube vertices. The source asks whether random orthogonal eigenvectors allow its pressure to be determined by the limiting eigenvalue distribution at every positive temperature. The source is OpenAI's September 2026 manuscript.
Setting
The pressure is the normalized logarithm of the average Boltzmann weight under the uniform spin prior. Eigenvectors are Haar orthogonal, and the deterministic empirical eigenvalue laws converge weakly to a compactly supported probability law. The variational formula uses an edge-extended spectral transform.
Formalization targets
EPN(β)→Vβ,PN(β)→Vβalmost surely(β>0).
For a spin system on N Ising spins σ ∈ {−1,1}^N, the rotated pressure for eigenvalues λ₁,…,λ_N, an orthogonal rotation U and a field c is (1/N) times the log of the average over all 2^N spin configurations of exp(H(σ)), where H(σ) = (1/2)∑ᵢ λᵢ (Uσ)ᵢ² + ∑ᵢ cᵢσᵢ. The theorem states that, for any probability space (Ω,P), random N×N orthogonal matrices U_N that are measurable and whose laws are right-invariant under multiplication (Haar-type), and any deterministic eigenvalue arrays (eig_N,i), the following holds under these assumptions: ν is a probability measure on ℝ whose support lies in [a,b] and contains both a and b; for every ε>0, eventually in N all eigenvalues lie in [a−ε,b+ε]; the empirical spectral laws (1/N)∑ᵢ δ_{eig_N,i} converge weakly to ν; and the inverse temperature β is strictly positive. Then, with eigenvalues β·eig_N,i, rotation U_N(ω)⁻¹ and zero external field, the expected rotated pressure ∫ pressure dP converges as N → ∞, and the pressure itself converges for P-almost every ω, in both cases to the real part of the variational functional of the measureR transform of the β-rescaled law of ν, evaluated at the edge β·b.
The selected goal is the all-positive-temperature zero-field pressure theorem. The attached results also cover field laws, random spectra, Gaussian-pattern pressure and ground energy, and related norm limits under their stated assumptions. The manuscript presents a mathematical argument; the attached goal currently has status Open, so the requested contribution is a Lean proof of the displayed formal target.
Difficulty
Weak spectral convergence does not control rare extreme eigenvalues. The explicit no-outlier condition is essential to the selected statement’s comparison with the limiting spectral functional.
Formalization scope
The spin prior is uniform probability, and temperature is implemented by multiplying eigenvalues by β. The no-outlier condition holds eventually within every enlargement of the limiting support interval. Almost-sure convergence and expectation convergence are both conclusions; other attached variants keep their own uniform-integrability and field hypotheses.
The shared definitions are supplied by InvariantIsing. The target uses Lean 4.33.1 with the pinned Mathlib environment. Contributions should preserve the existing statement and develop reusable supporting lemmas in its definitions.
Selected references
OpenAI, All-temperature pressure for orthogonally invariant Ising spin glasses, preprint, September 2026. Manuscript.
Phase Transition of the Largest Eigenvalue for Nonnull Complex Sample Covariance Matrices 2: Spikes above 1+γ⁻¹ Give √M Fluctuations of the Largest Eigenvalue with Limit G_k, the k×k GUE LawResearch Paper
Motivation
Sample covariance matrices are the basic object of multivariate statistics: principal component analysis, signal detection and factor models all start from the eigenvalues of S=M1∑k=1Mykyk∗ computed from M samples of an N-dimensional vector. When N is comparable to M, the eigenvalues of S no longer approximate those of the population covariance Σ. The question is then whether a few large population eigenvalues ("spikes") are visible in the sample spectrum at all, and how the top sample eigenvalue fluctuates when they are.
Baik, Ben Arous and Péché (Ann. Probab. 33 (2005) 1643–1697) answered this for complex Gaussian samples. They found a sharp threshold 1+γ−1, where γ2=M/N, now called the BBP phase transition. This mission formalizes the supercritical half of the answer, Theorem 1.1(b).
Timeline.
2000–2001: Johansson (Comm. Math. Phys. 209, 2000) and Johnstone (Ann. Statist. 29) proved that for Σ=I the largest eigenvalue, centred at (1+γ−1)2 and scaled by M2/3, has the Tracy–Widom law. Johansson treated the complex case, Johnstone the real case.
2005: Baik, Ben Arous and Péché treated Σ with finitely many eigenvalues different from 1. Spikes at or below 1+γ−1 give M2/3 fluctuations with limit laws Fk (part (a), a separate mission of this series). Spikes above 1+γ−1 give M fluctuations with limit Gk (part (b)).
2006: Baik and Silverstein (J. Multivariate Anal. 97) located the outlying eigenvalue for general, non-Gaussian samples. Bai and Yao (Ann. IHP 44, 2008) obtained Gaussian fluctuations of the outliers for general samples.
Setting
Let gkj, 1≤k≤M, 1≤j≤N, be independent standard complex Gaussians: g=a+ib with a,b independent real normal variables of mean 0 and variance 1/2. Fix a unitary matrix U and positive population eigenvaluesℓ1,…,ℓN. The samples are
yk=Udiag(ℓ1,…,ℓN)gk,
which are independent mean-zero complex Gaussian vectors with covariance Σ=Udiag(ℓ)U∗. The sample covariance matrix is S=M1∑kykyk∗, and λ1 is its largest eigenvalue.
The regime has M,N→∞ with M/N=γ2 and γ in a compact subset of [1,∞). A fixed number r of the ℓj differ from 1. For some 1≤k≤r the top k coincide, ℓ1=⋯=ℓk, with common value in a compact subset of (1+γ−1,∞). The others, ℓk+1,…,ℓr, lie in a compact subset of (0,ℓ1).
The limit law is the finite GUE distribution. Let Zk=∫Rk∏i<j∣ξi−ξj∣2∏ie−ξi2/2dξ. Then
Here pn are the orthonormal polynomials for the weight e−x2/2 and cn their leading coefficients.
Significance
The result. Below the threshold the top eigenvalue sticks to the bulk edge (1+γ−1)2. Above it, Theorem 1.1(b) and Corollary 1.1(b) show that λ1 separates from the bulk to the explicit location ℓ1(1+γ−2/(ℓ1−1)). Its fluctuations shrink from order M−2/3 to order M−1/2, and their law is the top eigenvalue of a k×k GUE, with k the multiplicity of the spike. This gives:
a detection threshold for spiked signals in high dimension;
the centring and scaling of tests based on the top sample eigenvalue;
the first instance of a k-dependent family of finite-GUE limits in a spiked model.
Formalizing it. The result is proved in the paper and is not open. Nothing in this mission has a machine-checked proof yet. A complete development would include:
the first formal statements of the spiked complex Wishart model;
the GUE law Gk and its Fredholm representation;
a uniform steepest-descent analysis with explicit contours.
The two contour lemmas (4.1, 4.2) and the Hermite identities are elementary and are footholds. Proposition 4.1 and Lemma 1.1 are substantial. The paper does not prove Lemma 1.1 but cites it as a standard result of random matrix theory (its references [28, 41]).
Difficulty
The obvious route is to diagonalise the sample matrix, write down the eigenvalue density and take a limit. That density involves the Harish-Chandra–Itzykson–Zuber integral. Its large-N limit is not accessible directly, because the spike enters through a determinant with N nearly coincident columns.
The paper instead starts from an exact Fredholm-determinant formula (Proposition 2.1, posed in mission 1 of this series). In it the kernel factors into two contour integrals H and J with phase f(z)=−μ(z−q)+logz−γ−2log(1−z). Above the threshold the two factors are governed by different points. J has a nondegenerate saddle at π1=ℓ1−1. For H the natural saddle 1/(μπ1) lies beyond the pole at π1, so the contour must be deformed through a pole of order k, and the leading term is a residue rather than a saddle contribution. The estimates must hold uniformly in γ and in the remaining spikes, and the convergence must be strong enough (Hilbert–Schmidt) to pass to Fredholm determinants.
Formalization scope
Model. Complex Gaussian samples with mean zero and no centring. S=M1∑kykyk∗, with the factor 1/M. Σ=Udiag(ℓ)U∗ for every unitary U. This is the model of the paper's (59), (61) and Proposition 2.1; the introduction's mentions of 1/N, of centring and of the real density (1) are inconsistent with those formulas. λ1 is the supremum of the eigenvalues of the Hermitian matrix S, and probabilities are measures of sets of sample arrays.
Regime. The regime is in sequence form:
Nn→∞ and γn=Mn/Nn∈[1,γ0];
1+γn−1+c≤ℓ1=⋯=ℓk≤C;
c≤ℓj≤ℓ1−c for k<j≤r;
ℓj=1 for j>r.
The fixed margins c,C encode the compact subsets whose open ends move with γ. Convergence is pointwise in x.
Analytic conventions.
The Fredholm determinant is the Fredholm series ∑nn!(−1)n∫(x,∞)ndet[K(ui,uj)].
H(k) takes its continuous diagonal value.
pn=Hen/((2π)1/4n!) with Mathlib's probabilists' Hermite polynomials, equal to the paper's (31).
The closed contours Γ, Σ in H, J are explicit circles satisfying the paper's constraints.
Σ∞ is the imaginary axis.
Residues of a−kφ(a) are Taylor coefficients.
Ruled-out trivialisations. Five shortcuts would make the statements trivial, and none is available:
division by zero on the kernel diagonal is replaced by the diagonal value;
the Fredholm series is shown to be 1 on the zero kernel, so it is not identically junk;
the conditionally convergent real form of contour integrals is not used;
Σ is not specialised to a diagonal matrix;
the regime is shown satisfiable in a sorry-free check.
Infrastructure. The needed infrastructure is the complex Wishart model, eigenvalues of random Hermitian matrices, Fredholm determinants of integral operators, and contour integrals with uniform estimates. The Hermite and Fredholm layers can be reused for any orthogonal-polynomial ensemble. Contributions welcome:
proofs of the elementary milestones ((218)–(219), (287), (295), (288), (292), (299), Lemmas 4.1, 4.2);
Lemma 1.1 via Christoffel–Darboux and Andréief;
the operator-theoretic step from Hilbert–Schmidt convergence of kernels to convergence of Fredholm series.
Selected references
J. Baik, G. Ben Arous, S. Péché, Phase transition of the largest eigenvalue for nonnull complex sample covariance matrices, Ann. Probab. 33(5) (2005) 1643–1697. https://doi.org/10.1214/009117905000000233
I. M. Johnstone, On the distribution of the largest eigenvalue in principal components analysis, Ann. Statist. 29 (2001) 295–327. https://doi.org/10.1214/aos/1009210544
J. Baik, J. W. Silverstein, Eigenvalues of large sample covariance matrices of spiked population models, J. Multivariate Anal. 97 (2006) 1382–1408. https://doi.org/10.1016/j.jmva.2005.08.003
Z. Bai, J. Yao, Central limit theorems for eigenvalues in a spiked population model, Ann. Inst. H. Poincaré Probab. Statist. 44 (2008) 447–474. https://doi.org/10.1214/07-AIHP118
Phase Transition of the Largest Eigenvalue for Nonnull Complex Sample Covariance Matrices 4: The Exponential Last Passage Time Has the Law of the Largest Sample EigenvalueResearch Paper
Motivation
Random growth models and random matrices share limit laws. The first exact instance was found by Johansson (Shape fluctuations and random matrices, Comm. Math. Phys. 2000), who showed that the last passage time of a lattice model with geometric or exponential weights has the law of the largest eigenvalue of a Laguerre (complex Wishart) random matrix. Baik, Ben Arous and Péché (Ann. Probab. 33 (2005)) extended the identity to weights whose rate depends on the row. On the matrix side this is a sample covariance matrix with a general population covariance Σ; on the growth side it is a corner growth model, or a series of exponential queues, with inhomogeneous service rates.
The identity is the reason the paper's main result, the phase transition of the largest sample eigenvalue as a few population eigenvalues ("spikes") cross the critical value 1+γ−1, is also a theorem about last passage percolation and tandem queues. In the queueing reading, a spike is a slow server, and the phase transition describes how slow a few servers must be before they change the centring and the fluctuation scale of the exit time.
Timeline:
2000, Johansson: equal rates; the geometric and exponential last passage time has the law of the largest Laguerre eigenvalue (his Proposition 1.4 is the case π1=⋯=πN of (307)).
2001, Baryshnikov and, independently, Gravner, Tracy and Widom: the tandem-queue and GUE-minor descriptions of the same object.
2005, Baik, Ben Arous and Péché: row-dependent rates πi (Proposition 6.1), via the Robinson–Schensted–Knuth (RSK) formula for geometric weights (310) and a scaling limit.
Setting
Last passage time. Attach a real weight X(i,j) to every site of the grid {1,…,N}×{1,…,M}. An up/right path from (1,1) to (N,M) is a sequence of N+M−1 sites starting at (1,1), ending at (N,M), each step adding (1,0) or (0,1). The last passage time is
L(N,M)=π:(1,1)↗(N,M)max(i,j)∈π∑X(i,j).(306)
Exponential environment. Given positive numbers π1,…,πN, the X(i,j) are independent and X(i,j) is exponential of mean 1/(πiM), i.e. density πiMe−πiMx on x≥0. All M sites of row i share one rate.
Sample covariance matrix. Let gkj, 1≤k≤M, 1≤j≤N, be independent standard complex Gaussians (real and imaginary parts independent N(0,1/2)). For a unitary U and ℓj=πj−1 (308), the samples yk=Udiag(ℓj)gk are mean-zero complex Gaussian vectors with covariance Σ=Udiag(ℓ)U∗, and
S=M1k=1∑Mykyk∗,λ1=largest eigenvalue of S.
Schur functions and geometric weights. For a partition λ, the Schur function sλ(x) is the sum over semistandard Young tableaux T of shape λ of ∏cxT(c). A geometric variable of parameter q∈[0,1) has P(Y=k)=(1−q)qk.
Formalization targets
Goal: Proposition 6.1
For 1≤N≤M, positive π1,…,πN and every unitary U,
P(L(N,M)≤x)=P(λ1(M,N)≤x)for all x∈R.(309)
The two sides are defined independently: one by a maximum over lattice paths of exponential weights, the other by the spectrum of a Gaussian random matrix.
Milestones
The recurrence (313): for every array of weights and every site with a,b≥2,
L(a,b)=max{L(a−1,b),L(a,b−1)}+X(a,b).
The Cauchy identity (311): ∑λsλ(x)sλ(y)=∏i,j(1−xiyj)−1 for xi,yj≥0, xiyj<1.
The geometric formula (310): for independent geometric weights Y(i,j) of parameter xiyj, xi,yj∈[0,1),
P(G(N,M)≤n)=i,j∏(1−xiyj)λ:λ1≤n∑sλ(x)sλ(y).
The exponential formula (307): for M≥N and distinct πi,
The proposition makes every distributional statement about λ1 in the paper a statement about last passage percolation with row-dependent rates: the Fk (generalised Tracy–Widom) fluctuations at the critical spike value, the Gaussian fluctuations above it, and the fixed-dimension limit. Through (312)–(313) it also covers the exit time of M customers from N exponential servers in series, an operations research object. The geometric formula (310) is the entry point of the RSK method for exactly solvable growth models.
These results are proved in the literature; the paper cites (310) and (311) and derives (307) from them by a limit. None of them is machine-checked as far as this series knows. Formalizing them requires a Lean development of Schur functions, the Cauchy identity and the RSK correspondence on matrices with nonnegative integer entries, and of the Wishart eigenvalue density, each reusable well beyond this mission.
Difficulty
The recurrence (313) is elementary. Everything else is not. The geometric formula (310) needs the RSK bijection between nonnegative integer matrices and pairs of semistandard tableaux of the same shape, together with Schensted's theorem that the first row of the shape is the last passage time; neither is in Mathlib. The passage from (310) to (307) is a scaling limit xi=1−Mπi/L, n=xL, L→∞, in which a sum over partitions must converge to a multiple integral. The matrix side needs the joint eigenvalue density of a complex Wishart matrix with general Σ, which uses the Harish-Chandra–Itzykson–Zuber integral. A direct coupling of the two sides is not known; the identity is an equality of laws, proved by computing both.
Formalization scope
The development lives in the namespace SpikedWishart.LastPassage. Conventions:
Sites and indices are 0-based; the paper's (1,1) and (N,M) are (0,0) and (N−1,M−1), and N,M≥1.
L is defined as the maximum over paths (306), never by the recurrence (313), so (313) is a genuine statement.
The exponential law is Mathlib's expMeasure with rateπiM.
The sample model is mean zero, uncentred, with factor 1/M and E∣g∣2=1; this is the model of (59), (61) and (307), and the page's centring by the sample mean and S=(1/N)XX∗ are printed slips. The covariance is Udiag(π−1)U∗ for every unitary U; specialising to diagonal Σ would prove a special case.
N≤M is a disclosed addition to the goal, matching the page's "for M≥N" in (307).
Proposition 6.1 prints L(M,N); the object is (306)'s L(N,M).
Schur functions are the tableau sum with variables extended by zero. The Cauchy identity is stated with the exponent −1 that the page omits. In (307) the constant C is the integral of the same integrand over (0,∞)N, and the πi are distinct so that V(π)=0; the goal has no distinctness hypothesis.
A formalization in which either side of (309) is defined through the other, or through (307), would be trivial and is excluded: both laws are defined from scratch. Contributions are welcome on RSK and Schur function infrastructure, on the Wishart density, and on the measurability of λ1.
Selected references
J. Baik, G. Ben Arous, S. Péché, Phase transition of the largest eigenvalue for nonnull complex sample covariance matrices, Ann. Probab. 33(5):1643–1697, 2005. https://doi.org/10.1214/009117905000000233
J. Gravner, C. A. Tracy, H. Widom, Limit theorems for height fluctuations in a class of discrete space and time growth models, J. Stat. Phys. 102:1085–1132, 2001. https://doi.org/10.1023/A:1004879725949
R. P. Stanley, Enumerative Combinatorics, Vol. 2, Cambridge University Press, 1999 (the paper's reference [36] for the Cauchy identity). https://doi.org/10.1017/CBO9780511609589
Phase Transition of the Largest Eigenvalue for Nonnull Complex Sample Covariance Matrices 1: Spikes at or below 1+γ⁻¹ Give M^{2/3} Fluctuations of the Largest Eigenvalue with Limit F_kResearch Paper
Motivation
Sample covariance matrices are the basic object of multivariate statistics: principal component analysis, factor models and signal detection all start from the eigenvalues of S=M1∑k=1Mykyk∗ built from M observations of N variables. When N is comparable to M, the largest eigenvalue λ1 of S is no longer a consistent estimate of the largest population eigenvalue, and the question of when a "spike" in the population covariance is visible in λ1 becomes a question about the law of λ1 at large M,N.
Timeline.
2000–2001: for null covariance Σ=I, Johansson (complex samples, arXiv:math/9903134) and Johnstone (real samples, doi:10.1214/aos/1009210544) showed that λ1, centred at (1+γ−1)2 and scaled by M2/3, converges to a Tracy–Widom law.
2003: Péché (reference [31] of the 2005 paper) showed that finitely many population eigenvalues below 2 (at M=N) leave this limit unchanged.
2005: Baik, Ben Arous and Péché (doi:10.1214/009117905000000233), for complex Gaussian samples, located the threshold exactly at 1+γ−1 and identified the limit laws on both sides and at the threshold. This transition is now called the BBP phase transition.
This mission formalizes the critical and subcritical side of that transition, Theorem 1.1(a) of the 2005 paper.
Setting
Let g=a+ib with a,b independent real normal variables of mean 0 and variance 1/2: a standard complex Gaussian. Let G=(gkj), 1≤k≤M, 1≤j≤N, be i.i.d. standard complex Gaussians. Fix an N×N unitary matrix U and positive reals ℓ1,…,ℓN, the population eigenvalues, and put Σ=Udiag(ℓ)U∗. The samples are yk=Udiag(ℓj)gk, mean-zero complex Gaussian vectors with covariance Σ. The sample covariance matrix is S=M1∑kykyk∗ and λ1 is its largest eigenvalue. Write γ=M/N≥1.
The limit laws are built from the Airy functionAi(u)=2π1∫eiua+ia3/3da, integrated along a contour from ∞e5iπ/6 to ∞eiπ/6, and the Airy kernel
A(u,v)=∫0∞Ai(u+z)Ai(z+v)dz.
For m≥1, s(m)(u)=2π1∫eiua+ia3/3(ia)−mda (with the pole a=0 above the contour) and t(m)(v)=2π1∫eiva+ia3/3(−ia)m−1da. The Fredholm determinant of a kernel K on L2((x,∞)) is its Fredholm series
F0 is the GUE Tracy–Widom distribution and F1=FGOE2.
Formalization targets
Goal: Theorem 1.1(a)
Fix integers 0≤k≤r. Let M,N→∞ with γ∈[1,γ0], and suppose ℓr+1=⋯=ℓN=1, ℓ1=⋯=ℓk=1+γ−1 and ℓk+1,…,ℓr stay in a compact subset of (0,1+γ−1). Then for every real x,
P((λ1−(1+γ−1)2)(1+γ)4/3γM2/3≤x)→Fk(x).
The goal leaves U arbitrary and allows γ to vary inside [1,γ0] along the sequence. A companion item states Corollary 1.1(a): λ1−(1+γ−1)2→0 in probability.
Milestones
The Airy equation and the identity of the two forms (11), (12) of the Airy kernel.
Lemma 3.3: Fk is well defined, and s(m) has the closed form (203).
The eigenvalue density (61) and Andréief's identity (65).
Hankel's formula (75).
Proposition 2.1: the exact identity P(λ1≤ξ)=det(1−KM,N)L2((ξ,∞)) for a double contour integral kernel.
The double critical point (109), (111) of the phase function f(z)=−μ(z−q)+logz−γ−2log(1−z).
The descent Lemmas 3.1 and 3.2 along explicit contour pieces.
Proposition 3.1: convergence of the rescaled contour integrals H,J at rate M−1/3 with exponential tails.
The kernel identity (200).
Significance
Theorem 1.1(a) shows that 1+γ−1 is the exact threshold. Spikes strictly below it do not change the Tracy–Widom limit of λ1. k spikes exactly at it produce a new family of limit laws Fk on the same M2/3 scale. Together with part (b), where λ1 separates and fluctuates on the scale M−1/2, it gives the first complete description of the transition. It is the reference point for spiked-model results on detection limits in PCA, for the real case (Bloemendal–Virág, Mo) and for finite-rank perturbations of Wigner matrices (Péché, Féral–Péché).
The paper's proof is complete and has been in the literature since 2005. No part of it is machine-checked. The mission produces:
a formal statement of the theorem;
formal definitions of the Airy function by its contour integral, of the Airy kernel and of Fredholm determinants as Fredholm series;
the exact finite-N determinantal formula of Proposition 2.1;
the uniform steepest-descent estimates;
formal statements of the classical identities used along the way (Andréief, Hankel).
Difficulty
The obvious approach is to compute the law of λ1 from the eigenvalue density, which is explicit (61). It fails because the density is an N-fold integral whose integrand is a signed determinant, so no direct limit exists. The paper converts it into a Fredholm determinant of a double contour integral kernel. Two steps then carry the difficulty.
First, the steepest-descent analysis. With k spikes at 1+γ−1, the poles of the integrand sit exactly at the double critical point pc=γ/(γ+1) of the phase function. The contour cannot pass through the critical point and must enclose these poles, so it has to be routed at distance of order M−1/3 to the left of pc. All estimates must be uniform in γ and in the remaining spikes.
Second, the passage from kernel convergence to convergence of Fredholm determinants. It needs bounds strong enough to control every term of the Fredholm series, which is what the exponential tails in Proposition 3.1 provide. The functions s(m) grow polynomially, so even the finiteness of Fk (Lemma 3.3) needs an argument.
Formalization scope
Lean encoding:
Samples are mean-zero with Eyy∗=Σ, S=M1∑kykyk∗, and there is no centring by the sample mean. This is the model of the paper's (59) and Proposition 2.1. The printed text on p. 1645 differs in three slips.
Σ=Udiag(ℓ)U∗ ranges over every unitary U.
λ1 is the largest eigenvalue of the Hermitian matrix S.
The asymptotic regime is written with sequences Mn,Nn, Nn→∞ and γn∈[1,γ0]. "In a compact subset of (0,1+γ−1)" is read with a fixed margin c: c≤ℓj≤1+γn−1−c.
Indices are 0-based.
Exponents 2/3, 4/3, 1/3 are real powers of positive reals.
Fk is defined as the Fredholm series of the kernel A+∑ms(m)⊗t(m), the paper's (201). The paper proves that this equals its Definition 1.1.
Contour integrals are Bochner integrals along two-ray broken lines, where the integrands decay like e−t3/3. Closed contours are circles with real centres.
Trivializing formalizations ruled out:
The Airy kernel is defined by (12), never by the divided difference (11), whose diagonal would be the junk value 0.
Ai is never written as the real integral π1∫0∞cos(t3/3+ut)dt, which is not Lebesgue integrable and would be 0.
A non-summable Fredholm series would be 0, so summability is a milestone.
The goal quantifies over every unitary U and every admissible sequence, so it does not specialize Σ. Its hypotheses are satisfiable, for example by Mn=Nn, ℓ≡1, r=k=0.
Infrastructure needed and reusable:
the Airy function and its decay;
trace-class or Hadamard-bound control of Fredholm series;
the complex Wishart eigenvalue density, which needs the Harish-Chandra–Itzykson–Zuber integral;
Andréief's identity;
steepest-descent estimates for contour integrals.
The Fredholm-determinant layer and the Airy function are reusable in any Tracy–Widom result. Andréief's identity and Hankel's formula are reusable well beyond random matrices.
Contributions welcome: proofs of any milestone, and lemmas that bound Fredholm series by Hadamard's inequality.
Selected references
J. Baik, G. Ben Arous, S. Péché, Phase transition of the largest eigenvalue for nonnull complex sample covariance matrices, Ann. Probab. 33(5) (2005), 1643–1697. https://doi.org/10.1214/009117905000000233
I. M. Johnstone, On the distribution of the largest eigenvalue in principal components analysis, Ann. Statist. 29 (2001), 295–327. https://doi.org/10.1214/aos/1009210544
C. A. Tracy, H. Widom, Level-spacing distributions and the Airy kernel, Comm. Math. Phys. 159 (1994), 151–174. https://arxiv.org/abs/hep-th/9211141
C. Andréief, Note sur une relation entre les intégrales définies des produits des fonctions, Mém. Soc. Sci. Phys. Nat. Bordeaux 2 (1883), 1–14.
Random Matrices: Universality of ESDs and the Circular Law 2: The ESD of A_n/√n Converges to μ iff the Normalized Log-Determinants of A_n/√n − zI Converge to μ's Log-Potential for a.e. zResearch Paper
Motivation
The eigenvalues of a large non-Hermitian random matrix are hard to study directly. For Hermitian matrices, the empirical spectral distribution (ESD) is controlled by the resolvent and the Stieltjes transform, both stable under small perturbations. For non-Hermitian matrices no such tool exists: eigenvalues can move a long way under tiny perturbations, and the moment method fails because moments do not determine a measure on C. The route that works, going back to Girko (1984) and used by Bai (1997), Götze–Tikhomirov (2007), Pan–Zhou (2007) and Tao–Vu (2008), passes through logarithmic determinants. The identity n1log∣det(A−zI)∣=∫log∣w−z∣dμA(w) links the ESD μA to a scalar quantity. That scalar is computable from the singular values of A−zI, which are eigenvalues of a Hermitian matrix.
Tao and Vu, with an appendix by Krishnapur (Ann. Probab. 38 (2010) 2023–2065), turn this link into an exact criterion. Theorem 1.15 of that paper states that the ESD of a normalized i.i.d. random matrix converges to a measure μif and only if the normalized log-determinants converge to the logarithmic potential of μ for almost every z. The same holds for a regularized version built from the singular values. This mission formalizes Theorem 1.15 and the main steps of its proof (§8 and Appendix A of the published article).
Setting
Let x be a complex random variable with Ex=0 and E∣x∣2=1, and let (xij)i,j≥1 be independent copies of x on a probability space (Ω,P). For each n, Xn=(xij)1≤i,j≤n. Let Mn be deterministic n×n complex matrices with
(1.3)nsupn21∥Mn∥22<∞,∥M∥22=i,j∑∣mij∣2,
and put An=Mn+Xn.
For A∈Mn(C) with eigenvalues λ1,…,λn, counted with algebraic multiplicity, the ESD is μA=n1∑iδλi. Its singular values are σ1(A)≥⋯≥σn(A)≥0. A sequence of random measures μnconverges in probability to a deterministic μ if, for every continuous compactly supported f:C→R, ∫fdμn−∫fdμ tends to 0 in probability. It converges almost surely if, with probability one, ∫fdμn→∫fdμ for all such f at once. The logarithmic potential of a measure μ is z↦∫log∣w−z∣dμ(w).
Hypothesis (1.4) states that, for almost every z, the ESDs of the deterministic matrices (n1Mn−zI)(n1Mn−zI)∗ converge to a limit.
In Lean everything lives in the namespace UnivESD.LogDet. The ESD is esd, the one-based singular value is sv, logPotential is the logarithmic potential, and ConvInProb/ConvAS/ESDConvInProb/ESDConvAS are the modes of convergence. IIDArray is the entry model, HSBound is (1.3), shiftA M xs z n ω is n1An−zI, and Cond14 is (1.4).
Formalization targets
Goal: Theorem 1.15
Let μ be a probability measure on C with ∫∣z∣2dμ<∞. The following are equivalent:
μn1An→μ in probability;
for a.e. z, n1logdet(n1An−zI)→∫Clog∣w−z∣dμ(w) in probability;
Under (1.4) the same equivalence holds with almost sure convergence throughout.
Milestones, in the order the proof uses them
(8.1): for a.e. z, w↦log∣w−z∣ is μ-integrable.
Lemma A.3 (Weyl's comparison for products): ∏j≤J∣λj∣≤∏j≤Jσj(A) and ∏j≥Jσj(A)≤∏j≥J∣λj∣.
(8.3): P(σn−i≤ci/n)=O(exp(−n0.01)) for 2n0.99≤i≤n−1.
(8.2): n1∑i=1n(logσi2+εn−logσi)→0 almost surely.
p. 2057: almost surely, eventually, n1∑(1−κ)n<i≤nlogσi1≤O(κlogκ1), and the same bound for log∣λi∣1.
Significance
Theorem 1.15 reduces convergence of a non-Hermitian ESD to a family of scalar limit statements. Each of these can be attacked through Hermitian tools, because the regularized log-determinant in (iii) is a smooth functional of the singular values. Corollary 1.16 of the paper derives from it a practical criterion: the ESD converges to μ once the singular value distributions of n1An−zI converge for a.e. z to laws ηz with ∫logtdηz=2∫log∣w−z∣dμ. With the universality principle (Theorem 1.5, the companion mission) and Mehta's computation for complex Gaussian matrices, this yields the circular law for i.i.d. matrices with finite variance.
The theorem is proved in the source. To our knowledge neither it nor any of its milestones has a machine-checked proof. Mathlib has the ingredients for the objects (characteristic polynomials and their roots, LinearMap.singularValues, the strong law of large numbers) but not the results. A formal proof would give the platform its first verified bridge between eigenvalue and singular value asymptotics. Lemma A.3 alone, Weyl's multiplicative majorization, is a standard linear-algebra fact absent from Mathlib.
Difficulty
Two things make the obvious argument fail. First, log is unbounded at 0. Vague convergence of μn1An controls ∫fdμn only for continuous compactly supported f, while log∣w−z∣ has a singularity at w=z and grows at infinity. Passing between (i) and (ii) therefore requires uniform control of the eigenvalues (and singular values) near 0 after the shift: the contribution of the smallest ones must vanish. Second, eigenvalues are not stable under perturbation, so this control cannot be read off the eigenvalues. It has to come from the singular values, through a lower-tail bound on σn−i that is exponentially small in a power of n, and through a deterministic inequality transferring bounds from singular values to eigenvalues. The smallest singular values (i below n0.99) need a separate polynomial lower bound (the paper's Lemma 4.1, posed in the companion mission). The tail bound in its printed form is false at that end of the range.
Formalization scope
Conventions committed to in Lean:
The ESD is n−1 times the sum of Dirac masses over the root multiset of the characteristic polynomial, so multiplicities are kept. Test functions are real-valued, continuous and compactly supported. Almost sure ESD convergence uses one null set for all test functions.
The matrices Xn are the top-left corners of one infinite i.i.d. array. In-probability statements do not depend on how the Xn are coupled across n, but almost-sure statements do, and the paper's proof applies the strong law to this array. The limit in (1.4) is required to be a probability measure, which is equivalent under (1.3).
The second-moment condition on μ is Integrable (‖w‖²) μ. The logarithmic potential is a Bochner integral, which is 0 when not integrable; by (8.1) this happens only for a null set of z, which "for a.e. z" ignores.
Real.log 0 = 0. In (ii), a singular matrix would contribute 0 instead of −∞; for this model the determinant is almost surely nonzero eventually. Every milestone with logσi or log∣λi∣ carries positivity of those quantities in its conclusion.
Singular values are one-based (sv A i is Mathlib's singularValues (i-1)). Eigenvalue enumerations are Fin n-indexed, and the paper's λi is index i−1.
The paper's O(⋅) constants are existential and fixed before n. In the κlog(1/κ) bounds the constant is fixed before κ; a κ-dependent constant would make the bounds empty.
Disclosed corrections to the printed text:
(iii)'s matrix. Theorem 1.15 prints a mis-bracketed product, and (1.8) prints (C+εnI)(C∗+εnI). The Lean uses CC∗+εnI with C=n1An−zI, the matrix of (8.2) and of the proof of Corollary 1.16.
(8.3)'s range. It is printed for 1≤i≤n−n0.99, which is false for small i: for Gaussian entries and Mn=0, nσn−1 has a non-degenerate limit law. It is posed on 2n0.99≤i≤n−1, the range the proof covers and the use needs.
Lemma A.3, second inequality. It is posed for 1≤J≤n, since σ0 is undefined. The eigenvalues are ordered ∣λ1∣≥⋯≥∣λn∣, not ∣λ1∣≤⋯≤∣λn∣ as printed: the printed order is a slip. With it, both inequalities are weaker than Weyl's and cannot support the use on p. 2057, which orders the eigenvalues decreasingly. The decreasing order implies the printed form.
(8.2) and (8.1). (8.2) is stated for every deterministic εn>0 with εn→0; the proof uses nothing else. (8.1) is stated as μ-integrability plus local integrability of z↦∫∣log∣w−z∣∣dμ(w).
A trivializing formalization is ruled out by construction. The goal mentions only An and μ. The proof's comparison ensemble Bn=ndiag(λi) is not a hypothesis. No statement takes a log-potential or a log-determinant as an input that could be fixed to make it hold.
The companion mission (Theorem 1.5, universality) poses Theorem 2.1 (the replacement principle), Lemma 1.7 (tightness) and Lemma 4.1 (least singular value), which the proof here also uses. They are not restated here. Once that mission is published they belong here as references. Contributions are welcome on the linear-algebra layer (Lemma A.3, multiplicative Weyl majorization), the Fubini argument of (8.1), and the singular-value tail bound (8.3). These are reusable well beyond this paper.
Selected references
T. Tao, V. Vu (appendix by M. Krishnapur), Random matrices: Universality of ESDs and the circular law, Ann. Probab. 38(5) (2010) 2023–2065. https://doi.org/10.1214/10-AOP534 (source of this mission; published version).
H. Weyl, Inequalities between the two kinds of eigenvalues of a linear transformation, Proc. Natl. Acad. Sci. USA 35 (1949) 408–411. https://doi.org/10.1073/pnas.35.7.408
Random Matrices: Universality of ESDs and the Circular Law 1: For i.i.d. Zero-Mean Unit-Variance Entries, the ESD of (M_n + X_n)/√n Does Not Depend on the Entry Distribution in the LimitResearch Paper
Motivation
The eigenvalues of a large random matrix with independent entries follow deterministic limiting laws. For non-Hermitian matrices with i.i.d. entries of mean zero and variance one, the circular law asserts that the eigenvalues of n1Xn become uniformly distributed on the unit disk. These laws are used across statistics (covariance and signal-plus-noise models), numerical analysis (spectra of perturbed matrices), mathematical physics and the theory of neural networks; what makes them usable is universality: the limit depends only on the mean and variance of the entries, not on their distribution.
Timeline of the circular law and its universality:
1965: Ginibre computes the joint eigenvalue density for complex Gaussian entries; Mehta (1967) derives the circular law in that case.
1984: Girko proposes a general approach via log-determinants; his argument has gaps.
1997: Bai gives the first rigorous proof for continuous entries with finite (2+η)-th moment and bounded density.
2007–2008: Götze–Tikhomirov, Pan–Zhou and Tao–Vu remove the density assumption under extra moment conditions, using lower bounds on the least singular value.
2010: Tao and Vu (Ann. Probab. 38 (2010) 2023–2065) prove universality under the minimal assumption of finite variance, and with a deterministic shift Mn; the circular law in full generality follows.
This mission formalizes the main universality theorem of that paper (Theorem 1.5) together with the lemmas its proof rests on.
Setting
Let A be an n×n complex matrix with eigenvalues λ1,…,λn, counted with algebraic multiplicity. Its empirical spectral distribution (ESD) is the probability measure
μA=n1i=1∑nδλi
on C. The Hilbert–Schmidt norm is ∥A∥22=trace(AA∗)=∑i,j∣aij∣2.
A complex random variable x has zero mean and unit variance if Ex=0 and E∣x∣2=1. Given such x and y, let Xn=(xij)1≤i,j≤n and Yn=(yij)1≤i,j≤n have entries that are i.i.d. copies of x, resp. y. Let Mn be deterministic n×n matrices with
nsupn21∥Mn∥22<∞,(1.3)
and set An=Mn+Xn, Bn=Mn+Yn.
Measures on C carry the vague topology: test functions are continuous compactly supported f:C→R. The random measures μn1An−μn1Bnconverge to zero in probability if, for every test function f and ε>0,
P(∫fdμn1An−∫fdμn1Bn≥ε)→0,
and almost surely if, with probability one, the difference of integrals tends to zero for every test function. The additional hypothesis
(1.4)μ(n1Mn−zI)(n1Mn−zI)∗converges for almost every z∈C
The statement involves no limiting law: it compares two ensembles, so it holds whether or not μn1An itself converges.
Milestones
The proof combines three results, each a milestone:
Theorem 2.1 (replacement principle), in probability and almost surely: if n21(∥An∥22+∥Bn∥22) is bounded and, for almost every z, n1log∣det(n1An−zI)∣−n1log∣det(n1Bn−zI)∣→0, then μn1An−μn1Bn→0.
Lemma 1.7 (tightness): n21∥An∥22 and ∫∣z∣2dμn1An are almost surely bounded.
Proposition 2.2 (converging determinant): for every z, the log-determinant difference (2.2) tends to zero in probability, and almost surely if (1.4) holds at z.
Supporting milestones: the linear-algebra Lemmas A.1 (interlacing), A.2 (Weyl's second-moment inequality) and A.4 (negative second moment); the probabilistic dominated convergence Lemma 3.1; Girko's identity (Lemma 3.3); the least singular value bound (Lemma 4.1); the lower tail bound for distances to subspaces (Proposition 5.1); and the high- and low-dimensional row contributions (Lemmas 4.2 and 4.3).
Significance
The result. Theorem 1.5 with Mn=0 and y complex Gaussian transfers Mehta's computation for the Ginibre ensemble to every zero-mean unit-variance entry distribution: this is the circular law under the optimal moment assumption (Theorem 1.10 of the paper). With nonzero Mn it gives universality for perturbed and non-centred matrices, including results with entries of nonzero mean. The replacement principle (Theorem 2.1) is a general tool: it reduces vague convergence of non-Hermitian ESDs to convergence of log-determinants and applies to ensembles without independent entries.
Formalizing it. The theorem is proved in the paper; no part of it, and no ESD of a non-Hermitian matrix, has been formalized. The mission produces a machine-checked statement of universality in the vague topology and a library of reusable pieces: ESDs of complex matrices, convergence of random measures in probability and almost surely, interlacing and Weyl-type inequalities for singular values, and Girko's identity. Lemmas 4.1 and 5.1 rest on results the paper cites (Tao–Vu's least singular value bound, Talagrand's concentration inequality); formalizing those is part of the work.
Difficulty
The ESD of a non-Hermitian matrix is not stable under small perturbations: eigenvalues of a nearly singular matrix can move a long way, so the moment method and the Stieltjes transform method that settle the Hermitian case do not apply. The log-determinant n1log∣det(n1An−zI)∣ is a stable proxy only if the small singular values of n1An−zI are controlled, and with only a second moment on the entries the entries can be unbounded, which defeats standard concentration arguments. The obvious approach, comparing An and Bn by swapping entries one at a time, needs smooth bounded statistics of the matrix, and the log-determinant is neither: it is unbounded near singular matrices.
Formalization scope
Matrices are Matrix (Fin n) (Fin n) ℂ; eigenvalues are the roots of the characteristic polynomial as a multiset, so the ESD counts algebraic multiplicity; for n=0 the ESD is the zero measure, harmless because every statement is asymptotic. The Hilbert–Schmidt norm is written as an explicit sum, not Mathlib's default (sup) matrix norm. Singular values are Mathlib's zero-indexed LinearMap.singularValues, shifted so that σ1 is the largest.
Conventions committed to:
Coupling.Xn is the top-left n×n corner of one infinite i.i.d. array; the almost-sure statements depend on this choice, the in-probability ones do not. No relation between the x- and y-arrays is assumed.
(1.4) asks for a probability measure as the vague limit; under (1.3) any vague limit of these ESDs is one.
Almost sure convergence of ESDs uses one null set for all test functions.
Junk values. Lean has log0=0 and 1/0=0. Theorem 2.1 therefore states its "determinants are nonzero" clause explicitly, and Proposition 2.2 concludes it (the paper's (2.2) is −∞ at a vanishing determinant, so its convergence includes nonvanishing); Lemmas 4.2 and 4.3 assert positivity of the row distances they take logarithms of; Lemma 3.1's (1+δ)-moment is a lower integral with values in [0,∞], since a Bochner integral would vanish for non-integrable functions and make the hypothesis vacuous.
O(⋅) constants are existential and fixed before the quantities they must not depend on: before ε in Lemmas 4.2 and 4.3, before the sample point in Lemma 4.1, after c and before n,d,W and the row shift in Proposition 5.1.
Section 4 setting. Lemmas 4.2 and 4.3 carry the standing reductions of §4 as hypotheses: z=0, the row bound (4.2) ∥Zi∥=O(n) for n/2≤i≤n (which the paper obtains by permuting rows), and, for Lemma 4.3, convergence of μn1MnMn∗. Proposition 5.1 quantifies over every deterministic row shift.
Printed slip. On p. 2045 the Yi are called "the rows of n1Bn" and then normalized again; here Yi are the rows of Bn, matching Xi.
A trivializing formalization is excluded: the goal mentions only ESDs, i.i.d. entries, (1.3) and (1.4) — no log-determinant or singular value hypothesis — and its in-probability half holds without (1.4). Lemmas 6.3 and 6.4 and the internal Lemmas 3.2, 3.4, 5.3, 6.1, 6.6, 6.7 and Theorems 5.2, B.1 are not posed. Contributions are welcome on any milestone, and particularly on the deterministic Lemmas A.1, A.2, A.4 and 3.3, which are reusable well beyond this mission.
Selected references
T. Tao, V. Vu (appendix by M. Krishnapur), Random matrices: universality of ESDs and the circular law, The Annals of Probability 38(5) (2010) 2023–2065. https://doi.org/10.1214/10-AOP534 (the source; the published article is the citation basis)
J. Ginibre, Statistical ensembles of complex, quaternion, and real matrices, J. Math. Phys. 6 (1965) 440–449. https://doi.org/10.1063/1.1704292
V. L. Girko, The circular law, Teor. Veroyatnost. i Primenen. 29 (1984) 669–679; English translation Theory Probab. Appl. 29 (1985) 694–706. https://doi.org/10.1137/1129095
R. B. Dozier, J. W. Silverstein, On the empirical distribution of eigenvalues of large dimensional information-plus-noise-type matrices, J. Multivariate Anal. 98 (2007) 678–694. https://doi.org/10.1016/j.jmva.2006.09.006
Guaranteed Minimum-Rank Solutions of Linear Matrix Equations via Nuclear Norm Minimization 2: Nearly Isometric Random Maps Have δ_r ≤ δ with Probability 1 − e^(−c₁p) Once p ≥ c₀r(m+n)log(mn)Research Paper
Motivation
Many problems in system identification, collaborative filtering, low-dimensional embedding and model reduction ask for the matrix of least rank in an affine space. Affine rank minimization — given a linear map A:Rm×n→Rp and b∈Rp, find a matrix of minimum rank with A(X)=b — is NP-hard in general. Recht, Fazel and Parrilo (arXiv:0706.4138; SIAM Review 52(3), 2010, doi:10.1137/070697835) showed that the convex relaxation which minimizes the nuclear norm∥X∥∗ (the sum of the singular values) instead of the rank recovers the minimum-rank solution exactly whenever A satisfies a restricted isometry property on low-rank matrices. That deterministic theorem is the subject of the companion mission in this series.
A deterministic condition is only useful if maps satisfying it exist and are common. This mission formalizes the paper's second main result: random linear maps from a broad class — including Gaussian, symmetric Bernoulli and sparse random matrices — satisfy the restricted isometry property with overwhelming probability once the number of measurements p is of order r(m+n)log(mn). This parallels the role of random matrices in compressed sensing (Candès–Tao 2005; Baraniuk, Davenport, DeVore and Wakin 2008), where the vector restricted isometry property is established for random measurement matrices by a concentration and covering argument.
Setting
Write ⟨X,Y⟩=Tr(X⊤Y) for the trace inner product on Rm×n and ∥X∥F for the Frobenius norm. A linear map A:Rm×n→Rp is given by p measurement matrices X1,…,Xp through A(X)i=⟨Xi,X⟩; ∥A(X)∥ is the Euclidean norm in Rp, and ∥A∥=sup{∥A(X)∥:∥X∥F≤1} is the operator norm.
The r-restricted isometry constantδr(A) (Definition 3.1) is the smallest δ≥0 such that
(1−δ)∥X∥F≤∥A(X)∥≤(1+δ)∥X∥Ffor all X with rankX≤r.
A random linear map A is nearly isometrically distributed (Definition 4.1) if for every X, E∥A(X)∥2=∥X∥F2; for every X=0 and 0<ϵ<1,
and for some γ>0 and all t>0, P(∥A∥≥1+mn/p+t)≤exp(−γpt2).
For subspaces T1,T2 of a Euclidean space, the projection distance is ρ(T1,T2)=∥PT1−PT2∥, the operator norm of the difference of the orthogonal projections (4.10). For subspaces V⊆Rm, W⊆Rn, the subspace Σ(V,W)⊆Rm×n consists of the matrices whose column space lies in V and whose row space lies in W.
Formalization targets
Goal: Theorem 4.2
Fix 0<δ<1. There are c0,c1>0 depending only on δ such that for all 1≤r≤m≤n with mn≥2, all p≥c0r(m+n)log(mn), and every nearly isometric random map A:Rm×n→Rp,
P(δr(A)>δ)≤e−c1p.
Milestones
Lemma 4.3. For a fixed subspace U with d=dimU≤p and 0<δ<1, the map is a δ-isometry on all of U except with probability at most 2(12/δ)dexp(−2p(δ2/8−δ3/24)).
Lemma 4.4. If A is a δ-isometry on a d-dimensional U1, it is a δ′-isometry on any d-dimensional U2, with δ′=δ+(1+∥A∥)ρ(U1,U2).
Lemma 4.5. For 0<ϵ≤1, the family {Σ(V,W):dimV=dimW=r} has an ϵ-net in ρ of size at most (2C0/ϵ)r(m+n−2r), with C0 absolute.
Significance
The theorem supplies the hypothesis of the paper's recovery theorem for random measurements: with δ=1/10 and rank 5r, it shows that O(r(m+n)log(mn)) random linear measurements suffice for nuclear norm minimization to recover every matrix of rank at most r exactly. Since a rank-r matrix has r(m+n−r) degrees of freedom, this is within a logarithmic factor of the information-theoretic count. The result made the nuclear norm heuristic a guaranteed method and seeded the theory of low-rank matrix recovery, including matrix completion.
The result is proved in the paper, though Lemma 4.5 rests on Szarek's covering estimates for the Grassmannian, cited without proof. To the best of the curators' search of the platform, none of these statements is machine-checked: there is no formal treatment of the matrix restricted isometry property, of nets on Grassmannians in the projection metric, or of nearly isometric random maps. A formalization would supply the probabilistic half of a fully verified low-rank recovery guarantee, together with reusable pieces — the projection distance, the stability of near isometry under subspace perturbation, and a covering bound for Grassmannians.
Difficulty
The set of matrices of rank at most r is not a subspace, so concentration for one matrix and a union bound over a single net in Rm×n do not suffice: a net of the unit ball of all matrices has size exponential in mn, which would require p of order mn. The set must instead be covered by finitely many r2-dimensional subspaces, which requires a quantitative covering of the Grassmannians G(m,r) and G(n,r) in the projection distance — the content of Lemma 4.5, whose source in the literature (Szarek) does not give the constant explicitly. A second subtlety is that the event "A is a near-isometry on all of a subspace" involves an uncountable family of matrices, and the bound for the supremum of ∥A(X)∥ over a unit sphere has to be controlled from the finitely many points of a net.
Formalization scope
All statements live in the namespace MinRankRecovery.RandomRIP. Matrices are Matrix (Fin m) (Fin n) ℝ; the Frobenius norm and the trace inner product come from the published module HighDimStat_MatrixRank_Core. A linear map is represented by its p measurement matrices Xs : Fin p → Matrix (Fin m) (Fin n) ℝ, which loses no generality. A random map is Xs : Ω → Fin p → Matrix … on a probability space (Ω, μ), with measurable entries.
Conventions committed to:
δr is the infimum of the admissible δ≥0 and is defined for every r; the paper's 1≤r≤m≤n appears as hypotheses of the goal.
∥A∥ is a supremum over the nonempty, bounded Frobenius unit ball.
(4.1) includes integrability. (4.2) is imposed for X=0 only, since at X=0 the printed inequality fails for large p. γ in (4.3) may depend on the distribution and dimensions.
Probability statements bound the (outer) measure of the failure event, since events quantified over all matrices need not be measurable.
For ρ and Σ(V,W), matrices are vectorized into EuclideanSpace ℝ (Fin m × Fin n), which carries the Frobenius inner product. Lemma 4.4 is stated on EuclideanSpace ℝ ι for any finite index type.
Real.log is the natural logarithm.
Two printed statements are corrected. Theorem 4.2 assumes mn≥2: at m=n=r=1, log(mn)=0 and a two-point random scalar map is nearly isometric while violating the conclusion. Lemma 4.5 is posed for 0<ϵ≤1, since for large ϵ the right side drops below 1.
The constants c0,c1 of the goal are quantified before the dimensions, the distribution and γ; a formalization in which they may depend on m,n,p,r or on the distribution, or in which the near-isometry hypothesis is unsatisfiable, would be trivial and is ruled out by these choices.
A complete development needs: orthogonal projections and operator norms on finite-dimensional inner product spaces (Mathlib), volumetric ϵ-nets of spheres in subspaces, covering bounds for real Grassmannians in the projection metric, and the measure-theoretic union bound. The Grassmannian covering bound and the stability lemma are reusable beyond this mission. Contributions of any milestone, including Lemma 4.5 on its own, are welcome.
Selected references
B. Recht, M. Fazel, P. A. Parrilo, Guaranteed Minimum-Rank Solutions of Linear Matrix Equations via Nuclear Norm Minimization, SIAM Review 52(3):471–501, 2010. Preprint arXiv:0706.4138v1. https://arxiv.org/abs/0706.4138
R. Baraniuk, M. Davenport, R. DeVore, M. Wakin, A Simple Proof of the Restricted Isometry Property for Random Matrices, Constructive Approximation 28(3), 2008. https://doi.org/10.1007/s00365-007-9003-x
S. J. Szarek, Metric entropy of homogeneous spaces, in Quantum Probability, Banach Center Publ. 43, 1998. https://arxiv.org/abs/math/9701213
E. J. Candès, Y. Plan, Tight Oracle Inequalities for Low-Rank Matrix Recovery From a Minimal Number of Noisy Random Measurements, IEEE Trans. Inform. Theory 57(4), 2011. https://arxiv.org/abs/1001.0339
High-Dimensional Probability XI: Dvoretzky-Milman's TheoremTextbook
Motivation
A striking fact discovered by Dvoretzky in the 1960s (conjectured by Grothendieck, and sharpened
into its modern quantitative form by Milman in 1971) is that every high-dimensional convex body,
however irregular, contains a round slice: a random low-dimensional section (or projection) of
any bounded convex set in Rn is, with high probability, close to a Euclidean ball —
provided the dimension of the slice is small enough relative to a single geometric parameter of
the body. This is remarkable because it holds for every bounded set, arbitrarily irregular; no
special structure is assumed beyond boundedness. This chapter proves the theorem in its Gaussian
form, as a culmination of every geometric and probabilistic tool the book develops: chaining and
Dudley's inequality (Chapter 8), the matrix deviation inequality (Chapter 9), and Gaussian width
and the stable dimension (Chapter 7) all combine into a single closing argument.
Setting
Fix a subset T⊆Rn. For a standard Gaussian vector g∼N(0,In), the
Gaussian width of T is w(T):=Esupx∈T⟨g,x⟩ (Chapter 7), and
the stable dimension of a bounded T is d(T):=w(T)2/diam(T)2 up to an absolute
constant factor (Definition 7.6.2) — a robust substitute for the ordinary linear-algebraic
dimension of T, which can jump discontinuously under a small perturbation of T, unlike d(T).
An m×nGaussian random matrix with i.i.d. N(0,1) entries is a random matrix A each
of whose mn entries is an independent standard normal random variable.
for every m×n Gaussian random matrix A with i.i.d. N(0,1) entries, every bounded
T⊆Rn containing the origin, and every ε∈(0,1), where B is the
Euclidean ball of radius w(T) centered at the origin. The probability 0.99 is the book's own
literal numeral, not a free parameter — this is the theorem the book actually states, not a
family of theorems indexed by a confidence level.
Significance
Dvoretzky-Milman's theorem is one of the foundational results of the local theory of Banach
spaces (asymptotic geometric analysis): it says every n-dimensional normed space contains an
almost-Euclidean subspace of dimension proportional to (a geometric invariant closely related to)
logn in the worst case, and much larger for spaces whose unit ball is already well-behaved
(the stable dimension of the cube [−1,1]n, for instance, is proportional to n itself — Example
11.3.6). This underlies results throughout convex geometry, compressed sensing, and high-dimensional
statistics wherever a random low-dimensional projection needs to be shown to preserve geometric
structure. The book's own framing makes clear why this chapter is placed last: the theorem's proof
is a genuine capstone, invoking Chevet's inequality (itself built from the matrix deviation
inequality of Chapter 9, which is built from chaining, Chapter 8) as its main technical tool.
The theorem and its proof are classical (Milman 1971; this book's specific route via Chevet's
inequality is a standard modern exposition). This mission formalizes the goal theorem's statement
— including its two supporting geometric quantities, Gaussian width and stable dimension, and the
notion of a Gaussian random matrix — as a complete, faithful target for a solver, in the book's own
sub-namespace built for this chapter (no dependency here is reusable from an earlier chunk, since
none of this book series' Chapter 7 or Chapter 9 definitions has yet been published).
Difficulty
The natural first idea — bound conv(AT) directly using concentration of ∥Ax∥2 for
each fixed x∈T — runs into exactly the uniform-supremum obstacle the whole book has been
building tools to overcome: a bound that holds for one x at a time, even with a union bound over
a net of T, does not obviously extend to the full convex hull without first controlling
supx∈T∣⟨Ax,y⟩−w(T)∥y∥2∣ uniformly over bothx∈T and y on the
unit sphere of the target space — a two-parameter supremum. The book's actual route goes through
Chevet's inequality, itself proved using the matrix deviation inequality's own chaining-based
argument, to control this two-sided supremum, and then converts the resulting inequality into the
containment (1−ε)B⊆conv(AT)⊆(1+ε)B via a support-
function duality argument (a convex body is pinned down by its support function, so bounding
supx∈T⟨Ax,y⟩ uniformly over y on the sphere is exactly what is needed).
Formalization scope
A is Ω → Matrix (Fin m) (Fin n) ℝ with an explicit IsGaussianMatrix hypothesis (entries i.i.d.
N(0,1), formalized entrywise with joint independence). conv(AT) is convexHull ℝ of the image
of T under A's mulVec, round-tripped through EuclideanSpace's continuous linear equivalence
with the underlying function type. w(T) reuses this mission series' ExpSup/GaussianWidth
convention (redefined locally, per the drafts-cannot-import-drafts rule, following the same
ProbabilityTheory.stdGaussian-based realization of a standard Gaussian vector as
08-matrix-deviation). The stable dimension d(T) is formalized directly as w(T)2/diam(T)2 rather than via the book's literal (but only asymptotically equivalent, per
Exercise 7.6.1) definition through a squared Gaussian width h(T−T)2 — the goal theorem's own
proof uses only the inequality direction of that equivalence, and the goal's hypothesis already
carries an unpinned absolute constant that absorbs the equivalence constant, so this substitution
preserves the theorem's exact truth content (see StableDimension's own doc-comment and
MODERATION_NOTES.md for the full argument) rather than approximating it.
Ball-center deviation, disclosed. The book's printed theorem statement carries no hypothesis
that T contains the origin; its proof opens by translating T so that it does ("Translating T
if necessary, we can assume that T contains the origin"), and Remark 11.3.4 then confirms the
ball is centered at the origin in that case. This mission states the WLOG-reduced case directly —
adding 0∈T as an explicit hypothesis — rather than also formalizing the translation argument
that recovers the fully general (untranslated) statement. This is disclosed as a genuine narrowing
of the literal printed statement, though not of what the book's own proof actually establishes.
This mission covers Theorem 11.3.3 only, with no milestones: BRIEF.md explicitly instructs that
if the chapter's full proof chain (general matrix deviation inequality, Chevet's inequality, random
projections of sets — Theorems 11.1.5, 11.2.4, 11.3.1) proves too heavy for the session, milestones
should be cut rather than the goal substituted. All three are left out, not approximated, given
this chapter's five from-scratch definitions already needed for the goal's own statement. ExpSup,
GaussianWidth, StableDimension and IsGaussianMatrix are reusable by any later development
needing Gaussian width, the stable dimension, or a Gaussian random matrix. Solvers' contributions
are welcome on the goal theorem itself and, beyond this mission's current scope, on the three
named milestones.
Selected references
A. Dvoretzky, Some results on convex bodies and Banach spaces, Proc. Internat. Sympos. Linear
Spaces (Jerusalem, 1960), 123–160.
V. D. Milman, A new proof of A. Dvoretzky's theorem on cross-sections of convex bodies,
Funkcional. Anal. i Priložen. 5 (1971), 28–37.
R. Vershynin, High-Dimensional Probability: An Introduction with Applications in Data
Science, Cambridge University Press, 2018, Chapter 11. https://doi.org/10.1017/9781108231596
Asymptotics of random density matrices: The Rescaled Empirical Spectral Measure of an Induced Random Density Matrix Converges Almost Surely to the Marchenko–Pastur LawResearch Paper
Motivation
In quantum mechanics, a system that interacts with an environment is described by a density matrix: a positive semidefinite complex matrix of trace one. When nothing is known about the joint state of the system and its environment, a natural model is to take that joint state to be a uniformly random pure state and to trace out the environment. The resulting law on density matrices, the induced measure of Życzkowski and Sommers (Życzkowski–Sommers 2001), is a standard model of a "typical" mixed state. It is used in quantum information to study the typical entanglement of bipartite pure states, as in Page's conjecture on average entropy (Page 1993).
Nechita (arXiv:quant-ph/0702154v3, Ann. Henri Poincaré 2007) studies the asymptotics of this ensemble. In the regime where the system dimension n and the environment dimension k(n) grow proportionally, the spectrum of the random density matrix, magnified by cn, has the same almost sure limit as the spectrum of a Wishart matrix: the Marchenko–Pastur law (Marchenko–Pastur 1967). The mission formalizes this result (Theorem 5) together with the chain of statements the paper uses to derive it.
Setting
Throughout, NC(0,1) denotes the law of Z=X+iY where X,Y are independent real Gaussians of mean 0 and variance 1/2, so that E∣Z∣2=1.
Wishart matrix. If X is an n×k complex matrix with i.i.d. NC(0,1) entries, W=XX∗ is a Wishart matrix of parameters n and k.
Pure states. A pure state of Cm is a ray ∣ψ⟩: a nonzero vector up to a nonzero scalar factor. Its pure density matrix is ∣ψ⟩⟨ψ∣=ψψ∗/∥ψ∥2, a rank-one orthogonal projection. The set Em of pure states carries a unique probability measure νm invariant under all unitaries U∈U(m).
Induced measure. Let H=Cn (the system) and K=Ck (the environment). If ∣ψ⟩ has law νnk on the pure states of H⊗K, the law of
ρ=TrK∣ψ⟩⟨ψ∣
is the induced measure μn,k on n×n density matrices. Here TrK is the partial trace over the environment.
Empirical spectral measure. For an n×n matrix A with eigenvalues λ1(A),…,λn(A) (counted with multiplicity) and a scale s, put Ls(A)=n1∑iδsλi(A).
Marchenko–Pastur law. For c>0, with a=(c−1)2 and b=(c+1)2,
μc=max{1−c,0}δ0+2πx(x−a)(b−x)1[a,b](x)dx.
Formalization targets
Goal: Theorem 5 (p. 14)
Let c∈(0,∞), let k(n) be integers with k(n)/n→c, and let ρn have law μn,k(n) for every n. Then, almost surely,
Ln=n1i=1∑nδcnλi(ρn)⟹μcweakly.
Milestones, in the order the proof uses them
Proposition 1 (p. 4): νn exists and is unique.
Proposition 2, item 1 (p. 4): the class of a vector with law NCn(0,In) has law νn.
Eq. (12) (p. 6): if the coordinates ψij are arranged in an n×k matrix X, then TrK∣ψ⟩⟨ψ∣=XX∗/Tr(XX∗).
Lemma 1 (p. 6): if X has i.i.d. NC(0,1) entries, then XX∗/Tr(XX∗) has law μn,k.
Eq. (24) (p. 9): the eigenvalues of W/TrW are λi/∑jλj.
Theorem 2 (p. 8, quoted from Haagerup–Thorbjørnsen): almost surely, n1∑iδn−1λi(Wn)⇒μc for Wishart matrices Wn of parameters n and k(n).
Lemma 2, (45) (p. 13): TrWn/(nk(n))→1 almost surely.
The perturbation step (p. 14, eqs. (51)–(53)): dilating the atoms of a weakly convergent sequence by factors tn→1 does not change a compactly supported weak limit.
Significance
The result. Theorem 5 says that the reduced state of a large random bipartite pure state has a deterministic spectral profile at scale 1/(cn). When c<1, the environment is smaller than the system, and an atom of mass 1−c at 0 records that ρn has rank k(n)<n. When c≥1, the spectrum fills the interval [a,b]. The result transfers the classical Wishart limit to the induced measure, and with it quantitative information on the typical entanglement spectrum of bipartite pure states.
Formalizing it. The theorem is proved on paper, and none of it is machine-checked. The mission produces the induced-measure layer in Lean: complex Gaussian matrices, the invariant law on pure states, the partial-trace construction of μn,k, and the Marchenko–Pastur law with its atom. Lemma 1 and eq. (12) connect these objects to Wishart matrices. The step that needs the most new library is Theorem 2, the Marchenko–Pastur theorem itself, which the paper quotes without proof. The perturbation step is posed as a statement about weak convergence. The paper argues it through convergence of moments, and in that argument weak convergence of LnW is used as if it gave convergence of moments. The statement posed here is true as stated and needs no moment control.
Difficulty
The reduction from Theorem 5 to Theorem 2 is short on paper. The work lies elsewhere.
Theorem 2 is a full random-matrix limit theorem. It needs a method that identifies the limit (moments, Stieltjes transforms, or free probability) and concentration strong enough to give almost sure convergence with no coupling between different n.
Proposition 1 needs the invariant measure on complex projective space and a uniqueness argument for it. Mathlib's Haar-measure uniqueness is stated for groups, not for homogeneous spaces of U(n).
Lemma 1 needs the unitary invariance of the complex Gaussian vector NCnk(0,I) and the image of a law under partial trace.
The naive route of "taking moments" in the perturbation step fails: weak convergence does not give convergence of moments, so (52)–(53) cannot be compared without extra tightness or moment bounds.
Formalization scope
Pure states. Rays of Cm are represented by their rank-one projections (IsPureDensity: Hermitian, idempotent, trace one). A law on Em is a probability measure on m×m complex matrices carried by that set. Unitary invariance is invariance under ρ↦UρU∗ (IsPureStateLaw).
Induced law. "ρ has law μn,k" is the predicate HasInducedLaw: the law of ρ is the image under the partial trace of some unitarily invariant law on pure states of Cn⊗Ck. Proposition 1 makes this law unique.
Indexing and partial trace.Cn⊗Ck is indexed by Fin n × Fin k with the environment on the right. The partial trace is the published WildeQIT.partialTraceRight. NC(0,1) is the published SpikedWishart.SoftEdge.stdComplexGaussian, with variance 1/2 per real coordinate.
Eigenvalues and convergence. Eigenvalues are roots of the characteristic polynomial, counted with multiplicity. Weak convergence is ∫fdLn→∫fdμc for all bounded continuous f, with one null set for all f.
Disclosed conventions. The hypotheses hold for n≥1 with k(n)≥1, since there are no pure states in dimension 0. All random matrices live on one probability space, and nothing is assumed about their joint law across different n: the paper never couples them, and the almost sure statements hold for every coupling.
Trivializations ruled out. Three trivializing formalizations are excluded:
stating the law of ρn through the Wishart formula Wn/TrWn, which would make Lemma 1 a hypothesis;
dropping the atom of μc or leaving the limit measure free;
stating "for each f, almost surely" in place of one null set for all f.
Solvers are welcome to contribute in four places:
the general measure-theoretic pieces: Gaussian invariance, image laws under partial trace, and the dilation lemma, which are reusable beyond this mission;
the deterministic linear algebra of eqs. (12) and (24);
The Weingarten Calculus: Haar Integrals of Matrix Elements Are Given by a Basis of Invariant Tensors and Its Weingarten Matrix, I = A W A*Research Paper
Motivation
Integrals of polynomial functions of the matrix entries of a random element of a compact group appear across mathematical physics and probability: in random matrix theory (moments of Haar-distributed unitary, orthogonal and symplectic matrices), in lattice gauge theory (the link integrals of U(N) gauge fields), and in quantum information (averages of quantum channels over random unitaries, twirling, unitary designs). Weingarten first computed asymptotics of such integrals for U(N) in 1978 (Weingarten, J. Math. Phys. 19 (1978)); Collins (IMRN 2003) and Collins–Śniady (CMP 2006) gave exact formulas for the unitary, orthogonal and symplectic groups in terms of what is now called the Weingarten function.
The survey of Collins, Matsumoto and Novak, The Weingarten Calculus (arXiv:2109.14890), isolates the single statement underneath all these formulas, valid for every compact group and every finite-dimensional unitary representation. This mission formalizes that statement, Theorem 2.1 of the survey, together with the steps of its proof as written in Section 2.
Setting
Let G be a compact Hausdorff topological group and μ its Haar probability measure, the unique Borel probability measure on G invariant under left translation. Let H=CN with its standard orthonormal basis e1,…,eN, and let U:G→U(N) be a continuous group homomorphism into the unitary N×N matrices. Its matrix elements are Uxy(g)=⟨ex,U(g)ey⟩, the (x,y) entry of U(g).
Fix d∈N. A multi-index is a function i:[d]→[N]; the set of them is Fun(d,N). The Weingarten integrals of degree d are
Iij=∫Gx=1∏dUi(x)j(x)(g)dμ(g),i,j∈Fun(d,N).
The tensor power H⊗d has the orthonormal basis ei=ei(1)⊗⋯⊗ei(d), and the tensor-power matrixU⊗d(g) has entries Uij⊗d(g)=∏x=1dUi(x)j(x)(g). The invariant tensors are
(H⊗d)G={t∈H⊗d:U⊗d(g)t=tfor all g∈G},
and the Haar average is the matrix P=∫GU⊗d(g)dμ(g). Given vectors a1,…,am∈H⊗d, let A=[⟨ei,ax⟩] be the Nd×m matrix whose columns are their coordinates, A∗ its conjugate transpose, A∗A=[⟨ax,ay⟩] their Gram matrix, and, when it is invertible, W=(A∗A)−1 the Weingarten matrix of a1,…,am.
Formalization targets
Goal: Theorem 2.1 (p. 4)
If a1,…,am is a basis of (H⊗d)G, then for all i,j∈Fun(d,N)
Iij=x,y=1∑mAixWxyAyj∗.
The identity is exact and holds for every compact group, every continuous unitary representation, every degree d and every choice of basis; nothing is fixed beyond the data.
Milestones (Section 2, pp. 3–4)
Iij=Pij: the Weingarten integrals are the matrix elements of the Haar average.
P∗=P, P2=P, and imP=(H⊗d)G: P is the orthogonal projection onto the invariant tensors (the existing platform theorem that a Haar average of a compact group representation is an idempotent fixing exactly the invariant vectors is included as a supporting reference item).
For linearly independent columns, the selfadjoint idempotent with image span{ax} is A(A∗A)−1A∗.
Significance
Theorem 2.1 reduces a family of analytic problems — Haar integrals of polynomials in matrix entries — to two algebraic ones: find a basis of the invariant tensors of G in H⊗d (classical invariant theory), and invert its Gram matrix. For U(N) the invariants come from Schur–Weyl duality and the Weingarten matrix becomes the Weingarten function on permutations; for O(N) and Sp(N) the invariants are indexed by pair partitions; for the symmetric group S(N) acting by permutation matrices the invariants are indicators of set partitions. Each of these case-specific formulas, surveyed in Sections 3–5 of the paper, is an instance of Theorem 2.1, so a formal proof of the theorem is the common entry point for formalizing any of them.
The theorem is classical and its proof is elementary once stated; it is not open. No machine-checked proof is known to exist in Mathlib or on this platform: Mathlib has Haar measure, unitary groups and Gram matrices (including the equivalence of linear independence with invertibility of the Gram matrix), but not the averaging projection of a representation in matrix form nor the formula A(A∗A)−1A∗ for orthogonal projections. The mission produces both as reusable lemmas and the Fundamental Theorem itself.
Difficulty
The argument is short on paper; the formal work sits in three places. First, selfadjointness of P needs ∫GF(g−1)dμ=∫GF(g)dμ, i.e. unimodularity of a compact group, which Mathlib does not provide as an instance for non-commutative groups. Second, identifying the image of P with the invariant tensors needs both directions: invariant vectors are fixed by P, and every vector Pt is invariant, which uses left invariance of μ inside a matrix-valued Bochner integral. Third, the projection formula needs invertibility of A∗A from linear independence and an argument that a selfadjoint idempotent is determined by its image. Commuting the integral with matrix entries and products is routine but must be done with care for the matrix norm chosen.
Formalization scope
H is CN with its standard basis (a finite-dimensional Hilbert space with a specified orthonormal basis is the same thing after identification); U is a monoid homomorphism G →* Matrix.unitaryGroup (Fin N) ℂ, assumed continuous. Indices are 0-based: [N] is Fin N, multi-indices are Fin d → Fin N, tensors are (Fin d → Fin N) → ℂ.
G carries [IsTopologicalGroup G] [CompactSpace G] [T2Space G] [BorelSpace G], and μ is a Haar measure that is a probability measure. Right invariance is not assumed; it is a consequence that solvers must prove.
d ranges over all of N, including d=0, where both sides equal 1; the paper takes d positive.
The inner product is conjugate-linear in the first slot, so Ayj∗=Ajy and A∗ is Mathlib's conjugate transpose.
P is a Bochner integral of matrices, taken with the entrywise sup norm; in finite dimension the integral is independent of the norm.
The basis hypothesis is two hypotheses: LinearIndependent ℂ a and span (range a) = invariants U d, with the invariant subspace defined from the representation. Without independence Mathlib's matrix inverse returns 0 on a singular Gram matrix, and the statement would be false; it is therefore not a trivializing gap but a necessary hypothesis.
Correction. The paper says "a basis a1,…,am of (H)⊗d"; the same paragraph calls the ax "the basic invariants", and with a basis of all of H⊗d the right-hand side would be the identity. The formal statement uses a basis of (H⊗d)G.
A trivializing formalization — defining the Weingarten integral as AWA∗, defining the invariant space as the image of P, or assuming that P projects onto the span of the ax — is excluded: the left side is the Haar integral of the product of matrix elements written out, and the invariant space is defined by U⊗d(g)t=t.
Needed infrastructure: matrix-valued Bochner integrals commuting with entries and with matrix multiplication; unimodularity of compact groups; the orthogonal-projection formula. All three are reusable well beyond this mission, and contributions of these general lemmas are welcome.
D. Weingarten, Asymptotic behavior of group integrals in the limit of infinite rank, J. Math. Phys. 19 (1978), 999–1001. https://doi.org/10.1063/1.523807
B. Collins, Moments and cumulants of polynomial random variables on unitary groups, the Itzykson–Zuber integral, and free probability, Int. Math. Res. Not. 2003, no. 17, 953–982. https://doi.org/10.1155/S107379280320917X
B. Collins, P. Śniady, Integration with respect to the Haar measure on unitary, orthogonal and symplectic group, Comm. Math. Phys. 264 (2006), 773–795. https://doi.org/10.1007/s00220-006-1554-3