Lectures on Modern Convex Optimization V: Semidefinite Representability of Symmetric Functions of EigenvaluesTextbook
Motivation
A convex optimization problem can be handed to a semidefinite programming solver only after its constraints and objective have been written as linear matrix inequalities (LMIs): conditions of the form , where is affine in the decision variables and in auxiliary variables , and means positive semidefinite. A large share of modern engineering optimization is built this way, in robust control, structural design, and statistics and machine learning (nuclear-norm and spectral-norm regularization, eigenvalue optimization). Modeling tools such as CVX and YALMIP rest on a catalog of functions known to admit such a description.
Lecture 4 of A. Ben-Tal and A. Nemirovski, Lectures on Modern Convex Optimization (SIAM, 2001, doi:10.1137/1.9780898718829), builds that catalog for the semidefinite cone. Its central entry is a general rule: a convex function of a symmetric matrix that depends only on the eigenvalues, symmetrically, is expressible by LMIs whenever the underlying function of a vector is. This covers the largest eigenvalue, sums of the largest eigenvalues (Ky Fan, 1949), Schatten norms and powers of the determinant at once. The tool behind it is the theory of majorization (Hardy, Littlewood and Pólya, 1929; Rado, 1952) together with Birkhoff's theorem on double-stochastic matrices (1946).
Setting
For a symmetric real matrix , denotes its eigenvalues counted with multiplicity in nonascending order, . is the sum of the largest. For a vector , is the sum of its largest entries, and is the convex hull of the coordinate permutations of . A function is symmetric if permuting the coordinates of its argument leaves its value unchanged. A matrix is double-stochastic if its entries are nonnegative and every row and column sums to .
A function is semidefinite representable (SDr) if there is a map , affine in with values in the symmetric matrices, such that
For a matrix (), the singular values are , and .
Formalization targets
Goal: Proposition 4.2.1 (p. 151)
Let be symmetric and SDr with representation . Then for every symmetric matrix and real ,
Milestones, in attack order
- (A), p. 148: for every .
- Example 18.c, p. 147: for , iff there are and with , , (4.2.2).
- Birkhoff's theorem, p. 281: the double-stochastic matrices are exactly the convex combinations of permutation matrices. This is referenced, already proved on the platform as
FamousTheorems.birkhoff_von_neumann. - (C), p. 281: for a convex symmetric , and double-stochastic .
- Claim (majorization principle), p. 285: .
- (D), p. 282: for a real-valued convex symmetric , .
- Proposition 4.2.2, p. 154: for symmetric, monotone on and SDr, iff with , , for (4.2.4).
A further level
A strengthened variant of the goal states that (4.2.3) remains exact when the ordering constraint is removed.
Significance
The result itself. Proposition 4.2.1 reduces semidefinite representability of a spectral function on , a space of dimension , to semidefinite representability of a symmetric function on . The extra cost is scalar variables, ordering inequalities, Ky Fan constraints (each an LMI of size by 18.c) and one trace equation. The representation is explicit, so a modeling language can generate it mechanically. Proposition 4.2.2 does the same for functions of singular values, which covers the nuclear norm and the operator norm of rectangular matrices. Fact (D) is the variational description that makes convex (Davis, 1957; Lewis, 1996).
Formalizing it. All results here are classical and proved in the book, partly through exercises (4.14–4.16 for the goal). None of them is formalized on Prove2Me beyond Birkhoff's theorem, which Mathlib has as doublyStochastic_eq_convexHull_permMatrix. A related proved platform item, Matrix.IsHermitian.sorted_eigenvalues_le_add_of_re_dotProduct_le, is a Weyl-type inequality that implies (A) when its slack is zero. Mathlib has sorted eigenvalues of Hermitian matrices (eigenvalues₀) but no Ky Fan sums, no majorization theory and no notion of semidefinite representation. This mission produces them.
Difficulty
The forward direction of every representation is a choice of witness. The content is in the converse: from an arbitrary feasible point of the LMI system one must recover the inequality for the eigenvalues. For (4.2.3) this means showing that partial-sum domination plus equal totals places inside the convex hull of the permutations of the witness . The obvious coordinatewise comparison fails, because is not dominated entrywise by . The hard half of the majorization principle (Exercise 4.15) is this step, and it does not follow from convexity of alone. In Proposition 4.2.2 the witness only dominates the singular values in partial sums, with no equality of totals, so majorization alone does not suffice there.
Formalization scope
Vectors are Fin m → ℝ (0-based indices) and matrices Matrix (Fin m) (Fin m) ℝ. Symmetric means Matrix.IsHermitian and means Matrix.PosSemidef, which includes symmetry. is Mathlib's antitone IsHermitian.eigenvalues₀; it is applied only to symmetric matrices. Functions with values in are EReal-valued and never , and only order comparisons with real numbers are made on them. A representation is an explicit affine map with symmetric coefficient matrices. In Proposition 4.2.2 the function , given on only, is extended by outside . The singular values are written as , which equals . Typos on the page are corrected and listed in the items: for in 18.c, and and in Proposition 4.2.2.
The claim " is SDr" admits a trivializing formalization: asserting that some LMI system projects onto the epigraph, with no control over its shape or size, says nothing beyond convexity. Every statement here instead pins the representation down as the book's explicit system (4.2.2), (4.2.3) or (4.2.4), quantified over every symmetric matrix and every real .
Beyond the paper. One strengthened variant is posed: Proposition 4.2.1 with the constraint removed from (4.2.3) (finding F1). The evidence is that never exceeds the sum of the largest entries of , so the remaining constraints already express majorization. The variant is not a milestone, and the book's statement stays the goal.
A complete development needs:
- majorization on (sums of largest entries, the permutation hull);
- Birkhoff's theorem, available in Mathlib;
- monotonicity of sorted eigenvalues in the Loewner order;
- spectral decomposition for constructing witnesses;
- singular values through .
Majorization, the Ky Fan representation and fact (A) are reusable well beyond this chapter, for instance in eigenvalue optimization and the analysis of interior-point methods. Contributions of these as standalone results are welcome.
Selected references
- A. Ben-Tal, A. Nemirovski, Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications, MPS/SIAM Series on Optimization, SIAM, 2001. doi:10.1137/1.9780898718829
- G. Birkhoff, Tres observaciones sobre el álgebra lineal, Univ. Nac. Tucumán Rev. Ser. A 5 (1946), 147–151.
- K. Fan, On a theorem of Weyl concerning eigenvalues of linear transformations I, Proc. Natl. Acad. Sci. USA 35 (1949), 652–655. doi:10.1073/pnas.35.11.652
- A. S. Lewis, Convex analysis on the Hermitian matrices, SIAM J. Optim. 6 (1996), 164–177. doi:10.1137/0806009
- A. W. Marshall, I. Olkin, B. C. Arnold, Inequalities: Theory of Majorization and Its Applications, 2nd ed., Springer, 2011. doi:10.1007/978-0-387-68276-1