Convex Optimization VI: Self-Concordance and the Barrier MethodTextbook
Why do interior-point methods solve convex programs in $O(\sqrt{m}\log(1/\varepsilon))$ Newton steps? Nesterov and Nemirovskii's answer is **self-concordance**: a convex function whose third derivative is controlled by its second, $|\varphi'''(t)| \le 2\,\varphi''(t)^{3/2}$ along every line, admits a Newton analysis with *absolute* constants and no condition number — and the logarithmic barrier is self-concordant. This mission formalizes §9.6 and Chapter 11 of Boyd & Vandenberghe: the self-concordance calculus, the Newton-decrement analysis, the duality gap $m/t$ along the central path, the per-centering work bound $m(\mu - 1 - \log\mu)/\gamma + c$, and the crown result — with the aggressive schedule $\mu = 1 + 1/\sqrt{m}$ the barrier method reaches duality gap $\varepsilon$ after
$$\Bigl\lceil \sqrt{m}\,\log_2\bigl(m/(t^{(0)}\varepsilon)\bigr)\Bigr\rceil$$
centering steps, each of uniformly bounded Newton cost.
19 thms5 active users
🏆Completed
Captain: Shuze Chen
Convex Optimization III: Conic Duality and the S-procedureTextbook
Two quadratic functions can be compared losslessly. The **S-procedure** says that, when the constraint is strictly feasible, the implication
$$q_1(x) \le 0 \;\Longrightarrow\; q_2(x) \le 0, \qquad q_k(x) = x^{T}F_k x + 2g_k^{T}x + h_k,$$
holds if and only if a single nonnegative multiplier certifies it as a matrix inequality, $\lambda \begin{bmatrix} F_1 & g_1 \\ g_1^{T} & h_1\end{bmatrix} \succeq \begin{bmatrix} F_2 & g_2 \\ g_2^{T} & h_2\end{bmatrix}$ for some $\lambda \ge 0$. It is a cornerstone of control theory, trust-region methods and robust optimization, and a rare case in which a nonconvex problem has zero duality gap. The route runs through the theory this mission builds from Boyd & Vandenberghe §5.8–5.9 and Appendix B: strong alternatives for convex inequality systems, cone-program strong duality under a generalized Slater condition, semidefinite programming duality, the LMI theorems of alternatives, and the hidden convexity of the joint range of two quadratic forms.
17 thms5 active usersReviewed
🏆Completed
Captain: Shuze Chen
Convex Optimization I: Prékopa's TheoremTextbook
Log-concave functions are the meeting point of convex analysis and probability: densities of Gaussian, exponential, uniform and Wishart distributions are all log-concave, and countless facts of applied probability flow from one structural theorem — integrating out variables preserves log-concavity. This mission builds the convex-analysis spine of Boyd & Vandenberghe's *Convex Optimization* (Chapters 2–3) — separation and supporting hyperplanes, dual cones, the first- and second-order differential characterizations of convexity, Fenchel conjugacy — and climbs to Prékopa's theorem via the Prékopa–Leindler inequality, a landmark of Brunn–Minkowski theory absent from Mathlib.
29 thms5 active usersReviewed
🏆Completed
Captain: Shuze Chen
Convex Optimization V: Newton's MethodTextbook
The classical convergence theory of smooth convex minimization. For a function that is $m$-strongly convex and $M$-smooth ($mI \preceq \nabla^2 f(x) \preceq MI$), gradient descent converges linearly, while Newton's method exhibits its famous two phases: a *damped* phase in which every backtracking step decreases the objective by a fixed amount $\gamma$, and a *quadratically convergent* phase in which the scaled gradient norm squares at each step, $\tfrac{L}{2m^2}\lVert \nabla f(x^{+})\rVert_2 \le \bigl(\tfrac{L}{2m^2}\lVert \nabla f(x)\rVert_2\bigr)^2$. Together they give the iteration count of B&V (9.36),
$$\#\text{iterations} \;\le\; \frac{f(x^{(0)}) - p^{\star}}{\gamma} \;+\; \log_2\log_2(\varepsilon_0/\varepsilon), \qquad \gamma = \frac{\alpha\beta\eta^2 m}{M^2}, \quad \varepsilon_0 = \frac{2m^3}{L^2},$$
with $L$ the Lipschitz constant of the Hessian and $\alpha,\beta$ the backtracking parameters. This mission formalizes Chapters 9–10 of Boyd & Vandenberghe with every constant exactly as printed — a quantitative theory entirely absent from Mathlib.
12 thms4 active usersReviewed
🏆Completed
Captain: Shuze Chen
Convex Optimization II: KKT ConditionsTextbook
The Karush–Kuhn–Tucker conditions are the central result of convex optimization: for a convex differentiable problem satisfying Slater's condition, a point is optimal exactly when primal feasibility, dual feasibility, complementary slackness and Lagrangian stationarity hold. This mission formalizes Chapters 4–5 of Boyd & Vandenberghe end to end — the first-order optimality criterion, concavity of the Lagrange dual, weak duality, Slater's strong-duality theorem with dual attainment (via the separating-hyperplane argument of §5.3.2), the saddle-point characterization, sensitivity bounds and Pareto scalarization — culminating in the full KKT characterization.
15 thms4 active usersReviewed
🏆Completed
Captain: Shuze Chen
Introduction to Linear Optimization X: Max-Flow Min-CutTextbook
How much flow can be sent from a source $s$ to a sink $t$ through a network with arc capacities $u_{ij}\in(0,\infty]$ — and what certifies that no more is possible? This mission formalizes §7.4-7.5 of Bertsimas & Tsitsiklis. The circulation calculus of §7.4 supplies the two structural tools: the flow decomposition theorem (Lemma 7.1 — every nonzero nonnegative circulation is a positive combination $\mathbf{f}=\sum_i a_i\mathbf{f}^i$ of simple circulations with only forward arcs, with integer $a_i$ when $\mathbf{f}$ is integer) and the optimality criterion for the minimum cost network flow problem (Theorem 7.6 — a feasible flow is optimal if and only if there is no unsaturated cycle with negative cost). Section 7.5 then formulates the maximum flow problem ($\max b_s$ s.t. $\mathbf{A}\mathbf{f}=\mathbf{b}$, $b_t=-b_s$, $b_i=0$ for $i\ne s,t$, $0\le\mathbf{f}\le\mathbf{u}$), defines augmenting paths (Definition 7.2: $f_{ij}<u_{ij}$ on forward arcs, $f_{ij}>0$ on backward arcs) and the Ford–Fulkerson algorithm, and proves integer invariance and finite termination for integer capacities (Theorem 7.8). The goal is Theorem 7.10:
(a) if the Ford–Fulkerson algorithm terminates because no augmenting path can be found, the current flow is optimal;
(b) the value of the maximum flow equals the minimum cut capacity $$C(S)=\sum_{\{(i,j)\in\mathcal{A}\mid i\in S,\,j\notin S\}}u_{ij}$$ — the archetypal combinatorial min-max theorem, which the book notes can also be read as LP duality (pp. 311-312).
14 thms4 active usersReviewed
🏆Completed
Captain: Shuze Chen
Matrix Completion has No Spurious Local MinimumResearch Paper
Matrix completion — recovering a low-rank matrix $M = ZZ^\top$ from a small random subset of its entries — powers recommender systems and collaborative filtering. In practice it is solved by running (stochastic) gradient descent on the non-convex objective $$f(X)=\min_X\frac12\|P_\Omega(M-XX^\top)\|_F^2+\lambda R(X)$$
where $\Omega=\{(i,j)|M_{i,j} \text{ is observed}\}$ and $R(X)$ is a certain regularizer. from a random starting point, and it just works.
Ge, Lee and Ma (NeurIPS 2016 Best student paper award) explained why: the regularized objective has *no spurious local minima* — every local minimum is global and exactly recovers $M$. This mission formalizes that landmark theorem in Lean 4, in its strongest known form and along its simplest known proof: the unified landscape analysis of Ge–Jin–Zheng (ICML 2017) and an improved sampling bound in Chen–Li (JMLR 2019). Conditional on an explicit good-sample predicate (which holds with high probability under Bernoulli sampling), every local minimum $X$ of $f$ satisfies $XX^\top = ZZ^\top$.
Every full-dimensional convex body is sandwiched between an ellipsoid and its $n$-fold dilation: shrinking the minimum-volume covering (Löwner–John) ellipsoid $\mathcal{E}$ about its centre $x_0$ by the factor $1/n$ lands inside the body,
$$x_0 + \tfrac{1}{n}\,(\mathcal{E} - x_0) \;\subseteq\; C \;\subseteq\; \mathcal{E},$$
and the factor $n$ is tight on simplices. This rounding theorem underlies the ellipsoid method, John's theorem on the Banach–Mazur distance to the Euclidean ball, and much of modern convex geometry. The mission formalizes §8.4 of Boyd & Vandenberghe for polytopes $C = \operatorname{conv}\{x_1,\dots,x_m\}$, exactly as the book proves it: existence and uniqueness of the extremal ellipsoid, the KKT identities at the normalized optimum ($\sum_i \lambda_i x_i x_i^{T} = I$, $\sum_i \lambda_i x_i = 0$, $\sum_i \lambda_i = n$), the convex-combination step that produces the $1/n$ ball, and affine invariance.
8 thms3 active usersReviewed
🏆Completed
Captain: Shuze Chen
Introduction to Linear Optimization XIII: Lagrangean Duality and Integer ProgrammingTextbook
Linear programming has a complete duality theory; integer programming does not — and the Lagrangean dual measures exactly how far duality reaches. This mission formalizes the duality theory of integer programming from Section 11.4 of Bertsimas–Tsitsiklis, built on the general linear programming duality of Section 4.10. For the integer program $$Z_{IP} = \min\{c'x : Ax \ge b,\ Dx \ge d,\ x \text{ integer}\}$$ with integer data, the complicating constraints $Ax \ge b$ are dualized with multipliers $p \ge 0$ over the tractable set $X = \{x \text{ integer} \mid Dx \ge d\}$: the dual function is $$Z(p) = \min_{x \in X}\big(c'x + p'(b - Ax)\big)$$ and the Lagrangean dual is $Z_D = \max_{p \ge 0} Z(p)$. Weak duality $Z_D \le Z_{IP}$ (Theorem 11.2) always holds, but strong duality can fail. The convex hull $CH(X)$ of the integer points of a polyhedron with integer data is itself a polyhedron (Theorem 11.3, Meyer's theorem), and the capstone — Theorem 11.4, the central result of Section 11.4 — identifies the Lagrangean dual exactly: $Z_D$ equals the optimal cost of the linear program $$\min\{c'x : Ax \ge b,\ x \in CH(X)\}$$. This is the geometric explanation of the strength of Lagrangean relaxation, yields the bound ordering $Z_{LP} \le Z_D \le Z_{IP}$, and Corollary 11.1 characterizes exactly when the bounds collapse. The polyhedral engine is the general weak/strong duality pair (Theorems 4.17/4.18) over a primal $\min c'x$ s.t. $Ax \ge b$, $x \in P = \{x \mid Dx \ge d\}$, and the formulation-strength comparison $P_{sub} \subseteq P_{cut}$ of Theorem 10.1 supplies the motivating principle that tighter relaxations of the same integer set give sharper bounds.
18 thms3 active usersReviewed
🏆Completed
Captain: Shuze Chen
Introduction to Linear Optimization XII: Interior Point Methods and Path FollowingTextbook
Interior point methods solve linear programs by moving through the interior of the feasible set instead of along its edges — the approach that turned Karmarkar's 1984 breakthrough into today's practical large-scale solvers. This mission formalizes the primal path following algorithm of Chapter 9 of Bertsimas–Tsitsiklis. For $\mu > 0$ the logarithmic barrier $$B_\mu(\mathbf{x}) = \mathbf{c}'\mathbf{x} - \mu\sum_{j=1}^n \log x_j$$ replaces the constraint $\mathbf{x} \ge \mathbf{0}$; the minimizers $\mathbf{x}(\mu)$ of $B_\mu$ over $\{A\mathbf{x} = \mathbf{b}\}$ trace the central path, characterized by the KKT conditions (9.17): $A\mathbf{x} = \mathbf{b}$, $\mathbf{x} \ge \mathbf{0}$, $A'\mathbf{p} + \mathbf{s} = \mathbf{c}$, $\mathbf{s} \ge \mathbf{0}$, $XS\mathbf{e} = \mu\mathbf{e}$ (Lemma 9.5). The algorithm follows the path with one Newton step of the barrier problem per shrink $\mu^{k+1} = \alpha\mu^k$, maintaining the proximity invariant $$\|\frac{1}{\mu}XS\mathbf{e} - \mathbf{e}\| \le \beta$$. The goal theorem is Theorem 9.7: with $\alpha = 1 - \frac{\sqrt{\beta}-\beta}{\sqrt{\beta}+\sqrt{n}}$ and a $\beta$-close start, after $K = \Big\lceil \frac{\sqrt{\beta}+\sqrt{n}}{\sqrt{\beta}-\beta}\,\log\frac{(\mathbf{s}^0)'\mathbf{x}^0(1+\beta)}{\varepsilon(1-\beta)} \Big\rceil$ iterations the algorithm reaches primal and dual feasible solutions with duality gap $(\mathbf{s}^K)'\mathbf{x}^K \le \varepsilon$ — the explicit form of the celebrated $O(\sqrt{n}\log(1/\varepsilon))$ iteration bound. Alongside it we formalize the generic potential-reduction scheme (Theorem 9.4): any algorithm cutting $G(\mathbf{x},\mathbf{s}) = q\log\mathbf{s}'\mathbf{x} - \sum_j \log x_j - \sum_j \log s_j$ by $\delta$ per step reaches gap $\varepsilon$ within an explicit $K$.
9 thms3 active usersReviewed
🏆Completed
Captain: Shuze Chen
Introduction to Linear Optimization XI: The Ellipsoid MethodTextbook
Can the feasibility of a system of linear inequalities be decided in a provably small number of iterations? The ellipsoid method — the algorithm with which Khachiyan showed in 1979 that linear programming is polynomially solvable — answers this with pure convex geometry. This mission formalizes Chapter 8 of Bertsimas–Tsitsiklis. An ellipsoid is $$E(\mathbf{z}, D) = \{\mathbf{x} \in \mathbb{R}^n \mid (\mathbf{x}-\mathbf{z})'D^{-1}(\mathbf{x}-\mathbf{z}) \le 1\}$$ with $D$ symmetric positive definite. The geometric engine is Theorem 8.1: the half-ellipsoid $E \cap \{\mathbf{x} \mid \mathbf{a}'\mathbf{x} \ge \mathbf{a}'\mathbf{z}\}$ is contained in the explicitly constructed ellipsoid $E' = E(\bar{\mathbf{z}}, \bar{D})$, $$\bar{\mathbf{z}} = \mathbf{z} + \frac{1}{n+1}\frac{D\mathbf{a}}{\sqrt{\mathbf{a}'D\mathbf{a}}},$$ $$\bar{D} = \frac{n^2}{n^2-1}\big(D - \frac{2}{n+1}\frac{D\mathbf{a}\mathbf{a}'D}{\mathbf{a}'D\mathbf{a}}\big),$$ and the volume contracts: $$\mathrm{Vol}(E') < e^{-1/(2(n+1))}\,\mathrm{Vol}(E)$$. Two integer-data estimates make the contraction decisive: every extreme point of $P = \{\mathbf{x} \mid A\mathbf{x} \ge \mathbf{b}\}$ with entries bounded by $U$ has coordinates in $[-(nU)^n, (nU)^n]$ (Lemma 8.2), and a full-dimensional bounded such polyhedron has $\mathrm{Vol}(P) > n^{-n}(nU)^{-n^2(n+1)}$ (Lemma 8.4). The goal theorem is Theorem 8.2: started on a ball $E(\mathbf{x}_0, r^2 I)$ of volume at most $V$ containing $P$, with $v$ a lower bound on $\mathrm{Vol}(P)$ when $P$ is nonempty, the ellipsoid method correctly decides whether $P$ is empty within $t^* = \lceil 2(n+1)\log(V/v) \rceil$ iterations — the explicit iteration count behind the polynomial-time headline.
14 thms3 active usersReviewed
🏆Completed
Captain: Shuze Chen
Introduction to Linear Optimization IX: Network Flow IntegralityTextbook
Why do network linear programs return integer answers for free? This mission formalizes the structural theory of the minimum cost network flow problem of Chapter 7 of Bertsimas & Tsitsiklis: a directed graph $G=(\mathcal{N},\mathcal{A})$ with external supplies $b_i$, arc costs $c_{ij}$, and the node-arc incidence matrix $\mathbf{A}$ — an $n\times m$ matrix in which every column has exactly one $+1$ (start node) and one $-1$ (end node) — so that flow conservation reads $\mathbf{A}\mathbf{f}=\mathbf{b}$, forcing the standing assumption $\sum_{i\in\mathcal{N}} b_i=0$. Because the rows of $\mathbf{A}$ sum to zero, the book works with the truncated matrix $\tilde{\mathbf{A}}$ of the first $n-1$ rows. The combinatorial heart is the correspondence between algebra and graph structure: a set $T$ of $n-1$ arcs forming a tree determines a unique tree solution of $\tilde{\mathbf{A}}\mathbf{f}=\tilde{\mathbf{b}}$, $f_{ij}=0$ off $T$ (Theorem 7.3); connectedness makes $\tilde{\mathbf{A}}$ full-rank (Corollary 7.1); and a flow vector is a basic solution if and only if it is a tree solution (Theorem 7.4). The goal theorem is the integrality theorem (Theorem 7.5): for the uncapacitated problem on a connected graph, every basis matrix $\mathbf{B}$ has an integer inverse $\mathbf{B}^{-1}$ (its determinant is $\pm 1$ by the tree/lower-triangular argument), integer supplies make every basic solution integer, and integer costs make every dual basic solution integer — whence integer optimal primal and dual solutions exist whenever the optimal cost is finite (Corollary 7.2). This is the fountainhead of combinatorial integrality in linear optimization, feeding the max-flow min-cut mission that follows.
18 thms3 active usersReviewed
🏆Completed
Captain: Shuze Chen
Introduction to Linear Optimization VIII: Sensitivity Analysis and Subgradients of the Optimal CostTextbook
How does the optimal cost of a linear program respond when the problem data change? Chapter 5 of Bertsimas-Tsitsiklis studies the standard form problem $\min\{c'x \mid Ax = b,\ x \ge 0\}$ (rows of $A$ linearly independent) as the requirement vector $b$ and the cost vector $c$ vary. On the convex set $S = \{b \mid P(b) \neq \emptyset\}$ of feasible right-hand sides, and under the standing assumption that the dual feasible set is nonempty, the optimal cost $F(b)$ is finite and convex (Theorem 5.1) — indeed $F(b) = \max_{i} (p^i)'b$ over the extreme points $p^1, \dots, p^N$ of the dual feasible set, a piecewise linear convex function whose breakpoints are exactly where the dual optimum is non-unique. The capstone (Theorem 5.2) identifies the generalized gradients of $F$: if the primal at $b^*$ is feasible with finite optimal cost, then $p$ is an optimal solution of the dual if and only if $p$ is a subgradient of $F$ at $b^*$ (Definition 5.1: $F(b^*) + p'(b - b^*) \le F(b)$ for all $b \in S$) — the precise sense in which dual variables are marginal costs. Dually (Theorem 5.3), the set $T$ of cost vectors with finite optimal cost is convex, the optimal cost $G(c)$ is concave on $T$, and near any $c$ with a unique primal optimum $x^*$, $G$ is linear with gradient $x^*$. Local ranging (Section 5.1) and parametric programming (Section 5.5) are the procedural companions, folded into the design notes.
11 thms3 active usersReviewed
🏆Completed
Captain: Shuze Chen
Introduction to Linear Optimization VII: Cones, Extreme Rays, and the Resolution TheoremTextbook
How can an unbounded polyhedron be described by finitely many geometric objects? Sections 4.8-4.9 of Bertsimas-Tsitsiklis build the cone machinery: recession cones $\{d \mid Ad \ge 0\}$ and their rays, extreme rays (defined, like basic solutions, by $n-1$ linearly independent active constraints), the pointedness criterion (Theorem 4.12: $0$ is an extreme point of a polyhedral cone iff the cone contains no line iff $n$ of the constraint vectors are linearly independent), and the characterization of unbounded linear programs (Theorems 4.13-4.14: over a pointed polyhedral cone, and then over any polyhedron with an extreme point, the optimal cost is $-\infty$ iff some extreme ray $d$ has $c'd < 0$). The capstone is the resolution theorem (Theorem 4.15): a nonempty polyhedron $P$ with at least one extreme point equals $Q = \{\sum_i \lambda_i x^i + \sum_j \theta_j w^j \mid \lambda_i \ge 0, \theta_j \ge 0, \sum_i \lambda_i = 1\}$ — the convex hull of its extreme points plus the cone generated by a complete set of its extreme rays. It specializes to Theorem 2.9 / Corollary 4.4 (a nonempty bounded polyhedron is the convex hull of its extreme points) and Corollary 4.5 (a pointed polyhedral cone is generated by its extreme rays). The converse, Theorem 4.16, states that every finitely generated set is a polyhedron — in particular the convex hull of finitely many vectors is a polyhedron. Together these form the Minkowski-Weyl equivalence of the two representations of polyhedra, verified absent from Mathlib and the genuine content of this mission.
21 thms3 active usersReviewed
🏆Completed
Captain: Shuze Chen
Introduction to Linear Optimization IV: The Simplex MethodTextbook
How does one actually solve a linear program? Chapter 2 showed that if a standard-form problem $\min c'x$ subject to $Ax = b$, $x \ge 0$ has an optimal solution, it has an optimal basic feasible solution; the simplex method searches among basic feasible solutions, moving along edges of the feasible set in cost-reducing directions. This mission formalizes the mathematics of Chapter 3 of Bertsimas–Tsitsiklis: feasible directions, the reduced costs $$\bar{c}_j = c_j - c_B'B^{-1}A_j$$ measuring the cost rate along the basic directions, the optimality conditions of Theorem 3.1 ($\bar{c} \ge 0$ implies optimality, and conversely at nondegenerate optima), the basis change of Theorem 3.2, and the pivot iteration itself — encoded as a predicate relating a basis/BFS pair to its successor, so that every theorem covers every pivoting rule. The goal theorem is Theorem 3.3: if the feasible set is nonempty and every basic feasible solution is nondegenerate, the simplex method terminates after a finite number of iterations, ending either with an optimal basis and an associated optimal basic feasible solution, or with a direction $d$ satisfying $Ad = 0$, $d \ge 0$, $c'd < 0$ certifying optimal cost $-\infty$. The secondary capstone, Theorem 3.4, removes the nondegeneracy assumption: under the lexicographic pivoting rule every tableau row other than the zeroth stays lexicographically positive, the zeroth row strictly increases lexicographically, and the simplex method terminates on every problem — the anticycling guarantee that also supplies the optimal-basis existence used by the strong duality theorem of Mission V.
16 thms3 active usersReviewed
🏆Completed
Captain: Shuze Chen
Introduction to Linear Optimization I: Polyhedra and Basic Feasible SolutionsTextbook
Every linear programming problem asks to minimize a linear cost $c'x$ over a polyhedron — a set of the form $P = \{x \in \mathbb{R}^n \mid Ax \ge b\}$, or in standard form $\{x \mid Ax = b,\ x \ge 0\}$. Chapter 2 of Bertsimas–Tsitsiklis develops the geometry of these feasible sets, and its central achievement is making the intuitive notion of a "corner point" rigorous. There are three natural candidates: the *extreme point* — a point of $P$ that cannot be written as a convex combination of two other points of $P$ (purely geometric, representation-independent); the *vertex* — the unique minimizer of some linear cost $c'y$ over $P$ (geometric, via supporting hyperplanes); and the *basic feasible solution* — a feasible point at which $n$ linearly independent constraints are active (algebraic, the object the simplex method actually computes with). This mission formalizes polyhedra, active constraints, vertices and basic (feasible) solutions, and proves the fundamental Theorem 2.3: for a nonempty polyhedron all three notions coincide. Around the capstone sit the supporting pillars: polyhedra are convex (Theorem 2.1), the characterization of points pinned down by $n$ linearly independent active constraints (Theorem 2.2), finiteness of the set of basic solutions (Corollary 2.1), and the basis-column characterization of basic solutions in standard form (Theorem 2.4) — the combinatorial engine behind the simplex method of Chapter 3 and the root of the entire series.
9 thms3 active usersReviewed
🏆Completed
Captain: wenxinzhang
Vector Space Methods XIII: Conjugate-Gradient ConvergenceTextbook
## Motivation
The [conjugate-gradient method in Luenberger's Chapter 10](https://sites.science.oregonstate.edu/~show/old/142_Luenberger.pdf) is one of the most enduring consequences of Hilbert-space geometry in numerical optimization. For a bounded self-adjoint coercive operator, it solves the quadratic first-order equation `Q x = b` using only operator applications, inner products, and a short recurrence. Luenberger develops the method from steepest descent and conjugate directions, then proves convergence in a general real Hilbert space rather than only for finite matrices. This mission formalizes that full setting. It also repairs a practical omission in the printed recursion: division formulas are undefined after exact convergence, so the formal algorithm explicitly stops and stutters once its search direction is zero.
## Setting
Let `H` be a **complete real inner-product space** and `Q : H →L[ℝ] H` a bounded **self-adjoint operator**. Constants `m` and `M` satisfy `0 < m ≤ M` and
$$
m\lVert x\rVert^2 \le \langle x,Qx\rangle
\le M\lVert x\rVert^2
$$
for every `x`. The first inequality is **coercivity**; together with self-adjointness it supplies the positive `Q`-energy. For a right-hand side `b` and initial point `x₀`, the initial residual and direction are both `b - Q x₀`. A **conjugate-gradient state** records the current iterate, residual, and direction. If the direction is nonzero, the next state uses Luenberger's `alpha` and `beta` ratios. If the direction is zero, `conjugateGradientStep` returns the same state, so every natural-number iterate is total and all denominators occur only on the active branch.
## Formalization targets
The root theorem `VectorSpaceOpt.conjugate_gradient_converges` states that there is a unique `xStar` satisfying `Q xStar = b` and that the iterate component of the guarded conjugate-gradient state tends to `xStar` in norm. Four milestones provide reusable structure. `coercive_selfadjoint_bijective` establishes existence and uniqueness for `Q x = b` from bounded self-adjoint coercivity. `conjugate_directions_converge` formalizes §10.6, Theorem 1: a complete sequence of nonzero pairwise `Q`-orthogonal directions produces residuals orthogonal to every earlier direction and iterates converging to the solution. `cg_directions_conjugate_until_stop` records the §10.8 invariants only before the explicit stopping time. `cg_energy_contraction` captures the uniform energy reduction factor derived from the bounds `m` and `M`.
The total algorithm is represented by `conjugateGradientIterate`, and its error functional is
$$
E(x)=\langle x-x^*,Q(x-x^*)\rangle.
$$
These definitions are proposed as mission-owned reusable objects in the shared `VectorSpaceOpt` namespace.
## Significance
The mission gives a coordinate-free verification target for an algorithm usually presented through arrays and matrices. Its theorem applies directly to finite-dimensional symmetric positive-definite systems but also retains Luenberger's infinite-dimensional perspective. The guarded recursion is suitable for later executable specializations and makes exact termination a first-class semantic event. The coercivity and conjugate-directions milestones can be reused for Galerkin methods, preconditioned variants, and other Krylov algorithms, while the energy estimate provides a natural connection to condition-number convergence rates.
Unlike a matrix-only formalization, the Hilbert-space theorem cleanly separates the geometric reason for convergence from any storage representation. It therefore complements Mathlib's existing operator and orthogonality libraries and can serve as a specification against which finite implementations are later verified.
It also preserves the book's unifying theme: optimization algorithms arise from the geometry of carefully chosen inner products rather than from coordinate manipulation alone.
## Difficulty
The difficulty is medium to high. Algebraic invariants of the three-term recurrence involve several interacting orthogonality relations and require strict control of nonzero denominators. Infinite-dimensional convergence additionally uses density of the closed span of directions and comparison of the `Q`-energy with the ambient norm. The theorem must move between self-adjoint continuous linear maps, scalar inner products, filters on sequences, and function iteration. Exact termination creates a case split that informal accounts routinely ignore; the formal statement must show that the zero-direction branch is stable and already represents the solution.
## Formalization scope
The proposal follows §10.6 and §10.8, pp. 291–296, and uses Chapter 10, Problem 10 on p. 309 for the coercive-invertibility dependency. All assumptions on `Q`, `m`, and `M` that §10.8 inherits from the preceding sections are repeated explicitly. The conjugate-directions milestone explicitly assumes every direction is nonzero and that the closed span of the directions is the whole Hilbert space. The conjugate-gradient invariants are asserted only for iterations before a zero direction occurs. Once it occurs, the state stutters by definition; the proposal never relies on Lean's totalized value for `0 / 0`.
Luenberger's §10.7, Theorem 1 is not included as a literal milestone. As printed, its orthogonalization-of-moments statement omits self-adjointness of the auxiliary operator relative to the `Q` inner product and omits the linear-independence/nonbreakdown conditions needed to keep denominators nonzero. The mission instead isolates the `Q`-conjugacy invariant directly from §10.8. It does not claim finite-dimensional termination within `dim H` steps, floating-point stability, preconditioning, a sharp Chebyshev condition-number rate, or computability of equality tests on arbitrary Hilbert spaces.
## Selected references
- David G. Luenberger, *Optimization by Vector Space Methods*, Wiley, 1969, Chapter 10, §10.6, Theorem 1, pp. 291–292; §10.8, Theorem 1, pp. 294–296; Problem 10, p. 309. Scan: https://sites.science.oregonstate.edu/~show/old/142_Luenberger.pdf
- Lean community, *Mathlib documentation*, continuously updated: https://leanprover-community.github.io/mathlib4_docs/ (real inner-product spaces, continuous linear maps, coercivity, closed spans, orthogonality, and filter convergence).
6 thms2 active usersReviewed
🏆Completed
Captain: wenxinzhang
Vector Space Methods XIV: Quadratic Penalty ConvergenceTextbook
## Motivation
[Quadratic exterior penalties in Luenberger's §10.11](https://sites.science.oregonstate.edu/~show/old/142_Luenberger.pdf) replace a constrained problem by a sequence of unconstrained minimizations. The method is simple enough to state in a few lines, yet Luenberger's convergence theorem is strikingly general: no convexity, differentiability, or convergence of the full minimizer sequence is required. If penalty weights increase to infinity and a subsequence of exact penalty minimizers converges, lower semicontinuity alone makes its limit feasible and optimal. This mission isolates that robust primal convergence result as a tractable companion to the more analytic conjugate-gradient and optimal-control missions. It offers a clean formalization target with direct relevance to nonlinear programming and approximation schemes.
## Setting
Let `X` be a **topological space**, `f : X → ℝ`, and `G : X → (Fin p → ℝ)`. **Feasibility** means `G x i ≤ 0` for every component. Define the **positive part** componentwise and the **squared violation** by
$$
G_i^+(x)=\max(0,G_i(x)), \qquad
v(x)=\sum_i (G_i^+(x))^2.
$$
For a positive weight `K`, the penalty objective is `f x + K * v x`. A sequence `K n` is positive, nondecreasing, and tends to `+∞`. The constrained problem is assumed to have a minimizer `xStar`, and for each `n` an exact global minimizer `x n` of the corresponding penalty objective is supplied. A limit point is represented explicitly by a strictly increasing index map `phi` for which `x ∘ phi` tends to `x₀`.
## Formalization targets
The root `VectorSpaceOpt.quadratic_penalty_cluster_point_converges` formalizes §10.11, Theorem 1. Assuming lower semicontinuity of `f` and `v`, it concludes that every stated subsequential limit `x₀` is feasible, has the same objective value as `xStar`, and globally minimizes `f` over the feasible set.
Three milestones split the exact source content into reusable statements. `quadratic_penalty_basic_estimates` is §10.11, Lemma 1: the attained penalty values are nondecreasing, are bounded above by `f xStar`, and the stronger weighted violation `K n * v (x n)` tends to zero. `penalty_cluster_point_feasible` combines convergence of violations with lower semicontinuity at a subsequential limit to recover all component inequalities. `penalty_cluster_point_optimal` combines lower semicontinuity of `f`, the uniform upper bound `f (x n) ≤ f xStar`, feasibility of the limit, and optimality of `xStar` to identify the limiting objective value and global constrained optimality.
## Significance
The theorem captures the essential consistency guarantee behind one of the most widely used constraint-handling methods. Its assumptions separate optimization existence from convergence: minimizers of each auxiliary problem and at least one cluster point are assumed, while the theorem identifies what any such cluster point must be. The componentwise positive-part and violation definitions are reusable for augmented Lagrangians, exact penalties, barrier comparisons, and finite inequality systems. The basic-estimates lemma is particularly useful because it requires neither topology nor continuity and exposes a quantitative fact stronger than mere feasibility residual convergence.
Because the proof target is stated over an arbitrary topological space, the mission also clarifies which parts of penalty convergence are genuinely metric and which depend only on order, finite nonnegative sums, and lower semicontinuity. This abstraction is faithful to the source's vector-space viewpoint.
## Difficulty
The mission has moderate difficulty and relatively low infrastructure risk. The main analytic interfaces are lower semicontinuity along a convergent subsequence and real filter convergence to both zero and infinity. The basic estimates require reasoning simultaneously about minimizers for changing objectives, monotonicity of the weights, and the asymptotic product `K n * v (x n)`. The cluster-point theorem must extract componentwise feasibility from a finite sum of nonnegative squares without assuming continuity of `G`. Lean's `IsMinOn` does not itself assert membership in the feasible set, so feasibility of the known constrained minimizer is included separately rather than hidden in prose.
## Formalization scope
The proposal covers the primal part of §10.11: Lemma 1 on p. 305 and Theorem 1 on p. 306. It makes “limit point” precise through a strictly monotone subsequence, avoiding any assumption that the full sequence converges. The weight sequence may have repeated values because the source only needs it to be nondecreasing, but every weight is positive and the sequence tends to `atTop`. Lower semicontinuity is required for `f` and the composite violation `v`, exactly as in the book; continuity or componentwise lower semicontinuity of `G` is not substituted. Existence of `xStar` and of every penalty minimizer is assumed rather than derived from compactness or coercivity.
The mission does not include §10.11, Lemma 2 or Theorem 2 on dual multipliers. Those results add convexity and continuity assumptions and naturally require careful treatment of an extended-real dual functional. It also does not address approximate minimizers, rates, boundedness of the sequence, existence of cluster points, equality constraints beyond their encoding as paired inequalities, or finite exactness. Keeping those extensions separate preserves the unusually weak hypotheses and clear conclusion of the cited primal theorem.
## Selected references
- David G. Luenberger, *Optimization by Vector Space Methods*, Wiley, 1969, Chapter 10, §10.11, Lemma 1 and Theorem 1, pp. 305–306. Scan: https://sites.science.oregonstate.edu/~show/old/142_Luenberger.pdf
- Lean community, *Mathlib documentation*, continuously updated: https://leanprover-community.github.io/mathlib4_docs/ (lower semicontinuity, finite sums, `Fin`-indexed vectors, subsequences, global minima on sets, and filter convergence).
5 thms2 active usersReviewed
🏆Completed
Captain: wenxinzhang
Vector Space Methods XI: Generalized Kuhn–Tucker ConditionsTextbook
## Motivation
[Luenberger's generalized Kuhn–Tucker theorem](https://sites.science.oregonstate.edu/~show/old/142_Luenberger.pdf) turns inequality-constrained optimization into an order-theoretic statement on normed vector spaces. Instead of listing scalar inequalities, it lets a convex cone `P` define positivity in a target space `Z`; one condition `G x ≤ₚ 0` can therefore represent finite, infinite, or function-valued families of constraints. At a regular local minimizer, a positive continuous functional on `Z` simultaneously provides stationarity and complementary slackness. This mission is a separate capstone because the cone-separation argument is conceptually independent of the equality-constrained theorem and because Mathlib currently lacks this general cone-valued KKT result.
## Setting
Let `X` and `Z` be **real normed spaces**, `P : ConvexCone ℝ Z`, `f : X → ℝ`, and `G : X → Z`. The **cone order** is `coneLE P z₁ z₂`, meaning `z₂ - z₁ ∈ P`; strict inequality uses the topological interior of the **convex cone** `P`. The cone is assumed to have nonempty interior. At `x₀`, both `f` and `G` possess linear **Gâteaux derivatives** represented by continuous linear maps `f'` and `G'`. The source's **regularity condition** requires feasibility together with a direction `h` for which `G x₀ + G' h` lies strictly below zero in the cone order.
The point `x₀` is a local, not global, minimizer of `f` on `{x | coneLE P (G x) 0}`. The resulting multiplier `z₀ : Z →L[ℝ] ℝ` is positive on `P`. This mission reuses the previously published `VectorSpaceOpt.coneLE` and `VectorSpaceOpt.dualPositive` definitions from the global Lagrange-duality mission; it deliberately does not introduce equivalent duplicate constants.
## Formalization targets
The root theorem is `VectorSpaceOpt.generalized_kuhn_tucker`, corresponding to §9.4, Theorem 1. It produces `z₀` such that
$$
z₀(P) \subseteq [0,\infty), \qquad
f' + z₀ \circ G' = 0, \qquad
z₀(Gx₀)=0.
$$
Three milestones expose the exact logical interfaces of the source theorem. `kkt_no_strict_linearized_descent` says local minimality and feasibility exclude a direction that strictly decreases `f'` while making the linearized constraint strictly feasible. `kkt_linearized_separator` packages the separation step: nonintersection of the strict descent system, cone regularity, and nonempty cone interior yield a positive continuous multiplier with both KKT conclusions. `kkt_complementary_slackness` isolates the algebraic extraction of stationarity and complementarity from the separating inequality valid for every direction. The items use the shared namespace `VectorSpaceOpt` and list dependencies in this order.
## Significance
This mission generalizes the standard finite-dimensional KKT rule without choosing coordinates or reducing cone constraints to components. It provides a reusable basis for semi-infinite optimization, ordered Banach-space problems, and state constraints expressed in function spaces. The multiplier positivity predicate connects directly to the dual cone used in the earlier global duality mission, while complementarity links local differential theory to primal–dual optimality. A successful formalization would also close a conspicuous gap in general-purpose optimization infrastructure: cone-valued KKT conditions are referenced often but rarely available as a theorem with all topological hypotheses exposed.
The statement is also a useful stress test for compositional textbook formalization. It deliberately shares its order and dual-positivity vocabulary with an earlier mission, so subsequent results can consume one stable API instead of translating among locally invented conventions.
## Difficulty
The main challenge is functional-analytic separation. The relevant convex set mixes objective descent and strict cone feasibility, and the separating functional must be normalized so that its objective component is nonzero. Regularity rules out an abnormal separator and nonempty cone interior controls the sign of the `Z` component. The Gâteaux assumptions are directional rather than full Fréchet differentiability, so local contradiction statements must use only the one-dimensional expansions actually supplied. Lean also requires careful sign discipline: feasibility is encoded as `0 - G x ∈ P`, while positivity is evaluated on elements of `P`. Small convention errors would reverse the dual cone or the stationarity equation.
## Formalization scope
The source says that `X` is a vector space, but its definition of Gâteaux differentiation and its local perturbation argument require a norm and topology. The proposal therefore makes both `X` and `Z` normed real spaces and represents derivatives by continuous linear maps. It keeps Luenberger's cone assumptions: convexity and nonempty interior are explicit; pointedness and closedness are not added because the printed separation argument does not need them. The optimality hypothesis is faithfully local through `IsLocalMinOn`. Feasibility is included in `IsConeRegularAt`, and the no-descent milestone states it separately.
This is proposed as “Vector Space Methods XI” and depends on the earlier global Lagrange-duality mission, proposed as “Vector Space Methods IX,” for `coneLE` and `dualPositive`; the missions should be submitted in numerical order. The proposal does not cover equality constraints, second-order KKT conditions, multiplier uniqueness, constraint qualifications other than Luenberger's strict linearized feasibility condition, or sufficient conditions based on convexity. It also does not specialize to a finite list of scalar inequalities. These omissions preserve the exact role and scale of §9.4.
## Selected references
- David G. Luenberger, *Optimization by Vector Space Methods*, Wiley, 1969, Chapter 9, §9.4, regular-point definition and Theorem 1, pp. 248–250. Scan: https://sites.science.oregonstate.edu/~show/old/142_Luenberger.pdf
- Lean community, *Mathlib documentation*, continuously updated: https://leanprover-community.github.io/mathlib4_docs/ (convex cones, continuous linear functionals, topological interiors, differential calculus, local extrema, and geometric separation).
5 thms2 active usersReviewed
🏆Completed
Captain: wenxinzhang
Vector Space Methods X: Equality-Constrained Lagrange MultipliersTextbook
## Motivation
Equality-constrained optimization is the point where the geometric language of vector spaces becomes an operational calculus. In finite dimensions, the familiar rule says that the gradient of an objective at a regular constrained optimum is a linear combination of the constraint gradients. [Luenberger's Chapter 9](https://sites.science.oregonstate.edu/~show/old/142_Luenberger.pdf) replaces coordinate gradients by continuous linear maps between Banach spaces and identifies the genuinely important hypothesis: the derivative of the constraint map is onto. The resulting theorem covers constraints with infinitely many degrees of freedom and prepares the functional-analytic form of optimal control. This mission formalizes the local theorem rather than a finite-dimensional specialization. It also records the generalized inverse theorem that makes regular level sets locally rich enough to test every tangent direction.
## Setting
Let `X` and `Z` be **real Banach spaces**, `U ⊆ X` an open set, `f : X → ℝ` an objective, and `H : X → Z` an equality-constraint map. The distinguished point `x₀` lies in `U` and satisfies `H x₀ = 0`. Both maps are **continuously Fréchet differentiable** on `U`; their derivatives at `x₀` are named `f'` and `H'`. A **regular point** is one at which `H' : X →L[ℝ] Z` is surjective. Local optimality is expressed relative to the actual feasible set `{x | x ∈ U ∧ H x = 0}`, and may be either a local minimum or a local maximum. Multipliers live in the **continuous dual** `Z →L[ℝ] ℝ`, never in an untopologized algebraic dual.
The mission also treats a map `T : X → Y` between Banach spaces. Surjectivity of its derivative at `x₀` yields local metric surjectivity: sufficiently nearby target points possess preimages in `U`, with displacement controlled linearly by their distance from `T x₀`. This is the Lyusternik–Graves form of the generalized inverse theorem, not the ordinary inverse theorem requiring a bijective derivative.
## Formalization targets
The main target is `VectorSpaceOpt.equality_lagrange_multiplier`, the exact regular equality-multiplier theorem from §9.3. Its conclusion is the existence of a continuous linear functional `z₀` satisfying
$$
f' + z₀ \circ H' = 0.
$$
Three source-aligned milestones organize the mission. First, `generalized_inverse_function` formalizes §9.2, Theorem 1: an onto derivative gives constants `ε > 0` and `K ≥ 0` so every `y` with `dist y (T x₀) < ε` has a preimage `x ∈ U` obeying `T x = y` and `‖x - x₀‖ ≤ K ‖y - T x₀‖`. Second, `constrained_extremum_tangent_stationary` states that `f' h = 0` for every `h` in the kernel of `H'` at a regular local extremum. Third, `abnormal_lagrange_multiplier` records Luenberger's closed-range corollary: without surjectivity there is a nonzero pair `(r₀,z₀)` satisfying `r₀ • f' + z₀ ∘ H' = 0`.
## Significance
This theorem is the Banach-space bridge between unconstrained differentiation and multiplier theory. It isolates the quotient-space geometry behind the multiplier rule and supplies an interface reusable in variational problems, PDE-constrained optimization, and smooth optimal control. The abnormal alternative matters independently: it represents the degeneracy that later appears in Fritz John conditions and endpoint-constrained control. Formalizing the quantitative generalized inverse statement also contributes infrastructure with uses beyond optimization, including nonlinear solvability, metric regularity, and perturbation estimates.
## Difficulty
The mission is mathematically compact but technically demanding. The hard object is local surjectivity from an onto, noninjective derivative. Its natural linear model passes through the Banach quotient by the kernel and the open mapping theorem, while the nonlinear statement must preserve the open domain and a quantitative norm estimate. At the multiplier stage, a functional defined on the range of `H'` must be shown well-defined, bounded, and represented as a continuous functional on `Z`. Lean must also reconcile `ContDiffOn`, pointwise Fréchet derivatives, kernels and ranges of continuous linear maps, and filter-based local extrema. These are substantial analytic interfaces even though the final equation is short.
## Formalization scope
The proposal follows printed pp. 240–244. All domain, completeness, differentiability, feasibility, and locality hypotheses that are inherited implicitly in the prose are explicit in the Lean statements. The primary theorem assumes surjectivity and therefore produces a normalized multiplier with coefficient one on the objective. The abnormal milestone assumes only that `Set.range H'` is closed and explicitly requires the pair `(r₀,z₀)` to be nonzero. No finite-dimensionality, choice of coordinates, second-order condition, constraint qualification weaker than surjectivity, or sufficiency theorem is claimed.
Boundary cases are intentional. The zero constraint space is allowed and reduces the conclusion to ordinary stationarity. A local maximum is covered alongside a local minimum because the tangent argument is symmetric. The generalized inverse target explicitly returns a preimage inside `U`; it does not silently rely on extending `T` outside its domain. The mission does not identify the feasible level set with a manifold or claim uniqueness of a multiplier. Those are natural later developments but are not statements in the cited pages.
## Selected references
- David G. Luenberger, *Optimization by Vector Space Methods*, Wiley, 1969, Chapter 9, §9.2, Theorem 1, pp. 240–242; §9.3, Lemma 1, Theorem 1, and Corollary 1, pp. 242–244. Scan: https://sites.science.oregonstate.edu/~show/old/142_Luenberger.pdf
- Lean community, *Mathlib documentation*, continuously updated: https://leanprover-community.github.io/mathlib4_docs/ (Fréchet derivatives, local extrema, continuous linear maps, Banach quotients, and Lagrange multipliers).
4 thms2 active usersReviewed
🏆Completed
Captain: wenxinzhang
Vector Space Methods V: Convex Separation and Distance DualityTextbook
## Motivation
Linear approximation is only one instance of distance minimization. Feasible sets in optimization are typically convex rather than subspaces, so a useful certificate must compare a target point with an entire convex set and must allow an affine offset. Chapter 5 of Luenberger's *Optimization by Vector Space Methods* builds this certificate through geometric forms of the Hahn--Banach theorem, supporting hyperplanes, and separation of convex sets. The resulting minimum-distance theorem expresses the distance from a point to a convex set as an optimal gap measured by a norm-bounded continuous linear functional ([Luenberger, §§5.12--5.13, pp. 130--137](https://sites.science.oregonstate.edu/~show/old/142_Luenberger.pdf)).
This mission advances the series from subspace annihilators to affine separation. It formalizes the Minkowski gauge used by the chapter, three progressively stronger separation statements, and a capstone distance-duality certificate. These results are standard infrastructure for constrained optimization: they turn a geometric exclusion or distance into a scalar inequality that can later become a multiplier or a dual bound.
## Setting
Let $X$ be a **real normed space** and $K\subseteq X$ a nonempty convex set. Convexity is represented by `Convex ℝ K`, and topological interior, closure, and infimum distance use Mathlib's `interior`, `closure`, and `Metric.infDist`. A **continuous affine separator** is described by a continuous linear functional $f:X\toL[\mathbb R]\mathbb R$ and a scalar level $c$. The inequality $f(k)\le c$ for all $k\in K$ places $K$ in one closed half-space.
When a convex set contains zero in its interior, its **Minkowski gauge** is the functional `gauge K`. The source characterizes it by nonnegativity, positive homogeneity, subadditivity, continuity, and the level sets
$$
\{x:g_K(x)\le 1\}=\overline K,
\qquad
\{x:g_K(x)<1\}=\operatorname{int}K.
$$
These properties are bundled into the first milestone, following Lemma 1 of §5.12 ([pp. 131--132](https://sites.science.oregonstate.edu/~show/old/142_Luenberger.pdf)).
For two convex sets $K_1,K_2$, **Eidelheit separation** means finding nonzero $f$ and $c$ with $f(x)\le c\le f(y)$ for $x\in K_1$ and $y\in K_2$. The source assumes that $K_1$ has nonempty interior and that its interior does not meet $K_2$. The Lean statement records the nonemptiness of $K_2$ explicitly, since otherwise nonzero separation is not forced.
## Formalization targets
### Gauge and geometric Hahn--Banach milestones
Formalize the six gauge properties above. Then, for a convex $K$ with nonempty interior and an affine subspace $V$ disjoint from that interior, produce $f\ne0$ and $c$ such that
$$
f(v)=c\quad(v\in V),
\qquad
f(k)<c\quad(k\in\operatorname{int}K).
$$
This is Mazur's geometric Hahn--Banach theorem as stated in §5.12, Theorem 1 ([p. 133](https://sites.science.oregonstate.edu/~show/old/142_Luenberger.pdf)).
### Supporting hyperplanes and convex-set separation
For $x\notin\operatorname{int}K$, formalize a nonzero functional satisfying $f(k)\le f(x)$ for all $k\in K$. Next formalize Eidelheit separation:
$$
f(x)\le c\le f(y)
\quad\text{for all }x\in K_1,\ y\in K_2.
$$
These are Theorems 2 and 3 of §5.12 ([pp. 133--134](https://sites.science.oregonstate.edu/~show/old/142_Luenberger.pdf)).
### Convex minimum-distance duality
Let $x_1$ have positive distance $d$ from $K$. Produce $f$ and a real upper-bound level $c$ with $\|f\|\le1$, $f(k)\le c$ on $K$, and
$$
f(x_1)-c=d.
$$
Every other feasible pair $(g,b)$ must satisfy $g(x_1)-b\le d$. If $x_0\in K$ realizes the distance, require $-f$ to align with $x_0-x_1$. This is the finite real certificate form of §5.13, Theorem 1 ([pp. 136--137](https://sites.science.oregonstate.edu/~show/old/142_Luenberger.pdf)).
## Significance
The capstone is an exact strong-duality statement for distance to a convex set. A feasible pair $(g,b)$ yields a certified lower bound on the distance, and the distinguished pair reaches the primal value. Unlike a nearest-point characterization, it remains meaningful when $K$ is not closed and no minimizing point exists. The conditional alignment clause identifies the equality case when attainment is available.
Formalizing the chapter's progression creates more than one isolated equality. The gauge package links convex geometry to sublinear analysis; Mazur separation handles affine constraints; the supporting-hyperplane and Eidelheit statements provide reusable interfaces for later multiplier rules. The results are known and proved in the 1969 text; the mission's contribution is a coherent machine-checked Lean layer that preserves the source hypotheses and can support later chapters on duality and optimization.
## Difficulty
A direct reuse of subspace distance duality is insufficient because a general convex set is neither closed under subtraction nor described by an annihilator. An affine level $c$ is unavoidable. The common shorthand $\sup_{k\in K} f(k)$ introduces a second problem: $K$ need not be bounded, so a real-valued supremum is not available for an arbitrary functional. The capstone therefore quantifies over a real upper bound $c$ and asserts its optimality through a universal inequality; this records the same finite support value without imposing boundedness absent from the source.
Topological hypotheses also differ across the milestones. Separation uses nonempty interior, whereas the final distance theorem only assumes convexity, nonemptiness, and positive distance. Replacing positive distance by mere exclusion $x_1\notin K$ would be invalid for a nonclosed set. Similarly, requiring closure or compactness would make formalization easier but would lose the theorem's intended infinite-dimensional scope.
## Formalization scope
The mission is restricted to real normed spaces. Sets use `Set X`; affine varieties use `AffineSubspace ℝ X`; separators use `ContinuousLinearMap`. The gauge is Mathlib's existing `gauge`, so no competing definition is introduced. The bundled gauge milestone deliberately includes both level-set identities as well as continuity, positive homogeneity for positive real scalars, subadditivity, and nonnegativity.
The Eidelheit theorem includes `K₂.Nonempty`, an assumption used implicitly by the source's separating conclusion. The capstone includes `K.Nonempty` and `0 < Metric.infDist x₁ K`; it does not assume closedness, boundedness, compactness, or attainment. Its pair $(f,c)$ represents a finite support level, and the universal comparison over all feasible $(g,b)$ rules out a weakened statement in which an arbitrarily loose upper bound could trivialize existence. The optional nearest-point clause uses the exact equality $\|x_0-x_1\|=d$ and fixes the sign of alignment. Contributions may add reusable lemmas on gauges, interiors, affine subspaces, or support bounds, but the public results should remain independent of finite-dimensionality and completeness.
## Selected references
- David G. Luenberger, *Optimization by Vector Space Methods*, John Wiley & Sons, 1969, Chapter 5, §§5.11--5.13, pp. 127--137. [Public scan](https://sites.science.oregonstate.edu/~show/old/142_Luenberger.pdf).
5 thms2 active usersReviewed
🏆Completed
Captain: wenxinzhang
Vector Space Methods VII: Euler–Lagrange EquationsTextbook
## Motivation
The calculus of variations replaces optimization over finitely many coordinates by optimization over paths. Its necessary conditions underlie geodesics, minimum-energy curves, classical mechanics, and many optimal-control models. Chapter 7 of David G. Luenberger's [*Optimization by Vector Space Methods*](https://openlibrary.org/books/OL7612943M/Optimization_by_Vector_Space_Methods) presents this transition as an application of differentiation in normed vector spaces: a local extremum first forces every directional derivative to vanish, and the resulting integral identity forces a differential equation along the optimizing path. This mission formalizes the scalar, fixed-endpoint version in §§7.4–7.5. The target is intentionally the theorem actually isolated by the source, not a stronger modern Sobolev-space variant.
## Setting
Fix real numbers $a<b$. A **$C^1$ path on the segment** is represented in Lean by two functions, $x,\dot x:\mathbb R\to\mathbb R$. Both are continuous on $[a,b]$, and $x$ has derivative $\dot x(t)$ at every $t\in(a,b)$. Ordinary two-sided derivatives are not demanded at $a$ or $b$; this makes the formal endpoint convention match the one-sided role of endpoints in a closed interval.
Let $L(y,v,t)$ be a scalar Lagrangian. Along a candidate path, write
$$
L_x(t)=\frac{\partial L}{\partial y}(x(t),\dot x(t),t),\qquad
L_v(t)=\frac{\partial L}{\partial v}(x(t),\dot x(t),t).
$$
The Lean statement records these partial derivatives with `HasDerivAt` and assumes that $L_x$ and $L_v$ are continuous on $[a,b]$. A **fixed-endpoint variation** is another $C^1$ pair $(h,\dot h)$ with $h(a)=h(b)=0$. The first variation already computed from the action is
$$
\delta J(x;h)=\int_a^b\bigl(L_x(t)h(t)+L_v(t)\dot h(t)\bigr)\,dt.
$$
The main theorem begins from the stationarity identity $\delta J(x;h)=0$ for every such variation. It does not claim that the complete passage from a local extremum in Luenberger's $C^1$ norm to this integral formula has already been bundled into the root statement.
## Formalization targets
### Main goal: Euler–Lagrange equation
From the computed first-variation identity, prove that
$$
\frac{d}{dt}L_v(t)=L_x(t)\qquad(t\in(a,b)).
$$
The conclusion is expressed as `HasDerivAt Lv (Lx t) t`, so it asserts both differentiability of $L_v$ and the equality of its derivative with $L_x$. This is equation (2) and the conclusion reached on printed pages 180–181.
### Milestones
The first milestone formalizes §7.4, Theorem 1: a local minimum or maximum of a real functional has zero derivative along every direction whenever that scalar directional derivative exists. The remaining milestones are the three fixed-endpoint fundamental lemmas from §7.5. They respectively show that a continuous coefficient annihilating all variations is zero, that a continuous coefficient annihilating all variation derivatives is constant, and that an identity involving both $h$ and $\dot h$ forces the second coefficient to have derivative equal to the first. These are stated with the same $C^1$ variation class used by the goal.
## Significance
The result turns an infinite family of scalar integral equalities into a pointwise differential equation. Once available, the same interface can support standard variational examples by supplying a concrete $L$, its two partial derivatives, and a stationary path. It also provides the analytic core needed before treating natural boundary conditions, vector-valued paths, higher derivatives, or weak Euler–Lagrange equations.
The formalization adds reusable interval-sensitive infrastructure. In particular, `IsC1OnSegment` separates a path from its chosen continuous derivative and avoids silently imposing derivatives outside the optimization interval. The three fundamental lemmas are useful independently of the named Euler–Lagrange theorem: they are test-function principles for interval integrals and can serve later missions involving integration by parts or weak formulations. The mathematics is classical and proved in the cited text; the open work is a machine-checked Lean development of these exact statements in the pinned Mathlib environment.
## Difficulty
The source argument uses informal phrases such as “arbitrary $C^1$ function vanishing at the endpoints” and treats endpoint differentiation according to standard calculus convention. In Lean, those phrases must determine a precise domain, derivative witness, continuity requirement, and interval-integral orientation. Replacing $C^1$ variations by merely continuous functions would change Lemmas 2 and 3, while requiring `HasDerivAt` at the endpoints would add a hypothesis not present in the book.
Another tempting shortcut is to assume from the outset that $L_v$ is differentiable and then use integration by parts. That would trivialize the central regularity conclusion of Lemma 3: the book derives differentiability of $L_v$ from stationarity and continuity. The root therefore assumes only continuity of the two coefficient functions and concludes a `HasDerivAt` assertion on the open interval. Conversely, constructing the first variation from a local extremum of the action requires a separate differentiation-under-the-integral development and a topology on bundled $C^1$ paths; it is not hidden inside the main goal.
## Formalization scope
The scalar field, path values, time variable, and action values are all real. The interval is nondegenerate through the explicit hypothesis $a<b$. Integrals use Mathlib's oriented interval integral, but all principal statements are made in the forward orientation. Paths and variations are total functions on $\mathbb R$ whose relevant regularity is restricted to $[a,b]$. The Lagrangian is finite-valued. No measurability or integrability premise is omitted: continuity of the coefficient and variation factors on the compact interval supplies the intended finite integrals.
The goal starts from an already computed first-variation identity. Contributions connecting a genuine local extremum of the action in the norm $\max|x|+\max|\dot x|$ to that identity are welcome as a strengthening, but they must not be advertised as part of the present root theorem. Other welcome contributions include reusable continuous test-function constructions and endpoint-aware interval integration lemmas. Sobolev paths, vector-valued state spaces, free endpoints, and weak derivatives are outside this mission and should be proposed separately rather than obtained by weakening the stated hypotheses until the result becomes vacuous.
## Selected references
- David G. Luenberger, *Optimization by Vector Space Methods*, John Wiley & Sons, 1969, Chapter 7, §§7.4–7.5, pp. 178–181; definition of $D[a,b]$ on p. 23. [Open Library record](https://openlibrary.org/books/OL7612943M/Optimization_by_Vector_Space_Methods)
6 thms2 active usersReviewed
🏆Completed
Captain: wenxinzhang
Vector Space Methods VI: Pseudoinverse OperatorsTextbook
## Motivation
Linear equations between Hilbert spaces need not have unique solutions and may not even be exactly solvable for a given right-hand side. Least squares selects a vector with the smallest residual; when several such vectors exist, minimum norm selects one canonical representative. Luenberger packages this two-stage optimization into the **pseudoinverse** of a continuous linear operator with closed range. The construction unifies exact equations, approximation, normal equations, and orthogonal projections, while retaining a bounded linear operator suitable for subsequent optimization methods ([Luenberger, §§6.9--6.11, pp. 159--165](https://sites.science.oregonstate.edu/~show/old/142_Luenberger.pdf)).
This mission continues the series into Chapter 6. Its capstone formalizes the structural identities of the pseudoinverse, including involution, compatibility with adjoints, reflexive inverse laws, self-adjoint projection products, and factorizations through the normal operators. Earlier milestones establish the adjoint facts and minimum-norm characterizations on which that operator calculus depends.
## Setting
Let $G$ and $H$ be **real Hilbert spaces**, represented in Lean by complete real inner-product spaces, and let $A:G\toL[\mathbb R]H$ be a continuous linear map whose range is closed. The **Hilbert adjoint** is written $A^\dagger$ in the Lean statements and is Mathlib's adjoint continuous linear map. It is characterized by the inner-product relation and satisfies $\|A^\dagger\|=\|A\|$ ([Luenberger, §6.5, Theorem 1, p. 151](https://sites.science.oregonstate.edu/~show/old/142_Luenberger.pdf)). Closed range gives the range-kernel identity
$$
\operatorname{range}(A^\dagger)=\ker(A)^\perp,
$$
the Hilbert-space specialization of the closed range theorem used in the chapter ([§6.6, Theorem 2, p. 156](https://sites.science.oregonstate.edu/~show/old/142_Luenberger.pdf)).
For $y\in H$, a vector $x\in G$ is a **least-squares solution** when $\|Ax-y\|$ is no larger than $\|Az-y\|$ for every $z$. A least-squares solution is **minimum norm** when its norm is no larger than that of every other least-squares solution. A continuous linear map $B:H\toL[\mathbb R]G$ satisfies `VectorSpaceOpt.IsPseudoinverse A B` when, for every $y$, $By$ has both properties. This predicate is the mission's one lightweight definition, directly encoding the definition in §6.11 ([pp. 163--164](https://sites.science.oregonstate.edu/~show/old/142_Luenberger.pdf)).
## Formalization targets
### Adjoint and closed-range milestones
Formalize $\|A^\dagger\|=\|A\|$. Under closed range, formalize
$$
\operatorname{range}(A^\dagger)=\ker(A)^\perp.
$$
These record §6.5, Theorem 1 and the Hilbert form of §6.6, Theorem 2.
### Normal equations and minimum-norm solutions
Formalize the least-squares equivalence
$$
x\text{ minimizes }\|y-Ax\|
\quad\Longleftrightarrow\quad
A^\dagger A x=A^\dagger y,
$$
as in §6.9, Theorem 1 ([p. 160](https://sites.science.oregonstate.edu/~show/old/142_Luenberger.pdf)). For solvable $Ax=y$ and closed-range $A$, characterize the minimum-norm solution by $x=A^\dagger z$ with $AA^\dagger z=y$, following §6.10, Theorem 1 ([pp. 161--162](https://sites.science.oregonstate.edu/~show/old/142_Luenberger.pdf)). Finally, formalize existence and uniqueness of a continuous linear $B$ satisfying `IsPseudoinverse A B`.
### Pseudoinverse identities
Given such a $B$, formalize that $A$ is the pseudoinverse of $B$, that $B^\dagger$ is the pseudoinverse of $A^\dagger$, and that
$$
BAB=B,\qquad ABA=A,\qquad (BA)^\dagger=BA.
$$
Also produce pseudoinverses $C$ of $A^\dagger A$ and $D$ of $AA^\dagger$ satisfying
$$
B=CA^\dagger,
\qquad
B=A^\dagger D.
$$
Together with the continuous-linear-map type of $B$, these clauses encode all nine items of §6.11, Proposition 1 ([p. 165](https://sites.science.oregonstate.edu/~show/old/142_Luenberger.pdf)).
## Significance
The pseudoinverse turns a possibly inconsistent or underdetermined equation into a canonical bounded linear solution operator. The normal equations connect residual minimization with the self-adjoint operator $A^\dagger A$; the minimum-norm theorem selects the component orthogonal to the kernel. The capstone identities show that the construction behaves like an inverse on the effective ranges and that $BA$ is self-adjoint, while the two factorizations reduce pseudoinverse questions to the normal operators.
The underlying results are proved in Luenberger's text. Their Lean formalization supplies a reusable predicate for minimum-norm least squares and an operator-level API linking adjoints, kernels, ranges, composition, and optimization characterizations. This bridges the earlier missions on minimum norm and estimation with later chapters that use normal operators and generalized inverses. It also records explicitly which conclusions require closed range, preventing accidental use of a bounded pseudoinverse where only an unbounded generalized inverse could exist.
## Difficulty
Pointwise existence of a best residual is not enough. The selected minimum-norm solutions must collectively form a linear bounded map, and closed range is the hypothesis that makes this global operator well behaved. Without closed range, least-squares minimizers may fail to exist and the inverse on the effective range need not be bounded. A formulation that chooses an arbitrary minimizer for each target would therefore miss the main analytic content.
Several notationally similar operations must also remain distinct. The book writes a star for the adjoint and a superscript dagger-like symbol for the pseudoinverse; Mathlib's displayed dagger denotes the Hilbert adjoint. The mission consequently names the generalized inverse through `IsPseudoinverse` instead of overloading dagger notation. Orthogonal complements apply to submodules, compositions must retain their source and target spaces, and each factorization involves a different normal operator. These typing constraints expose domain/codomain mistakes that paper notation suppresses.
## Formalization scope
The mission uses real Hilbert spaces only: `NormedAddCommGroup`, `InnerProductSpace ℝ`, and `CompleteSpace`. Operators are `ContinuousLinearMap`, composition is `∘L`, the Hilbert adjoint is Mathlib's `†`, and the closed-range assumption is `IsClosed (A.range : Set H)`. The orthogonal complement in the range theorem is the submodule `A.kerᗮ`.
`IsPseudoinverse A B` requires two pointwise inequalities for every target: `B y` minimizes residual norm among all inputs, then minimizes norm among all residual minimizers. The second clause cannot be dropped or weakened to exact solutions, because it is what makes the choice canonical for inconsistent as well as underdetermined systems. The minimum-norm-solution milestone states `y ∈ A.range` explicitly; the source treats solvability as part of speaking about a solution. The capstone accepts a continuous linear $B$ satisfying the predicate, so linearity and boundedness are represented by its type, corresponding to the first two items of Proposition 1. Contributions may add reusable lemmas about adjoints, orthogonal complements, closed range, normal equations, or uniqueness of optimizers, but must preserve the closed-range and completeness assumptions in the public operator theorems.
## Selected references
- David G. Luenberger, *Optimization by Vector Space Methods*, John Wiley & Sons, 1969, Chapter 6, especially §§6.5--6.11, pp. 151--165. [Public scan](https://sites.science.oregonstate.edu/~show/old/142_Luenberger.pdf).
7 thms2 active usersReviewed
🏆Completed
Captain: wenxinzhang
Vector Space Methods IV: Minimum-Distance DualityTextbook
## Motivation
Best approximation asks how closely a point can be represented by a prescribed linear model. In a Hilbert space, orthogonality turns this into a geometric projection problem. A general normed space has no inner product and may have no nearest point, so the corresponding certificate must live in the continuous dual rather than in the original space. Chapter 5 of Luenberger's *Optimization by Vector Space Methods* develops exactly this passage from geometry to duality: the Hahn--Banach theorem supplies continuous linear functionals that detect norms, separate points from closed subspaces, and certify an infimum distance even when that distance is not attained ([Luenberger, §§5.4--5.8, pp. 111--120](https://sites.science.oregonstate.edu/~show/old/142_Luenberger.pdf)).
This mission continues the book's vector-space formalization series at the point where minimum-norm arguments cease to be specifically Hilbertian. Its capstone identifies the distance from a point to a linear subspace with the largest value at that point among all norm-at-most-one continuous linear functionals annihilating the subspace. The statement is a prototype for dual certificates throughout approximation theory and convex optimization.
## Setting
Let $X$ be a **real normed space** and let $M$ be a linear subspace. In Lean, $M$ is represented by `Submodule ℝ X`; no topological closure assumption is imposed on the capstone. A **continuous linear functional** is an element $f : X \toL[\mathbb R] \mathbb R$, with operator norm $\|f\|$. It **annihilates** $M$ when $f(m)=0$ for every $m\in M$. The set of all such functionals is the annihilator $M^\perp$ in the book's terminology.
For $x\in X$, the **infimum distance** to $M$ is
$$
d(x,M)=\inf_{m\in M}\|x-m\|.
$$
The Lean target uses `Metric.infDist x (M : Set X)`. Since every submodule contains zero, the underlying set is nonempty and this extended geometric notion is an ordinary nonnegative real number here. A functional $f$ is **aligned** with a vector $v$ when $f(v)=\|f\|\,\|v\|$. Alignment is the normed-space replacement for the familiar inner-product equality associated with a projection direction.
Two auxiliary dual notions are also formalized. A norm-preserving Hahn--Banach extension takes a functional on a subspace and extends it to all of $X$ without changing its norm. A **norming functional** for $x$ is a nonzero functional aligned with $x$. Finally, for closed $M$, the preannihilator of its annihilator is exactly $M$: the functionals vanishing on $M$ distinguish every point outside it ([Luenberger, §§5.4 and 5.7, pp. 112--118](https://sites.science.oregonstate.edu/~show/old/142_Luenberger.pdf)).
## Formalization targets
### Norm-preserving extension and norming functionals
For a continuous functional $f$ on $M$, formalize an extension $F$ satisfying
$$
F|_M=f,\qquad \|F\|=\|f\|.
$$
For nontrivial $X$ and every $x\in X$, formalize the existence of a nonzero $f$ with $f(x)=\|f\|\,\|x\|$. These are Corollaries 1 and 2 of §5.4 ([pp. 112--113](https://sites.science.oregonstate.edu/~show/old/142_Luenberger.pdf)).
### Closed-subspace double annihilator
For closed $M$, formalize
$$
\{x\in X: \forall f,\ f|_M=0 \Rightarrow f(x)=0\}=M.
$$
This is the concrete set-valued form of Theorem 1 in §5.7 ([p. 118](https://sites.science.oregonstate.edu/~show/old/142_Luenberger.pdf)).
### Minimum-distance duality
For arbitrary $M$ and $x$, produce one functional $f$ with $\|f\|\le 1$, $f|_M=0$, and
$$
f(x)=d(x,M),\qquad
g(x)\le d(x,M)
$$
for every other $g$ of norm at most one annihilating $M$. Thus $f$ realizes the dual maximum. If a best approximant $m_0\in M$ exists, the same certificate also satisfies
$$
f(x-m_0)=\|f\|\,\|x-m_0\|.
$$
This packages both parts of the minimum-distance theorem in §5.8 ([Theorem 1, pp. 119--120](https://sites.science.oregonstate.edu/~show/old/142_Luenberger.pdf)).
## Significance
The capstone gives an exact lower-bound certificate for an infinite-dimensional approximation problem. Every feasible dual functional supplies the inequality $g(x)\le d(x,M)$, while the distinguished functional reaches equality. Consequently, the primal infimum is identified without assuming reflexivity, strict convexity, finite dimension, closedness of $M$, or existence of a nearest point. When a nearest point does exist, alignment records the equality case of the norm estimate and links the dual certificate back to the geometry of the residual.
The source result is classical and proved in the book; the open work here is its machine-checked Lean formalization in the same namespace as the earlier vector-space missions. The reusable output includes norm-controlled extension infrastructure, norming functionals, a concrete double-annihilator theorem, and a certificate form of distance duality suitable for later convex-separation and constrained-optimization missions.
## Difficulty
The obvious Hilbert-space formulation fails because a normed space has no canonical orthogonal complement and a minimizing element of $M$ need not exist. Replacing the minimum by `Metric.infDist` avoids an unjustified attainment assumption, but the desired dual maximizer must still be an actual continuous functional, not merely a limiting family. Norm control is essential: an algebraic separator without continuity cannot serve as a bounded dual certificate.
There are also degenerate cases that informal notation can hide. The distance may be zero even when $x\notin M$ if $M$ is not closed, and then the zero functional is the correct capstone witness. Conversely, the book's assertion that a norming functional is nonzero requires a nontrivial ambient space. The formal statements must handle these cases without silently strengthening the main theorem to closed subspaces or positive distance.
## Formalization scope
All spaces and functionals are real, matching the chapter and avoiding extra complex-scalar conjugation conventions. The ambient object uses Mathlib's `NormedAddCommGroup`, `NormedSpace`, `Submodule`, and `ContinuousLinearMap`; completeness is not assumed because the cited Hahn--Banach consequences do not require it. The distance is exactly `Metric.infDist`, and annihilation is written pointwise rather than by introducing a new annihilator definition. This keeps the capstone self-contained while the double-annihilator milestone states the same construction explicitly as a set.
No claim is made that a best approximant exists. The alignment clause is conditional on an element already satisfying the global minimum property. No closedness assumption may be added to the capstone, since the zero-distance/nonclosed case is part of the source theorem's generality. The norming-functional milestone alone assumes `[Nontrivial X]`; this prevents a vacuous encoding of “nonzero functional” on the zero space. Contributions may establish the four stated theorems and any generally useful lemmas about restrictions, quotient norms, annihilation, or `Metric.infDist`, provided the public statements retain these conventions.
## Selected references
- David G. Luenberger, *Optimization by Vector Space Methods*, John Wiley & Sons, 1969, Chapter 5, especially §§5.4, 5.7, and 5.8, pp. 111--120. [Public scan](https://sites.science.oregonstate.edu/~show/old/142_Luenberger.pdf).
4 thms2 active usersReviewed
🏆Completed
Captain: Shuze Chen
Introduction to Linear Optimization VI: Farkas' Lemma and Separating HyperplanesTextbook
When is a system of linear constraints infeasible? Sections 4.6-4.7 of Bertsimas-Tsitsiklis answer with the archetypal theorem of the alternative. The capstone is Farkas' lemma (Theorem 4.6): for an $m \times n$ matrix $A$ and $b \in \mathbb{R}^m$, exactly one of the following holds — (a) some $x \ge 0$ satisfies $Ax = b$, or (b) some $p$ satisfies $p'A \ge 0'$ and $p'b < 0$; such a $p$ is a certificate of infeasibility, geometrically a hyperplane separating $b$ from the cone of the columns of $A$. The mission also carries the cone-membership restatement (Corollary 4.3), the inequality form (Theorem 4.7: every solution of $Ax \le b$ satisfies $c'x \le d$ iff some $p \ge 0$ has $p'A = c'$ and $p'b \le d$), and the application to asset pricing (Theorem 4.8: a market's prices admit no arbitrage iff there is a nonnegative state-price vector $q$ with $p_i = \sum_s q_s r_{si}$). The book proves Farkas' lemma from LP strong duality; Section 4.7 then reverses the arrow from first principles: every polyhedron is closed (Theorem 4.9), Weierstrass' theorem (Theorem 4.10, already in Mathlib), and the separating hyperplane theorem (Theorem 4.11: for nonempty closed convex $S$ and $x^* \notin S$ there exists $c$ with $c'x^* < c'x$ for all $x \in S$), from which Farkas' lemma — and hence the duality theorem itself — follows geometrically.