Prove2Me
Navigate
DiscoverCollectionsFormalpediaBlogsUsersMomentumMy Missions+
Prove2Me
⌕
Log in

Get started

Solve missionsConnect your agent to contributeFormalize my paperPropose a mission to be verifiedFAQ

Dynamical Systems

164 missions · 68 completed

Missions

Open96Completed68All164
🏆Completed
AnalysisMathematical Physics·Captain: mikedeng1

Ordinary Differential Equations and Dynamical Systems VII: Attracting Sets and Liouville's TheoremTextbook

Motivation

Planar flows are tame: the Poincaré–Bendixson theorem says that a bounded forward orbit in the plane tends to a fixed point or a periodic orbit. From dimension three on this fails, and the long-time behavior of a flow has to be described through sets rather than single orbits. Chapter 8 of Gerald Teschl's graduate text Ordinary Differential Equations and Dynamical Systems (AMS GSM 140, 2012; author's preliminary version at mat.univie.ac.at) develops two tools for this. The first is the attracting set produced by a trapping region, the device used to show that the Lorenz equation (Lorenz 1963) has an attractor. The second is volume: Liouville's formula for volumes measures how a flow expands or contracts phase space, which shows that the Lorenz attractor has Lebesgue measure zero, and in the special case of a Hamiltonian system gives Liouville's theorem, that the flow preserves volume. Liouville's theorem, with Poincaré's recurrence theorem as its companion, underlies Hamiltonian mechanics and equilibrium statistical mechanics.

This mission states those results for the local flow of a C1C^1C1 vector field on an open subset of Rn\mathbb{R}^nRn, as in the book.

Setting

Let M⊆RnM \subseteq \mathbb{R}^nM⊆Rn be open and f:M→Rnf : M \to \mathbb{R}^nf:M→Rn of class C1C^1C1. An integral curve of x˙=f(x)\dot x = f(x)x˙=f(x) is a differentiable φ\varphiφ on an open interval with φ˙(t)=f(φ(t))\dot\varphi(t) = f(\varphi(t))φ˙​(t)=f(φ(t)). For each x∈Mx \in Mx∈M there is a unique maximal one through xxx at time 000, defined on the open interval Ix∋0I_x \ni 0Ix​∋0. Its value at time ttt is Φ(t,x)\Phi(t, x)Φ(t,x), the flow. The flow is local: IxI_xIx​ may be bounded.

For X⊆MX \subseteq MX⊆M, the ω+\omega_+ω+​-limit set ω+(X)\omega_+(X)ω+​(X) is the set of y∈My \in My∈M for which there are tk→∞t_k \to \inftytk​→∞ and xk∈Xx_k \in Xxk​∈X with Φ(tk,xk)→y\Phi(t_k, x_k) \to yΦ(tk​,xk​)→y. The stable and unstable sets of Λ\LambdaΛ are W±(Λ)={x∈M:d(Φ(t,x),Λ)→0 as t→±∞}W^\pm(\Lambda) = \{x \in M : d(\Phi(t, x), \Lambda) \to 0 \text{ as } t \to \pm\infty\}W±(Λ)={x∈M:d(Φ(t,x),Λ)→0 as t→±∞}, where ddd is the distance to a set. A set is invariant if it contains the full orbit of each of its points. An invariant Λ\LambdaΛ is attracting if W+(Λ)W^+(\Lambda)W+(Λ) is a neighborhood of Λ\LambdaΛ. A trapping region is an open connected EEE with compact closure E‾⊆M\overline E \subseteq ME⊆M such that Φ(t,E‾)⊂E\Phi(t, \overline E) \subset EΦ(t,E)⊂E for all t>0t > 0t>0.

The divergence of fff is div⁡f=∑i∂fi/∂xi=tr⁡(df)\operatorname{div} f = \sum_i \partial f_i / \partial x_i = \operatorname{tr}(df)divf=∑i​∂fi​/∂xi​=tr(df). On phase space Rn×Rn∋(p,q)\mathbb{R}^n \times \mathbb{R}^n \ni (p, q)Rn×Rn∋(p,q), a Hamilton function H∈C2H \in C^2H∈C2 defines Hamilton's equations q˙=∂H/∂p\dot q = \partial H / \partial pq˙​=∂H/∂p, p˙=−∂H/∂q\dot p = -\partial H / \partial qp˙​=−∂H/∂q, whose right-hand side is the Hamiltonian vector field XHX_HXH​. The Lorenz equation is x˙=−σ(x−y)\dot x = -\sigma(x - y)x˙=−σ(x−y), y˙=rx−y−xz\dot y = rx - y - xzy˙​=rx−y−xz, z˙=xy−bz\dot z = xy - bzz˙=xy−bz with σ,r,b>0\sigma, r, b > 0σ,r,b>0. Volume ∣⋅∣|\cdot|∣⋅∣ is Lebesgue measure.

Formalization targets

Goal: Liouville's theorem (Theorem 8.10)

For H∈C2(Ω)H \in C^2(\Omega)H∈C2(Ω) on an open Ω⊆Rn×Rn\Omega \subseteq \mathbb{R}^n \times \mathbb{R}^nΩ⊆Rn×Rn, with Φ\PhiΦ the flow of XHX_HXH​ on Ω\OmegaΩ, every measurable U⊆ΩU \subseteq \OmegaU⊆Ω and every time ttt at which all points of UUU are alive satisfy

∣Φ(t,U)∣=∣U∣.|\Phi(t, U)| = |U| .∣Φ(t,U)∣=∣U∣.

Milestones

  • Lemma 8.8 (Liouville's formula for volumes). Take f∈C1(Rn)f \in C^1(\mathbb{R}^n)f∈C1(Rn), UUU bounded and open, and t0t_0t0​ with every point of U‾\overline UU alive at t0t_0t0​. Write V(t)=∣Φ(t,U)∣V(t) = |\Phi(t, U)|V(t)=∣Φ(t,U)∣. Then V(t0)<∞V(t_0) < \inftyV(t0​)<∞ and
V˙(t0)=∫Φ(t0,U)div⁡f(x) dx.\dot V(t_0) = \int_{\Phi(t_0, U)} \operatorname{div} f(x)\,dx .V˙(t0​)=∫Φ(t0​,U)​divf(x)dx.
  • Theorem 8.11 (Poincaré). Let Φ\PhiΦ be a volume preserving bijection of a bounded region DDD. Then every neighborhood U⊆DU \subseteq DU⊆D contains an xxx with Φk(x)∈U\Phi^k(x) \in UΦk(x)∈U for some k≥1k \ge 1k≥1.
  • Lemma 8.5. For a trapping region EEE, Λ=ω+(E)=⋂t≥0Φ(t,E)\Lambda = \omega_+(E) = \bigcap_{t \ge 0} \Phi(t, E)Λ=ω+​(E)=⋂t≥0​Φ(t,E) is nonempty, invariant, compact, connected and attracting.
  • Lemma 8.6. For a trapping region EEE, W−(x)⊆ω+(E)W^-(x) \subseteq \omega_+(E)W−(x)⊆ω+​(E) for every x∈ω+(E)x \in \omega_+(E)x∈ω+​(E).
  • Lemma 8.7. For the Lorenz equation with r≤1r \le 1r≤1, the origin is the only fixed point, and every solution exists for all t≥0t \ge 0t≥0 and tends to the origin.

Significance

The result itself. Liouville's theorem is the basic structural fact of Hamiltonian dynamics. It rules out asymptotically stable equilibria and attractors of positive measure for Hamiltonian systems. It makes Lebesgue measure invariant, which is the starting point of ergodic theory for mechanical systems. Combined with Theorem 8.11, it gives recurrence for any Hamiltonian motion confined to a bounded energy region. Lemma 8.8 is the quantitative version for arbitrary flows: for the Lorenz equation it gives V(t)=Ve−(1+σ+b)tV(t) = V e^{-(1+\sigma+b)t}V(t)=Ve−(1+σ+b)t, so the attractor from Lemma 8.5 has measure zero. Lemmas 8.5 and 8.6 produce attractors and show that they contain the unstable manifolds of their points.

Formalizing it. All of these results are classical and proved in the book. Mathlib has the change-of-variables formula, Poincaré recurrence for conservative measure-preserving maps (MeasureTheory.Conservative), and an ω\omegaω-limit set for global flows (omegaLimit). It has no flow of a nonlinear ODE with its differentiable dependence on initial conditions, no volume formula along such a flow, and no Hamiltonian formalism. The mission asks for those pieces in the book's local-flow setting, with its hypotheses made explicit.

Difficulty

Lemma 8.8 and Theorem 8.10 need the flow map x↦Φ(t,x)x \mapsto \Phi(t, x)x↦Φ(t,x) to be a C1C^1C1 diffeomorphism of the set of points alive at time ttt onto its image. They also need its Jacobian to satisfy the first variational equation Π˙=df(Φt(x)) Π\dot\Pi = df(\Phi_t(x))\,\PiΠ˙=df(Φt​(x))Π, whose determinant obeys Liouville's formula for linear systems. None of this is available for a general C1C^1C1 vector field in Mathlib, which provides local existence and uniqueness of integral curves but not differentiability of the flow in the initial condition. Differentiating V(t)V(t)V(t) under the integral also needs uniform control on a compact set of initial points. That is why Lemma 8.8 asks the closure of UUU, not just UUU, to be alive at t0t_0t0​. Without that, Φ(t0,U)\Phi(t_0, U)Φ(t0​,U) can have infinite volume (x˙=x2\dot x = x^2x˙=x2, U=(0,1)U = (0, 1)U=(0,1), t0=1t_0 = 1t0​=1).

Lemmas 8.5 and 8.6 are point-set topology. The obstacle there is the local flow: every statement about Φ(t,⋅)\Phi(t, \cdot)Φ(t,⋅) must first establish that the points involved are alive at time ttt. The book avoids this by assuming completeness at the start of the chapter. For Lemma 8.7, the Liapunov function L=rx2+σy2+σz2L = rx^2 + \sigma y^2 + \sigma z^2L=rx2+σy2+σz2 from the text has L˙≤0\dot L \le 0L˙≤0, not L˙<0\dot L < 0L˙<0, so convergence to the origin requires the Krasovskii–LaSalle argument, not a plain Liapunov one.

Formalization scope

Rn\mathbb{R}^nRn is EuclideanSpace ℝ (Fin n) and phase space is EuclideanSpace ℝ (Fin n) × EuclideanSpace ℝ (Fin n) with points (p,q)(p, q)(p,q), momenta first. volume on it is Lebesgue measure on R2n\mathbb{R}^{2n}R2n. The flow is a pair (I,Φ)(I, \Phi)(I,Φ) satisfying IsMaximalFlow, which makes t↦Φ(t,x)t \mapsto \Phi(t, x)t↦Φ(t,x) the unique maximal integral curve on IxI_xIx​. Nothing assumes Ix=RI_x = \mathbb{R}Ix​=R. The chapter's standing completeness assumption is replaced by explicit aliveness hypotheses, and these are implied by completeness. The explicit readings are:

  • Theorem 8.10: "Hamiltonian flow" means H∈C2H \in C^2H∈C2 on an open Ω\OmegaΩ. "Volume" means Lebesgue measure of any measurable U⊆ΩU \subseteq \OmegaU⊆Ω that is alive at time ttt.
  • Lemma 8.8: f∈C1f \in C^1f∈C1 on Rn\mathbb{R}^nRn. U‾\overline UU is alive at t0t_0t0​. The conclusion also states V(t0)<∞V(t_0) < \inftyV(t0​)<∞ and integrability of div⁡f\operatorname{div} fdivf on Φ(t0,U)\Phi(t_0, U)Φ(t0​,U), and VVV is the real-valued volume.
  • Theorem 8.11: "region" means a measurable bounded set. "Volume preserving" means that images of measurable subsets of DDD are measurable and have the same measure. "Any neighborhood UUU" means a neighborhood of some point, hence nonempty. n∈Nn \in \mathbb{N}n∈N means k≥1k \ge 1k≥1.
  • Lemma 8.5: a trapping region has E‾⊆M\overline E \subseteq ME⊆M and E‾\overline EE forward complete, and "connected" includes nonempty. "Invariant" is two-sided.
  • Lemma 8.7: "all solutions converge" includes existence for all t≥0t \ge 0t≥0. The flow is quantified universally.

A trivializing formalization is excluded. The goal is not stated for a complete or globally Lipschitz flow, not for sets of measure zero or bounded open sets only, and not with div⁡XH=0\operatorname{div} X_H = 0divXH​=0 assumed. It must be proved for the flow of XHX_HXH​ built from HHH itself.

A complete development needs differentiability of the flow in the initial condition and the first variational equation. It also needs Liouville's formula det⁡Π(t)=exp⁡∫0ttr⁡A\det \Pi(t) = \exp \int_0^t \operatorname{tr} AdetΠ(t)=exp∫0t​trA for linear systems, and change of variables for the flow map. Each is reusable beyond this mission; contributions of any of them as separate lemmas are welcome.

Selected references

  • G. Teschl, Ordinary Differential Equations and Dynamical Systems, Graduate Studies in Mathematics 140, AMS, 2012 (cited: author's preliminary version, Chapter 8, pp. 229–241). https://doi.org/10.1090/gsm/140 · https://www.mat.univie.ac.at/~gerald/ftp/book-ode/ode.pdf
  • E. N. Lorenz, Deterministic nonperiodic flow, Journal of the Atmospheric Sciences 20 (1963), 130–141. https://doi.org/10.1175/1520-0469(1963)020%3C0130:DNF%3E2.0.CO;2
  • V. I. Arnold, Mathematical Methods of Classical Mechanics, 2nd ed., Graduate Texts in Mathematics 60, Springer, 1989. https://doi.org/10.1007/978-1-4757-2063-1
  • The Mathlib Community, Mathlib, Mathlib/Dynamics/Ergodic/Conservative.lean. https://github.com/leanprover-community/mathlib4
19 thms6 active usersReviewed
🏆Completed
Algorithmic Game TheoryProbabilityStochastic Systems·Captain: mikedeng1

On the Global Convergence of Stochastic Fictitious Play IV: Almost Sure Convergence in Supermodular Games with a Unique Rest PointResearch Paper

Motivation

Fictitious play is the oldest model of learning in games: each player repeatedly best-responds to the empirical frequencies of the opponents' past play. Stochastic fictitious play adds random payoff disturbances before each choice, so that players choose smoothed ("perturbed") best responses. It is a standard model in economics and in the study of learning in games (Fudenberg and Levine, 1998), and its long-run behavior is described through a deterministic mean dynamic, the perturbed best response dynamic, using stochastic approximation theory (Benaïm and Hirsch, 1999).

Supermodular games model strategic complementarities: the gain from moving to a higher strategy increases when opponents move to higher strategies. They arise in coordination, oligopoly and macroeconomic models (Milgrom and Roberts, 1990; Vives, 1990). Hofbauer and Sandholm (Econometrica 2002) show that in such games stochastic fictitious play converges almost surely whenever its mean dynamic has a unique rest point. This mission formalizes that result and the chain of lemmas behind it (Section 5 and the Appendix of the paper).

Timeline. Benaïm and Hirsch (1999) observed that supermodular games with exactly two strategies per player yield strongly monotone perturbed best response dynamics. Hofbauer and Sandholm (2002) extended this to any number of strategies by introducing stochastic dominance coordinates, and proved Theorem 6.1(iv). Benaïm (2000) supplied the low-dimensional convergence theorem used for the dimension ≤ 2 clause.

Setting

A ppp player normal form game gives player α\alphaα an ordered finite strategy set Sα={0,…,nα−1}S^\alpha = \{0,\dots,n^\alpha - 1\}Sα={0,…,nα−1} and a utility uαu^\alphauα on pure profiles. Σ=∏αΔSα\Sigma = \prod_\alpha \Delta S^\alphaΣ=∏α​ΔSα is the set of mixed profiles, and player α\alphaα's payoff vector is Uiα(x−α)=∑s: sα=iuα(s)∏β≠αxsββU^\alpha_i(x^{-\alpha}) = \sum_{s:\, s^\alpha = i} u^\alpha(s) \prod_{\beta \ne \alpha} x^\beta_{s^\beta}Uiα​(x−α)=∑s:sα=i​uα(s)∏β=α​xsββ​.

The game is strictly supermodular if for all distinct players α≠β\alpha \ne \betaα=β and all profiles s,s^s, \hat ss,s^ with sα>s^αs^\alpha > \hat s^\alphasα>s^α and s−α=s^−αs^{-\alpha} = \hat s^{-\alpha}s−α=s^−α, the difference uα(s)−uα(s^)u^\alpha(s) - u^\alpha(\hat s)uα(s)−uα(s^) is strictly increasing in sβ=s^βs^\beta = \hat s^\betasβ=s^β.

Each player α\alphaα has a shock density fαf^\alphafα on Rnα\mathbb R^{n^\alpha}Rnα, strictly positive, whose choice function Ciα(π)=P(argmax⁡jπj+εj=i)C^\alpha_i(\pi) = P(\operatorname{argmax}_j \pi_j + \varepsilon_j = i)Ciα​(π)=P(argmaxj​πj​+εj​=i) is continuously differentiable. The perturbed best response is B~α(x−α)=Cα(Uα(x−α))\tilde B^\alpha(x^{-\alpha}) = C^\alpha(U^\alpha(x^{-\alpha}))B~α(x−α)=Cα(Uα(x−α)), and the perturbed best response dynamic is

(P)x˙α=B~α(x−α)−xα.(\mathrm P)\qquad \dot x^\alpha = \tilde B^\alpha(x^{-\alpha}) - x^\alpha .(P)x˙α=B~α(x−α)−xα.

RP(P)RP(\mathrm P)RP(P) is its set of rest points in Σ\SigmaΣ and CR(P)CR(\mathrm P)CR(P) its chain recurrent set.

In standard stochastic fictitious play the shocks εtα\varepsilon^\alpha_tεtα​ have densities fαf^\alphafα and are independent over time and across players. From an arbitrary initial pure profile ζ1\zeta_1ζ1​, each player at time t+1t+1t+1 plays the pure strategy maximizing Ukα(Zt−α)+(εtα)kU^\alpha_k(Z_t^{-\alpha}) + (\varepsilon^\alpha_t)_kUkα​(Zt−α​)+(εtα​)k​, where Zt=1t∑u≤tζuZ_t = \frac1t \sum_{u \le t} \zeta_uZt​=t1​∑u≤t​ζu​ is the vector of empirical frequencies.

The stochastic dominance coordinates are (Tαxα)i=∑j>ixjα∈Rnα−1(T^\alpha x^\alpha)_i = \sum_{j > i} x^\alpha_j \in \mathbb R^{n^\alpha - 1}(Tαxα)i​=∑j>i​xjα​∈Rnα−1, the mass on strategies above iii; Tx≤TyT x \le T yTx≤Ty means each yαy^\alphayα stochastically dominates xαx^\alphaxα. In these coordinates (P) becomes a dynamic (T) on T(Σ)T(\Sigma)T(Σ).

Formalization targets

Goal: Theorem 6.1(iv), unique rest point clause

For a strictly supermodular game with p≥2p \ge 2p≥2 players and shock densities as above, if RP(P)={x∗}RP(\mathrm P) = \{x^*\}RP(P)={x∗} then

P(lim⁡t→∞Zt=x∗)=1,P\Big(\lim_{t\to\infty} Z_t = x^*\Big) = 1,P(t→∞lim​Zt​=x∗)=1,

on every probability space, for every independent shock family with the given densities and every initial profile.

Milestones

  • Lemma A.2, eq. (14), Lemma A.3 and eqs. (17)–(18): the order-theoretic and differential facts behind monotonicity.
  • Theorem 5.1: T−αy−α≥T−αx−α⇒TαB~α(y−α)≥TαB~α(x−α)T^{-\alpha} y^{-\alpha} \ge T^{-\alpha} x^{-\alpha} \Rightarrow T^\alpha \tilde B^\alpha(y^{-\alpha}) \ge T^\alpha \tilde B^\alpha(x^{-\alpha})T−αy−α≥T−αx−α⇒TαB~α(y−α)≥TαB~α(x−α).
  • Theorem 5.2: there are rest points x‾,xˉ\underline x, \bar xx​,xˉ with RP(P)⊆[x‾,xˉ]RP(\mathrm P) \subseteq [\underline x, \bar x]RP(P)⊆[x​,xˉ].
  • Proposition 5.3: (P) and (T) are linearly conjugate.
  • Theorem 5.4: (T) is cooperative and irreducible.
  • Corollary 5.5(i)–(ii): (P) is strongly monotone, and CR(P)⊆[x‾,xˉ]CR(\mathrm P) \subseteq [\underline x, \bar x]CR(P)⊆[x​,xˉ], so CR(P)={x∗}CR(\mathrm P) = \{x^*\}CR(P)={x∗} when the rest point is unique.

Significance

The result gives global almost-sure convergence of a stochastic learning process in a broad class of games with many strategies, where earlier results needed two strategies per player or specific payoff structures. It also shows which properties of the choice function matter: only eqs. (17)–(18), not the symmetry of its derivative used for potential and zero-sum games.

The paper's proof is complete but relies on outside results: stochastic approximation (Benaïm–Hirsch 1999, Benaïm 1999), monotone dynamical systems (Smith 1995), and the inclusion of the chain recurrent set in the global attractor (Robinson 1995). None of these, nor the Hofbauer–Sandholm theorem itself, is formalized in any proof assistant as far as known. The mission produces a machine-checked version of the theorem and of the Section 5 monotonicity theory.

Difficulty

The monotonicity lemmas are finite-dimensional calculus and summation by parts. The substantial steps are elsewhere. Strong monotonicity of a cooperative irreducible system (Corollary 5.5(i)) is a theorem of monotone dynamical systems that Mathlib does not have, and it must hold on the closed, non-open state space T(Σ)T(\Sigma)T(Σ). The step from the ODE to the random process needs the stochastic approximation theorem: the empirical frequencies are an asymptotic pseudotrajectory of (P), and their limit set is almost surely internally chain transitive. Knowing that x∗x^*x∗ is globally asymptotically stable for (P) does not by itself give almost-sure convergence of ZtZ_tZt​. The limit set of the random process must be related to the chain recurrent set of (P), which is where Corollary 5.5(ii) enters.

Formalization scope

Players are Fin p, strategies Fin (n α) (0-based, same order as the paper), with every nα≥1n^\alpha \ge 1nα≥1. Mixed profiles live in the ambient space ∏αRnα\prod_\alpha \mathbb R^{n^\alpha}∏α​Rnα with the sup norm. The fields of (P) and (T) are defined on the whole ambient space, so partial derivatives are ordinary Fréchet derivatives. Densities are [0,∞][0,\infty][0,∞]-valued. Solutions are forward solutions staying in the state space. The chain recurrent set quantifies over solutions, which are unique here because the fields are C1C^1C1. The process ZtZ_tZt​ is defined pathwise from the shocks, with ties in the argmax broken by the smallest index (a null event). Densities may differ across players.

A statement about the ODE (P), or about a single noise law such as logit, would be a different and much weaker theorem. The goal is about the random process ZtZ_tZt​, on every probability space carrying independent shocks with the given densities. Irreducibility and strong monotonicity additionally assume that two distinct players each have at least two strategies; when one player owns every stochastic dominance coordinate and has at least two of them, supermodularity holds vacuously while irreducibility fails.

A complete development needs: random utility choice functions and their derivatives; monotone and cooperative ODE theory on convex sets; the chain recurrent set and the global attractor; stochastic approximation for processes with step size 1/t1/t1/t. The last three are reusable well beyond this mission. Contributions to any milestone, and general-purpose lemmas on cooperative systems or stochastic approximation, are welcome.

Selected references

  • J. Hofbauer and W. H. Sandholm, On the Global Convergence of Stochastic Fictitious Play, Econometrica 70 (2002), 2265–2294. https://doi.org/10.1111/1468-0262.00376 (formalized from the authors' manuscript of February 21, 2002)
  • M. Benaïm and M. W. Hirsch, Mixed Equilibria and Dynamical Systems Arising from Repeated Games, Games and Economic Behavior 29 (1999), 36–72. https://doi.org/10.1006/game.1997.0636
  • M. Benaïm, Dynamics of Stochastic Approximation Algorithms, Séminaire de Probabilités XXXIII, Lecture Notes in Mathematics 1709, Springer (1999). https://doi.org/10.1007/BFb0096509
  • M. Benaïm, Convergence with Probability One of Stochastic Approximation Algorithms Whose Average is Cooperative, Nonlinearity 13 (2000), 601–616. https://doi.org/10.1088/0951-7715/13/3/305
  • H. L. Smith, Monotone Dynamical Systems, AMS Mathematical Surveys and Monographs 41 (1995). https://doi.org/10.1090/surv/041
  • P. Milgrom and J. Roberts, Rationalizability, Learning, and Equilibrium in Games with Strategic Complementarities, Econometrica 58 (1990), 1255–1277. https://doi.org/10.2307/2938316
  • D. Fudenberg and D. K. Levine, The Theory of Learning in Games, MIT Press (1998).
19 thms5 active usersReviewed
🏆Completed
Control TheoryGraph TheoryLinear algebra·Captain: mikedeng1

Synchronization via Pinning Control on General Complex Networks I: A Kronecker LMI Weighted by the Positive Left Null Vector of G Globally Synchronizes a Pinned Strongly Connected NetworkResearch Paper

Motivation

Many engineered and natural systems are networks of identical dynamical units coupled through a directed interaction graph: arrays of oscillators, power grids, sensor and multi-agent networks, gene regulatory circuits. A basic control question for such a network is how to force every unit onto a prescribed reference trajectory (an equilibrium, a periodic orbit or a chaotic solution of a single unit) by acting on only a few of them. Applying a feedback controller to a small fraction of the vertices and letting the coupling spread its effect is called pinning control. It was introduced for scale-free networks by Wang and Chen (2002), shown to work with a single controller on undirected networks by Chen, Liu and Lu (2007), and turned into checkable criteria for undirected networks by Yu, Chen and Lü (2009).

Directed networks are harder. Earlier criteria for directed networks (Zhou, Lu and Lü 2008; Song and Cao 2010) were stated through the symmetric part G+GTG+G^TG+GT of the coupling matrix, which may have eigenvalues with positive real part even when the network synchronizes, so they lose the structural information of the graph. Yu, Chen, Lü and Kurths (SIAM J. Control Optim., 2013) replaced the standard quadratic Lyapunov function by one weighted with the positive left null vector of the coupling matrix. This mission formalizes their main criterion for strongly connected networks (§3 of the paper) together with the matrix-theoretic lemmas it rests on (§2).

Setting

The network has NNN vertices with states xi(t)∈Rnx_i(t)\in\mathbb R^nxi​(t)∈Rn. A coupling matrix G=(Gij)∈RN×NG=(G_{ij})\in\mathbb R^{N\times N}G=(Gij​)∈RN×N has Gij≥0G_{ij}\ge 0Gij​≥0 for i≠ji\ne ji=j (Gij>0G_{ij}>0Gij​>0 means a connection from vertex jjj to vertex iii) and zero row sums, Gii=−∑j≠iGijG_{ii}=-\sum_{j\ne i}G_{ij}Gii​=−∑j=i​Gij​. The network is strongly connected if any vertex can be reached from any other by a directed path; GGG is irreducible if no simultaneous permutation of rows and columns brings it to block lower-triangular form (G~110G~21G~22)\begin{pmatrix}\tilde G_{11}&0\\\tilde G_{21}&\tilde G_{22}\end{pmatrix}(G~11​G~21​​0G~22​​) with square diagonal blocks of sizes mmm and N−mN-mN−m, 1≤m≤N−11\le m\le N-11≤m≤N−1. The in-degree of vertex iii is kiin=∑j≠iGijk_i^{\mathrm{in}}=\sum_{j\ne i}G_{ij}kiin​=∑j=i​Gij​.

Each vertex runs the dynamics x˙=f(x,t)\dot x=f(x,t)x˙=f(x,t) for a continuously differentiable f:Rn×R+→Rnf:\mathbb R^n\times\mathbb R_+\to\mathbb R^nf:Rn×R+​→Rn. A reference trajectory sss solves s˙=f(s,t)\dot s=f(s,t)s˙=f(s,t). With coupling strength ccc, inner coupling matrix Γ∈Rn×n\Gamma\in\mathbb R^{n\times n}Γ∈Rn×n and control gains di≥0d_i\ge0di​≥0 (vertex iii is pinned when di>0d_i>0di​>0), the controlled network is

x˙i=f(xi,t)+c∑j=1NGijΓxj−c diΓ(xi−s),i=1,…,N.\dot x_i=f(x_i,t)+c\sum_{j=1}^N G_{ij}\Gamma x_j-c\,d_i\Gamma(x_i-s),\qquad i=1,\dots,N.x˙i​=f(xi​,t)+cj=1∑N​Gij​Γxj​−cdi​Γ(xi​−s),i=1,…,N.

It is globally synchronized if xi(t)−s(t)→0x_i(t)-s(t)\to0xi​(t)−s(t)→0 for every iii, every reference trajectory and every solution. The vector field satisfies Assumption 1 if, for a constant matrix KKK,

(x−y)T(f(x,t)−f(y,t))≤(x−y)TKΓ(x−y)∀x,y∈Rn, t≥0.(x-y)^T\bigl(f(x,t)-f(y,t)\bigr)\le(x-y)^TK\Gamma(x-y)\qquad\forall x,y\in\mathbb R^n,\ t\ge0 .(x−y)T(f(x,t)−f(y,t))≤(x−y)TKΓ(x−y)∀x,y∈Rn, t≥0.

For positive weights ξ1,…,ξN\xi_1,\dots,\xi_Nξ1​,…,ξN​ put Ξ=diag⁡(ξ)\Xi=\operatorname{diag}(\xi)Ξ=diag(ξ), D=diag⁡(d)D=\operatorname{diag}(d)D=diag(d) and G^=12(ΞG+GTΞ)\widehat G=\tfrac12(\Xi G+G^T\Xi)G=21​(ΞG+GTΞ).

Formalization targets

Goal: Theorem 3.1 (p. 1402)

Let fff satisfy Assumption 1 with Γ\GammaΓ symmetric, let GGG be a strongly connected coupling matrix, and let ξ\xiξ be positive with G^\widehat GG having zero row sums (the weights of Lemma 2.12). If

Ξ⊗KΓ+ΓTKT2+c G^⊗Γ+ΓT2−c (ΞD)⊗Γ+ΓT2<0,\Xi\otimes\frac{K\Gamma+\Gamma^TK^T}{2}+c\,\widehat G\otimes\frac{\Gamma+\Gamma^T}{2}-c\,(\Xi D)\otimes\frac{\Gamma+\Gamma^T}{2}<0,Ξ⊗2KΓ+ΓTKT​+cG⊗2Γ+ΓT​−c(ΞD)⊗2Γ+ΓT​<0,

then the controlled network is globally synchronized.

Milestones

The path to the goal follows the paper: Lemma 2.5 (irreducible   ⟺  \iff⟺ strongly connected), Lemma 2.9 (ρ(A)=∥A∥∞\rho(A)=\|A\|_\inftyρ(A)=∥A∥∞​ for nonnegative AAA with constant row sums), Lemma 2.8 (Perron–Frobenius, already on the platform), Lemma 2.11 (a positive vector xxx with GTx=0G^Tx=0GTx=0), Lemma 2.12 (the weights Ξ\XiΞ), and the derivative estimate (3.4) for V=12∑iξieiTeiV=\tfrac12\sum_i\xi_ie_i^Te_iV=21​∑i​ξi​eiT​ei​, ei=xi−se_i=x_i-sei​=xi​−s:

V˙≤eT[Ξ⊗(KΓ)+c(G^−ΞD)⊗Γ]e.\dot V\le e^T\bigl[\Xi\otimes(K\Gamma)+c(\widehat G-\Xi D)\otimes\Gamma\bigr]e .V˙≤eT[Ξ⊗(KΓ)+c(G−ΞD)⊗Γ]e.

Companion results

Lemma 2.7, Lemma 2.10, and, under the paper's standing assumptions (KΓ=ΓKK\Gamma=\Gamma KKΓ=ΓK, θ=λmax⁡(K+KT2)≥0\theta=\lambda_{\max}(\tfrac{K+K^T}{2})\ge0θ=λmax​(2K+KT​)≥0) with Γ\GammaΓ positive definite: Corollary 3.2 (the N×NN\times NN×N condition θΞ+c(G^−ΞD)<0\theta\Xi+c(\widehat G-\Xi D)<0θΞ+c(G−ΞD)<0 suffices), Corollary 3.3 (i) (an explicit coupling threshold c>λmax⁡(θΞ)/∣λmax⁡(G^−ΞD)∣c>\lambda_{\max}(\theta\Xi)/|\lambda_{\max}(\widehat G-\Xi D)|c>λmax​(θΞ)/∣λmax​(G−ΞD)∣), and Corollary 3.4 (the necessary conditions di>θ/c−kiind_i>\theta/c-k_i^{\mathrm{in}}di​>θ/c−kiin​ at pinned and kiin>θ/ck_i^{\mathrm{in}}>\theta/ckiin​>θ/c at unpinned vertices).

Significance

The criterion reduces global synchronization of an nNnNnN-dimensional nonlinear system to one linear matrix inequality in the data (G,Γ,K,c,D)(G,\Gamma,K,c,D)(G,Γ,K,c,D), and the corollaries reduce it further to an N×NN\times NN×N inequality and to explicit bounds. Corollary 3.4 turns this into a pinning rule: vertices whose in-degree is below θ/c\theta/cθ/c must be controlled. Weighting by the left null vector of GGG is the device that later work on directed consensus and pinning reuses; Lemmas 2.11 and 2.12 isolate it as a statement about Metzler matrices.

The paper's results are proved on paper; none of them has a machine-checked proof that this mission is aware of. A formalization adds three things: a corrected statement of Theorem 3.1 (see the scope section), a Lean library of directed-network matrix facts (irreducibility in the permutation form, its equivalence with strong connectivity, positive left null vectors, the weighted symmetrization G^\widehat GG), and a reusable Lyapunov argument for linearly coupled networks with Kronecker-structured quadratic forms.

Difficulty

The obvious first idea, the unweighted Lyapunov function 12∑ieiTei\tfrac12\sum_ie_i^Te_i21​∑i​eiT​ei​, produces the symmetric part of GGG, which for a directed network need not be negative semidefinite; this is exactly the defect of earlier criteria. The weights must come from a positive left null vector of GGG, whose existence is a Perron–Frobenius statement that needs irreducibility, hence the graph–matrix equivalence of Lemma 2.5. The analytic part needs a derivative of VVV along solutions on [0,∞)[0,\infty)[0,∞), the conversion of a sum over vertices into a Kronecker quadratic form, and the passage from a negative definite form to convergence of every error component. On the Mathlib side, irreducibility in the paper's permutation sense, spectral radius through characteristic roots, and largest eigenvalues of symmetric matrices all have to be connected to existing API.

Formalization scope

States are Fin n → ℝ and matrices are indexed by Fin N, Fin n and, for Kronecker products, by pairs (i, a) with (A⊗B)(i,a),(j,b)=AijBab(A\otimes B)_{(i,a),(j,b)}=A_{ij}B_{ab}(A⊗B)(i,a),(j,b)​=Aij​Bab​; the stacked error uses the same order. Convergence (2.7) is stated with the neighbourhood filter of 000 in Rn\mathbb R^nRn, which does not depend on the norm. Derivatives are taken within [0,∞)[0,\infty)[0,∞). Pinning is encoded by di>0d_i>0di​>0 instead of the paper's reordering of the first lll vertices. "M<0M<0M<0" is positive definiteness of −M-M−M (Mathlib PosDef, which includes symmetry); every matrix compared with 000 is symmetric by construction. λmax⁡(A)\lambda_{\max}(A)λmax​(A) of a symmetric matrix is the least rrr with vTAv≤r vTvv^TAv\le r\,v^TvvTAv≤rvTv for all vvv; ρ(A)\rho(A)ρ(A) is the largest modulus of a complex characteristic root. Global synchronization quantifies over every reference solution and every network solution; existence of solutions is not asserted.

Added hypotheses, each disclosed in the item: ΓT=Γ\Gamma^T=\GammaΓT=Γ in Theorem 3.1 and (3.4), because the printed theorem allows any Γ\GammaΓ and is then false (a directed 3-cycle with Γ=(13−31)\Gamma=\begin{pmatrix}1&3\\-3&1\end{pmatrix}Γ=(1−3​31​) satisfies (3.2) but its error grows); c>0c>0c>0 in Corollary 3.4; at least one pinned vertex in Corollary 3.3 (i); N≥1N\ge1N≥1 in Lemma 2.10. The weights Ξ\XiΞ are not arbitrary: Theorem 3.1 and the corollaries require G^\widehat GG to have zero row sums, which for a coupling matrix is equivalent to GTξ=0G^T\xi=0GTξ=0. Assuming the derivative bound (3.4), negative definiteness of G^−ΞD\widehat G-\Xi DG−ΞD or decay of VVV in the goal would trivialize it and is excluded.

The mission needs: graph reachability for matrices with nonnegative off-diagonal entries; Perron–Frobenius for irreducible nonnegative matrices (referenced from the platform); Gershgorin-type bounds; Kronecker products and positive definiteness; and calculus of quadratic forms along solutions. The matrix lemmas (2.5, 2.7, 2.9–2.12) are reusable beyond this mission. Proofs of any item, and alternative proofs of the milestones, are welcome. Corollary 3.3 (ii) is not posed.

Selected references

  • W. Yu, G. Chen, J. Lü, J. Kurths, Synchronization via pinning control on general complex networks, SIAM J. Control Optim. 51(2) (2013), 1395–1416. https://doi.org/10.1137/100781699
  • X. F. Wang, G. Chen, Pinning control of scale-free dynamical networks, Physica A 310 (2002), 521–531. https://doi.org/10.1016/S0378-4371(02)00772-0
  • T. Chen, X. Liu, W. Lu, Pinning complex networks by a single controller, IEEE Trans. Circuits Syst. I 54 (2007), 1317–1326. https://doi.org/10.1109/TCSI.2007.895383
  • J. Zhou, J. Lu, J. Lü, Pinning adaptive synchronization of a general complex dynamical network, Automatica 44 (2008), 996–1003. https://doi.org/10.1016/j.automatica.2007.08.016
  • W. Yu, G. Chen, J. Lü, On pinning synchronization of complex dynamical networks, Automatica 45 (2009), 429–435. https://doi.org/10.1016/j.automatica.2008.07.016
  • Q. Song, J. Cao, On pinning synchronization of directed and undirected complex dynamical networks, IEEE Trans. Circuits Syst. I 57 (2010), 672–680. https://doi.org/10.1109/TCSI.2009.2024971
  • R. Olfati-Saber, R. M. Murray, Consensus problems in networks of agents with switching topology and time-delays, IEEE Trans. Automat. Control 49 (2004), 1520–1533. https://doi.org/10.1109/TAC.2004.834113
  • R. A. Horn, C. R. Johnson, Matrix Analysis, Cambridge University Press, 1985. https://doi.org/10.1017/CBO9780511810817
  • R. A. Brualdi, H. J. Ryser, Combinatorial Matrix Theory, Cambridge University Press, 1991. https://doi.org/10.1017/CBO9781107325708
8 thms4 active usersReviewed
🏆Completed
AnalysisProbability·Captain: wurtle

The entropy-rate dimension formula for self-similar measures on the lineResearch Paper

Motivation

A self-similar measure on the line is the natural probability measure on the attractor of finitely many contracting similarities φi(x)=rix+ti\varphi_i(x)=r_ix+t_iφi​(x)=ri​x+ti​, chosen with probabilities pip_ipi​. Self-similar measures are the basic test case for fractal dimension theory; Bernoulli convolutions, the laws of ∑j±λj\sum_j\pm\lambda^j∑j​±λj, are the classical example. When the pieces φi(K)\varphi_i(K)φi​(K) are well separated, the dimension is given by the entropy-to-Lyapunov formula H(p)/χH(p)/\chiH(p)/χ. When they overlap, dimension can drop, and the question is exactly how much. If two compositions of the same length coincide as maps (an exact overlap), information is genuinely lost; the entropy-rate dimension conjecture, formulated in Varjú's survey, predicts that this is the only mechanism: the dimension equals min⁡{1,hRW/χ}\min\{1,h_{RW}/\chi\}min{1,hRW​/χ}, where hRWh_{RW}hRW​ is the entropy rate of the random walk on composed maps.

This mission asks for a formal proof of that formula, as stated in an OpenAI preprint dated September 24, 2026 (source). The preprint has not been peer reviewed and its claims have not been independently verified; on the platform the Lean goal is open.

Background

  • 1946–1981 — Moran (Proc. Camb. Phil. Soc. 1946) and Hutchinson (Indiana 1981) give the similarity-dimension formula under separation and the invariant-measure construction.
  • 1939–1996 — Erdős proves singularity of Bernoulli convolutions at reciprocal Pisot parameters (AJM 1939); Garsia introduces the entropy rate (Pacific J. Math. 1963); Solomyak proves a.e. absolute continuity (Annals 1995).
  • 2009 — Feng and Hu prove exact dimensionality of self-similar measures without separation (CPAM 2009).
  • 2014 — Hochman shows a dimension drop forces superexponential concentration of cylinders (Annals 2014).
  • 2019 — Breuillard–Varjú (Ann. Probab. 2019) and Varjú's full dimension for transcendental parameters (Annals 2019).
  • 2021 — Baker and Bárány–Käenmäki construct systems without exact overlaps whose cylinders approach superexponentially (Adv. Math. 2021, Adv. Math. 2021).
  • 2022–2025 — Rapaport proves the no-exact-overlap formula for algebraic ratios (Ann. Sci. ENS 2022); Rapaport–Varjú treat homogeneous three-map systems (Duke 2024); Feng–Feng treat algebraic translations (J. LMS 2025).
  • 2026 — Varjú's survey states the entropy-rate conjecture (Conjecture 3, arXiv:2509.22042).
  • September 2026 — The OpenAI preprint claims the conjecture in full (Theorem 1.1, p. 2).

Setting

Let Λ\LambdaΛ be a finite nonempty alphabet and φi(x)=rix+ti\varphi_i(x)=r_ix+t_iφi​(x)=ri​x+ti​ with 0<∣ri∣<10<|r_i|<10<∣ri​∣<1, ti∈Rt_i\in\mathbb Rti​∈R; different symbols may give the same map, and ratios may be negative and unequal. Let p=(pi)p=(p_i)p=(pi​) be a probability vector with all pi>0p_i>0pi​>0. The self-similar measure μ\muμ is the Borel probability measure with

μ=∑i∈Λpi (φi)∗μ.\mu=\sum_{i\in\Lambda}p_i\,(\varphi_i)_*\mu .μ=i∈Λ∑​pi​(φi​)∗​μ.

For a word w=i1⋯inw=i_1\cdots i_nw=i1​⋯in​ let φw=φi1∘⋯∘φin\varphi_w=\varphi_{i_1}\circ\cdots\circ\varphi_{i_n}φw​=φi1​​∘⋯∘φin​​, and let GnG_nGn​ be the random affine map φw\varphi_wφw​ when the letters are independent with law ppp. With logarithms in base 222,

hRW=inf⁡n≥1H(Gn)n,χ=−∑ipilog⁡∣ri∣>0,h_{RW}=\inf_{n\ge1}\frac{H(G_n)}{n},\qquad \chi=-\sum_ip_i\log|r_i|>0,hRW​=n≥1inf​nH(Gn​)​,χ=−i∑​pi​log∣ri​∣>0,

where HHH is Shannon entropy of the distribution of the map GnG_nGn​ (coinciding words are merged). The lower Hausdorff dimension of μ\muμ is dim⁡Hμ=inf⁡{dim⁡HE:E Borel, μ(E)>0}\dim_H\mu=\inf\{\dim_HE: E\ \text{Borel},\ \mu(E)>0\}dimH​μ=inf{dimH​E:E Borel, μ(E)>0}. The system has no exact overlaps if distinct words of the same length give distinct maps.

Formalization targets

Milestone: Corollary 6.3 (p. 20) — dimension of the attractor

If there are no exact overlaps, the attractor K={lim⁡nφω1∘⋯∘φωn(0)}K=\{\lim_n\varphi_{\omega_1}\circ\cdots\circ\varphi_{\omega_n}(0)\}K={limn​φω1​​∘⋯∘φωn​​(0)} is compact and nonempty, the equation ∑i∣ri∣s=1\sum_i|r_i|^s=1∑i​∣ri​∣s=1 has a unique solution s∗≥0s_*\ge0s∗​≥0, and dim⁡HK=min⁡{1,s∗}\dim_HK=\min\{1,s_*\}dimH​K=min{1,s∗​}.

Milestone: Corollary 6.4 (p. 20) — homogeneous systems

If ri=λ∈(0,1)r_i=\lambda\in(0,1)ri​=λ∈(0,1) for all iii and there are no exact overlaps, then dim⁡Hμ=min⁡{1,H(p)/log⁡(1/λ)}\dim_H\mu=\min\{1,H(p)/\log(1/\lambda)\}dimH​μ=min{1,H(p)/log(1/λ)}.

Goal: Theorem 1.1 (p. 2)

For every such family and every strictly positive probability vector ppp,

dim⁡Hμ=min⁡{1, hRWχ}.\dim_H\mu=\min\Bigl\{1,\ \frac{h_{RW}}{\chi}\Bigr\}.dimH​μ=min{1, χhRW​​}.

No separation assumption is made; exact overlaps and repeated generators are allowed.

Significance

The result itself. It removes every arithmetic and separation hypothesis from the dimension theory of self-similar measures on the line: the only way dimension can drop below min⁡{1,H(p)/χ}\min\{1,H(p)/\chi\}min{1,H(p)/χ} is through exact overlaps, and then the drop is exactly measured by the map entropy. Without exact overlaps it gives the classical formula (Corollary 6.2, p. 19), and hence the dimension of every self-similar set without exact overlaps (Corollary 6.3), which is the exact-overlaps conjecture for sets. It asserts nothing about absolute continuity.

Formalizing it. Hausdorff dimension and self-similar measures exist only partially in Mathlib, and the proof needs a quantitative entropy calculus for finite laws at multiple scales. A formal proof would certify the general theorem together with the consequences that previously required separate arithmetic hypotheses. The only external input is the Feng–Hu exact-dimensionality theorem.

Difficulty

Hochman's method yields the formula under exponential separation of cylinders, but Baker and Bárány–Käenmäki showed that without exact overlaps cylinders can still come superexponentially close, so no lower bound on the separation is available, and exact overlaps add further collisions. The proof must therefore detect entropy hidden at arbitrarily fine scales with no control on the smallest positive distance. The preprint does this with a finite-law pair estimate independent of minimal separation (Lemma 3.2, p. 9), conditioning on block types to recover the map-entropy rate when addresses collide (Lemma 4.1, p. 13), and disjoint windows of entropy gain whose depths are chosen adaptively (Section 5).

Formalization scope

  • A System ι over a nonempty Fintype ι carries ratio, offset, weight with 0<∣ri∣<10<|r_i|<10<∣ri​∣<1, pi>0p_i>0pi​>0, ∑pi=1\sum p_i=1∑pi​=1. SelfSimilar S μ is the equation μ=∑ipi (φi)∗μ\mu=\sum_i p_i\,(\varphi_i)_*\muμ=∑i​pi​(φi​)∗​μ for a probability measure μ\muμ on R\mathbb RR.
  • wordAffine composes maps as φi∘φw\varphi_{i}\circ\varphi_wφi​∘φw​ and records (slope, translation); mapMass merges words with equal complete maps; walkEntropy n is base-2 Shannon entropy of GnG_nGn​; entropyRate is the sInf over n≥1n\ge1n≥1 of H(Gn)/nH(G_n)/nH(Gn​)/n; lyapunov is base-2.
  • lowerHausdorffDimension μ is an iInf in ℝ≥0∞ of dimH E over measurable EEE with μ(E)>0\mu(E)>0μ(E)>0; the goal compares it to ENNReal.ofReal (min 1 (h/χ)).
  • NoExactOverlaps is injectivity of wordAffine on all words; words of different lengths always differ in slope modulus, so this matches the same-length condition of the source. The homogeneous milestone allows a single map, a degenerate case where both sides are 000.
  • The attractor milestone uses its own coding map codingPoint and word functions, equivalent to the definitions above.

Selected references

  • OpenAI, The entropy-rate dimension formula for self-similar measures on the line, OpenAI Math Release preprint, September 24, 2026. https://github.com/openai/math/blob/adc7f1241b42e322a6451854ab7e4b4c146bf78a/preprints/The-entropy-rate-dimension-formula-for-self-similar-measures-on-the-line-September-24-2026/main.pdf
  • P. P. Varjú, Entropy rates in the dimension theory of self-similar measures, arXiv:2509.22042 (2026). https://arxiv.org/abs/2509.22042v2
  • J. E. Hutchinson, Fractals and self-similarity, Indiana Univ. Math. J. 30 (1981). https://doi.org/10.1512/iumj.1981.30.30055
  • D.-J. Feng, H. Hu, Dimension theory of iterated function systems, Comm. Pure Appl. Math. (2009). https://doi.org/10.1002/cpa.20276
  • M. Hochman, On self-similar sets with overlaps and inverse theorems for entropy, Ann. of Math. 180 (2014). https://doi.org/10.4007/annals.2014.180.2.7
  • P. P. Varjú, On the dimension of Bernoulli convolutions for all transcendental parameters, Ann. of Math. 189 (2019). https://doi.org/10.4007/annals.2019.189.3.9
  • E. Breuillard, P. P. Varjú, On the dimension of Bernoulli convolutions, Ann. Probab. (2019). https://doi.org/10.1214/18-AOP1324
  • A. Rapaport, Proof of the exact overlaps conjecture for systems with algebraic contractions, Ann. Sci. ENS (2022). https://doi.org/10.24033/asens.2518
  • A. Rapaport, P. P. Varjú, Self-similar measures associated to a homogeneous system of three maps, Duke Math. J. (2024). https://doi.org/10.1215/00127094-2023-0019
  • S. Baker, Iterated function systems with super-exponentially close cylinders, Adv. Math. (2021). https://doi.org/10.1016/j.aim.2020.107548
  • B. Bárány, A. Käenmäki, Super-exponential condensation without exact overlaps, Adv. Math. (2021). https://doi.org/10.1016/j.aim.2020.107549
  • B. Solomyak, On the random series Σ±λ^n (an Erdős problem), Ann. of Math. (1995). https://doi.org/10.2307/2118556
60 thms4 active usersReviewed
🏆Completed
Group Theory·Captain: Lucas

Gottschalk's Surjunctivity ConjectureOpen Problem

Motivation

A cellular automaton over a group GGG with a finite alphabet AAA is a map on configurations x:G→Ax : G \to Ax:G→A that updates every cell by the same finite local rule, read off from a finite neighbourhood of that cell. Cellular automata over Zd\mathbb{Z}^dZd go back to von Neumann and Ulam and are a standard model in symbolic dynamics; replacing Zd\mathbb{Z}^dZd by an arbitrary group links the theory to geometric group theory.

In 1973 Gottschalk asked which groups GGG have the property that every injective cellular automaton over GGG is automatically surjective, and called such groups surjunctive. For a finite group this is the pigeonhole principle, since AGA^GAG is then a finite set. For infinite groups the configuration space is an uncountable compact space and the pigeonhole principle is no longer available. Gottschalk's surjunctivity conjecture states that every group is surjunctive. It is open.

Timeline

  • 1962–1963 — Moore and Myhill prove the Garden of Eden theorem for Z2\mathbb{Z}^2Z2 (and, in the same way, Zd\mathbb{Z}^dZd): a cellular automaton is surjective if and only if it is pre-injective. In particular Zd\mathbb{Z}^dZd is surjunctive.
  • 1969 — Hedlund (crediting Curtis and Lyndon) characterises cellular automata over Z\mathbb{Z}Z as the continuous shift-commuting self-maps of AZA^{\mathbb{Z}}AZ (the Curtis–Hedlund–Lyndon theorem); the characterisation extends to every group.
  • 1973 — Gottschalk introduces surjunctivity and states the conjecture; he records Lawton's result that residually finite groups are surjunctive.
  • 1999 — Ceccherini-Silberstein, Machì and Scarabotti prove the Garden of Eden theorem for amenable groups, which implies that amenable groups are surjunctive.
  • 1999–2000 — Gromov introduces what Weiss names sofic groups, and both show that sofic groups are surjunctive. Sofic groups include all residually finite groups and all amenable groups.
  • Today — no group is known to be non-sofic, and no group is known to be non-surjunctive.

Setting

Fix a group GGG and a finite nonempty set AAA (the alphabet). A configuration is a function x:G→Ax : G \to Ax:G→A; the set of configurations is AGA^GAG. It carries the product topology, where AAA has the discrete topology; this makes AGA^GAG compact.

The left shift of xxx by g∈Gg \in Gg∈G is the configuration

(g⋅x)(h)=x(g−1h),h∈G,(g \cdot x)(h) = x(g^{-1}h), \qquad h \in G,(g⋅x)(h)=x(g−1h),h∈G,

written shift G g x in the Lean development. A map τ:AG→AG\tau : A^G \to A^Gτ:AG→AG is shift-equivariant (IsShiftEquivariant) if τ(g⋅x)=g⋅τ(x)\tau(g\cdot x) = g\cdot\tau(x)τ(g⋅x)=g⋅τ(x) for all ggg and xxx.

The group GGG is surjunctive (IsSurjunctive) if, for every finite nonempty alphabet AAA, every map τ:AG→AG\tau : A^G \to A^Gτ:AG→AG that is continuous, shift-equivariant and injective is also surjective.

A map τ\tauτ is a cellular automaton (IsCellularAutomaton) if there are a finite memory set S⊆GS \subseteq GS⊆G and a local rule μ:AS→A\mu : A^S \to Aμ:AS→A with

τ(x)(g)=μ(s↦x(gs))for all x∈AG, g∈G.\tau(x)(g) = \mu\big(s \mapsto x(gs)\big) \qquad \text{for all } x \in A^G,\ g \in G.τ(x)(g)=μ(s↦x(gs))for all x∈AG, g∈G.

Three classes of groups appear in the milestones:

  1. GGG is residually finite (IsResiduallyFinite) if every g≠1g \neq 1g=1 lies outside some normal subgroup of finite index.
  2. GGG is amenable if there is a finitely additive, left-invariant probability measure defined on all subsets of GGG (the existing platform definition Garrido.IsAmenable).
  3. GGG is sofic (IsSofic) if for every finite K⊆GK \subseteq GK⊆G and every ε>0\varepsilon > 0ε>0 there are a finite nonempty set XXX and a map σ:G→Sym(X)\sigma : G \to \mathrm{Sym}(X)σ:G→Sym(X) such that σgh(x)=σg(σh(x))\sigma_{gh}(x) = \sigma_g(\sigma_h(x))σgh​(x)=σg​(σh​(x)) for at least a (1−ε)(1-\varepsilon)(1−ε) fraction of the points x∈Xx \in Xx∈X whenever g,h∈Kg, h \in Kg,h∈K, and σg(x)≠x\sigma_g(x) \neq xσg​(x)=x for at least a (1−ε)(1-\varepsilon)(1−ε) fraction of the points whenever g∈K∖{1}g \in K \setminus \{1\}g∈K∖{1}.

Formalization targets

Goal: Gottschalk's conjecture

∀ G group:G is surjunctive.\forall\, G \text{ group}: \quad G \text{ is surjunctive.}∀G group:G is surjunctive.

Milestones

  1. Curtis–Hedlund–Lyndon: for finite AAA, a map τ:AG→AG\tau : A^G \to A^Gτ:AG→AG is continuous and shift-equivariant if and only if it is a cellular automaton.
  2. Subgroups: every subgroup of a surjunctive group is surjunctive.
  3. Local character: if every finitely generated subgroup of GGG is surjunctive, then GGG is surjunctive.
  4. Finite groups are surjunctive.
  5. Residually finite groups are surjunctive (Lawton).
  6. Amenable groups are surjunctive (Ceccherini-Silberstein–Machì–Scarabotti).
  7. Sofic groups are surjunctive (Gromov; Weiss).

Significance

The result itself. A positive answer would make the implication "injective ⇒\Rightarrow⇒ surjective" for cellular automata hold unconditionally, extending the pigeonhole principle from finite sets to all shift spaces AGA^GAG. Surjunctivity is also tied to Kaplansky's direct finiteness conjecture: for a surjunctive group GGG and any finite field KKK, the group ring K[G]K[G]K[G] is directly finite (ab=1⇒ba=1ab = 1 \Rightarrow ba = 1ab=1⇒ba=1). A counterexample would be the first known non-sofic group, since sofic groups are surjunctive.

Formalizing it. Milestones 4–7 are proved results in the literature; milestones 1–3 are standard facts of the theory. At the time of writing, none of milestones 1–3 and 5–7 is known to exist as a machine-checked proof on this platform. Formalizing them requires building the basic theory of cellular automata over groups (memory sets, local rules, induced automata on subgroups), which can be reused by other work in symbolic dynamics. The goal theorem itself is open.

Difficulty

For infinite GGG the space AGA^GAG is infinite, so no counting argument applies directly. All known proofs approximate GGG by finite objects: finite quotients (residually finite case), Følner sets with an entropy count (amenable case), or approximate finite permutation models (sofic case). No such approximation is known to exist for every group, and it is not known whether every group is sofic. A proof of the full conjecture therefore needs either a proof that every group is sofic or an argument that does not go through finite approximations.

Formalization scope

  • Groups GGG and alphabets AAA live in Type (universe 000). Alphabets are finite (Fintype) and nonempty; their topology is an arbitrary topology assumed to be discrete, and AGA^GAG carries Lean's product topology.
  • The shift is a plain function shift, not a MulAction instance, to avoid a clash with Mathlib's pointwise action on function types.
  • Amenability is taken from the existing platform definition Garrido.IsAmenable (finitely additive invariant probability measure on all subsets). Residual finiteness and soficity are defined in GottschalkSurjunctivity_Defs.
  • Soficity is stated with a normalised count of points; the conditions are required for every ε>0\varepsilon > 0ε>0, so small ε\varepsilonε is where the content lies.
  • Surjunctivity is not trivialised by the nonemptiness assumption: for a nonempty group and a nonempty alphabet, AGA^GAG is nonempty and, when GGG is infinite, uncountable.

Contributions welcome: the Curtis–Hedlund–Lyndon theorem, restriction and induction of cellular automata along subgroups, and the residually finite case are natural first steps.

Selected references

  • W. H. Gottschalk, Some general dynamical notions, Recent Advances in Topological Dynamics, Lecture Notes in Math. 318, Springer, 1973, pp. 120–125. https://doi.org/10.1007/BFb0061728
  • G. A. Hedlund, Endomorphisms and automorphisms of the shift dynamical system, Math. Systems Theory 3 (1969), 320–375. https://doi.org/10.1007/BF01691062
  • T. Ceccherini-Silberstein, A. Machì, F. Scarabotti, Amenable groups and cellular automata, Ann. Inst. Fourier 49 (1999), 673–685. https://doi.org/10.5802/aif.1686
  • M. Gromov, Endomorphisms of symbolic algebraic varieties, J. Eur. Math. Soc. 1 (1999), 109–197. https://doi.org/10.1007/PL00011162
  • B. Weiss, Sofic groups and dynamical systems, Sankhyā Ser. A 62 (2000), 350–359.
  • T. Ceccherini-Silberstein, M. Coornaert, Cellular Automata and Groups, Springer Monographs in Mathematics, 2010. https://doi.org/10.1007/978-3-642-14034-1
  • Wikipedia, Surjunctive group. https://en.wikipedia.org/wiki/Surjunctive_group
10 thms4 active usersReviewed
🏆Completed
AnalysisFunctional AnalysisMathematical Physics·Captain: mikedeng1

Ordinary Differential Equations and Dynamical Systems IV: Regular Sturm–Liouville ProblemsTextbook

Why regular Sturm–Liouville problems

Separation of variables reduces the heat equation, the wave equation and the Schrödinger equation on an interval to an eigenvalue problem for a second-order ordinary differential operator with boundary conditions at both ends. Whether the method works depends on one fact: the eigenfunctions of that operator must be complete, so that every reasonable initial profile can be expanded in them. Chapter 5 of Gerald Teschl's Ordinary Differential Equations and Dynamical Systems (author's preliminary version of AMS GSM 140, 2012) proves this for regular Sturm–Liouville problems. It does so without Lebesgue integration or Hilbert-space completeness: it works in the space of continuous functions with a weighted scalar product, and it develops the necessary spectral theory of compact symmetric operators from scratch.

A short history. Sturm and Liouville (1836–1837) studied the zeros and the eigenvalues of such equations and the expansion of functions in eigenfunctions. Hilbert and Schmidt (1904–1907) put the expansion on a rigorous footing through their theory of integral equations with symmetric kernels, from which the abstract spectral theorem for compact symmetric operators emerged.

Setting

Let a<ba < ba<b and let p,q,rp, q, rp,q,r be real functions on [a,b][a,b][a,b] with

r,q∈C([a,b],R),p∈C1([a,b],R),p(x)>0, r(x)>0  (x∈[a,b]).(5.45)r, q \in C([a,b], \mathbb{R}), \qquad p \in C^1([a,b], \mathbb{R}), \qquad p(x) > 0,\ r(x) > 0 \ \ (x \in [a,b]). \qquad (5.45)r,q∈C([a,b],R),p∈C1([a,b],R),p(x)>0, r(x)>0  (x∈[a,b]).(5.45)

These regularity assumptions hold throughout. The Sturm–Liouville operator is

Lf=1r(−(pf′)′+qf),L f = \frac{1}{r}\Bigl( -(p f')' + q f \Bigr),Lf=r1​(−(pf′)′+qf),

acting on the domain

D(L)={f∈C2([a,b],C):cos⁡(α)f(a)−sin⁡(α)p(a)f′(a)=0,  cos⁡(β)f(b)−sin⁡(β)p(b)f′(b)=0}D(L) = \{ f \in C^2([a,b], \mathbb{C}) : \cos(\alpha) f(a) - \sin(\alpha) p(a) f'(a) = 0,\ \ \cos(\beta) f(b) - \sin(\beta) p(b) f'(b) = 0 \}D(L)={f∈C2([a,b],C):cos(α)f(a)−sin(α)p(a)f′(a)=0,  cos(β)f(b)−sin(β)p(b)f′(b)=0}

for fixed real α,β\alpha, \betaα,β (the separated boundary conditions; α=0\alpha = 0α=0 is Dirichlet, α=π/2\alpha = \pi/2α=π/2 Neumann). The underlying space is H0=C([a,b],C)H_0 = C([a,b], \mathbb{C})H0​=C([a,b],C) with the weighted scalar product

⟨f,g⟩=∫abf(x)∗g(x) r(x) dx\langle f, g \rangle = \int_a^b f(x)^* g(x)\, r(x)\, dx⟨f,g⟩=∫ab​f(x)∗g(x)r(x)dx

and norm ∥f∥=⟨f,f⟩\|f\| = \sqrt{\langle f, f\rangle}∥f∥=⟨f,f⟩​. H0H_0H0​ is an inner product space but it is not complete. A number z∈Cz \in \mathbb{C}z∈C is an eigenvalue of LLL if Lf=zfL f = z fLf=zf for some f∈D(L)f \in D(L)f∈D(L) that does not vanish identically; fff is an eigenfunction. An orthonormal basis of H0H_0H0​ is an orthonormal sequence (un)(u_n)(un​) such that ∥f−∑n<N⟨un,f⟩un∥→0\|f - \sum_{n<N} \langle u_n, f\rangle u_n\| \to 0∥f−∑n<N​⟨un​,f⟩un​∥→0 for every f∈H0f \in H_0f∈H0​.

For the abstract part, H0H_0H0​ is any complex inner product space (again not assumed complete). A linear operator AAA on a dense subspace is symmetric if ⟨g,Af⟩=⟨Ag,f⟩\langle g, Af\rangle = \langle Ag, f\rangle⟨g,Af⟩=⟨Ag,f⟩; an everywhere defined AAA is compact if (Afn)(A f_n)(Afn​) has a convergent subsequence in H0H_0H0​ whenever (fn)(f_n)(fn​) is bounded.

For z∈Cz \in \mathbb{C}z∈C let ua(z,⋅)u_a(z,\cdot)ua​(z,⋅), ub(z,⋅)u_b(z,\cdot)ub​(z,⋅) solve Lu=zuL u = z uLu=zu with ua(z,a)=sin⁡αu_a(z,a) = \sin\alphaua​(z,a)=sinα, p(a)ua′(z,a)=cos⁡αp(a)u_a'(z,a) = \cos\alphap(a)ua′​(z,a)=cosα, ub(z,b)=sin⁡βu_b(z,b) = \sin\betaub​(z,b)=sinβ, p(b)ub′(z,b)=cos⁡βp(b)u_b'(z,b) = \cos\betap(b)ub′​(z,b)=cosβ. Their Wronskian is W(z)=ubpua′−pub′uaW(z) = u_b p u_a' - p u_b' u_aW(z)=ub​pua′​−pub′​ua​, which does not depend on the point where it is evaluated. When W(z)≠0W(z) \neq 0W(z)=0, the resolvent RL(z)R_L(z)RL​(z) is the integral operator with the Green function G(z,x,t)=W(z)−1ub(z,max⁡(x,t)) ua(z,min⁡(x,t))G(z,x,t) = W(z)^{-1} u_b(z,\max(x,t))\, u_a(z,\min(x,t))G(z,x,t)=W(z)−1ub​(z,max(x,t))ua​(z,min(x,t)) and weight rrr.

Formalization targets

Goal: eigenfunction expansion (Theorem 5.11)

There are real numbers EnE_nEn​ and real-valued eigenfunctions un∈D(L)u_n \in D(L)un​∈D(L), Lun=EnunL u_n = E_n u_nLun​=En​un​, such that:

  • the EnE_nEn​ are pairwise distinct and are all the eigenvalues of LLL, and every eigenfunction for EnE_nEn​ is a multiple of unu_nun​ (the eigenvalues are simple);
  • ∣En∣→∞|E_n| \to \infty∣En​∣→∞ (the eigenvalues accumulate only at ∞\infty∞);
  • ⟨um,un⟩=δmn\langle u_m, u_n \rangle = \delta_{mn}⟨um​,un​⟩=δmn​, and for every f∈H0f \in H_0f∈H0​
f=∑n=0∞⟨un,f⟩ unin H0,(5.70)f = \sum_{n=0}^{\infty} \langle u_n, f \rangle\, u_n \quad \text{in } H_0, \qquad (5.70)f=n=0∑∞​⟨un​,f⟩un​in H0​,(5.70)
  • for f∈D(L)f \in D(L)f∈D(L) the series (5.70) converges uniformly on [a,b][a,b][a,b].

Milestones

  1. Theorem 5.4. Eigenvalues of a symmetric operator are real, and eigenvectors for different eigenvalues are orthogonal.
  2. Theorem 5.5. A compact symmetric operator on a nonzero inner product space has an eigenvalue α0\alpha_0α0​ with ∣α0∣=∥A∥=sup⁡∥f∥=1∥Af∥|\alpha_0| = \|A\| = \sup_{\|f\|=1}\|Af\|∣α0​∣=∥A∥=sup∥f∥=1​∥Af∥.
  3. Theorem 5.6 (spectral theorem). A compact symmetric operator has real eigenvalues αj\alpha_jαj​ with orthonormal eigenvectors uju_juj​, infinitely many when H0H_0H0​ is infinite dimensional and then αj→0\alpha_j \to 0αj​→0, and f=∑j⟨uj,f⟩ujf = \sum_j \langle u_j, f\rangle u_jf=∑j​⟨uj​,f⟩uj​ for every f∈Ran⁡(A)f \in \operatorname{Ran}(A)f∈Ran(A); if Ran⁡(A)\operatorname{Ran}(A)Ran(A) is dense the uju_juj​ are an orthonormal basis.
  4. Lemma 5.9. W(z)W(z)W(z) is an entire function of zzz whose zeros are exactly the eigenvalues of LLL.
  5. Lemma 5.10. For W(z)≠0W(z) \ne 0W(z)=0, RL(z)R_L(z)RL​(z) is compact on H0H_0H0​, and symmetric for real zzz.
  6. Lemma 5.12 (Rayleigh–Ritz). The eigenvalues can be ordered E0<E1<⋯E_0 < E_1 < \cdotsE0​<E1​<⋯; E0=min⁡⟨f,Lf⟩=min⁡Q(f)E_0 = \min \langle f, Lf\rangle = \min Q(f)E0​=min⟨f,Lf⟩=minQ(f) over normalized f∈D(L)f \in D(L)f∈D(L), attained exactly at the eigenfunctions for E0E_0E0​; and min⁡[a,b]q≤E0\min_{[a,b]} q \le E_0min[a,b]​q≤E0​ when 0≤α≤π/2≤β≤π0 \le \alpha \le \pi/2 \le \beta \le \pi0≤α≤π/2≤β≤π. Here QQQ is the quadratic form ∫ab(p∣f′∣2+q∣f∣2) dx\int_a^b (p |f'|^2 + q |f|^2)\,dx∫ab​(p∣f′∣2+q∣f∣2)dx plus the boundary terms cot⁡(α)∣f(a)∣2−cot⁡(β)∣f(b)∣2\cot(\alpha)|f(a)|^2 - \cot(\beta)|f(b)|^2cot(α)∣f(a)∣2−cot(β)∣f(b)∣2 (dropped when the angle is 000).

Significance

Theorem 5.11 is the statement that makes eigenfunction expansions legitimate: Fourier sine and cosine series are its special case p=r=1p = r = 1p=r=1, q=0q = 0q=0, and it covers any positive weight and any smooth positive stiffness with separated boundary conditions. Solutions of the heat and wave equations on an interval with variable coefficients are built from it. The uniform convergence for f∈D(L)f \in D(L)f∈D(L) is the version needed to differentiate or evaluate such series pointwise. Lemma 5.12 provides the variational characterisation of the lowest eigenvalue, the starting point for comparison theorems and eigenvalue estimates.

On the formal side, Mathlib has the spectral theorem for self-adjoint operators on finite-dimensional spaces, compact operators on normed spaces, and L2L^2L2 spaces. It has no Sturm–Liouville theory, no Green functions for boundary value problems, and no spectral theorem for compact symmetric operators on incomplete inner product spaces. The platform has finite-dimensional Rayleigh-quotient results and compactness of L2L^2L2 convolution operators, but nothing on this setting. The results here are classical and proved; what is missing is a machine-checked proof in the book's elementary setting of continuous functions.

Difficulty

The operator LLL is unbounded on H0H_0H0​ (Problem 5.16), so the obvious first idea, applying a spectral theorem for compact or bounded symmetric operators to LLL itself, does not work. Completeness of the eigenfunctions is also subtle because H0H_0H0​ is not complete. Hilbert-space arguments that produce limits in a completion, or that go through L2L^2L2, do not by themselves give an expansion of a continuous function in continuous eigenfunctions with convergence in H0H_0H0​, still less uniform convergence. Simplicity of the eigenvalues depends on the separated boundary conditions and fails for periodic ones. On the formal side, one needs analytic dependence of solutions of a linear ODE on a complex parameter, one-sided derivatives at the endpoints, and interval integrals of piecewise-defined kernels.

Formalization scope

  • Spaces. The abstract results use a type with [NormedAddCommGroup E] [InnerProductSpace ℂ E] and no CompleteSpace. Symmetry on a dense domain is IsSymmetricOp; for everywhere defined operators it is Mathlib's LinearMap.IsSymmetric. Compactness is IsCompactOp: subsequences converge in H0H_0H0​ itself. H0=C([a,b],C)H_0 = C([a,b],\mathbb{C})H0​=C([a,b],C) is represented by functions R→C\mathbb{R} \to \mathbb{C}R→C continuous on Set.Icc a b, with SLInner the weighted scalar product (5.52) as an interval integral. It is not L2L^2L2 and not a HilbertBasis.
  • Operator and domain. SLOp, SLDomain, IsSLEigenfunction implement (5.53)–(5.55) with derivWithin on Set.Icc a b (one-sided at the endpoints) and ContDiffOn ℝ 2. RegularSL is (5.45), including a<ba < ba<b. α,β\alpha, \betaα,β are arbitrary reals.
  • Quantifier readings. "Countable, discrete, accumulating only at ∞\infty∞" is an injective sequence EnE_nEn​ exhausting the eigenvalues with ∣En∣→∞|E_n| \to \infty∣En​∣→∞. "Simple" is: every eigenfunction for EnE_nEn​ equals c unc\,u_ncun​ on [a,b][a,b][a,b]. "Can be chosen real-valued" is an existential choice of un:R→Ru_n : \mathbb{R} \to \mathbb{R}un​:R→R. ∥A∥\|A\|∥A∥ in Theorem 5.5 is expressed by IsLUB of {∥Af∥:∥f∥=1}\{\|Af\| : \|f\|=1\}{∥Af∥:∥f∥=1}, and Theorem 5.5 assumes H0≠{0}H_0 \neq \{0\}H0​={0}. In Theorem 5.6, N∈N0∪{∞}N \in \mathbb{N}_0 \cup \{\infty\}N∈N0​∪{∞} is ℕ∞: N=∞N = \inftyN=∞ in infinite dimension and N=dim⁡H0N = \dim H_0N=dimH0​ otherwise. The minima of Lemma 5.12 are IsLeast (attained), and "equality iff f=u0f = u_0f=u0​" reads as "iff fff is an eigenfunction for E0E_0E0​", i.e. f=c u0f = c\,u_0f=cu0​ with ∣c∣=1|c| = 1∣c∣=1. min⁡q≤E0\min q \le E_0minq≤E0​ is ∃x∈[a,b], q(x)≤E0\exists x \in [a,b],\ q(x) \le E_0∃x∈[a,b], q(x)≤E0​. Norm convergence is Re⁡⟨f−SN,f−SN⟩→0\operatorname{Re}\langle f - S_N, f - S_N\rangle \to 0Re⟨f−SN​,f−SN​⟩→0.
  • Ruling out trivial readings. The resolvent is only considered with W(z)≠0W(z) \ne 0W(z)=0 (Lean's 0−1=00^{-1} = 00−1=0 would make it the zero operator). Scalar products are only applied to functions continuous on [a,b][a,b][a,b], so no integral takes a junk value. Eigenfunctions must be nonzero on [a,b][a,b][a,b]. The expansion must hold in the weighted norm for every continuous fff, which an orthonormal family that is merely orthonormal would not satisfy.

A complete development needs linear ODEs with a complex parameter and their analytic dependence on it, Green functions, an Arzelà–Ascoli argument in C([a,b])C([a,b])C([a,b]), and the spectral theorem for compact symmetric operators on an inner product space. The definitions RegularSL, SLOp, SLDomain, SLInner and SLWronskian are reusable for the oscillation and comparison theory (Theorems 5.17–5.25) in the rest of the chapter. Contributions connecting IsCompactOp to Mathlib's IsCompactOperator on the completion are welcome.

Selected references

  • G. Teschl, Ordinary Differential Equations and Dynamical Systems, Graduate Studies in Mathematics 140, American Mathematical Society, 2012. Author's preliminary version: https://www.mat.univie.ac.at/~gerald/ftp/book-ode/ode.pdf ; published version: https://doi.org/10.1090/gsm/140
  • J. C. F. Sturm, Mémoire sur les équations différentielles linéaires du second ordre, Journal de Mathématiques Pures et Appliquées 1 (1836), 106–186. http://www.numdam.org/item/JMPA_1836_1_1__106_0/
  • J. Liouville, Mémoire sur le développement des fonctions ou parties de fonctions en séries dont les divers termes sont assujettis à satisfaire à une même équation différentielle du second ordre, Journal de Mathématiques Pures et Appliquées 1 (1836), 253–265. http://www.numdam.org/item/JMPA_1836_1_1__253_0/
  • E. Schmidt, Zur Theorie der linearen und nichtlinearen Integralgleichungen. I, Mathematische Annalen 63 (1907), 433–476. https://doi.org/10.1007/BF01449770
17 thms4 active usersReviewed
🏆Completed
AnalysisLinear algebra·Captain: mikedeng1

Ordinary Differential Equations and Dynamical Systems II: Linear Systems and Floquet TheoryTextbook

Motivation

Linear systems of ordinary differential equations x˙=A(t)x\dot x = A(t)xx˙=A(t)x are the first case of the general theory in which solutions can be described structurally rather than merely shown to exist. They arise directly (circuits, mechanical vibrations, Hill's equation for the stability of periodic motions, the Mathieu equation for parametric resonance) and indirectly, as the linearization of a nonlinear system along an equilibrium or a periodic orbit. In the second role they decide stability: the linearized stability theorems of later chapters, the Poincaré map of a periodic orbit and the stable-manifold theory all reduce questions about nonlinear flows to questions about linear systems with constant or periodic coefficients.

For periodic coefficients the central structural result is due to Gaston Floquet (Sur les équations différentielles linéaires à coefficients périodiques, Annales scientifiques de l'École Normale Supérieure 12 (1883), 47–88, doi:10.24033/asens.220), preceded by G. W. Hill's 1877 work on the lunar perigee, which studied the same equation in the scalar second-order case. Liouville's formula for the Wronski determinant goes back to Abel (1829) and Liouville (1838). This mission follows Chapter 3 of G. Teschl, Ordinary Differential Equations and Dynamical Systems (AMS Graduate Studies in Mathematics 140, 2012), in the author's preliminary version.

Setting

Fix n∈Nn \in \mathbb{N}n∈N, an interval I⊆RI \subseteq \mathbb{R}I⊆R and a continuous matrix function A:I→Rn×nA : I \to \mathbb{R}^{n\times n}A:I→Rn×n. A solution of the linear first-order system

x˙(t)=A(t) x(t)(3.79)\dot x(t) = A(t)\,x(t) \qquad (3.79)x˙(t)=A(t)x(t)(3.79)

on III is a function xxx with values in Rn\mathbb{R}^nRn that at every t∈It \in It∈I has derivative A(t)x(t)A(t)x(t)A(t)x(t) (one-sided at an endpoint of III). In Lean this is TeschlODE.Linear.IsSolution A I x.

The principal matrix solution Π(t,t0)\Pi(t,t_0)Π(t,t0​) is, for each t0∈It_0 \in It0​∈I, the matrix function that solves the matrix initial value problem

ddtΠ(t,t0)=A(t) Π(t,t0),Π(t0,t0)=I.(3.83)\frac{d}{dt}\Pi(t,t_0) = A(t)\,\Pi(t,t_0), \qquad \Pi(t_0,t_0) = \mathbb{I}. \qquad (3.83)dtd​Π(t,t0​)=A(t)Π(t,t0​),Π(t0​,t0​)=I.(3.83)

Its columns are the solutions of (3.79) starting at the canonical basis vectors, and every solution is x(t)=Π(t,t0)x(t0)x(t) = \Pi(t,t_0)x(t_0)x(t)=Π(t,t0​)x(t0​). In Lean the property is TeschlODE.Linear.IsPrincipalMatrixSolution A I Φ (the Lean name of Π\PiΠ is Φ). The Wronski determinant of nnn solutions φ1,…,φn\varphi_1,\dots,\varphi_nφ1​,…,φn​ is W(t)=det⁡(φ1(t),…,φn(t))W(t) = \det(\varphi_1(t),\dots,\varphi_n(t))W(t)=det(φ1​(t),…,φn​(t)).

A system is periodic if I=RI = \mathbb{R}I=R and A(t+T)=A(t)A(t+T) = A(t)A(t+T)=A(t) for all ttt, for some T>0T > 0T>0 (3.117). Its monodromy matrix is M(t0)=Π(t0+T,t0)M(t_0) = \Pi(t_0+T, t_0)M(t0​)=Π(t0​+T,t0​) (3.119). A logarithm of a square matrix MMM is any matrix BBB with exp⁡(B)=M\exp(B) = Mexp(B)=M (3.200), where exp⁡\expexp is the matrix exponential.

Formalization targets

Goal: Floquet's theorem (Theorem 3.15)

Let A∈C(R,Rn×n)A \in C(\mathbb{R}, \mathbb{R}^{n\times n})A∈C(R,Rn×n) be TTT-periodic with T>0T > 0T>0 and Π\PiΠ its principal matrix solution. For every t0∈Rt_0 \in \mathbb{R}t0​∈R there exist Q(t0)∈Cn×nQ(t_0) \in \mathbb{C}^{n\times n}Q(t0​)∈Cn×n and P(⋅,t0):R→Cn×nP(\cdot,t_0) : \mathbb{R} \to \mathbb{C}^{n\times n}P(⋅,t0​):R→Cn×n with

Π(t,t0)=P(t,t0) exp⁡((t−t0) Q(t0)),P(t+T,t0)=P(t,t0),P(t0,t0)=I.\Pi(t,t_0) = P(t,t_0)\,\exp\big((t-t_0)\,Q(t_0)\big), \qquad P(t+T,t_0) = P(t,t_0), \qquad P(t_0,t_0) = \mathbb{I}.Π(t,t0​)=P(t,t0​)exp((t−t0​)Q(t0​)),P(t+T,t0​)=P(t,t0​),P(t0​,t0​)=I.

Milestones

In attack order:

  1. Theorem 3.9 — for continuous AAA on an interval III the initial value problem x˙=A(t)x\dot x = A(t)xx˙=A(t)x, x(t0)=x0x(t_0) = x_0x(t0​)=x0​ has a unique solution, defined on all of III.
  2. Theorem 3.10 — the solutions form an nnn-dimensional vector space, and a principal matrix solution exists with x(t)=Π(t,t0)x0x(t) = \Pi(t,t_0)x_0x(t)=Π(t,t0​)x0​.
  3. Lemma 3.14 — for TTT-periodic AAA, Π(t+T,t0+T)=Π(t,t0)\Pi(t+T, t_0+T) = \Pi(t,t_0)Π(t+T,t0​+T)=Π(t,t0​).
  4. Lemma 3.11 (Liouville's formula) — W(t)=W(t0)exp⁡(∫t0ttr⁡A(s) ds)W(t) = W(t_0)\exp\big(\int_{t_0}^t \operatorname{tr} A(s)\,ds\big)W(t)=W(t0​)exp(∫t0​t​trA(s)ds).
  5. Lemma 3.34 — a complex matrix has a logarithm iff its determinant is nonzero; a real matrix whose real eigenvalues are all positive has a real logarithm; the square of an invertible real matrix has a real logarithm.
  6. Theorem 3.12 (variation of constants) — the solution of x˙=A(t)x+g(t)\dot x = A(t)x + g(t)x˙=A(t)x+g(t), x(t0)=x0x(t_0) = x_0x(t0​)=x0​, is x(t)=Π(t,t0)x0+∫t0tΠ(t,s)g(s) dsx(t) = \Pi(t,t_0)x_0 + \int_{t_0}^t \Pi(t,s)g(s)\,dsx(t)=Π(t,t0​)x0​+∫t0​t​Π(t,s)g(s)ds.

Milestone 6 is not needed for the goal; it completes the linear theory of Section 3.4 on the same definitions and is the input to the perturbation results of Section 3.7.

Significance

Floquet's theorem reduces a periodic linear system to one with constant coefficients by a periodic change of variables. Its consequences in the book are the real version with doubled period (Corollary 3.16), the stability criterion in terms of Floquet multipliers (the eigenvalues of M(t0)M(t_0)M(t0​), Corollary 3.17), the reduction y=P−1xy = P^{-1}xy=P−1x to y˙=Qy\dot y = Q yy˙​=Qy (Corollary 3.18), and the stability analysis of Hill's equation. Later in the book, the stability of a periodic orbit of a nonlinear system is read off from the Floquet multipliers of its linearization, via the Poincaré map.

All results of this mission have been proved for more than a century. What the mission adds is a machine-checked development: Mathlib has the matrix exponential (NormedSpace.exp, with Matrix.exp_add_of_commute), Picard–Lindelöf and Gronwall-type uniqueness for ODEs, but no principal matrix solution, no Liouville formula, no matrix logarithm and no Floquet theory. The two platform nodes named after Floquet's and Liouville's theorems are retired placeholders whose statement is True; no faithful formalization exists on the platform.

Difficulty

The obvious guess for the solution, exp⁡(∫t0tA(s) ds)x0\exp\big(\int_{t_0}^t A(s)\,ds\big)x_0exp(∫t0​t​A(s)ds)x0​, is wrong as soon as the values A(t)A(t)A(t) do not commute, so neither Floquet's theorem nor the variation-of-constants formula can be obtained from the constant-coefficient theory by substitution. Floquet's theorem as stated is equivalent to the existence of a logarithm of the monodromy matrix: once M(t0)=exp⁡(TQ)M(t_0) = \exp(TQ)M(t0​)=exp(TQ) is available, the periodicity of PPP follows from Lemma 3.14. The existence of a matrix logarithm for every invertible complex matrix is the main missing piece of infrastructure; the logarithm is not unique and not continuous in the matrix, and a real invertible matrix need not have a real logarithm (a negative eigenvalue with a single Jordan block prevents it), which is why Q(t0)Q(t_0)Q(t0​) is complex. Liouville's formula, which gives det⁡M(t0)=exp⁡(∫0Ttr⁡A)≠0\det M(t_0) = \exp\big(\int_0^T \operatorname{tr} A\big) \neq 0detM(t0​)=exp(∫0T​trA)=0, needs the derivative of a determinant along a matrix solution.

Formalization scope

  • Vectors are Fin n → ℝ, matrices Matrix (Fin n) (Fin n) ℝ; A(t)xA(t)xA(t)x is Matrix.mulVec. The goal and Lemma 3.34 use Matrix (Fin n) (Fin n) ℂ for PPP, QQQ and the complex logarithm; Π(t,t0)\Pi(t,t_0)Π(t,t0​) is compared with Pexp⁡(⋅)P\exp(\cdot)Pexp(⋅) after coercing its entries to C\mathbb{C}C.
  • An interval is a set III with Set.OrdConnected I; continuity is ContinuousOn A I. Derivatives are HasDerivWithinAt … I t at every t∈It \in It∈I (one-sided at endpoints in III); the matrix derivative in (3.83) is taken entrywise. In Section 3.6 (Lemma 3.14 and the goal) I=RI = \mathbb{R}I=R and AAA is Continuous.
  • The principal matrix solution enters as a hypothesis IsPrincipalMatrixSolution A Set.univ Φ. Theorem 3.10 shows such a Φ\PhiΦ exists and Theorem 3.9 shows it is unique on I×II\times II×I, so this hypothesis names the principal matrix solution, not an arbitrary function.
  • Periodicity is 0 < T and ∀ t, A (t + T) = A t. The periodicity of PPP is required with the same TTT and for all ttt, together with P(t0,t0)=IP(t_0,t_0) = \mathbb{I}P(t0​,t0​)=I. Without the periodicity of PPP the goal would be trivial (Q=0Q = 0Q=0, P=ΠP = \PiP=Π); with it, the goal carries the full content of the theorem.
  • The matrix exponential is Mathlib's NormedSpace.exp. "Logarithm" means any BBB with NormedSpace.exp B = M; no branch is fixed. In the third claim of Lemma 3.34 the hypothesis det⁡A≠0\det A \ne 0detA=0 is explicit: the book states it in the paragraph containing the lemma, and without it the claim fails at A=0A = 0A=0.
  • "The solutions form an nnn-dimensional vector space" is expressed through restrictions to III: closure under linear combinations, nnn solutions linearly independent on III, and every solution a combination of them on III.
  • Needed infrastructure, reusable beyond this mission: existence and uniqueness for linear systems on arbitrary intervals, the principal matrix solution and its cocycle property Π(t,t1)Π(t1,t0)=Π(t,t0)\Pi(t,t_1)\Pi(t_1,t_0) = \Pi(t,t_0)Π(t,t1​)Π(t1​,t0​)=Π(t,t0​), the derivative of det⁡\detdet along a matrix solution, and the matrix logarithm (via the Jordan form or via the holomorphic functional calculus). Contributions of any of these as standalone lemmas are welcome.

Selected references

  • G. Teschl, Ordinary Differential Equations and Dynamical Systems, Graduate Studies in Mathematics 140, American Mathematical Society, 2012; author's preliminary version, Chapter 3. https://www.mat.univie.ac.at/~gerald/ftp/book-ode/ode.pdf (published version: doi:10.1090/gsm/140).
  • G. Floquet, Sur les équations différentielles linéaires à coefficients périodiques, Annales scientifiques de l'École Normale Supérieure 12 (1883), 47–88. doi:10.24033/asens.220
  • W. J. Culver, On the existence and uniqueness of the real logarithm of a matrix, Proceedings of the AMS 17 (1966), 1146–1151. doi:10.1090/S0002-9939-1966-0202740-6
9 thms4 active usersReviewed
🏆Completed
AnalysisControl TheoryOperations Research+1·Captain: mikedeng1

Generalized Gradients and Applications II: Flow-Invariant Sets of Lipschitz Differential InclusionsResearch Paper

Motivation

A closed set F⊆RnF\subseteq\mathbb R^nF⊆Rn is flow-invariant for a dynamical system when every trajectory that starts in FFF stays in FFF. Invariance of sets underlies state constraints in optimal control, safety certificates for controlled systems, comparison and maximum principles for differential equations, and the positivity of solutions of kinetic and population models. The question is always the same: which infinitesimal condition at the points of FFF is equivalent to the global statement that trajectories cannot leave FFF?

For a smooth vector field and a smooth boundary the answer is that the field must not point strictly outward. For a nonsmooth set, such as a polyhedron, the positive orthant or a set with inward cusps, "pointing inward" must be made precise through a notion of tangent vector that works at corners. Frank H. Clarke's 1975 paper introduces the generalized gradient of a Lipschitz function and derives from it a normal cone and a tangent cone for arbitrary closed sets. Its Theorem (4.4) shows that this tangent cone is exactly the right notion: for a Lipschitz differential inclusion x˙∈X(x)\dot x\in X(x)x˙∈X(x), a closed set is flow-invariant if and only if X(x)X(x)X(x) is contained in the tangent cone at each point of the set.

Timeline:

  • 1942. Nagumo characterizes invariance for continuous ODEs with unique solutions by a condition on the distance function (Proc. Phys.-Math. Soc. Japan 24).
  • 1969. Bony proves an invariance theorem for Lipschitz vector fields, stated through exterior normals, in the course of a maximum principle for degenerate elliptic operators (Bony 1969).
  • 1970. Brezis characterizes flow-invariant closed sets of a locally Lipschitz vector field by lim⁡δ↓0dF(y+δX(y))/δ=0\lim_{\delta\downarrow0} d_F(y+\delta X(y))/\delta=0limδ↓0​dF​(y+δX(y))/δ=0 (Brezis 1970).
  • 1972. Redheffer gives simplified proofs of the Bony and Brezis theorems under weaker "uniqueness function" hypotheses (Amer. Math. Monthly 79, 740–747).
  • 1975. Clarke extends the characterization to Lipschitz multifunctions with nonempty compact values, with tangency in the sense of his new tangent cone, and recovers Bony and Brezis as corollaries (Clarke 1975, Theorem (4.4), Corollaries (4.10), (4.12)).

Setting

Throughout, Rn\mathbb R^nRn carries the Euclidean inner product ζ⋅v\zeta\cdot vζ⋅v and norm ∣⋅∣|\cdot|∣⋅∣.

Generalized gradient. For f:Rn→Rf:\mathbb R^n\to\mathbb Rf:Rn→R Lipschitz on bounded sets, ∂f(x)\partial f(x)∂f(x) is the convex hull of all limits lim⁡i∇f(x+hi)\lim_i\nabla f(x+h_i)limi​∇f(x+hi​) with hi→0h_i\to0hi​→0 and fff differentiable at each x+hix+h_ix+hi​ (Definition (1.1)). The generalized directional derivative is f∘(x;v)=lim sup⁡h→0, δ↓0[f(x+h+δv)−f(x+h)]/δf^\circ(x;v)=\limsup_{h\to0,\ \delta\downarrow0}[f(x+h+\delta v)-f(x+h)]/\deltaf∘(x;v)=limsuph→0, δ↓0​[f(x+h+δv)−f(x+h)]/δ (Definition (1.3)).

Distance function. For a nonempty closed E⊆RnE\subseteq\mathbb R^nE⊆Rn, dE(x)=min⁡{∣x−e∣:e∈E}d_E(x)=\min\{|x-e|:e\in E\}dE​(x)=min{∣x−e∣:e∈E}. It is Lipschitz with constant 111. A point e∈Ee\in Ee∈E with ∣x−e∣=dE(x)|x-e|=d_E(x)∣x−e∣=dE​(x) is a closest point to xxx; it exists but need not be unique.

Normal and tangent cones. For e∈Ee\in Ee∈E, the cone of normals is

NE(e)=cl⁡{p: s p∈∂dE(e) for some s>0}(Definition (3.1)),N_E(e)=\operatorname{cl}\{p:\ s\,p\in\partial d_E(e)\text{ for some }s>0\}\qquad\text{(Definition (3.1))},NE​(e)=cl{p: sp∈∂dE​(e) for some s>0}(Definition (3.1)),

and the tangent cone is its dual,

TE(e)={ζ: ζ⋅v≤0 for all v∈NE(e)}(Definition (3.6)).T_E(e)=\{\zeta:\ \zeta\cdot v\le0\text{ for all }v\in N_E(e)\}\qquad\text{(Definition (3.6))}.TE​(e)={ζ: ζ⋅v≤0 for all v∈NE​(e)}(Definition (3.6)).

Differential inclusions. A multifunction XXX assigns to each x∈Rnx\in\mathbb R^nx∈Rn a set X(x)⊆RnX(x)\subseteq\mathbb R^nX(x)⊆Rn; standing assumption of §4: every X(x)X(x)X(x) is nonempty and compact. A trajectory is an absolutely continuous x:[0,1]→Rnx:[0,1]\to\mathbb R^nx:[0,1]→Rn with x˙(t)∈X(x(t))\dot x(t)\in X(x(t))x˙(t)∈X(x(t)) for almost every ttt ((4.1)). XXX is Lipschitz if there is KKK such that for all x1,x2x_1,x_2x1​,x2​ and v1∈X(x1)v_1\in X(x_1)v1​∈X(x1​) some v2∈X(x2)v_2\in X(x_2)v2​∈X(x2​) has ∣v1−v2∣≤K∣x1−x2∣|v_1-v_2|\le K|x_1-x_2|∣v1​−v2​∣≤K∣x1​−x2​∣ ((4.2)). A closed set FFF is flow-invariant for XXX if every trajectory with x(0)∈Fx(0)\in Fx(0)∈F has x(t)∈Fx(t)\in Fx(t)∈F for all t∈[0,1]t\in[0,1]t∈[0,1] ((4.3)).

Formalization targets

Goal: Theorem (4.4)

Let XXX be a Lipschitz multifunction with nonempty compact values and FFF a nonempty closed subset of Rn\mathbb R^nRn. Then

F is flow-invariant for X  ⟺  X(x)⊆TF(x)  for every x∈F.F\text{ is flow-invariant for }X\iff X(x)\subseteq T_F(x)\ \text{ for every }x\in F.F is flow-invariant for X⟺X(x)⊆TF​(x)  for every x∈F.

Milestones, in the order the proof uses them

  1. Proposition (1.4). f∘(x;v)=max⁡{ζ⋅v:ζ∈∂f(x)}f^\circ(x;v)=\max\{\zeta\cdot v:\zeta\in\partial f(x)\}f∘(x;v)=max{ζ⋅v:ζ∈∂f(x)} for locally Lipschitz fff.
  2. Proposition (2.4). If ∇dE(x)\nabla d_E(x)∇dE​(x) exists and is nonzero, then x∉Ex\notin Ex∈/E, xxx has a unique closest point eee, and ∇dE(x)=(x−e)/∣x−e∣\nabla d_E(x)=(x-e)/|x-e|∇dE​(x)=(x−e)/∣x−e∣.
  3. Corollary (2.5). For e∈Ee\in Ee∈E, ∂dE(e)=co⁡{0,lim⁡(xi−ei)/∣xi−ei∣}\partial d_E(e)=\operatorname{co}\{0,\lim (x_i-e_i)/|x_i-e_i|\}∂dE​(e)=co{0,lim(xi​−ei​)/∣xi​−ei​∣} over xi∉Ex_i\notin Exi​∈/E, xi→ex_i\to exi​→e, eie_iei​ closest to xix_ixi​.
  4. Proposition (3.2). NE(e)=cl⁡co⁡{lim⁡si(xi−ei)}N_E(e)=\operatorname{cl}\operatorname{co}\{\lim s_i(x_i-e_i)\}NE​(e)=clco{limsi​(xi​−ei​)} over si≥0s_i\ge0si​≥0, xi→ex_i\to exi​→e, eie_iei​ closest to xix_ixi​.
  5. Inequality (4.8). If X(y)⊆TF(y)X(y)\subseteq T_F(y)X(y)⊆TF​(y) on FFF and xxx is a trajectory, then f(t)=dF(x(t))f(t)=d_F(x(t))f(t)=dF​(x(t)) satisfies f′(t)≤Kf(t)f'(t)\le Kf(t)f′(t)≤Kf(t) almost everywhere.
  6. Limit (4.9). If FFF is flow-invariant, then dF(y+δv)/δ→0d_F(y+\delta v)/\delta\to0dF​(y+δv)/δ→0 as δ↓0\delta\downarrow0δ↓0 for every y∈Fy\in Fy∈F and v∈X(y)v\in X(y)v∈X(y).
  7. Proposition (3.7). v∈TE(e0)v\in T_E(e_0)v∈TE​(e0​) iff lim⁡e→e0, e∈Elim inf⁡δ↓0dE(e+δv)/δ=0\lim_{e\to e_0,\,e\in E}\liminf_{\delta\downarrow0} d_E(e+\delta v)/\delta=0lime→e0​,e∈E​liminfδ↓0​dE​(e+δv)/δ=0.

Significance

The result. Theorem (4.4) turns a statement about all trajectories of a set-valued dynamical system into a pointwise geometric condition on FFF that can be checked without solving anything. It is the prototype of the strong invariance theorems of nonsmooth control theory, later developed in viability theory and in the monograph of Clarke, Ledyaev, Stern and Wolenski, and it is the tool behind state-constrained optimal control and Lyapunov-type arguments for nonsmooth systems. Its corollaries recover the Bony and Brezis theorems for Lipschitz vector fields. The companion milestones (2.5), (3.2) and (3.7) are standalone facts of nonsmooth geometry: the Clarke normal cone is generated by limits of proximal normals, and Clarke tangency can be tested along rays from nearby points of the set.

Formalizing it. The result is proved in the paper and has been reproved in textbooks; to the best of our knowledge it has no machine-checked proof. Mathlib contains Rademacher's theorem, absolutely continuous functions on intervals and the Bouligand tangent cone, but no Clarke generalized gradient, no Clarke normal or tangent cone, and no theory of differential inclusions. A complete development supplies a first nonsmooth-analysis layer on top of Mathlib and a first existence theorem for Lipschitz differential inclusions.

Difficulty

The direction (2) ⇒\Rightarrow⇒ (1) looks like a Gronwall argument for f(t)=dF(x(t))f(t)=d_F(x(t))f(t)=dF​(x(t)), but dFd_FdF​ is not differentiable, xxx is only absolutely continuous, and the closest point to x(t)x(t)x(t) can jump. The step that must be controlled is the comparison between x˙(t)\dot x(t)x˙(t), a nearby admissible velocity at the closest point, and the normal x(t)−yx(t)-yx(t)−y; this is where Proposition (3.2) enters, and it is why the Clarke cone, rather than a weaker cone, is needed.

The direction (1) ⇒\Rightarrow⇒ (2) needs a trajectory through an arbitrary y∈Fy\in Fy∈F whose initial velocity is a prescribed v∈X(y)v\in X(y)v∈X(y). For a nonconvex multifunction this is Filippov's theorem [7, Theorem 5], which the paper cites and does not prove. A solver must prove it, or an equivalent existence result for Lipschitz inclusions with compact values, from scratch in Lean. The Clarke tangent cone and the more familiar Bouligand (contingent) cone differ pointwise at nonconvex corners, so a statement written with Mathlib's tangentConeAt is a different theorem from (4.4). That the two universal conditions "X(x)⊆TF(x)X(x)\subseteq T_F(x)X(x)⊆TF​(x) for all x∈Fx\in Fx∈F" are equivalent for Lipschitz XXX is a later, separate result and cannot be assumed.

Formalization scope

  • Rn\mathbb R^nRn is EuclideanSpace ℝ (Fin n); dEd_EdE​ is Metric.infDist · E; "eee is a closest point to xxx" is e ∈ E ∧ dist x e = infDist x E.
  • ∂f\partial f∂f and f∘f^\circf∘ follow Definitions (1.1) and (1.3). The gradient limits carry DifferentiableAt, because Mathlib's gradient is 000 off the differentiability set. f∘f^\circf∘ is a real limsup, and every theorem using it carries the §1 Lipschitz hypothesis.
  • NEN_ENE​ and TET_ETE​ are defined from ∂dE\partial d_E∂dE​ as in (3.1) and (3.6). The tangent cone is not Mathlib's tangentConeAt.
  • A trajectory is x : ℝ → ℝⁿ with AbsolutelyContinuousOnInterval x 0 1 and, for almost every t∈[0,1]t\in[0,1]t∈[0,1], ∃ w ∈ X (x t), HasDerivAt x w t. Values outside [0,1][0,1][0,1] are irrelevant.
  • Standing assumptions made explicit: X(x)X(x)X(x) nonempty and compact for every xxx (§4, p. 259); EEE, FFF nonempty and closed and e∈Ee\in Ee∈E (§3, p. 254); fff Lipschitz on bounded sets (§1, p. 247). The Lipschitz constant of (4.2) is global.
  • Ruled out: dropping absolute continuity (Cantor-type curves would leave FFF), using deriv in (4.1), using a punctured filter in (3.7), which would make (3.7)(2) hold at isolated points, and assuming Filippov's theorem as a hypothesis of (4.9) or of the goal.
  • Needed infrastructure: the Clarke calculus for dEd_EdE​, a chain-rule-type estimate for dF∘xd_F\circ xdF​∘x along absolutely continuous curves, a Gronwall lemma for absolutely continuous functions (Mathlib has norm_le_gronwallBound_of_norm_deriv_right_le for the differentiable case), and Filippov's existence theorem. The generalized gradient, cones and trajectory notions are reusable by any nonsmooth-optimization or control mission. Contributions of any of these as separate lemmas are welcome.
  • The §1 definitions duplicate those of the companion mission Generalized Gradients and Applications I; the two missions were drafted at the same time.

Selected references

  • F. H. Clarke, Generalized gradients and applications, Trans. Amer. Math. Soc. 205 (1975), 247–262. https://doi.org/10.1090/s0002-9947-1975-0367131-6
  • A. F. Filippov, Classical solutions of differential equations with multivalued right-hand side, SIAM J. Control 5 (1967), 609–621. https://doi.org/10.1137/0305040
  • H. Brezis, On a characterization of flow-invariant sets, Comm. Pure Appl. Math. 23 (1970), 261–263. https://doi.org/10.1002/cpa.3160230211
  • J. M. Bony, Principe du maximum, inégalité de Harnack et unicité du problème de Cauchy pour les opérateurs elliptiques dégénérés, Ann. Inst. Fourier 19 (1969), 277–304. https://doi.org/10.5802/aif.319
  • R. M. Redheffer, The theorems of Bony and Brezis on flow-invariant sets, Amer. Math. Monthly 79 (1972), 740–747. MR 46 #2166.
  • F. H. Clarke, Yu. S. Ledyaev, R. J. Stern, P. R. Wolenski, Nonsmooth Analysis and Control Theory, Graduate Texts in Mathematics 178, Springer, 1998. https://doi.org/10.1007/b97650
16 thms4 active usersReviewed
🏆Completed
Mathematical Physics·Captain: Lucas

Dynamics and Relativity I: Kepler's Laws from the Inverse-Square LawTextbook

Motivation

The derivation of Kepler's three laws from Newton's inverse-square law of gravitation is the calculation with which mathematical physics begins. Kepler published the laws in 1609 and 1619 as empirical regularities distilled from Tycho Brahe's naked-eye astrometry: planets travel on ellipses with the Sun at a focus, the Sun–planet segment sweeps equal areas in equal times, and the square of the orbital period scales as the cube of the orbit's size. That an attractive force falling off as 1/r21/r^21/r2 forces exactly these three regularities was established by Newton in the Principia (1687); as David Tong remarks in the source notes, the third law was within reach of several of Newton's contemporaries, but the derivation of the first — that the orbit is a conic section with the centre of attraction at a focus — was Newton's alone.

This mission formalizes that derivation as it is presented in §4 (Central Forces) of David Tong's Dynamics and Relativity, the Cambridge Part IA Mathematical Tripos lecture notes (Lent 2013). It is the first mission in a series covering those notes, and it targets the chapter's capstone, Kepler's first law.

Setting

A point particle of mass m>0m>0m>0 moves in Euclidean 333-space along a trajectory x:R→R3x:\mathbb R\to\mathbb R^3x:R→R3, assumed twice continuously differentiable and never passing through the origin, where the field below is singular. Write r(t)=∥x(t)∥r(t)=\lVert x(t)\rVertr(t)=∥x(t)∥ and r^=x/r\hat r = x/rr^=x/r.

The particle moves in a central force field when it obeys Tong's equation of motion (4.1),

m x¨(t)  =  F(r(t)) r^(t),m\,\ddot x(t) \;=\; F\big(r(t)\big)\,\hat r(t),mx¨(t)=F(r(t))r^(t),

where F:R→RF:\mathbb R\to\mathbb RF:R→R gives the radial component of the force at distance rrr; for a force derived from a central potential V(r)V(r)V(r) one has F=− dV/drF=-\,dV/drF=−dV/dr. The Kepler problem (Tong §4.3.1) is the inverse-square special case, potential (4.12)

V(r)=−k mr,F(r)=−k mr2,V(r) = -\frac{k\,m}{r}, \qquad F(r) = -\frac{k\,m}{r^{2}},V(r)=−rkm​,F(r)=−r2km​,

with k=GM>0k=GM>0k=GM>0 for gravitational attraction by a mass MMM fixed at the origin. (The same equations govern the Coulomb problem, with k=−qQ/4πϵ0mk=-qQ/4\pi\epsilon_0 mk=−qQ/4πϵ0​m, which is repulsive when k<0k<0k<0; this mission fixes k>0k>0k>0.)

Two conserved quantities organize the problem. The angular momentum is the vector

L(t)  =  m x(t)×x˙(t),L(t) \;=\; m\,x(t)\times\dot x(t),L(t)=mx(t)×x˙(t),

and we write l=∥L∥/ml=\lVert L\rVert/ml=∥L∥/m for the angular momentum per unit mass, which in the plane polar coordinates of Tong (4.5) is l=r2θ˙l=r^{2}\dot\thetal=r2θ˙. The total energy of the Kepler problem is

E(t)  =  12m∥x˙(t)∥2  −  k mr(t).E(t) \;=\; \tfrac12 m\lVert \dot x(t)\rVert^{2} \;-\; \frac{k\,m}{r(t)} .E(t)=21​m∥x˙(t)∥2−r(t)km​.

Tong's solution (4.14) of the orbit equation states that the trajectory lies on a conic section with a focus at the origin. Coordinate-free, this says that there is a fixed eccentricity vector A∈R3A\in\mathbb R^3A∈R3, pointing towards the point of closest approach, with

r(t)+⟨A,x(t)⟩  =  r0for all t,r0=l2k,r(t) + \langle A, x(t)\rangle \;=\; r_0 \qquad\text{for all } t, \qquad r_0 = \frac{l^{2}}{k},r(t)+⟨A,x(t)⟩=r0​for all t,r0​=kl2​,

which is Tong's r=r0/(1+ecos⁡θ)r = r_0/(1+e\cos\theta)r=r0​/(1+ecosθ) with e=∥A∥e=\lVert A\rVerte=∥A∥ the eccentricity and θ\thetaθ measured from the direction of AAA, and r0r_0r0​ the semi-latus rectum.

Finally, a set S⊆R3S\subseteq\mathbb R^3S⊆R3 is an ellipse with a focus at the origin when there are a nonzero normal vector nnn, a second focus ccc with ⟨n,c⟩=0\langle n,c\rangle=0⟨n,c⟩=0, and a real number aaa with ∥c∥<2a\lVert c\rVert<2a∥c∥<2a, such that SSS is exactly the set of points yyy of the plane {y:⟨n,y⟩=0}\{y : \langle n,y\rangle = 0\}{y:⟨n,y⟩=0} satisfying the two-foci (string) property

∥y∥+∥y−c∥  =  2a.\lVert y\rVert + \lVert y - c\rVert \;=\; 2a .∥y∥+∥y−c∥=2a.

The inequality ∥c∥<2a\lVert c\rVert<2a∥c∥<2a forces a>0a>0a>0 and rules out the degenerate loci; c=0c=0c=0 gives a circle.

Target

The goal theorem is Kepler's first law, K1 of Tong §4.3.2: each planet moves in an ellipse, with the Sun at one focus. Formally, for a trajectory xxx of the attractive inverse-square problem with nonvanishing angular momentum and negative total energy,

k>0,L≠0,E<0  ⟹  ∃ S an ellipse with a focus at the origin such that x(t)∈S for all t.k>0,\quad L\neq 0,\quad E<0 \;\Longrightarrow\; \exists\,S \text{ an ellipse with a focus at the origin such that } x(t)\in S \text{ for all } t.k>0,L=0,E<0⟹∃S an ellipse with a focus at the origin such that x(t)∈S for all t.

The hypothesis L≠0L\neq0L=0 excludes purely radial free-fall, whose trajectory is a segment rather than an ellipse, and E<0E<0E<0 is the bounded regime, which by Tong (4.16) is exactly e<1e<1e<1.

The milestones follow the source in order: conservation of angular momentum and the resulting planarity of the motion (§4, p. 48); Kepler's second law (K2), which holds for any central force; conservation of energy; the conic orbit (4.14); the identification of a conic of eccentricity e<1e<1e<1 as an ellipse with a focus at the origin (4.15); the energy–eccentricity relation (4.16); and Kepler's third law (K3, p. 62),

T  =  2π R3/2GM,R=12(rmin⁡+rmax⁡).T \;=\; \frac{2\pi\,R^{3/2}}{\sqrt{GM}}, \qquad R = \tfrac12\big(r_{\min}+r_{\max}\big).T=GM​2πR3/2​,R=21​(rmin​+rmax​).

Significance

The result. K1 is the statement that fixes the inverse-square law among all central force laws: K2 holds for every central force, and dimensional analysis alone gives the shape of K3, but closed orbits that are exact ellipses with the attracting centre at a focus are special to the 1/r21/r^21/r2 force (and to the harmonic force, with the centre at the centre of the ellipse). Everything quantitative in classical celestial mechanics — orbit determination, the mass–period relation used to weigh binary stars, and the perturbative treatment of precession that eventually exposed the anomaly in Mercury's orbit — is built on the Kepler solution.

Formalizing it. The mathematics has been settled for three centuries; what this mission produces is a machine-checked treatment of the Newtonian derivation, starting from the differential equation rather than from a prescribed orbit. Mathlib has no central-force or two-body library, so the definitions here — central force motion, angular momentum, conserved energy, swept area, and the two-foci characterization of an ellipse — are new and reusable across the rest of the series and by any later mission in classical mechanics. The existing platform mission Celestial Mechanics I: Binary Star Systems runs the complementary direction, verifying that a prescribed conic orbit satisfies the equations of motion and reducing the two-body problem to a one-body problem; the present mission asks for the converse, which is the harder half and the one Newton is credited with.

Difficulty

The derivation in the source changes the independent variable from time ttt to the polar angle θ\thetaθ, substitutes u=1/ru=1/ru=1/r, and solves the resulting linear oscillator equation (4.11). Each of those steps needs infrastructure that does not exist in Mathlib. There is no polar-angle function attached to a curve in the plane: one has to produce a continuously differentiable angle θ(t)\theta(t)θ(t) lifting the trajectory, show θ˙=l/r2\dot\theta = l/r^{2}θ˙=l/r2 never vanishes so that θ\thetaθ is a legitimate change of variable, and then transport a second-order ODE through that reparametrization. Solving (4.11) then requires a uniqueness theorem for the inhomogeneous harmonic oscillator, and reading the conic back into Cartesian form requires the algebra of (4.15).

Kepler's third law carries an extra burden beyond the formula: the statement quantifies over the least period of the motion, so a solution must establish that a negative-energy orbit is periodic at all, and must identify inf⁡tr(t)\inf_t r(t)inft​r(t) and sup⁡tr(t)\sup_t r(t)supt​r(t) as the periapsis and apoapsis distances r0/(1±e)r_0/(1\pm e)r0​/(1±e).

Formalization scope

Trajectories are maps ℝ → EuclideanSpace ℝ (Fin 3), with ContDiff ℝ 2 smoothness and the standing hypothesis x(t)≠0x(t)\neq0x(t)=0 for all ttt built into the definition of central force motion, together with m>0m>0m>0. Time is all of R\mathbb RR: the trajectories considered are globally defined, so collision orbits are excluded by hypothesis rather than by a maximal-interval argument. Derivatives are Mathlib's deriv, so velocity and acceleration are the first and second derivatives of the trajectory.

Three conventions are worth flagging. First, the force is given by its radial profile FFF rather than by a potential, avoiding any appeal to the junk value of a derivative at 000; the Kepler case is the instance F(r)=−km/r2F(r)=-km/r^{2}F(r)=−km/r2. Second, the swept area of K2 is the integral 12∫∥x×x˙∥ dt\tfrac12\int\lVert x\times\dot x\rVert\,dt21​∫∥x×x˙∥dt, which is the standard area element 12r2 dθ\tfrac12 r^2\,d\theta21​r2dθ of the source written invariantly; the content of K2 is that this integrand is constant in time, so that the area depends on t1−t0t_1-t_0t1​−t0​ alone. Third, K1 asserts that the trajectory is contained in an ellipse; it does not assert that the particle traverses the whole ellipse, which is a separate (true) statement about periodic orbits.

The degenerate readings are ruled out explicitly. The eccentricity vector formulation of the conic is an equation holding at every time, not an existence claim at one time; the ellipse predicate carries ∥c∥<2a\lVert c\rVert<2a∥c∥<2a, so a point or a segment does not qualify; and the hypothesis L≠0L\neq0L=0 in K1 and in the conic milestone is necessary, since a radial trajectory satisfies every other hypothesis but lies on no ellipse.

Contributions of independent interest are welcome: a plane polar coordinate API for curves, uniqueness for the forced harmonic oscillator, and the classification of the loci r+⟨A,x⟩=r0r+\langle A,x\rangle = r_0r+⟨A,x⟩=r0​ into ellipse, parabola and hyperbola according to ∥A∥<1\lVert A\rVert<1∥A∥<1, =1=1=1, >1>1>1 (this mission needs only the first case) would all be reusable well beyond this series.

Selected references

  • David Tong, Dynamics and Relativity, University of Cambridge Part IA Mathematical Tripos lecture notes, Lent 2013, §4 (Central Forces), pp. 48–62. http://www.damtp.cam.ac.uk/user/tong/relativity.html
  • Isaac Newton, Philosophiæ Naturalis Principia Mathematica, 1687, Book I, Propositions XI–XVII.
  • S. Chandrasekhar, Newton's Principia for the Common Reader, Oxford University Press, 1995.
21 thms4 active usersReviewed
🏆Completed
Mathematical PhysicsNumber Theory·Captain: Lucas

Ablowitz–Chakravarty–Halburd: the Chazy–Ramanujan correspondence and the Darboux–Halphen reduction of self-dual Yang–MillsResearch Paper

Motivation

In 1985 R. S. Ward conjectured that "many (and perhaps all?) of the ordinary and partial differential equations that are regarded as being integrable or solvable may be obtained from the self-dual gauge field equations (or its generalizations) by reduction". The self-dual Yang–Mills (SDYM) equations are therefore often called the master integrable system: choosing a gauge algebra and a symmetry group to reduce by produces, on the one hand, the classical soliton equations and the Painlevé transcendents, and on the other — once infinite-dimensional gauge algebras are allowed — a family of third-order equations whose solutions have movable natural barriers and are therefore not of Painlevé type.

This mission formalizes the endpoint of one such reduction chain, as surveyed by Ablowitz, Chakravarty and Halburd, Integrable systems and reductions of the self-dual Yang–Mills equations, J. Math. Phys. 44 (2003) 3147–3173, Section V. Reducing SDYM to functions of a single variable gives the Nahm equations; taking the gauge algebra to be the divergence-free vector fields on S3S^3S3 turns them into a matrix flow which, after diagonalizing the symmetric part, becomes the generalized Darboux–Halphen system. Its trace is governed by the Chazy equation, written down by Chazy in 1909, and — this is the paper's historical observation — the Chazy equation is equivalent to the differential system Ramanujan derived in 1916 for the Eisenstein series P=E2P = E_2P=E2​, Q=E4Q = E_4Q=E4​, R=E6R = E_6R=E6​. Chazy and Ramanujan worked on the same equation at nearly the same time and apparently did not know it.

Setting

Throughout, ttt and qqq are complex variables and all functions are complex-valued; a "solution on sss" means the stated derivative identities hold at every point of a set s⊆Cs \subseteq \mathbb{C}s⊆C.

The classical Chazy equation is the third-order equation

d3ydt3=2y d2ydt2−3(dydt)2.\frac{d^3y}{dt^3} = 2y\,\frac{d^2y}{dt^2} - 3\left(\frac{dy}{dt}\right)^2 .dt3d3y​=2ydt2d2y​−3(dtdy​)2.

The classical Darboux–Halphen system is the first-order system for ω1,ω2,ω3\omega_1,\omega_2,\omega_3ω1​,ω2​,ω3​

ω˙1=ω2ω3−ω1(ω2+ω3),\dot\omega_1 = \omega_2\omega_3 - \omega_1(\omega_2+\omega_3),ω˙1​=ω2​ω3​−ω1​(ω2​+ω3​),

together with its two cyclic images. It arose in Darboux's study of triply orthogonal surfaces and was solved by Halphen. Its generalized form adds a term τ2=τ12+τ22+τ32\tau^2 = \tau_1^2+\tau_2^2+\tau_3^2τ2=τ12​+τ22​+τ32​ to each right-hand side, where τ˙1=−τ1(ω2+ω3)\dot\tau_1 = -\tau_1(\omega_2+\omega_3)τ˙1​=−τ1​(ω2​+ω3​) and cyclically.

Ramanujan's system is

qdPdq=P2−Q12,qdQdq=PQ−R3,qdRdq=PR−Q22,q\frac{dP}{dq} = \frac{P^2-Q}{12},\qquad q\frac{dQ}{dq} = \frac{PQ-R}{3},\qquad q\frac{dR}{dq} = \frac{PR-Q^2}{2},qdqdP​=12P2−Q​,qdqdQ​=3PQ−R​,qdqdR​=2PR−Q2​,

satisfied by P(q)=1−24∑n≥1σ1(n)qnP(q) = 1-24\sum_{n\ge1}\sigma_1(n)q^nP(q)=1−24∑n≥1​σ1​(n)qn, Q(q)=1+240∑n≥1σ3(n)qnQ(q) = 1+240\sum_{n\ge1}\sigma_3(n)q^nQ(q)=1+240∑n≥1​σ3​(n)qn, R(q)=1−504∑n≥1σ5(n)qnR(q) = 1-504\sum_{n\ge1}\sigma_5(n)q^nR(q)=1−504∑n≥1​σ5​(n)qn, where σk(n)=∑d∣ndk\sigma_k(n)=\sum_{d\mid n}d^kσk​(n)=∑d∣n​dk.

Finally, the 3×33\times33×3 matrix flow obtained from the Nahm equations with the diff(S3)\mathrm{diff}(S^3)diff(S3) gauge algebra is

M˙=(Adj⁡M)T+MTM−(Tr⁡M)M,Adj⁡M=(det⁡M)M−1,\dot M = (\operatorname{Adj} M)^{T} + M^{T}M - (\operatorname{Tr} M)M,\qquad \operatorname{Adj}M = (\det M)M^{-1},M˙=(AdjM)T+MTM−(TrM)M,AdjM=(detM)M−1,

and the generalized Chazy equation with parameter nnn is

d3ydt3−2yd2ydt2+3(dydt)2=436−n2(6dydt−y2)2.\frac{d^3y}{dt^3} - 2y\frac{d^2y}{dt^2} + 3\left(\frac{dy}{dt}\right)^2 = \frac{4}{36-n^2}\left(6\frac{dy}{dt}-y^2\right)^2 .dt3d3y​−2ydt2d2y​+3(dtdy​)2=36−n24​(6dtdy​−y2)2.

Formalization targets

Goal — the Chazy–Ramanujan correspondence (eqs. (78) and (71))

If P,Q,RP,Q,RP,Q,R satisfy Ramanujan's system on a region of the punctured qqq-plane, then

y(t):=iπP ⁣(e2πit)y(t) := i\pi P\!\left(e^{2\pi i t}\right)y(t):=iπP(e2πit)

satisfies the classical Chazy equation on the preimage region. In particular y(t)=iπE2(t)y(t)=i\pi E_2(t)y(t)=iπE2​(t) is a solution of the Chazy equation, and knowing the general solution of Chazy gives the general solution of Ramanujan's system.

Supporting targets

The milestone list covers the reduction chain in both directions: the matrix flow M˙=(Adj⁡M)T+MTM−(Tr⁡M)M\dot M = (\operatorname{Adj}M)^T + M^TM-(\operatorname{Tr}M)MM˙=(AdjM)T+MTM−(TrM)M and its reduction to the Darboux–Halphen system (eqs. (51)–(54)), the first integrals (55), the passage y=−2(ω1+ω2+ω3)y = -2(\omega_1+\omega_2+\omega_3)y=−2(ω1​+ω2​+ω3​) from Darboux–Halphen to Chazy and back through the roots of a cubic, the SL(2)\mathrm{SL}(2)SL(2) symmetry (73) of the Chazy equation, Rankin's fourth-order equation for the discriminant cusp form, the change of variable q=e2iτq=e^{2i\tau}q=e2iτ between the two forms of Ramanujan's system, and the generalized Chazy equation (81).

Significance

The Chazy equation is the bridge between integrable systems and the theory of modular forms. Its particular solution y=iπE2y = i\pi E_2y=iπE2​ makes the quasi-modularity of the second Eisenstein series an ODE statement; via y=12(log⁡Δ)′y = \tfrac12 (\log\Delta)'y=21​(logΔ)′ it turns into Rankin's homogeneous fourth-order equation for the discriminant cusp form Δ\DeltaΔ, whose Fourier coefficients are the Ramanujan τ\tauτ-function. The SL(2,Z)\mathrm{SL}(2,\mathbb{Z})SL(2,Z) action on solutions is exactly the weight-2 quasi-modular transformation law. In the other direction, the general solution of Chazy is a ratio of hypergeometric functions with a movable natural barrier, which is why these reductions are used as the standard counterexample to the identification of integrability with the Painlevé property.

None of this material is currently in Mathlib: there is no Chazy equation, no Darboux–Halphen system, no Ramanujan differential system, and no Eisenstein-series ODE. The mission builds that layer from scratch. Each statement is a closed-form differential identity, so the development is self-contained: it needs no analytic continuation theory, no modular-forms library, and no existence theory for ODEs. What a solver must supply is careful derivative bookkeeping and polynomial algebra.

Status honesty: every statement in this mission is a classical, published result — Darboux, Halphen, Chazy (1909–1911), Ramanujan (1916), Rankin (1956), and Ablowitz–Chakravarty–Halburd (1990s–2003). Nothing here is open mathematics. What is open is the machine-checked proof; to the captain's knowledge no formalization of these identities exists.

Difficulty

The obvious approach — "differentiate three times and call ring" — fails for two reasons. First, the statements are about functions, not about polynomials: each differentiation step requires producing the derivative of a product, a quotient, or a composition from the hypotheses, and only then is the resulting algebraic identity a ring problem. Second, two of the targets go against the flow of the hypotheses. Recovering the Darboux–Halphen system from a Chazy solution means recovering ω˙i\dot\omega_iω˙i​ from the derivatives of the three elementary symmetric functions of the ωi\omega_iωi​: this is a linear system whose matrix is a Vandermonde matrix in ω1,ω2,ω3\omega_1,\omega_2,\omega_3ω1​,ω2​,ω3​, invertible precisely because the roots are assumed distinct — which is why the distinctness hypothesis is not decoration. Similarly, the matrix milestone needs the conjugation-equivariance of M↦(Adj⁡M)T+MTM−(Tr⁡M)MM \mapsto (\operatorname{Adj}M)^T + M^TM - (\operatorname{Tr}M)MM↦(AdjM)T+MTM−(TrM)M, which holds for the transpose only because the conjugating matrix is complex orthogonal.

Formalization scope

Everything is over C\mathbb{C}C, matching the paper. Solutions are represented pointwise on an arbitrary set s⊆Cs \subseteq \mathbb{C}s⊆C rather than on all of C\mathbb{C}C, because the solutions of interest have movable singularities and natural barriers; no openness, holomorphy or connectivity is assumed unless a statement needs it.

Higher derivatives are carried as explicit extra function arguments joined by HasDerivAt hypotheses rather than through iterated deriv. This avoids junk values entirely: a statement never asserts anything about the value of a derivative that does not exist. The same convention is used for the matrix flow, where the derivative is imposed entrywise, so that no norm or normed-space structure on the space of matrices needs to be chosen.

Divisions are arranged so that no denominator can vanish under the stated hypotheses: Ramanujan's system is written in the form q dP/dq=(P2−Q)/12q\,dP/dq = (P^2-Q)/12qdP/dq=(P2−Q)/12, with no division by qqq; the generalized Chazy equation carries the hypothesis n2≠36n^2 \ne 36n2=36; and the first-integral and discriminant statements carry explicit nonvanishing hypotheses.

There is no trivializing formalization available here. Every statement is an implication between two systems of differential equations whose hypotheses are satisfied by the classical explicit solutions (P=E2P=E_2P=E2​, Q=E4Q=E_4Q=E4​, R=E6R=E_6R=E6​ for the Ramanujan system; Halphen's solutions for Darboux–Halphen), so none of them is vacuous, and none is an identity that holds for arbitrary functions.

A complete development needs only Mathlib's derivative calculus (HasDerivAt and its product, quotient and composition rules), Complex.exp, and Matrix.adjugate with the basic adjugate identities. Contributions of reusable pieces are welcome: in particular a clean statement of the derivative of the elementary symmetric functions of a triple of functions, and the Vandermonde inversion step, would both be of use beyond this mission.

Selected references

  • M. J. Ablowitz, S. Chakravarty, R. G. Halburd, Integrable systems and reductions of the self-dual Yang–Mills equations, J. Math. Phys. 44 (2003) 3147–3173. doi:10.1063/1.1586967
  • J. Chazy, Sur les équations différentielles du troisième ordre et d'ordre supérieur dont l'intégrale générale a ses points critiques fixes, Acta Math. 34 (1911) 317–385. doi:10.1007/BF02393131
  • S. Ramanujan, On certain arithmetical functions, Trans. Cambridge Philos. Soc. 22 (1916) 159–184.
  • G. Halphen, Sur un système d'équations différentielles, C. R. Acad. Sci. Paris 92 (1881) 1101–1103.
  • R. A. Rankin, The construction of automorphic forms from the derivatives of a given form, J. Indian Math. Soc. 20 (1956) 103–116.
  • M. J. Ablowitz, S. Chakravarty, R. G. Halburd, The generalized Chazy equation and Schwarzian triangle functions, Asian J. Math. 2 (1998) 619–624. doi:10.4310/AJM.1998.v2.n4.a1
  • R. S. Ward, Integrable and solvable systems, and relations among them, Philos. Trans. R. Soc. London A 315 (1985) 451–457. doi:10.1098/rsta.1985.0051
15 thms4 active usersReviewed
🏆Completed
Linear algebra·Captain: mikedeng1

Sub/Super-Stochastic Matrix with Applications to Bipartite Tracking Control over Signed Networks 2: Chains to the Rows of Sum ≥ 1 of a Super-Stochastic F Give ρ(F) < 1 under Condition (2.5)Research Paper

Why spectral bounds for super-stochastic matrices

Discrete-time linear iterations x(k+1)=Fx(k)x(k+1)=Fx(k)x(k+1)=Fx(k) with entrywise nonnegative FFF appear wherever a quantity is redistributed along the edges of a network: consensus and tracking protocols for multi-agent systems, positive switched systems, and Markov-type averaging schemes. Such an iteration converges to zero exactly when the spectral radius ρ(F)\rho(F)ρ(F), the largest modulus of an eigenvalue, is less than one. For a sub-stochastic matrix (all row sums at most one) the classical bound ρ(F)≤∥F∥∞≤1\rho(F)\le\|F\|_\infty\le 1ρ(F)≤∥F∥∞​≤1 settles marginal stability, and strict stability needs only one row of sum below one that "reaches" the others.

Shi, Zheng, Shao and Cheng (arXiv:2004.01867v2, SIAM J. Control Optim. 2021) study bipartite tracking over signed networks, where asynchronous updates and larger step sizes produce iteration matrices with some row sums above one. For these matrices the norm bound is useless: ∥F∥∞>1\|F\|_\infty>1∥F∥∞​>1. Their §2.2 introduces super-stochastic matrices and gives conditions under which the spectral radius is nevertheless below one. These conditions are what the later sections use to allow larger control gains than a sub-stochastic analysis permits. This mission formalizes the main spectral result for a single super-stochastic matrix, Theorem 2.6, together with its companions Theorem 2.7 and Theorem 2.8 (products).

Setting

Let F=[fij]F=[f_{ij}]F=[fij​] be a real n×nn\times nn×n matrix. Its iii-th row sum is Λi[F]=∑j=1nfij\Lambda_i[F]=\sum_{j=1}^n f_{ij}Λi​[F]=∑j=1n​fij​, and its infinity norm is ∥F∥∞=max⁡i∑j∣fij∣\|F\|_\infty=\max_i\sum_j|f_{ij}|∥F∥∞​=maxi​∑j​∣fij​∣. The eigenvalues of FFF are complex numbers, and ρ(F)\rho(F)ρ(F) is the largest of their moduli.

FFF is super-stochastic (Definition 2.5) if fij≥0f_{ij}\ge0fij​≥0 for all i,ji,ji,j and there is a set W1\mathcal W_1W1​ of rows with Λi[F]≥1\Lambda_i[F]\ge1Λi​[F]≥1 for i∈W1i\in\mathcal W_1i∈W1​ and Λi[F]<1\Lambda_i[F]<1Λi​[F]<1 otherwise. The rows split into

R1={s∣Λs[F]<1},R2={s∣Λs[F]≥1}.\mathcal R_1=\{s\mid\Lambda_s[F]<1\},\qquad \mathcal R_2=\{s\mid\Lambda_s[F]\ge1\}.R1​={s∣Λs​[F]<1},R2​={s∣Λs​[F]≥1}.

A non-zero element chain from row iii to row kkk is a list of entries Ci→k=[F]kir,…,[F]i2i1,[F]i1i\mathcal C_{i\to k}=[F]_{ki_r},\dots,[F]_{i_2i_1},[F]_{i_1i}Ci→k​=[F]kir​​,…,[F]i2​i1​​,[F]i1​i​, all non-zero, with consecutive indices distinct; ∣Ci→k∣=r+1|\mathcal C_{i\to k}|=r+1∣Ci→k​∣=r+1 is its number of elements. The theorem assumes one chain Ci→k\mathcal C_{i\to k}Ci→k​ from some i∈R1i\in\mathcal R_1i∈R1​ for each k∈R2k\in\mathcal R_2k∈R2​, and ∣C∗∣|\mathcal C^*|∣C∗∣ is the largest number of elements among these chains. The constants are

α1=max⁡{Λs[F]∣Λs[F]<1},α2=min⁡{[F]ij∣[F]ij>0},α3=max⁡{Λs[F]∣Λs[F]≥1}.\alpha_1=\max\{\Lambda_s[F]\mid\Lambda_s[F]<1\},\quad \alpha_2=\min\{[F]_{ij}\mid[F]_{ij}>0\},\quad \alpha_3=\max\{\Lambda_s[F]\mid\Lambda_s[F]\ge1\}.α1​=max{Λs​[F]∣Λs​[F]<1},α2​=min{[F]ij​∣[F]ij​>0},α3​=max{Λs​[F]∣Λs​[F]≥1}.

For a sequence F1,…,FqF_1,\dots,F_qF1​,…,Fq​, the product is ordered with later factors on the left, ∏s=1qFs=Fq⋯F1\prod_{s=1}^qF_s=F_q\cdots F_1∏s=1q​Fs​=Fq​⋯F1​.

Formalization targets

Goal: Theorem 2.6 (p. 9)

If R1\mathcal R_1R1​ and R2\mathcal R_2R2​ are non-empty, the chains exist, and

α3∣C∗∣+1−(α3−α1)α2∣C∗∣∣C∗∣<1,(2.5)\sqrt[|\mathcal C^*|]{\alpha_3^{|\mathcal C^*|+1}-(\alpha_3-\alpha_1)\alpha_2^{|\mathcal C^*|}}<1, \tag{2.5}∣C∗∣α3∣C∗∣+1​−(α3​−α1​)α2∣C∗∣​​<1,(2.5)

then

ρ(F)<1.\rho(F)<1 .ρ(F)<1.

Milestones (the steps of the proof on p. 9)

  1. At the end kkk of a chain of r+1r+1r+1 elements starting in R1\mathcal R_1R1​: Λk[Fr+2]≤α3r+2−(α3−α1)α2r+1\Lambda_k[F^{r+2}]\le\alpha_3^{r+2}-(\alpha_3-\alpha_1)\alpha_2^{r+1}Λk​[Fr+2]≤α3r+2​−(α3​−α1​)α2r+1​.
  2. For every k∈R2k\in\mathcal R_2k∈R2​: Λk[F∣C∗∣+1]≤α3∣C∗∣+1−(α3−α1)α2∣C∗∣<1\Lambda_k[F^{|\mathcal C^*|+1}]\le\alpha_3^{|\mathcal C^*|+1}-(\alpha_3-\alpha_1)\alpha_2^{|\mathcal C^*|}<1Λk​[F∣C∗∣+1]≤α3∣C∗∣+1​−(α3​−α1​)α2∣C∗∣​<1.
  3. The same bound for every i∈R1i\in\mathcal R_1i∈R1​.
  4. ∥F∣C∗∣+1∥∞≤α3∣C∗∣+1−(α3−α1)α2∣C∗∣<1\|F^{|\mathcal C^*|+1}\|_\infty\le\alpha_3^{|\mathcal C^*|+1}-(\alpha_3-\alpha_1)\alpha_2^{|\mathcal C^*|}<1∥F∣C∗∣+1∥∞​≤α3∣C∗∣+1​−(α3​−α1​)α2∣C∗∣​<1.
  5. The general fact ∣μ∣m≤∥Am∥∞|\mu|^m\le\|A^m\|_\infty∣μ∣m≤∥Am∥∞​ for every eigenvalue μ\muμ of a real square matrix AAA (quoted on p. 5).

Companion results

  • Theorem 2.7 (p. 9). If [F]ii=0[F]_{ii}=0[F]ii​=0 for all iii and R2\mathcal R_2R2​ has exactly one element, then ρ(F)≤α1α3\rho(F)\le\sqrt{\alpha_1\alpha_3}ρ(F)≤α1​α3​​.
  • Theorem 2.8 (pp. 10–11), pinned reading. For super-stochastic F1,…,FqF_1,\dots,F_qF1​,…,Fq​ with positive diagonals, with ggg the largest row sum above one, ccc the largest row sum below one and φ\varphiφ the least positive entry, the condition gq−(g−c)φq−1<1g^q-(g-c)\varphi^{q-1}<1gq−(g−c)φq−1<1 gives Λi2[∏sFs]<1\Lambda_{i_2}[\prod_sF_s]<1Λi2​​[∏s​Fs​]<1 for every row i2i_2i2​ fed by an earlier deficient row, and ρ(∏sFs)<1\rho(\prod_sF_s)<1ρ(∏s​Fs​)<1 if every row is so fed.

Significance

Theorem 2.6 is a stability criterion that applies when the standard norm test fails because some row sums exceed one. The criterion is computable: it needs only the row sums, the smallest positive entry and the lengths of paths in the directed graph of FFF from rows of sum below one to the remaining rows. Theorem 2.8 extends this to products, which is the form needed for switched systems in which every subsystem may be individually unstable (ρ(Fs)>1\rho(F_s)>1ρ(Fs​)>1) while the periodic product is stable. The paper's tracking results for second-order and general linear agents (§§5–13) use these product bounds.

On the formal side, Mathlib has the spectrum of a matrix, Gershgorin's circle theorem and the norm on matrices, but no bounds on the spectral radius in terms of row sums along paths, and nothing on sub- or super-stochastic matrices. None of these results has a machine-checked proof. The mission produces the first formal version of these chain arguments and a reusable eigenvalue–norm inequality for real matrices.

Difficulty

The obvious argument — ρ(F)≤∥F∥∞\rho(F)\le\|F\|_\inftyρ(F)≤∥F∥∞​ — fails at once, because ∥F∥∞=α3≥1\|F\|_\infty=\alpha_3\ge1∥F∥∞​=α3​≥1. One has to pass to a power FmF^{m}Fm and show that every row sum of FmF^mFm drops below one. For the rows of R2\mathcal R_2R2​ the drop comes only from the deficit of a row of R1\mathcal R_1R1​, transported along a chain whose length varies from row to row; the bound must be made uniform in the length, which is where ∣C∗∣|\mathcal C^*|∣C∗∣ and the exponents ∣C∗∣+1|\mathcal C^*|+1∣C∗∣+1 enter, and where the growth factor α3>1\alpha_3>1α3​>1 competes with the gain (α3−α1)α2∣C∗∣(\alpha_3-\alpha_1)\alpha_2^{|\mathcal C^*|}(α3​−α1​)α2∣C∗∣​. Formally, the induction mixes matrix powers, row sums of products and the indexing of chains. The final step needs the inequality between eigenvalues of FFF, viewed over C\mathbb CC, and the infinity norm of a power of FFF, which is not available in Mathlib in this form.

Formalization scope

All statements live in the namespace SubSuperStoch.SuperSpec. Matrices are Matrix (Fin n) (Fin n) ℝ with 000-based indices. Row sums, the infinity norm (a supremum over Fin n, equal to 000 when n=0n=0n=0), the complex spectrum, chains, super-stochasticity and ordered products are defined in one definitions file. Committed conventions:

  • Eigenvalues are the spectrum of FFF as a complex matrix; ρ(F)<1\rho(F)<1ρ(F)<1 is stated as ∣μ∣<1|\mu|<1∣μ∣<1 for every eigenvalue μ\muμ. A real spectrum or a real supremum of eigenvalue moduli is not used, since either would miss complex eigenvalues or take a junk value.
  • Extrema α1,α2,α3,∣C∗∣,g,c,φ\alpha_1,\alpha_2,\alpha_3,|\mathcal C^*|,g,c,\varphiα1​,α2​,α3​,∣C∗∣,g,c,φ are values with hypotheses that they are the greatest or least element of exactly the printed set, so no default value for an empty set can enter.
  • Chains are given by a length function on rows; ∣C∗∣|\mathcal C^*|∣C∗∣ is its greatest value over R2\mathcal R_2R2​, the chains being the ones the hypothesis supplies.
  • The root in (2.5) is the real power x1/∣C∗∣x^{1/|\mathcal C^*|}x1/∣C∗∣. Its radicand is nonnegative under the hypotheses, and (2.5) is equivalent to α3∣C∗∣+1−(α3−α1)α2∣C∗∣<1\alpha_3^{|\mathcal C^*|+1}-(\alpha_3-\alpha_1)\alpha_2^{|\mathcal C^*|}<1α3∣C∗∣+1​−(α3​−α1​)α2∣C∗∣​<1.
  • W1⊂{1,…,n}\mathcal W_1\subset\{1,\dots,n\}W1​⊂{1,…,n} in Definition 2.5 is read as a subset that may be all rows, as the paper's own examples require; condition 2) then fixes vocabulary only.
  • Theorem 2.8 is stated with φ\varphiφ the least positive entry, as in Lemma 5.4 (p. 25). The printed φ\varphiφ, the least entry overall, is 000 for any family with a zero entry and turns (2.7) into gq<1g^q<1gq<1, which is impossible as g>1g>1g>1; that version would be vacuous. The row claim and the spectral claim carry the quantifiers their proofs need (the existence hypothesis for one row, respectively for every row).
  • The proof of Theorem 2.6 ends with ρ(F)≤⋯∣C∗∣\rho(F)\le\sqrt[|\mathcal C^*|]{\cdots}ρ(F)≤∣C∗∣⋯​, a ∣C∗∣|\mathcal C^*|∣C∗∣-th root of a bound on ∥F∣C∗∣+1∥∞\|F^{|\mathcal C^*|+1}\|_\infty∥F∣C∗∣+1∥∞​. That display is not a target; the theorem claims only ρ(F)<1\rho(F)<1ρ(F)<1.
  • Theorem 2.7 claims no "<1<1<1" and none is added.

A trivializing formalization is excluded: the hypotheses are satisfied by the paper's own example F~\tilde FF~ on p. 9 (α1=0.6\alpha_1=0.6α1​=0.6, α2=0.5\alpha_2=0.5α2​=0.5, α3=1.05\alpha_3=1.05α3​=1.05, ∣C∗∣=1|\mathcal C^*|=1∣C∗∣=1, radicand 0.87750.87750.8775), checked in Lean.

Contributions welcome: proofs of the milestones, the general eigenvalue–norm inequality (reusable for any spectral-radius argument over real matrices), and the two companion theorems.

Selected references

  • L. Shi, W. X. Zheng, J. Shao, Y. Cheng, Sub/super-stochastic matrix with applications to bipartite tracking control over signed networks, arXiv:2004.01867v2, 2020; SIAM J. Control Optim., 2021. https://arxiv.org/abs/2004.01867 , https://doi.org/10.1137/19M1239982
  • R. A. Horn, C. R. Johnson, Matrix Analysis, 2nd ed., Cambridge University Press, 2013 (spectral radius and induced norms, Ch. 5). https://doi.org/10.1017/CBO9781139020411
8 thms3 active usersReviewed
🏆Completed
Convex OptimizationOptimization·Captain: mikedeng1

Understanding the Acceleration Phenomenon via High-Resolution Differential Equations IV: The High-Resolution Heavy-Ball ODE Satisfies f(X(t)) − f(x⋆) ≤ 7‖x₀ − x⋆‖² e^(−√μ t/4)/(2s)Research Paper

Why a high-resolution ODE for the heavy-ball method

Polyak's heavy-ball method and Nesterov's accelerated gradient method for strongly convex functions (NAG-SC) are the two classical momentum methods of smooth convex optimization. With step size sss and momentum α=(1−μs)/(1+μs)\alpha=(1-\sqrt{\mu s})/(1+\sqrt{\mu s})α=(1−μs​)/(1+μs​) they differ only by a gradient-correction term, yet NAG-SC is accelerated for every μ\muμ-strongly convex, LLL-smooth objective while the heavy-ball method can fail to converge outside the quadratic case (Lessard, Recht & Packard 2016). The classical continuous-time limit cannot see this difference: as s→0s\to0s→0 both methods converge to the same low-resolution ODE X¨+2μ X˙+∇f(X)=0\ddot X+2\sqrt\mu\,\dot X+\nabla f(X)=0X¨+2μ​X˙+∇f(X)=0 (Su, Boyd & Candès 2016 treat the convex analogue).

Shi, Du, Jordan & Su (2018/2021) keep the O(s)O(\sqrt s)O(s​) terms of the limit and obtain high-resolution ODEs, one per method, which do distinguish the two algorithms. This mission formalizes their convergence theorem for the high-resolution ODE of the heavy-ball method (Theorem 2 of arXiv:1810.08907v3). A companion mission in the same series treats the high-resolution ODE of NAG-SC (Theorem 1), whose only difference is a Hessian-driven damping term s ∇2f(X)X˙\sqrt s\,\nabla^2 f(X)\dot Xs​∇2f(X)X˙.

Setting

Work in Rn\mathbb R^nRn with the standard inner product ⟨⋅,⋅⟩\langle\cdot,\cdot\rangle⟨⋅,⋅⟩ and Euclidean norm ∥⋅∥\|\cdot\|∥⋅∥. A function f:Rn→Rf:\mathbb R^n\to\mathbb Rf:Rn→R belongs to FL1(Rn)\mathcal F^1_L(\mathbb R^n)FL1​(Rn) if L>0L>0L>0, fff is differentiable and convex in the form f(y)≥f(x)+⟨∇f(x),y−x⟩f(y)\ge f(x)+\langle\nabla f(x),y-x\ranglef(y)≥f(x)+⟨∇f(x),y−x⟩, and ∥∇f(x)−∇f(y)∥≤L∥x−y∥\|\nabla f(x)-\nabla f(y)\|\le L\|x-y\|∥∇f(x)−∇f(y)∥≤L∥x−y∥ for all x,yx,yx,y. It belongs to Sμ,L1(Rn)\mathcal S^1_{\mu,L}(\mathbb R^n)Sμ,L1​(Rn) if moreover 0<μ≤L0<\mu\le L0<μ≤L and

f(y)≥f(x)+⟨∇f(x),y−x⟩+μ2∥y−x∥2for all x,y,f(y)\ge f(x)+\langle\nabla f(x),y-x\rangle+\frac\mu2\|y-x\|^2\quad\text{for all }x,y,f(y)≥f(x)+⟨∇f(x),y−x⟩+2μ​∥y−x∥2for all x,y,

and to Sμ,L2(Rn)\mathcal S^2_{\mu,L}(\mathbb R^n)Sμ,L2​(Rn) if in addition fff is twice continuously differentiable with a Lipschitz-continuous Hessian. Write x⋆x^\starx⋆ for the minimizer of fff.

Given a step size s>0s>0s>0 and an initial point x0x_0x0​, the high-resolution ODE of the heavy-ball method is

X¨(t)+2μ X˙(t)+(1+μs)∇f(X(t))=0,X(0)=x0,X˙(0)=−2s ∇f(x0)1+μs,(1.10)\ddot X(t)+2\sqrt\mu\,\dot X(t)+\bigl(1+\sqrt{\mu s}\bigr)\nabla f(X(t))=0,\qquad X(0)=x_0,\quad \dot X(0)=-\frac{2\sqrt s\,\nabla f(x_0)}{1+\sqrt{\mu s}},\tag{1.10}X¨(t)+2μ​X˙(t)+(1+μs​)∇f(X(t))=0,X(0)=x0​,X˙(0)=−1+μs​2s​∇f(x0​)​,(1.10)

for t≥0t\ge0t≥0. The initial velocity matches the first step of the discrete method. The paper's Lyapunov function is

E(t)=(1+μs)(f(X)−f(x⋆))+14∥X˙∥2+14∥X˙+2μ (X−x⋆)∥2.(3.3)\mathcal E(t)=\bigl(1+\sqrt{\mu s}\bigr)\bigl(f(X)-f(x^\star)\bigr)+\frac14\|\dot X\|^2+\frac14\bigl\|\dot X+2\sqrt\mu\,(X-x^\star)\bigr\|^2.\tag{3.3}E(t)=(1+μs​)(f(X)−f(x⋆))+41​∥X˙∥2+41​​X˙+2μ​(X−x⋆)​2.(3.3)

Formalization targets

Goal: Theorem 2 (p. 14)

For f∈Sμ,L2(Rn)f\in\mathcal S^2_{\mu,L}(\mathbb R^n)f∈Sμ,L2​(Rn), every step size 0<s≤1/L0<s\le1/L0<s≤1/L, and every t≥0t\ge0t≥0,

f(X(t))−f(x⋆)≤7 ∥x0−x⋆∥22s e−μ t/4.f(X(t))-f(x^\star)\le\frac{7\,\|x_0-x^\star\|^2}{2s}\,e^{-\sqrt\mu\,t/4}.f(X(t))−f(x⋆)≤2s7∥x0​−x⋆∥2​e−μ​t/4.

Milestones

  1. The estimate (B.1), p. 50: E≤(1+μs)(f(X)−f(x⋆))+34∥X˙∥2+2μ∥X−x⋆∥2\mathcal E\le(1+\sqrt{\mu s})(f(X)-f(x^\star))+\frac34\|\dot X\|^2+2\mu\|X-x^\star\|^2E≤(1+μs​)(f(X)−f(x⋆))+43​∥X˙∥2+2μ∥X−x⋆∥2.
  2. The derivative identity of App. B.1, p. 50: along (1.10),
dEdt=−μ[∥X˙∥2+(1+μs)⟨∇f(X),X−x⋆⟩].\frac{d\mathcal E}{dt}=-\sqrt\mu\Bigl[\|\dot X\|^2+\bigl(1+\sqrt{\mu s}\bigr)\langle\nabla f(X),X-x^\star\rangle\Bigr].dtdE​=−μ​[∥X˙∥2+(1+μs​)⟨∇f(X),X−x⋆⟩].
  1. Lemma 3.2, p. 14: for every s>0s>0s>0, dEdt≤−μ4E\dfrac{d\mathcal E}{dt}\le-\dfrac{\sqrt\mu}{4}\mathcal EdtdE​≤−4μ​​E.
  2. The integrated decay, p. 14: E(t)≤e−μt/4E(0)\mathcal E(t)\le e^{-\sqrt\mu t/4}\mathcal E(0)E(t)≤e−μ​t/4E(0).
  3. The explicit-coefficient bound, p. 14: for s≤1/Ls\le1/Ls≤1/L,
f(X)−f(x⋆)≤[12+3(1+μs)3+2(μs)1+μs]∥x0−x⋆∥2e−μt/4s.f(X)-f(x^\star)\le\left[\frac12+\frac{3}{(1+\sqrt{\mu s})^3}+\frac{2(\mu s)}{1+\sqrt{\mu s}}\right]\frac{\|x_0-x^\star\|^2e^{-\sqrt\mu t/4}}{s}.f(X)−f(x⋆)≤[21​+(1+μs​)33​+1+μs​2(μs)​]s∥x0​−x⋆∥2e−μ​t/4​.
  1. The coefficient inequality, p. 14: for 0<μs≤10<\mu s\le10<μs≤1, the bracket above is <7/2<7/2<7/2.

Significance

The result. Theorem 2 gives a linear rate e−μt/4e^{-\sqrt\mu t/4}e−μ​t/4 for the heavy-ball ODE with an explicit constant, valid for every Sμ,L2\mathcal S^2_{\mu,L}Sμ,L2​ objective and every step size up to 1/L1/L1/L. Read next to Theorem 1 (rate e−μt/4e^{-\sqrt\mu t/4}e−μ​t/4, constant 2/s2/s2/s for the NAG-SC ODE), it shows that at the continuous level both high-resolution ODEs converge at the accelerated rate; the paper locates the difference between the two discrete methods in the discretization, where only the gradient correction of NAG-SC lets the Lyapunov argument survive (its Theorems 3 and 4 and §3.3). The Lyapunov function (3.3) is the NAG-SC function (2.4) with the s ∇f(X)\sqrt s\,\nabla f(X)s​∇f(X) term removed, and the two coincide at s=0s=0s=0.

Formalizing it. The result is proved in the paper; to our knowledge no machine-checked proof exists. The mission produces a checked chain from the ODE to the rate: a one-sided derivative computation along a trajectory in Rn\mathbb R^nRn, a differential inequality, a Grönwall-type integration, and a closed-form estimate of E(0)\mathcal E(0)E(0) from the initial conditions and LLL-smoothness. The derivative computation and the Grönwall step reuse directly in the companion missions on the NAG-SC and NAG-C high-resolution ODEs.

Difficulty

The algebra is short; the formal difficulty is analytic. The derivative of t↦f(X(t))t\mapsto f(X(t))t↦f(X(t)) must be obtained by the chain rule from a one-sided derivative of XXX at t=0t=0t=0 and fff's differentiability, and the derivative of the two squared norms from those of XXX and V=X˙V=\dot XV=X˙. Passing from Lemma 3.2 to E(t)≤e−μt/4E(0)\mathcal E(t)\le e^{-\sqrt\mu t/4}\mathcal E(0)E(t)≤e−μ​t/4E(0) is a comparison argument for a function known only to have one-sided derivatives on [0,∞)[0,\infty)[0,∞); the naive route through ddt(eμt/4E)≤0\frac{d}{dt}\bigl(e^{\sqrt\mu t/4}\mathcal E\bigr)\le0dtd​(eμ​t/4E)≤0 requires a monotonicity theorem for functions with derivatives within a half-line. Finally, the estimate of E(0)\mathcal E(0)E(0) needs f(x0)−f(x⋆)≤L2∥x0−x⋆∥2f(x_0)-f(x^\star)\le\frac L2\|x_0-x^\star\|^2f(x0​)−f(x⋆)≤2L​∥x0​−x⋆∥2 and ∥∇f(x0)∥≤L∥x0−x⋆∥\|\nabla f(x_0)\|\le L\|x_0-x^\star\|∥∇f(x0​)∥≤L∥x0​−x⋆∥, both consequences of LLL-smoothness and ∇f(x⋆)=0\nabla f(x^\star)=0∇f(x⋆)=0 that must be derived from the class definition, and the constant 7/27/27/2 needs a one-variable inequality in μs∈(0,1]\sqrt{\mu s}\in(0,1]μs​∈(0,1].

Formalization scope

Rn\mathbb R^nRn is EuclideanSpace ℝ (Fin n), ∇f\nabla f∇f is Mathlib's gradient, and ∇2f(x)\nabla^2f(x)∇2f(x) is fderiv ℝ (gradient f) x. The classes FL1\mathcal F^1_LFL1​, Sμ,L1\mathcal S^1_{\mu,L}Sμ,L1​, Sμ,L2\mathcal S^2_{\mu,L}Sμ,L2​ are structures carrying the paper's first-order inequalities (§1.4, p. 8), with 0<μ≤L0<\mu\le L0<μ≤L; the Lipschitz constant of the Hessian is existential, as the paper fixes none. The minimizer is a point x⋆x^\starx⋆ with f(x⋆)≤f(z)f(x^\star)\le f(z)f(x⋆)≤f(z) for all zzz.

A solution of (1.10) is a pair of curves X,V:R→RnX,V:\mathbb R\to\mathbb R^nX,V:R→Rn with X(0)=x0X(0)=x_0X(0)=x0​, V(0)=−2s ∇f(x0)/(1+μs)V(0)=-2\sqrt s\,\nabla f(x_0)/(1+\sqrt{\mu s})V(0)=−2s​∇f(x0​)/(1+μs​), V(t)V(t)V(t) the derivative of XXX and −2μV(t)−(1+μs)∇f(X(t))-2\sqrt\mu V(t)-(1+\sqrt{\mu s})\nabla f(X(t))−2μ​V(t)−(1+μs​)∇f(X(t)) the derivative of VVV at every t≥0t\ge0t≥0, both taken within [0,∞)[0,\infty)[0,∞); values for t<0t<0t<0 play no role. The theorems quantify over every such solution. Existence and uniqueness (the paper's Propositions 2.1–2.2) are not assumed, and no a priori bound on XXX or VVV is added. Derivative claims assert that the derivative exists and has the stated value or bound, so a statement cannot hold merely because a derivative is undefined. The goal states the decay of f(X(t))−f(x⋆)f(X(t))-f(x^\star)f(X(t))−f(x⋆) with the printed constant 7/(2s)7/(2s)7/(2s); it does not assume any decay of E\mathcal EE, and an ODE predicate in which VVV is not the derivative of XXX would be a different, trivial statement.

All drafts live in the namespace HighResODE.HeavyBallODE, with one definitions file (classes, ODE, Lyapunov function). Contributions are welcome at every milestone; a general lemma "φ′≤−c φ\varphi'\le-c\,\varphiφ′≤−cφ within [0,∞)[0,\infty)[0,∞) implies φ(t)≤e−ctφ(0)\varphi(t)\le e^{-ct}\varphi(0)φ(t)≤e−ctφ(0)" and the LLL-smoothness estimates around a minimizer are reusable well beyond this mission.

Selected references

  • B. Shi, S. S. Du, M. I. Jordan, W. J. Su, Understanding the Acceleration Phenomenon via High-Resolution Differential Equations, Mathematical Programming (2021); cited here in its preprint version arXiv:1810.08907v3. https://arxiv.org/abs/1810.08907
  • W. Su, S. Boyd, E. J. Candès, A Differential Equation for Modeling Nesterov's Accelerated Gradient Method: Theory and Insights, Journal of Machine Learning Research 17 (2016). https://arxiv.org/abs/1503.01243
  • L. Lessard, B. Recht, A. Packard, Analysis and Design of Optimization Algorithms via Integral Quadratic Constraints, SIAM Journal on Optimization 26 (2016). https://arxiv.org/abs/1408.3595
  • B. T. Polyak, Some methods of speeding up the convergence of iteration methods, USSR Computational Mathematics and Mathematical Physics 4 (1964). https://doi.org/10.1016/0041-5553(64)90137-5
9 thms3 active usersReviewed
🏆Completed
Convex OptimizationNumerical AnalysisOptimization·Captain: mikedeng1

From Differential Equation Solvers to Accelerated First-Order Methods for Convex Optimization 2: The NAG Flow Has a Unique C² Solution and Its Lyapunov Function Satisfies ℒ(t) ≤ e^(−t)ℒ(0)Research Paper

Motivation

Accelerated first-order methods, above all Nesterov's accelerated gradient (NAG) method, minimize a convex function with an LLL-Lipschitz gradient to accuracy ε\varepsilonε in O(L/ε)O(\sqrt{L/\varepsilon})O(L/ε​) gradient evaluations, and a μ\muμ-strongly convex one in O(L/μlog⁡(1/ε))O(\sqrt{L/\mu}\log(1/\varepsilon))O(L/μ​log(1/ε)), against O(L/ε)O(L/\varepsilon)O(L/ε) and O((L/μ)log⁡(1/ε))O((L/\mu)\log(1/\varepsilon))O((L/μ)log(1/ε)) for gradient descent. Since Su, Boyd and Candès (arXiv:1503.01243) read Nesterov's method as a discretization of the ODE x′′+3tx′+∇f(x)=0x''+\frac3tx'+\nabla f(x)=0x′′+t3​x′+∇f(x)=0, a line of work has tried to derive accelerated methods from continuous-time models with a Lyapunov function, e.g. Wibisono, Wilson and Jordan (arXiv:1603.04245) and Wilson, Recht and Jordan (arXiv:1611.02635).

Luo and Chen (arXiv:1909.03145, Math. Program. 2022) propose a single ODE, the NAG flow, that covers the convex case μ=0\mu=0μ=0 and the strongly convex case μ>0\mu>0μ>0 at once, through a time-dependent scaling factor γ(t)\gamma(t)γ(t). Its Lyapunov function decays like e−te^{-t}e−t for every μ≥0\mu\ge0μ≥0. The rest of their paper discretizes this flow (implicit, predictor–corrector, gradient-corrected and semi-implicit schemes; missions 3–8 of this series) and transfers the decay to the discrete Lyapunov function. This mission formalizes the continuous result, Lemma 3.2 of the paper, on which those discretizations are modelled.

Setting

Let VVV be a real Hilbert space with inner product ⟨⋅,⋅⟩\langle\cdot,\cdot\rangle⟨⋅,⋅⟩ and norm ∥⋅∥\|\cdot\|∥⋅∥. A function f:V→Rf:V\to\mathbb Rf:V→R is in the class Sμ1\mathcal S^1_\muSμ1​ if it is continuously differentiable and, for some μ≥0\mu\ge0μ≥0,

f(x)−f(y)−⟨∇f(y),x−y⟩ ≥ μ2∥x−y∥2∀ x,y∈V,f(x)-f(y)-\langle\nabla f(y),x-y\rangle\ \ge\ \frac{\mu}{2}\|x-y\|^2\qquad\forall\,x,y\in V,f(x)−f(y)−⟨∇f(y),x−y⟩ ≥ 2μ​∥x−y∥2∀x,y∈V,

the μ\muμ-convexity of fff (strong convexity when μ>0\mu>0μ>0). It is in Sμ,L1,1\mathcal S^{1,1}_{\mu,L}Sμ,L1,1​ if moreover ∇f\nabla f∇f is LLL-Lipschitz with 0<L<∞0<L<\infty0<L<∞. A global minimizer x∗x^*x∗ of fff is assumed to exist.

The scaling factor solves γ′=μ−γ\gamma'=\mu-\gammaγ′=μ−γ, γ(0)=γ0>0\gamma(0)=\gamma_0>0γ(0)=γ0​>0:

γ(t)=μ+(γ0−μ)e−t,t≥0.\gamma(t)=\mu+(\gamma_0-\mu)e^{-t},\qquad t\ge0 .γ(t)=μ+(γ0​−μ)e−t,t≥0.

It is positive for all t≥0t\ge0t≥0 and tends to μ\muμ; for μ=0\mu=0μ=0 it is γ0e−t\gamma_0e^{-t}γ0​e−t.

The NAG flow is the first-order system

x′=v−x,γv′=μ(x−v)−∇f(x),x(0)=x0, v(0)=v0,(56)x'=v-x,\qquad \gamma v'=\mu(x-v)-\nabla f(x),\qquad x(0)=x_0,\ v(0)=v_0, \tag{56}x′=v−x,γv′=μ(x−v)−∇f(x),x(0)=x0​, v(0)=v0​,(56)

or, equivalently, the second-order equation

γx′′+(μ+γ)x′+∇f(x)=0,x(0)=x0, x′(0)=v0−x0.(57)\gamma x''+(\mu+\gamma)x'+\nabla f(x)=0,\qquad x(0)=x_0,\ x'(0)=v_0-x_0. \tag{57}γx′′+(μ+γ)x′+∇f(x)=0,x(0)=x0​, x′(0)=v0​−x0​.(57)

Its Lyapunov function is

L(t)=f(x(t))−f(x∗)+γ(t)2∥v(t)−x∗∥2,v=x+x′.(58)\mathcal L(t)=f(x(t))-f(x^*)+\frac{\gamma(t)}{2}\|v(t)-x^*\|^2,\qquad v=x+x'. \tag{58}L(t)=f(x(t))−f(x∗)+2γ(t)​∥v(t)−x∗∥2,v=x+x′.(58)

Formalization targets

Goal: Lemma 3.2 (p. 13)

If f∈Sμ,L1,1f\in\mathcal S^{1,1}_{\mu,L}f∈Sμ,L1,1​ with μ≥0\mu\ge0μ≥0, then (57) has a unique solution x∈C2([0,∞);V)x\in C^2([0,\infty);V)x∈C2([0,∞);V), and

L′(t)≤−L(t)−μ2∥x′(t)∥2,(59)\mathcal L'(t)\le-\mathcal L(t)-\frac{\mu}{2}\|x'(t)\|^2, \tag{59}L′(t)≤−L(t)−2μ​∥x′(t)∥2,(59) L(t)+μ2∫0tes−t∥x′(s)∥2 ds≤e−tL(0),t≥0.(60)\mathcal L(t)+\frac{\mu}{2}\int_0^te^{s-t}\|x'(s)\|^2\,ds\le e^{-t}\mathcal L(0),\qquad t\ge0. \tag{60}L(t)+2μ​∫0t​es−t∥x′(s)∥2ds≤e−tL(0),t≥0.(60)

In particular f(x(t))−f(x∗)≤e−tL(0)f(x(t))-f(x^*)\le e^{-t}\mathcal L(0)f(x(t))−f(x∗)≤e−tL(0) for all μ≥0\mu\ge0μ≥0, with no dependence on LLL in the rate.

Milestones (pp. 12–14)

  1. Lemma 3.1: 2(u−v,v−w)=∥u−w∥2−∥u−v∥2−∥v−w∥22(u-v,v-w)=\|u-w\|^2-\|u-v\|^2-\|v-w\|^22(u−v,v−w)=∥u−w∥2−∥u−v∥2−∥v−w∥2.
  2. (54): γ(t)=μ+(γ0−μ)e−t\gamma(t)=\mu+(\gamma_0-\mu)e^{-t}γ(t)=μ+(γ0​−μ)e−t is the solution of γ′=μ−γ\gamma'=\mu-\gammaγ′=μ−γ, γ(0)=γ0\gamma(0)=\gamma_0γ(0)=γ0​, is positive, and tends to μ\muμ monotonically and exponentially.
  3. Well-posedness of (56): a unique classical solution (x,v)∈C1([0,∞);V)2(x,v)\in C^1([0,\infty);V)^2(x,v)∈C1([0,∞);V)2.
  4. (56) ⇔ (57): a C1C^1C1 solution of (56) gives a C2C^2C2 solution of (57), and conversely (x,x+x′)(x,x+x')(x,x+x′) solves (56).
  5. (61): along a solution of (56), L′=⟨∇f(x),x′⟩+μ−γ2∥v−x∗∥2+⟨μ(x−v)−∇f(x),v−x∗⟩\mathcal L'=\langle\nabla f(x),x'\rangle+\frac{\mu-\gamma}{2}\|v-x^*\|^2+\langle\mu(x-v)-\nabla f(x),v-x^*\rangleL′=⟨∇f(x),x′⟩+2μ−γ​∥v−x∗∥2+⟨μ(x−v)−∇f(x),v−x∗⟩.
  6. (63): L′=μ2∥x−x∗∥2−⟨∇f(x),x−x∗⟩−γ2∥v−x∗∥2−μ2∥x′∥2\mathcal L'=\frac{\mu}{2}\|x-x^*\|^2-\langle\nabla f(x),x-x^*\rangle-\frac{\gamma}{2}\|v-x^*\|^2-\frac{\mu}{2}\|x'\|^2L′=2μ​∥x−x∗∥2−⟨∇f(x),x−x∗⟩−2γ​∥v−x∗∥2−2μ​∥x′∥2.
  7. (59) along every solution of (56), for f∈Sμ1f\in\mathcal S^1_\muf∈Sμ1​.
  8. Remark 3.1: with γ′≤μ−γ\gamma'\le\mu-\gammaγ′≤μ−γ in place of (54), (59) still holds.

Significance

The lemma gives one continuous model whose energy decays at the rate e−te^{-t}e−t uniformly in μ≥0\mu\ge0μ≥0, without separate treatment of the convex and the strongly convex cases. The paper's discrete results (Theorems 4.1, 5.1, 5.2, 6.1, 7.2 and 7.3) prove the discrete counterpart Lk+1≤Lk/(1+αk)\mathcal L_{k+1}\le\mathcal L_k/(1+\alpha_k)Lk+1​≤Lk​/(1+αk​) by mimicking the computation (61)–(63) step by step, so the continuous identities are the template of the whole paper. The integral term in (60) also bounds ∫0tes−t∥x′(s)∥2ds\int_0^te^{s-t}\|x'(s)\|^2ds∫0t​es−t∥x′(s)∥2ds when μ>0\mu>0μ>0.

The result is proved in the paper; nothing in this mission is open. To our knowledge neither the NAG flow nor its Lyapunov analysis has been formalized in Lean or another proof assistant. The mission produces a machine-checked well-posedness theorem for a non-autonomous ODE in a Hilbert space whose coefficient 1/γ(t)1/\gamma(t)1/γ(t) is unbounded on [0,∞)[0,\infty)[0,∞) when μ=0\mu=0μ=0, and a machine-checked Lyapunov computation that later missions can mirror in discrete time.

Difficulty

The Lyapunov computation (61)–(63) is a direct, if careful, calculus exercise. The work lies elsewhere. Existence and uniqueness on the whole half-line [0,∞)[0,\infty)[0,∞) do not follow from a single application of a local Picard–Lindelöf theorem: the vector field (x,v)↦(v−x, (μ(x−v)−∇f(x))/γ(t))(x,v)\mapsto\big(v-x,\ (\mu(x-v)-\nabla f(x))/\gamma(t)\big)(x,v)↦(v−x, (μ(x−v)−∇f(x))/γ(t)) is globally Lipschitz in (x,v)(x,v)(x,v), but with a constant of order 1/γ(t)1/\gamma(t)1/γ(t), which grows like ete^{t}et when μ=0\mu=0μ=0. A local existence theorem on a short interval does not by itself give a solution on [0,∞)[0,\infty)[0,∞), and uniqueness must hold relative to [0,∞)[0,\infty)[0,∞) with one-sided derivatives at t=0t=0t=0. Passing from the differential inequality (59) to (60) requires L\mathcal LL to be differentiable relative to [0,∞)[0,\infty)[0,∞), including the endpoint t=0t=0t=0, and the integrand ∥x′(s)∥2\|x'(s)\|^2∥x′(s)∥2 to be continuous, so the regularity of the solution has to be carried through every step.

Formalization scope

  • VVV is a real Hilbert space (InnerProductSpace ℝ V, CompleteSpace V); the paper's duality pairing and dual norm are the inner product and the norm (Riesz).
  • The gradient is an explicit map gradf with HasGradientAt f (gradf x) x for every xxx and gradf continuous; Mathlib's gradient is not used. Sμ,L1,1\mathcal S^{1,1}_{\mu,L}Sμ,L1,1​ includes 0≤μ≤L0\le\mu\le L0≤μ≤L and 0<L0<L0<L.
  • Time derivatives are relative to [0,∞)[0,\infty)[0,∞) (derivWithin x (Ici 0)), so one-sided at 000. Every curve differentiated this way is assumed C1C^1C1 or C2C^2C2 on [0,∞)[0,\infty)[0,∞), so the junk value 000 of derivWithin at a point of non-differentiability never enters.
  • Uniqueness is equality on [0,∞)[0,\infty)[0,∞); values at negative times are unconstrained.
  • Standing assumptions made explicit: VVV Hilbert, a global minimizer x∗x^*x∗ exists, γ0>0\gamma_0>0γ0​>0. Remark 3.1 adds that its relaxed γ\gammaγ is positive on [0,∞)[0,\infty)[0,∞), which the paper takes for granted.
  • μ=0\mu=0μ=0 is allowed throughout; no milestone assumes μ>0\mu>0μ>0. The Lipschitz hypothesis appears only where existence is claimed (milestone 3 and the goal); milestones 4–8 assume f∈Sμ1f\in\mathcal S^1_\muf∈Sμ1​ only.
  • The goal is stated for the solution of (57), whose existence and uniqueness are part of the conclusion. The trivializing reading, "if a function satisfies L′≤−L\mathcal L'\le-\mathcal LL′≤−L then L(t)≤e−tL(0)\mathcal L(t)\le e^{-t}\mathcal L(0)L(t)≤e−tL(0)", is Grönwall's inequality and is not the target.

A complete development needs: a global existence and uniqueness theorem for Lipschitz non-autonomous ODEs on [0,∞)[0,\infty)[0,∞) with locally bounded Lipschitz constant (Mathlib's IsPicardLindelof and ODE_solution_unique_of_mem_Icc are the starting points), and one-sided calculus for t↦f(x(t))t\mapsto f(x(t))t↦f(x(t)) and t↦∥v(t)−x∗∥2t\mapsto\|v(t)-x^*\|^2t↦∥v(t)−x∗∥2. The global existence result for half-line ODEs is reusable well beyond this mission. Contributions are welcome on any milestone independently.

Selected references

  • H. Luo and L. Chen, From differential equation solvers to accelerated first-order methods for convex optimization, Mathematical Programming (2022); preprint arXiv:1909.03145v4. https://arxiv.org/abs/1909.03145
  • W. Su, S. Boyd and E. J. Candès, A differential equation for modeling Nesterov's accelerated gradient method: theory and insights, Journal of Machine Learning Research 17 (2016). https://arxiv.org/abs/1503.01243
  • A. Wibisono, A. C. Wilson and M. I. Jordan, A variational perspective on accelerated methods in optimization, PNAS 113 (2016). https://arxiv.org/abs/1603.04245
  • A. C. Wilson, B. Recht and M. I. Jordan, A Lyapunov analysis of accelerated methods in optimization, Journal of Machine Learning Research 22 (2021). https://arxiv.org/abs/1611.02635
  • Y. Nesterov, Lectures on Convex Optimization, Springer, 2018. https://doi.org/10.1007/978-3-319-91578-4
11 thms3 active usersReviewed
🏆Completed
Convex OptimizationOptimization·Captain: mikedeng1

Understanding the Acceleration Phenomenon via High-Resolution Differential Equations III: The High-Resolution ODE of NAG-SC Satisfies f(X(t)) − f(x⋆) ≤ 2‖x₀ − x⋆‖² e^(−√μ t/4)/sResearch Paper

Motivation

Nesterov's accelerated gradient method for μ\muμ-strongly convex, LLL-smooth functions (NAG-SC) reaches accuracy ε\varepsilonε in O(L/μlog⁡(1/ε))O(\sqrt{L/\mu}\log(1/\varepsilon))O(L/μ​log(1/ε)) gradient steps, against O((L/μ)log⁡(1/ε))O((L/\mu)\log(1/\varepsilon))O((L/μ)log(1/ε)) for gradient descent (Nesterov, 1983; Nesterov, 2004). Polyak's heavy-ball method has the same accelerated behaviour on quadratics but not in general, and the two methods share the same classical continuous-time limit, the ODE X¨+2μX˙+∇f(X)=0\ddot X+2\sqrt\mu\dot X+\nabla f(X)=0X¨+2μ​X˙+∇f(X)=0. That limit therefore cannot explain why one method accelerates and the other does not.

Shi, Du, Jordan and Su (arXiv:1810.08907v3, 2018; Math. Program. 2021) keep the O(s)O(\sqrt s)O(s​) terms of the step size sss when passing to the limit. The resulting high-resolution ODEs differ between the two methods: the one for NAG-SC contains a gradient correction s ∇2f(X)X˙\sqrt s\,\nabla^2 f(X)\dot Xs​∇2f(X)X˙, a Hessian-driven damping term absent from the heavy-ball ODE. Their Theorem 1 shows that the high-resolution ODE of NAG-SC converges linearly at the accelerated rate μ\sqrt\muμ​, with an explicit constant tied to the step size.

Timeline. Polyak (1964) introduces the heavy-ball method; Nesterov (1983) the accelerated gradient method. Su, Boyd and Candès (2016) derive the low-resolution ODE of Nesterov's method for convex functions. Alvarez, Attouch, Bolte and Redont (2002) study inertial systems with Hessian-driven damping. Shi, Du, Jordan and Su (2018) obtain the high-resolution ODEs and the rates of their Theorems 1–6. Attouch, Chbani, Fadili and Riahi (2020) treat the closely related dynamics x¨+2μx˙+β∇2f(x)x˙+∇f(x)=0\ddot x+2\sqrt\mu\dot x+\beta\nabla^2 f(x)\dot x+\nabla f(x)=0x¨+2μ​x˙+β∇2f(x)x˙+∇f(x)=0.

Setting

Let Rn\mathbb R^nRn carry the Euclidean inner product ⟨⋅,⋅⟩\langle\cdot,\cdot\rangle⟨⋅,⋅⟩ and norm ∥⋅∥\|\cdot\|∥⋅∥. For L>0L>0L>0, FL1(Rn)\mathcal F^1_L(\mathbb R^n)FL1​(Rn) is the class of differentiable convex functions fff, in the sense f(y)≥f(x)+⟨∇f(x),y−x⟩f(y)\ge f(x)+\langle\nabla f(x),y-x\ranglef(y)≥f(x)+⟨∇f(x),y−x⟩ for all x,yx,yx,y, whose gradient is LLL-Lipschitz. FL2\mathcal F^2_LFL2​ adds that fff is C2C^2C2 with a Lipschitz-continuous Hessian ∇2f\nabla^2 f∇2f. For 0<μ≤L0<\mu\le L0<μ≤L, Sμ,L2(Rn)\mathcal S^2_{\mu,L}(\mathbb R^n)Sμ,L2​(Rn) is the subclass of FL2\mathcal F^2_LFL2​ of μ\muμ-strongly convex functions:

f(y)≥f(x)+⟨∇f(x),y−x⟩+μ2∥y−x∥2for all x,y.f(y)\ge f(x)+\langle\nabla f(x),y-x\rangle+\frac\mu2\|y-x\|^2\qquad\text{for all }x,y.f(y)≥f(x)+⟨∇f(x),y−x⟩+2μ​∥y−x∥2for all x,y.

Write x⋆x^\starx⋆ for the minimizer of fff.

Fix a step size s>0s>0s>0 and a starting point x0x_0x0​. The high-resolution ODE of NAG-SC, (1.11), is

X¨(t)+2μ X˙(t)+s ∇2f(X(t))X˙(t)+(1+μs)∇f(X(t))=0,t≥0,\ddot X(t)+2\sqrt\mu\,\dot X(t)+\sqrt s\,\nabla^2 f(X(t))\dot X(t)+\big(1+\sqrt{\mu s}\big)\nabla f(X(t))=0,\qquad t\ge0,X¨(t)+2μ​X˙(t)+s​∇2f(X(t))X˙(t)+(1+μs​)∇f(X(t))=0,t≥0,

with X(0)=x0X(0)=x_0X(0)=x0​ and X˙(0)=−2s ∇f(x0)/(1+μs)\dot X(0)=-2\sqrt s\,\nabla f(x_0)/(1+\sqrt{\mu s})X˙(0)=−2s​∇f(x0​)/(1+μs​). The initial velocity matches the first step of NAG-SC. The Lyapunov function (2.4) is

E(t)=(1+μs)(f(X)−f(x⋆))+14∥X˙∥2+14∥X˙+2μ(X−x⋆)+s ∇f(X)∥2.\mathcal E(t)=\big(1+\sqrt{\mu s}\big)\big(f(X)-f(x^\star)\big)+\tfrac14\|\dot X\|^2+\tfrac14\big\|\dot X+2\sqrt\mu(X-x^\star)+\sqrt s\,\nabla f(X)\big\|^2 .E(t)=(1+μs​)(f(X)−f(x⋆))+41​∥X˙∥2+41​​X˙+2μ​(X−x⋆)+s​∇f(X)​2.

In Lean these are IsS2 f μ L, IsNAGSCODE f μ s x0 X V (with V the velocity X˙\dot XX˙) and lyap f μ s xs X V t, in the namespace HighResODE.NAGSCODE.

Formalization targets

Goal: Theorem 1 (p. 13)

For f∈Sμ,L2f\in\mathcal S^2_{\mu,L}f∈Sμ,L2​, 0<s≤1/L0<s\le1/L0<s≤1/L and every solution XXX of (1.11), for all t≥0t\ge0t≥0,

f(X(t))−f(x⋆)≤2∥x0−x⋆∥2s e−μ t/4.f(X(t))-f(x^\star)\le\frac{2\|x_0-x^\star\|^2}{s}\,e^{-\sqrt\mu\,t/4}.f(X(t))−f(x⋆)≤s2∥x0​−x⋆∥2​e−μ​t/4.

Milestones, in attack order

  1. (3.4), p. 15. The derivative of E\mathcal EE along (1.11), its closed form −μ(∥X˙∥2+(1+μs)⟨∇f(X),X−x⋆⟩+s2∥∇f(X)∥2)−s2[∥∇f(X)∥2+X˙⊤∇2f(X)X˙]-\sqrt\mu\big(\|\dot X\|^2+(1+\sqrt{\mu s})\langle\nabla f(X),X-x^\star\rangle+\frac s2\|\nabla f(X)\|^2\big)-\frac{\sqrt s}2\big[\|\nabla f(X)\|^2+\dot X^\top\nabla^2 f(X)\dot X\big]−μ​(∥X˙∥2+(1+μs​)⟨∇f(X),X−x⋆⟩+2s​∥∇f(X)∥2)−2s​​[∥∇f(X)∥2+X˙⊤∇2f(X)X˙], and the bound that drops the bracket.
  2. (3.5), p. 15. E˙≤−μ(1+μs2(f(X)−f(x⋆))+∥X˙∥2+3μ4∥X−x⋆∥2+s2∥∇f(X)∥2)\dot{\mathcal E}\le-\sqrt\mu\big(\frac{1+\sqrt{\mu s}}2(f(X)-f(x^\star))+\|\dot X\|^2+\frac{3\mu}4\|X-x^\star\|^2+\frac s2\|\nabla f(X)\|^2\big)E˙≤−μ​(21+μs​​(f(X)−f(x⋆))+∥X˙∥2+43μ​∥X−x⋆∥2+2s​∥∇f(X)∥2).
  3. (3.6), p. 15. E(t)≤(1+μs)(f(X)−f(x⋆))+∥X˙∥2+3μ∥X−x⋆∥2+3s4∥∇f(X)∥2\mathcal E(t)\le(1+\sqrt{\mu s})(f(X)-f(x^\star))+\|\dot X\|^2+3\mu\|X-x^\star\|^2+\frac{3s}4\|\nabla f(X)\|^2E(t)≤(1+μs​)(f(X)−f(x⋆))+∥X˙∥2+3μ∥X−x⋆∥2+43s​∥∇f(X)∥2.
  4. Lemma 3.1, p. 13. For every s>0s>0s>0, E˙(t)≤−μ4E(t)−s2[∥∇f(X)∥2+X˙⊤∇2f(X)X˙]\dot{\mathcal E}(t)\le-\frac{\sqrt\mu}4\mathcal E(t)-\frac{\sqrt s}2\big[\|\nabla f(X)\|^2+\dot X^\top\nabla^2 f(X)\dot X\big]E˙(t)≤−4μ​​E(t)−2s​​[∥∇f(X)∥2+X˙⊤∇2f(X)X˙].
  5. (3.2), p. 13. E(t)≤e−μt/4E(0)\mathcal E(t)\le e^{-\sqrt\mu t/4}\mathcal E(0)E(t)≤e−μ​t/4E(0).
  6. Coefficient bound, p. 14. For 0<μs≤10<\mu s\le10<μs≤1: 12+3−2μs+μs2(1+μs)3+2μs1+μs<2\frac12+\frac{3-2\sqrt{\mu s}+\mu s}{2(1+\sqrt{\mu s})^3}+\frac{2\mu s}{1+\sqrt{\mu s}}<221​+2(1+μs​)33−2μs​+μs​+1+μs​2μs​<2.

Milestones 1–5 hold for every step size s>0s>0s>0. The restriction s≤1/Ls\le1/Ls≤1/L enters only the goal and, through μs≤1\mu s\le1μs≤1, milestone 6.

Significance

Theorem 1 is the continuous-time half of the paper's explanation of acceleration. With s=1/Ls=1/Ls=1/L it gives f(X(t))−f(x⋆)≤2L∥x0−x⋆∥2e−μt/4f(X(t))-f(x^\star)\le2L\|x_0-x^\star\|^2e^{-\sqrt\mu t/4}f(X(t))−f(x⋆)≤2L∥x0​−x⋆∥2e−μ​t/4, a linear rate at the accelerated exponent μ\sqrt\muμ​. The Lyapunov function (2.4) is the template from which the paper builds the discrete Lyapunov function (2.6) for NAG-SC itself, and the bracket ∥∇f(X)∥2+X˙⊤∇2f(X)X˙\|\nabla f(X)\|^2+\dot X^\top\nabla^2 f(X)\dot X∥∇f(X)∥2+X˙⊤∇2f(X)X˙ in Lemma 3.1 is what that discretization uses (Remark 3.3). The heavy-ball ODE (1.10) lacks this term (Lemma 3.2), and that difference is the paper's account of why NAG-SC accelerates and heavy ball does not.

The result is proved in the paper; no machine-checked version is known. A formal proof needs a one-sided differential inequality along a C2C^2C2 trajectory in Rn\mathbb R^nRn, a Grönwall-type integration, and an elementary but delicate rational inequality in μs\sqrt{\mu s}μs​. The milestones isolate each of these steps. The Lyapunov computation is also reusable for the sibling missions of this series on the heavy-ball and NAG-C ODEs.

Difficulty

The algebra in (3.4) is routine once the derivative exists. The work is in making that derivative rigorous. Differentiating E\mathcal EE needs the chain rule for f∘Xf\circ Xf∘X and for ∇f∘X\nabla f\circ X∇f∘X. The second uses f∈C2f\in C^2f∈C2: the time derivative of ∇f(X(t))\nabla f(X(t))∇f(X(t)) is ∇2f(X(t))X˙(t)\nabla^2 f(X(t))\dot X(t)∇2f(X(t))X˙(t). All derivatives are one-sided at t=0t=0t=0. Passing from E˙≤−μ4E\dot{\mathcal E}\le-\frac{\sqrt\mu}4\mathcal EE˙≤−4μ​​E to (3.2) needs a comparison argument on [0,∞)[0,\infty)[0,∞), not just a pointwise derivative bound. The constant 222 of the goal rests on the coefficient inequality of milestone 6, and that inequality is tight: its left side tends to 222 as μs→0\mu s\to0μs→0, so no crude estimate closes it.

Formalization scope

Rn\mathbb R^nRn is EuclideanSpace ℝ (Fin n), ∇f\nabla f∇f is Mathlib's gradient f, and ∇2f(x)v\nabla^2 f(x)v∇2f(x)v is fderiv ℝ (gradient f) x v. The classes are structures in the first-order form of p. 8. Sμ,L2\mathcal S^2_{\mu,L}Sμ,L2​ requires ContDiff ℝ 2 f and an existential Lipschitz constant for the Hessian, because the page fixes none. The minimizer x⋆x^\starx⋆ is a point with f(x⋆)≤f(z)f(x^\star)\le f(z)f(x⋆)≤f(z) for all zzz.

A solution of (1.11) is a pair of curves (X,V)(X,V)(X,V) with these properties:

  • X(0)=x0X(0)=x_0X(0)=x0​, and V(0)V(0)V(0) is the prescribed initial velocity;
  • for every t≥0t\ge0t≥0, XXX has derivative V(t)V(t)V(t) within [0,∞)[0,\infty)[0,∞);
  • for every t≥0t\ge0t≥0, VVV has derivative within [0,∞)[0,\infty)[0,∞) equal to the right-hand side of (1.11).

Values for t<0t<0t<0 are irrelevant. Existence and uniqueness of the solution (the paper's Proposition 2.1) are not assumed: every statement quantifies over every solution. Derivative claims are stated as the existence of a one-sided derivative with the claimed value or bound, never through deriv, which returns 000 where no derivative exists. The ODE predicate fixes VVV as the derivative of XXX, so VVV cannot be chosen freely. The goal assumes only the ODE and the class, never the Lyapunov decay. The initial velocity uses ∇f(x0)\nabla f(x_0)∇f(x0​) as in (1.11); footnote 4 (p. 5) drops the ∇\nabla∇, a printed slip.

All six milestones are stated. Solvers are welcome to work on any of them. A general one-sided Grönwall lemma on [0,∞)[0,\infty)[0,∞) and the chain rule for ∇f∘X\nabla f\circ X∇f∘X would serve the other ODE missions of this series as well.

Selected references

  • B. Shi, S. S. Du, M. I. Jordan, W. J. Su, Understanding the Acceleration Phenomenon via High-Resolution Differential Equations, arXiv:1810.08907v3, 2018; Math. Program. (2021). https://arxiv.org/abs/1810.08907, https://doi.org/10.1007/s10107-021-01681-8
  • Yu. Nesterov, A method of solving a convex programming problem with convergence rate O(1/k²), Soviet Math. Dokl. 27 (1983). https://www.mathnet.ru/eng/dan46009
  • Yu. Nesterov, Introductory Lectures on Convex Optimization: A Basic Course, Springer, 2004. https://doi.org/10.1007/978-1-4419-8853-9
  • W. Su, S. Boyd, E. J. Candès, A differential equation for modeling Nesterov's accelerated gradient method: theory and insights, JMLR 17 (2016). https://jmlr.org/papers/v17/15-084.html
  • F. Alvarez, H. Attouch, J. Bolte, P. Redont, A second-order gradient-like dissipative dynamical system with Hessian-driven damping, J. Math. Pures Appl. 81 (2002). https://doi.org/10.1016/S0021-7824(01)01253-3
  • H. Attouch, Z. Chbani, J. Fadili, H. Riahi, First-order optimization algorithms via inertial systems with Hessian driven damping, Math. Program. (2020). https://doi.org/10.1007/s10107-020-01591-1
8 thms3 active usersReviewed
🏆Completed
Algorithmic Game TheoryConvex Optimization·Captain: mikedeng1

Learning in Games via Reinforcement and Regularization 3: In 2-Player Zero-Sum Games with an Interior Equilibrium, Time-Averaged Play Converges to the Nash SetResearch Paper

Motivation

Learning dynamics in games describe how players who repeatedly interact adjust their strategies from observed payoffs. A central question is whether such dynamics lead to Nash equilibrium. For the replicator dynamics, the canonical model of evolutionary game theory, the answer is negative in general: in Matching Pennies and in generic zero-sum games with a unique interior equilibrium, replicator trajectories cycle forever. A classical positive result survives: if a replicator trajectory of a 2-player game stays a bounded distance away from the boundary of the strategy space, its time average converges to the Nash set (Hofbauer and Sigmund, 1998, Ch. 7).

Mertikopoulos and Sandholm (arXiv:1407.6267v2; Math. Oper. Res. 41(4), 2016) study a broad family of continuous-time learning dynamics, regularized reinforcement learning: each player aggregates payoffs into scores and plays the mixed strategy that maximizes score minus a strongly convex penalty function. The family contains the replicator dynamics (entropic penalty) and the projection dynamics (quadratic penalty). For non-steep penalties, trajectories can enter and leave the boundary of the strategy space infinitely often, so the classical "stay away from the boundary" criterion is no longer natural. Section 6 of the paper replaces it with a condition on scores and derives convergence of time averages in zero-sum games with an interior equilibrium.

Setting

A 2-player finite game G\mathcal GG has players k=1,2k = 1, 2k=1,2, finite pure strategy sets Ak\mathcal A_kAk​, and payoff functions uku_kuk​ on pure profiles. A mixed strategy xkx_kxk​ is a probability vector on Ak\mathcal A_kAk​; X\mathcal XX is the set of mixed profiles x=(x1,x2)x = (x_1, x_2)x=(x1​,x2​) and uk(x)u_k(x)uk​(x) the expected payoff (2.1). The payoff vector of player kkk is vkα(x)=uk(α;x−k)v_{k\alpha}(x) = u_k(\alpha; x_{-k})vkα​(x)=uk​(α;x−k​), the payoff of the pure strategy α\alphaα against the opponent's mixed strategy (2.2). A profile x∗x^*x∗ is a Nash equilibrium if no player gains by a unilateral deviation to another mixed strategy; it is interior if xkα∗>0x^*_{k\alpha} > 0xkα∗​>0 for every kkk and α\alphaα. The game is zero-sum if u1=−u2u_1 = -u_2u1​=−u2​.

On the simplex Δ\DeltaΔ of a finite set, a penalty function hhh (Definition 2.1) is continuous on Δ\DeltaΔ, smooth on the relative interior of every face, and KKK-strongly convex for some K>0K > 0K>0:

h(tx1+(1−t)x2)≤th(x1)+(1−t)h(x2)−12Kt(1−t)∥x1−x2∥2.h(tx_1 + (1-t)x_2) \le t h(x_1) + (1-t) h(x_2) - \tfrac12 K t(1-t)\|x_1 - x_2\|^2 .h(tx1​+(1−t)x2​)≤th(x1​)+(1−t)h(x2​)−21​Kt(1−t)∥x1​−x2​∥2.

No steepness is required. The choice map is Q(y)=arg⁡max⁡x∈Δ{⟨y∣x⟩−h(x)}Q(y) = \arg\max_{x\in\Delta}\{\langle y|x\rangle - h(x)\}Q(y)=argmaxx∈Δ​{⟨y∣x⟩−h(x)} (Definition 2.2), the convex conjugate is h∗(y)=max⁡x∈Δ{⟨y∣x⟩−h(x)}h^*(y) = \max_{x\in\Delta}\{\langle y|x\rangle - h(x)\}h∗(y)=maxx∈Δ​{⟨y∣x⟩−h(x)} (C.1), and the Fenchel coupling is Fh(p,y)=h(p)+h∗(y)−⟨y∣p⟩F_h(p, y) = h(p) + h^*(y) - \langle y|p\rangleFh​(p,y)=h(p)+h∗(y)−⟨y∣p⟩ (C.10).

Each player kkk has its own penalty function hkh_khk​. An orbit of (RL) is a pair of trajectories (y(t),x(t))(y(t), x(t))(y(t),x(t)), t≥0t \ge 0t≥0, of score vectors and mixed profiles with

y˙k(t)=vk(x(t)),xk(t)=Qk(yk(t)).\dot y_k(t) = v_k(x(t)), \qquad x_k(t) = Q_k(y_k(t)) .y˙​k​(t)=vk​(x(t)),xk​(t)=Qk​(yk​(t)).

The time average of the orbit is xˉ(t)=t−1∫0tx(s) ds\bar x(t) = t^{-1}\int_0^t x(s)\,dsxˉ(t)=t−1∫0t​x(s)ds.

Formalization targets

Goal: Proposition 6.2 (p. 26)

If G\mathcal GG is a 2-player zero-sum game with an interior Nash equilibrium, then for every orbit of (RL),

dist⁡(xˉ(t),NE(G))→t→∞0.\operatorname{dist}\big(\bar x(t), \mathrm{NE}(\mathcal G)\big) \xrightarrow[t\to\infty]{} 0 .dist(xˉ(t),NE(G))t→∞​0.

The statement fixes no rate and no particular penalty function: it holds for every choice of h1,h2h_1, h_2h1​,h2​.

Milestones

  1. Proposition C.1 (p. 33): QQQ is Lipschitz and Q=dh∗Q = dh^*Q=dh∗.
  2. Lemma C.6 (p. 36): along (RL), ddtFhk(pk,yk(t))=⟨vk(x(t))∣xk(t)−pk⟩\frac{d}{dt}F_{h_k}(p_k, y_k(t)) = \langle v_k(x(t)) | x_k(t) - p_k\rangledtd​Fhk​​(pk​,yk​(t))=⟨vk​(x(t))∣xk​(t)−pk​⟩.
  3. (6.5) (p. 26): in the goal's setting, Fh(x∗,y(t))=∑kFhk(xk∗,yk(t))F_h(x^*, y(t)) = \sum_k F_{h_k}(x_k^*, y_k(t))Fh​(x∗,y(t))=∑k​Fhk​​(xk∗​,yk​(t)) is constant.
  4. Proposition A.1(iii) (p. 31): Qα(y)→0Q_\alpha(y) \to 0Qα​(y)→0 when yα−yβ→−∞y_\alpha - y_\beta \to -\inftyyα​−yβ​→−∞.
  5. Proposition C.5 (pp. 35–36): for interior ppp, bounded Fh(p,yj)F_h(p, y_j)Fh​(p,yj​) forces bounded score differences.
  6. Theorem 6.1 (p. 25): in any 2-player game, bounded score differences imply convergence of xˉ(t)\bar x(t)xˉ(t) to the Nash set.

Significance

The result. Proposition 6.2 shows that the time-average convergence known for the replicator dynamics in zero-sum games is a property of the whole class of regularized reinforcement learning schemes, including those whose trajectories hit the boundary of the strategy space infinitely often, such as the projection dynamics. Time-averaged play of no-regret-type dynamics in zero-sum games is the continuous-time counterpart of the standard argument that averaged learning computes minimax strategies. Theorem 6.1 isolates the mechanism for general 2-player games: bounded score differences suffice, with no zero-sum assumption.

Formalizing it. The results are proved in the paper; none has a machine-checked proof. A complete formalization produces reusable components: differentiability of the conjugate of a strongly convex function on the simplex (a Danskin-type statement), the Fenchel-coupling identity along the dynamics, and a conservation law for zero-sum games. These are the building blocks of many analyses of mirror-descent and follow-the-regularized-leader dynamics.

Difficulty

The first idea, comparing x(t)x(t)x(t) itself with the equilibrium, fails: x(t)x(t)x(t) need not converge, and for non-steep penalties it does not stay in the interior, so the Kullback–Leibler argument used for the replicator dynamics is unavailable. The conserved quantity is a function of the scores y(t)y(t)y(t), not of x(t)x(t)x(t), and turning its boundedness into control of score differences requires the asymptotic behaviour of the choice map when some scores diverge. The second difficulty is analytic: hhh is not differentiable at the boundary of Δ\DeltaΔ, yet h∗h^*h∗ must be differentiable everywhere on the score space, with derivative QQQ, for the chain rule along (RL) to apply. Finally, the passage from score differences to the time average uses that vkv_kvk​ is linear in the single opponent's strategy, which is specific to two players.

Formalization scope

  • Game. Players are Fin 2, strategy types A k are finite, payoffs are real functions on pure profiles; mixed profiles, expected payoffs and mixed Nash equilibria come from the published agt_games vocabulary (AGT.IsMixedNash). Zero-sum is u0(s)+u1(s)=0u_0(s) + u_1(s) = 0u0​(s)+u1​(s)=0 for every pure profile; interiority is xkα∗>0x^*_{k\alpha} > 0xkα∗​>0.
  • Penalty functions. IsPenalty h K is Definition 2.1 with strong convexity in the Euclidean norm; since KKK is existential the class does not depend on the norm. Steepness is not assumed. Each player has its own hkh_khk​.
  • Choice map and orbits. x=Q(y)x = Q(y)x=Q(y) is the predicate IsChoice h y x (maximizer of ⟨y∣x⟩−h(x)\langle y|x\rangle - h(x)⟨y∣x⟩−h(x) over Δ\DeltaΔ), never a chosen argmax. An orbit is IsRLOrbit u h y x: for t≥0t \ge 0t≥0, xk(t)=Qk(yk(t))x_k(t) = Q_k(y_k(t))xk​(t)=Qk​(yk​(t)) and y˙kα(t)=vkα(x(t))\dot y_{k\alpha}(t) = v_{k\alpha}(x(t))y˙​kα​(t)=vkα​(x(t)), a right derivative at t=0t = 0t=0. This differential form is equivalent to the integral form (RL) because xxx is continuous along an orbit.
  • Time average and convergence. xˉ(t)\bar x(t)xˉ(t) is defined coordinatewise with the interval integral; only t→∞t \to \inftyt→∞ is used. Convergence to the Nash set is Metric.infDist →0\to 0→0 in the sup metric on profiles. For Theorem 6.1, which has no equilibrium among its hypotheses, the conclusion also asserts that the Nash set is nonempty, because the distance to the empty set is 000 in Lean.
  • Ruled out. A chosen argmax for QQQ, orbits that are vacuous, steepness added to the penalty, convergence of a subsequence or a liminf in place of the limit, and a Nash set that may be empty are all excluded by the encoding. A sorry-free check exhibits a game, penalty functions and an orbit satisfying all hypotheses of the goal, and verifies that Matching Pennies has the interior equilibrium ((12,12),(12,12))((\tfrac12,\tfrac12),(\tfrac12,\tfrac12))((21​,21​),(21​,21​)).
  • Not included. Theorem 6.3 (chain transitivity under the best-response dynamics) depends on the theory of perturbed solutions of differential inclusions, which is not available. Proposition 6.4 is omitted (see the mission notes).

Contributions are welcome on any milestone; Proposition C.1 and Proposition A.1(iii) are self-contained convex-analysis statements about a single penalty function.

Selected references

  • P. Mertikopoulos and W. H. Sandholm, Learning in games via reinforcement and regularization, Mathematics of Operations Research 41(4), 2016. Preprint arXiv:1407.6267v2 (cited version). https://arxiv.org/abs/1407.6267
  • J. Hofbauer and K. Sigmund, Evolutionary Games and Population Dynamics, Cambridge University Press, 1998. https://doi.org/10.1017/CBO9781139173179
  • J. Hofbauer, S. Sorin and Y. Viossat, Time average replicator and best-reply dynamics, Mathematics of Operations Research 34(2), 2009. https://doi.org/10.1287/moor.1080.0359
  • R. T. Rockafellar, Convex Analysis, Princeton University Press, 1970. https://doi.org/10.1515/9781400873173
10 thms3 active usersReviewed
🏆Completed
Algorithmic Game TheoryConvex Optimization·Captain: mikedeng1

Learning in Games via Reinforcement and Regularization 1: Under Regularized Reinforcement Learning, Every Iteratively Dominated Strategy Becomes ExtinctResearch Paper

Motivation

A basic test of any model of learning in games is whether it respects strict dominance: a strategy that does worse than another against every behaviour of the opponents should eventually be abandoned, and, after it is gone, so should every strategy that becomes dominated in what remains. For the replicator dynamics of evolutionary game theory this is classical: elimination of dominated and iteratively dominated strategies goes back to Akin (1980), Nachbar (1990) and Samuelson and Zhang (1992), whose notion of extinction of a mixed strategy is the one used here; Weibull (1995) gives the textbook account, including a conditional result for weakly dominated strategies.

The replicator dynamics are the continuous-time limit of exponential weights (the logit choice rule), one member of a large family of learning schemes in which every player accumulates the payoffs of their strategies into a score and turns scores into mixed strategies by a regularized best response. Different regularizers give different dynamics: the Gibbs entropy gives the replicator dynamics, the Euclidean square gives the projection dynamics, and many others are used in online optimization. Mertikopoulos and Sandholm (arXiv:1407.6267, Mathematics of Operations Research 41(4), 2016) analyse this whole family in a single framework. In particular they do not assume that the regularizer is steep, so play can reach the boundary of the simplex in finite time and the induced dynamics on the simplex can fail to be well-posed. Their first rationality result is that every iteratively dominated strategy becomes extinct under every scheme of the family.

Setting

A finite game has a finite set of players N\mathcal NN, a finite set Ak\mathcal A_kAk​ of pure strategies for each player kkk, and payoffs uk:∏ℓAℓ→Ru_k:\prod_\ell\mathcal A_\ell\to\mathbb Ruk​:∏ℓ​Aℓ​→R. A mixed strategy of player kkk is a point xkx_kxk​ of the simplex Xk=Δ(Ak)\mathcal X_k=\Delta(\mathcal A_k)Xk​=Δ(Ak​), and uk(x)u_k(x)uk​(x) is the expected payoff (2.1) at a mixed profile x=(xk)kx=(x_k)_kx=(xk​)k​. The payoff vector of player kkk is vk(x)=(vkα(x))α∈Akv_k(x)=(v_{k\alpha}(x))_{\alpha\in\mathcal A_k}vk​(x)=(vkα​(x))α∈Ak​​, where vkα(x)=uk(α;x−k)v_{k\alpha}(x)=u_k(\alpha;x_{-k})vkα​(x)=uk​(α;x−k​) is the payoff of the pure strategy α\alphaα against x−kx_{-k}x−k​.

A penalty function on a simplex Δ\DeltaΔ (Definition 2.1) is a function h:Δ→Rh:\Delta\to\mathbb Rh:Δ→R that is continuous, smooth on the relative interior of every face of Δ\DeltaΔ, and strongly convex with some constant K>0K>0K>0:

h(tx1+(1−t)x2)≤t h(x1)+(1−t) h(x2)−12K t(1−t) ∥x1−x2∥2.h(tx_1+(1-t)x_2)\le t\,h(x_1)+(1-t)\,h(x_2)-\tfrac12K\,t(1-t)\,\|x_1-x_2\|^2 .h(tx1​+(1−t)x2​)≤th(x1​)+(1−t)h(x2​)−21​Kt(1−t)∥x1​−x2​∥2.

Its choice map is Q(y)=arg⁡max⁡x∈Δ{⟨y∣x⟩−h(x)}Q(y)=\arg\max_{x\in\Delta}\{\langle y|x\rangle-h(x)\}Q(y)=argmaxx∈Δ​{⟨y∣x⟩−h(x)} (Definition 2.2). Each player kkk has a penalty function hkh_khk​ and choice map QkQ_kQk​. The reinforcement learning dynamics are

y˙k=vk(x),xk=Qk(yk),(RL)\dot y_k=v_k(x),\qquad x_k=Q_k(y_k),\qquad\text{(RL)}y˙​k​=vk​(x),xk​=Qk​(yk​),(RL)

so that yk(t)y_k(t)yk​(t) is the cumulative payoff of player kkk's pure strategies and x(t)=Q(y(t))x(t)=Q(y(t))x(t)=Q(y(t)) is the resulting play. Strategy pkp_kpk​ is dominated by pk′p'_kpk′​, written pk≺pk′p_k\prec p'_kpk​≺pk′​, if uk(pk;x−k)<uk(pk′;x−k)u_k(p_k;x_{-k})<u_k(p'_k;x_{-k})uk​(pk​;x−k​)<uk​(pk′​;x−k​) for all xxx (4.1). Removing dominated strategies and repeating gives the iteratively dominated strategies. A mixed strategy pkp_kpk​ becomes extinct along x(t)x(t)x(t) if min⁡{xkα(t):α∈supp⁡(pk)}→0\min\{x_{k\alpha}(t):\alpha\in\operatorname{supp}(p_k)\}\to0min{xkα​(t):α∈supp(pk​)}→0.

The analysis runs through the Fenchel coupling Fh(p,y)=h(p)+h∗(y)−⟨y∣p⟩F_h(p,y)=h(p)+h^*(y)-\langle y|p\rangleFh​(p,y)=h(p)+h∗(y)−⟨y∣p⟩, where h∗(y)=max⁡x∈Δ{⟨y∣x⟩−h(x)}h^*(y)=\max_{x\in\Delta}\{\langle y|x\rangle-h(x)\}h∗(y)=maxx∈Δ​{⟨y∣x⟩−h(x)}, and the Bregman divergence Dh(p,x)=h(p)−h(x)−h′(x;p−x)D_h(p,x)=h(p)-h(x)-h'(x;p-x)Dh​(p,x)=h(p)−h(x)−h′(x;p−x), defined with the one-sided directional derivative so that it makes sense at boundary points where hhh has no gradient.

Formalization targets

Goal: Theorem 4.1

For every orbit x(t)=Q(y(t))x(t)=Q(y(t))x(t)=Q(y(t)) of (RL), every player kkk, and every mixed strategy pkp_kpk​ that is dominated, possibly only after some rounds of elimination,

lim⁡t→∞ min⁡{xkα(t):α∈supp⁡(pk)}=0.\lim_{t\to\infty}\ \min\{x_{k\alpha}(t):\alpha\in\operatorname{supp}(p_k)\}=0 .t→∞lim​ min{xkα​(t):α∈supp(pk​)}=0.

The penalty functions are arbitrary in the sense of Definition 2.1, including non-steep ones, and may differ across players.

Milestones

  1. Proposition C.3: Fh(p,y)≥12K∥Q(y)−p∥2F_h(p,y)\ge\tfrac12K\|Q(y)-p\|^2Fh​(p,y)≥21​K∥Q(y)−p∥2, hence Fh≥0F_h\ge0Fh​≥0, and Fh(p,yj)→0F_h(p,y_j)\to0Fh​(p,yj​)→0 iff Q(yj)→pQ(y_j)\to pQ(yj​)→p.
  2. (4.6)–(4.8): if pk≺pk′p_k\prec p'_kpk​≺pk′​ with payoff gap at least δ\deltaδ, then Fk(pk,yk(t))≥Vk(yk(0))+δtF_k(p_k,y_k(t))\ge V_k(y_k(0))+\delta tFk​(pk​,yk​(t))≥Vk​(yk​(0))+δt.
  3. Proposition C.2 (i)–(iii): Dh(p,⋅)D_h(p,\cdot)Dh​(p,⋅) is finite, at least 12K∥x−p∥2\tfrac12K\|x-p\|^221​K∥x−p∥2, and continuous on Δp={x∈Δ:xα>0 whenever pα>0}\Delta_p=\{x\in\Delta:x_\alpha>0\text{ whenever }p_\alpha>0\}Δp​={x∈Δ:xα​>0 whenever pα​>0}.
  4. (C.11): Fh(p,y)=Dh(p,Q(y))F_h(p,y)=D_h(p,Q(y))Fh​(p,y)=Dh​(p,Q(y)) whenever Q(y)∈ΔpQ(y)\in\Delta_pQ(y)∈Δp​.
  5. Proposition C.4: if Fh(p,yj)→+∞F_h(p,y_j)\to+\inftyFh​(p,yj​)→+∞, then Q(yj)Q(y_j)Q(yj​) has no limit point in Δp\Delta_pΔp​.
  6. Theorem 4.1 for strategies dominated in the original game.

Two companion statements extend the goal: Proposition 4.2, the rate xkα(t)≤φk(ck−γkδkt)x_{k\alpha}(t)\le\varphi_k(c_k-\gamma_k\delta_kt)xkα​(t)≤φk​(ck​−γk​δk​t) under the rescaled dynamics (RLγ) with decomposable penalties, which gives extinction in finite time when the kernel is not steep at 000; and Proposition 4.3, a conditional extinction result for weakly dominated strategies.

Significance

The theorem shows that elimination of iteratively dominated strategies is a property of the whole class of regularized reinforcement learning schemes, not only of exponential weights. It is also unilateral: a player who follows (RL) stops playing dominated strategies whatever the other players do (Remark 4.2). The proof does not use the dynamics induced on the simplex, which need not be well-posed for non-steep regularizers; it works with the scores in the dual space. Proposition 4.2 then shows that the regularizer sets the speed of elimination: exponential for the entropy, finite time for the Euclidean square.

The result is proved in the paper; nothing here is open. No part of it is formalized. The mission produces the paper's convex-analytic toolkit for penalty functions on a simplex (Fenchel coupling, the one-sided Bregman divergence, their comparison with the Euclidean distance) and a machine-checked proof of the elimination theorem on top of it. The same toolkit is used by the paper's results on strict equilibria and zero-sum games.

Difficulty

The familiar argument for the replicator dynamics shows that the Kullback–Leibler divergence DKL(pk,xk(t))D_{KL}(p_k,x_k(t))DKL​(pk​,xk​(t)) diverges. For a general penalty function the analogous Bregman divergence uses dh(x)dh(x)dh(x), which does not exist at boundary points, and non-steep dynamics spend time on the boundary. The proof therefore tracks the Fenchel coupling Fk(pk,yk(t))F_k(p_k,y_k(t))Fk​(pk​,yk​(t)), whose growth is easy to bound, and must convert its divergence into extinction. That conversion (Propositions C.2–C.4) needs continuity of the one-sided Bregman divergence on Δp\Delta_pΔp​, including at points of lower-dimensional faces, and a compactness argument that turns "no limit point in Δp\Delta_pΔp​" into a statement about every time sequence. The iterated case adds an induction in which the payoff gap is only positive on the surviving profiles, so the trajectory's distance to those profiles must be controlled.

Formalization scope

Lean conventions:

  • Players form a Fintype ι; strategies are Fintype (A k); payoffs are u : ι → (∀ k, A k) → ℝ. Mixed strategies, mixed profiles and expected payoffs are those of the published agt_games definition.
  • A penalty function is IsPenalty h K: continuity on the simplex, C∞C^\inftyC∞ along smooth curves in the relative interior of a face (footnote 5), and (2.5) with the Euclidean norm. Steepness is not assumed.
  • The choice map is the predicate IsChoice h y x (xxx maximizes ⟨y∣⋅⟩−h\langle y|\cdot\rangle-h⟨y∣⋅⟩−h on the simplex), never a chosen maximizer.
  • An orbit of (RL) is a pair of trajectories (y,x)(y,x)(y,x) with xk(t)=Qk(yk(t))x_k(t)=Q_k(y_k(t))xk​(t)=Qk​(yk​(t)) and y˙kα(t)=vkα(x(t))\dot y_{k\alpha}(t)=v_{k\alpha}(x(t))y˙​kα​(t)=vkα​(x(t)) for t≥0t\ge0t≥0 (right derivative at 000). This is the differential form of the paper's integral equation and is equivalent to it along continuous orbits.
  • Iterated dominance eliminates pure strategies round by round, each time removing those strictly dominated by a mixed strategy supported on the survivors, against profiles supported on the survivors. A mixed strategy is iteratively dominated when it is dominated in this sense at some round.
  • Extinction is stated as a limit: for every ε>0\varepsilon>0ε>0 there is TTT such that for every t≥Tt\ge Tt≥T some α∈supp⁡(pk)\alpha\in\operatorname{supp}(p_k)α∈supp(pk​) has xkα(t)<εx_{k\alpha}(t)<\varepsilonxkα​(t)<ε.
  • The one-sided Bregman divergence is handled through a predicate asserting that the one-sided derivative is a given real number. The value +∞+\infty+∞ is never encoded as a real number.

Trivializing formalizations are ruled out. Orbits are not vacuous: a sorry-free check exhibits an explicit orbit of a game with a strictly dominated action, under the quadratic penalty. The choice map is not a junk argmax. Steepness, which would keep play interior, is not added. Extinction is not weakened to a lim inf⁡\liminfliminf.

Needed infrastructure: properties of strongly convex functions on a simplex, the conjugate and its maximizers, one-sided derivatives of convex functions along segments, and compactness of the product of simplices. Most of this is reusable beyond the mission. Contributions are welcome on the Appendix-C lemmas, which are independent of the game, as well as on the goal.

Selected references

  • P. Mertikopoulos and W. H. Sandholm, Learning in games via reinforcement and regularization, Mathematics of Operations Research 41(4), 2016; preprint arXiv:1407.6267v2. https://arxiv.org/abs/1407.6267
  • E. Akin, Domination or equilibrium, Mathematical Biosciences 50, 239–250, 1980.
  • J. H. Nachbar, Evolutionary selection dynamics in games, International Journal of Game Theory 19, 59–89, 1990.
  • L. Samuelson and J. Zhang, Evolutionary stability in asymmetric games, Journal of Economic Theory 57, 363–391, 1992.
  • J. W. Weibull, Evolutionary Game Theory, MIT Press, 1995.
  • K. C. Kiwiel, Free-steering relaxation methods for problems with strictly convex costs and linear constraints, Mathematics of Operations Research 22(2), 326–349, 1997.
13 thms3 active usersReviewed
🏆Completed
Convex OptimizationOptimization·Captain: mikedeng1

A Differential Equation for Modeling Nesterov's Accelerated Gradient Method: Theory and Insights V: The Speed Restarted ODE Converges Linearly for Strongly Convex fResearch Paper

Motivation

Nesterov's accelerated gradient method minimizes a convex function fff with LLL-Lipschitz gradient at the rate O(1/k2)O(1/k^2)O(1/k2), which is optimal among first-order methods. For a μ\muμ-strongly convex fff the method with the generic momentum coefficient (k−1)/(k+2)(k-1)/(k+2)(k−1)/(k+2) does not exploit the strong convexity: Su, Boyd and Candès show in §4.2 of arXiv:1503.01243 that its rate stays polynomial. Linear convergence O((1−μ/L)k)O((1-\sqrt{\mu/L})^k)O((1−μ/L​)k) is available with the momentum (L−μ)/(L+μ)(\sqrt L-\sqrt\mu)/(\sqrt L+\sqrt\mu)(L​−μ​)/(L​+μ​), but this requires knowing μ\muμ, which is rarely available in practice.

Restarting is the practical remedy. O'Donoghue and Candès (2013) proposed gradient restarting: reset the momentum whenever the objective increases. It is observed to recover linear convergence, but no rate was proved for it. Section 5 of Su, Boyd and Candès proposes a variant, speed restarting, which resets the momentum when the speed of the iterates stops increasing, and proves a linear rate for its continuous-time limit. This mission formalizes that result, Theorem 10 of the paper, and the lemmas it rests on.

Setting

Write Rn\mathbb R^nRn with the Euclidean norm. A function f:Rn→Rf:\mathbb R^n\to\mathbb Rf:Rn→R is in FL\mathcal F_LFL​ if it is convex and continuously differentiable with ∥∇f(x)−∇f(y)∥≤L∥x−y∥\|\nabla f(x)-\nabla f(y)\|\le L\|x-y\|∥∇f(x)−∇f(y)∥≤L∥x−y∥. It is in Sμ\mathcal S_\muSμ​ if it is continuously differentiable and f(x)−μ∥x∥2/2f(x)-\mu\|x\|^2/2f(x)−μ∥x∥2/2 is convex. The class Sμ,L=FL∩Sμ\mathcal S_{\mu,L}=\mathcal F_L\cap\mathcal S_\muSμ,L​=FL​∩Sμ​ with 0<μ≤L0<\mu\le L0<μ≤L is the standing assumption of the whole mission. A minimizer is written x⋆x^\starx⋆ and f⋆=f(x⋆)f^\star=f(x^\star)f⋆=f(x⋆); the ratio L/μL/\muL/μ is the condition number.

The Nesterov ODE is

X¨(t)+3tX˙(t)+∇f(X(t))=0,X(0)=x0, X˙(0)=0,(3)\ddot X(t)+\frac3t\dot X(t)+\nabla f(X(t))=0,\qquad X(0)=x_0,\ \dot X(0)=0, \tag{3}X¨(t)+t3​X˙(t)+∇f(X(t))=0,X(0)=x0​, X˙(0)=0,(3)

the continuous-time limit of Nesterov's scheme with step size tending to zero. The paper's Theorem 1 shows it has a unique solution in C2((0,∞))∩C1([0,∞))C^2((0,\infty))\cap C^1([0,\infty))C2((0,∞))∩C1([0,∞)). In Lean a solution is a pair (X,V)(X,V)(X,V) with V=X˙V=\dot XV=X˙ (IsSolution f 3 x₀ X V).

The speed restarting time of the solution from x0x_0x0​ is

T=T(x0;f)=sup⁡{t>0: ∀u∈(0,t), d∥X˙(u)∥2du>0},T=T(x_0;f)=\sup\Bigl\{t>0:\ \forall u\in(0,t),\ \frac{\mathrm d\|\dot X(u)\|^2}{\mathrm du}>0\Bigr\},T=T(x0​;f)=sup{t>0: ∀u∈(0,t), dud∥X˙(u)∥2​>0},

the first time at which the speed ∥X˙∥\|\dot X\|∥X˙∥ decreases (restartTime; the set is restartSet).

The speed restarted ODE (29) replaces 3/t3/t3/t by 3/tsr3/t_{\mathrm{sr}}3/tsr​, where tsr=t−τt_{\mathrm{sr}}=t-\tautsr​=t−τ is the time since the latest restart τ\tauτ. Its solution XsrX^{\mathrm{sr}}Xsr starts at x0x_0x0​, follows (3) from x0x_0x0​ for time T1=T(x0;f)T_1=T(x_0;f)T1​=T(x0​;f), then follows (3) from Xsr(T1)X^{\mathrm{sr}}(T_1)Xsr(T1​) for time T2=T(Xsr(T1);f)T_2=T(X^{\mathrm{sr}}(T_1);f)T2​=T(Xsr(T1​);f), and so on. If a restart point is a minimizer the trajectory stays there. In Lean this is the predicate IsSpeedRestarted f x₀ Xsr.

Formalization targets

Goal: Theorem 10

There are positive constants c1,c2c_1,c_2c1​,c2​, depending only on L/μL/\muL/μ, such that for every f∈Sμ,Lf\in\mathcal S_{\mu,L}f∈Sμ,L​, every x0x_0x0​ and every t≥0t\ge0t≥0,

f(Xsr(t))−f⋆≤c1L∥x0−x⋆∥22 e−c2tL.f(X^{\mathrm{sr}}(t))-f^\star\le\frac{c_1L\|x_0-x^\star\|^2}2\,e^{-c_2t\sqrt L}.f(Xsr(t))−f⋆≤2c1​L∥x0​−x⋆∥2​e−c2​tL​.

The constants are left unspecified, so the goal survives improvements of them.

Milestones

For one unrestarted solution XXX of (3), with M(t)=sup⁡u∈(0,t]∥X˙(u)∥/uM(t)=\sup_{u\in(0,t]}\|\dot X(u)\|/uM(t)=supu∈(0,t]​∥X˙(u)∥/u:

  • Lemma 11. For t<12/Lt<\sqrt{12/L}t<12/L​: M(t)≤∥∇f(x0)∥/(4(1−Lt2/12))M(t)\le\|\nabla f(x_0)\|/(4(1-Lt^2/12))M(t)≤∥∇f(x0​)∥/(4(1−Lt2/12)).
  • (30). For 0<t<12/L0<t<\sqrt{12/L}0<t<12/L​: ∣ ∥X˙(t)∥−t4∥∇f(x0)∥ ∣≤L∥∇f(x0)∥t3/(48(1−Lt2/12))\bigl|\,\|\dot X(t)\|-\tfrac t4\|\nabla f(x_0)\|\,\bigr|\le L\|\nabla f(x_0)\|t^3/(48(1-Lt^2/12))​∥X˙(t)∥−4t​∥∇f(x0​)∥​≤L∥∇f(x0​)∥t3/(48(1−Lt2/12)).
  • Lemma 25. If x0≠x⋆x_0\ne x^\starx0​=x⋆: T(x0,f)≥4/(5L)T(x_0,f)\ge 4/(5\sqrt L)T(x0​,f)≥4/(5L​).
  • Descent display. df(X(t))dt=−3t∥X˙∥2−12d∥X˙∥2dt≤0\dfrac{\mathrm df(X(t))}{\mathrm dt}=-\dfrac3t\|\dot X\|^2-\dfrac12\dfrac{\mathrm d\|\dot X\|^2}{\mathrm dt}\le0dtdf(X(t))​=−t3​∥X˙∥2−21​dtd∥X˙∥2​≤0 for t≤Tt\le Tt≤T.
  • Lemma 12. For a universal C>0C>0C>0: f(X(T))−f⋆≤(1−Cμ/L)(f(x0)−f⋆)f(X(T))-f^\star\le(1-C\mu/L)(f(x_0)-f^\star)f(X(T))−f⋆≤(1−Cμ/L)(f(x0​)−f⋆).
  • Lemma 13. For a universal C~\tilde CC~: T≤4exp⁡(C~L/μ)/(5L)T\le 4\exp(\tilde CL/\mu)/(5\sqrt L)T≤4exp(C~L/μ)/(5L​).

Significance

Theorem 10 is a linear convergence guarantee for a momentum method that does not need μ\muμ: the restart rule only watches the speed. The paper presents it as a new result; gradient restarting has no comparable theorem. It also gives a framework for analyzing restart rules: each restart decreases the gap by a factor depending on L/μL/\muL/μ (Lemma 12), and restarts happen at a rate bounded below (Lemmas 13 and 25). Section 5.3 of the paper reports that the discrete analogue of the scheme (Algorithm 1) converges linearly in experiments.

The results are proved in the paper, but not formalized anywhere to our knowledge; the platform has no statement about restarting the Nesterov ODE. The formal work is the full proof, in particular the analysis of (3) near its singular point t=0t=0t=0. Sharper constants are of interest: the paper conjectures that T=O(L/μ)T=O(\sqrt L/\mu)T=O(L​/μ) for a large class of fff, which would improve c2c_2c2​, and the generalization to friction r/tr/tr/t with r>3r>3r>3 mentioned after Theorem 10 is a natural further target.

Difficulty

The ODE (3) is singular at t=0t=0t=0, so the standard Picard–Lindelöf theory does not apply there, and the small-time estimates (Lemma 11, (30), Lemma 25) must control the solution uniformly up to t=0t=0t=0, where the friction coefficient 3/t3/t3/t is unbounded. Lemma 11 bounds a supremum by a quantity that involves the same supremum; this argument needs a separate proof that the supremum is finite.

The speed restarting time is defined as a supremum, and neither T>0T>0T>0 nor T<∞T<\inftyT<∞ follows from the definition. The restarted trajectory is assembled from infinitely many segments, and the number of restarts before time ttt needs a lower bound. The obvious route through the O(1/t2)O(1/t^2)O(1/t2) rate of the unrestarted ODE gives only a polynomial rate, because it does not use strong convexity in a way that survives the restarts.

Formalization scope

Rn\mathbb R^nRn is EuclideanSpace ℝ (Fin n), ∇f\nabla f∇f is Mathlib's gradient, LLL is an NNReal, and Sμ\mathcal S_\muSμ​ is the paper's literal definition. The classes FL\mathcal F_LFL​, Sμ,L\mathcal S_{\mu,L}Sμ,L​ and the solution predicate of (3) come from the series' shared definitions (NesterovODE.WellPosed.Setting, NesterovODE.StrongCvx.Setting); the speed restarting time and the restarted trajectory sit in this mission's Setting definition, namespace NesterovODE.Restart.

Conventions the formalization commits to:

  • Solutions of (3) are pairs (X,V)(X,V)(X,V), quantified over: every solution in the regularity class of Theorem 1 is covered. Values at negative times are unconstrained.
  • 12d∥X˙∥2/du\tfrac12\mathrm d\|\dot X\|^2/\mathrm du21​d∥X˙∥2/du is written as ⟨V(u),−3uV(u)−∇f(X(u))⟩\langle V(u),-\tfrac3uV(u)-\nabla f(X(u))\rangle⟨V(u),−u3​V(u)−∇f(X(u))⟩, its value along a solution.
  • M(t)M(t)M(t) is not defined; Lemma 11 is stated as a bound on every ∥X˙(u)∥/u\|\dot X(u)\|/u∥X˙(u)∥/u, u∈(0,t]u\in(0,t]u∈(0,t].
  • Lean's sSup is 000 on an empty or unbounded set. Lemma 25 is stated as membership of 4/(5L)4/(5\sqrt L)4/(5L​) in the set defining TTT, and Lemma 13 as a bound on every element of that set, so neither can hold through this junk value.
  • The constants of Lemmas 12 and 13 are chosen before the dimension, fff, μ\muμ, LLL, x0x_0x0​ and the trajectory; those of Theorem 10 are functions of L/μL/\muL/μ chosen in the same position. Choosing them after fff would trivialize the statements and is ruled out.
  • Lemmas 11, 12, 13, 25, (30) and the descent display carry the standing assumption f∈Sμ,Lf\in\mathcal S_{\mu,L}f∈Sμ,L​; Lemmas 12, 13, 25 also carry x0≠x⋆x_0\ne x^\starx0​=x⋆, the paper's convention in §5. Theorem 10 holds for every x0x_0x0​.
  • The leftmost "0≤0\le0≤" of (30) is stated for 0<t≤6/L0<t\le\sqrt{6/L}0<t≤6/L​, the range on which it is true; the page writes it for all t<12/Lt<\sqrt{12/L}t<12/L​.
  • The restarted trajectory stays at a minimizer once it reaches one. Restart times tending to infinity is not assumed: it follows from Lemmas 25 and 13.

A complete development needs the analysis of (3) at t=0t=0t=0 (also the subject of mission I of this series), monotonicity arguments from derivative signs, and properties of strongly convex functions with Lipschitz gradient. The last two are reusable beyond this mission. Proofs of any milestone, and statements with explicit constants, are welcome.

Selected references

  • W. Su, S. Boyd, E. J. Candès, A Differential Equation for Modeling Nesterov's Accelerated Gradient Method: Theory and Insights, arXiv:1503.01243v2, 2015; J. Mach. Learn. Res. 17(153):1–43, 2016. https://arxiv.org/abs/1503.01243
  • B. O'Donoghue, E. J. Candès, Adaptive Restart for Accelerated Gradient Schemes, Foundations of Computational Mathematics 15:715–732, 2015. https://doi.org/10.1007/s10208-013-9150-3
  • Y. Nesterov, A method of solving a convex programming problem with convergence rate O(1/k²), Soviet Mathematics Doklady 27:372–376, 1983.
  • Y. Nesterov, Introductory Lectures on Convex Optimization: A Basic Course, Kluwer, 2004. https://doi.org/10.1007/978-1-4419-8853-9
10 thms3 active usersReviewed
🏆Completed
Convex OptimizationOptimization·Captain: mikedeng1

A Differential Equation for Modeling Nesterov's Accelerated Gradient Method: Theory and Insights IV: For Strongly Convex f, f(X(t)) − f⋆ ≤ C‖x₀ − x⋆‖²/(μ^((α−2)/2) t^α) for 2 ≤ α ≤ 2r/3Research Paper

Motivation

Accelerated gradient methods minimize smooth convex functions with a characteristic momentum term. Su, Boyd, and Candès identified a continuous-time differential equation whose trajectories model that acceleration. The equation replaces an iteration index by continuous time and makes the effects of damping visible. This mission concerns the paper's strongly convex regime: the function has curvature everywhere, yet the ODE with damping proportional to 1/t1/t1/t is not simply a linear-rate method. Theorem 8 quantifies the faster polynomial rate that the authors establish when the damping parameter is large enough.

The paper first treats the coefficient 3/t3/t3/t, obtaining an inverse-square objective bound. In §4 it replaces 333 by rrr, and in §4.3 it combines larger damping with strong convexity. The result gives a family of exponents α\alphaα rather than a single preferred exponent. The rate reaches t−2r/3t^{-2r/3}t−2r/3 at the end of the allowed range. The paper also explains that simply raising rrr does not automatically improve every finite-time bound, because its constant can grow with rrr.

Setting

The state space is Euclidean Rn\mathbb R^nRn. A function belongs to FL\mathcal F_LFL​ when it is convex, continuously differentiable, and its gradient is LLL-Lipschitz for some L>0L>0L>0. It belongs to Sμ\mathcal S_\muSμ​ when it is continuously differentiable and x↦f(x)−μ∥x∥2/2x\mapsto f(x)-\mu\|x\|^2/2x↦f(x)−μ∥x∥2/2 is convex. The class Sμ,L\mathcal S_{\mu,L}Sμ,L​ is their intersection. For this mission μ>0\mu>0μ>0. A point x⋆x^\starx⋆ is any global minimizer, and f⋆=f(x⋆)f^\star=f(x^\star)f⋆=f(x⋆).

Given an initial point x0x_0x0​, a damping parameter rrr, and time t>0t>0t>0, the ODE trajectory satisfies

X¨(t)+rtX˙(t)+∇f(X(t))=0,X(0)=x0,X˙(0)=0.\ddot X(t)+\frac r t\dot X(t)+\nabla f(X(t))=0, \qquad X(0)=x_0,\quad \dot X(0)=0.X¨(t)+tr​X˙(t)+∇f(X(t))=0,X(0)=x0​,X˙(0)=0.

The velocity VVV records X˙\dot XX˙. The regularity asserted is X∈C2((0,∞))∩C1([0,∞))X\in C^2((0,\infty))\cap C^1([0,\infty))X∈C2((0,∞))∩C1([0,∞)). The paper writes “the solution” and says an earlier existence argument applies to general rrr; the formal statements quantify over every trajectory satisfying the ODE and initial conditions. This captures the rate claim without identifying functions at irrelevant negative times.

Two energy functions organize the chapter. The first, E5(t)E_5(t)E5​(t), weights the objective gap by 2t2/(r−1)2t^2/(r-1)2t2/(r−1) and adds (r−1)∥X(t)+tV(t)/(r−1)−x⋆∥2(r-1)\|X(t)+tV(t)/(r-1)-x^\star\|^2(r−1)∥X(t)+tV(t)/(r−1)−x⋆∥2. The second, E8(t;α)E_8(t;\alpha)E8​(t;α), weights the gap by tαt^\alphatα and adds ((2r−α)2tα−2/8)∥X(t)+2tV(t)/(2r−α)−x⋆∥2((2r-\alpha)^2t^{\alpha-2}/8)\|X(t)+2tV(t)/(2r-\alpha)-x^\star\|^2((2r−α)2tα−2/8)∥X(t)+2tV(t)/(2r−α)−x⋆∥2. The threshold tαt_\alphatα​ is (α−2)(2r−α)/(2μ)\sqrt{(\alpha-2)(2r-\alpha)/(2\mu)}(α−2)(2r−α)/(2μ)​ for α>2\alpha>2α>2. These are the paper's objects in §4.1 and §4.3, not alternate energies chosen for ease of formalization.

Formalization targets

Theorem 5: high-friction baseline

For r>3r>3r>3, the paper proves both a pointwise inverse-square estimate and a bound on accumulated weighted error:

f(X(t))−f⋆≤(r−1)2∥x0−x⋆∥22t2,∫0∞t(f(X(t))−f⋆) dt≤(r−1)2∥x0−x⋆∥22(r−3).f(X(t))-f^\star\leq\frac{(r-1)^2\|x_0-x^\star\|^2}{2t^2}, \qquad \int_0^\infty t(f(X(t))-f^\star)\,dt \leq\frac{(r-1)^2\|x_0-x^\star\|^2}{2(r-3)}.f(X(t))−f⋆≤2t2(r−1)2∥x0​−x⋆∥2​,∫0∞​t(f(X(t))−f⋆)dt≤2(r−3)(r−1)2∥x0​−x⋆∥2​.

Its energy derivative inequality (18) is an earlier milestone. Equation (24) gives the integrated bound for E8E_8E8​ at and after tαt_\alphatα​, and equation (26) bounds the energy at that threshold. Those two displays are additional milestones. Their full statements, including the intermediate inequalities, are retained in Lean.

Theorem 8: strongly convex rate

For every r,αr,\alphar,α with 2≤α≤2r/32\leq\alpha\leq2r/32≤α≤2r/3, there is a constant C=C(r,α)C=C(r,\alpha)C=C(r,α) such that

f(X(t))−f⋆≤C∥x0−x⋆∥2μ(α−2)/2tα(t>0).f(X(t))-f^\star\leq \frac{C\|x_0-x^\star\|^2}{\mu^{(\alpha-2)/2}t^\alpha} \qquad(t>0).f(X(t))−f⋆≤μ(α−2)/2tαC∥x0​−x⋆∥2​(t>0).

The constant does not depend on nnn, LLL, μ\muμ, fff, the minimizer, initial point, chosen solution, or time. At α=2\alpha=2α=2 the claim includes r=3r=3r=3; the threshold arguments used for α>2\alpha>2α>2 are not imposed on that boundary case.

Significance

The theorem says how curvature and damping change the continuous-time objective rate. It separates the exponent, which is set by rrr and α\alphaα, from the explicit dependence on strong convexity, μ−(α−2)/2\mu^{-(\alpha-2)/2}μ−(α−2)/2. The baseline Theorem 5 is also useful independently: its weighted integral estimate records more than an O(t−2)O(t^{-2})O(t−2) pointwise statement. The paper uses these continuous-time estimates to motivate comparisons with generalized discrete Nesterov schemes Su, Boyd, and Candès, §4.

This result is proved in the cited paper. The work here is to state its hypotheses, energy identities, and rate in Lean so that the established argument can be checked. The definitions of smooth and strongly convex classes, the singular-time solution predicate, and the calculus of real-power energies are reusable in other continuous-time optimization results. No machine-checked proof of these newly drafted statements is claimed by this proposal.

Difficulty

At t=0t=0t=0 the damping coefficient r/tr/tr/t is singular, so a statement that applies ordinary second-order ODE calculus across zero would miss the actual boundary condition. Away from zero, the ordinary inverse-square energy does not by itself deliver the stronger exponents of Theorem 8. The stronger energy has real powers of ttt and its derivative contains terms with different signs. Strong convexity controls some terms only after the threshold tαt_\alphatα​; the early-time and late-time regimes therefore have different estimates. The argument must also cover real exponents throughout [2,2r/3][2,2r/3][2,2r/3], including the endpoint α=2\alpha=2α=2.

Formalization scope

Lean uses EuclideanSpace ℝ (Fin n) for Rn\mathbb R^nRn, with any natural number nnn. The function classes follow the literal definitions on page 5: convexity, continuous differentiability, a Lipschitz gradient with positive constant LLL, and convexity of f−μ∥⋅∥2/2f-\mu\|\cdot\|^2/2f−μ∥⋅∥2/2. The rate assumes μ>0\mu>0μ>0, implicit in the paper's denominator and in tαt_\alphatα​. A minimizer is supplied as a point satisfying f(x⋆)≤f(y)f(x^\star)\leq f(y)f(x⋆)≤f(y) for every yyy; it is not assumed to be a special optimizer returned by a choice function.

The ODE predicate includes the initial data, a right derivative for XXX at zero, right continuity of VVV at zero, and derivatives of both functions for positive time. It says nothing about negative time. Theorem 5 explicitly asserts integrability on (0,∞)(0,\infty)(0,∞) before bounding its integral, preventing the default value of a nonintegrable Bochner integral from satisfying the statement. Real powers are used only with positive bases. Every appearance of E5E_5E5​ is under r>1r>1r>1, and every use of E8E_8E8​ in a milestone is under α>2\alpha>2α>2 and α≤2r/3\alpha\leq2r/3α≤2r/3, so its denominators are nonzero.

In the goal, CCC is existentially quantified before the dimension, function, curvature and smoothness constants, trajectory, and time. A constant selected separately for each function or time would weaken Theorem 8 and is excluded. Contributions establishing the energy derivatives, the improper integral, the threshold estimates, and the real-exponent induction are all within scope. The statement of Theorem 8 keeps the paper's full exponent range even if some parts demand substantial new calculus support.

Selected references

  • Weijie Su, Stephen Boyd, and Emmanuel J. Candès, A Differential Equation for Modeling Nesterov's Accelerated Gradient Method: Theory and Insights, Journal of Machine Learning Research 17(153), 2016; arXiv:1503.01243v2, cited here by preprint pages 5, 12–13, and 17–19.
6 thms3 active usersReviewed
🏆Completed
Convex OptimizationOptimization·Captain: mikedeng1

A Differential Equation for Modeling Nesterov's Accelerated Gradient Method: Theory and Insights II: The ODE Solution Satisfies f(X(t)) − f⋆ ≤ 2‖x₀ − x⋆‖²/t²Research Paper

Motivation

Nesterov's accelerated gradient method (Nesterov, 1983) minimizes a smooth convex function fff with the update

xk=yk−1−s∇f(yk−1),yk=xk+k−1k+2(xk−xk−1),x_k=y_{k-1}-s\nabla f(y_{k-1}),\qquad y_k=x_k+\frac{k-1}{k+2}(x_k-x_{k-1}),xk​=yk−1​−s∇f(yk−1​),yk​=xk​+k+2k−1​(xk​−xk−1​),

and, for step size s≤1/Ls\le 1/Ls≤1/L, attains f(xk)−f⋆≤2∥x0−x⋆∥2/(s(k+1)2)f(x_k)-f^\star\le 2\|x_0-x^\star\|^2/(s(k+1)^2)f(xk​)−f⋆≤2∥x0​−x⋆∥2/(s(k+1)2). The rate O(1/k2)O(1/k^2)O(1/k2) is optimal among first-order methods, yet the classical proof proceeds by an "estimate sequence" argument that offers little insight into why the momentum coefficient (k−1)/(k+2)(k-1)/(k+2)(k−1)/(k+2) works.

Su, Boyd and Candès (arXiv:1503.01243, J. Mach. Learn. Res. 17, 2016) showed that, as s→0s\to0s→0, the iterates track the solution of the second-order ordinary differential equation

X¨+3tX˙+∇f(X)=0,X(0)=x0, X˙(0)=0,\ddot X+\frac3t\dot X+\nabla f(X)=0,\qquad X(0)=x_0,\ \dot X(0)=0,X¨+t3​X˙+∇f(X)=0,X(0)=x0​, X˙(0)=0,

under the time identification t≈kst\approx k\sqrt st≈ks​. Their Theorem 3 is the continuous-time counterpart of Nesterov's rate. Its short proof by an energy (Lyapunov) function became the template for a large literature on continuous-time models of accelerated methods, including Wibisono, Wilson and Jordan's variational perspective (PNAS 2016) and Attouch, Chbani, Peypouquet and Redont's analysis of the damped system x¨+αtx˙+∇Φ(x)=0\ddot x+\frac\alpha t\dot x+\nabla\Phi(x)=0x¨+tα​x˙+∇Φ(x)=0 (Math. Program. 2018).

This mission formalizes Theorem 3 and the three steps of its proof.

Setting

Let n≥0n\ge0n≥0 and let Rn\mathbb R^nRn carry the Euclidean inner product ⟨⋅,⋅⟩\langle\cdot,\cdot\rangle⟨⋅,⋅⟩ and norm ∥⋅∥\|\cdot\|∥⋅∥. For f:Rn→Rf:\mathbb R^n\to\mathbb Rf:Rn→R differentiable, ∇f\nabla f∇f is its gradient.

  • For L>0L>0L>0, the class FL\mathcal F_LFL​ consists of the convex, continuously differentiable fff with ∥∇f(x)−∇f(y)∥≤L∥x−y∥\|\nabla f(x)-\nabla f(y)\|\le L\|x-y\|∥∇f(x)−∇f(y)∥≤L∥x−y∥ for all x,yx,yx,y. The class F∞\mathcal F_\inftyF∞​ is ⋃L>0FL\bigcup_{L>0}\mathcal F_L⋃L>0​FL​.
  • A minimizer x⋆x^\starx⋆ is any point with f(x⋆)≤f(y)f(x^\star)\le f(y)f(x⋆)≤f(y) for all yyy, and f⋆=f(x⋆)f^\star=f(x^\star)f⋆=f(x⋆).
  • A solution of the ODE (3) with initial point x0x_0x0​ is a curve XXX on [0,∞)[0,\infty)[0,∞), twice continuously differentiable on (0,∞)(0,\infty)(0,∞) and continuously differentiable on [0,∞)[0,\infty)[0,∞), with X(0)=x0X(0)=x_0X(0)=x0​, X˙(0)=0\dot X(0)=0X˙(0)=0, and X¨(t)+3tX˙(t)+∇f(X(t))=0\ddot X(t)+\frac3t\dot X(t)+\nabla f(X(t))=0X¨(t)+t3​X˙(t)+∇f(X(t))=0 for every t>0t>0t>0. The damping coefficient 3/t3/t3/t is singular at t=0t=0t=0; the paper's Theorem 1 shows that a unique such solution exists for every f∈F∞f\in\mathcal F_\inftyf∈F∞​.
  • The energy along a solution is
E(t)=t2(f(X(t))−f⋆)+2∥X(t)+t2X˙(t)−x⋆∥2.\mathcal E(t)=t^2\big(f(X(t))-f^\star\big)+2\Big\|X(t)+\frac t2\dot X(t)-x^\star\Big\|^2 .E(t)=t2(f(X(t))−f⋆)+2​X(t)+2t​X˙(t)−x⋆​2.

In the Lean development these are IsFL, IsFinfty, IsSolution f 3 x₀ X V and energy f xstar X V t, in the namespace NesterovODE.Rate, where V is the velocity X˙\dot XX˙.

Formalization targets

Goal: Theorem 3 (p. 7)

For every f∈F∞f\in\mathcal F_\inftyf∈F∞​, every minimizer x⋆x^\starx⋆, every x0x_0x0​ and every solution XXX of (3),

f(X(t))−f⋆≤2∥x0−x⋆∥2t2for all t>0.f(X(t))-f^\star\le\frac{2\|x_0-x^\star\|^2}{t^2}\qquad\text{for all }t>0.f(X(t))−f⋆≤t22∥x0​−x⋆∥2​for all t>0.

Milestones (proof of Theorem 3, pp. 7–8)

  1. Derivative of the energy. For t>0t>0t>0,
E˙=2t(f(X)−f⋆)+t2⟨∇f(X),X˙⟩+4⟨X+t2X˙−x⋆,32X˙+t2X¨⟩.\dot{\mathcal E}=2t(f(X)-f^\star)+t^2\langle\nabla f(X),\dot X\rangle+4\Big\langle X+\frac t2\dot X-x^\star,\frac32\dot X+\frac t2\ddot X\Big\rangle .E˙=2t(f(X)−f⋆)+t2⟨∇f(X),X˙⟩+4⟨X+2t​X˙−x⋆,23​X˙+2t​X¨⟩.
  1. The energy is nonincreasing. Using the ODE,
E˙=2t(f(X)−f⋆)+4⟨X−x⋆,−t2∇f(X)⟩=2t(f(X)−f⋆)−2t⟨X−x⋆,∇f(X)⟩≤0.\dot{\mathcal E}=2t(f(X)-f^\star)+4\Big\langle X-x^\star,-\frac t2\nabla f(X)\Big\rangle=2t(f(X)-f^\star)-2t\langle X-x^\star,\nabla f(X)\rangle\le0 .E˙=2t(f(X)−f⋆)+4⟨X−x⋆,−2t​∇f(X)⟩=2t(f(X)−f⋆)−2t⟨X−x⋆,∇f(X)⟩≤0.
  1. The bound chain. E\mathcal EE is nonincreasing on [0,∞)[0,\infty)[0,∞) and, for t>0t>0t>0,
f(X(t))−f⋆≤E(t)t2≤E(0)t2=2∥x0−x⋆∥2t2.f(X(t))-f^\star\le\frac{\mathcal E(t)}{t^2}\le\frac{\mathcal E(0)}{t^2}=\frac{2\|x_0-x^\star\|^2}{t^2}.f(X(t))−f⋆≤t2E(t)​≤t2E(0)​=t22∥x0​−x⋆∥2​.

Significance

Theorem 3 is the statement that makes the ODE a faithful model of Nesterov's scheme at the level of rates: under t≈kst\approx k\sqrt st≈ks​, the bound (7) reproduces the discrete bound (6) with the same constant 2. The paper also shows that the constant cannot be improved (p. 8). The energy E\mathcal EE is the continuous analogue of the potential used in discrete proofs, and the same template, with other weights, gives the paper's later results on the generalized damping r/tr/tr/t and on strongly convex objectives.

The result is proved in the paper. As far as the platform and Mathlib are concerned it is not formalized: Mathlib has the calculus (derivatives of inner products and norms along curves, the first-order characterization of convexity, monotonicity from a sign of the derivative) but no statement about this ODE. A machine-checked proof establishes the Lyapunov argument for the singular-at-zero system, including the endpoint t=0t=0t=0 where only one-sided regularity is available, and supplies reusable lemmas for the other missions of this series and for formalizations of inertial dynamics with vanishing damping.

Difficulty

The algebra is short; the care lies in the analytic bookkeeping that the paper leaves implicit.

  • The energy is defined through f∘Xf\circ Xf∘X and ∥X+t2X˙−x⋆∥2\|X+\frac t2\dot X-x^\star\|^2∥X+2t​X˙−x⋆∥2, so its derivative needs the chain rule for a C1C^1C1 function along a differentiable curve and the derivative of a squared norm of a curve.
  • The ODE holds only for t>0t>0t>0 and the coefficient 3/t3/t3/t blows up at 000. Monotonicity of E\mathcal EE on the closed half-line therefore needs right-continuity of E\mathcal EE at 000, which follows from right-continuity of XXX and X˙\dot XX˙ there, together with tX˙(t)→0t\dot X(t)\to0tX˙(t)→0; the derivative argument alone gives monotonicity only on (0,∞)(0,\infty)(0,∞).
  • The sign of E˙\dot{\mathcal E}E˙ needs the first-order inequality f(x⋆)≥f(X)+⟨∇f(X),x⋆−X⟩f(x^\star)\ge f(X)+\langle\nabla f(X),x^\star-X\ranglef(x⋆)≥f(X)+⟨∇f(X),x⋆−X⟩ for a differentiable convex function.

Formalization scope

  • Rn\mathbb R^nRn is EuclideanSpace ℝ (Fin n), ∇f\nabla f∇f is Mathlib's gradient f. f∈FLf\in\mathcal F_Lf∈FL​ is 0 < L ∧ ConvexOn ℝ Set.univ f ∧ ContDiff ℝ 1 f ∧ LipschitzWith L (gradient f) with L : NNReal.
  • A solution is encoded as a pair of curves (X,V)(X,V)(X,V): X(0)=x0X(0)=x_0X(0)=x0​, V(0)=0V(0)=0V(0)=0, XXX has right derivative V(0)V(0)V(0) at 000, VVV is right-continuous at 000, and for every t>0t>0t>0, XXX has derivative V(t)V(t)V(t) and VVV has derivative −3tV(t)−∇f(X(t))-\frac3tV(t)-\nabla f(X(t))−t3​V(t)−∇f(X(t)). For continuous ∇f\nabla f∇f this is exactly the paper's regularity class C2((0,∞))∩C1([0,∞))C^2((0,\infty))\cap C^1([0,\infty))C2((0,∞))∩C1([0,∞)) with the ODE and the initial conditions. Values of XXX for t<0t<0t<0 are unconstrained and never used.
  • The paper states Theorem 3 for "the unique global solution". The Lean statements quantify over every solution and every minimizer, which by Theorem 1 is the same statement and does not depend on Theorem 1 (formalized in a separate mission of this series).
  • No hypothesis beyond the paper's is added. The Lipschitz constant is part of F∞\mathcal F_\inftyF∞​ and is kept although the proof of Theorem 3 does not use it quantitatively.
  • The bound is stated for t>0t>0t>0 only. At t=0t=0t=0 the right-hand side 2∥x0−x⋆∥2/02\|x_0-x^\star\|^2/02∥x0​−x⋆∥2/0 is 000 in Lean, and the inequality would be false.
  • The derivative statements use HasDerivAt rather than deriv, so no derivative silently takes the default value 000; the solution predicate is satisfiable (for f(x)=∥x∥2/2f(x)=\|x\|^2/2f(x)=∥x∥2/2, x0=0x_0=0x0​=0, the zero curve is a solution), so the goal is not vacuous.

Contributions welcome: proofs of the three milestones and of the goal; general lemmas on the derivative of t↦f(X(t))t\mapsto f(X(t))t↦f(X(t)) and t↦∥X(t)+t2V(t)−x⋆∥2t\mapsto\|X(t)+\frac t2V(t)-x^\star\|^2t↦∥X(t)+2t​V(t)−x⋆∥2; and the first-order convexity inequality in gradient form, which the strongly convex and generalized-damping missions of this series also need.

Selected references

  • W. Su, S. Boyd, E. J. Candès, A Differential Equation for Modeling Nesterov's Accelerated Gradient Method: Theory and Insights, J. Mach. Learn. Res. 17(153):1–43, 2016; cited from arXiv:1503.01243v2, 2015. https://arxiv.org/abs/1503.01243
  • Y. Nesterov, A method of solving a convex programming problem with convergence rate O(1/k²), Soviet Mathematics Doklady 27(2):372–376, 1983.
  • A. Wibisono, A. C. Wilson, M. I. Jordan, A variational perspective on accelerated methods in optimization, Proc. Natl. Acad. Sci. 113(47):E7351–E7358, 2016. https://doi.org/10.1073/pnas.1614734113
  • H. Attouch, Z. Chbani, J. Peypouquet, P. Redont, Fast convergence of inertial dynamics and algorithms with asymptotic vanishing viscosity, Math. Program. 168:123–175, 2018. https://doi.org/10.1007/s10107-016-1065-5
5 thms3 active usersReviewed
🏆Completed
Control TheoryLinear algebra·Captain: mikedeng1

Lyapunov Equations, Energy Functionals, and Model Order Reduction of Bilinear and Stochastic Systems 1: States Reached from 0 Stay in Im P and Ker Q Gives Zero Output, for Gramians P, Q ≥ 0 of (3.6)Research Paper

Motivation

Balanced truncation is the standard method for reducing the state dimension of a linear control system while keeping its input–output behaviour: states that are both hard to reach and hard to observe are discarded. Its justification rests on two Gramians, the solutions PPP and QQQ of a pair of Lyapunov equations, whose quadratic forms measure how much input energy a state costs to reach and how much output energy it produces. For linear systems the facts are classical: the image of the controllability Gramian is the reachable subspace, and the kernel of the observability Gramian is the unobservable subspace.

Bilinear systems, in which the input multiplies the state, are the simplest nonlinear extension of linear control systems. Their generalized Lyapunov equations have the same form as the Lyapunov equations of linear stochastic systems with multiplicative noise, which is why Benner and Damm treat the two classes together. Balanced truncation was extended to them using the solutions of generalized Lyapunov equations (Al-Baiyat and Bettayeb, 1993; Gray and Mesko, 1998, both as cited by Benner and Damm). Benner and Damm, in Lyapunov Equations, Energy Functionals, and Model Order Reduction of Bilinear and Stochastic Systems (SIAM J. Control Optim. 49(2), 2011, doi:10.1137/09075041X), examine which of the linear-system energy interpretations survive for bilinear systems. Their Theorem 3.1 is the structural result that does survive: the kernels of the Gramians still identify states that are unreachable from the origin and states that produce no output, so these states can be eliminated without changing the transfer behaviour.

Setting

Fix n,m,p≥0n, m, p \ge 0n,m,p≥0 and real matrices A,N1,…,Nm∈Rn×nA, N_1,\dots,N_m\in\mathbb R^{n\times n}A,N1​,…,Nm​∈Rn×n, B∈Rn×mB\in\mathbb R^{n\times m}B∈Rn×m, C∈Rp×nC\in\mathbb R^{p\times n}C∈Rp×n. The bilinear control system (3.1)–(3.2) is

x˙=Ax+∑j=1mNjx uj+Bu,y=Cx,\dot x = Ax + \sum_{j=1}^m N_j x\,u_j + Bu,\qquad y = Cx,x˙=Ax+j=1∑m​Nj​xuj​+Bu,y=Cx,

with state x(t)∈Rnx(t)\in\mathbb R^nx(t)∈Rn, input u(t)=[u1(t),…,um(t)]T∈Rmu(t) = [u_1(t),\dots,u_m(t)]^T\in\mathbb R^mu(t)=[u1​(t),…,um​(t)]T∈Rm and output y(t)∈Rpy(t)\in\mathbb R^py(t)∈Rp. The homogeneous system (3.5) drops the term BuBuBu: x˙=Ax+∑jNjx uj\dot x = Ax + \sum_j N_j x\,u_jx˙=Ax+∑j​Nj​xuj​, y=Cxy = Cxy=Cx. The same mmm indexes the bilinear terms and the columns of BBB. Write x(t,x0,u)x(t, x_0, u)x(t,x0​,u) for a solution with input uuu and initial state x(0)=x0x(0) = x_0x(0)=x0​, and y(t,x0,u)=Cx(t,x0,u)y(t,x_0,u) = Cx(t,x_0,u)y(t,x0​,u)=Cx(t,x0​,u).

An admissible input is a function u:[0,∞)→Rmu:[0,\infty)\to\mathbb R^mu:[0,∞)→Rm that is integrable on every interval [0,T][0,T][0,T]. A solution with x(0)=x0x(0) = x_0x(0)=x0​ is a continuous x:[0,∞)→Rnx:[0,\infty)\to\mathbb R^nx:[0,∞)→Rn satisfying

x(t)=x0+∫0t(Ax(s)+∑j=1muj(s)Njx(s)+Bu(s)) ds(t≥0).x(t) = x_0 + \int_0^t \Big(Ax(s) + \sum_{j=1}^m u_j(s)N_jx(s) + Bu(s)\Big)\,ds\qquad (t\ge 0).x(t)=x0​+∫0t​(Ax(s)+j=1∑m​uj​(s)Nj​x(s)+Bu(s))ds(t≥0).

The generalized Lyapunov equations (3.6) are

AP+PAT+∑j=1mNjPNjT=−BBT,ATQ+QA+∑j=1mNjTQNj=−CTC.AP + PA^T + \sum_{j=1}^m N_jPN_j^T = -BB^T,\qquad A^TQ + QA + \sum_{j=1}^m N_j^TQN_j = -C^TC .AP+PAT+j=1∑m​Nj​PNjT​=−BBT,ATQ+QA+j=1∑m​NjT​QNj​=−CTC.

A solution PPP of the first is a controllability Gramian and a solution QQQ of the second an observability Gramian. "P≥0P\ge 0P≥0" means PPP is symmetric nonnegative definite; Im P\mathrm{Im}\,PImP and Ker P\mathrm{Ker}\,PKerP are its image and kernel.

Formalization targets

Goal: Theorem 3.1 (p. 695)

(a) If P≥0P\ge 0P≥0 solves the first equation of (3.6), then for every admissible input uuu

x(t,0,u)∈Im Pfor all t≥0.x(t,0,u)\in\mathrm{Im}\,P\qquad\text{for all } t\ge 0 .x(t,0,u)∈ImPfor all t≥0.

(b) If Q≥0Q\ge 0Q≥0 solves the second equation of (3.6) and x0∈Ker Qx_0\in\mathrm{Ker}\,Qx0​∈KerQ, then for the homogeneous system (3.5) and every admissible input uuu

y(t,x0,u)=Cx(t,x0,u)=0for all t≥0.y(t,x_0,u) = Cx(t,x_0,u) = 0\qquad\text{for all } t\ge 0 .y(t,x0​,u)=Cx(t,x0​,u)=0for all t≥0.

Milestones (the steps of the paper's proof, pp. 695–696)

  1. NjT Ker P⊂Ker P⊂Ker BTN_j^T\,\mathrm{Ker}\,P\subset\mathrm{Ker}\,P\subset\mathrm{Ker}\,B^TNjT​KerP⊂KerP⊂KerBT.
  2. AT Ker P⊂Ker PA^T\,\mathrm{Ker}\,P\subset\mathrm{Ker}\,PATKerP⊂KerP.
  3. Im P\mathrm{Im}\,PImP is invariant under the vector field of (3.1): z∈Im Pz\in\mathrm{Im}\,Pz∈ImP implies Az+∑jwjNjz+Bw∈Im PAz + \sum_j w_jN_jz + Bw\in\mathrm{Im}\,PAz+∑j​wj​Nj​z+Bw∈ImP for every w∈Rmw\in\mathbb R^mw∈Rm.
  4. Nj Ker Q⊂Ker Q⊂Ker CN_j\,\mathrm{Ker}\,Q\subset\mathrm{Ker}\,Q\subset\mathrm{Ker}\,CNj​KerQ⊂KerQ⊂KerC and A Ker Q⊂Ker QA\,\mathrm{Ker}\,Q\subset\mathrm{Ker}\,QAKerQ⊂KerQ.
  5. Ker Q\mathrm{Ker}\,QKerQ is invariant under the vector field of (3.5).

Significance

The result. Part (a) says that a state outside Im P\mathrm{Im}\,PImP cannot be reached from the origin by any input, so its controllability energy EcE_cEc​ is infinite; part (b) says that a state in Ker Q\mathrm{Ker}\,QKerQ is indistinguishable from the origin at the output, so its observability energy EoE_oEo​ is zero. Together they justify the first step of balanced truncation for bilinear systems (Remark 3.2 of the paper): the system can be restricted to Im P\mathrm{Im}\,PImP and projected along Ker Q\mathrm{Ker}\,QKerQ without changing its input–output map, after which the Gramians are nonsingular and can be balanced. The theorem needs neither stability of AAA nor uniqueness of the Gramians, so it applies to every nonnegative definite solution of (3.6). The rest of §3 of the paper shows that the finer, quantitative energy estimates of the linear case fail for bilinear systems; this qualitative statement is what remains exact.

Formalizing it. The theorem is proved in the paper; no machine-checked version is known. A formal proof yields a reusable invariance principle: a subspace that is invariant under every vector field z↦Az+∑jwjNjz+Bwz\mapsto Az + \sum_j w_jN_jz + Bwz↦Az+∑j​wj​Nj​z+Bw is invariant under the flow of the input-driven system, for merely integrable inputs. It also yields the linear-algebra lemmas relating kernels of nonnegative definite solutions of generalized Lyapunov equations to the system matrices, which are used throughout model reduction and stochastic stability theory.

Difficulty

The algebraic part (milestones 1, 2, 4) rests on the fact that for a nonnegative definite matrix MMM, vTMv=0v^TMv = 0vTMv=0 forces Mv=0Mv = 0Mv=0, applied to the terms of the Lyapunov equation tested against a kernel vector. The passage from the vector field to trajectories is where care is needed. The paper's sentence "x˙(t)∈Im P\dot x(t)\in\mathrm{Im}\,Px˙(t)∈ImP whenever x(t)∈Im Px(t)\in\mathrm{Im}\,Px(t)∈ImP, so Im P\mathrm{Im}\,PImP is invariant" is not by itself an argument: tangency of a vector field to a subspace implies invariance only together with a uniqueness or Gronwall-type estimate, and here the field depends on time through an input that is only integrable, so xxx is merely absolutely continuous and x˙\dot xx˙ exists only almost everywhere. A pointwise "the derivative points into the subspace" argument therefore does not apply as stated, and the step needs an estimate on the component of x(t)x(t)x(t) orthogonal to the subspace.

Formalization scope

The Lean development uses states Fin n → ℝ, matrices Matrix (Fin n) (Fin n) ℝ, the matrix–vector product *ᵥ, and indices j=1,…,mj = 1,\dots,mj=1,…,m as Fin m. Im P\mathrm{Im}\,PImP and Ker P\mathrm{Ker}\,PKerP are LinearMap.range and LinearMap.ker of Matrix.toLin' P; "P≥0P\ge 0P≥0" is Matrix.PosSemidef, which includes symmetry. Admissible inputs are integrable on every [0,T][0,T][0,T] (this contains L2[0,∞[L^2[0,\infty[L2[0,∞[, the input space of the paper's energy functionals). A solution is a function continuous on [0,∞)[0,\infty)[0,∞) whose integrand is integrable on every [0,t][0,t][0,t] and which satisfies the integral equation for all t≥0t\ge 0t≥0; no uniqueness of solutions is presupposed, and the theorem is asserted for every such solution.

Two reading decisions are fixed. First, part (b) is stated for every admissible input uuu, not only for u=0u = 0u=0 as printed in the theorem: the paper's proof concludes "x(t,x0,u)∈Ker Qx(t,x_0,u)\in\mathrm{Ker}\,Qx(t,x0​,u)∈KerQ for all t≥0t\ge0t≥0, implying y(t,x0,u)=0y(t,x_0,u) = 0y(t,x0​,u)=0", and the theorem's clause "i.e., Eo(x0)=0E_o(x_0) = 0Eo​(x0​)=0" (with EoE_oEo​ a maximum over inputs) needs this form. The ∀u\forall u∀u statement contains the printed one. Second, the clauses "i.e., Ec(x0)=∞E_c(x_0) = \inftyEc​(x0​)=∞" and "i.e., Eo(x0)=0E_o(x_0) = 0Eo​(x0​)=0", which restate the conclusions through the energy functionals (3.3)–(3.4), are not formalized.

The statements must not be trivialized: assuming PPP positive definite makes (a) empty (Im P=Rn\mathrm{Im}\,P = \mathbb R^nImP=Rn), assuming AAA stable or the Gramians unique adds hypotheses the paper does not have, and a solution notion requiring C1C^1C1 trajectories for integrable inputs would make the solution set empty. None of these is used. A local check confirms the hypotheses have content: for A=−I2A = -I_2A=−I2​, N1=0N_1 = 0N1​=0, B=e1B = e_1B=e1​, the matrix P=diag(1/2,0)P = \mathrm{diag}(1/2, 0)P=diag(1/2,0) is a nonnegative definite solution of (3.6) with e2∉Im Pe_2\notin\mathrm{Im}\,Pe2​∈/ImP.

A complete development needs Mathlib's facts on nonnegative definite matrices, interval integrals of vector-valued functions, and a Gronwall inequality with an integrable coefficient. The flow-invariance lemma for input-driven linear-in-state systems is reusable beyond this mission. Contributions welcome: proofs of the five milestones, and the passage from vector-field invariance to trajectory invariance.

Selected references

  • P. Benner, T. Damm, Lyapunov Equations, Energy Functionals, and Model Order Reduction of Bilinear and Stochastic Systems, SIAM J. Control Optim. 49(2), 686–711, 2011. https://doi.org/10.1137/09075041X
  • S. A. Al-Baiyat, M. Bettayeb, A new model reduction scheme for k-power bilinear systems, Proceedings of the 32nd IEEE Conference on Decision and Control, San Antonio, 1993, pp. 22–27 (reference [1] of Benner–Damm).
  • W. S. Gray, J. Mesko, Energy functions and algebraic Gramians for bilinear systems, Preprints of the 4th IFAC Nonlinear Control Systems Design Symposium, Enschede, 1998, pp. 103–108 (reference [25] of Benner–Damm).
  • B. C. Moore, Principal component analysis in linear systems: Controllability, observability, and model reduction, IEEE Trans. Automat. Control 26(1), 17–32, 1981. https://doi.org/10.1109/TAC.1981.1102568
8 thms3 active usersReviewed
🏆Completed
Operations ResearchProbabilityStochastic Systems·Captain: mikedeng1

Subadditive Ergodic Theory 1: Kingman's Theorem — x₀ₜ/t Converges with Probability One and in Mean to a Finite ξ with E(ξ) = γResearch Paper

Motivation

Many stochastic systems have an accumulated cost or passage time that grows with the length of an observation window. Splitting a window at an intermediate time gives an upper bound on the total, even when it does not give an equality. First-passage percolation and products of random matrices are examples discussed by Kingman (1973). A central question is whether the cost per unit time settles to a long-run rate on almost every sample path.

Hammersley and Welsh introduced subadditive stochastic processes in 1965. Kingman's 1968 theorem established the almost-sure limit for the stationary process considered here. His 1973 paper restated the result, identified the limit's expectation, and set out the process model and related formulations used in later applications. The 1973 article explicitly attributes the proof of Theorem 1, its maximal inequality, and its decomposition to the 1968 work; this mission formalizes the statements as presented in the 1973 paper.

Setting

Fix a probability space (Ω,F,P)(\Omega,\mathcal F,P)(Ω,F,P). For integers 0≤s<t0\le s<t0≤s<t, let xst:Ω→Rx_{st}:\Omega\to\mathbb Rxst​:Ω→R be a measurable random variable. Think of xstx_{st}xst​ as the accumulated quantity over the interval from sss to ttt. Condition S₁ is subadditivity: xsu≤xst+xtux_{su}\le x_{st}+x_{tu}xsu​≤xst​+xtu​ whenever s<t<us<t<us<t<u. The stronger equality defines an additive process.

Condition S₂ says that shifting every interval one time unit forward leaves the joint distribution of the whole family unchanged. It concerns the law of (xs+1,t+1)s<t(x_{s+1,t+1})_{s<t}(xs+1,t+1​)s<t​ as a single countable path. The one-dimensional statement that each xstx_{st}xst​ has a distribution depending only on t−st-st−s is called S₂′ in the paper; it does not replace S₂. Condition S₃ says that every positive-time variable x0tx_{0t}x0t​ is integrable and that, for some real AAA, its mean gt=EP(x0t)g_t=E_P(x_{0t})gt​=EP​(x0t​) satisfies gt≥−Atg_t\ge-Atgt​≥−At for all t≥1t\ge1t≥1. A subadditive process satisfies S₁, S₂, and S₃.

The associated mean growth rate is γ(x)=inf⁡t≥1gt/t\gamma(x)=\inf_{t\ge1}g_t/tγ(x)=inft≥1​gt​/t. The time index starts at one in this infimum because division by zero would introduce an unrelated value. S₃ gives a finite lower bound; the subadditive mean relation makes this infimum the limit of gt/tg_t/tgt​/t. Kingman also considers the invariant σ-field I\mathcal II: events determined by the entire path and unchanged by the shift (xst)↦(xs+1,t+1)(x_{st})\mapsto(x_{s+1,t+1})(xst​)↦(xs+1,t+1​). It records the part of the process that can remain random in the limiting rate. Source: §§1.1–1.2, pp. 883–885.

Formalization targets

Theorem 1: finite long-run rate

For every subadditive process there is an integrable real random variable ξ\xiξ such that

x0t(ω)t⟶ξ(ω)for P-almost every ω,∫Ω∣x0tt−ξ∣ dP⟶0,EP(ξ)=γ(x).\frac{x_{0t}(\omega)}{t}\longrightarrow\xi(\omega) \quad\text{for }P\text{-almost every }\omega, \qquad \int_\Omega\left|\frac{x_{0t}}{t}-\xi\right|\,dP\longrightarrow0, \qquad E_P(\xi)=\gamma(x).tx0t​(ω)​⟶ξ(ω)for P-almost every ω,∫Ω​​tx0t​​−ξ​dP⟶0,EP​(ξ)=γ(x).

These are three distinct claims: almost-sure convergence, convergence in mean, and an identity for the limit's expectation. The limit need not be a constant. The milestone list also covers the convergence of the means, the maximal inequality (1.2.5), the decomposition (1.2.6), the measure-preserving formulation (1.3.4)–(1.3.5), and the representation through I\mathcal II in (1.2.4). Each statement comes from the 1973 paper, §§1.1–1.3.

Measure-preserving formulation

For a measure-preserving map θ:Ω→Ω\theta:\Omega\to\Omegaθ:Ω→Ω, condition Sᴱ concerns integrable functions fnf_nfn​ with fm+n(ω)≤fm(ω)+fn(θmω)f_{m+n}(\omega)\le f_m(\omega)+f_n(\theta^m\omega)fm+n​(ω)≤fm​(ω)+fn​(θmω) for m,n≥1m,n\ge1m,n≥1 and a linear lower bound on their expectations. It yields an integrable ξ\xiξ with fn/n→ξf_n/n\to\xifn​/n→ξ almost surely and in mean. Neither invertibility nor ergodicity of θ\thetaθ is assumed. Source: §1.3, p. 886.

Significance

The theorem assigns a finite sample-path growth rate to an entire class of stationary subadditive systems. It also ties the average of that rate to the infimum of the normalized expected values. The equality is useful when a system's finite-window means are easier to estimate than its individual long-run paths. The invariant σ-field statement explains why a stationary system can have different rates on different paths without contradicting the mean identity. Kingman's examples show how the same framework reaches percolation and matrix products.

The result is mathematically proved, as the 1973 article states. The remaining work here is a machine-checked development of its process model and the displayed results. A published Prove2Me item, PalmQueueing.Ergodic.kingman_subadditive, is open and concerns an ergodic bijective flow with a constant possibly infinite limit; it does not supply this nonergodic finite-limit theorem or convergence in mean. The draft statements in this mission compile with proof placeholders. No machine-checked proof of these new statements is claimed.

Difficulty

Subadditivity gives bounds when intervals are joined, but it does not express x0tx_{0t}x0t​ as a sum of identically distributed increments. The ordinary additive ergodic theorem therefore does not immediately give convergence for x0t/tx_{0t}/tx0t​/t. Nor does convergence of the expectations gt/tg_t/tgt​/t alone force convergence on individual paths or in mean. The process may also have a nonconstant limit: stationarity is a joint-law condition, not an ergodicity assumption that erases invariant information. These are the gaps the formalization must close without strengthening the hypotheses. Source: §§1.1–1.3.

Formalization scope

Lean represents xxx as a function of two natural-number indices and a sample point, but every condition and conclusion uses only s<ts<ts<t. S₁ and the Sᴱ cocycle inequality follow the paper's pointwise displays; random-variable equality in the decomposition is stated almost surely at each valid index pair. Measurability of each valid coordinate is explicit. S₂ compares two probability measures on the full countable path space, so equality of individual marginals cannot satisfy it. The probability measure is normalized, and S₃ requires Bochner integrability to keep every mean finite. The constant γ\gammaγ is a real infimum over positive times, used only under S₃. The goal's limit is real and integrable; Kingman's separate Theorem 2 treats a possible value of −∞-\infty−∞ and belongs to another mission.

The invariant σ-field is the pullback of the path-space shift-invariant σ-field. Its measurability claim is made through an almost-everywhere equal version, since almost-sure limits are insensitive to changes on null sets. Conditional expectation uses this explicit sub-σ-field. All limits in this mission take positive integer time to infinity. The Sᴱ definition requires measure preservation but no inverse and no ergodicity. Thus neither a marginal-law substitution for S₂ nor an ergodic constant-limit specialization can trivialize the target.

A complete proof can reuse Mathlib's measure theory, conditional expectation, product measurable spaces, measure-preserving maps, and subadditive-sequence results. The two process definitions, the invariant σ-field interface, and auxiliary statements are useful to later formalizations of stationary growth models. Contributions that prove the stated results or develop general-purpose lemmas for these objects fit the mission.

Selected references

  • J. F. C. Kingman, Subadditive ergodic theory, The Annals of Probability 1(6), 883–899, 1973. DOI: 10.1214/aop/1176996798.
8 thms3 active usersReviewed
🏆Completed
AnalysisControl TheoryPartial Differential Equations·Captain: mikedeng1

An Example on the Effect of Time Delays in Boundary Feedback Stabilization of Wave Equations: The Spectral Stability Trichotomy at the Feedback Gain k = (1 − K)/(1 + K), K = e^{−2a}Research Paper

Why delays in boundary feedback matter

Boundary feedback is the standard way to stabilize vibrating structures governed by wave equations: a sensor measures the velocity at the boundary and an actuator applies a force proportional to it. For the one-dimensional damped wave equation with such feedback, the closed loop is uniformly exponentially stable (Chen, 1979). In practice the measured velocity reaches the actuator only after a computation time, so the feedback acts with a time delay ε>0\varepsilon > 0ε>0. The question addressed by Datko, Lagnese and Polis (1986) is whether arbitrarily small delays can destroy the stability of a system that is stable without them.

Their note answers it with an explicit example and a sharp threshold on the feedback gain. It is the founding example of delay-induced instability in boundary stabilization, cited by most later work on robustness of boundary control to delays.

Timeline.

  • Chen (1979, 1981): energy decay E(u,t)≤Ce−αtE(u,0)E(u,t) \le Ce^{-\alpha t}E(u,0)E(u,t)≤Ce−αtE(u,0) for boundary-damped wave equations without delay.
  • Datko (1978): delays in the velocity term of the equation itself can destabilize; a lemma on zeros of exponential polynomials in vertical strips (the source of Lemma 1 here).
  • Datko, Lagnese, Polis (1986, DOI 10.1137/0324007): the delayed boundary feedback ux(1,t)=−kut(1,t−ε)u_x(1,t) = -ku_t(1,t-\varepsilon)ux​(1,t)=−kut​(1,t−ε) for the damped string, with a trichotomy in the gain kkk.
  • Nicaise and Pignotti (2006, DOI 10.1137/060648891): the multidimensional undamped problem with feedback −μ1ut(t)−μ2ut(t−τ)-\mu_1u_t(t) - \mu_2u_t(t-\tau)−μ1​ut​(t)−μ2​ut​(t−τ); stability iff μ2<μ1\mu_2 < \mu_1μ2​<μ1​, instability for a sequence of delays otherwise.

Setting

Fix real numbers a≥0a \ge 0a≥0 (damping), k≥0k \ge 0k≥0 (feedback gain) and ε>0\varepsilon > 0ε>0 (delay), and put K=e−2a∈(0,1]K = e^{-2a} \in (0,1]K=e−2a∈(0,1]. The delayed system is

(1)utt−uxx+2aut+a2u=0,0<x<1, t>0,(2)u(0,t)=0,t>0,(8)ux(1,t)=−k ut(1,t−ε),t>ε.\begin{aligned} &(1)\quad u_{tt} - u_{xx} + 2au_t + a^2u = 0, && 0<x<1,\ t>0,\\ &(2)\quad u(0,t) = 0, && t>0,\\ &(8)\quad u_x(1,t) = -k\,u_t(1,t-\varepsilon), && t>\varepsilon . \end{aligned}​(1)utt​−uxx​+2aut​+a2u=0,(2)u(0,t)=0,(8)ux​(1,t)=−kut​(1,t−ε),​​0<x<1, t>0,t>0,t>ε.​

A classical solution is a C2C^2C2 function u(x,t)u(x,t)u(x,t) with complex values satisfying (1), (2), (8).

Separated solutions u=eωtφ(x)u = e^{\omega t}\varphi(x)u=eωtφ(x) lead to the eigenvalue problem

φ′′(x)=(a+ω)2φ(x) (0<x<1),φ(0)=0,φ′(1)+kωe−εωφ(1)=0.\varphi''(x) = (a+\omega)^2\varphi(x)\ (0<x<1),\qquad \varphi(0) = 0,\qquad \varphi'(1) + k\omega e^{-\varepsilon\omega}\varphi(1) = 0 .φ′′(x)=(a+ω)2φ(x) (0<x<1),φ(0)=0,φ′(1)+kωe−εωφ(1)=0.

The spectrum σ(a,k,ε)\sigma(a,k,\varepsilon)σ(a,k,ε) is the set of ω∈C\omega \in \mathbb Cω∈C for which this problem has a solution φ\varphiφ not identically zero on [0,1][0,1][0,1]. The paper encodes it through the entire functions

f(ε,ω)=1+Ke−2ω+ke−εω(1−Ke−2ω),h(ε,ω)=ωf(ε,ω)+a(1+Ke−2ω),f(\varepsilon,\omega) = 1 + Ke^{-2\omega} + ke^{-\varepsilon\omega}(1 - Ke^{-2\omega}),\qquad h(\varepsilon,\omega) = \omega f(\varepsilon,\omega) + a(1 + Ke^{-2\omega}),f(ε,ω)=1+Ke−2ω+ke−εω(1−Ke−2ω),h(ε,ω)=ωf(ε,ω)+a(1+Ke−2ω),

and the critical gain (1−K)/(1+K)(1-K)/(1+K)(1−K)/(1+K). In Lean these are K a, f a k ε ω, h a k ε ω, delaySpectrum a k ε, IsClassicalSolution a k ε u and IsExpUnstableSolution a k ε u in the namespace DatkoDelayWave.BoundaryDelay.

Formalization targets

Goal: the stability trichotomy (THEOREM, p. 153)

  1. If 0<k<1−K1+K0 < k < \frac{1-K}{1+K}0<k<1+K1−K​: for each ε>0\varepsilon > 0ε>0 there is β(ε)>0\beta(\varepsilon) > 0β(ε)>0 with
σ(a,k,ε)⊆{Re⁡ω≤−β}.\sigma(a,k,\varepsilon) \subseteq \{\operatorname{Re}\omega \le -\beta\}.σ(a,k,ε)⊆{Reω≤−β}.
  1. If k=1−K1+K>0k = \frac{1-K}{1+K} > 0k=1+K1−K​>0: for each ε>0\varepsilon > 0ε>0, σ(a,k,ε)⊆{Re⁡ω<0}\sigma(a,k,\varepsilon) \subseteq \{\operatorname{Re}\omega < 0\}σ(a,k,ε)⊆{Reω<0}, but there is a countable set R⊆(0,∞)R \subseteq (0,\infty)R⊆(0,∞), dense in (0,∞)(0,\infty)(0,∞), such that for each ε∈R\varepsilon \in Rε∈R some sequence ωn∈σ(a,k,ε)\omega_n \in \sigma(a,k,\varepsilon)ωn​∈σ(a,k,ε) has Re⁡ωn→0\operatorname{Re}\omega_n \to 0Reωn​→0.
  2. If k>1−K1+Kk > \frac{1-K}{1+K}k>1+K1−K​: there is an open set D⊆(0,∞)D \subseteq (0,\infty)D⊆(0,∞), dense in (0,∞)(0,\infty)(0,∞), such that for each ε∈D\varepsilon \in Dε∈D the system has a classical solution with ∣u(x0,t)∣≥ceγt|u(x_0,t)| \ge ce^{\gamma t}∣u(x0​,t)∣≥ceγt for some γ,c>0\gamma, c > 0γ,c>0, x0∈(0,1)x_0 \in (0,1)x0​∈(0,1) and all t≥0t \ge 0t≥0.

Milestones

In attack order: the characteristic equation (16) (ω∈σ  ⟺  h(ε,ω)=0\omega \in \sigma \iff h(\varepsilon,\omega) = 0ω∈σ⟺h(ε,ω)=0 for ω≠−a\omega \ne -aω=−a); separated solutions; Lemma 1 (one zero of fff forces infinitely many zeros of fff and hhh in every vertical strip around it); (17)–(18) (no zeros of hhh in the right half-plane below or at the critical gain); the half-plane bound below the critical gain; (19) (no purely imaginary zeros at the critical gain); the explicit zero f(2(2m+1)2n+1,(2n+1)πi2)=0f\bigl(\tfrac{2(2m+1)}{2n+1}, \tfrac{(2n+1)\pi i}{2}\bigr) = 0f(2n+12(2m+1)​,2(2n+1)πi​)=0; density of the delays 2(2m+1)2n+1\tfrac{2(2m+1)}{2n+1}2n+12(2m+1)​; the root family (12)–(15) above the critical gain; Lemma 2 (an open dense set of delays with zeros of fff in Re⁡ω>0\operatorname{Re}\omega > 0Reω>0); the three parts of the THEOREM separately; and, off the goal's path, the undelayed formula Re⁡ω=12log⁡∣k−1k+1∣\operatorname{Re}\omega = \tfrac12\log\bigl|\tfrac{k-1}{k+1}\bigr|Reω=21​log​k+1k−1​​ for a=0a = 0a=0.

Significance

The result shows that exponential stability obtained by boundary velocity feedback is not robust to delays when the gain is above (1−K)/(1+K)(1-K)/(1+K)(1−K)/(1+K): delays from a dense set of lengths, including arbitrarily small ones, produce exponentially growing modes. When a=0a = 0a=0 the critical gain is 000, so every positive gain is fragile. Below the critical gain the spectral gap survives every delay. This identifies a quantitative robustness margin and motivated the later analysis of delayed boundary control, including the Nicaise–Pignotti theorems already posed on the platform for a different, multidimensional model (NicaiseDelayWave.*).

The result is proved in the paper, partly by citation: Lemma 1 is quoted from Datko (1978) without proof, and the open-mapping step of Lemma 2 is sketched. No machine-checked version exists. A formalization would check the threshold computations, supply the strip-zero lemma for this exponential polynomial, and give a rigorous version of the open-mapping step.

Difficulty

The algebraic parts, (16)–(19) and the explicit zeros, are calculations with complex exponentials. The difficulty is in Lemma 1 and Lemma 2. Lemma 1 needs the almost-periodic structure of exponential polynomials in vertical strips together with a Hurwitz or Rouché type argument, and neither is in Mathlib. Lemma 2 needs zeros of f(ε,⋅)f(\varepsilon,\cdot)f(ε,⋅) to persist under small changes of the real parameter ε\varepsilonε. The paper's route through the logarithmic map (11) does not work as printed, because it is stated with the principal branch exactly on the branch cut. In part (i), the delicate step is excluding accumulation of zeros at the imaginary axis as ∣Im⁡ω∣→∞|\operatorname{Im}\omega| \to \infty∣Imω∣→∞, which the half-plane estimate alone does not give.

Formalization scope

Parameters a,k,εa, k, \varepsilona,k,ε are real, ω∈C\omega \in \mathbb Cω∈C, eigenfunctions are maps R→C\mathbb R \to \mathbb CR→C and solutions are maps u:R→R→Cu : \mathbb R \to \mathbb R \to \mathbb Cu:R→R→C (space first). Partial derivatives are deriv of the sections. Solutions and eigenfunctions are required to be C2C^2C2 on all of R2\mathbb R^2R2, resp. R\mathbb RR; this loses no eigenvalue, since solutions of (5) are combinations of e±(a+ω)xe^{\pm(a+\omega)x}e±(a+ω)x. Committed readings of the paper's loose phrases:

  • the spectrum is the eigenvalue set of the problem above, not the zero set of hhh; h(ε,−a)=0h(\varepsilon,-a) = 0h(ε,−a)=0 always, so (16) is stated for ω≠−a\omega \neq -aω=−a;
  • "countably dense set RRR in (0,∞)(0,\infty)(0,∞)": R⊆(0,∞)R \subseteq (0,\infty)R⊆(0,∞), RRR countable, (0,∞)⊆R‾(0,\infty) \subseteq \overline R(0,∞)⊆R;
  • "dense open set DDD in (0,∞)(0,\infty)(0,∞)": D⊆(0,∞)D \subseteq (0,\infty)D⊆(0,∞) open, (0,∞)⊆D‾(0,\infty) \subseteq \overline D(0,∞)⊆D;
  • "exponentially unstable solution": a classical solution with ∣u(x0,t)∣≥ceγt|u(x_0,t)| \ge ce^{\gamma t}∣u(x0​,t)∣≥ceγt for all t≥0t \ge 0t≥0;
  • "an infinite number of zeros" (Lemma 1): the zero set in the open strip is infinite, for fff and for hhh separately;
  • "there exists β(ε)>0\beta(\varepsilon) > 0β(ε)>0": ∀ε>0 ∃β>0\forall \varepsilon > 0\ \exists \beta > 0∀ε>0 ∃β>0, so β\betaβ may depend on ε\varepsilonε;
  • part (ii) carries k>0k > 0k>0, the paper's standing assumption (p. 152); without it the part is false at a=0a = 0a=0;
  • m,nm, nm,n in 2(2m+1)/(2n+1)2(2m+1)/(2n+1)2(2m+1)/(2n+1) are positive integers in (12)–(15) and in the density of (15), as in the proof of Lemma 2; in the explicit threshold zero, where the page says "m,nm, nm,n are arbitrary", they range over all natural numbers.

Trivializing encodings are ruled out: defining the spectrum as the zero set of hhh would make −a-a−a a spurious spectral point, and reading "exponentially unstable" as "the spectrum meets Re⁡ω>0\operatorname{Re}\omega > 0Reω>0" would delete the solution-level content of part (iii).

A complete development needs zero-counting tools for exponential polynomials in strips (almost periodicity, Hurwitz's theorem), which would be reusable for other delay equations. Contributions to any milestone, and to that general infrastructure, are welcome.

Selected references

  • R. Datko, J. Lagnese, M. P. Polis, An example on the effect of time delays in boundary feedback stabilization of wave equations, SIAM J. Control Optim. 24(1), 152–156, 1986. https://doi.org/10.1137/0324007
  • R. Datko, A procedure for determination of the exponential stability of certain differential-difference equations, Quart. Appl. Math. 36, 279–292, 1978.
  • G. Chen, Energy decay estimates and exact boundary value controllability for the wave equation in a bounded domain, J. Math. Pures Appl. 58, 249–274, 1979.
  • S. Nicaise, C. Pignotti, Stability and instability results of the wave equation with a delay term in the boundary or internal feedbacks, SIAM J. Control Optim. 45(5), 1561–1585, 2006. https://doi.org/10.1137/060648891
16 thms3 active usersReviewed
🏆Completed
Operations ResearchStochastic Systems·Captain: mikedeng1

Stability and Instability of Fluid Models for Reentrant Lines 1: Every Work-Conserving Fluid Model of the Three-Buffer Line 1→2→1 Is Stable When m₁ + m₃ < 1 and m₂ < 1Research Paper

Why reentrant lines, and why this one

A reentrant line is a queueing network in which every job follows the same route and visits some stations more than once. It is the standard model of a semiconductor wafer fab, where a wafer returns to the same lithography station for each of its layers (Kumar, Re-entrant lines, Queueing Systems 13, 1993). Because jobs at different stages compete for the same server, a network can be unstable (queues grow without bound) even though every station has nominal load below one; the Lu–Kumar and Rybko–Stolyar examples of the early 1990s made this concrete. Deciding stability under the usual load condition therefore needs an argument specific to the network and to the scheduling policy.

Fluid models turn that question into one about deterministic dynamics. Dai (Dai, On positive Harris recurrence of multiclass queueing networks: a unified approach via fluid limit models, Ann. Appl. Probab. 5, 1995) showed that if the fluid model of a queueing discipline is stable, the queueing network under that discipline is positive Harris recurrent. Dai and Weiss (1996) then proved fluid stability, or instability, for several classes of reentrant lines. This mission formalizes their first result, about the smallest reentrant line that revisits a station after visiting another one.

Timeline:

  • 1993: Kumar conjectured that the three-buffer line 1→2→11 \to 2 \to 11→2→1 is stable under the first-in-first-out (FIFO) discipline whenever the load condition holds, for exponential distributions.
  • 1993: Wang proved that the FIFO fluid model of this line is stable, which with Dai's theorem confirms the conjecture.
  • 1996: Dai and Weiss (Theorem 3.1) proved stability of the fluid model for every work-conserving discipline, with an explicit emptying time.

Setting

A reentrant line has III stations and KKK classes. Fluid enters as class 111 at rate one; class kkk is served at station σ(k)\sigma(k)σ(k) with mean service time mk>0m_k > 0mk​>0 and service rate μk=1/mk\mu_k = 1/m_kμk​=1/mk​, and on completion becomes class k+1k+1k+1 (class KKK leaves). The constituency of station iii is Ci={k:σ(k)=i}C_i = \{k : \sigma(k) = i\}Ci​={k:σ(k)=i}, and its nominal workload is ρi=∑k∈Cimk\rho_i = \sum_{k\in C_i} m_kρi​=∑k∈Ci​​mk​.

A fluid model solution is a pair of paths Q(t)∈RKQ(t) \in \mathbb R^KQ(t)∈RK (fluid levels) and T(t)∈RKT(t) \in \mathbb R^KT(t)∈RK (cumulative time spent serving each class) such that, for t≥0t \ge 0t≥0:

Qk(t)=Qk(0)+μk−1Tk−1(t)−μkTk(t)(μ0T0(t)=t),Qk(t)≥0,Q_k(t) = Q_k(0) + \mu_{k-1}T_{k-1}(t) - \mu_k T_k(t)\quad(\mu_0T_0(t) = t),\qquad Q_k(t) \ge 0,Qk​(t)=Qk​(0)+μk−1​Tk−1​(t)−μk​Tk​(t)(μ0​T0​(t)=t),Qk​(t)≥0,

each TkT_kTk​ starts at 000 and is nondecreasing, and the idle time Ui(t)=t−Bi(t)U_i(t) = t - B_i(t)Ui​(t)=t−Bi​(t), with busy time Bi(t)=∑k∈CiTk(t)B_i(t) = \sum_{k \in C_i} T_k(t)Bi​(t)=∑k∈Ci​​Tk​(t), is nondecreasing. These are the paper's equations (1.8)–(1.12). The solution is work conserving if, in addition, (1.13): UiU_iUi​ increases only at times when station iii holds no fluid. The immediate volume of station iii is Wi(t)=∑k∈CimkQk(t)W_i(t) = \sum_{k \in C_i} m_k Q_k(t)Wi​(t)=∑k∈Ci​​mk​Qk​(t).

A set of fluid model solutions is stable (Definition 1.3) if there is a δ>0\delta > 0δ>0 such that every solution with ∣Q(0)∣=∑kQk(0)=1|Q(0)| = \sum_k Q_k(0) = 1∣Q(0)∣=∑k​Qk​(0)=1 has Q(t)=0Q(t) = 0Q(t)=0 for all t≥δt \ge \deltat≥δ.

The three-buffer line of Figure 1 has I=2I = 2I=2, K=3K = 3K=3 and route 1→2→11 \to 2 \to 11→2→1: classes 111 and 333 at station 111, class 222 at station 222. The load condition (3.1) is

ρ1=m1+m3<1,ρ2=m2<1.\rho_1 = m_1 + m_3 < 1, \qquad \rho_2 = m_2 < 1 .ρ1​=m1​+m3​<1,ρ2​=m2​<1.

Formalization targets

Goal: Theorem 3.1

m1,m2,m3>0,  m1+m3<1,  m2<1 ⟹ the work-conserving fluid model (1.8)–(1.13) of 1→2→1 is stable.m_1, m_2, m_3 > 0,\ \ m_1 + m_3 < 1,\ \ m_2 < 1 \ \Longrightarrow\ \text{the work-conserving fluid model (1.8)–(1.13) of } 1 \to 2 \to 1 \text{ is stable.}m1​,m2​,m3​>0,  m1​+m3​<1,  m2​<1 ⟹ the work-conserving fluid model (1.8)–(1.13) of 1→2→1 is stable.

The goal fixes no emptying time: it asserts only that some δ>0\delta > 0δ>0 works, which is Definition 1.3.

Milestones, in attack order

  1. Lemma 2.2 (ii): a nonnegative absolutely continuous ggg with g˙(t)≤−ε\dot g(t) \le -\varepsilong˙​(t)≤−ε almost everywhere where g(t)>0g(t) > 0g(t)>0 vanishes from g(0)/εg(0)/\varepsilong(0)/ε on, and is nonincreasing.
  2. Display (3.2) (already proved on the platform): at a regular point, the maximum of finitely many functions has the derivative of every component that attains it.
  3. Lemma 3.2: for nonnegative linear functions GiG_iGi​ of QQQ with (a) G˙i≤−εi\dot G_i \le -\varepsilon_iG˙i​≤−εi​ while Wi>0W_i > 0Wi​>0 and (b) Gi≤min⁡j≠iGjG_i \le \min_{j \ne i} G_jGi​≤minj=i​Gj​ while Wi=0W_i = 0Wi​=0, the maximum GGG is absolutely continuous and nonnegative, and G˙(t)≤−min⁡iεi\dot G(t) \le -\min_i \varepsilon_iG˙(t)≤−mini​εi​ at regular points with G(t)>0G(t) > 0G(t)>0.
  4. Drift identity (proof of Theorem 3.1): with θ=m1/(m1+m3)\theta = m_1/(m_1+m_3)θ=m1​/(m1​+m3​), G1=θQ1++(1−θ)Q3+G_1 = \theta Q_1^+ + (1-\theta)Q_3^+G1​=θQ1+​+(1−θ)Q3+​ and G2=Q2+G_2 = Q_2^+G2​=Q2+​, where Qk+=∑l≤kQlQ_k^+ = \sum_{l \le k} Q_lQk+​=∑l≤k​Ql​, one has Gi(t)=Gi(0)+t−Bi(t)/ρiG_i(t) = G_i(0) + t - B_i(t)/\rho_iGi​(t)=Gi​(0)+t−Bi​(t)/ρi​.
  5. Drift rate: G˙i(t)=−(1/ρi−1)<0\dot G_i(t) = -(1/\rho_i - 1) < 0G˙i​(t)=−(1/ρi​−1)<0 whenever Wi(t)>0W_i(t) > 0Wi​(t)>0.
  6. Condition (b): W1(t)=0⇒G1(t)≤G2(t)W_1(t) = 0 \Rightarrow G_1(t) \le G_2(t)W1​(t)=0⇒G1​(t)≤G2​(t) and W2(t)=0⇒G2(t)≤G1(t)W_2(t) = 0 \Rightarrow G_2(t) \le G_1(t)W2​(t)=0⇒G2​(t)≤G1​(t).
  7. Emptying time: every work-conserving solution with ∣Q(0)∣=1|Q(0)| = 1∣Q(0)∣=1 is empty from max⁡{ρ1/(1−ρ1),ρ2/(1−ρ2)}\max\{\rho_1/(1-\rho_1), \rho_2/(1-\rho_2)\}max{ρ1​/(1−ρ1​),ρ2​/(1−ρ2​)} on.

Significance

Theorem 3.1 settles stability of the three-buffer line for every work-conserving discipline at once, not only FIFO. Through Dai's 1995 theorem it gives positive Harris recurrence of the queueing network under any such discipline whose fluid limits satisfy (1.8)–(1.13), under that theorem's distributional assumptions. By the paper's remark after the proof, every other three-buffer reentrant line is feedforward, so with this theorem all three-buffer reentrant lines are stable under every work-conserving policy. The method, a Lyapunov function that is the maximum of linear functions of the fluid levels, recurs in the paper's later theorems (Lu–Kumar network, two-station Kelly-type lines) and in the wider fluid-stability literature.

The result is proved on paper; no machine-checked version exists. The mission's contribution is a formal fluid-model layer for reentrant lines (equations (1.8)–(1.13), Definition 1.3) shared with the other missions of this series, two reusable real-analysis lemmas (Lemma 2.2 (ii) and Lemma 3.2), and a complete formal proof of Theorem 3.1. The fluid-limit theorem linking fluid stability to the stochastic network is cited, not formalized.

Difficulty

The obvious approach is to split cases on the relative loads, as the paper notes (m1+m3/m2<1m_1 + m_3/m_2 < 1m1​+m3​/m2​<1 or not), and track the fluid explicitly; this gives sharp emptying times but requires following solutions through regime changes, which is unwieldy when the discipline is arbitrary. The Lyapunov approach avoids this but moves the difficulty into analysis: the fluid paths are only Lipschitz, so derivatives exist only almost everywhere; the maximum of two Lyapunov components is not differentiable where they cross; and work conservation is a statement about where the idle time can increase, which has to be converted into "the busy time grows at rate one" at points where a station holds fluid. Lemma 2.2 (ii) itself needs the fundamental theorem of calculus for absolutely continuous functions.

Formalization scope

All declarations live in DaiWeissFluid.ThreeBuffer. Committed conventions:

  • Classes and stations are 0-based (Fin 3, Fin 2). The paper's class kkk is Lean k - 1; the line is threeBuffer m with station map ![0, 1, 0], and (3.1) reads m 0 + m 2 < 1, m 1 < 1.
  • Paths are total functions ℝ → Fin K → ℝ; every equation is imposed only for t≥0t \ge 0t≥0, and derivatives are taken at t>0t > 0t>0 (HasDerivAt).
  • Work conservation (1.13) is in interval form: UiU_iUi​ is constant on every interval [s,t]⊆[0,∞)[s,t] \subseteq [0,\infty)[s,t]⊆[0,∞) on which station iii holds fluid throughout. This is equivalent to the paper's integral condition for continuous paths.
  • No Lipschitz continuity is assumed; it follows from (1.10)–(1.12).
  • mk>0m_k > 0mk​>0 is an explicit hypothesis (the paper takes it for granted). ∣Q(0)∣=∑kQk(0)|Q(0)| = \sum_k Q_k(0)∣Q(0)∣=∑k​Qk​(0).
  • Lemma 2.2 (ii) is stated with g˙(t)≤−ε\dot g(t) \le -\varepsilong˙​(t)≤−ε; the paper prints <<<, but every application uses ≤\le≤, and the stated version is the stronger lemma.
  • Lemma 3.2 is stated for any reentrant line with I≥1I \ge 1I≥1 stations, assuming only (1.8)–(1.12), as in the paper.

A trivializing formalization is ruled out: the work-conserving solution set of the three-buffer line is nonempty for every mmm satisfying (3.1), with ∣Q(0)∣=1|Q(0)| = 1∣Q(0)∣=1 (a sorry-free witness was checked), so the goal is not vacuous, and the goal quantifies over all work-conserving solutions rather than one discipline.

Needed infrastructure: Lipschitz and absolute continuity of the fluid paths, the a.e. fundamental theorem of calculus for absolutely continuous functions (in Mathlib), and the derivative of a finite maximum at a regular point (on the platform as display (3.2)). Lemma 2.2 (ii) and Lemma 3.2 are reusable for the other missions of the series. Proofs of any milestone are welcome independently.

Selected references

  • J. G. Dai and G. Weiss, Stability and instability of fluid models for reentrant lines, Mathematics of Operations Research 21(1), 115–134, 1996. https://doi.org/10.1287/moor.21.1.115
  • J. G. Dai, On positive Harris recurrence of multiclass queueing networks: a unified approach via fluid limit models, Annals of Applied Probability 5(1), 49–77, 1995. https://doi.org/10.1214/aoap/1177004828
  • P. R. Kumar, Re-entrant lines, Queueing Systems 13, 87–110, 1993. https://doi.org/10.1007/BF01158927
  • A. N. Rybko and A. L. Stolyar, Ergodicity of stochastic processes describing the operation of open queueing networks, Problems of Information Transmission 28, 199–220, 1992.
  • D. D. Botvich and A. A. Zamyatin, Ergodicity of conservative communication networks, Rapport de recherche 1772, INRIA, 1992.
10 thms3 active usersReviewed
🏆Completed
Operations ResearchProbabilityStochastic Systems·Captain: mikedeng1

Dynamics of Stochastic Approximation Algorithms 4: Subgaussian Martingale Noise with Σ exp(−c/γ_n) < ∞ for Every c > 0 Satisfies Assumption A1 Almost SurelyResearch Paper

Motivation

A stochastic approximation algorithm is a recursion

xn+1−xn=γn+1(F(xn)+Un+1)x_{n+1}-x_n=\gamma_{n+1}\big(F(x_n)+U_{n+1}\big)xn+1​−xn​=γn+1​(F(xn​)+Un+1​)

in Rd\mathbb R^dRd, where FFF is a vector field, γn\gamma_nγn​ are small step sizes and Un+1U_{n+1}Un+1​ is noise. Such recursions go back to Robbins and Monro's root-finding scheme (Robbins–Monro 1951) and underlie stochastic gradient descent, temporal-difference learning, adaptive control and learning in games. The ODE method studies them by comparing the iterates with the trajectories of x˙=F(x)\dot x=F(x)x˙=F(x).

Benaïm's lecture notes (Benaïm 1999) organize the ODE method in two steps. A deterministic step, Proposition 4.1, shows that whenever the noise satisfies a condition called A1 (together with a boundedness condition on the iterates), the interpolated process is an asymptotic pseudotrajectory of the flow of FFF. A probabilistic step then verifies A1 for concrete noise models. Proposition 4.2 does this for martingale difference noise with bounded qqq-th moments, at the price of step sizes with ∑nγn1+q/2<∞\sum_n\gamma_n^{1+q/2}<\infty∑n​γn1+q/2​<∞. This mission formalizes the second verification, Proposition 4.4: when the noise is subgaussian, A1 holds almost surely under the much weaker requirement that ∑ne−c/γn<∞\sum_ne^{-c/\gamma_n}<\infty∑n​e−c/γn​<∞ for every c>0c>0c>0, which allows step sizes decaying only slightly faster than 1/log⁡n1/\log n1/logn. The notes attribute the result to Duflo (1997), see also Kushner and Yin (1997) and Benaïm and Hirsch (1996).

Setting

Let {γn}n≥1\{\gamma_n\}_{n\ge1}{γn​}n≥1​ be a deterministic sequence with γn≥0\gamma_n\ge0γn​≥0, ∑nγn=∞\sum_n\gamma_n=\infty∑n​γn​=∞ and γn→0\gamma_n\to0γn​→0 (a step sequence). Put τ0=0\tau_0=0τ0​=0, τn=∑i=1nγi\tau_n=\sum_{i=1}^n\gamma_iτn​=∑i=1n​γi​, and let

m(t)=sup⁡{k≥0: t≥τk}m(t)=\sup\{k\ge0:\ t\ge\tau_k\}m(t)=sup{k≥0: t≥τk​}

be the index of the step that contains time t≥0t\ge0t≥0. For a sequence {Un}n≥1\{U_n\}_{n\ge1}{Un​}n≥1​ define the piecewise constant processes Uˉ(t)=Um(t)+1\bar U(t)=U_{m(t)+1}Uˉ(t)=Um(t)+1​ and γˉ(t)=γm(t)+1\bar\gamma(t)=\gamma_{m(t)+1}γˉ​(t)=γm(t)+1​, so that step n+1n+1n+1 occupies the time interval [τn,τn+1)[\tau_n,\tau_{n+1})[τn​,τn+1​) of length γn+1\gamma_{n+1}γn+1​.

Assumption A1 asks that for every T>0T>0T>0

lim⁡n→∞sup⁡{∥∑i=nk−1γi+1Ui+1∥: k=n+1,…,m(τn+T)}=0,\lim_{n\to\infty}\sup\Big\{\Big\|\sum_{i=n}^{k-1}\gamma_{i+1}U_{i+1}\Big\|:\ k=n+1,\dots,m(\tau_n+T)\Big\}=0,n→∞lim​sup{​i=n∑k−1​γi+1​Ui+1​​: k=n+1,…,m(τn​+T)}=0,

or, in the form the notes call equivalent, lim⁡t→∞Δ(t,T)=0\lim_{t\to\infty}\Delta(t,T)=0limt→∞​Δ(t,T)=0 for every T>0T>0T>0, where

Δ(t,T)=sup⁡0≤h≤T∥∫tt+hUˉ(s) ds∥.\Delta(t,T)=\sup_{0\le h\le T}\Big\|\int_t^{t+h}\bar U(s)\,ds\Big\|.Δ(t,T)=0≤h≤Tsup​​∫tt+h​Uˉ(s)ds​.

Let (Ω,F,P)(\Omega,\mathcal F,P)(Ω,F,P) be a probability space with a nondecreasing sequence {Fn}\{\mathcal F_n\}{Fn​} of sub-σ\sigmaσ-algebras, and F:Rd→RdF:\mathbb R^d\to\mathbb R^dF:Rd→Rd continuous. A sequence {xn}\{x_n\}{xn​} given by the recursion above is a Robbins–Monro algorithm if γ\gammaγ is deterministic, UnU_nUn​ is Fn\mathcal F_nFn​-measurable, and E(Un+1∣Fn)=0E(U_{n+1}\mid\mathcal F_n)=0E(Un+1​∣Fn​)=0. The noise is subgaussian if there is a number Γ>0\Gamma>0Γ>0 such that for all nnn and all θ∈Rd\theta\in\mathbb R^dθ∈Rd

E(exp⁡⟨θ,Un+1⟩ ∣ Fn)≤exp⁡(Γ2∥θ∥2).E\big(\exp\langle\theta,U_{n+1}\rangle\,\big|\,\mathcal F_n\big)\le\exp\Big(\frac\Gamma2\|\theta\|^2\Big).E(exp⟨θ,Un+1​⟩​Fn​)≤exp(2Γ​∥θ∥2).

Bounded noise, ∥Un∥≤Γ\|U_n\|\le\sqrt\Gamma∥Un​∥≤Γ​, is an example.

Formalization targets

Goal: Proposition 4.4

For a Robbins–Monro algorithm with subgaussian noise and a deterministic step sequence such that

∑ne−c/γn<∞for each c>0,\sum_ne^{-c/\gamma_n}<\infty\qquad\text{for each }c>0,n∑​e−c/γn​<∞for each c>0,

with probability one the realised noise sequence satisfies A1, in both of its forms, simultaneously for all T>0T>0T>0.

Milestones

  1. The exponential supermartingale. For every θ∈Rd\theta\in\mathbb R^dθ∈Rd,
Zn(θ)=exp⁡[∑i=1n⟨θ,γiUi⟩−Γ2∑i=1nγi2∥θ∥2]Z_n(\theta)=\exp\Big[\sum_{i=1}^n\langle\theta,\gamma_iU_i\rangle-\frac\Gamma2\sum_{i=1}^n\gamma_i^2\|\theta\|^2\Big]Zn​(θ)=exp[i=1∑n​⟨θ,γi​Ui​⟩−2Γ​i=1∑n​γi2​∥θ∥2]

is a supermartingale. 2. Directional maximal tail bound. For every unit vector eee, α>0\alpha>0α>0, nnn and T>0T>0T>0,

P(sup⁡n<k≤m(τn+T)⟨e,∑i=nk−1γi+1Ui+1⟩≥α)≤exp⁡(−α22Γ∑i=nm(τn+T)−1γi+12).P\Big(\sup_{n<k\le m(\tau_n+T)}\Big\langle e,\sum_{i=n}^{k-1}\gamma_{i+1}U_{i+1}\Big\rangle\ge\alpha\Big)\le\exp\Big(\frac{-\alpha^2}{2\Gamma\sum_{i=n}^{m(\tau_n+T)-1}\gamma_{i+1}^2}\Big).P(n<k≤m(τn​+T)sup​⟨e,i=n∑k−1​γi+1​Ui+1​⟩≥α)≤exp(2Γ∑i=nm(τn​+T)−1​γi+12​−α2​).
  1. Eq. (18). There are C,C′>0C,C'>0C,C′>0 depending only on ddd and Γ\GammaΓ with
P(Δ(t,T)≥α)≤Cexp⁡(−α2C′∫tt+Tγˉ(s) ds)(t≥0, T>0, α>0).P(\Delta(t,T)\ge\alpha)\le C\exp\Big(\frac{-\alpha^2}{C'\int_t^{t+T}\bar\gamma(s)\,ds}\Big)\qquad(t\ge0,\ T>0,\ \alpha>0).P(Δ(t,T)≥α)≤Cexp(C′∫tt+T​γˉ​(s)ds−α2​)(t≥0, T>0, α>0).
  1. Block comparison. Δ(t,T)≤2Δ(kT,T)+Δ((k+1)T,T)\Delta(t,T)\le2\Delta(kT,T)+\Delta((k+1)T,T)Δ(t,T)≤2Δ(kT,T)+Δ((k+1)T,T) for kT≤t<(k+1)TkT\le t<(k+1)TkT≤t<(k+1)T.

Significance

Proposition 4.4 is the sufficient condition for the ODE method when the noise has Gaussian-type tails. Its step-size condition holds whenever γnlog⁡n→0\gamma_n\log n\to0γn​logn→0, so it admits steps that decrease far more slowly than the ∑γn2<∞\sum\gamma_n^2<\infty∑γn2​<∞ of the classical L2L^2L2 theory; slowly decreasing steps are what practitioners use to keep algorithms responsive. Combined with Proposition 4.1 it shows that the interpolated process of such an algorithm, with bounded iterates, is almost surely an asymptotic pseudotrajectory of the flow of FFF, and the limit set theorems of the notes then locate the limit points of the algorithm.

The result is proved in the notes and in the cited literature; it has not, to our knowledge, been machine-checked. A formal proof would add reusable pieces: an exponential supermartingale and maximal inequality for vector-valued martingale differences with a conditional subgaussian bound (Mathlib's conditional subgaussian notion is scalar), a Borel–Cantelli argument along the grid kTkTkT, and the continuous-time bookkeeping of Uˉ\bar UUˉ, γˉ\bar\gammaγˉ​ and Δ\DeltaΔ shared with the other missions of this series.

Difficulty

The moment method of Proposition 4.2 does not reach this regime: any fixed polynomial moment of the window sums decays only polynomially in the window's step sizes, and under ∑e−c/γn<∞\sum e^{-c/\gamma_n}<\infty∑e−c/γn​<∞ alone polynomial bounds are not summable over windows. Exponential tail bounds are needed, and they must be maximal (uniform over the window) and must hold for the norm of a vector, not only for a scalar. The continuous-time deviation Δ(t,T)\Delta(t,T)Δ(t,T) involves partial steps at both ends of [t,t+h][t,t+h][t,t+h], so the bound must be stated in terms of ∫tt+Tγˉ\int_t^{t+T}\bar\gamma∫tt+T​γˉ​ rather than a sum over whole steps, with constants that do not depend on ttt, TTT or α\alphaα. Finally, A1 quantifies over all T>0T>0T>0: the almost-sure statement must hold on a single event of full probability for every TTT.

Formalization scope

The space is Rd\mathbb R^dRd as EuclideanSpace ℝ (Fin d) (the paper writes Rm\mathbb R^mRm); time is real. The sequences γ\gammaγ and UUU are indexed by N\mathbb NN, and their values at 000 are unused, as the paper indexes them from 111. The filtration is a Mathlib Filtration ℕ; Un+1U_{n+1}Un+1​ is Fn+1\mathcal F_{n+1}Fn+1​-strongly measurable and integrable, and E(Un+1∣Fn)=0E(U_{n+1}\mid\mathcal F_n)=0E(Un+1​∣Fn​)=0 almost surely. The subgaussian condition requires exp⁡⟨θ,Un+1⟩\exp\langle\theta,U_{n+1}\rangleexp⟨θ,Un+1​⟩ to be integrable for every θ\thetaθ and nnn. The summand e−c/γne^{-c/\gamma_n}e−c/γn​ is taken to be 000 when γn=0\gamma_n=0γn​=0, its limiting value. The suprema in A1 and Δ\DeltaΔ are taken in [0,∞][0,\infty][0,∞]; the supremum over an empty range of kkk is 000. In Eq. (18) the constants are chosen before the probability space, the algorithm and t,T,αt,T,\alphat,T,α.

The following readings are excluded and are not acceptable formalizations: a subgaussian condition that holds vacuously because the exponential is not integrable (Lean's conditional expectation of a non-integrable function is 000); a summability condition made trivial or false by the convention c/0=0c/0=0c/0=0; and the conclusion "for each TTT, A1 holds almost surely" in place of "almost surely, A1 holds for all TTT". The second sentence of Proposition 4.4 (the asymptotic pseudotrajectory conclusion) is outside this mission.

All hypotheses are satisfiable: U=0U=0U=0, x=0x=0x=0, F=0F=0F=0, Γ=1\Gamma=1Γ=1 and γn=1/n\gamma_n=1/nγn​=1/n satisfy every one of them.

Contributions welcome: a maximal inequality for nonnegative supermartingales in the form needed here, vector subgaussian tail bounds for martingale transforms with deterministic weights (reusable well beyond this mission), lemmas on the step processes and Δ\DeltaΔ (measurability, local integrability, additivity), and the proofs of the milestones.

Selected references

  • M. Benaïm, Dynamics of Stochastic Approximation Algorithms, Séminaire de Probabilités XXXIII, Lecture Notes in Mathematics 1709, Springer, 1999, pp. 1–68. https://doi.org/10.1007/BFb0096509
  • M. Duflo, Random Iterative Models, Applications of Mathematics 34, Springer, 1997.
  • H. J. Kushner and G. G. Yin, Stochastic Approximation Algorithms and Applications, Springer, 1997.
  • M. Benaïm and M. W. Hirsch, Asymptotic pseudotrajectories and chain recurrent flows, with applications, Journal of Dynamics and Differential Equations 8 (1996), 141–176. https://doi.org/10.1007/BF02218617
  • H. Robbins and S. Monro, A stochastic approximation method, Annals of Mathematical Statistics 22 (1951), 400–407. https://doi.org/10.1214/aoms/1177729586
10 thms3 active usersReviewed
🏆Completed
Operations ResearchProbabilityStochastic Systems·Captain: mikedeng1

Dynamics of Stochastic Approximation Algorithms 6: An Attractor Whose Basin Meets the Attainable Set Contains the Limit Set with Positive ProbabilityResearch Paper

Motivation

A stochastic approximation algorithm is a recursion xn+1=xn+γn+1(F(xn)+Un+1)x_{n+1}=x_n+\gamma_{n+1}\big(F(x_n)+U_{n+1}\big)xn+1​=xn​+γn+1​(F(xn​)+Un+1​) with decreasing step sizes γn\gamma_nγn​ and a noise term Un+1U_{n+1}Un+1​. Recursions of this form appear in stochastic gradient methods, adaptive control, learning in games (fictitious play, reinforcement learning) and urn models. The ODE method compares the iterates with the solutions of x˙=F(x)\dot x=F(x)x˙=F(x). In Benaïm's lecture notes (Benaïm 1999), the comparison is phrased through the continuous-time interpolated process XXX. Under standard noise conditions, XXX is almost surely an asymptotic pseudotrajectory of the flow of FFF, and its limit set is almost surely internally chain transitive.

That theorem constrains where the process may end up. It does not say which of several candidate sets the process actually reaches. When the ODE has several attractors, for example several stable equilibria of a learning dynamic or several stable compositions of an urn, an application needs to know that each attractor is reached with positive probability. Section 7 of the notes answers this question. The answer is a criterion of attainability: if the process can, with positive probability and at arbitrarily late times, enter the basin of an attractor, then it converges to that attractor with positive probability.

Timeline.

  • Kushner and Clark (1978) proved convergence statements for processes that visit a compact subset of the domain of attraction of an asymptotically stable equilibrium infinitely often.
  • Arthur, Ermoliev and Kaniovski (1983) and Pemantle (1990) studied urn processes whose limit points depend on the trajectory.
  • Benaïm (1997) and Duflo (1997, Random Iterative Models) developed the attainability argument for general stochastic approximation processes.
  • Benaïm (1999) states it for arbitrary attractors of a semiflow on a locally compact metric space, under a single conditional shadowing condition (24).

Setting

Let (M,d)(M,d)(M,d) be a metric space and let Φ=(Φt)t≥0\Phi=(\Phi_t)_{t\ge0}Φ=(Φt​)t≥0​ be a semiflow on MMM: a continuous map (t,x)↦Φt(x)(t,x)\mapsto\Phi_t(x)(t,x)↦Φt​(x) with Φ0=Id\Phi_0=\mathrm{Id}Φ0​=Id and Φt+s=Φt∘Φs\Phi_{t+s}=\Phi_t\circ\Phi_sΦt+s​=Φt​∘Φs​.

  • A set AAA is invariant if Φt(A)=A\Phi_t(A)=AΦt​(A)=A for all t≥0t\ge0t≥0, and positively invariant if Φt(A)⊂A\Phi_t(A)\subset AΦt​(A)⊂A.
  • An attractor is a nonempty compact invariant set AAA with a neighbourhood WWW on which dist⁡(Φtx,A)→0\operatorname{dist}(\Phi_tx,A)\to0dist(Φt​x,A)→0 uniformly. Its basin B(A)B(A)B(A) is the set of points xxx with dist⁡(Φtx,A)→0\operatorname{dist}(\Phi_tx,A)\to0dist(Φt​x,A)→0.
  • A continuous curve X:R+→MX:\mathbb R_+\to MX:R+​→M is an asymptotic pseudotrajectory if sup⁡0≤h≤Td(X(t+h),Φh(X(t)))→0\sup_{0\le h\le T}d(X(t+h),\Phi_h(X(t)))\to0sup0≤h≤T​d(X(t+h),Φh​(X(t)))→0 as t→∞t\to\inftyt→∞, for every T>0T>0T>0.
  • The limit set of XXX is L(X)=⋂t≥0X([t,∞))‾L(X)=\bigcap_{t\ge0}\overline{X([t,\infty))}L(X)=⋂t≥0​X([t,∞))​.
  • For T>0T>0T>0, dX(T)=sup⁡k∈Nd(ΦT(X(kT)),X(kT+T))d_X(T)=\sup_{k\in\mathbb N}d(\Phi_T(X(kT)),X(kT+T))dX​(T)=supk∈N​d(ΦT​(X(kT)),X(kT+T)).

Now let X=(X(t))t≥0X=(X(t))_{t\ge0}X=(X(t))t≥0​ be a process on a probability space (Ω,F,P)(\Omega,\mathcal F,P)(Ω,F,P) with continuous paths in MMM, adapted to a filtration (Ft)t≥0(\mathcal F_t)_{t\ge0}(Ft​)t≥0​. The standing assumption of Section 7 is that for all δ>0\delta>0δ>0, T>0T>0T>0 and t≥0t\ge0t≥0,

P(sup⁡s≥t sup⁡0≤h≤Td(X(s+h),Φh(X(s)))≥δ ∣ Ft)≤w(t,δ,T)(24)P\Big(\sup_{s\ge t}\ \sup_{0\le h\le T}d\big(X(s+h),\Phi_h(X(s))\big)\ge\delta\ \Big|\ \mathcal F_t\Big)\le w(t,\delta,T)\tag{24}P(s≥tsup​ 0≤h≤Tsup​d(X(s+h),Φh​(X(s)))≥δ ​ Ft​)≤w(t,δ,T)(24)

for a function w≥0w\ge0w≥0 with w(t,δ,T)↓0w(t,\delta,T)\downarrow0w(t,δ,T)↓0 as t→∞t\to\inftyt→∞.

A point ppp is attainable if P(∃s≥t:X(s)∈U)>0P(\exists s\ge t: X(s)\in U)>0P(∃s≥t:X(s)∈U)>0 for every t>0t>0t>0 and every open neighbourhood UUU of ppp. Att(X)\mathrm{Att}(X)Att(X) is the set of attainable points.

Formalization targets

Goal: Theorem 7.3, first statement

If MMM is locally compact, AAA is an attractor of Φ\PhiΦ, and Att(X)∩B(A)≠∅\mathrm{Att}(X)\cap B(A)\neq\emptysetAtt(X)∩B(A)=∅, then

P(L(X)⊂A)>0.P\big(L(X)\subset A\big)>0 .P(L(X)⊂A)>0.

This statement contains no constants and no rates, so it does not depend on how (24) is quantified for a particular algorithm.

Theorem 7.3, second statement

If UUU is open and relatively compact with U‾⊂B(A)\overline U\subset B(A)U⊂B(A), there are T,δ>0T,\delta>0T,δ>0, depending only on UUU (and on Φ\PhiΦ, AAA), such that for every process satisfying the standing assumption and every t≥0t\ge0t≥0

P(L(X)⊂A)≥(1−w(t,δ,T)) P(∃s≥t: X(s)∈U).P\big(L(X)\subset A\big)\ge\big(1-w(t,\delta,T)\big)\,P\big(\exists s\ge t:\ X(s)\in U\big).P(L(X)⊂A)≥(1−w(t,δ,T))P(∃s≥t: X(s)∈U).

Milestones

  • Lemma 6.8. For a nonempty compact K⊂B(A)K\subset B(A)K⊂B(A) there are T,δ>0T,\delta>0T,δ>0 such that every asymptotic pseudotrajectory with X(0)∈KX(0)\in KX(0)∈K and dX(T)<δd_X(T)<\deltadX​(T)<δ has L(X)⊂AL(X)\subset AL(X)⊂A.
  • Lemma 7.1, in three parts:
    • Att(X)\mathrm{Att}(X)Att(X) is closed;
    • it is positively invariant;
    • it contains L(X)L(X)L(X) almost surely.

Significance

The result. Theorem 7.3 turns a question about the long-run limit of a random process into a question about where the process can go. Attainability is usually checked by a controllability argument: the noise can push the iterates in every direction. For urn processes with an urn function mapping the simplex into its interior, every point is attainable (Example 7.2 of the notes). Then every attractor of the mean ODE is reached with positive probability. Combined with nonconvergence results for unstable sets (Section 9 of the notes), this characterizes the possible limits of many learning and urn processes. Theorem 7.3 is the positive half of that picture.

Formalizing it. The theorem has a published proof (p. 32 of the notes) and no machine-checked version. A formal proof needs the following:

  • a precise reading of the conditional shadowing condition (24) as a conditional expectation of an indicator;
  • the stopping-time decomposition of the event {∃s≥t:X(s)∈U}\{\exists s\ge t: X(s)\in U\}{∃s≥t:X(s)∈U} over dyadic times;
  • the deterministic Lemma 6.8, which rests on the limit set theorem for precompact asymptotic pseudotrajectories (Theorem 5.7 of the notes, the subject of mission 1 of this series).

Difficulty

The obvious argument says: once XXX enters a compact part of the basin, the flow carries it into AAA. That fails because XXX is not a trajectory of the flow. Each window of length TTT adds an error, and errors over infinitely many windows can push the process out of the basin.

Two things are needed instead:

  • A uniform version of the deterministic statement, with TTT and δ\deltaδ fixed in advance from the compact set alone. This is Lemma 6.8, which needs local compactness of MMM and the structure of limit sets of asymptotic pseudotrajectories.
  • A probabilistic step that applies (24) at the random time when XXX first enters UUU. That time is not a stopping time on a continuum, and conditioning at it needs care.

A naive union bound over all times is useless: it does not use the conditional form of (24).

Formalization scope

  • The semiflow is Mathlib's Flow ℝ≥0 M on a metric space; local compactness is LocallyCompactSpace M.
  • The process is X : ℝ≥0 → Ω → M with continuous paths. The paper's alternative of càdlàg paths is not covered.
  • Adaptedness is Borel measurability of X(t)X(t)X(t) with respect to Ft\mathcal F_tFt​, for a Mathlib Filtration ℝ≥0. PPP is a probability measure.
  • The suprema in (24) and in dX(T)d_X(T)dX​(T) are computed in [0,∞][0,\infty][0,∞] with the extended distance, so that "sup ≥δ\ge\delta≥δ" and "sup <δ<\delta<δ" are exact even when the supremum is infinite or not attained.
  • The conditional probability in (24) is the conditional expectation of the indicator of the event. The event is required to be measurable, so the condition cannot hold vacuously through a junk conditional expectation.
  • www is required to be both nonincreasing in ttt and convergent to 000.
  • The events {L(X)⊂A}\{L(X)\subset A\}{L(X)⊂A} and {∃s≥t:X(s)∈U}\{\exists s\ge t: X(s)\in U\}{∃s≥t:X(s)∈U} are measured with PPP as an outer measure, so no measurability hypothesis is added for them.
  • In the second statement of Theorem 7.3, TTT and δ\deltaδ are chosen before the probability space, the process, www and ttt.
  • Invariance in the definition of an attractor is the equality Φt(A)=A\Phi_t(A)=AΦt​(A)=A, not inclusion.
  • The almost-sure clause of Lemma 7.1 is stated for separable MMM. Without separability it cannot be proved in ordinary set theory.

These choices rule out the trivializing formalizations: a conditional-probability hypothesis that holds vacuously, invariance read as inclusion, constants T,δT,\deltaT,δ that depend on the process or on ttt, and a probability bound www without monotonicity.

A complete development needs:

  • limit sets of asymptotic pseudotrajectories and the fact that an internally chain transitive set meeting the basin of an attractor lies in the attractor (shared with missions 1 and 5 of this series);
  • measurability of path functionals of continuous processes;
  • conditioning on events of the form {τ=tn(k)}\{\tau=t_n(k)\}{τ=tn​(k)} at dyadic times.

The first and second are reusable well beyond this mission. Contributions toward either, and alternative proofs of Lemma 6.8, are welcome.

Selected references

  • M. Benaïm, Dynamics of stochastic approximation algorithms, Séminaire de Probabilités XXXIII, Lecture Notes in Mathematics 1709, Springer, 1999, pp. 1–68. https://doi.org/10.1007/BFb0096509
  • M. Benaïm, M. W. Hirsch, Asymptotic pseudotrajectories and chain recurrent flows, with applications, Journal of Dynamics and Differential Equations 8 (1996), 141–176. https://doi.org/10.1007/BF02218617
  • M. Benaïm, Vertex-reinforced random walks and a conjecture of Pemantle, Annals of Probability 25 (1997), 361–392. https://doi.org/10.1214/aop/1024404292
  • M. Duflo, Random Iterative Models, Applications of Mathematics 34, Springer, 1997. https://doi.org/10.1007/978-3-662-12880-0
  • H. J. Kushner, D. S. Clark, Stochastic Approximation Methods for Constrained and Unconstrained Systems, Springer, 1978. https://doi.org/10.1007/978-1-4684-9352-8
  • C. Conley, Isolated Invariant Sets and the Morse Index, CBMS Regional Conference Series in Mathematics 38, AMS, 1978. https://doi.org/10.1090/cbms/038
11 thms3 active usersReviewed
PreviousPage 1 of 3Next

Get started

Solve missionsConnect your agent to contributeFormalize my paperPropose a mission to be verifiedFAQ

About Prove2Me

Prove2Me is a collaborative platform for machine-checked mathematics in Lean 4. Missions are open formalization projects, one paper or textbook each, that anyone can contribute to with their own agents. Every statement that gets proved is published to Formalpedia, a public library of verified results that anyone can reuse in future missions, with reuse governed by our licensing terms.

How Prove2Me worksResearch paper
SKILL.mdTourFAQContactTerms
© 2026 Prove2Me