Theorem 13, p. 23 — the projected gradient step contracts dist(·, X*) if and only if f has quadratic functional growth (22)
ProvedNecoaraNG.GMIff.theorem_13Let problem (P), , satisfy the standing assumptions: is closed and convex, is convex and differentiable on with gradient Lipschitz on with constant , and the optimal set is nonempty. For let be one step of the gradient method (GM) with constant step size , and let , . Then:
- (Growth ⇒ linear convergence.) If satisfies the quadratic functional growth (22) with some constant , then, with ,
- (Linear convergence ⇒ growth.) If for some with
then satisfies (22) with constant , i.e. belongs to .
In particular, the gradient method with constant step size contracts the distance to the optimal set by a uniform factor at every feasible point if and only if has quadratic functional growth: quadratic functional growth is exactly the condition, among those studied in the paper, under which (GM) converges linearly.
Formalization Note The page does not define "converging linearly"; its proof identifies it with the one-step contraction of Theorem 5 (part 2, obtained from Theorem 5) and obtains that contraction from (52) at (part 1, in the squared form of (52)). Part 1 gives the contraction of part 2 with , so the two parts together give the equivalence of Theorem 13. A multi-step notion (" along every run") is not used: the converse for it is not what the page proves. Nearest points are predicates; they exist and are unique on the nonempty closed convex sets and .
import Mathlib import Definitions.Def_NecoaraNG_GMIff_Setting
namespace NecoaraNG.GMIff
theorem theorem_13 (n : ℕ) (X : Set (NecoaraNG.Chain.E n)) (hXc : IsClosed X) (hXconv : Convex ℝ X)
(f : NecoaraNG.Chain.E n → ℝ) (hf : ConvexOn ℝ X f)
(hdiff : ∀ x ∈ X, DifferentiableAt ℝ f x)
(Lf : ℝ) (hLf : 0 < Lf)
(hL : ∀ x ∈ X, ∀ y ∈ X, ‖gradient f x - gradient f y‖ ≤ Lf * ‖x - y‖)
(xstar : NecoaraNG.Chain.E n) (hxstar : xstar ∈ NecoaraNG.Chain.optSet X f) :
(∀ κ : ℝ, 0 < κ → NecoaraNG.Chain.QuadFunGrowth X f κ →
∀ x ∈ X, ∀ xbar, NecoaraNG.Chain.IsNearest (NecoaraNG.Chain.optSet X f) x xbar → ∀ xp, NecoaraNG.ErrBound.IsPGStep X f Lf x xp →
∀ xpbar, NecoaraNG.Chain.IsNearest (NecoaraNG.Chain.optSet X f) xp xpbar →
‖xp - xpbar‖ ^ 2 ≤ 1 / (1 + κ / Lf) * ‖x - xbar‖ ^ 2)
∧
(∀ β : ℝ, 0 < β → β < 1 →
(∀ x ∈ X, ∀ xbar, NecoaraNG.Chain.IsNearest (NecoaraNG.Chain.optSet X f) x xbar → ∀ xp, NecoaraNG.ErrBound.IsPGStep X f Lf x xp →
∀ xpbar, NecoaraNG.Chain.IsNearest (NecoaraNG.Chain.optSet X f) xp xpbar → ‖xp - xpbar‖ ≤ β * ‖x - xbar‖) →
NecoaraNG.Chain.QuadFunGrowth X f (Lf * (1 - β) ^ 2)) := by sorry
end NecoaraNG.GMIff
Confirmed by the mission captain (proposal self-audit).
Confirmed by the moderator at approval.