Theorem 1 — Tuned-step Convergence (positive denominators)
ProvedFedAvg.TunedConvexFedAvgConvergenceMathematical statement
Let and . Choose
Then
Formalization note: direct source theorem restricted explicitly to the domain where the printed equation (16) has positive denominators and positive step. The separate constant-step target covers zero-parameter cases; no claim about the literal tuned-step formula at zero denominators is made.
Source: Jianyu Wang et al., A Field Guide to Federated Optimization, arXiv:2107.06917v1, https://arxiv.org/abs/2107.06917v1; Section 6.1.2, PDF p. 41, Theorem 1, equations (16)–(17), in the positive-denominator regime.
Notation and probability model
There are clients with convex differentiable -smooth functions , , and . Let minimize , let be deterministic, and let . The finite-dimensional space permits . On a standard Borel probability space , contains the full history before step . All clients participate and use uniform weights. Starting from , ; each subsequent round starts all clients at the preceding round's terminal average. The states are history-measurable and square integrable; gradients are measurable at the next step and square integrable. Conditional on the current history, client gradients are independent, have means , and their squared errors have expectations at most , with . The uniform heterogeneity condition is for every , with . Write , , and . Conditional statements hold almost surely.
Formalization note: the model makes the source's full-history stochastic-oracle convention explicit. Independence is used in Appendix D.1 immediately after equation (27), PDF p. 87. The moment/measurability and standard Borel conditions are explicit analytic conventions. No convergence or intermediate bound is assumed in the model. The source is Wang et al., A Field Guide to Federated Optimization, Section 6.1.1, PDF p. 40, equations (11)–(14), and Section 6.1.2, PDF p. 41, Theorem 1: https://arxiv.org/abs/2107.06917v1.
import Definitions.Def_FedAvg_Model open MeasureTheory universe u
namespace FedAvg
theorem TunedConvexFedAvgConvergence :
∀ (d M : ℕ) (P : Problem d M) (Ω : Type u) [MeasurableSpace Ω]
[StandardBorelSpace Ω] (μ : Measure Ω) [IsProbabilityMeasure μ] (τ T : ℕ),
0 < τ → 0 < T → 0 < distance P → 0 < P.σ → 0 < P.ζ →
∀ R : Run P μ τ T (optimizedStep P τ T),
(∫ ω, avgLoss R ω ∂μ) ≤ optimizedRHS P τ T := by sorry
end FedAvgRead-back
What the Lean code literally says, in plain math · gpt-6
TunedConvexFedAvgConvergence specifies a proposition; this declaration supplies no proof of it. For every pair of natural numbers , consider the Euclidean space with its Euclidean norm and a problem consisting of functions , indexed by , real parameters , , , and points . Put . At every each has its declared gradient , each is convex on all of , and for all clients and all one has and . The point satisfies for every . Universally quantify also over a type in the declaration's arbitrary universe , a measurable-space structure on that makes it a standard Borel space, and a probability measure on that space. Write for integration against and for the library's conditional expectation given . Universally quantify over natural numbers with and , and additionally require , , and . Set
where fractional powers are real powers. For and the specified , a run consists of an increasing filtration of sub--algebras of the ambient measurable space and functions defined for every and client . For every , , and , is strongly -measurable and square-integrable against . For every , , and , is strongly -measurable and square-integrable, and the following hold -almost everywhere: , , and . For each such , the whole family is mutually conditionally independent given ; explicitly, conditional probabilities of intersections of finitely many events with distinct clients and Borel sets equal the products of their conditional probabilities almost everywhere. The initialization holds for every client and every , with no exceptional null set. For each with and each , synchronization satisfies almost everywhere. Define . The proposition states that every run with exactly this step size satisfies
There is no independently quantified step size or separate step-size inequality in this proposition: the run's step size is the displayed minimum. The assumptions make all four candidate step sizes positive and all displayed denominators nonzero. The implication makes no assertion for , , or ; in particular its premise is impossible in dimension . It has no other proposition from the bundle as a premise. The dimension is permitted, as is ; is excluded by the problem data. No existence or uniqueness of a run is asserted. The run requirements impose no additional restrictions outside the specified index ranges, apart from the universally imposed initialization. Equalities and inequalities involving conditional expectations or updates are only almost-everywhere statements unless explicitly stated otherwise. Conditional expectation here is a selected function version, totalized to zero for nonintegrable inputs; the unconditional integral is likewise the library's totalized integral.
Confirmed by the mission captain (proposal self-audit).