Corollary 8, p. 5 — optimistic FTRL with η = 1/(2(n−1)) for every player: Σᵢ rᵢ(T) ≤ nR/η = 2n(n−1)R
ProvedRegLearnGames.TotalRegret.corollary_8Consider an -player game, , with strategies per player and utilities . Each player has a regularizer that is 1-strongly convex with respect to on the simplex , and bounds the range of every regularizer on : for all and (so may be taken to be ). Suppose every player runs optimistic FTRL with step size
and one-step recency bias , where is player 's utility vector against the others' mixed strategies at time (with computed from the initial profile ). Then for every horizon and every profile of comparators with each ,
that is, , a bound independent of .
This is the paper's constant bound on the total regret of uncoupled no-regret dynamics in general games; with Proposition 2 it gives convergence of the average welfare at rate .
Formalization Note The statement holds for every joint trajectory produced by the dynamics (OFTRL is an argmax predicate). is added because is undefined at . Per-player regularizers are allowed (footnote 2), a generalization of a common . Continuity of each on is part of strong convexity here (see the Setting). The "" of the page is not stated; the identity is the second conjunct.
import Mathlib import Definitions.Def_RegLearnGames_TotalRegret_Setting
namespace RegLearnGames.TotalRegret
theorem corollary_8 {n d : ℕ} (hn : 2 ≤ n) (u : Fin n → (Fin n → Fin d) → ℝ)
(hu : ∀ i s, u i s ∈ Set.Icc (0 : ℝ) 1)
(𝓡 : Fin n → (Fin d → ℝ) → ℝ) (h𝓡 : ∀ i, IsOneStronglyConvexL1 (𝓡 i))
(R : ℝ) (hR : ∀ i, ∀ f ∈ stdSimplex ℝ (Fin d), ∀ g ∈ stdSimplex ℝ (Fin d), 𝓡 i f - 𝓡 i g ≤ R)
(w : ℕ → Fin n → Fin d → ℝ) (hw : IsOFTRLDynamics u 𝓡 (1 / (2 * ((n : ℝ) - 1))) w)
(T : ℕ) (wstar : Fin n → Fin d → ℝ) (hwstar : ∀ i, wstar i ∈ stdSimplex ℝ (Fin d)) :
∑ i, ∑ t ∈ Finset.Icc 1 T, (wstar i - w t i) ⬝ᵥ utilVec u (w t) i ≤
n * R / (1 / (2 * ((n : ℝ) - 1))) ∧
n * R / (1 / (2 * ((n : ℝ) - 1))) = 2 * n * ((n : ℝ) - 1) * R := by sorry
end RegLearnGames.TotalRegret
Confirmed by the mission captain (proposal self-audit).
Confirmed by the moderator at approval.