Proposition C.1, p. 33 — the choice map Q is Lipschitz and equals the derivative dh* of the convex conjugate
ProvedReinfRegGames.TimeAvg.proposition_C_1Let be a nonempty finite set, the simplex of probability vectors on , and a penalty function on with strong convexity constant (Definition 2.1). For a score vector , the choice map is the maximizer of over , and is the convex conjugate (C.1). Then:
- for every a maximizer exists;
- is Lipschitz: there is a constant such that for all ;
- is (Fréchet) differentiable at every , with
The identity is what turns the Fenchel coupling into a Lyapunov-type function along the reinforcement learning dynamics (Lemma C.6); the Lipschitz property makes the trajectory continuous.
Formalization Note The Lipschitz constant is existential, as on the page (which gives none). Distances are those of Lean's sup metric on ; Lipschitz continuity does not depend on the choice of norm. Item 2 implies that the maximizer is unique. The derivative is stated as HasFDerivAt with the linear functional . The hypothesis that is nonempty is the page's standing assumption that is the simplex of , (otherwise no maximizer exists).
import Mathlib import Definitions.Def_agt_games import Definitions.Def_ReinfRegGames_Extinction_Model
namespace ReinfRegGames.TimeAvg
theorem proposition_C_1 {B : Type*} [Fintype B] [DecidableEq B] [Nonempty B]
(h : (B → ℝ) → ℝ) (K : ℝ) (hpen : ReinfRegGames.Extinction.IsPenalty h K) :
(∀ y : B → ℝ, ∃ x, ReinfRegGames.Extinction.IsChoice h y x) ∧
(∃ L : ℝ, ∀ y y' x x' : B → ℝ, ReinfRegGames.Extinction.IsChoice h y x → ReinfRegGames.Extinction.IsChoice h y' x' →
dist x x' ≤ L * dist y y') ∧
(∀ y x : B → ℝ, ReinfRegGames.Extinction.IsChoice h y x →
HasFDerivAt (ReinfRegGames.Extinction.conj h) (∑ β, x β • ContinuousLinearMap.proj (R := ℝ) (φ := fun _ : B => ℝ) β) y) := by sorry
end ReinfRegGames.TimeAvg
Confirmed by the mission captain (proposal self-audit).
Confirmed by the moderator at approval.