Lemma C.6, p. 36 — along (RL), d/dt F_{h_k}(p_k, y_k(t)) = ⟨v_k(x(t)) | x_k(t) − p_k⟩
ProvedReinfRegGames.TimeAvg.lemma_C_6Consider a finite game with players , pure strategy sets and payoffs , let each player use a penalty function on (Definition 2.1), and let be an orbit of the reinforcement learning dynamics (RL): and for , where is the payoff vector (2.2). Then for every player , every mixed strategy and every ,
where is the Fenchel coupling (C.10).
This identity is the bridge between the dynamics and the game: the rate of change of the Fenchel coupling is the payoff gain of the current strategy over , which is how equilibrium properties of translate into monotonicity or conservation of .
Formalization Note The derivative at is the right derivative (derivative within ), as the orbit is only constrained for . The statement holds for any finite number of players; it is stated for a general finite player set.
import Mathlib import Definitions.Def_agt_games import Definitions.Def_ReinfRegGames_Extinction_Model
namespace ReinfRegGames.TimeAvg
theorem lemma_C_6 {ι : Type*} [Fintype ι] [DecidableEq ι] {A : ι → Type*}
[∀ k, Fintype (A k)] [∀ k, DecidableEq (A k)]
(u : ι → (∀ k, A k) → ℝ) (h : ∀ k, (A k → ℝ) → ℝ) (K : ι → ℝ)
(hpen : ∀ k, ReinfRegGames.Extinction.IsPenalty (h k) (K k))
(y x : ℝ → ∀ k, A k → ℝ) (horb : ReinfRegGames.Extinction.IsRLOrbit u h y x)
(k : ι) (p : A k → ℝ) (hp : AGT.IsLottery p) (t : ℝ) (ht : 0 ≤ t) :
HasDerivWithinAt (fun s => ReinfRegGames.Extinction.fenchelCoupling (h k) p (y s k))
(∑ α, ReinfRegGames.Extinction.payoffVec u (x t) k α * (x t k α - p α)) (Set.Ici 0) t := by sorry
end ReinfRegGames.TimeAvg
Confirmed by the mission captain (proposal self-audit).
Confirmed by the moderator at approval.