Motivation
Mean field games describe the limit of stochastic differential games with a large number of statistically identical players, each interacting with the others only through the empirical distribution of their states (Lasry and Lions, Mean field games, 2007; Huang, Malhamé and Caines, 2006). Models whose players move among finitely many states (an infection status, a cyber-security level, a choice among a few technologies) are used in epidemiology, botnet defense (Kolokoltsov and Bensoussan, 2016; reference [19] of the paper) and economics. For these finite state games the analytic approach of Gomes, Mohr and Souza (Continuous time finite state mean field games, 2013) characterizes equilibria by a coupled system of ordinary differential equations, and so restricts attention to Markovian controls and to interactions through the distribution of states only.
Carmona and Wang (arXiv:1808.07635, 2018; Mathematics of Operations Research, 2021) treat the finite state game in the weak formulation, by a change of measure on the canonical path space, following the diffusion case of Carmona and Lacker (A probabilistic weak formulation of mean field games and applications, 2015). This handles two extensions at once: strategies may be arbitrary closed-loop, path-dependent controls, and the rates and costs may depend on the distribution of the controls themselves (an extended mean field game).
Setting
There are m states, identified with the basis vectors e1,…,em of Rm. The canonical space Ω consists of the càdlàg paths [0,T]→{e1,…,em} that are continuous at T; Xt(ω)=ωt is the canonical process and F=(Ft) its natural filtration. The reference measure P makes X a continuous-time Markov chain with a given initial law p∘ and rate matrix Q0 (diagonal entries −(m−1), off-diagonal entries 1).
Controls take values in a compact set A⊆Rl. Write S for the probability simplex of Rm, P(A) for the probability measures on A and W1 for the 1-Wasserstein distance on P(A). A strategy α∈A is an F-predictable process with values in A. A mean field is a pair of measurable flows p:[0,T]→S (states) and ν:[0,T]→P(A) (controls). Transition rates q(t,i,j,α,p,ν), a running cost f(t,ei,α,p,ν) and a terminal cost g(ei,p) are given.
A strategy and a mean field change P into Q(α,p,ν), whose density is the Doléans-Dade exponential of (5)–(7); under it X jumps from ei to ej at rate q(t,i,j,αt,pt,νt). The cost is
J(α,p,ν)=EQ(α,p,ν)[∫0Tf(t,Xt,αt,pt,νt)dt+g(XT,pT)].
A Nash equilibrium (Definition 2.8) is a triple (α∗,p∗,ν∗) such that α∗ minimizes J(⋅,p∗,ν∗) over A, and for every t, pt∗ is the law of Xt and νt∗ the law of αt∗ under Q(α∗,p∗,ν∗).
The reduced Hamiltonian is Hi(t,z,α,p,ν)=f(t,ei,α,p,ν)+∑j=i(zj−zi)(q(t,i,j,α,p,ν)−1), and H^i=infα∈AHi.
Formalization targets
Goal: Theorem 4.1
Under Assumption 2.2 (each Q(t,α,p,ν) is a Q-matrix, off-diagonal rates lie in (C1,C2) with C1>0, rates are Lipschitz in (α,p,ν)), Assumption 2.7 (f and g Lipschitz) and Assumption 3.4 (Hi(t,z,⋅,p,ν) has a unique minimizer a^i(t,z,p), independent of ν, measurable and Lipschitz in the sense of (21)),
∃(α∗,p∗,ν∗) a Nash equilibrium in the sense of Definition 2.8.
Milestones
- Lemma 3.7: H^ and a^ are Lipschitz in z for the stochastic seminorm and in (p,ν) with factor 1+∥z∥.
- Lemma 3.8: the cost of any strategy is EP[Y0] for the BSDE (28).
- Proposition 3.9: the BSDE (29) gives the value, and α^t=a^(t,Xt−,Zt,pt) is the dt⊗dP-a.e. unique optimal control.
- Proposition 4.2: the set P0 of measures with L2-bounded densities is convex and relatively compact for the J1-weak topology.
- Lemma 4.7: R0 is convex and compact for the stable topology.
- Lemma 4.8: randomized rates and Hamiltonian are Lipschitz in Wˉ1.
- Proposition 4.10: for some C0>0, Pˉ0×R0 is stable for the fixed-point map Φ.
- Lemma 4.15: ∫0TW1(νt(n),νt(0))dt→0 implies stable convergence of L(dt)×δνt(n).
Significance
The theorem gives equilibria for finite state mean field games without the Markovian restriction and with interaction through the distribution of controls, in a setting where the analytic (ODE) approach does not apply. The same weak formulation supports the paper's uniqueness result under a Lasry–Lions monotonicity condition and the construction of approximate Nash equilibria for the N-player game (Theorems 5.4 and 6.5), which are not part of this mission.
The result is proved in the paper; nothing here is open mathematics. No part of it is machine-checked. A formalization would add, beyond the theorem itself: the reference Markov chain on the canonical càdlàg space, Girsanov changes of measure for chains with bounded rates, backward SDEs driven by a Markov chain (Cohen and Elliott's theory), the J1 topology on finite-valued paths, and the stable topology of Jacod and Mémin. While drafting, Proposition 4.5 of the paper was found to be false as printed (see Formalization scope); the formalization records the corrected argument structure.
Difficulty
The obvious approach, a fixed point of (p,ν)↦ the laws under the best response, fails for lack of compactness: sets of measurable flows [0,T]→S or [0,T]→P(A) have no useful compact subsets. The paper replaces the state flow by a law on path space, where compactness comes from tightness in the Skorokhod topology, and the control flow by a measure on [0,T]×P(A) with the stable topology, where compactness requires fixing the time marginal. Continuity of the fixed-point map then needs stability of Markov-chain BSDEs and L2 control of Girsanov densities, both uniform in the mean field. In Lean, Schauder's fixed point theorem, Prokhorov's theorem on Skorokhod space, Girsanov's theorem for jump processes and BSDE theory are all absent from Mathlib.
Formalization scope
All declarations are in the namespace ExtFSMFG.Existence. Time is indexed by R≥0, and paths are frozen after T (in bijection with the paper's paths on [0,T]); states are Fin m. The reference measure is a hypothesis, IsReference, fixing the finite-dimensional distributions through etQ0. Q(α,p,ν) is defined by the closed form of E(L(α,p,ν))T (an exponential of the compensator times the product of the rates at the finitely many jumps), and stochastic integrals against the chain's martingale are pathwise. ∥p−p′∥ is Euclidean. P(A) carries the weak topology and its Borel σ-field; W1 and Wˉ1 are infima over couplings. a^ appears as an argument of Assumption 3.4, which pins it down.
One hypothesis is added and disclosed: q and f are jointly measurable and f is bounded (RegularData). The paper uses both (for (5), (12) and the BSDE drivers) without stating them. BSDE solutions have a predictable Z (the paper writes "left-continuous"); Wˉ1 has the marginal constraint that (40) omits; (42) uses the rates of (5).
The equilibrium is stated through Definition 2.8 itself, so a formalization in which the equilibrium is a fixed point of Φ, the strategy set is shrunk to Markov feedback controls, the density is a junk value, or optimality is phrased through an infimum that may default to 0, is ruled out: optimality is "J(α∗)≤J(α) for every predictable A-valued α".
Proposition 4.5 ("R is compact, metrizable, and Polish") is not a milestone: as printed it is false, since δ(T/(n+1),m0) has no cluster point in R for the stable topology (test g(t,m)=1Sk(t) with Sk={T/(n+1):n≥k}). Only the compactness of R0 (Lemma 4.7) is used. Lemmas 3.1, 3.3, 4.3, Propositions 4.12, 4.14, 4.16 are further steps of the proof that may be added as milestones later. Contributions welcome: a proof of the sanity-level facts (the density integrates to 1; W1 is a metric on P(A)), Lemma 3.7 and Lemma 4.8, which are elementary, and Markov-chain BSDE infrastructure.
Selected references
- R. Carmona, P. Wang, A Probabilistic Approach to Extended Finite State Mean Field Games, Mathematics of Operations Research, 2021. https://arxiv.org/abs/1808.07635 (v1 is the version formalized)
- R. Carmona, D. Lacker, A probabilistic weak formulation of mean field games and applications, Annals of Applied Probability 25, 2015. https://doi.org/10.1214/14-AAP1020
- S. N. Cohen, R. J. Elliott, Comparisons for backward stochastic differential equations on Markov chains and related no-arbitrage conditions, Annals of Applied Probability 20, 2010. https://doi.org/10.1214/09-AAP619
- D. A. Gomes, J. Mohr, R. R. Souza, Continuous time finite state mean field games, Applied Mathematics & Optimization 68, 2013. https://doi.org/10.1007/s00245-013-9202-8
- J. Jacod, J. Mémin, Sur un type de convergence intermédiaire entre la convergence en loi et la convergence en probabilité, Séminaire de Probabilités XV, Lecture Notes in Mathematics 850, 1981 (reference [17] of Carmona–Wang).
- J.-M. Lasry, P.-L. Lions, Mean field games, Japanese Journal of Mathematics 2, 2007. https://doi.org/10.1007/s11537-007-0657-8