Prove2Me
Navigate
MissionsFormalpediaBlogsUsersMy Missions+
Prove2Me
⌕
Log in
← Formalpedia

Entanglement is bounded by the configuration's deviation (Lem. 2 / 8)

Proved
MarkovEntanglement.rmab_entanglement_le_configuration_deviation

by tianyipeng · Aug 28, 2026 · Mathlib c5ea003 (Lean v4.30.0)

Consider an NNN-agent restless multi-armed bandit under an index policy π\piπ with injective priority index ν\nuν and budget ⌊αN⌋\lfloor \alpha N \rfloor⌊αN⌋, and let μ1:Nπ\mu^\pi_{1:N}μ1:Nπ​ be a stationary occupancy distribution of the induced chain. Let m∗m^\astm∗ be a configuration (a point of the simplex) — in the application, the mean-field fixed point.

Then for every agent iii the measure of Markov entanglement of the joint chain, with respect to the occupancy-weighted agent-wise total variation distance, is bounded by the expected deviation of the system configuration from m∗m^\astm∗:

Ei(P1:Nπ)  ≤  ∣S∣2⋅E[∥m−m∗∥∞],\mathcal{E}_i(P^\pi_{1:N}) \;\le\; |S|^2 \cdot \mathbb{E}\big[\|m - m^\ast\|_\infty\big],Ei​(P1:Nπ​)≤∣S∣2⋅E[∥m−m∗∥∞​],

the expectation taken over the stationary occupancy measure.

This is where Proposition 1 is cashed in. Proposition 1 bounds the entanglement of a weakly-coupled system by the occupancy-weighted distance between the realised policy's per-agent marginal and any local policy; here the local policy chosen as witness is the mean-field limiting policy at m∗m^\astm∗, which activates an agent in state xxx with the probability the index policy would allot at the configuration m∗m^\astm∗. Homogeneity of the agents makes the mismatch the same for every agent, so the supremum over iii may be replaced by an average, and the average collapses into a sum over local states weighted by the configuration. The remaining per-state estimate compares the fraction the index policy actually activates in state xxx with the fraction it would activate at m∗m^\astm∗; both are ratios of a budget residual to a state occupancy, and both numerator and denominator move by at most ∣S∣⋅∥m−m∗∥∞|S| \cdot \|m - m^\ast\|_\infty∣S∣⋅∥m−m∗∥∞​, which gives the stated bound.

With this lemma the asymptotic analysis reduces entirely to a question about the configuration process: how far does mmm stray from the mean-field fixed point under the stationary distribution?

The stationary distribution is assumed exchangeable — invariant under permuting the agents. The source's proof exchanges agent indices and asserts the stationary distribution is unchanged, which is exactly exchangeability; it holds automatically for the unique stationary distribution of an ergodic symmetric chain, but a reducible chain also has non-exchangeable stationary distributions concentrated on asymmetric closed classes, for which the agent-averaging step (and with it the stated bound) is not available. The hypothesis records precisely the consequence of ergodicity the argument uses.

Preamble
import Mathlib
import Definitions.Def_markov_entanglement_meanfield

open scoped BigOperators
open MarkovEntanglement
Formal statement
namespace MarkovEntanglement

variable {S : Type*} [Fintype S] [DecidableEq S]

/-- Lemma 2 / Lemma 8.  For an index policy, the measure of Markov entanglement of any agent,
with respect to the occupancy-weighted agent-wise total variation distance, is bounded by the
expected deviation of the system configuration from the mean-field fixed point:

`Eᵢ(P^π_{1:N}) ≤ |S|² · E[‖m − m✦‖_∞]`,

the expectation taken over the stationary occupancy measure.  This is where Proposition 1 is
cashed in: the local policy witnessing the bound is the mean-field limiting policy at `m✦`. -/
theorem rmab_entanglement_le_configuration_deviation
    (P0 P1 : Matrix S S ℝ) (hP0 : IsTransitionMatrix P0) (hP1 : IsTransitionMatrix P1)
    (ν : S → ℝ) (hν : Function.Injective ν) (α : ℝ) (hα : 0 < α) (hα1 : α < 1)
    (mstar : S → ℝ) (hmstar : IsConfiguration mstar)
    (N : ℕ) (hN : 0 < N)
    (π : (Fin N → S) → (Fin N → Bool) → ℝ)
    (hπ : IsIndexPolicy ν ⌊α * (N : ℝ)⌋₊ π)
    (μ : Joint (StateAction (fun _ : Fin N => S) (fun _ : Fin N => Bool)) → ℝ)
    (hμ : IsDist μ) (hexch : IsExchangeableDist μ)
    (hstat : IsStationary (inducedTransition
      (fun s a s' => ∏ j, rmabKernel P0 P1 (s j) (a j) (s' j)) π) μ)
    (i : Fin N) :
    entanglementN i μ (inducedTransition
        (fun s a s' => ∏ j, rmabKernel P0 P1 (s j) (a j) (s' j)) π)
      ≤ (Fintype.card S : ℝ) ^ 2 *
          ∑ p : Joint (StateAction (fun _ : Fin N => S) (fun _ : Fin N => Bool)),
            μ p * supNorm (fun x => configuration (fun j => (p j).1) x - mstar x) := by
  sorry

/-! ### M4 — Lemma 9, one-step concentration (Gast, Gaujal and Yan) -/

end MarkovEntanglement
Source
Shuze Chen and Tianyi Peng, 'Multi-agent Markov Entanglement', arXiv:2506.02385v3, Section 7.1.2 p. 25 (Lemma 2) and Appendix I p. 42 (Lemma 8)
Read-back

What the Lean code literally says, in plain math · claude-opus-5

Fix a finite type SSS with decidable equality (its cardinality is ∣S∣=card S|S| = \text{card}\,S∣S∣=cardS), and write [N][N][N] for the NNN-element index type Fin N\mathrm{Fin}\,NFinN. The statement asserts: for every choice of the following data, satisfying all of the listed hypotheses, the displayed inequality holds.

Data and hypotheses.

  • Two real matrices P0,P1P^0, P^1P0,P1 indexed by S×SS \times SS×S, each a transition matrix: Pxy0≥0P^0_{x y} \ge 0Pxy0​≥0 and ∑y∈SPxy0=1\sum_{y \in S} P^0_{x y} = 1∑y∈S​Pxy0​=1 for every xxx, and likewise for P1P^1P1.
  • A function ν:S→R\nu : S \to \mathbb{R}ν:S→R that is injective.
  • A real number α\alphaα with 0<α<10 < \alpha < 10<α<1.
  • A function m⋆:S→Rm^\star : S \to \mathbb{R}m⋆:S→R that is a configuration: m⋆(x)≥0m^\star(x) \ge 0m⋆(x)≥0 for all xxx and ∑x∈Sm⋆(x)=1\sum_{x \in S} m^\star(x) = 1∑x∈S​m⋆(x)=1. (No relation whatsoever is assumed between m⋆m^\starm⋆ and P0,P1,ν,αP^0, P^1, \nu, \alphaP0,P1,ν,α, or the dynamics below; it is an arbitrary probability vector on SSS.)
  • A natural number NNN with N>0N > 0N>0. Set M:=⌊αN⌋M := \lfloor \alpha N \rfloorM:=⌊αN⌋, the natural-number floor (so M=0M = 0M=0 whenever αN<1\alpha N < 1αN<1, e.g. when N=1N = 1N=1).
  • A function π:(S[N])→({false,true}[N])→R\pi : (S^{[N]}) \to (\{\mathrm{false},\mathrm{true}\}^{[N]}) \to \mathbb{R}π:(S[N])→({false,true}[N])→R assigning to each joint state s=(sj)j∈[N]s = (s_j)_{j \in [N]}s=(sj​)j∈[N]​ and each joint action a=(aj)j∈[N]a = (a_j)_{j \in [N]}a=(aj​)j∈[N]​ (each aja_jaj​ a Boolean) a real number, subject to the three conditions of being an index policy for ν\nuν with budget MMM:
    1. π(s,a)≥0\pi(s,a) \ge 0π(s,a)≥0 for all s,as, as,a, and ∑a∈{false,true}[N]π(s,a)=1\sum_{a \in \{\mathrm{false},\mathrm{true\}}^{[N]}} \pi(s,a) = 1∑a∈{false,true}[N]​π(s,a)=1 for every sss;
    2. for all s,as,as,a: if π(s,a)≠0\pi(s,a) \neq 0π(s,a)=0 then aaa is exactly MMM-budgeted, i.e. #{j∈[N]:aj=true}=M\#\{j \in [N] : a_j = \mathrm{true}\} = M#{j∈[N]:aj​=true}=M (equality, not ≤\le≤);
    3. for every joint state sss and every agent j∈[N]j \in [N]j∈[N], the marginal probability that agent jjj is activated,
∑a : aj=trueπ(s,a),\sum_{a\,:\,a_j = \mathrm{true}} \pi(s,a),a:aj​=true∑​π(s,a),
 equals $\mathrm{iap}_\nu^M(s, s_j)$, where for $x \in S$ one writes $c_s(x) := \#\{k \in [N] : s_k = x\}$ (the count of agents in state $x$), $h_s(x) := \sum_{y : \nu(x) < \nu(y)} c_s(y)$ (the number of agents in strictly higher-$\nu$ states), and
 
iapνM(s,x):={0,cs(x)=0,min⁡(cs(x),  M−˙hs(x))cs(x),otherwise,\mathrm{iap}_\nu^M(s,x) := \begin{cases} 0, & c_s(x) = 0,\\[2pt] \dfrac{\min\bigl(c_s(x),\; M \dot- h_s(x)\bigr)}{c_s(x)}, & \text{otherwise,}\end{cases}iapνM​(s,x):=⎩⎨⎧​0,cs​(x)min(cs​(x),M−˙​hs​(x))​,​cs​(x)=0,otherwise,​
 with $\dot-$ denoting **truncated natural-number subtraction** (so $M \dot- h_s(x) = 0$ when $h_s(x) \ge M$).
  • A function μ\muμ on the joint state–action space Ω:=∏j∈[N](S×{false,true})\Omega := \prod_{j \in [N]} (S \times \{\mathrm{false},\mathrm{true}\})Ω:=∏j∈[N]​(S×{false,true}) — i.e. μ\muμ assigns a real number to each tuple p=(pj)j∈[N]p = (p_j)_{j\in[N]}p=(pj​)j∈[N]​ where pj=(statej,actionj)p_j = (\text{state}_j, \text{action}_j)pj​=(statej​,actionj​) — such that:
    • μ(p)≥0\mu(p) \ge 0μ(p)≥0 for all ppp and ∑p∈Ωμ(p)=1\sum_{p \in \Omega} \mu(p) = 1∑p∈Ω​μ(p)=1;
    • μ\muμ is exchangeable: μ(p∘σ)=μ(p)\mu(p \circ \sigma) = \mu(p)μ(p∘σ)=μ(p) for every permutation σ\sigmaσ of [N][N][N] and every ppp;
    • μ\muμ is stationary for the matrix TTT on Ω×Ω\Omega \times \OmegaΩ×Ω defined by
T(p,q)  =  (∏j∈[N]K(statej(p), actionj(p), statej(q)))⋅π(states(q), actions(q)),T(p, q) \;=\; \Bigl(\prod_{j \in [N]} K\bigl(\text{state}_j(p),\, \text{action}_j(p),\, \text{state}_j(q)\bigr)\Bigr)\cdot \pi\bigl(\text{states}(q),\, \text{actions}(q)\bigr),T(p,q)=(j∈[N]∏​K(statej​(p),actionj​(p),statej​(q)))⋅π(states(q),actions(q)),
where $K(x,a,y) = P^1_{x y}$ if $a = \mathrm{true}$ and $K(x,a,y) = P^0_{x y}$ if $a = \mathrm{false}$; stationarity meaning $\sum_{p \in \Omega} \mu(p)\, T(p,q) = \mu(q)$ for every $q \in \Omega$. (Note the next action in $q$ is drawn by $\pi$ from the *next* state in $q$, and $T$ is not itself asserted to be a transition matrix.)
  • A distinguished agent index i∈[N]i \in [N]i∈[N].

Conclusion. With TTT as above,

Ei(μ,T)  ≤  ∣S∣2⋅∑p∈Ωμ(p) (sup⁡x∈S∣cst(p)(x)/N  −  m⋆(x)∣),\mathcal{E}_i(\mu, T) \;\le\; |S|^2 \cdot \sum_{p \in \Omega} \mu(p)\, \Bigl(\sup_{x \in S} \bigl| c_{\mathrm{st}(p)}(x)/N \;-\; m^\star(x) \bigr|\Bigr),Ei​(μ,T)≤∣S∣2⋅p∈Ω∑​μ(p)(x∈Ssup​​cst(p)​(x)/N−m⋆(x)​),

where st(p)=(statej(p))j∈[N]\mathrm{st}(p) = (\text{state}_j(p))_{j \in [N]}st(p)=(statej​(p))j∈[N]​ is the state part of ppp, the inner quantity cst(p)(x)/Nc_{\mathrm{st}(p)}(x)/Ncst(p)​(x)/N is the empirical configuration (fraction of the NNN agents whose state is xxx, a real division by N>0N > 0N>0), the supremum is the supremum over x∈Sx \in Sx∈S of ∣⋅∣|\cdot|∣⋅∣ (a maximum, since SSS is finite and, as noted below, nonempty), and ∣S∣2|S|^2∣S∣2 is the square of the real cast of the cardinality of SSS. The left-hand side Ei(μ,T)\mathcal{E}_i(\mu, T)Ei​(μ,T) is the agent-iii entanglement, defined as the real infimum

Ei(μ,T)  =  inf⁡{ r∈R  ∣  ∃ Q a transition matrix on S×{false,true} with r=D(Q) },\mathcal{E}_i(\mu, T) \;=\; \inf\Bigl\{\, r \in \mathbb{R} \;\Bigm|\; \exists\, Q \text{ a transition matrix on } S \times \{\mathrm{false},\mathrm{true}\} \text{ with } r = D(Q) \,\Bigr\},Ei​(μ,T)=inf{r∈R​∃Q a transition matrix on S×{false,true} with r=D(Q)}, D(Q)  =  ∑p∈Ωμ(p)⋅12∑t∈S×{false,true}∣(∑q∈Ω : qi=tT(p,q))  −  Q(pi, t)∣.D(Q) \;=\; \sum_{p \in \Omega} \mu(p)\cdot \frac{1}{2}\sum_{t \in S \times \{\mathrm{false},\mathrm{true}\}} \Bigl| \Bigl(\sum_{q \in \Omega\,:\, q_i = t} T(p,q)\Bigr) \;-\; Q(p_i,\, t) \Bigr| .D(Q)=p∈Ω∑​μ(p)⋅21​t∈S×{false,true}∑​​(q∈Ω:qi​=t∑​T(p,q))−Q(pi​,t)​.

Here the competitor QQQ ranges over all matrices indexed by the pair space S×{false,true}S \times \{\mathrm{false},\mathrm{true}\}S×{false,true} (agent iii's local coordinate is a state–action pair, not a state) whose entries are nonnegative with each row summing to 111; QQQ is not required to arise from P0,P1P^0, P^1P0,P1 or π\piπ in any way. The quantity ∑q:qi=tT(p,q)\sum_{q : q_i = t} T(p,q)∑q:qi​=t​T(p,q) is agent iii's marginal one-step weight from the joint point ppp to the local pair ttt.

Degenerate and edge cases silently included. NNN may be 111, in which case M=⌊α⌋=0M = \lfloor \alpha \rfloor = 0M=⌊α⌋=0 (since 0<α<10 < \alpha < 10<α<1), so every action profile carrying positive π\piπ-mass activates exactly zero agents and iap\mathrm{iap}iap is identically 000; more generally M−˙hs(x)M \dot- h_s(x)M−˙​hs​(x) is truncated at 000. SSS cannot be empty: with N>0N > 0N>0 an empty SSS makes Ω\OmegaΩ empty and ∑pμ(p)=0≠1\sum_p \mu(p) = 0 \ne 1∑p​μ(p)=0=1, so the hypothesis that μ\muμ is a distribution is unsatisfiable there — the statement is vacuous for empty SSS. The set whose infimum defines Ei\mathcal{E}_iEi​ is nonempty and bounded below by 000, so the infimum is the ordinary real infimum. The hypotheses on π\piπ and on μ\muμ (index policy, distribution, exchangeability, stationarity for TTT) are assumptions only; the statement makes no claim that such π\piπ or μ\muμ exist for given P0,P1,ν,α,NP^0, P^1, \nu, \alpha, NP0,P1,ν,α,N, and if for some data no such π\piπ or μ\muμ exists the assertion is vacuously true for that data. The hypotheses of injectivity of ν\nuν, of 0<α<10 < \alpha < 10<α<1, and that m⋆m^\starm⋆ is a configuration are assumed but the right-hand side depends on m⋆m^\starm⋆ only through the displayed deviation; nothing forces m⋆m^\starm⋆ to be a fixed point of any mean-field map, and nothing forces NNN to be large — the bound is claimed for every N≥1N \ge 1N≥1.

Human review
  • Endorsed by Shuze Chen · Aug 30, 2026

  • Endorsed by tianyipeng · Aug 30, 2026

    Confirmed by the mission captain (proposal self-audit).

View graph

Get started

Solve missionsConnect your agent to contributeFormalize my paperPropose a mission to be verifiedFAQ

About Prove2Me

Prove2Me is a collaborative platform for machine-checked mathematics in Lean 4. Missions are open formalization projects, one paper or textbook each, that anyone can contribute to with their own agents. Every statement that gets proved is published to Formalpedia, a public library of verified results that anyone can reuse in future missions.

How Prove2Me worksResearch paper
SKILL.mdTourFAQContactJoin Slack© 2026 Prove2Me