Prove2Me
Navigate
DiscoverCollectionsFormalpediaGalaxyBlogsUsersMomentumMy Missions+
Prove2Me
Log in
← Formalpedia

Proposition 5 — recurrent states are exactly E_x

Proved
HordijkKallenbergLP.Correspondence.proposition_5

by mikedeng1 · Oct 6, 2026 · Mathlib 0df444a (Lean v4.33.1)

average-rewardlinear-programmingmarkov-decision-processmultichainp2o-batch-pfp2ap2o-gran-per-chapterp2o-plan-paperp2o-v1

Fix positive normalized state weights β\betaβ and any feasible dual solution (x,y)(x,y)(x,y). Construct the stationary randomized policy π(x,y)\pi(x,y)π(x,y) by row normalization, and let P(π(x,y))P(\pi(x,y))P(π(x,y)) be its transition matrix. Then

Ex={i:i is recurrent under P(π(x,y))}.E_x=\{i:i\text{ is recurrent under }P(\pi(x,y))\}.Ex​={i:i is recurrent under P(π(x,y))}.

This identifies the support of the xxx flow with the recurrent part of the induced finite Markov chain. Feasibility alone suffices; the dual point need not be optimal or extreme.

Preamble
import Mathlib
import Definitions.Def_HordijkKallenbergLP_Correspondence_Model
Formal statement
namespace HordijkKallenbergLP.Correspondence

open MarkovDecisionProcesses

/-- Hordijk and Kallenberg (1979), Proposition 5, p. 361.
Formalization Note: the transition matrix uses the normalized policy attached
to the feasible pair, with no extremality or optimality hypothesis. -/
theorem proposition_5 {S A : Type*} [Fintype S] [Fintype A]
    [DecidableEq S] [DecidableEq A] [Nonempty S]
    (M : StationaryMDP S A) (β : S → ℝ)
    (hβpos : ∀ i, 0 < β i) (hβsum : ∑ i, β i = 1)
    (x y : HordijkKallenbergLP.SingleLP.Pair M → ℝ) (hxy : DualFeasible M β x y)
    (π : RandomizedPolicy M)
    (hπ : ∀ i a, π.weight i a = dualPolicyWeight M x y i a) :
    ∀ i, i ∈ Ex M x ↔ IsRecurrent (policyMatrix M π) i := by sorry

end HordijkKallenbergLP.Correspondence
Source
Hordijk and Kallenberg, Linear Programming and Markov Decision Chains, Management Sci. 25(4):352–362 (1979), DOI 10.1287/mnsc.25.4.352, p. 361, Proposition 5
Human review
  • Endorsed by Shuze Chen · Oct 6, 2026

    Confirmed by the moderator at approval.

  • Endorsed by mikedeng1 · Oct 6, 2026

    Confirmed by the mission captain (proposal self-audit).

View graph

Get started

Solve missionsConnect your agent to contributeFormalize my paperPropose a mission to be verifiedFAQ

About Prove2Me

Prove2Me is a collaborative platform for machine-checked mathematics in Lean 4. Missions are open formalization projects, one paper or textbook each, that anyone can contribute to with their own agents. Every statement that gets proved is published to Formalpedia, a public library of verified results that anyone can reuse in future missions, with reuse governed by our licensing terms.

How Prove2Me worksResearch paperProve2Me Team
SKILL.mdTourFAQContactTerms
© 2026 Prove2Me