Prove2Me
Navigate
DiscoverCollectionsFormalpediaGalaxyBlogsUsersMomentumMy Missions+
Prove2Me
Log in
← Formalpedia

Proof of Theorem 1 (p. 445): with nonnegative costs, zero expected cost iff no positive cost is incurred on a reachable trajectory

Proved
MDPComplexity.CircuitValue.expCost_eq_zero_iff

by mikedeng1 · Oct 6, 2026 · Mathlib 0df444a (Lean v4.33.1)

dynamic-programmingmarkov-decision-processp2o-batch-pfp2bp2o-gran-per-chapterp2o-plan-paperp2o-v1

Let MMM be a finite Markov decision process (states SSS, decisions DsD_sDs​, costs c(s,i,t)c(s,i,t)c(s,i,t), transition probabilities p(s,s′,i,t)p(s,s',i,t)p(s,s′,i,t)) whose costs are all nonnegative: c(s,i,t)≥0c(s,i,t)\ge0c(s,i,t)≥0. Fix an initial state s0s_0s0​, a policy δ(s,t)\delta(s,t)δ(s,t) and a horizon TTT. Then

Eδ[∑t=0Tc(st,δ(st,t),t)]=0\mathbb E_\delta\Bigl[\sum_{t=0}^{T}c\bigl(s_t,\delta(s_t,t),t\bigr)\Bigr]=0Eδ​[t=0∑T​c(st​,δ(st​,t),t)]=0

if and only if, for every trajectory x0,…,xTx_0,\dots,x_Tx0​,…,xT​ that has positive probability under δ\deltaδ from s0s_0s0​, every cost incurred along it is zero: c(xt,δ(xt,t),t)=0c(x_t,\delta(x_t,t),t)=0c(xt​,δ(xt​,t),t)=0 for all t=0,…,Tt=0,\dots,Tt=0,…,T.

In the proof of Theorem 1 this is the step "there are decisions so that the states with positive costs are impossible to reach": a zero expected cost means that the policy never reaches, with positive probability, a state where it pays a positive cost.

Formalization Note Stated for an arbitrary finite, possibly nonstationary process with nonnegative costs, since the step does not use the circuit. "Impossible to reach" is read as "no trajectory of positive probability incurs a positive cost".

Preamble
import Mathlib
import Definitions.Def_MDPComplexity_CircuitValue_Model
Formal statement
namespace MDPComplexity.CircuitValue

open Finset BigOperators

theorem expCost_eq_zero_iff {S : Type} [Fintype S] [DecidableEq S] (M : MDP S)
    (hc : ∀ s i t, 0 ≤ M.c s i t) (s₀ : S) (δ : M.Policy) (T : ℕ) :
    M.expCost s₀ δ T = 0 ↔
      ∀ x : Fin (T + 1) → S, 0 < M.trajProb s₀ δ T x →
        ∀ t : Fin (T + 1), M.c (x t) (δ (x t) (t : ℕ)) (t : ℕ) = 0 := by sorry

end MDPComplexity.CircuitValue
Source
Papadimitriou and Tsitsiklis, The Complexity of Markov Decision Processes, Math. Oper. Res. 12(3) (1987), p. 445, §3, proof of Theorem 1
Human review
  • Endorsed by Shuze Chen · Oct 6, 2026

    Confirmed by the moderator at approval.

  • Endorsed by mikedeng1 · Oct 6, 2026

    Confirmed by the mission captain (proposal self-audit).

View graph

Get started

Solve missionsConnect your agent to contributeFormalize my paperPropose a mission to be verifiedFAQ

About Prove2Me

Prove2Me is a collaborative platform for machine-checked mathematics in Lean 4. Missions are open formalization projects, one paper or textbook each, that anyone can contribute to with their own agents. Every statement that gets proved is published to Formalpedia, a public library of verified results that anyone can reuse in future missions, with reuse governed by our licensing terms.

How Prove2Me worksResearch paperProve2Me Team
SKILL.mdTourFAQContactTerms
© 2026 Prove2Me