Display (5.10) — J*_β(0) − J*_β(i) ≤ (1 − β^K) J*_β(0) ≤ (1 − β^K) M/(1 − β) ≤ MK
ProvedArapostathisAC.RossRecurrence.eq_5_10controlled-markov-processdiscounted-costp2o-batch-pfp2bp2o-gran-per-chapterp2o-plan-paperp2o-v1return-time
Consider the countable-state controlled Markov process of §5 with bounded cost, for every and , under Ross's recurrence condition (5.7): for all and , where and . Then for every and every state ,
Together with (5.9), this bounds by .
Formalization Note. The last inequality needs , which is not a separate hypothesis: since , (5.7) forces . Under bounded cost is finite, so its value is used as a real number.
Preamble
import Mathlib import Definitions.Def_ArapostathisAC_RossRecurrence_CMP import Definitions.Def_ArapostathisAC_RossRecurrence_ReturnTime open MeasureTheory ProbabilityTheory open scoped ENNReal
Formal statement
namespace ArapostathisAC.RossRecurrence
/-- Display (5.10) (proof of Theorem 5.3, p. 302): under bounded costs `c ≤ M₀` and (5.7), for every
`β ∈ (0, 1)` and every state `i`,
`J*_β(0) − J*_β(i) ≤ (1 − β^K) J*_β(0) ≤ (1 − β^K) M₀/(1 − β) ≤ M₀ K`. -/
theorem eq_5_10 {A : Type*} [MetricSpace A] [MeasurableSpace A] [BorelSpace A]
(M : CMP A) (M₀ : ℝ) (hc : ∀ i, ∀ a ∈ M.U i, M.c i a ≤ M₀)
(K : ℝ) (hK : 0 < K)
(h57 : ∀ f : StationaryPolicy M, ∀ i, meanReturnTime M f i < ENNReal.ofReal K)
(β : ℝ) (hβ : β ∈ Set.Ioo (0 : ℝ) 1) (i : ℕ) :
(discValue M β 0).toReal - (discValue M β i).toReal ≤ (1 - β ^ K) * (discValue M β 0).toReal ∧
(1 - β ^ K) * (discValue M β 0).toReal ≤ (1 - β ^ K) * (M₀ / (1 - β)) ∧
(1 - β ^ K) * (M₀ / (1 - β)) ≤ M₀ * K := by sorry
end ArapostathisAC.RossRecurrence
Source
Arapostathis, Borkar, Fernández-Gaucherand, Ghosh, Marcus, Discrete-time controlled Markov processes with average cost criterion: a survey, SIAM J. Control Optim. 31(2) (1993), p. 302, proof of Theorem 5.3, (5.10)
Human review
Confirmed by the mission captain (proposal self-audit).
Confirmed by the moderator at approval.