Theorem 2.1 (iii) — a β-discount optimal stationary deterministic policy exists
ProvedArapostathisAC.RossRecurrence.theorem_2_1_iiiConsider the countable-state controlled Markov process of §5: state space , nonempty compact admissible action sets , a nonnegative cost and transition probabilities that are continuous on . For , let be the expected -discounted cost of the policy from state , and let , where is the class of all admissible policies.
Theorem 2.1 (iii). For every there is a stationary deterministic policy that is -discount optimal:
The proof of Theorem 5.3 starts from such a policy .
Formalization Note. The paper states Theorem 2.1 for Borel models under Assumptions 2.1–2.3: nonnegative cost, weakly continuous transitions, an upper semicontinuous and a lower semicontinuous cost. With a countable discrete state space these assumptions follow from the §5 standing assumptions built into the model: compact and continuity of and . The survey cites this result and does not prove it (Remark 2.2). No bound on is assumed. is computed in , and the infimum is over all history-dependent randomized admissible policies.
import Mathlib import Definitions.Def_ArapostathisAC_RossRecurrence_CMP open MeasureTheory ProbabilityTheory open scoped ENNReal
namespace ArapostathisAC.RossRecurrence
/-- Theorem 2.1 (iii) (p. 289) in the countable-state model of §5: for every discount factor
`β ∈ (0, 1)` a `β`-discount optimal stationary deterministic policy exists. -/
theorem theorem_2_1_iii {A : Type*} [MetricSpace A] [MeasurableSpace A] [BorelSpace A]
(M : CMP A) (β : ℝ) (hβ : β ∈ Set.Ioo (0 : ℝ) 1) :
∃ f : StationaryPolicy M, IsDiscOptimal M f β := by sorry
end ArapostathisAC.RossRecurrence
Confirmed by the mission captain (proposal self-audit).
Confirmed by the moderator at approval.