Theorem 5.3 (Ross) — uniformly bounded mean return times make h_β uniformly bounded
ProvedArapostathisAC.RossRecurrence.theorem_5_3Consider the countable-state controlled Markov process of §5: state space , nonempty compact admissible action sets , and a nonnegative cost and transition probabilities that are continuous on . As §5.1 assumes, the cost is bounded: for all and . For , let be the optimal -discounted cost from state over all admissible policies, and let
be the differential discounted value function. For , let be the return time to state of the process controlled by .
Theorem 5.3. Suppose there is a constant such that
Then is bounded uniformly in and : there is a constant with for all and .
This recurrence condition, due to Ross, implies the hypothesis of Theorem 5.2. Theorem 5.2 then gives a bounded solution of the average cost optimality equation and an average optimal stationary policy.
Formalization Note. The bounded-cost hypothesis is the standing assumption of §5.1 ("In this section, we assume that is bounded"). It makes finite, so is a genuine difference of real numbers. The return time is valued in and counts from , so it applies from too. Its mean is a lower Lebesgue integral, so (5.7) forces almost surely. is the infimum over all history-dependent randomized admissible policies. The bound is chosen before and .
import Mathlib import Definitions.Def_ArapostathisAC_RossRecurrence_CMP import Definitions.Def_ArapostathisAC_RossRecurrence_ReturnTime open MeasureTheory ProbabilityTheory open scoped ENNReal
namespace ArapostathisAC.RossRecurrence
/-- Theorem 5.3 (Ross; pp. 301–302), under the standing assumption of §5.1 that the cost is bounded:
if the mean return time `E^f_i[τ]`, `τ = min{t ≥ 1 : X_t = 0}`, is below a constant `K > 0` for
every `f ∈ Π_SD` and every initial state `i` (5.7), then `h_β(i) = J*_β(i) − J*_β(0)` is bounded
uniformly in `β ∈ (0, 1)` and `i`. -/
theorem theorem_5_3 {A : Type*} [MetricSpace A] [MeasurableSpace A] [BorelSpace A]
(M : CMP A) (M₀ : ℝ) (hc : ∀ i, ∀ a ∈ M.U i, M.c i a ≤ M₀)
(K : ℝ) (hK : 0 < K)
(h57 : ∀ f : StationaryPolicy M, ∀ i, meanReturnTime M f i < ENNReal.ofReal K) :
∃ B : ℝ, ∀ β ∈ Set.Ioo (0 : ℝ) 1, ∀ i, |hRel M β i| ≤ B := by sorry
end ArapostathisAC.RossRecurrence
Confirmed by the mission captain (proposal self-audit).
Confirmed by the moderator at approval.