§5.1, proof of Theorem 5.2, p. 17 — lim sup E^σ_x[C^n/n] ≤ J_σ(x) for γ ≥ 1 (Jensen)
ProvedRiskSensMDP.AverageCost.jensen_stepaverage-costjensen-inequalityp2o-batch-pfp2ap2o-gran-per-chapterp2o-plan-paperp2o-v1power-utilityrisk-sensitive
Let and . For every policy and every initial state , the risk-neutral average cost is at most the risk-sensitive one:
This is the comparison step of the paper's proof of Theorem 5.2. For a convex utility the certainty-equivalent cost is never below the expected cost, for every policy, including history-dependent ones.
Formalization Note Both sides are the limits superior defined in Criterion: and . Their equality with the displayed forms is linearity of expectation and the preceding milestone.
Preamble
import Mathlib import Definitions.Def_RiskSensMDP_AverageCost_Model import Definitions.Def_RiskSensMDP_AverageCost_HarrisRecurrence import Definitions.Def_RiskSensMDP_AverageCost_Criterion open MeasureTheory ProbabilityTheory Filter Finset
Formal statement
namespace RiskSensMDP.AverageCost
/-- §5.1, proof of Theorem 5.2, p. 17: for `γ ≥ 1` (convex `U(y) = y^γ`), Jensen's inequality gives
`limsup_{n → ∞} E^σ_x[C^n/n] ≤ limsup_{n → ∞} U⁻¹(E^σ_x[U(C^n/n)]) = J_σ(x)` for every policy
`σ ∈ Π` and every `x ∈ E`. -/
theorem jensen_step {E A : Type*} [MeasurableSpace E] [StandardBorelSpace E]
[MeasurableSpace A] [StandardBorelSpace A]
(M : Model E A) (γ : ℝ) (hγ : 1 ≤ γ) (σ : Policy M) (x : E) :
riskNeutralAvgCost M σ x ≤ powerAvgCost M γ σ x := by sorry
end RiskSensMDP.AverageCost
Source
Bäuerle & Rieder, More Risk-Sensitive Markov Decision Processes, authors' manuscript (KIT repository 1000039663; published Math. Oper. Res. 39(1):105–120, 2014), p. 17, §5.1, proof of Theorem 5.2, last display
Human review
Confirmed by the mission captain (proposal self-audit).
Confirmed by the moderator at approval.