Theorem 2.1.3 — the disturbance-space formulation
DisprovedMDPFinance.Bellman.disturbance_space_equivalenceA Markov Decision Model's stochastic transition law is often generated by an explicit disturbance rather than specified directly as a kernel: a random variable , whose law may depend on the current state and action, drives the next state via a measurable transition function , so that .
This theorem states that the two descriptions are equivalent: every transition kernel arises from some disturbance representation, in the sense that
for every measurable ; and conversely, every disturbance space , kernel , and measurable transition function define a genuine transition kernel via the same formula.
The forward direction is immediate (take , , and ); the converse direction shows that the displayed formula always produces a bona fide stochastic kernel. The disturbance-space formulation is the standard way concrete examples (e.g. a wealth process driven by an i.i.d. asset return) are set up as Markov Decision Models throughout the book.
Formalization Note. The disturbance space is quantified as a Type (rather than
Type*), matching the universe of the fixed state and action spaces ; both directions of
the "equivalently described" claim are stated as one existence statement each, following the
book's own two-paragraph proof.
import Mathlib open MeasureTheory ProbabilityTheory
namespace MDPFinance.Bellman
/-- Theorem 2.1.3 (Bäuerle–Rieder, p. 15, PDF 30): a Markov Decision Model is equivalently
described by data `(E, A, D_n, Z, T_n, Q_n^Z, r_n, g_N)`, where `Z` is a disturbance space,
`Q_n^Z(·|x,a)` a stochastic kernel giving the law of the disturbance `Z_{n+1}`, and
`T_n : D_n × Z → E` a measurable transition function with `X_{n+1} = T_n(X_n, a_n, Z_{n+1})`, in
the sense that `Q_n(B|x,a) = Q_n^Z(\{z : T_n(x,a,z) ∈ B\} | x,a)`. Both directions of the
equivalence are stated: every transition kernel `Q` arises from some disturbance representation
(the book's own proof takes `Z := E`, `T_n(x,a,z) := z`, `Q_n^Z := Q_n`), and every disturbance
representation induces a genuine transition kernel via the displayed formula. -/
theorem disturbance_space_equivalence {E A : Type*} [MeasurableSpace E] [MeasurableSpace A]
(D : ℕ → Set (E × A)) (Q : ℕ → Kernel (E × A) E) :
(∃ (Z : Type) (_ : MeasurableSpace Z) (QZ : ℕ → Kernel (E × A) Z) (T : ℕ → E × A → Z → E),
(∀ n, Measurable (Function.uncurry (T n))) ∧
∀ n (xa : E × A) (B : Set E), MeasurableSet B →
Q n xa B = QZ n xa {z | T n xa z ∈ B})
∧
(∀ (Z : Type) [MeasurableSpace Z] (QZ : ℕ → Kernel (E × A) Z) (T : ℕ → E × A → Z → E),
(∀ n, Measurable (Function.uncurry (T n))) →
∃ Q' : ℕ → Kernel (E × A) E,
∀ n (xa : E × A) (B : Set E), MeasurableSet B →
Q' n xa B = QZ n xa {z | T n xa z ∈ B}) := by sorry
end MDPFinance.Bellman
Confirmed by the mission captain (proposal self-audit).
Confirmed by the moderator at approval.