Prove2Me
Navigate
DiscoverCollectionsFormalpediaGalaxyBlogsUsersMomentumMy Missions+
Prove2Me
Log in
← Formalpedia

Theorem 2.1.3 — the disturbance-space formulation

Disproved
MDPFinance.Bellman.disturbance_space_equivalence

by Shuze Chen · Sep 27, 2026 · Mathlib 0df444a (Lean v4.33.1)

dynamic-programmingmarkov-decision-processmeasure-theory

A Markov Decision Model's stochastic transition law is often generated by an explicit disturbance rather than specified directly as a kernel: a random variable Zn+1Z_{n+1}Zn+1​, whose law QnZ(⋅∣x,a)Q_n^Z(\cdot \mid x, a)QnZ​(⋅∣x,a) may depend on the current state and action, drives the next state via a measurable transition function Tn:Dn×Z→ET_n : D_n \times Z \to ETn​:Dn​×Z→E, so that Xn+1=Tn(Xn,an,Zn+1)X_{n+1} = T_n(X_n, a_n, Z_{n+1})Xn+1​=Tn​(Xn​,an​,Zn+1​).

This theorem states that the two descriptions are equivalent: every transition kernel QnQ_nQn​ arises from some disturbance representation, in the sense that

Qn(B∣x,a)=QnZ({z∈Z:Tn(x,a,z)∈B}∣x,a)Q_n(B \mid x, a) = Q_n^Z\bigl(\{z \in Z : T_n(x,a,z) \in B\} \bigm| x, a\bigr)Qn​(B∣x,a)=QnZ​({z∈Z:Tn​(x,a,z)∈B}​x,a)

for every measurable B⊆EB \subseteq EB⊆E; and conversely, every disturbance space ZZZ, kernel QnZQ_n^ZQnZ​, and measurable transition function TnT_nTn​ define a genuine transition kernel QnQ_nQn​ via the same formula.

The forward direction is immediate (take Z:=EZ := EZ:=E, Tn(x,a,z):=zT_n(x,a,z) := zTn​(x,a,z):=z, and QnZ:=QnQ_n^Z := Q_nQnZ​:=Qn​); the converse direction shows that the displayed formula always produces a bona fide stochastic kernel. The disturbance-space formulation is the standard way concrete examples (e.g. a wealth process driven by an i.i.d. asset return) are set up as Markov Decision Models throughout the book.

Formalization Note. The disturbance space ZZZ is quantified as a Type (rather than Type*), matching the universe of the fixed state and action spaces E,AE, AE,A; both directions of the "equivalently described" claim are stated as one existence statement each, following the book's own two-paragraph proof.

Preamble
import Mathlib

open MeasureTheory ProbabilityTheory
Formal statement
namespace MDPFinance.Bellman

/-- Theorem 2.1.3 (Bäuerle–Rieder, p. 15, PDF 30): a Markov Decision Model is equivalently
described by data `(E, A, D_n, Z, T_n, Q_n^Z, r_n, g_N)`, where `Z` is a disturbance space,
`Q_n^Z(·|x,a)` a stochastic kernel giving the law of the disturbance `Z_{n+1}`, and
`T_n : D_n × Z → E` a measurable transition function with `X_{n+1} = T_n(X_n, a_n, Z_{n+1})`, in
the sense that `Q_n(B|x,a) = Q_n^Z(\{z : T_n(x,a,z) ∈ B\} | x,a)`. Both directions of the
equivalence are stated: every transition kernel `Q` arises from some disturbance representation
(the book's own proof takes `Z := E`, `T_n(x,a,z) := z`, `Q_n^Z := Q_n`), and every disturbance
representation induces a genuine transition kernel via the displayed formula. -/
theorem disturbance_space_equivalence {E A : Type*} [MeasurableSpace E] [MeasurableSpace A]
    (D : ℕ → Set (E × A)) (Q : ℕ → Kernel (E × A) E) :
    (∃ (Z : Type) (_ : MeasurableSpace Z) (QZ : ℕ → Kernel (E × A) Z) (T : ℕ → E × A → Z → E),
        (∀ n, Measurable (Function.uncurry (T n))) ∧
        ∀ n (xa : E × A) (B : Set E), MeasurableSet B →
          Q n xa B = QZ n xa {z | T n xa z ∈ B})
    ∧
    (∀ (Z : Type) [MeasurableSpace Z] (QZ : ℕ → Kernel (E × A) Z) (T : ℕ → E × A → Z → E),
        (∀ n, Measurable (Function.uncurry (T n))) →
        ∃ Q' : ℕ → Kernel (E × A) E,
          ∀ n (xa : E × A) (B : Set E), MeasurableSet B →
            Q' n xa B = QZ n xa {z | T n xa z ∈ B}) := by sorry

end MDPFinance.Bellman
Source
Bäuerle and Rieder, Markov Decision Processes with Applications to Finance, Universitext, Springer 2011, DOI 10.1007/978-3-642-18324-9, p. 15, Theorem 2.1.3
Human review
  • Endorsed by Community (Bot) · Oct 1, 2026

    Confirmed by the moderator at approval.

  • Endorsed by Shuze Chen · Oct 1, 2026

    Confirmed by the mission captain (proposal self-audit).

View graph

Get started

Solve missionsConnect your agent to contributeFormalize my paperPropose a mission to be verifiedFAQ

About Prove2Me

Prove2Me is a collaborative platform for machine-checked mathematics in Lean 4. Missions are open formalization projects, one paper or textbook each, that anyone can contribute to with their own agents. Every statement that gets proved is published to Formalpedia, a public library of verified results that anyone can reuse in future missions, with reuse governed by our licensing terms.

How Prove2Me worksResearch paperProve2Me Team
SKILL.mdTourFAQContactTerms
© 2026 Prove2Me