Rewards are almost surely bounded under the trajectory law
ProvedTPG.ae_abs_Y_lecausal-inferencemarkov-chainnonstationaritypolicy-gradienttreatment-effect
Setting. is a finite state space, actions are , time is (the source's time ). The environment has transition matrices , initial distribution and reward laws on . For a sequence of treatment probabilities, and denote the law of the trajectory when , the actions are independent with , , and the rewards are conditionally independent with (Env.trajLaw T q); the Bernoulli policy is .
Lemma. If for all , then for every horizon and every real sequence ,
i.e. almost surely for every .
The lemma lets bounded-reward arguments be applied to functions of the observed rewards, which are bounded only almost surely.
Preamble
import Mathlib import Definitions.Def_TPG_Model import Definitions.Def_TPG_Markov open MeasureTheory ProbabilityTheory open scoped BigOperators ENNReal
Formal statement
namespace TPG
theorem ae_abs_Y_le {S : Type} [Fintype S] [MeasurableSpace S] [MeasurableSingletonClass S]
(E : Env S) (T : ℕ) (q : ℕ → ℝ) (M : ℝ) (hM : E.RewardBound M) :
∀ᵐ ω ∂(E.trajLaw T q), ∀ t : Fin T, |Y t ω| ≤ M := by sorry
end TPGSource
Johari, Peng, Xing, Estimation of Treatment Effects Under Nonstationarity via the Truncated Policy Gradient Estimator, revised manuscript of October 2026 (revision of arXiv:2506.05308v3, https://arxiv.org/abs/2506.05308); numbering follows the revised manuscript. Auxiliary statement, not a numbered result of the paper: it transfers the boundedness assumption R([-M,M] | x,z) = 1 of §2.2, p. 9, to the trajectory.