Joint law of the state and action at one time: , independent
ProvedTPG.integral_fun_state_actioncausal-inferencemarkov-chain
Setting. is a finite state space, actions are , time is (the source's time ). The environment has transition matrices , initial distribution and reward laws on . For a sequence of treatment probabilities, and denote the law of the trajectory when , the actions are independent with , , and the rewards are conditionally independent with (Env.trajLaw T q); the Bernoulli policy is .
Lemma. Let for all . For every time and every function ,
So has distribution and is an independent Bernoulli variable. This is the one-time marginal behind all first-moment computations.
Preamble
import Mathlib import Definitions.Def_TPG_Model import Definitions.Def_TPG_Estimators import Definitions.Def_TPG_Markov import Definitions.Def_TPG_Bridge open MeasureTheory ProbabilityTheory open scoped BigOperators ENNReal
Formal statement
namespace TPG
theorem integral_fun_state_action {S : Type} [Fintype S] [DecidableEq S] [MeasurableSpace S] [MeasurableSingletonClass S]
(E : Env S) (T : ℕ) (q : ℕ → ℝ) (hq : ∀ t : ℕ, 0 ≤ q t ∧ q t ≤ 1)
(t : Fin T) (g : S → Bool → ℝ) :
∫ ω, g (X t ω) (Z t ω) ∂(E.trajLaw T q)
= ∑ a : S, E.stateVec q (t : ℕ) a
* (q (t : ℕ) * g a true + (1 - q (t : ℕ)) * g a false) := by sorry
end TPGSource
Johari, Peng, Xing, Estimation of Treatment Effects Under Nonstationarity via the Truncated Policy Gradient Estimator, revised manuscript of October 2026 (revision of arXiv:2506.05308v3, https://arxiv.org/abs/2506.05308); numbering follows the revised manuscript. Auxiliary statement, not a numbered result of the paper: it is the marginal distribution of (X_t, Z_t) implied by the model of §2.2–2.3, pp. 8–9, in the notation μ_q^t = ρ^⊤ P_q^{1→t} of Appendix D, p. 31.