One-step factorisation of the infinite-horizon canonical bandit model
ProvedBanditAlgorithm.banditTrajMeasure_joint_eq_compProdThe infinite-horizon canonical bandit model advances one round at a time through the one-round step kernel.
Let be the law of the infinite interaction trajectory of policy in environment , and write for the first completed rounds. Then the joint law of the pair factorises as
where is the one-round step kernel: given the history, draw the arm from the policy and then the reward from that arm's distribution.
In words: conditionally on the first rounds, round is generated by the step kernel, exactly as in the finite-horizon canonical model. This is the structural bridge between the two canonical models of Section 4.6 — the finite-horizon one, defined by an explicit recursion, and the infinite-horizon one, obtained from the Ionescu-Tulcea theorem — and it is what allows a computation on the infinite trajectory space (a divergence, a martingale increment) to be carried out one round at a time.
Formalization Note Trajectory coordinate is round , so banditTrajPrefix k n retains coordinates and the new observation is coordinate . Mathlib states the corresponding fact for Kernel.trajMeasure with histories indexed by an initial segment Iic a of the naturals; the reconciliation with the Fin n indexing of the canonical model is part of the content.
import Definitions.Def_BanditTrajectory open MeasureTheory ProbabilityTheory
theorem BanditAlgorithm.banditTrajMeasure_joint_eq_compProd {k : ℕ}
(ν : StochasticBandit k) (π : BanditPolicy k) (n : ℕ) :
(banditTrajMeasure ν π).map (fun ω ↦ (banditTrajPrefix k n ω, ω n))
= ((banditTrajMeasure ν π).map (banditTrajPrefix k n)) ⊗ₘ
(banditStepKernel ν π n) := by
sorry