Regularity of the finite Gittins terminal potential
ProvedBanditAlgorithm.gittins_terminal_potential_regularbanditsgittins-indexprobability
For a measurable reward function and discount factor , assume discounted absolute rewards are integrable. Then, at every finite horizon , the sum of the arms' current Gittins retirement values is a measurable function of the bandit history and is integrable under the history law generated by every Markov bandit policy and initial state vector.
This regularity theorem is the measure-theoretic interface required to take expectations of the terminal potential and to transport those expectations through common-stack couplings.
Preamble
import Definitions.Def_GittinsTerminalPotential
Formal statement
open MeasureTheory ProbabilityTheory ENNReal
theorem BanditAlgorithm.gittins_terminal_potential_regular
{k : ℕ} {S : Type*} [MeasurableSpace S]
(P : Kernel S S) [IsMarkovKernel P]
{r : S → ℝ} (hr : Measurable r)
{α : ℝ} (hα0 : 0 < α) (hα1 : α < 1)
(hint : DiscountedRewardIntegrable P r α) :
∀ n : ℕ,
Measurable (currentGittinsRetirementPotential P r α :
MarkovBanditHistory k S n → ℝ) ∧
∀ (π : MarkovBanditPolicy k S) (x : Fin k → S),
Integrable (currentGittinsRetirementPotential P r α)
(markovBanditMeasure P π x n) := by
sorry
Source
Tor Lattimore and Csaba Szepesvári, Bandit Algorithms, Assumption 35.6 and §35.4 proof of Lemma 35.10, printed pp. 447 and 452–453 (free PDF pp. 455 and 460–461), https://tor-lattimore.com/downloads/book/book.pdf