Theorem 7.9 / Corollary 7.10: for a reward process with a location parameter and a conjugate prior with location parameter x̄, r(x̄ + c, n) = r(x̄, n) + c and ν(x̄, n) = x̄ + ν(0, n)
ProvedAllocationIndices.location_invariance_of_indexTheorem 7.9 / Corollary 7.10. If the parameter for the family of distributions for a reward process is a location parameter, the sample mean is a sufficient statistic, and has a conjugate prior distribution with the parameters and , where is a location parameter and , and these parameters take the values and when is the next value sampled, then may be identified by its parameters and , and .
Formally: for a sampling model on with parameters , conjugate for , likelihood with location parameter (HasLocationParameter), prior family with location parameter (PriorHasLocationParameter), update meanUpdate, integrable observations (the mean of exists, p. 206), , and the discounted rewards of the chain integrable from every state (L&S Assumption 35.6): for every , and the Gittins index of the reward process in the state equals . The book's improper uniform prior is replaced, as in Corollary 7.10, by a proper conjugate family with a location parameter; the identity for the optimal payoff of is not restated.
Admissible parameters. Conjugacy is required, and the conclusion stated, for only (IsConjugateOn {p | 0 < p.2}, 0 < n): a proper prior has , the book's being the improper prior of the theorem itself. Conjugacy at every cannot hold together with a location parameter. At the update divides by zero (Lean's value is ), so every observation leads to the same state . Conjugacy would then force the posterior of the prior to be the prior for every , but by the location structure that posterior moves with . Stated with conjugacy over all of , the theorem had no instances.
import Definitions.Def_AllocationIndices_Sampling open MeasureTheory ProbabilityTheory BanditAlgorithm
namespace AllocationIndices
/-- **Theorem 7.9 / Corollary 7.10** (pp. 189–191). If the parameter `μ` of the distributions of a
reward process is a location parameter, the sample mean is a sufficient statistic, and `μ` has a
conjugate prior distribution with parameters `x̄` and `n`, where `x̄` is a location parameter of
the prior family and the parameters become `(n x̄ + x)/(n + 1)` and `n + 1` when `x` is the next
value sampled, then the state is identified by `(x̄, n)`, `r(x̄ + c, n) = r(x̄, n) + c`, and
`ν(x̄, n) = x̄ + ν(0, n)`. Stated for the Gittins index of the chain of parameters, under the
book's standing assumptions that the observations have a mean and the discounted rewards are
integrable (L&S Assumption 35.6). -/
theorem location_invariance_of_index (F : SamplingModel ℝ (ℝ × ℝ))
(hconj : F.IsConjugateOn {p | 0 < p.2})
(hlik : HasLocationParameter F.likelihood) (hprior : PriorHasLocationParameter F.prior)
(hupd : F.update = meanUpdate) (hmean : ∀ p, Integrable (fun x : ℝ ↦ x) (F.predictive p))
{a : ℝ} (ha0 : 0 < a) (ha1 : a < 1)
(hint : DiscountedRewardIntegrable F.rewardChain F.rewardOf a) (xb n : ℝ) (hn : 0 < n) :
(∀ c : ℝ, F.rewardOf (xb + c, n) = F.rewardOf (xb, n) + c) ∧
gittinsIndex F.rewardChain F.rewardOf a (xb, n) =
xb + gittinsIndex F.rewardChain F.rewardOf a (0, n) := by sorry
end AllocationIndices
Confirmed by the mission captain (proposal self-audit).
Confirmed by the moderator at approval.