Note 4.2 (corrected): under Condition D, S is selected in {S, Λ(λ)} iff ν(S, x) ≥ λ, and at λ = ν(S, x) a control u is optimal iff ν(S, x, u) = ν(S, x)
ProvedAllocationIndices.conditionD_optimal_control_iffNote 4.2. If a superprocess satisfies Condition D, a control is optimal for if and only if . This follows by supposing that in the statement of Condition D has the parameter .
Stated here, corrected. For a decision process on a countable state space with bounded rewards, , satisfying Condition D, and every state :
- for every , it is optimal to select in state in the SFAS (some optimal policy continues at time from ) if and only if ;
- for , the parameter the note's own justification uses, a control is optimal to apply (some optimal policy continues with at time from ) if and only if .
Why not every . The printed equivalence fails for , in both directions. Take and a state with two controls. pays and leads to a state paying forever. pays and leads to a state paying forever. Both have index , and Condition D holds with . At , applying earns against , so is not optimal, although . Conversely, let pay and then forever and pay and then forever. Then , Condition D holds with , and at both controls earn , so is optimal. At the note is right: selecting is exactly as good as retiring, and a control is optimal exactly when some stationary policy starting with it has index (the constrained stopping problem has an optimal stationary policy, so the supremum in (4.1) is attained where it matters).
import Definitions.Def_AllocationIndices_Superprocess open MeasureTheory ProbabilityTheory BanditAlgorithm
namespace AllocationIndices
theorem conditionD_optimal_control_iff {S U : Type*} [MeasurableSpace S] [Countable S]
[MeasurableSingletonClass S] [MeasurableSpace U] [Fintype U] [Nonempty U]
[MeasurableSingletonClass U] (D : DecisionProcess S U) (hD : D.BoundedRewards)
{a : ℝ} (ha0 : 0 < a) (ha1 : a < 1) (hcond : ConditionD D a) (x : S) :
(∀ lam : ℝ, OptimalToSelect D a lam x ↔ lam ≤ superIndexMax D a x) ∧
(∀ u ∈ D.avail x, OptimalToApply D a (superIndexMax D a x) x u ↔
superIndex D a x u = superIndexMax D a x) := by sorry
end AllocationIndices
Confirmed by the mission captain (proposal self-audit).
Confirmed by the moderator at approval.