§C.1 (p.30) —
ProvedCalibeatBandit.rec_identity_C_1bandit-feedbackcalibeatingonline-learningp2o-gran-per-chapterp2o-plan-paperp2o-strengthenedp2o-v1swap-regret
Consider any learner and any run of the interaction protocol, with the learner's mixed strategy in round . Then
and for every round . Hence the expected pseudo reward loss is the expected reward loss.
This converts the bound on into the bound on in Lemma 3.1 and Theorem 3.2.
Formalization Note is run.played t, the policy evaluated on the learner's observations.
Differences from the source:
- generalization: the learner of Algorithm 1 (and, by the proof of Theorem 3.2, the bandit algorithm) → every learner policy; reason: the argument uses only that has conditional law given the past and ; both algorithms are instances
- encoding: → the Bochner integral against ; reason: in every run the integrands are bounded and almost everywhere measurable (the conditional law pins a.s.), so the integrals are the expectations
- encoding: rounds , actions → rounds (
Finset.range T), actionsFin K; reason: a relabelling; same instances
Preamble
import Mathlib import Definitions.Def_CalibeatBandit_ProtocolRun import Definitions.Def_CalibeatBandit_pseudoRewardLoss import Definitions.Def_CalibeatBandit_rewardLoss set_option autoImplicit false open MeasureTheory Matrix
Formal statement
namespace CalibeatBandit
/-- **The expected pseudo reward loss is the expected reward loss** (§C.1, proof of Lemma C.1,
p.30). In every run of the §2.1 protocol, with `p_t` the learner's mixed strategy,
`E[R^{Rec,p}_T] = E[Rec_T] − ∑_{t=1}^T E[⟨p_t, y_t⟩ − y_t(I_t)]`, and the expectation of
`⟨p_t, y_t⟩ − y_t(I_t)` is zero for every round.
Hartline, Wang, Wu, *Calibeating Bandit Advice* (manuscript, sha256 a65ccb08…), §C.1, proof of Lemma C.1, p.30; reused in the proof of Theorem 3.2, p.36 ("the same
conversion … still holds here").
**Formalization Note.** Stated for an arbitrary learner `policy` (the paper uses it for
Algorithm 1 and for the bandit algorithm); `p_t` is `run.played t`. Rounds are `0, …, T − 1`;
`E` is the Bochner integral; `⟨p, y⟩` is `p ⬝ᵥ y`. -/
theorem rec_identity_C_1 {K T : ℕ} {Ω : Type*} [MeasurableSpace Ω] (P : Measure Ω) [IsProbabilityMeasure P]
(policy : Policy K) (run : ProtocolRun K T P policy) :
∫ ω, pseudoRewardLoss T (fun t => run.played t ω) (fun t => run.y t ω) (fun t => run.J t ω) ∂P
= ∫ ω, rewardLoss T (fun t => run.y t ω) (fun t => run.J t ω) (fun t => run.I t ω) ∂P
- ∑ t ∈ Finset.range T, ∫ ω, (run.played t ω ⬝ᵥ run.y t ω - run.y t ω (run.I t ω)) ∂P ∧
∀ t < T, ∫ ω, (run.played t ω ⬝ᵥ run.y t ω - run.y t ω (run.I t ω)) ∂P = 0 := by sorry
end CalibeatBandit
Source
Hartline, Wang, Wu, Calibeating Bandit Advice (manuscript, STOC 2027 submission, sha256 a65ccb0880e6e4698eee1717d9a7121019b84a5ebcb4dfaa2e1614dbd3de0b44), §C.1, proof of Lemma C.1, p.30 (PDF 30); reused in the proof of Theorem 3.2, p.36
Human review
Confirmed by the mission captain (proposal self-audit).
Confirmed by the moderator at approval.