The Elliptical Potential Lemma for General Distributions with an Application to Linear Thompson Sampling: For Any Prior and Noise, E[Σ Aₜᵀ Γₜ Aₜ] ≤ 2 max(σ², 1) log det(I + T Γ₁)Research Paper
Motivation
In a linear bandit problem, a learner chooses a vector and observes a noisy scalar response that carries information about an unknown parameter. Regret analyses often need to control the sum of the learner's uncertainty in the directions it chooses. The familiar elliptical potential lemma does this when uncertainty is represented by the inverse of a deterministic, successively updated Gram matrix. Bayesian learning presents a different object: the posterior covariance of the unknown parameter given the observations. It changes with the realized data and can increase on an individual sample path. Hamidi and Bayati's technical note establishes a potential bound for this posterior object under general prior and noise distributions and uses it to analyze linear Thompson sampling.
The deterministic lemma and the posterior result answer related but distinct questions. In the deterministic setting, the update is fixed by the chosen vectors. Here, actions may depend on earlier observations and the next posterior covariance is itself random. The mission concerns the latter claim, including the observation model that gives the covariance its meaning.
Setting
Let be a probability space with an increasing filtration . An unknown parameter obeys almost surely. At time , the learner chooses an -measurable action with almost surely. An observation , measurable at time , has conditional mean and conditional variance at most when both and are known. No Gaussian law or independent noise model is part of these assumptions.
Write for the posterior covariance matrix. The quantity is the remaining parameter uncertainty in the chosen direction. The matrix function used to measure accumulated uncertainty is
The matrix order means that is positive semidefinite. Conditional expectations of matrices are taken entrywise. The filtration can contain information beyond the actions and observations; it need not be generated by them.
Formalization targets
The first target is the source's stochastic variance reduction inequality. For every , almost surely,
The matrix potential targets are the concavity of on the positive semidefinite cone and equation (2.2): if , then
Two scalar and covariance bounds from the proof complete the milestone list: on , and almost surely. The main target, a conditional formulation of Theorem 2.1, is
The source prints an unconditional expectation on the left although can be random. Conditioning on supplies a coherent general statement; it becomes the printed display when the initial information is trivial. The goal retains the explicit factor and the horizon inside the log determinant.
Significance
The result controls a cumulative, adaptively selected posterior quadratic form using only the initial posterior covariance, the horizon and the conditional noise bound. It permits a regret analysis of linear Thompson sampling without replacing the prior by a Gaussian model. In the source, this bound feeds the Bayesian regret theorem for that algorithm; that later theorem is outside this mission's scope. Its proof would also require formalizing the sampling policy and its action-selection law.
The formal development fixes a reusable interface for finite-dimensional posterior covariance under a filtration: a parameter, adapted actions, conditional moments of observations, and an entrywise matrix covariance. It also states the matrix potential claims separately from the probability model, so those claims can serve other analyses of covariance updates. The statements compile locally as open Lean theorems; this mission does not claim machine-checked proofs of them. The deterministic elliptical potential statement already available elsewhere uses a Gram-matrix update and does not establish this posterior result.
Difficulty
Posterior covariance is not pathwise decreasing. The paper's Section 2 example has an observation after which . A bound that assumes for every outcome therefore fails at the first stochastic step. The issue matters especially when actions are adapted: later actions can point toward directions whose posterior variance became large after earlier data. Conditional covariance reduction is a matrix inequality, while the target is a scalar sum of quadratic forms; its potential must remain valid across those different objects.
Formalization scope
Lean uses vectors Fin d → ℝ and the Euclidean dot product, so the unit-ball assumptions mean . The posterior covariance is the conditional expectation of centered coordinate products. The expectation of a covariance matrix is defined entrywise; matrix order is positive semidefiniteness of the difference. The conditioning sigma-algebra for is , not merely . Every observation is explicitly square-integrable, so Lean's totalized conditional variance cannot silently return zero for a non-square-integrable input. The source's bound on the parameter already supplies its integrability.
Rounds start at ; the finite sum uses . At it is empty and both sides of the goal are zero. No positivity assumption on is imposed, since the statements use and the zero-denominator covariance correction has zero numerator. The log determinant is used on positive semidefinite covariance matrices with nonnegative scalar argument, where its determinant is positive. The paper's square-root factorization of is represented by its equal direct matrix formula. A deterministic Gram matrix in place of , an independent-observation construction in place of the conditional-moment model, or a supremum-norm ball would change the theorem. The separate matrix and probability lemmas, conditional expectation infrastructure, and finite-dimensional matrix order are the principal reusable components.
Selected references
- Nima Hamidi and Mohsen Bayati, The Elliptical Potential Lemma for General Distributions with an Application to Linear Thompson Sampling, technical note, Operations Research, 2023; formalization source: arXiv:2102.07987v3, 2022, §2 and Appendix A.