Optimal No-regret Learning in Repeated First-price Auctions 3: Even with Full Information and v_t ≡ 1, Every Bidding Policy Has Worst-Case Expected Regret at Least c√TResearch Paper
Learning to bid in first-price auctions
Most online advertising inventory is sold one impression at a time in automated auctions, and since about 2019 the major ad exchanges have moved from second-price to first-price rules: the highest bidder wins and pays its own bid. A bidder in a first-price auction must trade off winning probability against the price paid, and that trade-off depends on the unknown distribution of competing bids. A bidder who takes part in many such auctions can learn that distribution as it goes. The natural yardstick is regret: the expected shortfall of the bidder's cumulative reward relative to an oracle that knows the distribution of competing bids from the start.
Han, Zhou and Weissman (arXiv:2003.09795, Operations Research, 2025) give bidding policies whose expected regret is over auctions, both for iid and for adversarial private values, under the censored feedback that real exchanges provide (only the winning bid is announced). Their Theorem 9 shows that the rate cannot be improved, even in a much easier setting. This mission is that lower bound.
Related results put it in context. With binary feedback (only "won" or "lost"), regret of order is unavoidable (Kleinberg and Leighton, 2003). With a fixed value, repeated first-price bidding is a pricing problem on a one-dimensional action set whose reward need not be continuous in . Theorem 9 shows that the extra information of the full competing bid buys and nothing better.
Setting
In each round a bidder has a private value , submits a bid , and faces the highest other bid (HOB) , the largest bid of all competitors. The bidder wins when , and its payoff is
The HOBs are iid with an unknown cumulative distribution function on . No smoothness is assumed: may have atoms. The expected reward of bid at value is , and the regret of the bids is
A policy chooses from what it has observed before round , and it may use internal randomness. Theorem 9 makes the problem as easy as possible: the value is in every round, and the HOB is revealed at the end of every round (full information), rather than only when the bidder loses. A policy is then a rule , where is the policy's internal randomness.
The lower bound uses two HOB laws on . For , puts mass on and on , and the reverse. is the law of iid HOBs from . The total variation distance is , and is the Kullback–Leibler divergence.
Formalization targets
Goal: Theorem 9 (p. 36)
There is an absolute constant such that, for every horizon ,
where the supremum is over all HOB distributions on and the infimum is over all full-information policies, randomized ones included. The constant is left unspecified, as in the paper; the paper's proof gives .
Milestones (App. A–B, pp. 36–38)
- Oracle rewards (p. 37): with , , and .
- Separation (20): for every bid , the instantaneous regrets under and sum to at least .
- Two-point reduction (21): for every policy, .
- Lemma 12: .
- KL computation (p. 37): .
- (22): .
Significance
The theorem shows that the upper bounds of the paper, for iid values and for adversarial values, are optimal up to logarithmic factors. Because the lower bound holds with a constant value and with every HOB revealed, it is not caused by censoring or by the value process. The comes from the statistical difficulty of locating the maximizer of when is learned from samples. The same lower bound therefore applies to every richer feedback model and every value model that contains the constant sequence.
The milestones are general tools. Lemma 12 (a symmetrised Bretagnolle–Huber inequality), the product rule for KL divergences of iid laws, and the reduction (21) are the standard steps of Le Cam's two-point method. They recur in essentially every minimax lower bound for bandits and online learning.
The theorem has a complete proof in the paper (App. B). As far as is known it has not been machine-checked, and the needed pieces are only partly in Mathlib: there is a Kullback–Leibler divergence with a chain rule, but at the pinned revision there is no total variation distance and no Bretagnolle–Huber or Le Cam inequality. The work is to formalize the known proof.
Difficulty
The arithmetic is elementary, and the difficulty lies in the probabilistic reduction. The first idea, bounding the regret round by round under a single law, fails: for any fixed a policy that happens to bid the right point has zero regret, so the lower bound must come from a policy's inability to know which of two laws it faces. Making that precise requires the law of the observed history under each hypothesis, the identity , and monotonicity of total variation under marginalization from to coordinates. A randomized policy adds one more integration, over its internal randomness, before the per-round bound applies. The KL step needs the tensorization of KL over product measures and an elementary but explicit logarithmic inequality.
Formalization scope
All declarations live in the namespace FirstPriceLearning.LowerBound.
- Time. Rounds are
Fin T; Lean roundtis paper time . - HOB law. A measure
μGonℝ. The goal asks for a probability measure giving no mass outside . The CDF is(μG (Set.Iic x)).toReal, with atoms allowed. - Reward and regret. , and the oracle term is the supremum over (a bounded nonempty family, attained). The regret is the pseudo-regret (3), a sum of expected-reward gaps.
- Policies. A policy carries a probability space , with
U : Type, for its internal randomness and bid mapsbid t : U → (Fin t → ℝ) → ℝthat are jointly measurable with values in . The history typeFin t → ℝmakes the policy non-anticipating by construction: it cannot read or later HOBs, and it never sees . - Expectation. An integral over independent of (
Measure.prodwithMeasure.pi). - Quantifiers. The goal is stated as , which implies the inf–sup form. The constant precedes and the policy, and the law is chosen after the policy.
- Divergences. KL is Mathlib's
InformationTheory.klDiv, valued in . Total variation is defined locally as with Radon–Nikodym densities. In Lemma 12 the right-hand side is when a divergence is infinite (). - Hard laws. are explicit two-point measures; the milestones (20)–(22) are stated for as printed.
Two trivializing formalizations are excluded. A policy that observes (or ) before bidding would make the bound false rather than trivial, and the types forbid it. A non-measurable bid map would make the expectation the junk value , and measurability is required. Restricting the policy class to deterministic policies, or reading in Lemma 12, would change the theorem and is not done.
Reusable beyond this mission: the total variation distance, Lemma 12, and the tensorization bound for KL. Contributions are welcome at every milestone, especially Lemma 12 and the data-processing step behind (21).
Selected references
- Y. Han, Z. Zhou, T. Weissman, Optimal No-regret Learning in Repeated First-price Auctions, Operations Research, 2025; preprint arXiv:2003.09795v7, 2024. https://arxiv.org/abs/2003.09795
- A. B. Tsybakov, Introduction to Nonparametric Estimation, Springer, 2009 (Lemma 2.6). https://doi.org/10.1007/b13794
- J. Bretagnolle, C. Huber, Estimation des densités: risque minimax, Z. Wahrscheinlichkeitstheorie verw. Gebiete 47, 1979. https://doi.org/10.1007/BF00535278
- R. Kleinberg, T. Leighton, The Value of Knowing a Demand Curve: Bounds on Regret for Online Posted-Price Auctions, FOCS, 2003. https://doi.org/10.1109/SFCS.2003.1238232