Motivation
Display advertising is sold one impression at a time, mostly through second-price auctions with reserve prices. A reserve tailored to each buyer and to the features of the impression can raise the seller's revenue considerably, but the right reserve depends on the buyers' valuations, which the seller does not know and must learn from past auctions. The buyers participate in the same exchange day after day and know that their bids feed the seller's learning, so a patient buyer may bid untruthfully today to obtain lower reserves tomorrow. Learning reserve prices from strategic buyers is therefore a joint problem of statistics and incentives.
Golrezaei, Javanmard and Mirrokni (arXiv:2002.11137, Operations Research 69(1), 2021) study this problem in a contextual model and propose the policy CORP (Contextual Robust Pricing). Earlier work on learning against strategic buyers considered a single buyer facing posted prices, without contexts: Amin, Rostamizadeh and Syed (NeurIPS 2013) introduced the discounted strategic buyer, and contextual and revenue-optimization variants followed (Amin, Rostamizadeh and Syed, NeurIPS 2014; Mohri and Muñoz Medina, NeurIPS 2015). Personalized reserves in second-price auctions were studied by Paes Leme, Pál and Vassilvitskii (2016). The present paper treats many buyers, personalized reserves, contexts in Rd and censored observations (the seller sees who won, not the valuations).
Setting
In every period t=1,2,… the seller observes a feature vector xt∈Rd, drawn i.i.d. from a law D supported in the unit ball. Each of N buyers has the valuation
vit=⟨xt,βi⟩+zit,
where βi∈Rd is an unknown preference vector with ∥βi∥≤Bp and the noises zit are i.i.d. with a known density f on [−Bn,Bn], mean zero, and distribution function F. Assumption 1 asks F and 1−F to be log-concave on [−Bn,Bn].
Each period runs a lazy second-price auction with personalized reserves rit: the highest bidder i⋆ gets the item if bi⋆t≥ri⋆t and pays max{ri⋆t,maxj=i⋆bjt}; nobody else pays. The benchmark knows the βi and sets ri⋆(x)=argmaxyy(1−F(y−⟨x,βi⟩)) (4); its expected revenue against truthful buyers is revt⋆ (5).
CORP works in episodes Ek={ℓk,…,ℓk+1−1} with ℓk=2k−1. At the start of episode k it computes, for each buyer, a maximum-likelihood estimate β^ik over ∥β∥≤Bp from the censored outcomes of episode k−1 (8)–(9). In each period of episode k, with probability 1/ℓk it explores: one uniformly chosen buyer is offered the item at a price uniform on [0,Bp+Bn]. Otherwise it exploits with reserves rit=argmaxyy(1−F(y−⟨xt,β^ik⟩)) (10).
Buyer i's bid may depend on his own history of valuations, bids, allocations and payments, on xt, on vit and on private randomness. His utility is ∑tγtE[vitqit−pit] with a discount factor γ∈(0,1), and buyers are utility-maximizing. The regret after T periods is ∑t=1T(revt⋆−revt), worst case over β and D (7).
Formalization targets
Goal: Theorem 4.1 (p. 23)
There is C, depending only on Bp,Bn,M,f,cmin,cmax, such that for all N,d≥1, T≥2, γ∈(0,1), all admissible D,β and all utility-maximizing buyers,
Regret(T)≤CNd(log(Td)logT+log2(1/γ)log2T).
Milestones, in the order the proof uses them
- Proposition 3.1 (p. 15): the reserves (4) are revenue-optimal against truthful buyers.
- Lemma 1 (p. 56): the conditional revenue Wit is flat at rit⋆ and has a bounded second derivative.
- Lemma 2 (p. 56): ∣rit⋆−rit∣≤∣⟨xt,βi−β^ik⟩∣.
- Lemma 3 (p. 57): the second-highest bid and the competing bid move by at most the shading and overbidding.
- Proposition 9.2 (p. 53): per episode, a utility-maximizing buyer lies O(log(ℓ/δ)+log(ℓN)/log(1/γ)) times, and his total shading and overbidding are O(log(ℓN)/log(1/γ)), with high probability.
- Corollary 18.2 (p. 88): the vector Freedman inequality.
Significance
If true, Theorem 4.1 says that a seller can learn personalized, contextual reserves from censored data at a cost logarithmic in T and linear in Nd, even when buyers manipulate their bids. The strategic term vanishes as buyers become impatient. The episodic delay between a bid and its effect on reserves, together with the random exploration prices, is what makes untruthful bidding unprofitable. The lemmas are of independent use. Proposition 3.1 and Lemma 2 concern the monopoly price under log-concave noise. Lemma 3 is a deterministic fact about second-price auctions. Corollary 18.2 is a vector concentration inequality.
Status. None of these results is machine-checked anywhere known to us. The paper's printed proof has gaps. Proposition 9.1 (p. 52), which bounds the estimation error of the MLE by the number of lies, fails as printed. With uniform noise a single lie per episode keeps the estimate a constant distance from βi. Its supporting Lemma 5 uses a constant uF that is infinite for every compactly supported F (p. 71). Its curvature bound (89) also fails when a likelihood term leaves the support of f. Lemma 6 (p. 76) conditions on an event the exploration lottery itself moves. A proof of Theorem 4.1, or a counterexample to it, would settle the question. The goal is stated as the paper states it, and is neither asserted nor denied here.
Difficulty
The natural route is the paper's. First bound the estimation error of the censored MLE in terms of the number of lies. Then bound the number of lies by comparing a lie's discounted future gain with its immediate loss under the exploration lottery. Finally sum over episodes.
The first step is where it breaks. With noise supported on [−Bn,Bn], an observation whose threshold falls outside the support carries no curvature. Its likelihood term is constant or +∞, so the minimizer of (9) is in general a set, not a point. A minimizer at the wrong end of that set can keep the reserve below the benchmark for many episodes, even with truthful buyers. Any proof must say which minimizer CORP takes, or show that all of them work. The second step must handle buyers who react to one another through their histories; the paper's proof holds the other buyers' bids fixed. The concentration steps need a vector martingale inequality and a covariance concentration bound, neither of which is in Mathlib.
Formalization scope
Lean represents features as EuclideanSpace ℝ (Fin d) and buyers as Fin N. Periods are indexed from 0 internally (Lean's n is the paper's t=n+1), and episodes likewise (Lean's e is the paper's k=e+1, so ℓk=2e). Period 1 always explores. The per-period randomness consists of the features, the noises, the exploration coin, the chosen buyer, the exploration price and the buyers' seeds. It is independent across periods, under Measure.infinitePi, and the play is defined by recursion over periods. The likelihood (9) takes values in [0,+∞] with −log0=+∞; a real-valued Real.log would make impossible observations free and change the estimator. The estimator (8) and the reserve rule (10) are arbitrary measurable selections of the argmin and argmax, quantified universally. No tie-break or uniqueness is added. The same reserve rule defines the benchmark (4).
Standing assumptions and added hypotheses.
- The noise density is continuous on [−Bn,Bn], with mass one and mean zero, and Assumption 1 holds.
- ∥x∥≤1 almost surely, and cmin/d∥u∥2<E⟨x,u⟩2≤cmax/d∥u∥2; the proof uses these eigenvalue bounds (pp. 57, 72–73).
- Bids are at most M (p. 14) and pairwise distinct almost surely (p. 20).
- Strategies and selections are measurable, N,d≥1 and T≥2.
- Utility-maximizing is pinned as the inequality the proof uses on p. 77: no buyer gains by switching to truthful bidding from the start of any episode on. Every best-response profile satisfies it, and the all-truthful profile does too.
- Exploration is a posted price to the chosen buyer.
- Lemma 1 adds f∈C1(R), which its proof uses.
- Lemma 2 adds that both indices exceed −Bn; without that, the argmax is unbounded.
- Lemma 3's (45) adds that another buyer wins; as printed it fails when nobody wins.
- Corollary 18.2 adds u0=0.
Ruled out. The constant C comes before every instance, horizon, selection and strategy, so a bound depending on T or on the instance does not count. The policy is CORP as defined, not any reserve-setting rule. The rationality hypothesis is satisfiable.
Infrastructure. A complete development needs the following.
- Measurability of the recursive play.
- The monopoly price under log-concavity (Proposition 3.1, Lemma 2).
- Censored-likelihood estimation with compactly supported noise.
- A vector Freedman inequality (Corollary 18.2, reusable).
- Covariance concentration for bounded vectors.
Contributions on any milestone are welcome. So is a formal counterexample to the goal in the model as stated, for example with one truthful buyer, uniform noise and the smallest minimizer of (9).
Out of scope. CORP-II and SCORP (Theorems 5.1–5.3, Propositions 11.1–11.2) and the extensions of Appendix 8 are not part of this mission. Proposition 9.1, Lemma 5, Lemma 6 and Theorem 18.1 are not posed (see Significance).
Selected references
- N. Golrezaei, A. Javanmard, V. Mirrokni, Dynamic Incentive-Aware Learning: Robust Pricing in Contextual Auctions, Operations Research 69(1), 2021; read as arXiv:2002.11137v1. https://arxiv.org/abs/2002.11137
- K. Amin, A. Rostamizadeh, U. Syed, Learning Prices for Repeated Auctions with Strategic Buyers, NeurIPS 2013. https://papers.nips.cc/paper/2013/hash/1e6e0a04d20f50967c64dac2d639a577-Abstract.html
- R. Paes Leme, M. Pál, S. Vassilvitskii, A Field Guide to Personalized Reserve Prices, WWW 2016. https://doi.org/10.1145/2872427.2883071
- J. A. Tropp, Freedman's inequality for matrix martingales, Electronic Communications in Probability 16, 2011. https://doi.org/10.1214/ECP.v16-1624
- R. B. Myerson, Optimal Auction Design, Mathematics of Operations Research 6(1), 1981. https://doi.org/10.1287/moor.6.1.58