An Optimal Policy for Dynamic Assortment Planning Under Uncapacitated Multinomial Logit Models: The Adaptive-Confidence Trisection Policy Has N-Independent Regret at Most C√TResearch Paper
Learning which products to offer
A retailer that sells substitutable products must decide, for each arriving customer, which subset of products to display. Customer preferences are unknown at the start and have to be learned from purchases, while every suboptimal display costs revenue. This is dynamic assortment planning, a standard problem of revenue management in e-commerce and online advertising, and its learning version is a bandit problem whose arms are the subsets of products.
Under the multinomial logit (MNL) choice model the problem has been studied through the regret of a policy over customers (Chen, Wang and Zhou, §2 and Table 1):
- Rusmevichientong, Shen and Shmoys (2010) analysed the same revenue potential as this paper with a golden-ratio search; their bound involves and needs a separability (gap) assumption between candidate assortments.
- Sauré and Zeevi (2013) also cast MNL assortment learning as regret minimization.
- Agrawal, Avadhanula, Goyal and Zeevi gave UCB and Thompson-sampling policies with regret for assortments of capacity .
- Chen and Wang (2018) proved an lower bound in the capacitated model when .
In the capacitated model the regret must grow with . Chen, Wang and Zhou (arXiv:1805.04785v2, Mathematics of Operations Research 2021) show that without a capacity constraint it need not: a trisection policy achieves regret with a universal constant, and is a lower bound. This mission formalizes the upper bound.
Setting
There are items with known revenues and unknown utilities ; the no-purchase option has utility . In each period the retailer offers an assortment , and the customer buys with probability or buys nothing with probability , independently of the past. The expected revenue of is
and the regret of a policy is , where maximizes over all subsets.
The policy rests on two objects. The level set collects the items whose revenue is at least . The revenue potential is a piecewise-constant function of one real variable. Its maximum is , its right limit is , and denotes the fixed point .
The adaptive-confidence trisection policy keeps an interval , starting from . In epoch it forms the points and , at distance , and runs inner iterations. Each inner iteration does two things:
- while lies inside the current confidence interval for , it explores: it offers , adds the revenue to the running sum , and sets ;
- it then exploits by offering .
At the end of the epoch the policy sets if , and otherwise. The policy knows and but never .
Formalization targets
Goal: Theorem 3
There is a universal constant such that for every , every horizon , all and all , the adaptive-confidence trisection policy satisfies
No gap condition and no bound on is assumed. The constant is left unspecified, as in the paper.
Milestones
- Structure of . Lemma 1 (revenue-ordered optimality), Corollary 1 (), Lemma 2 (the fixed point, under ), the jump identity (S4)/(S8), and Lemmas 3–4 ( is below the diagonal to the right of and above it to the left).
- Concentration. Hoeffding's inequality (Lemma 15, already proved on the platform), the uniform inequality with growing confidence levels (Lemma 8, (11)), and the two tail bounds (S32) and (S33) used in the proof of Lemma 10.
- Recursion. With the expected regret of epochs conditioned on the relative position of , and ((14)): Lemma 11 (Case 3, ), Lemma 9 (Case 1) and Lemma 10 (Case 2).
Significance
Theorem 3, together with the lower bound of Theorem 4, settles the minimax regret of uncapacitated MNL assortment learning at , independent of the number of products. The policy uses no estimate of the individual utilities, and its cost per period is one level-set offer, so it applies to catalogues where is large compared with . The structural lemmas on the revenue potential (Lemmas 1–4) are standard tools for MNL assortment problems and are reusable by other missions on static and dynamic assortment optimization. Lemma 8 is a MOSS-style concentration inequality whose confidence level grows with the sample size; it is the device that removes the factor of the paper's baseline bound (Theorem 2).
The result is proved in the paper and is not presented as open; no machine-checked proof of any part of it is known. The platform has the MNL model, the UCB policy of Agrawal et al. (whose bound depends on ) and Hoeffding's inequality, but no level-set or trisection analysis. A formal proof would also check the paper's argument, whose printed slips are recorded in the scope section.
Difficulty
The regret concerns a stateful algorithm whose epochs end at random times: an inner iteration takes two periods while exploration is active and one afterwards, and the policy's internal state must be recomputed from the purchase history alone. The obvious argument fixes a confidence level , bounds each epoch's regret on the event that all confidence intervals hold, and sums a geometric series. That argument loses a factor, and it is the paper's own Theorem 2. Removing the factor needs confidence levels that weaken as data accumulate. A failure is then no longer rare enough to be ignored, so the analysis has to track its cost. This is done by a recursion on conditional expected regrets across three cases (interval to the right of, containing, or to the left of ), in which a wrong decision in one epoch passes to another case with controlled probability. Within an epoch, the purchases made while exploring a level set must be shown to behave like i.i.d. samples with mean , although the number of explorations is itself random.
Formalization scope
- Model. The model is the published
MNLBandit.UCB.Settingwith every subset admissible: histories are sequences of choices inOption (Fin N)(none= no purchase), policies are deterministic functions of the past choices, and expectations are finite sums over histories. Items and periods are 0-based in Lean; epochs and inner iterations are as on the page. - Policy. The policy is defined exactly. Confidence radius (δ = 1/T in (12)); ; trisection weights ; the update if and only if ; exploitation of in every inner iteration; exploration only while .
- Conventions.
- The matters only for tiny , where the printed is .
- The empty assortment is allowed (§3 asks ; ).
- §3's is relaxed to Theorem 3's .
- is defined on all of , and is Mathlib's
Function.rightLim. - is defined as .
- is a ratio of finite sums and is on a conditioning event of probability .
- Every and is an existential constant chosen before all other quantities.
- Added hypotheses. Lemma 2 assumes , which its proof uses. Lemmas 9 and 10 and (S33) assume and that epoch can start (); without these, can be negative.
- Repairs. Two printed statements are repaired.
- (S32) is stated with , which its printed justification gives; the printed fails on an explicit instance.
- Lemma 9 sums from , as its proof does, with exponent and the deterministic last epoch as upper limit.
- Ruled out. Three formalizations would make the goal trivial, and none is admissible: a policy that receives (it could offer in every period); a constant that depends on , or the instance (the regret is at most ); and an abstract "trisection oracle" with assumed guarantees in place of the algorithm above.
- Not included. Theorem 4 (the lower bound) and its Lemmas 12–14 are not part of the mission, and neither are Theorem 2 and its Lemmas 5–7 (the baseline policy) or Proposition 1.
- Contributions welcome. Lemmas 1–4 need only finite sums; Lemma 8 and (S32)/(S33) need Hoeffding's inequality and a union bound over sample sizes; the reduction of exploration purchases to i.i.d. samples is reusable for other level-set policies.
Selected references
- X. Chen, Y. Wang, Y. Zhou, An Optimal Policy for Dynamic Assortment Planning Under Uncapacitated Multinomial Logit Models, arXiv:1805.04785v2, 2019; Mathematics of Operations Research 46(4), 2021. https://arxiv.org/abs/1805.04785
- S. Agrawal, V. Avadhanula, V. Goyal, A. Zeevi, MNL-Bandit: A Dynamic Learning Approach to Assortment Selection, Operations Research 67(5), 2019. https://arxiv.org/abs/1706.03880
- X. Chen, Y. Wang, A note on tight lower bound for MNL-bandit assortment selection models, Operations Research Letters 46(5), 534–537, 2018.
- P. Rusmevichientong, Z.-J. M. Shen, D. B. Shmoys, Dynamic Assortment Optimization with a Multinomial Logit Choice Model and Capacity Constraint, Operations Research 58(6), 2010. https://doi.org/10.1287/opre.1100.0866
- D. Sauré, A. Zeevi, Optimal Dynamic Assortment Planning with Demand Learning, Manufacturing & Service Operations Management 15(3), 2013. https://doi.org/10.1287/msom.2013.0429
- K. Talluri, G. van Ryzin, Revenue Management Under a General Discrete Choice Model of Consumer Behavior, Management Science 50(1), 2004. https://doi.org/10.1287/mnsc.1030.0147
- J.-Y. Audibert, S. Bubeck, Minimax Policies for Adversarial and Stochastic Bandits, COLT 2009.