Continuous Assortment Optimization with Logit Choice Probabilities under Incomplete Information: The Stochastic Approximation Policy SAP(α, β) Has Worst-Case Regret O(log T)Research Paper
Motivation
A retailer choosing which products to display, a platform choosing which items to recommend, and a broadcaster choosing which content to stream all face an assortment problem: offer a set of products, observe what customers buy, and earn the revenue of the purchased item. The standard demand model is the multinomial logit (MNL) model, under which the optimal assortment without capacity constraints consists of the most profitable products (Talluri and Van Ryzin, 2004). When the customers' preferences are unknown, the seller must learn them from sales, and the quality of a learning policy is measured by its regret: the cumulative revenue lost relative to an omniscient seller.
When the product catalogue is very large, or products are described by a continuous attribute such as quality, the discrete model becomes a continuum. Peeters, den Boer and Mandjes (arXiv:1807.06497, published in Operations Research, 2022) introduced a continuous analogue of the MNL model and studied learning in it. Their first main result, the goal of this mission, is that a simple stochastic approximation policy has worst-case regret of order over customers.
- 1951: Robbins and Monro introduce stochastic approximation for finding the root of a function observed with noise (doi:10.1214/aoms/1177729586).
- 2010: Rusmevichientong, Shen and Shmoys give adaptive policies for capacitated MNL assortment (doi:10.1287/opre.1100.0866).
- 2011: Broadie, Cicek and Zeevi give mean-square rates for Kiefer–Wolfowitz type stochastic approximation (doi:10.1287/opre.1110.0970).
- 2019: Agrawal, Avadhanula, Goyal and Zeevi give an -regret UCB policy for discrete MNL bandits (doi:10.1287/opre.2018.1832).
- 2021/2022: Peeters, den Boer and Mandjes prove worst-case regret for the continuum model, with a matching lower bound.
Setting
The products are the points , and the revenue function is continuously differentiable with . An assortment is a Borel set ; is the set of all of them. A customer offered buys a product in a Borel set , or buys nothing (), with probabilities
where the preference function is unknown to the seller. The expected revenue of is , and .
The class (Assumption 1) contains the integrable with on that are differentiable on with , where and .
The generalized inverse is , and . The policy SAP starts at , offers in period , observes the revenue , and updates
Given the history, the purchase has the law of . The regret over periods is .
Formalization targets
Goal: Theorem 1 (p.12)
For and there is such that for all
The constant depends on , the class and the policy, but not on or .
Milestones, in proof order
- Well-posedness: an SAP purchase process exists for every horizon.
- §4.1: the optimum over is attained by an interval .
- Lemma 4: ; the drift inequality (23) ; the derivative and the curvature bound (27); the quadratic gap (24) with uniform over .
- The iterates stay in , and .
- The recursion (25) for , and Lemma 5, .
- The regret identity and .
Significance
The theorem shows that in the continuum logit model the seller does not need to estimate the preference function: a one-dimensional Robbins–Monro recursion on the observed revenue finds the optimal revenue level, and the loss is logarithmic in the horizon. This is faster than the rates typical of continuum-armed bandits, and the paper's Theorem 2 shows it is optimal. The bound is worst-case over a class of preference functions, which matters when nothing is known about the customers beyond the class.
The result is proved on paper. No part of it is formalized on the platform. The mission adds the platform's first continuum-indexed choice model, a measure-theoretic model of a sequential learning policy whose decisions depend on the full purchase history, and a complete regret analysis for a stochastic approximation policy. The discrete MNL bandit of Agrawal et al. is on the platform as MNLBandit.UCB; it is a related, not a reused, model.
Difficulty
The policy itself is three lines; the difficulty is in the analysis of the map . The obvious route — " is a contraction, so the iterates converge" — fails: is not a contraction, and is flat below and drops to zero above . The proof instead needs the one-sided drift inequality (23), whose constant must use the upper bound , and the quadratic gap (24), which requires a second-order Taylor expansion of with a curvature bound (27) that is uniform over the class. The bound (27) needs differentiable, and the uniformity of and over is what turns per-instance bounds into a worst-case one.
The induction in the proof of Lemma 5 multiplies by , which is negative for small when is small; the printed does not follow from the printed step there, although the lemma's statement (some ) remains true.
Formalization scope
Products are reals and purchases live in ℝ ⊕ Unit (Sum.inr () is no-purchase). carry hypotheses only on and . is the measurable subsets of (the uncapacitated case ), and is a supremum (sSup), shown to be a maximum in a milestone. The paper's is empty for , which SAP can reach since is arbitrary; there , so the offered assortment is a null set. Lean's sInf ∅ = 0 would instead offer and break (23). The class has a fixed derivative bound , since Assumption 1(ii)'s supremum over all admissible is otherwise infinite.
Time is 0-based (). A horizon- process is a probability space with measurable purchases whose conditional law given the history is the choice law of the offered assortment, written as an identity of measures on history events. A Lean proof that the SAP iterate is a measurable function of the history exists, so the expectations in the regret are genuine integrals. The goal quantifies over all such processes, and a milestone asserts that one exists, so the goal is not vacuous.
A complete development needs: second-order calculus for parameter integrals ; the construction of a process from a history-dependent kernel (Ionescu–Tulcea, ProbabilityTheory.Kernel.traj, or an iterated Measure.bind); and conditional expectations given a history σ-algebra. The mean-square recursion and the harmonic-sum bound are reusable for other Robbins–Monro analyses. Contributions to any milestone are welcome, in any order; milestones 2–3 and 6 are independent of the process construction.
Selected references
- Y. Peeters, A. V. den Boer, M. Mandjes, Continuous Assortment Optimization with Logit Choice Probabilities under Incomplete Information, arXiv:1807.06497v4, 2021; Operations Research, 2022. https://arxiv.org/abs/1807.06497
- H. Robbins, S. Monro, A Stochastic Approximation Method, Annals of Mathematical Statistics 22(3), 1951. https://doi.org/10.1214/aoms/1177729586
- K. Talluri, G. van Ryzin, Revenue Management Under a General Discrete Choice Model of Consumer Behavior, Management Science 50(1), 2004. https://doi.org/10.1287/mnsc.1030.0147
- P. Rusmevichientong, Z.-J. M. Shen, D. B. Shmoys, Dynamic Assortment Optimization with a Multinomial Logit Choice Model and Capacity Constraint, Operations Research 58(6), 2010. https://doi.org/10.1287/opre.1100.0866
- M. Broadie, D. Cicek, A. Zeevi, General Bounds and Finite-Time Improvement for the Kiefer-Wolfowitz Stochastic Approximation Algorithm, Operations Research 59(5), 2011. https://doi.org/10.1287/opre.1110.0970
- S. Agrawal, V. Avadhanula, V. Goyal, A. Zeevi, MNL-Bandit: A Dynamic Learning Approach to Assortment Selection, Operations Research 67(5), 2019. https://doi.org/10.1287/opre.2018.1832