Optimal Search and Discovery: Always Taking the Action with the Largest Purchase, Search or Discovery Value Is an Optimal PolicyResearch Paper
Consumer search when the set of options is not known in advance
In the standard model of costly sequential search, a consumer knows from the start which products exist and pays to learn how much each is worth. Weitzman (Econometrica 1979) solved this problem with a single number per product, the reservation value: inspect in decreasing order of reservation values, and stop as soon as the best product found is worth more than every remaining reservation value. This rule underlies most structural models of online search in marketing and industrial organization.
In many markets, though, consumers do not know every option at the start. On a retail platform they scroll through results a page at a time, and each page shows only partial information (price, picture, rating) about the products it reveals. Learning the full value of a product requires clicking on it. The consumer therefore chooses among three kinds of action: buy, inspect a product already seen, or discover more products. R. P. Greminger, Optimal Search and Discovery (Management Science, 2022), shows that this richer problem is still solved by comparing reservation values, now three kinds of them.
Timeline. Weitzman (1979) treats inspection with a known set of products. Keller and Oldale (JET 2003) prove that a Gittins index policy is optimal for branching bandits, in which taking an action creates new actions. Choi, Dai and Kim (Econometrica 2018) give the eventual purchase theorem for Weitzman's problem. Greminger (arXiv 2019, journal 2022) adds the discovery action and characterizes the optimal policy.
The search and discovery problem
A risk-neutral consumer gets utility from buying product . The partial valuation is visible as soon as the product is discovered. The rest of the value, , is revealed only by inspection. The pairs are independent draws of , where and .
In period the consumer has a consideration set of inspected products (valuation known), which also contains an outside option worth , and an awareness set of discovered products for which only is known. The available actions are:
- buy any product of , which ends search;
- inspect a product of at cost , which moves it to ;
- discover more products at cost , which adds them to .
With probability , the consumer believes, a discovery is the last one. Recall is free, payoffs are not discounted, and the objective is the expected purchase value minus the total costs paid.
The three reservation values are defined against a hypothetical outside option . The search value is the root of
The purchase value is . The discovery value is the root of
where is the optimal value of the reduced problem whose only products are the outside option and the newly discovered ones.
Formalization targets
Goal: Theorem 1
Let and . Every policy that obeys
is feasible and attains the largest expected payoff among all feasible policies, from every initial state.
Milestones
- Equation (6) has a unique solution, and is a tail integral of (Appendix B).
- The search value is the Gittins index of inspection: an outside option is optimal against one product if and only if .
- , with (Appendix B).
- Proposition 7: equation (10) has a unique solution.
- The discovery value is the Gittins index of discovery: against an outside option , discovering helps if and only if .
Significance
The theorem turns a dynamic program over an infinite state space into comparisons of three reservation values. The discovery value still requires optimizing inspection among the products in one newly discovered batch. The paper builds on the result in several directions. Its second main result, an eventual purchase theorem under limited awareness (Theorem 2), is derived from Theorem 1. So are its closed-form expected payoff, its comparisons with random and directed search, and the claim that the discovery value, and hence the optimal policy, does not depend on the belief .
Formalizing it gives the first machine-checked statement of a Weitzman-type optimality theorem with an outside option, partial information and a branching action. As far as a search of the Prove2Me catalog shows, Weitzman's rule itself is not formalized for undiscounted search with an outside option, so the milestones on the search value and on the closed form of are new formalizations in their own right. The paper's proof relies on two external results, Keller and Oldale's branching bandit theorem and Choi, Dai and Kim's Corollary 1. A complete formal development either proves these or replaces them with a direct argument.
Difficulty
The obvious argument is to treat the problem as a multi-armed bandit and apply the Gittins index theorem. It does not apply as stated. Discovery creates new arms, the problem is undiscounted and ends at a purchase, and the number of future discoveries is random. The branching bandit theory that covers creating arms is stated for discounted rewards and in a derivative form, , which needs almost-everywhere differentiability of value functions in the outside option. The step specific to this paper is to show that the index of discovery does not depend on whether later discoveries are possible. That needs the index to be non-increasing along search, which the paper derives from stationarity. Finally, the rule policy must be shown to stop almost surely with an integrable payoff even when infinitely many products can be discovered ().
Formalization scope
All objects live in the namespace SearchDiscovery.Reservation. A model SDModel carries , the kernel κ (), cs, cd, nd and . Its standing assumptions are , , finite second moments of and of each κ x, and a finite unconditional mean of (footnote 36). An information state lists the purchase values of (the outside option is one entry), the partial valuations of , and whether discovery is available. A run is driven by an explicit probability space: the hidden of each initial awareness product, and for each discovery i.i.d. pairs together with an independent Bernoulli() flag saying whether the discovery was the last.
The formalization commits to the following readings:
- Policies are deterministic functions of the information state.
- A feasible policy takes available actions and buys after finitely many periods almost surely, and has an integrable payoff.
- when is empty.
- enters the buy and inspect rules only while discovery is available.
- and are hypotheses satisfying (6) and (10), not definitions.
- Theorem 1 is read as sufficiency: every rule-obeying policy, with any tie-break, is optimal. The policy's payoff is assumed measurable.
Several shortcuts would make the statements trivial, and the formalization rules them out:
- the rule predicate is satisfiable at every state, and
FollowsRulesis not vacuous; - feasibility of the rule policy is a conclusion, not a hypothesis;
- maxima over empty sets are never given a junk value of 0;
- is defined from the optimal value of the reduced problem, not from its closed form.
A complete development needs:
- path-wise runs of history-dependent searches over infinite product measures;
- an undiscounted optimal stopping and index theory with an outside option;
- the layer-cake identities for and .
The Weitzman/Pandora layer (search values, the closed form of the reduced problem) is reusable well beyond this paper. Contributions to any milestone are welcome, as are alternative proofs that avoid the derivative form of the branching bandit theorem.
Selected references
- R. P. Greminger, Optimal Search and Discovery, Management Science, 2022, https://doi.org/10.1287/mnsc.2021.4085; preprint arXiv:1911.07773v5 (cited version), https://arxiv.org/abs/1911.07773
- M. L. Weitzman, Optimal Search for the Best Alternative, Econometrica 47(3), 641–654, 1979. https://doi.org/10.2307/1910412
- G. Keller and A. Oldale, Branching Bandits: A Sequential Search Process with Correlated Pay-offs, Journal of Economic Theory 113(2), 302–315, 2003. https://doi.org/10.1016/S0022-0531(03)00004-0
- M. Choi, A. Y. Dai and K. Kim, Consumer Search and Price Competition, Econometrica 86(4), 1257–1281, 2018. https://doi.org/10.3982/ECTA13803
- K. Adam, Learning While Searching for the Best Alternative, Journal of Economic Theory 101(1), 252–280, 2001. https://doi.org/10.1006/jeth.2000.2724