Learning while allocating scarce inventory
An online platform must allocate resources before it knows which customers will arrive later. Advertising budgets, limited product inventories and workers' task opportunities can all be exhausted by early decisions. When the probabilities of a successful allocation are also unknown, a platform must learn from the same interactions that consume its scarce resources. Reserving inventory for future customers and gathering useful observations are therefore coupled decisions.
Cheung, Ma, Simchi-Levi and Wang introduce Inventory Balancing with Online Learning (IBOL) to separate the inventory mechanism from the choice of learning algorithm. Their general guarantee relates the physical revenue of the allocation process to a linear-programming benchmark and a particular learning-regret quantity. This mission formalizes that framework result, Theorem 1, with its exact finite-capacity coefficient, rather than an application-specific oracle analysis. The source is the pinned preprint arXiv:1810.05640v2, Sections 2–4 and Appendix A.2.
Model, algorithm and benchmark
There are n>0 resources. Resource i has positive integer capacity bi and earns reward ri∈(0,1] each time one unit is consumed. Inventory is not replenished. Customers arrive over a finite horizon T>0. Each arrival reveals a context xt from a known finite set. The context sequence is arbitrary but fixed by an oblivious adversary before the run. The algorithm receives the current context and past observations, without future arrivals or the horizon as inputs.
The action space A is general. Given context x and action a, a fixed unknown distribution ρx,a produces a Boolean vector y∈{0,1}n. Several coordinates may be one, and coordinates may be correlated. Write px,a,i=Eρx,a[yi]. The chosen action generates a fresh conditional sample; outcomes for unchosen actions are unobserved. A successful coordinate consumes one unit only if inventory remains. Feedback is still observed for depleted resources. These are the model conventions in Section 2, pages 8–10.
Let Nit−1 count consumed units before round t. IBOL uses the perturbed potential
Ψϵ(u)=e1+ϵ−1e(1+ϵ)u−1
to assign discounted rewards rit=ri(1−Ψϵ(Nit−1/bi)). An arbitrary randomized bandit oracle receives the ordered history of observed triples, xt and (rit)i, and returns at. No maximization property or particular regret bound is required of the oracle. These operations specify Algorithm 1 in Section 3, pages 12–16.
Physical revenue is
ALG=t=1∑Ti∑riyt,i1{Nit−1<bi}.
The auxiliary mean reward is Rt(a)=∑iritpxt,a,i. Its dynamic comparator evaluates the best action at the very same realized inventory. The relaxed regret is the signed quantity
Regϵ=Et=1∑T[a∈AsupRt(a)−(1+ϵ)Rt(at)].
It can be negative. It is neither regret against a fixed action nor the revenue gap to an independently run optimal allocation policy.
The LP benchmark chooses a probability allocation st over actions in each period. It maximizes ∑t,i∫Aripxt,a,idst(a) subject to ∑t∫Apxt,a,idst(a)≤bi for every resource. Each measure has total mass exactly one. Its optimal value is OPT. For discrete actions these integrals recover equations (9)–(12) of the source.
Formalization targets
The goal is Theorem 1 on page 17, equations (13)–(14). For every model, oblivious arrival sequence, randomized oracle and ϵ∈[0,1],
E[ALG]≥fΨ({bi},ϵ)(OPT−Regϵ),
where the exact coefficient is
fΨ({bi},ϵ)=imin(bi+1+ϵ)(1−e−(1+ϵ)/bi)1−e−(1+ϵ).
The formal extended-order statement is OPT≤E[ALG]/fΨ+Regϵ. Since fΨ>0, this is equivalent to the displayed guarantee whenever the LP value is finite.
There are two supporting milestones, both with the full inherited domain ϵ≥0. Claim 1, page 40, asserts that Λi=riΨϵ(NiT/bi) and Γt=supaRt(a) form a feasible solution to the printed dual for every realized path, and that
OPT≤E[t∑Γt+i∑biΛi].
Lemma 4, page 41, asserts for every positive integer b and N∈{0,…,b} that
1/bΨϵ(min{b,N+1}/b)−Ψϵ(N/b)+(1+ϵ)(1−Ψϵ(N/b))≤1{N<b}1−e−(1+ϵ)(b+1+ϵ)(1−e−(1+ϵ)/b).
The two milestones are the numbered dependencies in Appendix A.2. Application-specific UCB and LazyUCB guarantees are outside this mission.
What the guarantee provides
The theorem gives an interface between a learning algorithm and an allocation guarantee: the oracle's relaxed regret enters a precise physical-revenue bound. Its coefficient retains every resource capacity and the exploitation parameter. At unit capacity and ϵ=0, the resource coefficient is 1/2; no large-inventory approximation is needed to interpret the target.
The paper supplies a mathematical proof. This mission supplies compiled statements and concrete definitions for a Lean proof of that result; the three theorem declarations remain open. The reusable outputs include a probability-measure packing LP and a model of randomized policies with correlated vector feedback. Establishing the stated theorems in Lean remains the solver's task.
Why the formal proof needs care
Discounted rewards change with realized inventory, which depends on earlier actions and outcomes. Treating the auxiliary reward sequence as independent would therefore lose the process that the theorem concerns. Likewise, assuming that the oracle selects an actual maximizer would remove the learning contribution from the statement.
Several boundary cases affect the mathematics. Resources at capacity still produce feedback, although their physical and discounted rewards vanish. Relaxed regret need not be nonnegative. The discrete inventory grid includes its depleted endpoint. A general action space need not admit a best action, and the equality-constrained primal LP need not be feasible when there is no zero-consumption action. The definitions retain these cases without inserting extra optimization or feasibility assumptions.
Formalization scope
Resources and finite contexts use finite index types; capacities are positive naturals. Histories carry the usual measurable structure obtained from the finite product space at each possible length. The run is built by successive probability-kernel sampling of an action and then its outcome. The oracle receives only observed history, current context and discounted rewards.
Action integrals extend the printed sums to general measurable spaces. Every discrete source allocation is included. Best-action suprema agree with the printed maxima whenever attained and additionally cover unattained values. The LP value uses extended reals, with −∞ for an empty feasible set. No indeterminate extended-real subtraction appears in the theorem. Claim 1 also asserts pathwise feasibility for arbitrary length-T observation strings, including probability-zero strings generated by the same physical update rule.
All objectives, coefficients, policies and certificates are constructed from data. Neither a performance bound nor dual feasibility is packaged as a model assumption. Mathlib supplies kernels, measures, integrals, finite sums, exponentials and order operations. The separate definition modules distinguish the general LP from its inventory-specific instance. The explicit history definition on page 15 determines the indexing used in Algorithm 1; the shifted update indices in its printed line 9 are recorded as a source finding.
Selected references
- Wang Chi Cheung, Will Ma, David Simchi-Levi and Xinshang Wang, Inventory Balancing with Online Learning, arXiv preprint, version 2, 30 August 2021. arXiv:1810.05640v2. All theorem indices and page references in this mission use this artifact.