Motivation
Periodic-review inventory systems with lost sales and a positive replenishment lead time are a basic model of retail and spare-parts operations: demand that cannot be met from stock is lost, and an order placed today arrives τ periods later. Even with a known demand distribution the optimal policy depends on the whole vector of outstanding orders and is intractable for long lead times, so practice and theory fall back on base-stock (order-up-to) policies, which are asymptotically optimal as the lost-sales penalty grows relative to the holding cost (Huh, Janakiraman, Muckstadt and Rusmevichientong, Management Science 2009).
Computing the best base-stock level still requires the demand distribution. In many settings the distribution is unknown and, worse, demand is censored: the manager records sales, min{Dt,It}, and never sees the demand lost to a stockout. The paper behind this mission, Huh, Janakiraman, Muckstadt and Rusmevichientong, An Adaptive Algorithm for Finding the Optimal Base-Stock Policy in Lost Sales Inventory Systems with Censored Demand (working paper of February 8, 2007; published in Mathematics of Operations Research 2009, DOI 10.1287/moor.1080.0367), gives an algorithm that learns the optimal base-stock level from sales data alone and bounds its regret against the best base-stock policy. It extended the zero-lead-time adaptive newsvendor analyses (e.g. Huh and Rusmevichientong, Math. Oper. Res. 2009) to positive lead times, where the inventory process is a Markov chain on Rτ.
Setting
Demands D1,D2,… are i.i.d. copies of a nonnegative, continuous random variable D with E[D]>0 and distribution function F. The inventory vector in period t is
Xt=(Qt−1,…,Qt−τ+1,It)∈R+τ,
the τ−1 outstanding orders and the on-hand inventory It; its inventory position is Xt⋅1τ. Under the order-up-to-S policy the manager orders Qt=[S−Xt⋅1τ]+, demand Dt is realized, and
It+1=[It−Dt]++Qt−τ+1.
A period with on-hand inventory y costs, in expectation, C(y)=hE[y−D]++bE[D−y]+, with holding cost h and lost-sales penalty b. The long-run average cost of level S is
C(I∞(S))=T→∞limT1t=1∑TE[C(It(S))],
and S∗ denotes a level minimizing it over S≥0. Write γ(S)=P[D≤S/(τ+1)].
The algorithm \textscAdaptive(α,β) runs in cycles k=1,2,… of length Tk=⌈kβ⌉; N(L)=∑k≤LTk. Cycle k uses a fixed level Sk. From sales data it computes a sample-path derivative I′ by a recursion over the last τ periods, forms a gradient estimate Hk∈{h,−b,0} from the last period of the cycle, and sets Sk+1=P[M,M](Sk−ϵkHk) with ϵk=(M−M)/(max{b,h}kα). The inventory is carried over between cycles. The regret after L cycles is
Λ(L)=E[k=1∑Lj=1∑TkC(I(k,j))]−C(I∞(S∗))N(L).
Formalization targets
Goal: Theorem 10, corrected
Under Assumption 1 (0≤M≤S∗≤M and γ(M)>0) and infinite support of D, for α,β∈(0,1) and L≥1, with N=N(L) and ν=max{1−γ(M)2τ,F(M),1/e},
NΛ(L)≤(b+h)[(M−M)(N1+β1−α4+N1+βα4/(1−α)+N1+β1C3)+MN1+ββ24τ/ln(1/ν)],
with C3=12(4τ)1/βΓ(1/β)(1/β)/(ln(1/ν))1/β. At α=β=1/2 this is O(N−1/3).
Milestones
Lemma 5 (tail of D1+⋯+Dt), Lemma 6 (the inventory position falls to S), Theorem 7 (existence, independence of the start, convexity and differentiability of C(I∞(S)), and existence of S∗ with γ(S∗)>0), Theorem 11 (projected stochastic gradient descent with biased gradients), Lemmas 12 and 13 (sums over cycles of length ⌈kβ⌉), Lemma 14 (the regret Λ1 from Sk=S∗) and Lemma 15 (the transient regret Λ2).
Significance
The theorem gives a nonparametric, sales-only policy whose average cost approaches that of the best base-stock policy at an explicit polynomial rate, uniformly over demand distributions satisfying Assumption 1. Its ingredients are reusable: an explicit geometric ergodicity rate for the lost-sales base-stock chain, convexity of the long-run base-stock cost in S, and a regret bound for projected gradient descent whose gradients are biased by Markov dependence.
The result is published; to our knowledge none of it has been machine-checked. The mission formalizes the regret bound, which requires Markov-chain ergodicity on a continuous state space, long-run averages, and stochastic approximation with biased gradients. It also repairs the printed constants: Theorem 10 and Lemma 15 are false as printed (see Formalization scope), and the formal statements make the dependence on M explicit.
Difficulty
The obvious route treats the cycle's last-period derivative as an unbiased gradient of C(I∞(Sk)) and applies standard online convex optimization. That fails: the estimate is computed from a Markov chain started at whatever state the previous cycle left, so it is biased, and the bias decays only as the chain mixes. Controlling the bias requires an explicit mixing rate for a chain on R+τ×{0,1}τ whose state includes the derivative process, uniformly over the levels in [M,M] and over starting positions above S. The ergodicity argument uses a coalescing demand pattern, which exists only when γ(S)>0; without that condition the chain need not be ergodic. A second obstacle is that the objective C(I∞(S)) is defined as a long-run average, so its convexity and differentiability in S must be established before any gradient argument applies.
Formalization scope
Everything is stated in Lean 4 with Mathlib. The inventory vector is a pair (pipeline Fin (τ-1) → ℝ, newest order first; on-hand ℝ). Lean periods are 0-based (Lean period t is the paper's t+1); cycles are numbered from 1. Demand is a sequence on a probability space that is i.i.d., nonnegative, integrable, with E[D]>0 and an atomless law; infinite support is F(x)<1 for all x; τ≥1, h,b>0. The long-run cost is a limsup of Cesàro averages from the empty start (Theorem 7 shows it is a limit from every start), and S∗ is any minimizer over [0,∞). The algorithm is a deterministic function of the demand path; the sample derivative is the recursion the algorithm computes, not a derivative. Integrability of every expectation follows from boundedness of the inventory and is not assumed.
Corrections, all disclosed in the items: Theorem 10 carries M instead of M−M on the C4 term, because the printed bound fails for M=M=S∗, X(1,1)=0, L=1, where the right side is 0 and Λ(1)=C(0)−C(I∞(S∗))>0. Lemma 15 carries M and 12τ+1. Lemma 14 keeps its printed constant; its proof appears to use the ergodicity theorem one period early. Theorem 11 is stated on an interval with the bias taken conditionally on the history. Lemmas 5 and 6 are stated for infinite support only.
A goal that bounds the regret by Λ1 alone (costs at the steady state of Sk), or that lets the levels be any sequence in [M,M], is not the theorem: the regret here uses the realized inventory of the algorithm. Lean's conventions x/0=0 and ln of a nonpositive number cannot trivialize the bound, because infinite support and γ(M)>0 give ν∈[1/e,1). The source is the authors' working paper of February 8, 2007, whose printed page numbers equal the PDF's.
Not yet stated: Theorem 3 and Lemma 4 (ergodicity rate), Theorem 8 and Lemma 9 (bias bounds), Theorem 1 (sample derivatives). They need the joint chain (Xt,Xt′) and are welcome as contributions, as are the general pieces: geometric ergodicity via coalescence, and the biased projected-gradient bound.
Selected references
- W. T. Huh, G. Janakiraman, J. A. Muckstadt, P. Rusmevichientong, An Adaptive Algorithm for Finding the Optimal Base-Stock Policy in Lost Sales Inventory Systems with Censored Demand, working paper, 2007; Mathematics of Operations Research 2009. https://doi.org/10.1287/moor.1080.0367
- W. T. Huh, G. Janakiraman, J. A. Muckstadt, P. Rusmevichientong, Asymptotic Optimality of Order-up-to Policies in Lost Sales Inventory Systems, Management Science 55(3), 2009. https://doi.org/10.1287/mnsc.1080.0945
- W. T. Huh, P. Rusmevichientong, A Nonparametric Asymptotic Analysis of Inventory Planning with Censored Demand, Mathematics of Operations Research 2009. https://doi.org/10.1287/moor.1080.0355
- G. Janakiraman, R. O. Roundy, Lost-Sales Problems with Stochastic Lead Times: Convexity Results for Base-Stock Policies, Operations Research 2004. https://doi.org/10.1287/opre.1040.0130