Rebalancing shared vehicles at scale
Shared bike and scooter systems (micromobility) must decide, every period, how to move vehicles between stations, which depleted vehicles to recharge, and how many vehicles to source from or return to a central warehouse. Customer trips redistribute the fleet in the meantime, and some vehicles come back with a depleted battery. For a realistic network, the exact Markov decision process has a state space that grows exponentially in the number of stations, so its optimal policy cannot be computed.
Akturk, Candogan and Gupta (Management Science, 2022; preprint SSRN 4155841) group stations into finitely many types and replace the stochastic network by a deterministic mean-field model that tracks, for each type, the fraction of stations holding each number of charged vehicles and the scaled stock of depleted vehicles. They show that the mean-field model is asymptotically exact as the number of stations grows (their Theorem 3.4), and they propose a control algorithm for it whose optimality gap they bound explicitly (their Theorem 4.2). This mission formalizes the second result.
The compact mean-field model
There are e^ station types, with ne stations of type e and n=∑ene in total. A customer leaving a type-e′ station returns the vehicle at a type-e station with probability pe′,e=newe′,e/∑e2we′,e2ne2. Demand at a type-e station has a law De on N0 with finite mean and variance; a returned vehicle is still charged with probability q, and costs are discounted by γ∈[0,1).
The extended types are 0 (the warehouse), 1,…,e^ (charged vehicles of each type) and e^+1,…,2e^ (the depleted stock of each type). Moving one vehicle from extended type e1 to e2 costs ce1,e2, with ce1,e2≥ce1,e1/2+ce2,e2/2.
An action a^∈A^ is a post-rebalancing distribution: fractions a^e,d≥0 of stations of type e holding d vehicles, supported on {Le,…,Ue} and summing to ne/n, and a scaled depleted stock a^e,r∈[Lr,e,Ur,e]. A state x˘∈X˘ records the fractions x˘e,d for d≤Ue, the depleted stock x˘e,r, and two aggregates x˘e,s,x˘e,u of the vehicles above Ue, within the bounds of the paper's Appendix F.
The next state x˘′(a^) depends on the action only, through the transitions (F.1)–(F.5): demand removes min(b,De) vehicles from a station holding b, and an independent Poisson number R^e of charged vehicles arrives, with a rate (3.2) that is linear in the action's expected trips. Each period costs c^2(x˘,a^)+N^(a^), where
- c^2 (F.6) is a truncated Wasserstein term plus n times the optimal value of a min-cost-flow problem between the extended types;
- N^ (3.4) is the expected holding and shortage cost.
The value function V^(x˘) is the infimum over action sequences in A^ of ∑k≥0γk(c^2(x˘k,ak)+N^(ak)), where x˘0=x˘ and x˘k+1=x˘′(ak).
The algorithm. Given x˘ and a transient horizon T≥0, solve the finite program (4.1): choose a~0,…,a~T∈A^ to minimize
k=0∑T−1γk(c^2(x˘k,a~k)+N^(a~k))+γTc^2(x˘T,a~T)+1−γγT(γc^2(x˘T+1,a~T)+N^(a~T)),
then play a~0,…,a~T−1 and repeat a~T forever ((4.2)–(4.3)). Call this policy π~ and its discounted cost V^π~(x˘).
Formalization targets
Goal: Theorem 4.2
For every T≥0, every x˘∈X˘ and every minimizer of (4.1),
V^π~(x˘)−V^(x˘)≤1−γγCγT,and, if q=0,V^π~(x˘)−V^(x˘)≤CγT,
where
C=2e=1∑e^(nemax(c0,e,ce,0)(Ue−Le)+nmax(c0,e^+e,ce^+e,0)(Ur,e−Lr,e)).
Milestones
- (3.6): V^ satisfies the Bellman equation V^(x˘)=mina^∈A^c^2(x˘,a^)+N^(a^)+γV^(x˘′(a^)) on X˘, with the minimum attained.
- §4.1, p. 26: the cost of π~ equals the optimal value of (4.1).
- Lemma H.7: c^2(x˘,a^)≤c^2(x˘,z^)+c^2(z^,a^), an action read as a state.
- Lemma H.9: c^2(a^1,a^2)≤C/2 for all a^1,a^2∈A^.
- Lemma H.8: under the triangle inequality, there is an optimal policy with π^∗(π^∗(x˘))=π^∗(x˘).
- Proposition H.3: for q=0, the dynamic program has a fixed point x˘f=x˘′(a^f) with a^f optimal at x˘f (Definition H.1).
Significance
Theorem 4.2 makes the algorithm a certified approximation scheme. It turns an infinite-horizon problem into a finite program of T+1 actions, and the loss from doing so decays geometrically in T, uniformly over the initial state, with a constant that depends only on the rebalancing costs and the action bounds. For q=0 the transitions are affine, the program (4.1) becomes a linear program (the paper's Appendix G), and the sharper bound CγT applies.
The result is proved in the paper (Appendix H), but not formalized anywhere. A formal proof combines a triangle inequality for an LP-valued cost, a dynamic-programming principle with attainment on a compact action set, and a fixed-point argument for a convex deterministic dynamic program. The paper has several small printed slips on this path (for example, Proposition H.10 states the q=0 constant without the factor 2 that its proof produces); the formal statements fix the corrected reading.
Difficulty
The bound is not a truncation estimate. Cutting the discounted sum at T gives a gap proportional to the per-period cost, not to C. The comparison has to be made against an optimal action sequence, whose first T+1 actions are feasible for (4.1). After period T, the optimal sequence and π~ diverge, and the cost of that divergence must be controlled by the cost of moving between two actions, which requires:
- the triangle inequality for c^2, whose flow component is the value of a linear program, not a closed form;
- the fact that, once rebalanced to an optimal action, staying there is optimal (Lemma H.8);
- the bound of Lemma H.9 on the cost between any two feasible actions.
The q=0 bound needs, in addition, a steady state of the controlled system (Proposition H.3), which rests on a fixed-point theorem for convex deterministic dynamic programs (Flynn 1979). The Bellman equation itself is not free: with V^ defined as an infimum over sequences, attainment of the minimum needs continuity of c^2, N^, the transition and V^ on the compact action set.
Formalization scope
Types are Fin ê (0-based) and the extended types are Fin (2ê+1) with chg e and dep e for the paper's e and e^+e. Level vectors are functions N→R that vanish outside their support. P[b−min(b,De)+R^e=d] is the convolution of the demand law with the Poisson law, so De and R^e are independent, as the paper presupposes. V^ is the infimum over the subtype of A^-valued sequences (p. 18 and Proposition F.3), and (3.6) is posed as a milestone rather than used as a definition. X˘ is formalized as printed on p. A-22. The policy cost is the discounted cost of a~0,…,a~T−1,a~T,a~T,…, where a~ is a hypothesis-given minimizer of (4.1). Everything is time-homogeneous and stated at t=0.
Standing assumptions, as one predicate on the model: 0≤γ<1, 0≤q≤1, ne≥1, w≥0 with positive normalizers, Le≤Ue, 0≤Lr,e≤Ur,e, finite second moments of De, the cost assumption ce1,e2≥ce1,e1/2+ce2,e2/2, c0,0=0, ce^+e,e^+e=0. Two additions are disclosed: γ≥0, and ce1,e2≥0, without which Lemma H.9 and the goal fail (the right-hand side could be negative).
Trivializing formalizations are excluded: V^ ranges only over A^-valued sequences, the policy is tied to a minimizer of (4.1) and to no other objective, the flow value is an infimum over a nonempty set bounded below (warehouse routing is always feasible), and C uses c0,e, ce,0, c0,e^+e, ce^+e,0 with the paper's indexing and the factor 2.
Two general objects are separate definitions on which the model is built: a discounted deterministic dynamic program (trajectory, discounted cost, value as an infimum over admissible action sequences) and the uncapacitated transshipment LP on a complete digraph with free nodes. Both are reusable beyond this mission. A complete development further needs Poisson tail sums, subadditivity of the flow value, compactness of A^, a dynamic-programming principle and a fixed-point theorem for convex deterministic programs. Proofs of any milestone, and alternative arguments for the q=0 fixed point, are welcome.
Selected references
- D. Akturk, O. Candogan, V. Gupta, Managing Resources for Shared Micromobility: Approximate Optimality in Large-Scale Systems, Management Science, 2022. https://doi.org/10.1287/mnsc.2022.02023 (read as SSRN 4155841, version of 2024-05-22: https://ssrn.com/abstract=4155841)
- J. Flynn, Steady state policies for deterministic dynamic programs, SIAM Journal on Applied Mathematics 37(1):128–147, 1979 (as cited in the paper's appendix references).