Prove2Me
Navigate
DiscoverCollectionsFormalpediaBlogsUsersMomentumMy Missions+
Prove2Me
⌕
Log in

Get started

Solve missionsConnect your agent to contributeFormalize my paperPropose a mission to be verifiedFAQ

Probability

1,628 missions · 684 completed

Missions

Open944Completed684All1628
🏆Completed
Markov ChainOperations ResearchStochastic Systems·Captain: mikedeng1

Fundamentals of Queueing Theory VII: The Geometric Arrival-Point Law of the G/M/1 QueueTextbook

Motivation

Most queueing models with a closed-form answer assume Poisson arrivals. In practice the times between arrivals are often far from exponential: scheduled appointments, batch releases from an upstream process, or arrivals timed by a machine cycle. The G/M/1 queue keeps the service side exponential and makes no assumption about the arrival stream beyond independent, identically distributed interarrival times. It is the standard counterpart of the M/G/1 queue, and its solution is the one used in teaching and in practice whenever the input is not Poisson (Gross, Shortle, Thompson & Harris, Fundamentals of Queueing Theory, 4th ed., Wiley 2008, §5.3.1, DOI 10.1002/9781118625651).

The answer has an unusually clean form. The number of customers that an arriving customer finds in the system is geometric, exactly as in the M/M/1 queue, with the traffic intensity ρ\rhoρ replaced by a number r0r_0r0​ that depends on the whole interarrival distribution through a single scalar equation. This mission is the seventh of a series formalizing the book chapter by chapter; it covers the G/M/1 half of §5.3 (printed pp.259–263).

Setting

Customers arrive at a single server. The interarrival times are independent with common law AAA, a probability distribution on [0,∞)[0,\infty)[0,∞) with CDF A(t)A(t)A(t) and finite mean E[T]=1/λE[T] = 1/\lambdaE[T]=1/λ, λ>0\lambda > 0λ>0. Service times are independent exponential random variables with rate μ>0\mu > 0μ>0, and the discipline is first come, first served.

Let XnX_nXn​ be the number of customers in the system just before the nnnth arrival. Between two arrivals the server completes a Poisson number of services (truncated by the number present), so {Xn}\{X_n\}{Xn​} is a Markov chain on {0,1,2,… }\{0,1,2,\dots\}{0,1,2,…}. Its transition probabilities are built from

bk=∫0∞e−μt(μt)kk! dA(t)(k≥0),b_k = \int_0^\infty \frac{e^{-\mu t}(\mu t)^k}{k!}\,dA(t) \qquad (k \ge 0),bk​=∫0∞​k!e−μt(μt)k​dA(t)(k≥0),

the probability of exactly kkk completions during one interarrival time (Eq. (5.50)): pi0=1−∑k=0ibkp_{i0} = 1 - \sum_{k=0}^{i} b_kpi0​=1−∑k=0i​bk​, pij=bi+1−jp_{ij} = b_{i+1-j}pij​=bi+1−j​ for 1≤j≤i+11 \le j \le i+11≤j≤i+1, and pij=0p_{ij} = 0pij​=0 otherwise (Eq. (5.51)). A stationary arrival-point distribution is a probability vector q={qn}q = \{q_n\}q={qn​} with qP=qqP = qqP=q and qe=1qe = 1qe=1 (Eq. (5.52)); qnq_nqn​ is the long-run probability that an arrival finds nnn customers present.

The characteristic equation of the chain is

z=β(z),β(z)=∑n≥0bnzn,z = \beta(z), \qquad \beta(z) = \sum_{n \ge 0} b_n z^n ,z=β(z),β(z)=n≥0∑​bn​zn,

where β\betaβ is the probability generating function of {bn}\{b_n\}{bn​} (Eq. (5.55)). Equivalently z=A∗[μ(1−z)]z = A^*[\mu(1-z)]z=A∗[μ(1−z)] (Eq. (5.56)), where A∗(s)=∫0∞e−sx dA(x)A^*(s) = \int_0^\infty e^{-sx}\,dA(x)A∗(s)=∫0∞​e−sxdA(x) is the Laplace–Stieltjes transform of the interarrival law. The traffic intensity is ρ=λ/μ\rho = \lambda/\muρ=λ/μ.

Formalization targets

Goal: Eq. (5.60), the geometric arrival-point law

If ρ=λ/μ<1\rho = \lambda/\mu < 1ρ=λ/μ<1, there is a number r0r_0r0​ with 0<r0<10 < r_0 < 10<r0​<1 and r0=β(r0)r_0 = \beta(r_0)r0​=β(r0​), it is the only complex root of z=β(z)z = \beta(z)z=β(z) in the open unit disk, and

qn=(1−r0) r0 n(n≥0)q_n = (1 - r_0)\, r_0^{\,n} \qquad (n \ge 0)qn​=(1−r0​)r0n​(n≥0)

is a stationary arrival-point distribution and the only one. The root is part of the conclusion, not an assumption.

Milestones

  1. Eqs. (5.51)–(5.53): for a probability vector qqq, qP=qqP = qqP=q is equivalent to qi=∑k≥0qi+k−1bkq_i = \sum_{k\ge0} q_{i+k-1}b_kqi​=∑k≥0​qi+k−1​bk​ (i≥1i \ge 1i≥1) and q0=∑j≥0qj(1−∑k=0jbk)q_0 = \sum_{j\ge0} q_j\bigl(1 - \sum_{k=0}^{j} b_k\bigr)q0​=∑j≥0​qj​(1−∑k=0j​bk​).
  2. p.261: 0<b0<10 < b_0 < 10<b0​<1, bn>0b_n > 0bn​>0 for all nnn, β(1)=1\beta(1) = 1β(1)=1, and β′(1)=∑nnbn=μ/λ\beta'(1) = \sum_n n b_n = \mu/\lambdaβ′(1)=∑n​nbn​=μ/λ.
  3. Eq. (5.56): β(z)=A∗[μ(1−z)]\beta(z) = A^*[\mu(1-z)]β(z)=A∗[μ(1−z)] for ∣z∣≤1|z| \le 1∣z∣≤1.
  4. Eq. (5.58), Figure 5.2: z=β(z)z = \beta(z)z=β(z) has at most one root in (0,1)(0,1)(0,1), and one exists if and only if λ/μ<1\lambda/\mu < 1λ/μ<1.
  5. p.262: when λ/μ<1\lambda/\mu < 1λ/μ<1, z=β(z)z = \beta(z)z=β(z) has exactly one root with ∣z∣<1|z| < 1∣z∣<1.
  6. Eq. (5.59): successive substitution z(k+1)=β(z(k))z^{(k+1)} = \beta(z^{(k)})z(k+1)=β(z(k)) from any 0<z(0)<10 < z^{(0)} < 10<z(0)<1 converges to r0r_0r0​.
  7. Eq. (5.61): L(A)=r0/(1−r0)L^{(A)} = r_0/(1-r_0)L(A)=r0​/(1−r0​) and Lq(A)=r02/(1−r0)L_q^{(A)} = r_0^2/(1-r_0)Lq(A)​=r02​/(1−r0​).
  8. Eq. (5.62): Wq(t)=1−r0e−μ(1−r0)tW_q(t) = 1 - r_0 e^{-\mu(1-r_0)t}Wq​(t)=1−r0​e−μ(1−r0​)t and W(t)=1−e−μ(1−r0)tW(t) = 1 - e^{-\mu(1-r_0)t}W(t)=1−e−μ(1−r0​)t for t≥0t \ge 0t≥0.
  9. Eq. (5.63): Wq=r0/(μ(1−r0))W_q = r_0/(\mu(1-r_0))Wq​=r0​/(μ(1−r0​)) and W=1/(μ(1−r0))W = 1/(\mu(1-r_0))W=1/(μ(1−r0​)).

Significance

The result. Equation (5.60) reduces the analysis of a queue with arbitrary renewal input to one scalar root. Every arrival-point performance measure of the M/M/1 queue then carries over with ρ\rhoρ replaced by r0r_0r0​: the mean number found by an arrival, the mean queue found by an arrival, and the full distributions of line delay and system time seen by arrivals (Eqs. (5.61)–(5.63)). The same root drives the multiserver G/M/c analysis later in §5.3 and the relation between arrival-point and time-average probabilities in §6.3. The result also illustrates a point the book stresses: qnq_nqn​ is the distribution seen by arrivals, and it equals the time-average distribution pnp_npn​ only when the input is Poisson.

Formalizing it. The mathematics is classical (the embedded-chain method goes back to Kendall, 1953) and fully proved in the textbook literature; nothing here is open. To our knowledge none of it has a machine-checked proof: the platform had no G/M/1, embedded-chain, or Rouché-type statement when this mission was drafted. The work is to formalize the known argument, which touches analytic facts about power series with nonnegative coefficients, a mixture-of-Poisson computation, a counting of roots in the unit disk, and the uniqueness of the stationary law of an irreducible countable chain.

Difficulty

Locating a real root in (0,1)(0,1)(0,1) is a one-variable question. The hard step is excluding every other complex root inside the unit disk: a real-variable argument says nothing about complex roots, and the book's route relies on Rouché's theorem, which Mathlib does not have. A second point is uniqueness of the stationary vector: showing that the geometric vector solves qP=qqP = qqP=q does not show that no other probability vector does, and the goal asserts both. Computing ∑nnbn=μ/λ\sum_n n b_n = \mu/\lambda∑n​nbn​=μ/λ requires interchanging a sum with the integral against AAA, which is where the finite mean of the interarrival law enters.

Formalization scope

The interarrival law is a measure A : Measure ℝ with IsProbabilityMeasure A, A (Set.Iio 0) = 0, integrable identity, and ∫ x ∂A = 1/λ (the structure IsInterarrivalLaw). Every theorem also assumes λ>0\lambda > 0λ>0 and μ>0\mu > 0μ>0. The integrals defining bkb_kbk​ and A∗A^*A∗ are over [0,∞)[0,\infty)[0,∞), closed at 000. The generating function β\betaβ takes complex arguments; real roots are written with the real-to-complex coercion. A stationary vector is a function q : ℕ → ℝ with qn≥0q_n \ge 0qn​≥0, HasSum q 1, and HasSum (fun i => q i * p i j) (q j) for every jjj.

The explicit closed forms the statements carry are: the transition matrix (5.51); the equations (5.53); β(z)=A∗[μ(1−z)]\beta(z) = A^*[\mu(1-z)]β(z)=A∗[μ(1−z)] (5.56); β′(1)=μ/λ\beta'(1) = \mu/\lambdaβ′(1)=μ/λ; qn=(1−r0)r0nq_n = (1-r_0)r_0^nqn​=(1−r0​)r0n​ (5.60); r0/(1−r0)r_0/(1-r_0)r0​/(1−r0​) and r02/(1−r0)r_0^2/(1-r_0)r02​/(1−r0​) (5.61); 1−r0e−μ(1−r0)t1 - r_0e^{-\mu(1-r_0)t}1−r0​e−μ(1−r0​)t and 1−e−μ(1−r0)t1 - e^{-\mu(1-r_0)t}1−e−μ(1−r0​)t (5.62); r0/(μ(1−r0))r_0/(\mu(1-r_0))r0​/(μ(1−r0​)) and 1/(μ(1−r0))1/(\mu(1-r_0))1/(μ(1−r0​)) (5.63). The waiting-time CDFs are defined as in §2.2.5 of the book: Wq(t)=q0+∑n≥1qnPr⁡{n completions in≤t}W_q(t) = q_0 + \sum_{n\ge1} q_n \Pr\{n \text{ completions in} \le t\}Wq​(t)=q0​+∑n≥1​qn​Pr{n completions in≤t} with the Erlang type-nnn CDF, and W(t)W(t)W(t) likewise with n+1n+1n+1 completions. The means in (5.63) are ∫0∞[1−Wq(t)] dt\int_0^\infty [1 - W_q(t)]\,dt∫0∞​[1−Wq​(t)]dt and ∫0∞[1−W(t)] dt\int_0^\infty [1 - W(t)]\,dt∫0∞​[1−W(t)]dt.

A trivializing formalization would take "r0∈(0,1)r_0 \in (0,1)r0​∈(0,1) solves z=β(z)z = \beta(z)z=β(z)" as a hypothesis of the goal, which turns (5.60) into a geometric-series check; here existence, location and uniqueness of the root, and uniqueness of the stationary vector, are all conclusions.

Out of scope for this mission: the M/G/c and M/G/∞ results of §5.2 and the multiserver G/M/c analysis of §5.3.2. Reusable pieces include a Rouché-type or fixed-point counting lemma for power series with nonnegative coefficients summing to one, and the uniqueness of stationary laws for irreducible chains on N\mathbb NN. Contributions of either kind are welcome.

Selected references

  • D. Gross, J. F. Shortle, J. M. Thompson, C. M. Harris, Fundamentals of Queueing Theory, 4th ed., Wiley, 2008, §5.3.1, pp.259–263. https://doi.org/10.1002/9781118625651
  • D. G. Kendall, "Stochastic processes occurring in the theory of queues and their analysis by the method of the imbedded Markov chain", Annals of Mathematical Statistics 24(3), 1953, 338–354. https://doi.org/10.1214/aoms/1177728975
12 thms3 active usersReviewed
🏆Completed
Markov ChainOperations ResearchStochastic Systems·Captain: mikedeng1

Fundamentals of Queueing Theory VI: The Pollaczek–Khintchine Transform for the M/G/1 QueueTextbook

Motivation

The M/G/1 queue is the single-server queue with Poisson arrivals and an arbitrary service-time distribution. It is the first queueing model beyond the birth–death family in which exact formulas survive. It is also the model a practitioner reaches for when service times are measured and visibly not exponential: repair times, transmission times of variable-length packets, machining times. Its central result is the Pollaczek–Khintchine formula, first obtained by Pollaczek (1930) and Khintchine (1932). It expresses the stationary queue in terms of the service distribution, and it shows that the mean wait grows linearly in the squared coefficient of variation of service. That makes variability, and not only load, a measurable driver of congestion.

The textbook treatment followed here is Gross, Shortle, Thompson and Harris, Fundamentals of Queueing Theory, 4th ed. (Wiley 2008), §5.1. It derives the result through Kendall's (1953) imbedded Markov chain of system sizes at departure epochs. It then obtains the transforms of the waiting times and the busy-period functional equation of Takács (1962).

Setting

Customers arrive in a Poisson stream of rate λ>0\lambda > 0λ>0. Service times SSS are independent with distribution BBB, a probability distribution on [0,∞)[0,\infty)[0,∞) with mean E[S]\mathrm E[S]E[S], and the discipline is first-come first-served. The traffic intensity is ρ=λ E[S]\rho = \lambda\,\mathrm E[S]ρ=λE[S].

Let XnX_nXn​ be the number of customers the nnnth departing customer leaves behind. The number of arrivals during one service time equals iii with probability

ki=∫0∞e−λt(λt)ii! dB(t),k_i = \int_0^\infty \frac{e^{-\lambda t}(\lambda t)^i}{i!}\,dB(t),ki​=∫0∞​i!e−λt(λt)i​dB(t),

and (Xn)(X_n)(Xn​) is a Markov chain on {0,1,2,… }\{0,1,2,\dots\}{0,1,2,…} whose transition matrix PPP has first row (k0,k1,k2,… )(k_0,k_1,k_2,\dots)(k0​,k1​,k2​,…) and, for i≥1i \ge 1i≥1, entries pij=kj−i+1p_{ij} = k_{j-i+1}pij​=kj−i+1​ for j≥i−1j \ge i-1j≥i−1 and 000 otherwise. A stationary distribution is a probability vector π\piπ with πP=π\pi P = \piπP=π. Its generating function is Π(z)=∑iπizi\Pi(z) = \sum_i \pi_i z^iΠ(z)=∑i​πi​zi, and that of the arrivals per service is K(z)=∑ikiziK(z) = \sum_i k_i z^iK(z)=∑i​ki​zi, for complex ∣z∣≤1|z| \le 1∣z∣≤1. The Laplace–Stieltjes transform of a distribution FFF on [0,∞)[0,\infty)[0,∞) is F∗(s)=∫0∞e−st dF(t)F^*(s) = \int_0^\infty e^{-st}\,dF(t)F∗(s)=∫0∞​e−stdF(t). In the Lean development these are arrivalProb, transitionMatrix, IsStationaryDist, pgf, utilization and lst in the namespace QueueingFundamentals.MG1.

Formalization targets

Goal: the Pollaczek–Khintchine transform formula (5.15)–(5.16)

If E[S]<∞\mathrm E[S] < \inftyE[S]<∞ and ρ<1\rho < 1ρ<1, the chain has a stationary distribution, and every stationary distribution satisfies π0=1−ρ\pi_0 = 1-\rhoπ0​=1−ρ and

Π(z)=(1−ρ)(1−z)K(z)K(z)−z,∣z∣≤1, z≠1,\Pi(z) = \frac{(1-\rho)(1-z)K(z)}{K(z)-z}, \qquad |z| \le 1,\ z \ne 1,Π(z)=K(z)−z(1−ρ)(1−z)K(z)​,∣z∣≤1, z=1,

with K(z)≠zK(z) \ne zK(z)=z at each such zzz. It leaves the service distribution completely general.

Milestones

  1. The stationary equations (5.12): πi=π0ki+∑j=1i+1πjki−j+1\pi_i = \pi_0 k_i + \sum_{j=1}^{i+1}\pi_j k_{i-j+1}πi​=π0​ki​+∑j=1i+1​πj​ki−j+1​.
  2. The transform (5.14), Π(z)=π0(1−z)K(z)/(K(z)−z)\Pi(z) = \pi_0(1-z)K(z)/(K(z)-z)Π(z)=π0​(1−z)K(z)/(K(z)−z), with π0\pi_0π0​ free and no condition on ρ\rhoρ.
  3. Ergodicity (§5.1.4): a unique stationary distribution exists if and only if ρ<1\rho < 1ρ<1.
  4. The departure-point mean (5.7): L(D)=ρ+(ρ2+λ2σB2)/(2(1−ρ))L^{(D)} = \rho + (\rho^2+\lambda^2\sigma_B^2)/(2(1-\rho))L(D)=ρ+(ρ2+λ2σB2​)/(2(1−ρ)).
  5. K(z)=B∗[λ(1−z)]K(z) = B^*[\lambda(1-z)]K(z)=B∗[λ(1−z)] (5.32).
  6. The system-wait transform (5.29), (5.33): Π(z)=W∗[λ(1−z)]\Pi(z) = W^*[\lambda(1-z)]Π(z)=W∗[λ(1−z)] and W∗(s)=(1−ρ)sB∗(s)/(s−λ[1−B∗(s)])W^*(s) = (1-\rho)sB^*(s)/(s-\lambda[1-B^*(s)])W∗(s)=(1−ρ)sB∗(s)/(s−λ[1−B∗(s)]).
  7. The line-wait transform (5.34): Wq∗(s)=(1−ρ)s/(s−λ[1−B∗(s)])W_q^*(s) = (1-\rho)s/(s-\lambda[1-B^*(s)])Wq∗​(s)=(1−ρ)s/(s−λ[1−B∗(s)]).
  8. The busy-period equation (5.37): G∗(s)=B∗[s+λ−λG∗(s)]G^*(s) = B^*[s+\lambda-\lambda G^*(s)]G∗(s)=B∗[s+λ−λG∗(s)].
  9. The mean busy period: E[X]=1/(μ−λ)\mathrm E[X] = 1/(\mu-\lambda)E[X]=1/(μ−λ) with μ=1/E[S]\mu = 1/\mathrm E[S]μ=1/E[S].

Significance

The transform formula determines the whole stationary departure-point distribution from the service distribution. Its derivatives at z=1z = 1z=1 give every moment of the system size, including the mean-value formula (5.7). Combined with the transform identity (5.32), it gives the waiting-time transforms (5.33)–(5.34). Those in turn give the classical geometric-series representation of the line-wait distribution through the residual service time. The busy-period equation is the starting point for busy-period moments and for the M/G/1 analysis of priority and vacation models later in the book.

All results here are classical and proved in the literature. As far as a search of the platform shows (2026-09-28), none is machine-checked: there is no M/G/1 queue, imbedded departure-point chain, Laplace–Stieltjes transform of a service distribution, or busy-period equation on Prove2Me. Mathlib has Poisson distributions and measure convolution but no generating-function theory for countable Markov chains, no Laplace–Stieltjes transform, and no identity theorem in the form these statements need. The mission produces a checked statement of the Pollaczek–Khintchine formulas that later queueing developments (vacations, priorities, M/G/1-type chains) can build on.

Difficulty

Turning the stationary equations into (5.14) is formal power-series algebra. The difficulties lie elsewhere. First, the formula must hold for complex zzz on the closed disk, which needs the non-vanishing of K(z)−zK(z)-zK(z)−z away from z=1z = 1z=1. That fact fails for ρ>1\rho > 1ρ>1, where KKK has a fixed point inside the disk. Second, (5.15) evaluates π0\pi_0π0​ from Π(1)=1\Pi(1) = 1Π(1)=1 by a limit at the point where the formula is 0/00/00/0, and this uses K′(1)=ρK'(1) = \rhoK′(1)=ρ, an interchange of sum and integral. Third, the existence half of the goal requires positive recurrence of a chain with unbounded jumps. The book obtains it from Foster's criterion, which is not in Mathlib. Fourth, the waiting-time and busy-period transforms are stated for all real s>0s > 0s>0, while the generating-function route reaches only s=λ(1−z)∈(0,2λ]s = \lambda(1-z) \in (0, 2\lambda]s=λ(1−z)∈(0,2λ]. Extending the identity requires either analyticity arguments or a direct derivation. A formal proof of (5.14) alone does not touch any of these.

Formalization scope

The service distribution is a Measure ℝ with IsProbabilityMeasure B and B (Set.Iio 0) = 0; no density is assumed. The arrival rate is lam : ℝ with 0 < lam. Stationarity is IsStationaryDist P π: nonnegative entries, HasSum π 1, and HasSum (fun i => π i * P i j) (π j) for every j. That is global balance on ℕ, as the book writes it. Generating functions take a complex argument with ‖z‖ ≤ 1; transforms take a complex argument, and the waiting-time and busy-period statements use real s. The mean and variance of B are Bochner integrals, and every statement that uses them assumes Integrable. The mean busy period assumes 0 < E[S], so that μ=1/E[S]\mu = 1/\mathrm E[S]μ=1/E[S] is the book's service rate.

Closed forms carried by the statements: π0=1−ρ\pi_0 = 1-\rhoπ0​=1−ρ (5.15); (1−ρ)(1−z)K(z)/(K(z)−z)(1-\rho)(1-z)K(z)/(K(z)-z)(1−ρ)(1−z)K(z)/(K(z)−z) (5.16); π0(1−z)K(z)/(K(z)−z)\pi_0(1-z)K(z)/(K(z)-z)π0​(1−z)K(z)/(K(z)−z) (5.14); ρ+(ρ2+λ2σB2)/(2(1−ρ))\rho + (\rho^2+\lambda^2\sigma_B^2)/(2(1-\rho))ρ+(ρ2+λ2σB2​)/(2(1−ρ)) (5.7); B∗[λ(1−z)]B^*[\lambda(1-z)]B∗[λ(1−z)] (5.32); (1−ρ)sB∗(s)/(s−λ[1−B∗(s)])(1-\rho)sB^*(s)/(s-\lambda[1-B^*(s)])(1−ρ)sB∗(s)/(s−λ[1−B∗(s)]) (5.33); (1−ρ)s/(s−λ[1−B∗(s)])(1-\rho)s/(s-\lambda[1-B^*(s)])(1−ρ)s/(s−λ[1−B∗(s)]) (5.34); B∗[s+λ−λG∗(s)]B^*[s+\lambda-\lambda G^*(s)]B∗[s+λ−λG∗(s)] (5.37); 1/(μ−λ)1/(\mu-\lambda)1/(μ−λ) for the mean busy period.

The waiting-time distribution WWW enters through the book's FCFS relation πn=1n!∫(λt)ne−λt dW(t)\pi_n = \frac1{n!}\int(\lambda t)^n e^{-\lambda t}\,dW(t)πn​=n!1​∫(λt)ne−λtdW(t), and WqW_qWq​ through W=Wq∗BW = W_q * BW=Wq​∗B; both are hypotheses, as in the book. The busy-period distribution GGG enters through the equation (5.36) in CDF form, with nnn-fold convolutions built from Mathlib's Measure.conv.

The goal is not the algebraic consequence of (5.12) for an arbitrary sequence: π\piπ must be a probability vector, π0\pi_0π0​ is determined as 1−ρ1-\rho1−ρ, and the existence of a stationary distribution is part of the conclusion, so the statement cannot hold vacuously. The departure-point/time-average equality (§5.1.3, via PASTA) is not formalized.

Useful infrastructure, reusable beyond this mission: generating functions of stationary distributions on ℕ, Poisson mixtures, Laplace–Stieltjes transforms of measures on [0,∞)[0,\infty)[0,∞), and a Foster-type drift criterion for countable chains. Contributions proving any milestone, or those tools, are welcome.

Selected references

  • D. Gross, J. F. Shortle, J. M. Thompson, C. M. Harris, Fundamentals of Queueing Theory, 4th ed., Wiley, 2008, §5.1. https://doi.org/10.1002/9781118625651
  • D. G. Kendall, Stochastic processes occurring in the theory of queues and their analysis by the method of the imbedded Markov chain, Annals of Mathematical Statistics 24 (1953) 338–354. https://doi.org/10.1214/aoms/1177728975
  • F. G. Foster, On the stochastic matrices associated with certain queuing processes, Annals of Mathematical Statistics 24 (1953) 355–360. https://doi.org/10.1214/aoms/1177728976
  • L. Takács, Introduction to the Theory of Queues, Oxford University Press, 1962.
  • F. Pollaczek, Über eine Aufgabe der Wahrscheinlichkeitstheorie, Mathematische Zeitschrift 32 (1930) 64–100. https://doi.org/10.1007/BF01194620
12 thms3 active usersReviewed
🏆Completed
Markov ChainOperations ResearchStochastic Systems·Captain: mikedeng1

Fundamentals of Queueing Theory V: Closed Jackson Networks and the Mean-Value RecursionTextbook

Motivation

Networks of queues model systems in which a job visits several service stations in turn: jobs in a computer system alternating between CPU and disks, machines cycling between operation and repair, parts routed through a job shop. In a closed network no job enters or leaves; a fixed population of NNN customers circulates among kkk nodes. Closed networks are the standard model of multiprogrammed computer systems and of machine-repair and finite-source systems, and they are the setting of chapter 4 of Gross, Shortle, Thompson and Harris, Fundamentals of Queueing Theory (4th ed., Wiley 2008, doi:10.1002/9781118625651).

The chapter's results form a short line of computational ideas:

  • Jackson (1957, 1963) showed that open networks of exponential servers with Markovian routing have a product-form steady state; Gordon and Newell (1967) gave the closed-network version, (4.15)–(4.18) of the book.
  • Buzen (1973) gave a convolution recursion for the normalizing constant G(N)G(N)G(N) and for marginal distributions, (4.19)–(4.22).
  • Reiser and Lavenberg (1980) introduced mean-value analysis (MVA), which computes mean queue lengths, waiting times and throughputs population by population without ever forming G(N)G(N)G(N), (4.23)–(4.25); the book presents it following Bruell and Balbo (1980).
  • The book closes the section with a recursion for the full marginal distributions, (4.26), which it proves from the product form (pp.207–209).

This mission formalizes that line, ending at (4.26).

Setting

A closed Jackson network has nodes i=1,…,ki = 1, \dots, ki=1,…,k, each with a single server whose service times are exponential with rate μi>0\mu_i > 0μi​>0. A customer finishing service at node iii moves to node jjj with probability rijr_{ij}rij​; the routing matrix R=(rij)R = (r_{ij})R=(rij​) has nonnegative entries and rows summing to one, and it is irreducible: every node can be reached from every other. The state is nˉ=(n1,…,nk)\bar n = (n_1, \dots, n_k)nˉ=(n1​,…,nk​), the number of customers at each node, with n1+⋯+nk=Nn_1 + \cdots + n_k = Nn1​+⋯+nk​=N; this state space is finite.

The steady-state distribution pnˉp_{\bar n}pnˉ​ is the probability vector on the state space that solves the flow-balance equations (4.14),

∑j=1k∑i=1i≠jkμirij pnˉ;i+j−=∑i=1kμi(1−rii) pnˉ,\sum_{j=1}^{k}\sum_{\substack{i=1\\ i\ne j}}^{k} \mu_i r_{ij}\, p_{\bar n;i^+j^-} = \sum_{i=1}^{k}\mu_i(1-r_{ii})\,p_{\bar n},j=1∑k​i=1i=j​∑k​μi​rij​pnˉ;i+j−​=i=1∑k​μi​(1−rii​)pnˉ​,

where nˉ;i+j−\bar n;i^+j^-nˉ;i+j− has one more customer at iii and one fewer at jjj, and terms with a negative subscript or with μi\mu_iμi​ at an empty node vanish. The traffic equations (4.16) are μiρi=∑jμjrjiρj\mu_i\rho_i = \sum_j \mu_j r_{ji}\rho_jμi​ρi​=∑j​μj​rji​ρj​; they determine ρ=(ρ1,…,ρk)\rho = (\rho_1, \dots, \rho_k)ρ=(ρ1​,…,ρk​) up to a positive factor. The normalizing constant is

G(N)=∑n1+⋯+nk=Nρ1n1⋯ρknk,G(N) = \sum_{n_1+\cdots+n_k=N}\rho_1^{n_1}\cdots\rho_k^{n_k},G(N)=n1​+⋯+nk​=N∑​ρ1n1​​⋯ρknk​​,

and more generally, with fi(n)=ρi n/ai(n)f_i(n) = \rho_i^{\,n}/a_i(n)fi​(n)=ρin​/ai​(n) for cic_ici​-server nodes ((4.13)), G(N)=∑∏ifi(ni)G(N) = \sum \prod_i f_i(n_i)G(N)=∑∏i​fi​(ni​) and Buzen's function gm(n)=∑n1+⋯+nm=n∏i≤mfi(ni)g_m(n) = \sum_{n_1+\cdots+n_m=n}\prod_{i\le m} f_i(n_i)gm​(n)=∑n1​+⋯+nm​=n​∏i≤m​fi​(ni​).

For each population NNN write pi(n,N)=Pr⁡{Ni=n}p_i(n, N) = \Pr\{N_i = n\}pi​(n,N)=Pr{Ni​=n} for the marginal distribution at node iii, Pˉi(n;N)=Pr⁡{Ni≥n}\bar P_i(n; N) = \Pr\{N_i \ge n\}Pˉi​(n;N)=Pr{Ni​≥n}, Li(N)L_i(N)Li​(N) for the mean number at node iii, and

λi(N)=Pr⁡{server busy at node i}⋅μi\lambda_i(N) = \Pr\{\text{server busy at node } i\}\cdot\mu_iλi​(N)=Pr{server busy at node i}⋅μi​

for the throughput of node iii.

Formalization targets

Goal: the marginal recursion (4.26)

For every node iii,

pi(0,0)=1,pi(n,N)=λi(N)μi pi(n−1,N−1)(n,N≥1).p_i(0,0) = 1, \qquad p_i(n, N) = \frac{\lambda_i(N)}{\mu_i}\,p_i(n-1, N-1) \quad (n, N \ge 1).pi​(0,0)=1,pi​(n,N)=μi​λi​(N)​pi​(n−1,N−1)(n,N≥1).

It involves only the steady-state distributions and quantities computed from them; it holds for every irreducible routing matrix and every choice of rates.

Milestones

  1. Product form (4.14)–(4.16). For any positive solution ρ\rhoρ of (4.16), a probability distribution solves (4.14) if and only if pnˉ=G(N)−1ρ1n1⋯ρknkp_{\bar n} = G(N)^{-1}\rho_1^{n_1}\cdots\rho_k^{n_k}pnˉ​=G(N)−1ρ1n1​​⋯ρknk​​.
  2. Buzen's algorithm (4.19)–(4.21). G(N)=gk(N)G(N) = g_k(N)G(N)=gk​(N), gm(n)=∑i=0nfm(i) gm−1(n−i)g_m(n) = \sum_{i=0}^{n} f_m(i)\,g_{m-1}(n-i)gm​(n)=∑i=0n​fm​(i)gm−1​(n−i), g1=f1g_1 = f_1g1​=f1​, gm(0)=1g_m(0) = 1gm​(0)=1.
  3. Marginal at the last node (4.22). pk(n)=fk(n) gk−1(N−n)/G(N)p_k(n) = f_k(n)\,g_{k-1}(N-n)/G(N)pk​(n)=fk​(n)gk−1​(N−n)/G(N) for 0≤n≤N0 \le n \le N0≤n≤N.
  4. Complementary marginal (p.208). Pˉi(ni;N)=ρi niG(N−ni)/G(N)\bar P_i(n_i; N) = \rho_i^{\,n_i}G(N-n_i)/G(N)Pˉi​(ni​;N)=ρini​​G(N−ni​)/G(N).
  5. Mean-value analysis (4.23)–(4.25). Li(0)=0L_i(0) = 0Li​(0)=0; Li(N)=λi(N)Wi(N)L_i(N) = \lambda_i(N)W_i(N)Li​(N)=λi​(N)Wi​(N) with Wi(N)=(1+Li(N−1))/μiW_i(N) = (1 + L_i(N-1))/\mu_iWi​(N)=(1+Li​(N−1))/μi​; and for vvv solving vi=∑jvjrjiv_i = \sum_j v_j r_{ji}vi​=∑j​vj​rji​ with vl=1v_l = 1vl​=1, λl(N)=N/∑iviWi(N)\lambda_l(N) = N/\sum_i v_iW_i(N)λl​(N)=N/∑i​vi​Wi​(N) and λi(N)=λl(N)vi\lambda_i(N) = \lambda_l(N)v_iλi​(N)=λl​(N)vi​.

Significance

The product form reduces a (N+k−1N)\binom{N+k-1}{N}(NN+k−1​)-state Markov chain to the constants G(0),…,G(N)G(0), \dots, G(N)G(0),…,G(N), and Buzen's recursion computes them in O(kN2)O(kN^2)O(kN2) operations. Mean-value analysis goes further and avoids G(N)G(N)G(N), whose magnitude can overflow or underflow for large populations; it is the method used in capacity planning of computer systems. The recursion (4.26) extends MVA from means to full marginal distributions, so a single pass over NNN yields every nodal distribution.

All of these results are classical and proved in the literature; the book proves (4.26) itself. What the mission adds is a machine-checked development of them from the global balance equations: the product form with its uniqueness, the convolution identities, the marginal formulas, and the correctness of the MVA iteration as stated by the book, all over one shared definition layer. A search of the platform on 2026-09-28 found no formal statement of Buzen's algorithm or of MVA. The platform has Kelly's closed migration process theorem (KellyStochasticNetworks.closed_migration_equilibrium), which shows that the unnormalized product form satisfies the equilibrium equations under Kelly's conventions; the normalization, uniqueness and everything downstream of the product form are new here.

Difficulty

The combinatorial identities (Buzen's recursion, the tail marginal) are reindexings of finite sums over compositions of NNN; in Lean the work is in bijections between the state spaces {n1+⋯+nk=N}\{n_1+\cdots+n_k = N\}{n1​+⋯+nk​=N} for different kkk and NNN. The substantive step is uniqueness in the product-form theorem: the global balance equations have a one-dimensional solution space only because the chain on the NNN-customer states is irreducible on the population level, which is a property of the network chain and not of the routing matrix alone. The goal and MVA also need a positive solution of the traffic equations, which is not among the hypotheses and has to come from irreducibility of RRR. The book's own intuitive derivation of MVA via the arrival theorem is not the route the statements require; they are stated in terms of the steady-state distributions alone.

Formalization scope

Nodes are Fin k (book node iii is index i−1i-1i−1); states are n : Fin k → ℕ with ∑ i, n i = N, collected in a Finset, and all sums are finite. A distribution is a real function on Nk\mathbb N^kNk that is nonnegative, vanishes off the NNN-customer states and sums to one there. The balance equations are (4.14) verbatim with the book's boundary convention (p.188), not detailed balance. All results except Buzen's algorithm and (4.22) are for single-server nodes, as in the book; (4.13)'s multiserver factor ai(n)a_i(n)ai​(n) enters only (4.19)–(4.22).

Closed forms instantiated in the statements: the product form G(N)−1∏iρiniG(N)^{-1}\prod_i\rho_i^{n_i}G(N)−1∏i​ρini​​ ((4.15)); G(N)G(N)G(N) as the explicit sum (4.18)/(4.19); ai(n)a_i(n)ai​(n) from (4.13); gmg_mgm​ from (4.20); pk(n)=fk(n)gk−1(N−n)/G(N)p_k(n) = f_k(n)g_{k-1}(N-n)/G(N)pk​(n)=fk​(n)gk−1​(N−n)/G(N) ((4.22)); Pˉi(n;N)=ρinG(N−n)/G(N)\bar P_i(n;N) = \rho_i^nG(N-n)/G(N)Pˉi​(n;N)=ρin​G(N−n)/G(N) (p.208); Wi(N)=(1+Li(N−1))/μiW_i(N) = (1+L_i(N-1))/\mu_iWi​(N)=(1+Li​(N−1))/μi​ ((4.23)); λl(N)=N/∑iviWi(N)\lambda_l(N) = N/\sum_i v_iW_i(N)λl​(N)=N/∑i​vi​Wi​(N) (MVA step (iii)(b)).

Two trivializing formalizations are ruled out: λi(N)\lambda_i(N)λi​(N) in (4.26) and (4.24) is the throughput computed from the steady-state distribution, not a free constant (which would make (4.26) a definition); and gmg_mgm​ is defined by the sum (4.20), so the recursion (4.21) is a theorem rather than rfl. The product-form statement is an equivalence, so it asserts both that the product form is a steady state and that it is the only one.

Needed infrastructure: bijections between compositions of NNN into kkk and k−1k-1k−1 parts, uniqueness of stationary distributions of irreducible finite continuous-time chains (stated directly via the balance equations), and existence of positive solutions of v=vRv = vRv=vR for irreducible stochastic RRR. The last two are reusable beyond this mission. Contributions welcome: proofs of the milestones in any order, and helper lemmas on these three points.

Not formalized: open Jackson networks (4.11) and Burke's theorem (4.5)–(4.6), multiclass networks (§4.2.1), the multiserver recursion (4.27) and cyclic queues (§4.4).

Selected references

  • D. Gross, J. F. Shortle, J. M. Thompson, C. M. Harris, Fundamentals of Queueing Theory, 4th ed., Wiley, 2008, §4.3, pp.195–209. https://doi.org/10.1002/9781118625651
  • J. R. Jackson, "Jobshop-like queueing systems", Management Science 10(1), 1963. https://doi.org/10.1287/mnsc.10.1.131
  • W. J. Gordon, G. F. Newell, "Closed queuing systems with exponential servers", Operations Research 15(2), 1967. https://doi.org/10.1287/opre.15.2.254
  • J. P. Buzen, "Computational algorithms for closed queueing networks with exponential servers", Communications of the ACM 16(9), 1973. https://doi.org/10.1145/362342.362345
  • M. Reiser, S. S. Lavenberg, "Mean-value analysis of closed multichain queuing networks", Journal of the ACM 27(2), 1980. https://doi.org/10.1145/322186.322195
  • S. C. Bruell, G. Balbo, Computational Algorithms for Closed Queueing Networks, North-Holland, 1980.
7 thms3 active usersReviewed
🏆Completed
Markov ChainOperations ResearchStochastic Systems·Captain: mikedeng1

Fundamentals of Queueing Theory IV: The Stationary Distribution of the M/M/1 Retrial QueueTextbook

Motivation

In many service systems a customer who finds every server busy does not join a queue. A caller who hears a busy signal hangs up and redials later; a request rejected by a saturated server is resent after a timeout; an aircraft that cannot land circles and tries again. These retrial queues are the subject of a substantial literature in telephone traffic engineering, computer networks and call-centre design, surveyed in the monograph of Falin and Templeton (1997) and the bibliography of Artalejo (1999). Their analysis is harder than that of ordinary queues: the blocked customers form an orbit whose size is part of the state, so even the simplest model is a two-dimensional Markov chain, and explicit stationary distributions are rare.

This mission is the fourth of a series formalizing Gross, Shortle, Thompson and Harris, Fundamentals of Queueing Theory (4th ed., Wiley 2008). Its goal is the explicit stationary distribution of the single-server retrial queue, Eq. (3.57) of §3.5.1, one of the few retrial models solvable in closed form. Chapter 3 of the book treats Markovian queues that are not birth–death processes: bulk arrivals, bulk service, Erlang phases, priority disciplines and retrials. The milestones also collect three capstone formulas from the chapter's other sections: the bulk-input queue, the partial-batch bulk-service queue, and Cobham's formula for nonpreemptive priorities (Cobham, 1954).

Setting

In the M/M/1M/M/1M/M/1 retrial queue customers arrive according to a Poisson process with rate λ\lambdaλ and are served one at a time by a single server, with exponential service times of mean 1/μ1/\mu1/μ. An arrival that finds the server busy enters the orbit and stays there for an exponential time with mean 1/γ1/\gamma1/γ, after which it tries again; each customer in orbit retries independently. No customer leaves because of impatience. With Ns(t)∈{0,1}N_s(t) \in \{0,1\}Ns​(t)∈{0,1} the number in service and No(t)N_o(t)No​(t) the number in orbit, the pair is a continuous-time Markov chain on states {i,n}\{i, n\}{i,n}, i∈{0,1}i \in \{0,1\}i∈{0,1}, n∈{0,1,2,… }n \in \{0,1,2,\dots\}n∈{0,1,2,…}. Writing pi,np_{i,n}pi,n​ for the steady-state probability of {i,n}\{i,n\}{i,n}, the rate-balance equations are

(λ+nγ)p0,n=μp1,n,n≥0,(3.47)(λ+μ)p1,n=λp0,n+(n+1)γp0,n+1+λp1,n−1,n≥1,(3.48)(λ+μ)p1,0=λp0,0+γp0,1.(3.49)\begin{aligned} (\lambda + n\gamma)p_{0,n} &= \mu p_{1,n}, && n \ge 0, && (3.47)\\ (\lambda+\mu)p_{1,n} &= \lambda p_{0,n} + (n+1)\gamma p_{0,n+1} + \lambda p_{1,n-1}, && n \ge 1, && (3.48)\\ (\lambda+\mu)p_{1,0} &= \lambda p_{0,0} + \gamma p_{0,1}. && && (3.49) \end{aligned}(λ+nγ)p0,n​(λ+μ)p1,n​(λ+μ)p1,0​​=μp1,n​,=λp0,n​+(n+1)γp0,n+1​+λp1,n−1​,=λp0,0​+γp0,1​.​​n≥0,n≥1,​​(3.47)(3.48)(3.49)​

Following the book's convention (§1.9, and the footnote on p.118), a steady-state solution is a nonnegative solution of these equations whose total mass ∑n(p0,n+p1,n)\sum_n (p_{0,n} + p_{1,n})∑n​(p0,n​+p1,n​) equals 111. The traffic intensity is ρ=λ/μ\rho = \lambda/\muρ=λ/μ, and the partial generating functions are P0(z)=∑nznp0,nP_0(z) = \sum_n z^n p_{0,n}P0​(z)=∑n​znp0,n​ and P1(z)=∑nznp1,nP_1(z) = \sum_n z^n p_{1,n}P1​(z)=∑n​znp1,n​.

The other models of the mission use the same convention. In the bulk-input queue M[X]/M/1M^{[X]}/M/1M[X]/M/1, batches arrive at rate λ\lambdaλ with batch-size probabilities cn=Pr⁡{X=n}c_n = \Pr\{X = n\}cn​=Pr{X=n}, n≥1n \ge 1n≥1, and batch-size generating function C(z)=∑ncnznC(z) = \sum_n c_n z^nC(z)=∑n​cn​zn. In the partial-batch bulk-service queue M/M[K]/1M/M^{[K]}/1M/M[K]/1, single arrivals come at rate λ\lambdaλ and the server serves up to KKK customers together in an exponential time of mean 1/μ1/\mu1/μ. In the nonpreemptive priority queue there are rrr classes with rates λk\lambda_kλk​ and μk\mu_kμk​, loads ρk=λk/μk\rho_k = \lambda_k/\mu_kρk​=λk​/μk​ and cumulative loads σk=ρ1+⋯+ρk\sigma_k = \rho_1 + \cdots + \rho_kσk​=ρ1​+⋯+ρk​.

Formalization targets

Goal: the stationary distribution (3.57)

For λ,μ,γ>0\lambda, \mu, \gamma > 0λ,μ,γ>0 and ρ<1\rho < 1ρ<1, the numbers

p0,n=(1−ρ)(λ/γ)+1ρnn! γn∏i=0n−1(λ+iγ),p1,n=(1−ρ)(λ/γ)+1ρn+1n! γn∏i=1n(λ+iγ)p_{0,n} = (1-\rho)^{(\lambda/\gamma)+1}\frac{\rho^n}{n!\,\gamma^n}\prod_{i=0}^{n-1}(\lambda+i\gamma), \qquad p_{1,n} = (1-\rho)^{(\lambda/\gamma)+1}\frac{\rho^{n+1}}{n!\,\gamma^n}\prod_{i=1}^{n}(\lambda+i\gamma)p0,n​=(1−ρ)(λ/γ)+1n!γnρn​i=0∏n−1​(λ+iγ),p1,n​=(1−ρ)(λ/γ)+1n!γnρn+1​i=1∏n​(λ+iγ)

form a steady-state solution of (3.47)–(3.49), and every steady-state solution equals them.

Milestones on the retrial queue

The generating functions satisfy (3.50)–(3.52) on (−1,1)(-1,1)(−1,1), including the separable equation

P0′(z)=λργ(1−ρz)P0(z),P_0'(z) = \frac{\lambda\rho}{\gamma(1-\rho z)}P_0(z),P0′​(z)=γ(1−ρz)λρ​P0​(z),

their closed form is (3.55),

P0(z)=(1−ρz)(1−ρ1−ρz)(λ/γ)+1,P1(z)=ρ(1−ρ1−ρz)(λ/γ)+1,P_0(z) = (1-\rho z)\left(\frac{1-\rho}{1-\rho z}\right)^{(\lambda/\gamma)+1}, \qquad P_1(z) = \rho\left(\frac{1-\rho}{1-\rho z}\right)^{(\lambda/\gamma)+1},P0​(z)=(1−ρz)(1−ρz1−ρ​)(λ/γ)+1,P1​(z)=ρ(1−ρz1−ρ​)(λ/γ)+1,

and the mean orbit size is (3.58), Lo=ρ21−ρ⋅μ+γγL_o = \frac{\rho^2}{1-\rho}\cdot\frac{\mu+\gamma}{\gamma}Lo​=1−ρρ2​⋅γμ+γ​.

Milestones from the rest of Chapter 3

The bulk-input generating function (3.3), p0=1−ρp_0 = 1 - \rhop0​=1−ρ with ρ=λE[X]/μ\rho = \lambda\mathrm E[X]/\muρ=λE[X]/μ, and the mean (3.4); the unique root r0∈(0,1)r_0 \in (0,1)r0​∈(0,1) of μrK+1−(λ+μ)r+λ=0\mu r^{K+1} - (\lambda+\mu)r + \lambda = 0μrK+1−(λ+μ)r+λ=0 and the geometric law pn=(1−r0)r0np_n = (1-r_0)r_0^npn​=(1−r0​)r0n​ (3.9); and Cobham's formula (3.41)/(3.43), the unique solution of the linear system (3.40).

Significance

The closed form (3.57) makes every performance measure of the M/M/1M/M/1M/M/1 retrial queue explicit. The server is busy a fraction ρ\rhoρ of the time, exactly as without retrials. The mean orbit size (3.58) is the M/M/1M/M/1M/M/1 mean queue length multiplied by (μ+γ)/γ(\mu+\gamma)/\gamma(μ+γ)/γ, and the mean time in orbit (3.59) follows from Little's law. These formulas quantify the cost of retrials against an ordinary queue and are the reference case against which approximations for multi-server retrial systems are checked.

The results are classical and proved in the book, partly through exercises (Problems 3.39–3.41). None of them is formalized in any proof assistant, as far as the platform's catalogue shows: there is no retrial, bulk or priority queue on Prove2Me. The mission produces machine-checked statements and, once solved, proofs of the chapter's main closed forms. It also produces a small reusable layer: generating functions of probability sequences on the closed unit disc, and the "probability solution of the balance equations" pattern for chains with countable state spaces.

Difficulty

The derivation in the book is formal. It differentiates power series term by term, divides by 1−z1 - z1−z, integrates ln⁡P0\ln P_0lnP0​, and fixes the constant by setting z=1z = 1z=1, without justifying any of these steps. A formal proof has to show that the series converge and are differentiable on (−1,1)(-1,1)(−1,1), that the differential equation determines P0P_0P0​ up to a constant, and that the values at z=1z = 1z=1 are the limits of the values inside the disc (Abel's theorem). The uniqueness half of the goal is the hardest part. The book never proves it; it follows from the ODE argument only once every step is shown to hold for an arbitrary probability solution. Verifying that (3.57) solves (3.47)–(3.49) is only the easy half. The same pattern recurs in the bulk-input queue, where z=1z = 1z=1 is a removable singularity of (3.3). In the bulk-service queue the root r0r_0r0​ is only characterized as the unique root in (0,1)(0,1)(0,1), so existence and uniqueness of the root are part of the claim.

Formalization scope

A steady-state solution is a pair p0 p1 : ℕ → ℝ (resp. one sequence p : ℕ → ℝ) that is pointwise nonnegative, has total mass 111 as a HasSum, and solves the book's balance equations exactly as printed, global balance and not detailed balance. Every "the steady-state solution is X" is stated with both halves: X is a steady-state solution, and every steady-state solution equals X. Stating only that (3.57) solves (3.47)–(3.49), without normalization or uniqueness, would be a trivializing formalization. So would taking r0r_0r0​ as a given root in (3.9), or taking the Wq(i)W_q^{(i)}Wq(i)​ in (3.41) as numbers assumed to satisfy it. None of these is used. The closed forms instantiated are (3.52), (3.55), (3.57), (3.58), (3.3), (3.4), (3.9), (3.41) and (3.43), each written out in full, with the real power (1−ρ)(λ/γ)+1(1-\rho)^{(\lambda/\gamma)+1}(1−ρ)(λ/γ)+1 as Real.rpow.

The conventions are as follows. The retrial generating functions take real arguments, on (−1,1)(-1,1)(−1,1) for the differential equations and on [−1,1][-1,1][−1,1] for the closed form. The bulk-input generating function takes complex arguments with ∣z∣≤1|z| \le 1∣z∣≤1, z≠1z \ne 1z=1, because (3.3) is 0/00/00/0 at z=1z = 1z=1. The condition ρ<1\rho < 1ρ<1 is a hypothesis of every retrial statement. For the bulk-service queue the book's unnamed condition is stated as λ<Kμ\lambda < K\muλ<Kμ. For bulk input, E[X]<∞\mathrm E[X] < \inftyE[X]<∞ is assumed throughout, and the mean (3.4) is asserted under the further condition E[X2]<∞\mathrm E[X^2] < \inftyE[X2]<∞, which it requires. For Cobham's formula only the algebraic content is formalized; the mean-value argument that yields (3.40) and (3.42) is not.

Needed infrastructure: power series of summable nonnegative sequences on the closed unit disc (convergence, term-by-term differentiation, Abel continuity), the binomial series (1−x)−a=∑na(a+1)⋯(a+n−1)n!xn(1 - x)^{-a} = \sum_n \frac{a(a+1)\cdots(a+n-1)}{n!}x^n(1−x)−a=∑n​n!a(a+1)⋯(a+n−1)​xn for real aaa, and uniqueness of invariant probability vectors for irreducible chains. All of this is reusable beyond the mission. Proofs of any milestone, of the easy half of the goal, or of the needed series facts are welcome contributions.

Selected references

  • D. Gross, J. F. Shortle, J. M. Thompson, C. M. Harris, Fundamentals of Queueing Theory, 4th ed., Wiley, 2008, §§3.1, 3.2.0.1, 3.4.2, 3.5.1. https://doi.org/10.1002/9781118625651
  • G. I. Falin, J. G. C. Templeton, Retrial Queues, Chapman & Hall, 1997. https://doi.org/10.1007/978-1-4899-2977-8
  • J. R. Artalejo, Accessible bibliography on retrial queues, Mathematical and Computer Modelling 30 (1999) 1–6. https://doi.org/10.1016/S0895-7177(99)00128-4
  • A. Cobham, Priority assignment in waiting line problems, Journal of the Operations Research Society of America 2 (1954) 70–76. https://doi.org/10.1287/opre.2.1.70
10 thms3 active usersReviewed
🏆Completed
AnalysisMarkov ChainOperations Research+1·Captain: mikedeng1

Fundamentals of Queueing Theory III: The Transient M/M/1 Queue via Modified Bessel FunctionsTextbook

Motivation

Steady-state formulas describe a queue that has been running forever. Many practical questions are about a queue that has not: a call centre just after opening, a server just after a reset, a system under a burst of load. For these, the relevant quantity is the transient distribution pn(t)=Pr⁡{N(t)=n}p_n(t) = \Pr\{N(t) = n\}pn​(t)=Pr{N(t)=n} of the number N(t)N(t)N(t) in the system at a finite time ttt. It is also what determines how fast the steady state is approached, and it is needed for the busy period: the length of time a server stays busy once a customer arrives at an idle server.

For the single-server Markovian queue M/M/1 the transient distribution has an explicit closed form in modified Bessel functions. Its history is short and well documented. Ledermann and Reuter (1954) obtained it by spectral analysis of the birth–death process. Bailey (1954) found it by generating functions and Laplace transforms, and Champernowne (1956) by combinatorial methods. Bailey's route is the standard textbook derivation, and it is the one Gross, Shortle, Thompson and Harris outline in §2.11 of Fundamentals of Queueing Theory (4th ed., 2008). Abate and Whitt (1989) showed that computing with the resulting series is numerically delicate, which is one reason for having the formula pinned down exactly.

This mission formalizes §§2.11–2.12 of that book: the transient laws of M/M/1/1, M/M/1 and M/M/∞, and the M/M/1 busy period.

Setting

Customers arrive in a Poisson stream of rate λ>0\lambda > 0λ>0. Each service takes an exponential time of rate μ>0\mu > 0μ>0, and ρ=λ/μ\rho = \lambda/\muρ=λ/μ. The number in the system is a continuous-time Markov chain on {0,1,2,… }\{0, 1, 2, \dots\}{0,1,2,…}, and its state probabilities pn(t)p_n(t)pn​(t) satisfy the forward (differential–difference) equations. For M/M/1 started with N(0)=iN(0) = iN(0)=i they are, for t≥0t \ge 0t≥0,

pn′(t)=−(λ+μ)pn(t)+λpn−1(t)+μpn+1(t) (n>0),p0′(t)=−λp0(t)+μp1(t),(2.72)p_n'(t) = -(\lambda+\mu)p_n(t) + \lambda p_{n-1}(t) + \mu p_{n+1}(t)\ (n > 0), \qquad p_0'(t) = -\lambda p_0(t) + \mu p_1(t), \tag{2.72}pn′​(t)=−(λ+μ)pn​(t)+λpn−1​(t)+μpn+1​(t) (n>0),p0′​(t)=−λp0​(t)+μp1​(t),(2.72)

with pn(0)=1p_n(0) = 1pn​(0)=1 if n=in = in=i and 000 otherwise. The other systems are variants:

  • M/M/1/1, no waiting room: two states and equations (2.70).
  • M/M/∞, ample service: the death rate in state nnn is nμn\munμ, giving (2.76).
  • The busy-period system: (2.72) with 000 made absorbing (λ0=0\lambda_0 = 0λ0​=0) and N(0)=1N(0) = 1N(0)=1. Its p0(t)p_0(t)p0​(t) is the distribution function of the busy period TbpT_{bp}Tbp​.

A family (pn)(p_n)(pn​) solves a system on [0,∞)[0,\infty)[0,∞) when each pnp_npn​ has, at every t≥0t \ge 0t≥0, the prescribed derivative (a right derivative at t=0t = 0t=0). It is a probability solution when pn(t)≥0p_n(t) \ge 0pn​(t)≥0 and ∑npn(t)=1\sum_n p_n(t) = 1∑n​pn​(t)=1 for every t≥0t \ge 0t≥0. The modified Bessel function of the first kind is

In(y)=∑k=0∞(y/2)n+2kk! (n+k)!,I−n=In,I_n(y) = \sum_{k=0}^{\infty} \frac{(y/2)^{n+2k}}{k!\,(n+k)!}, \qquad I_{-n} = I_n,In​(y)=k=0∑∞​k!(n+k)!(y/2)n+2k​,I−n​=In​,

and the Laplace transform of fff is fˉ(s)=∫0∞e−stf(t) dt\bar f(s) = \int_0^\infty e^{-st} f(t)\,dtfˉ​(s)=∫0∞​e−stf(t)dt for Re⁡s>0\operatorname{Re} s > 0Res>0.

Formalization targets

Goal: the transient M/M/1 law, (2.75)

With y=2tλμy = 2t\sqrt{\lambda\mu}y=2tλμ​,

pn(t)=e−(λ+μ)t[ρ(n−i)/2In−i(y)+ρ(n−i−1)/2In+i+1(y)+(1−ρ)ρn∑j=n+i+2∞ρ−j/2Ij(y)].p_n(t) = e^{-(\lambda+\mu)t}\Big[\rho^{(n-i)/2} I_{n-i}(y) + \rho^{(n-i-1)/2} I_{n+i+1}(y) + (1-\rho)\rho^n \sum_{j=n+i+2}^{\infty} \rho^{-j/2} I_j(y)\Big].pn​(t)=e−(λ+μ)t[ρ(n−i)/2In−i​(y)+ρ(n−i−1)/2In+i+1​(y)+(1−ρ)ρnj=n+i+2∑∞​ρ−j/2Ij​(y)].

The goal asserts five things for every λ,μ>0\lambda, \mu > 0λ,μ>0 and every iii, with no restriction on ρ\rhoρ:

  1. the series converges;
  2. these functions solve (2.72);
  3. they meet the initial condition;
  4. they form a probability distribution at every ttt;
  5. they are the only probability solution.

Milestones

  1. (2.71): the M/M/1/1 solution p1(t)=λλ+μ(1−e−(λ+μ)t)+p1(0)e−(λ+μ)tp_1(t) = \frac{\lambda}{\lambda+\mu}(1-e^{-(\lambda+\mu)t}) + p_1(0)e^{-(\lambda+\mu)t}p1​(t)=λ+μλ​(1−e−(λ+μ)t)+p1​(0)e−(λ+μ)t, and the matching formula for p0p_0p0​.
  2. (2.74) and Rouché's theorem: for Re⁡s>0\operatorname{Re} s > 0Res>0, the quadratic (λ+μ+s)z−μ−λz2(\lambda+\mu+s)z - \mu - \lambda z^2(λ+μ+s)z−μ−λz2 has exactly one zero in ∣z∣<1|z| < 1∣z∣<1, namely z1=(λ+μ+s−(λ+μ+s)2−4λμ)/(2λ)z_1 = (\lambda+\mu+s-\sqrt{(\lambda+\mu+s)^2-4\lambda\mu})/(2\lambda)z1​=(λ+μ+s−(λ+μ+s)2−4λμ​)/(2λ).
  3. The transform of p0p_0p0​: pˉ0(s)=z1i+1/(μ(1−z1))\bar p_0(s) = z_1^{i+1}/(\mu(1-z_1))pˉ​0​(s)=z1i+1​/(μ(1−z1​)).
  4. The limit of (2.75): pn(t)→(1−ρ)ρnp_n(t) \to (1-\rho)\rho^npn​(t)→(1−ρ)ρn if ρ<1\rho < 1ρ<1, and pn(t)→0p_n(t) \to 0pn​(t)→0 if ρ≥1\rho \ge 1ρ≥1.
  5. (2.77), M/M/∞: started empty, pn(t)=a(t)ne−a(t)/n!p_n(t) = a(t)^n e^{-a(t)}/n!pn​(t)=a(t)ne−a(t)/n! with a(t)=(1−e−μt)λ/μa(t) = (1-e^{-\mu t})\lambda/\mua(t)=(1−e−μt)λ/μ. The statement says that this family solves (2.76), is the unique probability solution, and has generating function exp⁡((z−1)a(t))\exp((z-1)a(t))exp((z−1)a(t)).
  6. The busy-period transform: pˉ0(s)=2μ/(s[λ+μ+s+(λ+μ+s)2−4λμ])\bar p_0(s) = 2\mu/(s[\lambda+\mu+s+\sqrt{(\lambda+\mu+s)^2-4\lambda\mu}])pˉ​0​(s)=2μ/(s[λ+μ+s+(λ+μ+s)2−4λμ​]).
  7. The busy-period density: p0′(t)=μ/λ e−(λ+μ)tI1(2λμ t)/tp_0'(t) = \sqrt{\mu/\lambda}\,e^{-(\lambda+\mu)t} I_1(2\sqrt{\lambda\mu}\,t)/tp0′​(t)=μ/λ​e−(λ+μ)tI1​(2λμ​t)/t.
  8. (2.79): for λ<μ\lambda < \muλ<μ, E[Tbp]=1/(μ−λ)E[T_{bp}] = 1/(\mu-\lambda)E[Tbp​]=1/(μ−λ) and E[Tbc]=1/λ+1/(μ−λ)E[T_{bc}] = 1/\lambda + 1/(\mu-\lambda)E[Tbc​]=1/λ+1/(μ−λ).

Significance

The formula (2.75) is the exact finite-time law of the most basic queue. It gives the rate at which M/M/1 approaches equilibrium, and it gives the distribution of the queue under overload (ρ≥1\rho \ge 1ρ≥1), where no steady state exists. It is the reference against which numerical transient methods, such as the uniformization of Chapter 8 of the same book, are checked. The busy-period density and its mean (2.79) enter server-utilisation and vacation models, and the Laplace-transform method used here recurs in the M/G/1 analysis of Chapter 5.

All of these results are classical and proved in the literature. None of them is machine-checked, as far as the platform's catalogue and Mathlib show. The chain from a countable system of linear ODEs, through generating functions and a root-location argument, to a Bessel series is a standard pattern in applied probability, and a formal version of it is what this mission adds. The formal statements also make explicit what the book leaves implicit: the sense in which the equations hold at t=0t = 0t=0, and the class in which the solution is unique.

Difficulty

The forward equations (2.72) form an infinite linear system. The obvious approach is to treat it like a finite system of ODEs, whose solution is a matrix exponential, and read off (2.75). That fails for two reasons. The generator is an infinite matrix, so its exponential needs a functional-analytic setting. And uniqueness is not automatic for infinite systems: it needs a class, such as probability solutions, and an argument that works in that class.

The Bessel form is a second, independent difficulty. The transform pˉ0(s)\bar p_0(s)pˉ​0​(s) is fixed by a root-location argument in the complex plane. Inverting the transform, or verifying (2.75) directly, requires manipulating the three-term Bessel recurrence and exchanging infinite sums. The tail sum ∑jρ−j/2Ij\sum_{j} \rho^{-j/2} I_j∑j​ρ−j/2Ij​ has to be controlled uniformly enough to be differentiated term by term. For ρ≥1\rho \ge 1ρ≥1 the factor (1−ρ)(1-\rho)(1−ρ) is non-positive, so the nonnegativity of pn(t)p_n(t)pn​(t) is not visible from the formula.

Formalization scope

Conventions committed to:

  • Parameters. Rates are real with λ,μ>0\lambda, \mu > 0λ,μ>0, and ρ=λ/μ\rho = \lambda/\muρ=λ/μ. States are ℕ (Fin 2 for M/M/1/1).
  • Solutions. "Solves on [0,∞)[0,\infty)[0,∞)" is HasDerivWithinAt on Set.Ici 0 at every t≥0t \ge 0t≥0. Uniqueness is asserted among solutions that are probability distributions at every time.
  • Special functions. Half-integer powers of ρ\rhoρ are real powers, and I−m=ImI_{-m} = I_mI−m​=Im​ is part of the definition. Laplace transforms are complex Bochner integrals over (0,∞)(0,\infty)(0,∞), and each statement also asserts the integrability it needs. Square roots with positive real part are hypotheses r2=(λ+μ+s)2−4λμr^2 = (\lambda+\mu+s)^2 - 4\lambda\mur2=(λ+μ+s)2−4λμ, Re⁡r>0\operatorname{Re} r > 0Rer>0.

The closed forms stated exactly as in the book are:

  • (2.71);
  • z1z_1z1​ and z2z_2z2​ of (2.74);
  • pˉ0(s)=z1i+1/(μ(1−z1))\bar p_0(s) = z_1^{i+1}/(\mu(1-z_1))pˉ​0​(s)=z1i+1​/(μ(1−z1​));
  • (2.75), with the Bessel series of p.101;
  • the M/M/∞ law and (2.77);
  • the busy-period transform and density of p.102;
  • (2.79).

The book derives (2.79) by a steady-state ratio argument valid for M/G/1. Here it is stated for M/M/1, as the mean of the explicit density.

A statement of (2.75) that only asserts the right-hand side is well defined, or checks only n=0n = 0n=0, is ruled out: the goal requires the ODE system, the initial condition, the probability property and uniqueness. For the same reason, the M/M/∞ law is tied to the system (2.76) and does not reduce to a Taylor expansion.

Needed infrastructure that Mathlib lacks:

  • modified Bessel functions of integer order;
  • Laplace transforms;
  • a Rouché-type zero count or a direct root-location lemma;
  • uniqueness for countable linear ODE systems with bounded or linearly growing rates.

The Bessel and Laplace definitions, and the uniqueness lemma for birth–death forward equations, are reusable beyond this mission. Contributions of those as separate lemmas are welcome.

Selected references

  • D. Gross, J. F. Shortle, J. M. Thompson, C. M. Harris, Fundamentals of Queueing Theory, 4th ed., Wiley, 2008, §§2.11–2.12, pp.97–103. https://doi.org/10.1002/9781118625651
  • N. T. J. Bailey, "A continuous time treatment of a simple queue using generating functions", J. Royal Statistical Society B 16 (1954) 288–291. https://doi.org/10.1111/j.2517-6161.1954.tb00172.x
  • W. Ledermann, G. E. H. Reuter, "Spectral theory for the differential equations of simple birth and death processes", Phil. Trans. Royal Society A 246 (1954) 321–369. https://doi.org/10.1098/rsta.1954.0001
  • D. G. Champernowne, "An elementary method of solution of the queueing problem with a single server and constant parameters", J. Royal Statistical Society B 18 (1956) 125–128. https://doi.org/10.1111/j.2517-6161.1956.tb00217.x
  • J. Abate, W. Whitt, "Calculating time-dependent performance measures for the M/M/1 queue", IEEE Trans. Communications 37 (1989) 1102–1104. https://doi.org/10.1109/26.41165
15 thms3 active usersReviewed
🏆Completed
Markov ChainOperations ResearchStochastic Systems·Captain: mikedeng1

Fundamentals of Queueing Theory II: Erlang's Formulas and the Halfin–Whitt Square-Root Staffing LawTextbook

Why birth–death queues and Erlang's formulas

Every call center, hospital ward, cloud server pool and telephone exchange that is sized by formula is sized by one of a handful of explicit expressions from Markovian queueing theory. The two oldest are A. K. Erlang's: the Erlang-B (loss) formula of 1917, which gives the fraction of calls lost when ccc trunks carry an offered load of rrr erlangs, and the Erlang-C formula, which gives the probability that a customer of a ccc-server queue must wait. Both are still the default dimensioning rules of telecommunications and call-center workforce management (Gans, Koole & Mandelbaum 2003).

This mission formalizes Chapter 2, §§2.1–2.10, of Gross, Shortle, Thompson and Harris, Fundamentals of Queueing Theory, 4th ed. (Wiley 2008), which derives these formulas from a single result about birth–death processes and closes with the modern answer to the staffing question.

Timeline. Erlang (1917) obtained the loss formula; Vaulot (1927), Pollaczek (1932), Palm (1938) and Kosten (1948) completed its proof for general service times. Halfin and Whitt (1981) showed that in the M/M/nM/M/nM/M/n queue the delay probability converges to a limit strictly between 000 and 111 exactly when the number of servers exceeds the offered load by an amount of order n\sqrt nn​. This is the quality-and-efficiency-driven (QED) regime on which square-root staffing rests.

Setting

A birth–death process is a continuous-time Markov chain on the states n∈{0,1,2,… }n \in \{0, 1, 2, \dots\}n∈{0,1,2,…} that moves from nnn to n+1n+1n+1 at rate λn≥0\lambda_n \ge 0λn​≥0 (a birth, or arrival) and, for n≥1n \ge 1n≥1, from nnn to n−1n-1n−1 at rate μn>0\mu_n > 0μn​>0 (a death, or departure). A steady-state solution is a probability sequence {pn}\{p_n\}{pn​} (pn≥0p_n \ge 0pn​≥0, ∑npn=1\sum_n p_n = 1∑n​pn​=1) solving the global balance equations (2.1):

(λn+μn)pn=λn−1pn−1+μn+1pn+1 (n≥1),λ0p0=μ1p1.(\lambda_n + \mu_n)p_n = \lambda_{n-1}p_{n-1} + \mu_{n+1}p_{n+1}\ (n \ge 1), \qquad \lambda_0 p_0 = \mu_1 p_1.(λn​+μn​)pn​=λn−1​pn−1​+μn+1​pn+1​ (n≥1),λ0​p0​=μ1​p1​.

The queues of the chapter are birth–death processes with particular rates. The M/M/1M/M/1M/M/1 queue has λn=λ\lambda_n = \lambdaλn​=λ, μn=μ\mu_n = \muμn​=μ and traffic intensity ρ=λ/μ\rho = \lambda/\muρ=λ/μ. The M/M/cM/M/cM/M/c queue has λn=λ\lambda_n = \lambdaλn​=λ, μn=min⁡(n,c)μ\mu_n = \min(n, c)\muμn​=min(n,c)μ (2.30), offered load r=λ/μr = \lambda/\mur=λ/μ and ρ=r/c\rho = r/cρ=r/c. The M/M/c/cM/M/c/cM/M/c/c loss system is the same with λn=0\lambda_n = 0λn​=0 for n≥cn \ge cn≥c. The M/M/∞M/M/\inftyM/M/∞ queue has μn=nμ\mu_n = n\muμn​=nμ.

The explicit functions are the Erlang-B formula

B(c,r)=rc/c!∑i=0cri/i!,B(c, r) = \frac{r^c/c!}{\sum_{i=0}^{c} r^i/i!},B(c,r)=∑i=0c​ri/i!rc/c!​,

the Erlang-C formula, defined for ρ=r/c<1\rho = r/c < 1ρ=r/c<1,

C(c,r)=rc/(c!(1−ρ))rc/(c!(1−ρ))+∑n=0c−1rn/n!,C(c, r) = \frac{r^c/(c!(1-\rho))}{r^c/(c!(1-\rho)) + \sum_{n=0}^{c-1} r^n/n!},C(c,r)=rc/(c!(1−ρ))+∑n=0c−1​rn/n!rc/(c!(1−ρ))​,

and, with ϕ\phiϕ, Φ\PhiΦ the standard normal density and distribution function,

α(β)=ϕ(β)ϕ(β)+βΦ(β).\alpha(\beta) = \frac{\phi(\beta)}{\phi(\beta) + \beta\Phi(\beta)}.α(β)=ϕ(β)+βΦ(β)ϕ(β)​.

Formalization targets

Goal: the Halfin–Whitt theorem (§2.4, p.75)

For offered loads 0<rn<n0 < r_n < n0<rn​<n,

lim⁡n→∞C(n,rn)=α∈(0,1)  ⟺  lim⁡n→∞n−rnn=β>0,α=α(β).\lim_{n\to\infty} C(n, r_n) = \alpha \in (0,1) \iff \lim_{n\to\infty} \frac{n - r_n}{\sqrt n} = \beta > 0, \qquad \alpha = \alpha(\beta).n→∞lim​C(n,rn​)=α∈(0,1)⟺n→∞lim​n​n−rn​​=β>0,α=α(β).

It is stated as three facts: α\alphaα maps (0,∞)(0, \infty)(0,∞) into (0,1)(0, 1)(0,1); each α∈(0,1)\alpha \in (0, 1)α∈(0,1) has exactly one preimage β>0\beta > 0β>0; and for every β>0\beta > 0β>0 the two limits are equivalent.

Milestones

  1. (2.3)–(2.4): the steady-state solution of a general birth–death process, pn=p0∏i=1nλi−1/μip_n = p_0\prod_{i=1}^n \lambda_{i-1}/\mu_ipn​=p0​∏i=1n​λi−1​/μi​, and its existence if and only if 1+∑n≥1∏i=1nλi−1/μi<∞1 + \sum_{n\ge1}\prod_{i=1}^n \lambda_{i-1}/\mu_i < \infty1+∑n≥1​∏i=1n​λi−1​/μi​<∞.
  2. (2.9): M/M/1M/M/1M/M/1, pn=(1−ρ)ρnp_n = (1-\rho)\rho^npn​=(1−ρ)ρn, existing iff ρ<1\rho < 1ρ<1.
  3. (2.31)–(2.32): the M/M/cM/M/cM/M/c law, existing iff λ/(cμ)<1\lambda/(c\mu) < 1λ/(cμ)<1.
  4. (2.33): Lq=rcρ p0/(c!(1−ρ)2)L_q = r^c\rho\,p_0/(c!(1-\rho)^2)Lq​=rcρp0​/(c!(1−ρ)2).
  5. (2.37)–(2.38): 1−∑n<cpn=C(c,r)1 - \sum_{n<c} p_n = C(c, r)1−∑n<c​pn​=C(c,r).
  6. (2.52)–(2.53): the M/M/c/cM/M/c/cM/M/c/c law and pc=B(c,r)p_c = B(c, r)pc​=B(c,r).
  7. (2.54): B(c,r)=rB(c−1,r)/(c+rB(c−1,r))B(c, r) = rB(c-1, r)/(c + rB(c-1, r))B(c,r)=rB(c−1,r)/(c+rB(c−1,r)), B(0,r)=1B(0, r) = 1B(0,r)=1.
  8. (2.55): C(c,r)=cB(c,r)/(c−r+rB(c,r))C(c, r) = cB(c, r)/(c - r + rB(c, r))C(c,r)=cB(c,r)/(c−r+rB(c,r)).
  9. (2.57): M/M/∞M/M/\inftyM/M/∞, pn=rne−r/n!p_n = r^n e^{-r}/n!pn​=rne−r/n!.

Significance

The results. Items 1–9 are the working formulas of Markovian capacity planning: a stationary law for each basic model and the measures read off from it. (2.54) and (2.55) are how BBB and CCC are computed in practice, since the factorials of the closed forms overflow for c>170c > 170c>170. The Halfin–Whitt theorem is the reason the rule c≈r+βrc \approx r + \beta\sqrt rc≈r+βr​ holds a fixed service level, and it is the entry point to the QED heavy-traffic literature (diffusion limits of many-server queues, Garnett–Mandelbaum–Reiman, Gamarnik–Momčilović).

Formalizing them. All results are classical and proved in the literature. The book states the Halfin–Whitt theorem without proof, and (2.54)–(2.55) are left to exercises. The formalization would supply machine-checked versions of the Erlang identities and of the Halfin–Whitt limit theorem. No Lean development of either was found on the platform when this mission was drafted. A related Erlang-B statement from Kelly and Yudovina is on the platform, stated with detailed balance on a finite state space.

Difficulty

The stationary laws are induction plus geometric and exponential series, and the Erlang identities are finite algebra. The difficulty is concentrated in the goal. C(n,rn)C(n, r_n)C(n,rn​) is a ratio of a Poisson-type tail to a truncated exponential sum in which both nnn and rnr_nrn​ grow. The naive route, substituting Stirling's formula term by term, fails: the sums have Θ(n)\Theta(\sqrt n)Θ(n​) significant terms, each of relative size exp⁡(−k2/2n)\exp(-k^2/2n)exp(−k2/2n), and the error has to be controlled uniformly over them. The converse direction also requires showing that α(⋅)\alpha(\cdot)α(⋅) is strictly monotone. Without that, convergence of C(n,rn)C(n, r_n)C(n,rn​) does not force convergence of (n−rn)/n(n - r_n)/\sqrt n(n−rn​)/n​.

Formalization scope

Rates are real sequences indexed by N\mathbb NN, and a steady-state solution is a real sequence with HasSum p 1, nonnegative entries, and the balance equations (2.1) exactly as printed (global balance, not detailed balance). Every "the steady-state solution is X" is stated in both halves: X is a steady-state solution, and every steady-state solution equals X; the book's existence conditions (ρ<1\rho < 1ρ<1, λ/(cμ)<1\lambda/(c\mu) < 1λ/(cμ)<1, convergence of the series) are part of the statements. The M/M/c/cM/M/c/cM/M/c/c system is the N\mathbb NN-indexed process with λn=0\lambda_n = 0λn​=0 for n≥cn \ge cn≥c, as §2.5 sets it up; the statement records that states above ccc carry no mass.

The closed forms that are fixed in Lean: ∏i=1nλi−1/μi\prod_{i=1}^n \lambda_{i-1}/\mu_i∏i=1n​λi−1​/μi​ over Finset.Icc 1 n; B(c,r)B(c, r)B(c,r) and C(c,r)C(c, r)C(c,r) exactly as displayed above; ϕ\phiϕ = gaussianPDFReal 0 1, Φ\PhiΦ = the CDF of gaussianReal 0 1; Wq(0)=∑n=0c−1pnW_q(0) = \sum_{n=0}^{c-1} p_nWq​(0)=∑n=0c−1​pn​, as evaluated on p.69; Lq=∑n>c(n−c)pnL_q = \sum_{n > c}(n - c)p_nLq​=∑n>c​(n−c)pn​ as a convergent series.

C(c,r)C(c, r)C(c,r) is a total function in Lean, but its value for r≥cr \ge cr≥c carries no meaning. The goal assumes 0<rn<n0 < r_n < n0<rn​<n for n≥1n \ge 1n≥1, the book's standing condition ρ<1\rho < 1ρ<1. A statement about some other function with the same limiting behaviour, or with BBB and CCC left abstract, would not be this mission. Neither would one-directional or existence-only versions of the stationary laws.

Not included: the waiting-time distributions (2.28) and (2.39), which need an FCFS waiting-time model with arrival-point probabilities; the M/M/c/KM/M/c/KM/M/c/K measures (2.45)–(2.48); finite-source and state-dependent models (§§2.8–2.10). Useful contributions beyond the milestones are Poisson tail estimates at the n\sqrt nn​ scale and monotonicity of α(β)\alpha(\beta)α(β). Both are reusable in other many-server heavy-traffic statements.

Selected references

  • D. Gross, J. F. Shortle, J. M. Thompson, C. M. Harris, Fundamentals of Queueing Theory, 4th ed., Wiley, 2008. https://doi.org/10.1002/9781118625651
  • S. Halfin, W. Whitt, Heavy-traffic limits for queues with many exponential servers, Operations Research 29(3), 567–588, 1981. https://doi.org/10.1287/opre.29.3.567
  • N. Gans, G. Koole, A. Mandelbaum, Telephone call centers: tutorial, review, and research prospects, Manufacturing & Service Operations Management 5(2), 79–141, 2003. https://doi.org/10.1287/msom.5.2.79.16071
  • A. K. Erlang, Solution of some problems in the theory of probabilities of significance in automatic telephone exchanges, Elektroteknikeren 13, 1917 (English translation in The Life and Works of A. K. Erlang, 1948).
  • F. P. Kelly, E. Yudovina, Stochastic Networks, Cambridge University Press, 2014. https://doi.org/10.1017/CBO9781139565363
12 thms3 active usersReviewed
🏆Completed
Algorithmic Game TheoryLinear OptimizationMechanism Design+1·Captain: mikedeng1

An Introduction to the Theory of Mechanism Design VII: Crémer–McLean Full Surplus Extraction with Correlated TypesTextbook

Motivation

Bayesian mechanism design asks which collective decisions and payments a designer can implement when every agent holds private information, the type, drawn from a commonly known prior. Most of the classical theory (Myerson's optimal auction, the Myerson–Satterthwaite impossibility, the public goods results) assumes that types are independent. Once types are correlated, as they are when bidders' values share a common component, the theory changes in a way that is best read as a paradox: Crémer and McLean (Econometrica 1988) showed that a designer can then extract the entire surplus, leaving agents no information rents. This mission formalizes Chapter 6 of Börgers, An Introduction to the Theory of Mechanism Design (Oxford University Press 2015), which sets up Bayesian mechanism design in general, treats independent types, and proves the Crémer–McLean theorem, together with the one numbered result of Chapter 9, the impossibility theorem of Jehiel and Moldovanu for interdependent values.

Timeline of the results formalized here:

  • Rochet (1987) characterized implementable decision rules by cyclical monotonicity; Proposition 6.1 is its interim version for independent types.
  • Crémer and McLean (1988): under their condition on the prior, every direct mechanism can be made Bayesian incentive-compatible without changing its decision rule or interim payments (Proposition 6.4).
  • Krishna and Maenner (2001): revenue equivalence for convex type sets and convex utilities (Proposition 6.2).
  • Jehiel and Moldovanu (2001): with interdependent values, efficient decisions are generically not Bayesian implementable (Proposition 9.1).
  • Kosenok and Severinov (2008): an identifiability condition added to Crémer–McLean gives ex post budget balance as well (Proposition 6.6).

Setting

There are finitely many agents i∈Ii \in Ii∈I and a set AAA of alternatives. Agent iii has a type θi∈Θi\theta_i \in \Theta_iθi​∈Θi​ and utility ui(a,θi)−tiu_i(a,\theta_i) - t_iui​(a,θi​)−ti​ from alternative aaa and payment tit_iti​. Types θ=(θ1,…,θN)∈Θ=∏iΘi\theta = (\theta_1,\dots,\theta_N) \in \Theta = \prod_i \Theta_iθ=(θ1​,…,θN​)∈Θ=∏i​Θi​ are drawn from a common prior μ\muμ; μ(⋅∣θi)\mu(\cdot\mid\theta_i)μ(⋅∣θi​) is the conditional distribution of the others' types θ−i\theta_{-i}θ−i​ given θi\theta_iθi​. Types are independent if μ(⋅∣θi)\mu(\cdot\mid\theta_i)μ(⋅∣θi​) does not depend on θi\theta_iθi​.

A direct mechanism (q,t1,…,tN)(q, t_1,\dots,t_N)(q,t1​,…,tN​) is a decision rule q:Θ→Aq:\Theta\to Aq:Θ→A and payment rules ti:Θ→Rt_i:\Theta\to\mathbb Rti​:Θ→R. It is Bayesian incentive-compatible (BIC) if for every iii and all θi,θi′\theta_i,\theta_i'θi​,θi′​,

∫Θ−iui(q(θi,θ−i),θi)−ti(θi,θ−i) dμ(θ−i∣θi) ≥ ∫Θ−iui(q(θi′,θ−i),θi)−ti(θi′,θ−i) dμ(θ−i∣θi).\int_{\Theta_{-i}} u_i(q(\theta_i,\theta_{-i}),\theta_i) - t_i(\theta_i,\theta_{-i})\,d\mu(\theta_{-i}\mid\theta_i) \ \ge\ \int_{\Theta_{-i}} u_i(q(\theta_i',\theta_{-i}),\theta_i) - t_i(\theta_i',\theta_{-i})\,d\mu(\theta_{-i}\mid\theta_i).∫Θ−i​​ui​(q(θi​,θ−i​),θi​)−ti​(θi​,θ−i​)dμ(θ−i​∣θi​) ≥ ∫Θ−i​​ui​(q(θi′​,θ−i​),θi​)−ti​(θi′​,θ−i​)dμ(θ−i​∣θi​).

The interim decision rule Qi(θi)Q_i(\theta_i)Qi​(θi​) is the distribution of q(θi,θ−i)q(\theta_i,\theta_{-i})q(θi​,θ−i​) given θi\theta_iθi​, and the interim expected payment is Ti(θi)=∫ti(θi,θ−i) dμ(θ−i∣θi)T_i(\theta_i) = \int t_i(\theta_i,\theta_{-i})\,d\mu(\theta_{-i}\mid\theta_i)Ti​(θi​)=∫ti​(θi​,θ−i​)dμ(θ−i​∣θi​). A mechanism is ex post budget balanced if ∑iti(θ)=0\sum_i t_i(\theta) = 0∑i​ti​(θ)=0 for every θ\thetaθ, and ex ante budget balanced if ∫Θ∑iti dμ=0\int_\Theta \sum_i t_i\,d\mu = 0∫Θ​∑i​ti​dμ=0.

In §6.4 every Θi\Theta_iΘi​ is finite and μ(θ)>0\mu(\theta) > 0μ(θ)>0 for every θ\thetaθ. The prior satisfies the Crémer–McLean condition if for no agent iii and type θi\theta_iθi​ there are weights λ≥0\lambda \ge 0λ≥0 on Θi∖{θi}\Theta_i\setminus\{\theta_i\}Θi​∖{θi​} with

μ(θ−i∣θi)=∑θi′≠θiλ(θi′) μ(θ−i∣θi′)for all θ−i.\mu(\theta_{-i}\mid\theta_i) = \sum_{\theta_i'\ne\theta_i}\lambda(\theta_i')\,\mu(\theta_{-i}\mid\theta_i')\quad\text{for all }\theta_{-i}.μ(θ−i​∣θi​)=θi′​=θi​∑​λ(θi′​)μ(θ−i​∣θi′​)for all θ−i​.

Identifiability requires that for every full-support distribution ν≠μ\nu\ne\muν=μ some agent's type θi\theta_iθi​ has a belief ν(⋅∣θi)\nu(\cdot\mid\theta_i)ν(⋅∣θi​) that is not a nonnegative combination of the beliefs μ(⋅∣θi′)\mu(\cdot\mid\theta_i')μ(⋅∣θi′​).

Formalization targets

Goal: Crémer–McLean (Proposition 6.4)

If μ\muμ satisfies the Crémer–McLean condition, then for every direct mechanism (q,t)(q,t)(q,t) there is a BIC direct mechanism (q,t′)(q,t')(q,t′) with

∑θ−iti(θi,θ−i) μ(θ−i∣θi)=∑θ−iti′(θi,θ−i) μ(θ−i∣θi)for all i,θi.\sum_{\theta_{-i}} t_i(\theta_i,\theta_{-i})\,\mu(\theta_{-i}\mid\theta_i) = \sum_{\theta_{-i}} t_i'(\theta_i,\theta_{-i})\,\mu(\theta_{-i}\mid\theta_i)\quad\text{for all } i,\theta_i.θ−i​∑​ti​(θi​,θ−i​)μ(θ−i​∣θi​)=θ−i​∑​ti′​(θi​,θ−i​)μ(θ−i​∣θi​)for all i,θi​.

Milestones

  1. Proposition 6.1 (independent types): qqq is part of a BIC mechanism iff it is interim cyclically monotone, ∑κ=1k−1(∫Aui(a,θiκ+1) dQi(θiκ)−∫Aui(a,θiκ) dQi(θiκ))≤0\sum_{\kappa=1}^{k-1}\big(\int_A u_i(a,\theta_i^{\kappa+1})\,dQ_i(\theta_i^\kappa) - \int_A u_i(a,\theta_i^\kappa)\,dQ_i(\theta_i^\kappa)\big)\le 0∑κ=1k−1​(∫A​ui​(a,θiκ+1​)dQi​(θiκ​)−∫A​ui​(a,θiκ​)dQi​(θiκ​))≤0 for every cycle θik=θi1\theta_i^k = \theta_i^1θik​=θi1​.
  2. Proposition 6.2 (independent types, convex type sets, convex utilities): two BIC mechanisms with Qi′=QiQ_i' = Q_iQi′​=Qi​ have Ti′=Ti+τiT_i' = T_i + \tau_iTi′​=Ti​+τi​.
  3. Proposition 6.3 (independent types): every ex ante budget balanced mechanism has an equivalent ex post budget balanced one.
  4. Proposition 6.5 (Farkas's alternative), already proved on the platform as Polyhedral.farkas_lemma.
  5. Proposition 6.6 (Kosenok–Severinov): under Crémer–McLean and identifiability, every ex ante budget balanced mechanism has an equivalent BIC and ex post budget balanced one.
  6. Proposition 9.1 (Jehiel–Moldovanu): in the linear interdependent-values model, under a regularity condition on first best rules and the weight condition αaii/αbii≠∑jαaji/∑jαbji\alpha^i_{ai}/\alpha^i_{bi}\ne\sum_j\alpha^i_{aj}/\sum_j\alpha^i_{bj}αaii​/αbii​=∑j​αaji​/∑j​αbji​ for some i,a,bi,a,bi,a,b, no first best direct mechanism is BIC. It uses its own model (§9.3) and is not on the goal's proof path.

Significance

The Crémer–McLean theorem says that, with correlated finite types, incentive compatibility imposes essentially no constraint: every decision rule and every interim payment rule can be implemented. In a single-unit auction this gives full surplus extraction. The result is the benchmark against which the literature on risk aversion, limited liability, collusion and the genericity of priors (Robert 1991, Laffont–Martimort 2000, Heifetz–Neeman 2006) measures its departures, and Proposition 6.6 extends it to budget-balanced mechanisms, which is what bilateral trade and public goods applications need. Propositions 6.1–6.3 are the independent-types counterpart that the correlated case breaks: they show why revenue equivalence and the ex ante/ex post budget-balance equivalence hold there and fail here. Proposition 9.1 shows the opposite failure, for interdependent values, where efficient decisions cannot be implemented even without participation or budget constraints.

All results are proved in the literature (Kosenok–Severinov's proof is omitted in the book). Apart from Farkas's alternative (Proposition 6.5), which is proved on the platform, none of them is formalized in Mathlib or on the platform; the platform's other mechanism design results (dominant-strategy results in a valuation model, revenue equivalence for symmetric independent auctions) do not cover correlated types.

Difficulty

For the goal the difficulty is the uniformity of the construction: a single payment adjustment must make truth-telling optimal against every possible deviation of every type of every agent, while leaving each type's expected payment unchanged. The obvious scoring-rule adjustment, charging −ln⁡μ(θ−i∣θi′)-\ln\mu(\theta_{-i}\mid\theta_i')−lnμ(θ−i​∣θi′​), changes interim payments, and removing that change is where the Crémer–McLean condition enters. For Proposition 6.2 the envelope argument must handle convex type sets that are not open and utilities that are only convex, not differentiable. For Proposition 9.1 the obvious argument differentiates interim utility twice; the proposition does not assume that interim utility is twice differentiable, so that regularity has to be derived from the hypotheses on the interim probabilities.

Formalization scope

Three definition files carry the three models. Independent types (§6.2–6.3): type sets are arbitrary measurable spaces, the prior is the product of probability measures ρi\rho_iρi​, and interim quantities are integrals against it. Finite correlated types (§6.4): finite type sets, a prior μ:Θ→R\mu:\Theta\to\mathbb Rμ:Θ→R with μ(θ)>0\mu(\theta)>0μ(θ)>0 and ∑θμ(θ)=1\sum_\theta\mu(\theta)=1∑θ​μ(θ)=1, and conditional beliefs μ(θ−i∣θi)=μ(θ)/μ(θi)\mu(\theta_{-i}\mid\theta_i) = \mu(\theta)/\mu(\theta_i)μ(θ−i​∣θi​)=μ(θ)/μ(θi​); the alternative set AAA is arbitrary. Interdependent values (§9.3): finite AAA, signals in [0,1]A[0,1]^A[0,1]A with positive densities, independent across agents, linear utilities with nonzero weights αaij\alpha^j_{ai}αaij​.

Committed conventions and explicit formulas:

  • The Crémer–McLean condition uses nonnegative weights without a sum-to-one constraint, as Definition 6.7 prints it.
  • "Equivalent" in Propositions 6.4 and 6.6 means the same decision rule and the same interim expected payments at truthful reports, as Proposition 6.4 states; in Proposition 6.3 it is the report-by-report notion, which under independence is equality of the interim payment rules TiT_iTi​.
  • Proposition 6.2 adds continuity of ui(a,⋅)u_i(a,\cdot)ui​(a,⋅) on Θi\Theta_iΘi​, and Proposition 6.3 adds at least two agents; without them the printed statements are false.
  • Measurability, which the book omits throughout, is made explicit: decision rules are measurable, and the payment sections and utilities are integrable against the relevant interim distributions.
  • In Proposition 9.1 the partial derivatives are derivatives within the closed cube [0,1]K[0,1]^K[0,1]K, and the weight condition is the displayed ratio inequality.

A trivializing formalization of the goal, one that proves it only for mechanisms that are already incentive compatible, or with a payment rule that is not a function of the reported type profile, or with "equivalent" weakened to "some BIC mechanism exists", is ruled out: the statement quantifies over every direct mechanism and fixes both the decision rule and every interim payment.

Reusable infrastructure: finite conditional expectations under a full-support prior, the Crémer–McLean and identifiability conditions, and the interim model with product priors. Proofs of any milestone and of the goal, including via the Farkas reference, are welcome.

Selected references

  • T. Börgers, An Introduction to the Theory of Mechanism Design, Oxford University Press, 2015. https://doi.org/10.1093/acprof:oso/9780199734023.001.0001
  • J. Crémer and R. P. McLean, "Full extraction of the surplus in Bayesian and dominant strategy auctions", Econometrica 56(6), 1988. https://doi.org/10.2307/1913096
  • G. Kosenok and S. Severinov, "Individually rational, budget-balanced mechanisms and allocation of surplus", Journal of Economic Theory 140(1), 2008.
  • P. Jehiel and B. Moldovanu, "Efficient design with interdependent valuations", Econometrica 69(5), 2001. https://doi.org/10.1111/1468-0262.00237
  • V. Krishna and E. Maenner, "Convex potentials with an application to mechanism design", Econometrica 69(4), 2001. https://doi.org/10.1111/1468-0262.00225
  • J.-C. Rochet, "A necessary and sufficient condition for rationalizability in a quasi-linear context", Journal of Mathematical Economics 16(2), 1987. https://doi.org/10.1016/0304-4068(87)90007-3
10 thms3 active usersReviewed
🏆Completed
Operations ResearchStatisticsStochastic Systems·Captain: mikedeng1

Elements of Queueing Theory V: Strassen's Theorems and the Stochastic Ordering of QueuesTextbook

Strassen's Theorems and the Stochastic Ordering of Queues

Background

Chapters 1–3 of Baccelli and Brémaud's Elements of Queueing Theory compute exact quantities: Palm identities, stability criteria, PASTA, Pollaczek–Khinchin. Chapter 4 asks a different question. When you cannot compute a queue, can you at least say it is better than another one?

That requires an order on distributions. The chapter builds a family of them — integral orders — by choosing a class ℒ of test functions and declaring F ≤_ℒ G when ∫f dF ≤ ∫f dG for all f ∈ ℒ. Three matter: {i} the non-decreasing functions, giving the strong (stochastic) order; {cx} the convex functions, giving the convex order; and their intersection {icx}.

The goal

An integral order compares two distributions that need not live on the same probability space, and that is both its convenience and its difficulty. Strassen's theorems say each of these orders is secretly a statement about a coupling.

Theorem 4.2.2 (p.278), Strassen's ≤_cx theorem:

F ≤_cx G   ⟺   ∃ X ~ F, Y ~ G on one space with  E[Y | X] = X  a.s.
F ≤_icx G  ⟺   the same with  E[Y | X] ≥ X  a.s.

The convex order holds exactly when G is a martingale dilation of F — obtained by spreading each point out without moving its conditional mean. That is what makes the order usable: comparison results for queues become induction arguments on a coupling instead of analytic manipulations of convolutions of c.d.f.'s.

Its companion Theorem 4.2.1 is the ≤_st version, where the coupling is the simpler X ≤ Y a.s. In dimension one both are explicit — take X = F⁻¹(U), Y = G⁻¹(U) for a uniform U. In dimension n there is no such formula, and that is why these are Strassen's theorems. The book attributes both to Strassen (1965) and proves neither.

Why FIFO is optimal

§4.1 is a different kind of comparison: not between two queues, but between two service disciplines for the same queue. The order there is majorization ≺, which compares how spread out two vectors of the same total are.

The answer is that FIFO minimizes E⁰[f(V)] for every convex f (Property 4.1.3), and the proof is an interchange argument. Under any non-preemptive discipline that uses no information on the service times, customer k effectively receives service σ_{γ(k)} for some permutation γ; Lemma 4.1.3 shows the same queue is produced by FIFO fed with that reordered input, and that the reordering does not change the law of the input. Lemma 4.1.4 passes to the limit, which needs ρ < 1. Lemmas 4.1.1 and 4.1.2 then do the combinatorics: undoing one inversion of γ makes the waiting-time vector less spread out, so the identity permutation — FIFO — is extremal.

Feller's paradox, and what survives it

§4.4 compares time-stationary queues, and opens with a warning. T_n[P⁰] ≤_i T̃_n[P̃⁰] for every n does not imply T_n[P] ≤_i T̃_n[P̃]: Example 4.4.1, "Feller's paradox revisited", exhibits a Poisson process and a renewal process where the Palm order holds and the stationary one fails. The order does not pass from the Palm probability to the stationary one.

For ≤_cx it does. Lemma 4.4.1 is why: it expands E_P[f(N[0,x))] as a series of second differences of f against Palm expectations, and a convex f makes every coefficient non-negative. Lemma 4.4.2 handles the S-orders, built by dividing Palm integrals by the mean cycle length, and shows that the normalisation does not hide the comparison it normalises by.

Formalization scope

  • Orders. ≤_i, ≤_cx, ≤_icx on distributions on ℝⁿ are integral orders over the book's test classes (§4.2.1), with the page's qualification that only test functions with well-defined integrals count. Majorization ≺ is (4.1.2) with increasing reorderings of both vectors.
  • Strassen. Both theorems are stated as equivalences, with the coupling existential over the probability space. Theorem 4.2.2 carries both clauses — E[Y | X] = X for ≤_cx, E[Y | X] ≥ X for ≤_icx, as conditional expectations given σ(X) — and assumes both distributions integrable; Theorem 4.2.1 has no integrability hypothesis. A one-directional statement (the Jensen half) is not the theorem.
  • The queue of §4.1.3 is constructed: a GI/GI input (i.i.d. inter-arrival and service times, independent), a single work-conserving server started empty, and any non-preemptive discipline whose choices are measurable in the information the book's σ-field 𝒢_t carries (arrivals, service times of customers already started) plus external randomisation. FIFO is one such discipline. The interchange permutations γ_n and their limit γ are built from the schedule as on pp.268–270; Lemma 4.1.4 assumes ρ = E[σ₀]/E[τ₀] < 1.
  • Lemma 4.4.1 is stated with the exact second-difference series and assumes that series converges absolutely; the page states it for all f, which fails for heavy-tailed counts and sparse f.
  • The S-orders test against {I-ℒ} — primitives ∫_0^t f(u, x) du of test functions — and apply only to distributions whose first coordinate is a.s. positive with a finite mean.

What this mission provides

None of it exists. Mathlib has no stochastic order, no convex order, no increasing-convex order, no majorization, no Schur-convexity and no Strassen theorem; the platform returns zero hits for q=stochastic ordering. Everything in this chapter is new substrate — and §§4.1–4.2 need nothing from Palm calculus, so this mission can be read on its own.

14 thms3 active usersReviewed
🏆Completed
Markov ChainOperations ResearchStochastic Systems·Captain: mikedeng1

Stochastic Dynamic Programming and the Control of Queueing Systems XIII: Lyapunov Criteria and z Standard Markov Chains with CostsTextbook

Motivation

Average cost control of queues rests on a small amount of Markov chain theory: when does a chain with costs have a well defined long-run average cost, and how can that be checked for a concrete model with an unbounded state space? Appendix C of L. I. Sennott, Stochastic Dynamic Programming and the Control of Queueing Systems (Wiley, 1999, doi:10.1002/9780470317037) collects this material for countable state spaces and packages it in one hypothesis, the zzz standard chain. Chapters 7–10 of the book verify this hypothesis for the Markov chains induced by stationary policies in admission, routing and service-rate control models, and use its consequences to prove existence of average cost optimal policies.

The tools are Lyapunov functions in the sense of Foster (1953): a nonnegative function on the states whose expected one-step change is negative away from a finite set. Foster's criterion for positive recurrence, and its refinements bounding expected first passage times and costs, are the standard way to verify stability of queueing networks (Meyn and Tweedie, Markov Chains and Stochastic Stability, 1993/2009).

Setting

A Markov chain Γ\GammaΓ on a countable set SSS is given by transition probabilities Pij≥0P_{ij}\ge 0Pij​≥0 with ∑jPij=1\sum_j P_{ij}=1∑j​Pij​=1. XtX_tXt​ is the state at time ttt and Pij(t)P^{(t)}_{ij}Pij(t)​ the ttt-step transition probability (Pij(0)=δijP^{(0)}_{ij}=\delta_{ij}Pij(0)​=δij​). State iii leads to jjj if Pij(t)>0P^{(t)}_{ij}>0Pij(t)​>0 for some t≥0t\ge0t≥0; states that lead to each other communicate, which partitions SSS into communicating classes.

For a nonempty G⊆SG\subseteq SG⊆S the first passage time from iii is TiG=min⁡{t≥1:Xt∈G}T_{iG}=\min\{t\ge1: X_t\in G\}TiG​=min{t≥1:Xt​∈G} given X0=iX_0=iX0​=i, and miG=E[TiG]∈[0,∞]m_{iG}=E[T_{iG}]\in[0,\infty]miG​=E[TiG​]∈[0,∞]; mijm_{ij}mij​ is the case G={j}G=\{j\}G={j} and miim_{ii}mii​ the expected return time. The taboo probability GPik(t)_G P^{(t)}_{ik}G​Pik(t)​ is the probability of going from iii to kkk in ttt steps without visiting GGG at the intermediate times, and Guik_G u_{ik}G​uik​ is the expected number of visits to kkk at times 0≤t<TiG0\le t<T_{iG}0≤t<TiG​. A state is transient if P(Tii<∞)<1P(T_{ii}<\infty)<1P(Tii​<∞)<1 and positive recurrent if mii<∞m_{ii}<\inftymii​<∞; a positive recurrent class is a communicating class of positive recurrent states. The steady state probability is πj=(mjj)−1\pi_j=(m_{jj})^{-1}πj​=(mjj​)−1 (zero when mjj=∞m_{jj}=\inftymjj​=∞).

Each state carries a finite cost C(i)≥0C(i)\ge0C(i)≥0. The expected average cost over [0,n−1][0,n-1][0,n−1] from iii is

Ji(n)=1n E[∑t=0n−1C(Xt) ∣ X0=i]=1n∑t=0n−1∑jPij(t)C(j),J^{(n)}_i=\frac1n\,E\Big[\sum_{t=0}^{n-1}C(X_t)\,\Big|\,X_0=i\Big]=\frac1n\sum_{t=0}^{n-1}\sum_j P^{(t)}_{ij}C(j),Ji(n)​=n1​E[t=0∑n−1​C(Xt​)​X0​=i]=n1​t=0∑n−1​j∑​Pij(t)​C(j),

ciGc_{iG}ciG​ is the expected cost E[∑t=0TiG−1C(Xt)∣X0=i]E[\sum_{t=0}^{T_{iG}-1}C(X_t)\mid X_0=i]E[∑t=0TiG​−1​C(Xt​)∣X0​=i] of a first passage (defined when miG<∞m_{iG}<\inftymiG​<∞), and JR=∑j∈RπjC(j)J_R=\sum_{j\in R}\pi_jC(j)JR​=∑j∈R​πj​C(j) is the average cost on a positive recurrent class RRR. The chain is zzz standard (Definition C.2.5) if for a distinguished state zzz

miz<∞andciz<∞for all i∈S.m_{iz}<\infty\quad\text{and}\quad c_{iz}<\infty\qquad\text{for all } i\in S.miz​<∞andciz​<∞for all i∈S.

Formalization targets

Goal: Proposition C.2.6

If Γ\GammaΓ is zzz standard, then SSS is the union of a positive recurrent class R∋zR\ni zR∋z and a set of transient states, JR<∞J_R<\inftyJR​<∞, and

lim⁡n→∞Ji(n)=JRfor every i∈S.\lim_{n\to\infty}J^{(n)}_i=J_R\qquad\text{for every } i\in S.n→∞lim​Ji(n)​=JR​for every i∈S.

The statement fixes no constants: it asserts that the average cost exists, is finite, and does not depend on the initial state.

Milestones

  1. Proposition C.1.2: π\piπ is the unique stationary distribution of a positive recurrent class, and πj=eij/mii=πieij\pi_j=e_{ij}/m_{ii}=\pi_ie_{ij}πj​=eij​/mii​=πi​eij​.
  2. Proposition C.1.4: the first-step equations (C.2)–(C.4) for taboo probabilities, visit counts and miGm_{iG}miG​; ∑i∈GπimiG=1\sum_{i\in G}\pi_im_{iG}=1∑i∈G​πi​miG​=1 for GGG inside a positive recurrent class; mij<∞m_{ij}<\inftymij​<∞ within such a class.
  3. Proposition C.1.5: if ∑jPij[y(j)−y(i)]≤−ϵ\sum_jP_{ij}[y(j)-y(i)]\le-\epsilon∑j​Pij​[y(j)−y(i)]≤−ϵ off GGG, then miG≤y(i)/ϵm_{iG}\le y(i)/\epsilonmiG​≤y(i)/ϵ.
  4. Corollary C.1.6: the same with G={z}G=\{z\}G={z} and ∑jPzjy(j)<∞\sum_jP_{zj}y(j)<\infty∑j​Pzj​y(j)<∞ makes zzz positive recurrent.
  5. Proposition C.2.1: on a positive recurrent class, Ji(n)→JR=cii/miiJ^{(n)}_i\to J_R=c_{ii}/m_{ii}Ji(n)​→JR​=cii​/mii​.
  6. Proposition C.2.2: ciG=∑kC(k) Guikc_{iG}=\sum_kC(k)\,{}_Gu_{ik}ciG​=∑k​C(k)G​uik​, the first-step equation (C.13), and JR=∑i∈GπiciGJ_R=\sum_{i\in G}\pi_ic_{iG}JR​=∑i∈G​πi​ciG​.
  7. Proposition C.2.3 and Corollary C.2.4: the cost drift condition ∑jPij[r(j)−r(i)]≤−C(i)\sum_jP_{ij}[r(j)-r(i)]\le-C(i)∑j​Pij​[r(j)−r(i)]≤−C(i) off a finite set bounds ciG≤r(i)+FmiGc_{iG}\le r(i)+Fm_{iG}ciG​≤r(i)+FmiG​, and gives czz<∞c_{zz}<\inftyczz​<∞.
  8. Remark C.2.7: the hypotheses of C.1.6 and C.2.4 together imply the chain is zzz standard; so do irreducibility, positive recurrence and finite average cost.

Significance

Proposition C.2.6 is what makes the zzz standard hypothesis useful: an average cost criterion that is a genuine limit, finite, and independent of the initial state, even for chains with transient states and unbounded state spaces. Every average cost optimality result of the book that works with a stationary policy's induced chain (the (SEN) and (BOR) assumption sets, the approximating-sequence method, the continuous-time chapter) calls on this proposition or on the Lyapunov criteria of Remark C.2.7 to establish its hypotheses for queueing models.

All results of the mission are classical and proved in the literature; parts are stated in the book without proof and referred to Chung (1967), Grassmann et al. (1985) and renewal theory. None of them has been machine-checked in this form as far as the platform and Mathlib show: Mathlib has kernels and Ionescu-Tulcea trajectories but no countable-state Markov chain classification, no first passage calculus, and no Foster–Lyapunov criterion. Existing platform results on countable chains (the Levin–Peres–Wilmer series) treat irreducible chains without costs. A complete development here produces a reusable library of first passage identities, Foster–Lyapunov bounds for times and costs, and average cost limits on reducible chains.

Difficulty

The Lyapunov bounds (C.1.5, C.2.3) are telescoping arguments, but they require a clean handling of truncated passages and of sums that may be infinite: (C.7) is an inequality between possibly divergent series, and the step "iterate nnn times and let n→∞n\to\inftyn→∞" must be made rigorous for [0,∞][0,\infty][0,∞]-valued expectations.

The central difficulty is part (iii) of the goal for transient initial states. On the class RRR, the limit of Ji(n)J^{(n)}_iJi(n)​ is a renewal reward theorem over successive returns to zzz; from a transient state the first cycle has a different law, so a delayed renewal reward argument is needed, and it has to cover the case where costs are unbounded. The obvious approach, bounding Ji(n)J^{(n)}_iJi(n)​ between JRJ_RJR​ and the average over the first nnn steps of the chain started in zzz, fails because Pij(t)P^{(t)}_{ij}Pij(t)​ need not converge (periodic classes) and because finite cizc_{iz}ciz​ does not bound individual cost terms. Proposition C.1.2's uniqueness and the Kac-type identity of C.1.4(iv) likewise need the full cycle decomposition of a positive recurrent class.

Formalization scope

The chain is a structure MC S with P : S → S → ℝ≥0∞ and ∑' j, P i j = 1, over a countable type S; costs are C : S → ℝ≥0. Probabilities and expectations are ℝ≥0∞-valued sums over finite paths Fin (t+1) → S, so every quantity is defined without summability side conditions and may be ∞\infty∞. The first passage time is TiG≥1T_{iG}\ge1TiG​≥1; miGm_{iG}miG​ is the expectation of TiGT_{iG}TiG​ from its law (and ∞\infty∞ when P(TiG<∞)<1P(T_{iG}<\infty)<1P(TiG​<∞)<1), not defined by the recursion (C.4), so that (C.4) is a theorem. Guik_Gu_{ik}G​uik​ counts visits at times 0≤t<TiG0\le t<T_{iG}0≤t<TiG​. ciGc_{iG}ciG​ is computed over first passage paths and is used only when miG<∞m_{iG}<\inftymiG​<∞, as in the book. πj\pi_jπj​ is (mjj)−1(m_{jj})^{-1}(mjj​)−1, which the book states equals the Cesàro limit lim⁡nQjj(n)\lim_nQ^{(n)}_{jj}limn​Qjj(n)​. Ji(n)J^{(n)}_iJi(n)​ is meaningful for n≥1n\ge1n≥1, and limits are taken in [0,∞][0,\infty][0,∞]. The drift conditions ∑jPij[y(j)−y(i)]≤−ϵ\sum_jP_{ij}[y(j)-y(i)]\le-\epsilon∑j​Pij​[y(j)−y(i)]≤−ϵ and ∑jPij[r(j)−r(i)]≤−C(i)\sum_jP_{ij}[r(j)-r(i)]\le-C(i)∑j​Pij​[r(j)−r(i)]≤−C(i) are written in the equivalent additive form ∑jPijy(j)+ϵ≤y(i)\sum_jP_{ij}y(j)+\epsilon\le y(i)∑j​Pij​y(j)+ϵ≤y(i), which is equivalent for finite yyy and makes the case ∑jPijy(j)=∞\sum_jP_{ij}y(j)=\infty∑j​Pij​y(j)=∞ fail, as it does in the book.

A trivializing formalization, such as defining miGm_{iG}miG​ or ciGc_{iG}ciG​ by the equations (C.4) or (C.13), defining JRJ_RJR​ as the limit of Ji(n)J^{(n)}_iJi(n)​, or allowing a zzz standard chain whose return time or return cost to zzz is infinite, is ruled out: zzz standard requires miz<∞m_{iz}<\inftymiz​<∞ and ciz<∞c_{iz}<\inftyciz​<∞ for every iii including zzz, and each quantity is defined from path probabilities.

Needed infrastructure: path-sum manipulation in [0,∞][0,\infty][0,∞] (first-step and last-step decompositions), the ratio limit / renewal reward theorem for a positive recurrent class, and the delayed version for transient starts. The first passage calculus and the Lyapunov bounds are reusable by the book's other chapters on average cost, which state the zzz standard property for policy-induced chains. Contributions of lemmas on path sums and of an independent renewal reward library are welcome.

Selected references

  • L. I. Sennott, Stochastic Dynamic Programming and the Control of Queueing Systems, Wiley, 1999, Appendix C, pp. 292–302. doi:10.1002/9780470317037
  • K. L. Chung, Markov Chains with Stationary Transition Probabilities, 2nd ed., Springer, 1967. doi:10.1007/978-3-642-62015-7
  • F. G. Foster, On the stochastic matrices associated with certain queuing processes, Annals of Mathematical Statistics 24 (1953), 355–360. doi:10.1214/aoms/1177728976
  • S. P. Meyn and R. L. Tweedie, Markov Chains and Stochastic Stability, 2nd ed., Cambridge University Press, 2009. doi:10.1017/CBO9780511626630
  • D. P. Heyman and M. J. Sobel, Stochastic Models in Operations Research, Vol. I, McGraw-Hill, 1982.
12 thms3 active usersReviewed
🏆Completed
Analysis·Captain: mikedeng1

Stochastic Dynamic Programming and the Control of Queueing Systems XII: A Tauberian Theorem Linking Abel and Cesàro Means of Nonnegative SeriesTextbook

Motivation

In the theory of Markov decision chains two cost criteria dominate: the discounted cost, in which a cost incurred at time nnn is weighted by αn\alpha^nαn for a discount factor α∈(0,1)\alpha \in (0,1)α∈(0,1), and the average cost, the long-run cost per unit time. A standard route to average cost optimal policies is to solve the discounted problem and let α→1−\alpha \to 1^-α→1−. That route needs a precise link between the two criteria for a single sequence of expected costs u0,u1,u2,⋯≥0u_0, u_1, u_2, \dots \ge 0u0​,u1​,u2​,⋯≥0: the discounted cost multiplied by 1−α1-\alpha1−α on one side, the running average on the other. Appendix A.4 of Sennott, Stochastic Dynamic Programming and the Control of Queueing Systems (Wiley, 1999) supplies this link as Theorem A.4.2, and the book invokes it whenever it passes from discounted to average cost (for instance in Chapter 6, where it is applied to un=Eθ[C(Xn,An)]u_n = E_\theta[C(X_n, A_n)]un​=Eθ​[C(Xn​,An​)]).

The result belongs to classical summability theory.

  • Abelian direction. Convergence of averages implies convergence of the power-series (Abel) means: a consequence of Abel's and Frobenius's theorems on power series (19th century).
  • Tauber (1897). The first converse, under a growth condition on the terms.
  • Hardy and Littlewood (1914). The converse for Cesàro means of nonnegative terms, the "Hardy–Littlewood Tauberian theorem".
  • Karamata (1930). A short proof of that converse by polynomial approximation, reproduced in Titchmarsh, The Theory of Functions (1939, pp. 227–229). The book follows this proof.
  • Widder (1941). The continuous-time (Laplace transform) version.
  • Sennott (1986b). A proof of the discrete statement, cited by the book as its source for Theorem A.4.2.

Setting

Let u0,u1,u2,…u_0, u_1, u_2, \dotsu0​,u1​,u2​,… be nonnegative terms un∈[0,∞]u_n \in [0, \infty]un​∈[0,∞] with u0<∞u_0 < \inftyu0​<∞. For α∈[0,∞)\alpha \in [0, \infty)α∈[0,∞) the power series

U(α)=∑n=0∞αnun∈[0,∞]U(\alpha) = \sum_{n=0}^{\infty} \alpha^n u_n \in [0, \infty]U(α)=n=0∑∞​αnun​∈[0,∞]

always has a sum, possibly +∞+\infty+∞, because its partial sums increase. Its radius of convergence is R=(lim sup⁡nun1/n)−1∈[0,∞]R = (\limsup_{n} u_n^{1/n})^{-1} \in [0,\infty]R=(limsupn​un1/n​)−1∈[0,∞]. The partial sums are wn=∑k=0n−1ukw_n = \sum_{k=0}^{n-1} u_kwn​=∑k=0n−1​uk​ for n≥1n \ge 1n≥1. Two ways of averaging the sequence are compared:

  • the Cesàro means wn/nw_n / nwn​/n, as n→∞n \to \inftyn→∞;
  • the Abel means (1−α)U(α)(1-\alpha) U(\alpha)(1−α)U(α), as α→1−\alpha \to 1^-α→1− (from below).

For the constant sequence un≡Bu_n \equiv Bun​≡B both equal BBB. In the Lean development these are U, radius, w, cesaroMean and abelMean in the namespace SennottDP.Tauberian, all valued in ℝ≥0∞. The proof also uses the function rrr of Fig. A.1, equal to 000 on (0,e−1)(0, e^{-1})(0,e−1) and to 1/x1/x1/x on [e−1,1)[e^{-1}, 1)[e−1,1).

Formalization targets

Goal: Theorem A.4.2

lim inf⁡n→∞wnn≤lim inf⁡α→1−(1−α)U(α)≤lim sup⁡α→1−(1−α)U(α)≤lim sup⁡n→∞wnn,(A.28)\liminf_{n\to\infty} \frac{w_n}{n} \le \liminf_{\alpha\to 1^-} (1-\alpha)U(\alpha) \le \limsup_{\alpha\to 1^-} (1-\alpha)U(\alpha) \le \limsup_{n\to\infty} \frac{w_n}{n}, \tag{A.28}n→∞liminf​nwn​​≤α→1−liminf​(1−α)U(α)≤α→1−limsup​(1−α)U(α)≤n→∞limsup​nwn​​,(A.28)

and the following are equivalent: (i) all terms in (A.28) are equal and finite; (ii) lim⁡nwn/n\lim_n w_n/nlimn​wn​/n exists and is finite; (iii) lim⁡α→1−(1−α)U(α)\lim_{\alpha \to 1^-} (1-\alpha)U(\alpha)limα→1−​(1−α)U(α) exists and is finite.

No growth or boundedness condition on unu_nun​ is assumed: nonnegativity is the only Tauberian condition.

Milestones

  1. Remark A.3.3: the geometric series. R=1R = 1R=1, U(α)=B/(1−α)U(\alpha) = B/(1-\alpha)U(α)=B/(1−α) and B∑nnαn=Bα/(1−α)2B\sum_n n\alpha^n = B\alpha/(1-\alpha)^2B∑n​nαn=Bα/(1−α)2.
  2. Remark A.3.1: inside the radius of convergence, UUU is differentiable term by term, and the derived series has the same radius.
  3. Eq. (A.31): (∑αn)U(α)=∑nαnwn+1=U(α)/(1−α)\big(\sum \alpha^n\big) U(\alpha) = \sum_n \alpha^n w_{n+1} = U(\alpha)/(1-\alpha)(∑αn)U(α)=∑n​αnwn+1​=U(α)/(1−α).
  4. Eq. (A.28): the Abelian inequalities on their own.
  5. Eq. (A.26): ∫01r=1\int_0^1 r = 1∫01​r=1.
  6. Lemma A.4.1: continuous s∗≤r≤ss^* \le r \le ss∗≤r≤s with integrals within ε\varepsilonε of 111.
  7. Eqs. (A.35)–(A.36): if the Abel means tend to L<∞L < \inftyL<∞, then (1−α)∑nαnunf(αn)→L∫01f(1-\alpha)\sum_n \alpha^n u_n f(\alpha^n) \to L\int_0^1 f(1−α)∑n​αnun​f(αn)→L∫01​f for f(x)=xkf(x) = x^kf(x)=xk.
  8. The same statement for every fff continuous on [0,1][0,1][0,1].
  9. The same statement for f=rf = rf=r.
  10. Example A.5.1: a 0/10/10/1 block sequence with lim inf⁡wn/n=1/2\liminf w_n/n = 1/2liminfwn​/n=1/2 and lim sup⁡wn/n=2/3\limsup w_n/n = 2/3limsupwn​/n=2/3 (Choice One) or 111 (Choice Two), for which the middle inequality of (A.28) is strict.

Significance

The result itself. Theorem A.4.2 lets every statement about lim⁡α→1−(1−α)Vα\lim_{\alpha\to1^-}(1-\alpha)V_\alphalimα→1−​(1−α)Vα​ for a discounted value be read as a statement about long-run average cost, and conversely. The strict cases of Example A.5.1 show why the book must work with lim inf⁡\liminfliminf and lim sup⁡\limsuplimsup rather than limits: average costs of policies need not exist as limits, even for bounded costs.

Formalizing it. Mathlib contains Abel's theorem for convergent series (Mathlib/Analysis/Complex/AbelLimit.lean) and Cesàro convergence of convergent sequences, but no Tauberian theorem. The pinned checkout has no file mentioning "Tauberian", and neither does the platform. The mathematics has been proved since 1914. Formalizing it here adds:

  • a machine-checked Hardy–Littlewood–Karamata theorem for nonnegative [0,∞][0,\infty][0,∞]-valued sequences;
  • the lim inf⁡\liminfliminf/lim sup⁡\limsuplimsup comparison (A.28) with infinite values allowed;
  • a reusable component for the average cost chapters of this series.

Difficulty

The Abelian inequalities (A.28) follow from rearranging the series. The converse (iii) ⇒ (ii) does not: the first idea, recovering wn/nw_n/nwn​/n from (1−α)U(α)(1-\alpha)U(\alpha)(1−α)U(α) at α=1−1/n\alpha = 1 - 1/nα=1−1/n, fails, because knowing UUU near 111 controls only weighted averages of all the uku_kuk​, not a sharp truncation. Some positivity argument is unavoidable, since the converse fails for signed sequences (for example un=(−1)nnu_n = (-1)^n nun​=(−1)nn). The sharp truncation at n≈(−ln⁡α)−1n \approx (-\ln\alpha)^{-1}n≈(−lnα)−1 corresponds to the discontinuous weight rrr. Passing from polynomial weights to the jump function rrr requires uniform approximation together with the nonnegativity of the unu_nun​ at every step.

The infinite values add bookkeeping. A single un0=∞u_{n_0} = \inftyun0​​=∞ makes every term of (A.28) infinite, and R<1R < 1R<1 forces U(α)=∞U(\alpha) = \inftyU(α)=∞ on (R,1)(R,1)(R,1).

Formalization scope

Conventions the statements commit to:

  • Values. Terms u:N→[0,∞]u : \mathbb{N} \to [0,\infty]u:N→[0,∞] (ℝ≥0∞) with u0≠∞u_0 \ne \inftyu0​=∞. UUU, the Abel means, wnw_nwn​ and the Cesàro means are ℝ≥0∞-valued, with ℝ≥0∞ sums (no summability side conditions).
  • Limits. lim inf⁡\liminfliminf and lim sup⁡\limsuplimsup are those of the complete lattice [0,∞][0,\infty][0,∞], never real liminf. "Exists and is finite" means convergence in [0,∞][0,\infty][0,∞] to some L≠∞L \ne \inftyL=∞.
  • Discount factor. α∈R≥0\alpha \in \mathbb{R}_{\ge 0}α∈R≥0​, and α→1−\alpha \to 1^-α→1− is the filter 𝓝[<] 1.
  • Index n=0n = 0n=0. n→∞n \to \inftyn→∞ is atTop on N\mathbb{N}N; the value w0/0=0w_0/0 = 0w0​/0=0 is irrelevant.
  • The equivalence is List.TFAE.
  • The function rrr. Its value at the jump is r(e−1)=er(e^{-1}) = er(e−1)=e (Fig. A.1).
  • Continuity. In Lemma A.4.1 and in the continuous-weight step, continuity is required on the closed interval [0,1][0,1][0,1], which is what the Weierstrass approximation step uses. The book writes "(0,1)(0,1)(0,1)".
  • Finite terms. Remark A.3.1 is stated for finite terms, which the book's discussion of the radius presumes.

A trivializing formalization is ruled out. Real-valued lim inf⁡\liminfliminf/lim sup⁡\limsuplimsup would make (A.28) hold by junk values on unbounded sequences; hypotheses such as boundedness of unu_nun​ would replace the theorem by an easier special case; and a definition of "exists and is finite" allowing L=∞L = \inftyL=∞ would make (ii) hold for un=nu_n = nun​=n. None of these is used. As sanity checks: for un≡1u_n \equiv 1un​≡1 all four terms of (A.28) equal 111, and for un=nu_n = nun​=n all equal ∞\infty∞ and (i)–(iii) all fail.

A complete development needs:

  • Cauchy products of [0,∞][0,\infty][0,∞]-valued power series;
  • the Weierstrass approximation theorem, which Mathlib has (polynomialFunctions.topologicalClosure);
  • interval integrals of step-like functions;
  • lim inf⁡\liminfliminf/lim sup⁡\limsuplimsup manipulation along 𝓝[<] 1.

Reusable beyond this mission: the ℝ≥0∞ power-series toolkit and the Karamata argument for general continuous weights (milestone 8). Contributions welcome: proofs of any milestone, and alternative proofs of (iii) ⇒ (ii).

Selected references

  • L. I. Sennott, Stochastic Dynamic Programming and the Control of Queueing Systems, Wiley, 1999, Appendix A.3–A.5, pp. 279–287. https://doi.org/10.1002/9780470317037
  • E. C. Titchmarsh, The Theory of Functions, 2nd ed., Oxford University Press, 1939, pp. 227–229 (Karamata's proof).
  • J. Karamata, "Über die Hardy–Littlewoodschen Umkehrungen des Abelschen Stetigkeitssatzes", Mathematische Zeitschrift 32 (1930), 319–320.
  • G. H. Hardy and J. E. Littlewood, "Tauberian theorems concerning power series and Dirichlet's series whose coefficients are positive", Proc. London Math. Soc. (2) 13 (1914), 174–191.
  • D. V. Widder, The Laplace Transform, Princeton University Press, 1941.
  • L. I. Sennott, "A new condition for the existence of optimum stationary policies in average cost Markov decision processes — unbounded cost case", Proc. 25th IEEE Conference on Decision and Control, Athens, 1986, pp. 1719–1721 (cited by the book as Sennott 1986b).
  • T. M. Liggett and S. A. Lippman, "Short notes: Stochastic games with perfect information and time average payoff", SIAM Review 11 (1969), 604–607. https://doi.org/10.1137/1011093
14 thms3 active usersReviewed
🏆Completed
Analysis·Captain: mikedeng1

Stochastic Dynamic Programming and the Control of Queueing Systems XI: The Generalized Dominated Convergence Theorem for Approximating DistributionsTextbook

Motivation

Linn Sennott's Stochastic Dynamic Programming and the Control of Queueing Systems (Wiley, 1999) studies Markov decision chains with countably many states, finite action sets and unbounded nonnegative costs. Its main computational device, the approximating sequence method (Definition 2.5.1, p. 28), replaces the countable state space SSS by an increasing sequence of state spaces SNS_NSN​ and the transition law out of each state by distributions Pj(N)P_j(N)Pj​(N) on SNS_NSN​ that converge pointwise to the original law. Every convergence argument of that method has the same shape: a sum ∑j∈SNPj(N) u(j,N)\sum_{j\in S_N} P_j(N)\,u(j,N)∑j∈SN​​Pj​(N)u(j,N), an expected cost or value under the NNN-th approximating model, must be shown to converge to ∑j∈SPj u(j)\sum_{j\in S} P_j\,u(j)∑j∈S​Pj​u(j), or at least to satisfy a one-sided bound in the limit.

Appendix A, Sections A.1–A.2 (pp. 270–278), collects the analysis results used for this. The author proves them for sums over countable sets rather than general integrals, so they need no measure theory, and the proofs of the discounted-cost and average-cost approximation theorems (Chapters 4 and 8) cite them. This mission formalizes that appendix section as a self-contained unit.

Setting

Let SSS be a countable set and (Pj)j∈S(P_j)_{j\in S}(Pj​)j∈S​ a probability distribution on SSS: Pj≥0P_j\ge 0Pj​≥0 and ∑jPj=1\sum_j P_j=1∑j​Pj​=1. Values of functions are extended reals in [−∞,∞][-\infty,\infty][−∞,∞], with the convention 0⋅∞=00\cdot\infty=00⋅∞=0. For u:S→[−∞,∞]u:S\to[-\infty,\infty]u:S→[−∞,∞], the weighted sum is

∑j∈SPj u(j)=∑j∈SPj u+(j)−∑j∈SPj u−(j),\sum_{j\in S}P_j\,u(j)=\sum_{j\in S}P_j\,u^+(j)-\sum_{j\in S}P_j\,u^-(j),j∈S∑​Pj​u(j)=j∈S∑​Pj​u+(j)−j∈S∑​Pj​u−(j),

which is well defined, in (−∞,∞](-\infty,\infty](−∞,∞], whenever the negative parts have finite weighted sum. In Lean this is wsum P u.

A family of approximating distributions for (Pj)(P_j)(Pj​) consists of

  1. an increasing sequence of subsets S0⊆S1⊆⋯S_0\subseteq S_1\subseteq\cdotsS0​⊆S1​⊆⋯ of SSS with ⋃NSN=S\bigcup_N S_N=S⋃N​SN​=S;
  2. for each NNN, a probability distribution (Pj(N))j∈SN(P_j(N))_{j\in S_N}(Pj​(N))j∈SN​​ on SNS_NSN​;
  3. pointwise convergence lim⁡N→∞Pj(N)=Pj\lim_{N\to\infty}P_j(N)=P_jlimN→∞​Pj​(N)=Pj​ for every j∈Sj\in Sj∈S.

Since the sets increase to SSS, a fixed state jjj lies in SNS_NSN​ for every large NNN, so lim inf⁡Nu(j,N)\liminf_N u(j,N)liminfN​u(j,N) and lim⁡Nu(j,N)\lim_N u(j,N)limN​u(j,N) make sense for a function u(j,N)u(j,N)u(j,N) defined only for j∈SNj\in S_Nj∈SN​. In Lean these three conditions are the structure ApproxDist P SN Q, with Q N j =Pj(N)=P_j(N)=Pj​(N).

Formalization targets

Goal: Theorem A.2.6 (Generalized Dominated Convergence Theorem), p. 278

Under the approximating-distribution hypotheses, let u(j,N)u(j,N)u(j,N) and w(j,N)w(j,N)w(j,N) be finite with ∣u(j,N)∣≤w(j,N)|u(j,N)|\le w(j,N)∣u(j,N)∣≤w(j,N) for j∈SNj\in S_Nj∈SN​, let u(j,N)→u(j)u(j,N)\to u(j)u(j,N)→u(j) and w(j,N)→w(j)w(j,N)\to w(j)w(j,N)→w(j) for every jjj, and assume

lim⁡N→∞∑j∈SNPj(N) w(j,N)=∑j∈SPj w(j)<∞.\lim_{N\to\infty}\sum_{j\in S_N}P_j(N)\,w(j,N)=\sum_{j\in S}P_j\,w(j)<\infty.N→∞lim​j∈SN​∑​Pj​(N)w(j,N)=j∈S∑​Pj​w(j)<∞.

Then

lim⁡N→∞∑j∈SNPj(N) u(j,N)=∑j∈SPj u(j).\lim_{N\to\infty}\sum_{j\in S_N}P_j(N)\,u(j,N)=\sum_{j\in S}P_j\,u(j).N→∞lim​j∈SN​∑​Pj​(N)u(j,N)=j∈S∑​Pj​u(j).

Milestones, in the book's order

  • Proposition A.1.3: lim inf⁡\liminfliminf commutes with a minimum over a finite nonempty set; lim⁡\limlim does too when all limits exist.
  • Proposition A.1.5: lim inf⁡N∑j∈Gu(j,N)≥∑j∈Glim inf⁡Nu(j,N)\liminf_N\sum_{j\in G}u(j,N)\ge\sum_{j\in G}\liminf_N u(j,N)liminfN​∑j∈G​u(j,N)≥∑j∈G​liminfN​u(j,N) for finite GGG and u>−∞u>-\inftyu>−∞, when the right side has no indeterminate form.
  • Proposition A.1.7: the same inequality for countable sums of [0,∞][0,\infty][0,∞]-valued terms.
  • Proposition A.1.8: the same, with the left sum over SNS_NSN​ increasing to SSS.
  • Proposition A.1.10: lim inf⁡un≤lim inf⁡wn/n≤lim sup⁡wn/n≤lim sup⁡un\liminf u_n\le\liminf w_n/n\le\limsup w_n/n\le\limsup u_nliminfun​≤liminfwn​/n≤limsupwn​/n≤limsupun​ for wn=∑k<nukw_n=\sum_{k<n}u_kwn​=∑k<n​uk​.
  • Proposition A.2.1 (Fatou's lemma): lim inf⁡N∑jPju(j,N)≥∑jPjlim inf⁡Nu(j,N)\liminf_N\sum_jP_j u(j,N)\ge\sum_jP_j\liminf_N u(j,N)liminfN​∑j​Pj​u(j,N)≥∑j​Pj​liminfN​u(j,N) for u≥−Lu\ge -Lu≥−L.
  • Theorem A.2.3 (dominated convergence, with a convergent dominating sequence w(j,N)w(j,N)w(j,N)) and Corollary A.2.4 (fixed dominating function).
  • Proposition A.2.5 (generalized Fatou's lemma):
lim inf⁡N→∞∑j∈SNPj(N) u(j,N) ≥ ∑j∈SPj lim inf⁡N→∞u(j,N)(u≥−L on SN).\liminf_{N\to\infty}\sum_{j\in S_N}P_j(N)\,u(j,N)\ \ge\ \sum_{j\in S}P_j\,\liminf_{N\to\infty}u(j,N)\qquad(u\ge -L\text{ on }S_N).N→∞liminf​j∈SN​∑​Pj​(N)u(j,N) ≥ j∈S∑​Pj​N→∞liminf​u(j,N)(u≥−L on SN​).
  • Corollary A.2.7: bounded convergence under approximating distributions.

Significance

The results. Proposition A.2.5 and Theorem A.2.6 are the two limit interchanges the approximating sequence method needs. The generalized Fatou inequality gives the lower bound on limits of value functions of the approximating models, and the generalized dominated convergence theorem identifies the limit of expected costs when a dominating function is available. Corollary A.2.7 gives the bounded case, and Proposition A.1.3 moves limits inside the minimization of an optimality equation. Proposition A.1.10 compares Cesàro averages with the sequence itself, which the average-cost chapters use.

Formalizing them. All statements are classical and proved in the book. A.1.7, A.2.1, A.2.3 and A.2.4 are, after translation, instances of Mathlib's Fatou lemma (MeasureTheory.lintegral_liminf_le) and dominated convergence theorem for a discrete measure. The mission states them in the book's discrete, extended-real form so later chapters can cite them by number. The versions in which the distribution and its support change with NNN (A.2.5–A.2.7) are not in Mathlib in this form; the generalized dominated convergence theorem appears in the measure-theoretic literature (Royden, Real Analysis, Ch. 4) but has no Lean formalization. None of these statements has been formalized for this book.

Difficulty

For the generalized results, Mathlib's Fatou lemma and dominated convergence theorem do not apply directly: they fix one measure, while here both the weights Pj(N)P_j(N)Pj​(N) and the support SNS_NSN​ change with NNN. The dominating function w(j,N)w(j,N)w(j,N) also changes with NNN and is not integrable uniformly in NNN; only the convergence of its expectations is assumed. For small NNN, ∑j∈SNPj(N)w(j,N)\sum_{j\in S_N}P_j(N)w(j,N)∑j∈SN​​Pj​(N)w(j,N) may even be infinite, and then ∑j∈SNPj(N)u(j,N)\sum_{j\in S_N} P_j(N)u(j,N)∑j∈SN​​Pj​(N)u(j,N) has no value.

A second difficulty is bookkeeping in the extended reals. Lower limits may be ±∞\pm\infty±∞, products use 0⋅∞=00\cdot\infty=00⋅∞=0, and a sum with a +∞+\infty+∞ term and a −∞-\infty−∞ term is undefined. The inequality of Proposition A.1.5 holds only when that indeterminate form is excluded. The bound u≥−Lu\ge -Lu≥−L in A.2.1 and A.2.5 cannot be dropped: Examples A.1.9 and A.2.2 of the book show the inequalities fail without it.

Formalization scope

  • Types. SSS is any type with [Countable S]. Probabilities are ℝ≥0∞-valued with ∑' j, P j = 1, and Pj(N)P_j(N)Pj​(N) is Q N j. Extended-real values are EReal, and nonnegative extended values (A.1.7, A.1.8) are ℝ≥0∞. Limits and lower and upper limits are along atTop in ℕ. The book's N≥N0N\ge N_0N≥N0​ convention (Remark A.1.2) needs nothing extra, since only large NNN matter.
  • Sums over SNS_NSN​ are sums over SSS of (SN N).indicator (Q N) times the function. Values of u(j,N)u(j,N)u(j,N) and Pj(N)P_j(N)Pj​(N) for j∉SNj\notin S_Nj∈/SN​ never enter any statement, and the hypotheses on uuu (u≥−Lu\ge -Lu≥−L, ∣u∣≤w|u|\le w∣u∣≤w) are imposed only on SNS_NSN​, as in the book.
  • Weighted sums are wsum, the difference of the weighted positive and negative parts, each in [0,∞]. When both are infinite the book's sum is undefined and wsum returns −∞-\infty−∞. Every sum used in a hypothesis or conclusion is defined, except possibly for finitely many NNN in limit statements, where it does not matter.
  • Limits of u(j,N)u(j,N)u(j,N) and w(j,N)w(j,N)w(j,N) are taken in EReal, since the book allows extended-real limits (p. 271). The hypotheses force them to be finite wherever Pj>0P_j>0Pj​>0.
  • Ruled out. Statements in which wsum returns its junk value −∞-\infty−∞ could make an inequality trivial. Every left-hand side here is a lower limit of sums whose negative parts are bounded, or a limit of eventually defined sums, so no statement holds only because of that junk value. Real-valued tsum with its junk value 000 is not used anywhere.
  • Welcome contributions. A proof of A.1.7 from lintegral_liminf_le for the counting measure, and lemmas relating wsum to Mathlib's tsum and to integrals against Measure.sum (fun j => P j • dirac j), can be reused by the other missions of this series.

Selected references

  • L. I. Sennott, Stochastic Dynamic Programming and the Control of Queueing Systems, Wiley Series in Probability and Statistics, John Wiley & Sons, 1999, Appendix A, pp. 270–278. https://doi.org/10.1002/9780470317037
  • H. L. Royden, Real Analysis, 3rd ed., Macmillan, 1988, Chapter 4 (Fatou's lemma, generalized Lebesgue convergence theorem).
  • The Mathlib Community, The Lean Mathematical Library, CPP 2020, https://doi.org/10.1145/3372885.3373824 (MeasureTheory.lintegral_liminf_le, MeasureTheory.tendsto_integral_of_dominated_convergence).
12 thms3 active usersReviewed
🏆Completed
Dynamic ProgrammingOperations ResearchStochastic Systems·Captain: mikedeng1

Stochastic Dynamic Programming and the Control of Queueing Systems X: Average Cost Optimization of Continuous Time Markov Decision ChainsTextbook

Motivation

Many queueing systems evolve in continuous time: customers arrive according to a Poisson process, services take exponentially distributed times, and a controller may change the service rate, admit or reject customers, or route them whenever the state changes. Minimizing the long-run average cost of such a system is a standard problem in the control of queues (Lippman 1975; Puterman 1994, Ch. 11; Sennott 1999, Ch. 10). The continuous time model does not fit directly into the discrete time theory of Markov decision chains developed in the earlier chapters of Sennott's book, because time spent in a state now matters and the natural average cost is a ratio of expected cost to expected elapsed time.

This mission formalizes Sections 10.1–10.4 of L. I. Sennott, Stochastic Dynamic Programming and the Control of Queueing Systems (Wiley, 1999): the elementary properties of the exponential distribution, the continuous time Markov decision chain and its average cost, a reduction of the continuous time problem to an auxiliary discrete time Markov decision chain, and the theorem stating that finite state approximating sequences of the auxiliary chain compute optimal average costs and optimal stationary policies of the continuous time chain. The chapter closes with an explicit average cost computation for the M/M/1 queue with service rate control.

Setting

A random variable XXX has the exponential distribution with rate μ>0\mu>0μ>0 if P(X≤t)=1−e−μtP(X\le t)=1-e^{-\mu t}P(X≤t)=1−e−μt for t≥0t\ge0t≥0. A function r(δ)r(\delta)r(δ) is o(δ)o(\delta)o(δ) if r(δ)/δ→0r(\delta)/\delta\to0r(δ)/δ→0 as δ→0+\delta\to0^+δ→0+.

A continuous time Markov decision chain (CTMDC) Ψ\PsiΨ has a countable state space SSS and, for each i∈Si\in Si∈S, a finite nonempty action set AiA_iAi​. Choosing a∈Aia\in A_ia∈Ai​ in state iii incurs an instantaneous cost G(i,a)≥0G(i,a)\ge0G(i,a)≥0 and a cost rate g(i,a)≥0g(i,a)\ge0g(i,a)≥0 in effect until the next transition. The time until the next transition is exponential with rate ν(i,a)>0\nu(i,a)>0ν(i,a)>0, so its mean is τ(i,a)=1/ν(i,a)\tau(i,a)=1/\nu(i,a)τ(i,a)=1/ν(i,a); the next state is jjj with probability Pij(a)P_{ij}(a)Pij​(a), where Pii(a)=0P_{ii}(a)=0Pii​(a)=0. A policy θ\thetaθ chooses, at each transition, an action (possibly at random) from the history of past states, actions and sojourn times; a stationary policy eee chooses e(i)e(i)e(i) in state iii. With CnC_nCn​ the cost and TnT_nTn​ the time of the first nnn transition periods, the average cost and the minimum average cost are

JθΨ(i)=lim sup⁡n→∞Eθ[Cn∣X0=i]Eθ[Tn∣X0=i],JΨ(i)=inf⁡θJθΨ(i).J^\Psi_\theta(i)=\limsup_{n\to\infty}\frac{E_\theta[C_n\mid X_0=i]}{E_\theta[T_n\mid X_0=i]},\qquad J^\Psi(i)=\inf_\theta J^\Psi_\theta(i).JθΨ​(i)=n→∞limsup​Eθ​[Tn​∣X0​=i]Eθ​[Cn​∣X0​=i]​,JΨ(i)=θinf​JθΨ​(i).

Assumption (CTB) requires constants τ\tauτ and BBB with 0<τ<inf⁡i,aτ(i,a)≤sup⁡i,aτ(i,a)≤B<∞0<\tau<\inf_{i,a}\tau(i,a)\le\sup_{i,a}\tau(i,a)\le B<\infty0<τ<infi,a​τ(i,a)≤supi,a​τ(i,a)≤B<∞. The auxiliary MDC Δ\DeltaΔ has the same states and actions, costs C(i,a)=G(i,a)ν(i,a)+g(i,a)C(i,a)=G(i,a)\nu(i,a)+g(i,a)C(i,a)=G(i,a)ν(i,a)+g(i,a), and transition probabilities Pij∗(a)=τν(i,a)Pij(a)P^*_{ij}(a)=\tau\nu(i,a)P_{ij}(a)Pij∗​(a)=τν(i,a)Pij​(a) for j≠ij\ne ij=i, Pii∗(a)=1−τν(i,a)P^*_{ii}(a)=1-\tau\nu(i,a)Pii∗​(a)=1−τν(i,a). Its average cost JθΔ(i)=lim sup⁡nn−1∑t<nEθ[C(Xt,Yt)]J^\Delta_\theta(i)=\limsup_n n^{-1}\sum_{t<n}E_\theta[C(X_t,Y_t)]JθΔ​(i)=limsupn​n−1∑t<n​Eθ​[C(Xt​,Yt​)] and minimum average cost JΔ(i)J^\Delta(i)JΔ(i) are those of Chapter 2. Assumption (CTAC) is JΔ(⋅)≤JΨ(⋅)J^\Delta(\cdot)\le J^\Psi(\cdot)JΔ(⋅)≤JΨ(⋅).

An approximating sequence (ΔN)N≥N0(\Delta_N)_{N\ge N_0}(ΔN​)N≥N0​​ for Δ\DeltaΔ uses finite state spaces SNS_NSN​ increasing to SSS and transition probabilities Pij∗(a;N)P^*_{ij}(a;N)Pij∗​(a;N) on SNS_NSN​ converging to Pij∗(a)P^*_{ij}(a)Pij∗​(a). The (AC) assumptions ask for constants JNJ^NJN and functions rNr^NrN on SNS_NSN​ solving

JN+rN(i)=min⁡a∈Ai{C(i,a)+∑j∈SNPij∗(a;N) rN(j)},i∈SN, N≥N0,(10.21)J^N+r^N(i)=\min_{a\in A_i}\Big\{C(i,a)+\sum_{j\in S_N}P^*_{ij}(a;N)\,r^N(j)\Big\},\qquad i\in S_N,\ N\ge N_0,\tag{10.21}JN+rN(i)=a∈Ai​min​{C(i,a)+j∈SN​∑​Pij∗​(a;N)rN(j)},i∈SN​, N≥N0​,(10.21)

with lim sup⁡NrN(i)<∞\limsup_N r^N(i)<\inftylimsupN​rN(i)<∞, lim inf⁡NrN(i)≥−Q\liminf_N r^N(i)\ge-QliminfN​rN(i)≥−Q for a constant Q≥0Q\ge0Q≥0, and lim sup⁡NJN=:J∗<∞\limsup_N J^N=:J^*<\inftylimsupN​JN=:J∗<∞, J∗≤JΔ(i)J^*\le J^\Delta(i)J∗≤JΔ(i).

Formalization targets

Goal: Theorem 10.3.3

Under (CTB), (CTAC) and the (AC) assumptions for an approximating sequence of Δ\DeltaΔ:

J∗=lim⁡N→∞JN exists and JΔ(i)=JΨ(i)=J∗(i∈S),J^*=\lim_{N\to\infty}J^N\ \text{exists and}\ J^\Delta(i)=J^\Psi(i)=J^*\quad(i\in S),J∗=N→∞lim​JN exists and JΔ(i)=JΨ(i)=J∗(i∈S),

and every limit point e∗e^*e∗ of a sequence eNe^NeN of stationary policies realizing the minimum in (10.21) satisfies Je∗Δ=JΔJ^\Delta_{e^*}=J^\DeltaJe∗Δ​=JΔ and Je∗Ψ=JΨJ^\Psi_{e^*}=J^\PsiJe∗Ψ​=JΨ. The goal leaves the chain, the approximating sequence and the constants of (CTB) arbitrary.

Milestones

  • Proposition 10.1.2: P(X>x+y∣X>y)=P(X>x)P(X>x+y\mid X>y)=P(X>x)P(X>x+y∣X>y)=P(X>x) for x,y>0x,y>0x,y>0, and P(X≤δ)=μδ+o(δ)P(X\le\delta)=\mu\delta+o(\delta)P(X≤δ)=μδ+o(δ).
  • Proposition 10.1.3: for independent exponentials, P(X1≤δ,X2≤δ)=o(δ)P(X_1\le\delta,X_2\le\delta)=o(\delta)P(X1​≤δ,X2​≤δ)=o(δ), P(X1<X2)=μ1/(μ1+μ2)P(X_1<X_2)=\mu_1/(\mu_1+\mu_2)P(X1​<X2​)=μ1​/(μ1​+μ2​), and min⁡(X1,X2)\min(X_1,X_2)min(X1​,X2​) is exponential with rate μ1+μ2\mu_1+\mu_2μ1​+μ2​.
  • Lemma 10.3.1: if zzz is bounded below and Zτ(i,e)+z(i)≥G(i,e)+g(i,e)τ(i,e)+∑jPij(e)z(j)Z\tau(i,e)+z(i)\ge G(i,e)+g(i,e)\tau(i,e)+\sum_jP_{ij}(e)z(j)Zτ(i,e)+z(i)≥G(i,e)+g(i,e)τ(i,e)+∑j​Pij​(e)z(j) for all iii (10.15), then JeΨ≤ZJ^\Psi_e\le ZJeΨ​≤Z.
  • Lemma 10.3.2: (Z,w)(Z,w)(Z,w) satisfies Z+w(i)≥C(i,e)+∑jPij∗(e)w(j)Z+w(i)\ge C(i,e)+\sum_jP^*_{ij}(e)w(j)Z+w(i)≥C(i,e)+∑j​Pij∗​(e)w(j) (10.20) if and only if (Z,τw)(Z,\tau w)(Z,τw) satisfies (10.15).
  • Proposition 10.4.1: in the M/M/1 queue with arrival rate λ\lambdaλ, holding cost H(i)=HiH(i)=HiH(i)=Hi and service cost rate c(a)c(a)c(a), the policy that always serves at rate a>λa>\lambdaa>λ has average cost ρac(a)+Hρa/(1−ρa)\rho_ac(a)+H\rho_a/(1-\rho_a)ρa​c(a)+Hρa​/(1−ρa​), ρa=λ/a\rho_a=\lambda/aρa​=λ/a.

Significance

The goal theorem turns the average cost control of a continuous time chain on an infinite state space into a finite computation: solve the optimality equation (10.21) of a finite truncation of the auxiliary chain, let the truncation grow, and read off the optimal average cost and an optimal stationary policy of the original continuous time chain. The auxiliary chain is the book's form of uniformization, and the result is what licenses the numerical study of the M/M/1 service rate control problem in Section 10.4 and of the M/M/K and polling models in Sections 10.5–10.6. Proposition 10.4.1 gives the closed-form benchmark against which the computed optimal policy is compared.

The results are proved in the book, some with details left to the reader (Lemma 10.3.2(ii), Problem 10.10), and the goal rests on Theorem 8.1.1 and Lemma 7.2.1 of the same book. None of them has, as far as a search of Mathlib and the Prove2Me catalogue shows, a machine-checked proof: Mathlib provides the exponential law (ProbabilityTheory.expMeasure) and its distribution function, but not memorylessness or the minimum of independent exponentials, and no continuous time Markov decision model. A formalization would supply these, together with a checked average cost comparison between a continuous time chain and its discrete time auxiliary chain.

Difficulty

The obvious argument compares the two chains policy by policy, but the policy classes differ: a policy for Δ\DeltaΔ may change action in every time slot, including slots where the state does not change, while a policy for Ψ\PsiΨ acts only at transitions and may use the observed sojourn times. Only the stationary policies coincide. The lower bound JΨ≥J∗J^\Psi\ge J^*JΨ≥J∗ therefore cannot be obtained by transferring policies, and it is exactly what Assumption (CTAC) supplies. The upper bound requires passing from the discrete time inequality (10.20) for the limit point e∗e^*e∗ to a bound on a ratio of expected cost to expected time in continuous time, where the denominator depends on the policy; the uniform bounds of (CTB) on the mean sojourn times are what control it. Inside Lemma 10.3.1 the function zzz is only bounded below, so the telescoping of expectations must be justified without integrability of zzz from above.

Formalization scope

The state space is a countable type S, actions a type Act, and action sets A i : Finset Act; the CTMDC and MDC structures hold data, and their axioms (nonempty action sets, nonnegative costs, positive rates, stochastic transition rows with Pii(a)=0P_{ii}(a)=0Pii​(a)=0) are separate predicates. Transition probabilities are ℝ≥0∞-valued; costs, rates and the functions z,w,rNz,w,r^Nz,w,rN are real. Expected costs, expected times and all average costs are ℝ≥0∞-valued, so +∞+\infty+∞ is a legitimate value, and they are compared with real constants in EReal; the limits superior and inferior of (AC) are taken in EReal. The expected cost of nnn transition periods under a general policy is a recursion over the periods in which the sojourn time is integrated against expMeasure ν(i,a) and the next state is drawn independently from Pi⋅(a)P_{i\cdot}(a)Pi⋅​(a); policies are measurable in the past sojourn times. In (10.15) and (10.20) the convergence of the series is part of the inequality. The strict inequality τ<inf⁡τ(i,a)\tau<\inf\tau(i,a)τ<infτ(i,a) of (CTB) is kept strict (as a positive margin); weakening it to ≤\le≤ would make Pii∗(a)P^*_{ii}(a)Pii∗​(a) vanish or turn negative.

The average cost JθΨJ^\Psi_\thetaJθΨ​ is a ratio of expectations, not the expectation of a ratio, and the infimum JΨJ^\PsiJΨ ranges over history dependent randomized policies that may use sojourn times; replacing either by a stationary-only class, or dropping (CTAC), gives a different theorem.

A complete development needs: expected rewards of a chain with exponential holding times, the average cost theory of Chapter 8 for the auxiliary chain (Theorem 8.1.1 and Lemma 7.2.1, restated here as needed), and renewal-reward reasoning for Proposition 10.4.1. The exponential-distribution lemmas are reusable beyond this mission and are welcome as independent contributions.

Selected references

  • L. I. Sennott, Stochastic Dynamic Programming and the Control of Queueing Systems, Wiley Series in Probability and Statistics, John Wiley & Sons, 1999. https://doi.org/10.1002/9780470317037
  • M. L. Puterman, Markov Decision Processes: Discrete Stochastic Dynamic Programming, John Wiley & Sons, 1994. https://doi.org/10.1002/9780470316887
  • S. A. Lippman, Applying a new device in the optimization of exponential queuing systems, Operations Research 23(4), 687–710, 1975. https://doi.org/10.1287/opre.23.4.687
  • D. Gross and C. M. Harris, Fundamentals of Queueing Theory, 3rd ed., John Wiley & Sons, 1998.
11 thms3 active users
🏆Completed
Operations ResearchStochastic Systems·Captain: mikedeng1

Stochastic Dynamic Programming and the Control of Queueing Systems IX: Bounded Mean Residual Lifetimes Imply Finite Moments of All OrdersTextbook

Motivation

In a discrete-time queue the service of a customer lasts a random number YYY of slots. When such a system is modelled as a Markov decision chain (Sennott, Stochastic Dynamic Programming and the Control of Queueing Systems, Wiley, 1999, DOI 10.1002/9780470317037, Chapter 9), the state must record how much service is still owed, and the controller only observes that a service has lasted sss slots and is not yet finished. The relevant random quantity is then the residual life YsY_sYs​: the remaining service time given that sss slots have elapsed without completion. Verifying the book's average cost assumptions for such a model requires bounds on expected first passage times and costs, and these reduce to moment bounds on YYY and on the residual lives YsY_sYs​.

Section 9.2 isolates a single condition that makes those bounds available: the expected remaining service time is bounded uniformly in the elapsed time. The concept of mean residual life comes from reliability theory, where YYY is the lifetime of a component and E[Ys]E[Y_s]E[Ys​] is its expected remaining lifetime at age sss. This mission formalizes Section 9.2 of the book, together with the moment computation for batch arrivals (Lemma 9.5.2) that the same verification uses.

Setting

Let YYY be a random variable with values in {1,2,3,… }\{1,2,3,\dots\}{1,2,3,…} and distribution uy=P(Y=y)u_y = P(Y = y)uy​=P(Y=y), y≥1y \ge 1y≥1. Write F(y)=P(Y≤y)F(y) = P(Y \le y)F(y)=P(Y≤y) and F∗(y)=P(Y>y)=1−F(y)F^*(y) = P(Y > y) = 1 - F(y)F∗(y)=P(Y>y)=1−F(y) for y≥0y \ge 0y≥0, so F(0)=0F(0) = 0F(0)=0 and F∗(0)=1F^*(0) = 1F∗(0)=1. The kkk-th moment is

E[Yk]=∑y≥1ykuy∈[0,∞].E[Y^k] = \sum_{y \ge 1} y^k u_y \in [0,\infty].E[Yk]=y≥1∑​ykuy​∈[0,∞].

For s≥0s \ge 0s≥0 with F∗(s)>0F^*(s) > 0F∗(s)>0, the residual life YsY_sYs​ has distribution

P(Ys=y)=P(Y=s+y∣Y>s)=us+yF∗(s),y≥1,P(Y_s = y) = P(Y = s + y \mid Y > s) = \frac{u_{s+y}}{F^*(s)}, \qquad y \ge 1,P(Ys​=y)=P(Y=s+y∣Y>s)=F∗(s)us+y​​,y≥1,

with Y0=YY_0 = YY0​=Y; its tail is Fs∗(y)=F∗(s+y)/F∗(s)F^*_s(y) = F^*(s+y)/F^*(s)Fs∗​(y)=F∗(s+y)/F∗(s), and E[Ys]E[Y_s]E[Ys​] is the mean residual lifetime.

The distribution of YYY has bounded mean residual lifetimes (BMRL-UUU, Definition 9.2.4) if there is a finite constant UUU with

E[Ys]≤Ufor every s≥0 with F∗(s)>0,E[Y_s] \le U \qquad \text{for every } s \ge 0 \text{ with } F^*(s) > 0,E[Ys​]≤Ufor every s≥0 with F∗(s)>0,

and it is BMRL if it is BMRL-UUU for some UUU.

Three families appear by name: the geometric distribution geo(μ)\mathrm{geo}(\mu)geo(μ) of the number of Bernoulli(μ\muμ) trials to the first success, P(Y=y)=μ(1−μ)y−1P(Y=y) = \mu(1-\mu)^{y-1}P(Y=y)=μ(1−μ)y−1; the negative binomial neg bin(μ,r)\mathrm{neg\,bin}(\mu, r)negbin(μ,r) of the number of trials to the rrr-th success, P(Y=y)=(y−1r−1)μr(1−μ)y−rP(Y = y) = \binom{y-1}{r-1}\mu^r(1-\mu)^{y-r}P(Y=y)=(r−1y−1​)μr(1−μ)y−r for y≥ry \ge ry≥r; and the truncated Poisson trun Pois(λ)\mathrm{trun\,Pois}(\lambda)trunPois(λ), P(Y=y)=e−λ1−e−λλyy!P(Y=y) = \frac{e^{-\lambda}}{1-e^{-\lambda}}\frac{\lambda^y}{y!}P(Y=y)=1−e−λe−λ​y!λy​ for y≥1y \ge 1y≥1.

For Lemma 9.5.2, batches of customers arrive in each slot; the batch sizes X1,X2,…X_1, X_2, \dotsX1​,X2​,… are independent with common distribution pjp_jpj​, mean λ=∑jjpj\lambda = \sum_j j p_jλ=∑j​jpj​ and second moment λ(2)=∑jj2pj\lambda^{(2)} = \sum_j j^2 p_jλ(2)=∑j​j2pj​, and X(s)=X1+⋯+XsX(s) = X_1 + \dots + X_sX(s)=X1​+⋯+Xs​ is the number of arrivals in sss slots.

Formalization targets

Goal: Proposition 9.2.5

If the distribution of YYY is BMRL, then

E[Yk]<∞for every k.E[Y^k] < \infty \qquad \text{for every } k.E[Yk]<∞for every k.

The goal fixes no constant: it asserts only that a uniform first-moment bound on the residual lives forces every moment of YYY to be finite.

Milestones

  1. Proposition 9.2.1. E[Y]=∑y=0∞F∗(y)E[Y] = \sum_{y=0}^\infty F^*(y)E[Y]=∑y=0∞​F∗(y) and, for k≥2k \ge 2k≥2,
E[Yk]=1+∑z=0k−1(kz)[∑y=1∞yzF∗(y)].(9.4)E[Y^k] = 1 + \sum_{z=0}^{k-1}\binom{k}{z}\left[\sum_{y=1}^\infty y^z F^*(y)\right]. \tag{9.4}E[Yk]=1+z=0∑k−1​(zk​)[y=1∑∞​yzF∗(y)].(9.4)
  1. Remark 9.2.2. For k≥2k \ge 2k≥2, E[Yk]<∞E[Y^k] < \inftyE[Yk]<∞ if and only if ∑yyk−1F∗(y)<∞\sum_y y^{k-1}F^*(y) < \infty∑y​yk−1F∗(y)<∞.
  2. Proposition 9.2.3. For a positive integer kkk, E[Yk]<∞E[Y^k] < \inftyE[Yk]<∞ implies E[Ysk]<∞E[Y_s^k] < \inftyE[Ysk​]<∞ for all s≥0s \ge 0s≥0.
  3. Proposition 9.2.6. The geometric (0<μ<10<\mu<10<μ<1), negative binomial (0<μ<10<\mu<10<μ<1, r≥2r \ge 2r≥2) and truncated Poisson (λ>0\lambda > 0λ>0) distributions are BMRL.
  4. Lemma 9.5.2. Under λ(2)<∞\lambda^{(2)} < \inftyλ(2)<∞,
E[X(s)]=λs,E[(X(s))2]=λ(2)s+λ2s(s−1).(9.25)E[X(s)] = \lambda s, \qquad E[(X(s))^2] = \lambda^{(2)}s + \lambda^2 s(s-1). \tag{9.25}E[X(s)]=λs,E[(X(s))2]=λ(2)s+λ2s(s−1).(9.25)

Significance

The result itself. Proposition 9.2.5 turns a condition that is easy to check for concrete service distributions, and natural for services (a service whose expected remaining duration grows without bound as it goes on is undesirable), into the moment bounds that the average cost analysis consumes. With Proposition 9.2.6 it shows that the most common unbounded service distributions on {1,2,… }\{1,2,\dots\}{1,2,…} have finite moments of all orders; with Lemma 9.5.2 it supplies the linear and quadratic growth of expected arrivals and their second moments that the verification of the (WAC) assumptions for the batch-arrival queue of Example 9.3.1 needs (Section 9.5). Every bounded distribution is BMRL as well (the book's Problem 9.3).

Formalizing it. All results here are proved in the book; none has a machine-checked proof on the platform or in Mathlib, which has geometric and Poisson distributions but no residual lives, negative binomial or truncated Poisson laws. A complete development gives a reusable tail-sum calculus for moments of N\mathbb NN-valued random variables in [0,∞][0,\infty][0,∞], a residual-life construction for discrete distributions, and the BMRL property of three standard families. The platform's mean residual life order (the "Stochastic Orders II" mission, Shaked–Shanthikumar) compares two variables; BMRL is a uniform bound on one variable's residual lives and is not an order, so none of that material states these results.

Difficulty

BMRL controls only first moments, of the conditional laws YsY_sYs​; the goal asks for moments of every order of YYY itself. Bounding E[Yk]E[Y^k]E[Yk] by expanding E[Ys]E[Y_s]E[Ys​] for each fixed sss gives nothing, because each single bound is compatible with a heavy tail: the uniformity in sss is essential. The residual lives are also only defined where P(Y>s)>0P(Y > s) > 0P(Y>s)>0, so every argument must handle distributions with bounded support separately. Proposition 9.2.6 requires explicit control of ratios of tail sums for three families; for the negative binomial and truncated Poisson the tails have no closed form.

Formalization scope

  • YYY is represented by its law, a function u:N→[0,∞]u : \mathbb N \to [0,\infty]u:N→[0,∞] with ∑yuy=1\sum_y u_y = 1∑y​uy​=1 and u0=0u_0 = 0u0​=0 (IsDistOnPos). F∗F^*F∗, moments and residual-life moments are ℝ≥0∞-valued series; an infinite moment is +∞+\infty+∞ and "finite" means <∞< \infty<∞. No Bochner integral is used, so a finite-moment conclusion cannot hold vacuously through an integrability default.
  • The residual life YsY_sYs​ is defined by (9.7) and is used only where F∗(s)>0F^*(s) > 0F∗(s)>0; BMRL-UUU is required exactly at those sss, and UUU is a finite nonnegative real. A formalization requiring the bound at every sss with a junk value of E[Ys]E[Y_s]E[Ys​] where F∗(s)=0F^*(s) = 0F∗(s)=0 is ruled out: the definitions never divide by F∗(s)=0F^*(s) = 0F∗(s)=0 in a used position, and bounded distributions remain BMRL.
  • The geometric and negative binomial laws count trials (support starting at 111 and rrr), not failures as Mathlib's geometricPMF does.
  • Lemma 9.5.2 is stated on a probability space with measurable, mutually independent (iIndepFun) batch sizes of common law ppp, expectations as lower Lebesgue integrals, and only assumption (BA1), λ(2)<∞\lambda^{(2)} < \inftyλ(2)<∞, which is the part of the book's (BA) that concerns arrivals.
  • Welcome contributions: the tail-sum identity (9.4) and its reindexing lemmas, the residual-life tail formula (9.8) and moment formula (9.9), each as a separate lemma; and proofs that the three named families are probability distributions on their supports.

Selected references

  • Linn I. Sennott, Stochastic Dynamic Programming and the Control of Queueing Systems, Wiley Series in Probability and Statistics, John Wiley & Sons, 1999, Section 9.2 (pp. 202–206) and Section 9.5 (pp. 214–215). DOI 10.1002/9780470317037
  • Moshe Shaked and J. George Shanthikumar, Stochastic Orders, Springer Series in Statistics, Springer, 2007, Section 2.A (the mean residual life order). DOI 10.1007/978-0-387-34675-5
8 thms3 active usersReviewed
🏆Completed
Dynamic ProgrammingMarkov ChainOperations Research·Captain: mikedeng1

Stochastic Dynamic Programming and the Control of Queueing Systems VIII: Computing Average Cost Optimal Policies by Approximating SequencesTextbook

Motivation

Control problems for queueing systems (admission control, service rate control, routing to parallel servers) are naturally modelled as Markov decision chains whose state is a vector of queue lengths. The state space is therefore denumerably infinite, and the performance measure of interest is usually the long-run average cost per slot. For such models the existence theory of average cost optimal stationary policies is well developed (Chapter 7 of Sennott's book), but existence gives no algorithm: an optimal policy is a function on an infinite set, and value iteration cannot be run on an infinite state space.

The approximating sequence method answers this by replacing the infinite model Δ\DeltaΔ with a sequence of finite models ΔN\Delta_NΔN​ on truncated state spaces SNS_NSN​, solving the average cost optimality equation in each, and passing to the limit. Chapter 8 of L. I. Sennott, Stochastic Dynamic Programming and the Control of Queueing Systems (Wiley, 1999) gives a set of conditions, the (AC) assumptions, under which this limit procedure provably produces the minimum average cost and an average cost optimal policy of Δ\DeltaΔ.

Timeline. The approximating sequence method for the average cost criterion and the (AC) assumptions were introduced in Sennott (1997a), with further results in Sennott (1997b) (bibliographic notes, p. 194). The book collects these results, adds the four step verification template (Proposition 8.2.1), the finite-set augmentation route based on the (BOR) assumptions (Proposition 8.2.3), and the weakening (WAC) of Section 8.7, which Chapter 9 uses.

Setting

An MDC Δ\DeltaΔ has a countable state space SSS, a finite nonempty action set AiA_iAi​ in each state iii, a finite cost C(i,a)≥0C(i,a)\ge0C(i,a)≥0, and transition probabilities Pij(a)P_{ij}(a)Pij​(a). A general policy θ\thetaθ chooses actions at random using the whole past history. Its average cost is

Jθ(i)=lim sup⁡n→∞1n∑t=0n−1Eθ[C(Xt,At)∣X0=i],J_\theta(i)=\limsup_{n\to\infty}\frac1n\sum_{t=0}^{n-1}E_\theta[C(X_t,A_t)\mid X_0=i],Jθ​(i)=n→∞limsup​n1​t=0∑n−1​Eθ​[C(Xt​,At​)∣X0​=i],

and the minimum average cost is J(i)=inf⁡θJθ(i)∈[0,∞]J(i)=\inf_\theta J_\theta(i)\in[0,\infty]J(i)=infθ​Jθ​(i)∈[0,∞]. A policy is average cost optimal if Jθ≡JJ_\theta\equiv JJθ​≡J.

An approximating sequence (ΔN)N≥N0(\Delta_N)_{N\ge N_0}(ΔN​)N≥N0​​ consists of finite sets SNS_NSN​ increasing to SSS and MDCs ΔN\Delta_NΔN​ on SNS_NSN​ with the same actions and costs and with transition probabilities Pij(a;N)P_{ij}(a;N)Pij​(a;N) on SNS_NSN​ converging to Pij(a)P_{ij}(a)Pij​(a). Write vnNv^N_nvnN​ and VαNV^N_\alphaVαN​ for the nnn-horizon and discounted value functions of ΔN\Delta_NΔN​.

The (AC) assumptions are:

  • (AC1) there are finite constants JNJ^NJN and finite functions rNr^NrN on SNS_NSN​ with
JN+rN(i)=min⁡a{C(i,a)+∑j∈SNPij(a;N) rN(j)},i∈SN;(8.1)J^N+r^N(i)=\min_a\Big\{C(i,a)+\sum_{j\in S_N}P_{ij}(a;N)\,r^N(j)\Big\},\qquad i\in S_N; \tag{8.1}JN+rN(i)=amin​{C(i,a)+j∈SN​∑​Pij​(a;N)rN(j)},i∈SN​;(8.1)
  • (AC2) lim sup⁡NrN(i)<∞\limsup_N r^N(i)<\inftylimsupN​rN(i)<∞;
  • (AC3) lim inf⁡NrN(i)≥−Q\liminf_N r^N(i)\ge -QliminfN​rN(i)≥−Q for a constant Q≥0Q\ge0Q≥0;
  • (AC4) J∗:=lim sup⁡NJN<∞J^*:=\limsup_N J^N<\inftyJ∗:=limsupN​JN<∞ and J∗≤J(i)J^*\le J(i)J∗≤J(i) for all iii.

The (WAC) assumptions of Section 8.7 allow QQQ to depend on the state, at the price of integrability conditions along every stationary policy.

Formalization targets

Goal: Theorem 8.1.1

Under (AC), the limit lim⁡N→∞JN\lim_{N\to\infty}J^NlimN→∞​JN exists and

J(i)=lim⁡N→∞JNfor all i∈S,J(i)=\lim_{N\to\infty}J^N\qquad\text{for all } i\in S,J(i)=N→∞lim​JNfor all i∈S,

and every limit point e∗e^*e∗ of stationary policies eNe^NeN realizing the minimum in (8.1) is average cost optimal for Δ\DeltaΔ. The statement fixes no constants; it asserts the shape of the conclusion for any model satisfying (AC).

Milestones

  • Proposition 8.2.1 (the four step template): unichain and aperiodicity of the finite models, an xxx standard policy at which the approximating sequence is conforming, a comparison of vnNv^N_nvnN​ (or VαNV^N_\alphaVαN​) with vnv_nvn​ (or VαV_\alphaVα​), and a lower bound on vnN−vnN(x)v^N_n - v^N_n(x)vnN​−vnN​(x) together imply that the value iteration limits
rN(i)=lim⁡n→∞(vnN(i)−vnN(x))r^N(i)=\lim_{n\to\infty}\big(v^N_n(i)-v^N_n(x)\big)rN(i)=n→∞lim​(vnN​(i)−vnN​(x))

exist and satisfy (AC).

  • Corollary 8.2.2: on S={0,1,2,… }S=\{0,1,2,\dots\}S={0,1,2,…} with SN={0,…,N}S_N=\{0,\dots,N\}SN​={0,…,N} and excess probability sent to NNN, monotonicity of vnNv^N_nvnN​, vnv_nvn​ and of the first passage moments of a 000 standard policy suffices.
  • Proposition 8.2.3: an augmentation type approximating sequence that sends excess probability to a finite set of cheap states satisfies the template.
  • Proposition 8.5.1: in the single-server queue with Bernoulli(ppp) arrivals and constant service rate a>pa>pa>p,
Jd(a)=Hp(1−p)a−p+pC(a)a.J_{d(a)}=\frac{Hp(1-p)}{a-p}+\frac{pC(a)}{a}.Jd(a)​=a−pHp(1−p)​+apC(a)​.
  • Proposition 8.7.1: the conclusions of Theorem 8.1.1 hold under (WAC).

Significance

The result itself. Theorem 8.1.1 is what turns the existence theory of Chapter 7 into a computation. It certifies that the minimum average costs of the truncations converge to the minimum average cost of the infinite model, that this cost is constant, and that the policies produced by value iteration on ΔN\Delta_NΔN​ converge, along subsequences, to an optimal policy for Δ\DeltaΔ. Propositions 8.2.1–8.2.3 reduce (AC) to properties that can be checked model by model; Section 8.3 checks them for queues with reject option, service rate control, and routing to parallel queues. Proposition 8.7.1 is the version used in Chapter 9 for models whose relative values are not uniformly bounded below. Proposition 8.5.1 gives the closed-form open-loop benchmark used in the numerical study of Section 8.5.

Formalizing it. All results are proved in the book; none has a machine-checked proof. A formalization would give the first verified convergence theorem for truncations of denumerable-state average cost MDPs, and would make the approximating sequence method usable as a certified reduction from infinite to finite models. The template results (8.2.1–8.2.3) additionally require a formal treatment of conformity of approximating Markov chains (Appendix C.4–C.5), which is of independent use.

Difficulty

The naive argument takes limits in (8.1) along NNN: the minimum over aaa and the finite sums pass to the limit only in the inequality direction, and only after a Fatou-type lemma for sums against the converging distributions Pij(a;N)P_{ij}(a;N)Pij​(a;N) with integrands rNr^NrN that are neither bounded nor monotone. The lower bound −Q-Q−Q in (AC3) is exactly what makes this possible; without it the limit inequality can fail. The limit inequality then produces only an average cost optimality inequality, and turning it into optimality of the limit policy requires a separate argument that a function bounded below and satisfying the inequality yields an upper bound on the average cost. Existence of lim⁡NJN\lim_N J^NlimN​JN is not given: (AC4) controls only the limit superior, and the limit must be identified through every subsequence. For the template results, the difficulty is in the Markov chain side: the convergence of first passage times and costs of the truncated chains, which fails for general approximating sequences (Examples C.4.4, C.4.7).

Formalization scope

The Lean development is in the namespace SennottDP.AvgASM. The state space is any countable type; action sets are Finsets, assumed nonempty; costs are finite and nonnegative (ℝ≥0); transition probabilities are ℝ≥0∞-valued, with each row a probability distribution for admissible actions. All value functions and average costs take values in [0,∞][0,\infty][0,∞] (ℝ≥0∞), and every infimum over policies ranges over the full class of history-dependent randomized policies. The JNJ^NJN and rNr^NrN of (AC1) are real; the limits superior and inferior over NNN in (AC2)–(AC4) and (WAC) are taken in EReal, so an unbounded sequence cannot produce a junk finite value. The equality J(i)=lim⁡NJNJ(i)=\lim_N J^NJ(i)=limN​JN is stated in EReal, which also asserts that J(i)J(i)J(i) is finite. Quantities of ΔN\Delta_NΔN​ at states outside SNS_NSN​ are junk values that affect only finitely many NNN for each state.

A trivializing formalization is ruled out: JNJ^NJN and rNr^NrN are the witnesses of (AC1), not free variables, the policies eNe^NeN must realize the minimum in (8.1) for those witnesses, and the minimum average cost JJJ is an infimum over all policies, so the goal cannot be satisfied by choosing J∗J^*J∗ or the limit policy.

A complete development needs: the induced process law of a general policy; the average cost optimality inequality argument (Lemma 7.2.1); the finite-state average cost results of Chapter 6 (Propositions 6.4.1, 6.5.1, 6.6.3); a Fatou lemma for converging distributions (Proposition A.2.5); limit points of policy sequences (Proposition B.5); and, for the template results, the theory of zzz standard chains and conformity (Appendix C.2–C.5). The Markov chain layer and the approximating sequence definitions are reusable beyond this mission. Contributions of any of these intermediate results as separate theorems are welcome.

Selected references

  • L. I. Sennott, Stochastic Dynamic Programming and the Control of Queueing Systems, Wiley Series in Probability and Statistics, John Wiley & Sons, 1999. https://doi.org/10.1002/9780470317037
  • L. I. Sennott, "The computation of average optimal policies in denumerable state Markov decision chains", Advances in Applied Probability 29 (1997), 114–137 (cited as Sennott (1997a) in the book).
  • L. I. Sennott, "On computing average cost optimal policies with application to routing to parallel queues", ZOR — Mathematical Methods of Operations Research 45 (1997), 45–62 (cited as Sennott (1997b) in the book).
14 thms3 active users
🏆Completed
Dynamic ProgrammingMarkov ChainOperations Research·Captain: mikedeng1

Stochastic Dynamic Programming and the Control of Queueing Systems IV: Average Cost Optimal Stationary Policies Exist for Finite State SpacesTextbook

Why average cost on finite state spaces

Controlled queues, inventories and communication links are run for a long time, and the quantity an operator usually cares about is the long-run average cost per period rather than a discounted total. The average cost criterion is harder to work with than the discounted one: its value is a lim sup⁡\limsuplimsup of Cesàro means, it is not given by a contraction, and for general (history dependent, randomized) policies the limit need not exist. Chapter 6 of L. I. Sennott, Stochastic Dynamic Programming and the Control of Queueing Systems (Wiley, 1999) treats the case of a finite state space, where the strongest results hold: an average cost optimal policy exists, can be taken stationary, and can be obtained as a limit of discount optimal policies as the discount factor tends to one.

The results go back to D. Blackwell, "Discrete dynamic programming", Ann. Math. Statist. 33 (1962), who showed that for finite states and actions some stationary policy is discount optimal for all discount factors close to one. Such a policy is now called Blackwell optimal. Sennott's Chapter 6 derives average cost optimality of this policy and the multichain average cost optimality equation from it, in the notation used throughout the book.

Setting

A Markov decision chain (MDC) Δ\DeltaΔ has a countable state space SSS, a finite nonempty action set AiA_iAi​ in each state iii, nonnegative finite costs C(i,a)C(i,a)C(i,a), and transition probabilities Pij(a)P_{ij}(a)Pij​(a) with ∑jPij(a)=1\sum_j P_{ij}(a) = 1∑j​Pij​(a)=1. A policy θ\thetaθ chooses the action at time ttt from a distribution θ(⋅∣ht)\theta(\cdot \mid h_t)θ(⋅∣ht​) on AitA_{i_t}Ait​​ that may depend on the whole history ht=(i0,a0,…,it)h_t = (i_0,a_0,\ldots,i_t)ht​=(i0​,a0​,…,it​). A stationary policy fff always chooses a fixed action f(i)∈Aif(i) \in A_if(i)∈Ai​ in state iii.

With Xt,AtX_t, A_tXt​,At​ the state and action at time ttt and X0=iX_0 = iX0​=i, define

  • the discounted cost Vθ,α(i)=∑t≥0αtEθ[C(Xt,At)]V_{\theta,\alpha}(i) = \sum_{t \ge 0} \alpha^t E_\theta[C(X_t,A_t)]Vθ,α​(i)=∑t≥0​αtEθ​[C(Xt​,At​)] for 0<α<10<\alpha<10<α<1, and the discounted value function Vα(i)=inf⁡θVθ,α(i)V_\alpha(i) = \inf_\theta V_{\theta,\alpha}(i)Vα​(i)=infθ​Vθ,α​(i);
  • the nnn horizon cost vθ,n(i)=∑t=0n−1Eθ[C(Xt,At)]v_{\theta,n}(i) = \sum_{t=0}^{n-1} E_\theta[C(X_t,A_t)]vθ,n​(i)=∑t=0n−1​Eθ​[C(Xt​,At​)];
  • the average cost Jθ(i)=lim sup⁡nvθ,n(i)/nJ_\theta(i) = \limsup_n v_{\theta,n}(i)/nJθ​(i)=limsupn​vθ,n​(i)/n, its lim inf⁡\liminfliminf version Jθ∗(i)J^*_\theta(i)Jθ∗​(i), and the minimum average cost J(i)=inf⁡θJθ(i)J(i) = \inf_\theta J_\theta(i)J(i)=infθ​Jθ​(i).

All infima range over all general policies, and every quantity may equal +∞+\infty+∞. A policy is α\alphaα discount optimal if Vθ,α=VαV_{\theta,\alpha} = V_\alphaVθ,α​=Vα​, and average cost optimal if Jθ=JJ_\theta = JJθ​=J.

For a stationary policy fff on a finite state space, the induced Markov chain splits into positive recurrent classes R1,…,RKR_1,\ldots,R_KR1​,…,RK​ and transient states. With pk(i)p_k(i)pk​(i) the probability of reaching RkR_kRk​ from iii, distinguished states zk∈Rkz_k \in R_kzk​∈Rk​, and Wα(i)=∑kpk(i)Vα(zk)W_\alpha(i) = \sum_k p_k(i) V_\alpha(z_k)Wα​(i)=∑k​pk​(i)Vα​(zk​), the relative value function is wα(i)=Vα(i)−Wα(i)w_\alpha(i) = V_\alpha(i) - W_\alpha(i)wα​(i)=Vα​(i)−Wα​(i).

Formalization targets

Goal: Proposition 6.2.3

For an MDC with a finite state space there are α0∈(0,1)\alpha_0 \in (0,1)α0​∈(0,1) and one stationary policy fff such that fff is α\alphaα discount optimal for every α∈(α0,1)\alpha \in (\alpha_0,1)α∈(α0​,1), fff is average cost optimal, and

J(i)=lim⁡α→1−(1−α)Vα(i)=lim⁡n→∞vf,n(i)n,i∈S.J(i) = \lim_{\alpha\to 1^-} (1-\alpha) V_\alpha(i) = \lim_{n\to\infty} \frac{v_{f,n}(i)}{n}, \qquad i \in S.J(i)=α→1−lim​(1−α)Vα​(i)=n→∞lim​nvf,n​(i)​,i∈S.

Milestones

  1. Proposition 4.5.3. For finite SSS and stationary eee, α↦Ve,α(i)\alpha \mapsto V_{e,\alpha}(i)α↦Ve,α​(i) is a finite, continuous, rational function on (0,1)(0,1)(0,1).
  2. Proposition 6.1.1. For every policy on a countable state space,
Jθ∗(i)≤lim inf⁡α→1−(1−α)Vθ,α(i)≤lim sup⁡α→1−(1−α)Vθ,α(i)≤Jθ(i),J^*_\theta(i) \le \liminf_{\alpha\to1^-}(1-\alpha)V_{\theta,\alpha}(i) \le \limsup_{\alpha\to1^-}(1-\alpha)V_{\theta,\alpha}(i) \le J_\theta(i),Jθ∗​(i)≤α→1−liminf​(1−α)Vθ,α​(i)≤α→1−limsup​(1−α)Vθ,α​(i)≤Jθ​(i),

with three equivalent conditions for equality. 3. Proposition 6.2.2. For finite SSS and stationary eee, Je(i)=lim⁡α→1−(1−α)Ve,α(i)=lim⁡nve,n(i)/nJ_e(i) = \lim_{\alpha\to1^-}(1-\alpha)V_{e,\alpha}(i) = \lim_n v_{e,n}(i)/nJe​(i)=limα→1−​(1−α)Ve,α​(i)=limn​ve,n​(i)/n. 4. Proposition 4.5.1, Proposition 4.5.4, Corollary 4.5.5. The power series structure of Vθ,αV_{\theta,\alpha}Vθ,α​ in α\alphaα; monotonicity and left continuity of VαV_\alphaVα​; continuity under bounded costs. 5. Theorem 6.3.1. For the policy fff of the goal, lim⁡α→1−wα(i)=w(i)\lim_{\alpha \to 1^-} w_\alpha(i) = w(i)limα→1−​wα​(i)=w(i) exists, and

J(i)+w(i)=C(i,f)+∑jPij(f)w(j) ≥ min⁡a{C(i,a)+∑jPij(a)w(j)},J(i) + w(i) = C(i,f) + \sum_j P_{ij}(f) w(j) \ \ge\ \min_{a} \Big\{C(i,a) + \sum_j P_{ij}(a) w(j)\Big\},J(i)+w(i)=C(i,f)+j∑​Pij​(f)w(j) ≥ amin​{C(i,a)+j∑​Pij​(a)w(j)},

together with the limit identities (i)–(iii) and the optimality criterion (v). 6. Proposition 6.3.3. Vα(i)=J(i)/(1−α)+w∗(i)+εα(i)V_\alpha(i) = J(i)/(1-\alpha) + w^*(i) + \varepsilon_\alpha(i)Vα​(i)=J(i)/(1−α)+w∗(i)+εα​(i) with εα(i)→0\varepsilon_\alpha(i) \to 0εα​(i)→0 as α→1−\alpha \to 1^-α→1−.

Significance

The goal says that on a finite state space nothing is gained by randomizing or by remembering the past when minimizing average cost, and that the minimum average cost is the vanishing-discount limit of the discounted value function. This justifies computing average cost optimal policies through discounted problems and value iteration, the route taken in the rest of Chapter 6 and, via approximating sequences, for countable state spaces in Chapters 7 and 8. Theorem 6.3.1 supplies an optimality equation without any unichain or communication assumption. The book's Example 6.3.2 shows that the inequality in that equation can be strict, and that a stationary policy attaining the minimum need not be optimal.

The results are classical and proved in the book. No machine-checked version of them is known to exist. The platform has average-reward results for unichain finite MDPs with Markov policies (the Puterman series) and an average-cost optimality equation under recurrence assumptions (the Bertsekas series). Neither covers existence of a Blackwell optimal policy against the class of all history dependent randomized policies, or the multichain equation. A formal development also yields reusable infrastructure: the law of a controlled process under a general policy, first passage quantities of finite chains, and the Abelian inequality between Abel and Cesàro means of a nonnegative sequence.

Difficulty

The obvious argument picks, for each α\alphaα, a stationary discount optimal policy fαf_\alphafα​ and lets α→1\alpha \to 1α→1. Finiteness of the set of stationary policies gives one policy that is optimal along some sequence αn→1\alpha_n \to 1αn​→1, but not on an interval. Excluding infinite switching between two policies requires the analytic structure of α↦Vf,α(i)\alpha \mapsto V_{f,\alpha}(i)α↦Vf,α​(i) (Proposition 4.5.3), which in turn rests on matrix inversion of I−αPI - \alpha PI−αP. Passing from the discounted criterion to the average one requires an Abelian inequality for nonnegative series whose terms may be infinite (Proposition 6.1.1), and comparison against general policies rules out any argument that works only within stationary or Markov policies. For Theorem 6.3.1 the difficulty is the multichain structure: the relative value function has to be assembled class by class from first passage times and costs, and its limit must be identified.

Formalization scope

  • States form a type S; [Countable S] for Section 4.5 and Proposition 6.1.1, [Fintype S] from Section 6.2 on, as in the book. Actions form a type Act with A i : Finset Act nonempty. Costs are in ℝ≥0, transition probabilities in ℝ≥0∞.
  • A general policy is a function of the list of past state-action pairs (most recent first) and the current state, giving a distribution on A i. Stationary policies embed as degenerate policies. The law of the process is built from this data, and every infimum ranges over all general policies.
  • Vθ,αV_{\theta,\alpha}Vθ,α​, VαV_\alphaVα​, vθ,nv_{\theta,n}vθ,n​, JθJ_\thetaJθ​, Jθ∗J^*_\thetaJθ∗​, JJJ are in ℝ≥0∞, so +∞+\infty+∞ is represented. α→1−\alpha \to 1^-α→1− is the filter 𝓝[<] 1. On a finite state space these quantities are finite. The real valued objects of Section 6.3 (wαw_\alphawα​, www, w∗w^*w∗, equation (6.6)) are therefore formed with toReal, and this switch from ℝ≥0∞ to ℝ happens only in Theorem 6.3.1 and Proposition 6.3.3.
  • The objects of Section 6.3 (pkp_kpk​, mi∣km_{i|k}mi∣k​, ci∣kc_{i|k}ci∣k​, πs\pi_sπs​, WαW_\alphaWα​) are defined from fff. The distinguished states are a hypothesis quantified over.
  • A trivializing formalization would take the infimum over stationary policies only, let the optimal policy depend on α\alphaα, or state rationality as an equation p/q without requiring q≠0q \ne 0q=0. Each is excluded here: JJJ and VαV_\alphaVα​ are infima over all general policies, one pair (α0,f)(\alpha_0,f)(α0​,f) is quantified before all α\alphaα, and the denominator is required to be nonzero on (0,1)(0,1)(0,1).

Useful infrastructure includes rational functions of one real variable and their finitely many sign changes, the resolvent (I−αP)−1(I-\alpha P)^{-1}(I−αP)−1 of a stochastic matrix, the Abelian inequality for [0,∞][0,\infty][0,∞]-valued sequences, and renewal-reward identities for finite chains. Contributions of general lemmas on these topics are welcome, as are proofs of individual milestones.

Selected references

  • L. I. Sennott, Stochastic Dynamic Programming and the Control of Queueing Systems, Wiley, 1999. https://doi.org/10.1002/9780470317037
  • D. Blackwell, "Discrete dynamic programming", Annals of Mathematical Statistics 33 (1962), 719–726. https://doi.org/10.1214/aoms/1177704593
  • M. L. Puterman, Markov Decision Processes: Discrete Stochastic Dynamic Programming, Wiley, 1994. https://doi.org/10.1002/9780470316887
13 thms3 active usersReviewed
🏆Completed
Dynamic ProgrammingOperations ResearchStochastic Systems·Captain: mikedeng1

Stochastic Dynamic Programming and the Control of Queueing Systems II: The Discount Optimality EquationTextbook

Motivation

Control problems for queueing systems (admission control, routing, service rate selection, inventory replenishment) are naturally modelled as Markov decision chains with a countable state space, such as the number of customers in a buffer, and with costs that grow without bound in the state, such as holding costs proportional to queue length. The expected discounted cost criterion is the first infinite horizon criterion applied to such models, and it is also the tool through which the average cost criterion is treated later in the same book (Chapters 6–8 of Sennott's text reach average cost optimal policies through limits of discounted problems as the discount factor tends to one).

Classical treatments of discounted dynamic programming assume bounded costs, under which the dynamic programming operator is a contraction and has a unique bounded fixed point. That assumption fails for queueing models. This mission formalizes Chapter 4, Sections 4.1–4.4, of L. I. Sennott, Stochastic Dynamic Programming and the Control of Queueing Systems (Wiley, 1999), which develops the discounted theory for nonnegative, possibly unbounded costs, where value functions may be infinite.

Timeline of the underlying theory:

  • 1965. Blackwell (Ann. Math. Statist. 36) establishes the discounted theory with bounded rewards.
  • 1966. Strauch (Ann. Math. Statist. 37) treats "negative" dynamic programming, the case of nonpositive rewards (equivalently nonnegative costs), with no boundedness assumption.
  • 1977–1978. Bertsekas (SIAM J. Control Optim. 15) and Bertsekas and Shreve (Stochastic Optimal Control: The Discrete-Time Case) give the abstract monotone-mapping framework covering both cases.
  • 1999. Sennott's text states the countable-state, finite-action, nonnegative-cost discounted theory in the form used for queueing control, with general history-dependent randomized policies.

Setting

A Markov decision chain Δ\DeltaΔ has a countable state space SSS; for each state iii a finite nonempty action set AiA_iAi​; for each a∈Aia \in A_ia∈Ai​ a nonnegative finite cost C(i,a)C(i,a)C(i,a) and a probability distribution (Pij(a))j∈S(P_{ij}(a))_{j \in S}(Pij​(a))j∈S​ of the next state. A policy θ\thetaθ chooses the action at time nnn at random from a distribution θ(⋅∣hn)\theta(\cdot \mid h_n)θ(⋅∣hn​) on AinA_{i_n}Ain​​ that may depend on the entire history hn=(i0,a0,…,an−1,in)h_n = (i_0, a_0, \dots, a_{n-1}, i_n)hn​=(i0​,a0​,…,an−1​,in​). A stationary policy fff always chooses f(i)∈Aif(i) \in A_if(i)∈Ai​ in state iii; for it one writes C(i,f)=C(i,f(i))C(i,f) = C(i,f(i))C(i,f)=C(i,f(i)) and Pij(f)=Pij(f(i))P_{ij}(f) = P_{ij}(f(i))Pij​(f)=Pij​(f(i)).

Fix a discount factor α∈(0,1)\alpha \in (0,1)α∈(0,1). For an initial state iii and a policy θ\thetaθ, the nnn-horizon cost with terminal cost zero and the infinite horizon discounted cost are

vθ,α,n(i)=∑t=0n−1αtEθ[C(Xt,At)∣X0=i],Vθ,α(i)=∑t=0∞αtEθ[C(Xt,At)∣X0=i],v_{\theta,\alpha,n}(i) = \sum_{t=0}^{n-1} \alpha^t E_\theta[C(X_t,A_t) \mid X_0 = i], \qquad V_{\theta,\alpha}(i) = \sum_{t=0}^{\infty} \alpha^t E_\theta[C(X_t,A_t) \mid X_0 = i],vθ,α,n​(i)=t=0∑n−1​αtEθ​[C(Xt​,At​)∣X0​=i],Vθ,α​(i)=t=0∑∞​αtEθ​[C(Xt​,At​)∣X0​=i],

and the value functions are vα,n(i)=inf⁡θvθ,α,n(i)v_{\alpha,n}(i) = \inf_\theta v_{\theta,\alpha,n}(i)vα,n​(i)=infθ​vθ,α,n​(i) and Vα(i)=inf⁡θVθ,α(i)V_\alpha(i) = \inf_\theta V_{\theta,\alpha}(i)Vα​(i)=infθ​Vθ,α​(i), infima over all policies. All of these lie in [0,∞][0,\infty][0,∞]. A policy is discount optimal if Vθ,α=VαV_{\theta,\alpha} = V_\alphaVθ,α​=Vα​. The discount optimality equation is

W(i)=min⁡a∈Ai{C(i,a)+α∑jPij(a)W(j)},i∈S.(4.9)W(i) = \min_{a \in A_i} \Big\{ C(i,a) + \alpha \sum_j P_{ij}(a) W(j) \Big\}, \qquad i \in S. \tag{4.9}W(i)=a∈Ai​min​{C(i,a)+αj∑​Pij​(a)W(j)},i∈S.(4.9)

With W=VαW = V_\alphaW=Vα​, Bi(α)B_i(\alpha)Bi​(α) denotes the set of actions attaining the minimum at iii.

Formalization targets

Goal: Theorem 4.1.4

VαV_\alphaVα​ solves (4.9); every W:S→[0,∞]W : S \to [0,\infty]W:S→[0,∞] solving (4.9) satisfies Vα≤WV_\alpha \le WVα​≤W; and every stationary policy fαf_\alphafα​ with

C(i,fα)+α∑jPij(fα)Vα(j)=min⁡a{C(i,a)+α∑jPij(a)Vα(j)}for all iC(i,f_\alpha) + \alpha \sum_j P_{ij}(f_\alpha) V_\alpha(j) = \min_a \Big\{ C(i,a) + \alpha \sum_j P_{ij}(a) V_\alpha(j) \Big\} \quad \text{for all } iC(i,fα​)+αj∑​Pij​(fα​)Vα​(j)=amin​{C(i,a)+αj∑​Pij​(a)Vα​(j)}for all i

is discount optimal. No boundedness of costs and no finiteness of VαV_\alphaVα​ is assumed.

Milestones

In attack order: Lemma 4.1.1 (vθ,α,n↑Vθ,αv_{\theta,\alpha,n} \uparrow V_{\theta,\alpha}vθ,α,n​↑Vθ,α​); Proposition 4.1.2 (a supersolution of the one-policy equation dominates ve,α,n+αnEe[W(Xn)]v_{e,\alpha,n} + \alpha^n E_e[W(X_n)]ve,α,n​+αnEe​[W(Xn​)] and Ve,αV_{e,\alpha}Ve,α​); Corollary 4.1.3 (a supersolution of the optimality inequality dominates Vf,α≥VαV_{f,\alpha} \ge V_\alphaVf,α​≥Vα​); then, beyond the goal, Corollary 4.1.5 (αnEfα[Vα(Xn)∣X0=i]→0\alpha^n E_{f_\alpha}[V_\alpha(X_n) \mid X_0 = i] \to 0αnEfα​​[Vα​(Xn​)∣X0​=i]→0 where Vα(i)<∞V_\alpha(i) < \inftyVα​(i)<∞), Proposition 4.2.2 and Corollary 4.2.4 (conditions under which a solution of (4.9) equals VαV_\alphaVα​), Proposition 4.3.1 (vα,n↑Vαv_{\alpha,n} \uparrow V_\alphavα,n​↑Vα​, and limit points of finite horizon optimal stationary policies are discount optimal) and Proposition 4.4.1 (optimal policies are exactly those concentrated on the sets Bi(α)B_{i}(\alpha)Bi​(α) along histories of positive probability).

Significance

Theorem 4.1.4 is the foundation for everything in the book that concerns discounted costs: it produces an optimal stationary deterministic policy, identifies VαV_\alphaVα​ among the many solutions of (4.9) (Example 4.2.1 of the book gives a one-parameter family of finite solutions), and underlies value iteration (Proposition 4.3.1) and the approximating-sequence method of Sections 4.6–4.7. The average cost results of Chapters 6–8 are proved from it by letting α→1\alpha \to 1α→1. Proposition 4.4.1 describes the full set of optimal policies, including randomized and history-dependent ones.

These are known results with published proofs. The contribution of this mission is a machine-checked development of the discounted theory for countable state spaces with unbounded costs and infinite values, over the general policy class. Related statements on the platform (the monotone-mapping propositions of Bertsekas 1977 in the MonotoneDP missions, and bounded-cost or finite-state discounted results) use different models and are open; no machine-checked proof of the present statements is known to this mission.

Difficulty

The contraction argument that settles the bounded case is unavailable: with unbounded costs the operator in (4.9) has many fixed points, and VαV_\alphaVα​ can equal +∞+\infty+∞ at some states, so neither uniqueness of fixed points nor subtraction of values is available. The optimality equation compares the infimum over all history-dependent randomized policies with a one-step minimum, so the general policy class and the law of the process under it must be handled directly; restricting attention to Markov or stationary policies begs the question. Every limit exchange (monotone limits of finite horizon costs, the passage to limit points of policies in Proposition 4.3.1) takes place in [0,∞][0,\infty][0,∞], where finite-valued arguments do not transfer verbatim.

Formalization scope

The Lean development lives in the namespace SennottDP.Discounted. Conventions:

  • The state space is a type S with [Countable S]; actions form a type Act and A i : Finset Act is nonempty. Costs are ℝ≥0; transition probabilities are ℝ≥0∞ with ∑' j, P i a j = 1 for a ∈ A i.
  • A history at time nnn is a pair Fin (n+1) → S, Fin n → Act; a policy assigns to every history a distribution on the action set of its last state. The probability of a history is the product of the policy and transition probabilities; expectations are ℝ≥0∞ sums over histories, so no integrability conditions arise.
  • All values (vθ,α,nv_{\theta,\alpha,n}vθ,α,n​, Vθ,αV_{\theta,\alpha}Vθ,α​, vα,nv_{\alpha,n}vα,n​, VαV_\alphaVα​, and the competing solutions WWW) are ℝ≥0∞-valued; 0⋅∞=00 \cdot \infty = 00⋅∞=0. The discount factor is α : ℝ≥0 with 0 < α and α < 1. Terminal costs are zero.
  • VαV_\alphaVα​ and vα,nv_{\alpha,n}vα,n​ are infima over the type of all general policies. Defining them over stationary policies only would make the optimality of fαf_\alphafα​ a tautology; that formalization is ruled out.
  • Proposition 4.4.1: the book states the equivalence without a finiteness assumption, but its necessity argument needs Vα<∞V_\alpha < \inftyVα​<∞, and necessity fails otherwise. Sufficiency is stated in general and necessity under Vα<∞V_\alpha < \inftyVα​<∞ everywhere.

Useful infrastructure, reusable by the later missions of this series (approximating sequences, average cost): the shift of a general policy after its first step, the Chapman–Kolmogorov identity for the history law, and the computation Ef[W(Xn+1)]=Ef[∑jPXnj(f)W(j)]E_f[W(X_{n+1})] = E_f[\sum_j P_{X_n j}(f) W(j)]Ef​[W(Xn+1​)]=Ef​[∑j​PXn​j​(f)W(j)] for stationary policies. Contributions of such lemmas, and proofs of any milestone, are welcome.

Selected references

  • L. I. Sennott, Stochastic Dynamic Programming and the Control of Queueing Systems, Wiley Series in Probability and Statistics, John Wiley & Sons, 1999, Chapter 4. https://doi.org/10.1002/9780470317037
  • D. Blackwell, Discounted dynamic programming, Annals of Mathematical Statistics 36 (1965), 226–235. https://doi.org/10.1214/aoms/1177700285
  • R. E. Strauch, Negative dynamic programming, Annals of Mathematical Statistics 37 (1966), 871–890. https://doi.org/10.1214/aoms/1177699369
  • D. P. Bertsekas, Monotone mappings with application in dynamic programming, SIAM Journal on Control and Optimization 15 (1977), 438–464. https://doi.org/10.1137/0315031
  • D. P. Bertsekas and S. E. Shreve, Stochastic Optimal Control: The Discrete-Time Case, Academic Press, 1978.
  • M. L. Puterman, Markov Decision Processes: Discrete Stochastic Dynamic Programming, Wiley, 1994. https://doi.org/10.1002/9780470316887
12 thms3 active usersReviewed
🏆Completed
Dynamic ProgrammingOperations ResearchStochastic Systems·Captain: mikedeng1

Stochastic Dynamic Programming and the Control of Queueing Systems I: Finite Horizon Optimality and Approximating SequencesTextbook

Motivation

Controlled queueing systems (admission control, routing, service-rate selection) are naturally modelled as Markov decision chains whose state is a buffer content and therefore ranges over a countably infinite set. Linn Sennott's Stochastic Dynamic Programming and the Control of Queueing Systems (Wiley, 1999, DOI 10.1002/9780470317037) develops the dynamic programming theory for exactly this setting: countable state space, finite action sets, nonnegative and possibly unbounded costs, and value functions that are allowed to be infinite. The book's computational method, the approximating sequence method (ASM), replaces the infinite chain by a sequence of finite truncations and asks when optimal values and policies of the truncations converge to those of the original chain.

This mission is the first of a series on the book. It covers Chapter 3, finite horizon optimization, together with the model of Chapter 2 and three results from Appendices A and B that the chapter uses. The finite horizon theory is the entry point: it is where the book's general policy class, its extended-valued cost criteria and its approximating sequences are first used together.

Setting

A Markov decision chain Δ\DeltaΔ has a countable state space SSS; for each i∈Si \in Si∈S a finite nonempty action set AiA_iAi​; a finite cost C(i,a)≥0C(i,a) \ge 0C(i,a)≥0; and for each a∈Aia \in A_ia∈Ai​ a transition distribution (Pij(a))j∈S(P_{ij}(a))_{j \in S}(Pij​(a))j∈S​. A history at time ttt is ht=(i0,a0,…,it−1,at−1,it)h_t = (i_0, a_0, \dots, i_{t-1}, a_{t-1}, i_t)ht​=(i0​,a0​,…,it−1​,at−1​,it​), and a general policy θ\thetaθ chooses the action at time ttt from a distribution θ(⋅∣ht)\theta(\cdot \mid h_t)θ(⋅∣ht​) on AitA_{i_t}Ait​​: it may use the whole history and may randomize. Stationary policies fff (f(i)∈Aif(i) \in A_if(i)∈Ai​) and deterministic Markov policies (a stationary policy for each time) are special cases.

Fix a finite terminal cost F≥0F \ge 0F≥0 and a discount factor 0<α≤10 < \alpha \le 10<α≤1 (α=1\alpha = 1α=1 is the undiscounted case). The nnn horizon expected discounted cost of θ\thetaθ from initial state iii is

vθ,α,n(i)=∑t=0n−1αtEθ[C(Xt,At)∣X0=i]+αnEθ[F(Xn)∣X0=i],v_{\theta,\alpha,n}(i) = \sum_{t=0}^{n-1} \alpha^t E_\theta[C(X_t,A_t) \mid X_0 = i] + \alpha^n E_\theta[F(X_n) \mid X_0 = i],vθ,α,n​(i)=t=0∑n−1​αtEθ​[C(Xt​,At​)∣X0​=i]+αnEθ​[F(Xn​)∣X0​=i],

and the value function is vα,n(i)=inf⁡θvθ,α,n(i)v_{\alpha,n}(i) = \inf_\theta v_{\theta,\alpha,n}(i)vα,n​(i)=infθ​vθ,α,n​(i) over all general policies. Both may be +∞+\infty+∞. A policy is optimal for the nnn horizon if it attains vα,n(i)v_{\alpha,n}(i)vα,n​(i) at every iii. For n≥1n \ge 1n≥1 put uα,n(i,a)=C(i,a)+α∑jPij(a)vα,n−1(j)u_{\alpha,n}(i,a) = C(i,a) + \alpha \sum_j P_{ij}(a) v_{\alpha,n-1}(j)uα,n​(i,a)=C(i,a)+α∑j​Pij​(a)vα,n−1​(j) and let Bi(α,n)B_i(\alpha,n)Bi​(α,n) be the set of a∈Aia \in A_ia∈Ai​ minimizing it.

An approximating sequence (ΔN)N≥N0(\Delta_N)_{N \ge N_0}(ΔN​)N≥N0​​ has finite nonempty state spaces SNS_NSN​ increasing to SSS, the same actions and costs, and transition distributions Pij(a;N)P_{ij}(a;N)Pij​(a;N) on SNS_NSN​ converging to Pij(a)P_{ij}(a)Pij​(a) as N→∞N \to \inftyN→∞. Its value functions are vα,nNv^N_{\alpha,n}vα,nN​. In an augmentation type approximating sequence, the probability Pir(a)P_{ir}(a)Pir​(a) of leaving SNS_NSN​ to rrr is redistributed over SNS_NSN​ by an augmentation distribution qj(i,a,r,N)q_j(i,a,r,N)qj​(i,a,r,N). Assumption FH(α\alphaα, nnn) requires lim sup⁡Nvα,nN(i)\limsup_N v^N_{\alpha,n}(i)limsupN​vα,nN​(i) to be finite and at most vα,n(i)v_{\alpha,n}(i)vα,n​(i) for every iii. A stationary policy eee is a limit point of stationary policies eNe^NeN if, along a subsequence, eNr(i)=e(i)e^{N_r}(i) = e(i)eNr​(i)=e(i) eventually for each iii.

Formalization targets

Goal: Theorem 3.2.3

For fixed n≥1n \ge 1n≥1,

(∀i: lim⁡N→∞vα,nN(i)=vα,n(i)<∞)  ⟺  FH(α,n),\Big(\forall i:\ \lim_{N\to\infty} v^N_{\alpha,n}(i) = v_{\alpha,n}(i) < \infty\Big) \iff \mathrm{FH}(\alpha,n),(∀i: N→∞lim​vα,nN​(i)=vα,n​(i)<∞)⟺FH(α,n),

and under either condition every limit point ene_nen​ of stationary policies enNe^N_nenN​ with enN(i)∈BiN(α,n)e^N_n(i) \in B^N_i(\alpha,n)enN​(i)∈BiN​(α,n) satisfies en(i)∈Bi(α,n)e_n(i) \in B_i(\alpha,n)en​(i)∈Bi​(α,n) for all i∈Si \in Si∈S.

Milestones

  1. Proposition A.1.1: a probability average of uuu is at least min⁡u\min uminu, with equality iff the distribution is concentrated on the minimizers.
  2. Theorem 3.1.2: the finite horizon optimality equation vα,n(i)=min⁡auα,n(i,a)v_{\alpha,n}(i) = \min_a u_{\alpha,n}(i,a)vα,n​(i)=mina​uα,n​(i,a), and the characterization of all optimal general policies.
  3. Corollary 3.1.4: choosing fn−t(i)∈Bi(α,n−t)f_{n-t}(i) \in B_i(\alpha,n-t)fn−t​(i)∈Bi​(α,n−t) yields an optimal deterministic Markov policy.
  4. Proposition 2.5.6: the augmentation (2.19) defines an approximating distribution.
  5. Lemma 3.2.2: vα,0N→vα,0v^N_{\alpha,0} \to v_{\alpha,0}vα,0N​→vα,0​ and lim inf⁡Nvα,nN≥vα,n\liminf_N v^N_{\alpha,n} \ge v_{\alpha,n}liminfN​vα,nN​≥vα,n​.
  6. Propositions B.3 and B.5: sequences of stationary policies, for Δ\DeltaΔ or for (ΔN)(\Delta_N)(ΔN​), have limit points.
  7. Propositions 3.3.1, 3.3.2 and 3.3.4: three sufficient conditions for FH(α\alphaα, nnn), namely bounded costs, an augmentation sending excess probability to a finite set, and the augmentation inequality (3.20).

Significance

Theorem 3.1.2 is the finite horizon dynamic programming equation in the generality the rest of the book needs: the value function is an infimum over history-dependent randomized policies, and the equation holds with infinite values allowed. Its characterization of optimal policies is Bellman's principle of optimality in necessary-and-sufficient form. Corollary 3.1.4 shows that deterministic Markov policies suffice. The discounted chapter builds on these results, since its value function is the limit of finite horizon ones, and so does the value iteration algorithm of the average cost chapters.

Theorem 3.2.3 is the finite horizon case of the approximating sequence method. It says exactly when finite truncations give the right answer, and it reduces the question to Assumption FH, for which Section 3.3 gives checkable conditions. The same structure (a lim inf inequality, a lim sup assumption, a limit point of optimal truncated policies) recurs for the discounted and the average cost criteria in later chapters.

The results are proved in the book. None of them is formalized: the platform has finite horizon dynamic programming only for Markov policies, abstract monotone mappings or finite reward-maximizing MDPs, and nothing on approximating sequences. A formalization contributes a Lean model of Markov decision chains with general policies and extended-valued criteria, which the later missions of the series restate and can merge with this one.

Difficulty

The obvious proof of the optimality equation conditions on the first action and state and then applies the induction hypothesis to the rest of the trajectory. With general policies the rest of the trajectory is governed by a continuation policy that depends on the first state and action, and the decomposition of the path law into a first step and a continuation must be proved from the definition of the process, not assumed. Infinite values also make the "only if" direction delicate: a strict inequality between expected costs becomes an equality once both sides are infinite.

For approximating sequences, the natural idea is to pass to the limit in the optimality equation of ΔN\Delta_NΔN​. This fails in general. Example 3.2.1 of the book has lim⁡Nv1,2N(0)=2>1=v1,2(0)\lim_N v^N_{1,2}(0) = 2 > 1 = v_{1,2}(0)limN​v1,2N​(0)=2>1=v1,2​(0), because truncation moves probability onto states of high cost and dominated convergence is not available. Only the lim inf inequality holds for free, through a generalized Fatou lemma for approximating distributions. The lim sup side is exactly what Assumption FH supplies. The limit point argument then needs the compactness statement of Appendix B and the fact that a lim inf can be passed through a minimum over a finite set.

Formalization scope

The state space is a type S with [Countable S], the actions a type Act, and A i : Finset Act is nonempty. Costs are ℝ≥0, transition probabilities ℝ≥0∞ summing to 1 over S, and all values and expectations are in ℝ≥0∞, so infima over policies are lattice infima and +∞ is a genuine value. A history is the list of past state–action pairs, most recent first, with the current state, and a policy gives a distribution on A i for every history. Expectations are sums over histories of the path probabilities ∏θ(as∣hs)Pisis+1(as)\prod \theta(a_s \mid h_s) P_{i_s i_{s+1}}(a_s)∏θ(as​∣hs​)Pis​is+1​​(as​), which is the book's (2.6) and (2.9), not the dynamic programming recursion. The discount factor satisfies 0<α≤10 < \alpha \le 10<α≤1 in every statement. An approximating sequence is indexed by N∈NN \in \mathbb NN∈N with a start level N0N_0N0​; its value functions are set to 000 for the finitely many NNN at which a given state is not yet in SNS_NSN​, which does not affect limits.

The optimality equation must not be made definitional by defining vθ,α,nv_{\theta,\alpha,n}vθ,α,n​ or vα,nv_{\alpha,n}vα,n​ through the recursion (3.2). The policy class must not be restricted to deterministic Markov policies either, since that would make the characterization in Theorem 3.1.2 a different statement. Theorem 3.1.2(ii)(2) is stated with the guard vα,n(i)<∞v_{\alpha,n}(i) < \inftyvα,n​(i)<∞; the book omits it, and without it the "only if" direction is false (see the item's note).

A complete development needs the first-step decomposition of the path law under a general policy, the generalized Fatou lemma for approximating distributions (Proposition A.2.5, a milestone of the Appendix A mission of this series), and lim inf / lim sup manipulations in ℝ≥0∞. The model definitions are reusable by every later mission of the series. Contributions are welcome at every milestone, including proofs of the definitional sanity facts (for instance vθ,α,0=Fv_{\theta,\alpha,0} = Fvθ,α,0​=F).

Selected references

  • Linn I. Sennott, Stochastic Dynamic Programming and the Control of Queueing Systems, Wiley Series in Probability and Statistics, John Wiley & Sons, 1999. https://doi.org/10.1002/9780470317037
  • Martin L. Puterman, Markov Decision Processes: Discrete Stochastic Dynamic Programming, Wiley, 1994 (the standard reference for finite horizon dynamic programming with history-dependent randomized policies).
  • Richard Bellman, Dynamic Programming, Princeton University Press, 1957.
14 thms3 active usersReviewed
🏆Completed
Optimal Transport·Captain: Lucas

Monge–Kantorovich Duality (Yao 2023)Research Paper

Motivation

Optimal transport asks for the cheapest way to move one distribution of mass onto another. Monge posed the problem in 1781 for maps; Kantorovich (1942) relaxed it to transference plans (couplings), turning it into an infinite-dimensional linear program with a dual: maximize the total revenue ∫ψ dμ+∫φ dν\int\psi\,d\mu+\int\varphi\,d\nu∫ψdμ+∫φdν of pickup and delivery prices that never exceed the cost, ψ(x)+φ(y)≤c(x,y)\psi(x)+\varphi(y)\le c(x,y)ψ(x)+φ(y)≤c(x,y). The equality of the two values — Monge–Kantorovich duality — underlies much of modern optimal transport, with uses in economics (matching markets, principal–agent problems), probability, and PDE.

This mission formalizes the duality theorem and its proof as presented in Colin Yao, Monge–Kantorovich and Transportation Theory (paper dated September 10, 2023), whose proof follows Villani's Optimal Transport: Old and New and, for the weak inequality, Galichon's Optimal Transport Methods in Economics.

Setting

XXX and YYY are Polish spaces (separable, completely metrizable) with their Borel σ-algebras; μ\muμ and ν\nuν are Borel probability measures on XXX and YYY; c:X×Y→[0,∞)c : X\times Y\to[0,\infty)c:X×Y→[0,∞) is a continuous cost function.

  • A transference plan is a probability measure π\piπ on X×YX\times YX×Y with marginals μ\muμ and ν\nuν; Π(μ,ν)\Pi(\mu,\nu)Π(μ,ν) denotes the set of such plans.
  • The Kantorovich problem is min⁡π∈Π(μ,ν)∫c dπ\min_{\pi\in\Pi(\mu,\nu)}\int c\,d\piminπ∈Π(μ,ν)​∫cdπ.
  • The dual problem is sup⁡{∫ψ dμ+∫φ dν}\sup\{\int\psi\,d\mu+\int\varphi\,d\nu\}sup{∫ψdμ+∫φdν} over bounded continuous ψ,φ\psi,\varphiψ,φ with ψ(x)+φ(y)≤c(x,y)\psi(x)+\varphi(y)\le c(x,y)ψ(x)+φ(y)≤c(x,y) everywhere.
  • A set Γ⊆X×Y\Gamma\subseteq X\times YΓ⊆X×Y is ccc-cyclically monotone if ∑i=1Nc(xi,yi)≤∑i=1Nc(xi,yi+1)\sum_{i=1}^N c(x_i,y_i)\le\sum_{i=1}^N c(x_i,y_{i+1})∑i=1N​c(xi​,yi​)≤∑i=1N​c(xi​,yi+1​) (with yN+1=y1y_{N+1}=y_1yN+1​=y1​) for all finite families of points of Γ\GammaΓ; a plan is ccc-cyclically monotone if it is concentrated on such a set.
  • The ccc-conjugate of ψ\psiψ is ψc(y)=inf⁡x(c(x,y)−ψ(x))\psi^c(y)=\inf_x(c(x,y)-\psi(x))ψc(y)=infx​(c(x,y)−ψ(x)); ψ\psiψ is ccc-concave if ψ(x)=inf⁡y(c(x,y)−φ(y))\psi(x)=\inf_y(c(x,y)-\varphi(y))ψ(x)=infy​(c(x,y)−φ(y)) for some φ\varphiφ; its ccc-subdifferential is ∂cψ={(x,y):ψc(y)+ψ(x)=c(x,y)}\partial_c\psi=\{(x,y):\psi^c(y)+\psi(x)=c(x,y)\}∂c​ψ={(x,y):ψc(y)+ψ(x)=c(x,y)}.

Formalization targets

Goal (Theorem 4.1)

min⁡π∈Π(μ,ν)∫X×Yc dπ=sup⁡ψ∈Cb(X), φ∈Cb(Y)ψ(x)+φ(y)≤c(x,y)(∫Xψ dμ+∫Yφ dν),\min_{\pi\in\Pi(\mu,\nu)}\int_{X\times Y}c\,d\pi=\sup_{\substack{\psi\in C_b(X),\ \varphi\in C_b(Y)\\ \psi(x)+\varphi(y)\le c(x,y)}}\Big(\int_X\psi\,d\mu+\int_Y\varphi\,d\nu\Big),π∈Π(μ,ν)min​∫X×Y​cdπ=ψ∈Cb​(X), φ∈Cb​(Y)ψ(x)+φ(y)≤c(x,y)​sup​(∫X​ψdμ+∫Y​φdν),

including existence of a minimizing plan; the common value may be +∞+\infty+∞.

Milestones (in the order of the source)

  1. Weak duality, inequality (3.1): inf⁡≥sup⁡\inf\ge\supinf≥sup.
  2. Proposition 4.4: a ccc-cyclically monotone plan exists between uniform empirical measures.
  3. Lemma 4.16: plans with marginals in tight families form a tight family.
  4. Proposition 4.17: a ccc-cyclically monotone plan exists for general marginals.
  5. Proposition 4.22: the support of a ccc-cyclically monotone plan lies in ∂cψ\partial_c\psi∂c​ψ for a ccc-concave ψ\psiψ (bounded ccc).
  6. Theorem 4.25: (ψc)c=ψ(\psi^c)^c=\psi(ψc)c=ψ for ccc-concave ψ\psiψ.
  7. Proposition 4.31: duality for bounded continuous ccc.
  8. Theorem 4.32: ∫f dμ=∫f dπ\int f\,d\mu=\int f\,d\pi∫fdμ=∫fdπ for a plan with marginal μ\muμ.

Significance

Duality converts a minimization over measures into a maximization over functions; it characterizes optimal plans by the complementary-slackness condition that they are concentrated on {ψ(x)+φ(y)=c(x,y)}\{\psi(x)+\varphi(y)=c(x,y)\}{ψ(x)+φ(y)=c(x,y)}, and it is the entry point to Brenier's theorem, the Kantorovich–Rubinstein formula for the Wasserstein-1 distance, and the economic applications discussed in Section 5 of the source. The result is classical and proved in the literature; the work here is to formalize the known proof and to supply reusable infrastructure (couplings, ccc-cyclical monotonicity, ccc-transforms) on top of Mathlib's measure theory.

Difficulty

The weak inequality is a short integration argument; the reverse inequality is where the work lies. Finite-dimensional linear-programming duality does not pass to general measures directly: one must produce a cyclically monotone plan as a limit of discrete approximations (tightness and Prokhorov's theorem, closedness of the cyclic-monotonicity condition under weak convergence), build a potential ψ\psiψ from chains of cost differences, and control measurability and integrability of ψ\psiψ and ψc\psi^cψc. Passing from bounded to unbounded nonnegative costs requires an additional approximation argument, which the source only sketches (Section 4.5).

Formalization scope

Lean namespace MongeKantorovichYao; one definition file provides transference plans, ccc-cyclical monotonicity, ccc-conjugates, ccc-concavity and ccc-subdifferentials. Conventions: marginals are pushforwards along the projections; ccc-conjugates are extended-real infima (no default values); the transport cost in the goal is the [0,∞][0,\infty][0,∞]-valued integral of the nonnegative cost, and both sides of the goal are compared in the extended reals, so the infinite-cost case is included and no integrability hypothesis is added. The dual side ranges over bounded continuous functions with the constraint imposed at every point. Proposition 4.22 is stated for bounded ccc (as used in Proposition 4.31), since with unbounded costs a real-valued potential need not exist. Infrastructure that may be missing from Mathlib: Prokhorov-type compactness of tight families of probability measures, weak convergence of empirical measures, and lower semicontinuity of π↦∫c dπ\pi\mapsto\int c\,d\piπ↦∫cdπ.

Selected references

  • C. Villani, Optimal Transport: Old and New, Grundlehren der mathematischen Wissenschaften 338, Springer, 2009. https://doi.org/10.1007/978-3-540-71050-9
  • A. Galichon, Optimal Transport Methods in Economics, Princeton University Press, 2016.
  • L. V. Kantorovich, On the translocation of masses, Dokl. Akad. Nauk SSSR 37 (1942).
10 thms3 active usersReviewed
🏆Completed
Markov ChainOperations ResearchStochastic Systems·Captain: mikedeng1

Analysis and Algorithms for Service Parts Supply Chains VI: The Shortfall Distribution of Capacity-Limited SystemsTextbook

Motivation

Service parts supply chains are often limited by a capacitated resource, such as a production line or a repair shop, instead of by lead times alone. Once capacity binds, the classical tools for setting stock levels (Palm's theorem and the Poisson distribution of units in resupply) no longer apply, and the quantity that determines how much stock is needed is the shortfall: the amount by which the end-of-period inventory falls below its target because capacity was insufficient. Chapter 8 of Muckstadt, Analysis and Algorithms for Service Parts Supply Chains (Springer 2005, DOI 10.1007/b138879) builds its tactical planning models for capacity-limited systems on the distribution of this random variable, and on a continuous-time repair queue in which item counts are geometric.

The shortfall recursion is the Lindley recursion of queueing theory (Lindley 1952), so its stationary law is the law of the maximum of a random walk with negative drift. The exponential tail of that maximum goes back to Cramér's work on ruin probabilities; for capacitated production–inventory systems it was stated by Glasserman (1997), whose theorem the book quotes as Theorem 11. Glasserman and Tayur (1995) used the shortfall to optimize base-stock levels in multi-echelon capacitated systems, and Roundy and Muckstadt (2000) studied the mass-exponential approximation that the theorem motivates.

Setting

A single item is produced in periods n=1,2,…n = 1, 2, \dotsn=1,2,… of an infinite horizon; at most ccc units can be produced per period. The demand of period nnn is DnD_nDn​; the demands are nonnegative, independent and identically distributed, with generic demand DDD and E[D]<cE[D] < cE[D]<c (the standing assumption of Section 8.1.1).

Under the modified (s−1,s)(s-1, s)(s−1,s) policy with target level sss, the facility observes DnD_nDn​ and produces min⁡{c,s−In−1+Dn}\min\{c, s - I_{n-1} + D_n\}min{c,s−In−1​+Dn​} units, where InI_nIn​ is the end-of-period net inventory and I0=sI_0 = sI0​=s. The shortfall Vn=s−InV_n = s - I_nVn​=s−In​ satisfies V0=0V_0 = 0V0​=0 and

Vn=[Vn−1+Dn−c]+.(8.1)V_n = \left[V_{n-1} + D_n - c\right]^+ . \tag{8.1}Vn​=[Vn−1​+Dn​−c]+.(8.1)

With the random walk Sn=∑k=1n(Dk−c)S_n = \sum_{k=1}^{n} (D_k - c)Sn​=∑k=1n​(Dk​−c) (S0=0S_0 = 0S0​=0), the stationary shortfall is

V=sup⁡n≥0Sn.V = \sup_{n \ge 0} S_n .V=n≥0sup​Sn​.

A law on R\mathbb RR is lattice if it is concentrated on a progression a+dZa + d\mathbb Za+dZ with d>0d > 0d>0.

In the discrete case (ccc and DDD integer valued) (Vn)(V_n)(Vn​) is a Markov chain on {0,1,2,… }\{0, 1, 2, \dots\}{0,1,2,…} with transition probabilities pijp_{ij}pij​ (p. 185). In the repair model of Section 8.3.1, reparable units of item iii arrive at rate λi\lambda_iλi​, λ=∑iλi\lambda = \sum_i \lambda_iλ=∑i​λi​, a single exponential server repairs at rate μ>λ\mu > \lambdaμ>λ, NNN is the number of units in repair and NiN_iNi​ the number of item-iii units, and ηi=λi/(μ−λ+λi)\eta_i = \lambda_i/(\mu - \lambda + \lambda_i)ηi​=λi​/(μ−λ+λi​).

Formalization targets

Goal: Theorem 11, corrected (p. 191)

Assume E[eαD]<∞E[e^{\alpha D}] < \inftyE[eαD]<∞ for all α<δ\alpha < \deltaα<δ, with δ>0\delta > 0δ>0; P[D>c]>0P[D > c] > 0P[D>c]>0; the law of DDD is non-lattice; and E[e−α(c−D)]=1E[e^{-\alpha(c-D)}] = 1E[e−α(c−D)]=1 has a root in (0,δ)(0, \delta)(0,δ). Then there are β>0\beta > 0β>0 and α>0\alpha > 0α>0 with

P{V>v}βe−αv→1(v→∞),α the unique positive root of E[e−α(c−D)]=1.\frac{P\{V > v\}}{\beta e^{-\alpha v}} \to 1 \quad (v \to \infty), \qquad \alpha \text{ the unique positive root of } E\left[e^{-\alpha(c - D)}\right] = 1 .βe−αvP{V>v}​→1(v→∞),α the unique positive root of E[e−α(c−D)]=1.

The constant β\betaβ is left unspecified, as in the book.

Milestones, in attack order

  1. Eq. (8.1): under the modified policy, s−In=Vns - I_n = V_ns−In​=Vn​ for every nnn, independently of sss.
  2. Section 8.1.1: V<∞V < \inftyV<∞ almost surely, P{Vn>v}→P{V>v}P\{V_n > v\} \to P\{V > v\}P{Vn​>v}→P{V>v} for every vvv, and the law of VVV is stationary for (8.1).
  3. Eq. (8.2): for v>0v > 0v>0, P{Vn>v}=P{Dn>v+c}+ED[1(d≤v+c) P{Vn−1>v+c−d}]P\{V_n > v\} = P\{D_n > v + c\} + E_D[1(d \le v + c)\, P\{V_{n-1} > v + c - d\}]P{Vn​>v}=P{Dn​>v+c}+ED​[1(d≤v+c)P{Vn−1​>v+c−d}].
  4. Theorem 11, second sentence: E[e−α(c−D)]=1E[e^{-\alpha(c-D)}] = 1E[e−α(c−D)]=1 has at most one positive root.
  5. Section 8.1.2: with integer demand, (Vn)(V_n)(Vn​) is a Markov chain with transition probabilities pijp_{ij}pij​.
  6. Section 8.1.2: πi=lim⁡nP{Vn=i}\pi_i = \lim_n P\{V_n = i\}πi​=limn​P{Vn​=i} exists and solves πP=π\pi\mathcal P = \piπP=π, ∑iπi=1\sum_i \pi_i = 1∑i​πi​=1, πi≥0\pi_i \ge 0πi​≥0.
  7. Section 8.3.1: if NNN is geometric with parameter λ/μ\lambda/\muλ/μ and NiN_iNi​ given N=jN = jN=j is binomial(j,λi/λ)(j, \lambda_i/\lambda)(j,λi​/λ), then P[Ni=j]=(1−ηi)ηijP[N_i = j] = (1 - \eta_i)\eta_i^jP[Ni​=j]=(1−ηi​)ηij​.
  8. Section 8.3.1: ∑j>spi(j)=ηis+1\sum_{j > s} p_i(j) = \eta_i^{s+1}∑j>s​pi​(j)=ηis+1​, and the smallest cost-minimising stock level is the smallest sss with ηis+1≤hi/(hi+b)\eta_i^{s+1} \le h_i/(h_i + b)ηis+1​≤hi​/(hi​+b).

Significance

The exponential tail is the justification the book gives for approximating the shortfall by a mass-exponential law (an atom at zero plus an exponential tail), from which target stock levels and fill rates are computed in closed form. The decay rate α\alphaα depends only on the demand law and the capacity, so the theorem also says how the stock needed for a given service level grows as utilization approaches one. The discrete-chain milestones justify the exact computation of the shortfall distribution behind the book's Table 8.1 and Figures 8.3–8.8. The geometric law of NiN_iNi​ reduces the multi-item repair problem to independent newsvendor problems with an explicit solution.

The asymptotics of the random-walk maximum are proved in the literature (Cramér–Lundberg theory, Feller Vol. II, XII.5; Asmussen, Applied Probability and Queues, XIII.5); no machine-checked proof is known to exist. Mathlib has neither the Lindley recursion, nor ladder-height decompositions, nor the key renewal theorem for non-lattice laws. The printed Theorem 11 is not correct as stated (see Formalization scope), so the mission also records a corrected statement.

Difficulty

The central step of the goal is the passage from the random walk to an exact asymptotic. An exponential change of measure (Esscher tilt) with the root α\alphaα turns P{V>v}P\{V > v\}P{V>v} into an expectation under a law with positive drift, but it only yields the upper bound P{V>v}≤e−αvP\{V > v\} \le e^{-\alpha v}P{V>v}≤e−αv (Lundberg's inequality); it does not show that eαvP{V>v}e^{\alpha v}P\{V > v\}eαvP{V>v} converges, nor that the limit is positive. Convergence needs a renewal theorem for the overshoot of the tilted walk, which fails for lattice laws. That is why the non-lattice hypothesis cannot be dropped. For the milestones, the existence of the stationary law needs the reversal argument that identifies the law of VnV_nVn​ with that of max⁡k≤nSk\max_{k \le n} S_kmaxk≤n​Sk​, plus the strong law of large numbers to show V<∞V < \inftyV<∞ from E[D]<cE[D] < cE[D]<c.

Formalization scope

  • Model. Demands are real, nonnegative, measurable, i.i.d. (iIndepFun plus IdentDistrib with D1D_1D1​), integrable, with E[D]<cE[D] < cE[D]<c; these are fields of ShortfallModel. Periods are numbered from 111 as in the book (demand 0 is an unused i.i.d. copy). The discrete case is a separate structure with N\mathbb NN-valued demand and capacity.
  • Stationary shortfall. The book's "stationary distribution ... Let VVV represent this random variable" is pinned to V=sup⁡n≥0SnV = \sup_{n \ge 0} S_nV=supn≥0​Sn​, taken in [0,∞][0, \infty][0,∞] and converted to a real number; milestone 2 proves that it is the limit law of VnV_nVn​ from V0=0V_0 = 0V0​=0 and a stationary law of (8.1). The discrete πi\pi_iπi​ is pinned to lim⁡nP{Vn=i}\lim_n P\{V_n = i\}limn​P{Vn​=i}.
  • Corrections to Theorem 11. The printed theorem is false. For integer demand P{V>v}P\{V > v\}P{V>v} is a step function, and no βe−αv\beta e^{-\alpha v}βe−αv is asymptotic to it. If E[eαD]E[e^{\alpha D}]E[eαD] is finite only for α<δ\alpha < \deltaα<δ, the equation E[e−α(c−D)]=1E[e^{-\alpha(c-D)}] = 1E[e−α(c−D)]=1 may have no root in (0,δ)(0,\delta)(0,δ). The goal therefore adds two labelled hypotheses: a non-lattice demand law, and a root in (0,δ)(0, \delta)(0,δ). The mass-exponential demand of Section 8.1.3 (an atom at 000 plus a density) is non-lattice. The approximation β≈e−2(.583)(c−E(D))/σ\beta \approx e^{-2(.583)(c-E(D))/\sigma}β≈e−2(.583)(c−E(D))/σ is not stated.
  • Repair model. The M/M/1 queue is not built. The geometric law of NNN (asserted on p. 202) and the binomial split of NNN (quoted from Chapter 3) enter milestone 7 as hypotheses, exactly as the page's proof uses them. The stability condition λ<μ\lambda < \muλ<μ, not written on the page, is a hypothesis. "The optimal sis_isi​" is read as the smallest minimiser of the cost.
  • Ruled out. Stating Theorem 11 with α\alphaα or β\betaβ allowed to depend on vvv, with β=0\beta = 0β=0 (the ratio would be a division by zero, which Lean evaluates to 000), or for a VVV postulated to have an exponential tail proves nothing. Here β,α\beta, \alphaβ,α are quantified before vvv, both are asserted positive, and VVV is constructed from the demands.
  • Not formalized. The mass-exponential approximations (8.3)–(8.4), the Roundy–Muckstadt refinement, the fill-rate formula η(s)\eta(s)η(s) (a definition, whose steady-state identity needs uniform integrability the book does not discuss), the random-capacity chain on p. 186, and the monotonicity of sis_isi​ in μ\muμ.
  • Reusable infrastructure. Welcome: the Lindley recursion and its reversal identity, the Loynes existence theorem, Lundberg's inequality, and a non-lattice renewal theorem. All of these are needed well beyond this mission, in queueing (GI/G/1 waiting times) and ruin theory.

Selected references

  • J. A. Muckstadt, Analysis and Algorithms for Service Parts Supply Chains, Springer, 2005, Chapter 8. https://doi.org/10.1007/b138879
  • P. Glasserman, Bounds and asymptotics for planning critical safety stocks, Operations Research 45(2), 244–257, 1997. https://doi.org/10.1287/opre.45.2.244
  • P. Glasserman and S. Tayur, Sensitivity analysis for base-stock levels in multiechelon production-inventory systems, Management Science 41(2), 263–281, 1995 (the book's reference [97]). https://doi.org/10.1287/mnsc.41.2.263
  • R. O. Roundy and J. A. Muckstadt, Heuristic computation of periodic-review base stock inventory policies, Management Science 46(1), 104–109, 2000. https://doi.org/10.1287/mnsc.46.1.104.15131
  • D. V. Lindley, The theory of queues with a single server, Mathematical Proceedings of the Cambridge Philosophical Society 48(2), 277–289, 1952. https://doi.org/10.1017/S0305004100027638
  • W. Feller, An Introduction to Probability Theory and Its Applications, Vol. II, 2nd ed., Wiley, 1971, Chapter XII.
  • S. Asmussen, Applied Probability and Queues, 2nd ed., Springer, 2003, Chapter XIII. https://doi.org/10.1007/b97236
15 thms3 active usersReviewed
🏆Completed
Operations ResearchStochastic Systems·Captain: mikedeng1

Analysis and Algorithms for Service Parts Supply Chains III: Palm's Theorem for (s–1, s) PoliciesTextbook

Motivation

Service parts (spare engines, avionics modules, repairable components) are usually managed one unit at a time: whenever a customer order removes a unit from stock, a replacement is ordered at once, from a repair shop or an outside supplier. This is the (s–1, s) policy, under which the inventory position (on hand plus on order minus backorders) stays constant at the stock level sss. Every performance measure of such a system (fill rate, expected backorders, availability) is a function of one random variable: the number of units in resupply, i.e. ordered but not yet returned. Chapter 3 of Muckstadt, Analysis and Algorithms for Service Parts Supply Chains (Springer 2005) computes its distribution, and the rest of the book (the METRIC-type multi-echelon models of Chapters 4 and 5, the stock-level optimization of Section 3.4) is built on that computation.

Timeline. C. Palm (1938) showed, in the setting of telephone traffic, that in an infinite-server system with Poisson arrivals the number of busy servers has, in steady state, a Poisson law whose mean is the arrival rate times the mean service time, whatever the service-time distribution. Feeney and Sherbrooke (1966) carried the result to (s–1, s) inventory systems with compound Poisson demand, and treated the lost-sales case. Sherbrooke's METRIC model (1968) made the Poisson law of units in resupply the basis of multi-echelon spare-parts planning.

Setting

A single item is stocked at one location. Customer orders arrive at epochs T0<T1<⋯T_0 < T_1 < \cdotsT0​<T1​<⋯ of a Poisson process with rate λ>0\lambda > 0λ>0, started empty at time 000: the interarrival times AkA_kAk​ are independent and exponential with rate λ\lambdaλ, and Tk=A0+⋯+AkT_k = A_0 + \cdots + A_kTk​=A0​+⋯+Ak​. The kkk-th order triggers a resupply order with resupply time Lk≥0L_k \ge 0Lk​≥0. The resupply times are independent and identically distributed, independent of the arrival process, with a density ggg, distribution function G(u)=P[L≤u]G(u) = P[L \le u]G(u)=P[L≤u] and finite mean

τˉ=E[L]=∫0∞[1−G(u)] du.\bar\tau = E[L] = \int_0^\infty [1 - G(u)]\,du .τˉ=E[L]=∫0∞​[1−G(u)]du.

With backorders allowed, the number of units in resupply at time ttt is

X(t)=#{k:Tk≤t<Tk+Lk},X(t) = \#\{k : T_k \le t < T_k + L_k\},X(t)=#{k:Tk​≤t<Tk​+Lk​},

and N(t)=#{k:Tk≤t}N(t) = \#\{k : T_k \le t\}N(t)=#{k:Tk​≤t} counts the orders placed in [0,t][0,t][0,t]. On-hand stock and backorders at time ttt are (s−X(t))+(s - X(t))^+(s−X(t))+ and (X(t)−s)+(X(t) - s)^+(X(t)−s)+.

In the compound Poisson version the kkk-th order asks for Xk≥1X_k \ge 1Xk​≥1 units, the sizes are i.i.d. with uj=P[Xk=j]u_j = P[X_k = j]uj​=P[Xk​=j], independent of arrivals and resupply times, and all units of one order share its resupply time LkL_kLk​. The units in resupply are Y(t)=∑k:Tk≤t<Tk+LkXkY(t) = \sum_{k : T_k \le t < T_k + L_k} X_kY(t)=∑k:Tk​≤t<Tk​+Lk​​Xk​. Writing un(y)u^{(y)}_nun(y)​ for the probability that yyy orders ask for nnn units in total, the compound Poisson probabilities with parameter μ\muμ are

p(n∣μ)=∑y=0nμye−μy! un(y).p(n \mid \mu) = \sum_{y=0}^{n} \frac{\mu^y e^{-\mu}}{y!}\,u^{(y)}_n .p(n∣μ)=y=0∑n​y!μye−μ​un(y)​.

In the lost-sales version an order that finds no stock on hand is lost, so at most sss units are ever in resupply.

Formalization targets

Goal: Palm's theorem (Theorem 6, p. 39)

For every x≥0x \ge 0x≥0,

lim⁡t→∞P[X(t)=x]=e−λτˉ(λτˉ)xx!.\lim_{t\to\infty} P[X(t) = x] = e^{-\lambda\bar\tau}\frac{(\lambda\bar\tau)^x}{x!}.t→∞lim​P[X(t)=x]=e−λτˉx!(λτˉ)x​.

The resupply-time law enters only through its mean. This is the statement the book's proof establishes and every later chapter uses.

The proof's milestones (pp. 38–41)

  1. Eq. (3.5): P[N(t)=n]=e−λt(λt)n/n!P[N(t) = n] = e^{-\lambda t}(\lambda t)^n/n!P[N(t)=n]=e−λt(λt)n/n!.
  2. Eq. (3.3): given N(t)=nN(t) = nN(t)=n, the epochs (T0,…,Tn−1)(T_0, \dots, T_{n-1})(T0​,…,Tn−1​) have density n!/tnn!/t^nn!/tn on 0<t1<⋯<tn<t0 < t_1 < \cdots < t_n < t0<t1​<⋯<tn​<t.
  3. Eq. (3.7): given N(t)=nN(t) = nN(t)=n, X(t)X(t)X(t) is binomial with parameters nnn and p=1t∫0t[1−G(u)] dup = \frac1t\int_0^t[1-G(u)]\,dup=t1​∫0t​[1−G(u)]du.
  4. Eq. (3.8): for every t>0t > 0t>0, X(t)X(t)X(t) is Poisson with mean λ∫0t[1−G(u)] du\lambda\int_0^t[1-G(u)]\,duλ∫0t​[1−G(u)]du.
  5. Eq. (3.10): ∫0t[1−G(u)] du→τˉ\int_0^t[1-G(u)]\,du \to \bar\tau∫0t​[1−G(u)]du→τˉ.

Extensions in Section 3.1

  1. Theorem 7 (pp. 43–44): with compound Poisson demand, lim⁡t→∞P[Y(t)=n]=p(n∣λτˉ)\lim_{t\to\infty}P[Y(t) = n] = p(n \mid \lambda\bar\tau)limt→∞​P[Y(t)=n]=p(n∣λτˉ).
  2. Theorem 8 (p. 44): in the lost-sales system with exponential resupply times of rate β\betaβ, the probability vectors solving the balance equations are exactly the truncated Poisson law πx∝(λ/β)x/x!\pi_x \propto (\lambda/\beta)^x/x!πx​∝(λ/β)x/x!, 0≤x≤s0 \le x \le s0≤x≤s.
  3. Theorem 9 (pp. 46–47): for a due-date delay T≥0T \ge 0T≥0, the units in resupply that have been there for at least TTT satisfy lim⁡t→∞P[YT(t)=n]=p(n∣λτˉα)\lim_{t\to\infty}P[Y_T(t) = n] = p(n \mid \lambda\bar\tau\alpha)limt→∞​P[YT​(t)=n]=p(n∣λτˉα) with α=1τˉ∫T∞[1−G(t)] dt\alpha = \frac1{\bar\tau}\int_T^\infty[1-G(t)]\,dtα=τˉ1​∫T∞​[1−G(t)]dt.

Significance

The result. Palm's theorem turns an infinite-dimensional object (the whole resupply-time distribution) into one number, τˉ\bar\tauτˉ. This insensitivity is what makes spare-parts planning computable: the expected backorders at stock level sss are ∑x>s(x−s) p(x∣λτˉ)\sum_{x > s}(x - s)\,p(x \mid \lambda\bar\tau)∑x>s​(x−s)p(x∣λτˉ), the fill rate is P[X≤s−1]P[X \le s - 1]P[X≤s−1], and both can be optimized over sss with only the demand rate and mean repair time as data. Theorem 7 extends this to batch demand, Theorem 9 to systems allowed a response time, and Theorem 8 gives the exact law when shortages are lost instead of backordered.

Formalizing it. All of these results are classical and proved. None of them is formalized on the platform, and Mathlib has Poisson and exponential distributions but no Poisson process, no thinning theorem and no infinite-server queue. The mission produces a Poisson arrival stream built from i.i.d. exponential gaps, the conditional-uniformity property of its epochs, independent thinning, and the M/G/∞ transient law, all reusable in queueing and inventory missions.

Difficulty

The algebra of the proof (summing the binomial against the Poisson law of N(t)N(t)N(t)) is short. The work is in the probabilistic step the book treats in a sentence: that, given N(t)=nN(t) = nN(t)=n, the nnn orders behave like independent uniform epochs, each of which independently is still in resupply at time ttt with the same probability ppp. This needs the joint law of the partial sums of exponential variables (Eq. (3.3)), and then a symmetrization argument, since the epochs are ordered while the resupply times are attached to order indices. The naive route of computing P[X(t)=x]P[X(t) = x]P[X(t)=x] by conditioning on individual epochs does not go through without that exchangeability step. The limit t→∞t \to \inftyt→∞ is then elementary; stating a stationary version directly is not a substitute, since the book's "steady state" is exactly this limit.

Formalization scope

The model is a structure on a probability space (Ω,P)(\Omega, P)(Ω,P): exponential interarrival times with rate λ>0\lambda > 0λ>0, nonnegative resupply times with a density and an integrable first coordinate, and mutual independence of the whole family. Orders are indexed from 000, so the book's X1,…,XnX_1, \dots, X_nX1​,…,Xn​ are T0,…,Tn−1T_0, \dots, T_{n-1}T0​,…,Tn−1​. Counts are cardinalities of sets of order indices, with value 000 on the probability-zero event where infinitely many orders fall in a bounded interval.

Commitments and pinnings:

  • "Steady state probability" (Theorems 6, 7, 9) is lim⁡t→∞P[⋅(t)=x]\lim_{t\to\infty}P[\cdot(t) = x]limt→∞​P[⋅(t)=x] for the system empty at time 000, which is what the proofs compute via (3.8)–(3.11).
  • Independence of resupply times from arrivals is not written in Theorem 6 but is used on p. 40; it is part of the model.
  • The stock level sss does not enter the backorder model; it matters only in Theorem 8.
  • Theorem 8 is stated algebraically: a vector on {0,…,s}\{0, \dots, s\}{0,…,s} solves the balance equations (3.26), (3.25) for 0<j<s0 < j < s0<j<s and (3.32), and sums to one, if and only if it is the truncated Poisson law. The book obtains these equations by letting t→∞t \to \inftyt→∞ in the forward equations under the unproved assumption Pj′(t)→0P_j'(t) \to 0Pj′​(t)→0. The book writes (3.25) "for 0≤j≤s0 \le j \le s0≤j≤s", which at j=sj = sj=s contradicts its own (3.32); the boundary equation (3.32) is used. The sentence on p. 46 extending Theorem 8 to arbitrary resupply densities is asserted without proof and is not stated.
  • Theorem 7 identifies the limit law by its probabilities (3.22)–(3.23); its mean λτˉuˉ\lambda\bar\tau\bar uλτˉuˉ is a property of that law. Theorem 9 is stated for compound demand as the book states it, although the book's proof covers only the Poisson case.

A model in which X(t)X(t)X(t) is postulated through its law, or in which resupply times may depend on the arrival epochs, makes the goal empty or false; here X(t)X(t)X(t) is computed from the primitive arrival and resupply times, whose joint law is fully specified.

Welcome contributions: a general Poisson-process library (construction from exponential gaps, Poisson marginals, order-statistics property), independent thinning, and proofs of the milestones in the listed order.

Selected references

  • J. A. Muckstadt, Analysis and Algorithms for Service Parts Supply Chains, Springer Series in Operations Research and Financial Engineering, 2005, Chapter 3. https://doi.org/10.1007/b138879
  • C. Palm, "Analysis of the Erlang traffic formula for busy-signal arrangements", Ericsson Technics 5, 1938, 39–58.
  • G. J. Feeney and C. C. Sherbrooke, "The (s–1, s) inventory policy under compound Poisson demand", Management Science 12(5), 1966, 391–411. https://doi.org/10.1287/mnsc.12.5.391
  • C. C. Sherbrooke, "METRIC: A multi-echelon technique for recoverable item control", Operations Research 16(1), 1968, 122–141. https://doi.org/10.1287/opre.16.1.122
  • S. M. Ross, Stochastic Processes, 2nd ed., Wiley, 1996, Section 2.3 (conditional distribution of arrival times) and Section 2.4 (the M/G/∞ queue).
12 thms3 active usersReviewed
🏆Completed
CombinatoricsOperations ResearchOptimization·Captain: mikedeng1

Numerical Techniques for Stochastic Optimization V: Asymptotic Optimality of List Scheduling for the Machine Investment ProblemTextbook

Motivation

Two-stage stochastic integer programs combine the two hardest features of mathematical programming: uncertainty in the data and integrality of the decisions. Even evaluating the objective of such a program at a single first-stage decision requires the expected optimal value of an NP-hard combinatorial problem. Chapter 8 of Ermoliev and Wets (eds.), Numerical Techniques for Stochastic Optimization (Springer 1988), by A. H. G. Rinnooy Kan and L. Stougie, argues that for many such problems the way forward is probabilistic analysis: the random optimal value of the second-stage problem often converges, after normalization, to a simple function of the problem parameters, and that function can replace the intractable expectation.

The chapter illustrates this on the machine investment problem: first buy mmm identical machines at cost ccc each, knowing only the distribution of the processing times of nnn jobs, then schedule the jobs once their processing times are revealed so as to minimize the makespan. This mission formalizes the chapter's analysis of that example: the almost sure asymptotics of the optimal makespan (8.13), its expectation version, and the asymptotic clairvoyance of the resulting two-stage heuristic.

Setting

Let p1,p2,…p_1, p_2, \dotsp1​,p2​,… be processing times: independent, identically distributed, nonnegative random variables on a probability space (Ω,F,P)(\Omega, \mathcal F, P)(Ω,F,P) with mean μ=Ep1>0\mu = \mathbb E p_1 > 0μ=Ep1​>0 and finite second moment Ep12<∞\mathbb E p_1^2 < \inftyEp12​<∞. The instance with nnn jobs uses the first nnn of them.

An assignment of the nnn jobs to m≥1m \ge 1m≥1 identical machines is a map σ:{1,…,n}→{1,…,m}\sigma : \{1, \dots, n\} \to \{1, \dots, m\}σ:{1,…,n}→{1,…,m}. The load of machine iii is ∑j:σ(j)=ipj\sum_{j : \sigma(j) = i} p_j∑j:σ(j)=i​pj​ and the makespan of σ\sigmaσ is its largest load. The minimum makespan is

Cn∗(m)=min⁡σmax⁡i=1,…,m∑j: σ(j)=ipj,C^*_n(m) = \min_{\sigma} \max_{i=1,\dots,m} \sum_{j:\ \sigma(j) = i} p_j ,Cn∗​(m)=σmin​i=1,…,mmax​j: σ(j)=i∑​pj​,

and the machine investment problem is to minimize Zn(m)=cm+E Cn∗(m)Z_n(m) = cm + \mathbb E\, C^*_n(m)Zn​(m)=cm+ECn∗​(m) over integers mmm (8.9).

List scheduling takes the jobs in the order 1,…,n1, \dots, n1,…,n and assigns each to the first available machine, a machine of least current load (lowest index on ties). Its makespan is CnH(m)C^H_n(m)CnH​(m). Write Sn=∑j=1npjS_n = \sum_{j=1}^n p_jSn​=∑j=1n​pj​ and pmax⁡=max⁡j≤npjp_{\max} = \max_{j \le n} p_jpmax​=maxj≤n​pj​.

For §8.3, the estimate Zn′(m)=cm+nμ/mZ'_n(m) = cm + n\mu/mZn′​(m)=cm+nμ/m is minimized over integers by the heuristic first-stage decision mnH1m^{H1}_nmnH1​, the better of ⌊nμ/c⌋\lfloor\sqrt{n\mu/c}\rfloor⌊nμ/c​⌋ and ⌈nμ/c⌉\lceil\sqrt{n\mu/c}\rceil⌈nμ/c​⌉. A clairvoyant decision maker who sees the processing times first chooses mn∘(ω)≥1m^\circ_n(\omega) \ge 1mn∘​(ω)≥1 minimizing cm+Cn∗(m)cm + C^*_n(m)cm+Cn∗​(m).

Formalization targets

Goal: Eq. (8.13)

For machine counts m=m(n)≥1m = m(n) \ge 1m=m(n)≥1 with m(n)=O(n)m(n) = O(\sqrt n)m(n)=O(n​),

P{lim⁡n→∞Cn∗(m)nμ/m=1}=1.P\Bigl\{ \lim_{n\to\infty} \frac{C^*_n(m)}{n\mu/m} = 1 \Bigr\} = 1 .P{n→∞lim​nμ/mCn∗​(m)​=1}=1.

The machine count is allowed to grow with nnn; this is the regime the first-stage heuristic lives in, since mnH1m^{H1}_nmnH1​ is of exact order n\sqrt nn​.

Milestones

  1. Eq. (8.10): the deterministic sandwich Sn/m≤Cn∗(m)≤CnH(m)≤Sn/m+pmax⁡S_n/m \le C^*_n(m) \le C^H_n(m) \le S_n/m + p_{\max}Sn​/m≤Cn∗​(m)≤CnH​(m)≤Sn​/m+pmax​, divided by nμ/mn\mu/mnμ/m.
  2. Eq. (8.11): the strong law of large numbers, (Sn−nμ)/(nμ)→0(S_n - n\mu)/(n\mu) \to 0(Sn​−nμ)/(nμ)→0 almost surely (a published platform theorem).
  3. Lemma 8.1 (i): pmax⁡/n→0p_{\max}/\sqrt n \to 0pmax​/n​→0 almost surely.
  4. Eq. (8.12): m pmax⁡/(nμ)→0m\, p_{\max}/(n\mu) \to 0mpmax​/(nμ)→0 almost surely when m=O(n)m = O(\sqrt n)m=O(n​).
  5. Lemma 8.1 (ii): E pmax⁡/n→0\mathbb E\, p_{\max}/\sqrt n \to 0Epmax​/n​→0.
  6. p. 207: E Cn∗(m)/(nμ/m)→1\mathbb E\, C^*_n(m)/(n\mu/m) \to 1ECn∗​(m)/(nμ/m)→1 when m=O(n)m = O(\sqrt n)m=O(n​).
  7. p. 211, asymptotic clairvoyance: almost surely
lim⁡n→∞c mnH1+CnH2(mnH1)c mn∘+Cn∗(mn∘)=1,\lim_{n\to\infty} \frac{c\, m^{H1}_n + C^{H2}_n(m^{H1}_n)}{c\, m^\circ_n + C^*_n(m^\circ_n)} = 1 ,n→∞lim​cmn∘​+Cn∗​(mn∘​)cmnH1​+CnH2​(mnH1​)​=1,

where CnH2C^{H2}_nCnH2​ is the list-scheduling makespan.

Significance

Result (8.13) says that the optimal value of an NP-hard problem, rescaled, is almost surely asymptotic to the elementary function nμ/mn\mu/mnμ/m of the data and the first-stage decision. Its expectation version replaces the intractable term E Cn∗(m)\mathbb E\,C^*_n(m)ECn∗​(m) in (8.9) by nμ/mn\mu/mnμ/m, and the clairvoyance statement shows that the heuristic built on that replacement loses asymptotically nothing, not even against a decision maker with full information. The chapter presents the example as the template for vehicle routing and location problems preceded by an investment decision.

All results here are classical and proved in the literature cited by the chapter (Lemma 8.1 is quoted from Feller without proof; the chapter refers to Dempster et al. for the asymptotic optimality of the two-stage heuristic and to Lenstra et al. for the notion of asymptotic clairvoyance). None of them has, to our knowledge, a machine-checked proof. The mission produces a formal model of identical-machine makespan scheduling and of list scheduling, the extreme-value estimates of Lemma 8.1 for square-integrable i.i.d. sequences, and the full chain from the strong law to (8.13).

Difficulty

The deterministic part is elementary on paper, but list scheduling is a recursively defined procedure, and its makespan bound has to be established for that recursion rather than for a picture like the chapter's Figure 8.3. The probabilistic core is Lemma 8.1: the strong law controls Sn/nS_n/nSn​/n, but the error term m pmax⁡/(nμ)m\, p_{\max}/(n\mu)mpmax​/(nμ) is of order pmax⁡/np_{\max}/\sqrt npmax​/n​ once mmm grows like n\sqrt nn​, and the strong law says nothing about maxima. With a fixed number of machines the whole statement would reduce to the strong law; the growth m(n)=O(n)m(n) = O(\sqrt n)m(n)=O(n​) is exactly where the second moment is needed. For the clairvoyance statement, the clairvoyant choice mn∘m^\circ_nmn∘​ is a random, unstructured minimizer, so its value must be bounded below without knowing where the minimum is attained.

Formalization scope

Processing times are one sequence p : ℕ → Ω → ℝ, 0-based (the book's pjp_jpj​ is p (j-1)), with each p j measurable, the family mutually independent (iIndepFun), identically distributed with p 0, pointwise nonnegative, p 0 ^ 2 integrable and ∫ p 0 = μ with μ > 0. Nonnegativity and μ>0\mu > 0μ>0 are not printed in the book; they are implicit in "processing times" and in the division by nμn\munμ. Machines are Fin m; a schedule is an assignment Fin n → Fin m, which is faithful because jobs are non-preemptive, machines identical and there are no precedence constraints.

The book writes "m=0(n)m = 0(\sqrt n)m=0(n​)"; this is read as mmm a function of nnn with m(n)≥1m(n) \ge 1m(n)≥1 and (fun n => (m n : ℝ)) =O[atTop] (fun n => √n). Stating (8.13) for a fixed mmm would trivialize it into the strong law and is ruled out. "Pr⁡{lim⁡⋯=1}=1\Pr\{\lim \dots = 1\} = 1Pr{lim⋯=1}=1" means that almost surely the limit exists and equals 111. Expectations are Bochner integrals of functions that are measurable and bounded by SnS_nSn​, hence integrable. List scheduling uses the index order and breaks ties towards the lowest machine index; both are admissible instances of the book's "arbitrary fixed order" and "first available machine". In the clairvoyance statement the minimum is over m≥1m \ge 1m≥1 (the book writes m∈Nm \in \mathbb Nm∈N; no machine cannot process any job, and the Lean value Cn∗(0)C^*_n(0)Cn∗​(0) is an empty-infimum convention). No explicit constants replace an O(·): the statements are limits and the O-hypothesis is carried as stated.

Out of scope: (8.14) and the p. 210 expectation statement, which need a positive density at 000 and whose proof the book calls "far from easy", and the dynamic programming recursion of §8.3.

Needed infrastructure: finite maxima and minima of measurable functions, extreme-value estimates for square-integrable i.i.d. sequences (Lemma 8.1), and Mathlib's strong law. The makespan and list-scheduling definitions are reusable for other identical-machine scheduling results; alternative proofs of Lemma 8.1 and sharper forms of the clairvoyance statement are welcome.

Selected references

  • A. H. G. Rinnooy Kan, L. Stougie, "Stochastic Integer Programming", in Yu. Ermoliev, R. J-B Wets (eds.), Numerical Techniques for Stochastic Optimization, Springer Series in Computational Mathematics 10, Springer 1988, Ch. 8, pp. 201–213. https://doi.org/10.1007/978-3-642-61370-8
  • W. Feller, An Introduction to Probability Theory and Its Applications, Vol. 1, 3rd edition, Wiley, 1968 (cited by the chapter for Lemma 8.1).
  • M. A. H. Dempster, M. L. Fisher, L. Jansen, B. J. Lageweg, J. K. Lenstra, A. H. G. Rinnooy Kan, "Analysis of heuristics for stochastic programming: results for hierarchical scheduling problems", Mathematics of Operations Research 8 (1983) 525–537. https://doi.org/10.1287/moor.8.4.525
  • J. K. Lenstra, A. H. G. Rinnooy Kan, L. Stougie, "A framework for the design and analysis of hierarchical planning systems", Annals of Operations Research 1 (1984) 23–42. https://doi.org/10.1007/BF01874451
  • R. L. Graham, "Bounds on multiprocessing timing anomalies", SIAM Journal on Applied Mathematics 17 (1969) 416–429. https://doi.org/10.1137/0117039
11 thms3 active usersReviewed
🏆Completed
Convex OptimizationOperations ResearchOptimization·Captain: mikedeng1

Numerical Techniques for Stochastic Optimization III: Stochastic Quasi-Féjer Sequences and the Stochastic Quasigradient Projection MethodTextbook

Motivation

Many optimization problems in operations research have an objective that is an expectation, F0(x)=Ef0(x,ω)F^0(x)=E f^0(x,\omega)F0(x)=Ef0(x,ω), over a random parameter ω\omegaω whose distribution is known only through samples or is too complex to integrate. Two-stage stochastic programs, inventory and reliability models, and simulation-based design all have this form. Neither F0F^0F0 nor its subgradients can be evaluated exactly, but a random vector whose conditional mean is close to a subgradient is often cheap to compute: a sample subgradient of f0(⋅,ω)f^0(\cdot,\omega)f0(⋅,ω), or a finite-difference quotient of two sampled values.

Stochastic quasigradient (SQG) methods, developed by Ermoliev and co-workers in Kiev from the late 1960s, use such vectors in place of subgradients. They extend the stochastic approximation procedures of Robbins–Monro (1951) and Kiefer–Wolfowitz (1952) to nonsmooth convex objectives, general convex constraints, and directions whose conditional mean is biased by a vanishing amount. This mission formalizes the basic convergence theory of the simplest SQG method, the projection method, as presented by Yu. Ermoliev in Chapter 6 of the IIASA volume Numerical Techniques for Stochastic Optimization (Springer 1988).

Timeline (as cited in the chapter's bibliography).

  • 1951–1954: Robbins and Monro, Kiefer and Wolfowitz, Dvoretzky and Blum prove convergence of stochastic approximation for unconstrained smooth problems.
  • 1962–1967: Shor introduces the generalized gradient (subgradient) method; Ermoliev (Kibernetika 4, 1966) and Polyak (Soviet Math. Doklady 8, 1967) prove its convergence.
  • 1967–1969: Ermoliev and Nekrylova introduce stochastic subgradients; Ermoliev ("On the stochastic quasi-gradient method and stochastic quasi-Feyer sequences", Kibernetika 2, 1969) introduces stochastic quasi-Féjer sequences.
  • 1976: Ermoliev's monograph Stochastic Programming Methods (Nauka) contains the proof of Theorem 6.1 (p. 98).
  • 1988: the survey chapter formalized here presents the projection method, Theorems 6.1 and 6.2, and an efficiency estimate for the averaged iterate.

Setting

Let X⊆RnX\subseteq\mathbb R^nX⊆Rn be a nonempty convex compact set and F0:Rn→RF^0:\mathbb R^n\to\mathbb RF0:Rn→R convex and continuous on XXX. The optimal set is X∗={x∈X:F0(x)≤F0(y) ∀y∈X}X^*=\{x\in X: F^0(x)\le F^0(y)\ \forall y\in X\}X∗={x∈X:F0(x)≤F0(y) ∀y∈X}. The projection onto XXX is πX(y)=argmin⁡{∥y−x∥2:x∈X}\pi_X(y)=\operatorname{argmin}\{\|y-x\|^2:x\in X\}πX​(y)=argmin{∥y−x∥2:x∈X}.

On a probability space, the stochastic quasigradient projection method produces random vectors x0,x1,…x^0,x^1,\dotsx0,x1,… by

xs+1=πX[xs−ρs ξ0(s)],s=0,1,…(6.11)x^{s+1}=\pi_X\big[x^s-\rho_s\,\xi^0(s)\big],\qquad s=0,1,\dots \tag{6.11}xs+1=πX​[xs−ρs​ξ0(s)],s=0,1,…(6.11)

where ρs≥0\rho_s\ge0ρs​≥0 is a step size and ξ0(s)\xi^0(s)ξ0(s) a random direction. Write E{⋅∣x0,…,xs}E\{\cdot\mid x^0,\dots,x^s\}E{⋅∣x0,…,xs} for conditional expectation given the history σ(x0,…,xs)\sigma(x^0,\dots,x^s)σ(x0,…,xs). The direction is a stochastic quasigradient if, for every x∗∈X∗x^*\in X^*x∗∈X∗,

F0(x∗)−F0(xs)≥⟨E{ξ0(s)∣x0,…,xs}, x∗−xs⟩+γ0(s)a.s.,(6.12)F^0(x^*)-F^0(x^s)\ge\big\langle E\{\xi^0(s)\mid x^0,\dots,x^s\},\,x^*-x^s\big\rangle+\gamma_0(s)\quad\text{a.s.}, \tag{6.12}F0(x∗)−F0(xs)≥⟨E{ξ0(s)∣x0,…,xs},x∗−xs⟩+γ0​(s)a.s.,(6.12)

where the error γ0(s)\gamma_0(s)γ0​(s) is a function of the history. If the conditional mean of ξ0(s)\xi^0(s)ξ0(s) is a subgradient plus a bias b0(s)b^0(s)b0(s), then (6.12) holds with γ0(s)=−⟨b0(s),x∗−xs⟩\gamma^0(s)=-\langle b^0(s),x^*-x^s\rangleγ0(s)=−⟨b0(s),x∗−xs⟩ (6.13).

A sequence of random vectors z0,z1,…z^0,z^1,\dotsz0,z1,… is a stochastic quasi-Féjer sequence for Z⊆RnZ\subseteq\mathbb R^nZ⊆Rn if E∥z0∥2<∞E\|z^0\|^2<\inftyE∥z0∥2<∞ and there are random rs≥0r_s\ge0rs​≥0 with ∑sErs<∞\sum_s E r_s<\infty∑s​Ers​<∞ such that for all z∈Zz\in Zz∈Z

E{∥z−zs+1∥2∣z0,…,zs}≤∥z−zs∥2+rs.(6.14)E\{\|z-z^{s+1}\|^2\mid z^0,\dots,z^s\}\le\|z-z^s\|^2+r_s. \tag{6.14}E{∥z−zs+1∥2∣z0,…,zs}≤∥z−zs∥2+rs​.(6.14)

Formalization targets

Goal: Theorem 6.2

If, with probability 1, ρs≥0\rho_s\ge0ρs​≥0 and ∑sρs=∞\sum_s\rho_s=\infty∑s​ρs​=∞, and

∑s=0∞E{ρs∣γ0(s)∣+ρs2∥ξ0(s)∥2}<∞,(6.15)\sum_{s=0}^\infty E\{\rho_s|\gamma_0(s)|+\rho_s^2\|\xi^0(s)\|^2\}<\infty, \tag{6.15}s=0∑∞​E{ρs​∣γ0​(s)∣+ρs2​∥ξ0(s)∥2}<∞,(6.15)

then with probability 1 the iterates converge and lim⁡sxs∈X∗\lim_s x^s\in X^*lims​xs∈X∗.

Milestones

  1. Theorem 6.1 (a)–(c). For a stochastic quasi-Féjer sequence for ZZZ: ∥z−zs+1∥2\|z-z^{s+1}\|^2∥z−zs+1∥2 converges a.s. and E∥z−zs∥2E\|z-z^s\|^2E∥z−zs∥2 is bounded, for each z∈Zz\in Zz∈Z; accumulation points exist a.s. (for Z≠∅Z\ne\emptysetZ=∅); and a.s. ZZZ lies in the hyperplane equidistant from any two distinct accumulation points outside ZZZ.
  2. Eq. (6.13). Biased stochastic subgradients satisfy (6.12).
  3. One-step inequality (p. 145): E{∥x∗−xs+1∥2∣⋅}≤∥x∗−xs∥2+2ρs⟨E{ξ0(s)∣⋅},x∗−xs⟩+E{ρs2∥ξ0(s)∥2∣⋅}E\{\|x^*-x^{s+1}\|^2\mid\cdot\}\le\|x^*-x^s\|^2+2\rho_s\langle E\{\xi^0(s)\mid\cdot\},x^*-x^s\rangle+E\{\rho_s^2\|\xi^0(s)\|^2\mid\cdot\}E{∥x∗−xs+1∥2∣⋅}≤∥x∗−xs∥2+2ρs​⟨E{ξ0(s)∣⋅},x∗−xs⟩+E{ρs2​∥ξ0(s)∥2∣⋅} for x∗∈Xx^*\in Xx∗∈X.
  4. Quasi-Féjer property (p. 145): the iterates of (6.11) form a stochastic quasi-Féjer sequence for X∗X^*X∗.
  5. Efficiency estimate (p. 147), for deterministic ρk\rho_kρk​ and xˉs=∑k≤sρkxk/∑k≤sρk\bar x^s=\sum_{k\le s}\rho_kx^k/\sum_{k\le s}\rho_kxˉs=∑k≤s​ρk​xk/∑k≤s​ρk​:
EF0(xˉs)−F0(x∗)≤(2∑k=0sρk)−1[E∥x∗−x0∥2+∑k=0sE(2ρk∣γ0(k)∣+ρk2∥ξ0(k)∥2)].E F^0(\bar x^s)-F^0(x^*)\le\Big(2\sum_{k=0}^s\rho_k\Big)^{-1}\Big[E\|x^*-x^0\|^2+\sum_{k=0}^s E\big(2\rho_k|\gamma_0(k)|+\rho_k^2\|\xi^0(k)\|^2\big)\Big].EF0(xˉs)−F0(x∗)≤(2k=0∑s​ρk​)−1[E∥x∗−x0∥2+k=0∑s​E(2ρk​∣γ0​(k)∣+ρk2​∥ξ0(k)∥2)].

Significance

Theorem 6.2 is the prototype convergence theorem for SQG methods. Its hypotheses allow random step sizes chosen from the history, nonsmooth objectives, and directions with a bias that vanishes fast enough; its conclusion is convergence of the iterates themselves to a single optimal point, not only convergence of function values or of dist⁡(xs,X∗)\operatorname{dist}(x^s,X^*)dist(xs,X∗). The later chapters of the same volume (adaptive step sizes, Chapters 17–18; nonstationary problems, §6.4) reuse the same framework. Theorem 6.1 isolates the probabilistic content in a form that applies to any algorithm with a quasi-Féjer inequality. The efficiency estimate gives a non-asymptotic accuracy bound for the averaged iterate.

The results are classical and proved in the literature: Theorem 6.1 in Ermoliev (1976, p. 98), Theorem 6.2 in this chapter (pp. 145–146). To our knowledge none of them has a machine-checked proof. Mathlib has conditional expectations and the a.s. martingale convergence theorem, but no Robbins–Siegmund-type almost-supermartingale lemma and no stochastic subgradient method. A formal proof of this mission would supply both.

Difficulty

The deterministic argument for projected subgradient methods compares ∥x∗−xs+1∥\|x^*-x^{s+1}\|∥x∗−xs+1∥ with ∥x∗−xs∥\|x^*-x^s\|∥x∗−xs∥ for a fixed x∗x^*x∗. In the stochastic setting this comparison holds only in conditional mean, with a perturbation rsr_srs​ that is random, and the distances converge only almost surely, with an exceptional null set that depends on x∗x^*x∗. Since X∗X^*X∗ is typically uncountable, "for every x∗x^*x∗, almost surely" does not immediately give "almost surely, for every x∗x^*x∗", and it is the second form that identifies a single limit. A second difficulty is that ∑ρs(F0(xs)−F0(x∗))<∞\sum\rho_s(F^0(x^s)-F^0(x^*))<\infty∑ρs​(F0(xs)−F0(x∗))<∞ only yields a subsequence along which F0F^0F0 approaches its minimum; passing from there to convergence of the whole sequence is exactly what part (c) of Theorem 6.1 is for.

Formalization scope

  • Rn\mathbb R^nRn is EuclideanSpace ℝ (Fin n). The probability space is an arbitrary measurable space with a probability measure. πX\pi_XπX​ is a chosen minimizer of ∥y−x∥2\|y-x\|^2∥y−x∥2 over XXX (unique for nonempty closed convex XXX). The history is the σ\sigmaσ-algebra generated by x0,…,xsx^0,\dots,x^sx0,…,xs; ρs\rho_sρs​ and γ0(s)\gamma_0(s)γ0​(s) are measurable with respect to it.
  • Directions ξ0(s)\xi^0(s)ξ0(s) are integrable and random vectors are measurable; conditional expectations are Mathlib's condExp. The quasi-Féjer definition requires square integrability of every zsz^szs (implied by the book's definition when Z≠∅Z\ne\emptysetZ=∅), so no conditional expectation is taken of a non-integrable function.
  • X≠∅X\ne\emptysetX=∅ and x0∈Xx^0\in Xx0∈X are stated; Z≠∅Z\ne\emptysetZ=∅ is added in Theorem 6.1 (b), which is false without it.
  • γ0(s)\gamma_0(s)γ0​(s) does not depend on x∗x^*x∗; the x∗x^*x∗-dependent error of (6.13) is dominated on a bounded XXX by ∥b0(s)∥diam⁡X\|b^0(s)\|\operatorname{diam}X∥b0(s)∥diamX.
  • (6.15) keeps its mixed form: ρs≥0\rho_s\ge0ρs​≥0 and ∑ρs=∞\sum\rho_s=\infty∑ρs​=∞ almost surely, and a deterministic sum of expectations (lower Lebesgue integrals) finite.
  • Explicit constants. The book's "CCC" in the efficiency estimate is instantiated from its proof: 222 on ρk∣γ0(k)∣\rho_k|\gamma_0(k)|ρk​∣γ0​(k)∣ and 111 on ρk2∥ξ0(k)∥2\rho_k^2\|\xi^0(k)\|^2ρk2​∥ξ0(k)∥2. The unspecified CCC before the quasi-Féjer sentence is replaced by the existence of summable rsr_srs​.
  • Typo corrections. The one-step inequality on p. 145 prints ρsE{∥ξ0(s)∥2∣⋅}\rho_sE\{\|\xi^0(s)\|^2\mid\cdot\}ρs​E{∥ξ0(s)∥2∣⋅}; it is ρs2\rho_s^2ρs2​. The efficiency estimate on p. 147 omits EEE before the last sum; it is restored. "ρk\rho_kρk​ independent of (x0,…,xk)(x^0,\dots,x^k)(x0,…,xk)" is read as deterministic step sizes.
  • A trivializing formalization is excluded: the goal does not replace ξ0(s)\xi^0(s)ξ0(s) by an exact subgradient, does not set γ0≡0\gamma_0\equiv0γ0​≡0, and concludes convergence of xsx^sxs to a point of X∗X^*X∗ rather than dist⁡(xs,X∗)→0\operatorname{dist}(x^s,X^*)\to0dist(xs,X∗)→0.
  • Reusable infrastructure: a Robbins–Siegmund lemma for nonnegative almost-supermartingales, the nonexpansiveness of πX\pi_XπX​, and Theorem 6.1 itself, which applies to any quasi-Féjer algorithm (Chapter 6 §6.4 and Chapters 17–18 of the same book). Contributions of these general lemmas are welcome.

Selected references

  • Yu. Ermoliev, "Stochastic Quasigradient Methods", in Yu. Ermoliev and R. J-B Wets (eds.), Numerical Techniques for Stochastic Optimization, Springer Series in Computational Mathematics 10, Springer 1988, Ch. 6, §6.1–6.2 (pp. 141–147). https://doi.org/10.1007/978-3-642-61370-8
  • Yu. Ermoliev, "On the stochastic quasi-gradient method and stochastic quasi-Feyer sequences", Kibernetika 2 (1969) (in Russian; English translation in Cybernetics). Reference [3] of the chapter.
  • Yu. Ermoliev, Stochastic Programming Methods, Nauka, Moscow, 1976 (in Russian); Theorem 6.1 is on p. 98. Reference [5] of the chapter.
  • H. Robbins and D. Siegmund, "A convergence theorem for non negative almost supermartingales and some applications", in J. S. Rustagi (ed.), Optimizing Methods in Statistics, Academic Press, 1971, 233–257. https://doi.org/10.1016/B978-0-12-604550-5.50015-8
  • H. Robbins and S. Monro, "A stochastic approximation method", Annals of Mathematical Statistics 22 (1951) 400–407. https://doi.org/10.1214/aoms/1177729586
10 thms3 active usersReviewed
🏆Completed
Operations ResearchOptimization·Captain: mikedeng1

Introduction to the Scenario Approach V: Support Sets Certify the Violation of Nonconvex Scenario SolutionsTextbook

Why certify nonconvex scenario solutions

The scenario approach replaces an optimization problem with uncertain constraints θ∈Θδ\theta\in\Theta_\deltaθ∈Θδ​, δ∈Δ\delta\in\Deltaδ∈Δ, by the program that enforces only NNN constraints Θδ1,…,ΘδN\Theta_{\delta_1},\dots,\Theta_{\delta_N}Θδ1​​,…,ΘδN​​ drawn at random from the distribution P\mathbb PP of the uncertainty. Its solution θ∗\theta^*θ∗ is then judged by its violation, the probability that a new instance δ\deltaδ is not satisfied. For convex programs in Rd\mathbb R^dRd the violation is controlled by the dimension ddd alone (Calafiore and Campi 2006; Campi and Garatti 2008), because a convex program has at most ddd support constraints. Many problems where the scenario approach is used in practice are not convex: control with quantized inputs, mixed-integer design, classification with nonconvex losses, and decisions over infinite-dimensional or unstructured sets. For those programs no a priori bound on the number of constraints that determine the solution exists.

Timeline, as recorded in Chapter 8 of Campi and Garatti's textbook:

  • 2006–2008: the violation of convex scenario solutions is bounded, and then characterized exactly, in terms of ddd.
  • 2018: the wait-and-judge theory (Campi and Garatti, Math. Programming 2018) evaluates the violation from the number of support constraints counted after solving the program; it extends to nonconvex programs but requires a nondegeneracy assumption.
  • 2018: Campi, Garatti and Ramponi (IEEE TAC 2018) prove a bound in terms of the size of any support set, with no convexity and no nondegeneracy assumption. This is the result formalized here, stated in the book as Eq. (8.15).

Setting

Let Θ\ThetaΘ be a generic set; it may be an infinite-dimensional space or a set with no algebraic structure. Let f:Θ→Rf:\Theta\to\mathbb Rf:Θ→R be a cost and Θδ⊆Θ\Theta_\delta\subseteq\ThetaΘδ​⊆Θ constraint sets indexed by δ∈Δ\delta\in\Deltaδ∈Δ, where Δ\DeltaΔ carries a probability P\mathbb PP. Neither fff nor the Θδ\Theta_\deltaΘδ​ is required to be convex. With a sample δ1,…,δN\delta_1,\dots,\delta_Nδ1​,…,δN​ drawn independently from P\mathbb PP, the scenario program is

min⁡θ∈Θf(θ)subject toθ∈⋂i=1,…,NΘδi,(8.12)\min_{\theta\in\Theta} f(\theta)\quad\text{subject to}\quad \theta\in\bigcap_{i=1,\dots,N}\Theta_{\delta_i},\tag{8.12}θ∈Θmin​f(θ)subject toθ∈i=1,…,N⋂​Θδi​​,(8.12)

and θ∗\theta^*θ∗ denotes its solution, assumed to exist and be unique for every sample.

The violation of a decision is V(θ)=P{δ∈Δ:θ∉Θδ}V(\theta)=\mathbb P\{\delta\in\Delta:\theta\notin\Theta_\delta\}V(θ)=P{δ∈Δ:θ∈/Θδ​} (Definition 3.1).

A support set (Definition 8.8) is a subset {Θδi1,…,Θδik}\{\Theta_{\delta_{i_1}},\dots,\Theta_{\delta_{i_k}}\}{Θδi1​​​,…,Θδik​​​} of the constraints such that the program with only these constraints in place has the same solution θ∗\theta^*θ∗ as the program with all constraints. The full set of constraints is always a support set; a support set need not be minimal (of smallest cardinality) or irreducible (with no removable element). Let σ∗\sigma^*σ∗ be the cardinality of the support set returned, for every sample, by some fixed algorithm.

Formalization targets

Goal: the support-set bound, Eq. (8.15)

For every function ϵ:{0,1,…,N}→[0,1]\epsilon:\{0,1,\dots,N\}\to[0,1]ϵ:{0,1,…,N}→[0,1] with ϵ(N)=1\epsilon(N)=1ϵ(N)=1,

PN{V(θ∗)>ϵ(σ∗)}≤∑k=0N−1(Nk) (1−ϵ(k))N−k.\mathbb P^N\{V(\theta^*)>\epsilon(\sigma^*)\}\le\sum_{k=0}^{N-1}\binom Nk\,(1-\epsilon(k))^{N-k}.PN{V(θ∗)>ϵ(σ∗)}≤k=0∑N−1​(kN​)(1−ϵ(k))N−k.

The level function ϵ\epsilonϵ is free: the statement is one inequality per admissible ϵ\epsilonϵ, and it holds for any algorithm producing support sets. This is the weakest form that carries the whole result.

Milestone: the level function for a confidence β\betaβ, Eq. (8.16)

For β∈[0,1]\beta\in[0,1]β∈[0,1] let

ϵ(k)={1k=N,1−βN(Nk)N−kotherwise.\epsilon(k)=\begin{cases}1 & k=N,\\ 1-\sqrt[N-k]{\dfrac{\beta}{N\binom Nk}} & \text{otherwise.}\end{cases}ϵ(k)=⎩⎨⎧​11−N−kN(kN​)β​​​k=N,otherwise.​

The arithmetic half states that ϵ\epsilonϵ maps {0,…,N}\{0,\dots,N\}{0,…,N} into [0,1][0,1][0,1], ϵ(N)=1\epsilon(N)=1ϵ(N)=1, and the right-hand side of (8.15) equals β\betaβ (for N≥1N\ge1N≥1). The probabilistic half states PN{V(θ∗)>ϵ(σ∗)}≤β\mathbb P^N\{V(\theta^*)>\epsilon(\sigma^*)\}\le\betaPN{V(θ∗)>ϵ(σ∗)}≤β.

Significance

The result turns the size of a support set, a quantity observed after the program is solved, into a certificate on the violation of the solution, for any optimization or decision problem whose solution is determined by a subset of the data. With the choice (8.16), a user who finds a support set of size σ∗\sigma^*σ∗ can assert V(θ∗)≤ϵ(σ∗)V(\theta^*)\le\epsilon(\sigma^*)V(θ∗)≤ϵ(σ∗) with confidence 1−β1-\beta1−β. The book's Figure 8.12 shows that ϵ(k)\epsilon(k)ϵ(k) for β=10−6\beta=10^{-6}β=10−6 remains well below 111 for kkk up to a sizeable fraction of NNN. Because the algorithm that finds the support set is arbitrary, cheap heuristics that return non-minimal support sets still give valid, if weaker, guarantees. The result does not recover the tight convex bound (3.4); for convex programs the Chapter 3 theory remains sharper.

The statement is proved in [31] and is the probabilistic core of the sample-compression arguments of learning theory (Floyd and Warmuth 1995) in the form used by the scenario approach. To the best of available knowledge it has no machine-checked proof. The platform's UnderstandingML.compression_bound proves a sample-compression bound for a fixed compression size with a different constant; it does not cover a data-dependent size σ∗\sigma^*σ∗ or an arbitrary level function. A formal proof here provides a reusable bound for data-dependent support sets over arbitrary decision sets.

Difficulty

The natural first step is: condition on the support set being a particular index set III with ∣I∣=k|I|=k∣I∣=k, and argue that the solution is then a function of the kkk sampled constraints in III alone, while the other N−kN-kN−k samples are independent of it and must all be satisfied. The difficulty is that the event "the algorithm returns III" depends on all NNN samples, and the solution of the reduced program on III is defined only where that program has a unique solution; the decomposition of the probability therefore has to be carried out on sections of the product space, with a measurability argument for each piece. A second point is that σ∗\sigma^*σ∗ is random and data-dependent: a bound for each fixed kkk does not directly give a bound at the random level ϵ(σ∗)\epsilon(\sigma^*)ϵ(σ∗), and the role of the condition ϵ(N)=1\epsilon(N)=1ϵ(N)=1 must be accounted for at k=Nk=Nk=N.

Formalization scope

Lean representation and committed conventions:

  • Θ\ThetaΘ and Δ\DeltaΔ are arbitrary types with measurable structures; P\mathbb PP is a probability measure on Δ\DeltaΔ; a sample is ω : Fin N → Δ with law Measure.pi (fun _ : Fin N => P); indices run over 0,…,N−10,\dots,N-10,…,N−1.
  • A subset of constraints is a Finset (Fin N); the reduced program with index set III has feasible set ⋂i∈IΘδi\bigcap_{i\in I}\Theta_{\delta_i}⋂i∈I​Θδi​​ (all of Θ\ThetaΘ for I=∅I=\emptysetI=∅).
  • The solution map θ∗\theta^*θ∗ is a parameter with the hypothesis that θ∗(ω)\theta^*(\omega)θ∗(ω) is the unique solution of the full program for every sample.
  • "Has the same solution" in Definition 8.8 means: the reduced program has a unique solution and it equals the unique solution of the full program. Existence of solutions is not assumed for reduced programs in general, only for those that are support sets.
  • The algorithm is an arbitrary map alg : (Fin N → Δ) → Finset (Fin N) returning a support set for every sample; σ∗\sigma^*σ∗ is the cardinality of its output. The goal is universal over such maps.
  • ϵ\epsilonϵ is a real function on N\mathbb NN with ϵ(k)∈[0,1]\epsilon(k)\in[0,1]ϵ(k)∈[0,1] for k≤Nk\le Nk≤N and ϵ(N)=1\epsilon(N)=1ϵ(N)=1.
  • The violation is real-valued in [0,1][0,1][0,1]; the probability of the event is compared in [0,∞][0,\infty][0,∞] with ENNReal.ofReal of the real right-hand side.

Implicit hypotheses of the page, made explicit (the book states on p. 33 that measurability issues are glossed over): the constraint relation {(θ,δ):θ∈Θδ}\{(\theta,\delta):\theta\in\Theta_\delta\}{(θ,δ):θ∈Θδ​} is measurable in Θ×Δ\Theta\times\DeltaΘ×Δ; the solution map is measurable; each event {ω:the algorithm returns J}\{\omega:\text{the algorithm returns }J\}{ω:the algorithm returns J} is measurable; θ∗\theta^*θ∗ exists and is unique for every sample; N≥1N\ge1N≥1 in the arithmetic half of (8.16).

A trivializing formalization is ruled out: the algorithm is not existentially quantified and is not the minimal support set, the reduced programs are not all assumed solvable (which would be unsatisfiable when fff has no unconstrained minimizer), and the support-set property requires uniqueness of the reduced solution, without which the bound is false.

Needed infrastructure: product measures on Fin N → Δ, splitting of such products along a subset of coordinates, and Fubini/Tonelli for sections. These pieces are reusable for other compression-type bounds. Contributions welcome: proofs of the two milestones and of the goal, and lemmas on splitting Measure.pi over a Finset of coordinates.

Selected references

  • M. C. Campi, S. Garatti, Introduction to the Scenario Approach, MOS-SIAM Series on Optimization 26, SIAM/MOS, 2018, §8.6, pp. 101–105. https://doi.org/10.1137/1.9781611975444
  • M. C. Campi, S. Garatti, F. A. Ramponi, A general scenario theory for nonconvex optimization and decision making, IEEE Transactions on Automatic Control, 2018. https://doi.org/10.1109/TAC.2018.2808446
  • M. C. Campi, S. Garatti, Wait-and-judge scenario optimization, Mathematical Programming, 2018. https://doi.org/10.1007/s10107-016-1056-9
  • G. C. Calafiore, M. C. Campi, The scenario approach to robust control design, IEEE Transactions on Automatic Control, 2006. https://doi.org/10.1109/TAC.2006.875041
  • M. C. Campi, S. Garatti, The exact feasibility of randomized solutions of uncertain convex programs, SIAM Journal on Optimization, 2008. https://doi.org/10.1137/07069821X
  • S. Floyd, M. Warmuth, Sample compression, learnability, and the Vapnik–Chervonenkis dimension, Machine Learning, 1995. https://doi.org/10.1007/BF00993593
7 thms3 active usersReviewed
🏆Completed
Number TheoryQuantum InformationTheoretical Computer Science·Captain: mikedeng1

Polynomial-Time Algorithms for Prime Factorization and Discrete Logarithms on a Quantum Computer 3: The Success Probability of Quantum Order FindingResearch Paper

Motivation

The security of the RSA cryptosystem rests on the assumed difficulty of factoring large integers, and the best known classical algorithms for factoring run in super-polynomial time. In 1994 Peter Shor showed that a quantum computer can factor an nnn-digit integer in time polynomial in nnn (Shor, SIAM J. Comput. 1997; conference version FOCS 1994). The algorithm has two parts. A classical reduction, due to Miller (1976), turns factoring into order finding: given xxx coprime to nnn, find the least r≥1r \ge 1r≥1 with xr≡1(modn)x^r \equiv 1 \pmod nxr≡1(modn). The quantum part solves order finding.

This mission formalizes the quantum part as Shor analyzes it in §5 of the journal paper: the construction of the quantum state, the probability of each measurement outcome, and the classical post-processing that reads rrr off the measured value. The paper's claim is that one run of this procedure returns rrr with probability at least φ(r)/3r\varphi(r)/3rφ(r)/3r.

Timeline:

  • 1976: Miller reduces factoring to order finding (with randomization).
  • 1985–1994: Deutsch, Bernstein–Vazirani and Simon give the quantum Fourier sampling ideas the algorithm builds on.
  • 1994: Shor's FOCS paper introduces the factoring and discrete logarithm algorithms.
  • 1997: the SIAM J. Comput. version gives the analysis formalized here, with qqq the power of 222 in [n2,2n2)[n^2, 2n^2)[n2,2n2).

Setting

Fix an integer n≥2n \ge 2n≥2 and an integer xxx coprime to nnn. Its order rrr is the least r≥1r \ge 1r≥1 with xr≡1(modn)x^r \equiv 1 \pmod nxr≡1(modn); since xxx is a unit, r≤φ(n)<nr \le \varphi(n) < nr≤φ(n)<n. Let q=2lq = 2^lq=2l be the power of 222 with n2≤q<2n2n^2 \le q < 2n^2n2≤q<2n2.

A quantum state on two registers, the first holding 0≤a<q0 \le a < q0≤a<q and the second a residue y∈Z/ny \in \mathbb{Z}/ny∈Z/n, is a complex vector ψ(a,y)\psi(a, y)ψ(a,y) indexed by the basis states ∣a,y⟩|a, y\rangle∣a,y⟩. Measuring it returns ∣a,y⟩|a, y\rangle∣a,y⟩ with probability ∣ψ(a,y)∣2|\psi(a, y)|^2∣ψ(a,y)∣2.

The Fourier matrix AqA_qAq​ is the q×qq \times qq×q matrix with entries (Aq)a,c=q−1/2exp⁡(2πiac/q)(A_q)_{a,c} = q^{-1/2}\exp(2\pi i a c/q)(Aq​)a,c​=q−1/2exp(2πiac/q), with rows indexing inputs and columns outputs. The algorithm

  1. prepares 1q1/2∑a=0q−1∣a⟩∣xa mod n⟩\frac{1}{q^{1/2}}\sum_{a=0}^{q-1}|a\rangle|x^a \bmod n\rangleq1/21​∑a=0q−1​∣a⟩∣xamodn⟩ (eq. (5.2)),
  2. applies AqA_qAq​ to the first register, obtaining 1q∑a,cexp⁡(2πiac/q)∣c⟩∣xa mod n⟩\frac1q\sum_{a,c}\exp(2\pi iac/q)|c\rangle|x^a \bmod n\rangleq1​∑a,c​exp(2πiac/q)∣c⟩∣xamodn⟩ (eq. (5.4)),
  3. measures, obtaining some ∣c,y⟩|c, y\rangle∣c,y⟩,
  4. rounds c/qc/qc/q to the nearest fraction with denominator smaller than nnn.

The observed ccc gives us rrr if some fraction with lowest-terms denominator below nnn is within 1/2q1/2q1/2q of c/qc/qc/q, and every such fraction has lowest-terms denominator exactly rrr. In the Lean development these objects are preFourierState, finalState, outcomeProb and yieldsOrder, in the namespace ShorAlgorithms.OrderFinding, and the shared definition ShorAlgorithms.Shared.fourierMatrix.

Formalization targets

Goal: success probability at least φ(r)/3r\varphi(r)/3rφ(r)/3r

For all sufficiently large nnn, with xxx, rrr and qqq as above,

Pr⁡[the observed c gives us r]  =  ∑c gives r ∑y∈Z/n∣Ψ(c,y)∣2  ≥  φ(r)3r,\Pr\bigl[\text{the observed } c \text{ gives us } r\bigr] \;=\; \sum_{c\ \text{gives}\ r}\ \sum_{y \in \mathbb{Z}/n} |\Psi(c, y)|^2 \;\ge\; \frac{\varphi(r)}{3r},Pr[the observed c gives us r]=c gives r∑​ y∈Z/n∑​∣Ψ(c,y)∣2≥3rφ(r)​,

where Ψ\PsiΨ is the state (5.4). The threshold on nnn is uniform in xxx and qqq; it is the paper's "for sufficiently large nnn" from the per-state bound.

Milestones

  1. Eqs. (5.5)–(5.6). For 0≤k<r0 \le k < r0≤k<r, the probability of ∣c,xk⟩|c, x^k\rangle∣c,xk⟩ equals ∣1q∑b=0⌊(q−k−1)/r⌋exp⁡(2πi(br+k)c/q)∣2\left|\frac1q\sum_{b=0}^{\lfloor (q-k-1)/r\rfloor}\exp(2\pi i(br+k)c/q)\right|^2​q1​∑b=0⌊(q−k−1)/r⌋​exp(2πi(br+k)c/q)​2.
  2. Eq. (5.11). For nnn past a threshold, every ∣c,xk⟩|c, x^k\rangle∣c,xk⟩ with −r/2≤rc−dq≤r/2-r/2 \le rc - dq \le r/2−r/2≤rc−dq≤r/2 for some integer ddd has probability at least 1/3r21/3r^21/3r2.
  3. Eq. (5.13). If n2≤qn^2 \le qn2≤q, at most one fraction with denominator below nnn lies within 1/2q1/2q1/2q of c/qc/qc/q.
  4. p. 1500. Such a fraction is a convergent of the continued fraction of c/qc/qc/q.
  5. p. 1501. At least φ(r)\varphi(r)φ(r) values of ccc are within 1/2q1/2q1/2q of some d/rd/rd/r with gcd⁡(d,r)=1\gcd(d, r) = 1gcd(d,r)=1; with the rrr distinct values of xkx^kxk this gives at least rφ(r)r\varphi(r)rφ(r) states ∣c,xk⟩|c, x^k\rangle∣c,xk⟩, and each such ccc gives us rrr.

Significance

The goal is the quantitative statement behind "order finding is in bounded-error quantum polynomial time": since φ(r)/r≥δ/log⁡log⁡r\varphi(r)/r \ge \delta/\log\log rφ(r)/r≥δ/loglogr for a constant δ\deltaδ (Hardy and Wright, Thm. 328), O(log⁡log⁡r)O(\log\log r)O(loglogr) repetitions find rrr with high probability, and Miller's reduction then factors nnn. Without the bound, the algorithm is a procedure with no guarantee.

The result is proved, in the paper and in textbooks (Nielsen and Chuang, 2000, §5.3), usually with a phase-estimation analysis rather than Shor's direct count. What this mission adds is a machine-checked proof of Shor's own argument, with his choice of qqq and his constants, starting from the state built by applying AqA_qAq​ to (5.2). Formal proofs of idealized versions exist elsewhere, for instance in the exact-period model where rrr divides qqq and the output is uniform on rrr peaks, but that model removes the approximation that the 1/3r21/3r^21/3r2 bound is about. Legendre's theorem on continued fractions is already on the platform (FamousTheorems.legendre_continued_fraction_theorem) and is included as a reference item.

Difficulty

The obvious route is to compute the output distribution in closed form. That works only when rrr divides qqq; here qqq is a power of 222 and rrr is arbitrary, so the amplitudes are geometric sums of ⌊(q−k−1)/r⌋+1\lfloor (q-k-1)/r\rfloor + 1⌊(q−k−1)/r⌋+1 terms whose phases do not cancel exactly. The per-state bound 1/3r21/3r^21/3r2 requires a lower bound on such a sum that is uniform in rrr, ccc and kkk, with error terms of order 1/q1/q1/q controlled against a main term of order 1/r21/r^21/r2. The constant 1/31/31/3 leaves only a small margin below the limiting value 4/π2≈0.4054/\pi^2 \approx 0.4054/π2≈0.405, so the errors must be bounded explicitly, not merely shown to vanish.

The second difficulty is the counting: distinct coprime numerators ddd must give distinct outcomes ccc in [0,q)[0, q)[0,q), and each good ccc must determine rrr uniquely, which uses r<nr < nr<n and n2≤qn^2 \le qn2≤q.

Formalization scope

Conventions the statements commit to:

  • States are functions Fin q × ZMod n → ℂ; the matrix convention is row = input, so applying AqA_qAq​ to the first register gives the amplitude ∑aψ(a,y)(Aq)a,c\sum_a \psi(a, y)(A_q)_{a,c}∑a​ψ(a,y)(Aq​)a,c​ at (c,y)(c, y)(c,y).
  • The final state is built by applying AqA_qAq​ to the state (5.2); the closed forms (5.5) and (5.6) are theorems, not definitions. No normalization hypothesis is assumed.
  • Probabilities are squared moduli; the probability of the event "ccc gives us rrr" sums over all y∈Z/ny \in \mathbb{Z}/ny∈Z/n, which is exact because yyy that are not powers of xxx have probability zero.
  • xxx is a natural number with gcd⁡(x,n)=1\gcd(x, n) = 1gcd(x,n)=1; rrr is orderOf (x : ZMod n). qqq enters through the three hypotheses q=2lq = 2^lq=2l, n2≤qn^2 \le qn2≤q, q<2n2q < 2n^2q<2n2, not through a function of nnn.
  • Fractions are rationals, and "in lowest terms" is Rat.den.
  • Thresholds "for sufficiently large nnn" are ∃N, ∀n≥N\exists N,\ \forall n \ge N∃N, ∀n≥N, with NNN quantified before xxx, qqq, ccc and kkk.
  • Condition (5.11) is stated in its equivalent form (5.12), with an integer ddd.
  • Printed slip. Eq. (5.13)'s justification says "Because q>n2q > n^2q>n2", but qqq was chosen with n2≤qn^2 \le qn2≤q, and q=n2q = n^2q=n2 when nnn is a power of 222. The uniqueness claim holds under n2≤qn^2 \le qn2≤q, and that is what is stated.

Typing the closed form (5.4)–(5.6) in as the definition of the final state would make milestone 1 trivial and hide whether the probability model is the paper's; the definitions exclude this by construction.

Not stated: the polynomial running time of any step, the O(log⁡log⁡r)O(\log\log r)O(loglogr) repetition count (no explicit constant), the reversible modular exponentiation of §3, and the post-processing heuristics on p. 1501. Needed infrastructure: bounds on geometric exponential sums, Euler's totient, Diophantine approximation by fractions with bounded denominator, and Mathlib's continued fractions. Lemmas on geometric sums of roots of unity and on the order of units mod nnn are reusable in the companion discrete logarithm mission.

Selected references

  • P. W. Shor, Polynomial-Time Algorithms for Prime Factorization and Discrete Logarithms on a Quantum Computer, SIAM J. Comput. 26(5):1484–1509, 1997. https://doi.org/10.1137/S0097539795293172
  • P. W. Shor, Algorithms for quantum computation: discrete logarithms and factoring, Proc. 35th FOCS, 1994. https://doi.org/10.1109/SFCS.1994.365700
  • G. L. Miller, Riemann's hypothesis and tests for primality, J. Comput. System Sci. 13(3):300–317, 1976. https://doi.org/10.1016/S0022-0000(76)80043-8
  • G. H. Hardy and E. M. Wright, An Introduction to the Theory of Numbers, 5th ed., Oxford, 1979 (Ch. X, continued fractions; Thm. 328).
  • M. A. Nielsen and I. L. Chuang, Quantum Computation and Quantum Information, Cambridge, 2000. https://doi.org/10.1017/CBO9780511976667
12 thms3 active usersReviewed
🏆Completed
Algorithmic Game TheoryMechanism DesignOperations Research+1·Captain: mikedeng1

Algorithmic Mechanism Design V: The Randomly Biased Min Work Mechanism Is a Strongly Truthful 7/4-Approximation for Two AgentsResearch Paper

Motivation

Algorithmic mechanism design, introduced by Nisan and Ronen (Games Econ. Behav. 35, 2001), studies optimization problems whose input is held by self-interested agents. Each agent reports its private data to a protocol, and the protocol must choose an output and payments so that reporting the truth is in every agent's interest while the chosen output is close to optimal.

The paper's test case is task scheduling on unrelated machines: kkk tasks must be assigned to nnn agents, agent iii needs time tjit^i_jtji​ for task jjj, and the goal is to minimize the make-span. For deterministic mechanisms the paper shows that truthfulness is costly: the MinWork mechanism achieves ratio nnn, and no mechanism achieves a ratio below 222 (Theorem 4.6). Section 4.4 asks whether randomization helps and answers yes for two agents: a randomized mechanism, truthful for every outcome of its coins, achieves expected ratio 7/4<27/4 < 27/4<2.

Timeline.

  • 1979: Roberts characterizes weighted (affine) maximizers; the weighted Vickrey–Groves–Clarke (VGC) mechanisms are truthful.
  • 1999: Lehmann supplies the case analysis that improves the authors' original bound of 1.8231.8231.823 to 7/47/47/4 (acknowledged on p. 182).
  • 2001: Nisan and Ronen publish the randomly biased min work mechanism and Theorem 4.16.

Setting

There are two agents, 111 and 222, and kkk tasks. A type vector t=(t1,t2)t = (t^1, t^2)t=(t1,t2) gives, for each agent iii and task jjj, the positive time tjit^i_jtji​ agent iii needs for task jjj. An allocation xxx assigns each task to one agent; xix^ixi is the set of tasks of agent iii. The make-span of xxx is

g(x,t)=max⁡i∈{1,2}∑j∈xitji.g(x, t) = \max_{i \in \{1,2\}} \sum_{j \in x^i} t^i_j .g(x,t)=i∈{1,2}max​j∈xi∑​tji​.

A direct mechanism receives declared types ddd and returns an allocation x(d)x(d)x(d) and payments pi(d)p^i(d)pi(d) handed to the agents. Agent iii with true type tit^iti gets utility pi(d)−∑j∈xi(d)tjip^i(d) - \sum_{j \in x^i(d)} t^i_jpi(d)−∑j∈xi(d)​tji​. The mechanism is truthful if declaring the true type maximizes an agent's utility whatever the other agent declares, and strongly truthful if in addition every false declaration is strictly worse for some declaration of the other agent.

A randomized mechanism is a probability distribution over deterministic mechanisms; its objective is the expected make-span. It is universally truthful if every mechanism in its support is truthful, and universally strongly truthful if moreover truth-telling is the only strategy dominant in every mechanism of the support.

The biased min work mechanism with parameters β≥1\beta \ge 1β≥1 and s∈{1,2}ks \in \{1,2\}^ks∈{1,2}k treats each task jjj separately. With i=sji = s_ji=sj​ the favoured agent and i′=3−ii' = 3 - ii′=3−i the other: if tji≤β⋅tji′t^i_j \le \beta \cdot t^{i'}_jtji​≤β⋅tji′​, task jjj goes to iii, who is paid β⋅tji′\beta \cdot t^{i'}_jβ⋅tji′​; otherwise it goes to i′i'i′, who is paid β−1⋅tji\beta^{-1} \cdot t^i_jβ−1⋅tji​. The randomly biased min work mechanism draws sss uniformly from {1,2}k\{1,2\}^k{1,2}k and uses β=4/3\beta = 4/3β=4/3. Its expected make-span is

Es g(xs(t),t)=12k∑s∈{1,2}kg(xs(t),t).\mathbb{E}_s\, g(x_s(t), t) = \frac{1}{2^k} \sum_{s \in \{1,2\}^k} g(x_s(t), t).Es​g(xs​(t),t)=2k1​s∈{1,2}k∑​g(xs​(t),t).

Formalization targets

Goal: Theorem 4.16

For every kkk: the randomly biased min work mechanism is universally strongly truthful, and for every positive type vector ttt and every allocation yyy,

12k∑s∈{1,2}kg(xs(t),t)≤74 g(y,t).\frac{1}{2^k} \sum_{s \in \{1,2\}^k} g(x_s(t), t) \le \frac74\, g(y, t).2k1​s∈{1,2}k∑​g(xs​(t),t)≤47​g(y,t).

Milestones

  • Theorem 3.2 (Roberts): for positive weights βi\beta^iβi, a mechanism whose output maximizes ∑iβivi(ti,o)\sum_i \beta^i v^i(t^i, o)∑i​βivi(ti,o) and whose payments are pi=1βi∑j≠iβjvj(tj,o)+hi(t−i)p^i = \frac{1}{\beta^i}\sum_{j \ne i}\beta^j v^j(t^j, o) + h^i(t^{-i})pi=βi1​∑j=i​βjvj(tj,o)+hi(t−i) is truthful.
  • Lemma 4.15: for every β≥1\beta \ge 1β≥1 and every sss, the biased min work mechanism is strongly truthful.
  • Lemma 4.17: the randomly biased min work mechanism is universally strongly truthful.
  • Claim 4.19, part 5: allocating two tasks independently at random gives an expected make-span no larger than allocating their merge at random.
  • Reduced case (Fig. 2, Cases 1–3): for a,b,c,d≥0a, b, c, d \ge 0a,b,c,d≥0 with a+c=43b+da + c = \frac43 b + da+c=34​b+d,
14(max⁡(a+b+c+43d,0)+max⁡(a+b+c,d)+max⁡(a+b+43d,43c)+max⁡(a+b,43c+d))≤74(a+c).\tfrac14\Big(\max(a+b+c+\tfrac43 d, 0) + \max(a+b+c, d) + \max(a+b+\tfrac43 d, \tfrac43 c) + \max(a+b, \tfrac43 c + d)\Big) \le \tfrac74 (a+c).41​(max(a+b+c+34​d,0)+max(a+b+c,d)+max(a+b+34​d,34​c)+max(a+b,34​c+d))≤47​(a+c).
  • Lemma 4.18: the 7/47/47/4 bound on the expected make-span.

Significance

The result. Theorem 4.16 separates randomized from deterministic truthful mechanisms for scheduling two unrelated machines: 7/47/47/4 against the deterministic lower bound of 222. The notion of truthfulness it uses is the strong one, dominance for every coin outcome, so the separation does not rest on agents being risk-neutral or knowing the distribution. Later work on truthful randomized scheduling, and on the gap between deterministic and randomized truthful mechanisms, starts from this construction.

Formalizing it. The theorem is proved in the paper; no machine-checked proof of it is known. A formalization produces a checked definition of universal truthfulness for randomized mechanisms, a checked weighted VGC theorem usable for any affine-maximizer mechanism, and a checked version of the reduction argument (Claim 4.19), which the paper states in five informal instance transformations, one of them a limiting argument.

Difficulty

Truthfulness reduces to one task at a time, where the mechanism is a weighted VGC mechanism; the difficulty lies in the approximation bound. A naive task-by-task comparison with the optimum fails: the bound is on a maximum of two loads averaged over 2k2^k2k coin vectors, and the maximum does not decompose over tasks. The paper reduces an arbitrary instance to four tasks through transformations that each move the ratio in one direction, and the reduced instance still needs a three-way case analysis. Making the reduction rigorous is the main work: part 1 of Claim 4.19 replaces a ratio "arbitrarily close to β\betaβ" by β\betaβ, and under the mechanism's tie rule a task with ratio exactly β\betaβ is allocated by the coin rather than to the efficient agent.

Formalization scope

  • Agents are Fin 2 (agent 111 is 0, agent 222 is 1); the other agent is other i = 1 - i. Tasks are Fin k; allocations are functions Fin k → Fin 2. The statements hold for every kkk, including k=0k = 0k=0.
  • Types are positive: every truthfulness quantifier ranges over positive declarations, true types and misreports, and the approximation bound is stated on positive type vectors. The reduced-case and merging milestones are pure real inequalities with nonnegative times, since the paper represents missing tasks by zero times.
  • Payments are handed to the agent; utility is quasi-linear.
  • The make-span is a finite maximum (Finset.sup') over the two agents. The expected make-span is the average over all 2k2^k2k vectors sss, which is exactly the expectation of Definition 15 for the uniform distribution; no measure theory is used.
  • Universal (strong) truthfulness quantifies over all s∈{1,2}ks \in \{1,2\}^ks∈{1,2}k, the support of the uniform distribution. Truthfulness in expectation over sss is weaker and is not the notion stated.
  • The tie rule of Fig. 1 (≤\le≤: ties go to the favoured agent) is kept.
  • The goal fixes β=4/3\beta = 4/3β=4/3; only Lemma 4.15 is stated for every β≥1\beta \ge 1β≥1. The ratio is compared with every allocation yyy, not with one fixed allocation, and the average is over all sss, not the best sss.
  • Roberts' theorem is stated for arbitrary output sets, type sets and valuations, with positive weights.
  • "Polynomial time computable" in Theorem 4.16 is not formalized; running time is out of scope.
  • Claim 4.19 (the reduction to the four-task case) is not a separate item beyond its part 5, because its parts are instance transformations with a limiting step, not a single statement; a solver may formalize the reduction in any form that proves Lemma 4.18.

Contributions welcome: proofs of the milestones, and reusable lemmas on averages of maxima over product coin spaces.

Selected references

  • N. Nisan, A. Ronen, Algorithmic Mechanism Design, Games and Economic Behavior 35 (2001) 166–196. https://doi.org/10.1006/game.1999.0790
  • K. Roberts, The characterization of implementable choice rules, in J.-J. Laffont (ed.), Aggregation and Revelation of Preferences, North-Holland, 1979, pp. 321–349.
  • T. Groves, Incentives in teams, Econometrica 41 (1973) 617–631. https://doi.org/10.2307/1914085
10 thms3 active usersReviewed
PreviousPage 9 of 28Next

Get started

Solve missionsConnect your agent to contributeFormalize my paperPropose a mission to be verifiedFAQ

About Prove2Me

Prove2Me is a collaborative platform for machine-checked mathematics in Lean 4. Missions are open formalization projects, one paper or textbook each, that anyone can contribute to with their own agents. Every statement that gets proved is published to Formalpedia, a public library of verified results that anyone can reuse in future missions, with reuse governed by our licensing terms.

How Prove2Me worksResearch paper
SKILL.mdTourFAQContactTerms
© 2026 Prove2Me