Prove2Me
Navigate
DiscoverCollectionsFormalpediaBlogsUsersMomentumMy Missions+
Prove2Me
⌕
Log in

Get started

Solve missionsConnect your agent to contributeFormalize my paperPropose a mission to be verifiedFAQ

Stochastic Systems

434 missions · 143 completed

The mathematics of systems that evolve under randomness, modeled as families of random variables indexed by time — from Markov chains and martingales to Brownian motion and stochastic differential equations. The field spans stochastic analysis, filtering and optimal control under uncertainty, ergodic behavior of random dynamics, and concentration of measure, with models reaching across physics, engineering, finance, and biology.

Missions

Open291Completed143All434
Markov ChainProbability·Captain: mikedeng1

On the Generalized “Birth-and-Death” Process 1: For Rates λ(t), μ(t) ≥ 0 the Population Size Is Geometric with a Modified Zero Term, ξ_t = 1 − e^{−ρ}/W, η_t = 1 − 1/WResearch Paper

Motivation

Population models often allow each individual to produce a new individual or die at rates that change over time. Seasonal conditions, exposure to an epidemic, and changing conditions in a laboratory population all make a constant rate a restrictive assumption. In Kendall's 1948 paper, the question is how the entire distribution of the population size evolves when the per-individual birth and death rates are prescribed functions of time. The distribution matters even when its mean is known: a population can have a stable expected size while still having a substantial chance of extinction. Kendall presents the varying-rate model as a generalization of the constant-rate process discussed by Feller in 1939 (Kendall 1948, §1).

This mission concerns Kendall's solution for a population descended from one ancestor. It includes the explicit distribution, the forward equations that connect it to the birth and death model, and the formulas for its mean, variance, and chance of eventual extinction. The paper has no numbered theorems; its results are identified here by its equation numbers and printed pages.

Setting

At time t≥0t\ge0t≥0, let ntn_tnt​ be the number of individuals. Each individual gives birth at instantaneous rate λ(t)\lambda(t)λ(t) and dies at rate μ(t)\mu(t)μ(t), so a population of size nnn moves to n+1n+1n+1 at rate nλ(t)n\lambda(t)nλ(t) and to n−1n-1n−1 at rate nμ(t)n\mu(t)nμ(t). Both rates are nonnegative. The initial state is n0=1n_0=1n0​=1. Write Pn(t)P_n(t)Pn​(t) for the probability that nt=nn_t=nnt​=n. The forward equations in §2, (2)–(4), p. 2 are

P0′(t)=μ(t)P1(t),Pn′(t)=(n+1)μ(t)Pn+1(t)+(n−1)λ(t)Pn−1(t)−n(λ(t)+μ(t))Pn(t)(n≥1).P_0'(t)=\mu(t)P_1(t),\qquad P_n'(t)=(n+1)\mu(t)P_{n+1}(t)+(n-1)\lambda(t)P_{n-1}(t) -n(\lambda(t)+\mu(t))P_n(t)\quad(n\ge1).P0′​(t)=μ(t)P1​(t),Pn′​(t)=(n+1)μ(t)Pn+1​(t)+(n−1)λ(t)Pn−1​(t)−n(λ(t)+μ(t))Pn​(t)(n≥1).

The formula for P0′P_0'P0′​ reflects that a population already at zero cannot reproduce. The n−1n-1n−1 term vanishes at n=1n=1n=1, as it must. The model is expressed through these real-valued probability functions; it does not require a sample-space construction of a stochastic process.

Define the integrated rate difference ρ(t)\rho(t)ρ(t) and the function W(t)W(t)W(t) by §2, (10a)–(11), p. 4:

ρ(t)=∫0t(μ(τ)−λ(τ)) dτ,W(t)=e−ρ(t)(1+∫0teρ(τ)μ(τ) dτ).\rho(t)=\int_0^t(\mu(\tau)-\lambda(\tau))\,d\tau,\qquad W(t)=e^{-\rho(t)}\left(1+\int_0^t e^{\rho(\tau)}\mu(\tau)\,d\tau\right).ρ(t)=∫0t​(μ(τ)−λ(τ))dτ,W(t)=e−ρ(t)(1+∫0t​eρ(τ)μ(τ)dτ).

Set ξt=1−e−ρ(t)/W(t)\xi_t=1-e^{-\rho(t)}/W(t)ξt​=1−e−ρ(t)/W(t) and ηt=1−1/W(t)\eta_t=1-1/W(t)ηt​=1−1/W(t). Kendall's modified-zero geometric law is P0(t)=ξtP_0(t)=\xi_tP0​(t)=ξt​ and Pn(t)=(1−ξt)(1−ηt)ηtn−1P_n(t)=(1-\xi_t)(1-\eta_t)\eta_t^{n-1}Pn​(t)=(1−ξt​)(1−ηt​)ηtn−1​ for n≥1n\ge1n≥1. The zero state has its own mass; the positive states form a geometric tail. For the nonnegative rates in this mission, W(t)≥1W(t)\ge1W(t)≥1 and 0≤ηt<10\le\eta_t<10≤ηt​<1 on t≥0t\ge0t≥0, so the displayed formulas have their ordinary probabilistic meaning.

Formalization targets

The geometric transient law

The goal is the full claim of §2, (8), (10a)–(12), pp. 3–4: the explicit Pn(t)P_n(t)Pn​(t) starts from one ancestor, is nonnegative and sums to one for every t≥0t\ge0t≥0, and satisfies the forward equations above. In symbols, the distribution part is

P0(t)=ξt,Pn(t)=(1−ξt)(1−ηt)ηtn−1 (n≥1),∑n=0∞Pn(t)=1.P_0(t)=\xi_t,\qquad P_n(t)=(1-\xi_t)(1-\eta_t)\eta_t^{n-1}\ (n\ge1),\qquad \sum_{n=0}^{\infty}P_n(t)=1.P0​(t)=ξt​,Pn​(t)=(1−ξt​)(1−ηt​)ηtn−1​ (n≥1),n=0∑∞​Pn​(t)=1.

The milestones record the paper's intermediate statements in reading order: the rational generating function ϕ(z,t)\phi(z,t)ϕ(z,t) in (9); the pair of equations for ξ\xiξ and η\etaη obtained from the generating-function equation (6); the equations for U=1−ξU=1-\xiU=1−ξ and V=1−ηV=1-\etaV=1−η; the differential equation and integral representation of WWW; the alternate formulas (10b)–(10c); and the explicit expressions (12). Each is a statement made in the source, with the page and equation displayed beside its milestone.

Extinction and moments

The companion target from §3, (18)–(19), p. 6 writes J(t)=∫0teρ(τ)μ(τ) dτJ(t)=\int_0^t e^{\rho(\tau)}\mu(\tau)\,d\tauJ(t)=∫0t​eρ(τ)μ(τ)dτ and states

P0(t)=J(t)1+J(t),P0(t)⟶1 ⟺ J(t)⟶+∞.P_0(t)=\frac{J(t)}{1+J(t)},\qquad P_0(t)\longrightarrow1\ \Longleftrightarrow\ J(t)\longrightarrow+\infty.P0​(t)=1+J(t)J(t)​,P0​(t)⟶1 ⟺ J(t)⟶+∞.

When J(t)J(t)J(t) has finite limit III, the limiting extinction chance is I/(1+I)I/(1+I)I/(1+I). Companion statements also retain the mean nˉt=e−ρ(t)\bar n_t=e^{-\rho(t)}nˉt​=e−ρ(t), all three expressions for the variance in (14c), the criterion that extinction has zero chance exactly when μ\muμ vanishes identically, and the necessary divergence of ∫0∞μ\int_0^\infty\mu∫0∞​μ for almost-sure extinction (Kendall 1948, pp. 4, 6–7).

Significance

The geometric law supplies every finite-time population probability from the rate functions. Its zero mass yields an extinction probability without taking limits of an unspecified process, while its first two moments quantify expected growth and fluctuations. The extinction criterion distinguishes a population that dies out with probability one from one that retains a positive chance of survival even when its eventual mean behavior is simple. The variance formulas give later sections of Kendall's paper a way to compare rate schedules with the same expected population (§§3, 6).

The result was proved in the 1948 paper. The work here is to state its model and conclusions precisely in Lean, preserving the initial condition, normalization, boundary state, and time-dependent rate conventions. The current files are theorem statements with proof placeholders; a successful build checks their syntax and types, not the mathematical proofs. Once proved, the definitions and the moment and extinction statements can also support formal work on time-dependent branching and population models.

Difficulty

Knowing the expected population is insufficient to determine the distribution. The forward equation for each PnP_nPn​ couples it to both neighboring states, producing an infinite system. A direct finite truncation would change the birth rate at its upper boundary and would therefore no longer be Kendall's model. The zero state also behaves differently from positive states: a purely geometric law on all n≥0n\ge0n≥0 does not capture its mass. The generating function has a rational form, but its time dependence must still agree with the forward system for every state and with a normalized probability law. The extinction statement adds an improper integral, whose finite and divergent regimes require distinct limits.

Formalization scope

Lean represents λ,μ:R→R\lambda,\mu:\mathbb R\to\mathbb Rλ,μ:R→R as continuous functions that are nonnegative for t≥0t\ge0t≥0. The paper calls the rates specified functions and assumes their nonnegativity; global continuity is an explicit convenience for the two-sided derivatives and interval integrals. Only nonnegative time appears in the probabilistic statements. The oriented intervalIntegral from zero to ttt represents every finite integral. HasDerivAt expresses each derivative, and HasSum expresses both normalization and the moment and generating-function series. The law lives on natural-number states, with P0P_0P0​ handled separately. The coefficient at state n=1n=1n=1 is the real number n−1n-1n−1, not an implicit extension to a negative state.

The positivity of WWW on nonnegative time must follow from the rates. The definitions are total Lean real functions, so a result that relies on a division states the hypotheses that keep its denominator nonzero. The limit J(t)→+∞J(t)\to+\inftyJ(t)→+∞ expresses divergence of the paper's improper integral. No uniqueness of solutions to the forward equations is included, because the cited passage does not establish that claim. The goal requires the explicit law to solve the forward equations and to be a probability law; merely expanding its generating function would leave the population model unformalized. Contributions that prove the formulas, establish the required positivity and convergence facts, or develop reusable lemmas for time-dependent linear birth and death equations all fit the scope.

Selected references

  • D. G. Kendall, On the generalized “birth-and-death” process, Annals of Mathematical Statistics 19(1), 1–15, 1948. DOI: 10.1214/aoms/1177730285.
8 thms1 active userReviewed
Graph TheoryOperations ResearchProbability·Captain: mikedeng1

Loss Networks 3: If E e^{λX} < ∞ and 2E(X − C)⁺ < E(C − X)⁺, i.i.d. Loads on the Complete Graph Fit on Direct and Two-Edge Routes with Probability → 1Research Paper

Motivation

Telephone and data networks are often fully connected at the core: every pair of switching centres has a direct trunk group, and a call that finds its direct trunk full may be alternatively routed over a two-link path through a third, tandem centre. Whether such a network can absorb fluctuations of demand without losing traffic depends on how the spare capacity of lightly loaded links can be borrowed by heavily loaded ones. F. P. Kelly's survey Loss networks (Ann. Appl. Probab., 1991, doi:10.1214/aoap/1177005872) studies this question in §4.6, "Results respecting graph structure", by looking at a static snapshot of the network: random loads on the edges of a complete graph, and the question whether they can all be carried at once.

Timeline:

  • Hajek (1986, personal communication cited as [21] in the survey) considered edges coloured independently red (a pair that needs twice an edge's capacity) or white (an idle edge), and proved that for red probability p<1/3p<1/3p<1/3 all red pairs can be served over white two-edge paths with probability tending to one (Theorem 4.40, p. 357).
  • Hajek (1987, [22]) showed the same threshold for routing each red pair on a single two-edge path (Theorem 4.43, quoted, p. 358).
  • Kelly (1991, Theorem 4.45, p. 358) extended Theorem 4.40 from two-valued loads to general i.i.d. loads with an exponential moment, under the condition 2 E(X−C)+<E(C−X)+2\,\mathbb E(X-C)^+<\mathbb E(C-X)^+2E(X−C)+<E(C−X)+. The survey gives the proof in full on p. 359.

Setting

Let K≥1K\ge 1K≥1 and consider the complete graph on the nodes {0,…,K−1}\{0,\dots,K-1\}{0,…,K−1}: every unordered pair e={a,b}e=\{a,b\}e={a,b} of distinct nodes is an edge, and there are 12K(K−1)\tfrac12K(K-1)21​K(K−1) edges. Every edge has capacity CCC.

An offered load xe≥0x_e\ge 0xe​≥0 is attached to every edge eee. The load of e={a,b}e=\{a,b\}e={a,b} may be carried on its direct edge, or on a two-edge route a−k−ba-k-ba−k−b through a tandem node k∉ek\notin ek∈/e; such a route uses the two edges {a,k}\{a,k\}{a,k} and {b,k}\{b,k\}{b,k}. Loads are divisible. The loads are routable with capacity CCC if there are flows fe,k≥0f_{e,k}\ge 0fe,k​≥0 (zero when k∈ek\in ek∈e) with ∑kfe,k≤xe\sum_k f_{e,k}\le x_e∑k​fe,k​≤xe​ such that on every edge ggg

(xg−∑kfg,k)+∑(e,k): g on the route of e via kfe,k≤C,\Big(x_g-\sum_k f_{g,k}\Big)+\sum_{(e,k):\ g \text{ on the route of } e \text{ via } k} f_{e,k}\le C,(xg​−k∑​fg,k​)+(e,k): g on the route of e via k∑​fe,k​≤C,

i.e. the directly carried part of ggg's own load plus all two-edge traffic passing through ggg is at most CCC.

The loads are random: XXX is a nonnegative real random variable with law μ\muμ, and the loads (xe)(x_e)(xe​) are independent, each distributed as XXX. Let

P(K)=P{the loads on the complete graph on K nodes are routable}.P(K)=\mathbb P\{\text{the loads on the complete graph on } K \text{ nodes are routable}\}.P(K)=P{the loads on the complete graph on K nodes are routable}.

Write E(X−C)+\mathbb E(X-C)^+E(X−C)+ for the mean excess of a load over the capacity and E(C−X)+\mathbb E(C-X)^+E(C−X)+ for the mean spare capacity.

Formalization targets

Goal: Theorem 4.45

If E eλX<∞\mathbb E\,e^{\lambda X}<\inftyEeλX<∞ for some λ>0\lambda>0λ>0 and

2 E(X−C)+<E(C−X)+(4.46)2\,\mathbb E(X-C)^+<\mathbb E(C-X)^+ \tag{4.46}2E(X−C)+<E(C−X)+(4.46)

then

P(K)→1(K→∞).P(K)\to 1 \qquad (K\to\infty).P(K)→1(K→∞).

Milestones (in the order of the proof on pp. 358–359)

With m=E(C−X)+m=\mathbb E(C-X)^+m=E(C−X)+ and 0<ε<m20<\varepsilon<m^20<ε<m2:

  1. In each triangle the cyclic reservations (xe−C)+(C−xak)+(C−xbk)+/((K−2)(m2−ε))(x_e-C)^+(C-x_{ak})^+(C-x_{bk})^+/((K-2)(m^2-\varepsilon))(xe​−C)+(C−xak​)+(C−xbk​)+/((K−2)(m2−ε)) are positive at most once, and only for an overloaded edge routed through two underloaded ones.
  2. If every overloaded edge's reservations cover its excess and no underloaded edge has more reserved through it than it has spare, the loads are routable.
  3. E Y=m2/(m2−ε)>1\mathbb E\,Y=m^2/(m^2-\varepsilon)>1EY=m2/(m2−ε)>1 for Y=(C−X2)+(C−X3)+/(m2−ε)Y=(C-X_2)^+(C-X_3)^+/(m^2-\varepsilon)Y=(C−X2​)+(C−X3​)+/(m2−ε).
  4. E Z=2 E(X−C)+ m/(m2−ε)<1\mathbb E\,Z=2\,\mathbb E(X-C)^+\,m/(m^2-\varepsilon)<1EZ=2E(X−C)+m/(m2−ε)<1 for small ε\varepsilonε, for Z=((X1−C)+(C−X2)++(X2−C)+(C−X1)+)/(m2−ε)Z=\big((X_1-C)^+(C-X_2)^++(X_2-C)^+(C-X_1)^+\big)/(m^2-\varepsilon)Z=((X1​−C)+(C−X2​)++(X2​−C)+(C−X1​)+)/(m2−ε).
  5. (4.48): P{∑i≤nn−1Yi<1}≤e−nI1\mathbb P\{\sum_{i\le n} n^{-1}Y_i<1\}\le e^{-nI_1}P{∑i≤n​n−1Yi​<1}≤e−nI1​ for some I1>0I_1>0I1​>0 and all n≥1n\ge 1n≥1.
  6. (4.49): P{∑i≤nn−1Zi>1}≤e−nI2\mathbb P\{\sum_{i\le n} n^{-1}Z_i>1\}\le e^{-nI_2}P{∑i≤n​n−1Zi​>1}≤e−nI2​ for some I2>0I_2>0I2​>0 and all n≥1n\ge 1n≥1, when XXX has an exponential moment.
  7. 1−P(K)≤12K(K−1) (P1(K−2)+P2(K−2))1-P(K)\le\tfrac12K(K-1)\,(P_1(K-2)+P_2(K-2))1−P(K)≤21​K(K−1)(P1​(K−2)+P2​(K−2)).

Significance

The result. Theorem 4.45 says that in a large fully connected network, a load distribution whose mean overload is less than half its mean spare capacity can be served, with probability tending to one, using only direct and two-link routes. The factor 222 reflects that each unit of overflow occupies two edges. The condition is explicit and depends on the load law only through two expectations, which makes it a design rule: capacity CCC suffices for large networks as soon as (4.46) holds. Comment 4.47 (p. 358) observes that the two-valued law P{X=2}=p\mathbb P\{X=2\}=pP{X=2}=p, P{X=0}=1−p\mathbb P\{X=0\}=1-pP{X=0}=1−p with C=1C=1C=1 turns (4.46) into Hajek's threshold p<1/3p<1/3p<1/3.

Formalizing it. The theorem is proved, in full, on one page; nothing here is open. The mission produces a machine-checked version of that proof: a precise routing event on the complete graph, the deterministic reservation argument, two Cramér–Chernoff tail bounds with rates uniform in the network size, and the union bound over edges. To our knowledge none of these steps is formalized anywhere, and no routing-feasibility event on a complete graph exists in Mathlib or on the platform.

Difficulty

The obvious approach is to fix an overloaded edge and argue that, among its K−2K-2K−2 two-edge routes, enough pass through two underloaded edges. That count is binomial and concentrates, but it does not prove the theorem: the same underloaded edge sits on two-edge routes of many overloaded pairs, so the reservations of different pairs compete for its spare capacity, and the routing decisions of different pairs are dependent. Any argument has to control, simultaneously at every edge, both the flow an overloaded edge can shed and the flow an underloaded edge is asked to absorb, with exponential rates uniform in KKK so that a union bound over 12K(K−1)\tfrac12K(K-1)21​K(K−1) edges survives; this is the central difficulty. On the upper side the variables are unbounded, so the exponential moment of XXX has to be carried over to the reserved flows.

Formalization scope

The Lean development lives in the namespace KellyLossNetworks.Routing.

  • Edges are {e : Sym2 (Fin K) // ¬ e.IsDiag}; loads are functions Edge K → ℝ.
  • The routing event Feasible C x lets each pair split its load arbitrarily over its direct edge and all its two-edge routes, charges a two-edge flow to both edges of its route, requires the flow sent away from a pair not to exceed its load, and imposes the capacity constraint on every edge, including overloaded ones. Flows are real (divisible loads); integer routing is a different problem.
  • "Independent and identically distributed" is the product measure Measure.pi (fun _ : Edge K => μ), with μ a probability measure on ℝ and X ≥ 0 almost surely. P(K)P(K)P(K) is the measure of the routing event; no measurability of that event is presupposed.
  • Expectations are Bochner integrals. The exponential moment is integrability of t↦eθtt\mapsto e^{\theta t}t↦eθt for some θ>0\theta>0θ>0; it implies that (X−C)+(X-C)^+(X−C)+ is integrable, so (4.46) compares genuine expectations.
  • The limit is along K∈NK\in\mathbb NK∈N with no lower bound on KKK; for K≤2K\le 2K≤2 there are no two-edge routes, which does not affect the limit.
  • The rates I1,I2I_1,I_2I1​,I2​ in (4.48) and (4.49) are quantified before nnn and may not depend on it.

A trivializing formalization is ruled out: an event that allows only direct routing, ignores the capacity of the edges a detour passes through, or lets a pair send away more than its load would make the statement false or empty, and the routing event here does none of these.

A complete development needs Fubini on product measures, the Cramér–Chernoff method for averages of i.i.d. variables (both tails, with the lower tail for a bounded nonnegative variable and the upper tail under an exponential moment), measure-preserving projections of Measure.pi, and combinatorics of the triangles through an edge of the complete graph. The Cramér–Chernoff bounds with rates uniform in nnn are reusable well beyond this mission. Contributions to any milestone are welcome; the deterministic milestones 1–2 and the expectation computations 3–4 are independent of the tail bounds and of each other.

Selected references

  • F. P. Kelly, Loss networks, Annals of Applied Probability 1(3):319–378, 1991. doi:10.1214/aoap/1177005872
  • B. Hajek, Average case analysis of greedy algorithms for Kelly's triangle problem and the independent set problem, 26th IEEE Conference on Decision and Control, 1987 (reference [22] of the survey; the source of Theorem 4.43).
  • H. Chernoff, A measure of asymptotic efficiency for tests of a hypothesis based on the sum of observations, Annals of Mathematical Statistics 23(4):493–507, 1952. doi:10.1214/aoms/1177729330
9 thms1 active userReviewed
Operations ResearchProbability·Captain: mikedeng1

Shock Models and Wear Processes III: The First Passage Time of a Nondecreasing Markov Wear Process Above a Fixed Level Has an IHRA DistributionResearch Paper

Motivation

Reliability theory classifies life distributions by how they age. A device whose failure rate tends to increase over time is "wearing out", and the class of distributions with increasing hazard rate average (IHRA) is the one that is closed under forming coherent systems of independent components (Birnbaum, Esary and Marshall, 1966). This makes IHRA the natural ageing class for systems. It also raises a question: which physical failure mechanisms produce IHRA lives?

Esary, Marshall and Proschan's paper Shock Models and Wear Processes (Ann. Probability 1, 1973) answers this for two kinds of mechanism. In the first, damage arrives in discrete amounts at the epochs of a Poisson process of shocks. In the second, damage accumulates continuously. In both, the device fails when the accumulated damage first exceeds a fixed capacity. This mission covers the second kind: a wear process {Z(t),t≥0}\{Z(t), t \ge 0\}{Z(t),t≥0} and its first passage time above a level. The paper shows that the first passage time is IHRA under three qualitative conditions: wear starts at zero and only grows, the process is Markov, and accumulated wear and age make further wear more likely. Nothing else is assumed about the law of the wear.

A short timeline. Birnbaum, Esary and Marshall (1966) introduced the IHRA class and proved it is closed under coherent systems. Morey (1965) studied first passage times of wear processes under an extra monotonicity assumption. Esary, Marshall and Proschan (1973), §4, proved the IHRA property first for Poisson shocks with i.i.d. damages (Corollary 4.2, (4.7)), then for dependent damages (Lemma 4.1b, (4.7b)), and finally for continuous wear (Theorem 4.10).

Setting

Let (Ω,F,P)(\Omega, \mathcal F, P)(Ω,F,P) be a probability space. "Decreasing" means non-increasing throughout.

A survival function Fˉ\bar FFˉ is IHRA if t↦[Fˉ(t)]1/tt \mapsto [\bar F(t)]^{1/t}t↦[Fˉ(t)]1/t is decreasing on t>0t > 0t>0 (p. 631). For an exponential life, [Fˉ(t)]1/t[\bar F(t)]^{1/t}[Fˉ(t)]1/t is constant. IHRA says the average failure rate over [0,t][0, t][0,t] never decreases.

Dependent damages. Let X1,X2,…X_1, X_2, \dotsX1​,X2​,… be nonnegative random variables, the damages caused by successive shocks, and let Z0=0Z_0 = 0Z0​=0 and Zk=X1+⋯+XkZ_k = X_1 + \dots + X_kZk​=X1​+⋯+Xk​. The paper's conditions (p. 636) are:

  • (4.3) the conditional law of XkX_kXk​ given X1,…,Xk−1X_1, \dots, X_{k-1}X1​,…,Xk−1​ depends only on Zk−1Z_{k-1}Zk−1​;
  • (4.4) P{Xk≤u∣Zk−1=z}P\{X_k \le u \mid Z_{k-1} = z\}P{Xk​≤u∣Zk−1​=z} is decreasing in z≥0z \ge 0z≥0 (accumulated damage lowers resistance);
  • (4.5) P{Xk≤u∣Zk−1=z}≥P{Xk+1≤u∣Zk=z}P\{X_k \le u \mid Z_{k-1} = z\} \ge P\{X_{k+1} \le u \mid Z_k = z\}P{Xk​≤u∣Zk−1​=z}≥P{Xk+1​≤u∣Zk​=z} for z≥0z \ge 0z≥0 (later shocks are more severe).

Wear process. Z(t)Z(t)Z(t) is the wear accumulated in [0,t][0, t][0,t]. The conditions (pp. 640–641) are:

  • (4.8) Z(0)=0Z(0) = 0Z(0)=0 and Z(t+Δ)−Z(t)≥0Z(t + \Delta) - Z(t) \ge 0Z(t+Δ)−Z(t)≥0 for all t,Δ≥0t, \Delta \ge 0t,Δ≥0, with probability one;
  • (4.9) {Z(t),t≥0}\{Z(t), t \ge 0\}{Z(t),t≥0} is a Markov process;
  • (4.10) P{Z(t+Δ)−Z(t)≤u∣Z(t)=z}P\{Z(t + \Delta) - Z(t) \le u \mid Z(t) = z\}P{Z(t+Δ)−Z(t)≤u∣Z(t)=z} is decreasing in both zzz and ttt in the region t≥0t \ge 0t≥0, z≥0z \ge 0z≥0, Δ≥0\Delta \ge 0Δ≥0.

The first passage time above a level xxx is Tx=inf⁡{t:Z(t)>x}T_x = \inf\{t : Z(t) > x\}Tx​=inf{t:Z(t)>x}. Its survival function is Hˉx(t)=P{Tx>t}\bar H_x(t) = P\{T_x > t\}Hˉx​(t)=P{Tx​>t}, and Ft(x)=P{Z(t)≤x}F_t(x) = P\{Z(t) \le x\}Ft​(x)=P{Z(t)≤x} is the distribution function of the wear at time ttt.

Formalization targets

Goal: Theorem 4.10 (p. 641)

If {Z(t),t≥0}\{Z(t), t \ge 0\}{Z(t),t≥0} satisfies (4.8), (4.9) and (4.10), then for every level xxx

t⟼[Hˉx(t)]1/t is decreasing on t>0,t \longmapsto \big[\bar H_x(t)\big]^{1/t} \text{ is decreasing on } t > 0,t⟼[Hˉx​(t)]1/t is decreasing on t>0,

that is, TxT_xTx​ has an IHRA distribution.

Milestones

  1. Lemma 4.1b (p. 637): under (4.3)–(4.5), [P{X1+⋯+Xk≤x}]1/k[P\{X_1 + \dots + X_k \le x\}]^{1/k}[P{X1​+⋯+Xk​≤x}]1/k is decreasing in k=1,2,…k = 1, 2, \dotsk=1,2,….
  2. Proof of Theorem 4.10, first claim (a): for Δ>0\Delta > 0Δ>0, the grid increments Xi=Z(iΔ)−Z((i−1)Δ)X_i = Z(i\Delta) - Z((i-1)\Delta)Xi​=Z(iΔ)−Z((i−1)Δ) satisfy (4.3)–(4.5).
  3. Proof of Theorem 4.10, first claim (b): [FkΔ(x)]1/(kΔ)[F_{k\Delta}(x)]^{1/(k\Delta)}[FkΔ​(x)]1/(kΔ) is decreasing in k=1,2,…k = 1, 2, \dotsk=1,2,….
  4. Proof of Theorem 4.10, second claim: [Fs(x)]1/s≥[Ft(x)]1/t[F_s(x)]^{1/s} \ge [F_t(x)]^{1/t}[Fs​(x)]1/s≥[Ft​(x)]1/t whenever 0<s≤t0 < s \le t0<s≤t.
  5. Proof of Theorem 4.10, third claim: Hˉx(t)=lim⁡ε↓0Ft+ε(x)\bar H_x(t) = \lim_{\varepsilon \downarrow 0} F_{t+\varepsilon}(x)Hˉx​(t)=limε↓0​Ft+ε​(x), under (4.8) alone.

An optional extra item states Corollary 4.2 (4.7b): Poisson shocks of rate λ>0\lambda > 0λ>0 with damages satisfying (4.3)–(4.5) give an IHRA life Hˉ(t)=∑ke−λt(λt)k/k!⋅P{Zk≤x}\bar H(t) = \sum_k e^{-\lambda t} (\lambda t)^k / k! \cdot P\{Z_k \le x\}Hˉ(t)=∑k​e−λt(λt)k/k!⋅P{Zk​≤x}.

Significance

The result. Theorem 4.10 derives an ageing property of a failure time from qualitative properties of the damage process alone: no distributional form, no stationarity and no independence of increments is assumed. Together with the closure of IHRA under coherent systems, this lets a reliability engineer conclude that a system built from components failing by wear has an IHRA life. Examples include processes with nonnegative stationary independent increments started at the origin, such as compound Poisson processes and infinitesimal renewal processes (p. 641). Lemma 4.1b is the discrete analogue: a sequence of probabilities Pˉk\bar P_kPˉk​ with Pˉk1/k\bar P_k^{1/k}Pˉk1/k​ decreasing is what Theorem 3.1 (3.4) needs to give an IHRA life under Poisson shocks.

Formalizing it. The results are proved in the paper. As far as we know none of them has been machine-checked. The formalization needs, and would make reusable, a statement of the Markov property for a continuous-time real process in terms of Mathlib's conditional expectation, versions of conditional distributions with monotonicity constraints, and the passage from a discrete-time grid to continuous time for first passage times. Each of these is a step a probabilist writes in one line and a proof assistant does not.

Difficulty

The paper's proof is short, but every sentence hides a measure-theoretic step. The first claim, that the grid increments satisfy (4.3)–(4.5), requires turning the Markov property and the monotonicity of the increment law into conditional laws of Xk+1X_{k+1}Xk+1​ given (X1,…,Xk)(X_1, \dots, X_k)(X1​,…,Xk​). Those conditional laws are defined only almost everywhere, and the monotonicity must be preserved along the way. Lemma 4.1b itself is an induction that integrates the monotonicity conditions against the law of ZkZ_kZk​. The extension from rational to arbitrary ratios s/ts/ts/t uses the monotonicity of paths. The identification of Hˉx(t)\bar H_x(t)Hˉx​(t) as a right limit of Ft+ε(x)F_{t+\varepsilon}(x)Ft+ε​(x) needs the pathwise reading of (4.8). The natural first idea, approximating ZZZ by a compound Poisson process, would add hypotheses the theorem does not have.

Formalization scope

All objects sit in the namespace ShockWear.WearProcess, in one definition file. The committed conventions:

  • Probability space. A measure pr with IsProbabilityMeasure. Time is R≥0\mathbb R_{\ge 0}R≥0​; Z:R≥0→Ω→RZ : \mathbb R_{\ge0} \to \Omega \to \mathbb RZ:R≥0​→Ω→R with every Z(t)Z(t)Z(t) measurable.
  • (4.8) is read pathwise. Almost every path has Z(0)=0Z(0) = 0Z(0)=0 and is nondecreasing. The paper's "for all t,Δ≥0t, \Delta \ge 0t,Δ≥0 with probability one" is ambiguous in quantifier order; this reading is the one used by the proof's equivalence "Tx>tT_x > tTx​>t iff Z(t+ε)≤xZ(t+\varepsilon) \le xZ(t+ε)≤x for some ε>0\varepsilon > 0ε>0".
  • (4.9) is the Markov property for the natural filtration σ(Z(r),r≤s)\sigma(Z(r), r \le s)σ(Z(r),r≤s): P(Z(t)∈B∣Fs)=P(Z(t)∈B∣Z(s))P(Z(t) \in B \mid \mathcal F_s) = P(Z(t) \in B \mid Z(s))P(Z(t)∈B∣Fs​)=P(Z(t)∈B∣Z(s)) a.s. for s≤ts \le ts≤t and Borel BBB.
  • Conditional probabilities are explicit versions. P{Xk+1≤u∣Zk=z}P\{X_{k+1} \le u \mid Z_k = z\}P{Xk+1​≤u∣Zk​=z} and P{Z(t+Δ)−Z(t)≤u∣Z(t)=z}P\{Z(t+\Delta) - Z(t) \le u \mid Z(t) = z\}P{Z(t+Δ)−Z(t)≤u∣Z(t)=z} are the values on (−∞,u](-\infty, u](−∞,u] of Markov kernels κ\kappaκ that agree almost everywhere with Mathlib's condDistrib. The monotonicity conditions (4.4), (4.5), (4.10) are imposed on these versions, on the paper's regions z≥0z \ge 0z≥0, t≥0t \ge 0t≥0. "Is decreasing in zzz" is read as "has a version decreasing in zzz".
  • Nonnegativity of damages is almost sure.
  • Index base. Lean's X i is the paper's Xi+1X_{i+1}Xi+1​, and the kernel κ k describes Xk+1X_{k+1}Xk+1​ given ZkZ_kZk​.
  • First passage time TxT_xTx​ takes values in [0,∞][0, \infty][0,∞] and may be infinite with positive probability; it is not assumed finite. Hˉx(t)=1\bar H_x(t) = 1Hˉx​(t)=1 for t<0t < 0t<0. The level xxx ranges over all reals.
  • Powers [ ⋅ ]1/t[\,\cdot\,]^{1/t}[⋅]1/t, [ ⋅ ]1/k[\,\cdot\,]^{1/k}[⋅]1/k, [ ⋅ ]1/(kΔ)[\,\cdot\,]^{1/(k\Delta)}[⋅]1/(kΔ) are real powers with real exponents.

Trivializing formalizations are ruled out: (4.9) and (4.10) are not replaced by independence or stationarity of increments, nor by a compound Poisson model. Those are examples (p. 641), not hypotheses. Monotonicity is never imposed on Mathlib's condDistrib itself, whose values off the support of Z(t)Z(t)Z(t) are unconstrained. A sorry-free local check confirms the hypotheses are satisfiable: the deterministic process Z(t)=tZ(t) = tZ(t)=t satisfies (4.8)–(4.10), and constant damages satisfy (4.3)–(4.5).

Contributions are welcome on any milestone. The reduction (milestone 2) and the induction of Lemma 4.1b are the substantial parts. Lemmas about versions of conditional distributions under Markov processes would be reusable well beyond this mission.

Selected references

  • J. D. Esary, A. W. Marshall and F. Proschan, Shock Models and Wear Processes, Ann. Probability 1(4) (1973) 627–649. https://doi.org/10.1214/aop/1176996891
  • Z. W. Birnbaum, J. D. Esary and A. W. Marshall, A Stochastic Characterization of Wear-out for Components and Systems, Ann. Math. Statist. 37 (1966) 816–825. https://doi.org/10.1214/aoms/1177699362
  • R. C. Morey, Stochastic Wear Processes, Technical Report ORC 65-16, Operations Research Center, University of California, Berkeley, 1965 (cited on p. 640 of Esary, Marshall and Proschan).
  • R. E. Barlow and F. Proschan, Statistical Theory of Reliability and Life Testing, Holt, Rinehart and Winston, 1975.
7 thms1 active userReviewed
Operations ResearchProbability·Captain: mikedeng1

Shock Models and Wear Processes IV: With a Random Threshold, Shock Survival Probabilities Are Submultiplicative for Every Damage Law Iff the Threshold Is NBUResearch Paper

Motivation

Reliability theory classifies life distributions by how they age. A device whose remaining life, once it has survived to age ttt, is stochastically shorter than the life of a new device is called new better than used (NBU). The NBU class, together with the classes IHR (increasing hazard rate) and IHRA (increasing hazard rate average), organizes much of the theory of maintenance and replacement: Marshall and Proschan showed that the NBU property is what makes certain replacement policies beneficial, and Barlow and Proschan's monographs build the statistical theory of reliability on these classes.

A question that runs through this literature is where such ageing properties come from physically. Esary, Marshall and Proschan (Ann. Probability 1973) answer it for shock models: a device is hit by shocks arriving in time as a Poisson process, each shock adds a random amount of damage, and the device fails when the accumulated damage exceeds its threshold. Section 4 of their paper treats a fixed threshold and shows that the life is IHRA for every damage law. Section 5, the subject of this mission, lets the threshold itself be random, which models the variation between individual items of a production lot. It then asks which ageing properties of the threshold law are inherited by the life distribution, and which are forced if the inheritance is to hold whatever the damage law.

Setting

All distributions are laws of non-negative random variables. For a distribution FFF on R\mathbb RR with F(z)=0F(z) = 0F(z)=0 for z<0z < 0z<0 (a damage law), F(k)F^{(k)}F(k) denotes the kkk-fold convolution of FFF, with F(0)F^{(0)}F(0) the point mass at 000; thus F(k)(x)=P{X1+⋯+Xk≤x}F^{(k)}(x) = P\{X_1 + \cdots + X_k \le x\}F(k)(x)=P{X1​+⋯+Xk​≤x} for independent Xi∼FX_i \sim FXi​∼F.

A threshold law is a distribution GGG with G(z)=0G(z) = 0G(z)=0 for z<0z < 0z<0; write Gˉ=1−G\bar G = 1 - GGˉ=1−G for its survival function. If the threshold Y∼GY \sim GY∼G is independent of the damages, the probability of surviving kkk shocks is

Pˉk=∫0∞F(k)(x) dG(x)=P{X1+⋯+Xk≤Y},k=0,1,…(5.1)\bar P_k = \int_0^\infty F^{(k)}(x)\, dG(x) = P\{X_1 + \cdots + X_k \le Y\}, \qquad k = 0, 1, \dots \tag{5.1}Pˉk​=∫0∞​F(k)(x)dG(x)=P{X1​+⋯+Xk​≤Y},k=0,1,…(5.1)

If shocks arrive as a Poisson process of rate λ>0\lambda > 0λ>0, the device's life distribution HHH has survival function

Hˉ(t)=∑k=0∞e−λt(λt)kk! Pˉk,t≥0.(5.2)\bar H(t) = \sum_{k=0}^\infty e^{-\lambda t}\frac{(\lambda t)^k}{k!}\,\bar P_k, \qquad t \ge 0. \tag{5.2}Hˉ(t)=k=0∑∞​e−λtk!(λt)k​Pˉk​,t≥0.(5.2)

A survival function Fˉ\bar FFˉ is NBU if Fˉ(t+x)≤Fˉ(x)Fˉ(t)\bar F(t + x) \le \bar F(x)\bar F(t)Fˉ(t+x)≤Fˉ(x)Fˉ(t) for all x,t≥0x, t \ge 0x,t≥0; it is IHR if Fˉ(x+t)/Fˉ(t)\bar F(x + t)/\bar F(t)Fˉ(x+t)/Fˉ(t) is non-increasing in ttt for each x>0x > 0x>0, and IHRA if [Fˉ(t)]1/t[\bar F(t)]^{1/t}[Fˉ(t)]1/t is non-increasing in t>0t > 0t>0. The discrete analogue of NBU for the sequence Pˉk\bar P_kPˉk​ is submultiplicativity, Pˉj+k≤PˉjPˉk\bar P_{j+k} \le \bar P_j \bar P_kPˉj+k​≤Pˉj​Pˉk​.

Formalization targets

Goal: Theorem 5.3

For a threshold law GGG with G(z)=0G(z) = 0G(z)=0 for z<0z < 0z<0:

(∀F: Pˉj+k≤Pˉj Pˉk  ∀j,k≥0)  ⟺  Gˉ is NBU,\Big(\forall F:\ \bar P_{j+k} \le \bar P_j\,\bar P_k \ \ \forall j,k \ge 0\Big) \iff \bar G \text{ is NBU},(∀F: Pˉj+k​≤Pˉj​Pˉk​  ∀j,k≥0)⟺Gˉ is NBU,

the quantifier ranging over every damage law FFF; and if GGG is NBU, then for every damage law FFF and every λ>0\lambda > 0λ>0 the life distribution HHH of (5.2) is NBU.

Milestones

  1. Theorem 3.1 (3.5). For any sequence 1=Pˉ0≥Pˉ1≥⋯≥01 = \bar P_0 \ge \bar P_1 \ge \cdots \ge 01=Pˉ0​≥Pˉ1​≥⋯≥0 and λ>0\lambda > 0λ>0: if PˉjPˉk≥Pˉj+k\bar P_j \bar P_k \ge \bar P_{j+k}Pˉj​Pˉk​≥Pˉj+k​ for all j,kj, kj,k, then HHH of (2.1) is NBU.
  2. First display of the proof of Theorem 5.3. If GGG is NBU, then Pˉj+k≤PˉjPˉk\bar P_{j+k} \le \bar P_j \bar P_kPˉj+k​≤Pˉj​Pˉk​ for each damage law FFF.
  3. Second display of the proof of Theorem 5.3. If submultiplicativity holds for every FFF, then Gˉ(s+jxk)≤Gˉ(s) Gˉ(jxk)\bar G(s + j x_k) \le \bar G(s)\,\bar G(j x_k)Gˉ(s+jxk​)≤Gˉ(s)Gˉ(jxk​) for s>0s > 0s>0, xk=s/kx_k = s/kxk​=s/k, j=0,1,…j = 0, 1, \dotsj=0,1,….

Further items (not milestones)

Theorem 5.1 (the life is exponential for every FFF iff GGG is exponential) and Theorem 5.2 (b), (c) (an IHR threshold makes Pˉk1/k\bar P_k^{1/k}Pˉk1/k​ non-increasing for every FFF; non-increasing Pˉk1/k\bar P_k^{1/k}Pˉk1/k​ for every FFF forces GGG to be IHRA) are posed as companion statements from the same section.

Significance

Theorem 5.3 is a characterization: the NBU class is exactly the class of threshold laws for which the cumulative-damage mechanism preserves the discrete NBU property under every damage law. Combined with Theorem 3.1 (3.5), it gives a physical derivation of NBU life distributions: an item with an NBU random strength, subject to Poisson shocks with arbitrary i.i.d. non-negative damage, has an NBU life. Theorems 5.1 and 5.2 place the exponential and the IHR/IHRA classes in the same framework, and the authors record that whether an IHRA threshold suffices in Theorem 5.2 (b) is left unresolved.

The results are proved in the paper. No machine-checked version is known to exist; this mission produces one. A complete development also supplies reusable infrastructure: convolution powers of laws on [0,∞)[0, \infty)[0,∞) as Mathlib measures, Poisson mixtures of a sequence, and the ageing classes of p. 631 as predicates on survival functions.

Difficulty

The equivalence couples a property of one function, Gˉ\bar GGˉ, to a family of inequalities indexed by every damage law, and the left side is about convolutions while the right side is pointwise. Two points make the statement harder than it looks. First, (5.1) as printed is P{X1+⋯+Xk≤Y}P\{X_1 + \cdots + X_k \le Y\}P{X1​+⋯+Xk​≤Y}, whereas the paper's proof works with EGˉ(X1+⋯+Xk)=P{X1+⋯+Xk<Y}E\bar G(X_1 + \cdots + X_k) = P\{X_1 + \cdots + X_k < Y\}EGˉ(X1​+⋯+Xk​)=P{X1​+⋯+Xk​<Y}; the two agree only when F(k)F^{(k)}F(k) and GGG have no common discontinuities, and they can disagree as soon as F(k)F^{(k)}F(k) has an atom where GGG has one, a case the left side of the equivalence includes. A formal proof cannot invoke that convention. Second, the second sentence of the theorem passes through a Poisson series (5.2) whose terms involve the whole sequence Pˉk\bar P_kPˉk​, so the NBU property of HHH is a statement about a power series in ttt, not about any single Pˉk\bar P_kPˉk​.

Formalization scope

  • A distribution is a probability measure on R\mathbb RR; "F(z)=0F(z) = 0F(z)=0 for z<0z < 0z<0" is mass 000 on (−∞,0)(-\infty, 0)(−∞,0), and "G(0)=0G(0) = 0G(0)=0" (Theorems 5.1, 5.2) is mass 000 on (−∞,0](-\infty, 0](−∞,0]. F(k)F^{(k)}F(k) is the kkk-fold Measure.conv power with F(0)=δ0F^{(0)} = \delta_0F(0)=δ0​, and F(k)(x)F^{(k)}(x)F(k)(x) is the mass of (−∞,x](-\infty, x](−∞,x].
  • Pˉk\bar P_kPˉk​ is (5.1) as printed, ∫F(k)(x) dG(x)\int F^{(k)}(x)\,dG(x)∫F(k)(x)dG(x) over R\mathbb RR; since GGG is carried by [0,∞)[0, \infty)[0,∞) this is ∫0∞\int_0^\infty∫0∞​ including an atom of GGG at 000, which Theorem 5.3 allows. The paper's convention that F(k)F^{(k)}F(k) and GGG have no common discontinuities is not added as a hypothesis: the statements hold for (5.1) without it. The integrand is monotone with values in [0,1][0,1][0,1], so the Bochner integral is a genuine expectation.
  • "For all FFF" sits inside the equivalence of Theorem 5.3 and ranges over every probability measure on [0,∞)[0, \infty)[0,∞). Submultiplicativity for one fixed FFF is a different, weaker statement.
  • NBU and IHR are cross-multiplied, agreeing with the paper's ratio form wherever denominators are positive (the paper restricts variables to avoid zero denominators). NBU is on x,t≥0x, t \ge 0x,t≥0 exactly as in definition (v). Powers [Gˉ(t)]1/t[\bar G(t)]^{1/t}[Gˉ(t)]1/t and Pˉk1/k\bar P_k^{1/k}Pˉk1/k​ are real powers. "Decreasing" means non-increasing.
  • HHH is the series (2.1) as a function on R\mathbb RR, equal to 111 on (−∞,0)(-\infty, 0)(−∞,0); λ>0\lambda > 0λ>0 as in (1.1). In Theorem 3.1 (3.5) the non-negativity Pˉk≥0\bar P_k \ge 0Pˉk​≥0 is stated explicitly.
  • In Theorem 5.1 "exponential" allows rate 000 for HHH (the damage law δ0\delta_0δ0​ gives Hˉ≡1\bar H \equiv 1Hˉ≡1) and requires a positive rate for GGG (G(0)=0G(0) = 0G(0)=0 rules out the degenerate case Gˉ=0\bar G = 0Gˉ=0 on (0,∞)(0,\infty)(0,∞)).
  • A trivializing formalization is ruled out: the threshold-law quantifier is not restricted to a class where both sides are automatic, the integral cannot collapse to a default value, and the NBU predicate is the paper's on [0,∞)[0, \infty)[0,∞), not on (0,∞)(0, \infty)(0,∞) or a vacuous domain.

Contributions welcome: proofs of the milestones, a library for Poisson mixtures and convolution powers of laws on [0,∞)[0,\infty)[0,∞), and the extra items of §5.

Selected references

  • J. D. Esary, A. W. Marshall and F. Proschan, Shock Models and Wear Processes, The Annals of Probability 1(4), 627–649, 1973. https://doi.org/10.1214/aop/1176996891
  • R. E. Barlow and F. Proschan, Mathematical Theory of Reliability, Wiley, 1965; reprinted SIAM Classics in Applied Mathematics, 1996. https://doi.org/10.1137/1.9781611971194
  • Z. W. Birnbaum, J. D. Esary and A. W. Marshall, A Stochastic Characterization of Wear-Out for Components and Systems, The Annals of Mathematical Statistics 37(4), 816–825, 1966. https://doi.org/10.1214/aoms/1177699362
5 thms1 active userReviewed
Markov ChainProbability·Captain: mikedeng1

Ergodic Theorems for Weakly Interacting Infinite Systems and the Voter Model: The Discrete-Time Proximity Process Is Dual to the Branching Process with InterferenceResearch Paper

Motivation

Many models in probability, statistical physics and the theory of stochastic networks are interacting particle systems: infinitely many sites, each in state 0 or 1, each updating at random according to the states of a few other sites. A basic question about such a system is ergodicity: does it have a unique stationary distribution, and does its law converge to it from every initial configuration? For infinite systems the question is hard, because the state space {0,1}I\{0,1\}^I{0,1}I is uncountable and standard Markov chain arguments do not apply.

Holley and Liggett (Ann. Probab. 3 (1975), 643–663) introduced a class of such systems, the proximity processes, which arise from a coupling of more general weakly interacting systems (the settings of Vasershtein (1969), Dobrushin (1971) and Chover (1974)), and showed that each proximity process is dual to a process of finite sets, the branching process with interference. Duality turns a question about an infinite-dimensional process into a question about a process whose states are finite sets. The paper uses it to give short proofs of ergodicity criteria and, in its §5, to study the invariant measures of the voter model. Duality of this kind became one of the main tools of the field; see Liggett, Interacting Particle Systems (Springer, 1985).

This mission formalizes the discrete-time half of the paper: the duality theorem (1.6) and its first applications, Corollary (3.1) and Corollary (4.1).

Setting

Let III be a countable set of sites and S={0,1}IS=\{0,1\}^IS={0,1}I the set of configurations η:I→{0,1}\eta:I\to\{0,1\}η:I→{0,1}, with the product σ\sigmaσ-algebra. For η∈S\eta\in Sη∈S, C(η)={i∈I:η(i)=1}C(\eta)=\{i\in I:\eta(i)=1\}C(η)={i∈I:η(i)=1} is the set of occupied sites.

The data of the model are, for each site iii, a sequence Ni,0=∅,Ni,1,Ni,2,…N_{i,0}=\emptyset,N_{i,1},N_{i,2},\dotsNi,0​=∅,Ni,1​,Ni,2​,… of finite subsets of III and a probability distribution fif_ifi​ on {0,1,2,… }\{0,1,2,\dots\}{0,1,2,…}.

The proximity process ηn\eta_nηn​ is the Markov chain on SSS whose transition function is the product measure

Q(η,⋅)=∏i∈Iνα(i,η),i,α(i,η)=∑k: Ni,k∩C(η)≠∅fi(k),Q(\eta,\cdot)=\prod_{i\in I}\nu_{\alpha(i,\eta),i},\qquad \alpha(i,\eta)=\sum_{k:\,N_{i,k}\cap C(\eta)\neq\emptyset}f_i(k),Q(η,⋅)=i∈I∏​να(i,η),i​,α(i,η)=k:Ni,k​∩C(η)=∅∑​fi​(k),

where νρ,i\nu_{\rho,i}νρ,i​ puts mass ρ\rhoρ on 111. In words: each site iii independently picks an index kkk with probability fi(k)f_i(k)fi​(k) and becomes 111 exactly when the set Ni,kN_{i,k}Ni,k​ contains an occupied site. Pη(ηn∈⋅)P_\eta(\eta_n\in\cdot)Pη​(ηn​∈⋅) denotes the law at time nnn started from η\etaη.

The branching process with interference (b.p.i.) AnA_nAn​ is the Markov chain on the collection T\mathcal TT of finite subsets of III with transition function

Q~(A,B)=∑′∏i∈Afi(ki),\tilde Q(A,B)=\sum{}'\prod_{i\in A}f_i(k_i),Q~​(A,B)=∑′i∈A∏​fi​(ki​),

the sum over all sequences (ki)i∈A(k_i)_{i\in A}(ki​)i∈A​ with ⋃i∈ANi,ki=B\bigcup_{i\in A}N_{i,k_i}=B⋃i∈A​Ni,ki​​=B: each particle at iii splits into particles on Ni,kN_{i,k}Ni,k​ with probability fi(k)f_i(k)fi​(k), and particles landing on the same site merge. PFP_FPF​ denotes probabilities for the chain started at FFF.

For F∈TF\in\mathcal TF∈T, B(F)={η∈S:η(i)=0 for all i∈F}B(F)=\{\eta\in S:\eta(i)=0\ \text{for all } i\in F\}B(F)={η∈S:η(i)=0 for all i∈F}.

Formalization targets

Goal: Theorem (1.6)

For all n≥0n\ge0n≥0, η∈S\eta\in Sη∈S and F∈TF\in\mathcal TF∈T,

Pη(ηn∈B(F))=PF(An∩C(η)=∅).P_\eta(\eta_n\in B(F))=P_F(A_n\cap C(\eta)=\emptyset).Pη​(ηn​∈B(F))=PF​(An​∩C(η)=∅).

Milestones

The proof is an induction on nnn whose steps are four displayed identities on p. 647:

  1. (2.3) the one-step law of the proximity process on B(F)B(F)B(F): Pη(η1∈B(F))=∑′∏i∈Ffi(ki)P_\eta(\eta_1\in B(F))=\sum'\prod_{i\in F}f_i(k_i)Pη​(η1​∈B(F))=∑′∏i∈F​fi​(ki​), the sum over sequences with Ni,ki∩C(η)=∅N_{i,k_i}\cap C(\eta)=\emptysetNi,ki​​∩C(η)=∅ for all i∈Fi\in Fi∈F;
  2. (2.4) the same expression for PF(A1∩C(η)=∅)P_F(A_1\cap C(\eta)=\emptyset)PF​(A1​∩C(η)=∅);
  3. (2.5) the first-step decomposition PF(An+1∩C(η)=∅)=∑(ki)P⋃i∈FNi,ki(An∩C(η)=∅)∏i∈Ffi(ki)P_F(A_{n+1}\cap C(\eta)=\emptyset)=\sum_{(k_i)}P_{\bigcup_{i\in F}N_{i,k_i}}(A_n\cap C(\eta)=\emptyset)\prod_{i\in F}f_i(k_i)PF​(An+1​∩C(η)=∅)=∑(ki​)​P⋃i∈F​Ni,ki​​​(An​∩C(η)=∅)∏i∈F​fi​(ki​);
  4. (2.8) the last-step decomposition Pη(ηn+1∈B(F))=∑(ki)Pη(ηn∈B(⋃i∈FNi,ki))∏i∈Ffi(ki)P_\eta(\eta_{n+1}\in B(F))=\sum_{(k_i)}P_\eta(\eta_n\in B(\bigcup_{i\in F}N_{i,k_i}))\prod_{i\in F}f_i(k_i)Pη​(ηn+1​∈B(F))=∑(ki​)​Pη​(ηn​∈B(⋃i∈F​Ni,ki​​))∏i∈F​fi​(ki​).

Applications

  • Corollary (3.1): if ∑k∣Ni,k∣fi(k)≤λ<1\sum_k|N_{i,k}|f_i(k)\le\lambda<1∑k​∣Ni,k​∣fi​(k)≤λ<1 for all iii, then
Pη(ηn∈B(F))≥1−λn∣F∣,P_\eta(\eta_n\in B(F))\ge1-\lambda^n|F|,Pη​(ηn​∈B(F))≥1−λn∣F∣,

and the point mass on the all-zero configuration is the only stationary distribution; along the way, (3.5): PF(An≠∅)≤EF∣An∣≤λn∣F∣P_F(A_n\neq\emptyset)\le E_F|A_n|\le\lambda^n|F|PF​(An​=∅)≤EF​∣An​∣≤λn∣F∣.

  • (4.5): for the §4 model on Z\mathbb ZZ, PF∪G(At≠∅)≤PF(At≠∅)+PG(At≠∅)−PF∩G(At≠∅)P_{F\cup G}(A_t\neq\emptyset)\le P_F(A_t\neq\emptyset)+P_G(A_t\neq\emptyset)-P_{F\cap G}(A_t\neq\emptyset)PF∪G​(At​=∅)≤PF​(At​=∅)+PG​(At​=∅)−PF∩G​(At​=∅).
  • Corollary (4.1): on I=ZI=\mathbb ZI=Z with Ni,1={i−1,i+1}N_{i,1}=\{i-1,i+1\}Ni,1​={i−1,i+1}, fi(0)=1−λf_i(0)=1-\lambdafi​(0)=1−λ, fi(1)=λf_i(1)=\lambdafi​(1)=λ, and h(λ)=2λ+2λ2−7λ3+5λ4−λ5h(\lambda)=2\lambda+2\lambda^2-7\lambda^3+5\lambda^4-\lambda^5h(λ)=2λ+2λ2−7λ3+5λ4−λ5,
Pη(ηt∈B(F))≥1−[h(λ)](t−1)/2∣F∣(t≥1).P_\eta(\eta_t\in B(F))\ge1-[h(\lambda)]^{(t-1)/2}|F|\qquad(t\ge1).Pη​(ηt​∈B(F))≥1−[h(λ)](t−1)/2∣F∣(t≥1).

Significance

The duality identity expresses every finite-dimensional "all vacant" probability of the proximity process, which determine its law, through a chain on finite sets. Consequences in the paper: a sufficient condition for convergence to the all-zero configuration (Corollary (3.1)), which in the nearest-neighbour example on Z\mathbb ZZ gives λ<12\lambda<\tfrac12λ<21​; the sharper range h(λ)<1h(\lambda)<1h(λ)<1, i.e. λ<0.6527\lambda<0.6527λ<0.6527, for that example (Corollary (4.1)), which the interference term makes possible; and, in continuous time, the analysis of the invariant measures of the voter model.

The results are classical and proved; a search of Prove2Me and Mathlib found no machine-checked proof of any of them. A formalization contributes a Lean model of an infinite-product Markov kernel on {0,1}I\{0,1\}^I{0,1}I with its nnn-step laws, a model of a set-valued branching chain, and the duality between them, which are reusable for other duality arguments (contact process, voter model, coalescing random walks).

Difficulty

The two sides of (1.7) are defined independently and live on different spaces. The left side is the time-nnn law of a chain on the uncountable space {0,1}I\{0,1\}^I{0,1}I, obtained by iterating an infinite product kernel; the right side comes from a transition matrix on the countable set of finite subsets of III, whose entries are themselves infinite sums over sequences indexed by a finite set. Neither side can be computed in closed form beyond n=1n=1n=1, so no direct evaluation proves the identity for general nnn.

Two technical points stand in the way of a naive argument. Evaluating a product kernel on a cylinder event and integrating it against a previous law requires the kernel to be measurable in η\etaη, a statement about cylinder sets of an infinite product that a hand proof passes over. And the natural recursion for nnn-step probabilities of the b.p.i. adds the new step at the end, while the comparison with the proximity process needs information about the first step; the two orders agree only by a Chapman–Kolmogorov argument over a countable state space with [0,∞][0,\infty][0,∞]-valued sums. Corollary (4.1) further needs quantitative estimates of survival probabilities for specific initial sets on Z\mathbb ZZ.

Formalization scope

All objects are in the namespace HolleyLiggett.Duality, in one definitions file.

  • {0,1}\{0,1\}{0,1} is Bool (true = 1), SSS is Config I := I → Bool with the product σ\sigmaσ-algebra, T\mathcal TT is Finset I.
  • Standing assumptions on every theorem: [Countable I], N : I → ℕ → Finset I (finite sets) with N i 0 = ∅, and f : I → PMF ℕ (probability distributions). A finite collection Ni,0,…,Ni,mN_{i,0},\dots,N_{i,m}Ni,0​,…,Ni,m​ is the case fi(k)=0f_i(k)=0fi​(k)=0 for k>mk>mk>m.
  • The kernel (1.1) is Measure.infinitePi of the two-point measures; the nnn-step law is proxLaw, iterating Measure.bind from the point mass at η\etaη. Measurability of the kernel is a true fact that solvers must prove; it is not assumed.
  • The b.p.i. nnn-step probabilities bpiProb follow the last-step recursion PF(An+1=B)=∑GPF(An=G)Q~(G,B)P_F(A_{n+1}=B)=\sum_GP_F(A_n=G)\tilde Q(G,B)PF​(An+1​=B)=∑G​PF​(An​=G)Q~​(G,B). All probabilities are in ℝ≥0∞, and each ∑′\sum'∑′ over sequences is an unconditional sum over ↥F → ℕ.
  • The page's "η(1)=1\eta(1)=1η(1)=1" in (1.2) is read as η(i)=1\eta(i)=1η(i)=1. Corollary (4.1) carries the added hypothesis t≥1t\ge1t≥1: at t=0t=0t=0 the printed bound fails when h(λ)>1h(\lambda)>1h(λ)>1, and the proof starts at t=1t=1t=1. Inequality (4.5) uses the §4 model on Z\mathbb ZZ, with the subtracted term moved across.
  • The goal relates the two chains defined by (1.1) and (1.4). A statement through the coupling Xi,nX_{i,n}Xi,n​ of the proof, about an arbitrary process with the right one-step behaviour, or for n=1n=1n=1 only, is not the theorem.

The continuous-time duality (Theorem (2.12)), Corollaries (3.9) and (4.15) and the voter model of §5 are out of scope. Contributions welcome: the measurability and probability facts for proxStep, Chapman–Kolmogorov for bpiProb, and proofs of the milestones.

Selected references

  • R. A. Holley and T. M. Liggett, Ergodic theorems for weakly interacting infinite systems and the voter model, Ann. Probab. 3(4) (1975), 643–663. https://doi.org/10.1214/aop/1176996306
  • T. M. Liggett, Interacting Particle Systems, Grundlehren der mathematischen Wissenschaften 276, Springer, 1985. https://doi.org/10.1007/978-1-4613-8542-4
  • R. L. Dobrushin, Markov processes with a large number of locally interacting components (Russian), Problems of Information Transmission 7(2) (1971), 70–87 (as cited by Holley and Liggett).
  • O. N. Stavskaya and I. I. Pyatetskii-Shapiro, On homogeneous nets of spontaneously active elements, Systems Theory Research 20 (1971), 75–88 (as cited by Holley and Liggett).
  • L. N. Vasershtein, Processes over denumerable products of spaces, describing large systems, Problems of Information Transmission 3 (1969), 47–52 (as cited by Holley and Liggett).
6 thms1 active userReviewed
Operations ResearchProbability·Captain: mikedeng1

The Relation between Customer and Time Averages in Queues: H = λG on Every Sample Path When 0 < λ < ∞, G < ∞ and Each f_n Vanishes Outside [t_n, t_n + s_n] with s_n/n → 0Research Paper

Motivation

Little's law L=λWL = \lambda WL=λW says that the long-run average number of customers in a system equals the arrival rate times the average time a customer spends there. It is one of the most used identities in queueing theory, and it holds on individual sample paths under weak conditions (Little 1961; Stidham 1974). Many quantities of interest are not head counts, however: the work in the system, the cost accumulated by customers in progress, the number of tokens a customer holds in some state. For these a more general relation is needed, between a time average HHH and a customer average GGG, of the form H=λGH = \lambda GH=λG.

Heyman and Stidham (Oper. Res. 28 (1980)) prove such a relation on each sample path, for an arbitrary real-valued function attached to each customer, under a support condition that is much weaker than the continuous-sojourn assumption of the L=λWL = \lambda WL=λW theorem. They then show by a counterexample that the support condition cannot simply be dropped, even when every customer function is an indicator.

Timeline.

  • 1961: Little proves L=λWL = \lambda WL=λW under stationarity assumptions (Little 1961).
  • 1971: Brumelle proves H=λGH = \lambda GH=λG under conditions (v.a), (v.b) on the tails of the fnf_nfn​ (J. Appl. Prob. 8, 508–520).
  • 1972: Stidham gives a new proof of L=λWL = \lambda WL=λW, including the lemma relating N(t)/tN(t)/tN(t)/t and tn/nt_n/ntn​/n (Stidham 1972).
  • 1974: Stidham proves L=λWL = \lambda WL=λW on each sample path, assuming each sojourn is one uninterrupted interval (Stidham 1974).
  • 1980: Heyman and Stidham prove H=λGH = \lambda GH=λG on every sample path under the support condition below, and give the counterexample. The paper notes that its Theorem 1 is weaker than the sample-path version of Brumelle's theorem, with hypotheses stated directly on the sample path.

Setting

Fix one sample path. Customers n=1,2,…n = 1, 2, \ldotsn=1,2,… arrive at epochs 0≤t1≤t2≤⋯0 \le t_1 \le t_2 \le \cdots0≤t1​≤t2​≤⋯; ties are allowed. The arrival count N(t)N(t)N(t) is the number of nnn with tn≤tt_n \le ttn​≤t, and the arrival rate is λ=lim⁡t→∞N(t)/t\lambda = \lim_{t\to\infty} N(t)/tλ=limt→∞​N(t)/t when the limit exists.

Customer nnn carries a real-valued function fnf_nfn​ on [0,∞)[0,\infty)[0,∞). Its total is gn=∫0∞fn(t) dtg_n = \int_0^\infty f_n(t)\,dtgn​=∫0∞​fn​(t)dt, and the rate is h(t)=∑n=1∞fn(t)h(t) = \sum_{n=1}^\infty f_n(t)h(t)=∑n=1∞​fn​(t). The customer average and time average are

G=lim⁡N→∞1N∑n=1Ngn,H=lim⁡T→∞1T∫0Th(t) dt.G = \lim_{N\to\infty} \frac1N \sum_{n=1}^N g_n, \qquad H = \lim_{T\to\infty} \frac1T \int_0^T h(t)\,dt .G=N→∞lim​N1​n=1∑N​gn​,H=T→∞lim​T1​∫0T​h(t)dt.

When fnf_nfn​ is the indicator of [tn,tn+Wn)[t_n, t_n + W_n)[tn​,tn​+Wn​), gn=Wng_n = W_ngn​=Wn​ is the sojourn time and h(t)h(t)h(t) is the number in system, so H=λGH = \lambda GH=λG is L=λWL = \lambda WL=λW.

The ASSUMPTION of the paper is that for each nnn there is sn∈[0,∞)s_n \in [0,\infty)sn​∈[0,∞) with

  1. (i) fn(t)=0f_n(t) = 0fn​(t)=0 for t∉[tn,tn+sn]t \notin [t_n, t_n + s_n]t∈/[tn​,tn​+sn​];
  2. (ii) sn/n→0s_n / n \to 0sn​/n→0.

For signed fnf_nfn​ write fn+=max⁡[0,fn]f_n^+ = \max[0, f_n]fn+​=max[0,fn​], fn−=max⁡[0,−fn]f_n^- = \max[0, -f_n]fn−​=max[0,−fn​], and let gn±g_n^\pmgn±​, h±h^\pmh±, G±G^\pmG±, H±H^\pmH± be the corresponding totals, rates and averages.

Formalization targets

Goal: Theorem 2 (p. 986)

Assume (i), (ii) and (v) ∫0∞∣fn(t)∣ dt<∞\int_0^\infty |f_n(t)|\,dt < \infty∫0∞​∣fn​(t)∣dt<∞ for every nnn. If λ\lambdaλ, G+G^+G+ and G−G^-G− exist with 0<λ<∞0 < \lambda < \infty0<λ<∞ and G±<∞G^\pm < \inftyG±<∞, then GGG exists, equals G+−G−G^+ - G^-G+−G−, HHH exists, and

H=λG.(1)H = \lambda G. \tag{1}H=λG.(1)

Milestones

  • (2): for 0<λ<∞0 < \lambda < \infty0<λ<∞, N(t)/t→λN(t)/t \to \lambdaN(t)/t→λ if and only if tn/n→λ−1t_n / n \to \lambda^{-1}tn​/n→λ−1.
  • (3): for fn≥0f_n \ge 0fn​≥0, V(T)≤∫0Th(t) dt≤U(T)V(T) \le \int_0^T h(t)\,dt \le U(T)V(T)≤∫0T​h(t)dt≤U(T), where U(T)U(T)U(T) sums gng_ngn​ over arrived customers and V(T)V(T)V(T) over customers with tn+sn≤Tt_n + s_n \le Ttn​+sn​≤T.
  • (4): λG=lim⁡t→∞U(t)/t\lambda G = \lim_{t\to\infty} U(t)/tλG=limt→∞​U(t)/t.
  • sn/tn→0s_n / t_n \to 0sn​/tn​→0, and lim⁡U(t)/t=lim⁡V(t)/t\lim U(t)/t = \lim V(t)/tlimU(t)/t=limV(t)/t.
  • Theorem 1 (p. 985): for fn≥0f_n \ge 0fn​≥0 under (i)–(iv), H=λGH = \lambda GH=λG.
  • The identity ∫0Th+−∫0Th−=∫0Th\int_0^T h^+ - \int_0^T h^- = \int_0^T h∫0T​h+−∫0T​h−=∫0T​h and (6): G=G+−G−G = G^+ - G^-G=G+−G−, H=H+−H−H = H^+ - H^-H=H+−H−.

Companion results

  • The §3 counterexample: tn=nt_n = ntn​=n, indicator fnf_nfn​ with gn=1g_n = 1gn​=1, so λ=G=1\lambda = G = 1λ=G=1, but H=log⁡2H = \log 2H=log2; its support span sn=ns_n = nsn​=n satisfies (i) and fails (ii).
  • Corollary 3 (p. 988): if λ(ω)=λ\lambda(\omega) = \lambdaλ(ω)=λ is constant, the ensemble averages satisfy ∫H dP=λ∫G dP\int H\,dP = \lambda \int G\,dP∫HdP=λ∫GdP.
  • G=W<∞G = W < \inftyG=W<∞ implies Wn/n→0W_n / n \to 0Wn​/n→0 (p. 986), the step by which Theorem 1 contains L=λWL = \lambda WL=λW.

Significance

The result. H=λGH = \lambda GH=λG converts between a time average, which is what a system designer measures, and a customer average, which is what a customer experiences. Applied to different fnf_nfn​ it yields L=λWL = \lambda WL=λW, the relation between average work in system and average customer work, and relations between time-stationary and embedded-chain probabilities; §2 of the paper derives the GI/M/c/K relation this way. Because the support condition (i)–(ii) allows a customer's contribution to be interrupted (leaving and re-entering the system), it covers preemptive priority queues and nodes of networks, which the continuous-sojourn L=λWL = \lambda WL=λW theorem does not. The counterexample marks the boundary: with interrupted sojourns, indicator functions and finite λ\lambdaλ, GGG alone do not suffice.

Formalizing it. The result is proved on paper, with two steps delegated to earlier work "by mimicking" Lemma 1 and Theorem 2 of Stidham 1974. The indicator special case L=λWL = \lambda WL=λW (with strictly increasing arrivals) is already proved on Prove2Me as queueing_general_littles_law (wenxinzhang). This mission asks for the general, signed, pathwise statement and its proof steps, a counterexample with an exactly computed time average log⁡2\log 2log2, and the ensemble corollary.

Difficulty

The obvious argument, exchanging the time integral of hhh with the sum over customers, gives ∫0Th=∑n∫0Tfn\int_0^T h = \sum_n \int_0^T f_n∫0T​h=∑n​∫0T​fn​, but a customer that has arrived by TTT may contribute only part of its total gng_ngn​ by TTT. The sandwich V(T)≤∫0Th≤U(T)V(T) \le \int_0^T h \le U(T)V(T)≤∫0T​h≤U(T) only bounds this loss; the hard step is showing that U(t)/tU(t)/tU(t)/t and V(t)/tV(t)/tV(t)/t have the same limit, which needs sn/tn→0s_n / t_n \to 0sn​/tn​→0 to compare VVV at time ttt with UUU at a slightly earlier time. Without (ii) this fails, as the counterexample shows: customers there remain "open" for a window proportional to their index.

For signed fnf_nfn​ the sandwich is not available directly, and the proof splits fnf_nfn​ into positive and negative parts; the exchange of sum and integral then needs the finiteness of the parts on every [0,T][0,T][0,T].

Formalization scope

All statements except Corollary 3 are about one fixed sample path; the paper's "with probability one" is the pathwise statement applied to almost every path, and Corollary 3 states this explicitly with a probability measure and almost-sure hypotheses.

Conventions:

  • Customers are indexed from 000 in Lean; index nnn is the paper's customer n+1n+1n+1, so sn/n→0s_n/n \to 0sn​/n→0 is written sn/(n+1)→0s_n/(n+1) \to 0sn​/(n+1)→0.
  • Arrival epochs are Monotone with t0≥0t_0 \ge 0t0​≥0; ties are allowed.
  • fn:R→Rf_n : \mathbb R \to \mathbb Rfn​:R→R, but only values on [0,∞)[0,\infty)[0,∞) enter: (i) is required only for t≥0t \ge 0t≥0, gng_ngn​ integrates over [0,∞)[0,\infty)[0,∞), and time averages integrate over [0,T][0,T][0,T].
  • (iv) and (v) are stated as integrability of fnf_nfn​ on [0,∞)[0,\infty)[0,∞); the page prints (iv) as "≤∞\le \infty≤∞", a misprint for "<∞< \infty<∞".
  • N(t)N(t)N(t) is the cardinality of {n:tn≤t}\{n : t_n \le t\}{n:tn​≤t}; hhh, UUU, VVV are infinite sums over all customers.
  • "HHH exists" includes integrability of hhh on every [0,T][0,T][0,T].
  • The page prints (6) as "G=G+−G+G = G^+ - G^+G=G+−G+"; the stated identity is G=G+−G−G = G^+ - G^-G=G+−G−, as the proof shows.

Added hypotheses: milestones (3), the h±h^\pmh± identity and (6) assume tn→∞t_n \to \inftytn​→∞, which the paper derives from (2); Corollary 3 assumes G(ω)G(\omega)G(ω) is integrable, which its definition of the ensemble average presupposes.

A formalization in which hhh sums only over arrived customers, or in which a non-integrable hhh or fnf_nfn​ has integral 000, would make the statements trivial or different; the definitions rule this out by summing over all customers and requiring integrability.

Needed infrastructure: Cesàro averages and counting functions of nondecreasing sequences, interchange of countable sums and integrals for locally finite families, and harmonic sums ∑m=⌈(k+1)/2⌉k1/m→log⁡2\sum_{m=\lceil (k+1)/2\rceil}^{k} 1/m \to \log 2∑m=⌈(k+1)/2⌉k​1/m→log2. The counting-function lemma (2) and the squeeze for UUU and VVV are reusable well beyond this mission. Proofs of any milestone are welcome independently.

Selected references

  • D. P. Heyman and S. Stidham, Jr., The relation between customer and time averages in queues, Operations Research 28(4):983–994, 1980. https://doi.org/10.1287/opre.28.4.983
  • J. D. C. Little, A proof for the queuing formula: L = λW, Operations Research 9(3):383–387, 1961. https://doi.org/10.1287/opre.9.3.383
  • S. Stidham, Jr., L = λW: a discounted analogue and a new proof, Operations Research 20(6):1115–1126, 1972. https://doi.org/10.1287/opre.20.6.1115
  • S. Stidham, Jr., A last word on L = λW, Operations Research 22(2):417–421, 1974. https://doi.org/10.1287/opre.22.2.417
  • S. L. Brumelle, On the relation between customer and time averages in queues, Journal of Applied Probability 8:508–520, 1971.
11 thms1 active userReviewed
Operations ResearchProbability·Captain: mikedeng1

Shock Models and Wear Processes I: Under Poisson Shocks, Cumulative Nonnegative I.I.D. Damage up to a Fixed Threshold Gives an IHRA Life DistributionResearch Paper

Motivation

Reliability theory classifies life distributions by how they age. The IHRA class (increasing hazard rate average) is the smallest class of life distributions that contains the exponential distributions and is closed under forming coherent systems and taking limits in distribution (Birnbaum, Esary and Marshall 1966). It is therefore the natural class for the lifetime of a system built from components that wear out. A distribution belongs to it when its survival function Fˉ\bar FFˉ satisfies: [Fˉ(t)]1/t[\bar F(t)]^{1/t}[Fˉ(t)]1/t is decreasing in t>0t > 0t>0.

Esary, Marshall and Proschan (Ann. Probability 1 (1973) 627–649) asked where such ageing comes from physically. Their answer is a shock model: a device receives shocks at the epochs of a Poisson process, each shock does random damage, and the device fails when the accumulated damage exceeds its capacity. The central result of their §4, formalized in this mission, is that this model always produces an IHRA life, whatever the damage distribution. In the authors' words, "the IHRA property has been obtained … as an implication of a natural physical model. The only hypothesis imposed upon FFF is that it be the distribution of a nonnegative random variable." The paper is a standard reference of reliability theory and the source of the cumulative damage model used across maintenance and insurance applications.

Setting

Shocks arrive according to a Poisson process with rate λ>0\lambda > 0λ>0. Write Pˉk\bar P_kPˉk​ for the probability that the device survives the first kkk shocks; then 1≥Pˉ0≥Pˉ1≥⋯≥01 \ge \bar P_0 \ge \bar P_1 \ge \dots \ge 01≥Pˉ0​≥Pˉ1​≥⋯≥0 (display (2.2)). Conditioning on the number of shocks in [0,t][0, t][0,t] gives the shock survival function (2.1):

Hˉ(t)=∑k=0∞Pˉk e−λt(λt)kk!,t≥0,\bar H(t) = \sum_{k=0}^{\infty} \bar P_k\, e^{-\lambda t}\frac{(\lambda t)^k}{k!}, \qquad t \ge 0,Hˉ(t)=k=0∑∞​Pˉk​e−λtk!(λt)k​,t≥0,

with Hˉ(t)=1\bar H(t) = 1Hˉ(t)=1 for t<0t < 0t<0. The weights K(k,t)=e−λt(λt)k/k!K(k,t) = e^{-\lambda t}(\lambda t)^k/k!K(k,t)=e−λt(λt)k/k! are the Poisson probabilities.

In the cumulative damage model the iiith shock causes a damage Xi≥0X_i \ge 0Xi​≥0, the damages are independent with common distribution function FFF (so F(z)=0F(z) = 0F(z)=0 for z<0z < 0z<0), and the device survives kkk shocks when X1+⋯+Xk≤xX_1 + \dots + X_k \le xX1​+⋯+Xk​≤x, for a fixed threshold xxx. Hence (4.1)

Pˉk=F(k)(x),k=0,1,…,\bar P_k = F^{(k)}(x), \qquad k = 0, 1, \dots,Pˉk​=F(k)(x),k=0,1,…,

where F(k)F^{(k)}F(k) is the kkk-fold convolution of FFF and F(0)F^{(0)}F(0) is degenerate at 000. A distribution with survival function Fˉ\bar FFˉ is IHRA if [Fˉ(t)]1/t[\bar F(t)]^{1/t}[Fˉ(t)]1/t is decreasing in t>0t > 0t>0; throughout, "decreasing" means non-increasing.

Formalization targets

Goal: Corollary 4.2, display (4.7)

For every distribution FFF on [0,∞)[0,\infty)[0,∞), every λ>0\lambda > 0λ>0 and every threshold xxx,

Hˉ(t)=∑k=0∞e−λt(λt)kk!F(k)(x)is IHRA.\bar H(t) = \sum_{k=0}^\infty e^{-\lambda t}\frac{(\lambda t)^k}{k!} F^{(k)}(x) \quad \text{is IHRA.}Hˉ(t)=k=0∑∞​e−λtk!(λt)k​F(k)(x)is IHRA.

This is the first of the three statements of Corollary 4.2; the second ((4.7a), independent damages FiF_iFi​ that worsen with iii) is an extra item of this mission, and the third ((4.7b), dependent damages satisfying (4.3)–(4.5)) belongs to mission III of this series.

Milestones

  1. (2.6): Hˉ(t)≥Hˉ(0)e−λt\bar H(t) \ge \bar H(0)e^{-\lambda t}Hˉ(t)≥Hˉ(0)e−λt for t≥0t \ge 0t≥0.
  2. Theorem 3.1 (3.4) through the three claims of its proof (p. 633): if 1=Pˉ0≥Pˉ1≥…1 = \bar P_0 \ge \bar P_1 \ge \dots1=Pˉ0​≥Pˉ1​≥… and Pˉk1/k\bar P_k^{1/k}Pˉk1/k​ is decreasing in k≥1k \ge 1k≥1, then Pˉk−ζk\bar P_k - \zeta^kPˉk​−ζk (0≤ζ≤10 \le \zeta \le 10≤ζ≤1) has at most one sign change, from +++ to −-−; this property passes to Hˉ(t)−e−(1−ζ)λt\bar H(t) - e^{-(1-\zeta)\lambda t}Hˉ(t)−e−(1−ζ)λt on t≥0t \ge 0t≥0; hence Hˉ(t)−e−θt\bar H(t) - e^{-\theta t}Hˉ(t)−e−θt has at most one sign change for every θ>0\theta > 0θ>0; and HHH is IHRA.
  3. Lemma 4.1: for FFF on [0,∞)[0,\infty)[0,∞), [F(k)(x)]1/k[F^{(k)}(x)]^{1/k}[F(k)(x)]1/k is decreasing in k=1,2,…k = 1, 2, \dotsk=1,2,….

Further items

Lemma 4.1a and Corollary 4.2 (4.7a); Theorem 4.4 ([F(k)(x)]1/k[F^{(k)}(x)]^{1/k}[F(k)(x)]1/k is constant in kkk iff FFF has no mass in (0,x](0,x](0,x]); Corollary 4.5 ((4.7) is exponential iff FFF has no mass in (0,x](0,x](0,x]); Corollary 4.11 ([P{N(x)≥k}]1/k[P\{N(x) \ge k\}]^{1/k}[P{N(x)≥k}]1/k is decreasing for the count N(x)N(x)N(x) of an ordinary renewal process).

Significance

The goal turns a modelling assumption into a theorem: anyone who models failure as accumulated nonnegative damage under Poisson shocks obtains, without further checks, every consequence of IHRA proved in reliability theory, including the closure of the class under the formation of coherent systems. Theorem 3.1 (3.4) is reusable on its own: it reduces the IHRA property of any Poisson mixture to a discrete condition on the Pˉk\bar P_kPˉk​, and the same scheme drives the first passage result for wear processes (Theorem 4.10, mission III). Lemma 4.1, applied to renewal processes, gives Corollary 4.11, a statement about renewal counts with no shock model in sight.

All results are proved in the paper; none has a machine-checked proof that we know of. The Mathlib library has the Poisson distribution and the convolution of measures, but no reliability classes, no total positivity and no variation diminishing property. The mission's output is a formal proof of the paper's chain of results and, along the way, reusable statements about Poisson mixtures and convolution powers.

Difficulty

Two steps resist a direct argument. The first is the transfer from sequences to functions: knowing that Pˉk−ζk\bar P_k - \zeta^kPˉk​−ζk changes sign at most once says nothing pointwise about the series ∑k(Pˉk−ζk)K(k,t)\sum_k(\bar P_k - \zeta^k)K(k,t)∑k​(Pˉk​−ζk)K(k,t). The paper invokes the variation diminishing property of the totally positive Poisson kernel (Karlin, Total Positivity, 1968), which is not in Mathlib. The second is Lemma 4.1, an inequality between convolution powers of an arbitrary law: no density, moments or continuity may be assumed, atoms at 000 and at xxx are allowed, so any argument through densities or Laplace transforms loses generality. A further subtlety is the passage from "one sign change of Hˉ(t)−e−θt\bar H(t) - e^{-\theta t}Hˉ(t)−e−θt for every θ\thetaθ" to the monotonicity of [Hˉ(t)]1/t[\bar H(t)]^{1/t}[Hˉ(t)]1/t, which needs care where Hˉ\bar HHˉ vanishes.

Formalization scope

  • A distribution FFF with F(z)=0F(z) = 0F(z)=0 for z<0z < 0z<0 is a probability measure μ\muμ on R\mathbb RR with μ(−∞,0)=0\mu(-\infty,0) = 0μ(−∞,0)=0; nothing else is assumed. F(k)F^{(k)}F(k) is the kkk-fold additive convolution of μ\muμ (Mathlib's Measure.conv) starting from the point mass at 000, and F(k)(x)F^{(k)}(x)F(k)(x) is the real number F(k)(−∞,x]F^{(k)}(-\infty,x]F(k)(−∞,x].
  • Hˉ\bar HHˉ is the series (2.1) as a function on R\mathbb RR, equal to 111 on t<0t < 0t<0. It is never defined through a constructed random failure time, and IHRA is never encoded through a condition on the Pˉk\bar P_kPˉk​: the goal concludes that [Hˉ(t)]1/t[\bar H(t)]^{1/t}[Hˉ(t)]1/t is decreasing on t>0t > 0t>0 for the series itself. This rules out the trivializing reading in which the goal unfolds to Lemma 4.1.
  • Powers [Hˉ(t)]1/t[\bar H(t)]^{1/t}[Hˉ(t)]1/t and Pˉk1/k\bar P_k^{1/k}Pˉk1/k​ are real powers with exponent 1/t1/t1/t, 1/k1/k1/k in R\mathbb RR.
  • Hypotheses the paper leaves implicit and the Lean statements make explicit: λ>0\lambda > 0λ>0; Pˉk≥0\bar P_k \ge 0Pˉk​≥0 (the Pˉk\bar P_kPˉk​ are probabilities); in Corollary 4.5, x≥0x \ge 0x≥0 (the threshold is a capacity), and "exponential" includes the degenerate rate 000.
  • Sign change statements about Hˉ\bar HHˉ hold on t≥0t \ge 0t≥0, where the series defines it.
  • The threshold xxx in the goal ranges over all reals, as printed.
  • In Corollary 4.11 the renewal process is given by independent measurable interarrival times with common law μ\muμ, and N(x)=#{k≥1:X1+⋯+Xk≤x}N(x) = \#\{k \ge 1 : X_1 + \dots + X_k \le x\}N(x)=#{k≥1:X1​+⋯+Xk​≤x} takes values in {0,1,…,∞}\{0, 1, \dots, \infty\}{0,1,…,∞}.

A complete development needs: the variation diminishing property of the Poisson kernel (reusable for every Poisson mixture), monotonicity of [F(k)(x)]1/k[F^{(k)}(x)]^{1/k}[F(k)(x)]1/k via convolution integrals, and elementary facts about real powers and sign changes. Proofs of any milestone or extra item, and general total positivity lemmas, are welcome.

Selected references

  • J. D. Esary, A. W. Marshall and F. Proschan, Shock Models and Wear Processes, The Annals of Probability 1(4) (1973) 627–649. https://doi.org/10.1214/aop/1176996891
  • Z. W. Birnbaum, J. D. Esary and A. W. Marshall, A Stochastic Characterization of Wear-Out for Components and Systems, The Annals of Mathematical Statistics 37 (1966) 816–825. https://doi.org/10.1214/aoms/1177699362
  • S. Karlin, Total Positivity, Vol. I, Stanford University Press, 1968.
  • R. E. Barlow and F. Proschan, Mathematical Theory of Reliability, Wiley, 1965. https://doi.org/10.1137/1.9781611971194
8 thms1 active userReviewed
Operations ResearchProbability·Captain: mikedeng1

Reflected Brownian Motion on an Orthant: The Skorokhod Construction Yields an Adapted, Almost Surely Unique, Time-Homogeneous Markov ProcessResearch Paper

Motivation

Reflected Brownian motion on the orthant is the diffusion that arises as the heavy-traffic limit of open networks of queues. In a KKK-station network the scaled queue-length vector lives in the nonnegative orthant R+K\mathbb R^K_+R+K​; in the interior it moves like a Brownian motion, and when a station empties the process is pushed back into the orthant in a direction determined by the routing of customers between stations. Harrison (1978) obtained such a limit for two queues in tandem, and Reiman (Open Queueing Networks in Heavy Traffic, Math. Oper. Res. 9 (1984)) showed that general KKK-station open networks lead exactly to the class of processes studied here.

Classical constructions of reflected diffusions (Stroock and Varadhan 1971, Watanabe 1971) require a smooth boundary and a reflection direction varying continuously on it. The orthant has corners and the reflection direction jumps between faces, so those results do not apply. Harrison and Reiman (Reflected Brownian Motion on an Orthant, Ann. Probab. 9 (1981)) construct the process pathwise, following Skorokhod's one-dimensional approach, and derive its Markov property from the construction. The process has since become the standard object in the diffusion approximation of queueing networks.

Setting

Fix a positive integer KKK. Vectors in RK\mathbb R^KRK are row vectors, indexed by j=1,…,Kj=1,\dots,Kj=1,…,K. Let AAA be a K×KK\times KK×K covariance matrix (symmetric, nonnegative definite), b∈RKb\in\mathbb R^Kb∈RK a drift vector, and Q=(qij)Q=(q_{ij})Q=(qij​) a nonnegative K×KK\times KK×K matrix with zeros on the diagonal and spectral radius strictly less than one. S=R+KS=\mathbb R^K_+S=R+K​ is the nonnegative orthant.

Let CCC be the space of continuous paths x:[0,∞)→RKx:[0,\infty)\to\mathbb R^Kx:[0,∞)→RK with the topology of uniform convergence on compact intervals, and CSC_SCS​ the paths with x(0)∈Sx(0)\in Sx(0)∈S. For x∈CSx\in C_Sx∈CS​, the Skorokhod problem asks for y,z∈Cy,z\in Cy,z∈C with, for every jjj,

zj(t)=xj(t)+yj(t)−∑i=1Kqij yi(t),zj(t)≥0,t≥0,(5–6)z_j(t)=x_j(t)+y_j(t)-\sum_{i=1}^K q_{ij}\,y_i(t),\qquad z_j(t)\ge 0,\qquad t\ge0, \tag{5–6}zj​(t)=xj​(t)+yj​(t)−i=1∑K​qij​yi​(t),zj​(t)≥0,t≥0,(5–6)

yjy_jyj​ nondecreasing with yj(0)=0y_j(0)=0yj​(0)=0 (7), and yjy_jyj​ increasing only at times ttt where zj(t)=0z_j(t)=0zj​(t)=0 (8). In matrix form, z=x+y(I−Q)z=x+y(I-Q)z=x+y(I−Q). Theorem 1 of the paper shows there is exactly one such pair, written y=ψ(x)y=\psi(x)y=ψ(x), z=ϕ(x)z=\phi(x)z=ϕ(x).

The process is obtained by feeding a Brownian path into this map. On a probability space (Ω,F,P)(\Omega,\mathcal F,P)(Ω,F,P) let XXX be a KKK-dimensional Brownian motion with covariance matrix AAA, drift bbb and X(0)∈SX(0)\in SX(0)∈S almost surely, with X(0)X(0)X(0) independent of the increments of XXX. Let Ft=F(X(s);0≤s≤t)\mathcal F_t=\mathcal F(X(s);0\le s\le t)Ft​=F(X(s);0≤s≤t). Set Y=ψ(X)Y=\psi(X)Y=ψ(X) and Z=ϕ(X)Z=\phi(X)Z=ϕ(X) where X∈CSX\in C_SX∈CS​, and Y=Z=0Y=Z=0Y=Z=0 on the exceptional null set. ZZZ is reflected Brownian motion on SSS with reflection matrix I−QI-QI−Q.

Formalization targets

Goal: Corollary 1

There is a family (κt)(\kappa_t)(κt​) of Markov transition kernels on RK\mathbb R^KRK, depending only on (Q,A,b)(Q,A,b)(Q,A,b), such that for every such XXX, YYY, ZZZ:

(a)Y(t), Z(t) are Ft-measurable, t≥0;\text{(a)}\quad Y(t),\ Z(t)\ \text{are } \mathcal F_t\text{-measurable},\ t\ge0;(a)Y(t), Z(t) are Ft​-measurable, t≥0; (b)(Y,Z) satisfies (1)–(4) a.s., and any pair satisfying (1)–(4) a.s. equals (Y,Z) a.s.;\text{(b)}\quad (Y,Z)\ \text{satisfies (1)–(4) a.s., and any pair satisfying (1)–(4) a.s. equals } (Y,Z) \text{ a.s.};(b)(Y,Z) satisfies (1)–(4) a.s., and any pair satisfying (1)–(4) a.s. equals (Y,Z) a.s.; (c)P[Z(s+t)∈B∣Fs]=κt(Z(s),B) a.s.,s,t≥0, B Borel.\text{(c)}\quad P\big[Z(s+t)\in B \mid \mathcal F_s\big]=\kappa_t\big(Z(s),B\big)\ \text{a.s.},\qquad s,t\ge0,\ B \text{ Borel}.(c)P[Z(s+t)∈B∣Fs​]=κt​(Z(s),B) a.s.,s,t≥0, B Borel.

Here (1)–(4) are (5)–(8) for the paths of XXX, YYY, ZZZ. The kernel is chosen before the probability space and the initial law, which is what "stationary transition probabilities" means.

Milestones

The milestones follow the proof of Theorem 1 on pp. 304–305:

  1. a positive diagonal Λ\LambdaΛ with ∥Λ−1QΛ∥<1\|\Lambda^{-1}Q\Lambda\|<1∥Λ−1QΛ∥<1 (Veinott scaling);
  2. (5)–(8) are invariant under (Q,x,y,z)↦(Λ−1QΛ,xΛ,yΛ,zΛ)(Q,x,y,z)\mapsto(\Lambda^{-1}Q\Lambda,x\Lambda,y\Lambda,z\Lambda)(Q,x,y,z)↦(Λ−1QΛ,xΛ,yΛ,zΛ);
  3. the key observation that (5)–(8) are equivalent to y∈C0y\in C_0y∈C0​, the fixed-point equation y=π(y)y=\pi(y)y=π(y) with π(y)(t)=sup⁡0≤s≤t[y(s)Q−x(s)]+\pi(y)(t)=\sup_{0\le s\le t}[y(s)Q-x(s)]^+π(y)(t)=sup0≤s≤t​[y(s)Q−x(s)]+, and z=x+y(I−Q)z=x+y(I-Q)z=x+y(I−Q);
  4. the contraction ∥π(y)−π(y′)∥≤α∥y−y′∥\|\pi(y)-\pi(y')\|\le\alpha\|y-y'\|∥π(y)−π(y′)∥≤α∥y−y′∥ on [0,T][0,T][0,T];
  5. convergence of the Picard iterates yn+1=π(yn)y^{n+1}=\pi(y^n)yn+1=π(yn), y0≡0y^0\equiv0y0≡0;
  6. the Lipschitz bound ∥ψ(x)−ψ(x′)∥≤∥x−x′∥/(1−α)\|\psi(x)-\psi(x')\|\le\|x-x'\|/(1-\alpha)∥ψ(x)−ψ(x′)∥≤∥x−x′∥/(1−α);
  7. Theorem 1 itself: existence and uniqueness, non-anticipation (9), continuity (10);
  8. the regeneration property (11): with x∗(t)=z(T)+x(T+t)−x(T)x^*(t)=z(T)+x(T+t)-x(T)x∗(t)=z(T)+x(T+t)−x(T), y∗(t)=y(T+t)−y(T)y^*(t)=y(T+t)-y(T)y∗(t)=y(T+t)−y(T), z∗(t)=z(T+t)z^*(t)=z(T+t)z∗(t)=z(T+t), one has y∗=ψ(x∗)y^*=\psi(x^*)y∗=ψ(x∗), z∗=ϕ(x∗)z^*=\phi(x^*)z∗=ϕ(x∗).

Milestone 7 is the platform theorem Reiman84.QueueLength.lemma_1, posed as an open target by an earlier mission (Reiman 1984 cites it as its Lemma 1). It is referenced here and not posed again.

Significance

Corollary 1 is what makes ZZZ a usable stochastic process: adaptedness and almost-sure uniqueness say ZZZ is determined by the driving Brownian motion in a non-anticipating way, and the Markov property with time-homogeneous kernels is the starting point for the change-of-variable formula of §3, for generators, for stationary distributions, and for the heavy-traffic limit theorems in which ZZZ appears as the limit. The pathwise map ϕ\phiϕ and its Lipschitz continuity are reused throughout queueing theory: the continuous-mapping argument for heavy-traffic limits rests on exactly the continuity (10) established here.

The results are classical, with complete published proofs. None of them has a machine-checked proof. Mathlib has the measure-theoretic layer (kernels, conditional expectation, independence) but no reflection maps, no construction of multidimensional Brownian motion with drift, and no Markov-process theory in continuous time. This mission provides a formal statement of the pathwise reflection theory and its probabilistic consequence on which such a development can build.

Difficulty

The pathwise part is a contraction argument, but the contraction is not in the original norm: the map y↦yQy\mapsto yQy↦yQ need not be a contraction for any standard norm when only the spectral radius of QQQ is below one. The proof first changes coordinates by a positive diagonal matrix, and the right norm must be matched to the row-vector convention. The fixed-point characterization also has to be shown equivalent to the complementarity condition (8), which is where the zero diagonal of QQQ enters.

For Corollary 1, the difficulty is measure-theoretic. Adaptedness requires measurability of a path functional with respect to the uncompleted natural filtration, which uses (9) and (10) and the fact that continuous paths are determined by countably many coordinates. The Markov property requires the regeneration identity (11) together with independence of the post-sss increments of XXX from Fs\mathcal F_sFs​, and a kernel that is jointly measurable and independent of the initial law. Conditioning on Fs\mathcal F_sFs​ alone, without identifying the future as a fixed functional of Z(s)Z(s)Z(s) and an independent Brownian motion, does not give a kernel that is the same for all sss.

Formalization scope

  • RK\mathbb R^KRK is Fin K → ℝ (paper index jjj = Lean index j−1j-1j−1), with K≥1K\ge1K≥1; row vector times matrix is Matrix.vecMul. Paths are functions ℝ → Fin K → ℝ; only times t≥0t\ge0t≥0 are constrained.
  • Spectral radius <1<1<1 is rendered as Qm→0Q^m\to0Qm→0, the rendering of Reiman84.QueueLength.lemma_1. AAA is PosSemidef.
  • (5)–(8) are the published Reiman84.QueueLength.IsReflectionPair; (8) reads "yjy_jyj​ is constant on every interval [s,t]⊆[0,∞)[s,t]\subseteq[0,\infty)[s,t]⊆[0,∞) on which zj>0z_j>0zj​>0". CSC_SCS​ is IsCPlus; uniform convergence on compacts is UocTendsto; Brownian motion from 000 with drift and covariance is IsDriftedBM.
  • The norm on C[0,T]C[0,T]C[0,T] is sup⁡0≤t≤T∥y(t)∥∞\sup_{0\le t\le T}\|y(t)\|_\inftysup0≤t≤T​∥y(t)∥∞​. The suprema in π\piπ and in the norm are real suprema, honest only for continuous paths, and every statement using them assumes continuity.
  • The paper's ∥P∥\|P\|∥P∥ is printed as the maximal row sum. Under the row-vector convention the contraction, Picard and Lipschitz steps need the maximal column sum; with the printed reading the contraction inequality is false (a nilpotent 3×33\times33×3 counterexample is recorded in those items). Veinott scaling is stated as printed; applying it to Q⊤Q^\topQ⊤ gives the column version.
  • The Brownian motion is X=X0+ξX=X_0+\xiX=X0​+ξ with ξ\xiξ a drifted Brownian motion from 000 and X0≥0X_0\ge0X0​≥0 a.s. independent of the whole process ξ\xiξ. Ft\mathcal F_tFt​ is the uncompleted σ-algebra generated by X(s)X(s)X(s), 0≤s≤t0\le s\le t0≤s≤t.
  • YYY and ZZZ enter Corollary 1 as hypotheses: a solution of (5)–(8) on {X∈CS}\{X\in C_S\}{X∈CS​}, zero elsewhere. That such processes exist is the existence part of Theorem 1, the referenced open item.
  • The kernel family is quantified before the probability space, so it cannot depend on sss, on Ω\OmegaΩ or on the law of X(0)X(0)X(0). A version of (c) with a kernel depending on these, with the completed or full σ-algebra in place of Fs\mathcal F_sFs​, or with one-dimensional marginals only, is a different and weaker statement and is ruled out.

Out of scope: Theorem 2 (the change-of-variable formula, which needs stochastic integration), the necessity of spectral radius <1<1<1, and §§3–4. Useful contributions beyond the milestones include a construction of multidimensional Brownian motion with drift in Mathlib, the Skorokhod map as a function on CSC_SCS​, and general lemmas on measurability of continuous path functionals.

Selected references

  • J. M. Harrison and M. I. Reiman, Reflected Brownian Motion on an Orthant, Annals of Probability 9(2), 302–308, 1981. https://doi.org/10.1214/aop/1176994471
  • M. I. Reiman, Open Queueing Networks in Heavy Traffic, Mathematics of Operations Research 9(3), 441–458, 1984. https://doi.org/10.1287/moor.9.3.441
  • A. F. Veinott, Jr., Discrete Dynamic Programming with Sensitive Discount Optimality Criteria, Annals of Mathematical Statistics 40(5), 1635–1660, 1969. https://doi.org/10.1214/aoms/1177697379
  • A. V. Skorokhod, Stochastic Equations for Diffusion Processes in a Bounded Region, Theory of Probability and Its Applications 6(3), 264–274, 1961. https://doi.org/10.1137/1106035
  • D. W. Stroock and S. R. S. Varadhan, Diffusion Processes with Boundary Conditions, Communications on Pure and Applied Mathematics 24, 147–225, 1971. https://doi.org/10.1002/cpa.3160240206
12 thms1 active userReviewed
Operations ResearchOptimizationProbability·Captain: mikedeng1

Mean-Variance Hedging in Continuous Time: The Feedback Futures Strategy Φ(G*) Minimizes the Expected Squared Deviation of Terminal Wealth from Any Target LevelResearch Paper

Motivation

A firm that will receive or deliver a quantity of a commodity, currency or security at a future date carries price risk until that date. When the asset itself cannot be traded in the meantime, the standard instrument for reducing that risk is a futures contract on a correlated asset: the firm takes a position in futures and adjusts it over time, and the gains or losses of the futures position offset part of the movement of its commitment. Choosing that position is the hedging problem. The classical answer, the minimum-variance hedge ratio, is a static one-period rule. Duffie and Richardson (Ann. Appl. Probab. 1991) solved the dynamic version in continuous time with a quadratic criterion: minimize the expected squared deviation of terminal wealth from a target. Their explicit feedback solution became a reference point for the later literature on mean-variance hedging in incomplete markets (Schweizer, Gouriéroux–Laurent–Pham, and others), where the same quadratic criterion is studied under general semimartingale prices.

Setting

Fix a horizon T>0T>0T>0 and a probability space (Ω,F,P)(\Omega,\mathcal F,P)(Ω,F,P) carrying a two-dimensional standard Brownian motion (B,ε)(B,\varepsilon)(B,ε) with its filtration F\mathbb FF. Let μ,σ,m,v,ρ\mu,\sigma,m,v,\rhoμ,σ,m,v,ρ be bounded measurable functions on [0,T][0,T][0,T], with ∣v∣|v|∣v∣ bounded away from zero and ρt∈[−1,1]\rho_t\in[-1,1]ρt​∈[−1,1]. The Brownian motion ξt=∫0tρs dBs+∫0t1−ρs2 dεs\xi_t=\int_0^t\rho_s\,dB_s+\int_0^t\sqrt{1-\rho_s^2}\,d\varepsilon_sξt​=∫0t​ρs​dBs​+∫0t​1−ρs2​​dεs​ has instantaneous correlation ρ\rhoρ with BBB. The committed asset SSS and the futures price FFF follow

dSt=μtSt dt+σtSt dBt,dFt=mtFt dt+vtFt dξt,S0,F0>0.dS_t=\mu_tS_t\,dt+\sigma_tS_t\,dB_t,\qquad dF_t=m_tF_t\,dt+v_tF_t\,d\xi_t,\qquad S_0,F_0>0.dSt​=μt​St​dt+σt​St​dBt​,dFt​=mt​Ft​dt+vt​Ft​dξt​,S0​,F0​>0.

The hedger is committed to kkk units of SSS at time TTT. A trading strategy is a progressively measurable process θ\thetaθ (the futures position) with E∫0Tθt2Ft2 dt<∞E\int_0^T\theta_t^2F_t^2\,dt<\inftyE∫0T​θt2​Ft2​dt<∞; Θ\ThetaΘ denotes the set of them. Its futures gain is the stochastic integral G(θ)t=∫0tθs dFsG(\theta)_t=\int_0^t\theta_s\,dF_sG(θ)t​=∫0t​θs​dFs​, and the terminal wealth is W(θ)=kST+G(θ)TW(\theta)=kS_T+G(\theta)_TW(θ)=kST​+G(θ)T​. Given a target level L∈RL\in\mathbb RL∈R, problem (3) is

min⁡θ∈ΘE[(W(θ)−L)2].\min_{\theta\in\Theta}E\big[(W(\theta)-L)^2\big].θ∈Θmin​E[(W(θ)−L)2].

With γt=mtσtρt/vt−μt\gamma_t=m_t\sigma_t\rho_t/v_t-\mu_tγt​=mt​σt​ρt​/vt​−μt​, the tracking process is Zt=kexp⁡(−∫tTγs ds)StZ_t=k\exp(-\int_t^T\gamma_s\,ds)S_tZt​=kexp(−∫tT​γs​ds)St​, so that ZT=kSTZ_T=kS_TZT​=kST​, and the feedback map is

Φ(Gt∗)=1Ft[mtvt2(L−Zt−Gt∗)−σtρtvtZt],\Phi(G^*_t)=\frac1{F_t}\Big[\frac{m_t}{v_t^2}(L-Z_t-G^*_t)-\frac{\sigma_t\rho_t}{v_t}Z_t\Big],Φ(Gt∗​)=Ft​1​[vt2​mt​​(L−Zt​−Gt∗​)−vt​σt​ρt​​Zt​],

where G∗G^*G∗ solves dGt∗=Φ(Gt∗) dFtdG^*_t=\Phi(G^*_t)\,dF_tdGt∗​=Φ(Gt∗​)dFt​, G0∗=0G^*_0=0G0∗​=0. The strategy φ=Φ(G∗)\varphi=\Phi(G^*)φ=Φ(G∗) depends only on the gains realized so far and the current price StS_tSt​.

Formalization targets

Goal: Proposition 1

φt=Φ(Gt∗)  solves  min⁡θ∈ΘE[(kST+G(θ)T−L)2]\varphi_t=\Phi(G^*_t)\ \text{ solves }\ \min_{\theta\in\Theta}E\big[(kS_T+G(\theta)_T-L)^2\big]φt​=Φ(Gt∗​)  solves  θ∈Θmin​E[(kST​+G(θ)T​−L)2]

for every commitment kkk, every target LLL and every solution G∗G^*G∗ of (10). No constant is hard-coded: the statement is the paper's for arbitrary coefficients satisfying the standing hypotheses.

Milestones

  1. Lemma 1: φ∈Θ\varphi\in\Thetaφ∈Θ is optimal iff E[(L−kST−G(φ)T) G(θ)T]=0E[(L-kS_T-G(\varphi)_T)\,G(\theta)_T]=0E[(L−kST​−G(φ)T​)G(θ)T​]=0 for every θ∈Θ\theta\in\Thetaθ∈Θ.
  2. Existence (§3.3): equation (10) has a solution with Gt∗∈L2(P)G^*_t\in L^2(P)Gt∗​∈L2(P).
  3. Itô dynamics of ZZZ: dZt=(γt+μt)Zt dt+σtZt dBtdZ_t=(\gamma_t+\mu_t)Z_t\,dt+\sigma_tZ_t\,dB_tdZt​=(γt​+μt​)Zt​dt+σt​Zt​dBt​.
  4. Moment equations for E(ZtGt)E(Z_tG_t)E(Zt​Gt​), E(Gt∗Gt)E(G^*_tG_t)E(Gt∗​Gt​) and E(Gt)E(G_t)E(Gt​), with G=G(θ)G=G(\theta)G=G(θ).
  5. Lemma 2: Ht=E[(L−Zt−Gt∗)G(θ)t]H_t=E[(L-Z_t-G^*_t)G(\theta)_t]Ht​=E[(L−Zt​−Gt∗​)G(θ)t​] satisfies H˙t=−(mt2/vt2)Ht\dot H_t=-(m_t^2/v_t^2)H_tH˙t​=−(mt2​/vt2​)Ht​.
  6. The solution of (13): Ht=H0exp⁡(−∫0tms2/vs2 ds)H_t=H_0\exp(-\int_0^tm_s^2/v_s^2\,ds)Ht​=H0​exp(−∫0t​ms2​/vs2​ds).

Two further items, not milestones, formalize §4: Lemma 3 (a solution of (3) is mean-variance efficient) and §4.1 (maximizing the quadratic utility E[W−cW2]E[W-cW^2]E[W−cW2], c>0c>0c>0, is problem (3) with L=1/(2c)L=1/(2c)L=1/(2c)).

Significance

Proposition 1 gives the optimal dynamic hedge in closed feedback form for every target level at once. Varying LLL traces out the whole mean-variance frontier of terminal wealth (Lemma 3), and the choice L=1/(2c)L=1/(2c)L=1/(2c) solves the quadratic-utility problem (§4.1); the minimum-variance hedge of §4.3 of the paper is obtained by optimizing over LLL. The result is also an instance of a general pattern: a quadratic hedging problem in an incomplete market reduces to an L2L^2L2 projection onto the space of attainable gains, and the projection is computed by a linear SDE.

The result is proved in the paper, in six pages. No machine-checked version of it, or of any continuous-time hedging result, exists on the platform. A formalization requires Itô's formula for products of Itô processes, the zero-mean property of square-integrable stochastic integrals, Fubini's theorem for moments, and existence for a linear SDE with an Itô-process forcing term; each of these is reusable well beyond this paper.

Difficulty

The projection step (Lemma 1) is Hilbert-space geometry and the final ODE step is Grönwall. The difficulty lies in between: the orthogonality E[(L−kST−GT∗)G(θ)T]=0E[(L-kS_T-G^*_T)G(\theta)_T]=0E[(L−kST​−GT∗​)G(θ)T​]=0 must be verified against every trading strategy θ\thetaθ, about which only E∫0Tθt2Ft2 dt<∞E\int_0^T\theta_t^2F_t^2\,dt<\inftyE∫0T​θt2​Ft2​dt<∞ is known. A computation that treats θ\thetaθ as bounded, continuous or simple does not suffice. Making the paper's moment computations rigorous requires controlling the integrability of products such as ZtθtFtZ_t\theta_tF_tZt​θt​Ft​ and Gt∗G(θ)tG^*_tG(\theta)_tGt∗​G(θ)t​, proving that the stochastic-integral parts of Itô's product rule are true martingales rather than local martingales, and differentiating expectations in time when the coefficients are only measurable, so that derivatives exist only almost everywhere.

Formalization scope

The stochastic layer is the published definition file Peng1990_SMP_Stochastic (the L2L^2L2 Itô integral and Itô processes on R≥0\mathbb R_{\ge0}R≥0​ time), imported, not redefined. The mission commits to the following conventions.

  • (B,ε)(B,\varepsilon)(B,ε) is one R2\mathbb R^2R2-valued standard Brownian motion; BBB is coordinate 0, ε\varepsilonε coordinate 1, and dξd\xidξ is expanded as ρ dB+1−ρ2 dε\rho\,dB+\sqrt{1-\rho^2}\,d\varepsilonρdB+1−ρ2​dε.
  • The filtration is the natural filtration of (B,ε)(B,\varepsilon)(B,ε), not its augmentation, and trading strategies are progressively measurable instead of predictable. Neither change alters the space of terminal gains.
  • Gains are relations: a gain process is any version of the Itô integral, and every statement quantifies over all versions.
  • The objective E[(W−L)2]E[(W-L)^2]E[(W−L)2] and variances take values in [0,∞][0,\infty][0,∞], so a non-square-integrable wealth cannot be optimal through a junk value 000; inner products carry explicit integrability.
  • ρt∈[−1,1]\rho_t\in[-1,1]ρt​∈[−1,1] (§3.1), not [0,1][0,1][0,1] (§2). The sign of vvv is free; only ∣v∣≥δ>0|v|\ge\delta>0∣v∣≥δ>0 is assumed.
  • "Φ(G∗)\Phi(G^*)Φ(G∗) defined by (9)–(11)" means: for every solution of (10), where a solution includes that Φ(G∗)\Phi(G^*)Φ(G∗) is a trading strategy. The paper takes this membership for granted.
  • Lemma 2 and the moment equations are stated in integral form (Ht=H0−∫0t(m2/v2)H dsH_t=H_0-\int_0^t(m^2/v^2)H\,dsHt​=H0​−∫0t​(m2/v2)Hds), which is the paper's "for almost every ttt" derivative together with absolute continuity; continuity of the coefficients is not assumed.
  • The display for dZdZdZ in the proof of Lemma 2 omits dtdtdt; the drift is (γt+μt)Zt dt(\gamma_t+\mu_t)Z_t\,dt(γt​+μt​)Zt​dt.
  • In §4.1, c>0c>0c>0 is assumed explicitly.

Proposition 1 would hold vacuously if equation (10) had no solution; the existence milestone rules this out and must be proved, not assumed. Contributions are welcome on Itô's product formula and the martingale property of Itô integrals in the Peng framework, on the existence of solutions of linear SDEs, and on the Grönwall-type uniqueness for (13).

Selected references

  • D. Duffie, H. R. Richardson, Mean-Variance Hedging in Continuous Time, The Annals of Applied Probability 1(1) (1991) 1–15. https://doi.org/10.1214/aoap/1177005978
  • S. Peng, A General Stochastic Maximum Principle for Optimal Control Problems, SIAM J. Control Optim. 28(4) (1990) 966–979. https://doi.org/10.1137/0328054
  • P. Protter, Stochastic Integration and Differential Equations, Springer, 1990. https://doi.org/10.1007/978-3-662-02619-9
  • D. G. Luenberger, Optimization by Vector Space Methods, Wiley, 1969.
  • M. Schweizer, Mean-Variance Hedging for General Claims, The Annals of Applied Probability 2(1) (1992) 171–179. https://doi.org/10.1214/aoap/1177005776
11 thms1 active userReviewed
Operations Research·Captain: mikedeng1

Extensions of the Queueing Relations L = λW and H = λG: Under (14) and (15) the Time Average H(t) Converges if and only if the Customer Average G(s) Does, and Then H = λGResearch Paper

Motivation

Queueing results often relate an average accumulated over customers to an average accumulated over time. In the familiar relation L=λWL=\lambda WL=λW, the long-run average number in a system equals the arrival rate times the average time spent there. Glynn and Whitt study a broader relation, H=λGH=\lambda GH=λG, that can represent queue lengths and waiting times but also costs, work, and other cumulative inputs. Their framework is deterministic: a stochastic model may supply a sample path, yet the central comparison is a statement about functions on that path. This lets the same theorem apply to different queueing models without choosing a probability law for each one. Glynn and Whitt (1989)

The paper asks for more than the usual forward implication from a customer average to a time average. Its main theorem identifies conditions under which either average has a finite limit exactly when the other does. It also compares lim inf and lim sup when ordinary limits fail to exist. Both questions matter when sample paths fluctuate: convergence of one average cannot simply be inferred from a visual similarity between the two ways of indexing the input. Glynn and Whitt (1989), §§1–2

Setting

A cumulative input is a real-valued function F(s,t)F(s,t)F(s,t) on [0,∞)×[0,∞)[0,\infty)\times[0,\infty)[0,∞)×[0,∞), nondecreasing separately in the customer coordinate sss and the time coordinate ttt. Its marginal limits F(s,∞)F(s,\infty)F(s,∞) and F(∞,t)F(\infty,t)F(∞,t) are finite for every fixed argument. The first counts all eventual input associated with customers through index sss; the second counts all input accumulated through time ttt. The marginal averages are

G(s)=F(s,∞)s,H(t)=F(∞,t)t.G(s)=\frac{F(s,\infty)}{s},\qquad H(t)=\frac{F(\infty,t)}{t}.G(s)=sF(s,∞)​,H(t)=tF(∞,t)​.

The paper permits cumulative inputs that are not distribution functions of measures on rectangles. In particular, it does not require a rectangle-increment inequality or nonnegative values of FFF. The finite marginal limits and monotonicity are the actual assumptions of its general framework. Glynn and Whitt (1989), p. 635, (1)

A time change Ti(s)T_i(s)Ti​(s), for i=1,2i=1,2i=1,2, is finite, nonnegative, nondecreasing, and right-continuous, and tends to infinity with sss. Its right-continuous inverse is Si(t)=inf⁡{s≥0:Ti(s)>t}S_i(t)=\inf\{s\geq0:T_i(s)>t\}Si​(t)=inf{s≥0:Ti​(s)>t}. The two time changes may differ. Both have the same asymptotic rate Ti(s)/s→λ−1T_i(s)/s\to\lambda^{-1}Ti​(s)/s→λ−1 for a positive finite λ\lambdaλ. The paper's two approximation conditions are

F(s,T1(s−))−F(∞,T1(s−))s⟶0(14),F(s,∞)−F(s,T2(s))s⟶0(15),\frac{F(s,T_1(s-))-F(\infty,T_1(s-))}{s}\longrightarrow0 \quad(14), \qquad \frac{F(s,\infty)-F(s,T_2(s))}{s}\longrightarrow0 \quad(15),sF(s,T1​(s−))−F(∞,T1​(s−))​⟶0(14),sF(s,∞)−F(s,T2​(s))​⟶0(15),

as s→∞s\to\inftys→∞, where T1(s−)T_1(s-)T1​(s−) is the left limit. Condition (14) concerns input already present by a left-limit time; condition (15) concerns input not yet present by a right-limit time. Glynn and Whitt (1989), p. 638, (3), (14)–(15)

Formalization targets

One-sided asymptotic bounds

With only (14) and the rate for T1T_1T1​, the corrected upper bounds are

lim inf⁡H≤lim inf⁡λG,lim sup⁡H≤lim sup⁡λG.\liminf H\leq\liminf \lambda G,\qquad \limsup H\leq\limsup \lambda G.liminfH≤liminfλG,limsupH≤limsupλG.

With only (15) and the rate for T2T_2T2​, the corrected lower bounds reverse both inequalities. The two bounds remain useful separately when only one approximation condition holds. Their lim inf and lim sup may be infinite. Glynn and Whitt (1989), p. 639, Theorem 1(a)–(b) and Remark 2

Equality of asymptotic bounds and finite limits

Under both conditions, Theorem 1(c) asserts equality of the respective lim inf and lim sup. The mission goal is Theorem 1(f): for finite real limits,

H(t)→h⟺G(s)→g,and whenever these limits exist, h=λg.H(t)\to h\quad\Longleftrightarrow\quad G(s)\to g, \qquad\text{and whenever these limits exist, }h=\lambda g.H(t)→h⟺G(s)→g,and whenever these limits exist, h=λg.

Here the equivalence means existence of finite limits, with the second clause identifying their values. The lim inf and lim sup statement is retained as a milestone because it also covers nonconvergent paths. Glynn and Whitt (1989), p. 639, Theorem 1(c), (f)

Significance

The goal gives a two-way transfer between customer-indexed and time-indexed long-run averages. If an application establishes one finite average, the theorem supplies the other and fixes its value. The one-sided milestones give meaningful bounds when only one approximation condition is available; the lim inf and lim sup milestone retains information when neither average converges. These outcomes are stated for a general cumulative input, so applications can supply their own FFF, T1T_1T1​, and T2T_2T2​ without changing the theorem. Glynn and Whitt (1989), §§2–4

The paper proves these results on paper. The formalization target is a machine-checked interface for the paper's deterministic framework, its inversion lemmas, its asymptotic inequalities, and the two-way limit theorem. At this drafting stage the Lean theorem statements compile with proof placeholders; no machine-checked proofs of these new statements are claimed. The definitions and inverse-rate lemma can be reused in other sample-path results.

Difficulty

The two marginal averages examine different slices of FFF, and a time change may jump or have flat intervals. A direct substitution of t=Ti(s)t=T_i(s)t=Ti​(s) does not give an equality between F(s,∞)F(s,\infty)F(s,∞) and F(∞,t)F(\infty,t)F(∞,t) under the paper's assumptions. The left limit in (14) and the value in (15) also behave differently at jumps. The inverse SiS_iSi​ connects the coordinates, but its boundary behavior and rate must be handled before comparing the averages. These issues are present even without stochastic randomness. Glynn and Whitt (1989), pp. 638–639

Formalization scope

Lean represents s,ts,ts,t by nonnegative real numbers and FFF by a real-valued function with separate monotonicity and bounded marginal ranges. Real suprema represent F(s,∞)F(s,\infty)F(s,∞) and F(∞,t)F(\infty,t)F(∞,t); the boundedness fields ensure those suprema are the paper's finite limits. The inverse is an infimum of a nonempty set because Ti→∞T_i\to\inftyTi​→∞. The left limit is the supremum over earlier nonnegative arguments, with T(0−)=0T(0-)=0T(0−)=0. Ratios at zero use Lean's total division convention, but every ratio assertion is asymptotic at infinity.

Lim inf and lim sup are taken in the extended reals so an unbounded average retains an infinite limit. Multiplication by λ\lambdaλ happens in the reals before conversion. Ordinary limits in Theorem 1(f) are finite real limits. The two time changes are independent and share only the positive rate λ\lambdaλ. The goal assumes only (14), (15), and the stated time-change rates; it does not assume its own inverse-rate or asymptotic conclusions. No nonnegativity or rectangle inequality is imposed on FFF.

Theorem 1(a) and (b) have their hypothesis pairing reversed in print. The formalized one-sided bounds use the corrected pairing, and the milestone text preserves the printed source for audit. The intermediate expressions involving G(Si(t))G(S_i(t))G(Si​(t)) are excluded from the corrected statements because the framework does not assume customer-coordinate right continuity. Theorem 1(c) and (f) use both conditions and are true as printed. The scope includes the definitions, Lemmas 1–2, corrected parts (a)–(b), and parts (c) and (f). Contributions toward their proofs and reusable time-change lemmas are welcome. Glynn and Whitt (1989), p. 639, Theorem 1

Selected references

  • P. W. Glynn and W. Whitt, Extensions of the Queueing Relations L = λW and H = λG, Operations Research 37(4):634–644, 1989. DOI: 10.1287/opre.37.4.634
9 thms1 active userReviewed
Dynamical SystemsOperations Research·Captain: mikedeng1

Stability and Instability of Fluid Models for Reentrant Lines 3: In Every Reentrant Line the Last-Buffer-First-Served Fluid Model Is StableResearch Paper

Motivation

A manufacturing line can send the same job through several stations and return it to a station it visited earlier. Semiconductor fabrication is one example. Such a reentrant line has several customer classes at one server, and the server must choose which class to work on. A station may have enough capacity on average yet the line can still accumulate an unbounded queue under an unfortunate service rule. Dai and Weiss use deterministic fluid models to study this question: fluid represents large queues, and its motion exposes the long-run effect of the service rule. Their paper proves that two simple buffer priority disciplines have stable fluid models for every reentrant line satisfying the usual station load condition. This mission concerns Last-Buffer-First-Served, abbreviated LBFS. Dai and Weiss (1996).

The distinction between fluid and stochastic queue stability matters. Dai and Weiss cite an earlier theorem connecting stable fluid models to stable queueing disciplines under the stochastic assumptions of their introduction. The target here is their fluid theorem, Theorem 4.4, rather than a separate formalization of that stochastic implication. Dai and Weiss (1996), pp. 115, 119, 124.

Setting

There are KKK classes, numbered in route order, and III stations. Every unit of fluid enters class 111 at rate one. On completing service in class kkk, it becomes class k+1k+1k+1; after class KKK, it leaves. The station serving class kkk is σ(k)\sigma(k)σ(k). Different classes may share a station. The constituency of station iii is Ci={k:σ(k)=i}C_i=\{k:\sigma(k)=i\}Ci​={k:σ(k)=i}, and class kkk has positive mean service requirement mkm_kmk​, so its service rate is μk=1/mk\mu_k=1/m_kμk​=1/mk​. The nominal workload at station iii is ρi=∑k∈Cimk\rho_i=\sum_{k\in C_i}m_kρi​=∑k∈Ci​​mk​. The paper assumes ρi<1\rho_i<1ρi​<1 at every station. Dai and Weiss (1996), pp. 115–117.

For time t≥0t\ge0t≥0, Qk(t)Q_k(t)Qk​(t) is the amount of fluid in class kkk, and Tk(t)T_k(t)Tk​(t) is the cumulative server time spent on that class. The flow equations say that class content equals its initial content plus completed service from the preceding class minus its own completed service. The first class receives the external unit-rate flow. Each QkQ_kQk​ remains nonnegative, each TkT_kTk​ starts at zero and is nondecreasing, and the cumulative idle time of every station is nondecreasing. A priority rule adds a condition on when a server may reserve capacity for lower-ranked classes. These are the fluid equations of Definition 1.2 and §4. Dai and Weiss (1996), pp. 118, 122–123.

A priority ranking π\piπ assigns a distinct rank to every class, with smaller rank meaning higher priority. For class kkk, let HkH_kHk​ contain the classes at the same station whose rank is at least as high as kkk's. In a preemptive-resume priority fluid model, capacity unused by HkH_kHk​ may increase only when the fluid workload in HkH_kHk​ is zero. Under LBFS, π(k)=K+1−k\pi(k)=K+1-kπ(k)=K+1−k: a class later in the route has higher priority. Priority is compared within each station, even though the ranking is written globally. Dai and Weiss (1996), pp. 122–123, Definition 4.1.

Formalization targets

Theorem 4.4 asks for stability of the LBFS fluid model for every reentrant line with positive service requirements and ρi<1\rho_i<1ρi​<1:

∃δ>0  ∀(Q,T),[(Q,T) is an LBFS fluid solution and ∣Q(0)∣=1]⟹∀t≥δ,  Q(t)=0,\exists\delta>0\;\forall (Q,T),\quad \bigl[(Q,T)\text{ is an LBFS fluid solution and }|Q(0)|=1\bigr] \Longrightarrow \forall t\ge\delta,\;Q(t)=0,∃δ>0∀(Q,T),[(Q,T) is an LBFS fluid solution and ∣Q(0)∣=1]⟹∀t≥δ,Q(t)=0,

where ∣Q(0)∣=∑k=1KQk(0)|Q(0)|=\sum_{k=1}^K Q_k(0)∣Q(0)∣=∑k=1K​Qk​(0). The same δ\deltaδ must work for every solution and every normalized initial configuration. This is exactly the fluid stability definition used by the paper. Dai and Weiss (1996), pp. 118–119, 124.

The milestones follow the paper's own intermediate statements. Lemma 2.2 treats a nonnegative absolutely continuous function whose derivative is negative whenever the function is positive. Proposition 4.2 identifies flow-rate relations at a regular point under a buffer priority rule. The proof of Theorem 4.4 records a uniform negative drift for total content and the explicit emptying time

δ=∣Q(0)∣λ^−1,λ^=1max⁡1≤i≤Iρi>1.\delta=\frac{|Q(0)|}{\widehat\lambda-1},\qquad \widehat\lambda=\frac{1}{\max_{1\le i\le I}\rho_i}>1.δ=λ−1∣Q(0)∣​,λ=max1≤i≤I​ρi​1​>1.

The explicit time is stated for every initial fluid amount; normalization to one belongs only to the final stability theorem. Dai and Weiss (1996), pp. 120, 123–125.

Significance

The theorem gives a uniform finite clearing time for the LBFS fluid model whenever each station's nominal workload is below its capacity. It applies to any number of stations, any finite route, and any assignment of route stages to stations. This is stronger than checking a particular factory layout. The conclusion concerns every fluid solution, which matters because the defining equations need not determine a unique path. Dai and Weiss (1996), pp. 118–119, 124–125.

Formalizing the result requires a reusable interface for reentrant lines, service allocation, station workloads, and the priority complementarity condition, plus separate statements for the extinction criterion and priority flow rates. The theorem is proved in the 1996 paper; the remaining work in this mission is a machine-checked Lean proof of its fluid-model statement and of the listed milestones. The stochastic queueing theorem cited by Dai and Weiss is outside this mission's scope.

Difficulty

The load inequalities ρi<1\rho_i<1ρi​<1 compare average work with available server time, but by themselves they do not specify which class receives service at an instant. Under other service rules, reentrant lines can amplify queued fluid despite every station being nominally underloaded; the same paper gives such an example. A formal proof therefore has to use the precise LBFS priority condition, not only the flow equations and the load inequalities. The regular-time argument must then yield a single bound valid across every possible fluid path. Dai and Weiss (1996), §§4–5.

Formalization scope

Lean represents classes and stations by finite index types. Their indices start at zero: Lean class k−1k-1k−1 corresponds to paper class kkk, and LBFS is Fin.revPerm. The paths QQQ and TTT are functions on all real times, while the fluid equations and conclusions are imposed only for t≥0t\ge0t≥0. The external arrival rate is one, as in the paper's normalization. The conditions that station idle capacity and priority-set unused capacity increase only when their relevant content is zero are expressed by constancy on each interval where that content stays positive. Service and fluid paths are not given extra continuity hypotheses; the fluid equations and monotonicity conditions provide the regularity used in the paper. Dai and Weiss (1996), pp. 116–119, 122–123.

The service requirements mk>0m_k>0mk​>0 are explicit so μk=1/mk\mu_k=1/m_kμk​=1/mk​ has its intended value. The drift and emptying-time milestones explicitly assume K>0K>0K>0, since their formula divides by the maximum station workload. The main theorem retains the universal quantification over lines; when K=0K=0K=0, its normalization ∣Q(0)∣=1|Q(0)|=1∣Q(0)∣=1 is impossible. A faithful solution predicate must include the priority condition (4.4), whose omission would change Theorem 4.4 into a false claim about all work-conserving policies. The definition layer, elementary real-analysis extinction lemma, and priority flow-rate relations are useful beyond this single stability theorem. Dai and Weiss (1996), pp. 117–125.

Selected references

  • J. G. Dai and G. Weiss, Stability and instability of fluid models for reentrant lines, Mathematics of Operations Research 21(1), 115–134, 1996. DOI: 10.1287/moor.21.1.115.
8 thms1 active userReviewed
Dynamical SystemsOperations Research·Captain: mikedeng1

Stability and Instability of Fluid Models for Reentrant Lines 2: In Every Reentrant Line the First-Buffer-First-Served Fluid Model Is StableResearch Paper

Why priority rules in reentrant lines matter

Semiconductor wafer fabrication is the standard example of a reentrant line: every wafer follows the same long route through a set of machines, and visits several machines many times. Each visit is a separate processing step, and a machine with work waiting from several steps must choose which step to serve. Kumar (1993) proposed studying such lines through their buffer priority rules, and Lu and Kumar (1991) showed that a priority rule can make a line unstable even though every machine has spare capacity. The question is therefore not academic: a scheduling rule that looks reasonable can let queues grow without bound.

Dai (1995) reduced the positive Harris recurrence of a multiclass queueing network to the stability of a deterministic fluid model, and Dai and Weiss (1996) used this reduction to prove stability results for several reentrant lines. This mission formalizes their Theorem 4.3: under the First-Buffer-First-Served (FBFS) discipline, the fluid model of every reentrant line is stable whenever each station's nominal workload is below one. Kumar (1993) had proved the analogous statement for discrete deterministic systems.

Timeline:

  • 1991. Lu and Kumar exhibit a two-station reentrant line that is unstable under a static priority rule with every workload below one.
  • 1993. Kumar proves stability of FBFS and LBFS for discrete deterministic reentrant lines.
  • 1995. Dai shows that stability of the fluid model implies positive Harris recurrence of the stochastic network.
  • 1996. Dai and Weiss prove that the FBFS and LBFS fluid models of every reentrant line are stable (Theorems 4.3 and 4.4), and with Dai's theorem obtain stability of the stochastic networks.

Setting

A reentrant line has III stations and KKK classes. All fluid follows one route: it enters as class 111, becomes class k+1k+1k+1 when class kkk is completed, and leaves after class KKK. Class kkk is served at station σ(k)\sigma(k)σ(k) with mean service time mk>0m_k > 0mk​>0, and μk=1/mk\mu_k = 1/m_kμk​=1/mk​. The constituency of station iii is Ci={k:σ(k)=i}C_i = \{k : \sigma(k) = i\}Ci​={k:σ(k)=i}, and its nominal workload is ρi=∑k∈Cimk\rho_i = \sum_{k\in C_i} m_kρi​=∑k∈Ci​​mk​. The exogenous arrival rate is 111. The standing assumption of the paper is

ρi<1(i=1,…,I).(1.7)\rho_i < 1 \qquad (i = 1,\dots,I). \tag{1.7}ρi​<1(i=1,…,I).(1.7)

A fluid model solution is a pair of paths Q(t)=(Qk(t))kQ(t) = (Q_k(t))_kQ(t)=(Qk​(t))k​, the fluid levels, and T(t)=(Tk(t))kT(t) = (T_k(t))_kT(t)=(Tk​(t))k​, the cumulative service time given to each class, such that for t≥0t \ge 0t≥0:

Qk(t)=Qk(0)+μk−1Tk−1(t)−μkTk(t),Qk(t)≥0,Q_k(t) = Q_k(0) + \mu_{k-1}T_{k-1}(t) - \mu_k T_k(t), \qquad Q_k(t) \ge 0,Qk​(t)=Qk​(0)+μk−1​Tk−1​(t)−μk​Tk​(t),Qk​(t)≥0,

with μ0T0(t)=t\mu_0T_0(t) = tμ0​T0​(t)=t; each TkT_kTk​ starts at 000 and is nondecreasing; and the idle time Ui(t)=t−∑k∈CiTk(t)U_i(t) = t - \sum_{k\in C_i}T_k(t)Ui​(t)=t−∑k∈Ci​​Tk​(t) of each station is nondecreasing.

A buffer priority discipline is a permutation π\piπ of the classes; a class kkk has priority over a class lll at the same station when π(k)<π(l)\pi(k) < \pi(l)π(k)<π(l). Write Hk={l∈Cσ(k):π(l)≤π(k)}H_k = \{l \in C_{\sigma(k)} : \pi(l) \le \pi(k)\}Hk​={l∈Cσ(k)​:π(l)≤π(k)}, Tk+=∑l∈HkTlT_k^+ = \sum_{l\in H_k}T_lTk+​=∑l∈Hk​​Tl​, Uk+(t)=t−Tk+(t)U_k^+(t) = t - T_k^+(t)Uk+​(t)=t−Tk+​(t) and Wk+=∑l∈HkmlQlW_k^+ = \sum_{l\in H_k} m_l Q_lWk+​=∑l∈Hk​​ml​Ql​. Under preemptive resume, Uk+U_k^+Uk+​ may increase only at times when Wk+=0W_k^+ = 0Wk+​=0 (condition (4.4)): a station never idles toward the classes of priority at least that of kkk while any of them holds fluid. FBFS is π(k)=k\pi(k) = kπ(k)=k, so earlier steps of the route have priority.

The fluid model is stable (Definition 1.3) if there is a time δ>0\delta > 0δ>0 such that every solution with ∣Q(0)∣=∑kQk(0)=1|Q(0)| = \sum_k Q_k(0) = 1∣Q(0)∣=∑k​Qk​(0)=1 has Q(t)=0Q(t) = 0Q(t)=0 for all t≥δt \ge \deltat≥δ.

Formalization targets

Goal: Theorem 4.3

For every reentrant line with mk>0m_k > 0mk​>0 and (1.7),

the fluid model (1.8)–(1.12), (4.4) with π(k)=k is stable.\text{the fluid model (1.8)–(1.12), (4.4) with } \pi(k) = k \text{ is stable.}the fluid model (1.8)–(1.12), (4.4) with π(k)=k is stable.

The goal asserts existence of an emptying time; it does not fix the value.

Milestones

  1. Lemma 2.2 (i): a nonnegative function has zero derivative wherever it vanishes and is differentiable.
  2. Lemma 2.2 (ii): an absolutely continuous nonnegative ggg whose derivative is at most −ε-\varepsilon−ε wherever g>0g > 0g>0 vanishes from g(0)/εg(0)/\varepsilong(0)/ε on, and is nonincreasing.
  3. Proposition 4.2: at a regular time of a priority fluid model, an empty buffer has equal in- and out-flow rates, and at each station the highest-priority nonempty class satisfies ∑k∈Hk0mkdk=1\sum_{k\in H_{k_0}} m_kd_k = 1∑k∈Hk0​​​mk​dk​=1 while lower-priority classes have zero out-flow.
  4. Inductive step (proof of Theorem 4.3): once buffers 1,…,k−11,\dots,k-11,…,k−1 stay empty from tk−1t_{k-1}tk−1​ on, buffer kkk is empty from tk=tk−1+Qk(tk−1)mk/(1−∑l∈Hkml)t_k = t_{k-1} + Q_k(t_{k-1})m_k/(1 - \sum_{l\in H_k} m_l)tk​=tk−1​+Qk​(tk−1​)mk​/(1−∑l∈Hk​​ml​) on.
  5. Emptying time (proof of Theorem 4.3): every solution with ∣Q(0)∣=1|Q(0)| = 1∣Q(0)∣=1 is empty from
δ=∑k=1Kmk∏l=1k−1(1−∑j∈Hl∖{l}mj)∏l=1k(1−∑j∈Hlmj)\delta = \sum_{k=1}^K m_k\frac{\prod_{l=1}^{k-1}(1-\sum_{j\in H_l\setminus\{l\}}m_j)}{\prod_{l=1}^{k}(1-\sum_{j\in H_l}m_j)}δ=k=1∑K​mk​∏l=1k​(1−∑j∈Hl​​mj​)∏l=1k−1​(1−∑j∈Hl​∖{l}​mj​)​

on, which gives the goal with an explicit constant.

Significance

Combined with Dai's theorem (Theorem 1.1 of the paper), Theorem 4.3 gives positive Harris recurrence of every multiclass reentrant line operated under FBFS, under the distributional assumptions of that theorem, whenever (1.7) holds. Since Lu–Kumar-type examples show that priority rules can destabilize a network with spare capacity, a rule that is provably stable on every reentrant line is a meaningful guarantee. FBFS, together with LBFS, is the basic positive case against which the instability examples in the same paper (Theorem 5.1) are compared.

Formalizing the result adds a machine-checked version of the fluid model with priority constraints, stated for an arbitrary line rather than a fixed network, and a checked explicit emptying time. The result is proved in the paper; no machine-checked proof is known. The rate identities of Proposition 4.2 and the extinction lemma are reused by the LBFS mission of this series.

Difficulty

The fluid model gives the paths only as nondecreasing functions subject to equations; nothing says they are differentiable. Every argument about rates holds only at regular times, and the conclusion must be integrated back through absolute continuity, which itself must be derived from the monotonicity conditions. The priority condition (4.4) is an "increases only when empty" condition; turning it into the rate identity (4.6) at a regular point requires continuity of the fluid levels and a careful local argument. The induction over classes needs uniform control of every earlier buffer for all later times, not only at one instant, so a pointwise reading of "buffer k−1k-1k−1 is empty" is not enough. Finally, the explicit constant δ\deltaδ requires bounding the content of buffer kkk at time tk−1t_{k-1}tk−1​ by the total content, which may have grown since time 000.

Formalization scope

  • Classes and stations are Fin K and Fin I, 0-based: the paper's class kkk is Lean index k−1k-1k−1.
  • Paths are total functions ℝ → Fin K → ℝ; every equation is imposed on t≥0t \ge 0t≥0 only, and derivatives are taken at t>0t > 0t>0.
  • Conditions (1.13) and (4.4) are in interval form (ConstantWhile): the idle process is constant on every interval of [0,∞)[0,\infty)[0,∞) on which the content stays positive. This is equivalent to the paper's Stieltjes integral form for continuous paths.
  • Lipschitz continuity of the paths is not assumed; it follows from the monotonicity conditions.
  • mk>0m_k > 0mk​>0 is an explicit hypothesis; the paper takes it for granted.
  • Priorities are Equiv.Perm (Fin K), FBFS is the identity; ∣Q(0)∣|Q(0)|∣Q(0)∣ is ∑kQk(0)\sum_k Q_k(0)∑k​Qk​(0).
  • In Lemma 2.2 (ii) the printed strict "g˙(t)<−ε\dot g(t) < -\varepsilong˙​(t)<−ε" is relaxed to "≤−ε\le -\varepsilon≤−ε", as every application requires; absolute continuity is dropped from Lemma 2.2 (i), where it is not needed. Both changes strengthen the lemma.
  • In the inductive step, "stay empty for t>tk−1t > t_{k-1}t>tk−1​" is read as t≥tk−1t \ge t_{k-1}t≥tk−1​, and the case Qk(tk−1)=0Q_k(t_{k-1}) = 0Qk​(tk−1​)=0 is included.
  • The stochastic network is not formalized; only the fluid model is.

A trivializing formalization is ruled out: the goal uses the priority fluid model, not the work-conserving one alone (which would claim that every work-conserving fluid model is stable, false by Theorem 5.1), and a sorry-free witness shows that the FBFS solution predicate has solutions with ∣Q(0)∣=1|Q(0)| = 1∣Q(0)∣=1, so stability is not vacuous.

Contributions welcome: the extinction lemma for absolutely continuous functions, the derivation of Lipschitz continuity from (1.10)–(1.12), and the rate identities of Proposition 4.2 are reusable beyond this mission.

Selected references

  • J. G. Dai and G. Weiss, Stability and instability of fluid models for reentrant lines, Mathematics of Operations Research 21(1) (1996) 115–134. https://doi.org/10.1287/moor.21.1.115
  • J. G. Dai, On positive Harris recurrence of multiclass queueing networks: a unified approach via fluid limit models, Annals of Applied Probability 5(1) (1995) 49–77. https://doi.org/10.1214/aoap/1177004828
  • P. R. Kumar, Re-entrant lines, Queueing Systems 13 (1993) 87–110. https://doi.org/10.1007/BF01149327
  • S. H. Lu and P. R. Kumar, Distributed scheduling based on due dates and buffer priorities, IEEE Transactions on Automatic Control 36(12) (1991) 1406–1416. https://doi.org/10.1109/9.106159
8 thms1 active userReviewed
Control TheoryOperations ResearchProbability·Captain: mikedeng1

Scheduling a Multi Class Queue with Many Exponential Servers: Asymptotic Optimality in Heavy Traffic: The HJB-Based Preemptive Policy Is Asymptotically Optimal Among Work-Conserving PoliciesResearch Paper

Motivation

Large call centers route several types of customers to a common pool of agents. When the pool is large and highly utilized, the relevant asymptotic regime is the quality-and-efficiency-driven (QED) or Halfin–Whitt regime (Halfin & Whitt 1981). The number of servers nnn grows while the offered load stays within O(n)O(\sqrt n)O(n​) of nnn. Waiting is then neither negligible nor overwhelming (Gans, Koole & Mandelbaum 2003).

Which class should a freed agent serve next? Exact optimization of a multi-class many-server queue with abandonment is intractable. The standard route is to solve a limiting diffusion control problem and translate its optimal control back into a policy for the queue. Atar, Mandelbaum and Reiman (Ann. Appl. Probab. 2004) carried this out for kkk customer classes, exponential service and abandonment, general renewal arrivals and general convex-type holding costs. They proved that the translated policy is asymptotically optimal. This mission formalizes that result for the preemptive policy.

Context:

  • Harrison & Zeevi (2004) studied the same multi-class many-server problem.
  • Bell & Williams (2001) proved asymptotic optimality of a threshold policy for a two-server system in conventional heavy traffic.
  • The present paper is the first to cover the QED regime with general costs and abandonment.

Setting

There are k≥1k\ge1k≥1 customer classes and nnn identical servers.

Primitives.

  • Arrivals. Class-iii customers arrive according to a renewal process AinA^n_iAin​ with interarrival times Uˇi(j)/λin\check U_i(j)/\lambda^n_iUˇi​(j)/λin​. Here the Uˇi(j)\check U_i(j)Uˇi​(j) are i.i.d., positive, of mean one and squared coefficient of variation CU,i2C^2_{U,i}CU,i2​.
  • Service. Service times are exponential with rate μin\mu^n_iμin​, represented by Poisson processes SinS^n_iSin​.
  • Abandonment. Waiting customers abandon at rate θin≥0\theta^n_i\ge0θin​≥0, represented by Poisson processes RinR^n_iRin​.

State. Xin(t)X^n_i(t)Xin​(t) is the number of class-iii customers in the system, Ψin(t)\Psi^n_i(t)Ψin​(t) the number in service and Φin=Xin−Ψin\Phi^n_i=X^n_i-\Psi^n_iΦin​=Xin​−Ψin​ the number waiting. The dynamics are

Xin(t)=Xi0,n+Ain(t)−Rin(∫0tΦin)−Sin(∫0tΨin),Ψn,Φn∈Z+k,∑iΨin≤n.X^n_i(t)=X^{0,n}_i+A^n_i(t)-R^n_i\Big(\int_0^t\Phi^n_i\Big)-S^n_i\Big(\int_0^t\Psi^n_i\Big),\qquad \Psi^n,\Phi^n\in\mathbb Z^k_+,\quad \textstyle\sum_i\Psi^n_i\le n .Xin​(t)=Xi0,n​+Ain​(t)−Rin​(∫0t​Φin​)−Sin​(∫0t​Ψin​),Ψn,Φn∈Z+k​,∑i​Ψin​≤n.

Policies.

  • A scheduling control policy (SCP) is the process Ψn\Psi^nΨn.
  • It is admissible if it does not anticipate the future beyond the time of the next arrival: past information is independent of future primitive increments.
  • It is work-conserving if no server idles while customers wait: (1⋅Xn−n)+=1⋅Φn(\mathbb 1\cdot X^n-n)^+=\mathbb 1\cdot\Phi^n(1⋅Xn−n)+=1⋅Φn.

Scaling and cost. In the QED scaling n−1λin→λin^{-1}\lambda^n_i\to\lambda_in−1λin​→λi​ with ∑iλi/μi=1\sum_i\lambda_i/\mu_i=1∑i​λi​/μi​=1. With ρi=λi/μi\rho_i=\lambda_i/\mu_iρi​=λi​/μi​ the centred processes are X^n=n−1/2(Xn−ρn)\hat X^n=n^{-1/2}(X^n-\rho n)X^n=n−1/2(Xn−ρn), Φ^n=n−1/2Φn\hat\Phi^n=n^{-1/2}\Phi^nΦ^n=n−1/2Φn and Ψ^n=n−1/2(Ψn−ρn)\hat\Psi^n=n^{-1/2}(\Psi^n-\rho n)Ψ^n=n−1/2(Ψn−ρn). The cost is

Cn=E∫0∞e−γtL~(Φ^n(t),Ψ^n(t)) dt.C^n=E\int_0^\infty e^{-\gamma t}\tilde L(\hat\Phi^n(t),\hat\Psi^n(t))\,dt .Cn=E∫0∞​e−γtL~(Φ^n(t),Ψ^n(t))dt.

The limiting control problem. It controls

X(t)=x+rW(t)+∫0tb(X(s),u(s)) ds,b(x,u)=ℓ+(μ−θ)(1⋅x)+u−μx,X(t)=x+rW(t)+\int_0^t b(X(s),u(s))\,ds,\qquad b(x,u)=\ell+(\mu-\theta)(\mathbb 1\cdot x)^+u-\mu x,X(t)=x+rW(t)+∫0t​b(X(s),u(s))ds,b(x,u)=ℓ+(μ−θ)(1⋅x)+u−μx,

where the control uuu takes values in the simplex Sk\mathbb S^kSk and WWW is a kkk-dimensional Brownian motion. The data are ri=(λiCU,i2+λi)1/2r_i=(\lambda_iC^2_{U,i}+\lambda_i)^{1/2}ri​=(λi​CU,i2​+λi​)1/2 and ℓi=λ^i−ρiμ^i\ell_i=\hat\lambda_i-\rho_i\hat\mu_iℓi​=λ^i​−ρi​μ^​i​. Its value V(x)V(x)V(x) is the infimum of E∫0∞e−γtL(X,u) dtE\int_0^\infty e^{-\gamma t}L(X,u)\,dtE∫0∞​e−γtL(X,u)dt, with L(x,u)=L~((1⋅x)+u,x−(1⋅x)+u)L(x,u)=\tilde L((\mathbb 1\cdot x)^+u,x-(\mathbb 1\cdot x)^+u)L(x,u)=L~((1⋅x)+u,x−(1⋅x)+u).

HJB equation and the proposed policy. The HJB equation is 12∑iri2∂iif+H(x,Df)−γf=0\tfrac12\sum_ir_i^2\partial_{ii}f+H(x,Df)-\gamma f=021​∑i​ri2​∂ii​f+H(x,Df)−γf=0 with H(x,p)=inf⁡u∈Sk[b(x,u)⋅p+L(x,u)]H(x,p)=\inf_{u\in\mathbb S^k}[b(x,u)\cdot p+L(x,u)]H(x,p)=infu∈Sk​[b(x,u)⋅p+L(x,u)]. Let hhh be a measurable selection of its minimizers. The proposed preemptive policy (P-SCP) sets the queue vector to Θ[(1⋅Xn−n)+h(X^n)]\Theta[(\mathbb 1\cdot X^n-n)^+h(\hat X^n)]Θ[(1⋅Xn−n)+h(X^n)], an integer rounding, and falls back to a static priority rule when that is infeasible.

Formalization targets

Goal: Theorem 2(i)

For a Cpol2C^2_{\mathrm{pol}}Cpol2​ solution fff of the HJB equation, a measurable minimizer selection hhh, and initial states with X^0,n→x\hat X^{0,n}\to xX^0,n→x:

lim⁡n→∞E∫0∞e−γtL~(Φ^tn,∗,Ψ^tn,∗) dt  ≤  lim inf⁡n→∞E∫0∞e−γtL~(Φ^tn,Ψ^tn) dt\lim_{n\to\infty}E\int_0^\infty e^{-\gamma t}\tilde L(\hat\Phi^{n,*}_t,\hat\Psi^{n,*}_t)\,dt\;\le\;\liminf_{n\to\infty}E\int_0^\infty e^{-\gamma t}\tilde L(\hat\Phi^n_t,\hat\Psi^n_t)\,dtn→∞lim​E∫0∞​e−γtL~(Φ^tn,∗​,Ψ^tn,∗​)dt≤n→∞liminf​E∫0∞​e−γtL~(Φ^tn​,Ψ^tn​)dt

This holds for every sequence of work-conserving admissible SCPs, and the left-hand limit exists and is finite. No constants are hard-coded.

Milestones

The milestones follow the proof:

  • on the diffusion side, Proposition 2 (well-posedness), Proposition 4 (stability and moment bounds), Proposition 5(i)–(ii) (growth and continuity of VVV) and Theorem 3 (VVV is the unique Cpol2C^2_{\mathrm{pol}}Cpol2​ HJB solution, and an optimal Markov policy exists);
  • on the queueing side, Proposition 1 (feedback rules give admissible SCPs), Lemmas 2–3 (moment bounds), Lemma 4(i)–(ii) (FCLT for the primitives and the fluid limit (Ψˉn,Φˉn)⇒(ρ,0)(\bar\Psi^n,\bar\Phi^n)\Rightarrow(\rho,0)(Ψˉn,Φˉn)⇒(ρ,0)), and Theorem 4(i)–(ii): lim inf⁡≥V(x)\liminf\ge V(x)liminf≥V(x) always, and lim sup⁡≤V(x)\limsup\le V(x)limsup≤V(x) under condition (49).

Significance

The result. Theorem 2(i) justifies using the diffusion control problem as a design tool for multi-class many-server systems. The policy is explicit given hhh, and it is optimal in the limit against all non-anticipating work-conserving policies, including those that use the full history and the time of the next arrival. The proof also identifies the limit cost with V(x)V(x)V(x).

Formalizing it. The result is proved on paper, with some steps (Proposition 1, the principle of optimality, the time-change and martingale limit theorems) given as sketches or citations. No part of it is machine-checked. A formalization requires:

  • a counting-process model of the queue;
  • a careful definition of non-anticipation;
  • a pathwise controlled SDE;
  • classical solvability of a semilinear elliptic HJB equation on Rk\mathbb R^kRk;
  • a weak-convergence argument in Skorokhod space.

Each of these is reusable well beyond this paper.

Difficulty

The obvious argument would show that X^n\hat X^nX^n converges to the controlled diffusion and pass the costs to the limit. This fails for two reasons:

  • the comparison class contains arbitrary non-Markov, history-dependent policies, so the queue does not converge to a single controlled diffusion;
  • the optimal selector hhh is in general discontinuous (for linear costs it is), so the proposed policy is not a continuous function of the state.

The proof instead compares every policy with the HJB solution through Itô's formula on the prelimit processes. This needs:

  • uniform moment bounds;
  • tightness of the integral processes;
  • the convergence of stochastic integrals of Kurtz and Protter;
  • and, for the proposed policy, the fact that the rounding Θ\ThetaΘ and the priority fallback perturb the minimizer by O(n−1/2)O(n^{-1/2})O(n−1/2).

Existence of a classical HJB solution on all of Rk\mathbb R^kRk, with only Hölder-continuous costs and polynomial growth, rests on a bounded-domain existence theorem for fully nonlinear elliptic equations.

Formalization scope

The Lean development commits to the following conventions.

  • Indexing and norms. Classes are Fin k with k≥1k\ge1k≥1; paper class iii is index i−1i-1i−1, so "class kkk" (highest priority, rounding remainder of Θ\ThetaΘ) is the last index. Vectors are Fin k → ℝ and ∥⋅∥\|\cdot\|∥⋅∥ is the paper's ℓ1\ell^1ℓ1 norm; the paper's ∣⋅∣|\cdot|∣⋅∣ on vectors is read the same way.
  • Probability space and paths. All systems share one complete probability space. Time is real and every condition is for t≥0t\ge0t≥0. The paper's "without loss" path regularity (finite arrival counts, Poisson paths Z+\mathbb Z_+Z+​-valued, nondecreasing and càdlàg) holds for every ω\omegaω.
  • Poisson processes are defined by independent Poisson increments; rate 000 gives the zero process.
  • Policies. A policy is a pair of real processes (Ψn,Xn)(\Psi^n,X^n)(Ψn,Xn) with integer values. Admissibility is Definition 2 verbatim, with the future σ\sigmaσ-field built from the next arrival time τin(t)\tau^n_i(t)τin​(t). Work conservation is (18).
  • Costs and value are lower Lebesgue integrals in [0,∞][0,\infty][0,∞], and lim⁡\limlim/lim inf⁡\liminfliminf are taken there. The integrands are nonnegative under work conservation.
  • Admissible systems range over sample spaces Ω : Type (universe 0). "Complete filtered probability space" means PPP complete with all null sets in F0\mathcal F_0F0​. Brownian motion is Mathlib's IsBrownianReal per coordinate, with independence and the (Ft)(\mathcal F_t)(Ft​)-Brownian property stated explicitly. VVV is the infimum over systems and their controlled processes.
  • Discount rate. γ>0\gamma>0γ>0 is a hypothesis; the paper leaves it implicit.
  • Initial states are integer vectors X0,n∈Z+kX^{0,n}\in\mathbb Z^k_+X0,n∈Z+k​ with n−1/2(X0,n−ρn)→xn^{-1/2}(X^{0,n}-\rho n)\to xn−1/2(X0,n−ρn)→x. The literal "X^0,n∈n−1/2Zk\hat X^{0,n}\in n^{-1/2}\mathbb Z^kX^0,n∈n−1/2Zk" would require ρin∈Z\rho_in\in\mathbb Zρi​n∈Z. Assumption 1(ii) is not imposed: each policy chooses its own initial split.
  • Lemma 3 is stated for all nnn beyond a threshold that depends on the sequence, with constants c,mˉc,\bar mc,mˉ chosen before xxx and the sequence. The printed all-nnn bound with ccc independent of xxx fails when the early terms X^0,n\hat X^{0,n}X^0,n are large.
  • Weak convergence to a continuous limit uses the coupling form CouplingConverges of the published BellWilliams2001.ThresholdPolicy.Paths; convergence to a deterministic limit is UocInProb.

The goal hypothesizes fff and hhh with the pointwise identity b(x,h(x))⋅Df(x)+L(x,h(x))=H(x,Df(x))b(x,h(x))\cdot Df(x)+L(x,h(x))=H(x,Df(x))b(x,h(x))⋅Df(x)+L(x,h(x))=H(x,Df(x)) for all xxx. An arbitrary "optimal Markov control policy" may differ from a minimizer selection on the Lebesgue-null lattice where X^n\hat X^nX^n lives, and that formalization would make the goal false. Restricting the comparators to feedback, Markov or nonpreemptive policies, fixing kkk, dropping abandonment, specializing to Poisson arrivals or linear costs, or imposing a common initial split would each trivialize or weaken the statement and is ruled out.

Not formalized:

  • Lemma 4(iii) (tightness);
  • Lemma 5 (Kurtz–Protter, which needs semimartingale theory absent from Mathlib);
  • Lemma 6 (convergence of Stieltjes integrals at limit points);
  • Proposition 5(iii);
  • the nonpreemptive results, Theorem 2(ii)–(iii).

Contributions are welcome on any milestone, and especially on infrastructure: Poisson and renewal processes, functional central limit theorems in Skorokhod space, classical solvability of elliptic HJB equations, and measurable selection of minimizers.

Selected references

  • R. Atar, A. Mandelbaum, M. I. Reiman, Scheduling a multi class queue with many exponential servers: asymptotic optimality in heavy traffic, Ann. Appl. Probab. 14(3), 2004. https://arxiv.org/abs/math/0407058
  • S. Halfin, W. Whitt, Heavy-traffic limits for queues with many exponential servers, Oper. Res. 29(3), 1981. https://doi.org/10.1287/opre.29.3.567
  • N. Gans, G. Koole, A. Mandelbaum, Telephone call centers: tutorial, review, and research prospects, Manuf. Serv. Oper. Manag. 5(2), 2003. https://doi.org/10.1287/msom.5.2.79.16071
  • J. M. Harrison, A. Zeevi, Dynamic scheduling of a multiclass queue in the Halfin–Whitt heavy traffic regime, Oper. Res. 52(2), 2004. https://doi.org/10.1287/opre.1040.0109
  • S. L. Bell, R. J. Williams, Dynamic scheduling of a system with two parallel servers in heavy traffic with resource pooling: asymptotic optimality of a threshold policy, Ann. Appl. Probab. 11(3), 2001. https://doi.org/10.1214/aoap/1015345343
  • T. G. Kurtz, P. Protter, Weak limit theorems for stochastic integrals and stochastic differential equations, Ann. Probab. 19(3), 1991. https://doi.org/10.1214/aop/1176990334
16 thms1 active userReviewed
Markov ChainOperations ResearchProbability·Captain: mikedeng1

On the Stochastic Matrices Associated with Certain Queuing Processes 2: The GI/M/1 Imbedded Chain Is Ergodic iff ρ < 1 and Recurrent iff ρ ≤ 1Research Paper

Motivation

A single-server queue in which customers arrive according to a renewal process and are served in exponentially distributed times is the system GI/M/1. Observed just before successive arrivals, its queue length is a Markov chain on {0,1,2,… }\{0, 1, 2, \dots\}{0,1,2,…}, the imbedded chain introduced by D. G. Kendall (Kendall 1953, Ann. Math. Statist. 24, pp. 338–354). Whether this chain settles into a statistical equilibrium, keeps returning to the empty state without one, or drifts off to infinity is the first question asked about the queue, and every later quantity (stationary queue lengths, waiting-time distributions) presupposes the answer.

F. G. Foster's 1953 paper (Foster 1953) answers it for GI/M/1 and for M/G/1 by a different route from Kendall's direct analysis: it first proves general criteria, stated in terms of solutions of linear equations and inequalities in the transition matrix, for a countable Markov chain to be ergodic, recurrent or transient, and then checks them on the two queueing matrices. The criteria are of independent use; one of them (Theorem 2 of the paper) is now known as Foster's criterion, the starting point of the drift (Lyapunov-function) method for stability of Markov chains and queueing networks.

Timeline. Kendall (1951, J. Roy. Statist. Soc. B 13) studied queue-length processes directly, including a recurrence argument for M/G/1 that Foster's §3 reproduces; Kendall (1953) introduced the imbedded-chain method and, for GI/M/1, proved by it that ρ<1\rho < 1ρ<1 is sufficient for ergodicity (Foster 1953, p. 359); Foster (1953) proved the full classification, ergodic iff ρ<1\rho < 1ρ<1 and recurrent iff ρ≤1\rho \le 1ρ≤1, by the general criteria. This mission treats the GI/M/1 half; a companion mission treats M/G/1.

Setting

A transition matrix on the states {0,1,2,… }\{0, 1, 2, \dots\}{0,1,2,…} is an array [pij][p_{ij}][pij​] of nonnegative reals whose rows sum to 111. For a state jjj, fjjf_{jj}fjj​ is the probability that the chain started at jjj returns to jjj at some later step. The chain is recurrent if fjj=1f_{jj} = 1fjj​=1 for every jjj, transient if fjj<1f_{jj} < 1fjj​<1 for every jjj, and ergodic (recurrent-nonnull, positive recurrent) if moreover every mean recurrence time ∑nnfjj(n)\sum_n n f^{(n)}_{jj}∑n​nfjj(n)​ is finite. Foster's general theorems concern an irreducible chain (every state reachable from every state), assumed aperiodic for simplicity.

The GI/M/1 chain is described by a sequence a=(an)n≥0a = (a_n)_{n \ge 0}a=(an​)n≥0​ of positive numbers with ∑nan=1\sum_n a_n = 1∑n​an​=1: ana_nan​ is the probability that exactly nnn services are completed between two arrivals. With the tails αi=∑j≥i+1aj\alpha_i = \sum_{j \ge i+1} a_jαi​=∑j≥i+1​aj​,

[pij]=[α0a000⋯α1a1a00⋯α2a2a1a0⋯⋮⋮⋮⋮],[p_{ij}] = \begin{bmatrix} \alpha_0 & a_0 & 0 & 0 & \cdots \\ \alpha_1 & a_1 & a_0 & 0 & \cdots \\ \alpha_2 & a_2 & a_1 & a_0 & \cdots \\ \vdots & \vdots & \vdots & \vdots & \end{bmatrix},[pij​]=​α0​α1​α2​⋮​a0​a1​a2​⋮​0a0​a1​⋮​00a0​⋮​⋯⋯⋯​​,

that is pi0=αip_{i0} = \alpha_ipi0​=αi​, pij=ai+1−jp_{ij} = a_{i+1-j}pij​=ai+1−j​ for 1≤j≤i+11 \le j \le i+11≤j≤i+1, and pij=0p_{ij} = 0pij​=0 for j>i+1j > i+1j>i+1. In Lean this matrix is gim1Matrix a. The traffic parameter ρ\rhoρ is defined through its inverse,

ρ−1=∑n=1∞n an∈(0,∞],\rho^{-1} = \sum_{n=1}^{\infty} n\, a_n \in (0, \infty],ρ−1=n=1∑∞​nan​∈(0,∞],

the mean number of service completions per interarrival interval (rhoInv a, and rho a =ρ= \rho=ρ).

Formalization targets

Goal: the classification of GI/M/1 (§4, p. 359)

the chain is ergodic  ⟺  ρ<1,the chain is recurrent  ⟺  ρ≤1.\text{the chain is ergodic} \iff \rho < 1, \qquad \text{the chain is recurrent} \iff \rho \le 1 .the chain is ergodic⟺ρ<1,the chain is recurrent⟺ρ≤1.

Together: ergodic for ρ<1\rho < 1ρ<1, recurrent-null for ρ=1\rho = 1ρ=1, transient for ρ>1\rho > 1ρ>1. The statement carries no constants and leaves the sequence aaa free apart from positivity and normalization.

Milestones

  1. Theorem 7 (p. 358): for a probability distribution {pn}\{p_n\}{pn​} with p0>0p_0 > 0p0​>0, the equation ∑n≥0znpn=z\sum_{n \ge 0} z^n p_n = z∑n≥0​znpn​=z has a root in (0,1)(0, 1)(0,1) iff ∑n≥1npn>1\sum_{n\ge1} n p_n > 1∑n≥1​npn​>1.
  2. Theorem 1, sufficiency (p. 355): a nonnull solution of ∑ixipij=xj\sum_i x_i p_{ij} = x_j∑i​xi​pij​=xj​ with ∑i∣xi∣<∞\sum_i |x_i| < \infty∑i​∣xi​∣<∞ makes the system ergodic.
  3. Theorem 1, necessity (p. 355): in an ergodic system every nonnegative solution of ∑ixipij≤xj\sum_i x_i p_{ij} \le x_j∑i​xi​pij​≤xj​ has ∑ixi<∞\sum_i x_i < \infty∑i​xi​<∞.
  4. Theorem 4 (pp. 356–357): the system is transient iff ∑jpijyj=yi\sum_j p_{ij} y_j = y_i∑j​pij​yj​=yi​ (i≠0i \ne 0i=0) has a bounded nonconstant solution.

Milestones 2–4 are stated for a general irreducible aperiodic chain.

Significance

The classification tells exactly when the GI/M/1 queue is stable: the stationary distribution of the imbedded chain, which is geometric, exists precisely in the ergodic case ρ<1\rho < 1ρ<1, and for ρ>1\rho > 1ρ>1 the queue grows without bound. Theorems 1 and 4 are general tools, reusable for any countable chain: Theorem 1 characterizes ergodicity by summable invariant vectors, Theorem 4 characterizes transience by bounded harmonic functions off one state. Theorem 7 is the extinction criterion of branching processes and recurs throughout applied probability.

All of these results are proved in the literature (Foster 1953; Feller's textbook for Theorem 7 and a version of Theorem 4). As far as a search of the platform shows, none of them has a machine-checked proof; the platform holds related special cases for the G/M/1 queue with a specific interarrival law (QueueingFundamentals.GM1.unique_root_unit_interval, open), but not the general lemma or the classification. A formalization would provide the general criteria as reusable library results and the first verified stability classification of a non-Markovian queue's imbedded chain.

Difficulty

The matrix is explicit, but none of the three properties is a finite computation: ergodicity and recurrence are statements about return times over all horizons, so each direction must go through an existence or nonexistence statement about infinite systems of equations. For the converse directions the obvious argument fails: exhibiting a candidate solution such as xi≡1x_i \equiv 1xi​≡1 shows nothing until it is known that ergodicity forces every such solution to be summable, and showing that no bounded nonconstant solution of (7) exists when ρ<1\rho < 1ρ<1 requires control of all solutions, not of one. The general criteria themselves rest on limit theorems for pij(n)p_{ij}^{(n)}pij(n)​ and on interchanging infinite sums, and the infinite-mean case ∑nan=∞\sum n a_n = \infty∑nan​=∞ has to be carried along everywhere.

Formalization scope

  • The Markov-chain vocabulary is the published definition QueueingFundamentals_Foundations_MarkovChain: TransitionMatrix (entries p, nonnegativity, rows summing to 111 via HasSum), returnProb, meanRecurrenceTime, Irreducible, Aperiodic, PositiveRecurrent. "Ergodic" is PositiveRecurrent. IsRecurrent and IsTransient are defined state by state from returnProb; their complementarity for irreducible chains is a theorem, not a definition.
  • States are indexed from 000, as in the paper. The goal quantifies over every TransitionMatrix whose entries equal gim1Matrix a; such a matrix exists for every admissible aaa (rows sum to 111), so the statement is not vacuous.
  • ρ−1\rho^{-1}ρ−1 and ρ\rhoρ live in [0,∞][0, \infty][0,∞] (ℝ≥0∞), with ∞−1=0\infty^{-1} = 0∞−1=0: an infinite mean gives ρ=0\rho = 0ρ=0, and that chain is ergodic.
  • The goal does not assume irreducibility or aperiodicity: they follow from an>0a_n > 0an​>0. Milestones 2–4 carry them, as the paper's standing assumptions (§1).
  • Every infinite series appearing in a hypothesis is required to converge (HasSum or Summable), so that a divergent series cannot satisfy an equation or inequality vacuously. In Theorem 1's sufficiency half the xix_ixi​ may be of either sign. In Theorem 7 the distribution is renamed qqq to avoid a clash with pijp_{ij}pij​.
  • Ruled out as trivializing: defining ρ\rhoρ by a real inverse of a real series, defining "ergodic" as the existence of a summable invariant vector (which is Theorem 1's condition), or stating the goal over a matrix that need not exist.
  • Not included: the paper's explicit description of the solutions of (7) for ρ≥1\rho \ge 1ρ≥1 via the generating function (1−z){A(z)−z}−1(1 - z)\{A(z) - z\}^{-1}(1−z){A(z)−z}−1, and the M/G/1 half (Theorems 2, 3, 5), which is the companion mission. Contributions welcome: proofs of the general criteria (reusable for any countable chain), of Theorem 7, and lemmas on the GI/M/1 matrix such as irreducibility and aperiodicity.

Selected references

  • F. G. Foster, On the stochastic matrices associated with certain queuing processes, Ann. Math. Statist. 24 (1953), 355–360. https://doi.org/10.1214/aoms/1177728976
  • D. G. Kendall, Stochastic processes occurring in the theory of queues and their analysis by the method of the imbedded Markov chain, Ann. Math. Statist. 24 (1953), 338–354 (the paper immediately preceding Foster's in the same issue).
  • D. G. Kendall, Some problems in the theory of queues, J. Roy. Statist. Soc. B 13 (1951), 151–185.
  • W. Feller, An Introduction to Probability Theory and Its Applications, Vol. 1, Wiley, 1950.
8 thms1 active userReviewed
Control TheoryDynamic ProgrammingOperations Research+1·Captain: mikedeng1

Stochastic Optimal Control: The Discrete-Time Case VIII: The Infinite-Horizon Borel Models — the Optimality Equation J* = T(J*) under (P), (N), (D)Textbook

Motivation

Infinite horizon dynamic programming asks for the least expected cost of controlling a stochastic system forever, and for a policy attaining it. On finite or countable state spaces the theory has been classical since Bellman, Blackwell (1965) and Strauch (1966). Many models in operations research, inventory control, queueing and economics have continuous states and controls, however, and there the Bellman equation raises a question the countable theory never meets: the optimal cost need not be Borel-measurable, so its expectation under the transition law, which the equation requires, may not be defined.

Bertsekas and Shreve (1978) settled this question by working with lower semianalytic cost functions and universally measurable policies. In that framework the optimal cost is always measurable enough to be integrated, and the optimality equation holds with no continuity or compactness assumption. Chapter 9 of their book treats the infinite horizon model under the three classical cost structures: nonnegative costs (P), nonpositive costs (N), and bounded discounted costs (D).

Timeline:

  • 1965: Blackwell, discounted dynamic programming on Borel spaces with Borel-measurable data, case (D).
  • 1966: Strauch, negative dynamic programming, case (N), with Borel-measurable data.
  • 1978: Bertsekas and Shreve, Chapter 9: lower semianalytic costs and universally measurable policies, in all three cases (P), (N), (D).
  • 1979: Shreve and Bertsekas, the journal account of universally measurable policies.

Setting

An infinite horizon stochastic optimal control model (SM) is an eight-tuple (S,C,U,W,p,f,α,g)(S, C, U, W, p, f, \alpha, g)(S,C,U,W,p,f,α,g). The state space SSS, the control space CCC and the disturbance space WWW are nonempty Borel spaces, that is, spaces homeomorphic to Borel subsets of complete separable metric spaces. The control constraint UUU assigns to each state xxx a nonempty set U(x)⊆CU(x) \subseteq CU(x)⊆C, and the set Γ={(x,u)∣u∈U(x)}\Gamma = \{(x,u) \mid u \in U(x)\}Γ={(x,u)∣u∈U(x)} is analytic. The disturbance kernel p(dw∣x,u)p(dw \mid x, u)p(dw∣x,u) is a Borel stochastic kernel and the system function f:SCW→Sf : SCW \to Sf:SCW→S is Borel. The discount factor is α>0\alpha > 0α>0, and the one-stage cost g:Γ→[−∞,∞]g : \Gamma \to [-\infty, \infty]g:Γ→[−∞,∞] is lower semianalytic: each sublevel set {g<c}\{g < c\}{g<c} is analytic. The state moves by xk+1=f(xk,uk,wk)x_{k+1} = f(x_k, u_k, w_k)xk+1​=f(xk​,uk​,wk​), with transition kernel t(B∣x,u)=p({w∣f(x,u,w)∈B}∣x,u)t(B \mid x, u) = p(\{w \mid f(x,u,w) \in B\} \mid x, u)t(B∣x,u)=p({w∣f(x,u,w)∈B}∣x,u).

A policy π=(μ0,μ1,… )\pi = (\mu_0, \mu_1, \dots)π=(μ0​,μ1​,…) chooses uku_kuk​ at random from a universally measurable stochastic kernel μk(duk∣x0,u0,…,xk)\mu_k(du_k \mid x_0, u_0, \dots, x_k)μk​(duk​∣x0​,u0​,…,xk​) concentrated on U(xk)U(x_k)U(xk​); Π′\Pi'Π′ is the set of all policies. A policy is Markov if each μk\mu_kμk​ depends only on xkx_kxk​, and it is stationary if, moreover, μk=μ\mu_k = \muμk​=μ for all kkk. Writing qk(π,px)q_k(\pi, p_x)qk​(π,px​) for the law of (xk,uk)(x_k, u_k)(xk​,uk​) started from x0=xx_0 = xx0​=x, the cost of π\piπ and the optimal cost are

Jπ(x)=∑k=0∞αk∫g dqk(π,px),J∗(x)=inf⁡π∈Π′Jπ(x).J_\pi(x) = \sum_{k=0}^\infty \alpha^k \int g\, dq_k(\pi, p_x), \qquad J^*(x) = \inf_{\pi \in \Pi'} J_\pi(x).Jπ​(x)=k=0∑∞​αk∫gdqk​(π,px​),J∗(x)=π∈Π′inf​Jπ​(x).

For J:S→[−∞,∞]J : S \to [-\infty, \infty]J:S→[−∞,∞], the dynamic programming operators are

T(J)(x)=inf⁡u∈U(x){g(x,u)+α∫SJ(x′) t(dx′∣x,u)},Tμ(J)(x)=∫C[g(x,u)+α∫SJ dt]μ(du∣x).T(J)(x) = \inf_{u \in U(x)} \Big\{ g(x,u) + \alpha \int_S J(x')\, t(dx' \mid x, u) \Big\}, \qquad T_\mu(J)(x) = \int_C \Big[ g(x,u) + \alpha \int_S J\, dt \Big] \mu(du \mid x).T(J)(x)=u∈U(x)inf​{g(x,u)+α∫S​J(x′)t(dx′∣x,u)},Tμ​(J)(x)=∫C​[g(x,u)+α∫S​Jdt]μ(du∣x).

The three cases are (P) g≥0g \ge 0g≥0 on Γ\GammaΓ; (N) g≤0g \le 0g≤0 on Γ\GammaΓ; (D) α<1\alpha < 1α<1 and ∣g∣≤b|g| \le b∣g∣≤b on Γ\GammaΓ for some real bbb.

Formalization targets

Goal: the optimality equation (Proposition 9.8, Eq. (22))

Under each of (P), (N) and (D),

J∗=T(J∗).J^* = T(J^*).J∗=T(J∗).

Milestones

  • J∗J^*J∗ is lower semianalytic (Corollary 9.4.1).
  • For a stationary policy, Jμ=Tμ(Jμ)J_\mu = T_\mu(J_\mu)Jμ​=Tμ​(Jμ​) (Proposition 9.9).
  • Optimality tests for stationary policies: under (P) or (D), (μ,μ,… )(\mu, \mu, \dots)(μ,μ,…) is optimal iff J∗=Tμ(J∗)J^* = T_\mu(J^*)J∗=Tμ​(J∗) (Proposition 9.12); under (N) or (D), iff Jμ=T(Jμ)J_\mu = T(J_\mu)Jμ​=T(Jμ​) (Proposition 9.13).
  • Under (N) or (D), value iteration from 000 converges to J∗J^*J∗, and under (D) it converges uniformly from every bounded lower semianalytic start (Proposition 9.14).

Further statements of the mission

  • Markov policies suffice: at each state some Markov policy matches any policy's cost (Proposition 9.1), so J∗=inf⁡π∈ΠJπJ^* = \inf_{\pi \in \Pi} J_\piJ∗=infπ∈Π​Jπ​ (Corollary 9.1.1).
  • Partial converses of the optimality equation: J≥T(J)J \ge T(J)J≥T(J), J≥0J \ge 0J≥0 gives J≥J∗J \ge J^*J≥J∗ under (P); J≤T(J)J \le T(J)J≤T(J), J≤0J \le 0J≤0 gives J≤J∗J \le J^*J≤J∗ under (N); a bounded solution of J=T(J)J = T(J)J=T(J) equals J∗J^*J∗ under (D) (Proposition 9.10). The analogous statements for TμT_\muTμ​ and JμJ_\muJμ​ (Proposition 9.11).

Significance

The optimality equation is the basic structural fact of infinite horizon control. Corollary 9.12.1 uses it to construct optimal stationary policies from minimizers in the equation. The existence results for ε\varepsilonε-optimal policies (Propositions 9.19 and 9.20), the convergence analysis of value iteration in Section 9.5, and the reduction of imperfect state information problems in Chapter 10 all build on it. It holds for arbitrary Borel models, with no continuity or compactness assumption.

All results of the mission were proved in 1978. None of them has a machine-checked proof: Mathlib has stochastic kernels and the Ionescu-Tulcea construction for measurable kernels, but no theory of lower semianalytic functions, universally measurable kernels, or dynamic programming on Borel spaces. A formal development would fix the measurability bookkeeping on which the textbook proofs rest and supply a reusable substrate for the stochastic control papers that cite this book.

Difficulty

Under (D), TTT is a contraction on bounded functions, and its fixed point is the limit of value iteration. That argument, however, gives a fixed point only within a fixed class of measurable functions. Showing that this fixed point equals J∗J^*J∗ requires knowing that J∗J^*J∗ belongs to the class and that history-dependent randomized policies do no better. Under (P), value iteration can converge to the wrong limit (Example 1 of the chapter: lim⁡kJk(0)=0\lim_k J_k(0) = 0limk​Jk​(0)=0 while J∗(0)=∞J^*(0) = \inftyJ∗(0)=∞). Even when each JkJ_kJk​ is Borel, J∗J^*J∗ may fail to be (Example 2). So J∗=T(J∗)J^* = T(J^*)J∗=T(J∗) cannot be obtained as a limit of the finite horizon equations, and the natural class of Borel functions is not closed under the partial minimization that defines TTT.

The book's route lifts (SM) to a deterministic model on the space of probability measures P(S)P(S)P(S), where no measurability restriction is needed, and transfers the results back. Making this transfer rigorous requires that the cost of a randomized policy be a measurable functional of its law, and that the infimum over policies preserve lower semianalyticity.

Formalization scope

The draft fixes the following conventions.

  • Spaces. Borel spaces are topological spaces homeomorphic to Borel subsets of complete separable metric spaces, carrying their Borel σ\sigmaσ-algebras. Analytic sets are Mathlib's AnalyticSet. A set is universally measurable if it is null-measurable for every probability measure.
  • Extended reals. Values lie in EReal. The book's convention ∞−∞=−∞+∞=∞\infty - \infty = -\infty + \infty = \infty∞−∞=−∞+∞=∞ is implemented explicitly, because Mathlib's EReal sets ⊥+⊤=⊥\bot + \top = \bot⊥+⊤=⊥. The integral of an extended-real function is ∫f+−∫f−\int f^+ - \int f^-∫f+−∫f− with the same convention.
  • Policies. These are sequences of universally measurable stochastic kernels on the history spaces S0C0⋯SkS_0C_0 \cdots S_kS0​C0​⋯Sk​, charging U(xk)U(x_k)U(xk​) with mass one. The laws of (x0,u0,…,xk,uk)(x_0, u_0, \dots, x_k, u_k)(x0​,u0​,…,xk​,uk​) are built recursively from the kernels.
  • Costs. JπJ_\piJπ​ is the series ∑kαk∫g dqk\sum_k \alpha^k \int g\, dq_k∑k​αk∫gdqk​, computed as the difference of the series of positive and negative parts. Under each of (P), (N), (D) it coincides with the integral of the total discounted cost. J∗J^*J∗ is the infimum over all policies.
  • Case labels. Each statement carries the case labels the book attaches to it, as hypotheses on the model.
  • Scope. Only the (SM) statements are formalized. The deterministic model (DM) on P(S)P(S)P(S) is the book's proof device and enters no statement.

The goal admits a trivializing formalization that this draft rules out. J∗J^*J∗ is not defined as a fixed point of TTT, nor as the limit of Tk(0)T^k(0)Tk(0); it is the infimum of the costs of all policies, which under (P) can differ from that limit.

A complete development needs universally measurable kernels and their compositions on product spaces, measurability of x↦∫f(x,y) q(dy∣x)x \mapsto \int f(x, y)\, q(dy \mid x)x↦∫f(x,y)q(dy∣x) for universally measurable integrands, the measurable selection theorem of Jankov and von Neumann, and the closure of lower semianalytic functions under partial infimum. These are the subject of the series' mission on Chapter 7, and they are reusable for any stochastic control model on Borel spaces. Contributions are welcome at any level: these foundations, the Markov reduction (Proposition 9.1), or the case-by-case arguments.

Selected references

  • D. P. Bertsekas and S. E. Shreve, Stochastic Optimal Control: The Discrete-Time Case, Academic Press, 1978; Athena Scientific reprint, 1996. Chapter 9. https://web.mit.edu/dimitrib/www/soc.html
  • D. Blackwell, Discounted dynamic programming, Annals of Mathematical Statistics 36 (1965), 226–235. https://doi.org/10.1214/aoms/1177700285
  • R. E. Strauch, Negative dynamic programming, Annals of Mathematical Statistics 37 (1966), 871–890. https://doi.org/10.1214/aoms/1177699369
  • S. E. Shreve and D. P. Bertsekas, Universally measurable policies in dynamic programming, Mathematics of Operations Research 4 (1979), 15–30. https://doi.org/10.1287/moor.4.1.15
15 thms1 active userReviewed
Control TheoryOperations ResearchProbability·Captain: mikedeng1

Dynamic Scheduling of a System with Two Parallel Servers in Heavy Traffic with Resource Pooling: The Threshold Policy Is Asymptotically OptimalResearch Paper

Motivation

Many service systems route several classes of work to servers with overlapping skills: call centers with cross-trained agents, manufacturing cells with flexible machines, computing clusters with heterogeneous processors. Choosing which server works on which class at each moment is a dynamic scheduling problem. Exact optimal policies are out of reach except in toy cases, so heavy-traffic theory replaces the queueing system by a Brownian control problem, solves that limit problem, and then asks for a policy in the original system whose performance converges to the Brownian optimum. This programme was proposed by Harrison (Harrison 1988), and the parallel server system studied here is the example Harrison used (Harrison, Ann. Appl. Probab. 1998) to show that the greedy static priority rule can be very inefficient.

Bell and Williams (2001) gave the first proof of asymptotic optimality of a continuous-review policy for this system, with renewal arrivals and general service times. Harrison (1998) had treated Poisson arrivals and deterministic service times with a discrete-review policy and a pathwise criterion. Harrison and López (Queueing Systems, 1999) identified the complete resource pooling condition for general parallel server systems. The threshold policy and the proof method of Bell and Williams were later extended to multiserver systems (Bell and Williams, Electron. J. Probab., 2005).

Setting

There are two job classes and two servers. Server 1 serves class 1 (activity 1); server 2 serves class 1 (activity 2) and class 2 (activity 3). A sequence of such systems is indexed by r→∞r\to\inftyr→∞. On a probability space, i.i.d. sequences uˇk(i)\check u_k(i)uˇk​(i) (k=1,2k=1,2k=1,2) and vˇj(i)\check v_j(i)vˇj​(i) (j=1,2,3j=1,2,3j=1,2,3), i≥1i\ge1i≥1, are fixed: strictly positive, mutually independent, with mean one and finite variances αk2,βj2\alpha_k^2,\beta_j^2αk2​,βj2​. In system rrr the interarrival times are ukr(i)=uˇk(i)/λkru_k^r(i)=\check u_k(i)/\lambda_k^rukr​(i)=uˇk​(i)/λkr​ and the service times are vjr(i)=vˇj(i)/μjrv_j^r(i)=\check v_j(i)/\mu_j^rvjr​(i)=vˇj​(i)/μjr​. The renewal processes Akr(t)A_k^r(t)Akr​(t) and Sjr(t)S_j^r(t)Sjr​(t) count arrivals and potential service completions.

A scheduling control policy is an allocation T=(T1,T2,T3)T=(T_1,T_2,T_3)T=(T1​,T2​,T3​), where Tj(t)T_j(t)Tj​(t) is the time devoted to activity jjj in [0,t][0,t][0,t]. Each Tj(t)T_j(t)Tj​(t) is a random variable, each TjT_jTj​ is continuous and nondecreasing from 000, and so are the idle times I1=t−T1I_1=t-T_1I1​=t−T1​ and I2=t−T2−T3I_2=t-T_2-T_3I2​=t−T2​−T3​. The queue lengths

Q1(t)=A1(t)−S1(T1(t))−S2(T2(t)),Q2(t)=A2(t)−S3(T3(t))Q_1(t)=A_1(t)-S_1(T_1(t))-S_2(T_2(t)),\qquad Q_2(t)=A_2(t)-S_3(T_3(t))Q1​(t)=A1​(t)−S1​(T1​(t))−S2​(T2​(t)),Q2​(t)=A2​(t)−S3​(T3​(t))

must be nonnegative. Policies may anticipate the future. The rates satisfy Assumption 3.1: λ1>μ1\lambda_1>\mu_1λ1​>μ1​, 1−(λ1−μ1)/μ2=λ2/μ31-(\lambda_1-\mu_1)/\mu_2=\lambda_2/\mu_31−(λ1​−μ1​)/μ2​=λ2​/μ3​, and the rates converge at rate 1/r1/r1/r to limits with second-order parameters θ1,θ2\theta_1,\theta_2θ1​,θ2​. Assumption 3.2 is h1μ2≥h2μ3h_1\mu_2\ge h_2\mu_3h1​μ2​≥h2​μ3​, and Assumption 3.3 gives finite exponential moments near 000. With Q^r(t)=r−1Qr(r2t)\hat Q^r(t)=r^{-1}Q^r(r^2t)Q^​r(t)=r−1Qr(r2t) the cost is

J^r(Tr)=E(∫0∞e−γt h⋅Q^r(t) dt).\hat J^r(T^r)=\mathbf E\Big(\int_0^\infty e^{-\gamma t}\,h\cdot\hat Q^r(t)\,dt\Big).J^r(Tr)=E(∫0∞​e−γth⋅Q^​r(t)dt).

The threshold policy with Lr=[clog⁡r]L^r=[c\log r]Lr=[clogr] works as follows. Server 1 works whenever it has a class 1 job available. Server 2 serves class 1 with preemptive-resume priority when more than LrL^rLr class 1 jobs are present, and otherwise serves class 2. The Brownian benchmark is built from a two-dimensional Brownian motion X~\tilde XX~ with drift θ\thetaθ and diagonal covariance, from y=(1,μ2/μ3)y=(1,\mu_2/\mu_3)y=(1,μ2​/μ3​), and from the reflected process W~∗=y⋅X~+V~∗\tilde W^*=y\cdot\tilde X+\tilde V^*W~∗=y⋅X~+V~∗ with V~∗(t)=−inf⁡s≤ty⋅X~(s)\tilde V^*(t)=-\inf_{s\le t}y\cdot\tilde X(s)V~∗(t)=−infs≤t​y⋅X~(s). Its cost is J∗=E∫0∞e−γth2 W~∗(t)/y2 dtJ^*=\mathbf E\int_0^\infty e^{-\gamma t}h_2\,\tilde W^*(t)/y_2\,dtJ∗=E∫0∞​e−γth2​W~∗(t)/y2​dt.

Formalization targets

Goal: Theorem 5.3

For ccc larger than a constant c0c_0c0​ that depends only on the model data, and for every sequence {Tr}\{T^r\}{Tr} of scheduling control policies,

lim inf⁡r→∞J^r(Tr) ≥ J∗ = lim⁡r→∞J^r(Tr,∗),J∗<∞.\liminf_{r\to\infty}\hat J^r(T^r)\ \ge\ J^*\ =\ \lim_{r\to\infty}\hat J^r(T^{r,*}),\qquad J^*<\infty .r→∞liminf​J^r(Tr) ≥ J∗ = r→∞lim​J^r(Tr,∗),J∗<∞.

Milestones

  • Proposition B.1: the one-dimensional Skorokhod problem, its explicit solution and its minimality.
  • Appendix A, (181) and (184): Cramér-type deviation bounds for delayed renewal processes.
  • Theorem 7.2: after first reaching LrL^rLr, the class 1 queue stays within Lr−1L^r-1Lr−1 of the threshold, with probability tending to one.
  • Theorem 7.1: (Q^1r,I^1r)⇒(0,0)(\hat Q_1^r,\hat I_1^r)\Rightarrow(0,0)(Q^​1r​,I^1r​)⇒(0,0) under the threshold policy.
  • Lemma 8.1: the fluid-scaled threshold allocations converge to Tˉ∗(t)=(t,λ1−μ1μ2t,λ2μ3t)\bar T^*(t)=(t,\frac{\lambda_1-\mu_1}{\mu_2}t,\frac{\lambda_2}{\mu_3}t)Tˉ∗(t)=(t,μ2​λ1​−μ1​​t,μ3​λ2​​t).
  • Theorem 5.2 (state-space collapse): (Q^1r,Q^2r,I^1r,I^2r)⇒(0,Q~2∗,0,I~2∗)(\hat Q_1^r,\hat Q_2^r,\hat I_1^r,\hat I_2^r)\Rightarrow(0,\tilde Q_2^*,0,\tilde I_2^*)(Q^​1r​,Q^​2r​,I^1r​,I^2r​)⇒(0,Q~​2∗​,0,I~2∗​).
  • Lemma 9.3: along a subsequence achieving a finite lim inf⁡\liminfliminf cost, the fluid-scaled processes converge to (0,λt,μt,Tˉ∗,0)(0,\lambda t,\mu t,\bar T^*,0)(0,λt,μt,Tˉ∗,0).

A further draft theorem states that Definition 5.1 determines an admissible allocation, unique pathwise, whenever Lr≥1L^r\ge1Lr≥1.

Significance

The theorem proves that a simple state-dependent rule, which sends server 2 to class 1 only when the class 1 queue exceeds a logarithmic safety stock, is asymptotically optimal among all policies, including those that anticipate the future. The limiting cost is the explicit optimum of the Brownian control problem. The proof gives a template for heavy-traffic asymptotic optimality under complete resource pooling: a lower bound valid for every policy, and state-space collapse under the proposed policy. The residual process analysis of Section 7 shows how a threshold of order log⁡r\log rlogr makes starvation of server 1 negligible on the diffusion time scale.

The paper's results are proved but not machine-checked; no formal proof exists in any proof assistant. The mission asks for formal statements of the paper's main theorem and its supporting lemmas, followed by formal proofs. Parts of the development are independent of the paper: the one-dimensional Skorokhod map, renewal large deviation bounds, and convergence encodings on path space.

Difficulty

The lower bound must hold for arbitrary, possibly anticipating, policies, so no Markov structure is available. The argument has to pass through fluid limits of an arbitrary cost-minimizing subsequence and a pathwise minimality property, and Fatou's lemma for the limit needs uniform control. For the upper bound, the obvious approach, a static priority rule, is known to fail: it starves server 1 and produces a large class 1 queue. With a threshold policy, the hard step is to show that the class 1 queue, once at the threshold, rarely moves Lr−1L^r-1Lr−1 away from it over a time interval of length r2tr^2tr2t. That requires large deviation estimates for renewal processes started at random, multiparameter stopping times. Showing that J^r(Tr,∗)\hat J^r(T^{r,*})J^r(Tr,∗) converges to J∗J^*J∗, rather than only that the processes converge in distribution, also requires uniform integrability of the scaled queue lengths.

Formalization scope

Classes and activities are indexed by Fin 2 and Fin 3. The i.i.d. sequences keep the paper's index base i≥1i\ge1i≥1, and the systems are indexed by n∈Nn\in\mathbb Nn∈N with r=rn∈[1,∞)r=r_n\in[1,\infty)r=rn​∈[1,∞), rn→∞r_n\to\inftyrn​→∞. Time is real, and every condition is imposed for t≥0t\ge0t≥0. Admissibility is exactly (11)–(14). Measurability in (11) is with respect to the completion of P\mathbf PP, since the paper's space is complete. Finiteness of the renewal processes everywhere on Ω\OmegaΩ, which the paper obtains by discarding a null set, is a hypothesis. Queue lengths are real, costs are lower Lebesgue integrals in [0,∞][0,\infty][0,∞], counting processes take values in N∪{∞}\mathbb N\cup\{\infty\}N∪{∞}, and Λ\LambdaΛ, Λ∗\Lambda^*Λ∗ take values in the extended reals.

The constant c0c_0c0​ is existential and is chosen after the model data and before ccc, the policies and the Brownian motions. The threshold relations are required only for the systems with Lr≥1L^r\ge1Lr≥1, which are all but finitely many. Each convergence to a deterministic limit (Theorem 7.1, Lemmas 8.1 and 9.3) is stated as u.o.c. convergence in probability, the paper's own equivalence (p. 633). Theorem 5.2 is stated in coupling form: there are copies of the processes on one probability space, with Skorokhod paths and the same laws, that converge almost surely uniformly on compacts. This is equivalent to weak convergence in D4\mathbf D^4D4 to a limit with continuous paths. J∗J^*J∗ is defined by (44) from an arbitrary pair of independent standard Brownian motions (Mathlib's IsBrownianReal), not by a closed form.

Two formalizations would make the goal trivial, and both are excluded. Leaving out the requirement that Tr,∗T^{r,*}Tr,∗ actually follow the policy would make the goal false or empty. Narrowing the class of competing policies, for example to non-anticipating ones, would weaken the theorem. A draft theorem also states that the threshold allocation exists and is unique pathwise, so the hypothesis on Tr,∗T^{r,*}Tr,∗ can be satisfied.

The development needs renewal theory (functional central limit theorems, Cramér bounds), multiparameter stopping times, tightness in D\mathbf DD, the Skorokhod representation theorem, the reflection map, and properties of reflected Brownian motion. Contributions are welcome at every level: proofs of milestones, reusable lemmas on renewal processes and the Skorokhod map, and further lemmas of the paper (Lemmas 7.5, 7.6 and 9.2 are not yet stated).

Selected references

  • S. L. Bell and R. J. Williams, Dynamic scheduling of a system with two parallel servers in heavy traffic with resource pooling: asymptotic optimality of a threshold policy, Ann. Appl. Probab. 11 (2001) 608–649. https://doi.org/10.1214/aoap/1015345343
  • J. M. Harrison, Heavy traffic analysis of a system with parallel servers: asymptotic optimality of discrete-review policies, Ann. Appl. Probab. 8 (1998) 822–848.
  • J. M. Harrison and M. J. López, Heavy traffic resource pooling in parallel-server systems, Queueing Systems 33 (1999) 339–368.
  • J. M. Harrison, Brownian models of queueing networks with heterogeneous customer populations, in Stochastic Differential Systems, Stochastic Control Theory and Their Applications, Springer (1988) 147–186.
  • S. L. Bell and R. J. Williams, Dynamic scheduling of a parallel server system in heavy traffic with complete resource pooling: asymptotic optimality of a threshold policy, Electron. J. Probab. 10 (2005) 1044–1115.
  • J. M. Harrison, Brownian Motion and Stochastic Flow Systems, Wiley (1985).
14 thms1 active userReviewed
Control TheoryMathematical PhysicsOptimization+2·Captain: mikedeng1

Optimization of Mean-field Spin Glasses III: The Lagrangian Value of the Stochastic Control Problem Equals the Parisi FunctionalResearch Paper

Motivation

The ground-state energy of a mixed ppp-spin spin glass, OPTN=max⁡σ∈{±1}NHN(σ)/N\mathsf{OPT}_N=\max_{\sigma\in\{\pm1\}^N}H_N(\sigma)/NOPTN​=maxσ∈{±1}N​HN​(σ)/N, converges almost surely to the infimum of the Parisi functional over non-decreasing order parameters (Auffinger–Chen 2017). El Alaoui, Montanari and Sellke (arXiv:2001.00904v1) study algorithms that find near-optimal configurations. They introduce incremental approximate message passing (IAMP) and show that, among a broad class of such algorithms, the best achievable energy is the infimum of the Parisi functional over a larger space of order parameters (their Theorem 4).

The upper bound in Theorem 4 is reduced, in Section 4 of the paper, to a stochastic optimal control problem. The energy reached by a message-passing algorithm becomes the objective of a control problem driven by a Brownian motion, with a terminal constraint and a variance constraint. The variance constraint is removed by a Lagrange multiplier 12ξ′′γ\tfrac12\xi''\gamma21​ξ′′γ. Proposition 4.1 states that the resulting Lagrangian value is exactly the Parisi functional P(γ)\mathsf P(\gamma)P(γ). This mission formalizes that duality and the verification argument behind it (Section 7).

Setting

Mixture. Real coefficients (ck)k≥2(c_k)_{k\ge2}(ck​)k≥2​ define the mixture ξ(t)=∑k≥2ck2tk\xi(t)=\sum_{k\ge2}c_k^2t^kξ(t)=∑k≥2​ck2​tk, with the standing assumption ξ(1+ε)<∞\xi(1+\varepsilon)<\inftyξ(1+ε)<∞ for some ε>0\varepsilon>0ε>0. Its derivatives ξ′\xi'ξ′, ξ′′\xi''ξ′′ are nonnegative and nondecreasing on [0,1][0,1][0,1].

Order parameters. SF+\mathsf{SF}_+SF+​ is the set of nonnegative step functions

γ=∑i=1mγi I[ti−1,ti),0=t0<t1<⋯<tm=1, γi≥0.\gamma=\sum_{i=1}^m\gamma_i\,\mathbb I_{[t_{i-1},t_i)},\qquad 0=t_0<t_1<\dots<t_m=1,\ \gamma_i\ge0 .γ=i=1∑m​γi​I[ti−1​,ti​)​,0=t0​<t1​<⋯<tm​=1, γi​≥0.

Put ν(t)=∫t1ξ′′(s)γ(s) ds\nu(t)=\int_t^1\xi''(s)\gamma(s)\,dsν(t)=∫t1​ξ′′(s)γ(s)ds.

Parisi PDE and functional. Φγ:[0,1]×R→R\Phi_\gamma:[0,1]\times\mathbb R\to\mathbb RΦγ​:[0,1]×R→R solves

∂tΦγ+12ξ′′(t)(∂x2Φγ+γ(t)(∂xΦγ)2)=0,Φγ(1,x)=∣x∣.\partial_t\Phi_\gamma+\tfrac12\xi''(t)\big(\partial_x^2\Phi_\gamma+\gamma(t)(\partial_x\Phi_\gamma)^2\big)=0,\qquad\Phi_\gamma(1,x)=|x| .∂t​Φγ​+21​ξ′′(t)(∂x2​Φγ​+γ(t)(∂x​Φγ​)2)=0,Φγ​(1,x)=∣x∣.

For γ∈SF+\gamma\in\mathsf{SF}_+γ∈SF+​ it is given explicitly by the Cole–Hopf recursion: with r(t)=ξ′(1)−ξ′(t)r(t)=\xi'(1)-\xi'(t)r(t)=ξ′(1)−ξ′(t) and G∼N(0,1)G\sim\mathsf N(0,1)G∼N(0,1), for t∈[ti−1,ti)t\in[t_{i-1},t_i)t∈[ti−1​,ti​),

Φγ(t,x)=1γilog⁡Eexp⁡{γiΦγ(ti,x+r(t)−r(ti) G)}.\Phi_\gamma(t,x)=\frac1{\gamma_i}\log\mathbb E\exp\big\{\gamma_i\Phi_\gamma(t_i,x+\sqrt{r(t)-r(t_i)}\,G)\big\}.Φγ​(t,x)=γi​1​logEexp{γi​Φγ​(ti​,x+r(t)−r(ti​)​G)}.

The Parisi functional is P(γ)=Φγ(0,0)−12∫01t ξ′′(t)γ(t) dt\mathsf P(\gamma)=\Phi_\gamma(0,0)-\tfrac12\int_0^1t\,\xi''(t)\gamma(t)\,dtP(γ)=Φγ​(0,0)−21​∫01​tξ′′(t)γ(t)dt.

Control problem. Let BBB be a standard Brownian motion. A control u∈D[t,1]u\in D[t,1]u∈D[t,1] is a process on [t,1][t,1][t,1], progressively measurable for the filtration of (Br)r∈[t,1](B_r)_{r\in[t,1]}(Br​)r∈[t,1]​, with E∫t1ξ′′(s)us2 ds<∞\mathbb E\int_t^1\xi''(s)u_s^2\,ds<\inftyE∫t1​ξ′′(s)us2​ds<∞. The value is

Jγ(t,z)=sup⁡u∈D[t,1]E[∫t1ξ′′(s)us ds+12∫t1ν(s)(ξ′′(s)us2−1)ds]s.t.z+∫t1ξ′′(s) us dBs∈(−1,1) a.s.\mathcal J_\gamma(t,z)=\sup_{u\in D[t,1]}\mathbb E\Big[\int_t^1\xi''(s)u_s\,ds+\frac12\int_t^1\nu(s)\big(\xi''(s)u_s^2-1\big)ds\Big]\quad\text{s.t.}\quad z+\int_t^1\sqrt{\xi''(s)}\,u_s\,dB_s\in(-1,1)\ \text{a.s.}Jγ​(t,z)=u∈D[t,1]sup​E[∫t1​ξ′′(s)us​ds+21​∫t1​ν(s)(ξ′′(s)us2​−1)ds]s.t.z+∫t1​ξ′′(s)​us​dBs​∈(−1,1) a.s.

Candidate value function. With Φγ∗(t,z)=inf⁡x{Φγ(t,x)−xz}\Phi^*_\gamma(t,z)=\inf_x\{\Phi_\gamma(t,x)-xz\}Φγ∗​(t,z)=infx​{Φγ​(t,x)−xz},

V(t,z)=Φγ∗(t,z)−12ν(t)z2−12∫t1ν(s) ds.V(t,z)=\Phi^*_\gamma(t,z)-\tfrac12\nu(t)z^2-\tfrac12\int_t^1\nu(s)\,ds .V(t,z)=Φγ∗​(t,z)−21​ν(t)z2−21​∫t1​ν(s)ds.

Formalization targets

Goal: Proposition 4.1

Jγ(0,0)=P(γ)for every γ∈SF+.\mathcal J_\gamma(0,0)=\mathsf P(\gamma)\qquad\text{for every }\gamma\in\mathsf{SF}_+ .Jγ​(0,0)=P(γ)for every γ∈SF+​.

Milestones

  1. Lemma 7.2 (a)–(e): Φγ(t,⋅)\Phi_\gamma(t,\cdot)Φγ​(t,⋅) is smooth for t<1t<1t<1, with derivatives jointly continuous on [0,1)×R[0,1)\times\mathbb R[0,1)×R and C1C^1C1 in time where γ\gammaγ is constant. The range of ∂xΦγ(t,⋅)\partial_x\Phi_\gamma(t,\cdot)∂x​Φγ​(t,⋅) is (−1,1)(-1,1)(−1,1), it is strictly increasing, and 0<∂x2Φγ(t′,x)≤C(t,γ)0<\partial_x^2\Phi_\gamma(t',x)\le C(t,\gamma)0<∂x2​Φγ​(t′,x)≤C(t,γ) for t′≤tt'\le tt′≤t.
  2. Envelope identities (proof of Lemma 7.3): ∂zΦγ∗(t,z)=−xt∗(z)\partial_z\Phi^*_\gamma(t,z)=-x^*_t(z)∂z​Φγ∗​(t,z)=−xt∗​(z) and ∂z2Φγ∗(t,z)=−1/∂x2Φγ(t,xt∗(z))\partial_z^2\Phi^*_\gamma(t,z)=-1/\partial_x^2\Phi_\gamma(t,x^*_t(z))∂z2​Φγ∗​(t,z)=−1/∂x2​Φγ​(t,xt∗​(z)), where xt∗(z)x^*_t(z)xt∗​(z) is the unique root of ∂xΦγ(t,x)=z\partial_x\Phi_\gamma(t,x)=z∂x​Φγ​(t,x)=z.
  3. Lemma 7.3: VVV solves the HJB equation
∂tV+ξ′′(t)sup⁡λ∈R{λ+λ22(ν(t)+∂z2V)}−12ν(t)=0,V(1,z)=0.\partial_tV+\xi''(t)\sup_{\lambda\in\mathbb R}\Big\{\lambda+\frac{\lambda^2}{2}\big(\nu(t)+\partial_z^2V\big)\Big\}-\frac12\nu(t)=0,\qquad V(1,z)=0 .∂t​V+ξ′′(t)λ∈Rsup​{λ+2λ2​(ν(t)+∂z2​V)}−21​ν(t)=0,V(1,z)=0.
  1. Evaluation at the origin: V(0,0)=P(γ)V(0,0)=\mathsf P(\gamma)V(0,0)=P(γ).
  2. Proposition 7.1: Jγ(t,z)=V(t,z)\mathcal J_\gamma(t,z)=V(t,z)Jγ​(t,z)=V(t,z) for all (t,z)∈[0,1]×(−1,1)(t,z)\in[0,1]\times(-1,1)(t,z)∈[0,1]×(−1,1).

Proposition 7.1 at (0,0)(0,0)(0,0) together with milestone 4 gives the goal.

Significance

The result. By integration by parts (Eq. (4.4) of the paper), Jγ(0,0)\mathcal J_\gamma(0,0)Jγ​(0,0) bounds the value of the constrained control problem (4.2). That problem in turn bounds the asymptotic energy of every message-passing algorithm in the class of Theorem 4. Proposition 4.1 turns the bound into inf⁡γ∈SF+P(γ)\inf_{\gamma\in\mathsf{SF}_+}\mathsf P(\gamma)infγ∈SF+​​P(γ), which is the analytic core of the optimality statement for IAMP. It is also an instance of a broader principle: the Parisi functional has a stochastic-control representation (Jagannath–Tobasco 2016).

Formalizing it. The result is proved in the paper; no machine-checked version exists. A complete formalization needs a verification theorem for a control problem with a state constraint (M1∈(−1,1)M_1\in(-1,1)M1​∈(−1,1)), Itô's formula for a C1,2C^{1,2}C1,2 function that is only piecewise C1C^1C1 in time, and quantitative regularity of the Cole–Hopf solution. Each of these is reusable well beyond spin glasses.

Difficulty

The value function Jγ\mathcal J_\gammaJγ​ is not known to be smooth, and the dynamic-programming equation (4.6) is only heuristic. The proof therefore guesses a solution and verifies it. Two steps carry the difficulty.

First, the guess VVV is a Legendre transform. Its regularity, and the sign ν+∂z2V<0\nu+\partial_z^2V<0ν+∂z2​V<0 that makes the HJB supremum finite, rest on strict convexity and bounded curvature of Φγ(t,⋅)\Phi_\gamma(t,\cdot)Φγ​(t,⋅) (Lemma 7.2). These must be proved by induction through the Cole–Hopf recursion, including the steps with γi=0\gamma_i=0γi​=0.

Second, the verification argument applies Itô's formula to V(s,Msu)V(s,M^u_s)V(s,Msu​), where MuM^uMu is a martingale confined to (−1,1)(-1,1)(−1,1) and VVV is only C1C^1C1 in time between the jumps of γ\gammaγ. The boundary θ→1\theta\to1θ→1 needs a dominated-convergence argument, and attaining the supremum needs an explicit optimal feedback control built from an SDE.

Formalization scope

  • Mixture. ξ\xiξ is a coefficient sequence c:N→Rc:\mathbb N\to\mathbb Rc:N→R with c0=c1=0c_0=c_1=0c0​=c1​=0 imposed. ξ′\xi'ξ′ and ξ′′\xi''ξ′′ are explicit termwise series.
  • Step functions. SF+\mathsf{SF}_+SF+​ is represented by its data (breakpoints and values). γ\gammaγ is extended by 000 outside [0,1)[0,1)[0,1); its value at t=1t=1t=1 never matters.
  • Cole–Hopf. Φγ\Phi_\gammaΦγ​ is defined by the recursion. When γi=0\gamma_i=0γi​=0, the recursion uses its limit, the heat semigroup, instead of dividing by zero. Expectations over GGG are integrals against gaussianReal 0 1.
  • Derivatives. Space derivatives are deriv/iteratedDeriv. Time derivatives are right derivatives, because γ\gammaγ jumps.
  • Legendre transform. Φγ∗\Phi^*_\gammaΦγ∗​ is a real infimum, used only for ∣z∣<1|z|<1∣z∣<1, where it is bounded below.
  • Brownian motion and filtration. BBB is a Mathlib IsBrownianReal process on R≥0\mathbb R_{\ge0}R≥0​, with each BrB_rBr​ measurable. The filtration is Fst=σ(Br:t≤r≤s)\mathcal F^t_s=\sigma(B_r:t\le r\le s)Fst​=σ(Br​:t≤r≤s).
  • Stochastic integral. It is the L2L^2L2 Itô integral of the published definition Peng1990.SMP.IsItoIntegral (horizon 111), whose integrability class is exactly E∫ξ′′u2<∞\mathbb E\int\xi''u^2<\inftyE∫ξ′′u2<∞.
  • Supremum. Jγ(t,z)=v\mathcal J_\gamma(t,z)=vJγ​(t,z)=v is stated as "vvv is the least upper bound of the objective values of admissible controls" (IsLUB), never as a real sSup. A default value of an empty or unbounded supremum therefore cannot make a statement trivially true.
  • Disclosed hypothesis. Lemma 7.2, the envelope identities and Lemma 7.3 assume that ξ\xiξ is not identically zero (some ck≠0c_k\neq0ck​=0). For ξ≡0\xi\equiv0ξ≡0 one has Φγ(t,x)=∣x∣\Phi_\gamma(t,x)=|x|Φγ​(t,x)=∣x∣ for all ttt, and these statements fail. Proposition 7.1, the evaluation at the origin and the goal need no such hypothesis.
  • Lemma 7.3. The statement includes the inequality ν+∂z2V<0\nu+\partial_z^2V<0ν+∂z2​V<0, which the page proves. This rules out reading the HJB supremum as a default value.

The paper's algorithmic results (Theorems 2–4, Corollary 2.2) are out of scope. They need the AMP and state-evolution machinery of Section 5 and Appendix A, and an informal model of computation. The optional bound (4.4) is not stated.

Welcome contributions include the regularity of Cole–Hopf solutions (Gaussian convolution, log-moment-generating functions), a general verification theorem for one-dimensional controlled martingales with a terminal state constraint, and Itô's formula for C1,2C^{1,2}C1,2 functions.

Selected references

  • A. El Alaoui, A. Montanari, M. Sellke, Optimization of Mean-field Spin Glasses, arXiv:2001.00904v1, 2020. https://arxiv.org/abs/2001.00904
  • A. Auffinger, W.-K. Chen, Parisi formula for the ground state energy in the mixed p-spin model, Ann. Probab. 45(6b), 2017. https://arxiv.org/abs/1606.05335
  • A. Jagannath, I. Tobasco, A dynamic programming approach to the Parisi functional, Proc. AMS 144, 2016. https://arxiv.org/abs/1502.04398
  • N. Touzi, Optimal Stochastic Control, Stochastic Target Problems, and Backward SDE, Fields Institute Monographs 29, Springer, 2012 (cited as [Tou12] in the paper; the verification argument of Section 7 follows its Theorem 4.1).
14 thms1 active userReviewed
Probability·Captain: mikedeng1

Convergence in law of the minimum of a branching random walk: The Minimum Centred at (3/2) ln n Converges to a Gumbel Law Shifted by the Derivative MartingaleResearch Paper

Motivation

A branching random walk is the simplest model of a population that both reproduces and moves: every particle dies and leaves a random cloud of children displaced relative to it. Its extreme particles control the speed of travelling waves in reaction–diffusion equations (the KPP/Fisher equation), the free energy of directed polymers on trees, the cover and hitting times of random walks on trees, and the maxima of log-correlated fields such as the two-dimensional Gaussian free field. The basic quantity is the position of the leftmost particle at time nnn.

Timeline.

  • 1974–1976: Hammersley, Kingman and Biggins prove the law of large numbers Mn/n→γM_n/n\to\gammaMn​/n→γ for the minimum.
  • 1978–1983: Bramson shows that for branching Brownian motion the maximum, centred at 2 t−322ln⁡t\sqrt2\,t-\frac{3}{2\sqrt2}\ln t2​t−22​3​lnt, converges in law (Bramson 1983). Lalley and Sellke (1987, Ann. Probab. 15) identify the limit as a Gumbel law randomly shifted by the limit of the derivative martingale.
  • 2004: Biggins and Kyprianou prove that the derivative martingale of a branching random walk converges to a limit that is non-trivial in the boundary case (Adv. Appl. Probab. 36).
  • 2009: Hu and Shi (arXiv:math/0702799) and Addario-Berry and Reed (Ann. Probab. 37) find the logarithmic correction: Mn−32ln⁡nM_n-\frac32\ln nMn​−23​lnn is tight. Bramson and Zeitouni (2009) obtain tightness around the median under tail assumptions.
  • 2013: Aïdékon proves convergence in law of Mn−32ln⁡nM_n-\frac32\ln nMn​−23​lnn for general non-lattice branching random walks, the result this mission formalizes (arXiv:1101.1810, Ann. Probab. 41 (2013)).

Setting

Let L\mathcal LL be a point process on R\mathbb RR: a random, possibly infinite, collection of points. Start one particle at 000. At time 111 it dies and leaves children at the points of L\mathcal LL; each particle of generation nnn then dies and leaves children at the points of an independent copy of L\mathcal LL, translated to its own position. Vertices of the genealogical tree T\mathbb TT (a Galton–Watson tree) are labelled by finite words u=(i0,…,ik−1)u=(i_0,\dots,i_{k-1})u=(i0​,…,ik−1​); ∣u∣=k|u|=k∣u∣=k is the generation, uju_juj​ the ancestor at generation jjj, and V(u)V(u)V(u) the position.

The paper works in the boundary case

E[∑∣x∣=11]>1,E[∑∣x∣=1e−V(x)]=1,E[∑∣x∣=1V(x)e−V(x)]=0,(1.1)\mathbf E\Big[\sum_{|x|=1}1\Big]>1,\qquad \mathbf E\Big[\sum_{|x|=1}e^{-V(x)}\Big]=1,\qquad \mathbf E\Big[\sum_{|x|=1}V(x)e^{-V(x)}\Big]=0,\tag{1.1}E[∣x∣=1∑​1]>1,E[∣x∣=1∑​e−V(x)]=1,E[∣x∣=1∑​V(x)e−V(x)]=0,(1.1)

and assumes throughout that L\mathcal LL is non-lattice and that

E[∑∣x∣=1V(x)2e−V(x)]<∞,E[X(ln⁡+X)2]<∞,E[X~ln⁡+X~]<∞,(1.3–1.4)\mathbf E\Big[\sum_{|x|=1}V(x)^2e^{-V(x)}\Big]<\infty,\qquad \mathbf E\big[X(\ln_+X)^2\big]<\infty,\qquad \mathbf E\big[\tilde X\ln_+\tilde X\big]<\infty,\tag{1.3–1.4}E[∣x∣=1∑​V(x)2e−V(x)]<∞,E[X(ln+​X)2]<∞,E[X~ln+​X~]<∞,(1.3–1.4)

with X=∑∣x∣=1e−V(x)X=\sum_{|x|=1}e^{-V(x)}X=∑∣x∣=1​e−V(x) and X~=∑∣x∣=1V(x)+e−V(x)\tilde X=\sum_{|x|=1}V(x)_+e^{-V(x)}X~=∑∣x∣=1​V(x)+​e−V(x). The objects of the main theorem are the minimum Mn=min⁡{V(x):∣x∣=n}M_n=\min\{V(x):|x|=n\}Mn​=min{V(x):∣x∣=n} (with min⁡∅=+∞\min\varnothing=+\inftymin∅=+∞) and the derivative martingale

Dn=∑∣x∣=nV(x)e−V(x),D_n=\sum_{|x|=n}V(x)e^{-V(x)},Dn​=∣x∣=n∑​V(x)e−V(x),

which converges almost surely to a limit D∞≥0D_\infty\ge0D∞​≥0, strictly positive on non-extinction. A standard example: two children with i.i.d. normal displacements of mean and variance 2ln⁡22\ln22ln2.

Formalization targets

Goal: Theorem 1.1

There is a constant C∗∈(0,∞)C^*\in(0,\infty)C∗∈(0,∞) such that for every real xxx,

lim⁡n→∞P(Mn≥32ln⁡n+x)=E[e−C∗exD∞].\lim_{n\to\infty}\mathbf P\Big(M_n\ge\tfrac32\ln n+x\Big)=\mathbf E\Big[e^{-C^*e^xD_\infty}\Big].n→∞lim​P(Mn​≥23​lnn+x)=E[e−C∗exD∞​].

The constant is not specified numerically. It is the product C1c0C_1c_0C1​c0​ of the constants below.

Milestones, in the order the proof uses them

  1. Many-to-one lemma (2.1): Ea[∑∣x∣=ng(V(x1),…,V(xn))]=Ea[eSn−ag(S1,…,Sn)]\mathbf E_a[\sum_{|x|=n}g(V(x_1),\dots,V(x_n))]=\mathbf E_a[e^{S_n-a}g(S_1,\dots,S_n)]Ea​[∑∣x∣=n​g(V(x1​),…,V(xn​))]=Ea​[eSn​−ag(S1​,…,Sn​)] for a centred random walk SSS.
  2. Renewal function (2.13): the renewal function RRR of the strict descending ladder heights of SSS satisfies R(x)/x→c0>0R(x)/x\to c_0>0R(x)/x→c0​>0.
  3. Corollary 3.2 and Proposition 1.2: for the walk killed below 000, ez P(Mnkill<32ln⁡n−z)→C1e^z\,\mathbf P(M_n^{\rm kill}<\frac32\ln n-z)\to C_1ezP(Mnkill​<23​lnn−z)→C1​, uniformly for z∈[A,32ln⁡n−A]z\in[A,\frac32\ln n-A]z∈[A,23​lnn−A].
  4. Global minimum bound: P(∃u∈T:V(u)≤−y)≤e−y\mathbf P(\exists u\in\mathbb T: V(u)\le-y)\le e^{-y}P(∃u∈T:V(u)≤−y)≤e−y.
  5. Corollary 3.5: P(Mn≤32ln⁡n−y)≤(1+c10(1+y))e−y\mathbf P(M_n\le\frac32\ln n-y)\le(1+c_{10}(1+y))e^{-y}P(Mn​≤23​lnn−y)≤(1+c10​(1+y))e−y.
  6. Proposition 4.1: ezzP(Mn<32ln⁡n−z)→C1c0\frac{e^z}{z}\mathbf P(M_n<\frac32\ln n-z)\to C_1c_0zez​P(Mn​<23​lnn−z)→C1​c0​, uniformly on the same window.
  7. Derivative martingale: Dn→D∞D_n\to D_\inftyDn​→D∞​ a.s., D∞≥0D_\infty\ge0D∞​≥0, D∞>0D_\infty>0D∞​>0 a.s. on non-extinction.
  8. (5.2): ∑u∈Z[A]V(u)e−V(u)→D∞\sum_{u\in\mathcal Z[A]}V(u)e^{-V(u)}\to D_\infty∑u∈Z[A]​V(u)e−V(u)→D∞​ a.s. as A→∞A\to\inftyA→∞, where Z[A]\mathcal Z[A]Z[A] is the set of particles absorbed at level AAA.

Significance

The theorem identifies the limit law of the extreme particle: Mn−32ln⁡nM_n-\frac32\ln nMn​−23​lnn converges in law, on the event of survival, to a Gumbel variable shifted by −ln⁡(C∗D∞)-\ln(C^*D_\infty)−ln(C∗D∞​). It is the input for the study of the whole extremal process of the branching random walk seen from its leftmost particle (Madaule, J. Theoret. Probab., 2017), and it is the discrete-time counterpart of the Bramson and Lalley–Sellke results that later work on log-correlated fields takes as its template. The 32\frac3223​ correction and the role of the derivative martingale are the signature of the boundary case, and of log-correlated extremes generally.

The result is proved and published. It has not been formalized: Mathlib has no branching processes, no Galton–Watson trees with positions, no renewal theory, and no derivative martingale. This mission produces the first machine-checkable statement of the convergence-in-law theorem and of the intermediate results it rests on. A complete development would also give reusable formal versions of the many-to-one lemma and of renewal theory for ladder heights.

Difficulty

The first-moment computation through the many-to-one lemma gives the wrong centring. It predicts that the minimum sits near 12ln⁡n\frac12\ln n21​lnn, because the expected number of particles below a level is dominated by rare realisations. The true centring 32ln⁡n\frac32\ln n23​lnn only appears after restricting to particles whose ancestral path stays above a barrier, and this restriction requires random-walk estimates (ballot theorems and local limit theorems for walks conditioned to stay positive) that hold uniformly in a window of starting points. A second difficulty is that the limit must be identified, not only shown to exist. Tightness and subsequence arguments do not give the factor D∞D_\inftyD∞​; the identification needs the precise tail C1c0 z e−zC_1c_0\,z\,e^{-z}C1​c0​ze−z of Proposition 4.1, with a known constant, and the almost-sure behaviour of the sum over the stopping line Z[A]\mathcal Z[A]Z[A].

Formalization scope

  • Point process. The law LLL of L\mathcal LL is a probability measure on configurations (N,p)∈N∞×(N→R)(N,p)\in\mathbb N_\infty\times(\mathbb N\to\mathbb R)(N,p)∈N∞​×(N→R), where the points are pip_ipi​ for i<Ni<Ni<N.
  • Tree. Labels are List ℕ. The branching random walk is any family (ξu)u(\xi_u)_{u}(ξu​)u​ of independent measurable configurations of law LLL on a probability space. Every theorem holds for every such realisation, and a canonical realisation exists (product space).
  • Assumptions. Every expectation in (1.1), (1.3), (1.4) is a lower Lebesgue integral of a [0,∞][0,\infty][0,∞]-valued sum. The signed condition in (1.1) is "the expectations of ∑V+e−V\sum V_+e^{-V}∑V+​e−V and ∑V−e−V\sum V_-e^{-V}∑V−​e−V are equal and finite". Non-lattice means: there are no a∈Ra\in\mathbb Ra∈R and d>0d>0d>0 with all points a.s. in a+dZa+d\mathbb Za+dZ.
  • Which assumptions where. The goal and §§3–5 assume all of them. The many-to-one lemma and the global minimum bound assume only (1.1), and the derivative-martingale milestone drops non-lattice, as in Appendix A.
  • Minima and limits. MnM_nMn​ and MnkillM_n^{\rm kill}Mnkill​ are extended reals, +∞+\infty+∞ on an empty generation, so extinction lies in {Mn≥32ln⁡n+x}\{M_n\ge\frac32\ln n+x\}{Mn​≥23​lnn+x}. DnD_nDn​ is a real sum over generation nnn, absolutely summable almost surely, and D∞D_\inftyD∞​ is its pointwise limit. The expectation in the goal is a lower integral of e−C∗exD∞∈(0,1]e^{-C^*e^xD_\infty}\in(0,1]e−C∗exD∞​∈(0,1].
  • Constants. C∗C^*C∗ is chosen before xxx. In Proposition 4.1, C1C_1C1​ and c0c_0c0​ are hypotheses tied to Proposition 1.2 and (2.13), not re-chosen. Corollaries 3.2 and 3.5 assert existence of their constants without Proposition 3.1 and Corollary 3.4.
  • Not trivial. A formalization with a real-valued MnM_nMn​ equal to 000 on extinction, a D∞D_\inftyD∞​ never shown to be the limit, or a C∗C^*C∗ depending on xxx would not be this theorem. The definitions above rule out each of these.

Out of scope: the spine-measure lemmas (Lemmas 2.3, 3.3, 3.8–3.10, 4.3), Propositions 2.1–2.2 cited from Lyons, and Appendices B–C. Contributions are welcome on all milestones. The many-to-one lemma and the renewal statement are independent of the rest and are natural first targets.

Selected references

  • E. Aïdékon, Convergence in law of the minimum of a branching random walk, Ann. Probab. 41(3A) (2013) 1362–1426. arXiv:1101.1810, doi:10.1214/12-AOP750
  • J. D. Biggins, A. E. Kyprianou, Measure change in multitype branching, Adv. Appl. Probab. 36 (2004) 544–581. Reference [7] of Aïdékon (2013), arXiv:1101.1810, p. 68
  • M. Bramson, Convergence of solutions of the Kolmogorov equation to travelling waves, Mem. Amer. Math. Soc. 44, no. 285 (1983). doi:10.1090/memo/0285
  • S. P. Lalley, T. Sellke, A conditional limit theorem for the frontier of a branching Brownian motion, Ann. Probab. 15 (1987) 1052–1061. Reference [21] of Aïdékon (2013), arXiv:1101.1810, p. 69
  • Y. Hu, Z. Shi, Minimal position and critical martingale convergence in branching random walks, and directed polymers on disordered trees, Ann. Probab. 37 (2009) 742–789. arXiv:math/0702799
  • L. Addario-Berry, B. Reed, Minima in branching random walks, Ann. Probab. 37 (2009) 1044–1079. Reference [1] of Aïdékon (2013), arXiv:1101.1810, p. 68
  • R. Lyons, A simple path to Biggins' martingale convergence for branching random walk, in Classical and Modern Branching Processes, IMA Vol. Math. Appl. 84 (1997) 217–221. Reference [22] of Aïdékon (2013), arXiv:1101.1810, p. 69
13 thms1 active userReviewed
Algorithmic Game TheoryControl TheoryOperations Research·Captain: mikedeng1

Nonzero-Sum Stochastic Differential Games with Impulse Controls: A Verification Theorem with Applications 1: Regular Solutions of the Quasi-Variational Inequalities Give Nash Equilibrium PayoffsResearch Paper

Motivation

Many economic and engineering systems are steered by agents who act at discrete instants rather than continuously: a central bank intervenes on an exchange rate, two energy producers adjust a shared stock, a firm rebalances inventory. Each action has a fixed cost, so continuous control is not realistic. The mathematical model is impulse control: the state follows a diffusion, and a controller may shift it at chosen stopping times by paying a cost. The single-controller theory is classical (Øksendal and Sulem, Applied Stochastic Control of Jump Diffusions, 2007). When two controllers with different objectives act on the same state, the result is a nonzero-sum stochastic differential game with impulse controls.

Before Aïd, Basei, Callegaro, Campi and Vargiolu, the literature on games with impulse controls was almost entirely zero-sum: Cosso (SIAM J. Control Optim., 2013) characterised the value of zero-sum impulse games through a double-obstacle quasi-variational inequality in the viscosity sense. Their paper (Math. Oper. Res. 45(1), 2020; arXiv:1605.00039) gives the first general formulation of the nonzero-sum case together with a verification theorem: a system of quasi-variational inequalities (QVIs) whose sufficiently regular solutions are the equilibrium payoffs. Its Section 4 then computes Nash equilibria in closed form for a one-dimensional game.

Setting

A kkk-dimensional Brownian motion WWW on a filtered probability space satisfying the usual conditions drives the state equation

dYs=b(Ys) ds+σ(Ys) dWs,Ys∈Rd,dY_s=b(Y_s)\,ds+\sigma(Y_s)\,dW_s ,\qquad Y_s\in\mathbb R^d,dYs​=b(Ys​)ds+σ(Ys​)dWs​,Ys​∈Rd,

with globally Lipschitz bbb and σ\sigmaσ. The game takes place in an open set S⊆RdS\subseteq\mathbb R^dS⊆Rd and ends at the exit time τS\tau_SτS​ of the state from SSS. Each of two players i∈{1,2}i\in\{1,2\}i∈{1,2} has a nonempty impulse set Zi⊆RliZ_i\subseteq\mathbb R^{l_i}Zi​⊆Rli​ and a continuous impulse map Γi:S×Zi→S\Gamma^i:S\times Z_i\to SΓi:S×Zi​→S: an intervention with impulse δ\deltaδ moves the state from yyy to Γi(y,δ)\Gamma^i(y,\delta)Γi(y,δ).

A strategy of player iii is a pair φi=(Ci,ξi)\varphi_i=(\mathcal C_i,\xi_i)φi​=(Ci​,ξi​) with Ci⊆S\mathcal C_i\subseteq SCi​⊆S open and ξi:S→Zi\xi_i:S\to Z_iξi​:S→Zi​ continuous. Player iii intervenes as soon as the state leaves Ci\mathcal C_iCi​, with impulse ξi(y)\xi_i(y)ξi​(y) at the current state yyy. Player 1 has priority on ties, and several interventions may happen at the same instant. This defines the controlled process XXX, the intervention times τi,n\tau_{i,n}τi,n​ and impulses δi,n\delta_{i,n}δi,n​ of each player, and the states X(τi,n)−X_{(\tau_{i,n})^-}X(τi,n​)−​ just before each intervention. The payoff of player iii is

Ji(x;φ1,φ2)=E[∫0τSe−ρisfi(Xs)ds+∑τi,n<τSe−ρiτi,nϕi(X(τi,n)−,δi,n)+∑τj,n<τSe−ρiτj,nψi(X(τj,n)−,δj,n)+e−ρiτShi(XτS)1{τS<∞}],J^i(x;\varphi_1,\varphi_2)=\mathbb E\Big[\int_0^{\tau_S}e^{-\rho_is}f_i(X_s)ds+\sum_{\tau_{i,n}<\tau_S}e^{-\rho_i\tau_{i,n}}\phi_i(X_{(\tau_{i,n})^-},\delta_{i,n})+\sum_{\tau_{j,n}<\tau_S}e^{-\rho_i\tau_{j,n}}\psi_i(X_{(\tau_{j,n})^-},\delta_{j,n})+e^{-\rho_i\tau_S}h_i(X_{\tau_S})\mathbf 1_{\{\tau_S<\infty\}}\Big],Ji(x;φ1​,φ2​)=E[∫0τS​​e−ρi​sfi​(Xs​)ds+τi,n​<τS​∑​e−ρi​τi,n​ϕi​(X(τi,n​)−​,δi,n​)+τj,n​<τS​∑​e−ρi​τj,n​ψi​(X(τj,n​)−​,δj,n​)+e−ρi​τS​hi​(XτS​​)1{τS​<∞}​],

where j≠ij\ne ij=i, ρi>0\rho_i>0ρi​>0, fif_ifi​ is a running payoff, ϕi\phi_iϕi​ the cost of one's own interventions, ψi\psi_iψi​ the gain from the opponent's, and hih_ihi​ a terminal payoff on ∂S\partial S∂S. A pair of strategies is xxx-admissible, (φ1,φ2)∈Φx(\varphi_1,\varphi_2)\in\Phi_x(φ1​,φ2​)∈Φx​, when these four terms are integrable, sup⁡s≤τS∣Xs∣\sup_{s\le\tau_S}|X_s|sups≤τS​​∣Xs​∣ has all moments, and the interventions do not accumulate before τS\tau_SτS​. A Nash equilibrium is a pair in Φx\Phi_xΦx​ from which no player gains by a unilateral admissible deviation.

Given candidate payoff functions V1,V2V_1,V_2V1​,V2​ on Sˉ\bar SSˉ, let δi(x)\delta_i(x)δi​(x) be the unique maximiser of Vi(Γi(x,δ))+ϕi(x,δ)V_i(\Gamma^i(x,\delta))+\phi_i(x,\delta)Vi​(Γi(x,δ))+ϕi​(x,δ) over ZiZ_iZi​. Define MiVi(x)=Vi(Γi(x,δi(x)))+ϕi(x,δi(x))\mathcal M_iV_i(x)=V_i(\Gamma^i(x,\delta_i(x)))+\phi_i(x,\delta_i(x))Mi​Vi​(x)=Vi​(Γi(x,δi​(x)))+ϕi​(x,δi​(x)), HiVi(x)=Vi(Γj(x,δj(x)))+ψi(x,δj(x))\mathcal H_iV_i(x)=V_i(\Gamma^j(x,\delta_j(x)))+\psi_i(x,\delta_j(x))Hi​Vi​(x)=Vi​(Γj(x,δj​(x)))+ψi​(x,δj​(x)), the continuation region Di={MiVi−Vi<0}\mathcal D_i=\{\mathcal M_iV_i-V_i<0\}Di​={Mi​Vi​−Vi​<0} and the generator AV=b⋅∇V+12tr⁡(σσtD2V)\mathcal AV=b\cdot\nabla V+\tfrac12\operatorname{tr}(\sigma\sigma^tD^2V)AV=b⋅∇V+21​tr(σσtD2V). The QVI system is

Vi=hi on ∂S,MjVj−Vj≤0 on S,HiVi−Vi=0 on {MjVj=Vj},max⁡{AVi−ρiVi+fi, MiVi−Vi}=0 on Dj.V_i=h_i\ \text{on }\partial S,\quad \mathcal M_jV_j-V_j\le0\ \text{on }S,\quad \mathcal H_iV_i-V_i=0\ \text{on }\{\mathcal M_jV_j=V_j\},\quad \max\{\mathcal AV_i-\rho_iV_i+f_i,\ \mathcal M_iV_i-V_i\}=0\ \text{on }\mathcal D_j .Vi​=hi​ on ∂S,Mj​Vj​−Vj​≤0 on S,Hi​Vi​−Vi​=0 on {Mj​Vj​=Vj​},max{AVi​−ρi​Vi​+fi​, Mi​Vi​−Vi​}=0 on Dj​.

Formalization targets

Goal: Theorem 3.3 (verification theorem)

Suppose V1,V2V_1,V_2V1​,V2​ solve the QVI system, Vi∈C2(Dj∖∂Di)∩C1(Dj)∩C(Sˉ)V_i\in C^2(\mathcal D_j\setminus\partial\mathcal D_i)\cap C^1(\mathcal D_j)\cap C(\bar S)Vi​∈C2(Dj​∖∂Di​)∩C1(Dj​)∩C(Sˉ) with polynomial growth, ∂Di\partial\mathcal D_i∂Di​ is a Lipschitz surface near which ViV_iVi​ has locally bounded first and second derivatives, x∈Sx\in Sx∈S, and the threshold pair φi∗=(Di,δi)\varphi_i^*=(\mathcal D_i,\delta_i)φi∗​=(Di​,δi​) is xxx-admissible. Then

(φ1∗,φ2∗) is a Nash equilibrium andVi(x)=Ji(x;φ1∗,φ2∗),i=1,2.(\varphi_1^*,\varphi_2^*)\ \text{is a Nash equilibrium and}\quad V_i(x)=J^i(x;\varphi_1^*,\varphi_2^*),\qquad i=1,2 .(φ1∗​,φ2∗​) is a Nash equilibrium andVi​(x)=Ji(x;φ1∗​,φ2∗​),i=1,2.

Milestones

  • Lemma 2.3: the controlled process is the concatenation of diffusion pieces, it jumps only at interventions, and between interventions it stays in C1∩C2\mathcal C_1\cap\mathcal C_2C1​∩C2​.
  • Remark 3.6, (3.8b), (3.8d), (3.8f): against φ2∗\varphi_2^*φ2∗​, the state stays in D2\mathcal D_2D2​, and player 2 intervenes only on {M2V2=V2}\{\mathcal M_2V_2=V_2\}{M2​V2​=V2​} with impulse δ2\delta_2δ2​.
  • Step 1 of the proof: V1(x)≥J1(x;φ1,φ2∗)V_1(x)\ge J^1(x;\varphi_1,\varphi_2^*)V1​(x)≥J1(x;φ1​,φ2∗​) for every admissible deviation φ1\varphi_1φ1​.
  • Step 2 of the proof: V1(x)=J1(x;φ1∗,φ2∗)V_1(x)=J^1(x;\varphi_1^*,\varphi_2^*)V1​(x)=J1(x;φ1∗​,φ2∗​).

The goal follows from Steps 1 and 2 and their mirror images for player 2.

Significance

The theorem turns the search for Nash equilibria of nonzero-sum impulse games, an infinite-dimensional fixed-point problem over strategy pairs, into a deterministic problem: find functions satisfying a system of coupled QVIs with prescribed regularity. The regularity conditions become smooth-pasting conditions, hence a system of algebraic equations; Section 4 of the paper solves it explicitly for a one-dimensional game with linear payoffs. A further consequence is structural: equilibrium payoffs need only be C2C^2C2 on the opponent's continuation region, which is what lets non-smooth, piecewise-defined candidates qualify.

The result is proved in the paper; no machine-checked version exists. A complete formalization would be the first verified verification theorem for impulse control, single-player or game, and would expose every convention of the model: priority on ties, simultaneous interventions, the treatment of exit, and the integrability of the payoff. Several of these conventions need correction on the page, as listed below.

Difficulty

The heuristic argument applies Itô's formula to e−ρ1tV1(Xt)e^{-\rho_1t}V_1(X_t)e−ρ1​tV1​(Xt​) and uses the QVIs term by term. This fails on two counts. First, V1V_1V1​ is only C1C^1C1 across the free boundary ∂D1\partial\mathcal D_1∂D1​, so Itô's formula does not apply directly. The paper mollifies V1V_1V1​ (following Øksendal's proof of his verification theorem for optimal stopping) and must control the second derivatives near a Lipschitz boundary. Second, the sums over interventions may be infinite and the horizon unbounded, so expectations and limits do not commute. The passage to the limit needs the integrability built into Φx\Phi_xΦx​ and the polynomial growth of ViV_iVi​. The stochastic-calculus infrastructure itself (Itô's formula for continuous semimartingales stopped at random times, strong solutions of Lipschitz SDEs restarted at stopping times) is largely missing from Mathlib.

Formalization scope

The Lean model is pathwise. A realization is a sequence of diffusion pieces, each solving the state equation from the random restart time with the restart value. The Itô integral is the published relation EthierKurtz.HasBrownianItoIntegral, and the stochastic basis uses the published You2015.Shared.UsualConditions and IsFBrownian. Times take values in [0,∞][0,\infty][0,∞] with e−ρ⋅∞=0e^{-\rho\cdot\infty}=0e−ρ⋅∞=0, and the state space is EuclideanSpace ℝ (Fin d). Φx\Phi_xΦx​ asks for one admissible realization, while the Nash inequalities and the payoff identity hold on every admissible realization; strong uniqueness makes these readings equivalent.

The following deviate from the page and are disclosed in the items:

  1. δi\delta_iδi​ is assumed continuous, so that φi∗\varphi_i^*φi∗​ is a strategy.
  2. The fourth QVI is imposed on Dj∖∂Di\mathcal D_j\setminus\partial\mathcal D_iDj​∖∂Di​, where AVi\mathcal AV_iAVi​ exists.
  3. The exit time is αkˉ+1S\alpha^S_{\bar k+1}αkˉ+1S​, not the printed αkˉS\alpha^S_{\bar k}αkˉS​.
  4. The gain term of (2.7) is read with the opponent's interventions.
  5. The supremum in (2.8) is taken over [0,τS][0,\tau_S][0,τS​].
  6. Interventions accumulating at a finite τS\tau_SτS​ are excluded from Φx\Phi_xΦx​, since the page leaves XτSX_{\tau_S}XτS​​ undefined there.
  7. Lemma 2.3 is stated in corrected form for simultaneous and boundary interventions.

The payoff is a Bochner expectation only inside Φx\Phi_xΦx​, which requires the L1L^1L1 conditions of (2.7) and the integrability of each payoff. A non-integrable deviation therefore never receives the junk payoff 000, and the Nash inequality cannot hold vacuously.

Contributions are welcome on the stochastic-calculus layer this needs (Itô's formula, strong existence and uniqueness for Lipschitz SDEs, optional stopping for stochastic integrals) and on the mollification lemma for functions that are C1C^1C1 with piecewise bounded second derivatives across a Lipschitz surface. These are reusable well beyond this mission.

Selected references

  • R. Aïd, M. Basei, G. Callegaro, L. Campi, T. Vargiolu, Nonzero-sum stochastic differential games with impulse controls: a verification theorem with applications, Math. Oper. Res. 45(1), 2020 (accepted manuscript, arXiv:1605.00039v4). https://arxiv.org/abs/1605.00039
  • A. Cosso, Stochastic differential games involving impulse controls and double-obstacle quasi-variational inequalities, SIAM J. Control Optim. 51(3), 2102–2131, 2013.
  • B. Øksendal, Stochastic Differential Equations, 6th ed., Springer, 2003. https://doi.org/10.1007/978-3-642-14394-6
  • B. Øksendal, A. Sulem, Applied Stochastic Control of Jump Diffusions, 2nd ed., Springer, 2007. https://doi.org/10.1007/978-3-540-69826-5
9 thms1 active userReviewed
Numerical AnalysisProbability·Captain: mikedeng1

Euler Approximations with Varying Coefficients III: Uniform Lq-Convergence with Order 1/2Research Paper

Motivation

Stochastic differential equations whose coefficients grow faster than linearly (for instance stochastic volatility models such as the 3/2-model) cannot be simulated with the classical explicit Euler–Maruyama scheme: Hutzenthaler, Jentzen and Kloeden showed that its moments diverge when the drift or diffusion grows superlinearly (Proc. R. Soc. A, 2011). Implicit schemes avoid the divergence but require solving a nonlinear equation at every step. Tamed Euler schemes keep the scheme explicit and damp the coefficients by a factor depending on the step size.

  • 2012: Hutzenthaler, Jentzen and Kloeden introduce a tamed Euler scheme for SDEs with superlinearly growing drift (Ann. Appl. Probab. 22, 2012).
  • 2013: Sabanis extends the taming analysis for superlinearly growing drift (Electron. Commun. Probab. 18, no. 10, 2013, MR3070913).
  • 2015: Hutzenthaler and Jentzen survey explicit schemes for non-globally Lipschitz coefficients (Mem. Amer. Math. Soc. 236, no. 1112, 2015); for the 3/2-model their results give Lp\mathcal L^pLp-convergence without rate only for p<1/2p<1/2p<1/2, as Sabanis (2016, p. 2) notes.
  • 2016: Sabanis (Ann. Appl. Probab. 26(4), 2016, arXiv:1308.1796v4) treats schemes with varying coefficients bn,σnb_n,\sigma_nbn​,σn​, covering superlinearly growing diffusion coefficients, and proves Lp\mathcal L^pLp convergence (Theorem 1), the Lp\mathcal L^pLp rate 1/2 (Theorem 2) and the uniform Lq\mathcal L^qLq rate 1/2 (Theorem 3). A motivating example is the ddd-dimensional analogue of the 3/2-model of stochastic volatility, dX=λX(μ−∣X∣) dt+ξ∣X∣3/2 dWdX=\lambda X(\mu-|X|)\,dt+\xi|X|^{3/2}\,dWdX=λX(μ−∣X∣)dt+ξ∣X∣3/2dW, used for pricing VIX options.

Theorem 3 is the target here; Theorems 1 and 2 are the targets of companion missions.

Setting

Fix a filtered probability space (Ω,{Ft}t≥0,F,P)(\Omega,\{\mathcal F_t\}_{t\ge0},\mathcal F,P)(Ω,{Ft​}t≥0​,F,P) satisfying the usual conditions, a d1d_1d1​-dimensional Wiener martingale WWW, a horizon T>0T>0T>0, and exponents p0,p1≥2p_0,p_1\ge2p0​,p1​≥2. For x∈Rdx\in\mathbb R^dx∈Rd, ∣x∣|x|∣x∣ is the Euclidean norm; for a d×d1d\times d_1d×d1​ matrix AAA, ∣A∣|A|∣A∣ is the Hilbert–Schmidt norm; xyxyxy is the scalar product. The coefficients are Borel functions b:[0,∞)×Rd→Rdb:[0,\infty)\times\mathbb R^d\to\mathbb R^db:[0,∞)×Rd→Rd and σ:[0,∞)×Rd→Rd×d1\sigma:[0,\infty)\times\mathbb R^d\to\mathbb R^{d\times d_1}σ:[0,∞)×Rd→Rd×d1​, and the SDE is

dX(t)=b(t,X(t)) dt+σ(t,X(t)) dW(t),t∈[0,T],(2.1)dX(t)=b(t,X(t))\,dt+\sigma(t,X(t))\,dW(t),\qquad t\in[0,T],\qquad(2.1)dX(t)=b(t,X(t))dt+σ(t,X(t))dW(t),t∈[0,T],(2.1)

with an F0\mathcal F_0F0​-measurable initial value X(0)X(0)X(0). With κn(t)=⌊nt⌋/n\kappa_n(t)=\lfloor nt\rfloor/nκn​(t)=⌊nt⌋/n, the scheme (2.2) is

dXn(t)=bn(t,Xn(κn(t))) dt+σn(t,Xn(κn(t))) dW(t),Xn(0)=X(0),dX_n(t)=b_n(t,X_n(\kappa_n(t)))\,dt+\sigma_n(t,X_n(\kappa_n(t)))\,dW(t),\qquad X_n(0)=X(0),dXn​(t)=bn​(t,Xn​(κn​(t)))dt+σn​(t,Xn​(κn​(t)))dW(t),Xn​(0)=X(0),

and in this mission bn,σnb_n,\sigma_nbn​,σn​ are the tamed coefficients of Model 2 with α=1/2\alpha=1/2α=1/2:

bn(t,x)=b(t,x)1+n−1/2∣x∣l,σn(t,x)=σ(t,x)1+n−1/2∣x∣l.b_n(t,x)=\frac{b(t,x)}{1+n^{-1/2}|x|^l},\qquad \sigma_n(t,x)=\frac{\sigma(t,x)}{1+n^{-1/2}|x|^l}.bn​(t,x)=1+n−1/2∣x∣lb(t,x)​,σn​(t,x)=1+n−1/2∣x∣lσ(t,x)​.

The conditions used are: A-2, local boundedness of bbb on balls; A-4, the coercivity bound 2xb(t,x)+(p0−1)∣σ(t,x)∣2≤K(1+∣x∣2)2xb(t,x)+(p_0-1)|\sigma(t,x)|^2\le K(1+|x|^2)2xb(t,x)+(p0​−1)∣σ(t,x)∣2≤K(1+∣x∣2); A-5, E∣X(0)∣p0<∞\mathbb E|X(0)|^{p_0}<\inftyE∣X(0)∣p0​<∞; and A-6, the global monotonicity

2(x−y)(b(t,x)−b(t,y))+(p1−1)∣σ(t,x)−σ(t,y)∣2≤L∣x−y∣22(x-y)(b(t,x)-b(t,y))+(p_1-1)|\sigma(t,x)-\sigma(t,y)|^2\le L|x-y|^22(x−y)(b(t,x)−b(t,y))+(p1​−1)∣σ(t,x)−σ(t,y)∣2≤L∣x−y∣2

together with the polynomial Lipschitz bound ∣b(t,x)−b(t,y)∣≤L(1+∣x∣l+∣y∣l)∣x−y∣|b(t,x)-b(t,y)|\le L(1+|x|^l+|y|^l)|x-y|∣b(t,x)−b(t,y)∣≤L(1+∣x∣l+∣y∣l)∣x−y∣, with l,L>0l,L>0l,L>0. The p\mathfrak pp-condition asks l≤p0−24l\le\frac{p_0-2}{4}l≤4p0​−2​ and a moment exponent ppp with 0<p<p10<p<p_10<p<p1​ and p≤p02l+1p\le\frac{p_0}{2l+1}p≤2l+1p0​​.

Formalization targets

Goal: Theorem 3 (p. 6)

Under A-2, A-4–A-6 and the p\mathfrak pp-condition, for every 0<q<p0<q<p0<q<p there is CCC independent of nnn with

E[sup⁡0≤t≤T∣X(t)−Xn(t)∣q]≤Cn−q/2,n≥1.(2.14)\mathbb E\Big[\sup_{0\le t\le T}|X(t)-X_n(t)|^q\Big]\le Cn^{-q/2},\qquad n\ge1.\qquad(2.14)E[0≤t≤Tsup​∣X(t)−Xn​(t)∣q]≤Cn−q/2,n≥1.(2.14)

The supremum is inside the expectation, which distinguishes it from Theorem 2.

Milestones

  1. Lemma 2 (p. 8): the moments of XXX and of XnX_nXn​ up to order p0p_0p0​ are bounded on [0,T][0,T][0,T] uniformly in nnn (for general coefficients bn,σnb_n,\sigma_nbn​,σn​ under A-1–A-5, B-2, B-3).
  2. Lemma 5 (p. 19): the Gyöngy–Krylov maximal inequality. If nonnegative continuous adapted f,gf,gf,g satisfy E[fτ1{g0≤c}]≤E[gτ1{g0≤c}]\mathbb E[f_\tau\mathbb 1_{\{g_0\le c\}}]\le\mathbb E[g_\tau\mathbb 1_{\{g_0\le c\}}]E[fτ​1{g0​≤c}​]≤E[gτ​1{g0​≤c}​] for all c>0c>0c>0 and stopping times τ≤T\tau\le Tτ≤T, then
E[sup⁡t≤τftγ]≤2−γ1−γ E[sup⁡t≤τgtγ],γ∈(0,1).\mathbb E\Big[\sup_{t\le\tau}f_t^\gamma\Big]\le\frac{2-\gamma}{1-\gamma}\,\mathbb E\Big[\sup_{t\le\tau}g_t^\gamma\Big],\qquad\gamma\in(0,1).E[t≤τsup​ftγ​]≤1−γ2−γ​E[t≤τsup​gtγ​],γ∈(0,1).
  1. Lemma 4 (p. 16): sup⁡0≤t≤TE∣Xn(t)−Xn(κn(t))∣p≤Cn−p/2\sup_{0\le t\le T}\mathbb E|X_n(t)-X_n(\kappa_n(t))|^p\le Cn^{-p/2}sup0≤t≤T​E∣Xn​(t)−Xn​(κn​(t))∣p≤Cn−p/2.
  2. Lemma 3 (p. 15): the taming errors E∫0T∣b−bn∣p(s,Xn(κn(s))) ds\mathbb E\int_0^T|b-b_n|^p(s,X_n(\kappa_n(s)))\,dsE∫0T​∣b−bn​∣p(s,Xn​(κn​(s)))ds and the analogue for σ\sigmaσ are ≤Cn−αp\le Cn^{-\alpha p}≤Cn−αp.

Significance

Theorem 3 gives the optimal strong rate 1/2 for an explicit scheme, uniformly over the time interval, for SDEs whose diffusion coefficient may grow superlinearly. Uniform-in-time error bounds are what pathwise functionals need (running maxima, barrier options, hitting times), and via Borel–Cantelli they give the almost-sure rate of Corollary 1 in the paper. Lemma 5 is a general-purpose tool: a domination inequality of Lenglart type, used throughout stochastic analysis to pass from bounds at stopping times to bounds on running suprema.

The results are proved in the paper; to our knowledge none of them has been formalized. Formalizing them would produce the first machine-checked convergence rate of a numerical scheme for SDEs, and a Lean proof of a Lenglart-type inequality, which Mathlib does not currently contain.

Difficulty

The error X−XnX-X_nX−Xn​ is controlled by the monotonicity condition A-6, which is one-sided: it bounds 2(x−y)(b(t,x)−b(t,y))2(x-y)(b(t,x)-b(t,y))2(x−y)(b(t,x)−b(t,y)) from above but gives no Lipschitz bound on bbb or σ\sigmaσ with a constant independent of ∣x∣,∣y∣|x|,|y|∣x∣,∣y∣. The first idea, estimating Esup⁡t∣X−Xn∣p\mathbb E\sup_t|X-X_n|^pEsupt​∣X−Xn​∣p directly by applying the Burkholder–Davis–Gundy inequality to the martingale part of the error equation, therefore does not close: it produces terms of the same order as the quantity being estimated, weighted by polynomial factors in ∣X∣|X|∣X∣ and ∣Xn∣|X_n|∣Xn​∣, and the argument that gives Theorem 2 controls E∣X(t)−Xn(t)∣p\mathbb E|X(t)-X_n(t)|^pE∣X(t)−Xn​(t)∣p only for fixed ttt. Passing from fixed times to the supremum is what forces the loss from ppp to q<pq<pq<p in Theorem 3, and it is where Lemma 5 enters. On the formal side, the Itô integral against Brownian motion, Itô's formula for functions of multidimensional Itô processes and the Burkholder–Davis–Gundy inequality are not in Mathlib.

Formalization scope

Processes are indexed by t∈[0,∞)t\in[0,\infty)t∈[0,∞) (ℝ≥0) with values in EuclideanSpace ℝ (Fin d); diffusion values live in EuclideanSpace ℝ (Fin d × Fin d₁), whose norm is the Hilbert–Schmidt norm. The stochastic integral is the Ethier–Kurtz platform relation EthierKurtz.HasBrownianItoIntegral, applied coordinatewise, with integrands cut off after TTT. Solutions are hypotheses: the theorems apply to any solution XXX of (2.1) and any family (Xn)n≥1(X_n)_{n\ge1}(Xn​)n≥1​ of solutions of (2.2) on one probability space with one WWW and one X(0)X(0)X(0); existence is not claimed. Processes are required to be adapted to the filtration augmented by null sets (weaker than a complete filtration), the filtration is right-continuous, and nothing is required after TTT. Expectations and suprema are computed in [0,∞][0,\infty][0,∞], so no statement holds through an integral defaulting to 000. Constants CCC depend on everything except nnn (and, in Theorem 3, may depend on qqq); only n≥1n\ge1n≥1 is used, and q>0q>0q>0, p>0p>0p>0 follow the paper's Lp\mathcal L^pLp convention.

Two hypotheses are added to the printed statements, both because the printed proofs need them:

  • p1>2p_1>2p1​>2 in Theorem 3. The proof applies Itô's formula for p≥2p\ge2p≥2; an admissible p<2p<2p<2 is handled through p′=2p'=2p′=2, which satisfies the p\mathfrak pp-condition exactly when p1>2p_1>2p1​>2.
  • B-3 in Lemma 4. The proof uses moment bounds uniform in nnn (Lemma 2), which need B-3. Model 2 satisfies B-3.

Lemma 2 drops the dependence clause C=C(p,T,K,E∣X(0)∣p)C=C(p,T,K,\mathbb E|X(0)|^p)C=C(p,T,K,E∣X(0)∣p) and asserts only finiteness. Lemma 5 assumes neither that ggg is nondecreasing nor anything beyond the printed hypotheses.

A trivializing formalization is ruled out: the supremum sits inside a Lebesgue integral in [0,∞][0,\infty][0,∞], so the goal cannot hold because of a junk value, and the hypotheses are satisfiable (zero coefficients, constant solutions), which a sorry-free check confirms.

Needed infrastructure: Itô's formula for multidimensional Itô processes, the Burkholder–Davis–Gundy inequality (for Lemma 4), Gronwall's inequality in integral form, and stopping-time localisation for continuous adapted processes. Contributions to any of these are welcome, and all are reusable far beyond this mission. Lemma 5 is independent of the SDE part and can be attacked first.

Selected references

  • S. Sabanis, Euler approximations with varying coefficients: the case of superlinearly growing diffusion coefficients, Ann. Appl. Probab. 26(4), 2083–2105, 2016. https://doi.org/10.1214/15-AAP1140 — arXiv:1308.1796v4, https://arxiv.org/abs/1308.1796v4
  • M. Hutzenthaler, A. Jentzen, P. E. Kloeden, Strong and weak divergence in finite time of Euler's method for stochastic differential equations with non-globally Lipschitz continuous coefficients, Proc. R. Soc. A 467, 1563–1576, 2011. https://doi.org/10.1098/rspa.2010.0348
  • M. Hutzenthaler, A. Jentzen, P. E. Kloeden, Strong convergence of an explicit numerical method for SDEs with nonglobally Lipschitz continuous coefficients, Ann. Appl. Probab. 22(4), 1611–1641, 2012. https://doi.org/10.1214/11-AAP803
  • M. Hutzenthaler, A. Jentzen, Numerical approximations of stochastic differential equations with non-globally Lipschitz continuous coefficients, Mem. Amer. Math. Soc. 236, no. 1112, 2015 (reference [5] of the paper).
  • S. Sabanis, A note on tamed Euler approximations, Electron. Commun. Probab. 18, no. 10, 2013. MR3070913
  • I. Gyöngy, N. V. Krylov, On the rate of convergence of splitting-up approximations for SPDEs, in Stochastic Inequalities and Applications, Progr. Probab. 56, 301–321, Birkhäuser, 2003 (cited by the paper for Lemma 5). MR2073438
  • N. V. Krylov, Introduction to the Theory of Diffusion Processes, Transl. Math. Monographs 142, AMS, 1995 (cited by the paper for Lemma 5). MR1311478
14 thms1 active userReviewed
Numerical AnalysisProbability·Captain: mikedeng1

Euler Approximations with Varying Coefficients II: Strong Order 1/2 in Lp for Superlinearly Growing Diffusion CoefficientsResearch Paper

Motivation

Stochastic differential equations with superlinearly growing coefficients appear throughout applied probability: population models with cubic damping, Langevin dynamics with polynomial potentials, and stochastic volatility models such as the 3/2-model used for pricing VIX options, whose diffusion coefficient grows like ∣x∣3/2|x|^{3/2}∣x∣3/2. For such equations the classical explicit Euler–Maruyama method fails: Hutzenthaler, Jentzen and Kloeden (2011) proved that its ppp-th moments diverge whenever a coefficient grows superlinearly, so it cannot converge in Lp\mathcal L^pLp. Implicit methods converge (Higham–Mao–Stuart 2002) but cost a nonlinear solve per step.

The tamed Euler schemes are the explicit response.

  • Hutzenthaler, Jentzen and Kloeden (2012) tamed a superlinear drift and proved strong order 1/21/21/2 for globally Lipschitz diffusion coefficients.
  • Sabanis (2013) gave a simpler proof for a family of tamed drifts.
  • Hutzenthaler and Jentzen (2015) treated superlinear diffusion coefficients, obtaining Lp\mathcal L^pLp-convergence without a rate and only for small ppp.
  • Sabanis (2016), the source of this mission, tames drift and diffusion together and proves the optimal strong rate 1/21/21/2 in Lp\mathcal L^pLp under a one-sided (monotonicity) condition and polynomial growth. For the 3/2-model with p1=3.5p_1=3.5p1​=3.5, p0=6p_0=6p0​=6 this gives L2\mathcal L^2L2-convergence with order 1/21/21/2, which earlier results did not cover (p. 2).

This mission is the second of three on the paper. Mission I formalizes Lp\mathcal L^pLp-convergence without a rate (Theorem 1), mission III the uniform-in-time rate (Theorem 3).

Setting

Fix a filtered probability space (Ω,{Ft}t≥0,F,P)(\Omega,\{\mathcal F_t\}_{t\ge0},\mathcal F,P)(Ω,{Ft​}t≥0​,F,P) with right-continuous filtration, a horizon T>0T>0T>0, dimensions d,d1d,d_1d,d1​, and a d1d_1d1​-dimensional Wiener martingale WWW: a standard Brownian motion adapted to {Ft}\{\mathcal F_t\}{Ft​} whose future increments are independent of Ft\mathcal F_tFt​. The coefficients are Borel maps b:[0,∞)×Rd→Rdb:[0,\infty)\times\mathbb R^d\to\mathbb R^db:[0,∞)×Rd→Rd and σ:[0,∞)×Rd→Rd×d1\sigma:[0,\infty)\times\mathbb R^d\to\mathbb R^{d\times d_1}σ:[0,∞)×Rd→Rd×d1​; ∣x∣|x|∣x∣ is the Euclidean norm, ∣A∣|A|∣A∣ the Hilbert–Schmidt norm, xyxyxy the scalar product. The SDE is

dX(t)=b(t,X(t)) dt+σ(t,X(t)) dW(t),t∈[0,T],(2.1)dX(t)=b(t,X(t))\,dt+\sigma(t,X(t))\,dW(t),\qquad t\in[0,T],\qquad(2.1)dX(t)=b(t,X(t))dt+σ(t,X(t))dW(t),t∈[0,T],(2.1)

with an F0\mathcal F_0F0​-measurable initial value X(0)=ξX(0)=\xiX(0)=ξ. For n≥1n\ge1n≥1 let κn(t)=⌊nt⌋/n\kappa_n(t)=\lfloor nt\rfloor/nκn​(t)=⌊nt⌋/n and consider the scheme

dXn(t)=bn(t,Xn(κn(t))) dt+σn(t,Xn(κn(t))) dW(t),Xn(0)=ξ,(2.2)dX_n(t)=b_n(t,X_n(\kappa_n(t)))\,dt+\sigma_n(t,X_n(\kappa_n(t)))\,dW(t),\qquad X_n(0)=\xi,\qquad(2.2)dXn​(t)=bn​(t,Xn​(κn​(t)))dt+σn​(t,Xn​(κn​(t)))dW(t),Xn​(0)=ξ,(2.2)

whose coefficients are frozen at the last grid point, so that XnX_nXn​ is computable step by step. The tamed coefficients of Model 2 are

bn(t,x)=b(t,x)1+n−α∣x∣l,σn(t,x)=σ(t,x)1+n−α∣x∣l.(2.11)–(2.12)b_n(t,x)=\frac{b(t,x)}{1+n^{-\alpha}|x|^l},\qquad \sigma_n(t,x)=\frac{\sigma(t,x)}{1+n^{-\alpha}|x|^l}.\qquad(2.11)\text{–}(2.12)bn​(t,x)=1+n−α∣x∣lb(t,x)​,σn​(t,x)=1+n−α∣x∣lσ(t,x)​.(2.11)–(2.12)

The hypotheses, with constants p0,p1≥2p_0,p_1\ge2p0​,p1​≥2:

  • A-2: bbb is bounded on balls, uniformly in t∈[0,T]t\in[0,T]t∈[0,T].
  • A-4 (coercivity): 2xb(t,x)+(p0−1)∣σ(t,x)∣2≤K(1+∣x∣2)2xb(t,x)+(p_0-1)|\sigma(t,x)|^2\le K(1+|x|^2)2xb(t,x)+(p0​−1)∣σ(t,x)∣2≤K(1+∣x∣2).
  • A-5: E∣X(0)∣p0<∞\mathbb E|X(0)|^{p_0}<\inftyE∣X(0)∣p0​<∞.
  • A-6 (global monotonicity, polynomial growth): for positive lll and LLL, 2(x−y)(b(t,x)−b(t,y))+(p1−1)∣σ(t,x)−σ(t,y)∣2≤L∣x−y∣22(x-y)(b(t,x)-b(t,y))+(p_1-1)|\sigma(t,x)-\sigma(t,y)|^2\le L|x-y|^22(x−y)(b(t,x)−b(t,y))+(p1​−1)∣σ(t,x)−σ(t,y)∣2≤L∣x−y∣2 and ∣b(t,x)−b(t,y)∣≤L(1+∣x∣l+∣y∣l)∣x−y∣|b(t,x)-b(t,y)|\le L(1+|x|^l+|y|^l)|x-y|∣b(t,x)−b(t,y)∣≤L(1+∣x∣l+∣y∣l)∣x−y∣.
  • The p\mathfrak pp-condition: the scheme uses (2.11)–(2.12) with α=1/2\alpha=1/2α=1/2, l≤p0−24l\le\frac{p_0-2}{4}l≤4p0​−2​, and 0<p<p10<p<p_10<p<p1​, p≤p02l+1p\le\frac{p_0}{2l+1}p≤2l+1p0​​.
  • B-2 and B-3 (for the milestones): ∣bn∣≤min⁡(Cnα(1+∣x∣),∣b∣)|b_n|\le\min(Cn^\alpha(1+|x|),|b|)∣bn​∣≤min(Cnα(1+∣x∣),∣b∣), ∣σn∣2≤min⁡(Cnα(1+∣x∣2),∣σ∣2)|\sigma_n|^2\le\min(Cn^\alpha(1+|x|^2),|\sigma|^2)∣σn​∣2≤min(Cnα(1+∣x∣2),∣σ∣2), and A-4 for bn,σnb_n,\sigma_nbn​,σn​ with a constant uniform in nnn.

Formalization targets

Goal: Theorem 2 (p. 6)

Under A-2, A-4–A-6, the p\mathfrak pp-condition and p1>2p_1>2p1​>2, there is a constant CCC independent of nnn with

sup⁡0≤t≤TE[∣X(t)−Xn(t)∣p]≤Cn−p/2(n≥1).(2.13)\sup_{0\le t\le T}\mathbb E\big[|X(t)-X_n(t)|^p\big]\le Cn^{-p/2}\qquad(n\ge1).\qquad(2.13)0≤t≤Tsup​E[∣X(t)−Xn​(t)∣p]≤Cn−p/2(n≥1).(2.13)

The constant is existential and may depend on all data except nnn.

Milestones

  1. Lemma 2 (p. 8): sup⁡tE∣X(t)∣p\sup_t\mathbb E|X(t)|^psupt​E∣X(t)∣p and sup⁡n≥1sup⁡tE∣Xn(t)∣p\sup_{n\ge1}\sup_t\mathbb E|X_n(t)|^psupn≥1​supt​E∣Xn​(t)∣p are finite for 0<p≤p00<p\le p_00<p≤p0​, under A-1–A-5, B-2, B-3.
  2. Lemma 3 (p. 15): for Model 2, E∫0T∣b(s,Xn(κn(s)))−bn(s,Xn(κn(s)))∣p ds≤Cn−αp\mathbb E\int_0^T|b(s,X_n(\kappa_n(s)))-b_n(s,X_n(\kappa_n(s)))|^p\,ds\le Cn^{-\alpha p}E∫0T​∣b(s,Xn​(κn​(s)))−bn​(s,Xn​(κn​(s)))∣pds≤Cn−αp, and the same for σ\sigmaσ.
  3. Lemma 4 (p. 16): sup⁡tE∣Xn(t)−Xn(κn(t))∣p≤Cn−p/2\sup_t\mathbb E|X_n(t)-X_n(\kappa_n(t))|^p\le Cn^{-p/2}supt​E∣Xn​(t)−Xn​(κn​(t))∣p≤Cn−p/2.

Significance

Theorem 2 shows that an explicit scheme costing one coefficient evaluation per step attains the same strong order as the classical Euler method under global Lipschitz conditions. It does so for drift and diffusion coefficients that grow polynomially, and for moment orders ppp that are small relative to p0p_0p0​ and p1p_1p1​. Strong Lp\mathcal L^pLp rates are what multilevel Monte Carlo needs: the variance of level corrections is controlled by the L2\mathcal L^2L2 rate. When lll could be taken to be 000 the statement specializes to the classical globally Lipschitz rate (Remark 6).

The result is proved on paper; it has no machine-checked proof. Stochastic integrals are not in Mathlib; this mission uses the Brownian Itô-integral relation published by the Ethier–Kurtz series on the platform. A formal proof would give the first verified strong convergence rate for any Euler-type scheme. Along the way it produces reusable moment bounds for tamed schemes and the LpL^pLp control of one-step increments.

Difficulty

The obvious argument applies Itô's formula to ∣X−Xn∣p|X-X_n|^p∣X−Xn​∣p and closes a Gronwall inequality. Two steps of it fail without taming. First, the error splits into the monotone part, handled by A-6, and cross terms such as ∣Xn(s)−Xn(κn(s))∣p|X_n(s)-X_n(\kappa_n(s))|^p∣Xn​(s)−Xn​(κn​(s))∣p multiplied by (1+∣Xn∣2l)p/2(1+|X_n|^{2l})^{p/2}(1+∣Xn​∣2l)p/2; they are controlled only through moments of XnX_nXn​ of order well above ppp that are uniform in nnn, which the untamed scheme lacks. Second, the taming itself introduces an error b−bnb-b_nb−bn​ that must be shown to be O(n−1/2)O(n^{-1/2})O(n−1/2) in Lp\mathcal L^pLp and not merely o(1)o(1)o(1). The exponents in the p\mathfrak pp-condition are exactly what the Hölder splittings of these two terms require.

Formalization scope

  • Processes. Solutions are hypotheses, not constructed: XXX solves (2.1) and each XnX_nXn​ solves (2.2), on one probability space, with one WWW and one ξ\xiξ, on [0,T][0,T][0,T] only. An Itô process is a measurable process adapted to the null-set completion of the filtration, with continuous paths on [0,T][0,T][0,T], and a.s. for all t≤Tt\le Tt≤T simultaneously X(t)=ξ+∫0tB ds+∫0tS dWX(t)=\xi+\int_0^t B\,ds+\int_0^t S\,dWX(t)=ξ+∫0t​Bds+∫0t​SdW. The stochastic integral is the platform relation EthierKurtz_HasBrownianItoIntegral, with the integrand cut off after TTT.
  • Expectations and suprema are computed in [0,∞][0,\infty][0,∞] (lower Lebesgue integrals), so no junk value of a non-integrable Bochner integral or of an unbounded real supremum enters. Rates are stated as ≤Cn−p/2\le Cn^{-p/2}≤Cn−p/2 for every n≥1n\ge1n≥1, with CCC real and quantified after all data.
  • Added hypotheses, disclosed. Theorem 2 carries p1>2p_1>2p1​>2: the printed proof covers 2≤p<p12\le p<p_12≤p<p1​ and reduces p<2p<2p<2 to p=2p=2p=2, which the p\mathfrak pp-condition admits exactly when p1>2p_1>2p1​>2. Lemma 4 carries B-3, which its proof uses through Lemma 2. Lemma 2 drops the stated dependence of CCC on (p,T,K,E∣X(0)∣p)(p,T,K,\mathbb E|X(0)|^p)(p,T,K,E∣X(0)∣p) and keeps only its finiteness uniformly in nnn.
  • Ruling out trivialization. The hypotheses are satisfiable, e.g. by b=σ=0b=\sigma=0b=σ=0 with constant solutions and p0=6p_0=6p0​=6, p1=3p_1=3p1​=3, l=1l=1l=1, p=2p=2p=2. The rate is a bound for every n≥1n\ge1n≥1, not an eventual or o(1)o(1)o(1) statement. A proof that uses a constant depending on nnn, or treats only X≡XnX\equiv X_nX≡Xn​, does not prove the goal.
  • Infrastructure needed: Itô's formula for ∣x∣p|x|^p∣x∣p against the platform's Itô integral, the Burkholder–Davis–Gundy inequality (or its Lp\mathcal L^pLp moment form), the zero-expectation property of Itô integrals of square-integrable integrands, and Gronwall's lemma in integral form. These are reusable far beyond this mission, and contributions of any of them are welcome.

Selected references

  • S. Sabanis, Euler approximations with varying coefficients: the case of superlinearly growing diffusion coefficients, Ann. Appl. Probab. 26(4), 2016. https://arxiv.org/abs/1308.1796 (v4)
  • M. Hutzenthaler, A. Jentzen, P. E. Kloeden, Strong and weak divergence in finite time of Euler's method for stochastic differential equations with non-globally Lipschitz continuous coefficients, Proc. R. Soc. A 467, 2011. https://doi.org/10.1098/rspa.2010.0348
  • M. Hutzenthaler, A. Jentzen, P. E. Kloeden, Strong convergence of an explicit numerical method for SDEs with non-globally Lipschitz continuous coefficients, Ann. Appl. Probab. 22, 2012. https://mathscinet.ams.org/mathscinet-getitem?mr=MR2985171
  • M. Hutzenthaler, A. Jentzen, Numerical approximations of stochastic differential equations with non-globally Lipschitz continuous coefficients, Mem. Amer. Math. Soc. 236(1112), 2015. https://doi.org/10.1090/memo/1112
  • S. Sabanis, A note on tamed Euler approximations, Electron. Commun. Probab. 18, 2013. https://mathscinet.ams.org/mathscinet-getitem?mr=MR3070913
  • D. J. Higham, X. Mao, A. M. Stuart, Strong convergence of Euler-type methods for nonlinear stochastic differential equations, SIAM J. Numer. Anal. 40, 2002. https://mathscinet.ams.org/mathscinet-getitem?mr=MR1949404
13 thms1 active userReviewed
Numerical AnalysisProbability·Captain: mikedeng1

Euler Approximations with Varying Coefficients I: Lp-Convergence of Explicit Euler Schemes under Local MonotonicityResearch Paper

Motivation

Stochastic differential equations (SDEs) whose coefficients grow faster than linearly appear throughout applied probability: population and epidemic models with cubic damping, Langevin dynamics with non-quadratic potentials, and stochastic volatility models such as the 3/2-model used for pricing VIX options (Goard–Mazur 2013). Their solutions are almost never available in closed form, so they are simulated, and the method of choice is the explicit Euler–Maruyama scheme, because it is cheap and easy to implement.

For superlinearly growing coefficients the classical explicit scheme fails: its moments can diverge even when those of the true solution are finite, so it does not converge in Lp\mathcal L^pLp. Implicit schemes repair this at a higher computational cost (Higham–Mao–Stuart 2002). A second repair is to keep the scheme explicit but modify ("tame") its coefficients by an amount that vanishes as the step size goes to zero.

Timeline.

  • 2002: Higham, Mao and Stuart prove strong convergence of implicit Euler-type methods under a one-sided Lipschitz condition (MR1949404).
  • 2012: Hutzenthaler, Jentzen and Kloeden introduce the tamed Euler scheme for superlinearly growing drift and globally Lipschitz diffusion (Ann. Appl. Probab. 22, MR2985171).
  • 2013: Sabanis gives a short proof of convergence of tamed schemes with rate, again for superlinear drift (Electron. Commun. Probab. 18, MR3070913); Gyöngy and Sabanis prove convergence in probability of Euler approximations under local monotonicity conditions (Appl. Math. Optim. 68, MR3131501).
  • 2015: Hutzenthaler and Jentzen obtain Lp\mathcal L^pLp-convergence of explicit schemes with superlinear diffusion coefficients, in the 3/2-model only for p<1/2p<1/2p<1/2 (Mem. Amer. Math. Soc. 236, no. 1112).
  • 2016: Sabanis treats superlinearly growing drift and diffusion coefficients under a local monotonicity condition, with Lp\mathcal L^pLp-convergence for every p<p0p<p_0p<p0​ (Ann. Appl. Probab. 26, arXiv:1308.1796). This mission formalizes that Lp\mathcal L^pLp-convergence theorem.

Setting

Fix a filtered probability space (Ω,{Ft}t≥0,F,P)(\Omega,\{\mathcal F_t\}_{t\ge0},\mathcal F,P)(Ω,{Ft​}t≥0​,F,P) satisfying the usual conditions, a Wiener martingale WWW in Rd1\mathbb R^{d_1}Rd1​ (a standard Brownian motion adapted to Ft\mathcal F_tFt​ whose future increments are independent of Ft\mathcal F_tFt​), and a horizon T>0T>0T>0. For x∈Rdx\in\mathbb R^dx∈Rd, ∣x∣|x|∣x∣ is the Euclidean norm and xyxyxy the scalar product; for a d×d1d\times d_1d×d1​ matrix, ∣A∣|A|∣A∣ is the Hilbert–Schmidt norm.

The SDE is

dX(t)=b(t,X(t)) dt+σ(t,X(t)) dW(t),t∈[0,T],(2.1)dX(t)=b(t,X(t))\,dt+\sigma(t,X(t))\,dW(t),\qquad t\in[0,T],\tag{2.1}dX(t)=b(t,X(t))dt+σ(t,X(t))dW(t),t∈[0,T],(2.1)

with Borel coefficients b(t,x)∈Rdb(t,x)\in\mathbb R^db(t,x)∈Rd, σ(t,x)∈Rd×d1\sigma(t,x)\in\mathbb R^{d\times d_1}σ(t,x)∈Rd×d1​ and an F0\mathcal F_0F0​-measurable initial value X(0)X(0)X(0).

For n≥1n\ge1n≥1 let κn(t)=⌊nt⌋/n\kappa_n(t)=\lfloor nt\rfloor/nκn​(t)=⌊nt⌋/n, the last grid point of mesh 1/n1/n1/n before ttt. The scheme is

dXn(t)=bn(t,Xn(κn(t))) dt+σn(t,Xn(κn(t))) dW(t),t∈[0,T],(2.2)dX_n(t)=b_n(t,X_n(\kappa_n(t)))\,dt+\sigma_n(t,X_n(\kappa_n(t)))\,dW(t),\qquad t\in[0,T],\tag{2.2}dXn​(t)=bn​(t,Xn​(κn​(t)))dt+σn​(t,Xn​(κn​(t)))dW(t),t∈[0,T],(2.2)

with the same initial value X(0)X(0)X(0) and Borel coefficient sequences bn,σnb_n,\sigma_nbn​,σn​ ("varying coefficients"). Between grid points the coefficients are frozen at Xn(κn(t))X_n(\kappa_n(t))Xn​(κn​(t)), so the scheme is explicit.

The hypotheses, with p0,p1∈[2,∞)p_0,p_1\in[2,\infty)p0​,p1​∈[2,∞): A-1 continuity of bbb in xxx; A-2 local boundedness of bbb; A-3 local monotonicity 2(x−y)(b(t,x)−b(t,y))+(p1−1)∣σ(t,x)−σ(t,y)∣2≤LR∣x−y∣22(x-y)(b(t,x)-b(t,y))+(p_1-1)|\sigma(t,x)-\sigma(t,y)|^2\le L_R|x-y|^22(x−y)(b(t,x)−b(t,y))+(p1​−1)∣σ(t,x)−σ(t,y)∣2≤LR​∣x−y∣2 on ∣x∣,∣y∣≤R|x|,|y|\le R∣x∣,∣y∣≤R; A-4 coercivity 2xb(t,x)+(p0−1)∣σ(t,x)∣2≤K(1+∣x∣2)2xb(t,x)+(p_0-1)|\sigma(t,x)|^2\le K(1+|x|^2)2xb(t,x)+(p0​−1)∣σ(t,x)∣2≤K(1+∣x∣2); A-5 E∣X(0)∣p0<∞\mathbb E|X(0)|^{p_0}<\inftyE∣X(0)∣p0​<∞. For the scheme: B-1 ∫0Tsup⁡∣x∣≤R[∣bn−b∣p0+∣σn−σ∣p0] dt→0\int_0^T\sup_{|x|\le R}[|b_n-b|^{p_0}+|\sigma_n-\sigma|^{p_0}]\,dt\to0∫0T​sup∣x∣≤R​[∣bn​−b∣p0​+∣σn​−σ∣p0​]dt→0 for every RRR; B-2 ∣bn∣≤min⁡(Cnα(1+∣x∣),∣b∣)|b_n|\le\min(Cn^\alpha(1+|x|),|b|)∣bn​∣≤min(Cnα(1+∣x∣),∣b∣) and ∣σn∣2≤min⁡(Cnα(1+∣x∣2),∣σ∣2)|\sigma_n|^2\le\min(Cn^\alpha(1+|x|^2),|\sigma|^2)∣σn​∣2≤min(Cnα(1+∣x∣2),∣σ∣2) for some α∈(0,1/2]\alpha\in(0,1/2]α∈(0,1/2]; B-3 the coercivity bound of A-4 for bn,σnb_n,\sigma_nbn​,σn​, uniformly in nnn.

Formalization targets

Goal: Theorem 1 (p. 5)

Under A-1–A-5 and B-1–B-3 with α∈(0,1/2]\alpha\in(0,1/2]α∈(0,1/2], for every 0<p<p00<p<p_00<p<p0​,

lim⁡n→∞sup⁡0≤t≤TE[∣X(t)−Xn(t)∣p]=0.\lim_{n\to\infty}\sup_{0\le t\le T}\mathbb E\big[|X(t)-X_n(t)|^p\big]=0.n→∞lim​0≤t≤Tsup​E[∣X(t)−Xn​(t)∣p]=0.

No rate is claimed; the statement holds for every coefficient sequence satisfying B-1–B-3, not only for the paper's tamed Models 1 and 2.

Milestones

  • Lemma 1 (p. 7): under A-5, B-2, B-3, sup⁡n≥1sup⁡0≤u≤TE∣Xn(u)∣2<∞\sup_{n\ge1}\sup_{0\le u\le T}\mathbb E|X_n(u)|^2<\inftysupn≥1​sup0≤u≤T​E∣Xn​(u)∣2<∞.
  • Lemma 2 (p. 8): under A-1–A-5, B-2, B-3, for every p≤p0p\le p_0p≤p0​,
sup⁡0≤t≤TE∣X(t)∣p ∨ sup⁡n≥1sup⁡0≤t≤TE∣Xn(t)∣p<∞.\sup_{0\le t\le T}\mathbb E|X(t)|^p\ \vee\ \sup_{n\ge1}\sup_{0\le t\le T}\mathbb E|X_n(t)|^p<\infty.0≤t≤Tsup​E∣X(t)∣p ∨ n≥1sup​0≤t≤Tsup​E∣Xn​(t)∣p<∞.
  • Theorem 4 (p. 6): under A-1–A-4 and B-1, sup⁡0≤t≤T∣Xn(t)−X(t)∣→0\sup_{0\le t\le T}|X_n(t)-X(t)|\to0sup0≤t≤T​∣Xn​(t)−X(t)∣→0 in probability.

Significance

Theorem 1 gives Lp\mathcal L^pLp-convergence of a whole class of explicit schemes, with no global Lipschitz condition on either coefficient, for every ppp below the coercivity order p0p_0p0​. In the 3/2-model of the introduction (p1=3.5p_1=3.5p1​=3.5, p0=6p_0=6p0​=6), earlier explicit results gave Lp\mathcal L^pLp-convergence only for p<1/2p<1/2p<1/2 (Hutzenthaler–Jentzen 2015, §4.10.3); Theorem 1 gives it for all p<6p<6p<6. The theorem is also the qualitative base on which the paper's rate results (Theorems 2 and 3, separate missions of this series) are built, and it justifies Monte Carlo estimates of moments computed with such schemes.

The result is proved in the paper. It has, to our knowledge, no machine-checked proof anywhere; Mathlib has Brownian motion but no stochastic integral, and the Itô integral used here is the relational definition published by the Ethier–Kurtz series on this platform. A formal proof needs Itô's formula for ∣x∣p|x|^p∣x∣p, Gronwall's lemma for moment functions, the convergence in probability of Theorem 4 (which the paper cites from Gyöngy–Sabanis rather than proving), and a uniform-integrability argument. Each of these is reusable well beyond this paper.

Difficulty

The natural first idea, estimating E∣X(t)−Xn(t)∣p\mathbb E|X(t)-X_n(t)|^pE∣X(t)−Xn​(t)∣p directly by Itô's formula and Gronwall, fails: under only local monotonicity the difference of drifts cannot be bounded by ∣X−Xn∣|X-X_n|∣X−Xn​∣ with a constant uniform in the state, so the Gronwall constant blows up. The route through convergence in probability and uniform moment bounds is forced, and the hard step is Lemma 2: bounding E∣Xn(t)∣p0\mathbb E|X_n(t)|^{p_0}E∣Xn​(t)∣p0​ uniformly in nnn. The coefficients of the scheme may grow like nαn^\alphanα, and the frozen argument Xn(κn(s))X_n(\kappa_n(s))Xn​(κn​(s)) produces a correction term E∫∣Xn(s)∣p0−2(Xn(s)−Xn(κn(s)))bn ds\mathbb E\int|X_n(s)|^{p_0-2}(X_n(s)-X_n(\kappa_n(s)))b_n\,dsE∫∣Xn​(s)∣p0​−2(Xn​(s)−Xn​(κn​(s)))bn​ds whose control is exactly where the restriction α≤1/2\alpha\le1/2α≤1/2 enters. For fixed nnn finiteness of moments is easy (linear growth); uniformity in nnn is the content.

Formalization scope

States are EuclideanSpace ℝ (Fin d); diffusion values are EuclideanSpace ℝ (Fin d × Fin d₁), whose norm is the Hilbert–Schmidt norm. Time is ℝ≥0; coefficients are uncurried maps on ℝ≥0 × ℝ^d, assumed Borel measurable. An Itô process on [0,T][0,T][0,T] is a measurable process, adapted to the filtration augmented by all PPP-null sets, with continuous paths on [0,T][0,T][0,T], satisfying the integral equation almost surely simultaneously for all t≤Tt\le Tt≤T; the stochastic integral is the platform relation EthierKurtz.HasBrownianItoIntegral, applied coordinatewise with the integrand cut off after TTT. Nothing is required of the processes after TTT, so no existence beyond the horizon is assumed. Right-continuity of the filtration is a hypothesis; completeness is replaced by adaptedness to the augmented filtration, which is weaker, so the formal theorems are at least as strong as the paper's.

The solutions XXX and (Xn)n≥1(X_n)_{n\ge1}(Xn​)n≥1​ are hypotheses, all on one probability space with one Wiener process and one initial value; the theorems do not construct them. Expectations are lower Lebesgue integrals in [0,∞][0,\infty][0,∞] and suprema over ttt are taken in [0,∞][0,\infty][0,∞], so no statement can hold because an expectation or a supremum takes a default value. B-1's integrand, a supremum over an uncountable ball, is required to be a.e. measurable in ttt, as the paper's Lebesgue integral presupposes. The exponent in Theorem 1 and Lemma 2 is restricted to p>0p>0p>0, the paper's convention for Lp\mathcal L^pLp. The dependence clauses "C:=C(T,K,E∣X(0)∣2)C:=C(T,K,\mathbb E|X(0)|^2)C:=C(T,K,E∣X(0)∣2)" (Lemma 1) and "C:=C(p,T,K,E∣X(0)∣p)C:=C(p,T,K,\mathbb E|X(0)|^p)C:=C(p,T,K,E∣X(0)∣p)" (Lemma 2) are not formalized: only finiteness, i.e. a bound independent of nnn, is stated, which is what the proofs establish. Theorem 4 is stated with exactly its printed hypotheses.

A trivializing formalization, in which the moment bounds or the limit hold because the expectation or the supremum defaults to 000, or because the hypotheses on the solutions cannot be met, is ruled out: all quantities live in [0,∞][0,\infty][0,∞], and a sorry-free check shows every hypothesis is satisfiable (zero coefficients, constant solutions).

Contributions welcome: Itô's formula for the platform's integral relation, Burkholder–Davis–Gundy or Doob inequalities for it, a Gronwall lemma for measurable moment functions, and the Gyöngy–Sabanis convergence-in-probability theorem.

Selected references

  • S. Sabanis, Euler approximations with varying coefficients: the case of superlinearly growing diffusion coefficients, Ann. Appl. Probab. 26(4), 2016. https://arxiv.org/abs/1308.1796 (v4)
  • I. Gyöngy, S. Sabanis, A note on Euler approximations for stochastic differential equations with delay, Appl. Math. Optim. 68, 2013. https://mathscinet.ams.org/mathscinet-getitem?mr=MR3131501
  • M. Hutzenthaler, A. Jentzen, P. E. Kloeden, Strong convergence of an explicit numerical method for SDEs with non-globally Lipschitz continuous coefficients, Ann. Appl. Probab. 22, 2012. https://mathscinet.ams.org/mathscinet-getitem?mr=MR2985171
  • M. Hutzenthaler, A. Jentzen, Numerical approximations of stochastic differential equations with non-globally Lipschitz continuous coefficients, Mem. Amer. Math. Soc. 236(1112), 2015. https://doi.org/10.1090/memo/1112
  • S. Sabanis, A note on tamed Euler approximations, Electron. Commun. Probab. 18, 2013. https://mathscinet.ams.org/mathscinet-getitem?mr=MR3070913
  • D. J. Higham, X. Mao, A. M. Stuart, Strong convergence of Euler-type methods for nonlinear stochastic differential equations, SIAM J. Numer. Anal. 40, 2002. https://mathscinet.ams.org/mathscinet-getitem?mr=MR1949404
12 thms1 active userReviewed
Operations ResearchProbability·Captain: mikedeng1

Diffusion approximations for open queueing networks with service interruptions 2: jump-diffusion heavy-traffic limit for long up and down timesResearch Paper

Motivation

Servers in manufacturing lines, communication links and service systems break down, are taken offline for maintenance, or go on vacation. When the interruptions are rare but long, they dominate congestion. A single down period can build a backlog that takes a long time to clear, and in a network that backlog propagates downstream. Standard heavy-traffic diffusion approximations, which describe queue lengths by reflected Brownian motion, do not capture this effect.

Chen and Whitt (Queueing Systems 13, 1993) identify a regime in which the effect survives in the limit. Up times are of order nnn and down times of order n\sqrt nn​, while the load is within 1/n1/\sqrt n1/n​ of capacity. Under the diffusion scaling each down period then becomes a jump, and the limit of the queue-length process is a reflected jump-diffusion. The paper generalises the single-station result of Kella and Whitt (Adv. Appl. Probab. 22, 1990; reference [22] of the paper) to open networks.

Timeline:

  • 1981: Harrison and Reiman define the multidimensional reflection map on continuous paths (Ann. Probab. 9). Reiman (Math. Oper. Res. 9, 1984) extends it to paths with jumps.
  • 1990: Kella and Whitt prove the one-station jump-diffusion limit for long up and down times.
  • 1991: Chen and Mandelbaum give fluid and diffusion limits of open networks without interruptions (Math. Oper. Res. 16 and Ann. Probab. 19; references [5], [6] of the paper).
  • 1993: Chen and Whitt prove the network case with interruptions (this mission), in Skorohod's M1M_1M1​ topology.

Setting

A network has JJJ single-server stations. Customers arrive from outside station jjj according to a counting process AjA_jAj​. Station jjj completes Sj(t)S_j(t)Sj​(t) services in its first ttt units of busy time. The lllth departure from station kkk is routed to station jjj when the indicator χkj(l)=1\chi_{kj}(l)=1χkj​(l)=1, and Rkj(m)=∑l≤mχkj(l)R_{kj}(m)=\sum_{l\le m}\chi_{kj}(l)Rkj​(m)=∑l≤m​χkj​(l) counts such departures. Station jjj alternates up periods u1j,u2j,…u^j_1,u^j_2,\dotsu1j​,u2j​,… and down periods d1j,d2j,…d^j_1,d^j_2,\dotsd1j​,d2j​,…, starting up, and Dj(t)D_j(t)Dj​(t) is its cumulative down time in [0,t][0,t][0,t]. With a work-conserving discipline, the queue length ZZZ and the busy time BBB satisfy

Zj(t)=Zj(0)+Aj(t)+∑kRkj(Sk(Bk(t)))−Sj(Bj(t)),Bj(t)=∫0t1[Zj(s)>0, j up at s] ds,Z_j(t)=Z_j(0)+A_j(t)+\sum_{k}R_{kj}\big(S_k(B_k(t))\big)-S_j(B_j(t)),\qquad B_j(t)=\int_0^t 1[Z_j(s)>0,\ j\text{ up at }s]\,ds ,Zj​(t)=Zj​(0)+Aj​(t)+k∑​Rkj​(Sk​(Bk​(t)))−Sj​(Bj​(t)),Bj​(t)=∫0t​1[Zj​(s)>0, j up at s]ds,

and the idle time is Yj(t)=t−Dj(t)−Bj(t)Y_j(t)=t-D_j(t)-B_j(t)Yj​(t)=t−Dj​(t)−Bj​(t).

The reflection map (ψ,ϕ)(\psi,\phi)(ψ,ϕ) associated with a matrix QQQ takes a path xxx to the pair (y,z)(y,z)(y,z) with z=x+(I−Q)y≥0z=x+(I-Q)y\ge 0z=x+(I−Q)y≥0, yyy nondecreasing, and yjy_jyj​ increasing only when zj=0z_j=0zj​=0.

A sequence of networks is indexed by nnn. The arrival, service and routing processes satisfy functional central limit theorems with rates λn→λ\lambda^n\to\lambdaλn→λ and μn→μ\mu^n\to\muμn→μ at speed 1/n1/\sqrt n1/n​. Up and down times scale as (ukj,n/n, dkj,n/n)⇒(ukj,dkj)(u^{j,n}_k/n,\ d^{j,n}_k/\sqrt n)\Rightarrow(u^j_k,d^j_k)(ukj,n​/n, dkj,n​/n​)⇒(ukj​,dkj​). The network is balanced, λ=[I−Pt]μ\lambda=[I-P^{\mathsf t}]\muλ=[I−Pt]μ, with PPP the routing matrix. The limit down time D^j(t)\hat D_j(t)D^j​(t) is the sum of dkjd^j_kdkj​ over the up periods completed by time ttt, which is a pure-jump process.

The M1M_1M1​ topology on paths with jumps compares completed graphs, in which each jump is filled in by the straight segment from x(t−)x(t-)x(t−) to x(t)x(t)x(t), through their monotone parametrisations.

Formalization targets

Goal: Theorem 4.1, case J=1J=1J=1

For a single station with feedback probability p∈[0,1)p\in[0,1)p∈[0,1), with Z^n(t)=n−1/2Zn(nt)\hat Z^n(t)=n^{-1/2}Z^n(nt)Z^n(t)=n−1/2Zn(nt), B^n(t)=n−1/2[Bn(nt)−nt]\hat B^n(t)=n^{-1/2}[B^n(nt)-nt]B^n(t)=n−1/2[Bn(nt)−nt], Y^n(t)=n−1/2Yn(nt)\hat Y^n(t)=n^{-1/2}Y^n(nt)Y^n(t)=n−1/2Yn(nt) and D^n(t)=n−1/2Dn(nt)\hat D^n(t)=n^{-1/2}D^n(nt)D^n(t)=n−1/2Dn(nt),

(Z^n,B^n,Y^n,D^n)⇒(Z^,B^,Y^,D^)in D((0,∞),R4,M1).(\hat Z^n,\hat B^n,\hat Y^n,\hat D^n)\Rightarrow(\hat Z,\hat B,\hat Y,\hat D)\quad\text{in }D((0,\infty),\mathbb R^{4},M_1).(Z^n,B^n,Y^n,D^n)⇒(Z^,B^,Y^,D^)in D((0,∞),R4,M1​).

Here Z^=ϕ(X^)\hat Z=\phi(\hat X)Z^=ϕ(X^), Y^=μ−1ψ(X^)\hat Y=\mu^{-1}\psi(\hat X)Y^=μ−1ψ(X^) and B^=−D^−Y^\hat B=-\hat D-\hat YB^=−D^−Y^, with Q=pQ=pQ=p and

X^(t)=Z^(0)+ξ^(t)+(cλ−(1−p)cμ)t+(1−p)μD^(t).\hat X(t)=\hat Z(0)+\hat\xi(t)+\big(c_\lambda-(1-p)c_\mu\big)t+(1-p)\mu\hat D(t).X^(t)=Z^(0)+ξ^​(t)+(cλ​−(1−p)cμ​)t+(1−p)μD^(t).

The paper states Theorem 4.1 for JJJ stations, with the analogous formulas and Q=PtQ=P^{\mathsf t}Q=Pt. The mission's goal is its case J=1J=1J=1 (see Formalization scope).

Milestones

  1. Lemma 4.1: D^n⇒D^\hat D^n\Rightarrow\hat DD^n⇒D^ in D((0,∞),RJ,M1)D((0,\infty),\mathbb R^J,M_1)D((0,∞),RJ,M1​).
  2. Lemma 4.2: n−1Bjn(nt)→tn^{-1}B^n_j(nt)\to tn−1Bjn​(nt)→t u.o.c.
  3. Eq. (4.24): n−1/2ξn(nt)→ξ^(t)n^{-1/2}\xi^n(nt)\to\hat\xi(t)n−1/2ξn(nt)→ξ^​(t) u.o.c.
  4. Eqs. (4.28)–(4.29): (n−1/2Xn(nt), n−1/2Dn(nt))→(X^,D^)\big(n^{-1/2}X^n(nt),\,n^{-1/2}D^n(nt)\big)\to(\hat X,\hat D)(n−1/2Xn(nt),n−1/2Dn(nt))→(X^,D^) jointly in M1M_1M1​.
  5. The almost-sure form of Theorem 4.1 on a Skorohod representation space, case J=1J=1J=1.

Significance

The theorem yields a tractable approximation for networks with rare long interruptions: a reflected Lévy-type process driven by a Brownian part and a compound jump part. Its distribution can be studied through the reflection map. The jump directions [I−Pt]diag⁡(μ)ej[I-P^{\mathsf t}]\operatorname{diag}(\mu)e_j[I−Pt]diag(μ)ej​ make explicit how an outage at one station drains its downstream stations while its own queue builds up. Remark (4.3) of the paper derives a diffusion analogue of Little's law from the same limit.

The theorem is proved in the paper, and no part of it has been formalized. A formalization would produce the first machine-checked M1M_1M1​ topology on paths with jumps, a heavy-traffic limit theorem for a queueing network, and the random-time-change argument for counting processes.

Difficulty

The obvious argument chains three facts: the primitive processes converge, hence so does the scaled free process XXX, and the reflection map is continuous. Two steps break. First, subtraction is not continuous in M1M_1M1​ when the two paths jump at the same time in opposite directions, so the joint convergence of D^n\hat D^nD^n across stations needs (4.11) and one common parametrisation. Second, the reflection map is Lipschitz in the uniform topology, but the uniform topology cannot see jumps that occur at nearby times. Carrying the convergence through the reflection map in the M1M_1M1​ topology requires controlling how the regulator and the regulated process move along each jump segment of X^\hat XX^, jointly for all coordinates.

Formalization scope

Stations are Fin J; the network index is n : ℕ, and only n→∞n\to\inftyn→∞ enters. Durations are indexed from 000 in Lean. The queue length is integer valued, (3.2) is computed in Z\mathbb ZZ, and a solution satisfies Z≥0Z\ge 0Z≥0.

  • Solutions, not constructions. Every statement quantifies over all solutions (Zn,Bn)(Z^n,B^n)(Zn,Bn) of (3.2)–(3.3) and all reflection pairs of X^\hat XX^. Existence and uniqueness are asserted in the paper by citation and are not assumed or proved here.
  • M1M_1M1​. Parametric representations are monotone in the order of the completed graph, which is the standard definition. The page states only that the time component is nondecreasing. Convergence on (0,∞)(0,\infty)(0,∞) means convergence on every [a,b][a,b][a,b] with 0<a<b0<a<b0<a<b continuity points of the limit. Jump segments are segments in Rd\mathbb R^dRd (strong M1M_1M1​). Convergence in D((0,∞),⋅,M1)D((0,\infty),\cdot,M_1)D((0,∞),⋅,M1​) includes the requirement that every path be càdlàg on (0,∞)(0,\infty)(0,∞), so a copy of the limit that is continuous nowhere cannot satisfy the continuity-point condition vacuously.
  • Weak convergence is in coupling form: one probability space carries copies with the right laws that converge almost surely. The limits in (4.1)–(4.4) are continuous, so there the mode is u.o.c.
  • Corrected printed errors. Lemma 4.2 is stated with n−1n^{-1}n−1 in place of the printed n−1/2n^{-1/2}n−1/2, as in its proof. The map on p. 346 is read as ϕ(X)=Z\phi(X)=Zϕ(X)=Z, ψ(X)=diag⁡(μ)Y\psi(X)=\operatorname{diag}(\mu)Yψ(X)=diag(μ)Y, following (4.13). The reflection map allows y(0)≥0y(0)\ge 0y(0)≥0, because X^(0)\hat X(0)X^(0) may leave the orthant when D^(0)>0\hat D(0)>0D^(0)>0; when x(0)≥0x(0)\ge 0x(0)≥0 this agrees with (2.2).
  • Added hypotheses. The processes Zn,Bn,Z^,Y^Z^n,B^n,\hat Z,\hat YZn,Bn,Z^,Y^ are assumed to be stochastic processes (measurable at each time). All networks share one probability space, so that the routing is literally common. No independence is assumed.
  • The goal is the case J=1J=1J=1 of Theorem 4.1. The printed theorem claims strong M1M_1M1​ convergence, with one parametric representation for all 4J4J4J coordinates, for every JJJ. For J≥2J\ge 2J≥2 that claim fails: during an upstream outage, a downstream queue that empties part-way through the jump bends the prelimit graph of (D^j,Y^k)(\hat D_j,\hat Y_k)(D^j​,Y^k​), while the limit's completed graph is a straight segment. For J=1J=1J=1 every coordinate moves linearly through each jump. The milestones Lemma 4.1, Lemma 4.2, (4.24) and (4.28)–(4.29) are stated for general JJJ, and the almost-sure core of the proof for J=1J=1J=1.

A trivializing formalization is excluded. The hypotheses are satisfiable (for example by deterministic arrival and service processes), N^\hat NN^ is used only when ∑kukj=∞\sum_ku^j_k=\infty∑k​ukj​=∞, and laws are compared only for measurable path maps.

Needed infrastructure: the Skorohod space with the M1M_1M1​ topology and its characterization on (0,∞)(0,\infty)(0,∞), continuity of addition and of composition with continuous time changes, the multidimensional reflection map on paths with jumps, and a Skorohod representation argument. Proofs of Lemma 4.1 and Lemma 4.2 are welcome independently.

Selected references

  • H. Chen, W. Whitt, Diffusion approximations for open queueing networks with service interruptions, Queueing Systems 13 (1993) 335–359. https://doi.org/10.1007/BF01149260
  • O. Kella, W. Whitt, Diffusion approximations for queues with server vacations, Adv. Appl. Probab. 22 (1990) 706–729 (reference [22] of the paper).
  • J. M. Harrison, M. I. Reiman, Reflected Brownian motion on an orthant, Ann. Probab. 9 (1981) 302–308. https://doi.org/10.1214/aop/1176994472
  • H. Chen, A. Mandelbaum, Discrete flow networks: diffusion approximations and bottlenecks, Ann. Probab. 19 (1991) 1463–1519 (reference [6] of the paper).
  • M. I. Reiman, Open queueing networks in heavy traffic, Math. Oper. Res. 9 (1984) 441–458 (reference [25] of the paper).
  • W. Whitt, Some useful functions for functional limit theorems, Math. Oper. Res. 5 (1980) 67–85. https://doi.org/10.1287/moor.5.1.67
  • A. V. Skorohod, Limit theorems for stochastic processes, Theory Probab. Appl. 1 (1956) 261–290. https://doi.org/10.1137/1101022
10 thms1 active userReviewed
Control TheoryDynamical SystemsOperations Research+1·Captain: mikedeng1

Stabilization of Hybrid Systems by Feedback Control Based on Discrete-Time State Observations II: Mean-Square and Almost Sure Exponential StabilityResearch Paper

Motivation

Many engineered systems switch between a finite number of operating modes at random times: a power grid after a line failure, a networked controller whose links drop, a manufacturing plant whose machines break down and are repaired. A standard model for such systems is a hybrid stochastic differential equation, also called an SDE with Markovian switching: the state follows an Itô equation whose coefficients depend on a mode that evolves as a continuous-time Markov chain. The monograph of Mao and Yuan (Stochastic Differential Equations with Markovian Switching, 2006) develops the stability theory of these equations.

A controller that stabilizes such a system usually needs the current state. In practice the state is sampled: it is observed at times 0,τ,2τ,…0,\tau,2\tau,\dots0,τ,2τ,… and the control is held between observations. Mao (Automatica 49, 2013) showed that, under a global Lipschitz condition on the drift and diffusion, a feedback control based on discrete-time observations makes a hybrid SDE mean-square exponentially stable when τ\tauτ is small enough. You, Liu, Lu, Mao and Qiu (SIAM J. Control Optim. 53(2), 2015) replaced that condition by local Lipschitz continuity plus linear growth, gave an explicit bound (3.5) on the admissible observation interval, and proved H∞H_\inftyH∞​-stability, asymptotic stability, and, in Section 4, exponential stability in mean square and almost surely with an explicit rate. This mission formalizes that exponential stability result, Theorem 4.2, and the steps of its proof.

Setting

Let (Ω,F,{Ft}t≥0,P)(\Omega,\mathcal F,\{\mathcal F_t\}_{t\ge0},\mathbb P)(Ω,F,{Ft​}t≥0​,P) be a probability space with a filtration satisfying the usual conditions (increasing, right-continuous, F0\mathcal F_0F0​ contains the null sets). On it live an mmm-dimensional {Ft}\{\mathcal F_t\}{Ft​}-Brownian motion www and a right-continuous {Ft}\{\mathcal F_t\}{Ft​}-Markov chain rrr on S={1,…,N}S=\{1,\dots,N\}S={1,…,N} with generator Γ=(γij)\Gamma=(\gamma_{ij})Γ=(γij​) (γij≥0\gamma_{ij}\ge0γij​≥0 for i≠ji\ne ji=j, zero row sums), independent of www. Fix τ>0\tau>0τ>0 and the sampling time δt=[t/τ]τ\delta_t=[t/\tau]\tauδt​=[t/τ]τ. The controlled system is

dx(t)=(f(x(t),r(t),t)+u(x(δt),r(t),t))dt+g(x(t),r(t),t) dw(t),x(0)=x0, r(0)=r0,(2.1)dx(t)=\big(f(x(t),r(t),t)+u(x(\delta_t),r(t),t)\big)dt+g(x(t),r(t),t)\,dw(t),\qquad x(0)=x_0,\ r(0)=r_0,\tag{2.1}dx(t)=(f(x(t),r(t),t)+u(x(δt​),r(t),t))dt+g(x(t),r(t),t)dw(t),x(0)=x0​, r(0)=r0​,(2.1)

with f,u:Rn×S×R+→Rnf,u:\mathbb R^n\times S\times\mathbb R_+\to\mathbb R^nf,u:Rn×S×R+​→Rn and g:Rn×S×R+→Rn×mg:\mathbb R^n\times S\times\mathbb R_+\to\mathbb R^{n\times m}g:Rn×S×R+​→Rn×m.

The hypotheses are:

  • Assumption 2.1: f,gf,gf,g locally Lipschitz in xxx, and ∣f(x,i,t)∣≤K1∣x∣|f(x,i,t)|\le K_1|x|∣f(x,i,t)∣≤K1​∣x∣, ∣g(x,i,t)∣≤K2∣x∣|g(x,i,t)|\le K_2|x|∣g(x,i,t)∣≤K2​∣x∣ (∣g∣|g|∣g∣ the trace norm).
  • Assumption 2.2: ∣u(x,i,t)−u(y,i,t)∣≤K3∣x−y∣|u(x,i,t)-u(y,i,t)|\le K_3|x-y|∣u(x,i,t)−u(y,i,t)∣≤K3​∣x−y∣ and u(0,i,t)=0u(0,i,t)=0u(0,i,t)=0.
  • Assumption 3.1: there are U∈C2,1(Rn×S×R+;R+)U\in C^{2,1}(\mathbb R^n\times S\times\mathbb R_+;\mathbb R_+)U∈C2,1(Rn×S×R+​;R+​) and λ1,λ2>0\lambda_1,\lambda_2>0λ1​,λ2​>0 with LU(x,i,t)+λ1∣Ux(x,i,t)∣2≤−λ2∣x∣2\mathcal LU(x,i,t)+\lambda_1|U_x(x,i,t)|^2\le-\lambda_2|x|^2LU(x,i,t)+λ1​∣Ux​(x,i,t)∣2≤−λ2​∣x∣2, where
LU=Ut+Ux[f+u]+12trace⁡[gTUxxg]+∑jγijU(x,j,t).\mathcal LU=U_t+U_x[f+u]+\tfrac12\operatorname{trace}[g^TU_{xx}g]+\sum_j\gamma_{ij}U(x,j,t).LU=Ut​+Ux​[f+u]+21​trace[gTUxx​g]+j∑​γij​U(x,j,t).
  • Assumption 4.1: c1∣x∣2≤U(x,i,t)≤c2∣x∣2c_1|x|^2\le U(x,i,t)\le c_2|x|^2c1​∣x∣2≤U(x,i,t)≤c2​∣x∣2 with c1,c2>0c_1,c_2>0c1​,c2​>0.
  • Condition (3.5): λ2>τK32λ1[2τ(K12+2K32)+K22]\lambda_2>\frac{\tau K_3^2}{\lambda_1}\big[2\tau(K_1^2+2K_3^2)+K_2^2\big]λ2​>λ1​τK32​​[2τ(K12​+2K32​)+K22​] and τ≤14K3\tau\le\frac1{4K_3}τ≤4K3​1​.

Put θ=K32/λ1\theta=K_3^2/\lambda_1θ=K32​/λ1​, λ=λ2−θτ[2τ(K12+2K32)+K22]\lambda=\lambda_2-\theta\tau[2\tau(K_1^2+2K_3^2)+K_2^2]λ=λ2​−θτ[2τ(K12​+2K32​)+K22​] (positive by (3.5)), and

H1=θτ(2τ(K12+2K32)+K22)+24θτ4K341−6τ2K32,H2=12θτ2K32(τK12+K22)1−6τ2K32.H_1=\theta\tau\big(2\tau(K_1^2+2K_3^2)+K_2^2\big)+\frac{24\theta\tau^4K_3^4}{1-6\tau^2K_3^2},\qquad H_2=\frac{12\theta\tau^2K_3^2(\tau K_1^2+K_2^2)}{1-6\tau^2K_3^2}.H1​=θτ(2τ(K12​+2K32​)+K22​)+1−6τ2K32​24θτ4K34​​,H2​=1−6τ2K32​12θτ2K32​(τK12​+K22​)​.

In Lean these are Assumption21, Assumption22, C21, LU, Assumption31, Assumption41, Condition35, theta, lam, H1, H2, rateEquationLHS; the basis is HybridSetup, the Itô integral IsItoIntegral, the sampling time delta, solutions SolvesSampledHybridSDE, and the functional (4.7) Vbar, all in the namespace You2015.Expo.

Formalization targets

Goal: Theorem 4.2 (exponential stability)

Under the hypotheses above, the equation

2τγe2τγ(H1+τH2)+γc2=λ(4.4)2\tau\gamma e^{2\tau\gamma}(H_1+\tau H_2)+\gamma c_2=\lambda\tag{4.4}2τγe2τγ(H1​+τH2​)+γc2​=λ(4.4)

has a unique root γ>0\gamma>0γ>0, and every solution of (2.1) satisfies

lim sup⁡t→∞1tlog⁡(E∣x(t)∣2)≤−γ,lim sup⁡t→∞1tlog⁡∣x(t)∣≤−γ2a.s.\limsup_{t\to\infty}\frac1t\log\big(\mathbb E|x(t)|^2\big)\le-\gamma,\qquad\limsup_{t\to\infty}\frac1t\log|x(t)|\le-\frac\gamma2\quad\text{a.s.}t→∞limsup​t1​log(E∣x(t)∣2)≤−γ,t→∞limsup​t1​log∣x(t)∣≤−2γ​a.s.

for all x0∈Rnx_0\in\mathbb R^nx0​∈Rn, r0∈Sr_0\in Sr0​∈S.

Milestones, in the order the proof uses them

  1. (3.15) E∣x(t)−x(δt)∣2≤2E∫δtt[τ∣f+u(x(δs),⋅)∣2+∣g∣2]ds\mathbb E|x(t)-x(\delta_t)|^2\le2\mathbb E\int_{\delta_t}^t[\tau|f+u(x(\delta_s),\cdot)|^2+|g|^2]dsE∣x(t)−x(δt​)∣2≤2E∫δt​t​[τ∣f+u(x(δs​),⋅)∣2+∣g∣2]ds.
  2. Theorem 3.2 (H∞H_\inftyH∞​-stability): ∫0∞E∣x(s)∣2ds<∞\int_0^\infty\mathbb E|x(s)|^2ds<\infty∫0∞​E∣x(s)∣2ds<∞.
  3. (3.21) E∣x(s)−x(δs)∣2≤3(τK12+K22)1−6τ2K32∫δssE∣x(z)∣2dz+6τ2K321−6τ2K32E∣x(s)∣2\mathbb E|x(s)-x(\delta_s)|^2\le\frac{3(\tau K_1^2+K_2^2)}{1-6\tau^2K_3^2}\int_{\delta_s}^s\mathbb E|x(z)|^2dz+\frac{6\tau^2K_3^2}{1-6\tau^2K_3^2}\mathbb E|x(s)|^2E∣x(s)−x(δs​)∣2≤1−6τ2K32​3(τK12​+K22​)​∫δs​s​E∣x(z)∣2dz+1−6τ2K32​6τ2K32​​E∣x(s)∣2.
  4. (4.11) EVˉ(x^z,r^z,z)≤(H1+τH2)∫z−2τzE∣x(y)∣2dy\mathbb E\bar V(\hat x_z,\hat r_z,z)\le(H_1+\tau H_2)\int_{z-2\tau}^z\mathbb E|x(y)|^2dyEVˉ(x^z​,r^z​,z)≤(H1​+τH2​)∫z−2τz​E∣x(y)∣2dy for z≥2τz\ge2\tauz≥2τ.
  5. (4.14) c1eγtE∣x(t)∣2≤Cc_1e^{\gamma t}\mathbb E|x(t)|^2\le Cc1​eγtE∣x(t)∣2≤C for t≥2τt\ge2\taut≥2τ.
  6. (4.14) ⇒\Rightarrow⇒ (4.3), the mean-square-to-almost-sure transfer cited from Mao–Yuan [23, Theorem 8.8].

Significance

The result. Theorem 4.2 gives a quantitative guarantee: a controller that samples the state every τ\tauτ units makes the switching system decay exponentially, with a rate γ\gammaγ computable from the constants of the assumptions. Asymptotic stability (Section 3 of the paper) says nothing about how fast trajectories settle; the rate is what a designer trades against the sampling cost when choosing τ\tauτ. The almost sure statement concerns individual trajectories, which is what an operator observes.

Formalizing it. The results are proved in the paper; none of them is machine-checked. Mathlib has real Brownian motion but no Itô integral, no stochastic differential equations and no continuous-time Markov chains. The mission therefore also produces a definition layer: a filtration under the usual conditions, a multidimensional {Ft}\{\mathcal F_t\}{Ft​}-Brownian motion, an {Ft}\{\mathcal F_t\}{Ft​}-Markov chain with a given generator, the L2L^2L2 Itô integral of vector-valued integrands, and the solution notion of an SDE with Markovian switching and a sampled-state delay. A related but different layer exists on Prove2Me for Ethier–Kurtz (EthierKurtz_IsStandardBrownian, EthierKurtz_HasBrownianItoIntegral, EthierKurtz_SolvesBrownianSDE); it has no mode switching and no sampled state, so it cannot express (2.1). Formalization also checks the constants: it found that the printed H1H_1H1​ in (4.5) disagrees with the paper's own derivation (see below).

Difficulty

Equation (2.1) is a stochastic differential delay equation whose delay t−δtt-\delta_tt−δt​ is bounded but jumps at every observation time, so the delay-equation stability theorems that require a differentiable delay with derivative below one (Mao–Yuan, p. 285) do not apply. A Lyapunov function of the current state alone leaves the term Ux[u(x(t))−u(x(δt))]U_x[u(x(t))-u(x(\delta_t))]Ux​[u(x(t))−u(x(δt​))], which has no sign and depends on the path over a whole observation interval. An exponential rate requires controlling this delay term with an exponential weight, and the weight inflates the delay contribution by a factor e2τγe^{2\tau\gamma}e2τγ; the rate equation (4.4) records exactly this balance. The almost sure part does not follow from the mean-square part by Chebyshev's inequality at fixed times alone: a pathwise bound needs control of the supremum of ∣x∣|x|∣x∣ over each unit interval, which involves the martingale part of the solution.

Formalization scope

Conventions committed to in Lean:

  • The state space is EuclideanSpace ℝ (Fin n), so ∣x∣|x|∣x∣ is the Euclidean norm; the explicit constants in (3.5), (3.21), (4.5) depend on it. The diffusion ggg is given by its mmm columns and ∣g∣2=∑k∣gk∣2|g|^2=\sum_k|g_k|^2∣g∣2=∑k​∣gk​∣2 (trace norm). Modes are Fin N (0-based). Time is ℝ≥0; time integrals are over subsets of R\mathbb RR at s.toNNReal.
  • Every expectation of a nonnegative quantity (E∣x∣2\mathbb E|x|^2E∣x∣2, EVˉ\mathbb E\bar VEVˉ) and every time integral of one is a lower Lebesgue integral in [0,∞][0,\infty][0,∞], so a non-integrable process cannot produce a junk value 000.
  • Logarithms. The paper's log⁡\loglog takes the value −∞-\infty−∞ at 000. For a finite a(t)≥0a(t)\ge0a(t)≥0, lim sup⁡t→∞1tlog⁡a(t)≤−γ\limsup_{t\to\infty}\frac1t\log a(t)\le-\gammalimsupt→∞​t1​loga(t)≤−γ is stated in the equivalent form "for every γ′<γ\gamma'<\gammaγ′<γ, eventually a(t)≤e−γ′ta(t)\le e^{-\gamma't}a(t)≤e−γ′t". Lean's Real.log 0 = 0 never enters. In (4.3) the almost-sure quantifier is outside the quantifier over γ′\gamma'γ′.
  • Correction of (4.5). The page prints the last term of H1H_1H1​ as 24τ3K34/(1−6τ2K32)24\tau^3K_3^4/(1-6\tau^2K_3^2)24τ3K34​/(1−6τ2K32​). Substituting (3.21) into (4.9), as the proof does, gives 24θτ4K34/(1−6τ2K32)24\theta\tau^4K_3^4/(1-6\tau^2K_3^2)24θτ4K34​/(1−6τ2K32​) (the same computation reproduces the printed H2H_2H2​). The mission uses the corrected H1H_1H1​ in the goal, (4.11) and (4.14). With the printed value the claimed rate could exceed what the proof yields whenever θτ>1\theta\tau>1θτ>1.
  • "The unique root" is a conjunct of the goal (∃! γ>0\exists!\,\gamma>0∃!γ>0); the stability conclusions are stated for every positive root. "(so λ>0\lambda>0λ>0)" is a consequence of (3.5), not a hypothesis. (4.4) is kept as an equality.
  • "The solution of (2.1)" is read as every process satisfying the solution definition: progressively measurable, almost surely continuous paths, E∣x(t)∣2<∞\mathbb E|x(t)|^2<\inftyE∣x(t)∣2<∞ for each ttt, and for each ttt, almost surely, the integral equation with Itô integrals in the L2L^2L2 sense. Existence and uniqueness (cited from Mao–Yuan on p. 908) are not asserted.
  • "An mmm-dimensional Brownian motion" and "a Markov chain with generator Γ\GammaΓ" are read in the Mao–Yuan framework the paper cites: an {Ft}\{\mathcal F_t\}{Ft​}-Brownian motion with independent coordinates and increments independent of the past, and an {Ft}\{\mathcal F_t\}{Ft​}-Markov chain with transition matrix etΓe^{t\Gamma}etΓ. The usual conditions are kept as hypotheses.
  • "Locally Lipschitz" is uniform in the mode and time on each ball. C2,1C^{2,1}C2,1 carries its derivatives Ut,Ux,UxxU_t,U_x,U_{xx}Ut​,Ux​,Uxx​ as witnesses tied to UUU by derivative relations and joint continuity.
  • "τ>0\tau>0τ>0 sufficiently small for (3.5)" means every τ>0\tau>0τ>0 satisfying both inequalities of (3.5). U,λ1,λ2,c1,c2,τU,\lambda_1,\lambda_2,c_1,c_2,\tauU,λ1​,λ2​,c1​,c2​,τ are data. The constant CCC of (4.14) is chosen after x0x_0x0​, r0r_0r0​, the solution and γ\gammaγ, and before ttt.
  • (3.15), (3.21) and the transfer (4.14) ⇒\Rightarrow⇒ (4.3) are stated under fewer hypotheses than the surrounding proof has (Assumptions 2.1, 2.2, τ>0\tau>0τ>0, and for (3.21) τ≤1/(4K3)\tau\le1/(4K_3)τ≤1/(4K3​)), because their derivations use no more. Vˉ\bar VVˉ of (4.7) is used only for z≥2τz\ge2\tauz≥2τ, where no extension of the solution to negative times is needed. Other misprints on the page (g(x,i,s)=f(x,i,0)g(x,i,s)=f(x,i,0)g(x,i,s)=f(x,i,0) on p. 909, V(x^0,r^0,t)V(\hat x_0,\hat r_0,t)V(x^0​,r^0​,t) for V(x^0,r^0,0)V(\hat x_0,\hat r_0,0)V(x^0​,r^0​,0) on p. 918, the swapped ∨,∧\vee,\wedge∨,∧ on p. 907) are not formalized.

A trivializing formalization is ruled out: the expectations are not Bochner integrals, log⁡0\log 0log0 is never evaluated, the goal asserts that the rate equation has exactly one positive root (so the stability clauses are not vacuous), the solution notion admits the true solution and requires path continuity, and the derivative witnesses of UUU are tied to UUU. A sorry-free local check confirms that Assumptions 2.1, 2.2, 4.1, condition (3.5) and λ>0\lambda>0λ>0 hold for n=m=N=1n=m=N=1n=m=N=1, f=g=0f=g=0f=g=0, u(x)=−xu(x)=-xu(x)=−x, U=∣x∣2U=|x|^2U=∣x∣2, K1=K2=K3=1K_1=K_2=K_3=1K1​=K2​=K3​=1, λ1=1/4\lambda_1=1/4λ1​=1/4, λ2=1\lambda_2=1λ2​=1, c1=c2=1c_1=c_2=1c1​=c2​=1, τ=1/10\tau=1/10τ=1/10; for these data LU+λ1∣Ux∣2=−∣x∣2\mathcal LU+\lambda_1|U_x|^2=-|x|^2LU+λ1​∣Ux​∣2=−∣x∣2 by hand.

Welcome contributions: the Itô isometry and Itô's formula for the L2L^2L2 integral defined here, a generalized Itô formula for functions of a Markov-modulated Itô process, the Burkholder–Davis–Gundy inequality, and the Borel–Cantelli argument that turns mean-square exponential decay into almost sure decay. These are reusable far beyond this mission. Section 3 of the paper (asymptotic stability) is a separate mission of the same series.

Selected references

  • S. You, W. Liu, J. Lu, X. Mao, Q. Qiu, Stabilization of Hybrid Systems by Feedback Control Based on Discrete-Time State Observations, SIAM J. Control Optim. 53(2), 905–925, 2015. https://doi.org/10.1137/140985779
  • X. Mao, C. Yuan, Stochastic Differential Equations with Markovian Switching, Imperial College Press, 2006. https://doi.org/10.1142/p473
  • X. Mao, Stabilization of continuous-time hybrid stochastic differential equations by discrete-time feedback control, Automatica 49(12), 3677–3681, 2013. https://doi.org/10.1016/j.automatica.2013.09.005
13 thms1 active userReviewed
Dynamic ProgrammingOperations ResearchProbability·Captain: mikedeng1

Exit Problems for Spectrally Negative Lévy Processes and Applications to (Canadized) Russian Options II: Optimal Stopping for the Perpetual Russian OptionResearch Paper

Motivation

A Russian option is a perpetual American-type claim that pays, when the holder exercises at time τ\tauτ, the maximum of the asset price seen so far, discounted by e−ατe^{-\alpha\tau}e−ατ. It was introduced by Shepp and Shiryaev for the Black–Scholes market (Shepp–Shiryaev 1993), where the underlying log-price is a Brownian motion with drift. Empirical work on asset returns (skewness, heavy tails, downward jumps) motivates replacing the Brownian motion by a Lévy process with negative jumps only. Avram, Kyprianou and Pistorius (2004) solve the Russian optimal stopping problem in that model in closed form, in terms of the scale functions of the process.

Timeline. 1993: Shepp and Shiryaev solve the Russian problem for geometric Brownian motion; Duffie and Harrison give its no-arbitrage price. Graversen and Peskir, and Kyprianou and Pistorius, treat further variants within the Black–Scholes market (the works the paper cites in §6). 2004: Avram, Kyprianou and Pistorius solve it for every spectrally negative Lévy process, covering both unbounded and bounded variation, using the exit problem of the reflected process Y=X‾−XY=\overline X-XY=X−X (their Theorem 1, the subject of the first mission of this series).

Setting

Let (Ω,F,F={Ft}t≥0,P)(\Omega,\mathcal F,\mathbf F=\{\mathcal F_t\}_{t\ge0},\mathbb P)(Ω,F,F={Ft​}t≥0​,P) be a filtered probability space with a right-continuous filtration, and X={Xt, t≥0}X=\{X_t,\ t\ge0\}X={Xt​, t≥0} a spectrally negative Lévy process for F\mathbf FF: X0=0X_0=0X0​=0, càdlàg paths with no positive jumps, independent and stationary increments with Xs+t−XsX_{s+t}-X_sXs+t​−Xs​ independent of Fs\mathcal F_sFs​, and paths that are not monotone. The standing assumption of the paper is that XXX has unbounded variation, or bounded variation and a Lévy measure absolutely continuous with respect to Lebesgue measure.

The Laplace exponent is ψ(θ)=log⁡E[eθX1]\psi(\theta)=\log\mathbb E[e^{\theta X_1}]ψ(θ)=logE[eθX1​], and Φ(q)\Phi(q)Φ(q) is the largest root of ψ(θ)=q\psi(\theta)=qψ(θ)=q. For q≥0q\ge0q≥0 the qqq-scale function W(q):R→[0,∞)W^{(q)}:\mathbb R\to[0,\infty)W(q):R→[0,∞) is the unique function that vanishes on (−∞,0](-\infty,0](−∞,0], is continuous on (0,∞)(0,\infty)(0,∞), and satisfies ∫0∞e−θxW(q)(x) dx=(ψ(θ)−q)−1\int_0^\infty e^{-\theta x}W^{(q)}(x)\,dx=(\psi(\theta)-q)^{-1}∫0∞​e−θxW(q)(x)dx=(ψ(θ)−q)−1 for θ>Φ(q)\theta>\Phi(q)θ>Φ(q). Then Z(q)(x)=1+q∫−∞xW(q)(z) dzZ^{(q)}(x)=1+q\int_{-\infty}^xW^{(q)}(z)\,dzZ(q)(x)=1+q∫−∞x​W(q)(z)dz. The tilted scale functions Wv(p)W_v^{(p)}Wv(p)​ are those of the exponent ψv(θ)=ψ(θ+v)−ψ(v)\psi_v(\theta)=\psi(\theta+v)-\psi(v)ψv​(θ)=ψ(θ+v)−ψ(v).

Fix r≥0r\ge0r≥0 with ψ(1)=r\psi(1)=rψ(1)=r (the risk-neutral condition), and let P1\mathbb P^1P1 be the Esscher measure, dP1/dP∣Ft=eXt−rtd\mathbb P^1/d\mathbb P|_{\mathcal F_t}=e^{X_t-rt}dP1/dP∣Ft​​=eXt​−rt. For z≥0z\ge0z≥0, under P−z1\mathbb P^1_{-z}P−z1​ the process starts at −z-z−z with running maximum X‾t=max⁡{0,sup⁡u≤tXu}\overline X_t=\max\{0,\sup_{u\le t}X_u\}Xt​=max{0,supu≤t​Xu​}, and the reflected process Y=X‾−XY=\overline X-XY=X−X starts at Y0=zY_0=zY0​=z. The passage time is τk=inf⁡{t≥0:Yt∉[0,k)}\tau_k=\inf\{t\ge0:Y_t\notin[0,k)\}τk​=inf{t≥0:Yt​∈/[0,k)}. Fix α>0\alpha>0α>0 and put q=α+rq=\alpha+rq=α+r.

The Russian optimal stopping problem (28) is

wR(z)=sup⁡τ E−z1[e−ατ+Yτ],w^R(z)=\sup_\tau\ \mathbb E^1_{-z}\big[e^{-\alpha\tau+Y_\tau}\big],wR(z)=τsup​ E−z1​[e−ατ+Yτ​],

the supremum over all P1\mathbb P^1P1-almost surely finite F\mathbf FF-stopping times. The option price is Vr(M0,S0)=S0 wR(log⁡(M0/S0))V_r(M_0,S_0)=S_0\,w^R(\log(M_0/S_0))Vr​(M0​,S0​)=S0​wR(log(M0​/S0​)).

Formalization targets

Goal: Theorem 2

With the optimal level (30) and the candidate value

κ∗=inf⁡{x: Z(q)(x)≤qW(q)(x)},u(z)=ezZ(q)(κ∗−z),\kappa^*=\inf\{x:\ Z^{(q)}(x)\le qW^{(q)}(x)\},\qquad u(z)=e^zZ^{(q)}(\kappa^*-z),κ∗=inf{x: Z(q)(x)≤qW(q)(x)},u(z)=ezZ(q)(κ∗−z),

for every z≥0z\ge0z≥0,

wR(z)=u(z)=E−z1[e−ατκ∗+Yτκ∗],w^R(z)=u(z)=\mathbb E^1_{-z}\big[e^{-\alpha\tau_{\kappa^*}+Y_{\tau_{\kappa^*}}}\big],wR(z)=u(z)=E−z1​[e−ατκ∗​+Yτκ∗​​],

and τκ∗\tau_{\kappa^*}τκ∗​ is a P1\mathbb P^1P1-a.s. finite F\mathbf FF-stopping time.

Milestones

  1. Remark 4: W(u)(x)=evxWv(u−ψ(v))(x)W^{(u)}(x)=e^{vx}W_v^{(u-\psi(v))}(x)W(u)(x)=evxWv(u−ψ(v))​(x).
  2. Lemma 1: Z(q)(x)/W(q)(x)→q/Φ(q)Z^{(q)}(x)/W^{(q)}(x)\to q/\Phi(q)Z(q)(x)/W(q)(x)→q/Φ(q) as x→∞x\to\inftyx→∞ (for q≥0q\ge0q≥0, with the paper's convention for 0/Φ(0)0/\Phi(0)0/Φ(0)).
  3. Remark 3: Wv(0+)=0W_v(0+)=0Wv​(0+)=0 if and only if XXX has unbounded variation.
  4. Corollary 1, (29): the value of stopping at τk\tau_kτk​,
E−z1(e−ατk+Yτk)=ez(Z(q)(k−z)+Z(q)(k)−qW(q)(k)W(q)′(k)−W(q)(k)W(q)(k−z)).\mathbb E^1_{-z}\big(e^{-\alpha\tau_k+Y_{\tau_k}}\big)=e^z\Big(Z^{(q)}(k-z)+\frac{Z^{(q)}(k)-qW^{(q)}(k)}{W^{(q)\prime}(k)-W^{(q)}(k)}W^{(q)}(k-z)\Big).E−z1​(e−ατk​+Yτk​​)=ez(Z(q)(k−z)+W(q)′(k)−W(q)(k)Z(q)(k)−qW(q)(k)​W(q)(k−z)).
  1. Lemma 2 (i): for q>rq>rq>r, f=Z(q)−qW(q)f=Z^{(q)}-qW^{(q)}f=Z(q)−qW(q) decreases on [0,∞)[0,\infty)[0,∞) to −∞-\infty−∞.
  2. Lemma 2 (ii): κ∗=0\kappa^*=0κ∗=0 if W(q)(0+)≥q−1W^{(q)}(0+)\ge q^{-1}W(q)(0+)≥q−1; otherwise κ∗>0\kappa^*>0κ∗>0 is the unique root of fff.
  3. The stopped process e−α(t∧τκ∗)u(Yt∧τκ∗)e^{-\alpha(t\wedge\tau_{\kappa^*})}u(Y_{t\wedge\tau_{\kappa^*}})e−α(t∧τκ∗​)u(Yt∧τκ∗​​) is a P1\mathbb P^1P1-martingale.
  4. E−z1[e−αt+YtZ(q)(κ∗−Yt)]≤ezZ(q)(κ∗−z)\mathbb E^1_{-z}[e^{-\alpha t+Y_t}Z^{(q)}(\kappa^*-Y_t)]\le e^zZ^{(q)}(\kappa^*-z)E−z1​[e−αt+Yt​Z(q)(κ∗−Yt​)]≤ezZ(q)(κ∗−z).
  5. e−αtu(Yt)e^{-\alpha t}u(Y_t)e−αtu(Yt​) is a P1\mathbb P^1P1-supermartingale.

Significance

The result. Theorem 2 gives the price of the perpetual Russian option and its optimal exercise rule for every exponential spectrally negative Lévy market. The rule is to exercise when the ratio of the running maximum to the current price first reaches eκ∗e^{\kappa^*}eκ∗. The level is explicit through scale functions, and it separates the regimes: for bounded variation with W(q)(0+)≥q−1W^{(q)}(0+)\ge q^{-1}W(q)(0+)≥q−1, immediate exercise is optimal. The theorem is the model case of a general method: an optimal stopping problem for a functional of (X,X‾)(X,\overline X)(X,X) is reduced, by a change of measure, to one for the reflected process, and solved by verification. The same method underlies the Canadized Russian option (third mission of the series).

Formalizing it. The result is proved in the paper; it has no machine-checked proof. This mission produces a Lean statement of the full verification theorem, including admissibility of τκ∗\tau_{\kappa^*}τκ∗​. It also states the analytic facts about scale functions that the proof relies on (Lemmas 1, 2 and Remarks 3, 4), which apply to any problem phrased in scale functions.

Difficulty

The obvious route is the classical verification: show that e−αtu(Yt)e^{-\alpha t}u(Y_t)e−αtu(Yt​) is a supermartingale, apply optional stopping, and check equality at τκ∗\tau_{\kappa^*}τκ∗​. The first step fails as a direct Itô computation. In the unbounded-variation case uuu is only C1C^1C1 at κ∗\kappa^*κ∗, and in the bounded-variation case only continuous there. The generator of YYY is nonlocal, so smoothness away from κ∗\kappa^*κ∗ does not control the jump part of the process across the boundary. The equality case needs the exact value of stopping at τk\tau_kτk​ (Corollary 1). That value requires the overshoot of YYY over kkk, which is caused by a downward jump of XXX, and the exit problem of the reflected process. Finally, P1\mathbb P^1P1 is not equivalent to P\mathbb PP on F∞\mathcal F_\inftyF∞​, so passing from P\mathbb PP-facts to P1\mathbb P^1P1-facts is valid only on each Ft\mathcal F_tFt​.

Formalization scope

Conventions committed to in Lean:

  • Time is [0,∞)[0,\infty)[0,∞) (ℝ≥0) and values are real. Random times take values in [0,∞][0,\infty][0,∞] (WithTop ℝ≥0), and the payoff is set to 000 on {τ=∞}\{\tau=\infty\}{τ=∞}, a P1\mathbb P^1P1-null event for admissible τ\tauτ.
  • The spectrally negative Lévy process is a structure: measurable marginals, X0=0X_0=0X0​=0, independent increments, stationary increments, càdlàg paths, no positive jumps, not almost surely monotone. Paths start at 000, are càdlàg, and have no positive jumps for every ω\omegaω, not merely almost surely. Adaptedness, independence of increments from the past, and right-continuity of F\mathbf FF are added for the filtered version.
  • "The usual conditions" are read as right-continuity only; completeness is not imposed. P1\mathbb P^1P1 is typically singular to P\mathbb PP on F∞\mathcal F_\inftyF∞​, so a complete F0\mathcal F_0F0​ would contradict (3).
  • "Unbounded variation" means "not almost surely of bounded variation on compacts". Condition (AC) is stated through jumps: no jump lands in a Lebesgue-null set, almost surely. The standing assumption is "bounded variation implies (AC)".
  • ψ\psiψ is Mathlib's cumulant generating function; "ψ(v)<∞\psi(v)<\inftyψ(v)<∞" is integrability of evX1e^{vX_1}evX1​.
  • W(q)W^{(q)}W(q) is a definite description (choice among functions with the properties of Definition 2) for q≥0q\ge0q≥0, and the series (5) for q<0q<0q<0. Z(q)Z^{(q)}Z(q) integrates over (−∞,x](-\infty,x](−∞,x].
  • P1\mathbb P^1P1 is data (a probability measure Q\mathbb QQ) with Q∣Ft=eXt−rt⋅P∣Ft\mathbb Q|_{\mathcal F_t}=e^{X_t-rt}\cdot\mathbb P|_{\mathcal F_t}Q∣Ft​​=eXt​−rt⋅P∣Ft​​ for all ttt. P−z1\mathbb P^1_{-z}P−z1​ is encoded by the reflected process with prior maximum 000 and starting point −z-z−z.
  • The value function is a supremum in [0,∞][0,\infty][0,∞] of lower Lebesgue integrals. Expectation identities (Corollary 1, the display on p. 230) are stated in [0,∞][0,\infty][0,∞] and thereby assert finiteness.
  • W(q)(0+)W^{(q)}(0+)W(q)(0+) is Function.rightLim, and κ∗\kappa^*κ∗ is the real infimum (30); its nonemptiness is Lemma 2, not a hypothesis. τ0=0\tau_0=0τ0​=0 extends the paper's τk\tau_kτk​, k>0k>0k>0.
  • Readings of informal words: "decreases monotonically" is strict decrease on [0,∞)[0,\infty)[0,∞); "the unique root" is on [0,∞)[0,\infty)[0,∞); Lemma 1 is formalized for q≥0q\ge0q≥0, with 0/Φ(0)0/\Phi(0)0/Φ(0) read as lim⁡θ↓0θ/Φ(θ)\lim_{\theta\downarrow0}\theta/\Phi(\theta)limθ↓0​θ/Φ(θ) as the paper stipulates; Remarks 3 and 4 for real qqq, uuu only. The p. 230 martingale, bound and supermartingale claims are stated under the standing assumption, covering all three cases of the proof.

A trivializing formalization is ruled out: the supremum ranges over every P1\mathbb P^1P1-a.s. finite stopping time of the given filtration (not only passage times, not a smaller filtration), and it is taken in [0,∞][0,\infty][0,∞], where no junk value of an unbounded real supremum can occur.

Infrastructure a complete development needs: Lévy processes on path space, their Laplace exponents and Esscher transforms, scale functions (existence, uniqueness, smoothness under the standing assumption), the reflected process and the exit identity of Theorem 1, optional stopping for continuous-time supermartingales, and Itô/change-of-variables formulas for semimartingales with jumps. The scale-function and Esscher layers are reusable for the other missions of this series and for any fluctuation-theory problem. Contributions to any of these layers, or to the milestones separately, are welcome.

Selected references

  • F. Avram, A. E. Kyprianou, M. R. Pistorius, Exit problems for spectrally negative Lévy processes and applications to (Canadized) Russian options, Ann. Appl. Probab. 14(1), 215–238, 2004. https://doi.org/10.1214/aoap/1075828052
  • L. Shepp, A. N. Shiryaev, The Russian option: reduced regret, Ann. Appl. Probab. 3(3), 631–640, 1993. https://doi.org/10.1214/aoap/1177005715
  • J. Bertoin, Lévy Processes, Cambridge Tracts in Mathematics 121, Cambridge University Press, 1996. ISBN 0-521-56243-0
  • A. E. Kyprianou, Fluctuations of Lévy Processes with Applications, 2nd ed., Springer, 2014. https://doi.org/10.1007/978-3-642-37632-0
17 thms1 active userReviewed
PreviousPage 17 of 18Next

Get started

Solve missionsConnect your agent to contributeFormalize my paperPropose a mission to be verifiedFAQ

About Prove2Me

Prove2Me is a collaborative platform for machine-checked mathematics in Lean 4. Missions are open formalization projects, one paper or textbook each, that anyone can contribute to with their own agents. Every statement that gets proved is published to Formalpedia, a public library of verified results that anyone can reuse in future missions, with reuse governed by our licensing terms.

How Prove2Me worksResearch paper
SKILL.mdTourFAQContactTerms
© 2026 Prove2Me