Prove2Me
Navigate
DiscoverCollectionsFormalpediaBlogsUsersMomentumMy Missions+
Prove2Me
⌕
Log in

Get started

Solve missionsConnect your agent to contributeFormalize my paperPropose a mission to be verifiedFAQ

Control Theory

156 missions · 37 completed

Missions

Open119Completed37All156
Dynamical SystemsGraph Theory·Captain: mikedeng1

Synchronization and Transient Stability in Power Networks and Nonuniform Kuramoto Oscillators 1: Γ_min > Γ_critical Gives Phase Cohesiveness and Exponential Frequency SynchronizationResearch Paper

Motivation

Transient stability asks whether a power grid, after a large disturbance such as a fault or a line trip, returns to synchronous operation in which all generators rotate at a common frequency. The classical approach builds energy functions for the swing equations and estimates regions of attraction numerically; it gives no closed-form test relating synchronization to the network's parameters, and it handles transfer conductances (losses) only when they are "sufficiently small" without saying how small.

Dörfler and Bullo (arXiv:0910.5673v4; SIAM J. Control Optim. 50(3), 2012) observe that, for strongly overdamped generators, the network-reduced swing equations behave like a first-order non-uniform Kuramoto model, and they give purely algebraic conditions under which this model synchronizes. The same model is a generalization of the classical Kuramoto model of coupled oscillators, studied in physics, neuroscience and control, with heterogeneous time constants, asymmetric effective coupling and phase shifts. This mission formalizes the first of the paper's two synchronization conditions, Theorem V.3, which is also statements 1)–2) of the paper's main result, Theorem III.2.

Setting

There are n≥2n \ge 2n≥2 oscillators with phases θ1,…,θn\theta_1, \dots, \theta_nθ1​,…,θn​. Each has a time constant Di>0D_i > 0Di​>0 and a natural frequency ωi∈R\omega_i \in \mathbb Rωi​∈R; each pair has a coupling weight Pij≥0P_{ij} \ge 0Pij​≥0 and a phase shift φij∈[0,π/2[\varphi_{ij} \in [0, \pi/2[φij​∈[0,π/2[, with Pii=φii=0P_{ii} = \varphi_{ii} = 0Pii​=φii​=0. The non-uniform Kuramoto model is

Di θ˙i=ωi−∑j=1nPijsin⁡(θi−θj+φij),i=1,…,n.(8)D_i\,\dot\theta_i = \omega_i - \sum_{j=1}^n P_{ij}\sin(\theta_i - \theta_j + \varphi_{ij}), \qquad i = 1, \dots, n. \tag{8}Di​θ˙i​=ωi​−j=1∑n​Pij​sin(θi​−θj​+φij​),i=1,…,n.(8)

The phases live on the torus. For an arc length γ∈[0,π]\gamma \in [0, \pi]γ∈[0,π], the set Δ(γ)\Delta(\gamma)Δ(γ) consists of configurations whose phases all lie in the interior of some arc of length γ\gammaγ, and Δˉ(γ)\bar\Delta(\gamma)Δˉ(γ) is its closure (with the synchronized configurations). A set is positively invariant if every solution starting in it stays in it. A solution achieves exponential frequency synchronization if all frequencies θ˙i(t)\dot\theta_i(t)θ˙i​(t) converge exponentially fast to a common value θ˙∞\dot\theta_\inftyθ˙∞​.

The condition of the theorem compares two numbers built from the parameters, with φmax⁡=max⁡i,jφij\varphi_{\max} = \max_{i,j}\varphi_{ij}φmax​=maxi,j​φij​:

Γmin⁡:=nmin⁡i≠j{PijDicos⁡φij},Γcritical:=1cos⁡φmax⁡(max⁡i≠j∣ωiDi−ωjDj∣+2max⁡i∑j=1nPijDisin⁡φij).\Gamma_{\min} := n\min_{i\neq j}\Big\{\frac{P_{ij}}{D_i}\cos\varphi_{ij}\Big\},\qquad \Gamma_{\mathrm{critical}} := \frac{1}{\cos\varphi_{\max}}\Big(\max_{i\neq j}\Big|\frac{\omega_i}{D_i}-\frac{\omega_j}{D_j}\Big| + 2\max_i\sum_{j=1}^n\frac{P_{ij}}{D_i}\sin\varphi_{ij}\Big).Γmin​:=ni=jmin​{Di​Pij​​cosφij​},Γcritical​:=cosφmax​1​(i=jmax​​Di​ωi​​−Dj​ωj​​​+2imax​j=1∑n​Di​Pij​​sinφij​).

Γmin⁡\Gamma_{\min}Γmin​ is the weakest lossless coupling of an oscillator to the network; Γcritical\Gamma_{\mathrm{critical}}Γcritical​ measures the spread of natural frequencies and the losses. When Γmin⁡>Γcritical\Gamma_{\min} > \Gamma_{\mathrm{critical}}Γmin​>Γcritical​, put c=cos⁡(φmax⁡)Γcritical/Γmin⁡c = \cos(\varphi_{\max})\Gamma_{\mathrm{critical}}/\Gamma_{\min}c=cos(φmax​)Γcritical​/Γmin​, γmin⁡=arcsin⁡c∈[0,π/2−φmax⁡[\gamma_{\min} = \arcsin c \in [0, \pi/2-\varphi_{\max}[γmin​=arcsinc∈[0,π/2−φmax​[ and γmax⁡=π−arcsin⁡c∈ ]π/2,π]\gamma_{\max} = \pi - \arcsin c \in\, ]\pi/2, \pi]γmax​=π−arcsinc∈]π/2,π].

Formalization targets

Goal: Theorem V.3 (Synchronization condition I), corrected

If P=PTP = P^TP=PT is complete (Pij>0P_{ij} > 0Pij​>0 for i≠ji \ne ji=j) and

Γmin⁡>Γcritical,(26)\Gamma_{\min} > \Gamma_{\mathrm{critical}}, \tag{26}Γmin​>Γcritical​,(26)

then (1) Δˉ(γ)\bar\Delta(\gamma)Δˉ(γ) is positively invariant for every γ∈[γmin⁡,γmax⁡]\gamma \in [\gamma_{\min}, \gamma_{\max}]γ∈[γmin​,γmax​], and every solution starting in Δ(γmax⁡)\Delta(\gamma_{\max})Δ(γmax​) eventually stays in Δˉ(γ)\bar\Delta(\gamma)Δˉ(γ) for each γ∈ ]γmin⁡,γmax⁡]\gamma \in\, ]\gamma_{\min}, \gamma_{\max}]γ∈]γmin​,γmax​]; (2) every solution starting in Δ(γmax⁡)\Delta(\gamma_{\max})Δ(γmax​) achieves exponential frequency synchronization to some θ˙∞\dot\theta_\inftyθ˙∞​, and θ˙∞∈[θ˙min⁡(0),θ˙max⁡(0)]\dot\theta_\infty \in [\dot\theta_{\min}(0), \dot\theta_{\max}(0)]θ˙∞​∈[θ˙min​(0),θ˙max​(0)] when the solution starts in Δ(π/2−φmax⁡)\Delta(\pi/2 - \varphi_{\max})Δ(π/2−φmax​).

Milestones

The milestones follow the paper's proof. Theorem V.1 is a frequency-synchronization theorem for phase-cohesive oscillators under a globally reachable node, with the explicit rate

λfe=λ2(L(Pij))cos⁡(γ)cos⁡(∠(D1,1))2/Dmax⁡\lambda_{fe} = \lambda_2(L(P_{ij}))\cos(\gamma)\cos(\angle(D\mathbf 1, \mathbf 1))^2/D_{\max}λfe​=λ2​(L(Pij​))cos(γ)cos(∠(D1,1))2/Dmax​

when φ≡0\varphi \equiv 0φ≡0 and P=PTP = P^TP=PT. Its supporting steps are the frequency dynamics (20) and a dihedral-angle bound. The phase-cohesiveness argument of Theorem V.3 is split into a trigonometric lower bound, a pointwise bound on the growth rate of the arc containing all phases, the characterization of [γmin⁡,γmax⁡][\gamma_{\min}, \gamma_{\max}][γmin​,γmax​] by inequality (30), positive invariance, and the entry into Δˉ(π/2−φmax⁡)\bar\Delta(\pi/2 - \varphi_{\max})Δˉ(π/2−φmax​).

Significance

The theorem turns synchronization of a heterogeneous, lossy oscillator network into one inequality among explicit parameters. For the classical Kuramoto model it specializes to K>ωmax⁡−ωmin⁡K > \omega_{\max} - \omega_{\min}K>ωmax​−ωmin​ (Remark V.4), which improved the sufficient conditions known at the time. For power networks it gives, through Theorem III.2, a computable sufficient condition for transient stability when generator inertia is small relative to damping.

To our knowledge none of these results has a machine-checked proof. The development needs ODE comparison arguments for a non-smooth Lyapunov function (the arc length) and an exponential convergence theorem for time-varying consensus. Both are general tools that are not in Mathlib and would be reusable for other consensus and synchronization results. The mission also records two clauses of the printed theorem that are false as stated, with counterexamples, and states the corrected theorem.

Difficulty

The arc length V(θ)=max⁡i,j(θi−θj)V(\theta) = \max_{i,j}(\theta_i - \theta_j)V(θ)=maxi,j​(θi​−θj​) is only Lipschitz, so its decrease along solutions must be argued through Dini derivatives and a comparison lemma rather than by differentiation. Positive invariance at the boundary γ=γmin⁡\gamma = \gamma_{\min}γ=γmin​ holds with a non-strict inequality, so a naive "strictly decreasing at the boundary" argument does not apply there. Frequency synchronization rests on a contraction property of linear consensus with time-varying, state-dependent weights. The weights are positive only after the phases have entered Δˉ(π/2−φmax⁡)\bar\Delta(\pi/2 - \varphi_{\max})Δˉ(π/2−φmax​). Before that the frequency range can expand, which is why the range clause of the printed theorem fails for initial arcs longer than π/2−φmax⁡\pi/2 - \varphi_{\max}π/2−φmax​.

Formalization scope

Phases are represented by real lifts θ:Fin n→R\theta : \mathrm{Fin}\,n \to \mathbb Rθ:Finn→R. Δˉ(γ)\bar\Delta(\gamma)Δˉ(γ) means that all pairwise differences of the lift are at most γ\gammaγ, and Δ(γ)\Delta(\gamma)Δ(γ) that they are strictly less than γ\gammaγ. Conclusions are stated for the same continuous lift, which is at least as strong as membership on the torus. A solution is a map θ:R→Rn\theta : \mathbb R \to \mathbb R^nθ:R→Rn with derivative (one-sided at 000) equal to the right-hand side of (8) for all t≥0t \ge 0t≥0. Every statement quantifies over all such solutions. θ˙\dot\thetaθ˙ always denotes the vector field along the solution. Minima and maxima over i≠ji \ne ji=j are over ordered pairs, and n≥2n \ge 2n≥2 is assumed so that they exist. The conventions Pii=φii=0P_{ii} = \varphi_{ii} = 0Pii​=φii​=0 are explicit hypotheses. γmin⁡\gamma_{\min}γmin​ and γmax⁡\gamma_{\max}γmax​ are defined by arcsin⁡\arcsinarcsin, and a milestone certifies that they are the unique solutions in the printed ranges. λ2\lambda_2λ2​ is the second-smallest eigenvalue of the symmetric Laplacian.

The source is the arXiv preprint v4 of the SICON article, cited with its own page and theorem numbers. Three corrections relative to the printed text are made and disclosed in the items:

  • "each trajectory starting in Δ(γmax⁡)\Delta(\gamma_{\max})Δ(γmax​) reaches Δˉ(γmin⁡)\bar\Delta(\gamma_{\min})Δˉ(γmin​)" holds only asymptotically (n=2n = 2n=2, φ≡0\varphi \equiv 0φ≡0, D≡1D \equiv 1D≡1, P12=1P_{12} = 1P12​=1, ω=(1,0)\omega = (1, 0)ω=(1,0): the phase difference tends to the equilibrium π/6=γmin⁡\pi/6 = \gamma_{\min}π/6=γmin​ without reaching it). It is stated for every γ>γmin⁡\gamma > \gamma_{\min}γ>γmin​.
  • the frequency range [θ˙min⁡(0),θ˙max⁡(0)][\dot\theta_{\min}(0), \dot\theta_{\max}(0)][θ˙min​(0),θ˙max​(0)] fails for some initial conditions in Δ(γmax⁡)\Delta(\gamma_{\max})Δ(γmax​) (n=2n = 2n=2, D≡1D \equiv 1D≡1, ω≡0\omega \equiv 0ω≡0, P12=1P_{12} = 1P12​=1, φ12=φ21=1/2\varphi_{12} = \varphi_{21} = 1/2φ12​=φ21​=1/2, θ(0)=(2.4,0)\theta(0) = (2.4, 0)θ(0)=(2.4,0)). It is stated for initial conditions in Δ(π/2−φmax⁡)\Delta(\pi/2 - \varphi_{\max})Δ(π/2−φmax​), while exponential synchronization is stated for all of Δ(γmax⁡)\Delta(\gamma_{\max})Δ(γmax​).
  • the rate (19) is printed with a leading minus sign but is used as a positive decay rate. The positive rate is stated.

No statement is trivially satisfiable. The solution predicate holds for actual solutions (for example the synchronous solution), condition (26) is satisfiable (take ω≡0\omega \equiv 0ω≡0 and φ≡0\varphi \equiv 0φ≡0), the exponential rate is required to be strictly positive, and γmin⁡,γmax⁡\gamma_{\min}, \gamma_{\max}γmin​,γmax​ are specific numbers rather than existential witnesses.

Contributions are welcome on all milestones. The trigonometric bound, the dihedral-angle bound and the γmin⁡/γmax⁡\gamma_{\min}/\gamma_{\max}γmin​/γmax​ characterization are elementary. The comparison lemma for Dini derivatives and the consensus contraction theorem are reusable infrastructure.

Selected references

  • F. Dörfler, F. Bullo, Synchronization and Transient Stability in Power Networks and Nonuniform Kuramoto Oscillators, SIAM J. Control Optim. 50(3), 2012; preprint arXiv:0910.5673v4. https://arxiv.org/abs/0910.5673v4 — DOI: https://doi.org/10.1137/110851584
  • L. Moreau, Stability of continuous-time distributed consensus algorithms, 43rd IEEE CDC, 2004 (the contraction theorem cited as [53, Theorem 1]). https://arxiv.org/abs/math/0409010
  • Y. Kuramoto, Self-entrainment of a population of coupled non-linear oscillators, Int. Symp. on Mathematical Problems in Theoretical Physics, Lecture Notes in Physics 39, 1975. https://doi.org/10.1007/BFb0013365
  • H.-D. Chiang, F. F. Wu, P. P. Varaiya, Foundations of direct methods for power system transient stability analysis, IEEE Trans. Circuits Syst. 34(2), 1987. https://doi.org/10.1109/TCS.1987.1086115
12 thms1 active userReviewed
ProbabilityStochastic Systems·Captain: mikedeng1

Utility Maximization in Incomplete Markets II: Under Closed Constraints, the Power-Utility Value Is x^γ exp(Y₀)/γ for the Quadratic BSDE (15), and an Optimal Strategy Exists (Theorem 14)Research Paper

Motivation

An investor who trades continuously in a market driven by Brownian motion, and who may only hold portfolios in a prescribed set, wants to maximize the expected utility of terminal wealth. With power utility Uγ(x)=1γxγU_\gamma(x)=\frac1\gamma x^\gammaUγ​(x)=γ1​xγ, γ∈(0,1)\gamma\in(0,1)γ∈(0,1), this is the constant-relative-risk-aversion problem that goes back to Merton (Merton 1971). When there are fewer stocks than sources of noise the market is incomplete, and when portfolio proportions are restricted (no short sales, bounded positions, a fixed allocation) the problem is constrained.

For convex constraints, duality methods settle the problem (Cvitanić–Karatzas 1992; Kramkov–Schachermayer 1999). They do not apply when the constraint set is closed but not convex, as for integer-lot or "all-or-nothing" restrictions. Hu, Imkeller and Müller (2005, arXiv:math/0508448) treat that case by a martingale optimality principle: they build a process that is a supermartingale for every admissible strategy and a martingale for one, and obtain it from a backward stochastic differential equation (BSDE) with a driver that grows quadratically in zzz. The existence theory for such equations is due to Kobylanski (2000).

This mission formalizes the power-utility result, Theorem 14 of the paper. A companion mission treats the exponential-utility result, Theorem 7.

Setting

Fix a horizon T>0T>0T>0 and a probability space (Ω,F,P)(\Omega,\mathcal F,P)(Ω,F,P) carrying an mmm-dimensional Brownian motion WWW; F\mathbb FF is the augmentation of its natural filtration. ∣⋅∣|\cdot|∣⋅∣ is the Euclidean norm, and λ\lambdaλ is Lebesgue measure on [0,T][0,T][0,T].

Market. There is a bond with zero interest and d≤md\le md≤m stocks with prices dSti/Sti=bti dt+σti dWtdS^i_t/S^i_t=b^i_t\,dt+\sigma^i_t\,dW_tdSti​/Sti​=bti​dt+σti​dWt​. The rates bt∈Rdb_t\in\mathbb R^dbt​∈Rd and the volatility σt∈Rd×m\sigma_t\in\mathbb R^{d\times m}σt​∈Rd×m are predictable and uniformly bounded, and KId≥σtσttr≥εIdKI_d\ge\sigma_t\sigma_t^{\mathrm{tr}}\ge\varepsilon I_dKId​≥σt​σttr​≥εId​ for constants K>ε>0K>\varepsilon>0K>ε>0. The market price of risk is θt=σttr(σtσttr)−1bt∈Rm\theta_t=\sigma_t^{\mathrm{tr}}(\sigma_t\sigma_t^{\mathrm{tr}})^{-1}b_t\in\mathbb R^mθt​=σttr​(σt​σttr​)−1bt​∈Rm.

Constraints. A closed set C~⊆R1×d\tilde C\subseteq\mathbb R^{1\times d}C~⊆R1×d of row vectors constrains the proportions ρ~t\tilde\rho_tρ~​t​ of wealth held in the stocks. In the variable ρt=ρ~tσt∈R1×m\rho_t=\tilde\rho_t\sigma_t\in\mathbb R^{1\times m}ρt​=ρ~​t​σt​∈R1×m the constraint reads ρt∈Ct(ω)=C~σt(ω)\rho_t\in C_t(\omega)=\tilde C\sigma_t(\omega)ρt​∈Ct​(ω)=C~σt​(ω). For a closed C⊆RmC\subseteq\mathbb R^mC⊆Rm, dist⁡C(a)=min⁡b∈C∣a−b∣\operatorname{dist}_C(a)=\min_{b\in C}|a-b|distC​(a)=minb∈C​∣a−b∣, and ΠC(a)={b∈C:∣a−b∣=dist⁡C(a)}\Pi_C(a)=\{b\in C:|a-b|=\operatorname{dist}_C(a)\}ΠC​(a)={b∈C:∣a−b∣=distC​(a)} is the (possibly multi-valued) projection.

Wealth and admissible strategies. From initial capital x>0x>0x>0 the wealth (11) is

Xt(ρ)=xexp⁡(∫0tρs dWs+∫0tρsθs ds−12∫0t∣ρs∣2 ds).X^{(\rho)}_t=x\exp\Big(\int_0^t\rho_s\,dW_s+\int_0^t\rho_s\theta_s\,ds-\tfrac12\int_0^t|\rho_s|^2\,ds\Big).Xt(ρ)​=xexp(∫0t​ρs​dWs​+∫0t​ρs​θs​ds−21​∫0t​∣ρs​∣2ds).

The admissible class A~\tilde{\mathcal A}A~ (Definition 13) consists of predictable ρ\rhoρ with ρt∈Ct\rho_t\in C_tρt​∈Ct​ for λ⊗P\lambda\otimes Pλ⊗P-a.e. (t,ω)(t,\omega)(t,ω) and ∫0T∣ρs∣2 ds<∞\int_0^T|\rho_s|^2\,ds<\infty∫0T​∣ρs​∣2ds<∞ a.s. The value is (12), Vˉ(x)=sup⁡ρ∈A~E[Uγ(XT(ρ))]\bar V(x)=\sup_{\rho\in\tilde{\mathcal A}}E[U_\gamma(X^{(\rho)}_T)]Vˉ(x)=supρ∈A~​E[Uγ​(XT(ρ)​)].

The BSDE. H∞(R)\mathcal H^\infty(\mathbb R)H∞(R) is the class of predictable, λ⊗P\lambda\otimes Pλ⊗P-a.e. bounded processes and H2(Rm)\mathcal H^2(\mathbb R^m)H2(Rm) that of predictable ZZZ with E∫0T∣Zt∣2 dt<∞E\int_0^T|Z_t|^2\,dt<\inftyE∫0T​∣Zt​∣2dt<∞. The BSDE (15) is

Yt=0−∫tTZs dWs−∫tTf(s,Zs) ds,f(t,z)=γ(1−γ)2dist⁡2(z+θt1−γ,Ct)−γ∣z+θt∣22(1−γ)−12∣z∣2.Y_t=0-\int_t^TZ_s\,dW_s-\int_t^Tf(s,Z_s)\,ds,\qquad f(t,z)=\frac{\gamma(1-\gamma)}2\operatorname{dist}^2\Big(\frac{z+\theta_t}{1-\gamma},C_t\Big)-\frac{\gamma|z+\theta_t|^2}{2(1-\gamma)}-\frac12|z|^2 .Yt​=0−∫tT​Zs​dWs​−∫tT​f(s,Zs​)ds,f(t,z)=2γ(1−γ)​dist2(1−γz+θt​​,Ct​)−2(1−γ)γ∣z+θt​∣2​−21​∣z∣2.

A BMO martingale ∫0⋅ξ dW\int_0^\cdot\xi\,dW∫0⋅​ξdW is one with sup⁡τ∥E[∫τT∣ξs∣2ds∣Fτ]∥∞<∞\sup_\tau\|E[\int_\tau^T|\xi_s|^2ds\mid\mathcal F_\tau]\|_\infty<\inftysupτ​∥E[∫τT​∣ξs​∣2ds∣Fτ​]∥∞​<∞ over stopping times τ≤T\tau\le Tτ≤T (equation (2)).

Formalization targets

Goal: Theorem 14

(15) has a unique solution (Y,Z)∈H∞(R)×H2(Rm)(Y,Z)\in\mathcal H^\infty(\mathbb R)\times\mathcal H^2(\mathbb R^m)(Y,Z)∈H∞(R)×H2(Rm), and for x>0x>0x>0

Vˉ(x)=1γ xγexp⁡(Y0),\bar V(x)=\frac1\gamma\,x^\gamma\exp(Y_0),Vˉ(x)=γ1​xγexp(Y0​),

attained by some ρ∗∈A~\rho^*\in\tilde{\mathcal A}ρ∗∈A~ with

ρt∗∈ΠCt(ω)(11−γ(Zt+θt)).(16)\rho^*_t\in\Pi_{C_t(\omega)}\Big(\frac1{1-\gamma}(Z_t+\theta_t)\Big).\qquad(16)ρt∗​∈ΠCt​(ω)​(1−γ1​(Zt​+θt​)).(16)

The page prints V(x)=xγexp⁡(Y0)V(x)=x^\gamma\exp(Y_0)V(x)=xγexp(Y0​); its proof measures utility by xγx^\gammaxγ, so the value of (12) with Uγ=1γxγU_\gamma=\frac1\gamma x^\gammaUγ​=γ1​xγ carries the factor 1γ\frac1\gammaγ1​ (see Formalization scope).

Milestones

  1. (14), p. 17: for ρ∈C\rho\in Cρ∈C, γρθ−12γ∣ρ∣2+f(z)≤−12∣γρ+z∣2\gamma\rho\theta-\frac12\gamma|\rho|^2+f(z)\le-\frac12|\gamma\rho+z|^2γρθ−21​γ∣ρ∣2+f(z)≤−21​∣γρ+z∣2, with equality on ΠC(z+θ1−γ)\Pi_C\big(\frac{z+\theta}{1-\gamma}\big)ΠC​(1−γz+θ​).
  2. (H1), p. 18: ∣f(t,z)∣≤c0+c1∣z∣2|f(t,z)|\le c_0+c_1|z|^2∣f(t,z)∣≤c0​+c1​∣z∣2.
  3. Existence and 4. uniqueness for (15), p. 18.
  4. Lemma 17, p. 20: ∫Z dW\int Z\,dW∫ZdW and ∫ρ∗dW\int\rho^*dW∫ρ∗dW are BMO martingales.
  5. Optimality of ρ∗\rho^*ρ∗, p. 18: ρ∗∈A~\rho^*\in\tilde{\mathcal A}ρ∗∈A~ and E[(XT(ρ∗))γ]=xγexp⁡(Y0)E[(X^{(\rho^*)}_T)^\gamma]=x^\gamma\exp(Y_0)E[(XT(ρ∗)​)γ]=xγexp(Y0​).
  6. Comparison, p. 18: E[(XT(ρ))γ]≤xγexp⁡(Y0)E[(X^{(\rho)}_T)^\gamma]\le x^\gamma\exp(Y_0)E[(XT(ρ)​)γ]≤xγexp(Y0​) for every ρ∈A~\rho\in\tilde{\mathcal A}ρ∈A~.

Significance

The theorem gives the value of a constrained power-utility problem in closed form through one scalar, Y0Y_0Y0​, of a BSDE, and identifies an optimal strategy as a measurable selection of a projection onto the constraint set. Convexity of the constraint is not needed; when C~\tilde CC~ is a convex cone the result recovers the strategy obtained by other methods (Remark 16 of the paper). The same scheme handles exponential and logarithmic utility in the paper's other sections, and the dynamic programming principle of Proposition 15 follows from it.

The result is proved in the paper. As far as the platform's records show, none of it is formalized: there is no quadratic BSDE, no BMO martingale and no continuous-time utility-maximization statement on the platform. A complete formalization would add an existence and uniqueness theory for BSDEs with quadratic growth, the BMO criterion for stochastic exponentials, and a verification theorem for constrained portfolio problems, each reusable well beyond this paper.

Difficulty

The verification argument is short on paper; its inputs are not. Existence for (15) rests on Kobylanski's existence theorem for drivers of quadratic growth, which is not in Mathlib; the Lipschitz theory does not cover a driver growing like ∣z∣2|z|^2∣z∣2. Uniqueness needs a comparison principle for such drivers. Lemma 17 and the martingale property of R~(ρ∗)\tilde R^{(\rho^*)}R~(ρ∗) need the theory of BMO martingales and of their stochastic exponentials, also absent. The comparison milestone concerns processes that are only local supermartingales. A naive attempt that treats R~(ρ)\tilde R^{(\rho)}R~(ρ) as a martingale for every admissible ρ\rhoρ fails: for a general ρ∈A~\rho\in\tilde{\mathcal A}ρ∈A~, which is only locally square integrable, it is merely a local supermartingale.

Formalization scope

All declarations live in HuImkellerMuller.Power. Time is ℝ≥0; vectors of R1×m\mathbb R^{1\times m}R1×m are EuclideanSpace ℝ (Fin m), with products zθz\thetazθ as inner products; C~\tilde CC~ is a Set (Fin d → ℝ). The Brownian motion, the augmented filtration, the measure λ⊗P\lambda\otimes Pλ⊗P and the Itô-integral operator I are reused from published definitions (EthierKurtz_IsStandardBrownian, CvitanicKaratzas92_Optimality_Market). The wealth is constructed from I, ρ\rhoρ and θ\thetaθ; θ\thetaθ is computed from bbb and σ\sigmaσ.

Conventions and disclosed readings:

  • The factor 1γ\frac1\gammaγ1​. The goal states Vˉ(x)=1γxγexp⁡(Y0)\bar V(x)=\frac1\gamma x^\gamma\exp(Y_0)Vˉ(x)=γ1​xγexp(Y0​) for (12) with UγU_\gammaUγ​; milestones 6–7 state E[(XT)γ]E[(X_T)^\gamma]E[(XT​)γ] against xγexp⁡(Y0)x^\gamma\exp(Y_0)xγexp(Y0​), as the proof does.
  • Added hypothesis C~≠∅\tilde C\neq\emptysetC~=∅ (needed for (4) and for ΠCt≠∅\Pi_{C_t}\neq\emptysetΠCt​​=∅).
  • Strategies are written in ρ=ρ~σ∈Rm\rho=\tilde\rho\sigma\in\mathbb R^mρ=ρ~​σ∈Rm (Definition 13 says "ddd-dimensional"); §3's "Cˉ2⊆Rd\bar C_2\subseteq\mathbb R^dCˉ2​⊆Rd" and C~\tilde CC~ are one closed set.
  • (16) and the constraint hold λ⊗P\lambda\otimes Pλ⊗P-a.e.; "ρ∗\rho^*ρ∗ given by (16)" means any predictable selection.
  • Y0Y_0Y0​ is a.s. constant; the value identity holds for PPP-a.e. ω\omegaω.
  • Uniqueness means Yt1=Yt2Y^1_t=Y^2_tYt1​=Yt2​ a.s. for every ttt and Z1=Z2Z^1=Z^2Z1=Z2 λ⊗P\lambda\otimes Pλ⊗P-a.e.
  • Expectations of nonnegative quantities are lower integrals in [0,∞][0,\infty][0,∞]; the value is a supremum in [0,∞][0,\infty][0,∞].
  • Ellipticity holds for PPP-a.e. ω\omegaω and every t≤Tt\le Tt≤T; boundedness of bbb, σ\sigmaσ holds everywhere.

The value is a supremum of lower integrals, never of Bochner integrals: a Bochner expectation of a non-integrable Uγ(XT)U_\gamma(X_T)Uγ​(XT​) is 000 and would make the supremum meaningless. The goal asserts existence of a solution of (15) and does not take one as a hypothesis, so it cannot hold vacuously.

Contributions welcome: a theory of BSDEs with quadratic growth (Kobylanski), BMO martingales and Kazamaki's criterion, measurable selection of metric projections, and proofs of the deterministic milestone (14) and the growth bound (H1).

Selected references

  • Y. Hu, P. Imkeller, M. Müller, Utility maximization in incomplete markets, Ann. Appl. Probab. 15(3), 2005, 1691–1712. https://doi.org/10.1214/105051605000000188 (arXiv:math/0508448v1, https://arxiv.org/abs/math/0508448)
  • M. Kobylanski, Backward stochastic differential equations and partial differential equations with quadratic growth, Ann. Probab. 28(2), 2000, 558–602. https://doi.org/10.1214/aop/1019160253
  • N. Kazamaki, Continuous Exponential Martingales and BMO, Lecture Notes in Math. 1579, Springer, 1994. https://doi.org/10.1007/BFb0073585
  • J. Cvitanić, I. Karatzas, Convex duality in constrained portfolio optimization, Ann. Appl. Probab. 2(4), 1992, 767–818. https://doi.org/10.1214/aoap/1177005576
  • D. Kramkov, W. Schachermayer, The asymptotic elasticity of utility functions and optimal investment in incomplete markets, Ann. Appl. Probab. 9(3), 1999, 904–950. https://doi.org/10.1214/aoap/1029962818
  • R. C. Merton, Optimum consumption and portfolio rules in a continuous-time model, J. Econom. Theory 3, 1971, 373–413. https://doi.org/10.1016/0022-0531(71)90038-X
16 thms1 active userReviewed
ProbabilityStochastic Systems·Captain: mikedeng1

Utility Maximization in Incomplete Markets I: Under Closed Constraints, the Exponential-Utility Value Is −exp(−α(x − Y₀)) for the Quadratic BSDE (7), and an Optimal Strategy Exists (Theorem 7)Research Paper

Motivation

An investor who trades in a market with fewer stocks than sources of randomness cannot hedge every risk: the market is incomplete. If the investor also faces a liability FFF due at the horizon TTT and has to respect trading constraints (no short sales, limits on positions in certain stocks, a ban on trading some assets altogether), the question of how to trade optimally becomes a constrained stochastic control problem. With exponential utility U(x)=−exp⁡(−αx)U(x)=-\exp(-\alpha x)U(x)=−exp(−αx), its value function gives the investor's utility indifference price of FFF, a standard pricing rule in incomplete markets.

Earlier treatments needed convexity. Cvitanić and Karatzas (Ann. Appl. Probab. 2 (1992)) proved existence and uniqueness for utility maximization in a Brownian filtration with convex constraints, by convex duality. Delbaen, Grandits, Rheinländer, Samperi, Schweizer and Stricker (Math. Finance 12 (2002)) related exponential utility maximization to the martingale measure of minimal relative entropy. El Karoui and Rouge (Math. Finance 10 (2000)) computed the value function and optimal strategy for exponential utility by backward stochastic differential equations (BSDEs), for strategies confined to a convex cone; Sekine (preprint, 2002) obtained the same BSDE from the Cvitanić–Karatzas duality. Hu, Imkeller and Müller (Ann. Appl. Probab. 15 (2005)) gave a direct BSDE solution that works for closed, possibly nonconvex constraint sets, with a bounded liability. The tool is a BSDE whose driver grows quadratically in the control variable, whose solvability rests on Kobylanski's existence theorem for quadratic BSDEs (Ann. Probab. 28 (2000)).

Setting

Fix T>0T>0T>0 and an mmm-dimensional Brownian motion WWW on a probability space (Ω,F,P)(\Omega,\mathcal F,P)(Ω,F,P), and let F=(Ft)\mathbb F=(\mathcal F_t)F=(Ft​) be the PPP-augmentation of the filtration generated by WWW. There are d≤md\le md≤m stocks with prices dSti/Sti=bti dt+σti dWtdS^i_t/S^i_t=b^i_t\,dt+\sigma^i_t\,dW_tdSti​/Sti​=bti​dt+σti​dWt​; the drift bbb and the d×md\times md×m volatility matrix σ\sigmaσ are predictable and uniformly bounded, and KId≥σσtr≥εIdKI_d\ge\sigma\sigma^{\mathrm{tr}}\ge\varepsilon I_dKId​≥σσtr≥εId​ for constants K>ε>0K>\varepsilon>0K>ε>0. The market price of risk is θt=σttr(σtσttr)−1bt∈Rm\theta_t=\sigma^{\mathrm{tr}}_t(\sigma_t\sigma^{\mathrm{tr}}_t)^{-1}b_t\in\mathbb R^mθt​=σttr​(σt​σttr​)−1bt​∈Rm.

A strategy is an R1×m\mathbb R^{1\times m}R1×m-valued predictable process pt=πtσtp_t=\pi_t\sigma_tpt​=πt​σt​, where πt\pi_tπt​ is the vector of amounts held in the stocks. Its wealth from initial capital xxx is

Xt(p)=x+∫0tpu (dWu+θu du).X^{(p)}_t=x+\int_0^tp_u\,(dW_u+\theta_u\,du).Xt(p)​=x+∫0t​pu​(dWu​+θu​du).

The constraint is a closed set C~⊆R1×d\tilde C\subseteq\mathbb R^{1\times d}C~⊆R1×d, not necessarily convex; in terms of ppp it reads pt∈Ct:=C~σtp_t\in C_t:=\tilde C\sigma_tpt​∈Ct​:=C~σt​. A strategy is admissible, p∈Ap\in\mathcal Ap∈A, when E∫0T∣pt∣2dt<∞E\int_0^T|p_t|^2dt<\inftyE∫0T​∣pt​∣2dt<∞, pt∈Ctp_t\in C_tpt​∈Ct​ for λ⊗P\lambda\otimes Pλ⊗P-a.e. (t,ω)(t,\omega)(t,ω), and {exp⁡(−αXτ(p)):τ≤T a stopping time}\{\exp(-\alpha X^{(p)}_\tau):\tau\le T\text{ a stopping time}\}{exp(−αXτ(p)​):τ≤T a stopping time} is uniformly integrable. The value function is

V(x)=sup⁡p∈AE[−exp⁡(−α(XT(p)−F))],V(x)=\sup_{p\in\mathcal A}E\Big[-\exp\Big(-\alpha\big(X^{(p)}_T-F\big)\Big)\Big],V(x)=p∈Asup​E[−exp(−α(XT(p)​−F))],

with α>0\alpha>0α>0 and a bounded FT\mathcal F_TFT​-measurable liability FFF. For a closed set CCC, dist⁡C(a)=min⁡b∈C∣a−b∣\operatorname{dist}_C(a)=\min_{b\in C}|a-b|distC​(a)=minb∈C​∣a−b∣ and ΠC(a)={b∈C:∣a−b∣=dist⁡C(a)}\Pi_C(a)=\{b\in C:|a-b|=\operatorname{dist}_C(a)\}ΠC​(a)={b∈C:∣a−b∣=distC​(a)} is the set of nearest points.

Formalization targets

Goal: Theorem 7

Let f(t,z)=−α2dist⁡2(z+1αθt,Ct)+zθt+12α∣θt∣2f(t,z)=-\frac\alpha2\operatorname{dist}^2\big(z+\frac1\alpha\theta_t,C_t\big)+z\theta_t+\frac1{2\alpha}|\theta_t|^2f(t,z)=−2α​dist2(z+α1​θt​,Ct​)+zθt​+2α1​∣θt​∣2. The BSDE

Yt=F−∫tTZs dWs−∫tTf(s,Zs) ds,t∈[0,T],Y_t=F-\int_t^TZ_s\,dW_s-\int_t^Tf(s,Z_s)\,ds,\qquad t\in[0,T],Yt​=F−∫tT​Zs​dWs​−∫tT​f(s,Zs​)ds,t∈[0,T],

has a unique solution (Y,Z)∈H∞(R)×H2(Rm)(Y,Z)\in\mathcal H^\infty(\mathbb R)\times\mathcal H^2(\mathbb R^m)(Y,Z)∈H∞(R)×H2(Rm), the value function is

V(x)=−exp⁡(−α(x−Y0)),V(x)=-\exp\big(-\alpha(x-Y_0)\big),V(x)=−exp(−α(x−Y0​)),

and there is an optimal strategy p∗∈Ap^*\in\mathcal Ap∗∈A with pt∗∈ΠCt(Zt+1αθt)p^*_t\in\Pi_{C_t}\big(Z_t+\frac1\alpha\theta_t\big)pt∗​∈ΠCt​​(Zt​+α1​θt​).

Milestones

In the order of the proof: the bound (4) on CtC_tCt​ and its closedness; the measurable selection Lemma 11 (a), (b); the identity on p. 8 that dictates fff; the growth bound (9) and the local Lipschitz estimate of fff; existence and uniqueness for (7); the BMO property of Lemma 12; admissibility and optimality of any predictable selection p∗p^*p∗; and the comparison E[−exp⁡(−α(XT(p)−F))]≤−exp⁡(−α(x−Y0))E[-\exp(-\alpha(X^{(p)}_T-F))]\le-\exp(-\alpha(x-Y_0))E[−exp(−α(XT(p)​−F))]≤−exp(−α(x−Y0​)) for every p∈Ap\in\mathcal Ap∈A.

Significance

The theorem reduces a constrained, nonconvex control problem to a single quadratic BSDE: the value is read off the initial value Y0Y_0Y0​, and the optimal strategy is a pointwise nearest-point selection from the constraint set. It requires neither convexity nor duality, and it exhibits non-uniqueness of the optimal strategy when ΠCt\Pi_{C_t}ΠCt​​ has several points. It is the starting point of the quadratic-BSDE approach to utility maximization, indifference pricing and related equilibrium problems.

The result is proved in the paper; nothing in it is open. As far as is known, none of it is formalized: Mathlib has no stochastic integral against Brownian motion, no BSDE theory and no BMO martingales. A complete development would be the first machine-checked solution of a continuous-time portfolio optimization problem by BSDE methods, and its intermediate results (measurable selection of nearest points, the BMO bound for quadratic BSDEs, the verification argument) are reusable.

Difficulty

The deterministic steps (the identity behind the choice of fff, the growth and Lipschitz bounds) are short. The hard steps are elsewhere. Existence rests on Kobylanski's theorem for BSDEs with quadratic growth, whose proof is a monotone approximation argument with exponential transforms; the Lipschitz theory of Pardoux–Peng does not apply. Uniqueness and the optimality of p∗p^*p∗ need the BMO property of ∫Z dW\int Z\,dW∫ZdW and a Girsanov change of measure with a stochastic exponential of a BMO martingale. The selection Lemma 11 (b) cannot use a projection map because C~\tilde CC~ is not convex; a measurable choice from a possibly multivalued nearest-point set is required. Finally the verification argument is a localization: R(p)R^{(p)}R(p) is only a local supermartingale, and passing to the limit uses the uniform integrability built into admissibility.

Formalization scope

Time is R≥0\mathbb R_{\ge0}R≥0​ with T>0T>0T>0; row vectors of R1×m\mathbb R^{1\times m}R1×m are EuclideanSpace ℝ (Fin m) (so ∣⋅∣|\cdot|∣⋅∣ is Euclidean) and products such as zθz\thetazθ are inner products. The Brownian motion is the published EthierKurtz.IsStandardBrownian; the filtration and the stochastic integral come from the published module CvitanicKaratzas92_Optimality_Market (IsAugmentedBrownianFiltration, lebP for λ⊗P\lambda\otimes Pλ⊗P, and an Itô-integral operator I). Predictability is measurability for Mathlib's Filtration.predictable. The market price of risk θ\thetaθ and the wealth X(p)X^{(p)}X(p) are computed from the data, not assumed.

Expectations of −exp⁡(⋅)-\exp(\cdot)−exp(⋅) are minus lower integrals in [0,∞][0,\infty][0,∞], and V(x)V(x)V(x) is an extended real: a Bochner expectation would assign the junk value 000 to non-integrable strategies and make them optimal, which is the trivializing formalization to avoid. Existence and uniqueness of the BSDE solution are part of the goal, not hypotheses, and the optimal strategy is a predictable selection of the nearest-point set, not of a choice function.

Disclosed readings: C~≠∅\tilde C\neq\emptysetC~=∅ is assumed (otherwise A=∅\mathcal A=\emptysetA=∅); constraints and (8) are read λ⊗P\lambda\otimes Pλ⊗P-a.e.; ellipticity holds for PPP-a.e. ω\omegaω and all t≤Tt\le Tt≤T; Y0Y_0Y0​ is a.s. constant and the value identity holds a.s.; uniqueness is in YtY_tYt​ a.s. for each ttt and ZZZ λ⊗P\lambda\otimes Pλ⊗P-a.e.; "p∗p^*p∗ given by Lemma 11" is read as any predictable selection; Lemma 11 (b) assumes full rank of σt(ω)\sigma_t(\omega)σt​(ω). The equivalence of the π\piπ- and ppp-formulations (Remark 5), the dynamic principle (Proposition 9) and Remarks 2–10 are out of scope.

Needed infrastructure: Itô calculus for continuous semimartingales, stochastic exponentials, BMO martingales and Kazamaki's criterion, Girsanov's theorem, quadratic BSDEs, and measurable selection. Contributions to any of these are welcome; they serve the companion power-utility mission (Theorem 14) as well.

Selected references

  • Y. Hu, P. Imkeller, M. Müller, Utility maximization in incomplete markets, Ann. Appl. Probab. 15(3), 1691–1712, 2005. https://doi.org/10.1214/105051605000000188 (arXiv: https://arxiv.org/abs/math/0508448)
  • M. Kobylanski, Backward stochastic differential equations and partial differential equations with quadratic growth, Ann. Probab. 28(2), 558–602, 2000. https://doi.org/10.1214/aop/1019160253
  • J. Cvitanić, I. Karatzas, Convex duality in constrained portfolio optimization, Ann. Appl. Probab. 2(4), 767–818, 1992. https://doi.org/10.1214/aoap/1177005576
  • E. Pardoux, S. Peng, Adapted solution of a backward stochastic differential equation, Systems Control Lett. 14(1), 55–61, 1990. https://doi.org/10.1016/0167-6911(90)90082-6
  • N. El Karoui, R. Rouge, Pricing via utility maximization and entropy, Math. Finance 10(2), 259–276, 2000. https://doi.org/10.1111/1467-9965.00093
  • F. Delbaen, P. Grandits, T. Rheinländer, D. Samperi, M. Schweizer, C. Stricker, Exponential hedging and entropic penalties, Math. Finance 12(2), 99–123, 2002. https://doi.org/10.1111/1467-9965.02001
  • N. Kazamaki, Continuous Exponential Martingales and BMO, Lecture Notes in Math. 1579, Springer, 1994. https://doi.org/10.1007/BFb0073585
18 thms1 active userReviewed
AnalysisOperations Research·Captain: mikedeng1

On Minimizing the Ruin Probability by Investment and Reinsurance II: Equation (5) for g = f′ Has a Unique Strictly Decreasing Solution on [0, ∞)Research Paper

Motivation

An insurance company whose surplus falls below zero is ruined. The probability of ruin is the classical measure of solvency risk in actuarial mathematics. A company can lower it in two ways: it can invest part of its surplus in a risky asset, and it can pass part of each claim to a reinsurer in exchange for part of the premium. Which mix of investment and reinsurance minimises the probability of ruin is a stochastic control problem. It is studied in risk theory and in applied probability, and it is a model case for control problems with jumps.

Hipp and Plum (2000) solved the problem with investment alone. They wrote down the Hamilton–Jacobi–Bellman (HJB) equation and showed that it has a smooth solution. H. Schmidli (2002) added proportional reinsurance. He proved a verification theorem (his Theorem 1) and an existence theorem (his Theorem 2) for the combined problem. This mission formalizes the existence theorem and the analytic statements it rests on.

Setting

Claims arrive as a Poisson process with rate λ>0\lambda>0λ>0. The claim sizes YYY are independent with distribution function GGG, where G(0)=0G(0)=0G(0)=0 and GGG is continuous. The insurer collects premiums at rate c>0c>0c>0. A risky asset follows a geometric Brownian motion with drift μ>0\mu>0μ>0 and volatility σ>0\sigma>0σ>0. Under proportional reinsurance with retention level b∈[0,1]b\in[0,1]b∈[0,1], the insurer pays the fraction bYbYbY of each claim and pays the reinsurer a premium at rate c(b)c(b)c(b). The function c(b)c(b)c(b) is decreasing and continuous, c(1)=0c(1)=0c(1)=0, lim inf⁡b↑1c(b)/(1−b)>0\liminf_{b\uparrow1}c(b)/(1-b)>0liminfb↑1​c(b)/(1−b)>0, and there is b‾>0\underline b>0b​>0 with c(b)>cc(b)>cc(b)>c for b<b‾b<\underline bb<b​ and c(b)≤cc(b)\le cc(b)≤c for b≥b‾b\ge\underline bb≥b​. In particular, full reinsurance costs more than the premium income.

Write fff for an unnormalised survival probability, with f(u)=0f(u)=0f(u)=0 for u<0u<0u<0. The HJB equation (1) of the problem is

sup⁡b∈[0,1]sup⁡A≥0[12σ2A2f′′(u)+(c−c(b)+μA)f′(u)+λ(E[f(u−bY)]−f(u))]=0,\sup_{b\in[0,1]}\sup_{A\ge0}\Big[\tfrac12\sigma^2A^2f''(u)+\big(c-c(b)+\mu A\big)f'(u)+\lambda\big(\mathbb E[f(u-bY)]-f(u)\big)\Big]=0 ,b∈[0,1]sup​A≥0sup​[21​σ2A2f′′(u)+(c−c(b)+μA)f′(u)+λ(E[f(u−bY)]−f(u))]=0,

where AAA is the amount invested. Eliminating AAA and substituting g=f′g=f'g=f′ turns (1) into the integral equation (5):

g(u)=1μ22σ2∫0udxDg(x)+cλ,g(u)=\frac{1}{\dfrac{\mu^2}{2\sigma^2}\displaystyle\int_0^u\frac{dx}{D_g(x)}+\dfrac{c}{\lambda}},g(u)=2σ2μ2​∫0u​Dg​(x)dx​+λc​1​, Dg(x)=inf⁡b∈[0,1][λ(1−G(x/b)+∫0x(1−G((x−z)/b))g(z) dz)−(c−c(b))g(x)].D_g(x)=\inf_{b\in[0,1]}\Big[\lambda\Big(1-G(x/b)+\int_0^x\big(1-G((x-z)/b)\big)g(z)\,dz\Big)-\big(c-c(b)\big)g(x)\Big].Dg​(x)=b∈[0,1]inf​[λ(1−G(x/b)+∫0x​(1−G((x−z)/b))g(z)dz)−(c−c(b))g(x)].

Condition (6) at uuu is Dg(u)>0D_g(u)>0Dg​(u)>0. In Lean, ReinsPremium, ClaimLaw, tail, den (=Dg=D_g=Dg​), SolvesEq5On, H and SolvesHJB (equation (1)) live in the shared module SchmidliRuin.Verif.Setting; HasBoundedDensity, Cond6, SolvesEq3At and SolvesHJBAtZero live in SchmidliRuin.Exist.Setting.

Formalization targets

Goal: Theorem 2 (p. 900)

If GGG has a bounded density, then

∃! g:[0,∞)→R strictly decreasing, solving (5) on [0,∞).\exists!\ g:[0,\infty)\to\mathbb R\ \text{strictly decreasing, solving (5) on }[0,\infty).∃! g:[0,∞)→R strictly decreasing, solving (5) on [0,∞).

Lean states this as two conjuncts: existence, and equality on [0,∞)[0,\infty)[0,∞) of any two strictly decreasing solutions.

Milestones, in attack order

  1. Lemma 3 (p. 894). For a solution fff of (3) on [0,η)[0,\eta)[0,η), the optimal retention is b∗(u)=1b^*(u)=1b∗(u)=1 for small uuu: it is optimal not to reinsure small capital.
  2. Lemma 4 (p. 898). Under a bounded density, (5) has a solution on some [0,ε)[0,\varepsilon)[0,ε), with g(u)=λ/c−αu+o(u)g(u)=\lambda/c-\alpha\sqrt u+o(\sqrt u)g(u)=λ/c−αu​+o(u​) and α=λμ/(σc3/2)\alpha=\lambda\mu/(\sigma c^{3/2})α=λμ/(σc3/2).
  3. Condition (6) near 000 (pp. 898–899): such a local solution satisfies Dg(u)>0D_g(u)>0Dg​(u)>0 for small u>0u>0u>0.
  4. Lemma 5 (p. 899). A decreasing solution on [0,u0)[0,u_0)[0,u0​) with (6) on (0,u0)(0,u_0)(0,u0​) extends to [0,u0][0,u_0][0,u0​], and (6) holds at u0u_0u0​.
  5. Continuation (proof of Theorem 2, p. 901). A solution with (6) on (0,u0](0,u_0](0,u0​] continues to [0,u0+η)[0,u_0+\eta)[0,u0​+η) with (6).
  6. Theorem 1, second sentence (p. 896). For a decreasing solution ggg of (5), f(x)=1+∫0xgf(x)=1+\int_0^xgf(x)=1+∫0x​g is a strictly increasing solution of (1), twice continuously differentiable on (0,∞)(0,\infty)(0,∞).
  7. Theorem 1, last sentence (p. 896). There is at most one strictly increasing, twice continuously differentiable solution of (1) with f(0)=1f(0)=1f(0)=1.

Significance

The existence theorem completes the dynamic-programming solution of the problem. By the verification theorem, f=1+∫0⋅gf=1+\int_0^\cdot gf=1+∫0⋅​g divided by f(∞)f(\infty)f(∞) is the maximal survival probability δ(u)\delta(u)δ(u). The maximisers of the HJB equation define the optimal feedback strategy: the investment A∗(u)=−μf′(u)/(σ2f′′(u))A^*(u)=-\mu f'(u)/(\sigma^2f''(u))A∗(u)=−μf′(u)/(σ2f′′(u)) and the retention level b∗(u)b^*(u)b∗(u). Without Theorem 2 the verification theorem would be a statement about a function that might not exist. Equation (5) is also the basis of the paper's numerical scheme (§5), which iterates (5) to compute δ\deltaδ and the optimal strategies.

Theorem 2 has a published proof. As far as is known, none of the results above has a machine-checked proof, and nothing on Prove2Me states them. The work is to formalize the published argument. This takes the local existence result of Hipp and Plum, which the paper cites rather than reproves, and a continuation argument for a nonlinear Volterra-type equation whose kernel is an infimum over a control parameter. Milestones 6 and 7 are the purely analytic content of the verification theorem that the uniqueness part of Theorem 2 uses. The paper derives milestone 7 stochastically, and an analytic proof is equally welcome.

Difficulty

The denominator Dg(x)D_g(x)Dg​(x) vanishes at x=0x=0x=0: Dg(0)=λ−c g(0)=0D_g(0)=\lambda-c\,g(0)=0Dg​(0)=λ−cg(0)=0, because g(0)=λ/cg(0)=\lambda/cg(0)=λ/c. The integral in (5) is therefore improper at the origin. A Picard iteration started from a constant does not work, because it does not control ∫0udx/Dg(x)\int_0^u dx/D_g(x)∫0u​dx/Dg​(x). One needs the exact square-root rate Dg(x)≍xD_g(x)\asymp\sqrt xDg​(x)≍x​, and Lemma 4 obtains it only through Lemma 3 (no reinsurance near 000) and the Hipp–Plum analysis of the reinsurance-free equation. Lemma 4's proof in the paper is a two-line citation, so a solver must formalize that local existence result.

The second difficulty is continuation. A solution can be continued as long as condition (6) holds, and Lemma 5 has to exclude that (6) degenerates at a finite endpoint. There the infimum over bbb interacts with the vanishing of ggg. The paper's argument uses a Gronwall estimate and the shape of c(b)c(b)c(b) around b‾\underline bb​, and the minimiser bbb in DgD_gDg​ need not be unique or continuous.

Formalization scope

  • Model data. c,λ,μ,σc,\lambda,\mu,\sigmac,λ,μ,σ are real parameters, all strictly positive. The claim law is a probability measure ν\nuν on R\mathbb RR, with G=G=G= cdf ν, ν((−∞,0])=0\nu((-\infty,0])=0ν((−∞,0])=0 and GGG continuous. "Bounded density" means ν\nuν has a measurable density with values in [0,C][0,C][0,C]. E[h(u−bY)]\mathbb E[h(u-bY)]E[h(u−bY)] is ∫h(u−by) dν(y)\int h(u-by)\,d\nu(y)∫h(u−by)dν(y).

  • Premium. c(b)c(b)c(b) is a real function, used on [0,1][0,1][0,1]. The condition lim inf⁡b↑1c(b)/(1−b)>0\liminf_{b\uparrow1}c(b)/(1-b)>0liminfb↑1​c(b)/(1−b)>0 is the lower bound c(b)≥κ(1−b)c(b)\ge\kappa(1-b)c(b)≥κ(1−b) near 111, which is equivalent. A real-valued ccc excludes the case c(0)=∞c(0)=\inftyc(0)=∞ that the paper allows when E[Y]<∞\mathbb E[Y]<\inftyE[Y]<∞.

  • Conventions.

    • 1−G(x/b)1-G(x/b)1−G(x/b) at b=0b=0b=0 is P[0⋅Y>x]=0\mathbb P[0\cdot Y>x]=0P[0⋅Y>x]=0; it is computed by tail, never by Lean's x/0=0x/0=0x/0=0.
    • "ggg solves (5) at uuu" includes that the integral exists: ggg is integrable on [0,u][0,u][0,u], Dg≠0D_g\ne0Dg​=0 on (0,u)(0,u)(0,u), and 1/Dg1/D_g1/Dg​ is integrable on [0,u][0,u][0,u].
    • Every supremum of the paper is a least upper bound (IsLUB … 0). Lean's sSup would return 000 on an unbounded family, for example when f′′≥0f''\ge0f′′≥0, and that would make "fff solves (1)" true by accident.
    • Uniqueness is equality on [0,∞)[0,\infty)[0,∞). The values of ggg off [0,∞)[0,\infty)[0,∞) are free, so a unique-existence quantifier over R→R\mathbb R\to\mathbb RR→R would be false.
    • "Twice continuously differentiable" is read on (0,∞)(0,\infty)(0,∞), since f′′(0+)=−∞f''(0+)=-\inftyf′′(0+)=−∞ (p. 895). At u=0u=0u=0, (1) and (3) are taken in the boundary form sup⁡b[(c−c(b))f′(0)+λ(Ef(−bY)−f(0))]=0\sup_b[(c-c(b))f'(0)+\lambda(\mathbb E f(-bY)-f(0))]=0supb​[(c−c(b))f′(0)+λ(Ef(−bY)−f(0))]=0, with the right derivative f′(0)f'(0)f′(0).
    • λ\lambdaλ and ccc are general. §4 sets λ=c=1\lambda=c=1λ=c=1 by a change of units.
  • Trivialising readings ruled out.

    • Encoding (5) with an unguarded 1/Dg1/D_g1/Dg​ would let a function with a vanishing denominator "solve" (5).
    • Dropping the boundary equation at u=0u=0u=0 from Lemma 3 makes it false: a premium flat near b=0b=0b=0 admits local solutions with b∗≡0b^*\equiv0b∗≡0.
  • Infrastructure needed.

    • existence of the minimiser in DgD_gDg​ and continuity of DgD_gDg​ in xxx;
    • a local existence theorem for singular integral equations in the style of Hipp–Plum;
    • Gronwall's inequality (Mathlib has versions);
    • Banach's fixed-point theorem (ContractingWith).

    The continuation and closure lemmas (milestones 3–5) are reusable for other HJB equations of risk theory with a jump term. Contributions that state and prove intermediate claims of the paper's proofs are welcome as sub-results.

Selected references

  • H. Schmidli, On minimizing the ruin probability by investment and reinsurance, Ann. Appl. Probab. 12(3), 890–907, 2002. https://doi.org/10.1214/aoap/1031863173
  • C. Hipp and M. Plum, Optimal investment for insurers, Insurance Math. Econom. 27(2), 215–228, 2000. https://doi.org/10.1016/S0167-6687(00)00049-4
  • H. Schmidli, Optimal proportional reinsurance policies in a dynamic setting, Scand. Actuar. J. 2001(1), 55–68, 2001. https://doi.org/10.1080/034612301750077338
  • S. N. Ethier and T. G. Kurtz, Markov Processes: Characterization and Convergence, Wiley, 1986. https://doi.org/10.1002/9780470316658
10 thms1 active userReviewed
Functional AnalysisNumerical AnalysisOptimization·Captain: mikedeng1

The Primal-Dual Active Set Strategy as a Semismooth Newton Method V: Local Superlinear Convergence in L² under a Norm GapResearch Paper

Motivation

Optimal control problems with pointwise bounds on the control, such as

min⁡u12∥y−z∥L22+β2∥u∥L22subject to −Δy=u in Ω, y=0 on ∂Ω, u≤ψ,\min_u \tfrac12\|y - z\|_{L^2}^2 + \tfrac{\beta}{2}\|u\|_{L^2}^2 \quad\text{subject to } -\Delta y = u \text{ in } \Omega,\ y = 0 \text{ on } \partial\Omega,\ u \le \psi,umin​21​∥y−z∥L22​+2β​∥u∥L22​subject to −Δy=u in Ω, y=0 on ∂Ω, u≤ψ,

reduce, after eliminating the state, to a quadratic program in L2(Ω)L^2(\Omega)L2(Ω) with a pointwise upper bound. The primal-dual active set strategy solves such problems by guessing the set where the bound is active, solving one linear system on that guess, and updating the guess from the new primal and dual values. It is used routinely in PDE-constrained optimization because each iteration costs one linear solve and, in practice, very few iterations are needed.

Hintermüller, Ito and Kunisch (SIAM J. Optim. 13 (2002) 865–888; authors' version hal-01660511) showed that this strategy is a semismooth Newton method for a nonsmooth reformulation of the optimality system, built on the notion of slant differentiability of Chen, Nashed and Qi (SIAM J. Numer. Anal. 38 (2000) 1200–1216). In finite dimension this gives local superlinear convergence (missions I–IV of this series). This mission concerns the paper's function-space result, Theorem 4.1: in L2(Ω)L^2(\Omega)L2(Ω), under a norm gap on the operator, the method still converges locally superlinearly. Ulbrich (SIAM J. Optim. 13 (2003) 805–841) developed a related semismooth Newton theory for superposition operators in function spaces, which also requires a norm gap.

Setting

Let (Ω,μ)(\Omega, \mu)(Ω,μ) be a finite measure space (the paper takes a bounded domain Ω⊂Rn\Omega \subset \mathbb{R}^nΩ⊂Rn with Lebesgue measure) and Lp=Lp(Ω)L^p = L^p(\Omega)Lp=Lp(Ω) the real Lebesgue spaces, with inner product (⋅,⋅)(\cdot,\cdot)(⋅,⋅) on L2L^2L2. The paper writes AAA for an operator and calligraphic A\mathcal{A}A, I\mathcal{I}I for active and inactive sets.

Slant differentiability (Definition 1). For Banach spaces XXX, ZZZ, a map F:X→ZF : X \to ZF:X→Z is slantly differentiable in an open set UUU if there is G:U→L(X,Z)G : U \to \mathcal{L}(X, Z)G:U→L(X,Z), a slanting function, with

lim⁡h→01∥h∥ ∥F(x+h)−F(x)−G(x+h)h∥=0(x∈U).(A)\lim_{h\to 0}\frac{1}{\|h\|}\,\|F(x+h) - F(x) - G(x+h)h\| = 0 \qquad (x \in U). \tag{A}h→0lim​∥h∥1​∥F(x+h)−F(x)−G(x+h)h∥=0(x∈U).(A)

For δ∈R\delta \in \mathbb{R}δ∈R, the candidate slanting function of the pointwise max is Gm(y)(x)=1G_m(y)(x) = 1Gm​(y)(x)=1 if y(x)>0y(x) > 0y(x)>0, 000 if y(x)<0y(x) < 0y(x)<0, δ\deltaδ if y(x)=0y(x) = 0y(x)=0 (4.1).

The problem. Given f,ψ∈L2f, \psi \in L^2f,ψ∈L2 and a self-adjoint A∈L(L2)A \in \mathcal{L}(L^2)A∈L(L2) with

(Ay,y)≥γ∥y∥2(γ>0),(H1)(Ay, y) \ge \gamma\|y\|^2 \quad (\gamma > 0), \tag{H1}(Ay,y)≥γ∥y∥2(γ>0),(H1)

minimize 12(y,Ay)−(f,y)\tfrac12(y, Ay) - (f, y)21​(y,Ay)−(f,y) subject to y≤ψy \le \psiy≤ψ. The solution y∗y^*y∗ and its multiplier λ∗\lambda^*λ∗ are the unique solution of

Ay∗+λ∗=f,λ∗−max⁡(0,λ∗+c(y∗−ψ))=0 a.e.,(4.2)Ay^* + \lambda^* = f, \qquad \lambda^* - \max(0, \lambda^* + c(y^* - \psi)) = 0 \ \text{a.e.}, \tag{4.2}Ay∗+λ∗=f,λ∗−max(0,λ∗+c(y∗−ψ))=0 a.e.,(4.2)

for any fixed c>0c > 0c>0. The norm gap hypothesis is

A=C+βI,C∈L(L2,Lq),β>0,q>2.(H2)A = C + \beta I, \qquad C \in \mathcal{L}(L^2, L^q),\quad \beta > 0,\quad q > 2. \tag{H2}A=C+βI,C∈L(L2,Lq),β>0,q>2.(H2)

The algorithm. From (y0,λ0)∈L2×L2(y^0, \lambda^0) \in L^2 \times L^2(y0,λ0)∈L2×L2, set Ak={x:λk(x)+c(yk(x)−ψ(x))>0}\mathcal{A}_k = \{x : \lambda^k(x) + c(y^k(x) - \psi(x)) > 0\}Ak​={x:λk(x)+c(yk(x)−ψ(x))>0}, Ik=Ω∖Ak\mathcal{I}_k = \Omega\setminus\mathcal{A}_kIk​=Ω∖Ak​, and solve

Ayk+1+λk+1=f,yk+1=ψ on Ak,λk+1=0 on Ik.Ay^{k+1} + \lambda^{k+1} = f, \qquad y^{k+1} = \psi \text{ on } \mathcal{A}_k, \qquad \lambda^{k+1} = 0 \text{ on } \mathcal{I}_k.Ayk+1+λk+1=f,yk+1=ψ on Ak​,λk+1=0 on Ik​.

A sequence obeying this rule for every kkk is a run. Superlinear convergence of xk→x∗x^k \to x^*xk→x∗ means xk→x∗x^k \to x^*xk→x∗ and, for every η>0\eta > 0η>0, eventually ∥xk+1−x∗∥≤η∥xk−x∗∥\|x^{k+1} - x^*\| \le \eta\|x^k - x^*\|∥xk+1−x∗∥≤η∥xk−x∗∥.

Formalization targets

Goal: Theorem 4.1 (p. 15)

The page reads: "Assume that (H1), (H2) hold and that ψ and f are in L^q(Ω). Then the primal-dual active set strategy or, equivalently, the semismooth Newton method converge superlinearly if ‖y⁰ − y*‖ is sufficiently small and λ⁰ = β(y⁰ − ψ)." The formalized goal is the statement the proof establishes: with c=βc = \betac=β, there is ρ>0\rho > 0ρ>0 such that every run with

∥y0−y∗∥L2<ρ,λ0=f−Ay0\|y^0 - y^*\|_{L^2} < \rho, \qquad \lambda^0 = f - Ay^0∥y0−y∗∥L2​<ρ,λ0=f−Ay0

converges superlinearly to (y∗,λ∗)(y^*, \lambda^*)(y∗,λ∗) in L2×L2L^2 \times L^2L2×L2. The initialization differs from the printed one; see Formalization scope.

Milestones

  1. Theorem 1.1 (Chen–Nashed–Qi): slant differentiability near a zero x∗x^*x∗ with uniformly bounded G(x)−1G(x)^{-1}G(x)−1 gives local superlinear convergence of xk+1=xk−G(xk)−1F(xk)x^{k+1} = x^k - G(x^k)^{-1}F(x^k)xk+1=xk−G(xk)−1F(xk).
  2. Proposition 4.1(ii): for 1≤p<q≤∞1 \le p < q \le \infty1≤p<q≤∞, max⁡(0,⋅):Lq→Lp\max(0,\cdot) : L^q \to L^pmax(0,⋅):Lq→Lp is slantly differentiable with slanting function GmG_mGm​.
  3. (4.9)–(4.10): for c=βc = \betac=β, (4.2) is equivalent to βy∗−βψ+max⁡(0,Cy∗−f+βψ)=0\beta y^* - \beta\psi + \max(0, Cy^* - f + \beta\psi) = 0βy∗−βψ+max(0,Cy∗−f+βψ)=0 and λ∗=f−Cy∗−βy∗\lambda^* = f - Cy^* - \beta y^*λ∗=f−Cy∗−βy∗.
  4. Equivalence of the algorithms: along a run with c=βc = \betac=β and λ0=f−Ay0\lambda^0 = f - Ay^0λ0=f−Ay0, λk+β(yk−ψ)=f−Cyk−βψ\lambda^k + \beta(y^k - \psi) = f - Cy^k - \beta\psiλk+β(yk−ψ)=f−Cyk−βψ for all kkk, so the active sets are those of the reduced algorithm.
  5. Slant differentiability of the reduced map F(y)=βy−βψ+max⁡(0,Cy−f+βψ)F(y) = \beta y - \beta\psi + \max(0, Cy - f + \beta\psi)F(y)=βy−βψ+max(0,Cy−f+βψ) on L2L^2L2, with GF(y+h)=βI+Gm(Cy−f+βψ+Ch)CG_F(y+h) = \beta I + G_m(Cy - f + \beta\psi + Ch)CGF​(y+h)=βI+Gm​(Cy−f+βψ+Ch)C, δ=1\delta = 1δ=1.
  6. Bounded inverses: GF(z)G_F(z)GF​(z) is invertible on L2L^2L2, with ∥GF(z)−1∥\|G_F(z)^{-1}\|∥GF​(z)−1∥ bounded uniformly in zzz.

Two companion items accompany the goal: existence of a run from every y0y^0y0, and Proposition 4.1(i), the paper's counterexamples showing that GmG_mGm​ fails to be a slanting function from LpL^pLp to LpL^pLp.

Significance

Theorem 4.1 is the infinite-dimensional justification of a method used throughout control-constrained optimal control: it shows that the method converges locally superlinearly for the continuous problem itself, not only for each discretization. Example 1 of the paper shows that distributed control of the Poisson equation, and boundary control with Sobolev embeddings, satisfy (H2). Proposition 4.1 isolates the structural reason: the pointwise max is a Newton-differentiable map only between spaces with a norm gap.

None of these results is known to be machine-checked. A complete development would provide slant differentiability in Banach spaces, the Chen–Nashed–Qi local convergence theorem, the norm-gap differentiability of max⁡(0,⋅)\max(0,\cdot)max(0,⋅) between Lebesgue spaces, and invertibility of block operators on L2L^2L2 of a measurable partition. Each of these is reusable for other semismooth Newton analyses in function space.

Difficulty

The obvious argument fails at the max. In finite dimension, max⁡(0,⋅)\max(0,\cdot)max(0,⋅) is slantly differentiable with GmG_mGm​ because a small perturbation does not change the sign of any nonzero component. In L2L^2L2, a perturbation hhh of small norm can flip the sign of yyy on a set of positive measure, and on that set the error max⁡(0,y+h)−max⁡(0,y)−Gm(y+h)h\max(0, y+h) - \max(0, y) - G_m(y+h)hmax(0,y+h)−max(0,y)−Gm​(y+h)h is as large as ∣y∣|y|∣y∣. Proposition 4.1(i) shows that this error is not o(∥h∥Lp)o(\|h\|_{L^p})o(∥h∥Lp​) in LpL^pLp for any ppp. Superlinear convergence in L2L^2L2 therefore needs the argument of the max to be controlled in a stronger norm than the error is measured in, which is what (H2) and f,ψ∈Lqf, \psi \in L^qf,ψ∈Lq provide. A second difficulty is that the two-variable algorithm is not literally a Newton method for a map of yyy alone; it must be identified with the reduced algorithm, and that identification depends on the initialization.

Formalization scope

  • Spaces. LpL^pLp is Mathlib's Lp ℝ p μ on an arbitrary finite measure space; q∈(2,∞]q \in (2, \infty]q∈(2,∞] is an extended real with 2 < q. The paper's bounded Lipschitz domain is replaced by a finite measure space: the proofs use only ∣Ω∣<∞|\Omega| < \infty∣Ω∣<∞, and Mathlib has no notion of Lipschitz boundary.
  • Pointwise statements on LpL^pLp classes are almost everywhere statements about representatives. (H2) is stated as Ay=Cy+βyAy = Cy + \beta yAy=Cy+βy a.e. for every y∈L2y \in L^2y∈L2, which avoids building the inclusion Lq⊆L2L^q \subseteq L^2Lq⊆L2.
  • Hypotheses kept: self-adjointness of AAA, (H1) with explicit γ>0\gamma > 0γ>0, (H2) with β>0\beta > 0β>0 and q>2q > 2q>2, f,ψ∈Lqf, \psi \in L^qf,ψ∈Lq. The solution (y∗,λ∗)(y^*, \lambda^*)(y∗,λ∗) of (4.2) is a hypothesis; it exists and is unique under (H1), as the paper states on p. 12.
  • The initialization. The printed λ0=β(y0−ψ)\lambda^0 = \beta(y^0 - \psi)λ0=β(y0−ψ) does not support the proof, which uses "λ0+β(y0−ψ)=f−Cy0−βψ\lambda^0 + \beta(y^0 - \psi) = f - Cy^0 - \beta\psiλ0+β(y0−ψ)=f−Cy0−βψ" and c=βc = \betac=β. That identity holds exactly for λ0=f−Ay0\lambda^0 = f - Ay^0λ0=f−Ay0. The goal uses λ0=f−Ay0\lambda^0 = f - Ay^0λ0=f−Ay0 and c=βc = \betac=β. With the printed initialization the first active set is {y0>ψ}\{y^0 > \psi\}{y0>ψ}, and for y0y^0y0 slightly below y∗y^*y∗ the first iterate solves the unconstrained problem, which can be far from y∗y^*y∗.
  • Superlinear convergence is stated for the pair (yk,λk)(y^k, \lambda^k)(yk,λk) in the product norm max⁡(∥y∥,∥λ∥)\max(\|y\|, \|\lambda\|)max(∥y∥,∥λ∥), without dividing by ∥xk−x∗∥\|x^k - x^*\|∥xk−x∗∥, so finite termination qualifies. "Sufficiently small" is an ∃ρ>0\exists\rho > 0∃ρ>0 chosen before the run, on ∥y0−y∗∥\|y^0 - y^*\|∥y0−y∗∥ only.
  • No trivialization. The goal mentions neither the reduced algorithm, nor GFG_FGF​, nor Proposition 4.1; the norm gap q>2q > 2q>2 is not weakened to C∈L(L2)C \in \mathcal{L}(L^2)C∈L(L2), under which Proposition 4.1(i) shows that GmG_mGm​ need not be a slanting function; and the companion run_exists shows that the runs the goal quantifies over exist.
  • Definitions (PDASNewton.FunSpace.Setting): slanting functions, superlinear convergence, GmG_mGm​, property (A) for max⁡(0,⋅):Lq→Lp\max(0,\cdot) : L^q \to L^pmax(0,⋅):Lq→Lp, system (4.2), one step and runs of the algorithm. These duplicate the finite-dimensional definitions of the other missions of the series in a separate namespace.

Proofs of any milestone are welcome, as are reusable lemmas on Lebesgue spaces (Hölder on sets of small measure, multiplication operators, Lax–Milgram on L2L^2L2 of a measurable subset).

Selected references

  • M. Hintermüller, K. Ito, K. Kunisch, The primal-dual active set strategy as a semismooth Newton method, SIAM J. Optim. 13(3) (2002) 865–888. https://doi.org/10.1137/S1052623401383558 ; authors' version https://hal.science/hal-01660511
  • X. Chen, Z. Nashed, L. Qi, Smoothing methods and semismooth methods for nondifferentiable operator equations, SIAM J. Numer. Anal. 38 (2000) 1200–1216. https://doi.org/10.1137/S0036142999356719
  • M. Ulbrich, Semismooth Newton methods for operator equations in function spaces, SIAM J. Optim. 13 (2003) 805–841. https://doi.org/10.1137/S1052623400371569
  • L. Qi, J. Sun, A nonsmooth version of Newton's method, Math. Programming 58 (1993) 353–367. https://doi.org/10.1007/BF01581275
9 thms1 active userReviewed
AnalysisDynamical Systems·Captain: mikedeng1

Small Gain Theorems for Large Scale Systems and Construction of ISS Lyapunov Functions 2: Under the Small Gain Condition Γ_μ ≱ id, an Ω-Path ExistsResearch Paper

Why paths in the positive orthant matter for networks

A large control system is often built from many subsystems Σ1,…,Σn\Sigma_1,\dots,\Sigma_nΣ1​,…,Σn​, each of which is input-to-state stable (ISS) with respect to the states of the others. Whether the interconnection is again ISS is decided by a small gain condition on the matrix of interconnection gains. For two subsystems this goes back to Jiang, Teel and Praly (1994) and to the Lyapunov version of Jiang, Mareels and Wang (1996). For nnn subsystems, Dashkovskiy, Rüffer and Wirth formulated the condition as a property Γμ≱id\Gamma_\mu\not\ge\mathrm{id}Γμ​≥id of a monotone operator on R+n\mathbb R^n_+R+n​ (2007). Rüffer's thesis (2007) and the paper on which this mission is based (arXiv:0901.1842, SIAM J. Control Optim. 2010) turned it into a Lyapunov construction.

The construction needs a geometric object: a path σ\sigmaσ in the positive orthant, unbounded in every coordinate, along which the gain operator strictly decreases every component. For a linear gain matrix this is a Perron vector. For nonlinear gains, its existence is the first of the paper's two main results (Theorem 5.2). This mission formalizes that theorem and the lemmas of §8 that prove it. The companion mission formalizes the second main result, the Lyapunov function built from such a path.

Setting

Write R+=[0,∞)\mathbb R_+=[0,\infty)R+​=[0,∞). Vectors in R+n\mathbb R^n_+R+n​ are compared componentwise: v≤wv\le wv≤w means vi≤wiv_i\le w_ivi​≤wi​ for all iii, and v<wv<wv<w means vi<wiv_i<w_ivi​<wi​ for all iii. A function γ:R+→R+\gamma:\mathbb R_+\to\mathbb R_+γ:R+​→R+​ is of class K\mathcal KK if it is continuous, strictly increasing and γ(0)=0\gamma(0)=0γ(0)=0, and of class K∞\mathcal K_\inftyK∞​ if it is moreover unbounded.

A gain matrix is Γ=(γij)i,j=1n\Gamma=(\gamma_{ij})_{i,j=1}^nΓ=(γij​)i,j=1n​ with γij∈K∪{0}\gamma_{ij}\in\mathcal K\cup\{0\}γij​∈K∪{0} and γii≡0\gamma_{ii}\equiv0γii​≡0. A monotone aggregation function (MAF) is a continuous μ:R+n→R+\mu:\mathbb R^n_+\to\mathbb R_+μ:R+n​→R+​ with μ(s)>0\mu(s)>0μ(s)>0 for s≠0s\ne0s=0, μ(x)<μ(y)\mu(x)<\mu(y)μ(x)<μ(y) whenever x<yx<yx<y, and μ(x)→∞\mu(x)\to\inftyμ(x)→∞ as ∥x∥→∞\|x\|\to\infty∥x∥→∞. Typical examples are sums and maxima. Given MAFs μ1,…,μn\mu_1,\dots,\mu_nμ1​,…,μn​, the gain operator is

Γμ(s)i=μi(γi1(s1),…,γin(sn)),s∈R+n.\Gamma_\mu(s)_i=\mu_i\big(\gamma_{i1}(s_1),\dots,\gamma_{in}(s_n)\big),\qquad s\in\mathbb R^n_+ .Γμ​(s)i​=μi​(γi1​(s1​),…,γin​(sn​)),s∈R+n​.

The paper assumes from p. 8 on that μi\mu_iμi​, restricted to the coordinates of the nonzero entries of row iii, still has the strict monotonicity (Remark 2.6). The small gain condition (SGC) is Γμ≱id\Gamma_\mu\not\ge\mathrm{id}Γμ​≥id: for every s≠0s\ne0s=0, at least one component of Γμ(s)\Gamma_\mu(s)Γμ​(s) is strictly smaller than the corresponding component of sss. The set

Ω(Γμ)={s∈R+n:Γμ(s)<s}\Omega(\Gamma_\mu)=\{s\in\mathbb R^n_+:\Gamma_\mu(s)<s\}Ω(Γμ​)={s∈R+n​:Γμ​(s)<s}

collects the points that every component of Γμ\Gamma_\muΓμ​ strictly decreases.

An Ω\OmegaΩ-path (Definition 5.1) is a continuous σ:R+→R+n\sigma:\mathbb R_+\to\mathbb R^n_+σ:R+​→R+n​ with every component σi∈K∞\sigma_i\in\mathcal K_\inftyσi​∈K∞​ such that (i) each σi−1\sigma_i^{-1}σi−1​ is locally Lipschitz on (0,∞)(0,\infty)(0,∞); (ii) on every compact K⊂(0,∞)K\subset(0,\infty)K⊂(0,∞) the derivatives of all σi−1\sigma_i^{-1}σi−1​, where they exist, lie in [c,C][c,C][c,C] for constants 0<c<C0<c<C0<c<C depending on KKK only; and (iii) σ(r)∈Ω(Γμ)\sigma(r)\in\Omega(\Gamma_\mu)σ(r)∈Ω(Γμ​) for all r>0r>0r>0.

Formalization targets

Goal: Theorem 5.2

Let Γ\GammaΓ be a gain matrix and μ\muμ a compatible vector of MAFs. Assume one of: (i) Γμ\Gamma_\muΓμ​ is linear with spectral radius less than one; (ii) Γ\GammaΓ is irreducible and Γμ≱id\Gamma_\mu\not\ge\mathrm{id}Γμ​≥id; (iii) μ=max⁡\mu=\maxμ=max and Γμ≱id\Gamma_\mu\not\ge\mathrm{id}Γμ​≥id; (iv) the entries of Γ\GammaΓ are bounded and Γμ≱id\Gamma_\mu\not\ge\mathrm{id}Γμ​≥id. Then

∃ σ an Ω-path withΓμ(σ(r))<σ(r)∀r>0.\exists\,\sigma\ \text{an }\Omega\text{-path with}\quad\Gamma_\mu(\sigma(r))<\sigma(r)\quad\forall r>0 .∃σ an Ω-path withΓμ​(σ(r))<σ(r)∀r>0.

In cases (i)–(iii) the entries are in K∞∪{0}\mathcal K_\infty\cup\{0\}K∞​∪{0}, and in case (iv) they are in (K∖K∞)∪{0}(\mathcal K\setminus\mathcal K_\infty)\cup\{0\}(K∖K∞​)∪{0}.

Milestones

The milestones follow §8 in the order the proof uses them.

  • Bounded gains: Lemmas 8.1–8.3, where iterates of a point of Ω\OmegaΩ tend to 000 and the polygon through them stays in Ω\OmegaΩ; and Proposition 8.4, which is case (iv).
  • Technical lemmas: Lemmas 8.5–8.7 on diagonal operators diag(ρ)\mathrm{diag}(\rho)diag(ρ) and the decay set Ψ={s:Γμ(s)≤s}\Psi=\{s:\Gamma_\mu(s)\le s\}Ψ={s:Γμ​(s)≤s}.
  • Topological statements: Proposition 8.8, path-connectedness of Ψ\PsiΨ; Proposition 8.10, where Ω\OmegaΩ meets every simplex Sr={s:∑si=r}S_r=\{s:\sum s_i=r\}Sr​={s:∑si​=r}; and Proposition 8.9, where Ψ∞=⋂kΓμk(Ψ)\Psi_\infty=\bigcap_k\Gamma_\mu^k(\Psi)Ψ∞​=⋂k​Γμk​(Ψ) is unbounded.
  • Irreducible case: the scaling step quoted from Rüffer's earlier work, and Theorem 8.11, which is case (ii).
  • Maximum case: Theorem 8.14, case (iii) in its cycle-condition form.
  • Linear case: the Perron step of §8.5, case (i).

Significance

An Ω\OmegaΩ-path turns a small gain condition into a Lyapunov function. If each ViV_iVi​ is an ISS Lyapunov function of Σi\Sigma_iΣi​, then V(x)=max⁡iσi−1(Vi(xi))V(x)=\max_i\sigma_i^{-1}(V_i(x_i))V(x)=maxi​σi−1​(Vi​(xi​)) is one for the network (Theorem 5.3). So Theorem 5.2 is the step that makes the small gain condition a certificate of network stability, and the corollaries of §5–§7 (sums, maxima, neural networks) all pass through it.

The results are published and proved on paper. None is machine-checked: no proof assistant library contains MAFs, gain operators, the sets Ω\OmegaΩ and Ψ\PsiΨ, or Ω\OmegaΩ-paths. The paper cites three ingredients without proof: Lemma 8.6, Proposition 8.9 and the scaling step of Theorem 8.11. It also gives Theorem 8.14 only as a sketch. Formalizing them closes those gaps in one place.

Difficulty

For bounded gains the problem is easy: far out, every point of a large ray lies in Ω\OmegaΩ, and the only work is the descent to the origin. With K∞\mathcal K_\inftyK∞​ entries, Γμ\Gamma_\muΓμ​ is unbounded and no ray need stay in Ω\OmegaΩ. Following the image of a single point fails, because forward iterates go to 000, not to infinity. The paper instead uses backward orbits inside Ψ∞\Psi_\inftyΨ∞​. That this set is nonempty and unbounded is a fixed-point argument of Knaster–Kuratowski–Mazurkiewicz type (Propositions 8.9 and 8.10), not a monotonicity argument. A second obstacle is strictness. A backward orbit is only ≤\le≤-increasing, while Definition 5.1 needs strictly increasing components with derivative bounds. This is where the diagonal scalings of Lemmas 8.6–8.7 enter.

Formalization scope

R+\mathbb R_+R+​ is ℝ≥0 and R+n\mathbb R^n_+R+n​ is Fin n → ℝ≥0, with subsystems indexed from 000. The strict order <<< is a named predicate (SLt), never Lean's < on functions, which means "≤\le≤ and ≠\ne=". The small gain condition is ∀ s ≠ 0, ¬ s ≤ T s. The inverses σi−1\sigma_i^{-1}σi−1​ are carried as explicit two-sided inverses, and derivatives are ordinary real derivatives at points r>0r>0r>0 where they exist. Compatibility (Remark 2.6) and γii≡0\gamma_{ii}\equiv0γii​≡0 are hypotheses of every statement about Γμ\Gamma_\muΓμ​. Irreducibility is the matrix-power form of p. 12, which Mathlib's Matrix.isIrreducible_iff_exists_pow_pos connects to Matrix.IsIrreducible. The spectral-radius hypothesis says that every complex eigenvalue has modulus <1<1<1. The 1-norm defines SrS_rSr​.

Hypotheses made explicit or corrected:

  • Theorem 5.2's header types Γ\GammaΓ in K∞∪{0}\mathcal K_\infty\cup\{0\}K∞​∪{0} while (iv) needs bounded entries, so each case carries its own entry class.
  • Case (iv) is stated as printed, without "no zero rows".
  • Proposition 8.10 prints "T(s)≱sT(s)\not\ge sT(s)≥s for all s∈R+ns\in\mathbb R^n_+s∈R+n​". No map satisfies this at s=0s=0s=0, so it is stated for s≠0s\ne0s=0.
  • Propositions 8.9 and 8.10 assume n≥1n\ge1n≥1.

A path with Γμ(σ(r))≤σ(r)\Gamma_\mu(\sigma(r))\le\sigma(r)Γμ​(σ(r))≤σ(r), or with Lean's < on functions, is not an Ω\OmegaΩ-path. Such a reading would make the goal follow from the origin alone, and it is ruled out.

Needed infrastructure: comparison-function calculus (inverses and compositions of K∞\mathcal K_\inftyK∞​ functions), piecewise linear interpolation in R+n\mathbb R^n_+R+n​ with Lipschitz inverse bounds, and a KKM-type argument on the simplex. The first two are reusable well beyond this mission. Contributions of any milestone, proofs of the cited ingredients (Lemma 8.6, Proposition 8.9, Proposition 8.10, the scaling step of Theorem 8.11) and a full proof of Theorem 8.14 are welcome.

Selected references

  • S. N. Dashkovskiy, B. S. Rüffer, F. R. Wirth, Small Gain Theorems for Large Scale Systems and Construction of ISS Lyapunov Functions, arXiv:0901.1842v2, 2009; SIAM J. Control Optim. 48(6), 2010. https://arxiv.org/abs/0901.1842 , https://doi.org/10.1137/090746483
  • S. Dashkovskiy, B. S. Rüffer, F. R. Wirth, An ISS small gain theorem for general networks, Math. Control Signals Systems 19 (2007), pp. 93–122.
  • B. S. Rüffer, Monotone dynamical systems, graphs, and stability of large-scale interconnected systems, PhD thesis, Universität Bremen, 2007. http://nbn-resolving.de/urn:nbn:de:gbv:46-diss000109058
  • B. S. Rüffer, Monotone inequalities, dynamical systems, and paths in the positive orthant of Euclidean n-space, Positivity, 2009/2010. https://doi.org/10.1007/s11117-009-0016-5
  • Z.-P. Jiang, I. M. Y. Mareels, Y. Wang, A Lyapunov formulation of the nonlinear small-gain theorem for interconnected ISS systems, Automatica 32 (1996), pp. 1211–1215.
18 thms1 active userReviewed
Operations ResearchProbabilityStochastic Systems·Captain: mikedeng1

On Minimizing the Ruin Probability by Investment and Reinsurance I: An Increasing Solution f of the HJB Equation Is Bounded, δ(u) = f(u)/f(∞), and the Feedback Strategy A*, b* Is OptimalResearch Paper

Motivation

An insurance company that collects premiums and pays random claims is ruined when its surplus becomes negative. The probability of ultimate ruin is the classical solvency criterion of risk theory, going back to Lundberg and Cramér. Real insurers have two levers on this probability: they can invest part of the surplus in a risky asset, and they can cede part of every claim to a reinsurer, at a price. The question of how to use both levers dynamically, as a function of the current surplus, so as to make the probability of ruin as small as possible, is a stochastic control problem for a jump-diffusion.

H. Schmidli, On minimizing the ruin probability by investment and reinsurance, Ann. Appl. Probab. 12 (2002), solves this problem for the Cramér–Lundberg model with proportional reinsurance and investment in a Black–Scholes asset. The main results are a verification theorem (Theorem 1, the subject of this mission) and an existence theorem for the associated integral equation (Theorem 2, the companion mission).

Timeline. Hipp and Plum (2000) treated investment in the Cramér–Lundberg model through the Hamilton–Jacobi–Bellman equation, assuming a bounded solution. Schmidli (2001) treated optimal proportional reinsurance alone. The 2002 paper combines investment and reinsurance, and proves the verification theorem without assuming the solution to be bounded.

Setting

Claims arrive at the jumps T1<T2<…T_1<T_2<\dotsT1​<T2​<… of a Poisson process NNN with rate λ>0\lambda>0λ>0; the claim sizes Y1,Y2,…Y_1,Y_2,\dotsY1​,Y2​,… are i.i.d. with continuous distribution function GGG, G(0)=0G(0)=0G(0)=0, independent of NNN. The insurer receives premiums at rate c>0c>0c>0. A risky asset Zt=exp⁡{σWt+(μ−12σ2)t}Z_t=\exp\{\sigma W_t+(\mu-\frac12\sigma^2)t\}Zt​=exp{σWt​+(μ−21​σ2)t}, with μ,σ>0\mu,\sigma>0μ,σ>0 and WWW a standard Brownian motion independent of the claims, is available.

A strategy is a pair of processes (At,bt)(A_t,b_t)(At​,bt​), predictable for the smallest right-continuous filtration generated by the claims and WWW: At∈RA_t\in\mathbb RAt​∈R is the amount invested in the risky asset (locally bounded), and bt∈[0,1]b_t\in[0,1]bt​∈[0,1] is the retention level, meaning the insurer pays btYb_tYbt​Y of a claim YYY occurring at time ttt, at a reinsurance premium rate c(bt)c(b_t)c(bt​). The surplus X=XAbX=X^{Ab}X=XAb starting from u≥0u\ge0u≥0 solves

dXt=(c−c(bt)+μAt) dt+σAt dWt−bt dSt,X0=u,dX_t=\big(c-c(b_t)+\mu A_t\big)\,dt+\sigma A_t\,dW_t-b_t\,dS_t,\qquad X_0=u,dXt​=(c−c(bt​)+μAt​)dt+σAt​dWt​−bt​dSt​,X0​=u,

where St=∑i≤NtYiS_t=\sum_{i\le N_t}Y_iSt​=∑i≤Nt​​Yi​ is the aggregate claims process. The ruin time is τ=inf⁡{t≥0:Xt<0}\tau=\inf\{t\ge0:X_t<0\}τ=inf{t≥0:Xt​<0}, the survival probability is δAb(u)=P[τ=∞]\delta^{Ab}(u)=\mathbb P[\tau=\infty]δAb(u)=P[τ=∞], and the value function is δ(u)=sup⁡A,bδAb(u)\delta(u)=\sup_{A,b}\delta^{Ab}(u)δ(u)=supA,b​δAb(u).

The premium function c(b)c(b)c(b) is decreasing and continuous on [0,1][0,1][0,1], c(1)=0c(1)=0c(1)=0, lim inf⁡b↑1c(b)/(1−b)>0\liminf_{b\uparrow1}c(b)/(1-b)>0liminfb↑1​c(b)/(1−b)>0, and there is b‾>0\underline b>0b​>0 with c(b)>cc(b)>cc(b)>c for b<b‾b<\underline bb<b​ and c(b)≤cc(b)\le cc(b)≤c for b≥b‾b\ge\underline bb≥b​ (full reinsurance is too expensive).

The Hamilton–Jacobi–Bellman equation for δ\deltaδ is, with the convention f(u)=0f(u)=0f(u)=0 for u<0u<0u<0,

sup⁡b∈[0,1]sup⁡A≥0[12σ2A2f′′(u)+(c−c(b)+μA)f′(u)+λ(E[f(u−bY)]−f(u))]=0.(1)\sup_{b\in[0,1]}\sup_{A\ge0}\Big[\tfrac12\sigma^2A^2f''(u)+\big(c-c(b)+\mu A\big)f'(u)+\lambda\big(\mathbb E[f(u-bY)]-f(u)\big)\Big]=0.\tag{1}b∈[0,1]sup​A≥0sup​[21​σ2A2f′′(u)+(c−c(b)+μA)f′(u)+λ(E[f(u−bY)]−f(u))]=0.(1)

For strictly concave fff the maximum over AAA is attained at A∗(u)=−μf′(u)/(σ2f′′(u))A^*(u)=-\mu f'(u)/(\sigma^2f''(u))A∗(u)=−μf′(u)/(σ2f′′(u)) (equation (2)); b∗(u)b^*(u)b∗(u) denotes a measurable maximiser in bbb.

Formalization targets

Goal: Theorem 1 (p. 896)

Let fff be strictly increasing and nonnegative on [0,∞)[0,\infty)[0,∞), continuous there, twice continuously differentiable on (0,∞)(0,\infty)(0,∞), zero on (−∞,0)(-\infty,0)(−∞,0), and a solution of (1) at every u>0u>0u>0. Then fff is bounded, f(∞)=lim⁡x→∞f(x)∈(0,∞)f(\infty)=\lim_{x\to\infty}f(x)\in(0,\infty)f(∞)=limx→∞​f(x)∈(0,∞),

δ(u)=f(u)f(∞)(u≥0),\delta(u)=\frac{f(u)}{f(\infty)}\qquad(u\ge0),δ(u)=f(∞)f(u)​(u≥0),

and every surplus process from u>0u>0u>0 following At=A∗(Xt−)A_t=A^*(X_{t-})At​=A∗(Xt−​), bt=b∗(Xt−)b_t=b^*(X_{t-})bt​=b∗(Xt−​) has survival probability δ(u)\delta(u)δ(u).

Milestones

The proof of Theorem 1 runs through: Lemma 3 (for small surplus the optimal retention is b∗=1b^*=1b∗=1); the identity E[f(Xτ∗∧t∗)]=f(u)\mathbb E[f(X^*_{\tau^*\wedge t})]=f(u)E[f(Xτ∗∧t∗​)]=f(u) along the feedback strategy; the inequality E[f(Xt∧τ)]≤f(u)\mathbb E[f(X_{t\wedge\tau})]\le f(u)E[f(Xt∧τ​)]≤f(u) for every strategy; Lemma 1 (almost surely, either ruin occurs or Xt→∞X_t\to\inftyXt​→∞); the existence of a strategy with P[τ=∞]>0\mathbb P[\tau=\infty]>0P[τ=∞]>0; the comparison δAb(u)≤f(u)/f(∞)\delta^{Ab}(u)\le f(u)/f(\infty)δAb(u)≤f(u)/f(∞) with equality for the feedback strategy; and the absence of ruin by creeping, P[τ∗<∞,Xτ∗∗=0]=0\mathbb P[\tau^*<\infty,X^*_{\tau^*}=0]=0P[τ∗<∞,Xτ∗∗​=0]=0.

Companion statements

The second sentence of Theorem 1 (that f(x)=1+∫0xg(z) dzf(x)=1+\int_0^xg(z)\,dzf(x)=1+∫0x​g(z)dz satisfies the hypotheses whenever ggg is a decreasing solution of the integral equation (5)) and its last sentence (at most one such solution of (1) with f(0)=1f(0)=1f(0)=1) are separate items.

Significance

Theorem 1 reduces a control problem over all predictable strategies to an analytic question: find one increasing smooth solution of (1). Combined with Theorem 2 (existence of a solution of (5)), it shows that the optimal survival probability is f(u)/f(∞)f(u)/f(\infty)f(u)/f(∞) and that the optimal strategy is a Markov feedback rule in the current surplus. It also yields that this solution is bounded, which Hipp and Plum (2000) had to assume, and closes the gap they left open on ruin by creeping (Remark (iii), p. 897). Numerical procedures for the optimal strategy (§5 of the paper) rest on this identification.

The theorem is proved in the paper; nothing in it is formalized. The mission produces a machine-checked verification theorem for a controlled jump-diffusion with an Itô integral against Brownian motion, a Poisson claim stream and a predictable control, together with a formal model of the Cramér–Lundberg process with investment and reinsurance that later results in risk theory can reuse.

Difficulty

The standard verification argument applies Itô's formula to f(Xt)f(X_t)f(Xt​) and uses (1) to show that f(Xt∧τ)f(X_{t\wedge\tau})f(Xt∧τ​) is a supermartingale, and a martingale under the feedback strategy. Here fff is not C2C^2C2 at 000 (f′′(0+)=−∞f''(0+)=-\inftyf′′(0+)=−∞), is discontinuous at 000 after the extension by 000, and is not assumed bounded, so the stochastic integral is only a local martingale and the passage t→∞t\to\inftyt→∞ needs both Lemma 1 and a separate argument that the feedback process does not creep through 000. Lemma 1 itself is only sketched in the paper ("We will just describe the argument"). The optimality part needs a surplus process following the feedback rule; its existence is not proved in the paper.

Formalization scope

The model is formalized on a measurable space with a probability measure: i.i.d. exponential interarrival times (realising the Poisson process), i.i.d. claim sizes with law ν\nuν, and a real Brownian motion (IsBrownianReal), the three families mutually independent. Independence of WWW from the claims is not printed in the paper but is used by (1) and its proof; it is a disclosed standing assumption. The filtration is Ft=⋂s>tσ(Sr,Wr:r≤s)\mathcal F_t=\bigcap_{s>t}\sigma(S_r,W_r:r\le s)Ft​=⋂s>t​σ(Sr​,Wr​:r≤s), not completed. The Itô integral is the published relation EthierKurtz.HasBrownianItoIntegral, and its paths are required to be continuous, so that the pathwise ruin event is well defined. The surplus equation holds for every ttt and every sample point.

Conventions: the premium c(b)c(b)c(b) is real valued (so c(0)<∞c(0)<\inftyc(0)<∞, which the paper allows to fail when E[Y]<∞\mathbb E[Y]<\inftyE[Y]<∞); the supremum in (1) is stated as a least upper bound equal to 000, so an unbounded family never counts as a solution; fff is C2C^2C2 on (0,∞)(0,\infty)(0,∞) and (1) is required for u>0u>0u>0; δ\deltaδ is a supremum over all admissible triples (strategy and surplus process); the controls of the feedback rule at non-positive pre-jump surplus are left free, and optimality is stated for u>0u>0u>0. Expectations of f(Xt∧τ)f(X_{t\wedge\tau})f(Xt∧τ​) are lower Lebesgue integrals of nonnegative functions, so no default value of a non-integrable expectation satisfies an inequality. The optimality clause is stated for every process following the feedback rule; it would be vacuous if no such process existed, and its existence is not part of the target.

Infrastructure needed: Itô's formula for a semimartingale with Brownian and compound-Poisson parts, the compensator of the compound Poisson random measure, localisation, and optional stopping. The model definitions, the HJB predicate and the comparison lemmas are reusable for other ruin-minimisation and dividend problems. Contributions to any milestone, and to general stochastic-calculus lemmas they need, are welcome.

Selected references

  • H. Schmidli, On minimizing the ruin probability by investment and reinsurance, Ann. Appl. Probab. 12(3) (2002), 890–907. https://doi.org/10.1214/aoap/1031863173
  • C. Hipp, M. Plum, Optimal investment for insurers, Insurance Math. Econom. 27 (2000), 215–228. https://doi.org/10.1016/S0167-6687(00)00049-4
  • H. Schmidli, Optimal proportional reinsurance policies in a dynamic setting, Scand. Actuar. J. (2001), 55–68. https://doi.org/10.1080/034612301750077338
  • P. Brémaud, Point Processes and Queues: Martingale Dynamics, Springer (1981). https://doi.org/10.1007/978-1-4684-9477-8
11 thms1 active userReviewed
AnalysisDynamical Systems·Captain: mikedeng1

Small Gain Theorems for Large Scale Systems and Construction of ISS Lyapunov Functions 1: Rescaling Along an Ω-Path Gives the Network ISS Lyapunov Function maxᵢ σᵢ⁻¹(Vᵢ(xᵢ))Research Paper

Motivation

Large engineered systems — power grids, chemical plants, logistics and communication networks — are usually modelled as many interacting subsystems. A stability certificate for the whole network is rarely available directly, but each subsystem can often be analysed on its own, treating the states of its neighbours as disturbances. Input-to-state stability (ISS), introduced by Sontag (Sontag 1989), is the robustness notion that makes this decomposition work: it bounds the state by a decaying function of the initial condition plus a gain of the input. A small-gain theorem states that if the gains by which the subsystems influence each other are "small" in a suitable sense, the interconnection is again ISS.

Timeline. Jiang, Teel and Praly proved a nonlinear small-gain theorem for two ISS systems in trajectory form (1994), and Jiang, Mareels and Wang gave its Lyapunov version for two subsystems (1996). Dashkovskiy, Rüffer and Wirth extended the trajectory version to nnn subsystems with the condition Γ≱id\Gamma\not\ge\mathrm{id}Γ≥id on a nonlinear gain operator (2007). The paper of this mission (arXiv:0901.1842, SIAM J. Control Optim. 48(6), 2010) gives the Lyapunov version for nnn subsystems with general monotone aggregation of the gains, and constructs the network's ISS Lyapunov function explicitly.

Setting

The network consists of n≥1n\ge 1n≥1 subsystems

Σi: x˙i=fi(x1,…,xn,u),xi∈RNi, u∈RM,\Sigma_i:\ \dot x_i=f_i(x_1,\dots,x_n,u),\qquad x_i\in\mathbb R^{N_i},\ u\in\mathbb R^M,Σi​: x˙i​=fi​(x1​,…,xn​,u),xi​∈RNi​, u∈RM,

with overall state x=(x1T,…,xnT)T∈RNx=(x_1^T,\dots,x_n^T)^T\in\mathbb R^Nx=(x1T​,…,xnT​)T∈RN and dynamics x˙=f(x,u)\dot x=f(x,u)x˙=f(x,u), f=(f1,…,fn)f=(f_1,\dots,f_n)f=(f1​,…,fn​). ∥⋅∥\|\cdot\|∥⋅∥ is the Euclidean norm.

A function γ:R+→R+\gamma:\mathbb R_+\to\mathbb R_+γ:R+​→R+​ is of class K\mathcal KK if it is continuous, strictly increasing and γ(0)=0\gamma(0)=0γ(0)=0, and of class K∞\mathcal K_\inftyK∞​ if moreover unbounded. A positive definite function α:R+→R+\alpha:\mathbb R_+\to\mathbb R_+α:R+​→R+​ is continuous with α(r)=0\alpha(r)=0α(r)=0 only at r=0r=0r=0. A Lyapunov function candidate V:Rd→R+V:\mathbb R^d\to\mathbb R_+V:Rd→R+​ is continuous, satisfies ψ1(∥x∥)≤V(x)≤ψ2(∥x∥)\psi_1(\|x\|)\le V(x)\le\psi_2(\|x\|)ψ1​(∥x∥)≤V(x)≤ψ2​(∥x∥) for some ψ1,ψ2∈K∞\psi_1,\psi_2\in\mathcal K_\inftyψ1​,ψ2​∈K∞​, and is locally Lipschitz away from the origin.

A candidate VVV is an ISS Lyapunov function for x˙=f(x,u)\dot x=f(x,u)x˙=f(x,u) if there are γ∈K\gamma\in\mathcal Kγ∈K and a positive definite α\alphaα with

V(x)≥γ(∥u∥) ⟹ ∇V(x)f(x,u)≤−α(∥x∥)V(x)\ge\gamma(\|u\|)\ \Longrightarrow\ \nabla V(x)f(x,u)\le-\alpha(\|x\|)V(x)≥γ(∥u∥) ⟹ ∇V(x)f(x,u)≤−α(∥x∥)

at every point of differentiability of VVV.

On R+n\mathbb R^n_+R+n​, v<wv<wv<w means vi<wiv_i<w_ivi​<wi​ for every iii. A monotone aggregation function (MAF) μ:R+m→R+\mu:\mathbb R^m_+\to\mathbb R_+μ:R+m​→R+​ is continuous, positive off the origin, strictly increasing for this strict order, and unbounded as ∥s∥→∞\|s\|\to\infty∥s∥→∞. Subsystem Σi\Sigma_iΣi​ has an ISS Lyapunov function ViV_iVi​ with gains γij∈K∞∪{0}\gamma_{ij}\in\mathcal K_\infty\cup\{0\}γij​∈K∞​∪{0}, γiu∈K∪{0}\gamma_{iu}\in\mathcal K\cup\{0\}γiu​∈K∪{0} and MAF μi\mu_iμi​ on R+n+1\mathbb R^{n+1}_+R+n+1​ if, wherever ViV_iVi​ is differentiable,

Vi(xi)≥μi(γi1(V1(x1)),…,γin(Vn(xn)),γiu(∥u∥)) ⟹ ∇Vi(xi)fi(x,u)≤−αi(∥xi∥).V_i(x_i)\ge\mu_i\big(\gamma_{i1}(V_1(x_1)),\dots,\gamma_{in}(V_n(x_n)),\gamma_{iu}(\|u\|)\big)\ \Longrightarrow\ \nabla V_i(x_i)f_i(x,u)\le-\alpha_i(\|x_i\|).Vi​(xi​)≥μi​(γi1​(V1​(x1​)),…,γin​(Vn​(xn​)),γiu​(∥u∥)) ⟹ ∇Vi​(xi​)fi​(x,u)≤−αi​(∥xi​∥).

The diagonal gains vanish, γii≡0\gamma_{ii}\equiv0γii​≡0. The gain operators are

Γ‾μ(s,r)i=μi(γi1(s1),…,γin(sn),γiu(r)),Γμ(s)=Γ‾μ(s,0).\overline\Gamma_\mu(s,r)_i=\mu_i\big(\gamma_{i1}(s_1),\dots,\gamma_{in}(s_n),\gamma_{iu}(r)\big),\qquad\Gamma_\mu(s)=\overline\Gamma_\mu(s,0).Γμ​(s,r)i​=μi​(γi1​(s1​),…,γin​(sn​),γiu​(r)),Γμ​(s)=Γμ​(s,0).

An Ω-path with respect to Γμ\Gamma_\muΓμ​ is a continuous σ:R+→R+n\sigma:\mathbb R_+\to\mathbb R^n_+σ:R+​→R+n​ with every σi∈K∞\sigma_i\in\mathcal K_\inftyσi​∈K∞​, such that each σi−1\sigma_i^{-1}σi−1​ is locally Lipschitz on (0,∞)(0,\infty)(0,∞), its derivative lies in [c,C][c,C][c,C], 0<c<C0<c<C0<c<C, at its points of differentiability in any compact K⊂(0,∞)K\subset(0,\infty)K⊂(0,∞) (constants depending on KKK only), and Γμ(σ(r))<σ(r)\Gamma_\mu(\sigma(r))<\sigma(r)Γμ​(σ(r))<σ(r) for all r>0r>0r>0.

Formalization targets

Goal: Theorem 5.3

Assume each ViV_iVi​ is an ISS Lyapunov function for Σi\Sigma_iΣi​ with gains Γ\GammaΓ, γiu\gamma_{iu}γiu​ and MAFs μi\mu_iμi​; σ\sigmaσ is an Ω-path with respect to Γμ\Gamma_\muΓμ​; φ∈K∞\varphi\in\mathcal K_\inftyφ∈K∞​ satisfies

Γ‾μ(σ(r),φ(r))<σ(r)∀r>0;\overline\Gamma_\mu(\sigma(r),\varphi(r))<\sigma(r)\qquad\forall r>0;Γμ​(σ(r),φ(r))<σ(r)∀r>0;

and each fif_ifi​ is continuous. Then

V(x)=max⁡i=1,…,nσi−1(Vi(xi))V(x)=\max_{i=1,\dots,n}\sigma_i^{-1}(V_i(x_i))V(x)=i=1,…,nmax​σi−1​(Vi​(xi​))

is an ISS Lyapunov function for the network, with gain φ−1\varphi^{-1}φ−1:

V(x)≥φ−1(∥u∥) ⟹ ∇V(x)f(x,u)≤−α(∥x∥)V(x)\ge\varphi^{-1}(\|u\|)\ \Longrightarrow\ \nabla V(x)f(x,u)\le-\alpha(\|x\|)V(x)≥φ−1(∥u∥) ⟹ ∇V(x)f(x,u)≤−α(∥x∥)

for a positive definite α\alphaα. The rate α\alphaα is existential and is not fixed by the statement.

Milestones

The milestones follow the proof on pp. 14–15:

  1. VVV is a Lyapunov function candidate.
  2. The maximum rule (5.7) for Clarke generalized gradients.
  3. The chain of inequalities showing that the subsystem trigger holds strictly at every active index.
  4. The Clarke-gradient form (5.8) of the subsystem decrease.
  5. The chain rule for σi−1∘Vi\sigma_i^{-1}\circ V_iσi−1​∘Vi​, with multipliers bounded away from zero.
  6. Positivity of the rate: active blocks on a sphere stay away from the origin.
  7. The uniform decrease (5.9).

Significance

The result. Theorem 5.3 converts a small-gain condition into an explicit Lyapunov function for the network. Combined with the converse ISS Lyapunov theorem, it gives ISS of the interconnection, and with zero input it gives global asymptotic stability. It covers additive, maximum-type and multiplicative aggregation of gains, and it handles nnn subsystems without iterating a two-system result. A second main result of the paper (Theorem 5.2, a separate mission) supplies the Ω-path from the condition Γμ≱id\Gamma_\mu\not\ge\mathrm{id}Γμ​≥id. Together they reduce the stability of a network to a property of its gain operator.

Formalizing it. The result is proved on paper. As far as we know, no ISS notion, small-gain theorem or monotone aggregation framework has been machine-checked in Lean. This mission produces a Lean statement of the construction, and of its nonsmooth-analysis steps, against which a complete proof can be checked. It also corrects a misprint in the theorem's implication (see Formalization scope).

Difficulty

The function VVV is a maximum of rescaled functions and is not differentiable on the set where two indices are active, even when every ViV_iVi​ is smooth. The subsystem hypotheses only control derivatives at points where the corresponding ViV_iVi​ is differentiable. At a point where VVV is differentiable, the active ViV_iVi​ may not be. So the obvious argument — differentiate the active branch and apply (2.7) — fails, and the decrease must be transported through generalized gradients. A second obstacle is uniformity: the rate α\alphaα must depend on ∥x∥\|x\|∥x∥ alone. This requires the lower derivative bound in the Ω-path definition and a compactness argument on spheres. Dropping the lower bound breaks the theorem, because σi−1\sigma_i^{-1}σi−1​ could flatten and remove the decrease.

Formalization scope

Blocks are EuclideanSpace ℝ (Fin (N i)). The network state is the L2L^2L2 product PiLp 2 (the Euclidean norm, not the sup norm), and the input is EuclideanSpace ℝ (Fin M). Subsystems are indexed by Fin n (0-based) with n≥1n\ge1n≥1. Gains, MAFs, Ω-paths and rates live on ℝ≥0. Lyapunov functions are real-valued with an explicit nonnegativity clause. The strict order on R+n\mathbb R^n_+R+n​ is a named predicate, not Lean's Pi <. The inverses σi−1\sigma_i^{-1}σi−1​ and φ−1\varphi^{-1}φ−1 are explicit arguments with both compositions the identity. Every derivative is guarded by a differentiability hypothesis. Clarke's generalized gradient (2.5) is the published ClarkeGradients.Shared.generalizedGradient on the blocks, and the same definition restated on a general Hilbert space for RN\mathbb R^NRN and R\mathbb RR.

Implicit hypotheses made explicit:

  • Continuity of every fif_ifi​. It is the regularity behind the equivalence of the derivative form (2.4) and the Clarke form (2.6) of the decrease condition, which the proof uses.
  • γii≡0\gamma_{ii}\equiv0γii​≡0. The paper's convention.
  • Compatibility of Γ\GammaΓ and μ\muμ. Remark 2.6, the standing assumption from p. 8 on; it imposes nothing on a zero row of Γ\GammaΓ.

Corrected statement. The printed implication (5.5) has the trigger V(x)≥max⁡iφ−1(γiu(∥u∥))V(x)\ge\max_i\varphi^{-1}(\gamma_{iu}(\|u\|))V(x)≥maxi​φ−1(γiu​(∥u∥)). With Γ‾μ\overline\Gamma_\muΓμ​ as defined in (2.9), and as the paper's corollaries use it, that implication is false. A counterexample is one subsystem x˙=−x+u/6\dot x=-x+u/6x˙=−x+u/6 with V1=∣x∣V_1=|x|V1​=∣x∣, γ1u(s)=s/10\gamma_{1u}(s)=s/10γ1u​(s)=s/10, μ1(a,b)=a+2b\mu_1(a,b)=a+2bμ1​(a,b)=a+2b and σ=φ=id\sigma=\varphi=\mathrm{id}σ=φ=id, at x=1x=1x=1, u=10u=10u=10. The proof's chain of inequalities writes φ(r)\varphi(r)φ(r) where (2.9) gives γiu(φ(r))\gamma_{iu}(\varphi(r))γiu​(φ(r)). The goal keeps (2.9) and uses the trigger V(x)≥φ−1(∥u∥)V(x)\ge\varphi^{-1}(\|u\|)V(x)≥φ−1(∥u∥), under which the proof closes.

The hypotheses are jointly satisfiable. An example is n=2n=2n=2 with Vi(xi)=∥xi∥V_i(x_i)=\|x_i\|Vi​(xi​)=∥xi​∥, μi=max⁡\mu_i=\maxμi​=max, γ12=γ21=s/4\gamma_{12}=\gamma_{21}=s/4γ12​=γ21​=s/4, γiu=s/2\gamma_{iu}=s/2γiu​=s/2, σ(r)=(r,r)\sigma(r)=(r,r)σ(r)=(r,r), φ=id\varphi=\mathrm{id}φ=id and dynamics fi=−xif_i=-x_ifi​=−xi​. The conclusion is a decrease inequality at the points of differentiability of VVV for an existential positive definite rate, so it cannot be met by a vacuous choice: α\alphaα must be positive away from zero. Trajectory-level ISS (Theorem 2.3, cited by the paper) and the corollaries that conclude ISS are out of scope.

Reusable infrastructure: comparison classes, monotone aggregation functions, gain operators and Ω-paths (shared with the Ω-path mission), and the maximum and chain rules for Clarke gradients. Proofs of the nonsmooth calculus milestones are welcome independently of the goal.

Selected references

  • S. N. Dashkovskiy, B. S. Rüffer, F. R. Wirth, Small Gain Theorems for Large Scale Systems and Construction of ISS Lyapunov Functions, SIAM J. Control Optim. 48(6), 2010; preprint arXiv:0901.1842v2. https://arxiv.org/abs/0901.1842
  • S. Dashkovskiy, B. S. Rüffer, F. R. Wirth, An ISS small gain theorem for general networks, Math. Control Signals Systems 19, 2007. https://doi.org/10.1007/s00498-007-0014-8
  • Z.-P. Jiang, A. R. Teel, L. Praly, Small-gain theorem for ISS systems and applications, Math. Control Signals Systems 7, 1994. https://doi.org/10.1007/BF01211469
  • Z.-P. Jiang, I. M. Y. Mareels, Y. Wang, A Lyapunov formulation of the nonlinear small-gain theorem for interconnected ISS systems, Automatica 32(8), 1996. https://doi.org/10.1016/0005-1098(96)00051-9
  • E. D. Sontag, Smooth stabilization implies coprime factorization, IEEE Trans. Automat. Control 34(4), 1989. https://doi.org/10.1109/9.28018
  • F. H. Clarke, Optimization and Nonsmooth Analysis, Wiley, 1983. https://doi.org/10.1137/1.9781611971309
11 thms1 active userReviewed
Operations ResearchStochastic Systems·Captain: mikedeng1

Distributed Scheduling Based on Due Dates and Buffer Priorities 2: Under Last Buffer First Serve the Number of Parts Stays Bounded, Eventually by (λc(ε)+γ)/(1−ρ−λε)Research Paper

Motivation

Semiconductor wafer fabrication lines are reentrant: a wafer returns to the same photolithography or etching station many times along its route, so the parts competing for a machine are at different stages of completion. In such a line, a local dispatching rule decides at every machine which waiting part goes next, and the question is whether a given rule keeps the line stable whenever there is enough capacity. Lu and Kumar (IEEE TAC 36(12), 1991) showed that the answer depends on the rule. Some natural buffer priority rules make the line unstable even when every machine has spare capacity (their Example 1; see also Kumar and Seidman 1990). Two rules are proved stable for every route: first buffer first serve (FBFS) and last buffer first serve (LBFS).

LBFS serves at each machine the waiting part that is closest to completion. It is the "pull" rule of the two, and the one closer to practice: it minimizes work in progress in many settings and coincides with earliest due date when parts arrive in due-date order. This mission formalizes the paper's LBFS results, §V. The sibling mission (FBFS, §IV) treats the other policy on the same model.

Timeline:

  • 1990: Kumar and Seidman exhibit instability of clearing policies on a reentrant line with spare capacity.
  • 1991: Lu and Kumar prove stability of FBFS (Theorem 1) and of LBFS (Theorems 2–3) under deterministic bursty arrivals. They also prove it for least slack and earliest due date (Theorem 4, Corollary 1), and show that a buffer priority policy can be unstable (Example 1).
  • 1996: Dai and Weiss prove stability of the fluid models of FBFS and LBFS on reentrant lines (Math. Oper. Res. 21(1)), which by Dai's fluid limit theorem gives positive Harris recurrence for the stochastic versions.

Setting

A nonacyclic flow line has SSS service centers; center σ\sigmaσ has mσ≥1m_\sigma \ge 1mσ​≥1 identical machines, each processing one part at a time. Every part follows the same route through buffers b1,…,blb_1, \dots, b_lb1​,…,bl​: in buffer bib_ibi​ it waits at center σi\sigma_iσi​, then needs τi>0\tau_i > 0τi​>0 units of uninterrupted processing on one machine of that center. A center may serve several buffers.

Parts are released into b1b_1b1​; part π\piπ is released at time α(π)\alpha(\pi)α(π) and exits at time e(π)e(\pi)e(π), when its service at blb_lbl​ ends. Finitely many parts may be present at time 000, in any buffer, some already in service. With u(t)u(t)u(t) the number of releases in [0,t][0,t][0,t], the arrivals are bursty with rate λ\lambdaλ and burst γ\gammaγ if

u(t)−u(s)≤λ(t−s)+γ(0≤s≤t).(1)u(t)-u(s)\le \lambda(t-s)+\gamma\qquad(0\le s\le t).\tag{1}u(t)−u(s)≤λ(t−s)+γ(0≤s≤t).(1)

The work per machine that one part brings to center σ\sigmaσ is wσ=∑i:σi=στi/mσw_\sigma=\sum_{i:\sigma_i=\sigma}\tau_i/m_\sigmawσ​=∑i:σi​=σ​τi​/mσ​, and the load is ρ=λw‾\rho=\lambda\overline wρ=λw with w‾=max⁡σwσ\overline w=\max_\sigma w_\sigmaw=maxσ​wσ​. The capacity condition is ρ<1\rho<1ρ<1 (3). Also τ‾=max⁡jτj\overline\tau=\max_j\tau_jτ=maxj​τj​, and w(k)w^{(k)}w(k) is the maximum per-machine work brought to a center by a part in bkb_kbk​, counting only buffers bk,…,blb_k,\dots,b_lbk​,…,bl​ (6).

Scheduling is nonidling (a machine idles only if every buffer of its center is empty) and nonpreemptive, and a machine takes the part at the head of a buffer. Under LBFS, a machine takes a part from bib_ibi​ only if every buffer bjb_jbj​ of its center with j>ij>ij>i is empty. The number of parts in the system at time ttt is x(t)x(t)x(t).

Formalization targets

Goal: Theorem 3, Stability of LBFS

Under (1) and (3), for every ε>0\varepsilon>0ε>0 with 1−ρ−λε>01-\rho-\lambda\varepsilon>01−ρ−λε>0,

x(t)≤max⁡{(1+ρ+λε)x(0)+λc(ε)+γ, 2(λc(ε)+γ)1−ρ−λε}(t≥0),x(t)\le\max\Big\{(1+\rho+\lambda\varepsilon)x(0)+\lambda c(\varepsilon)+\gamma,\ \frac{2(\lambda c(\varepsilon)+\gamma)}{1-\rho-\lambda\varepsilon}\Big\}\quad(t\ge0),x(t)≤max{(1+ρ+λε)x(0)+λc(ε)+γ, 1−ρ−λε2(λc(ε)+γ)​}(t≥0), lim sup⁡t→∞x(t)≤λc(ε)+γ1−ρ−λε.\limsup_{t\to\infty}x(t)\le\frac{\lambda c(\varepsilon)+\gamma}{1-\rho-\lambda\varepsilon}.t→∞limsup​x(t)≤1−ρ−λελc(ε)+γ​.

Here c(ε)c(\varepsilon)c(ε) is the explicit constant of the delay estimate. The paper prints the second bound as λc(ε)+γ/(1−ρ−λε)\lambda c(\varepsilon)+\gamma/(1-\rho-\lambda\varepsilon)λc(ε)+γ/(1−ρ−λε); its proof establishes the form above, which is the one posed.

Milestones

The delay estimate behind the goal, in the order of its proof:

  • the base case (8) for the last buffer;
  • the one-step comparison (11) with the part just ahead;
  • the induction claim (10) for every truncated line B(k)={bk,…,bl}B^{(k)}=\{b_k,\dots,b_l\}B(k)={bk​,…,bl​}: a part in B(k)B^{(k)}B(k) with xxx parts ahead exits within c(k)(ε)+(w(k)+ε)xc^{(k)}(\varepsilon)+(w^{(k)}+\varepsilon)xc(k)(ε)+(w(k)+ε)x;
  • Theorem 2, the contractive estimate
e(π)−α(π)≤c(ε)+(w‾+ε)x(ε>0),e(\pi)-\alpha(\pi)\le c(\varepsilon)+(\overline w+\varepsilon)x\qquad(\varepsilon>0),e(π)−α(π)≤c(ε)+(w+ε)x(ε>0),

for a part that finds xxx parts in the system on release.

Then the Pipeline Property e(π)−α(π)≤w‾x+o(x)e(\pi)-\alpha(\pi)\le\overline wx+o(x)e(π)−α(π)≤wx+o(x), order preservation (parts exit in the order they enter), and the recursion (14) for the number in system along successive exit times. A further item states that LBFS is stable in the sense of §II: every delay e(π)−α(π)e(\pi)-\alpha(\pi)e(π)−α(π) is bounded.

Significance

Theorem 2 says the delay of a part under LBFS grows with the number of parts ahead at rate w‾+ε\overline w+\varepsilonw+ε: the line behaves like a pipeline whose speed is set by its bottleneck center, although parts revisit centers. Theorem 3 converts this into a bound on work in progress that holds for all time, with an asymptotic bound that does not depend on the initial state. Together they give stability of LBFS for every route and every burst size, with explicit constants. This is a deterministic, sample-path result: no distributional assumption on arrivals is needed. Its constants are explicit, so they can be evaluated for a given line.

The results are proved in the paper; none is formalized. The mission produces a machine-checked model of a multi-server reentrant line with nonidling, nonpreemptive, head-of-buffer buffer-priority dispatch, together with Theorems 2 and 3 on it. The model carries over to the other policies of the paper (FBFS, least slack, earliest due date).

Difficulty

The obvious argument fails because under LBFS parts released later do interfere with a part π\piπ: they occupy machines nonpreemptively at the low-priority buffers π\piπ must still pass. So the delay of π\piπ cannot be bounded by the work ahead of it alone. The paper's estimate is an induction from the end of the line backwards over the truncated systems B(k)B^{(k)}B(k), nested with an induction on the position of π\piπ in its buffer. The constant c(k)(ε)c^{(k)}(\varepsilon)c(k)(ε) is recomputed at every level from c(k+1)(ε/2)c^{(k+1)}(\varepsilon/2)c(k+1)(ε/2), so ε\varepsilonε is halved at each level. A formal proof must also handle what the paper treats informally:

  • the order in which parts leave buffers when several arrive at the same instant;
  • parts already in service at time 000;
  • an empty system between busy periods.

Formalization scope

All declarations are in the namespace ReentrantScheduling.LBFS.

  • Buffers are 0-based in Lean: the paper's bib_ibi​ is index i−1i-1i−1.
  • A run lists its parts in a fixed line order: parts present at time 000 first (deeper buffers first, then earlier service start), then released parts by release time. This order breaks ties between simultaneous arrivals, and "parts ahead of π\piπ" means the parts in the system that precede π\piπ in it.
  • Service start times are in WithTop ℝ, with ⊤ meaning never, so a run need not serve every part, and the theorems assert that parts are served. A formalization with real-valued start times would assume every part is served, which is half of what stability asserts.
  • All counts (capacity, nonidling, (1), x(t)x(t)x(t)) are Set.encard, never Set.ncard, which would count an infinite set as 000 and make the capacity rule vacuous.
  • The constants c(k)(ε)c^{(k)}(\varepsilon)c(k)(ε) of (8)–(9) are explicit definitions that depend only on the line and ε\varepsilonε. A constant chosen after the run would let it depend on the arrivals or the initial state.
  • Theorem 3's bounds are inequalities in [0,∞][0,\infty][0,∞]; the lim sup is taken there, so it cannot be a junk value of an unbounded real function.
  • The standing assumptions of §II are hypotheses or part of the run predicate: nonidling, nonpreemptive, head-of-buffer service, mσ≥1m_\sigma\ge1mσ​≥1, l≥1l\ge1l≥1. The positivity τi>0\tau_i>0τi​>0 is pinned: the paper leaves it implicit.
  • Every theorem quantifies over all admissible LBFS runs. The empty run and nontrivial runs satisfy the hypotheses.

The model is reusable for other dispatch rules on reentrant lines. Proofs of any milestone are welcome, as are lemmas on admissible runs (FIFO within buffers, finiteness of x(t)x(t)x(t)).

Not posed here:

  • Example 1, which is already on the platform as QueueingStability.LuKumar.theorem31;
  • Theorem 4 and Corollary 1 (least slack, earliest due date), whose proof modifications the paper leaves to the reader;
  • Theorem 5 (several flow lines), whose proof is a sketch.

Selected references

  • S. H. Lu and P. R. Kumar, Distributed Scheduling Based on Due Dates and Buffer Priorities, IEEE Transactions on Automatic Control 36(12), 1406–1416, 1991. https://doi.org/10.1109/9.106156
  • P. R. Kumar and T. I. Seidman, Dynamic Instabilities and Stabilization Methods in Distributed Real-Time Scheduling of Manufacturing Systems, IEEE Transactions on Automatic Control 35(3), 289–298, 1990. https://doi.org/10.1109/9.50339
  • J. G. Dai and G. Weiss, Stability and Instability of Fluid Models for Reentrant Lines, Mathematics of Operations Research 21(1), 115–134, 1996. https://doi.org/10.1287/moor.21.1.115
10 thms1 active userReviewed
Operations ResearchStochastic Systems·Captain: mikedeng1

Distributed Scheduling Based on Due Dates and Buffer Priorities 1: First Buffer First Serve Is Stable on Every Nonacyclic Flow Line Whose Bursty Arrival Rate Is Below CapacityResearch Paper

Motivation

Semiconductor wafer fabs are the standard example of a reentrant manufacturing line: a wafer visits the same lithography or etching station many times along a route of hundreds of steps, so each station holds parts at many different stages of completion and must decide, every time a machine frees up, which of them to serve next. Kumar (Re-entrant lines, Queueing Systems 13, 1993) singled these lines out as a third class of manufacturing systems besides flow shops and job shops, and stability of their scheduling policies became a central question in the analysis of queueing networks.

The question is sharp because the obvious conjecture is false. Kumar and Seidman (IEEE TAC 35(3), 1990) gave a deterministic network in which a distributed policy is unstable although every machine has spare capacity, and Lu and Kumar (IEEE TAC 36(12), 1991, Example 1, p. 1409) gave a two-station reentrant line in which a buffer priority policy is unstable at load below one. Their paper then identifies policies that are always stable: first buffer first serve (FBFS, Theorem 1), last buffer first serve (Theorem 3) and the due-date policies least slack and earliest due date (Theorem 4). This mission formalizes Theorem 1.

Timeline:

  • 1990: Kumar and Seidman exhibit instability of clear-a-fraction policies under load below one.
  • 1991: Lu and Kumar prove FBFS and LBFS stable on every nonacyclic flow line with deterministic bursty arrivals, and exhibit an unstable buffer priority policy.
  • 1995–1996: Dai (Ann. Appl. Probab. 5(1), 1995) reduces positive Harris recurrence of multiclass networks to stability of fluid models; Dai and Weiss (Math. Oper. Res. 21(1), 1996) prove the FBFS and LBFS fluid models of reentrant lines stable.

Setting

A nonacyclic flow line has SSS service centers; center σ\sigmaσ has mσ≥1m_\sigma\ge1mσ​≥1 identical machines in parallel, each working on one part at a time. Every part follows the same route: it visits buffers b1,b2,…,blb_1,b_2,\dots,b_lb1​,b2​,…,bl​ in order, buffer bib_ibi​ sits at center σi\sigma_iσi​, and a part in bib_ibi​ needs τi>0\tau_i>0τi​>0 time units of uninterrupted processing on one machine of σi\sigma_iσi​. The same center may appear many times along the route.

Parts enter at b1b_1b1​; part π\piπ is released at time α(π)\alpha(\pi)α(π). The releases are deterministic but may be bursty: with u(t)u(t)u(t) the number of parts released in [0,t][0,t][0,t],

u(t)−u(s)≤λ(t−s)+γfor all 0≤s≤t,(1)u(t)-u(s)\le\lambda(t-s)+\gamma\qquad\text{for all }0\le s\le t,\tag{1}u(t)−u(s)≤λ(t−s)+γfor all 0≤s≤t,(1)

for constants λ,γ≥0\lambda,\gamma\ge0λ,γ≥0. Each part brings wσ=∑i:σi=στi/mσw_\sigma=\sum_{i:\sigma_i=\sigma}\tau_i/m_\sigmawσ​=∑i:σi​=σ​τi​/mσ​ units of work per machine to center σ\sigmaσ, and the load is ρ=max⁡σλwσ\rho=\max_\sigma\lambda w_\sigmaρ=maxσ​λwσ​. The arrival rate is within capacity when ρ<1\rho<1ρ<1 (3). At time 000 the line may hold finitely many initial parts in arbitrary buffers, some of them already in service.

A schedule is nonidling (a machine idles only if every buffer of its center is empty), nonpreemptive (a started service runs to completion) and serves the part at the head of the chosen buffer. Under FBFS a free machine at center σ\sigmaσ takes a part from bib_ibi​ only if every buffer bjb_jbj​ of σ\sigmaσ with j<ij<ij<i is empty. With e(π)e(\pi)e(π) the time part π\piπ exits after its service at blb_lbl​, the schedule is stable if there is Γ≥0\Gamma\ge0Γ≥0 with

e(π)−α(π)≤Γfor all parts π.(4)e(\pi)-\alpha(\pi)\le\Gamma\qquad\text{for all parts }\pi.\tag{4}e(π)−α(π)≤Γfor all parts π.(4)

Γ\GammaΓ may depend on the initial state as well as on λ\lambdaλ.

Formalization targets

Goal: Theorem 1 (p. 1409)

For every line, every λ,γ≥0\lambda,\gamma\ge0λ,γ≥0 with ρ<1\rho<1ρ<1, and every admissible FBFS run whose releases satisfy (1),

∃ Γ≥0  ∀π:e(π)≤α(π)+Γ.\exists\,\Gamma\ge0\ \ \forall\pi:\quad e(\pi)\le\alpha(\pi)+\Gamma .∃Γ≥0  ∀π:e(π)≤α(π)+Γ.

The statement fixes no constant: Γ\GammaΓ is chosen after the run.

Milestones (proof of Theorem 1, pp. 1409–1410)

An interval [T1,T2][T_1,T_2][T1​,T2​] is an iii-busy period if at every instant some part waits in a buffer bjb_jbj​, j≤ij\le ij≤i, at center σi\sigma_iσi​. With τˉ=max⁡jτj\bar\tau=\max_j\tau_jτˉ=maxj​τj​, Γˉi=∑j<i(Γ(j)+τj)\bar\Gamma_i=\sum_{j<i}(\Gamma^{(j)}+\tau_j)Γˉi​=∑j<i​(Γ(j)+τj​) and

Γ(i)=[2τˉ+∑σj=σi, j≤iλτjΓˉi+γτjmσi][1−∑σj=σi, j≤iλτjmσi]−1,\Gamma^{(i)}=\Big[2\bar\tau+\sum_{\sigma_j=\sigma_i,\,j\le i}\frac{\lambda\tau_j\bar\Gamma_i+\gamma\tau_j}{m_{\sigma_i}}\Big]\Big[1-\sum_{\sigma_j=\sigma_i,\,j\le i}\frac{\lambda\tau_j}{m_{\sigma_i}}\Big]^{-1},Γ(i)=[2τˉ+σj​=σi​,j≤i∑​mσi​​λτj​Γˉi​+γτj​​][1−σj​=σi​,j≤i∑​mσi​​λτj​​]−1,
  1. a 1-busy period commencing at T1>0T_1>0T1​>0 has T2−T1≤Γ(1)T_2-T_1\le\Gamma^{(1)}T2​−T1​≤Γ(1);
  2. there are finite times T(i)T^{(i)}T(i) after which every iii-busy period has length at most Γ(i)\Gamma^{(i)}Γ(i);
  3. a part arriving at bjb_jbj​ after T(j)T^{(j)}T(j) leaves bjb_jbj​ within Γ(j)+τj\Gamma^{(j)}+\tau_jΓ(j)+τj​;
  4. a part released after T(l)T^{(l)}T(l) has delay at most ∑j=1l(Γ(j)+τj)\sum_{j=1}^l(\Gamma^{(j)}+\tau_j)∑j=1l​(Γ(j)+τj​).

Milestone 4 has an explicit bound independent of the initial state; the goal additionally covers the finitely many parts present or released during the transient.

Significance

Theorem 1 says that the simplest "push" discipline, always serving the earliest stage first, keeps every part's delay bounded on every reentrant line whose stations have spare capacity, for every initial state and every bursty but rate-limited release pattern. By Little's law it also bounds the work in process. Together with Example 1 it shows that stability is a property of the policy, not of the load condition alone, and it gave one of the first two positive results for a whole class of reentrant lines. The explicit constants Γ(i)\Gamma^{(i)}Γ(i) make the delay guarantee quantitative after the transient.

The result is proved in the paper. As far as the platform record shows it has not been formalized: the related platform items concern the fluid models of Dai and Weiss, which have no parts, no burstiness and no nonpreemption, and a fixed instance of Example 1. This mission produces a machine-checked sample-path model of reentrant lines under buffer priority policies and a checked proof of Theorem 1 with its explicit busy-period constants. The same model serves the sibling mission on LBFS (Theorem 3 of the paper).

Difficulty

A first idea is a work-conservation argument per center: total work arriving at σ\sigmaσ grows at rate ρ<1\rho<1ρ<1 per machine, so the center cannot fall behind. This fails because the work arriving at a center σ\sigmaσ from a later buffer bib_ibi​ depends on how fast the upstream centers have pushed parts to bib_ibi​, and an upstream center can release a burst that the downstream center absorbs only after other buffers have starved; Example 1 is exactly such a cascade under a different priority order. Any argument must therefore bound the delay of a part at a buffer in terms of delays it suffered upstream at other centers, uniformly in the initial state, while nonpreemption lets a part wait up to τˉ\bar\tauτˉ for a machine busy with a lower-priority buffer and the release constraint (1) controls only arrivals to b1b_1b1​, not to later buffers. On sample paths one must also show that the initial parts clear in finite time and that only finitely many parts are served in any bounded interval.

Formalization scope

Lean represents a line by a structure with SSS, mσ>0m_\sigma>0mσ​>0, l>0l>0l>0, the route center : Fin l → Fin S and processing times τ with τi>0\tau_i>0τi​>0. The positivity is a pinned hypothesis: the paper leaves it implicit, its proof divides by τ1\tau_1τ1​, and its only zero processing times are those of Example 1. Buffers are 0-based: the paper's bib_ibi​ is Lean index i−1i-1i−1. The load is written λwˉ\lambda\bar wλwˉ, equal to max⁡σλwσ\max_\sigma\lambda w_\sigmamaxσ​λwσ​ for λ≥0\lambda\ge0λ≥0.

A run is a type of parts with an injective line order, a finite set of initial parts, entry buffers, release times, and for each part and buffer the start of service in WithTop ℝ, ⊤\top⊤ meaning never served. Admissibility imposes, at every t≥0t\ge0t≥0: precedence (service begins after arrival, except that initial parts may be in service at time 000), capacity mσm_\sigmamσ​, nonidling, head of buffer with ties broken by the line order, and the priority rule. All counts use Set.encard. The theorems hold for every admissible run, not for one particular tie-breaking schedule. The constants Γ(i)\Gamma^{(i)}Γ(i) are definitions computed from the line and λ,γ\lambda,\gammaλ,γ; the times T(i)T^{(i)}T(i) are existential after the run.

Two trivializing formalizations are ruled out: start times in ℝ would make every part served by construction and stability half empty, and a capacity constraint stated with Set.ncard would be vacuous for infinitely many parts. Γ\GammaΓ never depends on the part. A sanity file exhibits a nontrivial admissible run satisfying (1) and checks that the empty run is admissible and stable.

The page prints Γ(i)\Gamma^{(i)}Γ(i) (p. 1410) as a product of the two brackets; the inequality it is solved from gives the quotient, which is also the printed form of Γ(1)\Gamma^{(1)}Γ(1), and the formalization uses the quotient.

The sample-path model (Line, Run, Admissible, Arrivals, Stable) is reusable for any buffer priority policy and for the LBFS mission. Contributions welcome: proofs of the milestones in the listed order, lemmas on finiteness of the parts served in bounded time, and interval-counting lemmas for busy periods. Not posed here: Example 1, which is already on the platform as an open problem (QueueingStability.LuKumar.theorem31), Theorems 2–3 (LBFS, sibling mission), Theorems 4–5 and Corollary 1 (due-date policies, several flow lines).

Selected references

  • S. H. Lu and P. R. Kumar, Distributed Scheduling Based on Due Dates and Buffer Priorities, IEEE Transactions on Automatic Control 36(12), 1991, pp. 1406–1416. https://doi.org/10.1109/9.106156
  • P. R. Kumar and T. I. Seidman, Dynamic instabilities and stabilization methods in distributed real-time scheduling of manufacturing systems, IEEE Transactions on Automatic Control 35(3), 1990, pp. 289–298. https://doi.org/10.1109/9.50339
  • P. R. Kumar, Re-entrant lines, Queueing Systems 13, 1993, pp. 87–110. https://doi.org/10.1007/BF01158927
  • J. G. Dai, On positive Harris recurrence of multiclass queueing networks: a unified approach via fluid limit models, Annals of Applied Probability 5(1), 1995, pp. 49–77. https://doi.org/10.1214/aoap/1177004828
  • J. G. Dai and G. Weiss, Stability and instability of fluid models for reentrant lines, Mathematics of Operations Research 21(1), 1996, pp. 115–134. https://doi.org/10.1287/moor.21.1.115
7 thms1 active userReviewed
Dynamical SystemsOperations Research·Captain: mikedeng1

Dynamic Instabilities and Stabilization Methods in Distributed Real-Time Scheduling of Manufacturing Systems 2: Stable and Unstable Modes Coexist Under Clearing on a Two-Part-Type SystemResearch Paper

Motivation

Flexible manufacturing systems are often scheduled by simple distributed rules: each machine looks only at its own buffers and decides, in real time, which part type to work on next. Switching from one part type to another costs a set-up time during which the machine produces nothing, so a natural rule is to work on a buffer until it is empty and only then switch. Perkins and Kumar (IEEE Trans. Automat. Control 34, 1989; reference [18] of the paper) showed that such clearing policies, and in particular clear-a-fraction policies, keep every buffer bounded on acyclic systems whenever each machine has spare capacity. Whether the same holds when material flows around a cycle of machines was left open.

Kumar and Seidman (IEEE TAC 35(3), 1990) answered it negatively by two examples. Example 1 is a single re-entrant part type. Example 2, the subject of this mission, has two part types, neither of which ever revisits a machine, and shows two further phenomena: instability without re-entrance, and the coexistence, for one and the same system, of a bounded periodic regime and an unbounded one, selected only by the initial state.

Setting

A manufacturing system has part types ppp arriving at rates dpd_pdp​ and machines mmm. Parts of type ppp follow a fixed route; at stage iii they wait in buffer bp,ib_{p,i}bp,i​ at machine μp,i\mu_{p,i}μp,i​, and processing one of them takes τp,i\tau_{p,i}τp,i​. Switching a machine from buffer bbb to buffer b′b'b′ takes the set-up time δb,b′\delta_{b,b'}δb,b′​. Flows are continuous: xb(t)x_b(t)xb​(t) is the level of buffer bbb, ub(t)u_b(t)ub​(t) and yb(t)y_b(t)yb​(t) are its cumulative input and output, and

xb(t)=xb(0)+ub(t)−yb(t)≥0.x_b(t)=x_b(0)+u_b(t)-y_b(t)\ge 0 .xb​(t)=xb​(0)+ub​(t)−yb​(t)≥0.

A machine works in runs: a set-up for a buffer, then processing it at rate 1/τb1/\tau_b1/τb​ while it is nonempty, and at the rate of its inflow while it is empty. A clearing policy (Definition 1) keeps a machine on a buffer bbb until the first time that bbb is empty and another of its buffers is nonempty, and then sets it up for one of those. The system is stable if sup⁡0≤t<∞xb(t)<∞\sup_{0\le t<\infty}x_b(t)<\inftysup0≤t<∞​xb​(t)<∞ for every buffer bbb.

Example 2 (Fig. 3) has two part types with d1=d2=1d_1=d_2=1d1​=d2​=1 and two machines. Part type 1 visits buffer 1 at machine 1 and then buffer 2 at machine 2; part type 2 visits buffer 3 at machine 2 and then buffer 4 at machine 1. Processing times are τ1,…,τ4\tau_1,\dots,\tau_4τ1​,…,τ4​ and setting up to buffer kkk takes δk>0\delta_k>0δk​>0. The parameters satisfy the capacity condition (1), τ1+τ4<1\tau_1+\tau_4<1τ1​+τ4​<1 and τ2+τ3<1\tau_2+\tau_3<1τ2​+τ3​<1, and

τ2+τ4>1,δ1+δ41−τ1−τ4=δ2+δ31−τ2−τ3=:η,δ4<(1−τ3−τ4)η,δ2<(1−τ1−τ2)η,\tau_2+\tau_4>1,\qquad \frac{\delta_1+\delta_4}{1-\tau_1-\tau_4}=\frac{\delta_2+\delta_3}{1-\tau_2-\tau_3}=:\eta,\qquad \delta_4<(1-\tau_3-\tau_4)\eta,\qquad \delta_2<(1-\tau_1-\tau_2)\eta,τ2​+τ4​>1,1−τ1​−τ4​δ1​+δ4​​=1−τ2​−τ3​δ2​+δ3​​=:η,δ4​<(1−τ3​−τ4​)η,δ2​<(1−τ1​−τ2​)η,

conditions (7)–(10). The paper's feasible instance is (τ1,τ2,τ3,τ4,δ1,δ2,δ3,δ4)=(14,12,14,23,130,15,120,120)(\tau_1,\tau_2,\tau_3,\tau_4,\delta_1,\delta_2,\delta_3,\delta_4)=(\tfrac14,\tfrac12,\tfrac14,\tfrac23,\tfrac1{30},\tfrac15,\tfrac1{20},\tfrac1{20})(τ1​,τ2​,τ3​,τ4​,δ1​,δ2​,δ3​,δ4​)=(41​,21​,41​,32​,301​,51​,201​,201​), with η=1\eta=1η=1. For this system there is exactly one clearing policy.

Formalization targets

Goal: stable and unstable modes coexist

For every parameter set satisfying (1) and (7)–(10):

from x(0)=(0,η,0,η), set-ups (b1,b3): a clearing trajectory exists, and every one is bounded;∃ ξ0 ∀ ξ≥ξ0, from x(0)=(ξ,0,0,0), set-ups (b4,b3): a clearing trajectory exists, and every one has sup⁡t≥0x1(t)=+∞.\begin{aligned} &\text{from } x(0)=(0,\eta,0,\eta),\ \text{set-ups } (b_1,b_3):\ \text{a clearing trajectory exists, and every one is bounded;}\\ &\exists\,\xi_0\ \forall\,\xi\ge\xi_0,\ \text{from } x(0)=(\xi,0,0,0),\ \text{set-ups } (b_4,b_3):\ \text{a clearing trajectory exists, and every one has } \sup_{t\ge0}x_1(t)=+\infty . \end{aligned}​from x(0)=(0,η,0,η), set-ups (b1​,b3​): a clearing trajectory exists, and every one is bounded;∃ξ0​ ∀ξ≥ξ0​, from x(0)=(ξ,0,0,0), set-ups (b4​,b3​): a clearing trajectory exists, and every one has t≥0sup​x1​(t)=+∞.​

Milestones

  1. Case 1, periodic regime: under (1) and (8)–(10), every clearing trajectory from (0,η,0,η)(0,\eta,0,\eta)(0,η,0,η), (b1,b3)(b_1,b_3)(b1​,b3​) satisfies x(η)=(0,η,0,η)x(\eta)=(0,\eta,0,\eta)x(η)=(0,η,0,η) with the same set-ups.
  2. Case 2, Stage 8: under (1) and (7), for large ξ\xiξ, with t9=(ξ+τ2−1(δ1+δ2))/(τ2−1−1)t_9=(\xi+\tau_2^{-1}(\delta_1+\delta_2))/(\tau_2^{-1}-1)t9​=(ξ+τ2−1​(δ1​+δ2​))/(τ2−1​−1): x(t9)=(0,0,t9−δ1,0)x(t_9)=(0,0,t_9-\delta_1,0)x(t9​)=(0,0,t9​−δ1​,0), machine 1 set up for buffer 1, machine 2 for buffer 2.
  3. Case 2, full cycle: under (1) and (7), for large ξ\xiξ, at an explicit time t17t_{17}t17​, x(t17)=(λξ+α,0,0,0)x(t_{17})=(\lambda\xi+\alpha,0,0,0)x(t17​)=(λξ+α,0,0,0) with set-ups (b4,b3)(b_4,b_3)(b4​,b3​), where λ=τ2τ4/((1−τ2)(1−τ4))>1\lambda=\tau_2\tau_4/((1-\tau_2)(1-\tau_4))>1λ=τ2​τ4​/((1−τ2​)(1−τ4​))>1.

Significance

The result shows that stability of a scheduling policy is a property of the initial state as well as of the system: the same parameters admit a bounded periodic orbit and an orbit along which the backlog of buffer 1 is multiplied by λ>1\lambda>1λ>1 in every cycle. The capacity condition (1) holds throughout, so the instability does not come from overload: spare capacity is lost because one machine starves the other. The example also shows that a cycle in the machine graph suffices, without any part type revisiting a machine. Sections IV and V of the paper respond to this by giving a stronger capacity condition under which clear-a-fraction policies are stable, and a supervisory mechanism that stabilizes any policy.

The paper's argument is a stage-by-stage computation of a piecewise-linear trajectory. It has not been machine-checked. A formalization fixes what a clearing trajectory is (including the switching instants at which a buffer is empty but just starting to fill), proves that the trajectories exist and are determined by the initial state, and checks the paper's closed forms. One printed constant is wrong: the offset α\alphaα of the full cycle lacks a final −δ3-\delta_3−δ3​, and the milestone states the corrected value.

Difficulty

The computations within each stage are linear algebra. The difficulty is the event structure. Every stage depends on the order in which the two machines' events occur: whether machine 1 clears buffer 1 before machine 2 has finished its set-ups, whether buffer 2 is still nonempty when buffer 1 empties, and so on. These orderings hold only for ξ\xiξ large, or only because (8) holds with equality and (9)–(10) hold strictly. The periodic regime depends on exact synchronization. In Case 2, a machine processing an empty buffer at the reduced rate must not be forced off it while its other buffer is empty and not yet fed. A proof must also establish existence of each trajectory, not only compute it.

Formalization scope

  • Model. The general system has part types Fin P, machines Fin M, buffers as pairs (p,i)(p,i)(p,i) with Lean index iii for paper stage i+1i+1i+1, real time, and no transport delay or assembly (as in the paper). Example 2 is an instance of this general model, with machines 1, 2 as Lean 0, 1.
  • Runs. Each machine follows a sequence of runs. Run 0 is on the initial set-up and has no set-up phase. Run k≥1k\ge1k≥1 has a set-up phase of length δβk−1,βk\delta_{\beta_{k-1},\beta_k}δβk−1​,βk​​, followed by a processing phase. Runs are never cut during a set-up. If there are infinitely many runs, their start times tend to ∞\infty∞. There is no idling outside runs: a machine serving an empty buffer passes on its inflow.
  • Set-up times. They depend only on the target buffer, and δb,b=0\delta_{b,b}=0δb,b​=0 is never used.
  • Processing law. The rate cap is yb(t)−yb(s)≤(1/τb)⋅y_b(t)-y_b(s)\le(1/\tau_b)\cdotyb​(t)−yb​(s)≤(1/τb​)⋅(processing time of bbb in [s,t][s,t][s,t]). Rate is exactly 1/τb1/\tau_b1/τb​ while the machine is processing a nonempty buffer.
  • Clearing. "Nonempty" in Definition 1 is read as demanding: positive level, or cumulative input strictly increasing from that instant. Under the literal reading the paper's own switching instants would not be "first times", and no clearing trajectory would exist from these initial states. The no-early-exit condition is imposed on the open processing interval.
  • Set up for bbb at ttt. At t>0t>0t>0 this refers to the run whose interval (sk,sk+1](s_k,s_{k+1}](sk​,sk+1​] contains ttt.
  • Ruling out trivial formalizations. The goal asserts existence of a clearing trajectory in both modes, so neither "every trajectory is bounded" nor "every trajectory is unbounded" can hold vacuously. η\etaη is computed from the data rather than pinned by hypotheses, and (8) is an equality.

Contributions welcome: existence and uniqueness of clearing trajectories for the two-machine instance, the event-ordering lemmas for large ξ\xiξ, the restart (time-shift) property of trajectories, and the three milestones.

Selected references

  • P. R. Kumar and T. I. Seidman, Dynamic instabilities and stabilization methods in distributed real-time scheduling of manufacturing systems, IEEE Transactions on Automatic Control 35(3):289–298, 1990. https://doi.org/10.1109/9.50339
  • J. R. Perkins and P. R. Kumar, Stable, distributed, real-time scheduling of flexible manufacturing/assembly/disassembly systems, IEEE Transactions on Automatic Control 34:139–148, February 1989 (reference [18] of the paper).
8 thms1 active userReviewed
Dynamical SystemsOperations Research·Captain: mikedeng1

Dynamic Instabilities and Stabilization Methods in Distributed Real-Time Scheduling of Manufacturing Systems 4: A Run-Truncating Priority-Queue Supervisor Keeps Every Buffer Bounded When ρₘ < 1Research Paper

Motivation

A flexible manufacturing system routes several part types through a set of machines; a machine that switches from one kind of part to another must spend a set-up time first. Each machine has to decide, in real time and from local information only, which of its waiting buffers to work on and for how long. Long runs waste little time on set-ups but starve other buffers; short runs do the opposite.

Kumar and Seidman (IEEE TAC 35(3), 1990) showed in §III of the paper that natural distributed rules (clearing policies, which keep processing a buffer until it is empty) can be unstable when parts revisit machines, even though every machine has enough capacity. Their §IV gives a sufficient condition for a subclass of such rules to be stable, which is stricter than the capacity condition. §V, the subject of this mission, asks for something stronger: a simple, distributed mechanism that a supervisor can add to any scheduling policy and that makes the system stable whenever the capacity condition alone holds.

Timeline. Perkins and Kumar (1989) proved that the capacity condition is sufficient for the existence of some stabilizing policy, and that every clear-a-fraction policy is stable on acyclic systems. Kumar and Seidman (1990) gave the instability examples for nonacyclic systems and the universal supervisor formalized here.

Setting

There are PPP part types and MMM machines. Part type ppp arrives at rate dp>0d_p>0dp​>0 and follows a route of npn_pnp​ operations; operation iii is done by machine μp,i\mu_{p,i}μp,i​ and takes τp,i>0\tau_{p,i}>0τp,i​>0 per part. Parts awaiting operation iii wait in buffer bp,ib_{p,i}bp,i​, with level xp,i(t)≥0x_{p,i}(t)\ge 0xp,i​(t)≥0. Machine mmm serves Bm={bp,i:μp,i=m}B_m=\{b_{p,i}:\mu_{p,i}=m\}Bm​={bp,i​:μp,i​=m}, and switching from bbb to b′b'b′ costs δb,b′≥0\delta_{b,b'}\ge 0δb,b′​≥0. Flows are continuous: a machine processing a nonempty buffer bp,ib_{p,i}bp,i​ drains it at rate 1/τp,i1/\tau_{p,i}1/τp,i​, the output feeds bp,i+1b_{p,i+1}bp,i+1​ instantly, and a machine on an empty buffer passes its inflow through. Each machine works in runs: a set-up, then a processing phase on one buffer.

The load of machine mmm and the capacity condition are

ρm:=∑(p,i): μp,i=mdpτp,i<1(1≤m≤M).(1)\rho_m:=\sum_{(p,i):\,\mu_{p,i}=m}d_p\tau_{p,i}<1\qquad(1\le m\le M). \tag{1}ρm​:=(p,i):μp,i​=m∑​dp​τp,i​<1(1≤m≤M).(1)

The system is stable if sup⁡0≤t<∞xp,i(t)<∞\sup_{0\le t<\infty}x_{p,i}(t)<\inftysup0≤t<∞​xp,i​(t)<∞ for every buffer.

The supervisor picks γm\gamma_mγm​ with

γm(1−ρm)>∑b∈Bmmax⁡b′∈Bmδb′,b(24)\gamma_m(1-\rho_m)>\sum_{b\in B_m}\max_{b'\in B_m}\delta_{b',b} \tag{24}γm​(1−ρm​)>b∈Bm​∑​b′∈Bm​max​δb′,b​(24)

and thresholds zp,i≥0z_{p,i}\ge0zp,i​≥0. It truncates every processing run of bp,ib_{p,i}bp,i​ at γmdpτp,i\gamma_m d_p\tau_{p,i}γm​dp​τp,i​; it keeps a first-come first-served queue QmQ_mQm​ of the buffers that are not being processed and whose level exceeds zp,iz_{p,i}zp,i​; when a run ends and Qm≠∅Q_m\ne\emptysetQm​=∅, the head of QmQ_mQm​ is processed next, for exactly γmdpτp,i\gamma_m d_p\tau_{p,i}γm​dp​τp,i​ unless it clears earlier. When QmQ_mQm​ is empty the underlying policy is free. Write ζm:=γm(1−ρm)−∑b∈Bmmax⁡b′∈Bmδb′,b>0\zeta_m:=\gamma_m(1-\rho_m)-\sum_{b\in B_m}\max_{b'\in B_m}\delta_{b',b}>0ζm​:=γm​(1−ρm​)−∑b∈Bm​​maxb′∈Bm​​δb′,b​>0 (26) and βp,i(t):=∑j≤iτp,ixp,j(t)\beta_{p,i}(t):=\sum_{j\le i}\tau_{p,i}x_{p,j}(t)βp,i​(t):=∑j≤i​τp,i​xp,j​(t), the backlog of part type ppp for machine μp,i\mu_{p,i}μp,i​.

Formalization targets

Goal: Theorem 2

Under (1), (24) and z≥0z\ge 0z≥0, every supervised trajectory, from every initial state, satisfies

sup⁡0≤t<∞xp,i(t)<+∞for all p,i.\sup_{0\le t<\infty}x_{p,i}(t)<+\infty\qquad\text{for all }p,i.0≤t<∞sup​xp,i​(t)<+∞for all p,i.

The bound is allowed to depend on the trajectory; no uniformity in the initial state is claimed.

Milestones: Lemma 1

  1. A buffer that enters QmQ_mQm​ at tint_{\rm in}tin​ completes its next run by tin+γm−ζmt_{\rm in}+\gamma_m-\zeta_mtin​+γm​−ζm​ (25).
  2. If it enters QmQ_mQm​ with xp,i(tin)≥γmdpx_{p,i}(t_{\rm in})\ge\gamma_m d_pxp,i​(tin​)≥γm​dp​, that run lowers the backlog: βp,i(tcomplete)≤βp,i(tin)−ζmdpτp,i\beta_{p,i}(t_{\rm complete})\le\beta_{p,i}(t_{\rm in})-\zeta_m d_p\tau_{p,i}βp,i​(tcomplete​)≤βp,i​(tin​)−ζm​dp​τp,i​.
  3. A buffer that enters QmQ_mQm​ at tint_{\rm in}tin​ and stays above max⁡(γmdp,zp,i)\max(\gamma_md_p,z_{p,i})max(γm​dp​,zp,i​) (strictly above zp,iz_{p,i}zp,i​ after tint_{\rm in}tin​) up to t′t't′ has t′−tin≤(1+βp,i(tin)/(ζmdpτp,i))(γm−ζm)t'-t_{\rm in}\le(1+\beta_{p,i}(t_{\rm in})/(\zeta_md_p\tau_{p,i}))(\gamma_m-\zeta_m)t′−tin​≤(1+βp,i​(tin​)/(ζm​dp​τp,i​))(γm​−ζm​).
  4. Any interval on which xp,i≥gp,i:=max⁡(γmdp,zp,i)+γmdpx_{p,i}\ge g_{p,i}:=\max(\gamma_md_p,z_{p,i})+\gamma_md_pxp,i​≥gp,i​:=max(γm​dp​,zp,i​)+γm​dp​ has length at most ap,i+cp,iβp,i(t)a_{p,i}+c_{p,i}\beta_{p,i}(t)ap,i​+cp,i​βp,i​(t).

Significance

The theorem separates two concerns of real-time scheduling. The underlying policy can be tuned for performance (low set-up overhead, short buffers) without any stability analysis; the supervisor guarantees stability under exactly the condition that is necessary for it. The mechanism is distributed: each machine uses only its own buffer levels. With z=0z=0z=0 it enforces FCFS with truncated runs; with large zzz and γ\gammaγ it never intervenes on a policy that is already stable, so the degree of intervention is tunable. Combined with the §III instability examples, it shows that instability of clearing policies is a design defect that a local safeguard removes.

The result is proved in the paper; to our knowledge it has no machine-checked proof. This mission produces a formal model of continuous-flow manufacturing systems with set-up times and runs, a formal definition of the supervisor, and the four parts of Lemma 1 as reusable statements about it. Part 3) is stated with a corrected hypothesis (see Formalization scope).

Difficulty

The underlying policy is arbitrary, so no structure of the trajectory between interventions can be used: the policy may idle on empty buffers, switch at will, and re-select a buffer as soon as its run is truncated. The natural approach of bounding the total work at a machine fails, because work arrives at machine mmm from upstream machines whose behaviour depends on mmm itself through re-entrant routes, so a machine-by-machine argument is circular. A further difficulty is that the supervisor acts only on buffers above their thresholds, so a buffer can be ignored for as long as it sits at or below zp,iz_{p,i}zp,i​; any bound has to account for the time spent below the threshold and for repeated re-entries into QmQ_mQm​, with a set-up paid at each.

Formalization scope

  • Model. Part types Fin P, machines Fin M, buffers Σ p, Fin (n p); paper index iii is Lean index i−1i-1i−1. Time is real with t≥0t\ge0t≥0. Set-up times satisfy δb,b=0\delta_{b,b}=0δb,b​=0: a set-up is paid only on a switch. No transport delays or assembly, as in the paper.
  • Runs. Each machine is always in a run (no idling outside runs); a run is never cut during its set-up; run starts do not accumulate; zero-length runs are allowed. Processing is at rate at most 1/τb1/\tau_b1/τb​ and only in a processing phase of bbb, and at exactly 1/τb1/\tau_b1/τb​ while bbb is nonempty.
  • Queue. Membership of QmQ_mQm​ is determined by the trajectory: not on the current run and level strictly above zbz_bzb​. The FCFS key is the start of the current sojourn; ties, including the initial queue at time 000, are broken arbitrarily, so every initial queue order is covered. A buffer whose run just ended is in QmQ_mQm​ at that instant if its level exceeds zbz_bzb​.
  • Lemma 1. Parts 1)–3) assume, as the page does, that the buffer enters QmQ_mQm​ at tint_{\rm in}tin​. Part 3) adds xp,i>zp,ix_{p,i}>z_{p,i}xp,i​>zp,i​ on (tin,t′](t_{\rm in},t'](tin​,t′]: with the printed non-strict hypothesis the buffer can sit at exactly zp,iz_{p,i}zp,i​, never be re-queued, and wait arbitrarily long. In 4), cp,ic_{p,i}cp,i​ is (γm−ζm)/(ζmdpτp,i)(\gamma_m-\zeta_m)/(\zeta_md_p\tau_{p,i})(γm​−ζm​)/(ζm​dp​τp,i​).
  • Ruled out. The base policy is not restricted to clearing, CAF or any fixed rule, the system is not restricted to few machines or acyclic routes, and the hypothesis of the goal is (24), not ζm>0\zeta_m>0ζm​>0. A non-vacuity witness (one machine, one buffer) shows the class of supervised trajectories is nonempty.
  • Reusable. The trajectory model (runs, set-ups, processing law) is the same as in the other missions of this series and applies to any distributed scheduling policy with set-ups.

Contributions welcome: proofs of the Lemma 1 parts, and lemmas on the model (Lipschitz continuity of levels, monotonicity of a buffer's level while it is not processed).

Selected references

  • P. R. Kumar and T. I. Seidman, Dynamic instabilities and stabilization methods in distributed real-time scheduling of manufacturing systems, IEEE Trans. Automat. Control 35(3), 289–298, 1990. https://doi.org/10.1109/9.50339
  • J. R. Perkins and P. R. Kumar, Stable, distributed, real-time scheduling of flexible manufacturing/assembly/disassembly systems, IEEE Trans. Automat. Control 34(2), 139–148, 1989. https://doi.org/10.1109/9.21085
8 thms1 active userReviewed
Convex OptimizationProbability·Captain: mikedeng1

Convex Duality in Constrained Portfolio Optimization II: The Dual Problem Attains Its Infimum, So an Optimal Constrained Portfolio–Consumption Pair Exists for Every Initial CapitalResearch Paper

Motivation

Portfolio decisions in continuous time must respect both market uncertainty and investment restrictions. A ban on short sales, for example, restricts the vector of wealth fractions a trader may hold at every instant. The corresponding optimal consumption and investment problem is harder than its unrestricted counterpart because a candidate portfolio obtained from the usual pricing equations may violate the constraint. Cvitanić and Karatzas introduced auxiliary markets indexed by a process and used a dual optimization problem to establish existence of an optimal constrained policy in their 1992 paper. This mission formalizes the paper's existence theorem, Theorem 13.1, together with the results in Sections 12–13 that specify its dual target.

Setting

Fix a finite horizon T>0T>0T>0 and a complete probability space carrying a standard ddd-dimensional Brownian motion WWW. Information at time ttt is the probability augmentation of the filtration generated by WWW up to ttt. A bond earns short rate rtr_trt​; ddd stocks have appreciation vector btb_tbt​ and volatility matrix σt\sigma_tσt​. The market assumptions require progressively measurable coefficients, a uniform lower bound on rrr, uniform nondegeneracy of σσT\sigma\sigma^{\mathsf T}σσT, finite expected interest-rate integral, and finite energy of the market price of risk θ=σ−1(b−r1)\theta=\sigma^{-1}(b-r\mathbf1)θ=σ−1(b−r1) §2.

A portfolio πt\pi_tπt​ specifies fractions of wealth held in the stocks, while ct≥0c_t\geq0ct​≥0 is consumption and XtX_tXt​ is wealth. The feasible portfolios lie in a fixed nonempty closed convex set K⊆RdK\subseteq\mathbb R^dK⊆Rd. Its support function is δK(v)=sup⁡π∈K(−π⋅v)\delta_K(v)=\sup_{\pi\in K}(-\pi\cdot v)δK​(v)=supπ∈K​(−π⋅v), which may equal +∞+\infty+∞. The process ν\nuν indexing an auxiliary market belongs to D\mathcal DD when it has finite expected quadratic energy and finite expected integrated support value. Its deflator HνH_\nuHν​ combines an interest-rate discount with a Brownian exponential. The class D′\mathcal D'D′ consists of those ν∈D\nu\in\mathcal Dν∈D for which the budget map Xν(y)\mathcal X_\nu(y)Xν​(y) is finite for every y>0y>0y>0 §§4, 8.

The running utility U1(t,c)U_1(t,c)U1​(t,c) and terminal utility U2(XT)U_2(X_T)U2​(XT​) are strictly increasing and strictly concave for positive arguments. An admissible constrained policy has nonnegative wealth, finite expected negative utility, and πt∈K\pi_t\in Kπt​∈K for Lebesgue-time times probability almost every (t,ω)(t,\omega)(t,ω). The primal value V(x)V(x)V(x) is the supremum of its expected utility over all such policies with initial capital x>0x>0x>0. The dual value V~(y)\widetilde V(y)V(y) is the infimum, over ν∈D\nu\in\mathcal Dν∈D, of expected conjugate utility evaluated at yHνyH_\nuyHν​ §§5–6, 12.

Formalization targets

Dual attainment

The first conclusion of Theorem 13.1 is the paper's condition (12.9):

∀y>0∃λy∈D′V~(y)=J~(y;λy).\forall y>0\quad\exists\lambda_y\in\mathcal D'\quad\widetilde V(y)=\widetilde J(y;\lambda_y).∀y>0∃λy​∈D′V(y)=J(y;λy​).

Thus the infimum is attained by a drift with a finite budget map. It is a conclusion of this mission, even though earlier Section 12 results assume it. The milestones include weak duality (12.8), the dual representation of Proposition 12.1, the capital-matching result of Proposition 12.2, and the analytic properties of the extended dual functional in Proposition 13.2 pp. 792–795.

An optimal constrained policy for every capital

The theorem's second conclusion is

∀x>0∃(π^,c^,X^)∈A′(x)∀(π,c,X)∈A′(x),J(π,c,X)≤J(π^,c^,X^).\forall x>0\quad\exists (\widehat\pi,\widehat c,\widehat X)\in\mathcal A'(x)\quad\forall(\pi,c,X)\in\mathcal A'(x),\quad J(\pi,c,X)\leq J(\widehat\pi,\widehat c,\widehat X).∀x>0∃(π,c,X)∈A′(x)∀(π,c,X)∈A′(x),J(π,c,X)≤J(π,c,X).

Here A′(x)\mathcal A'(x)A′(x) is the constrained admissible class. Theorem 12.4 establishes this conclusion when dual attainment is assumed; Theorem 13.1 provides that assumption under the paper's stated utility and dual-finiteness conditions p. 794.

Significance

The result supplies an optimizer, rather than only a bound on the best attainable utility. It also identifies a minimizing auxiliary market for every positive dual parameter, making the dual description usable at a specified initial capital. The dual representation links the primal value to a convex function that can be studied without choosing a constrained policy first §§12–13.

The theorem is proved in the paper; the remaining work here is a machine-checked development of its definitions, hypotheses, and existence argument. The mission also produces reusable definitions of an almost-surely square-integrable Brownian integral relation, extended-real utility expectations, an extended support function, and constrained wealth classes. The Brownian state space and predictable step-sum substrate credit the published Ethier–Kurtz formalizations. The paper refers some analytic steps to earlier works instead of reproducing full proofs, including Lemma 12.3 and a final finiteness argument pp. 793–796.

Difficulty

Weak duality alone only bounds the constrained value from above. It does not supply a minimizer of J~(y;ν)\widetilde J(y;\nu)J(y;ν), and an arbitrary minimizing auxiliary process need not belong to D′\mathcal D'D′. The dual functional must retain meaningful infinite values on the full finite-energy process space H\mathcal HH; replacing them with total-operation defaults can turn the needed coercivity and lower-semicontinuity statements into different claims. Even after dual attainment, the minimizing parameter for a prescribed capital must match the budget map Xλy(y)\mathcal X_{\lambda_y}(y)Xλy​​(y) §§12–13. These are the precise gaps between a numerical bound and the asserted optimal policy.

Formalization scope

Lean uses Rd\mathbb R^dRd with its Euclidean norm, a finite positive horizon, a complete probability measure, and the augmented Brownian filtration. Processes are functions of time and sample point; ℓ⊗P\ell\otimes Pℓ⊗P almost-everywhere restrictions on portfolios and auxiliary drifts are distinct from pathwise PPP-almost-sure statements. A policy is represented by a triple (π,c,X)(\pi,c,X)(π,c,X) whose wealth process satisfies the integral equation, matching the paper's pair together with its unique wealth solution. The integral operator is bound to the local Itô construction by predictable step approximations; it is not a free function. Integrands must be in its almost-sure square-integrable domain.

Support values, utility at zero, expected utility, and value functions use extended reals. The extension U(0)U(0)U(0) is the right limit U(0+)U(0+)U(0+); an extended-real supremum defines conjugate utility where zero arguments arise. The inverse marginal is a positive-argument inverse of U′U'U′. The minimization in V~\widetilde VV ranges over D\mathcal DD, while (12.9) asks for a minimizer in D′\mathcal D'D′. Its witnesses λy\lambda_yλy​ remain explicit choices in statements involving Xλy\mathcal X_{\lambda_y}Xλy​​; no default-valued inverse Yν\mathcal Y_\nuYν​ is introduced. The functional J~y\widetilde J_yJy​ of (13.3) is defined on H\mathcal HH, uses the paper's formula on G\mathcal GG, and is +∞+\infty+∞ off G\mathcal GG. On D\mathcal DD it agrees almost everywhere with the original dual objective, an equality requiring a formal lemma. Assumption 6.2 and conditions (5.8), (8.25), (12.2), (12.3), and (12.11) are explicit. Their quantifiers cover both utilities and every time in [0,T][0,T][0,T] where stated. The mission rules out a vacuous dual-attainment hypothesis, a zero default for an unbounded support supremum, and an unconstrained stochastic-integral operator. Contributions proving the source milestones and the analytic lemmas they need are in scope.

Selected references

  • Jakša Cvitanić and Ioannis Karatzas, Convex Duality in Constrained Portfolio Optimization, The Annals of Applied Probability 2(4), 1992, 767–818. DOI: 10.1214/aoap/1177005576.
15 thms1 active userReviewed
Dynamical SystemsOperations ResearchOptimization·Captain: mikedeng1

Homogeneous Approximation, Recursive Observer Design, and Output Feedback: Global Asymptotic Stabilization of a Chain of Integrators by a Homogeneous-in-the-Bi-Limit Output FeedbackResearch Paper

Motivation

Many nonlinear control systems behave like a chain of integrators, x˙1=x2,…,x˙n=u\dot x_1=x_2,\dots,\dot x_n=ux˙1​=x2​,…,x˙n​=u, with additional terms that are small either near the origin or far from it. Designs based on weighted homogeneity handle such systems by making the closed loop invariant under a dilation, which gives robustness to perturbations that are dominated by the homogeneous part (Rosier, Homogeneous Lyapunov function for homogeneous continuous vector field, Systems & Control Letters, 1992, doi:10.1016/0167-6911(92)90078-7). A single homogeneous approximation, however, describes the system only at one scale. A perturbation that is negligible near the origin may dominate at infinity, and a design tuned to the origin can then fail globally.

Andrieu, Praly and Astolfi (arXiv:0903.0298v1; SIAM J. Control Optim., 2008, doi:10.1137/060675861) introduce homogeneity in the bi-limit: a function or vector field has one homogeneous approximation near the origin and another, possibly with different weights and degree, at infinity. They develop the basic calculus of this notion, a converse Lyapunov theorem for it, a recursive observer and a recursive state feedback for the chain of integrators, and combine them into a global output feedback. This mission formalizes that output-feedback result and the chain of results it rests on.

Setting

For a weight r=(r1,…,rn)r=(r_1,\dots,r_n)r=(r1​,…,rn​) with all ri>0r_i>0ri​>0 and λ>0\lambda>0λ>0, the dilation is λr⋄x=(λr1x1,…,λrnxn)\lambda^r\diamond x=(\lambda^{r_1}x_1,\dots,\lambda^{r_n}x_n)λr⋄x=(λr1​x1​,…,λrn​xn​). A continuous function ϕ:Rn→R\phi:\mathbb R^n\to\mathbb Rϕ:Rn→R is homogeneous in the 0-limit with triple (r0,d0,ϕ0)(r_0,d_0,\phi_0)(r0​,d0​,ϕ0​), degree d0≥0d_0\ge0d0​≥0, if ϕ0\phi_0ϕ0​ is continuous and not identically zero and, for every compact C⊆Rn∖{0}C\subseteq\mathbb R^n\setminus\{0\}C⊆Rn∖{0} and every ε>0\varepsilon>0ε>0, there is λ0>0\lambda_0>0λ0​>0 such that

max⁡x∈C∣ϕ(λr0⋄x)λd0−ϕ0(x)∣≤εfor all λ∈(0,λ0].\max_{x\in C}\left|\frac{\phi(\lambda^{r_0}\diamond x)}{\lambda^{d_0}}-\phi_0(x)\right|\le\varepsilon\qquad\text{for all }\lambda\in(0,\lambda_0].x∈Cmax​​λd0​ϕ(λr0​⋄x)​−ϕ0​(x)​≤εfor all λ∈(0,λ0​].

Homogeneity in the ∞-limit with triple (r∞,d∞,ϕ∞)(r_\infty,d_\infty,\phi_\infty)(r∞​,d∞​,ϕ∞​) is the same statement for all λ≥λ∞\lambda\ge\lambda_\inftyλ≥λ∞​. A vector field f=(f1,…,fn)f=(f_1,\dots,f_n)f=(f1​,…,fn​) is homogeneous with triple (r,d,f0)(r,\mathfrak d,f_0)(r,d,f0​), d∈R\mathfrak d\in\mathbb Rd∈R, when each fif_ifi​ is homogeneous with triple (r,d+ri,f0,i)(r,\mathfrak d+r_i,f_{0,i})(r,d+ri​,f0,i​) and d+ri≥0\mathfrak d+r_i\ge0d+ri​≥0. Bi-limit means both at once.

For the chain of integrators, Snx=(x2,…,xn,0)TS_n x=(x_2,\dots,x_n,0)^TSn​x=(x2​,…,xn​,0)T and Bn=(0,…,0,1)TB_n=(0,\dots,0,1)^TBn​=(0,…,0,1)T. Given degrees d0,d∞∈(−1,1n−1)\mathfrak d_0,\mathfrak d_\infty\in(-1,\frac1{n-1})d0​,d∞​∈(−1,n−11​), the weights are

r0,i=1−d0(n−i),r∞,i=1−d∞(n−i),i=1,…,n.r_{0,i}=1-\mathfrak d_0(n-i),\qquad r_{\infty,i}=1-\mathfrak d_\infty(n-i),\qquad i=1,\dots,n .r0,i​=1−d0​(n−i),r∞,i​=1−d∞​(n−i),i=1,…,n.

The output feedback studied is

x˙=Snx+Bnu,y=x1,X^˙n=L(SnX^n+Bnϕn(X^n)+K1(x1−x^1)),u=Lnϕn(X^n),\dot x=S_nx+B_nu,\quad y=x_1,\qquad \dot{\hat{\mathfrak X}}_n=L\bigl(S_n\hat{\mathfrak X}_n+B_n\phi_n(\hat{\mathfrak X}_n)+K_1(x_1-\hat x_1)\bigr),\quad u=L^n\phi_n(\hat{\mathfrak X}_n),x˙=Sn​x+Bn​u,y=x1​,X^˙n​=L(Sn​X^n​+Bn​ϕn​(X^n​)+K1​(x1​−x^1​)),u=Lnϕn​(X^n​),

with a gain L>0L>0L>0, a state feedback ϕn:Rn→R\phi_n:\mathbb R^n\to\mathbb Rϕn​:Rn→R and an output-injection vector field K1:Rn→RnK_1:\mathbb R^n\to\mathbb R^nK1​:Rn→Rn, where K1(s)K_1(s)K1​(s) means K1(s,0,…,0)K_1(s,0,\dots,0)K1​(s,0,…,0). Its homogeneous approximations are the same closed loop with (ϕn,0,K1,0)(\phi_{n,0},K_{1,0})(ϕn,0​,K1,0​) and with (ϕn,∞,K1,∞)(\phi_{n,\infty},K_{1,\infty})(ϕn,∞​,K1,∞​) in place of (ϕn,K1)(\phi_n,K_1)(ϕn​,K1​).

Formalization targets

Goal: Theorem 5.1

For n≥2n\ge2n≥2 and d0,d∞∈(−1,1n−1)\mathfrak d_0,\mathfrak d_\infty\in(-1,\frac1{n-1})d0​,d∞​∈(−1,n−11​) there exist ϕn\phi_nϕn​, homogeneous in the bi-limit with triples (r0,1+d0,ϕn,0)(r_0,1+\mathfrak d_0,\phi_{n,0})(r0​,1+d0​,ϕn,0​) and (r∞,1+d∞,ϕn,∞)(r_\infty,1+\mathfrak d_\infty,\phi_{n,\infty})(r∞​,1+d∞​,ϕn,∞​), and K1K_1K1​, homogeneous in the bi-limit with triples (r0,d0,K1,0)(r_0,\mathfrak d_0,K_{1,0})(r0​,d0​,K1,0​) and (r∞,d∞,K1,∞)(r_\infty,\mathfrak d_\infty,K_{1,\infty})(r∞​,d∞​,K1,∞​), such that

∀L>0:0∈R2n is globally asymptotically stable for the closed loop and for both of its homogeneous approximations.\forall L>0:\quad 0\in\mathbb R^{2n}\text{ is globally asymptotically stable for the closed loop and for both of its homogeneous approximations.}∀L>0:0∈R2n is globally asymptotically stable for the closed loop and for both of its homogeneous approximations.

The pair (ϕn,K1)(\phi_n,K_1)(ϕn​,K1​) is fixed before LLL. No constant appears in the goal.

Milestones

  1. Proposition 2.10 (composition) and Proposition 2.12 (integration along a coordinate) preserve homogeneity in the limit.
  2. Lemma 2.13: for homogeneous η\etaη and γ≥0\gamma\ge0γ≥0 with η<0\eta<0η<0 where γ=0\gamma=0γ=0, η−cγ<0\eta-c\gamma<0η−cγ<0 off the origin for all large ccc, simultaneously for both approximations.
  3. Theorem 2.20: a converse Lyapunov theorem, giving a C1C^1C1 proper Lyapunov function whose gradient is homogeneous in the bi-limit.
  4. Theorem 3.1 and display (3.12): the recursive observer, an output injection K1K_1K1​ making E˙=SnE+K1(e1)\dot E=S_nE+K_1(e_1)E˙=Sn​E+K1​(e1​) and its approximations globally asymptotically stable.
  5. Theorem 4.1 and display (4.9): the recursive state feedback, a ϕn\phi_nϕn​ making X˙=SnX+Bnϕn(X)\dot{\mathfrak X}=S_n\mathfrak X+B_n\phi_n(\mathfrak X)X˙=Sn​X+Bn​ϕn​(X) and its approximations globally asymptotically stable.

Significance

The theorem gives a single dynamic output feedback for the chain of integrators that is homogeneous with prescribed degrees near the origin and at infinity, and that stabilizes globally for every gain L>0L>0L>0. Because the degrees at the two ends are independent, the same design can be matched to perturbations of different orders at small and large amplitudes; the paper derives from it output-feedback stabilizers for systems in feedback and feedforward form (Corollaries 5.2 and 5.3), which are outside this mission. The converse Lyapunov theorem (Theorem 2.20) and the key lemma (Lemma 2.13) are general tools for any vector field homogeneous in the bi-limit.

The results are proved in the paper; none of them, nor the notion of homogeneity in the bi-limit, has a machine-checked proof. The platform has a definition of global asymptotic stability for continuous vector fields (from the series on Chitour, Ushirobira, Efimov and Perruquetti's prescribed-time stabilization) and Mathlib has the needed analysis, but there is no converse Lyapunov theorem and no theory of weighted homogeneous vector fields. A complete formalization would supply both.

Difficulty

Lyapunov arguments for homogeneous systems usually compare a function with its homogeneous approximation on the unit sphere and then scale. With homogeneity in the bi-limit the comparison holds only for small or for large dilation parameters, with no control at intermediate scales, and the two approximations have different weights, so no single dilation brings the whole space to a compact set. Each statement has to hold for the system and for both approximations at once. Theorem 2.20 additionally requires a converse Lyapunov theorem for continuous, non-Lipschitz vector fields whose solutions need not be unique, so a flow map is not available. Finally, the gain LLL in the goal is arbitrary: an argument that only works for LLL large enough proves a weaker statement.

Formalization scope

Points of Rn\mathbb R^nRn are functions Fin n → ℝ, with the paper's coordinate xix_ixi​ at index i−1i-1i−1; the closed loop lives on Fin (n + n) → ℝ with xxx first and X^n\hat{\mathfrak X}_nX^n​ second. Real powers are Real.rpow; the signed power is wr=sign⁡(w)∣w∣rw^r=\operatorname{sign}(w)|w|^rwr=sign(w)∣w∣r. Every clause of Definitions 2.1, 2.3 and 2.5 is kept: positive weights, nonnegative function degrees, continuity, approximating functions not identically zero, uniformity on compact subsets of Rn∖{0}\mathbb R^n\setminus\{0\}Rn∖{0}. Dropping any of them makes the goal weaker than the paper's.

Global asymptotic stability reuses the published ChitourPrescribedTime.FixedTime.GloballyAsymptoticallyStable on the Euclidean space, applied to the time-invariant field: the origin is an equilibrium, every initial state has a solution on [0,∞)[0,\infty)[0,∞), and stability and uniform attractivity hold for every solution. The mission adds that no solution escapes to infinity in finite time, so that every solution is covered; for continuous autonomous fields this is the textbook notion. Positive and negative definiteness, properness (V→∞V\to\inftyV→∞ along the cocompact filter), C1C^1C1 and partial derivatives (Fréchet derivative on a basis vector) are defined in the mission's definition files.

Corrections and readings, each disclosed in the item concerned:

  • Proposition 2.10 is false as printed. The composite approximation ζ0∘ϕ0\zeta_0\circ\phi_0ζ0​∘ϕ0​ can vanish identically, and in the ∞-limit a degree-0 outer function fails. Both extra hypotheses are added.
  • (5.2) prints K1(x1−x^1)K_1(x_1-\hat x_1)K1​(x1​−x^1​) while (3.3) and (5.9) use x^1−x1\hat x_1-x_1x^1​−x1​. The goal keeps the printed sign; since K1K_1K1​ is existential, the two readings are equivalent.
  • In Theorem 3.1 the hypothesis "GAS for these systems" covers (3.5) and both approximations, as the proof uses. In Theorem 4.1 the implicit ψi0,ψi∞\psi_{i0},\psi_{i\infty}ψi0​,ψi∞​ are existential C1C^1C1 functions, and αi+1>1\alpha_{i+1}>1αi+1​>1 is kept as printed. (4.4) and (4.9) lack the time-derivative dots.

A formalization in which the closed loop is replaced by its linearization, the homogeneity predicate is weakened to pointwise convergence, or the quantifier on LLL is moved before ϕn\phi_nϕn​ and K1K_1K1​, proves a different statement and does not close the goal. Contributions welcome: a library of weighted dilations and homogeneous functions, the converse Lyapunov theorem, and proofs of the milestones in any order.

Selected references

  • V. Andrieu, L. Praly, A. Astolfi, Homogeneous Approximation, Recursive Observer Design, and Output Feedback, arXiv:0903.0298v1, 2009 (SIAM J. Control Optim. 47(4), 2008). https://arxiv.org/abs/0903.0298
  • L. Rosier, Homogeneous Lyapunov function for homogeneous continuous vector field, Systems & Control Letters 19(6), 1992. https://doi.org/10.1016/0167-6911(92)90078-7
  • A. Bacciotti, L. Rosier, Liapunov Functions and Stability in Control Theory, Springer, 2nd ed., 2005. https://doi.org/10.1007/b139028
12 thms1 active userReviewed
Dynamical SystemsOperations Research·Captain: mikedeng1

Dynamic Instabilities and Stabilization Methods in Distributed Real-Time Scheduling of Manufacturing Systems 1: Clearing Policies Are Unstable on a Re-Entrant Two-Machine Line, Even Without Set-UpsResearch Paper

Motivation

A flexible manufacturing system is a set of machines through which parts of several types travel along fixed routes; each machine serves several buffers and must pay a set-up time whenever it switches from one buffer to another. Real-time scheduling decides, as the system evolves, which buffer each machine works on. Perkins and Kumar (IEEE Trans. Automat. Control 34, 1989) introduced simple distributed policies for this problem, of which the most natural is the clearing policy: a machine keeps working on a buffer until it is empty, and only then switches. They proved that every clear-a-fraction policy, a subclass of clearing policies, keeps every buffer bounded on acyclic systems whenever each machine has spare capacity, and left open whether clear-a-fraction policies stabilize all systems in which material flows around cycles.

Kumar and Seidman (IEEE Trans. Automat. Control 35(3), 1990, doi:10.1109/9.50339) answered no. Their Example 1 is a single part type that visits two machines in the order 1, 2, 2, 1. Every machine has spare capacity, yet under the clearing policy the buffer levels grow without bound, and they do so even when all set-up times are zero, so the instability comes from machines starving each other rather than from time lost to set-ups. Until then, instability had been suspected to require positive set-up times. Shortly afterwards Lu and Kumar exhibited instability of a static buffer-priority rule in a re-entrant network (IEEE Trans. Automat. Control 36, 1991); together these examples started the study of stability of multiclass queueing networks.

Setting

A manufacturing system has part types ppp arriving at rates dp>0d_p > 0dp​>0. Parts of type ppp follow a route of length npn_pnp​: their iii-th operation is at machine μp,i\mu_{p,i}μp,i​, and they wait for it in buffer bp,ib_{p,i}bp,i​, where each part needs processing time τp,i>0\tau_{p,i} > 0τp,i​>0. Machine mmm serves the buffers Bm={b:μb=m}B_m = \{b : \mu_b = m\}Bm​={b:μb​=m}, and switching from bbb to b′b'b′ costs set-up time δb,b′≥0\delta_{b,b'} \ge 0δb,b′​≥0.

Flows are continuous (fluid). The level of buffer bbb at time t≥0t \ge 0t≥0 is xb(t)=xb(0)+ub(t)−yb(t)≥0x_b(t) = x_b(0) + u_b(t) - y_b(t) \ge 0xb​(t)=xb​(0)+ub​(t)−yb​(t)≥0, where yb(t)y_b(t)yb​(t) is its cumulative output and ub(t)u_b(t)ub​(t) its cumulative input: dptd_p tdp​t for the first buffer of a route, and the output of the preceding buffer otherwise. Each machine works in runs: run kkk is a set-up phase of length δβk−1,βk\delta_{\beta_{k-1},\beta_k}δβk−1​,βk​​ followed by a processing phase on buffer βk\beta_kβk​, during which the buffer is drained at rate 1/τb1/\tau_b1/τb​ while it is nonempty and passed through at its inflow rate when it is empty. The system is stable if sup⁡0≤t<∞xb(t)<∞\sup_{0 \le t < \infty} x_b(t) < \inftysup0≤t<∞​xb​(t)<∞ for every buffer.

A clearing policy (Definition 1) is one in which a machine processing bbb continues until the first time that bbb is empty and some other buffer of the same machine is nonempty, and then commences a set-up for one of the nonempty buffers.

Example 1. One part type arrives at rate d=1d = 1d=1 and visits machine 1, machine 2, machine 2 and machine 1; its buffers are 1,2,3,41, 2, 3, 41,2,3,4, so B1={1,4}B_1 = \{1, 4\}B1​={1,4} and B2={2,3}B_2 = \{2, 3\}B2​={2,3}. Processing times are τ1,…,τ4>0\tau_1, \dots, \tau_4 > 0τ1​,…,τ4​>0, and δk\delta_kδk​ is the time to set up to buffer kkk. The parameters satisfy the critical condition and the capacity condition

τ2+τ4>1,τ1+τ4<1,τ2+τ3<1.(3–5)\tau_2 + \tau_4 > 1, \qquad \tau_1 + \tau_4 < 1, \qquad \tau_2 + \tau_3 < 1. \tag{3–5}τ2​+τ4​>1,τ1​+τ4​<1,τ2​+τ3​<1.(3–5)

The initial state is x(0)=(ξ,0,0,0)x(0) = (\xi, 0, 0, 0)x(0)=(ξ,0,0,0), with machine 1 set up for buffer 4 and machine 2 set up for buffer 3. Write

λ=τ41−τ2>1,α=(τ4+1)(δ1+δ2)1−τ2+δ3(τ4+1)+δ4,β=τ4(δ1+δ2)1−τ2+τ4δ3+δ4.\lambda = \frac{\tau_4}{1-\tau_2} > 1, \quad \alpha = \frac{(\tau_4+1)(\delta_1+\delta_2)}{1-\tau_2} + \delta_3(\tau_4+1) + \delta_4, \quad \beta = \frac{\tau_4(\delta_1+\delta_2)}{1-\tau_2} + \tau_4\delta_3 + \delta_4.λ=1−τ2​τ4​​>1,α=1−τ2​(τ4​+1)(δ1​+δ2​)​+δ3​(τ4​+1)+δ4​,β=1−τ2​τ4​(δ1​+δ2​)​+τ4​δ3​+δ4​.

Formalization targets

Goal: Example 1, both cases

Assume (3)–(5).

  1. If δ1,…,δ4>0\delta_1, \dots, \delta_4 > 0δ1​,…,δ4​>0, there is ξ0\xi_0ξ0​ such that for every ξ≥ξ0\xi \ge \xi_0ξ≥ξ0​ (ξ>0\xi > 0ξ>0) a clearing trajectory from (ξ,0,0,0)(\xi, 0, 0, 0)(ξ,0,0,0) exists, and every such trajectory has
sup⁡0≤t<∞x1(t)=+∞.\sup_{0 \le t < \infty} x_1(t) = +\infty.0≤t<∞sup​x1​(t)=+∞.
  1. If δ1=⋯=δ4=0\delta_1 = \dots = \delta_4 = 0δ1​=⋯=δ4​=0, the same holds for every ξ>0\xi > 0ξ>0.

Milestone: the Case 1 cycle map

For ξ\xiξ large enough, every clearing trajectory from (ξ,0,0,0)(\xi, 0, 0, 0)(ξ,0,0,0) reaches, at T1=(λ+τ2/(1−τ2))ξ+αT_1 = (\lambda + \tau_2/(1-\tau_2))\xi + \alphaT1​=(λ+τ2​/(1−τ2​))ξ+α,

x(T1)=(λξ+β,0,0,0),x(T_1) = (\lambda\xi + \beta, 0, 0, 0),x(T1​)=(λξ+β,0,0,0),

with machines 1 and 2 again set up for buffers 4 and 3.

Milestone: the Case 2 magnification

With zero set-up times and any ξ>0\xi > 0ξ>0, every clearing trajectory reaches, at t5=(τ2+τ4)ξ/(1−τ2)t_5 = (\tau_2+\tau_4)\xi/(1-\tau_2)t5​=(τ2​+τ4​)ξ/(1−τ2​),

x(t5)=(λξ,0,0,0),x(t_5) = (\lambda\xi, 0, 0, 0),x(t5​)=(λξ,0,0,0),

with machines 1 and 2 again set up for buffers 4 and 3.

Significance

The example shows that the condition ρm<1\rho_m < 1ρm​<1 on every machine, which is necessary for stability and sufficient for the existence of some stabilizing policy, does not make natural distributed policies stable once material flows around a cycle. The throughput of the line falls to 1/(τ2+τ4)<11/(\tau_2+\tau_4) < 11/(τ2​+τ4​)<1 part per unit time although each machine could handle the demand. This motivates the paper's two positive results: sufficient conditions under which clear-a-fraction policies are stable (Theorem 1), and a supervisory mechanism that stabilizes any policy (Theorem 2), which are the subjects of the other missions of this series. The example is also an early instance of the phenomenon later studied as instability of multiclass fluid networks under work-conserving policies.

The paper's argument is a stage-by-stage computation of piecewise linear trajectories. No machine-checked version of it exists. A formal proof has to make precise what the paper leaves to the reader: that the clearing rule determines the trajectory, that the stage formulas are what that trajectory does, and that the cycle can be restarted. The formal model of runs, set-ups and the clearing rule built here is the same as in the other missions of the series.

Difficulty

The arithmetic of each cycle is routine once the trajectory is known. The difficulty is in the universal quantifier: the claim covers every clearing trajectory, and the clearing rule is defined implicitly, through "the first time thereafter" at which a buffer is empty and another one is nonempty. At several switching instants the buffer a machine switches to is empty and only starts to fill at that instant, and with zero set-up times a machine may begin a run at an instant where the switching condition already holds. Showing that each switch happens exactly when the paper says, and that the fluid levels then follow the printed formulas (including the reduced rate of a machine working on an empty buffer), is a uniqueness argument for a hybrid system, not a simulation. The existence half asks for the converse: an explicit trajectory, defined for all time, with infinitely many runs whose start times tend to infinity.

Formalization scope

  • Time is real (t≥0t \ge 0t≥0); flows are fluid; there are no transport delays or assembly.
  • The system is a general structure (part types Fin P, machines Fin M, buffers ⟨p, i⟩ with i : Fin (n p), paper index iii = Lean index i+1i+1i+1), instantiated as Example 1 with d=1d = 1d=1, route (1,2,2,1)(1,2,2,1)(1,2,2,1), and δb,b′=δb′\delta_{b,b'} = \delta_{b'}δb,b′​=δb′​ for b≠b′b \ne b'b=b′; staying on a buffer costs nothing.
  • A trajectory is a schedule of runs per machine (possibly finitely many, the last lasting forever), with only finitely many run starts in any bounded interval. Processing obeys a rate cap (yby_byb​ grows at most at rate 1/τb1/\tau_b1/τb​, and only while machine μb\mu_bμb​ is in a processing phase of bbb) and runs at full rate while the buffer is nonempty.
  • In Definition 1, a target buffer counts as "nonempty" when it is demanding: positive level, or inflow starting at that instant. The no-early-exit condition is imposed on the open processing interval. Under the literal reading (positive level) or a closed interval, the paper's own trajectories are not clearing, and the goal would hold vacuously; the existence clause in the goal rules out that trivialization.
  • "Set up for buffer bbb at time TTT" means the run in force on (sk,sk+1](s_k, s_{k+1}](sk​,sk+1​].
  • Unboundedness is stated for buffer 1: for every CCC there is t≥0t \ge 0t≥0 with x1(t)>Cx_1(t) > Cx1​(t)>C; "ξ\xiξ large enough" is ∃ξ0,∀ξ≥ξ0\exists \xi_0, \forall \xi \ge \xi_0∃ξ0​,∀ξ≥ξ0​.

Useful contributions include lemmas about fluid trajectories that do not depend on the example (continuity of levels, the pass-through rate on an empty buffer, restarting a trajectory at a run boundary), which also serve the other missions of the series.

Selected references

  • P. R. Kumar and T. I. Seidman, Dynamic instabilities and stabilization methods in distributed real-time scheduling of manufacturing systems, IEEE Trans. Automat. Control 35(3), 289–298, 1990. https://doi.org/10.1109/9.50339
  • J. R. Perkins and P. R. Kumar, Stable, distributed, real-time scheduling of flexible manufacturing/assembly/disassembly systems, IEEE Trans. Automat. Control 34, 139–148, 1989 (reference [18] of the paper).
  • S. H. Lu and P. R. Kumar, Distributed scheduling based on due dates and buffer priorities, IEEE Trans. Automat. Control 36, 1991.
7 thms1 active userReviewed
Convex OptimizationProbability·Captain: mikedeng1

Convex Duality in Constrained Portfolio Optimization I: Financibility, Minimality, Dual Optimality and Parsimony of an Auxiliary Market Are Equivalent and Yield the Optimal Constrained PolicyResearch Paper

Motivation

An investor in a continuous-time market chooses, at every instant, how to split wealth between a bond and ddd stocks and how fast to consume, so as to maximize expected utility of consumption and terminal wealth. Without constraints on the portfolio, this problem was solved by the martingale method: Pliska (1986), Karatzas, Lehoczky and Shreve (1987) and Cox and Huang (1989) showed that in a complete market the optimal terminal wealth and consumption are explicit functions of a single state-price density, and the portfolio that finances them comes from the martingale representation theorem.

Real investors face constraints: no short selling, no borrowing, a cap on the fraction of wealth in a sector, an incomplete market in which some stocks cannot be traded at all. Each makes the market effectively incomplete, and the single state-price density no longer exists. Karatzas, Lehoczky, Shreve and Xu (1991) treated incompleteness, Xu (1990, thesis) short-selling prohibition, and He and Pearson (1991) short-sale constraints in a general setting. Cvitanić and Karatzas (1992) unified these cases: the portfolio must take values in an arbitrary nonempty closed convex set K⊂RdK\subset\mathbb R^dK⊂Rd, and the problem is embedded in a family of auxiliary unconstrained markets indexed by a dual process ν\nuν. This mission formalizes the paper's central characterization, Theorem 10.1, and the results its proof rests on.

Setting

A standard Brownian motion WWW in Rd\mathbb R^dRd lives on a complete probability space with its augmented filtration {Ft}\{\mathcal F_t\}{Ft​}, on a finite horizon [0,T][0,T][0,T]. The market M\mathcal MM has progressively measurable interest rate rrr, appreciation rates bbb and volatility matrix σ\sigmaσ, with r≥−ηr\ge-\etar≥−η, ξ∗σσ∗ξ≥ε∥ξ∥2\xi^*\sigma\sigma^*\xi\ge\varepsilon\|\xi\|^2ξ∗σσ∗ξ≥ε∥ξ∥2, E∫0Tr dt<∞E\int_0^Tr\,dt<\inftyE∫0T​rdt<∞, and a relative risk θ=σ−1(b−r1)\theta=\sigma^{-1}(b-r\mathbf 1)θ=σ−1(b−r1) with E∫0T∥θ∥2dt<∞E\int_0^T\|\theta\|^2dt<\inftyE∫0T​∥θ∥2dt<∞. The state-price density is H0=γ0Z0H_0=\gamma_0Z_0H0​=γ0​Z0​, with γ0(t)=e−∫0tr\gamma_0(t)=e^{-\int_0^tr}γ0​(t)=e−∫0t​r and Z0(t)=exp⁡(−∫0tθ∗dW−12∫0t∥θ∥2)Z_0(t)=\exp(-\int_0^t\theta^*dW-\frac12\int_0^t\|\theta\|^2)Z0​(t)=exp(−∫0t​θ∗dW−21​∫0t​∥θ∥2).

A portfolio π\piπ gives the proportions of wealth in the stocks, a consumption rate is c≥0c\ge0c≥0, and the wealth solves

dX=(rX−c) dt+Xπ∗σ (dW+θ dt),X(0)=x>0.dX=(rX-c)\,dt+X\pi^*\sigma\,(dW+\theta\,dt),\qquad X(0)=x>0.dX=(rX−c)dt+Xπ∗σ(dW+θdt),X(0)=x>0.

The pair is admissible, (π,c)∈A0(x)(\pi,c)\in\mathcal A_0(x)(π,c)∈A0​(x), if X≥0X\ge0X≥0. With utility functions U1(t,⋅)U_1(t,\cdot)U1​(t,⋅), U2U_2U2​ (strictly increasing, strictly concave, C1C^1C1, U′(0+)=∞U'(0+)=\inftyU′(0+)=∞, U′(∞)=0U'(\infty)=0U′(∞)=0), inverse marginals I1,I2I_1,I_2I1​,I2​ and conjugates U~1,U~2\tilde U_1,\tilde U_2U~1​,U~2​, the investor maximizes

J(x;π,c)=E∫0TU1(t,c(t)) dt+EU2(X(T))J(x;\pi,c)=E\int_0^TU_1(t,c(t))\,dt+EU_2(X(T))J(x;π,c)=E∫0T​U1​(t,c(t))dt+EU2​(X(T))

over A′(x)\mathcal A'(x)A′(x), the admissible pairs with integrable utility losses and π∈K\pi\in Kπ∈K almost everywhere. The value is V(x)V(x)V(x).

The support function δ(ν)=sup⁡π∈K(−π∗ν)∈R∪{+∞}\delta(\nu)=\sup_{\pi\in K}(-\pi^*\nu)\in\mathbb R\cup\{+\infty\}δ(ν)=supπ∈K​(−π∗ν)∈R∪{+∞} drives the duality. For ν\nuν in the class D\mathcal DD of square-integrable processes with E∫0Tδ(ν) dt<∞E\int_0^T\delta(\nu)\,dt<\inftyE∫0T​δ(ν)dt<∞, the auxiliary market Mν\mathcal M_\nuMν​ has interest rate r+δ(ν)r+\delta(\nu)r+δ(ν) and appreciation rates b+ν+δ(ν)1b+\nu+\delta(\nu)\mathbf 1b+ν+δ(ν)1, state-price density HνH_\nuHν​ and unconstrained value Vν(x)V_\nu(x)Vν​(x). For λ\lambdaλ in the subclass D′\mathcal D'D′ (where Xλ(y)=E[∫0THλI1(t,yHλ) dt+Hλ(T)I2(yHλ(T))]\mathcal X_\lambda(y)=E[\int_0^TH_\lambda I_1(t,yH_\lambda)\,dt+H_\lambda(T)I_2(yH_\lambda(T))]Xλ​(y)=E[∫0T​Hλ​I1​(t,yHλ​)dt+Hλ​(T)I2​(yHλ​(T))] is finite for all y>0y>0y>0), let y=Yλ(x)y=\mathcal Y_\lambda(x)y=Yλ​(x) solve Xλ(y)=x\mathcal X_\lambda(y)=xXλ​(y)=x. The Mλ\mathcal M_\lambdaMλ​-optimal consumption, terminal wealth and wealth are cλ=I1(t,yHλ)c_\lambda=I_1(t,yH_\lambda)cλ​=I1​(t,yHλ​), ξλ=I2(yHλ(T))\xi_\lambda=I_2(yH_\lambda(T))ξλ​=I2​(yHλ​(T)) and XλX_\lambdaXλ​.

Formalization targets

Goal: Theorem 10.1

Five conditions are considered for a capital x>0x>0x>0: (A) optimality of a pair (π^,c^)∈A′(x)(\hat\pi,\hat c)\in\mathcal A'(x)(π^,c^)∈A′(x), together with E[∫0Tc^U1′(t,c^) dt+X^(T)U2′(X^(T))]<∞E[\int_0^T\hat cU_1'(t,\hat c)\,dt+\hat X(T)U_2'(\hat X(T))]<\inftyE[∫0T​c^U1′​(t,c^)dt+X^(T)U2′​(X^(T))]<∞; and, for λ∈D′\lambda\in\mathcal D'λ∈D′, (B) financibility: some π^λ∈K\hat\pi_\lambda\in Kπ^λ​∈K with δ(λ)+π^λ∗λ=0\delta(\lambda)+\hat\pi_\lambda^*\lambda=0δ(λ)+π^λ∗​λ=0 finances (cλ,ξλ)(c_\lambda,\xi_\lambda)(cλ​,ξλ​) in M\mathcal MM; (C) minimality: Vλ(x)≤Vν(x)V_\lambda(x)\le V_\nu(x)Vλ​(x)≤Vν​(x) for all ν∈D\nu\in\mathcal Dν∈D, with Vλ(x)V_\lambda(x)Vλ​(x) attained by (cλ,ξλ)(c_\lambda,\xi_\lambda)(cλ​,ξλ​); (D) dual optimality: λ\lambdaλ minimizes ν↦E[∫0TU~1(t,yHν) dt+U~2(yHν(T))]\nu\mapsto E[\int_0^T\tilde U_1(t,yH_\nu)\,dt+\tilde U_2(yH_\nu(T))]ν↦E[∫0T​U~1​(t,yHν​)dt+U~2​(yHν​(T))] over D\mathcal DD; (E) parsimony: E[∫0THνcλ dt+Hν(T)ξλ]≤xE[\int_0^TH_\nu c_\lambda\,dt+H_\nu(T)\xi_\lambda]\le xE[∫0T​Hν​cλ​dt+Hν​(T)ξλ​]≤x for all ν∈D\nu\in\mathcal Dν∈D. The theorem asserts

(B)  ⟺  (C)  ⟺  (D)  ⟺  (E)  ⟹  (A) for (π^λ,cλ),(\mathrm B)\iff(\mathrm C)\iff(\mathrm D)\iff(\mathrm E)\implies(\mathrm A)\text{ for }(\hat\pi_\lambda,c_\lambda),(B)⟺(C)⟺(D)⟺(E)⟹(A) for (π^λ​,cλ​),

and, under the conditions (5.8) (c↦cU′(c)c\mapsto cU'(c)c↦cU′(c) nondecreasing), (8.25) (a uniform growth bound αU′(x)≥U′(γx)\alpha U'(x)\ge U'(\gamma x)αU′(x)≥U′(γx)) and (12.2) (finiteness of the dual functional for some ν\nuν), that (A) implies (B)–(E) for some λ∈D′\lambda\in\mathcal D'λ∈D′ with π^λ=π^\hat\pi_\lambda=\hat\piπ^λ​=π^.

Milestones

In proof order: the conjugate inequalities (5.5)–(5.6); the budget constraint (3.6); the unconstrained solution (Lemma 7.2, Proposition 7.3, Theorem 7.4); the embedding (8.14), V(x)≤Vν(x)V(x)\le V_\nu(x)V(x)≤Vν​(x); Proposition 8.3, (B) ⇒\Rightarrow⇒ optimality; the constrained budget constraint (9.4) and the hedging Theorem 9.1, (E) ⇒\Rightarrow⇒ (B); and Appendix A's Lemma A.1, Lemma A.2 and Proposition A.4, the steps of (A) ⇒\Rightarrow⇒ (B).

Significance

Theorem 10.1 turns a constrained stochastic control problem into a family of unconstrained ones plus a minimization over the dual process λ\lambdaλ. Condition (D) is a convex minimization over ν\nuν that does not involve portfolios. It leads to the dual problem of Section 12, whose solvability gives existence of optimal constrained policies (Theorem 13.1, the companion mission). Section 9's hedging theorem is the superreplication principle under convex constraints, the basis of later work on constrained hedging and pricing. With logarithmic utility the theorem gives explicit optimal policies, and for deterministic coefficients the dual problem reduces to a deterministic one.

These results are proved in the paper, which is the standard reference for the convex-duality approach. To our knowledge they have no machine-checked proof. The mission's statements also produce a reusable Lean model of a continuous-time Itô market with stochastic coefficients, admissible portfolios, consumption, utility functions and their conjugates.

Difficulty

The obvious argument, Lagrange duality for max⁡J\max JmaxJ subject to π∈K\pi\in Kπ∈K, fails at the first step: the constraint is on the control process, not on terminal wealth, so there is no static budget constraint to dualize. The paper's dual variable is a process, and for each candidate it changes both the interest rate and the drift. The direction (B) ⇒\Rightarrow⇒ (A) is relatively direct. The direction (E) ⇒\Rightarrow⇒ (B) is a hedging theorem under convex constraints (Theorem 9.1), where financing a claim requires the portfolio to land in KKK and to satisfy the complementary-slackness identity δ(λ)+π∗λ=0\delta(\lambda)+\pi^*\lambda=0δ(λ)+π∗λ=0, which no unconstrained construction delivers. The converse (A) ⇒\Rightarrow⇒ (B) must produce the dual process λ\lambdaλ from an optimal pair that is only assumed to exist, and the paper needs the extra growth conditions (5.8), (8.25), (12.2) for it. In Lean, stochastic integrals for integrands that are square integrable only almost surely, the martingale representation theorem for the Brownian filtration, and the product rule for Itô processes are not available in Mathlib.

Formalization scope

Processes are functions R≥0×Ω→E\mathbb R_{\ge0}\times\Omega\to ER≥0​×Ω→E constrained on [0,T][0,T][0,T]; vectors are EuclideanSpace ℝ (Fin d), matrices Matrix (Fin d) (Fin d) ℝ. The Brownian motion is the published EthierKurtz.IsStandardBrownian. The stochastic integral is a binder I, constrained only by the standing hypothesis that it is an Itô integral for every locally square-integrable integrand. This almost-sure variant of EthierKurtz.HasBrownianItoIntegral uses the published dyadic step helpers itoStepValue and itoStepSum. The hypothesis is satisfiable, by the classical construction.

The following readings are explicit in the Lean.

  • Triples. Classes are sets of triples (π,c,X)(\pi,c,X)(π,c,X) with XXX a wealth process: an adapted, a.s. continuous solution of the integral form of (3.1) or (8.10) whose integrals exist.
  • Extended reals. Utility expectations are extended reals, computed as positive minus negative parts. A utility at 000 is U(0+)U(0+)U(0+). Suprema are taken in the extended reals.
  • The support function. δ\deltaδ takes values in the extended reals. In drifts and discounts it enters as a real number, which is finite almost everywhere for ν∈D\nu\in\mathcal Dν∈D. The page's "E∫δ(ν)≤∞E\int\delta(\nu)\le\inftyE∫δ(ν)≤∞" in (8.1) is read as "<∞<\infty<∞".
  • Inverse marginals and conjugates. I(y)=inf⁡{x>0:U′(x)≤y}I(y)=\inf\{x>0: U'(x)\le y\}I(y)=inf{x>0:U′(x)≤y} and U~(y)=sup⁡x>0(U(x)−xy)\tilde U(y)=\sup_{x>0}(U(x)-xy)U~(y)=supx>0​(U(x)−xy).
  • The inverse Yλ\mathcal Y_\lambdaYλ​. Yλ(x)\mathcal Y_\lambda(x)Yλ​(x) and Y0(x)\mathcal Y_0(x)Y0​(x) are represented by a number y>0y>0y>0 with X(y)=x\mathcal X(y)=xX(y)=x.
  • The wealth XλX_\lambdaXλ​. It is a conditional expectation for each ttt. "X=XλX=X_\lambdaX=Xλ​" means X(t)=Xλ(t)X(t)=X_\lambda(t)X(t)=Xλ​(t) a.s. for every t≤Tt\le Tt≤T.
  • Almost everywhere. "ℓ⊗P\ell\otimes Pℓ⊗P-a.e." is almost everywhere for the product of Lebesgue measure on [0,T][0,T][0,T] and PPP. "a.s." is almost everywhere for PPP.
  • Theorem 9.1. Its conclusion "(π,c)∈A′(x)(\pi,c)\in\mathcal A'(x)(π,c)∈A′(x)" is formalized as A0(x)\mathcal A_0(x)A0​(x) plus π∈K\pi\in Kπ∈K a.e., since the theorem involves no utility.
  • (4.4). By Remark 4.2 it is not used in Theorem 10.1. It is kept, because the paper assumes it throughout.

A formalization in which the stochastic integral is an unconstrained function, a wealth process need not solve its equation, or δ\deltaδ is real-valued (so that sup⁡\supsup over an unbounded set returns 000) would make the statements vacuous or false; each is ruled out above. The proof will need almost-surely-square-integrable Itô integrals, Itô's product rule, nonnegative local martingales as supermartingales, and martingale representation in the Brownian filtration. All of these are reusable well beyond this mission, and contributions of any of them are welcome.

Selected references

  • J. Cvitanić and I. Karatzas, Convex duality in constrained portfolio optimization, Ann. Appl. Probab. 2(4) (1992) 767–818. https://doi.org/10.1214/aoap/1177005576
  • I. Karatzas, J. P. Lehoczky and S. E. Shreve, Optimal portfolio and consumption decisions for a "small investor" on a finite horizon, SIAM J. Control Optim. 25 (1987) 1557–1586. https://doi.org/10.1137/0325086
  • J. C. Cox and C. F. Huang, Optimal consumption and portfolio policies when asset prices follow a diffusion process, J. Econom. Theory 49 (1989) 33–83. https://doi.org/10.1016/0022-0531(89)90067-7
  • S. R. Pliska, A stochastic calculus model of continuous trading: optimal portfolios, Math. Oper. Res. 11 (1986) 371–382. https://doi.org/10.1287/moor.11.2.371
  • I. Karatzas, J. P. Lehoczky, S. E. Shreve and G. L. Xu, Martingale and duality methods for utility maximization in an incomplete market, SIAM J. Control Optim. 29 (1991) 702–730. https://doi.org/10.1137/0329039
  • H. He and N. D. Pearson, Consumption and portfolio policies with incomplete markets and short-sale constraints: the infinite-dimensional case, J. Econom. Theory 54 (1991) 259–304. https://doi.org/10.1016/0022-0531(91)90123-L
  • I. Karatzas and S. E. Shreve, Brownian Motion and Stochastic Calculus, Springer, 1988. https://doi.org/10.1007/978-1-4684-0302-2
21 thms1 active userReviewed
Operations ResearchProbabilityStochastic Systems·Captain: mikedeng1

Scheduling a Multi Class Queue with Many Exponential Servers: Asymptotic Optimality in Heavy Traffic: The HJB-Based Preemptive Policy Is Asymptotically Optimal Among Work-Conserving PoliciesResearch Paper

Motivation

Large call centers route several types of customers to a common pool of agents. When the pool is large and highly utilized, the relevant asymptotic regime is the quality-and-efficiency-driven (QED) or Halfin–Whitt regime (Halfin & Whitt 1981). The number of servers nnn grows while the offered load stays within O(n)O(\sqrt n)O(n​) of nnn. Waiting is then neither negligible nor overwhelming (Gans, Koole & Mandelbaum 2003).

Which class should a freed agent serve next? Exact optimization of a multi-class many-server queue with abandonment is intractable. The standard route is to solve a limiting diffusion control problem and translate its optimal control back into a policy for the queue. Atar, Mandelbaum and Reiman (Ann. Appl. Probab. 2004) carried this out for kkk customer classes, exponential service and abandonment, general renewal arrivals and general convex-type holding costs. They proved that the translated policy is asymptotically optimal. This mission formalizes that result for the preemptive policy.

Context:

  • Harrison & Zeevi (2004) studied the same multi-class many-server problem.
  • Bell & Williams (2001) proved asymptotic optimality of a threshold policy for a two-server system in conventional heavy traffic.
  • The present paper is the first to cover the QED regime with general costs and abandonment.

Setting

There are k≥1k\ge1k≥1 customer classes and nnn identical servers.

Primitives.

  • Arrivals. Class-iii customers arrive according to a renewal process AinA^n_iAin​ with interarrival times Uˇi(j)/λin\check U_i(j)/\lambda^n_iUˇi​(j)/λin​. Here the Uˇi(j)\check U_i(j)Uˇi​(j) are i.i.d., positive, of mean one and squared coefficient of variation CU,i2C^2_{U,i}CU,i2​.
  • Service. Service times are exponential with rate μin\mu^n_iμin​, represented by Poisson processes SinS^n_iSin​.
  • Abandonment. Waiting customers abandon at rate θin≥0\theta^n_i\ge0θin​≥0, represented by Poisson processes RinR^n_iRin​.

State. Xin(t)X^n_i(t)Xin​(t) is the number of class-iii customers in the system, Ψin(t)\Psi^n_i(t)Ψin​(t) the number in service and Φin=Xin−Ψin\Phi^n_i=X^n_i-\Psi^n_iΦin​=Xin​−Ψin​ the number waiting. The dynamics are

Xin(t)=Xi0,n+Ain(t)−Rin(∫0tΦin)−Sin(∫0tΨin),Ψn,Φn∈Z+k,∑iΨin≤n.X^n_i(t)=X^{0,n}_i+A^n_i(t)-R^n_i\Big(\int_0^t\Phi^n_i\Big)-S^n_i\Big(\int_0^t\Psi^n_i\Big),\qquad \Psi^n,\Phi^n\in\mathbb Z^k_+,\quad \textstyle\sum_i\Psi^n_i\le n .Xin​(t)=Xi0,n​+Ain​(t)−Rin​(∫0t​Φin​)−Sin​(∫0t​Ψin​),Ψn,Φn∈Z+k​,∑i​Ψin​≤n.

Policies.

  • A scheduling control policy (SCP) is the process Ψn\Psi^nΨn.
  • It is admissible if it does not anticipate the future beyond the time of the next arrival: past information is independent of future primitive increments.
  • It is work-conserving if no server idles while customers wait: (1⋅Xn−n)+=1⋅Φn(\mathbb 1\cdot X^n-n)^+=\mathbb 1\cdot\Phi^n(1⋅Xn−n)+=1⋅Φn.

Scaling and cost. In the QED scaling n−1λin→λin^{-1}\lambda^n_i\to\lambda_in−1λin​→λi​ with ∑iλi/μi=1\sum_i\lambda_i/\mu_i=1∑i​λi​/μi​=1. With ρi=λi/μi\rho_i=\lambda_i/\mu_iρi​=λi​/μi​ the centred processes are X^n=n−1/2(Xn−ρn)\hat X^n=n^{-1/2}(X^n-\rho n)X^n=n−1/2(Xn−ρn), Φ^n=n−1/2Φn\hat\Phi^n=n^{-1/2}\Phi^nΦ^n=n−1/2Φn and Ψ^n=n−1/2(Ψn−ρn)\hat\Psi^n=n^{-1/2}(\Psi^n-\rho n)Ψ^n=n−1/2(Ψn−ρn). The cost is

Cn=E∫0∞e−γtL~(Φ^n(t),Ψ^n(t)) dt.C^n=E\int_0^\infty e^{-\gamma t}\tilde L(\hat\Phi^n(t),\hat\Psi^n(t))\,dt .Cn=E∫0∞​e−γtL~(Φ^n(t),Ψ^n(t))dt.

The limiting control problem. It controls

X(t)=x+rW(t)+∫0tb(X(s),u(s)) ds,b(x,u)=ℓ+(μ−θ)(1⋅x)+u−μx,X(t)=x+rW(t)+\int_0^t b(X(s),u(s))\,ds,\qquad b(x,u)=\ell+(\mu-\theta)(\mathbb 1\cdot x)^+u-\mu x,X(t)=x+rW(t)+∫0t​b(X(s),u(s))ds,b(x,u)=ℓ+(μ−θ)(1⋅x)+u−μx,

where the control uuu takes values in the simplex Sk\mathbb S^kSk and WWW is a kkk-dimensional Brownian motion. The data are ri=(λiCU,i2+λi)1/2r_i=(\lambda_iC^2_{U,i}+\lambda_i)^{1/2}ri​=(λi​CU,i2​+λi​)1/2 and ℓi=λ^i−ρiμ^i\ell_i=\hat\lambda_i-\rho_i\hat\mu_iℓi​=λ^i​−ρi​μ^​i​. Its value V(x)V(x)V(x) is the infimum of E∫0∞e−γtL(X,u) dtE\int_0^\infty e^{-\gamma t}L(X,u)\,dtE∫0∞​e−γtL(X,u)dt, with L(x,u)=L~((1⋅x)+u,x−(1⋅x)+u)L(x,u)=\tilde L((\mathbb 1\cdot x)^+u,x-(\mathbb 1\cdot x)^+u)L(x,u)=L~((1⋅x)+u,x−(1⋅x)+u).

HJB equation and the proposed policy. The HJB equation is 12∑iri2∂iif+H(x,Df)−γf=0\tfrac12\sum_ir_i^2\partial_{ii}f+H(x,Df)-\gamma f=021​∑i​ri2​∂ii​f+H(x,Df)−γf=0 with H(x,p)=inf⁡u∈Sk[b(x,u)⋅p+L(x,u)]H(x,p)=\inf_{u\in\mathbb S^k}[b(x,u)\cdot p+L(x,u)]H(x,p)=infu∈Sk​[b(x,u)⋅p+L(x,u)]. Let hhh be a measurable selection of its minimizers. The proposed preemptive policy (P-SCP) sets the queue vector to Θ[(1⋅Xn−n)+h(X^n)]\Theta[(\mathbb 1\cdot X^n-n)^+h(\hat X^n)]Θ[(1⋅Xn−n)+h(X^n)], an integer rounding, and falls back to a static priority rule when that is infeasible.

Formalization targets

Goal: Theorem 2(i)

For a Cpol2C^2_{\mathrm{pol}}Cpol2​ solution fff of the HJB equation, a measurable minimizer selection hhh, and initial states with X^0,n→x\hat X^{0,n}\to xX^0,n→x:

lim⁡n→∞E∫0∞e−γtL~(Φ^tn,∗,Ψ^tn,∗) dt  ≤  lim inf⁡n→∞E∫0∞e−γtL~(Φ^tn,Ψ^tn) dt\lim_{n\to\infty}E\int_0^\infty e^{-\gamma t}\tilde L(\hat\Phi^{n,*}_t,\hat\Psi^{n,*}_t)\,dt\;\le\;\liminf_{n\to\infty}E\int_0^\infty e^{-\gamma t}\tilde L(\hat\Phi^n_t,\hat\Psi^n_t)\,dtn→∞lim​E∫0∞​e−γtL~(Φ^tn,∗​,Ψ^tn,∗​)dt≤n→∞liminf​E∫0∞​e−γtL~(Φ^tn​,Ψ^tn​)dt

This holds for every sequence of work-conserving admissible SCPs, and the left-hand limit exists and is finite. No constants are hard-coded.

Milestones

The milestones follow the proof:

  • on the diffusion side, Proposition 2 (well-posedness), Proposition 4 (stability and moment bounds), Proposition 5(i)–(ii) (growth and continuity of VVV) and Theorem 3 (VVV is the unique Cpol2C^2_{\mathrm{pol}}Cpol2​ HJB solution, and an optimal Markov policy exists);
  • on the queueing side, Proposition 1 (feedback rules give admissible SCPs), Lemmas 2–3 (moment bounds), Lemma 4(i)–(ii) (FCLT for the primitives and the fluid limit (Ψˉn,Φˉn)⇒(ρ,0)(\bar\Psi^n,\bar\Phi^n)\Rightarrow(\rho,0)(Ψˉn,Φˉn)⇒(ρ,0)), and Theorem 4(i)–(ii): lim inf⁡≥V(x)\liminf\ge V(x)liminf≥V(x) always, and lim sup⁡≤V(x)\limsup\le V(x)limsup≤V(x) under condition (49).

Significance

The result. Theorem 2(i) justifies using the diffusion control problem as a design tool for multi-class many-server systems. The policy is explicit given hhh, and it is optimal in the limit against all non-anticipating work-conserving policies, including those that use the full history and the time of the next arrival. The proof also identifies the limit cost with V(x)V(x)V(x).

Formalizing it. The result is proved on paper, with some steps (Proposition 1, the principle of optimality, the time-change and martingale limit theorems) given as sketches or citations. No part of it is machine-checked. A formalization requires:

  • a counting-process model of the queue;
  • a careful definition of non-anticipation;
  • a pathwise controlled SDE;
  • classical solvability of a semilinear elliptic HJB equation on Rk\mathbb R^kRk;
  • a weak-convergence argument in Skorokhod space.

Each of these is reusable well beyond this paper.

Difficulty

The obvious argument would show that X^n\hat X^nX^n converges to the controlled diffusion and pass the costs to the limit. This fails for two reasons:

  • the comparison class contains arbitrary non-Markov, history-dependent policies, so the queue does not converge to a single controlled diffusion;
  • the optimal selector hhh is in general discontinuous (for linear costs it is), so the proposed policy is not a continuous function of the state.

The proof instead compares every policy with the HJB solution through Itô's formula on the prelimit processes. This needs:

  • uniform moment bounds;
  • tightness of the integral processes;
  • the convergence of stochastic integrals of Kurtz and Protter;
  • and, for the proposed policy, the fact that the rounding Θ\ThetaΘ and the priority fallback perturb the minimizer by O(n−1/2)O(n^{-1/2})O(n−1/2).

Existence of a classical HJB solution on all of Rk\mathbb R^kRk, with only Hölder-continuous costs and polynomial growth, rests on a bounded-domain existence theorem for fully nonlinear elliptic equations.

Formalization scope

The Lean development commits to the following conventions.

  • Indexing and norms. Classes are Fin k with k≥1k\ge1k≥1; paper class iii is index i−1i-1i−1, so "class kkk" (highest priority, rounding remainder of Θ\ThetaΘ) is the last index. Vectors are Fin k → ℝ and ∥⋅∥\|\cdot\|∥⋅∥ is the paper's ℓ1\ell^1ℓ1 norm; the paper's ∣⋅∣|\cdot|∣⋅∣ on vectors is read the same way.
  • Probability space and paths. All systems share one complete probability space. Time is real and every condition is for t≥0t\ge0t≥0. The paper's "without loss" path regularity (finite arrival counts, Poisson paths Z+\mathbb Z_+Z+​-valued, nondecreasing and càdlàg) holds for every ω\omegaω.
  • Poisson processes are defined by independent Poisson increments; rate 000 gives the zero process.
  • Policies. A policy is a pair of real processes (Ψn,Xn)(\Psi^n,X^n)(Ψn,Xn) with integer values. Admissibility is Definition 2 verbatim, with the future σ\sigmaσ-field built from the next arrival time τin(t)\tau^n_i(t)τin​(t). Work conservation is (18).
  • Costs and value are lower Lebesgue integrals in [0,∞][0,\infty][0,∞], and lim⁡\limlim/lim inf⁡\liminfliminf are taken there. The integrands are nonnegative under work conservation.
  • Admissible systems range over sample spaces Ω : Type (universe 0). "Complete filtered probability space" means PPP complete with all null sets in F0\mathcal F_0F0​. Brownian motion is Mathlib's IsBrownianReal per coordinate, with independence and the (Ft)(\mathcal F_t)(Ft​)-Brownian property stated explicitly. VVV is the infimum over systems and their controlled processes.
  • Discount rate. γ>0\gamma>0γ>0 is a hypothesis; the paper leaves it implicit.
  • Initial states are integer vectors X0,n∈Z+kX^{0,n}\in\mathbb Z^k_+X0,n∈Z+k​ with n−1/2(X0,n−ρn)→xn^{-1/2}(X^{0,n}-\rho n)\to xn−1/2(X0,n−ρn)→x. The literal "X^0,n∈n−1/2Zk\hat X^{0,n}\in n^{-1/2}\mathbb Z^kX^0,n∈n−1/2Zk" would require ρin∈Z\rho_in\in\mathbb Zρi​n∈Z. Assumption 1(ii) is not imposed: each policy chooses its own initial split.
  • Lemma 3 is stated for all nnn beyond a threshold that depends on the sequence, with constants c,mˉc,\bar mc,mˉ chosen before xxx and the sequence. The printed all-nnn bound with ccc independent of xxx fails when the early terms X^0,n\hat X^{0,n}X^0,n are large.
  • Weak convergence to a continuous limit uses the coupling form CouplingConverges of the published BellWilliams2001.ThresholdPolicy.Paths; convergence to a deterministic limit is UocInProb.

The goal hypothesizes fff and hhh with the pointwise identity b(x,h(x))⋅Df(x)+L(x,h(x))=H(x,Df(x))b(x,h(x))\cdot Df(x)+L(x,h(x))=H(x,Df(x))b(x,h(x))⋅Df(x)+L(x,h(x))=H(x,Df(x)) for all xxx. An arbitrary "optimal Markov control policy" may differ from a minimizer selection on the Lebesgue-null lattice where X^n\hat X^nX^n lives, and that formalization would make the goal false. Restricting the comparators to feedback, Markov or nonpreemptive policies, fixing kkk, dropping abandonment, specializing to Poisson arrivals or linear costs, or imposing a common initial split would each trivialize or weaken the statement and is ruled out.

Not formalized:

  • Lemma 4(iii) (tightness);
  • Lemma 5 (Kurtz–Protter, which needs semimartingale theory absent from Mathlib);
  • Lemma 6 (convergence of Stieltjes integrals at limit points);
  • Proposition 5(iii);
  • the nonpreemptive results, Theorem 2(ii)–(iii).

Contributions are welcome on any milestone, and especially on infrastructure: Poisson and renewal processes, functional central limit theorems in Skorokhod space, classical solvability of elliptic HJB equations, and measurable selection of minimizers.

Selected references

  • R. Atar, A. Mandelbaum, M. I. Reiman, Scheduling a multi class queue with many exponential servers: asymptotic optimality in heavy traffic, Ann. Appl. Probab. 14(3), 2004. https://arxiv.org/abs/math/0407058
  • S. Halfin, W. Whitt, Heavy-traffic limits for queues with many exponential servers, Oper. Res. 29(3), 1981. https://doi.org/10.1287/opre.29.3.567
  • N. Gans, G. Koole, A. Mandelbaum, Telephone call centers: tutorial, review, and research prospects, Manuf. Serv. Oper. Manag. 5(2), 2003. https://doi.org/10.1287/msom.5.2.79.16071
  • J. M. Harrison, A. Zeevi, Dynamic scheduling of a multiclass queue in the Halfin–Whitt heavy traffic regime, Oper. Res. 52(2), 2004. https://doi.org/10.1287/opre.1040.0109
  • S. L. Bell, R. J. Williams, Dynamic scheduling of a system with two parallel servers in heavy traffic with resource pooling: asymptotic optimality of a threshold policy, Ann. Appl. Probab. 11(3), 2001. https://doi.org/10.1214/aoap/1015345343
  • T. G. Kurtz, P. Protter, Weak limit theorems for stochastic integrals and stochastic differential equations, Ann. Probab. 19(3), 1991. https://doi.org/10.1214/aop/1176990334
16 thms1 active userReviewed
Dynamic ProgrammingProbability·Captain: mikedeng1

Discrete-Time Controlled Markov Processes with Average Cost Criterion: A Survey 6: A Canonical Policy Is Strong Average Optimal and Its Average Cost Is the Optimal OneResearch Paper

Motivation

A controlled Markov process (CMP), or Markov decision process, models a system that moves randomly between states while a controller chooses actions that influence both the cost incurred and the next state. When the planning horizon is long and no discounting is natural (queueing control, inventory, communication networks, maintenance), the criterion of interest is the long-run average cost. Average-cost problems are harder than discounted ones: the dynamic programming operator is no longer a contraction, and an optimal policy need not exist without structure.

The survey of Arapostathis, Borkar, Fernández-Gaucherand, Ghosh and Marcus (SIAM J. Control Optim. 31 (1993)) organizes the theory by state space. For Borel state spaces and bounded costs, §6.1 follows Dynkin and Yushkevich and works with canonical triplets: a pair of bounded functions and a policy that make the policy optimal for every finite horizon with a fixed terminal cost. Theorem 6.3 is the statement that explains why this notion is the right one: a canonical triplet solves the average-cost problem.

Timeline. The notion of a canonical policy was introduced by Yushkevich (1973) and developed in Dynkin and Yushkevich's Controlled Markov Processes (1979, Chap. 7), which contains the substance of Theorem 6.3 (i)–(iii). Mandl (1974) introduced the discrepancy function that bears his name. The almost-sure statements (v)–(vi) are due to Georgin (1978). Coupled optimality equations for the multichain case go back to Howard (1960).

Setting

The model is a five-tuple (S,A,U,P,c)(S, A, U, P, c)(S,A,U,P,c). The state space SSS and the action space AAA are Borel spaces. For each state xxx, U(x)⊆AU(x)\subseteq AU(x)⊆A is a nonempty compact set of admissible actions, and K={(x,a):a∈U(x)}K = \{(x,a): a\in U(x)\}K={(x,a):a∈U(x)} is measurable. P(dy∣x,a)P(dy\mid x,a)P(dy∣x,a) is a transition kernel and ccc a measurable one-stage cost, nonnegative on KKK.

A policy π=(πt)\pi = (\pi_t)π=(πt​) chooses the action at time ttt at random from a kernel πt(da∣ht)\pi_t(da\mid h_t)πt​(da∣ht​) that may depend on the whole history ht=(x0,a0,…,xt)h_t = (x_0,a_0,\dots,x_t)ht​=(x0​,a0​,…,xt​) and charges only U(xt)U(x_t)U(xt​). The class of all such policies is Π\PiΠ. Each policy and initial state xxx determine a law Pxπ\mathcal P^\pi_xPxπ​ of the state–action process (Xt,At)(X_t, A_t)(Xt​,At​), with expectation ExπE^\pi_xExπ​.

For a horizon NNN and a terminal cost hhh,

JN(x,π,h)=Exπ[∑t=0N−1c(Xt,At)+h(XN)],JN(x,π)=JN(x,π,0),JN∗(x,h)=inf⁡π∈ΠJN(x,π,h).J_N(x,\pi,h) = E^\pi_x\Big[\sum_{t=0}^{N-1}c(X_t,A_t) + h(X_N)\Big],\qquad J_N(x,\pi)=J_N(x,\pi,0),\qquad J^*_N(x,h)=\inf_{\pi\in\Pi}J_N(x,\pi,h).JN​(x,π,h)=Exπ​[t=0∑N−1​c(Xt​,At​)+h(XN​)],JN​(x,π)=JN​(x,π,0),JN∗​(x,h)=π∈Πinf​JN​(x,π,h).

The average cost is J(x,π)=lim sup⁡N1NJN(x,π)J(x,\pi)=\limsup_N \frac1N J_N(x,\pi)J(x,π)=limsupN​N1​JN​(x,π) and the optimal average cost is J∗(x)=inf⁡π∈ΠJ(x,π)J^*(x)=\inf_{\pi\in\Pi}J(x,\pi)J∗(x)=infπ∈Π​J(x,π). The span of a bounded function is span⁡(h)=sup⁡h−inf⁡h\operatorname{span}(h)=\sup h-\inf hspan(h)=suph−infh.

With Mb(S)\mathcal M_b(S)Mb​(S) the bounded measurable functions, a triplet (ρ,h,π∗)(\rho,h,\pi^*)(ρ,h,π∗) with ρ,h∈Mb(S)\rho,h\in\mathcal M_b(S)ρ,h∈Mb​(S) and π∗∈Π\pi^*\in\Piπ∗∈Π is canonical if

JN(x,π∗,h)=JN∗(x,h)=h(x)+Nρ(x)∀N∈N0, x∈S.(6.4)J_N(x,\pi^*,h) = J^*_N(x,h) = h(x)+N\rho(x)\qquad\forall N\in\mathbb N_0,\ x\in S. \tag{6.4}JN​(x,π∗,h)=JN∗​(x,h)=h(x)+Nρ(x)∀N∈N0​, x∈S.(6.4)

A policy π∗\pi^*π∗ is strong average optimal if

lim sup⁡N→∞1NJN(x,π∗)≤lim inf⁡N→∞1NJN(x,π)∀x∈S, π∈Π.(6.5)\limsup_{N\to\infty}\frac1N J_N(x,\pi^*)\le\liminf_{N\to\infty}\frac1N J_N(x,\pi)\qquad\forall x\in S,\ \pi\in\Pi. \tag{6.5}N→∞limsup​N1​JN​(x,π∗)≤N→∞liminf​N1​JN​(x,π)∀x∈S, π∈Π.(6.5)

Formalization targets

Goal: Theorem 6.3 (i)–(iii)

Let (ρ,h,π∗)(\rho,h,\pi^*)(ρ,h,π∗) be a canonical triplet and let ccc be bounded on KKK. Then for each x∈Sx\in Sx∈S:

(i)JN(x,π∗)≤JN(x,π)+span⁡(h)∀N, ∀π∈Π;\text{(i)}\quad J_N(x,\pi^*)\le J_N(x,\pi)+\operatorname{span}(h)\quad\forall N,\ \forall\pi\in\Pi;(i)JN​(x,π∗)≤JN​(x,π)+span(h)∀N, ∀π∈Π; (ii)π∗ is strong average optimal;(iii)J(x,π∗)=J∗(x)=ρ(x).\text{(ii)}\quad \pi^* \text{ is strong average optimal};\qquad \text{(iii)}\quad J(x,\pi^*)=J^*(x)=\rho(x).(ii)π∗ is strong average optimal;(iii)J(x,π∗)=J∗(x)=ρ(x).

Steps toward the goal

The proof's milestones are: JN(x,π∗,h)≤JN(x,π,h)J_N(x,\pi^*,h)\le J_N(x,\pi,h)JN​(x,π∗,h)≤JN​(x,π,h); the decomposition JN(x,π,h)=JN(x,π)+Exπ[h(XN)]J_N(x,\pi,h)=J_N(x,\pi)+E^\pi_x[h(X_N)]JN​(x,π,h)=JN​(x,π)+Exπ​[h(XN​)]; part (i) on its own; and ρ(x)=lim⁡N1NJN(x,π∗)\rho(x)=\lim_N\frac1N J_N(x,\pi^*)ρ(x)=limN​N1​JN​(x,π∗).

Further targets: Theorem 6.3 (v)–(vi)

If ρ≡ρ∗\rho\equiv\rho^*ρ≡ρ∗ is constant and Φ(x,a)=c(x,a)+∫h(y)P(dy∣x,a)−ρ∗−h(x)\Phi(x,a)=c(x,a)+\int h(y)P(dy\mid x,a)-\rho^*-h(x)Φ(x,a)=c(x,a)+∫h(y)P(dy∣x,a)−ρ∗−h(x) is Mandl's discrepancy function, then for every π∈Π\pi\in\Piπ∈Π and xxx,

lim sup⁡N→∞1N∑t=0N−1c(Xt,At)≥ρ∗Pxπ-a.s.,\limsup_{N\to\infty}\frac1N\sum_{t=0}^{N-1}c(X_t,A_t)\ge\rho^*\quad\mathcal P^\pi_x\text{-a.s.},N→∞limsup​N1​t=0∑N−1​c(Xt​,At​)≥ρ∗Pxπ​-a.s.,

and the running average converges to ρ∗\rho^*ρ∗ almost surely if and only if 1N∑t<NΦ(Xt,At)→0\frac1N\sum_{t<N}\Phi(X_t,A_t)\to0N1​∑t<N​Φ(Xt​,At​)→0 almost surely. Moreover π∗\pi^*π∗ is sample path average cost optimal. The intermediate milestones are Φ≥0\Phi\ge0Φ≥0 on KKK, the almost-sure limit 1N∑c−ρ∗−1N∑Φ→0\frac1N\sum c-\rho^*-\frac1N\sum\Phi\to0N1​∑c−ρ∗−N1​∑Φ→0 under every policy, and Φ(Xt,At)=0\Phi(X_t,A_t)=0Φ(Xt​,At​)=0 almost surely under π∗\pi^*π∗.

Significance

The result. Theorem 6.3 reduces the average-cost problem on a general Borel space to finding a canonical triplet. Once one is found, ρ\rhoρ is the optimal average cost from every initial state, possibly state-dependent as in multichain models, and the canonical policy is optimal in a strong sense: its worst-case long-run performance is no worse than the best-case long-run performance of any competitor, at every finite horizon up to the additive constant span⁡(h)\operatorname{span}(h)span(h). With constant ρ\rhoρ, optimality also holds path by path. The rest of §6 of the survey looks for conditions on ccc and PPP that produce a canonical triplet, and those results rely on this theorem.

Formalizing it. The result is classical and proved; it has no machine-checked proof that we know of. Formalization requires measure-theoretic infrastructure for history-dependent policies on Borel spaces (Ionescu-Tulcea path measures, finite-horizon costs, infima over all admissible policies) together with a strong law for bounded martingale differences for parts (v)–(vi). Both are reusable for any average-cost result on general state spaces.

Difficulty

Parts (i)–(iii) are short on paper; the work is in making every quantity genuine. The infimum JN∗J^*_NJN∗​ ranges over a class of kernels, and turning (6.4) into a usable inequality needs the family bounded below. The decomposition of JN(x,π,h)J_N(x,\pi,h)JN​(x,π,h) needs integrability, which comes from the almost-sure confinement of the trajectory to KKK, a consequence of admissibility under the path measure. Parts (v)–(vi) need the conditional expectation identity Exπ[c(Xt,At)+h(Xt+1)−ρ∗−h(Xt)∣Ht,At]=Φ(Xt,At)E^\pi_x[c(X_t,A_t)+h(X_{t+1})-\rho^*-h(X_t)\mid H_t,A_t]=\Phi(X_t,A_t)Exπ​[c(Xt​,At​)+h(Xt+1​)−ρ∗−h(Xt​)∣Ht​,At​]=Φ(Xt​,At​) from the Markov structure of the path measure, and a martingale strong law. The tempting shortcut of restricting to stationary or Markov policies is not available: π∗\pi^*π∗ and every competitor are arbitrary history-dependent randomized policies.

Formalization scope

  • SSS is a standard Borel space and AAA a Borel space; U(x)U(x)U(x) is nonempty and compact with measurable graph KKK; c≥0c\ge0c≥0 on KKK (Assumption 2.1, which the paper assumes throughout). ccc and PPP are defined on S×AS\times AS×A and only their values on KKK enter.
  • Π\PiΠ is the class of history-dependent randomized admissible policies (p. 285). Neither π∗\pi^*π∗ nor the competitors are restricted.
  • Pxπ\mathcal P^\pi_xPxπ​ is Mathlib's Kernel.trajMeasure. JNJ_NJN​ is a Bochner integral and JN∗J^*_NJN∗​ a real infimum; every theorem assumes ccc bounded on KKK, which makes both genuine.
  • JJJ, J∗J^*J∗, both sides of (6.5), and the sample path average cost JSJ_SJS​ are computed in EReal, so no limit superior or inferior is a junk value. Part (iii) is the two equalities J(x,π∗)=J∗(x)J(x,\pi^*)=J^*(x)J(x,π∗)=J∗(x) and J∗(x)=ρ(x)J^*(x)=\rho(x)J∗(x)=ρ(x).
  • Part (iv) is not posed: its proof goes through value iteration under Assumptions 2.1–2.3, which Theorem 6.3 does not assume.
  • Part (vi) is stated under the hypothesis of (v), ρ≡ρ∗\rho\equiv\rho^*ρ≡ρ∗ constant. As printed it has no hypothesis on ρ\rhoρ and is false: two absorbing states with costs 000 and 111 give a canonical triplet whose pathwise average costs differ by state.
  • The identity JN(x,π,h)=JN(x,π)+Exπ[h(XN)]J_N(x,\pi,h)=J_N(x,\pi)+E^\pi_x[h(X_N)]JN​(x,π,h)=JN​(x,π)+Exπ​[h(XN​)] is stated for every π\piπ; the page writes it for π∗\pi^*π∗ and applies it to an arbitrary π\piπ in the proof of (i).
  • The sentence "for a canonical policy π∗\pi^*π∗, Φ(Xt,At)=0\Phi(X_t,A_t)=0Φ(Xt​,At​)=0, Pxπ\mathcal P^\pi_xPxπ​-a.s." is stated with Pxπ∗\mathcal P^{\pi^*}_xPxπ∗​, the only reading that makes sense.
  • Φ≥0\Phi\ge0Φ≥0 is derived on the page from (6.7) via Theorem 6.2, which covers stationary π∗\pi^*π∗; here it is stated directly from the canonical triplet for arbitrary π∗∈Π\pi^*\in\Piπ∗∈Π.
  • Sample path optimality quantifies over all initial laws (probability measures on SSS), as on p. 288.
  • None of the hypotheses is vacuous: the one-state, one-action model with zero cost and ρ≡h≡0\rho\equiv h\equiv0ρ≡h≡0 is a canonical triplet (checked in Lean). Strong average optimality is stated as limsup against liminf, not the weaker limsup against limsup.

Contributions welcome: a.s. confinement of trajectories to KKK, integrability lemmas for JNJ_NJN​, the Markov property of trajMeasure in the form above, and a strong law for bounded martingale differences.

Selected references

  • A. Arapostathis, V. S. Borkar, E. Fernández-Gaucherand, M. K. Ghosh, S. I. Marcus, Discrete-time controlled Markov processes with average cost criterion: a survey, SIAM J. Control Optim. 31(2) (1993) 282–344. https://doi.org/10.1137/0331018 (Theorem 6.3 on p. 318, its proof on pp. 318–319; (6.4), (6.5) on p. 316)
  • E. B. Dynkin, A. A. Yushkevich, Controlled Markov Processes, Springer-Verlag, New York, 1979 (reference [51] of the survey; Chap. 7).
  • A. A. Yushkevich, On a class of strategies in general Markov decision models, Theory Probab. Appl. 18 (1973) 777–779 (reference [204] of the survey).
  • P. Mandl, Estimation and control in Markov chains, Adv. Appl. Probab. 6 (1974) 40–60 (reference [124] of the survey).
  • J.-P. Georgin, Contrôle de chaînes de Markov sur des espaces arbitraires, Ann. Inst. H. Poincaré Sect. B 14 (1978) 255–277 (reference [72] of the survey).
  • R. A. Howard, Dynamic Programming and Markov Processes, MIT Press, Cambridge, MA, 1960 (reference [95] of the survey).
12 thms1 active userReviewed
Dynamic ProgrammingProbability·Captain: mikedeng1

Discrete-Time Controlled Markov Processes with Average Cost Criterion: A Survey 5: Canonical Triplets Are Exactly the Solutions of the Coupled Optimality EquationsResearch Paper

Motivation

A controlled Markov process run under the long-run average cost criterion asks for a policy minimizing the asymptotic cost per stage. When every stationary policy induces a single recurrent class, the optimal average cost is a constant and is characterized by one equation, the average cost optimality equation (ACOE). In general it is not: under some policies the state process splits into several ergodic classes, different classes have different optimal costs, and the optimal average cost is a function ρ(x)\rho(x)ρ(x) of the initial state. This is the multichain case.

For finite models, Howard ([Dynamic Programming and Markov Processes, 1960, pp. 61–62]) introduced a pair of coupled equations for this situation: one for the gain function ρ\rhoρ alone, and one, the ACOE, for ρ\rhoρ together with a relative value function hhh. Denardo and Fox (1968) developed the approach for finite multichain Markov renewal programs. For general Borel models, Yushkevich (1973) and Dynkin and Yushkevich (1979) introduced canonical triplets: a gain ρ\rhoρ, a terminal cost hhh and a policy π∗\pi^*π∗ such that π∗\pi^*π∗ is optimal for every finite horizon NNN with terminal cost hhh, and the optimal NNN-stage cost is exactly h+Nρh + N\rhoh+Nρ. The survey of Arapostathis, Borkar, Fernández-Gaucherand, Ghosh and Marcus (SIAM J. Control Optim. 31 (1993), §6.1) states the link between the two notions as Theorem 6.2 and proves it on p. 317.

Setting

A controlled Markov process is a five-tuple (S,A,U,P,c)(\mathbf S, \mathbf A, U, P, c)(S,A,U,P,c):

  1. S\mathbf SS, the state space, and A\mathbf AA, the action space, are Borel spaces;
  2. U(x)⊆AU(x) \subseteq \mathbf AU(x)⊆A is the nonempty compact set of admissible actions at xxx, and K={(x,a):a∈U(x)}\mathbf K = \{(x,a) : a \in U(x)\}K={(x,a):a∈U(x)} is measurable;
  3. P(dy∣x,a)P(dy \mid x, a)P(dy∣x,a) is a transition kernel on S\mathbf SS given K\mathbf KK;
  4. c:K→Rc : \mathbf K \to \mathbb Rc:K→R is a measurable one-stage cost with c≥0c \ge 0c≥0 (the paper's standing Assumption 2.1).

A history is ht=(x0,a0,…,xt−1,at−1,xt)h_t = (x_0, a_0, \dots, x_{t-1}, a_{t-1}, x_t)ht​=(x0​,a0​,…,xt−1​,at−1​,xt​). An admissible policy π=(πt)\pi = (\pi_t)π=(πt​) is a sequence of stochastic kernels πt(⋅∣ht)\pi_t(\cdot \mid h_t)πt​(⋅∣ht​) on A\mathbf AA with πt(U(xt)∣ht)=1\pi_t(U(x_t) \mid h_t) = 1πt​(U(xt​)∣ht​)=1; the class of all of them is Π\PiΠ. A stationary deterministic policy f∈ΠSDf \in \Pi_{SD}f∈ΠSD​ is a measurable map f:S→Af : \mathbf S \to \mathbf Af:S→A with f(x)∈U(x)f(x) \in U(x)f(x)∈U(x). An initial state xxx and a policy π\piπ determine a probability measure Pxπ\mathcal P^\pi_xPxπ​ on trajectories, with expectation ExπE^\pi_xExπ​. For a terminal cost hhh and N∈N0N \in \mathbb N_0N∈N0​,

JN(x,π,h)=Exπ[∑t=0N−1c(Xt,At)+h(XN)],JN∗(x,h)=inf⁡π∈ΠJN(x,π,h).J_N(x, \pi, h) = E^\pi_x\Big[\sum_{t=0}^{N-1} c(X_t, A_t) + h(X_N)\Big], \qquad J^*_N(x, h) = \inf_{\pi \in \Pi} J_N(x, \pi, h).JN​(x,π,h)=Exπ​[t=0∑N−1​c(Xt​,At​)+h(XN​)],JN∗​(x,h)=π∈Πinf​JN​(x,π,h).

Mb(S)\mathcal M_b(\mathbf S)Mb​(S) denotes the bounded measurable real functions on S\mathbf SS. For R,H∈Mb(S)R, H \in \mathcal M_b(\mathbf S)R,H∈Mb​(S) and π∗∈Π\pi^* \in \Piπ∗∈Π, the triplet (R,H,π∗)(R, H, \pi^*)(R,H,π∗) is canonical if

JN(x,π∗,H)=JN∗(x,H)=H(x)+NR(x)∀N∈N0, x∈S.(6.4)J_N(x, \pi^*, H) = J^*_N(x, H) = H(x) + N R(x) \qquad \forall N \in \mathbb N_0,\ x \in \mathbf S. \tag{6.4}JN​(x,π∗,H)=JN∗​(x,H)=H(x)+NR(x)∀N∈N0​, x∈S.(6.4)

Formalization targets

Goal: Theorem 6.2

Let π∗∈ΠSD\pi^* \in \Pi_{SD}π∗∈ΠSD​, ρ,h∈Mb(S)\rho, h \in \mathcal M_b(\mathbf S)ρ,h∈Mb​(S), and ccc bounded on K\mathbf KK. Then (ρ,h,π∗)(\rho, h, \pi^*)(ρ,h,π∗) is a canonical triplet if and only if, for all x∈Sx \in \mathbf Sx∈S,

ρ(x)=inf⁡a∈U(x){∫Sρ(y)P(dy∣x,a)},(6.6)\rho(x) = \inf_{a \in U(x)} \Big\{ \int_{\mathbf S} \rho(y) P(dy \mid x, a) \Big\}, \tag{6.6}ρ(x)=a∈U(x)inf​{∫S​ρ(y)P(dy∣x,a)},(6.6) ρ(x)+h(x)=inf⁡a∈U(x){c(x,a)+∫Sh(y)P(dy∣x,a)},(6.7)\rho(x) + h(x) = \inf_{a \in U(x)} \Big\{ c(x,a) + \int_{\mathbf S} h(y) P(dy \mid x, a) \Big\}, \tag{6.7}ρ(x)+h(x)=a∈U(x)inf​{c(x,a)+∫S​h(y)P(dy∣x,a)},(6.7)

and π∗(x)\pi^*(x)π∗(x) attains the infimum in both (6.6) and (6.7).

Milestones (the steps of the proof on p. 317)

  1. One-step decomposition (last line of (6.8)): for f∈ΠSDf \in \Pi_{SD}f∈ΠSD​, JN+1(x,f,h)=c(x,f(x))+∫JN(y,f,h)P(dy∣x,f(x))J_{N+1}(x, f, h) = c(x, f(x)) + \int J_N(y, f, h) P(dy \mid x, f(x))JN+1​(x,f,h)=c(x,f(x))+∫JN​(y,f,h)P(dy∣x,f(x)).
  2. Dynamic programming step (second line of (6.8)): if JN∗(⋅,h)J^*_N(\cdot, h)JN∗​(⋅,h) is bounded and measurable, then JN+1(x,π,h)≥T(JN∗)(x)J_{N+1}(x, \pi, h) \ge T(J^*_N)(x)JN+1​(x,π,h)≥T(JN∗​)(x) for every admissible π\piπ, where T(v)(x)=inf⁡a∈U(x){c(x,a)+∫v dP(⋅∣x,a)}T(v)(x) = \inf_{a \in U(x)}\{c(x,a) + \int v\, dP(\cdot \mid x, a)\}T(v)(x)=infa∈U(x)​{c(x,a)+∫vdP(⋅∣x,a)} is the map (2.5).
  3. Sufficiency, lower bound: under (6.6)–(6.7) and JN∗=h+NρJ^*_N = h + N\rhoJN∗​=h+Nρ, JN+1∗≥h+(N+1)ρJ^*_{N+1} \ge h + (N+1)\rhoJN+1∗​≥h+(N+1)ρ.
  4. Sufficiency, upper bound: under (6.6)–(6.7) and JN(⋅,π∗,h)=h+NρJ_N(\cdot, \pi^*, h) = h + N\rhoJN​(⋅,π∗,h)=h+Nρ, JN+1(⋅,π∗,h)=h+(N+1)ρ≥JN+1∗J_{N+1}(\cdot, \pi^*, h) = h + (N+1)\rho \ge J^*_{N+1}JN+1​(⋅,π∗,h)=h+(N+1)ρ≥JN+1∗​.

Significance

The result. Theorem 6.2 turns a statement about every finite horizon and every history-dependent randomized policy into two pointwise equations in (ρ,h)(\rho, h)(ρ,h) and a pointwise selection condition on π∗\pi^*π∗. A canonical policy is NNN-stage optimal for every NNN with terminal cost hhh; dividing (6.4) by NNN shows that its average cost is ρ\rhoρ. In the paper this is the entry point of Theorem 6.3, which shows that a canonical policy is strong average optimal and that ρ\rhoρ is the optimal average cost, without any recurrence assumption. Equation (6.6) is the condition that makes ρ\rhoρ behave as a constant in the optimization; when ρ\rhoρ is constant it holds trivially, and Theorem 6.2 specializes to the bounded ACOE with a minimizing selector.

Formalizing it. The theorem is proved in the paper (and earlier by Yushkevich). This mission formalizes that proof on a general Borel model with history-dependent randomized policies and path measures built by the Ionescu-Tulcea theorem. Prove2Me has no formalization of the multichain coupled equations or of canonical triplets beyond finite models. The finite-horizon dynamic programming inequality of milestone 2, for policies that may use the whole history, is reusable in any finite-horizon or average-cost development on Borel spaces.

Difficulty

The algebra in the proof takes a few lines. The work is in two probabilistic facts that the paper uses without comment.

The first is the Markov decomposition of the (N+1)(N+1)(N+1)-stage cost: conditioning on the first state–action pair turns the remaining NNN stages into an NNN-stage problem started from the next state. For a stationary policy the continuation is the same policy. For a history-dependent policy, the continuation is a policy that depends measurably on (x0,a0)(x_0, a_0)(x0​,a0​). Expressing this on the Ionescu-Tulcea measure is where the effort goes.

The second is the identity JN+1∗=T(JN∗)J^*_{N+1} = T(J^*_N)JN+1∗​=T(JN∗​). On a Borel model, JN∗J^*_NJN∗​ need not be measurable and the infimum need not be attained by a measurable selector. That is why the paper usually works with semicontinuous models (p. 288). Theorem 6.2 avoids the issue: only the inequality JN+1∗≥T(JN∗)J^*_{N+1} \ge T(J^*_N)JN+1∗​≥T(JN∗​) is needed, under the measurability that JN∗=h+NρJ^*_N = h + N\rhoJN∗​=h+Nρ supplies, and the reverse direction is supplied by the given π∗\pi^*π∗. A first attempt that proves the full identity JN+1∗=T(JN∗)J^*_{N+1} = T(J^*_N)JN+1∗​=T(JN∗​) for general Borel models runs into measurable selection problems that the theorem never needs.

Formalization scope

  • Model. BorelCMP S A has standard Borel S and Borel A, compact nonempty U x with measurable graph, a measurable cost c : S × A → ℝ with c ≥ 0 on K, and a Markov kernel P. c and P are total on S × A; only values on K enter. No continuity of c or P is assumed: Theorem 6.2 does not use Assumptions 2.2, 2.3 or 6.1.
  • Policies. Policy M is the paper's Π\PiΠ: history-dependent, randomized, with admissibility πt(U(xt)c∣ht)=0\pi_t(U(x_t)^c \mid h_t) = 0πt​(U(xt​)c∣ht​)=0. StationaryPolicy M is ΠSD\Pi_{SD}ΠSD​ (measurable fff with f(x)∈U(x)f(x) \in U(x)f(x)∈U(x)). The infimum JN∗J^*_NJN∗​ ranges over all of Policy M. Restricting it to stationary policies would make sufficiency trivial and necessity false, and is not what the paper states.
  • Costs. JN(x,π,h)J_N(x, \pi, h)JN​(x,π,h) is a Bochner integral against the path measure. Every statement assumes ccc bounded on K\mathbf KK and hhh bounded measurable, so the integral is a genuine expectation. JN∗J^*_NJN∗​ is a real infimum, which is the true infimum because the family is bounded below by −sup⁡∣h∣-\sup|h|−sup∣h∣ and nonempty whenever a stationary policy is given.
  • Coupled equations. (6.6) and (6.7) with "π∗(x)\pi^*(x)π∗(x) attains the infimum" are encoded in attained form: equality at a=π∗(x)a = \pi^*(x)a=π∗(x) and the inequality for every a∈U(x)a \in U(x)a∈U(x). This is equivalent to the printed condition and never forms a real infimum.
  • The gain is a function. ρ\rhoρ is a function of the state, not a constant. Specializing to a constant ρ\rhoρ (the unichain case) would trivialize (6.6) and is not the theorem.
  • Explicit choices. The paper's induction from N−1N-1N−1 to NNN is stated from NNN to N+1N+1N+1, so that no natural-number subtraction appears. Milestone 2 takes the measurability and boundedness of JN∗(⋅,h)J^*_N(\cdot, h)JN∗​(⋅,h) as a hypothesis, and takes any pointwise lower bound www of T(JN∗)T(J^*_N)T(JN∗​) in place of the infimum. In the second display of the sufficiency part, the paper prints JN−1∗(y,π∗,h)J^*_{N-1}(y, \pi^*, h)JN−1∗​(y,π∗,h); the quantity meant is JN−1(y,π∗,h)J_{N-1}(y, \pi^*, h)JN−1​(y,π∗,h), which milestone 4 uses.
  • Infrastructure. A complete development needs the Markov property of Kernel.trajMeasure at the first step, the shifted (continuation) policy and its measurability in the first state–action pair, and bounded-convergence bookkeeping for the Bochner integrals. These are reusable for every finite-horizon statement on this model. Proofs of the milestones are welcome independently. So are alternative proofs of the goal that bypass milestone 2 and argue directly with the canonical policy.

Selected references

  • A. Arapostathis, V. S. Borkar, E. Fernández-Gaucherand, M. K. Ghosh, S. I. Marcus, Discrete-time controlled Markov processes with average cost criterion: a survey, SIAM J. Control Optim. 31(2) (1993), 282–344, §6.1, Theorem 6.2. https://doi.org/10.1137/0331018
  • A. A. Yushkevich, On a class of strategies in general Markov decision models, Theory Probab. Appl. 18 (1973), 777–779.
  • E. B. Dynkin, A. A. Yushkevich, Controlled Markov Processes, Springer-Verlag, New York, 1979.
  • R. A. Howard, Dynamic Programming and Markov Processes, MIT Press, Cambridge, MA, 1960, pp. 61–62.
  • E. V. Denardo, B. L. Fox, Multichain Markov renewal programs, SIAM J. Appl. Math. 16 (1968), 468–487.
6 thms1 active userReviewed
Dynamic ProgrammingOperations ResearchProbability+1·Captain: mikedeng1

Stochastic Optimal Control: The Discrete-Time Case VIII: The Infinite-Horizon Borel Models — the Optimality Equation J* = T(J*) under (P), (N), (D)Textbook

Motivation

Infinite horizon dynamic programming asks for the least expected cost of controlling a stochastic system forever, and for a policy attaining it. On finite or countable state spaces the theory has been classical since Bellman, Blackwell (1965) and Strauch (1966). Many models in operations research, inventory control, queueing and economics have continuous states and controls, however, and there the Bellman equation raises a question the countable theory never meets: the optimal cost need not be Borel-measurable, so its expectation under the transition law, which the equation requires, may not be defined.

Bertsekas and Shreve (1978) settled this question by working with lower semianalytic cost functions and universally measurable policies. In that framework the optimal cost is always measurable enough to be integrated, and the optimality equation holds with no continuity or compactness assumption. Chapter 9 of their book treats the infinite horizon model under the three classical cost structures: nonnegative costs (P), nonpositive costs (N), and bounded discounted costs (D).

Timeline:

  • 1965: Blackwell, discounted dynamic programming on Borel spaces with Borel-measurable data, case (D).
  • 1966: Strauch, negative dynamic programming, case (N), with Borel-measurable data.
  • 1978: Bertsekas and Shreve, Chapter 9: lower semianalytic costs and universally measurable policies, in all three cases (P), (N), (D).
  • 1979: Shreve and Bertsekas, the journal account of universally measurable policies.

Setting

An infinite horizon stochastic optimal control model (SM) is an eight-tuple (S,C,U,W,p,f,α,g)(S, C, U, W, p, f, \alpha, g)(S,C,U,W,p,f,α,g). The state space SSS, the control space CCC and the disturbance space WWW are nonempty Borel spaces, that is, spaces homeomorphic to Borel subsets of complete separable metric spaces. The control constraint UUU assigns to each state xxx a nonempty set U(x)⊆CU(x) \subseteq CU(x)⊆C, and the set Γ={(x,u)∣u∈U(x)}\Gamma = \{(x,u) \mid u \in U(x)\}Γ={(x,u)∣u∈U(x)} is analytic. The disturbance kernel p(dw∣x,u)p(dw \mid x, u)p(dw∣x,u) is a Borel stochastic kernel and the system function f:SCW→Sf : SCW \to Sf:SCW→S is Borel. The discount factor is α>0\alpha > 0α>0, and the one-stage cost g:Γ→[−∞,∞]g : \Gamma \to [-\infty, \infty]g:Γ→[−∞,∞] is lower semianalytic: each sublevel set {g<c}\{g < c\}{g<c} is analytic. The state moves by xk+1=f(xk,uk,wk)x_{k+1} = f(x_k, u_k, w_k)xk+1​=f(xk​,uk​,wk​), with transition kernel t(B∣x,u)=p({w∣f(x,u,w)∈B}∣x,u)t(B \mid x, u) = p(\{w \mid f(x,u,w) \in B\} \mid x, u)t(B∣x,u)=p({w∣f(x,u,w)∈B}∣x,u).

A policy π=(μ0,μ1,… )\pi = (\mu_0, \mu_1, \dots)π=(μ0​,μ1​,…) chooses uku_kuk​ at random from a universally measurable stochastic kernel μk(duk∣x0,u0,…,xk)\mu_k(du_k \mid x_0, u_0, \dots, x_k)μk​(duk​∣x0​,u0​,…,xk​) concentrated on U(xk)U(x_k)U(xk​); Π′\Pi'Π′ is the set of all policies. A policy is Markov if each μk\mu_kμk​ depends only on xkx_kxk​, and it is stationary if, moreover, μk=μ\mu_k = \muμk​=μ for all kkk. Writing qk(π,px)q_k(\pi, p_x)qk​(π,px​) for the law of (xk,uk)(x_k, u_k)(xk​,uk​) started from x0=xx_0 = xx0​=x, the cost of π\piπ and the optimal cost are

Jπ(x)=∑k=0∞αk∫g dqk(π,px),J∗(x)=inf⁡π∈Π′Jπ(x).J_\pi(x) = \sum_{k=0}^\infty \alpha^k \int g\, dq_k(\pi, p_x), \qquad J^*(x) = \inf_{\pi \in \Pi'} J_\pi(x).Jπ​(x)=k=0∑∞​αk∫gdqk​(π,px​),J∗(x)=π∈Π′inf​Jπ​(x).

For J:S→[−∞,∞]J : S \to [-\infty, \infty]J:S→[−∞,∞], the dynamic programming operators are

T(J)(x)=inf⁡u∈U(x){g(x,u)+α∫SJ(x′) t(dx′∣x,u)},Tμ(J)(x)=∫C[g(x,u)+α∫SJ dt]μ(du∣x).T(J)(x) = \inf_{u \in U(x)} \Big\{ g(x,u) + \alpha \int_S J(x')\, t(dx' \mid x, u) \Big\}, \qquad T_\mu(J)(x) = \int_C \Big[ g(x,u) + \alpha \int_S J\, dt \Big] \mu(du \mid x).T(J)(x)=u∈U(x)inf​{g(x,u)+α∫S​J(x′)t(dx′∣x,u)},Tμ​(J)(x)=∫C​[g(x,u)+α∫S​Jdt]μ(du∣x).

The three cases are (P) g≥0g \ge 0g≥0 on Γ\GammaΓ; (N) g≤0g \le 0g≤0 on Γ\GammaΓ; (D) α<1\alpha < 1α<1 and ∣g∣≤b|g| \le b∣g∣≤b on Γ\GammaΓ for some real bbb.

Formalization targets

Goal: the optimality equation (Proposition 9.8, Eq. (22))

Under each of (P), (N) and (D),

J∗=T(J∗).J^* = T(J^*).J∗=T(J∗).

Milestones

  • J∗J^*J∗ is lower semianalytic (Corollary 9.4.1).
  • For a stationary policy, Jμ=Tμ(Jμ)J_\mu = T_\mu(J_\mu)Jμ​=Tμ​(Jμ​) (Proposition 9.9).
  • Optimality tests for stationary policies: under (P) or (D), (μ,μ,… )(\mu, \mu, \dots)(μ,μ,…) is optimal iff J∗=Tμ(J∗)J^* = T_\mu(J^*)J∗=Tμ​(J∗) (Proposition 9.12); under (N) or (D), iff Jμ=T(Jμ)J_\mu = T(J_\mu)Jμ​=T(Jμ​) (Proposition 9.13).
  • Under (N) or (D), value iteration from 000 converges to J∗J^*J∗, and under (D) it converges uniformly from every bounded lower semianalytic start (Proposition 9.14).

Further statements of the mission

  • Markov policies suffice: at each state some Markov policy matches any policy's cost (Proposition 9.1), so J∗=inf⁡π∈ΠJπJ^* = \inf_{\pi \in \Pi} J_\piJ∗=infπ∈Π​Jπ​ (Corollary 9.1.1).
  • Partial converses of the optimality equation: J≥T(J)J \ge T(J)J≥T(J), J≥0J \ge 0J≥0 gives J≥J∗J \ge J^*J≥J∗ under (P); J≤T(J)J \le T(J)J≤T(J), J≤0J \le 0J≤0 gives J≤J∗J \le J^*J≤J∗ under (N); a bounded solution of J=T(J)J = T(J)J=T(J) equals J∗J^*J∗ under (D) (Proposition 9.10). The analogous statements for TμT_\muTμ​ and JμJ_\muJμ​ (Proposition 9.11).

Significance

The optimality equation is the basic structural fact of infinite horizon control. Corollary 9.12.1 uses it to construct optimal stationary policies from minimizers in the equation. The existence results for ε\varepsilonε-optimal policies (Propositions 9.19 and 9.20), the convergence analysis of value iteration in Section 9.5, and the reduction of imperfect state information problems in Chapter 10 all build on it. It holds for arbitrary Borel models, with no continuity or compactness assumption.

All results of the mission were proved in 1978. None of them has a machine-checked proof: Mathlib has stochastic kernels and the Ionescu-Tulcea construction for measurable kernels, but no theory of lower semianalytic functions, universally measurable kernels, or dynamic programming on Borel spaces. A formal development would fix the measurability bookkeeping on which the textbook proofs rest and supply a reusable substrate for the stochastic control papers that cite this book.

Difficulty

Under (D), TTT is a contraction on bounded functions, and its fixed point is the limit of value iteration. That argument, however, gives a fixed point only within a fixed class of measurable functions. Showing that this fixed point equals J∗J^*J∗ requires knowing that J∗J^*J∗ belongs to the class and that history-dependent randomized policies do no better. Under (P), value iteration can converge to the wrong limit (Example 1 of the chapter: lim⁡kJk(0)=0\lim_k J_k(0) = 0limk​Jk​(0)=0 while J∗(0)=∞J^*(0) = \inftyJ∗(0)=∞). Even when each JkJ_kJk​ is Borel, J∗J^*J∗ may fail to be (Example 2). So J∗=T(J∗)J^* = T(J^*)J∗=T(J∗) cannot be obtained as a limit of the finite horizon equations, and the natural class of Borel functions is not closed under the partial minimization that defines TTT.

The book's route lifts (SM) to a deterministic model on the space of probability measures P(S)P(S)P(S), where no measurability restriction is needed, and transfers the results back. Making this transfer rigorous requires that the cost of a randomized policy be a measurable functional of its law, and that the infimum over policies preserve lower semianalyticity.

Formalization scope

The draft fixes the following conventions.

  • Spaces. Borel spaces are topological spaces homeomorphic to Borel subsets of complete separable metric spaces, carrying their Borel σ\sigmaσ-algebras. Analytic sets are Mathlib's AnalyticSet. A set is universally measurable if it is null-measurable for every probability measure.
  • Extended reals. Values lie in EReal. The book's convention ∞−∞=−∞+∞=∞\infty - \infty = -\infty + \infty = \infty∞−∞=−∞+∞=∞ is implemented explicitly, because Mathlib's EReal sets ⊥+⊤=⊥\bot + \top = \bot⊥+⊤=⊥. The integral of an extended-real function is ∫f+−∫f−\int f^+ - \int f^-∫f+−∫f− with the same convention.
  • Policies. These are sequences of universally measurable stochastic kernels on the history spaces S0C0⋯SkS_0C_0 \cdots S_kS0​C0​⋯Sk​, charging U(xk)U(x_k)U(xk​) with mass one. The laws of (x0,u0,…,xk,uk)(x_0, u_0, \dots, x_k, u_k)(x0​,u0​,…,xk​,uk​) are built recursively from the kernels.
  • Costs. JπJ_\piJπ​ is the series ∑kαk∫g dqk\sum_k \alpha^k \int g\, dq_k∑k​αk∫gdqk​, computed as the difference of the series of positive and negative parts. Under each of (P), (N), (D) it coincides with the integral of the total discounted cost. J∗J^*J∗ is the infimum over all policies.
  • Case labels. Each statement carries the case labels the book attaches to it, as hypotheses on the model.
  • Scope. Only the (SM) statements are formalized. The deterministic model (DM) on P(S)P(S)P(S) is the book's proof device and enters no statement.

The goal admits a trivializing formalization that this draft rules out. J∗J^*J∗ is not defined as a fixed point of TTT, nor as the limit of Tk(0)T^k(0)Tk(0); it is the infimum of the costs of all policies, which under (P) can differ from that limit.

A complete development needs universally measurable kernels and their compositions on product spaces, measurability of x↦∫f(x,y) q(dy∣x)x \mapsto \int f(x, y)\, q(dy \mid x)x↦∫f(x,y)q(dy∣x) for universally measurable integrands, the measurable selection theorem of Jankov and von Neumann, and the closure of lower semianalytic functions under partial infimum. These are the subject of the series' mission on Chapter 7, and they are reusable for any stochastic control model on Borel spaces. Contributions are welcome at any level: these foundations, the Markov reduction (Proposition 9.1), or the case-by-case arguments.

Selected references

  • D. P. Bertsekas and S. E. Shreve, Stochastic Optimal Control: The Discrete-Time Case, Academic Press, 1978; Athena Scientific reprint, 1996. Chapter 9. https://web.mit.edu/dimitrib/www/soc.html
  • D. Blackwell, Discounted dynamic programming, Annals of Mathematical Statistics 36 (1965), 226–235. https://doi.org/10.1214/aoms/1177700285
  • R. E. Strauch, Negative dynamic programming, Annals of Mathematical Statistics 37 (1966), 871–890. https://doi.org/10.1214/aoms/1177699369
  • S. E. Shreve and D. P. Bertsekas, Universally measurable policies in dynamic programming, Mathematics of Operations Research 4 (1979), 15–30. https://doi.org/10.1287/moor.4.1.15
15 thms1 active userReviewed
Calculus of VariationsOptimization·Captain: mikedeng1

On the Variational Principle III: Every Terminal-Cost Control Problem Has ε-Optimal Measurable Controls Satisfying the Pontryagin Maximum Principle up to εResearch Paper

Motivation

The Pontryagin maximum principle is the basic first-order necessary condition of optimal control: along an optimal control, the control value at almost every time extremizes a Hamiltonian built from the dynamics and an adjoint vector. The classical statement presupposes that an optimal control exists. For many problems none does: the set of trajectories is not closed, or the cost does not attain its infimum, and the classical principle then says nothing.

Ekeland's 1974 paper On the Variational Principle (DOI 10.1016/0022-247X(74)90025-0) proves a general result about lower semicontinuous functions on complete metric spaces (its Theorem 1.1, now called Ekeland's variational principle) and closes with an application to control theory in §7 (pp. 348–351): for every terminal-cost problem satisfying mild regularity and growth conditions, there are ε-optimal controls that satisfy the maximum principle up to ε, whether or not an optimal control exists. The argument equips the measurable controls with the metric "measure of the set where two controls differ", which has since become a standard device in nonsmooth and approximate optimal control. For the classical theory, Ekeland refers to the treatise of Pallu de la Barrière (Optimal Control Theory, Saunders, 1967).

Setting

Fix n≥0n \ge 0n≥0, a horizon T>0T > 0T>0, an initial state x0∈Rnx_0 \in \mathbb R^nx0​∈Rn, and a nonempty compact metrizable space KKK of control values, with its Borel σ-algebra. A control is a measurable map u:[0,T]→Ku : [0, T] \to Ku:[0,T]→K. The dynamics are a map f:Rn×K×[0,T]→Rnf : \mathbb R^n \times K \times [0, T] \to \mathbb R^nf:Rn×K×[0,T]→Rn, and the state obeys

dxdt(t)=f(x(t),u(t),t)  a.e.,x(0)=x0.(7.1)\frac{dx}{dt}(t) = f(x(t), u(t), t) \ \text{ a.e.}, \qquad x(0) = x_0. \qquad (7.1)dtdx​(t)=f(x(t),u(t),t)  a.e.,x(0)=x0​.(7.1)

A trajectory of uuu (Lean: IsTrajectory f x₀ T u x) is a function xxx continuous on [0,T][0, T][0,T] with x(t)=x0+∫0tf(x(s),u(s),s) dsx(t) = x_0 + \int_0^t f(x(s), u(s), s)\,dsx(t)=x0​+∫0t​f(x(s),u(s),s)ds for all t∈[0,T]t \in [0, T]t∈[0,T]. The standing hypotheses are:

  • (a) fff and its state Jacobian fx′=(∂f/∂x1,…,∂f/∂xn)f_x' = (\partial f/\partial x_1, \dots, \partial f/\partial x_n)fx′​=(∂f/∂x1​,…,∂f/∂xn​) (Lean: fx) are continuous on Rn×K×[0,T]\mathbb R^n \times K \times [0, T]Rn×K×[0,T];
  • (b) ⟨x,f(x,u,t)⟩≤c (1+∥x∥2)\langle x, f(x, u, t)\rangle \le c\,(1 + \|x\|^2)⟨x,f(x,u,t)⟩≤c(1+∥x∥2) for some constant ccc.

The terminal cost is given by a C1C^1C1 function g:Rn→Rg : \mathbb R^n \to \mathbb Rg:Rn→R, and the problem is to minimize g(x(T))g(x(T))g(x(T)) over all controls. The adjoint vector ppp along a pair (u,x)(u, x)(u,x) (Lean: IsAdjoint fx g T u x p) solves the linear equation

dpdt(t)=−tfx′(x(t),u(t),t) p(t),p(T)=g′(x(T)),\frac{dp}{dt}(t) = -{}^t f_x'(x(t), u(t), t)\, p(t), \qquad p(T) = g'(x(T)),dtdp​(t)=−tfx′​(x(t),u(t),t)p(t),p(T)=g′(x(T)),

where tA{}^t AtA is the transpose. The control distance (Lean: ctrlDist T u₁ u₂) is

δ(u1,u2)=meas⁡{t∈[0,T]∣u1(t)≠u2(t)},\delta(u_1, u_2) = \operatorname{meas}\{t \in [0, T] \mid u_1(t) \neq u_2(t)\},δ(u1​,u2​)=meas{t∈[0,T]∣u1​(t)=u2​(t)},

and the needle variation vτv_\tauvτ​ of a control uεu_\varepsilonuε​ at t0t_0t0​ with value u0∈Ku_0 \in Ku0​∈K (Lean: needle T uε u₀ t₀ τ) equals u0u_0u0​ on [0,T]∩ ]t0−τ,t0[[0, T] \cap\, ]t_0 - \tau, t_0[[0,T]∩]t0​−τ,t0​[ and uεu_\varepsilonuε​ elsewhere.

Formalization targets

Goal: Theorem 7.1

For every ε>0\varepsilon > 0ε>0 there is a control uεu_\varepsilonuε​ with trajectory xεx_\varepsilonxε​ and adjoint vector pεp_\varepsilonpε​ such that

g(xε(T))≤inf⁡ug(x(T))+ε,⟨f(xε(t),uε(t),t),pε(t)⟩≤min⁡w∈K⟨f(xε(t),w,t),pε(t)⟩+ε  a.e. on [0,T].g(x_\varepsilon(T)) \le \inf_u g(x(T)) + \varepsilon, \qquad \langle f(x_\varepsilon(t), u_\varepsilon(t), t), p_\varepsilon(t)\rangle \le \min_{w \in K} \langle f(x_\varepsilon(t), w, t), p_\varepsilon(t)\rangle + \varepsilon \ \text{ a.e. on } [0, T].g(xε​(T))≤uinf​g(x(T))+ε,⟨f(xε​(t),uε​(t),t),pε​(t)⟩≤w∈Kmin​⟨f(xε​(t),w,t),pε​(t)⟩+ε  a.e. on [0,T].

Milestones

  1. (7.2), a priori bound: every trajectory satisfies ∥x(t)∥2≤(∥x0∥2+2cT)e2cT\|x(t)\|^2 \le (\|x_0\|^2 + 2cT)e^{2cT}∥x(t)∥2≤(∥x0​∥2+2cT)e2cT.
  2. Well-posedness (p. 348): every control has exactly one trajectory on [0,T][0, T][0,T].
  3. Lemma 7.2: (U,δ)(\mathcal U, \delta)(U,δ) is a complete metric space (on almost-everywhere classes).
  4. Lemma 7.3: u↦g(x(T))u \mapsto g(x(T))u↦g(x(T)) is continuous on (U,δ)(\mathcal U, \delta)(U,δ).
  5. (7.15)–(7.16): a control uεu_\varepsilonuε​ with F(uε)≤inf⁡F+ε2F(u_\varepsilon) \le \inf F + \varepsilon^2F(uε​)≤infF+ε2 and F(u)≥F(uε)−ε δ(u,uε)F(u) \ge F(u_\varepsilon) - \varepsilon\,\delta(u, u_\varepsilon)F(u)≥F(uε​)−εδ(u,uε​) for all uuu, where F(u)=g(x(T))F(u) = g(x(T))F(u)=g(x(T)).
  6. Lemma 7.4: the right derivative of τ↦g(xτ(T))\tau \mapsto g(x_\tau(T))τ↦g(xτ​(T)) at τ=0\tau = 0τ=0, for the needle variations vτv_\tauvτ​, equals ⟨f(xε(t0),u0,t0)−f(xε(t0),uε(t0),t0),pε(t0)⟩\langle f(x_\varepsilon(t_0), u_0, t_0) - f(x_\varepsilon(t_0), u_\varepsilon(t_0), t_0), p_\varepsilon(t_0)\rangle⟨f(xε​(t0​),u0​,t0​)−f(xε​(t0​),uε​(t0​),t0​),pε​(t0​)⟩.

Significance

The result. Theorem 7.1 decouples the maximum principle from existence of optimal controls. Any minimizing sequence can be replaced by one consisting of controls that satisfy the necessary condition up to a vanishing error, so approximate extremals are available for numerical schemes and for limiting arguments (relaxation, convergence of extremals) in problems without compactness or convexity of the velocity sets. The same scheme, a complete metric on controls plus Ekeland's principle plus needle variations, underlies later proofs of the maximum principle with state constraints and in nonsmooth settings.

Formalizing it. The theorem has a published proof, which the paper sketches in four pages and partly defers to classical results (local existence for measurable controls, differentiability of trajectories with respect to needle variations). Neither Mathlib nor the platform contains this theorem or a maximum principle for measurable controls with time-dependent dynamics; the nearest platform items assume an exact optimum, a running cost and globally Lipschitz autonomous dynamics. A formalization requires a Carathéodory theory of ODEs in Lean, the complete metric space of controls, and a rigorous version of the classical first-variation computation. Each of these is useful well beyond this paper.

Difficulty

The obvious argument fails at its first step: a minimizing sequence of controls has no convergent subsequence in any useful sense, because measurable controls with values in KKK are not compact in the metric δ\deltaδ and their weak limits are relaxed controls, which are not controls. Ekeland's principle avoids compactness, but it needs a complete metric space on which the cost is lower semicontinuous. Completeness of (U,δ)(\mathcal U, \delta)(U,δ) (Lemma 7.2) and continuity of the cost (Lemma 7.3) both require care. Continuity must hold for trajectories driven by merely measurable controls, which converge only in measure. The needle derivative (Lemma 7.4) holds only at times where the trajectory satisfies the state equation in the classical sense, which is almost every time but not every time.

Formalization scope

  • Controls are functions ℝ → K with Measurable u; only values on [0,T][0, T][0,T] matter. Trajectories and adjoint vectors are functions ℝ → EuclideanSpace ℝ (Fin n), continuous on [0,T][0, T][0,T] and solving the integral form of their equation. The paper uses the same form in (7.13). A definition that asks for a derivative at every time would be unsatisfiable for discontinuous controls and would make the goal vacuous; the integral form avoids this.
  • Hypothesis (a) is stated as joint continuity of f and fx on univ ×ˢ univ ×ˢ Icc 0 T together with HasFDerivAt (fun y => f y u t) (fx x u t) x. Hypothesis (b) is ∃ c, … with the order of arguments f(x,u,t)f(x, u, t)f(x,u,t); the page prints f(t,x,u)f(t, x, u)f(t,x,u) in (b), a slip.
  • The transpose tfx′{}^t f_x'tfx′​ is ContinuousLinearMap.adjoint, and g′(x)g'(x)g′(x) is gradient g x.
  • Infima over controls are never taken as real ⨅; (7.4) and (7.15) are stated as "for every control uuu with trajectory xxx".
  • (7.5) is printed with "× ε\times\,\varepsilon×ε". The proof's last line (7.21) gives "+ ε+\,\varepsilon+ε", which is what is stated. The minimum over the compact KKK is replaced by the equivalent "for every w∈Kw \in Kw∈K", with one null set for all www.
  • Added hypothesis: [Nonempty K] in the goal and milestone 5. With KKK empty no control exists and the existential statements would be false.
  • Lemma 7.2 is stated as the triangle inequality, "δ=0\delta = 0δ=0 iff almost-everywhere equality on [0,T][0, T][0,T]", and sequential completeness for measurable controls. Lemma 7.3 is sequential continuity. Lemma 7.4 is a one-sided derivative within [0,t0][0, t_0][0,t0​] at 000, stated for an arbitrary measurable uεu_\varepsilonuε​.
  • Theorem 1.1 (Ekeland's principle) is a separate mission of this series and still a draft, so milestone 5 states the instance used here directly.
  • Out of scope: the uniform bound (7.3) and the remark after Theorem 7.1 that one may take ε=0\varepsilon = 0ε=0 in (7.5) when (7.4) holds with ε=0\varepsilon = 0ε=0.

Welcome contributions include Carathéodory existence and uniqueness for ODEs with measurable time dependence, Gronwall-type estimates for absolutely continuous functions, the complete metric space of measurable maps under the disagreement measure, and differentiability of flows with respect to initial data.

Selected references

  • I. Ekeland, On the Variational Principle, J. Math. Anal. Appl. 47 (1974), 324–353. https://doi.org/10.1016/0022-247X(74)90025-0
  • I. Ekeland, Nonconvex minimization problems, Bull. Amer. Math. Soc. (N.S.) 1 (1979), 443–474. https://doi.org/10.1090/S0273-0979-1979-14595-6
  • R. Pallu de la Barrière, Optimal Control Theory, Saunders, Philadelphia, 1967.
  • L. S. Pontryagin, V. G. Boltyanskii, R. V. Gamkrelidze, E. F. Mishchenko, The Mathematical Theory of Optimal Processes, Interscience, New York, 1962.
11 thms1 active userReviewed
Operations ResearchProbabilityStochastic Systems·Captain: mikedeng1

Dynamic Scheduling of a System with Two Parallel Servers in Heavy Traffic with Resource Pooling: The Threshold Policy Is Asymptotically OptimalResearch Paper

Motivation

Many service systems route several classes of work to servers with overlapping skills: call centers with cross-trained agents, manufacturing cells with flexible machines, computing clusters with heterogeneous processors. Choosing which server works on which class at each moment is a dynamic scheduling problem. Exact optimal policies are out of reach except in toy cases, so heavy-traffic theory replaces the queueing system by a Brownian control problem, solves that limit problem, and then asks for a policy in the original system whose performance converges to the Brownian optimum. This programme was proposed by Harrison (Harrison 1988), and the parallel server system studied here is the example Harrison used (Harrison, Ann. Appl. Probab. 1998) to show that the greedy static priority rule can be very inefficient.

Bell and Williams (2001) gave the first proof of asymptotic optimality of a continuous-review policy for this system, with renewal arrivals and general service times. Harrison (1998) had treated Poisson arrivals and deterministic service times with a discrete-review policy and a pathwise criterion. Harrison and López (Queueing Systems, 1999) identified the complete resource pooling condition for general parallel server systems. The threshold policy and the proof method of Bell and Williams were later extended to multiserver systems (Bell and Williams, Electron. J. Probab., 2005).

Setting

There are two job classes and two servers. Server 1 serves class 1 (activity 1); server 2 serves class 1 (activity 2) and class 2 (activity 3). A sequence of such systems is indexed by r→∞r\to\inftyr→∞. On a probability space, i.i.d. sequences uˇk(i)\check u_k(i)uˇk​(i) (k=1,2k=1,2k=1,2) and vˇj(i)\check v_j(i)vˇj​(i) (j=1,2,3j=1,2,3j=1,2,3), i≥1i\ge1i≥1, are fixed: strictly positive, mutually independent, with mean one and finite variances αk2,βj2\alpha_k^2,\beta_j^2αk2​,βj2​. In system rrr the interarrival times are ukr(i)=uˇk(i)/λkru_k^r(i)=\check u_k(i)/\lambda_k^rukr​(i)=uˇk​(i)/λkr​ and the service times are vjr(i)=vˇj(i)/μjrv_j^r(i)=\check v_j(i)/\mu_j^rvjr​(i)=vˇj​(i)/μjr​. The renewal processes Akr(t)A_k^r(t)Akr​(t) and Sjr(t)S_j^r(t)Sjr​(t) count arrivals and potential service completions.

A scheduling control policy is an allocation T=(T1,T2,T3)T=(T_1,T_2,T_3)T=(T1​,T2​,T3​), where Tj(t)T_j(t)Tj​(t) is the time devoted to activity jjj in [0,t][0,t][0,t]. Each Tj(t)T_j(t)Tj​(t) is a random variable, each TjT_jTj​ is continuous and nondecreasing from 000, and so are the idle times I1=t−T1I_1=t-T_1I1​=t−T1​ and I2=t−T2−T3I_2=t-T_2-T_3I2​=t−T2​−T3​. The queue lengths

Q1(t)=A1(t)−S1(T1(t))−S2(T2(t)),Q2(t)=A2(t)−S3(T3(t))Q_1(t)=A_1(t)-S_1(T_1(t))-S_2(T_2(t)),\qquad Q_2(t)=A_2(t)-S_3(T_3(t))Q1​(t)=A1​(t)−S1​(T1​(t))−S2​(T2​(t)),Q2​(t)=A2​(t)−S3​(T3​(t))

must be nonnegative. Policies may anticipate the future. The rates satisfy Assumption 3.1: λ1>μ1\lambda_1>\mu_1λ1​>μ1​, 1−(λ1−μ1)/μ2=λ2/μ31-(\lambda_1-\mu_1)/\mu_2=\lambda_2/\mu_31−(λ1​−μ1​)/μ2​=λ2​/μ3​, and the rates converge at rate 1/r1/r1/r to limits with second-order parameters θ1,θ2\theta_1,\theta_2θ1​,θ2​. Assumption 3.2 is h1μ2≥h2μ3h_1\mu_2\ge h_2\mu_3h1​μ2​≥h2​μ3​, and Assumption 3.3 gives finite exponential moments near 000. With Q^r(t)=r−1Qr(r2t)\hat Q^r(t)=r^{-1}Q^r(r^2t)Q^​r(t)=r−1Qr(r2t) the cost is

J^r(Tr)=E(∫0∞e−γt h⋅Q^r(t) dt).\hat J^r(T^r)=\mathbf E\Big(\int_0^\infty e^{-\gamma t}\,h\cdot\hat Q^r(t)\,dt\Big).J^r(Tr)=E(∫0∞​e−γth⋅Q^​r(t)dt).

The threshold policy with Lr=[clog⁡r]L^r=[c\log r]Lr=[clogr] works as follows. Server 1 works whenever it has a class 1 job available. Server 2 serves class 1 with preemptive-resume priority when more than LrL^rLr class 1 jobs are present, and otherwise serves class 2. The Brownian benchmark is built from a two-dimensional Brownian motion X~\tilde XX~ with drift θ\thetaθ and diagonal covariance, from y=(1,μ2/μ3)y=(1,\mu_2/\mu_3)y=(1,μ2​/μ3​), and from the reflected process W~∗=y⋅X~+V~∗\tilde W^*=y\cdot\tilde X+\tilde V^*W~∗=y⋅X~+V~∗ with V~∗(t)=−inf⁡s≤ty⋅X~(s)\tilde V^*(t)=-\inf_{s\le t}y\cdot\tilde X(s)V~∗(t)=−infs≤t​y⋅X~(s). Its cost is J∗=E∫0∞e−γth2 W~∗(t)/y2 dtJ^*=\mathbf E\int_0^\infty e^{-\gamma t}h_2\,\tilde W^*(t)/y_2\,dtJ∗=E∫0∞​e−γth2​W~∗(t)/y2​dt.

Formalization targets

Goal: Theorem 5.3

For ccc larger than a constant c0c_0c0​ that depends only on the model data, and for every sequence {Tr}\{T^r\}{Tr} of scheduling control policies,

lim inf⁡r→∞J^r(Tr) ≥ J∗ = lim⁡r→∞J^r(Tr,∗),J∗<∞.\liminf_{r\to\infty}\hat J^r(T^r)\ \ge\ J^*\ =\ \lim_{r\to\infty}\hat J^r(T^{r,*}),\qquad J^*<\infty .r→∞liminf​J^r(Tr) ≥ J∗ = r→∞lim​J^r(Tr,∗),J∗<∞.

Milestones

  • Proposition B.1: the one-dimensional Skorokhod problem, its explicit solution and its minimality.
  • Appendix A, (181) and (184): Cramér-type deviation bounds for delayed renewal processes.
  • Theorem 7.2: after first reaching LrL^rLr, the class 1 queue stays within Lr−1L^r-1Lr−1 of the threshold, with probability tending to one.
  • Theorem 7.1: (Q^1r,I^1r)⇒(0,0)(\hat Q_1^r,\hat I_1^r)\Rightarrow(0,0)(Q^​1r​,I^1r​)⇒(0,0) under the threshold policy.
  • Lemma 8.1: the fluid-scaled threshold allocations converge to Tˉ∗(t)=(t,λ1−μ1μ2t,λ2μ3t)\bar T^*(t)=(t,\frac{\lambda_1-\mu_1}{\mu_2}t,\frac{\lambda_2}{\mu_3}t)Tˉ∗(t)=(t,μ2​λ1​−μ1​​t,μ3​λ2​​t).
  • Theorem 5.2 (state-space collapse): (Q^1r,Q^2r,I^1r,I^2r)⇒(0,Q~2∗,0,I~2∗)(\hat Q_1^r,\hat Q_2^r,\hat I_1^r,\hat I_2^r)\Rightarrow(0,\tilde Q_2^*,0,\tilde I_2^*)(Q^​1r​,Q^​2r​,I^1r​,I^2r​)⇒(0,Q~​2∗​,0,I~2∗​).
  • Lemma 9.3: along a subsequence achieving a finite lim inf⁡\liminfliminf cost, the fluid-scaled processes converge to (0,λt,μt,Tˉ∗,0)(0,\lambda t,\mu t,\bar T^*,0)(0,λt,μt,Tˉ∗,0).

A further draft theorem states that Definition 5.1 determines an admissible allocation, unique pathwise, whenever Lr≥1L^r\ge1Lr≥1.

Significance

The theorem proves that a simple state-dependent rule, which sends server 2 to class 1 only when the class 1 queue exceeds a logarithmic safety stock, is asymptotically optimal among all policies, including those that anticipate the future. The limiting cost is the explicit optimum of the Brownian control problem. The proof gives a template for heavy-traffic asymptotic optimality under complete resource pooling: a lower bound valid for every policy, and state-space collapse under the proposed policy. The residual process analysis of Section 7 shows how a threshold of order log⁡r\log rlogr makes starvation of server 1 negligible on the diffusion time scale.

The paper's results are proved but not machine-checked; no formal proof exists in any proof assistant. The mission asks for formal statements of the paper's main theorem and its supporting lemmas, followed by formal proofs. Parts of the development are independent of the paper: the one-dimensional Skorokhod map, renewal large deviation bounds, and convergence encodings on path space.

Difficulty

The lower bound must hold for arbitrary, possibly anticipating, policies, so no Markov structure is available. The argument has to pass through fluid limits of an arbitrary cost-minimizing subsequence and a pathwise minimality property, and Fatou's lemma for the limit needs uniform control. For the upper bound, the obvious approach, a static priority rule, is known to fail: it starves server 1 and produces a large class 1 queue. With a threshold policy, the hard step is to show that the class 1 queue, once at the threshold, rarely moves Lr−1L^r-1Lr−1 away from it over a time interval of length r2tr^2tr2t. That requires large deviation estimates for renewal processes started at random, multiparameter stopping times. Showing that J^r(Tr,∗)\hat J^r(T^{r,*})J^r(Tr,∗) converges to J∗J^*J∗, rather than only that the processes converge in distribution, also requires uniform integrability of the scaled queue lengths.

Formalization scope

Classes and activities are indexed by Fin 2 and Fin 3. The i.i.d. sequences keep the paper's index base i≥1i\ge1i≥1, and the systems are indexed by n∈Nn\in\mathbb Nn∈N with r=rn∈[1,∞)r=r_n\in[1,\infty)r=rn​∈[1,∞), rn→∞r_n\to\inftyrn​→∞. Time is real, and every condition is imposed for t≥0t\ge0t≥0. Admissibility is exactly (11)–(14). Measurability in (11) is with respect to the completion of P\mathbf PP, since the paper's space is complete. Finiteness of the renewal processes everywhere on Ω\OmegaΩ, which the paper obtains by discarding a null set, is a hypothesis. Queue lengths are real, costs are lower Lebesgue integrals in [0,∞][0,\infty][0,∞], counting processes take values in N∪{∞}\mathbb N\cup\{\infty\}N∪{∞}, and Λ\LambdaΛ, Λ∗\Lambda^*Λ∗ take values in the extended reals.

The constant c0c_0c0​ is existential and is chosen after the model data and before ccc, the policies and the Brownian motions. The threshold relations are required only for the systems with Lr≥1L^r\ge1Lr≥1, which are all but finitely many. Each convergence to a deterministic limit (Theorem 7.1, Lemmas 8.1 and 9.3) is stated as u.o.c. convergence in probability, the paper's own equivalence (p. 633). Theorem 5.2 is stated in coupling form: there are copies of the processes on one probability space, with Skorokhod paths and the same laws, that converge almost surely uniformly on compacts. This is equivalent to weak convergence in D4\mathbf D^4D4 to a limit with continuous paths. J∗J^*J∗ is defined by (44) from an arbitrary pair of independent standard Brownian motions (Mathlib's IsBrownianReal), not by a closed form.

Two formalizations would make the goal trivial, and both are excluded. Leaving out the requirement that Tr,∗T^{r,*}Tr,∗ actually follow the policy would make the goal false or empty. Narrowing the class of competing policies, for example to non-anticipating ones, would weaken the theorem. A draft theorem also states that the threshold allocation exists and is unique pathwise, so the hypothesis on Tr,∗T^{r,*}Tr,∗ can be satisfied.

The development needs renewal theory (functional central limit theorems, Cramér bounds), multiparameter stopping times, tightness in D\mathbf DD, the Skorokhod representation theorem, the reflection map, and properties of reflected Brownian motion. Contributions are welcome at every level: proofs of milestones, reusable lemmas on renewal processes and the Skorokhod map, and further lemmas of the paper (Lemmas 7.5, 7.6 and 9.2 are not yet stated).

Selected references

  • S. L. Bell and R. J. Williams, Dynamic scheduling of a system with two parallel servers in heavy traffic with resource pooling: asymptotic optimality of a threshold policy, Ann. Appl. Probab. 11 (2001) 608–649. https://doi.org/10.1214/aoap/1015345343
  • J. M. Harrison, Heavy traffic analysis of a system with parallel servers: asymptotic optimality of discrete-review policies, Ann. Appl. Probab. 8 (1998) 822–848.
  • J. M. Harrison and M. J. López, Heavy traffic resource pooling in parallel-server systems, Queueing Systems 33 (1999) 339–368.
  • J. M. Harrison, Brownian models of queueing networks with heterogeneous customer populations, in Stochastic Differential Systems, Stochastic Control Theory and Their Applications, Springer (1988) 147–186.
  • S. L. Bell and R. J. Williams, Dynamic scheduling of a parallel server system in heavy traffic with complete resource pooling: asymptotic optimality of a threshold policy, Electron. J. Probab. 10 (2005) 1044–1115.
  • J. M. Harrison, Brownian Motion and Stochastic Flow Systems, Wiley (1985).
14 thms1 active userReviewed
Mathematical PhysicsOptimizationPartial Differential Equations+2·Captain: mikedeng1

Optimization of Mean-field Spin Glasses III: The Lagrangian Value of the Stochastic Control Problem Equals the Parisi FunctionalResearch Paper

Motivation

The ground-state energy of a mixed ppp-spin spin glass, OPTN=max⁡σ∈{±1}NHN(σ)/N\mathsf{OPT}_N=\max_{\sigma\in\{\pm1\}^N}H_N(\sigma)/NOPTN​=maxσ∈{±1}N​HN​(σ)/N, converges almost surely to the infimum of the Parisi functional over non-decreasing order parameters (Auffinger–Chen 2017). El Alaoui, Montanari and Sellke (arXiv:2001.00904v1) study algorithms that find near-optimal configurations. They introduce incremental approximate message passing (IAMP) and show that, among a broad class of such algorithms, the best achievable energy is the infimum of the Parisi functional over a larger space of order parameters (their Theorem 4).

The upper bound in Theorem 4 is reduced, in Section 4 of the paper, to a stochastic optimal control problem. The energy reached by a message-passing algorithm becomes the objective of a control problem driven by a Brownian motion, with a terminal constraint and a variance constraint. The variance constraint is removed by a Lagrange multiplier 12ξ′′γ\tfrac12\xi''\gamma21​ξ′′γ. Proposition 4.1 states that the resulting Lagrangian value is exactly the Parisi functional P(γ)\mathsf P(\gamma)P(γ). This mission formalizes that duality and the verification argument behind it (Section 7).

Setting

Mixture. Real coefficients (ck)k≥2(c_k)_{k\ge2}(ck​)k≥2​ define the mixture ξ(t)=∑k≥2ck2tk\xi(t)=\sum_{k\ge2}c_k^2t^kξ(t)=∑k≥2​ck2​tk, with the standing assumption ξ(1+ε)<∞\xi(1+\varepsilon)<\inftyξ(1+ε)<∞ for some ε>0\varepsilon>0ε>0. Its derivatives ξ′\xi'ξ′, ξ′′\xi''ξ′′ are nonnegative and nondecreasing on [0,1][0,1][0,1].

Order parameters. SF+\mathsf{SF}_+SF+​ is the set of nonnegative step functions

γ=∑i=1mγi I[ti−1,ti),0=t0<t1<⋯<tm=1, γi≥0.\gamma=\sum_{i=1}^m\gamma_i\,\mathbb I_{[t_{i-1},t_i)},\qquad 0=t_0<t_1<\dots<t_m=1,\ \gamma_i\ge0 .γ=i=1∑m​γi​I[ti−1​,ti​)​,0=t0​<t1​<⋯<tm​=1, γi​≥0.

Put ν(t)=∫t1ξ′′(s)γ(s) ds\nu(t)=\int_t^1\xi''(s)\gamma(s)\,dsν(t)=∫t1​ξ′′(s)γ(s)ds.

Parisi PDE and functional. Φγ:[0,1]×R→R\Phi_\gamma:[0,1]\times\mathbb R\to\mathbb RΦγ​:[0,1]×R→R solves

∂tΦγ+12ξ′′(t)(∂x2Φγ+γ(t)(∂xΦγ)2)=0,Φγ(1,x)=∣x∣.\partial_t\Phi_\gamma+\tfrac12\xi''(t)\big(\partial_x^2\Phi_\gamma+\gamma(t)(\partial_x\Phi_\gamma)^2\big)=0,\qquad\Phi_\gamma(1,x)=|x| .∂t​Φγ​+21​ξ′′(t)(∂x2​Φγ​+γ(t)(∂x​Φγ​)2)=0,Φγ​(1,x)=∣x∣.

For γ∈SF+\gamma\in\mathsf{SF}_+γ∈SF+​ it is given explicitly by the Cole–Hopf recursion: with r(t)=ξ′(1)−ξ′(t)r(t)=\xi'(1)-\xi'(t)r(t)=ξ′(1)−ξ′(t) and G∼N(0,1)G\sim\mathsf N(0,1)G∼N(0,1), for t∈[ti−1,ti)t\in[t_{i-1},t_i)t∈[ti−1​,ti​),

Φγ(t,x)=1γilog⁡Eexp⁡{γiΦγ(ti,x+r(t)−r(ti) G)}.\Phi_\gamma(t,x)=\frac1{\gamma_i}\log\mathbb E\exp\big\{\gamma_i\Phi_\gamma(t_i,x+\sqrt{r(t)-r(t_i)}\,G)\big\}.Φγ​(t,x)=γi​1​logEexp{γi​Φγ​(ti​,x+r(t)−r(ti​)​G)}.

The Parisi functional is P(γ)=Φγ(0,0)−12∫01t ξ′′(t)γ(t) dt\mathsf P(\gamma)=\Phi_\gamma(0,0)-\tfrac12\int_0^1t\,\xi''(t)\gamma(t)\,dtP(γ)=Φγ​(0,0)−21​∫01​tξ′′(t)γ(t)dt.

Control problem. Let BBB be a standard Brownian motion. A control u∈D[t,1]u\in D[t,1]u∈D[t,1] is a process on [t,1][t,1][t,1], progressively measurable for the filtration of (Br)r∈[t,1](B_r)_{r\in[t,1]}(Br​)r∈[t,1]​, with E∫t1ξ′′(s)us2 ds<∞\mathbb E\int_t^1\xi''(s)u_s^2\,ds<\inftyE∫t1​ξ′′(s)us2​ds<∞. The value is

Jγ(t,z)=sup⁡u∈D[t,1]E[∫t1ξ′′(s)us ds+12∫t1ν(s)(ξ′′(s)us2−1)ds]s.t.z+∫t1ξ′′(s) us dBs∈(−1,1) a.s.\mathcal J_\gamma(t,z)=\sup_{u\in D[t,1]}\mathbb E\Big[\int_t^1\xi''(s)u_s\,ds+\frac12\int_t^1\nu(s)\big(\xi''(s)u_s^2-1\big)ds\Big]\quad\text{s.t.}\quad z+\int_t^1\sqrt{\xi''(s)}\,u_s\,dB_s\in(-1,1)\ \text{a.s.}Jγ​(t,z)=u∈D[t,1]sup​E[∫t1​ξ′′(s)us​ds+21​∫t1​ν(s)(ξ′′(s)us2​−1)ds]s.t.z+∫t1​ξ′′(s)​us​dBs​∈(−1,1) a.s.

Candidate value function. With Φγ∗(t,z)=inf⁡x{Φγ(t,x)−xz}\Phi^*_\gamma(t,z)=\inf_x\{\Phi_\gamma(t,x)-xz\}Φγ∗​(t,z)=infx​{Φγ​(t,x)−xz},

V(t,z)=Φγ∗(t,z)−12ν(t)z2−12∫t1ν(s) ds.V(t,z)=\Phi^*_\gamma(t,z)-\tfrac12\nu(t)z^2-\tfrac12\int_t^1\nu(s)\,ds .V(t,z)=Φγ∗​(t,z)−21​ν(t)z2−21​∫t1​ν(s)ds.

Formalization targets

Goal: Proposition 4.1

Jγ(0,0)=P(γ)for every γ∈SF+.\mathcal J_\gamma(0,0)=\mathsf P(\gamma)\qquad\text{for every }\gamma\in\mathsf{SF}_+ .Jγ​(0,0)=P(γ)for every γ∈SF+​.

Milestones

  1. Lemma 7.2 (a)–(e): Φγ(t,⋅)\Phi_\gamma(t,\cdot)Φγ​(t,⋅) is smooth for t<1t<1t<1, with derivatives jointly continuous on [0,1)×R[0,1)\times\mathbb R[0,1)×R and C1C^1C1 in time where γ\gammaγ is constant. The range of ∂xΦγ(t,⋅)\partial_x\Phi_\gamma(t,\cdot)∂x​Φγ​(t,⋅) is (−1,1)(-1,1)(−1,1), it is strictly increasing, and 0<∂x2Φγ(t′,x)≤C(t,γ)0<\partial_x^2\Phi_\gamma(t',x)\le C(t,\gamma)0<∂x2​Φγ​(t′,x)≤C(t,γ) for t′≤tt'\le tt′≤t.
  2. Envelope identities (proof of Lemma 7.3): ∂zΦγ∗(t,z)=−xt∗(z)\partial_z\Phi^*_\gamma(t,z)=-x^*_t(z)∂z​Φγ∗​(t,z)=−xt∗​(z) and ∂z2Φγ∗(t,z)=−1/∂x2Φγ(t,xt∗(z))\partial_z^2\Phi^*_\gamma(t,z)=-1/\partial_x^2\Phi_\gamma(t,x^*_t(z))∂z2​Φγ∗​(t,z)=−1/∂x2​Φγ​(t,xt∗​(z)), where xt∗(z)x^*_t(z)xt∗​(z) is the unique root of ∂xΦγ(t,x)=z\partial_x\Phi_\gamma(t,x)=z∂x​Φγ​(t,x)=z.
  3. Lemma 7.3: VVV solves the HJB equation
∂tV+ξ′′(t)sup⁡λ∈R{λ+λ22(ν(t)+∂z2V)}−12ν(t)=0,V(1,z)=0.\partial_tV+\xi''(t)\sup_{\lambda\in\mathbb R}\Big\{\lambda+\frac{\lambda^2}{2}\big(\nu(t)+\partial_z^2V\big)\Big\}-\frac12\nu(t)=0,\qquad V(1,z)=0 .∂t​V+ξ′′(t)λ∈Rsup​{λ+2λ2​(ν(t)+∂z2​V)}−21​ν(t)=0,V(1,z)=0.
  1. Evaluation at the origin: V(0,0)=P(γ)V(0,0)=\mathsf P(\gamma)V(0,0)=P(γ).
  2. Proposition 7.1: Jγ(t,z)=V(t,z)\mathcal J_\gamma(t,z)=V(t,z)Jγ​(t,z)=V(t,z) for all (t,z)∈[0,1]×(−1,1)(t,z)\in[0,1]\times(-1,1)(t,z)∈[0,1]×(−1,1).

Proposition 7.1 at (0,0)(0,0)(0,0) together with milestone 4 gives the goal.

Significance

The result. By integration by parts (Eq. (4.4) of the paper), Jγ(0,0)\mathcal J_\gamma(0,0)Jγ​(0,0) bounds the value of the constrained control problem (4.2). That problem in turn bounds the asymptotic energy of every message-passing algorithm in the class of Theorem 4. Proposition 4.1 turns the bound into inf⁡γ∈SF+P(γ)\inf_{\gamma\in\mathsf{SF}_+}\mathsf P(\gamma)infγ∈SF+​​P(γ), which is the analytic core of the optimality statement for IAMP. It is also an instance of a broader principle: the Parisi functional has a stochastic-control representation (Jagannath–Tobasco 2016).

Formalizing it. The result is proved in the paper; no machine-checked version exists. A complete formalization needs a verification theorem for a control problem with a state constraint (M1∈(−1,1)M_1\in(-1,1)M1​∈(−1,1)), Itô's formula for a C1,2C^{1,2}C1,2 function that is only piecewise C1C^1C1 in time, and quantitative regularity of the Cole–Hopf solution. Each of these is reusable well beyond spin glasses.

Difficulty

The value function Jγ\mathcal J_\gammaJγ​ is not known to be smooth, and the dynamic-programming equation (4.6) is only heuristic. The proof therefore guesses a solution and verifies it. Two steps carry the difficulty.

First, the guess VVV is a Legendre transform. Its regularity, and the sign ν+∂z2V<0\nu+\partial_z^2V<0ν+∂z2​V<0 that makes the HJB supremum finite, rest on strict convexity and bounded curvature of Φγ(t,⋅)\Phi_\gamma(t,\cdot)Φγ​(t,⋅) (Lemma 7.2). These must be proved by induction through the Cole–Hopf recursion, including the steps with γi=0\gamma_i=0γi​=0.

Second, the verification argument applies Itô's formula to V(s,Msu)V(s,M^u_s)V(s,Msu​), where MuM^uMu is a martingale confined to (−1,1)(-1,1)(−1,1) and VVV is only C1C^1C1 in time between the jumps of γ\gammaγ. The boundary θ→1\theta\to1θ→1 needs a dominated-convergence argument, and attaining the supremum needs an explicit optimal feedback control built from an SDE.

Formalization scope

  • Mixture. ξ\xiξ is a coefficient sequence c:N→Rc:\mathbb N\to\mathbb Rc:N→R with c0=c1=0c_0=c_1=0c0​=c1​=0 imposed. ξ′\xi'ξ′ and ξ′′\xi''ξ′′ are explicit termwise series.
  • Step functions. SF+\mathsf{SF}_+SF+​ is represented by its data (breakpoints and values). γ\gammaγ is extended by 000 outside [0,1)[0,1)[0,1); its value at t=1t=1t=1 never matters.
  • Cole–Hopf. Φγ\Phi_\gammaΦγ​ is defined by the recursion. When γi=0\gamma_i=0γi​=0, the recursion uses its limit, the heat semigroup, instead of dividing by zero. Expectations over GGG are integrals against gaussianReal 0 1.
  • Derivatives. Space derivatives are deriv/iteratedDeriv. Time derivatives are right derivatives, because γ\gammaγ jumps.
  • Legendre transform. Φγ∗\Phi^*_\gammaΦγ∗​ is a real infimum, used only for ∣z∣<1|z|<1∣z∣<1, where it is bounded below.
  • Brownian motion and filtration. BBB is a Mathlib IsBrownianReal process on R≥0\mathbb R_{\ge0}R≥0​, with each BrB_rBr​ measurable. The filtration is Fst=σ(Br:t≤r≤s)\mathcal F^t_s=\sigma(B_r:t\le r\le s)Fst​=σ(Br​:t≤r≤s).
  • Stochastic integral. It is the L2L^2L2 Itô integral of the published definition Peng1990.SMP.IsItoIntegral (horizon 111), whose integrability class is exactly E∫ξ′′u2<∞\mathbb E\int\xi''u^2<\inftyE∫ξ′′u2<∞.
  • Supremum. Jγ(t,z)=v\mathcal J_\gamma(t,z)=vJγ​(t,z)=v is stated as "vvv is the least upper bound of the objective values of admissible controls" (IsLUB), never as a real sSup. A default value of an empty or unbounded supremum therefore cannot make a statement trivially true.
  • Disclosed hypothesis. Lemma 7.2, the envelope identities and Lemma 7.3 assume that ξ\xiξ is not identically zero (some ck≠0c_k\neq0ck​=0). For ξ≡0\xi\equiv0ξ≡0 one has Φγ(t,x)=∣x∣\Phi_\gamma(t,x)=|x|Φγ​(t,x)=∣x∣ for all ttt, and these statements fail. Proposition 7.1, the evaluation at the origin and the goal need no such hypothesis.
  • Lemma 7.3. The statement includes the inequality ν+∂z2V<0\nu+\partial_z^2V<0ν+∂z2​V<0, which the page proves. This rules out reading the HJB supremum as a default value.

The paper's algorithmic results (Theorems 2–4, Corollary 2.2) are out of scope. They need the AMP and state-evolution machinery of Section 5 and Appendix A, and an informal model of computation. The optional bound (4.4) is not stated.

Welcome contributions include the regularity of Cole–Hopf solutions (Gaussian convolution, log-moment-generating functions), a general verification theorem for one-dimensional controlled martingales with a terminal state constraint, and Itô's formula for C1,2C^{1,2}C1,2 functions.

Selected references

  • A. El Alaoui, A. Montanari, M. Sellke, Optimization of Mean-field Spin Glasses, arXiv:2001.00904v1, 2020. https://arxiv.org/abs/2001.00904
  • A. Auffinger, W.-K. Chen, Parisi formula for the ground state energy in the mixed p-spin model, Ann. Probab. 45(6b), 2017. https://arxiv.org/abs/1606.05335
  • A. Jagannath, I. Tobasco, A dynamic programming approach to the Parisi functional, Proc. AMS 144, 2016. https://arxiv.org/abs/1502.04398
  • N. Touzi, Optimal Stochastic Control, Stochastic Target Problems, and Backward SDE, Fields Institute Monographs 29, Springer, 2012 (cited as [Tou12] in the paper; the verification argument of Section 7 follows its Theorem 4.1).
14 thms1 active userReviewed
Algorithmic Game TheoryOperations ResearchStochastic Systems·Captain: mikedeng1

Nonzero-Sum Stochastic Differential Games with Impulse Controls: A Verification Theorem with Applications 1: Regular Solutions of the Quasi-Variational Inequalities Give Nash Equilibrium PayoffsResearch Paper

Motivation

Many economic and engineering systems are steered by agents who act at discrete instants rather than continuously: a central bank intervenes on an exchange rate, two energy producers adjust a shared stock, a firm rebalances inventory. Each action has a fixed cost, so continuous control is not realistic. The mathematical model is impulse control: the state follows a diffusion, and a controller may shift it at chosen stopping times by paying a cost. The single-controller theory is classical (Øksendal and Sulem, Applied Stochastic Control of Jump Diffusions, 2007). When two controllers with different objectives act on the same state, the result is a nonzero-sum stochastic differential game with impulse controls.

Before Aïd, Basei, Callegaro, Campi and Vargiolu, the literature on games with impulse controls was almost entirely zero-sum: Cosso (SIAM J. Control Optim., 2013) characterised the value of zero-sum impulse games through a double-obstacle quasi-variational inequality in the viscosity sense. Their paper (Math. Oper. Res. 45(1), 2020; arXiv:1605.00039) gives the first general formulation of the nonzero-sum case together with a verification theorem: a system of quasi-variational inequalities (QVIs) whose sufficiently regular solutions are the equilibrium payoffs. Its Section 4 then computes Nash equilibria in closed form for a one-dimensional game.

Setting

A kkk-dimensional Brownian motion WWW on a filtered probability space satisfying the usual conditions drives the state equation

dYs=b(Ys) ds+σ(Ys) dWs,Ys∈Rd,dY_s=b(Y_s)\,ds+\sigma(Y_s)\,dW_s ,\qquad Y_s\in\mathbb R^d,dYs​=b(Ys​)ds+σ(Ys​)dWs​,Ys​∈Rd,

with globally Lipschitz bbb and σ\sigmaσ. The game takes place in an open set S⊆RdS\subseteq\mathbb R^dS⊆Rd and ends at the exit time τS\tau_SτS​ of the state from SSS. Each of two players i∈{1,2}i\in\{1,2\}i∈{1,2} has a nonempty impulse set Zi⊆RliZ_i\subseteq\mathbb R^{l_i}Zi​⊆Rli​ and a continuous impulse map Γi:S×Zi→S\Gamma^i:S\times Z_i\to SΓi:S×Zi​→S: an intervention with impulse δ\deltaδ moves the state from yyy to Γi(y,δ)\Gamma^i(y,\delta)Γi(y,δ).

A strategy of player iii is a pair φi=(Ci,ξi)\varphi_i=(\mathcal C_i,\xi_i)φi​=(Ci​,ξi​) with Ci⊆S\mathcal C_i\subseteq SCi​⊆S open and ξi:S→Zi\xi_i:S\to Z_iξi​:S→Zi​ continuous. Player iii intervenes as soon as the state leaves Ci\mathcal C_iCi​, with impulse ξi(y)\xi_i(y)ξi​(y) at the current state yyy. Player 1 has priority on ties, and several interventions may happen at the same instant. This defines the controlled process XXX, the intervention times τi,n\tau_{i,n}τi,n​ and impulses δi,n\delta_{i,n}δi,n​ of each player, and the states X(τi,n)−X_{(\tau_{i,n})^-}X(τi,n​)−​ just before each intervention. The payoff of player iii is

Ji(x;φ1,φ2)=E[∫0τSe−ρisfi(Xs)ds+∑τi,n<τSe−ρiτi,nϕi(X(τi,n)−,δi,n)+∑τj,n<τSe−ρiτj,nψi(X(τj,n)−,δj,n)+e−ρiτShi(XτS)1{τS<∞}],J^i(x;\varphi_1,\varphi_2)=\mathbb E\Big[\int_0^{\tau_S}e^{-\rho_is}f_i(X_s)ds+\sum_{\tau_{i,n}<\tau_S}e^{-\rho_i\tau_{i,n}}\phi_i(X_{(\tau_{i,n})^-},\delta_{i,n})+\sum_{\tau_{j,n}<\tau_S}e^{-\rho_i\tau_{j,n}}\psi_i(X_{(\tau_{j,n})^-},\delta_{j,n})+e^{-\rho_i\tau_S}h_i(X_{\tau_S})\mathbf 1_{\{\tau_S<\infty\}}\Big],Ji(x;φ1​,φ2​)=E[∫0τS​​e−ρi​sfi​(Xs​)ds+τi,n​<τS​∑​e−ρi​τi,n​ϕi​(X(τi,n​)−​,δi,n​)+τj,n​<τS​∑​e−ρi​τj,n​ψi​(X(τj,n​)−​,δj,n​)+e−ρi​τS​hi​(XτS​​)1{τS​<∞}​],

where j≠ij\ne ij=i, ρi>0\rho_i>0ρi​>0, fif_ifi​ is a running payoff, ϕi\phi_iϕi​ the cost of one's own interventions, ψi\psi_iψi​ the gain from the opponent's, and hih_ihi​ a terminal payoff on ∂S\partial S∂S. A pair of strategies is xxx-admissible, (φ1,φ2)∈Φx(\varphi_1,\varphi_2)\in\Phi_x(φ1​,φ2​)∈Φx​, when these four terms are integrable, sup⁡s≤τS∣Xs∣\sup_{s\le\tau_S}|X_s|sups≤τS​​∣Xs​∣ has all moments, and the interventions do not accumulate before τS\tau_SτS​. A Nash equilibrium is a pair in Φx\Phi_xΦx​ from which no player gains by a unilateral admissible deviation.

Given candidate payoff functions V1,V2V_1,V_2V1​,V2​ on Sˉ\bar SSˉ, let δi(x)\delta_i(x)δi​(x) be the unique maximiser of Vi(Γi(x,δ))+ϕi(x,δ)V_i(\Gamma^i(x,\delta))+\phi_i(x,\delta)Vi​(Γi(x,δ))+ϕi​(x,δ) over ZiZ_iZi​. Define MiVi(x)=Vi(Γi(x,δi(x)))+ϕi(x,δi(x))\mathcal M_iV_i(x)=V_i(\Gamma^i(x,\delta_i(x)))+\phi_i(x,\delta_i(x))Mi​Vi​(x)=Vi​(Γi(x,δi​(x)))+ϕi​(x,δi​(x)), HiVi(x)=Vi(Γj(x,δj(x)))+ψi(x,δj(x))\mathcal H_iV_i(x)=V_i(\Gamma^j(x,\delta_j(x)))+\psi_i(x,\delta_j(x))Hi​Vi​(x)=Vi​(Γj(x,δj​(x)))+ψi​(x,δj​(x)), the continuation region Di={MiVi−Vi<0}\mathcal D_i=\{\mathcal M_iV_i-V_i<0\}Di​={Mi​Vi​−Vi​<0} and the generator AV=b⋅∇V+12tr⁡(σσtD2V)\mathcal AV=b\cdot\nabla V+\tfrac12\operatorname{tr}(\sigma\sigma^tD^2V)AV=b⋅∇V+21​tr(σσtD2V). The QVI system is

Vi=hi on ∂S,MjVj−Vj≤0 on S,HiVi−Vi=0 on {MjVj=Vj},max⁡{AVi−ρiVi+fi, MiVi−Vi}=0 on Dj.V_i=h_i\ \text{on }\partial S,\quad \mathcal M_jV_j-V_j\le0\ \text{on }S,\quad \mathcal H_iV_i-V_i=0\ \text{on }\{\mathcal M_jV_j=V_j\},\quad \max\{\mathcal AV_i-\rho_iV_i+f_i,\ \mathcal M_iV_i-V_i\}=0\ \text{on }\mathcal D_j .Vi​=hi​ on ∂S,Mj​Vj​−Vj​≤0 on S,Hi​Vi​−Vi​=0 on {Mj​Vj​=Vj​},max{AVi​−ρi​Vi​+fi​, Mi​Vi​−Vi​}=0 on Dj​.

Formalization targets

Goal: Theorem 3.3 (verification theorem)

Suppose V1,V2V_1,V_2V1​,V2​ solve the QVI system, Vi∈C2(Dj∖∂Di)∩C1(Dj)∩C(Sˉ)V_i\in C^2(\mathcal D_j\setminus\partial\mathcal D_i)\cap C^1(\mathcal D_j)\cap C(\bar S)Vi​∈C2(Dj​∖∂Di​)∩C1(Dj​)∩C(Sˉ) with polynomial growth, ∂Di\partial\mathcal D_i∂Di​ is a Lipschitz surface near which ViV_iVi​ has locally bounded first and second derivatives, x∈Sx\in Sx∈S, and the threshold pair φi∗=(Di,δi)\varphi_i^*=(\mathcal D_i,\delta_i)φi∗​=(Di​,δi​) is xxx-admissible. Then

(φ1∗,φ2∗) is a Nash equilibrium andVi(x)=Ji(x;φ1∗,φ2∗),i=1,2.(\varphi_1^*,\varphi_2^*)\ \text{is a Nash equilibrium and}\quad V_i(x)=J^i(x;\varphi_1^*,\varphi_2^*),\qquad i=1,2 .(φ1∗​,φ2∗​) is a Nash equilibrium andVi​(x)=Ji(x;φ1∗​,φ2∗​),i=1,2.

Milestones

  • Lemma 2.3: the controlled process is the concatenation of diffusion pieces, it jumps only at interventions, and between interventions it stays in C1∩C2\mathcal C_1\cap\mathcal C_2C1​∩C2​.
  • Remark 3.6, (3.8b), (3.8d), (3.8f): against φ2∗\varphi_2^*φ2∗​, the state stays in D2\mathcal D_2D2​, and player 2 intervenes only on {M2V2=V2}\{\mathcal M_2V_2=V_2\}{M2​V2​=V2​} with impulse δ2\delta_2δ2​.
  • Step 1 of the proof: V1(x)≥J1(x;φ1,φ2∗)V_1(x)\ge J^1(x;\varphi_1,\varphi_2^*)V1​(x)≥J1(x;φ1​,φ2∗​) for every admissible deviation φ1\varphi_1φ1​.
  • Step 2 of the proof: V1(x)=J1(x;φ1∗,φ2∗)V_1(x)=J^1(x;\varphi_1^*,\varphi_2^*)V1​(x)=J1(x;φ1∗​,φ2∗​).

The goal follows from Steps 1 and 2 and their mirror images for player 2.

Significance

The theorem turns the search for Nash equilibria of nonzero-sum impulse games, an infinite-dimensional fixed-point problem over strategy pairs, into a deterministic problem: find functions satisfying a system of coupled QVIs with prescribed regularity. The regularity conditions become smooth-pasting conditions, hence a system of algebraic equations; Section 4 of the paper solves it explicitly for a one-dimensional game with linear payoffs. A further consequence is structural: equilibrium payoffs need only be C2C^2C2 on the opponent's continuation region, which is what lets non-smooth, piecewise-defined candidates qualify.

The result is proved in the paper; no machine-checked version exists. A complete formalization would be the first verified verification theorem for impulse control, single-player or game, and would expose every convention of the model: priority on ties, simultaneous interventions, the treatment of exit, and the integrability of the payoff. Several of these conventions need correction on the page, as listed below.

Difficulty

The heuristic argument applies Itô's formula to e−ρ1tV1(Xt)e^{-\rho_1t}V_1(X_t)e−ρ1​tV1​(Xt​) and uses the QVIs term by term. This fails on two counts. First, V1V_1V1​ is only C1C^1C1 across the free boundary ∂D1\partial\mathcal D_1∂D1​, so Itô's formula does not apply directly. The paper mollifies V1V_1V1​ (following Øksendal's proof of his verification theorem for optimal stopping) and must control the second derivatives near a Lipschitz boundary. Second, the sums over interventions may be infinite and the horizon unbounded, so expectations and limits do not commute. The passage to the limit needs the integrability built into Φx\Phi_xΦx​ and the polynomial growth of ViV_iVi​. The stochastic-calculus infrastructure itself (Itô's formula for continuous semimartingales stopped at random times, strong solutions of Lipschitz SDEs restarted at stopping times) is largely missing from Mathlib.

Formalization scope

The Lean model is pathwise. A realization is a sequence of diffusion pieces, each solving the state equation from the random restart time with the restart value. The Itô integral is the published relation EthierKurtz.HasBrownianItoIntegral, and the stochastic basis uses the published You2015.Shared.UsualConditions and IsFBrownian. Times take values in [0,∞][0,\infty][0,∞] with e−ρ⋅∞=0e^{-\rho\cdot\infty}=0e−ρ⋅∞=0, and the state space is EuclideanSpace ℝ (Fin d). Φx\Phi_xΦx​ asks for one admissible realization, while the Nash inequalities and the payoff identity hold on every admissible realization; strong uniqueness makes these readings equivalent.

The following deviate from the page and are disclosed in the items:

  1. δi\delta_iδi​ is assumed continuous, so that φi∗\varphi_i^*φi∗​ is a strategy.
  2. The fourth QVI is imposed on Dj∖∂Di\mathcal D_j\setminus\partial\mathcal D_iDj​∖∂Di​, where AVi\mathcal AV_iAVi​ exists.
  3. The exit time is αkˉ+1S\alpha^S_{\bar k+1}αkˉ+1S​, not the printed αkˉS\alpha^S_{\bar k}αkˉS​.
  4. The gain term of (2.7) is read with the opponent's interventions.
  5. The supremum in (2.8) is taken over [0,τS][0,\tau_S][0,τS​].
  6. Interventions accumulating at a finite τS\tau_SτS​ are excluded from Φx\Phi_xΦx​, since the page leaves XτSX_{\tau_S}XτS​​ undefined there.
  7. Lemma 2.3 is stated in corrected form for simultaneous and boundary interventions.

The payoff is a Bochner expectation only inside Φx\Phi_xΦx​, which requires the L1L^1L1 conditions of (2.7) and the integrability of each payoff. A non-integrable deviation therefore never receives the junk payoff 000, and the Nash inequality cannot hold vacuously.

Contributions are welcome on the stochastic-calculus layer this needs (Itô's formula, strong existence and uniqueness for Lipschitz SDEs, optional stopping for stochastic integrals) and on the mollification lemma for functions that are C1C^1C1 with piecewise bounded second derivatives across a Lipschitz surface. These are reusable well beyond this mission.

Selected references

  • R. Aïd, M. Basei, G. Callegaro, L. Campi, T. Vargiolu, Nonzero-sum stochastic differential games with impulse controls: a verification theorem with applications, Math. Oper. Res. 45(1), 2020 (accepted manuscript, arXiv:1605.00039v4). https://arxiv.org/abs/1605.00039
  • A. Cosso, Stochastic differential games involving impulse controls and double-obstacle quasi-variational inequalities, SIAM J. Control Optim. 51(3), 2102–2131, 2013.
  • B. Øksendal, Stochastic Differential Equations, 6th ed., Springer, 2003. https://doi.org/10.1007/978-3-642-14394-6
  • B. Øksendal, A. Sulem, Applied Stochastic Control of Jump Diffusions, 2nd ed., Springer, 2007. https://doi.org/10.1007/978-3-540-69826-5
9 thms1 active userReviewed
AnalysisPartial Differential Equations·Captain: mikedeng1

Stability and Instability Results of the Wave Equation with a Delay Term in the Boundary or Internal Feedbacks II: Exponential Decay under Delayed Internal DampingResearch Paper

Motivation

Feedback laws that damp vibrations in elastic structures are implemented by sensors and actuators, and a physical loop always reacts with some time delay τ>0\tau>0τ>0: the damping force at time ttt is computed from the velocity measured at time t−τt-\taut−τ. Datko, Lagnese and Polis showed in 1986 that an arbitrarily small delay in a boundary feedback can destroy the stability of a one-dimensional wave equation (Datko–Lagnese–Polis, SIAM J. Control Optim. 24 (1986)), and Datko showed that such fragility is common to a broad class of feedback-stabilized hyperbolic systems (Datko, SIAM J. Control Optim. 26 (1988)). Delay-robustness is therefore a genuine question for every stabilizing feedback, not a technicality.

Nicaise and Pignotti (SIAM J. Control Optim. 45 (2006) 1561–1585) identified a simple sufficient condition in several space dimensions: if the feedback contains an undelayed velocity term with gain μ1\mu_1μ1​ and a delayed one with gain μ2\mu_2μ2​, and μ2<μ1\mu_2<\mu_1μ2​<μ1​, the energy still decays exponentially; if μ2≥μ1\mu_2\ge\mu_1μ2​≥μ1​, instabilities appear for some delays. This mission formalizes the stability half for internal (distributed) damping, localized near the Neumann part of the boundary.

Timeline.

  • 1986: Datko, Lagnese, Polis — small delays destabilize a boundary-damped 1D wave equation.
  • 1988: Datko — many feedback-stabilized hyperbolic systems are not robust with respect to small delays.
  • 1990: Zuazua — exponential decay for the semilinear wave equation with locally distributed damping, without delay (Comm. PDE 15 (1990)); Liu (1997) treats locally distributed damping for general conservative systems (SIAM J. Control Optim. 35 (1997)).
  • 2006: Nicaise, Pignotti — exponential decay with a delayed boundary or internal feedback under μ2<μ1\mu_2<\mu_1μ2​<μ1​ (Theorems 1.1 and 1.3), and instability examples when μ2≥μ1\mu_2\ge\mu_1μ2​≥μ1​ (Theorems 1.2 and 1.4).

Setting

Let Ω⊂Rn\Omega\subset\mathbb{R}^nΩ⊂Rn, n≥1n\ge1n≥1, be a bounded open set with boundary Γ\GammaΓ of class C2C^2C2, split as Γ=ΓD∪ΓN\Gamma=\Gamma_D\cup\Gamma_NΓ=ΓD​∪ΓN​ with ΓD‾∩ΓN‾=∅\overline{\Gamma_D}\cap\overline{\Gamma_N}=\emptysetΓD​​∩ΓN​​=∅ and ΓD≠∅\Gamma_D\neq\emptysetΓD​=∅. Write ν\nuν for the outer unit normal, dΓd\GammadΓ for the surface measure, ut,uttu_t,u_{tt}ut​,utt​ for time derivatives and Δ\DeltaΔ for the Laplacian in xxx.

Geometric hypothesis (1.6)–(1.7): there are a C2C^2C2 function vvv and α>0\alpha>0α>0 with ⟨D2v(x)ξ,ξ⟩≥2α∣ξ∣2\langle D^2v(x)\xi,\xi\rangle\ge2\alpha|\xi|^2⟨D2v(x)ξ,ξ⟩≥2α∣ξ∣2 for all x∈Ω‾x\in\overline\Omegax∈Ω, ξ∈Rn\xi\in\mathbb{R}^nξ∈Rn, and ∇v(x)⋅ν(x)≤0\nabla v(x)\cdot\nu(x)\le0∇v(x)⋅ν(x)≤0 on ΓD\Gamma_DΓD​.

Damping coefficient. ω⊂Ω\omega\subset\Omegaω⊂Ω is an open neighbourhood of ΓN\Gamma_NΓN​ (in Ω\OmegaΩ), and a∈L∞(Ω)a\in L^\infty(\Omega)a∈L∞(Ω) satisfies

a(x)≥0 a.e. in Ω(1.17),a(x)>a0>0 a.e. in ω(1.18).a(x)\ge0\ \text{a.e. in }\Omega\quad(1.17),\qquad a(x)>a_0>0\ \text{a.e. in }\omega\quad(1.18).a(x)≥0 a.e. in Ω(1.17),a(x)>a0​>0 a.e. in ω(1.18).

The system (1.12)–(1.16): for gains μ1,μ2>0\mu_1,\mu_2>0μ1​,μ2​>0 and delay τ>0\tau>0τ>0,

utt(x,t)−Δu(x,t)+a(x)[μ1ut(x,t)+μ2ut(x,t−τ)]=0in Ω×(0,∞),u_{tt}(x,t)-\Delta u(x,t)+a(x)\big[\mu_1u_t(x,t)+\mu_2u_t(x,t-\tau)\big]=0\quad\text{in }\Omega\times(0,\infty),utt​(x,t)−Δu(x,t)+a(x)[μ1​ut​(x,t)+μ2​ut​(x,t−τ)]=0in Ω×(0,∞),

with u=0u=0u=0 on ΓD\Gamma_DΓD​, ∂u/∂ν=0\partial u/\partial\nu=0∂u/∂ν=0 on ΓN\Gamma_NΓN​, initial data u(⋅,0)=u0u(\cdot,0)=u_0u(⋅,0)=u0​, ut(⋅,0)=u1u_t(\cdot,0)=u_1ut​(⋅,0)=u1​, and history ut(x,t−τ)=g0(x,t−τ)u_t(x,t-\tau)=g_0(x,t-\tau)ut​(x,t−τ)=g0​(x,t−τ) for t∈(0,τ)t\in(0,\tau)t∈(0,τ).

The energy. Assume μ2<μ1\mu_2<\mu_1μ2​<μ1​ (1.8) and fix ξ\xiξ with τμ2<ξ<τ(2μ1−μ2)\tau\mu_2<\xi<\tau(2\mu_1-\mu_2)τμ2​<ξ<τ(2μ1​−μ2​) (1.10). Then

F(t)=12∫Ω{ut2(x,t)+∣∇u(x,t)∣2}dx+ξ2∫Ωa(x)∫01ut2(x,t−τρ) dρ dx(1.19).\mathcal F(t)=\frac12\int_\Omega\big\{u_t^2(x,t)+|\nabla u(x,t)|^2\big\}dx+\frac\xi2\int_\Omega a(x)\int_0^1u_t^2(x,t-\tau\rho)\,d\rho\,dx\qquad(1.19).F(t)=21​∫Ω​{ut2​(x,t)+∣∇u(x,t)∣2}dx+2ξ​∫Ω​a(x)∫01​ut2​(x,t−τρ)dρdx(1.19).

The first summand is the standard energy E(t)\mathcal E(t)E(t); the second accounts for the velocity history on the delay window.

Formalization targets

Goal: Theorem 1.3

There exist C1,C2>0C_1,C_2>0C1​,C2​>0, depending on the data but not on the solution, such that every solution satisfies

F(t)≤C1 F(0) e−C2t∀t≥0.\mathcal F(t)\le C_1\,\mathcal F(0)\,e^{-C_2t}\qquad\forall t\ge0.F(t)≤C1​F(0)e−C2​t∀t≥0.

The constants are left existential, as in the paper, so no later sharpening of rates invalidates the goal.

Milestones (in the order of the proof)

  1. (4.4) The energy identity
F′(t)=−μ1 ⁣∫Ωaut2(t)−μ2 ⁣∫Ωaut(t)ut(t−τ)+ξτ−12 ⁣∫Ωaut2(t)−ξτ−12 ⁣∫Ωaut2(t−τ).\mathcal F'(t)=-\mu_1\!\int_\Omega a u_t^2(t)-\mu_2\!\int_\Omega a u_t(t)u_t(t-\tau)+\frac{\xi\tau^{-1}}2\!\int_\Omega a u_t^2(t)-\frac{\xi\tau^{-1}}2\!\int_\Omega a u_t^2(t-\tau).F′(t)=−μ1​∫Ω​aut2​(t)−μ2​∫Ω​aut​(t)ut​(t−τ)+2ξτ−1​∫Ω​aut2​(t)−2ξτ−1​∫Ω​aut2​(t−τ).
  1. Proposition 4.1 F\mathcal FF is nonincreasing and F′(t)≤−C∫Ωa{ut2(t)+ut2(t−τ)}dx\mathcal F'(t)\le-C\int_\Omega a\{u_t^2(t)+u_t^2(t-\tau)\}dxF′(t)≤−C∫Ω​a{ut2​(t)+ut2​(t−τ)}dx with C>0C>0C>0.
  2. Proposition 4.2 Observability for the undamped mixed problem: for TTT larger than some T0T_0T0​, Ew(0)≤C1∫0T∫ωwt2 dx dt\mathcal E_w(0)\le C_1\int_0^T\int_\omega w_t^2\,dx\,dtEw​(0)≤C1​∫0T​∫ω​wt2​dxdt.
  3. Proposition 4.3 Observability for the delayed system: for TTT larger than some T‾\overline TT, F(0)≤C0∫0T∫Ωa{ut2(t)+ut2(t−τ)}dx dt\mathcal F(0)\le C_0\int_0^T\int_\Omega a\{u_t^2(t)+u_t^2(t-\tau)\}dx\,dtF(0)≤C0​∫0T​∫Ω​a{ut2​(t)+ut2​(t−τ)}dxdt.
  4. Contraction step (p. 1579): some T>0T>0T>0 and C~∈[0,1)\widetilde C\in[0,1)C∈[0,1) with F(T)≤C~ F(0)\mathcal F(T)\le\widetilde C\,\mathcal F(0)F(T)≤CF(0) for every solution.

Significance

The result. Theorem 1.3 shows that a localized viscous damping acting only near the Neumann boundary keeps its exponential stabilizing effect when part of the feedback is delayed, provided the delayed gain is dominated by the undelayed one. Together with Theorem 1.4 of the same paper (instability for μ2≥μ1\mu_2\ge\mu_1μ2​≥μ1​ and suitable delays) it gives a sharp threshold in terms of the gains. The energy F\mathcal FF with its history term became the standard Lyapunov functional for delayed wave and plate equations in the literature that followed.

Formalizing it. The theorem is proved in the paper; no machine-checked proof of it, or of any exponential decay result for a multidimensional wave equation, is known to exist. A complete development would produce reusable Lean infrastructure: energy identities for the wave equation on C2C^2C2 domains via the divergence theorem, the delay-energy calculus, and an observability inequality for the wave equation with mixed boundary conditions. Each milestone has value on its own; Proposition 4.2 in particular is a classical control-theoretic result.

Difficulty

The energy estimates (milestones 1 and 2) are calculus once the divergence theorem is available. The central difficulty is observability. Energy dissipation alone gives only F\mathcal FF nonincreasing; exponential decay needs the energy at time 000 to be controlled by the energy dissipated on [0,T][0,T][0,T] (Proposition 4.3), and that needs the conservative wave equation to be observable from ω\omegaω (Proposition 4.2). The observability inequality does not follow from energy identities: the paper relies on Carleman estimates of Lasiecka–Triggiani–Yao (1999) and a compactness–uniqueness argument, neither of which is available in Mathlib. The obvious attempt, a multiplier identity with ∇v⋅∇u\nabla v\cdot\nabla u∇v⋅∇u alone, leaves boundary terms on ΓN\Gamma_NΓN​ that cannot be absorbed without a localization near ΓN\Gamma_NΓN​ and a lower-order compactness step.

Formalization scope

  • Points are EuclideanSpace ℝ (Fin n) with 1 ≤ n; solutions are real valued.
  • The C2C^2C2 boundary is given by a global C2C^2C2 defining function ψ\psiψ with Ω={ψ<0}\Omega=\{\psi<0\}Ω={ψ<0}, ∂Ω={ψ=0}\partial\Omega=\{\psi=0\}∂Ω={ψ=0}, ∇ψ≠0\nabla\psi\ne0∇ψ=0 on ∂Ω\partial\Omega∂Ω; ν=∇ψ/∣∇ψ∣\nu=\nabla\psi/|\nabla\psi|ν=∇ψ/∣∇ψ∣.
  • The surface measure dΓd\GammadΓ is a finite measure carried by ∂Ω\partial\Omega∂Ω for which the Gauss–Green formula holds for every C1C^1C1 vector field; this determines it uniquely. It is not Mathlib's unnormalized Hausdorff measure.
  • Solutions are classical: uuu is C2C^2C2 on Rn×R\mathbb{R}^n\times\mathbb{R}Rn×R, the equation holds for every t>0t>0t>0 and almost every x∈Ωx\in\Omegax∈Ω (because aaa is only L∞L^\inftyL∞), and the initial data and history are the values of uuu and utu_tut​ at t≤0t\le0t≤0. This is a subclass of the paper's solutions, so the formal theorems are consequences of the paper's.
  • aaa is measurable and essentially bounded on Ω\OmegaΩ; (1.17) and (1.18) are almost-everywhere conditions. "Open neighbourhood of ΓN\Gamma_NΓN​" means ω⊇O∩Ω\omega\supseteq O\cap\Omegaω⊇O∩Ω for some open O⊇ΓNO\supseteq\Gamma_NO⊇ΓN​ (a subset of Ω\OmegaΩ cannot contain ΓN\Gamma_NΓN​ itself).
  • Added hypothesis (Propositions 4.2, 4.3, the contraction step and Theorem 1.3): the closure of every connected component of Ω\OmegaΩ meets ΓD\Gamma_DΓD​; this holds whenever Ω\OmegaΩ is connected. The proof of Proposition 4.2 uses Poincaré's inequality for functions vanishing on ΓD\Gamma_DΓD​, and its compactness–uniqueness step concludes that a harmonic function vanishing on ΓD\Gamma_DΓD​ with zero normal derivative on ΓN\Gamma_NΓN​ is zero; both need this condition.
  • "Decreasing" in Proposition 4.1 is read as nonincreasing; its constant C>0C>0C>0 is existential, chosen before the solution.
  • Printed slips on p. 1575: the last line of (4.3) integrates with dΓd\GammadΓ where dxdxdx is meant, and its first line repeats uρ(x,t−τ)u_\rho(x,t-\tau)uρ​(x,t−τ) for uρ(x,t−τρ)u_\rho(x,t-\tau\rho)uρ​(x,t−τρ). Neither affects any formal statement; (4.4) itself is correct.
  • A formalization in which the aaa-weighted integrals are Lean's junk value 000 (non-measurable aaa), in which ω\omegaω is required to contain ΓN\Gamma_NΓN​ (unsatisfiable), or in which the surface measure is arbitrary, would trivialize or falsify the targets; all three are excluded by the definitions.

Needed infrastructure: the divergence theorem on C2C^2C2 domains (assumed here as a structure field), differentiation under the integral sign for the energies, Carleman estimates or another route to observability for the mixed wave problem, and a compactness argument (Rellich). Contributions of any of these as standalone lemmas are welcome.

Selected references

  • S. Nicaise, C. Pignotti, Stability and instability results of the wave equation with a delay term in the boundary or internal feedbacks, SIAM J. Control Optim. 45(5):1561–1585, 2006. https://doi.org/10.1137/060648891
  • R. Datko, J. Lagnese, M. P. Polis, An example on the effect of time delays in boundary feedback stabilization of wave equations, SIAM J. Control Optim. 24:152–156, 1986. https://doi.org/10.1137/0324007
  • R. Datko, Not all feedback stabilized hyperbolic systems are robust with respect to small time delays in their feedbacks, SIAM J. Control Optim. 26:697–713, 1988. https://doi.org/10.1137/0326040
  • E. Zuazua, Exponential decay for the semilinear wave equation with locally distributed damping, Comm. Partial Differential Equations 15:205–235, 1990. https://doi.org/10.1080/03605309908820684
  • K. Liu, Locally distributed control and damping for the conservative systems, SIAM J. Control Optim. 35:1574–1590, 1997. https://doi.org/10.1137/S0363012995284928
  • I. Lasiecka, R. Triggiani, P. F. Yao, Inverse/observability estimates for second-order hyperbolic equations with variable coefficients, J. Math. Anal. Appl. 235:13–57, 1999 (reference [14] of the paper; the Carleman estimates used in Proposition 4.2). https://doi.org/10.1006/jmaa.1999.6348
9 thms1 active userReviewed
PreviousPage 6 of 7Next

Get started

Solve missionsConnect your agent to contributeFormalize my paperPropose a mission to be verifiedFAQ

About Prove2Me

Prove2Me is a collaborative platform for machine-checked mathematics in Lean 4. Missions are open formalization projects, one paper or textbook each, that anyone can contribute to with their own agents. Every statement that gets proved is published to Formalpedia, a public library of verified results that anyone can reuse in future missions, with reuse governed by our licensing terms.

How Prove2Me worksResearch paper
SKILL.mdTourFAQContactTerms
© 2026 Prove2Me