Prove2Me
Navigate
DiscoverCollectionsFormalpediaGalaxyBlogsUsersMomentumMy Missions+
Prove2Me
Log in
← Formalpedia

Theorem 2, p. 9 — Expected-Tail-Optimized LRU is an optimal online caching policy for Tail Excess Latency in the stochastic conversation model

Proved
TailOptCaching.ETLRU.theorem_2

by mikedeng1 · Oct 22, 2026 · Mathlib 0df444a (Lean v4.33.1)

cachinglrumarkov-decision-processoptimal-policyp2o-batch-pfp2ap2o-gran-per-chapterp2o-plan-paperp2o-v1tail-latency

Theorem 2 (Optimality of Expected-Tail-Optimized LRU). In the stochastic conversation model — conversations born at rate λconv>0\lambda_{\mathrm{conv}} > 0λconv​>0, returning at rates λˉi>0\bar\lambda_i > 0λˉi​>0 while active, with exponentially distributed lifetimes tracked through belief rates, known prompt-length laws, arbitrary response-length laws, cache capacity CCC and threshold ξ≥0\xi \ge 0ξ≥0 — let σ\sigmaσ be any ET-LRU policy (Definition 1) and let π\piπ be any caching policy whose decisions always satisfy (9), (10) and the capacity constraint. Then for every number MMM of requests and every well-formed starting belief state zzz,

Eσ[∑k=1MTELk]≤Eπ[∑k=1MTELk],\mathbb E^\sigma\Big[\sum_{k=1}^{M} \mathrm{TEL}_k\Big] \le \mathbb E^\pi\Big[\sum_{k=1}^{M} \mathrm{TEL}_k\Big],Eσ[k=1∑M​TELk​]≤Eπ[k=1∑M​TELk​],

where TELk=(Lθ+Q−Xθ−ξ)+\mathrm{TEL}_k = (L_\theta + Q - X_\theta - \xi)^+TELk​=(Lθ​+Q−Xθ​−ξ)+ is the Tail Excess Latency of the kkk-th request.

ET-LRU is myopic: it minimizes only the expected TEL of the next turn. The theorem says that, in this model, myopia loses nothing over any horizon. With ξ=0\xi = 0ξ=0 and homogeneous rates it specializes to the optimality of LRU for average latency.

Formalization Note. Policies are Markov and deterministic: they map the stage, the belief state and the current arrival to a cache vector. The well-formedness hypothesis (0<λi≤λˉi0 < \lambda_i \le \bar\lambda_i0<λi​≤λˉi​ for arrived conversations, Xi≤LiX_i \le L_iXi​≤Li​, ∑iXi≤C\sum_i X_i \le C∑i​Xi​≤C, nothing beyond the arrived conversations) holds for the empty start and for every reachable state; it cannot be dropped, since the claim fails from some states with λi>λˉi\lambda_i > \bar\lambda_iλi​>λˉi​. The law of the decay factor e−μτe^{-\mu\tau}e−μτ is generalized to any probability law on (0,1](0,1](0,1] depending on the belief rates, which contains the paper's law. Values are in [0,∞][0,\infty][0,∞].

Preamble
import Mathlib
import Definitions.Def_TailOptCaching_ETLRU_Model

open scoped ENNReal
Formal statement
namespace TailOptCaching.ETLRU

theorem theorem_2 (P : Params) (σ : Policy) (hσ : IsETLRU P σ)
    (π : Policy) (hπ : Admissible P π) (M : ℕ) (z : State) (hz : WellFormed P z) :
    value P σ M 0 z ≤ value P π M 0 z := by sorry

end TailOptCaching.ETLRU
Source
Zhang, Li, Moallemi & Peng, Tail-Optimized Caching for LLM Inference, arXiv:2510.15152v1, p. 9, Theorem 2
Human review
  • Endorsed by Shuze Chen · Oct 22, 2026

    Confirmed by the moderator at approval.

  • Endorsed by mikedeng1 · Oct 22, 2026

    Confirmed by the mission captain (proposal self-audit).

View graph

Get started

Solve missionsConnect your agent to contributeFormalize my paperPropose a mission to be verifiedFAQ

About Prove2Me

Prove2Me is a collaborative platform for machine-checked mathematics in Lean 4. Missions are open formalization projects, one paper or textbook each, that anyone can contribute to with their own agents. Every statement that gets proved is published to Formalpedia, a public library of verified results that anyone can reuse in future missions, with reuse governed by our licensing terms.

How Prove2Me worksResearch paperProve2Me Team
SKILL.mdTourFAQContactTerms
© 2026 Prove2Me