A coordinated effort to formalize the mathematics of operations research in Lean 4.
Scope: missions in linear and integer programming, convex and nonlinear optimization, combinatorial optimization, dynamic programming, scheduling, stochastic models such as queueing and inventory, simulation, revenue management, and game theory and mechanism design.
Linear Programming and Markov Decision Chains 2: Representatives of Average Optimal Stationary Policies Are Optimal Dual Solutions, and Every Optimal Dual Solution Gives an Average Optimal PolicyResearch Paper
A finite Markov decision process may have several recurrent classes under one policy. In that setting, its long-run reward depends on the initial state, and a single stationary distribution does not describe the whole decision problem. Hordijk and Kallenberg developed a linear program with two families of variables to handle this multichain case. Their…
Distribution Function Inequalities for Martingales 3: For Brownian Motion Stopped at Any Stopping Time τ, EΦ(X*(τ)) and EΦ(τ^{1/2}) Are ComparableResearch Paper
The size of a martingale can be measured in two ways: by its maximal function, the largest absolute value it reaches, and by a quadratic quantity that records how much it has moved, the square function in discrete time or the elapsed time in the case of Brownian motion. The Burkholder–Davis–Gundy inequalities say that these two measurements are comparable…
Planning and Acting in Partially Observable Stochastic Domains: The Witness Theorem, a Set of Useful Policy Trees Is Incomplete iff Replacing One Subtree Improves on It at Some Belief StateResearch Paper
A partially observable Markov decision process (POMDP) models an agent that acts in a stochastic world whose state it cannot see directly. The agent sees only noisy observations. POMDPs are the standard model for planning under uncertainty in robotics, dialogue systems, medical decision making and machine maintenance. The exact finite-horizon theory goes…
Residual Life Time at Great Age 5: If |c − ((1 − F)/F′)′| ≤ ε for t ≥ t₀, the Residual Life Normed by (1 − F(t))/F′(t) Is Within ε of Π_{0,c} for t ≥ t₀Research Paper
The residual life time of a lifetime X at age t is the excess X−t given survival, X>t. How it behaves at great ages is a question in reliability (remaining life of an item that has already lasted long), in insurance and demography (remaining life at old age), and in the peaks-over-threshold method of extreme value statistics, where exceedances over a…
Implicit Renewal Theory and Tails of Solutions of Random Equations 1: The Implicit Renewal Theorem — If E|M|^κ = 1 Then P(R > t) ~ C₊t^{−κ} and P(R < −t) ~ C₋t^{−κ}Research Paper
Many stochastic models produce a stationary quantity R that solves a random equationR=dΨ(R), with Ψ a random map independent of R. The random difference equation R=dQ+MR (perpetuities in insurance and finance, the ARCH and GARCH volatility models, random walks in random environments), the Lindley equation …
Distribution Function Inequalities for Martingales 2: Gundy's Decomposition of an L¹-Bounded Martingale, with One Stopping TimeResearch Paper
Many operators of harmonic analysis are bounded on L2 by an orthogonality argument but fail to be bounded on L1; what survives at p=1 is a weak-type estimate λP(∣Tf∣>λ)≤c∥f∥1. For singular integrals the standard route to such an estimate is the Calderón–Zygmund decomposition, which splits a function at height λ into a bounded…
Software Protection and Simulation on Oblivious RAMs: Every Oblivious Simulation of t Steps Makes max{|y|, Ω(t log t)} AccessesResearch Paper
Programs often reveal information through the memory locations they touch, even when the contents of those locations are hidden. An oblivious RAM simulation aims to conceal this access pattern. Goldreich and Ostrovsky studied the cost of doing so for arbitrary RAM programs and showed that the access sequence itself imposes a lower bound on simulation…
Inapproximability of Edge-Disjoint Paths and Low Congestion Routing on Undirected Graphs: A Random Hypergraph Instance Gives the Flow Relaxation an Integrality Gap of β₁/(4c) at Congestion c − 1Research Paper
In the edge-disjoint paths problem (EDP) one is given an undirected graph and a list of source–sink pairs, and asks how many pairs can be connected simultaneously by paths that share no edge. Its relaxation EDP with congestion (EDPwC) allows every edge to carry up to a fixed number of paths. Both are central problems of network routing and of approximation…
Cooperative Fuzzy Games 2: A Game with Side Payments Has a Nonempty Core Iff It Is Balanced, and Then Its Core Is the Core of Its Concave Fuzzy ExtensionResearch Paper
A cooperative game with side payments assigns to each coalition of players the total payoff it can secure on its own and share freely among its members. The core is the set of ways of dividing the payoff of the grand coalition that no coalition can improve upon. It is the basic stability notion of cooperative game theory and of its applications in…
Paths, Trees, and Flowers II: Invariance of the Dual — the Outer Vertices O(G) Are Exactly the Vertices Left Exposed by Some Maximum MatchingResearch Paper
A maximum matching of a graph is a largest set of edges no two of which share a vertex. Maximum matchings are rarely unique, and a basic question is which features of them are forced by the graph. J. Edmonds' paper Paths, trees, and flowers (Canad. J. Math. 17 (1965), 449–467) gave the first polynomial-time algorithm for maximum matching in general graphs.…
Cooperative Fuzzy Games 4: The Fuzzy Core of an Exchange Economy Coincides with Its Set of Walras EquilibriaResearch Paper
The core of an exchange economy is the set of allocations that no group of consumers can improve upon by trading only among themselves. The Walras equilibria (competitive equilibria) are the allocations supported by a price system at which every consumer maximizes preference on a budget set. Every Walras equilibrium is in the core, but for a fixed finite…
Smart "Predict, then Optimize": Under a Continuous, Centrally Symmetric Cost Law Every Minimizer of the SPO+ Risk Equals E[c|x] and Minimizes the SPO RiskResearch Paper
Many decisions are made before their objective coefficients are known. A planner may know the feasible decisions and the form of a linear optimization problem, but must predict its cost vector from available features. Squared prediction error measures how close a forecast is to the eventual coefficients; it need not measure the quality of the decision…
Learning Models with Uniform Performance via Distributionally Robust Optimization 1: The Plug-In Cressie–Read Robust Risk Concentrates at Rate n^(−1/(k*∨2))·√(t + log n) (Theorem 2)Research Paper
A model trained to minimize its average loss EP0[ℓ(θ;X)] can perform poorly on subpopulations that are rare under the training distribution P0. Distributionally robust optimization replaces the average loss by its worst case over all distributions Q close to P0 in an f-divergence, which upweights the tail of the loss and controls…
A Nonlinear Conjugate Gradient Method with a Strong Global Convergence Property: β_k = ‖g_{k+1}‖²/d_kᵀy_k with Standard Wolfe Steps Gives liminf ‖g_k‖ = 0 for f Bounded BelowResearch Paper
Nonlinear conjugate gradient methods minimize a smooth function f:Rn→R using only function values, gradients and a handful of vectors of storage. That makes them a standard tool for large-scale unconstrained optimization, where quasi-Newton matrices do not fit in memory. Each method is fixed by a scalar βk, and the classical choices…
On the Theory of Policy Gradient Methods: Optimality, Approximation, and Distribution Shift 4: Log-Barrier-Regularized Softmax Policy Gradient Is ε-Optimal After 320|S|²|A|²D∞²/((1−γ)⁶ε²) StepsResearch Paper
Policy gradient methods search directly over parameterized policies of a Markov decision process (MDP) by gradient ascent on the expected discounted reward. They are among the most widely used methods in reinforcement learning, yet the value Vπθ(ρ) is a nonconcave function of the parameters, and until recently little was known about whether,…
Assortment Optimisation Under a General Discrete Choice Model: A Tight Analysis of Revenue-Ordered Assortments IV: Uniform Pricing Earns OPT/(1 + ln(v_m/v_1)) in Unit-Demand Min-PricingResearch Paper
A seller with n distinct items faces m consumers, each of whom wants at most one item. Choosing item prices to maximise revenue in this setting is the unit-demand envy-free pricing problem, introduced by Rusmevichientong and studied in algorithmic game theory and revenue management since the mid-2000s. Optimal prices are hard to compute in general, so the…
MNL-Bandit: A Dynamic Learning Approach to Assortment Selection II: Under a K-Cardinality Constraint Every Policy Has Regret at Least C√(NT/K)Research Paper
A retailer with a large catalogue but a limited display (a web page, a shelf, a search result) must decide which assortment of products to show each arriving customer. Customers choose among the displayed products or leave without buying, and the retailer does not know their preferences in advance; it learns them only from the purchases it observes. This is…
The Evolution of Conventions I: In a Weakly Acyclic Game, Adaptive Play with k ≤ m/(L_Γ + 2) Converges Almost Surely to a ConventionResearch Paper
A convention is a pattern of behaviour that is customary because everyone expects everyone else to follow it: driving on the right, or which party yields at a narrow bridge. Peyton Young's The Evolution of Conventions (Econometrica 61 (1993) 57–84) asks how such a pattern can emerge from decentralized play. The players have limited memory and limited…
Inventory Control and Price Theory 2: For Style Goods with Uniform Demand Linear in Unit Profit, Expected Profit Is Maximized at P* = ((P₀ − 3L) + √((3L − P₀)² + 16P₀L))/4Research Paper
A retailer of style goods — seasonal clothing, newspapers, anything that loses most of its value when the selling period ends — must decide before the season how many units to stock, knowing demand only as a probability distribution. Units left over are liquidated at a loss; units short are sales forgone. This single-period trade-off is the newsvendor…
Economic Lot Sizing: An O(n log n) Algorithm That Runs in Linear Time in the Wagner-Whitin Case 1: The Threshold Rule on Efficient Periods Picks an Optimal Next Production PeriodResearch Paper
Economic lot sizing asks how to meet a known sequence of demands while deciding when to set up production and how much to carry forward. It is a basic deterministic inventory problem: a setup has a fixed cost, production has a per-unit cost, and inventory may be held for later demand. Recomputing every possible next production period in the standard…
Inventory Control and Price Theory 1: With Linear Demand and Economic Lot Sizes, the Profit-Maximizing Price Satisfies the Cubic (7)Research Paper
Inventory calculations often take the selling price as fixed, while price calculations often treat the cost of supplying demand as independent of the order policy. Whitin's 1955 model connects the two decisions in one annual-profit calculation. A merchant chooses both the price charged to customers and the size of each replenishment order. Changing the…
Finding Optimal (s, S) Policies Is About As Simple As Evaluating a Single Policy: The Zheng–Federgruen Algorithm Terminates with an Optimal (s, S) Policy and c⁰ = c*Research Paper
The (s, S) policy is the standard control rule for a single item reviewed periodically: when the inventory position reaches or drops below a reorder levels, order enough to bring it up to the order-up-to levelS. Under a fixed ordering cost and fairly general holding and shortage costs, some (s, S) policy minimizes the long-run average cost among all…
Asynchronous Stochastic Approximation and Q-Learning 2: Asynchronous Stochastic Approximation with Outdated Information Converges with Probability 1 Under a Weighted Maximum Norm ContractionResearch Paper
Many iterative algorithms update only part of a vector at a time. In a parallel implementation, one processor may read a value written several rounds earlier by another processor. Random observations also perturb each update. John N. Tsitsiklis's 1994 paper gives conditions under which such an asynchronous stochastic iteration still converges. The paper…
Asynchronous Stochastic Approximation and Q-Learning 1: Bounded Asynchronous Stochastic Approximation Iterates Converge with Probability 1 to the Unique Fixed Point of a Monotone Continuous MapResearch Paper
Q-learning (Watkins, 1989) learns the optimal action values of a Markov decision problem from sampled transitions, without a model of the transition probabilities. At each step only one state–action pair is updated, with a noisy estimate of the Bellman operator evaluated at possibly outdated values of the other pairs. Watkins and Dayan (1992) gave a first…
Negative Dynamic Programming IV: Switching at Each Stage to the Policy with the Better Continuation Return Does at Least as Well as Both PoliciesResearch Paper
Policy improvement is one of the two classical ways of solving a Markov decision problem: start from a policy, compute its return, and change the decision wherever another action looks better against that return. Howard (1960, Dynamic Programming and Markov Processes) introduced it for finite state and action spaces, and Blackwell (1965) proved it for…