A coordinated effort to formalize the mathematics of operations research in Lean 4.
Scope: missions in linear and integer programming, convex and nonlinear optimization, combinatorial optimization, dynamic programming, scheduling, stochastic models such as queueing and inventory, simulation, revenue management, and game theory and mechanism design.
Negative Dynamic Programming I: With Non-Positive Rewards, If an Optimal Policy Exists Then an Optimal Stationary Policy ExistsResearch Paper
Many sequential decision problems accumulate costs that are never recovered: inventory holding and shortage costs, waiting times, the expected number of steps until a target is reached. Written as rewards, these are non-positive rewards summed over an infinite horizon without discounting. Ralph Strauch's paper Negative Dynamic Programming (Ann. Math.…
Technical Note: Q-Learning: With Bounded Rewards and Learning Rates with Σα = ∞, Σα² < ∞ at Every State–Action Pair, Tabular Q-Learning Converges to Q* with Probability 1Research Paper
Q-learning is the basic model-free algorithm of reinforcement learning: an agent that does not know the transition probabilities or the reward structure of a controlled Markov process learns the optimal action values from experience alone, by a one-line update applied after every observed transition. It was introduced in C. J. C. H. Watkins' 1989 thesis,…
Some Theoretical Aspects of Road Traffic Research IV: Optimum Phase Times of a Two-Phase Fixed-Time Signal — When the Minimum Cycle Minimizes Average Delay, and the Optimum OtherwiseResearch Paper
A fixed-time traffic signal is set by two numbers per phase: how long the green lasts and, through the other phase's green, how long the red lasts. Traffic engineers in the early 1950s set signals by Adams' formula, the shortest cycle that can clear the arriving traffic, on the working assumption that the shortest feasible cycle also gives the least delay.…
On Cores and Indivisibility 1: The Housing Market's Game Without Side Payments Is BalancedResearch Paper
A market in which each trader owns one indivisible good has a basic coordination question: can all goods be reassigned so that no group of traders can withdraw its own goods and improve every member's outcome? Houses are the paper's running example. Monetary transfers and fractional houses are absent, so ordinary convex allocation arguments do not describe…
Some Theoretical Aspects of Road Traffic Research II: Equal Journey Times on Alternative Routes — Exactly the Routes with bᵢ < t Are Used, qᵢ = pᵢ(1 − bᵢ/t)Research Paper
Planning a road improvement requires an estimate of how traffic will redistribute over the routes affected. Because speed falls as flow rises, sending every vehicle along the route that is quickest today changes which route is quickest. J. G. Wardrop posed the question in its simplest setting in 1952: a given flow Q chooses among D alternative routes…
Optimal Ordering and Rationing Policies in a Nonstationary Dynamic Inventory Model with n Demand Classes I: Critical Rationing Levels z̄ₜ¹ ≥ ⋯ ≥ z̄ₜⁿ Give an Optimal Policy When Each aₜ Is 0 or 1Research Paper
A firm that holds a single stock of a product often serves customers of different priority: military and civilian requisitions, contract and spot customers, emergency and routine orders for spare parts. When stock runs short, it must decide not only how much demand to satisfy but whose. Satisfying a low-priority order now may leave the firm unable to serve…
Average Optimality in Dynamic Programming with General State Space: Under (W) or (S) and Bounded Relative Discounted Values, a Limit of Discount-Optimal Policies Is Average OptimalResearch Paper
A Markov decision process with the average cost criterion asks for a policy that minimizes the long-run expected cost per unit time. The criterion is the natural one for inventory, queueing and maintenance systems that run indefinitely without a meaningful discount rate, and it is the standard benchmark in the theory of controlled Markov chains (Arapostathis…
On Finding Optimal Policies in Discrete Dynamic Programming with No Discounting 1: If G(f) ∪ H(f) Is Empty, f Maximizes the Gain and Then the Bias, and Each Improvement Step Raises (x, y, z)Research Paper
A controller observes a finite system period after period, chooses an action in each state, earns an income and moves the system at random. When future income is not discounted, the natural criterion is the average return per unit time (the gain), but many policies share the maximal gain while differing in their transient income. Blackwell (1962) showed…
On Cores and Indivisibility 2: Gale's Top Trading Cycles Give a Core Allocation Supported by Competitive PricesResearch Paper
Markets for indivisible goods, in which each participant owns one unit and wants one unit (houses, kidneys in paired donation, dormitory rooms, school seats), cannot be analysed with the convexity assumptions of classical general equilibrium theory. Shapley and Scarf (1974) introduced the housing market to ask whether the two central solution concepts of…
Estimating Variance From High, Low and Closing Prices I: S_t(S_t − X_t) + I_t(I_t − X_t) Has Mean σ²t for Brownian Motion With Any DriftResearch Paper
The logarithm of a share price is commonly modelled as a Brownian motion with drift, Xt=σBt+ct. The Black–Scholes option pricing formula needs the volatility σ but not the drift c, so a practitioner must estimate σ2 from data. Observing the whole path would give σ2 exactly through its quadratic variation, but a real observer sees…
Duality and Stability in Extremum Problems Involving Convex Functions 3: If f Is Finite at x and g Is Finite and Continuous at Ax, Then (P) Is Stably Set and inf (P(z)) Is Continuous Near z = 0Research Paper
A convex program is useful in practice only if its optimal value responds reasonably to small changes in the data. Lagrangian and Fenchel duality theory makes this precise: the dual problem attains its optimum and has no duality gap exactly when the primal value function does not fall off infinitely steeply under small perturbations. A constraint…
A Dual Approach to Solving Nonlinear Programming Problems by Unconstrained Optimization 1: Approximate Minimizers of L_r(·, yᵏ) along a Bounded Maximizing Dual Sequence Are Asymptotically MinimizingResearch Paper
Constrained convex programs are routinely solved by turning them into a sequence of unconstrained problems. The classical route is the ordinary LagrangianL0(x,y)=f0(x)+∑iyifi(x), whose dual function g0 is in general nonsmooth and equal to −∞ off the orthant y≥0, so maximizing it requires a constrained, nonsmooth method. In 1969…
Concentration of Measure and Isoperimetric Inequalities in Product Spaces VII: Bin Packing — P(|B_N − M| ≥ 1 + u) ≤ 8 exp(−u²/(64 N E X₁²))Textbook
Bin packing asks for the minimum number of unit-capacity bins into which a list of items with sizes in [0,1] can be packed. It is one of the basic NP-hard problems of combinatorial optimization and operations research, and its stochastic version, where the item sizes are random, has a large literature (Coffman and Lueker, Probabilistic Analysis of Packing…
Duality and Stability in Extremum Problems Involving Convex Functions 1: A Convex Program Is Stably Set If and Only If Its Infimum Equals the Attained Maximum of the Dual, inf (P) = max (P*)Research Paper
In convex optimization, a dual problem supplies a bound on what a primal minimization problem can achieve. The bound can be useful even when no minimizer is known, but an equality of the primal and dual values says more: it certifies that the bound is exact. In applications, one also wants a dual variable that actually attains the bound, since that variable…
Concentration of Measure and Isoperimetric Inequalities in Product Spaces X: The Chromatic Number of Random Graphs — χ(G(n,p), m) Lies in an Interval of Length k with High ProbabilityTextbook
The chromatic number of the random graph G(n,p) is one of the central random variables of probabilistic combinatorics. Its order of magnitude for constant p, χ(G(n,p))∼n/(2log1/(1−p)n), was settled by Bollobás ([B], 1988); the question that remains is how much χ fluctuates around its typical value. Shamir and Spencer ([S-S], 1987) used the…
Concentration of Measure and Isoperimetric Inequalities in Product Spaces V: The Convex Hull Distance — ∫ exp f_α(A,x) dP ≤ P(A)^{−α}Textbook
A product probability space (ΩN,P), P=μ⊗N, concentrates its mass: a set A of non-negligible measure is "close" to most points. How closeness is measured decides what the statement can be used for. The Hamming distance (Chapter 2 of the paper) counts coordinates in which a point differs from A; it gives Gaussian concentration, but only for…
Some Aspects of the Sequential Design of Experiments I: Greedy Sampling with Density-Zero Forced Draws Earns max(α, β) per Draw from Any Two Populations with Finite MeansResearch Paper
An experimenter can draw observations from two populations A and B, one at a time, and wants the sum of the observations to be as large as possible. The populations might be two treatments, two seed varieties or two manufacturing processes; their means are unknown. Each draw both earns a reward and tells the experimenter something about the population it…
Inventory Control in a Fluctuating Demand Environment I: With Linear Order Costs, a World-Dependent Basestock Policy Is Optimal over the Infinite HorizonResearch Paper
An inventory controller must decide how much to order while demand changes with an observed external condition. A single demand rate misses that dependence: the same inventory position can justify different orders when the condition changes. Song and Zipkin's 1993 study asks whether a simple policy remains optimal when the condition follows a…
Regenerative Stochastic Processes 1: The State Probabilities of an Aperiodic Equilibrium Process Converge to (1/μ₁)∫φ_A(v){1 − F(v)}dvResearch Paper
Many stochastic systems studied in operations research "start afresh" at certain random instants: a single-server queue each time a customer arrives to find the server idle, an inventory system each time stock is replenished to its order level, a reliability system each time a failed unit is replaced. Between such instants the system may evolve in a…
Regenerative Stochastic Processes 3: A Cumulative Process Has Mean (κ₁/μ₁)t + o(t) and Variance (t/μ₁){σ₂² − 2ρσ₁σ₂κ₁/μ₁ + σ₁²(κ₁/μ₁)²} + o(t)Research Paper
Many stochastic systems accumulate a real-valued quantity while repeatedly returning to a state at which a new cycle begins. A queue may accumulate work or cost, a component may accumulate wear, and a recurrent system may accumulate time in a state. The observations at a fixed clock time usually cut through a cycle. This makes the moments of the observed…
On Adaptive-Step Primal-Dual Interior-Point Algorithms for Linear Programming 1: The Predictor–Corrector Algorithm Between N₂(1/4) and N₂(1/2) Attains Precision t in O(√n·t) IterationsResearch Paper
Linear programming asks for the best value of a linear objective subject to linear equations and inequalities. Primal-dual interior-point methods seek feasible primal and dual solutions while keeping every constrained coordinate positive. The resulting pair has a measurable gap between its objective values, so an algorithm can stop when the gap is below a…
Random Walks in a Random Environment 1: Transience to +∞, to −∞, or Recurrence According to Σ n⁻¹P(ρ_n > 1) < ∞, Σ n⁻¹P(ρ_n < 1) < ∞, or Both Infinite (E ln σ < 0, > 0, = 0)Research Paper
A random walk on the integers in which the probability of stepping right depends on the current site is the simplest model of motion through a disordered medium: a particle diffusing in a crystal with impurities, a molecule moving along a heterogeneous polymer, or a gene frequency drifting under fluctuating selection. When the site-dependent probabilities…
Optimal Pricing and Return Policies for Perishable Commodities I: Unlimited Returns at Partial Credit Coordinate the Channel for Every Demand DistributionResearch Paper
A manufacturer of a perishable good (newspapers, magazines, bread, seasonal fashion, printed books) sells through a retailer who must commit to an order before demand is known. Unsold units lose most of their value; unmet demand costs goodwill. When the retailer alone bears the risk of leftovers, it orders less than an integrated firm would, and the channel…
A New Class of Random Number Generators 1: For Prime m = bʳ + bˢ − 1, Every Nontrivial Add-with-Carry Sequence Is Periodic After r Steps, with Period the Order of b mod mResearch Paper
Simulation, Monte Carlo integration and randomized algorithms consume long streams of numbers that should behave like independent uniform draws. A deterministic generator with finite state is eventually periodic, so its period is the first figure of merit: a stream cannot be longer than its cycle. G. Marsaglia and A. Zaman, A new class of random number…