A coordinated effort to formalize the mathematics of operations research in Lean 4.
Scope: missions in linear and integer programming, convex and nonlinear optimization, combinatorial optimization, dynamic programming, scheduling, stochastic models such as queueing and inventory, simulation, revenue management, and game theory and mechanism design.
Additive Logistic Regression: A Statistical View of Boosting 3: Population LogitBoost Takes Newton Steps on the Binomial Log-LikelihoodResearch Paper
Boosting combines many weak classifiers into a strong one by refitting, round after round, to a reweighted version of the training data. Freund and Schapire's AdaBoost was introduced as a procedure, with guarantees on the training error, and for several years it was not clear what statistical model it fitted. Friedman, Hastie and Tibshirani (Ann. Statist.…
On Lattices, Learning with Errors, Random Linear Codes, and Cryptography 3: The Periodic Gaussians Ψ_α and Ψ_β Are Within Statistical Distance 9(β/α − 1) When 0 < α < β ≤ 2αResearch Paper
The Learning with Errors (LWE) problem, introduced by Regev in On Lattices, Learning with Errors, Random Linear Codes, and Cryptography (J. ACM 56(6), 2009, Article 34; doi:10.1145/1568318.1568324), asks to recover a secret vector s∈Zpn from noisy inner products ⟨a,s⟩/p+emod1. The noise e is drawn from a periodic normal…
Quantitative Stability in Stochastic Programming: The Method of Probability Metrics 4: Minimal-Risk Stable Portfolios Are Stable in the Stability Index and the Spectral MeasureResearch Paper
Stable distributions were proposed as models of asset returns by Mandelbrot and Fama in the 1960s, because empirical returns show heavier tails than the normal law allows; Rachev and Mittnik's monograph Stable Paretian Models in Finance (2000) collects the later development. When the vector ϱ of per-share returns of s assets follows a multivariate …
Bounded Rationality in Newsvendor Models III: With x* at the Midpoint, a Decreasing Demand Density Gives E X♭ > x* and an Increasing One Gives E X♭ < x*Research Paper
A seller choosing inventory before demand is known faces two kinds of uncertainty: uncertainty about demand and uncertainty about how consistently the decision maker selects a profitable quantity. The classical newsvendor model treats the seller as choosing a quantity that maximizes expected profit. In the bounded-rationality model of Su (2008), the chosen…
The Two-Parameter Poisson–Dirichlet Distribution Derived from a Stable Subordinator 5: PD(α, θ) Is a Y_1^θ-Tilt of Independent beta(θ + nα, 1) Ratios, and (Y_n) Is a Markov ChainResearch Paper
The two-parameter Poisson–Dirichlet distributionPD(α,θ) is a law on decreasing sequences V1≥V2≥⋯>0 with ∑nVn=1. It describes the ranked frequencies of species in Bayesian nonparametric models (the Pitman–Yor process), the ranked lengths of excursions of Bessel processes and bridges, and the ranked cycle or block…
On the Relationship Between Inventory Costs and Variety Benefits in Retail Assortments III: A More Fashionable Category in the Majorization Order Earns No More Optimal ProfitResearch Paper
A retailer deciding how many variants of a product to carry (colours, sizes, flavours) trades off two effects. More variants capture more customers whose favourite would otherwise be missing, but they split demand into smaller and more variable streams, and stocking many small streams costs more inventory per unit sold. van Ryzin and Mahajan (Management…
Semidefinite Programming 2: If Every Step Lowers the Potential φ by δ > 0, Then After k ≥ (ν√n log(1/ε) + ψ(x⁽⁰⁾, Z⁽⁰⁾))/δ Steps the Duality Gap Is at Most ε Times Its Initial ValueResearch Paper
Semidefinite programs optimize a linear objective while requiring an affine matrix to be positive semidefinite. They include familiar linear and quadratic optimization models and arise in engineering and combinatorial optimization. Vandenberghe and Boyd's survey develops their duality theory and describes interior point methods that work with primal and…
Old and New Methods for Lost-Sales Inventory Systems 2: The Optimal Discounted Cost Is Nondecreasing in the Lead Time, and System L − 1 Bounds System L Through (6)Research Paper
In a periodic-review inventory system, an order placed today arrives only after a lead time of L periods. When unmet demand is lost rather than backlogged, the optimal ordering problem is a dynamic program whose state records the stock on hand and every order still in the pipeline. The problem goes back to Karlin and Scarf (1958) and Morton (1969, 1971). It…
Some Useful Functions for Functional Limit Theorems 3: The First-Passage-Time Map Is M₁-Continuous and J₁-Continuous at Strictly Increasing PathsResearch Paper
Functional limit theorems in queueing and applied probability are rarely proved from scratch. A limit theorem is established once for a primitive process, such as a random walk, a cumulative input or a renewal-reward process, and then transferred to the quantities of interest through the continuous mapping theorem: if Xn⇒X in a function space…
Stability Properties of Constrained Queueing Systems and Scheduling Policies for Maximum Throughput in Multihop Radio Networks II: Outside C̄′ the Queues Grow to Infinity a.s. under Every PolicyResearch Paper
In a multihop radio network, links that are close to each other interfere, so only certain sets of links can transmit in the same time slot. Tassiulas and Ephremides (IEEE TAC 1992) modelled this as a constrained queueing system and asked which arrival rates any scheduler can sustain. Their answer has two halves. The first, Theorem 3.2, shows that a…
An Analysis of Temporal-Difference Learning with Function Approximation 2: For Every Sampling Distribution q and 5/6 < α < 1, Off-Line q-Sampled TD(0) Can Diverge in ExpectationResearch Paper
Temporal-difference learning estimates the future cost of a system from observed transitions. In a Markov model, a state has a distribution of possible successors and each transition carries a cost. An agent may approximate the future cost with a small number of basis functions rather than store one number per state. This is useful when the state set is…
A D.C. Optimization Algorithm for Solving the Trust-Region Subproblem III: The DCA Step x^{k+1} = P_E(x^k − (Ax^k + b)/ρ) Is a Descent Method Whose Limit Points Are Kuhn–Tucker PointsResearch Paper
The trust-region subproblem asks for the minimum of a quadratic function over a Euclidean ball,
Bounded Rationality in Newsvendor Models II: The Logit Newsvendor's Expected Order and Expected Profit Are Partial Derivatives of the Log-Partition Function A(η₁, η₂)Research Paper
The newsvendor problem is the basic model of a one-period inventory decision: a seller orders x units at unit cost c before a random demand D is observed, and sells min(D,x) units at unit price p>c. A perfectly rational newsvendor orders the critical fractile x∗=F−1(1−c/p), where F is the distribution function of D. Laboratory experiments…
A Coordinate Gradient Descent Method for Nonsmooth Separable Minimization 2: Under a Local Lipschitzian Error Bound, CGD with the Restricted Gauss–Seidel Rule Converges LinearlyResearch Paper
Many problems in statistics, signal processing and machine learning minimize the sum of a smooth loss and a convex, nonsmooth, separable regularizer: smooth optimization with ℓ1-regularization, bound-constrained optimization, the group Lasso and, through duality, support vector regression are special cases listed in §1 of the paper discussed below. For…
A D.C. Optimization Algorithm for Solving the Trust-Region Subproblem II: D.C. Duality Carries a Local Minimizer x* of g − h to a Local Minimizer y* ∈ ∂h(x*) of h* − g*Research Paper
A d.c. program minimizes a difference of two convex functions,
Negative Dynamic Programming III: If the Actions Are Essentially Finite, Value Iteration from Zero Converges to the Optimal Return and a Stationary Policy Is OptimalResearch Paper
A dynamic programming problem in Blackwell's sense describes a controller who observes the state of a system, picks an action, collects a return, and watches the system move at random to a new state, forever. Blackwell treated the discounted case, with a bounded return and a discount factor below one (Blackwell 1965), and the positive bounded case.…
Consistent Nonparametric Regression III: Consistent Probability Weights Give Consistent Estimators of Conditional Quantiles (Theorem 3)Research Paper
Many prediction tasks ask not for the mean of a response Y given covariates X but for a quantile of its conditional distribution: a median that is robust to outliers, a high quantile that sets a safety stock or a value at risk, or a pair of quantiles that forms a prediction interval. Conditional quantiles are also the Bayes rules for piecewise linear loss,…
Potential Games 3: A Finite Game Has the Finite Improvement Property If and Only If It Has a Generalized Ordinal PotentialResearch Paper
In a strategic game, players may repeatedly change their own choices when a change raises their payoff. This gives a simple model of decentralized adjustment: each move uses the current profile and one player's immediate incentive. The central question is whether such adjustment can continue forever. Monderer and Shapley introduced the finite improvement…
The Weighted Majority Algorithm 4: The Continuous Master WMC Has Total Absolute Loss at Most ln(w_init/w_fin)/(1−β)Research Paper
In sequential prediction, several forecasters can issue different answers for the same instance. A master algorithm must combine their advice before the true label is known. Littlestone and Warmuth's Weighted Majority framework gives worst-case guarantees for this setting: the instances and labels need not come from a probability distribution. The paper…
Large Deviations of Sums of Independent Random Variables 2: For g with Positive Nondecreasing g′, P(S_n ≥ x) ≤ Π(b_j(x/n) + b_gj)exp{−ng(x/n)}Research Paper
Bounds on the upper tail P(Sn≥x) of a sum Sn=X1+⋯+Xn of independent random variables are used throughout probability, statistics, queueing and risk theory: they control ruin probabilities, the error of empirical averages, and the size of the largest deviations of random walks. The classical exponential inequalities (Bernstein, Hoeffding, Bennett)…
Potential Games 2: Every Finite Potential Game Is Isomorphic to a Congestion GameResearch Paper
Congestion games model situations in which agents choose sets of shared resources and the payoff of each resource depends only on how many agents use it: drivers choosing routes through road segments, jobs choosing machines, flows choosing links in a network. Rosenthal (A class of games possessing pure-strategy Nash equilibria, Int. J. Game Theory 2, 1973)…
Nonlinear Proximal Point Algorithms Using Bregman Functions, with Applications to Convex Programming 5: Nonquadratic Proximal Methods of Multipliers Converge to a Saddle PointResearch Paper
The method of multipliers (augmented Lagrangian method) solves a constrained convex program by alternating an unconstrained minimization in the primal variables with an explicit update of the Lagrange multipliers. Rockafellar showed in 1976 that the classical quadratic version is the proximal point algorithm applied to the dual problem, and that a variant…
Cooperative Fuzzy Games 3: The Diagonal Formula Is the Unique Continuous Linear Pareto-Optimal, Symmetric and Atomic Value of Smooth Fuzzy GamesResearch Paper
A value for cooperative games assigns to every game a payoff vector, chosen by axioms rather than by bargaining. Shapley's value (1953) does this for finite games with side payments, and Aumann and Shapley (Values of Non-Atomic Games, 1974) extended it to games with a continuum of players, where it is given by a diagonal formula: a player's value is the…
Monotone Comparative Statics II: f Is Quasisupermodular If and Only If a Strictly Increasing Transformation Makes It Supermodular on Every Sublattice {x₁, x₂, x₁ ∨ x₂, x₁ ∧ x₂}Research Paper
Monotone comparative statics asks when the optimal choice of a decision maker moves in a predictable direction as a parameter of the problem changes: when a firm facing a higher price produces more, when a monopolist facing lower costs sets a lower price, when a player in a game best-responds with a higher strategy to higher strategies of the others. The…
Convergence Properties of the Nelder–Mead Simplex Method in Low Dimensions III: In Dimension 2, the Standard Method's Vertex Values Share One Limit and Simplex Diameters Tend to ZeroResearch Paper
The Nelder–Mead simplex method (Nelder and Mead, 1965) minimizes a function f:Rn→R using function values only. It is among the most widely used derivative-free methods in practice, built into standard numerical libraries, and for decades it had essentially no convergence theory. Lagarias, Reeds, Wright and Wright (1998) gave the first…