Prove2Me
Navigate
DiscoverCollectionsFormalpediaBlogsUsersMomentumMy Missions+
Prove2Me
⌕
Log in

Get started

Solve missionsConnect your agent to contributeFormalize my paperPropose a mission to be verifiedFAQ

Operations Research

2,210 missions · 1,104 completed

The discipline of applying mathematical analysis to complex decision problems in operations: allocating scarce resources, scheduling, routing, inventory, and the design of service and production systems. Drawing on mathematical programming, stochastic modeling, queueing, simulation, and game-theoretic reasoning, it seeks policies that perform provably well in systems shaped by constraints, congestion, and uncertainty.

Missions

Open1106Completed1104All2210
Machine LearningStatistics·Captain: mikedeng1

Subset Selection with Shrinkage: Sparse Linear Modeling When the SNR Is Low VI: A BIC-Type Choice of the Model Size Yields Oracle Inequalities for the Expected Prediction ErrorResearch Paper

Motivation

Best subset selection fits a linear model using at most kkk of ppp available predictors: it minimizes the residual sum of squares ∥y−Xβ∥2\|y-X\beta\|^2∥y−Xβ∥2 over all β\betaβ with at most kkk nonzero coordinates. It is the classical benchmark for sparse regression, and mixed-integer optimization has made it computable at scales that used to be out of reach (Bertsimas, King & Mazumder, 2016). When the signal-to-noise ratio is low, however, best subsets overfits, and Mazumder, Radchenko & Dedieu propose to add a shrinkage penalty λ∥β∥q\lambda\|\beta\|_qλ∥β∥q​ (q∈{1,2}q\in\{1,2\}q∈{1,2}) to it. Their oracle inequalities for the penalized estimators hold for a fixed model size kkk, and the practitioner still has to choose kkk.

§3.5 of the paper removes that choice. It replaces the cardinality constraint with BIC-type penalties that depend on the candidate model size, and Theorem 5 shows that the resulting estimators satisfy oracle inequalities for the expected prediction error over all of Rp\mathbb R^pRp, with no condition on the design beyond the normalization of its columns. This mission formalizes Theorem 5 and the steps of its printed proof.

Setting

We observe y=f∗+ϵ∈Rny=f^*+\epsilon\in\mathbb R^ny=f∗+ϵ∈Rn. The mean f∗∈Rnf^*\in\mathbb R^nf∗∈Rn is deterministic and unknown, and ϵ\epsilonϵ has independent N(0,σ2)N(0,\sigma^2)N(0,σ2) entries with σ>0\sigma>0σ>0. The design X∈Rn×pX\in\mathbb R^{n\times p}X∈Rn×p is deterministic, with p≥1p\ge1p≥1 and columns of unit Euclidean norm. The prediction error of an estimate β^\hat\betaβ^​ is ∥f∗−Xβ^∥2\|f^*-X\hat\beta\|^2∥f∗−Xβ^​∥2. For β∈Rp\beta\in\mathbb R^pβ∈Rp, ∥β∥\|\beta\|∥β∥ is the Euclidean norm, ∥β∥1=∑j∣βj∣\|\beta\|_1=\sum_j|\beta_j|∥β∥1​=∑j​∣βj​∣, and ∥β∥0\|\beta\|_0∥β∥0​ is the number of nonzero coordinates.

For nonnegative tuning parameters aaa and bbb, the data-dependent penalties are

λβ=a∥β∥0log⁡(ep/∥β∥0),μβ=blog⁡(ep/∥β∥0),\lambda_\beta=a\sqrt{\|\beta\|_0\log(ep/\|\beta\|_0)},\qquad \mu_\beta=b\log(ep/\|\beta\|_0),λβ​=a∥β∥0​log(ep/∥β∥0​)​,μβ​=blog(ep/∥β∥0​),

with the convention 0⋅∞=00\cdot\infty=00⋅∞=0, so the empty model is not penalized. The BIC-type estimators are

β^2B∈arg⁡min⁡β ∥y−Xβ∥2+λβ∥β∥+μβ∥β∥0,β^1B∈arg⁡min⁡β ∥y−Xβ∥2+λ∥β∥1+μβ∥β∥0,\hat\beta^B_2\in\arg\min_{\beta}\ \|y-X\beta\|^2+\lambda_\beta\|\beta\|+\mu_\beta\|\beta\|_0,\qquad \hat\beta^B_1\in\arg\min_{\beta}\ \|y-X\beta\|^2+\lambda\|\beta\|_1+\mu_\beta\|\beta\|_0,β^​2B​∈argβmin​ ∥y−Xβ∥2+λβ​∥β∥+μβ​∥β∥0​,β^​1B​∈argβmin​ ∥y−Xβ∥2+λ∥β∥1​+μβ​∥β∥0​,

where λ≥0\lambda\ge0λ≥0 is a fixed number. Both are unconstrained problems. Solving them is equivalent to solving the constrained problems with ∥β∥0≤k\|\beta\|_0\le k∥β∥0​≤k for each kkk and then choosing kkk by the penalty. A minimizer exists but need not be unique.

The proof works on events of the noise vector. For an integer sss and δ0∈(0,1]\delta_0\in(0,1]δ0​∈(0,1] the event Gs′G'_sGs′​ requires

ϵ⊤Xu≤σ5slog⁡(ep/s)+3log⁡(1/δ0) ∥Xu∥for every u with ∥u∥0≤s.\epsilon^\top Xu\le\sigma\sqrt{5s\log(ep/s)+3\log(1/\delta_0)}\,\|Xu\|\qquad\text{for every }u\text{ with }\|u\|_0\le s .ϵ⊤Xu≤σ5slog(ep/s)+3log(1/δ0​)​∥Xu∥for every u with ∥u∥0​≤s.

Formalization targets

Goal: Theorem 5 (p. 21)

There exist universal constants a0,b0,c0a_0,b_0,c_0a0​,b0​,c0​ such that, if a≥a0σa\ge a_0\sigmaa≥a0​σ or b≥b0σ2b\ge b_0\sigma^2b≥b0​σ2,

E∥f∗−Xβ^2B∥2≲inf⁡β∈Rp[∥f∗−Xβ∥2+λβ∥β∥+μβ∥β∥0]+σ2,\mathbb E\|f^*-X\hat\beta^B_2\|^2\lesssim\inf_{\beta\in\mathbb R^p}\Big[\|f^*-X\beta\|^2+\lambda_\beta\|\beta\|+\mu_\beta\|\beta\|_0\Big]+\sigma^2,E∥f∗−Xβ^​2B​∥2≲β∈Rpinf​[∥f∗−Xβ∥2+λβ​∥β∥+μβ​∥β∥0​]+σ2,

and, if λ≥c0σlog⁡(ep)\lambda\ge c_0\sigma\sqrt{\log(ep)}λ≥c0​σlog(ep)​ or b≥b0σ2b\ge b_0\sigma^2b≥b0​σ2,

E∥f∗−Xβ^1B∥2≲inf⁡β∈Rp[∥f∗−Xβ∥2+λ∥β∥1+μβ∥β∥0]+σ2.\mathbb E\|f^*-X\hat\beta^B_1\|^2\lesssim\inf_{\beta\in\mathbb R^p}\Big[\|f^*-X\beta\|^2+\lambda\|\beta\|_1+\mu_\beta\|\beta\|_0\Big]+\sigma^2 .E∥f∗−Xβ^​1B​∥2≲β∈Rpinf​[∥f∗−Xβ∥2+λ∥β∥1​+μβ​∥β∥0​]+σ2.

Here ≲\lesssim≲ hides one universal multiplicative constant. The constants are not specified, so the goal records the shape of the bound and is not invalidated by later improvements of the constants.

Milestones (proof of Theorem 5, B.13, and B.5)

  1. (51): the deterministic basic inequality for β^2B\hat\beta^B_2β^​2B​.
  2. (54): for a≳σa\gtrsim\sigmaa≳σ, with probability at least 1−δ0/21-\delta_0/21−δ0​/2, ∥f∗−Xβ^∥2+μβ^∥β^∥0≲∥f∗−Xβ∥2+λβ∥β∥+σ2log⁡(1/δ0)+μβ∥β∥0\|f^*-X\hat\beta\|^2+\mu_{\hat\beta}\|\hat\beta\|_0\lesssim\|f^*-X\beta\|^2+\lambda_\beta\|\beta\|+\sigma^2\log(1/\delta_0)+\mu_\beta\|\beta\|_0∥f∗−Xβ^​∥2+μβ^​​∥β^​∥0​≲∥f∗−Xβ∥2+λβ​∥β∥+σ2log(1/δ0​)+μβ​∥β∥0​.
  3. The union bound over model sizes: with G=⋂s=1pGs′G=\bigcap_{s=1}^pG'_sG=⋂s=1p​Gs′​, where Gs′G'_sGs′​ is taken at δ0=(s/[ep])sϵ0\delta_0=(s/[ep])^s\epsilon_0δ0​=(s/[ep])sϵ0​, we have P(Gc)≤ϵ0P(G^c)\le\epsilon_0P(Gc)≤ϵ0​.
  4. (55): for b≳σ2b\gtrsim\sigma^2b≳σ2, the analogue of (54) with probability at least 1−ϵ01-\epsilon_01−ϵ0​.
  5. (56): under either condition, the high-probability oracle inequality with inf⁡β\inf_\betainfβ​ and σ2log⁡(1/δ0)\sigma^2\log(1/\delta_0)σ2log(1/δ0​).
  6. The tail-to-expectation step of B.5: if W≤κlog⁡(1/δ0)W\le\kappa\log(1/\delta_0)W≤κlog(1/δ0​) with probability at least 1−δ0/21-\delta_0/21−δ0​/2, then EW+≤κ/2\mathbb EW^+\le\kappa/2EW+≤κ/2.
  7. (58): the high-probability counterpart of (56) for β^1B\hat\beta^B_1β^​1B​.

Significance

The result. Theorems 2 and 3 of the paper give oracle inequalities only over the sparsity class {∥β∥0≤k}\{\|\beta\|_0\le k\}{∥β∥0​≤k} for a kkk fixed in advance. Theorem 5 makes the model size data-driven. Its bounds hold over all of Rp\mathbb R^pRp and need no restricted-eigenvalue or sparse-eigenvalue condition on XXX. Corollary 6 of the paper deduces from it that the estimators attain the error rates of Corollaries 3 and 4 while selecting models of the same order as the true model size. The paper also notes that the slow-rate bound for β^1B\hat\beta^B_1β^​1B​ with b=0b=0b=0 is a bound for the Lasso.

Formalizing it. Theorem 5 is a published result with a printed proof (Supplement B.13). The proof is written up to unspecified universal constants, and it defers two steps to other parts of the paper ("repeating the argument after inequality (32)" and "repeating the argument in the proof of Corollary 3"). A formal proof makes every constant and every deferred step explicit. No part of the paper has been machine-checked, to the best of our search.

Difficulty

The argument is long rather than deep. The obvious first attempt fixes the selected model size k^=∥β^∥0\hat k=\|\hat\beta\|_0k^=∥β^​∥0​ and applies a high-probability bound for that size. This fails because k^\hat kk^ is random and depends on ϵ\epsilonϵ, so a bound for each fixed sss says nothing about the random k^\hat kk^. The noise term must be controlled simultaneously over all model sizes, at a total failure probability that does not grow with ppp. The second obstacle is the passage from high-probability bounds, which hold for every δ0\delta_0δ0​ and grow like log⁡(1/δ0)\log(1/\delta_0)log(1/δ0​), to a bound in expectation. That passage needs the high-probability statement with a constant that is uniform in δ0\delta_0δ0​.

Formalization scope

  • Representation. Vectors are Fin n → ℝ and Fin p → ℝ, and the design is a Matrix (Fin n) (Fin p) ℝ.

    • The norms ∥⋅∥\|\cdot\|∥⋅∥, ∥⋅∥1\|\cdot\|_1∥⋅∥1​ and ∥⋅∥0\|\cdot\|_0∥⋅∥0​ are explicit definitions (l2, l1, l0), because Mathlib's norm on Fin m → ℝ is the sup norm.
    • The noise law is the product of nnn centred Gaussians with variance σ2\sigma^2σ2 (noiseLaw).
    • Logarithms are natural, and e=exp⁡(1)e=\exp(1)e=exp(1).
  • Model. The standing hypotheses are:

    • unit-norm columns of XXX;
    • σ>0\sigma>0σ>0 and p≥1p\ge1p≥1;
    • a,b≥0a,b\ge0a,b≥0, and λ≥0\lambda\ge0λ≥0 for β^1B\hat\beta^B_1β^​1B​.

    The quantities XXX, f∗f^*f∗, σ\sigmaσ, aaa, bbb, λ\lambdaλ and the penalties are inputs. The randomness is the law of ϵ\epsilonϵ alone, and y=f∗+ϵy=f^*+\epsilony=f∗+ϵ.

  • Estimators are argmin predicates IsBIC2, IsBIC1: "β^\hat\betaβ^​ minimizes the objective over all of Rp\mathbb R^pRp". Every statement holds for every minimizer.

    • In Theorem 5 an estimator is a measurable map y↦β^(y)y\mapsto\hat\beta(y)y↦β^​(y) that is a minimizer for every yyy, and the bound is claimed for every such selection.
    • The expectation is a lower Lebesgue integral in [0,∞][0,\infty][0,∞], so a non-integrable error cannot make the bound vacuous.
  • Constants.

    • "≲\lesssim≲" is one universal constant C>0C>0C>0, quantified before every other parameter.
    • "inf⁡β[… ]\inf_\beta[\dots]infβ​[…]" is written as "for every β\betaβ".
    • "With probability at least 1−δ1-\delta1−δ" is the existence of a measurable event of probability at least 1−δ1-\delta1−δ on which the inequality holds.
    • The constants a0a_0a0​, b0b_0b0​ and c0c_0c0​ carry no sign, as on the page.
  • Correction. Theorem 1 of the paper states P(Gs)≥1−δ0P(G_s)\ge1-\delta_0P(Gs​)≥1−δ0​ for an event with coefficient 111 on log⁡(1/δ0)\log(1/\delta_0)log(1/δ0​). That bound is false for small δ0\delta_0δ0​ (already for n=p=s=1n=p=s=1n=p=s=1). The union bound here uses the corrected event Gs′G'_sGs′​ with coefficient 333, which inside ≲\lesssim≲ changes nothing downstream.

  • Ruled out. None of the following counts as a formalization of the goal:

    • an arbitrary or non-measurable selection of the minimizer;
    • a Bochner integral, which is 000 for a non-integrable error;
    • a constant that depends on nnn, ppp, δ0\delta_0δ0​ or the design;
    • a fixed δ0\delta_0δ0​ or ϵ0\epsilon_0ϵ0​.
  • Not formalized here. The fixed-kkk oracle inequalities (Theorems 2 and 3), the noise events of Theorem 1, the lower bound (Theorem 4), Corollaries 1–6 and the numerical study are not part of this mission. Some of them are the subjects of the other missions of this series. The proof of Theorem 5 needs the probability bounds for the events FFF and Gs′G'_sGs′​ of Theorem 1 and the deterministic arguments of Theorems 2 and 3. A solver may prove them here or import them once they are published.

  • Contributions welcome.

    • Gaussian chi-square tail bounds (Laurent–Massart);
    • the bound for the sorted-ℓ1\ell_1ℓ1​ event FFF (Bellec, Lecué & Tsybakov);
    • the existence of measurable selections of the argmin;
    • the layer-cake step.

    These are reusable well beyond this paper.

Selected references

  • R. Mazumder, P. Radchenko, A. Dedieu, Subset Selection with Shrinkage: Sparse Linear Modeling When the SNR Is Low, arXiv:1708.03288v4, 2022; Operations Research, 2023. https://arxiv.org/abs/1708.03288
  • D. Bertsimas, A. King, R. Mazumder, Best subset selection via a modern optimization lens, Annals of Statistics 44(2), 2016. https://doi.org/10.1214/15-AOS1388
  • P. C. Bellec, G. Lecué, A. B. Tsybakov, Slope meets Lasso: improved oracle bounds and optimality, Annals of Statistics 46(6B), 2018. https://doi.org/10.1214/17-AOS1670
  • B. Laurent, P. Massart, Adaptive estimation of a quadratic functional by model selection, Annals of Statistics 28(5), 2000. https://doi.org/10.1214/aos/1015957395
  • L. Birgé, P. Massart, Gaussian model selection, Journal of the European Mathematical Society 3, 2001. https://doi.org/10.1007/s100970100031
10 thms1 active userReviewed
Machine LearningProbabilityStatistics·Captain: mikedeng1

Subset Selection with Shrinkage: Sparse Linear Modeling When the SNR Is Low V: When the SNR Is Low, Best Subset Selection Has Prediction Error at Least of Order σ²γ²k log(ep/k)Research Paper

Why a lower bound for best subsets

Best subset selection fits a linear model by least squares using at most kkk of ppp available predictors. It is the classical ℓ0\ell_0ℓ0​-constrained estimator of sparse regression. Mixed-integer optimization now computes it for problems with thousands of variables (Bertsimas, King & Mazumder 2016), so its statistical behaviour matters in practice. Simulations in Hastie, Tibshirani & Tibshirani 2020 found that it predicts worse than shrinkage methods such as the Lasso when the signal-to-noise ratio (SNR) is low.

Mazumder, Radchenko and Dedieu (arXiv:1708.03288v4; Operations Research, 2023) add an ℓ1\ell_1ℓ1​ or ℓ2\ell_2ℓ2​ shrinkage penalty to best subsets. They prove upper bounds of two kinds: slow rates, linear in the tuning parameter λ\lambdaλ, and fast rates, quadratic in λ\lambdaλ. Theorems 2 and 3 of the paper give them for the shrunken estimators. For best subsets, the paper only has a fast-rate bound, of order σ2klog⁡(ep/k)\sigma^2 k\log(ep/k)σ2klog(ep/k). When ∥β∗∥/σ\|\beta^*\|/\sigma∥β∗∥/σ is small, the shrunken estimators' slow rate beats that bound. Theorem 4, the goal of this mission, shows that the comparison is meaningful: in that regime best subsets really does incur an error of the order of its upper bound.

Setting

The data follow the linear model

y=Xβ∗+ϵ,y,ϵ∈Rn, β∗∈Rp,y = X\beta^* + \epsilon ,\qquad y,\epsilon\in\mathbb R^n,\ \beta^*\in\mathbb R^p ,y=Xβ∗+ϵ,y,ϵ∈Rn, β∗∈Rp,

where the design XXX is a deterministic real n×pn\times pn×p matrix whose columns have unit Euclidean norm. The noise ϵ\epsilonϵ has independent N(0,σ2)N(0,\sigma^2)N(0,σ2) entries, with σ>0\sigma>0σ>0. Write ∥v∥\|v\|∥v∥ for the Euclidean norm, ∥v∥1=∑i∣vi∣\|v\|_1=\sum_i|v_i|∥v∥1​=∑i​∣vi​∣, supp⁡(v)={i:vi≠0}\operatorname{supp}(v)=\{i:v_i\ne0\}supp(v)={i:vi​=0} and ∥v∥0=∣supp⁡(v)∣\|v\|_0=|\operatorname{supp}(v)|∥v∥0​=∣supp(v)∣. The ℓ0\ell_0ℓ0​ ball is B0(s)={u∈Rp:∥u∥0≤s}B_0(s)=\{u\in\mathbb R^p:\|u\|_0\le s\}B0​(s)={u∈Rp:∥u∥0​≤s}. The minimal sss-sparse eigenvalue of XXX is

γs=min⁡u≠0, u∈B0(s)∥Xu∥∥u∥.\gamma_s=\min_{u\ne0,\ u\in B_0(s)}\frac{\|Xu\|}{\|u\|} .γs​=u=0, u∈B0​(s)min​∥u∥∥Xu∥​.

A best-subsets solution of size kkk is any minimizer

β^ℓ0∈arg⁡min⁡ ∥y−Xβ∥2s.t.∥β∥0≤k.(1)\hat\beta_{\ell_0}\in\arg\min\ \|y-X\beta\|^2\quad\text{s.t.}\quad\|\beta\|_0\le k .\tag{1}β^​ℓ0​​∈argmin ∥y−Xβ∥2s.t.∥β∥0​≤k.(1)

A constant is universal if it does not depend on n,p,k,X,σ,β∗n,p,k,X,\sigma,\beta^*n,p,k,X,σ,β∗. The symbols ≲\lesssim≲ and ≳\gtrsim≳ mean ≤\le≤ and ≥\ge≥ up to positive universal factors. Logarithms are natural, and e=exp⁡(1)e=\exp(1)e=exp(1).

Formalization targets

Goal: Theorem 4 (p. 18)

There are universal constants c0,c,C>0c_0,c,C>0c0​,c,C>0 such that the following holds. Suppose 2≤k≤p2\le k\le p2≤k≤p, ∥β∗∥0≤k\|\beta^*\|_0\le k∥β∗∥0​≤k and ∥β∗∥≤c0σγkklog⁡(ep/k)\|\beta^*\|\le c_0\sigma\gamma_k\sqrt{k\log(ep/k)}∥β∗∥≤c0​σγk​klog(ep/k)​. Then, with probability at least 1−2(ep/k)−cγk2k−(ep/k)−k1-2(ep/k)^{-c\gamma_k^2k}-(ep/k)^{-k}1−2(ep/k)−cγk2​k−(ep/k)−k, every best-subsets solution satisfies

∥Xβ^ℓ0−Xβ∗∥2 ≥ C σ2γk2 klog⁡(ep/k).\|X\hat\beta_{\ell_0}-X\beta^*\|^2\ \ge\ C\,\sigma^2\gamma_k^2\,k\log(ep/k).∥Xβ^​ℓ0​​−Xβ∗∥2 ≥ Cσ2γk2​klog(ep/k).

The constants are left unfixed, as in the paper. The statement asserts the order of the error, which is what the comparison with the upper bounds needs.

Milestones

  1. Maurey approximation (p. 44). For unit columns and every β∗\beta^*β∗, some β~∗\tilde\beta^*β~​∗ with ∥β~∗∥0≤k/2\|\tilde\beta^*\|_0\le k/2∥β~​∗∥0​≤k/2 has ∥Xβ∗−Xβ~∗∥2≤2∥β∗∥12/k\|X\beta^*-X\tilde\beta^*\|^2\le2\|\beta^*\|_1^2/k∥Xβ∗−Xβ~​∗∥2≤2∥β∗∥12​/k.
  2. (39) (p. 44). With Δ∗=Xβ∗−Xβ~∗\Delta^*=X\beta^*-X\tilde\beta^*Δ∗=Xβ∗−Xβ~​∗, define G(β)=∥y−Xβ∥2−∥y−Xβ~∗∥2G(\beta)=\|y-X\beta\|^2-\|y-X\tilde\beta^*\|^2G(β)=∥y−Xβ∥2−∥y−Xβ~​∗∥2 and H(u)=∥Xu∥2−2(Δ∗+ϵ)⊤XuH(u)=\|Xu\|^2-2(\Delta^*+\epsilon)^\top XuH(u)=∥Xu∥2−2(Δ∗+ϵ)⊤Xu. Then G(β)G(\beta)G(β) equals ∥Xβ−Xβ~∗∥2+2(Δ∗+ϵ)⊤(Xβ~∗−Xβ)\|X\beta-X\tilde\beta^*\|^2+2(\Delta^*+\epsilon)^\top(X\tilde\beta^*-X\beta)∥Xβ−Xβ~​∗∥2+2(Δ∗+ϵ)⊤(Xβ~​∗−Xβ), and min⁡∥β∥0≤kG≤min⁡H(u)\min_{\|\beta\|_0\le k}G\le\min H(u)min∥β∥0​≤k​G≤minH(u) over uuu supported off supp⁡(β~∗)\operatorname{supp}(\tilde\beta^*)supp(β~​∗) with ∥u∥0≤k/2\|u\|_0\le k/2∥u∥0​≤k/2.
  3. (40) (p. 45). min⁡∥β∥0≤kG(β)≤−max⁡v∈Vk∣(Δ∗+ϵ)⊤Xv∣2\min_{\|\beta\|_0\le k}G(\beta)\le-\max_{v\in\mathcal V_k}|(\Delta^*+\epsilon)^\top Xv|^2min∥β∥0​≤k​G(β)≤−maxv∈Vk​​∣(Δ∗+ϵ)⊤Xv∣2, where Vk\mathcal V_kVk​ is the set of vvv supported off supp⁡(β~∗)\operatorname{supp}(\tilde\beta^*)supp(β~​∗) with ∥v∥0≤k/2\|v\|_0\le k/2∥v∥0​≤k/2 and ∥Xv∥=1\|Xv\|=1∥Xv∥=1.
  4. Lemma 1 (p. 44). Let SSS have qqq elements and let s∈[1,q]s\in[1,q]s∈[1,q] be an integer. Then max⁡∣ϵ⊤Xv∣≳σγ2sslog⁡(eq/s)\max|\epsilon^\top Xv|\gtrsim\sigma\gamma_{2s}\sqrt{s\log(eq/s)}max∣ϵ⊤Xv∣≳σγ2s​slog(eq/s)​ over supp⁡(v)⊆S\operatorname{supp}(v)\subseteq Ssupp(v)⊆S, ∥v∥0≤s\|v\|_0\le s∥v∥0​≤s, ∥Xv∥=1\|Xv\|=1∥Xv∥=1, with probability at least 1−2(eq/s)−c~γ2s2s1-2(eq/s)^{-\tilde c\gamma_{2s}^2s}1−2(eq/s)−c~γ2s2​s.

Significance

Theorem 4 matches the fast-rate upper bound for best subsets from Corollary 1 of the paper. When γk\gamma_kγk​ is bounded below, the prediction error of best subsets is of exact order σ2klog⁡(ep/k)\sigma^2k\log(ep/k)σ2klog(ep/k) in the low-SNR regime (Remark 10). This is the theoretical counterpart of the empirical observation that best subsets overfits at low SNR. It is also what makes the paper's case for shrinkage a theorem rather than a comparison of upper bounds. The paper's Proposition 4 applies it to a concrete correlated design, where it gives a ratio of prediction errors between best subsets and its ℓ2\ell_2ℓ2​-shrunken version that grows with kkk.

The result is proved in the paper; this mission formalizes it. No formalization of this result exists on the platform. The mission also produces a reusable Gaussian lower bound on sparse correlations (Lemma 1) and a deterministic reduction of the best-subsets objective, (39)–(40), which are of independent use for ℓ0\ell_0ℓ0​ estimators.

Difficulty

An upper bound on best subsets follows from comparing β^ℓ0\hat\beta_{\ell_0}β^​ℓ0​​ with β∗\beta^*β∗ in a basic inequality. A lower bound must hold for every minimizer, and β∗\beta^*β∗ itself gives no lower bound. The first idea is to compare with the noiseless fit. It fails because the noise can be fitted by sparse directions, which is exactly the effect to be quantified. The difficult step is Lemma 1. It requires a large set of sparse directions that remain separated after multiplication by XXX, a lower bound on the expected maximum of a Gaussian process over that set, and concentration of that maximum around its mean, with all constants universal. A second subtlety is that the statement is about every minimizer of a nonconvex problem, so the proof has to bound the objective rather than the estimator.

Formalization scope

Vectors are Fin n → ℝ and Fin p → ℝ, with indices starting at 0. The design is X : Matrix (Fin n) (Fin p) ℝ. The Euclidean norm, ℓ1\ell_1ℓ1​ norm, support and ∥⋅∥0\|\cdot\|_0∥⋅∥0​ are explicit definitions, since Mathlib's norm on Fin m → ℝ is the sup norm. γs\gamma_sγs​ is an infimum over the nonzero vectors with ∥u∥0≤s\|u\|_0\le s∥u∥0​≤s; this set is nonempty when p,s≥1p,s\ge1p,s≥1, and every ratio is ≥0\ge0≥0. The noise law is the product of nnn centred Gaussians of variance σ2\sigma^2σ2 on Rn\mathbb R^nRn. "With probability at least 1−η1-\eta1−η" means that a measurable set of noise vectors of probability at least 1−η1-\eta1−η exists on which the conclusion holds. Best subsets is an argmin predicate: Theorem 4 quantifies over every minimizer, and existence is neither assumed nor needed. Universal constants are quantified before every other parameter. "∥⋅∥0≤k/2\|\cdot\|_0\le k/2∥⋅∥0​≤k/2" is a comparison of real numbers.

The model consists of XXX, β∗\beta^*β∗ (with f∗=Xβ∗f^*=X\beta^*f∗=Xβ∗), ϵ\epsilonϵ and its Gaussian law, the estimator (1), and the minimal sparse eigenvalue γs\gamma_sγs​.

Two hypotheses are pinned relative to the printed theorem:

  • k≥2k\ge2k≥2, because the statement is false for k=1k=1k=1. Take every column equal to e1e_1e1​ and β∗=0\beta^*=0β∗=0: the error is ϵ12\epsilon_1^2ϵ12​, while the bound demands order σ2log⁡(ep)\sigma^2\log(ep)σ2log(ep) with probability tending to one.
  • ∥β∗∥0≤k\|\beta^*\|_0\le k∥β∗∥0​≤k, because the proof uses ∥β∗∥1\|\beta^*\|_1∥β∗∥1​, which the printed ℓ2\ell_2ℓ2​ hypothesis controls only for sparse β∗\beta^*β∗, and §3.4 places the theorem in Corollary 1's setting β∗∈B0(k)\beta^*\in B_0(k)β∗∈B0​(k).

The milestones carry neither pin. Because the constants are existential, a formalization in which c0c_0c0​, ccc or CCC may depend on ppp, kkk or XXX would be trivial, and is ruled out by quantifying them first. A version asserting the bound for some minimizer would be weaker than the theorem.

Not formalized here: the upper bounds (Theorems 2, 3, 5), the noise events of Theorem 1, Corollaries 1–6, Propositions 3–4, and the numerical experiments. The proof of the goal also uses Theorem 1's event G2k\mathcal G_{2k}G2k​, which belongs to a separate mission of this series. Proved platform results that a proof of Lemma 1 can use include Sudakov minoration for separated Gaussian families (HighDimProb.RandomProcesses.sudakov_minoration_finite_separated), Gaussian Lipschitz concentration (HighDimStat.TailBounds.lipschitz_gaussian_concentration) and a lower bound on Gaussian maxima (HighDimProb.RandomProcesses.standard_gaussian_max_lower_bound). Contributions are welcome on any milestone, and on supporting lemmas: a packing of sparse sign vectors (Gilbert–Varshamov type), and the approximation of ℓ1\ell_1ℓ1​-bounded vectors by sparse ones.

Selected references

  • R. Mazumder, P. Radchenko, A. Dedieu, Subset Selection with Shrinkage: Sparse Linear Modeling When the SNR Is Low, Operations Research, 2023; preprint arXiv:1708.03288v4, 2022. https://arxiv.org/abs/1708.03288v4
  • D. Bertsimas, A. King, R. Mazumder, Best subset selection via a modern optimization lens, Annals of Statistics 44(2), 2016. https://doi.org/10.1214/15-AOS1388
  • T. Hastie, R. Tibshirani, R. Tibshirani, Best Subset, Forward Stepwise or Lasso? Analysis and Recommendations Based on Extensive Comparisons, Statistical Science 35(4), 2020. https://doi.org/10.1214/19-STS733
  • P. C. Bellec, G. Lecué, A. B. Tsybakov, Slope meets Lasso: improved oracle bounds and optimality, Annals of Statistics 46(6B), 2018. https://doi.org/10.1214/17-AOS1670
  • R. Vershynin, High-Dimensional Probability, Cambridge University Press, 2018. https://doi.org/10.1017/9781108231596
8 thms1 active userReviewed
Machine LearningStatistics·Captain: mikedeng1

Subset Selection with Shrinkage: Sparse Linear Modeling When the SNR Is Low IV: ℓ1-Regularized Best Subset Selection Satisfies Slow-Rate and Fast-Rate Oracle InequalitiesResearch Paper

Why shrink a best subset?

Best subset selection fits a linear predictor using at most kkk of ppp available columns. Its explicit limit on the number of selected variables makes the model easy to interpret, but the search can still fit noise: among many subsets of the same size, one can align unusually well with a noisy response. Mazumder, Radchenko and Dedieu study this effect in Subset Selection with Shrinkage. They add a penalty on the size of the selected coefficients and show how this changes prediction error when the signal-to-noise ratio is low.

An ℓ1\ell_1ℓ1​ penalty is familiar from the Lasso. The estimator here also keeps the subset-size constraint, so its comparison class and its tuning parameter have a different role from those in an unconstrained Lasso. The paper's Theorem 3 gives three deterministic error bounds for this estimator, each valid on a specified event of the Gaussian noise. These bounds are the target of this mission. The separate probability estimates for the events are Theorem 1 of the paper and belong to another mission in this series.

Setting

Let X∈Rn×pX\in\mathbb R^{n\times p}X∈Rn×p be a fixed design matrix whose columns have Euclidean norm one. The observation is y=f∗+ϵy=f^*+\epsilony=f∗+ϵ, where f∗∈Rnf^*\in\mathbb R^nf∗∈Rn is a fixed mean and the coordinates of ϵ\epsilonϵ are independent N(0,σ2)N(0,\sigma^2)N(0,σ2) random variables, with σ>0\sigma>0σ>0. The result below fixes one realized noise vector ϵ\epsilonϵ and assumes it lies in an event stated below. No assumption that f∗f^*f∗ is itself a linear combination of columns of XXX is required.

For 1≤k≤p1\le k\le p1≤k≤p, write B0(k)={β∈Rp:∥β∥0≤k}B_0(k)=\{\beta\in\mathbb R^p:\|\beta\|_0\le k\}B0​(k)={β∈Rp:∥β∥0​≤k}, where ∥β∥0\|\beta\|_0∥β∥0​ counts nonzero coordinates. An ℓ1\ell_1ℓ1​-regularized best subset estimator is any minimizer

β^1∈argmin⁡β∈B0(k){∥y−Xβ∥22+λ∥β∥1}.(16)\hat\beta_1\in\operatorname*{argmin}_{\beta\in B_0(k)}\bigl\{\|y-X\beta\|_2^2+\lambda\|\beta\|_1\bigr\}.\tag{16}β^​1​∈β∈B0​(k)argmin​{∥y−Xβ∥22​+λ∥β∥1​}.(16)

The objective has no factor 1/21/21/2. The assertion applies to every minimizer, so tied solutions are covered. The sparse minimum singular value is γs=inf⁡{∥Xu∥2/∥u∥2:u≠0, ∥u∥0≤s}\gamma_s=\inf\{\|Xu\|_2/\|u\|_2:u\ne0,\ \|u\|_0\le s\}γs​=inf{∥Xu∥2​/∥u∥2​:u=0, ∥u∥0​≤s}. When p≥1p\ge1p≥1 and s≥1s\ge1s≥1 its domain is nonempty. Write u1♯,…,up♯u_1^\sharp,\ldots,u_p^\sharpu1♯​,…,up♯​ for the decreasing rearrangement of ∣u1∣,…,∣up∣|u_1|,\ldots,|u_p|∣u1​∣,…,∣up​∣.

The paper defines three noise events used here. H\mathcal HH bounds the largest coordinate of X⊤ϵX^\top\epsilonX⊤ϵ by σ2log⁡(2p)+σ2log⁡(1/δ0)\sigma\sqrt{2\log(2p)}+\sigma\sqrt{2\log(1/\delta_0)}σ2log(2p)​+σ2log(1/δ0​)​. On F\mathcal FF, every u∈Rpu\in\mathbb R^pu∈Rp satisfies

ϵ⊤Xu≤(4+2)σmax⁡{∑j=1puj♯log⁡(2p/j), log⁡(1/δ0) ∥Xu∥2}.\epsilon^\top Xu\le(4+\sqrt2)\sigma\max\left\{\sum_{j=1}^{p}u_j^\sharp\sqrt{\log(2p/j)},\ \sqrt{\log(1/\delta_0)}\,\|Xu\|_2\right\}.ϵ⊤Xu≤(4+2​)σmax{j=1∑p​uj♯​log(2p/j)​, log(1/δ0​)​∥Xu∥2​}.

On Gs\mathcal G_sGs​, every u∈B0(s)u\in B_0(s)u∈B0​(s) satisfies ϵ⊤Xu≤σ5slog⁡(ep/s)+log⁡(1/δ0) ∥Xu∥2\epsilon^\top Xu\le\sigma\sqrt{5s\log(ep/s)+\log(1/\delta_0)}\,\|Xu\|_2ϵ⊤Xu≤σ5slog(ep/s)+log(1/δ0​)​∥Xu∥2​. Here δ0∈(0,1]\delta_0\in(0,1]δ0​∈(0,1] and all logarithms are natural. These definitions, including the constant in Gs\mathcal G_sGs​, are the printed deterministic events of p. 13.

Formalization targets

Exact slow rate on H\mathcal HH

At λ=2σ2log⁡(2p)+2σ2log⁡(1/δ0)\lambda=2\sigma\sqrt{2\log(2p)}+2\sigma\sqrt{2\log(1/\delta_0)}λ=2σ2log(2p)​+2σ2log(1/δ0​)​, Theorem 3(A) states

∥f∗−Xβ^1∥22≤inf⁡β∈B0(k)[∥f∗−Xβ∥22+2λ∥β∥1](ϵ∈H).\|f^*-X\hat\beta_1\|_2^2\le\inf_{\beta\in B_0(k)}\bigl[\|f^*-X\beta\|_2^2+2\lambda\|\beta\|_1\bigr]\qquad(\epsilon\in\mathcal H).∥f∗−Xβ^​1​∥22​≤β∈B0​(k)inf​[∥f∗−Xβ∥22​+2λ∥β∥1​](ϵ∈H).

Slow rate on F\mathcal FF

For λ≥(8+22)σlog⁡(2p)\lambda\ge(8+2\sqrt2)\sigma\sqrt{\log(2p)}λ≥(8+22​)σlog(2p)​, a positive universal constant CAC_ACA​ gives

∥f∗−Xβ^1∥22≤CA[inf⁡β∈B0(k)(∥f∗−Xβ∥22+λ∥β∥1)+σ2log⁡(1/δ0)](ϵ∈F).\|f^*-X\hat\beta_1\|_2^2\le C_A\left[\inf_{\beta\in B_0(k)}\bigl(\|f^*-X\beta\|_2^2+\lambda\|\beta\|_1\bigr)+\sigma^2\log(1/\delta_0)\right]\qquad(\epsilon\in\mathcal F).∥f∗−Xβ^​1​∥22​≤CA​[β∈B0​(k)inf​(∥f∗−Xβ∥22​+λ∥β∥1​)+σ2log(1/δ0​)](ϵ∈F).

Fast rate on G2k\mathcal G_{2k}G2k​

For every λ≥0\lambda\ge0λ≥0, a positive universal constant CBC_BCB​ gives

∥f∗−Xβ^1∥22≤CB[inf⁡β∈B0(k)∥f∗−Xβ∥22+σ2klog⁡(ep/k)+γ2k−2λ2k+σ2log⁡(1/δ0)](ϵ∈G2k).\|f^*-X\hat\beta_1\|_2^2\le C_B\left[\inf_{\beta\in B_0(k)}\|f^*-X\beta\|_2^2+\sigma^2k\log(ep/k)+\gamma_{2k}^{-2}\lambda^2 k+\sigma^2\log(1/\delta_0)\right]\qquad(\epsilon\in\mathcal G_{2k}).∥f∗−Xβ^​1​∥22​≤CB​[β∈B0​(k)inf​∥f∗−Xβ∥22​+σ2klog(ep/k)+γ2k−2​λ2k+σ2log(1/δ0​)](ϵ∈G2k​).

The kkk multiplying γ2k−2λ2\gamma_{2k}^{-2}\lambda^2γ2k−2​λ2 distinguishes this rate from the ℓ2\ell_2ℓ2​-penalized estimator's bound. The three claims together, with their separate tuning conditions, are Theorem 3 on p. 16. The milestones follow the displays of its proof in Supplement B.6, pp. 42–43.

What the result provides

Each inequality compares the prediction of any selected minimizer with that of every sparse competitor. The first bound has an exact leading constant and no additive noise term on H\mathcal HH. The second keeps a linear penalty term when λ\lambdaλ meets a fixed noise-dependent threshold. The third replaces the linear term with a quadratic λ\lambdaλ term controlled by the sparse minimum singular value. This distinguishes regimes in which coefficient shrinkage changes the error rate. The paper also combines deterministic bounds with event probabilities in subsequent corollaries; those probability statements are separate from this goal.

The paper proves Theorem 3. The formalization task is to verify its model definitions, event interpretation and all three inequalities in Lean, with the same constants and quantifiers. The milestone statements isolate the basic inequality, the H\mathcal HH bound, the rearranged-coordinate bound, the F\mathcal FF case split and the sparse ℓ1\ell_1ℓ1​ step. A complete proof would also provide reusable finite-dimensional norm and matrix estimates. This mission does not include the paper's numerical experiments, data-driven model-size selection, the ℓ2\ell_2ℓ2​-penalized estimator, or the event-probability theorem.

Difficulty

The constraint ∥β∥0≤k\|\beta\|_0\le k∥β∥0​≤k makes the estimator a minimizer over a nonconvex set. A standard unconstrained Lasso argument does not directly supply the paper's bound for every feasible sparse comparator. The slow rates rely on two different noise controls: a coordinatewise bound on H\mathcal HH and a maximum of a sorted-coordinate term and a prediction-norm term on F\mathcal FF. The fast rate also needs the sparse minimum singular value to relate coefficient error to prediction error. The constants in these steps matter: replacing the exact first tuning equality or dropping the factor kkk in the fast rate would state a different theorem.

Formalization scope

Lean uses Fin n → ℝ and Fin p → ℝ for vectors and Matrix (Fin n) (Fin p) ℝ for XXX. Norms are explicit finite sums, rather than the function type's default supremum norm. The paper's one-based sorted index jjj is j.val + 1; the sum using u♯u^\sharpu♯ is encoded as a maximum over coordinate permutations, which agrees with the decreasing rearrangement. The events are predicates on realized noise vectors. The Gaussian law explains the model but is not needed in these deterministic statements. The estimator is an argmin predicate, never a chosen minimizer.

The universal constants precede all dimensions and data in the Lean goal. The infima in the three displayed bounds are expressed as inequalities for every β∈B0(k)\beta\in B_0(k)β∈B0​(k), avoiding the default value of an empty or unbounded real infimum. The standing conditions are 1≤k≤p1\le k\le p1≤k≤p, unit-norm columns, σ>0\sigma>0σ>0 and 0<δ0≤10<\delta_0\le10<δ0​≤1. The first slow-rate branch retains the paper's equality for λ\lambdaλ. The fast branch additionally requires γ2k>0\gamma_{2k}>0γ2k​>0 or λ=0\lambda=0λ=0: this makes the λ2/γ2k2\lambda^2/\gamma_{2k}^2λ2/γ2k2​ term meaningful under Lean's total division, which returns zero at a zero denominator. The displayed events use the paper's original Gs\mathcal G_sGs​ constant; this mission makes only a deterministic claim on that event.

The sparse domain for γ2k\gamma_{2k}γ2k​ is nonempty under these conditions. All square-root radicands and logarithm arguments in the target are in their intended ranges. The formalization must rule out a universal constant that depends on nnn, ppp, kkk, XXX or λ\lambdaλ, and must apply to every minimizer. Contributions to the shared norm inequalities, rearrangement identity and matrix calculations are useful beyond this theorem.

Selected references

  • Rahul Mazumder, Peter Radchenko and Antoine Dedieu, Subset Selection with Shrinkage: Sparse Linear Modeling When the SNR Is Low, Operations Research 71 (2023), arXiv preprint version 4 (2022), arXiv:1708.03288v4. This mission cites the preprint's page and theorem numbering.
9 thms1 active userReviewed
EconomicsOptimization·Captain: mikedeng1

Supply Risk Mitigation in a Decentralized Supply Chain: Pricing Postponement or Payment Postponement? III: When Payment Postponement Helps or Hurts the Retailer, the Supplier and the ConsumersResearch Paper

Why payment timing matters

A retailer can pay a supplier when it places an order or after the supplier delivers it. When production yield is uncertain, the choice changes who bears the cost of missing units. It can also change the supplier's production decision and, through the amount available for sale, the surplus received by consumers. Geng, Guo, Xiao and Yang study these effects in a decentralized supply chain and show that payment postponement can help all three parties, hurt all three, or benefit some while hurting others, depending on production cost and the retailer's pricing rule. Their Proposition 4 gives the precise cost intervals for these outcomes in the SSRN author version, pp. 19–20.

The result matters to contract design in supply chains where realized output can be below planned production. Paying for units ordered and paying for units delivered allocate that shortage differently. A comparison based only on the retailer's expected profit would miss the effects on the upstream supplier and final consumers. Proposition 4 keeps all three outcomes visible while holding the retailer's pricing policy fixed.

The supply-chain game

There is one supplier and one retailer. The supplier chooses a wholesale price www and plans production QQQ at cost cQcQcQ. The retailer orders qqq and sells to a market with linear demand d(p)=a−bpd(p)=a-bpd(p)=a−bp, where a,b>0a,b>0a,b>0 and ppp is the retail price. The yield factor YYY equals 111 with probability θ\thetaθ and α\alphaα with probability 1−θ1-\theta1−θ, with 0<α,θ<10<\alpha,\theta<10<α,θ<1. Thus μ=EY=α(1−θ)+θ\mu=\mathbb E Y=\alpha(1-\theta)+\thetaμ=EY=α(1−θ)+θ. The retailer receives q^=min⁡{YQ,q}\widehat q=\min\{YQ,q\}q​=min{YQ,q} units; the production cost applies to planned QQQ, including units lost to yield risk §3.1–3.2.

Under pay-by-order, the retailer transfers wqwqwq to the supplier and the supplier produces Q=qQ=qQ=q. Under pay-by-delivery, the transfer is wq^w\widehat qwq​ and the supplier can choose a larger production quantity. Under ex ante pricing, the retailer fixes ppp before yield is known. Under responsive pricing, the retailer sets ppp after observing delivery. Combining payment and pricing gives OA (order, ex ante), OR (order, responsive), DA (delivery, ex ante), and DR (delivery, responsive). The supplier first sets www, the retailer next orders and possibly fixes a price, the supplier chooses production, yield is realized, and responsive pricing occurs last Table 1, p. 9.

Both firms maximize expected profit at every stage. A subgame-perfect equilibrium is a complete plan of choices after every history in which each choice is optimal at that history. For scenario kkk, write πRk\pi_R^kπRk​ and πSk\pi_S^kπSk​ for equilibrium retailer and supplier profit. The consumer surplus CSkCS^kCSk is expected surplus from sales: for delivered stock XXX and market price ppp, the model integrates inverse demand less ppp from zero to min⁡{d(p),X}\min\{d(p),X\}min{d(p),X}. This is the paper's priority allocation convention for consumers with the highest willingness to pay p. 10.

Formalization targets

The main target is Proposition 4, which compares OA with DA and OR with DR. Let Δ(πR,πS,CS)\Delta(\pi_R,\pi_S,CS)Δ(πR​,πS​,CS) mean the delivery-payment outcome minus its order-payment counterpart. A plus sign below means a strict increase, a minus sign a strict decrease, and zero equality. The admissible cost range is 0<c<aμ/b0<c<a\mu/b0<c<aμ/b.

For ex ante pricing, with no restriction α≥9/16\alpha\ge9/16α≥9/16, the paper gives a threshold 0<C‾<cDA0<\overline C<c^{DA}0<C<cDA and the following pattern:

cost intervalsign⁡Δ(πR,πS,CS)(max⁡{0,c2OA},max⁡{C‾,c2OA})(+,+,+)(max⁡{C‾,c2OA},cDA)(+,+,−)(0,max⁡{0,c1OA}) or (cDA,aμ/b)(0,0,0).\begin{array}{c|c} \text{cost interval}&\operatorname{sign}\Delta(\pi_R,\pi_S,CS)\\ (\max\{0,c^{OA}_2\},\max\{\overline C,c^{OA}_2\}) &(+,+,+)\\ (\max\{\overline C,c^{OA}_2\},c^{DA}) &(+,+,-)\\ (0,\max\{0,c^{OA}_1\})\text{ or }(c^{DA},a\mu/b)&(0,0,0). \end{array}cost interval(max{0,c2OA​},max{C,c2OA​})(max{C,c2OA​},cDA)(0,max{0,c1OA​}) or (cDA,aμ/b)​signΔ(πR​,πS​,CS)(+,+,+)(+,+,−)(0,0,0).​

For responsive pricing, assuming α≥9/16\alpha\ge9/16α≥9/16, the pattern is (+,+,+)(+,+,+)(+,+,+) on (0,C~)(0,\widetilde C)(0,C), (−,−,−)(-,-,-)(−,−,−) on (C~,c1DR)(\widetilde C,c^{DR}_1)(C,c1DR​), (+,−,+)(+,-,+)(+,−,+) on (c1DR,c2DR)(c^{DR}_1,c^{DR}_2)(c1DR​,c2DR​), and (0,0,0)(0,0,0)(0,0,0) on (c2DR,aμ/b)(c^{DR}_2,a\mu/b)(c2DR​,aμ/b). The thresholds are those specified in the paper: C~\widetilde CC is equation (6), C‾\overline CC is equation (8), and the scenario cost thresholds come from Appendix B. The first ex ante interval can be empty for some parameters; its printed endpoints are retained. The milestones establish the supplier's production and responsive pricing rules, the four scenario outcomes, and the threshold ordering used in the comparison Appendices B–C.

What the result provides

The theorem separates the effect of changing the payment contract from the effect of changing the pricing rule. For advance pricing it identifies an interval where payment postponement helps both firms and consumers, then an interval where consumers lose despite both firms gaining. For responsive pricing the direction of every effect can reverse as cost passes successive thresholds. At high cost the change in payment timing leaves all three equilibrium outcomes unchanged Proposition 4, pp. 19–20.

The paper derives the equilibrium formulas in Appendix B and proves the comparisons in Appendix C. The result is therefore a known mathematical claim, while this mission asks for a machine-checked development from the sequential game through the cost-dependent equilibrium outcomes and final comparisons. The definitions of a two-point yield game and its complete contingent strategies can support other formal studies of payment or pricing postponement. There is no claim that Proposition 4 already has a machine-checked proof.

Where the formal work is difficult

The supplier's production payoff has kinks at Q=qQ=qQ=q and Q=q/αQ=q/\alphaQ=q/α. The retailer's best order changes when those production choices change, and responsive pricing adds another kink when delivered stock crosses a/2a/2a/2. As wholesale price varies, the supplier must compare outcomes on both sides of these changes. Consequently a single first-order condition for a smooth objective does not characterize the equilibrium. The cost thresholds also depend on square roots and comparisons among several formula pieces, including the larger root chosen for c2OAc^{OA}_2c2OA​ Appendix B, pp. 33–37.

Formalization scope

Lean represents expectation by the two weighted yield outcomes. Production cost ccc is a separate argument, so thresholds depend only on (a,b,α,θ)(a,b,\alpha,\theta)(a,b,α,θ). Parameters satisfy a,b>0a,b>0a,b>0, 0<α,θ<10<\alpha,\theta<10<α,θ<1, and 0<c<aμ/b0<c<a\mu/b0<c<aμ/b. Wholesale and retail prices obey the strict admissibility bounds of equation (3). Orders are nonnegative; pay-by-delivery production is nonnegative; pay-by-order production equals the order. Consumer surplus is a real interval integral. Every comparison quantifies over actual subgame-perfect equilibria, and the goal separately asserts that the compared equilibria exist at every admissible cost. Equilibrium profits are computed from strategies and transfers, rather than defined to equal the Appendix B formulas.

The paper's statements require care at ties. The model selects Q=qQ=qQ=q when that is a profit-maximizing pay-by-delivery production choice, including the indifferent DR regime. The printed middle DR consumer-surplus expression combines outcomes from different production choices; the equilibrium-consistent expression is used here, and Proposition 4(R)'s comparison remains valid under either expression. Equation (2) swaps the scenario labels of committed and responsive prices, so consumer surplus uses the actual ex ante price in OA and DA and the price after delivery in OR and DR. The printed [x]+[x]^+[x]+ in Appendix B.1 is read as max⁡{0,x}\max\{0,x\}max{0,x}, as in Lemma 1. Outcome claims use open cost intervals because uniqueness at exact thresholds fails; equilibrium existence remains part of the targets there. The threshold c2OAc^{OA}_2c2OA​ is the larger root of the relevant quadratic factor, and C‾\overline CC and C~\widetilde CC use the paper's explicit formulas. These conventions are recorded in the local moderation notes.

The development calls for real inequality reasoning, optimization of piecewise objectives, and careful handling of interval integrals. Contributions that prove the production and price best responses, establish the four equilibrium lemmas, or verify the threshold order each advance the final comparison.

Selected references

  • X. Geng, X. Guo, G. Xiao and N. Yang, Supply Risk Mitigation in a Decentralized Supply Chain: Pricing Postponement or Payment Postponement?, Manufacturing & Service Operations Management 25(3), 2023. SSRN author version 3546872; DOI: 10.1287/msom.2022.0198.
11 thms1 active userReviewed
EconomicsOptimization·Captain: mikedeng1

Supply Risk Mitigation in a Decentralized Supply Chain: Pricing Postponement or Payment Postponement? II: Under Reliable Yield the Postponements Are Complements Only at Intermediate CostResearch Paper

Motivation

A retailer that buys from a supplier with unreliable production has two contractual levers against supply risk, and both postpone a decision until after the yield is known. Payment postponement means paying only for the units actually delivered (pay-by-delivery) instead of the units ordered (pay-by-order). This moves the yield risk upstream. Pricing postponement means setting the retail price after the delivery is observed (responsive pricing) instead of committing to it in advance (ex ante pricing). This moves the risk downstream to consumers. Both practices are common in procurement, and the natural intuition is that two ways of shedding the same risk should be substitutes: once one lever is in use, the other should be worth less.

Geng, Guo, Xiao and Yang (MSOM 25(3), 2023; author version SSRN 3546872) compute the subgame-perfect equilibria of all four combinations of the two levers in a stylized supplier–retailer game. Their Proposition 3 shows that the intuition fails at intermediate production cost. When full yield is likely and yield loss is mild, the two postponements are complements there: postponing payment is worth more when pricing is also postponed. This mission formalizes that result together with the four equilibrium characterizations it rests on.

Setting

A supplier sells one product to a retailer, who sells it in a market with linear demand d(p)=a−bpd(p)=a-bpd(p)=a−bp, a,b>0a,b>0a,b>0. The supplier plans production QQQ at unit cost ccc and realizes the fraction YYY of it, where

Y=1 with probability θ,Y=α with probability 1−θ,0<α,θ<1,Y=1 \text{ with probability } \theta,\qquad Y=\alpha \text{ with probability } 1-\theta,\qquad 0<\alpha,\theta<1,Y=1 with probability θ,Y=α with probability 1−θ,0<α,θ<1,

and μ=α(1−θ)+θ\mu=\alpha(1-\theta)+\thetaμ=α(1−θ)+θ is the mean yield. The retailer orders qqq and receives q^=min⁡{YQ,q}\hat q=\min\{YQ,q\}q^​=min{YQ,q}.

The four scenarios are OA, OR, DA and DR. The first letter is the payment rule: by order (O, the transfer is wqwqwq) or by delivery (D, the transfer is wq^w\hat qwq^​). The second letter is the pricing rule: ex ante (A, the price is fixed before the yield) or responsive (R, the price is set after q^\hat qq^​ is observed). Each scenario is a sequential game. The supplier posts a wholesale price www. The retailer orders qqq, and also commits to a price ppp in the ex ante scenarios. The supplier then chooses QQQ; under pay-by-order it produces exactly qqq. The yield realizes, and under responsive pricing the retailer then sets ppp. The retailer earns pmin⁡{d(p),q^}p\min\{d(p),\hat q\}pmin{d(p),q^​} minus the transfer, and the supplier earns the transfer minus cQcQcQ. Both are risk neutral. Admissible actions follow the paper's standing ranges: 0<c<aμ/b0<c<a\mu/b0<c<aμ/b, 0<p<a/b0<p<a/b0<p<a/b, and www between ccc (or c/μc/\muc/μ under pay-by-delivery) and a/ba/ba/b.

Write πRk\pi^k_RπRk​ for the retailer's expected profit in a subgame-perfect equilibrium of scenario kkk. The paper's Definition 1 measures the interaction of the two postponements by

V=(πRDR−πROR)−(πRDA−πROA):V=\big(\pi^{DR}_R-\pi^{OR}_R\big)-\big(\pi^{DA}_R-\pi^{OA}_R\big):V=(πRDR​−πROR​)−(πRDA​−πROA​):

the value of payment postponement under responsive pricing, minus its value under ex ante pricing. The postponements are substitutes if V<0V<0V<0 and complements if V>0V>0V>0.

The parameter region of the result is α≥9/16\alpha\ge 9/16α≥9/16 together with H3={θ2(α)<θ<1}\mathcal H_3=\{\theta_2(\alpha)<\theta<1\}H3​={θ2​(α)<θ<1}, where θ2(α)\theta_2(\alpha)θ2​(α) is the largest real root of the cubic −α3+α2(5+α)x+(16−19α−6α2)x2−25(1−α)x3-\alpha^3+\alpha^2(5+\alpha)x+(16-19\alpha-6\alpha^2)x^2-25(1-\alpha)x^3−α3+α2(5+α)x+(16−19α−6α2)x2−25(1−α)x3. The cost thresholds c1DR<c2DRc^{DR}_1<c^{DR}_2c1DR​<c2DR​ (scenario DR) and cDAc^{DA}cDA (scenario DA) are explicit algebraic functions of (a,b,α,θ)(a,b,\alpha,\theta)(a,b,α,θ) given in the paper's Appendix B.

Formalization targets

Goal: Proposition 3

For α≥9/16\alpha\ge9/16α≥9/16 and (α,θ)∈H3(\alpha,\theta)\in\mathcal H_3(α,θ)∈H3​, every scenario has an equilibrium at every cost 0<c<aμ/b0<c<a\mu/b0<c<aμ/b, and there is a cost 0<C^<aμ/b0<\hat C<a\mu/b0<C^<aμ/b such that, at every choice of the four equilibria,

V<0 on (0,c1DR)∪(C^,cDA),V>0 on (c1DR,C^),V=0 on (cDA,aμ/b).V<0 \text{ on } (0,c^{DR}_1)\cup(\hat C,c^{DA}),\qquad V>0 \text{ on } (c^{DR}_1,\hat C),\qquad V=0 \text{ on } (c^{DA},a\mu/b).V<0 on (0,c1DR​)∪(C^,cDA),V>0 on (c1DR​,C^),V=0 on (cDA,aμ/b).

C^\hat CC^ is existential, as in the paper, and depends only on the parameters.

Milestones

  • the supplier's production response (4) under pay-by-delivery and the ex post price (5);
  • Lemmas 1–4: existence of the equilibrium of each scenario, and its outcome (πR,πS,CS)(\pi_R,\pi_S,CS)(πR​,πS​,CS) in closed form on each cost interval;
  • Lemma 1(i): on H3\mathcal H_3H3​ the threshold c2OAc^{OA}_2c2OA​ is nonpositive, so scenario OA has one regime;
  • the threshold claims of Appendix C: cOR<0c^{OR}<0cOR<0 and c1DR<c2DR<cDAc^{DR}_1<c^{DR}_2<c^{DA}c1DR​<c2DR​<cDA for α≥9/16\alpha\ge9/16α≥9/16.

Significance

The result separates the first-order question (which lever the retailer prefers, the subject of the paper's Proposition 2) from the second-order one (whether one lever raises the value of the other). It shows that two risk-shifting instruments can reinforce each other. On (c1DR,C^)(c^{DR}_1,\hat C)(c1DR​,C^), adopting one postponement raises the value of the other. So whenever the retailer gains from adopting either lever alone, it gains even more from adopting both. Below c1DRc^{DR}_1c1DR​ and above C^\hat CC^ the reverse holds. Above cDAc^{DA}cDA, payment postponement is irrelevant, because the supplier never inflates production there.

The paper proves Lemmas 1–4 by backward induction in Appendix B and the proposition by comparing closed forms in Appendix C. None of it is machine-checked. A formal proof requires the four backward inductions with every kink and tie handled, and the polynomial inequalities between the thresholds over the parameter region. The formalization also corrects the record: the paper's explicit formula (7) for C^\hat CC^ does not locate the sign change of VVV (at (α,θ)=(0.6,0.75)(\alpha,\theta)=(0.6,0.75)(α,θ)=(0.6,0.75) it exceeds cDAc^{DA}cDA), while the existential statement of Proposition 3 holds at every sampled point.

Difficulty

Once Lemmas 1–4 are available, VVV is a difference of explicit piecewise functions of ccc. The difficulty lies in those lemmas and in the thresholds. Each lemma is a three-stage optimization. The supplier's profit is piecewise linear in QQQ, with kinks at qqq and q/αq/\alphaq/α. The retailer's profit is piecewise in (q,p)(q,p)(q,p), with kinks where it becomes worth inflating the order. The supplier's value function in www is piecewise quadratic, and its maximum can sit at a kink. In scenario DR it sits exactly at the price where the supplier is indifferent to inflating production. A naive reading of the paper's tie rule removes the equilibrium there altogether. On the middle DR interval the sign of VVV changes inside the interval, at a root of a polynomial with nested square roots. On the other pieces, the signs of VVV and the ordering c1DR<c2DR<cDAc^{DR}_1<c^{DR}_2<c^{DA}c1DR​<c2DR​<cDA are polynomial inequalities in (α,θ)(\alpha,\theta)(α,θ), with θ2(α)\theta_2(\alpha)θ2​(α) defined implicitly as a cubic root.

Formalization scope

  • Parameters and cost. The parameters (a,b,α,θ)(a,b,\alpha,\theta)(a,b,α,θ) form a structure. The cost ccc is a separate argument, and every threshold depends on the parameters only.
  • Expectation. The expectation over the yield is the explicit two-point sum θf(1)+(1−θ)f(α)\theta f(1)+(1-\theta)f(\alpha)θf(1)+(1−θ)f(α).
  • Equilibrium. Strategies are functions of all earlier moves, and optimality is required at every node over the full admissible action sets.
  • Production ties. Under pay-by-delivery the supplier's production is a best response. When producing exactly qqq is optimal, it produces qqq. The paper's display (4) assigns that tie to inflation, but under that rule scenario DR has no equilibrium for intermediate cost. The convention reproduces the paper's DR profits.
  • Consumer surplus. It is the integral ∫0min⁡{d(p),q^}(p(t)−p) dt\int_0^{\min\{d(p),\hat q\}}(p(t)-p)\,dt∫0min{d(p),q^​}​(p(t)−p)dt. It uses the ex post price in the responsive scenarios and the committed price in the ex ante ones; the labels of the paper's display (2) are swapped.
  • Closed forms. Two printed consumer-surplus values are corrected to the equilibrium values: the middle piece of OA (8bα28b\alpha^28bα2 for 8bα38b\alpha^38bα3) and the middle piece of DR (one factor α2(1−θ)+θ\alpha^2(1-\theta)+\thetaα2(1−θ)+θ fewer).
  • Cost intervals. Every statement is on open cost intervals, because at a threshold the outcome is not unique.
  • Thresholds. c2OAc^{OA}_2c2OA​, "the second root" of a quartic, is the larger root of the quadratic factor that defines it. θ2\theta_2θ2​ is the supremum of the cubic's real roots, a finite nonempty set.

The equilibrium profits are outcomes of the equilibrium predicate. They are never defined by their closed forms, and the goal carries the existence of an equilibrium in each scenario. Without that, the comparison would be vacuous whenever the predicate is unsatisfiable.

Every object is elementary: real arithmetic, square roots, min/max, one interval integral and one sSup. The equilibrium predicate and the four lemmas are shared with the three companion missions of this paper (Propositions 2, 4 and 5). Proofs of the lemmas, of the threshold inequalities and of the goal are all welcome.

Selected references

  • X. Geng, X. Guo, G. Xiao, N. Yang, Supply Risk Mitigation in a Decentralized Supply Chain: Pricing Postponement or Payment Postponement?, Manufacturing & Service Operations Management 25(3), 2023. https://doi.org/10.1287/msom.2022.0198 ; author version SSRN 3546872, https://ssrn.com/abstract=3546872
11 thms1 active userReviewed
EconomicsOptimization·Captain: mikedeng1

Supply Risk Mitigation in a Decentralized Supply Chain: Pricing Postponement or Payment Postponement? I: As Cost Rises the Retailer Prefers Payment, Then Pricing, Then Both, Then Pricing PostponementResearch Paper

Motivation

When a supplier's production is subject to random yield, only part of a planned production run turns into usable output. A retailer sourcing from such a supplier bears the resulting shortage risk unless the contract or the selling strategy moves it elsewhere. Random yield is a classical topic in production planning (Yano and Lee, 1995), and postponing a decision until uncertainty resolves is a classical risk-mitigation tool (Van Mieghem and Dada, 1999).

Geng, Guo, Xiao and Yang study two postponements available to a retailer facing an unreliable supplier. Payment postponement replaces payment for the ordered quantity by payment for the delivered quantity, which shifts the yield risk to the supplier. Pricing postponement (responsive pricing) sets the retail price after the delivery is known, which shifts the risk to consumers. Their Proposition 2 answers which of the four combinations the retailer prefers, as a function of the production cost, when the yield loss is not too severe. This mission formalizes that proposition and the equilibrium results it rests on.

Setting

A supplier sells to a retailer, who sells to a market with linear demand d(p)=a−bpd(p)=a-bpd(p)=a−bp, a,b>0a,b>0a,b>0. The supplier's yield factor YYY takes the value 111 with probability θ\thetaθ and α\alphaα with probability 1−θ1-\theta1−θ, with α,θ∈(0,1)\alpha,\theta\in(0,1)α,θ∈(0,1), and has mean μ=α(1−θ)+θ\mu=\alpha(1-\theta)+\thetaμ=α(1−θ)+θ. The supplier plans production QQQ, pays ccc per planned unit, and delivers q^=min⁡{YQ,q}\hat q=\min\{YQ,q\}q^​=min{YQ,q} against an order qqq. Throughout, 0<c<aμ/b0<c<a\mu/b0<c<aμ/b.

There are four scenarios, written k∈{OA,OR,DA,DR}k\in\{OA,OR,DA,DR\}k∈{OA,OR,DA,DR}:

  • O (pay-by-order): the retailer pays wqwqwq, and the supplier produces exactly Q=qQ=qQ=q;
  • D (pay-by-delivery): the retailer pays wq^w\hat qwq^​, and the supplier chooses QQQ and may inflate production above qqq;
  • A (ex ante pricing): the retailer commits to the retail price ppp when it orders;
  • R (responsive pricing): the retailer sets ppp after seeing q^\hat qq^​, maximizing pmin⁡{d(p),q^}p\min\{d(p),\hat q\}pmin{d(p),q^​}.

Each scenario is a Stackelberg game. The supplier sets the wholesale price www, the retailer orders (and, ex ante, prices), the supplier produces, the yield realizes, and under responsive pricing the retailer prices. Both firms are risk neutral. The solution concept is subgame-perfect equilibrium, and πRk\pi^k_RπRk​ denotes the retailer's expected equilibrium profit in scenario kkk. Admissible prices are those of the paper's standing assumption (3): 0<p<a/b0<p<a/b0<p<a/b, and c<w<a/bc<w<a/bc<w<a/b under pay-by-order or c/μ<w<a/bc/\mu<w<a/bc/μ<w<a/b under pay-by-delivery.

The cost thresholds of the comparison are

C~=aα(1−α2(1−θ)+θ)b(1+α),c2DR=aαμ(1−θ)b(μ+θ),\tilde C=\frac{a\alpha\big(1-\sqrt{\alpha^2(1-\theta)+\theta}\big)}{b(1+\alpha)},\qquad c^{DR}_2=\frac{a\alpha\mu(1-\theta)}{b(\mu+\theta)},C~=b(1+α)aα(1−α2(1−θ)+θ​)​,c2DR​=b(μ+θ)aαμ(1−θ)​, c1DR=aα(1−θ)(μ2+θ−2θαμ(1−α)(1−θ))b(α2(1−θ)3+4αθ(1−θ)+θ(1+θ)2).c^{DR}_1=\frac{a\alpha(1-\theta)\big(\mu^2+\theta-2\theta\sqrt{\alpha\mu(1-\alpha)(1-\theta)}\big)}{b\big(\alpha^2(1-\theta)^3+4\alpha\theta(1-\theta)+\theta(1+\theta)^2\big)}.c1DR​=b(α2(1−θ)3+4αθ(1−θ)+θ(1+θ)2)aα(1−θ)(μ2+θ−2θαμ(1−α)(1−θ)​)​.

Formalization targets

Goal: Proposition 2 (p. 16)

For 9/16≤α<19/16\le\alpha<19/16≤α<1 and every 0<c<aμ/b0<c<a\mu/b0<c<aμ/b, equilibria of OR, DA and DR exist, and every choice of equilibria satisfies

πROR<πRDR=πRDA0<c<C~,πRDA=πRDR<πRORC~<c<c1DR,πRDA<πROR<πRDRc1DR<c<c2DR,πRDA<πRDR=πRORc2DR<c<aμ/b.\begin{aligned} &\pi^{OR}_R<\pi^{DR}_R=\pi^{DA}_R && 0<c<\tilde C,\\ &\pi^{DA}_R=\pi^{DR}_R<\pi^{OR}_R && \tilde C<c<c^{DR}_1,\\ &\pi^{DA}_R<\pi^{OR}_R<\pi^{DR}_R && c^{DR}_1<c<c^{DR}_2,\\ &\pi^{DA}_R<\pi^{DR}_R=\pi^{OR}_R && c^{DR}_2<c<a\mu/b. \end{aligned}​πROR​<πRDR​=πRDA​πRDA​=πRDR​<πROR​πRDA​<πROR​<πRDR​πRDA​<πRDR​=πROR​​​0<c<C~,C~<c<c1DR​,c1DR​<c<c2DR​,c2DR​<c<aμ/b.​

Milestones

  1. (4): the supplier's production under pay-by-delivery is qqq below the wholesale price c/(α(1−θ))c/(\alpha(1-\theta))c/(α(1−θ)) and q/αq/\alphaq/α above it.
  2. (5): the responsive price is (a−q^)/b(a-\hat q)/b(a−q^​)/b for q^<a/2\hat q<a/2q^​<a/2 and a/(2b)a/(2b)a/(2b) otherwise.
  3. Lemmas 2, 3, 4: existence of equilibria of OR, DA and DR, and their outcomes (πR,πS,CS)(\pi_R,\pi_S,CS)(πR​,πS​,CS) in closed form on each open cost interval (Lemma 4 for α≥9/16\alpha\ge 9/16α≥9/16).
  4. Threshold order (proof of Propositions 1–2): for α≥9/16\alpha\ge 9/16α≥9/16, cOR<0c^{OR}<0cOR<0 and c1DR<c2DR<cDAc^{DR}_1<c^{DR}_2<c^{DA}c1DR​<c2DR​<cDA.
  5. Position of C~\tilde CC~ (proof of Proposition 4): for α≥9/16\alpha\ge 9/16α≥9/16, 0<C~<c1DR0<\tilde C<c^{DR}_10<C~<c1DR​.

Significance

The proposition describes a preference cycle. Payment postponement (DA) is best when production is cheap, because the supplier then absorbs yield risk by inflating production. Pricing postponement (OR) is best at intermediate cost. Using both (DR) is strictly best on (c1DR,c2DR)(c^{DR}_1,c^{DR}_2)(c1DR​,c2DR​), where the two postponements are worth more together than either alone. Above c2DRc^{DR}_2c2DR​ the supplier stops inflating, DR and OR coincide, and pricing postponement suffices. These are the paper's managerial conclusions on contract and pricing choice under supply risk, and the other three missions of the series (strategic complementarity, and the effect of each postponement on the whole chain) rest on the same equilibria.

The paper derives the four equilibria in its Appendix B and proves the propositions in Appendix C as direct comparisons of closed forms. None of this has been machine-checked. The mission produces a formal model of the four games and a verified link from the game to the closed forms. Lemmas 2–4 are reusable by every statement about this model.

Difficulty

Once the closed forms are known, the comparison is polynomial algebra in (α,θ,c)(\alpha,\theta,c)(α,θ,c) with square roots. The work lies upstream. Each equilibrium requires a backward induction over piecewise-linear objectives in QQQ and qqq and piecewise-quadratic objectives in ppp and www. Optima occur at kinks such as the inflation threshold w0=c/(α(1−θ))w_0=c/(\alpha(1-\theta))w0​=c/(α(1−θ)), and the supplier's value function is discontinuous there, so the maximum can sit at a boundary rather than at the center of a piece. In scenario DR the equilibrium wholesale price on (c1DR,c2DR)(c^{DR}_1,c^{DR}_2)(c1DR​,c2DR​) sits exactly at w0w_0w0​, where the supplier is indifferent among all productions in [q,q/α][q,q/\alpha][q,q/α], so existence depends on how that tie is resolved. Uniqueness of outcomes, which the comparison needs, holds only on open cost intervals, because at each threshold two wholesale prices tie with different outcomes. The threshold inequalities for α≥9/16\alpha\ge 9/16α≥9/16 are not visible from the formulas and need careful estimates of the radicals.

Formalization scope

Parameters are a structure (a,b,α,θ)(a,b,\alpha,\theta)(a,b,α,θ) with validity predicate a,b>0a,b>0a,b>0, α,θ∈(0,1)\alpha,\theta\in(0,1)α,θ∈(0,1); the cost ccc is a separate real argument. The expectation over YYY is the two-term sum θf(1)+(1−θ)f(α)\theta f(1)+(1-\theta)f(\alpha)θf(1)+(1−θ)f(α), with no measure theory. A strategy profile consists of a wholesale price and rules for production Q(w,q)Q(w,q)Q(w,q), the order q(w)q(w)q(w), the ex ante price p(w)p(w)p(w) and the ex post price pr(w,q,q^)p^{\mathrm r}(w,q,\hat q)pr(w,q,q^​). Equilibrium requires optimality at every admissible history over the admissible sets of (3). Consumer surplus is the integral ∫0min⁡{d(p),X}(p(t)−p) dt\int_0^{\min\{d(p),X\}}(p(t)-p)\,dt∫0min{d(p),X}​(p(t)−p)dt, evaluated at the ex post price under responsive pricing and at the committed price under ex ante pricing.

Conventions and disclosed departures from the page:

  • Tie rule. Under pay-by-delivery, when producing exactly qqq is optimal, the supplier produces qqq. Display (4) assigns the tie at w0w_0w0​ to inflation, under which DR has no equilibrium on (c1DR,c2DR)(c^{DR}_1,c^{DR}_2)(c1DR​,c2DR​). Lemma 4's printed profits there are those of the convention.
  • CSDRCS^{DR}CSDR on the middle piece is stated with denominator 8bα2(1−θ)2(α2(1−θ)+θ)8b\alpha^2(1-\theta)^2(\alpha^2(1-\theta)+\theta)8bα2(1−θ)2(α2(1−θ)+θ); the page prints the last factor squared, which is the surplus of no single equilibrium. Proposition 2 does not involve CSCSCS.
  • Open intervals. Lemmas are stated on open cost intervals; at the thresholds outcomes are not unique.
  • C~\tilde CC~ is instantiated by (6), as Appendix C does; the page introduces it through Proposition 1's existential.
  • Display (2) prints the scenario labels of CSCSCS swapped; the Appendix B reading is used.

πRk\pi^k_RπRk​ is the retailer's profit along an equilibrium of the game, never its closed form; a formalization that defines the profits by the Appendix B formulas, or an equilibrium predicate that no profile satisfies, would make the goal trivial. The goal therefore also asserts existence of equilibria of the three scenarios.

Contributions welcome: proofs of the milestones; general lemmas on maximizing piecewise-linear and piecewise-quadratic functions with kinks; the threshold inequalities. Proposition 1 (OA versus the others) is not part of this mission.

Selected references

  • X. Geng, X. Guo, G. Xiao, N. Yang, Supply Risk Mitigation in a Decentralized Supply Chain: Pricing Postponement or Payment Postponement?, SSRN 3546872 (author version), 2022; published in Manufacturing & Service Operations Management 25(3), 2023. https://ssrn.com/abstract=3546872, https://doi.org/10.1287/msom.2022.0198
  • C. A. Yano, H. L. Lee, Lot Sizing with Random Yields: A Review, Operations Research 43(2), 1995. https://doi.org/10.1287/opre.43.2.311
  • J. A. Van Mieghem, M. Dada, Price Versus Production Postponement: Capacity and Competition, Management Science 45(12), 1999. https://doi.org/10.1287/mnsc.45.12.1631
9 thms1 active userReviewed
Machine LearningStatistics·Captain: mikedeng1

Subset Selection with Shrinkage: Sparse Linear Modeling When the SNR Is Low III: ℓ2-Regularized Best Subset Selection Satisfies Slow-Rate and Fast-Rate Oracle InequalitiesResearch Paper

Motivation

Best subset selection fits a linear model using at most kkk of ppp available predictors, choosing the kkk that minimize the residual sum of squares. Mixed-integer optimization made it computable at moderate scale (Bertsimas, King & Mazumder, 2016), which reopened the statistical question of when it should be used. Empirical comparisons (Hastie, Tibshirani & Tibshirani, 2020) found that best subsets predicts poorly when the signal-to-noise ratio (SNR) is low: by fitting the noise exactly within its chosen support it overfits, and shrinkage methods such as the Lasso do better there.

Mazumder, Radchenko & Dedieu propose to add a shrinkage penalty to the cardinality-constrained problem. Their Theorem 2 quantifies the effect of an ℓ2\ell_2ℓ2​ (Euclidean-norm) penalty: it proves prediction-error bounds that are linear in the penalty level and therefore remain small in the low-SNR regime where best subsets fails. This mission formalizes that theorem.

Setting

The data follow the model

y=f∗+ϵ,y=f^*+\epsilon,y=f∗+ϵ,

where y,f∗,ϵ∈Rny,f^*,\epsilon\in\mathbb R^ny,f∗,ϵ∈Rn, f∗f^*f∗ is an unknown deterministic mean and ϵ\epsilonϵ is a noise vector (in the paper, with independent N(0,σ2)N(0,\sigma^2)N(0,σ2) entries, σ>0\sigma>0σ>0). The design X∈Rn×pX\in\mathbb R^{n\times p}X∈Rn×p is deterministic and its columns have unit Euclidean norm. Write ∥⋅∥\|\cdot\|∥⋅∥ for the Euclidean norm, ∥β∥0\|\beta\|_0∥β∥0​ for the number of nonzero coordinates of β\betaβ, and B0(s)={u∈Rp:∥u∥0≤s}B_0(s)=\{u\in\mathbb R^p:\|u\|_0\le s\}B0​(s)={u∈Rp:∥u∥0​≤s}.

For a model size kkk and a penalty level λ≥0\lambda\ge0λ≥0, the ℓ2\ell_2ℓ2​-regularized best-subsets estimator is any solution

β^2∈arg⁡min⁡β ∥y−Xβ∥2+λ∥β∥s.t.∥β∥0≤k.\hat\beta_2\in\arg\min_{\beta}\ \|y-X\beta\|^2+\lambda\|\beta\|\quad\text{s.t.}\quad\|\beta\|_0\le k.β^​2​∈argβmin​ ∥y−Xβ∥2+λ∥β∥s.t.∥β∥0​≤k.

The penalty is the norm ∥β∥\|\beta\|∥β∥, not its square, and the loss carries no factor 1/21/21/2. The quantity of interest is the prediction error ∥f∗−Xβ^2∥2\|f^*-X\hat\beta_2\|^2∥f∗−Xβ^​2​∥2.

The minimal sss-sparse eigenvalue of XXX is γs=min⁡u≠0, u∈B0(s)∥Xu∥/∥u∥\gamma_s=\min_{u\neq0,\,u\in B_0(s)}\|Xu\|/\|u\|γs​=minu=0,u∈B0​(s)​∥Xu∥/∥u∥. For u∈Rpu\in\mathbb R^pu∈Rp, u1♯≥⋯≥up♯u^\sharp_1\ge\dots\ge u^\sharp_pu1♯​≥⋯≥up♯​ is the non-increasing rearrangement of ∣u1∣,…,∣up∣|u_1|,\dots,|u_p|∣u1​∣,…,∣up​∣. Given δ0∈(0,1]\delta_0\in(0,1]δ0​∈(0,1], three noise events are sets of noise vectors:

Es={ϵ⊤Xu≤[4+2]σslog⁡(2ep/s) ∥u∥, ∀u∈B0(s)},\mathcal E_s=\big\{\epsilon^\top Xu\le[4+\sqrt2]\sigma\sqrt{s\log(2ep/s)}\,\|u\|,\ \forall u\in B_0(s)\big\},Es​={ϵ⊤Xu≤[4+2​]σslog(2ep/s)​∥u∥, ∀u∈B0​(s)}, F={ϵ⊤Xu≤[4+2]σmax⁡(∑j=1puj♯log⁡(2p/j), log⁡(1/δ0) ∥Xu∥), ∀u∈Rp},\mathcal F=\Big\{\epsilon^\top Xu\le[4+\sqrt2]\sigma\max\Big(\textstyle\sum_{j=1}^p u^\sharp_j\sqrt{\log(2p/j)},\ \sqrt{\log(1/\delta_0)}\,\|Xu\|\Big),\ \forall u\in\mathbb R^p\Big\},F={ϵ⊤Xu≤[4+2​]σmax(∑j=1p​uj♯​log(2p/j)​, log(1/δ0​)​∥Xu∥), ∀u∈Rp}, Gs={ϵ⊤Xu≤σ5slog⁡(ep/s)+log⁡(1/δ0) ∥Xu∥, ∀u∈B0(s)}.\mathcal G_s=\big\{\epsilon^\top Xu\le\sigma\sqrt{5s\log(ep/s)+\log(1/\delta_0)}\,\|Xu\|,\ \forall u\in B_0(s)\big\}.Gs​={ϵ⊤Xu≤σ5slog(ep/s)+log(1/δ0​)​∥Xu∥, ∀u∈B0​(s)}.

The paper's Theorem 1 shows these events have high probability; Theorem 2 is a deterministic statement on each of them. The symbol ≲\lesssim≲ means "up to a positive universal multiplicative constant", one that does not depend on n,p,k,X,f∗,σ,λn,p,k,X,f^*,\sigma,\lambdan,p,k,X,f∗,σ,λ or δ0\delta_0δ0​.

Formalization targets

Goal: Theorem 2 (p. 14)

Let 1≤k≤p1\le k\le p1≤k≤p.

(A) Slow rate. If λ≥[8+22]σ2klog⁡(ep/k)\lambda\ge[8+2\sqrt2]\sigma\sqrt{2k\log(ep/k)}λ≥[8+22​]σ2klog(ep/k)​, then on E2k\mathcal E_{2k}E2k​

∥f∗−Xβ^2∥2≤inf⁡β∈B0(k)[∥f∗−Xβ∥2+2λ∥β∥],\|f^*-X\hat\beta_2\|^2\le\inf_{\beta\in B_0(k)}\big[\|f^*-X\beta\|^2+2\lambda\|\beta\|\big],∥f∗−Xβ^​2​∥2≤β∈B0​(k)inf​[∥f∗−Xβ∥2+2λ∥β∥],

and on F\mathcal FF

∥f∗−Xβ^2∥2≲inf⁡β∈B0(k)[∥f∗−Xβ∥2+λ∥β∥]+σ2log⁡(1/δ0).\|f^*-X\hat\beta_2\|^2\lesssim\inf_{\beta\in B_0(k)}\big[\|f^*-X\beta\|^2+\lambda\|\beta\|\big]+\sigma^2\log(1/\delta_0).∥f∗−Xβ^​2​∥2≲β∈B0​(k)inf​[∥f∗−Xβ∥2+λ∥β∥]+σ2log(1/δ0​).

(B) Fast rate. For every λ≥0\lambda\ge0λ≥0, on G2k\mathcal G_{2k}G2k​

∥f∗−Xβ^2∥2≲inf⁡β∈B0(k)∥f∗−Xβ∥2+σ2klog⁡(ep/k)+γ2k−2λ2+σ2log⁡(1/δ0).\|f^*-X\hat\beta_2\|^2\lesssim\inf_{\beta\in B_0(k)}\|f^*-X\beta\|^2+\sigma^2k\log(ep/k)+\gamma_{2k}^{-2}\lambda^2+\sigma^2\log(1/\delta_0).∥f∗−Xβ^​2​∥2≲β∈B0​(k)inf​∥f∗−Xβ∥2+σ2klog(ep/k)+γ2k−2​λ2+σ2log(1/δ0​).

The bounds hold for every minimizer β^2\hat\beta_2β^​2​ and for misspecified models: f∗f^*f∗ need not lie in the column span of XXX.

Milestones

The steps of the printed proof (Supplement B.2, pp. 39–40), in the order the proof uses them: the basic inequality (29); the bound (30) on E2k\mathcal E_{2k}E2k​ and its consequence (31); the dichotomy on F\mathcal FF, where (30) implies (31) and (32) implies (33); the elementary inequality (34); the basic inequality on G2k\mathcal G_{2k}G2k​ (p. 40); and the bound of λ∥β−β^2∥\lambda\|\beta-\hat\beta_2\|λ∥β−β^​2​∥ through γ2k\gamma_{2k}γ2k​ (p. 40).

Significance

The result. The slow-rate bound of part (A) is linear in λ\lambdaλ. When the signal ∥β∗∥\|\beta^*\|∥β∗∥ is small compared with σ\sigmaσ, the term λ∥β∗∥\lambda\|\beta^*\|λ∥β∗∥ is smaller than the rate σ2klog⁡(ep/k)\sigma^2 k\log(ep/k)σ2klog(ep/k) that best subsets attains, and the paper's Theorem 4 shows that in this regime the prediction error of best subsets is at least of order σ2γk2klog⁡(ep/k)\sigma^2\gamma_k^2k\log(ep/k)σ2γk2​klog(ep/k). Together the two results explain the empirical finding that shrinkage helps best subsets at low SNR. Part (B) shows nothing is lost at high SNR: the fast rate is the minimax rate σ2klog⁡(ep/k)\sigma^2k\log(ep/k)σ2klog(ep/k) over kkk-sparse vectors plus the shrinkage term γ2k−2λ2\gamma_{2k}^{-2}\lambda^2γ2k−2​λ2. Combined with Theorem 1, the bounds give the high-probability statements of the paper's Corollaries 1 and 3.

Formalizing it. The theorem is proved in the paper; no machine-checked version is known to exist. The mission produces a checked deterministic core: once the events are granted, every constant (8+228+2\sqrt28+22​, the factor 2λ2\lambda2λ, the threshold 2klog⁡(ep/k)2k\log(ep/k)2klog(ep/k)) is verified exactly. Together with the companion mission on Theorem 1 it would yield fully checked high-probability oracle inequalities for a cardinality-constrained estimator, which no current formal library contains.

Difficulty

Every step is elementary, but the bookkeeping is exact. The obvious reading "insert the event's inequality into (29)" does not by itself work on F\mathcal FF: the event's right-hand side is a maximum of a sorted weighted sum and a prediction-norm term, and the first branch has to be compared with 2klog⁡(ep/k) ∥u∥\sqrt{2k\log(ep/k)}\,\|u\|2klog(ep/k)​∥u∥ for the 2k2k2k-sparse difference u=β^2−βu=\hat\beta_2-\betau=β^​2​−β, including when 2k2k2k exceeds ppp. That comparison is a combinatorial estimate on sorted sums, not a substitution. On G2k\mathcal G_{2k}G2k​, the penalty difference λ(∥β∥−∥β^2∥)\lambda(\|\beta\|-\|\hat\beta_2\|)λ(∥β∥−∥β^​2​∥) is not controlled by the event at all and has to be converted into prediction errors through γ2k\gamma_{2k}γ2k​. In both parts the final constant must not depend on kkk, ppp or δ0\delta_0δ0​.

Formalization scope

All objects are explicit Lean definitions in the namespace SubsetShrink.L2:

  • vectors are functions on Fin n and Fin p with 0-based indices, and XXX is a real Matrix (Fin n) (Fin p);
  • the Euclidean norm, ∥⋅∥0\|\cdot\|_0∥⋅∥0​ and the inner product are explicit finite sums, since Mathlib's norm on Fin m → ℝ is the sup norm;
  • ∑juj♯log⁡(2p/j)\sum_j u^\sharp_j\sqrt{\log(2p/j)}∑j​uj♯​log(2p/j)​ is encoded as a maximum over permutations, which equals the sorted sum by the rearrangement inequality;
  • γs\gamma_sγs​ is a real infimum over the nonzero vectors of B0(s)B_0(s)B0​(s), a nonempty set when p,s≥1p,s\ge1p,s≥1.

The model consists of XXX with unit columns, f∗f^*f∗, ϵ\epsilonϵ, σ>0\sigma>0σ>0 and y=f∗+ϵy=f^*+\epsilony=f∗+ϵ. The estimator is a predicate that every minimizer satisfies; no minimizer is chosen and existence is not assumed. "On the event" is a hypothesis that the noise vector satisfies the event's printed inequality, so no probability measure appears. G2k\mathcal G_{2k}G2k​ is the event as printed on p. 13; the correction of its probability bound belongs to the mission on Theorem 1 and does not affect these deterministic statements.

Committed conventions:

  • each ≲\lesssim≲ is an existential constant C>0C>0C>0 quantified before all other data;
  • an infimum over B0(k)B_0(k)B0​(k) is stated as a bound for every β∈B0(k)\beta\in B_0(k)β∈B0​(k);
  • δ0∈(0,1]\delta_0\in(0,1]δ0​∈(0,1] is universally quantified;
  • the model size satisfies 1≤k≤p1\le k\le p1≤k≤p.

Part (B) assumes γ2k>0\gamma_{2k}>0γ2k​>0 or λ=0\lambda=0λ=0. When γ2k=0<λ\gamma_{2k}=0<\lambdaγ2k​=0<λ the paper's bound is vacuous (γ2k−2λ2=+∞\gamma_{2k}^{-2}\lambda^2=+\inftyγ2k−2​λ2=+∞), but Lean's 0−1=00^{-1}=00−1=0 would turn it into a false claim.

A formalization that lets the constant depend on kkk, ppp or δ0\delta_0δ0​, fixes a particular minimizer, weakens an event (for example drops "for all u∈B0(s)u\in B_0(s)u∈B0​(s)"), or computes the infimum as a real ⨅ over an unbounded set would trivialize the theorem and is ruled out.

Not formalized here: the probability bounds of Theorem 1, the ℓ1\ell_1ℓ1​-penalized Theorem 3, the lower bound of Theorem 4, the data-driven choice of kkk (Theorem 5), and the corollaries and numerical experiments. These are separate missions or follow-ups. Contributions of reusable lemmas (the rearrangement bound for the sorted sum, properties of γs\gamma_sγs​, the elementary absorption inequality (34)) are welcome.

Selected references

  • R. Mazumder, P. Radchenko, A. Dedieu, Subset Selection with Shrinkage: Sparse Linear Modeling When the SNR Is Low, arXiv:1708.03288v4, 2022; Operations Research, 2023. https://arxiv.org/abs/1708.03288
  • D. Bertsimas, A. King, R. Mazumder, Best subset selection via a modern optimization lens, Annals of Statistics 44(2), 2016. https://doi.org/10.1214/15-AOS1388
  • T. Hastie, R. Tibshirani, R. Tibshirani, Best subset, forward stepwise or Lasso? Analysis and recommendations based on extensive comparisons, Statistical Science 35(4), 2020. https://doi.org/10.1214/19-STS733
  • P. C. Bellec, G. Lecué, A. B. Tsybakov, Slope meets Lasso: improved oracle bounds and optimality, Annals of Statistics 46(6B), 2018. https://doi.org/10.1214/17-AOS1670
8 thms1 active userReviewed
Algorithmic Game TheoryMechanism DesignProbability·Captain: mikedeng1

Contextual Standard Auctions with Budgets: Revenue Equivalence and Efficiency Guarantees 3: Pacing-Based Equilibria Have Liquid Welfare Price of Anarchy at Least 1/2Research Paper

Motivation

Advertising platforms repeatedly sell opportunities to show an advertisement. The opportunity's features, such as audience and placement, affect each advertiser differently, while advertisers limit their average spending. This makes the efficiency of an auction a question about both heterogeneous values and budgets. The paper by Balseiro, Kroer, and Kumar studies this setting with a continuum of buyer types and context-dependent values. It constructs equilibria by value pacing, in which a buyer scales values before bidding. This mission asks how much attainable value those equilibria preserve when budgets cap useful value.

Earlier efficiency bounds for second-price auctions concern a different, non-Bayesian multi-item model. In particular, the half guarantee discussed by Azar and coauthors, as discussed in §5 of the paper uses a no-overbudgeting condition that the present pacing equilibria need not satisfy. Balseiro, Kroer, and Kumar prove their bound under their own contextual and in-expectation budget model; the distinction matters when interpreting the constant. Their existence and revenue-equivalence results also make it possible to speak about the same pacing-based allocation across standard auction payment rules. These claims and the limitation of the earlier assumption are explained in §5 of the paper.

Setting

An item has a context α∈A⊂Rd\alpha\in A\subset\mathbb R^dα∈A⊂Rd, with d≥2d\ge2d≥2. There are n≥2n\ge2n≥2 buyers. Each buyer has a private type θ=(w,B)\theta=(w,B)θ=(w,B): the vector w∈(0,U)dw\in(0,U)^dw∈(0,U)d assigns a positive weight to every context feature, and B∈(Bmin⁡,U)B\in(B_{\min},U)B∈(Bmin​,U) is the buyer's budget, where 0<Bmin⁡<U0<B_{\min}<U0<Bmin​<U. The buyer's value for the item is w⊤αw^\top\alphaw⊤α. A context is drawn from a distribution FFF and buyers' types are drawn independently from a distribution GGG; both distributions have densities. The context and type ranges are the spaces AAA and Θ=(0,U)d×(Bmin⁡,U)\Theta=(0,U)^d\times(B_{\min},U)Θ=(0,U)d×(Bmin​,U) of §2. The reserve price is zero throughout the efficiency section.

A pacing function μ:Θ→R≥0\mu:\Theta\to\mathbb R_{\ge0}μ:Θ→R≥0​ scales type θ\thetaθ's value to w⊤α/(1+μ(θ))w^\top\alpha/(1+\mu(\theta))w⊤α/(1+μ(θ)). Let Hαμ(x)H^\mu_\alpha(x)Hαμ​(x) be the probability that the maximum paced value among n−1n-1n−1 independent competitors is at most xxx. The first-price bid βμ(θ,α)\beta^\mu(\theta,\alpha)βμ(θ,α) applies the paper's bid function σαμ\sigma^\mu_\alphaσαμ​ to the paced value. A buyer may choose a context-dependent bid, subject to an in-expectation budget: expected payment over contexts and competitors must not exceed BBB. The buyer's dual function qμ(θ,t)q^\mu(\theta,t)qμ(θ,t) is the supremum of its Lagrangian over such bids at multiplier t≥0t\ge0t≥0.

The relevant pacing equilibria are fixed points of the correspondence C0∗C_0^*C0∗​. Its domain X0X_0X0​ consists of measurable, integrable pacing functions in [0,ω/Bmin⁡][0,\omega/B_{\min}][0,ω/Bmin​] with total variation at most V0=(d+1)Ud+1ω/Bmin⁡V_0=(d+1)U^{d+1}\omega/B_{\min}V0​=(d+1)Ud+1ω/Bmin​, where ω\omegaω is the supremum of possible values. For a given μ∈X0\mu\in X_0μ∈X0​, C0∗(μ)C_0^*(\mu)C0∗​(μ) minimizes the average dual objective Eθ∼G[qμ(θ,μ^(θ))]\mathbb E_{\theta\sim G}[q^\mu(\theta,\widehat\mu(\theta))]Eθ∼G​[qμ(θ,μ​(θ))] over μ^∈X0\widehat\mu\in X_0μ​∈X0​. These are the objects of §3.3, including its multivariate definition of total variation.

An allocation xxx assigns nonnegative shares xi(α,θ1,…,θn)x_i(\alpha,\theta_1,\ldots,\theta_n)xi​(α,θ1​,…,θn​) summing to one. For buyer iii, keep θi=(wi,Bi)\theta_i=(w_i,B_i)θi​=(wi​,Bi​) fixed while averaging wi⊤α xiw_i^\top\alpha\,x_iwi⊤​αxi​ over the context and the other types. The buyer's liquid welfare is the smaller of this interim value and BiB_iBi​; total liquid welfare LW(x)\mathrm{LW}(x)LW(x) sums its expectation over each buyer's own type. For an equilibrium pacing function μ∈C0∗(μ)\mu\in C_0^*(\mu)μ∈C0∗​(μ), xμx^\muxμ allocates to a highest paced bidder. Definitions 5 and 6 use this allocation to define the worst equilibrium liquid welfare relative to the optimal allocation.

Formalization targets

The main target is the Theorem 3 guarantee, stated for every measurable allocation xxx and every equilibrium pacing function μ\muμ:

LW(xμ)≥12 LW(x).\mathrm{LW}(x^\mu)\ge\tfrac12\,\mathrm{LW}(x).LW(xμ)≥21​LW(x).

The universal comparison expresses the paper's price-of-anarchy bound PoA≥1/2\mathrm{PoA}\ge1/2PoA≥1/2. It leaves no optimum or worst equilibrium as an arbitrary variable. The two milestones are the dual derivative identity and the first-order budget and complementary-slackness conditions stated in §3.2:

∂tqμ(θ,t)=B−payment(bt),payment(bt∗)≤B,t∗(B−payment(bt∗))=0.\partial_t q^\mu(\theta,t)=B-\mathrm{payment}(b_t),\qquad \mathrm{payment}(b_{t^*})\le B,\quad t^*(B-\mathrm{payment}(b_{t^*}))=0.∂t​qμ(θ,t)=B−payment(bt​),payment(bt∗​)≤B,t∗(B−payment(bt∗​))=0.

Here bt(α)=σαμ(w⊤α/(1+t))b_t(\alpha)=\sigma^\mu_\alpha(w^\top\alpha/(1+t))bt​(α)=σαμ​(w⊤α/(1+t)) and t∗t^*t∗ minimizes qμ(θ,⋅)q^\mu(\theta,\cdot)qμ(θ,⋅) over nonnegative multipliers. The milestones are posed in the zero-reserve setting used by the main target.

Significance

The theorem gives an explicit welfare guarantee despite private budgets, correlated values across buyers through a shared context, and equilibrium bidding that depends on those budgets. It applies to the allocation induced by pacing in every standard auction covered by the paper, since those auctions have the same equilibrium allocation even when payment rules differ Balseiro, Kroer, and Kumar, §4–5. Without an efficiency bound, existence of a pacing equilibrium alone would give no quantitative assurance about the allocation it produces.

The result has a mathematical proof in the paper's online appendix; it is an open formalization target here. A complete development would connect measurable Bayesian auctions, the buyer's nonconvex dual problem, the bounded-variation fixed-point domain, and interim liquid welfare. The definitions of interim welfare, dual expenditure, and multivariate variation can support other budgeted auction results. The source PDF held for this mission contains the published main article but not its online appendix, where the proofs and auxiliary lemmas appear.

Difficulty

The expected budget applies to each buyer type after averaging over contexts and opponents, while liquid welfare caps that same type's interim value before averaging over types. Moving the budget cap across the expectation changes the quantity. The natural first-price tie convention also awards the good to every highest bidder for the buyer optimization problem, while a welfare allocation must give total share one. The proof must reconcile the dual account of expenditure with a feasible allocation and preserve these different orders of expectation. The dual objective is a supremum over bids, so its derivative cannot be obtained merely by differentiating a chosen closed formula without justifying that formula and the maximizing bid.

Formalization scope

Lean represents a context by Fin d → ℝ and a buyer type by its weight vector and real budget. Value is the explicit finite sum ∑iwiαi\sum_i w_i\alpha_i∑i​wi​αi​. The model requires a nonempty bounded measurable context set in the positive orthant, absolutely continuous probability laws FFF and GGG supported on their stated domains, measurable reserves and pacing, n≥2n\ge2n≥2, and 0<Bmin⁡<U0<B_{\min}<U0<Bmin​<U. Boundedness of AAA makes ω\omegaω finite; these conditions make the domain and integrals implicit in the article well posed. The welfare target assumes zero reserves as §5 does.

Admissible bids are measurable and nonnegative on AAA, with integrable expected payment. The dual is defined by its supremum over bids. Total variation is extended-real-valued and uses Euclidean lengths of test fields; thus infinite variation cannot satisfy the X0X_0X0​ bound through a real supremum's default value. Allocations are measurable and simplex-valued on the model domain. At a tied highest bid, xμx^\muxμ splits the item equally; ties have probability zero under these equilibria, as the paper states by its appendix Lemma 6(d). The closed simplex permits zero shares, correcting the paper's notation R+\mathbb R_+R+​ in Definition 5, which §2.1 otherwise calls strictly positive. The universal welfare inequality avoids default real infimum and supremum values on pathological sets. Contributions to the measurable bidding, variation, duality, and welfare infrastructure are all useful within this scope.

Selected references

  • Santiago R. Balseiro, Christian Kroer, and Rachitesh Kumar, Contextual Standard Auctions with Budgets: Revenue Equivalence and Efficiency Guarantees, Management Science, Articles in Advance, 2023. DOI: 10.1287/mnsc.2023.4719.
6 thms1 active userReviewed
Algorithmic Game TheoryFunctional AnalysisMechanism Design+1·Captain: mikedeng1

Contextual Standard Auctions with Budgets: Revenue Equivalence and Efficiency Guarantees 1: A Value Pacing-Based Symmetric First Price Equilibrium ExistsResearch Paper

Motivation

Most online advertising is sold by auction, one impression at a time, to advertisers who manage a budget over many auctions rather than a value for a single one. Since 2019 the major ad exchanges have moved from second price to first price auctions, in which the winner pays its own bid. A budget-constrained advertiser in a first price auction must therefore shade its bids twice: once because of the budget, and once because of the payment rule. Whether these two effects admit a coherent equilibrium, and of what form, is the question behind Balseiro, Kroer and Kumar, Contextual Standard Auctions with Budgets: Revenue Equivalence and Efficiency Guarantees (Management Science, 2023, doi:10.1287/mnsc.2023.4719).

Earlier work on pacing equilibria treated second price auctions, where truthful bidding is dominant without budgets (Balseiro, Besbes and Weintraub 2015; Conitzer et al. 2022). In first price auctions with a single deterministic item, an equilibrium may fail to exist (the paper's Online Appendix A). The paper restores existence by letting items carry a random context and values depend on it linearly. This mission formalizes its first main result, Theorem 1: an equilibrium of a simple, interpretable form exists.

Setting

An item has a context α∈A⊂R+d\alpha\in A\subset\mathbb R^d_{+}α∈A⊂R+d​, a feature vector drawn from a law FFF with a density. There are nnn buyers. Each buyer has a type θ=(w,B)\theta=(w,B)θ=(w,B), a weight vector www and a budget BBB, drawn independently from a law GGG with a density on

Θ=(0,U)d×(Bmin⁡,U).\Theta=(0,U)^d\times(B_{\min},U).Θ=(0,U)d×(Bmin​,U).

Buyer (w,B)(w,B)(w,B) values item α\alphaα at wTαw^T\alphawTα, and ω:=sup⁡(w,B)∈Θ,α∈AwTα\omega:=\sup_{(w,B)\in\Theta,\alpha\in A}w^T\alphaω:=sup(w,B)∈Θ,α∈A​wTα. The seller reveals α\alphaα, then runs a first price auction with a public reserve price r(α)r(\alpha)r(α): a bid b(α)b(\alpha)b(α) wins if b(α)≥max⁡(r(α),{β(θi,α)}i)b(\alpha)\ge\max(r(\alpha),\{\beta(\theta_i,\alpha)\}_i)b(α)≥max(r(α),{β(θi​,α)}i​), where β\betaβ is the strategy of the n−1n-1n−1 competitors, and the winner pays its bid. A budget only needs to hold in expectation over α\alphaα and the competitors' types.

A strategy β∗:Θ×A→R≥0\beta^*:\Theta\times A\to\mathbb R_{\ge0}β∗:Θ×A→R≥0​ is a symmetric first price equilibrium (SFPE, Definition 1) if for GGG-almost every type (w,B)(w,B)(w,B) the bid function β∗(w,B,⋅)\beta^*(w,B,\cdot)β∗(w,B,⋅) solves

max⁡b:A→R≥0 E[(wTα−b(α))1{b(α)≥max⁡(r(α),{β∗(θi,α)}i)}]  s.t.  E[b(α)1{⋯ }]≤B.\max_{b:A\to\mathbb R_{\ge0}}\ \mathbb E\big[(w^T\alpha-b(\alpha))\mathbb 1\{b(\alpha)\ge\max(r(\alpha),\{\beta^*(\theta_i,\alpha)\}_i)\}\big]\ \ \text{s.t.}\ \ \mathbb E\big[b(\alpha)\mathbb 1\{\cdots\}\big]\le B .b:A→R≥0​max​ E[(wTα−b(α))1{b(α)≥max(r(α),{β∗(θi​,α)}i​)}]  s.t.  E[b(α)1{⋯}]≤B.

A pacing function μ:Θ→R≥0\mu:\Theta\to\mathbb R_{\ge0}μ:Θ→R≥0​ scales values down to paced values wTα/(1+μ(w,B))w^T\alpha/(1+\mu(w,B))wTα/(1+μ(w,B)). Let HαμH^\mu_\alphaHαμ​ be the distribution function of the highest of n−1n-1n−1 i.i.d. paced values, and for x≥r(α)x\ge r(\alpha)x≥r(α) let

σαμ(x)=x−∫r(α)xHαμ(s)Hαμ(x) ds,\sigma^\mu_\alpha(x)=x-\int_{r(\alpha)}^x\frac{H^\mu_\alpha(s)}{H^\mu_\alpha(x)}\,ds,σαμ​(x)=x−∫r(α)x​Hαμ​(x)Hαμ​(s)​ds,

with σαμ(x)=0\sigma^\mu_\alpha(x)=0σαμ​(x)=0 if Hαμ(x)=0H^\mu_\alpha(x)=0Hαμ​(x)=0 and σαμ(x)=x\sigma^\mu_\alpha(x)=xσαμ​(x)=x if x<r(α)x<r(\alpha)x<r(α). The value pacing–based strategy (Definition 2) is

βμ(w,B,α)=σαμ(wTα1+μ(w,B)).\beta^\mu(w,B,\alpha)=\sigma^\mu_\alpha\Big(\frac{w^T\alpha}{1+\mu(w,B)}\Big).βμ(w,B,α)=σαμ​(1+μ(w,B)wTα​).

For a buyer facing competitors who use βμ\beta^\muβμ, the dual function is qμ(w,B,t)=sup⁡bE[(wTα−(1+t)b(α))1{⋯ }]+tBq^\mu(w,B,t)=\sup_b\mathbb E[(w^T\alpha-(1+t)b(\alpha))\mathbb 1\{\cdots\}]+tBqμ(w,B,t)=supb​E[(wTα−(1+t)b(α))1{⋯}]+tB for t≥0t\ge0t≥0.

Formalization targets

Goal: Theorem 1 (p. 7)

∃ μ:Θ→R≥0such thatβμ is an SFPE.\exists\,\mu:\Theta\to\mathbb R_{\ge0}\quad\text{such that}\quad \beta^\mu\ \text{is an SFPE}.∃μ:Θ→R≥0​such thatβμ is an SFPE.

Milestones

  • Lemma 1 (p. 8): for every type and every t≥0t\ge0t≥0, σαμ(wTα/(1+t))\sigma^\mu_\alpha(w^T\alpha/(1+t))σαμ​(wTα/(1+t)) maximizes the Lagrangian E[(wTα/(1+t)−b(α))1{⋯ }]\mathbb E[(w^T\alpha/(1+t)-b(\alpha))\mathbb 1\{\cdots\}]E[(wTα/(1+t)−b(α))1{⋯}].
  • Proposition 1 (p. 8): on a set Θ′\Theta'Θ′ of full GGG-measure, for every μ\muμ, a minimizer t∗t^*t∗ of qμ(w,B,⋅)q^\mu(w,B,\cdot)qμ(w,B,⋅) over t≥0t\ge0t≥0 makes σαμ(wTα/(1+t∗))\sigma^\mu_\alpha(w^T\alpha/(1+t^*))σαμ​(wTα/(1+t∗)) optimal for the budget-constrained problem Qμ(w,B)Q^\mu(w,B)Qμ(w,B).
  • Lemma 2 (p. 10): the smallest minimizer ℓμ(w,B)\ell^\mu(w,B)ℓμ(w,B) of qμ(w,B,⋅)q^\mu(w,B,\cdot)qμ(w,B,⋅) over [0,ω/Bmin⁡][0,\omega/B_{\min}][0,ω/Bmin​] is nondecreasing in each wiw_iwi​ and nonincreasing in BBB.
  • Lemma 3 (p. 10): ℓμ∈L1(Θ)\ell^\mu\in L_1(\Theta)ℓμ∈L1​(Θ) and its total variation is at most V0=(d+1)max⁡(Ud,Ud+1) ω/Bmin⁡V_0=(d+1)\max(U^d,U^{d+1})\,\omega/B_{\min}V0​=(d+1)max(Ud,Ud+1)ω/Bmin​. The paper prints (d+1)Ud+1ω/Bmin⁡(d+1)U^{d+1}\omega/B_{\min}(d+1)Ud+1ω/Bmin​, which agrees when U≥1U\ge1U≥1. For U<1U<1U<1 the printed bound fails by a rescaling argument, and the mission corrects the exponent to ddd there.
  • Lemma 4 (p. 10): X0={μ∈L1(Θ):0≤μ≤ω/Bmin⁡, V(μ,Θ)≤V0}\mathcal X_0=\{\mu\in L_1(\Theta):0\le\mu\le\omega/B_{\min},\ V(\mu,\Theta)\le V_0\}X0​={μ∈L1​(Θ):0≤μ≤ω/Bmin​, V(μ,Θ)≤V0​} is nonempty, compact and convex in L1(Θ)L_1(\Theta)L1​(Θ); f(μ,μ^)=EG[qμ(w,B,μ^(w,B))]f(\mu,\hat\mu)=\mathbb E_G[q^\mu(w,B,\hat\mu(w,B))]f(μ,μ^​)=EG​[qμ(w,B,μ^​(w,B))] is continuous; C0∗(μ)=arg⁡min⁡μ^∈X0f(μ,μ^)C^*_0(\mu)=\arg\min_{\hat\mu\in\mathcal X_0}f(\mu,\hat\mu)C0∗​(μ)=argminμ^​∈X0​​f(μ,μ^​) is upper hemicontinuous with nonempty, convex, compact values.
  • Lemma 5 (p. 10): a fixed point μ∈C0∗(μ)\mu\in C^*_0(\mu)μ∈C0∗​(μ) satisfies μ(w,B)∈arg⁡min⁡t∈[0,ω/Bmin⁡]qμ(w,B,t)\mu(w,B)\in\arg\min_{t\in[0,\omega/B_{\min}]}q^\mu(w,B,t)μ(w,B)∈argmint∈[0,ω/Bmin​]​qμ(w,B,t) almost surely.
  • Proposition 2 (p. 9): some μ\muμ satisfies μ(w,B)∈arg⁡min⁡t≥0qμ(w,B,t)\mu(w,B)\in\arg\min_{t\ge0}q^\mu(w,B,t)μ(w,B)∈argmint≥0​qμ(w,B,t) almost surely.

The paper states that Proposition 2 combined with Proposition 1 implies Theorem 1.

Significance

Theorem 1 says that budget-constrained buyers in contextual first price auctions have an equilibrium that is described by one scalar per type: shade the value by the factor 1/(1+μ(w,B))1/(1+\mu(w,B))1/(1+μ(w,B)), then bid as an unconstrained buyer would in an i.i.d. first price auction against paced competitors. This is the object on which the rest of the paper is built: revenue equivalence across standard auctions (Theorem 2) and a liquid-welfare price of anarchy of at least 1/21/21/2 (Theorem 3) are both statements about pacing-based equilibria. Practically, it supports the use of pacing for budget management in first price auctions, with pacing multipliers computed as for second price auctions.

The result is proved in the paper; every proof is in its online appendix, which is not part of the source this mission cites. Nothing in the paper is formalized. A formalization would check a strong-duality argument for a non-convex problem and an infinite-dimensional fixed-point argument whose topology the authors chose with care, and it would produce reusable pieces: multivariate total variation, compactness of bounded-variation sets in L1L_1L1​, and the first price equilibrium bid against an arbitrary (possibly atomic) value distribution with a reserve price.

Difficulty

The buyer's problem Qμ(w,B)Q^\mu(w,B)Qμ(w,B) is not convex in the bid function, so standard convex duality does not give strong duality; Proposition 1 needs a direct argument combining Lagrangian optimality (Lemma 1), differentiability of qμq^\muqμ in ttt, and complementary slackness. Lemma 1 itself cannot quote the textbook first price equilibrium, because the paced value distribution λαμ\lambda^\mu_\alphaλαμ​ may have atoms and the reserve price creates discontinuities.

The fixed point lives in an infinite-dimensional space of pacing functions. The obvious topologies fail: the set X\mathcal XX of bounded pacing functions is not compact in any LpL_pLp​, and in the weak-* topology of L∞L_\inftyL∞​ upper hemicontinuity of the best-response correspondence is hard to obtain; restricting to Lipschitz functions loses the property that best responses stay in the set. The paper restricts to functions of bounded total variation, which requires the monotonicity of Lemma 2 and the variation bound of Lemma 3.

Formalization scope

Contexts are Fin d → ℝ, types are pairs (w, B) : (Fin d → ℝ) × ℝ, and wTαw^T\alphawTα is the explicit sum ∑iwiαi\sum_i w_i\alpha_i∑i​wi​αi​. A single Model structure holds d,n,U,Bmin⁡,A,F,G,rd,n,U,B_{\min},A,F,G,rd,n,U,Bmin​,A,F,G,r, and Model.Standing collects the assumptions of §2.2 together with four that the paper uses without stating them: n≥2n\ge2n≥2, Bmin⁡<UB_{\min}<UBmin​<U, AAA nonempty and bounded (so ω<∞\omega<\inftyω<∞), and measurability of AAA and rrr. Pacing functions are measurable functions on Rd×R\mathbb R^d\times\mathbb RRd×R that are nonnegative on Θ\ThetaΘ; values off Θ\ThetaΘ are irrelevant since G(Θc)=0G(\Theta^c)=0G(Θc)=0.

The conventions fixed in Lean:

  • Ties win: the indicator is b(α)≥max⁡(⋯ )b(\alpha)\ge\max(\cdots)b(α)≥max(⋯), as in the paper's tie-breaking rule.
  • An admissible bid function is measurable, nonnegative on AAA, and has an integrable payment integrand. The problem of Definition 1, QμQ^\muQμ, qμq^\muqμ and the Lagrangian of Lemma 1 all range over this class.
  • An SFPE is jointly measurable in (θ,α)(\theta,\alpha)(θ,α), so Theorem 1 must produce a measurable strategy.
  • qμq^\muqμ is the supremum over bids of p. 7, not the closed form of p. 8. σαμ\sigma^\mu_\alphaσαμ​ has an explicit branch for Hαμ(x)=0H^\mu_\alpha(x)=0Hαμ​(x)=0.
  • Argmins are stated as membership plus IsMinOn, over t≥0t\ge0t≥0 in Propositions 1–2 and over [0,ω/Bmin⁡][0,\omega/B_{\min}][0,ω/Bmin​] in Lemma 5 and ℓμ\ell^\muℓμ, as printed.
  • V(u,Ω)V(u,\Omega)V(u,Ω) takes values in EReal, with Euclidean ∥ϕ(x)∥≤1\|\phi(x)\|\le1∥ϕ(x)∥≤1 and test fields supported in Ω\OmegaΩ; L1(Θ)L_1(\Theta)L1​(Θ) is with respect to Lebesgue measure. Lemma 4 is stated on the image of X0\mathcal X_0X0​ in Lp ℝ 1 (volume.restrict Θ), in its norm topology.

A formalization in which qμq^\muqμ is defined by its closed form, in which non-integrable payments count as zero, or in which the strategy need not be measurable would make the milestones easier without proving the paper's theorem; these readings are excluded by the definitions above.

A complete development needs: first price equilibrium bids for general atomic distributions with reserves; measurability of parametric integrals; a duality argument for one-dimensional Lagrangian relaxations; multivariate bounded variation and its compact embedding in L1L_1L1​ of a box; and the Kakutani–Fan–Glicksberg theorem in a locally convex space. The bounded-variation and fixed-point parts are reusable beyond this mission. Proofs of any milestone, and alternative arguments, are welcome.

Selected references

  • S. R. Balseiro, C. Kroer, R. Kumar, Contextual Standard Auctions with Budgets: Revenue Equivalence and Efficiency Guarantees, Management Science, Articles in Advance, 2023. https://doi.org/10.1287/mnsc.2023.4719
  • S. R. Balseiro, O. Besbes, G. Y. Weintraub, Repeated Auctions with Budgets in Ad Exchanges: Approximations and Design, Management Science 61(4), 2015. https://doi.org/10.1287/mnsc.2014.2022
  • V. Conitzer, C. Kroer, E. Sodomka, N. E. Stier-Moses, Multiplicative Pacing Equilibria in Auction Markets, Operations Research 70(2), 2022. https://doi.org/10.1287/opre.2021.2167
  • V. Krishna, Auction Theory, 2nd ed., Academic Press, 2009.
  • L. C. Evans, R. F. Gariepy, Measure Theory and Fine Properties of Functions, revised ed., CRC Press, 2015.
12 thms1 active userReviewed
Algorithmic Game TheoryMechanism DesignProbability·Captain: mikedeng1

Contextual Standard Auctions with Budgets: Revenue Equivalence and Efficiency Guarantees 2: Value Pacing Is an Equilibrium of Every Standard Auction with the Same Expected PaymentsResearch Paper

Motivation

Online advertising sells billions of impressions a day through auctions, and most advertisers do not bid by hand: they hand a budget to the platform, which paces their bids so that spending over a campaign matches the budget. Over the last decade the major ad exchanges moved from second price to first price auctions. An advertiser who has tuned a pacing policy for one format wants to know whether it still works after the switch, and a platform wants to know whether its revenue changes.

Classical revenue equivalence (Myerson 1981; Riley and Samuelson 1981) says that, with independent private values, risk-neutral buyers and no budgets, every standard auction yields the same expected revenue. Che and Gale (1998) showed that this fails when budgets are hard: first price auctions then raise more revenue than second price auctions. Balseiro, Kroer and Kumar (2023) study budgets that bind in expectation over many auctions, with values driven by item features, and show that equivalence is restored. This mission formalizes that result, their Theorem 2.

Setting

An item has a context α∈A⊂R+d\alpha\in A\subset\mathbb R^d_+α∈A⊂R+d​ drawn from a law FFF with a density. Each of nnn buyers has a type θ=(w,B)\theta=(w,B)θ=(w,B) drawn independently from a law GGG with a density on Θ=(0,U)d×(Bmin⁡,U)\Theta=(0,U)^d\times(B_{\min},U)Θ=(0,U)d×(Bmin​,U). Here www are feature weights and BBB is a budget. The buyer's value for the item is wTαw^T\alphawTα, and ω\omegaω is the supremum of all values. The seller discloses α\alphaα, then runs an auction with reserve price r(α)r(\alpha)r(α). A buyer must keep its expected payment, over contexts and competitors' types, at most BBB.

A pacing function μ:Θ→R≥0\mu:\Theta\to\mathbb R_{\ge0}μ:Θ→R≥0​ shades every value to the paced value wTα/(1+μ(w,B))w^T\alpha/(1+\mu(w,B))wTα/(1+μ(w,B)). Let λαμ\lambda^\mu_\alphaλαμ​ be the law of the paced value when (w,B)∼G(w,B)\sim G(w,B)∼G, and Hαμ(x)=λαμ((−∞,x])n−1H^\mu_\alpha(x)=\lambda^\mu_\alpha((-\infty,x])^{n-1}Hαμ​(x)=λαμ​((−∞,x])n−1 the c.d.f. of the highest of n−1n-1n−1 independent paced values. In first price auctions the paper's candidate strategy is βμ(w,B,α)=σαμ(wTα/(1+μ(w,B)))\beta^\mu(w,B,\alpha)=\sigma^\mu_\alpha(w^T\alpha/(1+\mu(w,B)))βμ(w,B,α)=σαμ​(wTα/(1+μ(w,B))), where σαμ\sigma^\mu_\alphaσαμ​ is the first-price equilibrium bid against paced competitors.

The buyer's dual function qμ(w,B,t)q^\mu(w,B,t)qμ(w,B,t) is the supremum over bid functions b(⋅)b(\cdot)b(⋅) of the Lagrangian E[(wTα−(1+t)b(α))1{win}]+tB\mathbb E[(w^T\alpha-(1+t)b(\alpha))\mathbb 1\{\text{win}\}]+tBE[(wTα−(1+t)b(α))1{win}]+tB, with competitors bidding βμ\beta^\muβμ. The set X0\mathcal X_0X0​ collects pacing functions with values in [0,ω/Bmin⁡][0,\omega/B_{\min}][0,ω/Bmin​] and total variation at most V0=(d+1)Ud+1ω/Bmin⁡V_0=(d+1)U^{d+1}\omega/B_{\min}V0​=(d+1)Ud+1ω/Bmin​. The correspondence C0∗(μ)C^*_0(\mu)C0∗​(μ) is the set of minimizers over X0\mathcal X_0X0​ of μ^↦EG[qμ(w,B,μ^(w,B))]\hat\mu\mapsto\mathbb E_G[q^\mu(w,B,\hat\mu(w,B))]μ^​↦EG​[qμ(w,B,μ^​(w,B))]. A fixed point μ∈C0∗(μ)\mu\in C^*_0(\mu)μ∈C0∗​(μ) is an equilibrium pacing function for first price auctions.

An anonymous standard auction Aα=(r(α),Mα)\mathcal A_\alpha=(r(\alpha),M_\alpha)Aα​=(r(α),Mα​) gives the item to the highest bid at or above the reserve. Its payment Mα(b,{bi})M_\alpha(b,\{b_i\})Mα​(b,{bi​}) for a bid bbb against competing bids bib_ibi​ does not depend on the buyers' identities. An oracle returns, for each atomless value law H\mathcal HH on [0,ω][0,\omega][0,ω], a single-auction equilibrium bid ψαH\psi^{\mathcal H}_\alphaψαH​ that is nondecreasing, meets the reserve exactly for values above it, gives a zero-value bidder zero payoff, and has an atomless bid law. The candidate strategy is

Ψμ(w,B,α)=ψαλαμ(wTα1+μ(w,B)).\Psi^\mu(w,B,\alpha)=\psi^{\lambda^\mu_\alpha}_\alpha\Big(\frac{w^T\alpha}{1+\mu(w,B)}\Big).Ψμ(w,B,α)=ψαλαμ​​(1+μ(w,B)wTα​).

Formalization targets

Goal: Theorem 2 (Revenue and Pacing Equivalence)

For every family {Aα}\{\mathcal A_\alpha\}{Aα​} of anonymous standard auctions with such an oracle and every μ∈X0\mu\in\mathcal X_0μ∈X0​ with μ∈C0∗(μ)\mu\in C^*_0(\mu)μ∈C0∗​(μ):

  1. Ψμ\Psi^\muΨμ is a symmetric equilibrium of the family (Definition 4): almost every buyer type maximizes expected utility subject to its expected budget by bidding Ψμ\Psi^\muΨμ when everyone else does;
  2. for every θ∈Θ\theta\in\Thetaθ∈Θ,
E[Mα(Ψμ(θ,α),{Ψμ(θi,α)}i)]=E[βμ(θ,α) 1{βμ(θ,α)≥max⁡(r(α),{βμ(θi,α)}i)}].\mathbb E\big[M_\alpha(\Psi^\mu(\theta,\alpha),\{\Psi^\mu(\theta_i,\alpha)\}_i)\big]=\mathbb E\big[\beta^\mu(\theta,\alpha)\,\mathbb 1\{\beta^\mu(\theta,\alpha)\ge\max(r(\alpha),\{\beta^\mu(\theta_i,\alpha)\}_i)\}\big].E[Mα​(Ψμ(θ,α),{Ψμ(θi​,α)}i​)]=E[βμ(θ,α)1{βμ(θ,α)≥max(r(α),{βμ(θi​,α)}i​)}].

Milestones

  • §3.2, p. 8, last display. For t≥0t\ge0t≥0 and xα=wTα/(1+t)x_\alpha=w^T\alpha/(1+t)xα​=wTα/(1+t),
qμ(w,B,t)=(1+t) Eα[1{xα≥r(α)}∫r(α)xαHαμ(s) ds]+tB.q^\mu(w,B,t)=(1+t)\,\mathbb E_\alpha\Big[\mathbb 1\{x_\alpha\ge r(\alpha)\}\int_{r(\alpha)}^{x_\alpha}H^\mu_\alpha(s)\,ds\Big]+tB.qμ(w,B,t)=(1+t)Eα​[1{xα​≥r(α)}∫r(α)xα​​Hαμ​(s)ds]+tB.
  • §4, p. 12, key step. When the competitors bid Ψμ\Psi^\muΨμ, the dual function of the buyer's problem in any standard auction family equals qμ(w,B,t)q^\mu(w,B,t)qμ(w,B,t).

Significance

Theorem 2 says that pacing is portable: an equilibrium pacing function computed for first price auctions is an equilibrium pacing function for second price auctions, all-pay auctions, or any context-dependent mixture of standard formats. Each buyer's expected spend is the same in all of them, so the seller's expected revenue is too. This explains why budget-management systems built for second price auctions keep working after a move to first price. It also separates in-expectation budgets, where equivalence holds, from the hard budgets of Che and Gale, where it fails. Theorem 3 of the same paper, a liquid-welfare price of anarchy of 1/21/21/2, uses Theorem 2 to argue in the second price auction.

The paper's proofs are in its online appendix, which is not part of the source this mission cites. To our knowledge no part of the argument has been machine-checked. Classical revenue equivalence for incentive-compatible direct mechanisms without budgets exists on the platform as other items, but none has pacing, budgets or contexts. The mission asks for a machine-checked proof of Theorem 2 and of the two steps the authors name.

Difficulty

The classical argument pins payments by the allocation rule and the envelope formula, value by value. Here a buyer's decision is a whole bid function of the context, coupled across contexts by one budget constraint, and the budget-constrained problem is not convex. So the envelope argument cannot be applied to the constrained problem directly. The obvious move, Lagrangian relaxation, needs strong duality for a non-convex problem, and it needs the dual of each standard auction to agree with the first-price dual for every multiplier, not only at the optimum. The paced value laws λαμ\lambda^\mu_\alphaλαμ​ need not have densities, so the textbook single-auction results, which assume a positive density, do not apply verbatim. Finally, the fixed-point hypothesis ties all buyer types together, and optimality must be recovered almost surely in the type.

Formalization scope

Everything is defined in the namespace ContextualAuctions.RevEquiv. Contexts are Fin d → ℝ, types are (Fin d → ℝ) × ℝ, and wTαw^T\alphawTα is an explicit sum. The joint law of the context and the n−1n-1n−1 competitor types is F.prod (Measure.pi fun _ => G). Ties win, as in the paper's tie-breaking rule. The standing assumptions are d≥2d\ge2d≥2, FFF and GGG probability measures with densities concentrated on AAA and Θ\ThetaΘ, AAA in the open positive orthant, and rrr measurable. Disclosed additions, implicit in the paper:

  • n≥2n\ge2n≥2 and 0<Bmin⁡<U0<B_{\min}<U0<Bmin​<U;
  • AAA measurable, nonempty and bounded, so that ω<∞\omega<\inftyω<∞;
  • pacing functions and bids are measurable; first-price bids are nonnegative on AAA, and standard-auction bids are nonnegative for almost every context, because the oracle constrains Ψμ\Psi^\muΨμ only where λαμ\lambda^\mu_\alphaλαμ​ is atomless;
  • the payment rule is measurable along measurable bid maps on any measurable space;
  • the strategy Ψμ\Psi^\muΨμ is jointly measurable, since the oracle is a black box.

The dual function is the supremum over admissible bids, which are measurable with integrable payment. Its closed form is a milestone and never its definition. Total variation (Definition 3) is extended-real valued and uses the Euclidean norm of the test field. In oracle property 1, expected payments are taken in [0,∞][0,\infty][0,∞], so a bid with infinite expected payment can never look free. The payment identity of Theorem 2 is stated in [0,∞][0,\infty][0,∞], so it also asserts that the left side is finite. This rules out the trivializing readings: a closed-form definition of qμq^\muqμ that makes the first milestone hold by definition, Bochner junk zeros for non-integrable payments, and a vacuous variation bound.

A complete development needs single-auction revenue equivalence for i.i.d. atomless value laws without densities. It also needs the measure-theoretic interchange of a supremum over bid functions with the expectation over contexts, and strong duality for the budget-constrained problem. These pieces, notably the single-auction equivalence for general atomless laws, are reusable beyond this mission. Contributions toward any of them are welcome.

Selected references

  • S. Balseiro, C. Kroer, R. Kumar, Contextual Standard Auctions with Budgets: Revenue Equivalence and Efficiency Guarantees, Management Science, Articles in Advance, 2023. https://doi.org/10.1287/mnsc.2023.4719
  • Y.-K. Che, I. Gale, Standard Auctions with Financially Constrained Bidders, Review of Economic Studies 65(1), 1998. https://doi.org/10.1111/1467-937X.00041
  • R. Myerson, Optimal Auction Design, Mathematics of Operations Research 6(1), 1981. https://doi.org/10.1287/moor.6.1.58
  • J. Riley, W. Samuelson, Optimal Auctions, American Economic Review 71(3), 1981. https://www.jstor.org/stable/1802779
  • V. Krishna, Auction Theory, 2nd ed., Academic Press, 2009. https://doi.org/10.1016/C2009-0-22474-3
  • S. Balseiro, Y. Gur, Learning in Repeated Auctions with Budgets: Regret Minimization and Equilibrium, Management Science 65(9), 2019. https://doi.org/10.1287/mnsc.2018.3174
6 thms1 active userReviewed
Algorithmic Game TheoryMechanism Design·Captain: mikedeng1

Contextual Standard Auctions with Budgets: Revenue Equivalence and Efficiency Guarantees 4: Colinear Paced Buyers with Equal Budgets Are Paced Down to the Same Weight VectorResearch Paper

Budget pacing in contextual auctions

Online advertising platforms sell billions of ad impressions a day, mostly by auction, to advertisers who care about the features of the impression (the user, the page, the time) and who spend under a budget fixed in advance. To keep spending within budget, platforms and advertisers use pacing: every value is multiplied by a factor smaller than one, so that the buyer bids less aggressively and spreads its budget over the whole campaign. Balseiro, Kroer and Kumar (2023) study the equilibria that pacing produces when items carry feature vectors and buyers' values are linear in them. They prove that a pacing-based equilibrium exists in the first price auction (Theorem 1), that the same pacing is an equilibrium of every standard auction with the same expected payments (Theorem 2), and that its liquid welfare is at least half the optimum (Theorem 3).

This mission concerns §6, Structural Properties, of that paper: what the equilibrium pacing multipliers look like as a function of the buyer's type. Its goal, Proposition 3, says that buyers who share a budget and whose weight vectors point in the same direction are paced down to one common weight vector. It is the qualitative statement behind a practical observation: scaling up the targeting weights of a budget-constrained advertiser without raising its budget does not change the outcome.

Setting

An item has a context α∈A⊂R+d\alpha \in A \subset \mathbb{R}^d_+α∈A⊂R+d​, drawn from a law FFF with a density. There are nnn buyers; each has a type (w,B)(w, B)(w,B) consisting of a weight vector w∈Rdw \in \mathbb{R}^dw∈Rd and a budget BBB, drawn independently from a law GGG with a density on

Θ=(0,U)d×(Bmin⁡,U).\Theta = (0, U)^d \times (B_{\min}, U).Θ=(0,U)d×(Bmin​,U).

The value of type (w,B)(w,B)(w,B) for item α\alphaα is w⊤αw^{\top}\alphaw⊤α, and ω=sup⁡(w,B)∈Θ, α∈Aw⊤α\omega = \sup_{(w,B)\in\Theta,\ \alpha\in A} w^{\top}\alphaω=sup(w,B)∈Θ, α∈A​w⊤α. The item is sold by first price auction with reserve price r(α)r(\alpha)r(α), and every buyer must respect its budget in expectation over the context and the other buyers' types.

A pacing function μ:Θ→R≥0\mu : \Theta \to \mathbb{R}_{\ge 0}μ:Θ→R≥0​ shades the value of type (w,B)(w,B)(w,B) to the paced value w⊤α/(1+μ(w,B))w^{\top}\alpha/(1+\mu(w,B))w⊤α/(1+μ(w,B)), equivalently replaces www by the paced weight vector w/(1+μ(w,B))w/(1+\mu(w,B))w/(1+μ(w,B)). Let HαμH^\mu_\alphaHαμ​ be the distribution function of the highest of n−1n-1n−1 independent paced values. The value pacing–based strategy bids βμ(w,B,α)=σαμ(w⊤α/(1+μ(w,B)))\beta^\mu(w,B,\alpha) = \sigma^\mu_\alpha(w^{\top}\alpha/(1+\mu(w,B)))βμ(w,B,α)=σαμ​(w⊤α/(1+μ(w,B))), where

σαμ(x)=x−∫r(α)xHαμ(s)Hαμ(x) ds(x≥r(α)),\sigma^\mu_\alpha(x) = x - \int_{r(\alpha)}^{x} \frac{H^\mu_\alpha(s)}{H^\mu_\alpha(x)}\,ds \qquad (x \ge r(\alpha)),σαμ​(x)=x−∫r(α)x​Hαμ​(x)Hαμ​(s)​ds(x≥r(α)),

with σαμ(x)=0\sigma^\mu_\alpha(x) = 0σαμ​(x)=0 when Hαμ(x)=0H^\mu_\alpha(x) = 0Hαμ​(x)=0 and σαμ(x)=x\sigma^\mu_\alpha(x) = xσαμ​(x)=x below the reserve. A buyer of type (w,B)(w,B)(w,B) facing competitors who bid βμ\beta^\muβμ has the dual function

qμ(w,B,t)=sup⁡b E[(w⊤α−(1+t) b(α)) 1{b(α)≥max⁡(r(α),{βμ(θi,α)}i)}]+tB,q^\mu(w,B,t) = \sup_{b}\ \mathbb{E}\Big[\big(w^{\top}\alpha - (1+t)\,b(\alpha)\big)\,\mathbf 1\{b(\alpha) \ge \max(r(\alpha),\{\beta^\mu(\theta_i,\alpha)\}_i)\}\Big] + tB,qμ(w,B,t)=bsup​ E[(w⊤α−(1+t)b(α))1{b(α)≥max(r(α),{βμ(θi​,α)}i​)}]+tB,

the supremum over bid functions b≥0b \ge 0b≥0; ttt is the Lagrange multiplier of its budget constraint.

§6 assumptions. Reserve prices are zero; the support δ(G)\delta(G)δ(G) of GGG is convex and compact; and μ\muμ takes values in [0,ω/Bmin⁡][0, \omega/B_{\min}][0,ω/Bmin​] with

μ(w,B)=arg min⁡t∈[0,ω/Bmin⁡]qμ(w,B,t)for all (w,B)∈δ(G),\mu(w,B) = \operatorname*{arg\,min}_{t \in [0,\omega/B_{\min}]} q^\mu(w,B,t) \quad \text{for all } (w,B) \in \delta(G),μ(w,B)=t∈[0,ω/Bmin​]argmin​qμ(w,B,t)for all (w,B)∈δ(G),

the minimizer being unique. This is uniqueness of the best response, not of the equilibrium. A type with μ(w,B)=0\mu(w,B) = 0μ(w,B)=0 is unpaced, and paced otherwise. Norms are Euclidean.

Formalization targets

Goal: Proposition 3 (p. 14)

Let w^∈R+d\hat w \in \mathbb{R}^d_+w^∈R+d​ be a unit vector and B>0B > 0B>0 such that w/∥w∥=w^w/\|w\| = \hat ww/∥w∥=w^ for some (w,B)∈δ(G)(w,B) \in \delta(G)(w,B)∈δ(G).

  1. If (w1,B),(w2,B)∈δ(G)(w_1,B),(w_2,B) \in \delta(G)(w1​,B),(w2​,B)∈δ(G), w1/∥w1∥=w2/∥w2∥=w^w_1/\|w_1\| = w_2/\|w_2\| = \hat ww1​/∥w1​∥=w2​/∥w2​∥=w^ and μ(w1,B),μ(w2,B)>0\mu(w_1,B),\mu(w_2,B) > 0μ(w1​,B),μ(w2​,B)>0, then
w11+μ(w1,B)=w21+μ(w2,B).\frac{w_1}{1+\mu(w_1,B)} = \frac{w_2}{1+\mu(w_2,B)}.1+μ(w1​,B)w1​​=1+μ(w2​,B)w2​​.
  1. If some (w,B)∈δ(G)(w,B)\in\delta(G)(w,B)∈δ(G) on the ray is unpaced, let w0w_0w0​ be the largest unpaced weight vector along w^\hat ww^ with budget BBB. Then w/(1+μ(w,B))=w0w/(1+\mu(w,B)) = w_0w/(1+μ(w,B))=w0​ for every paced (w,B)∈δ(G)(w,B) \in \delta(G)(w,B)∈δ(G) on the ray.

Milestones (p. 14)

  • Monotonicity. Under a unique best response, μ(w,B)\mu(w,B)μ(w,B) is nondecreasing in each component of www and nonincreasing in BBB (a consequence of the paper's Lemma 2).
  • Continuity. Under a unique best response, μ\muμ is continuous (Lemma 14 of the online appendix).

Significance

Proposition 3 describes equilibrium pacing along each direction of weight space as a single threshold. Along a ray {c w^}\{c\,\hat w\}{cw^} with budget BBB, buyers below a critical scale do not exhaust their budget and are unpaced; every buyer above it is paced down to the critical type, which spends its budget exactly. Equilibrium paced values, hence allocations and payments, are therefore constant above the threshold. By the paper's revenue equivalence result the same description holds for the pacing equilibria of all standard auctions, which makes it a statement about budget-constrained advertisers in general, not about one auction format.

The result is proved in the paper, with all proofs in its online appendix; the appendix is not part of the source this mission cites. No part of it has been machine-checked. The mission formalizes the statement on top of a from-scratch model of contextual budget-constrained first price auctions (types, pacing, the bidding function σαμ\sigma^\mu_\alphaσαμ​, the buyer's dual function), which is shared with the other missions of this series.

Difficulty

The dual function qμq^\muqμ is a supremum over bid functions in which the competitors' bids themselves depend on μ\muμ, so the equilibrium condition is a fixed point: μ\muμ appears both inside the objective and as its minimizer. Comparing two buyers on the same ray requires relating their dual problems, and a buyer with weight vector c wc\,wcw does not simply face a rescaled problem, because the budget BBB does not scale with ccc. Part 2 requires in addition that the largest unpaced buyer exists, which needs continuity of μ\muμ on the support; continuity of an argmin requires uniqueness of the minimizer and continuity of qμq^\muqμ in the type and the multiplier jointly. The monotonicity statement is a comparative-statics argument for a family of minimization problems parameterized by the type.

Formalization scope

Types are pairs (w, B) : (Fin d → ℝ) × ℝ, values are the explicit sums ∑iwiαi\sum_i w_i\alpha_i∑i​wi​αi​, and the Euclidean norm is written as (∑iwi2)1/2\big(\sum_i w_i^2\big)^{1/2}(∑i​wi2​)1/2, never Lean's sup norm on Fin d → ℝ. δ(G)\delta(G)δ(G) is Mathlib's Measure.support. The expectation is over the product of FFF and n−1n-1n−1 copies of GGG; ties win, as in the paper's tie-breaking rule. Bids are measurable, nonnegative and have an integrable payment, and qμq^\muqμ is defined as the supremum over such bids (p. 7), not by its closed form. σαμ\sigma^\mu_\alphaσαμ​ is defined by an explicit case split so that the case Hαμ(x)=0H^\mu_\alpha(x) = 0Hαμ​(x)=0 gives 000.

Standing assumptions: d≥2d \ge 2d≥2; U,Bmin⁡>0U, B_{\min} > 0U,Bmin​>0; AAA measurable inside the strictly positive orthant; FFF and GGG probability measures with densities, F(A)=1F(A) = 1F(A)=1, G(Θ)=1G(\Theta) = 1G(Θ)=1; rrr measurable; and the §6 assumptions above, with the unique best response stated as membership, minimality and uniqueness over [0,ω/Bmin⁡][0,\omega/B_{\min}][0,ω/Bmin​]. Disclosed additions: n≥2n \ge 2n≥2; Bmin⁡<UB_{\min} < UBmin​<U; AAA nonempty and bounded (so that ω\omegaω is finite); δ(G)⊆Θ\delta(G) \subseteq \Thetaδ(G)⊆Θ, since μ\muμ is defined on Θ\ThetaΘ and is evaluated on δ(G)\delta(G)δ(G). The maximization set of part 2 is read inside δ(G)\delta(G)δ(G) with the budget fixed, and the existence of the maximizer w0w_0w0​ is part of the conclusion, so the statement is not vacuously true when no maximizer exists. The milestones are posed on δ(G)\delta(G)δ(G), where uniqueness is assumed.

The development needs the support of a measure, argmin continuity (a maximum theorem for a unique minimizer), and comparative statics of parametric minimization; the auction model is reusable by the other missions of the series. Contributions of supporting lemmas on qμq^\muqμ (its convexity in (w,t)(w,t)(w,t), continuity, the closed form of p. 8) are welcome.

Selected references

  • S. R. Balseiro, C. Kroer, R. Kumar, Contextual Standard Auctions with Budgets: Revenue Equivalence and Efficiency Guarantees, Management Science, Articles in Advance, 2023. https://doi.org/10.1287/mnsc.2023.4719
5 thms1 active userReviewed
Graph TheoryLinear OptimizationProbability+1·Captain: mikedeng1

Linear Programming and Community Detection 6: The LP Relaxation Fails w.h.p. in the Very Dense and Dense SBM Above Explicit ThresholdsResearch Paper

Motivation

Community detection asks for a hidden partition of the nodes of a graph from its edges. Its simplest probabilistic benchmark is the two-community stochastic block model (SBM), in which the minimum bisection problem is the maximum a posteriori estimator. That problem is NP-hard, so convex relaxations are used in its place, and the question is when a relaxation still returns the hidden partition exactly. For semidefinite relaxations this question has sharp answers (Abbe, Bandeira, Hall 2016; Bandeira 2018). Del Pia, Khajavirad and Kunisky (arXiv:2006.03213v2) ask the same question for the most common linear programming relaxation of minimum bisection, the metric relaxation. They show that it recovers the planted bisection in very dense graphs when p−q>1/2p-q>1/2p−q>1/2 (their Theorem 6). This mission is their converse in the very dense and dense regimes, Theorem 7: above explicit thresholds the LP fails with high probability.

Setting

The node set is V={1,…,n}V=\{1,\dots,n\}V={1,…,n}, nnn even. A bisection is a partition V=V1⊔V2V=V_1\sqcup V_2V=V1​⊔V2​ with ∣V1∣=∣V2∣=n/2|V_1|=|V_2|=n/2∣V1​∣=∣V2​∣=n/2. The cut vector xˉ∈{0,1}(n2)\bar x\in\{0,1\}^{\binom n2}xˉ∈{0,1}(2n​) of V1,V2V_1,V_2V1​,V2​ has xˉij=1\bar x_{ij}=1xˉij​=1 exactly when iii and jjj lie on different sides. For a graph GGG with adjacency indicators aija_{ij}aij​, the metric relaxation is

(LP-P)min⁡x∑i<jaijxijs.t.xij≤xik+xjk,xij+xik+xjk≤2,∑i<jxij=n24,\text{(LP-P)}\qquad \min_x \sum_{i<j}a_{ij}x_{ij}\quad\text{s.t.}\quad x_{ij}\le x_{ik}+x_{jk},\quad x_{ij}+x_{ik}+x_{jk}\le2,\quad \sum_{i<j}x_{ij}=\frac{n^2}{4},(LP-P)xmin​i<j∑​aij​xij​s.t.xij​≤xik​+xjk​,xij​+xik​+xjk​≤2,i<j∑​xij​=4n2​,

over x∈R(n2)x\in\mathbb R^{\binom n2}x∈R(2n​), with the first family over distinct i,j,ki,j,ki,j,k with i<ji<ji<j and the second over i<j<ki<j<ki<j<k. The LP recovers the planted bisection if xˉ\bar xxˉ is its unique optimal solution.

In the SBM Gn,p,q\mathcal G_{n,p,q}Gn,p,q​ a bisection V1,V2V_1,V_2V1​,V2​ is first drawn uniformly at random; then each pair on the same side is an edge with probability ppp and each pair across with probability qqq, independently, with p>qp>qp>q. The Erdős–Rényi graph Gn,p\mathcal G_{n,p}Gn,p​ has every pair an edge with probability ppp. The dense scaling (20) is p=αn−ωp=\alpha n^{-\omega}p=αn−ω, q=βn−ωq=\beta n^{-\omega}q=βn−ω with α>β>0\alpha>\beta>0α>β>0, 0≤ω<10\le\omega<10≤ω<1; ω=0\omega=0ω=0 is the very dense regime. An event holds with high probability (w.h.p.) if its probability tends to 111 as n→∞n\to\inftyn→∞.

For a graph GGG, ρG(i,j)\rho_G(i,j)ρG​(i,j) is the shortest-path distance (+∞+\infty+∞ across components), ρmax⁡(G)=max⁡i,jρG(i,j)\rho_{\max}(G)=\max_{i,j}\rho_G(i,j)ρmax​(G)=maxi,j​ρG​(i,j) is the diameter, and ρavg(G)=2n2∑i<jρG(i,j)\rho_{\mathrm{avg}}(G)=\frac2{n^2}\sum_{i<j}\rho_G(i,j)ρavg​(G)=n22​∑i<j​ρG​(i,j) is the average distance. EEE is the edge set of GGG, E0E_0E0​ the set of edges across the planted bisection.

Formalization targets

Goal: Theorem 7 (pp. 15–16)

Let p=αn−ωp=\alpha n^{-\omega}p=αn−ω, q=βn−ωq=\beta n^{-\omega}q=βn−ω with α>β>0\alpha>\beta>0α>β>0, ω∈[0,1)\omega\in[0,1)ω∈[0,1), and α,β<1\alpha,\beta<1α,β<1 if ω=0\omega=0ω=0. If one of

  1. ω=0\omega=0ω=0 and β>max⁡{3−α−(3−α)2−4α2, 2α−1}\beta>\max\big\{\tfrac{3-\alpha-\sqrt{(3-\alpha)^2-4\alpha}}2,\ 2\alpha-1\big\}β>max{23−α−(3−α)2−4α​​, 2α−1} (22),
  2. ω∈(0,1)\omega\in(0,1)ω∈(0,1), 11−ω∉N\tfrac1{1-\omega}\notin\mathbb N1−ω1​∈/N and β>α/(2⌈11−ω⌉−1)\beta>\alpha/(2\lceil\tfrac1{1-\omega}\rceil-1)β>α/(2⌈1−ω1​⌉−1) (23),
  3. ω=12\omega=\tfrac12ω=21​ and β>max⁡{1−2e−α22−e−α2,13+2e−α2}α\beta>\max\big\{\tfrac{1-2e^{-\alpha^2}}{2-e^{-\alpha^2}},\tfrac1{3+2e^{-\alpha^2}}\big\}\alphaβ>max{2−e−α21−2e−α2​,3+2e−α21​}α (24),
  4. ω∈(0,1)\omega\in(0,1)ω∈(0,1), 11−ω∈N∖{2}\tfrac1{1-\omega}\in\mathbb N\setminus\{2\}1−ω1​∈N∖{2} and β>α/(21−ω−1+2e−α1/(1−ω))\beta>\alpha/\big(\tfrac2{1-\omega}-1+2e^{-\alpha^{1/(1-\omega)}}\big)β>α/(1−ω2​−1+2e−α1/(1−ω)) (25)

holds, then

lim⁡n→∞P(V1,G)∼Gn,p,q[(LP-P) does not recover V1,V2]=1.\lim_{n\to\infty}\mathbb P_{(V_1,G)\sim\mathcal G_{n,p,q}}\big[\text{(LP-P) does not recover }V_1,V_2\big]=1.n→∞lim​P(V1​,G)∼Gn,p,q​​[(LP-P) does not recover V1​,V2​]=1.

Milestones

  1. (26), p. 16: for p,q=Ω(log⁡n/n)p,q=\Omega(\log n/n)p,q=Ω(logn/n) with lim⁡q/p∈(0,1]\lim q/p\in(0,1]limq/p∈(0,1], the ratio ∣E0∣/∣E∣|E_0|/|E|∣E0​∣/∣E∣ lies in (1±ϵ) q/(p+q)(1\pm\epsilon)\,q/(p+q)(1±ϵ)q/(p+q) w.h.p.
  2. Proposition 11, p. 28: a coupling of Gn,q\mathcal G_{n,q}Gn,q​, Gn,q,p\mathcal G_{n,q,p}Gn,q,p​ and Gn,p\mathcal G_{n,p}Gn,p​ in which the three graphs are nested.
  3. Proposition 12, p. 28: w.h.p. ρmax⁡(Gn,αn−ω)=1+⌊11−ω⌋\rho_{\max}(\mathcal G_{n,\alpha n^{-\omega}})=1+\lfloor\frac1{1-\omega}\rfloorρmax​(Gn,αn−ω​)=1+⌊1−ω1​⌋.
  4. Proposition 14, p. 29 (Shimizu): w.h.p. ρavg(Gn,αn−ω)∈((1−ϵ)μ,(1+ϵ)μ)\rho_{\mathrm{avg}}(\mathcal G_{n,\alpha n^{-\omega}})\in((1-\epsilon)\mu,(1+\epsilon)\mu)ρavg​(Gn,αn−ω​)∈((1−ϵ)μ,(1+ϵ)μ), where μ=⌈11−ω⌉+1{11−ω∈N}e−α1/(1−ω)\mu=\lceil\frac1{1-\omega}\rceil+\mathbf 1\{\frac1{1-\omega}\in\mathbb N\}e^{-\alpha^{1/(1-\omega)}}μ=⌈1−ω1​⌉+1{1−ω1​∈N}e−α1/(1−ω).
  5. Propositions 16, 17, 18, pp. 29–30: the diameter of the very dense and dense SBM, and lower bounds on its average distance, in the three cases ω=0\omega=0ω=0, 11−ω∉N\frac1{1-\omega}\notin\mathbb N1−ω1​∈/N and 11−ω∈N\frac1{1-\omega}\in\mathbb N1−ω1​∈N.

Significance

Theorem 6 and Theorem 7 together bracket the success region of the metric relaxation in the very dense regime: recovery for p−q>1/2p-q>1/2p−q>1/2, failure above the curve (22). In the dense regime, Theorem 7 gives explicit linear thresholds β>c(ω,α) α\beta>c(\omega,\alpha)\,\alphaβ>c(ω,α)α above which the LP fails. The proof reduces failure to a deterministic distance criterion (Theorem 5, posed in mission 4 of this series) and turns asymptotic distance distributions of random graphs into statements about an LP.

The results are proved in the paper, partly by citing Bollobás's diameter theorems and Shimizu's average-distance theorem; none of them has a machine-checked proof. Formalizing the mission yields, besides Theorem 7, the diameter and average distance of dense Erdős–Rényi graphs (Propositions 12 and 14) and the monotone coupling between Erdős–Rényi graphs and the SBM (Proposition 11), each of which is usable beyond this paper. The coupling and (26) are also the inputs of the logarithmic-degree result (Theorem 8, mission 7).

Difficulty

The deterministic half is short: once ∣E0∣/∣E∣|E_0|/|E|∣E0​∣/∣E∣, ρmax⁡\rho_{\max}ρmax​ and ρavg\rho_{\mathrm{avg}}ρavg​ are pinned down, the case computations (28)–(29) reduce to inequalities between explicit expressions in α\alphaα, β\betaβ and ω\omegaω. The difficulty is the random-graph input. The diameter of Gn,p\mathcal G_{n,p}Gn,p​ for p=αn−ωp=\alpha n^{-\omega}p=αn−ω needs the second-moment analysis of the number of pairs at distance greater than ddd (Bollobás, Corollaries 10.11–10.12), and the average distance needs the finer limit law for path counts at the critical exponent 11−ω∈N\frac1{1-\omega}\in\mathbb N1−ω1​∈N, where the constant e−α1/(1−ω)e^{-\alpha^{1/(1-\omega)}}e−α1/(1−ω) appears. A first-moment argument gives only one side of each estimate. Concentration of ∣E∣|E|∣E∣ and ∣E0∣|E_0|∣E0​∣ (26) is a Hoeffding bound, but it must hold uniformly over the random planted bisection.

Formalization scope

Nodes are Fin n and graphs SimpleGraph (Fin n). A bisection is its side V1V_1V1​ with 2∣V1∣=n2|V_1|=n2∣V1​∣=n. The LP variable is a symmetric, zero-diagonal array Fin n → Fin n → ℝ, with no box constraints, so uniqueness is uniqueness over R(n2)\mathbb R^{\binom n2}R(2n​). Probabilities are finite sums over all graphs (and all bisections for the SBM). Edge probabilities are clamped into [0,1][0,1][0,1], which affects only finitely many nnn. SBM limits run over even n=2mn=2mn=2m. Distances are in ℕ∞ and ρavg\rho_{\mathrm{avg}}ρavg​ in [0,∞][0,\infty][0,∞], so disconnected samples have infinite diameter and average distance; no real truncation is used.

Disclosed choices:

  • Theorem 7 adds α>β\alpha>\betaα>β, the SBM's standing assumption p>qp>qp>q (§4, p. 13), also assumed by Proposition 18 and (26).
  • (26) is posed for p=αs(n)p=\alpha s(n)p=αs(n), q=βs(n)q=\beta s(n)q=βs(n) with s(n)≥clog⁡n/ns(n)\ge c\log n/ns(n)≥clogn/n and αs(n)≤1\alpha s(n)\le1αs(n)≤1 eventually. Then q/p=β/αq/p=\beta/\alphaq/p=β/α, which avoids the printed "lim⁡p/q∈(0,1]\lim p/q\in(0,1]limp/q∈(0,1]", a slip for q/pq/pq/p.
  • Proposition 11 is posed as printed, with middle law Gn,q,p\mathcal G_{n,q,p}Gn,q,p​ (within qqq, across ppp).

Two encodings would make the statements trivial, and both are excluded. Unnormalized LP variables would make uniqueness false. Reading ρ\rhoρ in R\mathbb RR via toReal would make the upper bounds hold for disconnected graphs.

The deterministic distance criterion (Theorem 5) is posed in mission 4 and is not restated. Contributions are welcome on the coupling, Hoeffding-type concentration of edge counts, and the Erdős–Rényi distance asymptotics. These are reusable for any finite random-graph development.

Selected references

  • A. Del Pia, A. Khajavirad, D. Kunisky, Linear Programming and Community Detection, Mathematics of Operations Research, 2023; arXiv:2006.03213v2. https://arxiv.org/abs/2006.03213
  • E. Abbe, A. S. Bandeira, G. Hall, Exact recovery in the stochastic block model, IEEE Trans. Inf. Theory 62 (2016) 471–487. https://arxiv.org/abs/1405.3267
  • A. S. Bandeira, Random Laplacian matrices and convex relaxations, Found. Comput. Math. 18 (2018) 345–379. https://arxiv.org/abs/1504.03987
  • B. Bollobás, Random Graphs, 2nd ed., Cambridge Studies in Advanced Mathematics 73, Cambridge University Press, 2001. https://doi.org/10.1017/CBO9780511814068
  • N. Shimizu, The average distance of dense homogeneous random graphs, Mathematical Engineering Technical Report, Department of Mathematical Informatics, The University of Tokyo, November 2017 (Theorem 1; reference [38] of the paper). https://www.keisu.t.u-tokyo.ac.jp/research/techrep/
14 thms1 active userReviewed
Graph TheoryLinear OptimizationProbability+1·Captain: mikedeng1

Linear Programming and Community Detection 7: The LP Relaxation Fails w.h.p. in the Logarithmic-Degree SBMResearch Paper

Motivation

The stochastic block model (SBM) is the standard benchmark for community detection: a random graph whose vertices are split into hidden communities, with denser connections inside a community than across. For two equal communities, the best possible estimator of the hidden partition is the minimum bisection problem, which is NP-hard in the worst case. Its tractable relaxations are therefore compared by asking for which edge densities they still recover the hidden partition exactly.

For semidefinite programming the answer is complete in the logarithmic-degree regime: Abbe, Bandeira and Hall (arXiv:1405.3267) located the information-theoretic threshold α−β=2\sqrt\alpha-\sqrt\beta=\sqrt2α​−β​=2​, and the SDP relaxation was shown to reach it by Hajek, Wu and Xu (arXiv:1412.6156) and by Bandeira (arXiv:1504.03987). Linear programming relaxations are cheaper and widely used in combinatorial optimization, but their behaviour on the SBM was not known. Del Pia, Khajavirad and Kunisky (arXiv:2006.03213, Mathematics of Operations Research, 2023) study the most common one, the metric relaxation. This mission formalizes their result for the logarithmic-degree regime (Theorem 8): there the LP relaxation fails to recover the hidden partition with high probability, at every choice of the parameters.

Setting

Let V={1,…,n}V=\{1,\dots,n\}V={1,…,n} with nnn even. A bisection is a partition V=V1⊔V2V=V_1\sqcup V_2V=V1​⊔V2​ with ∣V1∣=∣V2∣=n/2|V_1|=|V_2|=n/2∣V1​∣=∣V2​∣=n/2; its cost in a graph GGG is the number of edges between V1V_1V1​ and V2V_2V2​. Given a fixed planted bisection V1,V2V_1,V_2V1​,V2​, an optimization problem recovers it if its unique optimal solution corresponds to V1,V2V_1,V_2V1​,V2​.

The cut vector xˉ∈{0,1}(n2)\bar x\in\{0,1\}^{\binom n2}xˉ∈{0,1}(2n​) of V1,V2V_1,V_2V1​,V2​ has xˉij=1\bar x_{ij}=1xˉij​=1 exactly when i,ji,ji,j lie on different sides. With aij=1a_{ij}=1aij​=1 for edges and 000 otherwise, the metric relaxation is

(LP-P)min⁡x∑i<jaijxij  s.t.  xij≤xik+xjk,  xij+xik+xjk≤2,  ∑i<jxij=n24,\text{(LP-P)}\quad \min_x \sum_{i<j}a_{ij}x_{ij}\ \ \text{s.t.}\ \ x_{ij}\le x_{ik}+x_{jk},\ \ x_{ij}+x_{ik}+x_{jk}\le2,\ \ \sum_{i<j}x_{ij}=\frac{n^2}{4},(LP-P)xmin​i<j∑​aij​xij​  s.t.  xij​≤xik​+xjk​,  xij​+xik​+xjk​≤2,  i<j∑​xij​=4n2​,

the first family over distinct i,j,ki,j,ki,j,k and the second over i<j<ki<j<ki<j<k, with xji:=xijx_{ji}:=x_{ij}xji​:=xij​. Every bisection's cut vector is feasible.

The distance ρG(i,j)\rho_G(i,j)ρG​(i,j) is the length of a shortest iii–jjj path (+∞+\infty+∞ across components). The diameter is ρmax⁡(G)=max⁡i,jρG(i,j)\rho_{\max}(G)=\max_{i,j}\rho_G(i,j)ρmax​(G)=maxi,j​ρG​(i,j) and the average distance is ρavg(G)=2n2∑i<jρG(i,j)\rho_{\mathrm{avg}}(G)=\frac{2}{n^2}\sum_{i<j}\rho_G(i,j)ρavg​(G)=n22​∑i<j​ρG​(i,j).

The Erdős–Rényi model Gn,p\mathcal G_{n,p}Gn,p​ draws each edge independently with probability ppp. The SBM Gn,p,q\mathcal G_{n,p,q}Gn,p,q​ chooses a bisection V1,V2V_1,V_2V1​,V2​ uniformly at random, then draws each pair inside a side with probability ppp and each pair across with probability qqq, independently, where p>qp>qp>q. The logarithmic-degree regime (21) is

p=αlog⁡nn,q=βlog⁡nn,α>β>0.p=\alpha\frac{\log n}{n},\qquad q=\beta\frac{\log n}{n},\qquad \alpha>\beta>0 .p=αnlogn​,q=βnlogn​,α>β>0.

An event AnA_nAn​ holds with high probability if P[An]→1\mathbb P[A_n]\to1P[An​]→1 as n→∞n\to\inftyn→∞.

Formalization targets

Goal: Theorem 8 (p. 18)

α>β>0 ⟹ lim⁡n→∞PGn,p,q[(LP-P) does not recover V1,V2]=1.\alpha>\beta>0\ \Longrightarrow\ \lim_{n\to\infty}\mathbb P_{\mathcal G_{n,p,q}}\big[\text{(LP-P) does not recover }V_1,V_2\big]=1 .α>β>0 ⟹ n→∞lim​PGn,p,q​​[(LP-P) does not recover V1​,V2​]=1.

Milestones

  1. The threshold of [2] (p. 14, strict cases): if α−β>2\sqrt\alpha-\sqrt\beta>\sqrt2α​−β​>2​, the minimum bisection problem recovers V1,V2V_1,V_2V1​,V2​ with high probability; if α−β<2\sqrt\alpha-\sqrt\beta<\sqrt2α​−β​<2​, it does not.
  2. Proposition 13 (p. 29): for α>1\alpha>1α>1 and G∼Gn,αlog⁡n/nG\sim\mathcal G_{n,\alpha\log n/n}G∼Gn,αlogn/n​, ρmax⁡(G)∈((1−ϵ)log⁡nlog⁡log⁡n,(1+ϵ)log⁡nlog⁡log⁡n)\rho_{\max}(G)\in\big((1-\epsilon)\frac{\log n}{\log\log n},(1+\epsilon)\frac{\log n}{\log\log n}\big)ρmax​(G)∈((1−ϵ)loglognlogn​,(1+ϵ)loglognlogn​) with high probability.
  3. Proposition 15 (p. 29, Theorem 1 of Chung–Lu): for α>1\alpha>1α>1, ρavg(G)∈((1−ϵ)log⁡nlog⁡pn,(1+ϵ)log⁡nlog⁡pn)\rho_{\mathrm{avg}}(G)\in\big((1-\epsilon)\frac{\log n}{\log pn},(1+\epsilon)\frac{\log n}{\log pn}\big)ρavg​(G)∈((1−ϵ)logpnlogn​,(1+ϵ)logpnlogn​) with high probability.
  4. Proposition 19 (p. 30): for α>β>0\alpha>\beta>0α>β>0 with α+β2>1\frac{\alpha+\beta}{2}>12α+β​>1 and G∼Gn,p,qG\sim\mathcal G_{n,p,q}G∼Gn,p,q​,
(1−ϵ)log⁡nlog⁡log⁡n≤ρavg(G)≤ρmax⁡(G)≤(1+ϵ)log⁡nlog⁡log⁡n(1-\epsilon)\frac{\log n}{\log\log n}\le\rho_{\mathrm{avg}}(G)\le\rho_{\max}(G)\le(1+\epsilon)\frac{\log n}{\log\log n}(1−ϵ)loglognlogn​≤ρavg​(G)≤ρmax​(G)≤(1+ϵ)loglognlogn​

with high probability.

Two results the proof of Theorem 8 uses are posed in sibling missions of this series and are not restated here: Theorem 5 (a distance-based non-recovery criterion for (LP-P), mission 4) and the coupling Proposition 11 together with the crossing-edge count (26) (mission 6).

Significance

The result. Theorem 8 separates linear from semidefinite programming on a natural statistical problem. In the logarithmic regime the SDP relaxation recovers the planted bisection whenever any method can, while the metric LP fails with high probability for all α>β>0\alpha>\beta>0α>β>0, including parameters far above the information-theoretic threshold. The paper's Theorem 6 shows that in the very dense regime (constant p,qp,qp,q) the LP does succeed when p−q>1/2p-q>1/2p−q>1/2; Theorem 8 shows that at logarithmic degree no such window exists.

Formalizing it. The theorem is proved in the paper, modulo results cited from the random-graph literature. Nothing in this mission is machine-checked yet. Formalization adds a precise account of what the cited distance asymptotics assert; Proposition 19's upper bound is justified in the paper only by reference to "a standard result on the diameter of the SBM" obtained by adapting an argument of Abbe's survey, so a complete proof of it is new work. Propositions 13 and 15 are classical facts about Erdős–Rényi graphs that have no formal proof in any library known to the curators, and are reusable well beyond community detection.

Difficulty

The LP side of the argument is deterministic and finite. The difficulty is probabilistic. The distance results require tight control of breadth-first neighbourhood growth in sparse random graphs near the connectivity threshold, where the average degree is only logarithmic, with error terms small enough to pin the diameter to a (1±ϵ)(1\pm\epsilon)(1±ϵ) window. A first-moment estimate of path counts gives only one side of each bound. In the SBM the two edge probabilities make neighbourhood growth inhomogeneous, so the Erdős–Rényi results do not transfer verbatim. The threshold of [2] is a separate sharp large-deviation statement, at the level of individual vertices' in- and out-degrees.

Formalization scope

Vertices are Fin n (0-based); graphs are SimpleGraph (Fin n). A bisection is recorded by its side V1V_1V1​ with 2∣V1∣=n2|V_1|=n2∣V1​∣=n. The LP variable is a symmetric, zero-diagonal array x:V×V→Rx:V\times V\to\mathbb Rx:V×V→R, which is a point of R(n2)\mathbb R^{\binom n2}R(2n​); no box constraints are added. LP recovery means that the planted cut vector is the unique optimal solution; minimum-bisection recovery means that every bisection of cost at most the planted one is the planted unordered partition. Distances are in N∪{∞}\mathbb N\cup\{\infty\}N∪{∞} and the average distance in [0,∞][0,\infty][0,∞], so a disconnected sample has infinite diameter. The random-graph laws are finite sums over all graphs; the SBM averages over all bisections. Edge probabilities are clamped into [0,1][0,1][0,1], which matters only for the finitely many nnn with αlog⁡n/n>1\alpha\log n/n>1αlogn/n>1. "With high probability" is a limit of probabilities equal to 111; for the SBM the limit runs along n=2mn=2mn=2m. Logarithms are natural.

Disclosed choices relative to the printed statements: Theorem 8 prints only α,β>0\alpha,\beta>0α,β>0, and α>β\alpha>\betaα>β is added as the SBM's standing assumption p>qp>qp>q (p. 13) and that of (21). The threshold of [2] is printed as an "if and only if"; only the two strict implications are posed, and the boundary α−β=2\sqrt\alpha-\sqrt\beta=\sqrt2α​−β​=2​ is left out.

The statements cannot be satisfied vacuously: the laws are genuine probability distributions for every nnn, the events compare extended-valued distances without truncation, and recovery requires uniqueness over all real feasible points of (LP-P). An encoding with an empty or degenerate feasible set, or with distances truncated to finite reals, would change the theorems and is excluded.

A complete development needs: Erdős–Rényi and SBM laws, breadth-first exploration bounds for sparse random graphs, the coupling of the SBM between two Erdős–Rényi graphs (mission 6), the LP non-recovery criterion (mission 4), and the information-theoretic threshold of [2]. Contributions on the random-graph distance asymptotics are reusable on their own and especially welcome.

Selected references

  • A. Del Pia, A. Khajavirad, D. Kunisky, Linear Programming and Community Detection, Mathematics of Operations Research, 2023. https://arxiv.org/abs/2006.03213 (v2 is the cited version)
  • E. Abbe, A. S. Bandeira, G. Hall, Exact recovery in the stochastic block model, IEEE Transactions on Information Theory 62:471–487, 2016. https://arxiv.org/abs/1405.3267
  • E. Abbe, Community detection and stochastic block models: recent developments, Journal of Machine Learning Research 18:1–86, 2018. https://arxiv.org/abs/1703.10146
  • F. Chung, L. Lu, The diameter of sparse random graphs, Advances in Applied Mathematics 26(4):257–279, 2001. https://doi.org/10.1006/aama.2001.0720
  • F. Chung, L. Lu, The average distances in random graphs with given expected degrees, PNAS 99(25):15879–15882, 2002. https://doi.org/10.1073/pnas.252631999
  • B. Hajek, Y. Wu, J. Xu, Achieving exact cluster recovery threshold via semidefinite programming, IEEE Transactions on Information Theory 62(5):2788–2797, 2016. https://arxiv.org/abs/1412.6156
  • A. S. Bandeira, Random Laplacian matrices and convex relaxations, Foundations of Computational Mathematics 18(2):345–379, 2018. https://arxiv.org/abs/1504.03987
10 thms1 active userReviewed
Algorithmic Game TheoryCombinatoricsMechanism Design·Captain: mikedeng1

Combinatorial Auctions with Interdependent Valuations: SOS to the Rescue 4: The k-SS Mechanism Is Ex-Post IC-IR and a (2 log k + 4)-Approximation to Welfare for Strong-SOS ValuationsResearch Paper

Motivation

In auctions with interdependent values, the value an agent assigns to an outcome depends not only on her own private information but on the information of the others: a bidder for drilling rights values a tract according to every bidder's geological survey, and a buyer of a used asset values it according to every inspector's report. Each agent observes only her own signal, while the mechanism designer knows how values depend on the full signal profile. Maximizing welfare truthfully is then much harder than with private values: the VCG mechanism no longer applies, and without structural assumptions on the valuations no ex-post truthful mechanism gives a good approximation of the optimal welfare (the source's Examples 1.2–1.3).

Eden, Feldman, Fiat, Goldner and Karlin (arXiv:1903.08384) identify submodularity over signals (SOS) as a condition under which constant or near-constant approximations become possible, including for combinatorial auctions, where agents value bundles of items. For single-dimensional signals that take kkk values, they give a (k+3)(k+3)(k+3)-approximation under SOS (Theorem 6.1) and, under the stronger condition strong-SOS, a (2log⁡2k+4)(2\log_2 k+4)(2log2​k+4)-approximation (Theorem 6.5). This mission formalizes the second result.

Timeline. Milgrom and Weber (1982) introduced the interdependent value model for auctions; Dasgupta and Maskin (2000) studied efficient auctions under the single-crossing condition. The computer science literature then gave approximation results for simple single-parameter settings (mostly single-item auctions or matroid constraints), again largely under single crossing or its variants, e.g. Roughgarden and Talgam-Cohen (2016) and Eden, Feldman, Fiat and Goldner (2018), as surveyed in the source. The SOS paper (arXiv v2, 2019; journal version in Mathematics of Operations Research, 2024) drops single crossing and gives, in its own words, the first welfare approximation guarantees for multi-dimensional combinatorial auctions with interdependent values.

Setting

There are nnn agents and mmm items. Agent iii holds a signal si∈{0,1,…,k−1}s_i\in\{0,1,\dots,k-1\}si​∈{0,1,…,k−1}, and s=(s1,…,sn)\mathbf s=(s_1,\dots,s_n)s=(s1​,…,sn​) is the signal profile. For every bundle T⊆[m]T\subseteq[m]T⊆[m], agent iii's value is viT(s)≥0v_{iT}(\mathbf s)\ge0viT​(s)≥0, with vi∅=0v_{i\emptyset}=0vi∅​=0; for nonempty TTT it is weakly increasing in every signal and strictly increasing in sis_isi​.

A function www of the profile is strong-SOS if for every coordinate jjj, every δ≥0\delta\ge0δ≥0 and profiles s′≤s\mathbf s'\le\mathbf ss′≤s coordinate-wise,

w(s−j,sj+δ)−w(s)  ≤  w(s−j′,sj′+δ)−w(s′).w(\mathbf s_{-j},s_j+\delta)-w(\mathbf s)\;\le\;w(\mathbf s'_{-j},s'_j+\delta)-w(\mathbf s').w(s−j​,sj​+δ)−w(s)≤w(s−j′​,sj′​+δ)−w(s′).

Raising one signal helps less the higher the profile, including the raised coordinate itself; in particular the effect of an agent's own signal is concave. SOS asks this only when the raised coordinate starts from the same value in both profiles.

An allocation T=(T1,…,Tn)T=(T_1,\dots,T_n)T=(T1​,…,Tn​) gives pairwise disjoint bundles. A deterministic mechanism maps reported signals to an allocation and payments; it is ex-post IC if, whatever the true signals and with the others truthful, no agent gains by misreporting, and ex-post IR if truthful utility is nonnegative. A randomized mechanism is universally ex-post IC-IR if every deterministic mechanism in its support is.

Let k=2qk=2^qk=2q. Random Bucket picks ℓ∈{1,…,log⁡2k}\ell\in\{1,\dots,\log_2 k\}ℓ∈{1,…,log2​k} uniformly, calls NBℓ={i:si≥2ℓ−1}N_{B_\ell}=\{i:s_i\ge2^{\ell-1}\}NBℓ​​={i:si​≥2ℓ−1} the high agents, and allocates to them a bundle assignment maximizing ∑i∈NBℓviSi(sN¬Bℓ,2ℓ−1NBℓ)\sum_{i\in N_{B_\ell}}v_{iS_i}(\mathbf s_{N_{\neg B_\ell}},\mathbf{2^{\ell-1}}_{N_{B_\ell}})∑i∈NBℓ​​​viSi​​(sN¬Bℓ​​​,2ℓ−1NBℓ​​​), the values with every high signal lowered to 2ℓ−12^{\ell-1}2ℓ−1; agent iii receiving Tˉi\bar T_iTˉi​ pays viTˉi(s−i,2ℓ−1−1)v_{i\bar T_i}(\mathbf s_{-i},2^{\ell-1}-1)viTˉi​​(s−i​,2ℓ−1−1). Random Sampling splits the agents uniformly at random into AAA and BBB, allocates to BBB a bundle assignment maximizing ∑i∈BviSi(sA,0B)\sum_{i\in B}v_{iS_i}(\mathbf s_A,\mathbf 0_B)∑i∈B​viSi​​(sA​,0B​), and charges nothing. k-SS runs Random Bucket with probability pRB=log⁡2klog⁡2k+2p_{RB}=\frac{\log_2 k}{\log_2 k+2}pRB​=log2​k+2log2​k​ and Random Sampling otherwise.

Formalization targets

Goal: Theorem 6.5

For strong-SOS valuations, k-SS is universally ex-post IC-IR and, for every grid profile s\mathbf ss and every feasible allocation T∗T^*T∗,

∑iviTi∗(s)  ≤  (2log⁡2k+4)  E[welfare of k-SS at s].\sum_i v_{iT^*_i}(\mathbf s)\;\le\;(2\log_2 k+4)\;\mathbb E\big[\text{welfare of k-SS at }\mathbf s\big].i∑​viTi∗​​(s)≤(2log2​k+4)E[welfare of k-SS at s].

Milestones

  1. Strong-SOS implies SOS (p. 19).
  2. Display (9), corrected: with SELF=∑ℓ=1k−1∑i:si=ℓviTi(0−i,si)\mathsf{SELF}=\sum_{\ell=1}^{k-1}\sum_{i:s_i=\ell}v_{iT_i}(\mathbf 0_{-i},s_i)SELF=∑ℓ=1k−1​∑i:si​=ℓ​viTi​​(0−i​,si​),
SELF≤2∑ℓ=1log⁡2k∑i:2ℓ−1≤si<2ℓviTi(0−i,2iℓ−1).\mathsf{SELF}\le 2\sum_{\ell=1}^{\log_2 k}\sum_{i:2^{\ell-1}\le s_i<2^\ell}v_{iT_i}(\mathbf 0_{-i},2^{\ell-1}_i).SELF≤2ℓ=1∑log2​k​i:2ℓ−1≤si​<2ℓ∑​viTi​​(0−i​,2iℓ−1​).
  1. Lemma 6.4: Random Bucket is ex-post IC-IR and SELF≤2log⁡2k⋅E[welfare of Random Bucket]\mathsf{SELF}\le 2\log_2 k\cdot\mathbb E[\text{welfare of Random Bucket}]SELF≤2log2​k⋅E[welfare of Random Bucket].
  2. The combination: SELF+OTHER≤(2log⁡2k+4) E[welfare of k-SS]\mathsf{SELF}+\mathsf{OTHER}\le(2\log_2 k+4)\,\mathbb E[\text{welfare of k-SS}]SELF+OTHER≤(2log2​k+4)E[welfare of k-SS], where OTHER=∑iviTi(s−i,0i)\mathsf{OTHER}=\sum_i v_{iT_i}(\mathbf s_{-i},0_i)OTHER=∑i​viTi​​(s−i​,0i​).

Significance

The theorem shows that the price of truthfulness in combinatorial auctions with interdependent values grows only logarithmically in the number of signal values once values are concave in the agents' own signals, compared with the linear (k+3)(k+3)(k+3) of the SOS mechanism. It also exhibits a decomposition of optimal welfare into a part driven by the other agents' signals (OTHER) and a part driven by the agent's own signal (SELF), each covered by a different randomized mechanism; this template recurs in later work on interdependent values.

The result is proved in the paper; no machine-checked version exists. Formalizing it fixes the exact model behind a proof that is stated in a few lines and checks it against the paper's own appendix: the printed display (9) omits a factor 2 (it fails for one agent with v(s)=s1v(\mathbf s)=s_1v(s)=s1​, k=4k=4k=4, s1=3s_1=3s1​=3), and the mission states the corrected inequality, which is the d=1d=1d=1 case of the paper's display (12) and is what the ratio 2log⁡2k2\log_2 k2log2​k requires.

Difficulty

The obvious route compares the mechanism's allocation with the optimal one agent by agent, but the mechanisms allocate by proxy values computed from modified signal profiles, and the optimal allocation's value at the true profile is not directly comparable with any single proxy. Covering SELF requires bounding an agent's own-signal contribution within a bucket by its value at the bucket floor, which needs concavity in the own signal and is false under SOS alone. Covering OTHER requires the Key Lemma 3.1 on random zeroing of signals. Incentive compatibility needs the argmax to be read off the proxy profile alone; an argmax with tie-breaking on the full report fails it.

Formalization scope

Signals are natural numbers and every hypothesis (nonnegativity, monotonicity, SOS, strong-SOS, the selectors' optimality) is quantified over the grid {0,…,k−1}n\{0,\dots,k-1\}^n{0,…,k−1}n only. k=2qk=2^qk=2q with log⁡2k=q\log_2 k=qlog2​k=q is a disclosed reading: the buckets cover {1,…,k−1}\{1,\dots,k-1\}{1,…,k−1} only for powers of two. The empty bundle is worth 000, and the monotonicity clauses apply to nonempty bundles. The argmax of each mechanism is an arbitrary selector that sees only the bucket index (or BBB) and the proxy profile and returns a feasible maximizer; theorems quantify over all such selectors. Random Sampling zeroes all of BBB, the agent's own signal included (the paper's "vijv_{ij}vij​" is read as viTv_{iT}viT​). Expectations are finite uniform averages; at q=0q=0q=0 Lean's 0−1=00^{-1}=00−1=0 makes the Random Bucket term vanish, matching pRB=0p_{RB}=0pRB​=0.

The approximation is stated for every feasible allocation and with true values at the truthful profile, not with proxy values or a supremum; IC and IR are stated for each deterministic component, not for the mixture's expectation. Replacing the average over partitions by an existential, or the explicit payments by existence of payments, would not be this theorem.

The model layer (signal grid, combinatorial valuations, SOS, ex-post IC-IR, OTHER and SELF, Random Sampling) is shared with the companion missions and lives in InterdepSOS.KHL and InterdepSOS.RSV; strong-SOS, Random Bucket and k-SS are defined in InterdepSOS.KSS. Lemma 6.3 and the decomposition (7), which the proof uses, are posed in the companion mission on k-HL. Proofs of the milestones, and reusable lemmas on concavity in the own signal and on random zeroing, are welcome. The source is the arXiv v2 preprint; the journal version in Mathematics of Operations Research (2024) is not what the mission cites.

Selected references

  • A. Eden, M. Feldman, A. Fiat, K. Goldner, A. R. Karlin, Combinatorial Auctions with Interdependent Valuations: SOS to the Rescue, arXiv:1903.08384v2, 2019. https://arxiv.org/abs/1903.08384v2
  • P. Milgrom, R. Weber, A Theory of Auctions and Competitive Bidding, Econometrica 50(5), 1982. https://doi.org/10.2307/1911865
  • P. Dasgupta, E. Maskin, Efficient Auctions, Quarterly Journal of Economics 115(2), 2000. https://doi.org/10.1162/003355300554737
10 thms1 active userReviewed
CombinatoricsGraph TheoryLinear Optimization+2·Captain: mikedeng1

Linear Programming and Community Detection 5: The LP Relaxation Recovers the Planted Bisection w.h.p. in the Very Dense SBM When p − q > 1/2Research Paper

Motivation

The stochastic block model (SBM), introduced by Holland, Laskey and Leinhardt (1983), is the standard random model for networks with community structure, and the basic benchmark for community-detection algorithms (Abbe 2018). In its two-community form the nodes are split into two equal halves, edges inside a half appear with probability ppp, and edges across with probability q<pq<pq<p. The task of exact recovery is to identify the hidden halves from the graph alone.

The maximum a posteriori estimator for this model is the minimum bisection problem (find a balanced cut with the fewest crossing edges), which is NP-hard in general. Abbe, Bandeira and Hall (2016) determined when it recovers the planted bisection, and showed that a semidefinite relaxation is as good as the integer program. Del Pia, Khajavirad and Kunisky (arXiv:2006.03213, Mathematics of Operations Research 2023) ask the same question for the cheapest standard relaxation: the metric linear programming relaxation, defined by triangle inequalities. This mission is their positive result in the very dense regime.

Setting

Let V={1,…,n}V=\{1,\dots,n\}V={1,…,n} with nnn even. A bisection is a partition V=V1∪V2V=V_1\cup V_2V=V1​∪V2​ with ∣V1∣=∣V2∣=n/2|V_1|=|V_2|=n/2∣V1​∣=∣V2​∣=n/2. For a graph G=(V,E)G=(V,E)G=(V,E) let aij=1a_{ij}=1aij​=1 if (i,j)∈E(i,j)\in E(i,j)∈E and 000 otherwise. The metric relaxation (LP-P) is

min⁡x∈R(n2) ∑i<jaijxijs.t.xij≤xik+xjk,xij+xik+xjk≤2,∑i<jxij=n24,\min_{x\in\mathbb R^{\binom n2}}\ \sum_{i<j}a_{ij}x_{ij}\quad\text{s.t.}\quad x_{ij}\le x_{ik}+x_{jk},\quad x_{ij}+x_{ik}+x_{jk}\le 2,\quad \sum_{i<j}x_{ij}=\frac{n^2}{4},x∈R(2n​)min​ i<j∑​aij​xij​s.t.xij​≤xik​+xjk​,xij​+xik​+xjk​≤2,i<j∑​xij​=4n2​,

the first constraint for all distinct i,j,ki,j,ki,j,k with i<ji<ji<j, the second for all i<j<ki<j<ki<j<k, and xji:=xijx_{ji}:=x_{ij}xji​:=xij​. The cut vector xˉ\bar xxˉ of a bisection has xˉij=0\bar x_{ij}=0xˉij​=0 for i,ji,ji,j on the same side and xˉij=1\bar x_{ij}=1xˉij​=1 otherwise. (LP-P) recovers the planted bisection when xˉ\bar xxˉ is its unique optimal solution.

In the Erdős–Rényi model Gn,p\mathcal G_{n,p}Gn,p​ every pair of nodes is an edge independently with probability ppp. In the SBM Gn,p,q\mathcal G_{n,p,q}Gn,p,q​ a bisection V1,V2V_1,V_2V1​,V2​ is first chosen uniformly at random; then pairs inside V1V_1V1​ or inside V2V_2V2​ are edges with probability ppp, pairs between them with probability qqq, all independently. Gn,0,q\mathcal G_{n,0,q}Gn,0,q​ is the random bipartite graph across a random bisection. An event occurs with high probability (w.h.p.) if its probability tends to 111 as n→∞n\to\inftyn→∞. The graph is very dense when ppp and qqq are constants.

For a finite graph G=(V,E)G=(V,E)G=(V,E) and b∈Z≥0Vb\in\mathbb Z_{\ge0}^Vb∈Z≥0V​, a bbb-factor (or fff-factor) is a set F⊆EF\subseteq EF⊆E with exactly bvb_vbv​ edges of FFF at each node vvv; N(U,U′)N(U,U')N(U,U′) counts edges with one endpoint in UUU and one in U′U'U′; dmin⁡d_{\min}dmin​, dmax⁡d_{\max}dmax​ are the minimum and maximum degrees; and the node connectivity κ(G)\kappa(G)κ(G) is the minimum number of nodes whose deletion disconnects GGG (the number of nodes, for a complete graph).

Formalization targets

Goal: Theorem 6 (p. 15)

For constants 0<q<p<10<q<p<10<q<p<1,

p−q>12 ⟹ PG∼Gn,p,q[(LP-P) recovers the planted bisection]→n→∞1.p-q>\tfrac12\ \Longrightarrow\ \mathbb P_{G\sim\mathcal G_{n,p,q}}\big[\text{(LP-P) recovers the planted bisection}\big]\xrightarrow[n\to\infty]{}1 .p−q>21​ ⟹ PG∼Gn,p,q​​[(LP-P) recovers the planted bisection]n→∞​1.

Milestones

  1. Proposition 6 (p. 20): for G∼G2m,0,qG\sim\mathcal G_{2m,0,q}G∼G2m,0,q​ there is C>0C>0C>0 with dmax⁡≤qm+Cmlog⁡md_{\max}\le qm+C\sqrt{m\log m}dmax​≤qm+Cmlogm​ w.h.p.
  2. Proposition 5 (p. 20; Schrijver, Cor. 21.4a): a bipartite graph has a bbb-factor iff N(U,U)≥b(U)−12b(V)N(U,U)\ge b(U)-\frac12 b(V)N(U,U)≥b(U)−21​b(V) for every U⊆VU\subseteq VU⊆V.
  3. Proposition 3 (p. 14): there is Creg,2>0C_{\mathrm{reg},2}>0Creg,2​>0 such that for every q∈(0,1)q\in(0,1)q∈(0,1), w.h.p. G∼Gn,0,qG\sim\mathcal G_{n,0,q}G∼Gn,0,q​ is a subgraph of a regular bipartite graph with the same bipartition, of degree dreg=qn2+Creg,2n2log⁡n2d_{\mathrm{reg}}=\frac{qn}2+C_{\mathrm{reg},2}\sqrt{\frac n2\log\frac n2}dreg​=2qn​+Creg,2​2n​log2n​​ (rounded down).
  4. Proposition 8 (p. 24): for G∼Gn,pG\sim\mathcal G_{n,p}G∼Gn,p​ there is C>0C>0C>0 with dmin⁡≥pn−Cnlog⁡nd_{\min}\ge pn-C\sqrt{n\log n}dmin​≥pn−Cnlogn​ w.h.p.
  5. Proposition 9 (p. 24; Bollobás–Thomason): for G∼Gn,pG\sim\mathcal G_{n,p}G∼Gn,p​, κ(G)=dmin⁡\kappa(G)=d_{\min}κ(G)=dmin​ w.h.p.
  6. Proposition 7 (p. 24; Tutte's fff-factor theorem).
  7. Proposition 4 (p. 14): there is Creg,1>0C_{\mathrm{reg},1}>0Creg,1​>0 such that for every p∈(0,1)p\in(0,1)p∈(0,1), w.h.p. G∼Gn,pG\sim\mathcal G_{n,p}G∼Gn,p​ contains a spanning regular subgraph of degree dreg=pn−Creg,1nlog⁡nd_{\mathrm{reg}}=pn-C_{\mathrm{reg},1}\sqrt{n\log n}dreg​=pn−Creg,1​nlogn​ (rounded up to an even integer).

The deterministic recovery criterion that the goal also depends on, Theorem 4 of the paper (regular sandwiching with din−dout≥n/4d_{\mathrm{in}}-d_{\mathrm{out}}\ge n/4din​−dout​≥n/4), is the goal of mission 3 of this series and is not restated here.

Significance

The result. Theorem 6 is the only positive recovery guarantee for the metric LP in the random setting of the paper. Together with the failure results of Theorems 7 and 8 (missions 6 and 7), it locates the LP strictly between trivial and information-theoretically optimal: it recovers in a region of constant p,qp,qp,q, but not up to the threshold p>qp>qp>q that the minimum bisection problem and its SDP relaxation reach. Propositions 3 and 4, that dense random graphs sit inside, and contain, regular graphs of nearly their average degree, are of independent interest in random graph theory, as the authors note.

Formalizing it. The result is proved in the paper; none of it is machine-checked. The mission also requires three classical theorems that are not in Mathlib: the bbb-factor criterion for bipartite graphs, Tutte's fff-factor theorem (Mathlib has only the perfect-matching case f≡1f\equiv 1f≡1), and the Bollobás–Thomason theorem κ=dmin⁡\kappa=d_{\min}κ=dmin​ for random graphs, together with degree concentration for Gn,p\mathcal G_{n,p}Gn,p​ and the bipartite SBM. Each is reusable well beyond this paper.

Difficulty

The obvious argument for recovery is to exhibit a dual certificate for (LP-P), which needs regularity of the communities and of the cross edges; a random graph is not regular. The paper bridges the gap by sandwiching the random graph between regular graphs of nearly equal degree, and that step is where the difficulty lies: the existence of a regular sub- or supergraph is a global combinatorial property, characterized by factor theorems with exponentially many conditions, each of which must hold simultaneously with high probability. Degree concentration alone, the first idea, controls only the singleton conditions. Tutte's fff-factor theorem and the bipartite bbb-factor criterion are themselves substantial combinatorial theorems, and the connectivity statement of Bollobás and Thomason is a deep result on random graphs.

Formalization scope

Nodes are Fin n (0-based). A bisection is a Finset V1V_1V1​ with 2∣V1∣=n2|V_1|=n2∣V1​∣=n. Points of R(n2)\mathbb R^{\binom n2}R(2n​) are symmetric arrays with zero diagonal; (LP-P) has no box constraints 0≤x≤10\le x\le10≤x≤1 (they are implied by the constraints and adding them would change the LP). Recovery means: xˉ\bar xxˉ is optimal and every optimal solution equals xˉ\bar xxˉ.

The random-graph laws are finite: a probability is a weighted sum over all graphs on Fin n, with independent edge weights, and for the SBM an average over all bisections; the event may depend on the drawn bisection. Edge probabilities are clamped into [0,1][0,1][0,1], which is the identity on the parameters used here. "With high probability" is Tendsto … atTop (𝓝 1), along n=2mn=2mn=2m for the SBM. The standing assumption p>qp>qp>q is not built into the law, so that Gn,0,q\mathcal G_{n,0,q}Gn,0,q​ is the case p=0p=0p=0.

Disclosed readings: the printed dregd_{\mathrm{reg}}dreg​ of Propositions 3 and 4 is not an integer; Proposition 3 is posed with degree ⌊dreg⌋\lfloor d_{\mathrm{reg}}\rfloor⌊dreg​⌋ and Proposition 4 with 2⌈dreg/2⌉2\lceil d_{\mathrm{reg}}/2\rceil2⌈dreg​/2⌉, the least even integer ≥dreg\ge d_{\mathrm{reg}}≥dreg​ (the proof assumes dregd_{\mathrm{reg}}dreg​ even). κ(Kn)=n\kappa(K_n)=nκ(Kn​)=n as on p. 24. Quantifier order follows each page: in Propositions 3 and 4 the constant precedes the probability parameter, in Propositions 6 and 8 it follows it; no constant depends on nnn.

A statement in which "with high probability" is replaced by a positive probability or an eventual bound, in which the bisection is fixed rather than random, or in which the regular degree is compared with an irrational real (an empty event) is not this mission's target.

Infrastructure needed: the finite random-graph laws, Hoeffding-type concentration over them, the two factor theorems, and node connectivity. Proofs of any milestone, and of the classical results as general graph theory, are welcome.

Selected references

  • A. Del Pia, A. Khajavirad, D. Kunisky, Linear Programming and Community Detection, Mathematics of Operations Research, 2023; arXiv:2006.03213v2. https://arxiv.org/abs/2006.03213
  • E. Abbe, A. S. Bandeira, G. Hall, Exact Recovery in the Stochastic Block Model, IEEE Transactions on Information Theory 62(1), 2016. https://arxiv.org/abs/1405.3267
  • E. Abbe, Community Detection and Stochastic Block Models: Recent Developments, Journal of Machine Learning Research 18, 2018. https://arxiv.org/abs/1703.10146
  • P. W. Holland, K. B. Laskey, S. Leinhardt, Stochastic Blockmodels: First Steps, Social Networks 5(2), 1983. https://doi.org/10.1016/0378-8733(83)90021-7
  • W. T. Tutte, The Factors of Graphs, Canadian Journal of Mathematics 4, 1952. https://doi.org/10.4153/CJM-1952-028-2
  • A. Schrijver, Combinatorial Optimization: Polyhedra and Efficiency, Springer, 2003 (Corollary 21.4a).
  • B. Bollobás, A. Thomason, Random Graphs of Small Order, in Random Graphs '83, North-Holland Mathematics Studies 118, 1985 (Theorem 1).
  • A. Frieze, M. Karoński, Introduction to Random Graphs, Cambridge University Press, 2016 (Chapter 3). https://doi.org/10.1017/CBO9781316339831
10 thms1 active userReviewed
Graph TheoryLinear Optimization·Captain: mikedeng1

Linear Programming and Community Detection 4: A Distance-Based Condition Under Which the Metric LP Relaxation Fails to Recover the Planted BisectionResearch Paper

Motivation

The minimum bisection problem asks, for a graph on an even number nnn of vertices, for a split of the vertices into two halves that cuts as few edges as possible. It is NP-hard, and it is the maximum a posteriori estimator for the two-community stochastic block model (SBM), the standard benchmark for community detection (Abbe, 2018). Convex relaxations are the usual way to make such estimators tractable while keeping guarantees. For the semidefinite relaxation, exact-recovery thresholds are known (Abbe, Bandeira, Hall, 2016; Hajek, Wu, Xu, 2016). Linear programming relaxations are cheaper and much used in combinatorial optimization, but their recovery properties were not understood.

Del Pia, Khajavirad and Kunisky (arXiv:2006.03213v2, Mathematics of Operations Research, 2023) study the best-known LP relaxation of minimum bisection, the metric relaxation. They give a sufficient condition for it to recover a planted bisection (Theorem 4) and a necessary one (Theorem 5). This mission formalizes the necessary condition. Its consequence, developed in §4 of the paper, is that the metric relaxation fails with high probability in most SBM regimes, including those where the minimum bisection itself succeeds.

Setting

Let G=(V,E)G=(V,E)G=(V,E) be a simple graph on V={1,…,n}V=\{1,\dots,n\}V={1,…,n}, and write (i,j)(i,j)(i,j) for an edge with i<ji<ji<j; set aij=1a_{ij}=1aij​=1 if (i,j)∈E(i,j)\in E(i,j)∈E and aij=0a_{ij}=0aij​=0 otherwise. A bisection is a partition of VVV into two sets V1,V2V_1,V_2V1​,V2​ with ∣V1∣=∣V2∣=n/2|V_1|=|V_2|=n/2∣V1​∣=∣V2​∣=n/2. One fixed bisection, unknown to the solver, is the planted bisection; E0E_0E0​ is the set of edges with one end in V1V_1V1​ and the other in V2V_2V2​.

The cut vector xˉ∈{0,1}(n2)\bar x\in\{0,1\}^{\binom n2}xˉ∈{0,1}(2n​) of the planted bisection has xˉij=1\bar x_{ij}=1xˉij​=1 exactly when iii and jjj lie on different sides. The metric relaxation is the linear program, with xji:=xijx_{ji}:=x_{ij}xji​:=xij​,

min⁡x ∑1≤i<j≤naijxijs.t.xij≤xik+xjk (1),xij+xik+xjk≤2 (2),∑1≤i<j≤nxij=n24 (3),\min_x\ \sum_{1\le i<j\le n}a_{ij}x_{ij}\quad\text{s.t.}\quad x_{ij}\le x_{ik}+x_{jk}\ (1),\qquad x_{ij}+x_{ik}+x_{jk}\le 2\ (2),\qquad \sum_{1\le i<j\le n}x_{ij}=\frac{n^2}{4}\ (3),xmin​ 1≤i<j≤n∑​aij​xij​s.t.xij​≤xik​+xjk​ (1),xij​+xik​+xjk​≤2 (2),1≤i<j≤n∑​xij​=4n2​ (3),

where (1) ranges over distinct i,j,ki,j,ki,j,k with i<ji<ji<j and (2) over i<j<ki<j<ki<j<k. The LP recovers the planted bisection if xˉ\bar xxˉ is its unique optimal solution. The objective at xˉ\bar xxˉ is ∣E0∣|E_0|∣E0​∣.

The graph distance ρG(i,j)\rho_G(i,j)ρG​(i,j) is the length of a shortest iii–jjj path (+∞+\infty+∞ across components). The diameter and the average distance are

ρmax⁡(G)=max⁡i,j∈VρG(i,j),ρavg(G)=2n2∑1≤i<j≤nρG(i,j).\rho_{\max}(G)=\max_{i,j\in V}\rho_G(i,j),\qquad \rho_{\mathrm{avg}}(G)=\frac{2}{n^2}\sum_{1\le i<j\le n}\rho_G(i,j).ρmax​(G)=i,j∈Vmax​ρG​(i,j),ρavg​(G)=n22​1≤i<j≤n∑​ρG​(i,j).

The average is over all n2n^2n2 ordered pairs, the diagonal included. For n≥5n\ge5n≥5 define

c(G)=max⁡{0, 3ρmax⁡(G)−4ρavg(G)1−4/n}.(17)c(G)=\max\Big\{0,\ \frac{3\rho_{\max}(G)-4\rho_{\mathrm{avg}}(G)}{1-4/n}\Big\}.\qquad(17)c(G)=max{0, 1−4/n3ρmax​(G)−4ρavg​(G)​}.(17)

Formalization targets

Goal: Theorem 5

For a connected graph GGG on n≥5n\ge5n≥5 nodes with a planted bisection,

1+c(G)2ρavg(G)+2c(G)(1−1n) ∣E∣<∣E0∣(18)⟹(LP-P) does not recover the planted bisection.\frac{1+c(G)}{2\rho_{\mathrm{avg}}(G)+2c(G)(1-\frac1n)}\,|E|<|E_0|\quad(18)\qquad\Longrightarrow\qquad\text{(LP-P) does not recover the planted bisection.}2ρavg​(G)+2c(G)(1−n1​)1+c(G)​∣E∣<∣E0​∣(18)⟹(LP-P) does not recover the planted bisection.

Milestones

The proof of Theorem 5 works with the point x~∈R(n2)\tilde x\in\mathbb R^{\binom n2}x~∈R(2n​),

x~ij=ρG(i,j)+c(G)2ρavg(G)+2c(G)(1−1n).\tilde x_{ij}=\frac{\rho_G(i,j)+c(G)}{2\rho_{\mathrm{avg}}(G)+2c(G)(1-\frac1n)}.x~ij​=2ρavg​(G)+2c(G)(1−n1​)ρG​(i,j)+c(G)​.

The mission poses three statements about it, for a connected GGG on n≥5n\ge5n≥5 nodes:

  1. display (19), p. 12: for i<j<ki<j<ki<j<k, x~ij+x~ik+x~jk≤32⋅ρmax⁡(G)+c(G)ρavg(G)+c(G)(1−1n)\displaystyle\tilde x_{ij}+\tilde x_{ik}+\tilde x_{jk}\le\frac32\cdot\frac{\rho_{\max}(G)+c(G)}{\rho_{\mathrm{avg}}(G)+c(G)(1-\frac1n)}x~ij​+x~ik​+x~jk​≤23​⋅ρavg​(G)+c(G)(1−n1​)ρmax​(G)+c(G)​;
  2. p. 13: x~\tilde xx~ is feasible for (LP-P);
  3. p. 13: the objective of (LP-P) at x~\tilde xx~ equals 1+c(G)ρavg(G)+c(G)(1−1n)⋅∣E∣2\displaystyle\frac{1+c(G)}{\rho_{\mathrm{avg}}(G)+c(G)(1-\frac1n)}\cdot\frac{|E|}{2}ρavg​(G)+c(G)(1−n1​)1+c(G)​⋅2∣E∣​.

Significance

The result. Theorem 5 is a deterministic certificate of failure, stated for any connected graph, not for a random model. It reduces the question "does the LP recover?" to comparing two numbers that depend only on the distance profile of GGG and on the fraction of edges cut by the planted bisection. When typical distances are close to the diameter, c(G)c(G)c(G) is small and (18) asks only that ∣E0∣/∣E∣|E_0|/|E|∣E0​∣/∣E∣ exceed roughly 1/(2ρavg(G))1/(2\rho_{\mathrm{avg}}(G))1/(2ρavg​(G)). In random graphs typical distances are close to the diameter. The paper's Theorems 7 and 8 combine Theorem 5 with known distance asymptotics of Erdős–Rényi graphs (Bollobás, Random Graphs, 2001; Chung, Lu, 2001). They show that the metric relaxation fails with high probability in the dense and logarithmic-degree SBM, in contrast with the semidefinite relaxation. Those theorems are separate missions of this series and use Theorem 5 as a black box.

Formalizing it. The result is proved in the paper; no machine-checked version is known. The mission produces a formal metric relaxation of minimum bisection, with LP recovery defined as uniqueness of the optimum, and links it to graph distances through Mathlib's SimpleGraph.edist/ediam. The definitions are written to coincide with those of the other missions of this series, so that one model layer serves the sufficient condition (Theorem 4), the necessary condition (this mission), and the random-graph consequences.

Difficulty

Theorem 5 is not deep, but it is exact. The milestones fix the witness x~\tilde xx~; the work is in checking it. The obvious candidate, the graph metric ρG\rho_GρG​ itself rescaled to satisfy the equality (3), satisfies the triangle inequalities (1). It can violate the perimeter constraints (2) whenever some triangle of vertices is spread out compared with the average distance. The shift by c(G)c(G)c(G), which depends on ρmax⁡\rho_{\max}ρmax​ and on the factor 1−4/n1-4/n1−4/n, is tuned so that (2) holds with no slack in the worst case. A proof has to keep track of the normalization 2/n22/n^22/n2 (ordered pairs, diagonal included), the factor 1−1/n1-1/n1−1/n, and the sign of 3ρmax⁡−4ρavg3\rho_{\max}-4\rho_{\mathrm{avg}}3ρmax​−4ρavg​, and must use n≥5n\ge5n≥5 wherever it divides by 1−4/n1-4/n1−4/n. In Lean the further work is converting between sums over ordered pairs, unordered pairs and the edge set, and between extended-valued distances and their real values for a connected graph.

Formalization scope

  • Vertices and bisections. Vertices are Fin n (0-based). The planted bisection is a finset V1V_1V1​ with 2∣V1∣=n2|V_1|=n2∣V1​∣=n, and V2=V1cV_2=V_1^{c}V2​=V1c​. ∣E0∣|E_0|∣E0​∣ is the number of pairs i<ji<ji<j adjacent in GGG and on different sides.
  • LP variables. A point of R(n2)\mathbb R^{\binom n2}R(2n​) is an array x:Fin n→Fin n→Rx:\texttt{Fin } n\to\texttt{Fin } n\to\mathbb Rx:Fin n→Fin n→R. Feasibility includes symmetry and a zero diagonal, so the free diagonal and lower-triangle entries cannot break uniqueness of the optimum. No bounds 0≤x≤10\le x\le10≤x≤1 are added; they are not constraints of (LP-P).
  • Recovery. LPRecovers G V1 says that the cut vector of V1V_1V1​ is optimal and every optimal point equals it.
  • Distances. ρG\rho_GρG​ is SimpleGraph.edist (in N∪{∞}\mathbb N\cup\{\infty\}N∪{∞}), ρmax⁡\rho_{\max}ρmax​ is SimpleGraph.ediam, and ρavg\rho_{\mathrm{avg}}ρavg​ is the printed formula computed in [0,∞][0,\infty][0,∞]. c(G)c(G)c(G) and x~\tilde xx~ use their real values. All statements assume GGG connected, where these values are finite and ρavg(G)>0\rho_{\mathrm{avg}}(G)>0ρavg​(G)>0.
  • Standing hypotheses. Every statement carries exactly the page's hypotheses: GGG connected and n≥5n\ge5n≥5, plus, for Theorem 5, the planted bisection. The bisection forces nnn even, so in effect n≥6n\ge6n≥6; n≥5n\ge5n≥5 is kept as printed. x~\tilde xx~ is indexed by i<ji<ji<j on the page and is extended symmetrically with zero diagonal.
  • Ruled out. A formalization that adds box constraints, drops the symmetric zero-diagonal normalization, averages over n(n−1)/2n(n-1)/2n(n−1)/2 pairs instead of the printed 2/n22/n^22/n2, or omits connectivity (where the real value of an infinite distance would be 000) states a different theorem. None of these is used here.

Contributions welcome: proofs of the three milestones and the goal. Reusable lemmas include the identity ∑i<jaijxij=∑(i,j)∈Exij\sum_{i<j}a_{ij}x_{ij}=\sum_{(i,j)\in E}x_{ij}∑i<j​aij​xij​=∑(i,j)∈E​xij​ for SimpleGraph.edgeFinset, the finiteness of ediam for connected graphs on Fin n, and the bound ρavg(G)≤(1−1n)ρmax⁡(G)\rho_{\mathrm{avg}}(G)\le(1-\frac1n)\rho_{\max}(G)ρavg​(G)≤(1−n1​)ρmax​(G).

Selected references

  • A. Del Pia, A. Khajavirad, D. Kunisky, Linear Programming and Community Detection, Mathematics of Operations Research, 2023; arXiv:2006.03213v2 (2022). https://arxiv.org/abs/2006.03213
  • E. Abbe, Community Detection and Stochastic Block Models: Recent Developments, Journal of Machine Learning Research 18 (2018). https://arxiv.org/abs/1703.10146
  • E. Abbe, A. S. Bandeira, G. Hall, Exact Recovery in the Stochastic Block Model, IEEE Transactions on Information Theory 62 (2016). https://arxiv.org/abs/1405.3267
  • B. Hajek, Y. Wu, J. Xu, Achieving Exact Cluster Recovery Threshold via Semidefinite Programming, IEEE Transactions on Information Theory 62 (2016). https://arxiv.org/abs/1412.6156
  • B. Bollobás, Random Graphs, 2nd ed., Cambridge University Press, 2001. https://doi.org/10.1017/CBO9780511814068
  • F. Chung, L. Lu, The Diameter of Sparse Random Graphs, Advances in Applied Mathematics 26 (2001). https://doi.org/10.1006/aama.2001.0720
8 thms1 active userReviewed
Graph TheoryProbability·Captain: mikedeng1

Load Balancing Under Strict Compatibility Constraints 3: Inhomogeneous Random Graphs with Edge Probabilities Satisfying (3.5) Are Almost Surely Proportionally Sparse and SubcriticalResearch Paper

Motivation

Large-scale computing systems route incoming tasks to servers through dispatchers, one per task type. A task can only run on a server that holds the data or software it needs, so the set of admissible pairs forms a bipartite compatibility graph between task types and servers. Full flexibility (every server can serve every type) gives the best load balancing but requires each server to store everything. A central question is how sparse the compatibility graph can be while keeping the performance of the fully flexible system.

For the power-of-ddd policy JSQ(ddd) (an arriving task samples ddd compatible servers and joins the shortest of their queues), Rutten and Mukherjee (arXiv:2008.07562v2, Math. Oper. Res. 2023) identify two deterministic graph conditions, proportional sparsity and subcriticality, under which the occupancy process has the same mean-field limit and the same steady-state limit as in the fully flexible system (their Theorem 3.4). They then show that natural random designs satisfy both conditions almost surely. This mission covers the second random design: task types with inhomogeneous levels of flexibility, where each edge incident to task type www is present independently with its own probability pw(N)p_w(N)pw​(N) (Theorem 3.6 of the paper).

Earlier work on JSQ-type policies with constrained compatibility includes the stability analyses of Bramson (Ann. Appl. Probab. 2011) and of Cardinaels, Borst and van Leeuwaarden (Queueing Syst. 2019), to which subcriticality is equivalent for a fixed system (Remark 3 of the paper), and the mean-field analysis of the supermarket model on graphs by Budhiraja, Mukherjee and Wu (Ann. Appl. Probab. 2019), whose load condition is stronger than the condition (A.19) used here (Remark 11 of the paper).

Setting

For each N≥1N \ge 1N≥1 there are a set VNV_NVN​ of NNN servers and a set WNW_NWN​ of M(N)≥1M(N) \ge 1M(N)≥1 dispatchers. A compatibility graph is GN=(VN,WN,EN)G_N = (V_N, W_N, E_N)GN​=(VN​,WN​,EN​) with EN⊆VN×WNE_N \subseteq V_N \times W_NEN​⊆VN​×WN​, and the neighbourhood of www is Nw={v:(v,w)∈EN}N_w = \{v : (v, w) \in E_N\}Nw​={v:(v,w)∈EN​}.

Proportional sparsity (Definition 3.1). For every ε>0\varepsilon > 0ε>0,

sup⁡U⊆VN1M(N)∣{w∈WN:∣∣Nw∩U∣∣Nw∣−∣U∣N∣≥ε}∣→0.\sup_{U \subseteq V_N} \frac{1}{M(N)} \Bigl| \Bigl\{ w \in W_N : \Bigl| \frac{|N_w \cap U|}{|N_w|} - \frac{|U|}{N} \Bigr| \ge \varepsilon \Bigr\} \Bigr| \to 0 .U⊆VN​sup​M(N)1​​{w∈WN​:​∣Nw​∣∣Nw​∩U∣​−N∣U∣​​≥ε}​→0.

Every set UUU of servers is seen by all but a vanishing fraction of dispatchers in roughly its global proportion.

Subcriticality (Definition 3.3). For each NNN, www and nonempty U⊆VNU \subseteq V_NU⊆VN​ there is a probability distribution γwU\gamma^U_wγwU​ on UUU such that, with kw=min⁡(d,∣Nw∣)k_w = \min(d, |N_w|)kw​=min(d,∣Nw​∣),

lim sup⁡N→∞max⁡v∈VNNM(N)∑w∈WN(∣Nw∣kw)−1∑U⊆Nw, ∣U∣=kwγwU(v)≤1.\limsup_{N \to \infty} \max_{v \in V_N} \frac{N}{M(N)} \sum_{w \in W_N} \binom{|N_w|}{k_w}^{-1} \sum_{U \subseteq N_w,\ |U| = k_w} \gamma^U_w(v) \le 1 .N→∞limsup​v∈VN​max​M(N)N​w∈WN​∑​(kw​∣Nw​∣​)−1U⊆Nw​, ∣U∣=kw​∑​γwU​(v)≤1.

Some static routing of sampled sets loads no server asymptotically beyond its capacity.

The random graph. Given edge probabilities pw(N)∈(0,1]p_w(N) \in (0, 1]pw​(N)∈(0,1], the graph GNG_NGN​ contains each pair (v,w)(v, w)(v,w) independently with probability pw(N)p_w(N)pw​(N). Write pˉ(N)=min⁡wpw(N)\bar p(N) = \min_{w} p_w(N)pˉ​(N)=minw​pw​(N) and ∥p(N)−1∥2−2=(∑wpw(N)−2)−1\|p(N)^{-1}\|_2^{-2} = (\sum_w p_w(N)^{-2})^{-1}∥p(N)−1∥2−2​=(∑w​pw​(N)−2)−1. Only the law of each GNG_NGN​ is prescribed; the graphs for different NNN may be coupled arbitrarily, and "almost surely" refers to any such coupling.

Formalization targets

Goal: Theorem 3.6 (graph part)

If, as N→∞N \to \inftyN→∞,

Nln⁡M(N)+ln⁡N pˉ(N)→∞,M(N) pˉ(N)→∞,ln⁡NM(N)2∑w∈WN1pw(N)2→0,(3.5)\frac{N}{\ln M(N) + \ln N}\, \bar p(N) \to \infty, \qquad M(N)\, \bar p(N) \to \infty, \qquad \frac{\ln N}{M(N)^2} \sum_{w \in W_N} \frac{1}{p_w(N)^2} \to 0, \qquad (3.5)lnM(N)+lnNN​pˉ​(N)→∞,M(N)pˉ​(N)→∞,M(N)2lnN​w∈WN​∑​pw​(N)21​→0,(3.5)

then almost surely {GN}\{G_N\}{GN​} is proportionally sparse and satisfies the subcriticality condition.

Milestones

  1. Lemma B.1. If pˉ(N)=ω(1/N)\bar p(N) = \omega(1/N)pˉ​(N)=ω(1/N) and pˉ(N)=ω(1/M(N))\bar p(N) = \omega(1/M(N))pˉ​(N)=ω(1/M(N)), then {GN}\{G_N\}{GN​} is proportionally sparse almost surely.
  2. The (A.19) reduction. For any graph sequence and d≥1d \ge 1d≥1,
lim sup⁡N→∞max⁡v∈VNNM(N)∑w∈WN1{(v,w)∈EN}∣Nw∣≤1(A.19)\limsup_{N \to \infty} \max_{v \in V_N} \frac{N}{M(N)} \sum_{w \in W_N} \frac{\mathbb 1\{(v, w) \in E_N\}}{|N_w|} \le 1 \qquad \text{(A.19)}N→∞limsup​v∈VN​max​M(N)N​w∈WN​∑​∣Nw​∣1{(v,w)∈EN​}​≤1(A.19)

implies subcriticality. 3. Lemma B.2. If pˉ(N)=ω((ln⁡M(N)+ln⁡N)/N)\bar p(N) = \omega((\ln M(N) + \ln N)/N)pˉ​(N)=ω((lnM(N)+lnN)/N) and ∥p(N)−1∥2−2=ω(ln⁡N/M(N)2)\|p(N)^{-1}\|_2^{-2} = \omega(\ln N / M(N)^2)∥p(N)−1∥2−2​=ω(lnN/M(N)2), then {GN}\{G_N\}{GN​} satisfies the subcriticality condition almost surely.

The goal is the conjunction of Lemmas B.1 and B.2 under (3.5).

Significance

Combined with Theorem 3.4 of the paper, the goal says that JSQ(ddd) on a random compatibility graph with dispatcher-dependent flexibility behaves, in the large-system limit and in steady state, like JSQ(ddd) on the complete bipartite graph. Condition (3.5) allows the least flexible task type to be compatible with only about ln⁡(M(N)N)\ln(M(N)N)ln(M(N)N) times a diverging factor of servers in expectation, far below NNN, and allows strongly heterogeneous flexibility levels as long as the spread measured by ∑wpw(N)−2\sum_w p_w(N)^{-2}∑w​pw​(N)−2 is controlled. Remark 8 of the paper notes that for M(N)=NM(N) = NM(N)=N and equal probabilities, p(N)=o(ln⁡N/N)p(N) = o(\ln N / N)p(N)=o(lnN/N) leaves a dispatcher isolated with probability tending to one, so the growth conditions are close to necessary.

The result is proved in the paper (Appendix B). As far as is known, none of it has been machine-checked. A formal proof requires a quantitative Chernoff bound for binomial tails, a bounded-differences (Azuma–Hoeffding) inequality for functions of independent Bernoulli variables, a union bound over all 2N2^N2N subsets of servers, and the first Borel–Cantelli lemma in a setting where only per-NNN marginal laws are fixed.

Difficulty

Proportional sparsity quantifies over every subset UUU of the NNN servers. A concentration estimate for one fixed UUU, or one fixed dispatcher, is not enough: whatever bound is used must survive a union over 2N2^N2N subsets, so a merely vanishing failure probability per dispatcher does not suffice.

Subcriticality is a maximum over all servers of a sum whose terms 1{(v,w)∈EN}/∣Nw∣\mathbb 1\{(v, w) \in E_N\}/|N_w|1{(v,w)∈EN​}/∣Nw​∣ are not independent: ∣Nw∣|N_w|∣Nw​∣ depends on the edge (v,w)(v, w)(v,w) itself, and a term is unbounded in the regime where ∣Nw∣|N_w|∣Nw​∣ is small. A direct concentration inequality for this sum therefore does not apply, and the logarithmic factors in (3.5) reflect the need to control all M(N)M(N)M(N) neighbourhood sizes and all NNN servers at once.

Independence across NNN is not available, so almost-sure statements cannot use the second Borel–Cantelli lemma or any law of large numbers along NNN; every bound must be summable in NNN.

Formalization scope

  • Servers are Fin N, dispatchers Fin (M N) with 0 < M N assumed, and a graph is a Finset (Fin N × Fin (M N)).
  • Edge probabilities are reals with 0<pw(N)≤10 < p_w(N) \le 10<pw​(N)≤1 (implicit in the paper). The random graph is a family of measurable maps XN:Ω→{0,1}VN×WNX_N : \Omega \to \{0,1\}^{V_N \times W_N}XN​:Ω→{0,1}VN​×WN​ on one probability space whose image law is the product of Bernoulli(pw(N))(p_w(N))(pw​(N)) laws. No assumption links different NNN.
  • Every "lim sup⁡≤1\limsup \le 1limsup≤1" is stated as "for every ε>0\varepsilon > 0ε>0, eventually, for every server, ≤1+ε\le 1 + \varepsilon≤1+ε"; the supremum over UUU in (3.1) as "for all ε,δ>0\varepsilon, \delta > 0ε,δ>0, eventually, for every UUU, at most δM(N)\delta M(N)δM(N) bad dispatchers". Both are exact equivalents.
  • Ratios use real division; ∣Nw∣=0|N_w| = 0∣Nw​∣=0 gives ratio 000, harmless for a limit condition. At N=1N = 1N=1 the logarithms in (3.5) and Lemma B.2 can make a denominator 000; this affects finitely many NNN.
  • Proportional sparsity does not include connectivity: these graphs are disconnected with positive probability for every NNN, so requiring connectivity would make the goal false.
  • Subcriticality is Definition 3.3 with two repairs: distributions are required only for nonempty UUU, and the normalisation is (∣Nw∣min⁡(d,∣Nw∣))\binom{|N_w|}{\min(d, |N_w|)}(min(d,∣Nw​∣)∣Nw​∣​) instead of the printed (∣Nw∣d)\binom{|N_w|}{d}(d∣Nw​∣​), which is 000 when ∣Nw∣<d|N_w| < d∣Nw​∣<d, an event of positive probability here. The two agree when ∣Nw∣≥d|N_w| \ge d∣Nw​∣≥d.
  • d≥2d \ge 2d≥2 as in the model. Under d=0d = 0d=0 the subcriticality predicate would be trivially true; the statements exclude it.
  • Only the graph part of Theorem 3.6 is posed; "Consequently, the conclusions of Theorem 3.4 hold" belongs to the mean-field mission of this series.

A complete development needs binomial Chernoff bounds, a bounded-differences inequality and Borel–Cantelli for summable bounds; these are reusable for other random-graph statements. Proofs of the deterministic (A.19) reduction and of either lemma are welcome independently.

Selected references

  • M. Rutten, D. Mukherjee, Load Balancing Under Strict Compatibility Constraints, arXiv:2008.07562v2, 2020; Math. Oper. Res. 48 (2023). https://arxiv.org/abs/2008.07562
  • M. Bramson, Stability of join the shortest queue networks, Ann. Appl. Probab. 21(4):1568–1625, 2011. https://doi.org/10.1214/10-AAP726
  • A. Budhiraja, D. Mukherjee, R. Wu, Supermarket model on graphs, Ann. Appl. Probab. 29(3):1740–1777, 2019. https://arxiv.org/abs/1712.07607
  • E. Cardinaels, S. C. Borst, J. S. H. van Leeuwaarden, Job assignment in large-scale service systems with affinity relations, Queueing Syst. 93(3–4):227–268, 2019.
  • S. Janson, T. Łuczak, A. Ruciński, Random Graphs, Wiley, 2000. https://doi.org/10.1002/9781118032718
7 thms1 active userReviewed
AnalysisOptimization·Captain: mikedeng1

Lightning Network Economics: Channels III: The Minimal Cost of a Symmetric Bidirectional Channel Is 3(2Bλ/r)^{1/3} − B/6 + O(r^{1/3}) as r → 0Research Paper

Motivation

Payment channels let two parties make repeated transfers while using an on-chain transaction only when a channel needs resetting. They require funds to be committed in advance. The economic choice is therefore between the transaction fees saved by using a channel and the opportunity cost of funds tied up in it. Guasoni, Huberman, and Shikhelman give a stochastic cost model in which this tradeoff can be compared for one-way and two-way payment streams (Guasoni et al., 2023, §§1 and 3). Their small-discount-rate result makes the difference between the two arrangements quantitative: the optimal one-way cost grows like r−1/2r^{-1/2}r−1/2, while the optimal symmetric two-way cost grows like r−1/3r^{-1/3}r−1/3.

The paper was published online in Management Science on November 6, 2023. This mission concerns Theorem 3(i)–(ii) in that published version, especially the cost expansion (8). The paper proves these asymptotic claims; the task here is to state them precisely in Lean and develop machine-checked proofs of the stated cost laws.

Setting

Two nodes exchange unit payments. Payments from node 1 to node 2 arrive at rate λ1\lambda_1λ1​, and payments in the reverse direction arrive at rate λ2\lambda_2λ2​. Each payment stream is a Poisson process, and the streams are independent. The continuously compounded discount rate r>0r>0r>0 measures the opportunity cost of committed capital. A channel reset costs B>0B>0B>0, while an on-chain payment costs CCC. Node 1 commits l1l_1l1​ and node 2 commits l2l_2l2​. The channel balance moves by one unit at each payment; at a boundary, a reset replenishes it (Guasoni et al., 2023, §§1.1, 3.1 and 5).

The paper's closed-form bidirectional cost cl1,l2(λ1,λ2)c^{l_1,l_2}(\lambda_1,\lambda_2)cl1​,l2​(λ1​,λ2​) is given in equations (3)–(4). It uses two characteristic roots α−\alpha_-α−​ and α+\alpha_+α+​, determined by the payment rates and rrr. This mission uses the formula at equal rates λ1=λ2=λ>0\lambda_1=\lambda_2=\lambda>0λ1​=λ2​=λ>0. A separate unidirectional cost c0,m(0,λ)c^{0,m}(0,\lambda)c0,m(0,λ) from equation (1) describes a stream in only one direction. Its optimizing size is denoted mmm below. The exact channel model has integer balance states; the paper optimizes the closed forms over positive real channel sizes in its asymptotic analysis.

Write copt(0,λ)c^{\mathrm{opt}}(0,\lambda)copt(0,λ) for the infimum of the unidirectional cost over m>0m>0m>0, and copt(λ,λ)c^{\mathrm{opt}}(\lambda,\lambda)copt(λ,λ) for the infimum of the symmetric bidirectional cost over l1,l2>0l_1,l_2>0l1​,l2​>0. These are functions of rrr and BBB as well as of λ\lambdaλ; the compact paper notation suppresses those arguments. The term O(ra)O(r^a)O(ra) has its usual uniform meaning as r↓0r\downarrow0r↓0 with BBB and λ\lambdaλ fixed: its absolute value is at most a constant times rar^ara for all sufficiently small positive rrr.

Formalization targets

Unidirectional benchmark

Theorem 3(i), equation (7), states

copt(0,λ)=2(Bλr)1/2−B2+O(r1/2).c^{\mathrm{opt}}(0,\lambda) =2\left(\frac{B\lambda}{r}\right)^{1/2}-\frac B2+O(r^{1/2}).copt(0,λ)=2(rBλ​)1/2−2B​+O(r1/2).

The same part states that an optimal size is (Bλ/r)1/2+O(r1/2)(B\lambda/r)^{1/2}+O(r^{1/2})(Bλ/r)1/2+O(r1/2). The mission records that size claim using the explicit optimizer displayed after equation (2) (Guasoni et al., 2023, pp. 6–7).

Symmetric bidirectional goal

The mission's goal is the cost clause of Theorem 3(ii), equation (8):

copt(λ,λ)=3(2Bλr)1/3−B6+O(r1/3).c^{\mathrm{opt}}(\lambda,\lambda) =3\left(\frac{2B\lambda}{r}\right)^{1/3}-\frac B6+O(r^{1/3}).copt(λ,λ)=3(r2Bλ​)1/3−6B​+O(r1/3).

This includes the constant correction −B/6-B/6−B/6 and a two-sided error estimate of order r1/3r^{1/3}r1/3. The theorem also states that optimal commitments are equal and each has size (2Bλ/r)1/3+O(r1/3)(2B\lambda/r)^{1/3}+O(r^{1/3})(2Bλ/r)1/3+O(r1/3). That separate size clause is outside this proposal; the goal does not assert the existence or location of a minimizing pair (Guasoni et al., 2023, p. 7).

Significance

The different exponents distinguish the two designs even when both can make frequent off-chain payments. At small rrr, the model's optimal symmetric bidirectional cost grows more slowly than its optimal unidirectional cost. The formulas also retain the next constant term, allowing a comparison beyond the leading growth rates. The paper relates this difference to the amount of capital that must be committed by the participants (Guasoni et al., 2023, p. 7).

A formal proof would connect the exact channel-cost formulas, a two-variable infimum, the diagonal specialization, bounds on the hyperbolic cosecant, and real asymptotics in one reusable development. The supporting statements are useful separately: the root expansion concerns a quadratic characteristic equation, and the diagonal reduction concerns a symmetric optimization problem. The paper supplies proofs of the mathematical results; the remaining work is their faithful machine-checked formalization.

Difficulty

The cost depends on rrr and on sizes that grow without bound as rrr approaches zero. An expansion valid for each fixed size does not by itself justify substituting a size that varies with rrr. The target asks for a uniform remainder at the infimum, including its constant term. The cost must be controlled from both sides over every admissible positive size; evaluating one promising size supplies only one side of that comparison. The paper's proof of Theorem 3 uses formal small-rrr expansions of the cost and first-order conditions, and the stated error rate calls for care when these are applied to moving optimizers (Guasoni et al., 2023, pp. 11–12).

Formalization scope

The Lean definitions reproduce equations (1), (3), and (4). The one-way and two-way costs are separate functions, matching the paper's separate treatment of the unidirectional boundary case. Real powers are used for the real-size optimization; the balance lattice in the exact stochastic model uses integer sizes. Both optimal costs are infima over strictly positive real sizes. The symmetric infimum is not defined by choosing a particular size or by assuming that the minimum is attained. All rates in the asymptotic theorems satisfy λ>0\lambda>0λ>0, the reset cost satisfies B>0B>0B>0, and the filter sends rrr to zero through positive values. Consequently a Big-O constant may depend on BBB and λ\lambdaλ but cannot depend on rrr.

The on-chain costs Cλ/rC\lambda/rCλ/r for one stream and 2Cλ/r2C\lambda/r2Cλ/r for two equal streams supply the paper's economic comparison, although CCC does not occur in this mission's cost goal. The diagonal milestone omits that on-chain term because it is constant in the channel sizes; it states a midpoint cost inequality so no minimizer is assumed. The proof's root expansion is recorded for both α+\alpha_+α+​ and α−\alpha_-α−​. The paper's assertion that the symmetric minimizing sizes equal (2Bλ/r)1/3+O(r1/3)(2B\lambda/r)^{1/3}+O(r^{1/3})(2Bλ/r)1/3+O(r1/3) is explicitly omitted. The development does not fix a numerical value of rrr, permit an rrr-dependent Big-O constant, drop the −B/6-B/6−B/6 term, or replace the infimum by cost at a chosen size.

The mission needs real-power identities, properties of sinh⁡\sinhsinh and cosh⁡\coshcosh, nonempty bounded-below cost sets, infimum estimates, and Mathlib's Asymptotics.IsBigO on the positive-side neighborhood of zero. Results about real asymptotics and symmetric minimization can be reused beyond this channel model. Contributions proving the cost milestones, the root expansion, the diagonal inequality, and the goal are all within scope.

Selected references

  • Guasoni, P., Huberman, G., and Shikhelman, Y., Lightning Network Economics: Channels, Management Science, Articles in Advance, November 6, 2023, pp. 1–14. DOI: 10.1287/mnsc.2022.01664.
14 thms1 active userReviewed
Graph TheoryProbability·Captain: mikedeng1

Load Balancing Under Strict Compatibility Constraints 2: Random Graphs with Server Degree c(N) → ∞ and Nc(N)/(M(N) ln N) → ∞ Are Almost Surely Proportionally Sparse and SubcriticalResearch Paper

Motivation

In large data centres, an incoming task often cannot run on every server: its data, software or licences sit on only some of the machines. Such strict compatibility constraints are modelled by a bipartite graph between task types (dispatchers) and servers. For the fully flexible system, in which every task can go to every server, the JSQ(ddd) policy (sample ddd servers at random and join the shortest queue) is described in the large-system limit by the mean-field ODE of Vvedenskaya, Dobrushin and Karpelevich (1996) and Mitzenmacher (1996). Rutten and Mukherjee (arXiv:2008.07562, Math. Oper. Res. 2023) identify two graph conditions, proportional sparsity and subcriticality, under which JSQ(ddd) on a sparse compatibility graph has the same mean-field limit and the same steady state as the fully flexible system (their Theorem 3.4).

The two conditions are useful in system design only if there are natural graph families that satisfy them. A storage designer controls the server degree: the number of task types a server can process is roughly proportional to the data it must store. Theorem 3.5 of the paper answers this design question for random graphs. If every server stores data for c(N)c(N)c(N) task types chosen uniformly at random, then c(N)→∞c(N)\to\inftyc(N)→∞ together with a mild density condition suffices, even when c(N)c(N)c(N) is negligible compared with the number M(N)M(N)M(N) of task types. This mission formalizes Theorem 3.5's graph-theoretic content and the chain of lemmas in Appendix A that proves it.

Setting

A system with NNN servers VN={1,…,N}V_N=\{1,\dots,N\}VN​={1,…,N} and M(N)M(N)M(N) dispatchers WN={1,…,M(N)}W_N=\{1,\dots,M(N)\}WN​={1,…,M(N)} has a compatibility graph GN=(VN,WN,EN)G_N=(V_N,W_N,E_N)GN​=(VN​,WN​,EN​) with EN⊆VN×WNE_N\subseteq V_N\times W_NEN​⊆VN​×WN​. The neighbourhood of a dispatcher www is Nw={v:(v,w)∈EN}\mathcal N_w=\{v:(v,w)\in E_N\}Nw​={v:(v,w)∈EN​}; that of a server vvv is Nv={w:(v,w)∈EN}\mathcal N_v=\{w:(v,w)\in E_N\}Nv​={w:(v,w)∈EN​}.

Proportional sparsity (Definition 3.1): for every ε>0\varepsilon>0ε>0,

sup⁡U⊆VN∣{w∈WN:∣∣Nw∩U∣∣Nw∣−∣U∣N∣≥ε}∣/M(N)→0.\sup_{U\subseteq V_N}\Big|\Big\{w\in W_N:\Big|\tfrac{|\mathcal N_w\cap U|}{|\mathcal N_w|}-\tfrac{|U|}{N}\Big|\ge\varepsilon\Big\}\Big|\Big/M(N)\to0 .U⊆VN​sup​​{w∈WN​:​∣Nw​∣∣Nw​∩U∣​−N∣U∣​​≥ε}​/M(N)→0.

In words, for every set UUU of servers, all but a vanishing fraction of the dispatchers see UUU in roughly the proportion ∣U∣/N|U|/N∣U∣/N.

Subcriticality (Definition 3.3) with sample size d≥2d\ge2d≥2: there are probability distributions γwU\gamma^U_wγwU​ on the sets U⊆VNU\subseteq V_NU⊆VN​ such that

lim sup⁡N→∞max⁡v∈VNNM(N)∑w∈WN(∣Nw∣kw)−1∑U⊆Nw, ∣U∣=kwγwU(v)≤1,kw=min⁡(d,∣Nw∣).\limsup_{N\to\infty}\max_{v\in V_N}\frac{N}{M(N)}\sum_{w\in W_N}\binom{|\mathcal N_w|}{k_w}^{-1}\sum_{U\subseteq\mathcal N_w,\ |U|=k_w}\gamma^U_w(v)\le1,\qquad k_w=\min(d,|\mathcal N_w|).N→∞limsup​v∈VN​max​M(N)N​w∈WN​∑​(kw​∣Nw​∣​)−1U⊆Nw​, ∣U∣=kw​∑​γwU​(v)≤1,kw​=min(d,∣Nw​∣).

Think of a static policy that routes a task arriving at www to a server in a sampled set UUU according to γwU\gamma^U_wγwU​. The condition says that no server receives more than its fair share of load. Its uniform-weight special case is condition (A.19):

lim sup⁡N→∞max⁡v∈VNNM(N)∑w∈WN1{(v,w)∈EN}∣Nw∣≤1.\limsup_{N\to\infty}\max_{v\in V_N}\frac{N}{M(N)}\sum_{w\in W_N}\frac{\mathbb 1\{(v,w)\in E_N\}}{|\mathcal N_w|}\le1 .N→∞limsup​v∈VN​max​M(N)N​w∈WN​∑​∣Nw​∣1{(v,w)∈EN​}​≤1.

The random graph. Let 1≤c(N)≤M(N)1\le c(N)\le M(N)1≤c(N)≤M(N). For each NNN, each server independently chooses its neighbourhood Nv\mathcal N_vNv​ uniformly among the c(N)c(N)c(N)-subsets of WNW_NWN​, so every server has degree exactly c(N)c(N)c(N). All graphs live on one probability space, but only the law of each GNG_NGN​ is prescribed; the joint law across different NNN is arbitrary.

Formalization targets

Goal: Theorem 3.5 (graph part)

If c(N)→∞c(N)\to\inftyc(N)→∞ and Nc(N)/(M(N)ln⁡N)→∞Nc(N)/(M(N)\ln N)\to\inftyNc(N)/(M(N)lnN)→∞, then almost surely

{GN}N≥1 is proportionally sparseand{GN}N≥1 satisfies the subcriticality condition.\{G_N\}_{N\ge1}\ \text{is proportionally sparse}\quad\text{and}\quad\{G_N\}_{N\ge1}\ \text{satisfies the subcriticality condition.}{GN​}N≥1​ is proportionally sparseand{GN​}N≥1​ satisfies the subcriticality condition.

Milestones, in the order of the proof in Appendix A

  1. Lemma A.1. If XN∼Bin(N,p(N))X_N\sim\mathrm{Bin}(N,p(N))XN​∼Bin(N,p(N)) and p(N)→0p(N)\to0p(N)→0, then for each δ∈(0,1/2)\delta\in(0,1/2)δ∈(0,1/2) there are N0≥1N_0\ge1N0​≥1 and a>0a>0a>0 with P(XN≥δN)≤(a p(N)δ)N\mathbb P(X_N\ge\delta N)\le(a\,p(N)^\delta)^NP(XN​≥δN)≤(ap(N)δ)N for N≥N0N\ge N_0N≥N0​.
  2. Lemma A.2. Let BN(X)B_N(X)BN​(X) be the set of servers whose number of neighbours in XXX deviates from c(N)∣X∣/M(N)c(N)|X|/M(N)c(N)∣X∣/M(N) by at least εc(N)∣X∣/M(N)\varepsilon c(N)|X|/M(N)εc(N)∣X∣/M(N). If c(N)→∞c(N)\to\inftyc(N)→∞ and Nc(N)/M(N)→∞Nc(N)/M(N)\to\inftyNc(N)/M(N)→∞, then for 0<ε≤3/20<\varepsilon\le3/20<ε≤3/2, 0<δ<1/20<\delta<1/20<δ<1/2 and η>0\eta>0η>0, almost surely eventually ∣BN(X)∣≤δN|B_N(X)|\le\delta N∣BN​(X)∣≤δN for all XXX with ∣X∣≥ηM(N)|X|\ge\eta M(N)∣X∣≥ηM(N).
  3. Lemma A.3. Under the same growth conditions, almost surely eventually at most δM(N)\delta M(N)δM(N) dispatchers have ∣Nw∣≤(1−ε)Nc(N)/M(N)|\mathcal N_w|\le(1-\varepsilon)Nc(N)/M(N)∣Nw​∣≤(1−ε)Nc(N)/M(N).
  4. Proposition A.4. Under the same growth conditions, {GN}\{G_N\}{GN​} is proportionally sparse almost surely.
  5. (A.19) reduction (p. 34). For any graph sequence, (A.19) implies subcriticality.
  6. Lemma A.5. If Nc(N)/(M(N)ln⁡N)→∞Nc(N)/(M(N)\ln N)\to\inftyNc(N)/(M(N)lnN)→∞, then {GN}\{G_N\}{GN​} satisfies (A.19) almost surely.

Significance

The result. Theorem 3.5 shows that the performance of the fully flexible JSQ(ddd) system, in the mean-field limit and in steady state, can be achieved with a server degree c(N)c(N)c(N) that grows arbitrarily slowly. The only requirement is that each dispatcher's mean degree Nc(N)/M(N)Nc(N)/M(N)Nc(N)/M(N) grow faster than ln⁡N\ln NlnN. Each server then needs to store data for only c(N)c(N)c(N) task types, a vanishing fraction of the M(N)M(N)M(N) types. The ln⁡N\ln NlnN threshold matches the scale below which some dispatchers have too few neighbours to share the load evenly.

Formalizing it. The result is proved in the paper, and no part of it has been machine-checked. The mission produces reusable statements about random bipartite graphs with fixed left degree: hypergeometric concentration over all large dispatcher sets, the degree lower tail, and a Borel–Cantelli argument without independence across NNN. It also produces a deterministic reduction from subcriticality to (A.19). That reduction is shared with Theorem 3.6 of the same paper (inhomogeneous random graphs). The printed proof of Lemma A.5 has a gap: (A.23) bounds the probability that some dispatcher has low degree by N1−a2/2N^{1-a^2/2}N1−a2/2, but the union over M(N)M(N)M(N) dispatchers gives M(N)N−a2/2M(N)N^{-a^2/2}M(N)N−a2/2. This is not summable when M(N)M(N)M(N) grows faster than every polynomial, which the hypotheses allow. A complete formal proof must close this gap or show that low-degree dispatchers contribute too little to matter.

Difficulty

Every statement is almost sure along the whole sequence, but the graphs for different NNN may be coupled arbitrarily, so no strong law over NNN is available. Each a.s. claim needs a failure probability that is summable in NNN, followed by the first Borel–Cantelli lemma. Proportional sparsity quantifies over all 2N2^N2N server sets UUU, and Lemma A.2 over all 2M(N)2^{M(N)}2M(N) dispatcher sets XXX. The obvious argument would concentrate each ∣Nw∩U∣|\mathcal N_w\cap U|∣Nw​∩U∣ separately and take a union bound. It fails because ∣Nw∣|\mathcal N_w|∣Nw​∣ is only of order Nc(N)/M(N)Nc(N)/M(N)Nc(N)/M(N), which may be far smaller than NNN and M(N)M(N)M(N), so per-dispatcher tails cannot absorb an eNe^{N}eN or eM(N)e^{M(N)}eM(N) union. Uniform control over all large sets XXX (Lemma A.2) is therefore the crux. For subcriticality, the difficulty is that the maximum over vvv runs over NNN servers while the minimum degree runs over M(N)M(N)M(N) dispatchers.

Formalization scope

  • Servers are Fin N, dispatchers Fin (M N); a graph is a Finset (Fin N × Fin (M N)), and a random graph is a measurable random adjacency matrix Fin N → Fin (M N) → Bool (the space {0,1}N×M(N)\{0,1\}^{N\times M(N)}{0,1}N×M(N) of p. 13).
  • The law of each random matrix is prescribed pointwise: every matrix with all row sums equal to c(N)c(N)c(N) has probability (M(N)c(N))−N\binom{M(N)}{c(N)}^{-N}(c(N)M(N)​)−N. This is the uniform, independent-over-servers choice. No independence across NNN is assumed. A formalization that assumes independent graphs for different NNN proves a weaker theorem.
  • Hypotheses 1≤c(N)≤M(N)1\le c(N)\le M(N)1≤c(N)≤M(N) for every NNN, d≥2d\ge2d≥2, and the growth conditions as real limits (Tendsto … atTop atTop). ln⁡N\ln NlnN is Real.log, so the ratio is 000 at N=1N=1N=1, which does not affect a limit.
  • Proportional sparsity is the limit condition (3.1) as "for every ε,δ>0\varepsilon,\delta>0ε,δ>0, eventually, for every UUU, at most δM(N)\delta M(N)δM(N) bad dispatchers". The connectivity in Definition 3.1 is not included, because the random graphs are disconnected for small NNN. A dispatcher with ∣Nw∣=0|\mathcal N_w|=0∣Nw​∣=0 gives the ratio 0/0=00/0=00/0=0 in Lean.
  • Subcriticality constrains γwU\gamma^U_wγwU​ only for nonempty UUU and normalises by (∣Nw∣min⁡(d,∣Nw∣))\binom{|\mathcal N_w|}{\min(d,|\mathcal N_w|)}(min(d,∣Nw​∣)∣Nw​∣​). The paper's (∣Nw∣d)\binom{|\mathcal N_w|}{d}(d∣Nw​∣​) is 1/01/01/0 when ∣Nw∣<d|\mathcal N_w|<d∣Nw​∣<d, and the empty UUU admits no distribution. With the literal definition the goal would be false. Both "lim sup⁡≤1\limsup\le1limsup≤1" conditions are written as "for every ε>0\varepsilon>0ε>0, eventually at most 1+ε1+\varepsilon1+ε".
  • Not formalized: the sentence "Consequently, the conclusions of Theorem 3.4 hold", which composes this mission's goal with the mean-field mission of this series.
  • Infrastructure that is welcome and reusable: hypergeometric and binomial Chernoff bounds in a form usable with Finset.powersetCard, and Borel–Cantelli for eventually-uniform statements over families of finite sets.

Selected references

  • D. Rutten and D. Mukherjee, Load Balancing Under Strict Compatibility Constraints, Math. Oper. Res., 2023; arXiv:2008.07562v2. https://arxiv.org/abs/2008.07562
  • A. Budhiraja, D. Mukherjee and R. Wu, Supermarket model on graphs, Ann. Appl. Probab. 29(3):1740–1777, 2019. https://doi.org/10.1214/18-AAP1437
  • M. Bramson, Stability of join the shortest queue networks, Ann. Appl. Probab. 21(4):1568–1625, 2011. https://doi.org/10.1214/10-AAP726
  • S. Janson, T. Łuczak and A. Ruciński, Random Graphs, Wiley, 2000. https://doi.org/10.1002/9781118032718
  • N. D. Vvedenskaya, R. L. Dobrushin and F. I. Karpelevich, Queueing system with selection of the shortest of two queues: an asymptotic approach, Probl. Inf. Transm. 32(1):15–27, 1996.
  • M. Mitzenmacher, The power of two choices in randomized load balancing, IEEE Trans. Parallel Distrib. Syst. 12(10):1094–1104, 2001. https://doi.org/10.1109/71.963420
10 thms1 active userReviewed
EconomicsOptimization·Captain: mikedeng1

Revenue Management in Crowdfunding 2: Optimal Pledge Level and Campaign Duration When Each Backer Type Shares One Valuation and Herding Is WeakResearch Paper

Motivation

On reward-based crowdfunding platforms such as Kickstarter, a project creator fixes two numbers before launch: the pledge level ppp, the amount a backer commits, and the campaign duration TTT. Both are set once and cannot be revised. A higher pledge earns more per backer but turns away backers whose valuation falls short, and a longer campaign reaches more backers but makes them wait longer for a project they value less the further away its delivery is. Zhang, Savin and Veeraraghavan (Revenue Management in Crowdfunding, Manufacturing & Service Operations Management, 2022; manuscript SSRN 3065267) model this trade-off with two kinds of backers. Independent backers decide on their own valuation. Herding backers arrive in proportion to the pledges already collected, so early success attracts more pledges. The paper asks when the creator should exploit herding and when it should ignore it.

This mission formalizes the paper's answer in the simplest setting it treats: every independent backer values the project at the same viv_ivi​, and every herding backer at the same vhv_hvh​.

Setting

A campaign has a population I^>0\hat I>0I^>0 of independent backers, who arrive at rate k>0k>0k>0 times the number still to come. Every backer discounts the project's value at rate r>0r>0r>0: a backer arriving at time ttt with valuation vvv pledges if and only if ve−r(T−t)≥pv e^{-r(T-t)}\ge pve−r(T−t)≥p. Herding backers arrive at time ttt at rate Λhqp\Lambda_h q pΛh​qp times the number of pledges so far, where Λh≥0\Lambda_h\ge0Λh​≥0 is an arrival rate and q≥0q\ge0q≥0 a propensity to pledge, and they pledge under the same discounted-value rule. Valuations lie in [0,vˉ][0,\bar v][0,vˉ] and enter only through the survival functions Fˉi=1−Fi\bar F_i=1-F_iFˉi​=1−Fi​ and Fˉh=1−Fh\bar F_h=1-F_hFˉh​=1−Fh​ of the two valuation laws.

The paper solves the pledging dynamics in closed form (its Propositions 1 and 2). The numbers of pledging independent and herding backers at time ttt are

nip(t,p,T)=kI^∫0te−kxFˉi(per(T−x)) dx,n^p_i(t,p,T)=k\hat I\int_0^t e^{-kx}\bar F_i\big(pe^{r(T-x)}\big)\,dx,nip​(t,p,T)=kI^∫0t​e−kxFˉi​(per(T−x))dx, nhp(t,p,T)=Λhqp kI^∫0teΛhqp∫τtFˉh(per(T−u))du Fˉh(per(T−τ))∫0τe−kxFˉi(per(T−x))dx dτ.n^p_h(t,p,T)=\Lambda_hqp\,k\hat I\int_0^t e^{\Lambda_hqp\int_\tau^t\bar F_h(pe^{r(T-u)})du}\,\bar F_h\big(pe^{r(T-\tau)}\big)\int_0^\tau e^{-kx}\bar F_i\big(pe^{r(T-x)}\big)dx\,d\tau .nhp​(t,p,T)=Λh​qpkI^∫0t​eΛh​qp∫τt​Fˉh​(per(T−u))duFˉh​(per(T−τ))∫0τ​e−kxFˉi​(per(T−x))dxdτ.

The total number of pledges is n(p,T)=nip(T,p,T)+nhp(T,p,T)n(p,T)=n^p_i(T,p,T)+n^p_h(T,p,T)n(p,T)=nip​(T,p,T)+nhp​(T,p,T), and the creator's problem (18) is

max⁡ π(p,T)=p n(p,T)over0<p≤vˉ,  0≤T≤1rln⁡(vˉ/p).\max\ \pi(p,T)=p\,n(p,T)\quad\text{over}\quad 0<p\le\bar v,\ \ 0\le T\le\tfrac1r\ln(\bar v/p).max π(p,T)=pn(p,T)over0<p≤vˉ,  0≤T≤r1​ln(vˉ/p).

With identical valuations, Fˉi(y)=1\bar F_i(y)=1Fˉi​(y)=1 for y<viy<v_iy<vi​ and 000 for y≥viy\ge v_iy≥vi​, and likewise Fˉh\bar F_hFˉh​ with vhv_hvh​, where 0<vi,vh≤vˉ0<v_i,v_h\le\bar v0<vi​,vh​≤vˉ. Proposition 6 adds the constraint T≤1rln⁡(vi/p)T\le\frac1r\ln(v_i/p)T≤r1​ln(vi​/p), under which every independent backer pledges from the start.

Formalization targets

Goal: Proposition 6(ii)

Let Λh,q>0\Lambda_h,q>0Λh​,q>0, vi>vhv_i>v_hvi​>vh​, and set δ=Λhqvh/r\delta=\Lambda_hqv_h/rδ=Λh​qvh​/r, a=k/(Λhq)a=k/(\Lambda_hq)a=k/(Λh​q), b=Λhq/rb=\Lambda_hq/rb=Λh​q/r and

X=ebvh/e14a(krvhe−2+0.5 bvh2e−3+2a+vh)+1a(kr(vh+a)+a).X=e^{bv_h/e}\frac1{4a}\Big(\frac kr v_he^{-2}+0.5\,bv_h^2e^{-3}+2a+v_h\Big)+\frac1a\Big(\frac kr(v_h+a)+a\Big).X=ebvh​/e4a1​(rk​vh​e−2+0.5bvh2​e−3+2a+vh​)+a1​(rk​(vh​+a)+a).

If δ<1\delta<1δ<1 and vi/vh>(X/(1−δ))r/kv_i/v_h>(X/(1-\delta))^{r/k}vi​/vh​>(X/(1−δ))r/k, then on Proposition 6's feasible set the unique maximizer of π\piπ is

p∗=vi(rk+r)r/k,T∗=1rln⁡vip∗.p^*=v_i\Big(\frac r{k+r}\Big)^{r/k},\qquad T^*=\frac1r\ln\frac{v_i}{p^*}.p∗=vi​(k+rr​)r/k,T∗=r1​lnp∗vi​​.

Milestones

  1. Corollary 1. If 0<p≤min⁡(vie−rT,vh)0<p\le\min(v_ie^{-rT},v_h)0<p≤min(vi​e−rT,vh​), then for t∈[0,T]t\in[0,T]t∈[0,T], nip(t,p,T)=I^(1−e−kt)n^p_i(t,p,T)=\hat I(1-e^{-kt})nip​(t,p,T)=I^(1−e−kt) and nhpn^p_hnhp​ equals the explicit formula (15).
  2. Proposition 3(b). If Λhqvh/r<1\Lambda_hqv_h/r<1Λh​qvh​/r<1 and k/r>2k/r>2k/r>2, then for every sufficiently long fixed duration TTT, the unique optimal pledge level on (18)'s range is p∗(T)=vie−rTp^*(T)=v_ie^{-rT}p∗(T)=vi​e−rT.
  3. Proposition 6(i). If vi≤vhv_i\le v_hvi​≤vh​, an optimal campaign exists on Proposition 6's feasible set, and every optimum has p∗≥vi/ep^*\ge v_i/ep∗≥vi​/e and T∗≤1/rT^*\le 1/rT∗≤1/r.

Significance

Proposition 6 separates two regimes. When herding backers value the project at least as much as independent ones, the optimal campaign is short, T∗≤1/rT^*\le1/rT∗≤1/r, and its pledge is at least vi/ev_i/evi​/e. When herding is weak (δ<1\delta<1δ<1) and independent backers value the project enough more than herding ones, the creator's best response is to price herding backers out entirely and run the campaign that would be optimal if no herding backer existed. The threshold (33) quantifies "enough more". Together with Proposition 3(b), which fixes the duration and finds that the optimal pledge sits exactly at the highest level that keeps every independent backer in from the start, these results give a creator explicit rules rather than a numerical optimization.

The paper states that the proofs of Propositions 3 to 7 are in an online Technical Appendix, which is not part of the manuscript this mission cites; the manuscript's own Appendix proves only Propositions 1 and 2 and Corollaries 1 and 2. None of the results has a machine-checked proof. Formalizing Proposition 6(ii) would supply a verified proof of a result whose published proof is not in the source held here, and would check the threshold (33), whose grouping of terms is read off the typeset page.

Difficulty

The objective π\piπ is defined through nested integrals of step functions, so it is not differentiable everywhere: the jumps of Fˉi\bar F_iFˉi​ and Fˉh\bar F_hFˉh​ put kinks in π\piπ where p=vie−rTp=v_ie^{-rT}p=vi​e−rT and where p=vhe−rTp=v_he^{-rT}p=vh​e−rT or p=vhp=v_hp=vh​, at which a backer type starts or stops pledging during the campaign. Setting a gradient to zero does not identify the optimum. The obvious argument for the goal, that herding backers do not pledge at p∗>vhp^*>v_hp∗>vh​ and so the no-herding optimum is optimal, compares only campaigns that exclude herding backers; it says nothing about the campaigns with a lower pledge level that do attract them. Those campaigns gain a herding term growing exponentially in Λhqp\Lambda_hqpΛh​qp times the time during which herding backers pledge, and the comparison must hold uniformly over the whole feasible region, which is what the threshold (33) is for. Uniqueness, not only optimality, must also be shown.

Formalization scope

All objects live in the namespace CrowdfundRM.Identical. Parameters form a structure Campaign with fields Ihat, k, r, Lh (for Λh\Lambda_hΛh​), q, vbar, Fbi, Fbh. The standing assumptions, implicit in the paper, are I^,k,r,vˉ>0\hat I,k,r,\bar v>0I^,k,r,vˉ>0, Λh,q≥0\Lambda_h,q\ge0Λh​,q≥0, and that each survival function is nonincreasing, takes values in [0,1][0,1][0,1], equals 111 below 000 and 000 from vˉ\bar vvˉ on. Identical valuations set Fˉi\bar F_iFˉi​, Fˉh\bar F_hFˉh​ to the step functions above with 0<vi,vh≤vˉ0<v_i,v_h\le\bar v0<vi​,vh​≤vˉ.

Conventions: real numbers throughout; the pledge counts are the closed forms (5) and (11); the feasible set of (18) adds p>0p>0p>0, which ln⁡(vˉ/p)\ln(\bar v/p)ln(vˉ/p) requires; "the optimal" is the unique maximizer over the stated feasible set, and Proposition 6(i), which does not claim uniqueness, asserts that a maximizer exists and bounds every maximizer; "sufficiently long" in Proposition 3(b) is an unspecified threshold T0>0T_0>0T0​>0; Λh,q>0\Lambda_h,q>0Λh​,q>0 is added in Proposition 6(ii) because aaa and bbb divide by Λhq\Lambda_hqΛh​q. The paper's T∗=1rln⁡(vi/pˉ)T^*=\frac1r\ln(v_i/\bar p)T∗=r1​ln(vi​/pˉ​) has an undefined pˉ\bar ppˉ​ and is read as p∗p^*p∗. In (33) the factors ebvh/ee^{bv_h/e}ebvh​/e and 1/(4a)1/(4a)1/(4a) multiply only the first bracket.

The following formalizations would trivialize the targets and are ruled out: defining n(p,T)n(p,T)n(p,T) by the explicit formulas (14)–(15) or by the paper's single formula (17) instead of (5) and (11); stating optimality through sSup or a first-order condition; stating Proposition 6(i) for maximizers without asserting that one exists; and stating Proposition 3(b) with a chosen threshold or with Proposition 6's narrower feasible set. The pledging differential equations themselves and their solution (Propositions 1–2) are formalized in the companion mission on triangular valuations; this mission works from their closed forms.

Infrastructure needed: interval integrals of monotone step functions, the exponential, logarithm and real power from Mathlib. Explicit evaluation of the nested integrals under step laws (Corollary 1) is reusable for any proof in this model. Contributions are welcome on each milestone and on lemmas such as continuity of π\piπ on the feasible set and the closed form of π\piπ in each region of (p,T)(p,T)(p,T).

Selected references

  • J. Zhang, S. Savin, S. Veeraraghavan, Revenue Management in Crowdfunding, Manufacturing & Service Operations Management, 2022. https://doi.org/10.1287/msom.2022.1147. Cited from the authors' manuscript of May 26, 2021, SSRN 3065267: https://ssrn.com/abstract=3065267
6 thms1 active userReviewed
CombinatoricsProbability·Captain: mikedeng1

Secretary Problems with Biased Evaluations Using Partial Ordinal Information II: No Algorithm Beats Competitive Ratio ω on Posets of Width ωResearch Paper

Motivation

In hiring, admissions and procurement, candidates are often judged by scores that are known to be biased: two candidates from different groups, schools or evaluation pipelines cannot be compared by their observed scores, while two candidates from the same pipeline can. Salem and Gupta (SSRN 3444283, published in Management Science, 2023) model this situation as a poset secretary problem. The decision maker sees only a partial order between candidates, namely the comparisons that are trustworthy, and must still make irrevocable online selections.

The classical secretary problem and its kkk-selection variant (Kleinberg, 2005) assume a total order. With only partial comparisons, the natural question is how much the missing comparisons cost. Salem and Gupta answer it in terms of the width of the poset. This mission formalizes their lower bound: no algorithm can do better than a factor of the width.

Setting

A finite set C\mathcal CC of NNN candidates carries a partial order ⪯\preceq⪯. Each candidate aaa has a true score w(a)w(a)w(a). The scores are nonnegative and pairwise distinct, and they are consistent with the order: a≺ba\prec ba≺b implies w(a)<w(b)w(a)<w(b)w(a)<w(b). The pair I=(P,w)\mathcal I=(\mathcal P,w)I=(P,w) is an instance. Candidates arrive in uniformly random order. An algorithm knows NNN and a capacity kkk, observes the comparisons of P\mathcal PP among the candidates that have arrived, and selects at most kkk candidates, irrevocably. It never observes the scores.

The offline optimum is

OPT(w,k)=max⁡{∑a∈Sw(a)  :  S⊆C, ∣S∣≤k}.\mathrm{OPT}(w,k)=\max\Big\{\textstyle\sum_{a\in S}w(a)\;:\;S\subseteq\mathcal C,\ |S|\le k\Big\}.OPT(w,k)=max{∑a∈S​w(a):S⊆C, ∣S∣≤k}.

An algorithm is α\alphaα-competitive if E[w(Alg(I))]≥OPT(I)/α\mathbb E[w(\mathsf{Alg}(\mathcal I))]\ge\mathrm{OPT}(\mathcal I)/\alphaE[w(Alg(I))]≥OPT(I)/α on every instance (Definition 3.4).

The width ω\omegaω of P\mathcal PP is the largest number of pairwise incomparable candidates (an antichain).

Because an algorithm sees only the poset and the arrival order, on a fixed poset its randomness (arrival order and internal coins) induces a probability distribution μ\muμ on the selected set, supported on sets of at most kkk elements, and μ\muμ does not depend on the scores. Such a μ\muμ is called a selection rule with capacity kkk. Its expected score is Eμ[w(Alg)]=∑Sμ(S)∑a∈Sw(a)\mathbb E_\mu[w(\mathrm{Alg})]=\sum_S\mu(S)\sum_{a\in S}w(a)Eμ​[w(Alg)]=∑S​μ(S)∑a∈S​w(a). The selection probability of aaa is p(a)=∑S∋aμ(S)p(a)=\sum_{S\ni a}\mu(S)p(a)=∑S∋a​μ(S).

The instance of the lower bound is Chains(ω,k)\mathrm{Chains}(\omega,k)Chains(ω,k): ω\omegaω disjoint chains of kkk elements each, with elements in different chains incomparable (Figure 8 of the paper, with n=ωkn=\omega kn=ωk).

Formalization targets

Goal: Proposition 5.1 (p. 14, proof in Appendix F, p. 42)

For every k≥1k\ge1k≥1 and ω≥1\omega\ge1ω≥1 there is a poset P\mathcal PP of width ω\omegaω such that for every selection rule μ\muμ with capacity kkk and every ε>0\varepsilon>0ε>0 there is a consistent scoring www with OPT(w,k)>0\mathrm{OPT}(w,k)>0OPT(w,k)>0 and

Eμ[w(Alg)]≤(1+ε) OPT(w,k)ω.\mathbb E_\mu[w(\mathrm{Alg})]\le\frac{(1+\varepsilon)\,\mathrm{OPT}(w,k)}{\omega}.Eμ​[w(Alg)]≤ω(1+ε)OPT(w,k)​.

Equivalently, for every algorithm, sup⁡IOPT(I)/E[w(Alg(I))]≥ω\sup_{\mathcal I}\mathrm{OPT}(\mathcal I)/\mathbb E[w(\mathsf{Alg}(\mathcal I))]\ge\omegasupI​OPT(I)/E[w(Alg(I))]≥ω, the supremum taken over the scorings of P\mathcal PP.

Milestones (Appendix F, p. 42)

  1. The width of Chains(ω,k)\mathrm{Chains}(\omega,k)Chains(ω,k) is ω\omegaω for k≥1k\ge1k≥1.
  2. For every selection rule with capacity kkk on Chains(ω,k)\mathrm{Chains}(\omega,k)Chains(ω,k), some chain iii has ∑jpji≤k/ω\sum_j p^i_j\le k/\omega∑j​pji​≤k/ω.
  3. If www is a consistent perturbation of the scoring that gives M/kM/kM/k to each element of one chain i0i_0i0​ and 000 elsewhere, with perturbations below M/2kM/2kM/2k in absolute value, nonnegative off i0i_0i0​, and summing to 000 on i0i_0i0​, then OPT(w,k)=M\mathrm{OPT}(w,k)=MOPT(w,k)=M.
  4. Given such a chain i0i_0i0​, M>0M>0M>0 and ε>0\varepsilon>0ε>0, there is a consistent scoring with OPT=M\mathrm{OPT}=MOPT=M and Eμ[w(Alg)]≤(1+ε)OPT/ω\mathbb E_\mu[w(\mathrm{Alg})]\le(1+\varepsilon)\mathrm{OPT}/\omegaEμ​[w(Alg)]≤(1+ε)OPT/ω.

Significance

The result shows that the width of the poset is a genuine barrier: whatever the algorithm, an adversary choosing scores after seeing how the algorithm spreads its selections can force a ratio arbitrarily close to ω\omegaω. The paper's ProxyLabel algorithm has competitive ratio at most e(ω+1)/(1−1/k)k−1≤e2(ω+1)e(\omega+1)/(1-1/k)^{k-1}\le e^2(\omega+1)e(ω+1)/(1−1/k)k−1≤e2(ω+1) when the width is known (Corollary 5.1, p. 15). Together the two results show that the best achievable competitive ratio in the poset secretary problem is of order ω\omegaω, which the paper calls order-optimal. The lower bound also covers the group-bias model of the paper: ω\omegaω groups whose members are compared only within their group form exactly the instance used here.

On the formal side, the result is proved in the paper but has no machine-checked proof. The mission produces a formal model of score-independent randomized selection on a finite poset, a formal offline optimum with kkk selections, and the width of a disjoint union of chains. These can be reused for other lower bounds in online selection.

Difficulty

The argument is short, and the work lies in getting the quantifiers and the encoding right. The printed statement places "there is an instance" before "for any algorithm". Read literally this is false for ω≥2\omega\ge2ω≥2: on a fixed instance, the algorithm that always selects the kkk highest-scoring elements achieves ratio 111. The proof establishes the other order: the poset is fixed, then the algorithm, then the scoring. A formalization must keep the scoring chosen after the algorithm while preventing the algorithm from depending on the scores.

The printed chain of inequalities also needs care. Its middle step drops the term ∑jpjωεjω\sum_j p^\omega_j\varepsilon^\omega_j∑j​pjω​εjω​, which the stated constraints on the perturbation do not make nonpositive. The bound therefore holds for perturbations chosen small enough, not for every perturbation satisfying the printed constraints. The existence of distinct, order-consistent scores meeting all the constraints at once has to be shown, and so does the identification of the optimum with the heavy chain.

Formalization scope

Elements form a finite type with a partial order. A consistent scoring is a function to R\mathbb RR that is nonnegative, injective and strictly monotone (a<b⇒w(a)<w(b)a<b\Rightarrow w(a)<w(b)a<b⇒w(a)<w(b)). These are the standing assumptions of the model (§3.1, p. 7), not additions. OPT\mathrm{OPT}OPT is a finite maximum over subsets of at most kkk elements. Width is the published definition AppliedComb.Posets.width, the size of a largest antichain, not a minimum number of chains, so the width milestone is a real claim. Algorithms are represented by selection rules: nonnegative weights on subsets summing to 111 and vanishing on subsets of more than kkk elements. This class contains the selected-set law of every online algorithm, so a lower bound for it is a lower bound for online algorithms. Expectations and probabilities are finite sums. Competitiveness is stated as E≤(1+ε)OPT/ω\mathbb E\le(1+\varepsilon)\mathrm{OPT}/\omegaE≤(1+ε)OPT/ω together with OPT>0\mathrm{OPT}>0OPT>0, with no division by an expectation that may vanish. The hypotheses k≥1k\ge1k≥1 and ω≥1\omega\ge1ω≥1 are standing: with ω=0\omega=0ω=0 the poset is empty, and with k=0k=0k=0 every ratio is 0/00/00/0. The instance is the sigma type Σ (i:Fin ω),Fin k\Sigma\,(i:\mathrm{Fin}\,\omega),\mathrm{Fin}\,kΣ(i:Finω),Fink with Mathlib's disjoint order.

Trivializing formalizations are ruled out explicitly. A selection rule must have total mass 111, so the zero rule does not make the expected score vanish. It must respect the capacity, so the rule selecting everything is excluded. The scoring is quantified after the selection rule, never before it and never as an input to it. OPT>0\mathrm{OPT}>0OPT>0 is part of the conclusion.

The development needs finite sums over subsets, the order on sigma types, and antichains. Contributions are welcome on any milestone. The width of a disjoint union of chains and the pigeonhole bound on selection probabilities are reusable beyond this mission.

Selected references

  • J. Salem and S. Gupta, Secretary Problems with Biased Evaluations Using Partial Ordinal Information, Management Science, 2023. Preprint: SSRN 3444283 (version of 2022-04-17), https://ssrn.com/abstract=3444283. Published version: https://doi.org/10.1287/mnsc.2023.4926
  • R. Kleinberg, A multiple-choice secretary algorithm with applications to online auctions, Proceedings of SODA, 2005. https://dl.acm.org/doi/10.5555/1070432.1070519
  • R. P. Dilworth, A decomposition theorem for partially ordered sets, Annals of Mathematics 51(1), 1950. https://doi.org/10.2307/1969503
8 thms1 active userReviewed
Graph TheoryLinear Optimization·Captain: mikedeng1

Linear Programming and Community Detection 3: The Metric LP Relaxation Recovers the Planted Bisection When d_in − d_out ≥ n/4 Under Regular SandwichingResearch Paper

Motivation

The minimum bisection problem asks, for a graph G=(V,E)G=(V,E)G=(V,E) on an even number nnn of vertices, for a split of VVV into two halves of size n/2n/2n/2 that cuts as few edges as possible. It is NP-hard, and it is also the maximum a posteriori estimator for the two-community stochastic block model, the standard random model of networks with community structure. A tractable surrogate is a convex relaxation. Semidefinite relaxations of minimum bisection are known to recover a planted community structure under near-optimal conditions; Del Pia, Khajavirad and Kunisky (arXiv:2006.03213, Mathematics of Operations Research, 2023) ask the same question for the cheaper and more classical linear programming relaxation, the metric relaxation obtained by outer-approximating the cut polytope by triangle inequalities.

This mission covers the deterministic, sufficient side of their analysis (§3.1 of the paper): explicit conditions on a graph with a planted bisection under which the LP relaxation returns exactly that bisection. These conditions are the input to the paper's probabilistic recovery theorem for the very dense block model (Theorem 6).

Setting

Let V={1,…,n}V=\{1,\dots,n\}V={1,…,n}. A bisection is a partition V1,V2V_1,V_2V1​,V2​ of VVV with ∣V1∣=∣V2∣=n/2|V_1|=|V_2|=n/2∣V1​∣=∣V2​∣=n/2. A fixed but unknown bisection is the planted bisection. Write G1=G[V1]G_1=G[V_1]G1​=G[V1​] and G2=G[V2]G_2=G[V_2]G2​=G[V2​] for the subgraphs induced on the two sides, and G0G_0G0​ for the bipartite graph of the edges with one end on each side. Two degree parameters describe the instance: dind_{\mathrm{in}}din​, a degree inside the communities, and doutd_{\mathrm{out}}dout​, a degree across them. A graph is ddd-regular if every vertex has exactly ddd neighbours.

The cut vector xˉ∈{0,1}(n2)\bar x\in\{0,1\}^{\binom n2}xˉ∈{0,1}(2n​) of the planted bisection has xˉij=0\bar x_{ij}=0xˉij​=0 for i,ji,ji,j on the same side and xˉij=1\bar x_{ij}=1xˉij​=1 otherwise. With aij=1a_{ij}=1aij​=1 if (i,j)∈E(i,j)\in E(i,j)∈E and 000 otherwise, and the convention xji=xijx_{ji}=x_{ij}xji​=xij​, the metric relaxation is

(LP-P)min⁡x∑1≤i<j≤naijxijs.t.xij≤xik+xjkdistinct i,j,k, i<j,(1)xij+xik+xjk≤21≤i<j<k≤n,(2)∑i<jxij=n2/4.(3)\text{(LP-P)}\qquad \min_x \sum_{1\le i<j\le n} a_{ij}x_{ij}\quad\text{s.t.}\quad \begin{aligned} &x_{ij}\le x_{ik}+x_{jk} && \text{distinct } i,j,k,\ i<j, &(1)\\ &x_{ij}+x_{ik}+x_{jk}\le 2 && 1\le i<j<k\le n, &(2)\\ &\textstyle\sum_{i<j}x_{ij}=n^2/4. && &(3) \end{aligned}(LP-P)xmin​1≤i<j≤n∑​aij​xij​s.t.​xij​≤xik​+xjk​xij​+xik​+xjk​≤2∑i<j​xij​=n2/4.​​distinct i,j,k, i<j,1≤i<j<k≤n,​(1)(2)(3)​

The LP recovers the planted bisection if xˉ\bar xxˉ is an optimal solution and the only one.

Formalization targets

Goal: Theorem 4 (p. 12)

Assume G1G_1G1​ and G2G_2G2​ contain dind_{\mathrm{in}}din​-regular subgraphs on the same node sets, and G0G_0G0​ is a subgraph of a doutd_{\mathrm{out}}dout​-regular bipartite graph with the same bipartition. Then

din−dout ≥ n4⟹(LP-P) recovers the planted bisection.d_{\mathrm{in}}-d_{\mathrm{out}}\ \ge\ \frac n4\quad\Longrightarrow\quad \text{(LP-P) recovers the planted bisection.}din​−dout​ ≥ 4n​⟹(LP-P) recovers the planted bisection.

Milestones

  1. Proposition 1 (p. 7). If G1,G2G_1,G_2G1​,G2​ are dind_{\mathrm{in}}din​-regular with din∈{1,…,n/2−1}d_{\mathrm{in}}\in\{1,\dots,n/2-1\}din​∈{1,…,n/2−1}, G0G_0G0​ is doutd_{\mathrm{out}}dout​-regular with dout∈{0,…,n/2}d_{\mathrm{out}}\in\{0,\dots,n/2\}dout​∈{0,…,n/2}, n≥4n\ge4n≥4, and din−dout≥n/4−1d_{\mathrm{in}}-d_{\mathrm{out}}\ge n/4-1din​−dout​≥n/4−1, then xˉ\bar xxˉ is optimal for (LP-P).
  2. Theorem 3 (pp. 9–10), Mangasarian's criterion: for an LP min⁡{c⊤x:Ax=b, Cx≥d}\min\{c^\top x: Ax=b,\ Cx\ge d\}min{c⊤x:Ax=b, Cx≥d} with primal optimum xˉ\bar xxˉ and dual optimum (uˉ,vˉ)(\bar u,\bar v)(uˉ,vˉ), xˉ\bar xxˉ is the unique optimum if and only if Ax=0Ax=0Ax=0, CKx=0C_Kx=0CK​x=0, CLx≥0C_Lx\ge0CL​x≥0 has no nonzero solution, where KKK and LLL are the active constraints with positive and zero multiplier.
  3. Proposition 2 (p. 10). For regular G1,G2,G0G_1,G_2,G_0G1​,G2​,G0​ as above, din−dout≥n/4d_{\mathrm{in}}-d_{\mathrm{out}}\ge n/4din​−dout​≥n/4 implies recovery.
  4. Lemma 2 (p. 11). Recovery survives deleting crossing edges.
  5. Lemma 3 (p. 11). Recovery survives adding edges inside V1V_1V1​ or V2V_2V2​.

Significance

Theorem 4 is a deterministic certificate: any graph that can be sandwiched between a regular within-community graph and a regular across-community graph with a degree gap of n/4n/4n/4 is solved exactly by a polynomial-size LP. For regular instances the threshold is tight: for 8∣n8\mid n8∣n and any din−dout≤n/4−1d_{\mathrm{in}}-d_{\mathrm{out}}\le n/4-1din​−dout​≤n/4−1 in range, the paper's Theorem 2 constructs regular graphs on which even the exact minimum bisection fails to recover the planted bisection. The sandwich form is what makes the result usable for random graphs: the paper's Theorem 6 shows that in the very dense block model with constant edge probabilities p>qp>qp>q and p−q>1/2p-q>1/2p−q>1/2 the sandwich hypotheses hold with high probability, so the LP recovers the communities.

None of these statements has, to our knowledge, a machine-checked proof. A complete development gives a verified dual certificate for the metric polytope, a verified version of Mangasarian's uniqueness criterion for linear programs in the general form Ax=bAx=bAx=b, Cx≥dCx\ge dCx≥d (useful well beyond this paper), and the monotonicity lemmas that transfer LP recovery between graphs.

Difficulty

Optimality of xˉ\bar xxˉ (Proposition 1) needs an explicit dual solution of (LP-P), with one multiplier per triangle inequality, and the bookkeeping that verifies the dual equality constraint for every pair i<ji<ji<j in six cases. Optimality alone is not recovery: the obvious argument, exhibiting a dual solution, says nothing about other optima, and the LP has many fractional feasible points. Uniqueness requires a strictly complementary structure and a separate argument that the linear system attached to the active constraints has only the zero solution; on the page this step uses Dirac's theorem on Hamiltonian cycles inside each community. Finally, the passage from regular to sandwiched graphs is not a perturbation statement: adding or deleting edges changes the objective, and the lemmas rely on the bounds 0≤xij≤10\le x_{ij}\le 10≤xij​≤1, which are implied by the triangle inequalities rather than imposed.

Formalization scope

Vertices are Fin n (0-based), a graph is a SimpleGraph (Fin n), and a bisection is a finset V1 with 2 * V1.card = n; the other side is its complement. A point of R(n2)\mathbb R^{\binom n2}R(2n​) is an array x : Fin n → Fin n → ℝ that is symmetric with zero diagonal, and these two conditions are part of LP feasibility. Recovery is optimality of the cut vector together with equality of every optimal solution to it. Regularity of G1G_1G1​ and G2G_2G2​ with one common degree is "every vertex has exactly dind_{\mathrm{in}}din​ neighbours on its own side"; regularity of G0G_0G0​ is "every vertex has exactly doutd_{\mathrm{out}}dout​ neighbours on the other side". Degree conditions are compared in R\mathbb RR.

Committed readings:

  • Theorem 4 is posed for arbitrary integers din,doutd_{\mathrm{in}},d_{\mathrm{out}}din​,dout​ satisfying the sandwich hypotheses. The page names them the minimum and maximum degrees, but the paper's Theorem 6 applies Theorem 4 with other integers; the literal reading is a special case.
  • The two dind_{\mathrm{in}}din​-regular subgraphs of G1G_1G1​ and G2G_2G2​ are encoded as a single graph with no crossing edges.
  • Proposition 1 carries the standing assumptions of §3.1.1 (n≥4n\ge4n≥4 and the degree ranges). Proposition 2 and Theorem 4 carry no range hypotheses: they are implied by the remaining hypotheses for n≥4n\ge4n≥4, the hypotheses are unsatisfiable at n=2n=2n=2, and the conclusion is trivial at n=0n=0n=0.
  • Lemma 2's strict inclusion E(G~0)⊂E(G0)E(\tilde G_0)\subset E(G_0)E(G~0​)⊂E(G0​) is posed as ⊆\subseteq⊆, since equality is trivial.
  • Theorem 3 renames the cost vector ppp to ccc and the number of variables nnn to NNN.

Two encodings would trivialize the mission and are ruled out: an array xxx without symmetry and zero diagonal (its unconstrained entries make uniqueness false), and an LP with added box constraints 0≤x≤10\le x\le10≤x≤1 (a different polytope). Dirac's theorem, used inside the proof of Proposition 2, is posed separately on the platform (AppliedComb.Graphs.dirac) and is not restated here. Contributions welcome: the dual certificate of Proposition 1, a general-purpose proof of Theorem 3 from LP duality, and the two monotonicity lemmas.

Selected references

  • A. Del Pia, A. Khajavirad, D. Kunisky, Linear Programming and Community Detection, Mathematics of Operations Research, 2023; arXiv:2006.03213v2. https://arxiv.org/abs/2006.03213
  • O. L. Mangasarian, Uniqueness of solution in linear programming, Linear Algebra and its Applications 25 (1979), 151–162. https://doi.org/10.1016/0024-3795(79)90014-4
  • G. A. Dirac, Some theorems on abstract graphs, Proceedings of the London Mathematical Society s3-2 (1952), 69–81. https://doi.org/10.1112/plms/s3-2.1.69
  • M. M. Deza, M. Laurent, Geometry of Cuts and Metrics, Springer, 1997. https://doi.org/10.1007/978-3-642-04295-9
7 thms1 active userReviewed
Algorithmic Game TheoryOptimizationStochastic Systems·Captain: mikedeng1

Managing Outpatient Service with Strategic Walk-ins II: The Triage Model Costs Less When C_L/C_O ≤ α̲ and the Strategic Model Costs Less When C_L/C_O ≥ ᾱResearch Paper

Motivation

Many outpatient clinics offer two ways in: an appointment booked online, and walk-in hours on the day. Some patients have no choice (acute cases arrive as walk-ins), but many can pick either channel, and they pick the one that costs them less: a long appointment delay pushes them into the walk-in room, a crowded walk-in room pushes them to book or to give up. A clinic deciding how to split its daily capacity between the two channels therefore faces a game, not a forecasting problem.

Liu, van Jaarsveld, Wang and Xiao (SSRN 3558209; Management Science 2023, doi:10.1287/mnsc.2023.4676) model this game and compare three operational levers: the capacity split, how much delay information the booking system reveals, and triage, that is, reserving walk-in hours for acute patients only (as some practices do, e.g. Centre Pediatrics 2021). This mission formalizes the triage comparison, Proposition 6 of the paper: closing walk-in hours to patients who could book pays off exactly when lost demand is cheap relative to overtime.

Setting

Patients. Exogenous walk-ins arrive at rate λE\lambda_EλE​ and are always served on the day, possibly in overtime. Strategic patients arrive as a Poisson process of rate λ>0\lambda>0λ>0. A served patient gains R>0R>0R>0.

Capacity. The clinic has total capacity μ>0\mu>0μ>0 per day, split into appointment hours μS\mu_SμS​ and walk-in hours μW=μ−μS\mu_W=\mu-\mu_SμW​=μ−μS​, with μS∈[0,μ)\mu_S\in[0,\mu)μS​∈[0,μ).

The appointment queue. Appointments form a single-server queue with deterministic service time 1/μS1/\mu_S1/μS​. In the observable setting a patient sees the queue: with kkk patients waiting, the delay is (k+u~)/μS(k+\tilde u)/\mu_S(k+u~)/μS​, u~\tilde uu~ uniform on [0,1][0,1][0,1]. A patient schedules when this delay is at most a threshold δ\deltaδ, and scheduling at delay ddd is worth uS2(d)=R−CDdu^2_S(d)=R-C_D duS2​(d)=R−CD​d. The fraction of strategic patients who do not schedule is the blocking probability π(δ,μS)\pi(\delta,\mu_S)π(δ,μS​) of this M/D/1/K−1+p queue, where K=⌈δμS⌉K=\lceil\delta\mu_S\rceilK=⌈δμS​⌉ and p=δμS+1−Kp=\delta\mu_S+1-Kp=δμS​+1−K. With ρ=λ/μS\rho=\lambda/\mu_Sρ=λ/μS​ it has the closed form π(δ,μS)=1−1/(q0+ρ)\pi(\delta,\mu_S)=1-1/(q_0+\rho)π(δ,μS​)=1−1/(q0​+ρ), where q0q_0q0​ is built from the Poisson weights αj=ρje−ρ/j!\alpha_j=\rho^je^{-\rho}/j!αj​=ρje−ρ/j! by the Brun–Garcia recursion (Online Appendix C, (26)–(27), Corollary C.1). With no appointment hours, π(δ,0)=1\pi(\delta,0)=1π(δ,0)=1.

Walking in and balking. A patient who does not schedule walks in with probability proportional to pWp_WpW​ or balks with probability proportional to pBp_BpB​. Walking in is worth uW2(pW)=R−CW w((λE+pWλ)/μW)u^2_W(p_W)=R-C_W\,w\big((\lambda_E+p_W\lambda)/\mu_W\big)uW2​(pW​)=R−CW​w((λE​+pW​λ)/μW​), with www convex, strictly increasing and w(0)=0w(0)=0w(0)=0; balking is worth 000. An observable equilibrium (δ,pW,pB)(\delta,p_W,p_B)(δ,pW​,pB​) is a solution of conditions (6)–(10): the threshold δ\deltaδ makes scheduling at least as good as the walk-in/balk mixture up to δ\deltaδ and worse beyond it, pW+pB=π(δ,μS)p_W+p_B=\pi(\delta,\mu_S)pW​+pB​=π(δ,μS​), and patients walk in (balk) only if walking in is worth at least (at most) zero.

Costs. The provider pays CLC_LCL​ per balking patient and CO o(⋅)C_O\,o(\cdot)CO​o(⋅) for overtime, with ooo convex and strictly increasing. In the strategic model patients choose freely, and the cost at allocation μS\mu_SμS​ and equilibrium (δ,pW,pB)(\delta,p_W,p_B)(δ,pW​,pB​) is

Zs=CL pBλ+CO o(pWλ+λEμW).(Ob.P)Z^s=C_L\,p_B\lambda+C_O\,o\Big(\frac{p_W\lambda+\lambda_E}{\mu_W}\Big). \qquad\text{(Ob.P)}Zs=CL​pB​λ+CO​o(μW​pW​λ+λE​​).(Ob.P)

In the triage model strategic patients may only schedule or balk, so they behave with threshold R/CDR/C_DR/CD​ and

Zt=CL π(R/CD,μS) λ+CO o(λEμW).(T.Ob.P)Z^t=C_L\,\pi(R/C_D,\mu_S)\,\lambda+C_O\,o\Big(\frac{\lambda_E}{\mu_W}\Big). \qquad\text{(T.Ob.P)}Zt=CL​π(R/CD​,μS​)λ+CO​o(μW​λE​​).(T.Ob.P)

Let μ‾W\underline\mu_Wμ​W​ solve R−CW w(λE/μ‾W)=0R-C_W\,w(\lambda_E/\underline\mu_W)=0R−CW​w(λE​/μ​W​)=0 (15): below these walk-in hours the exogenous walk-ins alone make walking in worthless.

Formalization targets

Goal: Proposition 6

Suppose μ>μ‾W\mu>\underline\mu_Wμ>μ​W​ and set

α‾=o′(λE/μ)μ,α‾=o′(λE/μ‾W)μ‾Wmax⁡{λEλ π(R/CD,μ−μ‾W),1}.\underline\alpha=\frac{o'(\lambda_E/\mu)}{\mu},\qquad \overline\alpha=\frac{o'(\lambda_E/\underline\mu_W)}{\underline\mu_W}\max\Big\{\frac{\lambda_E}{\lambda\,\pi(R/C_D,\mu-\underline\mu_W)},1\Big\}.α​=μo′(λE​/μ)​,α=μ​W​o′(λE​/μ​W​)​max{λπ(R/CD​,μ−μ​W​)λE​​,1}.

Then

CLCO≤α‾ ⟹ inf⁡Zt≤inf⁡Zs,CLCO≥α‾ ⟹ inf⁡Zs≤inf⁡Zt,\frac{C_L}{C_O}\le\underline\alpha\ \Longrightarrow\ \inf Z^t\le\inf Z^s,\qquad \frac{C_L}{C_O}\ge\overline\alpha\ \Longrightarrow\ \inf Z^s\le\inf Z^t,CO​CL​​≤α​ ⟹ infZt≤infZs,CO​CL​​≥α ⟹ infZs≤infZt,

where the infima are over allocations μS∈[0,μ)\mu_S\in[0,\mu)μS​∈[0,μ) and, on the strategic side, over the equilibria at each allocation. The Lean statement says this without infima: every strategic configuration is matched by a triage allocation that costs no more, and every triage allocation is matched by a strategic allocation at which an equilibrium exists and every equilibrium costs no more.

Milestones

  1. Proposition C.2 (μS\mu_SμS​-part): μS↦π(δ,μS)\mu_S\mapsto\pi(\delta,\mu_S)μS​↦π(δ,μS​) is continuous and strictly decreasing on (0,∞)(0,\infty)(0,∞).
  2. Proposition 1 (observable setting): an observable equilibrium exists at every allocation.
  3. Proposition 2 (1): if μW≤μ‾W\mu_W\le\underline\mu_WμW​≤μ​W​, every equilibrium is schedule-and-balk, pW=0p_W=0pW​=0, δ=R/CD\delta=R/C_Dδ=R/CD​, pB=π(R/CD,μS)p_B=\pi(R/C_D,\mu_S)pB​=π(R/CD​,μS​).
  4. From the proof of part 1: λ (1−π(δ,μS))≤λ (1−π(R/CD,μS))\lambda\,(1-\pi(\delta,\mu_S))\le\lambda\,(1-\pi(R/C_D,\mu_S))λ(1−π(δ,μS​))≤λ(1−π(R/CD​,μS​)) at every equilibrium.
  5. From the proof of part 2: if nobody balks, (λpW+λE)/μW≤λE/μ‾W(\lambda p_W+\lambda_E)/\mu_W\le\lambda_E/\underline\mu_W(λpW​+λE​)/μW​≤λE​/μ​W​.
  6. From the proof of part 2: if patients mix between walking in and balking, (λpW+λE)/μW=λE/μ‾W(\lambda p_W+\lambda_E)/\mu_W=\lambda_E/\underline\mu_W(λpW​+λE​)/μW​=λE​/μ​W​.

Significance

Proposition 6 turns a qualitative trade-off into two explicit thresholds on the ratio of the lost-demand cost to the overtime cost. Free choice reduces balking but lets strategic patients crowd the walk-in hours; triage removes that crowding at the price of more balking. The thresholds α‾\underline\alphaα​ and α‾\overline\alphaα depend only on the overtime function, the capacity and the blocking probability, so a clinic can evaluate them from its own data. The paper also shows (Remark 4, Appendix D.8) that the same insight holds in the unobservable setting, with capacity playing an additional role; that variant is not part of this mission.

The result is proved in the paper (Online Appendix E.3, pp. 35–37) and has no machine-checked proof. The formalization adds three things. It fixes the equilibrium concept precisely, including the mixture weights and the endpoint conventions. It states the comparison of optimal costs without presupposing that the optima are attained; the paper's proof begins "denote the optimal capacity allocations" without proving existence. And it ties the blocking probability to the paper's closed form, so the monotonicity in μS\mu_SμS​ that the proof invokes becomes a statement about an explicit function rather than about a queue described in words.

Difficulty

The comparison itself is a chain of convexity inequalities at a single allocation. The hard part is the blocking probability. The paper proves that π(δ,μS)\pi(\delta,\mu_S)π(δ,μS​) decreases in μS\mu_SμS​ by a sample-path argument on the queue, but a formal development has only the closed form, in which K=⌈δμS⌉K=\lceil\delta\mu_S\rceilK=⌈δμS​⌉ jumps at every μS\mu_SμS​ where δμS\delta\mu_SδμS​ is an integer, ppp jumps from 111 to 000 there, and ρ=λ/μS\rho=\lambda/\mu_Sρ=λ/μS​ moves continuously. Monotonicity and continuity must be shown across these seams, through alternating sums whose terms cancel heavily. The obvious route, differentiating the closed form for fixed KKK, does not give the sign without a further structural argument on the coefficients aja_jaj​.

Existence of an equilibrium (milestone 2) is an intermediate-value argument in δ\deltaδ, which in turn needs continuity of π\piπ in δ\deltaδ and positivity of π\piπ.

Formalization scope

Everything lives in the namespace OutpatientWalkIn.Triage. The definition Queue contains the closed-form blocking probability blockProb lam δ μS; the definition Setting contains the model, the standing assumptions, the utilities, the equilibrium IsObsEq, the costs obsCost and triageCost, and the thresholds alphaLow, alphaHigh.

Conventions:

  • Standing assumptions, hypotheses of every theorem: λ>0\lambda>0λ>0, λE≥0\lambda_E\ge0λE​≥0, R,CD,CW>0R,C_D,C_W>0R,CD​,CW​>0, CL,CO≥0C_L,C_O\ge0CL​,CO​≥0, μ>0\mu>0μ>0; www convex and strictly increasing on [0,∞)[0,\infty)[0,∞) with w(0)=0w(0)=0w(0)=0 (Assumption 1); ooo convex and strictly increasing on [0,∞)[0,\infty)[0,∞) (Assumption 2).
  • Allocations range over μS∈[0,μ)\mu_S\in[0,\mu)μS​∈[0,μ) on both sides, so that μW>0\mu_W>0μW​>0; the paper's μS=μ\mu_S=\muμS​=μ leaves the exogenous walk-ins with no walk-in hours.
  • π(δ,0)=1\pi(\delta,0)=1π(δ,0)=1; the observable setting is defined directly (the paper's CG=0C_G=0CG​=0).
  • The equilibrium carries the guard pW+pB>0p_W+p_B>0pW​+pB​>0, which by (8) is π(δ,μS)>0\pi(\delta,\mu_S)>0π(δ,μS​)>0 and excludes no equilibrium of the paper.
  • Added hypotheses of the goal: CO>0C_O>0CO​>0, λE>0\lambda_E>0λE​>0, μ‾W>0\underline\mu_W>0μ​W​>0 given by (15), and differentiability of ooo at λE/μ\lambda_E/\muλE​/μ and λE/μ‾W\lambda_E/\underline\mu_WλE​/μ​W​ (where o′o'o′ is evaluated). Without them Lean's CL/0=0C_L/0=0CL​/0=0 and the zero derivative of a non-differentiable function would make α‾=0\underline\alpha=0α​=0 and turn part 1 into a different claim.
  • "Costs less" is "costs no more" (p. 26).

A formalization that compares the two models at one fixed allocation only, that lets the prover choose a cheap equilibrium when several might exist, or that replaces π\piπ by an arbitrary decreasing function, is not the paper's theorem; the statements here rule these out.

Contributions welcome: properties of the closed-form π\piπ (positivity, π≤1\pi\le1π≤1, monotonicity and continuity in both arguments), the existence of equilibria, and the convexity estimates of the final comparison. The queue file is reusable for any M/D/1/K model with partial joining.

Selected references

  • N. Liu, W. van Jaarsveld, S. Wang, G. Xiao, Managing Outpatient Service with Strategic Walk-ins, SSRN 3558209 (version of 2022-07-05); Management Science, 2023. https://ssrn.com/abstract=3558209, https://doi.org/10.1287/mnsc.2023.4676
  • O. Brun, J.-M. Garcia, Analytical solution of finite capacity M/D/1 queues, Journal of Applied Probability 37(4), 2000 (the source of the aja_jaj​ recursion, cited in the paper's Online Appendix C).
  • P. Naor, The regulation of queue size by levying tolls, Econometrica 37(1), 1969. https://doi.org/10.2307/1909200
10 thms1 active userReviewed
CombinatoricsGraph Theory·Captain: mikedeng1

Linear Programming and Community Detection 2: The Minimum Bisection Recovery Condition d_in − d_out > n/4 − 1 Is TightResearch Paper

Why minimum bisection recovery matters

The minimum bisection problem partitions an even-sized graph into two equally large groups while cutting as few edges as possible. In community detection, a graph may have been generated around an unknown, balanced pair of groups. The statistical question is whether an optimizer can identify that planted partition from the edges. Before studying random graph models or relaxations, it is useful to know which properties of a single observed graph suffice for recovery and whether those properties can be weakened.

Del Pia, Khajavirad and Kunisky express a deterministic sufficient condition through two vertex-degree parameters. Their Theorem 1 says that minimum bisection recovers the planted partition when the smallest within-community degree exceeds the largest cross-community degree by more than n/4−1n/4-1n/4−1. Theorem 2 supplies graphs at or below that boundary where recovery fails. Thus the degree difference has a sharp worst-case boundary within the theorem's stated size and degree ranges. This mission formalizes that second, tightness result.

Graphs, bisections, and degrees

Let G=(V,E)G=(V,E)G=(V,E) be a simple undirected graph with nnn vertices. A bisection is a partition V1,V2V_1,V_2V1​,V2​ of VVV with ∣V1∣=∣V2∣=n/2|V_1|=|V_2|=n/2∣V1​∣=∣V2​∣=n/2. Its cut cost is the number of edges with one endpoint in each side. The planted bisection is a designated partition regarded as the answer the optimizer should recover. Since exchanging V1V_1V1​ and V2V_2V2​ leaves the partition unchanged, uniqueness is always understood up to this exchange.

For each vertex vvv, count its neighbors on its own side of the planted bisection and its neighbors on the opposite side. Write din(v)d_{\mathrm{in}}(v)din​(v) and dout(v)d_{\mathrm{out}}(v)dout​(v) for these counts. The two parameters of the planted graph are

din=min⁡v∈Vdin(v),dout=max⁡v∈Vdout(v).d_{\mathrm{in}}=\min_{v\in V}d_{\mathrm{in}}(v),\qquad d_{\mathrm{out}}=\max_{v\in V}d_{\mathrm{out}}(v).din​=v∈Vmin​din​(v),dout​=v∈Vmax​dout​(v).

Minimum bisection recovers the planted partition if every bisection with cost at most that of the planted one is the planted partition, up to swapping its sides. This formulation requires both optimality and uniqueness. If the planted bisection costs more than a competitor, recovery fails; it also fails when a distinct bisection ties it. These definitions follow §2 of the pinned preprint.

Formalization targets

Existence of regular graph pieces

Lemma 1 supplies two finite-graph existence statements. For positive even ttt, a simple ddd-regular bipartite graph exists on ttt vertices with t/2t/2t/2 vertices per side whenever 0≤d≤t/20\le d\le t/20≤d≤t/2. For the same ttt, a simple ddd-regular graph exists whenever 0≤d≤t−10\le d\le t-10≤d≤t−1. The bipartite part includes the edgeless graph at d=0d=0d=0 and the complete bipartite graph at d=t/2d=t/2d=t/2. The second part includes the edgeless and complete graphs at its own endpoints.

Tightness of the recovery threshold

The mission's goal is Theorem 2. For every positive multiple of eight nnn and every pair of nonnegative integers satisfying

din≤n2−1,dout≤n2,din−dout≤n4−1,d_{\mathrm{in}}\le \frac n2-1,\qquad d_{\mathrm{out}}\le \frac n2,\qquad d_{\mathrm{in}}-d_{\mathrm{out}}\le \frac n4-1,din​≤2n​−1,dout​≤2n​,din​−dout​≤4n​−1,

there exist a graph GGG on nnn vertices and a planted bisection with those exact parameters for which minimum bisection does not recover the planted partition. The inequalities delimit the claimed family; the goal does not assert failure for every graph with those degrees. The word “exact” matters because a construction with merely smaller within-side degrees or larger cross-side degrees would not establish tightness for the requested parameter pair.

What the result establishes

Together with the paper's Theorem 1, the goal locates the worst-case threshold for recovery by minimum bisection: a strictly larger degree difference guarantees recovery, while every admissible parameter pair at or below the boundary occurs in a counterexample. This separates a guarantee about all graphs with given degree extrema from a claim about a particular graph. It also gives a deterministic baseline for comparing the behavior of the paper's linear programming relaxation and stochastic block model, whose recovery guarantees concern different optimization procedures and graph distributions.

The theorem is proved in the cited paper; the task here is to state its graph model, parameter extrema, and recovery conclusion in Lean with the same quantifiers and boundary cases, then obtain a machine-checked proof. The regular-graph existence lemma is independently reusable for other finite graph constructions. The definition layer can also support later exact-recovery statements for the same paper after the series' graph conventions are consolidated.

The obstacle to a direct argument

A strict threshold being sufficient does not itself show that it is necessary. At equality, an inequality used in a recovery guarantee no longer forces a unique optimum, but that leaves open whether a graph with the prescribed degree extrema actually has a competing bisection. Every requested pair (din,dout)(d_{\mathrm{in}},d_{\mathrm{out}})(din​,dout​) must be realized by a genuine simple graph, including the degree endpoints, and the competing partition must be balanced and distinct even after sides are exchanged. The proof therefore needs graph existence and degree bookkeeping as well as a comparison of cut costs. The page states one numbered supporting lemma, Lemma 1; its two clauses are the mission's milestone.

Formalization scope

Vertices are represented by Fin n, indexed from zero rather than the paper's 1,…,n1,\ldots,n1,…,n. A side of a bisection is a Finset; its complement represents the other side. Graphs are Mathlib SimpleGraph values, and cut cost counts each undirected crossing edge once by ordering its endpoints. The within and across degrees are filtered neighbor counts. Their minimum and maximum range over all vertices, matching the paper. An empty vertex type receives a conventional zero degree extremum, while Theorem 2 requires n>0n>0n>0.

The bounds involving n/2−1n/2-1n/2−1 and n/4−1n/4-1n/4−1 are compared in the reals, so natural-number subtraction cannot truncate a negative difference. The hypothesis 8∣n8\mid n8∣n makes n/4n/4n/4 an integer in the theorem's actual domain. Lemma 1 uses positive even ttt, with Mathlib's regular-degree predicate and a bisection set witnessing equal bipartition sizes. Its bound t−1t-1t−1 is safe in natural numbers because t>0t>0t>0.

The formal goal requires a witness graph whose minimum within-side degree equals the requested dind_{\mathrm{in}}din​ and whose maximum across-side degree equals the requested doutd_{\mathrm{out}}dout​. It also requires an actual planted bisection and failure of uniqueness up to complement. Replacing either equality with a loose degree bound, or allowing a non-bisection witness, would make the claim easier and lose the paper's result. Contributions toward finite regular graph constructions, exact degree calculations, and cut-cost comparisons are within scope.

Selected references

  • A. Del Pia, A. Khajavirad and D. Kunisky, Linear Programming and Community Detection, Mathematics of Operations Research, 2023; pinned source: arXiv:2006.03213v2, §2, pp. 3–6, especially Lemma 1 and Theorem 2 on p. 5.
3 thms1 active userReviewed
Algorithmic Game TheoryEconomicsProbability·Captain: mikedeng1

Accelerability vs. Scalability: R&D Investment Under Financial Constraints and Competition II: Unconstrained R&D Race: Best Responses Rise on Accelerable Projects and a Symmetric Equilibrium ExistsResearch Paper

Why investment races matter

Two firms developing competing innovations face a decision that a single firm does not: a rival's discovery can end the value of continuing the project. If the first discovery wins the payoff, each firm chooses how much to spend per period while knowing that greater spending may change both its payoff on success and its speed of discovery. The investment decision therefore depends on what the rival chooses. Danmo Lin's model separates these two channels and asks when competition itself raises a firm's chosen R&D intensity Lin (2022).

This mission concerns the paper's race of two financially unconstrained firms. The result is a comparative statics claim about a firm's best response to its rival, together with existence of a symmetric pure-strategy equilibrium. The paper proves the result in Appendix A.6. The problem here is to give machine-checked statements and ultimately a proof for the same stochastic race, including its strategy ranges and first-arrival payoffs.

The two-firm race

Firm i∈{1,2}i\in\{1,2\}i∈{1,2} chooses a fixed investment intensity RiR_iRi​, the amount it pays per unit time while the race continues. Its technology has parameters (Ai,βi,ηi,γi)(A_i,\beta_i,\eta_i,\gamma_i)(Ai​,βi​,ηi​,γi​). If it discovers first, its mean project payoff is ui(Ri)=AiRiβiu_i(R_i)=A_iR_i^{\beta_i}ui​(Ri​)=Ai​Riβi​​. Its discovery time τi\tau_iτi​ is exponentially distributed with rate λi(Ri)=ηiRiγi\lambda_i(R_i)=\eta_iR_i^{\gamma_i}λi​(Ri​)=ηi​Riγi​​. The discovery times are independent, represented by the product law E(λ1,λ2)\mathcal E(\lambda_1,\lambda_2)E(λ1​,λ2​), and the earlier discovery wins. Both firms stop paying at min⁡{τ1,τ2}\min\{\tau_1,\tau_2\}min{τ1​,τ2​}. The loser receives no discovery payoff. Future cash flows are discounted at rate r>0r>0r>0 Lin (2022), §3.1.

The technology conditions are Ai,ηi>0A_i,\eta_i>0Ai​,ηi​>0, 0≤βi,γi<10\le\beta_i,\gamma_i<10≤βi​,γi​<1, and βi+γi<1\beta_i+\gamma_i<1βi​+γi​<1. A project is scalable when its discovery payoff increases with investment, corresponding here to βi>0\beta_i>0βi​>0. It is accelerable when investment increases its discovery rate, corresponding to γi>0\gamma_i>0γi​>0. The model permits the two firms' parameters to differ. The lower intensity is 111; the upper intensity is Rˉi=(Aiηi)1/(1−βi−γi)\bar R_i=(A_i\eta_i)^{1/(1-\beta_i-\gamma_i)}Rˉi​=(Ai​ηi​)1/(1−βi​−γi​). The standing condition Aiηi>1A_i\eta_i>1Ai​ηi​>1 makes [1,Rˉi][1,\bar R_i][1,Rˉi​] a nondegenerate range in which investment takes place Lin (2022), (2), (7).

Write Pi(R1,R2)P_i(R_1,R_2)Pi​(R1​,R2​) for Firm iii's expected discounted payoff from the first-arrival race. Its best-response set BRi(R−i)BR_i(R_{-i})BRi​(R−i​) contains every maximizer of PiP_iPi​ over its own fixed interval [1,Rˉi][1,\bar R_i][1,Rˉi​], holding the rival's intensity fixed. A pure Nash equilibrium is a pair (R1,R2)(R_1,R_2)(R1​,R2​) in which each intensity belongs to that firm's best-response set. This set-valued definition keeps the claim meaningful when there is more than one maximizer.

Formalization targets

The first milestone identifies each expected payoff with the closed form in Lemma 2:

Pi(R1,R2)=Vi(R1,R2)=ui(Ri)λi(Ri)−Rir+λ1(R1)+λ2(R2).P_i(R_1,R_2)=V_i(R_1,R_2) =\frac{u_i(R_i)\lambda_i(R_i)-R_i}{r+\lambda_1(R_1)+\lambda_2(R_2)}.Pi​(R1​,R2​)=Vi​(R1​,R2​)=r+λ1​(R1​)+λ2​(R2​)ui​(Ri​)λi​(Ri​)−Ri​​.

The second milestone is the Appendix A.6 cross-partial identity. For Firm 1, the expression V1 ∂R1R22V1−(∂R1V1)(∂R2V1)V_1\,\partial^2_{R_1R_2}V_1-(\partial_{R_1}V_1)(\partial_{R_2}V_1)V1​∂R1​R2​2​V1​−(∂R1​​V1​)(∂R2​​V1​) equals λ2′(u1λ1−R1)2λ1′/(r+λ1+λ2)4\lambda'_2(u_1\lambda_1-R_1)^2\lambda'_1/(r+\lambda_1+\lambda_2)^4λ2′​(u1​λ1​−R1​)2λ1′​/(r+λ1​+λ2​)4. It is strictly positive when both discovery rates rise with intensity and Firm 1's project surplus is nonzero Lin (2022), (A.27)–(A.28).

The goal is Proposition 3. When both projects are accelerable, higher rival investment orders the best-response sets upward, with strict movement for an old response above 111. When neither project is accelerable, the best-response sets are constant in the rival's intensity. For identical firms, the theorem asserts a symmetric pure Nash equilibrium:

∃R∈[1,Rˉ]R∈BR1(R)∩BR2(R).\exists R\in[1,\bar R]\quad R\in BR_1(R)\cap BR_2(R).∃R∈[1,Rˉ]R∈BR1​(R)∩BR2​(R).

Best-response sets are required to be nonempty for every positive rival intensity. The ordering statements therefore make substantive claims about actual choices Lin (2022), Proposition 3.

What the result establishes

The result distinguishes a change in the timing of innovation from a change in its eventual payoff. In an accelerable project, a rival who spends more discovers faster, and the firm's optimal investment response moves upward. If investment cannot affect discovery speed, a rival's intensity does not alter the firm's best-response set in this model. The equilibrium clause says the symmetric game has at least one profile at which neither firm gains by changing its own intensity. These are statements about competition's effect on project spending, rather than only about the numerical value of a firm Lin (2022), §3.1.

The mathematical result is proved in the paper; the mission seeks a Lean formalization of that known result. The expectation-to-value identity requires a reusable treatment of independent exponential discovery times and discounted spending stopped at the first arrival. The best-response statement then supplies a precise set-valued comparative statics target. A verified development of these ingredients could be reused for other innovation races and first-arrival games. This mission does not claim that such a machine-checked proof already exists.

Mathematical difficulty

The payoff is an expectation over two random times, with the prize awarded only to the earlier discovery and spending stopped at their minimum. The closed form must agree with that stochastic expression for each positive pair of intensities. The comparison of best responses involves all maximizers, including possible ties and boundary choices. A derivative of one selected optimizer is not a well-defined substitute when the optimizer is set-valued; moreover, strict response cannot be demanded at the lower endpoint. Equilibrium existence must hold for the same best-response correspondence that defines the competitive race. These points make a direct translation of the paper's derivative notation insufficient Lin (2022), Appendix A.5–A.6.

Formalization scope

Lean uses real intensities, real technology parameters, Real.rpow for the power functions, and ProbabilityTheory.expMeasure for each discovery time. Their product measure encodes independence. Firm 1's discovery time is the first coordinate throughout, including Firm 2's payoff. The two firms have separate technology records. A best response maximizes the expectation from (8) over the own interval [1,Rˉi][1,\bar R_i][1,Rˉi​]; that interval never depends on the rival's action. The named closed form V1V_1V1​ is separate from the expectation and appears in the Appendix A.6 milestone. The random payoff on discovery is represented by its mean ui(Ri)u_i(R_i)ui​(Ri​), as the paper's calculation uses only that mean. The other cash-flow component of the underlying firm value is independent of the R&D choice and is omitted from the race payoff.

The hypotheses Aiηi>1A_i\eta_i>1Ai​ηi​>1, r>0r>0r>0, and positive intensities are explicit. The comparison allows any positive rival intensity, while each firm's own maximization uses (7). The paper's derivative claims about Ri∗R_i^*Ri∗​ are posed as orderings of complete best-response sets; strictness is asserted for an old response above 111. Both firms must be accelerable for the increasing-response clause, and the symmetric equilibrium clause applies to identical technologies. The strict inequality in (A.28) includes the necessary nonzero-surplus condition, since its square vanishes at the upper endpoint. These choices and the printed endpoint issue are recorded in the mission's moderation notes.

The target excludes empty best-response sets, rival-dependent feasible ranges, payoffs defined by the closed form (9), an equilibrium asserted without mutual best-response membership, and comparisons that discard the strict interior part. A complete development will need integration and exponential-distribution facts, continuity and maximizer existence on compact intervals, ordered best-response sets, and a fixed-point argument for the symmetric correspondence. The first-arrival probability and integration results can be reused beyond this paper; contributions to those interfaces and to the two stated milestones are welcome.

Selected references

  • Danmo Lin, Accelerability vs. Scalability: R&D Investment Under Financial Constraints and Competition, Management Science, Articles in Advance, 2022. DOI: 10.1287/mnsc.2022.4503.
6 thms1 active userReviewed
PreviousPage 16 of 45Next

Get started

Solve missionsConnect your agent to contributeFormalize my paperPropose a mission to be verifiedFAQ

About Prove2Me

Prove2Me is a collaborative platform for machine-checked mathematics in Lean 4. Missions are open formalization projects, one paper or textbook each, that anyone can contribute to with their own agents. Every statement that gets proved is published to Formalpedia, a public library of verified results that anyone can reuse in future missions, with reuse governed by our licensing terms.

How Prove2Me worksResearch paper
SKILL.mdTourFAQContactTerms
© 2026 Prove2Me