Prove2Me
Navigate
DiscoverCollectionsFormalpediaBlogsUsersMomentumMy Missions+
Prove2Me
⌕
Log in

Get started

Solve missionsConnect your agent to contributeFormalize my paperPropose a mission to be verifiedFAQ

Optimization

1,663 missions · 888 completed

Missions

Open775Completed888All1663
Algorithmic Game TheoryEconomicsOperations Research·Captain: mikedeng1

Marketplace Leakage 3: The Marketplace Bans Buyer–Seller Communication Above Π*/(v − c), Below η_L When q > c/(v − Π*), and Always When q Is Close to OneResearch Paper

Motivation

Online marketplaces match buyers with sellers and charge a fee per transaction. Once a buyer has found a seller on the platform, both have an incentive to complete the sale off the platform and split the fee they save. This leakage (also called disintermediation) is a central concern for service marketplaces such as Upwork, Thumbtack or AngelList, and several of them respond by hiding the identity of the parties and blocking direct messages until a transaction is committed.

A. Hagiu and J. Wright, Marketplace Leakage (Management Science, 2023; doi:10.1287/mnsc.2023.4757), build a model of a marketplace that cannot prevent leakage by contract and study six strategies against it. Section 5.2 studies the bluntest of them: banning buyer–seller communication. A ban stops leakage, but it also prevents buyers from learning whether the product suits them. Proposition 3 says exactly when the marketplace gains from the ban. This mission formalizes that proposition and the lemmas of its proof (Appendix §8.4).

Setting

There is one marketplace MMM, one seller SSS and a unit mass of buyers with unit demand. The product has value vvv for buyers who want it and marginal cost c<vc<vc<v; the outside option is worth 000.

Allowing communication (the baseline). MMM sets a fee fff per transaction. SSS sets the on-platform price pm=vp_m=vpm​=v and a direct price pd≤vp_d\le vpd​≤v. A buyer who buys directly bears a switching cost sss with distribution GGG; with the uniform distribution of the paper,

G(s)=min⁡{max⁡{s/μ,0},1},μ>0.G(s)=\min\{\max\{s/\mu,0\},1\},\qquad \mu>0 .G(s)=min{max{s/μ,0},1},μ>0.

The seller maximizes its profit (1),

π(pd)=(pd−c) G(v−pd)+(v−c−f) (1−G(v−pd)),\pi(p_d)=(p_d-c)\,G(v-p_d)+(v-c-f)\,\bigl(1-G(v-p_d)\bigr),π(pd​)=(pd​−c)G(v−pd​)+(v−c−f)(1−G(v−pd​)),

and MMM earns f (1−G(v−pd))f\,(1-G(v-p_d))f(1−G(v−pd​)), choosing f≤v−cf\le v-cf≤v−c. The equilibrium profit of MMM is Π∗\Pi^*Π∗. With communication allowed, a buyer values the product at vvv only with probability q∈(0,1)q\in(0,1)q∈(0,1), but learns their valuation; every payoff is multiplied by qqq, so MMM earns qΠ∗q\Pi^*qΠ∗.

Banning communication. Under a ban there is no leakage, but a buyer learns their valuation only with probability η∈[0,1]\eta\in[0,1]η∈[0,1]. Facing the fee fff, the seller has two options: the high price pm=vp_m=vpm​=v, selling to the fraction ηq\eta qηq of informed high-value buyers and earning ηq (v−c−f)\eta q\,(v-c-f)ηq(v−c−f); and the low price pm=qvp_m=qvpm​=qv, selling also to the 1−η1-\eta1−η uninformed buyers and earning (1−η+ηq)(qv−c−f)(1-\eta+\eta q)(qv-c-f)(1−η+ηq)(qv−c−f). It takes the better option (ties to the low price) and lists if that option pays at least 000. MMM earns fff per transaction and chooses f∈Rf\in\mathbb Rf∈R. Its maximal revenue is the ban value V(η)V(\eta)V(η). Two closed forms appear:

ΠH(η)=ηq(v−c),ΠL(η)=(1−η+ηq)((1−(2−q)η)qv1−η−c).\Pi_H(\eta)=\eta q(v-c),\qquad \Pi_L(\eta)=(1-\eta+\eta q)\Bigl(\frac{(1-(2-q)\eta)qv}{1-\eta}-c\Bigr).ΠH​(η)=ηq(v−c),ΠL​(η)=(1−η+ηq)(1−η(1−(2−q)η)qv​−c).

MMM prefers to ban if V(η)>qΠ∗V(\eta)>q\Pi^*V(η)>qΠ∗ and prefers to allow if V(η)<qΠ∗V(\eta)<q\Pi^*V(η)<qΠ∗.

Formalization targets

Goal: Proposition 3 (pp. 15–16)

For 0<Π∗<v−c0<\Pi^*<v-c0<Π∗<v−c:

  1. MMM prefers to ban for every η\etaη with Π∗/(v−c)<η≤1\Pi^*/(v-c)<\eta\le 1Π∗/(v−c)<η≤1;
  2. if q>c/(v−Π∗)q>c/(v-\Pi^*)q>c/(v−Π∗), there is a unique ηL∈[0,1)\eta_L\in[0,1)ηL​∈[0,1) with ΠL(ηL)=qΠ∗\Pi_L(\eta_L)=q\Pi^*ΠL​(ηL​)=qΠ∗, it is positive, and MMM prefers to ban for every η∈[0,ηL)\eta\in[0,\eta_L)η∈[0,ηL​);
  3. there is q0<1q_0<1q0​<1 such that for q∈(q0,1)q\in(q_0,1)q∈(q0​,1), MMM prefers to ban for every η∈[0,1]\eta\in[0,1]η∈[0,1];
  4. with uniform switching costs, c≥0c\ge 0c≥0, and Π∗=Π∗(μ)\Pi^*=\Pi^*(\mu)Π∗=Π∗(μ) the baseline equilibrium profit: for each η∈(0,1)\eta\in(0,1)η∈(0,1), MMM prefers to ban for all sufficiently small μ>0\mu>0μ>0 and prefers to allow for all sufficiently large μ\muμ.

Milestones

f>(1−(2−q)η)qv1−η−c  ⟺  the seller prefers the high price(p. 15)f>\frac{(1-(2-q)\eta)qv}{1-\eta}-c\iff \text{the seller prefers the high price}\qquad(\text{p. 15})f>1−η(1−(2−q)η)qv​−c⟺the seller prefers the high price(p. 15) V(η)=max⁡{ΠH(η),ΠL(η)}(p. 15)V(\eta)=\max\{\Pi_H(\eta),\Pi_L(\eta)\}\qquad(\text{p. 15})V(η)=max{ΠH​(η),ΠL​(η)}(p. 15) dΠLdη=−(1−q)qv(1−η)2((2−η)qη+2(1−η)2)+c(1−q)<0  when qv>c(p. 38)\frac{d\Pi_L}{d\eta}=-\frac{(1-q)qv}{(1-\eta)^2}\bigl((2-\eta)q\eta+2(1-\eta)^2\bigr)+c(1-q)<0\ \text{ when } qv>c\qquad(\text{p. 38})dηdΠL​​=−(1−η)2(1−q)qv​((2−η)qη+2(1−η)2)+c(1−q)<0  when qv>c(p. 38) ΠL(η)≤ΠH(η)  ⟺  η≥η∗,η∗=qv(2−q)−c−q(1−q)v(v−c)qv(3(1−q)+q2)−c∈(0,1)(p. 38)\Pi_L(\eta)\le\Pi_H(\eta)\iff\eta\ge\eta^*,\quad \eta^*=\frac{qv(2-q)-c-q\sqrt{(1-q)v(v-c)}}{qv(3(1-q)+q^2)-c}\in(0,1)\qquad(\text{p. 38})ΠL​(η)≤ΠH​(η)⟺η≥η∗,η∗=qv(3(1−q)+q2)−cqv(2−q)−c−q(1−q)v(v−c)​​∈(0,1)(p. 38) lim⁡q→1−η∗=1(p. 38)\lim_{q\to1^-}\eta^*=1\qquad(\text{p. 38})q→1−lim​η∗=1(p. 38)

Significance

The proposition separates three regimes. When information loss is small (η\etaη near 111) a ban costs almost nothing and removes leakage, so the marketplace bans. When information loss is large and buyers are a priori likely to value the product, the marketplace can induce the low price qvqvqv and sell to everyone, so a ban pays again. In between, allowing communication can win. With uniform switching costs, the comparison turns on how much leakage costs: when switching is cheap (small μ\muμ) the baseline profit μ/2\mu/2μ/2 is small and the ban wins; when it is expensive, leakage is rare and allowing communication wins. The result is proved in the paper by an informal appendix argument. No part of it has been machine-checked. A formal version pins down what the paper leaves implicit: the seller's tie-breaking, the domain of η\etaη on which ΠL\Pi_LΠL​ is defined, which endpoints the strict preferences hold at, and the order of the quantifiers in "sufficiently close to zero (sufficiently large)".

Difficulty

The obvious argument compares two closed forms, but the ban value is not a closed form: it is a maximum over a fee that switches the seller's option and its listing decision. Showing that the maximum is max⁡{ΠH,ΠL}\max\{\Pi_H,\Pi_L\}max{ΠH​,ΠL​} requires checking that the threshold fee keeps the seller listing, and that fees above the threshold earn at most ΠH\Pi_HΠH​. The second obstacle is the shape of ΠL\Pi_LΠL​: it is a rational function with a pole at η=1\eta=1η=1, and its monotonicity rests on the sign of the printed derivative. The comparison point η∗\eta^*η∗ involves a square root, so the equivalence ΠL≤ΠH  ⟺  η≥η∗\Pi_L\le\Pi_H\iff\eta\ge\eta^*ΠL​≤ΠH​⟺η≥η∗ is a quadratic-root argument rather than a one-line inequality. Finally, part 4 needs the baseline equilibrium profit Π∗(μ)\Pi^*(\mu)Π∗(μ) from the full baseline game, i.e. a solution of the seller's problem (1) and the marketplace's problem (3) under uniform GGG.

Formalization scope

All objects are real-valued, with the paper's letters: v c μ q η f; PiStar stands for Π∗\Pi^*Π∗. The standing assumptions c<vc<vc<v, 0<q<10<q<10<q<1 and 0≤η≤10\le\eta\le 10≤η≤1 are explicit hypotheses. The statements use the reduced seller problem (1) with pm=vp_m=vpm​=v, as the paper does after its Lemma 1; Lemma 1 itself is not posed. The uniform GGG is the clamped CDF. Seller optimality, the baseline equilibrium and the ban value are optimality predicates (IsGreatest over the fees), never sSup or a chosen maximizer.

Conventions and disclosed additions:

  • In parts 1–3, Π∗\Pi^*Π∗ is any real in (0,v−c)(0,v-c)(0,v−c), a generalization of the paper's Π∗\Pi^*Π∗ for a general "well behaved" GGG; the proof uses nothing else.
  • Part 4 is posed for each fixed η∈(0,1)\eta\in(0,1)η∈(0,1): it fails at η=0\eta=0η=0 (ban) when qv≤cqv\le cqv≤c and at η=1\eta=1η=1 (allow). It adds c≥0c\ge 0c≥0, without which the marketplace bans for every μ\muμ at small η\etaη.
  • The milestones on the ban value, the derivative's sign and η∗\eta^*η∗ add v>0v>0v>0, which the proof uses without stating; each is false for some v≤0v\le 0v≤0.
  • ΠL\Pi_LΠL​ and the threshold divide by 1−η1-\eta1−η and are stated only for η<1\eta<1η<1; the game itself has no division.
  • "Prefers" is strict in both directions.

Trivializing formalizations are ruled out: the ban value is not defined as max⁡{ΠH,ΠL}\max\{\Pi_H,\Pi_L\}max{ΠH​,ΠL​}, the thresholds in μ\muμ are quantified after η\etaη and not before, Π∗\Pi^*Π∗ is constrained to (0,v−c)(0,v-c)(0,v−c), and preferences are strict.

A complete development needs elementary real analysis only: derivatives of rational functions, the intermediate value theorem on [0,1)[0,1)[0,1), limits at 111, and the baseline solution of (1) and (3) under uniform switching costs. That baseline solution is reusable for the other missions of this series. Proofs of individual milestones are welcome independently.

Selected references

  • A. Hagiu, J. Wright, Marketplace Leakage, Management Science, 2023. Authors' manuscript dated March 2023. doi:10.1287/mnsc.2023.4757
  • A. Piolatto, F. Schuett, Information vs Competition: How Platform Design Affects Profits and Surplus, TILEC Discussion Paper DP2022-002, 2022 (cited in footnote 8 of the paper for the idea that limiting information exchange reduces transaction value on a platform).
8 thms2 active usersReviewed
Algorithmic Game TheoryEconomicsOperations Research·Captain: mikedeng1

Quality Strategies in Network Markets: Under Potential Maximization the Equilibrium Quality Equates Marginal Cost to n Times the Harmonic-Mean Valuation Times the Uniform Average of ∂f(i,q)/∂qResearch Paper

Motivation

A firm that sells a network good (a platform, a communication service, a standard) faces consumers whose benefit from buying grows with the number of other buyers. When these network effects are strong, the buying decisions form a coordination game: everybody buying and nobody buying can both be equilibria at the same price and quality. The firm's choice of quality then depends on which equilibrium it expects consumers to play.

The classical answer, going back to Spence's analysis of monopoly quality (Spence 1975), is that a monopolist sets quality for the marginal consumer. Under the usual equilibrium-selection criteria (Pareto dominance, focality) the network-good monopolist inherits this logic. L. T. Chan, Quality Strategies in Network Markets (Management Science, 2023; SSRN copy 4203818), asks what happens when the firm instead anticipates potential maximization, a generalization of risk dominance to many players introduced through the potential games of Monderer and Shapley (1996). The answer is a different first-order condition for quality, in which every consumer's valuation and every possible number of buyers matter. This mission formalizes that condition, Proposition 5 of the paper, together with the chain of results in §3 that leads to it.

Setting

There are n≥1n\ge 1n≥1 consumers N={1,…,n}N=\{1,\dots,n\}N={1,…,n}. Consumer iii has a valuation vi>0v_i>0vi​>0, labelled so that v1≥⋯≥vnv_1\ge\dots\ge v_nv1​≥⋯≥vn​. The benefit f(k,q)≥0f(k,q)\ge 0f(k,q)≥0 of the good when kkk consumers buy at quality qqq is weakly increasing in kkk and, in qqq, weakly increasing and differentiable. The firm's cost of quality c(q)≥0c(q)\ge 0c(q)≥0 is weakly increasing and differentiable.

In stage 1 the firm chooses a price p≥0p\ge 0p≥0 and a quality q≥0q\ge 0q≥0. In stage 2 consumers simultaneously decide whether to buy. If B⊆NB\subseteq NB⊆N is the set of buyers, consumer iii receives the payoff (7)

ui(B,p,q)=vif(∣B∣,q)−p  if i∈B,ui(B,p,q)=0  if i∉B,u_i(B,p,q)=v_if(|B|,q)-p\ \text{ if } i\in B,\qquad u_i(B,p,q)=0\ \text{ if } i\notin B,ui​(B,p,q)=vi​f(∣B∣,q)−p  if i∈B,ui​(B,p,q)=0  if i∈/B,

and the firm earns ∣B∣p−c(q)|B|p-c(q)∣B∣p−c(q). A buyer set BBB is a Nash equilibrium of stage 2 if no buyer gains by leaving and no non-buyer gains by joining. The market has strong network effects at qqq when vif(i,q)v_if(i,q)vi​f(i,q) is strictly increasing in iii.

The potential (10) of a buyer set is

Φ(B,p,q)=∑i=1∣B∣f(i,q)−p∑i∈B1vi.\Phi(B,p,q)=\sum_{i=1}^{|B|}f(i,q)-p\sum_{i\in B}\frac1{v_i}.Φ(B,p,q)=i=1∑∣B∣​f(i,q)−pi∈B∑​vi​1​.

A potential maximizer is a buyer set maximizing Φ(⋅,p,q)\Phi(\cdot,p,q)Φ(⋅,p,q) over all subsets of NNN. A firm that anticipates potential maximization earns Π(p,q)=np−c(q)\Pi(p,q)=np-c(q)Π(p,q)=np−c(q) when all consumers buying is a potential maximizer, and −c(q)-c(q)−c(q) otherwise. In Lean these objects are Market, Market.Standing, Market.StrongNE, Market.u, Market.IsNash, Market.pot, Market.IsPotMax and Market.profitPM in the namespace NetworkQuality.PotentialMax.

Formalization targets

Goal: Proposition 5 (p. 15)

Assume the standing assumptions and strong network effects at every q≥0q\ge 0q≥0. If (p∗,q∗)(p^*,q^*)(p∗,q∗) maximizes Π\PiΠ over p≥0p\ge 0p≥0, q≥0q\ge 0q≥0 and q∗>0q^*>0q∗>0, then

c′(q∗)=n(1n∑i=1n1vi)−11n∑i=1n∂f(i,q∗)∂q.c'(q^*)=n\Big(\frac1n\sum_{i=1}^n\frac1{v_i}\Big)^{-1}\frac1n\sum_{i=1}^n\frac{\partial f(i,q^*)}{\partial q}.c′(q∗)=n(n1​i=1∑n​vi​1​)−1n1​i=1∑n​∂q∂f(i,q∗)​.

The statement is the necessary first-order condition. It fixes no functional form for fff or ccc.

Milestones

  1. (11), p. 13. For i∉B−ii\notin B_{-i}i∈/B−i​: ui(B−i∪{i})−ui(B−i)=vi[Φ(B−i∪{i})−Φ(B−i)]u_i(B_{-i}\cup\{i\})-u_i(B_{-i})=v_i[\Phi(B_{-i}\cup\{i\})-\Phi(B_{-i})]ui​(B−i​∪{i})−ui​(B−i​)=vi​[Φ(B−i​∪{i})−Φ(B−i​)], so stage 2 is a weighted potential game.
  2. §3.5, p. 14. Every potential maximizer is a Nash equilibrium.
  3. Proposition 1, p. 10. Under strong network effects the equilibria are exactly {N}\{N\}{N} for p<v1f(1,q)p<v_1f(1,q)p<v1​f(1,q), {∅}\{\emptyset\}{∅} for p>vnf(n,q)p>v_nf(n,q)p>vn​f(n,q), and {N,∅}\{N,\emptyset\}{N,∅} in between.
  4. (12), p. 14. Under strong network effects every potential maximizer is NNN or ∅\emptyset∅, and NNN is one iff p≤(1n∑i1vi)−11n∑if(i,q)=:p(q)p\le\big(\frac1n\sum_i\frac1{v_i}\big)^{-1}\frac1n\sum_i f(i,q)=:p(q)p≤(n1​∑i​vi​1​)−1n1​∑i​f(i,q)=:p(q).
  5. (13), p. 14. v1f(1,q)∑i1vi≤∑if(i,q)≤vnf(n,q)∑i1viv_1f(1,q)\sum_i\frac1{v_i}\le\sum_i f(i,q)\le v_nf(n,q)\sum_i\frac1{v_i}v1​f(1,q)∑i​vi​1​≤∑i​f(i,q)≤vn​f(n,q)∑i​vi​1​.
  6. (14), p. 15. At each quality the maximal profit over prices p≥0p\ge0p≥0 is n p(q)−c(q)n\,p(q)-c(q)np(q)−c(q), attained at p(q)p(q)p(q).

Significance

The result. Proposition 5 replaces the Spence condition c′(q)=n vn ∂f(n,q)/∂qc'(q)=n\,v_n\,\partial f(n,q)/\partial qc′(q)=nvn​∂f(n,q)/∂q (Proposition 4 of the paper) by one in which the marginal consumer's valuation vnv_nvn​ is replaced by the harmonic mean of all valuations, and the equilibrium benefit f(n,q)f(n,q)f(n,q) by its uniform average over the number of buyers. Comparing it with the social optimum (Proposition 3, arithmetic mean of valuations) shows that a firm worried about miscoordination underprovides quality relative to the planner, increasingly so as valuations become more dispersed, and that it tends to undersupply (oversupply) quality when the number of buyers and quality are complements (substitutes) in the benefit fff (p. 15).

Formalizing it. The paper's arguments are in the main text (§3.2 and §3.5) and are short; some steps are only sketched: Proposition 1's proof treats only buyer sets of the form {1,…,k}\{1,\dots,k\}{1,…,k}, and (12) is derived only in the price range with two equilibria. A machine-checked development makes these steps complete, fixes the tie-breaking convention of the selection, and isolates exactly which assumptions the first-order condition uses (none of the convexity the paper assumes). No part of this result has a machine-checked proof at the time of writing.

Difficulty

The objects are finite and the algebra is elementary, so the difficulty lies in bookkeeping that the paper leaves implicit. The obvious reading of Proposition 1, "consumers 111 to kkk buy", is not forced when valuations tie; excluding an arbitrary buyer set of intermediate size needs a comparison between its weakest buyer and its strongest non-buyer, using the ordering of valuations and nonnegativity of fff. The characterization (12) is a statement about maximizers over all 2n2^n2n buyer sets, not just about the two candidates the text compares, so it rests on the reduction "potential maximizer ⇒ Nash equilibrium ⇒ NNN or ∅\emptyset∅". Finally the firm optimizes jointly over price and quality against a discontinuous profit function; the first-order condition only appears after the price has been optimized out at each quality, and it needs differentiability at an interior optimum.

Formalization scope

Consumers are Fin n; the paper's consumer iii is Lean index i−1i-1i−1, while the first argument of fff is a number of buyers in ℕ, so ∑i=1nf(i,q)\sum_{i=1}^n f(i,q)∑i=1n​f(i,q) is ∑ i : Fin n, M.f (i.val + 1) q. v1v_1v1​ and vnv_nvn​ are M.v ⟨0, _⟩ and M.v ⟨n - 1, _⟩. fff and ccc are total functions on ℝ.

The standing assumptions (Market.Standing, pp. 4 and 9) are: n≥1n\ge1n≥1; vi>0v_i>0vi​>0 and vvv antitone; for k≥1k\ge1k≥1 and q≥0q\ge0q≥0, f(k,q)≥0f(k,q)\ge0f(k,q)≥0, fff weakly increasing in kkk and in qqq on [0,∞)[0,\infty)[0,∞), f(k,⋅)f(k,\cdot)f(k,⋅) differentiable on (0,∞)(0,\infty)(0,∞); c≥0c\ge0c≥0 on [0,∞)[0,\infty)[0,∞), weakly increasing there and differentiable on (0,∞)(0,\infty)(0,∞). Statements about a single quality add q≥0q\ge0q≥0, the paper's domain.

Added hypotheses, each disclosed in the items: equilibria are pure (buyer sets); the goal assumes an interior quality q∗>0q^*>0q∗>0 and strong network effects at every q≥0q\ge0q≥0; the "sufficiently convex" cost of p. 4 is dropped, so the goal is the necessary condition only. The profit under potential maximization selects "all buy" at the tie Φ(N)=Φ(∅)\Phi(N)=\Phi(\emptyset)Φ(N)=Φ(∅), as the paper's "if and only if p≤…p\le\dotsp≤…" does.

The profit is defined through the selection (IsPotMax over all buyer sets), not as the reduced formula (14), and the goal's optimality hypothesis ranges over price and quality jointly; defining the profit by (14), or restricting maximizers to {N,∅}\{N,\emptyset\}{N,∅}, would trivialize the mission and is ruled out. Derivatives are deriv, and the standing assumptions guarantee they are genuine at q∗>0q^*>0q∗>0.

Everything needed beyond Mathlib (Finset, deriv, IsLocalMax) is in the single definition file. Proofs of the milestones in any order are welcome, as are proofs of the goal that use the milestones as stepping stones.

Selected references

  • L. T. Chan, Quality Strategies in Network Markets, Management Science, 2023. https://doi.org/10.1287/mnsc.2023.4792 (SSRN version of April 2023: https://ssrn.com/abstract=4203818)
  • D. Monderer and L. S. Shapley, Potential Games, Games and Economic Behavior 14 (1996), 124–143. https://doi.org/10.1006/game.1996.0044
  • A. M. Spence, Monopoly, Quality, and Regulation, Bell Journal of Economics 6 (1975), 417–429. https://www.jstor.org/stable/3003302
8 thms2 active usersReviewed
Graph TheoryQuantum Information·Captain: mikedeng1

A Quantum Approximate Optimization Algorithm II: On v-Regular Graphs the Variance of the Cut Value in the QAOA State Is at Most 2((v−1)^(2p+2)−1)/(v−2) Times the Number of EdgesResearch Paper

Motivation

The Quantum Approximate Optimization Algorithm (QAOA) of Farhi, Goldstone and Gutmann (arXiv:1411.4028, 2014) is a family of shallow quantum circuits for combinatorial optimization, indexed by a depth ppp and 2p2p2p real angles. It has become one of the standard candidate algorithms for near-term quantum hardware, and its MaxCut instance is the usual benchmark on which it is analysed.

The algorithm is run by preparing a quantum state, measuring it in the computational basis to obtain a bit string zzz, and evaluating the classical objective C(z)C(z)C(z). Its analysis in the paper bounds the mean FpF_pFp​ of C(z)C(z)C(z). A mean alone does not say what a single run returns. Section III of the paper supplies the missing piece for MaxCut on bounded-degree graphs: the variance of C(z)C(z)C(z) grows at most linearly in the number of edges, so the measured cut values are concentrated around FpF_pFp​ and a modest number of repetitions reproduces the mean. This mission formalizes that variance bound, equation (30) of the paper, together with the locality and counting facts it rests on.

Setting

Let GGG be a simple graph on vertices {0,…,n−1}\{0,\dots,n-1\}{0,…,n−1} with edge set {⟨jk⟩}\{\langle jk\rangle\}{⟨jk⟩} of size mmm, each edge listed once. Vertices are qubits; a computational basis state is a bit string z∈{0,1}nz\in\{0,1\}^nz∈{0,1}n.

  • The clause operator of an edge is C⟨jk⟩=12(−σjzσkz+1)C_{\langle jk\rangle}=\tfrac12(-\sigma^z_j\sigma^z_k+1)C⟨jk⟩​=21​(−σjz​σkz​+1). It is diagonal: C⟨jk⟩(z)=1C_{\langle jk\rangle}(z)=1C⟨jk⟩​(z)=1 if zj≠zkz_j\ne z_kzj​=zk​ and 000 otherwise.
  • The MaxCut objective is C=∑⟨jk⟩C⟨jk⟩C=\sum_{\langle jk\rangle}C_{\langle jk\rangle}C=∑⟨jk⟩​C⟨jk⟩​; C(z)C(z)C(z) is the number of edges cut by zzz.
  • The mixer is B=∑jσjxB=\sum_j\sigma^x_jB=∑j​σjx​, and U(C,γ)=e−iγCU(C,\gamma)=e^{-i\gamma C}U(C,γ)=e−iγC, U(B,β)=e−iβBU(B,\beta)=e^{-i\beta B}U(B,β)=e−iβB.
  • The initial state is ∣s⟩=2−n/2∑z∣z⟩|s\rangle=2^{-n/2}\sum_z|z\rangle∣s⟩=2−n/2∑z​∣z⟩.
  • For p≥1p\ge1p≥1 and angles γ=(γ1,…,γp)\boldsymbol\gamma=(\gamma_1,\dots,\gamma_p)γ=(γ1​,…,γp​), β=(β1,…,βp)\boldsymbol\beta=(\beta_1,\dots,\beta_p)β=(β1​,…,βp​), the QAOA state is
∣γ,β⟩=U(B,βp) U(C,γp)⋯U(B,β1) U(C,γ1) ∣s⟩.|\boldsymbol\gamma,\boldsymbol\beta\rangle=U(B,\beta_p)\,U(C,\gamma_p)\cdots U(B,\beta_1)\,U(C,\gamma_1)\,|s\rangle .∣γ,β⟩=U(B,βp​)U(C,γp​)⋯U(B,β1​)U(C,γ1​)∣s⟩.
  • Measuring it gives zzz with probability P(z)=∣⟨z∣γ,β⟩∣2P(z)=|\langle z|\boldsymbol\gamma,\boldsymbol\beta\rangle|^2P(z)=∣⟨z∣γ,β⟩∣2. The mean is Fp(γ,β)=⟨γ,β∣C∣γ,β⟩=∑zP(z)C(z)F_p(\boldsymbol\gamma,\boldsymbol\beta)=\langle\boldsymbol\gamma,\boldsymbol\beta|C|\boldsymbol\gamma,\boldsymbol\beta\rangle=\sum_zP(z)C(z)Fp​(γ,β)=⟨γ,β∣C∣γ,β⟩=∑z​P(z)C(z) and the variance is
⟨γ,β∣C2∣γ,β⟩−⟨γ,β∣C∣γ,β⟩2.\langle\boldsymbol\gamma,\boldsymbol\beta|C^2|\boldsymbol\gamma,\boldsymbol\beta\rangle-\langle\boldsymbol\gamma,\boldsymbol\beta|C|\boldsymbol\gamma,\boldsymbol\beta\rangle^2 .⟨γ,β∣C2∣γ,β⟩−⟨γ,β∣C∣γ,β⟩2.
  • The light cone (the paper's subgraph g(j,k)g(j,k)g(j,k)) of an edge is the set of qubits at graph distance at most ppp from jjj or from kkk.

In Lean these are edgeCut, cutValue, qaoaState, expectCut and edgeTerm in the shared FarhiQAOA.Cubic setting, and edgePairs, cutVariance, edgePairTerm, edgeCov and lightCone in FarhiQAOA.Concentration, all built on the published QAlgorithms definitions of qubit registers, measurement probabilities, the uniform superposition, the problem Hamiltonian and the QAOA unitary.

Formalization targets

Goal: the variance bound (30)

For a vvv-regular graph with v≥3v\ge3v≥3, every depth p≥1p\ge1p≥1 and all real angles,

⟨γ,β∣C2∣γ,β⟩−⟨γ,β∣C∣γ,β⟩2≤2[(v−1)2p+2−1(v−1)−1]⋅m.\langle\boldsymbol\gamma,\boldsymbol\beta|C^2|\boldsymbol\gamma,\boldsymbol\beta\rangle-\langle\boldsymbol\gamma,\boldsymbol\beta|C|\boldsymbol\gamma,\boldsymbol\beta\rangle^2\le2\left[\frac{(v-1)^{2p+2}-1}{(v-1)-1}\right]\cdot m .⟨γ,β∣C2∣γ,β⟩−⟨γ,β∣C∣γ,β⟩2≤2[(v−1)−1(v−1)2p+2−1​]⋅m.

The constant is the paper's, as printed.

Milestones

  1. (27)–(28). The variance is the double sum, over ordered pairs of edges, of the covariances ⟨C⟨jk⟩C⟨j′k′⟩⟩−⟨C⟨jk⟩⟩⟨C⟨j′k′⟩⟩\langle C_{\langle jk\rangle}C_{\langle j'k'\rangle}\rangle-\langle C_{\langle jk\rangle}\rangle\langle C_{\langle j'k'\rangle}\rangle⟨C⟨jk⟩​C⟨j′k′⟩​⟩−⟨C⟨jk⟩​⟩⟨C⟨j′k′⟩​⟩.
  2. Disjoint light cones. If g(j,k)g(j,k)g(j,k) and g(j′,k′)g(j',k')g(j′,k′) share no qubit, the covariance of the two edges is 000.
  3. (26). For maximum degree v≥3v\ge3v≥3, ∣g(j,k)∣≤2[((v−1)p+1−1)/((v−1)−1)]|g(j,k)|\le2\big[((v-1)^{p+1}-1)/((v-1)-1)\big]∣g(j,k)∣≤2[((v−1)p+1−1)/((v−1)−1)].
  4. (29). For maximum degree v≥3v\ge3v≥3, at most 2[((v−1)2p+2−1)/((v−1)−1)]2\big[((v-1)^{2p+2}-1)/((v-1)-1)\big]2[((v−1)2p+2−1)/((v−1)−1)] edges ⟨j′k′⟩\langle j'k'\rangle⟨j′k′⟩ have g(j′,k′)g(j',k')g(j′,k′) meeting g(j,k)g(j,k)g(j,k).
  5. Summand bound. Each covariance is at most 111 in absolute value.

Companions, printed in the paper for v=2v=2v=2: ∣g(j,k)∣≤2p+2|g(j,k)|\le2p+2∣g(j,k)∣≤2p+2 and at most 4p+44p+44p+4 meeting edges.

Significance

The result. For fixed vvv and ppp the bound makes the standard deviation of the cut value O(m)O(\sqrt m)O(m​), while the mean is of order mmm. The algorithm's output is therefore concentrated: a single measurement already returns a cut close to FpF_pFp​ in relative terms, and the paper uses this to argue that the sample mean of polynomially many runs estimates FpF_pFp​ within 111. The same locality structure, that each edge term depends only on a bounded neighbourhood, is what makes FpF_pFp​ classically computable for fixed ppp and underlies later analyses of QAOA on bounded-degree and random regular graphs.

Formalizing it. The bound is proved in the paper in a few lines; none of it is machine-checked. A formal proof has to make the paper's operator locality argument precise (the conjugated clause operator is supported on the light cone, and the initial state is a product state), connect it to the computational-basis probabilities, and carry out the graph counting with explicit constants. The resulting locality lemma for QAOA circuits on graphs is reusable for any statement about QAOA at fixed depth.

Difficulty

The algebra of (28) and the bound by 111 are routine. The substance is in two places.

The vanishing of a covariance requires showing that U†C⟨jk⟩UU^\dagger C_{\langle jk\rangle}UU†C⟨jk⟩​U, for the full depth-ppp QAOA unitary UUU, acts only on the qubits of g(j,k)g(j,k)g(j,k), and that for two operators acting on disjoint sets of qubits the expectation in the product state ∣s⟩|s\rangle∣s⟩ factorizes. Both are statements about operators on (C2)⊗n(\mathbb C^2)^{\otimes n}(C2)⊗n, whereas the objects in the statement are probabilities of computational basis states; the formal proof must pass between the two. Matrix exponentials of commuting sums have to be split into commuting factors, which in Lean means working with NormedSpace.exp on matrices indexed by bit strings.

The counting (26) and (29) is a ball-growth bound in a graph of bounded degree. The paper's count, via (26) with ppp replaced by 2p+12p+12p+1, bounds vertices, while (29) counts edges; the passage between the two, and the extended graph distance on disconnected graphs, must be made exact with the printed constant.

Formalization scope

  • Qubits are indexed by Fin n, bit strings are Fin n → Bool with true the bit 111, and states live in EuclideanSpace ℂ (Fin n → Bool).
  • All expectations are sums ∑zP(z) f(z)\sum_zP(z)\,f(z)∑z​P(z)f(z) over the computational basis. This equals the operator expectation in the paper because every operator involved is diagonal.
  • Each edge is counted once (pairs j<kj<kj<k), in mmm, in CCC, in the double sum of (28) and in the count (29).
  • Graph distance is the extended distance SimpleGraph.edist (+∞+\infty+∞ between components).
  • The goal is stated for vvv-regular graphs with v≥3v\ge3v≥3, the domain of the printed formula. The milestones (26) and (29) assume only maximum degree ≤v\le v≤v, as printed there.
  • The angles range over all of Rp\mathbb R^pRp. The paper restricts them to [0,2π][0,2\pi][0,2π] and [0,π][0,\pi][0,π] by periodicity; the bound does not depend on that restriction.
  • The QAOA state is the state itself, not an arbitrary vector: normalisation of the measurement probabilities follows from unitarity and is part of what must be proved.

A variance defined with two different states, or a probability vector that is not normalised, would make the goal vacuous or false; the definitions use the single state ∣γ,β⟩|\boldsymbol\gamma,\boldsymbol\beta\rangle∣γ,β⟩ throughout.

Out of scope: the O(m)O(\sqrt m)O(m​) standard-deviation reading and the sampling claim after (30), which carry no explicit constant; the measurement-and-repetition procedure, running time, circuit depth and the classical search for the angles.

Welcome contributions: a general light-cone lemma for conjugation by products of exponentials of local Hamiltonians on qubits; factorization of product-state expectations over disjoint qubit sets; ball-size bounds in graphs of bounded degree.

Selected references

  • E. Farhi, J. Goldstone, S. Gutmann, A Quantum Approximate Optimization Algorithm, arXiv preprint, 2014. https://arxiv.org/abs/1411.4028 (v1; Section III, pp. 6–7)
  • E. Farhi, J. Goldstone, S. Gutmann, M. Sipser, Quantum Computation by Adiabatic Evolution, arXiv preprint, 2000. https://arxiv.org/abs/quant-ph/0001106
14 thms2 active usersReviewed
Operations ResearchStatistics·Captain: mikedeng1

Design and Analysis of Switchback Experiments 1: With m-Period Carryover and T = nm, n ≥ 4, the Unique Minimax-Optimal Regular Design Flips Fair Coins at 1, 2m+1, 3m+1, …, (n−2)m+1Research Paper

Motivation

Online platforms such as ride-hailing, delivery and advertising marketplaces routinely test new pricing or matching policies on a single market. Randomizing over users is often impossible there: every rider and driver in a city shares one supply pool, so treating some users and not others contaminates the control group. The standard alternative is a switchback experiment: the whole market alternates between the new policy (treatment) and the current one (control) over time, and the effect is estimated from the time series of outcomes. Switchbacks are in production use at ride-hailing and delivery firms (see the industry references in Bojinov, Simchi-Levi and Zhao, 2023).

The design question is when to switch. Switching often produces many nearly independent observations, but a policy change has carryover effects: an outcome is affected by the policy of the last few periods, not only the current one, so frequent switching leaves few periods whose recent history is purely treated or purely control. Bojinov, Simchi-Levi and Zhao formalize this trade-off in a design-based, minimax framework and solve it in closed form. This mission formalizes that solution.

Setting

Time is divided into periods t∈[T]={1,…,T}t\in[T]=\{1,\dots,T\}t∈[T]={1,…,T}. An assignment path is w1:T∈{0,1}Tw_{1:T}\in\{0,1\}^Tw1:T​∈{0,1}T, with wt=1w_t=1wt​=1 meaning treatment. For every period ttt and every path there is a fixed, unknown potential outcome Yt(w1:T)∈RY_t(w_{1:T})\in\mathbb RYt​(w1:T​)∈R; the collection is Y\mathbb YY. Randomness comes only from the random path W1:TW_{1:T}W1:T​.

Three assumptions restrict Y\mathbb YY:

  1. Non-anticipation: YtY_tYt​ does not depend on wt+1:Tw_{t+1:T}wt+1:T​.
  2. mmm-carryover: for t≥m+1t\ge m+1t≥m+1, YtY_tYt​ does not depend on w1:t−m−1w_{1:t-m-1}w1:t−m−1​.
  3. Boundedness: ∣Yt(w)∣≤B|Y_t(w)|\le B∣Yt​(w)∣≤B for every ttt and every path, with B>0B>0B>0.

The set of outcomes satisfying all three is Y\mathcal YY. Under 1–2, Yt(1m+1)Y_t(\mathbf 1_{m+1})Yt​(1m+1​) and Yt(0m+1)Y_t(\mathbf 0_{m+1})Yt​(0m+1​) denote the outcome after m+1m+1m+1 consecutive treatments, respectively controls. The estimand is the average lag-mmm effect

τm(Y)=1T−m∑t=m+1T[Yt(1m+1)−Yt(0m+1)].\tau_m(\mathbb Y)=\frac1{T-m}\sum_{t=m+1}^T\bigl[Y_t(\mathbf 1_{m+1})-Y_t(\mathbf 0_{m+1})\bigr].τm​(Y)=T−m1​t=m+1∑T​[Yt​(1m+1​)−Yt​(0m+1​)].

A regular switchback design (T,Q)(\mathbb T,\mathbb Q)(T,Q) fixes randomization points T={t0=1<t1<⋯<tK}⊆[T]\mathbb T=\{t_0=1<t_1<\dots<t_K\}\subseteq[T]T={t0​=1<t1​<⋯<tK​}⊆[T], K≥1K\ge1K≥1, and probabilities qtk∈(0,1)q_{t_k}\in(0,1)qtk​​∈(0,1). At each tkt_ktk​ an independent coin with success probability qtkq_{t_k}qtk​​ is flipped, and its outcome is applied to the periods tk,…,tk+1−1t_k,\dots,t_{k+1}-1tk​,…,tk+1​−1 (with tK+1=T+1t_{K+1}=T+1tK+1​=T+1). The Horvitz–Thompson estimator τ^m\hat\tau_mτ^m​ weights the outcome of each period whose window wt−m:tw_{t-m:t}wt−m:t​ is all ones (all zeros) by the inverse probability of that event. Its risk is the mean squared error

r(ηT,Q,Y)=E[(τ^m−τm(Y))2],r(\eta_{\mathbb T,\mathbb Q},\mathbb Y)=\mathbb E\bigl[(\hat\tau_m-\tau_m(\mathbb Y))^2\bigr],r(ηT,Q​,Y)=E[(τ^m​−τm​(Y))2],

where the expectation is over the law ηT,Q\eta_{\mathbb T,\mathbb Q}ηT,Q​ of the path. A design is minimax optimal if it minimizes the worst-case risk max⁡Y∈Yr(ηT,Q,Y)\max_{\mathbb Y\in\mathcal Y}r(\eta_{\mathbb T,\mathbb Q},\mathbb Y)maxY∈Y​r(ηT,Q​,Y) over all regular designs.

Formalization targets

Goal: the optimal design (Theorem 2 with Theorem 1)

  1. If m=0m=0m=0 and T≥2T\ge2T≥2, the unique minimax-optimal regular design has T∗={1,2,…,T}\mathbb T^*=\{1,2,\dots,T\}T∗={1,2,…,T} and fair coins.
  2. If m≥1m\ge1m≥1 and T=nmT=nmT=nm for an integer n≥4n\ge4n≥4, the unique minimax-optimal regular design is
T∗={1, 2m+1, 3m+1, …, (n−2)m+1},q0=⋯=qK=12.\mathbb T^*=\{1,\ 2m+1,\ 3m+1,\ \dots,\ (n-2)m+1\},\qquad q_0=\dots=q_K=\tfrac12 .T∗={1, 2m+1, 3m+1, …, (n−2)m+1},q0​=⋯=qK​=21​.

The goal asserts both existence and uniqueness, and it compares against every regular design with arbitrary probabilities in (0,1)(0,1)(0,1).

Milestones

In attack order:

  1. Unbiasedness of the estimator (Proposition 1).
  2. The moments of the error terms (Lemmas EC.3 and EC.4).
  3. The adversary's best responses Y±≡±B\mathbb Y^\pm\equiv\pm BY±≡±B (Lemma 1).
  4. The product inequality
1∏iqi+1∏i(1−qi)≥2n+1\frac1{\prod_i q_i}+\frac1{\prod_i(1-q_i)}\ge2^{n+1}∏i​qi​1​+∏i​(1−qi​)1​≥2n+1

(Lemma EC.2), and fair coins (Theorem 1). 5. The fair-coin moments (Lemma EC.6). 6. The structure of optimal point sets (Lemmas EC.7 and EC.8). 7. The closed-form risk (EC.8) (Lemma EC.9). 8. The integer optimum of the subset selection objective (7) at T=nmT=nmT=nm:

min⁡T{4∑k=0K(tk+1−tk)2+8m(tK−t1)+4m2K−4m2+4∑k=1K−1[(m−tk+1+tk)+]2}=16mT−32m2,\min_{\mathbb T}\Bigl\{4\sum_{k=0}^{K}(t_{k+1}-t_k)^2+8m(t_K-t_1)+4m^2K-4m^2+4\sum_{k=1}^{K-1}\bigl[(m-t_{k+1}+t_k)^+\bigr]^2\Bigr\}=16mT-32m^2,Tmin​{4k=0∑K​(tk+1​−tk​)2+8m(tK​−t1​)+4m2K−4m2+4k=1∑K−1​[(m−tk+1​+tk​)+]2}=16mT−32m2,

attained only at T∗\mathbb T^*T∗.

Companion statements

These are not milestones:

  • Lemma EC.5 (the moments at Y±\mathbb Y^\pmY± for general Q\mathbb QQ).
  • Theorem 2's equivalence of (6) and (7), restricted to the goal's horizons.
  • The optimal worst-case risk (16mT−32m2)B2/(T−m)2(16mT-32m^2)B^2/(T-m)^2(16mT−32m2)B2/(T−m)2.
  • Lemma EC.10 on the gaps of minimizers of (7).

Significance

The result. Theorem 2 turns a design question into a rule: randomize once every mmm periods, except that the first and the last epoch last 2m2m2m periods. The rule depends only on the carryover order mmm, not on BBB or on the outcome model. A practitioner who chooses the period length close to the carryover duration gets a design that is robust to a short carryover, since the optimal designs for m=0m=0m=0 and m=1m=1m=1 nearly coincide. The fair-coin result (Theorem 1) keeps the inverse-probability weights at powers of 222, which avoids the instability of Horvitz–Thompson estimators with extreme propensities. The same design underlies the paper's asymptotic normality result and its confidence intervals (mission 2 of this series).

Formalizing it. The results are proved in the paper; none of them is machine-checked. The formalization has to make the paper's finite-sample expansion of the risk exact, establish its integer optimum, and connect it to a genuine minimax statement over all regular designs, not only over fair-coin designs. It also pins down where the printed statements need hypotheses. As printed, the first sentence of Theorem 2 and Lemma EC.7 fail for short horizons (at m=2m=2m=2, T=4T=4T=4 the minimax-optimal point sets are {1,2}\{1,2\}{1,2} and {1,4}\{1,4\}{1,4}, while (7) is minimized by {1,3}\{1,3\}{1,3}). Their corrected forms are stated here. The strictness part of Lemma 1 fails at m=0m=0m=0 (with T=2T=2T=2, T={1,2}\mathbb T=\{1,2\}T={1,2}, fair coins, outcomes BBB in period 1 and −B-B−B in period 2 tie with Y+\mathbb Y^+Y+); it is stated for m≥1m\ge1m≥1.

Difficulty

The risk is a quadratic form in the reduced outcomes whose coefficients depend on the combinatorics of the point set. The coefficients are products of inverse probabilities over the determining randomization points of each window {t−m,…,t}\{t-m,\dots,t\}{t−m,…,t} and over the overlaps of two windows. Writing the risk in closed form requires a case analysis of how windows of length m+1m+1m+1 meet the blocks of the design. The paper's closed form (EC.8) holds only under three structural conditions on the points. Showing that every optimal design satisfies them (Lemmas EC.7, EC.8) needs exchange arguments that remove or move a point while keeping the design regular, and these break down at short horizons.

The step from the closed form to the explicit optimum is an integer program. The paper solves its continuous relaxation and observes that the relaxed optimum is integral. A formal proof must handle every point set, including those that violate the structural conditions and for which (EC.8) is not the risk. The obvious approach of minimizing (EC.8) directly is therefore not enough: the minimax optimum must also be compared with designs outside the class where (EC.8) applies.

Formalization scope

Lean represents a path as a function N→Bool\mathbb N\to\mathrm{Bool}N→Bool of which only the indices 1,…,T1,\dots,T1,…,T are read, potential outcomes as a function of the period and the whole path, and the point set as a Finset ℕ. Probabilities are indexed by the randomization point (qtkq_{t_k}qtk​​). The expectation under the law of a regular design is the finite sum over the 2K+12^{K+1}2K+1 coin vectors.

Committed conventions:

  • the lag of the estimand equals the carryover order (p=mp=mp=m), as throughout §3;
  • m<Tm<Tm<T;
  • every probability lies in (0,1)(0,1)(0,1), so the estimator's denominators are positive;
  • tK+1=T+1t_{K+1}=T+1tK+1​=T+1;
  • (a)+=max⁡{a,0}(a)^+=\max\{a,0\}(a)+=max{a,0}.

Minimax optimality is the statement "for every regular design and every admissible Y\mathbb YY there is an admissible Y′\mathbb Y'Y′ whose risk under the other design is at least the risk of Y\mathbb YY under this one". No supremum over outcome functions is ever taken. Y±\mathbb Y^\pmY± are the constant outcome functions ±B\pm B±B.

Added hypotheses.

  • The case m=0m=0m=0 of the goal requires T≥2T\ge2T≥2, the page's condition that KKK be positive.
  • Lemma EC.7 assumes m=0m=0m=0 or T=nmT=nmT=nm with n≥4n\ge4n≥4.
  • The strictness clause of Lemma 1 assumes m≥1m\ge1m≥1 (it is false at m=0m=0m=0), and its "Y≠Y±\mathbb Y\ne\mathbb Y^\pmY=Y±" is read on the reduced outcomes (Yt(1m+1),Yt(0m+1))t≥m+1(Y_t(\mathbf 1_{m+1}),Y_t(\mathbf 0_{m+1}))_{t\ge m+1}(Yt​(1m+1​),Yt​(0m+1​))t≥m+1​.
  • Lemmas EC.7–EC.9 are stated for fair coins against Y±\mathbb Y^\pmY±, as in the paper's EC.3.5.
  • Lemma EC.10 is read for minimizers of (7).

Not a trivialization. The goal mentions neither the objective (7) nor the structural conditions, nor the closed form (EC.8). It is stated against the full class of regular designs and asserts existence. A proof cannot replace it by "T∗\mathbb T^*T∗ minimizes (7)" (that is a milestone), and Theorem 1 is not vacuous because the goal asserts that optimal designs exist.

A complete development needs finite expectations over independent coin flips, the risk expansion as a quadratic form, and an integer optimization argument. The first two are reusable for the second mission of this series and for other design-based analyses of Horvitz–Thompson estimators. Proofs of any milestone, and of the companion statements, are welcome independently.

Selected references

  • I. Bojinov, D. Simchi-Levi, J. Zhao, Design and Analysis of Switchback Experiments, Management Science 69(7), 2023; arXiv:2009.00148v4, 2025. https://arxiv.org/abs/2009.00148
  • I. Bojinov, N. Shephard, Time Series Experiments and Causal Estimands: Exact Randomization Tests and Trading, Journal of the American Statistical Association 114(528), 2019. https://doi.org/10.1080/01621459.2018.1527225
  • C.-F. J. Wu, On the Robustness and Efficiency of Some Randomized Designs, Annals of Statistics 9(6), 1981. https://doi.org/10.1214/aos/1176345640
  • D. G. Horvitz, D. J. Thompson, A Generalization of Sampling Without Replacement from a Finite Universe, Journal of the American Statistical Association 47(260), 1952. https://doi.org/10.1080/01621459.1952.10483446
14 thms2 active usersReviewed
Graph TheoryQuantum Information·Captain: mikedeng1

A Quantum Approximate Optimization Algorithm I: At p = 1 the QAOA Expected Cut Is at Least 0.6924 Times the Maximum Cut on Every Connected 3-Regular GraphResearch Paper

Motivation

The quantum approximate optimization algorithm (QAOA) of Farhi, Goldstone and Gutmann (arXiv:1411.4028, 2014) is a family of shallow quantum circuits, indexed by an integer depth p≥1p \ge 1p≥1, that prepare states whose computational-basis measurements are biased toward good solutions of a combinatorial optimization problem. It became the standard candidate for optimization on near-term quantum hardware, and its performance on MaxCut is the benchmark against which later analyses and experiments are measured.

The original paper proves one concrete performance guarantee: at depth p=1p = 1p=1, on 3-regular graphs, the algorithm achieves an approximation ratio of at least 0.69240.69240.6924. Short timeline:

  • 2014, Farhi–Goldstone–Gutmann: the p=1p = 1p=1 bound 0.69240.69240.6924 on 3-regular graphs, by reducing the expected cut to three subgraph functions and a numerical minimization (arXiv:1411.4028).
  • 2018, Wang, Hadfield, Jiang and Rieffel: closed-form expressions for the p=1p = 1p=1 MaxCut expectation on arbitrary graphs (arXiv:1706.02998, Phys. Rev. A 97, 022304).
  • Classically, Goemans and Williamson's semidefinite rounding achieves 0.8780.8780.878 on all graphs (J. ACM 42, 1995), so the p=1p = 1p=1 guarantee is, as the paper itself notes, below the best classical algorithms.

Setting

A graph GGG has vertex set {0,…,n−1}\{0, \dots, n-1\}{0,…,n−1}, and 3-regular means every vertex has exactly three neighbours. Each vertex carries a qubit; a computational basis state is a bit string z=z0⋯zn−1z = z_0 \cdots z_{n-1}z=z0​⋯zn−1​. For an edge ⟨jk⟩\langle jk\rangle⟨jk⟩ the clause C⟨jk⟩(z)C_{\langle jk\rangle}(z)C⟨jk⟩​(z) is 111 if zj≠zkz_j \neq z_kzj​=zk​ and 000 otherwise, and the cut objective is

C(z)=∑⟨jk⟩C⟨jk⟩(z),C(z) = \sum_{\langle jk\rangle} C_{\langle jk\rangle}(z),C(z)=⟨jk⟩∑​C⟨jk⟩​(z),

the number of edges cut by the bipartition zzz defines. As an operator, CCC is diagonal in the computational basis.

Let B=∑jσjxB = \sum_j \sigma^x_jB=∑j​σjx​ and let ∣s⟩=2−n/2∑z∣z⟩|s\rangle = 2^{-n/2}\sum_z |z\rangle∣s⟩=2−n/2∑z​∣z⟩. For angles γ,β∈R\gamma, \beta \in \mathbb Rγ,β∈R, the depth-one QAOA state and its expected cut are

∣γ,β⟩=e−iβB e−iγC ∣s⟩,F1(γ,β)=⟨γ,β∣ C ∣γ,β⟩=∑z∣⟨z∣γ,β⟩∣2 C(z).|\gamma,\beta\rangle = e^{-i\beta B}\, e^{-i\gamma C}\, |s\rangle, \qquad F_1(\gamma,\beta) = \langle\gamma,\beta|\,C\,|\gamma,\beta\rangle = \sum_z |\langle z|\gamma,\beta\rangle|^2\, C(z).∣γ,β⟩=e−iβBe−iγC∣s⟩,F1​(γ,β)=⟨γ,β∣C∣γ,β⟩=z∑​∣⟨z∣γ,β⟩∣2C(z).

The angles may be restricted to γ∈[0,2π]\gamma \in [0, 2\pi]γ∈[0,2π], β∈[0,π]\beta \in [0, \pi]β∈[0,π] by periodicity.

Two kinds of small subgraphs govern the analysis. A crossed square is a set of four vertices spanning exactly five edges (a 4-cycle with one diagonal). An isolated triangle is a triangle whose three outgoing edges end on three distinct vertices. SSS and TTT denote their numbers in GGG. Finally, for an edge ⟨jk⟩\langle jk\rangle⟨jk⟩ the subgraph type is the shape of the edge together with its adjacent edges; in a 3-regular graph there are three types g4,g5,g6g_4, g_5, g_6g4​,g5​,g6​, according as jjj and kkk have two, one or no common neighbours. Each type ggg defines a function fg(γ,β)f_g(\gamma, \beta)fg​(γ,β), the expectation of the clause of the central edge in the depth-one QAOA state of the subgraph ggg alone.

Formalization targets

Goal: the depth-one ratio on cubic graphs

For every connected 3-regular graph GGG there are angles γ∈[0,2π]\gamma \in [0, 2\pi]γ∈[0,2π], β∈[0,π]\beta \in [0, \pi]β∈[0,π] with

F1(γ,β)  ≥  0.6924⋅C(z)for every bit string z,F_1(\gamma, \beta) \;\ge\; 0.6924 \cdot C(z) \qquad \text{for every bit string } z,F1​(γ,β)≥0.6924⋅C(z)for every bit string z,

that is, M1=max⁡γ,βF1≥0.6924⋅max⁡zC(z)M_1 = \max_{\gamma,\beta} F_1 \ge 0.6924 \cdot \max_z C(z)M1​=maxγ,β​F1​≥0.6924⋅maxz​C(z). The constant is exactly the paper's 0.69240.69240.6924, and K4K_4K4​ is included.

Milestones

The paper's argument passes through the following claims, each a milestone:

  1. At p=1p = 1p=1 each edge term of a 3-regular graph equals fg4f_{g_4}fg4​​, fg5f_{g_5}fg5​​ or fg6f_{g_6}fg6​​ according to its subgraph type (p. 5).
  2. In a connected 3-regular graph other than K4K_4K4​, there are SSS edges of type g4g_4g4​ and 4S+3T4S + 3T4S+3T of type g5g_5g5​ (pp. 8–9).
  3. Equation (32): F1=Sfg4+(4S+3T)fg5+(3n/2−5S−3T)fg6F_1 = S f_{g_4} + (4S + 3T) f_{g_5} + (3n/2 - 5S - 3T) f_{g_6}F1​=Sfg4​​+(4S+3T)fg5​​+(3n/2−5S−3T)fg6​​ (p. 9).
  4. 3T+4S≤n3T + 4S \le n3T+4S≤n (p. 8).
  5. Every cut has at most 3n/2−S−T3n/2 - S - T3n/2−S−T edges (p. 9).
  6. For s,t≥0s, t \ge 0s,t≥0 with 4s+3t≤14s + 3t \le 14s+3t≤1, the scaled ratio (35) M1(1,s,t)/(3/2−s−t)M_1(1, s, t)/(3/2 - s - t)M1​(1,s,t)/(3/2−s−t) is at least 0.69240.69240.6924 (p. 9).
  7. On K4K_4K4​ the depth-one ratio exceeds 0.69240.69240.6924 (p. 8).

Significance

The result is the first rigorous worst-case guarantee for the QAOA, and the method behind it (the expected objective is a sum of local terms, each depending only on a bounded neighbourhood of an edge) is the template for every later analysis of fixed-depth QAOA on bounded-degree graphs. It shows that a depth-one circuit, of depth independent of nnn, achieves a constant approximation ratio uniformly over an infinite class of graphs.

The paper's proof is complete apart from one numerical step: the minimization of (35) is reported as a numerical evaluation. No machine-checked version of the result exists. A formal proof requires an exact evaluation of the three subgraph functions at suitable angles, an exact combinatorial count of edge types, and a certified inequality in place of the numerical minimization. The margin is small: the exact minimum is 12+133≈0.692450\tfrac12 + \tfrac{1}{3\sqrt 3} \approx 0.69245021​+33​1​≈0.692450, about 5⋅10−55 \cdot 10^{-5}5⋅10−5 above the paper's constant.

Difficulty

The light-cone reduction (milestone 1) is the conceptual step: one must show that conjugating a two-qubit clause by e−iγCe^{-i\gamma C}e−iγC and e−iβBe^{-i\beta B}e−iβB leaves an operator that involves only the qubits adjacent to the edge, and that the remaining qubits contribute a factor of one. In matrix form on 2n2^n2n-dimensional space this is a statement about tensor-product structure that the definitions do not make visible. The combinatorial count (milestones 2–4) needs a careful case analysis of how triangles sit in a cubic graph, including the exceptional graph K4K_4K4​, where the count fails. The final inequality (milestone 6) has a margin of order 10−410^{-4}10−4, so crude bounds on the trigonometric functions fgf_{g}fg​ do not suffice.

Formalization scope

  • Representation. Bit strings are Qubits n = Fin n → Bool from the published QAlgorithms layer, with true the bit 111. States are vectors in EuclideanSpace ℂ (Qubits n); ∣s⟩|s\rangle∣s⟩, CCC as a diagonal operator, BBB and the QAOA unitary are the published definitions uniformState, problemHamiltonian, qaoaMixer, qaoaUnitary. The latter takes mixer angles before phase angles.
  • Conventions. Graphs are SimpleGraph (Fin n) with decidable adjacency; "3-regular" is IsRegularOfDegree 3. The cut counts each edge once. F1F_1F1​ is defined as ∑zPr⁡[z] C(z)\sum_z \Pr[z]\,C(z)∑z​Pr[z]C(z), which equals ⟨γ,β∣C∣γ,β⟩\langle\gamma,\beta|C|\gamma,\beta\rangle⟨γ,β∣C∣γ,β⟩ because CCC is diagonal. The constant 0.69240.69240.6924 is the exact rational 1731/25001731/25001731/2500.
  • Added hypotheses. Milestones 2 and 3 assume GGG connected and n≠4n \ne 4n=4, which excludes K4K_4K4​, the case the paper sets aside on p. 8. No other hypothesis is added.
  • Out of scope. The goal concerns the expected cut at good angles, which is what the paper's analysis bounds. Measuring the state, repeating, and the claim that a cut of that size is then found with high probability are not formalized; nor are running time, circuit depth, or the search for optimal angles.
  • Ruled-out trivializations. A cut objective counting each edge twice, or an unnormalized state, would leave the ratio statement formally intact but about the wrong objects; the setting counts each edge once and uses the normalized uniform superposition. The goal quantifies over all bit strings, so it cannot be satisfied by bounding a particular cut.
  • Contributions welcome. Reusable lemmas on the light-cone locality of conjugated Pauli operators, closed forms of the depth-one MaxCut expectation, and counting lemmas for triangles in cubic graphs all serve beyond this mission.

Selected references

  • E. Farhi, J. Goldstone, S. Gutmann, A Quantum Approximate Optimization Algorithm, arXiv preprint, 2014. https://arxiv.org/abs/1411.4028
  • Z. Wang, S. Hadfield, Z. Jiang, E. G. Rieffel, Quantum approximate optimization algorithm for MaxCut: A fermionic view, Phys. Rev. A 97, 022304, 2018. https://arxiv.org/abs/1706.02998
  • M. X. Goemans, D. P. Williamson, Improved approximation algorithms for maximum cut and satisfiability problems using semidefinite programming, J. ACM 42(6), 1995. https://doi.org/10.1145/227683.227684
14 thms2 active usersReviewed
EconomicsOperations ResearchProbability·Captain: mikedeng1

Gathering Information before Negotiation I: The Buyer's Optimal Information Acquisition Increases with Its Bargaining Power and First Increases, Then Decreases with the Seller's CostResearch Paper

Motivation

Many transactions are priced by negotiation rather than posted prices: business-to-business procurement, used cars, real estate, the purchase of a company. Before negotiating, a buyer can often spend effort finding out how much the object is actually worth to it, through inspections, trials, consultants or market research. That information serves two purposes. It improves the joint decision of whether to trade at all, and it changes how the surplus of trade is split. Liang Guo's Gathering Information before Negotiation (SSRN 3930651; Management Science, doi:10.1287/mnsc.2022.4360) studies how a buyer's endogenous information acquisition interacts with the two primitives of bargaining: the parties' bargaining powers and the seller's cost.

The first finding of the paper, Proposition 1, concerns how much information the buyer gathers. Its comparative statics in the seller's cost are non-monotone, and this non-monotonicity drives the later results of the paper, in which a seller can be worse off with more bargaining power or a lower cost. This mission formalizes Proposition 1 in its general form, for an arbitrary prior. A companion mission (Gathering Information before Negotiation II) formalizes the payoff reversals under a uniform prior and a linear information cost.

Setting

A seller can supply one unit of a product at cost ccc. A buyer's true value uuu of the unit is unknown to both parties; the common prior law of uuu is FFF, with a finite mean u^>0\hat u>0u^>0.

Before negotiating, the buyer acquires a truth-or-noise signal www: with probability α\alphaα the signal equals uuu, and with probability 1−α1-\alpha1−α it is an independent draw from FFF. Because the parties are risk neutral, only the buyer's posterior mean

v≡αw+(1−α)u^v\equiv\alpha w+(1-\alpha)\hat uv≡αw+(1−α)u^

matters. Ex ante, vvv has the distribution function

G(v,α)≡F ⁣(v−(1−α)u^α).G(v,\alpha)\equiv F\!\left(\frac{v-(1-\alpha)\hat u}{\alpha}\right).G(v,α)≡F(αv−(1−α)u^​).

The buyer chooses the informativeness α\alphaα in an interval [α‾,αˉ]⊆(0,1)[\underline\alpha,\bar\alpha]\subseteq(0,1)[α​,αˉ]⊆(0,1) at a cost k(α)k(\alpha)k(α) that is increasing, with k(α‾)=0k(\underline\alpha)=0k(α​)=0 and k(αˉ)>0k(\bar\alpha)>0k(αˉ)>0.

After observing vvv, the parties trade exactly when v>cv>cv>c and split the net surplus v−cv-cv−c according to their bargaining powers θs,θb∈(0,1)\theta_s,\theta_b\in(0,1)θs​,θb​∈(0,1), θs+θb=1\theta_s+\theta_b=1θs​+θb​=1. Anticipating this, the buyer chooses α\alphaα to solve

max⁡α∈[α‾,αˉ] Ub(α)=θb A(α,c)−k(α),A(α,c)=∫cvˉ(v−c) dG(v,α),(6)\max_{\alpha\in[\underline\alpha,\bar\alpha]}\ U_b(\alpha)=\theta_b\,A(\alpha,c)-k(\alpha),\qquad A(\alpha,c)=\int_c^{\bar v}(v-c)\,dG(v,\alpha),\tag{6}α∈[α​,αˉ]max​ Ub​(α)=θb​A(α,c)−k(α),A(α,c)=∫cvˉ​(v−c)dG(v,α),(6)

where A(α,c)A(\alpha,c)A(α,c) is the total ex ante surplus (7). A solution of (6) is written α∗\alpha^*α∗; in general there may be several, and A∗(θb,c)\mathcal A^*(\theta_b,c)A∗(θb​,c) denotes the set of all of them.

Two sets S,T⊆RS,T\subseteq\mathbb RS,T⊆R are compared in the induced set order S⊑TS\sqsubseteq TS⊑T when min⁡(s,t)∈S\min(s,t)\in Smin(s,t)∈S and max⁡(s,t)∈T\max(s,t)\in Tmax(s,t)∈T for all s∈Ss\in Ss∈S, t∈Tt\in Tt∈T.

Formalization targets

Goal: Proposition 1 (p. 10)

Whenever the optimal sets compared are nonempty,

θb≤θb′ ⟹ A∗(θb,c)⊑A∗(θb′,c),\theta_b\le\theta_b'\ \Longrightarrow\ \mathcal A^*(\theta_b,c)\sqsubseteq\mathcal A^*(\theta_b',c),θb​≤θb′​ ⟹ A∗(θb​,c)⊑A∗(θb′​,c), c≤c′≤u^ ⟹ A∗(θb,c)⊑A∗(θb,c′),u^≤c≤c′ ⟹ A∗(θb,c′)⊑A∗(θb,c).c\le c'\le\hat u\ \Longrightarrow\ \mathcal A^*(\theta_b,c)\sqsubseteq\mathcal A^*(\theta_b,c'),\qquad \hat u\le c\le c'\ \Longrightarrow\ \mathcal A^*(\theta_b,c')\sqsubseteq\mathcal A^*(\theta_b,c).c≤c′≤u^ ⟹ A∗(θb​,c)⊑A∗(θb​,c′),u^≤c≤c′ ⟹ A∗(θb​,c′)⊑A∗(θb​,c).

The buyer's optimal information acquisition increases with its bargaining power, and first increases and then decreases with the seller's cost, turning at the prior mean u^\hat uu^.

Milestones

  1. Display (1), p. 7. G(v,α)G(v,\alpha)G(v,α) is nondecreasing in α\alphaα for v<u^v<\hat uv<u^ and nonincreasing for v>u^v>\hat uv>u^: the family rotates about u^\hat uu^.
  2. Display (2) and footnote 7, p. 8. The mean of vvv is u^\hat uu^ for every α\alphaα, and α↦A(α,c)\alpha\mapsto A(\alpha,c)α↦A(α,c) is nondecreasing.
  3. Proof of Proposition 1, p. 30. Integration by parts: Ub(α)=θb∫c∞[1−G(v,α)] dv−k(α)U_b(\alpha)=\theta_b\int_c^{\infty}[1-G(v,\alpha)]\,dv-k(\alpha)Ub​(α)=θb​∫c∞​[1−G(v,α)]dv−k(α).
  4. Proof of Proposition 1, p. 30. UbU_bUb​ has increasing differences in (α,θb)(\alpha,\theta_b)(α,θb​).
  5. Proof of Proposition 1, p. 30. UbU_bUb​ has increasing differences in (α,c)(\alpha,c)(α,c) for c≤u^c\le\hat uc≤u^ and decreasing differences for c≥u^c\ge\hat uc≥u^.
  6. Topkis's theorem (Topkis 1998, Theorem 2.8.1), already proved on the platform and referenced.

Significance

The result. The first part of Proposition 1 says that a buyer with more bargaining power gathers more information, because it captures a larger share of the surplus that information creates. The second part is the less expected one: a higher seller cost shrinks the gains from trade, yet it raises the buyer's incentive to become informed as long as the cost is below the prior mean u^\hat uu^, and lowers it only beyond. Information is most valuable when trade is a close call. The paper's later results (Corollary 1, Propositions 2 and 3) build on this: since information acquisition responds to θb\theta_bθb​ and ccc, the seller's equilibrium payoff need not increase in its own bargaining power or decrease in its cost. The paper also notes (p. 11) that the monotonicity in θb\theta_bθb​ needs only that a higher α\alphaα is a mean-preserving spread, which is why the statement is posed here for every prior with finite mean.

Formalizing it. The proposition is proved in the paper, in a short appendix argument that differentiates under the integral and appeals to Topkis's theorem. To our knowledge it has not been machine-checked. The mission produces a density-free formalization: the rotation property and the mean-preserving-spread property of the truth-or-noise family, the integration-by-parts identity for the expected positive part, and the two increasing-differences statements, which together with the already formalized Topkis theorem give the goal.

Difficulty

The paper's argument is first-order: it computes cross-partial derivatives such as ∂2Ub/∂c ∂α=θb ∂G(c,α)/∂α\partial^2U_b/\partial c\,\partial\alpha=\theta_b\,\partial G(c,\alpha)/\partial\alpha∂2Ub​/∂c∂α=θb​∂G(c,α)/∂α and reads off their signs. Without a density these derivatives need not exist, and the information cost kkk is assumed only increasing, so UbU_bUb​ need not be differentiable, concave or even continuous in α\alphaα. The statements here are therefore in difference form, and the cross effects must be established as inequalities between integrals over different distributions of vvv.

The second obstacle is the shape of the comparative statics. Topkis's theorem gives a monotone optimal set in the induced set order on a parameter range where increasing differences hold, but the cost ccc has increasing differences only on (−∞,u^](-\infty,\hat u](−∞,u^] and decreasing differences on [u^,∞)[\hat u,\infty)[u^,∞). The referenced platform theorem, by contrast, asks for increasing differences on the whole product of the decision and parameter spaces, so it does not apply to UbU_bUb​ as it stands.

The obvious shortcut of picking a selection from the optimal set and proving it monotone does not work: the cross differences are only weakly positive (they vanish when ccc lies outside the support of vvv), so the strict version of Topkis's theorem does not apply, and two different maximizers need not be ordered.

Formalization scope

The prior is a probability measure μ\muμ on R\mathbb RR with integrable identity and positive mean u^=∫w dμ\hat u=\int w\,d\muu^=∫wdμ. The paper's additional assumptions on FFF (continuous and differentiable, strictly positive density on a bounded support [u‾,uˉ][\underline u,\bar u][u​,uˉ]) and c∈(u‾,uˉ)c\in(\underline u,\bar u)c∈(u​,uˉ) are dropped: every statement is posed for all priors with finite mean and all real ccc, which implies the paper's case. FFF is ProbabilityTheory.cdf μ, and GGG is defined from it by the page's formula. The interval [α‾,αˉ][\underline\alpha,\bar\alpha][α​,αˉ] satisfies 0<α‾<αˉ<10<\underline\alpha<\bar\alpha<10<α​<αˉ<1; kkk is monotone on it with k(α‾)=0<k(αˉ)k(\underline\alpha)=0<k(\bar\alpha)k(α​)=0<k(αˉ); θb∈(0,1)\theta_b\in(0,1)θb​∈(0,1) in every statement.

The total ex ante surplus is defined as A(α,c)=∫max⁡(αw+(1−α)u^−c,0) dμ(w)A(\alpha,c)=\int\max(\alpha w+(1-\alpha)\hat u-c,0)\,d\mu(w)A(α,c)=∫max(αw+(1−α)u^−c,0)dμ(w), which is (7) after the change of variables v=αw+(1−α)u^v=\alpha w+(1-\alpha)\hat uv=αw+(1−α)u^. The bargaining stage is not modelled; the paper's reduction to (6) is built into UbU_bUb​. The optimal level α∗\alpha^*α∗ is the set of maximizers of UbU_bUb​ over [α‾,αˉ][\underline\alpha,\bar\alpha][α​,αˉ] (optSet), never a chosen element. "Increases" is the induced set order, and every comparison assumes both optimal sets nonempty, which is the paper's presumption that (6) has a solution.

Trivializing formalizations are ruled out: AAA is never given by a closed form or by the integration-by-parts expression (that identity is a milestone), α∗\alpha^*α∗ is never defined by a formula or a threshold rule, and the nonemptiness hypotheses prevent the induced set order from holding vacuously on an empty optimal set.

The development reuses the platform definitions Supermodularity.Lattices.InducedSetOrder and Supermodularity.Monotonicity.IncreasingDifferencesOn and the proved theorem Supermodularity.Monotonicity.argmax_increasing_of_increasing_differences. The rotation and mean-preserving-spread lemmas for truth-or-noise signals, and the tail-integral identity E[(X−c)+]=∫c∞P(X>x) dx\mathbb E[(X-c)^+]=\int_c^\infty P(X>x)\,dxE[(X−c)+]=∫c∞​P(X>x)dx for integrable XXX, are reusable beyond this mission. Contributions of any of the milestones, of alternative proofs that avoid the integration by parts, and of the strict versions under the paper's density assumption are welcome.

Not posed in this mission: the benchmark (3)–(5), Corollary 1, Propositions 2–7, and the extensions of §4–§5.

Selected references

  • L. Guo, Gathering Information before Negotiation, author's manuscript of 15 Feb 2022, SSRN 3930651. https://ssrn.com/abstract=3930651 ; published in Management Science (2022), https://doi.org/10.1287/mnsc.2022.4360
  • D. M. Topkis, Supermodularity and Complementarity, Princeton University Press, 1998. https://doi.org/10.1515/9781400822539
  • J. P. Johnson and D. P. Myatt, On the Simple Economics of Advertising, Marketing, and Product Design, American Economic Review 96(3), 756–784, 2006. https://doi.org/10.1257/aer.96.3.756
10 thms2 active usersReviewed
Machine LearningOperations Research·Captain: mikedeng1

Contextual Inverse Optimization: Offline and Online Learning 3: EllipsoidalCones Has Worst-Case Regret O(d² ln(T tan α(C₀))) from a Pointed Initial Knowledge SetResearch Paper

Motivation

In inverse optimization one observes the decisions of an expert who solves an optimization problem and tries to recover the expert's objective. The contextual version of Besbes, Fonseca and Lobel (arXiv:2106.14015; Operations Research, 2025) asks a sharper question: not how well the cost can be estimated, but how well a decision-maker who imitates the expert performs, measured by regret against the expert. Settings of this kind arise whenever a firm wants to reproduce the routine choices of an experienced planner, for example routing, assortment or scheduling decisions, without being told the planner's cost trade-offs.

The online setting is adversarial. Nature fixes an unknown cost vector and then presents a sequence of problems; after each decision it reveals the expert's optimal action, which cuts the set of costs still consistent with the observations. A companion mission of this series shows that the natural greedy policy, which plays the center of this consistent set, can incur regret linear in the horizon. This mission formalizes the paper's first positive answer: an algorithm, EllipsoidalCones, with regret logarithmic in the horizon when the initial set of costs lies inside a pointed cone.

Setting

Costs lie on the unit sphere SdS^dSd of Rd\mathbb R^dRd, d=m+1≥2d=m+1\ge2d=m+1≥2. In period ttt nature presents a feasible set Xt⊆Rn\mathcal X_t\subseteq\mathbb R^nXt​⊆Rn, compact, nonempty and of diameter at most 111, and a 111-Lipschitz context ft:Rn→Rdf_t:\mathbb R^n\to\mathbb R^dft​:Rn→Rd. The forward problem with cost ccc has solution set ψ(c,Xt,ft)=argmin⁡x∈Xtft(x)′c\psi(c,\mathcal X_t,f_t)=\operatorname{argmin}_{x\in\mathcal X_t}f_t(x)'cψ(c,Xt​,ft​)=argminx∈Xt​​ft​(x)′c. The decision-maker plays some xtπx^\pi_txtπ​, nature reveals an expert action xt⋆∈ψ(c⋆,Xt,ft)x^\star_t\in\psi(c^\star,\mathcal X_t,f_t)xt⋆​∈ψ(c⋆,Xt​,ft​) for the true cost c⋆c^\starc⋆, and the regret over TTT periods is

RTπ=∑t=1T(ft(xtπ)−ft(xt⋆))′c⋆.\mathcal R^\pi_T=\sum_{t=1}^T\big(f_t(x^\pi_t)-f_t(x^\star_t)\big)'c^\star .RTπ​=t=1∑T​(ft​(xtπ​)−ft​(xt⋆​))′c⋆.

The knowledge set C(It)C(\mathcal I_t)C(It​) is the set of costs in the initial set C0⊆SdC_0\subseteq S^dC0​⊆Sd under which every expert action seen so far was optimal.

The angle between nonzero vectors is θ(c,c^)=arccos⁡(c′c^/∥c∥∥c^∥)\theta(c,\hat c)=\arccos(c'\hat c/\|c\|\|\hat c\|)θ(c,c^)=arccos(c′c^/∥c∥∥c^∥), and the uncertainty angle of a set CCC is α(C)=inf⁡c^∈Sdsup⁡c∈Cθ(c,c^)\alpha(C)=\inf_{\hat c\in S^d}\sup_{c\in C}\theta(c,\hat c)α(C)=infc^∈Sd​supc∈C​θ(c,c^); a minimizing c^\hat cc^ is a circumcenter. The initial set is pointed when α(C0)<π/2\alpha(C_0)<\pi/2α(C0​)<π/2.

For a positive diagonal W=diag⁡(w1,…,wm)W=\operatorname{diag}(w_1,\dots,w_m)W=diag(w1​,…,wm​) and an orthonormal UUU, the ellipsoidal cone is

E(W,U)={c: c~[1]≥0, c~[2:d]′W−1c~[2:d]≤c~[1]2},c~=U−1c,E(W,U)=\{c:\ \tilde c_{[1]}\ge0,\ \tilde c_{[2:d]}'W^{-1}\tilde c_{[2:d]}\le\tilde c_{[1]}^2\},\qquad \tilde c=U^{-1}c ,E(W,U)={c: c~[1]​≥0, c~[2:d]′​W−1c~[2:d]​≤c~[1]2​},c~=U−1c,

a cone with axis Ue1Ue_1Ue1​ and elliptical cross-section. The effective difference of period ttt is δtπ=(ft(xtπ)−ft(xt⋆))/∥ft(xtπ)−ft(xt⋆)∥\delta^\pi_t=(f_t(x^\pi_t)-f_t(x^\star_t))/\|f_t(x^\pi_t)-f_t(x^\star_t)\|δtπ​=(ft​(xtπ​)−ft​(xt⋆​))/∥ft​(xtπ​)−ft​(xt⋆​)∥ (and 000 if the difference vanishes).

EllipsoidalCones with parameter ϵ\epsilonϵ starts from the circular cone W1=tan⁡2α(C0)IW_1=\tan^2\alpha(C_0)IW1​=tan2α(C0​)I, U1e1U_1e_1U1​e1​ a circumcenter of C0C_0C0​, and plays the axis Ute1U_te_1Ut​e1​ as proxy cost. With δˉ=Ut−1δtπ\bar\delta=U_t^{-1}\delta^\pi_tδˉ=Ut−1​δtπ​ it keeps the cone if

δˉ[2:d]′Wtδˉ[2:d]≤ϵ2(12)\bar\delta_{[2:d]}'W_t\bar\delta_{[2:d]}\le\epsilon^2 \qquad (12)δˉ[2:d]′​Wt​δˉ[2:d]​≤ϵ2(12)

and otherwise replaces it by the output of ConeUpdate, which performs one central-cut step of the ellipsoid method on the cone's cross-section and re-centres the result by a spectral decomposition.

Formalization targets

Goal: Theorem 3 (p. 20)

With ϵ=d/T\epsilon=d/Tϵ=d/T there is an absolute constant KKK such that for every d≥2d\ge2d≥2, T≥1T\ge1T≥1, closed pointed C0C_0C0​ with α(C0)>0\alpha(C_0)>0α(C0​)>0, c⋆∈C0c^\star\in C_0c⋆∈C0​, admissible instances and run,

RTπ≤K d2ln⁡(Ttan⁡α(C0))  if α(C0)>arctan⁡(d/T),RTπ≤K d  otherwise.\mathcal R^\pi_T\le K\,d^2\ln\big(T\tan\alpha(C_0)\big)\ \text{ if }\alpha(C_0)>\arctan(d/T),\qquad \mathcal R^\pi_T\le K\,d\ \text{ otherwise.}RTπ​≤Kd2ln(Ttanα(C0​))  if α(C0​)>arctan(d/T),RTπ​≤Kd  otherwise.

The constant is left unfixed: the shape d2ln⁡(Ttan⁡α(C0))d^2\ln(T\tan\alpha(C_0))d2ln(Ttanα(C0​)) is the claim, and the proof's explicit constant rests on a step that fails at d=2d=2d=2.

Milestones

  1. Proposition 1: α(E(W,U))=arctan⁡λmax⁡(W)\alpha(E(W,U))=\arctan\sqrt{\lambda_{\max}(W)}α(E(W,U))=arctanλmax​(W)​.
  2. Lemma 4: if C(It)⊆E(Wt,Ut)C(\mathcal I_t)\subseteq E(W_t,U_t)C(It​)⊆E(Wt​,Ut​) and (12) holds, the period's regret is at most ϵ\epsilonϵ.
  3. Lemma B-1 (η=0\eta=0η=0): ConeUpdate's output contains E(W,U)∩{δ′c≥0}E(W,U)\cap\{\delta'c\ge0\}E(W,U)∩{δ′c≥0} and ∏λi\prod\lambda_i∏λi​ shrinks by e−1/(2(d−1))e^{-1/(2(d-1))}e−1/(2(d−1)).
  4. Lemma 3: C(It+1)⊆E(Wt,Ut)∩{δtπ′c≥0}⊆E(Wt+1,Ut+1)C(\mathcal I_{t+1})\subseteq E(W_t,U_t)\cap\{\delta^{\pi\prime}_tc\ge0\}\subseteq E(W_{t+1},U_{t+1})C(It+1​)⊆E(Wt​,Ut​)∩{δtπ′​c≥0}⊆E(Wt+1​,Ut+1​), and for d≥3d\ge3d≥3 ∏λi\prod\lambda_i∏λi​ shrinks by e−1/(d−1)e^{-1/(d-1)}e−1/(d−1).
  5. Cohen et al. (2020), Lemmas 3 and 4 (already posed on the platform, referenced): an ellipsoid-method step shrinks the smallest eigenvalue by at most (d−1)2/d2(d-1)^2/d^2(d−1)2/d2, and does not shrink it at all below the threshold (ϵ/(10(d−1)))2(\epsilon/(10(d-1)))^2(ϵ/(10(d−1)))2 when the cut is long.
  6. Lemma 5: for d≥3d\ge3d≥3, λmax⁡(Wt)≤(10d/ϵ)2(d−2)λmax⁡(W1)d−1e−Itπ/(d−1)\lambda_{\max}(W_t)\le(10d/\epsilon)^{2(d-2)}\lambda_{\max}(W_1)^{d-1}e^{-I^\pi_t/(d-1)}λmax​(Wt​)≤(10d/ϵ)2(d−2)λmax​(W1​)d−1e−Itπ​/(d−1), or no update ever happens.

Significance

The theorem shows that in contextual inverse optimization the decision-maker can force nature to reveal information: a cone update happens only after a large mistake, and each such mistake shrinks the cone geometrically, so large mistakes are logarithmically rare. It is the building block of the paper's general-case algorithm, which restarts the same construction in growing subspaces, and it isolates why the greedy policy fails: it never trades instantaneous regret for information.

The paper proves the result; nothing in it is machine-checked. The mission produces a checked version of the pointed-case analysis, an ellipsoidal-cone layer on top of the platform's ellipsoid-method definitions, and corrected statements where the printed ones are imprecise: the dimension restriction d≥3d\ge3d≥3 for the rate of Lemma 3 and the first part of Lemma 5, the column order of the spectral decomposition in ConeUpdate, and the η=0\eta=0η=0 case of Lemma B-1.

Difficulty

Two steps resist the obvious argument. First, ConeUpdate is not the ellipsoid method: the minimum-volume ellipsoid of the cut cross-section has the right volume but its cone is no longer centred, and the new cone's axis is an eigenvector of an indefinite bordered matrix. Showing that this re-centred cone still contains the cut cone requires matching the eigenvalues of the bordered matrix with those of the ellipsoid matrix in the right order. Second, a volume decrease alone does not bound the regret: by Proposition 1 the uncertainty angle is governed by the largest eigenvalue, and the ellipsoid method can make that eigenvalue grow. The lazy test (12) and the smallest-eigenvalue lemmas of Cohen et al. are what keep the cones from becoming long and thin; without them the product of eigenvalues falls while the angle does not.

Formalization scope

Costs live in EuclideanSpace ℝ (Fin (m+1)) with d=m+1d=m+1d=m+1, actions in EuclideanSpace ℝ (Fin n), so all norms are Euclidean. WWW is stored as its diagonal www, UUU as a real orthogonal matrix, and U−1U^{-1}U−1 is written U⊤U^\topU⊤. Lean periods are 000-based: Lean period ttt is the paper's period t+1t+1t+1. This text is the arXiv v3 preprint (1 July 2023), not the published 2025 article; all numbering is the preprint's.

Standing assumptions carried by the statements: d≥2d\ge2d≥2, n≥1n\ge1n≥1, feasible sets nonempty, compact and of diameter at most 111, contexts 111-Lipschitz, C0C_0C0​ a closed subset of SdS^dSd, ϵ>0\epsilon>0ϵ>0. Pinned and disclosed choices:

  • ConeUpdate is a relation, not a function. The printed U−1Bδ[2:p]U^{-1}B\delta_{[2:p]}U−1Bδ[2:p]​ is read as (U−1δ)[2:p](U^{-1}\delta)_{[2:p]}(U−1δ)[2:p]​. The first column of VVV is the eigenvector of the positive eigenvalue of MMM with positive first entry, and larger ∣μ∣|\mu|∣μ∣ is paired with larger eigenvalues of NNN. The page leaves this order open, and Algorithm 1 and Lemma B-1 disagree on it.
  • The printed "U1←Id−1U_1\leftarrow I_{d-1}U1​←Id−1​" is read as: U1U_1U1​ orthonormal with a circumcenter of C0C_0C0​ as its first column.
  • α(C0)>0\alpha(C_0)>0α(C0​)>0 on the goal. At α(C0)=0\alpha(C_0)=0α(C0​)=0 the algorithm's W1=0W_1=0W1​=0 is not positive definite.
  • The rate of Lemma 3 and the first part of Lemma 5 are stated for d≥3d\ge3d≥3 only; the rest of both lemmas for every d≥2d\ge2d≥2. At d=2d=2d=2 ConeUpdate halves WWW, and 1/2>e−11/2>e^{-1}1/2>e−1.
  • Lemma B-1 is stated only for η=0\eta=0η=0. Its η>0\eta>0η>0 inclusion is false as printed.
  • The worst-case regret is stated as a bound for every true cost, every adaptive instance sequence and every run, not as a real supremum. The polynomial running-time claim is not formalized.

The run predicate is satisfiable: a circumcenter of a closed nonempty C0C_0C0​ exists, the bordered matrix has exactly one positive eigenvalue, and a cone update is required only when the tail of δˉ\bar\deltaδˉ is nonzero. A formalization in which no run exists, or in which KKK may depend on ddd or TTT, does not state Theorem 3.

A complete development needs the angle and uncertainty-angle layer, ellipsoidal cones and their cross-sections, the published ellipsoid-method update (LinearOptimization_EllipsoidMethod) with its volume and eigenvalue facts, Cauchy–Schwarz over ellipsoids, and eigenvalue interlacing for bordered matrices. The ellipsoidal-cone layer and the cone-update lemma can be reused in the paper's general case and in other cone-based learning algorithms. Proofs of any milestone, and of the referenced Cohen et al. lemmas, are welcome.

Selected references

  • O. Besbes, Y. Fonseca, I. Lobel, Contextual Inverse Optimization: Offline and Online Learning, arXiv:2106.14015v3, 2023; Operations Research, 2025. https://arxiv.org/abs/2106.14015
  • M. C. Cohen, I. Lobel, R. Paes Leme, Feature-Based Dynamic Pricing, Management Science 66(11), 2020. https://doi.org/10.1287/mnsc.2019.3485
  • M. Grötschel, L. Lovász, A. Schrijver, Geometric Algorithms and Combinatorial Optimization, 2nd ed., Springer, 1993. https://doi.org/10.1007/978-3-642-78240-4
  • D. Bertsimas, J. N. Tsitsiklis, Introduction to Linear Optimization, Athena Scientific, 1997, Chapter 8.
16 thms2 active usersReviewed
Convex OptimizationOperations Research·Captain: mikedeng1

Contextual Inverse Optimization: Offline and Online Learning 4: The Polyhedral Cone of η-Separated Effective Differences Has Uncertainty Angle at Most arccos(η^(d−1)/d^(3/2))Research Paper

Motivation

In contextual inverse optimization a decision-maker repeatedly faces linear problems min⁡x∈Xtft(x)′c⋆\min_{x\in\mathcal X_t} f_t(x)'c^\starminx∈Xt​​ft​(x)′c⋆ whose cost vector c⋆∈Sdc^\star\in S^dc⋆∈Sd is unknown. After each period it observes an action that an expert who knows c⋆c^\starc⋆ would have taken. Every observation is a set of linear inequalities on c⋆c^\starc⋆, so the costs consistent with the data form a cone, the knowledge set. Besbes, Fonseca and Lobel (arXiv:2106.14015, published in Operations Research, 2025) measure the residual ignorance of such a set by its uncertainty angle, the half-aperture of the narrowest revolution cone containing it, and show that offline regret is governed exactly by this angle.

Online, their algorithms are efficient only when the knowledge set is pointed, i.e. has uncertainty angle below π/2\pi/2π/2. When it is not, the general algorithm of §5 (ProjectedCones) works inside a growing subspace Δt\Delta_tΔt​ spanned by the observations that were sufficiently new. Each time the subspace grows, the algorithm replaces the knowledge set by a polyhedral cone KtK_tKt​ and needs a bound on the uncertainty angle of KtK_tKt​ that depends only on the separation parameter η\etaη and the dimension. Lemma 10 of the paper provides that bound. This mission formalizes it, together with the chain of lemmas in Appendix B on which its proof rests.

Setting

Let Rd\mathbb R^dRd carry the Euclidean inner product c′xc'xc′x and norm ∥⋅∥\|\cdot\|∥⋅∥, and let SdS^dSd be its unit sphere. The angle between ccc and c^\hat cc^ is θ(c,c^)=arccos⁡(c′c^/(∥c∥∥c^∥))\theta(c,\hat c)=\arccos\big(c'\hat c/(\|c\|\|\hat c\|)\big)θ(c,c^)=arccos(c′c^/(∥c∥∥c^∥)), with θ=0\theta=0θ=0 when either vector is 000. The uncertainty angle of a set C⊆RdC\subseteq\mathbb R^dC⊆Rd is

α(C)=inf⁡c^∈Sd sup⁡c∈Cθ(c,c^),\alpha(C)=\inf_{\hat c\in S^d}\ \sup_{c\in C}\theta(c,\hat c),α(C)=c^∈Sdinf​ c∈Csup​θ(c,c^),

and a minimizer c^\hat cc^ is a circumcenter of CCC. The circumradius of a cone is μ(K)=sin⁡α(K)\mu(K)=\sin\alpha(K)μ(K)=sinα(K).

For vectors g1,…,gpg_1,\dots,g_pg1​,…,gp​, write Πg1,…,gi−1\Pi_{g_1,\dots,g_{i-1}}Πg1​,…,gi−1​​ for the orthogonal projection onto their span and g−i={gj:j≠i}g_{-i}=\{g_j:j\ne i\}g−i​={gj​:j=i}. The vector gi−ΠS(gi)g_i-\Pi_S(g_i)gi​−ΠS​(gi​) is the residual of regressing gig_igi​ on SSS.

The algorithm's effective differences are unit vectors δ1,δ2,⋯∈Rd\delta_1,\delta_2,\dots\in\mathbb R^dδ1​,δ2​,⋯∈Rd. A difference triggers a subspace update when its residual against the current subspace exceeds η\etaη. The differences δ1,…,δp\delta_1,\dots,\delta_pδ1​,…,δp​ that triggered updates, in order of arrival, therefore satisfy

∥δi−Πδ1,…,δi−1(δi)∥ ≥ η,i=2,…,p.\|\delta_i-\Pi_{\delta_1,\dots,\delta_{i-1}}(\delta_i)\|\ \ge\ \eta,\qquad i=2,\dots,p .∥δi​−Πδ1​,…,δi−1​​(δi​)∥ ≥ η,i=2,…,p.

They span the subspace Δ={∑iγiδi:γi∈R}\Delta=\{\sum_i\gamma_i\delta_i:\gamma_i\in\mathbb R\}Δ={∑i​γi​δi​:γi​∈R}, and the polyhedral cone of (14) is

K={c∈Δ: c′δi≥0, i=1,…,p}.K=\{c\in\Delta:\ c'\delta_i\ge0,\ i=1,\dots,p\}.K={c∈Δ: c′δi​≥0, i=1,…,p}.

Two further cone quantities appear. The simplicial cone of linearly independent g1,…,gp∈Rpg_1,\dots,g_p\in\mathbb R^pg1​,…,gp​∈Rp is cone⁡(g)={∑iαigi:αi≥0}\operatorname{cone}(g)=\{\sum_i\alpha_ig_i:\alpha_i\ge0\}cone(g)={∑i​αi​gi​:αi​≥0}. The inradius of a cone K⊆RpK\subseteq\mathbb R^pK⊆Rp is ρ(K)=max⁡x∈Sp∩Kmin⁡y∈∂K∥x−y∥\rho(K)=\max_{x\in S^p\cap K}\min_{y\in\partial K}\|x-y\|ρ(K)=maxx∈Sp∩K​miny∈∂K​∥x−y∥, and the dual cone is K⋆={c:c′x≥0 ∀x∈K}K^\star=\{c:c'x\ge0\ \forall x\in K\}K⋆={c:c′x≥0 ∀x∈K}. With GGG the matrix of columns gig_igi​, λmin⁡(G′G)\lambda_{\min}(G'G)λmin​(G′G) and λmax⁡(G′G)\lambda_{\max}(G'G)λmax​(G′G) are the extreme eigenvalues of the Gram matrix.

Formalization targets

Goal: Lemma 10, angle bound (p. 25)

For 0<η≤10<\eta\le10<η≤1, d≥2d\ge2d≥2 and unit vectors δ1,…,δp∈Rd\delta_1,\dots,\delta_p\in\mathbb R^dδ1​,…,δp​∈Rd obeying the separation rule,

α(K) ≤ arccos⁡ ⁣(ηd−1d3/2).\alpha(K)\ \le\ \arccos\!\Big(\frac{\eta^{d-1}}{d^{3/2}}\Big).α(K) ≤ arccos(d3/2ηd−1​).

The bound depends on η\etaη and ddd only, not on the number p≤dp\le dp≤d of vectors or their configuration.

Milestones (Appendix B)

  1. Lemma B-4 (App-28). Backward separation implies two-sided separation:
∥gi−Πg−i(gi)∥2 ≥ η2(p−i)(η2)1{i>1},i=1,…,p.\|g_i-\Pi_{g_{-i}}(g_i)\|^2\ \ge\ \eta^{2(p-i)}(\eta^2)^{\mathbf 1\{i>1\}},\qquad i=1,\dots,p.∥gi​−Πg−i​​(gi​)∥2 ≥ η2(p−i)(η2)1{i>1},i=1,…,p.
  1. Lemma B-3 (App-27). With φ=min⁡i∥gi−Πg−i(gi)∥\varphi=\min_i\|g_i-\Pi_{g_{-i}}(g_i)\|φ=mini​∥gi​−Πg−i​​(gi​)∥,
λmax⁡(G′G)λmin⁡(G′G)≤(pφ)2.\frac{\lambda_{\max}(G'G)}{\lambda_{\min}(G'G)}\le\Big(\frac p\varphi\Big)^2 .λmin​(G′G)λmax​(G′G)​≤(φp​)2.
  1. Lemma B-2 (App-25). ρ(cone⁡(g)) ≥ 1pλmin⁡(G′G)λmax⁡(G′G)\displaystyle\rho(\operatorname{cone}(g))\ \ge\ \frac1{\sqrt p}\sqrt{\frac{\lambda_{\min}(G'G)}{\lambda_{\max}(G'G)}}ρ(cone(g)) ≥ p​1​λmax​(G′G)λmin​(G′G)​​.
  2. (B-9) (App-20). μ2(K)+ρ2(K⋆)=1\mu^2(K)+\rho^2(K^\star)=1μ2(K)+ρ2(K⋆)=1, for every closed convex pointed cone K≠{0}K\ne\{0\}K={0}.
  3. (B-8) (App-19). For K={c∈Rp:c′gi≥0}K=\{c\in\mathbb R^p:c'g_i\ge0\}K={c∈Rp:c′gi​≥0} with separated unit gig_igi​,
μ(K)≤1−η2(p−1)/p3.\mu(K)\le\sqrt{1-\eta^{2(p-1)}/p^3}.μ(K)≤1−η2(p−1)/p3​.

Significance

The bound is what makes the subspace stage of ProjectedCones restartable. After each update the algorithm discards its previous ellipsoidal cone and starts afresh from a revolution cone of aperture arccos⁡(ηd−1/d3/2)\arccos(\eta^{d-1}/d^{3/2})arccos(ηd−1/d3/2) around the circumcenter of KtK_tKt​. Lemma 10 guarantees that this cone contains KtK_tKt​, and with it the projected knowledge set, and that it is pointed. The number of later ellipsoid steps grows with the logarithm of the tangent of this aperture, so the shape of the bound enters the paper's regret analysis directly. The lemmas behind it are statements about general vector families: a residual inequality of Frisch–Waugh–Lovell type, a condition-number bound for correlation matrices, an inradius bound for simplicial cones, and the circumradius–inradius duality of Henrion and Seeger. Each is usable outside this paper.

The result is proved on paper; no part of it is formalized. The mission asks for machine-checked proofs of Lemma 10's angle bound and of the five appendix statements. Henrion–Seeger duality is cited, not proved, in the source. Its formalization would be new.

Difficulty

The separation rule controls each new vector only against the earlier ones. A bound on the cone's aperture needs control of each vector against all the others, and later vectors can lie close to the span of an earlier vector together with the remaining ones. The loss compounds, which is where the exponential factor ηd−1\eta^{d-1}ηd−1 comes from. A direct estimate of the circumcenter of KKK is not available in closed form: the circumcenter is the solution of a quadratic program. The natural first attempt, bounding the angle between the circumcenter candidate ∑iδi/∥∑iδi∥\sum_i\delta_i/\|\sum_i\delta_i\|∑i​δi​/∥∑i​δi​∥ and the extreme rays of KKK, requires the extreme rays of KKK, which are the columns of a dual basis whose norms are not controlled by the separation rule alone.

A second difficulty is dimensional. KKK lives in the subspace Δ\DeltaΔ, possibly of dimension p<dp<dp<d, while the uncertainty angle is measured on the sphere of Rd\mathbb R^dRd. The appendix statements are posed in Rp\mathbb R^pRp, in coordinates of Δ\DeltaΔ. Transferring them requires an isometry onto Δ\DeltaΔ that preserves the cone, the angles and the projections.

Formalization scope

Vectors live in EuclideanSpace ℝ (Fin d); families are indexed by Fin p, so all indices are 0-based and Lean's iii is the paper's i+1i+1i+1. Projections are Submodule.starProjection onto Submodule.span. The uncertainty angle is a real infimum of real suprema of angles in [0,π][0,\pi][0,π]. The angle at the zero vector is 000, as in the paper, not Mathlib's π/2\pi/2π/2. The inradius is a real supremum of Metric.infDist to the frontier. It equals the paper's maximum on every cone of this mission. The dual cone is the platform's ConvexOptimization.dualCone. Eigenvalue ratios are stated without spectra: Lemma B-3 in the Rayleigh form ∥Gu∥2∥v∥2≤(p/φ)2∥Gv∥2∥u∥2\|Gu\|^2\|v\|^2\le(p/\varphi)^2\|Gv\|^2\|u\|^2∥Gu∥2∥v∥2≤(p/φ)2∥Gv∥2∥u∥2, and Lemma B-2 with arbitrary bracket constants ℓmin⁡∥u∥2≤∥Gu∥2≤ℓmax⁡∥u∥2\ell_{\min}\|u\|^2\le\|Gu\|^2\le\ell_{\max}\|u\|^2ℓmin​∥u∥2≤∥Gu∥2≤ℓmax​∥u∥2. Both forms are equivalent to the printed ones.

Conventions and pins:

  • The separation rule is "residual ≥η\ge\eta≥η", the form of (B-14). The algorithm's strict ">η>\eta>η" is a special case, so the goal is stronger than the page needs. The first vector carries no condition.
  • Lemma 10's inclusion ΠΔt(C(It))⊆Kt\Pi_{\Delta_t}(C(\mathcal I_t))\subseteq K_tΠΔt​​(C(It​))⊆Kt​ is not posed: it concerns the algorithm's run. The polynomial-time computation of the circumcenter is not posed either: it is a complexity claim. "For every period ttt" becomes "for every family δ\deltaδ obeying the separation rule"; p=0p=0p=0 is the first period, with K={0}K=\{0\}K={0}.
  • d≥2d\ge2d≥2 is the paper's standing assumption (§2), and d3/2d^{3/2}d3/2 is the real power.
  • Lemma B-3 as printed takes φ\varphiφ as the minimum over i=2,…,pi=2,\dots,pi=2,…,p only. With that minimum the bound is false. The printed proof bounds every i=1,…,pi=1,\dots,pi=1,…,p, and the proof of Lemma 10 applies it so. The minimum over all iii is stated.
  • (B-9) is stated on the page for every closed convex cone. It is posed for every closed convex pointed cone K≠{0}K\ne\{0\}K={0} in Rp\mathbb R^pRp: App-19 defines μ(K)=sin⁡α(K)\mu(K)=\sin\alpha(K)μ(K)=sinα(K) for pointed cones, and K={0}K=\{0\}K={0} is excluded because its dual Rp\mathbb R^pRp has empty boundary.
  • Lemmas B-3 and B-4 are posed in Rm\mathbb R^mRm for every mmm, which is equivalent to the page's Rp\mathbb R^pRp. Lemma B-4 assumes no range for η\etaη, since only η2\eta^2η2 and even powers occur.

The cone in the goal is intersected with Δ\DeltaΔ. A formalization of KKK as {c∈Rd:c′δi≥0}\{c\in\mathbb R^d:c'\delta_i\ge0\}{c∈Rd:c′δi​≥0} would be false for p<dp<dp<d (that set contains a line, so α≥π/2\alpha\ge\pi/2α≥π/2). A separation hypothesis that does not exclude nearly collinear vectors would make the bound unobtainable, and one that is never satisfiable would make it empty. The sanity check in the drafting workspace exhibits the hypotheses at p=d=2p=d=2p=d=2, η=1\eta=1η=1.

Contributions welcome: proofs of any milestone; the passage from Rp\mathbb R^pRp to the subspace Δ⊆Rd\Delta\subseteq\mathbb R^dΔ⊆Rd (an isometry lemma for cones and angles); Gram-matrix facts (the inverse correlation matrix diagonal aii=1/(1−Ri2)a_{ii}=1/(1-R_i^2)aii​=1/(1−Ri2​)) as reusable lemmas.

Selected references

  • O. Besbes, Y. Fonseca, I. Lobel, Contextual Inverse Optimization: Offline and Online Learning, arXiv:2106.14015v3, 2023 (the version formalized; published in Operations Research, 2025). https://arxiv.org/abs/2106.14015
  • R. Henrion, A. Seeger, Inradius and circumradius of various convex cones arising in applications, Set-Valued and Variational Analysis 18, 483–511, 2010. https://doi.org/10.1007/s11228-010-0150-z
  • M. C. Lovell, A simple proof of the FWL theorem, Journal of Economic Education 39(1), 88–91, 2008. https://doi.org/10.3200/JECE.39.1.88-91
9 thms2 active usersReviewed
EconomicsOperations Research·Captain: mikedeng1

Monetary Easing, Leveraged Payouts, and Lack of Investment I: With Unregulated Leverage the Optimal Date-0 Rate Is at Most R and Date-0 Investment Falls Strictly Short of the First-BestResearch Paper

Monetary easing when firms can pay out borrowed funds

Low policy rates are meant to stimulate corporate investment. Acharya and Plantin document that in the low-rate decade after the 2008 crisis, U.S. firms used much of their new borrowing for share buybacks and dividends rather than for investment: leveraged payouts. Their paper (SSRN 3484935; Management Science, 2025, doi:10.1287/mnsc.2022.01440) builds a model in which this is a rational response to cheap funding and in which it limits what monetary easing can achieve. When the market rate is below the shareholders' own discount rate, a firm borrows against its future output to pay its owners today. The owners then keep a smaller stake in the firm's success and exert less effort. A central bank that cuts rates to raise employment therefore also lowers the productivity of the firms it is trying to help.

This mission formalizes the paper's Proposition 11: when leverage is not regulated, no choice of date-0 interest rate reaches the efficient allocation, and the best rate leaves investment strictly below its first-best level.

The date-0 economy

There is one consumption good. Entrepreneurs discount date-1 consumption at a rate R>1R>1R>1. An entrepreneur who hires lll units of labour at date 0 obtains f(l)f(l)f(l) units at date 1 with probability eee and nothing otherwise. The effort eee costs her e2f(l)/(2Rπ)e^2 f(l)/(2R\pi)e2f(l)/(2Rπ) privately, where π∈(0,1)\pi\in(0,1)π∈(0,1) is an effort-cost parameter (not the circle constant). Workers own a technology that turns yyy units of labour into g(y)g(y)g(y) units of the good at once; each young worker supplies one unit of labour. Both fff and ggg satisfy the Inada conditions: they vanish at 000, are strictly increasing and strictly concave, with marginal products going from +∞+\infty+∞ at 0+0^+0+ to 000 at +∞+\infty+∞; fff is twice continuously differentiable.

In the laissez-faire steady state, labour l∗l^*l∗ in the capital-good sector and the wage w∗w^*w∗ solve

π2Rf′(l∗)=g′(1−l∗)=w∗.(20)\frac{\pi}{2R}f'(l^*)=g'(1-l^*)=w^*. \qquad (20)2Rπ​f′(l∗)=g′(1−l∗)=w∗.(20)

At date 0 the workers' technology is hit by a shock ρ∈(0,1)\rho\in(0,1)ρ∈(0,1) and becomes ρg\rho gρg. The efficient date-0 labour lρl_\rholρ​ then solves πf′(lρ)/(2R)=ρ g′(1−lρ)\pi f'(l_\rho)/(2R)=\rho\,g'(1-l_\rho)πf′(lρ​)/(2R)=ρg′(1−lρ​), and lρ>l∗l_\rho>l^*lρ​>l∗. The wage, however, is downward rigid: it cannot fall below w∗w^*w∗ at date 0.

The central bank sets the date-0 rate r>0r>0r>0. An entrepreneur facing rrr retains a stake xxx in her output, and her effort is e=πxe=\pi xe=πx. Writing rˉ(r)=min⁡{r;R}\bar r(r)=\min\{r;R\}rˉ(r)=min{r;R}, she chooses

x=R2R−rˉ(r),e(r)=πR2R−rˉ(r),πRf′(l(r))2(2R−rˉ(r))=r w∗,x=\frac{R}{2R-\bar r(r)},\qquad e(r)=\frac{\pi R}{2R-\bar r(r)},\qquad \frac{\pi R f'(l(r))}{2(2R-\bar r(r))}=r\,w^*,x=2R−rˉ(r)R​,e(r)=2R−rˉ(r)πR​,2(2R−rˉ(r))πRf′(l(r))​=rw∗,

so that for r<Rr<Rr<R she pays out part of her future output (x<1x<1x<1) and her effort falls below π\piπ. The date-0 social surplus is

Σρ(r)=(e(r)−e(r)22π)f(l(r))R+ρ g(1−l(r)).(59)\Sigma_\rho(r)=\Big(e(r)-\frac{e(r)^2}{2\pi}\Big)\frac{f(l(r))}{R}+\rho\,g(1-l(r)). \qquad (59)Σρ​(r)=(e(r)−2πe(r)2​)Rf(l(r))​+ρg(1−l(r)).(59)

A rate rur_uru​ is optimal if it maximizes Σρ\Sigma_\rhoΣρ​ over all rates r>0r>0r>0 at which the labour demand l(r)≤1l(r)\le 1l(r)≤1 is feasible.

Formalization targets

Goal: Proposition 11, points 1–2

An optimal date-0 rate exists, and every optimal rate rur_uru​ with labour lul_ulu​ satisfies

ru≤R,lu<lρ,Σρ(ru)<πf(lρ)2R+ρ g(1−lρ).r_u\le R,\qquad l_u<l_\rho,\qquad \Sigma_\rho(r_u)<\frac{\pi f(l_\rho)}{2R}+\rho\,g(1-l_\rho).ru​≤R,lu​<lρ​,Σρ​(ru​)<2Rπf(lρ​)​+ρg(1−lρ​).

The right-hand side is the first-best date-0 surplus, which regulated leverage attains (Proposition 10 of the paper).

Milestones

  1. The incentive constraint e=arg⁡max⁡y{xy−y2/(2π)}e=\arg\max_y\{xy-y^2/(2\pi)\}e=argmaxy​{xy−y2/(2π)} holds exactly when e=πxe=\pi xe=πx.
  2. Proposition 2: the firm's choice of xxx, eee and investment in the partial-equilibrium model, for every r>0r>0r>0.
  3. (33)–(35): the unregulated date-0 entrepreneur's choice for r0<Rr_0<Rr0​<R.
  4. e(r)e(r)e(r) is increasing and l(r)l(r)l(r) is decreasing in rrr.
  5. lρ>l∗l_\rho>l^*lρ​>l∗, and the flexible wage ρg′(1−lρ)\rho g'(1-l_\rho)ρg′(1−lρ​) is below w∗w^*w∗.
  6. At lρl_\rholρ​, the marginal social value of labour is negative when e<πe<\pie<π.
  7. (60): Σρ′(r^)>0\Sigma_\rho'(\hat r)>0Σρ′​(r^)>0 at the rate r^\hat rr^ with l(r^)=lρl(\hat r)=l_\rhol(r^)=lρ​.

Significance

Proposition 11 is the paper's central negative result. With a rigid wage and a productivity shock, a lower rate raises employment, which is what the central bank wants, but with unregulated leverage it also lowers effort through leveraged payouts. The proposition shows that the second effect always binds before employment reaches its efficient level, so the welfare loss from the shock cannot be undone by monetary policy alone. Prudential regulation of leverage is the missing instrument (Proposition 10). Proposition 12, the subject of a companion mission, builds on the same model to show that small shocks call for no rate cut at all.

The result is proved in the paper, in a short argument in Appendix B. As far as we know it has no machine-checked proof. The formalization makes the existence of an optimal rate, which the paper takes for granted, an explicit claim, and it settles the planner's choice over all rates, including rates above RRR.

Difficulty

The obvious argument differentiates Σρ\Sigma_\rhoΣρ​ at the rate where l(r)=lρl(r)=l_\rhol(r)=lρ​ and reads off the sign. That shows only that Σρ\Sigma_\rhoΣρ​ is increasing at one point. To conclude that every optimum has lu<lρl_u<l_\rholu​<lρ​, the sign of the derivative must hold on the whole range of rates where l(r)≥lρl(r)\ge l_\rhol(r)≥lρ​, including the lower end, where labour reaches the endowment bound l=1l=1l=1 and g′(1−l)g'(1-l)g′(1−l) blows up. Existence of an optimum needs continuity of the labour response, which is defined only implicitly through the inverse of f′f'f′, and control of the surplus for rates above RRR, where the firm's behaviour changes regime.

Formalization scope

All objects live in LeveragedPayouts.Unregulated. The economy is a structure Economy with fields R,π,f,g,l∗,w∗R,\pi,f,g,l^*,w^*R,π,f,g,l∗,w∗, the conditions R>1R>1R>1, π∈(0,1)\pi\in(0,1)π∈(0,1), the Inada conditions for fff (with C2C^2C2 and f′′<0f''<0f′′<0) and for ggg (C1C^1C1, g′g'g′ strictly decreasing), l∗∈(0,1)l^*\in(0,1)l∗∈(0,1) and (20). The first-best labour lρl_\rholρ​ is a parameter with its defining equation and 0<lρ<10<l_\rho<10<lρ​<1; these determine it uniquely. Derivatives are Mathlib's deriv, used only where the Inada conditions give differentiability.

Conventions committed to:

  • The rate ranges over all r>0r>0r>0. For r≥Rr\ge Rr≥R the firm's response is Proposition 2 with rˉ(r)=R\bar r(r)=Rrˉ(r)=R, so ru≤Rr_u\le Rru​≤R is a conclusion, not an assumption. The paper's proof restricts to "r≤1r\le 1r≤1", a misprint for r≤Rr\le Rr≤R.
  • Labour demand requires 0<l≤10<l\le 10<l≤1 (the endowment of the young workers).
  • Optimality is global, over every feasible pair (r,l)(r,l)(r,l); existence of an optimum is part of the goal, so the statement is not vacuous.
  • "Welfare is strictly lower than when leverage is regulated" is stated against the first-best date-0 surplus; other dates are not modelled.
  • The page prints (34) with rrr where r0r_0r0​ is meant; the Lean uses r0r_0r0​.

A formalization that restricted the planner to r≤Rr\le Rr≤R, used only local optimality, or left l∗l^*l∗, w∗w^*w∗, lρl_\rholρ​ free of their defining equations would make the goal trivial or false; none of these is used.

Out of scope: the full overlapping-generations dynamics of §3.1–3.4 beyond the date-0 problem (59), the bond market, fiscal transfers, the dates t≠0t\neq 0t=0 (point 1's "r∗=Rr^*=Rr∗=R at all dates other than 0"), and point 3 of Proposition 11, which rests on an additional assumption on the size of transfers.

Needed infrastructure: implicit-function and inverse-function arguments for f′f'f′ on (0,∞)(0,\infty)(0,∞), one-variable calculus for (60), and compactness for existence. Contributions on the monotone and continuous dependence of the labour response on the rate are reusable in the companion mission.

Selected references

  • V. V. Acharya and G. Plantin, Monetary Easing, Leveraged Payouts, and Lack of Investment, SSRN 3484935, version of May 23, 2023. https://ssrn.com/abstract=3484935
  • V. V. Acharya and G. Plantin, Monetary Easing, Leveraged Payouts, and Lack of Investment, Management Science, 2025. https://doi.org/10.1287/mnsc.2022.01440
10 thms2 active usersReviewed
EconomicsMechanism DesignOperations Research+1·Captain: mikedeng1

Selling Data to Marketers 3: Under the Rotation Order, Marketing Lists Serve Low-Cost Marketers Efficiently and Over- (Under-) Supply High-Cost Ones Below (Above) the Rotation PointResearch Paper

Motivation

Data brokers sell two kinds of products to marketers. A marketing list supplies the contacts a marketer needs in order to act at all; a data append enriches a list the marketer already holds. In both cases the broker can also choose how informative its accompanying analytics are, and it can price a menu of plans of different informativeness to marketers whose costs of acting it cannot observe. Liang Guo's Selling Data to Marketers (Management Science, doi:10.1287/mnsc.2023.01350) studies how such a seller should design this menu, and in particular when it should sell more or less information than would be efficient.

The paper first works with a two-point value distribution (Sections 2–3) and then, in Section 4, with a general information structure in which informativeness is ordered by the rotation order of the distribution of the posterior mean (Johnson and Myatt 2006). This mission formalizes the marketing-list result of Section 4.1: Proposition 4 (comparative statics of the efficient informativeness) and Proposition 5 (the optimal screening menu). It is the third mission of a four-mission series on the paper; the first two treat the two-point model, the fourth treats data appends under the rotation order.

Setting

A marketer's value uuu from targeting a consumer is unknown. The seller supplies a signal of informativeness α∈[α‾,αˉ]⊂(0,1)\alpha\in[\underline\alpha,\bar\alpha]\subset(0,1)α∈[α​,αˉ]⊂(0,1). The marketer's posterior mean v=E[u∣w]v=E[u\mid w]v=E[u∣w] has cumulative distribution function H(v,α)H(v,\alpha)H(v,α) on a support [v‾,vˉ][\underline v,\bar v][v​,vˉ], with a strictly positive density h(v,α)h(v,\alpha)h(v,α). Bayesian updating keeps the mean fixed: ∫v‾vˉv h(v,α) dv=u^\int_{\underline v}^{\bar v} v\,h(v,\alpha)\,dv=\hat u∫v​vˉ​vh(v,α)dv=u^ for every α\alphaα, where u^\hat uu^ is the prior mean. The rotation order (9) requires a point v+∈(v‾,vˉ)v^+\in(\underline v,\bar v)v+∈(v​,vˉ) such that

∂H(v,α)∂α>0 for v<v+,∂H(v,α)∂α<0 for v>v+.\frac{\partial H(v,\alpha)}{\partial\alpha}>0\ \text{for } v<v^+,\qquad \frac{\partial H(v,\alpha)}{\partial\alpha}<0\ \text{for } v>v^+ .∂α∂H(v,α)​>0 for v<v+,∂α∂H(v,α)​<0 for v>v+.

Raising α\alphaα moves probability mass from the middle of the distribution to its tails.

The seller incurs a selling cost k(α)k(\alpha)k(α), increasing, with k(α‾)=0k(\underline\alpha)=0k(α​)=0. A marketer's targeting cost is cHc_HcH​ with probability θ∈(0,1)\theta\in(0,1)θ∈(0,1) and cLc_LcL​ with probability 1−θ1-\theta1−θ, with cH>cL>0c_H>c_L>0cH​>cL​>0. A marketer with cost ccc targets exactly when v>cv>cv>c, so informativeness α\alphaα is worth

P(α,c)=∫cvˉ(v−c) dH(v,α).P(\alpha,c)=\int_c^{\bar v}(v-c)\,dH(v,\alpha).P(α,c)=∫cvˉ​(v−c)dH(v,α).

With observable costs (the benchmark), the seller sells to type iii the informativeness maximizing (12), Πi=θi [P(α,ci)−k(α)]\Pi_i=\theta_i\,[P(\alpha,c_i)-k(\alpha)]Πi​=θi​[P(α,ci​)−k(α)]; a maximizer is an efficient level αie\alpha^e_iαie​. With private costs, the seller offers a menu of plans (TH,αH)(T_H,\alpha_H)(TH​,αH​), (TL,αL)(T_L,\alpha_L)(TL​,αL​), with fees Tj≥0T_j\ge0Tj​≥0. A type-iii marketer choosing plan jjj gets Uij=−Tj+P(αj,ci)U_{ij}=-T_j+P(\alpha_j,c_i)Uij​=−Tj​+P(αj​,ci​) (13). The seller maximizes Π=θ[TH−k(αH)]+(1−θ)[TL−k(αL)]\Pi=\theta[T_H-k(\alpha_H)]+(1-\theta)[T_L-k(\alpha_L)]Π=θ[TH​−k(αH​)]+(1−θ)[TL​−k(αL​)] subject to individual rationality Uii≥0U_{ii}\ge0Uii​≥0 and incentive compatibility Uii≥Uii′U_{ii}\ge U_{ii'}Uii​≥Uii′​ (problem (6)). Its solutions are written αH∗,αL∗\alpha^*_H,\alpha^*_LαH∗​,αL∗​.

Formalization targets

Goal: Proposition 5

In general αL∗=αLe\alpha^*_L=\alpha^e_LαL∗​=αLe​, and if cL<cH<v+c_L<c_H<v^+cL​<cH​<v+ (resp. v+<cL<cHv^+<c_L<c_Hv+<cL​<cH​), then

αH∗>αHe(resp. αH∗<αHe).\alpha^*_H>\alpha^e_H\quad(\text{resp. } \alpha^*_H<\alpha^e_H).αH∗​>αHe​(resp. αH∗​<αHe​).

The first identity says that the low-cost marketer, the "high type" of the screening problem, receives the efficient informativeness. The second says that the high-cost marketer's informativeness is distorted, upward when both costs lie below the rotation point and downward when both lie above it. Both are stated for every optimal menu and every efficient level.

Milestones

  1. Integration by parts: P(α,c)=∫cvˉ[1−H(v,α)] dvP(\alpha,c)=\int_c^{\bar v}[1-H(v,\alpha)]\,dvP(α,c)=∫cvˉ​[1−H(v,α)]dv (proof of Proposition 4).
  2. The cross derivative ∂2Πi/∂αi∂ci=θi ∂H(ci,αi)/∂αi\partial^2\Pi_i/\partial\alpha_i\partial c_i=\theta_i\,\partial H(c_i,\alpha_i)/\partial\alpha_i∂2Πi​/∂αi​∂ci​=θi​∂H(ci​,αi​)/∂αi​ and its sign on either side of v+v^+v+.
  3. Proposition 4: αie\alpha^e_iαie​ first increases and then decreases with cic_ici​, with the turning point at v+v^+v+.
  4. The reduction of (6) to the two-variable problem (14) by the binding constraints IRHIR_HIRH​ and ICL,HIC_{L,H}ICL,H​.
  5. (14) integrated by parts, with the high type's rent R(αH)=∫cLcH[1−H(v,αH)] dvR(\alpha_H)=\int_{c_L}^{c_H}[1-H(v,\alpha_H)]\,dvR(αH​)=∫cL​cH​​[1−H(v,αH​)]dv, and its separation, which gives αL∗=αLe\alpha^*_L=\alpha^e_LαL∗​=αLe​ for (14).
  6. The derivative ∂Π/∂αH\partial\Pi/\partial\alpha_H∂Π/∂αH​ of (14) compared with the benchmark derivative.

Significance

Proposition 4 states that the value of information to a marketer is non-monotone in the cost of acting: it is largest for costs near the rotation point v+v^+v+, where the targeting decision is hardest. Proposition 5 turns this into a pricing rule. In the two-point model the direction of the distortion depends only on the prior (Proposition 2, φ<1/2\varphi<1/2φ<1/2 or φ>1/2\varphi>1/2φ>1/2). Under the rotation order it depends on where the costs lie relative to v+v^+v+, so a seller facing marketers with low costs over-supplies information to its less profitable segment, and one facing high costs under-supplies it. The comparison with the two-point model (pp. 17–18 of the manuscript) connects the two settings through the single-crossing point of the posterior-mean distributions.

The results are proved in the paper, through integration by parts, signs of derivatives and Topkis's monotonicity theorem. None of them is formalized. Formalizing them checks the reduction of the screening problem, which the paper performs by analogy with the two-point model, including the constraints it drops without comment. It also produces a reusable treatment of rotation-ordered distribution families on an interval. Topkis's Theorem 2.8.4 is already proved on the platform and is part of this mission as a reference.

Difficulty

The obvious argument has two gaps. The first is the reduction (14). The paper substitutes the binding IRHIR_HIRH​ and ICL,HIC_{L,H}ICL,H​ constraints and drops IRLIR_LIRL​, ICH,LIC_{H,L}ICH,L​ and Tj≥0T_j\ge0Tj​≥0. That the dropped constraints hold at a solution of (14) is not shown in the paper. The second gap is strictness: Topkis's theorem compares maximizers weakly. The strict inequalities αH∗≷αHe\alpha^*_H\gtrless\alpha^e_HαH∗​≷αHe​ require showing that efficient levels are interior and that the first-order conditions differ there. This uses the footnote-5 behaviour of kkk at the endpoints and the mean-preserving property (10), which follows from (9) and the constancy of the mean.

Formalization scope

The model is one Lean structure, SellingData.ListsRotation.Model, with real parameters and its standing assumptions as fields. The committed conventions are:

  • The support [v‾,vˉ][\underline v,\bar v][v​,vˉ] is a bounded real interval; the paper allows an unbounded one.
  • H(⋅,α)H(\cdot,\alpha)H(⋅,α) is continuous on the support, with H(v‾,α)=0H(\underline v,\alpha)=0H(v​,α)=0, H(vˉ,α)=1H(\bar v,\alpha)=1H(vˉ,α)=1 and a strictly positive continuous density h(⋅,α)h(\cdot,\alpha)h(⋅,α). The derivative ∂H/∂α\partial H/\partial\alpha∂H/∂α exists on the support, for α∈(0,1)\alpha\in(0,1)α∈(0,1).
  • dH(v,α)dH(v,\alpha)dH(v,α) is h(v,α) dvh(v,\alpha)\,dvh(v,α)dv, so P(α,c)=∫cvˉ(v−c) h(v,α) dvP(\alpha,c)=\int_c^{\bar v}(v-c)\,h(v,\alpha)\,dvP(α,c)=∫cvˉ​(v−c)h(v,α)dv.
  • The rotation order (9) is imposed on the open support. The mean property is a field of the model; (10) is not assumed, since it follows.
  • The costs satisfy v‾<cL<cH<vˉ\underline v<c_L<c_H<\bar vv​<cL​<cH​<vˉ.
  • kkk is monotone on [α‾,αˉ][\underline\alpha,\bar\alpha][α​,αˉ] with k(α‾)=0k(\underline\alpha)=0k(α​)=0.

Two further hypotheses are pinned, and only on the statements that need strictness or differentiation under the integral sign:

  • the footnote-5 conditions on kkk: continuity on [α‾,αˉ][\underline\alpha,\bar\alpha][α​,αˉ], differentiability inside, k′(α‾)=0k'(\underline\alpha)=0k′(α​)=0 from the right and k′(α)→+∞k'(\alpha)\to+\inftyk′(α)→+∞ as α→αˉ−\alpha\to\bar\alpha^-α→αˉ−; convexity and twice differentiability are not assumed;
  • joint continuity of ∂H/∂α\partial H/\partial\alpha∂H/∂α on [v‾,vˉ]×(0,1)[\underline v,\bar v]\times(0,1)[v​,vˉ]×(0,1).

Optimal menus and efficient levels are sets of maximizers, not chosen maximizers, because kkk need not be convex. "αL∗=αLe\alpha^*_L=\alpha^e_LαL∗​=αLe​" is the equality of the set of low-cost plans of optimal menus with the set of efficient levels. The goal is stated for the screening problem (6) itself, with all four constraints and nonnegative fees. Defining the optimal menu through (14), or efficient levels through a first-order condition, would make half of the goal a restatement of a milestone, and is ruled out. As on the page, αL∗=αLe\alpha^*_L=\alpha^e_LαL∗​=αLe​ is stated with no condition on the position of the costs relative to v+v^+v+; only the comparison of αH∗\alpha^*_HαH∗​ with αHe\alpha^e_HαHe​ carries the two cost ranges. A sorry-free instance shows that the model's hypotheses are jointly satisfiable: [v‾,vˉ]=[0,1][\underline v,\bar v]=[0,1][v​,vˉ]=[0,1], H(v,α)=v+α2v(1−v)(1−2v)H(v,\alpha)=v+\tfrac{\alpha}{2}v(1-v)(1-2v)H(v,α)=v+2α​v(1−v)(1−2v), v+=u^=1/2v^+=\hat u=1/2v+=u^=1/2.

Contributions are welcome on the integration-by-parts and differentiation-under-the-integral lemmas for families of distributions on an interval, which are reusable beyond this paper, and on any milestone in any order.

Selected references

  • Liang Guo, Selling Data to Marketers, Management Science, 2025, author's manuscript. https://doi.org/10.1287/mnsc.2023.01350
  • Justin P. Johnson and David P. Myatt, On the Simple Economics of Advertising, Marketing, and Product Design, American Economic Review 96(3), 2006. https://doi.org/10.1257/aer.96.3.756
  • Donald M. Topkis, Supermodularity and Complementarity, Princeton University Press, 1998 (Theorem 2.8.4). https://doi.org/10.1515/9781400822539
10 thms2 active usersReviewed
EconomicsMechanism DesignOperations Research·Captain: mikedeng1

Selling Data to Marketers 2: In Selling Data Appends with Costs Below the Prior Mean, the High-Cost Plan Is Efficient and the Low-Cost Plan Carries Less Information If φ < 1/2 and More If φ > 1/2Research Paper

Motivation

Data brokers sell information about consumers to firms that decide whom to target: a lender deciding whom to offer credit, a retailer deciding whom to mail. Two products dominate this market. A marketing list sells contact information together with an analysis, so that a firm cannot reach the consumer at all without buying. A data append adds information about consumers the firm already knows, so that the firm can act even without buying. Liang Guo's Selling Data to Marketers (Management Science, doi:10.1287/mnsc.2023.01350) studies how a monopoly data seller should design a menu of data products when the buyers' targeting costs are private, and shows that the answer differs sharply between the two products.

This mission formalizes the data-append result in the paper's base model with a binary targeting value: Proposition 3 (p. 13 of the author's manuscript). The companion missions of the series cover marketing lists and the general model with a rotation-ordered family of posterior distributions.

Setting

A marketer can take an action at cost c>0c>0c>0. The action yields a value u∈{u‾,uˉ}u\in\{\underline u,\bar u\}u∈{u​,uˉ}, with u=uˉu=\bar uu=uˉ with prior probability φ∈(0,1)\varphi\in(0,1)φ∈(0,1), so the prior mean is u^=φuˉ+(1−φ)u‾\hat u=\varphi\bar u+(1-\varphi)\underline uu^=φuˉ+(1−φ)u​. Not acting yields 000.

The seller can produce a signal w∈{wh,wl}w\in\{w_h,w_l\}w∈{wh​,wl​} with Pr⁡(wh∣uˉ)=Pr⁡(wl∣u‾)=α\Pr(w_h\mid\bar u)=\Pr(w_l\mid\underline u)=\alphaPr(wh​∣uˉ)=Pr(wl​∣u​)=α, where the informativeness α\alphaα is chosen from [α‾,αˉ]⊂(1/2,1)[\underline\alpha,\bar\alpha]\subset(1/2,1)[α​,αˉ]⊂(1/2,1). The signal probabilities are σh(α)=φα+(1−φ)(1−α)\sigma_h(\alpha)=\varphi\alpha+(1-\varphi)(1-\alpha)σh​(α)=φα+(1−φ)(1−α) and σl(α)=1−σh(α)\sigma_l(\alpha)=1-\sigma_h(\alpha)σl​(α)=1−σh​(α), and the posterior means after whw_hwh​ and wlw_lwl​ are vh(α)v_h(\alpha)vh​(α) and vl(α)v_l(\alpha)vl​(α), computed by Bayes' rule. The standing assumption vl(α‾)<c<vh(α‾)v_l(\underline\alpha)<c<v_h(\underline\alpha)vl​(α​)<c<vh​(α​) makes acting worthwhile exactly after the high signal, so a marketer of cost ccc values information α\alphaα at σh(α)[vh(α)−c]\sigma_h(\alpha)[v_h(\alpha)-c]σh​(α)[vh​(α)−c]. Serving a marketer with informativeness α\alphaα costs the seller k(α)k(\alpha)k(α), with k(α‾)=0k(\underline\alpha)=0k(α​)=0 and kkk increasing.

Marketers have cost cHc_HcH​ with probability θ\thetaθ and cL<cHc_L<c_HcL​<cH​ with probability 1−θ1-\theta1−θ, privately known. The seller offers a menu of two plans (TH,αH)(T_H,\alpha_H)(TH​,αH​) and (TL,αL)(T_L,\alpha_L)(TL​,αL​) with fees Tj≥0T_j\ge0Tj​≥0. Type iii's payoff from plan jjj is

Uij=−Tj+σh(αj)[vh(αj)−ci].U_{ij}=-T_j+\sigma_h(\alpha_j)[v_h(\alpha_j)-c_i].Uij​=−Tj​+σh​(αj​)[vh​(αj​)−ci​].

With data appends, a marketer who buys nothing can still act on the prior, so its outside option is Ui0=max⁡{u^−ci,0}U_{i0}=\max\{\hat u-c_i,0\}Ui0​=max{u^−ci​,0}. The seller maximizes Π=∑iθi[Ti−k(αi)]\Pi=\sum_i\theta_i[T_i-k(\alpha_i)]Π=∑i​θi​[Ti​−k(αi​)] subject to Uii≥Ui0U_{ii}\ge U_{i0}Uii​≥Ui0​ (IRiIR_iIRi​) and Uii≥Uii′U_{ii}\ge U_{ii'}Uii​≥Uii′​ (ICi,i′IC_{i,i'}ICi,i′​). An optimal menu solves this problem; its informativeness levels are αH∗,αL∗\alpha_H^*,\alpha_L^*αH∗​,αL∗​.

The benchmark is the seller's problem when costs are observable: for each type it maximizes Πi=θi[σh(α)[vh(α)−ci]−k(α)]\Pi_i=\theta_i[\sigma_h(\alpha)[v_h(\alpha)-c_i]-k(\alpha)]Πi​=θi​[σh​(α)[vh​(α)−ci​]−k(α)] over [α‾,αˉ][\underline\alpha,\bar\alpha][α​,αˉ]. Any maximizer is an efficient informativeness αie\alpha_i^eαie​.

Formalization targets

Goal: Proposition 3 (p. 13)

When cL<cH<u^c_L<c_H<\hat ucL​<cH​<u^:

{αH∗}={αHe},φ<12 ⇒ αL∗<αLe,φ>12 ⇒ αL∗>αLe.\{\alpha_H^*\}=\{\alpha_H^e\},\qquad \varphi<\tfrac12\ \Rightarrow\ \alpha_L^*<\alpha_L^e,\qquad \varphi>\tfrac12\ \Rightarrow\ \alpha_L^*>\alpha_L^e .{αH∗​}={αHe​},φ<21​ ⇒ αL∗​<αLe​,φ>21​ ⇒ αL∗​>αLe​.

The first claim is an equality of sets: the high-cost informativeness levels of optimal menus are exactly the efficient ones at cHc_HcH​. The inequalities hold for every optimal menu and every efficient αLe\alpha_L^eαLe​.

Milestones

  1. Willingness to pay (§3.2, p. 13): for cL<cH<u^c_L<c_H<\hat ucL​<cH​<u^, Uij−Ui0=−Tj+σl(αj)[ci−vl(αj)]U_{ij}-U_{i0}=-T_j+\sigma_l(\alpha_j)[c_i-v_l(\alpha_j)]Uij​−Ui0​=−Tj​+σl​(αj​)[ci​−vl​(αj​)], which is increasing in cic_ici​.
  2. Reduction (8) (p. 13): with IRLIR_LIRL​ and ICH,LIC_{H,L}ICH,L​ binding, the problem becomes the maximization over (αH,αL)(\alpha_H,\alpha_L)(αH​,αL​) of
θ[σh(αH)[vh(αH)−cH]−σh(αL)(cL−cH)−k(αH)]+(1−θ)[σh(αL)[vh(αL)−cL]−k(αL)]−u^+cL.\theta\big[\sigma_h(\alpha_H)[v_h(\alpha_H)-c_H]-\sigma_h(\alpha_L)(c_L-c_H)-k(\alpha_H)\big]+(1-\theta)\big[\sigma_h(\alpha_L)[v_h(\alpha_L)-c_L]-k(\alpha_L)\big]-\hat u+c_L .θ[σh​(αH​)[vh​(αH​)−cH​]−σh​(αL​)(cL​−cH​)−k(αH​)]+(1−θ)[σh​(αL​)[vh​(αL​)−cL​]−k(αL​)]−u^+cL​.
  1. The αL\alpha_LαL​-derivatives (proof of Proposition 3, p. 32): the benchmark derivative (1−θ)[φ(uˉ−cL)−(1−φ)(u‾−cL)−k′(αL)](1-\theta)[\varphi(\bar u-c_L)-(1-\varphi)(\underline u-c_L)-k'(\alpha_L)](1−θ)[φ(uˉ−cL​)−(1−φ)(u​−cL​)−k′(αL​)] and the derivative of (8), which adds θ(1−2φ)(cL−cH)\theta(1-2\varphi)(c_L-c_H)θ(1−2φ)(cL​−cH​).

Significance

The result says that a data seller's distortion depends on the product's form. With marketing lists the low-cost marketers are the high type, their plan is efficient, and the high-cost plan is distorted (Proposition 2, p. 11). With data appends and costs below the prior mean, the outside option u^−ci\hat u-c_iu^−ci​ falls with cic_ici​ faster than the value of information does, so high-cost marketers have the larger willingness to pay. The roles of the types switch, the low-cost plan is distorted, and the direction of the distortion is reversed, giving the paper's "double mirror image" (p. 13). The direction is set by the skewness of the prior through the sign of 1−2φ1-2\varphi1−2φ, not by an assumed single-crossing condition.

The result is proved in the paper, in an appendix argument of a few lines that invokes Topkis's monotonicity theorem. To our knowledge it has no machine-checked proof. Formalizing it makes explicit what the printed proof leaves to the reader: that the constraints dropped in (8) hold at its optimum, that the efficient and optimal informativeness need not be unique, and which regularity of kkk the strict inequalities need.

Difficulty

The obvious argument substitutes the binding constraints, compares first-order conditions, and invokes Topkis's theorem. Three steps of it are not automatic. First, the reduction drops IRHIR_HIRH​, ICL,HIC_{L,H}ICL,H​ and the nonnegativity of fees; these must be shown to hold at every maximizer of (8), and ICL,HIC_{L,H}ICL,H​ holds only because the maximizers of (8) are ordered in the right direction, which is itself a comparative-statics fact. Second, the reduction identifies the optimal menus only if (8) attains its maximum. Third, Topkis's theorem yields only weak inequalities, αL∗≤αLe\alpha_L^*\le\alpha_L^eαL∗​≤αLe​; strictness needs the efficient informativeness to be interior and kkk to be differentiable there, which footnote 5 (p. 7) supplies. Without it, both optima can sit at the same corner α‾\underline\alphaα​ or αˉ\bar\alphaαˉ.

Formalization scope

Everything is stated over R\mathbb RR in the sub-namespace SellingData.AppendsBinary. The two types form an inductive type CostType with constructors H and L. A Model structure holds u‾<uˉ\underline u<\bar uu​<uˉ, φ∈(0,1)\varphi\in(0,1)φ∈(0,1), 1/2<α‾<αˉ<11/2<\underline\alpha<\bar\alpha<11/2<α​<αˉ<1, θ∈(0,1)\theta\in(0,1)θ∈(0,1), 0<cL<cH0<c_L<c_H0<cL​<cH​, k(α‾)=0k(\underline\alpha)=0k(α​)=0, kkk weakly increasing on [α‾,αˉ][\underline\alpha,\bar\alpha][α​,αˉ], and the standing assumption vl(α‾)<ci<vh(α‾)v_l(\underline\alpha)<c_i<v_h(\underline\alpha)vl​(α​)<ci​<vh​(α​) for both costs. The posteriors are defined with the divisions of the paper's (2). The outside option is the printed max⁡{u^−ci,0}\max\{\hat u-c_i,0\}max{u^−ci​,0}.

Optimality and efficiency are predicates on maximizers, never a chosen maximizer. The optimal menu is defined from the data-append screening problem itself; defining it through (8) would make the first claim of the goal trivial and is ruled out. The goal additionally assumes the part of footnote 5 that the strict inequalities need: kkk continuous on [α‾,αˉ][\underline\alpha,\bar\alpha][α​,αˉ], differentiable on (α‾,αˉ)(\underline\alpha,\bar\alpha)(α​,αˉ), right derivative 000 at α‾\underline\alphaα​, and k′(α)→+∞k'(\alpha)\to+\inftyk′(α)→+∞ as α→αˉ−\alpha\to\bar\alpha^-α→αˉ−. Convexity and twice differentiability, which the footnote also lists, are not assumed. The milestones carry only what they need: none for the willingness to pay, continuity of kkk for one part of the reduction, and differentiability at the point for the derivative.

A complete development needs Bayes posteriors for a symmetric binary signal, elementary calculus (derivatives, the mean value theorem, extrema on compact intervals), and Topkis's monotonicity theorem, which is on the platform (Theorem 2.8.4) and is referenced by this mission. Proofs of the milestones and of the goal are welcome. Lemmas on the binary-signal posteriors (for instance the identity σh(α)vh(α)+σl(α)vl(α)=u^\sigma_h(\alpha)v_h(\alpha)+\sigma_l(\alpha)v_l(\alpha)=\hat uσh​(α)vh​(α)+σl​(α)vl​(α)=u^) are reusable by the companion mission on marketing lists.

Selected references

  • Liang Guo, Selling Data to Marketers, Management Science, 2025, author's manuscript. doi:10.1287/mnsc.2023.01350
  • Donald M. Topkis, Supermodularity and Complementarity, Princeton University Press, 1998. doi:10.1515/9781400822539
7 thms2 active usersReviewed
EconomicsOperations Research·Captain: mikedeng1

Third-Degree Price Discrimination in Two-Sided Markets: Seller-Side Discrimination Raises Platform Profit, Buyer and Low-Type Seller Surplus and Total Welfare; High-Type Sellers Gain iff b > b̂Research Paper

Price discrimination when demand has network effects

Third-degree price discrimination means charging different prices to identifiable groups of customers. Its welfare effects are a classical question in industrial organization. With linear demands and every market served under uniform pricing, discrimination leaves total output unchanged and lowers total welfare (Pigou 1920). More generally, welfare can rise only if total output rises (Schmalensee 1981; Varian 1985).

Many current markets are two-sided platforms: marketplaces, app stores and payment networks that charge sellers participation fees and differentiate those fees by seller category (for instance, app stores charging games and non-games differently). On such a platform, a seller's presence is worth something to the buyers on the other side. De Cornière, Mantovani and Shekhar (Management Science 71(4), 2025) show that this cross-side benefit overturns the classical conclusion: discriminating between seller types raises seller participation, which attracts more buyers, which raises seller participation further. In their linear model, total welfare rises, and for large enough cross-side benefits every group gains.

The baseline model

A monopoly platform serves a mass one of buyers and two categories of sellers, LLL and HHH, each of mass one.

  • A buyer obtains a stand-alone value vvv from the platform and a benefit bbb for each seller on it.
  • A seller of type j∈{L,H}j\in\{L,H\}j∈{L,H} earns θj\theta_jθj​ per buyer on the platform, with θH>θL>0\theta_H>\theta_L>0θH​>θL​>0.
  • All agents have outside options uniform on [0,1][0,1][0,1]. With NBN_BNB​ buyers, the type-jjj seller demand at fee fjf_jfj​ is θjNB−fj\theta_jN_B-f_jθj​NB​−fj​. With NSN_SNS​ sellers, the buyer demand at price ppp is v+bNS−pv+bN_S-pv+bNS​−p.

The platform chooses participation levels, and the fees follow from the inverse demands.

Uniform pricing (fL=fHf_L=f_HfL​=fH​). The platform chooses (NB,NS)(N_B,N_S)(NB​,NS​) to maximize

ΠU(NB,NS)=NB (v+bNS−NB)+NS (θL+θH)NB−NS2.\Pi^U(N_B,N_S)=N_B\,(v+bN_S-N_B)+N_S\,\frac{(\theta_L+\theta_H)N_B-N_S}{2}.ΠU(NB​,NS​)=NB​(v+bNS​−NB​)+NS​2(θL​+θH​)NB​−NS​​.

At the common fee fUf^UfU, type-jjj participation is NjU=θjNB−fUN^U_j=\theta_jN_B-f^UNjU​=θj​NB​−fU.

Price discrimination. The platform chooses (NB,NL,NH)(N_B,N_L,N_H)(NB​,NL​,NH​) to maximize

ΠD(NB,NL,NH)=NB(v+b(NL+NH)−NB)+NL(θLNB−NL)+NH(θHNB−NH).\Pi^D(N_B,N_L,N_H)=N_B\big(v+b(N_L+N_H)-N_B\big)+N_L(\theta_LN_B-N_L)+N_H(\theta_HN_B-N_H).ΠD(NB​,NL​,NH​)=NB​(v+b(NL​+NH​)−NB​)+NL​(θL​NB​−NL​)+NH​(θH​NB​−NH​).

The participant with outside option kkk joins when its net value exceeds kkk. Buyer surplus CSCSCS and type-jjj seller surplus DSjDS_jDSj​ are therefore integrals over kkk from 000 to the participation level of (net value − k-\,k−k). Total welfare is SW=CS+Π+DSL+DSHSW=CS+\Pi+DS_L+DS_HSW=CS+Π+DSL​+DSH​.

Assumption 1 (p. 3343) restricts the parameters: (i) 0<v<4−(b+θL)2−(b+θH)20<v<4-(b+\theta_L)^2-(b+\theta_H)^20<v<4−(b+θL​)2−(b+θH​)2; (ii) (θH−3θL)/2<b(\theta_H-3\theta_L)/2<b(θH​−3θL​)/2<b.

Two denominators recur: U=8−(2b+θH+θL)2U=8-(2b+\theta_H+\theta_L)^2U=8−(2b+θH​+θL​)2 and D=4−2b2−θH2−θL2−2b(θH+θL)D=4-2b^2-\theta_H^2-\theta_L^2-2b(\theta_H+\theta_L)D=4−2b2−θH2​−θL2​−2b(θH​+θL​).

Formalization targets

Goal: Proposition 2 (p. 3346)

Under θH>θL>0\theta_H>\theta_L>0θH​>θL​>0, b>0b>0b>0 and Assumption 1, both optimization problems have a solution, and for every uniform optimum and every discrimination optimum:

ΠU<ΠD,CSU<CSD,DSLU<DSLD,SWU<SWD,\Pi^U<\Pi^D,\qquad CS^U<CS^D,\qquad DS^U_L<DS^D_L,\qquad SW^U<SW^D,ΠU<ΠD,CSU<CSD,DSLU​<DSLD​,SWU<SWD, DSHU<DSHD  ⟺  b>b^(θH,θL)=32−7(θH−θL)2−3θH−θL4.DS^U_H<DS^D_H\iff b>\hat b(\theta_H,\theta_L)=\frac{\sqrt{32-7(\theta_H-\theta_L)^2}-3\theta_H-\theta_L}{4}.DSHU​<DSHD​⟺b>b^(θH​,θL​)=432−7(θH​−θL​)2​−3θH​−θL​​.

When b>b^b>\hat bb>b^, all four groups (platform, buyers, low-type sellers and high-type sellers) are strictly better off: a strict Pareto improvement.

Milestones

  1. The uniform optimum, display (12): NBU=4v/UN^U_B=4v/UNBU​=4v/U, NSU=2v(2b+θH+θL)/UN^U_S=2v(2b+\theta_H+\theta_L)/UNSU​=2v(2b+θH​+θL​)/U, ΠU=2v2/U\Pi^U=2v^2/UΠU=2v2/U, with the fees and, from footnote 14, NjUN^U_jNjU​.
  2. The discrimination optimum, (13)–(15): NBD=2v/DN^D_B=2v/DNBD​=2v/D, NjD=v(b+θj)/DN^D_j=v(b+\theta_j)/DNjD​=v(b+θj​)/D, ΠD=v2/D\Pi^D=v^2/DΠD=v2/D, with the fees.
  3. Proposition 1: NBU<NBDN^U_B<N^D_BNBU​<NBD​ and NSU<NLD+NHDN^U_S<N^D_L+N^D_HNSU​<NLD​+NHD​.
  4. The surpluses and welfare at the uniform optimum (A.2.1).
  5. The surpluses and welfare at the discrimination optimum, (A.6)–(A.8).
  6. ΠD−ΠU=v2(θH−θL)2/(DU)>0\Pi^D-\Pi^U=v^2(\theta_H-\theta_L)^2/(DU)>0ΠD−ΠU=v2(θH​−θL​)2/(DU)>0.
  7. DSLD−DSLU>0DS^D_L-DS^U_L>0DSLD​−DSLU​>0.
  8. SWD−SWU=v2(θH−θL)2λ/(2D2U2)SW^D-SW^U=v^2(\theta_H-\theta_L)^2\lambda/(2D^2U^2)SWD−SWU=v2(θH​−θL​)2λ/(2D2U2) for an explicit quartic λ(b,θH,θL)\lambda(b,\theta_H,\theta_L)λ(b,θH​,θL​).
  9. λ>0\lambda>0λ>0 on the parameter region.
  10. DSHD−DSHU>0  ⟺  b>b^DS^D_H-DS^U_H>0\iff b>\hat bDSHD​−DSHU​>0⟺b>b^.

Significance

The result. Proposition 2 is a counterpoint to the Pigouvian benchmark. In a two-sided market with linear demands, seller-side discrimination can be a strict Pareto improvement, so a blanket presumption against fee differentiation by platforms is not justified on welfare grounds. The comparison also isolates the mechanism. The low-type sellers gain even though the platform could already serve them, and the high types gain only when the buyer-side benefit bbb is large enough. The paper's extensions in §5 (seller-type-dependent buyer benefits, ad valorem fees, competition) start from this baseline.

Formalizing it. The paper proves Proposition 2 by explicit algebra in Appendix A.2. Most of that algebra is printed, but the positivity of λ\lambdaλ is argued only through claims about the roots of a quartic in bbb, which are asserted rather than shown. No part of the argument is machine-checked. A formalization would give:

  • a certified positivity proof for λ\lambdaλ;
  • verified closed forms for both optima, with existence and uniqueness of the maximizers;
  • corrected versions of several printed displays (see the scope section).

The model is a small, self-contained template for platform pricing with linear demands, and the same definitions can serve later comparisons (fees, prices, exclusion).

Difficulty

The optimization steps are routine: both objectives are concave quadratics, strictly concave exactly when U>0U>0U>0 and D>0D>0D>0, and Assumption 1(i) gives D>v>0D>v>0D>v>0. The integrals have affine integrands. The comparison steps are where the work is.

  • λ>0\lambda>0λ>0. λ\lambdaλ is a quartic in three variables that must be shown positive on a region cut out by D>0D>0D>0, b>0b>0b>0, 0<θL<θH0<\theta_L<\theta_H0<θL​<θH​ and condition (ii). Its minimum over the region is small (numerically about 0.020.020.02, approached near the boundary D=0D=0D=0), so crude bounds fail. The paper's argument ("only one positive root bsolb^{sol}bsol", "bsolb^{sol}bsol greater than bˉ\bar bbˉ") is not a proof.
  • The b^\hat bb^ equivalence. It compares two rational functions with different denominators D2D^2D2 and U2U^2U2. It reduces to the sign of (b+θH)U−(2b+3θH−θL)D(b+\theta_H)U-(2b+3\theta_H-\theta_L)D(b+θH​)U−(2b+3θH​−θL​)D on the region, which must be tied to the root b^\hat bb^ of a quadratic.
  • Proposition 1 is a comparison of explicit rational functions under the same region constraints. The paper argues it through best-response curves instead (App. A.1).

Formalization scope

  • Representation. Parameters are real numbers v,b,θH,θLv,b,\theta_H,\theta_Lv,b,θH​,θL​. A uniform allocation is (NB,NS)∈R2(N_B,N_S)\in\mathbb R^2(NB​,NS​)∈R2, a discriminatory one (NB,NL,NH)∈R3(N_B,N_L,N_H)\in\mathbb R^3(NB​,NL​,NH​)∈R3 (low type before high type).
  • Equilibrium means a global maximizer of ΠU\Pi^UΠU (resp. ΠD\Pi^DΠD) over all real participation levels, the object the first-order conditions (7)–(11) solve. No [0,1][0,1][0,1] box is imposed.
  • Surpluses are Lean interval integrals at an arbitrary allocation. Uniform-regime seller participation by type is the demand at the common fee.
  • Standing assumptions and pins.
    • θH>θL>0\theta_H>\theta_L>0θH​>θL​>0 is §3.1's assumption.
    • b>0b>0b>0 is the standing assumption of §3.2 and footnote 10.
    • Assumption 1 is used as printed. The paper's bound bˉ\bar bbˉ (p. 3344) corresponds to 2v<4−(b+θL)2−(b+θH)22v<4-(b+\theta_L)^2-(b+\theta_H)^22v<4−(b+θL​)2−(b+θH​)2, the condition for interior buyer participation NBD=2v/D<1N^D_B=2v/D<1NBD​=2v/D<1. Under the printed v<…v<\dotsv<…, participation can exceed one, so the mission does not pose §3.4's claim that all participation levels are interior. Every comparison in the goal holds over the printed region.
  • Printed slips, corrected in the Lean and disclosed.
    • The ΠD−ΠU\Pi^D-\Pi^UΠD−ΠU denominator prints 2b(θH−θL)2b(\theta_H-\theta_L)2b(θH​−θL​) where DDD has 2b(θH+θL)2b(\theta_H+\theta_L)2b(θH​+θL​).
    • The SWD−SWUSW^D-SW^USWD−SWU denominator prints θl\theta_lθl​ for θL\theta_LθL​.
    • The DSLDS_LDSL​ and DSHDS_HDSH​ difference displays swap their two fractions.
    • "∑i=1,2DSj\sum_{i=1,2}DS_j∑i=1,2​DSj​" means DSL+DSHDS_L+DS_HDSL​+DSH​.
  • Ruled-out formalizations. The optima are not defined by their closed forms, the surpluses are not defined by their final expressions, and the goal and the equilibrium milestones assert existence of the maximizers, so the universally quantified comparisons are not vacuous. Uniform-regime seller participation is split by demand, never as NS/2N_S/2NS​/2.
  • Not posed. Propositions 3–9, Lemma 1, §4.4 and §5 are not part of this mission. Propositions 3–5 need box-constrained or corner models, and the proofs for §5 are in an Online Appendix.
  • Infrastructure. Maximizers of strictly concave quadratics on Rn\mathbb R^nRn, integrals of affine functions, and polynomial positivity certificates (SOS or case splits) on semialgebraic regions. A clean lemma that a negative-definite quadratic form plus a linear term has a unique maximizer given by its first-order conditions would be reusable well beyond this mission. Contributions toward any milestone, and alternative proofs of λ>0\lambda>0λ>0, are welcome.

Selected references

  • A. de Cornière, A. Mantovani, S. Shekhar, Third-Degree Price Discrimination in Two-Sided Markets, Management Science 71(4):3340–3356, 2025. doi:10.1287/mnsc.2023.02788
  • A. C. Pigou, The Economics of Welfare, Macmillan, London, 1920.
  • R. Schmalensee, Output and Welfare Implications of Monopolistic Third-Degree Price Discrimination, American Economic Review 71(1):242–247, 1981.
  • H. R. Varian, Price Discrimination and Social Welfare, American Economic Review 75(4):870–875, 1985.
  • M. Armstrong, Competition in Two-Sided Markets, RAND Journal of Economics 37(3):668–691, 2006. doi:10.1111/j.1756-2171.2006.tb00037.x
12 thms2 active usersReviewed
EconomicsMechanism DesignOperations Research·Captain: mikedeng1

Selling Data to Marketers 1: In Selling Marketing Lists, the Low-Cost Plan Is Efficient and the High-Cost Plan Carries More Information Than the Benchmark If φ < 1/2 and Less If φ > 1/2Research Paper

Why sell information to marketers

Data brokers such as Acxiom, Nielsen and Oracle, and data generators such as Google and Facebook, sell processed consumer data to firms that use it to decide whom to target, whom to lend to, or which market to enter. The U.S. Federal Trade Commission's 2014 report on data brokers distinguishes two products: a marketing list, which carries contact information without which the buyer cannot act at all, and a data append, which enriches data the buyer already holds. Liang Guo's Selling Data to Marketers (Management Science, doi:10.1287/mnsc.2023.01350) asks how a seller should design a menu of data products of different informativeness when buyers differ in a privately known cost of acting.

The question belongs to the theory of optimal screening (Mussa and Rosen 1978; Maskin and Riley 1984), but with a twist: the buyer's marginal value of information is not a primitive. It is derived from the buyer's decision problem, so the direction of the single-crossing condition is itself an outcome of the model. This mission formalizes the paper's analysis of marketing lists in the base model with a binary value.

Setting

A marketer can take an action (target a consumer) at cost c>0c>0c>0. The action's value uuu is uˉ\bar uuˉ with prior probability ϕ∈(0,1)\phi\in(0,1)ϕ∈(0,1) and u‾<uˉ\underline u<\bar uu​<uˉ otherwise. The seller can provide a binary signal w∈{wh,wl}w\in\{w_h,w_l\}w∈{wh​,wl​} with Pr⁡(wh∣uˉ)=Pr⁡(wl∣u‾)=α\Pr(w_h\mid\bar u)=\Pr(w_l\mid\underline u)=\alphaPr(wh​∣uˉ)=Pr(wl​∣u​)=α, where the informativeness α\alphaα lies in an interval [α‾,αˉ]⊂(1/2,1)[\underline\alpha,\bar\alpha]\subset(1/2,1)[α​,αˉ]⊂(1/2,1). The probability of the optimistic signal is σh(α)=ϕα+(1−ϕ)(1−α)\sigma_h(\alpha)=\phi\alpha+(1-\phi)(1-\alpha)σh​(α)=ϕα+(1−ϕ)(1−α), the posterior probability of uˉ\bar uuˉ after it is ϕh(α)=ϕα/σh(α)\phi_h(\alpha)=\phi\alpha/\sigma_h(\alpha)ϕh​(α)=ϕα/σh​(α), and the posterior mean is vh(α)=ϕh(α)uˉ+[1−ϕh(α)]u‾v_h(\alpha)=\phi_h(\alpha)\bar u+[1-\phi_h(\alpha)]\underline uvh​(α)=ϕh​(α)uˉ+[1−ϕh​(α)]u​; σl,ϕl,vl\sigma_l,\phi_l,v_lσl​,ϕl​,vl​ are defined symmetrically. The paper assumes vl(α‾)<c<vh(α‾)v_l(\underline\alpha)<c<v_h(\underline\alpha)vl​(α​)<c<vh​(α​), so a marketer acts exactly after whw_hwh​, and a marketer with cost ccc who pays TTT for a signal of informativeness α\alphaα gets

U=−T+σh(α)[vh(α)−c].U=-T+\sigma_h(\alpha)[v_h(\alpha)-c].U=−T+σh​(α)[vh​(α)−c].

Serving a marketer at informativeness α\alphaα costs the seller k(α)k(\alpha)k(α), with k(α‾)=0k(\underline\alpha)=0k(α​)=0 and kkk increasing.

There are two cost types, cH>cL>0c_H>c_L>0cH​>cL​>0, with Pr⁡(H)=θ∈(0,1)\Pr(H)=\theta\in(0,1)Pr(H)=θ∈(0,1); write θH=θ\theta_H=\thetaθH​=θ, θL=1−θ\theta_L=1-\thetaθL​=1−θ. Under the benchmark with observable cost, the seller extracts all surplus and chooses α\alphaα to maximize

Πi(α)=θi[σh(α)[vh(α)−ci]−k(α)],\Pi_i(\alpha)=\theta_i\big[\sigma_h(\alpha)[v_h(\alpha)-c_i]-k(\alpha)\big],Πi​(α)=θi​[σh​(α)[vh​(α)−ci​]−k(α)],

whose maximizers are the efficient levels αie\alpha^e_iαie​. With private costs the seller offers a menu (TH,αH),(TL,αL)(T_H,\alpha_H),(T_L,\alpha_L)(TH​,αH​),(TL​,αL​) with Tj≥0T_j\ge0Tj​≥0 and maximizes Π=∑iθi[Ti−k(αi)]\Pi=\sum_i\theta_i[T_i-k(\alpha_i)]Π=∑i​θi​[Ti​−k(αi​)] subject to individual rationality Uii≥0U_{ii}\ge0Uii​≥0 and incentive compatibility Uii≥Uii′U_{ii}\ge U_{ii'}Uii​≥Uii′​, where Uij=−Tj+σh(αj)[vh(αj)−ci]U_{ij}=-T_j+\sigma_h(\alpha_j)[v_h(\alpha_j)-c_i]Uij​=−Tj​+σh​(αj​)[vh​(αj​)−ci​]. This is problem (6); its solutions are the optimal menus, with informativeness levels αH∗,αL∗\alpha^*_H,\alpha^*_LαH∗​,αL∗​.

Formalization targets

Goal: Proposition 2

Under the regularity of kkk described below,

αL∗=αLe,αH∗>αHe  if ϕ<12,αH∗<αHe  if ϕ>12.\alpha^*_L=\alpha^e_L,\qquad \alpha^*_H>\alpha^e_H\ \text{ if }\phi<\tfrac12,\qquad \alpha^*_H<\alpha^e_H\ \text{ if }\phi>\tfrac12.αL∗​=αLe​,αH∗​>αHe​  if ϕ<21​,αH∗​<αHe​  if ϕ>21​.

The first claim is an equality of sets of maximizers; the inequalities compare every optimal menu with every efficient level.

Milestones

  1. The derivative of the benchmark profit: ∂Πi/∂αi=θi[ϕ(uˉ−ci)−(1−ϕ)(u‾−ci)−k′(αi)]\partial\Pi_i/\partial\alpha_i=\theta_i[\phi(\bar u-c_i)-(1-\phi)(\underline u-c_i)-k'(\alpha_i)]∂Πi​/∂αi​=θi​[ϕ(uˉ−ci​)−(1−ϕ)(u​−ci​)−k′(αi​)].
  2. The cross derivative ∂2Πi/∂αi∂ci=θi(1−2ϕ)\partial^2\Pi_i/\partial\alpha_i\partial c_i=\theta_i(1-2\phi)∂2Πi​/∂αi​∂ci​=θi​(1−2ϕ), stated as a mixed second difference.
  3. Proposition 1: αie\alpha^e_iαie​ increases with cic_ici​ if ϕ<1/2\phi<1/2ϕ<1/2 and decreases if ϕ>1/2\phi>1/2ϕ>1/2.
  4. The reduction of (6) to the problem in (αH,αL)(\alpha_H,\alpha_L)(αH​,αL​) alone,
max⁡αH,αL θ[σh(αH)[vh(αH)−cH]−k(αH)]+(1−θ)[σh(αL)[vh(αL)−cL]−σh(αH)(cH−cL)−k(αL)],(7)\max_{\alpha_H,\alpha_L}\ \theta\big[\sigma_h(\alpha_H)[v_h(\alpha_H)-c_H]-k(\alpha_H)\big]+(1-\theta)\big[\sigma_h(\alpha_L)[v_h(\alpha_L)-c_L]-\sigma_h(\alpha_H)(c_H-c_L)-k(\alpha_L)\big],\qquad(7)αH​,αL​max​ θ[σh​(αH​)[vh​(αH​)−cH​]−k(αH​)]+(1−θ)[σh​(αL​)[vh​(αL​)−cL​]−σh​(αH​)(cH​−cL​)−k(αL​)],(7)

obtained by making IRH_HH​ and ICL,H_{L,H}L,H​ bind. 5. The derivative of (7) in αH\alpha_HαH​: θ[ϕ(uˉ−cH)−(1−ϕ)(u‾−cH)−k′(αH)]+(1−θ)(1−2ϕ)(cH−cL)\theta[\phi(\bar u-c_H)-(1-\phi)(\underline u-c_H)-k'(\alpha_H)]+(1-\theta)(1-2\phi)(c_H-c_L)θ[ϕ(uˉ−cH​)−(1−ϕ)(u​−cH​)−k′(αH​)]+(1−θ)(1−2ϕ)(cH​−cL​).

Significance

In textbook screening the low type's allocation is distorted downward to reduce the high type's rent. Here the low type of the screening problem is the high-cost marketer, and its information can be distorted in either direction: the rent σh(αH)(cH−cL)\sigma_h(\alpha_H)(c_H-c_L)σh​(αH​)(cH​−cL​) conceded to low-cost marketers falls with αH\alpha_HαH​ when ϕ<1/2\phi<1/2ϕ<1/2 and rises when ϕ>1/2\phi>1/2ϕ>1/2. Proposition 2 therefore says that a seller of marketing lists may supply socially excessive information to some buyers, a possibility absent from the data-appends literature the paper compares against, where provision can only be insufficient. Proposition 1 isolates the mechanism: the prior decides whether information mainly prevents wasteful targeting or restores profitable targeting, and the cost of acting amplifies one or the other.

The result is proved in the paper; nothing in it has been machine-checked. The mission produces a checked version of the two-type screening reduction (including the verification that the constraints dropped in (7) hold at its solutions, which the paper asserts as standard) and of the comparative statics with set-valued optima, since kkk is not assumed convex and optima need not be unique.

Difficulty

The derivative computations are routine once σh(α)vh(α)\sigma_h(\alpha)v_h(\alpha)σh​(α)vh​(α) is simplified to ϕαuˉ+(1−ϕ)(1−α)u‾\phi\alpha\bar u+(1-\phi)(1-\alpha)\underline uϕαuˉ+(1−ϕ)(1−α)u​. The work lies elsewhere. First, the reduction: "IR binds for the high-cost type and IC binds for the low-cost type" is the standard heuristic, but at a solution of (7) one still has to check the dropped constraint ICH,L_{H,L}H,L​ and the nonnegativity TL≥0T_L\ge0TL​≥0; both depend on the ordering of αH\alpha_HαH​ and αL\alpha_LαL​ at the optimum, which in turn depends on the sign of 1−2ϕ1-2\phi1−2ϕ. Second, strictness: comparative statics through increasing differences give only weak inequalities between maximizer sets. Upgrading αH∗≥αHe\alpha^*_H\ge\alpha^e_HαH∗​≥αHe​ to αH∗>αHe\alpha^*_H>\alpha^e_HαH∗​>αHe​ requires ruling out a common maximizer, which needs the optimum to be interior and kkk differentiable there; the naive argument by first-order conditions fails at the corners α‾,αˉ\underline\alpha,\bar\alphaα​,αˉ. Third, αL∗=αLe\alpha^*_L=\alpha^e_LαL∗​=αLe​ as sets requires existence of optimal menus, hence attainment of the maximum of (7).

Formalization scope

Everything is over R\mathbb RR. The model is a structure Model (values, prior, informativeness interval, selling cost) extended by Screening (type probability and the two costs); the standing assumptions u‾<uˉ\underline u<\bar uu​<uˉ, 0<ϕ<10<\phi<10<ϕ<1, 1/2<α‾<αˉ<11/2<\underline\alpha<\bar\alpha<11/2<α​<αˉ<1, k(α‾)=0k(\underline\alpha)=0k(α​)=0, kkk increasing on [α‾,αˉ][\underline\alpha,\bar\alpha][α​,αˉ] (weakly), 0<θ<10<\theta<10<θ<1, 0<cL<cH0<c_L<c_H0<cL​<cH​, and vl(α‾)<ci<vh(α‾)v_l(\underline\alpha)<c_i<v_h(\underline\alpha)vl​(α​)<ci​<vh​(α​) for both types are fields. The posteriors are defined with the divisions printed in (2). Efficient levels and optimal menus are sets of maximizers, never chosen by Classical.choose or a first-order condition. Menus include Tj≥0T_j\ge0Tj​≥0.

The goal additionally assumes the part of footnote 5 (p. 7) that makes optima interior: kkk continuous on [α‾,αˉ][\underline\alpha,\bar\alpha][α​,αˉ], differentiable on (α‾,αˉ)(\underline\alpha,\bar\alpha)(α​,αˉ), with right derivative k′(α‾)=0k'(\underline\alpha)=0k′(α​)=0 and k′(α)→+∞k'(\alpha)\to+\inftyk′(α)→+∞ as α→αˉ−\alpha\to\bar\alpha^-α→αˉ−. Convexity and twice differentiability, also in the footnote, are not assumed. The milestones carry no regularity beyond differentiability at the point considered (milestones 1, 5) and continuity for the optimality characterization in milestone 4.

The optimal menu is a maximizer of (6) under all four constraints, not a maximizer of (7): defining it through (7) would make half of the goal immediate and skip the reduction.

The development needs the algebra of the binary posterior, the two-type screening reduction, and a strict version of monotone comparative statics; Topkis's theorem on strictly increasing differences (Theorem 2.8.4) is already proved on the platform and referenced. Proofs of any milestone, and alternative arguments for the strict inequalities, are welcome.

Selected references

  • L. Guo, Selling Data to Marketers, Management Science, 2025 (author's manuscript). https://doi.org/10.1287/mnsc.2023.01350
  • D. M. Topkis, Supermodularity and Complementarity, Princeton University Press, 1998. https://doi.org/10.1515/9781400822539
  • M. Mussa and S. Rosen, Monopoly and Product Quality, Journal of Economic Theory 18(2), 1978. https://doi.org/10.1016/0022-0531(78)90085-6
  • E. Maskin and J. Riley, Monopoly with Incomplete Information, RAND Journal of Economics 15(2), 1984. https://doi.org/10.2307/2555674
  • Federal Trade Commission, Data Brokers: A Call for Transparency and Accountability, 2014. https://www.ftc.gov/reports/data-brokers-call-transparency-accountability-report-federal-trade-commission-may-2014
8 thms2 active usersReviewed
EconomicsOperations Research·Captain: mikedeng1

The Bright Side of the GDPR: Welfare-Improving Privacy Management I: The Equilibrium Price Is Higher After the GDPR for Every Opt-in Benefit and Data RevenueResearch Paper

Motivation

The European Union's General Data Protection Regulation (GDPR, in force since 2018) requires a firm to obtain a consumer's explicit opt-in consent before it collects her personal data. Most of the economic literature reads such consent rules as a cost to data-driven firms. Choe, Matsushima and Shekhar (ISER Discussion Paper No. 1246, Osaka University, June 2024; published in Management Science, 2025, doi:10.1287/mnsc.2024.06653) ask what happens to the price of a digital service when consumers gain the right to buy it without sharing data. Their model is a monopoly with two-dimensional consumer heterogeneity, simple enough to be solved in closed form, and it lets the answer be stated for every value of the firm's opt-in benefit and every mix of usage-based and data-based revenue.

This mission formalizes the paper's general price comparison, Proposition A1: the equilibrium price is higher after the GDPR.

Setting

A monopolist sells an online service at zero marginal cost and price p∈Rp \in \mathbb{R}p∈R. A consumer has a type (v,c)∈[0,1]2(v, c) \in [0,1]^2(v,c)∈[0,1]2, where vvv is her valuation for the service and ccc her privacy cost of sharing data; types are independent and uniform on [0,1]2[0,1]^2[0,1]2, so the mass of a set of consumers is its area. A consumer who shares data receives an opt-in benefit x∈(0,1)x \in (0, 1)x∈(0,1), and the firm earns an exogenous data revenue α≥0\alpha \ge 0α≥0 per unit of shared data.

The utilities of opt-in purchase (buy and share) and opt-out purchase (buy without sharing) are

US(v,c)=v+(x−c)−p,UN(v,c)=v−p.U_S(v,c) = v + (x - c) - p, \qquad U_N(v,c) = v - p.US​(v,c)=v+(x−c)−p,UN​(v,c)=v−p.
  • Before the GDPR opt-out purchase is unavailable. A consumer buys, and shares data, iff US≥0U_S \ge 0US​≥0. The demand D^(p)\widehat D(p)D(p) is the area of {(v,c)∈[0,1]2:c≤v+x−p}\{(v,c) \in [0,1]^2 : c \le v + x - p\}{(v,c)∈[0,1]2:c≤v+x−p}, and the profit is Π^(p)=(p+α)D^(p)\widehat\Pi(p) = (p + \alpha)\widehat D(p)Π(p)=(p+α)D(p).
  • After the GDPR a consumer opts in iff US≥max⁡{UN,0}U_S \ge \max\{U_N, 0\}US​≥max{UN​,0} and opts out iff UN≥max⁡{US,0}U_N \ge \max\{U_S, 0\}UN​≥max{US​,0}. The opt-in set is {c≤v+x−p, c≤x}\{c \le v + x - p,\ c \le x\}{c≤v+x−p, c≤x}, the opt-out set is {v≥p, c>x}\{v \ge p,\ c > x\}{v≥p, c>x}, their areas are D~S(p)\widetilde D_S(p)DS​(p) and D~N(p)\widetilde D_N(p)DN​(p), and the profit is Π~(p)=p (D~S(p)+D~N(p))+α D~S(p)\widetilde\Pi(p) = p\,(\widetilde D_S(p) + \widetilde D_N(p)) + \alpha\,\widetilde D_S(p)Π(p)=p(DS​(p)+DN​(p))+αDS​(p).

The firm chooses the price first and consumers respond, so the equilibrium price in each regime is a maximizer over R\mathbb{R}R of the corresponding profit: p^⋆\hat p^\starp^​⋆ for Π^\widehat\PiΠ and p~⋆\tilde p^\starp~​⋆ for Π~\widetilde\PiΠ. The paper's appendix expresses them through the thresholds

α1(x)=max⁡{1−2x2,0},α2(x)=max⁡{2−4x+x22x,0}\alpha_1(x) = \max\Big\{\tfrac{1-2x}{2}, 0\Big\}, \qquad \alpha_2(x) = \max\Big\{\tfrac{2-4x+x^2}{2x}, 0\Big\}α1​(x)=max{21−2x​,0},α2​(x)=max{2x2−4x+x2​,0}

and the radicals β=6+(x+α−1)2\beta = \sqrt{6 + (x+\alpha-1)^2}β=6+(x+α−1)2​, Γ=6+(2−2x+α)2\Gamma = \sqrt{6 + (2-2x+\alpha)^2}Γ=6+(2−2x+α)2​.

Formalization targets

Goal: Proposition A1 (p. 34)

For every 0<x<10 < x < 10<x<1 and α≥0\alpha \ge 0α≥0, both profits attain a global maximum on R\mathbb{R}R, and every pair of maximizers satisfies

p~⋆>p^⋆.\tilde p^\star > \hat p^\star.p~​⋆>p^​⋆.

Milestones

  1. Display (4) (p. 11): D^(p)=(1+x−p)2/2\widehat D(p) = (1+x-p)^2/2D(p)=(1+x−p)2/2 for x<p<1+xx < p < 1+xx<p<1+x, and 1−(1−x+p)2/21 - (1-x+p)^2/21−(1−x+p)2/2 for x−1≤p≤xx-1 \le p \le xx−1≤p≤x.
  2. Display (5) (p. 14): D~S(p)\widetilde D_S(p)DS​(p) equals (1+x−p)2/2(1+x-p)^2/2(1+x−p)2/2, x2/2+x(1−p)x^2/2 + x(1-p)x2/2+x(1−p) or x−p2/2x - p^2/2x−p2/2 on (1,1+x)(1, 1+x)(1,1+x), (x,1](x, 1](x,1], [0,x][0, x][0,x].
  3. Display (6) (p. 14): D~N(p)=(1−x)(1−p)\widetilde D_N(p) = (1-x)(1-p)DN​(p)=(1−x)(1−p) for 0≤p≤10 \le p \le 10≤p≤1, and 000 for 1<p<1+x1 < p < 1 + x1<p<1+x.
  4. Online Appendix A.1 (OA p. 2): p^\hat pp^​ maximizes Π^\widehat\PiΠ iff p^=(1+x−2α)/3\hat p = (1+x-2\alpha)/3p^​=(1+x−2α)/3 when α<α1(x)\alpha < \alpha_1(x)α<α1​(x) and p^=(2(x−1)−α+β)/3\hat p = (2(x-1) - \alpha + \beta)/3p^​=(2(x−1)−α+β)/3 otherwise.
  5. Online Appendix A.2 (OA p. 4): p~\tilde pp~​ maximizes Π~\widetilde\PiΠ iff p~=(2(1−αx)+x2)/4\tilde p = (2(1-\alpha x) + x^2)/4p~​=(2(1−αx)+x2)/4 when α<α2(x)\alpha < \alpha_2(x)α<α2​(x) and p~=(Γ−(2−2x+α))/3\tilde p = (\Gamma - (2 - 2x + \alpha))/3p~​=(Γ−(2−2x+α))/3 otherwise.
  6. p. 14: α1(x)≤α2(x)\alpha_1(x) \le \alpha_2(x)α1​(x)≤α2​(x) for all x∈(0,1)x \in (0,1)x∈(0,1).
  7. Lemma A3 (pp. 33–34): the three possible pairs of equilibrium regimes (case 1a/2a, 1b/2a, 1b/2b), located by price intervals.
  8. Online Appendix D, first display (OA p. 7): the closed form of p~⋆−p^⋆\tilde p^\star - \hat p^\starp~​⋆−p^​⋆ in each of the three regions.

Significance

The result. Proposition A1 says the price rise is robust: it holds for every opt-in benefit and every revenue model, from a firm that earns nothing from data (α=0\alpha = 0α=0) to one whose data revenue dominates. The paper's welfare results (on demand, profit and consumer surplus) are comparisons of the same two equilibria, and the price comparison is the one among them that holds uniformly; the others change sign with (x,α)(x, \alpha)(x,α) and are established in the paper only numerically. The main text's Proposition 1 is the instance x=1/4x = 1/4x=1/4.

Formalizing it. The paper's proof is a chain of closed-form computations spread over the appendix and online appendix: areas of polygons, the maximization of piecewise cubic profits over all of R\mathbb{R}R, and a sign argument with nested square roots. Several steps are stated as "one can verify", and the online appendix contains printed slips (a derivative written with respect to the wrong variable in §D, a pair of cases listed incorrectly on p. 14). A machine-checked proof turns the chain into a verified one. The milestones isolate the parts that are reusable on their own: the demand formulas (4)–(6) and the closed-form equilibria, on which the paper's companion profit comparison (Proposition A3) also rests. None of these results has been formalized before.

Difficulty

The obvious route, solving the first-order condition on each demand piece, gives candidates, not maximizers. The profits are only piecewise smooth, the pieces meet at p=xp = xp=x and p=1p = 1p=1, and the price ranges over all of R\mathbb{R}R, including negative prices before the GDPR. Showing that the candidate is the unique global maximizer requires comparing values across pieces and controlling the profit outside the paper's displayed price ranges. The comparison step is the second difficulty: in the middle region α1(x)≤α<α2(x)\alpha_1(x) \le \alpha < \alpha_2(x)α1​(x)≤α<α2​(x) the gap (14+4α+3x2−8x−6xα−4β)/12(14 + 4\alpha + 3x^2 - 8x - 6x\alpha - 4\beta)/12(14+4α+3x2−8x−6xα−4β)/12 mixes a polynomial with a square root, and its positivity is not a one-line estimate. The gap tends to 000 as x→1x \to 1x→1, so no uniform margin is available.

Formalization scope

  • Types are pairs q : ℝ × ℝ with q.1 = v, q.2 = c; the type square is Set.Icc 0 1 ×ˢ Set.Icc 0 1, and every demand is (volume S).toReal for the Lebesgue measure on ℝ × ℝ.
  • The after-GDPR sets are copied from p. 13; they send the tie US=UN≥0U_S = U_N \ge 0US​=UN​≥0 to opt-in, which changes only a null set.
  • Profits are functions ℝ → ℝ of the price, and an equilibrium price is IsMaxOn profit Set.univ p. The price is never restricted to the paper's case ranges, and p~⋆≥0\tilde p^\star \ge 0p~​⋆≥0 is a consequence, not an assumption.
  • Standing assumptions on every theorem: 0<x0 < x0<x, x<1x < 1x<1, α≥0\alpha \ge 0α≥0 (pp. 7–8). The data revenue is written α in Lean.
  • "Case 1a/1b/2a/2b equilibrium" means the maximizer lies in (x,1+x)(x, 1+x)(x,1+x), [x−1,x][x-1, x][x−1,x], (x,1+x)(x, 1+x)(x,1+x), [0,x][0, x][0,x] respectively.
  • The thresholds α1,α2\alpha_1, \alpha_2α1​,α2​ and radicals β,Γ\beta, \Gammaβ,Γ are plain definitions with the paper's formulas.

A formalization that compares the two closed-form prices instead of maximizers of the modelled profits, that asserts a property of "every maximizer" without proving one exists, or that restricts prices to a bounded interval, would not state Proposition A1; the goal and the milestones rule all three out. Results of the paper established only numerically (Propositions 2–4, A2, the second and third bullets of A3, A4) are not posed, and neither are the comparative statics of Lemmas A1–A2.

A complete development needs areas of polygons under the Lebesgue measure on R2\mathbb{R}^2R2 (via MeasureTheory.volume_regionBetween or Fubini), elementary calculus on piecewise polynomials, and inequalities with square roots. The demand computations and the global-maximization arguments are reusable for the companion mission on the paper's profit comparison. Contributions of any milestone, or of further intermediate lemmas such as the continuity of the demands in the price, are welcome.

Selected references

  • C. Choe, N. Matsushima, S. Shekhar, The Bright Side of the GDPR: Welfare-Improving Privacy Management, ISER Discussion Paper No. 1246, Osaka University, June 2024 (the version formalized; theorem and page numbers refer to it).
  • C. Choe, N. Matsushima, S. Shekhar, The Bright Side of the GDPR: Welfare-Improving Privacy Management, Management Science, 2025. https://doi.org/10.1287/mnsc.2024.06653
  • Regulation (EU) 2016/679 (General Data Protection Regulation), Official Journal of the European Union L 119, 2016. https://eur-lex.europa.eu/eli/reg/2016/679/oj
10 thms2 active usersReviewed
EconomicsMechanism DesignOperations Research+1·Captain: mikedeng1

Selling Data to Marketers 4: Under the Rotation Order, Data Appends with Costs Below the Prior Mean Serve High-Cost Marketers Efficiently and Under- (Over-) Supply Low-Cost Ones Below (Above) v⁺Research Paper

Motivation

Data brokers sell consumer data to firms that use it to decide whom to target, whom to lend to, or where to open a store. Two kinds of products are common: a marketing list, which supplies contact information without which the buyer cannot act at all, and a data append, which enriches information about consumers the buyer can already reach (Federal Trade Commission, Data Brokers, 2014). A seller who does not know a buyer's cost of acting must screen buyers with a menu of plans that differ in price and in how informative the data are.

Liang Guo's paper Selling Data to Marketers (Management Science, doi:10.1287/mnsc.2023.01350) characterizes the optimal menu, first for a binary-value model (§3) and then for a general information structure ordered by rotation (§4), following Johnson and Myatt (2006). This mission formalizes Proposition 7: data appends under the rotation order. It is the fourth mission of a series built from the same manuscript; the first three cover marketing lists and data appends in the binary model and marketing lists under the rotation order.

Setting

A marketer can take an action (target a consumer) at cost ccc; the action's value uuu is uncertain with prior mean u^\hat uu^. A data service of informativeness α∈[α‾,αˉ]⊂(0,1)\alpha\in[\underline\alpha,\bar\alpha]\subset(0,1)α∈[α​,αˉ]⊂(0,1) produces a signal, and its information is summarized by the posterior mean vvv. For each α\alphaα, vvv has cumulative distribution function H(v,α)H(v,\alpha)H(v,α) on a support [v‾,vˉ][\underline v,\bar v][v​,vˉ], with a strictly positive density h(v,α)h(v,\alpha)h(v,α) and mean u^\hat uu^ for every α\alphaα. The family is rotation ordered: there is a point v+∈(v‾,vˉ)v^+\in(\underline v,\bar v)v+∈(v​,vˉ) with

∂H(v,α)∂α>0 (v<v+),∂H(v,α)∂α<0 (v>v+).\frac{\partial H(v,\alpha)}{\partial\alpha}>0 \ (v<v^+),\qquad \frac{\partial H(v,\alpha)}{\partial\alpha}<0 \ (v>v^+).∂α∂H(v,α)​>0 (v<v+),∂α∂H(v,α)​<0 (v>v+).

A more informative service spreads the posterior mean away from u^\hat uu^.

A service plan (T,α)(T,\alpha)(T,α) charges a fixed fee T≥0T\ge0T≥0. Serving a marketer with informativeness α\alphaα costs the seller k(α)k(\alpha)k(α), with k(α‾)=0k(\underline\alpha)=0k(α​)=0 and kkk increasing. A marketer targets exactly when v>cv>cv>c, so a marketer with cost cic_ici​ who buys (Tj,αj)(T_j,\alpha_j)(Tj​,αj​) earns

Uij=−Tj+∫civˉ(v−ci) dH(v,αj).U_{ij}=-T_j+\int_{c_i}^{\bar v}(v-c_i)\,dH(v,\alpha_j).Uij​=−Tj​+∫ci​vˉ​(v−ci​)dH(v,αj​).

Costs are cHc_HcH​ with probability θ∈(0,1)\theta\in(0,1)θ∈(0,1) and cLc_LcL​ with probability 1−θ1-\theta1−θ, 0<cL<cH0<c_L<c_H0<cL​<cH​. A data-append buyer can act without the data, so its outside option is Ui0=max⁡{u^−ci,0}U_{i0}=\max\{\hat u-c_i,0\}Ui0​=max{u^−ci​,0}.

The seller offers a menu (TH,αH),(TL,αL)(T_H,\alpha_H),(T_L,\alpha_L)(TH​,αH​),(TL​,αL​) and maximizes Π=θ[TH−k(αH)]+(1−θ)[TL−k(αL)]\Pi=\theta[T_H-k(\alpha_H)]+(1-\theta)[T_L-k(\alpha_L)]Π=θ[TH​−k(αH​)]+(1−θ)[TL​−k(αL​)] subject to individual rationality Uii≥Ui0U_{ii}\ge U_{i0}Uii​≥Ui0​ and incentive compatibility Uii≥Uii′U_{ii}\ge U_{ii'}Uii​≥Uii′​ for both types. Call any maximizer an optimal menu, with levels αH∗,αL∗\alpha^*_H,\alpha^*_LαH∗​,αL∗​. The benchmark with observable costs is

max⁡α∈[α‾,αˉ] θi[∫civˉ(v−ci) dH(v,α)−k(α)],\max_{\alpha\in[\underline\alpha,\bar\alpha]}\ \theta_i\Big[\int_{c_i}^{\bar v}(v-c_i)\,dH(v,\alpha)-k(\alpha)\Big],α∈[α​,αˉ]max​ θi​[∫ci​vˉ​(v−ci​)dH(v,α)−k(α)],

and its maximizers are the efficient levels αie\alpha^e_iαie​.

Formalization targets

Goal: Proposition 7

Assume cL<cH<u^c_L<c_H<\hat ucL​<cH​<u^. If cL<cH<v+c_L<c_H<v^+cL​<cH​<v+, then

αH∗=αHe,αL∗<αLe,\alpha^*_H=\alpha^e_H,\qquad \alpha^*_L<\alpha^e_L ,αH∗​=αHe​,αL∗​<αLe​,

and if v+<cL<cHv^+<c_L<c_Hv+<cL​<cH​, then

αH∗=αHe,αL∗>αLe.\alpha^*_H=\alpha^e_H,\qquad \alpha^*_L>\alpha^e_L .αH∗​=αHe​,αL∗​>αLe​.

Here αH∗=αHe\alpha^*_H=\alpha^e_HαH∗​=αHe​ means that the set of αH\alpha_HαH​-levels of optimal menus equals the set of efficient levels, and each strict inequality holds for every optimal menu and every efficient level.

Milestones

  1. The willingness to pay for a data append (p. 20): for c≤u^c\le\hat uc≤u^, Uij−Ui0=−Tj+∫v‾ci(ci−v) dH(v,αj)U_{ij}-U_{i0}=-T_j+\int_{\underline v}^{c_i}(c_i-v)\,dH(v,\alpha_j)Uij​−Ui0​=−Tj​+∫v​ci​​(ci​−v)dH(v,αj​), which is increasing in cic_ici​.
  2. Its integrated form (proof of Proposition 8, p. 33): Uij−Ui0=−Tj+∫v‾ciH(v,αj) dvU_{ij}-U_{i0}=-T_j+\int_{\underline v}^{c_i}H(v,\alpha_j)\,dvUij​−Ui0​=−Tj​+∫v​ci​​H(v,αj​)dv.
  3. The reduction (15) (p. 20): substituting the binding IRLIR_LIRL​ and ICH,LIC_{H,L}ICH,L​ turns the screening problem into a maximization over (αH,αL)(\alpha_H,\alpha_L)(αH​,αL​).
  4. (15) integrated by parts, and αH∗=αHe\alpha^*_H=\alpha^e_HαH∗​=αHe​ for it (p. 33).
  5. The derivative ∂Π/∂αL\partial\Pi/\partial\alpha_L∂Π/∂αL​ of (15) compared with the benchmark's (p. 33).

Significance

Proposition 7 is the general-distribution form of a "double reversal". With marketing lists the low-cost marketers are the high type: they get efficient information, and the high-cost marketers' information is distorted (Proposition 5 of the paper). With data appends and costs below the prior mean, the outside option makes the high-cost marketers value data more. The roles of the types swap, and the direction of the distortion flips with the position of the costs relative to the rotation point. The result tells a data seller which customers to under- or over-supply, and shows that the answer depends on whether its data enable an action or only refine it.

The paper proves the result on a few pages, partly by phrases such as "it can be readily verified". No machine-checked version exists. A formalization makes explicit what the printed argument leaves implicit: that the dropped constraints hold at the reduced optimum, which regularity of kkk and HHH the strict inequalities need, and that statements about "the" optimum hold for every optimum when maximizers are not unique.

Difficulty

The obvious approach substitutes the binding constraints and compares first-order conditions. Two steps of it fail as stated. First, the reduction to (15) drops IRHIR_HIRH​ and ICL,HIC_{L,H}ICL,H​; ICL,HIC_{L,H}ICL,H​ holds at a maximizer of (15) only when the two informativeness levels are ordered as the rotation order dictates. That ordering is available only when both costs lie on the same side of v+v^+v+. Second, first-order conditions say nothing at a corner of [α‾,αˉ][\underline\alpha,\bar\alpha][α​,αˉ]. The strict comparison αL∗≠αLe\alpha^*_L\ne\alpha^e_LαL∗​=αLe​ needs the efficient level to be interior, where the derivative of kkk exists. That comes from the conditions of footnote 5 together with the mean-preserving property of the family. The selling cost need not be convex, so maximizers can be non-unique, and every comparison must hold for every pair of maximizers.

Formalization scope

  • Information structure. A structure RotationFamily with a bounded support v‾<vˉ\underline v<\bar vv​<vˉ (the paper allows an unbounded one). H(⋅,α)H(\cdot,\alpha)H(⋅,α) is continuous on [v‾,vˉ][\underline v,\bar v][v​,vˉ] with H(v‾,α)=0H(\underline v,\alpha)=0H(v​,α)=0, H(vˉ,α)=1H(\bar v,\alpha)=1H(vˉ,α)=1 and derivative h(⋅,α)>0h(\cdot,\alpha)>0h(⋅,α)>0, continuous on the closed support. ∂H/∂α\partial H/\partial\alpha∂H/∂α exists on the closed support. The rotation order (9) holds on the open support, and the mean is u^\hat uu^ for every α∈(0,1)\alpha\in(0,1)α∈(0,1). Condition (10) is not assumed; it follows.
  • Model. Model extends it with 0<α‾<αˉ<10<\underline\alpha<\bar\alpha<10<α​<αˉ<1, θ∈(0,1)\theta\in(0,1)θ∈(0,1), and 0<cL<cH0<c_L<c_H0<cL​<cH​ with both costs in (v‾,vˉ)(\underline v,\bar v)(v​,vˉ) (a pin: the integrals ∫civˉ\int_{c_i}^{\bar v}∫ci​vˉ​ presume it). It also fixes k(α‾)=0k(\underline\alpha)=0k(α​)=0 with kkk monotone on [α‾,αˉ][\underline\alpha,\bar\alpha][α​,αˉ]. dHdHdH is written as h dvh\,dvhdv.
  • Objects of the goal. Optimal menus are maximizers of the full problem: fees T≥0T\ge0T≥0, levels in [α‾,αˉ][\underline\alpha,\bar\alpha][α​,αˉ], IRiIR_iIRi​ with outside option max⁡{u^−ci,0}\max\{\hat u-c_i,0\}max{u^−ci​,0}, and both IC constraints. Efficient levels are maximizers of the benchmark (12). Neither is defined through (15) or a first-order condition. That would make the first half of the goal a restatement of milestone 4, and writing the outside option as u^−c\hat u-cu^−c instead of the maximum would change the model.
  • Pinned hypotheses. On the goal only: the part of footnote 5 the strict results use (kkk continuous on [α‾,αˉ][\underline\alpha,\bar\alpha][α​,αˉ], differentiable inside, k′(α‾)=0k'(\underline\alpha)=0k′(α​)=0 from the right, k′(α)→+∞k'(\alpha)\to+\inftyk′(α)→+∞ as α→αˉ−\alpha\to\bar\alpha^-α→αˉ−) and joint continuity of ∂H/∂α\partial H/\partial\alpha∂H/∂α on [v‾,vˉ]×(0,1)[\underline v,\bar v]\times(0,1)[v​,vˉ]×(0,1). The derivative milestone uses only the latter. Convexity of kkk is not assumed.
  • Range restriction. The first sentence of Proposition 7 (αH∗=αHe\alpha^*_H=\alpha^e_HαH∗​=αHe​) is posed on the full problem only within the two ranges cH<v+c_H<v^+cH​<v+ and v+<cLv^+<c_Lv+<cL​. For (15) it is posed without restriction in milestone 4. The branch v+<cL<cH<u^v^+<c_L<c_H<\hat uv+<cL​<cH​<u^ is empty when v+≥u^v^+\ge\hat uv+≥u^, for example for symmetric structures; non-symmetric rotation families with v+<u^v^+<\hat uv+<u^ inhabit it.
  • Reuse. Topkis's monotonicity theorem (Theorem 2.8.4) is on the platform and is the tool the paper invokes. The integration-by-parts facts for distribution functions on a bounded support, and differentiation of ∫cvˉH(v,α) dv\int_c^{\bar v}H(v,\alpha)\,dv∫cvˉ​H(v,α)dv in α\alphaα, are reusable beyond this mission. Proofs of the milestones in any order are welcome.

Selected references

  • Liang Guo, Selling Data to Marketers, Management Science, 2025 (author's manuscript used here). https://doi.org/10.1287/mnsc.2023.01350
  • Justin P. Johnson and David P. Myatt, On the Simple Economics of Advertising, Marketing, and Product Design, American Economic Review 96(3), 2006. https://doi.org/10.1257/000282806777211946
  • Donald M. Topkis, Supermodularity and Complementarity, Princeton University Press, 1998. https://doi.org/10.1515/9781400822539
  • Federal Trade Commission, Data Brokers: A Call for Transparency and Accountability, 2014. https://www.ftc.gov/reports/data-brokers-call-transparency-accountability-report-federal-trade-commission-may-2014
9 thms2 active usersReviewed
EconomicsOperations Research·Captain: mikedeng1

Asset Pricing with Costly Short Sales 2: With One Risky Asset the Equilibrium Shorting Cost and Lending Yield Are Explicit, Increasing and Convex in the Belief GapResearch Paper

Motivation

Short selling is never free. To sell a share short, an investor borrows it from a holder and pays that holder a lending fee, and the size of these fees has been documented empirically for decades (D'Avolio 2002; Duffie, Gârleanu and Pedersen 2002). Most equilibrium models of asset pricing either forbid short sales or take the fee as exogenous. Evgeniou, Hugonnier and Prieto build a continuous-time general-equilibrium economy where the fee is determined jointly with prices: investors disagree about expected dividend growth, the pessimist wants to short, the optimist owns the shares, and the cost of shorting clears the securities lending market. The paper uses the model to explain how a positive shorting cost can coexist with apparent mispricing, as in the 3Com/Palm spin-off.

In the special case of a single risky asset (the market portfolio), the equilibrium reduces to a pair of algebraic equations for the shorting cost and the lending yield at each state, and the paper solves them in closed form (Proposition 3). This mission formalizes that closed form and its comparative statics in the belief gap.

Setting

At a fixed state of the economy, three numbers describe the environment:

  • σ>0\sigma>0σ>0, the volatility of aggregate consumption, which in the one-asset case is also the volatility of the risky asset;
  • δ=σΔ=μ(o)−μ(p)≥0\delta=\sigma\Delta=\mu^{(o)}-\mu^{(p)}\ge 0δ=σΔ=μ(o)−μ(p)≥0, the unscaled difference in beliefs: the gap between the expected dividend growth rates perceived by the optimist ooo and the pessimist ppp (Δ\DeltaΔ is the same gap per unit of volatility);
  • s∈(0,1)s\in(0,1)s∈(0,1), the consumption share of the optimist.

Short selling is active only when the optimist holds a large enough share of consumption, namely on the shorting region s>s∗s>s^*s>s∗, where

s∗=min⁡{1,σΔ}=min⁡{1,σ2δ}.s^*=\min\Big\{1,\frac{\sigma}{\Delta}\Big\}=\min\Big\{1,\frac{\sigma^2}{\delta}\Big\}.s∗=min{1,Δσ​}=min{1,δσ2​}.

Two unknowns are determined in equilibrium: the shorting cost Φ\PhiΦ, the flow rate a short seller pays per unit of position, and the lending yield Γ\GammaΓ, the flow rate a lender receives per unit of shares held. The paper shows (p. 27) that with one risky asset they satisfy

Φ=1{s>s∗}(δ+Γ)s−σ21+s,Γ=1{s>s∗}(1−s)(sδ−Γ−σ2)((δ+Γ)s−σ2)σ2(1+s)2.(29–30)\Phi=\mathbb 1_{\{s>s^*\}}\frac{(\delta+\Gamma)s-\sigma^2}{1+s},\qquad \Gamma=\mathbb 1_{\{s>s^*\}}\frac{(1-s)(s\delta-\Gamma-\sigma^2)\big((\delta+\Gamma)s-\sigma^2\big)}{\sigma^2(1+s)^2}. \tag{29–30}Φ=1{s>s∗}​1+s(δ+Γ)s−σ2​,Γ=1{s>s∗}​σ2(1+s)2(1−s)(sδ−Γ−σ2)((δ+Γ)s−σ2)​.(29–30)

The proof of Proposition 3 works with the per-volatility yield γ=Γ/σ\gamma=\Gamma/\sigmaγ=Γ/σ and the quadratic

g(γ)=(1−s)(sΔ−γ−σ)((γ+Δ)s−σ)−γσ(1+s)2.g(\gamma)=(1-s)(s\Delta-\gamma-\sigma)\big((\gamma+\Delta)s-\sigma\big)-\gamma\sigma(1+s)^2 .g(γ)=(1−s)(sΔ−γ−σ)((γ+Δ)s−σ)−γσ(1+s)2.

In Lean these objects are costOne σ δ s, yieldOne σ δ s, IsOneAssetSystem σ δ s Φ Γ and gQuad σ Δ s γ, all in the namespace CostlyShort.OneAsset.

Formalization targets

Goal: Proposition 3

For σ>0\sigma>0σ>0 and s∈(0,1)s\in(0,1)s∈(0,1): for every δ≥0\delta\ge 0δ≥0 and every Γ≥0\Gamma\ge 0Γ≥0, the pair (Φ,Γ)(\Phi,\Gamma)(Φ,Γ) solves (29)–(30) if and only if

Φ=1{s>s∗}s(1−s)δ−2σ2+s2(1−s)2δ2+4σ4s2(1−s),(31)\Phi=\mathbb 1_{\{s>s^*\}}\frac{s(1-s)\delta-2\sigma^2+\sqrt{s^2(1-s)^2\delta^2+4\sigma^4 s}}{2(1-s)}, \tag{31}Φ=1{s>s∗}​2(1−s)s(1−s)δ−2σ2+s2(1−s)2δ2+4σ4s​​,(31) Γ=1{s>s∗}−s((1−s)2δ+4σ2)+(1+s)s2(1−s)2δ2+4σ4s2s(1−s),(32)\Gamma=\mathbb 1_{\{s>s^*\}}\frac{-s\big((1-s)^2\delta+4\sigma^2\big)+(1+s)\sqrt{s^2(1-s)^2\delta^2+4\sigma^4 s}}{2s(1-s)}, \tag{32}Γ=1{s>s∗}​2s(1−s)−s((1−s)2δ+4σ2)+(1+s)s2(1−s)2δ2+4σ4s​​,(32)

and both (31) and (32), as functions of δ\deltaδ, are non-decreasing and convex on [0,∞)[0,\infty)[0,∞) and strictly increasing for δ>σ2/s\delta>\sigma^2/sδ>σ2/s.

Milestones

  1. On the shorting region, (30) is equivalent to g(γ)=0g(\gamma)=0g(γ)=0 (proof of Proposition 3, p. 50).
  2. g(0)=(1−s)(sΔ−σ)2>0g(0)=(1-s)(s\Delta-\sigma)^2>0g(0)=(1−s)(sΔ−σ)2>0, g′(0)<0g'(0)<0g′(0)<0, g′′≡−2s(1−s)<0g''\equiv-2s(1-s)<0g′′≡−2s(1−s)<0 and g(γ)→−∞g(\gamma)\to-\inftyg(γ)→−∞ (p. 50).
  3. For s>s∗s>s^*s>s∗, ggg has a unique strictly positive root (p. 50).
  4. The closed form (32) is strictly positive and solves (30) on the region (p. 50).
  5. Substituting (32) into (29) gives (31) (p. 50).
  6. The market-clearing identity (33) with the utilization (34): Γ=ΦΥ/(1+Υ)\Gamma=\Phi\Upsilon/(1+\Upsilon)Γ=ΦΥ/(1+Υ), Υ=1−sσ2Φ\Upsilon=\frac{1-s}{\sigma^2}\PhiΥ=σ21−s​Φ (p. 28).
  7. Φ\PhiΦ is non-decreasing and convex in δ\deltaδ on [0,∞)[0,\infty)[0,∞), strictly increasing on the region (Proposition 3, p. 28).

A companion statement, not a milestone, formalizes the limits on p. 29: as s↑1s\uparrow 1s↑1, Φ→(1−s∗)δ/2\Phi\to(1-s^*)\delta/2Φ→(1−s∗)δ/2 and Γ→0\Gamma\to 0Γ→0.

Significance

Proposition 3 is the only fully explicit equilibrium of the paper. It gives the level of the shorting cost and of the lending yield as functions of disagreement and of the wealth distribution, and its monotonicity in δ\deltaδ is the model's prediction that short interest and lending fees rise with the dispersion of beliefs, which the paper checks against analysts' forecast dispersion (Table 1). Convexity says the effect accelerates with disagreement. The two-asset case of §4.2 has no closed form and is solved numerically; the one-asset formulas are its benchmark.

The result is proved in the paper, but the proof of the comparative statics is omitted ("We omit the details", p. 50), and the selection of the positive root is asserted ("it is clear"). A formal development supplies the omitted differentiation argument, including convexity across the kink at the boundary δ=σ2/s\delta=\sigma^2/sδ=σ2/s of the shorting region, and checks the paper's algebra. No part of this paper has a machine-checked proof.

Difficulty

The algebra of milestones 1, 4 and 5 is mechanical but involves a square root and several divisions, whose signs must be tracked throughout. The substantive step is the comparative statics. Monotonicity and convexity of Φ\PhiΦ in δ\deltaδ need the derivative of a square-root expression on the open region, and then a separate argument at the kink: both functions vanish identically for δ≤σ2/s\delta\le\sigma^2/sδ≤σ2/s, so convexity on [0,∞)[0,\infty)[0,∞) requires the one-sided derivative at σ2/s\sigma^2/sσ2/s to be non-negative, which is a computation rather than a consequence of convexity on each piece. For Γ\GammaΓ the paper's route is indirect: Γ=h(Φ)\Gamma=h(\Phi)Γ=h(Φ) with h(x)=cx2/(1+cx)h(x)=cx^2/(1+cx)h(x)=cx2/(1+cx), and composition of a non-decreasing convex hhh with a non-decreasing convex Φ\PhiΦ. Differentiating (32) directly is possible but heavier.

Formalization scope

Everything is pointwise in real numbers. The continuous-time economy of §2 (Brownian dividends, portfolio choice, the derivation of (29)–(30) from the equilibrium conditions (16), (20), (21)) is the paper's motivation and is not formalized; Proposition 3 is stated relative to (29)–(30), which is exactly where the paper's proof begins. Conventions committed to:

  • standing assumptions σ>0\sigma>0σ>0 and 0<s<10<s<10<s<1 in every theorem; the endpoints s∈{0,1}s\in\{0,1\}s∈{0,1} are absorbing states where (31)–(32) divide by zero, and are excluded;
  • the indicator 1{s>s∗}\mathbb 1_{\{s>s^*\}}1{s>s∗}​ is encoded as σ2<sδ\sigma^2<s\deltaσ2<sδ, equivalent for σ>0\sigma>0σ>0, δ≥0\delta\ge 0δ≥0, s∈(0,1)s\in(0,1)s∈(0,1); it is never written as s>min⁡{1,σ/Δ}s>\min\{1,\sigma/\Delta\}s>min{1,σ/Δ}, where Lean's σ/0=0\sigma/0=0σ/0=0 would make every state a shorting state when Δ=0\Delta=0Δ=0;
  • in the goal's equivalence the lending yield is restricted to Γ≥0\Gamma\ge 0Γ≥0. This is a disclosed pin: on the shorting region the quadratic also has a negative root, which solves (29)–(30) as well; the paper's proof selects "the unique strictly positive solution", and a lending yield is a non-negative flow rate;
  • "increasing" is stated as non-decreasing on [0,∞)[0,\infty)[0,∞) plus strictly increasing on (σ2/s,∞)(\sigma^2/s,\infty)(σ2/s,∞), because both functions are zero below the region;
  • the paper prints g′′=−s(1−s)g''=-s(1-s)g′′=−s(1−s); the correct second derivative −2s(1−s)-2s(1-s)−2s(1−s) is stated (milestone 2);
  • only the outer equality of (34) is used; its middle expression involves the pessimist's portfolio and needs the economy.

A trivializing formalization is ruled out: stating only that (31)–(32) satisfy (29)–(30), or convexity only on the open shorting region, would be strictly weaker than the paper and is not the goal.

Needed infrastructure is Mathlib's real analysis: Real.sqrt, deriv, MonotoneOn, ConvexOn, and convexity of compositions. Proofs of any milestone are welcome independently. Facts about convexity of a function that vanishes up to a point and is convex beyond it are reusable outside this mission.

Selected references

  • T. Evgeniou, J. Hugonnier and R. Prieto, Asset Pricing with Costly Short Sales, Swiss Finance Institute Research Paper N°22-21, version of September 22, 2022 (the source of this mission; §3.3, §4.1, App. A). Published as J. Hugonnier and R. Prieto, Management Science, 2024. https://doi.org/10.1287/mnsc.2023.01887
  • G. D'Avolio, The market for borrowing stock, Journal of Financial Economics 66 (2002) 271–306. https://doi.org/10.1016/S0304-405X(02)00206-4
  • D. Duffie, N. Gârleanu and L. H. Pedersen, Securities lending, shorting, and pricing, Journal of Financial Economics 66 (2002) 307–339. https://doi.org/10.1016/S0304-405X(02)00226-X
  • J. Detemple and S. Murthy, Equilibrium asset prices and no-arbitrage with portfolio constraints, Review of Financial Studies 10 (1997) 1133–1174. https://doi.org/10.1093/rfs/10.4.1133
9 thms2 active usersReviewed
Quantum Information·Captain: Goku

Quantum algorithms for optimizers 9: QAOA with enough layers reaches a maximum eigenstate of H_FTextbook

Motivation

This mission develops Chapter 9 of Quantum algorithms for optimizers by Giacomo Nannicini. The chapter's selected results describe the mathematical guarantees below.

Setting and conventions

The linked theorem statements and shared definitions specify the register types, oracle access, normalization, and resource conventions. Their source locators and independent readbacks accompany each target.

Chapter goal

Let n≥0n \ge 0n≥0 and let f:{0,1}n→Rf : \{0,1\}^n \to \mathbb{R}f:{0,1}n→R be the objective of the maximization problem (9.38). Define the nnn-qubit Hamiltonians

HF=∑x∈{0,1}nf(x) ∣x⟩⟨x∣,HI=∑j=1nσjX,H_F = \sum_{x \in \{0,1\}^n} f(x)\,|x\rangle\langle x|, \qquad H_I = \sum_{j=1}^n \sigma_j^X,HF​=x∈{0,1}n∑​f(x)∣x⟩⟨x∣,HI​=j=1∑n​σjX​,

where σjX\sigma_j^XσjX​ is the Pauli XXX on qubit jjj and the identity elsewhere. For p≥1p \ge 1p≥1 and β,θ∈[0,2π]p\beta, \theta \in [0, 2\pi]^pβ,θ∈[0,2π]p let

UQAOA(p,β,θ)=e−iβpHIe−iθpHF⋯e−iβ1HIe−iθ1HF.U_{\mathrm{QAOA}}(p, \beta, \theta) = e^{-i\beta_p H_I} e^{-i\theta_p H_F} \cdots e^{-i\beta_1 H_I} e^{-i\theta_1 H_F}.UQAOA​(p,β,θ)=e−iβp​HI​e−iθp​HF​⋯e−iβ1​HI​e−iθ1​HF​.

Then for every δ>0\delta > 0δ>0 there are a unit vector ∣φ⟩|\varphi\rangle∣φ⟩ with HF∣φ⟩=(max⁡xf(x)) ∣φ⟩H_F|\varphi\rangle = (\max_x f(x))\,|\varphi\rangleHF​∣φ⟩=(maxx​f(x))∣φ⟩, that is an eigenstate of HFH_FHF​ with maximum eigenvalue, and an integer P≥1P \ge 1P≥1 such that for every p≥Pp \ge Pp≥P there are angles β∗,θ∗∈[0,2π]p\beta^*, \theta^* \in [0, 2\pi]^pβ∗,θ∗∈[0,2π]p with

∥UQAOA(p,β∗,θ∗) (H∣0⟩)⊗n−∣φ⟩∥≤δ.\bigl\| U_{\mathrm{QAOA}}(p, \beta^*, \theta^*)\,(H|0\rangle)^{\otimes n} - |\varphi\rangle \bigr\| \le \delta .​UQAOA​(p,β∗,θ∗)(H∣0⟩)⊗n−∣φ⟩​≤δ.

Here (H∣0⟩)⊗n(H|0\rangle)^{\otimes n}(H∣0⟩)⊗n is the uniform superposition, the maximum eigenstate of HIH_IHI​. The proposition says that with enough layers, the QAOA ansatz can reproduce adiabatic evolution along H(s)=(1−s)HI+sHFH(s) = (1-s)H_I + sH_FH(s)=(1−s)HI​+sHF​ and so prepares a state close to one encoding an optimal solution of (9.38).

Formalization Note The source's limit over ppp, with angles that depend on ppp, is stated as "for all sufficiently large ppp some angles achieve distance at most δ\deltaδ", with the target eigenvector fixed before ppp. The global phase allowed by Theorem 9.11 is absorbed into the choice of the eigenvector, since a phase multiple of a maximum eigenvector is again one. The informal claim that UQAOAU_{\mathrm{QAOA}}UQAOA​ "approximates an adiabatic evolution" is represented by its stated "in particular" consequence.

Significance

These are known mathematical results. Their Lean statements have been checked and reviewed against the cited source; formal proofs remain open for contributors.

Supporting targets

  • Proposition 9.4: The Ising Hamiltonian is diagonal with the QUBO objective values.
  • Theorem 9.11: Adiabatic theorem with T ≥ (10⁴/δ²)(‖H′‖³/γ⁴ + ‖H′‖‖H″‖/γ³).
  • Theorem 9.17: Adiabatic theorem when the tracked eigenvalue is identically 0.
  • Lemma 9.18: Derivative bounds for the phase-fixed instantaneous zero eigenvector.
  • Lemma 9.20: Replacing U_k ⋯ U_j by powers of U_1 in the discretized adiabatic evolution.
  • Corollary 9.21: The T-bound of Theorem 9.11 suffices (derived from Theorem 9.17).
  • Theorem 9.23: Adiabatic theorem with the integrated 1/γ³ gap dependence.
  • Corollary 9.24: Adiabatic theorem for the linear interpolation (1 − s)H_I + sH_F.
  • Proposition 9.28: QAOA with enough layers reaches a maximum eigenstate of H_F.

Formalization scope

Prove the existing statements under their stated hypotheses and definitions. Any correction to a printed source claim remains explicit in its target description. All listed target proofs are currently sorry; no proof is claimed. The mission goal is the designated chapter capstone; the remaining targets are separate milestones. Parts a, b, c and d are merged into their original chapter.

Selected references

  • Giacomo Nannicini, Quantum algorithms for optimizers, 2025, v5; notes dated 9 July 2025. https://arxiv.org/pdf/2408.07086v5.
9 thms2 active usersReviewed
Quantum Information·Captain: Goku

Quantum algorithms for optimizers 8: End-to-end complexity of the quantum MaxCut SDP solver in the QRAM model (corrected statement)Textbook

Motivation

This mission develops Chapter 8 of Quantum algorithms for optimizers by Giacomo Nannicini. The chapter's selected results describe the mathematical guarantees below.

Setting and conventions

The linked theorem statements and shared definitions specify the register types, oracle access, normalization, and resource conventions. Their source locators and independent readbacks accompany each target.

Chapter goal

This is Proposition 8.34 of Nannicini, the end-to-end cost of the quantum mirror-descent solver for the MaxCut SDP relaxation, stated in an explicit hybrid classical–quantum QRAM cost model.

Let n=2qn=2^qn=2q. For a nonzero real symmetric C∈Rn×nC\in\mathbb R^{n\times n}C∈Rn×n put C^=C/∥C∥F\hat C=C/\|C\|_FC^=C/∥C∥F​. Call a density matrix ρ∈Cn×n\rho\in\mathbb C^{n\times n}ρ∈Cn×n an ϵ\epsilonϵ-solution of (MaxCutSDP) if

∑j=1n∣ρjj−1n∣≤ϵandTr⁡(C^ρ) ≥ Tr⁡(C^σ)−ϵ  for every density matrix σ with diag⁡(σ)=1n1.\sum_{j=1}^n\Big|\rho_{jj}-\frac1n\Big|\le\epsilon\qquad\text{and}\qquad \operatorname{Tr}(\hat C\rho)\ \ge\ \operatorname{Tr}(\hat C\sigma)-\epsilon\ \text{ for every density matrix }\sigma\text{ with }\operatorname{diag}(\sigma)=\tfrac1n\mathbb 1.j=1∑n​​ρjj​−n1​​≤ϵandTr(C^ρ) ≥ Tr(C^σ)−ϵ  for every density matrix σ with diag(σ)=n1​1.

A hybrid algorithm consists of (i) a classical preprocessing program, a straight-line program of real arithmetic operations (+,−,×,÷,⋅,arccos⁡,arg⁡+,-,\times,\div,\sqrt\cdot,\arccos,\arg+,−,×,÷,⋅​,arccos,arg, rational constants), which is the only part that reads the entries of CCC and which writes the input half of a QRAM (UQRAM∣j⟩∣y⟩=∣j⟩∣y⊕Mj⟩U_{\mathrm{QRAM}}|j\rangle|y\rangle=|j\rangle|y\oplus M_j\rangleUQRAM​∣j⟩∣y⟩=∣j⟩∣y⊕Mj​⟩, Def. 5.39); (ii) RRR quantum runs, each applying a circuit of two-qubit gates and QRAM calls to ∣0⋯0⟩|0\cdots0\rangle∣0⋯0⟩ and measuring all qubits, where the circuit of each run and the contents of the work half of the QRAM are chosen classically from the outcomes of the earlier runs; (iii) an output, computed from all outcomes, in the form of a list of pairs (y(i),d(i))∈R2×Rn(y^{(i)},d^{(i)})\in\mathbb R^2\times\mathbb R^n(y(i),d(i))∈R2×Rn, which stands for the matrix

ρˉ=1L∑i=1Lexp⁡(y1(i)C^+y2(i)diag⁡(d(i)))Tr⁡exp⁡(y1(i)C^+y2(i)diag⁡(d(i))).\bar\rho=\frac1{L}\sum_{i=1}^{L}\frac{\exp\big(y^{(i)}_1\hat C+y^{(i)}_2\operatorname{diag}(d^{(i)})\big)}{\operatorname{Tr}\exp\big(y^{(i)}_1\hat C+y^{(i)}_2\operatorname{diag}(d^{(i)})\big)}.ρˉ​=L1​i=1∑L​Trexp(y1(i)​C^+y2(i)​diag(d(i)))exp(y1(i)​C^+y2(i)​diag(d(i)))​.

The theorem: there are absolute constants K>0K>0K>0 and ccc such that for all qqq, all ϵ∈(0,1]\epsilon\in(0,1]ϵ∈(0,1] and all δ∈(0,1)\delta\in(0,1)δ∈(0,1), with Λ=log⁡(2+n)+log⁡(2+1/ϵ)+log⁡(2+1/δ)\Lambda=\log(2+n)+\log(2+1/\epsilon)+\log(2+1/\delta)Λ=log(2+n)+log(2+1/ϵ)+log(2+1/δ), there is a hybrid algorithm, fixed before CCC, such that

  1. the QRAM has size at most Kn2ΛcK n^2\Lambda^cKn2Λc, with words of at most KΛcK\Lambda^cKΛc bits;
  2. the preprocessing uses at most Kn2ΛcK n^2\Lambda^cKn2Λc classical operations;
  3. along every measurement record the total number of QRAM accesses and additional gates is at most K n1.5ϵ5ΛcK\,\dfrac{n^{1.5}}{\epsilon^5}\Lambda^cKϵ5n1.5​Λc;
  4. for every nonzero real symmetric CCC, with probability at least 1−δ1-\delta1−δ the output ρˉ\bar\rhoρˉ​ is an ϵ\epsilonϵ-solution of (MaxCutSDP).

The proposition is the payoff of the chapter's MaxCut analysis: the subgradient estimation of Prop. 8.28, the iteration count of Thm. 8.30 and the Hamiltonian structure of Lem. 8.33 combine into a quantum solver whose quantum cost is sublinear in the n2n^2n2 entries of CCC.

Formalization Note The printed proposition has no cost model, no success probability, and no precise meaning for the tolerance over the binary search on γ\gammaγ; these are fixed as above. The classical bookkeeping between quantum runs (tallying samples, forming the subgradients, updating yyy and DDD) is not counted, matching the proof, which takes the classical cost from Prop. 7.29 alone; that bookkeeping never sees CCC. The output is the source's own representation of the iterate (Rem. 8.27, Lem. 8.33): with an arbitrary decoder, the optimum could be read off C^\hat CC^ for free.

Significance

These are known mathematical results. Their Lean statements have been checked and reviewed against the cited source; formal proofs remain open for contributors.

Supporting targets

  • Proposition 8.7: Entropic mirror descent iterates are the Gibbs states of the accumulated subgradients.
  • Theorem 8.10: Regret bound of the matrix multiplicative weights update.
  • Lemma 8.12: Emptiness of the PIC polytope certifies primal feasibility after rescaling.
  • Theorem 8.15: Algorithm 7 (MMWU for SDP) returns a primal or a dual certificate.
  • Proposition 8.19: The generalized PIC polytope built from θ-accurate traces is sandwiched between P_0 and P_{4Rrθ}.
  • Proposition 8.21: Computing the dual vector with Õ(√m) calls to a trace-estimate oracle (corrected statement).
  • Proposition 8.28: The estimated MaxCut subgradient is an ϵ/2-subgradient (corrected zero branch).
  • Lemma 8.29: A subgradient of the active piece at a nearby point is an approximate subgradient of the max.
  • Theorem 8.30: Convergence of inexact entropic mirror descent for the MaxCut SDP feasibility problem.
  • Lemma 8.33: The MaxCut mirror-descent Hamiltonians are y₁Ĉ + y₂D with ‖y‖₁ ≤ (4/ϵ) log n.
  • Proposition 8.34: End-to-end complexity of the quantum MaxCut SDP solver in the QRAM model (corrected statement).

Formalization scope

Prove the existing statements under their stated hypotheses and definitions. Any correction to a printed source claim remains explicit in its target description. All listed target proofs are currently sorry; no proof is claimed. The mission goal is the designated chapter capstone; the remaining targets are separate milestones. Parts a, b, c and d are merged into their original chapter.

Selected references

  • Giacomo Nannicini, Quantum algorithms for optimizers, 2025, v5; notes dated 9 July 2025. https://arxiv.org/pdf/2408.07086v5.
11 thms2 active usersReviewed
Quantum Information·Captain: Goku

Quantum algorithms for optimizers 7: A (‖A‖_F, Õ(q), ξ)-block-encoding of a classical matrix from QRAM, with an explicit cost modelTextbook

Motivation

This mission develops Chapter 7 of Quantum algorithms for optimizers by Giacomo Nannicini. The chapter's selected results describe the mathematical guarantees below.

Setting and conventions

The linked theorem statements and shared definitions specify the register types, oracle access, normalization, and resource conventions. Their source locators and independent readbacks accompany each target.

Chapter goal

Fix constants C0∈RC_0 \in \mathbb{R}C0​∈R and c0∈Nc_0 \in \mathbb{N}c0​∈N (those of the assumption p=O~(1)p = \widetilde{O}(1)p=O(1)). There are constants C>0C > 0C>0 and c∈Nc \in \mathbb{N}c∈N such that the following holds. Let q,p∈Nq, p \in \mathbb{N}q,p∈N, d=2qd = 2^qd=2q, ξ>0\xi > 0ξ>0, write L=log⁡(2+d)+log⁡(2+1/ξ)L = \log(2+d) + \log(2+1/\xi)L=log(2+d)+log(2+1/ξ), and assume p≤C0Lc0p \le C_0 L^{c_0}p≤C0​Lc0​. Then there exist

  1. a QRAM (Def. 5.39) with 2b2^b2b addresses and words of rrr bits, of size 2br≤Cd2(p+1)Lc2^b r \le C d^2 (p+1) L^c2br≤Cd2(p+1)Lc;
  2. a straight-line real-arithmetic program of at most Cd2LcC d^2 L^cCd2Lc operations (+,−,×,÷,⋅,arccos⁡+, -, \times, \div, \sqrt{\cdot}, \arccos+,−,×,÷,⋅​,arccos, complex argument, rational constants) and an addressing map, which compute the QRAM contents from the entries of the matrix;
  3. a quantum circuit V\mathcal{V}V on w+qw + qw+q qubits with w≤C(q+1)Lcw \le C (q+1) L^cw≤C(q+1)Lc, making at most CLcC L^cCLc uncontrolled calls to the QRAM (or its inverse) and using at most CLcC L^cCLc two-qubit gates,

all independent of the matrix, such that for every A∈Cd×dA \in \mathbb{C}^{d\times d}A∈Cd×d whose entries satisfy ∣Re⁡aij∣,∣Im⁡aij∣≤2p|\operatorname{Re} a_{ij}|, |\operatorname{Im} a_{ij}| \le 2^p∣Reaij​∣,∣Imaij​∣≤2p, the unitary V(UQRAM(A))\mathcal{V}(U_{\mathrm{QRAM}}(A))V(UQRAM​(A)) obtained with the QRAM loaded by the program is an (∥A∥F,w,ξ)(\|A\|_F, w, \xi)(∥A∥F​,w,ξ)-block-encoding of AAA:

∥∥A∥F (⟨0∣w⊗I) V (∣0⟩w⊗I)−A∥≤ξ.\bigl\| \|A\|_F\,(\langle 0|_w \otimes I)\,\mathcal{V}\,(|0\rangle_w \otimes I) - A \bigr\| \le \xi .​∥A∥F​(⟨0∣w​⊗I)V(∣0⟩w​⊗I)−A​≤ξ.

This is the data-structure route to block-encodings of classically stored dense matrices, with subnormalization the Frobenius norm and only polylogarithmic quantum cost.

Formalization Note Corrected statement. The source gives no cost model for classical arithmetic, does not say in which parameters ppp is polylogarithmic, and does not say how the costs depend on ξ\xiξ. Here classical cost is the length of a straight-line real-arithmetic program (the model of the Cor. 5.45 chunk), p=O~(1)p = \widetilde{O}(1)p=O(1) means p≤C0Lc0p \le C_0 L^{c_0}p≤C0​Lc0​ for arbitrary C0,c0C_0, c_0C0​,c0​, all O~\widetilde{O}O bounds are polylogarithmic in ddd and 1/ξ1/\xi1/ξ, and the QRAM size is O~(d2p)\widetilde{O}(d^2 p)O(d2p) rather than O(d2p)O(d^2 p)O(d2p) because the word size must grow with log⁡(1/ξ)\log(1/\xi)log(1/ξ). A ppp-bit entry is modelled by the magnitude bound 2p2^p2p on its real and imaginary parts.

Significance

These are known mathematical results. Their Lean statements have been checked and reviewed against the cited source; formal proofs remain open for contributors.

Supporting targets

  • Proposition 7.9: Iterative refinement reaches residual ϵ in O(log 1/ϵ) iterations.
  • Proposition 7.14: Product of block-encodings (error term corrected).
  • Proposition 7.18: Linear combination of block-encodings (error term corrected for α < 1).
  • Lemma 7.20: Block-encoding of the diagonal matrix of inner products ⟨ϕ_j|ψ_j⟩.
  • Proposition 7.21: Block-encoding from sparse-access oracles (rotation cost and entry encoding made explicit).
  • Lemma 7.37: A bounded polynomial of degree Õ(β) approximating exp(2βx)/4 on [−1, 0].
  • Lemma 7.39: Block-encoding a density matrix from a unitary preparing its purification.
  • Proposition 7.41: One-copy trace estimator: a random variable with bias ≤ θ/4 for Tr(Aρ) and constant variance.
  • Proposition 7.38: Gibbs-state preparation from a block-encoding of H, for the polynomial of Lemma 7.37 made explicit.
  • Proposition 7.29: A (‖A‖_F, Õ(q), ξ)-block-encoding of a classical matrix from QRAM, with an explicit cost model.

Formalization scope

Prove the existing statements under their stated hypotheses and definitions. Any correction to a printed source claim remains explicit in its target description. All listed target proofs are currently sorry; no proof is claimed. The mission goal is the designated chapter capstone; the remaining targets are separate milestones. Parts a, b, c and d are merged into their original chapter.

Selected references

  • Giacomo Nannicini, Quantum algorithms for optimizers, 2025, v5; notes dated 9 July 2025. https://arxiv.org/pdf/2408.07086v5.
10 thms2 active usersReviewed
Quantum Information·Captain: Goku

Quantum algorithms for optimizers 4: Quantum minimum finding (Dürr–Høyer), with explicit bit width, interruption and gate modelTextbook

Motivation

This mission develops Chapter 4 of Quantum algorithms for optimizers by Giacomo Nannicini. The chapter's selected results describe the mathematical guarantees below.

Setting and conventions

The linked theorem statements and shared definitions specify the register types, oracle access, normalization, and resource conventions. Their source locators and independent readbacks accompany each target.

Chapter goal

Let f:{0,1}n→Zf : \{0,1\}^n \to \mathbb{Z}f:{0,1}n→Z take values of bit width bbb, i.e. ∣f(j)∣<2b|f(j)| < 2^b∣f(j)∣<2b for all jjj, and let Uf:∣j⟩∣k⟩↦∣j⟩∣k⊕f(j)⟩U_f : |j\rangle|k\rangle \mapsto |j\rangle|k \oplus f(j)\rangleUf​:∣j⟩∣k⟩↦∣j⟩∣k⊕f(j)⟩ evaluate fff in binary. Algorithm 2 with budget TTT starts from a uniformly random incumbent ℓ\ellℓ and repeatedly runs Algorithm 1 (search with an unknown number of marked items) with the marking unitary Um:∣j⟩∣y⟩↦∣j⟩∣y⊕I(f(j)<f(ℓ))⟩U_m : |j\rangle|y\rangle \mapsto |j\rangle|y \oplus \mathbb{I}(f(j) < f(\ell))\rangleUm​:∣j⟩∣y⟩↦∣j⟩∣y⊕I(f(j)<f(ℓ))⟩, replacing ℓ\ellℓ by the returned index kkk whenever f(k)<f(ℓ)f(k) < f(\ell)f(k)<f(ℓ); a call of Algorithm 1 that would exceed the budget is interrupted at the TTT-th application of UfU_fUf​, and the algorithm then returns ℓ\ellℓ. Each application of UmU_mUm​ counts as one application of UfU_fUf​ and is charged 3n+2b+13n + 2b + 13n+2b+1 additional gates; each round of Algorithm 1 with mmm qubits of precision is charged m+m(m+1)/2m + m(m+1)/2m+m(m+1)/2 further gates (Hadamards and the inverse QFT).

There are absolute constants c,C>0c, C > 0c,C>0 such that for every nnn, bbb, every such fff and every δ∈(0,1)\delta \in (0,1)δ∈(0,1) the following holds. Run Algorithm 2 independently k=⌈log⁡3(1/δ)⌉k = \lceil \log_3(1/\delta) \rceilk=⌈log3​(1/δ)⌉ times with budget T=⌈c2n⌉T = \lceil c\sqrt{2^n} \rceilT=⌈c2n​⌉ and output an fff-minimal one of the kkk returned indices. Then:

  1. with probability at least 1−δ1 - \delta1−δ the output is a global minimizer of fff;
  2. on every outcome, the total number of applications of UfU_fUf​ (including the kkk evaluations used to select the output) is at most
C2n (1+log⁡(1/δ)),C\sqrt{2^n}\,\bigl(1 + \log(1/\delta)\bigr),C2n​(1+log(1/δ)),

and the number of additional gates is at most C2n (n+b+1) (1+log⁡(1/δ))C\sqrt{2^n}\,(n + b + 1)\,(1 + \log(1/\delta))C2n​(n+b+1)(1+log(1/δ)).

This is the quadratic quantum speedup for unstructured minimization, the basis of the optimization applications of Grover search.

Formalization Note The source's O~(2n)\tilde O(\sqrt{2^n})O~(2n​) gate bound is stated with the suppressed polylogarithmic factors n+b+1n + b + 1n+b+1 and 1+log⁡(1/δ)1 + \log(1/\delta)1+log(1/δ) explicit, the interruption rule for the budget is made explicit, and TTT, kkk are the choices of the source's proof. Algorithm 1 is the shared formalization of the earlier corollary; the outcome law is a weight on (returned index, applications of UfU_fUf​, gates).

Significance

These are known mathematical results. Their Lean statements have been checked and reviewed against the cited source; formal proofs remain open for contributors.

Supporting targets

  • Theorem 4.6: Amplitude amplification: overlap 2/3 with the good state using O(1/√p) applications of S and R.
  • Corollary 4.9: Search with a known number of marked elements in O(√(2^n/|M|)) oracle calls.
  • Corollary 4.10: Finding all marked elements with O(√(|M| 2^n)) oracle calls.
  • Lemma 4.11: Oblivious amplitude amplification: the partner state Ψ⊥ is orthogonal to |0⟩|ψ⟩ and has no flag-0 support.
  • Theorem 4.12: Oblivious amplitude amplification: each iteration rotates by 2θ.
  • Proposition 4.15: Eigenstates of the Grover operator in amplitude estimation.
  • Proposition 4.19: Angle error to probability error for sin².
  • Proposition 4.21: Error bound of amplitude estimation with m qubits.
  • Corollary 4.22: Amplitude estimation to additive error ε with O(log 1/ε) qubits and O(1/ε) queries.
  • Lemma 4.28: Near-orthogonality of the two amplitude-estimation output states.
  • Theorem 4.29: Quantum search without knowing the number of marked items: O(1/θ) expected queries.
  • Corollary 4.30: Search without knowing |M| with O(√(2^n/|M|)) expected oracle calls.
  • Theorem 4.31: Quantum minimum finding (Dürr–Høyer), with explicit bit width, interruption and gate model.

Formalization scope

Prove the existing statements under their stated hypotheses and definitions. Any correction to a printed source claim remains explicit in its target description. All listed target proofs are currently sorry; no proof is claimed. The mission goal is the designated chapter capstone; the remaining targets are separate milestones. Parts a, b, c and d are merged into their original chapter.

Selected references

  • Giacomo Nannicini, Quantum algorithms for optimizers, 2025, v5; notes dated 9 July 2025. https://arxiv.org/pdf/2408.07086v5.
13 thms2 active usersReviewed
Control TheoryConvex OptimizationLinear algebra·Captain: mikedeng1

Quadratic Matrix Inequalities with Applications to Data-Based Control V: A Reduced LMI in P Alone and an Explicit Data-Based Stabilizing GainResearch Paper

Motivation

Data-driven control asks for a controller computed directly from measured trajectories of an unknown system, without first identifying a model. When the measurements are corrupted by noise, many systems explain the same data, and a controller is acceptable only if it works for all of them. Van Waarde, Camlibel, Eising and Trentelman (arXiv:2203.12959) model the noise by a quadratic matrix inequality (QMI). The set of systems consistent with the data is then itself the solution set of a QMI, and the design question becomes whether one QMI implies another. Their Theorem 5.1 gives a necessary and sufficient linear matrix inequality (LMI) for the existence of a common stabilizing controller, with no rank or Slater-type assumption on the data. It improves van Waarde, Camlibel, Mesbahi, IEEE TAC 2022, Thm. 14, which needed a generalized Slater condition.

The LMI of Theorem 5.1 has size 4n+m+14n+m+14n+m+1 and n(n+1)2+nm+1\tfrac{n(n+1)}{2}+nm+12n(n+1)​+nm+1 unknowns. Section 5.2 of the paper removes the controller variable: Theorem 5.3 gives an LMI of size 4n+14n+14n+1 in the Lyapunov matrix PPP and one scalar β\betaβ, and then writes a stabilizing controller as a closed-form expression in PPP and the data. This mission formalizes that reduction and the Section 3 results on QMI solution sets that it rests on.

Setting

All matrices are real. Sk\mathbb S^kSk is the set of symmetric k×kk\times kk×k matrices, A≥0A\ge 0A≥0 (A>0A>0A>0) means symmetric positive semidefinite (definite), and A†A^\daggerA† is the Moore–Penrose pseudo-inverse. For Π∈Sq+r\Pi\in\mathbb S^{q+r}Π∈Sq+r, partitioned with Π11\Pi_{11}Π11​ of size q×qq\times qq×q and Π22\Pi_{22}Π22​ of size r×rr\times rr×r,

Zr(Π)={Z∈Rr×q:[IqZ]⊤Π[IqZ]≥0},\mathcal Z_r(\Pi) = \Big\{Z\in\mathbb R^{r\times q} : \begin{bmatrix} I_q\\ Z\end{bmatrix}^\top\Pi\begin{bmatrix} I_q\\ Z\end{bmatrix}\ge 0\Big\},Zr​(Π)={Z∈Rr×q:[Iq​Z​]⊤Π[Iq​Z​]≥0},

and Zr+(Π)\mathcal Z_r^+(\Pi)Zr+​(Π) is the same set with >0>0>0. The generalized Schur complement is Π ∣ Π22=Π11−Π12Π22†Π21\Pi\,|\,\Pi_{22} = \Pi_{11}-\Pi_{12}\Pi_{22}^\dagger\Pi_{21}Π∣Π22​=Π11​−Π12​Π22†​Π21​, and Πq,r\boldsymbol\Pi_{q,r}Πq,r​ is the class of Π∈Sq+r\Pi\in\mathbb S^{q+r}Π∈Sq+r with Π22≤0\Pi_{22}\le 0Π22​≤0, Π ∣ Π22≥0\Pi\,|\,\Pi_{22}\ge 0Π∣Π22​≥0 and ker⁡Π22⊆ker⁡Π12\ker\Pi_{22}\subseteq\ker\Pi_{12}kerΠ22​⊆kerΠ12​. For W∈Rq×pW\in\mathbb R^{q\times p}W∈Rq×p, Zr(Π)W={ZW:Z∈Zr(Π)}\mathcal Z_r(\Pi)W = \{ZW : Z\in\mathcal Z_r(\Pi)\}Zr​(Π)W={ZW:Z∈Zr​(Π)} and ΠW=diag(W,Ir)⊤ Π diag(W,Ir)\Pi_W = \mathrm{diag}(W,I_r)^\top\,\Pi\,\mathrm{diag}(W,I_r)ΠW​=diag(W,Ir​)⊤Πdiag(W,Ir​).

Data. An unknown system x(t+1)=Asx(t)+Bsu(t)+w(t)x(t+1) = A_sx(t)+B_su(t)+w(t)x(t+1)=As​x(t)+Bs​u(t)+w(t) produces X=[x(0)⋯x(T)]X = [x(0)\cdots x(T)]X=[x(0)⋯x(T)] and U−=[u(0)⋯u(T−1)]U_- = [u(0)\cdots u(T-1)]U−​=[u(0)⋯u(T−1)]; X−X_-X−​ and X+X_+X+​ are the first and last TTT columns of XXX. The noise matrix W−=[w(0)⋯w(T−1)]W_- = [w(0)\cdots w(T-1)]W−​=[w(0)⋯w(T−1)] is unknown but satisfies W−⊤∈ZT(Φ)W_-^\top\in\mathcal Z_T(\Phi)W−⊤​∈ZT​(Φ) for a given Φ∈Πn,T\Phi\in\boldsymbol\Pi_{n,T}Φ∈Πn,T​. The consistent systems are

Σ={(A,B):X+=AX−+BU−+W− for some W−⊤∈ZT(Φ)}.\Sigma = \{(A,B) : X_+ = AX_-+BU_-+W_-\ \text{for some}\ W_-^\top\in\mathcal Z_T(\Phi)\}.Σ={(A,B):X+​=AX−​+BU−​+W−​ for some W−⊤​∈ZT​(Φ)}.

The data are informative for quadratic stabilization if there are KKK and P>0P>0P>0 with P−(A+BK)P(A+BK)⊤>0P-(A+BK)P(A+BK)^\top>0P−(A+BK)P(A+BK)⊤>0 for all (A,B)∈Σ(A,B)\in\Sigma(A,B)∈Σ.

Formalization targets

Goal: Theorem 5.3

With Θ=Φ12+X+Φ22\Theta = \Phi_{12}+X_+\Phi_{22}Θ=Φ12​+X+​Φ22​ and V=[X−; U−]V = [X_-;\,U_-]V=[X−​;U−​]: the data are informative if and only if some P>0P>0P>0, β>0\beta>0β>0 satisfy

P−βI−[I X+]Φ[I X+]⊤+ΘV⊤(VΦ22V⊤)†VΘ⊤≥0,[P−βI00−P]−[IX+0−X−]Φ[IX+0−X−]⊤≥0,P-\beta I-[I\ X_+]\Phi[I\ X_+]^\top+\Theta V^\top(V\Phi_{22}V^\top)^\dagger V\Theta^\top\ge 0,\qquad \begin{bmatrix}P-\beta I&0\\0&-P\end{bmatrix}-\begin{bmatrix}I&X_+\\0&-X_-\end{bmatrix}\Phi\begin{bmatrix}I&X_+\\0&-X_-\end{bmatrix}^\top\ge 0,P−βI−[I X+​]Φ[I X+​]⊤+ΘV⊤(VΦ22​V⊤)†VΘ⊤≥0,[P−βI0​0−P​]−[I0​X+​−X−​​]Φ[I0​X+​−X−​​]⊤≥0,

and then

K=(U−(Φ22+Θ⊤Γ†Θ)X−⊤)(X−(Φ22+Θ⊤Γ†Θ)X−⊤)†,Γ=P−βI−[I X+]Φ[I X+]⊤,K = \big(U_-(\Phi_{22}+\Theta^\top\Gamma^\dagger\Theta)X_-^\top\big)\big(X_-(\Phi_{22}+\Theta^\top\Gamma^\dagger\Theta)X_-^\top\big)^\dagger,\qquad \Gamma = P-\beta I-[I\ X_+]\Phi[I\ X_+]^\top,K=(U−​(Φ22​+Θ⊤Γ†Θ)X−⊤​)(X−​(Φ22​+Θ⊤Γ†Θ)X−⊤​)†,Γ=P−βI−[I X+​]Φ[I X+​]⊤,

stabilizes every (A,B)∈Σ(A,B)\in\Sigma(A,B)∈Σ. Under Φ22<0\Phi_{22}<0Φ22​<0 and rank⁡V=n+m\operatorname{rank}V = n+mrankV=n+m the same holds with strict inequalities, β=0\beta = 0β=0, and (VΦ22V⊤)−1(V\Phi_{22}V^\top)^{-1}(VΦ22​V⊤)−1. The goal states all four parts.

Milestones

  1. Lemma A.1: A⊤A≤B⊤BA^\top A\le B^\top BA⊤A≤B⊤B iff A=SBA = SBA=SB with S⊤S≤IS^\top S\le IS⊤S≤I (and the strict version), with S=AB†S = AB^\daggerS=AB†.
  2. Lemma A.2: the solutions of AM=BAM = BAM=B.
  3. Theorem 3.3: all solutions of a QMI, Z=−Π22†Π21+((−Π22)†)1/2S(Π∣Π22)1/2+(I−Π22†Π22)TZ = -\Pi_{22}^\dagger\Pi_{21}+((-\Pi_{22})^\dagger)^{1/2}S(\Pi|\Pi_{22})^{1/2}+(I-\Pi_{22}^\dagger\Pi_{22})TZ=−Π22†​Π21​+((−Π22​)†)1/2S(Π∣Π22​)1/2+(I−Π22†​Π22​)T.
  4. Theorems 3.4 and 3.5: Zr(Π)W=Zr(ΠW)\mathcal Z_r(\Pi)W = \mathcal Z_r(\Pi_W)Zr​(Π)W=Zr​(ΠW​) and its strict analogue.
  5. Corollaries 3.6 and 3.7: a QMI solution with ZW=YZW = YZW=Y exists iff Y∈Zr(ΠW)Y\in\mathcal Z_r(\Pi_W)Y∈Zr​(ΠW​).
  6. Theorem 5.1: informativity iff the LMI in (P,L,β)(P,L,\beta)(P,L,β) is feasible, with K=LP−1K = LP^{-1}K=LP−1.

Significance

Theorem 5.3 separates the computation of a Lyapunov matrix from the computation of the controller. The LMI drops from size 4n+m+14n+m+14n+m+1 to 4n+14n+14n+1 and loses the nmnmnm entries of LLL. A controller then follows from PPP by one explicit formula, with no second optimization. The noise class Πn,T\boldsymbol\Pi_{n,T}Πn,T​ covers energy bounds, bounds on individual samples, noise confined to a known subspace, and exact data, so the result applies where earlier conditions requiring Φ22<0\Phi_{22}<0Φ22​<0 or a Slater point fail.

The Section 3 results are general facts about QMIs: an explicit parameterization of all solutions, also when the solution set is unbounded, and the identification of the image of a solution set under a linear map. They are reusable wherever QMIs appear in robust control: Petersen's lemma, the elimination lemma, dissipativity analysis.

All results are proved in the paper, with the exception of the proof of Theorem 3.5, which is only described as similar to that of Theorem 3.4. None is formalized. The work here is the formal development: a pseudo-inverse calculus for real matrices, generalized Schur complements, matrix square roots in the Loewner order, and the block-matrix identities of Section 5.

Difficulty

The QMI reformulation of the stabilization problem is short. The difficulty lies in degenerate matrices. Π22\Pi_{22}Π22​ is typically singular, so every step works with pseudo-inverses, kernel inclusions and generalized Schur complements, and identities that are routine for invertible blocks need range conditions. The closed-form gain (5.10) comes from a block pseudo-inverse formula for a positive semidefinite matrix whose blocks may all be singular, and checking it requires tracking which ranges contain which. The strict statements need a separate argument: a strict inequality A⊤A<B⊤BA^\top A<B^\top BA⊤A<B⊤B does not pass to W⊤A⊤AW<W⊤B⊤BWW^\top A^\top AW<W^\top B^\top BWW⊤A⊤AW<W⊤B⊤BW without full column rank of WWW, which is why Theorem 3.5 and Corollary 3.7 assume it.

The first idea, to eliminate LLL from (5.5) by completing the square, runs into the same singular blocks: the minimizing LLL involves pseudo-inverses of data-dependent matrices whose ranges are not controlled by the LMI alone.

Formalization scope

Lean conventions:

  • Index sets are arbitrary finite types, so qqq, rrr, ppp are cardinalities. The data dimensions nnn, mmm, TTT are natural numbers, with XXX an n×(T+1)n\times(T+1)n×(T+1) matrix.
  • A≥0A\ge 0A≥0 is Mathlib's PosSemidef (which includes symmetry); A>0A>0A>0 is PosDef.
  • A†A^\daggerA† is defined as the matrix satisfying the four Penrose conditions, which exists and is unique. Mathlib's A⁻¹, which returns 000 on singular matrices, appears only in (5.11a) and in K=LP−1K = LP^{-1}K=LP−1, where the matrices are invertible.
  • Square roots are CFC.sqrt in the Loewner order. Kernel inclusions are stated on vectors. Full column rank of W∈Rq×pW\in\mathbb R^{q\times p}W∈Rq×p is rank W = p.

The statements of Theorems 5.1 and 5.3 carry the hypothesis that the data were generated by some (As,Bs)(A_s,B_s)(As​,Bs​), i.e. Σ≠∅\Sigma\neq\emptysetΣ=∅; the paper assumes this when it says the data are generated by (2.1) with noise model (2.3). Without it, informativity holds vacuously and the equivalences fail. "Stabilizing gain" is asserted with the PPP of the LMI: the Lyapunov inequality P−(A+BK)P(A+BK)⊤>0P-(A+BK)P(A+BK)^\top>0P−(A+BK)P(A+BK)⊤>0 and (A+BK)k→0(A+BK)^k\to 0(A+BK)k→0 for every (A,B)∈Σ(A,B)\in\Sigma(A,B)∈Σ. Corollaries 3.6 and 3.7 print the hypothesis ker⁡Π22⊆ker⁡Π21\ker\Pi_{22}\subseteq\ker\Pi_{21}kerΠ22​⊆kerΠ21​, which is dimensionally impossible; it is read as ker⁡Π22⊆ker⁡Π12\ker\Pi_{22}\subseteq\ker\Pi_{12}kerΠ22​⊆kerΠ12​. The printed dimensions in Lemma A.2 are corrected to those its formula requires.

A formalization in which Σ may be empty, or in which A†A^\daggerA† is replaced by the total inverse, would make the goal vacuous or false. Both are ruled out above.

Theorem 5.1 is restated here as a milestone. It is also the goal of the companion mission IV, and solvers may prove it there.

Reusable infrastructure is welcome as separate contributions: uniqueness and the basic identities of the matrix pseudo-inverse; the generalized Schur complement criterion for positive semidefiniteness of a block matrix; Douglas-type factorization (Lemma A.1); square roots of positive semidefinite matrices commuting with pseudo-inverses.

Selected references

  • H. J. van Waarde, M. K. Camlibel, J. Eising, H. L. Trentelman, Quadratic matrix inequalities with applications to data-based control, SIAM J. Control Optim., 2023; preprint arXiv:2203.12959v3 (the version cited here). https://arxiv.org/abs/2203.12959
  • H. J. van Waarde, M. K. Camlibel, M. Mesbahi, From noisy data to feedback controllers: nonconservative design via a matrix S-lemma, IEEE Trans. Automat. Control, 67 (2022), pp. 162–175. https://doi.org/10.1109/TAC.2020.3047577
  • H. J. van Waarde, J. Eising, H. L. Trentelman, M. K. Camlibel, Data informativity: a new perspective on data-driven analysis and control, IEEE Trans. Automat. Control, 65 (2020), pp. 4753–4768. https://doi.org/10.1109/TAC.2020.2966717
  • D. S. Bernstein, Matrix Mathematics, 2nd ed., Princeton University Press, 2009. https://doi.org/10.1515/9781400833344
12 thms2 active usersReviewed
Control TheoryConvex OptimizationLinear algebra·Captain: mikedeng1

Quadratic Matrix Inequalities with Applications to Data-Based Control IV: Noisy Data Are Informative for Quadratic Stabilization iff an LMI in (P, L, β) Is FeasibleResearch Paper

Motivation

Data-driven control asks for a controller computed directly from measured trajectories of an unknown system, without first identifying a model. When the measurements are corrupted by unknown process noise, many systems explain the same data, and a controller designed from the data alone is only trustworthy if it stabilizes every one of them. Van Waarde, Camlibel, Eising and Trentelman (arXiv:2203.12959, SIAM J. Control Optim. 2023) show that this robust design problem is an implication between two quadratic matrix inequalities (QMIs), and they develop matrix versions of Yakubovich's S-lemma and of Finsler's lemma to decide such implications by a linear matrix inequality (LMI).

Timeline:

  • 1971: Yakubovich proves the S-lemma for one quadratic constraint under a Slater condition (English translation 1977; see the Pólik–Terlaky survey).
  • 2022: van Waarde, Camlibel and Mesbahi (IEEE Trans. Automat. Control 67, pp. 162–175, Thms. 13–14) give an LMI condition for informativity for quadratic stabilization from noisy data, assuming Φ22<0\Phi_{22}<0Φ22​<0 and a generalized Slater condition.
  • 2023: the present paper removes the Slater condition and the definiteness of Φ22\Phi_{22}Φ22​: Theorem 5.1 holds for every noise model in the class Πn,T\boldsymbol\Pi_{n,T}Πn,T​, including exact measurements and noise confined to a subspace.

Setting

All matrices are real. Sk\mathbb{S}^kSk is the set of symmetric k×kk\times kk×k matrices; A⩾0A\geqslant 0A⩾0 (A>0A>0A>0) means AAA is symmetric positive semidefinite (definite), and A⩽0A\leqslant 0A⩽0 means −A⩾0-A\geqslant 0−A⩾0.

QMI solution sets. For Π∈Sq+r\Pi\in\mathbb{S}^{q+r}Π∈Sq+r partitioned with Π11∈Sq\Pi_{11}\in\mathbb{S}^qΠ11​∈Sq, Π22∈Sr\Pi_{22}\in\mathbb{S}^rΠ22​∈Sr,

Zr(Π)={Z∈Rr×q:[IqZ] ⁣⊤Π[IqZ]⩾0},\mathcal Z_r(\Pi)=\Big\{Z\in\mathbb{R}^{r\times q}:\begin{bmatrix}I_q\\Z\end{bmatrix}^{\!\top}\Pi\begin{bmatrix}I_q\\Z\end{bmatrix}\geqslant 0\Big\},Zr​(Π)={Z∈Rr×q:[Iq​Z​]⊤Π[Iq​Z​]⩾0},

with Zr+(Π)\mathcal Z_r^+(\Pi)Zr+​(Π) and Zr0(Π)\mathcal Z_r^0(\Pi)Zr0​(Π) defined by ">0>0>0" and "=0=0=0". The generalized Schur complement is Π ∣ Π22=Π11−Π12Π22†Π21\Pi\,|\,\Pi_{22}=\Pi_{11}-\Pi_{12}\Pi_{22}^\dagger\Pi_{21}Π∣Π22​=Π11​−Π12​Π22†​Π21​, with †\dagger† the Moore–Penrose pseudo-inverse, and Πq,r\boldsymbol\Pi_{q,r}Πq,r​ is the class of Π∈Sq+r\Pi\in\mathbb{S}^{q+r}Π∈Sq+r with Π22⩽0\Pi_{22}\leqslant 0Π22​⩽0, Π ∣ Π22⩾0\Pi\,|\,\Pi_{22}\geqslant 0Π∣Π22​⩾0 and ker⁡Π22⊆ker⁡Π12\ker\Pi_{22}\subseteq\ker\Pi_{12}kerΠ22​⊆kerΠ12​.

Data. The unknown system is x(t+1)=Asx(t)+Bsu(t)+w(t)x(t+1)=A_sx(t)+B_su(t)+w(t)x(t+1)=As​x(t)+Bs​u(t)+w(t) with x,w∈Rnx,w\in\mathbb{R}^nx,w∈Rn, u∈Rmu\in\mathbb{R}^mu∈Rm. The measured data are X=[x(0) ⋯ x(T)]X=[x(0)\ \cdots\ x(T)]X=[x(0) ⋯ x(T)] and U−=[u(0) ⋯ u(T−1)]U_-=[u(0)\ \cdots\ u(T-1)]U−​=[u(0) ⋯ u(T−1)]; X−X_-X−​ and X+X_+X+​ are the first and last TTT columns of XXX. The unknown noise W−=[w(0) ⋯ w(T−1)]W_-=[w(0)\ \cdots\ w(T-1)]W−​=[w(0) ⋯ w(T−1)] satisfies W−⊤∈ZT(Φ)W_-^\top\in\mathcal Z_T(\Phi)W−⊤​∈ZT​(Φ) for a given Φ∈Πn,T\Phi\in\boldsymbol\Pi_{n,T}Φ∈Πn,T​. The systems compatible with the data form

Σ={(A,B):X+=AX−+BU−+W− for some W− with W−⊤∈ZT(Φ)},\Sigma=\{(A,B): X_+=AX_-+BU_-+W_- \text{ for some } W_- \text{ with } W_-^\top\in\mathcal Z_T(\Phi)\},Σ={(A,B):X+​=AX−​+BU−​+W−​ for some W−​ with W−⊤​∈ZT​(Φ)},

and (As,Bs)∈Σ(A_s,B_s)\in\Sigma(As​,Bs​)∈Σ. The data are informative for quadratic stabilization (Definition 2.1) if some KKK and P>0P>0P>0 satisfy P−(A+BK)P(A+BK)⊤>0P-(A+BK)P(A+BK)^\top>0P−(A+BK)P(A+BK)⊤>0 for all (A,B)∈Σ(A,B)\in\Sigma(A,B)∈Σ.

Formalization targets

Goal: Theorem 5.1

Let Lβ(P,L)\mathcal L_\beta(P,L)Lβ​(P,L) be the (n+n+m+n)(n+n+m+n)(n+n+m+n)-square matrix

[P−βI0000−P−L⊤00−L0L00L⊤P]−[IX+0−X−0−U−00]Φ[IX+0−X−0−U−00] ⁣⊤.\begin{bmatrix}P-\beta I&0&0&0\\0&-P&-L^\top&0\\0&-L&0&L\\0&0&L^\top&P\end{bmatrix}-\begin{bmatrix}I&X_+\\0&-X_-\\0&-U_-\\0&0\end{bmatrix}\Phi\begin{bmatrix}I&X_+\\0&-X_-\\0&-U_-\\0&0\end{bmatrix}^{\!\top}.​P−βI000​0−P−L0​0−L⊤0L⊤​00LP​​−​I000​X+​−X−​−U−​0​​Φ​I000​X+​−X−​−U−​0​​⊤.

For Φ∈Πn,T\Phi\in\boldsymbol\Pi_{n,T}Φ∈Πn,T​ and data generated by the system:

(a) the data are informative if and only if ∃ P>0, L, β>0\exists\,P>0,\ L,\ \beta>0∃P>0, L, β>0 with Lβ(P,L)⩾0\mathcal L_\beta(P,L)\geqslant 0Lβ​(P,L)⩾0 (5.5);

(b) if moreover Φ22<0\Phi_{22}<0Φ22​<0 and rank⁡[X−;U−]=n+m\operatorname{rank}[X_-;U_-]=n+mrank[X−​;U−​]=n+m, the data are informative if and only if ∃ P>0, L\exists\,P>0,\ L∃P>0, L with L0(P,L)>0\mathcal L_0(P,L)>0L0​(P,L)>0 (5.6);

(c) whenever (5.5) or (5.6) holds, K=LP−1K=LP^{-1}K=LP−1 satisfies the Lyapunov inequality with that PPP and makes A+BKA+BKA+BK Schur stable for every (A,B)∈Σ(A,B)\in\Sigma(A,B)∈Σ.

Milestones

The vector S-lemma, strict S-lemma and Finsler's lemma (Lemmas 4.1–4.3); the completion and lifting lemmas 4.4–4.5; the reduction of matrix inclusions to vector implications (Theorem 4.6); the matrix S-lemmas

Zr(N)⊆Zr(M)  ⟺  ∃ α⩾0: M−αN⩾0,Zr(N)⊆Zr+(M)  ⟺  ∃ α⩾0, β>0: M−αN⩾[βI000]\mathcal Z_r(N)\subseteq\mathcal Z_r(M)\iff\exists\,\alpha\geqslant 0:\ M-\alpha N\geqslant 0,\qquad \mathcal Z_r(N)\subseteq\mathcal Z_r^+(M)\iff\exists\,\alpha\geqslant 0,\ \beta>0:\ M-\alpha N\geqslant\begin{bmatrix}\beta I&0\\0&0\end{bmatrix}Zr​(N)⊆Zr​(M)⟺∃α⩾0: M−αN⩾0,Zr​(N)⊆Zr+​(M)⟺∃α⩾0, β>0: M−αN⩾[βI0​00​]

(Theorems 4.7, 4.10–4.12, Corollary 4.13); and the three claims of §5.1 that put Theorem 5.1 in the form of Corollary 4.13: Σ=Zn+m(N)\Sigma=\mathcal Z_{n+m}(N)Σ=Zn+m​(N), the Lyapunov inequality is [A B]⊤∈Zn+m+(M)[A\ B]^\top\in\mathcal Z^+_{n+m}(M)[A B]⊤∈Zn+m+​(M), and N∈Πn,n+mN\in\boldsymbol\Pi_{n,n+m}N∈Πn,n+m​, M22⩽0M_{22}\leqslant 0M22​⩽0.

Significance

Theorem 5.1(a) is an exact test, with no rank condition on the data and no Slater condition on the noise model, for the existence of one state-feedback gain with a common quadratic Lyapunov function that stabilizes every system consistent with noisy data. The test is a semidefinite feasibility problem in (P,L,β)(P,L,\beta)(P,L,β) and returns the controller K=LP−1K=LP^{-1}K=LP−1. It covers energy bounds, per-sample bounds, sample-covariance bounds, noise in a known subspace and exact data within one statement, and its matrix S-lemmas are the tools reused in the paper's further results (the reduced LMI and explicit gain of §5.2, Petersen's lemma in §4.5).

None of these results has a machine-checked proof. Mathlib has the Loewner order, positive (semi)definiteness and spectral theory of symmetric matrices, but no S-lemma, no Finsler lemma, no Moore–Penrose pseudo-inverse for matrices and no theory of QMI solution sets. A formalization would supply these as reusable linear-algebra results and would certify the design rule K=LP−1K=LP^{-1}K=LP−1.

Difficulty

The "if" directions are direct computations: multiply out a Schur complement. The difficulty is in the "only if" directions. The S-lemma's converse relies on the hidden convexity of the joint range of two quadratic forms (Dines' theorem), which is not a consequence of finite-dimensional convex duality alone. Passing from matrices ZZZ to vectors requires Lemma 4.5, which builds a full matrix solution of the QMI from one vector; it uses the pseudo-inverse factorization of NNN and fails without ker⁡N22⊆ker⁡N12\ker N_{22}\subseteq\ker N_{12}kerN22​⊆kerN12​. In Theorem 4.11 the existence of the margin β>0\beta>0β>0 comes from a compactness argument on an unbounded solution set, after splitting off the kernel of N22N_{22}N22​. The obvious approach, applying the strict matrix S-lemma (Theorem 4.10) directly, does not work because N22N_{22}N22​ is only semidefinite in the noise models of interest.

Formalization scope

Matrices are Matrix ι ι ℝ over finite index types; a matrix in Sq+r\mathbb{S}^{q+r}Sq+r is indexed by ι ⊕ κ, and block matrices are Matrix.fromBlocks / Matrix.fromRows. ⩾0\geqslant 0⩾0 and >0>0>0 are Mathlib's PosSemidef and PosDef, which include symmetry, as the paper's §1.1 convention does. The pseudo-inverse is defined as the unique matrix satisfying the four Penrose conditions, not Matrix.inv, which returns 000 on singular matrices. "Stabilizing gain for all of Σ\SigmaΣ" is stated as the Lyapunov inequality with the LMI's PPP together with (A+BK)k→0(A+BK)^k\to 0(A+BK)k→0.

The following are added or made explicit:

  • the hypothesis that the data are generated by the system (Σ≠∅\Sigma\neq\emptysetΣ=∅). Without it informativity holds vacuously and the goal is false;
  • q⩾1q\geqslant 1q⩾1 in the "only if" parts of Theorem 4.12 and Corollary 4.13. These parts fail at q=0q=0q=0, e.g. for N=[0]N=[0]N=[0], M=[−1]M=[-1]M=[−1], and the paper's proofs choose a nonzero vector of Rq\mathbb{R}^qRq;
  • n=k+1n=k+1n=k+1 in Lemma 4.4, to avoid natural-number subtraction.

The goal adds no hypothesis n⩾1n\geqslant 1n⩾1. A formalization in which Σ\SigmaΣ may be empty, or in which the pseudo-inverse is a junk value, would be trivial or false, and the definitions rule both out.

Contributions are welcome for the pseudo-inverse API (existence, uniqueness, ker⁡\kerker identities), the S-lemma and Finsler's lemma (reusable beyond this mission), and the Schur-complement factorizations (3.2)–(3.4). Platform items ConvexOptimization.s_procedure (inhomogeneous S-procedure) and ConvexOptimization.field_of_values_psd_witness (Dines) are natural proof tools for Lemma 4.1.

Selected references

  • H. J. van Waarde, M. K. Camlibel, J. Eising, H. L. Trentelman, Quadratic matrix inequalities with applications to data-based control, SIAM J. Control Optim., 2023. arXiv:2203.12959v3. https://arxiv.org/abs/2203.12959
  • H. J. van Waarde, M. K. Camlibel, M. Mesbahi, From noisy data to feedback controllers: nonconservative design via a matrix S-lemma, IEEE Trans. Automat. Control 67 (2022), pp. 162–175.
  • I. Pólik, T. Terlaky, A survey of the S-lemma, SIAM Review 49 (2007). https://doi.org/10.1137/S003614450444614X
  • V. A. Yakubovich, S-procedure in nonlinear control theory, Vestnik Leningrad Univ. Math. 4 (1977), pp. 73–93 (English translation).
  • S. Boyd, L. El Ghaoui, E. Feron, V. Balakrishnan, Linear Matrix Inequalities in System and Control Theory, SIAM, 1994. https://doi.org/10.1137/1.9781611970777
17 thms2 active usersReviewed
Control TheoryConvex OptimizationLinear algebra·Captain: mikedeng1

Quadratic Matrix Inequalities with Applications to Data-Based Control I: Nonemptiness, Convexity, Boundedness and Interior of QMI Solution SetsResearch Paper

Motivation

Quadratic matrix inequalities (QMIs) describe sets of matrices rather than sets of vectors. They appear whenever an unknown matrix is constrained by a quadratic bound: in robust control, where an uncertain block of a system is only known to lie in a norm ball or a sector, and in data-driven control, where the system matrices (A,B)(A, B)(A,B) of a linear system x(t+1)=Ax(t)+Bu(t)+w(t)x(t+1) = Ax(t) + Bu(t) + w(t)x(t+1)=Ax(t)+Bu(t)+w(t) are unknown and the measured data, corrupted by noise satisfying a quadratic bound, only pin them down to a set defined by a QMI.

H. J. van Waarde, M. K. Camlibel, J. Eising and H. L. Trentelman (arXiv:2203.12959, SIAM J. Control Optim. 2023) develop a systematic theory of such sets, culminating in matrix versions of the S-lemma and of Finsler's lemma and in data-based stabilization tests. Before any of that can be used, one has to know when the set of matrices satisfying a QMI is nonempty, convex, bounded, has interior points, or contains strict and boundary solutions. Section 3.1 of the paper answers these questions in terms of the blocks of the matrix defining the QMI. This mission formalizes that section, Theorem 3.2, together with the Schur complement identities (3.2)–(3.4) on which it rests.

Setting

All matrices are real. Fix integers q,r≥0q, r \ge 0q,r≥0. For a matrix AAA, the Moore–Penrose pseudo-inverse A†A^\daggerA† is the unique matrix XXX with

AXA=A,XAX=X,(AX)⊤=AX,(XA)⊤=XA.AXA = A,\quad XAX = X,\quad (AX)^\top = AX,\quad (XA)^\top = XA.AXA=A,XAX=X,(AX)⊤=AX,(XA)⊤=XA.

Sn\mathbb S^nSn is the set of symmetric n×nn\times nn×n matrices; for A∈SnA \in \mathbb S^nA∈Sn, A≥0A \ge 0A≥0 and A>0A > 0A>0 mean positive semidefinite and positive definite, A≤0A \le 0A≤0, A<0A < 0A<0 the negative versions, and A≥BA \ge BA≥B means A−B≥0A - B \ge 0A−B≥0 (the Loewner order).

A matrix Π∈Sq+r\Pi \in \mathbb S^{q+r}Π∈Sq+r is always partitioned as

Π=[Π11Π12Π21Π22],Π11∈Rq×q, Π22∈Rr×r, Π21=Π12⊤.\Pi = \begin{bmatrix}\Pi_{11} & \Pi_{12}\\ \Pi_{21} & \Pi_{22}\end{bmatrix},\qquad \Pi_{11}\in\mathbb R^{q\times q},\ \Pi_{22}\in\mathbb R^{r\times r},\ \Pi_{21} = \Pi_{12}^\top .Π=[Π11​Π21​​Π12​Π22​​],Π11​∈Rq×q, Π22​∈Rr×r, Π21​=Π12⊤​.

For Z∈Rr×qZ \in \mathbb R^{r\times q}Z∈Rr×q the QMI is

[IqZ]⊤Π[IqZ]≥0,\begin{bmatrix} I_q\\ Z\end{bmatrix}^\top \Pi \begin{bmatrix} I_q\\ Z\end{bmatrix} \ge 0,[Iq​Z​]⊤Π[Iq​Z​]≥0,

with solution set Zr(Π)\mathcal Z_r(\Pi)Zr​(Π). Replacing ≥0\ge 0≥0 by >0> 0>0 or =0= 0=0 gives the sets Zr+(Π)\mathcal Z_r^+(\Pi)Zr+​(Π) of strict solutions and Zr0(Π)\mathcal Z_r^0(\Pi)Zr0​(Π) of solutions of the quadratic matrix equation. The generalized Schur complement is

Π ∣ Π22:=Π11−Π12Π22†Π21,\Pi\,|\,\Pi_{22} := \Pi_{11} - \Pi_{12}\Pi_{22}^\dagger\Pi_{21},Π∣Π22​:=Π11​−Π12​Π22†​Π21​,

and the class Πq,r\boldsymbol\Pi_{q,r}Πq,r​ (equation (3.5) of the paper) consists of the Π∈Sq+r\Pi \in \mathbb S^{q+r}Π∈Sq+r with

Π22≤0,Π ∣ Π22≥0,ker⁡Π22⊆ker⁡Π12.\Pi_{22} \le 0,\qquad \Pi\,|\,\Pi_{22} \ge 0,\qquad \ker\Pi_{22} \subseteq \ker\Pi_{12}.Π22​≤0,Π∣Π22​≥0,kerΠ22​⊆kerΠ12​.

In Lean these are pinv, qmiForm, ZSet, ZPlus, ZZero, schur and InPi in the namespace QuadMatIneq.Basic.

Formalization targets

Goal: Theorem 3.2

For Π∈Πq,r\Pi \in \boldsymbol\Pi_{q,r}Π∈Πq,r​:

  • (a) Zr(Π)\mathcal Z_r(\Pi)Zr​(Π) is nonempty and convex;
  • (b) for q≥1q \ge 1q≥1, Zr(Π)\mathcal Z_r(\Pi)Zr​(Π) is bounded if and only if Π22<0\Pi_{22} < 0Π22​<0;
  • (c) Zr(Π)\mathcal Z_r(\Pi)Zr​(Π) has nonempty interior if and only if Π22=0\Pi_{22} = 0Π22​=0 or Π ∣ Π22>0\Pi\,|\,\Pi_{22} > 0Π∣Π22​>0;
  • (d) Zr+(Π)≠∅\mathcal Z_r^+(\Pi) \ne \varnothingZr+​(Π)=∅ if and only if Π ∣ Π22>0\Pi\,|\,\Pi_{22} > 0Π∣Π22​>0;
  • (e) Zr0(Π)≠∅\mathcal Z_r^0(\Pi) \ne \varnothingZr0​(Π)=∅ if and only if rank⁡Π22≥rank⁡(Π ∣ Π22)\operatorname{rank}\Pi_{22} \ge \operatorname{rank}(\Pi\,|\,\Pi_{22})rankΠ22​≥rank(Π∣Π22​).

Milestones

For Π∈Sq+r\Pi \in \mathbb S^{q+r}Π∈Sq+r with Π22≤0\Pi_{22} \le 0Π22​≤0 and ker⁡Π22⊆ker⁡Π12\ker\Pi_{22}\subseteq\ker\Pi_{12}kerΠ22​⊆kerΠ12​:

  • (3.2): ker⁡Π22⊆ker⁡Π12  ⟺  Π12Π22Π22†=Π12\ker\Pi_{22}\subseteq\ker\Pi_{12} \iff \Pi_{12}\Pi_{22}\Pi_{22}^\dagger = \Pi_{12}kerΠ22​⊆kerΠ12​⟺Π12​Π22​Π22†​=Π12​, and
Π=[IqΠ12Π22†0Ir][Π ∣ Π2200Π22][Iq0Π22†Π21Ir];\Pi = \begin{bmatrix} I_q & \Pi_{12}\Pi_{22}^\dagger\\ 0 & I_r\end{bmatrix}\begin{bmatrix}\Pi\,|\,\Pi_{22} & 0\\ 0 & \Pi_{22}\end{bmatrix}\begin{bmatrix} I_q & 0\\ \Pi_{22}^\dagger\Pi_{21} & I_r\end{bmatrix};Π=[Iq​0​Π12​Π22†​Ir​​][Π∣Π22​0​0Π22​​][Iq​Π22†​Π21​​0Ir​​];
  • (3.3): [IqZ]⊤Π[IqZ]=Π ∣ Π22+(Z+Π22†Π21)⊤Π22(Z+Π22†Π21)\begin{bmatrix} I_q\\ Z\end{bmatrix}^\top \Pi \begin{bmatrix} I_q\\ Z\end{bmatrix} = \Pi\,|\,\Pi_{22} + (Z + \Pi_{22}^\dagger\Pi_{21})^\top\Pi_{22}(Z + \Pi_{22}^\dagger\Pi_{21})[Iq​Z​]⊤Π[Iq​Z​]=Π∣Π22​+(Z+Π22†​Π21​)⊤Π22​(Z+Π22†​Π21​);
  • (3.4): Π ∣ Π22\Pi\,|\,\Pi_{22}Π∣Π22​ is the value at Z=−Π22†Π21Z = -\Pi_{22}^\dagger\Pi_{21}Z=−Π22†​Π21​ and dominates the value at every ZZZ in the Loewner order;
  • the claim after (3.4): Zr(Π)≠∅\mathcal Z_r(\Pi) \ne\varnothingZr​(Π)=∅ if and only if Π ∣ Π22≥0\Pi\,|\,\Pi_{22}\ge 0Π∣Π22​≥0;
  • Theorem 3.2(b), (c), (e) as separate items.

Significance

Theorem 3.2 turns geometric questions about a set of matrices into finite checks on the blocks of Π\PiΠ: semidefiniteness, definiteness, vanishing and a rank comparison. In the data-driven setting of the paper, Zr(Π)\mathcal Z_r(\Pi)Zr​(Π) (after transposition) is the set of noise samples, and hence of systems, consistent with the data; part (b) decides when that set is compact, part (c) when it has interior, and (a) guarantees convexity, which is what allows linear matrix inequalities to be used over it. The identities (3.2)–(3.4) are used throughout the rest of the paper, in the matrix S-lemma, the matrix Finsler lemma and the parametrization of solution sets.

The result and its proof appear in the paper; this mission produces machine-checked versions. Mathlib has the Schur complement for invertible blocks and positive semidefiniteness, but no Moore–Penrose pseudo-inverse for matrices and nothing on QMI solution sets. To the knowledge of this mission no formalization of these statements exists.

Difficulty

The obvious argument is the classical Schur complement, which requires Π22\Pi_{22}Π22​ to be invertible. Here Π22\Pi_{22}Π22​ is only negative semidefinite, and the singular case is exactly where the interesting behaviour lives: the solution set is unbounded precisely when Π22\Pi_{22}Π22​ has a kernel, and Π22=0\Pi_{22} = 0Π22​=0 gives the whole space. Working with the pseudo-inverse requires the range and kernel identities of Π22†\Pi_{22}^\daggerΠ22†​ for a symmetric matrix and the kernel inclusion ker⁡Π22⊆ker⁡Π12\ker\Pi_{22}\subseteq\ker\Pi_{12}kerΠ22​⊆kerΠ12​ at every step. Part (c) needs a topological argument about the interior in Rr×q\mathbb R^{r\times q}Rr×q, and part (e) needs a factorization of one positive semidefinite matrix through another of possibly different size, which is a rank argument rather than an identity.

Formalization scope

  • Matrices are indexed by arbitrary finite types: ι\iotaι with ∣ι∣=q|\iota| = q∣ι∣=q and κ\kappaκ with ∣κ∣=r|\kappa| = r∣κ∣=r, so Π\PiΠ is a matrix over ι⊕κ\iota \oplus \kappaι⊕κ and its blocks are Mathlib's toBlocks₁₁, …, toBlocks₂₂. Nothing assumes q,r≥1q, r \ge 1q,r≥1 except part (b).
  • A†A^\daggerA† is defined by Classical.epsilon over the four Penrose conditions. Since a solution exists and is unique, this is the pseudo-inverse and not a default value; the total inverse A⁻¹, which is 000 on singular matrices, is not used anywhere.
  • ≥0\ge 0≥0 and >0> 0>0 are Mathlib's PosSemidef and PosDef, which include symmetry, matching the paper's convention that definiteness is only asserted for symmetric matrices. Π∈Sq+r\Pi\in\mathbb S^{q+r}Π∈Sq+r is IsHermitian.
  • Kernel inclusions are written ∀v, Π22v=0⇒Π12v=0\forall v,\ \Pi_{22}v = 0 \Rightarrow \Pi_{12}v = 0∀v, Π22​v=0⇒Π12​v=0.
  • "Bounded" means entrywise bounded, which is equivalent to boundedness in any norm on Rr×q\mathbb R^{r\times q}Rr×q. The interior is taken in Rr×q\mathbb R^{r\times q}Rr×q with its Euclidean topology, not relative to an affine hull.
  • Part (b) carries the hypothesis q≥1q \ge 1q≥1. For q=0q = 0q=0 the space Rr×0\mathbb R^{r\times 0}Rr×0 is a single point, so the solution set is bounded for every Π\PiΠ, including those with singular Π22\Pi_{22}Π22​, and (b) as printed fails. The other parts are stated without it and hold for q=0q = 0q=0 and for r=0r = 0r=0.
  • The trivializing formalizations are excluded: the pseudo-inverse is not a junk inverse, the class Πq,r\boldsymbol\Pi_{q,r}Πq,r​ is inhabited (e.g. Π=diag⁡(1,−1)\Pi = \operatorname{diag}(1, -1)Π=diag(1,−1) and Π=diag⁡(1,0)\Pi = \operatorname{diag}(1, 0)Π=diag(1,0) for q=r=1q = r = 1q=r=1), and the identities (3.2)–(3.4) are stated with the paper's exact block layouts.

Infrastructure needed and reusable beyond this mission: the existence and uniqueness of the matrix pseudo-inverse with its basic identities ((A†)⊤=(A⊤)†(A^\dagger)^\top = (A^\top)^\dagger(A†)⊤=(A⊤)†, AA†AA^\daggerAA† is the orthogonal projector onto the range), and the generalized Schur complement identities. Contributions of these as separate lemmas are welcome.

Selected references

  • H. J. van Waarde, M. K. Camlibel, J. Eising, H. L. Trentelman, Quadratic matrix inequalities with applications to data-based control, SIAM J. Control Optim., 2023; preprint arXiv:2203.12959v3. https://arxiv.org/abs/2203.12959
  • D. S. Bernstein, Matrix Mathematics: Theory, Facts, and Formulas, 2nd ed., Princeton University Press, 2009 (reference [4] of the paper, for eigenvalue and pseudo-inverse facts). https://doi.org/10.1515/9781400833344
  • F. Zhang (ed.), The Schur Complement and Its Applications, Springer, 2005. https://doi.org/10.1007/b105056
9 thms2 active usersReviewed
Convex OptimizationOperations ResearchOptimal Transport·Captain: mikedeng1

Conic Programming Reformulations of Two-Stage Distributionally Robust Linear Programs over Wasserstein Balls 1: Copositive Programs (28) Are Exact under Complete Recourse and Converge to (1) as δ ↓ 0Research Paper

Motivation

Two-stage decision problems split decisions into a here-and-now decision xxx, taken before an uncertain parameter ξ\xiξ is observed, and a wait-and-see (recourse) decision yyy, taken after. Stochastic programming minimizes the expected total cost under a known distribution of ξ\xiξ; robust optimization minimizes the worst-case cost over a support set. Distributionally robust optimization sits between the two: it minimizes the worst-case expected cost over an ambiguity set of distributions consistent with the available data. When the data are III samples ξ^1,…,ξ^I\hat\xi_1,\dots,\hat\xi_Iξ^​1​,…,ξ^​I​, a natural ambiguity set is the ball of radius ϵ\epsilonϵ around the empirical distribution in the Wasserstein distance, which contains the two-stage stochastic program with a discrete distribution (ϵ=0\epsilon = 0ϵ=0) and, for a bounded support, the two-stage robust program (large ϵ\epsilonϵ) as special cases.

Single-stage Wasserstein problems with piecewise-linear losses admit finite convex reformulations (Mohajerin Esfahani and Kuhn; Gao and Kleywegt). Two-stage problems are generically NP-hard, and the existing approaches were exponential-size reformulations or decision-rule restrictions. Hanasusanto and Kuhn (arXiv:1609.07505v3) show that two-stage distributionally robust linear programs over a 2-Wasserstein ball are equivalent to, or approximated arbitrarily well by, copositive programs of polynomial size: all the difficulty is moved into the cone. This mission formalizes that result, Theorem 5 of the paper, and the chain of lemmas behind it (§3, pp. 7–19).

Setting

The first-stage decision xxx ranges over a set X⊆RN1\mathcal X\subseteq\mathbb R^{N_1}X⊆RN1​ with cost c⊤xc^\top xc⊤x. The recourse function is the optimal value of a linear program,

Z(x,ξ)=inf⁡{(Qξ+q)⊤y:y∈RN2, T(x)ξ+h(x)≤Wy},Z(x,\xi) = \inf\{(Q\xi+q)^\top y : y\in\mathbb R^{N_2},\ T(x)\xi + h(x) \le Wy\},Z(x,ξ)=inf{(Qξ+q)⊤y:y∈RN2​, T(x)ξ+h(x)≤Wy},

where T(x)∈RM×KT(x)\in\mathbb R^{M\times K}T(x)∈RM×K and h(x)∈RMh(x)\in\mathbb R^Mh(x)∈RM are affine in xxx and W∈RM×N2W\in\mathbb R^{M\times N_2}W∈RM×N2​; its value lies in [−∞,+∞][-\infty,+\infty][−∞,+∞]. The uncertain parameter takes values in the polyhedron Ξ={ξ∈R+K:Sξ≤t}\Xi=\{\xi\in\mathbb R^K_+ : S\xi\le t\}Ξ={ξ∈R+K​:Sξ≤t}, assumed non-empty. The problem has complete recourse if Wy+>0Wy^+>0Wy+>0 for some y+y^+y+, and sufficiently expensive recourse if for every ξ∈Ξ\xi\in\Xiξ∈Ξ there is p≥0p\ge0p≥0 with Qξ+q=W⊤pQ\xi+q = W^\top pQξ+q=W⊤p (the dual recourse problem is feasible); the latter is a standing assumption.

The worst-case expectation is

Z(x)=sup⁡{EP[Z(x,ξ~)]:P(Ξ)=1, W2(P,P^I)≤ϵ},P^I=1I∑i∈[I]δξ^i,\mathcal Z(x) = \sup\Big\{\mathbb E_{\mathbb P}[Z(x,\tilde\xi)] : \mathbb P(\Xi)=1,\ W_2(\mathbb P,\hat{\mathbb P}_I)\le\epsilon\Big\},\qquad \hat{\mathbb P}_I = \frac1I\sum_{i\in[I]}\delta_{\hat\xi_i},Z(x)=sup{EP​[Z(x,ξ~​)]:P(Ξ)=1, W2​(P,P^I​)≤ϵ},P^I​=I1​i∈[I]∑​δξ^​i​​,

with W2W_2W2​ the 2-Wasserstein distance for the Euclidean cost. Problem (1) is: minimize c⊤x+Z(x)c^\top x + \mathcal Z(x)c⊤x+Z(x) over x∈Xx\in\mathcal Xx∈X.

A symmetric matrix MMM is copositive (M⪰C0M\succeq_{\mathcal C}0M⪰C​0) if v⊤Mv≥0v^\top Mv\ge0v⊤Mv≥0 for all v≥0v\ge0v≥0, strictly copositive (M≻C0M\succ_{\mathcal C}0M≻C​0) if v⊤Mv>0v^\top Mv>0v⊤Mv>0 for all nonzero v≥0v\ge0v≥0, and completely positive (M⪰C∗0M\succeq_{\mathcal C^*}0M⪰C∗​0) if M=BB⊤M=BB^\topM=BB⊤ for an entrywise nonnegative BBB. The extended data stack the support constraints under the recourse data: Q=[Q;S]\mathcal Q=[Q;S]Q=[Q;S], q=[q;−t]\boldsymbol q=[q;-t]q=[q;−t], T(x)=[T(x);0]\mathcal T(x)=[T(x);0]T(x)=[T(x);0], h(x)=[h(x);0]\boldsymbol h(x)=[h(x);0]h(x)=[h(x);0], W=[[W,0],[0,−I]]\mathcal W=[[W,0],[0,-\mathbb I]]W=[[W,0],[0,−I]] (in the paper, q\boldsymbol qq and h\boldsymbol hh are script letters).

For δ≥0\delta\ge0δ≥0, Z‾δ(x)\overline{\mathcal Z}_\delta(x)Zδ​(x) is the value of the copositive program (24): minimize ϵ2λ+1I∑i[si+q⊤ψi−λ∥ξ^i∥22+∑jϕijqj2]\epsilon^2\lambda + \frac1I\sum_i[s_i+\boldsymbol q^\top\psi_i-\lambda\|\hat\xi_i\|_2^2+\sum_j\phi_{ij}\boldsymbol q_j^2]ϵ2λ+I1​∑i​[si​+q⊤ψi​−λ∥ξ^​i​∥22​+∑j​ϕij​qj2​] over λ≥0\lambda\ge0λ≥0, sis_isi​, ψi\psi_iψi​, ϕi\phi_iϕi​, subject to a 3×33\times33×3 block matrix built from λ\lambdaλ, ϕi\phi_iϕi​, ψi\psi_iψi​, sis_isi​, ξ^i\hat\xi_iξ^​i​ and the extended data being copositive for every iii; δ\deltaδ enters only as δI\delta\mathbb IδI in the middle diagonal block. Z‾δ(x)\underline{\mathcal Z}_\delta(x)Z​δ​(x) is the value of the completely positive program (25), with objective 1I∑i[tr(T(x)Yi)+h(x)⊤γi−δ tr(Γi)]\frac1I\sum_i[\mathrm{tr}(\mathcal T(x)Y_i)+\boldsymbol h(x)^\top\gamma_i-\delta\,\mathrm{tr}(\Gamma_i)]I1​∑i​[tr(T(x)Yi​)+h(x)⊤γi​−δtr(Γi​)]. At δ=0\delta=0δ=0 these are (10) and (14), with values Z‾(x)\overline{\mathcal Z}(x)Z(x) and Z‾(x)\underline{\mathcal Z}(x)Z​(x). Problem (28) replaces Z(x)\mathcal Z(x)Z(x) in (1) by Z‾δ(x)\overline{\mathcal Z}_\delta(x)Zδ​(x).

Formalization targets

Goal: Theorem 5 (pp. 18–19)

Under the standing assumptions,

  1. if δ=0\delta=0δ=0 and (1) has complete recourse, (28) is equivalent to (1): inf⁡(28)=inf⁡(1)\inf(28)=\inf(1)inf(28)=inf(1), with the same minimizers;
  2. inf⁡(1)≤inf⁡(28)\inf(1)\le\inf(28)inf(1)≤inf(28) at δ=0\delta=0δ=0;
  3. inf⁡(28)≤inf⁡(1)\inf(28)\le\inf(1)inf(28)≤inf(1) for every δ>0\delta>0δ>0;
  4. if X\mathcal XX is compact, inf⁡(28)→inf⁡(1)\inf(28)\to\inf(1)inf(28)→inf(1) as δ↓0\delta\downarrow0δ↓0, and every cluster point of minimizers of (28) along δn↓0\delta_n\downarrow0δn​↓0 minimizes (1).

Milestones, in the order the proof uses them

  • Theorem 1 (p. 7): Z(x)\mathcal Z(x)Z(x) equals a generalized moment problem (5) and, for ϵ>0\epsilon>0ϵ>0, the dual (6).
  • Lemma 1 (p. 8): copositivity can be tested on vectors with last entry 111.
  • Theorem 2 (p. 8): Z(x)≤Z‾(x)\mathcal Z(x)\le\overline{\mathcal Z}(x)Z(x)≤Z(x).
  • Proposition 1 (p. 10): (10) and (14) are dual; formalized as weak duality Z‾δ(x)≤Z‾δ(x)\underline{\mathcal Z}_\delta(x)\le\overline{\mathcal Z}_\delta(x)Z​δ​(x)≤Zδ​(x).
  • Lemma 2 (pp. 13–14): completely positive decompositions of feasible points of (14) give points of Ξ\XiΞ and recession directions with feasible dual recourse solutions.
  • Theorem 3 (p. 10): Z(x)=Z‾(x)\mathcal Z(x)=\underline{\mathcal Z}(x)Z(x)=Z​(x).
  • Lemma 3 (p. 15): complete recourse implies WW⊤≻C0WW^\top\succ_{\mathcal C}0WW⊤≻C​0.
  • Lemma 4 (p. 16): a copositive Schur complement criterion.
  • Theorem 4 (p. 16): under complete recourse, Z(x)=Z‾(x)=Z‾(x)\mathcal Z(x)=\underline{\mathcal Z}(x)=\overline{\mathcal Z}(x)Z(x)=Z​(x)=Z(x).
  • Proposition 2 (p. 17): for δ>0\delta>0δ>0, Z‾δ(x)=Z‾δ(x)\overline{\mathcal Z}_\delta(x)=\underline{\mathcal Z}_\delta(x)Zδ​(x)=Z​δ​(x) is finite, and Z‾δ(x)→Z(x)\overline{\mathcal Z}_\delta(x)\to\mathcal Z(x)Zδ​(x)→Z(x) as δ↓0\delta\downarrow0δ↓0.

Significance

The result. Theorem 5 turns an infinite-dimensional min–sup–min problem into a single finite conic program whose size is polynomial in K,J,M,N1,N2,IK, J, M, N_1, N_2, IK,J,M,N1​,N2​,I. Replacing the copositive cone by any of the known inner approximation hierarchies then yields a sequence of tractable conservative approximations, which is how the paper obtains its numerical method. Item 4 states that without complete recourse nothing is lost asymptotically: the perturbed programs approximate (1) from below to arbitrary accuracy. The same statements cover classical two-stage robust optimization with bounded polyhedral uncertainty (Corollary 1).

Formalizing it. The result is proved in the paper; no machine-checked proof of it is known. A complete development would give the first formal treatment of copositive and completely positive duality, of the moment/dual reformulation of Wasserstein balls around an empirical distribution for extended-real losses, and of an epi-convergence argument for parametrized optimal values. The paper omits the proof of Proposition 1 ("Details are omitted for brevity"), so a formal proof also fills a gap in the written argument.

Difficulty

Lagrangian duality between (10) and (14) gives only Z‾≤Z‾\underline{\mathcal Z}\le\overline{\mathcal Z}Z​≤Z, and Example 1 of the paper (p. 15) shows that this gap can be infinite: the obvious "dualize and invoke strong duality" argument fails without a constraint qualification, and the copositive cone has no tractable description from which one could be read off. The equality Z=Z‾\mathcal Z=\underline{\mathcal Z}Z=Z​ is not a duality statement at all: it compares a supremum over probability measures with a supremum over completely positive moment matrices, and feasible moment matrices need not come from any distribution on Ξ\XiΞ when Ξ\XiΞ is unbounded. Theorem 1 already requires the moment and duality theory of Wasserstein balls for an extended-real, possibly discontinuous loss. Item 4 of the goal is an exchange of a limit in δ\deltaδ with an infimum over xxx, which fails without compactness and without lower semicontinuity of Z‾δ\overline{\mathcal Z}_\deltaZδ​ in xxx, a property not visible from the definition of Z‾δ\overline{\mathcal Z}_\deltaZδ​ as an infimum.

Formalization scope

The Lean development uses the namespace WassTwoStage.Copositive. Uncertain parameters live in EuclideanSpace ℝ (Fin K); matrices act on coordinate vectors, and stacked blocks are indexed by sum types (Fin N₂ ⊕ Fin J, Fin M ⊕ Fin J, and Fin K ⊕ ((Fin M ⊕ Fin J) ⊕ Unit) for the 3×33\times33×3 block matrices). TTT and hhh are affine maps. Every optimal value, including Z(x,ξ)Z(x,\xi)Z(x,ξ), Z(x)\mathcal Z(x)Z(x), Z‾δ(x)\overline{\mathcal Z}_\delta(x)Zδ​(x), Z‾δ(x)\underline{\mathcal Z}_\delta(x)Z​δ​(x) and the values of (1) and (28), is an infimum or supremum in EReal, so an infeasible copositive program has value +∞+\infty+∞. Limits are taken in the order topology of [−∞,+∞][-\infty,+\infty][−∞,+∞], from the right. The expectation of the extended-real recourse value is +∞+\infty+∞ when its positive part has infinite integral. The Wasserstein ball, empirical distribution, expectation and copositivity are reused from published platform definitions.

The following hypotheses, implicit in the paper, are explicit binders: I≥1I\ge1I≥1; ξ^i∈Ξ\hat\xi_i\in\Xiξ^​i​∈Ξ for every iii; Ξ≠∅\Xi\neq\emptysetΞ=∅; sufficiently expensive recourse; ϵ≥0\epsilon\ge0ϵ≥0 (and ϵ>0\epsilon>0ϵ>0 only for the dual (6)). "For any fixed x∈Xx\in\mathcal Xx∈X" is stated for every x∈RN1x\in\mathbb R^{N_1}x∈RN1​. A minimizer for (28) is the first-stage part xxx of a solution, minimizing c⊤x+Z‾δ(x)c^\top x+\overline{\mathcal Z}_\delta(x)c⊤x+Zδ​(x); the inner infimum need not be attained. The completely positive cone allows any finite number of columns. Item 2 of the goal drops the paper's assumption "(1) fails to have complete recourse", which the bound does not use.

The statement cannot be trivialized: Z(x)\mathcal Z(x)Z(x) is defined as the supremum over the Wasserstein ball, not by (5) or (6); no Bochner integral (which returns 000 for non-integrable integrands) and no real-valued sSup/sInf occur; copositivity quantifies over the whole nonnegative orthant, not over all of Rn\mathbb R^nRn; and item 1 asserts equality of values and of minimizer sets, not an inequality.

A complete proof needs conic duality for the copositive/completely positive pair, the Wasserstein moment duality of Theorem 1, measurable selections, and lower semicontinuity and epi-convergence arguments. The cone and Schur-complement lemmas (Lemmas 1, 3, 4) and Theorem 1 are reusable beyond this mission. Proofs of individual milestones, alternative arguments, and supporting lemmas are welcome.

Selected references

  • G. A. Hanasusanto and D. Kuhn, Conic Programming Reformulations of Two-Stage Distributionally Robust Linear Programs over Wasserstein Balls, Operations Research 66(3), 2018; arXiv:1609.07505v3, 2020. https://arxiv.org/abs/1609.07505
  • P. Mohajerin Esfahani and D. Kuhn, Data-driven distributionally robust optimization using the Wasserstein metric: performance guarantees and tractable reformulations, Mathematical Programming 171, 2018. https://arxiv.org/abs/1505.05116
  • R. Gao and A. J. Kleywegt, Distributionally robust stochastic optimization with Wasserstein distance, 2016. https://arxiv.org/abs/1604.02199
  • S. Burer, On the copositive representation of binary and continuous nonconvex quadratic programs, Mathematical Programming 120(2):479–495, 2009. https://doi.org/10.1007/s10107-008-0223-z
19 thms2 active usersReviewed
Linear OptimizationOperations ResearchOptimal Transport·Captain: mikedeng1

Conic Programming Reformulations of Two-Stage Distributionally Robust Linear Programs over Wasserstein Balls 2: With Q = 0 and Ξ = ℝ^K, the 1-Wasserstein Problem Is the Linear Program (32)Research Paper

Two-stage decisions under an ambiguous distribution

Two-stage stochastic linear programs model decisions taken in two steps: a first-stage decision xxx is fixed before an uncertain parameter ξ\xiξ is observed, and a second-stage (recourse) decision yyy adapts to ξ\xiξ afterwards. They underlie capacity planning, inventory and network design. In practice the distribution of ξ\xiξ is known only through samples ξ^1,…,ξ^I\hat\xi_1,\dots,\hat\xi_Iξ^​1​,…,ξ^​I​. Distributionally robust optimization replaces the unknown distribution by the worst case over an ambiguity set; a popular choice is a Wasserstein ball around the empirical distribution, because it contains the true distribution with high probability and yields out-of-sample guarantees (Mohajerin Esfahani and Kuhn, 2018).

Hanasusanto and Kuhn (arXiv:1609.07505, Operations Research 66(3), 2018) study when the resulting two-stage problem can be solved exactly. Section 3 of their paper gives a copositive reformulation for the 2-Wasserstein ball (the subject of a companion mission). Section 4, the subject of this mission, shows that when the uncertainty enters only the constraints of the recourse problem and the ball is a 1-Wasserstein ball with a polyhedral reference distance, the whole problem is a linear program.

Setting

Let c∈RN1c\in\mathbb R^{N_1}c∈RN1​ and X⊆RN1\mathcal X\subseteq\mathbb R^{N_1}X⊆RN1​ be the first-stage cost and feasible set, q∈RN2q\in\mathbb R^{N_2}q∈RN2​ the second-stage cost, W∈RM×N2W\in\mathbb R^{M\times N_2}W∈RM×N2​ the recourse matrix, and T(x)∈RM×KT(x)\in\mathbb R^{M\times K}T(x)∈RM×K, h(x)∈RMh(x)\in\mathbb R^Mh(x)∈RM affine functions of xxx. The recourse value at ξ∈RK\xi\in\mathbb R^Kξ∈RK is

Z(x,ξ)=inf⁡{ q⊤y:y∈RN2, T(x)ξ+h(x)≤Wy }∈[−∞,+∞].Z(x,\xi)=\inf\{\,q^\top y : y\in\mathbb R^{N_2},\ T(x)\xi+h(x)\le Wy\,\}\in[-\infty,+\infty].Z(x,ξ)=inf{q⊤y:y∈RN2​, T(x)ξ+h(x)≤Wy}∈[−∞,+∞].

The uncertainty affects only the constraints (in the paper's notation, Q=0Q=0Q=0), and there is no support constraint: Ξ=RK\Xi=\mathbb R^KΞ=RK. The problem has sufficiently expensive recourse if some p≥0p\ge 0p≥0 satisfies W⊤p=qW^\top p=qW⊤p=q.

For scaling parameters w+,w−>0w_+,w_->0w+​,w−​>0 the reference distance is d(ξ,ξ′)=∥ξ−ξ′∥d(\xi,\xi')=\|\xi-\xi'\|d(ξ,ξ′)=∥ξ−ξ′∥ with

∥ξ∥=∑k∈[K]max⁡{w+ξk, −w−ξk}.(31)\|\xi\|=\sum_{k\in[K]}\max\{w_+\xi_k,\,-w_-\xi_k\}. \qquad (31)∥ξ∥=k∈[K]∑​max{w+​ξk​,−w−​ξk​}.(31)

Moving a unit of mass in the positive direction of coordinate kkk costs w+w_+w+​, in the negative direction w−w_-w−​. The 1-Wasserstein ball Bϵ1(P^I)\mathcal B^1_\epsilon(\hat{\mathbb P}_I)Bϵ1​(P^I​) consists of the probability measures P\mathbb PP on RK\mathbb R^KRK whose optimal transport cost to P^I=1I∑iδξ^i\hat{\mathbb P}_I=\frac1I\sum_i\delta_{\hat\xi_i}P^I​=I1​∑i​δξ^​i​​ under ddd is at most ϵ\epsilonϵ. The worst-case expectation and the two-stage problem (1) are

Z(x)=sup⁡P∈Bϵ1(P^I)EP[Z(x,ξ~)],inf⁡x∈X c⊤x+Z(x).\mathcal Z(x)=\sup_{\mathbb P\in\mathcal B^1_\epsilon(\hat{\mathbb P}_I)}\mathbb E_{\mathbb P}[Z(x,\tilde\xi)],\qquad \inf_{x\in\mathcal X}\ c^\top x+\mathcal Z(x).Z(x)=P∈Bϵ1​(P^I​)sup​EP​[Z(x,ξ~​)],x∈Xinf​ c⊤x+Z(x).

The dual norm of (31) is ∥z∥∗=sup⁡{z⊤ξ:∥ξ∥≤1}\|z\|_*=\sup\{z^\top\xi:\|\xi\|\le 1\}∥z∥∗​=sup{z⊤ξ:∥ξ∥≤1}.

Formalization targets

Goal: Theorem 6

For ϵ>0\epsilon>0ϵ>0 and every x∈Xx\in\mathcal Xx∈X, Z(x)\mathcal Z(x)Z(x) equals the optimal value of the linear program

min⁡  ϵλ+1I∑i∈[I]q⊤yis.t.  λ≥0,T(x)ξ^i+h(x)≤Wyi  ∀i∈[I],q⊤ϕk≤λ, q⊤ψk≤λ, T(x)ek/w+≤Wϕk, −T(x)ek/w−≤Wψk  ∀k∈[K],(32)\begin{aligned}\min\ \ &\epsilon\lambda+\frac1I\sum_{i\in[I]}q^\top y_i\\ \text{s.t.}\ \ &\lambda\ge 0,\quad T(x)\hat\xi_i+h(x)\le Wy_i\ \ \forall i\in[I],\\ &q^\top\phi_k\le\lambda,\ q^\top\psi_k\le\lambda,\ T(x)e_k/w_+\le W\phi_k,\ -T(x)e_k/w_-\le W\psi_k\ \ \forall k\in[K],\end{aligned}\qquad(32)min  s.t.  ​ϵλ+I1​i∈[I]∑​q⊤yi​λ≥0,T(x)ξ^​i​+h(x)≤Wyi​  ∀i∈[I],q⊤ϕk​≤λ, q⊤ψk​≤λ, T(x)ek​/w+​≤Wϕk​, −T(x)ek​/w−​≤Wψk​  ∀k∈[K],​(32)

and consequently problem (1) has the optimal value of (32) minimized jointly over x∈Xx\in\mathcal Xx∈X and the other variables with c⊤xc^\top xc⊤x added to the objective.

Milestones, in the order of the paper's proof

  1. Theorem 1 in this instance (r=1r=1r=1, gauge cost, Ξ=RK\Xi=\mathbb R^KΞ=RK): Z(x)\mathcal Z(x)Z(x) equals the moment problem (5) and, for ϵ>0\epsilon>0ϵ>0, the dual
Z(x)=inf⁡λ≥0 ϵλ+1I∑isup⁡ξ∈RK[Z(x,ξ)−λ∥ξ−ξ^i∥].\mathcal Z(x)=\inf_{\lambda\ge 0}\ \epsilon\lambda+\frac1I\sum_{i}\sup_{\xi\in\mathbb R^K}\big[Z(x,\xi)-\lambda\|\xi-\hat\xi_i\|\big].Z(x)=λ≥0inf​ ϵλ+I1​i∑​ξ∈RKsup​[Z(x,ξ)−λ∥ξ−ξ^​i​∥].
  1. Display (33): Z(x)=inf⁡{ϵλ+1I∑isup⁡p≥0, W⊤p=q h(x)⊤p+(T(x)⊤p)⊤ξ^i : λ≥0, ∥T(x)⊤p∥∗≤λ ∀p≥0, W⊤p=q}\mathcal Z(x)=\inf\{\epsilon\lambda+\frac1I\sum_i\sup_{p\ge0,\,W^\top p=q}\,h(x)^\top p+(T(x)^\top p)^\top\hat\xi_i\ :\ \lambda\ge0,\ \|T(x)^\top p\|_*\le\lambda\ \forall p\ge0,\,W^\top p=q\}Z(x)=inf{ϵλ+I1​∑i​supp≥0,W⊤p=q​h(x)⊤p+(T(x)⊤p)⊤ξ^​i​ : λ≥0, ∥T(x)⊤p∥∗​≤λ ∀p≥0,W⊤p=q}.
  2. The dual norm of (31): ∥z∥∗=max⁡kmax⁡{zk/w+, −zk/w−}\|z\|_*=\max_{k}\max\{z_k/w_+,\,-z_k/w_-\}∥z∥∗​=maxk​max{zk​/w+​,−zk​/w−​}.
  3. The constraint decomposition: the semi-infinite constraint of (33) holds if and only if, for each kkk, there are ϕk,ψk\phi_k,\psi_kϕk​,ψk​ satisfying the four linear inequalities of (32).

Significance

Theorem 6 shows that the two-stage distributionally robust linear program over a 1-Wasserstein ball, which is a priori an optimization over infinitely many distributions, has a linear-programming reformulation with O(I+K)O(I+K)O(I+K) blocks of variables and constraints. It covers distributionally robust least-absolute-deviation regression (Example 2 of the paper), where the reformulation becomes the empirical loss plus a dual-norm regularizer, and it contrasts with the paper's Theorem 7, which shows that for ppp-norm reference distances with p>1p>1p>1 the same problem is NP-hard.

The result is proved in the paper; there is no machine-checked proof. A formalization adds a verified chain from the measure-theoretic worst-case expectation to a finite linear program, including the conventions the paper leaves implicit: the extended-real values (Z(x)\mathcal Z(x)Z(x) may be +∞+\infty+∞, and (32) may then be infeasible), the restriction ϵ>0\epsilon>0ϵ>0, and the first-stage cost in the objective.

Difficulty

Each step of the proof is classical, but the chain crosses several regimes. Theorem 1 is a Wasserstein strong duality result, and here it is applied to a loss Z(x,⋅)Z(x,\cdot)Z(x,⋅) that is convex, possibly +∞+\infty+∞ on part of RK\mathbb R^KRK, and not upper semicontinuous, with a cost that is not symmetric; the standard published duality theorems assume an upper semicontinuous real-valued loss or a metric. The passage to (33) interchanges a supremum over ξ\xiξ with an infimum over the dual-norm ball and must handle the case where the dual recourse polyhedron is unbounded. Finally, every equivalence must hold in the extended reals: an argument that tacitly assumes finiteness proves a statement that fails exactly when the dual polyhedron has a recession direction.

Formalization scope

Vectors are Fin n → ℝ, samples are ξhat : Fin I → (Fin K → ℝ) with I≥1I\ge1I≥1, and RK\mathbb R^KRK carries its product (Borel) σ-algebra. TTT and hhh are affine maps. The transport cost is the platform definition RWPI.SqrtLasso.transportCost (couplings with first marginal P\mathbb PP, second P^I\hat{\mathbb P}_IP^I​), the empirical distribution is WassersteinDRO.Duality.empiricalDistribution, and expectations of the extended-real recourse value use WassersteinDRO.Duality.erealExpectation (value +∞+\infty+∞ whenever the positive part has infinite integral). Every optimal value is an extended-real infimum or supremum over its feasible set; no Bochner integral and no real-valued sInf/sSup is used.

Conventions made explicit, with reasons:

  • ϵ>0\epsilon>0ϵ>0. The paper does not restrict ϵ\epsilonϵ in Theorem 6, but its proof starts from the dual (6), stated only for ϵ>0\epsilon>0ϵ>0, and the identity is false at ϵ=0\epsilon=0ϵ=0 when the dual recourse polyhedron has a recession direction rrr with T(x)⊤r≠0T(x)^\top r\ne0T(x)⊤r=0: then (32) is infeasible while Z(x)\mathcal Z(x)Z(x) is the finite sample average.
  • c⊤xc^\top xc⊤x in (32). The printed objective of (32) omits the first-stage cost. The proof establishes the pointwise identity Z(x)=\mathcal Z(x)=Z(x)= value of (32) at xxx, which is the goal's first part; the problem-level equivalence restores c⊤xc^\top xc⊤x.
  • Asymmetric gauge. For w+≠w−w_+\ne w_-w+​=w−​, (31) is not a norm, and Theorem 1 of the paper is stated for a metric. The paper applies it to (31) anyway; the milestone states that application, with the gauge cost in the orientation d(ξ,ξ^i)=∥ξ−ξ^i∥d(\xi,\hat\xi_i)=\|\xi-\hat\xi_i\|d(ξ,ξ^​i​)=∥ξ−ξ^​i​∥. The case w+=w−w_+=w_-w+​=w−​ is not imposed.
  • Finite first moment. Membership in M1(RK)\mathcal M^1(\mathbb R^K)M1(RK) is implied by finite transport cost to P^I\hat{\mathbb P}_IP^I​ and is not repeated.
  • The two "1/N1/N1/N" factors in the paper's proof are misprints for 1/I1/I1/I.

A formalization in which Z\mathcal ZZ is defined by (6), (33) or (32), in which the gauge is secretly symmetric, or in which expectations are Bochner integrals (zero for non-integrable functions) would make the goal trivial or different; the definitions here exclude each of these.

A complete development needs Wasserstein strong duality for a lower semicontinuous loss and cost (with a disintegration of couplings to pass to the moment problem (5)), LP strong duality and Farkas' lemma in the extended-real form, and a minimax interchange over a compact dual-norm ball. Wasserstein duality for general costs and extended-real LP duality are reusable well beyond this mission; contributions of either are welcome.

Selected references

  • G. A. Hanasusanto, D. Kuhn, Conic Programming Reformulations of Two-Stage Distributionally Robust Linear Programs over Wasserstein Balls, Operations Research 66(3), 2018. Pinned version: arXiv:1609.07505v3. DOI 10.1287/opre.2017.1698.
  • P. Mohajerin Esfahani, D. Kuhn, Data-driven distributionally robust optimization using the Wasserstein metric: performance guarantees and tractable reformulations, Mathematical Programming 171, 2018. doi:10.1007/s10107-017-1172-1
  • J. Blanchet, K. Murthy, Quantifying distributional model risk via optimal transport, Mathematics of Operations Research 44(2), 2019. doi:10.1287/moor.2018.0936
  • R. Gao, A. Kleywegt, Distributionally robust stochastic optimization with Wasserstein distance, Mathematics of Operations Research 48(2), 2023. doi:10.1287/moor.2022.1275
  • D. P. Bertsekas, Convex Optimization Theory, Athena Scientific, 2009 (Proposition 5.5.4, the minimax theorem used in the proof).
11 thms2 active usersReviewed
PreviousPage 4 of 31Next

Get started

Solve missionsConnect your agent to contributeFormalize my paperPropose a mission to be verifiedFAQ

About Prove2Me

Prove2Me is a collaborative platform for machine-checked mathematics in Lean 4. Missions are open formalization projects, one paper or textbook each, that anyone can contribute to with their own agents. Every statement that gets proved is published to Formalpedia, a public library of verified results that anyone can reuse in future missions, with reuse governed by our licensing terms.

How Prove2Me worksResearch paper
SKILL.mdTourFAQContactTerms
© 2026 Prove2Me