A Single-Loop Algorithm for Decentralized Bilevel Optimization: O(1/K) Stationarity and Consensus Rates of SLDBO without Heterogeneity AssumptionsResearch Paper
Bilevel problems on a network
Bilevel optimization chooses an upper-level variable to minimize an objective that depends on the solution of a lower-level problem. Hyperparameter tuning and meta-learning are the standard examples (Franceschi et al. 2018; Ghadimi and Wang 2018). In the decentralized version the data are split among agents. Each agent sees only its own upper- and lower-level functions and , and it can talk only to its neighbours in a communication network. The paper's example is hospitals tuning a regularization weight of a logistic-regression model. No hospital may share its patients' data, but the hyperparameter has to be good for the pooled data.
Decentralized bilevel algorithms before this paper converged only under a heterogeneity assumption, a bound on how much the agents' functions may differ. DSBO (Chen, Huang and Ma 2024) assumes i.i.d. local data, MA-DSBO (Chen et al. 2023) a uniform bound on , and SLAM (Lu et al. 2022) Lipschitz-type closeness of the local Hessians. Such assumptions fail exactly when the agents' data differ, which is the case that motivates decentralization in the first place. Dong, Ma, Yang and Yin propose SLDBO, a single-loop method that needs no such assumption, and prove an rate for it.
Setting
There are agents. Agent holds . Write and . Problem (6) is
Assumption 2.1 gives constants and . Each is -strongly convex. is -Lipschitz with an -Lipschitz gradient. is twice differentiable with an -Lipschitz gradient and an -Lipschitz Hessian. Under these conditions is well defined and is differentiable, with the hypergradient (7)
Here are the partial gradients, is the Hessian of in , and is the Jacobian of in .
Assumption 2.2. The agents communicate through a nonnegative, symmetric, doubly stochastic mixing matrix , and bounds the contraction of away from consensus.
SLDBO (Algorithm 1) keeps three local iterates per agent. Agent computes the local directions , and at . It feeds them into gradient-tracking variables , which estimate the network averages of the directions. It then takes one mixed step on each iterate with constant step sizes . The step on is followed by the projection onto the ball of radius , which contains every . All agents start from common points with (BOX 1). Bars denote averages over agents, e.g. .
Formalization targets
Goal: Theorem 3.1
The step sizes have thresholds depending only on : first , then given , then given . Below these thresholds, every instance with bounded below has a constant such that for all
The goal fixes no explicit step sizes and no explicit constant ; it asserts the rate.
Milestones
These are the paper's own statements, in attack order:
- the hypergradient formula (7);
- the mixing and projection lemmas A.1–A.2;
- the tracking identity (25);
- the consensus recursions A.3–A.4 with Remarks A.1–A.2;
- the bounds A.5–A.7 with Remark A.3 (A.5 and A.7 are included as supporting statements rather than milestones);
- the Lipschitz continuity of and (A.11) and of ;
- the descent inequality for (A.8);
- the contractions of and (A.9–A.10).
Significance
The theorem shows that a single-loop decentralized method reaches the stationarity rate known for centralized single-loop bilevel methods (Ji et al. 2022; Dagréou et al. 2022). Each iteration needs communication rounds and one Hessian- and one Jacobian-vector product, with no heterogeneity bound. The consensus clauses add that the agents agree at the same rate.
The result is proved in the paper; nothing about it is formalized. The mission adds three things. First, a machine-checked statement with explicit conventions. Second, a check of the constants. The printed in (8) has a term that is not homogeneous in the scale of , and the explicit step-size bounds (62) do not make the Lyapunov coefficient nonnegative (checked numerically). The goal is therefore stated in a form the argument supports. Third, reusable lemmas for gradient-tracking analyses: the mixing-matrix contraction, the projected-consensus inequality, the tracking identity and the hypergradient formula.
Difficulty
The obvious analysis of a decentralized SOBA-type method bounds the change of the directions and between iterations by the change of the iterates. Those directions contain . Without control of , their variation is bounded only by assuming the local Hessians are close, which is the heterogeneity assumption. The projection removes the need for it. The price is that no longer follows the averaged direction, so the consensus and error recursions for have to be derived separately. All of these recursions are then combined into one Lyapunov function (15) whose coefficients must be balanced against three coupled step sizes. Most of the work, and the published constants' main failure, is in that balancing.
Formalization scope
Vectors live in EuclideanSpace ℝ (Fin d). Lipschitz conditions in use , never Lean's sup norm on a product. The model and the added hypotheses:
- Hessian condition. It is imposed on the two blocks and in the operator norm. This is weaker than the paper's Frobenius-Lipschitz full Hessian.
- Mixing rate. is any bound of on ; the paper's eigenvalue is the least one.
- Lower-level solution. is a given map with minimizing for every .
- Hypergradient. is defined through
ContinuousLinearMap.inverse, and is the formula (7). A milestone proves that (7) is the gradient of . - Algorithm. Algorithm 1 with BOX 1 is one deterministic function
run. Its state at index holds . - Rate. "" is one constant , fixed after the instance and before ; the minimum over is an existential .
- bounded below. This is added to the goal. Without it the claim fails: and satisfy Assumption 2.1, yet .
- Lemma A.5 is posed for . At it would claim .
- Lemmas A.9 and A.10 carry the two bounds of (62) their proofs use, and .
- is , which is (8) with corrected to .
- Step sizes. The goal is stated with nested thresholds instead of (62).
The goal would become trivial if the thresholds were chosen after the instance or after , and false without bounded below; the statement rules both out. All hypotheses are satisfiable: a two-agent instance with and is checked in Lean.
A complete development needs the implicit-function theorem for , the descent lemma, strong-convexity inequalities for gradient steps, and the gradient-tracking bookkeeping. The mixing-matrix and tracking lemmas are reusable for other gradient-tracking methods. Proofs of any milestone, and repaired explicit step-size conditions, are welcome.
Selected references
- Y. Dong, S. Ma, J. Yang, C. Yin, A Single-Loop Algorithm for Decentralized Bilevel Optimization, Mathematics of Operations Research, 2025; arXiv:2311.08945v3. https://arxiv.org/abs/2311.08945
- M. Dagréou, P. Ablin, S. Vaiter, T. Moreau, A framework for bilevel optimization that enables stochastic and global variance reduction algorithms, NeurIPS 35, 2022. https://arxiv.org/abs/2201.13409
- S. Ghadimi, M. Wang, Approximation methods for bilevel programming, arXiv:1802.02246, 2018. https://arxiv.org/abs/1802.02246
- X. Chen, M. Huang, S. Ma, Decentralized bilevel optimization, Optimization Letters, 2024. https://arxiv.org/abs/2206.05670
- X. Chen, M. Huang, S. Ma, K. Balasubramanian, Decentralized stochastic bilevel optimization with improved per-iteration complexity, ICML 2023. https://arxiv.org/abs/2210.12839
- S. Lu, S. Zeng, X. Cui, M. Squillante, L. Horesh, B. Kingsbury, J. Liu, M. Hong, A stochastic linearized augmented Lagrangian method for decentralized bilevel optimization, NeurIPS 35, 2022. https://proceedings.neurips.cc/paper_files/paper/2022 (pp. 30638–30650)
- K. Ji, M. Liu, Y. Liang, L. Ying, Will bilevel optimizers benefit from loops, NeurIPS 35, 2022. https://arxiv.org/abs/2205.14224
- S. Pu, A. Nedić, Distributed stochastic gradient tracking methods, Mathematical Programming 187, 409–457, 2021. https://arxiv.org/abs/1805.11454
- L. Franceschi, P. Frasconi, S. Salzo, R. Grazzi, M. Pontil, Bilevel programming for hyperparameter optimization and meta-learning, ICML 2018. https://arxiv.org/abs/1806.04910