Motivation
The Douglas–Rachford algorithm is one of the most widely used splitting methods for minimizing a sum f+g of two convex functions, or more generally for finding a zero of a sum A+B of two maximally monotone operators. It touches each summand only through its proximal mapping (resolvent), which is why it underlies ADMM and a large share of first-order solvers in signal processing, statistics and operations research.
Its convergence theory is classical in the consistent case, when A+B has a zero: then the shadow sequence JATnx, the sequence actually used as the approximate solution, converges weakly to a zero of A+B (Lions and Mercier 1979; Svaiter 2011). In practice, however, problems are frequently inconsistent: the constraint sets do not meet, or domf∩domg=∅, so that minf+g has no solution in the usual sense. Then the governing sequence Tnx diverges, and the question is what the shadows do. Infeasibility detection in ADMM-based conic solvers (Banjac, Goulart, Stellato and Boyd 2019; Liu, Ryu and Yin 2019) is one place where this question arises.
Timeline (as recounted in §1.2 of the paper).
- 2004: Bauschke, Combettes and Luke treat two closed convex sets U,V: the shadows PUTnx are bounded and their weak cluster points minimize ιU+ιV(⋅−v).
- 2014: Bauschke, Hare and Moursi set up a static framework for generalized solutions of inconsistent monotone inclusions, the normal problem.
- 2016: the same authors relate ran(Id−T) to the domains and ranges of A and B.
- 2016: Bauschke and Moursi prove strong convergence, with linear rates, for two closed affine subspaces; with Dao, one affine subspace and one closed convex set; and, via a new Fejér monotonicity principle, full weak convergence of the shadows for two closed convex sets.
- 2020: Bauschke and Moursi treat f=ιU for a closed linear subspace U and general g∈Γ0(X), under the assumption vR=0.
- This paper (arXiv 2021; Mathematics of Operations Research 2024): weak and value convergence of the shadows for two general f,g∈Γ0(X), and vR=0 shown to be necessary.
Setting
Let X be a real Hilbert space and A,B:X⇉X maximally monotone operators. The resolvent of A is JA=(Id+A)−1, a single-valued map on all of X; RA=2JA−Id is the reflected resolvent, and JA−1=Id−JA. The Douglas–Rachford operator is
T=Id−JA+JBRA.
The minimal displacement vector of T is v=Pran(Id−T)(0), the element of least norm in the closure of ran(Id−T). With D=domA−domB and R=ranA+ranB, put vD=PD(0) and vR=PR(0). The paper works under the standing assumption
ran(Id−T)=D∩R=D∩R,(12)
which holds automatically in finite dimensions for subdifferential operators. The normal problem is to find x with 0∈−v+Ax+B(x−v); its solution set is Z.
In the optimization setting, f,g∈Γ0(X) are proper, convex and lower semicontinuous with values in ]−∞,+∞], (A,B)=(∂f,∂g), the resolvents become the proximal mappings Pf,Pg, and T=Id−Pf+PgRf. The normal problem becomes minxf(x)+g(x−v), which may have solutions even when domf∩domg=∅. The primal shadows are PfTnx and PgRfTnx.
Formalization targets
Goal: Theorem 6.9
Let f,g∈Γ0(X), assume (12), and suppose that 0∈domf∗+domg∗, that v∈ran(Id−T) and that zer(∂f+∂g(⋅−v))=∅. Let μ=minx(f(x)+g(x−v)) and x∈X. Then there is zˉ such that
PfTnx⇀zˉ,PgRfTnx⇀zˉ−v,
zˉ minimizes f+g(⋅−v) and zˉ−v minimizes f(⋅+v)+g, and
f(PfTnx)+g(PgRfTnx)→f(zˉ)+g(zˉ−v)=μ,
with f(PfTnx)→f(zˉ) and g(PgRfTnx)→g(zˉ−v).
Milestones
The milestones follow the paper's chain: the resolvent identities (Lemma 2.2) and Proposition 2.7; the splitting v=vD+vR (Proposition 3.3); the static results on Fix(v+T) and Z (Lemma 4.1, Proposition 4.2, Theorem 4.3); the orbit dynamics (Proposition 5.3, Theorem 5.5, Corollary 5.7); and §6: vR=0 from the conjugate condition, Z as the minimizers of the normal problem (Proposition 6.2), the prox lemma and value estimate (Lemma 6.3, Proposition 6.4), and convergence in value and weakly (Propositions 6.5, 6.6).
Significance
The result. Theorem 6.9 says that running the Douglas–Rachford algorithm on an inconsistent convex problem is still meaningful: the computed shadows converge to a solution of the normal problem, the closest consistent shift of the original problem, and the objective values converge to its optimal value. Practitioners can thus run the method without first deciding feasibility. The static results of §§3–4 (the location of vD, vR, the description of Z when vR=0) are general facts about pairs of maximally monotone operators and are reusable outside the algorithm.
Formalizing it. The theorem is proved on paper; no part of it is machine-checked. A formalization would be the first verified convergence result for a splitting method in the inconsistent case, and it requires a library of Hilbert-space monotone operator theory (resolvents, Minty's theorem, Fejér monotonicity, Opial-type arguments, the Brøndsted–Rockafellar theorem) that Mathlib currently lacks.
Difficulty
When v=0 the governing sequence Tnx diverges, roughly like −nv, so the Fejér monotonicity argument of the consistent case, which rests on a fixed point of T, has nothing to anchor to: T has no fixed points. The natural first idea, to apply the consistent theory to the shifted pair (−v+A,B(⋅−v)), does not by itself say anything about the shadows of the original iteration, and in general the primal shadows need not even be bounded (by Theorem 5.5 of the paper, when Fix(v+T)=∅ they are bounded exactly when vR=0). Identifying weak cluster points as minimizers requires an asymptotic value estimate rather than a fixed-point characterization, and the uniqueness of the weak limit needs a Fejér-type property in the product space X×X. Lower semicontinuity gives only liminf bounds on f and g separately, so convergence of each value is a separate claim.
Formalization scope
X is a real inner product space with CompleteSpace X. Operators are X → Set X; maximal monotonicity and resolvents come from published definitions. Resolvents and proximal maps are maps J : X → X characterized by x−Jx∈A(Jx) for all x, which is J=(Id+A)−1; JA−1 and Pf∗ are written Id−JA and Id−Pf, by the inverse resolvent identity. The projections v,vD,vR are least-norm points of the respective closures. Functions are X → EReal in the published class Γ0; the subdifferential is empty off the domain; every inequality is stated as an EReal sum, without extended-real subtraction. μ is the EReal infimum of f+g(⋅−v). Weak convergence is ⟨un,w⟩→⟨z,w⟩ for all w. Norms on X×X are written ∥a∥2+∥b∥2.
Restored hypotheses and corrections, disclosed on every item: the standing assumption (12) of p. 6 is an explicit hypothesis of every statement from §3 on; the §6 statements carry the section's standing assumptions (vR=0, and Z=∅ from (74) on). The page prints μ:=minx(f(x+v)−g(x)) in Theorem 6.9; the statement uses minx(f(x)+g(x−v)), as in Proposition 6.5 and §1.1. The goal does not assume vR=0, Z=∅ or boundedness of the shadows: these are consequences, and a version assuming them, or leaving the proximal maps or v unconstrained, would trivialize the statement.
The development needs Minty's theorem, the Brøndsted–Rockafellar density domf∗=ran∂f, closedness and convexity of ran(Id−T), Opial's lemma, and maximal monotonicity of subdifferentials of Γ0 functions. These are reusable throughout convex analysis and are welcome as separate contributions. Corollary 6.10 (dual shadows), Proposition 6.7 and the examples are not part of the mission.
Selected references
- H. H. Bauschke and W. M. Moursi, On the Douglas–Rachford algorithm for solving possibly inconsistent optimization problems, Mathematics of Operations Research, 2024; preprint arXiv:2106.11547v1 (2021), the version formalized here. https://arxiv.org/abs/2106.11547
- H. H. Bauschke, P. L. Combettes and D. R. Luke, Finding best approximation pairs relative to two closed convex sets in Hilbert spaces, Journal of Approximation Theory 127 (2004), 178–192.
- H. H. Bauschke, W. L. Hare and W. M. Moursi, Generalized solutions for the sum of two maximally monotone operators, SIAM Journal on Control and Optimization 52 (2014), 1034–1047.
- H. H. Bauschke, W. L. Hare and W. M. Moursi, On the range of the Douglas–Rachford operator, Mathematics of Operations Research 41 (2016), 884–897.
- H. H. Bauschke and W. M. Moursi, On the Douglas–Rachford algorithm, Mathematical Programming (Series A) 164 (2017), 263–284.
- H. H. Bauschke and W. M. Moursi, On the behaviour of the Douglas–Rachford algorithm for minimizing a convex function subject to a linear constraint, SIAM Journal on Optimization 30 (2020), 2559–2576.
- W. M. Moursi, The Douglas–Rachford operator in the possibly inconsistent case: static properties and dynamic behaviour, Ph.D. thesis, University of British Columbia, 2016. https://doi.org/10.14288/1.0340501
- P.-L. Lions and B. Mercier, Splitting algorithms for the sum of two nonlinear operators, SIAM Journal on Numerical Analysis 16 (1979), 964–979.
- B. F. Svaiter, On weak convergence of the Douglas–Rachford method, SIAM Journal on Control and Optimization 49 (2011), 280–287.
- H. H. Bauschke and P. L. Combettes, Convex Analysis and Monotone Operator Theory in Hilbert Spaces, 2nd edition, Springer, 2017.