On the Douglas-Rachford splitting method and the proximal point algorithm for maximal monotone operators
Summary (2 min read)
I. Introduction
- The theory of maximal set-valued monotone operators (see, for example, [4] ) provides a powerful general framework for the study of convex programming and variational inequalities.
- Some preliminary versions of their results have also appeared in [10] .
- This paper is organized as follows: Section 2 introduces the basic theory of monotone operators in Hilbert space, while Section 3 proves the convergence of a generalized form of the proximal point algorithm.
- Section 4 discusses Douglas-Rachford splitting, showing it to be a special case of the proximal point algorithm by means of a specially-constructed splitting operator.
2. Monotone operators
- The following lemma summarizes some well-known properties of firmly nonexpansive operators.
- The authors now give a critical theorem, The "only if" part of the following theorem has been well known for some time [48] , but the "if" part, just as easily obtained, appears to have been obscure.
- The purpose here is to stress the complete symmetry that exists between monotone operators and (full-domained) firmly nonexpansive operators over any Hilbert space. [37] , but is not identical (Minty did not use the concept of firm nonexpansiveness; but see also [28] ).
- In the case that T is the subdifferential map Of of a convex function f, zer(T) is the set of all global minima off.
- The zeroes of a monotone operator precisely coincide with the fixed points of its resolvents:.
3. A generalized proximal point algorithm
- Gol'shtein and Tret'yakov also allow resolvents to be evaluated approximately, but, unlike Rockafellar, do not allow the stepsize c to vary with k, restrict Y{ to be finitedimensional, and do not consider the case in which zer(T)=0.
- The following theorem effectively combines the results of Rockafellar and Gol'shtein-Tret'yakov.
- The notation "-~" denotes convergence in the weak topology on Y(, where "--*" denotes convergence in the strong topology induced by the usual norm (x, x} ~/2.
Vk>~O.
- In at least one real example [11, Section 7.2.3], using the generalized Douglas-Rachford splitting method with relaxation factors.
- Pk other than 1 has been shown to converge faster than regular Douglas-Rachford splitting.
- This example involved highly parallel algorithm for linear programming which will be described in a later paper.
- Thus, the inclusion of over-relaxation factors is of some practical significance.
- In addition, the convergence of Douglas-Rachford splitting with approximate calculation of resolvents had not been formerly established.
5. Some interesting special eases
- The authors can now state a new variation o~ the alternating direction method of multipliers for (P):.
- Then if (P) has a Kuhn-Tucker pair, {x k} converges to a solution of (P) and {pk} converges to a solution of the dual problem (D).
6. Concerning finite termination
- All operators of this form are staircase (in fact, for any y 6 V ~, 8(y) may be taken arbitrarily large).
- Define the following linear subspaces of E2:.
Did you find this useful? Give us your feedback
Citations
17,433 citations
Cites background from "On the Douglas-Rachford splitting m..."
...Viewed as a function of v, the righthand side is denoted proxf,ρ(v) and is called the proximity operator of f with penalty ρ [Mor62]....
[...]
4,487 citations
Cites background from "On the Douglas-Rachford splitting m..."
...Let us focus here on the Douglas-Rachford splitting algorithm (DRS) [19], which is known to be a special case of the proximal point algorithm (23), see [10]....
[...]
3,085 citations
Cites background from "On the Douglas-Rachford splitting m..."
...According to the theoretical results in [41], two conditions are sufficient (but may not necessary) for Algorithm 1 to converge: the first condition is that the dictionary matrixA is of full column rank; the second one is that the optimality gap produced in each iteration step is monotonically decreasing, namely th e error...
[...]
...For the monotonically decreasing condit ion, although it is not easy tostrictly prove it, the convexity of the Lagrange function could guarantee its validity to some exte nt [41]....
[...]
2,095 citations
References
5,597 citations
"On the Douglas-Rachford splitting m..." refers background in this paper
...An interesting presentation can be found in [15], and [3] provides a relative accessible exposition....
[...]
3,447 citations
3,238 citations