HAL Id: hal-00794044
https://hal.archives-ouvertes.fr/hal-00794044
Submitted on 25 Feb 2013
HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entic research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.
L’archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diusion de documents
scientiques de niveau recherche, publiés ou non,
émanant des établissements d’enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.
Primal-dual splitting algorithm for solving inclusions
with mixtures of composite, Lipschitzian, and
parallel-sum type monotone operators
Patrick Louis Combettes, Jean-Christophe Pesquet
To cite this version:
Patrick Louis Combettes, Jean-Christophe Pesquet. Primal-dual splitting algorithm for solving in-
clusions with mixtures of composite, Lipschitzian, and parallel-sum type monotone operators. Set-
Valued and Variational Analysis, Springer, 2012, 20 (2), pp.307-330. �10.1007/s11228-011-0191-y�.
�hal-00794044�
Primal-dual splitting algorithm for solving inclusions with
mixtures of composite, Lipschitzian, and parallel-sum type
monotone operators
∗
Patrick L. Combettes
1
and Jean-Christophe Pesquet
2
1
UPMC Universit´e Paris 06
Laboratoire Jacques-Louis Lions – UMR CNRS 7598
75005 Paris, France
plc@math.jussieu.fr
2
Universit´e Paris-Est
Laboratoire d’Informatique Gaspard Monge – UMR CNRS 8 049
77454 Marne la Vall´ee Cedex 2, France
jean-christophe.pesquet@univ-paris-est.fr
Abstract
We propose a primal-dual splitting algorithm for solving monotone inclusions involving a mixture
of sums, linear compositions, and parallel sums of set-valued a nd Lipschitzian operators. An
important feature of the algorithm is that the Lipschitzian operators present in the formulation
can be processed individually via explicit steps, while the set-valued operators are processed
individually via their resolvents. In addition, the algorithm is highly parallel in that most of its
steps can be executed simultaneously. This work brings together and notably extends various
types of structured monotone inclusion problems and their solutio n methods. The application to
convex minimization problems is given special attention.
Keywords maximal mon otone operator, monotone inclusion, nonsmooth convex optimization,
parallel sum, set-valued duality, splitting algorithm
Mathematics Subject Classifications (2010) 47H05, 49M29, 49M27, 90C25, 49N15.
∗
Contact author: P. L. Combettes, plc@math.jussieu.fr, phone: +33 1 4427 6319, fax: +33 1 4427 7200. This
work was supported by the Agence Nationale de la Recherche under grants ANR-08-BLAN-0294-02 and ANR-09-
EMER-004-03.
1
1 Introduction
Duality theory occupies a central place in classical optimization [
19, 24, 33, 40, 41]. Since the
mid 1960s it has expanded in various directions, e.g., variational inequalities [
2, 17, 21, 23, 26, 34],
minimax and saddle point problems [
27, 29, 32, 39], and, from a more global perspective, monotone
inclusions [
5, 9, 10, 16, 31, 37, 38]. In the present paper, we propose an algorithm for solving the
following structured duality framework for monotone inclusions that encompasses the above cited
works. In this formulation, we denote by B D the parallel sum of two set-valued operators B and
D (see (
2.5)). This operation plays a central role in convex analysis and monotone operator theory.
In p articular, B D can be seen as a regularization of B by D, and is natur ally connected to
addition through duality since (B + D)
−1
= B
−1
D
−1
. It is also strongly related to the infimal
convolution of functions through subdifferentials. We refer the reader to [
8, 28, 35, 36, 43] and the
references therein for background on the parallel sum.
Problem 1.1 Let H be a real Hilbert space, let z ∈ H, let m be a strictly positive integer, let
A: H → 2
H
be maximally monotone, and let C : H → H be monotone and µ-Lipschitzian for some
µ ∈ ]0, +∞[. For every i ∈ {1, . . . , m}, let G
i
be a real Hilber t space, let r
i
∈ G
i
, let B
i
: G
i
→ 2
G
i
be
maximally monotone, let D
i
: G
i
→ 2
G
i
be monotone and such that D
−1
i
is ν
i
-Lipschitzian, for some
ν
i
∈ ]0, +∞[, an d su ppose that L
i
: H → G
i
is a nonzero bounded linear operator. The problem is to
solve the primal inclusion
find
x ∈ H such that z ∈ Ax +
m
X
i=1
L
∗
i
(B
i
D
i
)(L
i
x − r
i
)
+ Cx, (1.1)
together with the dual inclusion
find v
1
∈ G
1
, . . . , v
m
∈ G
m
such that (∃ x ∈ H)
(
z −
P
m
i=1
L
∗
i
v
i
∈ Ax + Cx
(∀i ∈ {1, . . . , m})
v
i
∈ (B
i
D
i
)(L
i
x − r
i
).
(1.2)
Problem 1.1 captures and extends various existing problem formulations. Here are some examples
that illustrate its versatility and the breadth of its scope.
Example 1.2 In Problem 1.1 set
m = 1, C : x 7→ 0, and D
1
: y 7→
(
G
1
, if y = 0;
∅, if y 6= 0.
(1.3)
Then we recover a duality framework investigated in [
10, 16 , 37, 38], namely (we drop the subs cr ipt
‘1’ for br evity),
find (x, v) ∈ H ⊕ G such that
(
z ∈ A
x + L
∗
B(Lx − r)
−r ∈ −L
A
−1
(z − L
∗
v)
+ B
−1
v.
(1.4)
Example 1.3 In Example
1.2, let G = H, r = z = 0, and L = Id . Then we obtain th e duality
setting of [
5, 31], i.e.,
find (
x, u) ∈ H ⊕ H such that
(
0 ∈ A
x + Bx
0 ∈ −A
−1
(−u) + B
−1
u.
(1.5)
2
The special case of variational inequalities was first treated in [34].
Example 1.4 In Example
1.2, let A and B be the s ubdifferentials of lower semicontinuous convex
functions f : H → ]−∞, +∞] and g : G → ]−∞, +∞], respectively. Then, under suitable constraint
qualification, we obtain the classical Fench el-Rockafellar duality framework [40], i.e.,
minimize
x∈H
f(x) + g(Lx − r) − hx | zi
minimize
v∈G
f
∗
(z − L
∗
v) + g
∗
(v) + hv | ri.
(1.6)
Example 1.5 In Problem
1.1, set C : x 7→ 0, z = 0, and (∀i ∈ {1, . . . , m}) G
i
= H, r
i
= 0, L
i
= Id ,
and D
i
= ρ
−1
i
Id , where ρ
i
∈ ]0, +∞[. Then it follows from [
8, Proposition 23.6(ii)] that, for every
i ∈ {1, . . . , m}, B
i
D
i
= (Id −J
ρ
i
B
i
)/ρ
i
=
ρ
i
B
i
is the Yosida approximation of ind ex ρ
i
of B
i
. Thus,
(1.1) reduces to
find
x ∈ H such that 0 ∈ Ax +
m
X
i=1
ρ
i
B
i
x. (1.7)
This primal problem is investigated in [
13, Section 6.3]. In the case when m = 1, we obtain the
primal-dual problem (we drop the subscript ‘1’ for brevity)
find (
x, u) ∈ H ⊕ H such that
(
0 ∈ A
x +
ρ
Bx
0 ∈ −A
−1
(−u) + B
−1
u + ρu
(1.8)
investigated in [
9].
Example 1.6 In Problem
1.1, set m = 1, G
1
= G, L
1
= L, z = 0, and r
1
= 0, and let A and B
1
be the
subdifferentials of lower semicontinuous convex functions f : H → ]−∞, +∞] and g : G → ]−∞, +∞],
respectively. In addition, let C be the gradient of a differentiable convex function h: H → R, and
let D be the subdifferential of a lower semicontinuous s trongly convex function ℓ: G → ] −∞, +∞].
Then, under suitable constraint qualification, (1.1) assumes the form of the minim ization problem
minimize
x∈H
f(x) + (g ℓ)(Lx) + h(x), (1.9)
which can be rewritten as
minimize
x∈H, y∈G
f(x) + h(x) + g(y) + ℓ(Lx − y). (1.10)
In the special case when h = 0, G = H, L = Id , and ℓ is a quadratic coupling function, such
formulations have been investigated in [
1, 4, 6, 12, 15].
Example 1.7 In Problem
1.1, set m = 1, G
1
= H, L
1
= Id , B
1
= D
−1
1
: x 7→ {0}, and z = r
1
= 0.
Then (
1.1) yields the inclusion 0 ∈ Ax + Cx studied in [45], where an algorithm using explicit steps
for C was proposed.
Example 1.8 In Problem 1.1, s et A: x 7→ {0} and C = Id . Furthermore, for every i ∈ {1, . . . , m},
let B
i
be the subdifferential of a lower semicontinuous convex function g
i
: G
i
→ ]−∞, +∞] and
3
let D
−1
i
: y 7→ {0}. Then, under s uitable constraint qualification, we obtain the primal-dual pair
considered in [
14], namely
minimize
x∈H
m
X
i=1
g
i
(L
i
x − r
i
) +
1
2
kx − zk
2
(1.11)
and
minimize
v
1
∈G
1
,..., v
m
∈G
m
1
2
z −
m
X
i=1
L
∗
i
v
i
2
+
m
X
i=1
g
∗
i
(v
i
) + hv
i
| r
i
i
. (1.12)
Example 1.9 The special case of Problem
1.1 in which
A: x 7→ {0}, C : x 7→ 0, and (∀i ∈ {1, . . . , m}) D
i
: y 7→
(
G
i
, if y = 0;
∅, if y 6= 0
(1.13)
yields the primal-dual pair
find
x ∈ H such that z ∈
m
X
i=1
L
∗
i
B
i
(L
i
x − r
i
)
(1.14)
and
find
v
1
∈ G
1
, . . . , v
m
∈ G
m
such that
(
P
m
i=1
L
∗
i
v
i
= z
(∃ x ∈ H)(∀i ∈ {1, . . . , m}) v
i
∈ B
i
(L
i
x − r
i
).
(1.15)
This framework is considered in [
10, Theorem 3.8].
Conceptually, the primal problem (1.1) could be recast in the form of (1.14), namely
find x ∈ H such that z ∈
m
X
i=0
L
∗
i
E
i
(L
i
x − r
i
)
, (1.16)
where
G
0
= H, E
0
= A + C, L
0
= Id , r
0
= 0, and (∀i ∈ {1, . . . , m}) E
i
= B
i
D
i
. (1.17)
In turn, one could contemplate the possibility of using the primal-dual algorithm proposed in [
10,
Theorem 3.8] to solve Problem
1.1. However , this algorithm requires the computation of the resolvents
of the operators A + C and (B
−1
i
+ D
−1
i
)
1≤i≤m
, which are usually intractable. Thus, f or numerical
purposes, Problem 1.1 cannot be reduced to Examp le 1.9. Let us stress th at, even in the instance
of the simple inclusion 0 ∈ A
x + Cx, it is precisely the objective of the forward-backward splitting
algorithm and its variants [
8, 15, 30, 44, 45] to circumvent the computation of the resolvent of A+ C,
as would impose a naive application of the proximal point algorithm [
42].
The goal of this paper is to propose a fully split algorithm for solving Problem
1.1 that employs
the operators A, (L
i
)
1≤i≤m
, (B
i
)
1≤i≤m
, (D
i
)
1≤i≤m
, and C separately. An important feature of the
algorithm is to activate the single-valued operators (L
i
)
1≤i≤m
, (D
−1
i
)
1≤i≤m
, and C through explicit
steps. In addition, it exhibits a highly parallel structure which allows for the simultaneous activation
of the operators involved. This new splitting method goes significantly beyond the state-of-the-art,
which is limited to specific subclasses of Problem
1.1.
In Section
2, we briefly set our notation. The new s plitting method is proposed in Section 3, where
we also pr ove its convergence. The special case of minimization problems is discussed in Section
4.
4