A Message Passing Algorithm for the Minimum Cost Multicut Problem
Paul Swoboda
IST Austria
pswoboda@ist.ac.at
Bjoern Andres
MPI for Informatics
andres@mpi-inf.mpg.de
Abstract
We propose a dual decomposition and linear program
relaxation of the
NP-hard minimum cost multicut problem.
Unlike other polyhedral relaxations of the multicut polytope,
it is amenable to efficient optimization by message pass-
ing. Like other polyhedral relaxations, it can be tightened
efficiently by cutting planes. We define an algorithm that
alternates between message passing and efficient separation
of cycle- and odd-wheel inequalities. This algorithm is more
efficient than state-of-the-art algorithms based on linear pro-
gramming, including algorithms written in the framework
of leading commercial software, as we show in experiments
with large instances of the problem from applications in com-
puter vision, biomedical image analysis and data mining.
1. Introduction
Decomposing a graph into meaningful clusters is a fun-
damental primitive in computer vision, biomedical image
analysis and data mining. In settings where no information is
given about the number or size of clusters, and information
is given only about the pairwise similarity or dissimilarity of
nodes, a canonical mathematical abstraction is the minimum
cost multicut (or correlation clustering) problem [
17
]. The
feasible solutions of this problem, multicuts, relate one-to-
one to the decompositions of the graph. A multicut is the set
of edges that straddle distinct clusters. The cost of a multicut
is the sum of costs attributed to its edges.
In the field of computer vision, the minimum cost multi-
cut problem has been applied in [
5
,
6
,
47
,
8
] to the task of
unsupervised image segmentation defined by the BSDS data
sets and benchmarks [
36
] . In the field of biomedical image
analysis, the minimum cost multicut problem has been ap-
plied to an image segmentation task for connectomics [
7
]. In
the field of data mining, applications include [
9
,
40
,
15
,
16
].
Also, recently many computer vision problems that use
the multicut problem as a building block have been proposed:
image and mesh segmentation [
31
], instance-separating se-
mantic segmentation[
33
], multiple object tracking [
44
], cell
tracking[
26
] and articulated human body pose estimation [
3
].
Moreover, one of the tightest relaxations for the multi-label
Potts model is based on multicuts [30].
As the minimum cost multicut problem is NP-hard [
12
,
19
], even for planar graphs [
10
] large and complex instances
with millions of edges, especially those for connectomics,
pose a challenge for existing algorithms.
Related Work.
Due to the importance of multicuts for
applications, many algorithms for the minimum cost multicut
problem have been proposed. They are grouped below into
three categories: primal feasible local search algorithms,
linear programming algorithms and fusion algorithms.
Primal feasible local search algorithms [
42
,
38
,
23
,
21
,
22
] attempt to improve an initial feasible solution by means
of local transformations from a set that can be indexed or
searched efficiently. Local search algorithms are practical
for large instances, as the cost of all operations is small
compared to the cost of solving the entire problem at once.
On the downside, the feasible solution that is output typically
depends on the initialization. And even if a solution is found,
optimality is not certified, as no lower bound is computed.
Also, the multicut problem can be transformed into a
Markov random field and solved with primal heuristics there,
as done for the “scribbles” dataset in [37, 11].
Linear programming algorithms [
28
,
29
,
32
,
39
,
46
] op-
erate on an outer polyhedral relaxation of the feasible set.
Their output is independent of their initialization and pro-
vides a lower bound. This lower bound can be used directly
inside a branch-and-bound search for certified optimal so-
lutions. Alternatively, the LP relaxation can be tightened
by cutting planes. Several classes of planes are known that
define a facet of the multicut polytope and can be separated
efficiently [
17
]. On the downside, algorithms for general
LPs that are agnostic to the structure of the multicut problem
scale super-linearly with the size of the instance.
Fusion algorithms attempt to combine feasible solutions
of subproblems obtained by combinatorial or random proce-
dures into successively better multicuts. The fusion process
can either rely on column generation [
47
], binary quadratic
programming [
14
] or any algorithm for solving integer
LPs [
13
]. In particular, [
47
] provides dual lower bounds
1
1617
Figure 1
. Decomposition of a
graph into three components
(green). The corresponding
multicut consists of the edges
straddling distinct components
(red).
but is restricted to planar graphs. [
14
,
13
] explore the pri-
mal solution space in a clever way, but do not output dual
information.
Outline.
Below, a discussion of preliminaries (Sec. 2) is
followed by the definition of our proposed decomposition
(Sec. 3) and algorithm (Sec. 4) for the minimum cost multi-
cut problem. Our approach combines the efficiency of local
search with the lower bounds of LPs and the subproblems of
fusion, as we show in experiments with large and diverse in-
stances of the problem (Sec. 5). All code and data is available
at http://github.com/pawelswoboda/LP_MP.
2. Preliminaries
2.1. Minimum Cost Multicut Problem
A decomposition (or clustering) of a graph
G = (V, E)
is a partition
V
1
∪ . . . ∪ V
k
of the node set
V
such that
V
i
∩ V
j
= ∅ ∀i 6= j
and every cluster
V
i
,
i = 1, . . . , k
is connected. The multicut induced by a decomposition
is the subset of those edges that straddle distinct clusters
(cf. Fig.
1). Such edges are said to be cut. Every multicut
induced by any decomposition of
G
is called a multicut of
G. We denote by M
G
the set of all multicuts of G.
Given, for every edge
e ∈ E
, a cost
c
e
∈ R
of this
edge being cut, the instance of the minimum cost multicut
problem w.r.t. these costs is the optimization problem
(1)
whose feasible solutions are all multicuts of
G
. For any edge
{v, w} = e ∈ E
, negative costs
θ
e
< 0
favour the nodes
v
and
w
to be in distinct components. Positive costs
θ
e
> 0
favour these nodes to lie in the same component.
min
M∈M
G
X
e∈M
θ
e
(1)
This problem is NP-hard [
12
,
19
], even for planar
graphs [
10
]. Below, we recapitulate its formulation as a bi-
nary LP and then turn to LP relaxations: For any 01-labeling
x ∈ {0, 1}
E
of the edges of
G
, the subset
x
−1
(1)
of those
edges labeled 1 is a multicut of
G
if and only if
x
satisfies
the system
(3)
of cycle inequalities [
17
]. Hence,
(1)
can be
stated equivalently in the form of the binary LP (2)–(4).
min
x∈R
E
X
e∈E
θ
e
x
e
(2)
subject to ∀C ∈ cycles(G) : x
e
≤
X
e
′
∈C\{e}
x
e
′
(3)
x ∈ {0, 1}
E
(4)
u
v
1
v
2
v
3
v
4
v
5
Figure 2. Odd Wheel
u
1
u
2
v
1
v
2
v
3
v
4
v
5
Figure 3. Odd Bicycle Wheel
An LP relaxation is obtained by replacing the integrality
constraints
(4)
by
x ∈ P
with
P ⊆ [0, 1]
E
. This results in
an outer relaxation of the multicut polytope, which is the
convex hull of the characteristic functions of all multicuts of
G
. The LP relaxation obtained for
P := [0, 1]
E
, i.e., with
only the cycle inequalities, will not in general be tight.
A tighter LP relaxation is obtained by enforing also the
odd wheel inequalities [17]. A k-wheel is a cycle in G with
k
nodes all of which are connected to an additional node
u ∈ V
that is not in the cycle and is called the center of
the
k
-wheel (cf. Fig. 2). For any odd number
k ∈ N
, any
k
-
wheel of
G
, the cycle
C = (v
1
v
2
, . . . , v
k
v
1
)
and the center
u
of the
k
-wheel, every characteristic function
x ∈ {0, 1}
E
of a multicut
x
−1
(1)
of
G
satisfies the odd wheel inequality
k
X
i=1
x
v
i
v
i+1
−
k
X
i=1
x
uv
i
≤
k
2
with v
k+1
:= v
1
. (5)
For completeness, we note that other inqualities known
to further tighten the LP relaxation can be included in our
algorithm, e.g., the bicycle inequalities [
17
] defind on graphs
as in Fig. 3. We, however, do not consider inequalities other
than cycles and odd wheels in the algorithm we propose.
2.2. Integer relaxed pairwise separable LPs
LP relaxations of the multicut problem can in principle be
solved with algorithms for general LPs which are available
in excellent software such as CPlex [
2
] and Gurobi [
25
].
However, these algorithms scale super-linearly with the size
of the problem and are hence impractical for large instances.
We define in Sec. 3 an LP relaxation of the multicut
problem in form of an IRPS-LP (Def.
1). IRPS-LPs are a
special case of dual decomposition [
24
]. In Def. 1, every
i ∈ V
defines a subproblem, and every edge
ij ∈ E
defines
a dependency of subproblems. Def. 1 is more specific in that,
firstly, the subproblems are binary and, secondly, the linear
constraints
(9)
that describe the dependence of subproblems
are defined by 01-matrices that map 01-vectors to 01-vectors.
IRPS-LPs are amenable to efficient optimization by message
passing in the framework of [43].
Definition 1
(IRPS-LP [
43
])
.
Let
N ∈ N
and let
G = (V, E)
be a graph with
V = {1, . . . , N}
. For every
j ∈ V
, let
d
j
∈ N
, let
X
j
⊆ {0, 1}
d
j
, and let
θ
j
∈ R
d
j
. Let
Λ :=
conv(X
1
) ×· · · × conv(X
N
)
. For every
{j, k} = e ∈ E
, let
1618
v
1
v
2
v
3
v
4
v
5
Figure 4
. A triangulated
cycle (black) covered
by three triangles (red,
green and blue)
m
e
∈ N
,
A
(j,k)
∈ {0, 1}
m
e
×d
j
and
A
(k,j)
∈ {0, 1}
m
e
×d
k
such that
∀x ∈ X
j
: A
(j,k)
x ∈ {0, 1}
m
e
(6)
∀x ∈ X
k
: A
(k,j)
x ∈ {0, 1}
m
e
(7)
Then, the LP written below is called integer relaxed pairwise
separable w.r.t. the graph G.
min
µ∈Λ
X
j∈V
d
j
X
k=1
θ
jk
µ
jk
(8)
subject to ∀{j, k} ∈ E : A
(j,k)
µ
j
= A
(k,j)
µ
k
(9)
3. Dual Decomposition
A straight-forward decomposition of the minimum cost
multicut problem
(2)
–
(4)
in the form of an IRPS-LP (Def. 1)
consists of one subproblem for every edge, one subproblem
for every cycle inequality and one subproblem for every odd-
wheel inequality. From a computational perspective, it is
however advantageous to triangulate cycles and odd wheels,
and to consider the resulting smaller subproblems. Below,
three classes of subproblems are defined rigorously.
Edge Subproblems.
For every edge
e ∈ E
, we consider
a subproblem
e ∈ V
with the feasible set
X
e
:= {0, 1}
,
encoding whether edge e is cut (1) or uncut (0).
Triangle Subproblems
For every cycle
C =
{v
1
v
2
, v
2
v
3
, . . . v
k
v
1
} ⊆ E
, we consider the trian-
gles
v
1
v
2
v
3
to
v
k−1
v
k
v
1
, as depicted in Fig. 4. If some edge
uv
of a triangle
C
i
is not in
E
, we add it to
E
with cost
zero, i.e., we triangulate the cycle in
G
. For each triangle
uvw
, we introduce a subproblem
uvw ∈ V
whose feasible
set consists of the five feasible multicuts of the triangle, i.e.,
X
uv w
:= {(0, 0, 0), (0, 1, 1), (1, 0, 1), (1, 1, 0), (1, 1, 1)}.
Lollipop Subproblems
For every odd number
k ∈ N
and
every
k
-wheel of
G
consisting of a center node
u
and cycle
nodes
v
1
, . . . , v
k
, we introduce two classes of subproblems.
For the 5-wheel depicted in Fig. 2, these subproblems are
depicted in Fig. 5.
For every
j ∈ {2, . . . , k}
, we add the triangle subproblem
uv
1
v
j
∈ V, as described in the previous section.
For every
j ∈ {2, . . . , k − 1}
, we add the subproblem
uv
j
v
j+1
, v
1
∈ V
for the lollipop graph that consists of the
triangle
uv
j
v
j+1
and the additional edge
uv
1
. The feasible
set
X
uv w,s
of a lollipop graph
uvw, s
has ten elements, five
feasible multicuts of the triangle times two for accounting
for the additional edge.
3.1. Dependencies
The dependency between triangle subproblems and edge
subproblems are expressed below in the form of a linear
system. It fits into the form (9) of an IRPS-LP.
µ
uv
= µ
uv w
(1, 1, 0) + µ
uv w
(1, 0, 1) + µ
uv w
(1, 1, 1)
µ
uw
= µ
uv w
(1, 1, 0) + µ
uv w
(0, 1, 1) + µ
uv w
(1, 1, 1)
µ
v w
= µ
uv w
(1, 0, 1) + µ
uv w
(0, 1, 1) + µ
uv w
(1, 1, 1)
The dependency between a lollipop subproblem with edge
set
L = {e
1
, e
2
, e
3
, e
4
}
and a triangle subproblem with edge
set
T = {e
′
1
, e
′
2
, e
′
3
}
is stated below as a linear system with
sums over edges not shared between
L
and
T
. This linear
system has the form (9) of an IRPS-LP.
∀x
L∩T
:
X
x
L\T
µ
L
(x
L∩T
, x
L\T
) =
X
x
T \L
µ
T
(x
T ∩L
, x
T \C
)
3.2. Remarks
Remark 1.
The triangulation of cycles can be understood
as the constructing of a junction tree [
45
] in such a way that
the minimum cost multicut problem over the cycle can be
solved by dynamic programming. The triangulation of cycles
can also be understood as a tightening of an outer polyhedral
relaxation of the multicut polytope: A cycle inequality
(3)
defines a facet of the multicut polytope if and only if the
cycle is chordless [
17
]. By triangulating a cycle, we obtain
a set of minimal chordless cycles (triangles) whose cycle
inequalities together imply that of the entire cycle.
Remark 2.
Technically, we would not have needed to
include triangle subproblems for odd wheels. Instead, we
could have introduced dependencies between lollipops di-
rectly in the form of an IRPS-LP. However, by introducing
triangle factors in addition and by expressing dependencies
between lollipops and triangles, we couple lollipop factors
from different odd wheels more tightly whenever they share
the same triangles.
4. Algorithm
We now define an algorithm for the minimum cost mul-
ticut problem
(2)
–
(4)
. This algorithm takes an instance of
the problem as input and alternates for a fixed number of
iterations between two main procedures.
The first procedure, defined in Sec. 4.1, solves an in-
stance of a dual of the IRPS-LP relaxation defined in the
1619
v
1
v
2
u
v
1
v
2
v
3
u
v
1
v
3
u
v
1
v
3
v
4
u
v
1
v
4
u
v
1
v
4
v
5
u
v
1
v
5
u
Figure 5. Triangulation of the odd wheel from Figure 2. It con-
sists of the triangles
uv
1
v
2
, uv
1
v
3
, uv
1
v
4
, uv
1
v
5
and the lollipop
graphs (uv
2
v
3
, v
1
), (uv
3
v
4
, v
1
), (uv
4
v
5
, v
1
).
previous section. The output consists in a lower bound and
a re-parameterization of the instance of the minimum cost
multicut problem given as input. The second procedure tight-
ens the IRPS-LP relaxation by adding subproblems for cycle
inequalities
(3)
and odd wheel inequalities
(5)
violated by
the current solution. Separation procedures for finding such
violated inequalities, more efficiently than in cutting plane
algorithms for the primal [
28
,
29
,
32
], are defined in Sec. 4.2.
To find feasible solutions of the instance of the min-
imum cost multicut problem given as input, we apply a
state-of-the-art local search algorithm on the computed re-
parameterizations, a procedure commonly referred to as
rounding (Sec. 4.3).
4.1. Message Passing
Like other algorithms based on dual decomposition, the
algorithm we propose does not solve the IRPS-LP directly,
in the primal domain, but optimizes a dual of
(8)
–
(9)
. Specif-
ically, it operates on a space of re-parametrizations of the
problem defined below: For any two dependent subproblems
jk ∈ E
, we can change the costs
θ
j
and
θ
k
by an arbitrary
vector ∆ according to the update rules
θ
′
j
:= θ
j
+ A
⊤
(j,k)
∆ (10)
θ
′
k
:= θ
k
− A
⊤
(k,j)
∆ . (11)
We refer to any update of
θ
according to the rules
(10)
–
(11)
as message passing. Message passing does not change the
cost of any primal feasible solution, as
hθ
′
j
, µ
j
i + hθ
′
k
, µ
k
i
= hθ
j
+ A
⊤
(j,k)
∆, µ
j
i + hθ
k
− A
⊤
(k,j)
∆, µ
k
i (12)
= hθ
j
, µ
j
i + hθ
k
, µ
k
i + h∆, A
(j,k)
µ
j
− A
(k,j)
µ
k
i (13)
(
9)
= hθ
j
, µ
j
i + hθ
k
, µ
k
i . (14)
Message passing does, however, change the dual lower
bound L(θ) to (8) given by
L(θ) :=
X
j∈V
min
x∈X
i
hθ
j
, x
j
i . (15)
Algorithm 1:
Message passing for the multicut problem
Data: {i
1
, . . . , i
k
} = V, (θ
i
)
i∈V
, (A
(j,i)
, A
(i,j)
)
ij∈E
for i = i
1
, . . . , i
k
do
if i is an edge subproblem uv: then
Receive messages:
for w ∈ V : uvw ∈ T do
δ := min
x
uw
,x
vw
θ
uv w
(1, x
uw
, x
v w
)
− min
x
uw
,x
vw
θ
uv w
(0, x
uw
, x
v w
)
θ
uv
+= δ
∀x
uw
, x
v w
: θ
uv w
(1, x
uw
, x
v w
) -= δ
end
Send messages:
δ := |{w ∈ V : uvw ∈ T }|
−1
θ
uv
θ
uv
:= 0
for w ∈ V : uvw ∈ T do
∀x
uw
, x
v w
: θ
uv w
(1, x
uw
, x
v w
) += δ
end
end
if i is a triangle subproblem uvw with edges C:
then
Receive messages:
for lollipops L with L ∩ C 6= ∅ do
δ(x
L∩C
) := min
x
L\C
θ
L
(x
L∩C
, x
L\C
)
θ
C
(x
L∩C
, x
C\L
) += δ(x
L∩C
)
θ
L
(x
L∩C
, x
L\C
) += δ(x
L∩C
end
Send messages:
α := |{L a lollipop : L ∩ C 6= ∅}|
for lollipops L with L ∩ C 6= ∅ do
δ
L
(x
L∩C
) :=
min
x
C\L
θ
uv w
(x
L∩C
, x
C\L
)
θ
L
(x
L∩C
, x
L\C
) +=
1
1+α
δ
L
(x
L∩C
)
end
for lollipops L with L ∩ C 6= ∅ do
θ
C
(x
L∩C
, x
C\L
) +=
1
1+α
δ
L
(x
L∩C
)
end
end
end
The maximum of
L(θ)
over all costs obtainable by mes-
sage passing is equal to the minimum of
(8)
, by linear pro-
gramming duality. We seek to alter the costs
θ
by means
of message passing so as to maximize the lower bound
L(θ)
. For the general IRPS-LP, a framework of algorithms
to achieve this goal is d efined in [
43
]. For the minimum cost
multicut problem, we define and implement Alg. 1 within
this framework. The specifics of this algorithm for the min-
imum cost multicut problem are discussed below. General
properties of message passing for IRPS-LP s are discussed
in [43].
1620
Factor Order.
Alg. 1 iterates through all edge and triangle
subproblems. The order is specified as follows: We assume
that a node order is given. With respect to this node order,
edges
uv ∈ E
are ordered lexicographically. For every
triangle and its edge set
C = {e
1
, e
2
, e
3
} ⊆ E
with
e
1
<
e
2
< e
3
, we define the ordering constraint
e
1
< C < e
3
.
For every lollipop graph and its edge set
L = {e
1
, e
2
, e
3
, e
4
}
with
e
1
< e
2
< e
3
< e
4
, we define the ordering constraint
e
1
< L < e
4
. The strict partial order defined by these
constraints is extended to a total order by topological sorting.
Message Passing Description.
When an edge subprob-
lem
uv ∈ E
is visited, Alg. 1 receives messages from all
dependent triangle subproblems. Having received a message
from triangle
uvw ∈ E
, the costs
θ
uv w
satisfy the condition
min
x
uw
,x
vw
θ
uv w
(0, x
uw
, x
v w
) = min
x
uw
,x
vw
θ
uv w
(1, x
uw
, x
v w
) .
In other words, the cost of the triangle factor
θ
uv w
has no
preference for either
x
uv
= 0
or
x
uv
= 1
. Sending messages
from
θ
uv
is analoguous: Having sent messages from
uv
, we
have
θ
uv
= 0
, i.e., there is again no preference for either
x
uv
= 0 or x
uv
= 1.
When we visit a triangle subproblem
uvw
, we do the
analogous with all dependent lollipop subproblems: Once
messages have been received, lollipop subproblems have no
preference for incident edges. Once messages have been
sent, this holds true for the triangle subproblems.
Once Alg. 1 has visited all subproblems and terminates,
we reverse the order of subproblems and invoke Alg. 1 again.
This double call of Alg. 1 is repeated for a fixed number of
iterations that is a parameter of our algorithm.
4.2. Separation
Applying Alg. 1 with all cycles and all odd wheels of a
graph
G
is impractical, as the number of triangles for cycle
inequalities
(3)
is cubic, and the number of lollipop graphs
for odd wheels
(5)
is quartic in
|E|
. In order to arrive at
a practical algorithm, we take a cutting plane approach in
which we separate and add subproblems for violated cycle
and odd wheel inequalities periodically. Initially,
V
contains
only one element for every edge e ∈ E, and E is empty.
In the primal, given some fractional
x ∈ [0, 1]
E
, it is com-
mon to look for maximally violated inequalities (3) and (5).
This is possible in polynomial time via shortest path compu-
tations [
17
,
20
]. In our dual formulation, we have no primal
solution
x
to search for violated inequalities. Here, a suitable
criterion is to consider those additional triangle or lollipop
subproblems that necessarily increase the dual lower bound
L(θ)
by some constant
ǫ > 0
. Among these subproblems,
we choose those for which the increase is maximal and add
them to the graph
(V, E)
. A similar dual cutting plane ap-
proach has shown to be useful for graphical models in [
41
].
Algorithm 2: Separation of cycle inequalities (3)
Data: G = (V, E), ǫ ≥ 0, θ
e
∈ R
l := 1
for uv ∈ E do
if θ
uv
≥ ǫ then
union(u, v)
end
end
for uv ∈ E do
if θ
uv
≤ −ǫ and find(u) = find(v) then
C
l
:= shortest-path(u, v, ǫ)
l := l + 1
end
end
As we discuss below, separation is more efficient in the dual
than in the primal.
4.2.1 Cycle Inequalities
We characterize those cycles whose subproblem increases
the dual lower bound L(θ) by at least ǫ.
Proposition 1.
Let
C = {e
1
, . . . , e
k
}
be a cycle with
θ
e
1
≤
−ǫ
and
θ
e
l
≤ ǫ
for
l > 1
. Then, the dual lower bound
L(θ)
can be increased by ǫ by including a triangulation of C.
In order to find such cycles, we apply Alg. 2. This algo-
rithm first records in a disjoint set data structure
1
whether
distinct nodes
u, v ∈ V
are connected via edges with weight
≥ ǫ
. Specifically, the disjoint-set operation
union(u, v)
in
Algorithm
2 joins the connected components of
u
and
v
.
Then, we visit all edges
uv ∈ E
with
θ
uv
≤ −ǫ
. Querying
the disjoint-set datastructure via
find(u) = find(v)
reveals
whether
u
and
v
are connected via a path with edges of
weight
≥ ǫ
. If so we search for a shortest one with a breadth
first search.
In the primal, finding a maximally violated cycle inequal-
ity
(3)
is more expensive, requiring, for every edge
uv ∈ E
,
the search for a
uv
-path with minimum cost
x
[
17
] by, e.g.,
Dijkstra’s algorithm.
4.2.2 Odd Wheel Inequalities
We characterize those odd wheels whose lollipop subprob-
lem increases the lower bound L(θ) by at least ǫ.
Proposition 2.
Let
O
an odd wheel with center node
u
and
cycle nodes
v
1
, . . . , v
k
. Adding the lollipop subproblems for
O
increases
L(θ)
by at least
ǫ
if the costs
θ
uv
i
v
i+1
of each
triangle
uv
i
v
i+1
are such that the minimal cost of any edge
1
https://en.wikipedia.org/wiki/Disjoint-set_
data_structure
1621