IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 7, NO. 4, APRIL 1998 555
Antiextensive Connected Operators
for Image and Sequence Processing
Philippe Salembier, Member, IEEE, Albert Oliveras, Member, IEEE, and Luis Garrido
Abstract— This paper deals with a class of morphological
operators called connected operators. These operators filter the
signal by merging its flat zones. As a result, they do not create
any new contours and are very attractive for filtering tasks
where the contour information has to be preserved. This paper
shows that connected operators work implicitly on a structured
representation of the image made of flat zones. The max-tree is
proposed as a suitable and efficient structure to deal with the
processing steps involved in antiextensive connected operators.
A formal definition of the various processing steps involved
in the operator is proposed and, as a result, several lines of
generalization are developed. First, the notion of connectivity and
its definition are analyzed. Several modifications of the traditional
approach are presented. They lead to connected operators that
are able to deal with texture. They also allow the definition of
connected operators with less leakage than the classical ones.
Second, a set of simplification criteria are proposed and discussed.
They lead to simplicity-, entropy-, and motion-oriented operators.
The problem of using a nonincreasing criterion is analyzed. Its
solution is formulated as an optimization problem that can be
very efficiently solved by a Viterbi algorithm. Finally, several
implementation issues are discussed showing that these operators
can be very efficiently implemented.
Index Terms— Connected operators, connectivity, mathemat-
ical morphology, motion criterion, optimization, sequence pro-
cessing, simplicity criterion, Viterbi algorithm, watershed.
I. INTRODUCTION
T
HE FIRST connected operators reported in the literature
are known as binary opening by reconstruction [1]. These
operators independently act on each connected component of
the binary image: they eliminate the connected components
that would be totally removed by an erosion with a given
structuring element and they leave the other components
unchanged. This filtering approach offers the advantage of
simplifying the image, because some components are removed,
as well as preserving the contour information, because the
components that are not removed are perfectly preserved.
This approach has been generalized for gray-level functions
using the so-called reconstruction process [2]. Beside opening
by reconstruction,
- operators, area opening [3], dynam-
ics filters [4], and more recently, volumic [5], complexity
[6], motion [7], and moment-oriented [8] operators have been
proposed. These operators offer various simplification criteria
Manuscript received August 23, 1996; revised June 4, 1997. The associate
editor coordinating the review of this manuscript and approving it for
publication was Dr. Henri Maitre.
The authors are with E.T.S.E.T.B-Universitat Polit
`
ecnica de Catalunya,
08034 Barcelona, Spain (e-mail: philippe@gps.tsc.upc.es).
Publisher Item Identifier S 1057-7149(98)02462-2.
(size, contrast, shape, etc.) while preserving contours. This
property makes them very attractive for a large number of
applications such as noise cancellation, segmentation, pattern
recognition, etc.
The extensive use of connected operators has motivated
some theoretical studies. For instance, the notions of connected
operators and of flat zones are discussed in a formal way in
[9]–[11]. Connectivity issues related to connected operators
are analyzed in [12]–[14]. Finally, relations with structured
representations of images such as region adjacency graphs and
trees are discussed in [7] and [15].
The purpose of this paper is to focus on the class of
antiextensive connected operators (and by duality, extensive
connected operators). Based on a formal operator definition
involving a tree representation of the image called a max-
tree, several contributions are proposed. First, the notion
of connectivity is analyzed. Several modifications of the
traditional approach are presented. They lead to connected
operators that are able to deal with texture or to connected
operators that have much less leakage than classical operators.
Second, a set of new simplification criteria are proposed and
discussed. In particular, simplicity-, entropy-, and motion-
oriented operators are defined. The problem of using a non-
increasing criterion is analyzed and its solution is formulated
as an optimization problem that can be very efficiently solved
by a Viterbi algorithm. Finally, several applications as well
as implementation issues are discussed. Note that part of the
work reported here can be found in conference proceedings
[6], [7], [13]. One of the objectives of the paper is to review
these contributions. However, some new contributions are also
presented here. These original contributions mainly concern
the max-tree creation and processing, the use of the Viterbi
algorithm to deal with nonincreasing criteria, and the entropy
connected operator.
The organization of this paper is as follows. Section II is
devoted to the notion of binary and gray-level connected oper-
ators. This presentation will highlight three major processing
steps: tree creation, tree filtering, and image restitution. These
three steps are, respectively, discussed in Sections III–V. Fi-
nally, Section VI is devoted to the conclusions.
II. C
ONNECTED OPERATORS
A. Theoretical Definition
In [9], [10], the concept of binary connected operators is
formally defined as follows:
1057–7149/98$10.00 1998 IEEE
556 IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 7, NO. 4, APRIL 1998
Fig. 1. Binary connected operator.
Definition 1—Binary Connected Operators: A binary oper-
ator
is connected when for any binary image , the set
difference
is exclusively composed of connected
components of
or of its complement .
The extension of connected operators for gray-level func-
tions relies on the concept of partition [9], [10]. Let us recall
that a partition of the space
is a set of connected components
which are disjoint and the union of which is the entire
space. Each
is called a partition class. Moreover, a partition
is said to be finer than another partition if any
pair of points belonging to the same class
also belongs
to a unique partition class
. Consider now a binary image
and define its associated partition as the partition made of
the connected components of the binary sets and of their
complement. The definition of connected operators can be
expressed using associated partitions as follows.
Theorem 2—Binary Connected Operators via Partition: A
binary operator
is connected if and only if, for any binary
image
, the associated partition of is finer than the
associated partition of
.
The concept of gray-level connected operators can be intro-
duced if we define a partition associated to a function. To this
end, the use of flat zones was proposed in [9] and [10]. The set
of flat zones of a gray-level function
is the set of the largest
connected components of the space where
is constant (a flat
zone can be reduced to a single point). The set of flat zones of
a function is a partition, called the partition of flat zones and
leads to the following definition.
Definition 3—Gray-Level Connected Operators: An opera-
tor
acting on gray-level functions is connected if, for any
function
, the partition of flat zones of is finer than the
partition of flat zones of
.
Let us see how this definition implicitly means that the
operator works on a structured representation of the image.
B. Binary Antiextensive Connected Operators
In the sequel, we restrict ourselves to the case of antiex-
tensive operators (
). Therefore, a binary
antiextensive connected operator is an operator that can only
remove connected components of
. The filtering process can
easily be explained if a tree representation of the image is used
as shown in Fig. 1.
The original image
is composed of three connected
components. It can be represented by a tree structure with four
nodes: the root node
represents the set of pixels belonging
to the background
, and represent the three
connected components of the image. In this representation,
the filtering process consists in analyzing each node
by assessing the value of a particular criterion. Assume for
example that the criterion consists in counting the number of
pixels belonging to a node (area opening [3]). Then, for each
node, the criterion value is compared to a given threshold
and the node is removed if the criterion is lower than .In
the example of Fig. 1, node
is removed because its area
is small. As a result, its pixels are moved to the background
node
(the connected component is removed). As can be
seen, the tree links represent the pixels’ migration (toward
the father) when a node is removed. All antiextensive binary
connected operators can be described by this process, the only
modification being the criterion that is assessed.
C. Gray-Level Antiextensive Connected Operators
As seen in Definition 3, the extension of connected operators
to gray-level images uses the partition of flat zones. This
extension can also be seen as a simple generalization of the
tree representation to the gray-level case.
The idea consists in creating the tree recursively by a
study of the thresholded versions of the image at all possible
gray levels. An example is presented in Fig. 2. The original
image is composed of seven flat zones identified by a letter
. The number following each letter
defines the gray-level value of the flat zone. In our example,
the gray-level values range from zero to two. In the first step,
the threshold
is fixed to the gray-level value zero. The image
is binarized: all pixels at level
(pixels of region ) are
assigned to the root node of the tree
. Furthermore,
the pixels of gray-level value strictly higher than
form
two connected components that are temporarily assigned to
two nodes:
and . This
creates the first tree (for gray levels [0, 1]). This procedure is
the same as the one used for the binary image. In a second step,
the threshold is increased by one:
. Each node
is processed as the original image. Consider, for instance, the
node
; all its pixels at level
remain unchanged and create the final node . However,
pixels of gray-level values strictly higher than
(here )
create two different connected components and are moved to
two temporary child nodes
and . The
complete tree construction is done by iterating this process
for all nodes
at level and for all possible thresholds
(from zero to the highest gray-level value). The algorithm can
be summarized saying that, at each temporary node
,a
“local” background is defined by keeping all pixels of gray-
level value equal to
(the “local” background itself may not be
connected) and that the various connected components formed
by the pixels of gray-level value higher than
create the child
nodes of the tree.
In this procedure, some nodes may become empty. There-
fore, at the end of the tree construction, the empty nodes
SALEMBIER et al.: ANTIEXTENSIVE CONNECTED OPERATORS 557
Fig. 2. Max-tree representations.
Fig. 3. Connected operators with max-tree representations.
are removed. The final tree is called a max-tree in the sense
that it is a structured representation of the image which is
oriented toward the maxima of the image (maxima are simply
the leaves of the tree) and toward the implementation of
antiextensive operators. Note that this description does not
necessarily correspond to the actual implementation of the
tree construction. For this purpose, an efficient algorithm is
proposed in the Appendix.
The filtering itself is similar to the one used for the binary
case. A criterion
is assessed for each node . Based
on the
value, the node is either preserved or removed.
In this last case, the node’s pixels are moved toward its
father’s node. At the end of the process, the output max-tree
is transformed into a gray-level image by assigning the gray
value
to the pixels of .
D. Connected Operators and Max-Tree Representation
Based on the previous description, a general filtering scheme
is illustrated in Fig. 3. It involves a first step of max-tree cre-
ation, the goal of which is to structure the pixels in a suitable
way for the filtering process. The max-tree representation has
also the advantage of leading to very efficient implementations
of connected operators. The second step is the filtering itself,
which analyzes each node and takes a decision on which node
has to be preserved and which node has to be removed. Finally,
the last step restores the filtered image by transforming the
output max-tree into a gray-level image. The discussion of
this paper will focus on antiextensive operators and max-
trees. By the duality
, the same notions can be
applied to extensive operators and min-trees. In the following,
we describe the three steps of Fig. 3 with more details.
III. M
AX-TREE CREATION
The objective of this step is to create the max-tree, that is
the set of nodes
and the links between the father and child
nodes. As described in Section II-C, for each temporary node
, the set of pixels belonging to the local background is
defined and assigned to the max-tree node
. This is the
binarization step. Then, the set of pixels belonging to the
complement of the local background, that is
, are
analyzed and its connected components create the temporary
child nodes
(which will be further analyzed). This
is the connected components definition step. In the sequel, the
binarization and the connected components definition steps are
analyzed and some generalizations are proposed.
A. Binarization
The most natural way of defining the “local” background for
each node at level
consists in taking all pixels of gray-level
value
. Formally, the node is composed of the pixels of
level
of the temporary node , that is
such that (1)
where
represents the gray-level value of the pixel
.
This binarization process extracts the flat zones of the image.
This step is closely related to the definition of the basic entities
on which the filter is going to act. Equation (1) means that
the basic entities are characterized by a strictly flat gray-
level value. However, in practice, “visual” entities may not be
strictly flat because of noise or texture. To deal with such cases,
less strict binarization techniques can be used. To this end, a
useful criterion relies on the definition of a bound
on the
gray-level fluctuations. The corresponding “soft” binarization
rule is the following:
such that
either
or and neighbor of
(2)
A flat zone is composed of pixels with low gray-level
fluctuations. The particular case
corresponds to the
classical situation where the flat zones are strictly flat. Figs. 4
and 5 illustrate the evolution of the flat zones as a function
of
. To judge intuitively this evolution, we show, on the left
side of Fig. 4, images where each flat zone has been filled by
its mean and, on the right side, the corresponding contours of
the flat zones. When
, the image on the left side is the
original image and most flat zones involve one or two pixels
(right image). Fig. 5 shows the reduction of the number of
flat zones. Of course, this curve has a direct relation with the
evolution of the max-tree complexity.
The interest of this approach can be foreseen by looking
at the flat zones corresponding to the water areas. If a “strict”
binarization is used (
), the water is represented by a very
high number of small flat zones and will not be processed as a
single entity. By contrast, if a “soft” binarization is used, these
small flat zones are grouped together to form larger entities that
558 IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 7, NO. 4, APRIL 1998
Fig. 4. Flat zones resulting from the “nonflat” binarization (2). Left:
gray-level image where each flat zone has been filled with its mean. Right:
contour of the flat zones. Upper left: Flat zones
1=0
. Middle left: Flat
zones
1=4
. Bottom left: Flat zones
1=8
. Upper right: Contours
1=0
.
Middle right: Contours
1=4
. Bottom right: Contours
1=8
.
Fig. 5. Evolution of the number of flat zones of the image of Fig. 4.
will be processed in a coherent way by the connected operator.
Note, however, that the pixels assigned to each node of the
max-tree do not have the same gray-level value. Therefore, the
restitution step after the filtering cannot simply assign the gray-
level
to pixels belonging to the node . Let us postpone
this discussion until Section V dealing with restitution issues.
In the sequel (and until Section V), we assume that a “strict”
binarization rule (
) is used. Let us concentrate on the
problem of connected components definition: once the “local”
background (that is
) has been defined, its complement has
to be analyzed to create the new temporary nodes at level
(that is ).
B. Connected Components Definition
In the case of discrete images, the simplest approach consists
in selecting a connectivity (4-, 6-, or 8-connectivity) and in
labeling the set of pixels of the temporary node that does not
belong to the “local background” following this connectivity
rule. Note that, as the binarization, this step is very important
because it implicitly defines the notion of objects that will be
processed by the operator. The objective of this section is to
discuss modifications of the connectivity and their influence on
the resulting connected operator. Let us recall the connectivity
definition.
Definition 4—Connectivity Class: A connectivity class
is
defined on the subsets of a set
when
1)
and ;
2) for each family
i
of
i i
.
It was shown in [16] that this definition is equivalent to
the definition of a family of connected pointwise openings
, associated to each point of . Let us recall
the following result.
Theorem 5—Connectivity Characterized by Openings: The
definition of a connectivity class
is equivalent to the def-
inition of a family of openings
such that:
1)
;
2)
and and are either
equal or disjoint;
3)
and .
Intuitively, the opening
extracts the connected com-
ponent of
that contains . Based on this definition of the
connectivity, a generalization was proposed in [16]. It relies
on the definition of a new connected pointwise opening:
if
and
if (3)
where
is an extensive dilation. This new operator is
a connected pointwise opening and therefore defines a new
connectivity. This connectivity is less “strict” than the usual
ones in the sense that it considers that two objects that are
close to each other (that is they touch each other if they
are dilated by
) belong to the same connected component.
This generalization can lead to interesting connected filters.
However, in this paper we concentrate on a different issue: in
practice, connected operators are known to present a drawback
called “leakage” that results from the connection of different
objects. These connections are created because there exist
thin connected paths between large objects. A solution to this
problem consists in breaking the thin connections of the binary
connected components and in segmenting the components into
a set of elementary shapes to be processed separately. As a
result, the connected operator can take individual decision on
each elementary shape. Ideally, the shapes should correspond
to our perception of the main parts of the object. This approach
SALEMBIER et al.: ANTIEXTENSIVE CONNECTED OPERATORS 559
can be seen as the definition of a “strict” connectivity. To our
knowledge, two attempts have been reported in the literature
to define “strict” connectivities.
• Segmentation by Openings [17]: Given a family of con-
nected pointwise openings,
, and an opening with a
connected structuring element, a new family of connected
pointwise opening,
, can be created as follows:
if
and
if (4)
and as usual
if . It can be shown that
is actually a connected pointwise opening and therefore
defines a connectivity. Intuitively, this connectivity con-
siders that the connected components of a binary set are
made of the connected components of its opening by
.
The points that are removed by the opening are considered
as isolated points, that are connected components of size
one.
Even if this solution is theoretically sound, in practice
it turns out that this way of segmenting the connected
components leads to a loss of one of the main features
of connected operators. In practice, connected operators
are used because they can simplify while preserving the
shape information of the remaining image components.
Suppose now that we use an area opening of size larger
than one with the connectivity defined by the connected
pointwise opening of (4). The filter will eliminate all
the isolated points (area equal to one) and all the small
connected components resulting from the opening. The
shape information of the remaining components will
not be preserved because most of the time, this shape
information relies on the set of isolated points. To solve
this problem, we propose the following approach.
• Segmentation by Watershed [6], [13]: The idea of this ap-
proach is to rely on classical morphological segmentation
tools. Morphological segmentation generally involves two
steps: marker extraction and watershed segmentation [18].
The marker extraction defines the interior of the regions
that should be segmented and the watershed precisely
defines the contours of these regions. The segmentation
procedure is illustrated by Fig. 6. The original gray-level
image can be seen in Fig. 6(a), and the set of binary
connected components resulting from a thresholding at
level 70 in Fig. 6(b) (note that different gray-level values
have been assigned to each connected component). In this
last figure, there is a very large connected component
involving the two speakers, the screen in the background
of the scene and the letters of the word “MPEG.” These
objects are processed as single entity by the operator
because there exist thin connections between them.
— Marker Extraction: In the context of connected
operators, this step defines the number of con-
nected components created by the segmentation.
A simple idea consists in using as markers the
connected components of the ultimate erosion
of
, denoted by . However, in practice,
a segmentation driven by the ultimate erosion
creates a very large number of connected com-
ponents. As an illustration, Fig. 6(g) shows the
ultimate erosion of the binary components of
Fig. 6(b). The number of connected components
can be reduced by computing the union of
with the erosion of with a structuring element
of size [denoted by ]. The set of
markers is defined by
.
The erosion merges some connected components
of the ultimate erosion. In particular, if
, the
markers are the connected components of
itself,
(the connected components are not
broken), whereas if
, the set of markers is
the ultimate erosion,
. Fig. 6(d)
presents the set of markers
resulting from
an erosion with a structuring element of size 3
3.
— Watershed Segmentation: Once the markers are
defined, the watershed algorithm propagates them
to precisely define the shape of the connected
components. In [6] and [13], it is proposed to use
one of the classical tools for binary segmentation
that is to rely on the opposite of the distance
function
to perform the propagation.
This segmentation is geometric, since it only takes
into account the shape of
. More formally, let us
define
the transformation
that assigns to
the catchment basin of the func-
tion
that contains taking into account
the markers
. Consider now the operator
if
and
if (5)
This transformation reduces to the classical con-
nected pointwise opening
when . For ,
it only creates a pseudoconnectivity. Indeed, in
that case, all conditions of Theorem 5 are met
except one:
is not increasing and therefore
not an opening. This is a drawback but, using the
watershed as segmentation tool, our main concern
is to segment the components of
in a small
number of regions and to keep as much as possible
the contour information of
. Moreover, in prac-
tice for small values of
, this theoretical problem
does not prevent the creation of useful operators.
This segmentation is illustrated in Fig. 6(e) and
(h) for the two sets of markers
and
. As can be seen, for a small value of
[Fig. 6(e)] the segmentation corresponds well to
various objects of the scene. This is, however,
not the case for high values of
. In particular,
