IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICS—PART B: CYBERNETICS, VOL. 40, NO. 5, OCTOBER 2010 1267
Segmentation and Classification of Hyperspectral
Images Using Minimum Spanning Forest Grown
From Automatically Selected Markers
Yuliya Tarabalka, Student Member, IEEE, Jocelyn Chanussot, Senior Member, IEEE,and
Jón Atli Benediktsson, Fellow, IEEE
Abstract—A new method for segmentation and classification of
hyperspectral images is proposed. The method is based on the
construction of a minimum spanning forest (MSF) from region
markers. Markers are defined automatically from classification
results. For this purpose, pixelwise classification is performed, and
the most reliable classified pixels are chosen as markers. Each
classification-derived marker is associated with a class label. Each
tree in the MSF grown from a marker forms a region in the
segmentation map. By assigning a class of each marker to all the
pixels within the region grown from this marker, a spectral-spatial
classification map is obtained. Furthermore, the classification map
is refined using the results of a pixelwise classification and a ma-
jority voting within the spatially connected regions. Experimental
results are presented for three hyperspectral airborne images.
The use of different dissimilarity measures for the construction
of the MSF is investigated. The proposed scheme improves classi-
fication accuracies, when compared to previously proposed clas-
sification techniques, and provides accurate segmentation and
classification maps.
Index Terms—Classification, hyperspectral images, marker se-
lection, minimum spanning forest (MSF), segmentation.
I. INTRODUCTION
I
MAGE CLASSIFICATION, which can be defined as iden-
tification of objects in a scene captured by a vision system,
is one of the important tasks of a robotic system. On the one
side, the procedure of accurate object identification is known
to be more difficult for computers than for people [1]. On the
other side, recently developed image acquisition systems (for
instance, radar, lidar, and hyperspectral imaging technologies)
capture more data from the image scene than a human vision
Manuscript received May 29, 2009; revised September 4, 2009; accepted
November 3, 2009. Date of publication December 31, 2009; date of current ver-
sion September 15, 2010. This work was supported in part by the Marie Curie
Research Training Network “Hyper-I-Net.” This paper was recommended by
Associate Editor D. Goldgof.
Y. Tarabalka is with the Grenoble Images Speech Signals and Au-
tomatics Laboratory (GIPSA Lab), Grenoble Institute of Technology
(INPG), 38402 Grenoble, France, and the Faculty of Electrical and Com-
puter Engineering, University of Iceland, 107 Reykjavik, Iceland (e-mail:
yuliya.tarabalka@hyperinet.eu).
J. Chanussot is with the Grenoble Images Speech Signals and Automatics
Laboratory (GIPSA Lab), Grenoble Institute of Technology (INPG), 38402
Grenoble, France (e-mail: jocelyn.chanussot@gipsa-lab.grenoble-inp.fr).
J. A. Benediktsson is with the Faculty of Electrical and Computer Engineer-
ing, University of Iceland, 107 Reykjavik, Iceland (e-mail: benedikt@hi.is).
Color versions of one or more of the figures in this paper are available online
at http://ieeexplore.ieee.org.
Digital Object Identifier 10.1109/TSMCB.2009.2037132
Fig. 1. Structure of a hyperspectral image.
system. Therefore, efficient processing systems must be devel-
oped in order to use these data for accurate image classification.
Hyperspectral imagery records a detailed spectrum of light
arriving at each pixel [2]. Hyperspectral sensors measure the
energy of the received light in tens or hundreds of narrow spec-
tral bands (data channels) in each spatial position of the image
(Fig. 1 shows the structure of a hyperspectral image). This rich
information per pixel increases the capability to distinguish ma-
terials and objects and thus opens new perspectives for image
classification. However, a large number of spectral channels,
usually coupled with limited availability of reference data,
1
present challenges to image analysis. While pixelwise classi-
fication techniques process each pixel independently without
considering the information about spatial structures [3]–[5],
further improvement of classification results can be achieved
by considering spatial dependences between pixels, i.e., by
performing spectral-spatial classification [6]–[11].
Segmentation is an exhaustive partitioning of the input image
into homogeneous regions [12]. Segmentation techniques are a
powerful tool to define spatial dependences. In previous works,
we have performed unsupervised segmentation of hyperspectral
images in order to define spatial structures [9], [13], [14].
Watershed, partitional clustering, and hierarchical segmentation
techniques have been used for this purpose. Segmentation
and pixelwise classification were performed independently, and
then, the results were combined using a majority voting rule.
Thus, every region from a segmentation map was considered
as an adaptive homogeneous neighborhood for all the pixels
within this region. The described technique led to a signification
improvement of classification accuracies and provided more
1
By reference data, we mean manually labeled pixels which are used for
training classifiers followed by assessment of classification accuracies.
1083-4419/$26.00 © 2009 IEEE
1268 IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICS—PART B: CYBERNETICS, VOL. 40, NO. 5, OCTOBER 2010
homogeneous (less noisy) classification maps when compared
to classification techniques using local neighborhoods in order
to include spatial information into a classifier.
However, unsupervised image segmentation is a challenging
task. Segmentation aims at dividing an image into homo-
geneous regions, but the measure of homogeneity is image
dependent [12]. Depending on this measure, the process can
result in undersegmentation (several regions are detected as
one) or oversegmentation (one region is detected as several
ones) of the image. In previous works [13], [14], we preferred
oversegmentation to undersegmentation in order not to miss
objects in the classification map. In this work, we aim to reduce
oversegmentation and thus further improve segmentation and
classification results. This can be achieved by using markers
or region seeds [12], [15]. In previous studies, a marker (an
internal marker) was defined as a connected component belong-
ing to the image and associated with an object of interest [12],
[15]–[17]. In our study, we define a marker as a set of image
pixels (not necessarily connected; it can be composed of several
spatially disjoint subsets of adjacent pixels) which is associated
with one object in the image scene.
The problem of automatic marker selection has previously
been discussed in the literature, mostly for gray-scale and
color images. Markers are often defined by searching flat zones
(i.e., connected components of pixels of constant gray-level
value), zones of homogeneous texture, or image extrema [15].
Gómez et al. [18] applied histogram analysis to obtain a set
of representative pixel values, and the markers were gener-
ated with all the image pixels with representative gray values.
Jalba et al. [16] used connected operators filtering on the
gradient image in order to select markers for a gray-scale
diatom image. Noyel et al. [17], [19] performed classification
of the hyperspectral image (using different techniques, such as
Clara [20] and linear discriminant analysis) and then filtered
the classification maps class by class, using morphological
operators, in order to select large spatial regions as markers.
Furthermore, the authors proposed to use random balls (con-
nected sets of pixels of randomly selected sizes) extracted from
these large regions as markers. In the discussed studies [16],
[17], [19], the objective was to segment specific structures
(blood cells, diatoms, glue occlusions, and cancerous growth).
In our study, the objective is to mark (select a marker for)
each significant spatial object in the image. Here, by significant,
we mean an object of at least one-pixel size that belongs to
one of the classes of interest. As remote sensing images contain
small and complex structures, automatic selection of markers is
an especially challenging task.
In this paper, a new scheme for marker-based segmenta-
tion and classification of hyperspectral images is proposed.
In particular, we propose to perform a probabilistic pixelwise
classification first in order to choose the most reliable classified
pixels as markers of spatial regions [21]. Furthermore, image
pixels are grouped into a minimum spanning forest (MSF)
[22], where each tree is rooted on a classification-derived
marker. The decision to connect the pixel, which is not yet
in the forest, to one of the trees in the forest is based on its
similarity to one of the adjacent pixels already belonging to
the forest. By assigning a class of the marker to all the pixels
within the region grown from the considered marker, a spectral-
spatial classification map is obtained. Furthermore, the classi-
fication map is refined using the results of a pixelwise classi-
fication and a majority voting within the spatially connected
regions [14].
The construction of an MSF belongs to graph-based ap-
proaches for image segmentation [22]–[25]. They introduce the
Gestalt principles of perceptual grouping to the field of com-
puter vision. The image is associated with a graph, the vertices
of which correspond to the image entities (pixels or regions)
and the edges correspond to relations between these entities. A
weight associated with each edge indicates the (dis)similarity
between two entities (pixels or regions). Morris et al. [23] have
proposed to perform a graph-based image segmentation into R
regions by constructing a shortest spanning tree on the image
graph and then removing the R − 1 edges with the highest
weight. Furthermore, several graph-cut-based algorithms have
been developed for image segmentation [24], [25]. However,
these methods perform unsupervised segmentation by splitting
at each iteration one region into two subregions. This approach
is fundamentally different from the work described in this
paper. Several recent publications describe the use of an MSF
rooted on markers for image segmentation [22], [26], [27].
However, the authors of these works do not investigate the
problem of automatic marker selection. Their segmentation is
based on markers provided by the user.
The proposed procedure of defining markers for each spatial
object from probabilistic classification results and of build-
ing a spectral-spatial classification map for hyperspectral
images by constructing an MSF rooted on classification-
derived markers is a major contribution of this paper. Please
note that, while, in previous studies, markers were used as
seeds for image segmentation, in this paper, we introduce a
new concept of the automatic marker-based spectral-spatial
classification.
1) Markers are derived from probabilistic pixelwise classifi-
cation results.
2) Each marker can be composed of several spatially disjoint
subsets of adjacent pixels, and each marker has a class
label.
3) By performing a region growing from the classification-
derived markers, a spectral-spatial classification map is
obtained.
Although the classification scheme proposed in this paper has
been designed for hyperspectral data, the method is general and
can successfully be applied for other types of data as well. Ex-
perimental results are demonstrated on hyperspectral airborne
images recorded by the Airborne Visible/Infrared Imaging
Spectrometer (AVIRIS) over Northwestern Indiana’s Indiana
Pine site and over the region surrounding the volcano Hekla
in Iceland, and the image acquired by the Reflective Optics
System Imaging Spectrometer (ROSIS) over the University of
Pavia in Italy.
The outline of this paper is as follows. In the next section,
a classification scheme based on an MSF rooted on markers
is presented. Experimental results are discussed in Section III.
Finally, conclusions are drawn in Section IV.
TARABALKA et al.: SEGMENTATION AND CLASSIFICATION OF HYPERSPECTRAL IMAGES USING MSF 1269
II. SEGMENTATION AND CLASSIFICATION SCHEME
The flowchart of the proposed segmentation and classifica-
tion method is shown in Fig. 2. On the input, a B-band hyper-
spectral image is given, which can be considered as a set of n
pixel vectors X = {x
j
∈ R
B
,j =1, 2,...,n}. Classification
consists in assigning each pixel to one of the K classes of
interest. In the following, each step of the proposed procedure
is described.
A. Pixelwise Classification
The first step consists in performing a probabilistic pixelwise
classification of the hyperspectral image. We propose to use a
support vector machine (SVM) classifier [28] for this purpose.
Other classifiers could be used. However, SVMs are extremely
well suited to classify hyperspectral data [5], [29], [30]. We
refer the reader to [5] and [28] for details on SVMs. The outputs
of this step are the following:
1) classification map, containing class labels for each pixel;
2) probability map, containing probability estimates for
each pixel to belong to the assigned class.
Two techniques for computing probability estimates for mul-
ticlass classification by pairwise coupling are described in [31].
We propose to use one of these methods, which is implemented
in the LIBSVM library [32]. The objective is to estimate, for
each pixel x, the probabilities to belong to each class of interest
p = {p
k
= p (y = k|x),k =1,...,K} . (1)
For this purpose, first, pairwise class probabilities r
ij
≈
p(y = i|y = i or j, x) are estimated using an improved imple-
mentation [33] of [34]
r
ij
≈
1
1+e
A
ˆ
f+B
(2)
where A and B are estimated by minimizing the negative log-
likelihood function using known training data and decision
values
ˆ
f. Furthermore, the probabilities in (1) are computed by
solving the following optimization problem:
min
p
K
i=1
j:j=i
(r
ji
p
i
− r
ij
p
j
)
2
subject to
K
i=1
p
i
=1,p
i
≥ 0 ∀i. (3)
This problem has a unique solution and can be solved by a
simple linear system, as described in [31]. Finally, a probability
map is constructed by assigning to each pixel the maximum
probability estimate max(p
k
), k =1,...,K.
B. Selection of the Most Reliable Classified Pixels
The aim of this step is to choose the most reliable classified
pixels in order to define suitable markers. We propose to use
probability estimates obtained as a result of the pixelwise
classification for this purpose in order to keep the most reliable
classified pixels as markers. A simple way of marker selection
consists in thresholding the probability map. In other words, if
the probability of the considered pixel belonging to the assigned
class k is higher than a given threshold, this pixel is selected to
join the markers. In the resulting map of markers, each marker
pixel is associated with the class defined by the pixelwise
classifier. The marker pixels form connected components in the
map of markers so that each connected component represents
one marker. The main advantage of this technique of marker
selection is its simplicity. However, this method has the fol-
lowing disadvantage: Each marker leads to one region in the
segmentation map. Therefore, we need as many markers as the
desired number of regions. However, if classes k
i
and k
j
are
spectrally similar, pixels belonging to one of those classes have
a quasi-equal probability to belong to each of them. From here,
these classified pixels are not reliable. Therefore, we risk to lose
the regions corresponding to either class k
i
or k
j
in the final
segmentation map. This leads to undersegmentation, which is
highly undesired.
To mitigate this problem, we propose the following method
of marker selection [see the flowchart in Fig. 3(a)].
1) Perform a connected-component labeling of the pixel-
wise classification map. For this purpose, a classical
connected-component algorithm using the union-find
data structure can be used [35].
2) Analyze each connected region as follows.
• If a region is large enough, it should contain a
marker, which is determined as P % of the pixels
within the connected component with the highest
probability estimates.
• If a region is small, it should lead to a marker only
if it is very reliable; a potential marker is formed
by pixels with probability estimates higher than a
defined threshold.
The proposed procedure is deducted from the following
analysis: Based on the results of our previous studies [9], [13],
[14], it is common that almost no undersegmentation is present
in a pixelwise classification map. Therefore, each connected
spatial region from the classification map is analyzed if it
corresponds most probably to the spatial structure or if it is
rather a classification noise [see the illustrative example in
Fig. 3(b)]. If the size of the component is large enough to
consider it as a relevant region, the most reliable pixels within
this region are selected as its marker. If a component contains
only a few pixels, it is investigated if these pixels were classified
to a particular class with a high probability. If this is the case,
the considered connected component represents a small spatial
structure. Thus, a marker associated with this region should
be defined. Otherwise, the component is the consequence of
classification noise, and we tend to eliminate it. Therefore, no
marker within this component is selected. When performing
labeling of connected components for a pixelwise classification
map, we propose to use an eight-neighborhood connectivity.
For the proposed marker selection procedure, the following
parameters must be chosen.
1) A parameter M defining if a region is considered as being
large or small. We propose to use a number of pixels in
1270 IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICS—PART B: CYBERNETICS, VOL. 40, NO. 5, OCTOBER 2010
Fig. 2. Flowchart of the proposed segmentation and classification scheme.
Fig. 3. (a) Flowchart of the proposed marker selection procedure. (b) Illustrative example of the marker selection.
the region (i.e., an area of the region) as a criterion of
the region size. The threshold of the number of pixels
defining if the region is large depends on the resolution
of the image and typical sizes of the objects of interest.
For instance, if the image of the volcano is considered
(experimental results on the volcano image are illustrated
in Section III), where the goal is to classify lavas of
different eruption periods, it is known that the lava of dif-
ferent formations consists of large homogeneous regions.
Therefore, it can be assumed that the regions representing
structures (lavas) in the image scene have a size of at least
10 km
2
. Thus, for an airborne 20-m-resolution image, the
threshold of M =20pixels for dividing the regions in the
groups of large/small ones can be chosen.
2) A parameter P , defining the percentage of pixels within
the large region to be used as markers, depends on the
previous parameter. Since a marker for the large region
must be composed at least of one pixel, the following
condition must be fulfilled: P ≥ 100%/M .
3) The last parameter S, which is a threshold of probability
estimates defining potential markers for a small region,
depends on the probability of the presence of small
structures in the image (which also depends on the image
resolution and the classes of interest) and the importance
of the potential small structures (i.e., what is the cost of
losing the small structures in the classification map). For
instance, if we are interested in determining regions of
different lava formations in the volcano image, the small
objects in the image may have no importance for us, and a
high value of S can be chosen. However, if the classifica-
tion aims at determining regions of sick/damaged plants
in the field, it may be important not to lose any small
region of the damaged species. In this case, the threshold
S must be relaxed.
TARABALKA et al.: SEGMENTATION AND CLASSIFICATION OF HYPERSPECTRAL IMAGES USING MSF 1271
In Section III-D, the dependence of the classification accu-
racies from the chosen parameters for the marker selection is
investigated experimentally. As a conclusion, each connected
set of pixels with the same class in the classification map
provides either one or zero marker. One should stress that a
marker is not necessarily a connected set of pixels: It can
spatially be split into several subsets [see Fig. 3(b)].
C. Construction of an MSF
The previous two steps result in a map of markers defining
regions of interest in the image. The next step consists in the
grouping of all the image pixels into an MSF [22], where each
tree is rooted on a classification-derived marker.
For this purpose, each pixel is considered as a vertex v ∈ V
of an undirected graph G =(V,E,W), where V and E are the
sets of vertices and edges, respectively, and W is a mapping
of the set of edges E into R
+
. Each edge e
i,j
∈ E of this
graph connects a couple of vertices i and j corresponding to
the neighboring pixels (in the following, we simply call vertices
as pixels). Furthermore, a weight w
i,j
is assigned to each edge
e
i,j
, which indicates the degree of dissimilarity between two
pixels connected by this edge. Different dissimilarity measures
can be used for computing weights of edges, such as vector
norms, Spectral Angle Mapper (SAM), and spectral informa-
tion divergence (SID) [36].
The L1 vector norm between two pixel vectors x
i
=
(x
i1
,...,x
iB
)
T
and x
j
=(x
j1
,...,x
jB
)
T
is given as
L1(x
i
, x
j
)=
B
b=1
|x
ib
− x
jb
|. (4)
The SAM distance between x
i
and x
j
determines the spectral
similarity between two vectors by computing the angle between
them. It is defined as
SAM(x
i
, x
j
) = arccos
⎛
⎜
⎜
⎜
⎝
B
b=1
x
ib
x
jb
B
b=1
x
2
ib
1/2
B
b=1
x
2
jb
1/2
⎞
⎟
⎟
⎟
⎠
. (5)
The SID measure [37] computes the discrepancy of proba-
bilistic behaviors between the spectral signatures of two pixels.
It is defined as
SID(x
i
, x
j
)=
B
b=1
q
b
(x
i
) log
q
b
(x
i
)
q
b
(x
j
)
+q
b
(x
j
) log
q
b
(x
j
)
q
b
(x
i
)
(6)
where
q
b
(x
i
)=
x
ib
B
l=1
x
il
. (7)
Furthermore, more complex dissimilarity measures for im-
age segmentation have been proposed in [11] and [38].
Given a connected graph G =(V,E),aspanning tree T =
(V,E
T
) of G is a connected graph without cycles such that
E
T
⊂ E.Aspanning forest F =(V,E
F
) of G is a noncon-
nected graph without cycles such that E
F
⊂ E.
Given a graph G =(V,E,W), the minimum spanning tree
is defined as a spanning tree T
∗
=(V, E
T
∗
) of G such that the
sum of the edge weights of T
∗
is minimal
T
∗
∈ arg min
T ∈ST
⎧
⎨
⎩
e
i,j
∈E
T
w
i,j
⎫
⎬
⎭
(8)
where ST is a set of all spanning trees of G.
Given a graph G =(V,E,W),theMSF rooted on a set of
m distinct vertices {t
1
,...,t
m
} consists in finding a spanning
forest F
∗
=(V, E
F
∗
) of G, such that each distinct tree of F
∗
is
grown from one root t
i
, and the sum of the edge weights of F
∗
is minimal [22]
F
∗
∈ arg min
F ∈SF
⎧
⎨
⎩
e
i,j
∈E
F
w
i,j
⎫
⎬
⎭
(9)
where SF is a set of all spanning forests of G rooted on
{t
1
,...,t
m
}.
In order to obtain the MSF rooted on markers, m additional
vertices t
i
,i=1,...,m, are introduced. Each extra vertex
t
i
is connected by the edge with a null weight to the pixels
representing a marker i. Furthermore, an additional root vertex
r is added and is connected by the null-weight edges to the
vertices t
i
. The minimum spanning tree of the constructed
graph induces an MSF in G, where each tree is grown on a
vertex t
i
; the MSF is obtained after removing the vertex r.An
example of the construction of the MSF rooted on markers is
shown in Fig. 4. Prim’s algorithm can be used for building the
MSF (see Algorithm 1) [39]. The efficient implementation of
the algorithm using a binary min heap (for the implementation
of a min-priority queue) is possible [40]; the resulting time
complexity of the algorithm is O(|E| log |V |).
Algorithm 1 Prim’s Algorithm
Require: Connected graph G =(V,E,W)
Ensure: Tree T
∗
=(V
∗
,E
∗
,W
∗
)
V
∗
= {v}, v is an arbitrary vertex from V
whileV
∗
= V do
Choose edge e
i,j
∈ E with minimal weight such that i ∈
V
∗
and j/∈ V
∗
V
∗
= V
∗
∪{j}
E
∗
= E
∗
∪{e
i,j
}
end while
Each tree in the MSF forms a region in the segmentation
map (by mapping the resulting graph onto an image). Finally, a
spectral-spatial classification map is obtained by assigning the
class of each marker to all the pixels grown from this marker.
Thus, the proposed procedure of the construction of an
MSF from region markers is a region growing method, which
consists of the following steps: First, seed regions are chosen
to belong to the segmentation and classification maps. Then, at
each iteration, a new pixel i is added to the segmentation and
classification maps so that the dissimilarity criterion between
![Fig. 7. Assessment of the robustness of the parameter settings. (a)–(f) Overall and (g)–(l) average classification accuracies as functions of parameters for the marker selection procedure [(a), (d), (g), (j)] P (M = 20, T = 2), [(b), (e), (h), (k)] T (M = 20, P = 20), and [(c), (f), (i), (l)] M (P = 20, T = 5), for the [(a)–(c), (g)–(i)] Indiana and [(d)–(f), (j)–(l)] Hekla images.](/figures/fig-7-assessment-of-the-robustness-of-the-parameter-settings-35gjjo7o.webp)
