Human Perception based Image Retrieval using
Emergence Index and Fuzzy Similarity Measure
Sagarmay Deb
1
and Siddhivinayak Kulkarni
2
1
School of Commerce and Management
Southern Cross University
Military Road
East Lismore NSW 2480, Australia
Sagarmay.deb@scu.edu.au
2
School of Information Technology and Mathematical Science
University of Ballarat
Mt Helen Campus University Drive
PO Box 663, Ballarat
Victoria, 3353, Australia
S.Kulkarni@ballarat.edu.au
ABSTRACT
The main concern dealing with Content-based Image Retrieval
(CBIR) is to bridge the semantic gap. The high level query
posed by the user and low level features extracted by the
machine illustrates the problem of semantic gap. To solve the
problem of semantic gap, this paper presents a hybrid
technique using an emergence index and fuzzy logic for
efficient retrieval of images based on the colour feature.
Emergence Index (EI) is proposed to understand the hidden
meaning of the image. Fuzzy Similarity Measure is developed
to calculate the similarity between the target image and the
images in the database. The images were ranked based on
their similarity along with the fuzzy similarity distance
measure. The preliminary experiments conducted on small set
of images and promising results were obtained.
1.
INTRODUCTION
Content-based image retrieval (CBIR) is a bottleneck of
multimedia database access system. In the last decade, many
image retrieval techniques have been developed based on
colour, shape, texture and spatial locations where retrievals
were done in an automated way with least user interference.
Fuzzy logic based technique has been applied for solving the
problems in CBIR. Fuzzy logic is used for measuring the
similarity between the two images as fuzzy hamming distance,
region groupings, colour histogram as feature extraction
techniques, and posing the queries etc. Fuzzy colour
histogram is proposed by considering the colour similarity of
each pixel’s colour associated to all the histogram bins
through fuzzy-set membership function [1].
Fuzzy logic based colour histogram and their corresponding
fuzzy distances are proposed for the retrieval of colour images
for image database [2]. Fuzzy logic is applied to the traditional
colour histogram for solving the problem of inaccuracy in
typical colour feature. The similarity is defined through a
balanced combination between global and regional similarity
measures incorporating all the features [3].
The Fuzzy Hamming Distance (FHD) is an extension of
Hamming Distance for real valued vectors. Because the
feature space of each image is a real-valued, the fuzzy
Hamming Distance can be successfully used as image
similarity measure. FHD is applied for colour histograms of
the two images. The authors claim that FHD not only
considers the number of different colours but also the
magnitude of this difference [4] [5]. In [6] supports concept-
based image retrieval as well as the inexact match with a
fuzzy triple matching performed when evaluating queries.
In [7] an image is represented by a set of segmented regions
each of which is characterised by a fuzzy feature reflecting
colour, texture and shape properties.
A CBIR system which automatically clusters images using
features of those images which are fuzzy in nature. The
resultant clusters must be described by linguistic variables
which are more meaningful to humans than traditional
approaches, also fuzzy features result in better clustering than
traditional approach. Fuzzy image labelling method that
assigns multiple semantic labels together with confidence
measures to each region in an image [8].
1-4244-1502-0/07/$25.00 © 2007 IEEE ISSNIP 2007359
Figure 1: (a) Original shape, (b) and (c) emergent shapes derived from (a) [21]
Thorough and meaningful image segmentation is essential for
accurate image retrieval, is still a problem. Also finding the
semantic meanings out of an image from low-level features
like colour, shape, texture and spatial locations and connect it
to high-level features like chair, table, car, house etc. is
another unresolved problem. This is because of the inherent
problem of computer perception, as computer cannot replicate
a human mind. Image retrieval can be achieved based on low-
level visual features like colour [9], texture [10], shape [11] or
high level semantics [12] or both [13].
As quite a few commercial and non-commercial models are
developed, none of them made attempts to study the hidden or
implicit meanings of the images. We achieve more accurate
and different search outcomes when hidden meanings are also
taken into account for both precision and recall. For example,
we can consider a square with one diagonal. This is the
explicit or outer meaning of the image. But when we consider
implicit or hidden meanings, we get two triangles in it. This is
what we call emergence. Calculation of similarity between the
target image and the images in database is another important
issue for retrieving the images from the database. Various
distance measures such as city block distance [14], Euclidean
distance [15], histogram intersection distance [16],
Mahalanobis distance have been used in image retrieval. We
propose the calculation of similarity based on fuzzy logic.
Fuzzy logic has been already developed for natural language
queries for CBIR [17]. The performances of some of these
measures have been evaluated on image databases and results
are discussed in this paper.
The paper provides definitions and application of emergence
index in image query processing. The images are segmented
based on major object in an image. Similarity distance is
calculated using fuzzy similarity measure for various colours.
Section 2 details the concept of emergence index, Section 3
describes fuzzy logic based similarity distance measure,
Section 4 explains the experimental results and Section 5
deals with analysis and comparison and conclusion and future
research are described in Section 6.
2.
CONCEPT OF EMERGENCE INDEX
Shape emergence is defined as emergence of single or
multiple shapes from a particular shape. Figure 1 shows
examples of shape emergence. The symbolic representation of
shapes could be defined using infinite maximal lines which
are straight lines with no limit.
I = {N; constraints}, where N is the number of infinite
maximal lines, which effectively define an image I and
constraints are limitations, which define behaviours or
properties that come out from the infinite maximal lines [18].
In order to calculate the emergence index, we developed the
following equation for the emergence index, EI =
f(D,F,V,C,E), where D for domain where the image belongs,
F for features, V for variables which defines the features’
constraints under which the features are defined, C for
constraints and E for emergence characteristics of images.
Any image, static or in motion, could be expressed
semantically in terms of the above-mentioned five parameters
[19].
An application of emergence index in practical problems is in
geographic location. We consider the image of a map of a
township where there is a park, a lake, roads and residential
area. The roads surrounding the park and the lake form the
shape of a bowl. This is the example of an embedded shape
emergence where emergence is a set of the whole image. We
know images are generated in huge numbers. So if we want to
locate this particular image from a table containing large
volume of data, then we can have an input of a bowl. Then
this input of the bowl will find a match with the emergent
shape of the bowl in this image of the
map and select it for
display. This is the advantage of using emergence index [20].
Figure 2 shows the original images and along with segmented
images considering the emergence shape.
Figure 2: Segmentation of Images based on Emergent Shapes
Identification
360
3.
FUZZY LOGIC BASED SIMILARITY MEASURE
All the images in database are segmented based on the colour
regions using emergence index technique. The colour feature
is extracted from each image using colour histogram
techniques and stored in a database. N
p
represents the total
number of pixels in an image. For each colour value (red,
green, blue etc.) the number of the pixels that belong to the
value are recorded and denoted by N
f
where f ∈ F
u
. The
feature representation set F
u
is the set of colours useful for
retrieval. Due to the size of the image collection, it is
necessary to use a database for managing the image content
information. The chief priority is to store this data in such way
as to facilitate the fastest possible retrieval time in order to
make rapid browsing feasible. The numbers of the pixels are
calculated for each colour. The red colour feature component
F
R
is calculated for each colour in an image by the following
formula: F
R
= N
R
/N
P.
N
R
represents the number of red pixels
in an image. Similarly the feature component is calculated for
all the colours.
For image retrieval based on similarity measure, the fuzzy
distance between the two images should be calculated. This
distance is used to retrieve and index images based on the
similarity. Let us consider the extracted features for each
image are in the form of:
I
1
= [I
1
C
1
, I
1
C
2
... I
1
C
i
…I
1
C
n
], where I
1
is the first image in
database and C indicates the colour extracted from each
image. Similarly, these colour features are extracted from each
image and stored in feature database.
The images are represented by an n-dimensional colour
feature vectors. The similarity is calculated for each
component of the feature vectors. Hence for each pair of
images, we have n similarity measures. The global similarity
is given by:
FuzzyAND = Min [(Q
1
, T
1
), (Q
2
, T
2
)…(Q
n
, T
n
)]
FuzzyOR = Max [(Q
1
, T
1
), (Q
2
, T
2
)…(Q
n
, T
n
)],
where Q
1
represents the first colour vector for the query
image, similarly T
1
represents first colour vector for the target
image.
The global similarity measure is calculated by:
∑∑
==
−
=
k
j
n
i
FuzzyOR
FuzzyANDFuzzyOR
DF
00
..
An F.D. stand for Fuzzy Distance, n is the colour feature
vectors and k are the number of images in database. The
similarity measure for each image is calculated in accordance
with the query image and indexed based upon selection sort
algorithm.
4.
EXPERIMENTAL RESULTS
In the preliminary experiments, the prototype was kept simple.
Only 250 images were downloaded from World Wide Web
(WWW) in different categories like flowers, animals,
natural
scene, people etc.
Figure 3: Top five query results for three different images using
emergence index and fuzzy similarity distance
Colour Feature extraction for all the images have been
performed offline. To check the performance of the developed
fuzzy based similarity measure, number of experiments was
conducted. Some of the experimental results are discussed in
this paper. The Figure 3 shows the top five results returned for
a query. The image in the first row represents a query image
and below is the retrieved images with emergence index along
with their fuzzy similarity distance. Similar image has the
least fuzzy similarity distance and it increases from top to
bottom.
5. ANALYSIS AND COMPARISON
The performance of the proposed emergence index and fuzzy
similarity distance was compared with other developed
similarity measures such as Euclidean distance, Chi-square
distance, histogram intersection and city block distance. The
results were compared using precision and recall method
361
Figure 4: Comparison of retrieval performance for various similarity measures
using the consistent criteria and image database for each
similarity measure. Recall signifies the relevant images in the
database that are retrieved in response to a query. Precision is
the proportion of the retrieved images that are relevant to the
query.
Figure 4 shows the retrieval performance based on other
similarity measures. After comparing and analysing all the
similarity measure algorithms, it was observed that each
algorithm except histogram intersection performed relatively
well.
6. CONCLUSION AND FUTURE RESEARCH
Accessing the multimedia databases using emergence index
and fuzzy similarity distance is presented in this paper. The
emergence gives rise to altogether different meaning and
could be used in accessing large databases in more efficient
way. In the second stage, similarity measure has formed very
important step for retrieving the images based on their
features. There are many similarity function exists such as
Euclidean distance, histogram intersection etc. Along with
these existing measures, Fuzzy logic based image similarity
distance has been proposed and used for retrieving the images
based on their colour feature. The proposed similarity learning
is compared with existing similarity measures. Combining the
emergence index and fuzzy logic based similarity measure has
improved the overall retrieval results in CBIR. The future
research will lead to the extension of fusing emergence index
and fuzzy logic for other features of the image.
R
EFERENCES
[1] J. Han and K. Ma, Fuzzy Colour Histogram and Its Use in
Colour Image Retrieval, IEEE Transactions on Image
Processing, Vol. 11, No. 8, August 2002, pp. 944-952.
[2] C. Vertan and N. Boujemaa, Using Fuzzy Histograms
and Distances for Colour Image retrieval, Challenge of
Image Retrieval, Brighton, page numbers not available,
2000.
[3] R. Zhang and Z. Zhang, A Clustering based Approach to
Efficient Image Retrieval, 14
th
IEEE Conference on
Tools with Artificial Intelligence, pp. 339-346, 2002.
[4] M. Ionescu, and A. Ralescu, Fuzzy Hamming Distance in
a Content based Image Retrieval Systems, Proceedings
of FUZZ-IEEE 2004, Budapest, Page numbers not
available, 2004.
[5] M. Ionescu, and A. Ralescu, Image Clustering for a Fuzzy
Hamming Distance based CBIR Systems, Proceedings
of the Sixteen Midwest Artificial Intelligence and
Cognitive Science Conference, MAICS-2005, pp. 102-
108, 2005.
[6] J. Yang, An Image Retrieval Model based on Fuzzy
Triples, Journal on Fuzzy Sets and Systems, Vol.
121, No. 3, pp. 459 – 470, 2001.
[7] Y. Chen and J. Wang, A Region based Fuzzy Feature
Matching Approach to Content based Image Retrieval,
IEEE Transactions on Pattern Analysis and Machine
Intelligence, Vol. 24, No. 9, pp. 1252-1267, 2002.
362
[8] M. Paterno, F. Lim, and W. Leow, Fuzzy Semantic
Labelling for Image Retrieval, International Conference
on Multimedia and Expo, Vol. 2, pp. 767-770, 2004.
[9] G. Pass, R. Zabih, J. Miller, Comparing Images using
Colour Coherence Vector, ACM Multimedia,
Boston, MA, pp. 65-73, 1996.
[10] B. Manjunath, and W. Ma, Texture Features for
Browsing and Retrieval of Image Data”, IEEE
Transactions on Pattern Analysis and Machine
Intelligence, Vol. 18, No. 8, August 1996.
[11] M. Safar, C. Shahabi, C., and X. Sun, Image Retrieval by
Shape: A Comparative Study, Proceedings of IEEE
International Conference on Multimedia and Exposition
(ICME), USA, 2000.
[12] D. Forsyth, Finding Pictures of Objects in Large
Collections of Images”, Report of the NSF/ARPA
Workshop on 3D Object Representation for Computer
Vision, pp. 335, 1996.
[13] R. Zhao, and W. Grosky, Bridging the Semantic Gap in
Image Retrieval, Distributed Multimedia Databases:
Techniques and Applications, Idea Group Publishing,
Hershey, PA 17033-1240, USA, pp. 14-36, 2002.
[14] M. Stricker and M. Orengo, Similarity of Colour Images.
In the Proceedings of SPIE: Storage and Retrieval for
Image and Video Databases, Vol. 2420, pp. 381-392,
1995.
[15] H. Voorhees and T. Poggio, Computing Texture
Boundries from Images, Nature, pp. 333-367, 1988.
[16] J. Smith, Integrated Spatial and Feature Image System:
Retrieval, Analysis and Compression, PhD Thesis,
Columbia University, 1997.
[17] B. Verma and S. Kulkarni, Fuzzy Logic Based
Interpretation and Fusion of Colour Queries, Journal of
Fuzzy Sets and Systems, Volume 147, No. 1, pp. 99-118,
2004.
[18] J. Gero, Shape Emergence and Symbolic Reasoning
using Maximal Lines Unpublished Notes, Design
Computing Unit, Department of Architectural and Design
Science, University of Sydney, Sydney, 1992.
[19] S. Deb, and Y. Zhang, Concepts of Emergence Index in
Image Databases, Distributed Multimedia Databases:
Techniques and Applications, Idea Group Publishing,
Hershey, PA17033-1240, USA, pp. 73-88, 2001.
[20] S. Deb, and Y. Zhang, Retrieval of Geographic Location
with Emergence Index in Multimedia, Second
International Workshop on Intelligent Multimedia
Computing and Networking, Durham, North Carolina,
USA, pp. 976-979, March 8-13, 2002.
[21] J. Gero, Visual Emergence in Design Collaboration, Key
Center of Design Computing, International Journal of
CADCAM and Computer Graphics, Vol. 8, No. 3, 349-
357, 1993.
363