Image Retrieval Based on Fuzzy Mapping of Image Database and Fuzzy
Similarity Distance
Siddhivinayak Kulkarni
School of Information Technology and Mathematical Sciences
University of Ballarat, P. O. BOX 663, Ballarat, Victoria, 3353, Australia
E-mail: S.Kulkarni@ballarat.edu.au
Abstract
The on-line image retrieval process consists of a
query example image, given by the user as an input,
from which low-level image features are extracted.
These image features are used to find images in the
database which are most similar to the query image. A
drawback, however, is that these low level image
features are often too restricted to describe images on
a conceptual or semantic level. The gap between the
high level query from the user and low level features
extracted by a computer is known as the semantic gap.
Translating or converting the question posed by a
human to the low level features seen by the computer
illustrates the problem in bridging the semantic gap.
This paper proposes a novel fuzzy approach for
mapping the fuzzy database while extracting the colour
features from image and assigning the weights to this
fuzzy content when calculating the similarity between
the query image and the images in database. Number
of experiments was conducted on a small colour image
database and promising results were obtained.
1. Introduction
The size of the digital image collection is
increasing very rapidly due to the advancement in
technological devices. These images are stored
digitally and transmitted over the Internet at a very
high speed. To retrieve the images based on their
content effectively and efficiently further processing of
the images is essential. But how to retrieve the images
based on their content? There are few image retrieval
systems developed commercially as well as
academically.
Most of the Content-based Image Retrieval (CBIR)
systems such as QBIC [1], Virage [2], Photobook [3]
and Netra [4] use a weighted linear method to combine
similarity measurements of different feature classes.
QBIC [1] executes the queries by calculating the
similarity between the pre-extracted features of the
images in a database. QBIC allows queries based on
example images, user-constructed sketches or/and
selected colour and texture patterns. The percentage of
a specific colour in an image is adjusted by moving
sliders. To perform a query in Photobook [3], the user
selects some images from the grid of still images
displayed and/or enters annotation filter. The images
obtained with the query are refined to make another
search. VisualSEEK [5] determines the similarity by
measuring image regions using both colour parameters
and spatial relationships. To pose a query, user
sketches a number of positions and dimensions them
on the grid and selects a colour for each region. Also,
the user can indicate boundaries for location and size
and spatial relationships between regions. Netra [4]
depends upon image segmentation to carry out region
based searches that allow the user to select example
regions and lay emphasis on image attributes to focus
the search. The user can select any one image as query
image from 2500 images, clustered into 25 classes and
100 images for each class. The images are segmented
into various homogenous regions and the user can
select any one of the region for possible matching
based on colour, texture, spatial location and shape.
The main motivation for the development of this
system is that region-based search improves the quality
of the image retrieval. Therefore the system
incorporates an automated region identification
algorithm. Region-based querying is also used in
Blobworld [6] where global histograms are shown to
perform comparatively poorly on images containing
distinctive objects. The user first selects a category,
which already limits the search space. In an initial
image, the user selects a region (blob), and indicates
the importance of the blob. Next, the user indicates the
importance of each blob based on fuzzy terms such as
‘not’, ‘somewhat’, ‘very’. Multiple regions are used for
querying. In Chabot [7], the user is presented with a
list of search criteria such as keywords, colours,
photographer etc. The colour criterion offers the
options in terms of specific content of the each colour.
The concept in an image is demonstrated by a
6th IEEE/ACIS International Conference on Computer and Information Science (ICIS 2007)
0-7695-2841-4/07 $25.00 © 2007
descriptive keyword and specific colour related to the
keyword.
Fuzzy logic offers a good solution for posing a
query in terms of natural language based on the various
features of an image [8]. Fuzzy logic has been
extensively used at various stages of image retrieval
such as region groupings within the images as a feature
extraction technique, fuzzy image segmentation,
recognition of fuzzy objects etc. Fuzzy logic is
proposed for the computation of fuzzy colour
histogram as well as posing the queries in CBIR. Han
and Ma [9] propose a fuzzy colour histogram that
permits to consider colour similarity across different
bins and the colour dissimilarity between the different
bins. Colour naming system proposed by Sugano [10]
describes a technique to convert colour into set of
words such as dark red by using fuzzy membership
function to define saturation and lightness for a given
hue. The problem of image indexing and retrieval of
colour images is demonstrated in [11]. The lightness
and saturation are represented through linguistic
qualifiers also defined in fuzzy way. This paper
proposes a novel technique based on fuzzy logic to
reduce the semantic gap. Colour image features are
mapped to form image feature database and weights
are used to calculate the similarity based on fuzzy logic
between the two images. The rest of the paper is
organised as follows: Section 2 proposes fuzzy
mapping of image feature database, Section 3 details
the similarity function based on fuzzy logic,
experimental results for colour image retrieval are
discussed in Section 4, these results are compared and
analysed in Section 5 and the paper is concluded in
Section 6.
2. Fuzzy mapping
Colour feature extraction forms the basis of colour
image retrieval. The distribution of colour is a useful
feature for image representation. Colour distribution,
which is best represented as a histogram of intensity
values, is more appropriate as a global property which
does not require knowledge of how an image is
composed of different objects. So this technique works
extremely well to extract global colour components
from the images. A colour histogram technique is used
for extracting the colours from the images. The colour
of any pixel may be represented in terms of the
components of red, green and blue values. These
histograms are invariant under translation and rotation
about the view axis and change only under the change
of angle of view, change in scale and occlusion.
Therefore, the colour histogram is a suitable
quantitative representation of image content.
Let F
S
denote the set of features used to represent
colour content, F
S
= {colour}. The feature
representation set of colours is rep (colour) = {red,
green, blue, white, black, yellow, orange, pink,
purple}.
An image histogram refers to the probability mass
function of the image intensities. This is extended for
colour images to capture the joint probabilities of the
intensities of the three-colour channels. More formally,
the colour histogram is defined by
{}
bBgGrRprobNh
bgr
===⋅= ,, wher
e R, G, B represent the three-colour channels and N is
the number of pixels in an image. These RGB values
are converted into Hue [0, 360], Saturation [0, 1] and
Value [0, 1].
Maximum and minimum values for RGB were
calculated. Saturation (S) is the ratio of the difference
between maximum and minimum values to the
maximum value [12]. After getting these terms, HSV
were calculated.
Algorithm RGB_to_HSV (r, g, b:real; var h, s, v:real)
{Given: r, g, b, each in [0, 1].
Desired: h in [0,360), s and v in [0, 1] expect if s=0,
then h=UNDEFINED
begin
Red -> val[RED] > val[GREEN] + val[BLUE]
Green -> val[GREEN] > val[BLUE] && val[Green] >
val[RED]
BLUE -> val[BLUE] > 0.5 (val[RED] + val[GREEN])
WHITE -> val[RED] > 200 && val[BLUE] > 200 &&
val[GREEN]> 200
BLACK -> val[RED] < 30 && val[GREEN] < 30 &&
val[BLUE] <30
YELLOW -> h > 42 && h < 62 && s> 0.6 && v >
0.95
ORANGE -> h > 2 && h < 40 && s > .7 && v > .93
PINK-> h > 320 && s < .5
PURPLE -> (h > 320 && h < 330) && (s > .50) &&
(v > .60 && v < .8)
end
The fuzzy contents of the image are expressed as
follows:
F
contents
= {verylarge, large, ratherlarge, medium,
rathersmall, small, verysmall}
Instead of having numerical feature values, the
values are mapped based on fuzzy logic and stored in
feature database. Figure 2 describes the steps of storing
the fuzzy content of an image.
6th IEEE/ACIS International Conference on Computer and Information Science (ICIS 2007)
0-7695-2841-4/07 $25.00 © 2007
Figure 2. Fuzzy content representation for an image
Fuzzy terms for colour contents for each image:
verysmall ->[0, 0.3]
small -> [0, 0.4]
rathersmall -> [0.25, 0.45]
medium -> [0.30, 0.70]
ratherlarge -> [0.55, 0.75]
large -> [0.60, 1]
verylarge-> [0.70, 1]
3. Image similarity
Similarity measurement plays a vital role in content-
based image retrieval (CBIR), since without this
concept of similarity measurement; the retrieval of
images from a database would not be possible. After
the process of feature extraction has been carried out
on an image database, the stored image feature content
must be compared in terms of similarity taking into
account either colour, texture, or shape features. The
use of this extracted feature data allows for a database
to be indexed automatically according to its extracted
features, rather than having manually index the entire
database by keywords for example.
In this research, fuzzy logic based similarity is
proposed between the two images. The weights are
assigned to fuzzy colour content during the calculation
of similarity between the two images.
Assigning the weights for each fuzzy content term:
verysmall -> 0.1, small -> 0.25, rathersmall -> 0.4,
medium -> 0.55, ratherlarge -> 0.70, large -> 0.85,
verylarge-> 1
A fuzzy intersection is the lower membership in both
sets of each element. The fuzzy intersection of two
fuzzy sets A and B on universe of discourse X:
µ
A∩B(x) = min [
µ
A(x),
µ
B(x)] =
µ
A(x) ∩
µ
B(x),
where x∈X
The union is the largest membership value of the
element in either set. The fuzzy operation for forming
the union of two fuzzy sets A and B on universe X can
be given as:
µA∪B(x) = max [µA(x), µB(x)] = µA(x) ∪ µB(x),
where x∈X
Finally the fuzzy similarity distance is | µA∪B(x)-
µ
A∩B(x) |
Based on the fuzzy image features and similarity
measure, experiments were conducted on image
database. Next section describes the experimental
results.
4. Experimental results
To test the effectiveness of this proposed system,
the preliminary experiments were conducted on a small
colour image database. Around 1000 real world images
were downloaded from various websites. This image
database contains a wide variety of images, like the
images of flowers, scenery, animals, mountains etc.
Most of the images are in jpg format. All the nine
colours were extracted from each image; those colours
were converted into fuzzy terms before storing them in
database.
To check the performance of the developed fuzzy
based similarity measure, colour features were
extracted and fuzzy contents were stored in database
for each image. Some of the experimental results are
discussed in this paper. Figure 3 shows the top eleven
results returned for a query. The image-1 represents a
query image along with the retrieved images along
with their fuzzy distance.
5. Analysis and comparison
In order to compare the performance of fuzzy
image features and learning similarity measure based
on fuzzy logic, various distance formulae were
implemented and tested. The results are analysed using
precision and recall method using the consistent
criteria and image database for each similarity
measure. Recall signifies the relevant images in the
database that are retrieved in response to a query.
Precision is the proportion of the retrieved images that
are relevant to the query. More precisely, let A be the
set of relevant items, let B the set of retrieved items
and a, b, c and d are given in Figure 4. In the picture, a
stands for 'retrieved relevant' images, b for 'retrieved
irrelevant' images, c for 'unretrieved relevant' images
and d for 'unretrieved irrelevant' images. Then recall
and precision are defined as the following conditional
probabilities [13].
6th IEEE/ACIS International Conference on Computer and Information Science (ICIS 2007)
0-7695-2841-4/07 $25.00 © 2007
Figure 4. Sets for explaining retrieval
effectiveness
,
)(
)(
)(
ca
a
AP
BAP
ABPrecall
+
===
∩
,
)(
)(
)(
ba
a
BP
BAP
BAPprecision
+
===
∩
For 1000 images, 82% of the recall and 87%
precision has been achieved. With these conditions,
image retrieval is said to be more effective if precision
values are higher at the same recall values. Figure 5
shows the retrieval performance based on other
similarity measures. Every similarity functions perform
well except the histogram intersection. Euclidean
distance has performed better than proposed fuzzy
based similarity measure. The histogram intersection
distance filters out the irrelevant elements in the
matching of two feature vectors. Euclidean distance
has performed better than fuzzy distance and has been
used in many image retrieval systems to compare the
two images based on their features. City block distance
and chi-square are also desirable measures in terms of
both retrieval effectiveness and efficiency.
6. Conclusion
This paper presents a problem of semantic gap, low
level features extracted from image and high level
query expressed by the user. It is very important to
reduce this semantic gap to achieve the better retrieval
results. The paper proposes a novel approach of fuzzy
mapping on image database; therefore instead of
storing the actual numerical values in database, fuzzy
terms are stored for each colour for the image. While
calculating the similarity based on query image, these
terms are converted into numeric weights. Fuzzy based
similarity function is used to calculate the similarity
between the two images. Experiments were conducted
on small image database and promising results were
obtained.
7. References
[1]
M. Flickner, H. Sawhney, W. Niblack, J. Ashley, Q.
Huang, B. Dom, M. Gorkani, J. Hafner, D. Lee, D.
Petkovic, D. Steele and P. Yanker, Query By Image
and Video Content: The QBIC System, IEEE
Computer, Vol. 28, Number 9, pp. 23-32, September
1995.
[2] A. Gupta, Visual Information Retrieval: A Virage
Perspective, Technical Report Revision 4, Virage Inc,
San Diego, CA 92121, 1996, URL:
http://www.virage.com/wpaper.
[3] A. Pentland, R. Picard and S. Sclaroff, Photobook:
Content-based Manipulation of Image Databases,
International Journal of Computer Vision, Vol. 3, pp.
233-254, 1996.
[4] W. Ma and B. Manjunath, NETRA: A Toolbox for
Navigating Large Image Databases, Journal of ACM
Multimedia Systems, Vol. 7, Number 3, pp.184-198,
1999.
[5] J. Smith and S. Chang, Querying by Colour Regions
Using the VisualSEEK Content-Based Visual Query
System, Chapter in Intelligent Multimedia Information
Retrieval, 1996, URL:
http://www.ctr.columbia.edu/papers_advent/96/smith96
d.html.
[6] C. Carson, S. Belongie, H. Greenspan and J. Malik,
Blobworld: Image Segmentation using Expectation-
Maximization and Its Application to Image Querying,
Journal of Pattern Analysis and Machine Intelligence,
1998, URL: http://
elib.cs.berkeley.edu/carson/papers/pami.html.
[7] V. Ogle and M. Stonebraker, Chabot: Retrieval from a
Relational Database of Images, IEEE Computer,
Volume 28, Number 9, pp. 40-48, 1995.
[8] B. Verma and S. Kulkarni, Fuzzy Logic Based
Interpretation and Fusion of Colour Queries, Journal of
Fuzzy Sets and Systems, Volume 147, Number 1, pp.
99-118, 2004.
[9] J. Han and K. Ma, Fuzzy Colour Histogram and its Use
in Colour Image Retrieval, IEEE Transactions on
Image Processing, Volume 11, Number 8, pp. 944-952,
2002.
[10] N. Sugano, Colour Naming System using Fuzzy Set
Theoretical Approach, Proceedings of 10
th
IEEE
International Conference on Fuzzy Systems, Volume 1,
pp. 81-84, 2001.
[11] A. Younes, I. Truck. H. Akdag, Image Retrieval using
Fuzzy Representation of Colours, Journal of Soft
Computing, Volume 11, pp. 287-298, 2007.
[12] J. Foley, A. Dam, S. Feiner and J. Hughes, Computer
Graphics Principles and Practice, Addison-Wesley
Publishing Company, 1990.
[13] J. Smith and S. Chang, Tools and Techniques for
Colour Image Retrieval", In Symposium on Electronic
Imaging: Science and Technology - Storage &
Retrieval for Image and Video Databases IV, Vol.
2670, San Jose, CA, February 1996.
6th IEEE/ACIS International Conference on Computer and Information Science (ICIS 2007)
0-7695-2841-4/07 $25.00 © 2007
Figure 1. Fuzzy membership function for normalized colour contents
Figure 3. Query image (image-1) and retrieved images (image-2 to image-12)
6th IEEE/ACIS International Conference on Computer and Information Science (ICIS 2007)
0-7695-2841-4/07 $25.00 © 2007