Hybrid technique for colour image classification and efficient retrieval
based on fuzzy logic and neural networks. Paper presented at
Neural Networks (IJCNN), The 2012 International Joint Conference held 10-
15 June, 2012
Published version available at
http://dx.doi.org/10.1109/IJCNN.2012.6252587
Copyright 2012 IEEE Personal use of this material is permitted.
Permission from IEEE must be obtained for all other users, including
reprinting/ republishing this material for advertising or promotional
purposes, creating new collective works for resale or redistribution to
servers or lists, or reuse of any copyrighted components of this work in
other works
COPYRIGHT NOTICE
UB ResearchOnline
http://researchonline.ballarat.edu.au
Hybrid Technique for Colour Image Classification
and Efficient Retrieval based on Fuzzy Logic and
Neural Networks
Ranisha Fernando
School of Science, Information Technology and
Engineering
University of Ballarat
Ballarat, Australia
rani3fer@yahoo.com
Siddhivinayak Kulkarni
School of Science, Information Technology and
Engineering
University of Ballarat
Ballarat, Australia
S.Kulkarni@ballarat.edu.au
Abstract— Developments in the technology and the Internet have
led to increase in number of digital images and videos.
Thousands of images are added to WWW every day. To retrieve
the specific images efficiently from database or from Internet is
becoming a challenge now a day. As a result, the necessity of
retrieving images has emerged to be important to various
professional areas. This paper proposes a novel fuzzy approach
to classify the colour images based on their content, to pose a
query in terms of natural language and fuse the queries based on
neural networks for fast and efficient retrieval. Number of
experiments was conducted for classification and retrieval of
images on sets of images and promising results were obtained.
The results were analysed and compared with other similar
image retrieval system.
Keywords-image retrieval; fuzzy logic; neural networks;
classification
I. INTRODUCTION
Advances in technology have provided creation, storage
and share of the digital information including images and
videos. The rapid increase in digital information has led to its
own problems in the image retrieval process.
Images are vital part of everyday life in many professional
areas. Further, colour images have disctintive features which
are used in the research to extract the features for matching
purposes. Lot of research interest has been arisen into this area
of image retrieval which is done automatically on the basis of
colour, texture, shape or abstract features which a technology
is referred to as content-based image retrieval or CBIR.
Content based image retrieval application areas are
commercial areas, crime prevention including fingerprint and
face recognition, intellectually property including trade
marking, journalism and advertising and web searching.
Most of the image retrieval systems are based on example
image as query. However, this research proposes query in
terms of natural language content as very low, low, medium,
high and very high. These natural language terms are used
along with fuzzy logic to acquire the desired results. The
importance of this research is that the users are given freedom
to pose the query in terms of natural language and a novel
technique of fusion of queries using neural networks has been
proposed. Experiments were conducted on large image dataset
and promising results were obtained.
The rest of paper is organized as follows. Section 2 deals
with literature review and related work in this area, Section 3
describes the fuzzy classification, Section 4 deals with neural
based fusion of classes, Section 5 details experimental results
while Section 6 compares the result with other existing CBIR
system and Section 7 concludes the paper.
II. RELATED WORK
Most of the Content-based Image Retrieval (CBIR)
systems such as QBIC [1], Virage [2], Photobook [3] and
Netra [4] use a weighted linear method to combine similarity
measurements of different feature classes. Stricker and
Dimai’s method [5] segments each image into five partially
overlapping fuzzy regions and extracts first two colour
moments of each region both weighted by membership
functions of the region to form a feature vector for the image.
A colour space for CBIR is presented which provides both the
ability to measure similarity using fuzzy logic and
psychologically based set theoretic similarity measurement.
These properties are shown to be equal or superior to the
conventional colour space. C. Vertan et al. propose fuzzy
colour histogram that classifies fuzzy techniques as crude
fuzzy, fuzzy paradigm based, fuzzy aggregational and fuzzy
inferential [6].
The Fuzzy Hamming Distance (FHD) is an extension of
Hamming Distance for real valued vectors. Because the
feature space of each image is a real-valued, the fuzzy
Hamming Distance can be successfully used as image
similarity measure. FHD is applied for colour histograms of
the two images. The authors claim that FHD not only
considers the number of different colours but also the
magnitude of this difference [7]. In [8] supports concept-based
image retrieval as well as the inexact match with a fuzzy triple
matching performed when evaluating queries.
In [9] an image is represented by a set of segmented
regions each of which is characterised by a fuzzy feature
reflecting colour, texture and shape properties. The
resemblance between two images is then defined as the overall
similarity between two families of fuzzy features and
quantified by unified feature matching. Non-Boolean fuzzy
and similarity predicates are used to rank tuples according to
fuzzy based algebra [10]. Soft queries in image retrieval
systems present the use of soft computing and user defined
classifications in multimedia database systems for content
based queries [11]. A CBIR system which automatically
clusters images using features of those images which are fuzzy
in nature. The resultant clusters must be described by
linguistic variables which are more meaningful to humans
than traditional approaches, also fuzzy features result in better
clustering than traditional approach. Fuzzy image labelling
method that assigns multiple semantic labels together with
confidence measures to each region in an image [12]. E.
Walker describes several aspects of Internet information
retrieval where fuzzy logic can be applied [13]. Not much
work has been done in the area of fuzzy logic based linguistic
queries for image retrieval. Fuzzy logic has impressive power
to represent the queries in terms of natural content of the
image. The next section describes the proposed technique of
posing the queries in terms of natural language for colour and
texture features.
Colour and texture features are extracted using colour
median filtering and computed from gray-scale version of the
image respectively. Fuzzy C-means clustering has been used
to retrieve images in this research [14]. For each images,
colour and textures features are extracted and clustered to
generate regions which demonstrate similar features. Fuzzy
logic is used in the image retrieval process. There are three
major stages described in the research are using fuzzy logic
variable to describe similarity degree of features, using fuzzy
logic to describe the weight assignment and proposing an
improvement to the Average Area Histogram [15]. According
to the paper [16], a new histogram creation method has been
proposed and L*a*b* colour space components have been
used which are also known as fuzzy sets. The histogram is
obtained through linking of fuzzy sets according to 27 fuzzy
rules. A small number of bins are used in this research, only
10, to increase the efficiency of the program. The paper [17]
suggests the use of fuzzy logic in information retrieval on the
internet. The paper suggests the use of several fuzzy logic
techniques in different areas. The feature extraction problem
can use fuzzy C-means clustering. In the research done in [18]
has used an approach of both fuzzy logic and natural language
query. According to their research, the use of natural language
is compared to an intelligent approach which can be used to
interpret human language.
III. FUZZY CLASSIFICATION
Most of the image retrieval systems use the features those
are specific for an application. Colour, texture, shape and
object are the most prominent features used for retrieving
images in CBIR. In this feature extraction stage, colour feature
is extracted as RGB (Red, Green and Blue) values. These
extracted RGB values are converted into HSV (Hue,
Saturation and Value) values. Hue represents the colour (red,
blue), Saturation represents the amount of colour (bright red,
light red), and Value represents the amount of light (lightness
and darkness of a colour).
Colour {Red, Orange, Yellow, Green, Cyan, Blue, Purple,
Magenta, Pink, White, Black}
Content {Very low, Low, Medium, High, Very high}
The query is consisted of a colour and type of content in
natural language terms. There are total 55 classes (11 colours
and 5 content types). In this approach, the user has the
opportunity to select the colour and the content type in natural
language terms rather than having to use technical terms. The
user also has the advantage of retrieving images without
having to posses a similar image. Membership function for
various fuzzy content is shown in Figure1.
Membership Function
0
0.2
0.4
0.6
0.8
1
0 10 20 30 40 50 60 70 80 90 100
Percentage
Degree of Membership
Very Low
Low
Very High
High
Medium
No Colour
Figure 1. Membership Function used for Fuzzy Image Classification
F
content
(percentage) = 1 if percentage <= 10
0 if percentage <= 40
40 – percentage
(40 – 30)
Figure 1. Membership Function Equation
The above equation is used to calculate content type in terms
of fuzzy logic. For the above example equation, if the
percentage is between 10% and 40%, the range from 0 < x <
0.5 is classified as low and 0.5 < x < 1 is classified as medium.
As the above membership function, the following table and
figure 20 contains the fuzzy term for the relevant range of
percentage. Example shown below shows the procedure to
calculate the degree of membership and decide the correct
fuzzy term.
TABLE I. Fuzzy Terms
IV. NEURAL BASED FUSION OF CLASSES
It is very important to learn the meaning of the classes for
fusion of queries. A neural network based technique is the best
solution to learn those classes. Different neural network
algorithms can be categorized by, for example, the learning
method and architecture of the network. The supervised
learning neural network is efficient to learn the colours and
content types. Concept of neural network ensemble has been
used to implement the fusion of classes. Neural Network
Ensembles (NNE) divide the data into smaller areas and
becomes easier to learn the meaning of each class. NNE is
robust compared to single neural network for processing large
amount of data and therefore produces better final decision.
Separate neural networks were formed for each of the content
type ranging from very low to very high.
Fuzzy class for colour contents for each image:
very low [0.05, 0.1]
low [0.11, 0.35]
medium [0.36, 0.65]
high [0.66, 0.80]
very high [0.81, 1.0]
As each of the neural networks is learned on classes and not
on the database, it avoids retraining of the neural networks.
Figure 2. Neural based Fusion
NN indicates the Neural Network designed for a specific
content.
V. EXPERIMENTAL RESULTS
A. Image Dataset Preparation and Feature Extraction
Image Dataset consists of five thousand images. These
images are taken from various categories such as images of
babies, beaches, birds, boats, cars, dogs, fireworks, flowers,
landmarks, nature, planes, planets, sunsets, waterfalls and
weddings. Number of experiments were conducted by varies
queries and some of the results are mentioned in this section.
Table II shows percentage of colours for an image in
database. The image contains very high content for cyan and
other colours are in low content category.
TABLE II. EXAMPLE IMAGE AND COLOUR CONTENTS
Image
Percentage
Red = 3.49%
Yellow = 2.53%
Green = 5.38%
Cyan = 82.94%
Blue = 5.28%
Magenta = 0.37%
B. Image Retrieval for Single Query
In the image retrieval section, the images retrieved are
based on the query submitted by the user. In the below
example, the query submitted was medium as the content type
and blue as the desired colour.
Percentage
Fuzzy Logic Term
0% – 0.5%
No Significant Colour
0.5% – 10%
Very Low
11% – 35%
Low
36% – 65%
Medium
66% – 80%
High
81% – 100%
Very High
Classified Images for (Very High and
White)
Figure 3. Results for Single Query: White + Very High
The percentage of blue is also displayed at the bottom of
each image. These images are ranked in descending order and
retrieved accordingly. For ease of display, only top four
images are displayed.
C. Fusion of Queries using Neural Networks
The natural language query is used to match with the
image in the image set and the relevant images will be
classified into the query class. First experiments were
conducted on single colour and query type and later results
were fused using combination of neural networks based on
content type. The classified image will be retrieved and shown
in descending order long with their percentage for the relevant
colour. For example, if user provides Medium + Blue and
Medium Green, the names of the images are listed with the
percentages which have a medium content of Blue and Green
colours.
The images are retrieved with their percentage. Each
percentage is used for indexing. Sorting algorithm is used and
the percentages are sorted in descending order. The classified
images are then displayed in descending order.
Figure 4. Image Retrieval Results using Multiple Queries
VI. ANALYSIS AND COMPARISON
This section discusses analysis and comparison of the
results obtained in the last section, experimental results. The
feature extraction is done to significant number of images
(about five thousand) to check the accuracy of the proposed
system. The retrieval is also done using a significant number
of images which are the same images used in feature
extraction.
Proposed CBIR system was compared with IBM’S Query
By Image Content) QBIC. Firstly, the required colour is
selected and retrieval is performed.
Classified Retrieved Images
P = 98.8%
P = 85.9%
P = 85.7%
P = 85.4%
P = 53.8%
P = 51.4%
P = 52.1%
P = 51.2%
P = 50.1%
P = 48.6%
P = 48.2%
P = 47.9%