Showing papers on "Content-based image retrieval published in 2006"

Asymmetric bagging and random subspace for support vector machines-based relevance feedback in image retrieval

Abstract: Relevance feedback schemes based on support vector machines (SVM) have been widely used in content-based image retrieval (CBIR). However, the performance of SVM-based relevance feedback is often poor when the number of labeled positive feedback samples is small. This is mainly due to three reasons: 1) an SVM classifier is unstable on a small-sized training set, 2) SVM's optimal hyperplane may be biased when the positive feedback samples are much less than the negative feedback samples, and 3) overfitting happens because the number of feature dimensions is much higher than the size of the training set. In this paper, we develop a mechanism to overcome these problems. To address the first two problems, we propose an asymmetric bagging-based SVM (AB-SVM). For the third problem, we combine the random subspace method and SVM for relevance feedback, which is named random subspace SVM (RS-SVM). Finally, by integrating AB-SVM and RS-SVM, an asymmetric bagging and random subspace SVM (ABRS-SVM) is built to solve these three problems and further improve the relevance feedback performance

Content-Based Image Retrieval: Theory and Applications.

Abstract: Advances in data storage and image acquisition technologie s have enabled the creation of large image datasets. In this scenario, it is necess ary to develop appropriate information systems to efficiently manage these collect ions. The commonest approaches use the so-called Content-Based Image Retrieval (CBIR) systems . Basically, these systems try to retrieve images similar to a user-define d sp cification or pattern (e.g., shape sketch, image example). Their goal is to suppor t image retrieval based on contentproperties (e.g., shape, color, texture), usually encoded into feature vectors . One of the main advantages of the CBIR approach is the possibi lity of an automatic retrieval process, instead of the traditional keyword-bas ed approach, which usually requires very laborious and time-consuming previous annot ation of database images. The CBIR technology has been used in several applications su ch as fingerprint identification, biodiversity information systems, digital librar ies, crime prevention, medicine, historical research, among others. This paper aims to introduce the problems and challenges con cerned with the creation of CBIR systems, to describe the existing solutions and appl ications, and to present the state of the art of the existing research in this area.

Direct kernel biased discriminant analysis: a new content-based image retrieval relevance feedback algorithm

Abstract: In recent years, a variety of relevance feedback (RF) schemes have been developed to improve the performance of content-based image retrieval (CBIR). Given user feedback information, the key to a RF scheme is how to select a subset of image features to construct a suitable dissimilarity measure. Among various RF schemes, biased discriminant analysis (BDA) based RF is one of the most promising. It is based on the observation that all positive samples are alike, while in general each negative sample is negative in its own way. However, to use BDA, the small sample size (SSS) problem is a big challenge, as users tend to give a small number of feedback samples. To explore solutions to this issue, this paper proposes a direct kernel BDA (DKBDA), which is less sensitive to SSS. An incremental DKBDA (IDKBDA) is also developed to speed up the analysis. Experimental results are reported on a real-world image collection to demonstrate that the proposed methods outperform the traditional kernel BDA (KBDA) and the support vector machine (SVM) based RF algorithms

A unified log-based relevance feedback scheme for image retrieval

Abstract: Relevance feedback has emerged as a powerful tool to boost the retrieval performance in content-based image retrieval (CBIR). In the past, most research efforts in this field have focused on designing effective algorithms for traditional relevance feedback. Given that a CBIR system can collect and store users' relevance feedback information in a history log, an image retrieval system should be able to take advantage of the log data of users' feedback to enhance its retrieval performance. In this paper, we propose a unified framework for log-based relevance feedback that integrates the log of feedback data into the traditional relevance feedback schemes to learn effectively the correlation between low-level image features and high-level concepts. Given the error-prone nature of log data, we present a novel learning technique, named soft label support vector machine, to tackle the noisy data problem. Extensive experiments are designed and conducted to evaluate the proposed algorithms based on the COREL image data set. The promising experimental results validate the effectiveness of our log-based relevance feedback scheme empirically.

Multitraining Support Vector Machine for Image Retrieval

Abstract: Relevance feedback (RF) schemes based on support vector machines (SVMs) have been widely used in content-based image retrieval (CBIR). However, the performance of SVM-based RF approaches is often poor when the number of labeled feedback samples is small. This is mainly due to 1) the SVM classifier being unstable for small-size training sets because its optimal hyper plane is too sensitive to the training examples; and 2) the kernel method being ineffective because the feature dimension is much greater than the size of the training samples. In this paper, we develop a new machine learning technique, multitraining SVM (MTSVM), which combines the merits of the cotraining technique and a random sampling method in the feature space. Based on the proposed MTSVM algorithm, the above two problems can be mitigated. Experiments are carried out on a large image set of some 20 000 images, and the preliminary results demonstrate that the developed method consistently improves the performance over conventional SVM-based RFs in terms of precision and standard deviation, which are used to evaluate the effectiveness and robustness of a RF algorithm, respectively

Similarity-based online feature selection in content-based image retrieval

Abstract: Content-based image retrieval (CBIR) has been more and more important in the last decade, and the gap between high-level semantic concepts and low-level visual features hinders further performance improvement. The problem of online feature selection is critical to really bridge this gap. In this paper, we investigate online feature selection in the relevance feedback learning process to improve the retrieval performance of the region-based image retrieval system. Our contributions are mainly in three areas. 1) A novel feature selection criterion is proposed, which is based on the psychological similarity between the positive and negative training sets. 2) An effective online feature selection algorithm is implemented in a boosting manner to select the most representative features for the current query concept and combine classifiers constructed over the selected features to retrieve images. 3) To apply the proposed feature selection method in region-based image retrieval systems, we propose a novel region-based representation to describe images in a uniform feature space with real-valued fuzzy features. Our system is suitable for online relevance feedback learning in CBIR by meeting the three requirements: learning with small size training set, the intrinsic asymmetry property of training samples, and the fast response requirement. Extensive experiments, including comparisons with many state-of-the-arts, show the effectiveness of our algorithm in improving the retrieval performance and saving the processing time.

Image annotation by large-scale content-based image retrieval

Abstract: Image annotation has been an active research topic in recent years due to its potentially large impact on both image understanding and Web image search. In this paper, we target at solving the automatic image annotation problem in a novel search and mining framework. Given an uncaptioned image, first in the search stage, we perform content-based image retrieval (CBIR) facilitated by high-dimensional indexing to find a set of visually similar images from a large-scale image database. The database consists of images crawled from the World Wide Web with rich annotations, e.g. titles and surrounding text. Then in the mining stage, a search result clustering technique is utilized to find most representative keywords from the annotations of the retrieved image subset. These keywords, after salience ranking, are finally used to annotate the uncaptioned image. Based on search technologies, this framework does not impose an explicit training stage, but efficiently leverages large-scale and well-annotated images, and is potentially capable of dealing with unlimited vocabulary. Based on 2.4 million real Web images, comprehensive evaluation of image annotation on Corel and U. Washington image databases show the effectiveness and efficiency of the proposed approach.

ImprovingWeb-based Image Search via Content Based Clustering

Abstract: Current image search engines on the web rely purely on the keywords around the images and the filenames, which produces a lot of garbage in the search results. Alternatively, there exist methods for content based image retrieval that require a user to submit a query image, and return images that are similar in content. We propose a novel approach named ReSPEC (Re-ranking Sets of Pictures by Exploiting Consistency), that is a hybrid of the two methods. Our algorithm first retrieves the results of a keyword query from an existing image search engine, clusters the results based on extracted image features, and returns the cluster that is inferred to be the most relevant to the search query. Furthermore, it ranks the remaining results in order of relevance.

First steps toward an electronic field guide for plants

Abstract: We describe an ongoing project to digitize information about plant specimens and make it available to botanists in the field. This first requires digital images and models, and then effective retrieval and mobile computing mechanisms for accessing this information. We have almost completed a digital archive of the collection of type specimens at the Smithsonian Institution Department of Botany. Using these and additional images, we have also constructed prototype electronic field guides for the flora of Plummers Island. Our guides use a novel computer vision algorithm to compute leaf similarity. This algorithm is integrated into image browsers that assist a user in navigating a large collection of images to identify the species of a new specimen. For example, our systems allow a user to photograph a leaf and use this image to retrieve a set of leaves with similar shapes. We measured the effectiveness of one of these systems with recognition experiments on a large dataset of images, and with user studies of the complete retrieval system. In addition, we describe future directions for acquiring models of more complex, 3D specimens, and for using new methods in wearable computing to interact with data in the 3D environment in which it is acquired.

Ontological inference for image and video analysis

Abstract: This paper presents an approach to designing and implementing extensible computational models for perceiving systems based on a knowledge-driven joint inference approach. These models can integrate different sources of information both horizontally (multi-modal and temporal fusion) and vertically (bottom–up, top–down) by incorporating prior hierarchical knowledge expressed as an extensible ontology.Two implementations of this approach are presented. The first consists of a content-based image retrieval system that allows users to search image databases using an ontological query language. Queries are parsed using a probabilistic grammar and Bayesian networks to map high-level concepts onto low-level image descriptors, thereby bridging the ‘semantic gap’ between users and the retrieval system. The second application extends the notion of ontological languages to video event detection. It is shown how effective high-level state and event recognition mechanisms can be learned from a set of annotated training sequences by incorporating syntactic and semantic constraints represented by an ontology.

Fuzzy SVM for content-based image retrieval: a pseudo-label support vector machine framework

Abstract: Conventional relevance feedback in content-based image retrieval (CBIR) systems uses only the labeled images for learning. Image labeling, however, is a time-consuming task and users are often unwilling to label too many images during the feedback process. This gives rise to the small sample problem where learning from a small number of training samples restricts the retrieval performance. To address this problem, we propose a technique based on the concept of pseudo-labeling in order to enlarge the training data set. As the name implies, a pseudo-labeled image is an image not labeled explicitly by the users, but estimated using a fuzzy rule. Therefore, it contains a certain degree of uncertainty or fuzziness in its class information. Fuzzy support vector machine (FSVM), an extended version of SVM, takes into account the fuzzy nature of some training samples during its training. In order to exploit the advantages of pseudo-labeling, active learning and the structure of FSVM, we develop a unified framework called pseudo-label fuzzy support vector machine (PLFSVM) to perform content-based image retrieval. Experimental results based on a database of 10,000 images demonstrate the effectiveness of the proposed method

A content-based image retrieval scheme in jpeg compressed domain

Abstract: Nowadays, a large number of images are compressed in JPEG (Joint Photo- graphic Experts Group) format. Therefore, content-based image retrieval (CBIR) for the JPEG images has attracted many people's attention and a series of algorithms directly based on the discrete cosine transform (DCT) domain have been proposed. However, the existing methods are far from the practical application. Thus, in this paper, a new image retrieval scheme for JPEG formatted images is presented. The color, spatial and frequency (texture) features based on the DCT domain are extracted for the later image retrieval. It doesn't require decompressing the images but directly retrieving in the DCT domain. Thus, compared with the spatial domain based retrieval methods for JPEG im- ages, the computation complexity can be greatly reduced. In addition, this retrieval system is suitable for all color images with different sizes. Experimental results demonstrate the advantages of the proposed retrieval scheme.

Retrieval of trademark images by means of size functions

Abstract: We propose a new, effective system for content-based retrieval of figurative images, which is based on size functions, a geometrical-topological tool for shape description and matching. Three different classes of shape descriptors are introduced and integrated, for a total amount of 25 measuring functions. The evaluation of our fully automatic retrieval system has been performed on a benchmark database of 10,745 real trademark images, supplied by the United Kingdom Patent Office. Comparative results show that our method actually outperforms other existing whole-image matching techniques, comprising features incorporated in the MPEG-7 standard.

An adaptive technique for content-based image retrieval

Abstract: We discuss an adaptive approach towards Content-Based Image Retrieval. It is based on the Ostensive Model of developing information needs--a special kind of relevance feedback model that learns from implicit user feedback and adds a temporal notion to relevance. The ostensive approach supports content-assisted browsing through visualising the interaction by adding user-selected images to a browsing path, which ends with a set of system recommendations. The suggestions are based on an adaptive query learning scheme, in which the query is learnt from previously selected images. Our approach is an adaptation of the original Ostensive Model based on textual features only, to include content-based features to characterise images. In the proposed scheme textual and colour features are combined using the Dempster-Shafer theory of evidence combination. Results from a user-centred, work-task oriented evaluation show that the ostensive interface is preferred over a traditional interface with manual query facilities. This is due to its ability to adapt to the user's need, its intuitiveness and the fluid way in which it operates. Studying and comparing the nature of the underlying information need, it emerges that our approach elicits changes in the user's need based on the interaction, and is successful in adapting the retrieval to match the changes. In addition, a preliminary study of the retrieval performance of the ostensive relevance feedback scheme shows that it can outperform a standard relevance feedback strategy in terms of image recall in category search.

Collaborative image retrieval via regularized metric learning

Abstract: In content-based image retrieval (CBIR), relevant images are identified based on their similarities to query images. Most CBIR algorithms are hindered by the semantic gap between the low-level image features used for computing image similarity and the high-level semantic concepts conveyed in images. One way to reduce the semantic gap is to utilize the log data of users' feedback that has been collected by CBIR systems in history, which is also called "collaborative image retrieval." In this paper, we present a novel metric learning approach, named "regularized metric learning," for collaborative image retrieval, which learns a distance metric by exploring the correlation between low-level image features and the log data of users' relevance judgments. Compared to the previous research, a regularization mechanism is used in our algorithm to effectively prevent overfitting. Meanwhile, we formulate the proposed learning algorithm into a semidefinite programming problem, which can be solved very efficiently by existing software packages and is scalable to the size of log data. An extensive set of experiments has been conducted to show that the new algorithm can substantially improve the retrieval accuracy of a baseline CBIR system using Euclidean distance metric, even with a modest amount of log data. The experiment also indicates that the new algorithm is more effective and more efficient than two alternative algorithms, which exploit log data for image retrieval.

A Dynamic User Concept Pattern Learning Framework for Content-Based Image Retrieval

Abstract: A rapid increase in the amount of image data and the inefficiency of traditional text-based image retrieval systems have served to make content-based image retrieval an active research field. It is crucial to effectively discover users' concept patterns through an acquired understanding of the subjective role played by humans in the retrieval process for such systems. A learning and retrieval framework is used to achieve this. It seamlessly incorporates multiple instance learning for relevant feedback to discover users concept patterns-especially in the region of greatest user interest. It also maps the local feature vector of that region to the high-level concept pattern. This underlying mapping can be progressively discovered through feedback and learning. The user guides the retrieval systems learning process using his/her focus of attention. Retrieval performance is tested to establish the feasibility and effectiveness of the proposed learning and retrieval framework

A Simple Bayesian Framework for Content-Based Image Retrieval

Abstract: We present a Bayesian framework for content-based image retrieval which models the distribution of color and texture features within sets of related images. Given a userspecified text query (e.g. "penguins") the system first extracts a set of images, from a labelled corpus, corresponding to that query. The distribution over features of these images is used to compute a Bayesian score for each image in a large unlabelled corpus. Unlabelled images are then ranked using this score and the top images are returned. Although the Bayesian score is based on computing marginal likelihoods, which integrate over model parameters, in the case of sparse binary data the score reduces to a single matrix-vector multiplication and is therefore extremely efficient to compute. We show that our method works surprisingly well despite its simplicity and the fact that no relevance feedback is used. We compare different choices of features, and evaluate our results using human subjects.

Image retrieval and perceptual similarity

Abstract: Simple, low-level visual features are extensively used for content-based image retrieval. Our goal was to evaluate an image-indexing system based on some of the known properties of the early stages of human vision. We quantitatively measured the relationship between the similarity order induced by the indexes and perceived similarity. In contrast to previous evaluation approaches, we objectively measured similarity both for the few best-matching images and also for relatively distinct images. The results show that, to a large degree, the rank orders induced by the indexes predict the perceived similarity between images. The highest index concordance employing a single index was obtained using the chromaticity histogram. Combining different information sources substantially improved the correspondence with the observers. We conclude that image-indexing systems can provide useful measures for perceptual image similarity. The methods presented here can be used to evaluate and compare different image-retrieval systems.

Scalable search-based image annotation of personal images

Abstract: With the prevalence of digital cameras, more and more people have considerable digital images on their personal devices. As a result, there are increasing needs to effectively search these personal images. Automatic image annotation may serve the goal, for the annotated keywords could facilitate the search processes. Although many image annotation methods have been proposed in recent years, their effectiveness on arbitrary personal images is constrained by their limited scalability, i.e. limited lexicon of small-scale training set. To be scalable, we propose a search-based image annotation (SBIA) algorithm that is analogous to Web page search. First, content-based image retrieval (CBIR) technology is used to retrieve a set of visually similar images from a large-scale Web image set. Then, a text-based keyword search (TBKS) technique is used to obtain a ranked list of candidate annotations for each retrieved image. Finally, a fusion algorithm is used to combine the ranked lists into the final annotation list. The application of both efficient search technologies and Web-scale image set guarantees the scalability of the proposed algorithm. Experimental results on U. Washington dataset show not only the effectiveness and efficiency of the proposed algorithm but also the advantage of image retrieval using annotation results over that using visual features.

A New Way for Multidimensional Medical Data Management: Volume of Interest (VOI)-Based Retrieval of Medical Images With Visual and Functional Features

Abstract: The advances in digital medical imaging and storage in integrated databases are resulting in growing demands for efficient image retrieval and management. Content-based image retrieval (CBIR) refers to the retrieval of images from a database, using the visual features derived from the information in the image, and has become an attractive approach to managing large medical image archives. In conventional CBIR systems for medical images, images are often segmented into regions which are used to derive two-dimensional visual features for region-based queries. Although such approach has the advantage of including only relevant regions in the formulation of a query, medical images that are inherently multidimensional can potentially benefit from the multidimensional feature extraction which could open up new opportunities in visual feature extraction and retrieval. In this study, we present a volume of interest (VOI) based content-based retrieval of four-dimensional (three spatial and one temporal) dynamic PET images. By segmenting the images into VOIs consisting of functionally similar voxels (e.g., a tumor structure), multidimensional visual and functional features were extracted and used as region-based query features. A prototype VOI-based functional image retrieval system (VOI-FIRS) has been designed to demonstrate the proposed multidimensional feature extraction and retrieval. Experimental results show that the proposed system allows for the retrieval of related images that constitute similar visual and functional VOI features, and can find potential applications in medical data management, such as to aid in education, diagnosis, and statistical analysis

Shape-based image retrieval in botanical collections

Abstract: Apart from the computer vision community, an always increasing number of scientific domains show a great interest for image analysis techniques. This interest is often guided by practical needs. As examples, we can cite all the medical imagery systems, the satellites images treatment and botanical databases. A common point of these applications is the large image collections that are generated and therefore require some automatic tools to help the scientists. These tools should allow clear structuration of the visual information and provide fast and accurate retrieval process. In the framework of the plant genes expression study we designed a content-based image retrieval (CBIR) system to assist botanists in their work. We propose a new contour-based shape descriptor that satisfies the constraints of this application (accuracy and real-time search). It is called Directional Fragment Histogram (DFH). This new descriptor has been evaluated and compared to several shape descriptors.

Content-based Image Retrieval Using Gabor-Zernike Features

Abstract: Content-based image retrieval (CBIR) is an important research area for manipulating large amount of image databases and archives. Extraction of invariant features is the basis of CBIR. This paper focuses on the problem of texture and shape feature extractions. We investigate texture feature and shape feature for CBIR by successfully combining the Gabor filters and Zernike moments (GF+ZM). GF is used for texture feature extraction and ZM extracts shape features. Comprehensive performance evaluation of our method is based on three different databases: face database, fingerprint database, and MPEG-7 shape database. The experimental results demonstrate that GF+ZM presents robustness to all of the three databases with the best average retrieval rate while the GF and ZM are limited for certain databases. GF is effective for face database and fingerprint database but is weak for MPEG-7 shape database. ZM achieves high retrieval rate for face database and MPEG-7 shape database but gives relatively low retrieval rate for fingerprint database.

Content-based image retrieval using Legendre chromaticity distribution moments

Abstract: It is a well-known fact that the direct storing and comparison of the histogram for the purpose of content-based image retrieval (CBIR) is inefficient in terms of memory space and query processing time. It is shown that the set of Legendre chromaticity distribution moments (LCDM) provides a compact, fixed-length and computation effective representation of the colour contents of an image. Only a small fixed number of compact LCDM features need to be stored to effectively characterise the colour content of an image. The need to store the whole chromaticity histogram is circumvented. Consequently the time involved in database querying is reduced. It is also shown that LCDM can be computed directly from the chromaticity space without first having to evaluate the chromaticity histogram.

A digital library framework for biodiversity information systems

Abstract: Biodiversity Information Systems (BISs) involve all kinds of heterogeneous data, which include ecological and geographical features. However, available information systems offer very limited support for managing these kinds of data in an integrated fashion. Furthermore, such systems do not fully support image content (e.g., photos of landscapes or living organisms) management, a requirement of many BIS end-users. In order to meet their needs, these users--e.g., biologists, environmental experts--often have to alternate between separate biodiversity and image information systems to combine information extracted from them. This hampers the addition of new data sources, as well as cooperation among scientists. The approach provided in this paper to meet these issues is based on taking advantage of advances in digital library innovations to integrate networked collections of heterogeneous data. It focuses on creating the basis for a next-generation BIS, combining new techniques of content-based image retrieval and database query processing mechanisms. This paper shows the use of this component-based architecture to support the creation of two tailored BIS systems dealing with fish specimen identification using search techniques. Experimental results suggest that this new approach improves the effectiveness of the fish identification process, when compared to the traditional key-based method.

Colour and Texture Features for Content Based Image Retrieval

Abstract: The novel approach combines colour and texture features for content based image retrieval. Features like colour and texture are obtained by computing the measure of standard deviation in combination with energy on each colour band of image and sub band of wavelet. Wavelet transform is used for decomposing the image into 2times2 sub-bands. Feature database in content-based image retrieval of 640 visual texture (VisTex) color images is constructed. It is observed that proposed method outperforms the other conventional histograms and standard wavelet decomposition techniques

CLAIRE: A modular support vector image indexing and classification system

Abstract: Many users of image retrieval systems would prefer to express initial queries using keywords. However, manual keyword indexing is very time-consuming. Therefore, a content-based image retrieval system which can automatically assign keywords to images would be very attractive. Unfortunately, it has proved very challenging to build such systems, except where either the image domain is restricted or the keywords relate only to low-level concepts such as color. This article presents a novel image indexing and classification system, called CLAIRE (CLAssifying Images for REtrieval), composed of one image processing module and three modules of support vector machines for color, texture, and high-level concept classification for keyword assignment. The experimental prototype system described here assigns up to five keywords selected from a controlled vocabulary of 60 terms to each image. The system is trained offline by 1639 examples from the Corel stock photo library. For evaluation, five judges reviewed a sample of 800 unknown images to identify which automatically assigned keywords were actually relevant to the image. The system proved to have an 80p probability to assign at least one relevant keyword to an image.

An Interactive Semantic Video Mining and Retrieval Platform--Application in Transportation Surveillance Video for Incident Detection

Abstract: Understanding and retrieving videos based on their semantic contents is an important research topic in multimedia data mining and has found various real- world applications. Most existing video analysis techniques focus on the low level visual features of video data. However, there is a "semantic gap" between the machine-readable features and the high level human concepts i.e. human understanding of the video content. In this paper, an interactive platform for semantic video mining and retrieval is proposed using relevance feedback (RF), a popular technique in the area of content-based image retrieval (CBIR). By tracking semantic objects in a video and then modeling spatio-temporal events based on object trajectories and object interactions, the proposed interactive learning algorithm in the platform is able to mine the spatio-temporal data extracted from the video. An iterative learning process is involved in the proposed platform, which is guided by the user's response to the retrieved results. Although the proposed video retrieval platform is intended for general use and can be tailored to many applications, we focus on its application in traffic surveillance video database retrieval to demonstrate the design details. The effectiveness of the algorithm is demonstrated by our experiments on real-life traffic surveillance videos.