Showing papers on "Content-based image retrieval published in 2001"

SIMPLIcity: semantics-sensitive integrated matching for picture libraries

Abstract: We present here SIMPLIcity (semantics-sensitive integrated matching for picture libraries), an image retrieval system, which uses semantics classification methods, a wavelet-based approach for feature extraction, and integrated region matching based upon image segmentation. An image is represented by a set of regions, roughly corresponding to objects, which are characterized by color, texture, shape, and location. The system classifies images into semantic categories. Potentially, the categorization enhances retrieval by permitting semantically-adaptive searching methods and narrowing down the searching range in a database. A measure for the overall similarity between images is developed using a region-matching scheme that integrates properties of all the regions in the images. The application of SIMPLIcity to several databases has demonstrated that our system performs significantly better and faster than existing ones. The system is fairly robust to image alterations.

Constructing models for content-based image retrieval

Abstract: This paper presents a new method for constructing models from a set of positive and negative sample images; the method requires no manual extraction of significant objects or features. Our model representation is based on two layers. The first one consists of "generic" descriptors which represent sets of similar rotational invariant feature vectors. Rotation invariance allows to group similar, but rotated patterns and makes the method robust to model deformations. The second layer is the joint probability on the frequencies of the "generic" descriptors over neighborhoods. This probability is multi-modal and is represented by a set of "spatial-frequency" clusters. It adds a statistical spatial constraint which is rotationally invariant. Our two-layer representation is novel; it allows to efficiently capture "texture-like" visual structure. The selection of distinctive structure determines characteristic model features (common to the positive and rare in the negative examples) and increases the performance of the model. Models are retrieved and localized using a probabilistic score. Experimental results for "textured" animals and faces show a very good performance for retrieval as well as localization.

Content-Based Image Retrieval Systems

Abstract: In this paper we present image data representation, similarity image retrieval, the architecture of a generic content-based image retrieval system, and different content-based image retrieval systems. l o w we d es cribe a n umber of cont e n t-b a s ed image retri e va l s ystems, in al pha be tical o r der. If

Efficient matching and indexing of graph models in content-based retrieval

Abstract: In retrieval from image databases, evaluation of similarity, based both on the appearance of spatial entities and on their mutual relationships, depends on content representation based on attributed relational graphs. This kind of modeling entails complex matching and indexing, which presently prevents its usage within comprehensive applications. In this paper, we provide a graph-theoretical formulation for the problem of retrieval based on the joint similarity of individual entities and of their mutual relationships and we expound its implications on indexing and matching. In particular, we propose the usage of metric indexing to organize large archives of graph models, and we propose an original look-ahead method which represents an efficient solution for the (sub)graph error correcting isomorphism problem needed to compute object distances. Analytic comparison and experimental results show that the proposed lookahead improves the state-of-the-art in state-space search methods and that the combined use of the proposed matching and indexing scheme permits for the management of the complexity of a typical application of retrieval by spatial arrangement.

Emergent semantics through interaction in image databases

Abstract: In this paper, we briefly discuss some aspects of image semantics and the role that it plays for the design of image databases. We argue that images don't have an intrinsic meaning, but that they are endowed with a meaning by placing them in the context of other images and by the user interaction. From this observation, we conclude that, in an image, database users should be allowed to manipulate not only the individual images, but also the relation between them. We present an interface model based on the manipulation of configurations of images.

Unsupervised multiresolution segmentation for images with low depth of field

Abstract: Unsupervised segmentation of images with low depth of field (DOF) is highly useful in various applications. This paper describes a novel multiresolution image segmentation algorithm for low DOF images. The algorithm is designed to separate a sharply focused object-of-interest from other foreground or background objects. The algorithm is fully automatic in that all parameters are image independent. A multi-scale approach based on high frequency wavelet coefficients and their statistics is used to perform context-dependent classification of individual blocks of the image. Unlike other edge-based approaches, our algorithm does not rely on the process of connecting object boundaries. The algorithm has achieved high accuracy when tested on more than 100 low DOF images, many with inhomogeneous foreground or background distractions. Compared with he state of the art algorithms, this new algorithm provides better accuracy at higher speed.

Learning similarity measure for natural image retrieval with relevance feedback

Abstract: A new scheme of learning similarity measure is proposed for content-based image retrieval (CBIR). It learns a boundary that separates the images in the database into two parts. Images on the positive side of the boundary are ranked by their Euclidean distances to the query. The scheme is called restricted similarity measure (RSM), which not only takes into consideration the perceptual similarity between images, but also significantly improves the retrieval performance based on the Euclidean distance measure. Two techniques, support vector machine and AdaBoost, are utilized to learn the boundary, and compared with respect to their performance in boundary learning. The positive and negative examples used to learn the boundary are provided by the user with relevance feedback. The RSM metric is evaluated on a large database of 10,009 natural images with an accurate ground truth. Experimental results demonstrate the usefulness and effectiveness of the proposed similarity measure for image retrieval.

Content-based shape retrieval using different shape descriptors: a comparative study

Abstract: Shape representation is a fundamental issue in the newly emerging multimedia applications. In the content based image retrieval (CBIR), shape is an important low level image feature. Many shape representations have been proposed. However, for CBIR, a shape representation should satisfy several properties such as affine invariance, robustness, compactness, low computation complexity and perceptual similarity measurement. Against these properties, in this paper we attempt to study and compare several shape descriptors which have been widely adopted for CBIR, they are: Fourier descriptors (FD), curvature scale space (CSS) descriptors (CSSD), Zernike moment descriptors (ZMD) and grid descriptors (GD). The strengths and limitations of these methods are analyzed and clarified. Retrieval results are given to show the comparison.

Self-Organising Maps as a Relevance Feedback Technique in Content-Based Image Retrieval

Abstract: Self-Organising Maps (SOMs) can be used in implementing a powerful relevance feedback mechanism for Content-Based Image Retrieval (CBIR). This paper introduces the PicSOM CBIR system, and describes the use of SOMs as a relevance feedback technique in it. The technique is based on the SOM’s inherent property of topology-preserving mapping from a high-dimensional feature space to a two-dimensional grid of artificial neurons. On this grid similar images are mapped in nearby locations. As image similarity must, in unannotated databases, be based on low-level visual features, the similarity of images is dependent on the feature extraction scheme used. Therefore, in PicSOM there exists a separate tree-structured SOM for each different feature type. The incorporation of the relevance feedback and the combination of the outputs from the SOMs are performed as two successive processing steps. The proposed relevance feedback technique is described, analysed qualitatively, and visualised in the paper. Also, its performance is compared with a reference method.

Mining association rules between low-level image features and high-level concepts

Abstract: In image similarity retrieval systems, color is one of the most widely used features Users who are not well versed with the image domain characteristics might be more comfortable in working with an Image Retrieval System that allows specification of a query in terms of keywords, thus eliminating the usual intimidation in dealing with very primitive features In this paper we present two approaches to automatic image annotation, by finding those rules underlying the links between the low-level features and the high-level concepts associated with images One scheme uses global color image information and classification tree based techniques Through this supervised learning approach we are able to identify relationships between global color-based image features and some textual decriptors In the second approach, using low-level image features that capture local color information and through a k-means based clustering mechanism, images are organized in clusters such that images that are similar are located in the same cluster For each cluster, a set of rules is derived to capture the association between the localized color-based image features and the textual descriptors relevant to the cluster

Web mining for web image retrieval

Abstract: The popularity of digital images is rapidly increasing due to improving digital imaging technologies and convenient availability facilitated by the Internet. However, how to find user-intended images from the Internet is nontrivial. The main reason is that the Web images are usually not annotated using semantic descriptors. In this article, we present an effective approach to and a prototype system for image retrieval from the Internet using Web mining. The system can also serve as a Web image search engine. One of the key ideas in the approach is to extract the text information on the Web pages to semantically describe the images. The text description is then combined with other low-level image features in the image similarity assessment. Another main contribution of this work is that we apply data mining on the log of users' feedback to improve image retrieval performance in three aspects. First, the accuracy of the document space model of image representation obtained from the Web pages is improved by removing clutter and irrelevant text information. Second, to construct the user space model of users' representation of images, which is then combined with the document space model to eliminate mismatch between the page author's expression and the user's understanding and expectation. Third, to discover the relationship between low-level and high-level features, which is extremely useful for assigning the low-level features' weights in similarity assessment.

3D MARS: immersive virtual reality for content-based image retrieval

Abstract: 3D MARS is an interactive visualization system for Content-Based Image Retrieval (CBIR). In 3D MARS, the user browses and queries images in an immersive 3D Virtual Reality space of CAVE. The results of the query are displayed in 3D, so that the user can see the result with respect to three different criteria such as color, texture and structure. The user can examine the result from different view angles by flying-through the space with the joystick. Based on the user’s feedback, the system dynamically reorganize its visualization scheme. By giving meanings to each axis, the user can determine which features are important. In addition, the Sphere mode visualization is provided as a powerful analyzing tool for CBIR researchers.

Features in Content-based Image Retrieval Systems: a Survey

Abstract: This article provides a framework to describe and compare content-based image retrieval systems. Sixteen contemporary systems are described in detail, in terms of the following technical aspects: querying, relevance feedback, result presentation, features, and matching. For a total of 44 systems we list the features that are used. Of these systems, 35 use any kind of color features, 28 use texture, and only 25 use shape features.

Scalable integrated region-based image retrieval using IRM and statistical clustering

Abstract: Statistical clustering is critical in designing scalable image retriev al systems. In this paper, we present a scalable algorithm for indexing and retrieving images based on region segmentation. The method uses statistical clustering on region features and IRM (Integrated Region Matching), a measure developed to evaluate overall similarity between images that incorporates properties of all the regions in the images by a region-matching scheme. Compared with retrieval based on individual regions, our overall similarity approach (a) reduces the influence of inaccurate segmentation, (b) helps to clarify the semantics of a particular region, and (c) enables a simple querying interface for region-based image retrieval systems. The algorithm has been implemented as a part of our experimental SIMPLIcity image retrieval system and tested on large-scale image databases of both general-purpose images and pathology slides. Experiments have demonstrated that this technique maintains the accuracy and robustness of the original system while reducing the matching time significantly.

An adaptive and efficient clustering-based approach for content-based image retrieval in image databases

Abstract: The authors present a novel content based image retrieval (CBIR) approach, for image databases, based on cluster analysis. CBIR relies on the representation (metadata) of images' visual content. In order to produce such metadata, we propose an efficient and adaptive clustering algorithm to segment the images into regions of high similarity. This approach contrasts with those that use a single color histogram for the whole image (global methods), or local color histograms for a fixed number of image cells (partition based methods). Our experimental results show that our clustering approach offers high retrieval effectiveness with low space overhead. For example, using a database of 20000 images, we obtained higher retrieval effectiveness than partition based methods with about the same space overhead of global methods, which are typically regarded as storage-wise compact.

A Comparison of Shape Retrieval Using Fourier Descriptors and Short-Time Fourier Descriptors

Abstract: Shape is one of the primary features in Content Based Image Retrieval (CBIR). Many shape representations and retrieval methods exist. However, most of those methods either do not well capture shape features or are difficult to do normalization (or matching). Among them, methods based Fourier descriptors (FDs) achieve both good representation and easy normalization. FDs is often blamed for not being able to locate local shape features. Methods are proposed in attempt to overcome this drawback. These methods include short-time Fourier transform and wavelet transform. In this paper, we make a study and a comparison of shape retrieval using FDs and short-time Fourier descriptors (SFDs). Query data is given to show the retrieval performance of this two descriptors on a standard database.

Color constant ratio gradients for image segmentation and similarity of texture objects

Abstract: We aim for content-based image retrieval of texture objects in natural scenes under varying illumination and viewing conditions. To achieve this, image retrieval is based on matching feature distributions derived from color invariant gradients. To cope with object cluttering, region-based texture segmentation is applied on the target images prior to the actual image retrieval process. The retrieval scheme is empirically verified on color images taken from texture objects under different lighting, conditions.

Automated benchmarking in content-based image retrieval

Abstract: Benchmarking has always been a crucial problem in content-based image retrieval (CBIR). A key issue is the lack of a common access method to retrieval systems, such as SQL for relational databases. The Multimedia Retrieval Mark-up Language (MRML) solves this problem by standardizing access to CBIR systems (CBIRSs). Other difficult problems are also shortly addressed, such as obtaining relevance judgments and choosing a database for performance comparison. In this article we present a fully automated benchmark for CBIRSs based on MRML, which can be adapted to any image database and almost any kind of relevance judgment. The test evaluates the performance of positive and negative relevance feedback, which can be generated automatically from the relevance judgments. To illustrate our purpose, a freely available, non-copyright image collection is used to evaluate our CBIRS, Viper. All scripts described here are also freely available for download.

An intelligent content-based image retvieval system based oncolor, shape and spatial relations

Abstract: Content-based multimedia information retrieval is an interesting but difficult area of research. Current approaches include the use of color, texture, and shape information. In this paper, we present a hybrid approach, which incorporates color, shape and spatial relations among objects in a picture, and has been used to retrieve images from nearly 5000 pictures. A revised color scheme and its indexing technique are used to improve the efficiency of retrieval, based on our clustering method and color sensation. A seed filling-like mechanism is used to extract shape and spatial relations from objects. A qualitative approach is applied to similarity comparison of spatial differences. The system is also implemented with a friendly GUI, which allows sketch images as well as relevance feedback to improve the accuracy of retrieval. Our experience shows that the system is able to retrieve image information efficiently based on the proposed approach.

Category-based image retrieval

Abstract: This work presents a novel approach to content-based image retrieval in categorical multimedia databases. The images are indexed using a combination of text and content descriptors. The categories are viewed as semantic clusters of images and are used to confine the search space. Keywords are used to identify candidate categories. Content-based retrieval is performed in these categories using multiple image features. Relevance feedback is used to learn the user's intent-query specification and feature-weighting-with minimal user-interface abstraction. The method is applied to a large number of images collected from a popular categorical structure on the World Wide Web. Results show that efficient and accurate performance is achievable by exploiting the semantic classification represented by the categories. The relevance feedback loop allows the content descriptor weightings to be determined without exposing the calculations to the user.

Evaluating a content based image retrieval system

Abstract: Content Based Image Retrieval (CBIR) presents special challenges in terms of how image data is indexed, accessed, and how end systems are evaluated. This paper discusses the design of a CBIR system that uses global colour as the primary indexing key, and a user centered evaluation of the systems visual search tools. The results indicate that users are able to make use of a range of visual search tools, and that different tools are used at different points in the search process. The results also show that the provision of a structured navigation and browsing tool can support image retrieval, particularly in situations in which the user does not have a target image in mind. The results are discussed in terms of their implications for the design of visual search tools, and their implications for the use of user-centered evaluation for CBIR systems.

Content based image retrieval and information theroy: a general approach

Abstract: A fundamental aspect of content-based image retrieval (CBIR) is the extraction and the representation of a visual feature that is an effective discriminant between pairs of images. Among the many visual features that have been studied, the distribution of color pixels in an image is the most common visual feature studied. The standard representation of color for content-based indexing in image databases is the color histogram. Vector-based distance functions are used to compute the similarity between two images as the distance between points in the color histogram space. This paper proposes an alternative real valued representation of color based on the information theoretic concept of entropy. A theoretical presentation of image entropy is accompanied by a practical description of the merits and limitations of image entropy compared to color histograms. Specifically, the L1 norm for color histograms is shown to provide an upper bound on the difference between image entropy values. Our initial results suggest that image entropy is a promising approach to image description and representation.

Regions-of-Interest and Spatial Layout for Content-Based Image Retrieval

Abstract: To date most “content-based image retrieval” (CBIR) techniques rely on global attributes such as color or texture histograms which tend to ignore the spatial composition of the image. In this paper, we present an alternative image retrieval system based on the principle that it is the user who is most qualified to specify the query “content” and not the computer. With our system, the user can select multiple “regions-of-interest” and can specify the relevance of their spatial layout in the retrieval process. We also derive similarity bounds on histogram distances for pruning the database search. This experimental system was found to be superior to global indexing techniques as measured by statistical sampling of multiple users' “satisfaction” ratings.

Content-based image retrieval using wavelet-based salient points

Abstract: Content-based Image Retrieval (CBIR) has become one of the most active research areas in the past few years. Most of the attention from the research has been focused on indexing techniques based on global feature distributions. However, these global distributions have limited discriminating power because they are unable to capture local image information. Applying global Gabor texture features greatly improve the retrieval accuracy. But they are computationally complex. In this paper, we present a wavelet-based salient point extraction algorithm. We show that extracting the color and texture information in the locations given by these points provides significantly improved results in terms of retrieval accuracy, computational complexity and storage space of feature vectors as compared to the global feature approaches.

A Test Collection for the Evaluation of Content-Based Image Retrieval Algorithms—A User and Task-Based Approach

Abstract: Content-based image retrieval (CBIR) algorithms have been seen as a promising access method for digital photograph collections. Unfortunately, we have very little evidence of the usefulness of these algorithms in real user needs and contexts. In this paper, we introduce a test collection for the evaluation of CBIR algorithms. In the test collection, the performance testing is based on photograph similarity perceived by end-users in the context of realistic illustration tasks and environment. The building process and the characteristics of the resulting test collection are outlined, including a typology of similarity criteria expressed by the subjects judging the similarity of photographs. A small-scale study on the consistency of similarity assessments is presented. A case evaluation of two CBIR algorithms is reported. The results show clear correlation between the subjects' similarity assessments and the functioning of feature parameters of the tested algorithms.

Mapping low-level image features to semantic concepts

Abstract: Humans tend to use high-level semantic concepts when querying and browsing multimedia databases; there is thus, a need for systems that extract these concepts and make available annotations for the multimedia data. The system presented in this paper satisfies this need by automatically generating semantic concepts for images form their low-level visual features. The proposed system is built in two stages. First, an adaptation of k-means clustering using a non- Euclidean similarity metric is applied to discover the natural patterns of the data in the low-level feature space; the cluster prototype is designed to summarize the cluster in a manner that is suited for quick human comprehension of its components. Second, statistics measuring the variation within each cluster are used to derive a set of mappings between the most significant low-level features and the most frequent keywords of the corresponding cluster. The set of the derived rules could be used further to capture the semantic content and index new untagged images added to the image database. The attachment of semantic concepts to images will also give the system the advantage of handling queries expressed in terms of keywords and thus, it reduces the semantic gap between the user's conceptualization of a query and the query that is actually specified to the system. While the suggested scheme works with any kind of low-level features, our implementation and description of the system is centered on the use of image color information. Experiments using a 21 00 image database are presented to show the efficacy of the proposed system.

Relevance feedback for semantics based image retrieval

Abstract: Content based image retrieval is one of the most active research areas in the field of multimedia technology. Currently, the relevance feedback approach has attracted great attention since it can bridge the gap between low-level features and the semantics of images. We propose a new relevance feedback technique, which uses the normal mixture model for the high-level similarity metric of the user's intention and estimates the unknown parameters from the user's feedback. Our approach is based on a novel hybrid algorithm where the criterion for the selection of the display image set is evolved from the most informative to the most probable as the retrieval process progresses. Experiments on the Corel image set show that the proposed algorithm outperforms MindReader at the semantics based search.

Comparison of techniques for content-based image retrieval

Abstract: Content-based image retrieval (CBIR) is a new but widelyadopted method for finding images from vast and unannotated image databases. In CBIR images are indexed on the basis of low-level features, such as color, texture, and shape, that can automatically be derived from the visual content of the images. The operation of a CBIR system can be seen as a series of more or less independent processing stages. As there exists multiple choices for each of these stages, a multitude of CBIR systems can be implemented by combining a set of common building blocks. In this paper, we present a comparison of different techniques for three consecutive stages of a CBIR system. These include: (1) the initial per feature selection of considered images, (2) the combination of the lists of selected images, and (3) the final selection of images based on all available features simultaneously. The results of the performed experiments show that CBIR systems can be implemented using consecutive stages where, at each stage, a number of parallel techniques can be provided.