Showing papers on "Content-based image retrieval published in 1999"

NeTra: a toolbox for navigating large image databases

Abstract: We present here an implementation of NeTra, a prototype image retrieval system that uses color, texture, shape and spatial location information in segmented image regions to search and retrieve similar regions from the database. A distinguishing aspect of this system is its incorporation of a robust automated image segmentation algorithm that allows object- or region-based search. Image segmentation significantly improves the quality of image retrieval when images contain multiple complex objects. Images are segmented into homogeneous regions at the time, of ingest into the database, and image attributes that represent each of these regions are computed. In addition to image segmentation, other important components of the system include an efficient color representation, and indexing of color, texture, and shape features for fast search and retrieval. This representation allows the user to compose interesting queries such as "retrieve all images that contain regions that have the color of object A, texture of object B, shape of object C, and lie in the upper of the image", where the individual objects could be regions belonging to different images. A Java-based web implementation of NeTra is available at http://vivaldi.ece.ucsb.edu/Netra.

Spatial Color Indexing and Applications

Abstract: We define a new image feature called the color correlogram and use it for image indexing and comparison. This feature distills the spatial correlation of colors and when computed efficiently, turns out to be both effective and inexpensive for content-based image retrieval. The correlogram is robust in tolerating large changes in appearance and shape caused by changes in viewing position, camera zoom, etc. Experimental evidence shows that this new feature outperforms not only the traditional color histogram method but also the recently proposed histogram refinement methods for image indexing/retrieval. We also provide a technique to cut down the storage requirement of the correlogram so that it is the same as that of histograms, with only negligible performance penalty compared to the original correlogram. We also suggest the use of color correlogram as a generic indexing tool to tackle various problems arising from image retrieval and video browsing. We adapt the correlogram to handle the problems of image subregion querying, object localization, object tracking, and cut detection. Experimental results again suggest that the color correlogram is more effective than the histogram for these applications, with insignificant additional storage or processing cost.

Content-based hierarchical classification of vacation images

Abstract: Grouping images into (semantically) meaningful categories using low level visual features is a challenging and important problem in content based image retrieval. Using binary Bayesian classifiers, we attempt to capture high level concepts from low level image features under the constraint that the test image does belong to one of the classes of interest. Specifically, we consider the hierarchical classification of vacation images; at the highest level, images are classified into indoor/outdoor classes, outdoor images are further classified into city/landscape classes, and finally, a subset of landscape images is classified into sunset, forest, and mountain classes. We demonstrate that a small codebook (the optimal size of codebook is selected using a modified MDL criterion) extracted from a vector quantizer can be used to estimate the class-conditional densities of the observed features needed for the Bayesian methodology. On a database of 6931 vacation photographs, our system achieved an accuracy of 90.5% for indoor vs. outdoor classification, 95.3% for city vs. landscape classification, 96.6% for sunset vs. forest and mountain classification, and 95.5% for forest vs. mountain classification. We further develop a learning paradigm to incrementally train the classifiers as additional training samples become available and also show preliminary results for feature size reduction using clustering techniques.

Content-Based Image Retrieval Based on Local Affinely Invariant Regions

Abstract: This contribution develops a new technique for content-based image retrieval. Where most existing image retrieval systems mainly focus on color and color distribution or texture, we classify the images based on local invariants. These features represent the image in a very compact way and allow fast comparison and feature matching with images in the database. Using local features makes the system robust to occlusions and changes in the background. Using invariants makes it robust to changes in viewpoint and illumination.

Region-based shape representation and similarity measure suitable for content-based image retrieval

Abstract: A region-based approach to shape representation and similarity measure is presented. The shape representation is invariant to translation, scale and rotation. The similarity measure conforms to human similarity perception, i.e., perceptually similar shapes have high similarity measure. An experimental shape retrieval system has been developed and its performance has been studied. The shape retrieval performance of the proposed approach is better than that of the more established Fourier descriptor method.

A CBIR method based on color-spatial feature

Abstract: Along with the analysis of color features in the hue, saturation and value (HSV) space, a new dividing method to quantize the color space into 36 non-uniform bins is introduced in this paper. Based on this quantization method we propose a color-spatial method to include several spatial features of the colors in an image for retrieval. These features are area and position, which mean the zero-order and the first-order moments, respectively. Experiments on an image database of 838 images show that the algorithm performs well in precision and adaptability.

Spatial color histograms for content-based image retrieval

Abstract: The color histogram is an important technique for color image database indexing and retrieval. In this paper, the traditional color histogram is modified to capture the spatial layout information of each color, and three types of spatial color histograms are introduced: annular, angular and hybrid color histograms. Experiments show that, with a proper trade-off between the granularity in the color and spatial dimensions, these histograms outperform both the traditional color histogram and some existing histogram refinements such as the color coherent vector.

Content-based image retrieval in medical applications: a novel multistep approach

Abstract: In the past few years, immense improvement was obtained in the field of content-based image retrieval (CBIR). Nevertheless, existing systems still fail when applied to medical image databases. Simple feature-extraction algorithms that operate on the entire image for characterization of color, texture, or shape cannot be related to the descriptive semantics of medical knowledge that is extracted from images by human experts. In this paper, we present a novel multi-step approach, which is specially designed for content-based image retrieval in medical applications (IRMA). In contrast to common approaches, the IRMA-concept is based on a conceptual and algorithmic separation of: (a) image categorization using global features, (b) geometry and contrast registration with respect to prototypes within the categories, (c) extraction of local features, (d) category and query dependent local feature selection, (e) index generation resulting in hierarchical multi-scale blob representations, (f) object identification that links a-priori knowledge on image content to the blobs, and (g) image retrieval processed on the abstract blob-level. The IRMA-concept comprises several benefits when compared to existing CBIR-systems. The image categories enable semantics by prototypes. Furthermore, each image might belong to several categories. A-priori knowledge on both image and query content is adjuncted to indexing. Therefore, the IRMA-concept provides a high amount of content understanding and enables intelligent queries on an abstract level of information. Hence, IRMA promises satisfactory query completion in medical applications.

Relevance feedback and category search in image databases

Abstract: We present a sound framework for relevance feedback in content based image retrieval. The modeling is based on non parametric density estimation of relevant and non relevant items and Bayesian inference. This theory has been successfully applied to benchmark image databases, quantitatively demonstrating its performance for target search, selective control of precision and recall in category search, and improvement of retrieval effectiveness. The paper is illustrated with several experiments and retrieval results on real world data.

A relevance feedback mechanism for content-based image retrieval

Abstract: Content-based image retrieval systems require the development of relevance feedback mechanisms that allow the user to progressively refine the system's response to a query. In this paper a new relevance feedback mechanism is described which evaluates the feature distributions of the images judged relevant, or not relevant, by the user and dynamically updates both the similarity measure and the query in order to accurately represent the user's particular information needs. Experimental results demonstrate the effectiveness of this mechanism.

PicSOM: self-organizing maps for content-based image retrieval

Abstract: Content-based image retrieval is an important approach to the problem of processing the increasing amount of visual data. It is based on automatically extracted features from the content of the images, such as color, texture, shape and structure. We have started a project to study methods for content-based image retrieval using the self-organizing map (SOM) as the image similarity scoring method. Our image retrieval system, named PicSOM, can be seen as a SOM-based approach to relevance feedback which is a form of supervised learning to adjust the subsequent queries based on the user's responses during the information retrieval session. In PicSOM, a separate tree structured SOM (TS-SOM) is trained for each feature vector type in use. The system then adapts to the user's preferences by returning her more images from those SOMs where her responses have been most densely mapped.

Applying perceptual grouping to content-based image retrieval: building images

Abstract: This paper presents an application of perceptual grouping rules for content-based image retrieval. The semantic interrelationships between different primitive image features are exploited by perceptual grouping to detect the presence of manmade structures. A methodology based on these principles in a Bayesian framework for the retrieval of building images, and the results obtained are presented. The image database consists of monocular grayscale outdoor images taken from a ground-level camera.

Color-based image retrieval using spatial-chromatic histograms

Abstract: We propose a methodology to integrate color and spatial information for content-based image retrieval. This methodology, called Spatial-Chromatic Histogram (SCH), synthesizes in few values information about the location of pixels having the same color and their arrangement within the image. We have performed experiments on a three hundred images database and our results show a high retrieval accuracy.

Content-Based Image Retrieval over the Web Using Query by Sketch and Relevance Feedback

Abstract: This paper investigates the combined use of query by sketch and relevance feedback as techniques to ease user interaction and improve retrieval effectiveness in content-based image retrieval over the World Wide Web. To substantiate our ideas we implemented DrawSearch, a prototype image retrieval by content system that uses color, shape and texture to index and retrieve images. The system avails of Java applets for query by sketch and uses relevance feedback to allow users dynamically refine queries.

The PicToSeek WWW image search system

Abstract: We give an overview of the PicToSeek system for exploring visual information on the World Wide Web. PicToSeek automatically collects, indexes and catalogs visual information entirely on the basis of the pictorial content. PicToSeek allows for content-based image retrieval conducted in an interactive, iterative manner guided by the user by relevance feedback. Relevance feedback can be seen as a method of feature selection and weighting. The PicToSeek system has been implemented based on the client-server paradigm. The client is a Java applet and takes care of interactive query formulation, the display of the results, and the relevance feedback specification given by the user. The server is a Servlet using C-libraries and takes care of the image feature extraction, feature weighting from relevance feedback, k-nearest neighbour feature classification, and image sorting. The system is available at http://www.wins.uva.nl/research/isis/zomax/.

Features in Content-based Image Retrieval Systems: a Survey.

Abstract: This article provides a framework to describe and compare content-based image retrieval systems. Sixteen contemporary systems are described in detail, in terms of the following technical aspects: querying, relevance feedback, result presentation, features, and matching. For a total of 44 systems we list the features that are used. Of these systems, 35 use any kind of color features, 28 use texture, and only 25 use shape features.

An environment for content-based image retrieval from large spatial databases

Abstract: In this paper, we address the problem of content-based image retrieval using queries on shape and topology. We focus on the particularities of image databases encountered in typical topographic applications and present the development of a spatial data management system that enables such queries. The query requires user-provided sketches of the shape and spatial configuration of the object (or objects) which should appear in the images to be retrieved. The objective of the search is to retrieve images that contain a configuration of objects sufficiently similar to the one specified in the query. Our approach combines the design of an integrated database with the development of a feature library and the necessary matching tools. In this paper, we present our overall scheme, introduce some individual database components, and provide some implementation results.

Water-filling: a novel way for image structural feature extraction

Abstract: The performance of a content based image retrieval (CBIR) system is inherently constrained by the features adopted to represent the images in the database. In this paper, a new approach is proposed for image feature extraction based on edge maps. The feature vector with multiple feature components is computed through a “water-filling algorithm” applied on the edge map of the original image. The idea of this algorithm is to obtain measures of the edge length and complexity by graph traverse. The new feature is move generally applicable than texture or shape features. We call this structure feature. Experiments show that the new feature is capable of catching salient edge/structure information in the images. An experimental retrieval system utilizing the proposed new features yields better results in retrieving city/building images than some global texture features (wavelet moments). The new feature is ideal for images with clear edge structure. After combining the new features with other features in a relevance feedback framework, satisfactory retrieval results are observed.

Content-based image retrieval in picture archiving and communications systems

Abstract: We propose the concept of content-based image retrieval (CBIR) and demonstrate its potential use in picture archival and communication system (PACS) We address the importance of image retrieval in PACS and highlight the drawbacks existing in traditional textual-based retrieval We use a digital mammogram database as our testing data to illustrate the idea of CBIR, where retrieval is carried out based on object shape, size, and brightness histogram With a user-supplied query image, the system can find images with similar characteristics from the archive, and return them along with the corresponding ancillary data, which may provide a valuable reference for radiologists in a new case study Furthermore, CBIR can perform like a consultant in emergencies when radiologists are not available We also show that content-based retrieval is a more natural approach to man-machine communication

Evaluation of prostate tumor grades by content-based image retrieval

Abstract: As part of collaboration between the Pittsburgh Supercomputing Center and the University of Pittsburgh Medical Center we are developing methods for content based image retrieval to assist pathology diagnosis. We have been using Gleason grading of prostate tumor samples as an initial domain for evaluating the effectiveness of the method for specific tasks. In this application, the system does not attempt to directly reproduce pathologists' visual analysis. Rather, it relies on the comparison of image features from a sample image to key the retrieval of similar but previously graded images from a database. Appropriate features should be highly selective to architecture differences of the Gleason system so the grades of the retrieved images can be applied to the unknown sample. We have been investigating the usefulness of computational geometry structures, such as spanning trees, as components of feature sets providing accurate retrieval of matching grades.

Detecting sky and vegetation in outdoor images

Abstract: Developing semantic indices into large image databases is a challenging and important problem in content-based image retrieval We address the problem of detecting objects in an image based on color and texture features Specifically, we consider the following two problems of detecting sky and vegetation in outdoor images An image is divided into 16 X 16 sub-blocks and color, texture, and position features are extracted form every sub-block We demonstrate how a small set of codebook vectors, extracted from every sub- block We demonstrate how a small set of codebook vectors, extracted from a learning vector quantizer, can be used to estimate the class-conditional densities of the low-level observed feature needed for the Bayesian methodology The sky and vegetation detectors have been trained on over 400 color images from the Corel database We achieve classification accuracies of over 94 percent for both the classifiers on the training data We are currently extending our evaluation to a larger database of 1,700 images

Apply semantic template to support content-based image retrieval

Abstract: Content-based multimedia information retrieval is the hot point of researchers in many domains. But traditional featurevector based retrieval method can not provide retrieval on the semantic level. Integrated with our image retrieval system,we propose a new approach to generate semantic template automatically in the process of relevance feedback, and constructa network of semantic template with the support of WordNetTM j the retrieval process, which helps the user to do retrievalon the semantic level. By our approach, in the keyword query of the user, relevant images will be returned to the user bythe help of semantic template association even those images are not annotated by keyword. This paper introduces thisapproach in detail and presents an experiment result at the end of this paper.Keywords: Multimedia, Information retrieval, Relevance feedback, Semantic template