Advances in data storage and image acquisition technologie s have enabled the creation of large image datasets. In this scenario, it is necess ary to develop appropriate information systems to efficiently manage these collect ions. The commonest approaches use the so-called Content-Based Image Retrieval (CBIR) systems . Basically, these systems try to retrieve images similar to a user-define d sp cification or pattern (e.g., shape sketch, image example). Their goal is to suppor t image retrieval based on contentproperties (e.g., shape, color, texture), usually encoded into feature vectors . One of the main advantages of the CBIR approach is the possibi lity of an automatic retrieval process, instead of the traditional keyword-bas ed approach, which usually requires very laborious and time-consuming previous annot ation of database images. The CBIR technology has been used in several applications su ch as fingerprint identification, biodiversity information systems, digital librar ies, crime prevention, medicine, historical research, among others. This paper aims to introduce the problems and challenges con cerned with the creation of CBIR systems, to describe the existing solutions and appl ications, and to present the state of the art of the existing research in this area.

Content-Based Image Retrieval: Theory and Applications.

While it would seem that digital video libraries should benefit from access mechanisms directed to their visual contents, years of TREC Video Retrieval Evaluation (TRECVID) research have shown that text search against transcript narrative text provides almost all the retrieval capability, even with visually oriented generic topics. A within-subjects study involving 24 novice participants on TRECVID 2004 tasks again confirms this result. The study shows that satisfaction is greater and performance is significantly better on specific and generic information retrieval tasks from news broadcasts when transcripts are available for search. Additional runs with 7 expert users reveal different novice and expert interaction patterns with the video library interface, helping explain the novices' lack of success with image search and visual feature browsing for visual information needs. Analysis of TRECVID visual features well suited for particular tasks provides additional insights into the role of automated feature classification for digital image and video libraries.

Addressing the challenge of visual information access from digital image and video libraries

A perception-based approach to content-based image representation and retrieval is proposed in this paper. We consider textured images and propose to model their textural content by a set of features having a perceptual meaning and their application to content-based image retrieval. We present a new method to estimate a set of perceptual textural features, namely coarseness, directionality, contrast, and busyness. The proposed computational measures can be based upon two representations: the original images representation and the autocorrelation function (associated with original images) representation. The set of computational measures proposed is applied to content-based image retrieval on a large image data set, the well-known Brodatz database. Experimental results and benchmarking show interesting performance of our approach. First, the correspondence of the proposed computational measures to human judgments is shown using a psychometric method based upon the Spearman rank-correlation coefficient. Second, the application of the proposed computational measures in texture retrieval shows interesting results, especially when using results fusion returned by each of the two representations. Comparison is also given with related works and show excellent performance of our approach compared to related approaches on both sides: correspondence of the proposed computational measures with human judgments as well as the retrieval effectiveness.

Computational Perceptual Features for Texture Representation and Retrieval

With the aid of image content, the relevant images can be extracted from the image in the Content Based Image Retrieval (CBIR) system. Concise feature sets limit the retrieval efficiency, to eliminate this shape, colour, texture and contourlet features are extracted. For retrieving relevant images, the optimisation technique Genetic Algorithm (GA) is utilised and for similarity measure Squared Euclidean Distance (SED) is utilised for comparing query image featureset and database image featureset. Hence, from GA based similarity measure, relevant images are retrieved and evaluated by querying different images.

An effective similarity measure via genetic algorithm for Content-Based Image Retrieval with extensive features

An image processing system and method receives one or more digital images in the form of image data, including selected object data of a digital image, and determines, by an electronic recognition process, if a recognition match is available between the selected object data of the digital image and image object library data associated with image descriptor library data. An automated library user interface presents selectable matched object descriptor data associated with the image descriptor library data when a recognition match occurs between the selected object data of the digital image and the image descriptor library data. In response, the automated library user interface provides user feedback data to confirm that the image descriptor library data corresponds with the selected object data of the digital image, or entered descriptor data if no match or an incorrect match occurs, to create library descriptor associated image data.

Image processing system and method

Content-based image retrieval uses features that can be extracted from the images themselves. Using more than one representation of the images in a collection can improve the results presented to a user without changing the underlying feature extraction or search technologies. We present an example of this "multiple viewpoint" approach, multiple image channels, and discuss its advantages for an image-seeking user. This approach has also been shown to dramatically improve retrieval effectiveness in content-based image retrieval systems [J.C. French et al., (2003)].

An application of multiple viewpoints to content-based image retrieval

Multiple viewpoint systems are an approach to information retrieval that takes advantage of having more than one source of judgments on a body of information. Multiple viewpoints can improve retrieval effectiveness over the typical single-viewpoint approach, especially in cases when there is vocabulary mismatch between the user formulating queries and the information retrieval system. We are particularly interested in the interface issues for such a system. In this paper we describe our multiple viewpoint approach and describe two concrete realizations of such systems: a prototype browser for a body of Earth Science data sets and a new approach to content-based image retrieval. Introduction A multiple viewpoint system [French 2002] retrieves information from a collection of objects (documents, data sets, articles, images, et cetera) using more than one standard information retrieval component; for example, it may index the collection according to more than one set of keywords, may use two or more entirely different information retrieval strategies, or may include a component which retrieves auxiliary information related to the documents which may be used indirectly in searching the collection. When a searcher submits a description of an information need – a query – to an information retrieval system, the system returns those elements of a collection that the system judges most relevant to the query. When the vocabulary used by the searcher in constructing the query does not match the vocabulary used by the system in judging relevance, then the results are unlikely to be what the searcher is actually interested in. Often, due to this mismatch, a searcher may conclude that there are no relevant items in the document collection, when actually, the items exist but have been obscured by the mismatch in vocabulary. For example, a query “rain” might find no documents in a system that has dozens of documents indexed using the keyword “precipitation.” Such mismatches are not reserved for text-based queries. Content-based image retrieval (CBIR) systems use features that can be extracted from the images themselves in order to search collections of images. In image retrieval systems that are queried “by example,” the searcher submits a “query image” that the searcher has judged relevant to the information need, and the system searches based on features extracted from this image. But unless the searcher is very familiar with the behavior of the system, or the image retrieval task is very specifically defined, it is unlikely that the features the user identifies as important in judging this query image relevant to the information need will be the same features the system extracts and uses to judge the relevance of other images to the query image. Background Multiple viewpoints, as we intend the term here, were introduced by Powell and French [Powell 98] and are grounded in Wilbur's work on relevance judgments [Wilbur 98], which showed that retrieval rankings can be improved by using the relevance judgments of several persons, rather than those of one individual, and that of Rajashekar and Croft [Rajashekar 95], who have reported a consistent improvement in document retrieval when two or more document representations are used, rather than a single representation. Multiple viewpoints allow both multiple relevance judgments and multiple representations to be used together. Vocabulary mismatches between a searcher and a search index have been approached in Buckland's Unfamiliar Metadata project [Buckland 99], in which a mapping from ordinary language

A. C. Chapin

Papers

An application of multiple viewpoints to content-based image retrieval

Multiple Viewpoints: A Strategy for Searching Multimedia Content