Retrieving images from large and varied collections using image content as a key is a challenging and important problem We present a new image representation that provides a transformation from the raw pixel data to a small set of image regions that are coherent in color and texture This "Blobworld" representation is created by clustering pixels in a joint color-texture-position feature space The segmentation algorithm is fully automatic and has been run on a collection of 10,000 natural images We describe a system that uses the Blobworld representation to retrieve images from this collection An important aspect of the system is that the user is allowed to view the internal representation of the submitted image and the query results Similar systems do not offer the user this view into the workings of the system; consequently, query results from these systems can be inexplicable, despite the availability of knobs for adjusting the similarity metrics By finding image regions that roughly correspond to objects, we allow querying at the level of objects rather than global image properties We present results indicating that querying for images using Blobworld produces higher precision than does querying using color and texture histograms of the entire image in cases where the image contains distinctive objects

/pdf/blobworld-image-segmentation-using-expectation-maximization-5dehsdvird.pdf

Blobworld: image segmentation using expectation-maximization and its application to image querying

A review of content-based image retrieval systems in medical applications—clinical benefits and future directions

In many areas of commerce, government, academia, and hospitals, large collections of digital im-
ages are being created. Many of these collections are the product of digitizing existing collections of
analogue photographs, diagrams, drawings, paintings, and prints. Usually, the only way of search-
ing these collections was by keyword indexing, or simply by browsing. Digital images databases
however, open the way to content-based searching. In this paper we survey some technical aspects
of current content-based image retrieval systems.

Content-based image retrieval systems: A survey

Suggested by the structure of the visual nervous system, a new algorithm is proposed for pattern recognition. This algorithm can be realized with a multilayered network consisting of neuron-like cells. The network, “neocognitron”, is self-organized by unsupervised learning, and acquires the ability to recognize stimulus patterns according to the differences in their shapes: Any patterns which we human beings judge to be alike are also judged to be of the same category by the neocognitron. The neocognitron recognizes stimulus patterns correctly without being affected by shifts in position or even by considerable distortions in shape of the stimulus patterns.

Neocognitron--A New Algorithm for Pattern Recognition Tolerant of Deformations and Shifts in Position

Retrieving images from large and varied collections using image content as a key is a challenging and important problem. In this paper we present a new image representation which provides a transformation from the raw pixel data to a small set of image regions which are coherent in color and texture space. This so-called "blobworld" representation is based on segmentation using the expectation-maximization algorithm on combined color and texture features. The texture features we use for the segmentation arise from a new approach to texture description and scale selection. We describe a system that uses the blobworld representation to retrieve images. An important and unique aspect of the system is that, in the context of similarity-based querying, the user is allowed to view the internal representation of the submitted image and the query results. Similar systems do not offer the user this view into the workings of the system; consequently, the outcome of many queries on these systems can be quite inexplicable, despite the availability of knobs for adjusting the similarity metric.

/pdf/color-and-texture-based-image-segmentation-using-em-and-its-4zuz12eytx.pdf

Color- and texture-based image segmentation using EM and its application to content-based image retrieval

CANDID (Comparison Algorithm for Navigating Digital Image Databases) was developed to enable content-based retrieval of digital imagery from large databases using a query-by-example methodology. A user provides an example image to the system, and images in the database that are similar to that example are retrieved. The development of CANDID was inspired by the N-gram approach to document fingerprinting, where a ``global signature`` is computed for every document in a database and these signatures are compared to one another to determine the similarity between any two documents. CANDID computes a global signature for every image in a database, where the signature is derived from various image features such as localized texture, shape, or color information. A distance between probability density functions of feature vectors is then used to compare signatures. In this paper, the authors present CANDID and highlight two results from their current research: subtracting a ``background`` signature from every signature in a database in an attempt to improve system performance when using inner-product similarity measures, and visualizing the contribution of individual pixels in the matching process. These ideas are applicable to any histogram-based comparison technique.

/pdf/query-by-image-example-the-candid-approach-16lzawxgcf.pdf

Query by image example: The CANDID approach

CANDID (comparison algorithm for navigating digital image databases) was developed to enable content-based retrieval of digital imagery from large databases using a query-by- example methodology. A user provides an example image to the system, and images in the database that are similar to that example are retrieved. The development of CANDID was inspired by the N-gram approach to document fingerprinting, where a `global signature' is computed for every document in a database and these signatures are compared to one another to determine the similarity between any two documents. CANDID computes a global signature for every image in a database, where the signature is derived from various image features such as localized texture, shape, or color information. A distance between probability density functions of feature vectors is then used to compare signatures. In this paper, we present CANDID and highlight two results from our current research: subtracting a `background' signature from every signature in a database in an attempt to improve system performance when using inner-product similarity measures, and visualizing the contribution of individual pixels in the matching process. These ideas are applicable to any histogram-based comparison technique.

Query by image example: the comparison algorithm for navigating digital image databases (CANDID) approach

Quantitative methods are developed to use optical scatter to measure the critical dimensions of gratings etched into bulk Si and developed photoresist patterns on silicon substrates. Previous work either classified microstructures qualitatively or employed a 'chi-by-eye' method to find that structures were similar or dissimilar. A single detector scanning scatterometer is used to measure large 32 micrometers pitch structures while another instrument that varies the angle of incidence and tracks diffracted orders via the grating equation is used to measure 2 micrometers pitch structures. A rigorous coupled wave light scatter model is used to simulate diffraction from a set of test wafers. Partial least squares and neural network analysis techniques are then employed to use correlations between the simulated diffraction and the critical dimensions of the modeled structures to produce a capability to measure the critical dimensions from scattered light measurements. The marriage of rigorous coupled wave diffraction modeling and optical scatterometry directly addresses the needs of the industry for a rapid and nondestructive metrology tool.

Using scattered light modeling for semiconductor critical dimension metrology and calibration

This paper presents a Generalized Logistic (gLG) distribution as a unified model for Log-domain synthetic aperture Radar (SAR) data. This model stems from a special case of the G-distribution known as the G{sup 0}-distribution. The G-distribution arises from a multiplicative SAR model and has the classical K-distribution as another special case. The G{sup 0}-distribution, however, can model extremely heterogeneous clutter regions that the k-distribution cannot model. This flexibility is preserved in the unified gLG model, which is capable of modeling non-polarimetric SAR returns from clutter as well as man-made objects. Histograms of these two types of SAR returns have opposite skewness. The flexibility of the gLG model lies in its shape and shift parameters. The shape parameter describes the differing skewness between target and clutter data while the shift parameter compensates for movements in the mean as the shape parameter changes. A Maximum Likelihood (ML) estimate of the shape parameter gives an optimal measure of the skewness of the SAR data. This measure provides a basis for an optimal target detection algorithm.

/pdf/statistical-modeling-of-targets-and-clutter-in-single-look-u4hdvwt4sv.pdf

Statistical modeling of targets and clutter in single-look non-polarimetric SAR imagery

We have applied optical scatter techniques to improve several aspects of microelectronic manufacturing. One technique involves characterizing light scattered from two dimensional device structures, such as those from VLSI circuitry etched on a wafer, using a frosted dome which is imaged by a CCD camera. Previously, limited dynamic range available from affordable digital imaging systems has prevented the study of two dimensional scatter patterns. We have demonstrated a simple technique to increase the dynamic range by combining multiple images taken at different intensities. After the images have been acquired, image processing techniques are used to find and catalog the diffraction orders. Techniques such as inverse least squares, principal component analysis, and neural networks are then used to evaluate the dependence of the light scatter on a particular wafer characteristic under examination. Characterization of surface planarization over a VLSI structure and measurement of line edge roughness of diffraction gratings are presented as examples.© (1992) COPYRIGHT SPIE--The International Society for Optical Engineering. Downloading of the abstract is permitted for personal use only.

Donald R. Hush

Papers

Query by image example: The CANDID approach

Query by image example: the comparison algorithm for navigating digital image databases (CANDID) approach

Using scattered light modeling for semiconductor critical dimension metrology and calibration

Statistical modeling of targets and clutter in single-look non-polarimetric SAR imagery

Wafer examination and critical dimension estimation using scattered light