Presents a review of 200 references in content-based image retrieval. The paper starts with discussing the working conditions of content-based retrieval: patterns of use, types of pictures, the role of semantics, and the sensory gap. Subsequent sections discuss computational steps for image retrieval systems. Step one of the review is image processing for retrieval sorted by color, texture, and local geometry. Features for retrieval are discussed next, sorted by: accumulative and global features, salient points, object and shape features, signs, and structural combinations thereof. Similarity of pictures and objects in pictures is reviewed for each of the feature types, in close connection to the types and means of feedback the user of the systems is capable of giving by interaction. We briefly discuss aspects of system engineering: databases, system architecture, and evaluation. In the concluding section, we present our view on: the driving force of the field, the heritage from computer vision, the influence on computer vision, the role of similarity and of interaction, the need for databases, the problem of evaluation, and the role of the semantic gap.

/pdf/content-based-image-retrieval-at-the-end-of-the-early-years-q45iy9ejai.pdf

Content-based image retrieval at the end of the early years

Humans perceive the three-dimensional structure of the world with apparent ease. However, despite all of the recent advances in computer vision research, the dream of having a computer interpret an image at the same level as a two-year old remains elusive. Why is computer vision such a challenging problem and what is the current state of the art? Computer Vision: Algorithms and Applications explores the variety of techniques commonly used to analyze and interpret images. It also describes challenging real-world applications where vision is being successfully used, both for specialized applications such as medical imaging, and for fun, consumer-level tasks such as image editing and stitching, which students can apply to their own personal photos and videos. More than just a source of recipes, this exceptionally authoritative and comprehensive textbook/reference also takes a scientific approach to basic vision problems, formulating physical models of the imaging process before inverting them to produce descriptions of a scene. These problems are also analyzed using statistical models and solved using rigorous engineering techniques Topics and features: structured to support active curricula and project-oriented courses, with tips in the Introduction for using the book in a variety of customized courses; presents exercises at the end of each chapter with a heavy emphasis on testing algorithms and containing numerous suggestions for small mid-term projects; provides additional material and more detailed mathematical topics in the Appendices, which cover linear algebra, numerical techniques, and Bayesian estimation theory; suggests additional reading at the end of each chapter, including the latest research in each sub-field, in addition to a full Bibliography at the end of the book; supplies supplementary course material for students at the associated website, http://szeliski.org/Book/. Suitable for an upper-level undergraduate or graduate-level course in computer science or engineering, this textbook focuses on basic techniques that work under real-world conditions and encourages students to push their creative boundaries. Its design and exposition also make it eminently suitable as a unique reference to the fundamental techniques and current research literature in computer vision.

/pdf/computer-vision-algorithms-and-applications-25dn6wu83j.pdf

Computer Vision: Algorithms and Applications

We explore the effect of dimensionality on the nearest neighbor problem. We show that under a broad set of conditions (much broader than independent and identically distributed dimensions), as dimensionality increases, the distance to the nearest data point approaches the distance to the farthest data point. To provide a practical perspective, we present empirical results on both real and synthetic data sets that demonstrate that this effect can occur for as few as 10-15 dimensions. These results should not be interpreted to mean that high-dimensional indexing is never meaningful; we illustrate this point by identifying some high-dimensional workloads for which this effect does not occur. However, our results do emphasize that the methodology used almost universally in the database literature to evaluate high-dimensional indexing techniques is flawed, and should be modified. In particular, most such techniques proposed in the literature are not evaluated versus simple linear scan, and are evaluated over workloads for which nearest neighbor is not meaningful. Often, even the reported experiments, when analyzed carefully, show that linear scan would outperform the techniques being proposed on the workloads studied in high (10-15) dimensionality!.

When is nearest neighbor meaningful

This paper presents a new reflectance model for rendering computer synthesized images. The model accounts for the relative brightness of different materials and light sources in the same scene. It describes the directional distribution of the reflected light and a color shift that occurs as the reflectance changes with incidence angle. The paper presents a method for obtaining the spectral energy distribution of the light reflected from an object made of a specific real material and discusses a procedure for accurately reproducing the color associated with the spectral energy distribution. The model is applied to the simulation of a metal and a plastic.

A reflectance model for computer graphics

Images of outdoor scenes captured in bad weather suffer from poor contrast. Under bad weather conditions, the light reaching a camera is severely scattered by the atmosphere. The resulting decay in contrast varies across the scene and is exponential in the depths of scene points. Therefore, traditional space invariant image processing techniques are not sufficient to remove weather effects from images. We present a physics-based model that describes the appearances of scenes in uniform bad weather conditions. Changes in intensities of scene points under different weather conditions provide simple constraints to detect depth discontinuities in the scene and also to compute scene structure. Then, a fast algorithm to restore scene contrast is presented. In contrast to previous techniques, our weather removal algorithm does not require any a priori scene structure, distributions of scene reflectances, or detailed knowledge about the particular weather condition. All the methods described in this paper are effective under a wide range of weather conditions including haze, mist, fog, and conditions arising due to other aerosols. Further, our methods can be applied to gray scale, RGB color, multispectral and even IR images. We also extend our techniques to restore contrast of scenes with moving objects, captured using a video camera.

/pdf/contrast-restoration-of-weather-degraded-images-orn61jh9qu.pdf

Contrast restoration of weather degraded images

The authors present a polarization reflectance model that uses the Fresnel reflection coefficients. This reflectance model accurately predicts the magnitudes of polarization components of reflected light, and all the polarization-based methods presented follow from this model. The authors demonstrate the capability of polarization-based methods to segment material surfaces according to varying levels of relative electrical conductivity, in particular distinguishing dielectrics, which are nonconducting, and metals, which are highly conductive. Polarization-based methods can provide cues for distinguishing different intensity-edge types arising from intrinsic light-dark or color variations, intensity edges caused by specularities, and intensity edges caused by occluding contours where the viewing direction becomes nearly orthogonal to surface normals. Analysis of reflected polarization components is also shown to enable the separation of diffuse and specular components of reflection, unobscuring intrinsic surface detail saturated by specular glare. Polarization-based methods used for constraining surface normals are discussed. >

Constraining object features using a polarization reflectance model

A computationally simple yet powerful method for distinguishing metal and dielectric material surfaces from the polarization characteristics of specularly reflected light is introduced. The method is completely passive, requiring only the sensing of transmitted radiance of reflected light through a polarizing filter positioned in multiple orientations in front of a camera sensor. Precise positioning of lighting is not required. An advantage of using a polarization-based method for material classification is its immunity to color variations, which so commonly exist on uniform material samples. A simple polarization-reflectance model, called the Fresnel reflectance model, is developed. The fundamental assumptions are that the diffuse component of reflection is completely unpolarized and that the polarization state of the specular component of reflection is dictated by the Fresnel reflection coefficients. The material classification method presented results axiomatically from the Fresnel reflectance model, by estimating the polarization Fresnel ratio. No assumptions are required about the functional form of the diffuse and specular components of reflection. The method is demonstrated on some common objects consisting of metal and dielectric parts. >

Polarization-based material classification from specular reflection

Polarization vision: a new sensory approach to image understanding

A key problem for face recognition has been accurate identification under variable illumination conditions. Conventional video cameras sense reflected light so that image grayvalues are a product of both intrinsic skin reflectivity and external incident illumination, thus obfuscating the intrinsic reflectivity of skin. Thermal emission from skin, on the other hand, is an intrinsic measurement that can be isolated from external illumination. We examine the invariance of Long-Wave InfraRed (LWIR) imagery with respect to different illumination conditions from the viewpoint of performance comparisons of two well-known face recognition algorithms applied to LWIR and visible imagery. We develop rigourous data collection protocols that formalize face recognition analysis for computer vision in the thermal IR.

Lawrence B. Wolff

Papers

Constraining object features using a polarization reflectance model

Polarization-based material classification from specular reflection

Polarization-based material classification from specular reflection

Polarization vision: a new sensory approach to image understanding

Illumination invariant face recognition using thermal infrared imagery