The goal of this paper is to present a critical survey of existing literature on human and machine recognition of faces. Machine recognition of faces has several applications, ranging from static matching of controlled photographs as in mug shots matching and credit card verification to surveillance video images. Such applications have different constraints in terms of complexity of processing requirements and thus present a wide range of different technical challenges. Over the last 20 years researchers in psychophysics, neural sciences and engineering, image processing analysis and computer vision have investigated a number of issues related to face recognition by humans and machines. Ongoing research activities have been given a renewed emphasis over the last five years. Existing techniques and systems have been tested on different sets of images of varying complexities. But very little synergism exists between studies in psychophysics and the engineering literature. Most importantly, there exists no evaluation or benchmarking studies using large databases with the image quality that arises in commercial and law enforcement applications In this paper, we first present different applications of face recognition in commercial and law enforcement sectors. This is followed by a brief overview of the literature on face recognition in the psychophysics community. We then present a detailed overview of move than 20 years of research done in the engineering community. Techniques for segmentation/location of the face, feature extraction and recognition are reviewed. Global transform and feature based methods using statistical, structural and neural classifiers are summarized. >

Human and machine recognition of faces: a survey

A scheme is developed for classifying the types of motion perceived by a humanlike robot. It is assumed that the robot receives visual images of the scene using a perspective system model. Equations, theorems, concepts, clues, etc., relating the objects, their positions, and their motion to their images on the focal plane are presented. >

http://ieeexplore.ieee.org/iel2/815/3148/00100651.pdf

Robot vision

Most of the signal processing that we will study in this course involves local operations on a signal, namely transforming the signal by applying linear combinations of values in the neighborhood of each sample point. You are familiar with such operations from Calculus, namely, taking derivatives and you are also familiar with this from optics namely blurring a signal. We will be looking at sampled signals only. Let's start with a few basic examples. Local difference Suppose we have a 1D image and we take the local difference of intensities, DI(x) = 1 2 (I(x + 1) − I(x − 1)) which give a discrete approximation to a partial derivative. (We compute this for each x in the image.) What is the effect of such a transformation? One key idea is that such a derivative would be useful for marking positions where the intensity changes. Such a change is called an edge. It is important to detect edges in images because they often mark locations at which object properties change. These can include changes in illumination along a surface due to a shadow boundary, or a material (pigment) change, or a change in depth as when one object ends and another begins. The computational problem of finding intensity edges in images is called edge detection. We could look for positions at which DI(x) has a large negative or positive value. Large positive values indicate an edge that goes from low to high intensity, and large negative values indicate an edge that goes from high to low intensity. Example Suppose the image consists of a single (slightly sloped) edge:

http://ieeexplore.ieee.org/iel5/5/31307/01456462.pdf

Linear systems

Two-dimensional image motion is the projection of the three-dimensional motion of objects, relative to a visual sensor, onto its image plane. Sequences of time-orderedimages allow the estimation of projected two-dimensional image motion as either instantaneous image velocities or discrete image displacements. These are usually called the optical flow field or the image velocity field. Provided that optical flow is a reliable approximation to two-dimensional image motion, it may then be used to recover the three-dimensional motion of the visual sensor (to within a scale factor) and the three-dimensional surface structure (shape or relative depth) through assumptions concerning the structure of the optical flow field, the three-dimensional environment, and the motion of the sensor. Optical flow may also be used to perform motion detection, object segmentation, time-to-collision and focus of expansion calculations, motion compensated encoding, and stereo disparity measurement. We investigate the computation of optical flow in this survey: widely known methods for estimating optical flow are classified and examined by scrutinizing the hypothesis and assumptions they use. The survey concludes with a discussion of current research issues.

/pdf/the-computation-of-optical-flow-2n5afvnh1s.pdf

The computation of optical flow

The shape from focus method presented here uses different focus levels to obtain a sequence of object images. The sum-modified-Laplacian (SML) operator is developed to provide local measures of the quality of image focus. The operator is applied to the image sequence to determine a set of focus measures at each image point. A depth estimation algorithm interpolates a small number of focus measure values to obtain accurate depth estimates. A fully automated shape from focus system has been implemented using an optical microscope and tested on a variety of industrial samples. Experimental results are presented that demonstrate the accuracy and robustness of the proposed method. These results suggest shape from focus to be an effective approach for a variety of challenging visual inspection tasks. >

Shape from focus

A new method named STM is described for determining distance of objects and rapid autofocusing of camera systems. STM uses image defocus information and is based on a new Spatial-Domain Convolution/Deconvolution Transform. The method requires only two images taken with different camera parameters such as lens position, focal length, and aperture diameter. Both images can be arbitrarily blurred and neither of them needs to be a focused image. Therefore STM is very fast in comparison with Depth-from-Focus methods which search for the lens position or focal length of best focus. The method involves simple local operations and can be easily implemented in parallel to obtain the depth-map of a scene. STM has been implemented on an actual camera system named SPARCS. Experiments on the performance of STM and their results on real-world planar objects are presented. The results indicate that the accuracy of STM compares well with Depth-from-Focus methods and is useful in practical applications. The utility of the method is demonstrated for rapid autofocusing of electronic cameras.

Depth from defocus: a spatial domain approach

A method is described for selecting the optimal focus measure with respect to gray-level noise from a given set of focus measures in passive autofocusing and depth-from-focus applications. The method is based on two new metrics that have been defined for estimating the noise-sensitivity of different focus measures. The first metric-the autofocusing uncertainty measure (AUM)-is useful in understanding the relation between gray-level noise and the resulting error in lens position for autofocusing. The second metric-autofocusing root-mean-square error (ARMS error)-is an improved metric closely related to AUM. AUM and ARMS error metrics are based on a theoretical noise sensitivity analysis of focus measures, and they are related by a monotonic expression. The theoretical results are validated by actual and simulation experiments. For a given camera, the optimally accurate focus measure may change from one object to the other depending on their focused images. Therefore, selecting the optimal focus measure from a given set involves computing all focus measures in the set.

http://www.sc.ehu.es/ccwgrrom/transparencias/articulos-alumnos-doct-2002/josu-larra%2596aga/00709612.pdf

Selecting the optimal focus measure for autofocusing and depth-from-focus

A new shape-from-focus method is described which is based on a new concept, named focused image surface (FIS). FIS of an object is defined as the surface formed by the set of points at which the object points are focused by a camera lens. According to paraxial-geometric optics, there is a one-to-one correspondence between the shape of an object and the shape of its FIS. Therefore, the problem of shape recovery can be posed as the problem of determining the shape of the FIS. From the shape of FIS the shape of the object is easily obtained. In this paper the shape of the FIS is determined by searching for a shape which maximizes a focus measure. In contrast to previous literature where the focus measure is computed over the planar image detector of the camera, here the focus measure is computed over the FIS. This results in more accurate shape recovery than the traditional methods. Also, using FIS, a more accurate focused image can be reconstructed from a sequence of images than is possible with traditional methods. The new method has been implemented on an actual camera system, and the results of shape recovery and focused image reconstruction are presented. >

Accurate recovery of three-dimensional shape from image focus

A new method is described for recovering the distance of objects in a scene from images formed by lenses. The recovery is based on measuring the change in the scene’s image due to a known change in the three intrinsic camera parameters: (i) distance between the lens md the image detector, (ii) focal length of the lens, and (iii) diameter of the lens aperture. The method is parallel involving simple local computations. In comparison with stereo vision and structure-from-motion methods, the correspondence problem does not arise. This method for depthmap recovery may also be used for (i) obtaining focused images (i.e. images having large depth of field) from two images having finite depth of field, and (ii) rapid autofocusing of computer controlled video cameras.

/pdf/parallel-depth-recovery-by-changing-camera-parameters-py0ac17p4j.pdf

Parallel Depth Recovery By Changing Camera Parameters

Two new methods are presented for recovering the focused image of an object from only two blurred images recorded with different camera parameter settings. The camera parameters include lens position, focal length, and aperture diameter. First a blur parameter 0 is estimated using one of our two recently proposed depth-from-defocus methods. Then one of the two blurred images is deconvolved to recover the focused image. The first method is based on a recently proposed spatial domain convolutioddeconvolution transform. This method requires only the knowledge of ~7 of the camera’s point spread function (PSF). It does not require information about the actual form of the camera’s PSF. The second method, in contrast to the first, requires full knowledge of the form of the PSF. As part of the second method, we present a calibration procedure for estimating the camera’s PSF for different values of the blur parameter 0. In the second method, the focused image is obtained through deconvolution in the Fourier domain using the Wiener filter. For both methods, results of experiments on actual defocused images recorded by a CCD camera are given. The first method requires much less computation than the second method. The first method gives satisfactory results for up to medium levels of blur and the second method gives good results for up to relatively high levels of blur.

/pdf/focused-image-recovery-from-two-defocused-images-recorded-1rj29zhr25.pdf

Murali Subbarao

Papers

Depth from defocus: a spatial domain approach

Selecting the optimal focus measure for autofocusing and depth-from-focus

Accurate recovery of three-dimensional shape from image focus

Parallel Depth Recovery By Changing Camera Parameters

Focused image recovery from two defocused images recorded with different camera settings