A threshold selection method from gray level histograms

This paper describes a computational approach to edge detection. The success of the approach depends on the definition of a comprehensive set of goals for the computation of edge points. These goals must be precise enough to delimit the desired behavior of the detector while making minimal assumptions about the form of the solution. We define detection and localization criteria for a class of edges, and present mathematical forms for these criteria as functionals on the operator impulse response. A third criterion is then added to ensure that the detector has only one response to a single edge. We use the criteria in numerical optimization to derive detectors for several common image features, including step edges. On specializing the analysis to step edges, we find that there is a natural uncertainty principle between detection and localization performance, which are the two main goals. With this principle we derive a single operator shape which is optimal at any scale. The optimal detector has a simple approximate implementation in which edges are marked at maxima in gradient magnitude of a Gaussian-smoothed image. We extend this simple detector using operators of several widths to cope with different signal-to-noise ratios in the image. We present a general method, called feature synthesis, for the fine-to-coarse integration of information from operators at different scales. Finally we show that step edge detector performance improves considerably as the operator point spread function is extended along the edge.

A Computational Approach to Edge Detection

Texture is one of the important characteristics used in identifying objects or regions of interest in an image, whether the image be a photomicrograph, an aerial photograph, or a satellite image. This paper describes some easily computable textural features based on gray-tone spatial dependancies, and illustrates their application in category-identification tasks of three different kinds of image data: photomicrographs of five kinds of sandstones, 1:20 000 panchromatic aerial photographs of eight land-use categories, and Earth Resources Technology Satellite (ERTS) multispecial imagery containing seven land-use categories. We use two kinds of decision rules: one for which the decision regions are convex polyhedra (a piecewise linear decision rule), and one for which the decision regions are rectangular parallelpipeds (a min-max decision rule). In each experiment the data set was divided into two parts, a training set and a test set. Test set identification accuracy is 89 percent for the photomicrographs, 82 percent for the aerial photographic imagery, and 83 percent for the satellite imagery. These results indicate that the easily computable textural features probably have a general applicability for a wide variety of image-classification applications.

Textural Features for Image Classification

Multiresolution representations are effective for analyzing the information content of images. The properties of the operator which approximates a signal at a given resolution were studied. It is shown that the difference of information between the approximation of a signal at the resolutions 2/sup j+1/ and 2/sup j/ (where j is an integer) can be extracted by decomposing this signal on a wavelet orthonormal basis of L/sup 2/(R/sup n/), the vector space of measurable, square-integrable n-dimensional functions. In L/sup 2/(R), a wavelet orthonormal basis is a family of functions which is built by dilating and translating a unique function psi (x). This decomposition defines an orthogonal multiresolution representation called a wavelet representation. It is computed with a pyramidal algorithm based on convolutions with quadrature mirror filters. Wavelet representation lies between the spatial and Fourier domains. For images, the wavelet representation differentiates several spatial orientations. The application of this representation to data compression in image coding, texture discrimination and fractal analysis is discussed. >

/pdf/a-theory-for-multiresolution-signal-decomposition-the-530tw5pzq7.pdf

A theory for multiresolution signal decomposition: the wavelet representation

We make an analogy between images and statistical mechanics systems. Pixel gray levels and the presence and orientation of edges are viewed as states of atoms or molecules in a lattice-like physical system. The assignment of an energy function in the physical system determines its Gibbs distribution. Because of the Gibbs distribution, Markov random field (MRF) equivalence, this assignment also determines an MRF image model. The energy function is a more convenient and natural mechanism for embodying picture attributes than are the local characteristics of the MRF. For a range of degradation mechanisms, including blurring, nonlinear deformations, and multiplicative or additive noise, the posterior distribution is an MRF with a structure akin to the image model. By the analogy, the posterior distribution defines another (imaginary) physical system. Gradual temperature reduction in the physical system isolates low energy states (``annealing''), or what is the same thing, the most probable states under the Gibbs distribution. The analogous operation under the posterior distribution yields the maximum a posteriori (MAP) estimate of the image given the degraded observations. The result is a highly parallel ``relaxation'' algorithm for MAP estimation. We establish convergence properties of the algorithm and we experiment with some simple pictures, for which good restorations are obtained at low signal-to-noise ratios.

/pdf/stochastic-relaxation-gibbs-distributions-and-the-bayesian-4c0mbkv82j.pdf

Stochastic Relaxation, Gibbs Distributions, and the Bayesian Restoration of Images

Many types of common objects, such as tools and vehicles, usually move in simple ways when they are wielded or driven. The natural axes of the object tend to remain aligned with the local trihedron defined by the object's trajectory. Based on this observation we use a model called Frenet-Serret motion which corresponds to the motion of a moving trihedron along a space curve. Knowing how the Frenet-Serret frame is changing relative to the observer gives us essential information for understanding the object's motion. This is illustrated here for four examples, involving tools (a wrench and a saw) and vehicles (an accelerating van, a turning taxi).

/pdf/understanding-the-motions-of-tools-and-vehicles-go0jgggmgs.pdf

Understanding the motions of tools and vehicles

An approximation technique for photometric stereo

We can judge the importance of automatic detection and tracking of facial features, such as eyes, irises, eyelids, etc., by the list of potential applications. Some of these applications are in human-computer interaction; for instance, the computer may want to know what the user is looking at on the screen. Other applications can be found in compression techniques like MPEG4, where eye and iris information is part of the communication stream. Still another application can be found in driver behavior analysis. In this application, attentiveness on the driver's part is directly related to safety. For example, the car may want to know when the driver is tired and keeps closing his/her eyes; if the car finds that the driver is not attentive it may send visual or auditory signals to get the driver's attention. Of course, identifying and locating the face and the eyes are integral parts of most face recognition and classification algorithms, where eye information is primarily used to normalize the face. The contributions of this thesis are in two areas. The first is detection and verification of the face by locating the eyes in a color image of the face. The second is location of features of the eyes (the irises and eyelids) and identification of their behavior by tracking them over a sequence of images. 
The face is detected as a large flesh-colored region, and anthropometric data are then used to estimate the size and separation of the eyes. Our first method of eye detection uses a linear filtering approach applied to the gray-level image of the face; our second method uses nonlinear filters, applied to the color face image, to detect the corners of the eyes. Both methods were tested on two datasets. The first method had a good detection rate, but also gave many false alarms; the second method had a 90% detection rate with no false alarms. 
After locating the eye corners, we detect the eyelids and irises in every frame of an image sequence. We perform a frame-to-frame analysis of the movements of the irises and eyelids to determine changes in gaze direction and blinking, respectively. This analysis is improved upon by using motion information in the form of normal flow. We model the head and eyes and determine their respective motions. Using our models we determine the head-independent motions of the irises and eyelids by stabilizing for the head motion. The head-independent motions of the irises are used to determine behaviors like saccades and smooth pursuit. Tracking the upper eyelid and using the distance between its apex and the center of the iris, we detect instances of eye closure during blinking. In our experiments, we successfully located the irises in every frame in which the eye was fully or partially open. The eyelids were successfully located 80% of the time.

Eye detection and tracking

Sparse, opaque three-dimensional texture, 2B: Photometry

We propose an alternative way to study the problem of visual recognition which is closer to the spirit emerging from Brooks' work on building robots than to Marr's reconstructive approach. Our theory is purposive in the sense that recognition is considered in the context of an agent performing it in an environment, along with the agent's intentions that translate into a set of behaviors; it is qualitative in the sense that only partial recovery is needed; it is active in the sense that various partial recovery tasks need for recognition are achieved through active vision; and it is opportunistic in the sense that every available cue is used.© (1992) COPYRIGHT SPIE--The International Society for Optical Engineering. Downloading of the abstract is permitted for personal use only.

Azriel Rosenfeld

Papers

Understanding the motions of tools and vehicles

An approximation technique for photometric stereo

Eye detection and tracking

Sparse, opaque three-dimensional texture, 2B: Photometry

Purposive recognition: an active and qualitative approach