A general non-parametric technique is proposed for the analysis of a complex multimodal feature space and to delineate arbitrarily shaped clusters in it. The basic computational module of the technique is an old pattern recognition procedure: the mean shift. For discrete data, we prove the convergence of a recursive mean shift procedure to the nearest stationary point of the underlying density function and, thus, its utility in detecting the modes of the density. The relation of the mean shift procedure to the Nadaraya-Watson estimator from kernel regression and the robust M-estimators; of location is also established. Algorithms for two low-level vision tasks discontinuity-preserving smoothing and image segmentation - are described as applications. In these algorithms, the only user-set parameter is the resolution of the analysis, and either gray-level or color images are accepted as input. Extensive experimental results illustrate their excellent performance.

Mean shift: a robust approach toward feature space analysis

As a first step towards a perceptual user interface, a computer vision color tracking algorithm is developed and applied towards tracking human faces. Computer vision algorithms that are intended to form part of a perceptual user interface must be fast and efficient. They must be able to track in real time yet not absorb a major share of computational resources: other tasks must be able to run while the visual interface is being used. The new algorithm developed here is based on a robust nonparametric technique for climbing density gradients to find the mode (peak) of probability distributions called the mean shift algorithm. In our case, we want to find the mode of a color distribution within a video scene. Therefore, the mean shift algorithm is modified to deal with dynamically changing color probability distributions derived from video frame sequences. The modified algorithm is called the Continuously Adaptive Mean Shift (CAMSHIFT) algorithm. CAMSHIFT’s tracking accuracy is compared against a Polhemus tracker. Tolerance to noise, distractors and performance is studied. CAMSHIFT is then used as a computer interface for controlling commercial computer games and for exploring immersive 3D graphic worlds. Introduction This paper is part of a program to develop a Perceptual User Interface for computers. Perceptual interfaces are ones in which the computer is given the ability to sense and produce analogs of the human senses, such as allowing computers to perceive and produce localized sound and speech, giving computers a sense of touch and force feedback, and in our case, giving computers an ability to see. The work described in this paper is part of a larger effort aimed at giving computers the ability to segment, track, and understand the pose, gestures, and emotional expressions of humans and the tools they might be using in front of a computer or settop box. In this paper we describe the development of the first core module in this effort: a 4-degree of freedom color object tracker and its application to flesh-tone-based face tracking. Computer vision face tracking is an active and developing field, yet the face trackers that have been developed are not sufficient for our needs. Elaborate methods such as tracking contours with snakes [[10][12][13]], using Eigenspace matching techniques [14], maintaining large sets of statistical hypotheses [15], or convolving images with feature detectors [16] are far too computationally expensive. We want a tracker that will track a given face in the presence of noise, other faces, and hand movements. Moreover, it must run fast and efficiently so that objects may be tracked in real time (30 frames per second) while consuming as few system resources as possible. In other words, this tracker should be able to serve as part of a user interface that is in turn part of the computational tasks that a computer might routinely be expected to carry out. This tracker also needs to run on inexpensive consumer cameras and not require calibrated lenses. In order, therefore, to find a fast, simple algorithm for basic tracking, we have focused on color-based tracking [[7][8][9][10][11]], yet even these simpler algorithms are too computationally complex (and therefore slower at any given CPU speed) due to their use of color correlation, blob and region growing, Kalman filter smoothing and prediction, and contour considerations. The complexity of the these algorithms derives from their attempts to deal with irregular object motion due to perspective (near objects to the camera seem to move faster than distal objects); image noise; distractors, such as other faces in the scene; facial occlusion by hands or other objects; and lighting variations. We want a fast, computationally efficient algorithm that handles these problems in the course of its operation, i.e., an algorithm that mitigates the above problems “for free.” To develop such an algorithm, we drew on ideas from robust statistics and probability distributions. Robust statistics are those that tend to ignore outliers in the data (points far away from the region of interest). Thus, robust Intel Technology Journal Q2 ‘98 Computer Vision Face Tracking For Use in a Perceptual User Interface 2 algorithms help compensate for noise and distractors in the vision data. We therefore chose to use a robust nonparametric technique for climbing density gradients to find the mode of probability distributions called the mean shift algorithm [2]. (The mean shift algorithm was never intended to be used as a tracking algorithm, but it is quite effective in this role.) The mean shift algorithm operates on probability distributions. To track colored objects in video frame sequences, the color image data has to be represented as a probability distribution [1]; we use color histograms to accomplish this. Color distributions derived from video image sequences change over time, so the mean shift algorithm has to be modified to adapt dynamically to the probability distribution it is tracking. The new algorithm that meets all these requirements is called CAMSHIFT. For face tracking, CAMSHIFT tracks the X, Y, and Area of the flesh color probability distribution representing a face. Area is proportional to Z, the distance from the camera. Head roll is also tracked as a further degree of freedom. We then use the X, Y, Z, and Roll derived from CAMSHIFT face tracking as a perceptual user interface for controlling commercial computer games and for exploring 3D graphic virtual worlds. Choose initial search window size and location HSV Image Set calculation region at search window center but larger in size than the search window Color histogram lookup in calculation region Color probability distribution Find center of mass within the search window Center search window at the center of mass and find area under it Converged YES NO Report X, Y, Z, and Roll Use (X,Y) to set search window center, 2*area 1/2

/pdf/computer-vision-face-tracking-for-use-in-a-perceptual-user-53lqzf175w.pdf

Computer Vision Face Tracking For Use in a Perceptual User Interface

The investment in an automated fabric defect detection system is more than economical when reduction in labor cost and associated benefits are considered. The development of a fully automated web inspection system requires robust and efficient fabric defect detection algorithms. The inspection of real fabric defects is particularly challenging due to the large number of fabric defect classes, which are characterized by their vagueness and ambiguity. Numerous techniques have been developed to detect fabric defects and the purpose of this paper is to categorize and/or describe these algorithms. This paper attempts to present the first survey on fabric defect detection techniques presented in about 160 references. Categorization of fabric defect detection techniques is useful in evaluating the qualities of identified features. The characterization of real fabric surfaces using their structure and primitive set has not yet been successful. Therefore, on the basis of the nature of features from the fabric surfaces, the proposed approaches have been characterized into three categories; statistical, spectral and model-based. In order to evaluate the state-of-the-art, the limitations of several promising techniques are identified and performances are analyzed in the context of their demonstrated results and intended application. The conclusions from this paper also suggest that the combination of statistical, spectral and model-based approaches can give better results than any single approach, and is suggested for further research.

/pdf/computer-vision-based-fabric-defect-detection-a-survey-27mr8b2pf3.pdf

Computer-Vision-Based Fabric Defect Detection: A Survey

Saliency detection plays important roles in many image processing applications, such as regions of interest extraction and image resizing. Existing saliency detection models are built in the uncompressed domain. Since most images over Internet are typically stored in the compressed domain such as joint photographic experts group (JPEG), we propose a novel saliency detection model in the compressed domain in this paper. The intensity, color, and texture features of the image are extracted from discrete cosine transform (DCT) coefficients in the JPEG bit-stream. Saliency value of each DCT block is obtained based on the Hausdorff distance calculation and feature map fusion. Based on the proposed saliency detection model, we further design an adaptive image retargeting algorithm in the compressed domain. The proposed image retargeting algorithm utilizes multioperator operation comprised of the block-based seam carving and the image scaling to resize images. A new definition of texture homogeneity is given to determine the amount of removal block-based seams. Thanks to the directly derived accurate saliency information from the compressed domain, the proposed image retargeting algorithm effectively preserves the visually important regions for images, efficiently removes the less crucial regions, and therefore significantly outperforms the relevant state-of-the-art algorithms, as demonstrated with the in-depth analysis in the extensive experiments.

/pdf/saliency-detection-in-the-compressed-domain-for-adaptive-2yycmtymex.pdf

Saliency Detection in the Compressed Domain for Adaptive Image Retargeting

We present a novel approach to estimate traffic speed using a sequence of images from an uncalibrated camera. We assert that exact calibration is not necessary to estimate speed. Instead, we use: 1) geometric relationships inherently available in the image, 2) some common-sense assumptions that reduce the problem to a one-dimensional geometry, 3) frame differencing to isolate moving edges and track vehicles between frames, and 4) parameters from the distribution of vehicle lengths to estimate speed.

An algorithm to estimate mean traffic speed using uncalibrated cameras

Color image segmentation by pixel classification in an adapted hybrid color space: application to soccer image analysis

Catching moving objects with snakes for motion tracking

Color image segmentation by analysis of subset connectedness and color homogeneity properties

Clustering by mode boundary detection

This chapter presents a new color image segmentation method using fuzzy mathematical morphological filters applied to the 3D color histogram. Segmentation consists in detecting the different modes which are present in the 3D color histogram and associated with homogeneous regions. In order to detect these modes, we show how a color image can be considered as a fuzzy subset of pixels characterized by its membership function to a mode. This function is evaluated. thanks to a concavity analysis of the 3D color histogram. Then, a fuzzy morphological transformation is applied to this membership function in order to enhance the modes. The effectiveness of our proposed fuzzy morphological approach is then illustrated with color images.

Jack-Gérard Postaire

Papers

Color image segmentation by pixel classification in an adapted hybrid color space: application to soccer image analysis

Catching moving objects with snakes for motion tracking

Color image segmentation by analysis of subset connectedness and color homogeneity properties

Clustering by mode boundary detection

Color Image Segmentation by Analysis of 3D Histogram with Fuzzy Morphological Filters