Showing papers on "Three-dimensional face recognition published in 2004"

Facial action recognition for facial expression analysis from static face images

Abstract: Automatic recognition of facial gestures (i.e., facial muscle activity) is rapidly becoming an area of intense interest in the research field of machine vision. In this paper, we present an automated system that we developed to recognize facial gestures in static, frontal- and/or profile-view color face images. A multidetector approach to facial feature localization is utilized to spatially sample the profile contour and the contours of the facial components such as the eyes and the mouth. From the extracted contours of the facial features, we extract ten profile-contour fiducial points and 19 fiducial points of the contours of the facial components. Based on these, 32 individual facial muscle actions (AUs) occurring alone or in combination are recognized using rule-based reasoning. With each scored AU, the utilized algorithm associates a factor denoting the certainty with which the pertinent AU has been scored. A recognition rate of 86% is achieved.

A discriminative feature space for detecting and recognizing faces

Abstract: We introduce a novel discriminative feature space which is efficient not only for face detection but also for recognition. The face representation is based on local binary patterns (LBP) and consists of encoding both local and global facial characteristics into a compact feature histogram. The proposed representation is invariant with respect to monotonic gray scale transformations and can be derived in a single scan through the image. Considering the derived feature space, a second-degree polynomial kernel SVM classifier was trained to detect frontal faces in gray scale images. Experimental results using several complex images show that the proposed approach performs favorably compared to the state-of-the-art methods. Additionally, experiments with detecting and recognizing low-resolution faces from video sequences were carried out, demonstrating that the same facial representation can be efficiently used for both detection and recognition.

Appearance-based face recognition and light-fields

Abstract: Arguably the most important decision to be made when developing an object recognition algorithm is selecting the scene measurements or features on which to base the algorithm. In appearance-based object recognition, the features are chosen to be the pixel intensity values in an image of the object. These pixel intensities correspond directly to the radiance of light emitted from the object along certain rays in space. The set of all such radiance values over all possible rays is known as the plenoptic function or light-field. In this paper, we develop a theory of appearance-based object recognition from light-fields. This theory leads directly to an algorithm for face recognition across pose that uses as many images of the face as are available, from one upwards. All of the pixels, whichever image they come from, are treated equally and used to estimate the (eigen) light-field of the object. The eigen light-field is then used as the set of features on which to base recognition, analogously to how the pixel intensities are used in appearance-based face and object recognition.

Boosting local binary pattern (LBP)-Based face recognition

Abstract: This paper presents a novel approach for face recognition by boosting statistical local features based classifiers The face image is scanned with a scalable sub-window from which the Local Binary Pattern (LBP) histograms [14] are obtained to describe the local features of a face image The multi-class problem of face recognition is transformed into a two-class one by classifying every two face images as intra-personal or extra-personal ones [9] The Chi square distance between corresponding Local Binary Pattern histograms of two face images is used as discriminative feature for intra/extra-personal classification We use AdaBoost algorithm to learn a similarity of every face image pairs The proposed method was tested on the FERET FA/FB image sets and yielded an exciting recognition rate of 979%

Evaluation of Face Resolution for Expression Analysis

Abstract: Most automatic facial expression analysis (AFEA) systems attempt to recognize facial expressions from data collected in a highly controlled environment with very high resolution frontal faces ( face regions greater than 200 x 200 pixels). However, in real environments, the face image is often in lower resolution and with head motion. It is unclear that the performance of AFEA systems for low resolution face images. The general approach to AFEA consists of 3 steps: face acquisition, facial feature extraction, and facial expression recognition. This paper explores the effects of different image resolutions for each step of facial expression analysis. The different approaches are compared for face detection, face data extraction and expression recognition. A total of five different resolutions of the head region are studied (288x384, 144x192, 72x96, 36x48, and 18Xx24) based on a widely used public database. The lower resolution images are down-sampled from the originals.

Component-Based Face Recognition with 3D Morphable Models

Abstract: We present a system for pose and illumination invariant face recognition that combines two recent advances in the computer vision field: 3D morphable models and component-based recognition. A 3D morphable model is used to compute 3D face models from three input images of each subject in the training database. The 3D models are rendered under varying pose and illumination conditions to build a large set of synthetic images. These images are then used for training a component-based face recognition system. The face recognition module is preceded by a fast hierarchical face detector resulting in a system that can detect and identify faces in video images at about 4 Hz. The system achieved a recognition rate of 88% on a database of 2000 real images of ten people, which is significantly better than a comparable global face recognition system. The results clearly show the potential of the combination of morphable models and component-based recognition towards pose and illumination invariant face recognition.

Machine learning methods for fully automatic recognition of facial expressions and facial actions

Abstract: We present a systematic comparison of machine learning methods applied to the problem of fully automatic recognition of facial expressions. We explored recognition of facial actions from the facial action coding system (FACS), as well as recognition of fall facial expressions. Each video-frame is first scanned in real-time to detect approximately upright frontal faces. The faces found are scaled into image patches of equal size, convolved with a bank of Gabor energy filters, and then passed to a recognition engine that codes facial expressions into 7 dimensions in real time: neutral, anger, disgust, fear, joy, sadness, surprise. We report results on a series of experiments comparing recognition engines, including AdaBoost, support vector machines, linear discriminant analysis, as well as feature selection techniques. Best results were obtained by selecting a subset of Gabor filters using AdaBoost and then training support vector machines on the outputs of the filters selected by AdaBoost. The generalization performance to new subjects for recognition of full facial expressions in a 7-way forced choice was 93% correct, the best performance reported so far on the Cohn-Kanade FACS-coded expression dataset. We also applied the system to fully automated facial action coding. The present system classifies 18 action units, whether they occur singly or in combination with other actions, with a mean agreement rate of 94.5% with human FACS codes in the Cohn-Kanade dataset. The outputs of the classifiers change smoothly as a function of time and thus can be used to measure facial expression dynamics.

Face recognition using Ada-Boosted Gabor features

Abstract: Face representation based on Gabor features has attracted much attention and achieved great success in face recognition area for the advantages of the Gabor features. However, Gabor features currently adopted by most systems are redundant and too high dimensional. In this paper, we propose a face recognition method using AdaBoosted Gabor features, which are not only low dimensional but also discriminant. The main contribution of the paper lies in two points: (1) AdaBoost is successfully applied to face recognition by introducing the intra-face and extra-face difference space in the Gabor feature space; (2) an appropriate re-sampling scheme is adopted to deal with the imbalance between the amount of the positive samples and that of the negative samples. By using the proposed method, only hundreds of Gabor features are selected. Experiments on FERET database have shown that these hundreds of Gabor features are enough to achieve good performance comparable to that of methods using the complete set of Gabor features.

Automatic 3D reconstruction for face recognition

Abstract: An analysis-by-synthesis framework for face recognition with variant pose, illumination and expression (PIE) is proposed in this paper. First, an efficient 2D-to-3D integrated face reconstruction approach is introduced to reconstruct a personalized 3D face model from a single frontal face image with neutral expression and normal illumination. Then, realistic virtual faces with different PIE are synthesized based on the personalized 3D face to characterize the face subspace. Finally, face recognition is conducted based on these representative virtual faces. Compared with other related works, this framework has the following advantages: 1) only one single frontal face is required for face recognition, which avoids the burdensome enrollment work; 2) the synthesized face samples provide the capability to conduct recognition under difficult conditions like complex PIE; and 3) the proposed 2D-to-3D integrated face reconstruction approach is fully automatic and more efficient. The extensive experimental results show that the synthesized virtual faces significantly improve the accuracy of face recognition with variant PIE.

A survey of approaches to three-dimensional face recognition

Abstract: The vast majority of face recognition research has focused on the use of two-dimensional intensity images, and is covered in existing survey papers. This survey focuses on face recognition using three-dimensional data, either alone or in combination with two-dimensional intensity images. Challenges involved in developing more accurate three-dimensional face recognition are identified.

Three-dimensional model based face recognition

Abstract: The performance of face recognition systems that use two-dimensional (2D) images is dependent on consistent conditions such as lighting, pose and facial expression. We are developing a multi-view face recognition system that utilizes three-dimensional (3D) information about the face to make the system more robust to these variations. This work describes a procedure for constructing a database of 3D face models and matching this database to 2.5D face scans which are captured from different views, using coordinate system invariant properties of the facial surface. 2.5D is a simplified 3D (x, y, z) surface representation that contains at most one depth value (z direction) for every point in the (x, y) plane. A robust similarity metric is defined for matching, based on an iterative closest point (ICP) registration process. Results are given for matching a database of 18 3D face models with 113 2.5D face scans.

Real time facial expression recognition with AdaBoost

Abstract: In this paper, we propose a novel method for facial expression recognition. The facial expression is extracted from human faces by an expression classifier that is learned from boosting Haar feature based look-up-table type weak classifiers. The expression recognition system consists of three modules, face detection, facial feature landmark extraction and facial expression recognition. The implemented system can automatically recognize seven expressions in real time that include anger, disgust, fear, happiness, neutral, sadness and surprise. Experimental results are reported to show its potential applications in human computer interaction.

Fusion of Visual and Thermal Signatures with Eyeglass Removal for Robust Face Recognition

Abstract: This paper describes a fusion of visual and thermal infrared (IR) images for robust face recognition. Two types of fusion methods are discussed: data fusion and decision fusion. Data fusion produces an illumination-invariant face image by adaptively integrating registered visual and thermal face images. Decision fusion combines matching scores of individual face recognition modules. In the data fusion process, eyeglasses, which block thermal energy, are detected from thermal images and replaced with an eye template. Three fusion-based face recognition techniques are implemented and tested: Data fusion of visual and thermal images (Df), Decision fusion with highest matching score (Fh), and Decision fusion with average matching score (Fa). A commercial face recognition software FaceIt® is used as an individual recognition module. Comparison results show that fusion-based face recognition techniques outperformed individual visual and thermal face recognizers under illumination variations and facial expressions.

Automatic 3D face recognition combining global geometric features with local shape variation information

Abstract: Face recognition is a focused issue in pattern recognition over the past decades. In this paper, we have proposed a new scheme for face recognition using 3D information. In this scheme, the scattered 3D point cloud is first represented with a regular mesh using hierarchical mesh fitting. Then the local shape variation information is extracted to characterize the individual together with the global geometric features. Experimental results on 3D/spl I.bar/RMA, a likely largest 3D face database available currently, demonstrate that the local shape variation information is very important to improve the recognition accuracy and that the proposed algorithm has promising performance with a low computational cost.

Infrared and visible image fusion for face recognition

Abstract: Considerable progress has been made in face recognition research over the last decade especially with the development of powerful models of face appearance (i.e., eigenfaces). Despite the variety of approaches and tools studied, however, face recognition is not accurate or robust enough to be deployed in uncontrolled environments. Recently, a number of studies have shown that infrared (IR) imagery offers a promising alternative to visible imagery due to its relative insensitive to illumination changes. However, IR has other limitations including that it is opaque to glass. As a result, IR imagery is very sensitive to facial occlusion caused by eyeglasses. In this paper, we propose fusing IR with visible images, exploiting the relatively lower sensitivity of visible imagery to occlusions caused by eyeglasses. Two different fusion schemes have been investigated in this study: (1) image-based fusion performed in the wavelet domain and, (2) feature-based fusion performed in the eigenspace domain. In both cases, we employ Genetic Algorithms (GAs) to find an optimum strategy to perform the fusion. To evaluate and compare the proposed fusion schemes, we have performed extensive recognition experiments using the Equinox face dataset and the popular method of eigenfaces. Our results show substantial improvements in recognition performance overall, suggesting that the idea of fusing IR with visible images for face recognition deserves further consideration.

From still image to video-based face recognition: an experimental analysis

Abstract: In this work, we analyze the effects of face sequence length and image quality on the performance of video-based face recognition systems which use a spatio-temporal representation instead of a still image-based one. We experiment with two different databases and consider the temporal hidden Markov model as a baseline method for the spatio-temporal representation and PCA and LDA for the image-based one. We show that the face sequence length affects the joint spatio-temporal representation more than the static-image-based methods. On the other hand, the experiments indicate that static image-based systems are more sensitive to image quality than their spatio-temporal representation-based counterpart. The second major contribution in this work is the use of an efficient method for extracting the representative frames (exemplars) from raw video. We build an appearance-based face recognition system which uses the probabilistic voting strategy to assess the efficiency of our approach.

Curse of mis-alignment in face recognition: problem and a novel mis-alignment learning solution

Abstract: In this paper, we present the rarely concerned curse of mis-alignment problem in face recognition, and propose a novel mis-alignment learning solution. Mis-alignment problem is firstly empirically investigated through systematically evaluating Fisherface's sensitivity to mis-alignment on the FERET face database by perturbing the eye coordinates, which reveals that the imprecise localization of the facial landmarks abruptly degenerates the Fisherface system. We explicitly define this problem as curse of mis-alignment to highlight its graveness. We then analyze the sources of curse of mis-alignment and group the possible solutions into three categories: invariant features, mis-alignment modeling, and alignment retuning. And then we propose a set of measurement combining the recognition rate with the alignment error distribution to evaluate the overall performance of specific face recognition approach with its robustness against the mis-alignment considered. Finally, a novel mis-alignment learning method, named E-Fisherface, is proposed to reinforce the recognizer to model the mis-alignment variations. Experimental results have impressively indicated the effectiveness of the proposed E-Fisherface in tackling the curse of mis-alignment problem.

Robust precise eye location under probabilistic framework

Abstract: Eye feature location is an important step in automatic visual interpretation and human face recognition. In this paper, a novel approach for locating eye centers in face areas under probabilistic framework is devised. After grossly locating a face, we first find the areas which left and right eyes lies in. Then an appearance-based eye detector is used to detect the possible left and right eye separately. According to their probabilities, the candidates are subsampled to merge those in near positions. Finally, the remaining left and right eye candidates are paired; each possible eye pair is normalized and verified. According to their probabilities, the precise eye positions are decided. The experimental results demonstrate that our method can effectively cope with different eye variations and achieve better location performance on diverse test sets than some newly proposed methods. And the influence of precision of eye location on face recognition is also probed. The location of other face organs such as mouth and nose can be incorporated in the framework easily.

A Survey Of 3D and Multi-Modal 3D+2D Face Recognition

Abstract: The vast majority of face recognition research has focused on the use of two-dimensional intensity images, and is covered in existing survey papers. This survey focuses on face recognition performed by matching two three-dimensional shape models, either alone or in combination with two-dimensional intensity images. Challenges involved in developing more accurate three-dimensional face recognition are identied. These include the need for improved sensors, recognition algorithms, and experimental methodology.

Infrared face detection and recognition system

Abstract: A face detection and recognition system having several arrays imaging a scene at different bands of the infrared spectrum. The system may use weighted subtracting and thresholding to distinguish human skin in a sensed image. A feature selector may locate a face in the image. The face may be framed or the image cropped with a frame or border to incorporate essentially only the face. The border may be superimposed on an image direct from an imaging array. A sub-image containing the face may be extracted from within the border and compared with a database of face information to attain recognition of the face. A level of recognition of the face may be established. Infrared lighting may be used as needed to illuminate the scene.

Eigenphases vs eigenfaces

Abstract: In this paper, we present a novel method for performing robust illumination-tolerant and partial face recognition that is based on modeling the phase spectrum of face images. We perform principal component analysis in the frequency domain on the phase spectrum of the face images and we show that this improves the recognition performance in the presence of illumination variations dramatically compared to normal eigenface method and other competing face recognition methods such as the illumination subspace method and fisherfaces. We show that this method is robustly even when presented with partial views of the test faces, without performing any pre-processing and without needing any a-priori knowledge of the type or part of face that is occluded or missing. We show comparative results using the illumination subset of CMU-PIE database consisting of 65 people showing the performance gain of our proposed method using a variety of training scenarios using as little as three training images per person. We also present partial face recognition results that obtained by synthetically blocking parts of the face of the test faces (even though training was performed on the full face images) showing gain in recognition accuracy of our proposed method.

Facial similarity across age, disguise, illumination and pose

Abstract: Illumination, pose variations, disguises, aging effects and expression variations are some of the key factors that affect the performance of face recognition systems Face recognition systems have always been studied from a recognition perspective Our emphasis is on deriving a measure of similarity between faces The similarity measure provides insights into the role each of the above mentioned variations play in affecting the performance of face recognition systems In the process of computing the similarity measure between faces, we suggest a framework to compensate for pose variations and introduce the notion of 'half-faces' to circumvent the problem of non-uniform illumination We used the similarity measure to retrieve similar faces from a database containing multiple images of individuals Moreover, we devised experiments to study the effect age plays in affecting facial similarity In conclusion, the similarity measure helps in studying the significance facial features play in affecting the performance of face recognition systems

3D shape-based face recognition using automatically registered facial surfaces

Abstract: We address the use of three-dimensional facial shape information for human face identification. We propose a new method to represent faces as 3D registered point clouds. Fine registration of facial surfaces is done by first automatically finding important facial landmarks and then, establishing a dense correspondence between points on the facial surface with the help of a 3D face template-aided thin plate spline algorithm. After the registration of facial surfaces, similarity between two faces is defined as a discrete approximation of the volume difference between facial surfaces. Experiments done on the 3D RMA dataset show that the proposed algorithm performs as good as the point signature method, and it is statistically superior to the point distribution model-based method and the 2D depth imagery technique. In terms of computational complexity, the proposed algorithm is faster than the point signature method.

Color channel encoding with NMF for face recognition

Abstract: Colors act as cues for perceiving objects, particularly in complex scenes. Intuitively, color seems to play an important role in recognizing people in scenes. Recent research has evinced that color cues contribute in recognizing faces, especially when shape cues of the images are degraded. Although the input to many of the face recognition systems is color images, during preprocessing, these images are converted to gray scale images for the feature extraction. In this study, we use non negative matrix factorization (NMF) to recognize color face images. By using NMF, we encode color channels (red, green, blue), thereby, projecting these feature vectors to sparse subspaces. The implemented system is tested on a subset of color images in the AR database for robustness against facial expressions and illumination variations. Furthermore, color face recognition results are compared with the results obtained for the gray scale images of the same data set. Our results show improved accuracy of color image recognition over gray level image recognition when large facial expressions and illumination variations are present.

An accurate active shape model for facial feature extraction

Abstract: The active shape model (ASM) has been used successfully to extract the facial features of a face image under frontal view. However, its performance degrades when the face concerned is under perspective variations. In this paper, a modified shape model is proposed to make the model represent a face more flexibly, under different orientations. The model of the eyes, nose and mouth, and the face contour are separated. An energy function is defined that links up these two representations of a human face. Three models are employed to represent the important facial features under different poses. A genetic algorithm (GA) is applied to search for the best representation of face images. Experiments show a better face representation under different perspective variations and facial expressions than the conventional ASM can.

Fusion of Infrared and Visible Images for Face Recognition

Abstract: A number of studies have demonstrated that infrared (IR) imagery offers a promising alternative to visible imagery due to it’s insensitive to variations in face appearance caused by illumination changes. IR, however, has other limitations including that it is opaque to glass. The emphasis in this study is on examining the sensitivity of IR imagery to facial occlusion caused by eyeglasses. Our experiments indicate that IR-based recognition performance degrades seriously when eyeglasses are present in the probe image but not in the gallery image and vice versa. To address this serious limitation of IR, we propose fusing the two modalities, exploiting the fact that visible-based recognition is less sensitive to the presence or absence of eyeglasses. Our fusion scheme is pixel-based, operates in the wavelet domain, and employs genetic algorithms (GAs) to decide how to combine IR with visible information. Although our fusion approach was not able to fully discount illumination effects present in the visible images, our experimental results show substantial improvements recognition performance overall, and it deserves further consideration.

An algorithm for real time eye detection in face images

Abstract: The problem of eye detection in face images is very important for a large number of applications ranging from face recognition to gaze tracking. In this paper, we propose a new algorithm for eyes detection that uses iris geometrical information for determining in the whole image the region candidate to contain an eye, and then the symmetry for selecting the couple of eyes. The novelty of this work is that the algorithm works on complex images without constraints on the background, skin color segmentation and so on. Different experiments, carried out on images of subjects with different eyes colors, some of them wearing glasses, demonstrate the effectiveness and robustness of the proposed algorithm.