Showing papers on "Three-dimensional face recognition published in 2003"
TL;DR: In this paper, the authors provide an up-to-date critical survey of still-and video-based face recognition research, and provide some insights into the studies of machine recognition of faces.
Abstract: As one of the most successful applications of image analysis and understanding, face recognition has recently received significant attention, especially during the past several years. At least two reasons account for this trend: the first is the wide range of commercial and law enforcement applications, and the second is the availability of feasible technologies after 30 years of research. Even though current machine recognition systems have reached a certain level of maturity, their success is limited by the conditions imposed by many real applications. For example, recognition of face images acquired in an outdoor environment with changes in illumination and/or pose remains a largely unsolved problem. In other words, current systems are still far away from the capability of the human perception system.This paper provides an up-to-date critical survey of still- and video-based face recognition research. There are two underlying motivations for us to write this survey paper: the first is to provide an up-to-date review of the existing literature, and the second is to offer some insights into the studies of machine recognition of faces. To provide a comprehensive survey, we not only categorize existing recognition techniques but also present detailed descriptions of representative methods within each category. In addition, relevant topics such as psychophysical studies, system evaluation, and issues of illumination and pose variation are covered.
6,384 citations
TL;DR: It is found that recognition performance is not significantly different between the face and the ear, for example, 70.5 percent versus 71.6 percent in one experiment and multimodal recognition using both the ear and face results in statistically significant improvement over either individual biometric.
Abstract: Researchers have suggested that the ear may have advantages over the face for biometric recognition. Our previous experiments with ear and face recognition, using the standard principal component analysis approach, showed lower recognition performance using ear images. We report results of similar experiments on larger data sets that are more rigorously controlled for relative quality of face and ear images. We find that recognition performance is not significantly different between the face and the ear, for example, 70.5 percent versus 71.6 percent, respectively, in one experiment. We also find that multimodal recognition using both the ear and face results in statistically significant improvement over either individual biometric, for example, 90.9 percent in the analogous experiment.
597 citations
01 Jun 2003
TL;DR: The proposed algorithm is the first facial expression recognition model with the property of subject independence combined with robustness to variability in facial appearance and demonstrated the ability to discriminate smiling from talking based on the saliency score obtained from voting visual cues.
Abstract: Reliable detection of ordinary facial expressions (e.g. smile) despite the variability among individuals as well as face appearance is an important step toward the realization of perceptual user interface with autonomous perception of persons. We describe a rule-based algorithm for robust facial expression recognition combined with robust face detection using a convolutional neural network. In this study, we address the problem of subject independence as well as translation, rotation, and scale invariance in the recognition of facial expression. The result shows reliable detection of smiles with recognition rate of 97.6% for 5600 still images of more than 10 subjects. The proposed algorithm demonstrated the ability to discriminate smiling from talking based on the saliency score obtained from voting visual cues. To the best of our knowledge, it is the first facial expression recognition model with the property of subject independence combined with robustness to variability in facial appearance.
575 citations
TL;DR: This work proposes a new image preprocessing algorithm that compensates for illumination variations in images from a single brightness image, which does not require any training steps, knowledge of 3D face models or reflective surface models, and demonstrates large performance improvements.
Abstract: Face recognition algorithms have to deal with significant amounts of illumination variations between gallery and probe images. State-of-the-art commercial face recognition algorithms still struggle with this problem. We propose a new image preprocessing algorithm that compensates for illumination variations in images. From a single brightness image the algorithm first estimates the illumination field and then compensates for it to mostly recover the scene reflectance. Unlike previously proposed approaches for illumination compensation, our algorithm does not require any training steps, knowledge of 3D face models or reflective surface models. We apply the algorithm to face images prior to recognition. We demonstrate large performance improvements with several standard face recognition algorithms across multiple, publicly available face databases.
369 citations
TL;DR: A 3D morphable model is used to compute 3D face models from three input images of each subject in the training database and the system achieved a recognition rate significantly better than a comparable global face recognition system.
Abstract: We present a novel approach to pose and illumination invariant face recognition that combines two recent advances in the computer vision field: component-based recognition and 3D morphable models. First, a 3D morphable model is used to generate 3D face models from three input images from each person in the training database. The 3D models are rendered under varying pose and illumination conditions to build a large set of synthetic images. These images are then used to train a component-based face recognition system. The resulting system achieved 90% accuracy on a database of 1200 real images of six people and significantly outperformed a comparable global face recognition system. The results show the potential of the combination of morphable models and component-based recognition towards pose and illumination invariant face recognition based on only three training images of each subject.
364 citations
TL;DR: This work proposes to transfer the super-resolution reconstruction from pixel domain to a lower dimensional face space, and shows that face-space super- Resolution is more robust to registration errors and noise than pixel-domain super- resolution because of the addition of model-based constraints.
Abstract: Face images that are captured by surveillance cameras usually have a very low resolution, which significantly limits the performance of face recognition systems. In the past, super-resolution techniques have been proposed to increase the resolution by combining information from multiple images. These techniques use super-resolution as a preprocessing step to obtain a high-resolution image that is later passed to a face recognition system. Considering that most state-of-the-art face recognition systems use an initial dimensionality reduction method, we propose to transfer the super-resolution reconstruction from pixel domain to a lower dimensional face space. Such an approach has the advantage of a significant decrease in the computational complexity of the super-resolution reconstruction. The reconstruction algorithm no longer tries to obtain a visually improved high-quality image, but instead constructs the information required by the recognition system directly in the low dimensional domain without any unnecessary overhead. In addition, we show that face-space super-resolution is more robust to registration errors and noise than pixel-domain super-resolution because of the addition of model-based constraints.
338 citations
05 Nov 2003
TL;DR: A real time approach to emotion recognition through facial expression in live video is presented, employing an automatic facial feature tracker to perform face localization and feature extraction and evaluating the method in terms of recognition accuracy.
Abstract: Enabling computer systems to recognize facial expressions and infer emotions from them in real time presents a challenging research topic. In this paper, we present a real time approach to emotion recognition through facial expression in live video. We employ an automatic facial feature tracker to perform face localization and feature extraction. The facial feature displacements in the video stream are used as input to a Support Vector Machine classifier. We evaluate our method in terms of recognition accuracy for a variety of interaction and classification scenarios. Our person-dependent and person-independent experiments demonstrate the effectiveness of a support vector machine and feature tracking approach to fully automatic, unobtrusive expression recognition in live video. We conclude by discussing the relevance of our work to affective and intelligent man-machine interfaces and exploring further improvements.
332 citations
TL;DR: A novel 3D face recognition approach based on geometric invariants introduced by Elad and Kimmel, which allows mapping 2D facial texture images into special images that incorporate the 3D geometry of the face.
Abstract: We present a novel 3D face recognition approach based on geometric invariants introduced by Elad and Kimmel. The key idea of the proposed algorithm is a representation of the facial surface, invariant to isometric deformations, such as those resulting from different expressions and postures of the face. The obtained geometric invariants allow mapping 2D facial texture images into special images that incorporate the 3D geometry of the face. These signature images are then decomposed into their principal components. The result is an efficient and accurate face recognition algorithm that is robust to facial expressions. We demonstrate the results of our method and compare it to existing 2D and 3D face recognition algorithms.
285 citations
[...]
TL;DR: Results suggest that significant gains in recognition accuracy may be achieved by focussing more effort on the eye localization stage of the face recognition process.
Abstract: This paper evaluates the impact of eye localization on face recognition accuracy. To investigate its importance, we present an eye perturbation sensitivity analysis, as well as empirical evidence that reinforces the notion that eye localization plays a key role in the accuracy of face recognition systems. In particular, correct measurement of eye separation is shown to be more important than correct eye location, highlighting the critical role of eye separation in the scaling and normalization of face images. Results suggest that significant gains in recognition accuracy may be achieved by focussing more effort on the eye localization stage of the face recognition process.
194 citations
06 Jul 2003
TL;DR: This work uses the sum rule and RBF-based integration strategies to combine three commonly used face classifiers based on PCA, ICA and LDA representations and shows that the proposed classifier combination approaches outperform individual classifiers.
Abstract: Current two-dimensional face recognition approaches can obtain a good performance only under constrained environments. However, in the real applications, face appearance changes significantly due to different illumination, pose, and expression. Face recognizers based on different representations of the input face images have different sensitivity to these variations. Therefore, a combination of different face classifiers which can integrate the complementary information should lead to improved classification accuracy. We use the sum rule and RBF-based integration strategies to combine three commonly used face classifiers based on PCA, ICA and LDA representations. Experiments conducted on a face database containing 206 subjects (2,060 face images) show that the proposed classifier combination approaches outperform individual classifiers.
154 citations
17 Oct 2003
TL;DR: A new fully automatic framework to analyze facial action units, the fundamental building blocks of facial expression enumerated in Paul Ekman's facial action coding system (FACS), achieves a higher recognition accuracy on the Cohn-Kanade AU-coded facial expression database, which has been previously used to evaluate other facial action recognition system.
Abstract: We provide a new fully automatic framework to analyze facial action units, the fundamental building blocks of facial expression enumerated in Paul Ekman's facial action coding system (FACS). The action units examined here include upper facial muscle movements such as inner eyebrow raise, eye widening, and so forth, which combine to form facial expressions. Although prior methods have obtained high recognition rates for recognizing facial action units, these methods either use manually preprocessed image sequences or require human specification of facial features; thus, they have exploited substantial human intervention. We present a fully automatic method, requiring no such human specification. The system first robustly detects the pupils using an infrared sensitive camera equipped with infrared LEDs. For each frame, the pupil positions are used to localize and normalize eye and eyebrow regions, which are analyzed using PCA to recover parameters that relate to the shape of the facial features. These parameters are used as input to classifiers based on support vector machines to recognize upper facial action units and all their possible combinations. On a completely natural dataset with lots of head movements, pose changes and occlusions, the new framework achieved a recognition accuracy of 69.3% for each individual AU and an accuracy of 62.5% for all possible AU combinations. This framework achieves a higher recognition accuracy on the Cohn-Kanade AU-coded facial expression database, which has been previously used to evaluate other facial action recognition system.
TL;DR: This work proposes a method that outperforms state-of-the-art face detection methods in environments with stable lighting and can potentially perform well invironments with variable lighting conditions.
17 Oct 2003
TL;DR: Experimental results indicate that when there is substantial passage of time (greater than one week) between the gallery and probe images, recognition from typical visible-light images may outperform that from infrared images.
Abstract: Techniques for face recognition generally fall into global and local approaches, with the principal component analysis (PCA) being the most prominent global approach. We use the PCA algorithm to study the comparison and combination of infrared and typical visible-light images for face recognition. We examine the effects of lighting change, facial expression change and passage of time between the gallery image and probe image. Experimental results indicate that when there is substantial passage of time (greater than one week) between the gallery and probe images, recognition from typical visible-light images may outperform that from infrared images. Experimental results also indicate that the combination of the two generally outperforms either one alone. This is the only study that we know of to focus on the issue of how passage of time affects infrared face recognition.
Patent•
15 Jan 2003
TL;DR: In this article, a system and method for iris recognition including stereoscopic face recognition, which can recognize irises including stereo face recognition system in order to recognize an authenticatee, is presented.
Abstract: Disclosed herein is a system and method for iris recognition including stereoscopic face recognition, which can recognize irises including stereoscopic face recognition system in order to recognize an authenticatee. The system includes two or more face recognition cameras for photographing two or more face images of an authenticatee; a recognition system for receiving the face images photographed by the face recognition cameras from the face recognition cameras and creating stereoscopic face information on the basis of the face images; and one or more iris recognition cameras controlled by the recognition system to photograph focused irises of the authenticatee using the created stereoscopic face information.
01 May 2003
TL;DR: This work has proven the proposed idea that facial expressions can be characterized and recognized by caricatures and is thus suitable for real-time applications.
Abstract: The automatic recognition of facial expression presents a significant challenge to the pattern analysis and man-machine interaction research community. Recognition from a single static image is particularly a difficult task. In this paper, we present a methodology for facial expression recognition from a single static image using line-based caricatures. The recognition process is completely automatic. It also addresses the computational expensive problem and is thus suitable for real-time applications. The proposed approach uses structural and geometrical features of a user sketched expression model to match the line edge map (LEM) descriptor of an input face image. A disparity measure that is robust to expression variations is defined. The effectiveness of the proposed technique has been evaluated and promising results are obtained. This work has proven the proposed idea that facial expressions can be characterized and recognized by caricatures.
TL;DR: Experimental results show that the novel, pose-invariant face recognition system based on a deformable, generic 3D face model is capable of determining pose and recognizing faces accurately over a wide range of poses and with naturally varying lighting conditions.
16 Jul 2003
TL;DR: By using a large facial image database called CMU PIE database, a probabilistic model of how facial features change as the pose changes is developed, which achieves a better recognition rate than conventional face recognition methods over a much larger range of pose.
Abstract: Current automatic facial recognition systems are not robust against changes in illumination, pose, facial expression and occlusion. In this paper, we propose an algorithm based on a probabilistic approach for face recognition to address the problem of pose change by a probabilistic approach that takes into account the pose difference between probe and gallery images. By using a large facial image database called CMU PIE database, which contains images of the same set of people taken from many different angles, we have developed a probabilistic model of how facial features change as the pose changes. This model enables us to make our face recognition system more robust to the change of poses in the probe image. The experimental results show that this approach achieves a better recognition rate than conventional face recognition methods over a much larger range of pose. For example, when the gallery contains only images of a frontal face and the probe image varies its pose orientation, the recognition rate remains within a less than 10% difference until the probe pose begins to differ more than 45 degrees, whereas the recognition rate of a PCA-based method begins to drop at a difference as small as 10 degrees, and a representative commercial system at 30 degrees.
17 Oct 2003
TL;DR: The largest experimental study to date that investigates the comparison and combination of 2D and 3D face data for biometric recognition finds a multimodal rank-one recognition rate of 92.8%, which is statistically significantly greater than either 2D or 3D alone.
Abstract: Results are presented for the largest experimental study to date that investigates the comparison and combination of 2D and 3D face data for biometric recognition. To our knowledge, this is also the only such study to incorporate significant time lapse between gallery and probe image acquisition. Recognition results are presented for gallery and probe datasets of 166 subjects imaged in both 2D and 3D, with six to thirteen weeks time lapse between gallery and probe images of a given subject. Using a PCA-based approach tuned separately for 2D and for 3D, we find no statistically significant difference between the rank-one recognition rates of 83.1% for 2D and 83.7% for 3D. Using a certainty-weighted sum-of-distance approach to combining 2D and 3D, we find a multimodal rank-one recognition rate of 92.8%, which is statistically significantly greater than either 2D or 3D alone.
18 Jun 2003
TL;DR: A method in which Gabor wavelet features are used for modeling local image structure, in which the ability of W-ASM to accurately align and locate facial features is demonstrated.
Abstract: Active shape model (ASM) is a powerful statistical tool for face alignment by shape. However, it can suffer from changes in illumination and facial expression changes, and local minima in optimization. In this paper, we present a method, W-ASM, in which Gabor wavelet features are used for modeling local image structure. The magnitude and phase of Gabor features contain rich information about the local structural features of face images to be aligned, and provide accurate guidance for search. To a large extent, this repairs defects in gray scale based search. An E-M algorithm is used to model the Gabor feature distribution, and a coarse-to-fine grained search is used to position local features in the image. Experimental results demonstrate the ability of W-ASM to accurately align and locate facial features.
17 Oct 2003
TL;DR: A complete and automatic system to perform face authentication by analysis of 3D facial shape is demonstrated, to be contrasted with traditional face recognition methods, which compare pictures of faces.
Abstract: We demonstrate a complete and automatic system to perform face authentication by analysis of 3D facial shape. In this live demonstration of the system, the subject is first enrolled, and given a unique identifier. Subsequently, the user's identity is verified by providing the reference identifier. Our approach is to be contrasted with traditional face recognition methods, which compare pictures of faces. We also analyzed the image quality requirements in order to generate a good quality 3D reconstruction from stereo.
TL;DR: A novel method for the recognition of human faces in digital images using a new feature extraction method that combines the global and local information in frontal view of facial images and a newly defined parameter named axis correction ratio (ACR) of images for disregarding irrelevant information of face images is introduced.
Abstract: This paper introduces a novel method for the recognition of human faces in digital images using a new feature extraction method that combines the global and local information in frontal view of facial images. Radial basis function (RBF) neural network with a hybrid learning algorithm (HLA) has been used as a classifier. The proposed feature extraction method includes human face localization derived from the shape information. An efficient distance measure as facial candidate threshold (FCT) is defined to distinguish between face and nonface images. Pseudo-Zernike moment invariant (PZMI) with an efficient method for selecting moment order has been used. A newly defined parameter named axis correction ratio (ACR) of images for disregarding irrelevant information of face images is introduced. In this paper, the effect of these parameters in disregarding irrelevant information in recognition rate improvement is studied. Also we evaluate the effect of orders of PZMI in recognition rate of the proposed technique as well as RBF neural network learning speed. Simulation results on the face database of Olivetti Research Laboratory (ORL) indicate that the proposed method for human face recognition yielded a recognition rate of 99.3%.
17 Oct 2003
TL;DR: The subject is a challenge response mechanism used as an optional add-on to the face recognition part of the multi modal biometric authentication system "BioID", which greatly enhances security in regard to replay attacks.
Abstract: The subject is a challenge response mechanism used as an optional add-on to the face recognition part of the multi modal biometric authentication system "BioID" This mechanism greatly enhances security in regard to replay attacks The user is required to look into a certain direction, which is randomly chosen by the system By estimating the head pose, the system verifies the user's response to the direction challenge The pose estimation is based on detection and subsequent tracking of suitable facial features Experimental evaluations have shown the effectiveness of the approach against replay attacks
17 Oct 2003
TL;DR: Back-projection and a boundary-weighted XOR-based cost function for binary silhouette matching, coupled with a probabilistic "downhill-simplex" optimization for shape estimation and refinement are used for 3D face acquisition.
Abstract: We present a method for 3D face acquisition using a set or sequence of 2D binary silhouettes. Since silhouette images depend only on the shape and pose of an object, they are immune to lighting and/or texture variations (unlike feature or texture-based shape-from-correspondence). Our prior 3D face model is a linear combination of "eigenheads" obtained by applying PCA to a training set of laser-scanned 3D faces. These shape coefficients are the parameters for a near-automatic system for capturing the 3D shape as well as the 2D texture-map of a novel input face. Specifically, we use back-projection and a boundary-weighted XOR-based cost function for binary silhouette matching, coupled with a probabilistic "downhill-simplex" optimization for shape estimation and refinement. Experiments with a multicamera rig as well as monocular video sequences demonstrate the advantages of our 3D modeling framework and ultimately, its utility for robust face recognition with built-in invariance to pose and illumination.
01 Jan 2003
TL;DR: This paper describes a novel algorithm for exact eye contour detection in frontal face image that does not rely on deformable models or image luminance gradient (edge) map.
Abstract: This paper describes a novel algorithm for exact eye contour detection in frontal face image. The exact eye shape is a useful piece of input information for applications like facial expression recognition, feature-based face recognition and face modelling. In contrast to well-known eye-segmentation methods, we do not rely on deformable models or image luminance gradient (edge) map. The eye windows (rough eye regions) are assumed to be known. The detection algorithm works in several steps. First, iris center and radius is estimated, then, exact upper eyelid contour is detected by searching for luminance valley points. Finally, lower eyelid is estimated from the eye corners coordinates and iris. The proposed technique has been tested on images of about fifty individuals taken under different lighting conditions with different cameras. It proved to be sufficiently robust and accurate for wide variety of images.
01 Jan 2003
TL;DR: In this paper, depth areas of a 3D face image were extracted by the contour line from some depth value and resampled and stored in consecutive location in feature vector using multiple feature method.
Abstract: Depth information is one of the most important factors for the recognition of a digital face image Range images are very useful, when comparing one face with another, because of implicating depth information As the processing for the whole face produces a lot of calculations and data, face images can be represented in terms of a vector of feature descriptors for a local area In this paper, depth areas of a 3 dimensional (3D) face image were extracted by the contour line from some depth value These were resampled and stored in consecutive location in feature vector using multiple feature method A comparison between two faces was made based on their distance in the feature space, using Euclidian distance This paper reduced the number of index data in the database and used fewer feature vectors than other methods Proposed algorithm can be highly recognized for using local depth information and less feature vectors on the face
01 Jan 2003
TL;DR: Research aimed at increasing the robustness of single- and multi-modal biometric identity verification systems is reported, which addresses the illumination and pose variation problems in face recognition, as well as the challenge of effectively fusing information from multiple modalities under non-ideal conditions.
Abstract: Interest in biometric based identification and verification systems has increased considerably over the last decade. As an example, the shortcomings of security systems based on passwords can be addressed through the supplemental use of biometric systems based on speech signals, face images or fingerprints. Biometric recognition can also be applied to other areas, such as passport control (immigration checkpoints), forensic work (to determine whether a biometric sample belongs to a suspect) and law enforcement applications (e.g. surveillance). While biometric systems based on face images and/or speech signals can be useful, their performance can degrade in the presence of challenging conditions. In face based systems this can be in the form of a change in the illumination direction and/or face pose variations. Multi-modal systems use more than one biometric at the same time. This is done for two main reasons -- to achieve better robustness and to increase discrimination power. This thesis reviews relevant backgrounds in speech and face processing, as well as information fusion. It reports research aimed at increasing the robustness of single- and multi-modal biometric identity verification systems. In particular, it addresses the illumination and pose variation problems in face recognition, as well as the challenge of effectively fusing information from multiple modalities under non-ideal conditions.
Journal Article•
TL;DR: The goal of this paper is to present a critical survey of existing lite- ratures on human face recognition over the last 4-5 years.
Abstract: The goal of this paper is to present a critical survey of existing lite- ratures on human face recognition over the last 4-5 years Interest and research activities in face recognition have increased significantly over the past few years, especially after the American airliner tragedy on September 11 in 2001 While this growth largely is driven by growing application demands, such as static matching of controlled photographs as in mug shots matching, credit card verification to surveillance video images, identification for law enforcement and authentication for banking and security system access, advances in signal analysis techniques, such as wavelets and neural networks, are also important catalysts As the number of proposed techniques increases, survey and evaluation becomes important
Proceedings Article•
01 Jan 2003
TL;DR: This work investigates the effect of image processing techniques when applied as a pre-processing step to three methods of face recognition: the direct correlation method, the eigenface method and fisherface method to identify some key advantages and determine the best image processing technique for each face recognition method.
Abstract: We investigate the effect of image processing techniques when applied as a pre-processing step to three methods of face recognition: the direct correlation method, the eigenface method and fisherface method. Effectiveness is evaluated by comparing false acceptance rates, false rejection rates and equal error rates calculated from over 250,000 verification operations on a large test set of facial images, which present typical difficulties when attempting recognition, such as strong variations in lighting conditions and changes in facial expression. We identify some key advantages and determine the best image processing technique for each face recognition method.
18 Jun 2003
TL;DR: This research demonstrates that the proposed method provides a new way for face representation, the shape trace transform (STT), which is verified with experiments on the XM2VTS database.
Abstract: We introduce a face representation, the shape trace transform (STT), for recognizing faces in an authentication system. The STT offers an alternative representation for faces that has a very high discriminatory power. We estimate the dissimilarity between two shapes by a new measure. We propose the Hausdorff context. Reinforcement learning is used to search the optimal parameters of the algorithm, for which the within class variance of the STT is minimized. This research demonstrates that the proposed method provides a new way for face representation. Our system is verified with experiments on the XM2VTS database.