Robust Face Recognition for Uncontrolled Pose and Illumination Changes
01 Jan 2013-Vol. 43, Iss: 1, pp 149-163
TL;DR: A novel framework for real-world face recognition in uncontrolled settings named Face Analysis for Commercial Entities (FACE), which adopts reliability indices, which estimate the “acceptability” of the final identification decision made by the classifier.
Abstract: Face recognition has made significant advances in the last decade, but robust commercial applications are still lacking. Current authentication/identification applications are limited to controlled settings, e.g., limited pose and illumination changes, with the user usually aware of being screened and collaborating in the process. Among others, pose and illumination changes are limited. To address challenges from looser restrictions, this paper proposes a novel framework for real-world face recognition in uncontrolled settings named Face Analysis for Commercial Entities (FACE). Its robustness comes from normalization (“correction”) strategies to address pose and illumination variations. In addition, two separate image quality indices quantitatively assess pose and illumination changes for each biometric query, before submitting it to the classifier. Samples with poor quality are possibly discarded or undergo a manual classification or, when possible, trigger a new capture. After such filter, template similarity for matching purposes is measured using a localized version of the image correlation index. Finally, FACE adopts reliability indices, which estimate the “acceptability” of the final identification decision made by the classifier. Experimental results show that the accuracy of FACE (in terms of recognition rate) compares favorably, and in some cases by significant margins, against popular face recognition methods. In particular, FACE is compared against SVM, incremental SVM, principal component analysis, incremental LDA, ICA, and hierarchical multiscale local binary pattern. Testing exploits data from different data sets: CelebrityDB, Labeled Faces in the Wild, SCface, and FERET. The face images used present variations in pose, expression, illumination, image quality, and resolution. Our experiments show the benefits of using image quality and reliability indices to enhance overall accuracy, on one side, and to provide for individualized processing of biometric probes for better decision-making purposes, on the other side. Both kinds of indices, owing to the way they are defined, can be easily integrated within different frameworks and off-the-shelf biometric applications for the following: 1) data fusion; 2) online identity management; and 3) interoperability. The results obtained by FACE witness a significant increase in accuracy when compared with the results produced by the other algorithms considered.
Citations
More filters
TL;DR: A fully automatic face normalization and recognition system robust to most common face variations in unconstrained environments and improves the performance of AAM fitting by initializing the AAM with estimates of the locations of the facial landmarks obtained by a method based on flexible mixture of parts.
Abstract: We present a fully automatic face normalization and recognition system.It normalizes the face images for both in-plane and out-of-plane pose variations.The performance of AAM fitting is improved using a novel initialization technique.HOG and Gabor features are fused using CCA to have more discriminative features.The proposed system recognizes non-frontal faces using only a single gallery sample. Single sample face recognition have become an important problem because of the limitations on the availability of gallery images. In many real-world applications such as passport or driver license identification, there is only a single facial image per subject available. The variations between the single gallery face image and the probe face images, captured in unconstrained environments, make the single sample face recognition even more difficult. In this paper, we present a fully automatic face recognition system robust to most common face variations in unconstrained environments. Our proposed system is capable of recognizing faces from non-frontal views and under different illumination conditions using only a single gallery sample for each subject. It normalizes the face images for both in-plane and out-of-plane pose variations using an enhanced technique based on active appearance models (AAMs). We improve the performance of AAM fitting, not only by training it with in-the-wild images and using a powerful optimization technique, but also by initializing the AAM with estimates of the locations of the facial landmarks obtained by a method based on flexible mixture of parts. The proposed initialization technique results in significant improvement of AAM fitting to non-frontal poses and makes the normalization process robust, fast and reliable. Owing to the proper alignment of the face images, made possible by this approach, we can use local feature descriptors, such as Histograms of Oriented Gradients (HOG), for matching. The use of HOG features makes the system robust against illumination variations. In order to improve the discriminating information content of the feature vectors, we also extract Gabor features from the normalized face images and fuse them with HOG features using Canonical Correlation Analysis (CCA). Experimental results performed on various databases outperform the state-of-the-art methods and show the effectiveness of our proposed method in normalization and recognition of face images obtained in unconstrained environments.
131 citations
Cites methods from "Robust Face Recognition for Uncontr..."
...Table 4 shows that our proposed system outperforms all he above-mentioned methods including the recent method proosed in (De Marsico et al., 2013) with an impressive margin of 6% in the recognition rate....
[...]
...…& Cipolla, 007), a method using Support Vector Machines (SVM) (Guo, Li, & han, 2000), a recent approach based on Hierarchical Multiscale LBP HMLBP) (Guo, Zhang, & Mou, 2010), and the method called “FACE” roposed in (De Marsico et al., 2013), which is the most recent ethod evaluated on this dataset....
[...]
...Although LFW is basically designed for metric learning for face verification, (De Marsico et al., 2013) evaluated some of the most popular face recognition algorithms as well as their own method on a subset of this database....
[...]
TL;DR: A novel adjacency coefficient representation is proposed, which does not only capture the category information between different samples, but also reflects the continuity between similar samples and the similarity betweenDifferent samples.
Abstract: This paper develops a new dimensionality reduction method, named biomimetic uncorrelated locality discriminant projection (BULDP), for face recognition. It is based on unsupervised discriminant projection and two human bionic characteristics: principle of homology continuity and principle of heterogeneous similarity. With these two human bionic characteristics, we propose a novel adjacency coefficient representation, which does not only capture the category information between different samples, but also reflects the continuity between similar samples and the similarity between different samples. By applying this new adjacency coefficient into the unsupervised discriminant projection, it can be shown that we can transform the original data space into an uncorrelated discriminant subspace. A detailed solution of the proposed BULDP is given based on singular value decomposition. Moreover, we also develop a nonlinear version of our BULDP using kernel functions for nonlinear dimensionality reduction. The performance of the proposed algorithms is evaluated and compared with the state-of-the-art methods on four public benchmarks for face recognition. Experimental results show that the proposed BULDP method and its nonlinear version achieve much competitive recognition performance.
100 citations
Cites methods from "Robust Face Recognition for Uncontr..."
...Researchers have proposed a variety of methods [1]–[5], in which the methods based on subspace analysis is an important branch....
[...]
TL;DR: The role of computer vision technology applied to the development of monitoring systems to detect distraction is reviewed and some key points considered as both future work and challenges ahead yet to be solved will also be addressed.
Abstract: Driver distraction, defined as the diversion of attention away from activities critical for safe driving toward a competing activity, is increasingly recognized as a significant source of injuries and fatalities on the roadway Additionally, the trend towards increasing the use of in-vehicle information systems is critical because they induce visual, biomechanical and cognitive distraction and may affect driving performance in qualitatively different ways Non-intrusive methods are strongly preferred for monitoring distraction, and vision-based systems have appeared to be attractive for both drivers and researchers Biomechanical, visual and cognitive distractions are the most commonly detected types in video-based algorithms Many distraction detection systems only use a single visual cue and therefore, they may be easily disturbed when occlusion or illumination changes appear Moreover, the combination of these visual cues is a key and challenging aspect in the development of robust distraction detection systems These visual cues can be extracted mainly by using face monitoring systems but they should be completed with more visual cues (eg, hands or body information) or even, distraction detection from specific actions (eg, phone usage) Additionally, these algorithms should be included in an embedded device or system inside a car This is not a trivial task and several requirements must be taken into account: reliability, real-time performance, low cost, small size, low power consumption, flexibility and short time-to-market The key points for the development and implementation of sensors to carry out the detection of distraction will also be reviewed This paper shows a review of the role of computer vision technology applied to the development of monitoring systems to detect distraction Some key points considered as both future work and challenges ahead yet to be solved will also be addressed
95 citations
Cites methods from "Robust Face Recognition for Uncontr..."
...Karuppusamy et al. [226] proposed an embedded implementation of facial landmarks detection based on both Viola-Jones face detector and facial landmarks detection using extended Active Shape Model (ASM) [227]....
[...]
...[226] proposed an embedded implementation of facial landmarks detection based on both Viola-Jones face detector and facial landmarks detection using extended Active Shape Model (ASM) [227]....
[...]
01 Feb 2016
TL;DR: The proposed fully touchless fingerprint recognition system adopts an innovative and less-constrained acquisition setup, does not require contact with any surface or a finger placement guide, and simultaneously captures multiple images while the finger is moving, and proposes novel algorithms for computing 3-D models of the shape of a finger.
Abstract: Touchless fingerprint recognition systems do not require contact of the finger with any acquisition surface and thus provide an increased level of hygiene, usability, and user acceptability of fingerprint-based biometric technologies. The most accurate touchless approaches compute 3-D models of the fingertip. However, a relevant drawback of these systems is that they usually require constrained and highly cooperative acquisition methods. We present a novel, fully touchless fingerprint recognition system based on the computation of 3-D models. It adopts an innovative and less-constrained acquisition setup compared with other previously reported 3-D systems, does not require contact with any surface or a finger placement guide, and simultaneously captures multiple images while the finger is moving. To compensate for possible differences in finger placement, we propose novel algorithms for computing 3-D models of the shape of a finger. Moreover, we present a new matching strategy based on the computation of multiple touch-compatible images. We evaluated different aspects of the biometric system: acceptability, usability, recognition performance, robustness to environmental conditions and finger misplacements, and compatibility and interoperability with touch-based technologies. The proposed system proved to be more acceptable and usable than touch-based techniques. Moreover, the system displayed satisfactory accuracy, achieving an equal error rate of 0.06% on a dataset of 2368 samples acquired in a single session and 0.22% on a dataset of 2368 samples acquired over the course of one year. The system was also robust to environmental conditions and to a wide range of finger rotations. The compatibility and interoperability with touch-based technologies was greater or comparable to those reported in public tests using commercial touchless devices.
67 citations
TL;DR: An overview of the state-of-the-art FR algorithms, focusing their performances on publicly available databases, and highlights the conditions of the image databases with regard to the recognition rate of each approach.
Abstract: Automatic Face Recognition (FR) presents a challenging task in the field of pattern recognition and despite the huge research in the past several decades; it still remains an open research problem. This is primarily due to the variability in the facial images, such as non-uniform illuminations, low resolution, occlusion, and/or variation in poses. Due to its non-intrusive nature, the FR is an attractive biometric modality and has gained a lot of attention in the biometric research community. Driven by the enormous number of potential application domains, many algorithms have been proposed for the FR. This paper presents an overview of the state-of-the-art FR algorithms, focusing their performances on publicly available databases. We highlight the conditions of the image databases with regard to the recognition rate of each approach. This is useful as a quick research overview and for practitioners as well to choose an algorithm for their specified FR application. To provide a comprehensive survey, the paper divides the FR algorithms into three categories: (1) intensity-based, (2) video-based, and (3) 3D based FR algorithms. In each category, the most commonly used algorithms and their performance is reported on standard face databases and a brief critical discussion is carried out.
64 citations
References
More filters
01 Jan 1998
TL;DR: Presenting a method for determining the necessary and sufficient conditions for consistency of learning process, the author covers function estimates from small data pools, applying these estimations to real-life problems, and much more.
Abstract: A comprehensive look at learning and generalization theory. The statistical theory of learning and generalization concerns the problem of choosing desired functions on the basis of empirical data. Highly applicable to a variety of computer science and robotics fields, this book offers lucid coverage of the theory as a whole. Presenting a method for determining the necessary and sufficient conditions for consistency of learning process, the author covers function estimates from small data pools, applying these estimations to real-life problems, and much more.
26,531 citations
01 Dec 2001
TL;DR: A machine learning approach for visual object detection which is capable of processing images extremely rapidly and achieving high detection rates and the introduction of a new image representation called the "integral image" which allows the features used by the detector to be computed very quickly.
Abstract: This paper describes a machine learning approach for visual object detection which is capable of processing images extremely rapidly and achieving high detection rates. This work is distinguished by three key contributions. The first is the introduction of a new image representation called the "integral image" which allows the features used by our detector to be computed very quickly. The second is a learning algorithm, based on AdaBoost, which selects a small number of critical visual features from a larger set and yields extremely efficient classifiers. The third contribution is a method for combining increasingly more complex classifiers in a "cascade" which allows background regions of the image to be quickly discarded while spending more computation on promising object-like regions. The cascade can be viewed as an object specific focus-of-attention mechanism which unlike previous approaches provides statistical guarantees that discarded regions are unlikely to contain the object of interest. In the domain of face detection the system yields detection rates comparable to the best previous systems. Used in real-time applications, the detector runs at 15 frames per second without resorting to image differencing or skin color detection.
18,620 citations
"Robust Face Recognition for Uncontr..." refers background in this paper
...It first submits the image to a global face detector (Viola-Jones [44] or Rowley [38]), which extracts all regions of interest (ROI) from an image that includes at least one face....
[...]
TL;DR: This work considers the problem of automatically recognizing human faces from frontal views with varying expression and illumination, as well as occlusion and disguise, and proposes a general classification algorithm for (image-based) object recognition based on a sparse representation computed by C1-minimization.
Abstract: We consider the problem of automatically recognizing human faces from frontal views with varying expression and illumination, as well as occlusion and disguise. We cast the recognition problem as one of classifying among multiple linear regression models and argue that new theory from sparse signal representation offers the key to addressing this problem. Based on a sparse representation computed by C1-minimization, we propose a general classification algorithm for (image-based) object recognition. This new framework provides new insights into two crucial issues in face recognition: feature extraction and robustness to occlusion. For feature extraction, we show that if sparsity in the recognition problem is properly harnessed, the choice of features is no longer critical. What is critical, however, is whether the number of features is sufficiently large and whether the sparse representation is correctly computed. Unconventional features such as downsampled images and random projections perform just as well as conventional features such as eigenfaces and Laplacianfaces, as long as the dimension of the feature space surpasses certain threshold, predicted by the theory of sparse representation. This framework can handle errors due to occlusion and corruption uniformly by exploiting the fact that these errors are often sparse with respect to the standard (pixel) basis. The theory of sparse representation helps predict how much occlusion the recognition algorithm can handle and how to choose the training images to maximize robustness to occlusion. We conduct extensive experiments on publicly available databases to verify the efficacy of the proposed algorithm and corroborate the above claims.
9,658 citations
"Robust Face Recognition for Uncontr..." refers result in this paper
...As an example, Sparse Representation-based Classification (SRC) [49] requires precise alignment, notwithstanding claims made by the authors, and recent results cast doubt about the usefulness of SRC for image classification [37]....
[...]
01 Oct 2008
TL;DR: The database contains labeled face photographs spanning the range of conditions typically encountered in everyday life, and exhibits “natural” variability in factors such as pose, lighting, race, accessories, occlusions, and background.
Abstract: Most face databases have been created under controlled conditions to facilitate the study of specific parameters on the face recognition problem. These parameters include such variables as position, pose, lighting, background, camera quality, and gender. While there are many applications for face recognition technology in which one can control the parameters of image acquisition, there are also many applications in which the practitioner has little or no control over such parameters. This database, Labeled Faces in the Wild, is provided as an aid in studying the latter, unconstrained, recognition problem. The database contains labeled face photographs spanning the range of conditions typically encountered in everyday life. The database exhibits “natural” variability in factors such as pose, lighting, race, accessories, occlusions, and background. In addition to describing the details of the database, we provide specific experimental paradigms for which the database is suitable. This is done in an effort to make research performed with the database as consistent and comparable as possible. We provide baseline results, including results of a state of the art face recognition system combined with a face alignment system. To facilitate experimentation on the database, we provide several parallel databases, including an aligned version.
5,742 citations
"Robust Face Recognition for Uncontr..." refers methods in this paper
...In summary, FACE testing has access to a diversity of databases that includes four different data sets: CDB [56], LFW [22], SCface [17], and FERET [34]....
[...]
...databases that includes four different data sets: CDB [56], LFW [22], SCface [17], and FERET [34]....
[...]
...The LFW database contains 13 233 target face images of 5749 different individuals; 1680 people have two or more image instances, while the remaining 4069 people have just a single image instance in the database....
[...]
...F. Integrating Image Quality and System Reliability Measures The results reported in Table II show that, among databases with an acceptable resolution (therefore excluding LFW), CDB (3 img) is the most problematic one in terms of performances....
[...]
...However, while HMLBP is much better on CDB than on LFW, ILDA is better on LFW than CDB....
[...]
TL;DR: This paper presents a novel and efficient facial image representation based on local binary pattern (LBP) texture features that is assessed in the face recognition problem under different challenges.
Abstract: This paper presents a novel and efficient facial image representation based on local binary pattern (LBP) texture features. The face image is divided into several regions from which the LBP feature distributions are extracted and concatenated into an enhanced feature vector to be used as a face descriptor. The performance of the proposed method is assessed in the face recognition problem under different challenges. Other applications and several extensions are also discussed
5,563 citations
"Robust Face Recognition for Uncontr..." refers methods in this paper
...[18] Z. Guo, L. Zhang, D. Zhang, and X. Mou, “Hierarchical multiscale LBP for face and palmprint recognition,” in Proc....
[...]
...FACE is compared against SVM, incremental SVM (ISVM), PCA, incremental LDA (ILDA), ICA, and hierarchical multiscale local binary pattern (HMLBP)....
[...]
...However, while HMLBP is much better on CDB than on LFW, ILDA is better on LFW than CDB....
[...]
...The RRs found using LFW are much lower for both FACE (local correlation module) and HMLBP, even though CDB and LWF seem to contain images with similar characteristics....
[...]
...Even in this case, performances of FACE and HMLBP on CDB and LFW are very different....
[...]