Most face databases have been created under controlled conditions to facilitate the study of specific parameters on the face recognition problem. These parameters include such variables as position, pose, lighting, background, camera quality, and gender. While there are many applications for face recognition technology in which one can control the parameters of image acquisition, there are also many applications in which the practitioner has little or no control over such parameters. This database, Labeled Faces in the Wild, is provided as an aid in studying the latter, unconstrained, recognition problem. The database contains labeled face photographs spanning the range of conditions typically encountered in everyday life. The database exhibits “natural” variability in factors such as pose, lighting, race, accessories, occlusions, and background. In addition to describing the details of the database, we provide specific experimental paradigms for which the database is suitable. This is done in an effort to make research performed with the database as consistent and comparable as possible. We provide baseline results, including results of a state of the art face recognition system combined with a face alignment system. To facilitate experimentation on the database, we provide several parallel databases, including an aligned version.

/pdf/labeled-faces-in-the-wild-a-database-forstudying-face-2prti1m53k.pdf

Labeled Faces in the Wild: A Database forStudying Face Recognition in Unconstrained Environments

Making recognition more reliable under uncontrolled lighting conditions is one of the most important challenges for practical face recognition systems. We tackle this by combining the strengths of robust illumination normalization, local texture-based face representations, distance transform based matching, kernel-based feature extraction and multiple feature fusion. Specifically, we make three main contributions: 1) we present a simple and efficient preprocessing chain that eliminates most of the effects of changing illumination while still preserving the essential appearance details that are needed for recognition; 2) we introduce local ternary patterns (LTP), a generalization of the local binary pattern (LBP) local texture descriptor that is more discriminant and less sensitive to noise in uniform regions, and we show that replacing comparisons based on local spatial histograms with a distance transform based similarity metric further improves the performance of LBP/LTP based face recognition; and 3) we further improve robustness by adding Kernel principal component analysis (PCA) feature extraction and incorporating rich local appearance cues from two complementary sources-Gabor wavelets and LBP-showing that the combination is considerably more accurate than either feature set alone. The resulting method provides state-of-the-art performance on three data sets that are widely used for testing recognition under difficult illumination conditions: Extended Yale-B, CAS-PEAL-R1, and Face Recognition Grand Challenge version 2 experiment 4 (FRGC-204). For example, on the challenging FRGC-204 data set it halves the error rate relative to previously published methods, achieving a face verification rate of 88.1% at 0.1% false accept rate. Further experiments show that our preprocessing method outperforms several existing preprocessors for a range of feature sets, data sets and lighting conditions.

/pdf/enhanced-local-texture-feature-sets-for-face-recognition-4ixve2vsid.pdf

Enhanced Local Texture Feature Sets for Face Recognition Under Difficult Lighting Conditions

The capacity to estimate the head pose of another person is a common human ability that presents a unique challenge for computer vision systems. Compared to face detection and recognition, which have been the primary foci of face-related vision research, identity-invariant head pose estimation has fewer rigorously evaluated systems or generic solutions. In this paper, we discuss the inherent difficulties in head pose estimation and present an organized survey describing the evolution of the field. Our discussion focuses on the advantages and disadvantages of each approach and spans 90 of the most innovative and characteristic papers that have been published on this topic. We compare these systems by focusing on their ability to estimate coarse and fine head pose, highlighting approaches that are well suited for unconstrained environments.

Head Pose Estimation in Computer Vision: A Survey

For years, researchers in face recognition area have been representing and recognizing faces based on subspace discriminant analysis or statistical learning. Nevertheless, these approaches are always suffering from the generalizability problem. This paper proposes a novel non-statistics based face representation approach, local Gabor binary pattern histogram sequence (LGBPHS), in which training procedure is unnecessary to construct the face model, so that the generalizability problem is naturally avoided. In this approach, a face image is modeled as a "histogram sequence" by concatenating the histograms of all the local regions of all the local Gabor magnitude binary pattern maps. For recognition, histogram intersection is used to measure the similarity of different LGBPHSs and the nearest neighborhood is exploited for final classification. Additionally, we have further proposed to assign different weights for each histogram piece when measuring two LGBPHSes. Our experimental results on AR and FERET face database show the validity of the proposed approach especially for partially occluded face images, and more impressively, we have achieved the best result on FERET face database.

Local Gabor binary pattern histogram sequence (LGBPHS): a novel non-statistical model for face representation and recognition

Face alignment is a crucial step in face recognition tasks. Especially, using landmark localization for geometric face normalization has shown to be very effective, clearly improving the recognition results. However, no adequate databases exist that provide a sufficient number of annotated facial landmarks. The databases are either limited to frontal views, provide only a small number of annotated images or have been acquired under controlled conditions. Hence, we introduce a novel database overcoming these limitations: Annotated Facial Landmarks in the Wild (AFLW). AFLW provides a large-scale collection of images gathered from Flickr, exhibiting a large variety in face appearance (e.g., pose, expression, ethnicity, age, gender) as well as general imaging and environmental conditions. In total 25,993 faces in 21,997 real-world images are annotated with up to 21 landmarks per image. Due to the comprehensive set of annotations AFLW is well suited to train and test algorithms for multi-view face detection, facial landmark localization and face pose estimation. Further, we offer a rich set of tools that ease the integration of other face databases and associated annotations into our joint framework.

Annotated Facial Landmarks in the Wild: A large-scale, real-world database for facial landmark localization

In this paper, we describe the acquisition and contents of a large-scale Chinese face database: the CAS-PEAL face database. The goals of creating the CAS-PEAL face database include the following: 1) providing the worldwide researchers of face recognition with different sources of variations, particularly pose, expression, accessories, and lighting (PEAL), and exhaustive ground-truth information in one uniform database; 2) advancing the state-of-the-art face recognition technologies aiming at practical applications by using off-the-shelf imaging equipment and by designing normal face variations in the database; and 3) providing a large-scale face database of Mongolian. Currently, the CAS-PEAL face database contains 99 594 images of 1040 individuals (595 males and 445 females). A total of nine cameras are mounted horizontally on an arc arm to simultaneously capture images across different poses. Each subject is asked to look straight ahead, up, and down to obtain 27 images in three shots. Five facial expressions, six accessories, and 15 lighting changes are also included in the database. A selected subset of the database (CAS-PEAL-R1, containing 30 863 images of the 1040 subjects) is available to other researchers now. We discuss the evaluation protocol based on the CAS-PEAL-R1 database and present the performance of four algorithms as a baseline to do the following: 1) elementarily assess the difficulty of the database for face recognition algorithms; 2) preference evaluation results for researchers using the database; and 3) identify the strengths and weaknesses of the commonly used algorithms.

http://www.jdl.ac.cn/doc/2008/IEEE_SMC_A_gao_CAS-PEAL_Fianl.pdf

The CAS-PEAL Large-Scale Chinese Face Database and Baseline Evaluations

Evaluations of the state-of-the-art of both academic face recognition algorithms and commercial systems have shown that recognition performance of most current technologies degrades due to the variations of illumination. We investigate several illumination normalization methods and propose some novel solutions. The main contribution includes: (1) A gamma intensity correction (GIC) method is proposed to normalize the overall image intensity at the given illumination level; (2) A region-based strategy combining GIC and the histogram equalization (HE) is proposed to further eliminate the side-lighting effect; (3) A quotient illumination relighting (QIR) method is presented to synthesize images under a predefined normal lighting condition from the provided face images captured under nonnormal lighting condition. These methods are evaluated and compared on the Yale illumination face database B and Harvard illumination face database. Considerable improvements are observed. Some conclusions are given at last.

/pdf/illumination-normalization-for-robust-face-recognition-514zoqp9j3.pdf

Illumination normalization for robust face recognition against varying lighting conditions

In this paper, we present the rarely concerned curse of mis-alignment problem in face recognition, and propose a novel mis-alignment learning solution. Mis-alignment problem is firstly empirically investigated through systematically evaluating Fisherface's sensitivity to mis-alignment on the FERET face database by perturbing the eye coordinates, which reveals that the imprecise localization of the facial landmarks abruptly degenerates the Fisherface system. We explicitly define this problem as curse of mis-alignment to highlight its graveness. We then analyze the sources of curse of mis-alignment and group the possible solutions into three categories: invariant features, mis-alignment modeling, and alignment retuning. And then we propose a set of measurement combining the recognition rate with the alignment error distribution to evaluate the overall performance of specific face recognition approach with its robustness against the mis-alignment considered. Finally, a novel mis-alignment learning method, named E-Fisherface, is proposed to reinforce the recognizer to model the mis-alignment variations. Experimental results have impressively indicated the effectiveness of the proposed E-Fisherface in tackling the curse of mis-alignment problem.

http://www.jdl.ac.cn/tr/JDL-TR-03-FR-004.pdf

Curse of mis-alignment in face recognition: problem and a novel mis-alignment learning solution

The ICT-ISVISION Joint Research & Development Laboratory (JDL) for Face Recognition has constructed the CAS-PEAL face database supported by the National Hi-Tech Program and ISVISION Technologies Co., Ltd. The goals to create the PEAL face database include (1) providing the worldwide researchers of FR community a large-scale face database for training and evaluating their algorithms; (2) facilitating the development of FR by providing large-scale face images with different sources of variations, especially Pose, Expression, Accessories, and Lighting (PEAL); (3) advancing the state-of-the-art face recognition technologies aiming at practical applications especially for the oriental. Currently, the CAS-PEAL face database contains 99,594 images of 1040 individuals (595 males and 445 females) with varying Pose, Expression, Accessory, and Lighting (PEAL). For each subject, 9 cameras spaced equally in a horizontal semicircular shelf are setup to simultaneously capture images across different poses in one shot. Each subject is also asked to look up and down to capture 18 images in another two shots. We also considered 5 kinds of expressions, 6 kinds accessories (3 glasses, and 3 caps), and 15 lighting directions. This face database is now partly made available (a subset named by CAS-PEAL-R1, contain 30,900 images of 1040 subjects) for research purpose only on a case-by-case basis only. JDL is serving as the technical agent for distribution of the database and reserves the copyright of all the images in the database.

CAS-PEAL Large-Scale Chinese Face Database and Evaluation Protocols

Gabor feature has been widely recognized as better representation for face recognition in terms of rank-1 recognition rate. In this paper, we review the strength of Gabor feature for face recognition from the new angle of its robustness to mis-alignment using a novel quantificational evaluation method combining both the alignment precision and the recognition accuracy. Our experiments show that, compared with the gray-level intensity, Gabor feature is much more robust to image variation caused by the imprecision of facial feature localization, which further support the feasibility of Gabor representation.

Bo Cao

Papers

The CAS-PEAL Large-Scale Chinese Face Database and Baseline Evaluations

Illumination normalization for robust face recognition against varying lighting conditions

Curse of mis-alignment in face recognition: problem and a novel mis-alignment learning solution

CAS-PEAL Large-Scale Chinese Face Database and Evaluation Protocols

Review the strength of Gabor features for face recognition from the angle of its robustness to mis-alignment