Deep face recognition: A survey

A survey on deep learning based face recognition

Despite the large volume of face recognition datasets, there is a significant portion of subjects, of which the samples are insufficient and thus under-represented. Ignoring such significant portion results in insufficient training data. Training with under-represented data leads to biased classifiers in conventionally-trained deep networks. In this paper, we propose a center-based feature transfer framework to augment the feature space of under-represented subjects from the regular subjects that have sufficiently diverse samples. A Gaussian prior of the variance is assumed across all subjects and the variance from regular ones are transferred to the under-represented ones. This encourages the under-represented distribution to be closer to the regular distribution. Further, an alternating training regimen is proposed to simultaneously achieve less biased classifiers and a more discriminative feature representation. We conduct ablative study to mimic the under-represented datasets by varying the portion of under-represented classes on the MS-Celeb-1M dataset. Advantageous results on LFW, IJB-A and MS-Celeb-1M demonstrate the effectiveness of our feature transfer and training strategy, compared to both general baselines and state-of-the-art methods. Moreover, our feature transfer successfully presents smooth visual interpolation, which conducts disentanglement to preserve identity of a class while augmenting its feature space with non-identity variations such as pose and lighting.

/pdf/feature-transfer-learning-for-face-recognition-with-under-238pixqvou.pdf

Feature Transfer Learning for Face Recognition With Under-Represented Data

Deep metric learning papers from the past four years have consistently claimed great advances in accuracy, often more than doubling the performance of decade-old methods. In this paper, we take a closer look at the field to see if this is actually true. We find flaws in the experimental methodology of numerous metric learning papers, and show that the actual improvements over time have been marginal at best.

A Metric Learning Reality Check

Deep metric learning papers from the past four years have consistently claimed great advances in accuracy, often more than doubling the performance of decade-old methods. In this paper, we take a closer look at the field to see if this is actually true. We find flaws in the experimental methodology of numerous metric learning papers, and show that the actual improvements over time have been marginal at best. Code is available at github.com/KevinMusgrave/powerful-benchmarker.

In this paper we present Doppelganger mining - a method to learn better face representations. The main idea of this method is to maintain a list with the most similar identities for each identity in the training set. This list is used to generate better mini-batches by sampling pairs of similar-looking identities ("doppelgangers") together. It is especially useful for methods, based on exemplar-based supervision. Usually hard example mining comes with a price of necessity to use large mini-batches or substantial extra computation and memory cost, particularly for datasets with large numbers of identities. Our method needs only a negligible extra computation and memory. In our experiments on a benchmark dataset with 21,000 persons we show that Doppelganger mining, being inserted in the face representation learning process with joint prototype-based and exemplar-based supervision, significantly improves the discriminative power of learned face representations.

/pdf/doppelganger-mining-for-face-representation-learning-2uj5o670tc.pdf

Doppelganger Mining for Face Representation Learning

Hard example mining is an important part of the deep embedding learning. Most methods perform it at the mini-batch level. However, in the large-scale settings there is only a small chance that proper examples will appear in the same mini-batch and will be coupled into the hard example pairs or triplets. Doppelganger mining was previously proposed to increase this chance by means of class-wise similarity. This method ensures that examples of similar classes are sampled into the same mini-batch together. One of the drawbacks of this method is that it operates only at the class level, while there also might be a way to select appropriate examples within class in a more elaborated way than randomly. In this paper, we propose to use auxiliary embeddings for hard example mining. These embeddings are constructed in such way that similar examples have close embeddings in the cosine similarity sense. With the help of these embeddings it is possible to select new examples for the mini-batch based on their similarity with the already selected examples. We propose several ways to create auxiliary embeddings and use them to increase the number of potentially hard positive and negative examples in each mini-batch. Our experiments on the challenging Disguised Faces in the Wild (DFW) dataset show that hard example mining with auxiliary embeddings improves the discriminative power of learned representations.

/pdf/hard-example-mining-with-auxiliary-embeddings-112397r91r.pdf

Hard Example Mining with Auxiliary Embeddings

Although multi-modal (e.g. voice and face) biometric verification systems were in active development and showed impressive performance they need to be protected from spoofing attacks. In this paper we present methods for verifying face liveness based on estimation of synchrony between audio stream and lips movements track during the pronunciation of passphrase. The passphrase consists of a random set of the predetermined English words that are generated dynamically for each verification attempt. Lip movements extraction is performed by using of so-called Constrained Local Model of face shape. Audio stream is used to determine time intervals of pronounced words by means of automatic segmentation. Estimation of synchrony is done by analysis of lip movements for each word by employing a feedforward neural network and a Gaussian naive Bayes classifier. Finally, liveness score assessment is performed by averaging of individual word predictions during verification phrase utterance. For GRID corpus dataset average EER of 4.38% was achieved.

/pdf/audiovisual-liveness-detection-1n84ze6zq5.pdf

Audiovisual Liveness Detection

Multi-modal biometric verification systems are in active development and show impressive performance nowadays. However, such systems need additional protection from spoofing attacks. In our paper we present full pipeline of anti-spoofing method (based on our previous work) for bimodal audiovisual verification system. This method allows to evaluate parameters of quality for a sequence of face images during a verification process. Based on this parameters it’s decided whether the data is suitable for processing by the standard method (fiducial points based audiovisual liveness detection, FALD). If the quality of data is not sufficient, then our system switches to a new algorithm (svm-based audiovisual liveness detection, SALD), which provides less protection quality, but is able to operate when FALD is unsuitable. To improve the quality of the FALD algorithm we have collected the special dataset. This dataset allows to get better reliability of the algorithm for searching of fiducial points on the user’s face image. Tests show that developed system can significantly improve the quality of anti-spoofing protection versus our previous work.

Eugene Luckyanets

Papers

Doppelganger Mining for Face Representation Learning

Hard Example Mining with Auxiliary Embeddings

Audiovisual Liveness Detection

Bimodal Anti-Spoofing System for Mobile Security