Recognizing human faces under disguise and makeup
01 Feb 2016-pp 1-7
TL;DR: A new database for face images under disguised and make-up appearances the development of face recognition algorithms under such covariates is presented and the experimental results suggest significant performance degradation in the capability of these matchers in automatically recognizing these faces.
Abstract: The accuracy of automated human face recognition algorithms can significantly degrade while recognizing same subjects under make-up and disguised appearances. Increasing constraints on enhanced security and surveillance requires enhanced accuracy from face recognition algorithms for faces under disguise and/or makeup. This paper presents a new database for face images under disguised and make-up appearances the development of face recognition algorithms under such covariates. This database has 2460 images from 410 different subjects and is acquired under real environment, focuses on make-up and disguises covariates and also provides ground truth (eye glass, goggle, mustache, beard) for every image. This can enable developed algorithms to automatically quantify their capability for identifying such important disguise attribute during the face recognition We also present comparative experimental results from two popular commercial matchers and from recent publications. Our experimental results suggest significant performance degradation in the capability of these matchers in automatically recognizing these faces. We also analyze face detection accuracy from these matchers. The experimental results underline the challenges in recognizing faces under these covariates. Availability of this new database in public domain will help to advance much needed research and development in recognizing make-up and disguised faces.
Citations
More filters
TL;DR: The history of face recognition technology, the current state-of-the-art methodologies, and future directions are presented, specifically on the most recent databases, 2D and 3D face recognition methods.
Abstract: Face recognition is one of the most active research fields of computer vision and pattern recognition, with many practical and commercial applications including identification, access control, forensics, and human-computer interactions. However, identifying a face in a crowd raises serious questions about individual freedoms and poses ethical issues. Significant methods, algorithms, approaches, and databases have been proposed over recent years to study constrained and unconstrained face recognition. 2D approaches reached some degree of maturity and reported very high rates of recognition. This performance is achieved in controlled environments where the acquisition parameters are controlled, such as lighting, angle of view, and distance between the camera–subject. However, if the ambient conditions (e.g., lighting) or the facial appearance (e.g., pose or facial expression) change, this performance will degrade dramatically. 3D approaches were proposed as an alternative solution to the problems mentioned above. The advantage of 3D data lies in its invariance to pose and lighting conditions, which has enhanced recognition systems efficiency. 3D data, however, is somewhat sensitive to changes in facial expressions. This review presents the history of face recognition technology, the current state-of-the-art methodologies, and future directions. We specifically concentrate on the most recent databases, 2D and 3D face recognition methods. Besides, we pay particular attention to deep learning approach as it presents the actuality in this field. Open issues are examined and potential directions for research in facial recognition are proposed in order to provide the reader with a point of reference for topics that deserve consideration.
155 citations
Cites background from "Recognizing human faces under disgu..."
...[37] created in 2016 the disguise covariate and/or make-up facial database with ground truth (goggle, beard, mustache, eye-glasses), acquired under real environments....
[...]
...4 CFP [35] 2016 7000 500 >14 DMFD [37] 2016 2460 410 6 IJB-B [40] 2017 21,798 1845 ≈36....
[...]
...Database Apparition’s Date Images Subjects Images/Subject ORL [23] 1994 400 40 10 FERET [13] 1996 14,126 1199 - AR [24] 1998 3016 116 26 XM2VTS [25] 1999 - 295 - BANCA [26] 2003 - 208 - FRGC [14] 2006 50,000 - 7 LFW [10] 2007 13,233 5749 ≈2.3 CMU Multi PIE [29] 2009 >750,000 337 N/A IJB-A [31] 2015 5712 500 ≈11.4 CFP [35] 2016 7000 500 >14 DMFD [37] 2016 2460 410 6 IJB-B [40] 2017 21,798 1845 ≈36.2 MF2 [41] 2017 4.7 M 672,057 ≈7 DFW [42] 2018 11,157 1000 ≈5.26 IJB-C [43) 2018 31,334 3531 ≈6 LFR [44] 2020 30,000 542 10–260 RMFRD [45] 2020 95,000 525 - SMFRD [45] 2020 500,000 10,000 - Figure 17....
[...]
...4 CFP [35] 2016 7000 500 >14 DMFD [37] 2016 2460 410 IJB-B [40] 2 7 21,798 1845 ≈36....
[...]
...Database Apparition’s Date Images Subjects Images/Subject L [23] 1994 400 40 10 FERET [13] 1996 14,126 1199 - AR [24] 1998 3016 116 26 XM2VTS [25] 1999 - 295 - BANCA [26] 2003 - 208 - FRG [14] 2 6 50,000 - 7 LFW [10] 2007 13,233 5749 ≈2.3 CMU Multi PIE [29] 2009 >750,000 337 N/A IJB-A [31] 2015 5712 500 ≈11.4 CFP [35] 2016 7000 500 >14 DMFD [37] 2016 2460 410 IJB-B [40] 2 7 21,798 1845 ≈36.2 MF2 [41] 2017 4.7 M 672,057 ≈7 DFW [42] 2018 11,157 1000 ≈5.26 IJB-C [43] 2018 31,334 3531 ≈6 FR [44] 2020 30,000 542 10–260 RMFRD [45] 20 9 , 00 525 - SMFRD [45] 2 20 500,000 10,000 - Table 2....
[...]
18 Jun 2018
TL;DR: A novel Disguised Faces in the Wild (DFW) dataset, consisting of over 11,000 images for understanding and pushing the current state-of-the-art for disguised face recognition, along with the phase-I results of the CVPR2018 competition.
Abstract: Existing research in the field of face recognition with variations due to disguises focuses primarily on images captured in controlled settings. Limited research has been performed on images captured in unconstrained environments, primarily due to the lack of corresponding disguised face datasets. In order to overcome this limitation, this work presents a novel Disguised Faces in the Wild (DFW) dataset, consisting of over 11,000 images for understanding and pushing the current state-of-the-art for disguised face recognition. To the best of our knowledge, DFW is a first-of-a-kind dataset containing images pertaining to both obfuscation and impersonation for understanding the effect of disguise variations. A major portion of the dataset has been collected from the Internet, thereby encompassing a wide variety of disguise accessories and variations across other covariates. As part of CVPR2018, a competition and workshop are organized to facilitate research in this direction. This paper presents a description of the dataset, the baseline protocols and performance, along with the phase-I results of the competition.
76 citations
Cites methods from "Recognizing human faces under disgu..."
...AR Dataset [12] 1998 Yes 3,200 126 Yes National Geographic Dataset [15] 2004 Yes 46 1 No Synthetic Disguise Dataset [19] 2009 Yes 4,000 100 No IIIT-Delhi Disguise V1 Dataset [5] 2014 Yes 684 75 Yes Disguised and Makeup Faces Dataset [22] 2016 No 2,460 410 Yes...
[...]
01 Dec 2019
TL;DR: The primary concern to this work is about facial masks, and especially to enhance the recognition accuracy of different masked faces, and a feasible approach has been proposed that consists of first detecting the facial regions.
Abstract: Recognition from faces is a popular and significant technology in recent years. Face alterations and the presence of different masks make it too much challenging. In the real-world, when a person is uncooperative with the systems such as in video surveillance then masking is further common scenarios. For these masks, current face recognition performance degrades. An abundant number of researches work has been performed for recognizing faces under different conditions like changing pose or illumination, degraded images, etc. Still, difficulties created by masks are usually disregarded. The primary concern to this work is about facial masks, and especially to enhance the recognition accuracy of different masked faces. A feasible approach has been proposed that consists of first detecting the facial regions. The occluded face detection problem has been approached using Multi-Task Cascaded Convolutional Neural Network (MTCNN). Then facial features extraction is performed using the Google FaceNet embedding model. And finally, the classification task has been performed by Support Vector Machine (SVM). Experiments signify that this mentioned approach gives a remarkable performance on masked face recognition. Besides, its performance has been also evaluated within excessive facial masks and found attractive outcomes. Finally, a correlative study also made here for a better understanding.
64 citations
TL;DR: A conceptual categorisation of beautification is presented, relevant scenarios with respect to face recognition are discussed, and related publications are revisited, and technical considerations and trade-offs of the surveyed methods are summarized.
Abstract: Facial beautification induced by plastic surgery, cosmetics or retouching has the ability to substantially alter the appearance of face images. Such types of beautification can negatively affect the accuracy of face recognition systems. In this work, a conceptual categorisation of beautification is presented, relevant scenarios with respect to face recognition are discussed, and related publications are revisited. Additionally, technical considerations and trade-offs of the surveyed methods are summarized along with open issues and challenges in the field. This survey is targeted to provide a comprehensive point of reference for biometric researchers and practitioners working in the field of face recognition, who aim at tackling challenges caused by facial beautification.
55 citations
Cites background or result from "Recognizing human faces under disgu..."
...[50] T. Y. Wang and A. Kumar, ‘‘Recognizing human faces under disguise and makeup,’’ in Proc....
[...]
...Similar studies, confirming the above findings were conducted by Wang and Kumar [50] and Eckert et al. [41]....
[...]
...4 Another makeup database has been made publicly available5 in [50]....
[...]
...Similar studies, confirming the above findings were conducted by Wang and Kumar [50] and Eckert et al....
[...]
03 Apr 2020
TL;DR: Different ways in which the robustness of a face recognition algorithm is challenged, which can severely affect its intended working are summarized.
Abstract: Face recognition algorithms have demonstrated very high recognition performance, suggesting suitability for real world applications Despite the enhanced accuracies, robustness of these algorithms against attacks and bias has been challenged This paper summarizes different ways in which the robustness of a face recognition algorithm is challenged, which can severely affect its intended working Different types of attacks such as physical presentation attacks, disguise/makeup, digital adversarial attacks, and morphing/tampering using GANs have been discussed We also present a discussion on the effect of bias on face recognition models and showcase that factors such as age and gender variations affect the performance of modern algorithms The paper also presents the potential reasons for these challenges and some of the future research directions for increasing the robustness of face recognition models
53 citations
Cites background from "Recognizing human faces under disgu..."
...ormed person recognition using the nondisguised facial regions only. Up till 2016, most of the research on disguised face recognition involved face images captured in relatively constrained settings. Wang and Kumar (2016) presented a Disguised and Makeup Faces Dataset containing 2460 face images of 410 subjects. The dataset contains images collected from the Internet with variations across different disguise accessori...
[...]
...Wang and Kumar (2016) presented a Disguised and Makeup Faces Dataset containing 2460 face images of 410 subjects....
[...]
References
More filters
06 Dec 2012
TL;DR: The impact of a commonly used face altering technique that has received limited attention in the biometric literature, viz., non-permanent facial makeup is studied and it is suggested that this simple alteration can indeed compromise the accuracy of a biometric system.
Abstract: The matching performance of automated face recognition has significantly improved over the past decade. At the same time several challenges remain that significantly affect the deployment of such systems in security applications. In this work, we study the impact of a commonly used face altering technique that has received limited attention in the biometric literature, viz., non-permanent facial makeup. Towards understanding its impact, we first assemble two databases containing face images of subjects, before and after applying makeup. We present experimental results on both databases that reveal the effect of makeup on automated face recognition and suggest that this simple alteration can indeed compromise the accuracy of a bio-metric system. While these are early results, our findings clearly indicate the need for a better understanding of this face altering scheme and the importance of designing algorithms that can successfully overcome the obstacle imposed by the application of facial makeup.
162 citations
"Recognizing human faces under disgu..." refers background in this paper
...Increasing constraints on enhanced security and surveillance requires enhanced accuracy from face recognition algorithms for faces under disguise and/or makeup....
[...]
TL;DR: Experimental results based on the Southampton multibiometric tunnel database show that the use of soft biometric traits is able to improve the performance of face recognition based on sparse representation on real and ideal scenarios by adaptive fusion rules.
Abstract: Soft biometric information extracted from a human body (e.g., height, gender, skin color, hair color, and so on) is ancillary information easily distinguished at a distance but it is not fully distinctive by itself in recognition tasks. However, this soft information can be explicitly fused with biometric recognition systems to improve the overall recognition when confronting high variability conditions. One significant example is visual surveillance, where face images are usually captured in poor quality conditions with high variability and automatic face recognition systems do not work properly. In this scenario, the soft biometric information can provide very valuable information for person recognition. This paper presents an experimental study of the benefits of soft biometric labels as ancillary information based on the description of human physical features to improve challenging person recognition scenarios at a distance. In addition, we analyze the available soft biometric information in scenarios of varying distance between camera and subject. Experimental results based on the Southampton multibiometric tunnel database show that the use of soft biometric traits is able to improve the performance of face recognition based on sparse representation on real and ideal scenarios by adaptive fusion rules.
144 citations
04 Jun 2013
TL;DR: A method to automatically detect the presence of makeup in face images by extracting a feature vector that captures the shape, texture and color characteristics of the input face, and employs a classifier to determine the presence or absence of makeup.
Abstract: Facial makeup has the ability to alter the appearance of a person. Such an alteration can degrade the accuracy of automated face recognition systems, as well as that of meth-ods estimating age and beauty from faces. In this work, we design a method to automatically detect the presence of makeup in face images. The proposed algorithm extracts a feature vector that captures the shape, texture and color characteristics of the input face, and employs a classifier to determine the presence or absence of makeup. Besides extracting features from the entire face, the algorithm also considers portions of the face pertaining to the left eye, right eye, and mouth. Experiments on two datasets consisting of 151 subjects (600 images) and 125 subjects (154 images), respectively, suggest that makeup detection rates of up to 93.5% (at a false positive rate of 1%) can be obtained using the proposed approach. Further, an adaptive pre-processing scheme that exploits knowledge of the presence or absence of facial makeup to improve the matching accuracy of a face matcher is presented.
129 citations
TL;DR: An automated algorithm is developed to verify the faces presented under disguise variations using automatically localized feature descriptors which can identify disguised face patches and account for this information to achieve improved matching accuracy.
Abstract: Face verification, though an easy task for humans, is a long-standing open research area. This is largely due to the challenging covariates, such as disguise and aging, which make it very hard to accurately verify the identity of a person. This paper investigates human and machine performance for recognizing/verifying disguised faces. Performance is also evaluated under familiarity and match/mismatch with the ethnicity of observers. The findings of this study are used to develop an automated algorithm to verify the faces presented under disguise variations. We use automatically localized feature descriptors which can identify disguised face patches and account for this information to achieve improved matching accuracy. The performance of the proposed algorithm is evaluated on the IIIT-Delhi Disguise database that contains images pertaining to 75 subjects with different kinds of disguise variations. The experiments suggest that the proposed algorithm can outperform a popular commercial system and evaluates them against humans in matching disguised face images.
110 citations
04 Jun 2013
TL;DR: A framework, termed as Aravrta1, is proposed, which classifies the local facial regions of both visible and thermal face images into biometric (regions without disguise) and non-biometric (Regions with disguise) classes, and improves the performance compared to existing algorithms.
Abstract: Face verification, though for humans seems to be an easy task, is a long-standing research area. With challenging covariates such as disguise or face obfuscation, automatically verifying the identity of a person is assumed to be very hard. This paper explores the feasibility of face verification under disguise variations using multi-spectrum (visible and thermal) face images. We propose a framework, termed as Aravrta1, which classifies the local facial regions of both visible and thermal face images into biometric (regions without disguise) and non-biometric (regions with disguise) classes. The biometric patches are then used for facial feature extraction and matching. The performance of the algorithm is evaluated on the IHTD In and Beyond Visible Spectrum Disguise database that is prepared by the authors and contains images pertaining to 75 subjects with different kinds of disguise variations. The experimental results suggest that the proposed framework improves the performance compared to existing algorithms, however there is a need for more research to address this important covariate.
108 citations