Crafting A Panoptic Face Presentation Attack Detector
01 Jun 2019-pp 1-6
TL;DR: This paper designs a deep learning based panoptic algorithm for detection of both digital and physical presentation attacks using Cross Asymmetric Loss Function (CALF) and shows superior performance in three scenarios: ubiquitous environment, individual databases, and cross-attack/cross-database.
Abstract: With the advancements in technology and growing popularity of facial photo editing in the social media landscape, tools such as face swapping and face morphing have become increasingly accessible to the general public. It opens up the possibilities for different kinds of face presentation attacks, which can be taken advantage of by impostors to gain unauthorized access of a biometric system. Moreover, the wide availability of 3D printers has caused a shift from print attacks to 3D mask attacks. With increasing types of attacks, it is necessary to come up with a generic and ubiquitous algorithm with a panoptic view of these attacks, and can detect a spoofed image irrespective of the method used. The key contribution of this paper is designing a deep learning based panoptic algorithm for detection of both digital and physical presentation attacks using Cross Asymmetric Loss Function (CALF). The performance is evaluated for digital and physical attacks in three scenarios: ubiquitous environment, individual databases, and cross-attack/cross-database. Experimental results showcase the superior performance of the proposed presentation attack detection algorithm.
Citations
More filters
Posted Content•
TL;DR: A capsule network that can detect various kinds of attacks, from presentation attacks using printed images and replayed videos to attacks using fake videos created using deep learning, uses many fewer parameters than traditional convolutional neural networks with similar performance.
Abstract: The revolution in computer hardware, especially in graphics processing units and tensor processing units, has enabled significant advances in computer graphics and artificial intelligence algorithms. In addition to their many beneficial applications in daily life and business, computer-generated/manipulated images and videos can be used for malicious purposes that violate security systems, privacy, and social trust. The deepfake phenomenon and its variations enable a normal user to use his or her personal computer to easily create fake videos of anybody from a short real online video. Several countermeasures have been introduced to deal with attacks using such videos. However, most of them are targeted at certain domains and are ineffective when applied to other domains or new attacks. In this paper, we introduce a capsule network that can detect various kinds of attacks, from presentation attacks using printed images and replayed videos to attacks using fake videos created using deep learning. It uses many fewer parameters than traditional convolutional neural networks with similar performance. Moreover, we explain, for the first time ever in the literature, the theory behind the application of capsule networks to the forensics problem through detailed analysis and visualization.
109 citations
Cites methods from "Crafting A Panoptic Face Presentati..."
...Other methods have been developed that use the available CNN architectures with customized components and were trained on spoofing databases [40, 41, 15, 42, 43]....
[...]
TL;DR: A new framework for PAD is proposed using a one-class classifier, where the representation used is learned with a Multi-Channel Convolutional Neural Network (MCCNN) and a novel loss function is introduced, which forces the network to learn a compact embedding for bonafide class while being far from the representation of attacks.
Abstract: Face recognition has evolved as a widely used biometric modality. However, its vulnerability against presentation attacks poses a significant security threat. Though presentation attack detection (PAD) methods try to address this issue, they often fail in generalizing to unseen attacks. In this work, we propose a new framework for PAD using a one-class classifier, where the representation used is learned with a Multi-Channel Convolutional Neural Network ( MCCNN ). A novel loss function is introduced, which forces the network to learn a compact embedding for bonafide class while being far from the representation of attacks. A one-class Gaussian Mixture Model is used on top of these embeddings for the PAD task. The proposed framework introduces a novel approach to learn a robust PAD system from bonafide and available (known) attack classes. This is particularly important as collecting bonafide data and simpler attacks are much easier than collecting a wide variety of expensive attacks. The proposed system is evaluated on the publicly available WMCA multi-channel face PAD database, which contains a wide variety of 2D and 3D attacks. Further, we have performed experiments with MLFP and SiW-M datasets using RGB channels only. Superior performance in unseen attack protocols shows the effectiveness of the proposed approach. Software, data, and protocols to reproduce the results are made available publicly.
77 citations
Cites methods from "Crafting A Panoptic Face Presentati..."
...[27] trained an Alexnet model with a combination of cross-entropy and focal losses....
[...]
03 Apr 2020
TL;DR: Different ways in which the robustness of a face recognition algorithm is challenged, which can severely affect its intended working are summarized.
Abstract: Face recognition algorithms have demonstrated very high recognition performance, suggesting suitability for real world applications Despite the enhanced accuracies, robustness of these algorithms against attacks and bias has been challenged This paper summarizes different ways in which the robustness of a face recognition algorithm is challenged, which can severely affect its intended working Different types of attacks such as physical presentation attacks, disguise/makeup, digital adversarial attacks, and morphing/tampering using GANs have been discussed We also present a discussion on the effect of bias on face recognition models and showcase that factors such as age and gender variations affect the performance of modern algorithms The paper also presents the potential reasons for these challenges and some of the future research directions for increasing the robustness of face recognition models
53 citations
Cites background from "Crafting A Panoptic Face Presentati..."
...Given the vulnerabilities, it is our belief that future research should focus primarily on developing (i) robust PAD algorithms and (ii) universal detectors (Mehta et al. 2019) capable of handling multiple attacks....
[...]
TL;DR: This work proposes a novel anti-spoofing method, based on factorized bilinear coding of multiple color channels (namely MC\_FBC), that achieves the state-of-the-art performance on both the authors' own WFFD and other face spoofing databases under various intra-database and inter-database testing scenarios.
Abstract: We have witnessed rapid advances in both face presentation attack models and presentation attack detection (PAD) in recent years. When compared with widely studied 2D face presentation attacks, 3D face spoofing attacks are more challenging because face recognition systems are more easily confused by the 3D characteristics of materials similar to real faces. In this work, we tackle the problem of detecting these realistic 3D face presentation attacks and propose a novel anti-spoofing method from the perspective of fine-grained classification. Our method, based on factorized bilinear coding of multiple color channels (namely MC_FBC), targets at learning subtle fine-grained differences between real and fake images. By extracting discriminative and fusing complementary information from RGB and YCbCr spaces, we have developed a principled solution to 3D face spoofing detection. A large-scale wax figure face database (WFFD) with both images and videos has also been collected as super realistic attacks to facilitate the study of 3D face presentation attack detection. Extensive experimental results show that our proposed method achieves the state-of-the-art performance on both our own WFFD and other face spoofing databases under various intra-database and inter-database testing scenarios.
36 citations
Cites methods from "Crafting A Panoptic Face Presentati..."
...Existing methods tried to explore the difference between the real face skin and 3D fake face materials based on the reflectance properties using multispectral imaging [7], [8], texture analysis [9], [10], deep features [11], [12], or liveness cues [13], [14]....
[...]
TL;DR: Li et al. as mentioned in this paper proposed a two head contraction expansion convolutional neural network (CNN) architecture for robust presentation attack detection, which consists of raw image and edge enhanced image to learn discriminating features for binary classification.
10 citations
References
More filters
01 Sep 2016
TL;DR: The proposed algorithm extracts block-wise Haralick texture features from redundant discrete wavelet transformed frames obtained from a video and achieves state-of-the-art results for both frame-based and video- based approaches, including 100% accuracy on video-based spoofing detection.
Abstract: Face spoofing can be performed in a variety of ways such as replay attack, print attack, and mask attack to deceive an automated recognition algorithm. To mitigate the effect of spoofing attempts, face anti-spoofing approaches aim to distinguish between genuine samples and spoofed samples. The focus of this paper is to detect spoofing attempts via Haralick texture features. The proposed algorithm extracts block-wise Haralick texture features from redundant discrete wavelet transformed frames obtained from a video. Dimensionality of the feature vector is reduced using principal component analysis and two class classification is performed using support vector machine. Results on the 3DMAD database show that the proposed algorithm achieves state-of-the-art results for both frame-based and video-based approaches, including 100% accuracy on video-based spoofing detection. Further, the results are reported on existing benchmark databases on which the proposed feature extraction framework archives state-of-the-art performance.
106 citations
"Crafting A Panoptic Face Presentati..." refers methods in this paper
...If we compare with existing state-of-the-art algorithms [1] [7] on CASIA-FASD, the best reported results are 2....
[...]
...Majority of the existing anti-spoof methods involve extraction of discriminating features to analyze the face texture, such as Haralick texture features, local binary pattern (LBP), partial least square (PLS), and difference of Gaussian (DoG) [1, 2, 3, 20, 30]....
[...]
01 Jul 2017
TL;DR: A unique multispectral video face database for face presentation attack using latex and paper masks and it is observed that the thermal imaging spectrum is most effective in detecting face presentation attacks.
Abstract: Face recognition systems are susceptible to presentation attacks such as printed photo attacks, replay attacks, and 3D mask attacks. These attacks, primarily studied in visible spectrum, aim to obfuscate or impersonate a person's identity. This paper presents a unique multispectral video face database for face presentation attack using latex and paper masks. The proposed Multispectral Latex Mask based Video Face Presentation Attack (MLFP) database contains 1350 videos in visible, near infrared, and thermal spectrums. Since the database consists of videos of subjects without any mask as well as wearing ten different masks, the effect of identity concealment is analyzed in each spectrum using face recognition algorithms. We also present the performance of existing presentation attack detection algorithms on the proposed MLFP database. It is observed that the thermal imaging spectrum is most effective in detecting face presentation attacks.
104 citations
"Crafting A Panoptic Face Presentati..." refers methods in this paper
...Majority of the existing anti-spoof methods involve extraction of discriminating features to analyze the face texture, such as Haralick texture features, local binary pattern (LBP), partial least square (PLS), and difference of Gaussian (DoG) [1, 2, 3, 20, 30]....
[...]
01 Dec 2016
TL;DR: A novel multi-feature evidence aggregation method for face spoofing detection that fuses evidence from features encoding of both texture and motion properties in the face and also the surrounding scene regions and provides robustness to different attacks.
Abstract: Biometric systems can be attacked in several ways and the most common being spoofing the input sensor. Therefore, anti-spoofing is one of the most essential prerequisite against attacks on biometric systems. For face recognition it is even more vulnerable as the image capture is non-contact based. Several anti-spoofing methods have been proposed in the literature for both contact and non-contact based biometric modalities often using video to study the temporal characteristics of a real vs. spoofed biometric signal. This paper presents a novel multi-feature evidence aggregation method for face spoofing detection. The proposed method fuses evidence from features encoding of both texture and motion (liveness) properties in the face and also the surrounding scene regions. The feature extraction algorithms are based on a configuration of local binary pattern and motion estimation using histogram of oriented optical flow. Furthermore, the multi-feature windowed videolet aggregation of these orthogonal features coupled with support vector machine-based classification provides robustness to different attacks. We demonstrate the efficacy of the proposed approach by evaluating on three standard public databases: CASIA-FASD, 3DMAD and MSU-MFSD with equal error rate of 3.14%, 0%, and 0%, respectively.
98 citations
TL;DR: The proposed novel feature learning model can be used to adaptively weight the discriminability of the learned feature from different spatial regions or channels, which ensures that more discriminative deep dynamic textures play more important roles in face/mask classification.
Abstract: Three-dimensional mask spoofing attacks have been one of the main challenges in face recognition. Compared with a 3D mask, a real face displays different facial motion patterns that are reflected by different facial dynamic textures. However, a large portion of these facial motion differences is subtle. We find that the subtle facial motion can be fully captured by multiple deep dynamic textures from a convolutional layer of a convolutional neural network, but not all deep dynamic textures from different spatial regions and different channels of a convolutional layer are useful for differentiation of subtle motions between real faces and 3D masks. In this paper, we propose a novel feature learning model to learn discriminative deep dynamic textures for 3D mask face anti-spoofing. A novel joint discriminative learning strategy is further incorporated in the learning model to jointly learn the spatial- and channel-discriminability of the deep dynamic textures. The proposed joint discriminative learning strategy can be used to adaptively weight the discriminability of the learned feature from different spatial regions or channels, which ensures that more discriminative deep dynamic textures play more important roles in face/mask classification. Experiments on several publicly available data sets validate that the proposed method achieves promising results in intra- and cross-data set scenarios.
90 citations
"Crafting A Panoptic Face Presentati..." refers background in this paper
...On the other hand, existing state-of-the-art algorithms [17, 24] yield EER of 14....
[...]
TL;DR: This work examines a relatively new form of fraud, the use of photo-ID containing a graphical morph between two faces, and shows that a smartphone makes errors at a similar rate to ‘trained’ human viewers, accepting a small number of morphs as genuine ID.
Abstract: Matching unfamiliar faces is known to be difficult, and this can give an opportunity to those engaged in identity fraud. Here we examine a relatively new form of fraud, the use of photo-ID containing a graphical morph between two faces. Such a document may look sufficiently like two people to serve as ID for both. We present two experiments with human viewers, and a third with a smartphone face recognition system. In Experiment 1, viewers were asked to match pairs of faces, without being warned that one of the pair could be a morph. They very commonly accepted a morphed face as a match. However, in Experiment 2, following very short training on morph detection, their acceptance rate fell considerably. Nevertheless, there remained large individual differences in people's ability to detect a morph. In Experiment 3 we show that a smartphone makes errors at a similar rate to 'trained' human viewers-i.e. accepting a small number of morphs as genuine ID. We discuss these results in reference to the use of face photos for security.
89 citations
"Crafting A Panoptic Face Presentati..." refers background in this paper
...[22] performed an elaborate study, which accumulates responses of three experiments: human matching of two faces, human matching of two faces along with an option of fraudulent (morphed) image, and testing on an automated face recognition device....
[...]