scispace - formally typeset
Search or ask a question
Proceedings ArticleDOI

Face anti-spoofing with multifeature videolet aggregation

TL;DR: A novel multi-feature evidence aggregation method for face spoofing detection that fuses evidence from features encoding of both texture and motion properties in the face and also the surrounding scene regions and provides robustness to different attacks.
Abstract: Biometric systems can be attacked in several ways and the most common being spoofing the input sensor. Therefore, anti-spoofing is one of the most essential prerequisite against attacks on biometric systems. For face recognition it is even more vulnerable as the image capture is non-contact based. Several anti-spoofing methods have been proposed in the literature for both contact and non-contact based biometric modalities often using video to study the temporal characteristics of a real vs. spoofed biometric signal. This paper presents a novel multi-feature evidence aggregation method for face spoofing detection. The proposed method fuses evidence from features encoding of both texture and motion (liveness) properties in the face and also the surrounding scene regions. The feature extraction algorithms are based on a configuration of local binary pattern and motion estimation using histogram of oriented optical flow. Furthermore, the multi-feature windowed videolet aggregation of these orthogonal features coupled with support vector machine-based classification provides robustness to different attacks. We demonstrate the efficacy of the proposed approach by evaluating on three standard public databases: CASIA-FASD, 3DMAD and MSU-MFSD with equal error rate of 3.14%, 0%, and 0%, respectively.
Citations
More filters
Proceedings ArticleDOI
08 Jul 2019
TL;DR: A face anti-spoofing detection algorithm based on domain adaptation is proposed that outperforms state-of-the-art approaches and applies Maximum Mean Discrepancy to multi-layer network distribution adaptation, which improves the generalization ability of the model.
Abstract: With the popularity of face recognition technology, people have put forward higher requirements for the security of face recognition system. Face anti-spoofing detection attracts extensive attention and many methods been proposed. However, these methods perform poorly in cross scenes. To solve this problem, we propose a face anti-spoofing detection algorithm based on domain adaptation. We apply Maximum Mean Discrepancy (MMD) to multi-layer network distribution adaptation, which improves the generalization ability of the model. To further improve the performance of face anti-spoofing detection, we fuse the low-level features with the high-level features of convolutional neural network for face anti-spoofing detection. Two widely used datasets are used to test the proposed method. The experimental results show that the proposed algorithm outperforms state-of-the-art approaches.

12 citations


Cites methods from "Face anti-spoofing with multifeatur..."

  • ...Table 1 shows the EER and HTER of advanced face antispoofing methods: the LBP+HOOF based mothod [20], the IDA based method [3], the color analysis based methods [5, 2] and the dynamic texture based mothd [13]....

    [...]

  • ...Method EER HTER EER LBP+HOOF [20] - - 3....

    [...]

Book ChapterDOI
01 Jan 2019
TL;DR: This chapter introduces a novel two-stream CNN-based approach for the presentation attack detection, by extracting the patch-based features and holistic depth maps from the face images.
Abstract: Face is the most accessible biometric modality which can be used for identity verification in mobile phone applications, and it is vulnerable to many different presentation attacks, such as using a printed face/digital screen face to access the mobile phone. Presentation attack detection is a very critical step before feeding the face image to face recognition systems. In this chapter, we introduce a novel two-stream CNN-based approach for the presentation attack detection, by extracting the patch-based features and holistic depth maps from the face images. We also introduce a two-stream CNN v2 with model optimization, compression and a strategy of continuous updating. The CNN v2 shows great performances of both generalization and efficiency. Extensive experiments are conducted on the challenging databases (CASIA-FASD, MSU-USSA, replay attack, OULU-NPU, and SiW), with comparison to the state of the art.

12 citations

Journal ArticleDOI
TL;DR: Wang et al. as discussed by the authors proposed to use a Generative Adversarial Network (GAN) to transfer an input face image from the RGB domain to the depth domain, which enables biometric preservation against challenging appearance variations and diverse image qualities.
Abstract: With the rapid development in face recognition, most of the existing systems can perform very well in unconstrained scenarios. However, it is still a very challenging task to detect face spoofing attacks, thus face anti-spoofing has become one of the most important research topics in the community. Though various anti-spoofing models have been proposed, the generalisation capability of these models usually degrades for unseen attacks in the presence of challenging appearance variations, e.g. , background, illumination, diverse spoofing materials and low image quality. To address this issue, we propose to use a Generative Adversarial Network (GAN) that transfers an input face image from the RGB domain to the depth domain. The generated depth clue enables biometric preservation against challenging appearance variations and diverse image qualities. To be more specific, the proposed method has two main stages. The first one is a GAN-based domain transfer module that converts an input image to its corresponding depth map. By design, a live face image should be transferred to a depth map whereas a spoofing face image should be transferred to a plain (black) image. The aim is to improve the discriminative capability of the proposed system. The second stage is a classification model that determines whether an input face image is live or spoofing. Benefit from the use of the GAN-based domain transfer module, the latent variables can effectively represent the depth information, complementarily enhancing the discrimination of the original RGB features. The experimental results obtained on several benchmarking datasets demonstrate the effectiveness of the proposed method, with superior performance over the state-of-the-art methods. The source code of the proposed method is publicly available at https://github.com/coderwangson/DFA .

12 citations

Book ChapterDOI
01 Jan 2019
TL;DR: This chapter presents the design and results of the three international competitions assessed in software-based face anti-spoofing, and evaluates the generalization abilities of the proposed algorithms under some real-world variations faced in mobile scenarios, including previously unseen acquisition conditions, presentation attack instruments, and sensors.
Abstract: Face presentation attack detection has received increasing attention ever since the vulnerabilities to spoofing have been widely recognized. The state of the art in software-based face anti-spoofing has been assessed in three international competitions organized in conjunction with major biometrics conferences in 2011, 2013, and 2017, each introducing new challenges to the research community. In this chapter, we present the design and results of the three competitions. The particular focus is on the latest competition, where the aim was to evaluate the generalization abilities of the proposed algorithms under some real-world variations faced in mobile scenarios, including previously unseen acquisition conditions, presentation attack instruments, and sensors. We also discuss the lessons learnt from the competitions and future challenges in the field in general.

11 citations


Cites background from "Face anti-spoofing with multifeatur..."

  • ...In general, passive methods are based on analyzing different facial properties, such as frequency content [28, 46], texture [2, 12, 17, 27, 32, 53] and quality [18, 21, 23], or motion cues, such as eye blinking [3, 38, 45, 47], facial expression changes [3, 25, 45, 47], mouth movements [3, 25, 45, 47], or even color variation due to blood circulation (pulse) [15, 29, 31], to discriminate face artifacts from genuine ones....

    [...]

Proceedings ArticleDOI
01 Oct 2018
TL;DR: This work proposes a novel face anti spoofing method based on Multi Color Convolutional Neural Network (CNN) architecture named DeepColorFASD, which investigates the effect of space colors on CNN architectures and proposes a fusion based voting method for faceAnti spoofing.
Abstract: Despite a great deal of progress in face recognition technologies, current solutions are still vulnerable to spoof attacks. In fact, it is easy to access digital replicas of facial biometric information from readily available photos, videos and 3D masks. The literature contains several face anti spoofing methods that try to detect whether the face in the front of the recognition system is real or an artificial replica. However, these methods are not robust and require many improvements since they are sensitive to lightening conditions and pose variations. In order to address these issues, we propose a novel face anti spoofing method based on Multi Color Convolutional Neural Network (CNN) architecture named DeepColorFASD. Our approach investigates the effect of space colors (RGB, HSV and Y CbCr) on CNN architectures and proposes a fusion based voting method for face anti spoofing. In addition, we also explain the resulting feature maps visualizations. We evaluate our system through an experimental study using CASIA FASD: a well-known face anti spoofing database. The results using this challenging database demonstrate that our solution performs better than recent works as measured by Half Total Error Rate (HTER) and ROC curve.

11 citations


Cites methods from "Face anti-spoofing with multifeatur..."

  • ...[4] suggested a face antispoofing approach based on multi feature Videolet aggregation inspired from the fact of combining the evidence obtained from textural and motional information which were extracted from the face region and its surroundings....

    [...]

  • ...[4] Multi feature videolet aggregation SVM CASIA EER=3....

    [...]

References
More filters
Journal ArticleDOI
TL;DR: A novel approach for recognizing DTs is proposed and its simplifications and extensions to facial image analysis are also considered and both the VLBP and LBP-TOP clearly outperformed the earlier approaches.
Abstract: Dynamic texture (DT) is an extension of texture to the temporal domain. Description and recognition of DTs have attracted growing attention. In this paper, a novel approach for recognizing DTs is proposed and its simplifications and extensions to facial image analysis are also considered. First, the textures are modeled with volume local binary patterns (VLBP), which are an extension of the LBP operator widely used in ordinary texture analysis, combining motion and appearance. To make the approach computationally simple and easy to extend, only the co-occurrences of the local binary patterns on three orthogonal planes (LBP-TOP) are then considered. A block-based method is also proposed to deal with specific dynamic events such as facial expressions in which local information and its spatial locations should also be taken into account. In experiments with two DT databases, DynTex and Massachusetts Institute of Technology (MIT), both the VLBP and LBP-TOP clearly outperformed the earlier approaches. The proposed block-based method was evaluated with the Cohn-Kanade facial expression database with excellent results. The advantages of our approach include local processing, robustness to monotonic gray-scale changes, and simple computation

2,653 citations


"Face anti-spoofing with multifeatur..." refers background in this paper

  • ...Dynamic texture features such as LBP-TOP [22] are studied in this regard....

    [...]

Journal ArticleDOI
TL;DR: The inherent strengths of biometrics-based authentication are outlined, the weak links in systems employing biometric authentication are identified, and new solutions for eliminating these weak links are presented.
Abstract: Because biometrics-based authentication offers several advantages over other authentication methods, there has been a significant surge in the use of biometrics for user authentication in recent years. It is important that such biometrics-based authentication systems be designed to withstand attacks when employed in security-critical applications, especially in unattended remote applications such as e-commerce. In this paper we outline the inherent strengths of biometrics-based authentication, identify the weak links in systems employing biometrics-based authentication, and present new solutions for eliminating some of these weak links. Although, for illustration purposes, fingerprint authentication is used throughout, our analysis extends to other biometrics-based methods.

1,709 citations


"Face anti-spoofing with multifeatur..." refers background in this paper

  • ...Biometric systems have different points of vulnerability such as sensor attacks, overriding feature extraction, tampering feature representation, corrupting matcher, tampering stored template, and overriding decision [18]....

    [...]

01 Jan 2009
TL;DR: This thesis builds a human-assisted motion annotation system to obtain ground-truth motion, missing in the literature, for natural video sequences, and proposes SIFT flow, a new framework for image parsing by transferring the metadata information from the images in a large database to an unknown query image.
Abstract: The focus of motion analysis has been on estimating a flow vector for every pixel by matching intensities. In my thesis, I will explore motion representations beyond the pixel level and new applications to which these representations lead. I first focus on analyzing motion from video sequences. Traditional motion analysis suffers from the inappropriate modeling of the grouping relationship of pixels and from a lack of ground-truth data. Using layers as the interface for humans to interact with videos, we build a human-assisted motion annotation system to obtain ground-truth motion, missing in the literature, for natural video sequences. Furthermore, we show that with the layer representation, we can detect and magnify small motions to make them visible to human eyes. Then we move to a contour presentation to analyze the motion for textureless objects under occlusion. We demonstrate that simultaneous boundary grouping and motion analysis can solve challenging data, where the traditional pixel-wise motion analysis fails. In the second part of my thesis, I will show the benefits of matching local image structures instead of intensity values. We propose SIFT flow that establishes dense, semantically meaningful correspondence between two images across scenes by matching pixel-wise SIFT features. Using SIFT flow, we develop a new framework for image parsing by transferring the metadata information, such as annotation, motion and depth, from the images in a large database to an unknown query image. We demonstrate this framework using new applications such as predicting motion from a single image and motion synthesis via object transfer. Based on SIFT flow, we introduce a nonparametric scene parsing system using label transfer, with very promising experimental results suggesting that our system outperforms state-of-the-art techniques based on training classifiers. (Copies available exclusively from MIT Libraries, Rm. 14-0551, Cambridge, MA 02139-4307. Ph. 617-253-5668; Fax 617-253-1690.)

899 citations


"Face anti-spoofing with multifeatur..." refers methods in this paper

  • ...The orientation based optical flow vector is computed by solving the optimization problem 1 using conjugate gradient method [12]....

    [...]

Journal ArticleDOI
TL;DR: An efficient and rather robust face spoof detection algorithm based on image distortion analysis (IDA) that outperforms the state-of-the-art methods in spoof detection and highlights the difficulty in separating genuine and spoof faces, especially in cross-database and cross-device scenarios.
Abstract: Automatic face recognition is now widely used in applications ranging from deduplication of identity to authentication of mobile payment. This popularity of face recognition has raised concerns about face spoof attacks (also known as biometric sensor presentation attacks), where a photo or video of an authorized person’s face could be used to gain access to facilities or services. While a number of face spoof detection techniques have been proposed, their generalization ability has not been adequately addressed. We propose an efficient and rather robust face spoof detection algorithm based on image distortion analysis (IDA). Four different features (specular reflection, blurriness, chromatic moment, and color diversity) are extracted to form the IDA feature vector. An ensemble classifier, consisting of multiple SVM classifiers trained for different face spoof attacks (e.g., printed photo and replayed video), is used to distinguish between genuine (live) and spoof faces. The proposed approach is extended to multiframe face spoof detection in videos using a voting-based scheme. We also collect a face spoof database, MSU mobile face spoofing database (MSU MFSD), using two mobile devices (Google Nexus 5 and MacBook Air) with three types of spoof attacks (printed photo, replayed video with iPhone 5S, and replayed video with iPad Air). Experimental results on two public-domain face spoof databases (Idiap REPLAY-ATTACK and CASIA FASD), and the MSU MFSD database show that the proposed approach outperforms the state-of-the-art methods in spoof detection. Our results also highlight the difficulty in separating genuine and spoof faces, especially in cross-database and cross-device scenarios.

716 citations


"Face anti-spoofing with multifeatur..." refers background or methods in this paper

  • ...• On MSU dataset, HOOF obtains tremendous improvement in EER (from 30.41 to 2.50...

    [...]

  • ...Similarly, at the Inter Feature Fusion stage, the correlation of 0.51, 0.62, and 0.66 is observed for CASIA, MSU, and 3DMAD datasets, respectively....

    [...]

  • ...MSU dataset contains a higher fraction of replay attack videos compared to CASIA....

    [...]

  • ...• Performance of the Proposed Approach: The proposed fusion approach (using HOOF and multi-LBP with face and scene aggregated over videolets) provides 0% EER with uncontrolled illumination and background on both MSU and 3DMAD datasets....

    [...]

  • ...Orthogonal to the LBP texture descriptors based approaches, quality assessment metrics such as specular reflection, blurring and color density are also explored for anti-spoofing [10], [20]....

    [...]

Proceedings Article
27 Sep 2012
TL;DR: This paper inspects the potential of texture features based on Local Binary Patterns (LBP) and their variations on three types of attacks: printed photographs, and photos and videos displayed on electronic screens of different sizes and concludes that LBP show moderate discriminability when confronted with a wide set of attack types.
Abstract: Spoofing attacks are one of the security traits that biometric recognition systems are proven to be vulnerable to. When spoofed, a biometric recognition system is bypassed by presenting a copy of the biometric evidence of a valid user. Among all biometric modalities, spoofing a face recognition system is particularly easy to perform: all that is needed is a simple photograph of the user. In this paper, we address the problem of detecting face spoofing attacks. In particular, we inspect the potential of texture features based on Local Binary Patterns (LBP) and their variations on three types of attacks: printed photographs, and photos and videos displayed on electronic screens of different sizes. For this purpose, we introduce REPLAY-ATTACK, a novel publicly available face spoofing database which contains all the mentioned types of attacks. We conclude that LBP, with ∼15% Half Total Error Rate, show moderate discriminability when confronted with a wide set of attack types.

707 citations


Additional excerpts

  • ...The face anti-spoofing problem is extensively studied in literature, particularly with the introduction of Print Attack dataset [1], Replay Attack dataset [5], CASIA-FASD spoofing dataset [21], 3DMAD database [7], and MSU mobile face spoofing database [20]....

    [...]