Disguise detection and face recognition in visible and thermal spectrums

doi:10.1109/ICB.2013.6613019

Home
/
Papers
/
Disguise detection and face recognition in visible and thermal spectrums

Proceedings Article•DOI•

Disguise detection and face recognition in visible and thermal spectrums

Tejas I. Dhamecha¹, Aastha Nigam¹, Richa Singh¹, Mayank Vatsa¹•Institutions (1)

Indraprastha Institute of Information Technology¹

04 Jun 2013-pp 1-8

TL;DR: A framework, termed as Aravrta1, is proposed, which classifies the local facial regions of both visible and thermal face images into biometric (regions without disguise) and non-biometric (Regions with disguise) classes, and improves the performance compared to existing algorithms.

read less

Abstract: Face verification, though for humans seems to be an easy task, is a long-standing research area. With challenging covariates such as disguise or face obfuscation, automatically verifying the identity of a person is assumed to be very hard. This paper explores the feasibility of face verification under disguise variations using multi-spectrum (visible and thermal) face images. We propose a framework, termed as Aravrta1, which classifies the local facial regions of both visible and thermal face images into biometric (regions without disguise) and non-biometric (regions with disguise) classes. The biometric patches are then used for facial feature extraction and matching. The performance of the algorithm is evaluated on the IHTD In and Beyond Visible Spectrum Disguise database that is prepared by the authors and contains images pertaining to 75 subjects with different kinds of disguise variations. The experimental results suggest that the proposed framework improves the performance compared to existing algorithms, however there is a need for more research to address this important covariate.

...read moreread less

Citations

PDF

Open Access

More filters

Journal Article•DOI•

Biometric Antispoofing Methods: A Survey in Face Recognition

[...]

Javier Galbally¹, Sébastien Marcel², Julian Fierrez³•Institutions (3)

Institute for the Protection and Security of the Citizen¹, Idiap Research Institute², Autonomous University of Madrid³

18 Dec 2014-IEEE Access

TL;DR: The goal of this paper is to provide a comprehensive overview on the work that has been carried out over the last decade in the emerging field of antispoofing, with special attention to the mature and largely deployed face modality.

...read moreread less

Abstract: In recent decades, we have witnessed the evolution of biometric technology from the first pioneering works in face and voice recognition to the current state of development wherein a wide spectrum of highly accurate systems may be found, ranging from largely deployed modalities, such as fingerprint, face, or iris, to more marginal ones, such as signature or hand. This path of technological evolution has naturally led to a critical issue that has only started to be addressed recently: the resistance of this rapidly emerging technology to external attacks and, in particular, to spoofing. Spoofing, referred to by the term presentation attack in current standards, is a purely biometric vulnerability that is not shared with other IT security solutions. It refers to the ability to fool a biometric system into recognizing an illegitimate user as a genuine one by means of presenting a synthetic forged version of the original biometric trait to the sensor. The entire biometric community, including researchers, developers, standardizing bodies, and vendors, has thrown itself into the challenging task of proposing and developing efficient protection methods against this threat. The goal of this paper is to provide a comprehensive overview on the work that has been carried out over the last decade in the emerging field of antispoofing, with special attention to the mature and largely deployed face modality. The work covers theories, methodologies, state-of-the-art techniques, and evaluation databases and also aims at providing an outlook into the future of this very active field of research.

...read moreread less

366 citations

Cites background from "Disguise detection and face recogni..."

...Some initial efforts to study thermal imaging for liveness detection have already been carried out [159], including the acquisition of a significantly large database of thermal images for standard and disguised access attempts where very promising results have been obtained [122]....
[...]

Journal Article•DOI•

Person Recognition System Based on a Combination of Body Images from Visible Light and Thermal Cameras.

[...]

Dat Tien Nguyen¹, Hyung Gil Hong¹, Ki Wan Kim¹, Kang Ryoung Park¹•Institutions (1)

Dongguk University¹

16 Mar 2017-Sensors

TL;DR: The experimental results show that the proposed person recognition method using the information extracted from body images is efficient for enhancing recognition accuracy compared to systems that use only visible light or thermal images of the human body.

...read moreread less

Abstract: The human body contains identity information that can be used for the person recognition (verification/recognition) problem. In this paper, we propose a person recognition method using the information extracted from body images. Our research is novel in the following three ways compared to previous studies. First, we use the images of human body for recognizing individuals. To overcome the limitations of previous studies on body-based person recognition that use only visible light images for recognition, we use human body images captured by two different kinds of camera, including a visible light camera and a thermal camera. The use of two different kinds of body image helps us to reduce the effects of noise, background, and variation in the appearance of a human body. Second, we apply a state-of-the art method, called convolutional neural network (CNN) among various available methods, for image features extraction in order to overcome the limitations of traditional hand-designed image feature extraction methods. Finally, with the extracted image features from body images, the recognition task is performed by measuring the distance between the input and enrolled samples. The experimental results show that the proposed method is efficient for enhancing recognition accuracy compared to systems that use only visible light or thermal images of the human body.

...read moreread less

335 citations

Journal Article•DOI•

Biometric Face Presentation Attack Detection With Multi-Channel Convolutional Neural Network

[...]

Anjith George¹, Zohreh Mostaani¹, David Geissenbuhler¹, Olegs Nikisins¹, André Anjos¹, Sébastien Marcel¹ - Show less +2 more•Institutions (1)

Idiap Research Institute¹

01 Jan 2020-IEEE Transactions on Information Forensics and Security

TL;DR: In this article, a multi-channel Convolutional Neural Network-based approach for presentation attack detection (PAD) has been proposed, and the new Wide Multi-Channel presentation Attack (WMCA) database is introduced.

...read moreread less

Abstract: Face recognition is a mainstream biometric authentication method. However, the vulnerability to presentation attacks (a.k.a. spoofing) limits its usability in unsupervised applications. Even though there are many methods available for tackling presentation attacks (PA), most of them fail to detect sophisticated attacks such as silicone masks. As the quality of presentation attack instruments improves over time, achieving reliable PA detection with visual spectra alone remains very challenging. We argue that analysis in multiple channels might help to address this issue. In this context, we propose a multi-channel Convolutional Neural Network-based approach for presentation attack detection (PAD). We also introduce the new Wide Multi-Channel presentation Attack (WMCA) database for face PAD which contains a wide variety of 2D and 3D presentation attacks for both impersonation and obfuscation attacks. Data from different channels such as color, depth, near-infrared, and thermal are available to advance the research in face PAD. The proposed method was compared with feature-based approaches and found to outperform the baselines achieving an ACER of 0.3% on the introduced dataset. The database and the software to reproduce the results are made available publicly.

...read moreread less

139 citations

Journal Article•DOI•

A survey on heterogeneous face recognition

[...]

Shuxin Ouyang¹, Timothy M. Hospedales¹, Yi-Zhe Song¹, Xueming Li², Chen Change Loy³, Xiaogang Wang³ - Show less +2 more•Institutions (3)

Queen Mary University of London¹, Beijing University of Posts and Telecommunications², The Chinese University of Hong Kong³

01 Dec 2016-Image and Vision Computing

TL;DR: This survey provides a comprehensive review of established techniques and recent developments in HFR, and offers a detailed account of datasets and benchmarks commonly used for evaluation.

...read moreread less

114 citations

Cites background from "Disguise detection and face recogni..."

...sting and challenging covariate of face recognition. It includes intentional or unintentional changes through which one can either hide his/her identity or appear to be someone else. Dhamecha et al. [Dhamecha et al. 2013] have summarized the existing disguise detection and face recognition algorithms. In this survey, we rather focus on matching across plastic surgery variations. With reduction in cost and time requir...
[...]

Journal Article•DOI•

A Comprehensive Database for Benchmarking Imaging Systems

[...]

Karen Panetta¹, Arash Samani¹, Xin Yuan¹, Qianwen Wan¹, Sos S. Agaian², Srijith Rajeev¹, K M Shreyas Kamath¹, Rahul Rajendran¹, Shishir Paramathma Rao¹, Aleksandra Kaszowska¹, Holly A. Taylor¹ - Show less +7 more•Institutions (2)

Tufts University¹, City University of New York²

01 Mar 2020-IEEE Transactions on Pattern Analysis and Machine Intelligence

TL;DR: The Tufts Face Database is introduced that includes images acquired in various modalities: photograph images, thermal images, near infrared images, a recorded video, a computerized facial sketch, and 3D images of each volunteer's face.

...read moreread less

Abstract: Cross-modality face recognition is an emerging topic due to the wide-spread usage of different sensors in day-to-day life applications. The development of face recognition systems relies greatly on existing databases for evaluation and obtaining training examples for data-hungry machine learning algorithms. However, currently, there is no publicly available face database that includes more than two modalities for the same subject. In this work, we introduce the Tufts Face Database that includes images acquired in various modalities: photograph images, thermal images, near infrared images, a recorded video, a computerized facial sketch, and 3D images of each volunteer's face. An Institutional Research Board protocol was obtained and images were collected from students, staff, faculty, and their family members at Tufts University. The database includes over 10,000 images from 113 individuals from more than 15 different countries, various gender identities, ages, and ethnic backgrounds. The contributions of this work are: 1) Detailed description of the content and acquisition procedure for images in the Tufts Face Database; 2) The Tufts Face Database is publicly available to researchers worldwide, which will allow assessment and creation of more robust, consistent, and adaptable recognition algorithms; 3) A comprehensive, up-to-date review on face recognition systems and face datasets.

...read moreread less

111 citations

Additional excerpts

...Currently, feature extraction algorithms such as gray-scale value histogram features [30], thermal local binary pattern (LBP), SIFT features [31], and speeded up robust features (SURF) [32, 33], are widely applied in thermal face recognition....
[...]
...Currently, feature extraction algorithms such as gray-scale value histogram features [30], thermal local binary pattern (LBP), SIFT features [31], and speeded up robust features (SURF) [32], [33], are widely applied in thermal face recognition....
[...]

1
2
3
4
…
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22

Collapse

References

PDF

Open Access

More filters

Journal Article•DOI•

Support-Vector Networks

[...]

Corinna Cortes¹, Vladimir Vapnik¹•Institutions (1)

Bell Labs¹

15 Sep 1995-Machine Learning

TL;DR: High generalization ability of support-vector networks utilizing polynomial input transformations is demonstrated and the performance of the support- vector network is compared to various classical learning algorithms that all took part in a benchmark study of Optical Character Recognition.

...read moreread less

Abstract: The support-vector network is a new learning machine for two-group classification problems. The machine conceptually implements the following idea: input vectors are non-linearly mapped to a very high-dimension feature space. In this feature space a linear decision surface is constructed. Special properties of the decision surface ensures high generalization ability of the learning machine. The idea behind the support-vector network was previously implemented for the restricted case where the training data can be separated without errors. We here extend this result to non-separable training data. High generalization ability of support-vector networks utilizing polynomial input transformations is demonstrated. We also compare the performance of the support-vector network to various classical learning algorithms that all took part in a benchmark study of Optical Character Recognition.

...read moreread less

37,861 citations

"Disguise detection and face recogni..." refers methods in this paper

...For every patch, a decision score s is computed using SVM....
[...]
...The normalized descriptor is provided as input to SVM with Radial Basis Function kernel for patch classification....
[...]
...Face recognition with biometric patches is classified using ITE and SVM classifier 2....
[...]
...An SVM model is learned from the ITE descriptors of all the patches from training images (which are annotated manually)....
[...]
...Therefore, in this research, we have proposed the use of Support Vector Machine (SVM) [8], a discriminative classifier, for classifying biometric and non-biometric patches....
[...]

Journal Article•DOI•

Robust Face Recognition via Sparse Representation

[...]

John Wright¹, Allen Y. Yang², Arvind Ganesh¹, S. Shankar Sastry², Yi Ma¹ - Show less +1 more•Institutions (2)

University of Illinois at Urbana–Champaign¹, University of California, Berkeley²

01 Feb 2009-IEEE Transactions on Pattern Analysis and Machine Intelligence

TL;DR: This work considers the problem of automatically recognizing human faces from frontal views with varying expression and illumination, as well as occlusion and disguise, and proposes a general classification algorithm for (image-based) object recognition based on a sparse representation computed by C1-minimization.

...read moreread less

Abstract: We consider the problem of automatically recognizing human faces from frontal views with varying expression and illumination, as well as occlusion and disguise. We cast the recognition problem as one of classifying among multiple linear regression models and argue that new theory from sparse signal representation offers the key to addressing this problem. Based on a sparse representation computed by C1-minimization, we propose a general classification algorithm for (image-based) object recognition. This new framework provides new insights into two crucial issues in face recognition: feature extraction and robustness to occlusion. For feature extraction, we show that if sparsity in the recognition problem is properly harnessed, the choice of features is no longer critical. What is critical, however, is whether the number of features is sufficiently large and whether the sparse representation is correctly computed. Unconventional features such as downsampled images and random projections perform just as well as conventional features such as eigenfaces and Laplacianfaces, as long as the dimension of the feature space surpasses certain threshold, predicted by the theory of sparse representation. This framework can handle errors due to occlusion and corruption uniformly by exploiting the fact that these errors are often sparse with respect to the standard (pixel) basis. The theory of sparse representation helps predict how much occlusion the recognition algorithm can handle and how to choose the training images to maximize robustness to occlusion. We conduct extensive experiments on publicly available databases to verify the efficacy of the proposed algorithm and corroborate the above claims.

...read moreread less

9,658 citations

"Disguise detection and face recogni..." refers methods in this paper

...The proposed approach improves verification accuracy over direct application of LBP, COTS, and SRC....
[...]
...In case of SRC, for fusion of two spectrums, sum rule fusion of residuals of both individual spectrums is performed....
[...]
...Note that, SRC is one of the most important techniques in literature for addressing occlusion/disguise....
[...]
...The result section is divided into three subsections: patch classification using ITE, evaluation of the proposed algorithm, and comparison with sparse representation classifier (SRC) and COTS....
[...]
...As shown in Table 2, although the performance reported by the proposed approach is not as high as it is usually reported in face recognition literature, it outperforms one of the state-of-art commercial system, and a widely used technique, SRC, by at least 3% at 1.0% FAR....
[...]

Journal Article•DOI•

Face recognition: A literature survey

[...]

W. Zhao¹, Rama Chellappa², P. J. Phillips³, Azriel Rosenfeld²•Institutions (3)

Sarnoff Corporation¹, University of Maryland, College Park², National Institute of Standards and Technology³

01 Dec 2003-ACM Computing Surveys

TL;DR: In this paper, the authors provide an up-to-date critical survey of still-and video-based face recognition research, and provide some insights into the studies of machine recognition of faces.

...read moreread less

Abstract: As one of the most successful applications of image analysis and understanding, face recognition has recently received significant attention, especially during the past several years. At least two reasons account for this trend: the first is the wide range of commercial and law enforcement applications, and the second is the availability of feasible technologies after 30 years of research. Even though current machine recognition systems have reached a certain level of maturity, their success is limited by the conditions imposed by many real applications. For example, recognition of face images acquired in an outdoor environment with changes in illumination and/or pose remains a largely unsolved problem. In other words, current systems are still far away from the capability of the human perception system.This paper provides an up-to-date critical survey of still- and video-based face recognition research. There are two underlying motivations for us to write this survey paper: the first is to provide an up-to-date review of the existing literature, and the second is to offer some insights into the studies of machine recognition of faces. To provide a comprehensive survey, we not only categorize existing recognition techniques but also present detailed descriptions of representative methods within each category. In addition, relevant topics such as psychophysical studies, system evaluation, and issues of illumination and pose variation are covered.

...read moreread less

6,384 citations

The AR face database

[...]

Aleix M. Martinez

01 Jan 1998

3,650 citations

"Disguise detection and face recogni..." refers background in this paper

...Further, the visible spectrum databases generally used for disguise related research (AR [13] and Yale [2] face databases) contain very limited disguise variations, such as scarves and/or sunglasses....
[...]
...AR database [13] contains 3200+ images pertaining to 126 subjects with two kinds of disguises (sunglasses and scarves)....
[...]
...Note that most of the research has been performed in visible spectrum using the AR [13] and Yale [2] face databases which contains very limited disguise (sunglasses and scarves only)....
[...]

Journal Article•

The AR face databasae

[...]

A. M. Martinez

01 Jan 1998-CTIT technical reports series

2,952 citations

"Disguise detection and face recogni..." refers background in this paper

...Further, the visible spectrum databases generally used for disguise related research (AR [13] and Yale [2] face databases) contain very limited disguise variations, such as scarves and/or sunglasses....
[...]
...AR database [13] contains 3200+ images pertaining to 126 subjects with two kinds of disguises (sunglasses and scarves)....
[...]
...Note that most of the research has been performed in visible spectrum using the AR [13] and Yale [2] face databases which contains very limited disguise (sunglasses and scarves only)....
[...]