scispace - formally typeset
Search or ask a question

Showing papers by "Nalini K. Ratha published in 2018"


Posted Content
TL;DR: This paper attempts to unravel three aspects related to the robustness of DNNs for face recognition in terms of vulnerabilities to attacks inspired by commonly observed distortions in the real world, and presents several effective countermeasures to mitigate the impact of adversarial attacks and improve the overall robustnessof DNN-based face recognition.
Abstract: Deep neural network (DNN) architecture based models have high expressive power and learning capacity. However, they are essentially a black box method since it is not easy to mathematically formulate the functions that are learned within its many layers of representation. Realizing this, many researchers have started to design methods to exploit the drawbacks of deep learning based algorithms questioning their robustness and exposing their singularities. In this paper, we attempt to unravel three aspects related to the robustness of DNNs for face recognition: (i) assessing the impact of deep architectures for face recognition in terms of vulnerabilities to attacks inspired by commonly observed distortions in the real world that are well handled by shallow learning methods along with learning based adversaries; (ii) detecting the singularities by characterizing abnormal filter response behavior in the hidden layers of deep networks; and (iii) making corrections to the processing pipeline to alleviate the problem. Our experimental evaluation using multiple open-source DNN-based face recognition networks, including OpenFace and VGG-Face, and two publicly available databases (MEDS and PaSC) demonstrates that the performance of deep learning based face recognition algorithms can suffer greatly in the presence of such distortions. The proposed method is also compared with existing detection algorithms and the results show that it is able to detect the attacks with very high accuracy by suitably designing a classifier using the response of the hidden layers in the network. Finally, we present several effective countermeasures to mitigate the impact of adversarial attacks and improve the overall robustness of DNN-based face recognition.

103 citations


Proceedings Article
27 Apr 2018
TL;DR: In this article, the authors investigated the impact of adversarial attacks on the robustness of DNN-based face recognition models and proposed several effective countermeasures to mitigate the impact.
Abstract: Deep neural network (DNN) architecture based models have high expressive power and learning capacity. However, they are essentially a black box method since it is not easy to mathematically formulate the functions that are learned within its many layers of representation. Realizing this, many researchers have started to design methods to exploit the drawbacks of deep learning based algorithms questioning their robustness and exposing their singularities. In this paper, we attempt to unravel three aspects related to the robustness of DNNs for face recognition: (i) assessing the impact of deep architectures for face recognition in terms of vulnerabilities to attacks inspired by commonly observed distortions in the real world that are well handled by shallow learning methods along with learning based adversaries; (ii) detecting the singularities by characterizing abnormal filter response behavior in the hidden layers of deep networks; and (iii) making corrections to the processing pipeline to alleviate the problem. Our experimental evaluation using multiple open-source DNN-based face recognition networks, including OpenFace and VGG-Face, and two publicly available databases (MEDS and PaSC) demonstrates that the performance of deep learning based face recognition algorithms can suffer greatly in the presence of such distortions. The proposed method is also compared with existing detection algorithms and the results show that it is able to detect the attacks with very high accuracy by suitably designing a classifier using the response of the hidden layers in the network. Finally, we present several effective countermeasures to mitigate the impact of adversarial attacks and improve the overall robustness of DNN-based face recognition.

102 citations


Proceedings ArticleDOI
18 Jun 2018
TL;DR: A novel Disguised Faces in the Wild (DFW) dataset, consisting of over 11,000 images for understanding and pushing the current state-of-the-art for disguised face recognition, along with the phase-I results of the CVPR2018 competition.
Abstract: Existing research in the field of face recognition with variations due to disguises focuses primarily on images captured in controlled settings. Limited research has been performed on images captured in unconstrained environments, primarily due to the lack of corresponding disguised face datasets. In order to overcome this limitation, this work presents a novel Disguised Faces in the Wild (DFW) dataset, consisting of over 11,000 images for understanding and pushing the current state-of-the-art for disguised face recognition. To the best of our knowledge, DFW is a first-of-a-kind dataset containing images pertaining to both obfuscation and impersonation for understanding the effect of disguise variations. A major portion of the dataset has been collected from the Internet, thereby encompassing a wide variety of disguise accessories and variations across other covariates. As part of CVPR2018, a competition and workshop are organized to facilitate research in this direction. This paper presents a description of the dataset, the baseline protocols and performance, along with the phase-I results of the competition.

76 citations


Posted Content
TL;DR: Evidence is brought forth suggesting that differences in lip, eye and cheek structure across ethnicity lead to the differences in commercial face classification services, and lip and eye makeup are seen as strong predictors for a female face, which is a troubling propagation of a gender stereotype.
Abstract: Recent work shows unequal performance of commercial face classification services in the gender classification task across intersectional groups defined by skin type and gender. Accuracy on dark-skinned females is significantly worse than on any other group. In this paper, we conduct several analyses to try to uncover the reason for this gap. The main finding, perhaps surprisingly, is that skin type is not the driver. This conclusion is reached via stability experiments that vary an image's skin type via color-theoretic methods, namely luminance mode-shift and optimal transport. A second suspect, hair length, is also shown not to be the driver via experiments on face images cropped to exclude the hair. Finally, using contrastive post-hoc explanation techniques for neural networks, we bring forth evidence suggesting that differences in lip, eye and cheek structure across ethnicity lead to the differences. Further, lip and eye makeup are seen as strong predictors for a female face, which is a troubling propagation of a gender stereotype.

38 citations


Patent
01 Feb 2018
TL;DR: In this paper, a neural network processes adversarial input data and layer-wise comparison logic compares, for each intermediate layer of the neural network, a response generated by the intermediate layer based on processing the adversarial inputs, to the mean response associated with intermediate layer, to thereby generate a distance metric for the intermediate layers.
Abstract: Mechanisms are provided for training a classifier to identify adversarial input data. A neural network processes original input data representing a plurality of non-adversarial original input data and mean output learning logic determines a mean response for each intermediate layer of the neural network based on results of processing the original input data. The neural network processes adversarial input data and layer-wise comparison logic compares, for each intermediate layer of the neural network, a response generated by the intermediate layer based on processing the adversarial input data, to the mean response associated with the intermediate layer, to thereby generate a distance metric for the intermediate layer. The layer-wise comparison logic generates a vector output based on the distance metrics that is used to train a classifier to identify adversarial input data based on responses generated by intermediate layers of the neural network.

10 citations


Posted Content
TL;DR: In this article, a novel heterogeneity aware loss function within a deep learning framework was proposed to address the heterogeneous challenge of recognizing biometric patterns in unconstrained environment including variations in mobile camera sensors, illumination conditions, and capture distance.
Abstract: Mobile biometric approaches provide the convenience of secure authentication with an omnipresent technology. However, this brings an additional challenge of recognizing biometric patterns in unconstrained environment including variations in mobile camera sensors, illumination conditions, and capture distance. To address the heterogeneous challenge, this research presents a novel heterogeneity aware loss function within a deep learning framework. The effectiveness of the proposed loss function is evaluated for periocular biometrics using the CSIP, IMP and VISOB mobile periocular databases. The results show that the proposed algorithm yields state-of-the-art results in a heterogeneous environment and improves generalizability for cross-database experiments.

9 citations


Proceedings ArticleDOI
01 Oct 2018
TL;DR: A novel heterogeneity aware loss function within a deep learning framework for periocular biometrics that yields state-of-the-art results in a heterogeneous environment and improves generalizability for cross-database experiments is presented.
Abstract: Mobile biometric approaches provide the convenience of secure authentication with an omnipresent technology. However, this brings an additional challenge of recognizing biometric patterns in an unconstrained environment including variations in mobile camera sensors, illumination conditions, and capture distance. To address the heterogeneous challenge, this research presents a novel heterogeneity aware loss function within a deep learning framework. The effectiveness of the proposed loss function is evaluated for periocular biometrics using the CSIP, IMP and VISOB mobile periocular databases. The results show that the proposed algorithm yields state-of-the-art results in a heterogeneous environment and improves generalizability for cross-database experiments.

8 citations


Patent
03 Jul 2018
TL;DR: In this paper, the authors present an example operation where a blockchain is configured to use one or more smart contracts that specify transactions among a plurality of end-users, creating on the blockchain the smart contract defining authentication parameters for an authentication of an end-user from the plurality of the end users, executing the smart contracts to perform the authentication of the user associated with a transaction based on the authentication parameters by generating an authentication challenge for the transaction, and recording an authentication log produced by the authentication challenge into a metadata of a transaction payload for analytics.
Abstract: An example operation may include one or more of joining, by a host device, a blockchain managed by one or more devices on a decentralized network, the blockchain is configured to use one or more smart contracts that specify transactions among a plurality of end-users, creating on the blockchain the smart contract defining authentication parameters for an authentication of an end-user from the plurality of the end-users, executing the smart contract to perform the authentication of the end-user associated with a transaction based on the authentication parameters by generating an authentication challenge for the transaction, and recording an authentication log produced by the authentication challenge into a metadata of a transaction payload for analytics.

3 citations


Patent
04 Oct 2018
TL;DR: In this paper, an entity learning method, system, and computer program product include learning from at least one entity to produce augments entities such that an augmented entity is still recognizable as the original entity but differs sufficiently to produce a different feature representation of the entity to create a database for use.
Abstract: An entity learning recognition method, system, and computer program product include learning (i.e., in a training phase) from at least one entity to produce augments entities such that an augmented entity is still recognizable as the original entity but differs sufficiently to produce a different feature representation of the entity to create a database for use (i.e., in an implementation phase).

2 citations


Posted Content
TL;DR: The Disguised Faces in the Wild (DFW) dataset as discussed by the authors contains over 11,000 images of 1000 identities with different types of disguise accessories, including impersonator and genuine obfuscated face images for each subject.
Abstract: Research in face recognition has seen tremendous growth over the past couple of decades. Beginning from algorithms capable of performing recognition in constrained environments, the current face recognition systems achieve very high accuracies on large-scale unconstrained face datasets. While upcoming algorithms continue to achieve improved performance, a majority of the face recognition systems are susceptible to failure under disguise variations, one of the most challenging covariate of face recognition. Most of the existing disguise datasets contain images with limited variations, often captured in controlled settings. This does not simulate a real world scenario, where both intentional and unintentional unconstrained disguises are encountered by a face recognition system. In this paper, a novel Disguised Faces in the Wild (DFW) dataset is proposed which contains over 11000 images of 1000 identities with different types of disguise accessories. The dataset is collected from the Internet, resulting in unconstrained face images similar to real world settings. This is the first-of-a-kind dataset with the availability of impersonator and genuine obfuscated face images for each subject. The proposed dataset has been analyzed in terms of three levels of difficulty: (i) easy, (ii) medium, and (iii) hard in order to showcase the challenging nature of the problem. It is our view that the research community can greatly benefit from the DFW dataset in terms of developing algorithms robust to such adversaries. The proposed dataset was released as part of the First International Workshop and Competition on Disguised Faces in the Wild at CVPR, 2018. This paper presents the DFW dataset in detail, including the evaluation protocols, baseline results, performance analysis of the submissions received as part of the competition, and three levels of difficulties of the DFW challenge dataset.

2 citations


Patent
10 May 2018
TL;DR: In this paper, the age progression of a test facial image is facilitated by compiling training data, including a training set(s) having selected initial images of subjects by gender and age group.
Abstract: Age progression of a test facial image is facilitated by compiling training data, including a training set(s) having selected initial images of subjects by gender and age-group In addition, the age progression includes manipulating the training data, including: for a given age-group of a training set, substantially aligning respective face shapes; determining a common frame based on the aligned shapes; substantially aligning respective face appearances to generate a shape-free form corresponding to the face appearance of each subject, using the substantially aligned shapes to generate an age-specific shape-dictionary for each age-group, and a common shape-dictionary for the age-groups of the training set, and using the aligned appearances to generate at least an age-specific appearance-dictionary for each age-group, and a common appearance-dictionary for the age-groups of the training set The age specific appearance dictionary for each age group and the common appearance dictionary facilitate age progression of the facial image

Patent
18 Apr 2018
TL;DR: In this article, the authors present an example operation of detecting a suspected biometric authentication incident, submitting a first blockchain transaction, including a first report to a blockchain network, and submitting a second blockchain transaction including a second report to the blockchain network.
Abstract: An example operation may include one or more of detecting a suspected biometric authentication incident, submitting a first blockchain transaction including a first report to a blockchain network, submitting a second blockchain transaction including a second report to the blockchain network, and taking an action, by one or more blockchain nodes, in response to determining one or more of the first and second reports are relevant to the one or more blockchain nodes. The first report includes public and private data corresponding to the suspected biometric authentication incident, and the second report includes one or more of a root cause and mitigation steps for the incident.

Patent
18 Oct 2018
TL;DR: In this article, a computer-implemented method modifies physical classroom resources in a classroom by identifying and quantifying physical classroom resource constraints that impede learning by students in the classroom.
Abstract: A computer-implemented method modifies physical classroom resources in a classroom. One or more processors identify and quantify physical classroom resources in the classroom based on sensor readings received from sensors in a classroom. The processors determine physical classroom resource constraints that impede learning by students in the classroom based on the sensor readings from the sensors in the classroom. The processors detect one or more of the physical classroom resource constraints in the physical classroom resources identified by the sensor readings, and then adjust the one or more physical classroom resources based on the one or more detected physical classroom resource constraints.

Patent
08 Nov 2018
TL;DR: In this paper, a computer-implemented method, data processor and computer program product is used to determine exposure levels to external stimuli by integrating the intensity level over time, and a personal exposure level limitation is determined for the user based on the measured at least one human biometric quantity.
Abstract: A computer-implemented method, data processor and computer program product determine exposure levels to external stimuli. At least one environmental condition is monitored and an external stimulus event is identified based on the at least one environmental condition. An intensity level of the external stimulus event is determined to exceed a predetermined threshold. An exposure level of the external stimulus is determined by integrating the intensity level over time. At least one human biometric quantity of a user is measured and a personal exposure level limitation is determined for the user based on the measured at least one human biometric quantity. When the exposure level exceeds the personal exposure level limitation, the user is warned of the exposure level. The human biometric quantity is one of: heart rate, blood pressure, body temperature, glucose level, blood oxygen level, muscle activity, electrolyte level, and lactic acid level.