Showing papers by "Nalini K. Ratha published in 2019"

Diversity in Faces.

Abstract: Face recognition is a long standing challenge in the field of Artificial Intelligence (AI). The goal is to create systems that accurately detect, recognize, verify, and understand human faces. There are significant technical hurdles in making these systems accurate, particularly in unconstrained settings due to confounding factors related to pose, resolution, illumination, occlusion, and viewpoint. However, with recent advances in neural networks, face recognition has achieved unprecedented accuracy, largely built on data-driven deep learning methods. While this is encouraging, a critical aspect that is limiting facial recognition accuracy and fairness is inherent facial diversity. Every face is different. Every face reflects something unique about us. Aspects of our heritage - including race, ethnicity, culture, geography - and our individual identify - age, gender, and other visible manifestations of self-expression, are reflected in our faces. We expect face recognition to work equally accurately for every face. Face recognition needs to be fair. As we rely on data-driven methods to create face recognition technology, we need to ensure necessary balance and coverage in training data. However, there are still scientific questions about how to represent and extract pertinent facial features and quantitatively measure facial diversity. Towards this goal, Diversity in Faces (DiF) provides a data set of one million annotated human face images for advancing the study of facial diversity. The annotations are generated using ten well-established facial coding schemes from the scientific literature. The facial coding schemes provide human-interpretable quantitative measures of facial features. We believe that by making the extracted coding schemes available on a large set of faces, we can accelerate research and development towards creating more fair and accurate facial recognition systems.

Detecting and Mitigating Adversarial Perturbations for Robust Face Recognition

Abstract: Deep neural network (DNN) architecture based models have high expressive power and learning capacity. However, they are essentially a black box method since it is not easy to mathematically formulate the functions that are learned within its many layers of representation. Realizing this, many researchers have started to design methods to exploit the drawbacks of deep learning based algorithms questioning their robustness and exposing their singularities. In this paper, we attempt to unravel three aspects related to the robustness of DNNs for face recognition: (i) assessing the impact of deep architectures for face recognition in terms of vulnerabilities to attacks, (ii) detecting the singularities by characterizing abnormal filter response behavior in the hidden layers of deep networks; and (iii) making corrections to the processing pipeline to alleviate the problem. Our experimental evaluation using multiple open-source DNN-based face recognition networks, and three publicly available face databases demonstrates that the performance of deep learning based face recognition algorithms can suffer greatly in the presence of such distortions. We also evaluate the proposed approaches on four existing quasi-imperceptible distortions: DeepFool, Universal adversarial perturbations, $$l_2$$ , and Elastic-Net (EAD). The proposed method is able to detect both types of attacks with very high accuracy by suitably designing a classifier using the response of the hidden layers in the network. Finally, we present effective countermeasures to mitigate the impact of adversarial attacks and improve the overall robustness of DNN-based face recognition.

Towards Deep Neural Network Training on Encrypted Data

Abstract: While deep learning is a valuable tool for solving many tough problems in computer vision, the success of deep learning models is typically determined by: (i) availability of sufficient training data, (ii) access to extensive computational resources, and (iii) expertise in selecting the right model and hyperparameters for the selected task. Often, the availability of data is the hard part due to compliance, legal, and privacy constraints. Cryptographic techniques such as fully homomorphic encryption (FHE) offer a potential solution by enabling processing on encrypted data. While prior work has been done on using FHE for inferencing, training a deep neural network in the encrypted domain is an extremely challenging task due to the computational complexity of the operations involved. In this paper, we evaluate the feasibility of training neural networks on encrypted data in a completely non-interactive way. Our proposed system uses the open-source FHE toolkit HElib to implement a Stochastic Gradient Descent (SGD)-based training of a neural network. We show that encrypted training can be made more computationally efficient by (i) simplifying the network with minimal degradation of accuracy, (ii) choosing appropriate data representation and resolution, and (iii) packing the data elements within the ciphertext in a smart way so as to minimize the number of operations and facilitate parallelization of FHE computations. Based on the above optimizations, we demonstrate that it is possible to achieve more than 50x speed up while training a fully-connected neural network on the MNIST dataset while achieving reasonable accuracy (96%). Though the cost of training a complex deep learning model from scratch on encrypted data is still very high, this work establishes a solid baseline and paves the way for relatively simpler tasks such as fine-tuning of deep learning models based on encrypted data to be implemented in the near future.

Deep Learning for Face Recognition: Pride or Prejudiced?

Abstract: Do very high accuracies of deep networks suggest pride of effective AI or are deep networks prejudiced? Do they suffer from in-group biases (own-race-bias and own-age-bias), and mimic the human behavior? Is in-group specific information being encoded sub-consciously by the deep networks? This research attempts to answer these questions and presents an in-depth analysis of `bias' in deep learning based face recognition systems This is the first work which decodes if and where bias is encoded for face recognition Taking cues from cognitive studies, we inspect if deep networks are also affected by social in- and out-group effect Networks are analyzed for own-race and own-age bias, both of which have been well established in human beings The sub-conscious behavior of face recognition models is examined to understand if they encode race or age specific features for face recognition Analysis is performed based on 36 experiments conducted on multiple datasets Four deep learning networks either trained from scratch or pre-trained on over 10M images are used Variations across class activation maps and feature visualizations provide novel insights into the functioning of deep learning systems, suggesting behavior similar to humans It is our belief that a better understanding of state-of-the-art deep learning networks would enable researchers to address the given challenge of bias in AI, and develop fairer systems

Recognizing Disguised Faces in the Wild

Abstract: Research in face recognition has seen tremendous growth over the past couple of decades. Beginning from algorithms capable of performing recognition in constrained environments, existing face recognition systems achieve very high accuracies on large-scale unconstrained face datasets. While upcoming algorithms continue to achieve improved performance, many of them are susceptible to reduced performance under disguise variations, one of the most challenging covariate of face recognition. In this paper, the disguised faces in the wild (DFW) dataset is presented, which contains over 11000 images of 1000 identities with variations across different types of disguise accessories (the DFW dataset link: http://iab-rubric.org/resources/dfw.html ). The images are collected from the Internet, resulting in unconstrained variations similar to real-world settings. This is a unique dataset that contains impersonator and genuine obfuscated face images for each subject. The DFW dataset has been analyzed in terms of three levels of difficulty: 1) easy; 2) medium; and 3) hard, in order to showcase the challenging nature of the problem. The dataset was released as part of the First International Workshop and Competition on DFW at the Conference on Computer Vision and Pattern Recognition, 2018. This paper presents the DFW dataset in detail, including the evaluation protocols, baseline results, performance analysis of the submissions received as part of the competition, and three levels of difficulties of the DFW challenge dataset.

DeepRing: Protecting Deep Neural Network With Blockchain

Abstract: Several computer vision applications such as object detection and face recognition have started to completely rely on deep learning based architectures. These architectures, when paired with appropriate loss functions and optimizers, produce state-of-the-art results in a myriad of problems. On the other hand, with the advent of "blockchain", the cybersecurity industry has developed a new sense of trust which was earlier missing from both the technical and commercial perspectives. Employment of cryptographic hash as well as symmetric/asymmetric encryption and decryption algorithms ensure security without any human intervention (i.e., centralized authority). In this research, we present the synergy between the best of both these worlds. We first propose a model which uses the learned parameters of a typical deep neural network and is secured from external adversaries by cryptography and blockchain technology. As the second contribution of the proposed research, a new parameter tampering attack is proposed to properly justify the role of blockchain in machine learning.

Color-Theoretic Experiments to Understand Unequal Gender Classification Accuracy From Face Images

Abstract: Recent work shows unequal performance of commercial face classification services in the gender classification task across intersectional groups defined by skin type and gender. Accuracy on dark-skinned females is significantly worse than on any other group. We provide initial evidence that skin type alone is not the driver for this disparity by conducting novel stability experiments that vary an image's skin type via color-theoretic methods, namely luminance mode-shift and optimal transport. We evaluate the effect of skin type change on the gender classification decision of a pair of state-of-the-art commercial and open-source gender classifiers. The results raise the possibility that broader differences in ethnicity, as opposed to the skin type alone, are what contribute to unequal gender classification accuracy in face images.

Securing CNN Model and Biometric Template using Blockchain

Abstract: Blockchain has emerged as a leading technology that ensures security in a distributed framework. Recently, it has been shown that blockchain can be used to convert traditional blocks of any deep learning models into secure systems. In this research, we model a trained biometric recognition system in an architecture which leverages the blockchain technology to provide fault tolerant access in a distributed environment. The advantage of the proposed approach is that tampering in one particular component alerts the whole system and helps in easy identification of ‘any’ possible alteration. Experimentally, with different biometric modalities, we have shown that the proposed approach provides security to both deep learning model and the biometric template.

Biometric token for blockchain

Abstract: An example operation may include one or more of obtaining a first biometric sample of a user from a user device. extracting, by an issuing node of a permissioned blockchain network, a biometric template from the first biometric sample, encrypting the biometric template, distributing an issuetoken proposal comprising the encrypted biometric template to the blockchain network, and generating and distributing a biometric token to the user device. In response to the user indicating to the user device to redeem the biometric token, the method includes one or more of presenting, by the user device, the biometric token to a verifying node of the blockchain network, validating, by the verifying node, the biometric token, receiving, by the verifying node, a second biometric sample from the user device, distributing a redeemtoken proposal to the blockchain network, committing a transaction corresponding to the biometric token, to the blockchain network, and invalidating the biometric token.

Proving Multimedia Integrity using Sanitizable Signatures Recorded on Blockchain

Abstract: While significant advancements have been made in the field of multimedia forensics to detect altered content, existing techniques mostly focus on enabling the content recipient to verify the content integrity without any inputs from the content creator. In many application scenarios, the creator has a strong incentive to establish the provenance and integrity of the multimedia data created and released by him. Hence, there is a strong need for mechanisms that allow the content creator to prove the authenticity of the released content. Since blockchain technology provides an immutable distributed database, it is an ideal solution for reliably time-stamping content with its creation time and storing an irrefutable signature of the content at the time of its creation. However, a simple digital signature scheme does not allow modification of the content after the initial commitment. Authorized multimedia content alteration by its creator is often necessary (e.g., redaction of faces to protect the privacy of individuals in a video, redaction of sensitive fields in a text document) before the content is distributed. The main contributions of this paper are: (i) a novel sanitizable signature scheme that enables the content creator to prove the integrity of the redacted content, while preventing the recipients from reconstructing the redacted segments based on the published commitment, and (ii) a blockchain-based solution for securely managing the sanitizable signature. The proposed solution employs a robust hashing scheme using chameleon hash function and Merkle tree to generate the initial signature, which is stored on the blockchain. The auxiliary data required for the integrity verification step is retained by the content creator and only a signature of this auxiliary data is stored on the blockchain. Any modifications to the multimedia content requires only updating the signature of the auxiliary data, which is securely recorded on the blockchain. We demonstrate that the proposed approach enables verification of integrity of redacted multimedia content without compromising the content privacy requirements.

Device authentication using synchronized activity signature comparison

Abstract: A system, method and program product for authenticating a device. An authentication service is provided having: a data management system for periodically collecting and storing signature data from each of a set of registered devices, wherein the signature data includes a plurality of data points, and wherein at least one of the data points includes a device usage characteristic; a system for obtaining a temporal signature state (TSS) vector of a device in response to a transaction request from the device, wherein the TSS vector includes values for a selected subset of the data points forming the signature data; and an authenticator for comparing the TSS vector of the device with stored signature data in order to authenticate the device.

Using robot plans as parallel linguistic corpora

Abstract: Parallel linguistic corpora used to train cognitive translation systems are built from robot plans. A collection of robot plans in various languages is analyzed in candidate pairings to see if any two plans are directed to the same task. If so, verbal instructions in different languages from the two plans (such as plan names) are assumed to have the same meaning, and are stored in association as part of the bilingual corpus. If each plan has multiple steps, the corresponding steps from the two plans can also be stored in association as part of the corpus. Robot plans are modeled as finite state automata having a plurality of links representing sensory states and a plurality of nodes representing motor actions, and the determination of plan similarity is based on a comparison of the two FSAs.

Airborne anxiety detection and remediation

Abstract: Embodiments include methods, systems, and computer program products for detection and remediation of anxiety. Aspects include receiving an anxiety indicator. Aspects also include analyzing the anxiety indicator to determine whether an anxiety level exceeds an anxiety threshold. Aspects also include, based upon a determination that the anxiety level exceeds the anxiety threshold, selecting a first sound. Aspects also include outputting the first sound. Aspects also include receiving an anxiety feedback. Aspects also include determining, based upon the anxiety feedback, whether the anxiety level is decreasing.