A survey on deep learning based face recognition

Racial bias is an important issue in biometric, but has not been thoroughly studied in deep face recognition. In this paper, we first contribute a dedicated dataset called Racial Faces in-the-Wild (RFW) database, on which we firmly validated the racial bias of four commercial APIs and four state-of-the-art (SOTA) algorithms. Then, we further present the solution using deep unsupervised domain adaptation and propose a deep information maximization adaptation network (IMAN) to alleviate this bias by using Caucasian as source domain and other races as target domains. This unsupervised method simultaneously aligns global distribution to decrease race gap at domain-level, and learns the discriminative target representations at cluster level. A novel mutual information loss is proposed to further enhance the discriminative ability of network output without label information. Extensive experiments on RFW, GBU, and IJB-A databases show that IMAN successfully learns features that generalize well across different races and across different databases.

/pdf/racial-faces-in-the-wild-reducing-racial-bias-by-information-28wpjqufeq.pdf

Racial Faces in the Wild: Reducing Racial Bias by Information Maximization Adaptation Network

Systems incorporating biometric technologies have become ubiquitous in personal, commercial, and governmental identity management applications. Both cooperative (e.g., access control) and noncooperative (e.g., surveillance and forensics) systems have benefited from biometrics. Such systems rely on the uniqueness of certain biological or behavioral characteristics of human beings, which enable for individuals to be reliably recognized using automated algorithms. Recently, however, there has been a wave of public and academic concerns regarding the existence of systemic bias in automated decision systems (including biometrics). Most prominently, face recognition algorithms have often been labeled as “racist” or “biased” by the media, nongovernmental organizations, and researchers alike. The main contributions of this article are: 1) an overview of the topic of algorithmic bias in the context of biometrics; 2) a comprehensive survey of the existing literature on biometric bias estimation and mitigation; 3) a discussion of the pertinent technical and social matters; and 4) an outline of the remaining challenges and future work items, both from technological and social points of view.

Demographic Bias in Biometrics: A Survey on an Emerging Challenge

A comprehensive overview of biometric fusion

Drones, or general UAVs, equipped with cameras have been fast deployed with a wide range of applications, including agriculture, aerial photography, and surveillance. Consequently, automatic understanding of visual data collected from drones becomes highly demanding, bringing computer vision and drones more and more closely. To promote and track the evelopments of object detection and tracking algorithms, we have organized two challenge workshops in conjunction with ECCV 2018, and ICCV 2019, attracting more than 100 teams around the world. We provide a large-scale drone captured dataset, VisDrone, which includes four tracks, i.e., (1) image object detection, (2) video object detection, (3) single object tracking, and (4) multi-object tracking. In this paper, we first presents a thorough review of object detection and tracking datasets and benchmarks, and discuss the challenges of collecting large-scale drone-based object detection and tracking datasets with fully manual annotations. After that, we describe our VisDrone dataset, which is captured over various urban/suburban areas of 14 different cities across China from North to South. Being the largest such dataset ever published, VisDrone enables extensive evaluation and investigation of visual analysis algorithms on the drone platform. We provide a detailed analysis of the current state of the field of large-scale object detection and tracking on drones, and conclude the challenge as well as propose future directions. We expect the benchmark largely boost the research and development in video analysis on drone platforms. All the datasets and experimental results can be downloaded from the website: this https URL.

Vision Meets Drones: Past, Present and Future

Do very high accuracies of deep networks suggest pride of effective AI or are deep networks prejudiced? Do they suffer from in-group biases (own-race-bias and own-age-bias), and mimic the human behavior? Is in-group specific information being encoded sub-consciously by the deep networks? 
This research attempts to answer these questions and presents an in-depth analysis of `bias' in deep learning based face recognition systems This is the first work which decodes if and where bias is encoded for face recognition Taking cues from cognitive studies, we inspect if deep networks are also affected by social in- and out-group effect Networks are analyzed for own-race and own-age bias, both of which have been well established in human beings The sub-conscious behavior of face recognition models is examined to understand if they encode race or age specific features for face recognition Analysis is performed based on 36 experiments conducted on multiple datasets Four deep learning networks either trained from scratch or pre-trained on over 10M images are used Variations across class activation maps and feature visualizations provide novel insights into the functioning of deep learning systems, suggesting behavior similar to humans It is our belief that a better understanding of state-of-the-art deep learning networks would enable researchers to address the given challenge of bias in AI, and develop fairer systems

Deep Learning for Face Recognition: Pride or Prejudiced?

Face recognition algorithms have demonstrated very high recognition performance, suggesting suitability for real world applications Despite the enhanced accuracies, robustness of these algorithms against attacks and bias has been challenged This paper summarizes different ways in which the robustness of a face recognition algorithm is challenged, which can severely affect its intended working Different types of attacks such as physical presentation attacks, disguise/makeup, digital adversarial attacks, and morphing/tampering using GANs have been discussed We also present a discussion on the effect of bias on face recognition models and showcase that factors such as age and gender variations affect the performance of modern algorithms The paper also presents the potential reasons for these challenges and some of the future research directions for increasing the robustness of face recognition models

/pdf/on-the-robustness-of-face-recognition-algorithms-against-4pm6ra3h9o.pdf

On the Robustness of Face Recognition Algorithms Against Attacks and Bias

Unmanned Aerial Vehicles (UAVs) or drones are often used to reach remote areas or regions which are inaccessible to humans. Equipped with a large field of view, compact size, and remote control abilities, drones are deemed suitable for monitoring crowded or disaster-hit areas, and performing aerial surveillance. While research has focused on area monitoring, object detection and tracking, limited attention has been given to person identification, especially face recognition, using drones. This research presents a novel large-scale drone dataset, DroneSURF: Drone Surveillance of Faces, in order to facilitate research for face recognition. The dataset contains 200 videos of 58 subjects, captured across 411K frames, having over 786K face annotations. The proposed dataset demonstrates variations across two surveillance use cases: (i) active and (ii) passive, two locations, and two acquisition times. DroneSURF encapsulates challenges due to the effect of motion, variations in pose, illumination, background, altitude, and resolution, especially due to the large and varying distance between the drone and the subjects. This research presents a detailed description of the proposed DroneSURF dataset, along with information regarding the data distribution, protocols for evaluation, and baseline results.

DroneSURF: Benchmark Dataset for Drone-based Face Recognition

Body weight variations are an integral part of a person’s aging process. However, the lack of association between the age and the weight of an individual makes it challenging to model these variations for automatic face recognition. In this paper, we propose a regularizer-based approach to learn weight invariant facial representations using two different deep learning architectures, namely, sparse-stacked denoising autoencoders and deep Boltzmann machines. We incorporate a body-weight aware regularization parameter in the loss function of these architectures to help learn weight-aware features. The experiments performed on the extended WIT database show that the introduction of weight aware regularization improves the identification accuracy of the architectures both with and without dropout.

Regularized Deep Learning for Face Recognition With Weight Variations

Enhancing low resolution images via super-resolution or synthesis algorithms for cross-resolution face recognition has been well studied. Several image processing and machine learning paradigms have been explored for addressing the same. In this research, we propose Synthesis via Hierarchical Sparse Representation (SHSR) algorithm for synthesizing a high resolution face image from a low resolution input image. The proposed algorithm learns multilevel sparse representation for both high and low resolution gallery images, along with identity aware dictionaries and a transformation function between the two representations for face identification scenarios. With low resolution test data as input, a high resolution test image is synthesized using the identity aware dictionaries and transformation, which is then used for face recognition. The performance of the proposed SHSR algorithm is evaluated on four datasets, including one real world dataset. Experimental results and comparison with seven existing algorithms demonstrate the efficacy of the proposed algorithm in terms of both face identification and image quality measures.

/pdf/identity-aware-synthesis-for-cross-resolution-face-268v7aelcb.pdf

Shruti Nagpal

Papers

Deep Learning for Face Recognition: Pride or Prejudiced?

On the Robustness of Face Recognition Algorithms Against Attacks and Bias

DroneSURF: Benchmark Dataset for Drone-based Face Recognition

Regularized Deep Learning for Face Recognition With Weight Variations

Identity Aware Synthesis for Cross Resolution Face Recognition