Systems incorporating biometric technologies have become ubiquitous in personal, commercial, and governmental identity management applications. Both cooperative (e.g., access control) and noncooperative (e.g., surveillance and forensics) systems have benefited from biometrics. Such systems rely on the uniqueness of certain biological or behavioral characteristics of human beings, which enable for individuals to be reliably recognized using automated algorithms. Recently, however, there has been a wave of public and academic concerns regarding the existence of systemic bias in automated decision systems (including biometrics). Most prominently, face recognition algorithms have often been labeled as “racist” or “biased” by the media, nongovernmental organizations, and researchers alike. The main contributions of this article are: 1) an overview of the topic of algorithmic bias in the context of biometrics; 2) a comprehensive survey of the existing literature on biometric bias estimation and mitigation; 3) a discussion of the pertinent technical and social matters; and 4) an outline of the remaining challenges and future work items, both from technological and social points of view.

Demographic Bias in Biometrics: A Survey on an Emerging Challenge

When training a deep neural network for image classification, one can broadly distinguish between two types of latent features of images that will drive the classification. We can divide latent features into (i) ‘core’ or ‘conditionally invariant’ features $$C$$
 whose distribution $$C\vert Y$$
 , conditional on the class Y, does not change substantially across domains and (ii) ‘style’ features $$S$$
 whose distribution $$S\vert Y$$
 can change substantially across domains. Examples for style features include position, rotation, image quality or brightness but also more complex ones like hair color, image quality or posture for images of persons. Our goal is to minimize a loss that is robust under changes in the distribution of these style features. In contrast to previous work, we assume that the domain itself is not observed and hence a latent variable. We do assume that we can sometimes observe a typically discrete identifier or “
 $$\mathrm {ID}$$
 variable”. In some applications we know, for example, that two images show the same person, and $$\mathrm {ID}$$
 then refers to the identity of the person. The proposed method requires only a small fraction of images to have $$\mathrm {ID}$$
 information. We group observations if they share the same class and identifier $$(Y,\mathrm {ID})=(y,\mathrm {id})$$
 and penalize the conditional variance of the prediction or the loss if we condition on $$(Y,\mathrm {ID})$$
 . Using a causal framework, this conditional variance regularization (CoRe) is shown to protect asymptotically against shifts in the distribution of the style variables in a partially linear structural equation model. Empirically, we show that the CoRe penalty improves predictive accuracy substantially in settings where domain changes occur in terms of image quality, brightness and color while we also look at more complex changes such as changes in movement and posture.

/pdf/conditional-variance-penalties-and-domain-shift-robustness-4rxs9wf3vo.pdf

Conditional variance penalties and domain shift robustness

Human hand not only possesses distinctive feature for gender information, it is also considered one of the primary biometric traits used to identify a person. Unlike face images, which are usually unconstrained, an advantage of hand images is they are usually captured under a controlled position. Most state-of-the-art methods, that rely on hand images for gender recognition or biometric identification, employ handcrafted features to train an off-the-shelf classifier or be used by a similarity metric for biometric identification. In this work, we propose a deep learning-based method to tackle the gender recognition and biometric identification problems. Specifically, we design a two-stream convolutional neural network (CNN) which accepts hand images as input and predicts gender information from these hand images. This trained model is then used as a feature extractor to feed a set of support vector machine classifiers for biometric identification. As part of this effort, we propose a large dataset of human hand images, 11K Hands, which contains dorsal and palmar sides of human hand images with detailed ground-truth information for different problems including gender recognition and biometric identification. By leveraging thousands of hand images, we could effectively train our CNN-based model achieving promising results. One of our findings is that the dorsal side of human hands is found to have effective distinctive features similar to, if not better than, those available in the palmar side of human hand images. To facilitate access to our 11K Hands dataset, the dataset, the trained CNN models, and our Matlab source code are available at (
 https://goo.gl/rQJndd
 ).

11K Hands: Gender recognition and biometric identification using a large dataset of hand images

The human hand possesses distinctive features which can reveal gender information. In addition, the hand is considered one of the primary biometric traits used to identify a person. In this work, we propose a large dataset of human hand images with detailed ground-truth information for gender recognition and biometric identification. The proposed dataset comprises of 11,076 hand images (dorsal and palmar sides), from 190 subjects of different ages under the same lighting conditions. Using this dataset, a convolutional neural network (CNN) can be trained effectively for the gender recognition task. Based on this, we design a two-stream CNN to tackle the gender recognition problem. This trained model is then used as a feature extractor to feed a set of support vector machine classifiers for the biometric identification task. To facilitate access to the proposed dataset and replication of our experiments, the dataset, trained CNN models, and Matlab source code are available at (this https URL).

Gender recognition and biometric identification using a large dataset of hand images.

When training a deep neural network for image classification, one can broadly distinguish between two types of latent features of images that will drive the classification. We can divide latent features into (i) "core" or "conditionally invariant" features $X^\text{core}$ whose distribution $X^\text{core}\vert Y$, conditional on the class $Y$, does not change substantially across domains and (ii) "style" features $X^{\text{style}}$ whose distribution $X^{\text{style}} \vert Y$ can change substantially across domains. Examples for style features include position, rotation, image quality or brightness but also more complex ones like hair color, image quality or posture for images of persons. Our goal is to minimize a loss that is robust under changes in the distribution of these style features. In contrast to previous work, we assume that the domain itself is not observed and hence a latent variable. 
We do assume that we can sometimes observe a typically discrete identifier or "$\mathrm{ID}$ variable". In some applications we know, for example, that two images show the same person, and $\mathrm{ID}$ then refers to the identity of the person. The proposed method requires only a small fraction of images to have $\mathrm{ID}$ information. We group observations if they share the same class and identifier $(Y,\mathrm{ID})=(y,\mathrm{id})$ and penalize the conditional variance of the prediction or the loss if we condition on $(Y,\mathrm{ID})$. Using a causal framework, this conditional variance regularization (CoRe) is shown to protect asymptotically against shifts in the distribution of the style variables. Empirically, we show that the CoRe penalty improves predictive accuracy substantially in settings where domain changes occur in terms of image quality, brightness and color while we also look at more complex changes such as changes in movement and posture.

Conditional Variance Penalties and Domain Shift Robustness

The adoption of large-scale iris recognition systems around the world has brought to light the importance of detecting presentation attack images (textured contact lenses and printouts). This paper presents a new approach in iris presentation attack detection (PAD) by exploring combinations of convolutional neural networks (CNNs) and transformed input spaces through binarized statistical image features (BSIFs). Our method combines lightweight CNNs to classify multiple BSIF views of the input image. Following explorations on complementary input spaces leading to more discriminative features to detect presentation attacks, we also propose an algorithm to select the best (and most discriminative) predictors for the task at hand. An ensemble of predictors makes use of their expected individual performances to aggregate their results into a final prediction. Results show that this technique improves on the current state of the art in iris PAD, outperforming the winner of LivDet-Iris 2017 competition both for intra- and cross-dataset scenarios, and illustrating the very difficult nature of the cross-dataset scenario.

Ensemble of Multi-View Learning Classifiers for Cross-Domain Iris Presentation Attack Detection

Predicting a person's gender based on the iris texture has been explored by several researchers. This paper considers several dimensions of experimental work on this problem, including person-disjoint train and test, and the effect of cosmetics on eyelash occlusion and imperfect segmentation. We also consider the use of multi-layer perceptron and convolutional neural networks as classifiers, comparing the use of data-driven and hand-crafted features. Our results suggest that the gender-from-iris problem is more difficult than has so far been appreciated. Estimating accuracy using a mean of N person-disjoint train and test partitions, and considering the effect of makeup - a combination of experimental conditions not present in any previous work - we find a much weaker ability to predict genderfrom-iris texture than has been suggested in previous work.

/pdf/gender-from-iris-or-gender-from-mascara-4sgtriq513.pdf

Gender-from-Iris or Gender-from-Mascara?

The adoption of large-scale iris recognition systems around the world has brought to light the importance of detecting presentation attack images (textured contact lenses and printouts). This work presents a new approach in iris Presentation Attack Detection (PAD), by exploring combinations of Convolutional Neural Networks (CNNs) and transformed input spaces through binarized statistical image features (BSIF). Our method combines lightweight CNNs to classify multiple BSIF views of the input image. Following explorations on complementary input spaces leading to more discriminative features to detect presentation attacks, we also propose an algorithm to select the best (and most discriminative) predictors for the task at hand.An ensemble of predictors makes use of their expected individual performances to aggregate their results into a final prediction. Results show that this technique improves on the current state of the art in iris PAD, outperforming the winner of LivDet-Iris2017 competition both for intra- and cross-dataset scenarios, and illustrating the very difficult nature of the cross-dataset scenario.

Post-mortem biometrics entails utilizing the biometric data of a deceased individual for determining or verifying human identity. Due to fundamental biological changes that occur in a person’s biometric traits after death, post-mortem data can be significantly different from ante-mortem data, introducing new challenges for biometric sensors, feature extractors and matchers. This paper surveys research to date on the problem of using iris images acquired after death for automated human recognition. A comprehensive review of existing literature is complemented by a summary of the most recent results and observations offered in these publications. This survey is unique in several elements. Firstly, it is the first publication to consider iris recognition where gallery images are acquired before death (peri-mortem images) and the probe images are acquired after death from the same subjects. Secondly, results are presented from the largest database of peri-mortem and post-mortem iris images, collected from 213 subjects by two independent institutions located in the U.S. and Poland. Thirdly, post-mortem recognition viability is assessed using more than 20 iris recognition algorithms, ranging from the classic (e.g., Gabor filtering-based) to the modern (e.g., deep learning-based). Finally, we provide a medically informed commentary on post-mortem iris, analyze the reasons for recognition failures, and identify key directions for future research.

/pdf/post-mortem-iris-recognition-a-survey-and-assessment-of-the-1uqw2k9ao6.pdf

Post-Mortem Iris Recognition—A Survey and Assessment of the State of the Art

Predicting gender from iris images has been reported by several researchers as an application of machine learning in biometrics. Recent works on this topic have suggested that the preponderance of the gender cues is located in the periocular region rather than in the iris texture itself. This paper focuses on teasing out whether the information for gender prediction is in the texture of the iris stroma, the periocular region, or both. We present a larger dataset for gender from iris, and evaluate gender prediction accuracy using linear SVM and CNN, comparing hand-crafted and deep features. We use probabilistic occlusion masking to gain insight on the problem. Results suggest the discriminative power of the iris texture for gender is weaker than previously thought, and that the gender-related information is primarily in the periocular region.

Andrey Kuehlkamp

Papers

Ensemble of Multi-View Learning Classifiers for Cross-Domain Iris Presentation Attack Detection

Gender-from-Iris or Gender-from-Mascara?

Ensemble of Multi-View Learning Classifiers for Cross-Domain Iris Presentation Attack Detection

Post-Mortem Iris Recognition—A Survey and Assessment of the State of the Art

Predicting Gender From Iris Texture May Be Harder Than It Seems