Can holistic representations be used for face biometric quality assessment
01 Sep 2013-pp 2792-2796
TL;DR: This paper investigates the use of holistic super-ordinate representations, namely, Gist and sparsely pooled Histogram of Orientated Gradient, in classifying images into different quality categories that are derived from matching performance.
Abstract: A face quality metric must quantitatively measure the usability of an image as a biometric sample. Though it is well established that quality measures are an integral part of robust face recognition systems, automatic measurement of bio-metric quality in face is still challenging. Inspired by scene recognition research, this paper investigates the use of holistic super-ordinate representations, namely, Gist and sparsely pooled Histogram of Orientated Gradient (HOG), in classifying images into different quality categories that are derived from matching performance. The experiments on the CAS-PEAL and SCFace databases containing covariates such as illumination, expression, pose, low-resolution and occlusion by accessories, suggest that the proposed algorithm can efficiently classify input face image into relevant quality categories and be utilized in face recognition systems.
Citations
More filters
TL;DR: The analysis of the characteristic function of quality and match scores shows that a careful selection of complimentary set of quality metrics can provide more benefit to various applications of biometric quality.
Abstract: Biometric systems encounter variability in data that influence capture, treatment, and u-sage of a biometric sample. It is imperative to first analyze the data and incorporate this understanding within the recognition system, making assessment of biometric quality an important aspect of biometrics. Though several interpretations and definitions of quality exist, sometimes of a conflicting nature, a holistic definition of quality is indistinct. This paper presents a survey of different concepts and interpretations of biometric quality so that a clear picture of the current state and future directions can be presented. Several factors that cause different types of degradations of biometric samples, including image features that attribute to the effects of these degradations, are discussed. Evaluation schemes are presented to test the performance of quality metrics for various applications. A survey of the features, strengths, and limitations of existing quality assessment techniques in fingerprint, iris, and face biometric are also presented. Finally, a representative set of quality metrics from these three modalities are evaluated on a multimodal database consisting of 2D images, to understand their behavior with respect to match scores obtained from the state-of-the-art recognition systems. The analysis of the characteristic function of quality and match scores shows that a careful selection of complimentary set of quality metrics can provide more benefit to various applications of biometric quality.
119 citations
Cites background from "Can holistic representations be use..."
...Recently, holistic descriptors extracted from the face region are shown to be good indicators of performance of face recognition systems [87]....
[...]
TL;DR: This is the first work to utilize human assessments of face image quality in designing a predictor of unconstrained face quality that is shown to be effective in cross-database evaluation.
Abstract: Face image quality can be defined as a measure of the utility of a face image to automatic face recognition. In this paper, we propose (and compare) two methods for learning face image quality based on target face quality values from: 1) human assessments of face image quality (matcher-independent) and 2) quality values computed from similarity scores (matcher-dependent). A support vector regression model trained on face features extracted using a deep convolutional neural network (ConvNet) is used to predict the quality of a face image. The proposed methods are evaluated on two unconstrained face image databases, Labeled Faces in the Wild and IARPA Janus Benchmark-A (IJB-A), which both contain facial variations encompassing a multitude of quality factors. Evaluation of the proposed automatic face image quality measures shows we are able to reduce the false non-match rate at 1% false match rate by at least 13% for two face matchers (a commercial off-the-shelf matcher and a ConvNet matcher) by using the proposed face quality to select subsets of face images and video frames for matching templates (i.e., multiple faces per subject) in the IJB-A protocol. To the best of our knowledge, this is the first work to utilize human assessments of face image quality in designing a predictor of unconstrained face quality that is shown to be effective in cross-database evaluation.
79 citations
Posted Content•
TL;DR: This survey provides an overview of the face image quality assessment literature, which predominantly focuses on visible wavelength face image input and a trend towards deep learning based methods is observed, including notable conceptual differences among the recent approaches.
Abstract: The performance of face analysis and recognition systems depends on the quality of the acquired face data, which is influenced by numerous factors. Automatically assessing the quality of face data in terms of biometric utility can thus be useful to filter out low quality data. This survey provides an overview of the face quality assessment literature in the framework of face biometrics, with a focus on face recognition based on visible wavelength face images as opposed to e.g. depth or infrared quality assessment. A trend towards deep learning based methods is observed, including notable conceptual differences among the recent approaches. Besides image selection, face image quality assessment can also be used in a variety of other application scenarios, which are discussed herein. Open issues and challenges are pointed out, i.a. highlighting the importance of comparability for algorithm evaluations, and the challenge for future work to create deep learning approaches that are interpretable in addition to providing accurate utility predictions.
51 citations
Cites background or methods from "Can holistic representations be use..."
...LFW [79] 2011 to 2021 17: B [181][9][23] E [113][198][120][46][47] [25][123][140][24][73][22][176][166][193] FERET [146] 2007 to 2020 9: B [3][76][23][189] C [176] E [2][1][120] [160] VGGFace2 [21] 2019 to 2021 7: B [73][74] C [191] E [46][47][24][193] CASIA-WebFace [197] 2017 to 2021 7: B [198][204][166] C [140][9] E [113][193] CAS-PEAL [55] 2009 to 2018 6: B [2][76][17] C [1][184] E [201] FRGC [145] 2006 to 2018 6: B [102][101][75][23] C [152][184] MS-Celeb-1M [68] 2019 to 2020 5: B [166] C [22][176][193] E [113] CFP [164] 2019 to 2021 5: E [25][123][24][22][166] IJB-C [122] 2019 to 2021 5: E [123][140][191][22][166] YTF [188] 2014 to 2020 5: B [136] E [198][30][22][166] MS1MV2 [34] 2021 4: C [25][123][140][24] IJB-A [105] 2017 to 2019 4: B [113] C [193] E [166][9] ChokePoint [189] 2011 to 2018 4: B [150][179] E [184][189] SCface [64] 2011 to 2018 4: B [17] E [120][184][23] Extended Yale [110] 2010 to 2018 4: B [153][151][163] C [184] CPLFW [205] 2021 3: E [25][123][24] IJB-B [187] 2021 3: E [25][123][24] Adience [40] 2020 to 2021 3: E [25][140][176] BioSecure [139] 2019 to 2021 3: E [46][73][74] GBU [143] 2012 to 2014 3: B [2][1] E [142] AT&T [158] 2010 to 2016 3: B [76][163] C [152] CMU-PIE [167] 2009 to 2011 3: C [11] E [160][189] FRVT 2006 [147] 2008 to 2010 3: E [11][13][14] Yale [57] 2007 to 2014 3: B [2][1] E [56] BANCA [8] 2006 to 2008 3: B [108][109] E [156] AgeDB [129] 2021 2: E [123][24] CALFW [206] 2021 2: E [123][24] MEDS-II [44] 2019 to 2020 2: B [155][154] MegaFace [99] 2019 to 2020 2: E [22][166] AR [121] 2014 to 2018 2: C [152][184] PaSC [12] 2013 to 2018 2: B [150] E [142] MBGC [144] 2012 to 2014 2: E [2][1] Q-FIRE [95] 2012 to 2014 2: E [2][77]...
[...]
...[17] trained a one-vs-all SVM for 4 quality bins using either sparsely pooled Histogram of Oriented Gradient (HOG) or Gist [137] input features....
[...]
...• FR-based ground truth training (Dfrt): These approaches obtained training data from FRmodels [17][179][76][9] [150][74][73][191][140][25]....
[...]
...2013 [17] Dfrt 4-class SVM on Gist[137] or HOG....
[...]
...Some of the works evaluated FIQA performance exclusively by means other than the ERC - for example, FR performance was evaluated for 4 FIQA-derived quality bins in [17]....
[...]
TL;DR: This paper considered five categories of common homogeneous distortion in video suvillance applications, i.e. low-resolution, blurring, additive Gaussian white noise, salt and pepper noise, and Poisson noise and proposed a novel biometric quality assessment (BQA) method for face images and explored its applications in face recognition.
31 citations
Journal Article•
TL;DR: This paper proposes an approach for standardization of facial image quality, and develops facial symmetry based methods for the assessment of it by measuring facial asymmetries caused by non-frontal lighting and improper facial pose.
Abstract: Performance of biometric systems is dependent on quality of acquired biometric samples. Poor sample quality is a main reason for matching errors in biometric systems and may be the main weakness of some implementations. This paper proposes an approach for standardization of facial image quality, and develops facial symmetry based methods for the assessment of it by measuring facial asymmetries caused by non-frontal lighting and improper facial pose. Experimental results are provided to illustrate the concepts, definitions and effectiveness.
28 citations
References
More filters
TL;DR: This paper develops a new approach to understand and measure variations in biometric sample quality with limited data samples by introducing an algorithm which regularizes a Gaussian model of the feature covariances.
Abstract: This paper develops a new approach to understand and measure variations in biometric sample quality We begin with the intuition that degradations to a biometric sample will reduce the amount of identifiable information available In order to measure the amount of identifiable information, we define biometric information as the decrease in uncertainty about the identity of a person due to a set of biometric measurements We then show that the biometric information for a person may be calculated by the relative entropy D(p||q) between the population feature distribution q and the person's feature distribution p The biometric information for a system is the mean D(p||q) for all persons in the population In order to practically measure D(p||q) with limited data samples, we introduce an algorithm which regularizes a Gaussian model of the feature covariances An example of this method is shown for PCA, Fisher linear discriminant (FLD) and ICA based face recognition, with biometric information calculated to be 450 bits (PCA), 370 bits (FLD), 390 bits (ICA) and 556 bits (fusion of PCA and FLD features) Based on this definition of biometric information, we simulate degradations of biometric images and calculate the resulting decrease in biometric information Results show a quasi-linear decrease for small levels of blur with an asymptotic behavior at larger blur
26 citations
"Can holistic representations be use..." refers background in this paper
...[9] R. Youmaran and A. Adler, “Measuring biometric sample qual- ity in terms of biometric information,” in Proceedings of Biometric Consortium, 2006, pp. 1–6....
[...]
...Youmaran and Adler [9] Biometric information defined from information theory....
[...]
TL;DR: Experimental results on different multi-modal databases involving face and fingerprint show that the proposed quality-based classifier selection framework yields good performance even when the quality of the biometric sample is sub-optimal.
Abstract: Multibiometric systems fuse the evidence (e.g., match scores) pertaining to multiple biometric modalities or classifiers. Most score-level fusion schemes discussed in the literature require the processing (i.e., feature extraction and matching) of every modality prior to invoking the fusion scheme. This paper presents a framework for dynamic classifier selection and fusion based on the quality of the gallery and probe images associated with each modality with multiple classifiers. The quality assessment algorithm for each biometric modality computes a quality vector for the gallery and probe images that is used for classifier selection. These vectors are used to train Support Vector Machines (SVMs) for decision making. In the proposed framework, the biometric modalities are arranged sequentially such that the stronger biometric modality has higher priority for being processed. Since fusion is required only when all unimodal classifiers are rejected by the SVM classifiers, the average computational time of the proposed framework is significantly reduced. Experimental results on different multi-modal databases involving face and fingerprint show that the proposed quality-based classifier selection framework yields good performance even when the quality of the biometric sample is sub-optimal.
20 citations
"Can holistic representations be use..." refers background in this paper
...[17] present evidence indicating that a comprehensive quality measure must be a vector rather than a scalar....
[...]
01 Jan 2007
TL;DR: Results validate a relative improvement of up to 26% in FRR after assessment and restoration of high magnification face images and Magnification blur proves to be a major degradation source for face recognition and is addressed via blur assessment and deblurring algorithms.
Abstract: Most existing face related research is restricted to close range applications with low and constant system magnifications (camera zoom). To improve the performance of face recognition algorithms in wide area surveillance applications, we initiate a study regarding the effects of increased system magnifications and observation distances on face recognition rates (FRR). We first describe a new face video database including still face images and video sequences from long distances (indoor: 10m-20m and outdoor: 50m-300m). The corresponding system magnification is elevated from less than 3× to 20× for indoor and up to 375× for outdoor. Deteriorations unique to high magnification and long range face images are investigated. Magnification blur proves to be a major degradation source for face recognition and is addressed via blur assessment and deblurring algorithms. Experimental results validate a relative improvement of up to 26% in FRR after assessment and restoration of high magnification face images.
9 citations
Additional excerpts
...[14] Sharpness measure for frame selection....
[...]
13 May 2011
TL;DR: A general Bayesian framework that can utilize the quality measures effectively in multiple classifier combination of multimodal biometrics is proposed, leading to more efficient implementation and achieving performance comparable to, or better than the state of the art.
Abstract: This paper proposes a unified framework for quality-based fusion of multimodal biometrics. Quality- dependent fusion algorithms aim to dynamically combine several classifier (biometric expert) outputs as a function of automatically derived (biometric) sample quality. Quality measures used for this purpose quantify the degree of conformance of biometric samples to some predefined criteria known to influence the system performance. Designing a fusion classifier to take quality into consideration is difficult because quality measures cannot be used to distinguish genuine users from impostors, i.e., they are non- discriminative; yet, still useful for classification. We propose a general Bayesian framework that can utilize the quality infor- mation effectively. We show that this framework encompasses several recently proposed quality-based fusion algorithms in the literature -- Nandakumar et al., 2006; Poh et al., 2007; Kryszczuk and Drygajo, 2007; Kittler et al., 2007; Alonso- Fernandez, 2008; Maurer and Baker, 2007; Poh et al., 2010. Furthermore, thanks to the systematic study concluded herein, we also develop two alternative formulations of the problem, leading to more efficient implementation (with fewer parameters) and achieving performance comparable to, or better than the state of the art. Last but not least, the framework also improves the understanding of the role of quality in multiple classifier combination.
7 citations
"Can holistic representations be use..." refers background in this paper
...Further, considerable research on leveraging simple quality metrics to improve multibiometrics recognition is summarized in [7]....
[...]