Can holistic representations be used for face biometric quality assessment

doi:10.1109/ICIP.2013.6738575

Home
/
Papers
/
Can holistic representations be used for face biometric quality assessment

Proceedings Article•DOI•

Can holistic representations be used for face biometric quality assessment

Samarth Bharadwaj¹, Mayank Vatsa¹, Richa Singh¹•Institutions (1)

Indraprastha Institute of Information Technology¹

01 Sep 2013-pp 2792-2796

TL;DR: This paper investigates the use of holistic super-ordinate representations, namely, Gist and sparsely pooled Histogram of Orientated Gradient, in classifying images into different quality categories that are derived from matching performance.

read less

Abstract: A face quality metric must quantitatively measure the usability of an image as a biometric sample. Though it is well established that quality measures are an integral part of robust face recognition systems, automatic measurement of bio-metric quality in face is still challenging. Inspired by scene recognition research, this paper investigates the use of holistic super-ordinate representations, namely, Gist and sparsely pooled Histogram of Orientated Gradient (HOG), in classifying images into different quality categories that are derived from matching performance. The experiments on the CAS-PEAL and SCFace databases containing covariates such as illumination, expression, pose, low-resolution and occlusion by accessories, suggest that the proposed algorithm can efficiently classify input face image into relevant quality categories and be utilized in face recognition systems.

...read moreread less

Citations

PDF

Open Access

More filters

Journal Article•DOI•

Biometric quality: a review of fingerprint, iris, and face

[...]

Samarth Bharadwaj¹, Mayank Vatsa¹, Richa Singh¹•Institutions (1)

Indian Institutes of Information Technology¹

02 Jul 2014-Eurasip Journal on Image and Video Processing

TL;DR: The analysis of the characteristic function of quality and match scores shows that a careful selection of complimentary set of quality metrics can provide more benefit to various applications of biometric quality.

...read moreread less

Abstract: Biometric systems encounter variability in data that influence capture, treatment, and u-sage of a biometric sample. It is imperative to first analyze the data and incorporate this understanding within the recognition system, making assessment of biometric quality an important aspect of biometrics. Though several interpretations and definitions of quality exist, sometimes of a conflicting nature, a holistic definition of quality is indistinct. This paper presents a survey of different concepts and interpretations of biometric quality so that a clear picture of the current state and future directions can be presented. Several factors that cause different types of degradations of biometric samples, including image features that attribute to the effects of these degradations, are discussed. Evaluation schemes are presented to test the performance of quality metrics for various applications. A survey of the features, strengths, and limitations of existing quality assessment techniques in fingerprint, iris, and face biometric are also presented. Finally, a representative set of quality metrics from these three modalities are evaluated on a multimodal database consisting of 2D images, to understand their behavior with respect to match scores obtained from the state-of-the-art recognition systems. The analysis of the characteristic function of quality and match scores shows that a careful selection of complimentary set of quality metrics can provide more benefit to various applications of biometric quality.

...read moreread less

119 citations

Cites background from "Can holistic representations be use..."

...Recently, holistic descriptors extracted from the face region are shown to be good indicators of performance of face recognition systems [87]....
[...]

Journal Article•DOI•

Learning Face Image Quality From Human Assessments

[...]

Lacey Best-Rowden¹, Anil K. Jain¹•Institutions (1)

Michigan State University¹

30 Jan 2018-IEEE Transactions on Information Forensics and Security

TL;DR: This is the first work to utilize human assessments of face image quality in designing a predictor of unconstrained face quality that is shown to be effective in cross-database evaluation.

...read moreread less

Abstract: Face image quality can be defined as a measure of the utility of a face image to automatic face recognition. In this paper, we propose (and compare) two methods for learning face image quality based on target face quality values from: 1) human assessments of face image quality (matcher-independent) and 2) quality values computed from similarity scores (matcher-dependent). A support vector regression model trained on face features extracted using a deep convolutional neural network (ConvNet) is used to predict the quality of a face image. The proposed methods are evaluated on two unconstrained face image databases, Labeled Faces in the Wild and IARPA Janus Benchmark-A (IJB-A), which both contain facial variations encompassing a multitude of quality factors. Evaluation of the proposed automatic face image quality measures shows we are able to reduce the false non-match rate at 1% false match rate by at least 13% for two face matchers (a commercial off-the-shelf matcher and a ConvNet matcher) by using the proposed face quality to select subsets of face images and video frames for matching templates (i.e., multiple faces per subject) in the IJB-A protocol. To the best of our knowledge, this is the first work to utilize human assessments of face image quality in designing a predictor of unconstrained face quality that is shown to be effective in cross-database evaluation.

...read moreread less

79 citations

Posted Content•

Face Image Quality Assessment: A Literature Survey

[...]

Torsten Schlett, Christian Rathgeb, Olaf Henniger, Javier Galbally, Julian Fierrez, Christoph Busch - Show less +2 more

02 Sep 2020-arXiv: Computer Vision and Pattern Recognition

TL;DR: This survey provides an overview of the face image quality assessment literature, which predominantly focuses on visible wavelength face image input and a trend towards deep learning based methods is observed, including notable conceptual differences among the recent approaches.

...read moreread less

Abstract: The performance of face analysis and recognition systems depends on the quality of the acquired face data, which is influenced by numerous factors. Automatically assessing the quality of face data in terms of biometric utility can thus be useful to filter out low quality data. This survey provides an overview of the face quality assessment literature in the framework of face biometrics, with a focus on face recognition based on visible wavelength face images as opposed to e.g. depth or infrared quality assessment. A trend towards deep learning based methods is observed, including notable conceptual differences among the recent approaches. Besides image selection, face image quality assessment can also be used in a variety of other application scenarios, which are discussed herein. Open issues and challenges are pointed out, i.a. highlighting the importance of comparability for algorithm evaluations, and the challenge for future work to create deep learning approaches that are interpretable in addition to providing accurate utility predictions.

...read moreread less

51 citations

Cites background or methods from "Can holistic representations be use..."

...LFW [79] 2011 to 2021 17: B [181][9][23] E [113][198][120][46][47] [25][123][140][24][73][22][176][166][193] FERET [146] 2007 to 2020 9: B [3][76][23][189] C [176] E [2][1][120] [160] VGGFace2 [21] 2019 to 2021 7: B [73][74] C [191] E [46][47][24][193] CASIA-WebFace [197] 2017 to 2021 7: B [198][204][166] C [140][9] E [113][193] CAS-PEAL [55] 2009 to 2018 6: B [2][76][17] C [1][184] E [201] FRGC [145] 2006 to 2018 6: B [102][101][75][23] C [152][184] MS-Celeb-1M [68] 2019 to 2020 5: B [166] C [22][176][193] E [113] CFP [164] 2019 to 2021 5: E [25][123][24][22][166] IJB-C [122] 2019 to 2021 5: E [123][140][191][22][166] YTF [188] 2014 to 2020 5: B [136] E [198][30][22][166] MS1MV2 [34] 2021 4: C [25][123][140][24] IJB-A [105] 2017 to 2019 4: B [113] C [193] E [166][9] ChokePoint [189] 2011 to 2018 4: B [150][179] E [184][189] SCface [64] 2011 to 2018 4: B [17] E [120][184][23] Extended Yale [110] 2010 to 2018 4: B [153][151][163] C [184] CPLFW [205] 2021 3: E [25][123][24] IJB-B [187] 2021 3: E [25][123][24] Adience [40] 2020 to 2021 3: E [25][140][176] BioSecure [139] 2019 to 2021 3: E [46][73][74] GBU [143] 2012 to 2014 3: B [2][1] E [142] AT&T [158] 2010 to 2016 3: B [76][163] C [152] CMU-PIE [167] 2009 to 2011 3: C [11] E [160][189] FRVT 2006 [147] 2008 to 2010 3: E [11][13][14] Yale [57] 2007 to 2014 3: B [2][1] E [56] BANCA [8] 2006 to 2008 3: B [108][109] E [156] AgeDB [129] 2021 2: E [123][24] CALFW [206] 2021 2: E [123][24] MEDS-II [44] 2019 to 2020 2: B [155][154] MegaFace [99] 2019 to 2020 2: E [22][166] AR [121] 2014 to 2018 2: C [152][184] PaSC [12] 2013 to 2018 2: B [150] E [142] MBGC [144] 2012 to 2014 2: E [2][1] Q-FIRE [95] 2012 to 2014 2: E [2][77]...
[...]
...[17] trained a one-vs-all SVM for 4 quality bins using either sparsely pooled Histogram of Oriented Gradient (HOG) or Gist [137] input features....
[...]
...• FR-based ground truth training (Dfrt): These approaches obtained training data from FRmodels [17][179][76][9] [150][74][73][191][140][25]....
[...]
...2013 [17] Dfrt 4-class SVM on Gist[137] or HOG....
[...]
...Some of the works evaluated FIQA performance exclusively by means other than the ERC - for example, FR performance was evaluated for 4 FIQA-derived quality bins in [17]....
[...]

Journal Article•DOI•

Face biometric quality assessment via light CNN

[...]

Jun Yu¹, Kejia Sun¹, Fei Gao¹, Suguo Zhu¹•Institutions (1)

Hangzhou Dianzi University¹

01 Aug 2017-Pattern Recognition Letters

TL;DR: This paper considered five categories of common homogeneous distortion in video suvillance applications, i.e. low-resolution, blurring, additive Gaussian white noise, salt and pepper noise, and Poisson noise and proposed a novel biometric quality assessment (BQA) method for face images and explored its applications in face recognition.

...read moreread less

31 citations

Journal Article•

Standardization of Face Image Sample Quality

[...]

LI Zi-qing

01 Jan 2009-Journal of Chinese Computer Systems

TL;DR: This paper proposes an approach for standardization of facial image quality, and develops facial symmetry based methods for the assessment of it by measuring facial asymmetries caused by non-frontal lighting and improper facial pose.

...read moreread less

Abstract: Performance of biometric systems is dependent on quality of acquired biometric samples. Poor sample quality is a main reason for matching errors in biometric systems and may be the main weakness of some implementations. This paper proposes an approach for standardization of facial image quality, and develops facial symmetry based methods for the assessment of it by measuring facial asymmetries caused by non-frontal lighting and improper facial pose. Experimental results are provided to illustrate the concepts, definitions and effectiveness.

...read moreread less

28 citations

References

PDF

Open Access

More filters

Journal Article•DOI•

Performance of Biometric Quality Measures

[...]

Patrick J. Grother, Elham Tabassi

01 Apr 2007-IEEE Transactions on Pattern Analysis and Machine Intelligence

TL;DR: This work documents methods for the quantitative evaluation of systems that produce a scalar summary of a biometric sample's quality, motivated by a need to test claims that quality measures are predictive of matching performance.

...read moreread less

Abstract: We document methods for the quantitative evaluation of systems that produce a scalar summary of a biometric sample's quality. We are motivated by a need to test claims that quality measures are predictive of matching performance. We regard a quality measurement algorithm as a black box that converts an input sample to an output scalar. We evaluate it by quantifying the association between those values and observed matching results. We advance detection error trade-off and error versus reject characteristics as metrics for the comparative evaluation of sample quality measurement algorithms. We proceed this with a definition of sample quality, a description of the operational use of quality measures. We emphasize the performance goal by including a procedure for annotating the samples of a reference corpus with quality values derived from empirical recognition scores

...read moreread less

338 citations

"Can holistic representations be use..." refers background or methods in this paper

...As shown by Grother and Tabassi [15], there is a relationship between quality of a biometric sample and recognition accuracy....
[...]
...To evaluate the correctness of quality labels, the identification and verification performance of each bin are computed separately using the better performing COTS, similar to the experimental procedure in [15]....
[...]
...[15] G. Patrick and E. Tabassi, “Performance of biometric quality measures,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 29, no. 4, pp. 531–524, 2007....
[...]
...The training samples are annotated based on the identification performance on the training set, inspired from [15]....
[...]

Proceedings Article•DOI•

Patch-based probabilistic image quality assessment for face selection and improved video-based face recognition

[...]

Yongkang Wong¹, Shaokang Chen¹, Sandra Mau¹, Conrad Sanderson¹, Brian C. Lovell¹ - Show less +1 more•Institutions (1)

University of Queensland¹

20 Jun 2011

TL;DR: An efficient patch-based face image quality assessment algorithm which quantifies the similarity of a face image to a probabilistic face model, representing an ‘ideal’ face is proposed.

...read moreread less

Abstract: In video based face recognition, face images are typically captured over multiple frames in uncontrolled conditions, where head pose, illumination, shadowing, motion blur and focus change over the sequence. Additionally, inaccuracies in face localisation can also introduce scale and alignment variations. Using all face images, including images of poor quality, can actually degrade face recognition performance. While one solution it to use only the ‘best’ of images, current face selection techniques are incapable of simultaneously handling all of the abovementioned issues. We propose an efficient patch-based face image quality assessment algorithm which quantifies the similarity of a face image to a probabilistic face model, representing an ‘ideal’ face. Image characteristics that affect recognition are taken into account, including variations in geometric alignment (shift, rotation and scale), sharpness, head pose and cast shadows. Experiments on FERET and PIE datasets show that the proposed algorithm is able to identify images which are simultaneously the most frontal, aligned, sharp and well illuminated. Further experiments on a new video surveillance dataset (termed ChokePoint) show that the proposed method provides better face subsets than existing face selection techniques, leading to significant improvements in recognition accuracy.

...read moreread less

314 citations

"Can holistic representations be use..." refers background in this paper

...[12] Comparison of a facial image with ideal face models....
[...]

Journal Article•DOI•

A Unified Framework for Biometric Expert Fusion Incorporating Quality Measures

[...]

Norman Poh¹, J. Kittler¹•Institutions (1)

University of Surrey¹

01 Jan 2012-IEEE Transactions on Pattern Analysis and Machine Intelligence

TL;DR: A general Bayesian framework that can utilize the quality information effectively is proposed that encompasses several recently proposed quality-based fusion algorithms in the literature and improves the understanding of the role of quality in multiple classifier combination.

...read moreread less

Abstract: This paper proposes a unified framework for quality-based fusion of multimodal biometrics. Quality-dependent fusion algorithms aim to dynamically combine several classifier (biometric expert) outputs as a function of automatically derived (biometric) sample quality. Quality measures used for this purpose quantify the degree of conformance of biometric samples to some predefined criteria known to influence the system performance. Designing a fusion classifier to take quality into consideration is difficult because quality measures cannot be used to distinguish genuine users from impostors, i.e., they are nondiscriminative yet still useful for classification. We propose a general Bayesian framework that can utilize the quality information effectively. We show that this framework encompasses several recently proposed quality-based fusion algorithms in the literature-Nandakumar et al., 2006; Poh et al., 2007; Kryszczuk and Drygajo, 2007; Kittler et al., 2007; Alonso-Fernandez, 2008; Maurer and Baker, 2007; Poh et al., 2010. Furthermore, thanks to the systematic study concluded herein, we also develop two alternative formulations of the problem, leading to more efficient implementation (with fewer parameters) and achieving performance comparable to, or better than, the state of the art. Last but not least, the framework also improves the understanding of the role of quality in multiple classifier combination.

...read moreread less

114 citations

Book Chapter•DOI•

Standardization of face image sample quality

[...]

Xiufeng Gao¹, Stan Z. Li², Rong Liu², Peiren Zhang¹•Institutions (2)

University of Science and Technology of China¹, Chinese Academy of Sciences²

27 Aug 2007

TL;DR: An approach for standardization of facial image quality is presented, and facial symmetry based methods for its assessment by which facial asymmetries caused by non-frontal lighting and improper facial pose can be measured are developed.

...read moreread less

Abstract: Performance of biometric systems is dependent on quality of acquired biometric samples. Poor sample quality is a main reason for matching errors in biometric systems and may be the main weakness of some implementations. In this paper, we present an approach for standardization of facial image quality, and develop facial symmetry based methods for its assessment by which facial asymmetries caused by non-frontal lighting and improper facial pose can be measured. Experimental results are provided to illustrate the concepts, definitions and effectiveness.

...read moreread less

113 citations