Can holistic representations be used for face biometric quality assessment

doi:10.1109/ICIP.2013.6738575

Home
/
Papers
/
Can holistic representations be used for face biometric quality assessment

Proceedings Article•DOI•

Can holistic representations be used for face biometric quality assessment

Samarth Bharadwaj¹, Mayank Vatsa¹, Richa Singh¹•Institutions (1)

Indraprastha Institute of Information Technology¹

01 Sep 2013-pp 2792-2796

TL;DR: This paper investigates the use of holistic super-ordinate representations, namely, Gist and sparsely pooled Histogram of Orientated Gradient, in classifying images into different quality categories that are derived from matching performance.

read less

Abstract: A face quality metric must quantitatively measure the usability of an image as a biometric sample. Though it is well established that quality measures are an integral part of robust face recognition systems, automatic measurement of bio-metric quality in face is still challenging. Inspired by scene recognition research, this paper investigates the use of holistic super-ordinate representations, namely, Gist and sparsely pooled Histogram of Orientated Gradient (HOG), in classifying images into different quality categories that are derived from matching performance. The experiments on the CAS-PEAL and SCFace databases containing covariates such as illumination, expression, pose, low-resolution and occlusion by accessories, suggest that the proposed algorithm can efficiently classify input face image into relevant quality categories and be utilized in face recognition systems.

...read moreread less

Citations

PDF

Open Access

More filters

Journal Article•DOI•

Biometric quality: a review of fingerprint, iris, and face

[...]

Samarth Bharadwaj¹, Mayank Vatsa¹, Richa Singh¹•Institutions (1)

Indian Institutes of Information Technology¹

02 Jul 2014-Eurasip Journal on Image and Video Processing

TL;DR: The analysis of the characteristic function of quality and match scores shows that a careful selection of complimentary set of quality metrics can provide more benefit to various applications of biometric quality.

...read moreread less

Abstract: Biometric systems encounter variability in data that influence capture, treatment, and u-sage of a biometric sample. It is imperative to first analyze the data and incorporate this understanding within the recognition system, making assessment of biometric quality an important aspect of biometrics. Though several interpretations and definitions of quality exist, sometimes of a conflicting nature, a holistic definition of quality is indistinct. This paper presents a survey of different concepts and interpretations of biometric quality so that a clear picture of the current state and future directions can be presented. Several factors that cause different types of degradations of biometric samples, including image features that attribute to the effects of these degradations, are discussed. Evaluation schemes are presented to test the performance of quality metrics for various applications. A survey of the features, strengths, and limitations of existing quality assessment techniques in fingerprint, iris, and face biometric are also presented. Finally, a representative set of quality metrics from these three modalities are evaluated on a multimodal database consisting of 2D images, to understand their behavior with respect to match scores obtained from the state-of-the-art recognition systems. The analysis of the characteristic function of quality and match scores shows that a careful selection of complimentary set of quality metrics can provide more benefit to various applications of biometric quality.

...read moreread less

119 citations

Cites background from "Can holistic representations be use..."

...Recently, holistic descriptors extracted from the face region are shown to be good indicators of performance of face recognition systems [87]....
[...]

Journal Article•DOI•

Learning Face Image Quality From Human Assessments

[...]

Lacey Best-Rowden¹, Anil K. Jain¹•Institutions (1)

Michigan State University¹

30 Jan 2018-IEEE Transactions on Information Forensics and Security

TL;DR: This is the first work to utilize human assessments of face image quality in designing a predictor of unconstrained face quality that is shown to be effective in cross-database evaluation.

...read moreread less

Abstract: Face image quality can be defined as a measure of the utility of a face image to automatic face recognition. In this paper, we propose (and compare) two methods for learning face image quality based on target face quality values from: 1) human assessments of face image quality (matcher-independent) and 2) quality values computed from similarity scores (matcher-dependent). A support vector regression model trained on face features extracted using a deep convolutional neural network (ConvNet) is used to predict the quality of a face image. The proposed methods are evaluated on two unconstrained face image databases, Labeled Faces in the Wild and IARPA Janus Benchmark-A (IJB-A), which both contain facial variations encompassing a multitude of quality factors. Evaluation of the proposed automatic face image quality measures shows we are able to reduce the false non-match rate at 1% false match rate by at least 13% for two face matchers (a commercial off-the-shelf matcher and a ConvNet matcher) by using the proposed face quality to select subsets of face images and video frames for matching templates (i.e., multiple faces per subject) in the IJB-A protocol. To the best of our knowledge, this is the first work to utilize human assessments of face image quality in designing a predictor of unconstrained face quality that is shown to be effective in cross-database evaluation.

...read moreread less

79 citations

Posted Content•

Face Image Quality Assessment: A Literature Survey

[...]

Torsten Schlett, Christian Rathgeb, Olaf Henniger, Javier Galbally, Julian Fierrez, Christoph Busch - Show less +2 more

02 Sep 2020-arXiv: Computer Vision and Pattern Recognition

TL;DR: This survey provides an overview of the face image quality assessment literature, which predominantly focuses on visible wavelength face image input and a trend towards deep learning based methods is observed, including notable conceptual differences among the recent approaches.

...read moreread less

Abstract: The performance of face analysis and recognition systems depends on the quality of the acquired face data, which is influenced by numerous factors. Automatically assessing the quality of face data in terms of biometric utility can thus be useful to filter out low quality data. This survey provides an overview of the face quality assessment literature in the framework of face biometrics, with a focus on face recognition based on visible wavelength face images as opposed to e.g. depth or infrared quality assessment. A trend towards deep learning based methods is observed, including notable conceptual differences among the recent approaches. Besides image selection, face image quality assessment can also be used in a variety of other application scenarios, which are discussed herein. Open issues and challenges are pointed out, i.a. highlighting the importance of comparability for algorithm evaluations, and the challenge for future work to create deep learning approaches that are interpretable in addition to providing accurate utility predictions.

...read moreread less

51 citations

Cites background or methods from "Can holistic representations be use..."

...LFW [79] 2011 to 2021 17: B [181][9][23] E [113][198][120][46][47] [25][123][140][24][73][22][176][166][193] FERET [146] 2007 to 2020 9: B [3][76][23][189] C [176] E [2][1][120] [160] VGGFace2 [21] 2019 to 2021 7: B [73][74] C [191] E [46][47][24][193] CASIA-WebFace [197] 2017 to 2021 7: B [198][204][166] C [140][9] E [113][193] CAS-PEAL [55] 2009 to 2018 6: B [2][76][17] C [1][184] E [201] FRGC [145] 2006 to 2018 6: B [102][101][75][23] C [152][184] MS-Celeb-1M [68] 2019 to 2020 5: B [166] C [22][176][193] E [113] CFP [164] 2019 to 2021 5: E [25][123][24][22][166] IJB-C [122] 2019 to 2021 5: E [123][140][191][22][166] YTF [188] 2014 to 2020 5: B [136] E [198][30][22][166] MS1MV2 [34] 2021 4: C [25][123][140][24] IJB-A [105] 2017 to 2019 4: B [113] C [193] E [166][9] ChokePoint [189] 2011 to 2018 4: B [150][179] E [184][189] SCface [64] 2011 to 2018 4: B [17] E [120][184][23] Extended Yale [110] 2010 to 2018 4: B [153][151][163] C [184] CPLFW [205] 2021 3: E [25][123][24] IJB-B [187] 2021 3: E [25][123][24] Adience [40] 2020 to 2021 3: E [25][140][176] BioSecure [139] 2019 to 2021 3: E [46][73][74] GBU [143] 2012 to 2014 3: B [2][1] E [142] AT&T [158] 2010 to 2016 3: B [76][163] C [152] CMU-PIE [167] 2009 to 2011 3: C [11] E [160][189] FRVT 2006 [147] 2008 to 2010 3: E [11][13][14] Yale [57] 2007 to 2014 3: B [2][1] E [56] BANCA [8] 2006 to 2008 3: B [108][109] E [156] AgeDB [129] 2021 2: E [123][24] CALFW [206] 2021 2: E [123][24] MEDS-II [44] 2019 to 2020 2: B [155][154] MegaFace [99] 2019 to 2020 2: E [22][166] AR [121] 2014 to 2018 2: C [152][184] PaSC [12] 2013 to 2018 2: B [150] E [142] MBGC [144] 2012 to 2014 2: E [2][1] Q-FIRE [95] 2012 to 2014 2: E [2][77]...
[...]
...[17] trained a one-vs-all SVM for 4 quality bins using either sparsely pooled Histogram of Oriented Gradient (HOG) or Gist [137] input features....
[...]
...• FR-based ground truth training (Dfrt): These approaches obtained training data from FRmodels [17][179][76][9] [150][74][73][191][140][25]....
[...]
...2013 [17] Dfrt 4-class SVM on Gist[137] or HOG....
[...]
...Some of the works evaluated FIQA performance exclusively by means other than the ERC - for example, FR performance was evaluated for 4 FIQA-derived quality bins in [17]....
[...]

Journal Article•DOI•

Face biometric quality assessment via light CNN

[...]

Jun Yu¹, Kejia Sun¹, Fei Gao¹, Suguo Zhu¹•Institutions (1)

Hangzhou Dianzi University¹

01 Aug 2017-Pattern Recognition Letters

TL;DR: This paper considered five categories of common homogeneous distortion in video suvillance applications, i.e. low-resolution, blurring, additive Gaussian white noise, salt and pepper noise, and Poisson noise and proposed a novel biometric quality assessment (BQA) method for face images and explored its applications in face recognition.

...read moreread less

31 citations

Journal Article•

Standardization of Face Image Sample Quality

[...]

LI Zi-qing

01 Jan 2009-Journal of Chinese Computer Systems

TL;DR: This paper proposes an approach for standardization of facial image quality, and develops facial symmetry based methods for the assessment of it by measuring facial asymmetries caused by non-frontal lighting and improper facial pose.

...read moreread less

Abstract: Performance of biometric systems is dependent on quality of acquired biometric samples. Poor sample quality is a main reason for matching errors in biometric systems and may be the main weakness of some implementations. This paper proposes an approach for standardization of facial image quality, and develops facial symmetry based methods for the assessment of it by measuring facial asymmetries caused by non-frontal lighting and improper facial pose. Experimental results are provided to illustrate the concepts, definitions and effectiveness.

...read moreread less

28 citations

References

PDF

Open Access

More filters

Proceedings Article•DOI•

Histograms of oriented gradients for human detection

[...]

Navneet Dalal¹, Bill Triggs¹•Institutions (1)

French Institute for Research in Computer Science and Automation¹

20 Jun 2005

TL;DR: It is shown experimentally that grids of histograms of oriented gradient (HOG) descriptors significantly outperform existing feature sets for human detection, and the influence of each stage of the computation on performance is studied.

...read moreread less

Abstract: We study the question of feature sets for robust visual object recognition; adopting linear SVM based human detection as a test case. After reviewing existing edge and gradient based descriptors, we show experimentally that grids of histograms of oriented gradient (HOG) descriptors significantly outperform existing feature sets for human detection. We study the influence of each stage of the computation on performance, concluding that fine-scale gradients, fine orientation binning, relatively coarse spatial binning, and high-quality local contrast normalization in overlapping descriptor blocks are all important for good results. The new approach gives near-perfect separation on the original MIT pedestrian database, so we introduce a more challenging dataset containing over 1800 annotated human images with a large range of pose variations and backgrounds.

...read moreread less

31,952 citations

"Can holistic representations be use..." refers background or methods in this paper

...In this research, two prominent holistic representations, Gist [18] and sparsely pooled HOG [19] are considered....
[...]
...The illustrated instances are obtained from the set of images classified to a quality bin by both Gist and HOG....
[...]
...HOG: Dalal and Triggs [19] present a simple descriptor known as histogram of orientated gradient that is popularly used for humans, vehicles and animals detection in still imagery and videos due to the low computation time as well as high accuracy....
[...]
...5 show the performance of COTS on each of the quality bins obtained from both GIST and HOG....
[...]
...Further, compared to local image descriptors, the feature length of Gist (512) and HOG (81) can ensure low computational time for quality assessment....
[...]

Journal Article•DOI•

Modeling the Shape of the Scene: A Holistic Representation of the Spatial Envelope

[...]

Aude Oliva¹, Antonio Torralba²•Institutions (2)

Brigham and Women's Hospital¹, Carleton College²

01 May 2001-International Journal of Computer Vision

TL;DR: The performance of the spatial envelope model shows that specific information about object shape or identity is not a requirement for scene categorization and that modeling a holistic representation of the scene informs about its probable semantic category.

...read moreread less

Abstract: In this paper, we propose a computational model of the recognition of real world scenes that bypasses the segmentation and the processing of individual objects or regions. The procedure is based on a very low dimensional representation of the scene, that we term the Spatial Envelope. We propose a set of perceptual dimensions (naturalness, openness, roughness, expansion, ruggedness) that represent the dominant spatial structure of a scene. Then, we show that these dimensions may be reliably estimated using spectral and coarsely localized information. The model generates a multidimensional space in which scenes sharing membership in semantic categories (e.g., streets, highways, coasts) are projected closed together. The performance of the spatial envelope model shows that specific information about object shape or identity is not a requirement for scene categorization and that modeling a holistic representation of the scene informs about its probable semantic category.

...read moreread less

6,882 citations

"Can holistic representations be use..." refers background or methods in this paper

...In this research, two prominent holistic representations, Gist [18] and sparsely pooled HOG [19] are considered....
[...]
...The illustrated instances are obtained from the set of images classified to a quality bin by both Gist and HOG....
[...]
...Further, compared to local image descriptors, the feature length of Gist (512) and HOG (81) can ensure low computational time for quality assessment....
[...]
...The results with Gist and HOG show promise towards a robust solution to the important problem of quality assessment in face biometrics....
[...]
...Gist: Olivia and Torralba [18] propose a holistic representation of the spatial envelope of a scene image....
[...]

Book•

Modern image quality assessment

[...]

Zhou Wang¹, A.C. Bovik²•Institutions (2)

University of Texas at Arlington¹, University of Texas at Austin²

01 Jan 2006

TL;DR: This book is about objective image quality assessment to provide computational models that can automatically predict perceptual image quality and to provide new directions for future research by introducing recent models and paradigms that significantly differ from those used in the past.

...read moreread less

Abstract: This book is about objective image quality assessmentwhere the aim is to provide computational models that can automatically predict perceptual image quality. The early years of the 21st century have witnessed a tremendous growth in the use of digital images as a means for representing and communicating information. A considerable percentage of this literature is devoted to methods for improving the appearance of images, or for maintaining the appearance of images that are processed. Nevertheless, the quality of digital images, processed or otherwise, is rarely perfect. Images are subject to distortions during acquisition, compression, transmission, processing, and reproduction. To maintain, control, and enhance the quality of images, it is important for image acquisition, management, communication, and processing systems to be able to identify and quantify image quality degradations. The goals of this book are as follows; a) to introduce the fundamentals of image quality assessment, and to explain the relevant engineering problems, b) to give a broad treatment of the current state-of-the-art in image quality assessment, by describing leading algorithms that address these engineering problems, and c) to provide new directions for future research, by introducing recent models and paradigms that significantly differ from those used in the past. The book is written to be accessible to university students curious about the state-of-the-art of image quality assessment, expert industrial R&D engineers seeking to implement image/video quality assessment systems for specific applications, and academic theorists interested in developing new algorithms for image quality assessment or using existing algorithms to design or optimize other image processing applications.

...read moreread less

1,041 citations

Journal Article•DOI•

The CAS-PEAL Large-Scale Chinese Face Database and Baseline Evaluations

[...]

Wen Gao¹, Bo Cao¹, Shiguang Shan, Xilin Chen, Delong Zhou², Xiaohua Zhang³, Debin Zhao⁴ - Show less +3 more•Institutions (4)

Chinese Academy of Sciences¹, Zhejiang University of Technology², University of Rochester³, Harbin Institute of Technology⁴

01 Jan 2008

TL;DR: The evaluation protocol based on the CAS-PEAL-R1 database is discussed and the performance of four algorithms are presented as a baseline to do the following: elementarily assess the difficulty of the database for face recognition algorithms; preference evaluation results for researchers using the database; and identify the strengths and weaknesses of the commonly used algorithms.

...read moreread less

Abstract: In this paper, we describe the acquisition and contents of a large-scale Chinese face database: the CAS-PEAL face database. The goals of creating the CAS-PEAL face database include the following: 1) providing the worldwide researchers of face recognition with different sources of variations, particularly pose, expression, accessories, and lighting (PEAL), and exhaustive ground-truth information in one uniform database; 2) advancing the state-of-the-art face recognition technologies aiming at practical applications by using off-the-shelf imaging equipment and by designing normal face variations in the database; and 3) providing a large-scale face database of Mongolian. Currently, the CAS-PEAL face database contains 99 594 images of 1040 individuals (595 males and 445 females). A total of nine cameras are mounted horizontally on an arc arm to simultaneously capture images across different poses. Each subject is asked to look straight ahead, up, and down to obtain 27 images in three shots. Five facial expressions, six accessories, and 15 lighting changes are also included in the database. A selected subset of the database (CAS-PEAL-R1, containing 30 863 images of the 1040 subjects) is available to other researchers now. We discuss the evaluation protocol based on the CAS-PEAL-R1 database and present the performance of four algorithms as a baseline to do the following: 1) elementarily assess the difficulty of the database for face recognition algorithms; 2) preference evaluation results for researchers using the database; and 3) identify the strengths and weaknesses of the commonly used algorithms.

...read moreread less

971 citations

"Can holistic representations be use..." refers methods in this paper

...CAS-PEAL [21] 1040 (312/728) pose, illumination, expression, accessories, background, distance Combined 1170 (351/819) all of the above...
[...]
...Hence, in this research, a heterogeneous combination of two face databases, namely, the SCFace [20] and CAS-PEAL [21], with pose, illumination, expression, accessories, background, distance and resolution variations is used....
[...]
...[21] W. Gao, B. Cao, S. Shan, X. Chen, D. Zhou, X. Zhang, and D. Zhao, “The CAS-PEAL large-scale chinese face database and baseline evaluations,” IEEE Transactions on System Man, and Cybernetics-A, vol. 38, no. 1, pp. 149–161, 2008....
[...]

Journal Article•DOI•

SCface --- surveillance cameras face database

[...]

Mislav Grgic¹, Kresimir Delac¹, Sonja Grgic¹•Institutions (1)

University of Zagreb¹

01 Feb 2011-Multimedia Tools and Applications

TL;DR: A database of static images of human faces taken in uncontrolled indoor environment using five video surveillance cameras of various qualities to enable robust face recognition algorithms testing, emphasizing different law enforcement and surveillance use case scenarios is described.

...read moreread less

Abstract: In this paper we describe a database of static images of human faces. Images were taken in uncontrolled indoor environment using five video surveillance cameras of various qualities. Database contains 4,160 static images (in visible and infrared spectrum) of 130 subjects. Images from different quality cameras should mimic real-world conditions and enable robust face recognition algorithms testing, emphasizing different law enforcement and surveillance use case scenarios. In addition to database description, this paper also elaborates on possible uses of the database and proposes a testing protocol. A baseline Principal Component Analysis (PCA) face recognition algorithm was tested following the proposed protocol. Other researchers can use these test results as a control algorithm performance score when testing their own algorithms on this dataset. Database is available to research community through the procedure described at www.scface.org .

...read moreread less

483 citations

"Can holistic representations be use..." refers background or methods in this paper

...The experiments on the CASPEAL and SCFace databases containing covariates such as illumination, expression, pose, low-resolution and occlusion by accessories, suggest that the proposed algorithm can efficiently classify input face image into relevant quality categories and be utilized in face recognition systems....
[...]
...Hence, in this research, a heterogeneous combination of two face databases, namely, the SCFace [20] and CAS-PEAL [21], with pose, illumination, expression, accessories, background, distance and resolution variations is used....
[...]
...Database Subjects (Train/Test) Description SCFace [20] 130 (39/91) pose, low resolution...
[...]