scispace - formally typeset

Proceedings ArticleDOI

On rank aggregation for face recognition from videos

01 Sep 2013-pp 2993-2997

TL;DR: A video based face recognition algorithm that computes a discriminative video signature as an ordered list of still face images to facilitate matching two videos with large variations is presented.

AbstractFace recognition from still face images suffers due to intrapersonal variations caused by pose, illumination, and expression that degrade the performance. On the other hand, videos provide abundant information that can be leveraged to compensate the limitations of still face images and enhance face recognition performance. This paper presents a video based face recognition algorithm that computes a discriminative video signature as an ordered list of still face images. The video signature embeds diverse intra-personal and temporal variations across multiple frames, thus facilitates matching two videos with large variations. Two videos are matched by comparing their discriminative signatures using the Kendall tau similarity distance measure. Performance comparison with the benchmark results and a commercial face recognition system on the publicly available YouTube faces database show the efficacy of the proposed video based face recognition algorithm.

...read more


Citations
More filters
Journal ArticleDOI
TL;DR: A video-based face recognition algorithm that computes a discriminative video signature as an ordered list of still face images from a large dictionary, which embeds diverse intra-personal variations and facilitates in matching two videos with large variations.
Abstract: Due to widespread applications, availability of large intra-personal variations in video and limited information content in still images, video-based face recognition has gained significant attention. Unlike still face images, videos provide abundant information that can be leveraged to address variations in pose, illumination, and expression as well as enhance the face recognition performance. This paper presents a video-based face recognition algorithm that computes a discriminative video signature as an ordered list of still face images from a large dictionary. A three-stage approach is proposed for optimizing ranked lists across multiple video frames and fusing them into a single composite ordered list to compute the video signature. This signature embeds diverse intra-personal variations and facilitates in matching two videos with large variations. For matching two videos, a discounted cumulative gain measure is utilized, which uses the ranking of images in the video signature as well as the usefulness of images in characterizing the individual in the video. The efficacy of the proposed algorithm is evaluated under different video-based face recognition scenarios such as matching still face images with videos and matching videos with videos. The efficacy of the proposed algorithm is demonstrated on the YouTube faces database and the MBGC v2 video challenge database that comprise different types of video-based face recognition challenges such as matching still face images with videos and matching videos with videos. Performance comparison with the benchmark results on both the databases and a commercial face recognition system shows the efficiency of the proposed algorithm for video-based face recognition.

42 citations


Cites background or methods from "On rank aggregation for face recogn..."

  • ...[7] proposed to compute a video signature as an ordered list of still face images from a large dictionary....

    [...]

  • ...However, unlike the proposed algorithm, existing algorithm [7] does not optimize every ranked list before fusion which results in lower performance....

    [...]

  • ...gorithm [7] for all the three matching scenarios i....

    [...]

  • ...[7] Rank aggregation YouTube Faces [42] 78....

    [...]

  • ...[7] is also a rank aggregation based approach that combines multiple ranked lists for a video using Markov chain...

    [...]

Proceedings ArticleDOI
01 Sep 2013
TL;DR: This work demonstrates that all three COTS matchers individually are superior to previously published face recognition results on the unconstrained YouTube Faces database and achieves a 20% improvement in accuracy over previously published results.
Abstract: Face recognition in video is becoming increasingly important due to the abundance of video data captured by surveillance cameras, mobile devices, Internet uploads, and other sources. Given the aggregate of facial information contained in a video (i.e., a sequence of face images or frames), video-based face recognition solutions can potentially alleviate classic challenges caused by variations in pose, illumination, and expression. However, with this increased focus on the development of algorithms specifically crafted for video-based face recognition, it is important to establish a baseline for the accuracy using state-of-the-art still image matchers. Note that most commercial-off-the-shelf (COTS) offerings are still limited to single frame matching. In order to measure the accuracy of COTS face recognition systems on video data, we first investigate the effectiveness of multi-frame score-level fusion and analyze the consistency across three COTS face matchers. We demonstrate that all three COTS matchers individually are superior to previously published face recognition results on the unconstrained YouTube Faces database. Further, fusion of scores from the three COTS matchers achieves a 20% improvement in accuracy over previously published results. We encourage the use of these results as a competitive baseline for video-to-video face matching on the YouTube Faces database.

39 citations


Cites methods from "On rank aggregation for face recogn..."

  • ...The accuracies of the proposed COTS fusion schemes are benchmarked against Wolf et al.’s Matched Background Similarity (MBGS) [24], Li et al.’s Adaptive Probabilistic Elastic Matching (APEM) Fusion [15], Cui et al.’s Spatio-Temporal Face Region Descriptor Pairwiseconstrained Multiple Metric Learning (STFRD+PMML) [5], and Bhatt et al.’s method which we call Rank Aggregation [3]....

    [...]

  • ...All three COTS face matchers and fusion of three matchers significantly outperform previous methods: Rank Aggregation [3], APEM Fusion [15], and STFRD+PMML [5]....

    [...]

  • ..., the interpupillary distances remain the same) between the images used here and those used by [3, 15, 24]....

    [...]

  • ...’s method which we call Rank Aggregation [3]....

    [...]

Proceedings ArticleDOI
04 May 2015
TL;DR: This paper proposes a video-based face recognition method which improves upon the sparse representation framework with an intelligent and adaptive sparse dictionary that updates the current probe image into the training matrix based on continuously monitoring the probe video through a novel confidence criterion and a Bayesian inference scheme.
Abstract: Sparse representation-based face recognition has gained considerable attention recently due to its robustness against illumination and occlusion. Recognizing faces from videos has become a topic of importance to alleviate the limit of information content in still images. However, the sparse recognition framework is not applicable to video-based face recognition due to its sensitivity towards pose and alignment changes. In this paper, we propose a video-based face recognition method which improves upon the sparse representation framework. Our key contribution is an intelligent and adaptive sparse dictionary that updates the current probe image into the training matrix based on continuously monitoring the probe video through a novel confidence criterion and a Bayesian inference scheme. Due to this novel approach, our method is robust to pose and alignment and hence can be used to recognize faces from unconstrained videos successfully. Moreover, in a moving scene, camera angle, illumination and other imaging conditions may change quickly leading to performance loss in accuracy. In such situations, it is impractical to re-enroll the individual and re-train the classifiers on a continuous basis. Our novel approach addresses these practical issues. Experimental results on the well known YouTube Face database demonstrates the effectiveness of our method.

7 citations


Cites methods from "On rank aggregation for face recogn..."

  • ...In [15], Markov chain-based rank aggregation technique was used to calculate a video signature as an ordered set of frame images....

    [...]

Patent
31 Dec 2015
Abstract: The method includes a dictionary including a target collection defined by images that are known with a defined level of certainty to include a subject and an imposter collection defined by images of individuals other than the subject. In the method, images of an area are captured over a period of time. In respect of each image: a matching calculation is carried out, based upon a comparison of the image captured with the images in the dictionary to result in a measure of confidence that the subject is in the area; and an inference determination is made to replace one of the target collection images with a further image that is known with the defined level of certainty, the determination being a function of the measure of confidence resultant from the captured image, the measure resultant from one or more previously captured images and the associated capture times.

5 citations

Journal ArticleDOI
TL;DR: A novel approach for recognizing faces in videos with high recognition rate that embeds diverse intra-personal variations such as poses, expressions and facilitates in matching two videos with large variations and exhibits significant performance improvement when compared with the existing techniques.
Abstract: This paper proposes a novel approach for recognizing faces in videos with high recognition rate. Initially, the feature vector based on Normalized Local Binary Patterns is obtained for the face region. A set of training and testing videos are used in this face recognition procedure. Each frame in the query video is matched with the signature of the faces in the database using Euclidean distance and a rank list is formed. Each ranked list is clustered and its reliability is analyzed for re-ranking. Multiple re-ranked lists of the query video is fused together to form a video signature. This video signature embeds diverse intra-personal variations such as poses, expressions and facilitates in matching two videos with large variations. For matching two videos, their composite ranked lists are compared using a Kendall Tau distance measure. The developed methods are deployed on the YouTube and ChokePoint videos, and they exhibit significant performance improvement owing to their novel approach when compared with the existing techniques.

2 citations


References
More filters
Journal ArticleDOI
TL;DR: A generalized gray-scale and rotation invariant operator presentation that allows for detecting the "uniform" patterns for any quantization of the angular space and for any spatial resolution and presents a method for combining multiple operators for multiresolution analysis.
Abstract: Presents a theoretically very simple, yet efficient, multiresolution approach to gray-scale and rotation invariant texture classification based on local binary patterns and nonparametric discrimination of sample and prototype distributions. The method is based on recognizing that certain local binary patterns, termed "uniform," are fundamental properties of local image texture and their occurrence histogram is proven to be a very powerful texture feature. We derive a generalized gray-scale and rotation invariant operator presentation that allows for detecting the "uniform" patterns for any quantization of the angular space and for any spatial resolution and presents a method for combining multiple operators for multiresolution analysis. The proposed approach is very robust in terms of gray-scale variations since the operator is, by definition, invariant against any monotonic transformation of the gray scale. Another advantage is computational simplicity as the operator can be realized with a few operations in a small neighborhood and a lookup table. Experimental results demonstrate that good discrimination can be achieved with the occurrence statistics of simple rotation invariant local binary patterns.

13,021 citations


"On rank aggregation for face recogn..." refers methods in this paper

  • ...In our research, the dictionary comprises 38, 488 images pertaining to 337 individuals from the CMU Multi-PIE [18] database captured in four sessions....

    [...]

Journal ArticleDOI
TL;DR: This paper presents a novel and efficient facial image representation based on local binary pattern (LBP) texture features that is assessed in the face recognition problem under different challenges.
Abstract: This paper presents a novel and efficient facial image representation based on local binary pattern (LBP) texture features. The face image is divided into several regions from which the LBP feature distributions are extracted and concatenated into an enhanced feature vector to be used as a face descriptor. The performance of the proposed method is assessed in the face recognition problem under different challenges. Other applications and several extensions are also discussed

5,237 citations


"On rank aggregation for face recogn..." refers methods in this paper

  • ...In our research, the dictionary comprises 38, 488 images pertaining to 337 individuals from the CMU Multi-PIE [18] database captured in four sessions....

    [...]

Proceedings ArticleDOI
01 Apr 2001
TL;DR: A set of techniques for the rank aggregation problem is developed and compared to that of well-known methods, to design rank aggregation techniques that can be used to combat spam in Web searches.
Abstract: We consider the problem of combining ranking results from various sources. In the context of the Web, the main applications include building meta-search engines, combining ranking functions, selecting documents based on multiple criteria, and improving search precision through word associations. We develop a set of techniques for the rank aggregation problem and compare their performance to that of well-known methods. A primary goal of our work is to design rank aggregation techniques that can e ectively combat \spam," a serious problem in Web searches. Experiments show that our methods are simple, e cient, and e ective.

1,857 citations


"On rank aggregation for face recogn..." refers background in this paper

  • ...Multiple ranked lists computed across n frames of a video have significant amount of overlap....

    [...]

  • ...If r(J) r(I), where r(·) represents the rank, for a majority of ranked lists that ranked both I and J , then Sk+1 = J , otherwise similarity transition with a probability γ (γ = 1) is executed....

    [...]

BookDOI
31 Aug 2011
TL;DR: This highly anticipated new edition provides a comprehensive account of face recognition research and technology, spanning the full range of topics needed for designing operational face recognition systems, as well as offering challenges and future directions.
Abstract: This highly anticipated new edition provides a comprehensive account of face recognition research and technology, spanning the full range of topics needed for designing operational face recognition systems. After a thorough introductory chapter, each of the following chapters focus on a specific topic, reviewing background information, up-to-date techniques, and recent results, as well as offering challenges and future directions. Features: fully updated, revised and expanded, covering the entire spectrum of concepts, methods, and algorithms for automated face detection and recognition systems; provides comprehensive coverage of face detection, tracking, alignment, feature extraction, and recognition technologies, and issues in evaluation, systems, security, and applications; contains numerous step-by-step algorithms; describes a broad range of applications; presents contributions from an international selection of experts; integrates numerous supporting graphs, tables, charts, and performance data.

1,576 citations

Proceedings ArticleDOI
20 Jun 2011
TL;DR: A comprehensive database of labeled videos of faces in challenging, uncontrolled conditions, the ‘YouTube Faces’ database, along with benchmark, pair-matching tests are presented and a novel set-to-set similarity measure, the Matched Background Similarity (MBGS), is described.
Abstract: Recognizing faces in unconstrained videos is a task of mounting importance. While obviously related to face recognition in still images, it has its own unique characteristics and algorithmic requirements. Over the years several methods have been suggested for this problem, and a few benchmark data sets have been assembled to facilitate its study. However, there is a sizable gap between the actual application needs and the current state of the art. In this paper we make the following contributions. (a) We present a comprehensive database of labeled videos of faces in challenging, uncontrolled conditions (i.e., ‘in the wild’), the ‘YouTube Faces’ database, along with benchmark, pair-matching tests1. (b) We employ our benchmark to survey and compare the performance of a large variety of existing video face recognition techniques. Finally, (c) we describe a novel set-to-set similarity measure, the Matched Background Similarity (MBGS). This similarity is shown to considerably improve performance on the benchmark tests.

1,242 citations


"On rank aggregation for face recogn..." refers background in this paper

  • ...Index Terms— Video based face recognition, Rank ag- gregation, Dictionary based face recognition...

    [...]