Soft-Margin Learning for Multiple Feature-Kernel Combinations with Domain Adaptation, for Recognition in Surveillance Face Datasets
01 Jun 2016-pp 237-242
TL;DR: A novel soft-margin based learning method for multiple feature-kernel combinations, followed by feature transformed using Domain Adaptation, which outperforms many recent state-of-the-art techniques, when tested using three real-world surveillance face datasets.
Abstract: Face recognition (FR) is the most preferred mode for biometric-based surveillance, due to its passive nature of detecting subjects, amongst all different types of biometric traits. FR under surveillance scenario does not give satisfactory performance due to low contrast, noise and poor illumination conditions on probes, as compared to the training samples. A state-of-the-art technology, Deep Learning, even fails to perform well in these scenarios. We propose a novel soft-margin based learning method for multiple feature-kernel combinations, followed by feature transformed using Domain Adaptation, which outperforms many recent state-of-the-art techniques, when tested using three real-world surveillance face datasets.
Citations
More filters
TL;DR: A transfer-CNN architecture of deep learning tailor-made for domain adaptation (DA), to overcome the difference in feature distributions between the gallery and probe samples and to provide enhanced domain-invariant representation for efficient deep-DA learning and classification is proposed.
28 citations
TL;DR: A deep network based on a generative adversarial network (GAN), termed LR-GAN, which helps to reconstruct realistic mugshot images from low-resolution probe samples, which provides rich performances for FR, as evident by the high rank-1 recognition rates, over 4 real-world degraded face datasets.
10 citations
TL;DR: A novel method for solving facial expression recognition (FER) tasks which uses a self-adaptive weighted synthesised local directional pattern (SW-SLDP) descriptor integrating sparse autoencoder (SA) features based on improved multiple kernel learning (IMKL) strategy is presented.
Abstract: This study presents a novel method for solving facial expression recognition (FER) tasks which uses a self-adaptive weighted synthesised local directional pattern (SW-SLDP) descriptor integrating sparse autoencoder (SA) features based on improved multiple kernel learning (IMKL) strategy. The authors' work includes three parts. Firstly, the authors propose a novel SW-SLDP feature descriptor which divides the facial images into patches and extracts sub-block features synthetically according to both distribution information and directional intensity contrast. Then self-adaptive weights are assigned to each sub-block feature according to the projection error between the expressional image and neutral image of each patch, which can highlight such areas containing more expressional texture information. Secondly, to extract a discriminative high-level feature, they introduce SA for feature representation, which extracts the hidden layer representation including more comprehensive information. Finally, to combine the above two kinds of features, an IMKL strategy is developed by effectively integrating both soft margin learning and intrinsic local constraints, which is robust to noisy condition and thus improve the classification performance. Extensive experimental results indicate their model can achieve competitive or even better performance with existing representative FER methods.
7 citations
30 Mar 2020
TL;DR: A novel dual deep-shallow channeled generative adversarial network (D2SC-GAN) which performs supervised domain adaptation (DA) by mapping LR degraded probe samples to their corresponding HR gallery-like counterparts to perform closed-set face recognition.
Abstract: Face Recognition using convolutional neural networks have achieved considerable success in constrained environments in the recent past. However, the performance of these methods deteriorates in case of mismatch of training and test distributions, under classroom/surveillance scenarios. These test (probe) samples suffer from degradations such as noise, poor illumination, pose variations, occlusion, low-resolution (LR), blur as well as aliasing, when compared to the crisp, rich training (gallery) set, comprising mostly of high-resolution (HR) mugshot images captured in laboratory settings. To cope with this scenario, we propose a novel dual deep-shallow channeled generative adversarial network (D2SC-GAN) which performs supervised domain adaptation (DA) by mapping LR degraded probe samples to their corresponding HR gallery-like counterparts to perform closed-set face recognition. D2SC-GAN uses a multi-component loss function comprising of multi-resolution patchwise MSE and normalized chi-squared distance loss functions, along with a Kullback-Leibler divergence based loss function. Moreover, we propose a novel classroom face dataset called the Indian Classroom Face Dataset (ICFD), which, to the best of our knowledge, is a first of its kind and will be helpful to explore the challenges of face recognition when used for automatically recording the attendance in classroom conditions. The proposed network achieves superior results on five real-world face datasets when compared with recent state-of-the-art deep as well as shallow supervised domain adaptation (DA), super-resolution (SR), and degraded face recognition (DFR) methods, which show the effectiveness of our proposed method.
5 citations
Cites methods from "Soft-Margin Learning for Multiple F..."
...(Corresponding author: Sukhendu Das.)...
[...]
...The SMLMFKC method proposed by Banerjee and Das [28] employs an optimal feature-kernel combination for adaptation to the target domain....
[...]
...The SMLMFKC method proposed by Banerjee and Das [28] employs an optimal feature-kernel combination for adaptation to the...
[...]
...The work proposed by Banerjee and Das [8] employs a novel 3-stage mutually exclusive training algorithm for deep domain adaptation to solve the problem of degraded FR....
[...]
TL;DR: The authors propose the design of an end-to-end siamese convolutional neural network (SCNN) that simultaneously replicates the facial make-up of a subject using its target image on a query face image and verifies the identity of the query face sample either with or withoutMake-up.
Abstract: Facial make-up changes the appearance of a person and significantly degrades the performance of automated face verification (FV) systems. Here, the authors propose the design of an end-to-end siamese convolutional neural network (SCNN) that simultaneously replicates the facial make-up of a subject using its target image (under facial make-up) on a query face image and verifies the identity of the query face sample either with or without make-up. The SCNN model is designed using loss functions to deal with the variations due to make-up. The proposed architecture can reciprocate the make-up at appropriate locations of the face without any human interventions. Rigorous experimentations on four benchmark facial make-up datasets reveal the efficiency of their proposed model. Ablation studies show improvement of 4% for genuine acceptance rate at 0.1% false acceptance rate and reduction of equal error rate by 42% for FV in case of YouTube Make-up dataset, and ‘10%’ in case of Virtual Make-up dataset, when compared to the nearest state-of-the-art method. For the transfer of make-up, the similarity measures also show the effectiveness of their method, where the peak signal-to-noise ratio and structural similarity values show an improvement by ∼20–24 and ∼29–32%, respectively, when compared to a recent state-of-the-art technique.
4 citations
References
More filters
TL;DR: A near-real-time computer system that can locate and track a subject's head, and then recognize the person by comparing characteristics of the face to those of known individuals, and that is easy to implement using a neural network architecture.
Abstract: We have developed a near-real-time computer system that can locate and track a subject's head, and then recognize the person by comparing characteristics of the face to those of known individuals. The computational approach taken in this system is motivated by both physiology and information theory, as well as by the practical requirements of near-real-time performance and accuracy. Our approach treats the face recognition problem as an intrinsically two-dimensional (2-D) recognition problem rather than requiring recovery of three-dimensional geometry, taking advantage of the fact that faces are normally upright and thus may be described by a small set of 2-D characteristic views. The system functions by projecting face images onto a feature space that spans the significant variations among known face images. The significant features are known as "eigenfaces," because they are the eigenvectors (principal components) of the set of faces; they do not necessarily correspond to features such as eyes, ears, and noses. The projection operation characterizes an individual face by a weighted sum of the eigenface features, and so to recognize a particular face it is necessary only to compare these weights to those of known individuals. Some particular advantages of our approach are that it provides for the ability to learn and later recognize new faces in an unsupervised manner, and that it is easy to implement using a neural network architecture.
14,562 citations
"Soft-Margin Learning for Multiple F..." refers background in this paper
...Several recent state-of-the art as-well-as traditional features are taken into account viz., Eigenfaces [29], Fisherfaces [7], Weberfaces [31], Local binary pattern (LBP) [1], Gaborfaces [17], Bag-of-words (BOW) [11], Fisher vector encoding on dense-SIFT features (FV-SIFT) [20] and VLAD encoding on dense-SIFT features (VLAD-SIFT) [3]....
[...]
..., Eigenfaces [29], Fisherfaces [7], Weberfaces [31], Local binary pattern (LBP) [1], Gaborfaces [17], Bag-of-words (BOW) [11], Fisher vector encoding on dense-SIFT features (FV-SIFT) [20] and VLAD encoding on dense-SIFT features (VLAD-SIFT) [3]....
[...]
TL;DR: In this paper, a face detection framework that is capable of processing images extremely rapidly while achieving high detection rates is described. But the detection performance is limited to 15 frames per second.
Abstract: This paper describes a face detection framework that is capable of processing images extremely rapidly while achieving high detection rates. There are three key contributions. The first is the introduction of a new image representation called the “Integral Image” which allows the features used by our detector to be computed very quickly. The second is a simple and efficient classifier which is built using the AdaBoost learning algorithm (Freund and Schapire, 1995) to select a small number of critical visual features from a very large set of potential features. The third contribution is a method for combining classifiers in a “cascade” which allows background regions of the image to be quickly discarded while spending more computation on promising face-like regions. A set of experiments in the domain of face detection is presented. The system yields face detection performance comparable to the best previous systems (Sung and Poggio, 1998; Rowley et al., 1998; Schneiderman and Kanade, 2000; Roth et al., 2000). Implemented on a conventional desktop, face detection proceeds at 15 frames per second.
13,037 citations
TL;DR: A face recognition algorithm which is insensitive to large variation in lighting direction and facial expression is developed, based on Fisher's linear discriminant and produces well separated classes in a low-dimensional subspace, even under severe variations in lighting and facial expressions.
Abstract: We develop a face recognition algorithm which is insensitive to large variation in lighting direction and facial expression. Taking a pattern classification approach, we consider each pixel in an image as a coordinate in a high-dimensional space. We take advantage of the observation that the images of a particular face, under varying illumination but fixed pose, lie in a 3D linear subspace of the high dimensional image space-if the face is a Lambertian surface without shadowing. However, since faces are not truly Lambertian surfaces and do indeed produce self-shadowing, images will deviate from this linear subspace. Rather than explicitly modeling this deviation, we linearly project the image into a subspace in a manner which discounts those regions of the face with large deviation. Our projection method is based on Fisher's linear discriminant and produces well separated classes in a low-dimensional subspace, even under severe variation in lighting and facial expressions. The eigenface technique, another method based on linearly projecting the image space to a low dimensional subspace, has similar computational requirements. Yet, extensive experimental results demonstrate that the proposed "Fisherface" method has error rates that are lower than those of the eigenface technique for tests on the Harvard and Yale face databases.
11,674 citations
07 Jun 2015
TL;DR: A system that directly learns a mapping from face images to a compact Euclidean space where distances directly correspond to a measure offace similarity, and achieves state-of-the-art face recognition performance using only 128-bytes perface.
Abstract: Despite significant recent advances in the field of face recognition [10, 14, 15, 17], implementing face verification and recognition efficiently at scale presents serious challenges to current approaches. In this paper we present a system, called FaceNet, that directly learns a mapping from face images to a compact Euclidean space where distances directly correspond to a measure of face similarity. Once this space has been produced, tasks such as face recognition, verification and clustering can be easily implemented using standard techniques with FaceNet embeddings as feature vectors.
8,289 citations
TL;DR: This paper presents a novel and efficient facial image representation based on local binary pattern (LBP) texture features that is assessed in the face recognition problem under different challenges.
Abstract: This paper presents a novel and efficient facial image representation based on local binary pattern (LBP) texture features. The face image is divided into several regions from which the LBP feature distributions are extracted and concatenated into an enhanced feature vector to be used as a face descriptor. The performance of the proposed method is assessed in the face recognition problem under different challenges. Other applications and several extensions are also discussed
5,563 citations
"Soft-Margin Learning for Multiple F..." refers background in this paper
..., Eigenfaces [29], Fisherfaces [7], Weberfaces [31], Local binary pattern (LBP) [1], Gaborfaces [17], Bag-of-words (BOW) [11], Fisher vector encoding on dense-SIFT features (FV-SIFT) [20] and VLAD encoding on dense-SIFT features (VLAD-SIFT) [3]....
[...]
...Several recent state-of-the art as-well-as traditional features are taken into account viz., Eigenfaces [29], Fisherfaces [7], Weberfaces [31], Local binary pattern (LBP) [1], Gaborfaces [17], Bag-of-words (BOW) [11], Fisher vector encoding on dense-SIFT features (FV-SIFT) [20] and VLAD encoding on dense-SIFT features (VLAD-SIFT) [3]....
[...]