Memetically Optimized MCWLD for Matching Sketches With Digital Face Images
01 Oct 2012-IEEE Transactions on Information Forensics and Security (IEEE)-Vol. 7, Iss: 5, pp 1522-1535
TL;DR: An automated algorithm to extract discriminating information from local regions of both sketches and digital face images is presented and yields better identification performance compared to existing face recognition algorithms and two commercial face recognition systems.
Abstract: One of the important cues in solving crimes and apprehending criminals is matching sketches with digital face images. This paper presents an automated algorithm to extract discriminating information from local regions of both sketches and digital face images. Structural information along with minute details present in local facial regions are encoded using multiscale circular Weber's local descriptor. Further, an evolutionary memetic optimization algorithm is proposed to assign optimal weight to every local facial region to boost the identification performance. Since forensic sketches or digital face images can be of poor quality, a preprocessing technique is used to enhance the quality of images and improve the identification performance. Comprehensive experimental evaluation on different sketch databases show that the proposed algorithm yields better identification performance compared to existing face recognition algorithms and two commercial face recognition systems.
Citations
More filters
TL;DR: In this article, a graphical representation based HFR method (G-HFR) is proposed to represent heterogeneous image patches separately, which takes the spatial compatibility between neighboring image patches into consideration.
Abstract: Heterogeneous face recognition (HFR) refers to matching face images acquired from different sources (i.e., different sensors or different wavelengths) for identification. HFR plays an important role in both biometrics research and industry. In spite of promising progresses achieved in recent years, HFR is still a challenging problem due to the difficulty to represent two heterogeneous images in a homogeneous manner. Existing HFR methods either represent an image ignoring the spatial information, or rely on a transformation procedure which complicates the recognition task. Considering these problems, we propose a novel graphical representation based HFR method (G-HFR) in this paper. Markov networks are employed to represent heterogeneous image patches separately, which takes the spatial compatibility between neighboring image patches into consideration. A coupled representation similarity metric (CRSM) is designed to measure the similarity between obtained graphical representations. Extensive experiments conducted on multiple HFR scenarios (viewed sketch, forensic sketch, near infrared image, and thermal infrared image) show that the proposed method outperforms state-of-the-art methods.
122 citations
08 Oct 2016
TL;DR: Experimental results show that CNNs trained on visible spectrum images can be used to obtain results that are on par or improve over the state-of-the-art for heterogeneous recognition with near-infrared images and sketches.
Abstract: Heterogeneous face recognition aims to recognize faces across different sensor modalities. Typically, gallery images are normal visible spectrum images, and probe images are infrared images or sketches. Recently significant improvements in visible spectrum face recognition have been obtained by CNNs learned from very large training datasets. In this paper, we are interested in the question to what extent the features from a CNN pre-trained on visible spectrum face images can be used to perform heterogeneous face recognition. We explore different metric learning strategies to reduce the discrepancies between the different modalities. Experimental results show that we can use CNNs trained on visible spectrum images to obtain results that are on par or improve over the state-of-the-art for heterogeneous recognition with near-infrared images and sketches.
115 citations
TL;DR: A novel multiple representations-based face sketch-photo-synthesis method that adaptively combines multiple representations to represent an image patch that combines multiple features from face images processed using multiple filters and deploys Markov networks to exploit the interacting relationships between the neighboring image patches.
Abstract: Face sketch–photo synthesis plays an important role in law enforcement and digital entertainment. Most of the existing methods only use pixel intensities as the feature. Since face images can be described using features from multiple aspects, this paper presents a novel multiple representations-based face sketch–photo-synthesis method that adaptively combines multiple representations to represent an image patch. In particular, it combines multiple features from face images processed using multiple filters and deploys Markov networks to exploit the interacting relationships between the neighboring image patches. The proposed framework could be solved using an alternating optimization strategy and it normally converges in only five outer iterations in the experiments. Our experimental results on the Chinese University of Hong Kong (CUHK) face sketch database, celebrity photos, CUHK Face Sketch FERET Database, IIIT-D Viewed Sketch Database, and forensic sketches demonstrate the effectiveness of our method for face sketch–photo synthesis. In addition, cross-database and database-dependent style-synthesis evaluations demonstrate the generalizability of this novel method and suggest promising solutions for face identification in forensic science.
115 citations
Cites methods from "Memetically Optimized MCWLD for Mat..."
...Thus, we utilize RS-LDA as the classifier and compared the proposed strategy with three recent methods [17], [56], [57]....
[...]
...The same protocol, as in [56], was followed, and our method achieves a rank-50 accuracy of 37....
[...]
...The method in [56] utilized modified Weber’s local descriptor and memetic optimization....
[...]
TL;DR: This survey provides a comprehensive review of established techniques and recent developments in HFR, and offers a detailed account of datasets and benchmarks commonly used for evaluation.
114 citations
Cites background from "Memetically Optimized MCWLD for Mat..."
...These mid-level facial features were manually annotated for each image, and used together with automatically extracted LBP [112] features....
[...]
...Feature design strategies [29, 30, 31, 32] focus on engineering or learning features that are invariant to the modalities in question, while simultaneously being discriminative for person identity....
[...]
...Cross-domain Strategies Feature-centric crossdomain strategies [29, 30, 31, 32, 11, 27, 34, 33, 21, 39] can be seen as designing improved feature extractors...
[...]
...The most widely used image feature descriptors are Scale-invariant feature transform (SIFT), Gabor transform, Histogram of Averaged Oriented Gradients (HAOG) and Local Binary Pattern (LBP)....
[...]
...Multi-class (Tr) Bayesian [8], Metric learning [32] Metric learning [53] SVM [48]...
[...]
TL;DR: An automated algorithm is developed to verify the faces presented under disguise variations using automatically localized feature descriptors which can identify disguised face patches and account for this information to achieve improved matching accuracy.
Abstract: Face verification, though an easy task for humans, is a long-standing open research area. This is largely due to the challenging covariates, such as disguise and aging, which make it very hard to accurately verify the identity of a person. This paper investigates human and machine performance for recognizing/verifying disguised faces. Performance is also evaluated under familiarity and match/mismatch with the ethnicity of observers. The findings of this study are used to develop an automated algorithm to verify the faces presented under disguise variations. We use automatically localized feature descriptors which can identify disguised face patches and account for this information to achieve improved matching accuracy. The performance of the proposed algorithm is evaluated on the IIIT-Delhi Disguise database that contains images pertaining to 75 subjects with different kinds of disguise variations. The experiments suggest that the proposed algorithm can outperform a popular commercial system and evaluates them against humans in matching disguised face images.
110 citations
Cites methods from "Memetically Optimized MCWLD for Mat..."
...Earlier research has primarily focused on the challenges or covariates of pose, illumination and expression whereas recently, face alterations due to plastic surgery [9], sketch-to-photo matching [10,11], multi-spectrum matching [12–14], aging [15–17], and disguise [18–20] are also being explored....
[...]
References
More filters
01 Sep 1996
TL;DR: A multi-scale retinex (MSR) which overcomes this limitation for most scenes and both color rendition and dynamic range compression are successfully accomplished except for some "pathological" scenes that have very strong spectral characteristics in a single band.
Abstract: The retinex is a human perception-based image processing algorithm which provides color constancy and dynamic range compression. We have previously reported on a single-scale retinex (SSR) and shown that it can either achieve color/lightness rendition or dynamic range compression, but not both simultaneously. We now present a multi-scale retinex (MSR) which overcomes this limitation for most scenes. Both color rendition and dynamic range compression are successfully accomplished except for some "pathological" scenes that have very strong spectral characteristics in a single band.
560 citations
"Memetically Optimized MCWLD for Mat..." refers methods in this paper
...MCWLD histograms for every region are then concatenated to form the facial representation....
[...]
TL;DR: This paper is going to study a method for representing face which is based on the features which uses geometric relationship among the facial features like mouth, nose and eyes called Principal Component Analysis followed by Feed Forward Neural Network called PCA-NN.
Abstract: Today in Modern Society Face Recognition has gained much attention in the field of network multimedia access. After the 9/11 tragedy in India, the need for technologies for identification, detection and recognition of suspects has increased. One of the most common biometric recognition techniques is face recognition since face is the convenient way used by the people to identify each other. In this paper we are going to study a method for representing face which is based on the features which uses geometric relationship among the facial features like mouth, nose and eyes .Feature based face representation is done by independently matching templates of three facial regions i.e eyes, mouth and nose .Principal Component Analysis method which is also called Eigen faces is appearance based technique used widely for the dimensionality reduction and recorded a greater performance in face recognition. Here we are going to study about PCA followed by Feed Forward Neural Network called PCA-NN.
485 citations
"Memetically Optimized MCWLD for Mat..." refers background in this paper
...Wang and Tang [6] proposed Markov Random Fields based algorithm to automatically synthesize sketches from digital face images and vice-versa....
[...]
TL;DR: A generic HFR framework is proposed in which both probe and gallery images are represented in terms of nonlinear similarities to a collection of prototype face images, and Random sampling is introduced into the H FR framework to better handle challenges arising from the small sample size problem.
Abstract: Heterogeneous face recognition (HFR) involves matching two face images from alternate imaging modalities, such as an infrared image to a photograph or a sketch to a photograph. Accurate HFR systems are of great value in various applications (e.g., forensics and surveillance), where the gallery databases are populated with photographs (e.g., mug shot or passport photographs) but the probe images are often limited to some alternate modality. A generic HFR framework is proposed in which both probe and gallery images are represented in terms of nonlinear similarities to a collection of prototype face images. The prototype subjects (i.e., the training set) have an image in each modality (probe and gallery), and the similarity of an image is measured against the prototype images from the corresponding modality. The accuracy of this nonlinear prototype representation is improved by projecting the features into a linear discriminant subspace. Random sampling is introduced into the HFR framework to better handle challenges arising from the small sample size problem. The merits of the proposed approach, called prototype random subspace (P-RS), are demonstrated on four different heterogeneous scenarios: 1) near infrared (NIR) to photograph, 2) thermal to photograph, 3) viewed sketch to photograph, and 4) forensic sketch to photograph.
358 citations
"Memetically Optimized MCWLD for Mat..." refers background in this paper
...In their recent approach, Klare and Jain [15] proposed a framework for heterogeneous face recognition where both probe and gallery images are represented in terms of non-linear kernel similarities....
[...]
20 Jun 2005
TL;DR: This paper presents a face recognition system based on face sketches that is based on pseudo-sketch synthesis and sketch recognition, and experimental results show that the performance of the proposed method is encouraging.
Abstract: Most face recognition systems focus on photo-based face recognition. In this paper, we present a face recognition system based on face sketches. The proposed system contains two elements: pseudo-sketch synthesis and sketch recognition. The pseudo-sketch generation method is based on local linear preserving of geometry between photo and sketch images, which is inspired by the idea of locally linear embedding. The nonlinear discriminate analysis is used to recognize the probe sketch from the synthesized pseudo-sketches. Experimental results on over 600 photo-sketch pairs show that the performance of the proposed method is encouraging.
352 citations
"Memetically Optimized MCWLD for Mat..." refers background in this paper
...Liu et al. [3] proposed non-linear discriminative classifier based approach for synthesizing sketches by preserving face geometry....
[...]
20 Jun 2011
TL;DR: A new face descriptor based on coupled information-theoretic encoding is used to capture discriminative local face structures and to effectively match photos and sketches by reducing the modality gap at the feature extraction stage.
Abstract: Automatic face photo-sketch recognition has important applications for law enforcement. Recent research has focused on transforming photos and sketches into the same modality for matching or developing advanced classification algorithms to reduce the modality gap between features extracted from photos and sketches. In this paper, we propose a new inter-modality face recognition approach by reducing the modality gap at the feature extraction stage. A new face descriptor based on coupled information-theoretic encoding is used to capture discriminative local face structures and to effectively match photos and sketches. Guided by maximizing the mutual information between photos and sketches in the quantized feature spaces, the coupled encoding is achieved by the proposed coupled information-theoretic projection tree, which is extended to the randomized forest to further boost the performance. We create the largest face sketch database including sketches of 1, 194 people from the FERET database. Experiments on this large scale dataset show that our approach significantly outperforms the state-of-the-art methods.
338 citations
"Memetically Optimized MCWLD for Mat..." refers background in this paper
...Recently, Zhanget al. [17] proposed an information theoretic encoding band descriptor to capture discriminative information and random forest based matching to maximize the mutual information between a sketch and a photo....
[...]