Memetically Optimized MCWLD for Matching Sketches With Digital Face Images
01 Oct 2012-IEEE Transactions on Information Forensics and Security (IEEE)-Vol. 7, Iss: 5, pp 1522-1535
TL;DR: An automated algorithm to extract discriminating information from local regions of both sketches and digital face images is presented and yields better identification performance compared to existing face recognition algorithms and two commercial face recognition systems.
Abstract: One of the important cues in solving crimes and apprehending criminals is matching sketches with digital face images. This paper presents an automated algorithm to extract discriminating information from local regions of both sketches and digital face images. Structural information along with minute details present in local facial regions are encoded using multiscale circular Weber's local descriptor. Further, an evolutionary memetic optimization algorithm is proposed to assign optimal weight to every local facial region to boost the identification performance. Since forensic sketches or digital face images can be of poor quality, a preprocessing technique is used to enhance the quality of images and improve the identification performance. Comprehensive experimental evaluation on different sketch databases show that the proposed algorithm yields better identification performance compared to existing face recognition algorithms and two commercial face recognition systems.
Citations
More filters
17 Jul 2019
TL;DR: This paper proposes a new two-branch network architecture, termed as Residual Compensation Networks (RCN), to learn separated features for different modalities in HFR, which outperforms other state-of-the-art methods significantly.
Abstract: Heterogeneous Face Recognition (HFR) is a challenging task due to large modality discrepancy as well as insufficient training images in certain modalities. In this paper, we propose a new two-branch network architecture, termed as Residual Compensation Networks (RCN), to learn separated features for different modalities in HFR. The RCN incorporates a residual compensation (RC) module and a modality discrepancy loss (MD loss) into traditional convolutional neural networks. The RC module reduces modal discrepancy by adding compensation to one of the modalities so that its representation can be close to the other modality. The MD loss alleviates modal discrepancy by minimizing the cosine distance between different modalities. In addition, we explore different architectures and positions for the RC module, and evaluate different transfer learning strategies for HFR. Extensive experiments on IIIT-D Viewed Sketch, Forensic Sketch, CASIA NIR-VIS 2.0 and CUHK NIR-VIS show that our RCN outperforms other state-of-the-art methods significantly.
34 citations
Cites methods from "Memetically Optimized MCWLD for Mat..."
...In this section, we evaluate our proposed RCN on IIIT-D Viewed Sketch (Bhatt et al. 2012b), Forensic Sketch (Klare, Li, and Jain 2011), CASIA NIR-VIS 2....
[...]
...Our RCN outperforms those methods with large margin, i.e. 6.1% higher than MCWLD, 4.99% higher than CDL and 6.27% higher than Center Loss and Light CNN....
[...]
...In this section, we evaluate our proposed RCN on IIIT-D Viewed Sketch (Bhatt et al. 2012b), Forensic Sketch (Klare, Li, and Jain 2011), CASIA NIR-VIS 2.0 (Li et al. 2013) and CUHK NIR-VIS....
[...]
...They only show slightly superior to hand-crafted feature based method MCWLD (Bhatt et al. 2012a)....
[...]
TL;DR: A matching method on the basis of the Histograms of Oriented Gradients (HOG) descriptor and Principal Component Analysis (PCA) to handle the similarities between a forensic sketch and a synthesized pseudo-sketch, called HOG-PCA is proposed.
Abstract: In this paper, we propose a simple but yet effective method for synthesizing a pseudo face sketch (pseudo-sketch) from a photo, to be used for face recognition based on sketches drawn by a forensic artist. In contrast to current methods, the proposed method does not require training samples while fairly maintains the salient facial features as the artist do. We also propose a matching method on the basis of the Histograms of Oriented Gradients (HOG) descriptor and Principal Component Analysis (PCA), called HOG-PCA, to handle the similarities between a forensic sketch and a synthesized pseudo-sketch. In this method, we first extract the HOG features for the sketch and pseudo-sketch at regular grid and overlapped patches. The PCA is then applied to address the redundancy in feature representation due to several overlapped patches. Finally, the Nearest Neighbors Classifier (NNC) with the cosine distance is used to classify the sketch and pseudo-sketch pairs as matched or mismatched. Experimental results on CUHK and AR face sketch databases demonstrate that our proposed methods outperform state-of-the-art methods.
26 citations
Cites methods from "Memetically Optimized MCWLD for Mat..."
...In [4], a technique for matching a sketch with a photo using the modified Webers local descriptor [7] was also developed....
[...]
...ing between the sketch and photo were conducted [2, 4, 14]....
[...]
TL;DR: In this paper, a graph-structured module called Relational Graph Module (RGM) was proposed to extract global relational information in addition to general facial features, which can help cross-domain matching.
Abstract: Heterogeneous Face Recognition (HFR) is a task that matches faces across two different domains such as visible light (VIS), near-infrared (NIR), or the sketch domain. Due to the lack of databases, HFR methods usually exploit the pre-trained features on a large-scale visual database that contain general facial information. However, these pre-trained features cause performance degradation due to the texture discrepancy with the visual domain. With this motivation, we propose a graph-structured module called Relational Graph Module (RGM) that extracts global relational information in addition to general facial features. Because each identity's relational information between intra-facial parts is similar in any modality, the modeling relationship between features can help cross-domain matching. Through the RGM, relation propagation diminishes texture dependency without losing its advantages from the pre-trained features. Furthermore, the RGM captures global facial geometrics from locally correlated convolutional features to identify long-range relationships. In addition, we propose a Node Attention Unit (NAU) that performs node-wise recalibration to concentrate on the more informative nodes arising from relation-based propagation. Furthermore, we suggest a novel conditional-margin loss function (C-softmax) for the efficient projection learning of the embedding vector in HFR. The proposed method outperforms other state-of-the-art methods on five HFR databases. Furthermore, we demonstrate performance improvement on three backbones because our module can be plugged into any pre-trained face recognition backbone to overcome the limitations of a small HFR database.
25 citations
01 Oct 2017
TL;DR: DeepTransformer as mentioned in this paper learns a transformation and mapping function between the features of two domains, which can be applied with any existing learned or hand-crafted feature and can be used for sketch-to-sketch matching.
Abstract: Face sketch to digital image matching is an important challenge of face recognition that involves matching across different domains. Current research efforts have primarily focused on extracting domain invariant representations or learning a mapping from one domain to the other. In this research, we propose a novel transform learning based approach termed as DeepTransformer, which learns a transformation and mapping function between the features of two domains. The proposed formulation is independent of the input information and can be applied with any existing learned or hand-crafted feature. Since the mapping function is directional in nature, we propose two variants of DeepTransformer: (i) semi-coupled and (ii) symmetrically-coupled deep transform learning. This research also uses a novel IIIT-D Composite Sketch with Age (CSA) variations database which contains sketch images of 150 subjects along with age-separated digital photos. The performance of the proposed models is evaluated on a novel application of sketch-to-sketch matching, along with sketch-to-digital photo matching. Experimental results demonstrate the robustness of the proposed models in comparison to existing state-of-the-art sketch matching algorithms and a commercial face recognition system.
25 citations
01 Jun 2018
TL;DR: A new loss function, called attribute-centered loss, is proposed to train a Deep Coupled Convolutional Neural Network (DCCNN) for facial attribute guided sketch to photo matching, which significantly outperforms the state-of-the-art.
Abstract: Face sketches are able to capture the spatial topology of a face while lacking some facial attributes such as race, skin, or hair color. Existing sketch-photo recognition approaches have mostly ignored the importance of facial attributes. In this paper, we propose a new loss function, called attribute-centered loss, to train a Deep Coupled Convolutional Neural Network (DCCNN) for facial attribute guided sketch to photo matching. Specifically, an attribute-centered loss is proposed which learns several distinct centers, in a shared embedding space, for photos and sketches with different combinations of attributes. The DCCNN simultaneously is trained to map photos and pairs of testified attributes and corresponding forensic sketches around their associated centers, while preserving the spatial topology information. Importantly, the centers learn to keep a relative distance from each other, related to their number of contradictory attributes. Extensive experiments are performed on composite (E-PRIP) and semi-forensic (IIIT-D Semi-forensic) databases. The proposed method significantly outperforms the state-of-the-art.
24 citations
Cites methods from "Memetically Optimized MCWLD for Mat..."
...On the other hand, discriminative approaches perform feature extraction, such as the scale-invariant feature transform (SIFT) [17], Weber’s local descriptor (WLD) [3], and multiscale local binary pattern (MLBP) [8]....
[...]
References
More filters
TL;DR: This paper presents a method for extracting distinctive invariant features from images that can be used to perform reliable matching between different views of an object or scene and can robustly identify objects among clutter and occlusion while achieving near real-time performance.
Abstract: This paper presents a method for extracting distinctive invariant features from images that can be used to perform reliable matching between different views of an object or scene. The features are invariant to image scale and rotation, and are shown to provide robust matching across a substantial range of affine distortion, change in 3D viewpoint, addition of noise, and change in illumination. The features are highly distinctive, in the sense that a single feature can be correctly matched with high probability against a large database of features from many images. This paper also describes an approach to using these features for object recognition. The recognition proceeds by matching individual features to a database of features from known objects using a fast nearest-neighbor algorithm, followed by a Hough transform to identify clusters belonging to a single object, and finally performing verification through least-squares solution for consistent pose parameters. This approach to recognition can robustly identify objects among clutter and occlusion while achieving near real-time performance.
46,906 citations
01 Jan 2011
TL;DR: The Scale-Invariant Feature Transform (or SIFT) algorithm is a highly robust method to extract and consequently match distinctive invariant features from images that can then be used to reliably match objects in diering images.
Abstract: The Scale-Invariant Feature Transform (or SIFT) algorithm is a highly robust method to extract and consequently match distinctive invariant features from images. These features can then be used to reliably match objects in diering images. The algorithm was rst proposed by Lowe [12] and further developed to increase performance resulting in the classic paper [13] that served as foundation for SIFT which has played an important role in robotic and machine vision in the past decade.
14,708 citations
Additional excerpts
...On the other hand, sparse descriptor such as Scale Invariant Feature Transform (SIFT ) [23] is based on interest point detection and computing the descriptor in the vicinity of detected interest points....
[...]
TL;DR: In this article, the regularity of compactly supported wavelets and symmetry of wavelet bases are discussed. But the authors focus on the orthonormal bases of wavelets, rather than the continuous wavelet transform.
Abstract: Introduction Preliminaries and notation The what, why, and how of wavelets The continuous wavelet transform Discrete wavelet transforms: Frames Time-frequency density and orthonormal bases Orthonormal bases of wavelets and multiresolutional analysis Orthonormal bases of compactly supported wavelets More about the regularity of compactly supported wavelets Symmetry for compactly supported wavelet bases Characterization of functional spaces by means of wavelets Generalizations and tricks for orthonormal wavelet bases References Indexes.
14,157 citations
TL;DR: This paper presents a novel and efficient facial image representation based on local binary pattern (LBP) texture features that is assessed in the face recognition problem under different challenges.
Abstract: This paper presents a novel and efficient facial image representation based on local binary pattern (LBP) texture features. The face image is divided into several regions from which the LBP feature distributions are extracted and concatenated into an enhanced feature vector to be used as a face descriptor. The performance of the proposed method is assessed in the face recognition problem under different challenges. Other applications and several extensions are also discussed
5,563 citations