“Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks”の学習報告

This paper details MORPH a longitudinal face database developed for researchers investigating all facets of adult age-progression, e.g. face modeling, photo-realistic animation, face recognition, etc. This database contributes to several active research areas, most notably face recognition, by providing: the largest set of publicly available longitudinal images; longitudinal spans from a few months to over twenty years; and, the inclusion of key physical parameters that affect aging appearance. The direct contribution of this data corpus for face recognition is highlighted in the evaluation of a standard face recognition algorithm, which illustrates the impact that age-progression, has on recognition rates. Assessment of the efficacy of this algorithm is evaluated against the variables of gender and racial origin. This work further concludes that the problem of age-progression on face recognition (FR) is not unique to the algorithm used in this work.

MORPH: A Longitudinal Image Database of Normal Adult Age-Progression.

Face recognition is one of the most fundamental and long-standing topics in computer vision community. With the recent developments of deep convolutional neural networks and large-scale datasets, deep face recognition has made remarkable progress and been widely used in the real-world applications. Given a natural image or video frame as input, an end-to-end deep face recognition system outputs the face feature for recognition. To achieve this, the whole system is generally built with three key elements: face detection, face preprocessing, and face representation. The face detection locates faces in the image or frame. Then, the face preprocessing is proceeded to calibrate the faces to a canonical view and crop them to a normalized pixel size. Finally, in the stage of face representation, the discriminative features are extracted from the preprocessed faces for recognition. All of the three elements are fulfilled by deep convolutional neural networks. In this paper, we present a comprehensive survey about the recent advances of every element of the end-to-end deep face recognition, since the thriving deep learning techniques have greatly improved the capability of them. To start with, we introduce an overview of the end-to-end deep face recognition, which, as mentioned above, includes face detection, face preprocessing, and face representation. Then, we review the deep learning based advances of each element, respectively, covering many aspects such as the up-to-date algorithm designs, evaluation metrics, datasets, performance comparison, existing challenges, and promising directions for future research. We hope this survey could bring helpful thoughts to one for better understanding of the big picture of end-to-end face recognition and deeper exploration in a systematic way.

The Elements of End-to-end Deep Face Recognition: A Survey of Recent Advances

In recent years, researchers have proposed many deep learning (DL) methods for various tasks, and particularly face recognition (FR) made an enormous leap using these techniques. Deep FR systems benefit from the hierarchical architecture of the DL methods to learn discriminative face representation. Therefore, DL techniques significantly improve state-of-the-art performance on FR systems and encourage diverse and efficient real-world applications. In this paper, we present a comprehensive analysis of various FR systems that leverage the different types of DL techniques, and for the study, we summarize 171 recent contributions from this area. We discuss the papers related to different algorithms, architectures, loss functions, activation functions, datasets, challenges, improvement ideas, current and future trends of DL-based FR systems. We provide a detailed discussion of various DL methods to understand the current state-of-the-art, and then we discuss various activation and loss functions for the methods. Additionally, we summarize different datasets used widely for FR tasks and discuss challenges related to illumination, expression, pose variations, and occlusion. Finally, we discuss improvement ideas, current and future trends of FR tasks.

Recent Advances in Deep Learning Techniques for Face Recognition

Heart disease is the leading cause of death globally. The most common type of heart disease is coronary heart disease, which occurs when there is a build-up of plaque inside the arteries that supply blood to the heart, making blood circulation difficult. The prediction of heart disease is a challenge in clinical machine learning. Early detection of people at risk of the disease is vital in preventing its progression. This paper proposes a deep learning approach to achieve improved prediction of heart disease. An enhanced stacked sparse autoencoder network (SSAE) is developed to achieve efficient feature learning. The network consists of multiple sparse autoencoders and a softmax classifier. Additionally, in deep learning models, the algorithm’s parameters need to be optimized appropriately to obtain efficient performance. Hence, we propose a particle swarm optimization (PSO) based technique to tune the parameters of the stacked sparse autoencoder. The optimization by the PSO improves the feature learning and classification performance of the SSAE. Meanwhile, the multilayer architecture of autoencoders usually leads to internal covariate shift, a problem that affects the generalization ability of the network; hence, batch normalization is introduced to prevent this problem. The experimental results show that the proposed method effectively predicts heart disease by obtaining a classification accuracy of 0.973 and 0.961 on the Framingham and Cleveland heart disease datasets, respectively, thereby outperforming other machine learning methods and similar studies.

Improved Heart Disease Prediction Using Particle Swarm Optimization Based Stacked Sparse Autoencoder

It is well known that expression, pose variations, and especially age factors always affect the performance of a face recognition system in practical conditions. In this paper, we propose a novel framework called age factor removal network (AFRN) for cross-age face recognition, which combines the concepts of transfer learning and adversarial learning to enhance the performance of a pretrained face recognition network and suppress the influence of attributes, such as aging, expression, and pose variations. Similar to the generative adversarial network (GAN), our model consists of two networks: an identity feature generator G and an age discriminator D. First, the D network is trained to discriminate the age information from the feature extracted by G. Second, the G network simultaneously learns two tasks: extracting the feature through transfer learning from a pretrained face recognition network and suppressing age information through adversarial learning with D. Through the optimization of the two networks, the age factor in feature extraction process of G is removed. Besides, our network only requires the attributes' label to train and preserves the identity discriminant power through transfer learning, and thus, it does not depend on multi-label face databases. The extensive experiments have been performed on the benchmark cross-age face datasets, including MORPH Album2, CACD-VS, and Cross Age LFW, which verify the effectiveness of our model. What is more, our model can be extended to other practical face recognition tasks and decreases the influence of expressions and poses, which is verified by the experimental results on CMU Multi-PIE database.

Age Factor Removal Network Based on Transfer Learning and Adversarial Learning for Cross-Age Face Recognition

Age variations bring a large challenge for face recognition tasks. Existing Cross-Age Face Recognition (CAFR) methods have two limitations. Firstly, many CAFR approaches require both age labels and identity labels for training. However, it is difficult to collect images under a large age span from each individual. Secondly, many works are based on the assumption that age and identity information are independent of each other, which may not satisfy various conditions. In this paper, a Cycle Age-Adversarial Model (CAAM) is proposed for CAFR, which only uses the age labels for training without considering independence hypothesis. CAAM includes two different branch networks. Firstly, the branch of Age-robust Feature Extracting Model (AFEM) is designed to adaptively learn age-invariant features by adversarial learning scheme, which includes an age discriminator network and a feature generator network. The age discriminator network is trained to discriminate the age information, and the generator extracts age-invariant features through adversarial learning with discriminator. Secondly, a branch of the Identity Preserving Network (IPN) is proposed to keep identity information, which introduces Unsupervised Identity Loss (UIL) to enlarge the inter-class distance, and decrease the loss of identity information in the learning process. Finally, the features of the two branches are cyclically optimized through minmizing Feature Consistency Loss (FCL), which integrates age invariance learning and identity discrimination learning into final feature representation. Different from existing CAFR networks, our adversarial learning strategy for age-robust feature learning can be generalized to other attributes including pose and expression. Moreover, we introduce cycle optimization strategy to merge the advantages of two branch networks, which is a novel strategy to fuse multi-task features. Extensive CAFR experiments performed on the benchmark MORPH Album2, CACD-VS and Cross Age LFW databases demonstrate the effectiveness and superiority of CAAM.

Cycle Age-Adversarial Model Based on Identity Preserving Network and Transfer Learning for Cross-Age Face Recognition

This study presents a novel method for solving facial expression recognition (FER) tasks which uses a self-adaptive weighted synthesised local directional pattern (SW-SLDP) descriptor integrating sparse autoencoder (SA) features based on improved multiple kernel learning (IMKL) strategy. The authors' work includes three parts. Firstly, the authors propose a novel SW-SLDP feature descriptor which divides the facial images into patches and extracts sub-block features synthetically according to both distribution information and directional intensity contrast. Then self-adaptive weights are assigned to each sub-block feature according to the projection error between the expressional image and neutral image of each patch, which can highlight such areas containing more expressional texture information. Secondly, to extract a discriminative high-level feature, they introduce SA for feature representation, which extracts the hidden layer representation including more comprehensive information. Finally, to combine the above two kinds of features, an IMKL strategy is developed by effectively integrating both soft margin learning and intrinsic local constraints, which is robust to noisy condition and thus improve the classification performance. Extensive experimental results indicate their model can achieve competitive or even better performance with existing representative FER methods.

https://ietresearch.onlinelibrary.wiley.com/doi/epdf/10.1049/iet-cvi.2018.5127

Self-adaptive weighted synthesised local directional pattern integrating with sparse autoencoder for expression recognition based on improved multiple kernel learning strategy

Cross Age Face Recognition (CAFR) is a challenging task in the field of face recognition. There still exist some limitations in mainstream CAFR methods. On one hand, some methods need synthesizing multiple groups of features and fusing the results under different ages, where the similarity score in some age groups may affect final classification results. On the other hand, many other methods takes a facial image as a linear combination of identity information and age information, which treats age factor as a value independent of identity information, but may be inconsistent with the aging pattern of many individuals. And these methods require both age labels and identity labels in training, which is limited by the scale of existing CAFR datasets. To address the above limitations, this work proposes the Parallel Multi-path Age Distinguish Network (PMADN) model. Specifically, our model consists of two cascading networks, an Age Distinguish Mapping Network (ADMN) and a Cross-Age Feature Recombination Network (CFRN). Firstly, the face features are mapped into different age groups by parallel multi-path full connected layers in ADMN, which can better extract the identity features in a small age span. Secondly, CFRN nonlinearly recombines the mapped features to extract the age robust features that are beneficial to identity classification, which can avoid the simple linear combination of identity factor and age factor in the existing methods. What’s more, our algorithm is combined with transfer learning and only uses the age label and a pre-trained ordinary face recognition network for training, which can make use of a larger aging face dataset for training. Extensive CAFR experiments performed on the benchmark MORPH Album2, CACD-VS and Cross Age LFW databases demonstrate the effectiveness and superiority of our method.

Parallel Multi-Path Age Distinguish Network for Cross-Age Face Recognition

The invention provides an expression recognition method of antagonism enhancement. The expression recognition network comprises a feature encoder and a feature classifier. The antagonism network is generated comprising a feature encoder, a feature decoder and an discriminator. Firstly, a facial expression and an expressionless picture set are constructed, and the pictures of the picture set are subjected to posture normalization processing. The facial expression picture is input into the feature encoder of the facial expression recognition network to obtain the facial expression feature encoding, and then the facial expression feature encoding is input into the feature classifier for classification output. The expressionless picture is inputted into a feature encoder which generates an antagonistic network for encoding, and the expressionless feature encoding is obtained. The two feature codes and the Gaussian noise are serially connected to obtain a fusion feature code which is inputto a feature decoder to output a new expression picture. The new facial expression image is taken as negative sample and the input facial expression image is taken as positive sample. The discriminator discriminates the class of the new facial expression image and repeats the above steps to train the facial expression recognition network and the generated antagonism network alternately and updatethe parameters.

Yongbo Wu

Papers

Age Factor Removal Network Based on Transfer Learning and Adversarial Learning for Cross-Age Face Recognition

Cycle Age-Adversarial Model Based on Identity Preserving Network and Transfer Learning for Cross-Age Face Recognition

Self-adaptive weighted synthesised local directional pattern integrating with sparse autoencoder for expression recognition based on improved multiple kernel learning strategy

Parallel Multi-Path Age Distinguish Network for Cross-Age Face Recognition

An expression recognition method of antagonism enhancement