“Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks”の学習報告

This paper details MORPH a longitudinal face database developed for researchers investigating all facets of adult age-progression, e.g. face modeling, photo-realistic animation, face recognition, etc. This database contributes to several active research areas, most notably face recognition, by providing: the largest set of publicly available longitudinal images; longitudinal spans from a few months to over twenty years; and, the inclusion of key physical parameters that affect aging appearance. The direct contribution of this data corpus for face recognition is highlighted in the evaluation of a standard face recognition algorithm, which illustrates the impact that age-progression, has on recognition rates. Assessment of the efficacy of this algorithm is evaluated against the variables of gender and racial origin. This work further concludes that the problem of age-progression on face recognition (FR) is not unique to the algorithm used in this work.

MORPH: A Longitudinal Image Database of Normal Adult Age-Progression.

Face recognition is one of the most fundamental and long-standing topics in computer vision community. With the recent developments of deep convolutional neural networks and large-scale datasets, deep face recognition has made remarkable progress and been widely used in the real-world applications. Given a natural image or video frame as input, an end-to-end deep face recognition system outputs the face feature for recognition. To achieve this, the whole system is generally built with three key elements: face detection, face preprocessing, and face representation. The face detection locates faces in the image or frame. Then, the face preprocessing is proceeded to calibrate the faces to a canonical view and crop them to a normalized pixel size. Finally, in the stage of face representation, the discriminative features are extracted from the preprocessed faces for recognition. All of the three elements are fulfilled by deep convolutional neural networks. In this paper, we present a comprehensive survey about the recent advances of every element of the end-to-end deep face recognition, since the thriving deep learning techniques have greatly improved the capability of them. To start with, we introduce an overview of the end-to-end deep face recognition, which, as mentioned above, includes face detection, face preprocessing, and face representation. Then, we review the deep learning based advances of each element, respectively, covering many aspects such as the up-to-date algorithm designs, evaluation metrics, datasets, performance comparison, existing challenges, and promising directions for future research. We hope this survey could bring helpful thoughts to one for better understanding of the big picture of end-to-end face recognition and deeper exploration in a systematic way.

The Elements of End-to-end Deep Face Recognition: A Survey of Recent Advances

In this paper, we propose a progressive margin loss (PML) approach for unconstrained facial age classification. Conventional methods make strong assumption on that each class owns adequate instances to outline its data distribution, likely leading to bias prediction where the training samples are sparse across age classes. Instead, our PML aims to adaptively refine the age label pattern by enforcing a couple of margins, which fully takes in the in-between discrepancy of the intra-class variance, inter-class variance and class center. Our PML typically incorporates with the ordinal margin and the variational margin, simultaneously plugging in the globally-tuned deep neural network paradigm. More specifically, the ordinal margin learns to exploit the correlated relationship of the real-world age labels. Accordingly, the variational margin is leveraged to minimize the influence of head classes that misleads the prediction of tailed samples. Moreover, our optimization carefully seeks a series of indicator curricula to achieve robust and efficient model training. Extensive experimental results on three face aging datasets demonstrate that our PML achieves compelling performance compared to state of the art. Code will be made publicly.

/pdf/pml-progressive-margin-loss-for-long-tailed-age-1t71c3e8ii.pdf

PML: Progressive Margin Loss for Long-tailed Age Classification

Face recognition (FR) is one of the most popular and long-standing topics in computer vision. With the recent development of deep learning techniques and large-scale datasets, deep face recognition has made remarkable progress and has been widely used in many real-world applications. Given a natural image or video frame as input, an end-to-end deep face recognition system outputs the face feature for recognition. To achieve this, a typical end-to-end system is built with three key elements: face detection, face alignment, and face representation. Face detection locates faces in the image or frame. Then, the face alignment is proceeded to calibrate the faces to the canonical view and crop them with a normalized pixel size. Finally, in the stage of face representation, the discriminative features are extracted from the aligned face for recognition. Nowadays, all of the three elements are fulfilled by the technique of deep convolutional neural network. In this survey article, we present a comprehensive review about the recent advance of each element of the end-to-end deep face recognition, since the thriving deep learning techniques have greatly improved their capability of them. To start with, we present an overview of the end-to-end deep face recognition. Then, we review the advance of each element, respectively, covering many aspects such as the to-date algorithm designs, evaluation metrics, datasets, performance comparison, existing challenges, and promising directions for future research. Also, we provide a detailed discussion about the effect of each element on its subsequent elements and the holistic system. Through this survey, we wish to bring contributions in two aspects: first, readers can conveniently identify the methods which are quite strong-baseline style in the subcategory for further exploration; second, one can also employ suitable methods for establishing a state-of-the-art end-to-end face recognition system from scratch. 

https://dl.acm.org/doi/pdf/10.1145/3507902

In this paper, we propose a similarity-aware deep adversarial learning (SADAL) approach for facial age estimation. Instead of making full access to the limited training samples which likely leads to bias age prediction, our SADAL aims to seek batches of unobserved hard-negative samples based on existing training samples, which typically reinforces the discriminativeness of the learned feature representation for facial ages. Motivated by the fact that age labels are usually correlated in real-world scenarios, we carefully develop a similarity-aware function to well measure the distance of each face pair based on the age value gaps. Consequently, the age-difference information is exploited in the synthetic feature space for robust age estimation. During the learning process, we jointly optimize both procedures of generating hard negatives and learning discriminative age ranker via a sequence of adversarial-game iterations. Another major issue lies on that existing methods only enforce the indiscriminativeness within each class, which is probably trapped into model overfitting and thus the generation capacity is limited particularly on unseen age classes with many individuals. To circumvent this problem, we propose a variational deep adversarial learning (VDAL) paradigm, which learns to encode each face sample in two factorized parts, i.e., the intra-class variance distribution and the intra-class invariant class center. Moreover, our VDAL principally optimizes the variational confidence lower bound on the variational factorized feature representation. To better enhance the discriminativeness of the age representation, our VDAL further learns to encode the ordinal relationship among age labels in the reconstructed subspace. Experimental results on folds of widely-evaluated benchmarking datasets demonstrate that our approach achieves promising performance in contrast to most state-of-the-art age estimation methods.

Similarity-Aware and Variational Deep Adversarial Learning for Robust Facial Age Estimation

In this paper, we propose an end-to-end reasoning-decision networks (RDN) approach for robust face alignment via policy gradient. Unlike the conventional coarse-to-fine approaches which likely lead to bias prediction due to poor initialization, our approach aims to learn a policy by leveraging raw pixels to reason a subset of shape candidates, sequentially making plausible decisions to remove outliers for robust initialization. To achieve this, we formulate face alignment as a Markov decision process by defining an agent, which typically interacts with a trajectory of states, actions, state transitions and rewards. The agent seeks an optimal shape searching policy over the whole shape space by maximizing a discounted sum of the received values. To further improve the alignment performance, we develop an LSTM-based value function to evaluate the shape quality. During the training procedure, we adjust the gradient of our value function in directions of the policy gradient. This prevents our training goal from being trapped into local optima entangled by both the pose deformations and appearance variations especially in unconstrained environments. Experimental results show that our proposed RDN consistently outperforms most state-of-the-art approaches on four widely-evaluated challenging datasets.

Learning Reasoning-Decision Networks for Robust Face Alignment

Unconstrained face alignment usually undergoes extreme deformations and severe occlusions, which likely gives rise to biased shape prediction. Most existing methods simply exploit shape structure by directly concatenating all landmarks, which leads to losses of facial details in extreme deformation regions. In this paper, we propose a relational-structural networks (RSN) approach to learn both local and global feature representation for robust face alignment. To achieve this goal, we built a structural branch network to disentangle the local geometric relationship among neighboring facial sub-regions. Moreover, we develop a reinforcement learning approach to reason the robust iterative process. Our RSN generates three candidate shapes. Then a Q-net evaluates three candidate shapes by a reward function, which select the best shape to re-initialize network input to alleviate the local optimization problem of cascade regression methods. Authentic experimental results indicate that our approach consistently outperforms the most state-of-the-art methods on widely evaluated challenging datasets.

Learning Relational-Structural Networks for Robust Face Alignment

In this paper, we propose a deformable hourglass networks (DHGN) approach to investigate the problem of face alignment, especially in such challenging cases when faces undergo large variations including severe poses, diverse expressions and partial occlusions in unconstrained environments. Unlike conventional feature extractions which cannot explicitly exploit irregular geometric structures for facial shapes, our DHGN learns a deformable mask to reduce the variances of facial deformation and extract attentional facial regions for robust feature representation. To achieve this, we carefully design a differential module, dubbed the deformable transformer, which typically incorporates with a regression sub-net to predict a set of offsets and a masking operator to filter the semantic facial parts for feature representation learning. To further reinforce the alignment performance, we integrate our designed modules in the paradigm of stacked hourglass networks and jointly optimize the network parameters in an end-to-end manner. Extensive experimental results demonstrate very compelling performance in comparisons to most state-of-the-art methods.

Learning Deformable Hourglass Networks (DHGN) for Unconstrained Face Alignment

Plant disease detection plays an important role in agricultural production and ecological protection. However, it is always a challenge to detect the severity of plant diseases in multi-species and multi-disease conditions. Unlike most existing classification methods which are difficult to solve multi-properties detection, we propose a disentangled representation interactive network (DRIN), which disentangles the global features of each plant leaf and learns the discriminative representation of multiple sub-properties, including plant species, disease types and disease severity. To achieve it, the disentangled representation network transform the joint probability into the conditional probability through the information interaction between the sub-properties. Moreover, data filtering was introduced to reduce the error messages in property interactions. Experimental results demonstrate the effectiveness of our DRIN on the plant disease detection dataset.

Suping Wu

Papers

Similarity-Aware and Variational Deep Adversarial Learning for Robust Facial Age Estimation

Learning Reasoning-Decision Networks for Robust Face Alignment

Learning Relational-Structural Networks for Robust Face Alignment

Learning Deformable Hourglass Networks (DHGN) for Unconstrained Face Alignment

Disentangled Representation Learning for Leaf Diseases Recognition