Joint Face Alignment and 3D Face Reconstruction with Application to Face Recognition
Reads0
Chats0
TLDR
Zhang et al. as discussed by the authors proposed a joint face alignment and 3D face reconstruction method to simultaneously solve these two problems for 2D face images of arbitrary poses and expressions, which can fully automatically generate both pose-and-expression-normalized (PEN) and expressive 3D faces and localize both visible and invisible 2D landmarks.Abstract:
Face alignment and 3D face reconstruction are traditionally accomplished as separated tasks. By exploring the strong correlation between 2D landmarks and 3D shapes, in contrast, we propose a joint face alignment and 3D face reconstruction method to simultaneously solve these two problems for 2D face images of arbitrary poses and expressions. This method, based on a summation model of 3D faces and cascaded regression in 2D and 3D shape spaces, iteratively and alternately applies two cascaded regressors, one for updating 2D landmarks and the other for 3D shape. The 3D shape and the landmarks are correlated via a 3D-to-2D mapping matrix, which is updated in each iteration to refine the location and visibility of 2D landmarks. Unlike existing methods, the proposed method can fully automatically generate both pose-and-expression-normalized (PEN) and expressive 3D faces and localize both visible and invisible 2D landmarks. Based on the PEN 3D faces, we devise a method to enhance face recognition accuracy across poses and expressions. Both linear and nonlinear implementations of the proposed method are presented and evaluated in this paper. Extensive experiments show that the proposed method can achieve the state-of-the-art accuracy in both face alignment and 3D face reconstruction, and benefit face recognition owing to its reconstructed PEN 3D face.read more
Citations
More filters
Journal ArticleDOI
Real-Time 3D Face Alignment Using an Encoder-Decoder Network With an Efficient Deconvolution Layer
TL;DR: This study presents a real-time 3D face-alignment method that uses an encoder-decoder network with an efficient deconvolution layer and applies the L1 norm to select useful features and generate abundant ones through linear operations.
Journal ArticleDOI
Multi‐view frontal face image generation: A survey
Proceedings ArticleDOI
3D Face Modeling From Diverse Raw Scan Data
Feng Liu,Luan Tran,Xiaoming Liu +2 more
TL;DR: An innovative framework to jointly learn a nonlinear face model from a diverse set of raw 3D scan databases and establish dense point-to-point correspondence among their scans is proposed and its contribution to single-image 3D face reconstruction is demonstrated.
Journal ArticleDOI
Towards Efficient U-Nets: A Coupled and Quantized Approach
TL;DR: The results show that the proposed couple stacked U-Nets for efficient visual landmark localization achieves state-of-the-art localization accuracy but using fewer parameters, less inference time, and saving model size.
Journal ArticleDOI
An end-to-end shape modeling framework for vectorized building outline generation from aerial images
TL;DR: PolygonCNN is introduced, a learnable end-to-end vector shape modeling framework for generating building outlines from aerial images and proposes a simplify-and-densify sampling strategy to generate homogeneously sampled polygon with well-kept geometric signals for shape prior learning.
References
More filters
Journal ArticleDOI
Distinctive Image Features from Scale-Invariant Keypoints
TL;DR: This paper presents a method for extracting distinctive invariant features from images that can be used to perform reliable matching between different views of an object or scene and can robustly identify objects among clutter and occlusion while achieving near real-time performance.
Proceedings ArticleDOI
FaceNet: A unified embedding for face recognition and clustering
TL;DR: A system that directly learns a mapping from face images to a compact Euclidean space where distances directly correspond to a measure offace similarity, and achieves state-of-the-art face recognition performance using only 128-bytes perface.
Journal ArticleDOI
Active appearance models
Abstract: We describe a new method of matching statistical models of appearance to images. A set of model parameters control modes of shape and gray-level variation learned from a training set. We construct an efficient iterative matching algorithm by learning the relationship between perturbations in the model parameters and the induced image errors.
Proceedings ArticleDOI
DeepFace: Closing the Gap to Human-Level Performance in Face Verification
TL;DR: This work revisits both the alignment step and the representation step by employing explicit 3D face modeling in order to apply a piecewise affine transformation, and derive a face representation from a nine-layer deep neural network.
Proceedings ArticleDOI
Deep face recognition
TL;DR: It is shown how a very large scale dataset can be assembled by a combination of automation and human in the loop, and the trade off between data purity and time is discussed.