MaskGAN: Towards Diverse and Interactive Facial Image Manipulation
Cheng-Han Lee,Ziwei Liu,Lingyun Wu,Ping Luo +3 more
- pp 5549-5558
Reads0
Chats0
TLDR
MaskGAN as mentioned in this paper proposes MaskGAN to enable diverse and interactive face manipulation by learning style mapping between a free-form user modified mask and a target image, enabling diverse generation results.Abstract:
Facial image manipulation has achieved great progress in recent years. However, previous methods either operate on a predefined set of face attributes or leave users little freedom to interactively manipulate images. To overcome these drawbacks, we propose a novel framework termed MaskGAN, enabling diverse and interactive face manipulation. Our key insight is that semantic masks serve as a suitable intermediate representation for flexible face manipulation with fidelity preservation. MaskGAN has two main components: 1) Dense Mapping Network (DMN) and 2) Editing Behavior Simulated Training (EBST). Specifically, DMN learns style mapping between a free-form user modified mask and a target image, enabling diverse generation results. EBST models the user editing behavior on the source mask, making the overall framework more robust to various manipulated inputs. Specifically, it introduces dual-editing consistency as the auxiliary supervision signal. To facilitate extensive studies, we construct a large-scale high-resolution face dataset with fine-grained mask annotations named CelebAMask-HQ. MaskGAN is comprehensively evaluated on two challenging tasks: attribute transfer and style copy, demonstrating superior performance over other state-of-the-art methods. The code, models, and dataset are available at https://github.com/switchablenorms/CelebAMask-HQ.read more
Citations
More filters
Journal ArticleDOI
Prior-Guided Multi-View 3D Head Reconstruction
TL;DR: Zhang et al. as discussed by the authors proposed a prior-guided implicit neural rendering network to reconstruct 3D head geometry with a learnable signed distance field (SDF) and optimize it via an implicit differentiable renderer with the guidance of human head priors, including the facial prior knowledge, head semantic segmentation information and 2D hair orientation maps.
Proceedings ArticleDOI
A Conservative Approach for Unbiased Learning on Unknown Biases
TL;DR: UBNet as discussed by the authors proposes a conservative framework that employs this internal information for unbiased learning, which is implemented via hierarchical features captured along the multiple layers and orthogonal regularization, which does not necessitate a predefined bias.
Posted Content
AD-NeRF: Audio Driven Neural Radiance Fields for Talking Head Synthesis
TL;DR: In this paper, the feature of input audio signal is directly fed into a conditional implicit function to generate a dynamic neural radiance field, from which a high-fidelity talking-head video corresponding to the audio signal was synthesized using volume rendering.
Posted Content
Repurposing GANs for One-shot Semantic Part Segmentation
TL;DR: In this paper, a simple and effective approach based on GANs was proposed for semantic part segmentation that requires as few as one label example along with an unlabeled dataset.
Posted Content
LOHO: Latent Optimization of Hairstyles via Orthogonalization
TL;DR: In this paper, an optimization-based approach using GAN inversion to infill missing hair structure details in latent space during hairstyle transfer is proposed, which decomposes hair into three attributes: perceptual structure, appearance, and style.
References
More filters
Proceedings ArticleDOI
Deep Residual Learning for Image Recognition
TL;DR: In this article, the authors proposed a residual learning framework to ease the training of networks that are substantially deeper than those used previously, which won the 1st place on the ILSVRC 2015 classification task.
Proceedings Article
Very Deep Convolutional Networks for Large-Scale Image Recognition
Karen Simonyan,Andrew Zisserman +1 more
TL;DR: This work investigates the effect of the convolutional network depth on its accuracy in the large-scale image recognition setting using an architecture with very small convolution filters, which shows that a significant improvement on the prior-art configurations can be achieved by pushing the depth to 16-19 weight layers.
Proceedings ArticleDOI
ImageNet: A large-scale hierarchical image database
TL;DR: A new database called “ImageNet” is introduced, a large-scale ontology of images built upon the backbone of the WordNet structure, much larger in scale and diversity and much more accurate than the current image datasets.
Book ChapterDOI
U-Net: Convolutional Networks for Biomedical Image Segmentation
TL;DR: Neber et al. as discussed by the authors proposed a network and training strategy that relies on the strong use of data augmentation to use the available annotated samples more efficiently, which can be trained end-to-end from very few images and outperforms the prior best method (a sliding-window convolutional network) on the ISBI challenge for segmentation of neuronal structures in electron microscopic stacks.
Journal ArticleDOI
Generative Adversarial Nets
Ian Goodfellow,Jean Pouget-Abadie,Mehdi Mirza,Bing Xu,David Warde-Farley,Sherjil Ozair,Aaron Courville,Yoshua Bengio +7 more
TL;DR: A new framework for estimating generative models via an adversarial process, in which two models are simultaneously train: a generative model G that captures the data distribution and a discriminative model D that estimates the probability that a sample came from the training data rather than G.