scispace - formally typeset
Open AccessProceedings ArticleDOI

MaskGAN: Towards Diverse and Interactive Facial Image Manipulation

Reads0
Chats0
TLDR
MaskGAN as mentioned in this paper proposes MaskGAN to enable diverse and interactive face manipulation by learning style mapping between a free-form user modified mask and a target image, enabling diverse generation results.
Abstract
Facial image manipulation has achieved great progress in recent years. However, previous methods either operate on a predefined set of face attributes or leave users little freedom to interactively manipulate images. To overcome these drawbacks, we propose a novel framework termed MaskGAN, enabling diverse and interactive face manipulation. Our key insight is that semantic masks serve as a suitable intermediate representation for flexible face manipulation with fidelity preservation. MaskGAN has two main components: 1) Dense Mapping Network (DMN) and 2) Editing Behavior Simulated Training (EBST). Specifically, DMN learns style mapping between a free-form user modified mask and a target image, enabling diverse generation results. EBST models the user editing behavior on the source mask, making the overall framework more robust to various manipulated inputs. Specifically, it introduces dual-editing consistency as the auxiliary supervision signal. To facilitate extensive studies, we construct a large-scale high-resolution face dataset with fine-grained mask annotations named CelebAMask-HQ. MaskGAN is comprehensively evaluated on two challenging tasks: attribute transfer and style copy, demonstrating superior performance over other state-of-the-art methods. The code, models, and dataset are available at https://github.com/switchablenorms/CelebAMask-HQ.

read more

Content maybe subject to copyright    Report

Citations
More filters
Journal ArticleDOI

Repurposing GANs for One-shot Semantic Part Segmentation

TL;DR: This paper used GANs for semantic part segmentation and landmark detection, and achieved state-of-the-art performance on both tasks using only one labeled example along with an unlabeled dataset, rather than thousands of examples.
Posted Content

Bridging Unpaired Facial Photos And Sketches By Line-drawings

TL;DR: In this article, the authors propose a novel method to learn face sketch synthesis models by using unpaired data by bridging the photo domain and the sketch domain by using the line-drawing domain.
Journal ArticleDOI

PaintInStyle: One-Shot Discovery of Interpretable Directions by Painting

TL;DR: This work proposes a framework that finds a specific manipulation direction using only a single simple sketch drawn on an image and performs image manipulations comparable with state-of-the-art methods.
Journal ArticleDOI

Generative Adversarial Network for Overcoming Occlusion in Images: A Survey

TL;DR: A survey of generative adversarial networks (GANs) for amodal segmentation, content completion, order recovery, and acquisition of training data can be found in this article .

Automatic Generation of Semantic Parts for Face Image Synthesis

TL;DR: In this article , a network architecture is described to address the problem of automatically manipulating or generating the shape of object classes in semantic segmentation masks, with specific focus on human faces, by embedding the mask class-wise into a latent space where each class embedding can be independently edited.
References
More filters
Proceedings ArticleDOI

Deep Residual Learning for Image Recognition

TL;DR: In this article, the authors proposed a residual learning framework to ease the training of networks that are substantially deeper than those used previously, which won the 1st place on the ILSVRC 2015 classification task.
Proceedings Article

Very Deep Convolutional Networks for Large-Scale Image Recognition

TL;DR: This work investigates the effect of the convolutional network depth on its accuracy in the large-scale image recognition setting using an architecture with very small convolution filters, which shows that a significant improvement on the prior-art configurations can be achieved by pushing the depth to 16-19 weight layers.
Proceedings ArticleDOI

ImageNet: A large-scale hierarchical image database

TL;DR: A new database called “ImageNet” is introduced, a large-scale ontology of images built upon the backbone of the WordNet structure, much larger in scale and diversity and much more accurate than the current image datasets.
Book ChapterDOI

U-Net: Convolutional Networks for Biomedical Image Segmentation

TL;DR: Neber et al. as discussed by the authors proposed a network and training strategy that relies on the strong use of data augmentation to use the available annotated samples more efficiently, which can be trained end-to-end from very few images and outperforms the prior best method (a sliding-window convolutional network) on the ISBI challenge for segmentation of neuronal structures in electron microscopic stacks.
Journal ArticleDOI

Generative Adversarial Nets

TL;DR: A new framework for estimating generative models via an adversarial process, in which two models are simultaneously train: a generative model G that captures the data distribution and a discriminative model D that estimates the probability that a sample came from the training data rather than G.
Related Papers (5)