MaskGAN: Towards Diverse and Interactive Facial Image Manipulation
Cheng-Han Lee,Ziwei Liu,Lingyun Wu,Ping Luo +3 more
- pp 5549-5558
Reads0
Chats0
TLDR
MaskGAN as mentioned in this paper proposes MaskGAN to enable diverse and interactive face manipulation by learning style mapping between a free-form user modified mask and a target image, enabling diverse generation results.Abstract:
Facial image manipulation has achieved great progress in recent years. However, previous methods either operate on a predefined set of face attributes or leave users little freedom to interactively manipulate images. To overcome these drawbacks, we propose a novel framework termed MaskGAN, enabling diverse and interactive face manipulation. Our key insight is that semantic masks serve as a suitable intermediate representation for flexible face manipulation with fidelity preservation. MaskGAN has two main components: 1) Dense Mapping Network (DMN) and 2) Editing Behavior Simulated Training (EBST). Specifically, DMN learns style mapping between a free-form user modified mask and a target image, enabling diverse generation results. EBST models the user editing behavior on the source mask, making the overall framework more robust to various manipulated inputs. Specifically, it introduces dual-editing consistency as the auxiliary supervision signal. To facilitate extensive studies, we construct a large-scale high-resolution face dataset with fine-grained mask annotations named CelebAMask-HQ. MaskGAN is comprehensively evaluated on two challenging tasks: attribute transfer and style copy, demonstrating superior performance over other state-of-the-art methods. The code, models, and dataset are available at https://github.com/switchablenorms/CelebAMask-HQ.read more
Citations
More filters
Proceedings ArticleDOI
SEAN: Image Synthesis With Semantic Region-Adaptive Normalization
TL;DR: Semantic Region Adaptive Normalization (SEAN) as mentioned in this paper is a simple but effective building block for Generative Adversarial Networks conditioned on segmentation masks that describe the semantic regions in the desired output image.
Posted Content
StyleSpace Analysis: Disentangled Controls for StyleGAN Image Generation
TL;DR: The latent style space of Style-GAN2, a state-of-the-art architecture for image generation, is explored and StyleSpace, the space of channel-wise style parameters, is shown to be significantly more disentangled than the other intermediate latent spaces explored by previous works.
Proceedings ArticleDOI
Towards Photo-Realistic Virtual Try-On by Adaptively Generating↔Preserving Image Content
TL;DR: This work proposes a novel visual try-on network, namely Adaptive Content Generating and Preserving Network (ACGPN), which can generate photo-realistic images with much better perceptual quality and richer fine-details.
Proceedings ArticleDOI
PD-GAN: Probabilistic Diverse GAN for Image Inpainting
TL;DR: PD-GAN as mentioned in this paper modulates deep features of input random noise from coarse-to-fine by injecting an initially restored image and the hole regions in multiple scales to generate multiple inpainting results with diverse and visually realistic content.
Proceedings ArticleDOI
StyleSpace Analysis: Disentangled Controls for StyleGAN Image Generation
TL;DR: In this paper, the authors explore and analyze the latent style space of Style-GAN2, a state-of-the-art architecture for image generation, using models pretrained on several different datasets.
References
More filters
Proceedings ArticleDOI
Generative Image Inpainting with Contextual Attention
TL;DR: Yu et al. as discussed by the authors proposed a new deep generative model-based approach which can not only synthesize novel image structures but also explicitly utilize surrounding image features as references during network training to make better predictions.
Proceedings ArticleDOI
Compositing digital images
TL;DR: In this article, a matte component can be computed similarly to the color channels for four-channel pictures, and guidelines for the generation of elements and arithmetic for their arbitrary compositing are discussed.
Book ChapterDOI
Interactive facial feature localization
TL;DR: An improvement to the Active Shape Model is proposed that allows for greater independence among the facial components and improves on the appearance fitting step by introducing a Viterbi optimization process that operates along the facial contours.
Journal ArticleDOI
FaceWarehouse: A 3D Facial Expression Database for Visual Computing
TL;DR: There is a much richer matching collection of expressions, enabling depiction of most human facial actions, in FaceWarehouse, a database of 3D facial expressions for visual computing applications.
Proceedings ArticleDOI
Free-Form Image Inpainting With Gated Convolution
TL;DR: Yu et al. as mentioned in this paper proposed a generative image inpainting system to complete images with free-form mask and guidance, which is based on gated convolutions learned from millions of images without additional labeling efforts.