Semantic Image Inpainting with Deep Generative Models
Raymond A. Yeh,Chen Chen,Teck Yian Lim,Alexander G. Schwing,Alexander G. Schwing,Mark Hasegawa-Johnson,Minh N. Do +6 more
- pp 6882-6890
Reads0
Chats0
TLDR
A novel method for semantic image inpainting, which generates the missing content by conditioning on the available data, and successfully predicts information in large missing regions and achieves pixel-level photorealism, significantly outperforming the state-of-the-art methods.Abstract:
Semantic image inpainting is a challenging task where large missing regions have to be filled based on the available visual data. Existing methods which extract information from only a single image generally produce unsatisfactory results due to the lack of high level context. In this paper, we propose a novel method for semantic image inpainting, which generates the missing content by conditioning on the available data. Given a trained generative model, we search for the closest encoding of the corrupted image in the latent image manifold using our context and prior losses. This encoding is then passed through the generative model to infer the missing content. In our method, inference is possible irrespective of how the missing content is structured, while the state-of-the-art learning based method requires specific information about the holes in the training phase. Experiments on three datasets show that our method successfully predicts information in large missing regions and achieves pixel-level photorealism, significantly outperforming the state-of-the-art methods.read more
Citations
More filters
Posted Content
Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network
Christian Ledig,Lucas Theis,Ferenc Huszar,Jose Caballero,Andrew Cunningham,Alejandro Acosta,Andrew Peter Aitken,Alykhan Tejani,Johannes Totz,Zehan Wang,Wenzhe Shi +10 more
TL;DR: SRGAN, a generative adversarial network (GAN) for image super-resolution (SR), is presented, to its knowledge, the first framework capable of inferring photo-realistic natural images for 4x upscaling factors and a perceptual loss function which consists of an adversarial loss and a content loss.
Journal ArticleDOI
Generative adversarial networks
Ian Goodfellow,Jean Pouget-Abadie,Mehdi Mirza,Bing Xu,David Warde-Farley,Sherjil Ozair,Aaron Courville,Yoshua Bengio +7 more
TL;DR: A generative adversarial networks algorithm designed to solve the generative modeling problem and its applications in medicine, education and robotics are studied.
Book ChapterDOI
Unsupervised Anomaly Detection with Generative Adversarial Networks to Guide Marker Discovery
TL;DR: AnoGAN, a deep convolutional generative adversarial network is proposed to learn a manifold of normal anatomical variability, accompanying a novel anomaly scoring scheme based on the mapping from image space to a latent space.
Proceedings ArticleDOI
Unlabeled Samples Generated by GAN Improve the Person Re-identification Baseline in Vitro
TL;DR: A simple semisupervised pipeline that only uses the original training set without collecting extra data, which effectively improves the discriminative ability of learned CNN embeddings and proposes the label smoothing regularization for outliers (LSRO).
Book ChapterDOI
Image Inpainting for Irregular Holes Using Partial Convolutions
TL;DR: This work proposes the use of partial convolutions, where the convolution is masked and renormalized to be conditioned on only valid pixels, and outperforms other methods for irregular masks.
References
More filters
Proceedings ArticleDOI
Texture synthesis by non-parametric sampling
Alexei A. Efros,Thomas Leung +1 more
TL;DR: A non-parametric method for texture synthesis that aims at preserving as much local structure as possible and produces good results for a wide variety of synthetic and real-world textures.
Posted Content
Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps
TL;DR: The authors compute the gradient of the class score with respect to the input image and compute a class saliency map, which can be used for weakly supervised object segmentation using classification ConvNets.
Journal ArticleDOI
PatchMatch: a randomized correspondence algorithm for structural image editing
TL;DR: This paper presents interactive image editing tools using a new randomized algorithm for quickly finding approximate nearest-neighbor matches between image patches, and proposes additional intuitive constraints on the synthesis process that offer the user a level of control unavailable in previous methods.
Posted Content
Deep Learning Face Attributes in the Wild
TL;DR: Zhang et al. as mentioned in this paper proposed a novel deep learning framework for attribute prediction in the wild, which cascades two CNNs, LNet and ANet, which are fine-tuned jointly with attribute tags, but pre-trained differently.
Proceedings ArticleDOI
Poisson image editing
TL;DR: Using generic interpolation machinery based on solving Poisson equations, a variety of novel tools are introduced for seamless editing of image regions, which permits the seamless importation of both opaque and transparent source image regions into a destination region.