Semantic Image Inpainting with Deep Generative Models
Raymond A. Yeh,Chen Chen,Teck Yian Lim,Alexander G. Schwing,Alexander G. Schwing,Mark Hasegawa-Johnson,Minh N. Do +6 more
- pp 6882-6890
Reads0
Chats0
TLDR
A novel method for semantic image inpainting, which generates the missing content by conditioning on the available data, and successfully predicts information in large missing regions and achieves pixel-level photorealism, significantly outperforming the state-of-the-art methods.Abstract:
Semantic image inpainting is a challenging task where large missing regions have to be filled based on the available visual data. Existing methods which extract information from only a single image generally produce unsatisfactory results due to the lack of high level context. In this paper, we propose a novel method for semantic image inpainting, which generates the missing content by conditioning on the available data. Given a trained generative model, we search for the closest encoding of the corrupted image in the latent image manifold using our context and prior losses. This encoding is then passed through the generative model to infer the missing content. In our method, inference is possible irrespective of how the missing content is structured, while the state-of-the-art learning based method requires specific information about the holes in the training phase. Experiments on three datasets show that our method successfully predicts information in large missing regions and achieves pixel-level photorealism, significantly outperforming the state-of-the-art methods.read more
Citations
More filters
Posted Content
Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network
Christian Ledig,Lucas Theis,Ferenc Huszar,Jose Caballero,Andrew Cunningham,Alejandro Acosta,Andrew Peter Aitken,Alykhan Tejani,Johannes Totz,Zehan Wang,Wenzhe Shi +10 more
TL;DR: SRGAN, a generative adversarial network (GAN) for image super-resolution (SR), is presented, to its knowledge, the first framework capable of inferring photo-realistic natural images for 4x upscaling factors and a perceptual loss function which consists of an adversarial loss and a content loss.
Journal ArticleDOI
Generative adversarial networks
Ian Goodfellow,Jean Pouget-Abadie,Mehdi Mirza,Bing Xu,David Warde-Farley,Sherjil Ozair,Aaron Courville,Yoshua Bengio +7 more
TL;DR: A generative adversarial networks algorithm designed to solve the generative modeling problem and its applications in medicine, education and robotics are studied.
Book ChapterDOI
Unsupervised Anomaly Detection with Generative Adversarial Networks to Guide Marker Discovery
TL;DR: AnoGAN, a deep convolutional generative adversarial network is proposed to learn a manifold of normal anatomical variability, accompanying a novel anomaly scoring scheme based on the mapping from image space to a latent space.
Proceedings ArticleDOI
Unlabeled Samples Generated by GAN Improve the Person Re-identification Baseline in Vitro
TL;DR: A simple semisupervised pipeline that only uses the original training set without collecting extra data, which effectively improves the discriminative ability of learned CNN embeddings and proposes the label smoothing regularization for outliers (LSRO).
Book ChapterDOI
Image Inpainting for Irregular Holes Using Partial Convolutions
TL;DR: This work proposes the use of partial convolutions, where the convolution is masked and renormalized to be conditioned on only valid pixels, and outperforms other methods for irregular masks.
References
More filters
Proceedings ArticleDOI
3D Object Representations for Fine-Grained Categorization
TL;DR: This paper lifts two state-of-the-art 2D object representations to 3D, on the level of both local feature appearance and location, and shows their efficacy for estimating 3D geometry from images via ultra-wide baseline matching and 3D reconstruction.
Posted Content
Context Encoders: Feature Learning by Inpainting
TL;DR: Context Encoders as mentioned in this paper is a convolutional neural network trained to generate the contents of an arbitrary image region conditioned on its surroundings, which can be used for semantic inpainting tasks, either stand-alone or as initialization for nonparametric methods.
Proceedings Article
Deep generative image models using a Laplacian pyramid of adversarial networks
TL;DR: A generative parametric model capable of producing high quality samples of natural images using a cascade of convolutional networks within a Laplacian pyramid framework to generate images in a coarse-to-fine fashion.
Journal ArticleDOI
Sparse Representation for Color Image Restoration
TL;DR: This work puts forward ways for handling nonhomogeneous noise and missing information, paving the way to state-of-the-art results in applications such as color image denoising, demosaicing, and inpainting, as demonstrated in this paper.
Proceedings Article
Autoencoding beyond pixels using a learned similarity metric
TL;DR: In this article, an autoencoder that leverages learned representations to better measure similarities in data space is presented, which can use learned feature representations in the GAN discriminator as basis for the VAE reconstruction objective.