Semantic Image Inpainting with Deep Generative Models
Raymond A. Yeh,Chen Chen,Teck Yian Lim,Alexander G. Schwing,Alexander G. Schwing,Mark Hasegawa-Johnson,Minh N. Do +6 more
- pp 6882-6890
Reads0
Chats0
TLDR
A novel method for semantic image inpainting, which generates the missing content by conditioning on the available data, and successfully predicts information in large missing regions and achieves pixel-level photorealism, significantly outperforming the state-of-the-art methods.Abstract:
Semantic image inpainting is a challenging task where large missing regions have to be filled based on the available visual data. Existing methods which extract information from only a single image generally produce unsatisfactory results due to the lack of high level context. In this paper, we propose a novel method for semantic image inpainting, which generates the missing content by conditioning on the available data. Given a trained generative model, we search for the closest encoding of the corrupted image in the latent image manifold using our context and prior losses. This encoding is then passed through the generative model to infer the missing content. In our method, inference is possible irrespective of how the missing content is structured, while the state-of-the-art learning based method requires specific information about the holes in the training phase. Experiments on three datasets show that our method successfully predicts information in large missing regions and achieves pixel-level photorealism, significantly outperforming the state-of-the-art methods.read more
Citations
More filters
Posted Content
Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network
Christian Ledig,Lucas Theis,Ferenc Huszar,Jose Caballero,Andrew Cunningham,Alejandro Acosta,Andrew Peter Aitken,Alykhan Tejani,Johannes Totz,Zehan Wang,Wenzhe Shi +10 more
TL;DR: SRGAN, a generative adversarial network (GAN) for image super-resolution (SR), is presented, to its knowledge, the first framework capable of inferring photo-realistic natural images for 4x upscaling factors and a perceptual loss function which consists of an adversarial loss and a content loss.
Journal ArticleDOI
Generative adversarial networks
Ian Goodfellow,Jean Pouget-Abadie,Mehdi Mirza,Bing Xu,David Warde-Farley,Sherjil Ozair,Aaron Courville,Yoshua Bengio +7 more
TL;DR: A generative adversarial networks algorithm designed to solve the generative modeling problem and its applications in medicine, education and robotics are studied.
Book ChapterDOI
Unsupervised Anomaly Detection with Generative Adversarial Networks to Guide Marker Discovery
TL;DR: AnoGAN, a deep convolutional generative adversarial network is proposed to learn a manifold of normal anatomical variability, accompanying a novel anomaly scoring scheme based on the mapping from image space to a latent space.
Proceedings ArticleDOI
Unlabeled Samples Generated by GAN Improve the Person Re-identification Baseline in Vitro
TL;DR: A simple semisupervised pipeline that only uses the original training set without collecting extra data, which effectively improves the discriminative ability of learned CNN embeddings and proposes the label smoothing regularization for outliers (LSRO).
Book ChapterDOI
Image Inpainting for Irregular Holes Using Partial Convolutions
TL;DR: This work proposes the use of partial convolutions, where the convolution is masked and renormalized to be conditioned on only valid pixels, and outperforms other methods for irregular masks.
References
More filters
Journal ArticleDOI
Image completion using planar structure guidance
TL;DR: This work proposes a method for automatically guiding patch-based image completion using mid-level structural cues by first estimates planar projection parameters, softly segments the known region into planes, and discovers translational regularity within these planes.
Proceedings ArticleDOI
Generalized Nonconvex Nonsmooth Low-Rank Minimization
TL;DR: In theory, it is proved that IRNN decreases the objective function value monotonically, and any limit point is a stationary point, which enhances the low-rank matrix recovery compared with state-of-the-art convex algorithms.
Book ChapterDOI
Statistics of Patch Offsets for Image Completion
Kaiming He,Jian Sun +1 more
TL;DR: This paper observation that if the authors match similar patches in the image and obtain their offsets (relative positions), the statistics of these offsets are sparsely distributed means that a few dominant offsets provide reliable information for completing the image.
Proceedings ArticleDOI
Inversion of multilayer nets
Linden,Kindermann +1 more
TL;DR: The method of inversion for arbitrary continuous multilayer nets is developed in this article, where the inversion is done by computing iteratively an input vector which minimizes the least-mean-square errors to approximate a given output target.
Proceedings Article
Shepard convolutional neural networks
TL;DR: This paper draws on Shepard interpolation and design Shepard Convolutional Neural Networks (ShCNN) which efficiently realizes end-to-end trainable TVI operators in the network and shows that by adding only a few feature maps in the new Shepard layers, the network is able to achieve stronger results than a much deeper architecture.