Semi-Parametric Image Synthesis
Xiaojuan Qi,Qifeng Chen,Jiaya Jia,Vladlen Koltun +3 more
- pp 8808-8816
TLDR
A semi-parametric approach to photographic image synthesis from semantic layouts that combines the complementary strengths of parametric and nonparametric techniques is presented.Abstract:
We present a semi-parametric approach to photographic image synthesis from semantic layouts. The approach combines the complementary strengths of parametric and nonparametric techniques. The nonparametric component is a memory bank of image segments constructed from a training set of images. Given a novel semantic layout at test time, the memory bank is used to retrieve photographic references that are provided as source material to a deep network. The synthesis is performed by a deep network that draws on the provided photographic material. Experiments on multiple semantic segmentation datasets show that the presented approach yields considerably more realistic images than recent purely parametric techniques.read more
Citations
More filters
Proceedings ArticleDOI
Multi-Modal Fusion Transformer for End-to-End Autonomous Driving
TL;DR: TransFuser as discussed by the authors integrates image and LiDAR representations using attention and achieves state-of-the-art driving performance while reducing collisions by 76% compared to geometry-based fusion.
Journal ArticleDOI
State of the Art on Neural Rendering
Ayush Tewari,Ohad Fried,Justus Thies,Vincent Sitzmann,Stephen Lombardi,Kalyan Sunkavalli,Ricardo Martin-Brualla,Tomas Simon,Jason Saragih,Matthias Nießner,Rohit Pandey,Sean Fanello,Gordon Wetzstein,Jun-Yan Zhu,Christian Theobalt,Maneesh Agrawala,Eli Shechtman,Dan B. Goldman,Michael Zollhöfer +18 more
TL;DR: Neural rendering as discussed by the authors is a new and rapidly emerging field that combines generative machine learning techniques with physical knowledge from computer graphics, e.g., by the integration of differentiable rendering into network training.
Posted Content
State of the Art on Neural Rendering
Ayush Tewari,Ohad Fried,Justus Thies,Vincent Sitzmann,Stephen Lombardi,Kalyan Sunkavalli,Ricardo Martin-Brualla,Tomas Simon,Jason Saragih,Matthias Nießner,Rohit Pandey,Sean Fanello,Gordon Wetzstein,Jun-Yan Zhu,Christian Theobalt,Maneesh Agrawala,Eli Shechtman,Dan B. Goldman,Michael Zollhöfer +18 more
TL;DR: This state‐of‐the‐art report summarizes the recent trends and applications of neural rendering and focuses on approaches that combine classic computer graphics techniques with deep generative models to obtain controllable and photorealistic outputs.
Posted Content
Semantic Image Synthesis with Spatially-Adaptive Normalization
TL;DR: In this article, a spatially-adaptive normalization layer is proposed for synthesizing photorealistic images given an input semantic layout, which allows user control over both semantic and style.
Proceedings ArticleDOI
Cross-Domain Correspondence Learning for Exemplar-Based Image Translation
TL;DR: In this paper, an exemplar-based image translation method is proposed to synthesize a photo-realistic image from the input in a distinct domain (e.g., semantic segmentation mask, or edge map, or pose keypoints) given an image exemplar image.
References
More filters
Proceedings Article
Very Deep Convolutional Networks for Large-Scale Image Recognition
Karen Simonyan,Andrew Zisserman +1 more
TL;DR: In this paper, the authors investigated the effect of the convolutional network depth on its accuracy in the large-scale image recognition setting and showed that a significant improvement on the prior-art configurations can be achieved by pushing the depth to 16-19 layers.
Proceedings Article
Rectified Linear Units Improve Restricted Boltzmann Machines
Vinod Nair,Geoffrey E. Hinton +1 more
TL;DR: Restricted Boltzmann machines were developed using binary stochastic hidden units that learn features that are better for object recognition on the NORB dataset and face verification on the Labeled Faces in the Wild dataset.
Proceedings ArticleDOI
Image-to-Image Translation with Conditional Adversarial Networks
TL;DR: Conditional adversarial networks are investigated as a general-purpose solution to image-to-image translation problems and it is demonstrated that this approach is effective at synthesizing photos from label maps, reconstructing objects from edge maps, and colorizing images, among other tasks.
Proceedings ArticleDOI
Pyramid Scene Parsing Network
TL;DR: This paper exploits the capability of global context information by different-region-based context aggregation through the pyramid pooling module together with the proposed pyramid scene parsing network (PSPNet) to produce good quality results on the scene parsing task.
Journal ArticleDOI
Backpropagation applied to handwritten zip code recognition
Yann LeCun,Bernhard E. Boser,John S. Denker,D. Henderson,Richard Howard,W. Hubbard,Lawrence D. Jackel +6 more
TL;DR: This paper demonstrates how constraints from the task domain can be integrated into a backpropagation network through the architecture of the network, successfully applied to the recognition of handwritten zip code digits provided by the U.S. Postal Service.