Learning Parallax Attention for Stereo Image Super-Resolution
Longguang Wang,Yingqian Wang,Zhengfa Liang,Zaiping Lin,Jungang Yang,Wei An,Yulan Guo +6 more
- pp 12250-12259
TLDR
A parallax-attention mechanism with a global receptive field along the epipolar line to handle different stereo images with large disparity variations is introduced and a new and the largest dataset for stereo image SR is proposed.Abstract:Â
Stereo image pairs can be used to improve the performance of super-resolution (SR) since additional information is provided from a second viewpoint. However, it is challenging to incorporate this information for SR since disparities between stereo images vary significantly. In this paper, we propose a parallax-attention stereo superresolution network (PASSRnet) to integrate the information from a stereo image pair for SR. Specifically, we introduce a parallax-attention mechanism with a global receptive field along the epipolar line to handle different stereo images with large disparity variations. We also propose a new and the largest dataset for stereo image SR (namely, Flickr1024). Extensive experiments demonstrate that the parallax-attention mechanism can capture correspondence between stereo images to improve SR performance with a small computational and memory cost. Comparative results show that our PASSRnet achieves the state-of-the-art performance on the Middlebury, KITTI 2012 and KITTI 2015 datasets.read more
Citations
More filters
Journal ArticleDOI
Deep Learning for Image Super-Resolution: A Survey
TL;DR: A survey on recent advances of image super-resolution techniques using deep learning approaches in a systematic way, which can roughly group the existing studies of SR techniques into three major categories: supervised SR, unsupervised SR, and domain-specific SR.
Journal ArticleDOI
Deformable 3D Convolution for Video Super-Resolution
TL;DR: A deformable 3D convolution network (D3Dnet) is proposed to incorporate spatio-temporal information from both spatial and temporal dimensions for video SR, and achieves state-of-the-art SR performance.
Journal ArticleDOI
ANU-Net: Attention-based nested U-Net to exploit full resolution features for medical image segmentation
TL;DR: An attention-based nested segmentation network, named ANU-Net, which has a deep supervised encoder-decoder architecture and a redesigned dense skip connection and achieved very competitive performance for four kinds of medical image segmentation tasks.
Proceedings ArticleDOI
Attention Unet++: A Nested Attention-Aware U-Net for Liver CT Image Segmentation
TL;DR: A nested attention-aware segmentation network, named Attention UNet++, which has a deep supervised encoder-decoder architecture and a redesigned dense skip connection and achieved very competitive performance on MICCAI 2017 Liver Tumor Segmentation Challenge Dataset.
Journal Article
VRT: A Video Restoration Transformer
Jingyun Liang,Jiezhang Cao,Yuchen Fan,Kai Zhang,Rakesh Ranjan,Yawei Li,Radu Timofte,Luc Van Gool +7 more
TL;DR: Experimental results on video super-resolution, video deblurring, video denoising, video frame interpolation and space-time videosuper-resolution demonstrate that VRT outperforms the state-of-the-art methods by large margins.
References
More filters
Proceedings Article
Adam: A Method for Stochastic Optimization
Diederik P. Kingma,Jimmy Ba +1 more
TL;DR: This work introduces Adam, an algorithm for first-order gradient-based optimization of stochastic objective functions, based on adaptive estimates of lower-order moments, and provides a regret bound on the convergence rate that is comparable to the best known results under the online convex optimization framework.
Proceedings Article
Attention is All you Need
Ashish Vaswani,Noam Shazeer,Niki Parmar,Jakob Uszkoreit,Llion Jones,Aidan N. Gomez,Lukasz Kaiser,Illia Polosukhin +7 more
TL;DR: This paper proposed a simple network architecture based solely on an attention mechanism, dispensing with recurrence and convolutions entirely and achieved state-of-the-art performance on English-to-French translation.
Proceedings ArticleDOI
Are we ready for autonomous driving? The KITTI vision benchmark suite
TL;DR: The autonomous driving platform is used to develop novel challenging benchmarks for the tasks of stereo, optical flow, visual odometry/SLAM and 3D object detection, revealing that methods ranking high on established datasets such as Middlebury perform below average when being moved outside the laboratory to the real world.
Proceedings ArticleDOI
Non-local Neural Networks
TL;DR: In this article, the non-local operation computes the response at a position as a weighted sum of the features at all positions, which can be used to capture long-range dependencies.
Journal ArticleDOI
A taxonomy and evaluation of dense two-frame stereo correspondence algorithms
TL;DR: This paper has designed a stand-alone, flexible C++ implementation that enables the evaluation of individual components and that can easily be extended to include new algorithms.