Analyzing Perception-Distortion Tradeoff Using Enhanced Perceptual Super-Resolution Network
Subeesh Vasu,Nimisha Thekke Madam,A. N. Rajagopalan +2 more
- pp 114-131
TLDR
The proposed network, called enhanced perceptual super-resolution network (EPSR), is trained with a combination of mean squared error loss, perceptual loss, and adversarial loss and achieves the state-of-the-art trade-off between distortion and perceptual quality while the existing methods perform well in either of these measures alone.Abstract:
Convolutional neural network (CNN) based methods have recently achieved great success for image super-resolution (SR). However, most deep CNN based SR models attempt to improve distortion measures (e.g. PSNR, SSIM, IFC, VIF) while resulting in poor quantified perceptual quality (e.g. human opinion score, no-reference quality measures such as NIQE). Few works have attempted to improve the perceptual quality at the cost of performance reduction in distortion measures. A very recent study has revealed that distortion and perceptual quality are at odds with each other and there is always a trade-off between the two. Often the restoration algorithms that are superior in terms of perceptual quality, are inferior in terms of distortion measures. Our work attempts to analyze the trade-off between distortion and perceptual quality for the problem of single image SR. To this end, we use the well-known SR architecture- enhanced deep super-resolution (EDSR) network and show that it can be adapted to achieve better perceptual quality for a specific range of the distortion measure. While the original network of EDSR was trained to minimize the error defined based on per-pixel accuracy alone, we train our network using a generative adversarial network framework with EDSR as the generator module. Our proposed network, called enhanced perceptual super-resolution network (EPSR), is trained with a combination of mean squared error loss, perceptual loss, and adversarial loss. Our experiments reveal that EPSR achieves the state-of-the-art trade-off between distortion and perceptual quality while the existing methods perform well in either of these measures alone.read more
Citations
More filters
Journal ArticleDOI
Progressive perception-oriented network for single image super-resolution
TL;DR: This paper proposes a novel perceptual image super-resolution method that progressively generates visually high-quality results by constructing a stage-wise network and explores a new generator that adopts multi-scale hierarchical features fusion.
Posted Content
Deep Learning for Image Super-resolution: A Survey
TL;DR: In this paper, a comprehensive survey on recent advances of image super-resolution using deep learning approaches is provided, which can roughly group the existing studies of SR techniques into three major categories: supervised SR, unsupervised SR, and domain-specific SR.
Journal ArticleDOI
A comprehensive review of deep learning-based single image super-resolution.
TL;DR: A detailed survey of recent progress in single-image super-resolution in the perspective of deep learning can be found in this article, where the authors classify the image SR methods into four categories, i.e., classical methods, supervised learning-based methods, unsupervised learning based methods, and domain-specific SR methods.
Posted Content
Image Quality Assessment for Perceptual Image Restoration: A New Dataset, Benchmark and Metric.
TL;DR: Inspired by the find that the existing IQA methods have an unsatisfactory performance on the GAN-based distortion partially because of their low tolerance to spatial misalignment, a novel Space Warping Difference Network is proposed, which includes the novel l_2 pooling layers and Space Warped Difference layers.
Journal ArticleDOI
Deep Back-ProjectiNetworks for Single Image Super-Resolution
TL;DR: Deep Back-Projection Networks (DBPN) as discussed by the authors exploit iterative up-and down-sampling layers to exploit the mutual dependencies of low and high-resolution images, which is the winner of two image super-resolution challenges (NTIRE2018 and PIRM2018).
References
More filters
Proceedings Article
Adam: A Method for Stochastic Optimization
Diederik P. Kingma,Jimmy Ba +1 more
TL;DR: This work introduces Adam, an algorithm for first-order gradient-based optimization of stochastic objective functions, based on adaptive estimates of lower-order moments, and provides a regret bound on the convergence rate that is comparable to the best known results under the online convex optimization framework.
Proceedings Article
Very Deep Convolutional Networks for Large-Scale Image Recognition
Karen Simonyan,Andrew Zisserman +1 more
TL;DR: In this paper, the authors investigated the effect of the convolutional network depth on its accuracy in the large-scale image recognition setting and showed that a significant improvement on the prior-art configurations can be achieved by pushing the depth to 16-19 layers.
Journal ArticleDOI
Image quality assessment: from error visibility to structural similarity
TL;DR: In this article, a structural similarity index is proposed for image quality assessment based on the degradation of structural information, which can be applied to both subjective ratings and objective methods on a database of images compressed with JPEG and JPEG2000.
Proceedings ArticleDOI
Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network
Christian Ledig,Lucas Theis,Ferenc Huszar,Jose Caballero,Andrew Cunningham,Alejandro Acosta,Andrew Peter Aitken,Alykhan Tejani,Johannes Totz,Zehan Wang,Wenzhe Shi +10 more
TL;DR: SRGAN as mentioned in this paper proposes a perceptual loss function which consists of an adversarial loss and a content loss, which pushes the solution to the natural image manifold using a discriminator network that is trained to differentiate between the super-resolved images and original photo-realistic images.
Book ChapterDOI
Perceptual Losses for Real-Time Style Transfer and Super-Resolution
TL;DR: In this paper, the authors combine the benefits of both approaches, and propose the use of perceptual loss functions for training feed-forward networks for image style transfer, where a feedforward network is trained to solve the optimization problem proposed by Gatys et al. in real-time.