Book ChapterDOI
Space-Time Super-Resolution using Deep Learning-based Framework
Manoj Sharma,Santanu Chaudhury,Brejesh Lall +2 more
- pp 582-590
TLDR
The experimental results prove that the proposed H.264/AVC compatible framework performs better than the state-of-art techniques on space-time SR in terms of quality and time complexity.Abstract:
This paper introduces a novel end-to-end deep learning framework to learn space-time super-resolution (SR) process. We propose a coupled deep convolutional auto-encoder (CDCA) which learns the non-linear mapping between convolutional features of up-sampled low-resolution (LR) video sequence patches and convolutional features of high-resolution (HR) video sequence patches. The upsampling in LR video refers to tri-cubic interpolation both in space and time. We also propose a H.264/AVC compatible video space-time SR framework by using learned CDCA, which enables to super-resolve compressed LR video with less computational complexity. The experimental results prove that the proposed H.264/AVC compatible framework performs better than the state-of-art techniques on space-time SR in terms of quality and time complexity.read more
Citations
More filters
Proceedings ArticleDOI
Space-Time-Aware Multi-Resolution Video Enhancement
TL;DR: In this paper, the authors proposed a model called STARnet, which super-resolves jointly in space and time to leverage mutually informative relationships between time and space, which can provide more detailed information about motion and higher frame-rate can provide better pixel alignment.
Book ChapterDOI
Deep Space-Time Video Upsampling Networks
TL;DR: An end-to-end DNN framework for the space-time video upsampling by efficiently merging VSR and FI into a joint framework is proposed and a novel weighting scheme is proposed to fuse input frames effectively without explicit motion compensation for efficient processing of videos.
Proceedings ArticleDOI
Dual-Stream Fusion Network for Spatiotemporal Video Super-Resolution
TL;DR: Wang et al. as mentioned in this paper proposed a dual-stream fusion network to adaptively fuse the intermediate results produced by two spatio-temporal up-sampling streams, where the first stream applies the spatial superresolution followed by the temporal super-resolution, while the second one is with the reverse order of cascade.
Proceedings ArticleDOI
Spatiotemporal super-resolution with cross-task consistency and its semi-supervised extension
TL;DR: It turns out that the proposed cross-stream consistency does not consume labeled training data and can guide network training in an unsupervised manner, and can derive an effective model with few high-resolution and high-framerate videos, achieving the state-of-the-art performance.
Posted Content
Space-Time-Aware Multi-Resolution Video Enhancement
TL;DR: The components of the model that generate latent low- and high-resolution representations during ST-SR can be used to finetune a specialized mechanism for just spatial or just temporal super-resolution.
References
More filters
Proceedings ArticleDOI
Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network
Wenzhe Shi,Jose Caballero,Ferenc Huszar,Johannes Totz,Andrew Peter Aitken,Rob Bishop,Daniel Rueckert,Zehan Wang +7 more
TL;DR: This paper presents the first convolutional neural network capable of real-time SR of 1080p videos on a single K2 GPU and introduces an efficient sub-pixel convolution layer which learns an array of upscaling filters to upscale the final LR feature maps into the HR output.
Book ChapterDOI
Learning a Deep Convolutional Network for Image Super-Resolution
TL;DR: This work proposes a deep learning method for single image super-resolution (SR) that directly learns an end-to-end mapping between the low/high-resolution images and shows that traditional sparse-coding-based SR methods can also be viewed as a deep convolutional network.
Proceedings ArticleDOI
Accurate Image Super-Resolution Using Very Deep Convolutional Networks
TL;DR: In this article, a very deep convolutional network inspired by VGG-net was used for image superresolution, which achieved state-of-the-art performance in accuracy.
Posted Content
Accurate Image Super-Resolution Using Very Deep Convolutional Networks
TL;DR: This work presents a highly accurate single-image superresolution (SR) method using a very deep convolutional network inspired by VGG-net used for ImageNet classification and uses extremely high learning rates enabled by adjustable gradient clipping.
Journal ArticleDOI
Video Super-Resolution With Convolutional Neural Networks
TL;DR: This paper proposes a CNN that is trained on both the spatial and the temporal dimensions of videos to enhance their spatial resolution and shows that by using images to pretrain the model, a relatively small video database is sufficient for the training of the model to achieve and improve upon the current state-of-the-art.