Proceedings ArticleDOI
Deep Video Deblurring for Hand-Held Cameras
Shuochen Su,Mauricio Delbracio,Jue Wang,Guillermo Sapiro,Wolfgang Heidrich,Oliver Wang +5 more
- pp 237-246
TLDR
This work introduces a deep learning solution to video deblurring, where a CNN is trained end-to-end to learn how to accumulate information across frames, and shows that the features learned extend todeblurring motion blur that arises due to camera shake in a wide range of videos.Abstract:
Motion blur from camera shake is a major problem in videos captured by hand-held devices. Unlike single-image deblurring, video-based approaches can take advantage of the abundant information that exists across neighboring frames. As a result the best performing methods rely on the alignment of nearby frames. However, aligning images is a computationally expensive and fragile procedure, and methods that aggregate information must therefore be able to identify which regions have been accurately aligned and which have not, a task that requires high level scene understanding. In this work, we introduce a deep learning solution to video deblurring, where a CNN is trained end-to-end to learn how to accumulate information across frames. To train this network, we collected a dataset of real videos recorded with a high frame rate camera, which we use to generate synthetic motion blur for supervision. We show that the features learned from this dataset extend to deblurring motion blur that arises due to camera shake in a wide range of videos, and compare the quality of results to a number of other baselines.read more
Citations
More filters
Posted Content
The Unreasonable Effectiveness of Deep Features as a Perceptual Metric
TL;DR: A new dataset of human perceptual similarity judgments is introduced and it is found that deep features outperform all previous metrics by large margins on this dataset, and suggests that perceptual similarity is an emergent property shared across deep visual representations.
Proceedings ArticleDOI
The Unreasonable Effectiveness of Deep Features as a Perceptual Metric
TL;DR: In this paper, the authors introduce a new dataset of human perceptual similarity judgments, and systematically evaluate deep features across different architectures and tasks and compare them with classic metrics, finding that deep features outperform all previous metrics by large margins on their dataset.
Proceedings ArticleDOI
EDVR: Video Restoration With Enhanced Deformable Convolutional Networks
TL;DR: This work proposes a novel Video Restoration framework with Enhanced Deformable convolutions, termed EDVR, and proposes a Temporal and Spatial Attention (TSA) fusion module, in which attention is applied both temporally and spatially, so as to emphasize important features for subsequent restoration.
Proceedings ArticleDOI
DeblurGAN-v2: Deblurring (Orders-of-Magnitude) Faster and Better
TL;DR: It is demonstrated that DeblurGAN-V2 has very competitive performance on several popular benchmarks, in terms of deblurring quality (both objective and subjective), as well as efficiency, and is effective for general image restoration tasks too.
Proceedings ArticleDOI
NTIRE 2019 Challenge on Video Deblurring and Super-Resolution: Dataset and Study
Seungjun Nah,Sungyong Baik,Seokil Hong,Gyeongsik Moon,Sanghyun Son,Radu Timofte,Kyoung Mu Lee +6 more
TL;DR: It is found that the NTIRE 2019 challenges push the state-of-the-art in video deblurring and super-resolution, reaching compelling performance on the newly proposed REDS dataset.
References
More filters
Proceedings ArticleDOI
Deep Residual Learning for Image Recognition
TL;DR: In this article, the authors proposed a residual learning framework to ease the training of networks that are substantially deeper than those used previously, which won the 1st place on the ILSVRC 2015 classification task.
Proceedings Article
Adam: A Method for Stochastic Optimization
Diederik P. Kingma,Jimmy Ba +1 more
TL;DR: This work introduces Adam, an algorithm for first-order gradient-based optimization of stochastic objective functions, based on adaptive estimates of lower-order moments, and provides a regret bound on the convergence rate that is comparable to the best known results under the online convex optimization framework.
Proceedings ArticleDOI
Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network
Wenzhe Shi,Jose Caballero,Ferenc Huszar,Johannes Totz,Andrew Peter Aitken,Rob Bishop,Daniel Rueckert,Zehan Wang +7 more
TL;DR: This paper presents the first convolutional neural network capable of real-time SR of 1080p videos on a single K2 GPU and introduces an efficient sub-pixel convolution layer which learns an array of upscaling filters to upscale the final LR feature maps into the HR output.
Book ChapterDOI
Learning a Deep Convolutional Network for Image Super-Resolution
TL;DR: This work proposes a deep learning method for single image super-resolution (SR) that directly learns an end-to-end mapping between the low/high-resolution images and shows that traditional sparse-coding-based SR methods can also be viewed as a deep convolutional network.
Proceedings ArticleDOI
Context Encoders: Feature Learning by Inpainting
TL;DR: It is found that a context encoder learns a representation that captures not just appearance but also the semantics of visual structures, and can be used for semantic inpainting tasks, either stand-alone or as initialization for non-parametric methods.