Proceedings ArticleDOI
Learning Structure-And-Motion-Aware Rolling Shutter Correction
Bingbing Zhuang,Quoc-Huy Tran,Pan Ji,Loong-Fah Cheong,Manmohan Chandraker +4 more
- pp 4551-4560
Reads0
Chats0
TLDR
This paper first makes a theoretical contribution by showing that RS two-view geometry is degenerate in the case of pure translational camera motion, and proposes a Convolutional Neural Network (CNN)-based method which learns the underlying geometry from just a single RS image and performs RS image correction.Abstract:
An exact method of correcting the rolling shutter (RS) effect requires recovering the underlying geometry, i.e. the scene structures and the camera motions between scanlines or between views. However, the multiple-view geometry for RS cameras is much more complicated than its global shutter (GS) counterpart, with various degeneracies. In this paper, we first make a theoretical contribution by showing that RS two-view geometry is degenerate in the case of pure translational camera motion. In view of the complex RS geometry, we then propose a Convolutional Neural Network (CNN)-based method which learns the underlying geometry (camera motion and scene structure) from just a single RS image and perform RS image correction. We call our method structure-and-motion-aware RS correction because it reasons about the concealed motions between the scanlines as well as the scene structure. Our method learns from a large-scale dataset synthesized in a geometrically meaningful way where the RS effect is generated in a manner consistent with the camera motion and scene structure. In extensive experiments, our method achieves superior performance compared to other state-of-the-art methods for single image RS correction and subsequent Structure from Motion (SfM) applications.read more
Citations
More filters
Book ChapterDOI
Learning Monocular Visual Odometry via Self-Supervised Long-Term Modeling
TL;DR: This paper model the long-term dependency in pose prediction using a pose network that features a two-layer convolutional LSTM module, and proposes a stage-wise training mechanism, where the first stage operates in a local time window and the second stage refines the poses with a "global" loss given the firststage features.
Proceedings ArticleDOI
Deep Shutter Unrolling Network
TL;DR: A novel network for rolling shutter effect correction that can be trained end-to-end and only requires the global shutter image for supervision is presented and experimental results demonstrate that it outperforms the state-of-the-art methods.
Proceedings ArticleDOI
Depth-Guided Dense Dynamic Filtering Network for Bokeh Effect Rendering
TL;DR: The proposed network is composed of an efficient densely connected encoder-decoder backbone structure with a pyramid pooling module that leverages the task-specific efficacy of joint intensity estimation and dynamic filter synthesis for the spatially-aware blurring process.
Proceedings ArticleDOI
Degeneracy in Self-Calibration Revisited and a Deep Learning Solution for Uncalibrated SLAM
TL;DR: A theoretical proof that explicitly shows the ambiguity between radial distortion and scene depth when two-view geometry is used to self-calibrate the radial distortion is provided, and a learning approach that trains a convolutional neural network on a large amount of synthetic data is proposed.
Journal ArticleDOI
LSTM and Filter Based Comparison Analysis for Indoor Global Localization in UAVs
TL;DR: In this paper, a sequential, end-to-end, and multimodal deep neural network based monocular visual-inertial localization framework was proposed for indoor UAVs.
References
More filters
Proceedings ArticleDOI
Deep Residual Learning for Image Recognition
TL;DR: In this article, the authors proposed a residual learning framework to ease the training of networks that are substantially deeper than those used previously, which won the 1st place on the ILSVRC 2015 classification task.
Proceedings Article
Adam: A Method for Stochastic Optimization
Diederik P. Kingma,Jimmy Ba +1 more
TL;DR: This work introduces Adam, an algorithm for first-order gradient-based optimization of stochastic objective functions, based on adaptive estimates of lower-order moments, and provides a regret bound on the convergence rate that is comparable to the best known results under the online convex optimization framework.
Book
Multiple view geometry in computer vision
Richard Hartley,Andrew Zisserman +1 more
TL;DR: In this article, the authors provide comprehensive background material and explain how to apply the methods and implement the algorithms directly in a unified framework, including geometric principles and how to represent objects algebraically so they can be computed and applied.
Multiple View Geometry in Computer Vision.
TL;DR: This book is referred to read because it is an inspiring book to give you more chance to get experiences and also thoughts and it will show the best book collections and completed collections.
Automatic differentiation in PyTorch
Adam Paszke,Sam Gross,Soumith Chintala,Gregory Chanan,Edward Z. Yang,Zachary DeVito,Zeming Lin,Alban Desmaison,Luca Antiga,Adam Lerer +9 more
TL;DR: An automatic differentiation module of PyTorch is described — a library designed to enable rapid research on machine learning models that focuses on differentiation of purely imperative programs, with a focus on extensibility and low overhead.