Semantically Guided Depth Upsampling
Nick Schneider,Nick Schneider,Lukas Schneider,Lukas Schneider,Peter Pinggera,Uwe Franke,Marc Pollefeys,Christoph Stiller +7 more
- pp 37-48
TLDR
This work presents a novel method for accurate and efficient upsampling of sparse depth data, guided by high-resolution imagery that determines globally consistent solutions and preserves fine details and sharp depth boundaries.Abstract:
We present a novel method for accurate and efficient upsampling of sparse depth data, guided by high-resolution imagery. Our approach goes beyond the use of intensity cues only and additionally exploits object boundary cues through structured edge detection and semantic scene labeling for guidance. Both cues are combined within a geodesic distance measure that allows for boundary-preserving depth interpolation while utilizing local context. We model the observed scene structure by locally planar elements and formulate the upsampling task as a global energy minimization problem. Our method determines globally consistent solutions and preserves fine details and sharp depth boundaries. In our experiments on several public datasets at different levels of application, we demonstrate superior performance of our approach over the state-of-the-art, even for very sparse measurements.read more
Citations
More filters
Proceedings ArticleDOI
Sparsity Invariant CNNs
TL;DR: This paper proposes a simple yet effective sparse convolution layer which explicitly considers the location of missing data during the convolution operation, and demonstrates the benefits of the proposed network architecture in synthetic and real experiments with respect to various baseline approaches.
Proceedings ArticleDOI
Self-Supervised Sparse-to-Dense: Self-Supervised Depth Completion from LiDAR and Monocular Camera
TL;DR: A deep regression model is developed to learn a direct mapping from sparse depth (and color images) input to dense depth prediction and a self-supervised training framework that requires only sequences of color and sparse depth images, without the need for dense depth labels is proposed.
Posted Content
Sparsity Invariant CNNs
TL;DR: In this article, the location of missing data is considered in the convolutional layer of the network and a simple sparse convolution layer is proposed for depth upsampling from sparse laser scan data.
Journal ArticleDOI
Learning Guided Convolutional Network for Depth Completion
TL;DR: Inspired by the guided image filtering, a novel guided network is designed to predict kernel weights from the guidance image and these predicted kernels are then applied to extract the depth image features.
Proceedings ArticleDOI
PENet: Towards Precise and Efficient Image Guided Depth Completion
TL;DR: PENet-ICRA2021 as mentioned in this paper proposes a two-branch backbone that consists of a color-dominant branch to exploit and fuse two modalities thoroughly, and a simple geometric convolutional layer to encode 3D geometric cues.
References
More filters
Proceedings ArticleDOI
Fully convolutional networks for semantic segmentation
TL;DR: The key insight is to build “fully convolutional” networks that take input of arbitrary size and produce correspondingly-sized output with efficient inference and learning.
Proceedings ArticleDOI
Are we ready for autonomous driving? The KITTI vision benchmark suite
TL;DR: The autonomous driving platform is used to develop novel challenging benchmarks for the tasks of stereo, optical flow, visual odometry/SLAM and 3D object detection, revealing that methods ranking high on established datasets such as Middlebury perform below average when being moved outside the laboratory to the real world.
Proceedings ArticleDOI
The Cityscapes Dataset for Semantic Urban Scene Understanding
Marius Cordts,Mohamed Omran,Sebastian Ramos,Timo Rehfeld,Markus Enzweiler,Rodrigo Benenson,Uwe Franke,Stefan Roth,Bernt Schiele +8 more
TL;DR: This work introduces Cityscapes, a benchmark suite and large-scale dataset to train and test approaches for pixel-level and instance-level semantic labeling, and exceeds previous attempts in terms of dataset size, annotation richness, scene variability, and complexity.
Journal ArticleDOI
Vision meets robotics: The KITTI dataset
TL;DR: A novel dataset captured from a VW station wagon for use in mobile robotics and autonomous driving research, using a variety of sensor modalities such as high-resolution color and grayscale stereo cameras and a high-precision GPS/IMU inertial navigation system.
Journal ArticleDOI
The Pascal Visual Object Classes Challenge: A Retrospective
TL;DR: A review of the Pascal Visual Object Classes challenge from 2008-2012 and an appraisal of the aspects of the challenge that worked well, and those that could be improved in future challenges.
Related Papers (5)
Sparse-to-Dense: Depth Prediction from Sparse Depth Samples and a Single Image
Fangchang Mal,Sertac Karaman +1 more