A Novel 3D-Unet Deep Learning Framework Based on High-Dimensional Bilateral Grid for Edge Consistent Single Image Depth Estimation
Mansi Sharma,Abheesht Sharma,Kadvekar Rohit Tushar,Avinash Panneer +3 more
- pp 1-8
Reads0
Chats0
TLDR
Wang et al. as discussed by the authors proposed a novel Bilateral grid based 3D convolutional neural network, dubbed as 3DBG-UNet, that parameterize high dimensional feature space by encoding compact 3D bilateral grids with UNets and infers sharp geometric layout of the scene.Abstract:
The task of predicting smooth and edge-consistent depth maps is notoriously difficult for single image depth estimation. This paper proposes a novel Bilateral Grid based 3D convolutional neural network, dubbed as 3DBG-UNet, that parameterize high dimensional feature space by encoding compact 3D bilateral grids with UNets and infers sharp geometric layout of the scene. Further, an another novel 3DBGES-UNet model is introduced that integrate 3DBG-UNet for inferring an accurate depth map given a single color view. The 3DBGES-UNet concatenate 3DBG-UNet geometry map with the inception network edge accentuation map and a spatial object's boundary map obtained by leveraging semantic segmentation and train the UNet model with ResNet backbone. Both models are designed with a particular attention to explicitly account for edges or minute details. Preserving sharp discontinuities at depth edges is critical for many applications such as realistic integration of virtual objects in AR video or occlusion-aware view synthesis for 3D display applications. The proposed depth prediction network achieves state-of-the-art performance in both qualitative and quantitative evaluations on the challenging NYUv2-Depth data. The code and corresponding pre-trained weights will be made publicly available.read more
Citations
More filters
Journal ArticleDOI
Deep learning for reduced order modelling and efficient temporal evolution of fluid simulations
TL;DR: In this paper, the authors develop a deep learning framework DL-ROM (deep learning-reduced order modeling) to create a neural network capable of non-linear projections to reduced order states.
Journal ArticleDOI
MEStereo-Du2CNN: A Novel Dual Channel CNN for Learning Robust Depth Estimates from Multi-exposure Stereo Images for HDR 3D Applications
TL;DR: In this article , a dual-encoder single-decoder CNN with different weights for feature fusion is proposed for depth estimation of multi-exposure stereo image sequences in 3D HDR video content.
Journal ArticleDOI
2T-UNET: A Two-Tower UNet with Depth Clues for Robust Stereo Depth Estimation
TL;DR: The depth estimation problem is revisits, avoiding the explicit stereo matching step using a simple two-tower convolutional neural network, and the proposed algorithm is entitled 2T-UNet, which surpasses state-of-the-art monocular and stereo depth estimation methods on the challenging Scene dataset.
Journal ArticleDOI
An unsupervised monocular image depth prediction algorithm using Fourier domain analysis
Lifang Chen,Xiaojiao Tang +1 more
TL;DR: Zhang et al. as discussed by the authors proposed an unsupervised monocular image depth prediction algorithm based on Fourier domain analysis to take advantage of the complementary properties of small-scale and large-scale images.
Journal ArticleDOI
Nested DWT–Based CNN Architecture for Monocular Depth Estimation
TL;DR: NDWTN as mentioned in this paper proposes a moderately dense encoder-decoder network based on discrete wavelet decomposition and trainable coefficients (LL, LH, HL, HH), which preserves the highfrequency information that is otherwise lost during the downsampling process in the encoder.
References
More filters
Proceedings ArticleDOI
Digging Into Self-Supervised Monocular Depth Estimation
TL;DR: In this paper, the authors propose a set of improvements, which together result in both quantitatively and qualitatively improved depth maps compared to competing self-supervised methods, and demonstrate the effectiveness of each component in isolation, and show high quality, state-of-theart results on the KITTI benchmark.
Journal ArticleDOI
A Fast Approximation of the Bilateral Filter Using a Signal Processing Approach
Sylvain Paris,Frédo Durand +1 more
TL;DR: In this article, the authors proposed a new signal processing analysis of the bilateral filter, which complements the recent studies that analyzed it as a PDE or as a robust statistical estimator.
Proceedings ArticleDOI
Real-time edge-aware image processing with the bilateral grid
TL;DR: A new data structure---the bilateral grid, that enables fast edge-aware image processing that parallelize the algorithms on modern GPUs to achieve real-time frame rates on high-definition video.
Proceedings ArticleDOI
Multi-scale Continuous CRFs as Sequential Deep Networks for Monocular Depth Estimation
TL;DR: In this article, a deep model which fuses complementary information derived from multiple CNN side outputs is proposed, which is obtained by means of continuous Conditional Random Fields (CRFs).
Posted Content
High Quality Monocular Depth Estimation via Transfer Learning.
Ibraheem Alhashim,Peter Wonka +1 more
TL;DR: A convolutional neural network for computing a high-resolution depth map given a single RGB image with the help of transfer learning, which outperforms state-of-the-art on two datasets and also produces qualitatively better results that capture object boundaries more faithfully.
Related Papers (5)
A Two-Streamed Network for Estimating Fine-Scaled Depth Maps from Single RGB Images
Jun Li,Reinhard Klein,Angela Yao +2 more