Proceedings ArticleDOI
Predicting Forward & Backward Facial Depth Maps From a Single RGB Image For Mobile 3d AR Application
P Avinash,Mansi Sharma +1 more
- pp 1-8
Reads0
Chats0
TLDR
A novel deep learning based solution to predict robust depth maps of a face, one forward facing and the other backward facing, from a single image from the wild, by training a fully convolutional neural network to learn the dual depth maps.Abstract:
Cheap and fast 3D asset creation to enable AR/VR applications is a fast growing domain. This paper addresses a significant problem of reconstructing complete 3D information of a face in near real-time speed on a mobile phone. We propose a novel deep learning based solution to predict robust depth maps of a face, one forward facing and the other backward facing, from a single image from the wild. A critical contribution is that the proposed network is capable of learning the depths of the occluded part of the face too. This is achieved by training a fully convolutional neural network to learn the dual (forward and backward) depth maps, with a common encoder and two separate decoders. The 300W-LP, a cloud point dataset, is used to compute the required dual depth maps from the training data. The code and results will be made available at project page.read more
Citations
More filters
Journal ArticleDOI
Pano-SfMLearner: Self-Supervised Multi-Task Learning of Depth and Semantics in Panoramic Videos
TL;DR: Li et al. as discussed by the authors proposed a self-supervised framework for multi-task learning on depth, camera motion and semantics from panoramic videos, which is based on differentiable warping of adjacent views to the target.
Journal ArticleDOI
Geometry Sampling-Based Adaption to DCGAN for 3D Face Generation
Guoliang Luo,Guoming Xiong,Xiaojun Huang,Xin Zhao,Yang Tong,Qiang Chen,Zhiliang Zhu,Haopeng Lei,Juncong Lin +8 more
TL;DR: In this article , a geometric sampling method for the structured representation of 3D faces based on the intersection of iso-geodesic curves and radial curves, and a depth-like map sampling method using the average depth of grid cells on the front surface are proposed.
Journal ArticleDOI
2T-UNET: A Two-Tower UNet with Depth Clues for Robust Stereo Depth Estimation
TL;DR: The depth estimation problem is revisits, avoiding the explicit stereo matching step using a simple two-tower convolutional neural network, and the proposed algorithm is entitled 2T-UNet, which surpasses state-of-the-art monocular and stereo depth estimation methods on the challenging Scene dataset.
References
More filters
Proceedings Article
Adam: A Method for Stochastic Optimization
Diederik P. Kingma,Jimmy Ba +1 more
TL;DR: This work introduces Adam, an algorithm for first-order gradient-based optimization of stochastic objective functions, based on adaptive estimates of lower-order moments, and provides a regret bound on the convergence rate that is comparable to the best known results under the online convex optimization framework.
Proceedings ArticleDOI
Fully convolutional networks for semantic segmentation
TL;DR: The key insight is to build “fully convolutional” networks that take input of arbitrary size and produce correspondingly-sized output with efficient inference and learning.
Posted Content
MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
Andrew Howard,Menglong Zhu,Bo Chen,Dmitry Kalenichenko,Weijun Wang,Tobias Weyand,M. Andreetto,Hartwig Adam +7 more
TL;DR: This work introduces two simple global hyper-parameters that efficiently trade off between latency and accuracy and demonstrates the effectiveness of MobileNets across a wide range of applications and use cases including object detection, finegrain classification, face attributes and large scale geo-localization.
Journal ArticleDOI
DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs
TL;DR: This work addresses the task of semantic image segmentation with Deep Learning and proposes atrous spatial pyramid pooling (ASPP), which is proposed to robustly segment objects at multiple scales, and improves the localization of object boundaries by combining methods from DCNNs and probabilistic graphical models.
Posted Content
DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs
TL;DR: DeepLab as discussed by the authors proposes atrous spatial pyramid pooling (ASPP) to segment objects at multiple scales by probing an incoming convolutional feature layer with filters at multiple sampling rates and effective fields-of-views.