Learning to Fuse 2D and 3D Image Cues for Monocular Body Pose Estimation
Bugra Tekin,Pablo Márquez-Neila,Mathieu Salzmann,Pascal Fua +3 more
- pp 3961-3970
Reads0
Chats0
TLDR
In this paper, a trainable fusion scheme is proposed to fuse the information optimally instead of being hand-designed, which yields significant improvements on standard 3D human pose estimation benchmarks.Abstract:
Most recent approaches to monocular 3D human pose estimation rely on Deep Learning. They typically involve regressing from an image to either 3D joint coordinates directly or 2D joint locations from which 3D coordinates are inferred. Both approaches have their strengths and weaknesses and we therefore propose a novel architecture designed to deliver the best of both worlds by performing both simultaneously and fusing the information along the way. At the heart of our framework is a trainable fusion scheme that learns how to fuse the information optimally instead of being hand-designed. This yields significant improvements upon the state-of-the-art on standard 3D human pose estimation benchmarks.read more
Citations
More filters
Proceedings ArticleDOI
End-to-End Recovery of Human Shape and Pose
TL;DR: This work introduces an adversary trained to tell whether human body shape and pose parameters are real or not using a large database of 3D human meshes, and produces a richer and more useful mesh representation that is parameterized by shape and 3D joint angles.
Journal ArticleDOI
VNect: real-time 3D human pose estimation with a single RGB camera
Dushyant Mehta,Srinath Sridhar,Oleksandr Sotnychenko,Helge Rhodin,Mohammad Shafiei,Hans-Peter Seidel,Weipeng Xu,Dan Casas,Christian Theobalt +8 more
TL;DR: In this paper, a fully-convolutional pose formulation was proposed to regress 2D and 3D joint positions jointly in real-time and does not require tightly cropped input frames.
Proceedings ArticleDOI
Learning to Reconstruct 3D Human Pose and Shape via Model-Fitting in the Loop
TL;DR: SPIN as discussed by the authors uses a deep network to initialize an iterative optimization routine that fits the body model to 2D joints within the training loop, and the fitted estimate is subsequently used to supervise the network.
Journal ArticleDOI
VNect: Real-time 3D Human Pose Estimation with a Single RGB Camera
Dushyant Mehta,Srinath Sridhar,Oleksandr Sotnychenko,Helge Rhodin,Mohammad Shafiei,Hans-Peter Seidel,Weipeng Xu,Dan Casas,Christian Theobalt +8 more
TL;DR: This work presents the first real-time method to capture the full global 3D skeletal pose of a human in a stable, temporally consistent manner using a single RGB camera and shows that the approach is more broadly applicable than RGB-D solutions, i.e., it works for outdoor scenes, community videos, and low quality commodity RGB cameras.
Proceedings ArticleDOI
Learning to Estimate 3D Human Pose and Shape from a Single Color Image
TL;DR: This work addresses the problem of estimating the full body 3D human pose and shape from a single color image and proposes an efficient and effective direct prediction method based on ConvNets, incorporating a parametric statistical body shape model (SMPL) within an end-to-end framework.
References
More filters
Proceedings Article
Adam: A Method for Stochastic Optimization
Diederik P. Kingma,Jimmy Ba +1 more
TL;DR: This work introduces Adam, an algorithm for first-order gradient-based optimization of stochastic objective functions, based on adaptive estimates of lower-order moments, and provides a regret bound on the convergence rate that is comparable to the best known results under the online convex optimization framework.
Proceedings Article
Very Deep Convolutional Networks for Large-Scale Image Recognition
Karen Simonyan,Andrew Zisserman +1 more
TL;DR: In this paper, the authors investigated the effect of the convolutional network depth on its accuracy in the large-scale image recognition setting and showed that a significant improvement on the prior-art configurations can be achieved by pushing the depth to 16-19 layers.
Book ChapterDOI
U-Net: Convolutional Networks for Biomedical Image Segmentation
TL;DR: Neber et al. as discussed by the authors proposed a network and training strategy that relies on the strong use of data augmentation to use the available annotated samples more efficiently, which can be trained end-to-end from very few images and outperforms the prior best method (a sliding-window convolutional network) on the ISBI challenge for segmentation of neuronal structures in electron microscopic stacks.
Posted Content
U-Net: Convolutional Networks for Biomedical Image Segmentation
TL;DR: It is shown that such a network can be trained end-to-end from very few images and outperforms the prior best method (a sliding-window convolutional network) on the ISBI challenge for segmentation of neuronal structures in electron microscopic stacks.
Book ChapterDOI
Stacked Hourglass Networks for Human Pose Estimation
TL;DR: This work introduces a novel convolutional network architecture for the task of human pose estimation that is described as a “stacked hourglass” network based on the successive steps of pooling and upsampling that are done to produce a final set of predictions.