Monocular 3D Human Pose Estimation in the Wild Using Improved CNN Supervision
Dushyant Mehta,Helge Rhodin,Dan Casas,Pascal Fua,Oleksandr Sotnychenko,Weipeng Xu,Christian Theobalt +6 more
- Vol. 271, Iss: 20, pp 506-516
Reads0
Chats0
TLDR
In this article, a CNN-based approach for 3D human body pose estimation from single RGB images is proposed to address the issue of limited generalizability of models trained solely on the starkly limited publicly available 3D pose data.Abstract:
We propose a CNN-based approach for 3D human body pose estimation from single RGB images that addresses the issue of limited generalizability of models trained solely on the starkly limited publicly available 3D pose data. Using only the existing 3D pose data and 2D pose data, we show state-of-the-art performance on established benchmarks through transfer of learned features, while also generalizing to in-the-wild scenes. We further introduce a new training set for human body pose estimation from monocular images of real humans that has the ground truth captured with a multi-camera marker-less motion capture system. It complements existing corpora with greater diversity in pose, human appearance, clothing, occlusion, and viewpoints, and enables an increased scope of augmentation. We also contribute a new benchmark that covers outdoor and indoor scenes, and demonstrate that our 3D pose dataset shows better in-the-wild performance than existing annotated data, which is further improved in conjunction with transfer learning from 2D pose data. All in all, we argue that the use of transfer learning of representations in tandem with algorithmic and data contributions is crucial for general 3D body pose estimation.read more
Citations
More filters
Proceedings ArticleDOI
End-to-End Recovery of Human Shape and Pose
TL;DR: This work introduces an adversary trained to tell whether human body shape and pose parameters are real or not using a large database of 3D human meshes, and produces a richer and more useful mesh representation that is parameterized by shape and 3D joint angles.
Journal ArticleDOI
VNect: real-time 3D human pose estimation with a single RGB camera
Dushyant Mehta,Srinath Sridhar,Oleksandr Sotnychenko,Helge Rhodin,Mohammad Shafiei,Hans-Peter Seidel,Weipeng Xu,Dan Casas,Christian Theobalt +8 more
TL;DR: In this paper, a fully-convolutional pose formulation was proposed to regress 2D and 3D joint positions jointly in real-time and does not require tightly cropped input frames.
Proceedings ArticleDOI
Learning to Reconstruct 3D Human Pose and Shape via Model-Fitting in the Loop
TL;DR: SPIN as discussed by the authors uses a deep network to initialize an iterative optimization routine that fits the body model to 2D joints within the training loop, and the fitted estimate is subsequently used to supervise the network.
Proceedings ArticleDOI
VIBE: Video Inference for Human Body Pose and Shape Estimation
TL;DR: This work defines a novel temporal network architecture with a self-attention mechanism and shows that adversarial training, at the sequence level, produces kinematically plausible motion sequences without in-the-wild ground-truth 3D labels.
Book ChapterDOI
Recovering Accurate {3D} Human Pose in the Wild Using {IMUs} and a Moving Camera
TL;DR: This work proposes a method that combines a single hand-held camera and a set of Inertial Measurement Units (IMUs) attached at the body limbs to estimate accurate 3D poses in the wild and obtains an accuracy of 26 mm, which makes it accurate enough to serve as a benchmark for image-based 3D pose estimation in theWild.
References
More filters
Proceedings ArticleDOI
Deep Residual Learning for Image Recognition
TL;DR: In this article, the authors proposed a residual learning framework to ease the training of networks that are substantially deeper than those used previously, which won the 1st place on the ILSVRC 2015 classification task.
Journal ArticleDOI
ImageNet Large Scale Visual Recognition Challenge
Olga Russakovsky,Jia Deng,Hao Su,Jonathan Krause,Sanjeev Satheesh,Sean Ma,Zhiheng Huang,Andrej Karpathy,Aditya Khosla,Michael S. Bernstein,Alexander C. Berg,Li Fei-Fei +11 more
TL;DR: The ImageNet Large Scale Visual Recognition Challenge (ILSVRC) as mentioned in this paper is a benchmark in object category classification and detection on hundreds of object categories and millions of images, which has been run annually from 2010 to present, attracting participation from more than fifty institutions.
Proceedings ArticleDOI
Fully convolutional networks for semantic segmentation
TL;DR: The key insight is to build “fully convolutional” networks that take input of arbitrary size and produce correspondingly-sized output with efficient inference and learning.
Journal ArticleDOI
A Survey on Transfer Learning
Sinno Jialin Pan,Qiang Yang +1 more
TL;DR: The relationship between transfer learning and other related machine learning techniques such as domain adaptation, multitask learning and sample selection bias, as well as covariate shift are discussed.
Proceedings Article
How transferable are features in deep neural networks
TL;DR: In this paper, the authors quantify the transferability of features from the first layer to the last layer of a deep neural network and show that transferability is negatively affected by two distinct issues: (1) the specialization of higher layer neurons to their original task at the expense of performance on the target task and (2) optimization difficulties related to splitting networks between co-adapted neurons.