Open AccessPosted Content
OpenPose: Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields
Reads0
Chats0
TLDR
OpenPose is released, the first open-source realtime system for multi-person 2D pose detection, including body, foot, hand, and facial keypoints, and the first combined body and foot keypoint detector, based on an internal annotated foot dataset.Abstract:
Realtime multi-person 2D pose estimation is a key component in enabling machines to have an understanding of people in images and videos. In this work, we present a realtime approach to detect the 2D pose of multiple people in an image. The proposed method uses a nonparametric representation, which we refer to as Part Affinity Fields (PAFs), to learn to associate body parts with individuals in the image. This bottom-up system achieves high accuracy and realtime performance, regardless of the number of people in the image. In previous work, PAFs and body part location estimation were refined simultaneously across training stages. We demonstrate that a PAF-only refinement rather than both PAF and body part location refinement results in a substantial increase in both runtime performance and accuracy. We also present the first combined body and foot keypoint detector, based on an internal annotated foot dataset that we have publicly released. We show that the combined detector not only reduces the inference time compared to running them sequentially, but also maintains the accuracy of each component individually. This work has culminated in the release of OpenPose, the first open-source realtime system for multi-person 2D pose detection, including body, foot, hand, and facial keypoints.read more
Citations
More filters
Posted Content
Objects as Points
TL;DR: The center point based approach, CenterNet, is end-to-end differentiable, simpler, faster, and more accurate than corresponding bounding box based detectors and performs competitively with sophisticated multi-stage methods and runs in real-time.
Journal ArticleDOI
Cellpose: a generalist algorithm for cellular segmentation
TL;DR: This work introduces a generalist, deep learning-based segmentation method called Cellpose, which can precisely segment cells from a wide range of image types and does not require model retraining or parameter adjustments.
Proceedings ArticleDOI
VIBE: Video Inference for Human Body Pose and Shape Estimation
TL;DR: This work defines a novel temporal network architecture with a self-attention mechanism and shows that adversarial training, at the sequence level, produces kinematically plausible motion sequences without in-the-wild ground-truth 3D labels.
Proceedings ArticleDOI
Soft Rasterizer: A Differentiable Renderer for Image-Based 3D Reasoning
TL;DR: This work proposes a truly differentiable rendering framework that is able to directly render colorized mesh using differentiable functions and back-propagate efficient supervision signals to mesh vertices and their attributes from various forms of image representations, including silhouette, shading and color images.
Proceedings ArticleDOI
Pose-Guided Feature Alignment for Occluded Person Re-Identification
TL;DR: This paper introduces a novel method named Pose-Guided Feature Alignment (PGFA), exploiting pose landmarks to disentangle the useful information from the occluded noise, and largely outperforms existing person re-id methods on three occlusion datasets, while remains top performance on two holistic datasets.
References
More filters
Proceedings ArticleDOI
Detect-and-Track: Efficient Pose Estimation in Videos
TL;DR: In this paper, the authors propose an extremely lightweight yet highly effective approach that operates in two-stages: keypoint estimation in frames or short clips, followed by lightweight tracking to generate keypoint predictions linked over the entire video.
Proceedings ArticleDOI
Articulated part-based model for joint object detection and pose estimation
Min Sun,Silvio Savarese +1 more
TL;DR: An Articulated Part-based Model for jointly detecting objects and estimating their poses is proposed and extensive quantitative and qualitative experiment results on public datasets show that APM outperforms state-of-the-art methods.
Book ChapterDOI
Recycle-GAN: Unsupervised Video Retargeting
TL;DR: In this paper, the authors introduce a data-driven approach for unsupervised video retargeting that translates content from one domain to another while preserving the style native to a domain, i.e., if contents of John Oliver's speech were to be transferred to Stephen Colbert, then the generated content/speech should be in Stephen Colbert's style.
Proceedings Article
Articulated Pose Estimation by a Graphical Model with Image Dependent Pairwise Relations
Xianjie Chen,Alan L. Yuille +1 more
TL;DR: In this paper, a graphical model for human pose estimation from a single static image is proposed, which exploits the fact the local image measurements can be used both to detect parts and also to predict the spatial relationships between them.
Book ChapterDOI
MultiPoseNet: Fast Multi-Person Pose Estimation using Pose Residual Network
Abstract: In this paper, we present MultiPoseNet, a novel bottom-up multi-person pose estimation architecture that combines a multi-task model with a novel assignment method. MultiPoseNet can jointly handle person detection, person segmentation and pose estimation problems. The novel assignment method is implemented by the Pose Residual Network (PRN) which receives keypoint and person detections, and produces accurate poses by assigning keypoints to person instances. On the COCO keypoints dataset, our pose estimation method outperforms all previous bottom-up methods both in accuracy (+4-point mAP over previous best result) and speed; it also performs on par with the best top-down methods while being at least 4x faster. Our method is the fastest real time system with \(\sim 23\) frames/sec.