DUST: Dual Union of Spatio-Temporal Subspaces for Monocular Multiple Object 3D Reconstruction
Antonio Agudo,Francesc Moreno-Noguer +1 more
- pp 1513-1521
Reads0
Chats0
TLDR
An approach to reconstruct the 3D shape of multiple deforming objects from incomplete 2D trajectories acquired by a single camera is presented, which achieves state-of-the-art 3D reconstruction results, while it also provides spatial and temporal segmentation.Abstract:
We present an approach to reconstruct the 3D shape of multiple deforming objects from incomplete 2D trajectories acquired by a single camera. Additionally, we simultaneously provide spatial segmentation (i.e., we identify each of the objects in every frame) and temporal clustering (i.e., we split the sequence into primitive actions). This advances existing work, which only tackled the problem for one single object and non-occluded tracks. In order to handle several objects at a time from partial observations, we model point trajectories as a union of spatial and temporal subspaces, and optimize the parameters of both modalities, the non-observed point tracks and the 3D shape via augmented Lagrange multipliers. The algorithm is fully unsupervised and results in a formulation which does not need initialization. We thoroughly validate the method on challenging scenarios with several human subjects performing different activities which involve complex motions and close interaction. We show our approach achieves state-of-the-art 3D reconstruction results, while it also provides spatial and temporal segmentation.read more
Citations
More filters
Proceedings ArticleDOI
C3DPO: Canonical 3D Pose Networks for Non-Rigid Structure From Motion
TL;DR: This work proposes C3DPO, a method for extracting 3D models of deformable objects from 2D keypoint annotations in unconstrained images by learning a deep network that reconstructs a 3D object from a single view at a time, and introduces a novel regularization technique.
Posted Content
Human Motion Prediction via Spatio-Temporal Inpainting
TL;DR: This work argues that the L2 metric, considered so far by most approaches, fails to capture the actual distribution of long-term human motion, and proposes two alternative metrics, based on the distribution of frequencies, that are able to capture more realistic motion patterns.
Proceedings ArticleDOI
Human Motion Prediction via Spatio-Temporal Inpainting
TL;DR: In this article, a Generative Adversarial Network (GAN) is proposed to forecast 3D human motion given a sequence of past 3D skeleton poses. But their method is limited to a relatively short period of time and typically ignore the absolute position of the skeleton w.r.t. the camera.
Proceedings ArticleDOI
Geometry-Aware Network for Non-rigid Shape Prediction from a Single View
Albert Pumarola,Antonio Agudo,Lorenzo Porzi,Alberto Sanfeliu,Vincent Lepetit,Francesc Moreno-Noguer +5 more
TL;DR: A geometry-aware deep architecture that tackles the problem as usually done in analytic solutions: first perform 2D detection of the mesh and then estimate a 3D shape that is geometrically consistent with the image with a significantly lower computational time.
Proceedings ArticleDOI
Deep Facial Non-Rigid Multi-View Stereo
TL;DR: This method optimizes the 3D face shape by explicitly enforcing multi-view appearance consistency, which is known to be effective in recovering shape details according to conventional multi- view stereo methods.
References
More filters
Book
Distributed Optimization and Statistical Learning Via the Alternating Direction Method of Multipliers
TL;DR: It is argued that the alternating direction method of multipliers is well suited to distributed convex optimization, and in particular to large-scale problems arising in statistics, machine learning, and related areas.
Journal ArticleDOI
A Singular Value Thresholding Algorithm for Matrix Completion
TL;DR: This paper develops a simple first-order and easy-to-implement algorithm that is extremely efficient at addressing problems in which the optimal solution has low rank, and develops a framework in which one can understand these algorithms in terms of well-known Lagrange multiplier algorithms.
Journal ArticleDOI
Robust Recovery of Subspace Structures by Low-Rank Representation
TL;DR: It is shown that the convex program associated with LRR solves the subspace clustering problem in the following sense: When the data is clean, LRR exactly recovers the true subspace structures; when the data are contaminated by outliers, it is proved that under certain conditions LRR can exactly recover the row space of the original data.
Journal ArticleDOI
Shape and motion from image streams under orthography: a factorization method
Carlo Tomasi,Takeo Kanade +1 more
TL;DR: In this paper, the singular value decomposition (SVDC) technique is used to factor the measurement matrix into two matrices which represent object shape and camera rotation respectively, and two of the three translation components are computed in a preprocessing stage.
Journal ArticleDOI
A collaborative framework for 3D alignment and classification of heterogeneous subvolumes in cryo-electron tomography
Oleg Kuybeda,Gabriel A. Frank,Alberto Bartesaghi,Mario J. Borgnia,Sriram Subramaniam,Guillermo Sapiro +5 more
TL;DR: The genetic identity of each virus particle present in the mixture can be assigned based solely on the structural information derived from single envelope glycoproteins displayed on the virus surface by the nuclear norm-based, collaborative alignment method presented here.