SE3-nets: Learning rigid body motion using deep neural networks
Arunkumar Byravan,Dieter Fox +1 more
- pp 173-180
TLDR
SE3-Nets as discussed by the authors are deep neural networks designed to model and learn rigid body motion from raw point cloud data based on sequences of depth images along with action vectors and point wise data associations.Abstract:
We introduce SE3-Nets which are deep neural networks designed to model and learn rigid body motion from raw point cloud data. Based only on sequences of depth images along with action vectors and point wise data associations, SE3-Nets learn to segment effected object parts and predict their motion resulting from the applied force. Rather than learning point wise flow vectors, SE3-Nets predict SE(3) transformations for different parts of the scene. Using simulated depth data of a table top scene and a robot manipulator, we show that the structure underlying SE3-Nets enables them to generate a far more consistent prediction of object motion than traditional flow based networks. Additional experiments with a depth camera observing a Baxter robot pushing objects on a table show that SE3-Nets also work well on real data.read more
Citations
More filters
Proceedings ArticleDOI
Digging Into Self-Supervised Monocular Depth Estimation
TL;DR: In this paper, the authors propose a set of improvements, which together result in both quantitatively and qualitatively improved depth maps compared to competing self-supervised methods, and demonstrate the effectiveness of each component in isolation, and show high quality, state-of-theart results on the KITTI benchmark.
Posted Content
Digging Into Self-Supervised Monocular Depth Estimation
TL;DR: It is shown that a surprisingly simple model, and associated design choices, lead to superior predictions, and together result in both quantitatively and qualitatively improved depth maps compared to competing self-supervised methods.
Journal ArticleDOI
The limits and potentials of deep learning for robotics
Niko Sünderhauf,Oliver Brock,Walter J. Scheirer,Raia Hadsell,Dieter Fox,Jürgen Leitner,Ben Upcroft,Pieter Abbeel,Wolfram Burgard,Michael Milford,Peter Corke +10 more
TL;DR: The need for better evaluation metrics is explained, the importance and unique challenges for deep robotic learning in simulation are highlighted, and the spectrum between purely data-driven and model-driven approaches is explored.
Journal ArticleDOI
Recent Advances in Robot Learning from Demonstration
TL;DR: In the context of robotics and automation, learning from demonstration (LfD) is the paradigm in which robots acquire new skills by learning to imitate an expert.
Journal ArticleDOI
Visual SLAM and Structure from Motion in Dynamic Environments: A Survey
TL;DR: This article presents for the first time a survey of visual SLAM and SfM techniques that are targeted toward operation in dynamic environments and identifies three main problems: how to perform reconstruction, how to segment and track dynamic objects, and how to achieve joint motion segmentation and reconstruction.
References
More filters
Proceedings ArticleDOI
Fully convolutional networks for semantic segmentation
TL;DR: The key insight is to build “fully convolutional” networks that take input of arbitrary size and produce correspondingly-sized output with efficient inference and learning.
Proceedings Article
Spatial transformer networks
TL;DR: This work introduces a new learnable module, the Spatial Transformer, which explicitly allows the spatial manipulation of data within the network, and can be inserted into existing convolutional architectures, giving neural networks the ability to actively spatially transform feature maps.
Proceedings ArticleDOI
MuJoCo: A physics engine for model-based control
TL;DR: A new physics engine tailored to model-based control, based on the modern velocity-stepping approach which avoids the difficulties with spring-dampers, which can compute both forward and inverse dynamics.
Proceedings Article
Torch7: A Matlab-like Environment for Machine Learning
TL;DR: Torch7 is a versatile numeric computing framework and machine learning library that extends Lua that can easily be interfaced to third-party software thanks to Lua’s light interface.
Proceedings Article
PILCO: A Model-Based and Data-Efficient Approach to Policy Search
TL;DR: PILCO reduces model bias, one of the key problems of model-based reinforcement learning, in a principled way by learning a probabilistic dynamics model and explicitly incorporating model uncertainty into long-term planning.