Proceedings ArticleDOI
Learning Agent-Aware Affordances for Closed-Loop Interaction with Articulated Objects
Giulio Schiavi,Paula Wulkop,Giuseppe Maria Rizzi,Lionel Ott,Roland Siegwart,Jen Jen Chung +5 more
- pp 5916-5922
TLDR
The concept of agent-aware affordances which fully reflect the agent’s capabilities and embodiment are introduced and it is shown that they outperform their state-of-the-art counterparts which are only conditioned on the end-effector geometry.Abstract:
Interactions with articulated objects are a challenging but important task for mobile robots. To tackle this challenge, we propose a novel closed-loop control pipeline, which integrates manipulation priors from affordance estimation with sampling-based whole-body control. We introduce the concept of agent-aware affordances which fully reflect the agent's capabilities and embodiment and we show that they outperform their state-of-the-art counterparts which are only conditioned on the end-effector geometry. Additionally, closed-loop affordance inference is found to allow the agent to divide a task into multiple non-continuous motions and recover from failure and unexpected states. Finally, the pipeline is able to perform long-horizon mobile manipulation tasks, i.e. opening and closing an oven, in the real world with high success rates (opening: 71%, closing: 72%).read more
Citations
More filters
Journal ArticleDOI
Revisiting Proprioceptive Sensing for Articulated Object Manipulation
Thomas Lips,Francis wyffels +1 more
TL;DR: In this paper , a system that uses proprioceptive sensing to open cabinets with a position-controlled robot and a parallel gripper is presented, and a qualitative evaluation of this system is performed, where the slip between the gripper and handle limits the performance.
References
More filters
Book
The Ecological Approach to Visual Perception
TL;DR: The relationship between Stimulation and Stimulus Information for visual perception is discussed in detail in this article, where the authors also present experimental evidence for direct perception of motion in the world and movement of the self.
Proceedings Article
Mask R-CNN
TL;DR: This work presents a conceptually simple, flexible, and general framework for object instance segmentation that outperforms all existing, single-model entries on every task, including the COCO 2016 challenge winners.
Posted Content
PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space
TL;DR: A hierarchical neural network that applies PointNet recursively on a nested partitioning of the input point set and proposes novel set learning layers to adaptively combine features from multiple scales to learn deep point set features efficiently and robustly.
Proceedings ArticleDOI
PartNet: A Large-Scale Benchmark for Fine-Grained and Hierarchical Part-Level 3D Object Understanding
TL;DR: PartNet as discussed by the authors is a large-scale dataset of 3D objects annotated with fine-grained, instance-level, and hierarchical 3D part information, consisting of 573,585 part instances over 26,671 3D models.
Proceedings ArticleDOI
Learning Synergies Between Pushing and Grasping with Self-Supervised Deep Reinforcement Learning
TL;DR: This work demonstrates that it is possible to discover and learn complex synergies between non-prehensile and prehensile actions from scratch through model-free deep reinforcement learning, and achieves better grasping success rates and picking efficiencies than baseline alternatives after a few hours of training.