We use reinforcement learning (RL) to learn dexterous in-hand manipulation policies that can perform vision-based object reorientation on a physical Shadow Dexterous Hand. The training is performed...

Learning dexterous in-hand manipulation:

In this paper, we study the problem of learning vision-based dynamic manipulation skills using a scalable reinforcement learning approach. We study this problem in the context of grasping, a longstanding challenge in robotic manipulation. In contrast to static learning behaviors that choose a grasp point and then execute the desired grasp, our method enables closed-loop vision-based control, whereby the robot continuously updates its grasp strategy based on the most recent observations to optimize long-horizon grasp success. To that end, we introduce QT-Opt, a scalable self-supervised vision-based reinforcement learning framework that can leverage over 580k real-world grasp attempts to train a deep neural network Q-function with over 1.2M parameters to perform closed-loop, real-world grasping that generalizes to 96% grasp success on unseen objects. Aside from attaining a very high success rate, our method exhibits behaviors that are quite distinct from more standard grasping systems: using only RGB vision-based perception from an over-the-shoulder camera, our method automatically learns regrasping strategies, probes objects to find the most effective grasps, learns to reposition objects and perform other non-prehensile pre-grasp manipulations, and responds dynamically to disturbances and perturbations.

QT-Opt: Scalable Deep Reinforcement Learning for Vision-Based Robotic Manipulation

We demonstrate that models trained only in simulation can be used to solve a manipulation problem of unprecedented complexity on a real robot. This is made possible by two key components: a novel algorithm, which we call automatic domain randomization (ADR) and a robot platform built for machine learning. ADR automatically generates a distribution over randomized environments of ever-increasing difficulty. Control policies and vision state estimators trained with ADR exhibit vastly improved sim2real transfer. For control policies, memory-augmented models trained on an ADR-generated distribution of environments show clear signs of emergent meta-learning at test time. The combination of ADR with our custom robot platform allows us to solve a Rubik's cube with a humanoid robot hand, which involves both control and state estimation problems. Videos summarizing our results are available: this https URL

Solving Rubik's Cube with a Robot Hand.

Universal picking (UP), or reliable robot grasping of a diverse range of novel objects from heaps, is a grand challenge for e-commerce order fulfillment, manufacturing, inspection, and home service robots. Optimizing the rate, reliability, and range of UP is difficult due to inherent uncertainty in sensing, control, and contact physics. This paper explores “ambidextrous” robot grasping, where two or more heterogeneous grippers are used. We present Dexterity Network (Dex-Net) 4.0, a substantial extension to previous versions of Dex-Net that learns policies for a given set of grippers by training on synthetic datasets using domain randomization with analytic models of physics and geometry. We train policies for a parallel-jaw and a vacuum-based suction cup gripper on 5 million synthetic depth images, grasps, and rewards generated from heaps of three-dimensional objects. On a physical robot with two grippers, the Dex-Net 4.0 policy consistently clears bins of up to 25 novel objects with reliability greater than 95% at a rate of more than 300 mean picks per hour.

Learning ambidextrous robot grasping policies

This paper presents a robotic pick-and-place system that is capable of grasping and recognizing both known and novel objects in cluttered environments. The key new feature of the system is that it handles a wide range of object categories without needing any task-specific training data for novel objects. To achieve this, it first uses a category-agnostic affordance prediction algorithm to select and execute among four different grasping primitive behaviors. It then recognizes picked objects with a cross-domain image classification framework that matches observed images to product images. Since product images are readily available for a wide range of objects (e.g., from the web), the system works out-of-the-box for novel objects without requiring any additional training data. Exhaustive experimental results demonstrate that our multi-affordance grasping achieves high success rates for a wide variety of objects in clutter, and our recognition algorithm achieves high accuracy for both known and novel grasped objects. The approach was part of the MIT-Princeton Team system that took 1st place in the stowing task at the 2017 Amazon Robotics Challenge. All code, datasets, and pre-trained models are available online at http://arc.cs.princeton.edu

/pdf/robotic-pick-and-place-of-novel-objects-in-clutter-with-12ysl5pg39.pdf

Robotic Pick-and-Place of Novel Objects in Clutter with Multi-Affordance Grasping and Cross-Domain Image Matching

Vacuum-based end effectors are widely used in industry and are often preferred over parallel-jaw and multifinger grippers due to their ability to lift objects with a single point of contact. Suction grasp planners often target planar surfaces on point clouds near the estimated centroid of an object. In this paper, we propose a compliant suction contact model that computes the quality of the seal between the suction cup and local target surface and a measure of the ability of the suction grasp to resist an external gravity wrench. To characterize grasps, we estimate robustness to perturbations in end-effector and object pose, material properties, and external wrenches. We analyze grasps across 1,500 3D object models to generate Dex-Net 3.0, a dataset of 2.8 million point clouds, suction grasps, and grasp robustness labels. We use Dex-Net 3.0 to train a Grasp Quality Convolutional Neural Network (GQ-CNN) to classify robust suction targets in point clouds containing a single object. We evaluate the resulting system in 350 physical trials on an ABB YuMi fitted with a pneumatic suction gripper. When evaluated on novel objects that we categorize as Basic (prismatic or cylindrical), Typical (more complex geometry), and Adversarial (with few available suction-grasp points) Dex-Net 3.0 achieves success rates of 98%, 82%, and 58% respectively, improving to 81% in the latter case when the training set includes only adversarial objects. Code, datasets, and supplemental material can be found at http://berkeleyautomation.github.io/dex-net.

Dex-Net 3.0: Computing Robust Vacuum Suction Grasp Targets in Point Clouds Using a New Analytic Model and Deep Learning

Vacuum-based end effectors are widely used in industry and are often preferred over parallel-jaw and multifinger grippers due to their ability to lift objects with a single point of contact. Suction grasp planners often target planar surfaces on point clouds near the estimated centroid of an object. In this paper, we propose a compliant suction contact model that computes the quality of the seal between the suction cup and local target surface and a measure of the ability of the suction grasp to resist an external gravity wrench. To characterize grasps, we estimate robustness to perturbations in end-effector and object pose, material properties, and external wrenches. We analyze grasps across 1,500 3D object models to generate Dex-Net 3.0, a dataset of 2.8 million point clouds, suction grasps, and grasp robustness labels. We use Dex-Net 3.0 to train a Grasp Quality Convolutional Neural Network (GQ-CNN) to classify robust suction targets in point clouds containing a single object. We evaluate the resulting system in 350 physical trials on an ABB YuMi fitted with a pneumatic suction gripper. When evaluated on novel objects that we categorize as Basic (prismatic or cylindrical), Typical (more complex geometry), and Adversarial (with few available suction-grasp points) Dex-Net 3.0 achieves success rates of 98$\%$, 82$\%$, and 58$\%$ respectively, improving to 81$\%$ in the latter case when the training set includes only adversarial objects. Code, datasets, and supplemental material can be found at this http URL .

Dex-Net 3.0: Computing Robust Robot Suction Grasp Targets in Point Clouds using a New Analytic Model and Deep Learning

Robots built from cable-driven tensegrity (`tension-integrity') structures have many of the advantages of soft robots, such as flexibility and robustness, yet still obey simple statics and dynamics models. However, existing approaches cannot natively model tensegrity robots with arbitrary rigid bodies in their tension network. This work presents a method to calculate the cable tensions in static equilibrium for such robots, here defined as compound tensegrities. First, a static equilibrium model for compound tensegrity robots is reformulated from the standard force density method used with other tensegrity structures. Next, we pose the problem of calculating tension forces in the robot's cables under our model. A solution is proposed as a quadratic optimization problem with practical constraints. Simulations illustrate how this inverse statics optimization problem can be used for both the design and control of two different compound tensegrity applications: a spine robot and a quadruped robot built from that spine. Finally, we verify the accuracy of the inverse statics model through a hardware experiment, demonstrating the feasibility of low-error open-loop control using our methodology.

Inverse Statics Optimization for Compound Tensegrity Robots

Robots built from cable-driven tensegrity (`tension-integrity') structures have many of the advantages of soft robots, such as flexibility and robustness, while still obeying simple statics and dynamics models. However, existing tensegrity modeling approaches cannot natively describe robots with arbitrary rigid bodies in their tension network. This work presents a method to calculate the cable tensions in static equilibrium for such tensegrity robots, here defined as compound tensegrity. First, a static equilibrium model for compound tensegrity robots is reformulated from the standard force density method used with other tensegrity structures. Next, we pose the problem of calculating tension forces in the robot's cables under our proposed model. A solution is proposed as a quadratic optimization problem with practical constraints. Simulations illustrate how this inverse statics optimization problem can be used for both the design and control of two different compound tensegrity applications: a spine robot and a quadruped robot built from that spine. Finally, we verify the accuracy of the inverse statics model through a hardware experiment, demonstrating the feasibility of low-error open-loop control using our proposed methodology.

This paper describes the design of a lightweight wing rib structure by combining topology optimisation with additive manufacturing. In addition, a deep feed-forward neural network model is proposed to perform the load prediction for the constructed wing structure incorporated with this optimised rib. The strain energy of the front rib has a minimum strain energy 1330 J for the initial volume state when the topology is not optimised for lightweighting. The relative error of the load prediction values obtained by the output layer of the deep feed-forward neural network is less than 0.02%. The absolute error of the small load prediction was less than 0.30 N. The presented results demonstrate the viability of additive manufactured rib and its implementation in global wing load prediction model for faster designs.

Albert H. Li

Papers

Dex-Net 3.0: Computing Robust Vacuum Suction Grasp Targets in Point Clouds Using a New Analytic Model and Deep Learning

Dex-Net 3.0: Computing Robust Robot Suction Grasp Targets in Point Clouds using a New Analytic Model and Deep Learning

Inverse Statics Optimization for Compound Tensegrity Robots

Inverse Statics Optimization for Compound Tensegrity Robots

Topology Optimization of Wing Ribs for Additive Manufacturing