To reduce data collection time for deep learning of robust robotic grasp plans, we explore training from a synthetic dataset of 6.7 million point clouds, grasps, and analytic grasp metrics generated from thousands of 3D models from Dex-Net 1.0 in randomized poses on a table. We use the resulting dataset, Dex-Net 2.0, to train a Grasp Quality Convolutional Neural Network (GQ-CNN) model that rapidly predicts the probability of success of grasps from depth images, where grasps are specified as the planar position, angle, and depth of a gripper relative to an RGB-D sensor. Experiments with over 1,000 trials on an ABB YuMi comparing grasp planning methods on singulated objects suggest that a GQ-CNN trained with only synthetic data from Dex-Net 2.0 can be used to plan grasps in 0.8sec with a success rate of 93% on eight known objects with adversarial geometry and is 3x faster than registering point clouds to a precomputed dataset of objects and indexing grasps. The Dex-Net 2.0 grasp planner also has the highest success rate on a dataset of 10 novel rigid objects and achieves 99% precision (one false positive out of 69 grasps classified as robust) on a dataset of 40 novel household objects, some of which are articulated or deformable. Code, datasets, videos, and supplementary material are available at http://berkeleyautomation.github.io/dex-net .

http://www.roboticsproceedings.org/rss13/p58.pdf

Dex-Net 2.0: Deep Learning to Plan Robust Grasps with Synthetic Point Clouds and Analytic Grasp Metrics

To reduce data collection time for deep learning of robust robotic grasp plans, we explore training from a synthetic dataset of 6.7 million point clouds, grasps, and analytic grasp metrics generated from thousands of 3D models from Dex-Net 1.0 in randomized poses on a table. We use the resulting dataset, Dex-Net 2.0, to train a Grasp Quality Convolutional Neural Network (GQ-CNN) model that rapidly predicts the probability of success of grasps from depth images, where grasps are specified as the planar position, angle, and depth of a gripper relative to an RGB-D sensor. Experiments with over 1,000 trials on an ABB YuMi comparing grasp planning methods on singulated objects suggest that a GQ-CNN trained with only synthetic data from Dex-Net 2.0 can be used to plan grasps in 0.8sec with a success rate of 93% on eight known objects with adversarial geometry and is 3x faster than registering point clouds to a precomputed dataset of objects and indexing grasps. The Dex-Net 2.0 grasp planner also has the highest success rate on a dataset of 10 novel rigid objects and achieves 99% precision (one false positive out of 69 grasps classified as robust) on a dataset of 40 novel household objects, some of which are articulated or deformable. Code, datasets, videos, and supplementary material are available at this http URL .

Our goal is to train a policy for autonomous driving via imitation learning that is robust enough to drive a real vehicle. We find that standard behavior cloning is insufficient for handling complex driving scenarios, even when we leverage a perception system for preprocessing the input and a controller for executing the output on the car: 30 million examples are still not enough. We propose exposing the learner to synthesized data in the form of perturbations to the expert's driving, which creates interesting situations such as collisions and/or going off the road. Rather than purely imitating all data, we augment the imitation loss with additional losses that penalize undesirable events and encourage progress -- the perturbations then provide an important signal for these losses and lead to robustness of the learned model. We show that the ChauffeurNet model can handle complex situations in simulation, and present ablation experiments that emphasize the importance of each of our proposed changes and show that the model is responding to the appropriate causal factors. Finally, we demonstrate the model driving a car in the real world.

ChauffeurNet: Learning to Drive by Imitating the Best and Synthesizing the Worst

Objective This review evaluates the effectiveness of quality improvement (QI) strategies in reducing disparities in health and health care. Data sources We identified papers published in English between 1983 and 2011 from the MEDLINE® database, the Cumulative Index of Nursing and Allied Health Literature (CINAHL), Web of Science Social Science Index, and PsycINFO. Review methods All abstracts and full-text articles were dually reviewed. Studies were eligible if they reported data on effectiveness of QI interventions on processes or health outcomes in the United States such that the impact on a health disparity could be measured. The review focused on the following clinical conditions: breast cancer, colorectal cancer, diabetes, heart failure, hypertension, coronary artery disease, asthma, major depressive disorder, cystic fibrosis, pneumonia, pregnancy, and end-stage renal disease. It assessed health disparities associated with race or ethnicity, socioeconomic status, insurance status, sexual orientation, health literacy/numeracy, and language barrier. We evaluated the risk of bias of individual studies and the overall strength of the body of evidence based on risk of bias, consistency, directness, and precision. Results Nineteen papers, representing 14 primary research studies, met criteria for inclusion. All but one of the studies incorporated multiple components into their QI approach. Patient education was part of most interventions (12 of 14), although the specific approach differed substantially across the studies. Ten of the studies incorporated self-management; this would include, for example, teaching individuals with diabetes to check their blood sugar regularly. Most (8 of 14) included some sort of provider education, which may have focused on the clinical issue or on raising awareness about disparities affecting the target population. Studies evaluated the effect of these strategies on disparities in the prevention or treatment of breast or colorectal cancer, cardiovascular disease, depression, or diabetes. Overall, QI interventions were not shown to reduce disparities. Most studies have focused on racial or ethnic disparities, with some targeted interventions demonstrating greater effect in racial minorities--specifically, supporting individuals in tracking their blood pressure at home to reduce blood pressure and collaborative care to improve depression care. In one study, the effect of a language-concordant breast cancer screening intervention was helpful in promoting mammography in Spanish-speaking women. For some depression care outcomes, the collaborative care model was more effective in less-educated individuals than in those with more education and in women than in men. Conclusions The literature on QI interventions generally and their ability to improve health and health care is large. Whether those interventions are effective at reducing disparities remains unclear. This report should not be construed to assess the general effectiveness of QI in the health care setting; rather, QI has not been shown specifically to reduce known disparities in health care or health outcomes. In a few instances, some increased effect is seen in disadvantaged populations; these studies should be replicated and the interventions studied further as having potential to address disparities.

Closing the quality gap: revisiting the state of the science (vol. 3: quality improvement interventions to address health disparities).

One approach to Imitation Learning is Behavior Cloning, in which a robot observes a supervisor and infers a control policy. A known problem with this "off-policy" approach is that the robot's errors compound when drifting away from the supervisor's demonstrations. On-policy, techniques alleviate this by iteratively collecting corrective actions for the current robot policy. However, these techniques can be tedious for human supervisors, add significant computation burden, and may visit dangerous states during training. We propose an off-policy approach that injects noise into the supervisor's policy while demonstrating. This forces the supervisor to demonstrate how to recover from errors. We propose a new algorithm, DART (Disturbances for Augmenting Robot Trajectories), that collects demonstrations with injected noise, and optimizes the noise level to approximate the error of the robot's trained policy during data collection. We compare DART with DAgger and Behavior Cloning in two domains: in simulation with an algorithmic supervisor on the MuJoCo tasks (Walker, Humanoid, Hopper, Half-Cheetah) and in physical experiments with human supervisors training a Toyota HSR robot to perform grasping in clutter. For high dimensional tasks like Humanoid, DART can be up to $3x$ faster in computation time and only decreases the supervisor's cumulative reward by $5\%$ during training, whereas DAgger executes policies that have $80\%$ less cumulative reward than the supervisor. On the grasping in clutter task, DART obtains on average a $62\%$ performance increase over Behavior Cloning.

/pdf/dart-noise-injection-for-robust-imitation-learning-4tn99m9es3.pdf

DART: Noise Injection for Robust Imitation Learning

For applications such as Amazon warehouse order fulfillment, robots must grasp a desired object amid clutter: other objects that block direct access. This can be difficult to program explicitly due to uncertainty in friction and push mechanics and the variety of objects that can be encountered. Deep Learning networks combined with Online Learning from Demonstration (LfD) algorithms such as DAgger and SHIV have potential to learn robot control policies for such tasks where the input is a camera image and system dynamics and the cost function are unknown. To explore this idea, we introduce a version of the grasping in clutter problem where a yellow cylinder must be grasped by a planar robot arm amid extruded objects in a variety of shapes and positions. To reduce the burden on human experts to provide demonstrations, we propose using a hierarchy of three levels of supervisors: a fast motion planner that ignores obstacles, crowd-sourced human workers who provide appropriate robot control values remotely via online videos, and a local human expert. Physical experiments suggest that with 160 expert demonstrations, using the hierarchy of supervisors can increase the probability of a successful grasp (reliability) from 55% to 90%.

Robot grasping in clutter: Using a hierarchy of supervisors for learning from demonstrations

Motivated by recent advances in Deep Learning for robot control, this paper considers two learning algorithms in terms of how they acquire demonstrations from fallible human supervisors. Human-Centric (HC) sampling is a standard supervised learning algorithm, where a human supervisor demonstrates the task by teleoperating the robot to provide trajectories consisting of state-control pairs. Robot-Centric (RC) sampling is an increasingly popular alternative used in algorithms such as DAgger, where a human supervisor observes the robot execute a learned policy and provides corrective control labels for each state visited. We suggest RC sampling can be challenging for human supervisors and prone to mislabeling. RC sampling can also induce error in policy performance because it repeatedly visits areas of the state space that are harder to learn. Although policies learned with RC sampling can be superior to HC sampling for standard learning models such as linear SVMs, policies learned with HC sampling may be comparable to RC when applied to expressive learning models such as deep learning and hyper-parametric decision trees, which can achieve very low training error provided there is enough data. We compare HC and RC using a grid world environment and a physical robot singulation task. In the latter the input is a binary image of objects on a planar worksurface and the policy generates a motion in the gripper to separate one object from the rest. We observe in simulation that for linear SVMs, policies learned with RC outperformed those learned with HC but that using deep models this advantage disappears. We also find that with RC, the corrective control labels provided by humans can be highly inconsistent. We prove there exists a class of examples in which at the limit, HC is guaranteed to converge to an optimal policy while RC may fail to converge. These results suggest a form of HC sampling may be preferable for highly-expressive learning models and human supervisors.

Comparing human-centric and robot-centric sampling for robot deep learning from demonstrations

Background: The Stanford University Chronic Disease Self-Management Program (CDSMP) has resulted in moderate beneficial outcomes in randomized controlled trials. A study of a modified CDSMP in England suggested that younger participants, those with lower initial self-efficacy and those with greater depression benefited most from the program.Design: Using data from previous CDSMP programs in English and Spanish, we examined whether there were statistically significant interactions between baseline statuses (demographic, disease and health status variables) and randomization (intervention or usual-care control) in estimating 6-month changes in health status (health distress, activity limitation/role function, self-efficacy and self-reported general health). If an interaction was found, post hoc examinations of the relationships between the baseline variables and outcomes determined the directions of the relationships.Results: Six moderating variables were found in the original English-language CDSMP and thr...

Jonathan Lee

Papers

DART: Noise Injection for Robust Imitation Learning

Robot grasping in clutter: Using a hierarchy of supervisors for learning from demonstrations

DART: Noise Injection for Robust Imitation Learning

Comparing human-centric and robot-centric sampling for robot deep learning from demonstrations

Moderators of chronic disease self-management programs: who benefits?: