Human-robot collaborative manipulation planning using early prediction of human motion
01 Nov 2013-pp 299-306
TL;DR: The results show that the proposed framework enables the robot to avoid the human while still accomplishing the robot's task, even in cases where the initial prediction of the human's motion is incorrect.
Abstract: In this paper we present a framework that allows a human and a robot to perform simultaneous manipulation tasks safely in close proximity. The proposed framework is based on early prediction of the human's motion. The prediction system, which builds on previous work in the area of gesture recognition, generates a prediction of human workspace occupancy by computing the swept volume of learned human motion trajectories. The motion planner then plans robot trajectories that minimize a penetration cost in the human workspace occupancy while interleaving planning and execution. Multiple plans are computed in parallel, one for each robot task available at the current time, and the trajectory with the least cost is selected for execution. We test our framework in simulation using recorded human motions and a simulated PR2 robot. Our results show that our framework enables the robot to avoid the human while still accomplishing the robot's task, even in cases where the initial prediction of the human's motion is incorrect. We also show that taking into account the predicted human workspace occupancy in the robot's motion planner leads to safer and more efficient interactions between the user and the robot than only considering the human's current configuration.
Citations
More filters
TL;DR: This survey paper review, extend, compare, and evaluate experimentally model-based algorithms for real-time collision detection, isolation, and identification that use only proprioceptive sensors that cover the context-independent phases of the collision event pipeline for robots interacting with the environment.
Abstract: Robot assistants and professional coworkers are becoming a commodity in domestic and industrial settings. In order to enable robots to share their workspace with humans and physically interact with them, fast and reliable handling of possible collisions on the entire robot structure is needed, along with control strategies for safe robot reaction. The primary motivation is the prevention or limitation of possible human injury due to physical contacts. In this survey paper, based on our early work on the subject, we review, extend, compare, and evaluate experimentally model-based algorithms for real-time collision detection, isolation, and identification that use only proprioceptive sensors. This covers the context-independent phases of the collision event pipeline for robots interacting with the environment, as in physical human–robot interaction or manipulation tasks. The problem is addressed for rigid robots first and then extended to the presence of joint/transmission flexibility. The basic physically motivated solution has already been applied to numerous robotic systems worldwide, ranging from manipulators and humanoids to flying robots, and even to commercial products.
467 citations
Cites background from "Human-robot collaborative manipulat..."
...Other approaches like [11], [21]–[23] aim for effectively planning collision-free robot motions also incorporating the prediction of human behavior....
[...]
...Offline motion planning techniques are computationally expensive, and their efficient conversion to online methods capable of handling instantaneous changes is still on-going research [8], [9], in particular when considering human-aware situations [10], [11]....
[...]
Book•
22 May 2017TL;DR: A Survey of Methods for Safe Human-Robot Interaction organizes and summarizes the large body of research related to facilitation of safe human-robot interaction and organizes them into subcategories, characterizes relationships between the strategies, and identifies potential gaps in the existing knowledge that warrant further research.
Abstract: Ensuring human safety is one of the most important considerations within the field of human-robot interaction (HRI). This does not simply involve preventing collisions between humans and robots operating within a shared space; we must consider all possible ways in which harm could come to a person, ranging from physical contact to adverse psychological effects resulting from unpleasant or dangerous interaction. A Survey of Methods for Safe Human-Robot Interaction organizes and summarizes the large body of research related to facilitation of safe human-robot interaction. It describes the strategies and methods that have been developed thus far, organizes them into subcategories, characterizes relationships between the strategies, and identifies potential gaps in the existing knowledge that warrant further research. By creating an organized categorization of the field, A Survey of Methods for Safe Human-Robot Interaction is intended to support future research and the development of new technologies for safe HRI, as well as facilitate the use of these techniques by researchers within the HRI community.
287 citations
Cites background from "Human-robot collaborative manipulat..."
...Mainprice and Berenson (2013) developed a framework that utilizes labeled demonstrations of reaching motions to generate models for prediction of workspace occupancy....
[...]
TL;DR: An overview of symbiotic human-robot collaborative assembly is provided and future research directions for voice processing, gesture recognition, haptic interaction, and brainwave perception are highlighted.
273 citations
Cites methods from "Human-robot collaborative manipulat..."
...Mainprice and Berenson [146] categorised human actions through the use of Gaussian Mixture Models (GMMs) and Regression (GMR)....
[...]
07 Dec 2015
TL;DR: In this paper, an Autoregressive Input-Output HMM is proposed to model the contextual information along with the maneuvers, which can anticipate maneuvers 3.5 seconds before they occur with over 80% F1 score in real-time.
Abstract: Advanced Driver Assistance Systems (ADAS) have made driving safer over the last decade. They prepare vehicles for unsafe road conditions and alert drivers if they perform a dangerous maneuver. However, many accidents are unavoidable because by the time drivers are alerted, it is already too late. Anticipating maneuvers beforehand can alert drivers before they perform the maneuver and also give ADAS more time to avoid or prepare for the danger. In this work we anticipate driving maneuvers a few seconds before they occur. For this purpose we equip a car with cameras and a computing device to capture the driving context from both inside and outside of the car. We propose an Autoregressive Input-Output HMM to model the contextual information alongwith the maneuvers. We evaluate our approach on a diverse data set with 1180 miles of natural freeway and city driving and show that we can anticipate maneuvers 3.5 seconds before they occur with over 80% F1-score in real-time.
231 citations
TL;DR: A comprehensive review of EMG-based motor intention prediction of continuous human upper limb motion, which will cover the models and approaches used in continuous motion estimation, the kinematic motion parameters estimated from EMG signal, and the performance metrics utilized for system validation.
216 citations
References
More filters
Book•
01 Aug 2006
TL;DR: Looking for competent reading resources?
Abstract: Looking for competent reading resources? We have pattern recognition and machine learning information science and statistics to read, not only read, but also download them or even check out online. Locate this fantastic book writtern by by now, simply here, yeah just here. Obtain the reports in the kinds of txt, zip, kindle, word, ppt, pdf, as well as rar. Once again, never ever miss to review online and download this book in our site right here. Click the link.
8,923 citations
"Human-robot collaborative manipulat..." refers background in this paper
...The reader may refer to [24] for a more detailed explanation of this procedure....
[...]
TL;DR: In this article, the optimal data selection techniques have been used with feed-forward neural networks and showed how the same principles may be used to select data for two alternative, statistically-based learning architectures: mixtures of Gaussians and locally weighted regression.
Abstract: For many types of machine learning algorithms, one can compute the statistically "optimal" way to select training data. In this paper, we review how optimal data selection techniques have been used with feedforward neural networks. We then show how the same principles may be used to select data for two alternative, statistically-based learning architectures: mixtures of Gaussians and locally weighted regression. While the techniques for neural networks are computationally expensive and approximate, the techniques for mixtures of Gaussians and locally weighted regression are both efficient and accurate. Empirically, we observe that the optimality criterion sharply decreases the number of training examples the learner needs in order to achieve good performance.
2,122 citations
01 Apr 2007
TL;DR: A programming-by-demonstration framework for generically extracting the relevant features of a given task and for addressing the problem of generalizing the acquired knowledge to different contexts is presented.
Abstract: We present a programming-by-demonstration framework for generically extracting the relevant features of a given task and for addressing the problem of generalizing the acquired knowledge to different contexts. We validate the architecture through a series of experiments, in which a human demonstrator teaches a humanoid robot simple manipulatory tasks. A probability-based estimation of the relevance is suggested by first projecting the motion data onto a generic latent space using principal component analysis. The resulting signals are encoded using a mixture of Gaussian/Bernoulli distributions (Gaussian mixture model/Bernoulli mixture model). This provides a measure of the spatio-temporal correlations across the different modalities collected from the robot, which can be used to determine a metric of the imitation performance. The trajectories are then generalized using Gaussian mixture regression. Finally, we analytically compute the trajectory which optimizes the imitation metric and use this to generalize the skill to different contexts
1,089 citations
"Human-robot collaborative manipulat..." refers methods in this paper
...human’s motion and querying it using Gaussian Mixture Regression (GMR) is similar to [21], although, again, we extend this work to predict the workspace occupancy of the human, as well as integrating the prediction with the robot motion planning....
[...]
...More precisely, we apply the technique as established in [21], as it provides a way to reconstruct a general motion for the class....
[...]
TL;DR: A simple system of observation and notation with a view to standardizing the reporting of a narrow range of microcultural events is presented, far from perfect; but if it directs attention to certain behavior, it will have achieved its purpose.
Abstract: T HIS is one of a series of papers on Proxemics,2 the study of how man unconsciously structures microspace-the distance between men in the conduct of daily transactions, the organization of space in his houses and buildings, and ultimately the layout of his towns. The aim of this paper is to present a simple system of observation and notation with a view to standardizing the reporting of a narrow range of microcultural events. The system is far from perfect; but if it directs attention to certain behavior, it will have achieved its purpose. However, before proceeding to the descriptive portion of this paper, certain theoretical matters have to be dealt with.
878 citations
"Human-robot collaborative manipulat..." refers background in this paper
...In the closely-related topic of robot navigation in the presence of humans, early works were inspired by the social behavior demonstrated by humans [9]....
[...]
09 May 2011
TL;DR: It is experimentally show that the stochastic nature of STOMP allows it to overcome local minima that gradient-based methods like CHOMP can get stuck in.
Abstract: We present a new approach to motion planning using a stochastic trajectory optimization framework. The approach relies on generating noisy trajectories to explore the space around an initial (possibly infeasible) trajectory, which are then combined to produced an updated trajectory with lower cost. A cost function based on a combination of obstacle and smoothness cost is optimized in each iteration. No gradient information is required for the particular optimization algorithm that we use and so general costs for which derivatives may not be available (e.g. costs corresponding to constraints and motor torques) can be included in the cost function. We demonstrate the approach both in simulation and on a mobile manipulation system for unconstrained and constrained tasks. We experimentally show that the stochastic nature of STOMP allows it to overcome local minima that gradient-based methods like CHOMP can get stuck in.
817 citations
"Human-robot collaborative manipulat..." refers background or methods in this paper
...STOMP [23] is a trajectory optimizer that iteratively deforms an initial solution by estimating stochastically the gradient in trajectory space....
[...]
...We compute robot motions that minimize this cost along the robot trajectory using STOMP [23]....
[...]
...The initial algorithm presented in [23] optimizes a combination of two classical criteria, namely obstacle cost and smoothness cost....
[...]
...Our framework uses the STOMP algorithm, which has proven effective for the type of manipulator motion planning we consider [23]....
[...]
...At replanning step n, the robot executes the motion planned in Algorithm 1: Multiple goal planning with prediction input : Workspace W K goal configurations goals begin t← 0 trajs← initStraightLines(goals) while taskCompletion() do ξ ← updateHumanMotion() X ← predictVoxelOccupancy(ξ) for g ∈ goals do τ ← reconnectPrevious(trajs[g], τbest, t) trajs[g]← STOMP(τ,X,W) τbest ← getBestTrajectory(trajs) t← execute(τbest) end step n − 1, it also records the current human motion and predicts the human workspace occupancy....
[...]