Optimal Robot Motion Planning in Constrained Workspaces Using Reinforcement Learning
24 Oct 2020-pp 6917-6922
TL;DR: In this article, Reinforcement Learning (RL) is applied to adjust appropriately the parameters of the underlying potential field towards minimizing the Hamilton-Jacobi-Bellman (HJB) error.
Abstract: In this work, a novel solution to the optimal motion planning problem is proposed, through a continuous, deterministic and provably correct approach, with guaranteed safety and which is based on a parametrized Artificial Potential Field (APF). In particular, Reinforcement Learning (RL) is applied to adjust appropriately the parameters of the underlying potential field towards minimizing the Hamilton-Jacobi-Bellman (HJB) error. The proposed method, outperforms consistently a Rapidly-exploring Random Trees (RRT*) method and consists a fertile advancement in the optimal motion planning problem.
Citations
More filters
19 Feb 2021
TL;DR: In this paper, a reinforcement learning algorithm is proposed to solve the optimal motion planning problem, which inherits the strong traits from both artificial potential fields, i.e., reactivity, as well as sampling-based methods, and opens up new paths to the age-old problem of motion planning.
Abstract: In this work, we propose a novel reinforcement learning algorithm to solve the optimal motion planning problem. Particular emphasis is given on the rigorous mathematical proof of safety, convergence as well as optimality w.r.t. to an integral quadratic cost function, while reinforcement learning is adopted to enable the cost function's approximation. Both offline and online solutions are proposed, and an implementation of the offline method is compared to a state-of-the-art RRT $^{\star }$ approach. This novel approach inherits the strong traits from both artificial potential fields, i.e., reactivity, as well as sampling-based methods, i.e., optimality, and opens up new paths to the age-old problem of motion planning, by merging modern tools and philosophies from various corners of the field.
14 citations
TL;DR: A novel reactive method for robot trajectory planning within unknown 2D workspaces is presented, which provides a velocity profile for each position of the robot within the workspace, its smoothness, and its safety and convergence guarantees.
Abstract: A novel reactive method for robot trajectory planning within unknown 2D workspaces is presented in this paper. The trajectories provided by this method stem from an underlying potential field and are provably safe, with asymptotic convergence to the desired position. Given an initially unknown workspace, and a sensing process for obtaining boundary information, the scope of this work is to provide an underlying vector field that can be used as a reference signal for higher order, non-linear dynamical systems (robots). This can be achieved by tracking, either the field itself, or the trajectories that result from the latter. In this spirit, our method is advantageous due to: a) its reactivity, as open-loop approaches may exhibit large errors during the transient phase within unknown workspaces, whereas our method provides a velocity profile for each position of the robot within the workspace, b) its smoothness, and c) its safety and convergence guarantees. We prove the asserted claims and provide rigorous comparative simulations in various benchmark workspaces, red as well as in a high-fidelity ROS-Gazebo implementation.
4 citations
TL;DR: This method aims at bridging the gap between the control theoretic framework of mathematical rigor, with the data-driven Reinforcement Learning (RL) paradigm, while preserving the strong traits of each approach.
Abstract: A novel motion planning scheme for optimal navigation in unknown workspaces is proposed in this letter. Based upon the Artificial Harmonic Potential Fields (AHPFs) theory, a robust framework for provably correct (i.e., safe and globally convergent) navigation is enhanced through Integral Reinforcement Learning (IRL)1 to obtain a provably complete solution for optimal motion planning in unknown workspaces. Our method aims at bridging the gap between the control theoretic framework of mathematical rigor, with the data-driven Reinforcement Learning (RL) paradigm, while preserving the strong traits of each approach. Finally, it is compared against an RRT$^\star$
3 citations
TL;DR: In this article , a survey of path planning and obstacle avoidance methods for mobile robots is presented, including graph-based search, heuristic intelligence, local obstacle avoidance, artificial intelligence, sampling-based, planner-based and constraint problem satisfaction-based algorithms.
Abstract: Mobile robots, including ground robots, underwater robots, and unmanned aerial vehicles, play an increasingly important role in people’s work and lives. Path planning and obstacle avoidance are the core technologies for achieving autonomy in mobile robots, and they will determine the application prospects of mobile robots. This paper introduces path planning and obstacle avoidance methods for mobile robots to provide a reference for researchers in this field. In addition, it comprehensively summarizes the recent progress and breakthroughs of mobile robots in the field of path planning and discusses future directions worthy of research in this field. We focus on the path planning algorithm of a mobile robot. We divide the path planning methods of mobile robots into the following categories: graph-based search, heuristic intelligence, local obstacle avoidance, artificial intelligence, sampling-based, planner-based, constraint problem satisfaction-based, and other algorithms. In addition, we review a path planning algorithm for multi-robot systems and different robots. We describe the basic principles of each method and highlight the most relevant studies. We also provide an in-depth discussion and comparison of path planning algorithms. Finally, we propose potential research directions in this field that are worth studying in the future.
3 citations
TL;DR: In this article , a reactive method for robot trajectory planning within unknown 2D workspaces is presented, where trajectories provided by this method stem from an underlying potential field and are provably safe, with asymptotic convergence to the desired position.
Abstract: A novel reactive method for robot trajectory planning within unknown 2D workspaces is presented in this letter. The trajectories provided by this method stem from an underlying potential field and are provably safe, with asymptotic convergence to the desired position. Given an initially unknown workspace, and a sensing process for obtaining boundary information, the scope of this work is to provide an underlying vector field that can be used as a reference signal for higher order, non-linear dynamical systems (robots). This can be achieved by tracking, either the field itself, or the trajectories that result from the latter. In this spirit, our method is advantageous due to: a) its reactivity, as open-loop approaches may exhibit large errors during the transient phase within unknown workspaces, whereas our method provides a velocity profile for each position of the robot within the workspace, b) its smoothness, and c) its safety and convergence guarantees. We prove the asserted claims and provide rigorous comparative simulations in various benchmark workspaces, as well as in a high-fidelity ROS-Gazebo implementation.
2 citations
References
More filters
01 Aug 1996
TL;DR: Experimental results show that path planning can be done in a fraction of a second on a contemporary workstation (/spl ap/150 MIPS), after learning for relatively short periods of time (a few dozen seconds).
Abstract: A new motion planning method for robots in static workspaces is presented. This method proceeds in two phases: a learning phase and a query phase. In the learning phase, a probabilistic roadmap is constructed and stored as a graph whose nodes correspond to collision-free configurations and whose edges correspond to feasible paths between these configurations. These paths are computed using a simple and fast local planner. In the query phase, any given start and goal configurations of the robot are connected to two nodes of the roadmap; the roadmap is then searched for a path joining these two nodes. The method is general and easy to implement. It can be applied to virtually any type of holonomic robot. It requires selecting certain parameters (e.g., the duration of the learning phase) whose values depend on the scene, that is the robot and its workspace. But these values turn out to be relatively easy to choose, Increased efficiency can also be achieved by tailoring some components of the method (e.g., the local planner) to the considered robots. In this paper the method is applied to planar articulated robots with many degrees of freedom. Experimental results show that path planning can be done in a fraction of a second on a contemporary workstation (/spl ap/150 MIPS), after learning for relatively short periods of time (a few dozen seconds).
4,977 citations
TL;DR: In this paper, the authors studied the asymptotic behavior of the cost of the solution returned by stochastic sampling-based path planning algorithms as the number of samples increases.
Abstract: During the last decade, sampling-based path planning algorithms, such as probabilistic roadmaps (PRM) and rapidly exploring random trees (RRT), have been shown to work well in practice and possess theoretical guarantees such as probabilistic completeness. However, little effort has been devoted to the formal analysis of the quality of the solution returned by such algorithms, e.g. as a function of the number of samples. The purpose of this paper is to fill this gap, by rigorously analyzing the asymptotic behavior of the cost of the solution returned by stochastic sampling-based algorithms as the number of samples increases. A number of negative results are provided, characterizing existing algorithms, e.g. showing that, under mild technical conditions, the cost of the solution returned by broadly used sampling-based algorithms converges almost surely to a non-optimal value. The main contribution of the paper is the introduction of new algorithms, namely, PRM* and RRT*, which are provably asymptotically optimal, i.e. such that the cost of the returned solution converges almost surely to the optimum. Moreover, it is shown that the computational complexity of the new algorithms is within a constant factor of that of their probabilistically complete (but not asymptotically optimal) counterparts. The analysis in this paper hinges on novel connections between stochastic sampling-based path planning algorithms and the theory of random geometric graphs.
3,438 citations
Book•
29 Jun 1988TL;DR: John Canny resolves long-standing problems concerning the complexity of motion planning and, for the central problem of finding a collision free path for a jointed robot in the presence of obstacles, obtains exponential speedups over existing algorithms by applying high-powered new mathematical techniques.
Abstract: The Complexity of Robot Motion Planning makes original contributions both to robotics and to the analysis of algorithms. In this groundbreaking monograph John Canny resolves long-standing problems concerning the complexity of motion planning and, for the central problem of finding a collision free path for a jointed robot in the presence of obstacles, obtains exponential speedups over existing algorithms by applying high-powered new mathematical techniques.Canny's new algorithm for this "generalized movers' problem," the most-studied and basic robot motion planning problem, has a single exponential running time, and is polynomial for any given robot. The algorithm has an optimal running time exponent and is based on the notion of roadmaps - one-dimensional subsets of the robot's configuration space. In deriving the single exponential bound, Canny introduces and reveals the power of two tools that have not been previously used in geometric algorithms: the generalized (multivariable) resultant for a system of polynomials and Whitney's notion of stratified sets. He has also developed a novel representation of object orientation based on unnormalized quaternions which reduces the complexity of the algorithms and enhances their practical applicability.After dealing with the movers' problem, the book next attacks and derives several lower bounds on extensions of the problem: finding the shortest path among polyhedral obstacles, planning with velocity limits, and compliant motion planning with uncertainty. It introduces a clever technique, "path encoding," that allows a proof of NP-hardness for the first two problems and then shows that the general form of compliant motion planning, a problem that is the focus of a great deal of recent work in robotics, is non-deterministic exponential time hard. Canny proves this result using a highly original construction.John Canny received his doctorate from MIT And is an assistant professor in the Computer Science Division at the University of California, Berkeley. The Complexity of Robot Motion Planning is the winner of the 1987 ACM Doctoral Dissertation Award.
1,538 citations
25 Mar 1985
TL;DR: In this article, a real-time obstacle avoidance approach for manipulators and mobile robots based on the "artificial potential field" concept is presented, where collision avoidance, traditionally considered a high level planning problem, can be effectively distributed between different levels of control.
Abstract: This paper presents a unique real-time obstacle avoidance approach for manipulators and mobile robots based on the "artificial potential field" concept. In this approach, collision avoidance, traditionally considered a high level planning problem, can be effectively distributed between different levels of control, allowing real-time robot operations in a complex environment. We have applied this obstacle avoidance scheme to robot arm using a new approach to the general problem of real-time manipulator control. We reformulated the manipulator control problem as direct control of manipulator motion in operational space-the space in which the task is originally described-rather than as control of the task's corresponding joint space motion obtained only after geometric and kinematic transformation. This method has been implemented in the COSMOS system for a PUMA 560 robot. Using visual sensing, real-time collision avoidance demonstrations on moving obstacles have been performed.
1,088 citations
TL;DR: It is shown that the constrained optimal control law has the largest region of asymptotic stability (RAS) and the result is a nearly optimal constrained state feedback controller that has been tuned a priori off-line.
1,045 citations