RRT-HX: RRT With Heuristic Extend Operations for Motion Planning in Robotic Systems
21 Aug 2016-
TL;DR: This paper presents a sampling-based method for path planning in robotic systems without known cost-to-go information that uses trajectories generated from random search to heuristically learn the cost- to-go of regions within the configuration space.
Abstract: This paper presents a sampling-based method for path planning in robotic systems without known cost-to-go information. It uses trajectories generated from random search to heuristically learn the cost-to-go of regions within the configuration space. Gradually, the search is increasingly directed towards lower cost regions of the configuration space, thereby producing paths that converge towards the optimal path. The proposed framework builds on Rapidly-exploring Random Trees for random sampling-based search and Reinforcement Learning is used as the learning method. A series of experiments were performed to evaluate and demonstrate the performance of the proposed method.
Citations
More filters
TL;DR: A novel sampling-based algorithm is proposed which is able to plan in an unknown environment and provides solutions with lower cost in terms of path length, runtime and stability of the results.
Abstract: Despite the proven advantages of sampling-based motion planning algorithms, their inability to handle online navigation tasks and providing low-cost solutions make them less efficient in practice. In this paper, a novel sampling-based algorithm is proposed which is able to plan in an unknown environment and provides solutions with lower cost in terms of path length, runtime and stability of the results. First, a fuzzy controller is designed which incorporates the heuristic rules of Tabu search to enable the planner for solving online navigation tasks. Then, an adaptive neuro-fuzzy inference system (ANFIS) is proposed such that it constructs and optimizes the fuzzy controller based on a set of given input/output data. Furthermore, a heuristic dataset generator is implemented to provide enough data for the ANFIS using a randomized procedure. The performance of the proposed algorithm is evaluated through simulation in different motion planning queries. Finally, the proposed planner is compared to some of the similar motion planning algorithms to support the claim of superiority of its performance.
15 citations
Cites methods from "RRT-HX: RRT With Heuristic Extend O..."
...A sampling-based method has been introduced for improving the solution cost in singlequery planners [53]....
[...]
01 Nov 2018
TL;DR: The Experience-Based-Heuristic-Search algorithm is proposed, which overcomes the statistical failure rate of a Deep-reinforcement-Iearning-based planner and still benefits computationally from the pre-learned optimal policy.
Abstract: Interaction-aware planning for autonomous driving requires an exploration of a combinatorial solution space when using conventional search- or optimization-based motion planners. With Deep Reinforcement Learning, optimal driving strategies for such problems can be derived also for higher-dimensional problems. However, these methods guarantee optimality of the resulting policy only in a statistical sense, which impedes their usage in safety critical systems, such as autonomous vehicles. Thus, we propose the Experience-Based-Heuristic-Search algorithm, which overcomes the statistical failure rate of a Deep-reinforcement-Iearning-based planner and still benefits computationally from the pre-learned optimal policy. Specifically, we show how experiences in the form of a Deep Q-Network can be integrated as heuristic into a heuristic search algorithm. We benchmark our algorithm in the field of path planning in semi-structured valet parking scenarios. There, we analyze the accuracy of such estimates and demonstrate the computational advantages and robustness of our method. Our method may encourage further investigation of the applicability of reinforcement-learning-based planning in the field of self-driving vehicles.
11 citations
Cites methods from "RRT-HX: RRT With Heuristic Extend O..."
...[12] use value iteration to iteratively create a quality grid map during planning, which guides the node expansion of a RRT planner....
[...]
TL;DR: In this article, the authors proposed the Experience-Based-Heuristic-Search algorithm, which overcomes the statistical failure rate of a deep-reinforcement learning-based planner and still benefits computationally from the pre-learned optimal policy.
Abstract: Interaction-aware planning for autonomous driving requires an exploration of a combinatorial solution space when using conventional search- or optimization-based motion planners. With Deep Reinforcement Learning, optimal driving strategies for such problems can be derived also for higher-dimensional problems. However, these methods guarantee optimality of the resulting policy only in a statistical sense, which impedes their usage in safety critical systems, such as autonomous vehicles. Thus, we propose the Experience-Based-Heuristic-Search algorithm, which overcomes the statistical failure rate of a Deep-reinforcement-learning-based planner and still benefits computationally from the pre-learned optimal policy. Specifically, we show how experiences in the form of a Deep Q-Network can be integrated as heuristic into a heuristic search algorithm. We benchmark our algorithm in the field of path planning in semi-structured valet parking scenarios. There, we analyze the accuracy of such estimates and demonstrate the computational advantages and robustness of our method. Our method may encourage further investigation of the applicability of reinforcement-learning-based planning in the field of self-driving vehicles.
5 citations
24 Sep 2021
TL;DR: In this paper, an improved central circle sampling bidirectional RRT (CC-BRRT) algorithm is proposed, which searches for the next sampling point by a Central Circle Sampling strategy to reduce the number of searching nodes, and reduces the randomness by a target biasing strategy to speed up the convergence of the algorithm.
Abstract: Path planning through a bidirectional fast extended random tree algorithm cannot converge quickly, which does not meet the requirements of mobile robot path planning. To address this problem, an improved, central circle sampling bidirectional RRT (CC-BRRT) algorithm is proposed in this paper. The algorithm searches for the next sampling point by a central circle sampling strategy to reduce the numbers of searching nodes, and reduces the randomness by a target biasing strategy to speed up the convergence of the algorithm. For the obtained path, a sextic spline interpolation method is used to generate a smooth and executable path. Finally, experiments on mobile robot path planning are conducted both in simple with fewer obstacles and complex with more obstacles scenarios. The results show that the proposed CC-BRRT algorithm is superior to several other algorithms, with substantially fewer nodes sampled and a good smoothness and feasibility of the planned path.
2 citations
18 Aug 2018
TL;DR: The proposed system evaluates a RRT algorithm based on the individual cost of nodes and the optimized reconnection of the final path based on Dijkstra and Floyd criteria to achieve a definitive algorithm in mobile robotics.
Abstract: In this article, we present the application of Graph Theory in the development of an algorithm of path planning for mobile robots. The proposed system evaluates a RRT algorithm based on the individual cost of nodes and the optimized reconnection of the final path based on Dijkstra and Floyd criteria. Our proposal includes the comparisons between different RRT* algorithms and the simulation of the environments in different platforms. The results identify that these criteria must be considered in all the variations of RRT to achieve a definitive algorithm in mobile robotics.
1 citations
References
More filters
Book•
01 Jan 1988TL;DR: This book provides a clear and simple account of the key ideas and algorithms of reinforcement learning, which ranges from the history of the field's intellectual foundations to the most recent developments and applications.
Abstract: Reinforcement learning, one of the most active research areas in artificial intelligence, is a computational approach to learning whereby an agent tries to maximize the total amount of reward it receives when interacting with a complex, uncertain environment. In Reinforcement Learning, Richard Sutton and Andrew Barto provide a clear and simple account of the key ideas and algorithms of reinforcement learning. Their discussion ranges from the history of the field's intellectual foundations to the most recent developments and applications. The only necessary mathematical background is familiarity with elementary concepts of probability. The book is divided into three parts. Part I defines the reinforcement learning problem in terms of Markov decision processes. Part II provides basic solution methods: dynamic programming, Monte Carlo methods, and temporal-difference learning. Part III presents a unified view of the solution methods and incorporates artificial neural networks, eligibility traces, and planning; the two final chapters present case studies and consider the future of reinforcement learning.
37,989 citations
"RRT-HX: RRT With Heuristic Extend O..." refers methods in this paper
...Reinforcement Learning is a learning method that uses interaction with the system to learn its characteristics [12]....
[...]
TL;DR: How heuristic information from the problem domain can be incorporated into a formal mathematical theory of graph searching is described and an optimality property of a class of search strategies is demonstrated.
Abstract: Although the problem of determining the minimum cost path through a graph arises naturally in a number of interesting applications, there has been no underlying theory to guide the development of efficient search procedures. Moreover, there is no adequate conceptual framework within which the various ad hoc search strategies proposed to date can be compared. This paper describes how heuristic information from the problem domain can be incorporated into a formal mathematical theory of graph searching and demonstrates an optimality property of a class of search strategies.
10,366 citations
TL;DR: In this paper, the authors presented the first randomized approach to kinodynamic planning (also known as trajectory planning or trajectory design), where the task is to determine control inputs to drive a robot from an unknown position to an unknown target.
Abstract: This paper presents the first randomized approach to kinodynamic planning (also known as trajectory planning or trajectory design). The task is to determine control inputs to drive a robot from an ...
2,993 citations
"RRT-HX: RRT With Heuristic Extend O..." refers methods in this paper
...methods such as Rapidly-expanding Random Trees (RRT) have been proven to be effective in path planning for high-dimensional systems even with the presence of obstacles [2]....
[...]
...Difficulty in linearization also prevented use of popular methods like RRTLQR [6] and Kinodynamic RRT* [2]....
[...]
Posted Content•
TL;DR: The main contribution of the paper is the introduction of new algorithms, namely, PRM and RRT*, which are provably asymptotically optimal, i.e. such that the cost of the returned solution converges almost surely to the optimum.
Abstract: During the last decade, sampling-based path planning algorithms, such as Probabilistic RoadMaps (PRM) and Rapidly-exploring Random Trees (RRT), have been shown to work well in practice and possess theoretical guarantees such as probabilistic completeness. However, little effort has been devoted to the formal analysis of the quality of the solution returned by such algorithms, e.g., as a function of the number of samples. The purpose of this paper is to fill this gap, by rigorously analyzing the asymptotic behavior of the cost of the solution returned by stochastic sampling-based algorithms as the number of samples increases. A number of negative results are provided, characterizing existing algorithms, e.g., showing that, under mild technical conditions, the cost of the solution returned by broadly used sampling-based algorithms converges almost surely to a non-optimal value. The main contribution of the paper is the introduction of new algorithms, namely, PRM* and RRT*, which are provably asymptotically optimal, i.e., such that the cost of the returned solution converges almost surely to the optimum. Moreover, it is shown that the computational complexity of the new algorithms is within a constant factor of that of their probabilistically complete (but not asymptotically optimal) counterparts. The analysis in this paper hinges on novel connections between stochastic sampling-based path planning algorithms and the theory of random geometric graphs.
2,210 citations
"RRT-HX: RRT With Heuristic Extend O..." refers methods in this paper
...For similar reasons, route planning in graph based methods such as PRM [8] and BIT* are also not a viable option for our system....
[...]
TL;DR: In this article, the authors considered the swing-up control problem of a two-degree-of-freedom planar robot with a single actuator and gave conditions under which the response of either degree of freedom may be globally decoupled from the response on the other and linearized.
Abstract: Underactuated mechanical systems are those possessing fewer actuators than degrees of freedom. Examples of such systems abound, including flexible joint and flexible link robots, space robots, mobile robots, and robot models that include actuator dynamics and rigid body dynamics together. Complex internal dynamics, nonholonomic behavior, and lack of feedback linearizability are often exhibited by such systems, making the class a rich one from a control standpoint. In this article the author studies a particular underactuated system known as the Acrobot: a two-degree-of-freedom planar robot with a single actuator. The author considers the so-called swing up control problem using the method of partial feedback linearization. The author gives conditions under which the response of either degree of freedom may be globally decoupled from the response of the other and linearized. This result can be used as a starting point to design swing up control algorithms. Analysis of the resulting zero dynamics as well as analysis of the energy of the system provides an understanding of the swing up algorithms. Simulation results are presented showing the swing up motion resulting from partial feedback linearization designs. >
978 citations
"RRT-HX: RRT With Heuristic Extend O..." refers methods in this paper
...The aim of the control is to achieve balance in the inverted position using application of torque in the actuated joint [13]....
[...]