Many methods for planning under uncertainty operate in the belief space, i.e., the set of probability distributions over states. Although the problem is computationally hard, recent advances have shown that belief-space planning is becoming practical for many medium size problems. Some of the most successful methods utilize sampling and often rely on distances between beliefs to partially guide the search process. This paper deals with the question of what is a suitable distance function for belief space planning, which despite its importance remains unanswered. This work indicates that the rarely used Wasserstein distance (also known as Earth Mover’s Distance (\(\mathtt{EMD} \))) is a more suitable metric than the commonly used \(\mathtt{L1} \) and Kullback–Leibler (\(\mathtt{KL} \)) for belief-space planning. Simulation results on Non-Observable Markov Decision Problems, i.e., the simplest class of belief-space planning, indicate that as the problem becomes more complex, the differences on the effectiveness of different distance functions become quite prominent. In fact, in state spaces with more than 4 dimensions, by just replacing \(\mathtt{L1} \) or \(\mathtt{KL} \) distance with \(\mathtt{EMD} \), the problems become from virtually unsolvable to solvable within a reasonable time frame. Furthermore, preliminary results on Partially Observable Markov Decision Processes indicate that point-based solvers with \(\mathtt{EMD} \) use a smaller number of samples to generate policies with similar qualities, compared to those with \(\mathtt{L1} \) and \(\mathtt{KL} \). This paper also shows that \(\mathtt{EMD} \) carries the Lipschitz continuity of the state’s cost function to Lipschitz continuity of the expected cost of the beliefs. Such a continuity property is often critical for convergence to optimal solutions.

The Importance of a Suitable Distance Function in Belief-Space Planning

We consider robot navigation in environments given a known static map, but where dynamic obstacles of varying and unknown lifespans appear and disappear over time We describe a roadmap-based formulation of the problem that takes the sensing and transition uncertainty into account, and an efficient online planner for this problem The planner displays behaviors such as persistence and obstacle timeouts that would normally be hardcoded into an executive It is also able to make inferences about obstacle types even with impoverished sensors We present empirical results on simulated domains and on a PR2 robot

http://www.roboticsproceedings.org/rss08/p37.pdf

Robust Navigation Execution by Planning in Belief Space

In this survey we present different approaches that allow an intelligent agent to explore autonomous its environment to gather information and learn multiple tasks. Different communities proposed different solutions, that are in many cases, similar and/or complementary. These solutions include active learning, exploration/exploitation, online-learning and social learning. The common aspect of all these approaches is that it is the agent to selects and decides what information to gather next. Applications for these approaches already include tutoring systems, autonomous grasping learning, navigation and mapping and human-robot interaction. We discuss how these approaches are related, explaining their similarities and their differences in terms of problem assumptions and metrics of success. We consider that such an integrated discussion will improve inter-disciplinary research and applications.

Active Learning for Autonomous Intelligent Agents: Exploration, Curiosity, and Interaction

We consider a problem called task ordering with path uncertainty (TOP-U) where multiple robots are provided with a set of task locations to visit in a bounded environment, but the length of the path between a pair of task locations is initially known only coarsely by the robots. The objective of the robots is to find the order of tasks that reduces the path length (or, energy expended) to visit the task locations in such a scenario. To solve this problem, we propose an abstraction called a task reachability graph (TRG) that integrates the task ordering with the path planning by the robots. The TRG is updated dynamically based on inter-task path costs calculated using a sampling-based motion planner, and, a Hidden Markov Model (HMM)-based technique that calculates the belief in the current path costs based on the environment perceived by the robot's sensors and task completion information received from other robots. We then describe a Markov Decision Process (MDP)-based algorithm that can select the paths that reduce the overall path length to visit the task locations and a coordination algorithm that resolves path conflicts between robots. We have shown analytically that our task selection algorithm finds the lowest cost path returned by the motion planner, and, that our proposed coordination algorithm is deadlock free. We have also evaluated our algorithm on simulated Corobot robots within different environments while varying the number of task locations, obstacle geometries and number of robots, as well as on physical Corobot robots. Our results show that the TRG-based approach can perform considerably better in planning and locomotion times, and number of re-plans, while traveling almost-similar distances as compared to a closest first, no uncertainty (CFNU) task selection algorithm.

Integrated Task and Motion Planning for Multiple Robots under Path and Communication Uncertainties

Traditionally, pathfinding is solved using classical search algorithms such as the Dijkstra's, A*, probabilistic roadmaps and jump point search. These algorithms are still more practical in a familiar environment that has minimum changes. However, these generated navigation path may lose their appropriateness as they cannot handle dynamic changes in the complex environment, restricting independent living of visually impaired people. Nowadays, a network of smart physical devices called Internet of Things has become a foundation infrastructure for indoor navigation and pathfinding. Although, variations in the environment are identified and stored by sensors, there is absence of a reasonable system that adapts to the variable circumstances and learns to react to the changes. In this paper, we introduce a learning based autonomous system DynaPATH that classifies events of dynamic environments and adapts to the changes. We have performed simulation in order to evaluate the effectiveness of our approach. The results of our approach are compared with performance of different pathfinding algorithms for VIP people. The simulation results display strong conclusion that our proposed system has high stability and is VIP friendly for navigation in a complex environment.

DynaPATH: Dynamic Learning Based Indoor Navigation for VIP in IoT Based Environments

Probabilistic Roadmaps (PRM) are a commonly used class of algorithms for robot navigation tasks where obstacles are present in the environment. We examine the situation where the obstacle positions are not precisely known. A subset of the edges in the PRM graph may possibly intersect the obstacles, and as the robot traverses the graph it can make noisy observations of these uncertain edges to determine if it can traverse them or not. The problem is to traverse the graph from an initial vertex to a goal without taking a blocked edge, and to do this optimally the robot needs to consider the observations it can make as well as the structure of the graph. In this paper we show how this problem can be represented as a POMDP. We show that while too large to be solved with exact methods, approximate point based methods can provide a good quality solution. While feasible for smaller examples, this approach isn't scalable. By exploiting the structure in the belief space, we can construct an approximate belief-space MDP that can be solved efficiently. We demonstrate that this gives near optimal results in most cases while achieving an order of magnitude speed-up in policy generation time.

https://www.aaai.org/ocs/index.php/ICAPS/ICAPS09/paper/download/717/1113

Navigation planning in probabilistic roadmaps with uncertainty

Probabilistic roadmaps (PRMs) are a commonly used approach to path planning in continuous spaces with obstacles. We examine the case where the obstacle locations are not known with certainty but can be observed during execution of the plan. We abstract the problem to one of traversing a graph where some edges (referred to as uncertain edges) may or may not be present, and where noisy observations of these edges can be made from some of the vertices of the graph. We show that this problem can be represented as a POMDP, and then use the structure in the problem to derive a number of MDP approximations to the POMDP. We show that using these approximations we can solve larger PRMs efficiently while producing policies that are close to optimal for many problems, and that we can produce optimal solutions for PRMs with smaller numbers of uncertain edges.

/pdf/uncertain-probabilistic-roadmaps-with-observations-56kjzmwx3x.pdf

Michael Kneebone

Papers

Navigation planning in probabilistic roadmaps with uncertainty

Uncertain Probabilistic Roadmaps with Observations