Planning algorithms are impacting technical disciplines and industries around the world, including robotics, computer-aided design, manufacturing, computer graphics, aerospace applications, drug design, and protein folding. This coherent and comprehensive book unifies material from several sources, including robotics, control theory, artificial intelligence, and algorithms. The treatment is centered on robot motion planning but integrates material on planning in discrete spaces. A major part of the book is devoted to planning under uncertainty, including decision theory, Markov decision processes, and information spaces, which are the “configuration spaces” of all sensor-based planning problems. The last part of the book delves into planning under differential constraints that arise when automating the motions of virtually any mechanical system. Developed from courses taught by the author, the book is intended for students, engineers, and researchers in robotics, artificial intelligence, and control theory as well as computer graphics, algorithms, and computational biology.

Planning Algorithms: Introductory Material

A text that makes the mathematical underpinnings of robot motion accessible and relates low-level details of implementation to high-level algorithmic concepts. Robot motion planning has become a major focus of robotics. Research findings can be applied not only to robotics but to planning routes on circuit boards, directing digital actors in computer graphics, robot-assisted surgery and medicine, and in novel areas such as drug design and protein folding. This text reflects the great advances that have taken place in the last ten years, including sensor-based planning, probabalistic planning, localization and mapping, and motion planning for dynamic and nonholonomic systems. Its presentation makes the mathematical underpinnings of robot motion accessible to students of computer science and engineering, rleating low-level implementation details to high-level algorithmic concepts.

Principles of Robot Motion: Theory, Algorithms, and Implementations

As robots and other intelligent agents move from simple environments and problems to more complex, unstructured settings, manually programming their behavior has become increasingly challenging and expensive. Often, it is easier for a teacher to demonstrate a desired behavior rather than attempt to manually engineer it. This process of learning from demonstrations, and the study of algorithms to do so, is called imitation learning. This work provides an introduction to imitation learning. It covers the underlying assumptions, approaches, and how they relate; the rich set of algorithms developed to tackle the problem; and advice on effective tools and implementation. We intend this paper to serve two audiences. First, we want to familiarize machine learning experts with the challenges of imitation learning, particularly those arising in robotics, and the interesting theoretical and practical distinctions between it and more familiar frameworks like statistical supervised learning theory and reinforcement learning. Second, we want to give roboticists and experts in applied artificial intelligence a broader appreciation for the frameworks and tools available for imitation learning. We pay particular attention to the intimate connection between imitation learning approaches and those of structured prediction Daume III et al. [2009]. To structure this discussion, we categorize imitation learning techniques based on the following key criteria which drive algorithmic decisions:

1) The structure of the policy space. Is the learned policy a time-index trajectory (trajectory learning), a mapping from observations to actions (so called behavioral cloning [Bain and Sammut, 1996]), or the result of a complex optimization or planning problem at each execution as is common in inverse optimal control methods [Kalman, 1964, Moylan and Anderson, 1973].

2) The information available during training and testing. In particular, is the learning algorithm privy to the full state that the teacher possess? Is the learner able to interact with the teacher and gather corrections or more data? Does the learner have a (typically a priori) model of the system with which it interacts? Does the learner have access to the reward (cost) function that the teacher is attempting to optimize?

3) The notion of success. Different algorithmic approaches provide varying guarantees on the resulting learned behavior. These guarantees range from weaker (e.g., measuring disagreement with the agent’s decision) to stronger (e.g., providing guarantees on the performance of the learner with respect to a true cost function, either known or unknown). We organize our work by paying particular attention to distinction (1): dividing imitation learning into directly replicating desired behavior (sometimes called behavioral cloning) and learning the hidden objectives of the desired behavior from demonstrations (called inverse optimal control or inverse reinforcement learning [Russell, 1998]). In the latter case, behavior arises as the result of an optimization problem solved for each new instance that the learner faces. In addition to method analysis, we discuss the design decisions a practitioner must make when selecting an imitation learning approach. Moreover, application examples—such as robots that play table tennis [Kober and Peters, 2009], programs that play the game of Go [Silver et al., 2016], and systems that understand natural language [Wen et al., 2015]— illustrate the properties and motivations behind different forms of imitation learning. We conclude by presenting a set of open questions and point towards possible future research directions for machine learning.

/pdf/an-algorithmic-perspective-on-imitation-learning-n55kqg2cp5.pdf

An Algorithmic Perspective on Imitation Learning

The need for combined task and motion planning in robotics is well understood. Solutions to this problem have typically relied on special purpose, integrated implementations of task planning and motion planning algorithms. We propose a new approach that uses off-the-shelf task planners and motion planners and makes no assumptions about their implementation. Doing so enables our approach to directly build on, and benefit from, the vast literature and latest advances in task planning and motion planning. It uses a novel representational abstraction and requires only that failures in computing a motion plan for a high-level action be identifiable and expressible in the form of logical predicates at the task level. We evaluate the approach and illustrate its robustness through a number of experiments using a state-of-the-art robotics simulator and a PR2 robot. These experiments show the system accomplishing a diverse set of challenging tasks such as taking advantage of a tray when laying out a table for dinner and picking objects from cluttered environments where other objects need to be re-arranged before the target object can be reached.

/pdf/combined-task-and-motion-planning-through-an-extensible-14tlm2u6vw.pdf

Combined task and motion planning through an extensible planner-independent interface layer

In this paper we outline an approach to the integration of task planning and motion planning that has the following key properties: It is aggressively hierarchical; it makes choices and commits to them in a top-down fashion in an attempt to limit the length of plans that need to be constructed, and thereby exponentially decrease the amount of search required It operates on detailed, continuous geometric representations and does not require a-priori discretization of the state or action spaces

/pdf/hierarchical-task-and-motion-planning-in-the-now-2iyacxddt2.pdf

Hierarchical task and motion planning in the now

We propose a representation and a planning algorithm able to deal with problems integrating task planning as well as motion and manipulation planning knowledge involving several robots and objects. Robot plans often include actions where the robot has to place itself in some position in order to perform some other action or to "modify" the configuration of its environment by displacing objects. Our approach aims at establishing a bridge between task planning and manipulation planning that allows a rigorous treatment of geometric preconditions and effects of robot actions in realistic environments. We show how links can be established between a symbolic description and its geometric counterpart and how they can be used in an integrated planning process that is able to deal with intricate symbolic and geometric constraints. Finally, we describe the main features of an implemented planner and discuss several examples of its use.

/pdf/a-hybrid-approach-to-intricate-motion-manipulation-and-task-u23w5hr9y1.pdf

A Hybrid Approach to Intricate Motion, Manipulation and Task Planning

We propose an original approach to integrate symbolic task planning, and geometric motion and manipulation planning. We focus more particularly on one key aspect: the relation between the symbolic positions and their geometric counterparts. Indeed, we have developed an instantiation process that is able to propagate incrementally task-dependent as well as 3D environment-dependent constraints and to guide efficiently the search until valid geometric configurations are found that satisfy the plan at both levels. The overall process is discussed and illustrated through an implemented example.

/pdf/asymov-a-planner-that-deals-with-intricate-symbolic-and-4j4269k9ak.pdf

aSyMov: A Planner That Deals with Intricate Symbolic and Geometric Problems

We have developed an original planner, aSyMov, that has been specially designed to address intricate robot planning problems where geometric constraints cannot be simply "abstracted" in a way that has no influence on the symbolic plan. This paper presents the ingredients that allowed us to establish an effective link between the representations used by a symbolic task planner and the representations used by a realistic motion and manipulation planning library. The architecture and the main plan search strategies are presented together with an illustrative example solved by a prototype implementation of aSyMov. At each step of the planning process both symbolic and geometric constraints are considered. Besides, the planning process tries to arbitrate between finding a plan with the level of knowledge it has already acquired, or "investing" more in a deeper knowledge of the topology of the different configuration spaces it manipulates.

/pdf/a-robot-task-planner-that-merges-symbolic-and-geometric-e9rq6kanbk.pdf

A robot task planner that merges symbolic and geometric reasoning

In this paper we propose a new and integrated approach to solve planning problems with both symbolic and geometric aspects. Typical concerned domains are intricate manipulation and task planning problems involving multiple robots in three dimensional worlds. The approach involves a task planner that guides a probabilistic roadmap method used to capture the topology of the free space in various contexts. At each step of our hybrid planner aSyMov both symbolic and geometric data are taken into account. Specific predicates are used to link both aspects. Preliminary results obtained by a prototype implementation are shortly presented and discussed.

/pdf/overview-of-asymov-integrating-motion-manipulation-and-task-27vwheglqg.pdf

Overview of aSyMov: Integrating Motion, Manipulation and Task Planning

L'objet des recherches en planification de tâches est d'elaborer des algorithmes et des representations qui participent a l'autonomie des robots en les dotant de capacites decisionnelles. Classiquement, un plan symbolique synthetise par un planificateur permet au robot de decider des prochaines actions qu'il doit accomplir pour satisfaire sa mission. Dans cette these, nous developpons l'idee que les planificateurs actuels, bases sur une representation logique du monde, ne sont pas suffisants pour traiter des missions ou les contraintes de mouvements des robots et celles liees a la manipulation d'objet ne peuvent pas etre ignorees a cette etape. Souvent, en robotique, la faisabilite des mouvements et des manipulations est un pre-requis a la bonne execution d'un plan. Dans cette these, nous introduisons un formalisme permettant de prendre en consideration ces contraintes dans le modele utilise par un planificateur. En nous appuyant sur des resultats issus de la planification de mouvements et de manipulations autant que sur ceux de la planification de tâches, nous proposons un nouveau planificateur capable de raisonner sur ce formalisme. Nous produisons alors des plans definissant non seulement un ordre d'actions mais egalement les mouvements sans collision necessaires a son execution. L'implementation de ce planificateur est presentee et evaluee.

https://tel.archives-ouvertes.fr/tel-00009845/document

Stéphane Cambon

Papers

A Hybrid Approach to Intricate Motion, Manipulation and Task Planning

aSyMov: A Planner That Deals with Intricate Symbolic and Geometric Problems

A robot task planner that merges symbolic and geometric reasoning

Overview of aSyMov: Integrating Motion, Manipulation and Task Planning

Planifier avec les contraintes géométriques du mouvement et de la manipulation