Motion planning is a fundamental research area in robotics. Sampling-based methods offer an efficient solution for what is otherwise a rather challenging dilemma of path planning. Consequently, these methods have been extended further away from basic robot planning into further difficult scenarios and diverse applications. A comprehensive survey of the growing body of work in sampling-based planning is given here. Simulations are executed to evaluate some of the proposed planners and highlight some of the implementation details that are often left unspecified. An emphasis is placed on contemporary research directions in this field. We address planners that tackle current issues in robotics. For instance, real-life kinodynamic planning, optimal planning, replanning in dynamic environments, and planning under uncertainty are discussed. The aim of this paper is to survey the state of the art in motion planning and to assess selected planners, examine implementation details and above all shed a light on the current challenges in motion planning and the promising approaches that will potentially overcome those problems.

Sampling-Based Robot Motion Planning: A Review

A longstanding goal in character animation is to combine data-driven specification of behavior with a system that can execute a similar behavior in a physical simulation, thus enabling realistic responses to perturbations and environmental variation. We show that well-known reinforcement learning (RL) methods can be adapted to learn robust control policies capable of imitating a broad range of example motion clips, while also learning complex recoveries, adapting to changes in morphology, and accomplishing user-specified goals. Our method handles keyframed motions, highly-dynamic actions such as motion-captured flips and spins, and retargeted motions. By combining a motion-imitation objective with a task objective, we can train characters that react intelligently in interactive settings, e.g., by walking in a desired direction or throwing a ball at a user-specified target. This approach thus combines the convenience and motion quality of using motion clips to define the desired style and appearance, with the flexibility and generality afforded by RL methods and physics-based animation. We further explore a number of methods for integrating multiple clips into the learning process to develop multi-skilled agents capable of performing a rich repertoire of diverse skills. We demonstrate results using multiple characters (human, Atlas robot, bipedal dinosaur, dragon) and a large variety of skills, including locomotion, acrobatics, and martial arts.

DeepMimic: example-guided deep reinforcement learning of physics-based character skills

Physics-based simulation and control of biped locomotion is difficult because bipeds are unstable, underactuated, high-dimensional dynamical systems. We develop a simple control strategy that can be used to generate a large variety of gaits and styles in real-time, including walking in all directions (forwards, backwards, sideways, turning), running, skipping, and hopping. Controllers can be authored using a small number of parameters, or their construction can be informed by motion capture data. The controllers are applied to 2D and 3D physically-simulated character models. Their robustness is demonstrated with respect to pushes in all directions, unexpected steps and slopes, and unexpected variations in kinematic and dynamic parameters. Direct transitions between controllers are demonstrated as well as parameterized control of changes in direction and speed. Feedback-error learning is applied to learn predictive torque models, which allows for the low-gain control that typifies many natural motions as well as producing smoother simulated motion.

/pdf/simbicon-simple-biped-locomotion-control-483awhcdyv.pdf

SIMBICON: simple biped locomotion control

Technical Section: Sketch-based modeling: A survey

DeepMimic: Example-Guided Deep Reinforcement Learning of Physics-Based Character Skills

This paper proposes a new variation of the RRT planner which demonstrates good performance on both loosely-constrained and highly-constrained environments. The key to the planner is an implicit flood-fill-like mechanism, a technique that is well suited to escaping local minima in highly constrained problems. We show sample results for a variety of problems and environments, and discuss future improvements

/pdf/rrt-blossom-rrt-with-a-local-flood-fill-behavior-50f6zo1vo7.pdf

RRT-blossom: RRT with a local flood-fill behavior

We present a method for computing controllers for stable planar-biped walking gaits that follow a particular style. The desired style is specified with a kinematic target trajectory that may or may not be physically realizable. A nearest-neighbor controller representation is used and its free parameters are optimized using a local parameter search technique. The optimization function is constructed by integrating a mass-distance metric over fixed time intervals, which serves to measure the deviation of a simulated motion from a desired target motion. We demonstrate simulated bipedal walks having user-specified styles, walks for bipeds of varying dimensions, walks over terrain of known slopes, and walks that are robust with respect to unobserved terrain variations and modeling errors.

/pdf/synthesis-of-controllers-for-stylized-planar-bipedal-walking-1h5zstwcg0.pdf

Synthesis of Controllers for Stylized Planar Bipedal Walking

Current motion planners, in general, can neither "see" the world around them, nor learn from experience. That is, their reliance on collision tests as the only means of sensing the environment yields a tactile, myopic perception of the world. Such short-sightedness greatly limits any potential for detection, learning, or reasoning about frequently encountered situations. As a result, it is common for current planners to solve and re-solve the same general scenarios over and over, each time none the wiser. We thus propose a general approach for motion planning, as well as a specific illustrative algorithm, in which local sensory information, in conjunction with prior accumulated experience, are exploited to improve planner performance. Our approach relies on learning viability models for the agent's "perceptual space", and the use thereof to direct planning effort. Experiments with three test agents show significant speedups and skill-transfer between environments.

/pdf/faster-motion-planning-using-learned-local-viability-models-4usb2smu96.pdf

Faster Motion Planning Using Learned Local Viability Models

Sketch-based modeling shares many of the difficulties of the branch of computer vision that deals with single image interpretation. Most obviously, they must both identify the parts observed in a given 2D drawing or image.We draw on constellation models first proposed in the computer vision literature to develop probabilistic models for object sketches, based on multiple example drawings. These models are then applied to estimate the most-likely labels for a new sketch. A multi-pass branch-and-bound algorithm allows well-formed sketches to be quickly labelled, while still supporting the recognition of more ambiguous sketches. Results are presented for five classes of objects.

https://www.cs.ubc.ca/~van/papers/2006-sbim-constellation.pdf

Constellation models for sketch recognition

A numerical method is proposed for the constraint of the state of a dynamical system such that it cannot enter a predefined failure region. The proposed approach to this viability problem involves an explicit numerical approximation of a viability envelope, coupled with a practical strategy for enforcing containment that is based upon a predictive look-ahead strategy. The approach can be applied to achieve automated "intervention when necessary" to enforce system safety at interactive rates. Applications are shown to several low-dimensional systems, including steering control of a vehicle constrained to a given environment geometry.

/pdf/approximate-safety-enforcement-using-computed-viability-2rr6we5z1a.pdf

M. van de Panne

Papers

RRT-blossom: RRT with a local flood-fill behavior

Synthesis of Controllers for Stylized Planar Bipedal Walking

Faster Motion Planning Using Learned Local Viability Models

Constellation models for sketch recognition

Approximate safety enforcement using computed viability envelopes