Cooperative pursuit with Voronoi partitions

One of the primary challenges for autonomous robotics in uncertain and dynamic environments is planning and executing a collision-free path. Hybrid dynamic obstacles present an even greater challenge as the obstacles can change dynamics without warning and potentially invalidate paths. Artificial potential field (APF)-based techniques have shown great promise in successful path planning in highly dynamic environments due to their low cost at runtime. We utilize the APF framework for runtime planning but leverage a formal validation method, Stochastic Reachable (SR) sets, to generate accurate potential fields for moving obstacles. A small number of SR sets are computed a priori , then used to generate a potential field that represents the obstacle's stochastic motion for online path planning. Our method is novel and scales well with the number of obstacles, maintaining a relatively high probability of reaching the goal without collision, as compared to other traditional Gaussian APF methods. Here, we demonstrate our method with up to 900 hybrid dynamic obstacles and show that it outperforms the traditional Gaussian APF method by up to 60% in the holonomic case and up to 20% in the unicycle case.

Hybrid Dynamic Moving Obstacle Avoidance Using a Stochastic Reachable Set-Based Potential Field

Highly dynamic environments pose a particular challenge for motion planning due to the need for constant evaluation or validation of plans. However, due to the wide range of applications, an algorithm to safely plan in the presence of moving obstacles is required. In this paper, we propose a novel technique that provides computationally efficient planning solutions in environments with static obstacles and several dynamic obstacles with stochastic motions. Path-Guided APF-SR works by first applying a sampling-based technique to identify a valid, collision-free path in the presence of static obstacles. Then, an artificial potential field planning method is used to safely navigate through the moving obstacles using the path as an attractive intermediate goal bias. In order to improve the safety of the artificial potential field, repulsive potential fields around moving obstacles are calculated with stochastic reachable sets, a method previously shown to significantly improve planning success in highly dynamic environments. We show that Path-Guided APF-SR outperforms other methods that have high planning success in environments with 300 stochastically moving obstacles. Furthermore, planning is achievable in environments in which previously developed methods have failed.

Path-guided artificial potential fields with stochastic reachable sets for motion planning in highly dynamic environments

Recently, deep reinforcement learning (RL) methods have been applied successfully to multi-agent scenarios. Typically, these methods rely on a concatenation of agent states to represent the information content required for decentralized decision making. However, concatenation scales poorly to swarm systems with a large number of homogeneous agents as it does not exploit the fundamental properties inherent to these systems: (i) the agents in the swarm are interchangeable and (ii) the exact number of agents in the swarm is irrelevant. Therefore, we propose a new state representation for deep multi-agent RL based on mean embeddings of distributions. We treat the agents as samples of a distribution and use the empirical mean embedding as input for a decentralized policy. We define different feature spaces of the mean embedding using histograms, radial basis functions and a neural network learned end-to-end. We evaluate the representation on two well known problems from the swarm literature (rendezvous and pursuit evasion), in a globally and locally observable setup. For the local setup we furthermore introduce simple communication protocols. Of all approaches, the mean embedding representation using neural network features enables the richest information exchange between neighboring agents facilitating the development of more complex collective strategies.

Deep Reinforcement Learning for Swarm Systems

We consider autonomous racing of two cars and present an approach to formulate racing decisions as a noncooperative nonzero-sum game. We design three different games where the players aim to fulfill static track constraints as well as avoid collision with each other; the latter constraint depends on the combined actions of the two players. The difference between the games is the collision constraints and the payoff. In the first game, collision avoidance is only considered by the follower, and each player maximizes their own progress toward the finish line. We show that, thanks to the sequential structure of this game, equilibria can be computed through an efficient sequential maximization approach. Furthermore, we show that these actions, if feasible, are also a Stackelberg and Nash equilibrium in pure strategies of our second game where both players consider the collision constraints. The payoff of our third game is designed to promote blocking, by additionally rewarding the cars for staying ahead at the end of the horizon. We show that this changes the Stackelberg equilibrium, but has a minor influence on the Nash equilibria. For online implementation, we propose to play the games in a moving horizon fashion and discuss two methods for guaranteeing feasibility of the resulting coupled repeated games. Finally, we study the performance of the proposed approaches in simulation for a setup that replicates the miniature race car tested at the Automatic Control Laboratory, ETH Zurich, Switzerland. The simulation study shows that the presented games can successfully model different racing behaviors and generate interesting racing situations.

A Noncooperative Game Approach to Autonomous Racing

A reach-avoid game is one in which an agent attempts to reach a predefined goal, while avoiding some adversarial circumstance induced by an opposing agent or disturbance. Their analysis plays an important role in problems such as safe motion planning and obstacle avoidance, yet computing solutions is often computationally expensive due to the need to consider adversarial inputs. In this work, we present an open-loop formulation of a two-player reach-avoid game whereby the players define their control inputs prior to the start of the game. We define two open-loop games, each of which is conservative towards one player, show how the solutions to these games are related to the optimal feedback strategy for the closed-loop game, and demonstrate a modified Fast Marching Method to efficiently compute those solutions.

A general, open-loop formulation for reach-avoid games

Efficient path planning algorithms in reach-avoid problems

We propose a PDE approach for computing time-optimal trajectories of a vehicle which travels under certain curvature constraints. We derive a class of Hamilton---Jacobi equations which models such motions; it unifies two well-known vehicular models, the Dubins' and Reeds---Shepp's cars, and gives further generalizations. Numerical methods (finite difference for the Reeds---Shepp's car and semi-Lagrangian for the Dubins' car) are investigated for two-dimensional domains and surfaces.

Optimal Trajectories of Curvature Constrained Motion in the Hamilton---Jacobi Formulation

We present an efficient algorithm which computes, for a kinematic point mass moving in the plane, a time-optimal path that visits a sequence of target sets while conservatively avoiding collision with moving obstacles, also modelled as kinematic point masses, but whose trajectories are unknown. The problem is formulated as a pursuit-evasion differential game, and the underlying construction is based on optimal control. The algorithm, which is a variant of the fast marching method for shortest path problems, can handle general dynamical constraints on the players and arbitrary domain geometry (e.g. obstacles, non-polygonal boundaries). Applications to a two-stage game, capture-the-flag, is presented.

Time-optimal multi-stage motion planning with guaranteed collision avoidance via an open-loop game formulation

: We present an algorithm which computes the value function and optimal paths for a two-player static game, where the goal of one player is to maintain visibility of an adversarial player for as long as possible, and that of the adversarial player is to minimize this time. In a static game both players choose their controls at initial time and run in open-loop for t 0 until the end-game condition is met. Compared to closed-loop (feedback strategy) games which require solving a PDE in high dimensions, we demonstrate that the static game can be solved directly in the state space by the proposed PDE-based technique. This results in significant savings in both memory and computational cost, at the expense of a simpler information pattern that is more conservative towards one player. In addition, we describe how this algorithm can be easily generalized to games with multiple evaders. Applications to target tracking and an extension to a feedback control game are also presented.

https://www.intlpress.com/site/pub/files/_fulltext/journals/cms/2014/0012/0007/CMS-2014-0012-0007-a007.pdf

Ryo Takei

Papers

A general, open-loop formulation for reach-avoid games

Efficient path planning algorithms in reach-avoid problems

Optimal Trajectories of Curvature Constrained Motion in the Hamilton---Jacobi Formulation

Time-optimal multi-stage motion planning with guaranteed collision avoidance via an open-loop game formulation

An Efficient Algorithm for a Visibility-Based Surveillance-Evasion Game