Collision avoidance algorithms are essential for safe and efficient robot operation among pedestrians. This work proposes using deep reinforcement (RL) learning as a framework to model the complex interactions and cooperation with nearby, decision-making agents, such as pedestrians and other robots. Existing RL-based works assume homogeneity of agent properties, use specific motion models over short timescales, or lack a principled method to handle a large, possibly varying number of agents. Therefore, this work develops an algorithm that learns collision avoidance among a variety of heterogeneous, non-communicating, dynamic agents without assuming they follow any particular behavior rules. It extends our previous work by introducing a strategy using Long Short-Term Memory (LSTM) that enables the algorithm to use observations of an arbitrary number of other agents, instead of a small, fixed number of neighbors. The proposed algorithm is shown to outperform a classical collision avoidance algorithm, another deep RL-based algorithm, and scales with the number of agents better (fewer collisions, shorter time to goal) than our previously published learning-based approach. Analysis of the LSTM provides insights into how observations of nearby agents affect the hidden state and quantifies the performance impact of various agent ordering heuristics. The learned policy generalizes to several applications beyond the training scenarios: formation control (arrangement into letters), demonstrations on a fleet of four multirotors and on a fully autonomous robotic vehicle capable of traveling at human walking speed among pedestrians.

Collision Avoidance in Pedestrian-Rich Environments With Deep Reinforcement Learning

We consider the following differential game of pursuit and evasion involving two participating players: an evader, which has limited maneuverability, and an agile pursuer. The agents move on the Euclidean plane with different but constant speeds. Whereas the pursuer can change the orientation of its velocity vector arbitrarily fast, that is, he is a “pedestrian” a la Isaacs, the evader cannot make turns having a radius smaller than a specified minimum turning radius. This problem can be seen as a reversed Homicidal Chauffeur game, hence the name “Suicidal Pedestrian Differential Game.” The aim of this paper is to derive the optimal strategies of the two players and characterize the initial conditions that lead to capture if the pursuer acts optimally, and areas that guarantee evasion regardless of the pursuer’s strategy. Both proximity-capture and point-capture are considered. After applying the optimal strategy for the evader, it is shown that the case of point-capture reduces to a special version of Zermelo’s Navigation Problem (ZNP) for the pursuer. Therefore, the well-known ZNP solution can be used to validate the results obtained through the differential game framework, as well as to characterize the time-optimal trajectories. The results are directly applicable to collision avoidance in maritime and Air Traffic Control applications.

On the Suicidal Pedestrian Differential Game

Isaacs' formulation of the game of two cars, which assumes players of circular shape, is extended to include the effect of elongated players and elliptical shapes in particular. The geometry is used to model the problem of ship-collision avoidance as a differential game which suggests the name of ‘game of two ships’. The game of two ships is formulated as a free-time unbounded game of kind. A general solution for the optimal controls on the terminal manifold is given and is also used to derive a model analytical solution for the primary optimal path emanating from the target set. The occurrence of singular arcs as well as the condition for switching are also discussed. Finally, a capture criterion for the pursuit-evasion game between a faster elliptical pursuer and a more manoeuvrable circular evader is also derived. This capture criterion is given in the form of a relationship between the geometrical dimensions of the players, their speed and acceleration ratios, and the maximum range that the manoeuvrable, yet slower, evader can indefinitely guarantee without being captured. If both players have circular shapes, our solution reduces to an already-known capture criterion for the ‘game of two cars’.

The game of two elliptical ships

A differential game method has been developed which divides the state-space into two regions. In one region ship P2 can always cause a collision, in the other region ship P1 can prevent a collision, provided it uses an appropriate strategy. In a future publication slightly different sets of ships will be considered such as to get a feeling for the sensitivity of the collision avoidance capacities w.r.t. the three parameters which characterize a ship (i.e. length, velocity, maximum turnrate).

A differential game approach to collision avoidance of ships

In this paper we consider a differential game of pursuit and evasion involving two players with constant, but different, speeds, and different maneuverability constraints. Specifically, the evader has limited maneuverability, while the pursuer is completely agile. This problem is an asymmetric version of the well-known Game of Two Cars. The aim of this paper is to derive the optimal strategies of the two players and characterize areas of initial conditions that lead to capture if the pursuer acts optimally, and areas that guarantee evasion regardless of the pursuer's strategy. It is shown that the problem reduces to a special version of Zermelo's Navigation Problem (ZNP) for the pursuer. Therefore, the well-known ZNP solution can be used to validate the results obtained through the differential game framework as well as to characterize the time-optimal trajectories. The results are directly applicable to collision avoidance problems.

/pdf/an-asymmetric-version-of-the-two-car-pursuit-evasion-game-314kuylj7s.pdf

An asymmetric version of the two car pursuit-evasion game

When considering the subject of ship collision avoidance many factors or principles must be included. Even with the proper use of modern navigational aids such as RADAR many collisions still occur and still others are avoided only through last second actions. This article reviews the principles of proper collision avoidance along with many of the common problems and errors associated with the use of RADAR. The use of automated plotting in heavy traffic is also discussed.

A G Dunne

Papers

Ship collision avoidance