Pursuit-evasion is the problem of capturing mobile targets with one or more pursuers. We use deep reinforcement learning for pursuing an omnidirectional target with multiple, homogeneous agents that are subject to unicycle kinematic constraints. We use shared experience to train a policy for a given number of pursuers, executed independently by each agent at run-time. The training uses curriculum learning, a sweeping-angle ordering to locally represent neighboring agents, and a reward structure that encourages a good formation and combines individual and group rewards. Simulated experiments with a reactive evader and up to eight pursuers show that our learning-based approach outperforms recent reinforcement learning techniques as well as non-holonomic adaptations of classical algorithms. The learned policy is successfully transferred to the real-world in a proof-of-concept demonstration with three motion-constrained pursuer drones.

Decentralized Multi-Agent Pursuit Using Deep Reinforcement Learning

Pursuit-evasion is the problem of capturing mobile targets with one or more pursuers. We use deep reinforcement learning for pursuing an omni-directional target with multiple, homogeneous agents that are subject to unicycle kinematic constraints. We use shared experience to train a policy for a given number of pursuers that is executed independently by each agent at run-time. The training benefits from curriculum learning, a sweeping-angle ordering to locally represent neighboring agents and encouraging good formations with reward structure that combines individual and group rewards. Simulated experiments with a reactive evader and up to eight pursuers show that our learning-based approach, with non-holonomic agents, performs on par with classical algorithms with omni-directional agents, and outperforms their non-holonomic adaptations. The learned policy is successfully transferred to the real world in a proof-of-concept demonstration with three motion-constrained pursuer drones.

Decentralized Multi-Agent Pursuit using Deep Reinforcement Learning

Abstract A behavioral-based strategy for cooperative hunting using drones is proposed in this paper. In this decentralized scheme, each drone acts as an individual agent computing its guidance strategy toward the target based on the relative position of its neighbors without the use of direct communication. The algorithm is based on the deviated pure pursuit methodology, and the emerged behavior mimics a natural hunting formation. Simulations and real-time experiments with varying conditions were carried out to validate the effectiveness of the proposed hunting scheme. Videos of the system in action can be seen on: https://youtu.be/g2dODbd6ZLA . 

/pdf/local-interaction-and-navigation-guidance-for-hunters-drones-3fzb9nf8.pdf

Local interaction and navigation guidance for hunters drones: a chase behavior approach with real-time tests

An energy reduction strategy for a fixed-wing drone with classic configuration is presented in this paper. The strategy is developed using the longitudinal non-linear motion equations of the aircraft, considering the stall effects in the determination of lift and drag forces. Our strategy imposes the aircraft to track a special trajectory allowing the vehicle to perform a curve in ascend, acting against the gravity force. As a consequence of this tracking, the angle of attack of the vehicle is increased arriving to critical values that need to be compensated for avoiding the crash of the vehicle. Therefore, a strategy to avoid the stall stage and recovering the flight stability in the vehicle is proposed. The strategy is evaluated in simulations for validating the performance of the whole system.

Least Airspeed Reduction Strategy & Flight Recuperation of a Fixed-Wing Drone

Abstract A behavioral-based strategy for cooperative hunting using drones is proposed in this paper. In this decentralized scheme, each drone acts as an individual agent computing its guidance strategy toward the target based on the relative position of its neighbors without the use of direct communication. The algorithm is based on the deviated pure pursuit methodology, and the emerged behavior mimics a natural hunting formation. Simulations and real-time experiments with varying conditions were carried out to validate the effectiveness of the proposed hunting scheme. Videos of the system in action can be seen on: https://youtu.be/g2dODbd6ZLA.

An experimental validation for improving pose estimation using a linear Kalman Filter (KF) is presented in this paper The procedure is designed to lead with localization data degraded or lost The methodology is focused on determination, tuning and dynamics changes in the covariance matrices in the KF algorithm Several simulations are carried out in order to validate the methodology Similarly several flights tests are conducted in real time for validating the observer scheme A localization system is used and modified for emulating the GPS performance Main results show the good behavior of the proposed methodology and a video of them is available for showing the capabilities of the algorithm

Enhanced UAV pose estimation using a KF: experimental validation

In this work, the problem for trajectory generation and tracking for phugoid maneuvers using a fixed-wing aerial vehicle is studied. The goal of using this kind of trajectories is to reduce the kinetic energy produced by the aerial vehicle during forward flight. In our study, the phugoid trajectory contains a recovery phase for safely continue to fly the aerial vehicle. In order to track the phugoid trajectory, separate controllers generated by using Lyapunov Analysis are proposed. Several simulations are carried out to validate experimentally the proposed theory, using several tools such as Gazebo, QGroundControl, PX4, and ROS (Robot Operating System).

Trajectory generation and tracking for phugoid maneuvers using a mini-airplane

A simple and reactive algorithm for drone pursuit and collision avoidance for non-cooperative agents, not requiring global path planning is proposed in this paper. The pursuit algorithm is based on the geometrical rule PN- Parallel Navigation, using the same principle to track the target and for evading obstacles. Besides, our strategy does not require separate modes for pursuit and collision avoidance, as common in numerous works. The algorithm is validated numerically when tracking a mobile target in a confined environment in the presence of dynamic objects. Results show the good performance of the algorithm and confirm that the collision route property can be used to avoid reactively un-predicted and dynamic objects.

C. de Souza

Papers

Enhanced UAV pose estimation using a KF: experimental validation

Trajectory generation and tracking for phugoid maneuvers using a mini-airplane

Reactive drone pursuit and obstacle avoidance based in parallel navigation