Design and Analysis of State-Feedback Optimal Strategies for the Differential Game of Active Defense
TL;DR: This paper provides a complete, closed form solution of the active target defense differential game; synthesize closed-loop state feedback optimal strategies for the agents and obtain the Value function of the game.
Abstract: This paper is concerned with a scenario of active target defense modeled as a zero-sum differential game. The differential game theory as developed by Isaacs provides the correct framework for the analysis of pursuit-evasion conflicts and the design of optimal strategies for the players involved in the game. This paper considers an Attacker missile pursuing a Target aircraft protected by a Defender missile which aims at intercepting the Attacker before the latter reaches the Target aircraft. A differential game is formulated where the two opposing players/teams try to minimize/maximize the distance between the Target and the Attacker at the time of interception of the Attacker by the Defender and such time indicates the termination of the game. The Attacker aims to minimize the terminal distance between itself and the Target at the moment of its interception by the Defender. The opposing player/team consists of two cooperating agents: The Target and the Defender. These two agents cooperate in order to accomplish the two objectives: Guarantee interception of the Attacker by the Defender and maximize the terminal Target-Attacker separation. In this paper, we provide a complete, closed form solution of the active target defense differential game; we synthesize closed-loop state feedback optimal strategies for the agents and obtain the Value function of the game. We characterize the Target's escape set and show that the Value function is continuous and continuously differentiable over the Target's escape set, and that it satisfies the Hamilton–Jacobi–Isaacs equation everywhere in this set.
Citations
More filters
TL;DR: A broad understanding gained from the survey overall will assist with the design of a holistic CUS and inspire cross-domain research across physical layer designs in wireless communications, CUS network designs, control theory, mechanics, and computer science, to enhance counter UAV techniques further.
Abstract: Recognizing the various and broad range of applications of unmanned aerial vehicles (UAVs) and unmanned aircraft systems (UAS) for personal, public and military applications, recent un-intentional malfunctions of uncontrollable UAVs or intentional attacks on them divert our attention and motivate us to devise a protection system, referred to as a counter UAV system (CUS). The CUS, also known as a counter-drone system, protects personal, commercial, public, and military facilities and areas from uncontrollable and belligerent UAVs by neutralizing or destroying them. This paper provides a comprehensive survey of the CUS to describe the key technologies of the CUS and provide sufficient information with wich to comprehend this system. The first part starts with an introduction of general UAVs and the concept of the CUS. In the second part, we provide an extensive survey of the CUS through a top-down approach: i) the platform of CUS including ground and sky platforms and related networks; ii) the architecture of the CUS consisting of sensing systems, command-and-control (C2) systems, and mitigation systems; and iii) the devices and functions with the sensors for detection-and-identification and localization-and-tracking actions and mitigators for neutralization. The last part is devoted to a survey of the CUS market with relevant challenges and future visions. From the CUS market survey, potential readers can identify the major players in a CUS industry and obtain information with which to develop the CUS industry. A broad understanding gained from the survey overall will assist with the design of a holistic CUS and inspire cross-domain research across physical layer designs in wireless communications, CUS network designs, control theory, mechanics, and computer science, to enhance counter UAV techniques further.
63 citations
20 May 2019
TL;DR: This work shows how a time-discounted modification of the problem of maximizing the minimum payoff over time, central to safety analysis, through a modified dynamic programming equation that induces a contraction mapping can render reinforcement learning techniques amenable to quantitative safety analysis as tools to approximate the safe set and optimal safety policy.
Abstract: Safety analysis is a necessary component in the design and deployment of autonomous robotic systems. Techniques from robust optimal control theory, such as Hamilton-Jacobi reachability analysis, allow a rigorous formalization of safety as guaranteed constraint satisfaction. Unfortunately, the computational complexity of these tools for general dynamical systems scales poorly with state dimension, making existing tools impractical beyond small problems. Modern reinforcement learning methods have shown promising ability to find approximate yet proficient solutions to optimal control problems in complex and high-dimensional systems, however their application has in practice been restricted to problems with an additive payoff over time, unsuitable for reasoning about safety. In recent work, we introduced a time-discounted modification of the problem of maximizing the minimum payoff over time, central to safety analysis, through a modified dynamic programming equation that induces a contraction mapping. Here, we show how a similar contraction mapping can render reinforcement learning techniques amenable to quantitative safety analysis as tools to approximate the safe set and optimal safety policy. This opens a new avenue of research connecting control-theoretic safety analysis and the reinforcement learning domain. We validate the correctness of our formulation by comparing safety results computed through Q-learning to analytic and numerical solutions, and demonstrate its scalability by learning safe sets and control policies for simulated systems of up to 18 state dimensions using value learning and policy gradient techniques.
62 citations
Cites background from "Design and Analysis of State-Feedba..."
...While analytic solutions exist in rare instances [6, 7], and efficient decompositions are occasionally possible [8], computing safety-ensuring controllers is intractable for many systems of interest....
[...]
TL;DR: In this paper, the authors considered the case of a team of pursuers and evaders, and provided a foundation to formally analyze complex and high-dimensional conflicts between teams by means of differential game theory, where the players' optimal strategies require codesign of cooperative optimal assignments and optimal guidance laws.
Abstract: In this article an $N$ -pursuer versus $M$ -evader team conflict is studied. This article extends classical differential game theory to simultaneously address weapon assignments and multiplayer pursuit-evasion scenarios. Saddle-point strategies that provide guaranteed performance for each team regardless of the actual strategies implemented by the opponent are devised. The players’ optimal strategies require the codesign of cooperative optimal assignments and optimal guidance laws. A representative measure of performance is employed and the Value function of the attendant game is obtained. It is shown that the Value function is continuously differentiable and that it satisfies the Hamilton–Jacobi–Isaacs equation—the curse of dimensionality is overcome and the optimal strategies are obtained. The cases of $N=M$ and $N>M$ are considered. In the latter case, cooperative guidance strategies are also developed in order for the pursuers to exploit their numerical advantage. This article provides a foundation to formally analyze complex and high-dimensional conflicts between teams of $N$ pursuers and $M$ evaders by means of differential game theory.
48 citations
TL;DR: This work considers the problem of steering a single evader to a target location, while avoiding capture by multiple pursuers, and proposes a feasible control strategy for the evader, against a group of pursuers that adopts a semi-cooperative strategy.
Abstract: We address a planar multiagent pursuit–evasion game with a terminal constraint (reach-avoid game). Specifically, we consider the problem of steering a single evader to a target location, while avoiding capture by multiple pursuers. We propose a feasible control strategy for the evader, against a group of pursuers that adopts a semi-cooperative strategy. First, we characterize a partition of the game’s state-space, that allows us to determine the existence of a solution to the game based on the initial conditions of the players. Next, based on the time-derivative of an appropriately defined risk metric, we develop a nonlinear state feedback strategy for the evader which provides a feasible solution to the game. This control strategy involves switching between different control laws in different parts of the state-space. We demonstrate the efficacy of our proposed feedback control in terms of the evader’s performance, through numerical simulations. We also show that for the special case of the reach-avoid game with only one pursuer, the proposed control law is successful in guiding the evader to the target location from almost all initial conditions, and ensures that the evader will remain uncaptured.
34 citations
Cites background from "Design and Analysis of State-Feedba..."
...This class of games is a preferred tool to model situations where a team of agents must reach a target location while avoiding obstacles or defending a target from an offensive team of agents [16]–[20]....
[...]
01 Jul 2020
TL;DR: In this article, the authors present an organized introduction of pursuit-evasion differential games with an overview of recent advances in the area and present two representative pursuit evasion differential games: the two-cutters and fugitive ship differential game and the active target defense differential game.
Abstract: Pursuit and evasion conflicts represent challenging problems with important applications in aerospace and robotics. In pursuit-evasion problems, synthesis of intelligent actions must consider the adversary's potential strategies. Differential game theory provides an adequate framework to analyze possible outcomes of the conflict without assuming particular behaviors by the opponent. This article presents an organized introduction of pursuit-evasion differential games with an overview of recent advances in the area. First, a summary of the seminal work is outlined, highlighting important contributions. Next, more recent results are described by employing a classification based on the number of players: one-pursuer-one-evader, N-pursuers-one-evader, one-pursuer-M-evaders, and N-pursuer-M-evader games. In each scenario, a brief summary of the literature is presented. Finally, two representative pursuit-evasion differential games are studied in detail: the two-cutters and fugitive ship differential game and the active target defense differential game. These problems provide two important applications and, more importantly, they give great insight into the realization of cooperation between friendly agents in order to form a team and defeat the adversary.
34 citations
References
More filters
TL;DR: In this paper, the authors introduced the notion of "viscosity solutions" of scalar nonlinear first order partial differential equations and proved several new facts and reproved various known results in a simpler manner.
Abstract: : Recently M. G. Crandall and P. L. Lions introduced the notion of 'viscosity solutions' of scalar nonlinear first order partial differential equations. Viscosity solutions need not be differentiable anywhere and thus are not sensitive to the classical problem of the crossing of characteristics. The value of this concept is established by the fact that very general existence, uniqueness and continuous dependence results hold for viscosity solutions of many problems arising in fields of application. The notion of a 'viscosity solution' admits several equivalent formulations. Here we look more closely at two of these equivalent criteria and exhibit their virtues by both proving several new facts and reproving various known results in a simpler manner. Moreover, by forsaking technical generality we hereby provide a more congenial introduction to this subject than the original paper. (Author)
1,243 citations
"Design and Analysis of State-Feedba..." refers background in this paper
...For instance, on an equivocal singular surface the Value function is not differentiable so it does not satisfy the HJI PDE in the classical sense, but it does in the viscosity sense....
[...]
...(16) The Value function is C1 , it satisfies the HJI PDE, and is explicitly given by V (x) = α [ 1 4 ( (xA − xD )2 + (yA − yD )2 ) + ( x− 1 2 (xA + xD ) )2 + ( y − 1 2 (yA + yD ) )2]1/2 + √ (xT − x)2 + (yT − y)2 ....
[...]
...The Value function is C1 , it satisfies the HJI PDE, and is explicitly given by V (x) = α [ 1 4 ( (xA − xD )2 + (yA − yD )2 ) + ( x− 1 2 (xA + xD ) )2 + ( y − 1 2 (yA + yD ) )2]1/2 − √ (x− xT )2 + (y − yT )2 ....
[...]
...The possible nondifferentiability of the solution of the Hamilton–Jacobi–Isaacs partial differential equation (HJI PDE) is a concern in differential games and the viscosity solution of the HJI PDE [19], [20] provides a generalized solution concept....
[...]
...Therefore, the results in this paper represent an important contribution where, for a differential game with higher dimensional state space, a (closed-form) continuous and continuously differentiable Value function in the Target’s escape set is obtained which is the classical solution of the HJI PDE....
[...]
TL;DR: A new guidance problem with the impact time constraint is investigated, which can be applied to salvo attack of anti-ship missiles and the closed form solution based on the linear formulation is derived, suggesting an additional loop for adjusting the impactTime in addition to the traditional optimal guidance loop.
Abstract: In this paper, a new guidance problem with the impact time constraint is investigated, which can be applied to salvo attack of anti-ship missiles. The closed form solution based on the linear formulation is derived, suggesting an additional loop for adjusting the impact time in addition to the traditional optimal guidance loop. This solution is a combination of the well-known PNG law and the feedback of the impact time error, which is the difference between the impact time by PNG and the prescribed impact time. The new guidance law called ITCG (Impact-Time-Control Guidance) can be used to guide multiple missiles to hit a stationary target simultaneously at a desirable impact time. Nonlinear simulation of several engagement situations demonstrates the performance and feasibility of ITCG. In addition, the similarity of the closed form solution and APNG is investigated and the switching rule for practical implementation is discussed.
507 citations
"Design and Analysis of State-Feedba..." refers background in this paper
...For instance, the work in [9] describes the multimissile cooperative attacks on a single stationary target (ship) and in [10] for moving targets....
[...]
TL;DR: In this article, a new guidance law is proposed to control both impact time and impact angle for a flight vehicle's homing problem, which can be applied for an efficient salvo attack of antiship missiles or a cooperative mission of UAVs.
Abstract: This paper proposes a new guidance law to control both impact time and impact angle for a flight vehicle's homing problem, which can be applied for an efficient salvo attack of antiship missiles or a cooperative mission of unmanned aerial vehicles (UAVs). The proposed law can lead vehicles to home on a target at a designated impact time with a prescribed impact angle. It comprises a feedback loop and an additional control command, the first to achieve the desired impact angle with zero miss distance, and the second to control the impact time. Numerical simulations demonstrate the performance of the proposed law in the accuracy of impact angle and impact time
362 citations
"Design and Analysis of State-Feedba..." refers background in this paper
...For instance, the work in [9] describes the multimissile cooperative attacks on a single stationary target (ship) and in [10] for moving targets....
[...]
TL;DR: The main advantage of the approach proposed is that it can be applied to a general class of target-hitting continuous dynamic games with nonlinear dynamics, and has very good properties in terms of its numerical solution, since the value function and the Hamiltonian of the system are both continuous.
Abstract: A new framework for formulating reachability problems with competing inputs, nonlinear dynamics, and state constraints as optimal control problems is developed. Such reach-avoid problems arise in, among others, the study of safety problems in hybrid systems. Earlier approaches to reach-avoid computations are either restricted to linear systems, or face numerical difficulties due to possible discontinuities in the Hamiltonian of the optimal control problem. The main advantage of the approach proposed in this paper is that it can be applied to a general class of target-hitting continuous dynamic games with nonlinear dynamics, and has very good properties in terms of its numerical solution, since the value function and the Hamiltonian of the system are both continuous. The performance of the proposed method is demonstrated by applying it to a case study, which involves the target-hitting problem of an underactuated underwater vehicle in the presence of obstacles.
193 citations
"Design and Analysis of State-Feedba..." refers background in this paper
...[15] K. Margellos and J. Lygeros, “Hamilton–jacobi formulation for reach– avoid differential games,” IEEE Trans....
[...]
...Margellos and Lygeros [15] provided a game formulation to solve reach and avoid problems involving nonlinear systems....
[...]
TL;DR: A new continuous level-set approach for characterizing the optimal times and the backward-reachability sets is given, which leads to a characterization via a Hamilton-Jacobi equation, without assuming any controllability assumption.
Abstract: We consider a target problem for a nonlinear system under state constraints. We give a new continuous level-set approach for characterizing the optimal times and the backward-reachability sets. This approach leads to a characterization via a Hamilton-Jacobi equation, without assuming any controllability assumption. We also treat the case of time-dependent state constraints, as well as a target problem for a two-player game with state constraints. Our method gives a good framework for numerical approximations, and some numerical illustrations are included in the paper.
169 citations
"Design and Analysis of State-Feedba..." refers methods in this paper
...The nondifferentiability of solutions of the Hamilton–Jacobi equation in optimal control with state constraints is addressed in [21] and [22]....
[...]