Showing papers by "Efstathios Bakolas published in 2021"

Feedback Strategies for a Reach-Avoid Game With a Single Evader and Multiple Pursuers

Abstract: We address a planar multiagent pursuit–evasion game with a terminal constraint (reach-avoid game). Specifically, we consider the problem of steering a single evader to a target location, while avoiding capture by multiple pursuers. We propose a feasible control strategy for the evader, against a group of pursuers that adopts a semi-cooperative strategy. First, we characterize a partition of the game’s state-space, that allows us to determine the existence of a solution to the game based on the initial conditions of the players. Next, based on the time-derivative of an appropriately defined risk metric, we develop a nonlinear state feedback strategy for the evader which provides a feasible solution to the game. This control strategy involves switching between different control laws in different parts of the state-space. We demonstrate the efficacy of our proposed feedback control in terms of the evader’s performance, through numerical simulations. We also show that for the special case of the reach-avoid game with only one pursuer, the proposed control law is successful in guiding the evader to the target location from almost all initial conditions, and ensures that the evader will remain uncaptured.

Collision Avoidance for Unmanned Aerial Vehicles in the Presence of Static and Moving Obstacles

Abstract: This paper presents a new collision avoidance procedure for unmanned aerial vehicles in the presence of static and moving obstacles. The proposed procedure is based on a new form of local parametrized guidance vector fields, called collision avoidance vector fields, that produce smooth and intuitive maneuvers around obstacles. The maneuvers follow nominal collision-free paths which we refer to as streamlines of the collision avoidance vector fields. In the case of multiple obstacles, the proposed procedure determines a mixed vector field that blends the collision avoidance vector field of each obstacle and assumes its form whenever a pre-defined distance threshold is reached. Then, in accordance to the computed guidance vector fields, different collision avoidance controllers that generate collision-free maneuvers are developed. Furthermore, it is shown that any tracking controller with convergence guarantees can be used with the avoidance controllers to track the streamlines of the collision avoidance vector fields. Finally, numerical simulations demonstrate the efficacy of the proposed approach and its ability to avoid collisions with static and moving pop-up threats in three different practical scenarios.

Covariance Steering of Discrete-Time Stochastic Linear Systems Based on Wasserstein Distance Terminal Cost

Abstract: We consider a class of stochastic optimal control problems for discrete-time linear systems whose objective is the characterization of control policies that will steer the probability distribution of the terminal state of the system close to a desired Gaussian distribution. In our problem formulation, the closeness between the terminal state distribution and the desired (goal) distribution is measured in terms of the squared Wasserstein distance which is associated with a corresponding terminal cost term. We recast the stochastic optimal control problem as a finite-dimensional nonlinear program whose performance index can be expressed as the difference of two convex functions. This representation of the performance index allows us to find local minimizers of the original nonlinear program via the so-called convex-concave procedure [1] . Finally, we present non-trivial numerical simulations to demonstrate the efficacy of the proposed technique by comparing it with sequential quadratic programming methods in terms of computation time.

Distributed Covariance Steering with Consensus ADMM for Stochastic Multi-Agent Systems

Abstract: In this paper, we address the problem of steering a team of agents under stochastic linear dynamics to prescribed final state means and covariances. The agents operate in a common environment where inter-agent constraints may also be present. In order for our method to be scalable to large-scale systems and computationally efficient, we approach the problem in a distributed control framework using the Alternating Direction Method of Multipliers (ADMM). Each agent solves its own covariance steering problem in parallel, while additional copy variables for its closest neighbors are introduced to ensure that the inter-agent constraints will be satisfied. The inclusion of these additional variables creates a requirement for consensus between original and copy variables that involve the same agent. For this reason, we employ a variation of ADMM for consensus optimization. Simulation results on multi-vehicle systems under uncertainty with collision avoidance constraints illustrate the effectiveness of our algorithm. The substantially improved scalability of our distributed approach with respect to the number of agents is also demonstrated, in comparison with an equivalent centralized scheme.

Safe nonlinear control design for input constrained polynomial systems using sum-of-squares programming

Abstract: We study the feedback stabilisation problem for input-affine polynomial systems subject to polytopic input constraints. First, we characterise a subset of the state-space, which we refer to as the ...

Far-Field Minimum-Fuel Spacecraft Rendezvous using Koopman Operator and l 2 /l 1 Optimization

Abstract: We propose a method to compute approximate solutions to the minimum-fuel far-field rendezvous problem for thrust-vectoring spacecraft. When the distance between the active and the target spacecraft is significantly greater than the distance between the target spacecraft and the center of gravity of the planet, linearization-based approximations of the nonlinear rendezvous dynamics may not be sufficiently accurate. For this reason, control methods that rely on such linearizations may not be appropriate for far-field rendezvous. In this paper, we address the control design problem based on a nonlinear state space model. To overcome the well-known challenges of nonlinear control design, we utilize a Koopman operator based approach in which the nonlinear spacecraft rendezvous dynamics is lifted into a higher dimensional space over which the nonlinear dynamics can be approximated by a linear system which is more suitable for control design purposes than the original nonlinear model. An Iteratively Recursive Least Squares (IRLS) algorithm from compressive sensing is then used to solve the minimum fuel control problem based on the lifted linear system. Numerical simulations are performed to show the efficacy of the proposed Koopman operator based approach.

Estimation and Control of Fluid Flows Using Sparsity-Promoting Dynamic Mode Decomposition

Abstract: Control and estimation of fluid systems is a challenging problem that requires approximating high-dimensional, nonlinear dynamics with computationally tractable models. A number of techniques, such as proper orthogonal decomposition (POD) and dynamic mode decomposition (DMD) have been developed to derive such reduced-order models. In this letter, the problem of selecting the dynamically important modes of dynamic mode decomposition with control (DMDc) is addressed. Similar to sparsity-promoting DMD, the method described in this letter solves a convex optimization problem in order to determine the most important modes. The proposed algorithm produces sparse dynamical models for systems with inputs by solving a regularized least-squares problem that minimizes the reweighted $\mathcal {L}_{1}$ norm of the relative mode weights and can work even with snapshot data that are not sequential. In addition, the process of estimating the modeling errors and designing a Kalman filter for flow estimation from limited measurements is presented. The method is demonstrated in the feedback control and estimation of the unsteady wake past an inclined flat plate in a high-fidelity direct numerical simulation.

Workspace Partitioning and Topology Discovery Algorithms for Heterogeneous Multiagent Networks

Abstract: In this article, we consider a class of workspace partitioning problems that arise in the context of area coverage for spatially distributed heterogeneous multiagent networks. It is assumed that each agent has certain directions of motion or directions for sensing that are preferable to others. These preferences are measured by means of convex and anisotropic (direction-dependent) quadratic proximity metrics, which can be different for each agent. These proximity metrics induce Voronoi-like partitions of the network's workspace, whose cells may not always be convex (or even connected) sets but are necessarily contained in a priori known ellipsoids. The main contributions of this article are as follows: 1) a distributed algorithm for the computation of a Voronoi-like partition of the workspace of a heterogeneous multiagent network and 2) a systematic process to discover the network topology induced by the latter partition. The distributed implementation of the proposed algorithms is enabled by the utilization of a hypothetical agent which determines when the performance of each agent is acceptable. Numerical simulations that illustrate the efficacy of the proposed algorithms are also presented.

Model Predictive Control of Material Volumes with Application to Vortical Structures

Abstract: In this paper, a model predictive control algorithm for selectively steering material volumes in a boundary layer is proposed. Using direct numerical simulations of a laminar boundary layer with a ...

Greedy Decentralized Auction-based Task Allocation for Multi-Agent Systems.

Abstract: We propose a decentralized auction-based algorithm for the solution of dynamic task allocation problems for spatially distributed multi-agent systems. In our approach, each member of the multi-agent team is assigned to at most one task from a set of spatially distributed tasks, while several agents can be allocated to the same task. The task assignment is dynamic since it is updated at discrete time stages (iterations) to account for the current states of the agents as the latter move towards the tasks assigned to them at the previous stage. Our proposed methods can find applications in problems of resource allocation by intelligent machines such as the delivery of packages by a fleet of unmanned or semi-autonomous aerial vehicles. In our approach, the task allocation accounts for both the cost incurred by the agents for the completion of their assigned tasks (e.g., energy or fuel consumption) and the rewards earned for their completion (which may reflect, for instance, the agents' satisfaction). We propose a Greedy Coalition Auction Algorithm (GCAA) in which the agents possess bid vectors representing their best evaluations of the task utilities. The agents propose bids, deduce an allocation based on their bid vectors and update them after each iteration. The solution estimate of the proposed task allocation algorithm converges after a finite number of iterations which cannot exceed the number of agents. Finally, we use numerical simulations to illustrate the effectiveness of the proposed task allocation algorithm (in terms of performance and computation time) in several scenarios involving multiple agents and tasks distributed over a spatial 2D domain.

Covariance Control of Discrete-Time Gaussian Linear Systems Using Affine Disturbance Feedback Control Policies

Abstract: In this paper, we present a new control policy parametrization for the finite-horizon covariance steering problem for discrete-time Gaussian linear systems (DTGLS) which can reduce the latter stochastic optimal control problem to a tractable optimization problem. The covariance steering problem seeks for a feedback control policy that will steer the state covariance of a DTGLS to a desired positive definite matrix in finite time. We consider two different formulations of the covariance steering problem, one with hard terminal LMI constraints (hard-constrained covariance steering) and another one with soft terminal constraints in the form of a terminal cost which corresponds to the squared Wasserstein distance between the actual terminal state (Gaussian) distribution and the desired one (soft-constrained covariance steering). We propose a solution approach that relies on the affine disturbance feedback parametrization for both problem formulations. We show that this particular parametrization allows us to reduce the hard-constrained covariance steering problem into a semi-definite program (SDP) and the soft-constrained covariance steering problem into a difference of convex functions program(DCP). Finally, we show the advantages of our approach over other covariance steering algorithms in terms of computational complexity and computation time by means of theoretical analysis and numerical simulations.

Distributed Coverage Control of Multi-Agent Networks with Guaranteed Collision Avoidance in Cluttered Environments

Abstract: We propose a distributed control algorithm for a multi-agent network whose agents deploy over a cluttered region in accordance with a time-varying coverage density function while avoiding collisions with all obstacles they encounter. Our algorithm is built on a two-level characterization of the network. The first level treats the multi-agent network as a whole based on the distribution of the locations of its agents over the spatial domain. In the second level, the network is described in terms of the individual positions of its agents. The aim of the multi-agent network is to attain a spatial distribution that resembles that of a reference coverage density function (high-level problem) by means of local (microscopic) interactions of its agents (low-level problem). In addition, as the agents deploy, they must avoid collisions with all the obstacles in the region at all times. Our approach utilizes a modified version of Voronoi tessellations which are comprised of what we refer to as Obstacle-Aware Voronoi Cells (OAVC) in order to enable coverage control while ensuring obstacle avoidance. We consider two control problems. The first problem which we refer to as the high-level coverage control problem corresponds to an interpolation problem in the class of Gaussian mixtures (no collision avoidance requirement), which we solve analytically. The second problem which we refer to as the low-level coverage control problem corresponds to a distributed control problem (collision avoidance requirement is now enforced at all times) which is solved by utilizing Lloyd's algorithm together with the modified Voronoi tessellation (OAVC) and a time-varying coverage density function which corresponds to the solution of the high-level coverage control problem. Finally, simulation results for coverage in a cluttered environment are provided to demonstrate the efficacy of the proposed approach.

An efficient and direct method for trajectory optimization of robots constrained by contact kinematics and forces

Abstract: In this work, we propose a trajectory generation method for robotic systems with contact kinematics and force constraints based on optimal control and reachability analysis tools. Normally, the dynamics and constraints of a contact-constrained robot are nonlinear and coupled to each other. Instead of linearizing the model and constraints, we solve the optimal control problem directly to obtain feasible state trajectories and their corresponding control inputs. A tractable optimal control problem is formulated and subsequently addressed by dual approaches, which rely on sampling-based dynamic programming and rigorous reachability analysis tools. In particular, a sampling-based method together with a Partially Observable Markov Decision Process solution approach are used to break down the end-to-end trajectory generation problem by generating a sequence of subregions that the system’s trajectory will have to pass through to reach its final destination. The distinctive characteristic of the proposed trajectory optimization algorithm is its ability to handle the intricate contact constraints, coupled with the system dynamics, in a computationally efficient way. We validate our method using extensive numerical simulations with two legged robots.

Koopman Operator Based Modeling and Control of Rigid Body Motion Represented by Dual Quaternions.

Abstract: In this paper, we systematically derive a finite set of Koopman based observables to construct a lifted linear state space model that describes the rigid body dynamics based on the dual quaternion representation. In general, the Koopman operator is a linear infinite dimensional operator, which means that the derived linear state space model of the rigid body dynamics will be infinite-dimensional, which is not suitable for modeling and control design purposes. Recently, finite approximations of the operator computed by means of methods like the Extended Dynamic Mode Decomposition (EDMD) have shown promising results for different classes of problems. However, without using an appropriate set of observables in the EDMD approach, there can be no guarantees that the computed approximation of the nonlinear dynamics is sufficiently accurate. The major challenge in using the Koopman operator for constructing a linear state space model is the choice of observables. State-of-the-art methods in the field compute the approximations of the observables by using neural networks, standard radial basis functions (RBFs), polynomials or heuristic approximations of these functions. However, these observables might not providea sufficiently accurate approximation or representation of the dynamics. In contrast, we first show the pointwise convergence of the derived observable functions to zero, thereby allowing us to choose a finite set of these observables. Next, we use the derived observables in EDMD to compute the lifted linear state and input matrices for the rigid body dynamics. Finally, we show that an LQR type (linear) controller, which is designed based on the truncated linear state space model, can steer the rigid body to a desired state while its performance is commensurate with that of a nonlinear controller. The efficacy of our approach is demonstrated through numerical simulations.

Optimal Strategies for Guarding a Compact and Convex Target Area: A Differential Game Approach.

Abstract: We revisit the two-player planar target-defense game posed in [1], a special class of pursuit-evasion games in which the pursuer (or defender) strives to defend a stationary target area from the evader (or intruder) who desires to reach it, if possible, or approach it as close as possible In this paper, the target area is assumed to be a compact and convex set Unlike classical two-player pursuit-evasion games, this game involves two subgames: a capture game and an escape game In the capture game, where capture is assured, the evader attempts to minimize the distance between her final position and the target area whereas the pursuer tries to maximize the same distance In the escape game, where capture is not guaranteed, the evader attempts to maximize the distance between herself and the pursuer at the moment that she reaches the target for the first time Our solution approach is based on Isaacs classical method in differential games We first identify the barrier surface that demarcates the state space of the game into two subspaces, each of which corresponds to the two aforementioned subgames, by means of geometric arguments Thereafter, we derive the optimal strategies for the players in each subspace We show that, as long as the target area is compact and convex, the value of the game in each subspace is always continuously differentiable, and the proposed optimal strategies correspond to the unique saddle-point state-feedback strategies for the players We illustrate our proposed solutions by means of numerical simulations

Relay Pursuit of an Evader by a Heterogeneous Group of Pursuers using Potential Games

Abstract: In this paper, we propose a decentralized game-theoretic pursuit policy for a heterogeneous group of pursuers who individually attempt to, without any prescribed cooperative pursuit strategy, capture a single evader who strives to delay or avoid capture if possible. We assume that the pursuers are rational (self-interested) agents who are not necessarily connected via communication network. Our proposed pursuit policy is motivated from the semi-cooperative pursuit policy called relay pursuit [1] under which only the pursuer who can capture the evader faster than the others is active while the rest stay put. In contrast to the latter strategy, our proposed method does not rely on geometric tools. It relies instead on reducing the noncooperative pursuit-evasion game into a sequence of maximum weighted bipartite matching problems which seek to find the pursuer-evader assignments which will result in minimum time of capture. To find the optimal assignment in a decentralized manner, the graph matching problem at each time instant is formulated as a static potential game whose pure strategy Nash equilibria correspond to the optimal assignments. Such equilibria are found by iteratively executing a game-theoretic learning algorithm called Joint Strategy Fictitious Play (JSFP) under which every pursuer synchronously takes his best reply strategy (pursue or stay put), depending on the joint actions of other pursuers, until they reach a Nash equilibrium. We illustrate the performance of our method by means of extensive numerical simulations.

Covariance Steering of Discrete-Time Stochastic Linear Systems Based on Wasserstein Distance Terminal Cost

Abstract: We consider a class of stochastic optimal control problems for discrete-time linear systems whose objective is the characterization of control policies that will steer the probability distribution of the terminal state of the system close to a desired Gaussian distribution. In our problem formulation, the closeness between the terminal state distribution and the desired (goal) distribution is measured in terms of the squared Wasser-stein distance which is associated with a corresponding terminal cost term. We recast the stochastic optimal control problem as a finite-dimensional nonlinear program whose performance index can be expressed as the difference of two convex functions. This representation of the performance index allows us to find local minimizers of the original nonlinear program via the socalled convex-concave procedure [1]. Finally, we present nontrivial numerical simulations to demonstrate the efficacy of the proposed technique by comparing it with sequential quadratic programming methods in terms of computation time.

Constrained Covariance Steering Based Tube-MPPI

Abstract: In this paper, we present a new trajectory optimization algorithm for stochastic linear systems which combines Model Predictive Path Integral (MPPI) control with Constrained Covariance Steering (CSS) to achieve high performance with safety guarantees (robustness). Although MPPI can be used to solve complex nonlinear trajectory optimization problems, it may not always handle constraints effectively and its performance may degrade in the presence of unmodeled disturbances. By contrast, CCS can handle probabilistic state and / or input constraints (e.g., chance constraints) and also steer the state covariance of the system to a desired positive definite matrix (control of uncertainty) which both imply that CCS can provide robustness against stochastic disturbances. CCS, however, suffers from scalability issues and cannot handle complex cost functions in general. We argue that the combination of the two methods yields a class of trajectory optimization algorithms that can achieve high performance (a feature of MPPI) while ensuring safety with high probability (a feature of CCS). The efficacy of our algorithm is demonstrated in an obstacle avoidance problem and a circular track path generation problem.

Decentralized Game-Theoretic Control for Dynamic Task Allocation Problems for Multi-Agent Systems

Abstract: We propose a decentralized game-theoretic framework for dynamic task allocation problems for multi-agent systems. In our problem formulation, the agents' utilities depend on both the rewards and the costs associated with the successful completion of the tasks assigned to them. The rewards reflect how likely is for the agents to accomplish their assigned tasks whereas the costs reflect the effort needed to complete these tasks (this effort is determined by the solution of corresponding optimal control problems). The task allocation problem considered herein corresponds to a dynamic game whose solution depends on the states of the agents in contrast with classic static (or single-act) game formulations. We propose a greedy solution approach in which the agents negotiate with each other to find a mutually agreeable (or individually rational) task assignment profile based on evaluations of the task utilities that reflect their current states. We illustrate the main ideas of this work by means of extensive numerical simulations.

Guarding a Target Set from a Single Attacker in the Euclidean Space.

Abstract: This paper addresses a two-player target defense game in the $n$-dimensional Euclidean space where an attacker attempts to enter a closed convex target set while a defender strives to capture the attacker beforehand. We provide a complete and universal differential game-based solution which not only encompasses recent work associated with similar problems whose target sets have simple, low-dimensional geometric shapes, but can also address problems that involve nontrivial geometric shapes of high-dimensional target sets. The value functions of the game are derived in a semi-analytical form that includes a convex optimization problem. When the latter problem has a closed-form solution, one of the value functions is used to analytically construct the barrier surface that divides the state space of the game into the winning sets of players. For the case where the barrier surface has no analytical expression but the target set has a smooth boundary, the bijective map between the target boundary and the projection of the barrier surface is obtained. By using Hamilton-Jacobi-Isaacs equation, we verify that the proposed optimal state feedback strategies always constitute the game's unique saddle point whether or not the optimization problem has a closed-form solution. We illustrate our solutions via numerical simulations.

Minimum-fuel Spacecraft Rendezvous based on Sparsity Promoting Optimization.

Abstract: In this paper, we consider the classical spacecraft rendezvous problem in which the so-called active spacecraft has to approach the target spacecraft which is moving in an elliptical orbit around a planet by using the minimum possible amount of fuel. Instead of using standard convex optimization tools which can be computationally expensive, we use modified versions of the Iteratively Reweighted Least Squares (IRLS) algorithm from compressive sensing to compute sparse optimal control sequences which minimize the fuel consumption for both thrust vectoring and orthogonal vectoring (active) spacecraft. Numerical simulations are performed to verify the efficacy of our approach.

On the Convexity of Discrete Time Covariance Steering in Stochastic Linear Systems with Wasserstein Terminal Cost.

Abstract: In this work, we analyze the properties of the solution to the covariance steering problem for discrete time Gaussian linear systems with a squared Wasserstein distance terminal cost. In our previous work, we have shown that by utilizing the state feedback control policy parametrization, this stochastic optimal control problem can be associated with a difference of convex functions program. Here, we revisit the same covariance control problem but this time we focus on the analysis of the problem. Specifically, we establish the existence of solutions to the optimization problem and derive the first and second order conditions for optimality. We provide analytic expressions for the gradient and the Hessian of the performance index by utilizing specialized tools from matrix calculus. Subsequently, we prove that the optimization problem always admits a global minimizer, and finally, we provide a sufficient condition for the performance index to be a strictly convex function (under the latter condition, the problem admits a unique global minimizer). In particular, we show that when the terminal state covariance is upper bounded, with respect to the L\"{o}wner partial order, by the covariance matrix of the desired terminal normal distribution, then our problem admits a unique global minimizing state feedback gain. The results of this paper set the stage for the development of specialized control design tools that exploit the structure of the solution to the covariance steering problem with a squared Wasserstein distance terminal cost.

Collision Avoidance Using Spherical Harmonics.

Abstract: In this paper, we propose a novel optimization-based trajectory planner that utilizes spherical harmonics to estimate the collision-free solution space around an agent. The space is estimated using a constrained over-determined least-squares estimator to determine the parameters that define a spherical harmonic approximation at a given time step. Since spherical harmonics produce star-convex shapes, the planner can consider all paths that are in line-of-sight for the agent within a given radius. This contrasts with other state-of-the-art planners that generate trajectories by estimating obstacle boundaries with rough approximations and using heuristic rules to prune a solution space into one that can be easily explored. Those methods cause the trajectory planner to be overly conservative in environments where an agent must get close to obstacles to accomplish a goal. Our method is shown to perform on-par with other path planners and surpass these planners in certain environments. It generates feasible trajectories while still running in real-time and guaranteeing safety when a valid solution exists.

Koopman Operator Based Modeling for Quadrotor Control on $SE(3)$

Abstract: In this paper, we propose a Koopman operator based approach to describe the nonlinear dynamics of a quadrotor on SE(3) in terms of an infinite-dimensional linear system which evolves in the space of observable functions (lifted space) and which is more appropriate for control design purposes. The major challenge when using the Koopman operator is the characterization of a set of observable functions that can span the lifted space. Recent methods either use tools from machine learning to learn the observable functions or guess a suitable set of observables that best describes the nonlinear dynamics. Instead of guessing or learning the observables, in this work we derive them in a systematic way for the quadrotor dynamics on SE(3). In addition, we prove that the proposed sequence of observable functions converges pointwise to the zero function, which allows us to select only a finite set of observable functions to form (an approximation of) the lifted space. Our theoretical analysis is also confirmed by numerical simulations which demonstrate that by increasing the dimension of the lifted space, the derived linear state space model can approximate the nonlinear quadrotor dynamics more accurately.

Feedback Strategies for Hypersonic Pursuit of a Ground Evader.

Abstract: In this paper, we present a game-theoretic feedback terminal guidance law for an autonomous, unpowered hypersonic pursuit vehicle that seeks to intercept an evading ground target whose motion is constrained along an axis. We formulate this problem as a pursuit-evasion game whose saddle point solution is in general difficult to compute onboard the hypersonic vehicle due to its highly nonlinear dynamics. To overcome this difficulty, we linearize the nonlinear hypersonic dynamics around a reference trajectory and subsequently utilize feedback control design techniques from Linear Quadratic Differential Games (LQDGs). In our proposed guidance algorithm, the hypersonic vehicle computes its open-loop optimal state and input trajectories off-line and prior to the commencement of the game. These trajectories are then used to linearize the nonlinear equations of hypersonic motion. Subsequently, using this linearized system model, we formulate an auxiliary two-player zero-sum LQDG which is effective in the neighborhood of the given reference trajectory and derive its feedback saddle point strategy that allows the hypersonic vehicle to modify its trajectory online in accordance with the changes/dispersion to the target's position due to its evasive maneuvers. We provide numerical simulations to verify the performance of our proposed guidance law.