Showing papers in "Siam Journal on Control and Optimization in 2009"

Controllability of Multi-Agent Systems from a Graph-Theoretic Perspective

Abstract: In this work, we consider the controlled agreement problem for multi-agent networks, where a collection of agents take on leader roles while the remaining agents execute local, consensus-like protocols. Our aim is to identify reflections of graph-theoretic notions on system-theoretic properties of such systems. In particular, we show how the symmetry structure of the network, characterized in terms of its automorphism group, directly relates to the controllability of the corresponding multi-agent system. Moreover, we introduce network equitable partitions as a means by which such controllability characterizations can be extended to the multileader setting.

Convergence Speed in Distributed Consensus and Averaging

Abstract: We study the convergence speed of distributed iterative algorithms for the consensus and averaging problems, with emphasis on the latter. We first consider the case of a fixed communication topology. We show that a simple adaptation of a consensus algorithm leads to an averaging algorithm. We prove lower bounds on the worst-case convergence time for various classes of linear, time-invariant, distributed consensus methods, and provide an algorithm that essentially matches those lower bounds. We then consider the case of a time-varying topology, and provide a polynomial-time averaging algorithm.

Coordination and Consensus of Networked Agents with Noisy Measurements: Stochastic Algorithms and Asymptotic Behavior

Abstract: This paper considers the coordination and consensus of networked agents where each agent has noisy measurements of its neighbors' states. For consensus seeking, we propose stochastic approximation-type algorithms with a decreasing step size, and introduce the notions of mean square and strong consensus. Although the decreasing step size reduces the detrimental effect of the noise, it also reduces the ability of the algorithm to drive the individual states towards each other. The key technique is to ensure a trade-off for the decreasing rate of the step size. By following this strategy, we first develop a stochastic double array analysis in a two-agent model, which leads to both mean square and strong consensus, and extend the analysis to a class of well-studied symmetric models. Subsequently, we consider a general network topology, and introduce stochastic Lyapunov functions together with the so-called direction of invariance to establish mean square consensus. Finally, we apply the stochastic Lyapunov analysis to a leader following scenario.

Large-Population LQG Games Involving a Major Player: The Nash Certainty Equivalence Principle

Abstract: We consider linear-quadratic-Gaussian (LQG) games with a major player and a large number of minor players. The major player has a significant influence on others. The minor players individually have negligible impact, but they collectively contribute mean field coupling terms in the individual dynamics and costs. To overcome the dimensionality difficulty and obtain decentralized strategies, the so-called Nash certainty equivalence methodology is applied. The control synthesis is preceded by a state space augmentation via a set of aggregate quantities giving the mean field approximation. Subsequently, within the population limit the LQG game is decomposed into a family of limiting two-player games as each is locally seen by a representative minor player. Next, when solving these limiting two-player games, we impose certain interaction consistency conditions such that the aggregate quantities initially assumed coincide with the ones replicated by the closed loop of a large number of minor players. This procedure leads to a set of decentralized strategies for the original LQG game, which is an $\varepsilon$-Nash equilibrium.

Consensus Optimization on Manifolds

Abstract: The present paper considers distributed consensus algorithms that involve $N$ agents evolving on a connected compact homogeneous manifold. The agents track no external reference and communicate their relative state according to a communication graph. The consensus problem is formulated in terms of the extrema of a cost function. This leads to efficient gradient algorithms to synchronize (i.e., maximizing the consensus) or balance (i.e., minimizing the consensus) the agents; a convenient adaptation of the gradient algorithms is used when the communication graph is directed and time-varying. The cost function is linked to a specific centroid definition on manifolds, introduced here as the induced arithmetic mean, that is easily computable in closed form and may be of independent interest for a number of manifolds. The special orthogonal group $SO(n)$ and the Grassmann manifold $\text{{\it Grass\/}}(p,n)$ are treated as original examples. A link is also drawn with the many existing results on the circle.

Payoff-Based Dynamics for Multiplayer Weakly Acyclic Games

Abstract: We consider repeated multiplayer games in which players repeatedly and simultaneously choose strategies from a finite set of available strategies according to some strategy adjustment process. We focus on the specific class of weakly acyclic games, which is particularly relevant for multiagent cooperative control problems. A strategy adjustment process determines how players select their strategies at any stage as a function of the information gathered over previous stages. Of particular interest are “payoff-based” processes in which, at any stage, players know only their own actions and (noise corrupted) payoffs from previous stages. In particular, players do not know the actions taken by other players and do not know the structural form of payoff functions. We introduce three different payoff-based processes for increasingly general scenarios and prove that, after a sufficiently large number of stages, player actions constitute a Nash equilibrium at any stage with arbitrarily high probability. We also show how to modify player utility functions through tolls and incentives in so-called congestion games, a special class of weakly acyclic games, to guarantee that a centralized objective can be realized as a Nash equilibrium. We illustrate the methods with a simulation of distributed routing over a network.

Coordinated Path-Following in the Presence of Communication Losses and Time Delays

Abstract: This paper addresses the problem of steering a group of vehicles along given spatial paths while holding a desired time-varying geometrical formation pattern. The solution to this problem, henceforth referred to as the coordinated path-following (CPF) problem, unfolds in two basic steps. First, a path-following (PF) control law is designed to drive each vehicle to its assigned path, with a nominal speed profile that may be path dependent. This is done by making each vehicle approach a virtual target that moves along the path according to a conveniently defined dynamic law. In the second step, the speeds of the virtual targets (also called coordination states) are adjusted about their nominal values so as to synchronize their positions and achieve, indirectly, vehicle coordination. In the problem formulation, it is explicitly considered that each vehicle transmits its coordination state to a subset of the other vehicles only, as determined by the communications topology adopted. It is shown that the system that is obtained by putting together the PF and coordination subsystems can be naturally viewed as either the feedback or the cascade connection of the latter two. Using this fact and recent results from nonlinear systems and graph theory, conditions are derived under which the PF and the coordination errors are driven to a neighborhood of zero in the presence of communication losses and time delays. Two different situations are considered. The first captures the case where the communication graph is alternately connected and disconnected (brief connectivity losses). The second reflects an operational scenario where the union of the communication graphs over uniform intervals of time remains connected (uniformly connected in mean). To better root the paper in a nontrivial design example, a CPF algorithm is derived for multiple underactuated autonomous underwater vehicles (AUVs). Simulation results are presented and discussed.

Network Synthesis of Linear Dynamical Quantum Stochastic Systems

Abstract: The purpose of this paper is to develop a synthesis theory for linear dynamical quantum stochastic systems that are encountered in linear quantum optics and in phenomenological models of linear quantum circuits. In particular, such a theory will enable the systematic realization of coherent/fully quantum linear stochastic controllers for quantum control, amongst other potential applications. We show how general linear dynamical quantum stochastic systems can be constructed by assembling an appropriate interconnection of one degree of freedom open quantum harmonic oscillators and, in the quantum optics setting, discuss how such a network of oscillators can be approximately synthesized or implemented in a systematic way from some linear and nonlinear quantum optical elements. An example is also provided to illustrate the theory.

Analysis and Design of Unconstrained Nonlinear MPC Schemes for Finite and Infinite Dimensional Systems

Abstract: We present a technique for computing stability and performance bounds for unconstrained nonlinear model predictive control (MPC) schemes. The technique relies on controllability properties of the system under consideration, and the computation can be formulated as an optimization problem whose complexity is independent of the state space dimension. Based on the insight obtained from the numerical solution of this problem, we derive design guidelines for nonlinear MPC schemes which guarantee stability of the closed loop for small optimization horizons. These guidelines are illustrated by a finite and an infinite dimensional example.

Control of Minimally Persistent Formations in the Plane

Abstract: This paper studies the problem of controlling the shape of a formation of point agents in the plane. A model is considered where the distance between certain agent pairs is maintained by one of the agents making up the pair; if enough appropriately chosen distances are maintained, with the number growing linearly with the number of agents, then the shape of the formation will be maintained. The detailed question examined in the paper is how one may construct decentralized nonlinear control laws to be operated at each agent that will restore the shape of the formation in the presence of small distortions from the nominal shape. Using the theory of rigid and persistent graphs, the question is answered. As it turns out, a certain submatrix of a matrix known as the rigidity matrix can be proved to have nonzero leading principal minors, which allows the determination of a stabilizing control law.

Average Consensus with Packet Drop Communication

Abstract: Average consensus consists in the problem of determining the average of some quantities by means of a distributed algorithm. It is a simple instance of problems arising when designing estimation algorithms operating on data produced by sensor networks. Simple solutions based on linear estimation algorithms have already been proposed in the literature and their performance has been analyzed in detail. If the communication links which allow the data exchange between the sensors have some loss, then the estimation performance will degrade. In this contribution the performance degradation due to this data loss is evaluated.

Symbolic Models for Nonlinear Control Systems: Alternating Approximate Bisimulations

Abstract: Symbolic models are abstract descriptions of continuous systems in which symbols represent aggregates of continuous states In the last few years there has been a growing interest in the use of symbolic models as a tool for mitigating complexity in control design In fact, symbolic models enable the use of well-known algorithms in the context of supervisory control and algorithmic game theory for controller synthesis Since the 1990s many researchers faced the problem of identifying classes of dynamical and control systems that admit symbolic models In this paper we make further progress along this research line by focusing on control systems affected by disturbances Our main contribution is to show that incrementally globally asymptotically stable nonlinear control systems with disturbances admit symbolic models

A Finite Horizon Optimal Multiple Switching Problem

Abstract: -1We consider the problem of optimal multiple switching in a finite horizon when the state of the system, including the switching costs, is a general adapted stochastic process. The problem is formulated as an extended impulse control problem and solved using probabilistic tools such as the Snell envelope of processes and reflected backward stochastic differential equations. Finally, when the state of the system is a Markov process, we show that the associated vector of value functions provides a viscosity solution to a system of variational inequalities with interconnected obstacles.

Maintaining Limited-Range Connectivity Among Second-Order Agents

Abstract: In this paper we consider ad-hoc networks of robotic agents with double integrator dynamics. For such networks, the connectivity maintenance problems are as follows: (i) Do there exist control inputs for each agent to maintain network connectivity, and (ii) given desired controls for each agent, can we compute the closest connectivity-maintaining controls in a distributed fashion? The proposed solution is based on three contributions. First, we define and characterize admissible sets for double integrators to remain inside disks. Second, we establish an existence theorem for the connectivity maintenance problem by introducing a novel state-dependent graph, called the double-integrator disk graph. Specifically, we show that one can always maintain connectivity by maintaining a spanning tree of this new graph, but one will not always maintain connectivity of a particular agent pair that happens to be connected at one instant of time. Finally, we design a distributed “flow-control” algorithm for distributed computation of connectivity-maintaining controls.

Invariance Entropy for Control Systems

Abstract: For continuous time control systems, this paper introduces invariance entropy as a measure for the amount of information necessary to achieve invariance of weakly invariant compact subsets of the state space. Upper and lower bounds are derived; in particular, finiteness is proven. For linear control systems with compact control range, the invariance entropy is given by the sum of the real parts of the unstable eigenvalues of the uncontrolled system. A characterization via covers and corresponding feedbacks is provided.

Approximating Submodular Functions Everywhere

Abstract: Submodular functions are a key concept in combinatorial optimization. Algorithms that involve submodular functions usually assume that they are given by a (value) oracle. Many interesting problems involving submodular functions can be solved using only polynomially many queries to the oracle, e.g., exact minimization or approximate maximization.In this paper, we consider the problem of approximating a non-negative, monotone, submodular function f on a ground set of size n everywhere, after only poly(n) oracle queries. Our main result is a deterministic algorithm that makes poly(n) oracle queries and derives a function f such that, for every set S, f(S) approximates f(S) within a factor α(n), where α(n) = √n+1 for rank functions of matroids and α(n), = O(√n log n) for general monotone submodular functions. Our result is based on approximately finding a maximum volume inscribed ellipsoid in a symmetrized polymatroid, and the analysis involves various properties of submodular functions and polymatroids.Our algorithm is tight up to logarithmic factors. Indeed, we show that no algorithm can achieve a factor better than Ω(√n/log n), even for rank functions of a matroid.

A Parallel Splitting Method for Coupled Monotone Inclusions

Abstract: A parallel splitting method is proposed for solving systems of coupled monotone inclusions in Hilbert spaces, and its convergence is established under the assumption that solutions exist. Unlike existing alternating algorithms, which are limited to two variables and linear coupling, our parallel method can handle an arbitrary number of variables as well as nonlinear coupling schemes. The breadth and flexibility of the proposed framework is illustrated through applications in the areas of evolution inclusions, variational problems, best approximation, and network flows.

Stochastic Target Problems with Controlled Loss

Abstract: We consider the problem of finding the minimal initial data of a controlled process which guarantees to reach a controlled target with a given probability of success or, more generally, with a given level of expected loss. By suitably increasing the state space and the controls, we show that this problem can be converted into a stochastic target problem, i.e., finding the minimal initial data of a controlled process which guarantees to reach a controlled target with probability one. Unlike in the existing literature on stochastic target problems, our increased controls are valued in an unbounded set. In this paper, we provide a new derivation of the dynamic programming equation for general stochastic target problems with unbounded controls, together with the appropriate boundary conditions. These results are applied to the problem of quantile hedging in financial mathematics and are shown to recover the explicit solution of Follmer and Leukert [Finance Stoch., 3 (1999), pp. 251-273].

Dynamic Output Feedback Control of Discrete-Time Markov Jump Linear Systems through Linear Matrix Inequalities

Abstract: This paper addresses the ${\cal H}_2$ and ${\cal H}_\infty$ dynamic output feedback control design problems of discrete-time Markov jump linear systems. Under the mode-dependent assumption, which means that the Markov parameters are available for feedback, the main contribution is the complete characterization of all full order proper Markov jump linear controllers such that the ${\cal H}_2$ or ${\cal H}_\infty$ norm of the closed loop system remains bounded by a given prespecified level, yielding the global solution to the corresponding mode-dependent optimal control design problem, expressed in terms of pure linear matrix inequalities. Some academic examples are solved for illustration and comparison. As a more consequent practical application, the networked control of a vehicle platoon using measurement signals transmitted in a Markov channel, as initially proposed in [P. Seiler and R. Sengupta, IEEE Trans. Automat. Control, 50 (2005), pp. 356-364], is considered.

Maximum Principles for Optimal Control of Forward-Backward Stochastic Differential Equations with Jumps

Abstract: We present various versions of the maximum principle for optimal control of forward-backward stochastic differential equations (SDE) with jumps. Our study is motivated by risk minimization via $g$-expectations. We first prove a general sufficient maximum principle for optimal control with partial information of a stochastic system consisting of a forward and a backward SDE driven by Levy processes. We then present a Malliavin calculus approach which allows us to handle non-Markovian systems. Finally, we give examples of applications.

Consensus Problems with Distributed Delays, with Application to Traffic Flow Models

Abstract: This paper focuses on consensus problems for a class of linear systems with distributed delay that are encountered in modeling traffic flow dynamics. In the application problems the distributed delay, whose kernel is a $\gamma$-distribution with a gap, represents the human drivers' behavior in the average. The aim of the paper is to give a characterization of the regions in the corresponding delay parameter space, where a consensus is reached for all initial conditions. The structure and properties of the system are fully exploited, which leads to explicit and computationally tractable expressions. As a by-product a stability theory for distributed delay systems with a $\gamma$-distribution kernel is developed. Also explicit expressions for the consensus function(al) of time-delay systems with constant and distributed delays that solve a consensus problem are provided. Several illustrative examples complete the presentation.

Linear Programming Approach to Deterministic Infinite Horizon Optimal Control Problems with Discounting

Abstract: We investigate relationships between the deterministic infinite time horizon optimal control problem with discounting, in which the state trajectories remain in a given compact set $Y$, and a certain infinite dimensional linear programming (IDLP) problem. We introduce the problem dual with respect to this IDLP problem and obtain some duality results. We construct necessary and sufficient optimality conditions for the optimal control problem under consideration, and we give a characterization of the viability kernel of $Y$. We also indicate how one can use finite dimensional approximations of the IDLP problem and its dual for construction of near optimal feedback controls. The construction is illustrated with a numerical example.

General Projective Splitting Methods for Sums of Maximal Monotone Operators

Abstract: We describe a general projective framework for finding a zero of the sum of $n$ maximal monotone operators over a real Hilbert space Unlike prior methods for this problem, we neither assume $n=2$ nor first reduce the problem to the case $n=2$ Our analysis defines a closed convex extended solution set for which we can construct a separating hyperplane by individually evaluating the resolvent of each operator At the cost of a single, computationally simple projection step, this framework gives rise to a family of splitting methods of unprecedented flexibility: numerous parameters, including the proximal stepsize, may vary by iteration and by operator The order of operator evaluation may vary by iteration and may be either serial or parallel The analysis essentially generalizes our prior results for the case $n=2$ We also include a relative error criterion for approximately evaluating resolvents, which was not present in our earlier work

Optimal Stochastic Control and Carbon Price Formation

Abstract: To meet the targets of the Kyoto Protocol, the European Union established the European Emission Trading Scheme, a mandatory market for carbon emission allowances. This regulatory framework has introduced a market for emission allowances and created a variety of emission-related financial instruments. In this work, we show that the economic mechanism of carbon allowance price formation can be formulated in the framework of competitive stochastic equilibrium models, and we show that its solution reduces to an optimal stochastic control problem. Using this mathematical setup, we identify the main allowance price drivers and show how stochastic control can be used to treat quantitative problems in carbon price risk management.

Estimation of Spatially Distributed Processes Using Mobile Spatially Distributed Sensor Network

Abstract: The problem of estimating a spatially distributed process described by a partial differential equation (PDE), whose observations are contaminated by a zero mean Gaussian noise, is considered in this work. The basic premise of this work is that a set of mobile sensors achieve better estimation performance than a set of immobile sensors. To enhance the performance of the state estimator, a network of sensors that are capable of moving within the spatial domain is utilized. Specifically, such an estimation process is achieved by using a set of spatially distributed mobile sensors. The objective is to provide mobile sensor control policies that aim to improve the state estimate. The metric for such an estimate improvement is taken to be the expected state estimation error. Using different spatial norms, two guidance policies are proposed. The current approach capitalizes on the efficient filter gain design in order to avoid intense computational requirements resulting from the solution to filter Riccati equations. Simulation studies implementing and comparing the two proposed control policies are provided.

Quantized Coordination Algorithms for Rendezvous and Deployment

Abstract: In this paper we study motion coordination problems for groups of robots that exchange information through a rate-constrained communication network. For one-dimensional rendezvous and deployment problems with communication path graphs, we propose an integrated control and communication scheme combining a logarithmic coder/decoder with linear coordination algorithms. We show that the closed-loop performance is comparable to that achievable in the quantization-free model: the time complexity is, asymptotically on the size of the network, unchanged, and the exponential convergence factor degrades smoothly as the quantization accuracy becomes coarser.

Finite Element Approximation of Dirichlet Boundary Control for Elliptic PDEs on Two- and Three-Dimensional Curved Domains

Abstract: We consider the variational discretization of elliptic Dirichlet optimal control problems with constraints on the control. The underlying state equation, which is considered on smooth two- and three-dimensional domains, is discretized by linear finite elements taking into account domain approximation. The control variable is not discretized. We obtain optimal error bounds for the optimal control in two and three space dimensions and prove a superconvergence result in two dimensions, provided that the underlying mesh satisfies some additional condition. We confirm our analytical findings by numerical experiments.

Null Controllability for Forward and Backward Stochastic Parabolic Equations

Abstract: This paper is concerned with the null controllability for general forward and backward linear stochastic parabolic equations. To develop the duality argument, we establish observability estimates for linear backward and forward stochastic parabolic equations with general coefficients, by means of a global Carleman estimate. Our Carleman inequality (Theorem 6.1) and observability estimate (Theorem 2.3) for backward stochastic parabolic equations are new in their forms. By adding a control variable to act on the white noise, we give a partial solution to the null controllability of forward stochastic heat equations, which was regarded as a challenging topic (see pages 99 and 108-110 in [Barbu, Rascanu, and Tessitore, Appl. Math. Optim., 47 (2003), pp. 97-120]).

Optimal Control in Networks of Pipes and Canals

Abstract: This paper deals with the optimal control of systems governed by nonlinear systems of conservation laws at junctions. The applications considered range from gas compressors in pipelines to open channels management. The existence of an optimal control is proved. From the analytical point of view, these results are based on the well posedness of a suitable initial boundary value problem and on techniques for quasidifferential equations in a metric space.