Showing papers in "Siam Journal on Control and Optimization in 2012"

Synchronization and Transient Stability in Power Networks and Nonuniform Kuramoto Oscillators

Abstract: Motivated by recent interest for multiagent systems and smart grid architectures, we discuss the synchronization problem for the network-reduced model of a power system with nontrivial transfer conductances. Our key insight is to exploit the relationship between the power network model and a first-order model of coupled oscillators. Assuming overdamped generators (possibly due to local excitation controllers), a singular perturbation analysis shows the equivalence between the classic swing equations and a nonuniform Kuramoto model. Here, nonuniform Kuramoto oscillators are characterized by multiple time constants, nonhomogeneous coupling, and nonuniform phase shifts. Extending methods from transient stability, synchronization theory, and consensus protocols, we establish sufficient conditions for synchronization of nonuniform Kuramoto oscillators. These conditions reduce to necessary and sufficient tests for the standard Kuramoto model. Combining our singular perturbation and Kuramoto analyses, we derive ...

Time-Inconsistent Stochastic Linear--Quadratic Control

Abstract: In this paper, we formulate a general time-inconsistent stochastic linear--quadratic (LQ) control problem. The time-inconsistency arises from the presence of a quadratic term of the expected state as well as a state-dependent term in the objective functional. We define an equilibrium, instead of optimal, solution within the class of open-loop controls, and derive a sufficient condition for equilibrium controls via a flow of forward--backward stochastic differential equations. When the state is one dimensional and the coefficients in the problem are all deterministic, we find an explicit equilibrium control. As an application, we then consider a mean--variance portfolio selection model in a complete financial market where the risk-free rate is a deterministic function of time but all the other market parameters are possibly stochastic processes. Applying the general sufficient condition, we obtain explicit equilibrium strategies when the risk premium is both deterministic and stochastic.

Mean Field Games: Numerical Methods for the Planning Problem

Abstract: Mean field games describe the asymptotic behavior of differential games in which the number of players tends to $+\infty$. Here we focus on the optimal planning problem, i.e., the problem in which the positions of a very large number of identical rational agents, with a common value function, evolve from a given initial spatial density to a desired target density at the final horizon time. We propose a finite difference semi-implicit scheme for the optimal planning problem, which has an optimal control formulation. The latter leads to existence and uniqueness of the discrete control problem. We also study a penalized version of the semi-implicit scheme. For solving the resulting system of equations, we propose a strategy based on Newton iterations. We describe some numerical experiments.

Opinion Dynamics in Heterogeneous Networks: Convergence Conjectures and Theorems

Abstract: Recently, significant attention has been dedicated to the models of opinion dynamics in which opinions are described by real numbers and agents update their opinions synchronously by averaging their neighbors' opinions. The neighbors of each agent can be defined as either (1) those agents whose opinions are in its “confidence range” or (2) those agents whose “influence range” contain the agent's opinion. The former definition is employed in Hegselmann and Krause's bounded confidence model, and the latter is novel here. As the confidence and influence ranges are distinct for each agent, the heterogeneous state-dependent interconnection topology leads to a poorly-understood complex dynamic behavior. In both models, we classify the agents via their interconnection topology and, accordingly, compute the equilibria of the system. Then, we define a positive invariant set centered at each equilibrium opinion vector. We show that if a trajectory enters one such set, then it converges to a steady state with consta...

Stability and Resolution Analysis for a Topological Derivative Based Imaging Functional

Abstract: The aim of this paper is to study a topological derivative based anomaly detection algorithm. We compare its performance with other imaging approaches such as MUltiple SIgnal Classification, backpropagation, and Kirchhoff migration. We also investigate its stability with respect to medium and measurement noises as well as its resolution. A simple postprocessing of the data set is introduced and shown to be essential in order to obtain an efficient topological based imaging functional, both in terms of resolution and stability.

Approximation of Elliptic Control Problems in Measure Spaces with Sparse Solutions

Abstract: Optimal control problems in measure spaces governed by elliptic equations are considered for distributed and Neumann boundary control, which are known to promote sparse solutions. Optimality conditions are derived and some of the structural properties of their solutions, in particular sparsity, are discussed. A framework for their approximation is proposed which is efficient for numerical computations and for which we prove convergence and provide error estimates.

Mean Field Games for Large-Population Multiagent Systems with Markov Jump Parameters

Abstract: In this paper, distributed games for large-population multiagent systems with random time-varying parameters are investigated, where the agents are coupled via their individual costs and the structure parameters are a family of independent Markov chains with identical generators. The cost function of each agent is a long-run average tracking-type functional with an unknown mean field coupling nonlinear term as “reference signal.” To reduce the computational complexity, the mean field approach is applied to construct distributed strategies. The population statistics effect (PSE) is used to approximate the average effect of all the agents, and the distributed strategies are given through solving a Markov jump tracking problem. Here the PSE is a deterministic quantity and can be obtained by solving the Stackelberg equilibrium of an auxiliary two-player game. It is shown that the closed-loop system is uniformly stable, and the distributed strategies are asymptotically optimal in the sense of Nash equilibrium,...

Linear-Quadratic-Gaussian Mixed Games with Continuum-Parametrized Minor Players

Abstract: We consider a mean field linear-quadratic-Gaussian game with a major player and a large number of minor players parametrized by a continuum set. The mean field generated by the minor players is approximated by a random process depending only on the initial state and the Brownian motion of the major player, and this leads to two limiting optimal control problems with random coefficients, which are solved subject to a consistency requirement on the mean field approximation. The set of decentralized strategies constructed from the limiting control problems has an $\varepsilon$-Nash equilibrium property when applied to the large but finite population model.

Directional Sparsity in Optimal Control of Partial Differential Equations

Abstract: We study optimal control problems in which controls with certain sparsity patterns are preferred. For time-dependent problems the approach can be used to find locations for control devices that allow controlling the system in an optimal way over the entire time interval. The approach uses a nondifferentiable cost functional to implement the sparsity requirements; additionally, bound constraints for the optimal controls can be included. We study the resulting problem in appropriate function spaces and present two solution methods of Newton type, based on different formulations of the optimality system. Using elliptic and parabolic test problems we research the sparsity properties of the optimal controls and analyze the behavior of the proposed solution algorithms.

Adaptive Sampling for Linear State Estimation

Abstract: When a sensor has continuous measurements but sends occasional messages over a data network to a supervisor which estimates the state, the available packet rate fixes the achievable quality of state estimation. When such rate limits turn stringent, the sensor's messaging policy should be designed anew. What are good causal messaging policies? What should message packets contain? What is the lowest possible distortion in a causal estimate at the supervisor? Is Delta sampling better than periodic sampling? We answer these questions for a Markov state process under an idealized model of the network and the assumption of perfect state measurements at the sensor. If the state is a scalar, or a vector of low dimension, then we can ignore sample quantization. If in addition we can ignore jitter in the transmission delays over the network, then our search for efficient messaging policies simplifies. First, each message packet should contain the value of the state at that time. Thus a bound on the number of data packets becomes a bound on the number of state samples. Second, the remaining choice in messaging is entirely about the times when samples are taken. For a scalar, linear diffusion process, we study the problem of choosing the causal sampling times that will give the lowest aggregate squared error distortion. We stick to finite horizons and impose a hard upper bound $N$ on the number of allowed samples. We cast the design as a problem of choosing an optimal sequence of stopping times. We reduce this to a nested sequence of problems, each asking for a single optimal stopping time. Under an unproven but natural assumption about the least-square estimate at the supervisor, each of these single stopping problems are of standard form. The optimal stopping times are random times when the estimation error exceeds designed envelopes. For the case where the state is a Brownian motion, we give analytically: the shape of the optimal sampling envelopes, the shape of the envelopes under optimal Delta sampling, and their performances. Surprisingly, we find that Delta sampling performs badly. Hence, when the rate constraint is a hard limit on the number of samples over a finite horizon, we should not use Delta sampling.

An Eulerian Approach to the Analysis of Krause's Consensus Models

Abstract: In this paper we analyze a class of multiagent consensus dynamical systems inspired by Krause's original model. As in Krause's model, the basic assumption is the so-called bounded confidence: two agents can influence each other only when their state values are below a given distance threshold $R$. We study the system under an Eulerian point of view considering (possibly continuous) probability distributions of agents, and we present original convergence results. The limit distribution is always necessarily a convex combination of delta functions at least $R$ far apart from each other: in other terms these models are locally aggregating. The Eulerian perspective provides the natural framework for designing a numerical algorithm, by which we obtain several simulations in $1$ and $2$ dimensions.

Stochastic Collocation for Optimal Control Problems with Stochastic PDE Constraints

Abstract: We discuss the use of stochastic collocation for the solution of optimal control problems which are constrained by stochastic partial differential equations (SPDE). Thereby the constraining SPDE depends on data which is not deterministic but random. Assuming a deter- ministic control, randomness within the states of the input data will propagate to the states of the system. For the solution of SPDEs there has recently been an increasing effort in the development of efficient numerical schemes based upon the mathematical concept of generalized polynomial chaos. Modal-based stochastic Galerkin and nodal-based stochastic collocation versions of this methodol- ogy exist, both of which rely on a certain level of smoothness of the solution in the random space to yield accelerated convergence rates. In this paper we apply the stochastic collocation method to develop a gradient descent as well as a sequential quadratic program (SQP) for the minimization of objective functions constrained by an SPDE. The stochastic function involves several higher-order moments of the random states of the system as well as classical regularization of the control. In particular we discuss several objective functions of tracking type. Numerical examples are presented to demonstrate the performance of our new stochastic collocation minimization approach.

Optimization and Convergence of Observation Channels in Stochastic Control

Abstract: This paper studies the optimization of observation channels (stochastic kernels) in partially observed stochastic control problems. In particular, existence and continuity properties are investigated, mostly (but not exclusively) concentrating on the single-stage case. Continuity properties of the optimal cost in channels are explored under total variation, setwise convergence, and weak convergence. Sufficient conditions for compactness of a class of channels under total variation and setwise convergence are presented, and applications to quantization are explored.

Variational Inequalities for Set-Valued Vector Fields on Riemannian Manifolds: Convexity of the Solution Set and the Proximal Point Algorithm

Abstract: We consider variational inequality problems for set-valued vector fields on general Riemannian manifolds. The existence results of the solution, convexity of the solution set, and the convergence property of the proximal point algorithm for the variational inequality problems for set-valued mappings on Riemannian manifolds are established. Applications to convex optimization problems on Riemannian manifolds are provided.

A Stochastic Maximum Principle for a Markov Regime-Switching Jump-Diffusion Model and Its Application to Finance

Abstract: This paper develops a sufficient stochastic maximum principle for a stochastic optimal control problem, where the state process is governed by a continuous-time Markov regime-switching jump-diffusion model. We also establish the relationship between the stochastic maximum principle and the dynamic programming principle in a Markovian case. Applications of the stochastic maximum principle to the mean-variance portfolio selection problem are discussed.

Second Order Analysis for Bang-Bang Control Problems of PDEs

Abstract: In this paper, we derive some sufficient second order optimality conditions for control problems of partial differential equations (PDEs) when the cost functional does not involve the usual quadratic term for the control or higher nonlinearities for it. Though not always, in this situation the optimal control is typically bang-bang. Two different control problems are studied. The second differs from the first in the presence of the $L^1$ norm of the control. This term leads to optimal controls that are sparse and usually take only three different values (we call them bang-bang-bang controls). Though the proofs are detailed in the case of a semilinear elliptic state equation, the approach can be extended to parabolic control problems. Some hints are provided in the last section to extend the results.

Distributed Optimal Control of the Cahn-Hilliard System Including the Case of a Double-Obstacle Homogeneous Free Energy Density

Abstract: In this paper we study the distributed optimal control for the Cahn-Hilliard system. A general class of free energy potentials is allowed which, in particular, includes the double-obstacle potential. The latter potential yields an optimal control problem of a parabolic variational inequality which is of fourth order in space. We show the existence of optimal controls to approximating problems where the potential is replaced by a mollified version of its Moreau-Yosida approximation. Corresponding first-order optimality conditions for the mollified problems are given. For this purpose a new result on the continuous Frechet differentiability of superposition operators with values in Sobolev spaces is established. Besides the convergence of optimal controls of the mollified problems to an optimal control of the original problem, we also derive first-order optimality conditions for the original problem by a limit process. The newly derived stationarity system corresponds to a function space version of C-stationarity.

Gossip Coverage Control for Robotic Networks: Dynamical Systems on the Space of Partitions

Abstract: Future applications in environmental monitoring, delivery of services, and transportation of goods motivate the study of deployment and partitioning tasks for groups of autonomous mobile agents. These tasks may be achieved by recent coverage algorithms, based upon the classic methods by Lloyd. These algorithms, however, rely upon critical requirements on the communication network: information is exchanged synchronously among all agents and long-range communication is sometimes required. This work proposes novel coverage algorithms that require only gossip communication, i.e., asynchronous, pairwise, and possibly unreliable communication. Which robot pair communicates at any given time may be selected deterministically or randomly. A key innovative idea is describing coverage algorithms for robot deployment and environment partitioning as dynamical systems on a space of partitions. In other words, we study the evolution of the regions assigned to each agent rather than the evolution of the agents' positions. The proposed gossip algorithms are shown to converge to centroidal Voronoi partitions under mild technical conditions. Our treatment features a broad variety of results in topology, analysis, and geometry. First, we establish the compactness of a suitable space of partitions with respect to the symmetric difference metric. Second, with respect to this metric, we establish the continuity of various geometric maps, including the Voronoi diagram as a function of its generators, the location of a centroid as a function of a set, and the widely known multicenter function studied in facility location problems. Third, we prove two convergence theorems for dynamical systems on metric spaces described by deterministic and stochastic switches.

Convergence Speed of Binary Interval Consensus

Abstract: We consider the convergence time for solving the binary consensus problem using the interval consensus algorithm proposed by Benezit, Thiran, and Vetterli [Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP), IEEE Press, Piscataway, NJ, 2009, pp. 3661–3664]. In the binary consensus problem, each node initially holds one of two states and the goal for each node is to correctly decide which one of these two states was initially held by a majority of nodes. We derive an upper bound on the expected convergence time that holds for arbitrary connected graphs, which is based on the location of eigenvalues of some contact rate matrices. We instantiate our bound for particular networks of interest, including complete graphs, paths, cycles, star-shaped networks, and Erdos–Renyi random graphs; for these graphs, we compare our bound with alternative computations. We find that for all these examples our bound is tight, yielding the exact order with respect to the numbe...

Distributed Optimal Control of the Cahn-Hilliard System Including the Case of a Double-Obstacle Homogeneous Free Energy Density

Abstract: In this paper we study the distributed optimal control for the Cahn-Hilliard system. A general class of free energy potentials is allowed which, in particular, includes the double-obstacle potentia...

Superhedging and Dynamic Risk Measures under Volatility Uncertainty

Abstract: We consider dynamic sublinear expectations (i.e., time-consistent coherent risk mea- sures) whose scenario sets consist of singular measures corresponding to a general form of volatility uncertainty. We derive a cadlag nonlinear martingale which is also the value process of a superhedg- ing problem. The superhedging strategy is obtained from a representation similar to the optional decomposition. Furthermore, we prove an optional sampling theorem for the nonlinear martingale and characterize it as the solution of a second order backward SDE. The uniqueness of dynamic extensions of static sublinear expectations is also studied.

On the Equivalence of Minimal Time and Minimal Norm Controls for Internally Controlled Heat Equations

Abstract: We prove the equivalence of the minimal time and minimal norm control problems for heat equations on bounded smooth domains of the Euclidean space with homogeneous Dirichlet boundary conditions and controls distributed internally on an open subset of the domain where the equation evolves. We consider the problem of null controllability whose aim is to drive solutions to rest in a finite final time. As a consequence of this equivalence, using the well-known variational characterization of minimal norm controls we establish necessary and sufficient conditions for the minimal time and the corresponding control.

Minimal Bit Rates and Entropy for Exponential Stabilization

Abstract: Minimal bit rates and entropy are studied for exponential stabilization of control systems in continuous time. Upper and lower bounds for the stabilization entropy are derived. In particular, for linear systems, a formula is given in terms of the real parts of eigenvalues. Then the minimal bit rate is related to the stabilization entropy.

Diagonal and Low-Rank Matrix Decompositions, Correlation Matrices, and Ellipsoid Fitting

Abstract: In this paper we establish links between, and new results for, three problems that are not usually considered together. The first is a matrix decomposition problem that arises in areas such as statistical modeling and signal processing: given a matrix X formed as the sum of an unknown diagonal matrix and an unknown low rank positive semidefinite matrix, decompose X into these constituents. The second problem we consider is to determine the facial structure of the set of correlation matrices, a convex set also known as the elliptope. This convex body, and particularly its facial structure, plays a role in applications from combinatorial optimization to mathematical finance. The third problem is a basic geometric question: given points v1, v2, … , vn ∈ R^k (where n > k) determine whether there is a centered ellipsoid passing exactly through all of the points. We show that in a precise sense these three problems are equivalent. Furthermore we establish a simple sufficient condition on a subspace U that ensures any positive semidefinite matrix L with column space U can be recovered from D+L for any diagonal matrix D using a convex optimization-based heuristic known as minimum trace factor analysis. This result leads to a new understanding of the structure of rank-deficient correlation matrices and a simple condition on a set of points that ensures there is a centered ellipsoid passing through them.

Anthropomorphic Image Reconstruction via Hypoelliptic Diffusion

Abstract: In this paper we study a model of geometry of vision due to Petitot, Citti, and Sarti. One of the main features of this model is that the primary visual cortex V1 lifts an image from $\mathbb{R}^2$ to the bundle of directions of the plane. Neurons are grouped into orientation columns, each of them corresponding to a point of this bundle. In this model a corrupted image is reconstructed by minimizing the energy necessary for the activation of the orientation columns corresponding to regions in which the image is corrupted. The minimization process intrinsically defines a hypoelliptic heat equation on the bundle of directions of the plane. In the original model, directions are considered both with and without orientation, giving rise, respectively, to a problem on the group of rototranslations of the plane $SE(2)$ or on the projective tangent bundle of the plane $PT\mathbb{R}^2$. We provide a mathematical proof of several important facts for this model. We first prove that the model is mathematically consis...

Least Squares Temporal Difference Methods: An Analysis under General Conditions

Abstract: We consider approximate policy evaluation for finite state and action Markov de- cision processes (MDP) with the least squares temporal difference (LSTD) algorithm, LSTD(λ), in an exploration-enhanced learning context, where policy costs are computed from observations of a Markov chain different from the one corresponding to the policy under evaluation. We establish for the discounted cost criterion that LSTD(λ) converges almost surely under mild, minimal conditions. We also analyze other properties of the iterates involved in the algorithm, including convergence in mean and boundedness. Our analysis draws on theories of both finite space Markov chains and weak Feller Markov chains on a topological space. Our results can be applied to other temporal difference algorithms and MDP models. As examples, we give a convergence analysis of a TD(λ) algorithm and extensions to MDP with compact state and action spaces, as well as a convergence proof of a new LSTD algorithm with state-dependent λ-parameters.

Exponentially Stable Interval Observers for Linear Systems with Delay

Abstract: This paper focuses on the analysis and design of families of interval observers for linear systems with a pointwise delay. First, it is proved that classical interval observers for systems without delays are not robust with respect to the presence of delays, no matter how small delays are. Next, it is shown that, in general, for linear systems with delay, the classical interval observers endowed with a pointwise delay are unstable. A new type of design of interval observers enabling circumvention of these obstacles is proposed. It provides framers that incorporate distributed delay terms. The proposed interval observers are assessed through a nonlinear biotechnological model.

Variational Approach to Second-Order Optimality Conditions for Control Problems with Pure State Constraints

Abstract: For optimal control problems with set-valued control constraints and pure state constraints we propose new second-order necessary optimality conditions. In addition to the usual second-order derivative of the Hamiltonian, these conditions contain extra terms involving second-order tangents to the set of feasible trajectory-control pairs at the extremal process under consideration. The second-order necessary optimality conditions of the present work are obtained by using a variational approach. In particular, we present a new second-order variational equation. This approach allows us to make direct proofs as opposed to the classical way of obtaining second-order necessary conditions by using an abstract infinite dimensional optimization problem. No convexity assumptions on the constraints are imposed and optimal controls are required to be merely measurable.

Weak Dynamic Programming for Generalized State Constraints

Abstract: We provide a dynamic programming principle for stochastic optimal control problems with expectation constraints. A weak formulation, using test functions and a probabilistic relaxation of the constraint, avoids restrictions related to a measurable selection but still implies the Hamilton--Jacobi--Bellman equation in the viscosity sense. We treat open state constraints as a special case of expectation constraints and prove a comparison theorem to obtain the equation for closed state constraints.

Coordination of Passive Systems under Quantized Measurements

Abstract: In this paper we investigate a passivity approach to collective coordination and synchronization problems in the presence of quantized measurements and show that coordination tasks can be achieved in a practical sense for a large class of passive systems.