Showing papers in "Siam Journal on Control and Optimization in 2007"
TL;DR: It is proved that asymptotic state agreement is achieved if and only if the dynamic interaction digraph has the property of being sufficiently connected over time.
Abstract: Two related problems are treated in continuous time. First, the state agreement problem is studied for coupled nonlinear differential equations. The vector fields can switch within a finite family. Associated to each vector field is a directed graph based in a natural way on the interaction structure of the subsystems. Generalizing the work of Moreau, under the assumption that the vector fields satisfy a certain subtangentiality condition, it is proved that asymptotic state agreement is achieved if and only if the dynamic interaction digraph has the property of being sufficiently connected over time. The proof uses nonsmooth analysis. Second, the rendezvous problem for kinematic point-mass mobile robots is studied when the robots’ fields of view have a fixed radius. The circumcenter control law of Ando e [IEEE Trans. Robotics Automation, 15 (1999), pp. 818-828] is shown to solve the problem. The rendezvous problem is a kind of state agreement problem, but the interaction structure is state dependent.
518 citations
TL;DR: In this paper, an introduction to quantum probability theory is given, focusing on the spectral theorem and the conditional expectation as a least squares estimate, and culminating in the construction of Wiener and Poisson processes on the Fock space.
Abstract: This paper provides an introduction to quantum filtering theory. An introduction to quantum probability theory is given, focusing on the spectral theorem and the conditional expectation as a least squares estimate, and culminating in the construction of Wiener and Poisson processes on the Fock space. We describe the quantum Ito calculus and its use in the modeling of physical systems. We use both reference probability and innovations methods to obtain quantum filtering equations for system-probe models from quantum optics.
509 citations
TL;DR: Using Liapunov functions, necessary and sufficient conditions for positive recurrence are developed and ergodicity of positive recurrent regime-switching diffusions is obtained by constructing cycles using the associated discrete-time Markov chains.
Abstract: In response to the increasing needs for control and optimization of hybrid systems, this work is concerned with such asymptotic properties as recurrence (also known as weak stochastic stability in the literature) and ergodicity of regime-switching diffusions. Using Liapunov functions, necessary and sufficient conditions for positive recurrence are developed. Then, ergodicity of positive recurrent regime-switching diffusions is obtained by constructing cycles using the associated discrete-time Markov chains.
352 citations
TL;DR: The properties of controlled quantum filtering equations as classical stochastic differential equations are studied and methods for global feedback stabilization of a class of quantum filters around a particular eigenstate of the measurement operator are developed.
Abstract: No quantum measurement can give full information on the state of a quantum system; hence any quantum feedback control problem is necessarily one with partial observations and can generally be converted into a completely observed control problem for an appropriate quantum filter as in classical stochastic control theory Here we study the properties of controlled quantum filtering equations as classical stochastic differential equations We then develop methods, using a combination of geometric control and classical probabilistic techniques, for global feedback stabilization of a class of quantum filters around a particular eigenstate of the measurement operator
261 citations
TL;DR: A family of unsynchronized strategies for solving the multi-agent rendezvous problem of a group of mobile autonomous agents, labelled 1 through n, which can all move in the plane are described.
Abstract: This paper is concerned with the collective behavior of a group of $n>1$ mobile autonomous agents, labelled $1$ through $n$, which can all move in the plane. Each agent is able to continuously track the positions of all other agents currently within its “sensing region,” where by an agent's sensing region we mean a closed disk of positive radius $r$ centered at the agent's current position. The multi-agent rendezvous problem is to devise “local” control strategies, one for each agent, which without any active communication between agents cause all members of the group to eventually rendezvous at a single unspecified location. This paper describes a family of unsynchronized strategies for solving the problem. Correctness is established appealing to the concept of “analytic synchronization.”
231 citations
TL;DR: This paper investigates the problems of passivity analysis and passification for network-based linear control systems and proposes procedures for designing passification controllers, which guarantee that the closed-loop networked control system (NCS) is passive.
Abstract: This paper investigates the problems of passivity analysis and passification for network-based linear control systems. A new sampled-data model is first formulated based on the updating instants of the ZOH (zeroth order hold), where the physical plant and the controller are, respectively, in continuous time and discrete time. In this model, network-induced delays, data packet dropouts, and signal measurement quantization have been taken into account. The measurement quantizer is assumed to be logarithmic, and the network-induced delays are assumed to have both a lower bound and an upper bound, which is more general than those assumptions used in the literature. The key idea is to transform the sampled-data model into a linear system with two successive delay components in the state. Then, by using a Lyapunov-Krasovskii approach plus the free weighting matrix technique, a passivity performance condition is formulated in the form of linear matrix inequalities (LMIs). Based on this condition, two procedures are proposed for designing passification controllers, which guarantee that the closed-loop networked control system (NCS) is passive. Finally, two illustrative examples are presented: one shows the advantage of introducing the lower bound of transmission delays and shows how much the quantization behavior affects the passivity performance; the other illustrates the applicability and effectiveness of the proposed passification results.
167 citations
TL;DR: A mathematical model for antiangiogenic treatments based on a biologically validated model by Hahnfeldt et al is analyzed as an optimal control problem and a full solution of the problem is given.
Abstract: Antiangiogenic therapy is a novel treatment approach in cancer therapy that aims at preventing a tumor from developing its own blood supply system that it needs for growth. In this paper a mathematical model for antiangiogenic treatments based on a biologically validated model by Hahnfeldt et al. is analyzed as an optimal control problem and a full solution of the problem is given. Geometric methods from optimal control theory are utilized to arrive at the solution.
166 citations
TL;DR: The present paper addresses the problem of the existence of an (output) feedback law that asymptotically steers to zero prescribed outputs, while keeping all state variables bounded, for any initial conditions in a given compact set.
Abstract: The present paper addresses the problem of the existence of an (output) feedback law that asymptotically steers to zero prescribed outputs, while keeping all state variables bounded, for any initial conditions in a given compact set. The problem can be viewed as an extension of the classical problem of semiglobally stabilizing the trajectories of a controlled system to a compact set. The problem also encompasses a version of the classical problem of output regulation. Under only a weak minimum phase assumption, it is shown that there exists a controller solving the problem at hand. The paper is deliberately focused on theoretical results regarding the existence of such a controller. Practical aspects involving the design and the implementation of the controller are left to a forthcoming work.
153 citations
TL;DR: The main result is a performance bound of the resulting policies expressed in terms of the $L_p$-norm errors introduced by the successive approximations, which takes into account a concentration coefficient that estimates how much the discounted future-state distributions starting from a probability measure can possibly differ from the distribution used in the regression operation.
Abstract: Approximate value iteration (AVI) is a method for solving large Markov decision problems by approximating the optimal value function with a sequence of value function representations $V_n$ processed according to the iterations $V_{n+1}=\mathcal{AT} V_n$, where $\mathcal{T}$ is the so-called Bellman operator and $\mathcal{A}$ an approximation operator, which may be implemented by a supervised learning (SL) algorithm. Usual bounds on the asymptotic performance of AVI are established in terms of the $L_\infty$-norm approximation errors induced by the SL algorithm. However, most widely used SL algorithms (such as least squares regression) return a function (the best fit) that minimizes an empirical approximation error in $L_p$-norm ($p\geq 1$). In this paper, we extend the performance bounds of AVI to weighted $L_p$-norms, which enables us to directly relate the performance of AVI to the approximation power of the SL algorithm, hence assuring the tightness and practical relevance of these bounds. The main result is a performance bound of the resulting policies expressed in terms of the $L_p$-norm errors introduced by the successive approximations. The new bound takes into account a concentration coefficient that estimates how much the discounted future-state distributions starting from a probability measure used to assess the performance of AVI can possibly differ from the distribution used in the regression operation. We illustrate the tightness of the bounds on an optimal replacement problem.
147 citations
TL;DR: The first control objective is tracking, by the output $y$, with prescribed accuracy: determine a feedback strategy which ensures that, for every reference signal $r$ and every system of class $\Sigma_{\rho}$, the tracking error is ultimately bounded by l.
Abstract: Tracking of a reference signal (assumed bounded with essentially bounded derivative) is considered in the context of a class $\Sigma_{\rho}$ of multi-input, multi-output dynamical systems, modelled by functional differential equations, affine in the control and satisfying the following structural assumptions: (i) arbitrary—but known—relative degree $\rho \ge 1; (ii) the “high-frequency gain” is sign definite—but possibly of unknown sign. The class encompasses a wide variety of nonlinear and infinite-dimensional systems and contains (as a prototype subclass) all finite-dimensional, linear, $m$-input, $m$-output, minimum-phase systems of known strict relative degree. The first control objective is tracking, by the output $y$, with prescribed accuracy: given $\lambda >0$ (arbitrarily small), determine a feedback strategy which ensures that, for every reference signal $r$ and every system of class $\Sigma_{\rho}$, the tracking error $e=y-r$ is ultimately bounded by l (that is, $\|e(t)\| < \lambda$ for all $t$ sufficiently large). The second objective is guaranteed output transient performance: the tracking error is required to evolve within a prescribed performance funnel $\mathcal{F}_\varphi$ (determined by a function J). Both objectives are achieved using a filter in conjunction with a feedback function of the tracking error, the filter states, and the funnel parameter J.
145 citations
TL;DR: This work considers HJB PDEs in which the Hamiltonian takes the form of a (pointwise) maximum of linear/quadratic forms, and obtains a numerical method not subject to the curse of dimensionality.
Abstract: In previous works of the author and others, max-plus methods have been explored for the solution of first-order, nonlinear Hamilton-Jacobi-Bellman partial differential equations (HJB PDEs) and corresponding nonlinear control problems. These methods exploit the max-plus linearity of the associated semigroups. In particular, although the problems are nonlinear, the semigroups are linear in the max-plus sense. These methods have been used successfully to compute solutions. Although they provide certain computational-speed advantages, they still generally suffer from the curse of dimensionality. Here we consider HJB PDEs in which the Hamiltonian takes the form of a (pointwise) maximum of linear/quadratic forms. The approach to the solution will be rather general, but in order to ground the work, we consider only constituent Hamiltonians corresponding to long-run average-cost-per-unit-time optimal control problems for the development. We obtain a numerical method not subject to the curse of dimensionality. The method is based on construction of the dual-space semigroup corresponding to the HJB PDE. This dual-space semigroup is constructed from the dual-space semigroups corresponding to the constituent linear/quadratic Hamiltonians. The dual-space semigroup is particularly useful due to its form as a max-plus integral operator with a kernel obtained from the originating semigroup. One considers repeated application of the dual-space semigroup to obtain the solution.
TL;DR: The problem of determining the optimal sequence of stopping times for a diffusion process subject to regime switching decisions is considered, using a viscosity solutions approach combined with the smooth-fit property to solve the problem in the two-regime case when the state process is of geometric Brownian nature.
Abstract: This paper considers the problem of determining the optimal sequence of stopping times for a diffusion process subject to regime switching decisions. This is motivated in the economics literature by the investment problem under uncertainty for a multi-activity firm involving opening and closing decisions. We use a viscosity solutions approach combined with the smooth-fit property, and explicitly solve the problem in the two-regime case when the state process is of geometric Brownian nature. The results of our analysis take several qualitatively different forms, depending on model parameter values.
TL;DR: An adaptive control scheme for a Hammerstein nonlinear sampled-data system in which the output sampling period is an integer multiple of the input updating period is derived by using a polynomial transformation technique, suitable for parameter estimation with dual-rate measurement data.
Abstract: This paper is motivated by the practical control considerations that nonlinearity is abundant in industrial processes and output sampling rates are often limited due to hardware constraints. In particular, for a Hammerstein nonlinear sampled-data system in which the output sampling period is an integer multiple of the input updating period, we derive, by using a polynomial transformation technique, a mathematical model which is suitable for parameter estimation with dual-rate measurement data. Further, we present an adaptive control scheme for such a dual-rate nonlinear system; the parameter estimation-based adaptive algorithm can achieve virtually asymptotically optimal control and ensure that the closed-loop system is stable and globally convergent. The simulation results are included.
TL;DR: It is proved that the nonlinear term gives the local exact controllability around the origin provided that the time of control is large enough.
Abstract: We consider the boundary controllability problem for a nonlinear Korteweg-de Vries equation with the Dirichlet boundary condition. We study this problem for a spatial domain with a critical length for which the linearized control system is not controllable. In order to deal with the nonlinearity, we use a power series expansion of second order. We prove that the nonlinear term gives the local exact controllability around the origin provided that the time of control is large enough.
TL;DR: In this article, a posteriori error estimate for space-time finite element discretizations of parabolic optimization problems is derived for a given quantity of interest and separate the influences of different parts of the discretization.
Abstract: In this paper we derive a posteriori error estimates for space-time finite element discretizations of parabolic optimization problems. The provided error estimates assess the discretization error with respect to a given quantity of interest and separate the influences of different parts of the discretization (time, space, and control discretization). This allows us to set up an efficient adaptive algorithm which successively improves the accuracy of the computed solution by construction of locally refined meshes for time and space discretizations.
TL;DR: This paper approaches optimal control problems for discrete-time controlled Markov processes by representing the value of the problem in a dual Lagrangian form, which opens up the possibility of numerical methods based on Monte Carlo simulation, which may be advantageous in high-dimensional problems or in problems with complicated constraints.
Abstract: This paper approaches optimal control problems for discrete-time controlled Markov processes by representing the value of the problem in a dual Lagrangian form; the value is expressed as an infimum over a family of Lagrangian martingales of an expectation of a pathwise supremum of the objective adjusted by the Lagrangian martingale term. This representation opens up the possibility of numerical methods based on Monte Carlo simulation, which may be advantageous in high-dimensional problems or in problems with complicated constraints.
TL;DR: It is shown that for noise-free plants, the Shannon capacity of the channel constitutes the border separating the cases where stabilization and reliable detection with arbitrarily large probability are and are not possible, respectively.
Abstract: The paper addresses both detection and stabilization problems involving communication errors and capacity constraints. Discrete-time partially observed linear systems are studied. Unlike the classic theory, the sensor signals are transmitted to the estimator/controller over a noisy digital communication link modeled as a stochastic stationary discrete memoryless channel. It is shown that for noise-free plants, the Shannon capacity of the channel constitutes the border separating the cases where stabilization and reliable detection (asymptotic state estimation) with arbitrarily large probability are and are not possible, respectively.
TL;DR: A global exact controllability result is obtained for a class of multidimensional semilinear hyperbolic equations with a superlinear nonlinearity and variable coefficients, in which the crucial observability constant is estimated explicitly by a function of the norm of the potential.
Abstract: In this paper, we obtain a global exact controllability result for a class of multidimensional semilinear hyperbolic equations with a superlinear nonlinearity and variable coefficients. For this purpose, we establish an observability estimate for the linear hyperbolic equation with an unbounded potential, in which the crucial observability constant is estimated explicitly by a function of the norm of the potential. Such an estimate is obtained by a combination of a pointwise estimate and a global Carleman estimate for the hyperbolic differential operators and analysis on the regularity of the optimal solution to an auxiliary optimal control problem.
TL;DR: The main methodological contribution of the paper is to employ the particle system representation to develop analytical and numerical approaches in obtaining the filter as well as solving the related backward stochastic differential equation.
Abstract: This paper is concerned with a continuous-time mean-variance portfolio selection problem in a (possibly incomplete) market with multiple stocks and a bond. Only the past price movements of the stocks and the bond are the information available to the investors. A separation principle is shown to hold in this setting. Efficient strategies based on the aforementioned partial information are derived, which involve the optimal filter of the stock appreciation rate processes. The main methodological contribution of the paper is to employ the particle system representation to develop analytical and numerical approaches in obtaining the filter as well as solving the related backward stochastic differential equation.
TL;DR: In this paper, a small-gain theorem can be applied to a wide class of systems that include systems satisfying the weak semigroup property, which generalizes all existing results in the literature and exploits notions of weighted, uniform, and nonuniform input-to-output stability properties.
Abstract: A small-gain theorem, which can be applied to a wide class of systems that includes systems satisfying the weak semigroup property, is presented in the present work The result generalizes all existing results in the literature and exploits notions of weighted, uniform, and nonuniform input-to-output stability properties Applications to partial state feedback stabilization problems with sampled-data feedback applied with zero order hold and positive sampling rate are also presented
TL;DR: A new procedure is presented for determining the kernel and the offspring hypersurfaces for general linear time invariant (LTI) dynamics with multiple delays using the extraordinary features of the “extended Kronecker summation” operation.
Abstract: A new procedure is presented for determining the kernel and the offspring hypersurfaces for general linear time invariant (LTI) dynamics with multiple delays. These hypersurfaces, as they have very recently been introduced in a concept paper [R. Sipahi and N. Olgac, Automatica, 41 (2005), pp. 1413-1422], form the basis of the overriding paradigm which is called the cluster treatment of characteristic roots (CTCR). In fact, these two sets of hypersurfaces exhaustively represent the locations in the domain of the delays where the system possesses at least one pair of imaginary characteristic roots. To determine the kernel and offspring we use the extraordinary features of the “extended Kronecker summation” operation in this paper. The end result is that the infinite-dimensional problem reduces to a finite-dimensional one (and preferably into an eigenvalue problem). Following the procedure described in this paper, we are able to shorten the computational time considerably in determining these hypersurfaces. We demonstrate these concepts via some example case studies. One of the examples treats a 3-delay system. For this case another interesting perspective, called the “building block,” is also utilized to display the kernel in three-dimensional space in the domain of “spectral delays.”
TL;DR: The problem of determining the optimal investment level that a firm should maintain in the presence of random price and/or demand fluctuations is considered, by means of a geometric Brownian motion, and general running payoff functions are considered.
Abstract: We consider the problem of determining the optimal investment level that a firm should maintain in the presence of random price and/or demand fluctuations. We model market uncertainty by means of a geometric Brownian motion, and we consider general running payoff functions. Our model allows for capacity expansion as well as for capacity reduction, with each of these actions being associated with proportional costs. The resulting optimization problem takes the form of a singular stochastic control problem that we solve explicitly. We illustrate our results by means of the so-called Cobb-Douglas production function. The problem that we study presents a model in which the associated Hamilton-Jacobi-Bellman equation admits a classical solution that conforms with the underlying economic intuition but does not necessarily identify with the corresponding value function, which may be identically equal to $\infty$. Thus, our model provides a situation that highlights the need for rigorous mathematical analysis when addressing stochastic optimization applications in finance and economics, as well as in other fields.
TL;DR: First, the existence of insensitizing controls for the $L^2$ norm of the gradient of solutions of linear heat equations is proved, and in the worst situation where null controllability for a system of two parabolic equations can hold, this result is proved.
Abstract: In this paper we establish some exact controllability results for systems of two parabolic equations. First, we prove the existence of insensitizing controls for the $L^2$ norm of the gradient of solutions of linear heat equations. Then, in the worst situation where null controllability for a system of two parabolic equations can hold, we prove this result for some general couplings.
TL;DR: It is shown in this paper that reachability of continuous systems can also be verified through convex programming, and the convexity of the methods are exploited to derive a converse theorem for safety verification using barrier certificates.
Abstract: A methodology for safety verification of continuous and hybrid systems using barrier certificates has been proposed recently. Conditions that must be satisfied by a barrier certificate can be formulated as a convex program, and the feasibility of the program implies system safety in the sense that there is no trajectory starting from a given set of initial states that reaches a given unsafe region. The dual of this problem, i.e., the reachability problem, concerns proving the existence of a trajectory starting from the initial set that reaches another given set. Using insights from the linear programming duality appearing in the discrete shortest path problem, we show in this paper that reachability of continuous systems can also be verified through convex programming. Several convex programs for verifying safety and reachability, as well as other temporal properties such as eventuality, avoidance, and their combinations, are formulated. Some examples are provided to illustrate the application of the proposed methods. Finally, we exploit the convexity of our methods to derive a converse theorem for safety verification using barrier certificates.
TL;DR: Approximation by uniformly ergodic controlled Markov processes is introduced, which allows us to show the existence of solutions to the infinite horizon risk sensitive Bellman equation.
Abstract: Risk sensitive control of Markov processes satisfying the minorization property is studied using splitting techniques. Existence of solutions to the multiplicative Poisson equation is shown. Approximation by uniformly ergodic controlled Markov processes is introduced, which allows us to show the existence of solutions to the infinite horizon risk sensitive Bellman equation.
Journal Article•
TL;DR: This work considers the task of network exploration by a mobile agent (robot) with small memory, and presents an algorithm to accomplish tree exploration (with return) using O(log n)-bit memory for all n-node trees.
Abstract: We consider the task of network exploration by a mobile agent (robot) with small memory. The agent has to traverse all nodes and edges of a network (represented as an undirected connected graph), and return to the starting node. Nodes of the network are unlabeled and edge ports are locally labeled at each node. The agent has no a priori knowledge of the topology of the network or of its size, and cannot mark nodes in any way. Under such weak assumptions, cycles in the network may prevent feasibility of exploration, hence we restrict attention to trees. We present an algorithm to accomplish tree exploration (with return) using O(log n)-bit memory for all n-node trees. This strengthens the result from [15], where O(log2n)-bit memory was used for tree exploration, and matches the lower bound on memory size proved there.
TL;DR: A discretization based on space-time finite elements is proposed and numerical examples are included and its global and local superlinear convergences are shown.
Abstract: Optimal Dirichlet boundary control based on the very weak solution of a parabolic state equation is analyzed. This approach allows us to consider the boundary controls in $L^2$, which has advantages over approaches which consider control in Sobolev spaces involving (fractional) derivatives. Pointwise constraints on the boundary are incorporated by the primal-dual active set strategy. Its global and local superlinear convergences are shown. A discretization based on space-time finite elements is proposed and numerical examples are included.
TL;DR: Existence and convergence of approximate solutions are proved, provided that the infinite dimensional shape problem admits a stable second order optimizer.
Abstract: The present paper aims at analyzing the existence and convergence of approximate solutions in shape optimization. Motivated by illustrative examples, an abstract setting of the underlying shape optimization problem is suggested, taking into account the so-called two norm discrepancy. A Ritz-Galerkin-type method is applied to solve the associated necessary condition. Existence and convergence of approximate solutions are proved, provided that the infinite dimensional shape problem admits a stable second order optimizer. The rate of convergence is confirmed by numerical results.
TL;DR: The (global) exact controllability property to nonzero constant states is proved with control functions whose norms in an appropriate space are bounded independently of $
u$, which belongs to a suitably small interval.
Abstract: In this paper, we deal with the viscous Burgers equation with a small dissipation coefficient $
u$. We prove the (global) exact controllability property to nonzero constant states, that is to say, the possibility of finding boundary values such that the solution of the associated Burgers equation is driven to a constant state. The main objective of this paper is to do so with control functions whose norms in an appropriate space are bounded independently of $
u$, which belongs to a suitably small interval. This result is obtained for a sufficiently large time.
TL;DR: In this paper, the authors obtained error estimates for the numerical approximation of a distributed control problem governed by the stationary Navier-Stokes equations, with pointwise control constraints, and established a second order necessary optimality condition.
Abstract: We obtain error estimates for the numerical approximation of a distributed control problem governed by the stationary Navier-Stokes equations, with pointwise control constraints. We show that the $L^2$-norm of the error for the control is of order $h^2$ if the control set is not discretized, while it is of order $h$ if it is discretized by piecewise constant functions. These error estimates are obtained for local solutions of the control problem, which are nonsingular in the sense that the linearized Navier-Stokes equations around these solutions define some isomorphisms, and which satisfy a second order sufficient optimality condition. We establish a second order necessary optimality condition. The gap between the necessary and sufficient second order optimality conditions is the usual gap known for finite dimensional optimization problems.