Showing papers in "Siam Journal on Control and Optimization in 2015"

Stabilization of hybrid systems by feedback control based on discrete-time state observations

Abstract: Recently, Mao [Automatica J. IFAC, 49 (2013), pp. 3677--3681] initiated the study the mean-square exponential stabilization of continuous-time hybrid stochastic differential equations by feedback controls based on discrete-time state observations. In the same paper Mao also obtains an upper bound on the duration $\tau$ between two consecutive state observations. However, it is due to the general technique used there that the bound on $\tau$ is not very sharp. In this paper, we will be able to establish a better bound on $\tau$ making use of Lyapunov functionals. We will discuss the stabilization not only in the sense of exponential stability (as Mao does in [Automatica J. IFAC, 49 (2013), pp. 3677--3681]) but also in other sense---that of $H_\infty$ stability or asymptotic stability. We will consider not only the mean square stability but also the almost sure stability.

Approximate Controllability of Fractional Evolution Systems with Riemann--Liouville Fractional Derivatives

Abstract: In this paper, we deal with the control systems governed by fractional evolution differential equations involving Riemann--Liouville fractional derivatives in Banach spaces. Our main purpose in this article is to establish suitable assumptions to guarantee the existence and uniqueness results of mild solutions. Under these conditions, the approximate controllability of the associated fractional evolution systems involving Riemann--Liouville fractional derivatives is formulated and proved.

Controllability Analysis and Control Design for Switched Boolean Networks with State and Input Constraints

Abstract: This paper investigates the controllability analysis and the control design for switched Boolean networks (SBNs) with state and input constraints by using the semi-tensor product method and presents a number of new results on their controllability, optimal control, and stabilization. First, the constrained incidence matrix is proposed for SBNs, based on which several necessary and sufficient conditions are obtained for the controllability. Second, using the results on the controllability, two algorithms are established to design proper switching sequence and controls that minimize the cost functional in a fixed/designed shortest termination time. Third, the constrained SBN is converted to an equivalent unconstrained one, based on which some necessary and sufficient conditions are presented for the stabilization of the constrained SBN with open-loop and closed-loop controls, respectively. Finally, a practical example of the apoptosis network is studied by using the new results obtained in this paper. The s...

The Maximum Principle for Global Solutions of Stochastic Stackelberg Differential Games

Abstract: National Science Foundation (DMS 1303775); Research Grant Council of the HKSAR, (CityU 500113)

CONSTRUCTION OF LYAPUNOV FUNCTIONS FOR INTERCONNECTED PARABOLIC SYSTEMS: AN iISS APPROACH ∗

Abstract: This paper is devoted to two issues. The first one is to provide Lyapunov-based tools to establish integral input-to-state stability (iISS) and input-to-state stability (ISS) for some classes of nonlinear parabolic equations. The other one is to provide a stability criterion for interconnections of iISS parabolic systems. The results addressing the former problem allow us to overcome obstacles arising in tackling the latter one. The results for the latter problem are a small-gain condition and a formula of Lyapunov functions which can be constructed for interconnections whenever the small-gain condition holds. It is demonstrated that for interconnections of partial differential equations, the choice of a right state and input spaces is crucial, in particular for iISS subsystems which are not ISS. As illustrative examples, stability of two highly nonlinear reaction-diffusion systems is established by the proposed small-gain criterion.

A Non-Markovian Liquidation Problem and Backward SPDEs with Singular Terminal Conditions

Abstract: We establish the existence, uniqueness, and regularity of solution results for a class of backward stochastic partial differential equations with singular terminal condition. The equation describes the value function of non-Markovian stochastic optimal control problem in which the terminal state of the controlled process is prespecified. The analysis of such control problems is motivated by models of optimal portfolio liquidation.

Optimal Boundary Control of a Viscous Cahn--Hilliard System with Dynamic Boundary Condition and Double Obstacle Potentials

Abstract: In this paper, we investigate optimal boundary control problems for Cahn--Hilliard variational inequalities with a dynamic boundary condition involving double obstacle potentials and the Laplace--Beltrami operator. The cost functional is of standard tracking type, and box constraints for the controls are prescribed. We prove existence of optimal controls and derive first-order necessary conditions of optimality. The general strategy, which follows the lines of the recent approach by Colli, Farshbaf-Shaker, and Sprekels [Appl. Math. Optim., 71 (2015), pp. 1--24] to the (simpler) Allen--Cahn case, is the following: we use the results that were recently established by Colli, Gilardi, and Sprekels [Appl. Math. Optim., Online First DOI:10.1007/s00245-015-9299-z, 2015] for the case of (differentiable) logarithmic potentials and perform a so-called deep quench limit. Using compactness and monotonicity arguments, it is shown that this strategy leads to the desired first-order necessary optimality conditions for t...

A FEM for an Optimal Control Problem of Fractional Powers of Elliptic Operators

Abstract: We study solution techniques for a linear-quadratic optimal control problem involving fractional powers of elliptic operators. These fractional operators can be realized as the Dirichlet-to-Neumann map for a nonuniformly elliptic problem posed on a semi-infinite cylinder in one more spatial dimension. Thus, we consider an equivalent formulation with a nonuniformly elliptic operator as the state equation. The rapid decay of the solution to this problem suggests a truncation that is suitable for numerical approximation. We discretize the proposed truncated state equation using first-degree tensor product finite elements on anisotropic meshes. For the control problem we analyze two approaches: one that is semidiscrete based on the so-called variational approach, where the control is not discretized, and the other one that is fully discrete via the discretization of the control by piecewise constant functions. For both approaches, we derive a priori error estimates with respect to degrees of freedom. Numerica...

A Convex Analytic Approach to Risk-Aware Markov Decision Processes

Abstract: In classical Markov decision process (MDP) theory, we search for a policy that, say, minimizes the expected infinite horizon discounted cost. Expectation is, of course, a risk neutral measure, which does not suffice in many applications, particularly in finance. We replace the expectation with a general risk functional, and call such models risk-aware MDP models. We consider minimization of such risk functionals in two cases, the expected utility framework, and conditional value-at-risk, a popular coherent risk measure. Later, we consider risk-aware MDPs wherein the risk is expressed in the constraints. This includes stochastic dominance constraints, and the classical chance-constrained optimization problems. In each case, we develop a convex analytic approach to solve such risk-aware MDPs. In most cases, we show that the problem can be formulated as an infinite-dimensional linear program (LP) in occupation measures when we augment the state space. We provide a discretization method and finite approximati...

Dissipative boundary conditions for one-dimensional quasi-linear hyperbolic systems: Lyapunov stability for the {$C^1$}-norm

Abstract: This paper is concerned with boundary dissipative conditions that guarantee the exponential stability of classical solutions of one-dimensional quasi-linear hyperbolic systems. We present a comprehensive review of the results that are available in the literature. The main result of the paper is then to supplement these previous results by showing how a new Lyapunov stability approach can be used for the analysis of boundary conditions that are known to be dissipative for the $C^1$-norm.

Optimal Distributed Control of a Nonlocal Convective Cahn--Hilliard Equation by the Velocity in Three Dimensions

Abstract: In this paper we study a distributed optimal control problem for a nonlocal convective Cahn--Hilliard equation with degenerate mobility and singular potential in three dimensions of space. While the cost functional is of standard tracking type, the control problem under investigation cannot easily be treated via standard techniques for two reasons: the state system is a highly nonlinear system of PDEs containing singular and degenerating terms, and the control variable, which is given by the velocity of the motion occurring in the convective term, is nonlinearly coupled to the state variable. The latter fact makes it necessary to state rather special regularity assumptions for the admissible controls, which, while looking a bit nonstandard, are, however, quite natural in the corresponding analytical framework. In fact, they are indispensable prerequisites to guarantee the well-posedness of the associated state system. In this paper, we employ recently proved existence, uniqueness, and regularity results f...

A dynamic model of barter exchange

Abstract: We consider the problem of efficient operation of a barter exchange platform for indivisible goods. We introduce a dynamic model of barter exchange where in each period one agent arrives with a single item she wants to exchange for a different item. We study a homogeneous and stochastic environment: an agent is interested in the item possessed by another agent with probability p, independently for all pairs of agents. We consider two settings with respect to the types of allowed exchanges: a) Only two-way cycles, in which two agents swap their items, b) Two or three-way cycles. The goal of the platform is to minimize the average waiting time of an agent.Somewhat surprisingly, we find that in each of these settings, a policy that conducts exchanges in a greedy fashion is near optimal, among a large class of policies that includes batching policies. Further, we find that for small p, allowing three-cycles can greatly improve the waiting time over the two-cycles only setting. Specifically, we find that a greedy policy achieves an average waiting time of Θ(1/p2) in setting a), and Θ(1/p3/2) in setting b). Thus, a platform can achieve the smallest waiting times by using a greedy policy, and by facilitating three cycles, if possible.Our findings are consistent with empirical and computational observations which compare batching policies in the context of kidney exchange programs.

Optimal Control of an Allen--Cahn Equation with Singular Potentials and Dynamic Boundary Condition

Abstract: In this paper, we investigate optimal control problems for Allen--Cahn equations with differentiable singular nonlinearities and a dynamic boundary condition involving differentiable singular nonlinearities and the Laplace--Beltrami operator. The approach covers both the cases of distributed controls and of boundary controls. The cost functional is of standard tracking type, and box constraints for the controls are prescribed. Parabolic problems with nonlinear dynamic boundary conditions involving the Laplace--Beltrami operator have recently drawn increasing attention due to their importance in applications, while their optimal control was apparently never studied before. In this paper, we first extend known well-posedness and regularity results for the state equation and then show the existence of optimal controls and that the control-to-state mapping is twice continuously Frechet differentiable between appropriate function spaces. Based on these results, we establish the first-order necessary optimality...

Minimax Lagrangian Approach to the Differentiability of Nonlinear PDE Constrained Shape Functions Without Saddle Point Assumption

Abstract: The object of this paper is the computation of the domain or boundary expression of a state constrained shape function without explicitly resorting to the material derivative. Our main theorem bypasses the restrictive saddle point assumption in existing differentiability theorems of minimax by introducing the notion of the averaged adjoint state, thus extending the use of minimax theorems to some classes of nonlinear state equations. As an illustration, the theorem is applied to a shape function that depends on a quasi-linear transmission problem. Using a Gagliardo penalization the existence of optimal shapes is established.

Minimizing the Probability of Lifetime Ruin Under Ambiguity Aversion

Abstract: We determine the optimal robust investment strategy of an individual who targets at a given rate of consumption and seeks to minimize the probability of lifetime ruin when she does not have perfect confidence in the drift of the risky asset. Using stochastic control, we characterize the value function as the unique classical solution of an associated Hamilton--Jacobi--Bellman (HJB) equation, obtain feedback forms for the optimal investment and drift distortion, and discuss their dependence on various model parameters. In analyzing the HJB equation, we establish the existence and uniqueness of a viscosity solution using Perron's method, and then upgrade regularity by working with an equivalent convex problem obtained via the Cole--Hopf transformation. We show the original value function may lose convexity for a class of parameters and the Isaacs condition may fail. Numerical examples are also included to illustrate our results.

Online Learning of Nash Equilibria in Congestion Games

Abstract: We study the repeated, nonatomic congestion game, in which multiple populations of players share resources and make, at each iteration, a decentralized decision on which resources to utilize. We investigate the following question: given a model of how individual players update their strategies, does the resulting dynamics of strategy profiles converge to the set of Nash equilibria of the one-shot game? We consider in particular a model in which players update their strategies using algorithms with sublinear discounted regret. We show that the resulting sequence of strategy profiles converges to the set of Nash equilibria in the sense of Cesaro means. However, convergence of the actual sequence is not guaranteed in general. We show that it can be guaranteed for a class of algorithms with a sublinear discounted regret and which satisfy an additional condition. We call such algorithms AREP (approximate replicator) algorithms, as they can be interpreted as a discrete-time approximation of the replicator equat...

Mean field stackelberg games: aggregation of delayed instructions ∗

Abstract: In this paper, we consider an $N$-player interacting strategic game in the presence of a (endogenous) dominating player, who gives direct influence on individual agents, through its impact on their control in the sense of Stackelberg game, and then on the whole community. Each individual agent is subject to a delay effect on collecting information, specifically at a delay time, from the dominating player. The size of his delay is completely known by the agent, while to others, including the dominating player, his delay plays as a hidden random variable coming from a common fixed distribution. By invoking a noncanonical fixed point property, we show that for a general class of finite $N$-player games, each of them converges to the mean field counterpart which may possess an optimal solution that can serve as an $\epsilon$-Nash equilibrium for the corresponding finite $N$-player game. Second, we provide, with explicit solutions, a comprehensive study on the corresponding linear quadratic mean field games of...

Strong Solutions and Strong Feller Properties for Regime-Switching Diffusion Processes in An Infinite State Space

Abstract: We establish the existence and pathwise uniqueness of regime-switching diffusion processes in an infinite state space, which could be time-inhomogeneous and state-dependent. Then the strong Feller properties of these processes are investigated by using the theory of parabolic differential equations and dimensional-free Harnack inequalities.

Structured Inverse Modeling in Parabolic Diffusion Problems

Abstract: Often, the unknown diffusivity in diffusive processes is structured by piecewise constant patches. This paper is devoted to efficient methods for the determination of such structured diffusion parameters by exploiting shape calculus. A novel shape gradient is derived in parabolic processes. Furthermore, quasi-Newton techniques are used in order to accelerate shape gradient based iterations in shape space. Numerical investigations support the theoretical results.

Dynamic Programming for General Linear Quadratic Optimal Stochastic Control with Random Coefficients

Abstract: We are concerned with the linear-quadratic optimal stochastic control problem where all the coefficients of the control system and the running weighting matrices in the cost functional are allowed to be predictable (but essentially bounded) processes and the terminal state-weighting matrix in the cost functional is allowed to be random. Under suitable conditions, we prove that the value field $\mathbb{V}(t,x,\omega), (t,x,\omega)\in [0,T]\times R^n\times \Omega$, is quadratic in $x$ and has the following form: $\mathbb{V}(t,x)=\langle \mathbb{K}_tx, x\rangle$, where $\mathbb{K}$ is an essentially bounded nonnegative symmetric matrix-valued adapted processes. Using the dynamic programming principle (DPP), we prove that $\mathbb{K}$ is a continuous semimartingale of the form $\mathbb{K}_t=\mathbb{K}_0-\int_0^t \, dk_s+\int_0^t\sum_{i=1}^d L_s^i\, dW_s^i, \ t\in [0,T]$, with $k$ being a continuous process of bounded variation and the stochastic integral $\int_0^\cdot\sum_{i=1}^d L^i_s\, dW^i_s$ being a bound...

The Principle of Least Action and Fundamental Solutions of Mass-Spring and N-Body Two-Point Boundary Value Problems

Abstract: Two-point boundary value problems for conservative systems are studied in the context of the least action principle. One obtains a fundamental solution, whereby two-point boundary value problems are converted to initial value problems via an idempotent convolution of the fundamental solution with a cost function related to the terminal data. The classical mass-spring problem is included as a simple example. The $N$-body problem under gravitation is also studied. There, the least action principle optimal control problem is converted to a differential game, where an opposing player maximizes over an indexed set of quadratics to yield the gravitational potential. Solutions are obtained as indexed sets of solutions of Riccati equations.

An Optimal Feedback Control-Strategy Pair For Zero-Sum Linear-Quadratic Stochastic Differential Game: the Riccati Equation Approach

Abstract: In this paper, we study a two-person zero-sum linear-quadratic stochastic differential game problem. From a new viewpoint, we construct an optimal feedback control-strategy pair for the game in a closed-loop form based on the solution of a Riccati equation. A key part of our analysis involves proving the global solvability of this Riccati equation, which is interesting in its own right. Moreover, we demonstrate an indefinite phenomenon arising from the linear-quadratic game.

Kernels for Linear Time Invariant System Identification

Abstract: In this paper, we study the problem of identifying the impulse response of a linear time invariant (LTI) dynamical system from the knowledge of the input signal and a finite set of noisy output observations. We adopt an approach based on regularization in a reproducing kernel Hilbert space (RKHS) that takes into account both continuous- and discrete-time systems. The focus of the paper is on designing spaces that are well suited for temporal impulse response modeling. To this end, we construct and characterize general families of kernels that incorporate system properties such as stability, relative degree, absence of oscillatory behavior, smoothness, or delay. In addition, we discuss the possibility of automatically searching over these classes by means of kernel learning techniques, so as to capture different modes of the system to be identified.

Second Order and Stability Analysis for Optimal Sparse Control of the FitzHugh--Nagumo Equation

Abstract: Optimal sparse control problems are considered for the FitzHugh--Nagumo system including the so-called Schlogl model. The nondifferentiable objective functional of tracking type includes a quadratic Tikhonov regularization term and the $L^1$-norm of the control that accounts for the sparsity. Though the objective functional is not differentiable, a theory of second order sufficient optimality conditions is established for Tikhonov regularization parameter $ u>0$ and also for the case $ u = 0$. In this context, also local minima are discussed that are strong in the sense of the calculus of variations. The second order conditions are used as the main assumption for proving the stability of locally optimal solutions with respect to $ u \to 0$ and with respect to perturbations of the desired state functions. The theory is confirmed by numerical examples that are resolved with high precision to confirm that the optimal solution obeys the system of necessary optimality conditions.

On the Existence of Optimal Policies for a Class of Static and Sequential Dynamic Teams

Abstract: In this paper, we identify sufficient conditions under which static teams and a class of sequential dynamic teams admit team-optimal solutions. We first investigate the existence of optimal solutions in static teams where the observations of the decision makers are conditionally independent given the state and satisfy certain regularity conditions. Building on these findings and the static reduction method of Witsenhausen, we then extend the analysis to sequential dynamic teams. In particular, we show that a large class of dynamic linear-quadratic-Gaussian (LQG) teams, including the vector version of the well-known Witsenhausen's counterexample and the Gaussian relay channel problem viewed as a dynamic team, admit team-optimal solutions. Results in this paper substantially broaden the class of stochastic control problems with nonclassical information known to have optimal solutions.

Networked Control Systems in the Presence of Scheduling Protocols and Communication Delays

Abstract: This paper develops the time-delay approach to networked control systems in the presence of multiple sensor nodes, communication constraints, variable transmission delays, and sampling intervals. Due to communication constraints, only one sensor node is allowed to transmit its packet at a time. The scheduling of sensor information toward the controller is ruled by a weighted try-once-discard or by round-robin protocols. A unified hybrid system model under both protocols for the closed-loop system is presented; it contains time-varying delays in the continuous dynamics and in the reset conditions. A new Lyapunov--Krasovskii method, which is based on discontinuous in time Lyapunov functionals, is introduced for the stability analysis of the delayed hybrid systems. The resulting conditions can be applied to the system with polytopic type uncertainties. The efficiency of the time-delay approach is illustrated on the examples of uncertain cart-pendulum and of batch reactor.

Distributed adaptive control for synchronization in directed complex networks

Abstract: The dynamics of a complex network is generally very complicated due to the self-dynamics of the node and their interactions. Many existing conditions for reaching certain desirable dynamics in a complex network require global information of the network, for example, the spectrum of its Laplacian matrix. A challenging problem is how the network structure affects the network dynamics in a distributed way especially for directed networks, which is still unclear today. This paper investigates the impact of the network structure on a general directed complex network and develops a scheme to change the weights in a local manner to achieve a desired behavior. In particular, network synchronization is investigated, for which some distributed adaptive laws are designed on the coupling weights for reaching synchronization in a directed complex network. It is found that the directed spanning tree structure is helpful while the aggregation and circular structures are harmful for achieving network synchronization. It ...

Poisson's equation in nonlinear filtering

Abstract: The aim of this paper is to provide a variational interpretation of the nonlinear filter in continuous time. A time-stepping procedure is introduced, consisting of successive minimization problems in the space of probability densities. The weak form of the nonlinear filter is derived via analysis of the first-order optimality conditions for these problems. The derivation shows the nonlinear filter dynamics may be regarded as a gradient flow, or a steepest descent, for a certain energy functional with respect to the Kullback--Leibler divergence. The second part of the paper is concerned with derivation of the feedback particle filter algorithm, based again on the analysis of the first variation. The algorithm is shown to be exact. That is, the posterior distribution of the particle matches exactly the true posterior, provided the filter is initialized with the true prior.

A Probabilistic Approach to Large Time Behavior of Mild Solutions of HJB Equations in Infinite Dimension

Abstract: We study the large time behaviour of mild solutions of HJB equations in infinite dimension by a purely probabilistic approach. For that purpose, we show that the solution of a BSDE in finite horizon $T$ taken at initial time behaves like a linear term in $T$ shifted with the solution of the associated EBSDE taken at initial time. Moreover we give an explicit speed of convergence, which seems to appear very rarely in literature.