Showing papers in "Siam Journal on Control and Optimization in 2000"
TL;DR: It is shown that the regularity properties of the Lyapunov function and those of the settling-time function are related and converse Lyap Unov results can only assure the existence of continuous Lyap unov functions.
Abstract: Finite-time stability is defined for equilibria of continuous but non-Lipschitzian autonomous systems. Continuity, Lipschitz continuity, and Holder continuity of the settling-time function are studied and illustrated with several examples. Lyapunov and converse Lyapunov results involving scalar differential inequalities are given for finite-time stability. It is shown that the regularity properties of the Lyapunov function and those of the settling-time function are related. Consequently, converse Lyapunov results can only assure the existence of continuous Lyapunov functions. Finally, the sensitivity of finite-time-stable systems to perturbations is investigated.
3,894 citations
TL;DR: A modification to the forward-backward splitting method for finding a zero of the sum of two maximal monotone mappings is proposed, under which the method converges assuming only the forward mapping is (Lipschitz) continuous on some closed convex subset of its domain.
Abstract: We consider the forward-backward splitting method for finding a zero of the sum of two maximal monotone mappings. This method is known to converge when the inverse of the forward mapping is strongly monotone. We propose a modification to this method, in the spirit of the extragradient method for monotone variational inequalities, under which the method converges assuming only the forward mapping is (Lipschitz) continuous on some closed convex subset of its domain. The modification entails an additional forward step and a projection step at each iteration. Applications of the modified method to decomposition in convex programming and monotone variational inequalities are discussed.
935 citations
TL;DR: It is shown here that Stability of the stochastic approximation algorithm is implied by the asymptotic stability of the origin for an associated ODE, which implies convergence of the algorithm.
Abstract: It is shown here that stability of the stochastic approximation algorithm is implied by the asymptotic stability of the origin for an associated ODE. This in turn implies convergence of the algorithm. Several specific classes of algorithms are considered as applications. It is found that the results provide (i) a simpler derivation of known results for reinforcement learning algorithms; (ii) a proof for the first time that a class of asynchronous stochastic approximation algorithms are convergent without using any a priori assumption of stability; (iii) a proof for the first time that asynchronous adaptive critic and Q-learning algorithms are convergent for the average cost optimal control problem.
515 citations
TL;DR: A stochastic calculus is given for the fractional Brownian motions that have the Hurst parameter in (1/2, 1) so that the integral has zero mean and an explicit expression for the second moment.
Abstract: In this paper a stochastic calculus is given for the fractional Brownian motions that have the Hurst parameter in (1/2, 1). A stochastic integral of Ito type is defined for a family of integrands so that the integral has zero mean and an explicit expression for the second moment. This integral uses the Wick product and a derivative in the path space. Some Ito formulae (or change of variables formulae) are given for smooth functions of a fractional Brownian motion or some processes related to a fractional Brownian motion. A stochastic integral of Stratonovich type is defined and the two types of stochastic integrals are explicitly related. A square integrable functional of a fractional Brownian motion is expressed as an infinite series of orthogonal multiple integrals.
471 citations
TL;DR: An asymptotic expansion of a design functional with respect to the creation of a small hole is obtained by using an adaptation of the adjoint method and a domain truncation technique for linear elasticity for general functionals and arbitrary shaped holes.
Abstract: The aim of the topological sensitivity analysis is to obtain an asymptotic expansion of a design functional with respect to the creation of a small hole. In this paper, such an expansion is obtained and analyzed in the context of linear elasticity for general functionals and arbitrary shaped holes by using an adaptation of the adjoint method and a domain truncation technique. The method is general and can be easily adapted to other linear PDEs and other types of boundary conditions.
429 citations
TL;DR: The problem of solving LQR models with system diffusions dependent on both state and control variables, which is left open in part I, is tackled and an optimal control is explicitly constructed.
Abstract: In part I of this paper [S. Chen, X. Li, and X. Zhou, SIAM J. Control Optim., 36 (1998), pp. 1685--1702], an optimization model of stochastic linear quadratic regulators (LQRs) with indefinite control cost weighting matrices is proposed and studied. In this sequel, the problem of solving LQR models with system diffusions dependent on both state and control variables, which is left open in part I, is tackled. First, the solvability of the associated stochastic Riccati equations (SREs) is studied in the normal case (namely, all the state and control weighting matrices and the terminal matrix in the cost functional are nonnegative definite, with at least one positive definite), which in turn leads to an optimal state feedback control of the LQR problem. In the general indefinite case, the problem is decomposed into two optimal LQR problems, one with a forward dynamics and the other with a backward dynamics. The well-posedness and solvability of the original LQR problem are then obtained by solving these two subproblems, and an optimal control is explicitly constructed. Examples are presented to illustrate the results.
385 citations
TL;DR: A new approach to error control and mesh adaptivity is described for the discretization of optimal control problems governed by elliptic partial differential equations and the computed state and costate variables can be used as sensitivity factors multiplying the local cell-residuals in the error estimators.
Abstract: A new approach to error control and mesh adaptivity is described for the discretization of optimal control problems governed by elliptic partial differential equations. The Lagrangian formalism yields the first-order necessary optimality condition in form of an indefinite boundary value problem which is approximated by an adaptive Galerkin finite element method. The mesh design in the resulting reduced models is controlled by residual-based a posteriori error estimates. These are derived by duality arguments employing the cost functional of the optimization problem for controlling the discretization error. In this case, the computed state and costate variables can be used as sensitivity factors multiplying the local cell-residuals in the error estimators. This results in a generic and simple algorithm for mesh adaptation within the optimization process. This method is developed and tested for simple boundary control problems in semiconductor models.
319 citations
TL;DR: It is proved that if the potential is bounded from below, then the solution trajectories are minimizing for it and converge weakly towards a minimizer of $\Phi$ if one exists; this convergence is strong when $\ Phi$ is even or when the optimal set has a nonempty interior.
Abstract: We study the asymptotic behavior at infinity of solutions of a second order evolution equation with linear damping and convex potential. The differential system is defined in a real Hilbert space. It is proved that if the potential is bounded from below, then the solution trajectories are minimizing for it and converge weakly towards a minimizer of $\Phi$ if one exists; this convergence is strong when $\Phi$ is even or when the optimal set has a nonempty interior. We introduce a second order proximal-like iterative algorithm for the minimization of a convex function. It is defined by an implicit discretization of the continuous evolution problem and is valid for any closed proper convex function. We find conditions on some parameters of the algorithm in order to have a convergence result similar to the continuous case.
312 citations
TL;DR: This paper investigates relaxations of parameterized L MI problems into standard LMI problems using techniques relying on directional convexity concepts, and discusses the impact of the proposed techniques in quadratic programming, Lyapunov-based stability and performance analysis, $\mu$ analysis, and linear parameter-varying control.
Abstract: A wide variety of problems in control system theory fall within the class of parameterized linear matrix inequalities (LMIs), that is, LMIs whose coefficients are functions of a parameter confined to a compact set. Such problems, though convex, involve an infinite set of LMI constraints and hence are inherently difficult to solve numerically.
This paper investigates relaxations of parameterized LMI problems into standard LMI problems using techniques relying on directional convexity concepts. An in-depth discussion of the impact of the proposed techniques in quadratic programming, Lyapunov-based stability and performance analysis, $\mu$ analysis, and linear parameter-varying control is provided. Illustrative examples are given to demonstrate the usefulness and practicality of the approach.
268 citations
TL;DR: An LMI-based procedure is proposed which is a sequential linearization programming approach which can be necessarily restated as a nonconvex optimization problem with a bilinear, multiobjective functional under suitably chosen linear matrix inequality (LMI) constraints.
Abstract: We consider the problem of designing a suboptimal ${\cal H}_2/{\cal H}_{\infty}$ feedback control law for a linear time-invariant control system when a complete set of state variables is not available. This problem can be necessarily restated as a nonconvex optimization problem with a bilinear, multiobjective functional under suitably chosen linear matrix inequality (LMI) constraints. To solve such a problem, we propose an LMI-based procedure which is a sequential linearization programming approach. The properties and the convergence of the algorithm are discussed in detail. Finally, several numerical examples for static ${\cal H}_2/{\cal H}_{\infty}$ output feedback problems demonstrate the applicability of the considered algorithm and also verify the theoretical results numerically.
252 citations
TL;DR: Utility maximization problems of mixed optimal stopping/control type are considered, which can be solved by reduction to a family of related pure optimal stopping problems, including one which demonstrates that optimal strategies need not always exist.
Abstract: Utility maximization problems of mixed optimal stopping/control type are considered, which can be solved by reduction to a family of related pure optimal stopping problems. Sufficient conditions for the existence of optimal strategies are provided in the context of continuous-time, Ito process models for complete markets. The mathematical tools used are those of optimal stopping theory, continuous-time martingales, convex analysis, and duality theory. Several examples are solved explicitly, including one which demonstrates that optimal strategies need not always exist.
TL;DR: In this paper, necessary and sufficient characterizations of several notions of input to output stability were presented, and the results given here extend their validity to the case when the output, but not necessarily the entire internal state, is being regulated.
Abstract: This paper presents necessary and sufficient characterizations of several notions of input to output stability. Similar Lyapunov characterizations have been found to play a key role in the analysis of the input to state stability property, and the results given here extend their validity to the case when the output, but not necessarily the entire internal state, is being regulated.
TL;DR: In this article, Lyapunov characterizations of the input-output-to-state stability (IOSS) property for nonlinear systems are explored. But the main contribution of this paper is to establish a complete equivalence between the IOSS property and the existence of a certain type of smooth LyapUNF function, which can be seen as a natural generalization of the zero-detectability property used in the linear case.
Abstract: This work explores Lyapunov characterizations of the input-output-to-state stability (IOSS) property for nonlinear systems The notion of IOSS is a natural generalization of the standard zero-detectability property used in the linear case The main contribution of this work is to establish a complete equivalence between the IOSS property and the existence of a certain type of smooth Lyapunov function As corollaries, one shows the existence of "norm-estimators," and obtains characterizations of nonlinear detectability in terms of relative stability and of finite-energy estimates
TL;DR: This paper attempts to explore the relationship between BSDEs and stochastic controls by interpreting BSDE’s as some Stochastic optimal control problems, and a modified model is investigated, where the difference between the state and the expectation of the given terminal value at any time is taken into account.
Abstract: It is well known that backward stochastic differential equations (BSDEs) stem from the study on the Pontryagin type maximum principle for optimal stochastic controls. A solution of a BSDE hits a given terminal value (which is a random variable) by virtue of an it additional martingale term and an indefinite initial state. This paper attempts to explore the relationship between BSDEs and stochastic controls by interpreting BSDEs as some stochastic optimal control problems. More specifically, associated with a BSDE, a new stochastic control problem is introduced with the same dynamics but a definite given initial state. The martingale term in the original BSDE is regarded as the control, and the objective is to minimize the second moment of the difference between the terminal state and the terminal value given in the BSDE. This problem is solved in a closed form by the stochastic linear-quadratic (LQ) theory developed recently. The general result is then applied to the Black--Scholes model, where an optimal mean-variance hedging portfolio is obtained explicitly in terms of the option price. Finally, a modified model is investigated, where the difference between the state and the expectation of the given terminal value at any time is taken into account.
TL;DR: It is shown that a sequence of generalized eigenfunctions of an Euler--Bernoulli beam equation with a tip mass under boundary linear feedback control forms a Riesz basis for the state Hilbert space.
Abstract: Using an abstract condition of Riesz basis generation of discrete operators in the Hilbert spaces, we show, in this paper, that a sequence of generalized eigenfunctions of an Euler--Bernoulli beam equation with a tip mass under boundary linear feedback control forms a Riesz basis for the state Hilbert space. In the meanwhile, an asymptotic expression of eigenvalues and the exponential stability are readily obtained. The main results of [ SIAM J. Control Optim., 36 (1998), pp. 1962--1986] are concluded as a special case, and the additional conditions imposed there are removed.
TL;DR: Given a locally defined, nondifferentiable but Lipschitz Lyapunov function, it is established that the feedback in question possesses a robustness property relative to measurement error, despite the fact that it may not be continuous.
Abstract: Given a locally defined, nondifferentiable but Lipschitz Lyapunov function, we employ it in order to construct a (discontinuous) feedback law which stabilizes the underlying system to any given tolerance. A converse result shows that suitable Lyapunov functions of this type exist under mild assumptions. We also establish that the feedback in question possesses a robustness property relative to measurement error, despite the fact that it may not be continuous.
TL;DR: It is shown that by a suitable change of variables, both design problems can be converted into convex programming problems written in terms of linear matrix inequalities (LMI) and the optimal guaranteed performance H2 and $H_\infty$ filters are proven to be of the same order as the order of the system.
Abstract: This paper investigates robust filtering design problems in H2 and $H_\infty$ spaces for discrete-time systems subjected to parameter uncertainty which is assumed to belong to a convex bounded polyhedral domain. It is shown that, by a suitable change of variables, both design problems can be converted into convex programming problems written in terms of linear matrix inequalities (LMI). The results generalize the ones available in the literature to date in several directions. First, all system matrices can be corrupted by parameter uncertainty and the admissible uncertainty may be structured. Then, assuming the order of the uncertain system is known, the optimal guaranteed performance H2 and $H_\infty$ filters are proven to be of the same order as the order of the system. Comparisons with robust filters for systems subjected to norm-bounded uncertainty are provided in both theoretical and practical settings. In particular, it is shown that under the same assumptions the results here are generally better as far as the minimization of a guaranteed cost expressed in terms of H2 or $H_\infty$ norms is considered. Some numerical examples illustrate the theoretical results.
TL;DR: It is shown that for any asymptotically controllable homogeneous system in euclidean space there exists a homogeneous control Lyapunov function and aHomogeneous, possibly discontinuous state feedback law stabilizing the corresponding sampled closed loop system.
Abstract: We show that for any asymptotically controllable homogeneous system in euclidean space (not necessarily Lipschitz at the origin) there exists a homogeneous control Lyapunov function and a homogeneous, possibly discontinuous state feedback law stabilizing the corresponding sampled closed loop system. If the system satisfies the usual local Lipschitz condition on the whole space we obtain semiglobal stability of the sampled closed loop system for each sufficiently small fixed sampling rate. If the system satisfies a global Lipschitz condition we obtain global exponential stability for each sufficiently small fixed sampling rate. The control Lyapunov function and the feedback are based on the Lyapunov exponents of a suitable auxiliary system and admit a numerical approximation.
TL;DR: The Hamilton--Jacobi--Bellman (HJB) equation associated with the {robust/\hinfty} filter (as well as the Mortensen filter) is considered.
Abstract: The Hamilton--Jacobi--Bellman (HJB) equation associated with the {robust/\hinfty} filter (as well as the Mortensen filter) is considered. These filters employ a model where the disturbances have finite power. The HJB equation for the filter information state is a first-order equation with a term that is quadratic in the gradient. Yet the solution operator is linear in the max-plus algebra. This property is exploited by the development of a numerical algorithm where the effect of the solution operator on a set of basis functions is computed off-line. The precomputed solutions are stored as vectors of coefficients of the basis functions. These coefficients are then used directly in the real-time computations.
TL;DR: A complete proof of the corresponding sufficiency theorem is given, a slightly weaker version of which had been stated in an earlier article, with only a rough outline of the proof.
Abstract: We propose a definition of "regular synthesis" that is more general than those suggested by other authors such as Boltyanskii [SIAM J. Control Optim, 4 (1966), pp. 326--361] and Brunovský [Math. Slovaca, 28 (1978), pp. 81--100], and an even more general notion of "regular presynthesis." We give a complete proof of the corresponding sufficiency theorem, a slightly weaker version of which had been stated in an earlier article, with only a rough outline of the proof. We illustrate the strength of our result by showing that the optimal synthesis for the famous Fuller problem satisfies our hypotheses. We also compare our concept of synthesis with the simpler notion of a "family of solutions of the closed-loop equation arising from an optimal feedback law," and show by means of examples why the latter is inadequate, and why the difficulty cannot be resolved by using other concepts of solution---such as Filippov solutions, or the limits of sample-and-hold solutions recently proposed as feedback solutions by Clarke et al. [IEEE Trans. Automat. Control, 42 (1997), pp. 1394--1407]---for equations with a non-Lipschitz and possibly discontinuous right-hand side.
TL;DR: This work studies the energy decay of a Bernoulli--Euler beam which is subject to a pointwise feedback force and deduces decay estimates from observability inequalities for the associated undamped problem via sharp trace regularity results.
Abstract: We study the energy decay of a Bernoulli--Euler beam which is subject to a pointwise feedback force. We show that both uniform and nonuniform energy decay may occur. The uniform or nonuniform decay depends on the boundary conditions. In the case of nonuniform decay in the energy space we give explicit polynomial decay estimates valid for regular initial data. Our method consists of deducing the decay estimates from observability inequalities for the associated undamped problem via sharp trace regularity results.
TL;DR: The asymptotic stability is studied and the energy decay rates for solutions of the wave equation with boundary dissipation of the memory type are given.
Abstract: We study the asymptotic stability and give the energy decay rates for solutions of the wave equation with boundary dissipation of the memory type.
TL;DR: Given a locally Lipschitz control system which is globally asymptotically controllable at the origin, a control-Lyapunov function is constructed and the existence of another one which is semiconcave outside the origin is deduced.
Abstract: Given a locally Lipschitz control system which is globally asymptotically controllable at the origin, we construct a control-Lyapunov function for the system which is Lipschitz on bounded sets, and we deduce the existence of another one which is semiconcave (and so locally Lipschitz) outside the origin. The proof relies on value functions and nonsmooth calculus.
TL;DR: This model includes standard mathematical programs with parameter-dependent variational inequalities or complementarity problems as side constraints, and uses Mordukhovich's generalized differential calculus to derive necessary optimality conditions and apply them to problems, where the equilibria are governed by implicit complementarity issues and by hemivariational inequalities.
Abstract: The paper concerns an optimization problem with a generalized equation among the constraints. This model includes standard mathematical programs with parameter-dependent variational inequalities or complementarity problems as side constraints. Using Mordukhovich's generalized differential calculus, we derive necessary optimality conditions and apply them to problems, where the equilibria are governed by implicit complementarity problems and by hemivariational inequalities.
TL;DR: Some systematic approaches to the mathematical formulation and numerical approximation of the time-dependent optimal control problem of tracking the velocity for Navier--Stokes flows in a bounded, two-dimensional domain with boundary control are presented.
Abstract: We present some systematic approaches to the mathematical formulation and numerical approximation of the time-dependent optimal control problem of tracking the velocity for Navier--Stokes flows in a bounded, two-dimensional domain with boundary control. We study the existence of optimal solutions and derive an optimality system from which optimal solutions may be determined. We also define and analyze semidiscrete-in-time and full space-time discrete approximations of the optimality system and a gradient method for the solution of the fully discrete system. The results of some computational experiments are provided.
TL;DR: It is demonstrated that given these conditions increased stochastic fluctuations decrease the value and increase the optimal threshold, thus postponing the exercise of the irreversible policy.
Abstract: We consider a class of singular stochastic control problems arising frequently in applications of stochastic control. We state a set of conditions under which the optimal policy and its value can be derived in terms of the minimal r-excessive functions of the controlled diffusion, and demonstrate that the optimal policy is of the standard local time type. We then state a set of weak smoothness conditions under which the value function is increasing and concave, and demonstrate that given these conditions increased stochastic fluctuations decrease the value and increase the optimal threshold, thus postponing the exercise of the irreversible policy. In line with previous studies of singular stochastic control, we also establish a connection between singular control and optimal stopping, and show that the marginal value of the singular control problem coincides with the value of the associated stopping problem whenever 0 is not a regular boundary for the controlled diffusion.
TL;DR: It is shown that the optimal strategy is of the form of a knock- out option with payoff C, where the "domain of the knock-out" depends on the value of the optimal dual variable.
Abstract: We study the problem of minimizing the expected discounted loss $$ E\left[e^{-\int_0^Tr(u)du}( C- X^{x,\pi}(T))^+\right] $$ when hedging a liability C at time t=T, using an admissible portfolio strategy $\pi(\cdot)$ and starting with initial wealth x. The existence of an optimal solution is established in the context of continuous-time Ito process incomplete market models, by studying an appropriate dual problem. It is shown that the optimal strategy is of the form of a knock-out option with payoff C, where the "domain of the knock-out" depends on the value of the optimal dual variable. We also discuss a dynamic measure for the risk associated with the liability C, defined as the supremum over different scenarios of the minimal expected loss of hedging C.
TL;DR: This paper obtains directly two observability inequalities for the linear wave equation with time-variant nonsmooth lower order terms with constant estimated by an explicit function of the norm of the involved coefficients in the equation.
Abstract: In this paper, by means of Carleman estimates and the usual energy estimate, we obtain directly two observability inequalities for the linear wave equation with time-variant nonsmooth lower order terms. We do not need any unique continuation property of the linear equation a priori, since this is actually one of the by-products of our analysis. Furthermore, the constant in the observability inequality is estimated by an explicit function of the norm of the involved coefficients in the equation. Also, we apply our observability estimates to exact controllability for wave equations.
TL;DR: The strong convergence of a broad class of outer approximation methods for minimizing a convex function over the intersection of an arbitrary number of convex sets in a reflexive Banach space is studied in a unified framework.
Abstract: The strong convergence of a broad class of outer approximation methods for minimizing a convex function over the intersection of an arbitrary number of convex sets in a reflexive Banach space is studied in a unified framework. The generic outer approximation algorithm under investigation proceeds by successive minimizations over the intersection of convex supersets of the feasibility set determined in terms of the current iterate and variable blocks of constraints. The convergence analysis involves flexible constraint approximation and aggregation techniques as well as relatively mild assumptions on the constituents of the problem. Various well-known schemes are recovered as special realizations of the generic algorithm and parallel block-iterative extensions of these schemes are devised within the proposed framework. The case of inconsistent constraints is also considered.
TL;DR: This work shows how the extended linear complementarity problem (ELCP), which is a mathematical programming problem, can be used to describe the set of system trajectories of a first order linear hybrid system with saturation, and shows that if there is no upper saturation, then the globally optimal switching time sequence can be computed very efficiently.
Abstract: We consider a class of first order linear hybrid systems with saturation. A system that belongs to this class can operate in several modes or phases; in each phase each state variable of the system exhibits a linear growth until a specified upper or lower saturation level is reached, and after that the state variable stays at that saturation level until the end of the phase. A typical example of such a system is a traffic signal controlled intersection. We develop methods to determine optimal switching time sequences for first order linear hybrid systems with saturation that minimize criteria such as average queue length, worst case queue length, average waiting time, and so on. First we show how the extended linear complementarity problem (ELCP), which is a mathematical programming problem, can be used to describe the set of system trajectories of a first order linear hybrid system with saturation. Optimization over the solution set of the ELCP then yields an optimal switching time sequence. Although this method yields globally optimal switching time sequences, it is not feasible in practice due to its computational complexity. Therefore, we also present some methods to compute suboptimal switching time sequences. Furthermore, we show that if there is no upper saturation, then for some objective functions the globally optimal switching time sequence can be computed very efficiently. We also discuss some approximations that lead to suboptimal switching time sequences that can be computed very efficiently. Finally, we use these results to design optimal switching time sequences for traffic signal controlled intersections.