Showing papers in "Siam Journal on Control and Optimization in 1984"
TL;DR: In this article, the problem of finding the Hessian of a smooth real-valued function on a convex subset of a linear space with respect to a Hilbert space norm was studied.
Abstract: This paper is concerned with the problem min $\{ f(x)\mid x \in X\} $ where X is a convex subset of a linear space H, and f is a smooth real-valued function on H. We propose the class of methods $x_{k + 1} = P(x_k - \alpha _k g_k )$, where P denotes projection on X with respect to a Hilbert space norm $\| \cdot \|$ , $g_k $ denotes the Frechet derivative of f at $x_k $ with respect to another Hilbert space norm $\| \cdot \|_k $ on H, and $\alpha _k $ is a positive scalar stepsize. We thus remove an important restriction in the original proposal of Goldstein [1] and Levitin and Poljak [2], where the norms $\| \cdot \|$ and $\| \cdot \|_k $ must be the same. It is therefore possible to match the norm $\| \cdot \|$ with the structure of X so that the projection operation is simplified while at the same time reserving the option to choose $\| \cdot \|_k $on the basis of approximations to the Hessian of f so as to attain a typically superlinear rate of convergence. The resulting methods are particularly attrac...
248 citations
TL;DR: In this article, the stochastic control problem of tracking a Brownian motion by a non-decreasing process (Monotone Follower) is related to the question of optimal stopping.
Abstract: The stochastic control problem of tracking a Brownian motion by a nondecreasing process (Monotone Follower) is related to a question of Optimal Stopping. Direct probabilistic arguments are employed to show that the two problems are equivalent, and that both admit optimal solutions.
233 citations
TL;DR: In this article, two stochastic control problems of the storage or inventory type are considered for general diffusion processes, and the objective function to be maximized is the expected discounted value of the withdrawals plus a discounted penalty for absorption.
Abstract: Two stochastic control problems of the storage or inventory type are considered for general diffusion processes. In the absorption problem, a diffusion process is controlled by subtracting a nondecreasing withdrawal process. The controlled process is absorbed if it reaches zero. The objective function to be maximized is the expected discounted value of the withdrawals plus a discounted penalty for absorption. In the reflection problem, the process can be controlled by subtracting a withdrawal process and adding a deposit process, and the controlled process must be nonnegative. One seeks to maximize an expected discounted weighted difference in withdrawals and deposits. The value function is computed, and a necessary and sufficient condition for the existence of an optimal policy is given. When they exist, optimal policies are found to be the minimal processes which keep the controlled process inside an interval.
217 citations
TL;DR: In this article, an approximation framework is presented for computation of Riccati operators that can be guaranteed to converge to the RICCati operator in feedback control for abstract evolution systems in a Hilbert space.
Abstract: An approximation framework is presented for computation (in finite imensional spaces) of Riccati operators that can be guaranteed to converge to the Riccati operator in feedback controls for abstract evolution systems in a Hilbert space. It is shown how these results may be used in the linear optimal regulator problem for a large class of parabolic systems.
206 citations
TL;DR: In this paper, a nonconvex, non-differentiable problem of optimal control where the state of the system is defined by an elliptic variational inequality with obstacle, and where the cost function is quadratic is studied.
Abstract: We study a nonconvex, nondifferentiable problem of optimal control where the state of the system is defined by an elliptic variational inequality with obstacle, and where the cost function is quadr...
202 citations
TL;DR: In this article, reachability, observability, and realizability of single-input, single-output linear time-invariant systems are discussed, in which state variables and/or input functions are restricted to be nonnegative to reflect physical constraints frequently encountered in real systems.
Abstract: This paper discusses reachability, observability, and realizability of single-input, single-output linear time-invariant systems, in which state variables and/or input (output) functions are restricted to be nonnegative to reflect physical constraints frequently encountered in real systems We define a set reachable from the origin with nonnegative inputs, and also a set observable with nonnegative outputs We investigate geometrical structures of the sets through convex analysis, and a duality relation between them is established Next we consider positive realization of a given transfer function Using the reachable set and the observable set, we give a necessary and sufficient condition for positive realizability An example is given to demonstrate that a positive realizable transfer function does not in general have a jointly controllable and observable positive realization
179 citations
TL;DR: In this article, a self-contained approach to lower semicontinuity and lower closure evolves from an extension of relaxed control theory, which is based on a central relative weak compactness criterion (called tightness) and relaxation in all but one variable.
Abstract: A self-contained approach to lower semicontinuity and lower closure evolves from an extension of relaxed control theory, which is based on a central relative weak compactness criterion (called tightness) and relaxation in all but one variable. Two lower closure results for outer integral functionals with variable abstract time domain are developed. The first of these has a convexity condition for the integrand and generalizes all similar results in the literature. The second lower closure result is of a new kind; among other things, it implies a quite general version of Fatou's lemma in several dimensions.
171 citations
TL;DR: In this article, the asymptotic convergence of the proximal point algorithm (PPA) for the solution of equations of type $0 \in Tz, where T is a multivalued maximal monotone operator in a real Hilbert space, is analyzed.
Abstract: The asymptotic convergence of the proximal point algorithm (PPA), for the solution of equations of type $0 \in Tz$, where T is a multivalued maximal monotone operator in a real Hilbert space, is analyzed. When $0 \in Tz$ has a nonempty solution set $\bar Z$, convergence rates are shown to depend on how rapidly $T^{ - 1} $ grows away from $\bar Z$ in a neighbourhood of 0. When this growth is bounded by a power function with exponent s, then for a sequence $\{ z^k \} $ generated by the PPA, $\{ | {z^k - \bar Z} |\} $ converges to zero, like $o(k^{ - {s / 2}} )$, linearly, superlinearly, or in a finite number of steps according to whether, $s \in (0,1)$, $s = 1$, $s \in (1, + \infty )$, or $s = + \infty $.
169 citations
TL;DR: In this article, the variation under perturbation of isolated local minimizers of a nonlinear and on-differentiable optimization problem is studied, and the notion of lower second-order directional derivative is introduced.
Abstract: We study the variation under perturbation of isolated local minimizers of a nonlinear and ondifferentiable optimization problem. For this we extend to the Lipschitzian case a fundamental result concerning regular points. Then we introduce the notion of lower second-order directional derivative, from which we obtain a second-order sufficiency theorem. These two results are finally used for obtaining bounds for the variations of some classes of isolated minimizers.
162 citations
TL;DR: In this article, the authors consider the problem of controlling an ordinary differential equation, subject to positive switching costs, and show in particular that the value functions form the "viscosity solution" of the dynamic programming quasi-variational inequalities.
Abstract: We consider the problem of controlling an ordinary differential equation, subject to positive switching costs, and show in particular that the value functions form the “viscosity solution” (cf. [6], [7]) of the dynamic programming quasi-variational inequalities. This interpretation allows for a rigorous application of various dynamic programming techniques.
146 citations
TL;DR: In this article, new concepts of differentiability of multifunctions are introduced and special attention is paid to relations given by perturbed inequalities, and applications are given to the study of the value (or marginal) function of a perturbed nonlinear program.
Abstract: Several new concepts of differentiability of multifunctions are introduced. Special attention is paid to relations given by perturbed inequalities. Applications are given to the study of the value (or marginal) function of a perturbed nonlinear program.
TL;DR: In this article, it was proved that for a class of parabolic distributed parameter systems with unbounded control and observation, there exists a finite dimensional compensator using dynamic output feedback.
Abstract: It is proved that for a class of parabolic distributed parameter systems with unbounded control and observation there exists a finite dimensional compensator using dynamic output feedback. Finite dimensional means here that the dynamics relating the input to the output is finite dimensional. A constructive design algorithm is presented and several examples are considered.
TL;DR: In this paper, the authors show how the structure of the recursive nonlinear filtering problem leads naturally to the use of methods from nonlinear system theory and the theory of Lie algebras.
Abstract: The purpose of this paper is to show how the structure of the recursive nonlinear filtering problem leads naturally to the use of methods from nonlinear system theory and the theory of Lie algebras...
TL;DR: This paper deals with Ritz–Galerkin approximations of the following two problems: boundary-value problems with $L_2 $-boundary data given in the form of Dirichlet boundary conditions and time optimal control problems for parabolic systems with control acting on the boundary.
Abstract: This paper deals with Ritz–Galerkin approximations of the following two problems: (i) boundary-value problems with $L_2 $-boundary data given in the form of Dirichlet boundary conditions; (ii) time...
TL;DR: In this article, a dynamic compensator is designed on the basis of a finite-dimensional modal model of the system, and a feedback control system is constructed by using sensor outputs.
Abstract: Feedback control is developed for a class of distributed systems described by second order evolution equations with slight damping. In order to increase the degree of stability of the system, a dynamic compensator is designed on the basis of a finite-dimensional model of the system, and a feedback control system is constructed by using sensor outputs. It is shown that the degree of stability of the whole system including the compensator is improved by “modal control” based on a finite-dimensional modal model. This proves the mathematical validity of the modal control of distributed systems.
TL;DR: In this article, an infinite horizon stochastic production planning problem with the constraint that production rate must be nonnegative is considered and an optimal feedback solution exists for the problem, which is characterized and is then compared with the solution of the unconstrained problem.
Abstract: This paper considers an infinite horizon stochastic production planning problem with the constraint that production rate must be nonnegative. It is shown that an optimal feedback solution exists for the problem. Moreover, this solution is characterized and is then compared with the solution of the unconstrained problem. Also obtained, by using a policy iteration procedure, are computational solutions to the related problems with upper bounds on the production rate. which determines production rates over time to minimize an integral representing a discounted quadratic loss function. The loss function is defined in terms of the deviations of production and inventory levels from their rated or factory-optimal values. The model is solved both with and without nonnegative production constraints. A stochastic extension of this paper involving a white noise process (1) is analyzed by Sethi and Thompson (9), (10). Closed-form solutions for optimal feedback production policy for both finite and infinite horizon versions of the model without production constraints are obtained. In particular, the model must allow negative production rates or disposals. In this paper, we consider the stochastic production planning problem with the constraint that production rates must be nonnegative. Only the infinite horizon problem is treated. It is shown that an optimal feedback solution exists for the problem. This solution is characterized. Also obtained, by using a policy iteration procedure, are computational solutions to the related problems with upper bounds on the production rate.
TL;DR: In this article, it was shown that the notion of controllability introduced by Kalman (K-controllability) is equivalent to the characterization of controLLability (H-Controllability), which was first proposed by Hewer as a natural extension to the periodic case of an eigenvalue-eigenvector condition independently introduced by several authors for time-invariant systems.
Abstract: With reference to linear periodic systems, the classical notion of controllability introduced by Kalman (K-controllability) is shown to be equivalent to the characterization of controllability (H-controllability) first proposed by Hewer as a natural extension to the periodic case of an eigenvalue-eigenvector controllability condition independently introduced by several authors for time-invariant systems only. The proof of such an equivalence also leads to the conclusion that if a T-periodic system S is K-controllable and k is the degree of the minimal polynomial of the monodromy matrix associated with S, then, at any time t, S is controllable over $(t,t + kT)$. Since k is lower than or equal to the system order, a well-known result due to Brunovsky is slightly strengthened. By duality, the corresponding observability results follow.
TL;DR: In this article, a weaker than persistent excitation condition is used to establish the strong consistency of MLS for MIMO stochastic systems without monitoring, and a comparison between various conditions for consistency is given.
Abstract: In this paper we first use a weaker than persistent excitation condition to establish the strong consistency of MLS for MIMO stochastic systems without monitoring, and give a comparison between various conditions for consistency. Then we show the global convergence for adaptive tracking based on MLS and finally the adaptive tracking and strong consistency results are combined.
TL;DR: In this paper, the minimum cost per unit time (MPUT) control problem is studied for a class of Markov chains that does not fit the conventional framework for this problem.
Abstract: The “minimum cost per unit time” control problem is studied for a class of Markov chains that, though important in applications, does not fit the conventional framework for this problem. Existence of optimal stationary strategies and necessary and sufficient conditions for optimality are established.
TL;DR: In this paper, the authors proved global existence for the solution of a Riccati differential equation connected with the synthesis of a boundary control problem governed by parabolic partial differential equations.
Abstract: Global existence is proved for the solution, of a Riccati differential equation connected with the synthesis of a boundary control problem governed by parabolic partial differential equations.
TL;DR: A dual approximation for the solution to an optimal control problem is analyzed with the differential equation handled with a Lagrange multiplier while other constraints are treated explicitly.
Abstract: A dual approximation for the solution to an optimal control problem is analyzed. The differential equation is handled with a Lagrange multiplier while other constraints are treated explicitly. An algorithm for solving the dual problem is presented.
TL;DR: In this article, a complete optimality condition in the standard inverse problem of optimal control for the multi-input case is established and the optimality of a feedback control law is characterized completely in terms of a new geometric condition as well as the well-known return difference condition.
Abstract: A complete optimality condition in the standard inverse problem of optimal control for the multi-input case is established. The optimality of a feedback control law is characterized completely in terms of a new geometric condition as well as the well-known return difference condition. The new condition not only provides a better insight into the well-known sensitivity reduction property of optimal control for the multi-input case, but also indicates an essential difference between the solutions of the single- and multi-input inverse problem.
TL;DR: In this paper, the pole placement and stabilization for generic linear systems with prescribed state, input and output dimensions are considered. And the rationality and solvability by radicals of pole positioning gains are discussed.
Abstract: We consider questions of pole placement and stabilization for generic linear systems with prescribed state, input and output dimensions, where the controller must be implemented by linear memoryless output feedback. We present a criterion, in terms of a special pole placement property, for generic stabilizability and apply this to describe constraints on the dimensions which are consistent with generic stabilizability. We also discuss the rationality and solvability by radicals of stabilizing or pole positioning gains, and we describe how decision algebra can theoretically handle existence questions for generic systems.
TL;DR: In this paper, a class of quasidifferentiable functions whose directional derivatives are representable as a difference of two sublinear functions is introduced, and an optimization problem subject to quasi-ifferentiable equality and inequality constraints is studied.
Abstract: A class of quasidifferentiable functions, whose directional derivatives are representable as a difference of two sublinear functions, is introduced. An optimization problem subject to quasidifferentiable equality and inequality constraints is studied. Optimality conditions for this problem are given in terms of sets inclusion. This refines the results of Demyanov, Rubinov and co-workers.
TL;DR: In this paper, the exact reachability and approximate controllability of a semilinear control system with an abstract algebraic abstract equation was studied. But the authors focused on the problem of finding the minimum norm optimal control and the construction of the reachable set of the corresponding linear system.
Abstract: We consider control systems described by a semilinear abstract equation $y' + Ay = F(y,u) + Bu$ in Hilbert space. General conditions for exact reachability and approximate controllability are given which are related with two families of associated quadratic optimal control problems. The minimum norm optimal control and the construction of the reachable set of the corresponding linear system $y' + Ay = Bu$ are characterized respectively. Exact reachability for a class of semilinear control systems is obtained under some assumptions on the range of a nonlinear operator $\mathcal{F}_{(t_0 ,T)} $) (see definition in §4) generated by the nonlinear function F.
TL;DR: In this paper, the authors consider affine families of vector fields on a finite dimensional vector space V and show that such families are necessarily transitive on V, and also show that they remain transitive under small perturbations.
Abstract: This paper deals with transitivity (controllability) of affine families of vector fields on a finite dimensional vector space V. In particular we focus on affine families whose corresponding families of linear fields are transitive on $V - \{ 0\} $, and which in addition have no fixed points in V. We show that such families are necessarily transitive on V, and we also show that they remain transitive under small perturbations. In general, however, affine families need not remain transitive under small perturbations—for example, small affine perturbations of transitive linear systems are not necessarily transitive. Since any affine system $\mathcal{F}$ naturally defines a system $\mathcal{F}_r $ of right-invariant vector fields on the semi-direct product of V with $GL(V)$ we also investigate transitivity properties $\mathcal{F}_r $. Our result is that if $\mathcal{F}$ is an affine family satisfying the preceding conditions then $\mathcal{F}_r $ generates the full Lie algebra on the semi-direct product.
TL;DR: In this article, the authors extend the results of Rothschild and Stein [Acts. Math., 137 (1976), pp. 247-320] and Goodman [Lecture Notes in Mathematics 562, Springer-Verlag, New York, 1976], in which a finite set of vector fields $g_1, \cdots,g_m $ are lifted to vector fields approximated by generators of a free nilpotent Lie algebra.
Abstract: This paper is concerned with extending certain results of Rothschild and Stein [Acts. Math., 137 (1976), pp. 247–320] and Goodman [Lecture Notes in Mathematics 562, Springer-Verlag, New York, 1976], in which a finite set of vector fields $g_1 , \cdots ,g_m $ are lifted to vector fields approximated by generators of a free nilpotent Lie algebra. We wish to add a vector field f, to the set $g_1 , \cdots ,g_m $ and lift these vector fields to ones on a finite dimensional vector space V, approximated by generators F, $G_1 , \cdots ,G_m $ of a solvable Lie algebra, in which $ad^j F(G_i )$ generate a nilpotent, but not free, ideal. This procedure is accomplished in the context of a nonlinear control system, with outputs, in which f vanishes at the initial state, and in such a way that the output functions lift to the state space V, to define a system whose input output map is the same as the original system. The approximating system is obtained from a suitable realization of a truncation of the Volterra series ...
TL;DR: In this paper, the authors consider the control of a particular one-dimensional diffusion process over a finite time interval and consider the case where the diffusion coefficient is chosen by the rule: choose the smaller diffusion coefficient if and only if the current state is nonnegative.
Abstract: We consider the control of a particular one-dimensional diffusion process over a finite time interval $[0,T]$. Two drift-diffusion pairs $(\mu _1 ,\sigma _1 )$ and $(\mu _2 ,\sigma _2 )$ are given and the process is controlled by switching between these pairs. The objective is to maximize the probability that the process lies in the half line $[0,\infty )$ at final time T.The case where $\mu _1 = \mu _2 = 0$ is considered first. Let the control $\sigma _0 $ be given by the rule: choose the smaller diffusion coefficient if and only if the current state is nonnegative. A result is proved which, loosely speaking, says that $\sigma _0 $ is optimal in this special case, and remains optimal even if we know the final value of the driving Brownian motion in advance.The general problem (with drift) is then solved by an application of this result and the Girsanov transformation.
TL;DR: In this article, a fundamental relationship between appropriate pairs of polynomial matrices is presented, termed canonical Diophantine equations, which can be used to resolve a number of standard matrix problems.
Abstract: A fundamental relationship between appropriate pairs of polynomial matrices is presented. This relationship, termed canonical Diophantine equations, can be used to resolve a number of standard polynomial matrix problems. Here, the general Diophantine equation is constructively resolved in a unique minimal way; in addition, prime canonical factorizations of a system transfer matrix are derived from knowledge of any dual factorization.
TL;DR: In this article, an optimal control of a Markov process submitted to a sequence of changes of state and law at times of constant impulse cost and continuous cost is studied, where the cost is made with an impulse cost which is positive and a continuous cost.
Abstract: This paper is concerned with the optimal control of a Markov process which is submitted to a sequence of changes of state and law at times $T_1 ,T_2 , \cdots ,T_n \cdots $. The cost is made with an impulse cost which is positive, and a continuous cost. The modelization is deeply inspired by the theory of renewal for Markov processes by P. A. Meyer. The main result is the Markovian form of the value function of this problem. Then under smooth additional conditions, we prove the existence of an optimal control.