Constrained optimal control of bilinear systems using neural network based HJB solution
01 Jun 2008-pp 4137-4142
TL;DR: In this paper, a Hamilton-Jacobi-Bellman (HJB) equation based optimal control algorithm is proposed for a bilinear system and the controller is shown to be optimal with respect to a cost functional.
Abstract: In this paper, a Hamilton-Jacobi-Bellman (HJB) equation based optimal control algorithm is proposed for a bilinear system. Utilizing the Lyapunov direct method, the controller is shown to be optimal with respect to a cost functional, which includes penalty on the control effort and the system states. In the proposed algorithm, Neural Network (NN) is used to find approximate solution of HJB equation using least squares method. Proposed algorithm has been applied on bilinear systems. Necessary theoretical and simulation results are presented to validate proposed algorithm.
Citations
More filters
TL;DR: A new architecture, called the “cost function-based single network adaptive critic” is presented that eliminates one of the networks, applicable to a wide class of nonlinear systems in engineering where the optimal control equation can be explicitly expressed in terms of the state and cost-related variables.
Abstract: Approximate dynamic programming formulation implemented with an “adaptive critic-based” neural network structure has been shown to be a powerful technique to solve theHamilton–Jacobi–Bellman equations. As interest in this technique grows, it is important to consider the enabling factors for their possible implementations. A typical adaptive critic structure consists of two interacting neural networks; in this paper, a new architecture, called the “cost function-based single network adaptive critic” is presented that eliminates one of the networks. This approach is applicable to a wide class of nonlinear systems in engineering where the optimal control equation can be explicitly expressed in terms of the state and cost-related variables. After the “training,” the output of the neural network represents the optimal cost. Optimal control is obtained by finding the derivatives of the output of the network with respect to its input and using it in the expression for optimal control. In practical applications, there usually exist uncertainties in modeling and in system parameters. Furthermore, the controllers have operational limits. The first concern is taken care of through an “approximated system” that contains an estimate of the uncertainties from an online neural network andhelps calculate the optimal control for the changed plant. Regarding the second concern, a nonquadratic term that incorporates the control constraints is used in the performance index. Necessary conditions for optimal control are derived and an algorithm to solve the constrained-control problem with cost function-based single network adaptive critic is developed. Two aerospace systems are used to illustrate the working of the proposed technique.
12 citations
Additional excerpts
...In [43], an HJB equation-based 62 J....
[...]
TL;DR: A solution for a new class of optimization problem, which is defined as inhomogeneous discrete-time bilinear plant with control trajectory constraints and a biquadratic performance index, is presented.
Abstract: Bilinear state-space models, a type of simple nonlinear model, have different applications. Optimal control of such models is often used for performance improvement. A solution for a new class of optimization problem, which is defined as inhomogeneous discrete-time bilinear plant with control trajectory constraints and a biquadratic performance index, is presented. Krotov's method is used for computing a candidate solution for this problem. A suitable sequence of improving functions, which are essential for the utilization of this method is formulated. The obtained algorithm is convergent and monotonic by means of the defined performance index. The method's effectiveness is demonstrated by a numerical example.
3 citations
01 Dec 2010
TL;DR: A novel architecture, called the Cost Function Based Single Network Adaptive Critic (J-SNAC) is used to solve control-constrained optimal control problems and an algorithm to solve the constrained-control problem with J- SNAC is developed.
Abstract: Approximate dynamic programming formulation implemented with an Adaptive Critic (AC) based neural network (NN) structure has evolved as a powerful alternative technique that eliminates the need for excessive computations and storage requirements needed for solving the Hamilton-Jacobi-Bellman (HJB) equations. A typical AC structure consists of two interacting NNs. In this paper, a novel architecture, called the Cost Function Based Single Network Adaptive Critic (J-SNAC) is used to solve control-constrained optimal control problems. Only one network is used that captures the mapping between states and the cost function. This approach is applicable to a wide class of nonlinear systems where the optimal control (stationary) equation can be explicitly expressed in terms of the state and costate variables. A non-quadratic cost function is used that incorporates the control constraints. Necessary equations for optimal control are derived and an algorithm to solve the constrained-control problem with J-SNAC is developed. Benchmark nonlinear systems are used to illustrate the working of the proposed technique. Extensions to optimal control-constrained problems in the presence of uncertainties are also considered.
1 citations
04 Feb 2009
TL;DR: A Hamilton-Jacobi-Bellman (HJB) equation based optimal control algorithm for robust controller design, is proposed for a nonlinear system and is shown to be optimal with respect to a cost functional that includes maximum bound on system uncertainty.
Abstract: In this paper, a Hamilton-Jacobi-Bellman (HJB) equation based optimal control algorithm for robust controller design, is proposed for a nonlinear system. Utilizing the Lyapunov direct method, controller is shown to be optimal with respect to a cost functional that includes maximum bound on system uncertainty. Controller is continuous and requires the knowledge of the upper bound of system uncertainty. In the proposed algorithm, Neural Network (NN) is used to find approximate solution of HJB equation. Proposed algorithm has been applied on a nonlinear uncertain system.
1 citations
References
More filters
TL;DR: It is shown that the constrained optimal control law has the largest region of asymptotic stability (RAS) and the result is a nearly optimal constrained state feedback controller that has been tuned a priori off-line.
1,045 citations
TL;DR: This paper states sufficient conditions that guarantee that the Galerkin approximation converges to the solution of the GHJB equation and that the resulting approximate control is stabilizing on the same region as the initial control.
580 citations
TL;DR: This work is intended to motivate the interest of bilinear systems and to present the current state of research in its various aspects.
Abstract: Recently, attention has been focused on the class of bilinear systems, both for its applicative interest and intrinsic simplicity. In fact, it appears that many important processes, not only in engineering, but also in biology, socio-economics, and ecology, may be modeled by bilinear systems. Moreover, since their nonlinearity is due to products between input and state variables, this class frequently may be studied by techniques similar to those employed for linear systems. This work is intended to motivate the interest of bilinear systems and to present the current state of research in its various aspects. After an introductory section, in which theoretical and applicative aspects of bilinear systems are enlightened, four other sections follow, respectively, devoted to structural properties, mathematical models, identification and optimization. In a final section, some concluding remarks are made on still open problems and possible trends for future research.
495 citations
TL;DR: It is shown that these so-called bilinear systems have a variable dynamical structure that makes them quite controllable and may utilize appropriately controlled unstable modes of response to enhance controllability.
Abstract: A nonlinear class of models for biological and physical processes is surveyed. It is shown that these so-called bilinear systems have a variable dynamical structure that makes them quite controllable. While control systems are classically designed so there are no unstable modes, bilinear systems may utilize appropriately controlled unstable modes of response to enhance controllability.
447 citations