This book, written by two major figures in adaptive control, provides a wealth of material for researchers, practitioners, and students. While some researchers in adaptive control may note the absence of a particular topic, the book‘s scope represents a high-gain instrument. It can be used by designers of control systems to enhance their work through the information on many new theoretical developments, and can be used by mathematical control theory specialists to adapt their research to practical needs. The book is strongly recommended to anyone interested in adaptive control.

Adaptive Control

Adaptive dynamic programming (ADP) and reinforcement learning are quite relevant to each other when performing intelligent optimization. They are both regarded as promising methods involving important components of evaluation and improvement, at the background of information technology, such as artificial intelligence, big data, and deep learning. Although great progresses have been achieved and surveyed when addressing nonlinear optimal control problems, the research on robustness of ADP-based control strategies under uncertain environment has not been fully summarized. Hence, this survey reviews the recent main results of adaptive-critic-based robust control design of continuous-time nonlinear systems. The ADP-based nonlinear optimal regulation is reviewed, followed by robust stabilization of nonlinear systems with matched uncertainties, guaranteed cost control design of unmatched plants, and decentralized stabilization of interconnected systems. Additionally, further comprehensive discussions are presented, including event-based robust control design, improvement of the critic learning rule, nonlinear $ {H_{\infty }}$ control design, and several notes on future perspectives. By applying the ADP-based optimal and robust control methods to a practical power system and an overhead crane plant, two typical examples are provided to verify the effectiveness of theoretical results. Overall, this survey is beneficial to promote the development of adaptive critic control methods with robustness guarantee and the construction of higher level intelligent systems.

Adaptive Critic Nonlinear Robust Control: A Survey

In this paper, the infinite horizon optimal robust guaranteed cost control of continuous-time uncertain nonlinear systems is investigated using neural-network-based online solution of Hamilton–Jacobi–Bellman (HJB) equation. By establishing an appropriate bounded function and defining a modified cost function, the optimal robust guaranteed cost control problem is transformed into an optimal control problem. It can be observed that the optimal cost function of the nominal system is nothing but the optimal guaranteed cost of the original uncertain system. A critic neural network is constructed to facilitate the solution of the modified HJB equation corresponding to the nominal system. More importantly, an additional stabilizing term is introduced for helping to verify the stability, which reinforces the updating process of the weight vector and reduces the requirement of an initial stabilizing control. The uniform ultimate boundedness of the closed-loop system is analyzed by using the Lyapunov approach as well. Two simulation examples are provided to verify the effectiveness of the present control approach.

Neural-Network-Based Online HJB Solution for Optimal Robust Guaranteed Cost Control of Continuous-Time Uncertain Nonlinear Systems

SUMMARY

In this paper, we introduce an online algorithm that uses integral reinforcement knowledge for learning the continuous-time optimal control solution for nonlinear systems with infinite horizon costs and partial knowledge of the system dynamics. This algorithm is a data-based approach to the solution of the Hamilton–Jacobi–Bellman equation, and it does not require explicit knowledge on the system's drift dynamics. A novel adaptive control algorithm is given that is based on policy iteration and implemented using an actor/critic structure having two adaptive approximator structures. Both actor and critic approximation networks are adapted simultaneously. A persistence of excitation condition is required to guarantee convergence of the critic to the actual optimal value function. Novel adaptive control tuning algorithms are given for both critic and actor networks, with extra terms in the actor tuning law being required to guarantee closed loop dynamical stability. The approximate convergence to the optimal controller is proven, and stability of the system is also guaranteed. Simulation examples support the theoretical result. Copyright © 2013 John Wiley & Sons, Ltd.

Online adaptive algorithm for optimal control with integral reinforcement learning

Guaranteed cost neural tracking control for a class of uncertain nonlinear systems using adaptive dynamic programming

In this paper we develop an optimality-based framework for designing controllers for discrete-time non-linear cascade systems. Specifically, using a non-linear—non-quadratic optimal control framework we develop a family of globally stabilizing backstepping-type controllers parameterized by the cost functional that is minimized. Furthermore, it is shown that the control Lyapunov function guaranteeing closed-loop stability is a solution to the steady-state Bellman equation for the controlled system and thus guarantees both optimality and stability.

/pdf/optimal-discrete-time-control-for-non-linear-cascade-systems-2doczwtlgj.pdf

Optimal discrete-time control for non-linear cascade systems

In this paper we develop an optimality-based robust control framework for non-linear uncertain systems with structured parametric uncertainty. Specifically, using an optimal non-linear robust control framework, we develop a family of globally stabilizing robust backstepping controllers parametrized by the cost functional that is minimized. Furthermore, it is shown that the Lyapunov function for the closed-loop system guaranteeing robust stability over a prescribed range of structured system parametric uncertainty is a solution to the steady-state Hamilton-Jacobi-Bellman equation for the controlled system and thus guarantees robust stability and robust performance. The results are then used to design robust controllers for jet engine compression systems with uncertain system dynamics.

Optimal non-linear robust control for non-linear uncertain systems

Robust nonlinear feedback control for uncertain linear systems with nonquadratic performance criteria

A Lyapunov-based optimal adaptive control-system design problem for nonlinear uncertain systems with exogenous L/sub 2/ disturbances is considered. Specifically, an inverse optimal adaptive nonlinear control framework is developed to explicitly characterize globally stabilizing disturbance rejection adaptive controllers that minimize a nonlinear-nonquadratic performance functional for nonlinear systems with parametric uncertainty. It is shown that the adaptive control Lyapunov function guaranteeing closed-loop stability is a solution to the Hamilton-Jacobi-Isaacs equation for the controlled system and thus guarantees both optimality and robust stability. Additionally, the adaptive control Lyapunov function is dissipative with respect to a weighted input-output energy supply rate guaranteeing closed-loop disturbance rejection.

Inverse optimal adaptive control for nonlinear uncertain systems with exogenous disturbances

We develop an optimality-based robust control framework for uncertain cascade systems with structured parametric uncertainty. Specifically, using an optimal nonlinear robust control framework we develop a family of globally stabilizing robust backstepping controllers parametrized by the cost functional that is minimized. Furthermore, it is shown that the robust control Lyapunov function guaranteeing closed-loop stability over a prescribed range of structured system parametric uncertainty is a solution to the steady-state Hamilton-Jacobi-Bellman equation for the controlled system and thus guarantees robust stability and robust performance. The results are then used to design robust controllers for jet engine compression systems with uncertain compressor characteristic performance maps.

Jerry L. Fausz

Papers

Optimal discrete-time control for non-linear cascade systems

Optimal non-linear robust control for non-linear uncertain systems

Robust nonlinear feedback control for uncertain linear systems with nonquadratic performance criteria

Inverse optimal adaptive control for nonlinear uncertain systems with exogenous disturbances

Optimal nonlinear robust control for nonlinear uncertain cascade systems