Journal ArticleDOI
Model-free Q-learning designs for discrete-time zero-sum games with application to H-infinity control
A. Al-Tamimi,Frank L. Lewis,Murad Abu-Khalaf +2 more
- Vol. 43, Iss: 3, pp 473-481
TLDR
It is proven that the algorithm ends up to be a model-free iterative algorithm to solve the (GARE) of the linear quadratic discrete-time zero-sum game.Abstract:
In this paper, the optimal strategies for discrete-time linear system quadratic zero-sum games related to the H-infinity optimal control problem are solved in forward time without knowing the system dynamical matrices. The idea is to solve for an action dependent value function Q(x,u,w) of the zero-sum game instead of solving for the state dependent value function V(x) which satisfies a corresponding game algebraic Riccati equation (GARE). Since the state and actions spaces are continuous, two action networks and one critic network are used that are adaptively tuned in forward time using adaptive critic methods. The result is a Q-learning approximate dynamic programming model-free approach that solves the zero-sum game forward in time. It is shown that the critic converges to the game value function and the action networks converge to the Nash equilibrium of the game. Proofs of convergence of the algorithm are shown. It is proven that the algorithm ends up to be a model-free iterative algorithm to solve the (GARE) of the linear quadratic discrete-time zero-sum game. The effectiveness of this method is shown by performing an H-infinity control autopilot design for an F-16 aircraft.read more
Citations
More filters
Journal ArticleDOI
Reinforcement learning and adaptive dynamic programming for feedback control
Frank L. Lewis,Draguna Vrabie +1 more
TL;DR: This work describes mathematical formulations for reinforcement learning and a practical implementation method known as adaptive dynamic programming that give insight into the design of controllers for man-made engineered systems that both learn and exhibit optimal behavior.
Journal ArticleDOI
Discrete-Time Nonlinear HJB Solution Using Approximate Dynamic Programming: Convergence Proof
TL;DR: It is shown that HDP converges to the optimal control and the optimal value function that solves the Hamilton-Jacobi-Bellman equation appearing in infinite-horizon discrete-time (DT) nonlinear optimal control.
Journal ArticleDOI
From model-based control to data-driven control: Survey, classification and perspective
Zhongsheng Hou,Zhuo Wang +1 more
TL;DR: This paper is a brief survey on the existing problems and challenges inherent in model-based control (MBC) theory, and some important issues in the analysis and design of data-driven control (DDC) methods are here reviewed and addressed.
Journal ArticleDOI
Adaptive Dynamic Programming: An Introduction
TL;DR: Some recent research trends within the field of adaptive/approximate dynamic programming (ADP), including the variations on the structure of ADP schemes, the development of ADPs algorithms and applications, and many recent papers have provided convergence analysis associated with the algorithms developed.
Journal ArticleDOI
Computational adaptive optimal control for continuous-time linear systems with completely unknown dynamics
Yu Jiang,Zhong-Ping Jiang +1 more
TL;DR: This paper presents a novel policy iteration approach for finding online adaptive optimal controllers for continuous-time linear systems with completely unknown system dynamics, using the approximate/adaptive dynamic programming technique to iteratively solve the algebraic Riccati equation using the online information of state and input.
References
More filters
Book
Dynamic Noncooperative Game Theory
Tamer Basar,Geert Jan Olsder +1 more
TL;DR: In this paper, the authors present a general formulation of non-cooperative finite games: N-Person nonzero-sum games, Pursuit-Evasion games, and Stackelberg Equilibria of infinite dynamic games.
Neuro-Dynamic Programming.
TL;DR: In this article, the authors present the first textbook that fully explains the neuro-dynamic programming/reinforcement learning methodology, which is a recent breakthrough in the practical application of neural networks and dynamic programming to complex problems of planning, optimal decision making, and intelligent control.
Book
Neuro-dynamic programming
TL;DR: This is the first textbook that fully explains the neuro-dynamic programming/reinforcement learning methodology, which is a recent breakthrough in the practical application of neural networks and dynamic programming to complex problems of planning, optimal decision making, and intelligent control.
Journal ArticleDOI
Neuronlike adaptive elements that can solve difficult learning control problems
TL;DR: In this article, a system consisting of two neuron-like adaptive elements can solve a difficult learning control problem, where the task is to balance a pole that is hinged to a movable cart by applying forces to the cart base.
Related Papers (5)
Reinforcement learning and adaptive dynamic programming for feedback control
Frank L. Lewis,Draguna Vrabie +1 more