Journal ArticleDOI
Optimal Tracking Control of Unknown Discrete-Time Linear Systems Using Input-Output Measured Data
TLDR
An output-feedback solution to the infinite-horizon linear quadratic tracking (LQT) problem for unknown discrete-time systems is proposed and a novel Bellman equation is developed that evaluates the value function related to a fixed policy by using only the input, output, and reference trajectory data from the augmented system.Abstract:
In this paper, an output-feedback solution to the infinite-horizon linear quadratic tracking (LQT) problem for unknown discrete-time systems is proposed. An augmented system composed of the system dynamics and the reference trajectory dynamics is constructed. The state of the augmented system is constructed from a limited number of measurements of the past input, output, and reference trajectory in the history of the augmented system. A novel Bellman equation is developed that evaluates the value function related to a fixed policy by using only the input, output, and reference trajectory data from the augmented system. By using approximate dynamic programming, a class of reinforcement learning methods, the LQT problem is solved online without requiring knowledge of the augmented system dynamics only by measuring the input, output, and reference trajectory from the augmented system. We develop both policy iteration (PI) and value iteration (VI) algorithms that converge to an optimal controller that require only measuring the input, output, and reference trajectory data. The convergence of the proposed PI and VI algorithms is shown. A simulation example is used to verify the effectiveness of the proposed control scheme.read more
Citations
More filters
Journal ArticleDOI
Optimal and Autonomous Control Using Reinforcement Learning: A Survey
TL;DR: Q-learning and the integral RL algorithm as core algorithms for discrete time (DT) and continuous-time (CT) systems, respectively are discussed, and a new direction of off-policy RL for both CT and DT systems is discussed.
Reinforcement Learning and Adaptive Dynamic Programming for Feedback Control
TL;DR: A transversal view through microfluidics theory and applications, covering different kinds of phenomena, from continuous to multiphase flow, and a vision of two phasemicrofluidic phenomena is given through nonlinear analyses applied to experimental time series.
Journal ArticleDOI
Model-Free Optimal Tracking Control via Critic-Only Q-Learning
TL;DR: This paper aims to solve the model-free optimal tracking control problem of nonaffine nonlinear discrete-time systems with a critic-only Q-learning (CoQL) method, which avoids solving the tracking Hamilton-Jacobi-Bellman equation.
Journal ArticleDOI
Adaptive Dynamic Programming for Control: A Survey and Recent Advances
TL;DR: In this article, the adaptive dynamic programming (ADP) with applications in control is reviewed, and the use of ADP to solve game problems, mainly nonzero-sum game problems is elaborated.
Journal ArticleDOI
An Event-Triggered ADP Control Approach for Continuous-Time System With Unknown Internal States
Xiangnan Zhong,Haibo He +1 more
TL;DR: A neural-network-based observer is integrated to recover the system internal states from the measurable feedback to reduce the computation cost and transmission load of the event-triggered adaptive dynamic programming control method.
References
More filters
Book
Reinforcement Learning: An Introduction
TL;DR: This book provides a clear and simple account of the key ideas and algorithms of reinforcement learning, which ranges from the history of the field's intellectual foundations to the most recent developments and applications.
Neuro-Dynamic Programming.
TL;DR: In this article, the authors present the first textbook that fully explains the neuro-dynamic programming/reinforcement learning methodology, which is a recent breakthrough in the practical application of neural networks and dynamic programming to complex problems of planning, optimal decision making, and intelligent control.
Book
Neuro-dynamic programming
TL;DR: This is the first textbook that fully explains the neuro-dynamic programming/reinforcement learning methodology, which is a recent breakthrough in the practical application of neural networks and dynamic programming to complex problems of planning, optimal decision making, and intelligent control.
BookDOI
Approximate dynamic programming : solving the curses of dimensionality
TL;DR: This book discusses the challenges of dynamic programming, the three curses of dimensionality, and some experimental comparisons of stepsize formulas that led to the creation of ADP for online applications.
Related Papers (5)
Reinforcement learning and adaptive dynamic programming for feedback control
Frank L. Lewis,Draguna Vrabie +1 more