Q1. What are the contributions in this paper?
In this paper, a policy search algorithm is proposed to find a policy that performs well over the entire state space, which is not inherently related to the number of state variables.
Reinforcement Learning and Dynamic Programming Using Function Approximators
TLDR
Reinforcement Learning and Dynamic Programming Using Function Approximators provides a comprehensive and unparalleled exploration of the field of RL and DP, with a focus on continuous-variable problems.Abstract:
From household appliances to applications in robotics, engineered systems involving complex dynamics can only be as effective as the algorithms that control them. While Dynamic Programming (DP) has provided researchers with a way to optimally solve decision and control problems involving complex dynamic systems, its practical value was limited by algorithms that lacked the capacity to scale up to realistic problems. However, in recent years, dramatic developments in Reinforcement Learning (RL), the model-free counterpart of DP, changed our understanding of what is possible. Those developments led to the creation of reliable methods that can be applied even when a mathematical model of the system is unavailable, allowing researchers to solve challenging control problems in engineering, as well as in a variety of other disciplines, including economics, medicine, and artificial intelligence. Reinforcement Learning and Dynamic Programming Using Function Approximators provides a comprehensive and unparalleled exploration of the field of RL and DP. With a focus on continuous-variable problems, this seminal text details essential developments that have substantially altered the field over the past decade. In its pages, pioneering experts provide a concise introduction to classical RL and DP, followed by an extensive presentation of the state-of-the-art and novel methods in RL and DP with approximation. Combining algorithm development with theoretical guarantees, they elaborate on their work with illustrative examples and insightful comparisons. Three individual chapters are dedicated to representative algorithms from each of the major classes of techniques: value iteration, policy iteration, and policy search. The features and performance of these algorithms are highlighted in extensive experimental studies on a range of control applications. The recent development of applications involving complex systems has led to a surge of interest in RL and DP methods and the subsequent need for a quality resource on the subject. For graduate students and others new to the field, this book offers a thorough introduction to both the basics and emerging methods. And for those researchers and practitioners working in the fields of optimal and adaptive control, machine learning, artificial intelligence, and operations research, this resource offers a combination of practical algorithms, theoretical analysis, and comprehensive examples that they will be able to adapt and apply to their own work. Access the authors' website at www.dcsc.tudelft.nl/rlbook/ for additional material, including computer code used in the studies and information concerning new developments.read more
Figures
TABLE 6.2 Parameters of the bicycle. 
FIGURE 6.7 Results of DIRECT for the double integrator, compared with the results of CE policy search. 
FIGURE 3.9 A conceptual illustration of the projected Bellman equation. Applying Th and thenPw to an ordinary approximate Q-function̂Q leads to a different point in the space of approximate Qfunctions (left). In contrast, applyingTh and thenPw to the fixed pointQ̂h of the projected Bellman equation leads back to the same point (right). 
FIGURE 4.12 Results of fuzzy Q-iteration for the inverted pendulum. 
FIGURE 5.9 Execution time of online PI with LSPE-Q for varyingKθ , compared with the execution time of online LSPI, in the inverted pendulum problem. 
FIGURE 4.15 Comparison between fuzzy Q-iteration with optimized and equidistant MFs for the car on the hill.
Citations
More filters
Proceedings ArticleDOI
Convolutional fitted Q iteration for vision-based control problems
TL;DR: A deep reinforcement learning (DRL) method is proposed to solve the control problem which takes raw image pixels as input states and successfully learns a satisfied policy which has better performance than the results of previous researches.
Proceedings Article
A Hybrid Lyapunov Fuzzy Reinforcement Learning Controller
Rakhi Saxena,Rajneesh Sharma +1 more
TL;DR: A Hybrid Lyapunov theory based Fuzzy Reinforcement Learning Controller with guaranteed stability for non linear systems is proposed and simulated on the benchmark problem of Inverted Pendulum Control to showcase effectiveness and efficacy.
Journal ArticleDOI
Symbolic Regression Methods for Reinforcement Learning
TL;DR: In this paper, the authors propose a new approach to construct smooth value functions in the form of analytic expressions by using symbolic regression, which is shown to yield well-performing policies and is easy to plug into other algorithms.
Proceedings ArticleDOI
Coverage path planning optimization based on Q-learning algorithm
TL;DR: An optimization approach of the coverage path planning using Q-Learning algorithm is addressed and comparison with other methods allows to validate the methodology.
Posted Content
Explainable Reinforcement Learning for Broad-XAI: A Conceptual Framework and Survey.
TL;DR: The Causal XRL Framework (CXF) as mentioned in this paper is a conceptual framework that unifies the current XRL research and uses RL as a backbone to the development of Broad-XAI.
References
More filters
Journal ArticleDOI
A tutorial on support vector regression
TL;DR: This tutorial gives an overview of the basic ideas underlying Support Vector (SV) machines for function estimation, and includes a summary of currently used algorithms for training SV machines, covering both the quadratic programming part and advanced methods for dealing with large datasets.
Journal ArticleDOI
Application of Fuzzy Logic to Approximate Reasoning Using Linguistic Synthesis
TL;DR: In this article, a fuzzy logic is used to synthesize linguistic control protocol of a skilled operator for industrial plants, which has been applied to pilot scale plants as well as in practical situations.
Book ChapterDOI
Integrated architecture for learning, planning, and reacting based on approximating dynamic programming
TL;DR: This paper extends previous work with Dyna, a class of architectures for intelligent systems based on approximating dynamic programming methods, and presents and shows results for two Dyna architectures, based on Watkins's Q-learning, a new kind of reinforcement learning.
Journal ArticleDOI
Least-squares policy iteration
TL;DR: The new algorithm, least-squares policy iteration (LSPI), learns the state-action value function which allows for action selection without a model and for incremental policy improvement within a policy-iteration framework.
Journal ArticleDOI
Cooperative Multi-Agent Learning: The State of the Art
Liviu Panait,Sean Luke +1 more
TL;DR: This survey attempts to draw from multi-agent learning work in a spectrum of areas, including RL, evolutionary computation, game theory, complex systems, agent modeling, and robotics, and finds that this broad view leads to a division of the work into two categories.