The Linear Programming Approach to Approximate Dynamic Programming
TLDR
In this article, an efficient method based on linear programming for approximating solutions to large-scale stochastic control problems is proposed. But the approach is not suitable for large scale queueing networks.Abstract:
The curse of dimensionality gives rise to prohibitive computational requirements that render infeasible the exact solution of large-scale stochastic control problems. We study an efficient method based on linear programming for approximating solutions to such problems. The approach "fits" a linear combination of pre-selected basis functions to the dynamic programming cost-to-go function. We develop error bounds that offer performance guarantees and also guide the selection of both basis functions and "state-relevance weights" that influence quality of the approximation. Experimental results in the domain of queueing network control provide empirical support for the methodology.read more
Citations
More filters
Posted Content
The Value Function Polytope in Reinforcement Learning
TL;DR: In this paper, Aigner et al. established geometric and topological properties of the space of value functions in finite state-action Markov decision processes and showed that the value functions of policies constrained on all but one state describe a line segment.
Journal ArticleDOI
Approximate linear programming for networks
TL;DR: This paper uses approximate linear programming (ALP) to compute average cost bounds for queueing network control problems and finds that the ALPs offer more accurate bounds than other methods and the simplicity of just solving an LP.
Journal ArticleDOI
Information Relaxation and Dual Formulation of Controlled Markov Diffusions
TL;DR: This paper extends the dual formulation of Markov decision processes to controlled Markov diffusions and demonstrates the use of this dual representation in a classic dynamic portfolio choice problem through a new class of penalties, which require little extra computation and produce small duality gap on the optimal value.
Approximate Linear Programming for Solving Hybrid Factored MDPs.
TL;DR: This work presents the HALP framework and discusses several representational and computational issues that make the approach appropriate for large MDPs, and demonstrates the feasibility of the approach on high-dimensional distributed control problems.
The Linear Programming Approach to Solving Large Scale Dynamic Stochastic Games
TL;DR: This paper introduces a new method to approximate Markov perfect equilibrium in large scale dynamic stochastic games that are not amenable to exact solution due to the curse of dimensionality, and suggests that the approach proposed significantly expands the set of models that can be analyzed computationally.
References
More filters
Book
Reinforcement Learning: An Introduction
TL;DR: This book provides a clear and simple account of the key ideas and algorithms of reinforcement learning, which ranges from the history of the field's intellectual foundations to the most recent developments and applications.
Book
Neural networks for pattern recognition
TL;DR: This is the first comprehensive treatment of feed-forward neural networks from the perspective of statistical pattern recognition, and is designed as a text, with over 100 exercises, to benefit anyone involved in the fields of neural computation and pattern recognition.
Book
Dynamic Programming and Optimal Control
TL;DR: The leading and most up-to-date textbook on the far-ranging algorithmic methododogy of Dynamic Programming, which can be used for optimal control, Markovian decision problems, planning and sequential decision making under uncertainty, and discrete/combinatorial optimization.
Journal ArticleDOI
Learning to Predict by the Methods of Temporal Differences
TL;DR: This article introduces a class of incremental learning procedures specialized for prediction – that is, for using past experience with an incompletely known system to predict its future behavior – and proves their convergence and optimality for special cases and relation to supervised-learning methods.
Book
Neuro-dynamic programming
TL;DR: This is the first textbook that fully explains the neuro-dynamic programming/reinforcement learning methodology, which is a recent breakthrough in the practical application of neural networks and dynamic programming to complex problems of planning, optimal decision making, and intelligent control.