The Linear Programming Approach to Approximate Dynamic Programming
TLDR
In this article, an efficient method based on linear programming for approximating solutions to large-scale stochastic control problems is proposed. But the approach is not suitable for large scale queueing networks.Abstract:
The curse of dimensionality gives rise to prohibitive computational requirements that render infeasible the exact solution of large-scale stochastic control problems. We study an efficient method based on linear programming for approximating solutions to such problems. The approach "fits" a linear combination of pre-selected basis functions to the dynamic programming cost-to-go function. We develop error bounds that offer performance guarantees and also guide the selection of both basis functions and "state-relevance weights" that influence quality of the approximation. Experimental results in the domain of queueing network control provide empirical support for the methodology.read more
Citations
More filters
Journal ArticleDOI
Learning Representation and Control in Markov Decision Processes: New Frontiers
TL;DR: Methods for automatically compressing Markov decision processes by learning a low-dimensional linear approximation defined by an orthogonal set of basis functions are described, whose matrix representations have non-positive off-diagonal elements and zero row sums.
Book ChapterDOI
Tetris: A Study of Randomized Constraint Sampling
Vivek F. Farias,Benjamin Van Roy +1 more
TL;DR: Results from Randomized constraint sampling suggest that in fact, such a scheme is capable of producing good solutions to the linear program that arises in the context of approximate Dynamic Programming.
Journal ArticleDOI
Elective Patient Admission and Scheduling under Multiple Resource Constraints
Christiane Barz,Kumar Rajaram +1 more
TL;DR: This work forms the control process as a Markov decision process to maximize expected contribution net of overbooking costs, develops bounds using approximate dynamic programming, and uses them to construct heuristics and finds that the intuitive newsvendor-based heuristic performs well across all scenarios.
Journal ArticleDOI
Optimally solving Dec-POMDPs as continuous-state MDPs
TL;DR: The idea of transforming a Dec-POMDP into a continuous-state deterministic MDP with a piecewise-linear and convex value function is introduced, and a feature-based heuristic search that relies on feature- based compact representations, point-based updates and efficient action selection is introduced.
Posted Content
Stochastic Primal-Dual Methods and Sample Complexity of Reinforcement Learning
Yichen Chen,Mengdi Wang +1 more
TL;DR: A class of Stochastic Primal-Dual methods which exploit the inherent minimax duality of Bellman equations are proposed which use small storage and has low computational complexity per iteration.
References
More filters
Book
Reinforcement Learning: An Introduction
TL;DR: This book provides a clear and simple account of the key ideas and algorithms of reinforcement learning, which ranges from the history of the field's intellectual foundations to the most recent developments and applications.
Book
Neural networks for pattern recognition
TL;DR: This is the first comprehensive treatment of feed-forward neural networks from the perspective of statistical pattern recognition, and is designed as a text, with over 100 exercises, to benefit anyone involved in the fields of neural computation and pattern recognition.
Book
Dynamic Programming and Optimal Control
TL;DR: The leading and most up-to-date textbook on the far-ranging algorithmic methododogy of Dynamic Programming, which can be used for optimal control, Markovian decision problems, planning and sequential decision making under uncertainty, and discrete/combinatorial optimization.
Journal ArticleDOI
Learning to Predict by the Methods of Temporal Differences
TL;DR: This article introduces a class of incremental learning procedures specialized for prediction – that is, for using past experience with an incompletely known system to predict its future behavior – and proves their convergence and optimality for special cases and relation to supervised-learning methods.
Book
Neuro-dynamic programming
TL;DR: This is the first textbook that fully explains the neuro-dynamic programming/reinforcement learning methodology, which is a recent breakthrough in the practical application of neural networks and dynamic programming to complex problems of planning, optimal decision making, and intelligent control.