The Linear Programming Approach to Approximate Dynamic Programming
TLDR
In this article, an efficient method based on linear programming for approximating solutions to large-scale stochastic control problems is proposed. But the approach is not suitable for large scale queueing networks.Abstract:
The curse of dimensionality gives rise to prohibitive computational requirements that render infeasible the exact solution of large-scale stochastic control problems. We study an efficient method based on linear programming for approximating solutions to such problems. The approach "fits" a linear combination of pre-selected basis functions to the dynamic programming cost-to-go function. We develop error bounds that offer performance guarantees and also guide the selection of both basis functions and "state-relevance weights" that influence quality of the approximation. Experimental results in the domain of queueing network control provide empirical support for the methodology.read more
Citations
More filters
Proceedings ArticleDOI
Deep Reinforcement Learning in Linear Discrete Action Spaces
TL;DR: In this article, a deep reinforcement learning algorithm that combines linear programming and neural network VFAs was proposed to solve the problem of solving stochastic multi-period optimization problems.
Proceedings ArticleDOI
Parallel Least-Squares Policy Iteration
Jun-Kun Wang,Shou-De Lin +1 more
TL;DR: Preliminary analysis of the proposed parallel least-squares policy iteration (parallel LSPI) shows that the sample complexity improved from O(1/√n) towards O( 1/ ∼Mn) for each worker, where n isThe number of samples and M is the number of workers.
Journal ArticleDOI
Computing controlled invariant sets from data using convex optimization
TL;DR: In this paper, the authors present a data-driven method for approximation of the maximum positively invariant (MPI) set and the maximum controlled invariant set for nonlinear dynamical systems.
Journal ArticleDOI
Fully polynomial time $$(\Sigma ,\Pi )$$ ( Σ , Π ) -approximation schemes for continuous nonlinear newsvendor and continuous stochastic dynamic programs
Nir Halman,Giacomo Nannicini +1 more
TL;DR: In this article, the authors study the nonlinear newsvendor problem concerning goods of a non-discrete nature, and a class of stochastic dynamic programs with several application areas such as supply chain management and economics.
Reference EntryDOI
Performance Bounds in Queueing Networks
TL;DR: This paper surveys computable performance bounds for sequencing and scheduling control in open networks to minimize the long run average number of customers, or a weighted average over different customer classes.
References
More filters
Book
Reinforcement Learning: An Introduction
TL;DR: This book provides a clear and simple account of the key ideas and algorithms of reinforcement learning, which ranges from the history of the field's intellectual foundations to the most recent developments and applications.
Book
Neural networks for pattern recognition
TL;DR: This is the first comprehensive treatment of feed-forward neural networks from the perspective of statistical pattern recognition, and is designed as a text, with over 100 exercises, to benefit anyone involved in the fields of neural computation and pattern recognition.
Book
Dynamic Programming and Optimal Control
TL;DR: The leading and most up-to-date textbook on the far-ranging algorithmic methododogy of Dynamic Programming, which can be used for optimal control, Markovian decision problems, planning and sequential decision making under uncertainty, and discrete/combinatorial optimization.
Journal ArticleDOI
Learning to Predict by the Methods of Temporal Differences
TL;DR: This article introduces a class of incremental learning procedures specialized for prediction – that is, for using past experience with an incompletely known system to predict its future behavior – and proves their convergence and optimality for special cases and relation to supervised-learning methods.
Book
Neuro-dynamic programming
TL;DR: This is the first textbook that fully explains the neuro-dynamic programming/reinforcement learning methodology, which is a recent breakthrough in the practical application of neural networks and dynamic programming to complex problems of planning, optimal decision making, and intelligent control.