The Linear Programming Approach to Approximate Dynamic Programming
TLDR
In this article, an efficient method based on linear programming for approximating solutions to large-scale stochastic control problems is proposed. But the approach is not suitable for large scale queueing networks.Abstract:
The curse of dimensionality gives rise to prohibitive computational requirements that render infeasible the exact solution of large-scale stochastic control problems. We study an efficient method based on linear programming for approximating solutions to such problems. The approach "fits" a linear combination of pre-selected basis functions to the dynamic programming cost-to-go function. We develop error bounds that offer performance guarantees and also guide the selection of both basis functions and "state-relevance weights" that influence quality of the approximation. Experimental results in the domain of queueing network control provide empirical support for the methodology.read more
Citations
More filters
Proceedings Article
Towards exploiting duality in approximate linear programming for MDPs
TL;DR: This paper proposes an LP formulation, which is called a composite ALP, that approximates both the primal and the dual optimization coordinates (the value function and the occupation measure), which is equivalent to approximating both the objective functions and the feasible regions of the LPs.
Journal ArticleDOI
Computing monotone policies for Markov decision processes: a nearly-isotonic penalty approach
TL;DR: A two-stage alternating convex optimization scheme that can accelerate the search for an optimal policy by exploiting the monotone property is proposed and it is shown that the alternating method of multipliers (ADMM) can be significantly accelerated using the regularization step.
Posted Content
On the Synthesis of Bellman Inequalities for Data-Driven Optimal Control.
TL;DR: In this article, a relatively small but sufficiently rich dataset can be exploited to generate new constraints offline and without observing the corresponding transitions, and the authors show how to reconstruct the associated unknown stage-costs.
Journal ArticleDOI
Adaptive polyhedral meshing for approximate dynamic programming in control
TL;DR: In this paper , a new criterion for adaptive meshing in polyhedral partitions which interpolate a value function in approximate dynamic programming (ADP) in optimal control problems is proposed. But this criterion adds new points to a simplicial mesh, based on: a user-defined initial condition probability density function which determines ‘influential’ regions of the state space, uncertainty (variance) propagation, and temporal difference error.
Posted Content
A Markov Decision Process Approach to Active Meta Learning.
TL;DR: This work proposes actively selecting samples on which to train by discerning covariates inside and between meta- training sets, and casts the problem of selecting a sample from a number of meta-training sets as either a multi-armed bandit or a Markov Decision Process (MDP), depending on how one encapsulates correlation across tasks.
References
More filters
Book
Reinforcement Learning: An Introduction
TL;DR: This book provides a clear and simple account of the key ideas and algorithms of reinforcement learning, which ranges from the history of the field's intellectual foundations to the most recent developments and applications.
Book
Neural networks for pattern recognition
TL;DR: This is the first comprehensive treatment of feed-forward neural networks from the perspective of statistical pattern recognition, and is designed as a text, with over 100 exercises, to benefit anyone involved in the fields of neural computation and pattern recognition.
Book
Dynamic Programming and Optimal Control
TL;DR: The leading and most up-to-date textbook on the far-ranging algorithmic methododogy of Dynamic Programming, which can be used for optimal control, Markovian decision problems, planning and sequential decision making under uncertainty, and discrete/combinatorial optimization.
Journal ArticleDOI
Learning to Predict by the Methods of Temporal Differences
TL;DR: This article introduces a class of incremental learning procedures specialized for prediction – that is, for using past experience with an incompletely known system to predict its future behavior – and proves their convergence and optimality for special cases and relation to supervised-learning methods.
Book
Neuro-dynamic programming
TL;DR: This is the first textbook that fully explains the neuro-dynamic programming/reinforcement learning methodology, which is a recent breakthrough in the practical application of neural networks and dynamic programming to complex problems of planning, optimal decision making, and intelligent control.