A model approximation scheme for planning in partially observable stochastic domains
Nevin L. Zhang,Wenju Liu +1 more
Reads0
Chats0
TLDR
This paper proposes a new approximation scheme to transform a POMDP into another one where additional information is provided by an oracle and uses its optimal policy to construct an approximate policy for the original PomDP.Abstract:
Partially observable Markov decision processes (POMDPs) are a natural model for planning problems where effects of actions are nondeterministic and the state of the world is not completely observable. It is difficult to solve POMDPs exactly. This paper proposes a new approximation scheme. The basic idea is to transform a POMDP into another one where additional information is provided by an oracle. The oracle informs the planning agent that the current state of the world is in a certain region. The transformed POMDP is consequently said to be region observable. It is easier to solve than the original POMDP. We propose to solve the transformed POMDP and use its optimal policy to construct an approximate policy for the original POMDP. By controlling the amount of additional information that the oracle provides, it is possible to find a proper tradeoff between computational time and approximation quality. In terms of algorithmic contributions, we study in details how to exploit region observability in solving the transformed POMDP. To facilitate the study, we also propose a new exact algorithm for general POMDPs. The algorithm is conceptually simple and yet is significantly more efficient than all previous exact algorithms.read more
Citations
More filters
Book
Dynamic Programming and Optimal Control
TL;DR: The leading and most up-to-date textbook on the far-ranging algorithmic methododogy of Dynamic Programming, which can be used for optimal control, Markovian decision problems, planning and sequential decision making under uncertainty, and discrete/combinatorial optimization.
Journal ArticleDOI
Decision-theoretic planning: structural assumptions and computational leverage
TL;DR: In this article, the authors present an overview and synthesis of MDP-related methods, showing how they provide a unifying framework for modeling many classes of planning problems studied in AI.
Journal ArticleDOI
Value-function approximations for partially observable Markov decision processes
TL;DR: This work surveys various approximation methods, analyzes their properties and relations and provides some new insights into their differences, and presents a number of new approximation methods and novel refinements of existing techniques.
Dynamic Programming and Optimal Control 3rd Edition, Volume II
TL;DR: This is an updated version of the research-oriented Chapter 6 on Approximate Dynamic Programming, which includes an account of new research, which is collected mostly in Sections 6.3 and 6.8.
Journal ArticleDOI
Complexity of finite-horizon Markov decision process problems
TL;DR: This work analyzes the computational complexity of evaluating policies and of determining whether a sufficiently good policy exists for a Markov decision process, based on a number of confounding factors, including the observability of the system state; the succinctness of the representation; the type of policy; even the number of actions relative to theNumber of states.
References
More filters
Book
Dynamic Programming
TL;DR: The more the authors study the information processing aspects of the mind, the more perplexed and impressed they become, and it will be a very long time before they understand these processes sufficiently to reproduce them.
Book
Markov Decision Processes: Discrete Stochastic Dynamic Programming
TL;DR: Puterman as discussed by the authors provides a uniquely up-to-date, unified, and rigorous treatment of the theoretical, computational, and applied research on Markov decision process models, focusing primarily on infinite horizon discrete time models and models with discrete time spaces while also examining models with arbitrary state spaces, finite horizon models, and continuous time discrete state models.
MonographDOI
Markov Decision Processes
P. Whittle,M. L. Puterman +1 more
TL;DR: Markov Decision Processes covers recent research advances in such areas as countable state space models with average reward criterion, constrained models, and models with risk sensitive optimality criteria, and explores several topics that have received little or no attention in other books.
Book
Dynamic Programming: Deterministic and Stochastic Models
TL;DR: As one of the part of book categories, dynamic programming deterministic and stochastic models always becomes the most wanted book.
Proceedings Article
Acting Optimally in Partially Observable Stochastic Domains
TL;DR: The existing algorithms for computing optimal control strategies for partially observable stochastic environments are found to be highly computationally inefficient and a new algorithm is developed that is empirically more efficient.