Open Access
The Witness Algorithm: Solving Partially Observable Markov Decision Processes
Reads0
Chats0
TLDR
It is argued that the witness algorithm is superior to existing algorithms for solving POMDP problems in an important complexity-theoretic sense.Abstract:
Markov decision processes (MDP''s) are a mathematical formalization of problems in which a decision-maker must choose how to act to maximize its reward over a series of interactions with its environment Partially observable Markov decision processes (POMDP''s) generalize the MDP framework to the case where the agent must make its decisions in partial ignorance of its current situation This paper describes the POMDP framework and presents some well-known results from the field It then presents a novel method called the witness algorithm for solving POMDP problems and analyzes its computational complexity The paper argues that the witness algorithm is superior to existing algorithms for solving POMDP''s in an important complexity-theoretic senseread more
Citations
More filters
MonographDOI
Planning Algorithms: Introductory Material
TL;DR: This coherent and comprehensive book unifies material from several sources, including robotics, control theory, artificial intelligence, and algorithms, into planning under differential constraints that arise when automating the motions of virtually any mechanical system.
Journal ArticleDOI
Planning and Acting in Partially Observable Stochastic Domains
TL;DR: A novel algorithm for solving pomdps off line and how, in some cases, a finite-memory controller can be extracted from the solution to a POMDP is outlined.
Book ChapterDOI
Learning policies for partially observable environments: scaling up
TL;DR: This paper discusses several simple solution methods and shows that all are capable of finding near- optimal policies for a selection of extremely small POMDP'S taken from the learning literature, but shows that none are able to solve a slightly larger and noisier problem based on robot navigation.
Journal ArticleDOI
The Hidden Information State model: A practical framework for POMDP-based spoken dialogue management
Steve Young,Milica Gasic,Simon Keizer,François Mairesse,Jost Schatzmann,Blaise Thomson,Kai Yu +6 more
TL;DR: This paper explains how Partially Observable Markov Decision Processes (POMDPs) can provide a principled mathematical framework for modelling the inherent uncertainty in spoken dialogue systems and describes a form of approximation called the Hidden Information State model which does scale and which can be used to build practical systems.
Algorithms for Sequential Decision Making
TL;DR: This thesis shows how to answer the question ``What should I do now?
References
More filters
Journal ArticleDOI
Learning from delayed rewards
TL;DR: The invention relates to a circuit for use in a receiver which can receive two-tone/stereo signals which is intended to make a choice between mono or stereo reproduction of signal A or of signal B and vice versa.
Book
Dynamic Programming: Deterministic and Stochastic Models
TL;DR: As one of the part of book categories, dynamic programming deterministic and stochastic models always becomes the most wanted book.
Proceedings Article
Acting Optimally in Partially Observable Stochastic Domains
TL;DR: The existing algorithms for computing optimal control strategies for partially observable stochastic environments are found to be highly computationally inefficient and a new algorithm is developed that is empirically more efficient.
Journal ArticleDOI
The Optimal Search for a Moving Target When the Search Path Is Constrained
TL;DR: A search is conducted for a target moving in discrete time among a finite number of cells according to a known Markov process and a finite time horizon POMDP solution technique is presented which is simpler than the standard linear programming methods.