scispace - formally typeset
Topic

Markov decision process

About: Markov decision process is a(n) research topic. Over the lifetime, 14258 publication(s) have been published within this topic receiving 351684 citation(s). The topic is also known as: MDP & MDPs.

...read more

Papers
  More

Open accessBook
15 Apr 1994-
Abstract: From the Publisher: The past decade has seen considerable theoretical and applied research on Markov decision processes, as well as the growing use of these models in ecology, economics, communications engineering, and other fields where outcomes are uncertain and sequential decision-making processes are needed. A timely response to this increased activity, Martin L. Puterman's new work provides a uniquely up-to-date, unified, and rigorous treatment of the theoretical, computational, and applied research on Markov decision process models. It discusses all major research directions in the field, highlights many significant applications of Markov decision processes models, and explores numerous important topics that have previously been neglected or given cursory coverage in the literature. Markov Decision Processes focuses primarily on infinite horizon discrete time models and models with discrete time spaces while also examining models with arbitrary state spaces, finite horizon models, and continuous-time discrete state models. The book is organized around optimality criteria, using a common framework centered on the optimality (Bellman) equation for presenting results. The results are presented in a "theorem-proof" format and elaborated on through both discussion and examples, including results that are not available in any other book. A two-state Markov decision process model, presented in Chapter 3, is analyzed repeatedly throughout the book and demonstrates many results and algorithms. Markov Decision Processes covers recent research advances in such areas as countable state space models with average reward criterion, constrained models, and models with risk sensitive optimality criteria. It also explores several topics that have received little or no attention in other books, including modified policy iteration, multichain models with average reward criterion, and sensitive optimality. In addition, a Bibliographic Remarks section in each chapter comments on relevant historic

...read more

11,593 Citations


Open accessMonographDOI: 10.1002/9780470316887
Abstract: From the Publisher: The past decade has seen considerable theoretical and applied research on Markov decision processes, as well as the growing use of these models in ecology, economics, communications engineering, and other fields where outcomes are uncertain and sequential decision-making processes are needed. A timely response to this increased activity, Martin L. Puterman's new work provides a uniquely up-to-date, unified, and rigorous treatment of the theoretical, computational, and applied research on Markov decision process models. It discusses all major research directions in the field, highlights many significant applications of Markov decision processes models, and explores numerous important topics that have previously been neglected or given cursory coverage in the literature. Markov Decision Processes focuses primarily on infinite horizon discrete time models and models with discrete time spaces while also examining models with arbitrary state spaces, finite horizon models, and continuous-time discrete state models. The book is organized around optimality criteria, using a common framework centered on the optimality (Bellman) equation for presenting results. The results are presented in a \"theorem-proof\" format and elaborated on through both discussion and examples, including results that are not available in any other book. A two-state Markov decision process model, presented in Chapter 3, is analyzed repeatedly throughout the book and demonstrates many results and algorithms. Markov Decision Processes covers recent research advances in such areas as countable state space models with average reward criterion, constrained models, and models with risk sensitive optimality criteria. It also explores several topics that have received little or no attention in other books, including modified policy iteration, multichain models with average reward criterion, and sensitive optimality. In addition, a Bibliographic Remarks section in each chapter comments on relevant historic

...read more

Topics: Markov decision process (71%)

4,062 Citations


Open accessBook
01 Jan 1967-
Abstract: Overview of the operations research modelling approach introduction to linear programming solving linear programming problems - the Simplex Method the theory of the Simplex Method duality theory and sensitivity analysis other algorithms for linear programming the transportation and assignment problems network analysis, including Pert-CPM dynamic programming game theory integer programming non-linear programming Markov chains queueing theory the application of queueing theory inventory theory forecasting Markovian decision processes and applications decision analysis simulation.

...read more

Topics: Inductive programming (67%), Constraint programming (65%), Linear programming (64%) ...read more

3,828 Citations


Open accessBook
Wendell H. Fleming1, H. Mete SonerInstitutions (1)
18 Dec 1992-
Abstract: This book is intended as an introduction to optimal stochastic control for continuous time Markov processes and to the theory of viscosity solutions. The authors approach stochastic control problems by the method of dynamic programming. The text provides an introduction to dynamic programming for deterministic optimal control problems, as well as to the corresponding theory of viscosity solutions. A new Chapter X gives an introduction to the role of stochastic optimal control in portfolio optimization and in pricing derivatives in incomplete markets. Chapter VI of the First Edition has been completely rewritten, to emphasize the relationships between logarithmic transformations and risk sensitivity. A new Chapter XI gives a concise introduction to two-controller, zero-sum differential games. Also covered are controlled Markov diffusions and viscosity solutions of Hamilton-Jacobi-Bellman equations. The authors have tried, through illustrative examples and selective material, to connect stochastic control theory with other mathematical areas (e.g. large deviations theory) and with applications to engineering, physics, management, and finance. In this Second Edition, new material on applications to mathematical finance has been added. Concise introductions to risk-sensitive control theory, nonlinear H-infinity control and differential games are also included.

...read more

Topics: Markov decision process (60%), Stochastic control (60%), Viscosity solution (59%) ...read more

3,765 Citations


Open accessJournal ArticleDOI: 10.1016/S0004-3702(98)00023-X
Abstract: In this paper, we bring techniques from operations research to bear on the problem of choosing optimal actions in partially observable stochastic domains. We begin by introducing the theory of Markov decision processes (mdps) and partially observable MDPs (pomdps). We then outline a novel algorithm for solving pomdps off line and show how, in some cases, a finite-memory controller can be extracted from the solution to a POMDP. We conclude with a discussion of how our approach relates to previous work, the complexity of finding exact solutions to pomdps, and of some possibilities for finding approximate solutions.

...read more

3,746 Citations


Performance
Metrics
No. of papers in the topic in previous years
YearPapers
202231
20211,243
20201,341
20191,229
2018864
2017745

Top Attributes

Show by:

Topic's top 5 most impactful authors

Krishnendu Chatterjee

95 papers, 2K citations

Ufuk Topcu

83 papers, 733 citations

Eugene A. Feinberg

72 papers, 1.4K citations

Shie Mannor

60 papers, 2.7K citations

Michael L. Littman

51 papers, 6.4K citations

Network Information
Related Topics (5)
Convex optimization

24.9K papers, 908.7K citations

90% related
Optimization problem

96.4K papers, 2.1M citations

89% related
Reinforcement learning

46K papers, 1M citations

88% related
Linear programming

32.1K papers, 920.3K citations

87% related
Computational complexity theory

30.8K papers, 711.2K citations

87% related