Open AccessBook
Constrained Markov Decision Processes
TLDR
In this paper, a unified approach for the study of constrained Markov decision processes with a countable state space and unbounded costs is presented, where a single controller has several objectives; it is desirable to design a controller that minimize one of cost objectives, subject to inequality constraints on other cost objectives.Abstract:
This report presents a unified approach for the study of constrained Markov decision processes with a countable state space and unbounded costs. We consider a single controller having several objectives; it is desirable to design a controller that minimize one of cost objective, subject to inequality constraints on other cost objectives. The objectives that we study are both the expected average cost, as well as the expected total cost (of which the discounted cost is a special case). We provide two frameworks: the case were costs are bounded below, as well as the contracting framework. We characterize the set of achievable expected occupation measures as well as performance vectors. This allows us to reduce the original control dynamic problem into an infinite Linear Programming. We present a Lagrangian approach that enables us to obtain sensitivity analysis. In particular, we obtain asymptotical results for the constrained control problem: convergence of both the value and the policies in the time horizon and in the discount factor. Finally, we present and several state truncation algorithms that enable to approximate the solution of the original control problem via finite linear programs.read more
Citations
More filters
Journal ArticleDOI
Markov Decision Processes
Nicole Bäuerle,Ulrich Rieder +1 more
TL;DR: The theory of Markov Decision Processes is the theory of controlled Markov chains as mentioned in this paper, which has found applications in various areas like e.g. computer science, engineering, operations research, biology and economics.
Journal ArticleDOI
Update or Wait: How to Keep Your Data Fresh
TL;DR: In this paper, the authors study how to optimally manage the freshness of information updates sent from a source node to a destination via a channel and develop efficient algorithms to find the optimal update policy among all causal policies and establish sufficient and necessary conditions for the optimality of the zero-wait policy.
Book
A First Course in Stochastic Models
TL;DR: In this article, the authors present an analysis of queuing models useful tools in applied probability useful probability distributions generating functions the discrete fast Fourier transform Laplace transformtheory numerical Laplace inversion the root-finding problem.
Proceedings Article
Constrained policy optimization
TL;DR: Constrained Policy Optimization (CPO) as discussed by the authors is the first general-purpose policy search algorithm for constrained reinforcement learning with guarantees for near-constraint satisfaction at each iteration.
Proceedings ArticleDOI
Delay-optimal computation task scheduling for mobile-edge computing systems
TL;DR: By analyzing the average delay of each task and the average power consumption at the mobile device, a power-constrained delay minimization problem is formulated, and an efficient one-dimensional search algorithm is proposed to find the optimal task scheduling policy.
References
More filters
Book
Matrix Analysis
Roger A. Horn,Charles R. Johnson +1 more
TL;DR: In this article, the authors present results of both classic and recent matrix analyses using canonical forms as a unifying theme, and demonstrate their importance in a variety of applications, such as linear algebra and matrix theory.
Book
Convergence of Probability Measures
TL;DR: Weak Convergence in Metric Spaces as discussed by the authors is one of the most common modes of convergence in metric spaces, and it can be seen as a form of weak convergence in metric space.
Book
Markov Decision Processes: Discrete Stochastic Dynamic Programming
TL;DR: Puterman as discussed by the authors provides a uniquely up-to-date, unified, and rigorous treatment of the theoretical, computational, and applied research on Markov decision process models, focusing primarily on infinite horizon discrete time models and models with discrete time spaces while also examining models with arbitrary state spaces, finite horizon models, and continuous time discrete state models.
Book
Markov Chains and Stochastic Stability
Sean P. Meyn,Richard L. Tweedie +1 more
TL;DR: This second edition reflects the same discipline and style that marked out the original and helped it to become a classic: proofs are rigorous and concise, the range of applications is broad and knowledgeable, and key ideas are accessible to practitioners with limited mathematical background.
MonographDOI
Markov Decision Processes
P. Whittle,M. L. Puterman +1 more
TL;DR: Markov Decision Processes covers recent research advances in such areas as countable state space models with average reward criterion, constrained models, and models with risk sensitive optimality criteria, and explores several topics that have received little or no attention in other books.