Journal ArticleDOI
Lipschitz Continuity of Value Functions in Markovian Decision Processes
Reads0
Chats0
TLDR
Tools and guidelines for investigating Lipschitz continuity of the value functions in MDP’s, using the Hausdorff metric and the Kantorovich metric for measuring the influence of the constraint set and the transition law, respectively are presented.Abstract:
We present tools and guidelines for investigating Lipschitz continuity of the value functions in MDP’s, using the Hausdorff metric and the Kantorovich metric for measuring the influence of the constraint set and the transition law, respectively. The methods are explained by examples. Additional topics include an application to the the discretization algorithm of Bertsekas (1975).read more
Citations
More filters
Posted Content
Algorithmic Framework for Model-based Deep Reinforcement Learning with Theoretical Guarantees
TL;DR: A novel algorithmic framework for designing and analyzing model-based RL algorithms with theoretical guarantees is introduced and a meta-algorithm with a theoretical guarantee of monotone improvement to a local maximum of the expected reward is designed.
Journal ArticleDOI
Policy gradient in Lipschitz Markov Decision Processes
TL;DR: This paper shows that both the expected return of a policy and its gradient are Lipschitz continuous w.r.t. policy parameters and defines policy-parameter updates that guarantee a performance improvement at each iteration.
Journal ArticleDOI
Approximation of Markov decision processes with general state space
TL;DR: A state and action discretization procedure for approximating the optimal value function and an optimal policy of the original control model is proposed and explicit bounds on the approximation errors are provided.
Posted Content
Lipschitz Continuity in Model-based Reinforcement Learning
TL;DR: The authors examined the impact of learning Lipschitz continuous models in the context of model-based reinforcement learning and provided a bound on multi-step prediction error using the Wasserstein metric.
Journal ArticleDOI
Finite Linear Programming Approximations of Constrained Discounted Markov Decision Processes
TL;DR: This work proposes a finite state approximation of the linear programming formulation of the constrained MDP to a finite-dimensional static optimization problem that can be used to obtain explicit numerical approximations of the corresponding optimal constrained cost.
References
More filters
Book
Real and abstract analysis
Edwin Hewitt,Karl R. Stromberg +1 more
TL;DR: The first € price and the £ and $ price are net prices, subject to local VAT, and the €(D) includes 7% for Germany, the€(A) includes 10% for Austria.
Journal ArticleDOI
Real and Abstract Analysis. By E. Hewitt and K. Stromberg Pp. viii, 476. 1965. (Springer-Verlag.)
Book
Real analysis and probability
TL;DR: This book discusses set theory, vector spaces, and Taylor's theorem with remainder, as well as general topology, measurement, and differentiation, and introduces probability theory.
Journal ArticleDOI
Integral Probability Metrics and Their Generating Classes of Functions
TL;DR: A unified study of integral probability metrics of the following type are given and how some interesting properties of these probability metrics arise directly from conditions on the generating class of functions is shown.
Book ChapterDOI
Controlled Markov Processes
TL;DR: This chapter introduces the stochastic control processes, also known as Markov decision processes or Markov dynamic programs, and discusses (briefly) more general control systems, such as non-stationary CMP’s and semi-Markov control models.