A survey of multi-objective sequential decision-making
TLDR
This article surveys algorithms designed for sequential decision-making problems with multiple objectives and proposes a taxonomy that classifies multi-objective methods according to the applicable scenario, the nature of the scalarization function, and the type of policies considered.Abstract:
Sequential decision-making problems with multiple objectives arise naturally in practice and pose unique challenges for research in decision-theoretic planning and learning, which has largely focused on single-objective settings. This article surveys algorithms designed for sequential decision-making problems with multiple objectives. Though there is a growing body of literature on this subject, little of it makes explicit under what circumstances special methods are needed to solve multi-objective problems. Therefore, we identify three distinct scenarios in which converting such a problem to a single-objective one is impossible, infeasible, or undesirable. Furthermore, we propose a taxonomy that classifies multi-objective methods according to the applicable scenario, the nature of the scalarization function (which projects multi-objective values to scalar ones), and the type of policies considered. We show how these factors determine the nature of an optimal solution, which can be a single policy, a convex hull, or a Pareto front. Using this taxonomy, we survey the literature on multi-objective methods for planning and learning. Finally, we discuss key applications of such methods and outline opportunities for future work.read more
Citations
More filters
Journal ArticleDOI
Synthesizing safe policies under probabilistic constraints with reinforcement learning and Bayesian model checking
Lenz Belzner,Martin Wirsing +1 more
TL;DR: In this article, the authors leverage epistemic uncertainty about constraint satisfaction of a reinforcement learner in safety critical domains, and introduce a framework for specification of requirements for reinforcement learners in constrained settings, including confidence about results.
Posted Content
A Practical Guide to Multi-Objective Reinforcement Learning and Planning.
Conor Hayes,Roxana Radulescu,Eugenio Bargiacchi,Johan Källström,Matthew Macfarlane,Mathieu Reymond,Timothy Verstraeten,Luisa M. Zintgraf,Richard Dazeley,Fredrik Heintz,Enda Howley,Athirai A. Irissappane,Patrick Mannion,Ann Nowé,Gabriel de Oliveira Ramos,Marcello Restelli,Peter Vamplew,Diederik M. Roijers +17 more
TL;DR: In this article, a guide to the application of multi-objective decision-making methods to difficult problems is presented, aimed at researchers who are already familiar with singleobjective reinforcement learning and planning methods and who wish to adopt a multiobjective perspective on their research.
Journal Article
Behaviour-Diverse Automatic Penetration Testing: A Curiosity-Driven Multi-Objective Deep Reinforcement Learning Approach
Yizhou Yang,Xin Liu +1 more
TL;DR: This work forms the automatic penetration testing in the Multi-Objective Reinforcement Learning (MORL) framework and proposes a Chebyshev decomposition critic to find diverse adversary strategies that balance different objectives in the penetration test.
Posted Content
Expected Scalarised Returns Dominance: A New Solution Concept for Multi-Objective Decision Making.
TL;DR: In this paper, a new dominance criterion, known as expected scalarised returns (ESR) dominance, was proposed, which extends first-order stochastic dominance to allow a set of optimal policies to be learned in practice.
Journal ArticleDOI
Evolutionary Reinforcement Learning: A Survey
Hui Bai,Ran Cheng,Yaochu Jin +2 more
TL;DR: A comprehensive survey of state-of-the-art methods for integrating evolutionary computation into reinforcement learning (EvoRL) is presented in this article , including hyperparameter optimization, policy search, exploration, reward shaping, meta-RL, and multi-objective RL.
References
More filters
Book
Dynamic Programming
TL;DR: The more the authors study the information processing aspects of the mind, the more perplexed and impressed they become, and it will be a very long time before they understand these processes sufficiently to reproduce them.
Book
Markov Decision Processes: Discrete Stochastic Dynamic Programming
TL;DR: Puterman as discussed by the authors provides a uniquely up-to-date, unified, and rigorous treatment of the theoretical, computational, and applied research on Markov decision process models, focusing primarily on infinite horizon discrete time models and models with discrete time spaces while also examining models with arbitrary state spaces, finite horizon models, and continuous time discrete state models.
Book
Introduction to Reinforcement Learning
TL;DR: In Reinforcement Learning, Richard Sutton and Andrew Barto provide a clear and simple account of the key ideas and algorithms of reinforcement learning.
Book
Evolutionary algorithms for solving multi-objective problems
TL;DR: This paper presents a meta-anatomy of the multi-Criteria Decision Making process, which aims to provide a scaffolding for the future development of multi-criteria decision-making systems.
Proceedings Article
Policy Gradient Methods for Reinforcement Learning with Function Approximation
TL;DR: This paper proves for the first time that a version of policy iteration with arbitrary differentiable function approximation is convergent to a locally optimal policy.
Related Papers (5)
Human-level control through deep reinforcement learning
Mastering the game of Go with deep neural networks and tree search
David Silver,Aja Huang,Chris J. Maddison,Arthur Guez,Laurent Sifre,George van den Driessche,Julian Schrittwieser,Ioannis Antonoglou,Veda Panneershelvam,Marc Lanctot,Sander Dieleman,Dominik Grewe,John Nham,Nal Kalchbrenner,Ilya Sutskever,Timothy P. Lillicrap,Madeleine Leach,Koray Kavukcuoglu,Thore Graepel,Demis Hassabis +19 more