A survey of multi-objective sequential decision-making
TLDR
This article surveys algorithms designed for sequential decision-making problems with multiple objectives and proposes a taxonomy that classifies multi-objective methods according to the applicable scenario, the nature of the scalarization function, and the type of policies considered.Abstract:
Sequential decision-making problems with multiple objectives arise naturally in practice and pose unique challenges for research in decision-theoretic planning and learning, which has largely focused on single-objective settings. This article surveys algorithms designed for sequential decision-making problems with multiple objectives. Though there is a growing body of literature on this subject, little of it makes explicit under what circumstances special methods are needed to solve multi-objective problems. Therefore, we identify three distinct scenarios in which converting such a problem to a single-objective one is impossible, infeasible, or undesirable. Furthermore, we propose a taxonomy that classifies multi-objective methods according to the applicable scenario, the nature of the scalarization function (which projects multi-objective values to scalar ones), and the type of policies considered. We show how these factors determine the nature of an optimal solution, which can be a single policy, a convex hull, or a Pareto front. Using this taxonomy, we survey the literature on multi-objective methods for planning and learning. Finally, we discuss key applications of such methods and outline opportunities for future work.read more
Citations
More filters
Proceedings Article
Random Hypervolume Scalarizations for Provable Multi-Objective Black Box Optimization
Richard Zhang,Daniel Golovin +1 more
TL;DR: This paper introduces a novel scalarization function, which it is shown that drawing random scalarizations from an appropriately chosen distribution can be used to efficiently approximate the hypervolume indicator metric, and highlights the general utility of this framework by showing that any provably convergent single-objective optimization process can be effortlessly converted to a multi-objectives optimization process with provable convergence guarantees.
Journal ArticleDOI
Many-objective stochastic path finding using reinforcement learning
TL;DR: This approach is the first to provide optimal performance for an autonomous, intelligent system operating in a many objective environment, and is demonstrated with multiple deterministic and stochastic many-objective path finding problems that are solved optimally without any advance knowledge of the problem or interaction with a decision maker.
Journal ArticleDOI
Steering approaches to Pareto-optimal multiobjective reinforcement learning
Peter Vamplew,Rustam Issabekov,Richard Dazeley,Cameron Foale,Adam Berry,Tim Moore,Douglas Creighton +6 more
TL;DR: This paper investigates two novel algorithms for learning non-stationary policies which produce Pareto-optimal behaviour (w- Steering and Q-steering), by extending prior work based on the concept of geometric steering to demonstrate broader applicability.
Book ChapterDOI
Simple Strategies in Multi-Objective MDPs
TL;DR: It is shown that checking whether a point is achievable by a pure stationary strategy is NP-complete, even for two objectives, and the author provides an MILP encoding to solve the corresponding problem.
Book ChapterDOI
Multi-cost Bounded Reachability in MDP
TL;DR: The need for output beyond Pareto curves is discussed and the available information from the algorithm is exploited to support decision makers and show the algorithm’s scalability.
References
More filters
Book
Dynamic Programming
TL;DR: The more the authors study the information processing aspects of the mind, the more perplexed and impressed they become, and it will be a very long time before they understand these processes sufficiently to reproduce them.
Book
Markov Decision Processes: Discrete Stochastic Dynamic Programming
TL;DR: Puterman as discussed by the authors provides a uniquely up-to-date, unified, and rigorous treatment of the theoretical, computational, and applied research on Markov decision process models, focusing primarily on infinite horizon discrete time models and models with discrete time spaces while also examining models with arbitrary state spaces, finite horizon models, and continuous time discrete state models.
Book
Introduction to Reinforcement Learning
TL;DR: In Reinforcement Learning, Richard Sutton and Andrew Barto provide a clear and simple account of the key ideas and algorithms of reinforcement learning.
Book
Evolutionary algorithms for solving multi-objective problems
TL;DR: This paper presents a meta-anatomy of the multi-Criteria Decision Making process, which aims to provide a scaffolding for the future development of multi-criteria decision-making systems.
Proceedings Article
Policy Gradient Methods for Reinforcement Learning with Function Approximation
TL;DR: This paper proves for the first time that a version of policy iteration with arbitrary differentiable function approximation is convergent to a locally optimal policy.
Related Papers (5)
Human-level control through deep reinforcement learning
Mastering the game of Go with deep neural networks and tree search
David Silver,Aja Huang,Chris J. Maddison,Arthur Guez,Laurent Sifre,George van den Driessche,Julian Schrittwieser,Ioannis Antonoglou,Veda Panneershelvam,Marc Lanctot,Sander Dieleman,Dominik Grewe,John Nham,Nal Kalchbrenner,Ilya Sutskever,Timothy P. Lillicrap,Madeleine Leach,Koray Kavukcuoglu,Thore Graepel,Demis Hassabis +19 more