scispace - formally typeset
Open AccessJournal ArticleDOI

A survey of multi-objective sequential decision-making

TLDR
This article surveys algorithms designed for sequential decision-making problems with multiple objectives and proposes a taxonomy that classifies multi-objective methods according to the applicable scenario, the nature of the scalarization function, and the type of policies considered.
Abstract
Sequential decision-making problems with multiple objectives arise naturally in practice and pose unique challenges for research in decision-theoretic planning and learning, which has largely focused on single-objective settings. This article surveys algorithms designed for sequential decision-making problems with multiple objectives. Though there is a growing body of literature on this subject, little of it makes explicit under what circumstances special methods are needed to solve multi-objective problems. Therefore, we identify three distinct scenarios in which converting such a problem to a single-objective one is impossible, infeasible, or undesirable. Furthermore, we propose a taxonomy that classifies multi-objective methods according to the applicable scenario, the nature of the scalarization function (which projects multi-objective values to scalar ones), and the type of policies considered. We show how these factors determine the nature of an optimal solution, which can be a single policy, a convex hull, or a Pareto front. Using this taxonomy, we survey the literature on multi-objective methods for planning and learning. Finally, we discuss key applications of such methods and outline opportunities for future work.

read more

Content maybe subject to copyright    Report

Citations
More filters
Proceedings Article

Bounded approximations for linear multi-objective planning under uncertainty

TL;DR: It is shown empirically that the two algorithms proposed are able to produce (near-)optimal alternative sets orders of magnitude faster than existing techniques, thereby demonstrating that the methods provide sensible approximations in stochastic multi-objective domains.
Proceedings ArticleDOI

Modular Multi-Objective Deep Reinforcement Learning with Decision Values

TL;DR: In this article, the authors propose an architecture in which separate DQNs are used to control the agent's behaviour with respect to particular objectives and introduce decision values to improve the scalarization of multiple DQN into a single action.
Book ChapterDOI

Queued Pareto Local Search for Multi-Objective Optimization

TL;DR: It is proved that QPLS terminates and it is shown that it can be embedded in a genetic search scheme that improves the approximate Pareto front with every iteration and produces good approximations faster, and leads to better approximation than popular alternative MOEAs.
Journal ArticleDOI

A Temporal Difference Method for Multi-Objective Reinforcement Learning

TL;DR: This work describes MPQ-learning, an algorithm that approximates the set of all deterministic non-dominated policies in multi-objective Markov decision problems, where rewards are vectors and each component stands for an objective to maximize.

Quality Assessment of MORL Algorithms: A Utility-Based Approach

TL;DR: This paper proposes two metrics that express either the expected utility, or the maximal utility loss with respect to the optimal solution set, and proposes a generalised benchmark in order to compare algorithms more reliably.
References
More filters
Book

Dynamic Programming

TL;DR: The more the authors study the information processing aspects of the mind, the more perplexed and impressed they become, and it will be a very long time before they understand these processes sufficiently to reproduce them.
Book

Markov Decision Processes: Discrete Stochastic Dynamic Programming

TL;DR: Puterman as discussed by the authors provides a uniquely up-to-date, unified, and rigorous treatment of the theoretical, computational, and applied research on Markov decision process models, focusing primarily on infinite horizon discrete time models and models with discrete time spaces while also examining models with arbitrary state spaces, finite horizon models, and continuous time discrete state models.
Book

Introduction to Reinforcement Learning

TL;DR: In Reinforcement Learning, Richard Sutton and Andrew Barto provide a clear and simple account of the key ideas and algorithms of reinforcement learning.
Book

Evolutionary algorithms for solving multi-objective problems

TL;DR: This paper presents a meta-anatomy of the multi-Criteria Decision Making process, which aims to provide a scaffolding for the future development of multi-criteria decision-making systems.
Proceedings Article

Policy Gradient Methods for Reinforcement Learning with Function Approximation

TL;DR: This paper proves for the first time that a version of policy iteration with arbitrary differentiable function approximation is convergent to a locally optimal policy.
Related Papers (5)