scispace - formally typeset
Open AccessJournal ArticleDOI

Normative Uncertainty as a Voting Problem

William MacAskill
- 01 Oct 2016 - 
- Vol. 125, Iss: 500, pp 967-1004
TLDR
In this paper, the Borda Rule is used to compare magnitudes of choice-worthiness across different theories of social choice in the context of decision-making in the face of normative uncertainty and intertheoretic incomparability.
Abstract
Some philosophers have recently argued that decision-makers ought to take normative uncertainty into account in their decisionmaking. These philosophers argue that, just as it is plausible that we should maximize expected value under empirical uncertainty, it is plausible that we should maximize expected choice-worthiness under normative uncertainty. However, such an approach faces two serious problems: how to deal with merely ordinal theories, which do not give sense to the idea of magnitudes of choice-worthiness; and how, even when theories do give sense to magnitudes of choice-worthiness, to compare magnitudes of choice-worthiness across different theories. Some critics have suggested that these problems are fatal to the project of developing a normative account of decision-making under normative uncertainty. The primary purpose of this article is to show that this is not the case. To this end, I develop an analogy between decisionmaking under normative uncertainty and the problem of social choice, and then argue that the Borda Rule provides the best way of making decisions in the face of merely ordinal theories and intertheoretic incomparability.

read more

Content maybe subject to copyright    Report

1"
"
Normative Uncertainty as a Voting Problem
WILLIAM MACASKILL
University of Oxford University
William.Macaskill@philosophy.ox.ac.uk

2"
"
1. Introduction
Often, you are not certain about what you ought to do. You might have heard the
arguments for vegetarianism, but be only partially convinced, and so be uncertain
whether you ought not to eat meat. On a larger scale, the government of a country might
be uncertain whether it should act solely in the interests of those it currently governs, or
whether it should take into account the interests of future generations as well. Indeed,
ethics is a subject that is rife with controversy: for almost any ethical view, there seems to
be something to be said in its favour. For even moderately reflective decision-makers,
therefore, normative uncertainty is the norm.
Recently, some philosophers
1
have argued that decision-makers ought to take normative
uncertainty into account in their decision-making. Consider the following case.
Suit
Jo Bloggs has £1000 to spend, and she wants to buy a nice suit. Just before
buying it, however, she remembers Peter Singer’s article ‘Famine, Affluence and
Morality’.
2
She believes that, if Singer is right, buying the suit would be as wrong
as letting a child which she could have easily saved drown in front of her. But she
is not that convinced by his argument: she believes with 80% certainty that she
ought to buy the suit, because of the pleasure it would give her, and gives Singer’s
conclusion only 20% credence.
By her own lights, should Jo buy the suit, or should she donate the money? Assume, for
simplicity, that these are the only two options. There are two cases in which she makes
the correct decision: (1) she buys the suit, and she ought to buy the suit; (2) she donates,
and she ought to donate. And there are two cases in which she makes the wrong decision:
(3) she buys the suit, but she ought to donate; (4) she donates, but she ought to buy the
suit.
One might think that she should act on the belief in which she has most credence, and
buy the suit. But that overlooks an important asymmetry in the bad outcomes. (3) is a far,
far worse outcome, morally, than (4): if (4) is true, Jo’s act is only wrong insofar as she
loses a small amount of aesthetic pleasure; but if (3) is true, her act is as wrong as walking
past a drowning child. So, given what she believes, she should not buy the suit. To do so
would be morally reckless: ignoring the small risk of grave wrongdoing.
"""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""
1
For example, Oddie 1995; Lockhart 2000; Ross 2006; Guerrero 2007; Sepielli 2009. John Broome flags
the importance of this idea, though he does not state his views on it, in Broome 2010.
2
Singer 1972.

3"
"
This reasoning just imports the idea, familiar from decision theory, that decision-makers
ought to take both the probability of an outcome and the magnitude of the value of that
outcome into account when making their decision. If we ought to do this under empirical
uncertainty, it is plausible to suggest that we should do this under normative uncertainty
as well,
3
and, just as it is plausible that we should maximize expected value under
empirical uncertainty, it is plausible that we should maximize expected choice-worthiness
under normative uncertainty.
This idea is at least prima facie compelling when applied to the Suit case. But when we
look to apply the idea more generally, two problems immediately rear their heads. First,
what should you do if one of the theories you have credence in does not give sense to the
idea of magnitudes of choice-worthiness? Some theories will tell you that murder is a
more serious wrong than lying is, but will not give any way of determining how much more
serious a wrong murder is than lying. But if it does not make sense to talk about
magnitudes of choice-worthiness, on a particular theory, then we will not be able to take
an expectation over that theory. I will call this the problem of merely ordinal theories.
A second problem is that, even when all theories under consideration give sense to the
idea of magnitudes of choice-worthiness, we need to be able to compare these magnitudes
of choice-worthiness across different theories. But it seems that we cannot always do this.
A rights-based theory claims that it would be wrong to kill one person in order to save
fifty; utilitarianism claims that it would be wrong not to do so. But how can we compare
the seriousness of the wrongs, according to these different theories? For which theory is
there more at stake? In line with the literature,
4
I will call this the problem of intertheoretic
comparisons.
Some philosophers have suggested that these problems are fatal to the project of
developing a normative account of decision-making under normative uncertainty.
5
The
primary purpose of this article is to show that this is not the case. To this end, I will
develop an analogy between decision-making under normative uncertainty and the
problem of social choice, and then argue that the Borda Rule provides the best way of
making decisions in the face of merely ordinal theories and intertheoretic
incomparability.
"""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""
3
Advocates of this idea include Lockhart 2000; Ross 2006; and Sepielli 2009.
4
E.g. Lockhart 2000; Ross 2006; Sepielli 2009.
5
E.g. Gracely 1996; Hudson 1989; Ross 2006. In conversation, John Broome has suggested that the
problem of intertheoretic comparisons is ‘devastating’ for accounts of decision-making under normative
uncertainty; Derek Parfit has described the problem of intertheoretic comparison as ‘fatal’.

4"
"
Before I begin, let me introduce the framework within which I work. What I will call a
decision-situation is a quintuple 𝑆, 𝑡, 𝐴, 𝑇, 𝐶, where S is a decision-maker, t is a time, 𝐴 is a
set of options that the decision-maker has the power to bring about at that time, 𝑇 is the
set, assumed to be finite, of first-order normative theories considered by the decision-
maker at that time, and 𝐶 is a credence function representing the decision-maker’s beliefs
about first-order normative theories. I will now elaborate on each of the last three
members of this quintuple in turn.
First, options. An option is a proposition, understood as a set of centred possible worlds,
that the decision-maker has the power to make true at a time. A centred possible world is
a triple of a world, an agent in that world, and a time in the history of that world.
6
By
convention, the options in 𝐴 are specified so as to be mutually exclusive and jointly
exhaustive. Furthermore, all the worlds in all the options in A are centred on the same
agent, S, and the same time, t. I will use the letters A, B, C etc to refer to specific options,
and italicised letters A, B, C etc to refer to variables. I assume that the constitution of the
relevant option-set is not something about which different first-order normative theories
disagree.
7
Second, normative theories. A first-order normative theory (or ‘theory’ for short) is an
ordering, for all possible sets of options 𝐴, of the options in that set in terms of their
choice-worthiness. I therefore assume that the theories in 𝑇 are mutually exclusive and
jointly exhaustive.
8
I take choice-worthiness to be defined as the ordering that first-order
normative theories produce; put intuitively, the choice-worthiness of an option on a
particular first-order normative theory is the degree to which the decision-maker ought to
choose that option, according to that theory. I use 𝐴
!
𝐵 to mean A is at least as
choice-worthy as B, according to 𝑇
!
,’ and I define A is strictly more choice-worthy than
"""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""
6
An option therefore can include the action available to the decision-maker, as well as the decision-maker’s
intention, motive, the outcome of the action, and everything else that could be normatively relevant to the
decision-maker’s decision.
7
This assumption does some real work: for example, Bergström (1971), argues that which option-set is
relevant, in a given decision-situation, is in part a normative question, which may therefore vary from
theory to theory. (I thank an anonymous referee for pointing out this issue.) However, while this issue is
interesting and important, it is one that I will have to leave to the side for the purposes of this paper. If one
endorses Bergström’s view, then one should assume that I am only discussing those situations where all
theories in which the decision-maker has positive credence agree on what the relevant option-set is.
8
In defining normative theories this way I also assume that the decision-maker cannot be uncertain about
what is entailed by a given normative theory. This, of course, is highly unrealistic, if we are using
‘normative theory’ within its usual meaning. (I thank an anonymous referee for pressing this point).
However, I am using ‘normative theory’ in a technical sense. If a decision-maker is unsure about whether,
according to Kantianism, A is more choice-worthy than B or vice-versa, then I would describe her as
uncertain between two distinct normative theories: Kantianism’, according to which A is more choice-
worthy than B, and Kantianism’’, according to which B is more choice-worthy than A. To my knowledge,
nothing hangs on this.

5"
"
B, according to 𝑇
!
(or 𝐴
!
𝐵) as (𝐴
𝑖
𝐵)!&!¬(𝐵
𝑖
𝐴), and ‘A is equally as choice-
worthy as B, according to 𝑇
!
(or 𝐴~
!
𝐵) as (𝐴
𝑖
𝐵)!&!(𝐵
𝑖
𝐴). Note that by ‘choice-
worthiness’ I do not mean merely moral choice-worthiness, in the sense in which the moral
choice-worthiness of an act might be weighed against the prudential choice-worthiness of
it. I mean all-things-considered choice-worthiness: the normative ordering of options after all
normatively relevant considerations have been taken into account (according to a
particular first-order normative theory).
9
First-order normative theories’ choice-
worthiness orderings can be represented by a choice-worthiness function, which is a function
from options to numbers such that 𝐶𝑊
!
(𝐴) 𝐶𝑊
!
(𝐵) iff 𝐴
!
𝐵, using 𝐶𝑊
!
(𝐴) to
represent the numerical value that the choice-worthiness function 𝐶𝑊
!
assigns to A.
10
The fifth element in a decision-situation is the credence function. The credence function
𝐶 is a representation of the decision-makers partial beliefs over first-order normative
theories. It is a function from every theory 𝑇
!
𝑇 to a real number in the interval [0,1],
such that the sum of credences across all theories in!𝑇 equals 1.
11,12
I introduce a new term, appropriateness, which is defined as the ordering that
metanormative theories produce; put intuitively, the appropriateness of an option is the
degree to which the decision-maker ought to choose that option, in the sense of ‘ought’
that is relevant to decision-making under normative uncertainty. I represent that A is at
least as appropriate as B as A
!
B, defining the relations ‘more appropriate than’ and
‘equally as appropriate as’ in the standard way. As long as the appropriateness relation is
reflexive, transitive, and complete, it may be represented with an appropriateness function: a
function from options to real numbers such that 𝐴𝑃(𝐴) 𝐴𝑃(𝐵)!iff 𝐴
!
𝐵, where
𝐴𝑃(𝐴)’ is the numerical value that the appropriateness function assigns to A.
"""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""
9
In this article, I do not consider how to accommodate theories that involve supererogation. This is an
important issue, but one that must be left for another time. So I only discuss theories according to which all
permissible options are maximally choice-worthy.
10
In a purely ordinal and non-comparable information setting, one could simply talk about orderings,
rather than choice-worthiness functions, as Arrow did. However, insofar as we might wish to use the social
choice analogy in order to develop metanormative theories in conditions of interval-scale measurability or
intertheoretic comparability, then it is better to rely on choice-worthiness functions, which are able to
accommodate this.
11
For simplicity I assume empirical certainty, and I assume that the decision-makers’ credences across
theories are evidentially independent of her actions; nothing will hang on this in what follows.
12
One might worry that, if non-cognitivism true, then one cannot make sense of credences over normative
theories. There has been debate on this issue in the literature, e.g. (Smith 2002; Bykvist and Olson 2009;
Sepielli 2012; Olson and Bykvist 2012). However, I wish to sidestep this debate in this article, so for the
purpose of this paper I assume that cognitivism is true.

Citations
More filters
Journal ArticleDOI

Artificial Intelligence, Values, and Alignment

Iason Gabriel
- 03 Oct 2020 - 
TL;DR: In this paper, a principle-based approach to AI alignment is proposed, which combines normative and technical aspects of the AI alignment problem, creating space for productive engagement between people working in both domains.
Journal ArticleDOI

Hard choices in artificial intelligence

TL;DR: The vagueness in debates about the safety and ethical behavior of AI systems is examined, and it is shown how it cannot be resolved through mathematical formalism alone, instead requiring deliberation about the politics of development as well as the context of deployment.
Journal ArticleDOI

Implementation of Moral Uncertainty in Intelligent Machines

TL;DR: This paper argues that the proper response to the development of artificial intelligence is to design machines to be fundamentally uncertain about morality, and describes a computational framework for doing so that efficiently resolves common obstacles to the implementation of moral philosophy in intelligent machines.
Journal ArticleDOI

Why Maximize Expected Choice-Worthiness?1

TL;DR: In this paper, the authors argue in favor of a particular account of decision-making under normative uncertainty: that, when it is possible to do so, one should maximize expected choice-worthiness.
References
More filters
Book

Theory of Games and Economic Behavior

TL;DR: Theory of games and economic behavior as mentioned in this paper is the classic work upon which modern-day game theory is based, and it has been widely used to analyze a host of real-world phenomena from arms races to optimal policy choices of presidential candidates, from vaccination policy to major league baseball salary negotiations.
Journal ArticleDOI

On the Theory of Scales of Measurement

TL;DR: The current issues will remain at 32 pages until a more adequate supply of paper is assured, due to a shortage of paper for Bacto-Agar research.
Book

Collective Choice and Social Welfare

Amartya Sen
TL;DR: The second edition of Collective Choice and Social Welfare as discussed by the authors was published in 1970 and has been widely used in the social choice literature since its early 1970s, and is considered a classic work in social choice.
Book

What We Owe to Each Other

TL;DR: The Structure of Contractualism and Responsibility Promises Relativism as discussed by the authors, and Williams on Internal and External Reasons and Values Reasons Values Well-Being Right and Wrong Wrongness and Reasons
Frequently Asked Questions (1)
Q1. What have the authors contributed in "Normative uncertainty as a voting problem" ?

This paper showed that normative uncertainty is a fact of human life and should be taken into account in decision-making.