A Zero Attraction Effect in Naturalistic Choice

TL;DR: In the attraction effect, adding a dominated third option to a choice set of two options can reverse the preference for the original two options, and even increase one of the option's choice share as discussed by the authors.

Abstract: In the attraction effect, adding a dominated third option to a choice set of two options can reverse the preference for the original two options, and even increase one of the option’s choice share. This constitutes a violation of the axioms of regularity and independence from irrelevant alternatives, which are core properties of any choice model in which the utility of each option is stable across choice sets. Consequently, in the past 20 years, the attraction effect has driven the development of a set of influential models of multiattribute choice. However, Frederick, Lee, and Baskin (2014) have recently claimed that the attraction effect is only limited to options with numerical attributes, and does not hold for choices between naturalistic options (e.g., snacks, movies) — a claim which would severely undermine its theoretical importance. Huber, Payne, and Puto (2014) criticised Frederick et al.’s experiments, laying down a set of criteria that should be met by any experiment wishing to test for the attraction effect in real-world consumer choices. This article presents the first experiment that meets these criteria. The results show a precisely zero attraction effect.

Summary

Introduction

• This can constitute a violation of the axioms of regularity and independence from irrelevant alternatives, which are core properties of any choice model in which the utility of each option is stable across choice sets.
• In the past 20 years, the attraction effect has driven the development of a set of influential models of multiattribute choice.
• Two studies (Frederick et al., 2014; Yang & Lynn, 2014) involving a series of experiments with naturalistic choice options (e.g., snacks, movies) found no evidence for the attraction effect in choice contexts where alternatives are represented without objectively defined (e.g., numerical) attribute dimensions—a finding that would severely undermine its theoretical importance.
• The results show a precisely zero attraction effect.

The Attraction Effect

• Imagine that you are having a nice meal in a restaurant, and you are looking at the two dessert options available on the menu: cheesecake and pecan pie.
• The introduction of the apple pie to the choice set makes you more likely to choose the pecan pie over the cheesecake .
• This article has been published under the terms of the Creative Commons Attribution License (http://creativecommons.org/ licenses/by/3.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
• Choice options that are represented along two attribute dimensions (perceptual or numerical) can be relatively easily manipulated within a choice experiment, but most real-world choices involve complex, naturalistic objects with a large number of underlying attributes that often cannot be represented in a stylized format.
• The authors describe a rigorous test of the attraction effect with complex, naturalistic choice options, using a carefully developed experimental methodology that addresses all of the critical conditions discussed by Huber et al. (2014).

Testing the Attraction Effect With Real-World Stimuli

• The goal of this experiment was to test the attraction effect with naturalistic stimuli.
• When the stimuli are represented with welldefined attribute dimensions, it is straightforward to construct choice triplets with a target, competitor, and decoy option.
• First, such stimuli can have a high number of attribute dimensions that might vary across individuals.
• They chose pairs of movies that are part of the same series or are starring the same actor to create target– decoy pairs.
• Figure 1 shows an example of two choice triplets created from one such quadruplet of movies (in the experiment participants were presented with real movie posters).

Candidate Choice Set Selection

• To determine the choice sets used in the experiment, the first step was to create a set of quadruplets, each consisting of two movies that are very similar (for the two target–decoy pairs; i.e., A–A,= B–B=), while making sure that the two pairs are sufficiently different from each other (for the target–competitor pair; i.e., A–B).
• To obtain the set of 400 movies, the authors first retrieved 40 of the most popular movies from each of 10 distinct genre categories (romance, drama, sci-fi, thriller, comedy, horror, animation, fantasy, crime, action) from IMDb.
• The authors selected a movie pair as target–decoy candidate if, for a given target candidate movie, the number of overlapping genres with a candidate decoy was equal to the maximum overlap seen for that target across all candidate decoy movies.
• Participants to rate the similarity of a randomly chosen group of movie pairs, obtaining 10 independent similarity ratings for each of the 1,242 target– decoy candidates.
• Therefore, the authors decided to pair up the least similar 253 target– decoy pairs (where similarity is captured by genre overlap) to create the quadruplets.

Experimental Procedures

• The experiment consisted of three stages: rating stage, choice stage, and a similarity rating stage.
• Before the choice stage, the authors created a bespoke set of movie triplets for each participant using their ratings from the rating stage.
• First, based on each participant’s preference ratings, the authors identified the subset of quadruplets where (a) the target and competitor were both rated 4, both rated 5, both rated 6, or both rated 7 and (b) the two decoy movies were rated at least 3 points lower than the two target candidates.
• The authors recruited 297 English-speaking participants from Prolific Academic who were paid €8 per hour.
• Out of the 179 participants who were invited back, 152 took part in the choice stage of the experiment.

Exclusion Criteria

• To conduct a rigorous test of the attraction effect, it is crucial that people take the task seriously and reveal their true preferences.
• Given that individually rating 231 movies can seem somewhat mundane, the authors specified a set of exclusion criteria to filter those people out who did not take the rating task sufficiently seriously.
• The authors excluded people who fell into the fastest 5% of the reaction time (RT) distribution, the lowest 5% of the entropy distribution, and the upper and lower 5% of the autocorrelation distribution.
• Thus, this procedure filtered out response patterns where people (a) spent an unusually short time completing the task, (b) did not use the whole of the ratings scale, (c) often gave the same ratings for consecutive movies, or (d) were giving ratings randomly.
• When testing for the attraction effect, the authors have also removed all trials where the decoy was chosen.

Results

• The median number of choice trials per participant was 16 (range: 6–54), and 84% of participants were presented with at least 8 choice trials.
• In addition, 72% of participants never chose the decoy, and only 2% chose it in more than 25% of the trials.
• As Figure 6 shows, the overwhelming majority of target–competitor pairs were perceived as not similar, while the majority of target–decoy pairs were perceived as similar, exactly as the authors intended.
• Using Model 1, the authors calculate that the overall probability of choosing the target is .50, 95% CI [.48, .52], the same as what they see in the mean over subject proportions from Figure 5.
• None of the ratings modulate the strength of the attraction effect.

Further Analyses

• During the peer-review process, the authors also completed the following exploratory analyses.
• Faced with two subsequent choices involving two equally highly rated A–B movie pairs and two different but undesirable decoys, it is possible that the first choice is “sticky” and will be repeated.
• When all of the covariates are included (Model 8), the authors still do not find evidence for the attraction effect, although the effect is less precisely estimated.
• Overall, while the results indicate that overall genre preferences slightly influenced choices between the target and competitor, the authors found no evidence that this had any effect on the strength of the attraction effect.
• While the authors have not found any evidence suggesting that the target– decoy rating difference influenced the strength of the attraction effect (see Model 2 in Table 1), a nonlinear association between target–decoy preference and the attraction effect might still exist.

General Discussion

• The authors tested for the attraction effect in a choice task with naturalistic choice options.
• The decoy was only chosen in 4.3% of the trials, which clearly shows that participants were able to spot and avoid the dominated option.
• Finally, in their analysis, the authors controlled for familiarity with the choice options, perceived similarity of the target– decoy and target– competitor pair, and preference difference between the target and the decoy, but they found that none of these modulated or revealed an attraction effect.
• Attribute information is spatially separate in a numerical matrix form presentation, but attribute information occurs in the same spatial location for movie thumbnails.

Figures

Figure 3 The Distribution of the Average Similarity Rating for Each Target–Decoy Candidate Pair (N 1,242)

Table 3 Odds Ratios and 95% CIs From Three Mixed-Effects Logistic Models Fit by Maximum Likelihood With Subject-Specific Intercepts

Figure 4 Choice Stage

Table 2 Odds Ratios and 95% CIs From Three Mixed-Effects Logistic Models Fit by Maximum Likelihood With Subject-Specific Intercepts, First Choices Only

Figure 5 The Proportion of Trials Where the Target Was Chosen Instead of the Competitor

Figure 1 Two Choice Triplets Used in the Experiment

Figure 8 The Proportion of Trials Where the Target Was Chosen Instead of the Competitor by Target–Competitor Rating

Figure 7 The Predicted Proportion of Trials Where the Target Was Chosen Instead of the Competitor, by Target Decoy Rating Difference and Similarity Rating

Table 1 Odds Ratios and 95% CIs From Two Mixed-Effects Logistic Models Fit by Maximum Likelihood With Subject-Specific Intercepts

Figure 6 Distribution of the Target–Competitor and Target–Decoy Similarity Ratings

Figure 2 Summary of the Quadruplet Construction Process

Full text

A Zero Attraction Effect in Naturalistic Choice
Anna Trendl, Neil Stewart, and Timothy L. Mullett
Behavioural Science Group, Warwick Business School, University of Warwick
In the attraction effect, adding a dominated third option to a choice set of two options
can alter preferences for the original two options and increase the dominating option’s
choice share. This can constitute a violation of the axioms of regularity and indepen-
dence from irrelevant alternatives, which are core properties of any choice model in
which the utility of each option is stable across choice sets. In the past 20 years, the
attraction effect has driven the development of a set of influential models of multiat-
tribute choice. However, two studies (Frederick et al., 2014; Yang & Lynn, 2014)
involving a series of experiments with naturalistic choice options (e.g., snacks, movies)
found no evidence for the attraction effect in choice contexts where alternatives are
represented without objectively defined (e.g., numerical) attribute dimensions—a find-
ing that would severely undermine its theoretical importance. Huber et al. (2014)
criticized these studies, laying down a set of criteria that should be met by any
experiment wishing to test for the attraction effect in real-world consumer choices.
Based on these criteria, this article presents a carefully designed experiment testing the
attraction effect with movies as naturalistic choice options. The results show a precisely
zero attraction effect.
Keywords: attraction effect, context effects, consumer choice
The Attraction Effect
Imagine that you are having a nice meal in
a restaurant, and you are looking at the two
dessert options available on the menu:
cheesecake and pecan pie. You are torn be-
tween the creamy texture of the cheesecake
and the rich, nutty flavor of the pecan pie.
You resolve to have the cheesecake. As the
waiter approaches to take your order, he in-
forms you that a third dessert option, apple
pie—which you find quite bland—is also
available today. But now you have changed
your mind and decide to order the pecan pie.
Tsetsos et al. (2010) give just this hypotheti-
cal example: The availability of the third op-
tion, which you do not want, makes you
switch between the original two options.
This is an example of the attraction effect
(also known as asymmetric dominance effect).
The introduction of the apple pie (decoy) to the
choice set makes you more likely to choose the
pecan pie (target) over the cheesecake (compet-
itor). In essence, it states that when the decision
maker is indifferent between the target and the
competitor (pecan pie and cheesecake in the
example), the addition of an inferior decoy op-
tion that resembles the target (apple pie is sim-
ilar to pecan pie but is less liked by the decision
maker) increases the likelihood that the target
will be chosen.
Anna Trendl X https://orcid.org/0000-0003-1994-4207
Neil Stewart X
https://orcid.org/0000-0002-2202-018X
Timothy L. Mullett X https://orcid.org/0000-0002-4082-2813
This research was funded in part by Economic and Social
Research Council Grants ES/K002201/1, ES/P008976/1, and
ES/N018192/1 and Leverhulme Trust Grant RP2012-V-022.
This article has been published under the terms of the Creative
Commons Attribution License (http://creativecommons.org/
licenses/by/3.0/), which permits unrestricted use, distribu-
tion, and reproduction in any medium, provided the original
author and source are credited. Copyright for this article is
retained by the author(s). Author(s) grant(s) the American
Psychological Association the exclusive right to publish the
article and identify itself as the original publisher.
Correspondence concerning this article should be addressed
to Anna Trendl, Behavioural Science Group, Warwick Busi-
ness School, University of Warwick, Coventry CV4 7AL,
United Kingdom. Email: anna.trendl@wbs.ac.uk
Decision
© 2021 The Author(s) 2021, Vol. 8, No. 1, 55– 68
ISSN: 2325-9965 https://doi.org/10.1037/dec0000145
55

The attraction effect is important, because it
poses a challenge to all choice models that rely
on the assumption of simple scalability, which
holds when options in a choice set can be given
a scale value, and choice probability is repre-
sented as a monotone function of these scale
values (Trueblood et al., 2013). The attraction
effect also violates the property of indepen-
dence from irrelevant alternatives, which re-
quires that the relative choice probability of two
options should not be affected by adding new
options to the choice set (Pleskac, 2015). In
addition, the attraction effect can violate the
regularity condition, which states that an op-
tion’s choice probability cannot increase when
the choice set is extended (Luce, 1977; Tversky,
1972).
Due to its theoretical importance, the attrac-
tion effect has played a substantial role in the
evolution of multialternative, multiattribute
models of choice, with a significant number of
various theoretical accounts being developed
over the past 20 years (e.g., multialternative
decision field theory, Roe et al., 2001; leaky
competing accumulators, Usher & McClelland,
2004; multialternative attentional drift– diffu-
sion model, Krajbich et al., 2010; range-
normalization model, Soltani et al., 2012; asso-
ciative accumulation model, Bhatia, 2013;
multiattribute linear ballistic accumulator, True-
blood et al., 2014; multialternative decision by
sampling, Noguchi & Stewart, 2018).
The first multiattribute choice experiments
demonstrating the attraction effect (e.g., Huber
et al., 1982; Simonson & Tversky, 1992) almost
exclusively used stylized stimuli with objec-
tively defined attribute dimensions (e.g., cars
presented as numerical values for gas mileage
and ride quality). Trueblood et al. (2013) have
also found evidence for the attraction effect in a
perceptual choice experiment, where partici-
pants were asked to select the largest from three
rectangles with varying widths and heights.
However, recent research suggests that the
attraction effect might only occur under very
specific conditions. In particular, it had been
shown that the effect is much more likely to
manifest when an attribute-wise comparison
strategy is employed in the choice process, as
opposed to an alternative-wise strategy (Nogu-
chi & Stewart, 2014). In addition, the attraction
effect seems to be highly dependent on the exact
presentation format of the choice options (e.g.,
Cataldo & Cohen, 2019; Frederick et al., 2014;
Spektor et al., 2018). Since stimulus presenta-
tion format fundamentally affects the underly-
ing comparison strategy, a natural concern is
then whether this hugely influential decision
bias generalizes to real-world choice situations
with nonstylized alternatives (i.e., options that
cannot be represented with objectively defined
attribute dimensions).
Choice options that are represented along two
attribute dimensions (perceptual or numerical)
can be relatively easily manipulated within a
choice experiment, but most real-world choices
involve complex, naturalistic objects with a
large number of underlying attributes that often
cannot be represented in a stylized format.
These naturalistic options are represented in a
pictorial format without objectively defined at-
tributes.
Recent experiments in multiattribute choice
introduced what can perhaps be considered as
more ecologically valid stimuli (e.g., movies
presented as thumbnails and titles on Netflix or
photographs of popular snacks). Results from
these experiments indicate that significant dif-
ferences might exist in the processing of styl-
ized and nonstylized (naturalistic) stimuli. For
example, Bhatia and Stewart (2018) find that,
with naturalistic stimuli, the weighted additive
model on a high-dimensional semantic repre-
sentation generalizes to new choices better than
the simpler heuristic models that perform so
well on stimuli presented in a stylized format.
Recently, the existence of the attraction effect
in choices that involve naturalistic options has
become a contentious issue in the decision-
making literature. Based on 38 experiments,
Frederick et al. (2014) presented an extensive
investigation of the boundary conditions of the
attraction effect. These experiments included
choice options represented with numerical attri-
butes, as well as complex, real-world stimuli
(e.g., fruits, bottled water, apartments), and in
some of these experiments, participants could
even sample the choice options (e.g., Kool-Aid,
facial tissue, jelly beans). The overall conclu-
sion of this study was that while the presence of
the decoy seemed to affect choices when the
options were represented in a numerical format,
it was absent in experiments with more com-
plex, naturalistic stimuli. In light of these re-
sults, Frederick et al. posited that the psycho-
logical processes underlying decisions where
56 TRENDL, STEWART, AND MULLETT

options are represented with numerical attri-
butes are fundamentally different from those
employed in decisions where the stimuli have a
more naturalistic format. This conclusion was
also supported by Yang and Lynn (2014), who
demonstrated difficulties replicating the attrac-
tion effect in experiments where the stimuli had
qualitative-verbal or pictorial depictions, as op-
posed to cases where the attributes were pre-
sented numerically.
These two studies sparked considerable inter-
est among decision-making researchers and led
to the reexamination of the boundary conditions
of the attraction effect. While the results from
these studies are consistent in showing no evi-
dence for the attraction effect across a wide
variety of naturalistic choice options, the degree
to which the individual experiments presented
in these studies invoked an attraction effect–
type choice scenario, thus constituting a strin-
gent test of the attraction effect, has been sub-
sequently questioned.
In particular, Huber et al. (2014) discussed
five critical conditions that can inhibit the at-
traction effect and argued that many of these are
present in the experiments reported by Freder-
ick et al. (2014) and Yang and Lynn (2014). The
five critical conditions are to avoid (a) strong
prior preferences over the target and competitor,
(b) inability to identify the dominance relation-
ship between the target and the decoy, (c) het-
erogeneity in prior preferences over the target
and competitor, (d) an undesirable decoy, and
(e) a decoy that is too desirable. In addition,
Simonson (2014) further stressed the impor-
tance of detecting the dominance relationship in
observing the attraction effect.
Due to the theoretical significance of the at-
traction effect, it is important to know if the
attraction effect is confined to choice settings
with stimuli presented in an attribute-by-
alternative format. Arguably, while most con-
sumer choices in the real world involve stimuli
that are often presented in a rich naturalistic
form (Bhatia & Stewart, 2018), the develop-
ment of formal models of choice has been al-
most exclusively reliant on results from exper-
iments where the options are represented with
numerical attributes. Exploring how the
strength of the effect depends on the presenta-
tion format of the choice alternatives also pro-
vides us with invaluable information about the
cognitive process underlying the attraction ef-
fect.
In this article, we describe a rigorous test of
the attraction effect with complex, naturalistic
choice options, using a carefully developed ex-
perimental methodology that addresses all of
the critical conditions discussed by Huber et al.
(2014). Our results show a precisely zero attrac-
tion effect. Taken together with earlier results
from Frederick et al. (2014) and Yang and Lynn
(2014), we see this as strong evidence for the
claim that the attraction effect is limited to
choice tasks where options are represented in a
stylized format with objectively defined attri-
butes.
Testing the Attraction Effect With
Real-World Stimuli
The goal of this experiment was to test the
attraction effect with naturalistic stimuli. We
chose to use the most popular movies on IMDb
as stimuli. Choosing between movies is a real-
istic, everyday task. In addition, the movie
space is rich, and thus it allows us to create a
wide range of unique choice triplets to test the
attraction effect.
When the stimuli are represented with well-
defined attribute dimensions, it is straightfor-
ward to construct choice triplets with a target,
competitor, and decoy option. However, with
naturalistic stimuli, this task is significantly
more complicated. First, such stimuli can have a
high number of attribute dimensions that might
vary across individuals. In addition, it is entirely
possible that preferences are not monotonic
over these attribute dimensions (while this
could also be the case for alternative-by-
attribute representations, these dimensions are
usually constructed to ensure monotonic prefer-
ences, e.g., probability of winning, amount to
win).
Frederick et al. (2014) have also used movie
stimuli in two of their experiments: They chose
pairs of movies that are part of the same series
or are starring the same actor to create target–
decoy pairs. In these experiments, the role of
each of the three movies (target, competitor,
decoy) was always the same for all participants
and based upon average population-level rat-
ings rather than individual ratings.
Our novel experimental design takes individ-
ual preferences into account when creating tar-
57ZERO ATTRACTION EFFECT IN NATURALISTIC CHOICE

get– competitor– decoy triplets and ensures that
decision makers are indifferent between the tar-
get and competitor and are able to clearly iden-
tify the lowest-rated option in the choice set. To
increase the statistical power of our experiment,
we used a within-subjects design. We presented
participants with both A, B, A= and B, A, B=
triplet pairs (where X= is the dominated option).
Two triplet pairs were created from “quadru-
plets” of movies, using two distinctly different
target– decoy pairs. Figure 1 shows an example
of two choice triplets created from one such
quadruplet of movies (in the experiment partic-
ipants were presented with real movie posters).
Method
Preregistration
The study design, exclusion criteria, and all
the analyses were planned and registered before
we collected any choice data. The preregistration
can be accessed here https://osf.io/fme6c/?view_
only⫽31da4193689f4247a76af93b2f98fcef.
Candidate Choice Set Selection
To determine the choice sets used in the
experiment, the first step was to create a set of
quadruplets, each consisting of two movies that
are very similar (for the two target– decoy pairs;
i.e., A–A,= B–B=), while making sure that the
two pairs are sufficiently different from each
other (for the target–competitor pair; i.e., A–B).
The details of the construction of these qua-
druplets are somewhat arbitrary—a different
recipe could have been used. However, the main
point is that the choice triplets created from
these quadruplets pass Huber et al.’s (2014)
criteria, as we detail below. The main steps in
the quadruplet construction process we describe
in detail in the remainder of this section are
depicted in Figure 2.
To obtain the set of 400 movies, we first
retrieved 40 of the most popular movies from
each of 10 distinct genre categories (romance,
drama, sci-fi, thriller, comedy, horror, anima-
tion, fantasy, crime, action) from IMDb. The
popularity of each movie was defined by the
number of ratings it has received. We selected a
wide range of genres to obtain a sufficiently rich
stimuli space, as we wanted our participants to
have a variety of preferences over our stimuli.
We omitted any sequels.
Due to the multidimensional nature of the
stimuli, one of the main difficulties in creating
attraction effect choice triplets from real-world
Figure 1
Two Choice Triplets Used in the Experiment
Decoy
Decoy
Target Competitor
TargetCompetitor
Goodfellas
The
Godfather
The
Godfather
Knocked Up
Friends
with
Benefits
Friends
with
Benefits
Note. In the experiment, participants were presented with actual movie posters.
58 TRENDL, STEWART, AND MULLETT

objects is establishing a criterion for matching
up similar objects. We used genre and subgenre
information from the website allmovie.com to
create target– decoy pairs with many shared
genres that are likely to be perceived as similar,
and target– competitor pairs with no genre over-
lap that are likely to be perceived as different.
We conjectured that it will be harder to find
movie pairs that will be perceived as similar, be-
cause any given movie is similar to only a few
movies and dissimilar to all of the others. For this
reason, we started the quadruplet creation with
selecting potential target– decoy pairs.
The genre information on allmovie.com is
very rich: Compared to the 18 genre categories
on IMDb, there are 156 genre and subgenre
categories, capturing many important aspects of
the movies. Using this rich genre information,
we created a movie-by-movie (400 ⫻ 400) ma-
trix, where each cell was the number of over-
lapping genre categories between the two mov-
ies. We selected a movie pair as target– decoy
candidate if, for a given target candidate movie,
the number of overlapping genres with a candi-
date decoy was equal to the maximum overlap
seen for that target across all candidate decoy
movies. This resulted in 2,271 target– decoy
candidate movie pairs overall. We also added
806 movie pairs obtained from the mutually
closest 10% of movies based on a latent seman-
tic analysis
1
that were not already in our list of
target– decoy candidates. The rationale behind
using semantic proximity as an additional crite-
rion was to capture movie pairs that are very
close to each other in terms of the story themes
but are not the closest on the genre dimension.
Overall, we had 3,011 unique target–decoy can-
didate pairs at this point.
We then reduced the size of this list by se-
lecting the most similar movie pairs. This was
done manually by two researchers, who inde-
pendently judged the similarity of each movie
pair (not similar at all/similar). We then only
kept the movie pairs that were judged as similar
by both researchers, weeding out the movie
pairs that were obviously not similar, resulting
in 1,242 target– decoy candidates. We then di-
vided the 1,242 pairs into six groups of 207
pairs and ran a pilot study where we asked 60
1
The latent semantic analysis assesses the similarity of
two items based on the text associated with them. For this
analysis, we used the summary text about the movies as well
as plot keywords, actor names, and director names, all
retrieved from IMDb.
Figure 2
Summary of the Quadruplet Construction Process
59ZERO ATTRACTION EFFECT IN NATURALISTIC CHOICE