ARTICLE
Associations between aversive learning processes
and transdiagnostic psychiatric symptoms in a
general population sample
Toby Wise
1,2,3
✉
& Raymond J. Dolan
1,2
Symptom expression in psychiatric conditions is often linked to altered threat perception,
however how computational mechanisms that support aversive learning relate to specific
psychiatric symptoms remains undetermined. We answer this question using an online
game-based aversive learning task together with measures of common psychiatric symptoms
in 400 subjects. We show that physiological symptoms of anxiety and a transdiagnostic
compulsivity-related factor are associated with enhanced safety learning, as measured using
a probabilistic computational model, while trait cognitive anxiety symptoms are associated
with enhanced learning from danger. We use data-driven partial least squares regression to
identify two separable components across behavioural and questionnaire data: one linking
enhanced safety learning and lower estimated uncertainty to physiological anxiety, com-
pulsivity, and impulsivity; the other linking enhanced threat learning and heightened uncer-
tainty estimation to symptoms of depression and social anxiety. Our findings implicate
aversive learning processes in the expression of psychiatric symptoms that transcend diag-
nostic boundaries.
https://doi.org/10.1038/s41467-020-17977-w
OPEN
1
Wellcome Centre for Human Neuroimaging, University College London, London, UK.
2
Max Planck UCL Centre for Computational Psychiatry and Ageing
Research, University College London, London, UK.
3
Division of the Humanities and Social Sciences, California Institute of Technology, Pasadena, CA, USA.
✉
email: t.wise@ucl.ac.uk
NATURE COMMUNICATIONS | (2020) 11:4179 | https://doi.org/10.1038/s41467-020-17977-w | www.nature.com/naturecommunications 1
1234567890():,;
M
any core symptoms of mental illness are linked to
learning about unpleasant events in our environment.
In particular, symptoms of mood and anxiety disorders,
such as apprehension, worry, and low mood, can intuitively be
related to altered perception of the likelihood of aversive out-
comes. Indeed, the importance of altered threat perception is a
feature of many diagnoses that extend beyond disorders of mood
to encompass conditions such as psychosis
1
and eating dis-
orders
2
. As a result, research into how individuals learn about
aversive events holds great promise for enhancing our under-
standing across a diverse range of mental health problems.
Computational approaches are a powerful means to char-
acterise the precise mechanisms underpinning learning, as well as
uncovering how these relate to psychiatric symptom
expression
3,4
. Recent studies have leveraged computational
modelling to capture associations between learning processes and
psychiatrically relevant dimensions in non-clinical samples
5–8
,as
well as in clinical conditions ranging from anxiety and depression
to psychosis
9–12
. A common finding across studies is that of
altered learning rates, where psychopathology is linked to inap-
propriate weighting of evidence when updating value
estimates
7,13,14
. Notably, there is evidence suggesting that people
with clinically significant symptoms of anxiety and depression
show biased learning as a function of the valence of information,
updating faster in response to negative than positive outcomes
presented as monetary losses and gains
12
, a bias that might
engender a negative view of the environment. However, we pre-
viously found an opposite pattern in a non-clinical study using
mild electric shocks as aversive stimuli, whereby more anxious
individuals learned faster from safety than from punishment, and
underestimated the likelihood of aversive outcomes
15
. This latter
finding highlights a need for a more extensive investigation using
larger samples.
In addition to aberrant learning, another process implicated in
the genesis of psychiatric disorder relates to the estimation of
uncertainty
16
. While there are multiple types of uncertainty, here
we use the term to refer to estimation uncertainty, describing the
precision of a learned association. Estimation uncertainty is
highest when there is a lack of experience, or the association to be
learned is unstable. For example, having seen two coin flips and
observing one head and one tail, one might believe the likelihood
of observing a head is 50%, though they are highly uncertain
about this estimate due to a lack of evidence. This kind of
uncertainty plays a fundamental role in learning, and computa-
tional formulations optimise learning in the face of non-
stationary probabilistic outcomes based on uncertainty
11,17–20
.
While psychiatric symptoms, including anxiety, have been linked
to an inability to adapt learning in response to environmental
statistics such as volatility
5,9
, little research has investigated how
individuals estimate, or respond to, uncertainty in aversive
environments and its potential association with psychiatric
symptoms. This is a crucial question given that core features of
anxiety revolve around the concept of uncertainty. For example,
individuals with anxiety disorders report feeling more uncertain
about threat and being less comfortable in situations involving
uncertainty
21–24
. In an earlier lab-based study we observed a
surprising relationship, finding that more anxious individuals
were more certain about stimulus–outcome relationships
15
.
However, this was in a relatively small sample and therefore
warrants further investigation.
Existing work on aversive learning has had a particular focus
on symptoms of anxiety and depression
7,12
. However, these
approaches have not been designed optimally for identifying
mechanisms that span traditional diagnostic boundaries. This
assumes importance in light of recent studies, using large sam-
ples, showing several aspects of learning and decision-making
relate more strongly to transdiagnostic factors (symptom
dimensions that are not unique to any one disorder) than to any
specific categorical conception of psychiatric disorder
6,8,25–27
.
Applying such an approach to aversive learning may yield better
insights into the role of learning in psychiatric disorders. In
addition, computationally defined measures of learning and
decision-making can facilitate identi fication of novel transdiag-
nostic factors, going beyond those identified based solely on
correlated symptom clusters in self-report and clinical interview
measures
6,28–30
.
Here, we aim to clarify the nature of the relationship between
aversive learning processes and traditional measures of anxiety, as
well as transdiagnostic psychiatric factors identified in prior
work
6
in a large, preregistered study conducted online. This
allows us to measure effects with high precision, potentially
helping to resolve mixed findings from previous studies
12,15
,in
addition to identifying small but meaningful effects that cross
traditional diagnostic boundaries
6
. As in similar prior
studies
6,8,25,26,31
, we do not purposefully recruit subjects diag-
nosed with mental health problems, instead focusing on exploring
relationships with the variation in symptoms present in the
general population. While this does not allow concrete conclu-
sions about clinical disorders per se, we note findings using
similar approaches have replicated those seen in clinical
samples
8,32
, providing reassurance these methods can generate
insights into psychiatrically relevant phenomena. Thus, we use a
computational approach to test whether anxiety and transdiag-
nostic symptoms are associated with biased learning from safety
and threat, whether these factors relate to altered estimates of
threat likelihood, and whether they are associated with different
levels of uncertainty during threat learning. We then use partial
least squares (PLS) regression, a data-driven multivariate method,
to derive transdiagnostic latent components of psychopathology
grounded in both self-report and computational measures. In
contrast to our primary analyses, this analysis is exploratory and
data-driven, enabling us to generate hypotheses for future
research. Given difficulties in using traditional aversive stimuli in
an online setting, we develop a game-based avoidance task
designed to engage threat and avoidance processes without the
need for administration of painful or noxious stimuli. Both the
task and modelling are, in principle, similar to our previous lab-
based task
15
, but their implementation here allows straightfor-
ward administration in large samples recruited online. Our results
demonstrate that learning from safety and danger are associated
with distinct symptom dimensions that cut across diagnostic
boundaries, implicating aversive learning processes in a range of
psychiatric symptoms.
Results
Task performance. Four hundred subjects recruited online
through Prolific
33
performed a game-based aversive learning task,
where the aim was to fly a spaceship through asteroid belts
without being hit (Fig. 1). Getting hit by the asteroids reduced the
integrity of the spaceship, and after sufficient hits the game ter-
minated. Crucially, there were two zones at the top and bottom of
the screen where subjects could encounter a hole in the asteroid
belt, each associated with a changing probability of being safe. In
order to perform well at the task subjects needed to learn which
zone was safest and behave accordingly.
Subjects were engaged and performed well at the task, with a
median number of spaceship destructions of 1 (Interquartile
range = 2) over the course of the task. They also reported high
motivation to perform the task, providing a mean rating of 85.70
(SD = 18.44) when asked to rate how motivated they were to
avoid asteroids on a scale from 0–100. Reassuringly, no subjects
ARTICLE NATURE COMMUNICATIONS | https://doi.org/10.1038/s41467-020-17977-w
2 NATURE COMMUNICATIONS | (2020) 11:4179 | https://doi.org/10.1038/s41467-020-17977-w | www.nature.com/naturecommunications
met our exclusion criteria designed to remove those not engaging
in the task.
Computational modelling of behaviour. To quantitatively
describe behaviour, we fit a series of computational models to
subjects’ position data during the task (see “Methods” and Sup-
plementary Methods for a full description of tested models). The
winning model was a probabilistic model incorporating different
updates parameters for safety and danger, as well as a stickiness
parameter representing a tendency for subjects to stick with their
previous position. This model represents an extension of one we
have previously used successfully as a lab-based aversive learning
task
15
, and is described fully in the “Methods” section. Briefly, this
winning model assumes that subjects in the task represent the
safety probability of each zone using a beta distribution, which is
updated on each trial based on encounters with danger or safety.
Simulating responses using the model, using each subject’sesti-
mated parameter values, produced behavioural profiles that
demonstrated a high concordance with the true data, reproducing
broad behavioural patterns seen in the true data (Supplementary
Fig. 1). We note that we do not wish to make strong claims
regarding the strategy employed by subjects in performing this
task; our central focus was on exploring relationships with symp-
toms, and we focused on probabilistic models as they provide a
natural measure of uncertainty that can be associated with symp-
toms. It is possible that more complex reinforcement learning
models for example could produce equally good fits to the data.
Task and model validation. It was important to first ensure that
the task has content validity, and that it produces behaviour
reminiscent of more traditional tasks. Likewise, the computa-
tional models used should provide measures and parameter
estimates that reflect the behaviour they aim to describe. We
therefore conducted extensive validation exercises. These are
reported fully in Supplementary Methods, but we summarise
them here.
First, we ensured the task did induce states of subjective anxiety
in the majority of subjects (Fig. 2a), and this level of anxiety was
correlated with self-report state and trait anxiety (Fig. 2c, d).
Importantly, subjects adjusted their position to a greater extent
following danger than following safety (Fig. 2e), indicating that
they were adapting their behaviour in response to outcomes in
the task, rather than behaving randomly. With respect to our
computational model, we verified that the model’s update
parameters were robustly correlated with subjects’ tendency to
move, or stay, following danger and safety, respectively (Fig. 3d).
We also assessed whether safety and uncertainty values, produced
by simulating data from our model with best fitting parameters,
related to subjects’ model-free behaviour. We found that subjects
changed their position more when model-derived uncertainty was
high, and when the difference between the safety value of the two
zones was small. This pattern (Fig. 3c) indicates that value and
uncertainty measures do reflect meaningful quantities for
behaviour. Finally, we verified that our model’s update para-
meters showed greater updating from danger relative to safety, as
we found in a previous lab-based study
15
, finding this was indeed
the case (Fig. 3e).
Parameter recovery analyses are also reported in Supplemen-
tary Fig. 2 and indicated good recoverability (lowest r between
true and recovered parameter value = 0.61). It should be noted
that we did find a moderate negative correlation between the two
update parameters (r = −0.49), which may be an artefact of the
fact that safety and danger outcomes in the task were partially
anticorrelated (i.e., when one zone was likely to be safe, the other
was likely to be dangerous). However, despite this, parameter
recovery tests indicated good recoverability of these parameters.
Relationships with anxiety. First, we asked whether our four
behavioural variables of interest (threat update parameter, danger
update parameter, mean estimated safety probability, and mean
estimated uncertainty) were associated with anxiety (both state
and trait) and intolerance of uncertainty. The strongest rela-
tionships, with highest posterior density (HPD) intervals that did
not include zero, were positive effects of state anxiety on safety
update rates and mean estimated safety probability (Fig. 4,
Table 1), although effects for trait anxiety were in the same
direction and of a similar magnitude for some measures, indi-
cating more anxious individuals learned faster about safety and
perceived safety as more likely overall.
However, it is possible that our state and trait anxiety sum scores
may obscure more nuanced effects relating to different symptom
dimensions. To test this, we performed exploratory analyses on the
two subscales of our trait anxiety measure, which represent
cognitive symptoms, such as worry, and physical symptoms, which
reflect aspects of physiological arousal. This analysis revealed a
dissociation between cognitive and somatic trait anxiety, whereby
cognitive symptoms were associated with heightened learning from
danger, heightened uncertainty, reduced learning from safety, and
lower safety probability, while somatic anxiety showed an opposite
pattern (Table 1 and Fig. 5). The same was true for state anxiety to
a lesser extent; while effects for somatic anxiety remained strong,
those for cognitive anxiety were weaker and had 95% HDPIs that
ab
Asteroid belt
Safety
zone B
Safety
zone A
Shields:
Score:
9260
Spaceship
Fig. 1 Task design overview. a Subjects were tasked with playing a game that had a cover story involving flying a spaceship through asteroid belts. Each
asteroid belt featured two locations that could potentially contain escape holes (safety zones), and subjects were instructed to aim to fly their spaceship
through these to gain the highest number of points. Subjects were only able to move the spaceship in the Y-dimension, while asteroid belts moved towards
the spaceship. The probability of each zone being safe varied over the course of the task but this could be learned, and learning this probability facilitated
performance. b Screenshot of the task, showing the spaceship, an asteroid belt with a hole in the lower safety zone (safety zone B), a representation of the
spaceship’s integrity (shown by the coloured bar in the top left corner) and the current score.
NATURE COMMUNICATIONS | https://doi.org/10.1038/s41467-020-17977-w ARTICLE
NATURE COMMUNICATIONS | (2020) 11:4179 | https://doi.org/10.1038/s41467-020-17977-w | www.nature.com/naturecommunications 3
included zero (Table 1 and Fig. 5). Together, these results indicate
that different components of trait anxiety have distinct relation-
ships with aversive learning processes.
Associations between aversive learning and transdiagnostic
factors. Following this, we examined the extent to which task
behaviour was associated with three transdiagnostic factors of
psychopathology identified through self-report assessments in
previous research
6
. Here, we observed effects of a factor labelled
compulsivity and intrusive thought (Fig. 4, Table 2), reflecting the
fact that subjects scoring higher on this factor learned faster about
safety and had higher safety probability estimates. There was also
a weak effect of this factor on uncertainty, although the HPDI for
this included zero. Other effects were weak, and including
reported task motivation as a covariate had a negligible effect on
the results (see Supplementary Figs. 5 and 6). Importantly, all of
these analyses were determined a priori and are included in our
preregistration. We also examined effects of age and sex, and their
interactions with our primary variables. These results are
described in Supplementary Fig. 4.
Psychiatric constructs derived from behaviour and self-report.
Numerous studies have used dimensionality reduction procedures
such as factor analysis on questionnaire-based data to identify
factors of psychopathology that cut across diagnostic
boundaries
6,28–30
. This, in turn, has revealed that many beha-
viourally defined phenotypes are more strongly associated with
transdiagnostic factors than any single disorder
6,8,26
. We built
upon this work by incorporating computationally derived indexes
of behaviour into this dimensionality reduction procedure, where
the aim was to identify latent constructs grounded in both self-
report and behaviour. We used PLS regression, a method that
identifies latent components linking multivariate data from mul-
tiple domains based on their shared covariance. This general
a
60
How anxious did the game
make you feel?
How much did you want to
avoid the asteroids?
40
Count
20
0
0
100
80
60
Task-induced anxiety
rating
40
20
Density
0
20
0.0 0.2 0.4
Magnitude
0.6 0.8
Danger
200
150
100
Number of subjects
50
0
Safety
1.0
0510
Number of crashes
15 20
40
STICSA state anxiety score STICSA trait anxiety score
60 20 40 60
25 50
Rating
State anxiety
R = 0.25, p = 4e–07
Trait anxiety
R = 0.21, p = 2.09e–05
Rating
75 100 0 25 50 75 100
150
100
50
0
b
cd
ef
Fig. 2 Subjective and behavioural responses during the task. a Distribution of task-induced anxiety ratings recorded after the task. b Distribution of task
motivation ratings. c, d Relationships between task-induced anxiety ratings and state and trait anxiety scores. Correlations represent Pearson r statistics,
and p values are two sided without correction for multiple comparisons. e Degree of location switching after encountering danger and safety across
subjects. The switch magnitude is the average absolute change in position between trial n and trial n + 1. As expected, subjects showed more switching
behaviour after encountering danger and were more likely to stay in the same position following a safe outcome. f Distribution of crash number
(representing the number of subjects hitting enough asteroids to end the game) across subjects.
ARTICLE NATURE COMMUNICATIONS | https://doi.org/10.1038/s41467-020-17977-w
4 NATURE COMMUNICATIONS | (2020) 11:4179 | https://doi.org/10.1038/s41467-020-17977-w | www.nature.com/naturecommunications
method has been employed successfully in prior studies to provide
insight into how panels of cognitive and behavioural measures
relate to multivariate neuroimaging-derived phenotypes
34–36
.
PLS-like analyses can be problematic if not properly validated (for
example producing spurious results due to overfitting), and so we
adopted best-practice methods for validating these results
37–39
,
selecting the optimal number of components using cross-
validation and training the model on 75% of the data, before
testing its performance on the remaining 25% of the data.
Importantly, in contrast to other analyses, this was an exploratory
analysis where our aim was to provide indicative results that can
be validated in future research.
We first identified the number of components that best
describe our data by evaluating the performance of a predictive
a
b
de
c
Responses
Uncertainty
Safety value
Tr ia l
1.0
0.5
Subject
position
0.0
1.0
0.5
Value
0.0
0
–10,000
–20,000
WAIC
–30,000
ALB
Learning from danger Learning from safety
1.0
0.7
0.6
0.5
0.4
Parameter
0.3
0.2
0.1
0.0
0.8
0.6
–
+
n
(danger)
p
(safety)
0.4
0.2
0.0
0.0 0.2 0.4
Response shift
0.6 0.0 0.2 0.4
Response shift
0.6
0.0 0.2 0.4
Estimated value
0.6 0.8 1.0
Danger (R = 0.25))
Safety (R = –0.24))
Danger (R = –0.31))
Safety (R = 0.12))
ALB
softmax
ALB
sticky
ALB
UCB
Model
ALB
variance
Dual
learning
rate RW
Rescorla
wagner
0.1
Uncertainty
0.0
0 50 100 150 200 250
Tr ia l
0 50 100 150 200 250
Tr ia l
0 50 100 150
Value
difference
–0.05
Uncertainty
0.00 0.05
Regression coefficient
(± 95% HPD)
0.10
0.15
Predictor
200 250
NATURE COMMUNICATIONS | https://doi.org/10.1038/s41467-020-17977-w ARTICLE
NATURE COMMUNICATIONS | (2020) 11:4179 | https://doi.org/10.1038/s41467-020-17977-w | www.nature.com/naturecommunications 5
