scispace - formally typeset
Search or ask a question
Journal ArticleDOI

Online evaluation of novel choices by simultaneous representation of multiple memories

TL;DR: Using functional magnetic resonance imaging repetition suppression in humans, it is suggested that, in the absence of direct experience, coactivation of multiple relevant memories can provide a training signal to the valuation system that allows the consequences of new experiences to be imagined and acted on.
Abstract: This study used fMRI repetition suppression to demonstrate that human subjects can represent and evaluate novel choice options by invoking multiple memories for previous experiences in hippocampus and medial prefrontal cortex.

Summary (3 min read)

Deciding between novel goods

  • Each good was a novel combination of two different familiar foods (Fig. 1a ).
  • Participants were given the opportunity to observe these novel goods without being allowed to sample them by either taste or smell.
  • This involvement accords with recent evidence that vmPFC encodes value preference for executable choices and dmPFC does so for choices that are modeled abstractly 7 .
  • Thus, their only recourse was to construct, online, an expectation of the compound's value from knowledge of the individual components.
  • A key question is whether subjects constructed a novel representation of the compound by explicitly combining the representations of each component and, if so, which brain regions support this construction process.

Constructing representations of novel goods using memories

  • Participants trained extensively on these associations between food items and abstract shapes.
  • The key comparison of interest here was the brain activity elicited by novel goods when preceded by related components (for example, A or B followed by AB) compared with novel goods when preceded by unrelated components (for example, C or D followed by AB).
  • The authors result implies that these brain regions construct a value representation of a novel item from component memories, and do so by simultaneously engaging neuronal representations of these components.

Plasticity between simultaneously active memories

  • If this is the case then it follows that during the construction of the compound good AB, the neuronal ensembles representing components A and B should be simultaneously active.
  • This can also be tested using fMRI adaptation, which predicts a differential effect for components that were part of the same compound compared with components that were not.
  • This suggests that the mechanism underlying this suppression occurred during the earlier construction of the novel good and not during the participant's elicitation of the component item at the time that this signal was measured.
  • It is important to note that these three de facto tests of mPFC function (valuation, construction and plasticity) do not rely on the same data.
  • MPFC can therefore evaluate novel goods by constructing explicit representations of expected outcomes from familiar components, a process that engenders plasticity between simultaneously active component representations.

The influence of sensory experience upon construction

  • The authors then asked whether consummatory exposure to the novel goods would reduce a need to construct value online.
  • Any difference between the two groups in the representation or evaluation of novel goods could therefore be attributed to the effect of sensory exposure.
  • In both groups, the neural activity observed in mPFC was consistent with a role for this brain region in the evaluation of compound goods (Fig. 3b,c ).
  • To determine whether this single experience was enough to reduce a need for online value construction, the authors compared adaptation effects across the two groups.
  • To avoid selection bias, the authors used ROIs derived from whole-brain adaptation effects averaged across both adaptation contrasts in the two groups (Online Methods and Fig. 4a,b ).

Temporal dynamics of construction mechanism

  • If experiential and constructed valuation use distinct neural mechanisms, it is possible that the value construction mechanism could itself substitute for a direct experience and train experiential valuation mechanisms.
  • As the experiment progressed, subjects gained substantial experience in constructing the representation of the novel good.
  • As their experiment extended over three separate blocks, the authors were able to study changes in value construction-related adaptation effects over time (Figs. 5 and 6 ).
  • This was not true for components that had been used to construct high-value novel goods.
  • When averaging across the final two blocks, both the mPFC and hippocampus showed a significant positive correlation with the value of the compound items (mPFC, r = 0.64, P = 0.002; hippocampus, r = 0.63, P = 0.003, Fig. 6b,d and Supplementary Fig. 5 ), after accounting for variance explained by the value of the component items in both cases.

DISCUSSION

  • The neural mechanisms by which these processes are achieved have remained unclear, particularly in circumstances in which anticipated outcomes have not previously been experienced.
  • Rather, the most plausible explanation for this change is that, through repeated representation of a novel compound, previously unrelated memories were recruited simultaneously, inducing a form of plasticity between the underlying representations of necessary components.
  • If subjects are asked to ignore all of their own experiences and preferences and to instead guess what a very different individual would choose, mPFC value signals can immediately reflect the preferences of this new individual 7, 38 .
  • These findings show that a potential new experience can be prospectively represented and evaluated by invoking multiple memories simultaneously in hippocampus and mPFC.

METHODS

  • Methods and any associated references are available in the online version of the paper.
  • Any Supplementary Information and Source Data files are available in the online version of the paper, also known as Note.

ONLINE METHODS

  • 39 healthy volunteers participated in the fMRI experiment and were assigned to one of two groups (unfamiliar and familiar) by drawing from Matlab's pseudo-random number generator.
  • The experimenter chose two novel goods, AB and CD, for each participant, under the constraint that the participant liked all four individual component foods (A and B, C and D) from which the two novel goods were formed.
  • The trials were sorted into seven principal categories with 32 trials of each category per scan session, presented in a randomized order.
  • Preprocessing and statistical analyses were carried out using SPM8 (Wellcome Trust Centre for Neuroimaging, http://www.fil.ion.ucl.ac.uk/spm).
  • To detect brain regions involved in constructing the novel goods (component to compound), the authors used the contrast [(AB preceded by C) − (AB preceded by A)], averaging across all possible permutations (that is, explanatory variables (2) − ( 1)).

Did you find this useful? Give us your feedback

Figures (4)

Content maybe subject to copyright    Report

1 4 9 2 VOLUME 16 | NUMBER 10 | OCTOBER 2013 nature neurOSCIenCe
a r t I C l e S
Humans display remarkable flexibility in their behavior. Like other
animals, we guide our behavior through direct experience, but we
can also infer the likely consequences of actions that have never been
taken
1,2
. Through generalizing principles and applying them to new
situations
3,4
, we can predict new relationships and statistical struc-
tures in our environment and use these to estimate the value of new
events
1,5,6
. Although some progress has been made in uncovering
the brain regions that underlie these complex abilities
1,3–7
, little or
no progress has been made in understanding how neuronal networks
support these complex computations, partly because it is unclear to
what extent such computations exist in species in which we can readily
measure single-cell activity.
One potential mechanism that allows for upcoming events to be
evaluated involves using past experience to predict consequences of
future possible scenarios. In rodents, hippocampal firing sequences at
choice points predict or ‘preplay’ the forthcoming environment
8
, and
the likely outcomes of their decision can later be decoded in the orbito-
frontal cortex
9
. In contrast, when choosing between novel options,
there is no direct experience from which to preplay and evaluate
future options. However, it is possible that the representation of an
upcoming novel outcome may be constructed by combining multiple
distinct relevant experiences, preplayed simultaneously.
To test these predictions, we required access to the information
content of neural populations underlying the representation of a novel
experience. Despite the poor spatial resolution of functional magnetic
resonance imaging (fMRI), there are well-validated strategies that can
reveal underlying cellular representations. For example, fMRI adapta-
tion takes advantage of the fact that activated cellular ensembles in
a voxel show a relative suppression in their activity in response to
repetition of a stimulus to which they recently responded. Despite
ambiguity in the biophysical mechanism underlying repetition sup-
pression
10,11
, when combined with careful experimental design the
technique allows for inferences to be made about the underlying neu-
ronal representations
12,13
.
We used fMRI adaptation to probe the neural representation of a
novel food reward. We hypothesized that, if the representation of a
novel food was constructed by explicit combination of multiple dis-
tinct experiences, we would observe fMRI adaptation when subjects
evaluated a novel reward immediately after evaluating a component
ingredient. Furthermore, if multiple experiences were replayed simul-
taneously, plasticity might result between the underlying neuronal
assemblies. Hence, experiences used to construct the same novel good
would later adapt to each other. Lastly, we hypothesized that this com-
plex construction process would not be required after an independent
neuronal representation of the novel good had been established. We
should therefore observe a reduction in each adaptation effect after
allowing the subjects to either experience the novel good directly
or simulate the novel good repeatedly. This repetition suppression
procedure allowed us to probe the neural mechanisms that underlie
human capacity for flexible, online, value construction.
RESULTS
Deciding between novel goods
We created 13 novel goods whose values were unknown to the sub-
jects (Fig. 1). However, each good was a novel combination of two
different familiar foods (Fig. 1a). Participants were given the oppor-
tunity to observe these novel goods without being allowed to sample
them by either taste or smell.
To first establish that these goods activate known value-related
brain regions, we measured fMRI activity in 19 subjects while they
evaluated and chose from pairs of these novel goods (Fig. 1b). After
the scan session, subjects performed a Becker-DeGroot-Marschak
(BDM) auction
14
that allowed us to measure subjects’ constructed
value for each good. Consistent with reports in simpler valuation
1
The Wellcome Trust Centre for Neuroimaging, Institute of Neurology, University College London, London, UK.
2
The Oxford Centre for Functional Magnetic Resonance
Imaging of the Brain, University of Oxford, Oxford, UK. Correspondence should be addressed to H.C.B. (helen.barron.10@ucl.ac.uk) or T.E.J.B. (behrens@fmrib.ox.ac.uk).
Received 5 June; accepted 9 August; published online 8 September 2013; doi:10.1038/nn.3515
Online evaluation of novel choices by simultaneous
representation of multiple memories
Helen C Barron
1
, Raymond J Dolan
1
& Timothy E J Behrens
1,2
Prior experience is critical for decision-making. It enables explicit representation of potential outcomes and provides training
to valuation mechanisms. However, we can also make choices in the absence of prior experience by merely imagining the
consequences of a new experience. Using functional magnetic resonance imaging repetition suppression in humans, we examined
how neuronal representations of novel rewards can be constructed and evaluated. A likely novel experience was constructed by
invoking multiple independent memories in hippocampus and medial prefrontal cortex. This construction persisted for only a
short time period, during which new associations were observed between the memories for component items. Together, these
findings suggest that, in the absence of direct experience, coactivation of multiple relevant memories can provide a training
signal to the valuation system that allows the consequences of new experiences to be imagined and acted on.
npg
© 2013 Nature America, Inc. All rights reserved.

nature neurOSCIenCe VOLUME 16 | NUMBER 10 | OCTOBER 2013 1 4 9 3
a r t I C l e S
contexts, we observed a signal that correlated
with the value of the chosen option in a net-
work of brain regions that included ventral
and dorsal medial prefrontal cortex (vmPFC
and dmPFC, respectively), and posterior
cingulate cortex (mPFC: P = 0.001 familywise
error (FWE) corrected on cluster level, peak
t
17
= 6.30; Fig. 2a). The involvement of both vmPFC and dmPFC is
of particular interest given that the task required subjects to con-
struct and evaluate a model of a future outcome. This involvement
accords with recent evidence that vmPFC encodes value prefer-
ence for executable choices and dmPFC does so for choices that are
modeled abstractly
7
.
To evaluate these novel goods, subjects could not rely on pre-learnt
values. Thus, their only recourse was to construct, online, an expec-
tation of the compound’s value from knowledge of the individual
components. A key question is whether subjects constructed a novel
representation of the compound by explicitly combining the repre-
sentations of each component and, if so, which brain regions support
this construction process. We reasoned that this construction process
could be measured using fMRI adaptation. Activity relating to the
construction of the compound value would be suppressed when pre-
ceded by a related component if, and only if, the subject had engaged
the neuronal ensembles of the components when constructing a
representation of the compound.
Constructing representations of novel goods using memories
For every participant, we selected 2 of the 13 novel compounds, here
referred to as AB and CD, each consisting of two familiar individual
components (A and B, C and D) that subjects had tasted immediately
before the experiment (Fig. 1a). To avoid visual confounds in a later
analysis, we trained subjects to associate each of the six component
and compound foods (A, B, C, D, AB and CD) with two different
abstract shapes (Fig. 1c). Participants trained extensively on these
associations between food items and abstract shapes. In the final
block of trials, the mean accuracy was 97.8%, with a mean reaction
time of 845.2 ms.
On each trial in the scanner, we presented a distinct shape that
served as an instruction cue for subjects to elicit an explicit mental
representation of the associated food (Fig. 1d). The key comparison
of interest here was the brain activity elicited by novel goods when
preceded by related components (for example, A or B followed by AB)
compared with novel goods when preceded by unrelated components
(for example, C or D followed by AB).
Early in the experiment (block 1 of 3), we observed fMRI adapta-
tion between the representation of novel goods and their constituent
components in both mPFC (P < 0.001, FWE corrected on cluster
level, peak t
18
= 4.45; Fig. 2b) and bilateral hippocampus (t
18
= 2.55,
P = 0.010 using region of interest (ROI) analysis; Online Methods and
Fig. 2b). These two brain regions are components of a network that is
commonly activated in studies of value
7,15–18
, episodic memory
4,19,20
and spatial navigation
12
. Our result implies that these brain regions
construct a value representation of a novel item from component
memories, and do so by simultaneously engaging neuronal represen-
tations of these components.
Plasticity between simultaneously active memories
If this is the case then it follows that during the construction of the
compound good AB, the neuronal ensembles representing compo-
nents A and B should be simultaneously active. We reasoned that this
simultaneous activity, which first occurred during the stimulus-item
training phase before scanning, would induce experience-dependent
plasticity between cellular elements in these two ensembles—a plastic-
ity evident in the scanning trials as a shadow of this value construction
process. For example, after constructing a representation of ‘tea-jelly
dessert’, we reasoned that cellular representations of tea would induce
activity in jelly-preferring ensembles and vice versa. This can also be
tested using fMRI adaptation, which predicts a differential effect for
components that were part of the same compound compared with
components that were not. Indeed, when we compared early trials of
A that were preceded by B to those that were preceded by C, we again
found relative suppression in mPFC activity (P
= 0.014, FWE cor-
rected on cluster level, peak t
18
= 4.24; Fig. 2c), but not hippocampus
(t
18
= 0.34, P = 0.367 using ROI analysis, Online Methods).
Notably, across all three blocks, the extent to which individual par-
ticipants showed adaptation between related components in mPFC,
and in the hippocampus, was predicted by the average value of the
novel items (mPFC, r = 0.47 and P = 0.040; hippocampus, r = 0.58
and P = 0.010), but not component items (mPFC, r = −0.05, P = 0.833;
hippocampus, r = −0.09, P = 0.730). This suggests that the mechanism
underlying this suppression occurred during the earlier construction
of the novel good and not during the participant’s elicitation of the
component item at the time that this signal was measured. Indeed, in
both structures, the correlation with the value of the novel good sur-
vived the removal of any signal attributable to the component values
a
Components
Novel good
A
B
C
D
AB
CD
b
c
d
Evaluation
Choice
Feedback
ITI
+
tomjam
tomjam
tomjam
yogfee
yogfee
yogfee
4s...
?
?
4,000 ms
600 ms
800 ms
6,000 ms
Stim
800 ms
800 ms
Stim
ITI
ITI
+
+
2,500 ms
2,500 ms
Figure 1 Experimental design. (a) We made
13 novel goods from the combination of two
familiar food types that had not previously
been tasted together. Two examples are shown
here: avocado and raspberry smoothie (AB) and
tea-jelly (CD). (b) Participants made binary
decisions between the novel goods while in
the scanner. (c) Prior to entering the scanner,
two of the novel goods were chosen for each
participant. Participants learned to associate
each of these novel goods and their respective
components with two abstract stimuli. (d) In
the scanner, participants vividly imagined the
sensory properties of the food items in response
to each abstract stimulus presented.
npg
© 2013 Nature America, Inc. All rights reserved.

1 4 9 4 VOLUME 16 | NUMBER 10 | OCTOBER 2013 nature neurOSCIenCe
a r t I C l e S
(mPFC, r = 0.51, P = 0.015, Fig. 2d; hippocampus, r = 0.60, P = 0.004;
Fig. 2e and Supplementary Fig. 1). Together, these findings support
value-dependent plasticity in related components as a consequence
of coactivation during construction of the novel goods.
It is important to note that these three de facto tests of mPFC func-
tion (valuation, construction and plasticity) do not rely on the same
data. Despite slight differences in thresholded peak locations of the
two adaptation effects, they showed similar patterns of activity in
mPFC (Fig. 2f). mPFC can therefore evaluate novel goods by con-
structing explicit representations of expected outcomes from familiar
components, a process that engenders plasticity between simultane-
ously active component representations.
The influence of sensory experience upon construction
We then asked whether consummatory exposure to the novel goods
would reduce a need to construct value online. To test this idea, we
repeated the experiment in a second group of 20 subjects with one
important difference. This second group (familiar) was given a single
sample of each of the 13 novel compound goods to taste before the
experiment. Notably, both groups underwent the same item-stimulus
learning task before entering the scanner, and there was no significant
difference between groups in reaction time or accuracy on the final block
of trials during the learning task (P > 0.150; Supplementary Table 1).
Any difference between the two groups in the representation or evalu-
ation of novel goods could therefore be attributed to the effect of
sensory exposure.
We first assessed value effects during decision trials. Both groups
showed similar consistency in their choices (Supplementary Fig. 2).
As was the case for the unfamiliar group, the familiar group encoded
chosen value activity in a network of value-related brain regions that
included mPFC (Fig. 3a). In both groups, the neural activity observed
in mPFC was consistent with a role for this brain region in the evalu-
ation of compound goods (Fig. 3b,c).
To determine whether this single experience was enough to reduce
a need for online value construction, we compared adaptation effects
across the two groups. To avoid selection bias, we used ROIs derived
from whole-brain adaptation effects averaged across both adapta-
tion contrasts in the two groups (Online Methods and Fig. 4a,b).
A between-group comparison in these ROIs revealed significant
differences in the adaptation effects between the familiar and unfa-
miliar participants in both mPFC (group × condition interaction,
P = 0.018, F
1,144
= 5.76, three-way ANOVA, Online Methods) and
hippocampus (group × adaptation type × condition interaction,
P = 0.035, F
1,144
= 4.52, three-way ANOVA, Online Methods).
Figure 2 Neural correlates of constructing and
evaluating a novel good. (a) While participants
made binary choices between novel goods,
the mPFC (extending into dmPFC) encoded
chosen value. (b) The mPFC and hippocampus
showed repetition suppression to a novel good
when preceded by a related component (for
example, tea-jelly preceded by tea) compared
with when preceded by an unrelated component
(for example, tea-jelly preceded by avocado).
(c) The mPFC showed repetition suppression to
a component food item when preceded by the
related component (for example, tea preceded
by jelly) compared with when preceded by an
unrelated component (for example, tea preceded
by avocado). (d,e) In mPFC and hippocampus,
a significant positive correlation was revealed
between the amount of suppression between
related components (across all blocks) and the
average value participants assigned to the novel
goods (after removing effects attributable to the
value of the components; for mPFC: r = 0.51,
P = 0.015; hippocampus, r = 0.60, P = 0.004),
respectively. (f) Both adaptation effects showed
comparable effect size across the ventral-to-
dorsal gradient of mPFC (mean ± s.e.m. across
participants). The locations of the ROIs are shown and the effect size for both adaptation measures was scaled such that the peak value was equal to 1.
There was no significant difference between the two adaptation effects at any point on this gradient (P > 0.300 for all ROIs).
a b
c d e
f
x = –4
x = 24
y = –10
x = 4
x = –6
z = 12
(AB) < (CB)
(AAB) < (CAB)
“AB or CD?”
z = –8
z = 10
2
2
Adaptation effect size
Adaptation effect size
Adaptation effect size
1
1
0
0
Value of novel goods
Ventral to dorsal
–1
1
0
–1
1.5
0.5
–0.5
0
1.0
–1 210
Value of novel goods
–1
Figure 3 Sensory exposure to a novel good:
comparison between the unfamiliar and
familiar groups during the decision-making
task. (a) In the familiar group, the mPFC
correlated with chosen value during the
decision-making task (thresholded at
P < 0.01, uncorrected for visualization).
(b) ROI used to assess value signals in both
groups of participants during the decision
task. (c) During the decision-making task, the
unfamiliar and familiar groups showed comparable chosen value signals in mPFC (left, average of all task blocks for each group), and in the unfamiliar
group there was no change in the chosen value signal across time (right, block 1 versus blocks 2 and 3). Parameter estimates were extracted from ROI
shown in b (mean ± s.e.m. across participants). a.u., arbitrary units.
0.075
x = 2
x = –8
z = –12
a b c
0.050
0.025
0
Effect size (a.u.)
Unfamiliar
Familiar
B1
B2 and B3
npg
© 2013 Nature America, Inc. All rights reserved.

nature neurOSCIenCe VOLUME 16 | NUMBER 10 | OCTOBER 2013 1 4 9 5
a r t I C l e S
Using post hoc two-sample t tests to decompose these interactions,
we found that, relative to the unfamiliar group, the familiar group
showed reduced adaptation between the novel goods and their related
components in mPFC (group difference: trend, t
18
= 1.70, P = 0.053;
Fig. 4c) and in the hippocampus (group difference: t
18
= 3.11,
P = 0.003; Fig. 4c). Furthermore, the familiar group did not show
plasticity in mPFC between the representation of the constituent
components of a novel good (group difference: t
18
= 1.96, P = 0.033,
Fig. 4c). Crucially, there was no significant difference between groups
in their ability to accurately elicit the correct representations during
the imagination task (group comparison of accuracy, P = 0.82; reac-
tion time, P = 0.89) or in the average subjective value assigned to any
of the novel goods used in the adaptation task (P > 0.05 for all assigned
novel goods; Supplementary Fig. 3). This result therefore suggests
that even a single previous experience of a good is sufficient to reduce
a requirement for online value construction. This is particularly nota-
ble given that extensive experience is required to reduce goal-oriented
behavior and establish habitual actions
21
.
Temporal dynamics of the construction mechanism
If experiential and constructed valuation use distinct neural mecha-
nisms, it is possible that the value construction mechanism could
itself substitute for a direct experience and train experiential valuation
mechanisms. As the experiment progressed, subjects gained substan-
tial experience in constructing the representation of the novel good.
We asked whether, after multiple previous simulations of an experi-
ence, it was still necessary to construct and evaluate the representation
of novel goods anew on each trial. Alternatively, were values learned
despite participants never having experienced the novel good? As
our experiment extended over three separate blocks, we were able to
study changes in value construction–related adaptation effects over
time (Figs. 5 and 6).
Previous studies have found that goal-directed choice mechanisms
exhibit marked differences early and late in choice experiments
17
. We
used a three-way ANOVA (Online Methods) to identify attenuation
of adaptation effects in mPFC and hippocampus in the unfamiliar
group across the scanning session (block × condition interaction for
mPFC, P = 0.004, F
1,144
= 8.44; block × adaptation-type interaction
for hippocampus, P = 0.011, F
1,144
= 6.56). Post hoc t tests comparing
block 1 with all remaining blocks revealed a significant reduction in
adaptation over time of a novel good to its related component (mPFC,
t
18
= 2.12, P = 0.024; hippocampus, t
18
= 2.13, P = 0.024; Fig. 5a)
and in the plasticity between related components (mPFC, t
18
= 1.85,
P = 0.041; but not hippocampus, t
18
= 0.81, P = 0.785; Fig. 6a).
To ensure that sensitivity to the construction process was main-
tained across the duration of the experiment, we also considered
temporal dynamics of other adaptation effects and of value signals
encoded on decision trials. In the unfamiliar group, both adaptation
in mPFC to repetition of any item (but not stimulus) and adaptation
in visual areas to repetition of a stimulus did not show a reduction
over time (one-tailed paired t tests, t
18
= 0.46, P = 0.326; Fig. 5a;
t
18
= 0.50, P = 0.312; Supplementary Fig. 4a). Furthermore, the cho-
sen value signal encoded by mPFC also did not reduce over time, but
instead remained consistent across sessions (Fig. 3c). In addition, per-
formance on the imagination task improved across blocks (Fig. 5b,c).
Rather than a loss of sensitivity, this suggests that the diminishing
adaptation effects demonstrate that simulated experience is sufficient
x = 0
z = 10
x = 22
(AAB) < (CAB) (AB) < (CB) (AAB) < (CAB)
a b
c
Effect size (a.u.)
1.5
2.0
0.5
–0.5
0
1.0
Unfamiliar
Familiar
*
*
Figure 4 Sensory exposure to a novel good: comparison between the
unfamiliar and familiar groups during construction of a novel good.
(a) ROI used to compare mPFC adaptation effects. (b) ROI used to
compare hippocampus adaptation effects. (c) In mPFC, the familiar
group showed less adaptation between the novel goods and their related
components (left, P = 0.053, trend) and significantly less adaptation
between related components (middle, P = 0.033; both extracted from
the ROI shown in a). In the hippocampus, the familiar group showed
significantly less adaptation between the novel goods and their related
components (right, P = 0.008, extracted from ROIs shown in b).
*P < 0.05. Data are presented as mean ± s.e.m. across participants.
a
B1
B2 and B3
2.0
*
1.5
1.0
(AAB) < (CAB) (AAB) < (CAB) (AA) < (CA)
Effect size (a.u.)
0.5
0
*
b
100
80
60
40
Percentage accuracy
20
0
B1 B2 B3
Component Compound
B1 B2 B3
c
3
2
1
Reaction time (s)
0
B1 B2 B3
Component Compound
B1 B2 B3
Figure 5 In the absence of sensory exposure,
there was evidence for the construction
mechanism only in early trials: block 1 compared
with blocks 2 and 3 for unfamiliar subjects.
(a) There was significantly less adaptation in
blocks 2 and 3 between the novel goods and their
related components in mPFC and hippocampus,
respectively (left and middle, P = 0.024 each;
ROIs are shown). There was no significant
reduction across time in the mPFC adaptation
of a component item to itself when predicted
by two different stimuli (right, P = 0.326, ROI
is shown). (b,c) On the imagination task, the
unfamiliar group showed an increase in accuracy
(b) and a decrease in reaction time (c) across
blocks. *P < 0.05. Data are presented as mean ±
s.e.m. across participants.
npg
© 2013 Nature America, Inc. All rights reserved.

1 4 9 6 VOLUME 16 | NUMBER 10 | OCTOBER 2013 nature neurOSCIenCe
a r t I C l e S
to establish an independent representation of the novel good that no
longer needs to be reconstructed anew on each trial.
Despite the overall reduction of cross-component suppres-
sion over the course of the experiment, this was not true for com-
ponents that had been used to construct high-value novel goods.
When averaging across the final two blocks, both the mPFC and
hippocampus showed a significant positive correlation with the
value of the compound items (mPFC, r = 0.64, P
= 0.002; hippo-
campus, r = 0.63, P = 0.003, Fig. 6b,d and Supplementary Fig. 5),
after accounting for variance explained by the value of the compo-
nent items in both cases. A median split of participants according
to the value assigned to the novel goods subsequently verified that
there was long-lasting plasticity in mPFC and hippocampus in the
final two blocks for those participants who attributed high, but not
low, values to the novel goods (mPFC: high, t
8
= 2.84, P = 0.022;
high versus low, t
8
= 2.68, P = 0.028; hippocampus: high, t
8
= 3.52,
P = 0.008; high versus low, t
8
= 5.36, P < 0.001; Fig. 6c,e). Suggestive
evidence that value-dependent adaptation between related compo-
nent items emerged later in hippocampus relative to mPFC (Fig. 6c,e)
could not be verified statistically (t
8
= 1.30, P = 0.229). Together, these
results suggest that the plasticity is long-lasting if value is attributed
to the original association.
DISCUSSION
The role of memory in prospective evaluation and inference has been
emphasized in both animals
22
and humans
3,4,20
. Simulation and pre-
play can be used to explore an internal model of the environment
and evaluate anticipated outcomes
8,23
. However, the neural mecha-
nisms by which these processes are achieved have remained unclear,
particularly in circumstances in which anticipated outcomes have not
previously been experienced. We used repetition suppression in fMRI
to reveal a neuronal mechanism that supports prospective representa-
tion and evaluation of novel experiences.
Repetition suppression has been used extensively in sensory brain
regions to probe the information content of neural activations and,
more recently, in more frontal brain regions, including orbitofrontal
cortex
24
. However, a number of different hypotheses have been
proposed to explain the underlying physiological mechanisms behind
the phenomenon, including fatigue, sparse coding and predictive
coding
10,25–27
. Although there is not yet a consensus on which mecha-
nism provides the most appropriate explanation for the phenomenon,
the consequences of this ambiguity are mitigated when used in a care-
fully controlled experimental design, as all models make the same
prediction: if a neural population is sensitive to a particular feature
or dimension, then suppression will occur in response to a repetition
of this feature, but not others.
The repetition suppression procedure that we used was designed
to allow interrogation of the underlying representation of a novel
reward. By asking people to imagine and evaluate novel rewards in
the scanner, we found that the neural representation of a novel reward
was dependent on representations of multiple related and previously
experienced rewards. Our data suggest that neuronal networks can
construct a novel experience by simultaneous activation of multiple
previous memories so that this constructed experience may be evalu-
ated. Although signals in the anterior hippocampus were found to be
related to construction, those in mPFC were related to both construc-
tion and valuation.
Crucially, unlike other goal-directed decision mechanisms that have
been reported
21,23,28,29
, we only found evidence for a construction
mechanism when subjects had no direct experience of an outcome,
and even then only fleetingly. It is therefore possible that constructed
value can provide a substitute for direct experience and train the expe-
riential goal-directed systems that have been studied previously. This
training signal may be considered analogous to off-line training of
an habitual system that makes use of simulations from an internal
goal-directed model
23,30–32
. Whereas the teaching signal provided to a
habitual system replicates, or fine tunes, previous sensory experience,
the teaching signal provided to a goal-directed system may establish
an internal model of the future world by repeated imagination of a
novel experience.
During the construction process, a second repetition suppression
effect was observed between distinct and previously unassociated
memories that contributed to the construction. This effect implies that
the neural representation of related, compared with unrelated, compo-
nent items became more similar as a consequence of the pre-scan train-
ing task, during which the participants were first exposed to the novel
compounds. Notably, given that the suppression was not observed
in the familiar group, it seems highly unlikely that this suppression
Figure 6 In the absence of sensory exposure,
repetition suppression between related
components was maintained across the
duration of the experiment only if participants
assigned high value to the compound goods.
(a) Participants from the unfamiliar group
showed significant reduction in adaptation
between related components over time in
mPFC, but not hippocampus (P = 0.041 and
P = 0.785, respectively, ROIs are shown).
(b,d) The correlations shown in Figure 2 were
also significant in mPFC (b) and hippocampus
(d) when considering suppression effects
between related components in blocks 2 and
3 alone: the amount of suppression across
participants correlated positively with the
average value of the compound goods (mPFC,
r = 0.64, P = 0.002; hippocampus, r = 0.63,
P = 0.003). (c,e) A median split of participants
into those that assigned high and low values
to the compound goods revealed significant
suppression between related components in blocks 2 and 3 only in those participants who assigned high value (c, mPFC, High, P = 0.022; High versus
Low, P = 0.028; e, hippocampus, High, P = 0.008; High versus Low, P < 0.001). *P < 0.05. Data are presented as mean ± s.e.m. across participants.
a b c
d e
1.5
1.0
0.5
0
Effect size (a.u.)
(AB) < (CB)
B1
B2 and B3
2
1
0
–1
210–1
Value of novel goods
210–1
Value of novel goods
Adaptation effect size
1
0
–1
Adaptation effect size
1.5
1.0
0.5
0
–0.5
HighHigh LowLow
B1
High Low
B1
B2 and B3
High Low
B2 and B3
1.0
0.5
0
–0.5
Effect size (a.u.) Effect size (a.u.)
*
*
*
*
*
npg
© 2013 Nature America, Inc. All rights reserved.

Citations
More filters
Journal ArticleDOI
17 Jun 2016-Science
TL;DR: These findings suggest that global relational codes may be used to organize nonspatial conceptual representations and that these codes may have a hexagonal gridlike pattern when conceptual knowledge is laid out in two continuous dimensions.
Abstract: It has been hypothesized that the brain organizes concepts into a mental map, allowing conceptual relationships to be navigated in a manner similar to that of space. Grid cells use a hexagonally symmetric code to organize spatial representations and are the likely source of a precise hexagonal symmetry in the functional magnetic resonance imaging signal. Humans navigating conceptual two-dimensional knowledge showed the same hexagonal signal in a set of brain regions markedly similar to those activated during spatial navigation. This gridlike signal is consistent across sessions acquired within an hour and more than a week apart. Our findings suggest that global relational codes may be used to organize nonspatial conceptual representations and that these codes may have a hexagonal gridlike pattern when conceptual knowledge is laid out in two continuous dimensions.

575 citations

Journal ArticleDOI
24 Oct 2018-Neuron
TL;DR: Experimental evidence and theoretical frameworks that point to principles unifying how to learn and use abstract, generalizable knowledge are reviewed and it is suggested that map-like representations observed in a spatial context may be an instance of general coding mechanisms capable of organizing knowledge of all kinds.

539 citations


Cites background from "Online evaluation of novel choices ..."

  • ...Indeed, when subjects imagine the taste of a new food that they have never experienced but is constructed from known ingredients (such as Tea-Jelly), both regions show evidence that cellular ensembles for the ingredients are active simultaneously (Barron et al., 2013)....

    [...]

Journal ArticleDOI
09 Nov 2018-Science
TL;DR: It is argued that spatial-processing principles in the hippocampalentorhinal region provide a geometric code to map information domains of cognitive spaces for high-level cognition and discuss recent evidence for this proposal.
Abstract: The hippocampal formation has long been suggested to underlie both memory formation and spatial navigation. We discuss how neural mechanisms identified in spatial navigation research operate across information domains to support a wide spectrum of cognitive functions. In our framework, place and grid cell population codes provide a representational format to map variable dimensions of cognitive spaces. This highly dynamic mapping system enables rapid reorganization of codes through remapping between orthogonal representations across behavioral contexts, yielding a multitude of stable cognitive spaces at different resolutions and hierarchical levels. Action sequences result in trajectories through cognitive space, which can be simulated via sequential coding in the hippocampus. In this way, the spatial representational format of the hippocampal formation has the capacity to support flexible cognition and behavior.

360 citations

Journal ArticleDOI
TL;DR: Fundamental similarities between the species and one human frontal area with no monkey counterpart are identified, suggesting that everyday human decision-making capitalizes on a neural apparatus similar to the one that supports monkeys when foraging in the wild.
Abstract: Reward-guided decision-making depends on a network of brain regions. Among these are the orbitofrontal and the anterior cingulate cortex. However, it is difficult to ascertain if these areas constitute anatomical and functional unities, and how these areas correspond between monkeys and humans. To address these questions we looked at connectivity profiles of these areas using resting-state functional MRI in 38 humans and 25 macaque monkeys. We sought brain regions in the macaque that resembled 10 human areas identified with decision making and brain regions in the human that resembled six macaque areas identified with decision making. We also used diffusion-weighted MRI to delineate key human orbital and medial frontal brain regions. We identified 21 different regions, many of which could be linked to particular aspects of reward-guided learning, valuation, and decision making, and in many cases we identified areas in the macaque with similar coupling profiles.

330 citations


Cites background from "Online evaluation of novel choices ..."

  • ...Similar activations have been reported nearby (10, 29, 30)....

    [...]

  • ...A dorsal medial frontal area in humans, area 9, was linked to decision making when the outcome had to be imagined or modeled in some way (30, 40), and its activity coupling pattern resembled area 9 (79) in the monkey medial frontal cortex....

    [...]

  • ...Area 9’s coupling pattern resembled that of the area linked to imagination of other peoples’ values and unexperienced rewards (30, 40) (Fig....

    [...]

  • ...A region near area 14m has been linked to decisions or attentional selection of choices (21, 23, 29, 30)....

    [...]

  • ...A similar region was active when people imagined howmuch they would like new and unexperienced food items that were nevertheless composed of familiar components (30)....

    [...]

Journal ArticleDOI
TL;DR: The mechanism of subjective decision-making involves the dynamic activation of OFC states associated with each choice alternative, tracked by recovering temporally precise neural states from multidimensional data in OFC.
Abstract: When making a subjective choice, the brain must compute a value for each option and compare those values to make a decision. The orbitofrontal cortex (OFC) is critically involved in this process, but the neural mechanisms remain obscure, in part due to limitations in our ability to measure and control the internal deliberations that can alter the dynamics of the decision process. Here we tracked these dynamics by recovering temporally precise neural states from multidimensional data in OFC. During individual choices, OFC alternated between states associated with the value of two available options, with dynamics that predicted whether a subject would decide quickly or vacillate between the two alternatives. Ensembles of value-encoding neurons contributed to these states, with individual neurons shifting activity patterns as the network evaluated each option. Thus, the mechanism of subjective decision-making involves the dynamic activation of OFC states associated with each choice alternative.

310 citations

References
More filters
Journal ArticleDOI
TL;DR: Most of the rat investigations, which I shall report, were carried out in the Berkeley laboratory, and a few, though a very few, were even carried out by me myself.
Abstract: I shall devote the body of this paper to a description of experiments with rats. But I shall also attempt in a few words at the close to indicate the significance of these findings on rats for the clinical behavior of men. Most of the rat investigations, which I shall report, were carried out in the Berkeley laboratory. But I shall also include, occasionally, accounts of the behavior of non-Berkeley rats who obviously have misspent their lives in out-of-State laboratories. Furthermore, in reporting our Berkeley experiments I shall have to omit a very great many. The ones I shall talk about were carried out by graduate students (or underpaid research assistants) who, supposedly, got some of their ideas from me. And a few, though a very few, were even carried out by me myself. Let me begin by presenting diagrams for a couple of typical mazes, an alley maze and an elevated maze. In the typical experiment a hungry rat is put at the entrance of the maze (alley or elevated), and wanders about through the various true path segments and blind alleys until he finally comes to the food box and eats. This is repeated (again in the typical experiment) one trial every 24 hours and the animal tends to make fewer and fewer errors (that is, blindalley entrances) and to take less and less time between start and goal-box until finally he is entering no blinds at all and running in a very few seconds from start to goal. The results are usually presented in the form of average curves of blind-entrances, or of seconds from start to finish, for groups of rats. All students agree as to the facts. They disagree, however, on theory and explanation. (1) First, there is a school of animal psychologists which believes that the maze behavior of rats is a matter of mere simple stimulus-response connections. Learning, according to them, consists in the strengthening of some of these connections and in the weakening of others. According to this ‘stimulus-response’ school the rat in progressing down the maze is helplessly responding to a succession of external stimulisights, sounds, smells, pressures, etc. impinging upon his external sense organs-plus internal stimuli coming from the viscera and from the skeletal muscles. Figure 1: Plan of maze 14-Unit T-Alley Maze. (From M.H. Elliot, The effect of change of reward on the maze performance of rats. University of California Publications in Psychology, 1928, 4, 20.)

5,735 citations

Journal ArticleDOI
TL;DR: A sequential experiment that provides, at each stage in the sequence, an estimate of the utility to the subject of some amount of a commodity, and to present a few experimental results obtained with the method.
Abstract: The purpose of this paper is to describe a sequential experiment that provides, at each stage in the sequence, an estimate of the utility to the subject of some amount of a commodity (e.g., money), and to present a few experimental results obtained with the method. The procedure is based upon the following well-known ‘expected utility hypothesis’. For each person there exist numerical constants, called utilities, associated with the various possible outcomes of his actions, given the external events not under his control. If, for a given subject, we could know the values of these constants and the (‘personal’) probabilities he assigns to the various external events we could, according to this model, predict his choice from among any available set of actions. He will choose an action with the highest expected utility; i.e., with the highest average of utilities of outcomes, weighted by the probabilities he assigns to the corresponding events. He will be indifferent between any two actions with equal expected utilities. Note that (by the nature of weighted averages) the comparison between expected utilities does not depend on which two particular outcomes are regarded as having zero-utility and unit-utility.

2,426 citations

Journal ArticleDOI
TL;DR: This work considers three models that have been proposed to account for repetition-related reductions in neural activity, and evaluates them in terms of their ability to accounts for the main properties of this phenomenon as measured with single-cell recordings and neuroimaging techniques.

2,185 citations

Journal ArticleDOI
TL;DR: This work considers dual-action choice systems from a normative perspective, and suggests a Bayesian principle of arbitration between them according to uncertainty, so each controller is deployed when it should be most accurate.
Abstract: A broad range of neural and behavioral data suggests that the brain contains multiple systems for behavioral choice, including one associated with prefrontal cortex and another with dorsolateral striatum. However, such a surfeit of control raises an additional choice problem: how to arbitrate between the systems when they disagree. Here, we consider dual-action choice systems from a normative perspective, using the computational theory of reinforcement learning. We identify a key trade-off pitting computational simplicity against the flexible and statistically efficient use of experience. The trade-off is realized in a competition between the dorsolateral striatal and prefrontal systems. We suggest a Bayesian principle of arbitration between them according to uncertainty, so each controller is deployed when it should be most accurate. This provides a unifying account of a wealth of experimental evidence about the factors favoring dominance by either system.

2,171 citations

Journal ArticleDOI
TL;DR: Functional magnetic resonance imaging is used to show that neural activity in several brain regions—particularly the ventral striatum, medial prefrontal cortex and posterior cingulate cortex—tracks the revealed subjective value of delayed monetary rewards.
Abstract: Neuroimaging studies of decision-making have generally related neural activity to objective measures (such as reward magnitude, probability or delay), despite choice preferences being subjective. However, economic theories posit that decision-makers behave as though different options have different subjective values. Here we use functional magnetic resonance imaging to show that neural activity in several brain regions—particularly the ventral striatum, medial prefrontal cortex and posterior cingulate cortex—tracks the revealed subjective value of delayed monetary rewards. This similarity provides unambiguous evidence that the subjective value of potential rewards is explicitly represented in the human brain.

1,743 citations

Frequently Asked Questions (13)
Q1. What are the contributions mentioned in the paper "Online evaluation of novel choices by simultaneous representation of multiple memories" ?

To test these predictions, the authors required access to the information content of neural populations underlying the representation of a novel experience. The authors used fMRI adaptation to probe the neural representation of a novel food reward. The authors hypothesized that, if the representation of a novel food was constructed by explicit combination of multiple distinct experiences, they would observe fMRI adaptation when subjects evaluated a novel reward immediately after evaluating a component ingredient. Lastly, the authors hypothesized that this complex construction process would not be required after an independent neuronal representation of the novel good had been established. Furthermore, if multiple experiences were replayed simultaneously, plasticity might result between the underlying neuronal assemblies. 

Whereas the teaching signal provided to a habitual system replicates, or fine tunes, previous sensory experience, the teaching signal provided to a goal-directed system may establish an internal model of the future world by repeated imagination of a novel experience. This value dependence effect suggests that the representations of the component memories were simultaneously present during valuation of the novel compounds. Their data suggest that mPFC can combine previous experiences to construct prospective outcomes de novo on each trial and can then evaluate these constructed outcomes. Consistent with the proposed function of memory in prospective inference44,45, the formation of associative links46,47 and constructive episodic simulation48,49, their data suggest that hippocampal activity can also have an active role in constructing de novo experiences in non-spatial contexts. 

In the absence of sensory exposure, there was evidence for the construction mechanism only in early trials: block 1 compared with blocks 2 and 3 for unfamiliar subjects. 

To detect plasticity effects between the related components (component to component), the authors used the contrast [(A preceded by C) − (A preceded by B)], again averaging across all possible permutations (that is, explanatory variables (4) − (3)). 

the most plausible explanation for this change is that, through repeated representation of a novel compound, previously unrelated memories were recruited simultaneously, inducing a form of plasticity between the underlying representations of necessary components. 

Consistent with the proposed function of memory in prospective inference44,45, the formation of associative links46,47 and constructive episodic simulation48,49, their data suggestthat hippocampal activity can also have an active role in constructing de novo experiences in non-spatial contexts. 

Participants were required to continue with this stimulus-item learning task until their average reaction time per block approached 800 ms with 100% accuracy. 

On each trial, 1 of the 12 abstract shapes was shown for 400 ms before all six possible items were presented in randomized positions across the screen. 

hippocampal activity is often recorded in concert with a network involving mPFC in studies of spatial memory and scene construction12,19,43. 

To detect brain regions involved in constructing the novel goods (component to compound), the authors used the contrast [(AB preceded by C) − (AB preceded by A)], averaging across all possible permutations (that is, explanatory variables (2) − (1)). 

During the construction process, a second repetition suppression effect was observed between distinct and previously unassociated memories that contributed to the construction. 

This effect implies that the neural representation of related, compared with unrelated, component items became more similar as a consequence of the pre-scan training task, during which the participants were first exposed to the novel compounds. 

To detect brain regions showing adaptation to repeated item, but not stimulus (item to self), the authors used the contrast [(item preceded by different item) − (item preceded by itself but paired with a different stimulus)] (that is, explanatory variables (4) − (5)).