An extended reinforcement learning model of basal ganglia to understand the contributions of serotonin and dopamine in risk-based decision making, reward prediction, and punishment learning.
Pragathi Priyadharsini Balasubramani,V. Srinivasa Chakravarthy,Balaraman Ravindran,Ahmed A. Moustafa +3 more
Reads0
Chats0
TLDR
A model of risk based decision making in a modified Reinforcement Learning (RL)-framework that reconciles several existing theories of 5HT and DA in the BG is presented.Abstract:
Although empirical and neural studies show that serotonin (5HT) plays many functional roles in the brain, prior computational models mostly focus on its role in behavioral inhibition. In this study, we present a model of risk based decision making in a modified Reinforcement Learning (RL)-framework. The model depicts the roles of dopamine (DA) and serotonin (5HT) in Basal Ganglia (BG). In this model, the DA signal is represented by the temporal difference error (δ), while the 5HT signal is represented by a parameter (α) that controls risk prediction error. This formulation that accommodates both 5HT and DA reconciles some of the diverse roles of 5HT particularly in connection with the BG system. We apply the model to different experimental paradigms used to study the role of 5HT: (1) Risk-sensitive decision making, where 5HT controls risk assessment, (2) Temporal reward prediction, where 5HT controls time-scale of reward prediction, and (3) Reward/Punishment sensitivity, in which the punishment prediction error depends on 5HT levels. Thus the proposed integrated RL model reconciles several existing theories of 5HT and DA in the BG.read more
Citations
More filters
Journal ArticleDOI
Learning Reward Uncertainty in the Basal Ganglia
TL;DR: A class of models that encode both the mean reward and the spread of the rewards, the former in the difference between the synaptic weights of D1 and D2 neurons, and the latter in their sum are presented.
Journal ArticleDOI
The Protective Action Encoding of Serotonin Transients in the Human Brain
Rosalyn J. Moran,Kenneth T. Kishida,Terry Lohrenz,Ignacio Saez,Adrian W. Laxton,Mark R. Witcher,Stephen B. Tatter,Thomas L. Ellis,Paul E. M. Phillips,Peter Dayan,P. Read Montague,P. Read Montague +11 more
TL;DR: Serotonergic concentrations transiently increase as a whole following negative reward prediction errors, while reversing when counterfactual losses predominate, which provides initial evidence that the serotonergic system acts as an opponent to dopamine signaling, as anticipated by theoretical models.
Journal ArticleDOI
Nicotinic receptors in the ventral tegmental area promote uncertainty-seeking
Jérémie Naudé,Jérémie Naudé,Jérémie Naudé,Stefania Tolu,Stefania Tolu,Stefania Tolu,Malou Dongelmans,Malou Dongelmans,Malou Dongelmans,Nicolas Torquet,Nicolas Torquet,Nicolas Torquet,Sébastien Valverde,Sébastien Valverde,Sébastien Valverde,Guillaume Rodriguez,Guillaume Rodriguez,Guillaume Rodriguez,Stéphanie Pons,Uwe Maskos,Alexandre Mourot,Alexandre Mourot,Alexandre Mourot,Fabio Marti,Fabio Marti,Fabio Marti,Philippe Faure,Philippe Faure,Philippe Faure +28 more
TL;DR: An unanticipated role for subcortical nAChRs in motivation induced by expected uncertainty is revealed and a parsimonious account for a wealth of behaviors related to nA cholinergic receptors expressed in the VTA expressing the β2 subunit is provided.
Journal ArticleDOI
A network model of basal ganglia for understanding the roles of dopamine and serotonin in reward-punishment-risk based decision making.
Pragathi Priyadharsini Balasubramani,V. Srinivasa Chakravarthy,Balaraman Ravindran,Ahmed A. Moustafa,Ahmed A. Moustafa +4 more
TL;DR: The model predicts that optimizing 5HT levels along with DA medications might be essential for improving the patients' reward-punishment learning deficits, and describes how DA and 5HT mediate activity in these classes of neurons (D1R, D2R-, D1R-D2R- MSNs).
Journal ArticleDOI
Amantadine preserves dopamine level and attenuates depression-like behavior induced by traumatic brain injury in rats.
Liang Tan,Hongfei Ge,Jun Tang,Chuhua Fu,Wangsheng Duanmu,Yujie Chen,Rong Hu,Jian-feng Sui,Xin Liu,Hua Feng +9 more
TL;DR: It is concluded that DA plays a critical role in post-TBI depression, and that amantadine shows its potential value in anti-depression treatment for TBI.
References
More filters
Book
Genetic algorithms in search, optimization, and machine learning
TL;DR: In this article, the authors present the computer techniques, mathematical tools, and research results that will enable both students and practitioners to apply genetic algorithms to problems in many fields, including computer programming and mathematics.
Book
Reinforcement Learning: An Introduction
TL;DR: This book provides a clear and simple account of the key ideas and algorithms of reinforcement learning, which ranges from the history of the field's intellectual foundations to the most recent developments and applications.
Book ChapterDOI
Prospect theory: an analysis of decision under risk
Daniel Kahneman,Amos Tversky +1 more
TL;DR: In this paper, the authors present a critique of expected utility theory as a descriptive model of decision making under risk, and develop an alternative model, called prospect theory, in which value is assigned to gains and losses rather than to final assets and in which probabilities are replaced by decision weights.
Journal ArticleDOI
Prospect theory: analysis of decision under risk
Daniel Kahneman,Amos Tversky +1 more
Journal ArticleDOI
A Neural Substrate of Prediction and Reward
TL;DR: Findings in this work indicate that dopaminergic neurons in the primate whose fluctuating output apparently signals changes or errors in the predictions of future salient and rewarding events can be understood through quantitative theories of adaptive optimizing control.