Rethinking dopamine as generalized prediction error
Matthew P.H. Gardner,Geoffrey Schoenbaum,Geoffrey Schoenbaum,Geoffrey Schoenbaum,Samuel J. Gershman +4 more
TLDR
A new theory of dopamine function is developed that embraces a broader conceptualization of prediction errors and concludes that by signaling errors in both sensory and reward predictions, dopamine supports a form of reinforcement learning that lies between model-based and model-free algorithms.Abstract:
Midbrain dopamine neurons are commonly thought to report a reward prediction error, as hypothesized by reinforcement learning theory. While this theory has been highly successful, several lines of evidence suggest that dopamine activity also encodes sensory prediction errors unrelated to reward. Here we develop a new theory of dopamine function that embraces a broader conceptualization of prediction errors. By signaling errors in both sensory and reward predictions, dopamine supports a form of reinforcement learning that lies between model-based and model-free algorithms. This account remains consistent with current canon regarding the correspondence between dopamine transients and reward prediction errors, while also accounting for new data suggesting a role for these signals in phenomena such as sensory preconditioning and identity unblocking, which ostensibly draw upon knowledge beyond reward predictions.read more
Citations
More filters
Posted ContentDOI
The cognitive basis of intracranial self-stimulation of midbrain dopamine neurons
Samuel J. Millard,Ivy B. Hoang,Zara Greer,Shayna L. O'Connor,Kate M. Wassum,Morgan H. James,David J. Barker,Melissa J. Sharpe +7 more
TL;DR: Dopamine neurons only support ICSS at supraphysiological frequencies and in a manner not reflecting dopamine’s role in learning, a manner that does not reflect the authors' subjective experience with endogenous firing of dopamine neurons during reinforcement learning.
Journal ArticleDOI
Leveraging Social Networks for the Assessment and Management of Neurological Patients
Amar Dhand,Archana Podury,Niteesh K. Choudhry,Shrikanth S. Narayanan,Min Seop Shin,Matthias R. Mehl +5 more
TL;DR: In this paper , a review of the biology and psychology of social networks, assessment methods including novel social sensors, and the design of network interventions and social therapeutics is presented, along with a set of scalable and quantitative tools increasing familiarity with social network effects and mechanisms.
Journal ArticleDOI
Reinforcement learning with associative or discriminative generalization across states and actions: fMRI at 3 T and 7 T
Jaron T. Colas,Neil M. Dundon,Raphael T. Gerraty,Natalie Saragosa-Harris,Karol P. Szymula,Koranis Tanwisuth,J. Michael Tyszka,Camilla van Geen,Harang Ju,Arthur W. Toga,Joshua I. Gold,Danielle S. Bassett,Catherine A. Hartley,Daphna Shohamy,Scott T. Grafton,John P. O'Doherty +15 more
TL;DR: Factoring in generalization as a multidimensional process in value‐based learning, these findings shed light on complexities that, while challenging classic RL, can still be resolved within the bounds of its core computations.
Journal ArticleDOI
Repeating patterns: Predictive processing suggests an aesthetic learning role of the basal ganglia in repetitive stereotyped behaviors
Blanca Spee,Ronald Sladky,Joerg Fingerhut,Alice Laciny,Christoph Kraus,Sidney Carls-Diamante,Christof Brücke,Matthew Pelowski,Marco Treven +8 more
TL;DR: It is suggested that basal ganglia feedback plays a central role in preconditioning cortical networks to anticipate self-generated, movement-related perception, and appears ideally situated to adjust the salience of sensory signals through precision weighting of (external) new sensory information, relative to the precision of predictions based on prior generated models.
Posted ContentDOI
A probabilistic successor representation for context-dependent prediction
TL;DR: A theory of learning SRs using prediction errors which includes optimally balancing uncertainty in new observations versus existing knowledge is introduced, allowing the model to learn and maintain multiple task-specific SRs and infer which one to use at any moment based on the accuracy of its predictions.
References
More filters