With or without you: predictive coding and Bayesian inference in the brain
TL;DR: It is argued that predictive coding is an algorithmic/representational motif that can serve several different computational goals of which Bayesian inference is but one, and that whileBayesian inference can utilize predictive coding, it can also be realized by a variety of other representations.
About: This article is published in Current Opinion in Neurobiology.The article was published on 2017-10-01 and is currently open access. It has received 172 citations till now. The article focuses on the topics: Frequentist inference & Predictive inference.
Citations
More filters
TL;DR: Recent advances in the understanding of the neural sources and targets of expectations in perception are reviewed and Bayesian theories of perception that prescribe how an agent should integrate prior knowledge and sensory information are discussed.
521 citations
TL;DR: A complete mathematical synthesis of active inference on discrete state-space models, which derives neuronal dynamics from first principles and relates this dynamics to biological processes is provided.
129 citations
Cites background from "With or without you: predictive cod..."
...This is the point of contact between active inference and the Bayesian brain [58–60]....
[...]
01 Jun 2014
TL;DR: This article found that higher order correlations in natural scenes induced a sparser code, in which information is encoded by reliable activation of a smaller set of neurons and can be read out more easily.
Abstract: Neural codes are believed to have adapted to the statistical properties of the natural environment. However, the principles that govern the organization of ensemble activity in the visual cortex during natural visual input are unknown. We recorded populations of up to 500 neurons in the mouse primary visual cortex and characterized the structure of their activity, comparing responses to natural movies with those to control stimuli. We found that higher order correlations in natural scenes induced a sparser code, in which information is encoded by reliable activation of a smaller set of neurons and can be read out more easily. This computationally advantageous encoding for natural scenes was state-dependent and apparent only in anesthetized and active awake animals, but not during quiet wakefulness. Our results argue for a functional benefit of sparsification that could be a general principle governing the structure of the population activity throughout cortical microcircuits.
125 citations
TL;DR: It is suggested that the late posterior positivity/P600 is triggered when the comprehender detects a conflict between the input and her model of the communicator and communicative environment, which leads to an initial failure to incorporate the unpredicted input into the situation model, which may be followed by second-pass attempts to make sense of the discourse through reanalysis, repair, or reinterpretation.
Abstract: It has been proposed that hierarchical prediction is a fundamental computational principle underlying neurocognitive processing. Here, we ask whether the brain engages distinct neurocognitive mechanisms in response to inputs that fulfill versus violate strong predictions at different levels of representation during language comprehension. Participants read three-sentence scenarios in which the third sentence constrained for a broad event structure, for example, {Agent caution animate-Patient}. High constraint contexts additionally constrained for a specific event/lexical item, for example, a two-sentence context about a beach, lifeguards, and sharks constrained for the event, {Lifeguards cautioned Swimmers}, and the specific lexical item swimmers. Low constraint contexts did not constrain for any specific event/lexical item. We measured ERPs on critical nouns that fulfilled and/or violated each of these constraints. We found clear, dissociable effects to fulfilled semantic predictions (a reduced N400), to event/lexical prediction violations (an increased late frontal positivity), and to event structure/animacy prediction violations (an increased late posterior positivity/P600). We argue that the late frontal positivity reflects a large change in activity associated with successfully updating the comprehender's current situation model with new unpredicted information. We suggest that the late posterior positivity/P600 is triggered when the comprehender detects a conflict between the input and her model of the communicator and communicative environment. This leads to an initial failure to incorporate the unpredicted input into the situation model, which may be followed by second-pass attempts to make sense of the discourse through reanalysis, repair, or reinterpretation. Together, these findings provide strong evidence that confirmed and violated predictions at different levels of representation manifest as distinct spatiotemporal neural signatures.
102 citations
TL;DR: Evidence is described for the importance of an important distinction between two forms of prediction that may advance the understanding of brain function and it is argued that it is critical for the development of the predictive processing framework and for an understanding of the perturbations that drive the emergence of neuropsychiatric symptoms and experiences.
Abstract: The idea that predictions shape how we perceive and comprehend the world has become increasingly influential in the field of systems neuroscience. It also forms an important framework for understanding neuropsychiatric disorders, which are proposed to be the result of disturbances in the mechanisms through which prior information influences perception and belief, leading to the production of suboptimal models of the world. There is a widespread tendency to conceptualize the influence of predictions exclusively in terms of ‘top-down’ processes, whereby predictions generated in higher-level areas exert their influence on lower-level areas within an information processing hierarchy. However, this excludes from consideration the predictive information embedded in the ‘bottom-up’ stream of information processing. We describe evidence for the importance of this distinction and argue that it is critical for the development of the predictive processing framework and, ultimately, for an understanding of the perturbations that drive the emergence of neuropsychiatric symptoms and experiences.
89 citations
References
More filters
Book•
01 Jan 1988TL;DR: This book provides a clear and simple account of the key ideas and algorithms of reinforcement learning, which ranges from the history of the field's intellectual foundations to the most recent developments and applications.
Abstract: Reinforcement learning, one of the most active research areas in artificial intelligence, is a computational approach to learning whereby an agent tries to maximize the total amount of reward it receives when interacting with a complex, uncertain environment. In Reinforcement Learning, Richard Sutton and Andrew Barto provide a clear and simple account of the key ideas and algorithms of reinforcement learning. Their discussion ranges from the history of the field's intellectual foundations to the most recent developments and applications. The only necessary mathematical background is familiarity with elementary concepts of probability. The book is divided into three parts. Part I defines the reinforcement learning problem in terms of Markov decision processes. Part II provides basic solution methods: dynamic programming, Monte Carlo methods, and temporal-difference learning. Part III presents a unified view of the solution methods and incorporates artificial neural networks, eligibility traces, and planning; the two final chapters present case studies and consider the future of reinforcement learning.
37,989 citations
TL;DR: A fast, greedy algorithm is derived that can learn deep, directed belief networks one layer at a time, provided the top two layers form an undirected associative memory.
Abstract: We show how to use "complementary priors" to eliminate the explaining-away effects that make inference difficult in densely connected belief nets that have many hidden layers. Using complementary priors, we derive a fast, greedy algorithm that can learn deep, directed belief networks one layer at a time, provided the top two layers form an undirected associative memory. The fast, greedy algorithm is used to initialize a slower learning procedure that fine-tunes the weights using a contrastive version of the wake-sleep algorithm. After fine-tuning, a network with three hidden layers forms a very good generative model of the joint distribution of handwritten digit images and their labels. This generative model gives better digit classification than the best discriminative learning algorithms. The low-dimensional manifolds on which the digits lie are modeled by long ravines in the free-energy landscape of the top-level associative memory, and it is easy to explore these ravines by using the directed connections to display what the associative memory has in mind.
15,055 citations
"With or without you: predictive cod..." refers background in this paper
...Pure direct coding models have enjoyed great success at a number of challenging supervised and unsupervised learning tasks, and their dynamics typically take a biologically plausible form, requiring neurons to integrate their inputs linearly and apply a spiking nonlinearity [60] or (a possibly stochastic) threshold [61, 62]....
[...]
TL;DR: Findings in this work indicate that dopaminergic neurons in the primate whose fluctuating output apparently signals changes or errors in the predictions of future salient and rewarding events can be understood through quantitative theories of adaptive optimizing control.
Abstract: The capacity to predict future events permits a creature to detect, model, and manipulate the causal structure of its interactions with its environment. Behavioral experiments suggest that learning is driven by changes in the expectations about future salient events such as rewards and punishments. Physiological work has recently complemented these studies by identifying dopaminergic neurons in the primate whose fluctuating output apparently signals changes or errors in the predictions of future salient and rewarding events. Taken together, these findings can be understood through quantitative theories of adaptive optimizing control.
8,163 citations
"With or without you: predictive cod..." refers background in this paper
...Neurally, there is strong evidence that this reward prediction error is instantiated by dopamine [47], which has indeed been shown to be a potent modulator of synaptic plasticity [48]....
[...]
TL;DR: This article presents a general class of associative reinforcement learning algorithms for connectionist networks containing stochastic units that are shown to make weight adjustments in a direction that lies along the gradient of expected reinforcement in both immediate-reinforcement tasks and certain limited forms of delayed-reInforcement tasks, and they do this without explicitly computing gradient estimates.
Abstract: This article presents a general class of associative reinforcement learning algorithms for connectionist networks containing stochastic units. These algorithms, called REINFORCE algorithms, are shown to make weight adjustments in a direction that lies along the gradient of expected reinforcement in both immediate-reinforcement tasks and certain limited forms of delayed-reinforcement tasks, and they do this without explicitly computing gradient estimates or even storing information from which such estimates could be computed. Specific examples of such algorithms are presented, some of which bear a close relationship to certain existing algorithms while others are novel but potentially interesting in their own right. Also given are results that show how such algorithms can be naturally integrated with backpropagation. We close with a brief discussion of a number of additional issues surrounding the use of such algorithms, including what is known about their limiting behaviors as well as further considerations that might be used to help develop similar but potentially more powerful reinforcement learning algorithms.
7,930 citations
6,631 citations