The authors conceptualize these memory systems from both behavioural and neurobiological perspectives, guided by three related principles: * First, that our understanding of a wide range of memory phenomena can be advanced by breaking down memory into multiple forms with different operating characteristics. * Second, that different forms of memory representation are supported by distinct brain pathways with circuitry and neural coding properties. * Third, that the contributions of different brain systems can be compared and contrasted by distinguishing between dedicated (or specific) and elaborate (or general) memory systems. A primary goal of this work is to relate the neurobiological properties of dedicated and elaborate systems to their neuropsychological counterparts, and in so doing, account for the phenomenology of memory, from conditioning to conscious recollection.

From Conditioning to Conscious Recollection Memory Systems of the Brain. Oxford Psychology Series, Volume 35.

The Tolman-Eichenbaum Machine: Unifying Space and Relational Memory through Generalization in the Hippocampal Formation.

Artificial neural networks suffer from catastrophic forgetting. Unlike humans, when these networks are trained on something new, they rapidly forget what was learned before. In the brain, a mechanism thought to be important for protecting memories is the reactivation of neuronal activity patterns representing those memories. In artificial neural networks, such memory replay can be implemented as ‘generative replay’, which can successfully – and surprisingly efficiently – prevent catastrophic forgetting on toy examples even in a class-incremental learning scenario. However, scaling up generative replay to complicated problems with many tasks or complex inputs is challenging. We propose a new, brain-inspired variant of replay in which internal or hidden representations are replayed that are generated by the network’s own, context-modulated feedback connections. Our method achieves state-of-the-art performance on challenging continual learning benchmarks (e.g., class-incremental learning on CIFAR-100) without storing data, and it provides a novel model for replay in the brain. One challenge that faces artificial intelligence is the inability of deep neural networks to continuously learn new information without catastrophically forgetting what has been learnt before. To solve this problem, here the authors propose a replay-based algorithm for deep learning without the need to store data.

/pdf/brain-inspired-replay-for-continual-learning-with-artificial-232i4neh86.pdf

Brain-inspired replay for continual learning with artificial neural networks.

Embracing Change: Continual Learning in Deep Neural Networks

Neuroscience research is undergoing a minor revolution. Recent advances in machine learning and artificial intelligence research have opened up new ways of thinking about neural computation. Many researchers are excited by the possibility that deep neural networks may offer theories of perception, cognition and action for biological brains. This approach has the potential to radically reshape our approach to understanding neural systems, because the computations performed by deep networks are learned from experience, and not endowed by the researcher. If so, how can neuroscientists use deep networks to model and understand biological brains? What is the outlook for neuroscientists who seek to characterize computations or neural codes, or who wish to understand perception, attention, memory and executive functions? In this Perspective, our goal is to offer a road map for systems neuroscience research in the age of deep learning. We discuss the conceptual and methodological challenges of comparing behaviour, learning dynamics and neural representations in artificial and biological systems, and we highlight new research questions that have emerged for neuroscience as a direct consequence of recent advances in machine learning.

If deep learning is the answer, what is the question?

Human Replay Spontaneously Reorganizes Experience

Replay bursts in humans coincide with activation of the default mode and parietal alpha networks

To make effective decisions, people need to consider the relationship between actions and outcomes. These are often separated by time and space. The neural mechanisms by which disjoint actions and outcomes are linked remain unknown. One promising hypothesis involves neural replay of nonlocal experience. Using a task that segregates direct from indirect value learning, combined with magnetoencephalography, we examined the role of neural replay in human nonlocal learning. After receipt of a reward, we found significant backward replay of nonlocal experience, with a 160-millisecond state-to-state time lag, which was linked to efficient learning of action values. Backward replay and behavioral evidence of nonlocal learning were more pronounced for experiences of greater benefit for future behavior. These findings support nonlocal replay as a neural mechanism for solving complex credit assignment problems during learning.

/pdf/experience-replay-is-associated-with-efficient-nonlocal-41nlo66466.pdf

Experience replay is associated with efficient nonlocal learning

Retrieval of everyday experiences is fundamental for informing our future decisions. The fine-grained neurophysiological mechanisms that support such memory retrieval are largely unknown. We studied participants who first experienced, without repetition, unique multicomponent 40-80-s episodes. One day later, they engaged in cued retrieval of these episodes while undergoing magnetoencephalography. By decoding individual episode elements, we found that trial-by-trial successful retrieval was supported by the sequential replay of episode elements, with a temporal compression factor of >60. The direction of replay supporting retrieval, either backward or forward, depended on whether the task goal was to retrieve elements of an episode that followed or preceded, respectively, a retrieval cue. This sequential replay was weaker in very-high-performing participants, in whom instead we found evidence for simultaneous clustered reactivation. Our results demonstrate that memory-mediated decisions are supported by a rapid replay mechanism that can flexibly shift in direction in response to task goals.

Yunzhe Liu

Papers

Human Replay Spontaneously Reorganizes Experience

Replay bursts in humans coincide with activation of the default mode and parietal alpha networks

Experience replay is associated with efficient nonlocal learning

Episodic memory retrieval success is associated with rapid replay of episode content.

Functional neuroimaging in psychiatry and the case for failing better