Showing papers by "Claudia Clopath published in 2019"

A deep learning framework for neuroscience

Abstract: Systems neuroscience seeks explanations for how the brain implements a wide variety of perceptual, cognitive and motor tasks. Conversely, artificial intelligence attempts to design computational systems based on the tasks they will have to solve. In artificial neural networks, the three components specified by design are the objective functions, the learning rules and the architectures. With the growing success of deep learning, which utilizes brain-inspired architectures, these three designed components have increasingly become central to how we model, engineer and optimize complex artificial learning systems. Here we argue that a greater focus on these components would also benefit systems neuroscience. We give examples of how this optimization-based framework can drive theoretical and experimental progress in neuroscience. We contend that this principled perspective on systems neuroscience will help to generate more rapid progress.

A diversity of interneurons and Hebbian plasticity facilitate rapid compressible learning in the hippocampus.

Abstract: The hippocampus is able to rapidly learn incoming information, even if that information is only observed once. Furthermore, this information can be replayed in a compressed format in either forward or reverse modes during sharp wave–ripples (SPW–Rs). We leveraged state-of-the-art techniques in training recurrent spiking networks to demonstrate how primarily interneuron networks can achieve the following: (1) generate internal theta sequences to bind externally elicited spikes in the presence of inhibition from the medial septum; (2) compress learned spike sequences in the form of a SPW–R when septal inhibition is removed; (3) generate and refine high-frequency assemblies during SPW–R-mediated compression; and (4) regulate the inter-SPW interval timing between SPW–Rs in ripple clusters. From the fast timescale of neurons to the slow timescale of behaviors, interneuron networks serve as the scaffolding for one-shot learning by replaying, reversing, refining, and regulating spike sequences. How does the hippocampus learn with only a single presentation of a stimulus? Nicola and Clopath show how interneuron networks can dilate pre-existing spike sequences during sharp waves into theta sequences for one-shot learning.

Interneuron-specific plasticity at parvalbumin and somatostatin inhibitory synapses onto CA1 pyramidal neurons shapes hippocampal output

Abstract: Summary The formation and maintenance of spatial representations within hippocampal cell assemblies is strongly dictated by patterns of inhibition from diverse interneuron populations. Although it is known that inhibitory synaptic strength is malleable, induction of long-term plasticity at distinct inhibitory synapses and its regulation of hippocampal network activity is not well understood. Here, we show that inhibitory synapses from parvalbumin and somatostatin expressing interneurons undergo long-term depression and potentiation respectively (PV-iLTD and SST-iLTP) during physiological activity patterns. Both forms of plasticity rely on T-type calcium channel activation to confer synapse specificity but otherwise employ distinct mechanisms. Since parvalbumin and somatostatin interneurons preferentially target perisomatic and distal dendritic regions respectively of CA1 pyramidal cells, PV-iLTD and SST-iLTP coordinate a reprioritisation of excitatory inputs from entorhinal cortex and CA3. Furthermore, circuit-level modelling reveals that PV-iLTD and SST-iLTP cooperate to stabilise place cells while facilitating representation of multiple unique environments within the hippocampal network.

Inhibitory microcircuits for top-down plasticity of sensory representations

Abstract: Rewards influence plasticity of early sensory representations, but the underlying changes in circuitry are unclear. Recent experimental findings suggest that inhibitory circuits regulate learning. In addition, inhibitory neurons are highly modulated by diverse long-range inputs, including reward signals. We, therefore, hypothesise that inhibitory plasticity plays a major role in adjusting stimulus representations. We investigate how top-down modulation by rewards interacts with local plasticity to induce long-lasting changes in circuitry. Using a computational model of layer 2/3 primary visual cortex, we demonstrate how interneuron circuits can store information about rewarded stimuli to instruct long-term changes in excitatory connectivity in the absence of further reward. In our model, stimulus-tuned somatostatin-positive interneurons develop strong connections to parvalbumin-positive interneurons during reward such that they selectively disinhibit the pyramidal layer henceforth. This triggers excitatory plasticity, leading to increased stimulus representation. We make specific testable predictions and show that this two-stage model allows for translation invariance of the learned representation. Rewards can improve stimulus processing in early sensory areas but the underlying neural circuit mechanisms are unknown. Here, the authors build a computational model of layer 2/3 primary visual cortex and suggest that plastic inhibitory circuits change first and then increase excitatory representations beyond the presence of rewards.

Audio-visual experience strengthens multisensory assemblies in adult mouse visual cortex

Abstract: We experience the world through multiple senses simultaneously. To better understand mechanisms of multisensory processing we ask whether inputs from two senses (auditory and visual) can interact and drive plasticity in neural-circuits of the primary visual cortex (V1). Using genetically-encoded voltage and calcium indicators, we find coincident audio-visual experience modifies both the supra and subthreshold response properties of neurons in L2/3 of mouse V1. Specifically, we find that after audio-visual pairing, a subset of multimodal neurons develops enhanced auditory responses to the paired auditory stimulus. This cross-modal plasticity persists over days and is reflected in the strengthening of small functional networks of L2/3 neurons. We find V1 processes coincident auditory and visual events by strengthening functional associations between feature specific assemblies of multimodal neurons during bouts of sensory driven co-activity, leaving a trace of multisensory experience in the cortical network. Sensory stimuli usually arrive simultaneously but the neural-circuit mechanisms that combine multiple streams of sensory information are incompletely understood. The authors here show that visual-auditory pairing drives plasticity in multi-modal neuron networks within the mouse visual cortex.

Policy Consolidation for Continual Reinforcement Learning

Abstract: We propose a method for tackling catastrophic forgetting in deep reinforcement learning that is \textit{agnostic} to the timescale of changes in the distribution of experiences, does not require knowledge of task boundaries, and can adapt in \textit{continuously} changing environments. In our \textit{policy consolidation} model, the policy network interacts with a cascade of hidden networks that simultaneously remember the agent's policy at a range of timescales and regularise the current policy by its own history, thereby improving its ability to learn without forgetting. We find that the model improves continual learning relative to baselines on a number of continuous control tasks in single-task, alternating two-task, and multi-agent competitive self-play settings.

Cortical credit assignment by Hebbian, neuromodulatory and inhibitory plasticity

Abstract: The cortex learns to make associations between stimuli and spiking activity which supports behaviour. It does this by adjusting synaptic weights. The complexity of these transformations implies that synapses have to change without access to the full error information, a problem typically referred to as "credit-assignment". However, it remains unknown how the cortex solves this problem. We propose that a combination of plasticity rules, 1) Hebbian, 2) acetylcholine-dependent and 3) noradrenaline-dependent excitatory plasticity, together with 4) inhibitory plasticity restoring E/I balance, effectively solves the credit assignment problem. We derive conditions under-which a neuron model can learn a number of associations approaching its theoretical capacity. We confirm our predictions regarding acetylcholine-dependent and inhibitory plasticity by reanalysing experimental data. Our work suggests that detailed cortical E/I balance reduces the dimensionality of the problem of associating inputs with outputs, thereby allowing imperfect "supervision" by neuromodulatory systems to guide learning effectively.

Learning spatiotemporal signals using a recurrent spiking network that discretizes time

Abstract: Learning to produce spatiotemporal sequences is a common task the brain has to solve. The same neural substrate may be used by the brain to produce different sequential behaviours. The way the brain learns and encodes such tasks remains unknown as current computational models do not typically use realistic biologically-plausible learning. Here, we propose a model where a spiking recurrent network of excitatory and inhibitory biophysical neurons drives a read-out layer: the dynamics of the recurrent network is constrained to encode time while the read-out neurons encode space. Space is then linked with time through plastic synapses that follow common Hebbian learning rules. We demonstrate that the model is able to learn spatiotemporal dynamics on a timescale that is behaviourally relevant. Learned sequences are robustly replayed during a regime of spontaneous activity. Author summary The brain has the ability to learn flexible behaviours on a wide range of time scales. Previous studies have successfully build spiking network models that learn a variety of computational tasks. However, often the learning involved is not local. Here, we investigate a model using biological-plausible plasticity rules for a specific computational task: spatiotemporal sequence learning. The architecture separates time and space into two different parts and this allows learning to bind space to time. Importantly, the time component is encoded into a recurrent network which exhibits sequential dynamics on a behavioural time scale. This network is then used as an engine to drive spatial read-out neurons. We demonstrate that the model can learn complicated spatiotemporal spiking dynamics, such as the song of a bird, and replay the song robustly.

Deep Reinforcement Learning for Subpixel Neural Tracking

Abstract: Automatically tracing elongated structures, such as axons and blood vessels, is a challenging problem in the field of biomedical imaging, but one with many downstream applications. Real, labelled data is sparse, and existing algorithms either lack robustness to different datasets, or otherwise require significant manual tuning. Here, we instead learn a tracking algorithm in a synthetic environment, and apply it to tracing axons. To do so, we formulate tracking as a reinforcement learning problem, and apply deep reinforcement learning techniques with a continuous action space to learn how to track at the subpixel level. We train our model on simple synthetic data and test it on mouse cortical two-photon microscopy images. Despite the domain gap, our model approaches the performance of a heavily engineered tracker from a standard analysis suite for neuronal microscopy. We show that fine-tuning on real data improves performance, allowing better transfer when real labelled data is available. Finally, we demonstrate that our model’s uncertainty measure—a feature lacking in hand-engineered trackers—corresponds with how well it tracks the structure.

Policy Consolidation for Continual Reinforcement Learning

Abstract: We propose a method for tackling catastrophic forgetting in deep reinforcement learning that is \textit{agnostic} to the timescale of changes in the distribution of experiences, does not require knowledge of task boundaries, and can adapt in \textit{continuously} changing environments. In our \textit{policy consolidation} model, the policy network interacts with a cascade of hidden networks that simultaneously remember the agent's policy at a range of timescales and regularise the current policy by its own history, thereby improving its ability to learn without forgetting. We find that the model improves continual learning relative to baselines on a number of continuous control tasks in single-task, alternating two-task, and multi-agent competitive self-play settings.

Synaptic plasticity onto inhibitory neurons as a mechanism for ocular dominance plasticity

Abstract: Ocular dominance plasticity is a well-documented phenomenon allowing us to study properties of cortical maturation. Understanding this maturation might be an important step towards unravelling how cortical circuits function. However, it is still not fully understood which mechanisms are responsible for the opening and closing of the critical period for ocular dominance and how changes in cortical responsiveness arise after visual deprivation. In this article, we present a theory of ocular dominance plasticity. Following recent experimental work, we propose a framework where a reduction in inhibition is necessary for ocular dominance plasticity in both juvenile and adult animals. In this framework, two ingredients are crucial to observe ocular dominance shifts: a sufficient level of inhibition as well as excitatory-to-inhibitory synaptic plasticity. In our model, the former is responsible for the opening of the critical period, while the latter limits the plasticity in adult animals. Finally, we also provide a possible explanation for the variability in ocular dominance shifts observed in individual neurons and for the counter-intuitive shifts towards the closed eye.

A Unifying Framework for Neuro-Inspired, Data-Driven Detection of Low-Level Auditory Features

Abstract: Our understanding of hearing and speech recognition rests on controlled experiments requiring simple stimuli. However, these stimuli often lack the characteristics of complex sounds such as speech. We propose an approach that combines neural modelling with machine learning to determine relevant low-level auditory features. Our approach bridges the gap between detailed neuronal models that capture specific auditory responses, and research on the statistics of real-world speech data and speech recognition. First, we introduce a feature detection model with a modest number of parameters that is compatible with auditory physiology. In order to objectively determine relevant feature detectors within the model parameter space, the model is tested in a speech classification task, using a simple classifier that approximates the information bottleneck. This framework allows us to determine the best model parameters and their neurophysiological and psychoacoustic implications. We show that our model can capture a variety of well-studied features (such as amplitude modulations and onsets) and allows us to unify concepts from different areas of hearing research. Our approach has various potential applications. Firstly, it could lead to new, testable experimental hypotheses for understanding hearing. Moreover, promising features could be directly applied as a new acoustic front-end for speech recognition systems.