Showing papers by "Kevin Gurney published in 2015"

A New Framework for Cortico-Striatal Plasticity: Behavioural Theory Meets In Vitro Data at the Reinforcement-Action Interface

Abstract: Operant learning requires that reinforcement signals interact with action representations at a suitable neural interface. Much evidence suggests that this occurs when phasic dopamine, acting as a reinforcement prediction error, gates plasticity at cortico-striatal synapses, and thereby changes the future likelihood of selecting the action(s) coded by striatal neurons. But this hypothesis faces serious challenges. First, cortico-striatal plasticity is inexplicably complex, depending on spike timing, dopamine level, and dopamine receptor type. Second, there is a credit assignment problem—action selection signals occur long before the consequent dopamine reinforcement signal. Third, the two types of striatal output neuron have apparently opposite effects on action selection. Whether these factors rule out the interface hypothesis and how they interact to produce reinforcement learning is unknown. We present a computational framework that addresses these challenges. We first predict the expected activity changes over an operant task for both types of action-coding striatal neuron, and show they co-operate to promote action selection in learning and compete to promote action suppression in extinction. Separately, we derive a complete model of dopamine and spike-timing dependent cortico-striatal plasticity from in vitro data. We then show this model produces the predicted activity changes necessary for learning and extinction in an operant task, a remarkable convergence of a bottom-up data-driven plasticity model with the top-down behavioural requirements of learning theory. Moreover, we show the complex dependencies of cortico-striatal plasticity are not only sufficient but necessary for learning and extinction. Validating the model, we show it can account for behavioural data describing extinction, renewal, and reacquisition, and replicate in vitro experimental data on cortico-striatal plasticity. By bridging the levels between the single synapse and behaviour, our model shows how striatum acts as the action-reinforcement interface.

Decision-making and action selection in insects: inspiration from vertebrate-based theories

Abstract: Effective decision-making, one of the most crucial functions of the brain, entails the analysis of sensory information and the selection of appropriate behavior in response to stimuli. Here, we consider the current state of knowledge on the mechanisms of decision-making and action selection in the insect brain, with emphasis on the olfactory processing system. Theoretical and computational models of decision-making emphasize the importance of using inhibitory connections to couple evidence-accumulating pathways; this coupling allows for effective discrimination between competing alternatives and thus enables a decision maker to reach a stable unitary decision. Theory also shows that the coupling of pathways can be implemented using a variety of different mechanisms and vastly improves the performance of decision-making systems. The vertebrate basal ganglia appear to resolve stable action selection by being a point of convergence for multiple excitatory and inhibitory inputs such that only one possible response is selected and all other alternatives are suppressed. Similar principles appear to operate within the insect brain. The insect lateral protocerebrum (LP) serves as a point of convergence for multiple excitatory and inhibitory channels of olfactory information to effect stable decision and action selection, at least for olfactory information. The LP is a rather understudied region of the insect brain, yet this premotor region may be key to effective resolution of action section. We argue that it may be beneficial to use models developed to explore the operation of the vertebrate brain as inspiration when considering action selection in the invertebrate domain. Such an approach may facilitate the proposal of new hypotheses and furthermore frame experimental studies for how decision-making and action selection might be achieved in insects.

Probabilistic Decision Making with Spikes: From ISI Distributions to Behaviour via Information Gain.

Abstract: Computational theories of decision making in the brain usually assume that sensory 'evidence' is accumulated supporting a number of hypotheses, and that the first accumulator to reach threshold triggers a decision in favour of its associated hypothesis. However, the evidence is often assumed to occur as a continuous process whose origins are somewhat abstract, with no direct link to the neural signals - action potentials or 'spikes' - that must ultimately form the substrate for decision making in the brain. Here we introduce a new variant of the well-known multi-hypothesis sequential probability ratio test (MSPRT) for decision making whose evidence observations consist of the basic unit of neural signalling - the inter-spike interval (ISI) - and which is based on a new form of the likelihood function. We dub this mechanism s-MSPRT and show its precise form for a range of realistic ISI distributions with positive support. In this way we show that, at the level of spikes, the refractory period may actually facilitate shorter decision times, and that the mechanism is robust against poor choice of the hypothesized data distribution. We show that s-MSPRT performance is related to the Kullback-Leibler divergence (KLD) or information gain between ISI distributions, through which we are able to link neural signalling to psychophysical observation at the behavioural level. Thus, we find the mean information needed for a decision is constant, thereby offering an account of Hick's law (relating decision time to the number of choices). Further, the mean decision time of s-MSPRT shows a power law dependence on the KLD offering an account of Pieron's law (relating reaction time to stimulus intensity). These results show the foundations for a research programme in which spike train analysis can be made the basis for predictions about behavior in multi-alternative choice tasks.

A neural model of the optomotor system accounts for ordered responses to decreasing stimulus spatial frequencies

Abstract: In insects the optomotor response produces a motor action to compensate for unintended body rotation. The response is generally modeled as a Reichardt-Hassenstein (HSD) or Barlow-Levick (BL) correlation detector, as anatomical and physiological studies in Drosophila melanogaster have demonstrated consistent neural pathways and responses in the insect brain [1]. Recordings from the descending neurons carrying the optomotor response signal in honeybees indicate an ordering effect for different stimulus spatial frequencies, with a greater response with decreasing frequency [2] (see Figure ​Figure1A),1A), which is not accounted for by HSD or BL correlation detectors. Figure 1 A. Cartoon of ordering effect indicated by honeybee descending neuron responses. Spatial frequency decreases from blue to green. B. Model diagram showing annealed synapses (coloured, same colours indicate same synaptic conductance). C. Slice of annealing ... We present a model in the SpineML format of the optomotor system, using Izhikevich point neurons tuned to match the respective physiological responses, which is shown in Figure ​Figure1B.1B. To examine if the model reproduces the ordering effect found in the honeybee we performed simulated annealing on four conductance values in the model, as shown in Figure ​Figure1.1. The objective function is designed to maximize: correct ordering; a 10Hz maximum response; and contrast between responses to forward and reverse motion. A. The data was imported into a commercial software package (MATLAB 7.14, The MathWorks Inc., Natick, MA, 2012) for analysis and interpolated onto a 414 grid. A 3D slice of this 4D grid can be seen in Figure ​Figure1C.1C. Spatial frequencies of 32.7, 18.9 and 9.5 Hz are used. A stable region in which a high value of the objective function, and thus correct spatial frequency ordering, could be obtained was found. In the stable region the onset pathway activity is low, leading to offset activity dominating. A RHD using the model up to the Medulla does not show correct ordering.