https://www2.econ.iastate.edu/tesfatsi/DeepLearningInNeuralNetworksOverview.JSchmidhuber2015.pdf

Deep learning in neural networks

Clocks tick, bridges and skyscrapers vibrate, neuronal networks oscillate. Are neuronal oscillations an inevitable by-product, similar to bridge vibrations, or an essential part of the brain’s design? Mammalian cortical neurons form behavior-dependent oscillating networks of various sizes, which span five orders of magnitude in frequency. These oscillations are phylogenetically preserved, suggesting that they are functionally relevant. Recent findings indicate that network oscillations bias input selection, temporally link neurons into assemblies, and facilitate synaptic plasticity, mechanisms that cooperatively support temporal representation and long-term consolidation of information.

/pdf/neuronal-oscillations-in-cortical-networks-3d909qh5dm.pdf

Neuronal Oscillations in Cortical Networks

Prelude. Cycle 1. Introduction. Cycle 2. Structure defines function. Cycle 3. Diversity of cortical functions is provided by inhibition. Cycle 4. Windows on the brain. Cycle 5. A system of rhythms: from simple to complex dynamics. Cycle 6. Synchronization by oscillation. Cycle 7. The brain's default state: self-organized oscillations in rest and sleep. Cycle 8. Perturbation of the default patterns by experience. Cycle 9. The gamma buzz: gluing by oscillations in the waking brain. Cycle 10. Perceptions and actions are brain state-dependent. Cycle 11. Oscillations in the "other cortex:" navigation in real and memory space. Cycle 12. Coupling of systems by oscillations. Cycle 13. The tough problem. References.

Rhythms of the brain

The ability to find one's way depends on neural algorithms that integrate information about place, distance and direction, but the implementation of these operations in cortical microcircuits is poorly understood. Here we show that the dorsocaudal medial entorhinal cortex (dMEC) contains a directionally oriented, topographically organized neural map of the spatial environment. Its key unit is the 'grid cell', which is activated whenever the animal's position coincides with any vertex of a regular grid of equilateral triangles spanning the surface of the environment. Grids of neighbouring cells share a common orientation and spacing, but their vertex locations (their phases) differ. The spacing and size of individual fields increase from dorsal to ventral dMEC. The map is anchored to external landmarks, but persists in their absence, suggesting that grid cells may be part of a generalized, path-integration-based map of the spatial environment.

/pdf/microstructure-of-a-spatial-map-in-the-entorhinal-cortex-511e186u79.pdf

Microstructure of a spatial map in the entorhinal cortex

Interneurons of the hippocampus

O'Keefe and Recce [1993] Hippocampus 3:317-330 described an interaction between the hippocampal theta rhythm and the spatial firing of pyramidal cells in the CA1 region of the rat hippocampus: they found that a cell's spike activity advances to earlier phases of the theta cycle as the rat passes through the cell's place field. The present study makes use of large-scale parallel recordings to clarify and extend this finding in several ways: 1) Most CA1 pyramidal cells show maximal activity at the same phase of the theta cycle. Although individual units exhibit deeper modulation, the depth of modulation of CA1 population activity is about 50%. The peak firing of inhibitory interneurons in CA1 occurs about 60 degrees in advance of the peak firing of pyramidal cells, but different interneurons vary widely in their peak phases. 2) The first spikes, as the rat enters a pyramidal cell's place field, come 90 degrees-120 degrees after the phase of maximal pyramidal cell population activity, near the phase where inhibition is least. 3) The phase advance is typically an accelerating, rather than linear, function of position within the place field. 4) These phenomena occur both on linear tracks and in two-dimensional environments where locomotion is not constrained to specific paths. 5) In two-dimensional environments, place-related firing is more spatially specific during the early part of the theta cycle than during the late part. This is also true, to a lesser extent, on a linear track. Thus, spatial selectivity waxes and wanes over the theta cycle. 6) Granule cells of the fascia dentata are also modulated by theta. The depth of modulation for the granule cell population approaches 100%, and the peak activity of the granule cell population comes about 90 degrees earlier in the theta cycle than the peak firing of CA1 pyramidal cells. 7) Granule cells, like pyramidal cells, show robust phase precession. 8) Cross-correlation analysis shows that portions of the temporal sequence of CA1 pyramidal cell place fields are replicated repeatedly within individual theta cycles, in highly compressed form. The compression ratio can be as much as 10:1. These findings indicate that phase precession is a very robust effect, distributed across the entire hippocampal population, and that it is likely to be inherited from the fascia dentata or an earlier stage in the hippocampal circuit, rather than generated intrinsically within CA1. It is hypothesized that the compression of temporal sequences of place fields within individual theta cycles permits the use of long-term potentiation for learning of sequential structure, thereby giving a temporal dimension to hippocampal memory traces.

Theta phase precession in hippocampal neuronal populations and the compression of temporal sequences.

The correlated activity of rat hippocampal pyramidal cells during sleep reflects the activity of those cells during earlier spatial exploration. Now the patterns of activity during sleep have also been found to reflect the order in which the cells fired during spatial exploration. This relation was reliably stronger for sleep after the behavioral session than before it; thus, the activity during sleep reflects changes produced by experience. This memory for temporal order of neuronal firing could be produced by an interaction between the temporal integration properties of long-term potentiation and the phase shifting of spike activity with respect to the hippocampal theta rhythm.

https://science.sciencemag.org/content/sci/271/5257/1870.full.pdf

Replay of Neuronal Firing Sequences in Rat Hippocampus During Sleep Following Spatial Experience

Hippocampal 'place' cells and the head-direction cells of the dorsal presubiculum and related neocortical and thalamic areas appear to be part of a preconfigured network that generates an abstract internal representation of two-dimensional space whose metric is self-motion. It appears that viewpoint-specific visual information (e.g. landmarks) becomes secondarily bound to this structure by associative learning. These associations between landmarks and the preconfigured path integrator serve to set the origin for path integration and to correct for cumulative error. In the absence of familiar landmarks, or in darkness without a prior spatial reference, the system appears to adopt an initial reference for path integration independently of external cues. A hypothesis of how the path integration system may operate at the neuronal level is proposed.

/pdf/deciphering-the-hippocampal-polyglot-the-hippocampus-as-a-3db59brwzy.pdf

Deciphering the hippocampal polyglot: the hippocampus as a path integration system

When rats forage for randomly dispersed food in a high walled cylinder the firing of their hippocampal "place" cells exhibits little dependence on the direction faced by the rat. On radial arm mazes and similar tasks, place cells are strongly directionally selective within their fields. These tasks differ in several respects, including the visual environment, configuration of the traversable space, motor behavior (e.g., linear and angular velocities), and behavioral context (e.g., presence of specific, consistent goal locations within the environment). The contributions of these factors to spatial and directional tuning of hippocampal neurons was systematically examined in rats performing several tasks in either an enriched or a sparse visual environment, and on different apparati. Place fields were more spatially and directionally selective on a radial maze than on an open, circular platform, regardless of the visual environment. On the platform, fields were more directional when the rat searched for food at fixed locations, in a stereotypic and directed manner, than when the food was scattered randomly. Thus, it seems that place fields are more directional when the animal is planning or following a route between points of special significance. This might be related to the spatial focus of the rat's attention (e.g., a particular reference point). Changing the behavioral task was also accompanied by a change in firing location in about one-third of the cells. Thus, hippocampal neuronal activity appears to encode a complex interaction between locations, their significance and the behaviors the rat is called upon to execute.

https://www.jneurosci.org/content/jneuro/15/11/7079.full.pdf

Interactions between location and task affect the spatial and directional firing of hippocampal neurons

Information theory is used to derive a simple formula for the amount of information conveyed by the firing rate of a neuron about any experimentally measured variable or combination of variables (e.g. running speed, head direction, location of the animal, etc.). The derivation treats the cell as a communication channel whose input is the measured variable and whose output is the cell's spike train. Applying the formula, we find systematic differences in the information content of hippocampal "place cells" in different experimental conditions.

/pdf/an-information-theoretic-approach-to-deciphering-the-q6hapi2fcw.pdf

William E. Skaggs

Papers

Theta phase precession in hippocampal neuronal populations and the compression of temporal sequences.

Replay of Neuronal Firing Sequences in Rat Hippocampus During Sleep Following Spatial Experience

Deciphering the hippocampal polyglot: the hippocampus as a path integration system

Interactions between location and task affect the spatial and directional firing of hippocampal neurons

An Information-Theoretic Approach to Deciphering the Hippocampal Code