While logistic sigmoid neurons are more biologically plausible than hyperbolic tangent neurons, the latter work better for training multi-layer neural networks. This paper shows that rectifying neurons are an even better model of biological neurons and yield equal or better performance than hyperbolic tangent networks in spite of the hard non-linearity and non-dierentiabil ity

/pdf/deep-sparse-rectifier-neural-networks-hvuorf8vbu.pdf

Deep Sparse Rectifier Neural Networks

The majority of functional neuroscience studies have focused on the brain's response to a task or stimulus. However, the brain is very active even in the absence of explicit input or output. In this Article we review recent studies examining spontaneous fluctuations in the blood oxygen level dependent (BOLD) signal of functional magnetic resonance imaging as a potentially important and revealing manifestation of spontaneous neuronal activity. Although several challenges remain, these studies have provided insight into the intrinsic functional architecture of the brain, variability in behaviour and potential physiological correlates of neurological and psychiatric disease.

/pdf/spontaneous-fluctuations-in-brain-activity-observed-with-32z1nyfc7n.pdf

Spontaneous fluctuations in brain activity observed with functional magnetic resonance imaging.

Prelude. Cycle 1. Introduction. Cycle 2. Structure defines function. Cycle 3. Diversity of cortical functions is provided by inhibition. Cycle 4. Windows on the brain. Cycle 5. A system of rhythms: from simple to complex dynamics. Cycle 6. Synchronization by oscillation. Cycle 7. The brain's default state: self-organized oscillations in rest and sleep. Cycle 8. Perturbation of the default patterns by experience. Cycle 9. The gamma buzz: gluing by oscillations in the waking brain. Cycle 10. Perceptions and actions are brain state-dependent. Cycle 11. Oscillations in the "other cortex:" navigation in real and memory space. Cycle 12. Coupling of systems by oscillations. Cycle 13. The tough problem. References.

Rhythms of the brain

https://www.cell.com/trends/cognitive-sciences/pdf/S1364-6613(05)00242-1.pdf

A mechanism for cognitive dynamics: neuronal communication through neuronal coherence

The peroxisome proliferator-activated receptor-gamma (PPAR-gamma) is a member of the nuclear receptor superfamily of ligand-dependent transcription factors that is predominantly expressed in adipose tissue, adrenal gland and spleen PPAR-gamma has been demonstrated to regulate adipocyte differentiation and glucose homeostasis in response to several structurally distinct compounds, including thiazolidinediones and fibrates Naturally occurring compounds such as fatty acids and the prostaglandin D2 metabolite 15-deoxy-delta prostaglandin J2 (15d-PGJ2) bind to PPAR-gamma and stimulate transcription of target genes Prostaglandin D2 metabolites have not yet been identified in adipose tissue, but are major products of arachidonic-acid metabolism in macrophages, raising the possibility that they might serve as endogenous PPAR-gamma ligands in this cell type Here we show that PPAR-gamma is markedly upregulated in activated macrophages and inhibits the expression of the inducible nitric oxide synthase, gelatinase B and scavenger receptor A genes in response to 15d-PGJ2 and synthetic PPAR-gamma ligands PPAR-gamma inhibits gene expression in part by antagonizing the activities of the transcription factors AP-1, STAT and NF-kappaB These observations suggest that PPAR-gamma and locally produced prostaglandin D2 metabolites are involved in the regulation of inflammatory responses, and raise the possibility that synthetic PPAR-gamma ligands may be of therapeutic value in human diseases such as atherosclerosis and rheumatoid arthritis in which activated macrophages exert pathogenic effects

The peroxisome proliferator-activated receptor-gamma is a negative regulator of macrophage activation

For years we have known that cortical neurons collectively have synchronous or oscillatory patterns of activity, the frequencies and temporal dynamics of which are associated with distinct behavioural states. Although the function of these oscillations has remained obscure, recent experimental and theoretical results indicate that correlated fluctuations might be important for cortical processes, such as attention, that control the flow of information in the brain.

/pdf/correlated-neuronal-activity-and-the-flow-of-neural-38xcnzfz6d.pdf

Correlated neuronal activity and the flow of neural information.

Recent studies combining psychophysical and neurophysiological experiments in behaving monkeys have provided new insights into how several cortical areas integrate efforts to solve a vibrotactile discrimination task. In particular, these studies have addressed how neural codes are related to perception, working memory and decision making in this model. The primary somatosensory cortex drives higher cortical areas where past and current sensory information are combined, such that a comparison of the two evolves into a behavioural decision. These and other observations in visual tasks indicate that decisions emerge from highly-distributed processes in which the details of a scheduled motor plan are gradually specified by sensory information.

/pdf/flutter-discrimination-neural-codes-perception-memory-and-19upac464k.pdf

Flutter discrimination: neural codes, perception, memory and decision making.

In a number of systems including wind detection in the cricket, visual motion perception and coding of arm movement direction in the monkey and place cell response to position in the rat hippocampus, firing rates in a population of tuned neurons are correlated with a vector quantity. We examine and compare several methods that allow the coded vector to be reconstructed from measured firing rates. In cases where the neuronal tuning curves resemble cosines, linear reconstruction methods work as well as more complex statistical methods requiring more detailed information about the responses of the coding neurons. We present a new linear method, the optimal linear estimator (OLE), that on average provides the best possible linear reconstruction. This method is compared with the more familiar vector method and shown to produce more accurate reconstructions using far fewer recorded neurons.

Vector reconstruction from firing rates

Cortical neurons are typically driven by thousands of synaptic inputs. The arrival of a spike from one input may or may not be correlated with the arrival of other spikes from different inputs. How does this interdependence alter the probability that the postsynaptic neuron will fire? We constructed a simple random walk model in which the membrane potential of a target neuron fluctuates stochastically, driven by excitatory and inhibitory spikes arriving at random times. An analytic expression was derived for the mean output firing rate as a function of the firing rates and pairwise correlations of the inputs. This stochastic model made three quantitative predictions. (1) Correlations between pairs of excitatory or inhibitory inputs increase the fluctuations in synaptic drive, whereas correlations between excitatory‐inhibitory pairs decrease them. (2) When excitation and inhibition are fully balanced (the mean net synaptic drive is zero), firing is caused by the fluctuations only. (3) In the balanced case, firing is irregular. These theoretical predictions were in excellent agreement with simulations of an integrate-and-fire neuron that included multiple conductances and received hundreds of synaptic inputs. The results show that, in the balanced regime, weak correlations caused by signals shared among inputs may have a multiplicative effect on the input-output rate curve of a postsynaptic neuron, i.e. they may regulate its gain; in the unbalanced regime, correlations may increase firing probability mainly around threshold, when output rate is low; and in all cases correlations are expected to increase the variability of the output spike train.

https://www.jneurosci.org/content/jneuro/20/16/6193.full.pdf

Impact of Correlated Synaptic Input on Output Firing Rate and Variability in Simple Neuronal Models

The sensation of flutter is produced when mechanical vibrations in the range of 5-50Hz are applied to the skin. A flutter stimulus activates neurons in the primary somatosensory cortex (S1) that somatotopically map to the site of stimulation. A subset of these neurons-those with quickly adapting properties, associated with Meissner's corpuscles-are strongly entrained by periodic flutter vibrations, firing with a probability that oscillates at the input frequency. Hence, quickly adapting neurons provide a dynamic representation of such flutter stimuli. However, are these neurons directly involved in the perception of flutter? Here we investigate this in monkeys trained to discriminate the difference in frequency between two flutter stimuli delivered sequentially on the fingertips. Microelectrodes were inserted into area 3b of S1 and the second stimulus was substituted with a train of injected current pulses. Animals reliably indicated whether the frequency of the second (electrical) signal was higher or lower than that of the first (mechanical) signal, even though both frequencies changed from trial to trial. Almost identical results were obtained with periodic and aperiodic stimuli of equal average frequencies. Thus, the quickly adapting neurons in area 3b activate the circuit leading to the perception of flutter. Furthermore, as far as can be psychophysically quantified during discrimination, the neural code underlying the sensation of flutter can be finely manipulated, to the extent that the behavioural responses produced by natural and artificial stimuli are indistinguishable.

Emilio Salinas

Papers

Correlated neuronal activity and the flow of neural information.

Flutter discrimination: neural codes, perception, memory and decision making.

Vector reconstruction from firing rates

Impact of Correlated Synaptic Input on Output Firing Rate and Variability in Simple Neuronal Models

Somatosensory discrimination based on cortical microstimulation