Journal ArticleDOI
Learning in neural networks with material synapses
Daniel J. Amit,Stefano Fusi +1 more
Reads0
Chats0
TLDR
It is shown that a network with synapses that have two stable states can dynamically learn with optimal storage efficiency, be a palimpsest, and maintain its (associative) memory for an indefinitely long time provided the coding level is low and depression is equilibrated against potentiation.Abstract:
We discuss the long term maintenance of acquired memory in synaptic connections of a perpetually learning electronic device. This is affected by ascribing each synapse a finite number of stable states in which it can maintain for indefinitely long periods. Learning uncorrelated stimuli is expressed as a stochastic process produced by the neural activities on the synapses. In several interesting cases the stochastic process can be analyzed in detail, leading to a clarification of the performance of the network, as an associative memory, during the process of uninterrupted learning. The stochastic nature of the process and the existence of an asymptotic distribution for the synaptic values in the network imply generically that the memory is a palimpsest but capacity is as low as log N for a network of N neurons. The only way we find for avoiding this tight constraint is to allow the parameters governing the learning process (the coding level of the stimuli; the transition probabilities for potentiation and depression and the number of stable synaptic levels) to depend on the number of neurons. It is shown that a network with synapses that have two stable states can dynamically learn with optimal storage efficiency, be a palimpsest, and maintain its (associative) memory for an indefinitely long time provided the coding level is low and depression is equilibrated against potentiation. We suggest that an option so easily implementable in material devices would not have been overlooked by biology. Finally we discuss the stochastic learning on synapses with variable number of stable synaptic states.read more
Citations
More filters
Journal ArticleDOI
The Theory of Stochastic Processes. By D. R. Cox and H. D. Miller. Pp. x, 398. 70s. (Methuen)
Journal ArticleDOI
Model of global spontaneous activity and local structured activity during delay periods in the cerebral cortex.
Daniel J Amit,Nicolas Brunel +1 more
TL;DR: In this paper, the authors investigate self-sustaining stable states (attractors) in networks of integrate-and-fire neurons and study the effect of learning in a local module, expressed in synaptic modifications in specific populations of synapses.
Journal ArticleDOI
A VLSI array of low-power spiking neurons and bistable synapses with spike-timing dependent plasticity
TL;DR: In this article, a mixed-mode analog/digital VLSI device comprising an array of leaky integrate-and-fire (I&F) neurons, adaptive synapses with spike-timing dependent plasticity, and an asynchronous event based communication infrastructure is presented.
Journal ArticleDOI
Cascade models of synaptically stored memories.
TL;DR: It is suggested that memory storage requires synapses with multiple states exhibiting dynamics over a wide range of timescales, and the model constructed here combines high levels of memory storage with long retention times and significantly outperforms alternative models.
Journal ArticleDOI
The Hebbian paradigm reintegrated: Local reverberations as internal representations
TL;DR: Cognitive and neurophysiological predictions are made, many following directly from the language used to describe the activity in the experimental delay period, others from the details of how the model captures the properties of the internal representations.
References
More filters
Journal ArticleDOI
Neural networks and physical systems with emergent collective computational abilities
TL;DR: A model of a system having a large number of simple equivalent components, based on aspects of neurobiology but readily adapted to integrated circuits, produces a content-addressable memory which correctly yields an entire memory from any subpart of sufficient size.
Journal ArticleDOI
The Theory of Stochastic Processes. By D. R. Cox and H. D. Miller. Pp. x, 398. 70s. (Methuen)
Journal ArticleDOI
Non-holographic associative memory
TL;DR: The features of a hologram that commend it as a model of associative memory can be improved on by other devices.
Journal ArticleDOI
Neuronal correlate of visual associative long-term memory in the primate temporal cortex.
TL;DR: The results indicate that the selectivity acquired by cells in the anterior ventral temporal cortex of monkeys represents a neuronal correlate of the associative long-term memory of pictures.
Journal ArticleDOI
Statistical mechanics of neural networks near saturation
TL;DR: In this paper, the Hopfield model of a neural network is studied near its saturation, i.e., when the number p of stored patterns increases with the size of the network, as p = αN.