Non-holographic associative memory
TL;DR: The features of a hologram that commend it as a model of associative memory can be improved on by other devices.
Abstract: The features of a hologram that commend it as a model of associative memory can be improved on by other devices.
Citations
More filters
•
01 Jan 1997
TL;DR: An Introduction to Nueral Networks will be warmly welcomed by a wide readership seeking an authoritative treatment of this key subject without an intimidating level of mathematics in the presentation.
Abstract: From the Publisher:
An Introduction to Nueral Networks will be warmly welcomed by a wide readership seeking an authoritative treatment of this key subject without an intimidating level of mathematics in the presentation.
2,135 citations
••
01 Jan 1992TL;DR: A speculative neurophysiological model illustrating how the backpropagation neural network architecture might plausibly be implemented in the mammalian brain for corticocortical learning between nearby regions of the cerebral cortex is presented.
Abstract: Publisher Summary This chapter presents a survey of the elementary theory of the basic backpropagation neural network architecture, covering the areas of architectural design, performance measurement, function approximation capability, and learning. The survey includes a formulation of the backpropagation neural network architecture to make it a valid neural network and a proof that the backpropagation mean squared error function exists and is differentiable. Also included in the survey is a theorem showing that any L2 function can be implemented to any desired degree of accuracy with a three-layer backpropagation neural network. An appendix presents a speculative neurophysiological model illustrating the way in which the backpropagation neural network architecture might plausibly be implemented in the mammalian brain for corticocortical learning between nearby regions of cerebral cortex. One of the crucial decisions in the design of the backpropagation architecture is the selection of a sigmoidal activation function.
1,729 citations
••
01 Jan 1989
TL;DR: A speculative neurophysiological model illustrating how the backpropagation neural network architecture might plausibly be implemented in the mammalian brain for corticocortical learning between nearby regions of the cerebral cortex is presented.
Abstract: The author presents a survey of the basic theory of the backpropagation neural network architecture covering architectural design, performance measurement, function approximation capability, and learning. The survey includes previously known material, as well as some new results, namely, a formulation of the backpropagation neural network architecture to make it a valid neural network (past formulations violated the locality of processing restriction) and a proof that the backpropagation mean-squared-error function exists and is differentiable. Also included is a theorem showing that any L/sub 2/ function from (0, 1)/sup n/ to R/sup m/ can be implemented to any desired degree of accuracy with a three-layer backpropagation neural network. The author presents a speculative neurophysiological model illustrating how the backpropagation neural network architecture might plausibly be implemented in the mammalian brain for corticocortical learning between nearby regions of the cerebral cortex. >
1,668 citations
••
TL;DR: Recent physiological recordings from sensory neurons have indicated that sparse coding could be a ubiquitous strategy employed in several different modalities across different organisms.
1,414 citations
••
TL;DR: A simple neuronal model capable of superimposing multiple memory traces within the same matrix of connections is outlined, and the correspondence between such models and the properties of LTE in the context of the hippocampal circuitry in which it occurs is considered.
1,254 citations
References
More filters
••
TL;DR: The theory of three-dimensional optical storage lends support to Beurle’s proposed mechanism of information storage in the brain, and makes three- dimensional storage very suitable for associative memories.
Abstract: In photography, information is stored in a medium which is essentially two-dimensional. Three-dimensional optical storage is possible in semitransparent colored materials, like alkali halides with color centers. With the use of coherent light sources, like lasers, large amounts of information can be stored in the volume, and retrieved with little interference. The storage of information is accomplished by the formation of interference patterns between each two plane parallel waves. This paper develops the theory of this form of storage. It turns out that the information storage capacity is as if every little cube with sides equal to the wavelength of light acts as an independent information storage cell, and the essential noise in recovering this information is only the statistical fluctuation in the number of color centers in such a cube. The storage capacity is therefore of the order of 1012–1013 bits per cm3. The main property of this way of information storage is the appearance of a “ghost image,” partly but not completely analogous to the one described previously. This property makes three-dimensional storage very suitable for associative memories. The theory lends support to Beurle’s proposed mechanism of information storage in the brain.
565 citations
••
TL;DR: It is shown that there is at least one other mathematical model, which is considered to be a closer analogue of holography, because it operates with triple products of the temporal function to be recognized or recalled.
Abstract: PROFESSOR LONGUET-HIGGINS1 has called attention to the remarkable property of human memory of recognizing and recalling long sequences of which at first only a small fraction is consciously remembered, and he has devised a most interesting, physically realizable mathematical model which achieves just this. He has rightly called this a temporal analogue of holography. I wish to show that there is at least one other mathematical model, which I consider to be a closer analogue of holography, because it operates with triple products of the temporal function to be recognized or recalled.
74 citations
••
TL;DR: The modified static system described here is an alternative to Longuet-Higgins' model and takes account of the weaknesses of the nervous system as well as of its powers to imitate human memory with models.
Abstract: THE static model of temporal recall which I recently proposed1 as an alternative to Longuet-Higgins' model2 is unsatisfactory in one respect. From a fragment which is remembered it can recall earlier parts of the whole sequence. This is unlike human memory which can readily recall later parts, but earlier parts only with great difficulty, if at all, especially if the sequence has been memorized by rote. If we want to imitate human memory with models, we must take account of the weaknesses of the nervous system as well as of its powers. Longuet-Higgins' model possesses this property, and so does the modified static system described here.
72 citations
••
TL;DR: It is argued that if the conditioning and memory mechanisms of the nervous system store information by means of modifiable synapses, some of these must be of class B or C, and if the models are to be given the property of extinction, classical conditioning requires modifiablesynapses of classes A and B or of class C.
Abstract: Modifiable synapses are analysed theoretically. It is proved that they fall into three classes (called A, B and C), such that any two members of the same class can, with the aid of non-remembering elements that perform simple logical operations, replace one another in any net; but a member of class A cannot replace one of class B or C. A member of class B can replace one of class A or C only if non-logical elements, for example noise generators, are included in the net. Models are designed that use modifiable synapses and show the principal features of classical and operant conditioning. If the models are to be given the property of extinction, classical conditioning requires modifiable synapses of classes A and B or of class C, and operant conditioning of class B or C. It is argued that if the conditioning and memory mechanisms of the nervous system store information by means of modifiable synapses, some of these must be of class B or C.
53 citations