Self-organization and associative memory

TLDR: In this paper, the problem of infinite-state memory is addressed in the context of biological memory using an adaptive filtering approach based on the classical laws of association, which is used for the purpose and nature of the biological memory.

Abstract: 1 Various Aspects of Memory- 11 On the Purpose and Nature of Biological Memory- 111 Some Fundamental Concepts- 112 The Classical Laws of Association- 113 On Different Levels of Modelling- 12 Questions Concerning the Fundamental Mechanisms of Memory- 121 Where Do the Signals Relating to Memory Act Upon?- 122 What Kind of Encoding is Used for Neural Signals?- 123 What are the Variable Memory Elements?- 124 How are Neural Signals Addressed in Memory?- 13 Elementary Operations Implemented by Associative Memory- 131 Associative Recall- 132 Production of Sequences from the Associative Memory- 133 On the Meaning of Background and Context- 14 More Abstract Aspects of Memory- 141 The Problem of Infinite-State Memory- 142 Invariant Representations- 143 Symbolic Representations- 144 Virtual Images- 145 The Logic of Stored Knowledge- 2 Pattern Mathematics- 21 Mathematical Notations and Methods- 211 Vector Space Concepts- 212 Matrix Notations- 213 Further Properties of Matrices- 214 Matrix Equations- 215 Projection Operators- 216 On Matrix Differential Calculus- 22 Distance Measures for Patterns- 221 Measures of Similarity and Distance in Vector Spaces- 222 Measures of Similarity and Distance Between Symbol Strings- 223 More Accurate Distance Measures for Text- 3 Classical Learning Systems- 31 The Adaptive Linear Element (Adaline)- 311 Description of Adaptation by the Stochastic Approximation- 32 The Perceptron- 33 The Learning Matrix- 34 Physical Realization of Adaptive Weights- 341 Perceptron and Adaline- 342 Classical Conditioning- 343 Conjunction Learning Switches- 344 Digital Representation of Adaptive Circuits- 345 Biological Components- 4 A New Approach to Adaptive Filters- 41 Survey of Some Necessary Functions- 42 On the "Transfer Function" of the Neuron- 43 Models for Basic Adaptive Units- 431 On the Linearization of the Basic Unit- 432 Various Cases of Adaptation Laws- 433 Two Limit Theorems- 434 The Novelty Detector- 44 Adaptive Feedback Networks- 441 The Autocorrelation Matrix Memory- 442 The Novelty Filter- 5 Self-Organizing Feature Maps- 51 On the Feature Maps of the Brain- 52 Formation of Localized Responses by Lateral Feedback- 53 Computational Simplification of the Process- 531 Definition of the Topology-Preserving Mapping- 532 A Simple Two-Dimensional Self-Organizing System- 54 Demonstrations of Simple Topology-Preserving Mappings- 541 Images of Various Distributions of Input Vectors- 542 "The Magic TV"- 543 Mapping by a Feeler Mechanism- 55 Tonotopic Map- 56 Formation of Hierarchical Representations- 561 Taxonomy Example- 562 Phoneme Map- 57 Mathematical Treatment of Self-Organization- 571 Ordering of Weights- 572 Convergence Phase- 58 Automatic Selection of Feature Dimensions- 6 Optimal Associative Mappings- 61 Transfer Function of an Associative Network- 62 Autoassociative Recall as an Orthogonal Projection- 621 Orthogonal Projections- 622 Error-Correcting Properties of Projections- 63 The Novelty Filter- 631 Two Examples of Novelty Filter- 632 Novelty Filter as an Autoassociative Memory- 64 Autoassociative Encoding- 641 An Example of Autoassociative Encoding- 65 Optimal Associative Mappings- 651 The Optimal Linear Associative Mapping- 652 Optimal Nonlinear Associative Mappings- 66 Relationship Between Associative Mapping, Linear Regression, and Linear Estimation- 661 Relationship of the Associative Mapping to Linear Regression- 662 Relationship of the Regression Solution to the Linear Estimator- 67 Recursive Computation of the Optimal Associative Mapping- 671 Linear Corrective Algorithms- 672 Best Exact Solution (Gradient Projection)- 673 Best Approximate Solution (Regression)- 674 Recursive Solution in the General Case- 68 Special Cases- 681 The Correlation Matrix Memory- 682 Relationship Between Conditional Averages and Optimal Estimator- 7 Pattern Recognition- 71 Discriminant Functions- 72 Statistical Formulation of Pattern Classification- 73 Comparison Methods- 74 The Subspace Methods of Classification- 741 The Basic Subspace Method- 742 The Learning Subspace Method (LSM)- 75 Learning Vector Quantization- 76 Feature Extraction- 77 Clustering- 771 Simple Clustering (Optimization Approach)- 772 Hierarchical Clustering (Taxonomy Approach)- 78 Structural Pattern Recognition Methods- 8 More About Biological Memory- 81 Physiological Foundations of Memory- 811 On the Mechanisms of Memory in Biological Systems- 812 Structural Features of Some Neural Networks- 813 Functional Features of Neurons- 814 Modelling of the Synaptic Plasticity- 815 Can the Memory Capacity Ensue from Synaptic Changes?- 82 The Unified Cortical Memory Model- 821 The Laminar Network Organization- 822 On the Roles of Interneurons- 823 Representation of Knowledge Over Memory Fields- 824 Self-Controlled Operation of Memory- 83 Collateral Reading- 831 Physiological Results Relevant to Modelling- 832 Related Modelling- 9 Notes on Neural Computing- 91 First Theoretical Views of Neural Networks- 92 Motives for the Neural Computing Research- 93 What Could the Purpose of the Neural Networks be?- 94 Definitions of Artificial "Neural Computing" and General Notes on Neural Modelling- 95 Are the Biological Neural Functions Localized or Distributed?- 96 Is Nonlinearity Essential to Neural Computing?- 97 Characteristic Differences Between Neural and Digital Computers- 971 The Degree of Parallelism of the Neural Networks is Still Higher than that of any "Massively Parallel" Digital Computer- 972 Why the Neural Signals Cannot be Approximated by Boolean Variables- 973 The Neural Circuits do not Implement Finite Automata- 974 Undue Views of the Logic Equivalence of the Brain and Computers on a High Level- 98 "Connectionist Models"- 99 How can the Neural Computers be Programmed?- 10 Optical Associative Memories- 101 Nonholographic Methods- 102 General Aspects of Holographic Memories- 103 A Simple Principle of Holographic Associative Memory- 104 Addressing in Holographic Memories- 105 Recent Advances of Optical Associative Memories- Bibliography on Pattern Recognition- References