Publisher Summary This chapter provides an account of different neural network architectures for pattern recognition. A neural network consists of several simple processing elements called neurons. Each neuron is connected to some other neurons and possibly to the input nodes. Neural networks provide a simple computing paradigm to perform complex recognition tasks in real time. The chapter categorizes neural networks into three types: single-layer networks, multilayer feedforward networks, and feedback networks. It discusses the gradient descent and the relaxation method as the two underlying mathematical themes for deriving learning algorithms. A lot of research activity is centered on learning algorithms because of their fundamental importance in neural networks. The chapter discusses two important directions of research to improve learning algorithms: the dynamic node generation, which is used by the cascade correlation algorithm; and designing learning algorithms where the choice of parameters is not an issue. It closes with the discussion of performance and implementation issues.

Neural Networks for Pattern Recognition

Sleep is characterized by synchronized events in billions of synaptically coupled neurons in thalamocortical systems. The activation of a series of neuromodulatory transmitter systems during awakening blocks low-frequency oscillations, induces fast rhythms, and allows the brain to recover full responsiveness. Analysis of cortical and thalamic networks at many levels, from molecules to single neurons to large neuronal assemblies, with a variety of techniques, ranging from intracellular recordings in vivo and in vitro to computer simulations, is beginning to yield insights into the mechanisms of the generation, modulation, and function of brain oscillations.

https://science.sciencemag.org/content/sci/262/5134/679.full.pdf

Thalamocortical oscillations in the sleeping and aroused brain

Introduction to neural networks.

Over more than a century of research has established the fact that sleep benefits the retention of memory. In this review we aim to comprehensively cover the field of "sleep and memory" research by providing a historical perspective on concepts and a discussion of more recent key findings. Whereas initial theories posed a passive role for sleep enhancing memories by protecting them from interfering stimuli, current theories highlight an active role for sleep in which memories undergo a process of system consolidation during sleep. Whereas older research concentrated on the role of rapid-eye-movement (REM) sleep, recent work has revealed the importance of slow-wave sleep (SWS) for memory consolidation and also enlightened some of the underlying electrophysiological, neurochemical, and genetic mechanisms, as well as developmental aspects in these processes. Specifically, newer findings characterize sleep as a brain state optimizing memory consolidation, in opposition to the waking brain being optimized for encoding of memories. Consolidation originates from reactivation of recently encoded neuronal memory representations, which occur during SWS and transform respective representations for integration into long-term memory. Ensuing REM sleep may stabilize transformed memories. While elaborated with respect to hippocampus-dependent memories, the concept of an active redistribution of memory representations from networks serving as temporary store into long-term stores might hold also for non-hippocampus-dependent memory, and even for nonneuronal, i.e., immunological memories, giving rise to the idea that the offline consolidation of memory during sleep represents a principle of long-term memory formation established in quite different physiological systems.

https://www.zora.uzh.ch/id/eprint/77770/1/physiol_rev_93.pdf

About sleep's role in memory

Overview: The Dynamics of Complex Systems-Examples, Questions, Methods and Concepts Introduction and Preliminaries Neural Networks I: Subdivision and Hierarchy Neural Networks II: Models of Mind Protein Folding I: Size Scaling of Time Protein Folding II: Kinetic Pathways Life I: Evolution-Origin of Complex Organisms Life II: Developmental Biology-Complex by Design Human Civilization I: Defining Complexity Human Civilization II: A Complex(ity) Transition.

Dynamics Of Complex Systems

Crick and Mitchison have presented a hypothesis for the functional role of dream sleep involving an 'unlearning' process. We have independently carried out mathematical and computer modelling of learning and 'unlearning' in a collective neural network of 30-1,000 neurones. The model network has a content-addressable memory or 'associative memory' which allows it to learn and store many memories. A particular memory can be evoked in its entirety when the network is stimulated by any adequate-sized subpart of the information of that memory. But different memories of the same size are not equally easy to recall. Also, when memories are learned, spurious memories are also created and can also be evoked. Applying an 'unlearning' process, similar to the learning processes but with a reversed sign and starting from a noise input, enhances the performance of the network in accessing real memories and in minimizing spurious ones. Although our model was not motivated by higher nervous function, our system displays behaviours which are strikingly parallel to those needed for the hypothesized role of 'unlearning' in rapid eye movement (REM) sleep.

D. I. Feinstein

Papers

‘Unlearning’ has a stabilizing effect in collective memories