https://www2.econ.iastate.edu/tesfatsi/DeepLearningInNeuralNetworksOverview.JSchmidhuber2015.pdf

Deep learning in neural networks

A comprehensive review of spatiotemporal pattern formation in systems driven away from equilibrium is presented, with emphasis on comparisons between theory and quantitative experiments. Examples include patterns in hydrodynamic systems such as thermal convection in pure fluids and binary mixtures, Taylor-Couette flow, parametric-wave instabilities, as well as patterns in solidification fronts, nonlinear optics, oscillatory chemical reactions and excitable biological media. The theoretical starting point is usually a set of deterministic equations of motion, typically in the form of nonlinear partial differential equations. These are sometimes supplemented by stochastic terms representing thermal or instrumental noise, but for macroscopic systems and carefully designed experiments the stochastic forces are often negligible. An aim of theory is to describe solutions of the deterministic equations that are likely to be reached starting from typical initial conditions and to persist at long times. A unified description is developed, based on the linear instabilities of a homogeneous state, which leads naturally to a classification of patterns in terms of the characteristic wave vector q0 and frequency ω0 of the instability. Type Is systems (ω0=0, q0≠0) are stationary in time and periodic in space; type IIIo systems (ω0≠0, q0=0) are periodic in time and uniform in space; and type Io systems (ω0≠0, q0≠0) are periodic in both space and time. Near a continuous (or supercritical) instability, the dynamics may be accurately described via "amplitude equations," whose form is universal for each type of instability. The specifics of each system enter only through the nonuniversal coefficients. Far from the instability threshold a different universal description known as the "phase equation" may be derived, but it is restricted to slow distortions of an ideal pattern. For many systems appropriate starting equations are either not known or too complicated to analyze conveniently. It is thus useful to introduce phenomenological order-parameter models, which lead to the correct amplitude equations near threshold, and which may be solved analytically or numerically in the nonlinear regime away from the instability. The above theoretical methods are useful in analyzing "real pattern effects" such as the influence of external boundaries, or the formation and dynamics of defects in ideal structures. An important element in nonequilibrium systems is the appearance of deterministic chaos. A greal deal is known about systems with a small number of degrees of freedom displaying "temporal chaos," where the structure of the phase space can be analyzed in detail. For spatially extended systems with many degrees of freedom, on the other hand, one is dealing with spatiotemporal chaos and appropriate methods of analysis need to be developed. In addition to the general features of nonequilibrium pattern formation discussed above, detailed reviews of theoretical and experimental work on many specific systems are presented. These include Rayleigh-Benard convection in a pure fluid, convection in binary-fluid mixtures, electrohydrodynamic convection in nematic liquid crystals, Taylor-Couette flow between rotating cylinders, parametric surface waves, patterns in certain open flow systems, oscillatory chemical reactions, static and dynamic patterns in biological media, crystallization fronts, and patterns in nonlinear optics. A concluding section summarizes what has and has not been accomplished, and attempts to assess the prospects for the future.

/pdf/pattern-formation-outside-of-equilibrium-1601b9znum.pdf

Pattern formation outside of equilibrium

This book explains the relationship of electrophysiology, nonlinear dynamics, and the computational properties of neurons, with each concept presented in terms of both neuroscience and mathematics and illustrated using geometrical intuition In order to model neuronal behavior or to interpret the results of modeling studies, neuroscientists must call upon methods of nonlinear dynamics This book offers an introduction to nonlinear dynamical systems theory for researchers and graduate students in neuroscience It also provides an overview of neuroscience for mathematicians who want to learn the basic facts of electrophysiology "Dynamical Systems in Neuroscience" presents a systematic study of the relationship of electrophysiology, nonlinear dynamics, and computational properties of neurons It emphasizes that information processing in the brain depends not only on the electrophysiological properties of neurons but also on their dynamical properties The book introduces dynamical systems starting with one- and two-dimensional Hodgkin-Huxley-type models and continuing to a description of bursting systems Each chapter proceeds from the simple to the complex, and provides sample problems at the end The book explains all necessary mathematical concepts using geometrical intuition; it includes many figures and few equations, making it especially suitable for non-mathematicians Each concept is presented in terms of both neuroscience and mathematics, providing a link between the two disciplines Nonlinear dynamical systems theory is at the core of computational neuroscience research, but it is not a standard part of the graduate neuroscience curriculum - or taught by math or physics department in a way that is suitable for students of biology This book offers neuroscience students and researchers a comprehensive account of concepts and methods increasingly used in computational neuroscience

/pdf/dynamical-systems-in-neuroscience-51dkewn57v.pdf

Dynamical Systems in Neuroscience

Neurons in the brain communicate by short electrical pulses, the so-called action potentials or spikes. How can we understand the process of spike generation? How can we understand information transmission by neurons? What happens if thousands of neurons are coupled together in a seemingly random network? How does the network connectivity determine the activity patterns? And, vice versa, how does the spike activity influence the connectivity pattern? These questions are addressed in this 2002 introduction to spiking neurons aimed at those taking courses in computational neuroscience, theoretical biology, biophysics, or neural networks. The approach will suit students of physics, mathematics, or computer science; it will also be useful for biologists who are interested in mathematical modelling. The text is enhanced by many worked examples and illustrations. There are no mathematical prerequisites beyond what the audience would meet as undergraduates: more advanced techniques are introduced in an elementary, concrete fashion when needed.

/pdf/spiking-neuron-models-single-neurons-populations-plasticity-58gzrd6ih6.pdf

Spiking Neuron Models: Single Neurons, Populations, Plasticity

Preface This is a guide to the software package AUTO for continuation and bifurcation problems in ordinary differential equations. graphics program PLAUT and the pendula animation program. An earlier graphical user interface for AUTO on SGI machines was written by Taylor & Kevrekidis (1989). Special thanks are due to Sheila Shull, California Institute of Technology, for her cheerful assistance in the distribution of AUTO over a long period of time. Over the years, the development of AUTO has been supported by various agencies through the California Institute of Technology. Work on this updated version was supported by a general research grant from NSERC (Canada). The development of HomCont has much benefitted from various pieces of help and advice from, among others, W. This manual uses the following conventions. command This font is used for commands which you can type in. PAR This font is used for AUTO parameters. filename This font is used for file and directory names. variable This font is used for environment variable. site This font is used for world wide web and ftp sites. function This font is used for function names.

AUTO 2000 : CONTINUATION AND BIFURCATION SOFTWARE FOR ORDINARY DIFFERENTIAL EQUATIONS (with HomCont)

To electronically simulate an animal nerve axon, the authors made an active pulse transmission line using tunnel diodes. The equation of propagation for this line is the same as that for a simplified model of nerve membrane treated elsewhere. This line shapes the signal waveform during transmission, that is, there being a specific pulse-like waveform peculiar to this line, smaller signals are amplified, larger ones are attenuated, narrower ones are widened and those which are wider are shrunk, all approaching the above-mentioned specific waveform. In addition, this line has a certain threshold value in respect to the signal height, and signals smaller than the threshold or noise are eliminated in the course of transmission. Because of the above-mentioned shaping action and the existence of a threshold, this line makes possible highly reliable pulse transmission, and will be useful for various kinds of information-processing systems.

An Active Pulse Transmission Line Simulating Nerve Axon

The paper introduces two types of active transmission lines having two stable states of equilibrium. In these, a transition from one state to the other is transmitted along the line and the transition waveforms are shaped during transmission. The properties of the lines have been investigated theoretically, using computers, and experimentally by using lumped constant elements and tunnel diodes.

S. Yoshizawa

Papers

An Active Pulse Transmission Line Simulating Nerve Axon

Bistable Transmission Lines