https://www2.econ.iastate.edu/tesfatsi/DeepLearningInNeuralNetworksOverview.JSchmidhuber2015.pdf

Deep learning in neural networks

Probability Distributions.- Linear Models for Regression.- Linear Models for Classification.- Neural Networks.- Kernel Methods.- Sparse Kernel Machines.- Graphical Models.- Mixture Models and EM.- Approximate Inference.- Sampling Methods.- Continuous Latent Variables.- Sequential Data.- Combining Models.

Pattern Recognition and Machine Learning

Intelligent tasks, such as visual perception, auditory perception, and language understanding require the construction of good internal representations of the world (or "features")? which must be invariant to irrelevant variations of the input while, preserving relevant information. A major question for Machine Learning is how to learn such good features automatically. Convolutional Networks (ConvNets) are a biologically-inspired trainable architecture that can learn invariant features. Each stage in a ConvNets is composed of a filter bank, some nonlinearities, and feature pooling layers. With multiple stages, a ConvNet can learn multi-level hierarchies of features. While ConvNets have been successfully deployed in many commercial applications from OCR to video surveillance, they require large amounts of labeled training samples. We describe new unsupervised learning algorithms, and new non-linear stages that allow ConvNets to be trained with very few labeled samples. Applications to visual object recognition and vision navigation for off-road mobile robots are described.

/pdf/convolutional-networks-and-applications-in-vision-2bpdhxxadr.pdf

Convolutional networks and applications in vision

This paper describes a 128 times 128 pixel CMOS vision sensor. Each pixel independently and in continuous time quantizes local relative intensity changes to generate spike events. These events appear at the output of the sensor as an asynchronous stream of digital pixel addresses. These address-events signify scene reflectance change and have sub-millisecond timing precision. The output data rate depends on the dynamic content of the scene and is typically orders of magnitude lower than those of conventional frame-based imagers. By combining an active continuous-time front-end logarithmic photoreceptor with a self-timed switched-capacitor differencing circuit, the sensor achieves an array mismatch of 2.1% in relative intensity event threshold and a pixel bandwidth of 3 kHz under 1 klux scene illumination. Dynamic range is > 120 dB and chip power consumption is 23 mW. Event latency shows weak light dependency with a minimum of 15 mus at > 1 klux pixel illumination. The sensor is built in a 0.35 mum 4M2P process. It has 40times40 mum2 pixels with 9.4% fill factor. By providing high pixel bandwidth, wide dynamic range, and precisely timed sparse digital output, this silicon retina provides an attractive combination of characteristics for low-latency dynamic vision under uncontrolled illumination with low post-processing requirements.

/pdf/a-128-times-128-120-db-15-mu-s-latency-asynchronous-temporal-3bv4hgi11i.pdf

A 128 $\times$ 128 120 dB 15 $\mu$ s Latency Asynchronous Temporal Contrast Vision Sensor

Hardware implementations of spiking neurons can be extremely useful for a large variety of applications, ranging from high-speed modeling of large-scale neural systems to real-time behaving systems, to bidirectional brain-machine interfaces. The specific circuit solutions used to implement silicon neurons depend on the application requirements. In this paper we describe the most common building blocks and techniques used to implement these circuits, and present an overview of a wide range of neuromorphic silicon neurons, which implement different computational models, ranging from biophysically realistic and conductance-based Hodgkin-Huxley models to bi-dimensional generalized adaptive integrate and fire models. We compare the different design methodologies used for each silicon neuron design described, and demonstrate their features with experimental results, measured from a wide range of fabricated VLSI chips.

/pdf/neuromorphic-silicon-neuron-circuits-3fphnmpvq9.pdf

Neuromorphic Silicon Neuron Circuits

European Union COST Actions provide the opportunity for researchers who are geographically dispersed to work together towards an ambitious, multidisciplinary goal, whilst learning from each other and avoiding effort duplication. This paper gives a brief overview of work done by, but not limited to, members of EU COST Action IC1401. The presented work summary is organized around the four workgroups that tackle devices, circuit theory, circuit implementations, bioinspired and sensory systems.

EU COST action IC1401 — Pushing the frontiers of memristive devices to systems

This paper proposed a vision system which implements a foveal mechanism to concentrate the attention and dynamically control the center and size of region of interest. The core of the system is an electronically-foveated dynamic vision sensor. An architecture and implementation of an electronically-foveated dynamic vision sensor is proposed. Simulation results demonstrating its operation are provided.

Electronically Foveated Dynamic Vision Sensor

In this live demonstration we exploit the use of a serial link for fast asynchronous communication in massively parallel processing platforms connected to a DVS for real-time implementation of bio-inspired vision processing on spiking neural networks.

Live demonstration: Multiplexing AER asynchronous channels over LVDS links with flow-control and clock-correction for scalable neuromorphic systems

Spiking Hardware for neuromorhic computing

Latest advances in CMOS technology, neuromorphic computing and flexible electronics are leading the way to a new era of brain implantable devices that promise to provide innovative and more effective treatments for brain disorders. Real-time signal detection and classification, as well as anticipation of brain electrical activity are yet under development. Microelectrode arrays (MEA) arise as a promising technology enabling detection of local field potentials from multiple locations and permitting the acquisition of more information on brain network electrical activity than conventional electrophysiology techniques. However, whereas most of the electrophysiological studies addressing brain activity have focused on events/patterns analysis, no one has so far addressed the features that might be hidden within the signal baseline. Such features might be particularly relevant in the context of epilepsy, as the signal baseline may carry relevant information for seizure prediction. Here, we present a preliminary processing and analysis of signal baseline acquired through MEA in an in vitro model of limbic seizures. The signal conditioning was implemented using an infinite impulse response (IIR) digital filter. After signal preprocessing, we applied an averaging method to 16 baseline sections to find common patterns and to study the frequency spectrum of this type of signal. We have found signal components between 0.5-2 Hz and peaks at 350, 390 and 650 Hz. In addition, the reconstruction of the averaged signal may provide insights into the main baseline wave patterns. These results might be a preliminary step to study the influence of those components on a biological basis. Based on these results, we propose a possible electronic architecture implementation of the signal processing method.

Baseline Features Extraction from Microelectrode Array Recordings in an in vitro model of Acute Seizures using Digital Signal Processing for Electronic Implementation

Bernabe Linares-Barranco

Papers

EU COST action IC1401 — Pushing the frontiers of memristive devices to systems

Electronically Foveated Dynamic Vision Sensor

Live demonstration: Multiplexing AER asynchronous channels over LVDS links with flow-control and clock-correction for scalable neuromorphic systems

Spiking Hardware for neuromorhic computing

Baseline Features Extraction from Microelectrode Array Recordings in an in vitro model of Acute Seizures using Digital Signal Processing for Electronic Implementation