Trends in extreme learning machines

Neuromorphic computing has come to refer to a variety of brain-inspired computers, devices, and models that contrast the pervasive von Neumann computer architecture This biologically inspired approach has created highly connected synthetic neurons and synapses that can be used to model neuroscience theories as well as solve challenging machine learning problems The promise of the technology is to create a brain-like ability to learn and adapt, but the technical challenges are significant, starting with an accurate neuroscience model of how the brain works, to finding materials and engineering breakthroughs to build devices to support these models, to creating a programming framework so the systems can learn, to creating applications with brain-like capabilities In this work, we provide a comprehensive survey of the research and motivations for neuromorphic computing over its history We begin with a 35-year review of the motivations and drivers of neuromorphic computing, then look at the major research areas of the field, which we define as neuro-inspired models, algorithms and learning approaches, hardware and devices, supporting systems, and finally applications We conclude with a broad discussion on the major research topics that need to be addressed in the coming years to see the promise of neuromorphic computing fulfilled The goals of this work are to provide an exhaustive review of the research conducted in neuromorphic computing since the inception of the term, and to motivate further work by illuminating gaps in the field where new research is needed

A Survey of Neuromorphic Computing and Neural Networks in Hardware.

Neuro-biology inspired Spiking Neural Network (SNN) enables efficient learning and recognition tasks. To achieve a large scale network akin to biology, a power and area efficient electronic neuron is essential. Earlier, we had demonstrated an LIF neuron by a novel 4-terminal impact ionization based n+/p/n+ with an extended gate (gated-INPN) device by physics simulation. Excellent improvement in area and power compared to conventional analog circuit implementations was observed. In this paper, we propose and experimentally demonstrate a compact conventional 3-terminal partially depleted (PD) SOI- MOSFET (100 nm gate length) to replace the 4-terminal gated-INPN device. Impact ionization (II) induced floating body effect in SOI-MOSFET is used to capture LIF neuron behavior to demonstrate spiking frequency dependence on input. MHz operation enables attractive hardware acceleration compared to biology. Overall, conventional PD-SOI-CMOS technology enables very-large-scale-integration (VLSI) which is essential for biology scale (~1011 neuron based) large neural networks.

/pdf/leaky-integrate-and-fire-neuron-by-charge-discharge-dynamics-1zlow0lmpg.pdf

Leaky Integrate and Fire Neuron by Charge-Discharge Dynamics in Floating-Body MOSFET.

The following work presents an overview of smart sensors and sensor fusion targeted at biomedical applications and sports areas. In this work, the integration of these areas is demonstrated, promoting a reflection about techniques and applications to collect, quantify and qualify some physical variables associated with the human body. These techniques are presented in various biomedical and sports applications, which cover areas related to diagnostics, rehabilitation, physical monitoring, and the development of performance in athletes, among others. Although some applications are described in only one of two fields of study (biomedicine and sports), it is very likely that the same application fits in both, with small peculiarities or adaptations. To illustrate the contemporaneity of applications, an analysis of specialized papers published in the last six years has been made. In this context, the main characteristic of this review is to present the largest quantity of relevant examples of sensor fusion and smart sensors focusing on their utilization and proposals, without deeply addressing one specific system or technique, to the detriment of the others.

https://www.mdpi.com/1424-8220/16/10/1569/pdf

Sensor Fusion and Smart Sensor in Sports and Biomedical Applications.

Deep extreme learning machines

Silicon spiking neurons for hardware implementation of extreme learning machines

We describe a smart sensor for EEG acquisition comprising a programmable gain low-noise amplifier followed by an integrated feature extraction and classification circuits. The feature extraction block comprises a bank of four band-pass filters followed by a wide dynamic range peak detector. The output of the peak detector is fed into a spiking neural network implementing the extreme learning machine (ELM) for classification. The advantage of ELM is that it has been shown to attain comparable performance to support vector machine (SVM) but with fewer computational nodes. We describe simulation results of each block designed in 0.35 um CMOS and demonstrate system level performance by using this to detect seizure onset in epileptic patients. The system can be reconfigured for other applications like speech classification.

/pdf/a-low-power-reconfigurable-smart-sensor-system-for-eeg-pez69qfs0w.pdf

A low-power, reconfigurable smart sensor system for EEG acquisition and classification

In this paper, we describe a methodical approach for reducing errors due to mismatch in neuron circuits. We chose the neuron's current-frequency (f-i) curve as the desired output and use a sensitivity analysis to determine which transistors contribute most significantly to its variation. This allows us to identify the most critical transistors that need to be matched. For the special case in which floating-gate (FG) transistors are used to reduce this mismatch, we propose a method to further reduce the number of FG devices to be used in the circuit resulting in a corresponding reduction in “calibration” time. In addition to reducing mismatch between neurons, the usage of FG devices allows the user to independently set the parameters of each neuron. Since the calibration is based on f-i curve, it can be obtained through address-event representation (AER) circuits that are included in the neuron array for normal functionality. We use one example of commonly used integrate and fire neuron to illustrate this mismatch correction procedure. The method presented allows the corrected neurons to compute both rate codes and spike time codes in a mismatch resilient fashion.

/pdf/analysis-and-reduction-of-mismatch-in-silicon-neurons-4unb5s5auo.pdf

Analysis and reduction of mismatch in silicon neurons

The welding problems of large and thick plates are becoming more prominent as the application of large-scale and thick-plate metal structures grows. Due to the issue of excessive welding deformation between the 60mm thick steam turbine valve body and the pipe joint, a new process method is employed to connect. In this paper, the welding technology of flux strip confined arc ultra-narrow gap is proposed to carry out welding test on the ZG13Cr9Mo2Co1NiVNbNB cast steel test block of steam turbine valve body material with a thickness of 60 mm. The welding temperature field is measured by means of a K-type thermocouple and numerical simulation. The results show that the thermal cycle curve obtained by the homogeneous body heat source simulation is basically consistent with the thermal cycle curve measured during the experiment, and the simulation results of the molten pool morphology are also consistent with the actual macroscopic morphology of the weld.

Numerical Simulation of Temperature Field in Ultra-Narrow Arc Welding of Thick-Walled Steam Turbine Valve Body Material

In this work, an MIV-OSELM prediction model is constructed to predict the state of charge (SOC) of lead-acid battery, which combines the mean impact value (MIV) algorithm and the online sequence extreme learning machine (OSELM) algorithm. This model uses the MIV method to quantitatively calculate the impact value of the input variables on the output variables, and completes selection of the input variables of model; the OSELM method is used to carry out incremental learning of new samples generated during the use of battery, and track the potential impact of battery's state of health (SOH) on the SOC prediction of battery in a timely manner. Compared with the prediction results of other models, the MIV-OSELM method can improve the prediction accuracy of SOC during the charging and discharging processes of lead-acid batteries, which also has the adaptive ability to make dynamic adjustment of the model parameters according to the information of new samples.

Sun Shuo

Papers

Silicon spiking neurons for hardware implementation of extreme learning machines

A low-power, reconfigurable smart sensor system for EEG acquisition and classification

Analysis and reduction of mismatch in silicon neurons

Numerical Simulation of Temperature Field in Ultra-Narrow Arc Welding of Thick-Walled Steam Turbine Valve Body Material

The SOC Prediction of Lead-Acid Battery Based on MIV-OSELM Algorithm