Neuromorphic computing has come to refer to a variety of brain-inspired computers, devices, and models that contrast the pervasive von Neumann computer architecture This biologically inspired approach has created highly connected synthetic neurons and synapses that can be used to model neuroscience theories as well as solve challenging machine learning problems The promise of the technology is to create a brain-like ability to learn and adapt, but the technical challenges are significant, starting with an accurate neuroscience model of how the brain works, to finding materials and engineering breakthroughs to build devices to support these models, to creating a programming framework so the systems can learn, to creating applications with brain-like capabilities In this work, we provide a comprehensive survey of the research and motivations for neuromorphic computing over its history We begin with a 35-year review of the motivations and drivers of neuromorphic computing, then look at the major research areas of the field, which we define as neuro-inspired models, algorithms and learning approaches, hardware and devices, supporting systems, and finally applications We conclude with a broad discussion on the major research topics that need to be addressed in the coming years to see the promise of neuromorphic computing fulfilled The goals of this work are to provide an exhaustive review of the research conducted in neuromorphic computing since the inception of the term, and to motivate further work by illuminating gaps in the field where new research is needed

A Survey of Neuromorphic Computing and Neural Networks in Hardware.

Artificial Intelligence has been the field of study for exploring the principles underlying thought, and utilizing their discovery to develop useful computers. Traditional AI models have been, consciously or subconsciously, optimized for available computing resources which has led AI in certain directions. The emergence of massively parallel computers liberates the way intelligence may be modeled. Although the AI community has yet to make a quantum leap, there are attempts to make use of the opportunities offered by massively parallel computers, such as memory-based reasoning, genetic algorithms, and other novel models. Even within the traditional AI approach, researchers have begun to realize that the needs for high performance computing and very large knowledge bases to develop intelligent systems requires massively parallel AI techniques. In this Computers and Thought Award lecture, I will argue that massively parallel artificial intelligence will add new dimensions to the ways that the AI goals are pursued, and demonstrate that massively parallel artificial intelligence is where AI meets the real world.

Challenges of massive parallelism

A neural processing element for use in a modular neural network is provided. One embodiment provides a neural network comprising an array of autonomous modules (300). The modules (300) can be arranged in a variety of configurations to form neural networks with various topologies, for example, with a hierarchical modular structure. Each module (300) contains sufficient neurons (100) to enable it to do useful work as a stand alone system, with the advantage that many modules (300) can be connected together to create a wide variety of configurations and network sizes. This modular approach results in a scaleable system that meets increased workload with an increase in parallelism and thereby avoids the usually extensive increases in training times associated with unitary implementations.

Neural processing element for use in a neural network

In this research, feature extraction and classification algorithms for high dimensional data are investigated. Developments with regard to sensors for Earth observation are moving in the direction of providing much higher dimensional multispectral imagery than is now possible.
In analyzing such high dimensional data, processing time becomes an important factor. With large increases in dimensionality and the number of classes, processing time will increase significantly. To address this problem, a multistage classification scheme is proposed which reduces the processing time substantially by eliminating unlikely classes from further consideration at each stage. Several truncation criteria are developed and the relationship between thresholds and the error caused by the truncation is investigated.
Next a novel approach to feature extraction for classification is proposed based directly on the decision boundaries. It is shown that all the features needed for classification can be extracted from decision boundaries. A novel characteristic of the proposed method arises by noting that only a portion of the decision boundary is effective in discriminating between classes, and the concept of the effective decision boundary is introduced. The proposed feature extraction algorithm has several desirable properties: (1) it predicts the minimum number of features necessary to achieve the same classification accuracy as in the original space for a given pattern recognition problem; (2) it finds the necessary feature vectors. The proposed algorithm does not deteriorate under the circumstances of equal means or equal covariances as some previous algorithms do. In addition, the decision boundary feature extraction algorithm can be used both for parametric and non-parametric classifiers.
Finally, we study some problems encountered in analyzing high dimensional data and propose possible solutions. We first recognize the increased importance of the second order statistics in analyzing high dimensional data. By investigating the characteristics of high dimensional data, we suggest the reason why the second order statistics must be taken into account in high dimensional data. Recognizing the importance of the second order statistics, there is a need to represent the second order statistics. We propose a method to visualize statistics using a color code. By representing statistics using color coding, one can easily extract and compare the first and the second statistics.

/pdf/feature-extraction-and-classification-algorithms-for-high-35eoz5denz.pdf

Feature extraction and classification algorithms for high-dimensional data

This paper presents a survey of digital systems to implement neural networks. We consider two basic options for designing these systems: parallel systems with standard digital components and parallel systems with custom processors. We describe many examples under each option, with an emphasis on commercially available systems. We report a first trend toward more general architectures and a second trend toward simple and fast structures. We discuss our experience in running a small ANN problem on two of these machines. After a reasonable programming effort, we obtain good convergence, but most of the training times are actually slower or moderately faster than on a serial workstation. We conclude that it is important to chose one's problems carefully, and that support software and in general, system integration, is only beginning to reach the level of versatility that many researchers will require.

Special-purpose digital hardware for neural networks: an architectural survey

A large-scale, dual-network architecture using wafer-scale integration (WSI) technology is proposed. By using 0.8 mu m CMOS technology, up to 144 self-learning digital neurons were integrated on each of eight 5 in silicon wafers. Neural functions and the back-propagation (BP) algorithm were mapped to digital circuits. The complete hardware system packaged more than 1000 neurons within a 30 cm cube. The dual-network architecture allowed high-speed learning at more than 2 gigaconnections updated per second (GCUPS). The high fault tolerance of the neural network and proposed defect-handling techniques overcame the yield problem of WSI. This hardware can be connected to a host workstation and used to simulating a wide range of artificial neural networks. Signature verification and stock price prediction have already been demonstrated with this hardware. >

A self-learning digital neural network using wafer-scale LSI

The design, fabrication, and evaluation of a compact self-learning neural network made up of more than 1000 neurons are described. A time-sharing bus architecture decreases the number of circuits required and makes possible flexible and expandable networks. Neural functions and the back propagation (BP) algorithm were mapped to binary digital circuits. A dual-network architecture allows high-speed learning. This hardware can be connected to a host workstation and used for a wide range of artificial neural networks. Signature verification and stock price prediction have already been demonstrated with this hardware. The peak learning speed was about 10 times faster than BP simulation by an S-820 Hitachi supercomputer. >

M. Yasunaga

Papers

A self-learning digital neural network using wafer-scale LSI

A self-learning neural network composed of 1152 digital neurons in wafer-scale LSIs