Suggested by the structure of the visual nervous system, a new algorithm is proposed for pattern recognition. This algorithm can be realized with a multilayered network consisting of neuron-like cells. The network, “neocognitron”, is self-organized by unsupervised learning, and acquires the ability to recognize stimulus patterns according to the differences in their shapes: Any patterns which we human beings judge to be alike are also judged to be of the same category by the neocognitron. The neocognitron recognizes stimulus patterns correctly without being affected by shifts in position or even by considerable distortions in shape of the stimulus patterns.

Neocognitron--A New Algorithm for Pattern Recognition Tolerant of Deformations and Shifts in Position

One of the most complicated ANN models, the neocognitron (NC), is adapted to an efficient all-digital implementation for VLSI. The new model, the digi-neocognitron (DNC), has the same pattern recognition performance as the NC. The DNC model is derived from the NC model by a combination of preprocessing approximation and the definition of new model functions, e.g., multiplication and division are eliminated by conversion of factors to powers of 2, requiring only shift operations. The NC model is reviewed, the DNC model is presented, a methodology to convert NC models to DNC models is discussed, and the performances of the two models are compared on a character recognition example. The DNC model has substantial advantages over the NC model for VLSI implementation. The area-delay product is improved by two to three orders of magnitude, and I/O and memory requirements are reduced by representation of weights with 3 bits or less and neuron outputs with 4 bits or 7 bits. >

The digi-neocognitron: a digital neocognitron neural network model for VLSI

Shift invariance and the neocognitron

We describe a sequence of experiments investigating the strengths and limitations of Fukushima's neocognitron as a handwritten digit classifier. Using the results of these experiments as a foundation, we propose and evaluate improvements to Fukushima's original network in an effort to obtain higher recognition performance. The neocognitron's performance is shown to be strongly dependent on the choice of selectivity parameters and we present two methods to adjust these variables. Performance of the network under the more effective of the two new selectivity adjustment techniques suggests that the network fails to exploit the features that distinguish different classes of input data. To avoid this shortcoming, the network's final layer cells were replaced by a nonlinear classifier (a multilayer perceptron) to create a hybrid architecture. Tests of Fukushima's original system and the novel systems proposed in this paper suggest that it may be difficult for the neocognitron to achieve the performance of existing digit classifiers due to its reliance upon the supervisor's choice of selectivity parameters and training data. These findings pertain to Fukushima's implementation of the system and should not be seen as diminishing the practical significance of the concept of hierarchical feature extraction embodied in the neocognitron. © 1997 IEEE.

An evaluation of the neocognitron

We describe a sequence of experiments investigating the strengths and limitations of Fukushima's neocognitron as a handwritten digit classifier. Using the results of these experiments as a foundation, we propose and evaluate improvements to Fukushima's original network in an effort to obtain higher recognition performance. The neocognitron performance is shown to be strongly dependent on the choice of selectivity parameters and we present two methods to adjust these variables. Performance of the network under the more effective of the two new selectivity adjustment techniques suggests that the network fails to exploit the features that distinguish different classes of input data. To avoid this shortcoming, the network's final layer cells were replaced by a nonlinear classifier (a multilayer perceptron) to create a hybrid architecture. Tests of Fukushima's original system and the novel systems proposed in this paper suggest that it may be difficult for the neocognitron to achieve the performance of existing digit classifiers due to its reliance upon the supervisor's choice of selectivity parameters and training data.

The authors present theoretical and numerical developments in the understanding of feature extraction in the Neocognitron. First, they show that the feature extraction process is equivalent to a generalized nonlinear discriminant. Second, they show that the operation of the feature-extraction process can be linked to the eigenvectors and eigenvalues of a matrix comprised of the excitatory and inhibitory convolution masks. Third, the authors show how the choice of parameters for the feature extraction and learning process affects the feature extraction capabilities of the machine. >

Burman

Papers

Feature extraction in the Neocognitron