A key task for connectionist research is the development and analysis of learning algorithms. An examination is made of several supervised learning algorithms for single-cell and network models. The heart of these algorithms is the pocket algorithm, a modification of perceptron learning that makes perceptron learning well-behaved with nonseparable training data, even if the data are noisy and contradictory. Features of these algorithms include speed algorithms fast enough to handle large sets of training data; network scaling properties, i.e. network methods scale up almost as well as single-cell models when the number of inputs is increased; analytic tractability, i.e. upper bounds on classification error are derivable; online learning, i.e. some variants can learn continually, without referring to previous data; and winner-take-all groups or choice groups, i.e. algorithms can be adapted to select one out of a number of possible classifications. These learning algorithms are suitable for applications in machine learning, pattern recognition, and connectionist expert systems. >

Perceptron-based learning algorithms

Apparatus, and an accompanying method, for a neural network, particularly one suited for use in optical character recognition (OCR) systems, which through controlling back propagation and adjustment of neural weight and bias values through an output confidence measure, smoothly, rapidly and accurately adapts its response to actual changing input data (characters). Specifically, the results of appropriate actual unknown input characters, which have been recognized with an output confidence measure that lies within a pre-defined range, are used to adaptively re-train the network during pattern recognition. By limiting the maximum value of the output confidence measure at which this re-training will occur, the network re-trains itself only when the input characters have changed by a sufficient margin from initial training data such that this re-training is likely to produce a subsequent noticeable increase in the recognition accuracy provided by the network. Output confidence is measured as a ratio between the highest and next highest values produced by output neurons in the network. By broadening the entire base of training data to include actual dynamically changing input characters, the inventive neural network provides more robust performance than which heretofore occurs in neural networks known in the art.

Neural network with back propagation controlled through an output confidence measure

Segmentation of merged characters by neural networks and shortest path

Character images which are to be sent to a neural network trained to recognize a predetermined set of symbols are first processed by an optical character recognition pre-processor which normalizes the character images. The output of the neural network is processed by an optical character recognition post-processor. The post-processor corrects erroneous symbol identifications made by the neural network. The post-processor identifies special symbols and symbol cases not identifiable by the neural network following character normalization. For characters identified by the neural network with low scores, the post-processor attempts to find and separate adjacent characters which are kerned and characters which are touching. The touching characters are separated in one of nine successively initiated processes depending upon the geometric parameters of the image. When all else fails, the post-processor selects either the second or third highest scoring symbol identified by the neural network based upon the likelihood of the second or third highest scoring symbol being confused with the highest scoring symbol.

Optical character recognition neural network system for machine-printed characters

A major problem with a neural network-based approach to printed character recognition is the segmentation of merged characters. A hybrid method is proposed which combines a neural network-based deferred segmentation scheme with conventional immediate segmentation techniques. In the deferred segmentation, a neural network is employed to distinguish single characters from composites. To find a proper vertical cut that separates a composite, a shortest-path algorithm seeking minimal-penalty curved cuts is used. Integrating those components with a multiresolution neural network OCR and an efficient spelling checker, the resulting system significantly improves its ability to read omnifont document text.

Segmentation of merged characters by neural networks and shortest-path

Summary form only given. New results are presented which were obtained by applying a connectionist model with the pocket algorithm proposed by Gallant to problems in alphanumeric character recognition. An evaluation is made of the recognition (classification) capability of the connectionist model for 62 and 93 alphanumeric characters of a single font having different kinds of typeface quality and 76 alphanumeric characters of multiple fonts having the same typeface quality. A useful technique is proposed to distinguish between characters that closely resemble each other by using the structure information of characters. An activation criterion of output cells for character recognition is also proposed. In the recognition of characters having different typeface qualities, a markedly high degree of accuracy (99.96% maximum, 99.74% on average) of the individual fonts was attained. In the recognition of the 76 alphanumeric characters having multiple fonts, the degree of accuracy achieved was 99.64% maximum. >

Ohhashi

Papers

Alphanumeric character recognition using a connectionist model with the pocket algorithm