Showing papers on "Feature (machine learning) published in 1991"

Pattern Recognition: Statistical, Structural and Neural Approaches

Abstract: STATISTICAL PATTERN RECOGNITION (StatPR). Supervised Learning (Training) Using Parametric and Nonparametric Approaches. Linear Discriminant Functions and the Discrete and Binary Feature Cases. Unsupervised Learning and Clustering. SYNTACTIC PATTERN RECOGNITION (SyntPR). Overview. Syntactic Recognition via Parsing and Other Grammars. Graphical Approaches to SyntPR. Learning via Grammatical Inference. NEURAL PATTERN RECOGNITION (NeurPR). Introduction to Neural Networks. Introduction to Neural Pattern Associators and Matrix Approaches. Feedforward Networks and Training by Backpropagation. Content Addressable Memory Approaches and Unsupervised Learning in NeurPR. Appendices. References. Permission Source Notes. Index.

Assessment of features technology

Abstract: Features encapsulate the engineering significance of portions of the geometry of a part or assembly, and, as such, are important in product design, product definition, and reasoning, for a variety of applications. Feature-based systems have demonstrated some potential in creating attractive design environments and in automating the geometric reasoning required in applications such as process planning and manufacturability evaluation. The paper reviews the major concepts and approaches that are in use in feature-based modelling. Several methodologies are prevalent for creating feature models and databases. These fall broadly into the categories of interactive definition, automatic recognition/extraction, and design by features. Within each, there are several subcategories, which are discussed and compared in the paper. Also presented are several schemes popular for representing features. They include augmented graphs, syntactic strings in grammars, and objects in object-oriented programming. Feature interactions and validation issues are outlined. Attempts at developing feature taxonomies are also summarized.

Method of speech recognition

Abstract: A set of "m" feature parameters is generated every frame from reference speech which is spoken by at least one speaker and which represents recognition-object words, where "m" denotes a preset integer. A set of "n" types of standard patterns is previously generated on the basis of speech data of a plurality of speakers, where "n" denotes a preset integer. Matching between the feature parameters of the reference speech and each of the standard patterns is executed to generate a vector of "n" reference similarities between the feature parameters of the reference speech and each of the standard patterns every frame. The reference similarity vectors of respective frames are arranged into temporal sequences corresponding to the recognition-object words respectively. The reference similarity vector sequences are previously registered as dictionary similarity vector sequences. Input speech to be recognized is analyzed to generate "m" feature parameters from the input speech. Matching between the feature parameters of the input speech and the standard patterns is executed to generate a vector of "n" input-speech similarities between the feature parameters of the input speech and the standard patterns every frame. The input-speech similarity vectors of respective frames are arranged into a temporal sequence. The input-speech similarity vector sequence is collated with the dictionary similarity vector sequences to recognize the input speech.

Multi-Digit Recognition Using a Space Displacement Neural Network

Abstract: We present a feed-forward network architecture for recognizing an unconstrained handwritten multi-digit string. This is an extension of previous work on recognizing isolated digits. In this architecture a single digit recognizer is replicated over the input. The output layer of the network is coupled to a Viterbi alignment module that chooses the best interpretation of the input. Training errors are propagated through the Viterbi module. The novelty in this procedure is that segmentation is done on the feature maps developed in the Space Displacement Neural Network (SDNN) rather than the input (pixel) space.

Applications of neural networks to character recognition

Abstract: Among the many applications that have been proposed for neural networks, character recognition has been one of the most successful. Compared to other methods used in pattern recognition, the advantage of neural networks is that they offer a lot of flexibility to the designer, i.e. expert knowledge can be introduced into the architecture to reduce the number of parameters determined by training by examples. In this paper, a general introduction to neural network architectures and learning algorithms commonly used for pattern recognition problems is given. The design of a neural network character recognizer for on-line recognition of handwritten characters is then described in detail.

Incremental constructive induction: an instance-based approach

Abstract: This paper introduces IB3-CI, the first instance-based learning algorithm that performs feature construction. Results with two learning tasks show that it attains higher predictive accuracies, lower storage requirements, and uses fewer attributes to compute similarities than previous instance-based learning algorithms, but only when its feature construction process is appropriately constrainted. The incremental IB3-CI algorithm, which partially integrates IB3 with STAGGER, performs as well as or better than several non-incremental algorithms on these tasks.

System and method for automatically classifying heterogeneous business forms

Abstract: Business forms are a special class of documents typically used to collect or distribute data; they represent a vast majority of the paperwork need to conduct business. The present invention provides a pattern recognition system that classifies digitized images of business forms according to a predefined set of templates. The process involves a training phase, during which images of the template forms are scanned, analyzed and stored in a data dictionary, and a recognition phase, during which images of actual forms are compared to the templates in the dictionary to determine their class membership. The invention provides the feature extraction and matching methods, as well as the organization of the form dictionary. The performance of the system was evaluated using a collection of computer generated test forms. The methodology for creating these forms, and the results of the evaluation are also described. Business forms are characterized by the presence of horizontal and vertical lines that delimit the useable space. The present invention identifies these so called regular lines in bi-level digital images to separate text from graphics before applying an optical character recognizer; or as a feature extractor in a form recognition system. The approach differs from existing vectorization, line extraction, and text-graphics separation methods, in that it focuses exclusively on the recognition of horizontal and vertical lines.

Genetic algorithm for optical pattern recognition.

Abstract: A genetic algorithm is used to generate binary reference functions for optical pattern recognition and classification. Procedures based on the properties of convex functions can be implemented directly on hybrid electro-optical systems. Computer simulations demonstrate the efficiency of this novel approach.

Method and apparatus for utilizing multiple data fields for character recognition

Abstract: A system (10) for recognizing numerical characters which appear as both numerical characters and words on a document includes an image capture device (12) for capturing the image of a document. A recognition circuit (14) performs recognition of the numerical characters. A recognition circuit (16) performs recognition of the words corresponding to the numerical characters. A comparator circuit (20) compares the recognition signals generated by the recognition circuit (16) to the recognition signals generated by the recognition circuit (14) to determine if the numerical characters recognized by the recognition circuit (14) is correct.

Speech recognition apparatus using neural network and fuzzy logic

Abstract: A speech recognition apparatus has: a speech input unit for inputting a speech; a speech analysis unit for analyzing the inputted speech to output the time series of a feature vector; a candidates selection unit for inputting the time series of a feature vector from the speech analysis unit to select a plurality of candidates of recognition result from the speech categories; and a discrimination processing unit for discriminating the selected candidates to obtain a final recognition result. The discrimination processing unit includes three components in the form of a pair generation unit for generating all of the two combinations of the n-number of candidates selected by said candidate selection unit a pair discrimination unit for discriminating which of the candidates of the combinations is more certain for each of all n C 2 -number of combinations (or pairs) on the basis of the extracted result of the acoustic feature intrinsic to each of said candidate speeches and a final decision unit for collecting all the pair discrimination results obtained from the pair discrimination unit for each of all the n C 2 -number of combinations (or pairs) to decide the final result. The pair discrimination unit handles the extracted result of the acoustic feature intrinsic to each of the candidate speeches as fuzzy information and accomplishes the discrimination processing on the basis of fuzzy logic algorithms, and the final decision unit accomplishes its collections on the basis of the fuzzy logic algorithms.

A segment-based approach to automatic language identification

Abstract: A segment-based approach to automatic language identification is discussed which is based on the idea that the acoustic structure of languages can be estimated by segmenting speech into broad phonetic categories. Automatic language identification can then be achieved by computing features that describe the phonetic and prosodic characteristics of the language, and using these feature measurements to train a classifier to distinguish between languages. As a first step in this approach, a multilanguage neural-network-based segmentation and broad classification algorithm using seven broad phonetic categories has been built. The algorithm was trained and tested on separate sets of speakers of American English, Japanese, Mandarin Chinese, and Tamil. It currently performs with an accuracy of 82.3% on the utterances of the test set. >

Fast affine point matching: an output-sensitive method

Abstract: A model-based recognition method that runs in time proportional to the actual number of instances of a model that are found in an image is presented. The key idea is to filter out many of the possible matches without having to explicitly consider each one. This contrasts with the hypothesize-and-test paradigm, commonly used in model-based recognition, where each possible match is tested and either accepted or rejected. For most recognition problems the number of possible matches is very large, whereas the number of actual matches is quite small, making output-sensitive methods such as this one very attractive. The method is based on an affine invariant representation of an object that uses distance ratios defined by quadruples of feature points. A central property of this representation is that it can be recovered from an image using only pairs of feature points. >

Features and formulae

Abstract: Feature structures are a representational device used in several current linguistic theories. This paper shows how these structures can be axiomatized in a decidable class of first-order logic, which can also be used to express constraints on these structures. Desirable properties, such as compactness and decidability, follow directly. Moreover, additional types of feature values, such as "set-valued" features, can be incorporated into the system simply by axiomatizing their properties.

Feature reasoning for sheet metal components

Abstract: The objective of this paper is to develop a set of principles for extracting, recognizing and reasoning about features from sheet metal parts which are created in the CAD system. Recognition of such features will enable the automatic evaluation of designs and the development of process plans by among other things, mapping form features to tools needed to produce the form. The principles are developed by studying the face-oriented representations of features and deriving concepts which relate the features geometrically and topologically. Subsequently, the processes that relate to the geometrical forms are used to further uniquely identify the features.

Unsupervised learning and the information retrieval problem

Abstract: The author presents two implemented neuronal methods for free-text database search in details. In the first method, a specific interest (or query) is taught to a Kohonen feature map. By using this network as a neural filter on a dynamic free-text database, only the associated subjects are selected from this database. The second method can be used in a more static environments. Statistical properties (n-grams) from various texts are taught to a feature map. A comparison of a query with this feature map results in the selection of texts with are closely related with respect to their contents. Both methods are compared with classical statistical information-retrieval algorithms. Various simulations show that the neural net converges towards a proper representation of the query as well as the objects in the database. The first algorithm exhibits much better scalability than its statistical counterparts, resulting in higher speeds, less memory needs, and easier maintainability. The second one shows an elegant and uniform generalization and association method, increasing the selection quality. >

Depth of processing approach to face recognition: A test of two theories.

Abstract: Three experiments investigated feature quantity and semantic quality accounts for level of processing effects in face recognition

HyperBF networks for real object recognition

Abstract: Even if represented in a way which is invariant to illumination conditions, a 3D object gives rise to an infinite number of 2D views, depending on its pose. It has been recently shown ([13]) that it is possible to synthesize a module that can recognize a specific 3D object from any viewpoint, by using a new technique of learning from examples, which are, in this case, a small set of 2D views of the object. In this paper we extend the technique, a) to deal with real objects (isolated paper clips) that suffer from noise and occlusions and b) to exploit negative examples during the learning phase. We also compare different versions of the multi-layer networks corresponding to our technique among themselves and with a standard Nearest Neighbor classifier. The simplest version, which is a Radial Basis Functions network, performs less well than a Nearest Neighbor classi-fier. The more powerful versions, trained with positive and negative examples, perform significantly better. Our results, which may have interesting implications for computer vision despite the relative simplicity of the task studied, are especially interesting for understanding the process of object recognition in biological vision. 1 Introduction Shape-based visual recognition of 3D objects may be solved by first hypothesizing the viewpoint (e.g., using information on feature correspondences between the image and a 3D model), then computing the appearance of the model of the object to be recognized from that viewpoint and comparing it with the actual image ([6; 20; 9; 11; 21]). Most recognition schemes developed in computer vision over the last few years employ 3D models of objects. Automatic learning of 3D models, however, is in itself a difficult problem that has not been much addressed in the past and which presents difficulties, especially for any theory that wants to account for human ability in visual recognition. Recently, recognition schemes have been suggested that, relying on a set of 2D views of the object instead of a 3D model ([2; 5; 13]), offer a natural solution to the problem of model acquisition. In particular, Poggio and Edelman ([13]) have argued that for each object there exists a smooth function mapping any perspective view into a "standard" view of the object and that this mul-tivariate function may be approximatevely synthesized from a small number of views of the object. Such a function would be object specific, with different functions corresponding to different 3D objects. …

Velocity and acceleration features in speaker recognition

Abstract: The performance of dynamic features in automatic speaker recognition is examined. Second- and third-order regression analysis examining the performance of the associated feature sets independently, in combination, and in the presence of noise is included. It is shown that each regression order has a clear optimum. These are independent of the analysis order of the static feature from which the dynamic features are derived, and insensitive to low-level noise added to the test speech. It is also demonstrated that while the static feature gives the best individual performance, multiple linear combinations of feature sets based on regression analysis can reduce error rates. >

Pattern recognition apparatus

Abstract: A pattern recognition apparatus using the hidden Markov model technique in which parameters for defining a mean vector representing the probability density function in each one of plural states for composing the hidden Markov model vary with time. Accordingly, the recognition precision may be enhanced when this apparatus is used in, for example, voice recognition.

A business form recognition system

Abstract: A pattern recognition system is described that classifies digitized images of business forms according to a predefined set of templates. The process involves a training phase, where images of the template forms are scanned, analyzed and stored in a data dictionary; and a recognition phase, during which scanned form images are compared to templates in a dictionary to determine their class membership. The system has been tested under a variety of conditions and its performance has been proven to be satisfactory. >

Character recognition with selective attention

Abstract: Selective attention is one of the most essential mechanisms for visual pattern recognition. One of the authors had previously proposed a model of selective attention, which has a function of segmenting patterns, as well as the function of recognizing patterns. The idea of this selective attention model can be extended to be used for several applications. The structure of the model used for connected character recognition is discussed. The authors offer two examples of its applications. One is the recognition and segmentation of connected characters in cursive handwriting of English words. Another example is the recognition of Chinese characters. Preliminary experiments with computer simulation, in which only a small number of characters have been taught to the models, show that this approach is promising for both of the applications. >

ARTMAP: a self-organizing neural network architecture for fast supervised learning and pattern recognition

Abstract: The authors present a neural network architecture, called ARTMAP, that autonomously learns to classify arbitrarily many, arbitrarily ordered vectors into recognition categories based on predictive success. This supervised learning system is built up from a pair of adaptive resonance theory modules that are capable of self-organizing stable recognition categories in response to arbitrary sequences of input patterns. Tested on a benchmark machine learning database in both online and offline simulations, the ARTMAP system learned orders of magnitude more quickly, efficiently, and accurately than alternative algorithms, and achieves 100% accuracy after training on less than half the input patterns in the database. It achieves these properties by using an internal controller that conjointly maximizes predictive generalization and minimizes predictive error by linking predictive success to category size on a trial-by-trial basis, using only local operations. >

Compound feature recognition by web grammar parsing

Abstract: A method is presented to automatically recognize compound features using web grammar parsing on a solid model graph. A compound feature is a class of regional shapes with a variable topology. Examples of compound features are connected features, which contain a variable number of simple features connected together, and intersecting features, which intersect one another creating different shapes from the original simple features conceived. The simple features are regional shapes related to preestablished application processes. A solid object, stored as a boundary representation, is transformed to a web representation (a node-labeled graph) which is the input of a web parsing system for feature recognition. And a compound feature is described as a web grammar. From the web representation of an object shape, the compound feature recognition can be accomplished by parsing the web with the web grammar. The application of a web grammar enables a user to define a compound feature and makes feature recognition a formalized process for subsequent recognition.

Learning polynomial functions by feature construction

Abstract: We present a method for learning higher-order polynomial functions from examples using linear regression and feature construction Regression is used on a set of training instances to produce a weight vector for a linear function over the feature set If this hypothesis is imperfect, a new feature is constructed by forming the product of the two features that most effectively predict the squared error of the current hypothesis The algorithm is then repeated In an extension to this method, the specific pair of features to combine is selected by measuring their joint ability to predict the hypothesis' error

An Incremental Deductive Strategy for Controlling Constructive Induction in Learning from Examples

Abstract: This article describes LAIR, a constructive induction system that acquires conjunctive concepts by applying a domain theory to introduce new features into the evolving concept description. Each acquired concept is added to the domain theory, making LAIR a closed-loop learning system that weakens the inductive bias with each iteration of the learning loop. LAIR's novel feature is the use of an incremental deductive strategy for constructive induction, reducing the amount of inference required for learning. A series of experiments manipulated features of learning tasks to assess this incremental method of constructive induction relative to an uncontrolled constructive induction process that extends each example description with all derivable features. These learning tasks differed in global characteristics of the domain theory, the training sequence, and the percentage of irrelevant features in the example descriptions. The results show that LAIR's constructive induction approach saves considerable inferencing effort, with little or no cost in the number of examples needed to reach a learning criterion. The experimental results also underscored the importance of viewing a domain theory as a search space, identifying several factors that impact the deductive and inductive aspects of constructive induction, such as concept definition overlap, density of features, and fan-in and fan-out of inference chains. The paper also discusses LAIR's operation as a pac-learner and its relation to other constructive induction techniques.

Analysis of a biologically motivated neural network for character recognition

Abstract: A neural network architecture for size-invariant and local shape-invariant digit recognition has been developed. The network is based on known biological data on the structure of vertebrate vision but is implemented using more conventional numerical methods for image feature extraction and pattern classification. The input receptor field structure of the network uses Gabor function feature selection. The classification section of the network uses back-propagation. Using these features as neurode inputs, an implementation of back-propagation on a serial machine achieved 100% accuracy when trained and tested on a single font size and style while classifying at a rate of 2 ms per character. Taking the same trained network, recognition greater than 99.9% accuracy was achieved when tested with digits of different font sizes. A network trained on multiple font styles when tested achieved greater than 99.9% accuracy and, when tested with digits of different font sizes, achieved greater than 99.8% accuracy. These networks, trained only with good quality prototypes, recognized images degraded with 15% random noise with an accuracy of 89%. In addition to raw recognition results, a study was conducted where activation distributions of correct responses from the network were compared against activation distributions of incorrect responses. By establishing a threshold between these two distributions, a reject mechanism was developed to minimize substitutional errors. This allowed substitutional errors on images degraded with 10% random noise to be reduced from 2.08% to 0.25%.

Methods for enhancing neural network handwritten character recognition

Abstract: An efficient method for increasing the generalization capacity of neural character recognition is presented. The network uses a biologically inspired architecture for feature extraction and character classification. The numerical methods used are optimized for use on massively parallel array processors. The method for training set construction, when applied to handwritten digit recognition, yielded a writer-independent recognition rate of 92%. The activation strength produced by network recognition is an effective statistical confidence measure of the accuracy of recognition. A method of using the activation strength for reclassification is described which, when applied to handwritten digit recognition, reduced substitutional errors to 2.2%. >

Unified Description of Neural Algorithms for Time-Independent Pattern Recognition

Abstract: Results obtained by Pineda for supervised learning in arbitrarily structured neural nets (including feed-back) are extended to nonsupervised learning. For the first time a unique set of 3 equations is derived which governs the learning dynamics of neural models that make use of objective functions. A general method to construct objective functions is outlined that helps organize the network output according to application-specific constraints. Several well-known learning algorithms are deduced exemplarily within the general frame. The unification turns out to result in economical design of software as well as hardware.