Showing papers on "Feature (machine learning) published in 1993"

High confidence visual recognition of persons by a test of statistical independence

Abstract: A method for rapid visual recognition of personal identity is described, based on the failure of a statistical test of independence. The most unique phenotypic feature visible in a person's face is the detailed texture of each eye's iris. The visible texture of a person's iris in a real-time video image is encoded into a compact sequence of multi-scale quadrature 2-D Gabor wavelet coefficients, whose most-significant bits comprise a 256-byte "iris code". Statistical decision theory generates identification decisions from Exclusive-OR comparisons of complete iris codes at the rate of 4000 per second, including calculation of decision confidence levels. The distributions observed empirically in such comparisons imply a theoretical "cross-over" error rate of one in 131000 when a decision criterion is adopted that would equalize the false accept and false reject error rates. In the typical recognition case, given the mean observed degree of iris code agreement, the decision confidence levels correspond formally to a conditional false accept probability of one in about 10/sup 31/. >

Connectionist Speech Recognition: A Hybrid Approach

Abstract: From the Publisher: Connectionist Speech Recognition: A Hybrid Approach describes the theory and implementation of a method to incorporate neural network approaches into state-of-the-art continuous speech recognition systems based on Hidden Markov Models (HMMs) to improve their performance. In this framework, neural networks (and in particular, multilayer perceptrons or MLPs) have been restricted to well-defined subtasks of the whole system, i.e., HMM emission probability estimation and feature extraction. The book describes a successful five year international collaboration between the authors. The lessons learned form a case study that demonstrates how hybrid systems can be developed to combine neural networks with more traditional statistical approaches. The book illustrates both the advantages and limitations of neural networks in the framework of a statistical system. Using standard databases and comparing with some conventional approaches, it is shown that MLP probability estimation can improve recognition performance. Other approaches are discussed, though there is no such unequivocal experimental result for these methods. Connectionist Speech Recognition: A Hybrid Approach is of use to anyone intending to use neural networks for speech recognition or within the framework provided by an existing successful statistical approach. This includes research and development groups working in the field of speech recognition, both with standard and neural network approaches, as well as other pattern recognition and/or neural network researchers. This book is also suitable as a text for advanced courses on neural networks or speech processing.

Using Genetic Algorithms for Concept Learning

Abstract: In this article, we explore the use of genetic algorithms (GAs) as a key element in the design and implementation of robust concept learning systems. We describe and evaluate a GA-based system called GABIL that continually learns and refines concept classification rules from its interaction with the environment. The use of GAs is motivated by recent studies showing the effects of various forms of bias built into different concept learning systems, resulting in systems that perform well on certain concept classes (generally, those well matched to the biases) and poorly on others. By incorporating a GA as the underlying adaptive search mechanism, we are able to construct a concept learning system that has a simple, unified architecture with several important features. First, the system is surprisingly robust even with minimal bias. Second, the system can be easily extended to incorporate traditional forms of bias found in other concept learning systems. Finally, the architecture of the system encourages explicit representation of such biases and, as a result, provides for an important additional feature: the ability to dynamically adjust system bias. The viability of this approach is illustrated by comparing the performance of GABIL with that of four other more traditional concept learners (AQ14, C4.5, ID5R, and IACL) on a variety of target concepts. We conclude with some observations about the merits of this approach and about possible extensions.

Using decision trees to improve case-based learning

Abstract: This paper shows that decision trees can be used to improve the performance of case- based learning (CBL) systems. We introduce a performance task for machine learning systems called semi-flexible prediction that lies between the classification task performed by decision tree algorithms and the flexible prediction task performed by conceptual clustering systems. In semi-flexible prediction, learning should improve prediction of a specific set of features known a priori rather than a single known feature (as in classification) or an arbitrary set of features (as in conceptual clustering). We describe one such task from natural language processing and present experiments that compare solutions to the problem using decision trees, CBL, and a hybrid approach that combines the two. In the hybrid approach, decision trees are used to specify the features to be included in k-nearest neighbor case retrieval. Results from the experiments show that the hybrid approach outperforms both the decision tree and case-based approaches as well as two case-based systems that incorporate expert knowledge into their case retrieval algorithms. Results clearly indicate that decision trees can be used to improve the performance of CBL systems and do so without reliance on potentially expensive expert knowledge.

Further Research on Feature Selection and Classification Using Genetic Algorithms

Abstract: This paper summarizes work on an approach that combines feature selection and data classiication using Genetic Algorithms. First, it describes our use of Genetic Algorithms combined with a K-nearest neighbor algorithm to optimize classiication by searching for an optimal feature weight-ing, essentially warping the feature space to coalesce individuals within groups and to separate groups from one another. This approach has proven especially useful with large data sets where standard feature selection techniques are computationally expensive. Second, it describes our implementation of the approach in a parallel processing environment, giving nearly linear speed-up in processing time. Third, it will summarize our present results in using the technique to discover the relative importance of features in large biological test sets. Finally, it will indicate areas for future research. 1 The Problem We live in the age of information where data is plentiful , to the extent that we are typically unable to process all of it usefully. Computer science has been challenged to discover approaches that can sort through the mountains of data available and discover the essential features needed to answer a speciic question. These approaches must be able to process large quantities of data, in reasonable time and in the presence of oisy" data i.e., irrelevant or erroneous data. Consider a typical example in biology. Researchers in the Center for Microbial Ecology (CME) have selected soil samples from three environments found in agriculture. The environments were: near the roots of a crop (rhizosphere), away from the innuence of the crop roots (non-rhizosphere), and from a fallow eld (crop residue). The CME researchers wished to investigate whether samples from those three environments could be distinguished. In particular, they wanted to see if diversity decreased in the rhizosphere as a result of the symbiotic relationship between the roots and its near-neighbor microbes, and if so in what ways. Their rst experiments used the Biolog c test as the discriminator. Biolog consists of a plate of 96 wells, with a diierent substrate in each well. These sub-strates (various sugars, amino acids and other nutrients) are assimilated by some microbes and not by others. If the microbial sample processes the substrate in the well, that well changes color which can be recorded photometrically. Thus large numbers of samples can be processed and characterized based on the substrates they can assimilate. The CME researchers applied the Biolog test to 3 sets of 100 samples …

Self-Organizing Neural Networks for Visualisation and Classification

Abstract: This paper presents the usage of an artificial neural network, Kohonen’s self organizing feature map, for visualisation and classification of high dimensional data. Through a learning process, this neural network creates a mapping from a N-dimensional space to a two-dimensional plane of units (neurons). This mapping is known to preserve topological relations of the N-dimensional space. A specially developed technique, called U-matrix method has been developed in order to detect nonlinearities in the resulting mapping. This method can be used to visualize structures of the N-dimensional space. Boundaries between different subsets of input data can be detectet. This allows to use this method for a clustering of the data. New data can be classified in an associative way. It has been demonstrated, that the method can be used also for knowledge acquisition and exploratory data analysis purposes.

Speech recognition by concatenating fenonic allophone hidden Markov models in parallel among subwords

Abstract: Analysis of a word input from a speech input device 1 for its features is made by a feature extractor 4 to obtain a feature vector sequence corresponding to said word, or to obtain a label sequence by applying a further transformation in a labeler 8. Fenonic hidden Markov models for speech transformation candidates are combined with N-gram probabilities (where N is all integer greater than or equal to 2) to produce models of words. The recognizer determines the probability that the speech model composed for each candidate word would output the label sequence or feature vector sequence input as speech, and outputs the candidate word corresponding to the speech model having the highest probability to a display 19.

Memory-based automaticity in the discrimination of visual numerosity.

Abstract: In the development of memory-based models of automaticity, it is crucial to specify the nature of the memory representation. Seven experiments with 94 students use a counting task to determine whether a feature (i.e., identity, color, or orientation) is explicitly represented in memory. It is assumed that the degree of transfer to a pattern differing on one feature is determined by that feature's importance in supporting skilled performance. Experiment 1 determined the practice necessary to obtain automaticity. In Experiments 2a, 3a, and 4a, which investigated the nature of the representation after extended practice, changing neither the identity nor color of elements had strong effects on transfer, but changing pattern orientation did impair memory retrieval, thus suggesting that for the counting task, pattern orientation is more important than element identity or color. Experiments 2b, 3b, and 4b replicated these results after limited practice.

Sophisticated topology of hidden Markov models for cursive script recognition

Abstract: The paper describes an adaptation of hidden Markov models (HMM) to automatic recognition of unrestricted handwritten words. Many interesting details of a 50,000 vocabulary recognition system for US city names are described. This system includes feature extraction, classification, estimation of model parameters, and word recognition. The feature extraction module transforms a binary image to a sequence of feature vectors. The classification module consists of a transformation based on linear discriminant analysis and Gaussian soft-decision vector quantizers which transform feature vectors into sets of symbols and associated likelihoods. Symbols and likelihoods form the input to both HMM training and recognition. HMM training performed in several successive steps requires only a small amount of gestalt labeled data on the level of characters for initialization. HMM recognition based on the Viterbi algorithm runs on subsets of the whole vocabulary. >

A Review of Probabilistic, Fuzzy, and Neural Models for Pattern Recognition

Abstract: Fuzzy sets were introduced by Zadeh in 1965 to represent and manipulate data and information that possess nonstatistical uncertainty. Computational neural networks were first discussed by McCullough and Pitts in 1943 as a means of imitating the power of biologic systems for data and information processing. Probabilistic models for data analysis, are, of course, several hundred years old. This article discusses the basic ideas of and some synergisms between probabilistic, fuzzy, and computational neural networks models as they apply to pattern recognition. We also provide a brief discussion of the relationship of both approaches to statistical pattern recognition methodologies.

The behavior-knowledge space method for combination of multiple classifiers

Abstract: A new method called the "Behavior-Knowledge Space Method" halJ been developed which can efficientlr aggregate decilJionlJ ol the individual c1alJlJifierlJ and de­ rive better relJuitlJ lor the unconlJtrained handwritten character recognition. It containlJ two IJtages: (1) the knowledge-modeling IJtage, which enroctlJ knowledge from the former behavior of classifiers and constructlJ a K-dimenlJional behavior-knowledge space; and (8) the operation stage, which is carried out lor each telJt sample, and which combines decisions generoted from individual classifiers, enters a specific unit 01 the con­ structed space, and makes a final decision by a rule which utilizelJ the knowledge inside the unit. Many important properties can be derived from this method which make it very promising.

Voice identification using nearest-neighbor distance measure

Abstract: An algorithm for attributing a sample of unconstrained speech to one of several known speakers is described. The algorithm is based on measurement of the similarity of distributions of features extracted from reference speech samples and from the sample to be attributed. The measure of feature distribution similarity employed is not based on any assumed form of the distributions involved. The theoretical basis of the algorithm is examined, and a plausible connection is shown to the divergence statistic of Kullback (1972). Experimental results are presented for the King telephone database and the Switchboard database. The performance of the algorithm is better than that reported for algorithms based on Gaussian modeling and robust discrimination. >

A connectionist approach to speech recognition

Abstract: The task discussed in this paper is that of learning to map input sequences to output sequences. In particular, problems of phoneme recognition in continuous speech are considered, but most of the discussed techniques could be applied to other tasks, such as the recognition of sequences of handwritten characters. The systems considered in this paper are based on connectionist models, or artificial neural networks, sometimes combined with statistical techniques for recognition of sequences of patterns, stressing the integration of prior knowledge and learning. Different architectures for sequence and speech recognition are reviewed, including recurrent networks as well as hybrid systems involving hidden Markov models.

Biosignal pattern recognition and interpretation systems

Abstract: A general framework is given to describe pattern recognition and interpretation. Pattern analysis stages are described, with consideration of difficulties in implementation and uncertainties present at each level. The main forms of pattern analysis-statistical, syntactic, and artificial intelligence (connectionist and symbolic) methods-have different strengths and weaknesses, depending on the stage of pattern analysis at which they are used. In general, statistical, syntactic, and connectionist techniques are used for pattern recognition, and statistical and symbolic techniques are used for pattern interpretation. Largely, pattern interpretation involves reasoning with uncertainty. Multichannel recordings increase the information available about specific physiologic events, at the expense of processing complexity. >

Sparse-pixel recognition of primitives in engineering drawings

Abstract: Recognition of primitives in technical drawings is the first stage in their higher level interpretation. It calls for processing of voluminous scanned raster files. This is a difficult task if each pixel must be addressed at least once, as required by Hough transform or thinning-based methods. This work presents a set of algorithms that recognize drawing primitives by examining the raster file sparsely. Bars (straight line segments), arcs, and arrowheads are identified by the orthogonal zig-zag, perpendicular Bisector tracing, and self-supervised arrowhead recognition algorithms, respectively. The common feature of these algorithms is that rather than applying massive pixel addressing, they recognize the sought primitives by screening a carefully selected sample of the image and focusing attention on identified key areas. The sparse-pixel-based algorithms yield high quality recognition, as demonstrated on a sample of engineering drawings.

Three-Dimensional Object Recognition Systems

Abstract: Contributors. 3-D object recognition: Inspirations and lessons from biological vision. Range sensing for computer vision. Feature extraction for 3-D model building and object recognition. Three-dimensional surface reconstruction: Theory and implementation. CAD-based object recognition in range images using pre-compiled strategy trees. Active 3-D object models. Image prediction for computer vision. Tools for 3-D object location from geometrical features by monocular vision. Part-based modeling and qualitative recognition. Appearance-based vision and the automatic generation of object recognition programs. Recognizing 3-D objects using constrained search. Recognition of superquadric models in dense range data. Recognition by alignment. Representations and algorithms for 3-D curved object recognition. Structural indexing: efficient three dimensional object recognition. Building a 3-D world model for outdoor scenes from multiple sensor data. Understanding object configurations. Modal descriptions for modeling, recognition, and tracking. Function-based generic recognition for multiple object categories.

Pattern recognition: neural networks in perspective

Abstract: Invariant pattern recognition will be a problem facing neural networks for some time, and the challenge is to overcome the limitation of Hamming distance generalization. Four representative architectures that are able to generalize are reviewed. The architectures are the backpropagation network, the ART architecture, the dynamic link architecture, and associate memories. Image representation, segmentation, and invariance are discussed. >

Design of a discriminating fingerprint for G-protein-coupled receptors.

Abstract: A systematic method for designing discriminating protein sequence fingerprints is described. The approach used is iterative, and diagnostic performance is evaluated in terms of the relative abilities of sequences to match with individual elements of the fingerprint. The method allows complete protein folds to be characterized in terms of a number of separate 'features', without the requirement to define specific intervals between them, and is described here with reference to the derivation of a fingerprint for G-protein-coupled receptors: this comprises the seven hydrophobic regions shown by protein chemistry approaches to be membrane-spanning. The fingerprint is potently diagnostic of all sequences of this type in the database in which it was derived (the OWL composite sequence database, version 8.1), and has continued to perform well on subsequent database updates, identifying 240 receptors in OWL17.0. Results are compared with a commonly used pattern template for this class of receptors. The investigation suggests that discriminating power is improved in the fingerprint approach because the recognition of individual features is made mutually conditional. Furthermore, by avoiding the definition of predetermined feature separations, members of protein families possessing all or only part of the fingerprint may be identified.

Writer-adaptation for on-line handwritten character recognition

Abstract: The authors have designed a writer-adaptable character recognition system for online characters entered on a touch terminal. It is based on a Time Delay Neural Network (TDNN) that is pre-trained on examples from many writers to recognize digits and uppercase letters. The TDNN without its last layer serves as a preprocessor for an optimal hyperplane classifier that can be easily retrained to peculiar writing styles. This combination allows for fast writer-dependent learning of new letters and symbols. The system is memory and speed efficient. >

On-line Japanese character recognition experiments by an off-line method based on normalization-cooperated feature extraction

Abstract: It is shown that an offline character recognition method is effective for use in an online Japanese character recognition. Major conventional online recognition methods have restricted the number and the order of strokes. The offline method removes these restrictions, based on pattern matching of orientation feature patterns. It has been improved with developments in nonlinear shape normalization, nonlinear pattern matching, and the normalization-cooperated feature extraction method. It was used to examine 52,944 online Kanji characters in 1,064 categories. The recognition rate achieved 95.1%, and the cumulation recognition rate within the best five candidates was 99.3%. >

Incidental concept learning, feature frequency, and correlated properties.

Abstract: Four experiments examined sensitivity to feature frequencies and feature correlations as a function of intentional and incidental concept learning. Feature frequencies were encoded equally well across variations in learning strategies, and although classification decisions in both intentional and incidental conditions preserved correlated features, this sensitivity was achieved through different processes. With intentional learning, sensitivity to correlations resulted from explicit rules, whereas incidental encoding preserved correlations through a similarity-based analogical process. In incidental tasks that promoted exemplar storage, classification decisions were mediated by similarity to retrieved examples, and correlated features were indirectly preserved in this process. The results are discussed in terms of the diversity of encoding processes and representations that can occur with incidental category learning.

Applications of feature recognition in the product life-cycle

Abstract: The concept of form features originated in the process planning of machined parts. However, it is now perceived to have applications in many other phases of the engineering process, from initial design through to inspection and assembly, and, as in process planning, the use of form features appears to be crucial in the automation of these activities. From the viewpoints of the different processes occurring in the product life-cycle a given product has different characterizations in terms of form features. This paper explores some of the implications of the resulting requirement for multiple feature models. A designer will most naturally wish to generate a CAD model in terms of functional features, and the system should then be capable of transmuting the designer's feature model into the appropriate feature model for any desired downstream application. This appears to be a simpler process than that of automatic feature recognition from a pure geometric model, a currently popular but difficult rese...

Representation scheme for defining and operating form features

Abstract: A representation scheme is presented that is suitable for defining and operating form features. Five basic constituents, B-rep solid components, measure entities, size, location and constraints, are used to represent a single form feature. With these constituents, form features are defined structurally and procedurally. Parts are then represented by a higher-level structure, called a feature-dependency graph, in which feature-position operators link the form features. In the FDG model, dimensions are used to determine the size and locations of form features. This property makes the manipulation of form features more consistent with the designer's intent. A C ++ implementation of the physical data structure according to the logical representation scheme is discussed.

Feature-based component model for computer-aided process planning systems

Abstract: The aim of this paper is to report on some of our research findings in developing generative process planning systems. A feature-based model for describing component geometry and connectivity, and a hierarchical structure for form features definition and classification are presented, as are methods for representing the capabilities of machine tools. These models form the basis for decision making in our prototype process planning system GENPLAN. As an example, the paper reports on how the models are being used for reasoning about component geometry and the linguistic approach adopted for machine tool selection in process planning. The component and processing system models and the linguistic approach used for representing their information are proving useful for geometric reasoning and machine tool selection tasks in computer aided process planning.

Word recognition in a segmentation-free approach to OCR

Abstract: Segmentation is a key step in current OCR systems. It has been estimated that half the errors in character recognition are due to segmentation. A novel approach that performs OCR without the segmentation step was developed. The approach starts by extracting significant geometric features from the input document image of the page. Each feature then votes for the character that could have generated that feature. Thus, even if some of the features are occluded or lost due to degradation, the remaining features can successfully identify the character. In extreme cases, the degradation may be severe enough to prevent recognition of some of the characters in a word. In such cases, a lexicon-based word recognition technique is used to resolve ambiguity. Inexact matching and probabilistic evaluation used in the technique make it possible to identify the correct word, by detecting a partial set of characters. The authors first present an overview of their segmentation-free OCR system and then focus on the word recognition technique. Preliminary experimental results show that this is a very promising approach. >