Showing papers on "Feature (machine learning) published in 1972"

Introduction to Statistical Pattern Recognition

Abstract: This completely revised second edition presents an introduction to statistical pattern recognition Pattern recognition in general covers a wide range of problems: it is applied to engineering problems, such as character readers and wave form analysis as well as to brain modeling in biology and psychology Statistical decision and estimation, which are the main subjects of this book, are regarded as fundamental to the study of pattern recognition This book is appropriate as a text for introductory courses in pattern recognition and as a reference book for workers in the field Each chapter contains computer projects as well as exercises

On linguistic, statistical and mixed models for pattern recognition

Abstract: The “pure” linguistic model is usually presented as a contrast to the “pure” statistical classification model. Neither pure model is relevant to applications. Rather, different models are appropriate for different kinds of prior information. The viewpoint espoused in this paper can be termed “heuristic complexity decomposition,” which corresponds to reducing a complex problem to a set of problems of complexity of lower order. It is shown that it is hybrid linguistic-statistical models that are useful in practice. This viewpoint reconciles the dichotomy that is currently believed to exist between “structural” and “geometric” approaches. We present a selective discussion of some aspects of linguistics, statistical and mixed approaches to pattern recognition, and consider the potential of transformational grammars, a proposed formalism for pattern analysis and recognition, and heuristic and formal procedures for inference of pattern grammars. We point out the utility of first-order predicate calculus formalism for pattern description in view of the availability of a large body of heuristic techniques for analysis.

On syntactic pattern recognition and stochastic languages

Abstract: Publisher Summary This chapter discusses syntactic pattern recognition and stochastic languages. Emphasis is on the description of noisy and/or distorted patterns and the learning of grammar from the actual pattern samples. It has been demonstrated by several very simple examples and sometimes with rather heuristic justifications, that the use of probability information in syntactic pattern recognition would make the syntactic approach more flexible and attractive. It is expected that the use of probability information in syntactic analysis, would probably improve the efficiency and flexibility of the analysis procedure. The many different techniques used to solve pattern recognition problems may be grouped into two general approaches: (1) decision-theoretic or discriminant approach and (2) syntactic or linguistic approach. In the decision-theoretic approach, a set of characteristic measurements are extracted from the patterns; the recognition of each pattern, assignment to a pattern class is usually made by partitioning the feature space The area of syntactic pattern recognition, though very promising, is still in its infancy. Many problems, such as primitive selection, flexible, and powerful pattern description techniques, and efficient analysis and inference procedures still need to be solved.

Structural pattern recognition: primitives and juxtaposition relations*

Abstract: Publisher Summary This chapter discusses structural pattern recognition. Research in pattern recognition deals usually either with the general classification techniques or with techniques strongly dependent on specific applications as typewritten character recognition, chromosome classification, etc. The major exception to this trend is linguistic methods dealing with picture processing. This includes printed or handwritten text, many biomedical pictures, recognition of objects in photographs through their silhouettes, certain types of wave-forms and all thin line pictures. It does not include recognition of faces in photographs or other similarly complex tasks. It is probably too optimistic to expect that mathematical techniques developed for other applications will provide a solution to the problem of shape recognition. The main effort will be to show how such pictures can be expressed in terms of simpler pictures and the juxtaposition relations of the latter. The end result will be a labeled graph. Further analysis of the graph can be made in a number of ways which will depend mostly on the specific problem. The chapter discusses the general features of approach and its connection to other methods rather than the analytical details involved.

Recognition of Handwritten Characters by Topological Feature Extraction and Multilevel Categorization

Abstract: A handwritten character recognition system has been designed by making use of topological feature extraction and multilevel decision making. By properly specifying a set of easily detectable topological features, it is possible to convert automatically the handwritten characters into stylized forms and to classify them into primary classes with similar topological configurations. Final recognition is accomplished by a secondary stage that performs local analysis on the characters in each primary category. The recognition system consists of two stages: global recognition, followed by local recognition. Automatic character stylization results in pattern clustering which simplifies the classification tasks considerably, while allowing a high degree of generality in the acceptable writing format. Simulation of this scheme on a digital computer has shown only 6 percent misrecognition.

Dynamic programming as applied to feature subset selection in a pattern recognition system

Abstract: This paper proposes dynamic programming search procedures to expedite the feature subset selection processes in a pattern recognition system. It is shown that in general the proposed procedures require far less number of subsets to be evaluated than the exhaustive search procedure. For example, a problem of selecting the best subset of 4 features from a set of 24 features requires an evaluation of [equation] = 10626 subsets by using the exhaustive search procedure; on the other hand, it requires only 175 and 136 subsets to be considered by employing the proposed procedures I and II, respectively, to solve the same problem. While the number of subsets to be evaluated for the dynamic programming search procedures is slightly greater than that for the without-replacement search procedure, the best feature subset selected by the proposed methods may, however, not necessarily contain all of the best single features selected in the previous stages.

Machine recognition of printed Chinese characters via transformation algorithms

Abstract: This paper presents some novel results concerning the recognition of single-font printed Chinese characters via the transformation algorithms of Fourier, Hadamard, and Rapid. The new design philosophy of a three-stage structure is believed to offer at least a suboptimal search strategy for recognizing printed Chinese characters with a dictionary of 7000 - 8000 characters. The transformation algorithms discussed in this paper will be used in the last two stages. Extensive experiments and simulations concerning feature extraction and noisy or abnormal pattern recognition have been carried out (the simulations have been restricted to a 63-character subset called "Radicals"). Comparison has been made of all three transforms according to their ability to recognize characters.

Automatic recognition of handwritten characters via feature extraction and multi-level decision

Abstract: Automatic recognition of handwritten alphanumeric characters is designed by making use of topological feature extraction and multi-level decision making. By properly specifying a set of easily detectable topological features, it is possible to convert automatically the handwritten characters into stylized forms and to classify them into primary categories. Each category contains one or several character pattern classes with similar topological configurations. Final recognition is accomplished by a secondary stage which performs local analysis on the characters in each primary category. The recognition system consists of two stages, global recognition followed by local recognition. Automatic character stylization results in pattern clustering which simplifies the classification tasks considerably, while allowing a high degree of generality in the acceptable writing format. Simulation of this scheme on a digital computer has shown only 2% misrecognition.

Feature enhancement character recognition system

Abstract: A character recognition system in which the degree of acceptability of possible recognition decisions is determined and in which the recognition problem of choosing between a large number of characters may be reduced to a decision between a small number of characters which decision may be performed by specialized logic. A memory is provided which has stored information corresponding to features which could be found in each of the characters to be recognized. A second memory is provided which has stored information corresponding to features which should not be found in each of the characters. An up-down counting means is associated with each of the characters and with both memories. The memories are read out to the counters in accordance with feature signals inputted thereto and after all feature signals for a character have been inputted the counts obtained by the counters are evaluated. If any counter has greater than a predetermined minimum count and another counter has a count within a second predetermined minimum count away, a character pair is defined which is inputted to specialized logic to determine which of the two characters has been read.

On the Correctness of Some Sequential Classification Schemes in Pattern Recognition

Abstract: This correspondence points out some discrepancies observed in [2]-[5] in the formulation of the sequential decision process for simplifying approximations to the probability distributions. The authors use two mutually contradictory conditions in introducing the approximations. The correct form of the functional equation for statistically independent features is presented.

Adaptive pattern recognition: a state-variable approach

Abstract: A state-variable approach to Bayes-optimal adaptive pattern recognition is presented for continuous data systems Both structure and parameter adaptation, as well as supervised and unsupervised learning are considered and Bayes-optimal as well as suboptimal, recursive recognition algorithms are given The state-variable approach consists of modeling random processes involved as the outputs of dynamic systems, linear or nonlinear, excited by white noise, and describing the systems in state-variable form Several fundamental pattern recognition results obtained using the state-variable approach are discussed Specifically, for the class of adaptive pattern recognition problems with signals modeled by nonlinear dynamic systems excited by white gaussian noise and observed in white gaussian noise, the following results are presented and discussed a) The fundamental relationship between pattern recognition and estimation is established Namely, it is shown that pattern recognition/detection constitutes mean-square nonlinear estimation; b) A “partition theorem” is derived that enables decomposition of the nonlinear adaptive pattern recognition system into two parts, a nonadaptive part consisting of recursive matched filters, and an adaptive part that incorporates the learning nature of the adaptive recognition system; c) For the special class of pattern recognition problems with linear dynamic models, the “partition theorem” partitions the nonlinear adaptive recognition system into a linear nonadaptive part consisting of Kalman filters, and a nonlinear adaptive part; d) Several simplified recursive recognition algorithms are presented with substantial computational advantages and high performance; and finally, e) Recursive and computationally efficient algorithms are given for the on-line performance evaluation of the adaptive recognition systems Moreover, two special cases are considered, namely that of supervised learning, treated previously by Lainiotis, and the case of independent signalling random processes For the special case of independent signalling random processes, the results for continuous data are similar to those obtained by Fralick for discrete, conditionally independent data Both deterministic decision-directed learning as well as random decision-directed learning algorithms (Agrawala's LPT) for continuous data are also obtained Moreover, suboptimal, recursive, unsupervised learning algorithms are obtained based on approximate nonlinear estimation procedures

A New Data Base for Syntax-Directed Pattern Analysis and Recognition

Abstract: The problem of developing an appropriate data base for syntax-directed pattern analysis and recognition is considered. A new data base is introduced by generalizing the notion of concatenation in representing patterns with a relationship matrix. The characteristics of relationship matrix are demonstrated in teh context of formal language theory. It is shown that this data base will allow us to remove many of the present restrictions placed on the types of patterns that can be handled by syntax-directed systems. Problems in pattern analysis (description and generation) as well as in pattern recognition are discussed and examples are given to illustrate the potential application of this data base in both of these areas.

End Points, Complexity, and Visual Illusions

Abstract: One aspect of a new theory of feature perception is considered. An algorithm is presented which can perceive and locate various features of a pattern by analyzing a statistic of the "chords" of the pattern. The procedure is illustrated by applying the algorithm to a pattern containing the Muller-Lyer figures. In measuring the length of the figures it is found that the algorithm has a visual illusion. A machine capable of executing the algorithm is described.

On feature extraction in multi-class pattern recognition†

Abstract: The problem of feature selection in multi-class pattern recognition is viewed as that of a mapping of vector samples from n-dimensional space to that in m-dimensional space (m

Experiments with an on-line picture language

Abstract: During the investigation of diverse pattern recognition problems it has become increasingly apparent that recognition of objects in a complex picture cannot be treated as an open loop process consisting of feature extraction and classification. A paradox is encountered in that reliable features can only be extracted if the object in the picture has been recognized, but the object cannot be recognized before its features are identified. The recognition process is better characterized by a closed-loop process containing an hypothesis testing procedure. Starting even from a single feature, which has been detected and recognized only to a certain degree as being a known feature, the machine will have to form an hypothesis as to which objects this feature may belong to and what additional information has to be gathered from the picture before the feature can be recognized with greater reliability. Only then can the machine improve on feature detection, search out other features expected to belong to the hypothesized object and decide whether its hypothesis can be maintained in the context of the given recognition problem. Some of the problems encountered in trying to realize such a system are discussed. The proposed solutions have been tested to some degree by using an online picture language program. A preliminary version of the online picture language has been completed. By using a controllable flying spot scanner the computer has direct access to a picture of about 4000 × 4000 resolution elements. The picture is treated as a read only memory and it may contain many different objects. The entire program consists of three major subdivisions labelled (a), (b), (c). (a) Atom Formation, Description and Recognition: A complicated object is fragmented (“atomized”) by using an algorithm which operates with or without guidance. If guided, the operator may only point out the approximate location of the atom that he proposes to the machine. The algorithm decides whether the area can be considered to be an atom and how much of the object it should include. A description of the atom is formed and normalized, before it is compared against already known “ideal” atoms. If the match is sufficiently good, the presented fragment is considered to be a realization of the best matching ideal atom. If the match is inadequate, the fragment becomes a new “ideal” atom. (b) Teaching of Object-Inter-relationships: When all the fragments (or a sufficient number of them) have been extracted, the operator instructs the computer how these atoms are to be interrelated for a given object. The operator can teach the machine expressions of the type: “If you see atom a1 (at location (xo, yo)), go search for atom a2 (at location (x1, y1))”, or “an object 02 consists of atoms a1, a2 and a3 (01 = a1 . a2 . a3)”, or “object 1 (01) is the same as object 2 (02), i.e., 01 = 02”. The position, size and rotation relationships between the atoms and objects are retained. (c) Recognition of Objects: In the recognition part of the system, the taught relationships are used to guide the machine from the identified atoms to the expected atoms until an object of the desired class has been identified or all the usable taught relationships have been exhausted. During running of the recognition programs the machine appeared to behave as if it “understood” the problem by going from one atom to the next in a systematic exploration of the picture. However, it also became apparent that the machine needs a goal-seeking algorithm in the recognition stage to guide it through the maze of instructions via the shortest path. At present the object fragmentation stage (a) can be run without human interference and there appear to exist no unsurmountable programming problems to automate step (b), i.e., to allow the machine to teach itself. The present programs are written for two-dimensional objects. Extension to three-dimensional objects is conceptually simple.

Predicate Calculus Feature Generation

Abstract: A summary description (proof of theorems omitted): An adaptive pattern representation and recognition strategy for application to mechanized interpretation of (sampled) pictorial data (other applications are appropriate) is described. The system generates its own features which are formulae in a subset of the weak (in the sense that only quantification over finite sets is permitted) second order predicate calculus. The models of such formulae define the "objects" in a description of the data, which is hierarchical both with respect to features and extensions. The hierarchy is automatically constructed, thereby implementing changes in "problem representation". Relations between the syntax and semantics of formulae in the weak second order predicate calculus are derived (by extending the syntax of the calculus) and utilized. Minimal use is made of the finiteness of the input data by the methods employed. That is, in pictorial pattern recognitions, the adaptive feature generation (i.e., "learning") algorithms are independent of the fineness of grain of the sampling of an input picture. Because this approach is used, many difficult problems of a purely mathematical nature acquire practical importance. Computation is reduced through the use of topological methods and the system is at present in a stage of development appropriate for programming and use in a variety of practical applications.

A Modified Training Procedure for Linear Threshold Devices

Abstract: A modification of conventional training techniques for linear threshold dichotomous pattern recognition is described. The modification is capable of convergence in some cases where the classical training techniques are not. The utility of this technique for signal detection problems is discussed.

Computer‐Assisted Reading of Spectrograms

Abstract: A simple user‐oriented computer language has been developed (using LISP) to be used as an aid in reading spectrograms of sentences containing words from a fixed vocabulary. The lexicon is stored in the computer in terms of phonetic segments and features. Questions such as “Give me all two‐word combinations that contain the following sequence of phonetic segments or features….” are easily translatable into simple and concise commands in this language. All commands are expressed in terms of phonetic segments or features which the user deduces from the spectrogram. In cases where segmentation is not obvious, a feature may be specified over a variable number of segments. It has been found that the use of such a language in conjunction with the reading of spectrograms provides insight into the kinds of acoustic analysis that are necessary in developing strategies for the machine recognition of connected speech. The language and some implications of its usage, based on experience with a 200‐word vocabulary, wil...

On a Method of Sequential Pattern Recognition

Abstract: In this paper, a multistage linear programming method of pattern recognition is proposed. The usual n-dimensional linear program has been split up into n stages of a one-dimensional linear program in such a way that more and more patterns belonging to two classes A and B are correctly classified as we proceed to higher and higher stages. It has been shown that if the threshold is kept fixed for ali the stages, the formulation of the problem entails a preprocessing of the feature coordinates; on the other hand, if the threshold is allowed to be different values in different stages, no such preprocessing is required. The application of the proposed method for the recognition of two handwritten Bengali characters has been discussed and compared with the results of other methods.

An Automatic Speech Recognition System Using a Vocoder Input.

Abstract: : Speech recognition is accomplished by off-line machine processes based on visual pattern recognition techniques. The fundamental system uses digitized data output from a KY-585 Vocoder, and two-dimensional discrete Fourier transforms with spatial frequency filters.. Two male speakers generated data for the computer processes which include a speaker adaptation routine. A relation to the human physiology is maintained through an elementary model. For a 39 word vocabulary, recognition rates reached 92% for the single speaker process, and 79% for an either-of-two-speaker process. Departures from visual pattern recognition techniques are introduced and proven effective. (Author)

A Cross‐Language Study of Perceptual Asymmetry

Abstract: Recent findings that the speechdominant hemisphere is specialized at the level of distinctive feature analysis entail a prediction that the processing of acoustic cues embodying a feature distinction in a particular language is asymmetric for “native” listeners, but not asymmetric for nonspeakers of the language if their own language does not employ the contrast. Russian‐ and English‐speaking listeners were presented dichotic sequences of syllables for identification of the consonants, i.e., the stops, categorized in Russian, but not in English, as palatalized or nonpalatalized. The results are discussed in the context of a recognition model equipped with a “filter” for feature selection.

On the Fisher criterion and divergence in pattern recognition

Abstract: The Fisher criterion is proposed for feature selection in pattern recognition and its relationship with the divergence is presented