Adaptive Systems in Pattern Recognition
TL;DR: The perceptual superiority of the human visual system over automata is outlined comparing the properties of both systems and the existing classification methods are outlined and discussed with regard to adaptive systems.
Abstract: The perceptual superiority of the human visual system over automata is outlined comparing the properties of both systems. The most effective property with regard to pattern recognition is the internal adaptability and the ability of abstracting. Both properties are well performed by human beings. A mechanical perceptor for complex pattern recognition must also have these capabilities. The use of adaptation for pattern recognition is discussed. The realization of these properties by machines is difficult, especially the development of an adequate feature generator which performs the internal adaptability and thus solves the problem of identification-criteria invariance of patterns. This is assumed to be the main task in pattern recognition research. External teaching processes may be accomplished by adaptive categorizers. The existing classification methods are outlined and discussed with regard to adaptive systems. Adaptive categorizers of a learning matrix type and a perceptron type are compared as to structure, linear classification performance, and training routine. It is assumed, however, that the somewhat passive external adaptation of categorizers must be supplemented by a more active adaptation by the system itself.
Citations
More filters
Book•
01 Jan 1996
TL;DR: The Bayes Error and Vapnik-Chervonenkis theory are applied as guide for empirical classifier selection on the basis of explicit specification and explicit enforcement of the maximum likelihood principle.
Abstract: Preface * Introduction * The Bayes Error * Inequalities and alternatedistance measures * Linear discrimination * Nearest neighbor rules *Consistency * Slow rates of convergence Error estimation * The regularhistogram rule * Kernel rules Consistency of the k-nearest neighborrule * Vapnik-Chervonenkis theory * Combinatorial aspects of Vapnik-Chervonenkis theory * Lower bounds for empirical classifier selection* The maximum likelihood principle * Parametric classification *Generalized linear discrimination * Complexity regularization *Condensed and edited nearest neighbor rules * Tree classifiers * Data-dependent partitioning * Splitting the data * The resubstitutionestimate * Deleted estimates of the error probability * Automatickernel rules * Automatic nearest neighbor rules * Hypercubes anddiscrete spaces * Epsilon entropy and totally bounded sets * Uniformlaws of large numbers * Neural networks * Other error estimates *Feature extraction * Appendix * Notation * References * Index
3,598 citations
TL;DR: The survey discusses distance functions, smoothing parameters, weighting functions, local model structures, regularization of the estimates and bias, assessing predictions, handling noisy data and outliers, improving the quality of predictions by tuning fit parameters, and applications of locally weighted learning.
Abstract: This paper surveys locally weighted learning, a form of lazy learning and memory-based learning, and focuses on locally weighted linear regression. The survey discusses distance functions, smoothing parameters, weighting functions, local model structures, regularization of the estimates and bias, assessing predictions, handling noisy data and outliers, improving the quality of predictions by tuning fit parameters, interference between old and new data, implementing locally weighted learning efficiently, and applications of locally weighted learning. A companion paper surveys how locally weighted learning can be used in robot learning and control.
1,863 citations
Cites background from "Adaptive Systems in Pattern Recogni..."
...Special purpose hardware for nding nearest neighbors has a long history (Taylor, 1959, 1960; Steinbuch, 1961; Steinbuch and Piske, 1963; Kazmierczak and Steinbuch, 1963; Batchelor, 1974)....
[...]
01 Jan 1983
TL;DR: The study and computer modeling of learning processes in their multiple manifestations constitutes the subject matter of machine learning.
Abstract: Learning is a many-faceted phenomenon. Learning processes include the acquisition of new declarative knowledge, the development of motor and cognitive skills through instruction or practice, the organization of new knowledge into general, effective representations, and the discovery of new facts and theories through observation and experimentation. Since the inception of the computer era, researchers have been striving to implant such capabilities in computers. Solving this problem has been, and remains, a most challenging and fascinating long-range goal in artificial intelligence (AI). The study and computer modeling of learning processes in their multiple manifestations constitutes the subject matter of machine learning.
383 citations
IBM1
TL;DR: This paper reviews statistical, adaptive, and heuristic techniques used in laboratory investigations of pattern recognition problems and includes correlation methods, discriminant analysis, maximum likelihood decisions minimax techniques, perceptron-like algorithms, feature extraction, preprocessing, clustering and nonsupervised learning.
Abstract: This paper reviews statistical, adaptive, and heuristic techniques used in laboratory investigations of pattern recognition problems. The discussion includes correlation methods, discriminant analysis, maximum likelihood decisions minimax techniques, perceptron-like algorithms, feature extraction, preprocessing, clustering and nonsupervised learning. Two-dimensional distributions are used to illustrate the properties of the various procedures. Several experimental projects, representative of prospective applications, are also described.
317 citations
TL;DR: A memory-based local modeling approach (locally weighted regression) is used to represent a learned model of the task to be performed, and an exploration algorithm is developed that explicitly deals with prediction accuracy requirements during exploration.
Abstract: Issues involved in implementing robot learning for a challenging dynamic task are explored in this article, using a case study from robot juggling. We use a memory-based local modeling approach (locally weighted regression) to represent a learned model of the task to be performed. Statistical tests are given to examine the uncertainty of a model, to optimize its prediction quality, and to deal with noisy and corrupted data. We develop an exploration algorithm that explicitly deals with prediction accuracy requirements during exploration. Using all these ingredients in combination with methods from optimal control, our robot achieves fast real-time learning of the task within 40 to 100 trials. >
270 citations
References
More filters
TL;DR: This paper examines the notion of a single number statistic for each characteristic which would have certain desirable properties related to the "goodness" of the characteristic, and shows that, in general, no such statistic exists.
Abstract: Design of pattern recognition systems usually involves a number of uncertainties which can be resolved only by experiment. In statistical recognition systems, pattern information is stored in the machine in the form of pattern characteristics with statistics relating the characteristics to the patterns. If the number of characteristics a system can process is limited, and if the system designer, while able to conceive of a large number of relevant characteristics, does not know which are the most important, then an experimental selection of these characteristics is required. As tools, the designer may have at his disposal a large computer, a large sample of the patterns to be recognized, and a set of programs to measure the characteristics. Thus, he can compare certain statistics relating the patterns to the characteristics. The problem is: what statistics should he calculate in order to select the best characteristics? Assuming the characteristics to be independent in their effect on the decision, this paper examines the notion of a single number statistic for each characteristic which would have certain desirable properties related to the "goodness" of the characteristic. It is shown that, in general, no such statistic exists. However, a statistic is proposed which, while not having these properties in general, at least has them in a wide range of situations. An experimental study of the validity of this choice is reported together with the design of a letter recognition system. Using a sample of 15 complete 62 symbol alphabets, 13 characteristics were selected. The resulting system recognized correctly 81.9% of the letters presented to it.
169 citations
01 Jun 1962
TL;DR: This paper is concerned with the study of a particular class of categorizers, the linear decision function, which can be empirically designed without making any assumptions whatsoever about either the distribution of the receptor measurements or the a priori probabilities of occurrence of the pattern classes, providing an appropriate pattern source is available.
Abstract: Many pattern recognition machines may be considered to consist of two principal parts, a receptor and a categorizer. The receptor makes certain measurements on the unknown pattern to be recognized; the categorizer determines from these measurements the particular allowable pattern class to which the unknown pattern belongs. This paper is concerned with the study of a particular class of categorizers, the linear decision function. The optimum linear decision function is the best linear approximation to the optimum decision function in the following sense: 1) "Optimum" is taken to mean minimum loss (which includes minimum error systems). 2) "Linear" is taken to mean that each pair of pattern classes is separated by one and only one hyperplane in the measurement space. This class of categorizers is of practical interest for two reasons: 1) It can be empirically designed without making any assumptions whatsoever about either the distribution of the receptor measurements or the a priori probabilities of occurrence of the pattern classes, providing an appropriate pattern source is available. 2) Its implementation is quite simple and inexpensive. Various properties of linear decision functions are discussed. One such property is that a linear decision function is guaranteed to perform at least as well as a minimum distance categorizer. Procedures are then developed for the estimation (or design) of the optimum linear decision function based upon an appropriate sampling from the pattern classes to be categorized.
150 citations
TL;DR: A character recognition system has been developed for the recognition of handwritten numerals using a logically controlled cathode ray tube scanner to generate basic measurements that characterize significant features of the numeral shapes.
Abstract: A character recognition system has been developed for the recognition of handwritten numerals. This system uses a logically controlled cathode ray tube scanner to generate basic measurements that characterize significant features of the numeral shapes. A contour-follower procedure is used to control the scanner. In addition, special scanner subroutines initiated by feedback from the recognition logic are utilized. Character shape data are generated in a sequential form, which can be analyzed for recognition with an easily realizable logic.
An experimental model has been built that recognized 99.3% of numerals written by 45 subjects after 30 minutes of training. The error rate ftohre se characters was 0.11%. The rejected character ratew as 0.59%.
91 citations
TL;DR: A computer program has been written to design character recognition logic based on the processing of data samples to search for logic circuits having certain constraints on hardware design and evaluate these logics in terms of their discriminating ability over samples of the character set they are expected to recognize.
Abstract: A computer program has been written to design character recognition logic based on the processing of data samples. This program consists of two subroutines: (1) to search for logic circuits having certain constraints on hardware design, and (2) to evaluate these logics in terms of their discriminating ability over samples of the character set they are expected to recognize. An executive routine is used to apply these subroutines to select a complete logic with a given performance and complexity. This logic consists of 39 to 96 AND gates connected to a shift register and a table look-up or resistance network comparison system.
The methods were applied to the design of recognitionl ogics for the 52 upper and lower case characters of IBM Electric Modern Pica type font and lower case Cyrillic characters scanned from Russian text. In both cases when the logics were tested on data different from that used to design the logics, the substitution rate was about one error per thousand. A single logic was designed to read two different Cyrillic fonts. For this design, an error rate of one error per hundred characters was observed.
Several experiments are reported ona number of logics designed for typewritten data, and single- and two-font Cyrillic data. The performances of different recognitionsy stems are compared as a function of the complexity of the recognition logics.
90 citations
TL;DR: It is shown that a small defect of stabilisation is important and that normal vision may be restored by introducing small movements which simulate some of the normal movements of a retinal image.
Abstract: A beam of light reflected from a mirror attached to a contact lens, worn by the subject, produces a test object whose image does not move across the retina in response to eye-movements. This is called the stabilised retinal image. In this paper, conditions for accurate stabilisation are discussed. It is shown that a small defect of stabilisation is important and that normal vision may be restored by introducing small movements which simulate some of the normal movements of a retinal image. Experiments on the stabilised image with light interrupted at different frequencies are described and it is shown that normal vision is obtained near the flicker fusion frequency. Un rayon lumineux, reflechi par un miroir fixe sur un verre de contact porte par le sujet, forme l'image d'un test objet dont l'image est immobile sur la retine quels que soient les mouvements du globe oculaire; cette image est dite « stabilisee ». Le present article traite des conditions requises pour une stabilisation tres precise. Il sera m...
85 citations