The methodology used to construct tree structured rules is the focus of this monograph. Unlike many other statistical procedures, which moved from pencil and paper to calculators, this text's use of trees was unthinkable before computers. Both the practical and theoretical sides have been developed in the authors' study of tree methods. Classification and Regression Trees reflects these two sides, covering the use of trees as a data analysis method, and in a more mathematical framework, proving some of their fundamental properties.

Classification and regression trees

Probability Distributions.- Linear Models for Regression.- Linear Models for Classification.- Neural Networks.- Kernel Methods.- Sparse Kernel Machines.- Graphical Models.- Mixture Models and EM.- Approximate Inference.- Sampling Methods.- Continuous Latent Variables.- Sequential Data.- Combining Models.

Pattern Recognition and Machine Learning

An efficient and intuitive algorithm is presented for the design of vector quantizers based either on a known probabilistic model or on a long training sequence of data. The basic properties of the algorithm are discussed and demonstrated by examples. Quite general distortion measures and long blocklengths are allowed, as exemplified by the design of parameter vector quantizers of ten-dimensional vectors arising in Linear Predictive Coded (LPC) speech compression with a complicated distortion measure arising in LPC analysis that does not depend only on the error vector.

/pdf/an-algorithm-for-vector-quantizer-design-266rr8cpmt.pdf

An Algorithm for Vector Quantizer Design

In the choice of an estimator for the spectrum of a stationary time series from a finite sample of the process, the problems of bias control and consistency, or "smoothing," are dominant. In this paper we present a new method based on a "local" eigenexpansion to estimate the spectrum in terms of the solution of an integral equation. Computationally this method is equivalent to using the weishted average of a series of direct-spectrum estimates based on orthogonal data windows (discrete prolate spheroidal sequences) to treat both the bias and smoothing problems. Some of the attractive features of this estimate are: there are no arbitrary windows; it is a small sample theory; it is consistent; it provides an analysis-of-variance test for line components; and it has high resolution. We also show relations of this estimate to maximum-likelihood estimates, show that the estimation capacity of the estimate is high, and show applications to coherence and polyspectrum estimates.

Spectrum estimation and harmonic analysis

This paper focuses on the class of speech enhancement systems which capitalize on the major importance of the short-time spectral amplitude (STSA) of the speech signal in its perception. A system which utilizes a minimum mean-square error (MMSE) STSA estimator is proposed and then compared with other widely used systems which are based on Wiener filtering and the "spectral subtraction" algorithm. In this paper we derive the MMSE STSA estimator, based on modeling speech and noise spectral components as statistically independent Gaussian random variables. We analyze the performance of the proposed STSA estimator and compare it with a STSA estimator derived from the Wiener estimator. We also examine the MMSE STSA estimator under uncertainty of signal presence in the noisy observations. In constructing the enhanced signal, the MMSE STSA estimator is combined with the complex exponential of the noisy phase. It is shown here that the latter is the MMSE estimator of the complex exponential of the original phase, which does not affect the STSA estimation. The proposed approach results in a significant reduction of the noise, and provides enhanced speech with colorless residual noise. The complexity of the proposed algorithm is approximately that of other systems in the discussed class.

Speech enhancement using a minimum-mean square error short-time spectral amplitude estimator

Several properties, interrelations, and interpretations are developed for various speech spectral distortion measures. The principle results are 1) the development of notions of relative strength and equivalence of the various distortion measures both in a mathematical sense corresponding to subjective equivalence and in a coding sense when used in minimum distortion or nearest neighbor speech processing systems; 2) the demonstration that the Itakura-Saito and related distortion measures possess a property similar to the triangle inequality when used in nearest neighbor systems such as quantization and cluster analysis; and 3) that the Itakura-Saito and normalized model distortion measures yield efficient computation algorithms for generalized centroids or minimum distortion points of groups or clusters of speech frames, an important computation in both classical cluster analysis techniques and in algorithms for optimal quantizer design. We also argue that the Itakura-Saito and related distortions are well-suited computationally, mathematically, and intuitively for such applications.

Distortion measures for speech processing

A new likelihood maximization algorithm called the /spl alpha/-EM algorithm (/spl alpha/-expectation-maximization algorithm) is presented. This algorithm outperforms the traditional or logarithmic EM algorithm in terms of convergence speed for an appropriate range of the design parameter /spl alpha/. The log-EM algorithm is a special case corresponding to /spl alpha/=-1. The main idea behind the /spl alpha/-EM algorithm is to search for an effective surrogate function or a minorizer for the maximization of the observed data's likelihood ratio. The surrogate function adopted in this paper is based upon the /spl alpha/-logarithm which is related to the convex divergence. The convergence speed of the /spl alpha/-EM algorithm is theoretically analyzed through /spl alpha/-dependent update matrices and illustrated by numerical simulations. Finally, general guidelines for using the /spl alpha/-logarithmic methods are given. The choice of alternative surrogate functions is also discussed.

The /spl alpha/-EM algorithm: surrogate likelihood maximization using /spl alpha/-logarithmic information measures

Database retrieval for similar images using ICA and PCA bases

Harmonic competition is a learning strategy based upon winner-take-all or winner-take-quota with respect to a composite of heterogeneous subcosts This learning is unsupervised and organizes itself The subcosts may conflict with each other Thus, the total learning system realizes a self-organizing multiple criteria optimization The subcosts are combined additively and multiplicatively using adjusting parameters For such a total cost, a general successive learning algorithm is derived first Then, specific problems in the Euclidian space are addressed Vector quantization with various constraints and traveling salesperson problems are selected as test problems The former is a typical class of problems where the number of neurons is less than that of the data The latter is an opposite case Duality exists in these two classes In both cases, the combination parameters of the subcosts show wide dynamic ranges in the course of learning It is possible, however, to decide the parameter control from the structure of the total cost This method finds a preferred solution from the Pareto optimal set of the multiple object optimization Controlled mutations motivated by genetic algorithms are proved to be effective in finding near-optimal solutions

Harmonic competition: a self-organizing multiple criteria optimization

Competitive learning in neural networks involving cooperation and categorization is discussed. Extended vehicle routing problems in the Euclidean space are also discussed. A fixed number of vehicles with a shared depot make subtours around precategorized cities and collect demands. The minimal tour length and even loaded demands are conflicting requirements for the optimization. This situation does not appear in a simple traveling salesman problem. The self-organization method gives qualified approximate solutions without computational backtracks. Experiments were made on the USA532 set. All computations can be carried out by a conventional workstation. >

Yasuo Matsuyama

Papers

Distortion measures for speech processing

The /spl alpha/-EM algorithm: surrogate likelihood maximization using /spl alpha/-logarithmic information measures

Database retrieval for similar images using ICA and PCA bases

Harmonic competition: a self-organizing multiple criteria optimization

Self-organization via competition, cooperation and categorization applied to extended vehicle routing problems