Survey of previous comparisons and theoretical work descriptions of methods dataset descriptions criteria for comparison and methodology (including validation) empirical results machine learning on machine learning.

Machine Learning: Neural and Statistical Classification

Pattern classification using neural networks and statistical methods is discussed. We give a tutorial overview in which popular classifiers are grouped into distinct categories according to their underlying mathematical principles; also, we assess what makes a classifier neural. The overview is complemented by two case studies using handwritten digit and phoneme data that test the performance of a number of most typical neural-network and statistical classifiers. Four methods of our own are included: reduced kernel discriminant analysis, the learning k-nearest neighbors classifier, the averaged learning subspace method, and a version of kernel discriminant analysis.

Neural and statistical classifiers-taxonomy and two case studies

The Mean Shift procedure is a well established clustering technique that is widely used in imaging applications such as image and video segmentation, denoising, object tracking, texture classification, and others. However, the Mean Shift procedure has relatively high time complexity which is superlinear in the number of data points. In this paper we present a novel fast Mean Shift procedure which is based on the random sampling of the Kernel Density Estimate (KDE). We show theoretically that the resulting reduced KDE is close to the complete data KDE, to within a given accuracy. Moreover, we prove that the time complexity of the proposed fast Mean Shift procedure based on the reduced KDE is considerably lower than that of the original Mean Shift; the typical gain is of several orders for big data sets. Experiments show that image and video segmentation results of the proposed fast Mean Shift method are similar to those based on the standard Mean shift procedure. We also present a new application of the Fast Mean Shift method to the efficient construction of graph hierarchies for images; the resulting structure is potentially useful for solving computer vision problems which can be posed as graph problems, including stereo, semi-automatic segmentation, and optical flow.

Fast Mean Shift by compact density representation

A conventional feature compensation module for robust automatic speech recognition is usually designed separately from the training of hidden Markov model (HMM) parameters of the recognizer, albeit a maximum-likelihood (ML) criterion might be used in both designs. In this paper, we present an environment-compensated minimum classification error (MCE) training approach for the joint design of the feature compensation module and the recognizer itself. The feature compensation module is based on a stochastic vector mapping function whose parameters have to be learned from stereo data in a previous approach called SPLICE. In our proposed MCE joint design approach, by initializing the parameters with an approximate ML training procedure, the requirement of stereo data can be removed. By evaluating the proposed approach on Auroral connected digits database, a digit recognition error rate, averaged on all three test sets, of 5.66% is achieved for multicondition training. In comparison with the performance achieved by the baseline system using ETSI advanced front-end, our approach achieves an additional overall error rate reduction of 12.4%

/pdf/an-environment-compensated-minimum-classification-error-5b19ptp6io.pdf

An Environment-Compensated Minimum Classification Error Training Approach Based on Stochastic Vector Mapping

The problem of reducing the number of terms in a kernal density estimator is considered. An online algorithm is proposed, where the kernel centers are tuned using the self-organizing feature map. The performance of the algorithm is compared with standard methods in numerical simulations. >

The self-organizing reduced kernel density estimator

Based on the assumption that most probability densities in real life can be approximated by a mixture of Gaussian densities, we propose here a three-layer adaptive network with each neuron in the lower hidden layer representing a Gaussian basis function (covariance matrix equal to where I is a unit matrix) to estimate various probability densities and serve as a Bayes classifier. The width of the basis function may be the same for all neurons in this layer or it may vary from one neuron to another. This paper investigates the effectiveness of the network for both cases and presents a localized learning algorithm to adjust the network parameters. The network was trained with artificial data derived from known mixtures of memoryless Gaussian sources as well as exponential and Gamma densities. The performance of the network as a pattern density estimator was measured in terms of the relative difference between the target probability density function (p.d.f.) which generates the training and testing data and the network output representing the estimation. Samples from two mixtures corresponding to two classes were used to test the network capability as a classifier by comparing its error rate against that of a Bayes classifier. Both one- and two-dimensional cases were explored. The successfulness of the network depended on how well the target p.d.f.’s were represented by the training samples, the number of hidden neurons employed in the network and how thoroughly the network was trained. It was also found that allowing each basis function to have an independent width had a predominant effect on the network performance.

A three-layer adaptive network for pattern density estimation and classification

A preliminary study on the static representation of short-timed speech dynamics.

Recognition of phonetic labels of the TIMIT speech corpus by means of an artificial neural network

This paper proposes a novel method of speaker normalization by means of input space optimization for continuous density hidden Markov models (CDHMM). The parameters of a linear feature transformation function are so determined that, together with the previously trained CDHMM parameters, a mis-classification cost function is minimized for the normalizing data set. Preliminary experimental results on the task of sex adaptation for speaker-independent stop consonant discrimination, evaluated from the DARPA TIMIT speech database, demonstrates the effectiveness of the proposed method. >

Jian-xiong Wu

Papers

A three-layer adaptive network for pattern density estimation and classification

A preliminary study on the static representation of short-timed speech dynamics.

Recognition of phonetic labels of the TIMIT speech corpus by means of an artificial neural network

Speaker normalization by input space optimization for continuous density hidden Markov models

Maximum Likelihood Density Estimation by means of an Artificial Neural Network.