Showing papers on "Dimensionality reduction published in 1990"

Hierarchical constrained automatic learning network for character recognition

Abstract: Highly accurate, reliable optical character recognition is afforded by a layered network having several layers of constrained feature detection wherein each layer of constrained feature detection includes a plurality of constrained feature maps and a corresponding plurality of feature reduction maps. Each feature reduction map is connected to only one constrained feature map in the same layer for undersampling that constrained feature map. Units in each constrained feature map of the first constrained feature detection layer respond as a function of a corresponding kernel and of different portions of the pixel image of the character captured in a receptive field associated with the unit. Units in each feature map of the second constrained feature detection layer respond as a function of a corresponding kernel and of different portions of an individual feature reduction map or a combination of several feature reduction maps in the first constrained feature detection layer as captured in a receptive field of the unit. The feature reduction maps of the second constrained feature detection layer are fully connected to each unit in the final character classification layer. Kernels are automatically learned by constrained back propagation during network initialization or training.

A dynamical systems approach to speech processing

Abstract: An approach to speech processing, based on nonlinear dynamical systems, is presented. It is shown that two fundamental problems in speech processing, dimensionality reduction and nonlinear temporal variability, can be addressed using geometrical methods from nonlinear dynamics. An effective dynamical system is extracted by training a nonlinear predictor of the signal samples. A variety of signal characteristics is then obtained from the properties of the resulting dynamical system such as the dimensionality and stability of its trajectories. The problem of time warping of speech is approached using a similar dynamical predictor, now acting directly on the acoustic features, provided that the magnitude of the time derivative of the feature vector is included in the predictor input. For the latter case the existence of a nonlinear predictor whose functional form is invariant with respect to nonlinear transformations of time is proven. The use of such dynamical predictors can replace or enhance existing methods for speech recognition. >

Extensions of a Theory of Networks for Approximation and Learning: Dimensionality Reduction and Clustering

Abstract: The theory developed in Poggio and Girosi (1989) shows the equivalence between regularization and a class of three-layer networks that we call regularization networks or Hyper Basis Functions. These networks are also closely related to the classical Radial Basis Functions used for interpolation tasks and to several pattern recognition and neural network algorithms. In this note, we extend the theory by defining a general form of these networks with two sets of modifiable parameters in addition to the coefficients $c_\alpha$: {\it moving centers} and {\it adjustable norm- weight}.

Hebbian feature discovery improves classifier efficiency

Abstract: Two neural network implementations of principal component analysis (PCA) are used to reduce the dimension of speech signals. The compressed signals are then used to train a feedforward classification network for vowel recognition. A comparison is made of classification performance, network size, and training time for networks trained with both compressed and uncompressed data. Results show that a significant reduction in training time, fivefold in the present case, can be achieved without a sacrifice in classifier accuracy. This reduction includes the time required to train the compression network. Thus, dimension reduction, as performed by unsupervised neural networks, is a viable tool for enhancing the efficiency of neural classifiers

Modified fisherrsquo;s linear discriminant function with reduction of dimensionality

Abstract: The use of principal components to reduce the number of dimensions is an optimum procedure for data representation, but may involve the loss of valuable information for discriminant analysis. In this paper a simple approach is proposed to improve the discrimination based on a given number (k) of principal components, without requiring the calculation of additional ones. This is achieved by introducing a metric which is a linear combination of the Mahalanobis distance in the subspace of the first k components and the euclidean distance in its orthogonal complement. This concept, applied to Fisher's linear discriminant function, yields an optimum combination of the two distances mentioned. The empirical performance of this procedure for estimated parameters is investigated by a simulation study. Some suggestions for further extensions of this method to nonlinear discrimination are briefly discussed.

Exploratory Feature Extraction in Speech Signals

Abstract: A novel unsupervised neural network for dimensionality reduction which seeks directions emphasizing multimodality is presented, and its connection to exploratory projection pursuit methods is discussed. This leads to a new statistical insight to the synaptic modification equations governing learning in Bienenstock, Cooper, and Munro (BCM) neurons (1982). The importance of a dimensionality reduction principle based solely on distinguishing features, is demonstrated using a linguistically motivated phoneme recognition experiment, and compared with feature extraction using back-propagation network.

Data Compression Using Multilayer Perceptrons

Abstract: This paper describes the use of Multilayer Pereeptrons to perform data compression tasks. It has been shown that an MLP with two fully connected layers of linear units, performs a principal component analysis when used for auto-association [2], Further work has been carried out in the field of MLP and data analysis which has proven that such networks can also perform discriminant analysis when used for classification [3]. The main property of MLPs is that the compressed representation of an input is the activity vector of the cells contained in the hidden layer, therefore the rank of dimensionality reduction is the size of this layer. We applied this idea to other network architectures and trained them with the backpropagation algorithm [4, 5] for French phoneme classification and auto-association.

Statistical feature selection for isolated word recognition

Abstract: A procedure for feature selection in isolated word recognition is discussed. The feature selection is performed in two steps. The first step takes into account the temporal correlation among feature vectors in order to obtain a transformation matrix which projects the initial template of N feature vectors to a new space where they are uncorrelated. This step gives a new template of M feature vectors, where M >

Neural networks for data compression and data fusion

Abstract: This chapter focuses on the use of neural networks for data compression and data fusion. Several neural network methods have been developed to deal with data compression, data dimensionality reduction, and multi-sensor data correlation/fusion. The approaches that have been most explored include Learning Vector Quantization network for data compression and special architecture of the back-propagation network for data compression; dimensionality reduction; and sensor data fusion. The self-organizing Topology-Preserving Map can be used for dimensionality reduction, and special architectures have been developed for multi-spectral, multi-sensor, and/or stereo image data correlation. The chapter describes the compression and dimensionality reduction methods, the desired characteristics of which are (1) robust discrimination of novel data from baseline; (2) potential for very rapid operation (both software and hardware optimization); (3) ability to specify loss involved in data compression and reconstruction; and (4) high compression ratios to facilitate data storage.

On en eduction and txme-bandwidth products in signal reconstruction *

Abstract: A dimensionality reduction method based on time-bandwidth products is proposed for tlie reconstruction of bandlimited, essentially time-limited signals from a small number of observations. A criterion is presented for choosing the dimension of the reduced signal space based on knowledge of the passband, time-concentration interval, energy concentration factor, and bounds on the tolerable reconstruction error. The reconstruction is constrained to lie in this lower dimensional signal space, and parameters characterizing the reconstruction are obtained from the d ata by solving a linear system of equations. For certain sampling patterns the system is ikonditioned, and a gecond rank reduction is needed to reduce the deleterious effects of observation noise and modeling error. A novel criterion for rank selection based on knowledge of bounds on noise power and modeling error is presented.