Showing papers on "Dimensionality reduction published in 1992"

Dimensions of meaning

Abstract: The representation of documents and queries as vectors in a high-dimensional space is well-established in information retrieval. The author proposes that the semantics of words and contexts in a text be represented as vectors. The dimensions of the space are words and the initial vectors are determined by the words occurring close to the entity to be represented, which implies that the space has several thousand dimensions (words). This makes the vector representations (which are dense) too cumbersome to use directly. Therefore, dimensionality reduction by means of a singular value decomposition is employed. The author analyzes the structure of the vector representations and applies them to word sense disambiguation and thesaurus induction. >

Pattern recognition and image preprocessing

Abstract: Pattern recognition: supervised and unsupervised learning in pattern recognition nonparametric decision theoretic classification nonparametric (distribution-free) training of discriminant functions statistical discriminant functions clusteringanalysis and unsupervised learning dimensionality reduction and feature selection. Neural networks for pattern recognition: multilayer perception radial basis function networks hamming net and Kohonen self-organizing feature map the Hopfield model.Data preprocessing for pictorial pattern recognition: preprocessing in the spatial domain pictorial data preposessing and shape analysis transforms and image processing in the transform doamin wavelets and wavelet transforms. Applications: exemplaryapplications. Practical concerns of image processing and pattern recognition: computer system architectures for image processing and pattern recognition. Appendices: digital images image model and discrete mathematics digital image fundamentals matrixmanipulation Eigenvectors and Eigenvalves of an operator notation.

Non-Linear Dimensionality Reduction

Abstract: A method for creating a non-linear encoder-decoder for multidimensional data with compact representations is presented. The commonly used technique of autoassociation is extended to allow non-linear representations, and an objective function which penalizes activations of individual hidden units is shown to result in minimum dimensional encodings with respect to allowable error in reconstruction.

Enhancements to probabilistic neural networks

Abstract: Probabilistic neural networks (PNNs) learn quickly from examples in one pass and asymptotically achieve the Bayes-optimal decision boundaries. The major disadvantage of a PNN stems from the fact that it requires one node or neuron for each training pattern. Various clustering techniques have been proposed to reduce this requirement to one node per cluster center. The correct choice of clustering technique will depend on the data distribution, data rate, and hardware implementation. Adaptation of kernel shape provides a tradeoff of increased accuracy for increased complexity and training time. The technique described also provides a basis for automatic feature selection and dimensionality reduction. >

Feature extraction using an unsupervised neural network

Abstract: A novel unsupervised neural network for dimensionality reduction that seeks directions emphasizing multimodality is presented, and its connection to exploratory projection pursuit methods is discussed. This leads to a new statistical insight into the synaptic modification equations governing learning in Bienenstock, Cooper, and Munro (BCM) neurons (1982). The importance of a dimensionality reduction principle based solely on distinguishing features is demonstrated using a phoneme recognition experiment. The extracted features are compared with features extracted using a backpropagation network.

Feature classification using supervised statistical pattern recognition

Abstract: Feature classification using a novel supervised statistical pattern recognition approach is described. A tree-like hierarchical decomposition of n-dimensional feature space is created off-line from an image processing system. The hierarchical tree is created through a minimax-type decompositional segregation of n-dimensional feature vectors of different feature classifications within the corresponding feature space. Each cell preferably contains feature vectors of only one feature classification, or is empty, or is of a predefined minimum cell size. Once created, the hierarchical tree is made available to the image processing system for real-time defect classification of features in a static or moving pattern. Each feature is indexed to the classification tree by locating its corresponding feature vector in the appropriate feature space cell as determined by a depth-first search of the hierarchical tree. The smallest leaf node which includes that feature vector provides the statistical information on the vector's classification.

Dimensions of meaning

Abstract: The representation of documents and queries as vectors in a high-dimensional space is well-established in information retrieval. The author proposes that the semantics of words and contexts in a text be represented as vectors. The dimensions of the space are words and the initial vectors are determined by the words occurring close to the entity to be represented, which implies that the space has several thousand dimensions (words). This makes the vector representations (which are dense) too cumbersome to use directly. Therefore, dimensionality reduction by means of a singular value decomposition is employed. The author analyzes the structure of the vector representations and applies them to word sense disambiguation and thesaurus induction.

Learning to Recognize Faces from Examples

Abstract: We describe an implemented system that learns to recognize human faces under varying pose and illumination conditions. The system relies on symmetry operations to detect the eyes and the mouth in a face image, uses the locations of these features to normalize the appearance of the face, performs simple but effective dimensionality reduction by a convolution with a set of Gaussian receptive fields, and subjects the vector of activities of the receptive fields to a Radial Basis Function interpolating classifier. The performance of the system compares favorably with the state of the art in machine recognition of faces.

Discriminative analysis for feature reduction in automatic speech recognition

Abstract: A dimensionality reduction method of the frame feature space based on discriminative analysis is discussed. A significant dimensionality reduction is obtained without loss of recognition performance in speaker independent experiments on a variety of speech databases. In addition, this procedure allows the selective incorporation of new feature components into an existing feature set. >

On the existence of complete invariant feature spaces in pattern recognition

Abstract: This paper presents a general and model-independent analysis of the problem of feature extraction in pattern recognition. Two criteria are derived which ensure the existence of a complete feature space. This is a space which contains exactly the information relevant for the classification process following feature extraction. Several possibilities for the construction of such a complete feature space are discussed and experimental results which indicate the potential of the proposed methods for practical applications are presented. >

Forecasting Demand for Electric Power

Abstract: We are developing a forecaster for daily extremes of demand for electric power encountered in the service area of a large midwestern utility and using this application as a testbed for approaches to input dimension reduction and decomposition of network training. Projection pursuit regression representations and the ability of algorithms like SIR to quickly find reasonable weighting vectors enable us to confront the vexing architecture selection problem by reducing high-dimensional gradient searchs to fitting single-input single-output (SISO) subnets. We introduce dimension reduction algorithms, to select features or relevant subsets of a set of many variables, based on minimizing an index of level-set dispersions (closely related to a projection index and to SIR), and combine them with backfitting to implement a neural network version of projection pursuit. The performance achieved by our approach, when trained on 1989, 1990 data and tested on 1991 data, is comparable to that achieved in our earlier study of backpropagation trained networks.

Statistical feature extraction and selection for IC test pattern analysis

Abstract: Complexity reduction and automatic test point selection are discussed in the context of statistical pattern classification. Different types of feedforward neural networks capable of IC fault diagnosis are examined. To reduce diagnostic complexity, principal component analysis (PCA) and full stepwise feature selection are employed to reduce network input dimension without sacrificing accuracy. For fault analysis purposes, it seems that feature selection by stepwise variable selection appears much more useful than feature extraction by PCA, since the latter requires that all original test measurements be made while the former helps eliminate redundant measurements. >

Using Recurrent Networks for Dimensionality Reduction

Abstract: This report explores how recurrent neural networks can be exploited for learning high-dimensional mappings. Since recurrent networks are as powerful as Turing machines, an interesting question is how recurrent networks can be used to simplify the problem of learning from examples. The main problem with learning high-dimensional functions is the curse of dimensionality which roughly states that the number of examples needed to learn a function increases exponentially with input dimension. This thesis proposes a way of avoiding this problem by using a recurrent network to decompose a high-dimensional function into many lower dimensional functions connected in a feedback loop.

Unsupervised feature reduction in image segmentation by local Karhunen-Loeve transform

Abstract: Proposes to reduce the dimensionality of feature vectors by using the principles of Karhunen-Loeve transform, (KL) applied to the feature images locally and globally. The reduction is achieved by choosing the resulting basis vectors which are closest to those of the classical KL transform. An efficient implementation technique using pyramids is proposed. Experimental results are presented. >

Accuracy of feature selection and extraction in statistical and neural net pattern classification

Abstract: Feature selection and feature extraction are common information processing stages in statistical pattern recognition and ANN classifier design. The number of samples used to evaluate the quality of feature subset and the use of simplified measures to speed up the evaluation procedures can cause a significant increase in a generalization error. Factors that determine the increase mentioned are analyzed and a method to determine this increase in practical work is proposed. >

Dimensionality reduction for nonlinear time series

Abstract: A technique for recoding multidimensional data in a representation of reduced dimensionality is presented. A non-linear encoder-decoder for multidimensional data with compact representations is developed. The technique of training a neural network to learn the identity map through a `bottleneck' is extended to networks with non-linear representations, and an objective function which penalizes entropy of the hidden unit activations is shown to result in low dimensional encodings. For scalar time series data, a common technique is phase-space reconstruction by embedding the time-lagged scalar signal in a higher dimensional space. Choosing the proper embedding dimension is difficult. By using non-linear dimensionality reduction, the intrinsic dimensionality of the underlying system may be estimated.

Topological feature map and automatic feature selection

Abstract: A method for automatic feature selection is described. The method is based on a suitable transform of an image and an estimated histogram of magnitudes of the transformed image. The estimation is done by a self-organizing process. The self-organizing process creates a one- dimensional topological feature map that is used as a feature vector. The method is demonstrated on four textured images.

Discriminant feature extraction for parametric and non-parametric classifier

Abstract: Feature extraction (FE) is considered as preserving the value of the discriminant function for a given classifier which uses a posteriori probabilities P( omega /sub i/ mod X) while reducing dimensionality For classification minimizing Bayes' error, a posteriori probabilities would be the best features In this feature space, the probability of error is the same as in the original space, assuming Bayes' classifier The authors consider FE as eliminating features which have no impact on the value of the discriminant function and propose an FE algorithm which eliminates those irrelevant features and retains only useful features The proposed algorithm does not deteriorate even when there is no difference in the mean vectors or covariance matrices, and it can be used for both parametric and nonparametric classifiers >

Directed spreading activation in multiple layers for low-level feature extraction

Abstract: Spreading activation neural networks have been proposed in literature. The paper proposes a directed spreading activation neural network model which performs a large number of early vision tasks. It is shown how directed two-dimensional (2D) diffusion followed by detection of local maxima can effectively perform feature extraction, feature centroid determination and feature clustering all on multiple scales in a purely data-driven manner. The feature map, which is the result of this directed spreading activation process can be used in learning and recognition of 2D object shapes from their binary patterns invariant to affine transformations. >

Robust object extraction using normalized principal component features

Abstract: A new general purpose method for object extraction and detection, RONPaC (Robust Object Extraction (Detection) using NPaC Features) method is presented. RONPaC employs normalized principal component (NPaC) features as a measure of similarity between corresponding regions of a target image and a background image. No a priori knowledge of objects and no assumptions about the environment are required. The object extraction problem is dealt with as a discriminant problem of two classes, 'object' and 'background', in the feature space. The performance of the method is quantitatively evaluated using various real images and compared with conventional methods using two criteria, separability between two classes in feature space and ease of binarizing expressed by the maximum discriminant criterion. Experimental results confirm the extraction accuracy and applicability of the proposed method. >

A Decision Boundary Method for Embedding Selection

Abstract: Abetract Previously, we have presented a method for embedding selection based on cluster analysis. In this paper, we described an embedding selection method based on a feature reduction transformation matrix. This method extracts features that are important for maintaining decision boundaries in the supervised clusters. Experimentally, we demonstrate that our method allow accurate prediction of the Mackay-Glass chaotic time series. Three important properties of the feature reduction transformation are proved in this paper.

Learning algorithm based on modified structure of pattern classes

Abstract: A procedure for classification and learning based on l-classifier is presented. Real data of five pattern classes have been used to demonstrate the applicability and efficiency of proposed procedure. Moment-invariant method is used for feature extraction, and dimensionality is reduced in order to maximize separability in reduced feature space. Experiment has shown that learning algorithm based on l-classifier has improved classification. Substitution procedure doesn't change the number of initial patterns, so that optimality is achieved without expanding storage space and increasing computational complexity. >

A proposal for an artificial neural network that optimizes reference vectors: FMNET

Abstract: A new artificial layered neural network model called FMNET (feature mapping and matching network) is proposed. FMNET has a feature mapping layer (F-layer) and a subsequent matching layer (M-layer). The F-layer maps the training set to the univariate Gaussian form and the M-layer creates or integrates the output neurons under the likelihood criterion to attain the unimodal Gaussian form. The well optimized FMNET extracts the feature vectors as the expectation value of the output vectors of the F-layer, and the backpropagation learning method becomes consistent with the maximum likelihood estimation method in an asymptotic condition. Furthermore, a good generalizing property is attained by an experiment using mixed Gaussian test patterns. >

Near-optimal algorithm for dimension reduction

Abstract: Dimension reduction is a process of transforming the multidimensional observations into low-dimensional space. In pattern recognition this process should not cause loss of classification accuracy. This goal is best accomplished using Bayes error as a criterion for dimension reduction. Since the criterion is not usable for practical purposes, the authors suggest the use of the k-nearest neighbor estimate of the Bayes error instead. They experimentally demonstrate the superior performance of the linear dimension reduction algorithm based on this criterion, as compared to the traditional techniques. >

Specific Distance for Feature Selection in Speech Recognition

Abstract: In this paper, the use of a specific metric as a feature selection step is investigated. The feature selection step tries to model the correlation among adjacent feature vectors and the variability of the speech. We propose a new procedure which performs the feature selection in two steps. The first step takes into account the temporal correlation among the N feature vectors of a template in order to obtain a new set of feature vectors which are uncorrelated. This step gives a new template of M feature vectors, with M ≪ N. The second step defines a specific distance among feature vectors to take into account the frequency discrimination features which discriminate each word of the vocabulary from the others or a set of them. Thus, the new feature vectors are uncorrelated in time and discriminant in frequency.