Showing papers on "Dimensionality reduction published in 1995"

Local discriminant bases and their applications

Abstract: We describe an extension to the “best-basis” method to select an orthonormal basis suitable for signal/image classification problems from a large collection of orthonormal bases consisting of wavelet packets or local trigonometric bases The original best-basis algorithm selects a basis minimizing entropy from such a “library of orthonormal bases” whereas the proposed algorithm selects a basis maximizing a certain discriminant measure (eg, relative entropy) among classes Once such a basis is selected, a small number of most significant coordinates (features) are fed into a traditional classifier such as Linear Discriminant Analysis (LDA) or Classification and Regression Tree (CARTTM) The performance of these statistical methods is enhanced since the proposed methods reduce the dimensionality of the problem at hand without losing important information for that problem Here, the basis functions which are well-localized in the time-frequency plane are used as feature extractors We applied our method to two signal classification problems and an image texture classification problem These experiments show the superiority of our method over the direct application of these classifiers on the input signals As a further application, we also describe a method to extract signal component from data consisting of signal and textured background

Finding compact and sparse-distributed representations of visual images

Abstract: Some recent work has investigated the dichotomy between compact coding using dimensionality reduction and sparse-distributed coding in the context of understanding biological information processing. We introduce an artificial neural network which self-organizes on the basis of simple Hebbian learning and negative feedback of activation and show that it is capable both of forming compact codings of data distributions and of identifying filters most sensitive to sparse-distributed codes. The network is extremely simple and its biological relevance is investigated via its response to a set of images which are typical of everyday life. However, an analysis of the network's identification of the filter for sparse coding reveals that this coding may not be globally optimal and that there exists an innate limiting factor which cannot be transcended.

Self-organization of pattern data with dimension reduction through learning of non-linear variance-constrained mapping

Abstract: The subject system provides a self-organized reduced-dimension remapping of pattern data. The system functions to a mapping from an original pattern space to a reduced-dimension space in an unsupervised nonlinear manner, but with a constraint that the overall variance in a representation of the data be conserved. This approach relates to but is different from both the Karhuren-Loeve (K-L) transform and auto-associative approaches which emphasize feature extraction, and also from the Advanced Reasoning Tool (ART) and feature mapping approaches which emphasize category formation based on similarity in the original representation. The subject system is highly efficient computationally. The reduced-dimension representation is suitably further simplified with ART or feature mapping techniques, as appropriate and as desired.

Linear dimension reduction of sequences of medical images: II. Direct sum decomposition

Abstract: Using unitary transformations together with a previously described statistical theory for optimal linear dimension reduction it is shown how pixels in a sequence of images can be decomposed into a sum of variates, covariates, and residual vectors, with all covariances equal to zero. It is demonstrated that this decomposition is optimal with respect to noise. In addition, it results in simplified and well conditioned equations for dimension reduction and elimination of covariates. The factor images are not degraded by subdivision of the time intervals. In contrast to traditional factor analysis, the factors can be measured directly or calculated based on physiological models. This procedure not only solves the rotation problem associated with factor analysis, but also eliminates the need for calculation of the principal components altogether. Examples are given of factor images of the heart, generated from a dynamic study using oxygen-15-labelled water and positron emission tomography. As a special application of the method, it is shown that the factor images may reveal any contamination of the blood curve derived from the original dynamic images with myocardial activity.

Feature modification in incremental feature generation

Abstract: Feature interaction is a common problem in feature generation methods such as incremental feature generation, automatic feature extraction, feature-based design and manual feature definition. Research on feature interaction involves analysing the interaction relationship, decomposing the interacted features into atomic or single features, and defining their relationship, human interpretation being required. The paper presents an approach to handling the feature interaction problem in incremental feature generation. As a protrusion or depression feature is defined, the interaction between the features is analysed. The relationship is then defined and the interacting existing feature is redefined according to the interaction cases. The approach involves feature existence analysis and modification procedures. The existence analysis classifies the existing features into three groups; features remaining without interaction, features to be removed, and features remaining after being partially removed. It further classifies the last group into only the face set being removed, only the boundary edge path being removed, and the face set including the boundary edge path being removed. The three interaction cases in the last group require modification procedures. The modification procedures are (a) decide whether the remaining part of an existing feature is valid for a feature definition, (b) update it as a new feature, and (c) define the feature's relationship. The above procedures are performed through tracing the Boolean operations and convexity of an intersection edge loop generated during design evolution.

Dimensionality reduction of multi-scale feature spaces using a separability criterion

Abstract: An algorithm for classification task dependent multiscale feature extraction is suggested. The algorithm focuses on dimensionality reduction of the feature space subject to maximum preservation of classification information. It has been shown that, for classification tasks, class separability based features are appropriate alternatives to features selected based on energy and entropy criteria. Application of this idea to feature extraction from multi-scale wavelet packets is presented. At each level of decomposition an optimal linear transform that preserves class separabilities and results in a reduced dimensional feature space is obtained. Classification and feature extraction is performed at each scale and resulting "soft decisions" are integrated across scales. The suggested scheme can also be applied to other orthogonal or non-orthogonal multiscale transforms e.g. local cosine transform or Gabor transform. The suggested algorithm has been tested on classification and segmentation of some radar target signatures as well as textured and document images.

Fast feature-based vector quantization algorithm of image coding

Abstract: A fast vector quantization algorithm is presented that exploits the spatial redundancy between neighboring vectors of pixels in an image to improve the performance of the triangle inequality elimination ruler, and employs the integral projection technique, a dimension-reduction method, to reduce the computational complexity of calculating distortion measure. Application of dimension reduction in distortion measures may result in some degradation of objective image quality. But a significant complexity reduction of over 90% in comparison with the conventional full-search method can be achieved. The degradation of image quality is only less than 0.2 dB in peak signal-to-noise ratio (PSNR). Acceptable image quality should be obtained successfully.

Self-organizing operator map for nonlinear dimension reduction

Abstract: Dimension reduction is an important problem arising e.g. in feature extraction, pattern recognition, and data compression. Often this is done using principal component analysis (PCA), but this approach is suitable only when the data are sufficiently linearly distributed. In this paper, neural network learning algorithms combining Kohonen's self-organizing map (SOM) and Oja's PCA rule are studied for the challenging task of nonlinear dimension reduction. The neural network has a structure of a self-organizing operator map where neurons, i.e. operators, are affine spaces instead of vectors. Adaptive algorithms derived from an optimization criterion are shortly reviewed, but the emphasis is on computationally more efficient and stable learning-rate free, K-means type batch algorithms. Simulations using image data show that the methods outperform the sequential methods proposed earlier.

Feature extraction of machinery diagnosis using neural network

Abstract: Vibration monitoring of machinery involves the collection of vibration data from machine components and detailed analysis to extract features that reflect the running state of the machinery. The machinery state can be described accurately if the feature is correctly selected. A new approach is developed in this paper, in which the optimization feature subset is extracted, making full use of the information processing ability of neural networks, and using sensitivity of the feature parameter as the criterion of selection. In this approach, feature parameter sensitivity and feature parameter consistency are appraised simultaneously when accomplishing the training of neural networks. In addition, combining with a logical rule, an optimization feature subset is obtained. This method solves the problem of the conventional inefficient way where the optimization feature subset is extracted from many feature parameter types of vibration signals. The new feature subset of the reduced dimensions provides accurate data for the precision analysis. As a result, the accuracy of the automonitoring system can be improved.

Experimental improvements of a language Id system

Abstract: Previously, automatic language identification systems provided good results by using syllabic "on-set" spectral features; they identified languages by finding the "nearest match" speakers who were closet to the test utterance. The present authors we show that augmenting the training data by adding speakers achieves a better gender balance in the data and reduces the error rate by more than 10%. Adding features like syllabic "coda" and "prosodic" features show very different results which can then be merged with the syllabic "on-set" spectral features to reduce errors an additional 10%. A dimensionality reduction by means of the principal components shows not only a reduction in computation and memory requirements, but also improves language identification performance when the eigenvectors are normalized with different weights. The combination of all these factors yields a significant improvement in performance when compared with the previous baseline system.

Multiple Self-Organizing Maps for Supervised Learning

Abstract: A scheme for supervised learning based on multiple self-organizing maps is presented and its performance is compared with other methods in several pattern classification benchmarks using both synthetic and real data The advantage of this approach is that the learning method is simplified because the problem is divided into several SOMs, which are trained in the standard unsupervised way The resulting network preserves the SOM properties like dimensionality reduction and cluster formation, while classifying with an accuaracy comparable to other supervised methods on a wide range of problems

An Example of Unsupervised Networks Kohonen's Self-Organizing Feature Map

Abstract: Kohonen's self-organizing feature map belongs to a class of unsupervised artificial neural network commonly referred to as topographic maps. It serves two purposes, the quantization and dimensionality reduction of date. A short description of its history and its biological context is given. We show that the inherent classification properties of the feature map make it a suitable candidate for solving the classification task in power system areas like load forecasting, fault diagnosis and security assessment.

Self-Organizing Maps for Supervision in Robot Pick-And-Place Operations

Abstract: A scheme for supervised learning based on multiple self-organizing maps (SOMs) is presented, and its application to robotic tasks, namely pick-and-place operations, is outlined. The advantage of this multiple organization is that the learning method is simplified because the problem is divided into several SOMs, which are trained in the standard unsupervised way. The resulting network preserves the SOM properties like dimensionality reduction and cluster formation, and its classification performance is comparable to other supervised methods like backpropagation networks.

Successive Feature Extraction from High-Dimensional Data Based on Significance Weighted Criterion

Abstract: High-dimensional data require a lot of computational work for processing. In order to efficiently and accurately obtain the results of measurement from high-dimensional data, significant features should be extracted before processing. Here we present a feature extraction method for significance weighted supervised classification. Conventional methods of extracting features consider only the average classification accuracy. In contrast with this, we present a new method of feature extraction which is based on a significance weighted criterion. The purpose is to extract features which separate particular pair of classes and give high classification accuracy for important classes. In the first step, assuming that all the classes have the same within-class covariance matrices and normally distributed, all the data are reduced by principal component analysis. Most of the information can be expressed in low dimensional space because many of the dimensions of high-dimensional data are highly correlated. In the next step, feature vectors are determined in the space of reduced data. Each feature is extracted successively by selecting the feature vectors which separate the hardest-to-separate pair of classes among the important pairs of classes to be separated. A feature vector which separates two classes is set according to the Fisher's linear discriminant function. The method is applied to about 500 dimensional hyperspectral data which are required to be classified into five categories. The feature extraction technique, together with the results of numerical simulations which confirm the validity of this approach are presented.