Showing papers on "Principal component analysis published in 1995"

Methods of multivariate analysis

Abstract: Introduction. Matrix Algebra. Characterizing and Displaying Multivariate Data. The Multivariate Normal Distribution. Tests on One or Two Mean Vectors. Multivariate Analysis of Variance. Tests on Covariance Matrices. Discriminant Analysis: Description of Group Separation. Classification Analysis: Allocation of Observations to Groups. Multivariate Regression. Canonical Correlation. Principal Component Analysis. Factor Analysis. Cluster Analysis. Graphical Procedures. Tables. Answers and Hints to Problems. Data Sets and SAS Files. References. Index.

Parallel Analysis: a method for determining significant principal components

Abstract: Numerous ecological studies use Principal Compo- nents Analysis (PCA) for exploratory analysis and data reduc- tion. Determination of the number of components to retain is the most crucial problem confronting the researcher when using PCA. An incorrect choice may lead to the underextraction of components, but commonly results in overextraction. Of several methods proposed to determine the significance of principal components, Parallel Analysis (PA) has proven con- sistently accurate in determining the threshold for significant components, variable loadings, and analytical statistics when decomposing a correlation matrix. In this procedure, eigen- values from a data set prior to rotation are compared with those from a matrix of random values of the same dimensionality (p variables and n samples). PCA eigenvalues from the data greater than PA eigenvalues from the corresponding random data can be retained. All components with eigenvalues below this threshold value should be considered spurious. We illus- trate Parallel Analysis on an environmental data set. We reviewed all articles utilizing PCA or Factor Analysis

A review of canonical coordinates and an alternative to correspondence analysis using Hellinger distance

Abstract: In this paper a general theory of canonical coordinates is developed for reduction of dimensionality in multivariate data, assessing the loss of information and plotting higher dimensional data in two or three dimensions for visual displays. The theory is applied to data in two way tables with variables in one category and samples (individual or populations) in the other. Two types of data are considered, one with continuous measurements on the variables and another with frequencies of attributes. An alternative to the usual correspondence analysis of contingency tables based on the Hellinger rather than the chi-square distance is suggested. The new method has some attractive features and does not suffer from some inherent drawbacks resulting from the use of the chi-square distance and variable sample sizes for the populations in the correspondence analysis. The technique of biplots where the populations and the variables are represented on the same chart is discussed

Robust principal component analysis by self-organizing rules based on statistical physics approach

Abstract: This paper applies statistical physics to the problem of robust principal component analysis (PCA). The commonly used PCA learning rules are first related to energy functions. These functions are generalized by adding a binary decision field with a given prior distribution so that outliers in the data are dealt with explicitly in order to make PCA robust. Each of the generalized energy functions is then used to define a Gibbs distribution from which a marginal distribution is obtained by summing over the binary decision field. The marginal distribution defines an effective energy function, from which self-organizing rules have been developed for robust PCA. Under the presence of outliers, both the standard PCA methods and the existing self-organizing PCA rules studied in the literature of neural networks perform quite poorly. By contrast, the robust rules proposed here resist outliers well and perform excellently for fulfilling various PCA-like tasks such as obtaining the first principal component vector, the first k principal component vectors, and directly finding the subspace spanned by the first k vector principal component vectors without solving for each vector individually. Comparative experiments have been made, and the results show that the authors' robust rules improve the performances of the existing PCA algorithms significantly when outliers are present. >

Discriminant Analysis with Singular Covariance Matrices: Methods and Applications to Spectroscopic Data

Abstract: SUMMARY Currently popular techniques such as experimental spectroscopy and computer-aided molecular modelling lead to data having very many variables observed on each of relatively few individuals. A common objective is discrimination between two or more groups, but the direct application of standard discriminant methodology fails because of singularity of covariance matrices. The problem has been circumvented in the past by prior selection of a few transformed variables, using either principal component analysis or partial least squares. Although such selection ensures nonsingularity of matrices, the decision process is arbitrary and valuable information on group structure may be lost. We therefore consider some ways of estimating linear discriminant functions without such prior selection. Several spectroscopic data sets are analysed with each method, and questions of bias of assessment procedures are investigated. All proposed methods seem worthy of consideration in practice.

Comparing wavelet transforms for recognizing cardiac patterns

Abstract: The authors' study made use of wavelet transforms to describe and recognize isolated cardiac beats. The choice of the wavelet family as well as the selection of the analyzing function into these families are discussed. The criterion used in the first case was the correct classification rate, and in the second case, the correlation coefficient between the original pattern and the reconstructed one. Two types of description have been considered-the energy-based representation and the extrema distribution estimated at each decomposition level-and their quality has been assessed by using principal component analysis. Their capability of discrimination between normal, premature ventricular contraction, and ischemic beats has been studied by means of linear discriminant analysis. This work leads also, for the problem at hand, to the identification of the most relevant resolution levels. >

Principal component analysis with missing data and its application to polyhedral object modeling

Abstract: Observation-based object modeling often requires integration of shape descriptions from different views. To overcome the problems of errors and their accumulation, we have developed a weighted least-squares (WLS) approach which simultaneously recovers object shape and transformation among different views without recovering interframe motion. We show that object modeling from a range image sequence is a problem of principal component analysis with missing data (PCAMD), which can be generalized as a WLS minimization problem. An efficient algorithm is devised. After we have segmented planar surface regions in each view and tracked them over the image sequence, we construct a normal measurement matrix of surface normals, and a distance measurement matrix of normal distances to the origin for all visible regions over the whole sequence of views, respectively. These two matrices, which have many missing elements due to noise, occlusion, and mismatching, enable us to formulate multiple view merging as a combination of two WLS problems. A two-step algorithm is presented. After surface equations are extracted, spatial connectivity among the surfaces is established to enable the polyhedral object model to be constructed. Experiments using synthetic data and real range images show that our approach is robust against noise and mismatching and generates accurate polyhedral object models. >

Rotation of principal components: choice of normalization constraints

Abstract: Following a principal component analysis, it is fairly common practice to rotate some of the components, often using orthogonal rotation. It is a frequent misconception that orthogonal rotation will produce rotated components which are pairwise uncorrelated, and/or whose loadings are orthogonal In fact, it is not possible, using the standard definition of rotation, to preserve both these properties. Which of the two properties is preserved depends on the normalization chosen for the loadings, prior to rotation. The usual ‘default’ normalization leads to rotated components which possess neither property.

Principal component filter banks for optimal multiresolution analysis

Abstract: An important issue in multiresolution analysis is that of optimal basis selection. An optimal P-band perfect reconstruction filter bank (PRFB) is derived in this paper, which minimizes the approximation error (in the mean-square sense) between the original signal and its low-resolution version. The resulting PRFB decomposes the input signal into uncorrelated, low-resolution principal components with decreasing variance. Optimality issues are further analyzed in the special case of stationary and cyclostationary processes. By exploiting the connection between discrete-time filter banks and continuous wavelets, an optimal multiresolution decomposition of L/sub 2/(R) is obtained. Analogous results are also derived for deterministic signals. Some illustrative examples and simulations are presented. >

Reducing data dimensionality through optimizing neural network inputs

Abstract: A neural network method for reducing data dimensionality based on the concept of input training, in which each input pattern is not fixed but adjusted along with internal network parameters to reproduce its corresponding output pattern, is presented. With input adjustment, a property configured network can be trained to reproduce a given data set with minimum distortion; the trained network inputs provide reduced data. A three-layer network with input training can perform all functions of a flue-layer autoassociative network, essentially capturing nonlinear correlations among data. In addition, simultaneous training of a network and its inputs is shown to be significantly more efficient in reducing data dimensionality than training an autoassociative network The concept of input training is closely related to principal component analysis (PCA) and the principal curve method, which is a nonlinear extension of PCA.

Multivariate analysis of spatial patterns: a unified approach to local and global structures

Abstract: We propose a new approach to the multivariate analysis of data sets with known sampling site spatial positions. A between-sites neighbouring relationship must be derived from site positions and this relationship is introduced into the multivariate analyses through neighbouring weights (number of neighbours at each site) and through the matrix of the neighbouring graph. Eigenvector analysis methods (e.g. principal component analysis, correspondence analysis) can then be used to detect total, local and global structures. The introduction of the D-centring (centring with respect to the neighbouring weights) allows us to write a total variance decomposition into local and global components, and to propose a unified view of several methods. After a brief review of the matrix approach to this problem, we present the results obtained on both simulated and real data sets, showing how spatial structure can be detected and analysed. Freely available computer programs to perform computations and graphical displays are proposed.

Principal component extraction using recursive least squares learning

Abstract: A new neural network-based approach is introduced for recursive computation of the principal components of a stationary vector stochastic process. The neurons of a single-layer network are sequentially trained using a recursive least squares squares (RLS) type algorithm to extract the principal components of the input process. The optimality criterion is based on retaining the maximum information contained in the input sequence so as to be able to reconstruct the network inputs from the corresponding outputs with minimum mean squared error. The proof of the convergence of the weight vectors to the principal eigenvectors is also established. A simulation example is given to show the accuracy and speed advantages of this algorithm in comparison with the existing methods. Finally, the application of this learning algorithm to image data reduction and filtering of images degraded by additive and/or multiplicative noise is considered. >

Quantitative Evaluation of Soybean (Glycine max L. Merr.) Leaflet Shape by Principal Component Scores Based on Elliptic Fourier Descriptor

Abstract: Leaflet shape of thirty-nine soybean cultivars/strains selected to cover the possible diversity of leaf shape, was quantitatively evaluated by principal components scores based on the elliptic Fourier descriptor of contours. After central leaflets of fully expanded compound-leaves of the cultivars/strains were videotaped, binary images of the leaflets were obtained from those video images by image processing. Then, the closed contour of each leaflet was extracted from the binary images and chain-coded by image processing. Because the first twenty harmonics could sufficiently represent soybean leaf contours, 77 elliptic Fourier coefficients were calculated for each chain-coded contour. Then, the Fourier coefficients were standardized so that the coefficients were invariant of the size, rotation, shift and chain-code starting-point of any contour. The principal component analysis about the standardized Fourier coefficients, showed that the cumulative contribution at the fifth principal component was about 96 o/o' Moreover, the effect of each principal component on the leaf shape was clarified by drawing the contours of leaflets using the Fourier coefficients inversely estimated under some typical values of the principal component scores. Consequently, it was indicated that the principal components scores about the standardized elliptic Fourier coefficients gave us powerful quantitative measures to evaluate soybean leaf shape. The analysis of variance and multiple comparison indicated that the genotypic differences on the first, the second and the fifth principal components were significantly large. Because the variations of those principal components were con-tinuous, the effects of the polygenes on the (size-invariant) shape were also suggested.

The Kohonen self-organizing map method: An assessment

Abstract: The “self-organizing map” method, due to Kohonen, is a well-known neural network method. It is closely related to cluster analysis (partitioning) and other methods of data analysis. In this article, we explore some of these close relationships. A number of properties of the technique are discussed. Comparisons with various methods of data analysis (principal components analysis, k-means clustering, and others) are presented.

Principal component analysis and the scaled subprofile model compared to intersubject averaging and statistical parametric mapping. I: Functional connectivity of the human motor system studied with [15O]water PET

Abstract: Using [15O]water PET and a previously well studied motor activation task, repetitive finger-to-thumb opposition, we compared the spatial activation patterns produced by (1) global normalization and intersubject averaging of paired-image subtractions, (2) the mean differences of ANCOVA-adjusted voxels in Statistical Parametric Mapping, (3) ANCOVA-adjusted voxels followed by principal component analysis (PCA), (4) ANCOVA-adjustment of mean image volumes (mean over subjects at each time point) followed by F-masking and PCA, and (5) PCA with Scaled Subprofile Model pre- and postprocessing. All data analysis techniques identified large positive focal activations in the contralateral sensorimotor cortex and ipsilateral cerebellar cortex, with varying levels of activation in other parts of the motor system, e.g., supplementary motor area, thalamus, putamen; techniques 1-4 also produced extensive negative areas. The activation signal of interest constitutes a very small fraction of the total nonrandom signal in the original dataset, and the exact choice of data preprocessing steps together with a particular analysis procedure have a significant impact on the identification and relative levels of activated regions. The challenge for the future is to identify those preprocessing algorithms and data analysis models that reproducibly optimize the identification and quantification of higher-order sensorimotor and cognitive responses.

A combined theory for PCA and PLS

Abstract: We present here an algorithmic approach to modelling data that includes principal component analysis (PCA) and partial least squares (PLS). In fact, the numerical algorithm presented can carry out PCA or PLS. The algorithm for linear analysis and extensions to non-linear analysis applies to both PCA and PLS. The algorithm allows for combination of PCA and PLS types of models and therefore extends modelling to new types of models that involve combination of regression models and ‘selection of variation’ models, which is the idea of PCA-type models. The fact that the algorithm carries out both PCA and PLS shows that PCA and PLS are based on the same theory. This theory is based on the H-principle of mathematical modelling. The algorithm allows tests for outliers, sensitivity analysis and tests of submodels. These aspects of the algorithm are treated in detail. We compute various measures of sizes, e.g. of components, of the covariance matrix, of its inverse, etc. that show how much the algorithm has selected at each step. The analysis is illustrated by data from practice.

A correlation principal component regression analysis of NIR data

Abstract: The use of principal component regression (PCR) as a multivariate calibration method has been discussed by a number of authors. In most situations principal components are included in the regression model in sequence based on the variances of the components, and the principal components with small variances are rarely used in regression. As pointed out by some authors, a low variance for a component does not necessarily imply that the corresponding component is unimportant, especially when prediction is of primary interest. In this paper we investigate a different version of PCR, correlation principal component regression (CPCR). In CPCR the importance of principal components in terms of predicting the response variable is used as a basis for the inclusion of principal components in the regression model. Two typical examples arising from calibrating near-infrared (NIR) instruments are discussed for the comparison of the two different versions of PCR along with partial least squares (PLS), a commonly used regression approach in NIR analysis. In both examples the three methods show similar optimal prediction ability, but CPCR performs better than standard PCR and PLS in terms of the number of components needed to achieve the optimal prediction ability. Similar results are also seen in other NIR examples.

Application of the Parametric Bootstrap to Models that Incorporate a Singular Value Decomposition

Abstract: SUMMARY Simulation is a standard technique for investigating the sampling distribution of parameter estimators. The bootstrap is a distribution-free method of assessing sampling variability based on resampling from the empirical distribution; the parametric bootstrap resamples from a fitted parametric model. However, if the parameters of the model are constrained, and the application of these constraints is a function of the realized sample, then the resampling distribution obtained from the parametric bootstrap may become badly biased and overdispersed. Here we discuss such problems in the context of estimating parameters from a bilinear model that incorporates the singular value decomposition (SVD) and in which the parameters are identified by the standard orthogonality relationships of the SVD. Possible effects of the SVD parameter identification are arbitrary changes in the sign of singular vectors, inversion of the order of singular values and rotation of the plotted co-ordinates. This paper proposes inverse transformation or 'filtering' techniques to avoid these problems. The ideas are illustrated by assessing the variability of the location of points in a principal co-ordinates diagram and in the marginal sampling distribution of singular values. An application to the analysis of a biological data set is described. In the discussion it is pointed out that several exploratory multivariate methods may benefit by using resampling with filtering.