scispace - formally typeset
Search or ask a question

Showing papers on "Principal component analysis published in 1997"


Book ChapterDOI
08 Oct 1997
TL;DR: A new method for performing a nonlinear form of Principal Component Analysis by the use of integral operator kernel functions is proposed and experimental results on polynomial feature extraction for pattern recognition are presented.
Abstract: A new method for performing a nonlinear form of Principal Component Analysis is proposed. By the use of integral operator kernel functions, one can efficiently compute principal components in highdimensional feature spaces, related to input space by some nonlinear map; for instance the space of all possible d-pixel products in images. We give the derivation of the method and present experimental results on polynomial feature extraction for pattern recognition.

2,223 citations


Journal ArticleDOI
TL;DR: The user interface is simple and homogeneous among all the programs; this contributes to making the use of ADE-4 very easy for non- specialists in statistics, data analysis or computer science.
Abstract: We present ADE-4, a multivariate analysis and graphical display software. Multivariate analysis methods available in ADE-4 include usual one-table methods like principal component analysis and correspondence analysis, spatial data analysis methods (using a total variance decomposition into local and global components, analogous to Moran and Geary indices), discriminant analysis and within/between groups analyses, many linear regression methods including lowess and polynomial regression, multiple and PLS (partial least squares) regression and orthogonal regression (principal component regression), projection methods like principal component analysis on instrumental variables, canonical correspondence analysis and many other variants, coinertia analysis and the RLQ method, and several three-way table (k-table) analysis methods. Graphical display techniques include an automatic collection of elementary graphics corresponding to groups of rows or to columns in the data table, thus providing a very efficient way for automatic k-table graphics and geographical mapping options. A dynamic graphic module allows interactive operations like searching, zooming, selection of points, and display of data values on factor maps. The user interface is simple and homogeneous among all the programs; this contributes to making the use of ADE-4 very easy for non- specialists in statistics, data analysis or computer science.

1,651 citations


Journal ArticleDOI
TL;DR: This paper reports the application of a method based on information theory that decomposes one or more ERPs recorded at multiple scalp sensors into a sum of components with fixed scalp distributions and sparsely activated, maximally independent time courses.
Abstract: Averaged event-related potential (ERP) data recorded from the human scalp reveal electroencephalographic (EEG) activity that is reliably time-locked and phase-locked to experimental events. We report here the application of a method based on information theory that decomposes one or more ERPs recorded at multiple scalp sensors into a sum of components with fixed scalp distributions and sparsely activated, maximally independent time courses. Independent component analysis (ICA) decomposes ERP data into a number of components equal to the number of sensors. The derived components have distinct but not necessarily orthogonal scalp projections. Unlike dipole-fitting methods, the algorithm does not model the locations of their generators in the head. Unlike methods that remove second-order correlations, such as principal component analysis (PCA), ICA also minimizes higher-order dependencies. Applied to detected—and undetected—target ERPs from an auditory vigilance experiment, the algorithm derived ten components that decomposed each of the major response peaks into one or more ICA components with relatively simple scalp distributions. Three of these components were active only when the subject detected the targets, three other components only when the target went undetected, and one in both cases. Three additional components accounted for the steady-state brain response to a 39-Hz background click train. Major features of the decomposition proved robust across sessions and changes in sensor number and placement. This method of ERP analysis can be used to compare responses from multiple stimuli, task conditions, and subject states.

1,172 citations


Proceedings Article
01 Dec 1997
TL;DR: An expectation-maximization (EM) algorithm for principal component analysis (PCA) which allows a few eigenvectors and eigenvalues to be extracted from large collections of high dimensional data and defines a proper density model in the data space.
Abstract: I present an expectation-maximization (EM) algorithm for principal component analysis (PCA). The algorithm allows a few eigenvectors and eigenvalues to be extracted from large collections of high dimensional data. It is computationally very efficient in space and time. It also naturally accommodates missing information. I also introduce a new variant of PCA called sensible principal component analysis (SPCA) which defines a proper density model in the data space. Learning for SPCA is also done with an EM algorithm. I report results on synthetic and real data showing that these EM algorithms correctly and efficiently find the leading eigenvectors of the covariance of datasets in a few iterations using up to hundreds of thousands of datapoints in thousands of dimensions.

928 citations


Journal ArticleDOI
TL;DR: A local linear approach to dimension reduction that provides accurate representations and is fast to compute is developed and it is shown that the local linear techniques outperform neural network implementations.
Abstract: Reducing or eliminating statistical redundancy between the components of high-dimensional vector data enables a lower-dimensional representation without significant loss of information. Recognizing the limitations of principal component analysis (PCA), researchers in the statistics and neural network communities have developed nonlinear extensions of PCA. This article develops a local linear approach to dimension reduction that provides accurate representations and is fast to compute. We exercise the algorithms on speech and image data, and compare performance with PCA and with neural network implementations of nonlinear PCA. We find that both nonlinear techniques can provide more accurate representations than PCA and show that the local linear techniques outperform neural network implementations.

702 citations


Journal ArticleDOI
TL;DR: This paper proposes neural structures related to multilayer feedforward networks for performing complete independent component analysis (ICA) and modify the previous nonlinear PCA type algorithms so that their separation capabilities are greatly improved.
Abstract: Independent component analysis (ICA) is a recently developed, useful extension of standard principal component analysis (PCA). The ICA model is utilized mainly in blind separation of unknown source signals from their linear mixtures. In this application only the source signals which correspond to the coefficients of the ICA expansion are of interest. In this paper, we propose neural structures related to multilayer feedforward networks for performing complete ICA. The basic ICA network consists of whitening, separation, and basis vector estimation layers. It can be used for both blind source separation and estimation of the basis vectors of ICA. We consider learning algorithms for each layer, and modify our previous nonlinear PCA type algorithms so that their separation capabilities are greatly improved. The proposed class of networks yields good results in test examples with both artificial and real-world data.

421 citations


Journal ArticleDOI
TL;DR: A variation of this technique in which the factors that reconstruct the modified EEG from the original are stored as a matrix is developed, which acts as a spatial filter with useful properties and successfully applied this method to remove artifacts, including ocular movement and electrocardiographic artifacts.
Abstract: Principal component analysis (PCA) by singular value decomposition (SVD) may be used to analyze an epoch of a multichannel electroencephalogram (EEG) into multiple linearly independent (temporally and spatially noncorrelated) components, or features; the original epoch of the EEG may be reconstructed as a linear combination of the components. The result of SVD includes the components, expressible as time series waveforms, and the factors that determine how much each component waveform contributes to each EEG channel. By omission of some component waveforms from the linear combination, a new EEG can be reconstructed, differing from the original in useful ways. For example, artifacts can be removed and features such as ictal or interictal discharges can be enhanced by suppressing the remainder of the EEG. We developed a variation of this technique in which the factors that reconstruct the modified EEG from the original are stored as a matrix. This matrix is applied to multichannel EEG at successive times to create a new EEG continuously in real time, without redoing the time-consuming SVD. This matrix acts as a spatial filter with useful properties. We successfully applied this method to remove artifacts, including ocular movement and electrocardiographic artifacts. Removal of myogenic artifacts was much less complete, but there was significant improvement in the ability to visualize underlying activity in the presence of myogenic artifacts. The major limitations of the method are its inability to completely separate some artifacts from cerebral activity, especially when both have similar amplitudes, and the possibility that a spatial filter may distort the distribution of activities that overlap with the artifacts being removed.

315 citations


Posted Content
TL;DR: It is shown that the overall stock price can be reconstructed surprisingly well by using a small number of thresholded weighted ICs, and when using shocks derived from principal components instead of independent components, the reconstructed price is less similar to the original one.
Abstract: This paper discusses the application of a modern signal processing technique known as independentcomponent analysis (ICA) or blind source separation to multivariate financial time series such as aportfolio of stocks. The key idea of ICA is to linearly map the observed multivariate time series into a newspace of statistically independent components (ICs). This can be viewed as a factorization of the portfoliosince joint probabilities become simple products in the coordinate system of the ICs.We apply ICA to three years of daily returns of the 28 largest Japanese stocks and compare the results withthose obtained using principal component analysis. The results indicate that the estimated ICs fall into twocategories, (i) infrequent but large shocks (responsible for the major changes in the stock prices), and (ii)frequent smaller fluctuations (contributing little to the overall level of the stocks). We show that the overallstock price can be reconstructed surprisingly well by using a small number of thresholded weighted ICs.In contrast, when using shocks derived from principal components instead of independent components, thereconstructed price is less similar to the original one. Independent component analysis is a potentially powerfulmethod of analyzing and understanding driving mechanisms in financial markets. There are furtherpromising applications to risk management since ICA focuses on higher-order statistics.

291 citations


Journal ArticleDOI
TL;DR: In this paper, the authors apply independent component analysis (ICA) to multivariate financial time series such as a portfolio of stocks and show that the overall stock price can be reconstructed surprisingly well by using a small number of thresholded independent components.
Abstract: This paper explores the appliation of a signal processing technique known as independent component analysis (ICA) or blind source separation to multivariate financial time series such as a portfolio of stocks. The key idea of ICA is to linearly map the observed multivariate time series into a new space of statistically independent components (ICs). We apply ICA to three years of daily returns of the 28 largest Japanese stocks and compare the results with those obtained using principal component analysis. The results indicate that the estimated ICs fall into two categories, (i) infrequent large shocks (responsible for the major changes in the stock prices), and (ii) frequent smaller fluctuations (contributing little to the overall level of the stocks). We show that the overall stock price can be reconstructed surprisingly well by using a small number of thresholded weighted ICs. In contrast, when using shocks derived from principal components instead of independent components, the reconstructed price is le...

272 citations


Journal ArticleDOI
TL;DR: In this paper, an effective procedure on the basis of principal component analysis (PCA) was proposed to optimize the multi-response problems in the Taguchi method, where a set of original responses can be transformed into a set uncorrelated components.
Abstract: Most previous Taguchi method applications have only addressed a single-response problem. However, more than one correlated response normally occurs in a manufactured product. The multi-response problem has received only limited attention. In this work, we propose an effective procedure on the basis of principal component analysis (PCA) to optimize the multi-response problems in the Taguchi method. With the PCA, a set of original responses can be transformed into a set of uncorrelated components. Therefore, the conflict for determining the optimal settings of the design parameters for the multi-response problems can be reduced. Two case studies are evaluated, indicating that the proposed procedure yields a satisfactory result.

254 citations


Journal ArticleDOI
TL;DR: It has been verified experimentally that when nonlinear Principal Component Analysis (PCA) learning rules are used for the weights of a neural layer, the neurons have signal separation capabilities and can be used for image and speech signal separation.

Journal ArticleDOI
TL;DR: The theoretical principles and practical implementation of a new method for multivariate data analysis, maximum likelihood principal component analysis (MLPCA), are described in this article, which is an analog to PCA that incorporates information about measurement errors to develop PCA models that are optimal in a maximum likelihood sense.
Abstract: The theoretical principles and practical implementation of a new method for multivariate data analysis, maximum likelihood principal component analysis (MLPCA), are described. MLCPA is an analog to principal component analysis (PCA) that incorporates information about measurement errors to develop PCA models that are optimal in a maximum likelihood sense. The theoretical foundations of MLPCA are initially established using a regression model and extended to the framework of PCA and singular value decomposition (SVD). An efficient and reliable algorithm based on an alternating regression method is described. Generalization of the algorithm allows its adaptation to cases of correlated errors provided that the error covariance matrix is known. Models with intercept terms can also be accommodated. Simulated data and near-infrared spectra, with a variety of error structures, are used to evaluate the performance of the new algorithm. Convergence times depend on the error structure but are typically around a few minutes. In all cases, models determined by MLPCA are found to be superior to those obtained by PCA when non-uniform error distributions are present, although the level of improvement depends on the error structure of the particular data set. © 1997 John Wiley & Sons, Ltd.

Proceedings ArticleDOI
07 Jul 1997
TL;DR: This paper shows how PCA can be derived from a maximum-likelihood procedure, based on a specialisation of factor analysis, to develop a well-defined mixture model of principal component analyzers, and an expectation-maximisation algorithm for estimating all the model parameters is given.
Abstract: Principal component analysis (PCA) is a ubiquitous technique for data analysis but one whose effective application is restricted by its global linear character. While global nonlinear variants of PCA have been proposed, an alternative paradigm is to capture data nonlinearity by a mixture of local PCA models. However, existing techniques are limited by the absence of a probabilistic formalism with an appropriate likelihood measure and so require an arbitrary choice of implementation strategy. This paper shows how PCA can be derived from a maximum-likelihood procedure, based on a specialisation of factor analysis. This is then extended to develop a well-defined mixture model of principal component analyzers, and an expectation-maximisation algorithm for estimating all the model parameters is given.

Journal ArticleDOI
TL;DR: In this article, the authors examined the increasing use by analytical chemists of chemometric methods for treating classification problems, including principal component analysis (PCA), canonical variates analysis (CVA), discriminant analysis (DA), and discriminant partial least squares (PLS).
Abstract: In this article, we examine the increasing use by analytical chemists of chemometric methods for treating classification problems. The methods considered are principal component analysis (PCA), canonical variates analysis (CVA), discriminant analysis (DA), and discriminant partial least squares (PLS). Overfitting, a potential hazard of multivariate modelling, is illustrated using examples of real and simulated data, and the importance of model validation is discussed.

Journal ArticleDOI
TL;DR: In this paper, the multivariate statistical technique of principal component analysis (PCA) is described and demonstrated to be a valuable tool to consolidate the large amount of information obtained with spectroscopic imaging observations of the interstellar medium.
Abstract: The multivariate statistical technique of principal component analysis (PCA) is described and demonstrated to be a valuable tool to consolidate the large amount of information obtained with spectroscopic imaging observations of the interstellar medium. Simple interstellar cloud models with varying degrees of complexity and Gaussian noise are constructed and analyzed to demonstrate the ability of PCA to statistically extract physical features and phenomena from the data and to gauge the effects of random noise upon the analysis. Principal components are calculated for high spatial dynamic range 12CO and 13CO data cubes of the Sh 155 (Cep OB3) cloud complex. These identify the three major emission components within the cloud and the spatial differences between 12CO and 13CO emissions. Higher order eigenimages identify small velocity fluctuations and therefore provide spatial information to the turbulent velocity field within the cloud. A size line width relationship δv ~ Rα is derived from spatial and kinematic characterizations of the principal components of 12CO emission from the Sh 155, Sh 235, Sh 140, and Gem OB1 cloud complexes. The power-law indices for these clouds range from 0.42 to 0.55 and are similar to those derived from an ensemble of clouds within the Galaxy found by Larson (1981) and Solomon et al. (1987). The size-line width relationship within a given cloud provides an important diagnostic to the variation of kinetic energy with size scale within turbulent flows of the interstellar medium.

Proceedings ArticleDOI
03 Nov 1997
TL;DR: This paper proposes an entropy measure for ranking features, and conducts extensive experiments to show that the method is able to find the important features and compares well with a similar feature ranking method that requires class information unlike this method.
Abstract: Dimensionality reduction is an important problem for efficient handling of large databases. Many feature selection methods exist for supervised data having class information. Little work has been done for dimensionality reduction of unsupervised data in which class information is not available. Principal component analysis (PCA) is often used. However, PCA creates new features. It is difficult to obtain intuitive understanding of the data using the new features only. We are concerned with the problem of determining and choosing the important original features for unsupervised data. Our method is based on the observation that removing an irrelevant feature from the feature set may not change the underlying concept of the data, but not so otherwise. We propose an entropy measure for ranking features, and conduct extensive experiments to show that our method is able to find the important features. Also it compares well with a similar feature ranking method (Relief) that requires class information unlike our method.

Journal ArticleDOI
TL;DR: The hypothesis of constant covariances must be rejected, but the magnitudes of divergence in covariance structure appear to be small, and Matrix correlations are very sensitive to sampling error, while CPC is not.
Abstract: Applications of quantitative techniques to understanding macroevolutionary patterns typically assume that genetic variances and covariances remain constant. That assumption is tested among 28 populations of the Phyllotis darwini species group (leaf-eared mice). Phenotypic covariances are used as a surrogate for genetic covariances to allow much greater phylogenetic sampling. Two new approaches are applied that extend the comparative method to multivariate data. The efficacy of these techniques are compared, and their sensitivity to sampling error examined. Pairwise matrix correlations of correlation matrices are consistently very high (> 0.90) and show no significant association between matrix similarity and phylogenetic relatedness. Hierarchical decomposition of common principal component (CPC) analyses applied to each clade in the phylogeny rejects the hypothesis that common principal component structure is shared in clades more inclusive than subspecies. Most subspecies also lack a common covariance structure as described by the CPC model. The hypothesis of constant covariances must be rejected, but the magnitudes of divergence in covariance structure appear to be small. Matrix correlations are very sensitive to sampling error, while CPC is not. CPC is a powerful statistical tool that allows detailed testing of underlying patterns of covariation.

Journal ArticleDOI
TL;DR: Two methods for examining growth of a microbial community on a particular Biolog substrate using cluster analysis and principal coordinate analysis and an analysis of summary statistics of the profiles, to identify substrates whose profiles distinguish different treatments are described.

Book
01 Jan 1997
TL;DR: In this paper, the general linear model is used for spatial analysis and nonlinear regression is applied in the context of matrix algebra and spatial autoregressive analysis, with a focus on nonlinear regressions.
Abstract: I. INTRODUCTION AND REVIEW. 1. Elementary Statistics Background. 2. Information Content in Geo-Referenced Data. 3. Introduction to Matrix Algebra. 4. Multiple Linear Regression Analysis and Correlation Analysis. II. INSTANCES OF THE GENERAL LINEAR MODEL. 5. Multivariate Analysis of Variance. 6. Principal Components and Factor Analysis. 7. Discriminant Function Analysis. 8. Cluster Analysis. 9. Canonical Correlation Analysis. III. NONLINEAR AND CATEGORICAL DATA MODELING. 10. Nonlinear Regression Analysis. 11. Spatial Autoregressive Analysis. 12. Special Nonlinear Regression Applications in Spatial Analysis. Epilogue. Appendices. Index.

Journal ArticleDOI
TL;DR: In this article, the authors considered the impedance tomography problem of estimating the conductivity distribution within the body from static current/voltage measurements on the body's surface and presented a new method of implementing prior information of the conductivities in the optimization algorithm.
Abstract: In this paper, we consider the impedance tomography problem of estimating the conductivity distribution within the body from static current/voltage measurements on the body's surface. We present a new method of implementing prior information of the conductivities in the optimization algorithm. The method is based on the approximation of the prior covariance matrix by simulated samples of feasible conductivities. The reduction of the dimensionality of the optimization problem is performed by principal component analysis (PCA).

Journal ArticleDOI
TL;DR: A self-validating inferential sensor approach based on principal component analysis (PCA) is proposed, where the input sensors are validated using a fault identification and reconstruction approach proposed in Dunia et al.
Abstract: Inferential sensors, or soft sensors, refer to a modeling approach to estimating hard-to-measure process variables from other easy-to-measure, on-line sensors. Since many sensors are used as input variables to estimate the output, the probability that one of the sensors fails increases significantly. In this paper, we propose a self-validating inferential sensor approach based on principal component analysis (PCA). The input sensors are validated using a fault identification and reconstruction approach proposed in Dunia et al. AIChE J. 1996, 42, 2797−2812. A principal component model is built for the input sensors for sensor validation, and the validated principal components are used to predict output variables using linear regression or neural networks. If a sensor fails, the sensor is identified and reconstructed with the best estimate from its correlation to other sensors. The principal components are also reconstructed accordingly for prediction. The number of principal components used in sensor valid...

Journal ArticleDOI
TL;DR: In this paper, the authors proposed using principal component analysis (PCA) as a means of weighting inputs and outputs and summarizing parsimoniously them rather than selecting them in DEA.
Abstract: In Data Envelopment Analysis (DEA), when there are more inputs and outputs, there are more efficient Decision Making Units (DMus). For example, if the specific inputs or outputs advantageous for a particular DMU are used, the DMU will become efficient. Usually the variables used as inputs or outputs are correlated. Therefore, the inputs and outputs should be selected appropriately by experts who know their characteristics very well. People who are less familiar with those characteristics require tools to assist in the selection. We propose using principal component analysis as a means of weighting inputs and/or outputs and summarizing parsimoniously them rather than selecting them. A basic model and its modification are proposed. In principal component analysis, many weights for the variables that define principal components (PCs) have negative values. This may cause a negative integrated input that is a denominator of the objective function in fractional programming. The denominator should be positive. In the basic model, a condition that the denominator must be positive is added. When the number of PCs is less than the number of original variables, a part of original information is neglected. In the modified model, a part of the neglected information is also used.

Journal ArticleDOI
TL;DR: In this paper, a functional principal component analysis (FPCPA) is proposed to simultaneously estimate n curves corrupted by noise, this means several observations of a random process, which leads to a new kind of functional principal components analysis which simultaneously takes into account a dimensionality and a smoothness constraint.


01 Jan 1997
TL;DR: In this paper, a novel algorithm for analysis of stochastic processes is presented, which can be used to find the required solutions in the cases of principal component analysis (PCA), partial leas...
Abstract: This paper presents a novel algorithm for analysis of stochastic processes. The algorithm can be used to find the required solutions in the cases of principal component analysis (PCA), partial leas ...

Journal ArticleDOI
TL;DR: In this paper, the authors compared the performance of ridge regression and principal component regression (PCR) on spectral data with OLS and partial least squares (PLS) on the basis of two data sets.
Abstract: Ridge regression (RR) and principal component regression (PCR) are two popular methods intended to overcome the problem of multicollinearity which arises with spectral data. The present study compares the performances of RR and PCR in addition to ordinary least squares (OLS) and partial least squares (PLS) on the basis of two data sets. An alternative procedure that combines both PCR and RR is also introduced and is shown to perform well. Furthermore, the performance of the combination of RR and PCR is stable in so far as sufficient information is taken into account. This result suggests discarding those components that are unquestionably identified as noise, when the ridge constant tackles the degeneracy caused by components with small variances.

Journal ArticleDOI
TL;DR: In this paper, a new method, called maximum likelihood principal component analysis (MLPCA), is proposed for multivariate analysis of incomplete data sets, which incorporates measurement error variance information in the decomposition of multivariate data.

Journal ArticleDOI
TL;DR: In this paper, the singular value decomposition analysis (SVD) method is discussed in the context of the simultaneous orthogonal rotation of two matrices and it is demonstrated that the singular vectors are rotated EOFs and the SVD expansion coefficients are rotated sets of principal component expansion coefficients.
Abstract: The singular value decomposition analysis (SVD) method is discussed in the context of the simultaneous orthogonal rotation of two matrices. It is demonstrated that the singular vectors are rotated EOFs and the SVD expansion coefficients are rotated sets of principal component expansion coefficients. This way of thinking about SVD aids in the interpretation of results and provides guidance as to when and how to use SVD.

Journal ArticleDOI
TL;DR: In this paper, principal component analysis (PCA) and correspondence analysis (CA) were applied to analyse potential barriers to total quality management (TQM) implementation in Hong Kong's service and manufacturing industries.
Abstract: Most quality management researchers make inadequate use of statistical techniques, especially multivariate statistical methods. Applies two multivariate analysis techniques, principal component analysis (PCA) and correspondence analysis (CA), to analyse potential barriers to total quality management (TQM) implementation in Hong Kong’s service and manufacturing industries. Describes and demonstrates the applicability of these techniques as analysis tools for quality researchers and practitioners. Conducts PCA on a set of survey data and produces four orthogonal dimensions to potential barriers to TQM implementation, then applies CA in order to corroborate the findings of PCA. In addition, CA provides a simultaneous graphical representation of the data organized under different categories which shows how the potential barriers relate to one another and to the different types of industry.

Journal ArticleDOI
TL;DR: In this article, a forward selection procedure PCR (FSPCR) is evaluated and compared to top-down selection and correlation principal component regression (CPCR) for multivariate calibration.