Showing papers on "Principal component analysis published in 2019"

Robust Sparse Linear Discriminant Analysis

Abstract: Linear discriminant analysis (LDA) is a very popular supervised feature extraction method and has been extended to different variants. However, classical LDA has the following problems: 1) The obtained discriminant projection does not have good interpretability for features; 2) LDA is sensitive to noise; and 3) LDA is sensitive to the selection of number of projection directions. In this paper, a novel feature extraction method called robust sparse linear discriminant analysis (RSLDA) is proposed to solve the above problems. Specifically, RSLDA adaptively selects the most discriminative features for discriminant analysis by introducing the $l_{2,1}$ norm. An orthogonal matrix and a sparse matrix are also simultaneously introduced to guarantee that the extracted features can hold the main energy of the original data and enhance the robustness to noise, and thus RSLDA has the potential to perform better than other discriminant methods. Extensive experiments on six databases demonstrate that the proposed method achieves the competitive performance compared with other state-of-the-art feature extraction methods. Moreover, the proposed method is robust to the noisy data.

Fault Diagnosis Method Based on Principal Component Analysis and Broad Learning System

Abstract: Traditional feature extraction methods are used to extract the features of signal to construct the fault feature matrix, which exists the complex structure, higher correlation, and redundancy. This will increase the complex fault classification and seriously affect the accuracy and efficiency of fault identification. In order to solve these problems, a new fault diagnosis (PABSFD) method based on the principal component analysis (PCA) and the broad learning system (BLS) is proposed for rotor system in this paper. In the proposed PABSFD method, the PCA with revealing the signal essence is used to reduce the dimension of the constructed feature matrix and decrease the linear feature correlation between data and eliminate the redundant attributes in order to obtain the low-dimensional feature matrix with retaining the essential features for the classification model. Then, the BLS with low time complexity and high classification accuracy is regarded as a classification model to realize the fault identification; it can efficiently accomplish the fault classification of rotor system. Finally, the actual vibration data of rotor system are selected to test and verify the effectiveness of the PABSFD method. The experimental results show that the PCA method can effectively eliminate the feature correlation and realize the dimension reduction of the feature matrix, the BLS can take on better adaptability, faster computation speed, and higher classification accuracy, and the PABSFD method can efficiently and accurately obtain the fault diagnosis results.

Combining Principal Component Analysis, Discrete Wavelet Transform and XGBoost to trade in the financial markets

Abstract: When investing in financial markets it is crucial to determine a trading signal that can provide the investor with the best entry and exit points of the financial market, however this is a difficult task and has become a very popular research topic in the financial area. This paper presents an expert system in the financial area that combines Principal Component Analysis (PCA), Discrete Wavelet Transform (DWT), Extreme Gradient Boosting (XGBoost) and a Multi-Objective Optimization Genetic Algorithm (MOO-GA) in order to achieve high returns with a low level of risk. PCA is used to reduce the dimensionality of the financial input data set and the DWT is used to perform a noise reduction to every feature. The resultant data set is then fed to an XGBoost binary classifier that has its hyperparameters optimized by a MOO-GA. The importance of the PCA is analyzed and the results obtained show that it greatly improves the performance of the system. In order to improve even more the results obtained in the system using PCA, the PCA and the DWT are then applied together in one system and the results obtained show that this system is capable of outperforming the Buy and Hold (B&H) strategy in three of the five analyzed financial markets, achieving an average rate of return of 49.26% in the portfolio, while the B&H achieves on average 32.41%.

Distributed estimation of principal eigenspaces.

Abstract: Principal component analysis (PCA) is fundamental to statistical machine learning. It extracts latent principal factors that contribute to the most variation of the data. When data are stored across multiple machines, however, communication cost can prohibit the computation of PCA in a central location and distributed algorithms for PCA are thus needed. This paper proposes and studies a distributed PCA algorithm: each node machine computes the top K eigenvectors and transmits them to the central server; the central server then aggregates the information from all the node machines and conducts a PCA based on the aggregated information. We investigate the bias and variance for the resulting distributed estimator of the top K eigenvectors. In particular, we show that for distributions with symmetric innovation, the empirical top eigenspaces are unbiased and hence the distributed PCA is "unbiased". We derive the rate of convergence for distributed PCA estimators, which depends explicitly on the effective rank of covariance, eigen-gap, and the number of machines. We show that when the number of machines is not unreasonably large, the distributed PCA performs as well as the whole sample PCA, even without full access of whole data. The theoretical results are verified by an extensive simulation study. We also extend our analysis to the heterogeneous case where the population covariance matrices are different across local machines but share similar top eigen-structures.

Principal Component Analysis of High-Frequency Data

Abstract: We develop the necessary methodology to conduct principal component analysis at high frequency. We construct estimators of realized eigenvalues, eigenvectors, and principal components, and provide ...

Applied Compositional Data Analysis: With Worked Examples in R

Abstract: Preface.- Acknowledgements.- Compositional data as a methodological concept.- Analyzing compositional data using R.- Geometrical properties of compositional data.- Exploratory data analysis and visualization.- First steps for a statistical analysis.- Cluster analysis.- Principal component analysis.- Correlation analysis.- Discriminant analysis.- Regression analysis.- Methods for high-dimensional compositional data.- Compositional tables.- Preprocessing issues.- Index.-

Face Detection and Recognition Using OpenCV

Abstract: Face detection and picture or video recognition is a popular subject of research on biometrics. Face recognition in a real-time setting has an exciting area and a rapidly growing challenge. Framework for the use of face recognition application authentication. This proposes the PCA (Principal Component Analysis) facial recognition system. The key component analysis (PCA) is a statistical method under the broad heading of factor analysis. The aim of the PCA is to reduce the large amount of data storage to the size of the feature space that is required to represent the data economically. The wide 1-D pixel vector made of the 2-D face picture in compact main elements of the space function is designed for facial recognition by the PCA. This is called a projection of self-space. The proper space is determined with the identification of the covariance matrix’s own vectors, which are centered on a collection of fingerprint images. I build a camera-based real-time face recognition system and set an algorithm by developing programming on OpenCV, Haar Cascade, Eigenface, Fisher Face, LBPH, and Python.

A comparison of dimension reduction techniques for support vector machine modeling of multi-parameter manufacturing quality prediction

Abstract: Manufacturing quality prediction model, as an effective measure to monitor the quality in advance, has been developed using various data-driven techniques. However, multi-parameter in multi-stage of the modern manufacturing industry brings about the curse of dimensionality, leading to the difficulties for feature extraction, learning and quality modeling. To address this issue, three dimension reduction techniques are investigated in this paper, i.e., principal component analysis (PCA), locally linear embedding (LLE), and isometric mapping (Isomap). Specifically, the PCA is a linear dimension reduction technique, the LLE is a nonlinear reduction technique with local perspective, and the Isomap is a nonlinear reduction technique from global perspective. After getting the low-dimensional information from the PCA, the LLE, and the Isomap methods respectively, a support vector machine (SVM) is utilized for modeling. To reveal the effectiveness of the dimension reduction techniques and compare the difference of the three dimension reduction techniques, two experimental manufacturing data are collected from a competition about manufacturing quality control in Tianchi Data Lab of China. The comparison experiments indicate that the dimension reduction techniques have capacity for improving the SVM modeling performance indeed, and the Isomap–SVM model with the nonlinear global dimension reduction outperforms all the candidate models in terms of qualitative and quantitative analysis.

A New Formulation of Linear Discriminant Analysis for Robust Dimensionality Reduction

Abstract: Dimensionality reduction is a critical technology in the domain of pattern recognition, and linear discriminant analysis (LDA) is one of the most popular supervised dimensionality reduction methods. However, whenever its distance criterion of objective function uses $L_2$ -norm, it is sensitive to outliers. In this paper, we propose a new formulation of linear discriminant analysis via joint $L_{2,1}$ -norm minimization on objective function to induce robustness, so as to efficiently alleviate the influence of outliers and improve the robustness of proposed method. An efficient iterative algorithm is proposed to solve the optimization problem and proved to be convergent. Extensive experiments are performed on an artificial data set, on UCI data sets, and on four face data sets, which sufficiently demonstrates the efficiency of comparing to other methods and robustness to outliers of our approach.

Use and Misuse of PCA for Measuring Well-Being

Abstract: The measurement of well-being of people is very difficult because it is characterized by a multiplicity of aspects or dimensions. Principal Components Analysis (PCA) is probably the most popular multivariate statistical technique for reducing data with many dimensions and, often, well-being indicators are reduced to a single index of well-being by using PCA. However, PCA is implicitly based on a reflective measurement model that is not suitable for all types of indicators. In this paper, we discuss the use and misuse of PCA for measuring well-being, and we show some applications to real data.

Bat algorithm with principal component analysis

Abstract: The bat algorithm (BA) is a novel evolutionary optimization algorithm, most studies of which have been performed with low-dimensional problems. To test and improve the global search ability of BA with large-scale problems, two new variants using principal component analysis (PCA_BA and PCA_LBA) are designed in this paper. A correlation threshold and generation threshold are determined using the golden section method to enhance the effectiveness of this new strategy. To test performance, CEC’2008 large-scale benchmark functions are utilized and compared with other algorithms; simulation results indicate the validity of this modification.

Combining Hierarchical Clustering approaches using the PCA Method

Abstract: In expert systems, data mining methods are algorithms that simulate humans’ problem-solving capabilities. Clustering methods as unsupervised machine learning methods are crucial approaches to categorize similar samples in the same categories. The use of different clustering algorithms to a given dataset produces clusters with different qualities. Hence, many researchers have applied clustering combination methods to reduce the risk of choosing an inappropriate clustering algorithm. In these methods, the outputs of several clustering algorithms are combined. In these research works, the input hierarchical clusterings are transformed to descriptor matrices and their combination is achieved by aggregating their descriptor matrices. In previous works, only element-wise aggregation operators have been used and the relation between the elements of each descriptor matrix has been ignored. However, the value of each element of the descriptor matrix is meaningful in comparison with its other elements. The current study proposes a novel method of combining hierarchical clustering approaches based on principle component analysis (PCA). PCA as an aggregator allows considering all elements of the descriptor matrices. In the proposed approach, basic clusters are made and transformed to descriptor matrices. Then, a final matrix is extracted from the descriptor matrices using PCA. Next, a final dendrogram is constructed from the matrix that is used to summarize the results of the diverse clustering. The experimental results on popular available datasets show the superiority of the clustering accuracy of the proposed method over basic clustering methods such as single, average and centroid linkage and previously combined hierarchical clustering methods. In addition, statistical tests show that the proposed method significantly outperformed hierarchical clustering combination methods with element-wise averaging operators in almost all tested datasets. Several experiments have also been conducted which confirm the robustness of the proposed method for its parameter setting.

Power Quality Disturbance Monitoring and Classification Based on Improved PCA and Convolution Neural Network for Wind-Grid Distribution Systems

Abstract: The excessive use of power semiconductor devices in a grid utility increases the malfunction of the control system, produces power quality disturbances (PQDs) and reduces the electrical component life. The present work proposes a novel algorithm based on Improved Principal Component Analysis (IPCA) and 1-Dimensional Convolution Neural Network (1-D-CNN) for detection and classification of PQDs. Firstly, IPCA is used to extract the statistical features of PQDs such as Root Mean Square, Skewness, Range, Kurtosis, Crest Factor, Form Factor. IPCA is decomposed into four levels. The principal component (PC) is obtained by IPCA, and it contains a maximum amount of original data as compare to PCA. 1-D-CNN is also used to extract features such as mean, energy, standard deviation, Shannon entropy, and log-energy entropy. The statistical analysis is employed for optimal feature selection. Secondly, these improved features of the PQDs are fed to the 1-D-CNN-based classifier to gain maximum classification accuracy. The proposed IPCA-1-D-CNN is utilized for classification of 12 types of synthetic and simulated single and multiple PQDs. The simulated PQDs are generated from a modified IEEE bus system with wind energy penetration in the balanced distribution system. Finally, the proposed IPCA-1-D-CNN algorithm has been tested with noise (50 dB to 20 dB) and noiseless environment. The obtained results are compared with SVM and other existing techniques. The comparative results show that the proposed method gives significantly higher classification accuracy.

Motor Fault Detection and Feature Extraction Using RNN-Based Variational Autoencoder

Abstract: In most of the fault detection methods, the time domain signals collected from the mechanical equipment usually need to be transformed into frequency domain or other high-level data, highly relying on professional knowledge such as signal processing and fault pattern recognition. Contrary to those existing approaches, we proposed a two-stage machine learning analysis architecture which can accurately predict the motor fault modes only by using motor vibration time-domain signals without any complicated preprocessing. In the first stage, the method RNN-based VAE was proposed which is highly suitable for dimension reduction of time series data. In addition to reducing the dimension of sequential data from 150*3 to 25 dimensions, our method furthermore improves the prediction accuracy evaluated by several classification algorithms. While other dimension reduction methods such as Autoencoder and Variational Autoencoder cannot improve the classification accuracy effectively or even decreased. It indicates that the sequential data after dimension reduction via the RNN-based VAE still can maintain the high-dimensional data information. Furthermore, the experimental results demonstrate that it can be well applied to time series data dimension reduction and shows a significant improvement of the prediction performance, even with a simple double-layer Neural Network can reach over 99% of accuracy. In the second stage, Principal Components Analysis (PCA) and Linear Discriminant Analysis (LDA) are used to further perform the second dimension reduction, such that the different or unknown fault modes can be clearly visualized and detected.

Be careful with your principal components.

Abstract: Principal components analysis (PCA) is a common method to summarize a larger set of correlated variables into a smaller and more easily interpretable axes of variation. However, the different components need to be distinct from each other to be interpretable otherwise they only represent random directions. This is a fundamental assumption of PCA and, thus, needs to be tested every time. Sample correlation matrices will always result in a pattern of decreasing eigenvalues even if there is no structure. Tests are, therefore, needed to discern real patterns from illusionary ones. Furthermore, the loadings of the vectors need to be larger than expected by random data to be useful in the calculation of PC-scores. PC-scores calculated from nondistinct PC's have very large standard errors and cannot be used for biological interpretations. I give a number of examples to illustrate the potential problems with PCA. Robustness of the PC's increases with increasing sample size but not with the number of traits. I review a few simple test statistics appropriate for testing PC's and use a real-world example to illustrate how this can be done using randomization tests. PCA can be very useful but great care is needed to avoid spurious results.

Deep Principal Component Analysis Based on Layerwise Feature Extraction and Its Application to Nonlinear Process Monitoring

Abstract: In order to deeply exploit intrinsic data feature information hidden among the process data, an improved kernel principal component analysis (KPCA) method is proposed, which is referred to as deep principal component analysis (DePCA). Specifically, motivated by the deep learning strategy, we design a hierarchical statistical model structure to extract multilayer data features, including both the linear and nonlinear principal components. To reduce the computation complexity in nonlinear feature extraction, the feature-samples’ selection technique is applied to build the sparse kernel model for DePCA. To integrate the monitoring statistics at each feature layer, Bayesian inference is used to transform the monitoring statistics into fault probabilities, and then, two probability-based DePCA monitoring statistics are constructed by weighting the fault probabilities at all the feature layers. Two case studies involving a simulated nonlinear system and the benchmark Tennessee Eastman process demonstrate the superior fault detection performance of the proposed DePCA method over the traditional KPCA-based methods.

Partially Linear Functional Additive Models for Multivariate Functional Data

Abstract: We investigate a class of partially linear functional additive models (PLFAM) that predicts a scalar response by both parametric effects of a multivariate predictor and nonparametric effects of a multivariate functional predictor. We jointly model multiple functional predictors that are cross-correlated using multivariate functional principal component analysis (mFPCA), and model the nonparametric effects of the principal component scores as additive components in the PLFAM. To address the high-dimensional nature of functional data, we let the number of mFPCA components diverge to infinity with the sample size, and adopt the component selection and smoothing operator (COSSO) penalty to select relevant components and regularize the fitting. A fundamental difference between our framework and the existing high-dimensional additive models is that the mFPCA scores are estimated with error, and the magnitude of measurement error increases with the order of mFPCA. We establish the asymptotic convergence ...

A Novel Hybrid Method Integrating ICA-PCA With Relevant Vector Machine for Multivariate Process Monitoring

Abstract: This brief proposes an independent component analysis-principal component analysis (ICA-PCA) integrating with relevance vector machine (RVM) for multivariate process monitoring. Given the fact that the distribution of industrial process variables is mostly non-Gaussian and PCA cannot well deal with the non-Gaussian part. A hybrid ICA-PCA method is proposed to simultaneously extract the non-Gaussian and Gaussian information of multivariate processes. ICA is first used to monitor the non-Gaussian part of the process and then the Gaussian part of the residual process can be extracted using PCA. After feature extraction, a Bayesian-based classifier named RVM is established to make fault detection for the sake of both preventing the chosen of threshold as in traditional method and compensating for the single statistic. The performance of the proposed approach is validated using the Tennessee Eastman process. Simulation results verified the effectiveness of the proposed method.