Large datasets are increasingly common and are often difficult to interpret. Principal component analysis (PCA) is a technique for reducing the dimensionality of such datasets, increasing interpretability but at the same time minimizing information loss. It does so by creating new uncorrelated variables that successively maximize variance. Finding such new variables, the principal components, reduces to solving an eigenvalue/eigenvector problem, and the new variables are defined by the dataset at hand, not a priori , hence making PCA an adaptive data analysis technique. It is adaptive in another sense too, since variants of the technique have been developed that are tailored to various different data types and structures. This article will begin by introducing the basic ideas of PCA, discussing what it can and cannot do. It will then describe some variants of PCA and their application.

https://royalsocietypublishing.org/doi/pdf/10.1098/rsta.2015.0202

Principal component analysis: a review and recent developments

Introduction.- Data management and software.- Advice for teachers.- Exploration.- Linear regression.- Generalised linear modelling.- Additive and generalised additive modelling.- Introduction to mixed modelling.- Univariate tree models.- Measures of association.- Ordination--first encounter.- Principal component analysis and redundancy analysis.- Correspondence analysis and canonical correspondence analysis.- Introduction to discriminant analysis.- Principal coordinate analysis and non-metric multidimensional scaling.- Time series analysis--Introduction.- Common trends and sudden changes.- Analysis and modelling lattice data.- Spatially continuous data analysis and modelling.- Univariate methods to analyse abundance of decapod larvae.- Analysing presence and absence data for flatfish distribution in the Tagus estuary, Portugual.- Crop pollination by honeybees in an Argentinean pampas system using additive mixed modelling.- Investigating the effects of rice farming on aquatic birds with mixed modelling.- Classification trees and radar detection of birds for North Sea wind farms.- Fish stock identification through neural network analysis of parasite fauna.- Monitoring for change: using generalised least squares, nonmetric multidimensional scaling, and the Mantel test on western Montana grasslands.- Univariate and multivariate analysis applied on a Dutch sandy beach community.- Multivariate analyses of South-American zoobenthic species--spoilt for choice.- Principal component analysis applied to harbour porpoise fatty acid data.- Multivariate analysis of morphometric turtle data--size and shape.- Redundancy analysis and additive modelling applied on savanna tree data.- Canonical correspondence analysis of lowland pasture vegetation in the humid tropics of Mexico.- Estimating common trends in Portuguese fisheries landings.- Common trends in demersal communities on the Newfoundland-Labrador Shelf.- Sea level change and salt marshes in the Wadden Sea: a time series analysis.- Time series analysis of Hawaiian waterbirds.- Spatial modelling of forest community features in the Volzhsko-Kamsky reserve.

Analysing Ecological Data

We introduce glmulti, an R package for automated model selection and multi-model inference with glm and related functions. From a list of explanatory variables, the provided function glmulti builds all possible unique models involving these variables and, optionally, their pairwise interactions. Restrictions can be specified for candidate models, by excluding specific terms, enforcing marginality, or controlling model complexity. Models are fitted with standard R functions like glm. The n best models and their support (e.g., (Q)AIC, (Q)AICc, or BIC) are returned, allowing model selection and multi-model inference through standard R functions. The package is optimized for large candidate sets by avoiding memory limitation, facilitating parallelization and providing, in addition to exhaustive screening, a compiled genetic algorithm method. This article briefly presents the statistical framework and introduces the package, with applications to simulated and real data.

glmulti: An R Package for Easy Automated Model Selection with (Generalized) Linear Models

In many multivariate statistical techniques, a set of linear functions of the original p variables is produced. One of the more difficult aspects of these techniques is the interpretation of the linear functions, as these functions usually have nonzero coefficients on all p variables. A common approach is to effectively ignore (treat as zero) any coefficients less than some threshold value, so that the function becomes simple and the interpretation becomes easier for the users. Such a procedure can be misleading. There are alternatives to principal component analysis which restrict the coefficients to a smaller number of possible values in the derivation of the linear functions, or replace the principal components by “principal variables.” This article introduces a new technique, borrowing an idea proposed by Tibshirani in the context of multiple regression where similar problems arise in interpreting regression equations. This approach is the so-called LASSO, the “least absolute shrinkage and selection o...

/pdf/a-modified-principal-component-technique-based-on-the-lasso-5gjjknb261.pdf

A Modified Principal Component Technique Based on the LASSO

In recent years, there has been an exponential growth in the number of complex documents and texts that require a deeper understanding of machine learning methods to be able to accurately classify texts in many applications. Many machine learning approaches have achieved surpassing results in natural language processing. The success of these learning algorithms relies on their capacity to understand complex models and non-linear relationships within data. However, finding suitable structures, architectures, and techniques for text classification is a challenge for researchers. In this paper, a brief overview of text classification algorithms is discussed. This overview covers different text feature extractions, dimensionality reduction methods, existing algorithms and techniques, and evaluations methods. Finally, the limitations of each technique and their application in real-world problems are discussed.

Text Classification Algorithms: A Survey

Computational aspects of algorithms for variable selection in the context of principal components

Principal component analysis is widely used in the analysis of multivariate data in the agricultural, biological, and environmental sciences. The first few principal components (PCs) of a set of variables are derived variables with optimal properties in terms of approximating the original variables. This paper considers the problem of identifying subsets of variables that best approximate the full set of variables or their first few PCs, thus stressing dimensionality reduction in terms of the original variables rather than in terms of derived variables (PCs) whose definition requires all the original variables. Criteria for selecting variables are often ill defined and may produce inappropriate subsets. Indicators of the performance of different subsets of the variables are discussed and two criteria are defined. These criteria are used in stepwise selection-type algorithms to choose good subsets. Examples are given that show, among other things, that the selection of variable subsets should not be based only on the PC loadings of the variables.

Variable selection and the interpretation of principal subspaces

Principal component analysis (PCA) can be seen as a singular value decomposition (SVD) of a column-centred data matrix. In a number of applications, no pre-processing of the data is carried out, and it is the uncentred data matrix that is subjected to an SVD, in what is often called an uncentred PCA. This paper explores the relationships between the results from both the standard, column-centred, PCA, and its uncentred counterpart. In particular, it obtains both exact results and bounds relating the eigenvalues and eigenvectors of the covariation matrices, as well as the principal components, in both types of analysis. These relationships highlight how the eigenvalues of both the covariance matrix and the matrix of non-central second moments contain much information that is highly informative for a comparative assessment of PCA and its uncentred variant. The relations and the examples also suggest that the results of both types of PCA have more in common than might be supposed.

On relationships between uncentred and column-centred principal component analysis

The separation of morphometric variation into a component related to size and other components associated with shape is of considerable interest and has generated much discussion in the literature. One class of approaches to achieving this separation is based on principal component analysis. A new technique is proposed within this class, which overcomes some of the disadvantages of existing approaches.

https://www.jstor.org/stable/info/2532909

Jorge Cadima

Papers

Principal component analysis: a review and recent developments

Computational aspects of algorithms for variable selection in the context of principal components

Variable selection and the interpretation of principal subspaces

On relationships between uncentred and column-centred principal component analysis

Size- and shape-related principal component analysis