https://libpls.net/publication/PLS_basic_2001.pdf

PLS-regression: a basic tool of chemometrics

Partial least squares (PLS) regression (a.k.a. projection on latent structures) is a recent technique that combines features from and generalizes principal component analysis (PCA) and multiple linear regression. Its goal is to predict a set of dependent variables from a set of independent variables or predictors. This prediction is achieved by extracting from the predictors a set of orthogonal factors called latent variables which have the best predictive power. These latent variables can be used to create displays akin to PCA displays. The quality of the prediction obtained from a PLS regression model is evaluated with cross-validation techniques such as the bootstrap and jackknife. There are two main variants of PLS regression: The most common one separates the roles of dependent and independent variables; the second one—used mostly to analyze brain imaging data—gives the same roles to dependent and independent variables. Copyright © 2010 John Wiley & Sons, Inc.

For further resources related to this article, please visit the WIREs website.

https://www.utdallas.edu/~herve/abdi-wireCS-PLS2010.pdf

Partial least squares regression and projection on latent structure regression (PLS Regression)

This article provides an overview of methods for reliability assessment of quantitative structure-activity relationship (QSAR) models in the context of regulatory acceptance of human health and environmental QSARs. Useful diagnostic tools and data analytical approaches are highlighted and exemplified. Particular emphasis is given to the question of how to define the applicability borders of a QSAR and how to estimate parameter and prediction uncertainty. The article ends with a discussion regarding QSAR acceptability criteria. This discussion contains a list of recommended acceptability criteria, and we give reference values for important QSAR performance statistics. Finally, we emphasize that rigorous and independent validation of QSARs is an essential step toward their regulatory acceptance and implementation.

Methods for reliability and uncertainty assessment and for applicability evaluations of classification- and regression-based QSARs.

https://www.utdallas.edu/~herve/abdi-kwmaPLS4NeuroImage2010.pdf

Partial Least Squares (PLS) methods for neuroimaging: a tutorial and review.

Partial least squares (PLS) is an efficient statistical regression technique that is highly suited for the analysis of genomic and proteomic data. In this article, we review both the theory underlying PLS as well as a host of bioinformatics applications of PLS. In particular, we provide a systematic comparison of the PLS approaches currently employed, and discuss analysis problems as diverse as, e.g. tumor classification from transcriptome data, identification of relevant genes, survival analysis and modeling of gene networks and transcription factor activities.

https://epub.ub.uni-muenchen.de/1826/1/paper_457.pdf

Partial least squares: a versatile tool for the analysis of high-dimensional genomic data

A set of frameworks for latent variable multivariate regression method is developed. The first two of these frameworks describe the objective functions satisfied by the latent variables chosen in canonical coordinates regression (CCR), reduced rank regression (RRR) and SIMPLS. These frameworks show the methods as a natural progression from CCR (maximizing correlation) to SIMPLS (maximizing covariance) via RRR (which is an intermediate method). These frameworks are unique in that they look at these methods in terms of latent variables in both the X- and Y-spaces. This adds insight to the nature of the latent variables being chosen. These frameworks are then extended to include PLS for latent variables beyond the first component. This new framework provides a detailed description of the objective function satisfied by PLS latent variables for the multivariate case. It also includes CCR, RRR and SIMPLS, allowing comparisons between the methods. A further framework suggests a new method, undeflated PLS (UDPLS), which adds insight to the effect of the deflation process on PLS. The impact of the objective functions on each of the methods is illustrated on real data from a mineral sorting plant.

Frameworks for latent variable multivariate regression

Latent variable multivariate regression modeling

In standard linear regression where the predictor matrix X is of full rank, the regression coefficients are clearly defined as the parameters B appearing in the linear regression model. In latent variable models there is no direct relationship between the predictor variables and response variables. Rather they are both related to an underlying reduced-rank set of latent variables. Recent papers have proposed different methods for obtaining approximate covariance matrices for the estimates of the regression coefficients from methods such as partial least squares (PLS) and for using them to determine ‘confidence intervals’, for variable selection and for judging variable importance. However, in the latent variable model a matrix of regression coefficients, B, does not even appear as a parameter matrix. In the situation where the data follow such a model, it is therefore uncertain how the regression coefficients and, by extension, any covariance matrices and ‘confidence intervals’ should be interpreted. In this paper we show that any inference is critically dependent upon how one defines these regression coefficients. Two definitions for the regression coefficients are given that are consistent with the latent variable model. Which of these definitions is more relevant is shown to be highly dependent on the goals of the analysis. Therefore one must be clear on the definition one is using for these coefficients when building predictive models, when screening variables based on them or when using them to make interpretations about the system. Under standard normality assumptions, different estimation methods such as ordinary least squares (OLS) and PLS are shown to provide very different distributions for the regression coefficient estimates when the data follow a latent variable model. This is shown to be not just a matter of the PLS coefficients being biased or the OLS estimates having large variance, but of more complex differences implied by the structure of the model parameters in the latent variable model. How the distributions for these estimates relate to the definitions given in this paper is explored here. It is shown for a simple case that the relative size of the PLS estimates, on average, tends to reflect the latent variable loadings, whereas the relative size of the OLS estimates, on average, is a function not only of the loadings but also of the error variances for the predictor variables. Thus in this particular case it appears that the relative size of the B parameters from PLS reflects the underlying latent structure, whereas those from OLS also reflect the error structure of the predictor variables. Copyright © 2001 John Wiley & Sons, Ltd.

Interpretation of regression coefficients under a latent variable regression model

A statistical framework is developed to contrast methods used for parameter estimation for a latent variable multivariate regression (LVMR) model. This model involves two sets of variables, X and Y, both with multiple variables and sharing a common latent structure with additive random errors. The methods contrasted are partial least squares (PLS) regression, principal component regression (PCR), reduced rank regression (RRR) and canonical co-ordinate regression (CCR). The framework is based on a constrained maximum likelihood analysis of the model under assumptions of multivariate normality. The constraint is that the estimates of the latent variables are restricted to be linear functions of the X variables, which is the form of the estimates for the methods being contrasted. The resulting framework is a continuum regression that goes from RRR to PCR depending on the ratio of error variances in the X and Y spaces. PLS does not arise as a member of the continuum; however, the method does offer some insight into why PLS would work well in practice. The constrained maximum likelihood result is also compared with the unconstrained maximum likelihood analysis to investigate the impact of the constraint. The results are illustrated on a simulated example. Copyright © 1999 John Wiley & Sons, Ltd.

Alison J. Burnham

Papers

Frameworks for latent variable multivariate regression

Latent variable multivariate regression modeling

Interpretation of regression coefficients under a latent variable regression model

A statistical framework for multivariate latent variable regression methods based on maximum likelihood

Predicting multivariate responses in multiple linear regression - Discussion