A generic preprocessing method for multivariate data, called orthogonal projections to latent structures (O-PLS), is described. O-PLS removes variation from X (descriptor variables) that is not correlated to Y (property variables, e.g. yield, cost or toxicity). In mathematical terms this is equivalent to removing systematic variation in X that is orthogonal to Y. In an earlier paper, Wold et al. (Chemometrics Intell. Lab. Syst. 1998; 44: 175-185) described orthogonal signal correction (OSC). In this paper a method with the same objective but with different means is described. The proposed O-PLS method analyzes the variation explained in each PLS component. The non-correlated systematic variation in X is removed, making interpretation of the resulting PLS model easier and with the additional benefit that the non-correlated variation itself can be analyzed further. As an example, near-infrared (NIR) reflectance spectra of wood chips were analyzed. Applying O-PLS resulted in reduced model complexity with preserved prediction ability, effective removal of non-correlated variation in X and, not least, improved interpretational ability of both correlated and non-correlated variation in the NIR spectra.

Orthogonal projections to latent structures (O-PLS)

Pre-processing of near-infrared (NIR) spectral data has become an integral part of chemometrics modeling. The objective of the pre-processing is to remove physical phenomena in the spectra in order to improve the subsequent multivariate regression, classification model or exploratory analysis. The most widely used pre-processing techniques can be divided into two categories: scatter-correction methods and spectral derivatives. This review describes and compares the theoretical and algorithmic foundations of current pre-processing methods plus the qualitative and quantitative consequences of their application. The aim is to provide NIR users with better end-models through fundamental knowledge on spectral pre-processing.

Review of the most common pre-processing techniques for near-infrared spectra

This chapter provides a review on the state of soil visible–near infrared (vis–NIR) spectroscopy Our intention is for the review to serve as a source of up-to-date information on the past and current role of vis–NIR spectroscopy in soil science It should also provide critical discussion on issues surrounding the use of vis–NIR for soil analysis and on future directions To this end, we describe the fundamentals of visible and infrared diffuse reflectance spectroscopy and spectroscopic multivariate calibrations A review of the past and current role of vis–NIR spectroscopy in soil analysis is provided, focusing on important soil attributes such as soil organic matter (SOM), minerals, texture, nutrients, water, pH, and heavy metals We then discuss the performance and generalization capacity of vis–NIR calibrations, with particular attention on sample pretratments, covariations in data sets, and mathematical data preprocessing Field analyses and strategies for the practical use of vis–NIR are considered We conclude that the technique is useful to measure soil water and mineral composition and to derive robust calibrations for SOM and clay content Many studies show that we also can predict properties such as pH and nutrients, although their robustness may be questioned For future work we recommend that research should focus on: (i) moving forward with more theoretical calibrations, (ii) better understanding of the complexity of soil and the physical basis for soil reflection, and (iii) applications and the use of spectra for soil mapping and monitoring, and for making inferences about soils quality, fertility and function To do this, research in soil spectroscopy needs to be more collaborative and strategic The development of the Global Soil Spectral Library might be a step in the right direction

/pdf/visible-and-near-infrared-spectroscopy-in-soil-science-l2f7v53g20.pdf

Visible and Near Infrared Spectroscopy in Soil Science

A review of near infrared spectroscopy and chemometrics in pharmaceutical technologies.

Orthogonal signal correction of near-infrared spectra

Particle size, scatter, and multi-collinearity are long-standing problems encountered in diffuse reflectance spectrometry. Multiplicative combinations of these effects are the major factor inhibiting the interpretation of near-infrared diffuse reflectance spectra. Sample particle size accounts for the majority of the variance, while variance due to chemical composition is small. Procedures are presented whereby physical and chemical variance can be separated. Mathematical transformations—standard normal variate (SNV) and de-trending (DT)—applicable to individual NIR diffuse reflectance spectra are presented. The standard normal variate approach effectively removes the multiplicative interferences of scatter and particle size. De-trending accounts for the variation in baseline shift and curvilinearity, generally found in the reflectance spectra of powdered or densely packed samples, with the use of a second-degree polynomial regression. NIR diffuse NIR diffuse reflectance spectra transposed by these methods are free from multi-collinearity and are not confused by the complexity of shape encountered with the use of derivative spectroscopy.

http://journals.sagepub.com/doi/pdf/10.1366/0003702894202201

Standard Normal Variate Transformation and De-trending of Near-Infrared Diffuse Reflectance Spectra

We demonstrate that set-dependent multiplicative scatter correction and set-independent standard normal variate transformations of NIR spectra are linearly related as theoretically expected. It is shown that the mean and standard deviation of the set-mean-spectrum together with the correlation coefficient between each individual spectrum and set-mean-spectrum are required to link these two transformations. It is through these three quantities, that set-dependency is incorporated into spectra derived by application of multiplicative scatter correction. MSC and SNV are two alternative approaches to reduce particle size effects and they are interconvertible.

The link between Multiplicative Scatter Correction (MSC) and Standard Normal Variate (SNV) transformations of NIR spectra

Correction to the description of Standard Normal Variate (SNV) and De-Trend (DT) Transformations in Practical Spectroscopy with Applications in Food and Beverage Analysis-2"d Edition R.J. Barnes2, M.S. Dhanoa1and S.J. Lister1 11nstitute of Grassland and Environmental Research, Plas Gogerddan, Aberystwyth SY23 3EB, UK 2N1RSystems, Highfield House, Foundation Park, Roxborough Way, Maidenhead, Berkshire SL6 3UD, UK

Correction to the Description of Standard Normal Variate (SNV) and De-Trend (DT) Transformations in Practical Spectroscopy with Applications in Food and Beverage Analysis—2nd Edition

Scale differences of individual near-infrared spectra are identified when set-independent standard normal variate (SNV) and de-trend (DT) transformations are applied in either SNV followed by DT or DT then SNV order. The relationship of set-dependent multiplicative scatter correction (MSC) to SNV is also referred to. A simple correction factor is proposed to convert derived spectra from one order to the other. It is suggested that the suitable order for the study of changes using difference spectra (when removing baselines) should be DT followed by SNV, which leads to all derived spectra on the scale of mean zero and variance equal to one. If baselines are identical, then SNV scale spectra can be used to calculate differences.

On the Scales Associated with Near-Infrared Reflectance Difference Spectra

Monitoring of calibration equation performance is essential if high quality of predicted analytical data is to be sustained. In this paper we outline and illustrate the use of some statistical methods which are well suited for post-prediction data scrutiny. Mean square prediction error is partitioned into three components, viz. mean bias, systematic bias and random error. Reproducibility measures such as concordance correlation (rc), intraclass correlation (r2) and correlation between difference and sum (r(X – Y)(X + Y)) are also discussed. Other topics discussed include the maximisation of R2, type II regression (both variables with error model) and new graphical displays.

R.J. Barnes

Papers

Standard Normal Variate Transformation and De-trending of Near-Infrared Diffuse Reflectance Spectra

The link between Multiplicative Scatter Correction (MSC) and Standard Normal Variate (SNV) transformations of NIR spectra

Correction to the Description of Standard Normal Variate (SNV) and De-Trend (DT) Transformations in Practical Spectroscopy with Applications in Food and Beverage Analysis—2nd Edition

On the Scales Associated with Near-Infrared Reflectance Difference Spectra

Use of Mean Square Prediction Error Analysis and Reproducibility Measures to Study near Infrared Calibration Equation Performance