Standard Normal Variate Transformation and De-trending of Near-Infrared Diffuse Reflectance Spectra
TL;DR: In this article, the standard normal variate (SNV) and de-trending (DT) approaches are applied to individual NIR diffuse reflectance spectra to remove the multiplicative interferences of scatter and particle size.
Abstract: Particle size, scatter, and multi-collinearity are long-standing problems encountered in diffuse reflectance spectrometry. Multiplicative combinations of these effects are the major factor inhibiting the interpretation of near-infrared diffuse reflectance spectra. Sample particle size accounts for the majority of the variance, while variance due to chemical composition is small. Procedures are presented whereby physical and chemical variance can be separated. Mathematical transformations—standard normal variate (SNV) and de-trending (DT)—applicable to individual NIR diffuse reflectance spectra are presented. The standard normal variate approach effectively removes the multiplicative interferences of scatter and particle size. De-trending accounts for the variation in baseline shift and curvilinearity, generally found in the reflectance spectra of powdered or densely packed samples, with the use of a second-degree polynomial regression. NIR diffuse NIR diffuse reflectance spectra transposed by these methods are free from multi-collinearity and are not confused by the complexity of shape encountered with the use of derivative spectroscopy.
Citations
More filters
••
TL;DR: In this article, a generic preprocessing method for multivariate data, called orthogonal projections to latent structures (O-PLS), is described, which removes variation from X (descriptor variables) that is not correlated to Y (property variables, e.g. yield, cost or toxicity).
Abstract: A generic preprocessing method for multivariate data, called orthogonal projections to latent structures (O-PLS), is described. O-PLS removes variation from X (descriptor variables) that is not correlated to Y (property variables, e.g. yield, cost or toxicity). In mathematical terms this is equivalent to removing systematic variation in X that is orthogonal to Y. In an earlier paper, Wold et al. (Chemometrics Intell. Lab. Syst. 1998; 44: 175-185) described orthogonal signal correction (OSC). In this paper a method with the same objective but with different means is described. The proposed O-PLS method analyzes the variation explained in each PLS component. The non-correlated systematic variation in X is removed, making interpretation of the resulting PLS model easier and with the additional benefit that the non-correlated variation itself can be analyzed further. As an example, near-infrared (NIR) reflectance spectra of wood chips were analyzed. Applying O-PLS resulted in reduced model complexity with preserved prediction ability, effective removal of non-correlated variation in X and, not least, improved interpretational ability of both correlated and non-correlated variation in the NIR spectra.
2,096 citations
••
TL;DR: This review describes and compares the theoretical and algorithmic foundations of current pre- processing methods plus the qualitative and quantitative consequences of their application to provide NIR users with better end-models through fundamental knowledge on spectral pre-processing.
Abstract: Pre-processing of near-infrared (NIR) spectral data has become an integral part of chemometrics modeling. The objective of the pre-processing is to remove physical phenomena in the spectra in order to improve the subsequent multivariate regression, classification model or exploratory analysis. The most widely used pre-processing techniques can be divided into two categories: scatter-correction methods and spectral derivatives. This review describes and compares the theoretical and algorithmic foundations of current pre-processing methods plus the qualitative and quantitative consequences of their application. The aim is to provide NIR users with better end-models through fundamental knowledge on spectral pre-processing.
1,942 citations
••
TL;DR: A review on the state of soil visible-near infrared (vis-NIR) spectroscopy is provided in this article, focusing on important soil attributes such as soil organic matter (SOM), minerals, texture, nutrients, water, pH, and heavy metals.
Abstract: This chapter provides a review on the state of soil visible–near infrared (vis–NIR) spectroscopy Our intention is for the review to serve as a source of up-to-date information on the past and current role of vis–NIR spectroscopy in soil science It should also provide critical discussion on issues surrounding the use of vis–NIR for soil analysis and on future directions To this end, we describe the fundamentals of visible and infrared diffuse reflectance spectroscopy and spectroscopic multivariate calibrations A review of the past and current role of vis–NIR spectroscopy in soil analysis is provided, focusing on important soil attributes such as soil organic matter (SOM), minerals, texture, nutrients, water, pH, and heavy metals We then discuss the performance and generalization capacity of vis–NIR calibrations, with particular attention on sample pretratments, covariations in data sets, and mathematical data preprocessing Field analyses and strategies for the practical use of vis–NIR are considered We conclude that the technique is useful to measure soil water and mineral composition and to derive robust calibrations for SOM and clay content Many studies show that we also can predict properties such as pH and nutrients, although their robustness may be questioned For future work we recommend that research should focus on: (i) moving forward with more theoretical calibrations, (ii) better understanding of the complexity of soil and the physical basis for soil reflection, and (iii) applications and the use of spectra for soil mapping and monitoring, and for making inferences about soils quality, fertility and function To do this, research in soil spectroscopy needs to be more collaborative and strategic The development of the Global Soil Spectral Library might be a step in the right direction
1,063 citations
Cites background from "Standard Normal Variate Transformat..."
...9D) with or without detrending (Barnes et al., 1989) and orthogonal signal correction (OSC) (Wold et al....
[...]
...9D) with or without detrending (Barnes et al., 1989) and orthogonal signal correction (OSC) (Wold et al., 1998)....
[...]
••
TL;DR: It is shown how a variant of PLS can be used to achieve a signal correction that is as close to orthogonal as possible to a given Y-vector or Y-matrix and is applied to four different data sets of multivariate calibration.
1,003 citations
••
TL;DR: In this article, the authors present an overview of the type of information that can be obtained based on some developed theory and food research of near infrared reflectance spectroscopy (NIRS), and some problems which need to be solved or investigated further are also discussed.
Abstract: Near infrared reflectance spectroscopy (NIRS) is a non-destructive and rapid technique applied increasingly for food quality evaluation in recent years. It provides us more information to research the quality of food products. This review intends to give an overview of the type of information that can be obtained based on some developed theory and food research of NIRS. It includes the principle of NIRS technique, the specific techniques with chemometrics for data pre-processing methods, qualitative and quantitative analysis and model transfer, and the wide applications of NIRS in food science. In addition, the promise of NIRS technique for food quality evaluation is demonstrated, and some problems which need to be solved or investigated further are also discussed.
812 citations
References
More filters
••
TL;DR: In this paper, the statistical technique of principal components is used to analyze two sets of near-infrared spectra, wheat flour samples for which % moisture and % protein values are included, and milled barley samples for whose hot water extract values were included.
Abstract: The statistical technique of principal components is used to analyze two sets of near-infrared spectra, wheat flour samples for which % moisture and % protein values are included, and milled barley samples for which hot water extract values are included. The methodology and interpretation of this technique are described within the context of NIR data, and its advantages both in providing insight into the variation of the spectra, and as a method of avoiding the problems caused by highly correlated reflectance energy values in the derivation of calibration equations, are highlighted. In each set of samples the first principal component accounts for the vast majority of the variation. These components also have an almost identical shape, which is interpreted as reflecting particle size. The second wheat component and the third barley component are also almost identical, with a shape very similar to that of the spectrum of water. Both fourth components share peaks at points in the spectrum which are used by fixed-filter instruments to measure protein in cereals.
266 citations
••
TL;DR: In this paper, a factorially designed experiment was carried out using mixtures of four pure chemicals to assess the effectiveness of several statistical techniques to detect the known structure of sample spectra.
Abstract: To assess the effectiveness of several statistical techniques to detect the known structure of sample spectra, a factorially designed experiment was carried out using mixtures of four pure chemicals. Analyses of the variation between spectra as expressed by correlation graphs and principal components are shown to be powerful techniques for relating the spectra of constituents to those of samples. A method of interpreting correlation graphs is proposed that identifies the existence of dominant effects such as particle size variation. For these samples, a standardisation algorithm was shown to reduce interference effects due to particle size and to allow easier interpretation of both correlation graphs and principal components.
30 citations