From the Publisher:
This is the first comprehensive treatment of feed-forward neural networks from the perspective of statistical pattern recognition. After introducing the basic concepts, the book examines techniques for modelling probability density functions and the properties and merits of the multi-layer perceptron and radial basis function network models. Also covered are various forms of error functions, principal algorithms for error function minimalization, learning and generalization in neural networks, and Bayesian techniques and their applications. Designed as a text, with over 100 exercises, this fully up-to-date work will benefit anyone involved in the fields of neural computation and pattern recognition.

/pdf/neural-networks-for-pattern-recognition-34puigiau1.pdf

Neural networks for pattern recognition

This paper describes a simple method of calculating a heteroskedasticity and autocorrelation consistent covariance matrix that is positive semi-definite by construction. It also establishes consistency of the estimated covariance matrix under fairly general conditions.

/pdf/a-simple-positive-semi-definite-heteroskedasticity-and-1h9nwsy8i5.pdf

A simple, positive semi-definite, heteroskedasticity and autocorrelation consistent covariance matrix

Structural equation modeling (SEM) has become a quasi-standard in marketing and management research when it comes to analyzing the cause-effect relations between latent constructs. For most researchers, SEM is equivalent to carrying out covariance-based SEM (CB-SEM). While marketing researchers have a basic understanding of CB-SEM, most of them are only barely familiar with the other useful approach to SEM-partial least squares SEM (PLS-SEM). The current paper reviews PLS-SEM and its algorithm, and provides an overview of when it can be most appropriately applied, indicating its potential and limitations for future research. The authors conclude that PLS-SEM path modeling, if appropriately applied, is indeed a "silver bullet" for estimating causal models in many theoretical models and empirical data situations.

PLS-SEM: Indeed a Silver Bullet

Generalized Additive Models.

Donoho and Johnstone (1994) proposed a method for reconstructing an unknown function f on [0,1] from noisy data d/sub i/=f(t/sub i/)+/spl sigma/z/sub i/, i=0, ..., n-1,t/sub i/=i/n, where the z/sub i/ are independent and identically distributed standard Gaussian random variables. The reconstruction f/spl circ/*/sub n/ is defined in the wavelet domain by translating all the empirical wavelet coefficients of d toward 0 by an amount /spl sigma//spl middot//spl radic/(2log (n)/n). The authors prove two results about this type of estimator. [Smooth]: with high probability f/spl circ/*/sub n/ is at least as smooth as f, in any of a wide variety of smoothness measures. [Adapt]: the estimator comes nearly as close in mean square to f as any measurable estimator can come, uniformly over balls in each of two broad scales of smoothness classes. These two properties are unprecedented in several ways. The present proof of these results develops new facts about abstract statistical inference and its connection with an optimal recovery model. >

De-noising by soft-thresholding

A fast and memory-saving PLS regression algorithm for matrices with large numbers of objects is presented. It is called the kernel algorithm for PLS. Long (meaning having many objects, N) matrices X (N × K) and Y (N × M) are condensed into a small (K × K) square ‘kernel’ matrix XTYYTX of size equal to the number of X-variables. Using this kernel matrix XTYYTX together with the small covariance matrices XTX (K × K), XTY (K × M) and YTY (M × M), it is possible to estimate all necessary parameters for a complete PLS regression solution with some statistical diagnostics. The new developments are presented in equation form. A comparison of consumed floating point operations is given for the kernel and the classical PLS algorithm. As appendices, a condensed matrix algebra version of the kernel algorithm is given together with the MATLAB code.

The kernel algorithm for PLS

The O2-PLS method is derived from the basic partial least squares projections to latent structures (PLS) prediction approach. The importance of the covariation matrix (YX) is pointed out in relation to both the prediction model and the structured noise in both X and Y. Structured noise in X (or Y) is defined as the systematic variation of X (or Y) not linearly correlated with Y (or X). Examples in spectroscopy include baseline, drift and scatter effects. If structured noise is present in X, the existing latent variable regression (LVR) methods, e.g. PLS, will have weakened score-loading correspondence beyond the first component. This negatively affects the interpretation of model parameters such as scores and loadings. The O2-PLS method models and predicts both X and Y and has an integral orthogonal signal correction (OSC) filter that separates the structured noise in X and Y from their joint X-Y covariation used in the prediction model. This leads to a minimal number of predictive components with full score-loading correspondence and also an opportunity to interpret the structured noise. In both a real and a simulated example, O2-PLS and PLS gave very similar predictions of Y. However, the interpretation of the prediction models was clearly improved with O2-PLS, because structured noise was present. In the NIR example, O2-PLS revealed a strong water peak and baseline offset in the structured noise components. In the simulated example the O2-PLS plot of observed versus predicted Y-scores (u vs U) showed good predictions. The corresponding loading vectors provided good interpretation of the covarying analytes in X and Y.

O2‐PLS, a two‐block (X–Y) latent variable regression (LVR) method with an integral OSC filter

Statistical Validation of QSAR Results

A fast PLS regression algorithm dealing with large data matrices with many variables (K) and fewer objects (N) is presented For such data matrices the classical algorithm is computer-intensive and memory-demanding. Recently, Lindgren et al. (J. Chemometrics, 7, 45–49 (1993)) developed a quick and efficient kernel algorithm for the case with many objects and few variables. The present paper is focused on the opposite case, i.e. many variables and fewer objects. A kernel algorithm is presented based on eigenvectors to the ‘kernel’ matrix XXTYYT, which is a square, non-symmetric matrix of size N × N, where N is the number of objects. Using the kernel matrix and the association matrices XXT (N × N) and YYT (N × N), it is possible to calculate all score and loading vectors and hence conduct a complete PLS regression including diagnostics such as R2. This is done without returning to the original data matrices X and Y. The algorithm is presented in equation form, with proofs of some new properties and as MATLAB code.

Svante Wold

Papers

The kernel algorithm for PLS

O2‐PLS, a two‐block (X–Y) latent variable regression (LVR) method with an integral OSC filter

Statistical Validation of QSAR Results

A PLS kernel algorithm for data sets with many variables and fewer objects. Part 1: Theory and algorithm

An evaluation of orthogonal signal correction applied to calibration transfer of near infrared spectra