Principal component analysis is one of the most important and powerful methods in chemometrics as well as in a wealth of other areas. This paper provides a description of how to understand, use, and interpret principal component analysis. The paper focuses on the use of principal component analysis in typical chemometric areas but the results are generally applicable.

/pdf/principal-component-analysis-1sba76ou47.pdf

Principal component analysis

Clustering: A neural network approach

Widrow (1971) proposed the least mean squares (LMS) algorithm, which has been extensively applied in adaptive signal processing and adaptive control. The LMS algorithm is based on the minimum mean squares error. On the basis of the total least mean squares error or the minimum Raleigh quotient, we propose the total least mean squares (TLMS) algorithm. The paper gives the statistical analysis for this algorithm, studies the global asymptotic convergence of this algorithm by an equivalent energy function, and evaluates the performances of this algorithm via computer simulations.

/pdf/total-least-mean-squares-algorithm-12x46x32yg.pdf

Total least mean squares algorithm

The minor component analysis (MCA) deals with the recovery of the eigenvector associated to the smallest eigenvalue of the autocorrelation matrix of the input data and is a very important tool for signal processing and data analysis. It is almost exclusively solved by linear neurons. This paper presents a linear neuron endowed with a novel learning law, called MCA EXINn and analyzes its features. The neural literature about MCA is very poor, in the sense that both a little theoretical basis is given (almost always focusing on the ODE asymptotic approximation) and only experiments on toy problems (at most four-dimensional problems) are presented, without any numerical analysis. This work addresses these problems and lays sound theoretical foundations for the neural MCA theory. In particular, it classifies the MCA neurons according to the Riemannian metric and justifies, from the analysis of the degeneracy of the error cost; the different behavior in approaching convergence. The cost landscape is studied and used as a basis for the analysis of the asymptotic behavior. All the phases of the dynamics of the MCA algorithms are investigated in detail and, together with the numerical analysis, lead to the identification of three possible kinds of divergence, here called sudden, dynamic, and numerical. The importance of the choice of low initial conditions is also explained. A lot of importance is given to the experimental part, where simulations on high-dimensional problems are,presented and analyzed. The orthogonal regression or total least squares (TLS) technique is also presented, together with a real-world application on the identification of the parameters of an electrical machine. It can be concluded that MCA EXIN is the best MCA neuron in terms of stability (no finite time divergence), speed, and accuracy.

The MCA EXIN neuron for the minor component analysis

Neural methods for antenna array signal processing: a review

In this paper, a new Hebbian-type learning algorithm for the total least-squares parameter estimation is presented. The algorithm is derived from the classical Hebbian rule. An asymptotic analysis is carried out to show that the algorithm allows the weight vector of a linear neuron unit to converge to the eigenvector associated with the smallest eigenvalue of the correlation matrix of the input signal. When the algorithm is applied to solve parameter estimation problems, the converged weights directly yield the total least-squares solution. Since the process of obtaining the estimate is optimal in the total least-squares sense, its noise rejection capability is superior to those of the least-squares-based algorithms. It is shown that the implementations of the proposed algorithm have the simplicity of those of the LMS algorithm. The applicability and performance of the algorithm are demonstrated through computer simulations of adaptive FIR and IIR parameter estimation problems. >

A constrained anti-Hebbian learning algorithm for total least-squares estimation with applications to adaptive FIR and IIR filtering

A new adaptive learning algorithm, called the constrained anti-Hebbian algorithm, is presented. This algorithm is optimal in the total least-squares sense, simple to use, and can be applied to on-line adaptive FIR and IIR filtering directly. Simulation results confirm that the proposed algorithm provides significantly better performance over the least mean-squares (LMS) and recursive least-squares (RLS) algorithms.

Learning algorithm for total least-squares adaptive signal processing

A constrained anti-Hebbian algorithm that is used for processing complex signals is presented. It is shown that the algorithm adaptively extracts the eigenvector associated with the smallest eigenvalue of the correlation matrix of the input signal. The operation of the algorithm is simple, similar to that of the LMS (least mean square) algorithm, and it can be applied to an adaptive prediction-error filter directly, giving an estimate of the parameters that is optimal in the total least-squares sense. Simulation results on estimating the frequencies of sinusoids corrupted by white noise are presented. >

A modified Hebbian learning rule for total least-squares estimation with complex-valued arguments

Least-squares (LS) estimation with a standard feedback neural network (SFBNN) which is based on an electrical model is investigated. In the energy function of a SFBNN, a non-quadratic term is included which is often neglected while solving an optimization problem. It is shown that the non-quadratic term affects the solution of a continuous optimization problem. Properties of the non-quadratic term and the relation between the estimation error and several parameters of the SFBNN are discussed. A technique, called extended space iterative search (ESIS), is introduced to reduce the estimation error. Simulation results are presented to confirm the analysis result and the effectiveness of the proposed technique. >

Neural LS estimator with a non-quadratic energy function

The application of self-organizing neural networks in processing nonlinear dynamic signals directly is investigated. The processing of a signal uses a model-based approach. The signal generating system is modeled by decomposing it into simpler subsystems and each subsystems is associated with a neuron on a single-layer network. Each subsystem is implemented using a temporally local linear combiner. The network is trained with a self-organizing procedure and the parameters of the linear combiners are updated by using the Widrow-Hoff adaptive rule. A competitive rule which takes into consideration the temporal dependence among the signal samples is presented. Simulation results are presented to illustrate the method. >

K. Gao

Papers

A constrained anti-Hebbian learning algorithm for total least-squares estimation with applications to adaptive FIR and IIR filtering

Learning algorithm for total least-squares adaptive signal processing

A modified Hebbian learning rule for total least-squares estimation with complex-valued arguments

Neural LS estimator with a non-quadratic energy function

Nonlinear signal processing with self-organizing neural networks