This survey provides an overview of higher-order tensor decompositions, their applications, and available software. A tensor is a multidimensional or $N$-way array. Decompositions of higher-order tensors (i.e., $N$-way arrays with $N \geq 3$) have applications in psycho-metrics, chemometrics, signal processing, numerical linear algebra, computer vision, numerical analysis, data mining, neuroscience, graph analysis, and elsewhere. Two particular tensor decompositions can be considered to be higher-order extensions of the matrix singular value decomposition: CANDECOMP/PARAFAC (CP) decomposes a tensor as a sum of rank-one tensors, and the Tucker decomposition is a higher-order form of principal component analysis. There are many other tensor decompositions, including INDSCAL, PARAFAC2, CANDELINC, DEDICOM, and PARATUCK2 as well as nonnegative variants of all of the above. The N-way Toolbox, Tensor Toolbox, and Multilinear Engine are examples of software packages for working with tensors.

/pdf/tensor-decompositions-and-applications-46vl2ih5gn.pdf

Tensor Decompositions and Applications

A Scaling Method for Priorities in Hierarchical Structures

This article is about a curious phenomenon. Suppose we have a data matrix, which is the superposition of a low-rank component and a sparse component. Can we recover each component individuallyq We prove that under some suitable assumptions, it is possible to recover both the low-rank and the sparse components exactly by solving a very convenient convex program called Principal Component Pursuit; among all feasible decompositions, simply minimize a weighted combination of the nuclear norm and of the e1 norm. This suggests the possibility of a principled approach to robust principal component analysis since our methodology and results assert that one can recover the principal components of a data matrix even though a positive fraction of its entries are arbitrarily corrupted. This extends to the situation where a fraction of the entries are missing as well. We discuss an algorithm for solving this optimization problem, and present applications in the area of video surveillance, where our methodology allows for the detection of objects in a cluttered background, and in the area of face recognition, where it offers a principled way of removing shadows and specularities in images of faces.

/pdf/robust-principal-component-analysis-59kf1xh72d.pdf

Robust principal component analysis

Principal component analysis PCA is a multivariate technique that analyzes a data table in which observations are described by several inter-correlated quantitative dependent variables. Its goal is to extract the important information from the table, to represent it as a set of new orthogonal variables called principal components, and to display the pattern of similarity of the observations and of the variables as points in maps. The quality of the PCA model can be evaluated using cross-validation techniques such as the bootstrap and the jackknife. PCA can be generalized as correspondence analysis CA in order to handle qualitative variables and as multiple factor analysis MFA in order to handle heterogeneous sets of variables. Mathematically, PCA depends upon the eigen-decomposition of positive semi-definite matrices and upon the singular value decomposition SVD of rectangular matrices. Copyright © 2010 John Wiley & Sons, Inc.

/pdf/principal-component-analysis-rvdwo0sk0j.pdf

Principal component analysis

Multivariate analyses are well known and widely used to identify and understand structures of ecological communities. The ade4 package for the R statistical environment proposes a great number of multivariate methods. Its implementation follows the tradition of the French school of "Analyse des Donnees" and is based on the use of the duality diagram. We present the theory of the duality diagram and discuss its implementation in ade4. Classes and main functions are presented. An example is given to illustrate the ade4 philosophy.

/pdf/the-ade4-package-implementing-the-duality-diagram-for-r095ij3wwz.pdf

The ade4 Package: Implementing the Duality Diagram for Ecologists

The mathematical problem of approximating one matrix by another of lower rank is closely related to the fundamental postulate of factor-theory. When formulated as a least-squares problem, the normal equations cannot be immediately written down, since the elements of the approximate matrix are not independent of one another. The solution of the problem is simplified by first expressing the matrices in a canonic form. It is found that the problem always has a solution which is usually unique. Several conclusions can be drawn from the form of this solution. A hypothetical interpretation of the canonic components of a score matrix is discussed.

The approximation of one matrix by another of lower rank

Necessary and sufficient conditions are given for a set of numbers to be the mutual distances of a set of real points in Euclidean space, and matrices are found whose ranks determine the dimension of the smallest Euclidean space containing such points. Methods are indicated for determining the configuration of these points, and for approximating to them by points in a space of lower dimensionality.

Discussion of a set of points in terms of their mutual distances

each two nonparallel elements of G cross each other. Obviously the conclusions of the theorem do not hold. The following example will show that the condition that no two elements of the collection G shall have a complementary domain in common is also necessary. In the cartesian plane let M be a circle of radius 1 and center at the origin, and iVa circle of radius 1 and center at the point (5, 5). Let d be a collection which contains each continuum which is the sum of M and a horizontal straight line interval of length 10 whose left-hand end point is on the circle M and which contains no point within M. Let G2 be a collection which contains each continuum which is the sum of N and a vertical straight line interval of length 10 whose upper end point is on the circle N and which contains no point within N. Let G = Gi+G2 . No element of G crosses any other element of G, but uncountably many have a complementary domain in common with some other element of the collection. However, it is evident that no countable subcollection of G covers the set of points each of which is common to two continua of the collection G. I t is not known whether or not the condition that each element of G shall separate some complementary domain of every other one can be omitted.

/pdf/a-principal-axis-transformation-for-non-hermitian-matrices-v9prrgd12u.pdf

A principal axis transformation for non-hermitian matrices

(1938). Matrix Approximation and Latent Roots. The American Mathematical Monthly: Vol. 45, No. 3, pp. 165-171.

Matrix Approximation and Latent Roots

Fisher's method of maximum likelihood is applied to the problem of estimation in factor analysis, as initiated by Lawley, and found to lead to a generalization of the Eckart matrix approximation problem. The solution of this in a special case is applied to show how test fallability enters into factor determination, it being noted that the method of communalities underestimates the number of factors.

Gale Young

Papers

The approximation of one matrix by another of lower rank

Discussion of a set of points in terms of their mutual distances

A principal axis transformation for non-hermitian matrices

Matrix Approximation and Latent Roots

Maximum likelihood estimation and factor analysis