This survey provides an overview of higher-order tensor decompositions, their applications, and available software. A tensor is a multidimensional or $N$-way array. Decompositions of higher-order tensors (i.e., $N$-way arrays with $N \geq 3$) have applications in psycho-metrics, chemometrics, signal processing, numerical linear algebra, computer vision, numerical analysis, data mining, neuroscience, graph analysis, and elsewhere. Two particular tensor decompositions can be considered to be higher-order extensions of the matrix singular value decomposition: CANDECOMP/PARAFAC (CP) decomposes a tensor as a sum of rank-one tensors, and the Tucker decomposition is a higher-order form of principal component analysis. There are many other tensor decompositions, including INDSCAL, PARAFAC2, CANDELINC, DEDICOM, and PARATUCK2 as well as nonnegative variants of all of the above. The N-way Toolbox, Tensor Toolbox, and Multilinear Engine are examples of software packages for working with tensors.

/pdf/tensor-decompositions-and-applications-46vl2ih5gn.pdf

Tensor Decompositions and Applications

The Theory of Matrices. By F R. Gantmacher. Two volumes, pp. 374 and 276. 1959. (Translated from the Russian by K. A. Hirsch; Chelsea Publishing Company, New York)

Tensors or multiway arrays are functions of three or more indices $(i,j,k,\ldots)$ —similar to matrices (two-way arrays), which are functions of two indices $(r,c)$ for (row, column). Tensors have a rich history, stretching over almost a century, and touching upon numerous disciplines; but they have only recently become ubiquitous in signal and data analytics at the confluence of signal processing, statistics, data mining, and machine learning. This overview article aims to provide a good starting point for researchers and practitioners interested in learning about and working with tensors. As such, it focuses on fundamentals and motivation (using various application examples), aiming to strike an appropriate balance of breadth and depth that will enable someone having taken first graduate courses in matrix algebra and probability to get started doing research and/or developing tensor algorithms and software. Some background in applied optimization is useful but not strictly required. The material covered includes tensor rank and rank decomposition; basic tensor factorization models and their relationships and properties (including fairly good coverage of identifiability); broad coverage of algorithms ranging from alternating optimization to stochastic gradient; statistical performance analysis; and applications ranging from source separation to collaborative filtering, mixture and topic modeling, classification, and multilinear subspace learning.

Tensor Decomposition for Signal Processing and Machine Learning

The widespread use of multisensor technology and the emergence of big data sets have highlighted the limitations of standard flat-view matrix models and the necessity to move toward more versatile data analysis tools. We show that higher-order tensors (i.e., multiway arrays) enable such a fundamental paradigm shift toward models that are essentially polynomial, the uniqueness of which, unlike the matrix methods, is guaranteed under very mild and natural conditions. Benefiting from the power of multilinear algebra as their mathematical backbone, data analysis techniques using tensor decompositions are shown to have great flexibility in the choice of constraints which match data properties and extract more general latent components in the data than matrix-based methods.

/pdf/tensor-decompositions-for-signal-processing-applications-2cvd1sgtij.pdf

Tensor Decompositions for Signal Processing Applications: From two-way to multiway component analysis

We prove that multilinear (tensor) analogues of many efficiently computable problems in numerical linear algebra are NP-hard. Our list includes: determining the feasibility of a system of bilinear equations, deciding whether a 3-tensor possesses a given eigenvalue, singular value, or spectral norm; approximating an eigenvalue, eigenvector, singular vector, or the spectral norm; and determining the rank or best rank-1 approximation of a 3-tensor. Furthermore, we show that restricting these problems to symmetric tensors does not alleviate their NP-hardness. We also explain how deciding nonnegative definiteness of a symmetric 4-tensor is NP-hard and how computing the combinatorial hyperdeterminant is NP-, #P-, and VNP-hard.

/pdf/most-tensor-problems-are-np-hard-cqgnykhb6t.pdf

Most Tensor Problems Are NP-Hard

We present a new proof of the classification of complex simple Lie algebras via the projective geometry of homogeneous varieties. Our proof proceeds by constructing homogeneous varieties using the ideals of the secant and tangential varieties of homogeneous varieties already constructed. 
Our algorithms make no reference to root systems. Our proofs use properties of root systems, but not their classification.

Classification of complex simple Lie algebras via projective geometry geometry

Upper bounds on projective rigidity of each homogeneously embedded homogeneous variety are determined; and a new, invariant characterization of the Fubini forms is given.

Fubini-Griffiths-Harris rigidity of homogeneous varieties

We determine defining equations for the set of concise tensors of minimal border rank in $${\mathbb {C}}^m{\mathord { \otimes } }{\mathbb {C}}^m{\mathord { \otimes } }{\mathbb {C}}^m$$ C m ⊗ C m ⊗ C m when $$m=5$$ m = 5 and the set of concise minimal border rank $$1_*$$ 1 ∗ -generic tensors when $$m=5,6$$ m = 5 , 6 . We solve the classical problem in algebraic complexity theory of classifying minimal border rank tensors in the special case $$m=5$$ m = 5 . Our proofs utilize two recent developments: the 111-equations defined by Buczyńska–Buczyński and results of Jelisiejew–Šivic on the variety of commuting matrices. We introduce a new algebraic invariant of a concise tensor, its 111-algebra, and exploit it to give a strengthening of Friedland’s normal form for 1-degenerate tensors satisfying Strassen’s equations. We use the 111-algebra to characterize wild minimal border rank tensors and classify them in $${\mathbb {C}}^5{\mathord { \otimes } }{\mathbb {C}}^5{\mathord { \otimes } }{\mathbb {C}}^5$$ C 5 ⊗ C 5 ⊗ C 5 .

Concise tensors of minimal border rank

Grenet's determinantal representation for the permanent is optimal among determinantal representations that are equivariant with respect to left multiplication by permutation and diagonal matrices (roughly half the symmetry group of the permanent). In particular, if any optimal determinantal representation of the permanent must be polynomially related to one with such symmetry, then Valiant's conjecture on permanent v. determinant is true.

/pdf/permanent-v-determinant-an-exponential-lower-bound-assuming-4j98kih3o8.pdf

Permanent v. Determinant: An Exponential Lower Bound Assuming Symmetry

In this note we discuss the geometry of matrix product states with periodic boundary conditions and provide three infinite sequences of examples where the quantum max-flow is strictly less than the quantum min-cut In the first we fix the underlying graph to be a 4-cycle and verify a prediction of Hastings that inequality occurs for infinitely many bond dimensions In the second we generalize this result to a 2d-cycle In the third we show that the 2d-cycle with periodic boundary conditions gives inequality for all d when all bond dimensions equal two, namely a gap of at least 2^{d-2} between the quantum max-flow and the quantum min-cut

Joseph M. Landsberg

Papers

Classification of complex simple Lie algebras via projective geometry geometry

Fubini-Griffiths-Harris rigidity of homogeneous varieties

Concise tensors of minimal border rank

Permanent v. Determinant: An Exponential Lower Bound Assuming Symmetry

Matrix product states and the quantum max-flow/min-cut conjectures