This survey provides an overview of higher-order tensor decompositions, their applications, and available software. A tensor is a multidimensional or $N$-way array. Decompositions of higher-order tensors (i.e., $N$-way arrays with $N \geq 3$) have applications in psycho-metrics, chemometrics, signal processing, numerical linear algebra, computer vision, numerical analysis, data mining, neuroscience, graph analysis, and elsewhere. Two particular tensor decompositions can be considered to be higher-order extensions of the matrix singular value decomposition: CANDECOMP/PARAFAC (CP) decomposes a tensor as a sum of rank-one tensors, and the Tucker decomposition is a higher-order form of principal component analysis. There are many other tensor decompositions, including INDSCAL, PARAFAC2, CANDELINC, DEDICOM, and PARATUCK2 as well as nonnegative variants of all of the above. The N-way Toolbox, Tensor Toolbox, and Multilinear Engine are examples of software packages for working with tensors.

/pdf/tensor-decompositions-and-applications-46vl2ih5gn.pdf

Tensor Decompositions and Applications

The Theory of Matrices. By F R. Gantmacher. Two volumes, pp. 374 and 276. 1959. (Translated from the Russian by K. A. Hirsch; Chelsea Publishing Company, New York)

Tensors or multiway arrays are functions of three or more indices $(i,j,k,\ldots)$ —similar to matrices (two-way arrays), which are functions of two indices $(r,c)$ for (row, column). Tensors have a rich history, stretching over almost a century, and touching upon numerous disciplines; but they have only recently become ubiquitous in signal and data analytics at the confluence of signal processing, statistics, data mining, and machine learning. This overview article aims to provide a good starting point for researchers and practitioners interested in learning about and working with tensors. As such, it focuses on fundamentals and motivation (using various application examples), aiming to strike an appropriate balance of breadth and depth that will enable someone having taken first graduate courses in matrix algebra and probability to get started doing research and/or developing tensor algorithms and software. Some background in applied optimization is useful but not strictly required. The material covered includes tensor rank and rank decomposition; basic tensor factorization models and their relationships and properties (including fairly good coverage of identifiability); broad coverage of algorithms ranging from alternating optimization to stochastic gradient; statistical performance analysis; and applications ranging from source separation to collaborative filtering, mixture and topic modeling, classification, and multilinear subspace learning.

Tensor Decomposition for Signal Processing and Machine Learning

The widespread use of multisensor technology and the emergence of big data sets have highlighted the limitations of standard flat-view matrix models and the necessity to move toward more versatile data analysis tools. We show that higher-order tensors (i.e., multiway arrays) enable such a fundamental paradigm shift toward models that are essentially polynomial, the uniqueness of which, unlike the matrix methods, is guaranteed under very mild and natural conditions. Benefiting from the power of multilinear algebra as their mathematical backbone, data analysis techniques using tensor decompositions are shown to have great flexibility in the choice of constraints which match data properties and extract more general latent components in the data than matrix-based methods.

/pdf/tensor-decompositions-for-signal-processing-applications-2cvd1sgtij.pdf

Tensor Decompositions for Signal Processing Applications: From two-way to multiway component analysis

We prove that multilinear (tensor) analogues of many efficiently computable problems in numerical linear algebra are NP-hard. Our list includes: determining the feasibility of a system of bilinear equations, deciding whether a 3-tensor possesses a given eigenvalue, singular value, or spectral norm; approximating an eigenvalue, eigenvector, singular vector, or the spectral norm; and determining the rank or best rank-1 approximation of a 3-tensor. Furthermore, we show that restricting these problems to symmetric tensors does not alleviate their NP-hardness. We also explain how deciding nonnegative definiteness of a symmetric 4-tensor is NP-hard and how computing the combinatorial hyperdeterminant is NP-, #P-, and VNP-hard.

/pdf/most-tensor-problems-are-np-hard-cqgnykhb6t.pdf

Most Tensor Problems Are NP-Hard

On secant varieties of compact Hermitian symmetric spaces

The sphericality of tangential varieties of rational homogeneous varieties is determined. The homogeneous coordinate rings and rings of covariants of the tangential varieties of homogenously embedded compact Hermitian symmetric spaces (CHSS) are determined. Bounds on the degrees of generators of the ideals of tangential varieties of CHSS are given, and more explicit information is obtained about the ideals in certain cases.

On Tangential varieties of rational homogeneous varieties

We characterize complete intersections in terms of local differential geometry. Let X ⊂ CPn+a be a variety. We first localize the problem; we give a criterion for X to be a complete intersection that is testable at any smooth point of X. We rephrase the criterion in the language of projective differential geometry and derive a sufficient condition for X to be a complete intersection that is computable at a general point x ∈ X. The sufficient condition has a geometric interpretation in terms of restrictions on the spaces of osculating hypersurfaces at x. When this sufficient condition holds, we are able to define systems of partial differential equations that generalize the classical Monge equation that characterizes conic curves in CP2. Using our sufficent condition, we show that if the ideal of X is generated by quadrics and a < n−(b+1)+3 3 , where b =dimXsing, then X is a complete intersection. §0. Introduction Local and global geometry Projective differential geometry has been used to study the local geometry of subvarieties of projective space by various authors (e.g. [C], [F], [GH], [JM], [T]). However, there are few examples where global conclusions are drawn from the local picture. One (global) fact about projective varieties that has been encoded into the infinitesimal geometry is the following: if there is a line on a variety X ⊂ CP along which the embedded tangent space is constant, then X must be singular. Griffiths and Harris realized this fact had implications for the projective second fundamental form (that its singular locus must be empty at general points) which 1991 Mathematics Subject Classification. primary 14e335, secondary 533a20.

Differential-geometric characterizations of complete intersections

We analyze tensors in the tensor product of three m-dimensional vector spaces satisfying Strassen's equations for border rank m Results include: two purely geometric characterizations of the Coppersmith-Winograd tensor, a reduction to the study of symmetric tensors under a mild genericity hypothesis, and numerous additional equations and examples This study is closely connected to the study of the variety of m-dimensional abelian subspaces of the space of endomorphisms of an m-dimensional vector space, and the subvariety consisting of the Zariski closure of the variety of maximal tori, called the variety of reductions

Abelian Tensors

Let X be a nonsingular simply connected projective variety of dimension m, E a rank n vector bundle on X, and L a line bundle on X. Suppose that $S^2(E^{*}) \otimes L$ is an ample vector bundle and that there is a constant even rank $r \ge 2$ symmetric bundle map $E \to E^{*} \otimes L$. We prove that $m \le n-r$. We use this result to solve the constant rank problem for symmetric matrices, proving that the maximal dimension of a linear subspace of the space of $m\times m$ symmetric matrices such that each nonzero element has even rank $r \ge 2$ is $m-r+1$. We explain how this result relates to the study of dual varieties in projective geometry and give some applications and examples.

https://arxiv.org/pdf/alg-geom/9611025.pdf

Joseph M. Landsberg

Papers

On secant varieties of compact Hermitian symmetric spaces

On Tangential varieties of rational homogeneous varieties

Differential-geometric characterizations of complete intersections

Abelian Tensors

On symmetric degeneracy loci, spaces of symmetric matrices of constant rank and dual varieties