Recent work by Kilmer and Martin [Linear Algebra Appl., 435 (2011), pp. 641--658] and Braman [Linear Algebra Appl., 433 (2010), pp. 1241--1253] provides a setting in which the familiar tools of linear algebra can be extended to better understand third-order tensors. Continuing along this vein, this paper investigates further implications including (1) a bilinear operator on the matrices which is nearly an inner product and which leads to definitions for length of matrices, angle between two matrices, and orthogonality of matrices, and (2) the use of t-linear combinations to characterize the range and kernel of a mapping defined by a third-order tensor and the t-product and the quantification of the dimensions of those sets. These theoretical results lead to the study of orthogonal projections as well as an effective Gram--Schmidt process for producing an orthogonal basis of matrices. The theoretical framework also leads us to consider the notion of tensor polynomials and their relation to tensor eigentupl...

Third-Order Tensors as Operators on Matrices: A Theoretical and Computational Framework with Applications in Imaging

Factorization strategies for third-order tensors

In this paper we propose novel methods for completion (from limited samples) and de-noising of multilinear (tensor) data and as an application consider 3-D and 4- D (color) video data completion and de-noising. We exploit the recently proposed tensor-Singular Value Decomposition (t-SVD)[11]. Based on t-SVD, the notion of multilinear rank and a related tensor nuclear norm was proposed in [11] to characterize informational and structural complexity of multilinear data. We first show that videos with linear camera motion can be represented more efficiently using t-SVD compared to the approaches based on vectorizing or flattening of the tensors. Since efficiency in representation implies efficiency in recovery, we outline a tensor nuclear norm penalized algorithm for video completion from missing entries. Application of the proposed algorithm for video recovery from missing entries is shown to yield a superior performance over existing methods. We also consider the problem of tensor robust Principal Component Analysis (PCA) for de-noising 3-D video data from sparse random corruptions. We show superior performance of our method compared to the matrix robust PCA adapted to this setting as proposed in [4].

https://www.cv-foundation.org/openaccess/content_cvpr_2014/papers/Zhang_Novel_Methods_for_2014_CVPR_paper.pdf

Novel Methods for Multilinear Data Completion and De-noising Based on Tensor-SVD

In this paper, we focus on the problem of completion of multidimensional arrays (also referred to as tensors), in particular three-dimensional (3-D) arrays, from limited sampling. Our approach is based on a recently proposed tensor algebraic framework where 3-D tensors are treated as linear operators over the set of 2-D tensors. In this framework, one can obtain a factorization for 3-D data, referred to as the tensor singular value decomposition (t-SVD), which is similar to the SVD for matrices. t-SVD results in a notion of rank referred to as the tubal-rank. Using this approach we consider the problem of sampling and recovery of 3-D arrays with low tubal-rank. We show that by solving a convex optimization problem, which minimizes a convex surrogate to the tubal-rank, one can guarantee exact recovery with high probability as long as number of samples is of the order $O(rnk \log (nk))$ , given a tensor of size $n\times n\times k$ with tubal-rank $r$ . The conditions under which this result holds are similar to the incoherence conditions for low-rank matrix completion under random sampling. The difference is that we define incoherence under the algebraic setup of t-SVD, which is different from the standard matrix incoherence conditions. We also compare the numerical performance of the proposed algorithm with some state-of-the-art approaches on real-world datasets.

https://ieeexplore.ieee.org/ielaam/78/7820239/7782758-aam.pdf

Exact Tensor Completion Using t-SVD

This paper studies the Tensor Robust Principal Component (TRPCA) problem which extends the known Robust PCA [4] to the tensor case. Our model is based on a new tensor Singular Value Decomposition (t-SVD) [14] and its induced tensor tubal rank and tensor nuclear norm. Consider that we have a 3-way tensor X e Rn1×n2×n3 such that X = L0 + S0, where L0 has low tubal rank and S0 is sparse. Is that possible to recover both components? In this work, we prove that under certain suitable assumptions, we can recover both the low-rank and the sparse components exactly by simply solving a convex program whose objective is a weighted combination of the tensor nuclear norm and the l1-norm, i.e., min L, E ||L||* + λ||e||1, s.t. X = L + e, where λ = 1/√max(n1, n2)n3. Interestingly, TRPCA involves RPCA as a special case when n3 = 1 and thus it is a simple and elegant tensor extension of RPCA. Also numerical experiments verify our theory and the application for the image denoising demonstrates the effectiveness of our method.

/pdf/tensor-robust-principal-component-analysis-exact-recovery-of-3zcap3joqy.pdf

Tensor Robust Principal Component Analysis: Exact Recovery of Corrupted Low-Rank Tensors via Convex Optimization

As computing power increases, many more problems in engineering and data analysis involve computation with tensors, or multi-way data arrays. Most applications involve computing a decomposition of a tensor into a linear combination of rank-1 tensors. Ideally, the decomposition involves a minimal number of terms, i.e. computation of the rank of the tensor. Tensor rank is not a straight-forward extension of matrix rank. A constructive proof based on an eigenvalue criterion is provided that shows when a 2 × 2 × 2 tensor over ℝ is rank-3 and when it is rank-2. The results are extended to show that n × n × 2 tensors over ℝ have maximum possible rank n + k where k is the number of complex conjugate eigenvalue pairs of the matrices forming the two faces of the tensor cube.

The rank of a 2 × 2 × 2 tensor

Are there analogues to the SVD, LU, QR, and other matrix decompositions for tensors (i.e., higher-order or multiway arrays)? What exactly do we mean by “analogue,” anyway? If such decompositions exist, are there efficient ways to compute them? These are some of the questions asked, and partially answered, at the Workshop on Tensor Decompositions, held July 19–23, 2004, at the American Institute of Mathematics in Palo Alto, California. Gene Golub, Tammy Kolda, James Nagy, and Charles Van Loan were the organizers. About 35 people—computer scientists, mathematicians, and a broad range of scholars who use tensor decompositions in their research—had come from eleven countries to participate in the weeklong workshop. Large group discussions and smaller break-out sessions were interleaved with invited talks, making for lively exchanges and a creative learning environment. Carla Martin, a graduate student at Cornell University, opened the week with an overview of multilinear algebra and tensor decompositions, beginning with the definition of a pth-order tensor A as a multiway array with p indices. A third-order tensor, for example, is written

Carla D. Martin

Papers

Factorization strategies for third-order tensors

The rank of a 2 × 2 × 2 tensor

Decomposing a Tensor