This paper proposes a novel approach to tensor completion, which recovers missing entries of data represented by tensors. The approach is based on the tensor train (TT) rank, which is able to capture hidden information from tensors thanks to its definition from a well-balanced matricization scheme. Accordingly, new optimization formulations for tensor completion are proposed as well as two new algorithms for their solution. The first one called simple low-rank tensor completion via TT (SiLRTC-TT) is intimately related to minimizing a nuclear norm based on TT rank. The second one is from a multilinear matrix factorization model to approximate the TT rank of a tensor, and is called tensor completion by parallel matrix factorization via TT (TMac-TT). A tensor augmentation scheme of transforming a low-order tensor to higher orders is also proposed to enhance the effectiveness of SiLRTC-TT and TMac-TT. Simulation results for color image and video recovery show the clear advantage of our method over all other methods.

/pdf/efficient-tensor-completion-for-color-image-and-video-1bp9yvh2g7.pdf

Efficient Tensor Completion for Color Image and Video Recovery: Low-Rank Tensor Train

Tensor networks have in recent years emerged as the powerful tools for solving the large-scale optimization problems. One of the most popular tensor network is tensor train (TT) decomposition that acts as the building blocks for the complicated tensor networks. However, the TT decomposition highly depends on permutations of tensor dimensions, due to its strictly sequential multilinear products over latent cores, which leads to difficulties in finding the optimal TT representation. In this paper, we introduce a fundamental tensor decomposition model to represent a large dimensional tensor by a circular multilinear products over a sequence of low dimensional cores, which can be graphically interpreted as a cyclic interconnection of 3rd-order tensors, and thus termed as tensor ring (TR) decomposition. The key advantage of TR model is the circular dimensional permutation invariance which is gained by employing the trace operation and treating the latent cores equivalently. TR model can be viewed as a linear combination of TT decompositions, thus obtaining the powerful and generalized representation abilities. For optimization of latent cores, we present four different algorithms based on the sequential SVDs, ALS scheme, and block-wise ALS techniques. Furthermore, the mathematical properties of TR model are investigated, which shows that the basic multilinear algebra can be performed efficiently by using TR representaions and the classical tensor decompositions can be conveniently transformed into the TR representation. Finally, the experiments on both synthetic signals and real-world datasets were conducted to evaluate the performance of different algorithms.

Tensor Ring Decomposition

Part 2 of this monograph builds on the introduction to tensor networks and their operations presented in Part 1. It focuses on tensor network models for super-compressed higher-order representation of data/parameters and related cost functions, while providing an outline of their applications in machine learning and data analytics. A particular emphasis is on the tensor train (TT) and Hierarchical Tucker (HT) decompositions, and their physically meaningful interpretations which reflect the scalability of the tensor network approach. Through a graphical approach, we also elucidate how, by virtue of the underlying low-rank tensor approximations and sophisticated contractions of core tensors, tensor networks have the ability to perform distributed computations on otherwise prohibitively large volumes of data/parameters, thereby alleviating or even eliminating the curse of dimensionality. The usefulness of this concept is illustrated over a number of applied areas, including generalized regression and classification (support tensor machines, canonical correlation analysis, higher order partial least squares), generalized eigenvalue decomposition, Riemannian optimization, and in the optimization of deep neural networks. Part 1 and Part 2 of this work can be used either as stand-alone separate texts, or indeed as a conjoint comprehensive review of the exciting field of low-rank tensor networks and tensor decompositions.

Tensor Networks for Dimensionality Reduction and Large-Scale Optimizations. Part 2 Applications and Future Perspectives

Tensor completion is a problem of filling the missing or unobserved entries of partially observed tensors. Due to the multidimensional character of tensors in describing complex datasets, tensor completion algorithms and their applications have received wide attention and achievement in areas like data mining, computer vision, signal processing, and neuroscience. In this survey, we provide a modern overview of recent advances in tensor completion algorithms from the perspective of big data analytics characterized by diverse variety, large volume, and high velocity. We characterize these advances from the following four perspectives: general tensor completion algorithms, tensor completion with auxiliary information (variety), scalable tensor completion algorithms (volume), and dynamic tensor completion algorithms (velocity). Further, we identify several tensor completion applications on real-world data-driven problems and present some common experimental frameworks popularized in the literature along with several available software repositories. Our goal is to summarize these popular methods and introduce them to researchers and practitioners for promoting future research and applications. We conclude with a discussion of key challenges and promising research directions in this community for future exploration.

https://dl.acm.org/doi/pdf/10.1145/3278607

Tensor Completion Algorithms in Big Data Analytics

Using the matrix product state (MPS) representation of the recently proposed tensor ring (TR) decompositions, in this paper we propose a TR completion algorithm, which is an alternating minimization algorithm that alternates over the factors in the MPS representation. This development is motivated in part by the success of matrix completion algorithms that alternate over the (low-rank) factors. We propose a novel initialization method and analyze the computational complexity of the TR completion algorithm. The numerical comparison between the TR completion algorithm and the existing algorithms that employ a low rank tensor train (TT) approximation for data completion shows that our method outperforms the existing ones for a variety of real computer vision settings, and thus demonstrates the improved expressive power of tensor ring as compared to tensor train.

Efficient Low Rank Tensor Ring Completion

We consider the problem of fitting a low rank tensor $A\in\mathbb{R}^{{\mathcal I}}$, ${\mathcal I} = \{1,\ldots,n\}^{d}$, to a given set of data points $\{M_i\in\mathbb{R} \mid i\in P\}$, $P\subset{\mathcal I}$. The low rank format under consideration is the hierarchical or tensor train or matrix product states format. It is characterized by rank bounds $r$ on certain matricizations of the tensor. The number of degrees of freedom is in ${\cal O}(r^2dn)$. For a fixed rank and mode size $n$ we observe that it is possible to reconstruct random (but rank structured) tensors as well as certain discretized multivariate (but rank structured) functions from a number of samples that is in ${\cal O}(\log N)$ for a tensor having $N=n^d$ entries. We compare an alternating least squares (ALS) fit to an overrelaxation scheme inspired by the LMaFit method for matrix completion. Both approaches aim at finding a tensor $A$ that fulfills the first order optimality conditions by a nonlinear Gauss--Seidel-type solver that c...

Variants of Alternating Least Squares Tensor Completion in the Tensor Train Format

Low rank tensor completion is a highly ill-posed inverse problem, particularly when the data model is not accurate, and some sort of regularization is required in order to solve it. In this article we focus on the calibration of the data model. For alternating optimization, we observe that existing rank adaption methods do not enable a continuous transition between manifolds of different ranks. We denote this characteristic as $\textit{instability (under truncation)}$. As a consequence of this property, arbitrarily small changes in the iterate can have arbitrarily large influence on the further reconstruction. We therefore introduce a singular value based regularization to the standard alternating least squares (ALS), which is motivated by averaging in microsteps. We prove its $\textit{stability}$ and derive a natural semi-implicit rank adaption strategy. We further prove that the standard ALS microsteps for completion problems are only stable on manifolds of fixed ranks, and only around points that have what we define as $\textit{internal tensor restricted isometry property, iTRIP}$. In conclusion, numerical experiments are provided that show improvements of the reconstruction quality up to orders of magnitude in the new Stable ALS Approximation (SALSA) compared to standard ALS and the well known Riemannian optimization RTTC.

Stable ALS Approximation in the TT-Format for Rank-Adaptive Tensor Completion

Low rank tensor completion is a highly ill-posed inverse problem, particularly when the data model is not accurate, and some sort of regularization is required in order to solve it. In this article we focus on the calibration of the data model. For alternating optimization, we observe that existing rank adaption methods do not enable a continuous transition between manifolds of different ranks. We denote this characteristic as instability (under truncation). As a consequence of this property, arbitrarily small changes in the iterate can have arbitrarily large influence on the further reconstruction. We therefore introduce a singular value based regularization to the standard alternating least squares (ALS), which is motivated by averaging in microsteps. We prove its stability and derive a natural semi-implicit rank adaption strategy. We further prove that the standard ALS microsteps for completion problems are only stable on manifolds of fixed ranks, and only around points that have what we define as internal tensor restricted isometry property, iTRIP. In conclusion, numerical experiments are provided that show improvements of the reconstruction quality up to orders of magnitude in the new Stable ALS Approximation (SALSA) compared to standard ALS and the well known Riemannian optimization RTTC.

Stable ALS approximation in the TT-format for rank-adaptive tensor completion

We consider the problem of fitting a low rank tensor $A\in\mathbb{R}^{\mathcal I}$, ${\mathcal I} = \{1,\ldots,n\}^{d}$, to a given set of data points $\{M_i\in\mathbb{R}\mid i\in P\}$, $P\subset{\mathcal I}$. 
The low rank format under consideration is the hierarchical or TT or MPS format. It is characterized by rank bounds $r$ on certain matricizations of the tensor. The number of degrees of freedom is in ${\cal O}(r^2dn)$. 
For a fixed rank and mode size $n$ we observe that it is possible to reconstruct random (but rank structured) tensors as well as certain discretized multivariate (but rank structured) functions from a number of samples that is in ${\cal O}(\log N)$ for a tensor having $N=n^d$ entries. 
We compare an alternating least squares fit (ALS) to an overrelaxation scheme inspired by the LMaFit method for matrix completion. 
Both approaches aim at finding a tensor $A$ that fulfils the first order optimality conditions by a nonlinear Gauss-Seidel type solver that consists of an alternating fit cycling through the directions $\mu=1,\ldots,d$. 
The least squares fit is of complexity ${\cal O}(r^4d\#P)$ per step, whereas each step of ADF is in ${\cal O}(r^2d\#P)$, albeit with a slightly higher number of necessary steps. 
In the numerical experiments we observe robustness of the completion algorithm with respect to noise and good reconstruction capability. 
Our tests provide evidence that the algorithm is suitable in higher dimension ($>$10) as well as for moderate ranks. 
Keywords: MPS, Tensor Completion, Tensor Train, TT, Hierarchical Tucker, HT, ALS.

Alternating Least Squares Tensor Completion in The TT-Format

Die vorliegende Erfindung betrifft ein Verfahren und eine Vorrichtung zur Uberwachung und Regelung der Bearbeitungsbahn bei einem Laser-Fugeprozess. The present invention relates to a method and apparatus for monitoring and control of the machining path with a laser joining process. Bei dem Verfahren wird ein Bearbeitungslaserstrahl uber eine Bearbeitungsoptik (3) auf eine Oberflache (16) zu fugender Werkstucke gerichtet und auf der Bearbeitungsbahn entlang einer Stosfuge der Werkstucke gefuhrt. In the method, a machining laser beam over a processing optics (3) onto a surface (16) is directed to mating workpieces and out on the machining path along a butt joint of the workpieces. Ein Bereich der Oberflache (16) wird koaxial zum Bearbeitungslaserstrahl mit Beleuchtungsstrahlung homogen beleuchtet und von dem Bereich reflektierte Beleuchtungsstrahlung durch die Bearbeitungsoptik (3) hindurch ortsaufgelost mit einem Bildsensor (20) erfasst und ausgewertet. A region of the surface (16) is coaxial with the machining laser beam homogeneously illuminated with illumination radiation and of the area reflected illumination radiation through the processing optics (3) through a spatially resolved with an image sensor (20) is detected and evaluated. Aus den vom Bildsensor (20) erhaltenen Bildern wird mittels texturbasierter Bildverarbeitung eine Position der Stosfuge und eine Position der vom Bearbeitungslaserstrahl induzierten Kapillare bestimmt. From the off the image sensor (20) images obtained a position of the butt joint and a position of the processing laser beam induced by the capillary is determined by texture-based image processing. Die Bearbeitungsbahn des Bearbeitungslaserstrahls wird dann in Abhangigkeit von den jeweils erfassten Positionen geregelt. The machining path of the machining laser beam is then controlled as a function of the positions respectively detected. Das Verfahren und die Vorrichtung ermoglichen eine robuste Detektion der Stosfuge und gleichzeitige Prozessuberwachung bei einem kompakten Aufbau des Bearbeitungskopfes. The method and apparatus enable a robust detection of the butt joint and simultaneous process monitoring in a compact structure of the machining head.

Sebastian Krämer

Papers

Variants of Alternating Least Squares Tensor Completion in the Tensor Train Format

Stable ALS Approximation in the TT-Format for Rank-Adaptive Tensor Completion

Stable ALS approximation in the TT-format for rank-adaptive tensor completion

Alternating Least Squares Tensor Completion in The TT-Format

Method and device for monitoring and controlling the processing path of a laser joining process