This survey provides an overview of higher-order tensor decompositions, their applications, and available software. A tensor is a multidimensional or $N$-way array. Decompositions of higher-order tensors (i.e., $N$-way arrays with $N \geq 3$) have applications in psycho-metrics, chemometrics, signal processing, numerical linear algebra, computer vision, numerical analysis, data mining, neuroscience, graph analysis, and elsewhere. Two particular tensor decompositions can be considered to be higher-order extensions of the matrix singular value decomposition: CANDECOMP/PARAFAC (CP) decomposes a tensor as a sum of rank-one tensors, and the Tucker decomposition is a higher-order form of principal component analysis. There are many other tensor decompositions, including INDSCAL, PARAFAC2, CANDELINC, DEDICOM, and PARATUCK2 as well as nonnegative variants of all of the above. The N-way Toolbox, Tensor Toolbox, and Multilinear Engine are examples of software packages for working with tensors.

/pdf/tensor-decompositions-and-applications-46vl2ih5gn.pdf

Tensor Decompositions and Applications

This paper presents an account of the current state of sampling, 50 years after Shannon's formulation of the sampling theorem. The emphasis is on regular sampling, where the grid is uniform. This topic has benefitted from a strong research revival during the past few years, thanks in part to the mathematical connections that were made with wavelet theory. To introduce the reader to the modern, Hilbert-space formulation, we reinterpret Shannon's sampling procedure as an orthogonal projection onto the subspace of band-limited functions. We then extend the standard sampling paradigm for a presentation of functions in the more general class of "shift-in-variant" function spaces, including splines and wavelets. Practically, this allows for simpler-and possibly more realistic-interpolation models, which can be used in conjunction with a much wider class of (anti-aliasing) prefilters that are not necessarily ideal low-pass. We summarize and discuss the results available for the determination of the approximation error and of the sampling rate when the input of the system is essentially arbitrary; e.g., nonbandlimited. We also review variations of sampling that can be understood from the same unifying perspective. These include wavelets, multiwavelets, Papoulis generalized sampling, finite elements, and frames. Irregular sampling and radial basis functions are briefly mentioned.

/pdf/sampling-50-years-after-shannon-1i3rgczukl.pdf

Sampling-50 years after Shannon

Tensors or multiway arrays are functions of three or more indices $(i,j,k,\ldots)$ —similar to matrices (two-way arrays), which are functions of two indices $(r,c)$ for (row, column). Tensors have a rich history, stretching over almost a century, and touching upon numerous disciplines; but they have only recently become ubiquitous in signal and data analytics at the confluence of signal processing, statistics, data mining, and machine learning. This overview article aims to provide a good starting point for researchers and practitioners interested in learning about and working with tensors. As such, it focuses on fundamentals and motivation (using various application examples), aiming to strike an appropriate balance of breadth and depth that will enable someone having taken first graduate courses in matrix algebra and probability to get started doing research and/or developing tensor algorithms and software. Some background in applied optimization is useful but not strictly required. The material covered includes tensor rank and rank decomposition; basic tensor factorization models and their relationships and properties (including fairly good coverage of identifiability); broad coverage of algorithms ranging from alternating optimization to stochastic gradient; statistical performance analysis; and applications ranging from source separation to collaborative filtering, mixture and topic modeling, classification, and multilinear subspace learning.

Tensor Decomposition for Signal Processing and Machine Learning

A novel theory is introduced for analyzing image compression methods that are based on compression of wavelet decompositions. This theory precisely relates (a) the rate of decay in the error between the original image and the compressed image as the size of the compressed image representation increases (i.e., as the amount of compression decreases) to (b) the smoothness of the image in certain smoothness classes called Besov spaces. Within this theory, the error incurred by the quantization of wavelet transform coefficients is explained. Several compression algorithms based on piecewise constant approximations are analyzed in some detail. It is shown that, if pictures can be characterized by their membership in the smoothness classes considered, then wavelet-based methods are near-optimal within a larger class of stable transform-based, nonlinear methods of image compression. Based on previous experimental research it is argued that in most instances the error incurred in image compression should be measured in the integral sense instead of the mean-square sense. >

Image compression through wavelet transform coding

Preface 1. The approximation problem and existence of best approximations 2. The uniqueness of best approximations 3. Approximation operators and some approximating functions 4. Polynomial interpolation 5. Divided differences 6. The uniform convergence of polynomial approximations 7. The theory of minimax approximation 8. The exchange algorithm 9. The convergence of the exchange algorithm 10. Rational approximation by the exchange algorithm 11. Least squares approximation 12. Properties of orthogonal polynomials 13. Approximation of periodic functions 14. The theory of best L1 approximation 15. An example of L1 approximation and the discrete case 16. The order of convergence of polynomial approximations 17. The uniform boundedness theorem 18. Interpolation by piecewise polynomials 19. B-splines 20. Convergence properties of spline approximations 21. Knot positions and the calculation of spline approximations 22. The Peano kernel theorem 23. Natural and perfect splines 24. Optimal interpolation Appendices Index.

Approximation theory and methods, by M. J. D. Powell. Pp 339. £25 (hardcover), £8·50 (paperback). 1981. ISBN 0-521-22472-1/29514-9 (Cambridge University Press)

I want to thank the organizers for inviting me to this meeting as it gives me the opportunity to help celebrate Sam Conte who gave me my first academic job. More than that, he provided my children with many years of summer camp in the wilds of New Hampshire and Wisconsin, and at least two of them think that those summers in New Hampshire were essential for their growing up (and I tend to agree with them).

Polynomial interpolation in several variables

Spline approximation by quasiinterpolants

: Local support bases for piecewise polynomial spaces are important for applications such as finite element methods, data fitting etc. In (BH sub 1) a general construction principle for such B-splines was described. A special case are the so called box-splines. They have a particularly regular discontinuity pattern and coincide in special cases with standard finite elements. It is hoped that using translates of box-splines will lead, at least in two variables, to a unified theory for piecewise polynomial functions on regular meshes. This note is a first attempt in this direction and deals with basic approximation properties of translates of one box-splines such as stability, degree of approximation etc. (Author)

B-Splines From Parallelepipeds.

: One of the important properties of univariate splines is that in most senses smooth splines approximate just as well as do piecewise polynomials on the same mesh. This report shows this to be untrue in the multivariate setting. In particular, it details the cost in approximating power one may have to pay for the luxury of a smooth piecewise polynomial approximant. In an extreme case, piecewise polynomials of total degree r on a rectangular grid with all derivatives of order or = RHo continuous will fail to approximate certain smooth functions at all (as the grid goes to zero) unless RHo is kept below (r-3)/2. During the analysis of approximation on a certain regular triangular grid, a novel kind of bivariate B-spline is introduced. This B-spline, in contrast to the established multivariate B-spline derived from a simplex, can be made to have all its breaklines in a given regular grid. This makes it a prime candidate for use in the construction of smooth multivariate piecewise polynomial approximation, and its properties will be explored further.

/pdf/approximation-by-smooth-multivariate-splines-2k46f7h8e7.pdf

C. de Boor

Papers

Polynomial interpolation in several variables

Spline approximation by quasiinterpolants

B-Splines From Parallelepipeds.

Approximation by smooth multivariate splines

Bivariate box splines and smooth pp functions on a three direction mesh