This survey provides an overview of higher-order tensor decompositions, their applications, and available software. A tensor is a multidimensional or $N$-way array. Decompositions of higher-order tensors (i.e., $N$-way arrays with $N \geq 3$) have applications in psycho-metrics, chemometrics, signal processing, numerical linear algebra, computer vision, numerical analysis, data mining, neuroscience, graph analysis, and elsewhere. Two particular tensor decompositions can be considered to be higher-order extensions of the matrix singular value decomposition: CANDECOMP/PARAFAC (CP) decomposes a tensor as a sum of rank-one tensors, and the Tucker decomposition is a higher-order form of principal component analysis. There are many other tensor decompositions, including INDSCAL, PARAFAC2, CANDELINC, DEDICOM, and PARATUCK2 as well as nonnegative variants of all of the above. The N-way Toolbox, Tensor Toolbox, and Multilinear Engine are examples of software packages for working with tensors.

/pdf/tensor-decompositions-and-applications-46vl2ih5gn.pdf

Tensor Decompositions and Applications

Today's Web-enabled deluge of electronic data calls for automated methods of data analysis. Machine learning provides these, developing methods that can automatically detect patterns in data and then use the uncovered patterns to predict future data. This textbook offers a comprehensive and self-contained introduction to the field of machine learning, based on a unified, probabilistic approach. The coverage combines breadth and depth, offering necessary background material on such topics as probability, optimization, and linear algebra as well as discussion of recent developments in the field, including conditional random fields, L1 regularization, and deep learning. The book is written in an informal, accessible style, complete with pseudo-code for the most important algorithms. All topics are copiously illustrated with color images and worked examples drawn from such application domains as biology, text processing, computer vision, and robotics. Rather than providing a cookbook of different heuristic methods, the book stresses a principled model-based approach, often using the language of graphical models to specify models in a concise and intuitive way. Almost all the models described have been implemented in a MATLAB software package--PMTK (probabilistic modeling toolkit)--that is freely available online. The book is suitable for upper-level undergraduates with an introductory-level college math background and beginning graduate students.

Machine Learning : A Probabilistic Perspective

A new variant ‘PMF’ of factor analysis is described. It is assumed that X is a matrix of observed data and σ is the known matrix of standard deviations of elements of X. Both X and σ are of dimensions n × m. The method solves the bilinear matrix problem X = GF + E where G is the unknown left hand factor matrix (scores) of dimensions n × p, F is the unknown right hand factor matrix (loadings) of dimensions p × m, and E is the matrix of residuals. The problem is solved in the weighted least squares sense: G and F are determined so that the Frobenius norm of E divided (element-by-element) by σ is minimized. Furthermore, the solution is constrained so that all the elements of G and F are required to be non-negative. It is shown that the solutions by PMF are usually different from any solutions produced by the customary factor analysis (FA, i.e. principal component analysis (PCA) followed by rotations). Usually PMF produces a better fit to the data than FA. Also, the result of PF is guaranteed to be non-negative, while the result of FA often cannot be rotated so that all negative entries would be eliminated. Different possible application areas of the new method are briefly discussed. In environmental data, the error estimates of data can be widely varying and non-negativity is often an essential feature of the underlying models. Thus it is concluded that PMF is better suited than FA or PCA in many environmental applications. Examples of successful applications of PMF are shown in companion papers.

Positive matrix factorization: A non-negative factor model with optimal utilization of error estimates of data values†

Non-negative matrix factorization (NMF) is a recently developed technique for finding parts-based, linear representations of non-negative data. Although it has successfully been applied in several applications, it does not always result in parts-based representations. In this paper, we show how explicitly incorporating the notion of 'sparseness' improves the found decompositions. Additionally, we provide complete MATLAB code both for standard NMF and for our extension. Our hope is that this will further the application of these methods to solving novel data-analysis problems.

/pdf/non-negative-matrix-factorization-with-sparseness-40ldavp7a5.pdf

Non-negative Matrix Factorization with Sparseness Constraints

This book provides a broad survey of models and efficient algorithms for Nonnegative Matrix Factorization (NMF) This includes NMFs various extensions and modifications, especially Nonnegative Tensor Factorizations (NTF) and Nonnegative Tucker Decompositions (NTD) NMF/NTF and their extensions are increasingly used as tools in signal and image processing, and data analysis, having garnered interest due to their capability to provide new insights and relevant information about the complex latent relationships in experimental data sets It is suggested that NMF can provide meaningful components with physical interpretations; for example, in bioinformatics, NMF and its extensions have been successfully applied to gene expression, sequence analysis, the functional characterization of genes, clustering and text mining As such, the authors focus on the algorithms that are most useful in practice, looking at the fastest, most robust, and suitable for large-scale models Key features: Acts as a single source reference guide to NMF, collating information that is widely dispersed in current literature, including the authors own recently developed techniques in the subject area Uses generalized cost functions such as Bregman, Alpha and Beta divergences, to present practical implementations of several types of robust algorithms, in particular Multiplicative, Alternating Least Squares, Projected Gradient and Quasi Newton algorithms Provides a comparative analysis of the different methods in order to identify approximation error and complexity Includes pseudo codes and optimized MATLAB source codes for almost all algorithms presented in the book The increasing interest in nonnegative matrix and tensor factorizations, as well as decompositions and sparse representation of data, will ensure that this book is essential reading for engineers, scientists, researchers, industry practitioners and graduate students across signal and image processing; neuroscience; data mining and data analysis; computer science; bioinformatics; speech processing; biomedical engineering; and multimedia

Nonnegative Matrix and Tensor Factorizations: Applications to Exploratory Multi-way Data Analysis and Blind Source Separation

Analysis of different modes of factor analysis as least squares fit problems

Source identification of bulk wet deposition in Finland by positive matrix factorization

[1] Ultrafine particles sampled during new particle formation bursts observed in the coastal zone were studied with transmission electron microscopy (TEM) and elemental analysis using energy-dispersive X ray (EDX). It was observed that both iodine and sulphur were present in the new particles with diameter below 10 mn. Gaseous emissions of halogen compounds from seaweeds were also measured at the same location during low-tide particle nucleation episodes. Based on the presence of iodine in the particle phase during low-tide nucleation bursts, and the significant emission of iodine compounds from the seaweeds during these periods, it is apparent that part of the biogenic iodine species emitted from the seaweeds end up in the ultrafine particulate phase. It was not possible to quantitatively determine the iodine content in the particles; however, in most cases the relative contribution from iodine and sulphate was similar, while some cases indicated no sulphate. On larger sized particles the contribution of sulphate was significantly higher than iodine. It appears that the condensable species leading to the appearance of new particles in the coastal atmosphere is an iodine species. Whether or not this iodine species also participates in the nucleation of new stable clusters could not be completely verified.

http://www.nuigalway.ie/staff-sites/david_payne/asrg/documents/makela02.pdf

Biogenic iodine emissions and identification of end-products in coastal ultrafine particles during nucleation bursts

Ultrafine particles sampled during new particle formation bursts observed in the coastal zone were studied with transmission electron microscopy (TEM) and elemental analysis using energy-dispersive X ray (EDX). It was observed that both iodine and sulphur were present in the new particles with diameter below 10 mn. Gaseous emissions of halogen compounds from seaweeds were also measured at the same location during low-tide particle nucleation episodes. Based on the presence of iodine in the particle phase during low-tide nucleation bursts, and the significant emission of iodine compounds from the seaweeds during these periods, it is apparent that part of the biogenic iodine species emitted from the seaweeds end up in the ultrafine particulate phase. It was not possible to quantitatively determine the iodine content in the particles; however, in most cases the relative contribution from iodine and sulphate was similar, while some cases indicated no sulphate. On larger sized particles the contribution of sulphate was significantly higher than iodine. It appears that the condensable species leading to the appearance of new particles in the coastal atmosphere is an iodine species. Whether or not this iodine species also participates in the nucleation of new stable clusters could not be completely verified.

Biogenic iodine emissions and identification of end-products in coastal ultrafine particles during nucleation bursts : New Particle Formation and Fate in the Coastal Environment (PARFORCE)

Unto Tapper

Papers

Positive matrix factorization: A non-negative factor model with optimal utilization of error estimates of data values†

Analysis of different modes of factor analysis as least squares fit problems

Source identification of bulk wet deposition in Finland by positive matrix factorization

Biogenic iodine emissions and identification of end-products in coastal ultrafine particles during nucleation bursts

Biogenic iodine emissions and identification of end-products in coastal ultrafine particles during nucleation bursts : New Particle Formation and Fate in the Coastal Environment (PARFORCE)