Nonnegative Matrix Approximation: Algorithms and Applications

TLDR: Generic methods for minimizingeralized divergences between the input and its low rank approximant and interesting extensions such as the use of penalty function, non-linear relationships via “link” functions, weighted errors, and multi-factor approximations are considered.

Abstract: Low dimensional data representations are crucial to numerous applicatio ns in machine learning, statistics, and signal processing. Nonnegative matrix approximation (NNMA) is a method for dimensionality reduction that respects the nonnegativity of the input data while constructin g a low-dimensional approximation. NNMA has been used in a multitude of applications, though without com mensurate theoretical development. In this report we describe generic methods for minimizing g e eralized divergences between the input and its low rank approximant. Some of our general methods are even extensible to arbitrary convex penalties. Our methods yield efficient multiplicative iterative schemes for s olving the proposed problems. We also consider interesting extensions such as the use of penalty function s, non-linear relationships via “link” functions, weighted errors, and multi-factor approximations. We p r sent some experiments as an illustration of our algorithms. For completeness, the report also includes a brief literature survey of the various algorithms and the applications of NNMA.