Recent work by Kilmer and Martin [Linear Algebra Appl., 435 (2011), pp. 641--658] and Braman [Linear Algebra Appl., 433 (2010), pp. 1241--1253] provides a setting in which the familiar tools of linear algebra can be extended to better understand third-order tensors. Continuing along this vein, this paper investigates further implications including (1) a bilinear operator on the matrices which is nearly an inner product and which leads to definitions for length of matrices, angle between two matrices, and orthogonality of matrices, and (2) the use of t-linear combinations to characterize the range and kernel of a mapping defined by a third-order tensor and the t-product and the quantification of the dimensions of those sets. These theoretical results lead to the study of orthogonal projections as well as an effective Gram--Schmidt process for producing an orthogonal basis of matrices. The theoretical framework also leads us to consider the notion of tensor polynomials and their relation to tensor eigentupl...

Third-Order Tensors as Operators on Matrices: A Theoretical and Computational Framework with Applications in Imaging

The authors' survey paper is devoted to the present state of computational methods in linear algebra. Questions discussed are the means and methods of estimating the quality of numerical solution of computational problems, the generalized inverse of a matrix, the solution of systems with rectangular and poorly conditioned matrices, the inverse eigenvalue problem, and more traditional questions such as algebraic eigenvalue problems and the solution of systems with a square matrix (by direct and iterative methods).

Computational methods of linear algebra

Random matrix theory is now a big subject with applications in many disciplines of science, engineering and finance. This article is a survey specifically oriented towards the needs and interests of a numerical analyst. This survey includes some original material not found anywhere else. We include the important mathematics which is a very modern development, as well as the computational software that is transforming the theory into useful practice.

https://stuff.mit.edu/people/raj/Acta05rmt.pdf

Random matrix theory

Theory of Equations. By J. V. Uspensky. Pp. vii, 353. 27s. 1948. (McGraw-Hill)

We introduce a flexible optimization framework for nuclear norm minimization of matrices with linear structure, including Hankel, Toeplitz, and moment structures and catalog applications from diverse fields under this framework. We discuss various first-order methods for solving the resulting optimization problem, including alternating direction methods of multipliers, proximal point algorithms, and gradient projection methods. We perform computational experiments to compare these methods on system identification problems and system realization problems. For the system identification problem, the gradient projection method (accelerated by Nesterov's extrapolation techniques) and the proximal point algorithm usually outperform other first-order methods in terms of CPU time on both real and simulated data, for small and large regularization parameters, respectively, while for the system realization problem, the alternating direction method of multipliers, as applied to a certain primal reformulation, usuall...

https://faculty.washington.edu/mfazel/Hankelrm23.pdf

Hankel Matrix Rank Minimization with Applications to System Identification and Realization

A lower bound for the smallest singular value of a matrix

We derive a stable technique, based upon matrix pencils, for the reconstruction of (or approximation by) polygonal shapes from moments. We point out that this problem can be considered the dual of 2-D numerical quadrature over polygonal domains. An analysis of the sensitivity of the problem is presented along with some numerical examples illustrating the relevant points. Finally, an application to the problem of gravimetry is explored where the shape of a gravitationally anomalous region is to be recovered from measurements of its exterior gravitational field.

A Stable Numerical Method for Inverting Shape from Moments

The prolate matrix

SUMMARY

Circulant matrices play a central role in a recently proposed formulation of three-way data computations. In this setting, a three-way table corresponds to a matrix where each ‘scalar’ is a vector of parameters defining a circulant. This interpretation provides many generalizations of results from matrix or vector-space algebra. These results and algorithms are closely related to standard decoupling techniques on block-circulant matrices using the fast Fourier transform. We derive the power and Arnoldi methods in this algebra. In the course of our derivation, we define inner products, norms, and other notions. These extensions are straightforward in an algebraic sense, but the implications are dramatically different from the standard matrix case. For example, the number of eigenpairs may exceed the dimension of the matrix, although it is still polynomial in it. It is thus necessary to take an extra step and carefully select a smaller, canonical set of size equal to the dimension of the matrix from which all possible eigenpairs can be formed. Copyright © 2012 John Wiley & Sons, Ltd.

/pdf/the-power-and-arnoldi-methods-in-an-algebra-of-circulants-3lnbn0fl7f.pdf

The power and Arnoldi methods in an algebra of circulants

We consider a positive definite block preconditioner for solving saddle point linear systems. An approach based on augmenting the (1,1) block while keeping its condition number small is described, and algebraic analysis is performed. Ways of selecting the parameters involved are discussed, and analytical and numerical observations are given.

James M. Varah

Papers

A lower bound for the smallest singular value of a matrix

A Stable Numerical Method for Inverting Shape from Moments

The prolate matrix

The power and Arnoldi methods in an algebra of circulants

An Algebraic Analysis of a Block Diagonal Preconditioner for Saddle Point Systems