We present an iterative method for solving linear systems, which has the property of minimizing at every step the norm of the residual vector over a Krylov subspace. The algorithm is derived from t...

/pdf/gmres-a-generalized-minimal-residual-algorithm-for-solving-4xovqxzqlw.pdf

GMRES: a generalized minimal residual algorithm for solving nonsymmetric linear systems

We describe the University of Florida Sparse Matrix Collection, a large and actively growing set of sparse matrices that arise in real applications The Collection is widely used by the numerical linear algebra community for the development and performance evaluation of sparse matrix algorithms It allows for robust and repeatable experiments: robust because performance results with artificially generated matrices can be misleading, and repeatable because matrices are curated and made publicly available in many formats Its matrices cover a wide spectrum of domains, include those arising from problems with underlying 2D or 3D geometry (as structural engineering, computational fluid dynamics, model reduction, electromagnetics, semiconductor devices, thermodynamics, materials, acoustics, computer graphics/vision, robotics/kinematics, and other discretizations) and those that typically do not have such geometry (optimization, circuit simulation, economic and financial modeling, theoretical and quantum chemistry, chemical process simulation, mathematics and statistics, power networks, and other networks and graphs) We provide software for accessing and managing the Collection, from MATLAB™, Mathematica™, Fortran, and C, as well as an online search capability Graph visualization of the matrices is provided, and a new multilevel coarsening scheme is proposed to facilitate this task

The university of Florida sparse matrix collection

Low-rank matrix approximations, such as the truncated singular value decomposition and the rank-revealing QR decomposition, play a central role in data analysis and scientific computing. This work surveys and extends recent research which demonstrates that randomization offers a powerful tool for performing low-rank matrix approximation. These techniques exploit modern computational architectures more fully than classical methods and open the possibility of dealing with truly massive data sets. This paper presents a modular framework for constructing randomized algorithms that compute partial matrix decompositions. These methods use random sampling to identify a subspace that captures most of the action of a matrix. The input matrix is then compressed—either explicitly or implicitly—to this subspace, and the reduced matrix is manipulated deterministically to obtain the desired low-rank factorization. In many cases, this approach beats its classical competitors in terms of accuracy, robustness, and/or speed. These claims are supported by extensive numerical experiments and a detailed error analysis. The specific benefits of randomized techniques depend on the computational environment. Consider the model problem of finding the $k$ dominant components of the singular value decomposition of an $m \times n$ matrix. (i) For a dense input matrix, randomized algorithms require $\bigO(mn \log(k))$ floating-point operations (flops) in contrast to $ \bigO(mnk)$ for classical algorithms. (ii) For a sparse input matrix, the flop count matches classical Krylov subspace methods, but the randomized approach is more robust and can easily be reorganized to exploit multiprocessor architectures. (iii) For a matrix that is too large to fit in fast memory, the randomized techniques require only a constant number of passes over the data, as opposed to $\bigO(k)$ passes for classical algorithms. In fact, it is sometimes possible to perform matrix approximation with a single pass over the data.

/pdf/finding-structure-with-randomness-probabilistic-algorithms-11r587jy79.pdf

Finding Structure with Randomness: Probabilistic Algorithms for Constructing Approximate Matrix Decompositions

Petroleum Reservoir Simulation

Low-rank matrix approximations, such as the truncated singular value decomposition and the rank-revealing QR decomposition, play a central role in data analysis and scientific computing. This work surveys and extends recent research which demonstrates that randomization offers a powerful tool for performing low-rank matrix approximation. These techniques exploit modern computational architectures more fully than classical methods and open the possibility of dealing with truly massive data sets. 
This paper presents a modular framework for constructing randomized algorithms that compute partial matrix decompositions. These methods use random sampling to identify a subspace that captures most of the action of a matrix. The input matrix is then compressed---either explicitly or implicitly---to this subspace, and the reduced matrix is manipulated deterministically to obtain the desired low-rank factorization. In many cases, this approach beats its classical competitors in terms of accuracy, speed, and robustness. These claims are supported by extensive numerical experiments and a detailed error analysis.

Finding structure with randomness: Probabilistic algorithms for constructing approximate matrix decompositions

We consider a class of iterative algorithms for solving systems of linear equations where the coefficient matrix is nonsymmetric with positive-definite symmetric part. The algorithms are modelled after the conjugate gradient method, and are well suited for large sparse systems. They do not make use of any associated symmetric problems. Convergence results and error bounds are presented.

Variational Iterative Methods for Nonsymmetric Systems of Linear Equations

We investigate several ways to improve the performance of sparse LU factorization with partial pivoting, as used to solve unsymmetric linear systems. We introduce the notion of unsymmetric supernodes to perform most of the numerical computation in dense matrix kernels. We introduce unsymmetric supernode-panel updates and two-dimensional data partitioning to better exploit the memory hierarchy. We use Gilbert and Peierls's depth-first search with Eisenstat and Liu's symmetric structural reductions to speed up symbolic factorization. 
We have developed a sparse LU code using all these ideas. We present experiments demonstrating that it is significantly faster than earlier partial pivoting codes. We also compare its performance with UMFPACK, which uses a multifrontal approach; our code is very competitive in time and storage requirements, especially for large problems.

/pdf/a-supernodal-approach-to-sparse-partial-pivoting-30rkhd31hw.pdf

A Supernodal Approach to Sparse Partial Pivoting

Given anm n matrixM withm > n, it is shown that there exists a permutation FI and an integer k such that the QR factorization MYI= Q(Ak ckBk) reveals the numerical rank of M: the k k upper-triangular matrix Ak is well conditioned, IlCkll2 is small, and Bk is linearly dependent on Ak with coefficients bounded by a low-degree polynomial in n. Existing rank-revealing QR (RRQR) algorithms are related to such factorizations and two algorithms are presented for computing them. The new algorithms are nearly as efficient as QR with column pivoting for most problems and take O (ran2) floating-point operations in the worst case.

/pdf/efficient-algorithms-for-computing-a-strong-rank-revealing-27z8orqr7o.pdf

Efficient algorithms for computing a strong rank-revealing QR factorization

: Consider the NxN system of linear equations M x = b, where the coefficient matrix M is large, sparse, and nonsymmetric. Assume that M can be factored in the form M = L D U, where L is a lower triangular matrix, D is a diagonal matrix, and U is a unit upper triangular matrix. Such systems arise frequently in scientific computation, e.g., in finite difference and finite element approximations to non-self-adjoint elliptic boundary value problems. This report presents a package of efficient, reliable, well-documented, and portable FORTRAN subroutines for solving these systems.

Yale sparse matrix package I: The symmetric codes

The authors present a stable and efficient divide-and-conquer algorithm for computing the spectral decomposition of an $N \times N$ symmetric tridiagonal matrix. The key elements are a new, stable method for finding the spectral decomposition of a symmetric arrowhead matrix and a new implementation of deflation. Numerical results show that this algorithm is competitive with bisection with inverse iteration, Cuppen's divide-and-conquer algorithm, and the QR algorithm for solving the symmetric tridiagonal eigenproblem.

/pdf/a-divide-and-conquer-algorithm-for-the-symmetric-4esv08oidz.pdf

Stanley C. Eisenstat

Papers

Variational Iterative Methods for Nonsymmetric Systems of Linear Equations

A Supernodal Approach to Sparse Partial Pivoting

Efficient algorithms for computing a strong rank-revealing QR factorization

Yale sparse matrix package I: The symmetric codes

A Divide-and-Conquer Algorithm for the Symmetric TridiagonalEigenproblem