https://www2.econ.iastate.edu/tesfatsi/DeepLearningInNeuralNetworksOverview.JSchmidhuber2015.pdf

Deep learning in neural networks

Preface 1. Background in linear algebra 2. Discretization of partial differential equations 3. Sparse matrices 4. Basic iterative methods 5. Projection methods 6. Krylov subspace methods Part I 7. Krylov subspace methods Part II 8. Methods related to the normal equations 9. Preconditioned iterations 10. Preconditioning techniques 11. Parallel implementations 12. Parallel preconditioners 13. Multigrid methods 14. Domain decomposition methods Bibliography Index.

/pdf/iterative-methods-for-sparse-linear-systems-34ozhpwq89.pdf

Iterative Methods for Sparse Linear Systems

We present an iterative method for solving linear systems, which has the property of minimizing at every step the norm of the residual vector over a Krylov subspace. The algorithm is derived from t...

/pdf/gmres-a-generalized-minimal-residual-algorithm-for-solving-4xovqxzqlw.pdf

GMRES: a generalized minimal residual algorithm for solving nonsymmetric linear systems

In this article we discuss our experience designing and implementing a statistical computing language. In developing this new language, we sought to combine what we felt were useful features from two existing computer languages. We feel that the new language provides advantages in the areas of portability, computational efficiency, memory management, and scoping.

R: A Language for Data Analysis and Graphics

In recent years, spectral clustering has become one of the most popular modern clustering algorithms. It is simple to implement, can be solved efficiently by standard linear algebra software, and very often outperforms traditional clustering algorithms such as the k-means algorithm. On the first glance spectral clustering appears slightly mysterious, and it is not obvious to see why it works at all and what it really does. The goal of this tutorial is to give some intuition on those questions. We describe different graph Laplacians and their basic properties, present the most common spectral clustering algorithms, and derive those algorithms from scratch by several different approaches. Advantages and disadvantages of the different spectral clustering algorithms are discussed.

/pdf/a-tutorial-on-spectral-clustering-536pmg9onu.pdf

A tutorial on spectral clustering

Preliminaries. Norms and Metrics. Linear Systems and Least Squares Problems. The Perturbation of Eigenvalues. Invariant Subspaces. Generalized Eigenvalue Problems.

/pdf/matrix-perturbation-theory-5e7zlkig8j.pdf

Matrix perturbation theory

Preliminaries Practicalities The Direct Solution of Linear Systems Norms, Limits, and Condition Numbers The Linear Least Squares Problem Eigenvalues and Eigenvectors The QR Algorithm The Greek Alphabet and Latin Notational Correspondents Determinants Rounding-Error Analysis of Solution of Triangular Systems and of Gaussian Elimination Of Things Not Treated Bibliography Index

Introduction to matrix computations

and Canada) or $18.00 (elsewhere). If the user sends a small tape (wt. less than 1 lb.) the algorithm will be copied on it and returned to him at a charge of $10.O0 (U.S. only). All orders are to be prepaid with checks payable to ACM Algorithms. The algorithm is re corded as one file of BCD 80 character card images at 556 B.P.I., even parity, on seven ~rack tape. We will supply the algorithm at a density of 800 B.P.I. if requested. The cards for the algorithm are sequenced starting at 10 and incremented by 10. The sequence number is right justified in colums 80. Although we will make every attempt to insure that the algorithm conforms to the description printed here, we cannot guarantee it, nor can we guarantee that the algorithm is correct.-L.D.F. Descdption The following programs are a collection of Fortran IV sub-routines to solve the matrix equation AX-.}-XB = C (1) where A, B, and C are real matrices of dimensions m X m, n X n, and m X n, respectively. Additional subroutines permit the efficient solution of the equation ArX + xa = C, (2) where C is symmetric. Equation (1) has applications to the direct solution of discrete Poisson equations [2]. It is well known that (1) has a unique solution if and only if the One proof of the result amounts to constructing the solution from complete systems of eigenvalues and eigenvectors of A and B, when they exist. This technique has been proposed as a computational method (e.g. see [1 ]); however, it is unstable when the eigensystem is ill conditioned. The method proposed here is based on the Schur reduction to triangular form by orthogonal similarity transformations. Equation (1) is solved as follows. The matrix A is reduced to lower real Schur form A' by an orthogonal similarity transformation U; that is A is reduced to the real, block lower triangular form.

Solution of the matrix equation AX + XB = C [F4]

General matrices Band matrices Positive definite matrices Positive definite band matrices Symmetric Indefinite Matrices Triangular matrices Tridiagonal matrices The Cholesky decomposition The QR decomposition Updating QR and Cholesky decompositions The singular value decomposition References Basic linear algebra subprograms Timing data Program listings BLA Listings.

LINPACK Users' Guide

A new method, called the $QZ$ algorithm, is presented for the solution of the matrix eigenvalue problem $Ax = \lambda Bx$ with general square matrices A and B. Particular attention is paid to the degeneracies which result when B is singular. No inversions of B or its submatrices are used. The algorithm is a generalization of the $QR$ algorithm, and reduces to it when $B = I$. Problems involving higher powers of $\lambda $ are also mentioned.

G. W. Stewart

Papers

Matrix perturbation theory

Introduction to matrix computations

Solution of the matrix equation AX + XB = C [F4]

LINPACK Users' Guide

An Algorithm for Generalized Matrix Eigenvalue Problems.