There is, I think, something ethereal about i —the square root of minus one. I remember first hearing about it at school. It seemed an odd beast at that time—an intruder hovering on the edge of reality.

Usually familiarity dulls this sense of the bizarre, but in the case of i it was the reverse: over the years the sense of its surreal nature intensified. It seemed that it was impossible to write mathematics that described the real world in …

I and i

https://www2.econ.iastate.edu/tesfatsi/DeepLearningInNeuralNetworksOverview.JSchmidhuber2015.pdf

Deep learning in neural networks

Preface 1. Background in linear algebra 2. Discretization of partial differential equations 3. Sparse matrices 4. Basic iterative methods 5. Projection methods 6. Krylov subspace methods Part I 7. Krylov subspace methods Part II 8. Methods related to the normal equations 9. Preconditioned iterations 10. Preconditioning techniques 11. Parallel implementations 12. Parallel preconditioners 13. Multigrid methods 14. Domain decomposition methods Bibliography Index.

/pdf/iterative-methods-for-sparse-linear-systems-34ozhpwq89.pdf

Iterative Methods for Sparse Linear Systems

/pdf/scalable-molecular-dynamics-with-namd-3j7xyc70g4.pdf

Scalable Molecular Dynamics with NAMD

Recently, a number of researchers have investigated a class of graph partitioning algorithms that reduce the size of the graph by collapsing vertices and edges, partition the smaller graph, and then uncoarsen it to construct a partition for the original graph [Bui and Jones, Proc. of the 6th SIAM Conference on Parallel Processing for Scientific Computing, 1993, 445--452; Hendrickson and Leland, A Multilevel Algorithm for Partitioning Graphs, Tech. report SAND 93-1301, Sandia National Laboratories, Albuquerque, NM, 1993]. From the early work it was clear that multilevel techniques held great promise; however, it was not known if they can be made to consistently produce high quality partitions for graphs arising in a wide range of application domains. We investigate the effectiveness of many different choices for all three phases: coarsening, partition of the coarsest graph, and refinement. In particular, we present a new coarsening heuristic (called heavy-edge heuristic) for which the size of the partition of the coarse graph is within a small factor of the size of the final partition obtained after multilevel refinement. We also present a much faster variation of the Kernighan--Lin (KL) algorithm for refining during uncoarsening. We test our scheme on a large number of graphs arising in various domains including finite element methods, linear programming, VLSI, and transportation. Our experiments show that our scheme produces partitions that are consistently better than those produced by spectral partitioning schemes in substantially smaller time. Also, when our scheme is used to compute fill-reducing orderings for sparse matrices, it produces orderings that have substantially smaller fill than the widely used multiple minimum degree algorithm.

/pdf/a-fast-and-high-quality-multilevel-scheme-for-partitioning-2i240yved4.pdf

A Fast and High Quality Multilevel Scheme for Partitioning Irregular Graphs

Consider the ridge estimate (λ) for β in the model unknown, (λ) = (X T X + nλI)−1 X T y. We study the method of generalized cross-validation (GCV) for choosing a good value for λ from the data. The estimate is the minimizer of V(λ) given by where A(λ) = X(X T X + nλI)−1 X T . This estimate is a rotation-invariant version of Allen's PRESS, or ordinary cross-validation. This estimate behaves like a risk improvement estimator, but does not require an estimate of σ2, so can be used when n − p is small, or even if p ≥ 2 n in certain cases. The GCV method can also be used in subset selection and singular value truncation methods for regression, and even to choose from among mixtures of these methods.

/pdf/generalized-cross-validation-as-a-method-for-choosing-a-good-ni6kr99a0h.pdf

Generalized Cross-Validation as a Method for Choosing a Good Ridge Parameter

From the Publisher:
"This book presents a broad overview of numerical methods for solving all the major problems in scientific computing, including linear and nonlinear equations, least squares, eigenvalues, optimization, interpolation, integration, ordinary and partial differential equations, fast Fourier transforms, and random number generators The treatment is comprehensive yet concise, software-oriented yet compatible with a variety of software packages and programming languages The book features more than 160 examples, 500 review questions, 240 exercises, and 200 computer problems" "Changes for the second edition include: expanded motivational discussions and examples; formal statements of all major algorithms; expanded discussions of existence, uniqueness, and conditioning for each type of problem so that students can recognize "good" and "bad" problem formulations and understand the corresponding quality of results producted; and expanded coverage of several topics, particularly eigenvalues and constrained optimization" The book contains a wealth of material and can be used in a variety of one- or two-term courses in computer science, mathematics, or engineering Its comprehensiveness and modern perspective, as well as the software pointers provided, also make it a highly useful reference for practicing professionals who need to solve computational problems

Scientific Computing: An Introductory Survey

An algorithm is presented in this paper for computing state-space balancing transformations directly from a state-space realization. The algorithm requires no "squaring up" or unnecessary matrix products. Various algorithmic aspects are discussed in detail. A key feature of the algorithm is the determination of a contragredient transformation through computing the singular value decomposition of a certain product of matrices without explicitly forming the product. Other contragredient transformation applications are also described. It is further shown that a similar approach may be taken, involving the generalized singular value decomposition, to the classical simultaneous diagonalization problem. These SVD-based simultaneous diagonalization algorithms provide a computational alternative to existing methods for solving certain classes of symmetric positive definite generalized eigenvalue problems.

Computation of system balancing transformations and other applications of simultaneous diagonalization algorithms

ParaGraph, a software tool that provides a detailed, dynamic, graphical animation of the behavior of message-passing parallel programs and graphical summaries of their performance, is presented. ParaGraph animates trace information from actual runs to depict behavior and obtain the performance summaries. It provides twenty-five perspectives on the same data, lending insight that might otherwise be missed. ParaGraph's features are described, its use is explained, its software design is briefly discussed, and its displays are examined in some detail. Future work on ParaGraph is indicated. >

Michael T. Heath

Papers

Generalized Cross-Validation as a Method for Choosing a Good Ridge Parameter

Scientific Computing: An Introductory Survey

Computation of system balancing transformations and other applications of simultaneous diagonalization algorithms

Visualizing the performance of parallel programs

Visualizing the performance of parallel programs