Preface 1. Background in linear algebra 2. Discretization of partial differential equations 3. Sparse matrices 4. Basic iterative methods 5. Projection methods 6. Krylov subspace methods Part I 7. Krylov subspace methods Part II 8. Methods related to the normal equations 9. Preconditioned iterations 10. Preconditioning techniques 11. Parallel implementations 12. Parallel preconditioners 13. Multigrid methods 14. Domain decomposition methods Bibliography Index.

/pdf/iterative-methods-for-sparse-linear-systems-34ozhpwq89.pdf

Iterative Methods for Sparse Linear Systems

This paper describes mathematical and software developments for a suite of programs for solving ordinary differential equations in MATLAB.

The MATLAB ODE Suite

Stan is a probabilistic programming language for specifying statistical models. A Stan program imperatively defines a log probability function over parameters conditioned on specified data and constants. As of version 2.14.0, Stan provides full Bayesian inference for continuous-variable models through Markov chain Monte Carlo methods such as the No-U-Turn sampler, an adaptive form of Hamiltonian Monte Carlo sampling. Penalized maximum likelihood estimates are calculated using optimization methods such as the limited memory Broyden-Fletcher-Goldfarb-Shanno algorithm. Stan is also a platform for computing log densities and their gradients and Hessians, which can be used in alternative algorithms such as variational Bayes, expectation propagation, and marginal inference using approximate integration. To this end, Stan is set up so that the densities, gradients, and Hessians, along with intermediate quantities of the algorithm such as acceptance probabilities, are easily accessible. Stan can be called from the command line using the cmdstan package, through R using the rstan package, and through Python using the pystan package. All three interfaces support sampling and optimization-based inference with diagnostics and posterior analysis. rstan and pystan also provide access to log probabilities, gradients, Hessians, parameter transforms, and specialized plotting.

/pdf/stan-a-probabilistic-programming-language-38sfkv8win.pdf

Stan: A Probabilistic Programming Language.

Petroleum Reservoir Simulation

SUNDIALS is a suite of advanced computational codes for solving large-scale problems that can be modeled as a system of nonlinear algebraic equations, or as initial-value problems in ordinary differential or differential-algebraic equations. The basic versions of these codes are called KINSOL, CVODE, and IDA, respectively. The codes are written in ANSI standard C and are suitable for either serial or parallel machine environments. Common and notable features of these codes include inexact Newton-Krylov methods for solving large-scale nonlinear systems; linear multistep methods for time-dependent problems; a highly modular structure to allow incorporation of different preconditioning and/or linear solver methods; and clear interfaces allowing for users to provide their own data structures underneath the solvers. We describe the current capabilities of the codes, along with some of the algorithms and heuristics used to achieve efficiency and robustness. We also describe how the codes stem from previous and widely used Fortran 77 solvers, and how the codes have been augmented with forward and adjoint methods for carrying out first-order sensitivity analysis with respect to model parameters or initial conditions.

SUNDIALS: Suite of nonlinear and differential/algebraic equation solvers

VODE is a new initial value ODE solver for stiff and nonstiff systems. It uses variable-coefficient Adams-Moulton and Backward Differentiation Formula (BDF) methods in Nordsieck form, as taken from the older solvers EPISODE and EPISODEB, treating the Jacobian as full or banded. Unlike the older codes, VODE has a highly flexible user interface that is nearly identical to that of the ODEPACK solver LSODE.In the process, several algorithmic improvements have been made in VODE, aside from the new user interface. First, a change in stepsize and/or order that is decided upon at the end of one successful step is not implemented until the start of the next step, so that interpolations performed between steps use the more correct data. Second, a new algorithm for setting the initial stepsize has been included, which iterates briefly to estimate the required second derivative vector. Efficiency is often greatly enhanced by an added algorithm for saving and reusing the Jacobian matrix J, as it occurs in the Newton m...

VODE: a variable-coefficient ODE solver

Two new packages are available for the numerical solution of the initial value problem for stiff and nonstiff systems of ordinary differential equations (ODE's). LSODE solves explicitly given ODE systems, while LSODI solves systems given in linearly implicit form, including differential-algebraic systems. Both are based on older packages that have proved highly popular, but both reflect a wide variety of improvements in the design of the user interface, algorithms, and program structure.

LSODE and LSODI, two new initial value ordinary differential equation solvers

CVODE is a package written in C for solving initial value problems for ordinary di erential equations. It provides the capabilities of two older Fortran packages, VODE and VODPK. CVODE solves both sti and nonsti systems, using variable-coe cient Adams and BDF methods. In the sti case, options for treating the Jacobian of the system include dense and band matrix solvers, and a preconditioned Krylov (iterative) solver. In the highly modular organization of CVODE, the core integrator module is independent of the linear system solvers, and all operations on N -vectors are isolated in a module of vector kernels. A set of parallel extenstions of CVODE, called PVODE, is being developed. CVODE is available from Netlib, and comes with an extensive user guide.

CVODE, a stiff/nonstiff ODE solver in C

LSODE, the Livermore Solver for Ordinary Differential Equations, is a package of FORTRAN subroutines designed for the numerical solution of the initial value problem for a system of ordinary differential equations. It is particularly well suited for 'stiff' differential systems, for which the backward differentiation formula method of orders 1 to 5 is provided. The code includes the Adams-Moulton method of orders 1 to 12, so it can be used for nonstiff problems as well. In addition, the user can easily switch methods to increase computational efficiency for problems that change character. For both methods a variety of corrector iteration techniques is included in the code. Also, to minimize computational work, both the step size and method order are varied dynamically. This report presents complete descriptions of the code and integration methods, including their implementation. It also provides a detailed guide to the use of the code, as well as an illustrative example problem.

/pdf/description-and-use-of-lsode-the-livermore-solver-for-53jijk9i1e.pdf

Alan C. Hindmarsh

Papers

SUNDIALS: Suite of nonlinear and differential/algebraic equation solvers

VODE: a variable-coefficient ODE solver

LSODE and LSODI, two new initial value ordinary differential equation solvers

CVODE, a stiff/nonstiff ODE solver in C

Description and use of LSODE, the Livermore Solver for Ordinary Differential Equations