Numerical Optimization presents a comprehensive and up-to-date description of the most effective methods in continuous optimization. It responds to the growing interest in optimization in engineering, science, and business by focusing on the methods that are best suited to practical problems. For this new edition the book has been thoroughly updated throughout. There are new chapters on nonlinear interior methods and derivative-free methods for optimization, both of which are used widely in practice and the focus of much current research. Because of the emphasis on practical methods, as well as the extensive illustrations and exercises, the book is accessible to a wide audience. It can be used as a graduate text in engineering, operations research, mathematics, computer science, and business. It also serves as a handbook for researchers and practitioners in the field. The authors have strived to produce a text that is pleasant to read, informative, and rigorous - one that reveals both the beautiful nature of the discipline and its practical side.

Numerical Optimization

Preface 1. Background in linear algebra 2. Discretization of partial differential equations 3. Sparse matrices 4. Basic iterative methods 5. Projection methods 6. Krylov subspace methods Part I 7. Krylov subspace methods Part II 8. Methods related to the normal equations 9. Preconditioned iterations 10. Preconditioning techniques 11. Parallel implementations 12. Parallel preconditioners 13. Multigrid methods 14. Domain decomposition methods Bibliography Index.

/pdf/iterative-methods-for-sparse-linear-systems-34ozhpwq89.pdf

Iterative Methods for Sparse Linear Systems

We present an iterative method for solving linear systems, which has the property of minimizing at every step the norm of the residual vector over a Krylov subspace. The algorithm is derived from t...

/pdf/gmres-a-generalized-minimal-residual-algorithm-for-solving-4xovqxzqlw.pdf

GMRES: a generalized minimal residual algorithm for solving nonsymmetric linear systems

This article reviews the current status of lattice-dynamical calculations in crystals, using density-functional perturbation theory, with emphasis on the plane-wave pseudopotential method. Several specialized topics are treated, including the implementation for metals, the calculation of the response to macroscopic electric fields and their relevance to long-wavelength vibrations in polar materials, the response to strain deformations, and higher-order responses. The success of this methodology is demonstrated with a number of applications existing in the literature.

/pdf/phonons-and-related-crystal-properties-from-density-3mdybdn8w1.pdf

Phonons and related crystal properties from density-functional perturbation theory

In this book, which focuses on the use of iterative methods for solving large sparse systems of linear equations, templates are introduced to meet the needs of both the traditional user and the high-performance specialist Templates, a description of a general algorithm rather than the executable object or source code more commonly found in a conventional software library, offer whatever degree of customization the user may desire

/pdf/templates-for-the-solution-of-linear-systems-building-blocks-28xlh68wjc.pdf

Templates for the Solution of Linear Systems: Building Blocks for Iterative Methods

We consider a class of iterative algorithms for solving systems of linear equations where the coefficient matrix is nonsymmetric with positive-definite symmetric part. The algorithms are modelled after the conjugate gradient method, and are well suited for large sparse systems. They do not make use of any associated symmetric problems. Convergence results and error bounds are presented.

Variational Iterative Methods for Nonsymmetric Systems of Linear Equations

: Consider the NxN system of linear equations M x = b, where the coefficient matrix M is large, sparse, and nonsymmetric. Assume that M can be factored in the form M = L D U, where L is a lower triangular matrix, D is a diagonal matrix, and U is a unit upper triangular matrix. Such systems arise frequently in scientific computation, e.g., in finite difference and finite element approximations to non-self-adjoint elliptic boundary value problems. This report presents a package of efficient, reliable, well-documented, and portable FORTRAN subroutines for solving these systems.

Yale sparse matrix package I: The symmetric codes

This paper presents a unified formulation of a class of the conjugate gradient-like algorithms for solving nonsymmetric linear systems. The common framework is the Petrov- Galerkin method on Krylov subspaces. We discuss some practical points concerning the methods and point out some of the interrelations between them. 1. Introduction. In the recent few years, a large number of generalizations of the conjugate gradient and conjugate residual methods, which are very successful in solving symmetric positive-definite linear systems, have been proposed for solving nonsymmetric linear systems (3), (6), (5), (7), (12). In this paper we present an abstract framework which includes most of these methods and many new ones. Our goal is to understand the relationships among the methods and to synthesize. Consider the general linear system:

Conjugate gradient-like algorithms for solving nonsymmetric linear systems

In this paper we consider different methods for exchanging data among processors in parallel computers. The most common data exchange operations in parallel numerical methods are examined and different methods for performing them efficiently on each of several ensemble architectures are proposed and analyzed. Our objective is to compare the performance of each particular architecture on each of the data exchange operations.

Martin H. Schultz

Papers

GMRES: a generalized minimal residual algorithm for solving nonsymmetric linear systems

Variational Iterative Methods for Nonsymmetric Systems of Linear Equations

Yale sparse matrix package I: The symmetric codes

Conjugate gradient-like algorithms for solving nonsymmetric linear systems

Data communication in parallel architectures