Showing papers by "Andrea Walther published in 2000"

Algorithm 799: revolve: an implementation of checkpointing for the reverse or adjoint mode of computational differentiation

Abstract: In its basic form, the reverse mode of computational differentiation yields the gradient of a scalar-valued function at a cost that is a small multiple of the computational work needed to evaluate the function itself. However, the corresponding memory requirement is proportional to the run-time of the evaluation program. Therefore, the practical applicability of the reverse mode in its original formulation is limited despite the availability of ever larger memory systems. This observation leads to the development of checkpointing schedules to reduce the storage requirements. This article presents the function revolve, which generates checkpointing schedules that are provably optimal with regard to a primary and a secondary criterion. This routine is intended to be used as an explicit “controller” for running a time-dependent applications program.

Evaluating higher derivative tensors by forward propagation of univariate Taylor series

Abstract: This article considers the problem of evaluating all pure and mixed partial derivatives of some vector function defined by an evaluation procedure. The natural approach to evaluating derivative tensors might appear to be their recursive calculation in the usual forward mode of computational differentiation. However, with the approach presented in this article, much simpler data access patterns and similar or lower computational counts can be achieved through propagating a family of univariate Taylor series of a suitable degree. It is applicable for arbitrary orders of derivatives. Also it is possible to calculate derivatives only in some directions instead of the full derivative tensor. Explicit formulas for all tensor entries as well as estimates for the corresponding computational complexities are given.

New results on program reversals

Abstract: For adjoint calculations, parameter estimation, and similar purposes, one may need to produce all qualities calculated during the execution of a computer program in reverse order. The simplest possible approach is to record a complete execution log and then to read it backwards. This may require massive amounts of storage. Instead one may generate the execution log piecewise by restarting the "forward" calculation repeatedly from suitably placed checkpoints. For such-program execution reversals we present parallel reversal schedules that are probably optimal with regards to the number of concurrent processes and the total amount of memory required.

Differentiation methods for industrial strength problems

Abstract: The importance of simulation has been growing in industrial production for many years. Because of reduced product cycles, new and more complicated computer models have to be developed more quickly. The correct description and implementation of the interaction between different components of the entire simulation model as well as the nonlinear behaviour of these components lead in many cases to the need for derivative information. In this chapter, we use software packages for automatic differentiation (AD) in three real world simulation systems typical for a wide range of tasks that have to be solved in numerous industrial applications. We consider difficulties arising from particular aspects of the modelling such as the integration of ordinary differential equations or fixed-point iterations for the solution of equations. Furthermore, we discuss challenges caused by technical software issues such as inhomogeneous source codes written in different languages or table lookups. Several results concerning the use of tools such as ADIFOR, Odyssee, and ADOL-C are presented. We discuss the benefits and the difficulties of current AD techniques applied to real industrial codes. Finally, we outline possible future developments.

Application of techniques of computational differentiation to a cooling system

Abstract: Mathematical functions are often evaluated by computer programs. Using the technique of computational differentiation (CD), one can determine in addition exact gradients of these functions. This article considers two software tools implementing the forward and the reverse mode of CD, namely ADOL-C and FADBAD. Then the model of a cooling system within steel manufacturing is described. The gradient of the corresponding function is calculated by CD applying both software tools separately and alternatively, by divided differences. After a short account of our experience installing and running the software tools the results obtained are presented. A comparison of the run-time needed by CD to the run-time needed by the divided difference method is given

Optimal program execution reversal

Abstract: For adjoint calculations, debugging, and similar purposes one may need to reverse the execution of a computer program. The simplest option of recording a complete execution log and then reading it backwards requires massive amounts of storage. Instead one may generate the execution log piecewise by restarting the ``forward'' calculation repeatedly from suitably placed checkpoints. Our goal is to minimize the temporal and spatial complexity as measured by the number of evaluation repeats and the number of checkpoints, respectively. We present optimal checkpointing schedules for one-step and multi-step evolutions. These might arise for example as discretizations of ODEs by Euler's methods or multi-step schemes, respectively. Furthermore, we present parallel extensions, where auxiliary processors perform the repeated forward evaluations such that one processor can run backward without any interruption. For either case the length of the evolution that can be reversed is shown to grow exponentially with the number of checkpoints and either the number of repetitions or the number of processors.