Journal ArticleDOI
Algorithm 799: revolve: an implementation of checkpointing for the reverse or adjoint mode of computational differentiation
Andreas Griewank,Andrea Walther +1 more
Reads0
Chats0
TLDR
This article presents the function revolve, which generates checkpointing schedules that are provably optimal with regard to a primary and a secondary criterion and is intended to be used as an explicit “controller” for running a time-dependent applications program.Abstract:
In its basic form, the reverse mode of computational differentiation yields the gradient of a scalar-valued function at a cost that is a small multiple of the computational work needed to evaluate the function itself. However, the corresponding memory requirement is proportional to the run-time of the evaluation program. Therefore, the practical applicability of the reverse mode in its original formulation is limited despite the availability of ever larger memory systems. This observation leads to the development of checkpointing schedules to reduce the storage requirements. This article presents the function revolve, which generates checkpointing schedules that are provably optimal with regard to a primary and a secondary criterion. This routine is intended to be used as an explicit “controller” for running a time-dependent applications program.read more
Citations
More filters
Patent
Simultaneous source inversion for marine streamer data with cross-correlation objective function
TL;DR: In this paper, a method for simultaneous full-wavefield inversion of gathers of source (or receiver) encoded geophysical data was proposed to determine a physical properties model for a subsurface region, especially suitable for surveys where fixed-receiver geometry conditions were not satisfied in the data acquisition.
Journal ArticleDOI
Automated derivation of the adjoint of high-level transient finite element programs
TL;DR: In this article, a technique for deriving discrete adjoint and tangent linear models of finite element models is presented, which relies on a high-level symbolic representation of the forward problem.
Patent
Simultaneous source encoding and source separation as a practical solution for full wavefield inversion
Partha S. Routh,Sunwoong Lee,Ramesh Neelamani,Jerome R. Krebs,Lazaratos Spyridon,Carey Marcinkovich +5 more
TL;DR: In this article, a method for simultaneous full-wavefield inversion of gathers of source (or receiver) encoded geophysical data to determine a physical properties model for a subsurface region, especially suitable for surveys where fixed receiver geometry conditions were not satisfied in the data acquisition, is presented.
Posted Content
Dynamic Tensor Rematerialization
Marisa Kirisame,Steven Lyubomirsky,Altan Haan,Jennifer Brennan,Mike He,Jared Roesch,Tianqi Chen,Zachary Tatlock +7 more
TL;DR: Dynamic Tensor Rematerialization is presented, a greedy online algorithm for heuristically checkpointing arbitrary models and it is proved it can train an $N$-layer feedforward network on an $\Omega(\sqrt{N})$ memory budget with only $\mathcal{O}(N)$ tensor operations.
Journal ArticleDOI
MultiStage Approaches for Optimal Offline Checkpointing
Philipp Stumm,Andrea Walther +1 more
TL;DR: The write and read counts for each checkpoint in a binomial checkpointing approach are examined and show that checkpointing techniques may reduce the overall computing time despite the required recalculations.
References
More filters
Book
Numerical methods for conservation laws
TL;DR: In this paper, the authors describe the derivation of conservation laws and apply them to linear systems, including the linear advection equation, the Euler equation, and the Riemann problem.
Book
Optimal Control of Systems Governed by Partial Differential Equations
TL;DR: In this paper, the authors consider the problem of minimizing the sum of a differentiable and non-differentiable function in the context of a system governed by a Dirichlet problem.
Book
Evaluating Derivatives: Principles and Techniques of Algorithmic Differentiation
Andreas Griewank,Andrea Walther +1 more
TL;DR: This second edition has been updated and expanded to cover recent developments in applications and theory, including an elegant NP completeness argument by Uwe Naumann and a brief introduction to scarcity, a generalization of sparsity.
Journal ArticleDOI
Upwind difference schemes for hyperbolic systems of conservation laws
Stanley Osher,Fred Solomon +1 more
TL;DR: In this article, a new upwind finite difference approximation to systems of nonlinear hyperbolic conservation laws has been derived. But the scheme has desirable properties for shock calculations, such as unique and sharp shocks.
Journal ArticleDOI
Achieving logarithmic growth of temporal and spatial complexity in reverse automatic differentiation
TL;DR: It is shown here that, by a recursive scheme related to the multilevel differentiation approach of Volin and Ostrovskii, the growth in both temporal and spatial complexity can be limited to a fixed multiple of log(T).
Related Papers (5)
Evaluating Derivatives: Principles and Techniques of Algorithmic Differentiation
Andreas Griewank,Andrea Walther +1 more
Evaluating Derivatives: Principles and Techniques of Algorithmic Differentiation, Second Edition
Andreas Griewank,Andrea Walther +1 more