Algorithm 799: revolve: an implementation of checkpointing for the reverse or adjoint mode of computational differentiation

TLDR: This article presents the function revolve, which generates checkpointing schedules that are provably optimal with regard to a primary and a secondary criterion and is intended to be used as an explicit “controller” for running a time-dependent applications program.

Abstract: In its basic form, the reverse mode of computational differentiation yields the gradient of a scalar-valued function at a cost that is a small multiple of the computational work needed to evaluate the function itself. However, the corresponding memory requirement is proportional to the run-time of the evaluation program. Therefore, the practical applicability of the reverse mode in its original formulation is limited despite the availability of ever larger memory systems. This observation leads to the development of checkpointing schedules to reduce the storage requirements. This article presents the function revolve, which generates checkpointing schedules that are provably optimal with regard to a primary and a secondary criterion. This routine is intended to be used as an explicit “controller” for running a time-dependent applications program.