Efficient calculation of sensitivities for optimization problems

TLDR: The tapeless forward mode of ADOL-C as discussed by the authors enables the joint computation of function and derivative values directly from main memory within one sweep, and shorter runtimes are achieved due to the avoidance of tape handling and a more effective, joint optimization for function and derivatives.

Abstract: Sensitivity information is required by numerous applications such as, for example, optimization algorithms, parameter estimations or real time control. Sensitivities can be computed with working accuracy using the forward mode of automatic differentiation (AD). ADOL-C is an AD-tool for programs written in C or C++. Originally, when applying ADOL-C, tapes for values, operations and locations are written during the function evaluation to generate an internal function representation. Subsequently, these tapes are evaluated to compute the derivatives, sparsity patterns etc., using the forward or reverse mode of AD. The generation of the tapes can be completely avoided by applying the recently implemented tapeless variant of the forward mode for scalar and vector calculations. The tapeless forward mode enables the joint computation of function and derivative values directly from main memory within one sweep. Compared to the original approach shorter runtimes are achieved due to the avoidance of tape handling and a more effective, joint optimization for function and derivative code. Advantages and disadvantages of the tapeless forward mode provided by ADOL-C will be discussed. Furthermore, runtime comparisons for two implemented variants of the tapeless forward mode are presented. The results are based on two numerical examples that require the computation of sensitivity information.