In this article, we describe an automatic differentiation module of PyTorch — a library designed to enable rapid research on machine learning models. It builds upon a few projects, most notably Lua Torch, Chainer, and HIPS Autograd [4], and provides a high performance environment with easy access to automatic differentiation of models executed on different devices (CPU and GPU). To make prototyping easier, PyTorch does not follow the symbolic approach used in many other deep learning frameworks, but focuses on differentiation of purely imperative programs, with a focus on extensibility and low overhead. Note that this preprint is a draft of certain sections from an upcoming paper covering all PyTorch features.

/pdf/automatic-differentiation-in-pytorch-gbuafkf7qo.pdf

Automatic differentiation in PyTorch

Deposits of clastic carbonate-dominated (calciclastic) sedimentary slope systems in the rock record have been identified mostly as linearly-consistent carbonate apron deposits, even though most ancient clastic carbonate slope deposits fit the submarine fan systems better. Calciclastic submarine fans are consequently rarely described and are poorly understood. Subsequently, very little is known especially in mud-dominated calciclastic submarine fan systems. Presented in this study are a sedimentological core and petrographic characterisation of samples from eleven boreholes from the Lower Carboniferous of Bowland Basin (Northwest England) that reveals a >250 m thick calciturbidite complex deposited in a calciclastic submarine fan setting. Seven facies are recognised from core and thin section characterisation and are grouped into three carbonate turbidite sequences. They include: 1) Calciturbidites, comprising mostly of highto low-density, wavy-laminated bioclast-rich facies; 2) low-density densite mudstones which are characterised by planar laminated and unlaminated muddominated facies; and 3) Calcidebrites which are muddy or hyper-concentrated debrisflow deposits occurring as poorly-sorted, chaotic, mud-supported floatstones. These

Phd by thesis

The brms package implements Bayesian multilevel models in R using the probabilistic programming language Stan. A wide range of distributions and link functions are supported, allowing users to fit - among others - linear, robust linear, binomial, Poisson, survival, ordinal, zero-inflated, hurdle, and even non-linear models all in a multilevel context. Further modeling options include autocorrelation of the response variable, user defined covariance structures, censored data, as well as meta-analytic standard errors. Prior specifications are flexible and explicitly encourage users to apply prior distributions that actually reflect their beliefs. In addition, model fit can easily be assessed and compared with the Watanabe-Akaike information criterion and leave-one-out cross-validation.

brms: An R Package for Bayesian Multilevel Models Using Stan

Structured additive regression models are perhaps the most commonly used class of models in statistical applications. It includes, among others, (generalized) linear models, (generalized) additive models, smoothing spline models, state space models, semiparametric regression, spatial and spatiotemporal models, log-Gaussian Cox processes and geostatistical and geoadditive models. We consider approximate Bayesian inference in a popular subset of structured additive regression models, latent Gaussian models, where the latent field is Gaussian, controlled by a few hyperparameters and with non-Gaussian response variables. The posterior marginals are not available in closed form owing to the non-Gaussian response variables. For such models, Markov chain Monte Carlo methods can be implemented, but they are not without problems, in terms of both convergence and computational time. In some practical applications, the extent of these problems is such that Markov chain Monte Carlo sampling is simply not an appropriate tool for routine analysis. We show that, by using an integrated nested Laplace approximation and its simplified version, we can directly compute very accurate approximations to the posterior marginals. The main benefit of these approximations is computational: where Markov chain Monte Carlo algorithms need hours or days to run, our approximations provide more precise estimates in seconds or minutes. Another advantage with our approach is its generality, which makes it possible to perform Bayesian analysis in an automatic, streamlined way, and to compute model comparison criteria and various predictive measures so that models can be compared and the model under study can be challenged.

/pdf/approximate-bayesian-inference-for-latent-gaussian-models-by-2m3x22eic2.pdf

Approximate Bayesian inference for latent Gaussian models by using integrated nested Laplace approximations

We consider prediction and uncertainty analysis for systems which are approximated using complex mathematical models. Such models, implemented as computer codes, are often generic in the sense that by a suitable choice of some of the model's input parameters the code can be used to predict the behaviour of the system in a variety of specific applications. However, in any specific application the values of necessary parameters may be unknown. In this case, physical observations of the system in the specific context are used to learn about the unknown parameters. The process of fitting the model to the observed data by adjusting the parameters is known as calibration. Calibration is typically effected by ad hoc fitting, and after calibration the model is used, with the fitted input values, to predict the future behaviour of the system. We present a Bayesian calibration technique which improves on this traditional approach in two respects. First, the predictions allow for all sources of uncertainty, including the remaining uncertainty over the fitted parameters. Second, they attempt to correct for any inadequacy of the model which is revealed by a discrepancy between the observed data and the model predictions from even the best-fitting parameter values. The method is illustrated by using data from a nuclear radiation release at Tomsk, and from a more complex simulated nuclear accident exercise.

https://www.jstor.org/stable/pdfplus/2680584.pdf

Bayesian Calibration of computer models

A complete characterisation of piezo ceramic materials typically requires several differently processed specimen. Since the results of all these measurements are combined to form one single set of material parameters, inconsistencies occur naturally. The problem when considering only one specimen is the lack of sensitivity for several material parameters. Therefore, a triple-ring-electrode set-up is presented that is optimised to balance the sensitivity of the frequency dependent impedance for the full set of material parameters. This specific set-up is designed to characterise the material properties using only one specimen, hence reaching consistent material parameters while still aiming to be sufficiently sensitive to each parameter. This more complex set-up calls for a FEM-based inverse problem, that makes it possible to find parameters that fit the measured impedance. The whole optimisation procedure is described exemplarily for one specimen of PIC255 (PI ceramic, Lederhose, Germany) leading to feasi...

Material parameter determination of a piezoelectric disc with triple-ring-electrodes for increased sensitivity

Combinatorial Problems in Algorithmic Differentiation

A gradient-based optimisation of a parametric CAD model requires the calculation of shape sensitivities, i.e., the derivatives of surface points with respect to the design parameters. Typically, this information is not available within a CAD system and can be obtained by applying Automatic Differentiation (AD) to the CAD sources. This paper demonstrates the differentiated open-source CAD kernel OpenCascade Technology (OCCT) using the AD software tool ADOL-C (Automatic Differentiation by OverLoading in C++) for the optimisation of pressure loss in a U-bend pipe. The U-bend geometry is parametrised in OCCT and its derivatives are used in CFD optimisation loops.

Shape Optimisation With Differentiated Cad-Kernel For U-Bend Testcase

The progress in numerical methods and simulation tools promotes the use of inverse problems in material characterisation problems. A newly developed procedure can be used to identify the behaviour of piezoceramic discs over a wide frequency range using a single specimen via fitting simulated and measured impedances by optimising the underlying material parameters. Since there is no generally accepted damping model for piezoelectric ceramics, several mechanical damping models are examined for the material identification. Three models have been chosen and their ability to replicate the measured impedances is evaluated. On the one hand, the common Rayleigh model is considered as a reference. On the other hand, a Zener model and a model using complex constants are extended to model the transversely isotropic material. As the Rayleigh model is only valid for a limited frequency range, it fails to model the broadband behaviour of the material. The model using complex constants leads to the best fit over a wide frequency range while at the same time only adding three additional parameters for modelling damping. Thus, damping can be assumed approximately frequency-independent in piezoceramics.

Modelling damping in piezoceramics: A comparative study

For numerous applications, the computation and provision of exact derivative information plays an important role for optimizing the considered system. This paper introduces the technique of algorithmic differentiation, a method to compute derivatives of arbitrary order within working precision. This derivative information will be combined with a calculus-based optimization algorithm to optimize a non-trivially shaped laser pulse which coherently steers the electron dynamics in a semiconductor quantum wire. Numerical results illustrating the cost for the derivative computation and the optimization process are presented and discussed.

Andrea Walther

Papers

Material parameter determination of a piezoelectric disc with triple-ring-electrodes for increased sensitivity

Combinatorial Problems in Algorithmic Differentiation

Shape Optimisation With Differentiated Cad-Kernel For U-Bend Testcase

Modelling damping in piezoceramics: A comparative study

Calculus-based optimization of the electron dynamics in nanostructures