Stochastic Gradient Descent as Approximate Bayesian Inference
TLDR
It is demonstrated that constant SGD gives rise to a new variational EM algorithm that optimizes hyperparameters in complex probabilistic models and a scalable approximate MCMC algorithm, the Averaged Stochastic Gradient Sampler is proposed.Abstract:
Stochastic Gradient Descent with a constant learning rate (constant SGD) simulates a Markov chain with a stationary distribution. With this perspective, we derive several new results. (1) We show t...read more
Citations
More filters
Journal ArticleDOI
Inferring change points in the spread of COVID-19 reveals the effectiveness of interventions.
Jonas Dehning,Johannes Zierenberg,F. Paul Spitzner,Michael Wibral,Joao Pinheiro Neto,Michael Wilczek,Michael Wilczek,Viola Priesemann,Viola Priesemann +8 more
TL;DR: Modeling and Bayesian inference reveal the time dependence of SARS-CoV-2 interventions on the number of new infections using the example of Germany and the impact of these measures on the disease spread using change point analysis.
Journal ArticleDOI
Deep Multi-Modal Object Detection and Semantic Segmentation for Autonomous Driving: Datasets, Methods, and Challenges
Di Feng,Christian Haase-Schutz,Lars Rosenbaum,Heinz Hertlein,Claudius Gläser,Fabian Timm,Werner Wiesbeck,Klaus Dietmayer +7 more
TL;DR: In this article, the authors systematically summarize methodologies and discuss challenges for deep multi-modal object detection and semantic segmentation in autonomous driving and provide an overview of on-board sensors on test vehicles, open datasets, and background information for object detection.
Posted Content
Deep Ensembles: A Loss Landscape Perspective
TL;DR: Developing the concept of the diversity--accuracy plane, it is shown that the decorrelation power of random initializations is unmatched by popular subspace sampling methods and the experimental results validate the hypothesis that deep ensembles work well under dataset shift.
Journal ArticleDOI
Advances in Variational Inference
TL;DR: Variational inference (VI) as mentioned in this paper approximates a high-dimensional Bayesian posterior with a simpler variational distribution by solving an optimization problem, which has been successfully applied to various models and large-scale applications.
Posted Content
Hands-on Bayesian Neural Networks -- a Tutorial for Deep Learning Users
TL;DR: This tutorial provides deep learning practitioners with an overview of the relevant literature and a complete toolset to design, implement, train, use and evaluate Bayesian neural networks, i.e., stochastic artificial neural networks trained using Bayesian methods.