Probability Distributions.- Linear Models for Regression.- Linear Models for Classification.- Neural Networks.- Kernel Methods.- Sparse Kernel Machines.- Graphical Models.- Mixture Models and EM.- Approximate Inference.- Sampling Methods.- Continuous Latent Variables.- Sequential Data.- Combining Models.

Pattern Recognition and Machine Learning

Initial-value ordinary differential equation solution via variable order Adams method (SIVA/DIVA) package is collection of subroutines for solution of nonstiff ordinary differential equations. There are versions for single-precision and double-precision arithmetic. Requires fewer evaluations of derivatives than other variable-order Adams predictor/ corrector methods. Option for direct integration of second-order equations makes integration of trajectory problems significantly more efficient. Written in FORTRAN 77.

Solving Ordinary Differential Equations

Summary. Markov chain Monte Carlo and sequential Monte Carlo methods have emerged as the two main tools to sample from high dimensional probability distributions. Although asymptotic convergence of Markov chain Monte Carlo algorithms is ensured under weak assumptions, the performance of these algorithms is unreliable when the proposal distributions that are used to explore the space are poorly chosen and/or if highly correlated variables are updated independently. We show here how it is possible to build efficient high dimensional proposal distributions by using sequential Monte Carlo methods. This allows us not only to improve over standard Markov chain Monte Carlo schemes but also to make Bayesian inference feasible for a large class of statistical models where this was not previously so. We demonstrate these algorithms on a non-linear state space model and a Levy-driven stochastic volatility model.

https://rss.onlinelibrary.wiley.com/doi/pdf/10.1111/j.1467-9868.2009.00736.x

Particle Markov chain Monte Carlo methods

Filtering and smoothing methods are used to produce an accurate estimate of the state of a time-varying system based on multiple observational inputs (data). Interest in these methods has exploded in recent years, with numerous applications emerging in fields such as navigation, aerospace engineering, telecommunications and medicine. This compact, informal introduction for graduate students and advanced undergraduates presents the current state-of-the-art filtering and smoothing methods in a unified Bayesian framework. Readers learn what non-linear Kalman filters and particle filters are, how they are related, and their relative advantages and disadvantages. They also discover how state-of-the-art Bayesian parameter estimation methods can be combined with state-of-the-art filtering and smoothing algorithms. The book's practical and algorithmic approach assumes only modest mathematical prerequisites. Examples include MATLAB computations, and the numerous end-of-chapter exercises include computational assignments. MATLAB/GNU Octave source code is available for download at www.cambridge.org/sarkka, promoting hands-on work with the methods.

/pdf/bayesian-filtering-and-smoothing-4w1c72z7m8.pdf

Bayesian Filtering and Smoothing

Noel Cressie and Christopher WikleHardcover: 624 pagesYear: 2011Publisher: John WileyISBN-13: 978-0471692744Here is the new reference book about spatial and spatio-temporal statistical modeling! No...

Statistics for Spatio-Temporal Data

Modern parallel computing devices, such as the graphics processing unit (GPU), have gained significant traction in scientific and statistical computing. They are particularly well-suited to data-parallel algorithms such as the particle filter, or more generally sequential Monte Carlo (SMC), which are increasingly used in statistical inference. SMC methods carry a set of weighted particles through repeated propagation, weighting, and resampling steps. The propagation and weighting steps are straightforward to parallelize, as they require only independent operations on each particle. The resampling step is more difficult, as standard schemes require a collective operation, such as a sum, across particle weights. Focusing on this resampling step, we analyze two alternative schemes that do not involve a collective operation (Metropolis and rejection resamplers), and compare them to standard schemes (multinomial, stratified, and systematic resamplers). We find that, in certain circumstances, the alternative re...

Parallel Resampling in the Particle Filter

LibBi is a software package for state-space modelling and Bayesian inference on modern computer hardware, including multi-core central processing units (CPUs), many-core graphics processing units (GPUs) and distributed-memory clusters of such devices. The software parses a domain-specific language for model specification, then optimises, generates, compiles and runs code for the given model, inference method and hardware platform. In presenting the software, this work serves as an introduction to state-space models and the specialised methods developed for Bayesian inference with them. The focus is on sequential Monte Carlo (SMC) methods such as the particle filter for state estimation, and the particle Markov chain Monte Carlo (PMCMC) and SMC^2 methods for parameter estimation. All are well-suited to current computer hardware. Two examples are given and developed throughout, one a linear three-element windkessel model of the human arterial system, the other a nonlinear Lorenz '96 model. These are specified in the prescribed modelling language, and LibBi demonstrated by performing inference with them. Empirical results are presented, including a performance comparison of the software with different hardware configurations.

Bayesian State-Space Modelling on High-Performance Hardware Using LibBi

The pacific islands of Micronesia have experienced several outbreaks of mosquito-borne diseases over the past decade. In outbreaks on small islands, the susceptible population is usually well defined, and there is no co-circulation of pathogens. Because of this, analysing such outbreaks can be useful for understanding the transmission dynamics of the pathogens involved, and particularly so for yet understudied pathogens such as Zika virus. Here, we compared three outbreaks of dengue and Zika virus in two different island settings in Micronesia, the Yap Main Islands and Fais, using a mathematical model of transmission dynamics and making full use of commonalities in disease and setting between the outbreaks. We found that the estimated reproduction numbers for Zika and dengue were similar when considered in the same setting, but that, conversely, reproduction number for the same disease can vary considerably by setting. On the Yap Main Islands, we estimated a reproduction number of 8.0-16 (95% Credible Interval (CI)) for the dengue outbreak and 4.8-14 (95% CI) for the Zika outbreak, whereas for the dengue outbreak on Fais our estimate was 28-102 (95% CI). We further found that the proportion of cases of Zika reported was smaller (95% CI 1.4%-1.9%) than that of dengue (95% CI: 47%-61%). We confirmed these results in extensive sensitivity analysis. They suggest that models for dengue transmission can be useful for estimating the predicted dynamics of Zika transmission, but care must be taken when extrapolating findings from one setting to another.

/pdf/comparative-analysis-of-dengue-and-zika-outbreaks-reveals-563z3t2242.pdf

Comparative Analysis of Dengue and Zika Outbreaks Reveals Differences by Setting and Virus.

We consider the use of commodity graphics processing units (GPUs) for the common task of numerically integrating ordinary differential equations (ODEs), achieving speedups of up to 115-fold over comparable serial CPU implementations, and 15-fold over multithreaded CPU code with SIMD intrinsics. Using Lorenz '96 models as a case study, single and double precision benchmarks are established for both the widely used DOPRI5 method and computationally tailored low-storage RK4(3)5[2R+]C. A range of configurations are assessed on each, including multithreading and SIMD intrinsics on the CPU, and GPU kernels parallelized over both the dimensionality of the ODE system and number of trajectories. On the GPU, we draw particular attention to the problem of variable task-length among threads of the same warp, proposing a lightweight strategy of assigning multiple data items to each thread to reduce the prevalence of redundant operations. A simple analysis suggests that the strategy can draw performance close to that of ideal parallelism, while empirical results demonstrate up to a 10 percent improvement over the standard approach.

GPU Acceleration of Runge-Kutta Integrators

In modern applications, statisticians are faced with integrating heterogeneous data modalities relevant for an inference, prediction, or decision problem. In such circumstances, it is convenient to use a graphical model to represent the statistical dependencies, via a set of connected "modules", each relating to a specific data modality, and drawing on specific domain expertise in their development. In principle, given data, the conventional statistical update then allows for coherent uncertainty quantification and information propagation through and across the modules. However, misspecification of any module can contaminate the estimate and update of others, often in unpredictable ways. In various settings, particularly when certain modules are trusted more than others, practitioners have preferred to avoid learning with the full model in favor of approaches that restrict the information propagation between modules, for example by restricting propagation to only particular directions along the edges of the graph. In this article, we investigate why these modular approaches might be preferable to the full model in misspecified settings. We propose principled criteria to choose between modular and full-model approaches. The question arises in many applied settings, including large stochastic dynamical systems, meta-analysis, epidemiological models, air pollution models, pharmacokinetics-pharmacodynamics, and causal inference with propensity scores.

Lawrence M. Murray

Papers

Parallel Resampling in the Particle Filter

Bayesian State-Space Modelling on High-Performance Hardware Using LibBi

Comparative Analysis of Dengue and Zika Outbreaks Reveals Differences by Setting and Virus.

GPU Acceleration of Runge-Kutta Integrators

Better together? Statistical learning in models made of modules