We introduce a stable, well tested Python implementation of the affine-invariant ensemble sampler for Markov chain Monte Carlo (MCMC) proposed by Goodman & Weare (2010). The code is open source and has already been used in several published projects in the astrophysics literature. The algorithm behind emcee has several advantages over traditional MCMC sampling methods and it has excellent performance as measured by the autocorrelation time (or function calls per independent sample). One major advantage of the algorithm is that it requires hand-tuning of only 1 or 2 parameters compared to ~N2 for a traditional algorithm in an N-dimensional parameter space. In this document, we describe the algorithm and the details of our implementation. Exploiting the parallelism of the ensemble method, emcee permits any user to take advantage of multiple CPU cores without extra effort. The code is available online at http://dan.iel.fm/emcee under the GNU General Public License v2.

https://iopscience.iop.org/article/10.1086/670067/pdf

emcee: The MCMC Hammer

This book describes the new generation of discrete choice methods, focusing on the many advances that are made possible by simulation. Researchers use these statistical methods to examine the choices that consumers, households, firms, and other agents make. Each of the major models is covered: logit, generalized extreme value, or GEV (including nested and cross-nested logits), probit, and mixed logit, plus a variety of specifications that build on these basics. Simulation-assisted estimation procedures are investigated and compared, including maximum simulated likelihood, method of simulated moments, and method of simulated scores. Procedures for drawing from densities are described, including variance reduction techniques such as anithetics and Halton draws. Recent advances in Bayesian procedures are explored, including the use of the Metropolis-Hastings algorithm and its variant Gibbs sampling. No other book incorporates all these fields, which have arisen in the past 20 years. The procedures are applicable in many fields, including energy, transportation, environmental studies, health, labor, and marketing.

/pdf/discrete-choice-methods-with-simulation-l1x4cghjtt.pdf

Discrete Choice Methods with Simulation

How should ecologists and evolutionary biologists analyze nonnormal data that involve random effects? Nonnormal data such as counts or proportions often defy classical statistical procedures. Generalized linear mixed models (GLMMs) provide a more flexible approach for analyzing nonnormal data when random effects are present. The explosion of research on GLMMs in the last decade has generated considerable uncertainty for practitioners in ecology and evolution. Despite the availability of accurate techniques for estimating GLMM parameters in simple cases, complex GLMMs are challenging to fit and statistical inference such as hypothesis testing remains difficult. We review the use (and misuse) of GLMMs in ecology and evolution, discuss estimation and inference and summarize 'best-practice' data analysis procedures for scientists facing this challenge.

/pdf/generalized-linear-mixed-models-a-practical-guide-for-3gh61pkir0.pdf

Generalized linear mixed models: a practical guide for ecology and evolution

We generalize the method proposed by Gelman and Rubin (1992a) for monitoring the convergence of iterative simulations by comparing between and within variances of multiple chains, in order to obtain a family of tests for convergence. We review methods of inference from simulations in order to develop convergence-monitoring summaries that are relevant for the purposes for which the simulations are used. We recommend applying a battery of tests for mixing based on the comparison of inferences from individual sequences and from the mixture of sequences. Finally, we discuss multivariate analogues, for assessing convergence of several parameters simultaneously.

/pdf/general-methods-for-monitoring-convergence-of-iterative-45g63vyekv.pdf

General methods for monitoring convergence of iterative simulations

Generalized linear mixed models provide a flexible framework for modeling a range of data, although with non-Gaussian response variables the likelihood cannot be obtained in closed form. Markov chain Monte Carlo methods solve this problem by sampling from a series of simpler conditional distributions that can be evaluated. The R package MCMCglmm implements such an algorithm for a range of model fitting problems. More than one response variable can be analyzed simultaneously, and these variables are allowed to follow Gaussian, Poisson, multi(bi)nominal, exponential, zero-inflated and censored distributions. A range of variance structures are permitted for the random effects, including interactions with categorical or continuous variables (i.e., random regression), and more complicated variance structures that arise through shared ancestry, either through a pedigree or through a phylogeny. Missing values are permitted in the response variable(s) and data can be known up to some level of measurement error as in meta-analysis. All simu- lation is done in C/ C++ using the CSparse library for sparse linear systems.

https://cran.r-project.org/web//packages/MCMCglmm/vignettes/Overview.pdf

MCMC Methods for Multi-Response Generalized Linear Mixed Models: The MCMCglmm R Package

A critical issue for users of Markov chain Monte Carlo (MCMC) methods in applications is how to determine when it is safe to stop sampling and use the samples to estimate characteristics of the distribution of interest. Research into methods of computing theoretical convergence bounds holds promise for the future but to date has yielded relatively little of practical use in applied work. Consequently, most MCMC users address the convergence problem by applying diagnostic tools to the output produced by running their samplers. After giving a brief overview of the area, we provide an expository review of 13 convergence diagnostics, describing the theoretical basis and practical implementation of each. We then compare their performance in two simple models and conclude that all of the methods can fail to detect the sorts of convergence failure that they were designed to identify. We thus recommend a combination of strategies aimed at evaluating and accelerating MCMC sampler convergence, including ap...

https://stat.duke.edu/~scs/Courses/Stat376/Papers/ConvergeDiagnostics/CowlesCarlin.1996.pdf

Markov Chain Monte Carlo Convergence Diagnostics: A Comparative Review

The ordinal probit, univariate or multivariate, is a generalized linear model (GLM) structure that arises frequently in such disparate areas of statistical applications as medicine and econometrics. Despite the straightforwardness of its implementation using the Gibbs sampler, the ordinal probit may present challenges in obtaining satisfactory convergence. We present a multivariate Hastings-within-Gibbs update step for generating latent data and bin boundary parameters jointly, instead of individually from their respective full conditionals. When the latent data are parameters of interest, this algorithm substantially improves Gibbs sampler convergence for large datasets. We also discuss Monte Carlo Markov chain (MCMC) implementation of cumulative logit (proportional odds) and cumulative complementary log-log (proportional hazards) models with latent data.

Accelerating Monte Carlo Markov chain convergence for cumulative-link generalized linear models

Poor adherence to medication regimens is a well-documented phenomenon in clinical practice and an ever-present concern in clinical trials. Little is known about adherence to inhaled medication regimens over extended periods. The present paper describes the 2-yr results of the Lung Health Study (LHS) program, which was developed to maintain long-term adherence to an inhaled medication regimen in 3,923 special intervention participants (as measured by self-report and medication canister weight). The LHS is a double-blind, multicenter, randomized controlled clinical trial of smoking intervention and bronchodilator therapy (ipratropium bromide or placebo) for early intervention in chronic obstructive pulmonary disease (COPD). At the first 4-mo follow-up visit, nearly 70% of participants reported satisfactory or better adherence. Over the next 18 mo, self-reported satisfactory or better adherence declined to about 60%. Canister weight classified adherence as satisfactory or better in 72% of participants returning all canisters at 1 yr, and in 70% of the participants returning all canisters at the 2-yr follow-up. Self-reporting confirmed by canister weight classified 48% of participants at 1 yr as showing satisfactory or better adherence. Overusers were 50% more likely than others to misrepresent their true smoking status, suggesting that canister weights indicating overuse may be deceptive. Results of multiple logistic regression analysis indicate that the best compliance was found in participants who were married, older, white, had more severe airways obstruction, less shortness of breath, and fewer hospitalizations, and who had not been confined to bed for respiratory illnesses.(ABSTRACT TRUNCATED AT 250 WORDS)

Long-term metered-dose inhaler adherence in a clinical trial

In the Lung Health Study (LHS), compliance with the use of inhaled medication was assessed at each follow-up visit both by self-report and by weighing the used medication canisters. One or both of these assessments were missing if the participant failed to attend the visit or to return all canisters. Approximately 30% of canister-weight data and 5% to 15% of self-report data were missing at different visits. We use Gibbs sampling with data augmentation and a multivariate Hastings update step to implement a Bayesian hierarchical model for LHS inhaler compliance. Incorporating individual-level random effects to account for correlations among repeated measures on the same participant, our model is a longitudinal extension of the Tobit models used in econometrics to deal with partially unobservable data. It enables (a) assessment of the relationships among visit attendance, canister return, self-reported compliance level, and canister weight compliance, and (b) determination of demographic, physiolog...

Bayesian Tobit Modeling of Longitudinal Ordinal Clinical Trial Compliance Data with Nonignorable Missingness

Markov chain Monte Carlo (MCMC) methods, including the Gibbs sampler and the Metropolis–Hastings algorithm, are very commonly used in Bayesian statistics for sampling from complicated, high-dimensional posterior distributions. A continuing source of uncertainty is how long such a sampler must be run in order to converge approximately to its target stationary distribution. A method has previously been developed to compute rigorous theoretical upper bounds on the number of iterations required to achieve a specified degree of convergence in total variation distance by verifying drift and minorization conditions. We propose the use of auxiliary simulations to estimate the numerical values needed in this theorem. Our simulation method makes it possible to compute quantitative convergence bounds for models for which the requisite analytical computations would be prohibitively difficult or impossible. On the other hand, although our method appears to perform well in our example problems, it cannot provide the guarantees offered by analytical proof.

/pdf/a-simulation-approach-to-convergence-rates-for-markov-chain-tdnjfzkz18.pdf

Mary Kathryn Cowles

Papers

Markov Chain Monte Carlo Convergence Diagnostics: A Comparative Review

Accelerating Monte Carlo Markov chain convergence for cumulative-link generalized linear models

Long-term metered-dose inhaler adherence in a clinical trial

Bayesian Tobit Modeling of Longitudinal Ordinal Clinical Trial Compliance Data with Nonignorable Missingness

A simulation approach to convergence rates for Markov chain Monte Carlo algorithms