Alan E. Gelfand

Bio: Alan E. Gelfand is an academic researcher from Duke University. The author has contributed to research in topics: Bayesian probability & Markov chain Monte Carlo. The author has an hindex of 73, co-authored 354 publications receiving 37619 citations. Previous affiliations of Alan E. Gelfand include Northern Arizona University & University of California, Los Angeles.

Sampling-Based Approaches to Calculating Marginal Densities

Abstract: Stochastic substitution, the Gibbs sampler, and the sampling-importance-resampling algorithm can be viewed as three alternative sampling- (or Monte Carlo-) based approaches to the calculation of numerical estimates of marginal probability distributions. The three approaches will be reviewed, compared, and contrasted in relation to various joint probability structures frequently encountered in applications. In particular, the relevance of the approaches to calculating Bayesian posterior densities for a variety of structured models will be discussed and illustrated.

Sampling-based approaches to calculating marginal densities

Abstract: Stochastic substitution, the Gibbs sampler, and the sampling-importance-resampling algorithm can be viewed as three alternative sampling- (or Monte Carlo-) based approaches to the calculation of numerical estimates of marginal probability distributions. The three approaches will be reviewed, compared, and contrasted in relation to various joint probability structures frequently encountered in applications. In particular, the relevance of the approaches to calculating Bayesian posterior densities for a variety of structured models will be discussed and illustrated.

Hierarchical Modeling and Analysis for Spatial Data

Abstract: OVERVIEW OF SPATIAL DATA PROBLEMS Introduction to Spatial Data and Models Fundamentals of Cartography Exercises BASICS OF POINT-REFERENCED DATA MODELS Elements of Point-Referenced Modeling Spatial Process Models Exploratory Approaches for Point-Referenced Data Classical Spatial Prediction Computer Tutorials Exercises BASICS OF AREAL DATA MODELS Exploratory Approaches for Areal Data Brook's Lemma and Markov Random Fields Conditionally Autoregressive (CAR) Models Simultaneous Autoregressive (SAR) Models Computer Tutorials Exercises BASICS OF BAYESIAN INFERENCE Introduction to Hierarchical Modeling and Bayes Theorem Bayesian Inference Bayesian Computation Computer Tutorials Exercises HIERARCHICAL MODELING FOR UNIVARIATE SPATIAL DATA Stationary Spatial Process Models Generalized Linear Spatial Process Modeling Nonstationary Spatial Process Models Areal Data Models General Linear Areal Data Modeling Exercises SPATIAL MISALIGNMENT Point-Level Modeling Nested Block-Level Modeling Nonnested Block-Level Modeling Misaligned Regression Modeling Exercises MULTIVARIATE SPATIAL MODELING Separable Models Coregionalization Models Other Constructive Approaches Multivariate Models for Areal Data Exercises SPATIOTEMPORAL MODELING General Modeling Formulation Point-Level Modeling with Continuous Time Nonseparable Spatio-Temporal Models Dynamic Spatio-Temporal Models Block-Level Modeling Exercises SPATIAL SURVIVAL MODELS Parametric Models Semiparametric Models Spatio-Temporal Models Multivariate Models Spatial Cure Rate Models Exercises SPECIAL TOPICS IN SPATIAL PROCESS MODELING Process Smoothness Revisited Spatially Varying Coefficient Models Spatial CDFs APPENDICES Matrix Theory and Spatial Computing Methods Answers to Selected Exercises REFERENCES AUTHOR INDEX SUBJECT INDEX Short TOC

Bayesian Model Choice: Asymptotics and Exact Calculations

Abstract: : Model determination is a fundamental data analytic task. Here we consider the problem of choosing amongst a finite (with loss of generality we assume two) set of models. After briefly reviewing classical and Bayesian model choice strategies we present a general predictive density which includes all proposed Bayesian approaches we are aware of. Using Laplace approximations we can conveniently assess and compare asymptotic behavior of these approaches. Concern regarding the accuracy of these approximation for small to moderate sample sizes encourages the use of Monte Carlo techniques to carry out exact calculations. A data set fit with nested non linear models enables comparison between proposals and between exact and asymptotic values.

Gaussian predictive process models for large spatial data sets

Abstract: With scientific data available at geocoded locations, investigators are increasingly turning to spatial process models for carrying out statistical inference. Over the last decade, hierarchical models implemented through Markov chain Monte Carlo methods have become especially popular for spatial modelling, given their flexibility and power to fit models that would be infeasible with classical methods as well as their avoidance of possibly inappropriate asymptotics. However, fitting hierarchical spatial models often involves expensive matrix decompositions whose computational complexity increases in cubic order with the number of spatial locations, rendering such models infeasible for large spatial data sets. This computational burden is exacerbated in multivariate settings with several spatially dependent response variables. It is also aggravated when data are collected at frequent time points and spatiotemporal process models are used. With regard to this challenge, our contribution is to work with what we call predictive process models for spatial and spatiotemporal data. Every spatial (or spatiotemporal) process induces a predictive process model (in fact, arbitrarily many of them). The latter models project process realizations of the former to a lower dimensional subspace, thereby reducing the computational burden. Hence, we achieve the flexibility to accommodate non-stationary, non-Gaussian, possibly multivariate, possibly spatiotemporal processes in the context of large data sets. We discuss attractive theoretical properties of these predictive processes. We also provide a computational template encompassing these diverse settings. Finally, we illustrate the approach with simulated and real data sets.

Inference from Iterative Simulation Using Multiple Sequences

Abstract: The Gibbs sampler, the algorithm of Metropolis and similar iterative simulation methods are potentially very helpful for summarizing multivariate distributions. Used naively, however, iterative simulation can give misleading answers. Our methods are simple and generally applicable to the output of any iterative simulation; they are designed for researchers primarily interested in the science underlying the data and models they are analyzing, rather than for researchers interested in the probability theory underlying the iterative simulations themselves. Our recommended strategy is to use several independent sequences, with starting points sampled from an overdispersed distribution. At each step of the iterative simulation, we obtain, for each univariate estimand of interest, a distributional estimate and an estimate of how much sharper the distributional estimate might become if the simulations were continued indefinitely. Because our focus is on applied inference for Bayesian posterior distributions in real problems, which often tend toward normality after transformations and marginalization, we derive our results as normal-theory approximations to exact Bayesian inference, conditional on the observed simulations. The methods are illustrated on a random-effects mixture model applied to experimental measurements of reaction times of normal and schizophrenic patients.

Bayesian measures of model complexity and fit

Abstract: Summary. We consider the problem of comparing complex hierarchical models in which the number of parameters is not clearly defined. Using an information theoretic argument we derive a measure pD for the effective number of parameters in a model as the difference between the posterior mean of the deviance and the deviance at the posterior means of the parameters of interest. In general pD approximately corresponds to the trace of the product of Fisher's information and the posterior covariance, which in normal models is the trace of the ‘hat’ matrix projecting observations onto fitted values. Its properties in exponential families are explored. The posterior mean deviance is suggested as a Bayesian measure of fit or adequacy, and the contributions of individual observations to the fit and complexity can give rise to a diagnostic plot of deviance residuals against leverages. Adding pD to the posterior mean deviance gives a deviance information criterion for comparing models, which is related to other information criteria and has an approximate decision theoretic justification. The procedure is illustrated in some examples, and comparisons are drawn with alternative Bayesian and classical proposals. Throughout it is emphasized that the quantities required are trivial to compute in a Markov chain Monte Carlo analysis.

Multimodel Inference Understanding AIC and BIC in Model Selection

Abstract: The model selection literature has been generally poor at reflecting the deep foundations of the Akaike information criterion (AIC) and at making appropriate comparisons to the Bayesian information...

Machine Learning : A Probabilistic Perspective

Abstract: Today's Web-enabled deluge of electronic data calls for automated methods of data analysis. Machine learning provides these, developing methods that can automatically detect patterns in data and then use the uncovered patterns to predict future data. This textbook offers a comprehensive and self-contained introduction to the field of machine learning, based on a unified, probabilistic approach. The coverage combines breadth and depth, offering necessary background material on such topics as probability, optimization, and linear algebra as well as discussion of recent developments in the field, including conditional random fields, L1 regularization, and deep learning. The book is written in an informal, accessible style, complete with pseudo-code for the most important algorithms. All topics are copiously illustrated with color images and worked examples drawn from such application domains as biology, text processing, computer vision, and robotics. Rather than providing a cookbook of different heuristic methods, the book stresses a principled model-based approach, often using the language of graphical models to specify models in a concise and intuitive way. Almost all the models described have been implemented in a MATLAB software package--PMTK (probabilistic modeling toolkit)--that is freely available online. The book is suitable for upper-level undergraduates with an introductory-level college math background and beginning graduate students.

Novel approach to nonlinear/non-Gaussian Bayesian state estimation

Abstract: An algorithm, the bootstrap filter, is proposed for implementing recursive Bayesian filters. The required density of the state vector is represented as a set of random samples, which are updated and propagated by the algorithm. The method is not restricted by assumptions of linear- ity or Gaussian noise: it may be applied to any state transition or measurement model. A simula- tion example of the bearings only tracking problem is presented. This simulation includes schemes for improving the efficiency of the basic algorithm. For this example, the performance of the bootstrap filter is greatly superior to the standard extended Kalman filter.