Showing papers in "Statistical Science in 2013"

MCMC Methods for Functions: Modifying Old Algorithms to Make Them Faster

Abstract: Many problems arising in applications result in the need to probe a probability distribution for functions. Examples include Bayesian nonparametric statistics and conditioned diffusion processes. Standard MCMC algorithms typically become arbitrarily slow under the mesh refinement dictated by nonparametric description of the unknown function. We describe an approach to modifying a whole range of MCMC methods, applicable whenever the target measure has density with respect to a Gaussian process or Gaussian random field reference measure, which ensures that their speed of convergence is robust under mesh refinement. Gaussian processes or random fields are fields whose marginal distributions, when evaluated at any finite set of NNpoints, are ℝ^N-valued Gaussians. The algorithmic approach that we describe is applicable not only when the desired probability measure has density with respect to a Gaussian process or Gaussian random field reference measure, but also to some useful non-Gaussian reference measures constructed through random truncation. In the applications of interest the data is often sparse and the prior specification is an essential part of the overall modelling strategy. These Gaussian-based reference measures are a very flexible modelling tool, finding wide-ranging application. Examples are shown in density estimation, data assimilation in fluid mechanics, subsurface geophysics and image registration. The key design principle is to formulate the MCMC method so that it is, in principle, applicable for functions; this may be achieved by use of proposals based on carefully chosen time-discretizations of stochastic dynamical systems which exactly preserve the Gaussian reference measure. Taking this approach leads to many new algorithms which can be implemented via minor modification of existing algorithms, yet which show enormous speed-up on a wide range of applied problems.

A comparative review of dimension reduction methods in approximate Bayesian computation

Abstract: Approximate Bayesian computation (ABC) methods make use of comparisons between simulated and observed summary statistics to overcome the problem of computationally intractable likelihood functions. As the practical implementation of ABC requires computations based on vectors of summary statistics, rather than full data sets, a central question is how to derive low-dimensional summary statistics from the observed data with minimal loss of information. In this article we provide a comprehensive review and comparison of the performance of the principal methods of dimension reduction proposed in the ABC literature. The methods are split into three nonmutually exclusive classes consisting of best subset selection methods, projection techniques and regularization. In addition, we introduce two new methods of dimension reduction. The first is a best subset selection method based on Akaike and Bayesian information criteria, and the second uses ridge regression as a regularization procedure. We illustrate the performance of these dimension reduction techniques through the analysis of three challenging models and data sets.

New Important Developments in Small Area Estimation

Abstract: The problem of small area estimation (SAE) is how to produce reliable estimates of characteristics of interest such as means, counts, quantiles, etc., for areas or domains for which only small samples or no samples are available, and how to assess their precision. The purpose of this paper is to review and discuss some of the new important developments in small area estimation methods. Rao (2003) wrote a very comprehensive book, which covers all the main developments in this topic until that time. A few review papers have been written after 2003 but they are limited in scope. Hence, the focus of this review is on new developments in the last 7-8 years but to make the review more self-contained, I also mention shortly some of the older developments. The review covers both design-based and model-dependent methods, with the latter methods further classified into frequentist and Bayesian methods. The style of the paper is similar to the style of my previous review on SAE published in 2002, explaining the new problems investigated and describing the proposed solutions, but without dwelling on theoretical details, which can be found in the original articles. I hope that this paper will be useful both to researchers who like to learn more on the research carried out in SAE and to practitioners who might be interested in the application of the new methods.

Wind Energy: Forecasting Challenges for Its Operational Management

Abstract: Renewable energy sources, especially wind energy, are to play a larger role in providing electricity to industrial and domestic consumers. This is already the case today for a number of European countries, closely followed by the US and high growth countries, for example, Brazil, India and China. There exist a number of technological, environmental and political challenges linked to supplementing existing electricity generation capacities with wind energy. Here, mathematicians and statisticians could make a substantial contribution at the interface of meteorology and decision-making, in connection with the generation of forecasts tailored to the various operational decision problems involved. Indeed, while wind energy may be seen as an environmentally friendly source of energy, full benefits from its usage can only be obtained if one is able to accommodate its variability and limited predictability. Based on a short presentation of its physical basics, the importance of considering wind power generation as a stochastic process is motivated. After describing representative operational decision-making problems for both market participants and system operators, it is underlined that forecasts should be issued in a probabilistic framework. Even though, eventually, the forecaster may only communicate single-valued predictions. The existing approaches to wind power forecasting are subsequently described, with focus on single-valued predictions, predictive marginal densities and space–time trajectories. Upcoming challenges related to generating improved and new types of forecasts, as well as their verification and value to forecast users, are finally discussed.

Spatial and spatio-temporal Log-Gaussian Cox processes:extending the geostatistical paradigm

Abstract: In this paper we first describe the class of log-Gaussian Cox processes (LGCPs) as models for spatial and spatio-temporal point process data. We discuss inference, with a particular focus on the computational challenges of likelihood-based inference. We then demonstrate the usefulness of the LGCP by describing four applications: estimating the intensity surface of a spatial point process; investigating spatial segregation in a multi-type process; constructing spatially continuous maps of disease risk from spatially discrete data; and real-time health surveillance. We argue that problems of this kind fit naturally into the realm of geostatistics, which traditionally is defined as the study of spatially continuous processes using spatially discrete observations at a finite number of locations. We suggest that a more useful definition of geostatistics is by the class of scientific problems that it addresses, rather than by particular models or data formats.

Model Selection in Linear Mixed Models

Abstract: Linear mixed effects models are highly flexible in handling a broad range of data types and are therefore widely used in applications. A key part in the analysis of data is model selection, which often aims to choose a parsimonious model with other desirable properties from a possibly very large set of candidate statistical models. Over the last 5–10 years the literature on model selection in linear mixed models has grown extremely rapidly. The problem is much more complicated than in linear regression because selection on the covariance structure is not straightforward due to computational issues and boundary problems arising from positive semidefinite constraints on covariance matrices. To obtain a better understanding of the available methods, their properties and the relationships between them, we review a large body of literature on linear mixed model selection. We arrange, implement, discuss and compare model selection methods based on four major approaches: information criteria such as AIC or BIC, shrinkage methods based on penalized loss functions such as LASSO, the Fence procedure and Bayesian techniques.

Uncertainty Quantification in Complex Simulation Models Using Ensemble Copula Coupling

Abstract: Critical decisions frequently rely on high-dimensional output from complex computer simulation models that show intricate cross-variable, spatial and temporal dependence structures, with weather and climate predictions being key examples. There is a strongly increasing recognition of the need for uncertainty quantification in such settings, for which we propose and review a general multi-stage procedure called ensemble copula coupling (ECC), proceeding as follows: 1. Generate a raw ensemble, consisting of multiple runs of the computer model that differ in the inputs or model parameters in suitable ways. 2. Apply statistical postprocessing techniques, such as Bayesian model averaging or nonhomogeneous regression, to correct for systematic errors in the raw ensemble, to obtain calibrated and sharp predictive distributions for each univariate output variable individually. 3. Draw a sample from each postprocessed predictive distribution. 4. Rearrange the sampled values in the rank order structure of the raw ensemble to obtain the ECC postprocessed ensemble. The use of ensembles and statistical postprocessing have become routine in weather forecasting over the past decade. We show that seemingly unrelated, recent advances can be interpreted, fused and consolidated within the framework of ECC, the common thread being the adoption of the empirical copula of the raw ensemble. Depending on the use of Quantiles, Random draws or Transformations at the sampling stage, we distinguish the ECC-Q, ECC-R and ECC-T variants, respectively. We also describe relations to the Schaake shuffle and extant copula-based techniques. In a case study, the ECC approach is applied to predictions of temperature, pressure, precipitation and wind over Germany, based on the 50-member European Centre for Medium-Range Weather Forecasts (ECMWF) ensemble.

What Is Meant by "Missing at Random"?

Abstract: The concept of missing at random is central in the literature on statistical analysis with missing data. In general, inference using incomplete data should be based not only on observed data values but should also take account of the pattern of missing values. However, it is often said that if data are missing at random, valid inference using likelihood approaches (including Bayesian) can be obtained ignoring the missingness mechanism. Unfortunately, the term “missing at random” has been used inconsistently and not always clearly; there has also been a lack of clarity around the meaning of “valid inference using likelihood”. These issues have created potential for confusion about the exact conditions under which the missingness mechanism can be ignored, and perhaps fed confusion around the meaning of “analysis ignoring the missingness mechanism”. Here we provide standardised precise definitions of “missing at random” and “missing completely at random”, in order to promote unification of the theory. Using these definitions we clarify the conditions that suffice for “valid inference” to be obtained under a variety of inferential paradigms.

Elliptical Insights: Understanding Statistical Methods through Elliptical Geometry

Abstract: Visual insights into a wide variety of statistical methods, for both didactic and data analytic purposes, can often be achieved through geometric diagrams and geometrically based statistical graphs. This paper extols and illustrates the virtues of the ellipse and her higher-dimensional cousins for both these purposes in a variety of contexts, including linear models, multivariate linear models and mixed-effect models. We emphasize the strong relationships among statistical methods, matrix-algebraic solutions and geometry that can often be easily understood in terms of ellipses.

MCMC for Normalized Random Measure Mixture Models

Abstract: This paper concerns the use of Markov chain Monte Carlo methods for posterior sampling in Bayesian nonparametric mixture models with normalized random measure priors. Making use of some recent posterior characterizations for the class of normalized random measures, we propose novel Markov chain Monte Carlo methods of both marginal type and conditional type. The proposed marginal samplers are generalizations of Neal’s well-regarded Algorithm 8 for Dirichlet process mixture models, whereas the conditional sampler is a variation of those recently introduced in the literature. For both the marginal and conditional methods, we consider as a running example a mixture model with an underlying normalized generalized Gamma process prior, and describe comparative simulation results demonstrating the efficacies of the proposed methods.

Lookahead Strategies for Sequential Monte Carlo

Abstract: Based on the principles of importance sampling and resampling, sequential Monte Carlo (SMC) encompasses a large set of powerful techniques dealing with complex stochastic dynamic systems. Many of these systems possess strong memory, with which future information can help sharpen the inference about the current state. By providing theoretical justification of several existing algorithms and introducing several new ones, we study systematically how to construct efficient SMC algorithms to take advantage of the “future” information without creating a substantially high computational burden. The main idea is to allow for lookahead in the Monte Carlo process so that future information can be utilized in weighting and generating Monte Carlo samples, or resampling from samples of the current state.

Wildfire Prediction to Inform Fire Management: Statistical Science Challenges

Abstract: Wildfire is an important system process of the earth that occurs across a wide range of spatial and temporal scales. A variety of methods have been used to predict wildfire phenomena during the past century to better our understanding of fire processes and to inform fire and land management decision-making. Statistical methods have an important role in wildfire prediction due to the inherent stochastic nature of fire phenomena at all scales. Predictive models have exploited several sources of data describing fire phenomena. Experimental data are scarce; observational data are dominated by statistics compiled by government fire management agencies, primarily for administrative purposes and increasingly from remote sensing observations. Fires are rare events at many scales. The data describing fire phenomena can be zero-heavy and nonstationary over both space and time. Users of fire modeling methodologies are mainly fire management agencies often working under great time constraints, thus, complex models have to be efficiently estimated. We focus on providing an understanding of some of the information needed for fire management decision-making and of the challenges involved in predicting fire occurrence, growth and frequency at regional, national and global scales.

Handling Attrition in Longitudinal Studies: The Case for Refreshment Samples

Abstract: Panel studies typically suffer from attrition, which reduces sample size and can result in biased inferences. It is impossible to know whether or not the attrition causes bias from the observed panel data alone. Refreshment samples—new, randomly sampled respondents given the questionnaire at the same time as a subsequent wave of the panel—offer information that can be used to diagnose and adjust for bias due to attrition. We review and bolster the case for the use of refreshment samples in panel studies. We include examples of both a fully Bayesian approach for analyzing the concatenated panel and refreshment data, and a multiple imputation approach for analyzing only the original panel. For the latter, we document a positive bias in the usual multiple imputation variance estimator. We present models appropriate for three waves and two refreshment samples, including nonterminal attrition. We illustrate the three-wave analysis using the 2007–2008 Associated Press–Yahoo! News Election Poll.

Modeling with Normalized Random Measure Mixture Models

Abstract: The Dirichlet process mixture model and more general mixtures based on discrete random probability measures have been shown to be flexible and accurate models for density estimation and clustering. The goal of this paper is to illustrate the use of normalized random measures as mixing measures in nonparametric hierarchical mixture models and point out how possible computational issues can be successfully addressed. To this end, we first provide a concise and accessible introduction to normalized random measures with independent increments. Then, we explain in detail a particular way of sampling from the posterior using the Ferguson–Klass representation. We develop a thorough comparative analysis for location-scale mixtures that considers a set of alternatives for the mixture kernel and for the nonparametric component. Simulation results indicate that normalized random measure mixtures potentially represent a valid default choice for density estimation problems. As a byproduct of this study an R package to fit these models was produced and is available in the Comprehensive R Archive Network (CRAN).

Variational Inference for Generalized Linear Mixed Models Using Partially Noncentered Parametrizations

Abstract: The effects of different parametrizations on the convergence of Bayesian computational algorithms for hierarchical models are well explored. Techniques such as centering, noncentering and partial noncentering can be used to accelerate convergence in MCMC and EM algorithms but are still not well studied for variational Bayes (VB) methods. As a fast deterministic approach to posterior approximation, VB is attracting increasing interest due to its suitability for large high-dimensional data. Use of different parametrizations for VB has not only computational but also statistical implications, as different parametrizations are associated with different factorized posterior approximations. We examine the use of partially noncentered parametrizations in VB for generalized linear mixed models (GLMMs). Our paper makes four contributions. First, we show how to implement an algorithm called nonconjugate variational message passing for GLMMs. Second, we show that the partially noncentered parametrization can adapt to the quantity of information in the data and determine a parametrization close to optimal. Third, we show that partial noncentering can accelerate convergence and produce more accurate posterior approximations than centering or noncentering. Finally, we demonstrate how the variational lower bound, produced as part of the computation, can be useful for model selection.

Assessment of Point Process Models for Earthquake Forecasting

Abstract: Models for forecasting earthquakes are currently tested prospectively in well-organized testing centers, using data collected after the models and their parameters are completely specified. The extent to which these models agree with the data is typically assessed using a variety of numerical tests, which unfortunately have low power and may be misleading for model comparison purposes. Promising alternatives exist, especially residual methods such as super-thinning and Voronoi residuals. This article reviews some of these tests and residual methods for determining the goodness of fit of earthquake forecasting models.

Defining Predictive Probability Functions for Species Sampling Models.

Abstract: We review the class of species sampling models (SSM). In particular, we investigate the relation between the exchangeable partition probability function (EPPF) and the predictive probability function (PPF). It is straightforward to define a PPF from an EPPF, but the converse is not necessarily true. In this paper we introduce the notion of putative PPFs and show novel conditions for a putative PPF to define an EPPF. We show that all possible PPFs in a certain class have to define (unnormalized) probabilities for cluster membership that are linear in cluster size. We give a new necessary and sufficient condition for arbitrary putative PPFs to define an EPPF. Finally, we show posterior inference for a large class of SSMs with a PPF that is not linear in cluster size and discuss a numerical method to derive its PPF.

Component-Wise Markov Chain Monte Carlo: Uniform and Geometric Ergodicity under Mixing and Composition

Abstract: It is common practice in Markov chain Monte Carlo to update the simulation one variable (or sub-block of variables) at a time, rather than conduct a single full-dimensional update. When it is possible to draw from each full-conditional distribution associated with the target this is just a Gibbs sampler. Often at least one of the Gibbs updates is replaced with a Metropolis-Hastings step, yielding a Metropolis-Hastings-within-Gibbs al- gorithm. Strategies for combining component-wise updates include compo- sition, random sequence and random scans. While these strategies can ease MCMC implementation and produce superior empirical performance com- pared to full-dimensional updates, the theoretical convergence properties of the associated Markov chains have received limited attention. We present conditions under which some component-wise Markov chains converge to the stationary distribution at a geometric rate. We pay particular attention to the connections between the convergence rates of the various component- wise strategies. This is important since it ensures the existence of tools that an MCMC practitioner can use to be as confident in the simulation results as if they were based on independent and identically distributed samples. We illustrate our results in two examples including a hierarchical linear mixed model and one involving maximum likelihood estimation for mixed models.

Cluster and Feature Modeling from Combinatorial Stochastic Processes

Abstract: One of the focal points of the modern literature on Bayesian nonparametrics has been the problem of clustering, or partitioning, where each data point is modeled as being associated with one and only one of some collection of groups called clusters or partition blocks. Underlying these Bayesian nonparametric models are a set of interrelated stochastic processes, most notably the Dirichlet process and the Chinese restaurant process. In this paper we provide a formal development of an analogous problem, called feature modeling, for associating data points with arbitrary nonnegative integer numbers of groups, now called features or topics. We review the existing combinatorial stochastic process representations for the clustering problem and develop analogous representations for the feature modeling problem. These representations include the beta process and the Indian buffet process as well as new representations that provide insight into the connections between these processes. We thereby bring the same level of completeness to the treatment of Bayesian nonparametric feature modeling that has previously been achieved for Bayesian nonparametric clustering.

Modern Statistical Methods in Oceanography: A Hierarchical Perspective

Abstract: Processes in ocean physics, air–sea interaction and ocean biogeochemistry span enormous ranges in spatial and temporal scales, that is, from molecular to planetary and from seconds to millennia. Identifying and implementing sustainable human practices depend critically on our understandings of key aspects of ocean physics and ecology within these scale ranges. The set of all ocean data is distorted such that three- and four-dimensional (i.e., time-dependent) in situ data are very sparse, while observations of surface and upper ocean properties from space-borne platforms have become abundant in the past few decades. Precisions in observations of all types vary as well. In the face of these challenges, the interface between Statistics and Oceanography has proven to be a fruitful area for research and the development of useful models. With the recognition of the key importance of identifying, quantifying and managing uncertainty in data and models of ocean processes, a hierarchical perspective has become increasingly productive. As examples, we review a heterogeneous mix of studies from our own work demonstrating Bayesian hierarchical model applications in ocean physics, air–sea interaction, ocean forecasting and ocean ecosystem models. This review is by no means exhaustive and we have endeavored to identify hierarchical modeling work reported by others across the broad range of ocean-related topics reported in the statistical literature. We conclude by noting relevant ocean-statistics problems on the immediate research horizon, and some technical challenges they pose, for example, in terms of nonlinearity, dimensionality and computing.

On Quantifying Dependence: A Framework for Developing Interpretable Measures

Abstract: We present a framework for selecting and developing measures of dependence when the goal is the quantification of a relationship between two variables, not simply the establishment of its existence. Much of the literature on dependence measures is focused, at least implicitly, on detection or revolves around the inclusion/exclusion of particular axioms and discussing which measures satisfy said axioms. In contrast, we start with only a few nonrestrictive guidelines focused on existence, range and interpretability, which provide a very open and flexible framework. For quantification, the most crucial is the notion of interpretability, whose foundation can be found in the work of Goodman and Kruskal [Measures of Association for Cross Classifications (1979) Springer], and whose importance can be seen in the popularity of tools such as the $R^{2}$ in linear regression. While Goodman and Kruskal focused on probabilistic interpretations for their measures, we demonstrate how more general measures of information can be used to achieve the same goal. To that end, we present a strategy for building dependence measures that is designed to allow practitioners to tailor measures to their needs. We demonstrate how many well-known measures fit in with our framework and conclude the paper by presenting two real data examples. Our first example explores U.S. income and education where we demonstrate how this methodology can help guide the selection and development of a dependence measure. Our second example examines measures of dependence for functional data, and illustrates them using data on geomagnetic storms.

Conflict Diagnostics in Directed Acyclic Graphs, with Applications in Bayesian Evidence Synthesis

Abstract: Complex stochastic models represented by directed acyclic graphs (DAGs) are increasingly employed to synthesise multiple, imperfect and disparate sources of evidence, to estimate quantities that are difficult to measure directly. The various data sources are dependent on shared parameters and hence have the potential to conflict with each other, as well as with the model. In a Bayesian framework, the model consists of three components: the prior distribution, the assumed form of the likelihood and structural assumptions. Any of these components may be incompatible with the observed data. The detection and quantification of such conflict and of data sources that are inconsistent with each other is therefore a crucial component of the model criticism process. We first review Bayesian model criticism, with a focus on conflict detection, before describing a general diagnostic for detecting and quantifying conflict between the evidence in different partitions of a DAG. The diagnostic is a $p$-value based on splitting the information contributing to inference about a “separator” node or group of nodes into two independent groups and testing whether the two groups result in the same inference about the separator node(s). We illustrate the method with three comprehensive examples: an evidence synthesis to estimate HIV prevalence; an evidence synthesis to estimate influenza case-severity; and a hierarchical growth model for rat weights.

Advection–Dispersion Across Interfaces

Abstract: This article concerns a systemic manifestation of small scale in- terfacial heterogeneities in large scale quantities of interest to a variety of diverse applications spanning the earth, biological and ecological sciences. Beginning with formulations in terms of partial differential equations gov- erning the conservative, advective-dispersive transport of mass concentra- tions in divergence form, the specific interfacial heterogeneities are intro- duced in terms of (spatial) discontinuities in the diffusion coefficient across a lower-dimensional hypersurface. A pathway to an equivalent stochastic for- mulation is then developed with special attention to the interfacial effects in various functionals such as first passage times, occupation times and local times. That an appreciable theory is achievable within a framework of ap- plications involving one-dimensional models having piecewise constant co- efficients greatly facilitates our goal of a gentle introduction to some rather dramatic mathematical consequences of interfacial effects that can be used to predict structure and to inform modeling.

Mean–Variance and Expected Utility: The Borch Paradox

Abstract: The model of rational decision-making in most of economics and statistics is expected utility theory (EU) axiomatised by von Neumann and Morgenstern, Savage and others. This is less the case, however, in financial economics and mathematical finance, where investment decisions are commonly based on the methods of mean–variance (MV) introduced in the 1950s by Markowitz. Under the MV framework, each available investment opportunity (“asset”) or portfolio is represented in just two dimensions by the ex ante mean and standard deviation $(\mu,\sigma)$ of the financial return anticipated from that investment. Utility adherents consider that in general MV methods are logically incoherent. Most famously, Norwegian insurance theorist Borch presented a proof suggesting that two-dimensional MV indifference curves cannot represent the preferences of a rational investor (he claimed that MV indifference curves “do not exist”). This is known as Borch’s paradox and gave rise to an important but generally little-known philosophical literature relating MV to EU. We examine the main early contributions to this literature, focussing on Borch’s logic and the arguments by which it has been set aside.

Benoît Mandelbrot and Fractional Brownian Motion

Abstract: Although fractional Brownian motion was not invented by Benoit Mandelbrot, it was he who recognized the importance of this random process and gave it the name by which it is known today. This is a personal account of the history behind fractional Brownian motion and some subsequent developments.

The Whetstone and the Alum Block: Balanced Objective Bayesian Comparison of Nested Models for Discrete Data

Abstract: When two nested models are compared, using a Bayes factor, from an objective standpoint, two seemingly conflicting issues emerge at the time of choosing parameter priors under the two models. On the one hand, for moderate sample sizes, the evidence in favor of the smaller model can be inflated by diffuseness of the prior under the larger model. On the other hand, asymptotically, the evidence in favor of the smaller model typically ac- cumulates at a slower rate. With reference to finitely discrete data models, we show that these two issues can be dealt with jointly, by combining intrinsic priors and nonlocal priors in a new unified class of priors. We illustrate our ideas in a running Bernoulli example, then we apply them to test the equality of two proportions, and finally we deal with the more general case of logistic regression models

The True Title of Bayes’s Essay

Abstract: New evidence is presented that Richard Price gave Thomas Bayes's famous essay a very different title from the commonly reported one. It is argued that this implies Price almost surely and Bayes not improbably embarked upon this work seeking a defensive tool to combat David Hume on an issue in theology.

A Prospect of Earthquake Prediction Research

Abstract: Earthquakes occur because of abrupt slips on faults due to accumulated stress in the Earth’s crust. Because most of these faults and their mechanisms are not readily apparent, deterministic earthquake prediction is difficult. For effective prediction, complex conditions and uncertain elements must be considered, which necessitates stochastic prediction. In particular, a large amount of uncertainty lies in identifying whether abnormal phenomena are precursors to large earthquakes, as well as in assigning urgency to the earthquake. Any discovery of potentially useful information for earthquake prediction is incomplete unless quantitative modeling of risk is considered. Therefore, this manuscript describes the prospect of earthquake predictability research to realize practical operational forecasting in the near future.

Scaling Integral Projection Models for Analyzing Size Demography

Abstract: Historically, matrix projection models (MPMs) have been employed to study population dynamics with regard to size, age or structure. To work with continuous traits, in the past decade, integral projection models (IPMs) have been proposed. Following the path for MPMs, currently, IPMs are handled first with a fitting stage, then with a projection stage. Model fitting has, so far, been done only with individual-level transition data. These data are used in the fitting stage to estimate the demographic functions (survival, growth, fecundity) that comprise the kernel of the IPM specification. The estimated kernel is then iterated from an initial trait distribution to obtain what is interpreted as steady state population behavior. Such projection results in inference that does not align with observed temporal distributions. This might be expected; a model for population level projection should be fitted with population level transitions. Ghosh, Gelfand and Clark [J. Agric. Biol. Environ. Stat. 17 (2012) 641–699] offer a remedy by viewing the observed size distribution at a given time as a point pattern over a bounded interval, driven by an operating intensity. They propose a three-stage hierarchical model. At the deepest level, demography is driven by an unknown deterministic IPM. The operating intensities are allowed to vary around this deterministic specification. Further uncertainty arises in the realization of the point pattern given the operating intensities. Such dynamic modeling, optimized by fitting data observed over time, is better suited to projection. Here, we address scaling of population IPM modeling, with the objective of moving from projection at plot level to projection at the scale of the eastern U.S. Such scaling is needed to capture climate effects, which operate at a broader geographic scale, and therefore anticipated demographic response to climate change at larger scales. We work with the Forest Inventory Analysis (FIA) data set, the only data set currently available to enable us to attempt such scaling. Unfortunately, this data set has more than 80% missingness; less than 20% of the 43,396 plots are inventoried each year. We provide a hierarchical modeling approach which still enables us to implement the desired scaling at annual resolution. We illustrate our methodology with a simulation as well as with an analysis for two tree species, one generalist, one specialist.

Bayes Model Selection with Path Sampling: Factor Models and Other Examples

Abstract: We prove a theorem justifying the regularity conditions which are needed for Path Sampling in Factor Models. We then show that the remaining ingredient, namely, MCMC for calculating the integrand at each point in the path, may be seriously flawed, leading to wrong estimates of Bayes factors. We provide a new method of Path Sampling (with Small Change) that works much better than standard Path Sampling in the sense of estimating the Bayes factor better and choosing the correct model more often. When the more complex factor model is true, PS-SC is substantially more accurate. New MCMC diagnostics is provided for these problems in support of our conclusions and recommendations. Some of our ideas for diagnostics and improvement in computation through small changes should apply to other methods of computation of the Bayes factor for model selection.