Showing papers on "Bayes' theorem published in 2009"

Bayesian T Tests for Accepting and Rejecting the Null Hypothesis

Abstract: Progress in science often comes from discovering invariances in relationships among variables; these invariances often correspond to null hypotheses. As is commonly known, it is not possible to state evidence for the null hypothesis in conventional significance testing. Here we highlight a Bayes factor alternative to the conventional t test that will allow researchers to express preference for either the null hypothesis or the alternative. The Bayes factor has a natural and straightforward interpretation, is based on reasonable assumptions, and has better properties than other methods of inference that have been advocated in the psychological literature. To facilitate use of the Bayes factor, we provide an easy-to-use, Web-based program that performs the necessary calculations.

Approximate Bayesian computation scheme for parameter inference and model selection in dynamical systems

Abstract: Approximate Bayesian computation (ABC) methods can be used to evaluate posterior distributions without having to calculate likelihoods. In this paper, we discuss and apply an ABC method based on sequential Monte Carlo (SMC) to estimate parameters of dynamical models. We show that ABC SMC provides information about the inferability of parameters and model sensitivity to changes in parameters, and tends to perform better than other ABC approaches. The algorithm is applied to several well-known biological systems, for which parameters and their credible intervals are inferred. Moreover, we develop ABC SMC as a tool for model selection; given a range of different mathematical descriptions, ABC SMC is able to choose the best model using the standard Bayesian model selection apparatus.

Bayesian phylogeography finds its roots.

Abstract: As a key factor in endemic and epidemic dynamics, the geographical distribution of viruses has been frequently interpreted in the light of their genetic histories. Unfortunately, inference of historical dispersal or migration patterns of viruses has mainly been restricted to model-free heuristic approaches that provide little insight into the temporal setting of the spatial dynamics. The introduction of probabilistic models of evolution, however, offers unique opportunities to engage in this statistical endeavor. Here we introduce a Bayesian framework for inference, visualization and hypothesis testing of phylogeographic history. By implementing character mapping in a Bayesian software that samples time-scaled phylogenies, we enable the reconstruction of timed viral dispersal patterns while accommodating phylogenetic uncertainty. Standard Markov model inference is extended with a stochastic search variable selection procedure that identifies the parsimonious descriptions of the diffusion process. In addition, we propose priors that can incorporate geographical sampling distributions or characterize alternative hypotheses about the spatial dynamics. To visualize the spatial and temporal information, we summarize inferences using virtual globe software. We describe how Bayesian phylogeography compares with previous parsimony analysis in the investigation of the influenza A H5N1 origin and H5N1 epidemiological linkage among sampling localities. Analysis of rabies in West African dog populations reveals how virus diffusion may enable endemic maintenance through continuous epidemic cycles. From these analyses, we conclude that our phylogeographic framework will make an important asset in molecular epidemiology that can be easily generalized to infer biogeogeography from genetic data for many organisms.

Bayesian statistical methods for genetic association studies

Abstract: Bayesian statistical methods have recently made great inroads into many areas of science, and this advance is now extending to the assessment of association between genetic variants and disease or other phenotypes. We review these methods, focusing on single-SNP tests in genome-wide association studies. We discuss the advantages of the Bayesian approach over classical (frequentist) approaches in this setting and provide a tutorial on basic analysis steps, including practical guidelines for appropriate prior specification. We demonstrate the use of Bayesian methods for fine mapping in candidate regions, discuss meta-analyses and provide guidance for refereeing manuscripts that contain Bayesian analyses.

Bayesian Mediation Analysis

Abstract: In this article, we propose Bayesian analysis of mediation effects. Compared with conventional frequentist mediation analysis, the Bayesian approach has several advantages. First, it allows researchers to incorporate prior information into the mediation analysis, thus potentially improving the efficiency of estimates. Second, under the Bayesian mediation analysis, inference is straightforward and exact, which makes it appealing for studies with small samples. Third, the Bayesian approach is conceptually simpler for multilevel mediation analysis. Simulation studies and analysis of 2 data sets are used to illustrate the proposed methods.

Additive Genetic Variability and the Bayesian Alphabet

Abstract: The use of all available molecular markers in statistical models for prediction of quantitative traits has led to what could be termed a genomic-assisted selection paradigm in animal and plant breeding This article provides a critical review of some theoretical and statistical concepts in the context of genomic-assisted genetic evaluation of animals and crops First, relationships between the (Bayesian) variance of marker effects in some regression models and additive genetic variance are examined under standard assumptions Second, the connection between marker genotypes and resemblance between relatives is explored, and linkages between a marker-based model and the infinitesimal model are reviewed Third, issues associated with the use of Bayesian models for marker-assisted selection, with a focus on the role of the priors, are examined from a theoretical angle The sensitivity of a Bayesian specification that has been proposed (called “Bayes A”) with respect to priors is illustrated with a simulation Methods that can solve potential shortcomings of some of these Bayesian regression procedures are discussed briefly

Bayesian inference for nonnegative matrix factorisation models

Abstract: We describe nonnegative matrix factorisation (NMF) with a Kullback-Leibler (KL) error measure in a statistical framework, with a hierarchical generative model consisting of an observation and a prior component. Omitting the prior leads to the standard KL-NMF algorithms as special cases, where maximum likelihood parameter estimation is carried out via the Expectation-Maximisation (EM) algorithm. Starting from this view, we develop full Bayesian inference via variational Bayes or Monte Carlo. Our construction retains conjugacy and enables us to develop more powerful models while retaining attractive features of standard NMF such as monotonic convergence and easy implementation. We illustrate our approach on model order selection and image reconstruction.

Bayes factors for genome‐wide association studies: comparison with P‐values

Abstract: The Bayes factor is a summary measure that provides an alternative to the P-value for the ranking of associations, or the flagging of associations as "significant". We describe an approximate Bayes factor that is straightforward to use and is appropriate when sample sizes are large. We consider various choices of the prior on the effect size, including those that allow effect size to vary with the minor allele frequency (MAF) of the marker. An important contribution is the description of a specific prior that gives identical rankings between Bayes factors and P-values, providing a link between the two approaches, and allowing the implications of the use of P-values to be more easily understood. As a summary measure of noteworthiness P-values are difficult to calibrate since their interpretation depends on MAF and, crucially, on sample size. A consequence is that a consistent decision-making procedure using P-values requires a threshold for significance that reduces with sample size, contrary to common practice.

Modeling demographic processes in marked populations

Abstract: Population Dynamics - Growth, Density-Dependence and Decomposing ?.- Bayesian Hierarchical Models for Inference About Population Growth.- Assessing Density-Dependence: Where Are We Left?.- The Efficient Semiparametric Regression Modeling of Capture-Recapture Data: Assessing the Impact of Climate on Survival of Two Antarctic Seabird Species.- Multivariate State Space Modelling of Bird Migration Count Data.- Evolutionary Ecology.- Contribution of Capture-Mark-Recapture Modeling to Studies of Evolution by Natural Selection.- Application of Capture-Recapture to Addressing Questions in Evolutionary Ecology.- Estimating Reproductive Costs with Multi-State Mark-Recapture Models, Multiple Observable States, and Temporary Emigration.- Estimating Latent Time of Maturation and Survival Costs of Reproduction in Continuous Time from Capture-Recapture Data.- Abundance Estimation - Direct Methods, Proxies, Occupancy Models and Point Count Data.- Inferences About Landbird Abundance from Count Data: Recent Advances and Future Directions.- Sources of Measurement Error, Misclassification Error, and Bias in Auditory Avian Point Count Data.- Density Estimation by Spatially Explicit Capture-Recapture: Likelihood-Based Methods.- A Generalized Mixed Effects Model of Abundance for Mark-Resight Data When Sampling is Without Replacement.- Evaluation of the Linkage Disequilibrium Method for Estimating Effective Population Size.- Dispersal, Movement and Migration - Methods and Multi-State Models.- Migration and Movement - The Next Stage.- Stopover Duration Analysis with Departure Probability Dependent on Unknown Time Since Arrival.- Habitat Selection, Age-Specific Recruitment and Reproductive Success in a Long-Lived Seabird.- Cubic Splines for Estimating the Distribution of Residence Time Using Individual Resightings Data.- Detecting Invisible Migrants: An Application of Genetic Methods to Estimate Migration Rates.- Wildlife and Conservation Management.- Stochastic Variation in Avian Survival Rates: Life-History Predictions, Population Consequences, and the Potential Responses to Human Perturbations and Climate Change.- Filling a Void: Abundance Estimation of North American Populations of Arctic Geese Using Hunter Recoveries.- Integration of Demographic Analyses and Decision Modeling in Support of Management of Invasive Monk Parakeets, an Urban and Agricultural Pest.- Combing Sources of Information - Kalman Filters, Matrix Methods and Joint Likelihoods.- Completing the Ecological Jigsaw.- Using a State-Space Model of the British Song Thrush Turdus philomelos Population to Diagnose the Causes of a Population Decline.- A Hierarchical Covariate Model for Detection, Availability and Abundance of Florida Manatees at a Warm Water Aggregation Site.- An Integrated Analysis of Multisite Recruitment, Mark-Recapture-Recovery and Multisite Census Data.- Bayesian Applications - Advances, Random Effects and Hierarchical Models.- Bayes Factors and Multimodel Inference.- Estimating Demographic Parameters from Complex Data Sets: A Comparison of Bayesian Hierarchical and Maximum-Likelihood Methods for Estimating Survival Probabilities of Tawny Owls, Strix aluco in Finland.- Inference About Species Richness and Community Structure Using Species-Specific Occupancy Models in the National Swiss Breeding Bird Survey MHB.- Time-Varying Covariates and Semi-Parametric Regression in Capture-Recapture: An Adaptive Spline Approach.- A Further Step Toward the Mother-of-All-Models: Flexibility and Functionality in the Modeling of Capture-Recapture Data.- The Robust Design - Sampling, Applications and Advances.- Exploring Extensions to Multi-State Models with Multiple Unobservable States.- Extending the Robust Design for DNA-Based Capture-Recapture Data Incorporating Genotyping Error and Laboratory Data.- A Traditional and a Less-Invasive Robust Design: Choices in Optimizing Effort Allocation for Seabird Population Studies.- Non-random Temporary Emigration and the Robust Design: Conditions for Bias at the End of a Time Series.- State Uncertainty - Assignmant Error and Unobservable States.- One Size Does Not Fit All: Adapting Mark-Recapture and Occupancy Models for State Uncertainty.- The Stakes of Capture-Recapture Models with State Uncertainty.- Rank and Redundancy of Multistate Mark-Recapture Models for Seabird Populations with Unobservable States.- Mark-Recapture Jolly-Seber Abundance Estimation with Classification Uncertainty.- Program E-Surge: A Software Application for Fitting Multievent Models.- Estimation of Lifetime Reproductive Success When Reproductive Status Cannot Always Be Assessed.- New Software Developments for Modeling Demographic Processes.- WinBUGS for Population Ecologists: Bayesian Modeling Using Markov Chain Monte Carlo Methods.- Comparison of Fixed Effect, Random Effect, and Hierarchical Bayes Estimators for Mark Recapture Data Using AD Model Builder.- Open Forum.- On Adjusting for Missed Visits in the Indexing of Abundance from "Constant Effort" Ringing.- Simulation Performance of Bayesian Estimators of Abundance Employing Age-at-Harvest and Mark-Recovery Data.- A Spatial Model for Estimating Mortality Rates, Abundance and Movement Probabilities from Fishery Tag-Recovery Data.- Gaussian Semiparametric Analysis Using Hierarchical Predictive Models.- Effect of Senescence on Estimation of Survival Probability When Age Is Unknown.- Weak Identifiability in Models for Mark-Recapture-Recovery Data.- Estimating N: A Robust Approach to Capture Heterogeneity.- Evaluation of Bias, Precision and Accuracy of Mortality Cause Proportion Estimators from Ring Recovery Data.- Standardising Terminology and Notation for the Analysis of Demographic Processes in Marked Populations.- Estimating the Seasonal Distribution of Migrant Bird Species: Can Standard Ringing Data Be Used?.- Evaluation of a Bayesian MCMC Random Effects Inference Methodology for Capture-Mark-Recapture Data.- On Adjusting for Missed Visits in the Indexing of Abundance from "Constant Effort" Ringing.- Simulation Performance of Bayesian Estimators of Abundance Employing Age-at-Harvest and Mark-Recovery Data.

Nonparametric factor analysis with beta process priors

Abstract: We propose a nonparametric extension to the factor analysis problem using a beta process prior. This beta process factor analysis (BP-FA) model allows for a dataset to be decomposed into a linear combination of a sparse set of factors, providing information on the underlying structure of the observations. As with the Dirichlet process, the beta process is a fully Bayesian conjugate prior, which allows for analytical posterior calculation and straightforward inference. We derive a varia-tional Bayes inference algorithm and demonstrate the model on the MNIST digits and HGDP-CEPH cell line panel datasets.

A Novel Bayes Model: Hidden Naive Bayes

Abstract: Because learning an optimal Bayesian network classifier is an NP-hard problem, learning-improved naive Bayes has attracted much attention from researchers. In this paper, we summarize the existing improved algorithms and propose a novel Bayes model: hidden naive Bayes (HNB). In HNB, a hidden parent is created for each attribute which combines the influences from all other attributes. We experimentally test HNB in terms of classification accuracy, using the 36 UCI data sets selected by Weka, and compare it to naive Bayes (NB), selective Bayesian classifiers (SBC), naive Bayes tree (NBTree), tree-augmented naive Bayes (TAN), and averaged one-dependence estimators (AODE). The experimental results show that HNB significantly outperforms NB, SBC, NBTree, TAN, and AODE. In many data mining applications, an accurate class probability estimation and ranking are also desirable. We study the class probability estimation and ranking performance, measured by conditional log likelihood (CLL) and the area under the ROC curve (AUC), respectively, of naive Bayes and its improved models, such as SBC, NBTree, TAN, and AODE, and then compare HNB to them in terms of CLL and AUC. Our experiments show that HNB also significantly outperforms all of them.

Variational Bayesian Inference for a Nonlinear Forward Model

Abstract: Variational Bayes (VB) has been proposed as a method to facilitate calculations of the posterior distributions for linear models, by providing a fast method for Bayesian inference by estimating the parameters of a factorized approximation to the posterior distribution. Here a VB method for nonlinear forward models with Gaussian additive noise is presented. In the case of noninformative priors the parameter estimates obtained from this VB approach are identical to those found via nonlinear least squares. However, the advantage of the VB method lies in its Bayesian formulation, which permits prior information to be included in a hierarchical structure and measures of uncertainty for all parameter estimates to be obtained via the posterior distribution. Unlike other Bayesian methods VB is only approximate in comparison with the sampling method of MCMC. However, the VB method is found to be comparable and the assumptions made about the form of the posterior distribution reasonable. Practically, the VB approach is substantially faster than MCMC as fewer calculations are required. Some of the advantages of the fully Bayesian nature of the method are demonstrated through the extension of the noise model and the inclusion of automatic relevance determination (ARD) within the VB algorithm.

Elicitation by design in ecology: using expert opinion to inform priors for Bayesian statistical models

Abstract: Bayesian statistical modeling has several benefits within an ecological context. In particular, when observed data are limited in sample size or representativeness, then the Bayesian framework provides a mechanism to combine observed data with other "prior" information. Prior information may be obtained from earlier studies, or in their absence, from expert knowledge. This use of the Bayesian framework reflects the scientific "learning cycle," where prior or initial estimates are updated when new data become available. In this paper we outline a framework for statistical design of expert elicitation processes for quantifying such expert knowledge, in a form suitable for input as prior information into Bayesian models. We identify six key elements: determining the purpose and motivation for using prior information; specifying the relevant expert knowledge available; formulating the statistical model; designing effective and efficient numerical encoding; managing uncertainty; and designing a practical elicitation protocol. We demonstrate this framework applies to a variety of situations, with two examples from the ecological literature and three from our experience. Analysis of these examples reveals several recurring important issues affecting practical design of elicitation in ecological problems.

Disentangling molecular relationships with a causal inference test

Abstract: There has been intense effort over the past couple of decades to identify loci underlying quantitative traits as a key step in the process of elucidating the etiology of complex diseases. Recently there has been some effort to coalesce non-biased high-throughput data, e.g. high density genotyping and genome wide RNA expression, to drive understanding of the molecular basis of disease. However, a stumbling block has been the difficult question of how to leverage this information to identify molecular mechanisms that explain quantitative trait loci (QTL). We have developed a formal statistical hypothesis test, resulting in a p-value, to quantify uncertainty in a causal inference pertaining to a measured factor, e.g. a molecular species, which potentially mediates a known causal association between a locus and a quantitative trait. We treat the causal inference as a 'chain' of mathematical conditions that must be satisfied to conclude that the potential mediator is causal for the trait, where the inference is only as good as the weakest link in the chain. P-values are computed for the component conditions, which include tests of linkage and conditional independence. The Intersection-Union Test, in which a series of statistical tests are combined to form an omnibus test, is then employed to generate the overall test result. Using computer simulated mouse crosses, we show that type I error is low under a variety of conditions that include hidden variables and reactive pathways. We show that power under a simple causal model is comparable to other model selection techniques as well as Bayesian network reconstruction methods. We further show empirically that this method compares favorably to Bayesian network reconstruction methods for reconstructing transcriptional regulatory networks in yeast, recovering 7 out of 8 experimentally validated regulators. Here we propose a novel statistical framework in which existing notions of causal mediation are formalized into a hypothesis test, thus providing a standard quantitative measure of uncertainty in the form of a p-value. The method is theoretically and computationally accessible and with the provided software may prove a useful tool in disentangling molecular relationships.

Modularization in Bayesian analysis, with emphasis on analysis of computer models

Abstract: Bayesian analysis incorporates different sources of information into a single analysis through Bayes theorem. When one or more of the sources of information are suspect (e.g., if the model assumed for the information is viewed as quite possibly being significantly flawed), there can be a concern that Bayes theorem allows this suspect information to overly influence the other sources of information. We consider a variety of situations in which this arises, and give methodological suggestions for dealing with the problem. After consideration of some pedagogical examples of the phenomenon, we focus on the interface of statistics and the development of complex computer models of processes. Three testbed computer models are considered, in which this type of issue arises.

Model criticism based on likelihood-free inference, with an application to protein network evolution

Abstract: Mathematical models are an important tool to explain and comprehend complex phenomena, and unparalleled computational advances enable us to easily explore them without any or little understanding of their global properties. In fact, the likelihood of the data under complex stochastic models is often analytically or numerically intractable in many areas of sciences. This makes it even more important to simultaneously investigate the adequacy of these models—in absolute terms, against the data, rather than relative to the performance of other models—but no such procedure has been formally discussed when the likelihood is intractable. We provide a statistical interpretation to current developments in likelihood-free Bayesian inference that explicitly accounts for discrepancies between the model and the data, termed Approximate Bayesian Computation under model uncertainty (ABCμ). We augment the likelihood of the data with unknown error terms that correspond to freely chosen checking functions, and provide Monte Carlo strategies for sampling from the associated joint posterior distribution without the need of evaluating the likelihood. We discuss the benefit of incorporating model diagnostics within an ABC framework, and demonstrate how this method diagnoses model mismatch and guides model refinement by contrasting three qualitative models of protein network evolution to the protein interaction datasets of Helicobacter pylori and Treponema pallidum. Our results make a number of model deficiencies explicit, and suggest that the T. pallidum network topology is inconsistent with evolution dominated by link turnover or lateral gene transfer alone.

Bayesian inverse problems for functions and applications to fluid mechanics

Abstract: In this paper we establish a mathematical framework for a range of inverse problems for functions, given a finite set of noisy observations. The problems are hence underdetermined and are often ill-posed. We study these problems from the viewpoint of Bayesian statistics, with the resulting posterior probability measure being defined on a space of functions. We develop an abstract framework for such problems which facilitates application of an infinite-dimensional version of Bayes theorem, leads to a well-posedness result for the posterior measure (continuity in a suitable probability metric with respect to changes in data), and also leads to a theory for the existence of maximizing the posterior probability (MAP) estimators for such Bayesian inverse problems on function space. A central idea underlying these results is that continuity properties and bounds on the forward model guide the choice of the prior measure for the inverse problem, leading to the desired results on well-posedness and MAP estimators; the PDE analysis and probability theory required are thus clearly dileneated, allowing a straightforward derivation of results. We show that the abstract theory applies to some concrete applications of interest by studying problems arising from data assimilation in fluid mechanics. The objective is to make inference about the underlying velocity field, on the basis of either Eulerian or Lagrangian observations. We study problems without model error, in which case the inference is on the initial condition, and problems with model error in which case the inference is on the initial condition and on the driving noise process or, equivalently, on the entire time-dependent velocity field. In order to undertake a relatively uncluttered mathematical analysis we consider the two-dimensional Navier–Stokes equation on a torus. The case of Eulerian observations—direct observations of the velocity field itself—is then a model for weather forecasting. The case of Lagrangian observations—observations of passive tracers advected by the flow—is then a model for data arising in oceanography. The methodology which we describe herein may be applied to many other inverse problems in which it is of interest to find, given observations, an infinite-dimensional object, such as the initial condition for a PDE. A similar approach might be adopted, for example, to determine an appropriate mathematical setting for the inverse problem of determining an unknown tensor arising in a constitutive law for a PDE, given observations of the solution. The paper is structured so that the abstract theory can be read independently of the particular problems in fluid mechanics which are subsequently studied by application of the theory.

Naive Bayes Classification of Uncertain Data

Abstract: Traditional machine learning algorithms assume that data are exact or precise. However, this assumption may not hold in some situations because of data uncertainty arising from measurement errors, data staleness, and repeated measurements, etc. With uncertainty, the value of each data item is represented by a probability distribution function (pdf). In this paper, we propose a novel naive Bayes classification algorithm for uncertain data with a pdf. Our key solution is to extend the class conditional probability estimation in the Bayes model to handle pdf’s. Extensive experiments on UCI datasets show that the accuracy of naive Bayes model can be improved by taking into account the uncertainty information.

Granger causality vs. dynamic Bayesian network inference: a comparative study

Abstract: Background In computational biology, one often faces the problem of deriving the causal relationship among different elements such as genes, proteins, metabolites, neurons and so on, based upon multi-dimensional temporal data. Currently, there are two common approaches used to explore the network structure among elements. One is the Granger causality approach, and the other is the dynamic Bayesian network inference approach. Both have at least a few thousand publications reported in the literature. A key issue is to choose which approach is used to tackle the data, in particular when they give rise to contradictory results.

Crash Injury Severity Analysis Using Bayesian Ordered Probit Models

Abstract: Understanding the underlying relationship between crash injury severity and factors such as driver's characteristics, vehicle type, and roadway conditions is very important for improving traffic safety. Most previous studies on this topic used traditional statistical models such as ordered probit (OP), multinomial logit, and nested logit models. This research introduces the Bayesian inference and investigates the application of a Bayesian ordered probit (BOP) model in driver's injury severity analysis. The OP and BOP models are compared based on datasets with different sample sizes from the 2003 National Automotive Sampling System General Estimates System (NASSGES). The comparison results show that these two types of models produce similar results for large sample data. When the sample data size is small, with proper prior setting, the BOP model can produce more reasonable parameter estimations and better prediction performance than the OP model. This research also shows that the BOP model provides a flexible framework that can combine information contained in the data with the prior knowledge of the parameters to improve model performance.

Bayesian committee of neural networks to predict travel times with confidence intervals

Abstract: Short-term prediction of travel time is one of the central topics in current transportation research and practice. Among the more successful travel time prediction approaches are neural networks and combined prediction models (a 'committee'). However, both approaches have disadvantages. Usually many candidate neural networks are trained and the best performing one is selected. However, it is difficult and arbitrary to select the optimal network. In committee approaches a principled and mathematically sound framework to combine travel time predictions is lacking. This paper overcomes the drawbacks of both approaches by combining neural networks in a committee using Bayesian inference theory. An 'evidence' factor can be calculated for each model, which can be used as a stopping criterion during training, and as a tool to select and combine different neural networks. Along with higher prediction accuracy, this approach allows for accurate estimation of confidence intervals for the predictions. When comparing the committee predictions to single neural network predictions on the A12 motorway in the Netherlands it is concluded that the approach indeed leads to improved travel time prediction accuracy.

Comparison of Hierarchical Bayesian Models for Overdispersed Count Data using DIC and Bayes' Factors

Abstract: When replicate count data are overdispersed, it is common practice to incorporate this extra-Poisson variability by including latent parameters at the observation level. For example, the negative binomial and Poisson-lognormal (PLN) models are obtained by using gamma and lognormal latent parameters, respectively. Several recent publications have employed the deviance information criterion (DIC) to choose between these two models, with the deviance defined using the Poisson likelihood that is obtained from conditioning on these latent parameters. The results herein show that this use of DIC is inappropriate. Instead, DIC was seen to perform well if calculated using likelihood that was marginalized at the group level by integrating out the observation-level latent parameters. This group-level marginalization is explicit in the case of the negative binomial, but requires numerical integration for the PLN model. Similarly, DIC performed well to judge whether zero inflation was required when calculated using the group-marginalized form of the zero-inflated likelihood. In the context of comparing multilevel hierarchical models, the top-level DIC was obtained using likelihood that was further marginalized by additional integration over the group-level latent parameters, and the marginal densities of the models were calculated for the purpose of providing Bayes' factors. The computational viability and interpretability of these different measures is considered.

Nonnegative matrix factorizations as probabilistic inference in composite models

Abstract: We develop an interpretation of nonnegative matrix factorization (NMF) methods based on Euclidean distance, Kullback-Leibler and Itakura-Saito divergences in a probabilistic framework. We describe how these factorizations are implicit in a well-defined statistical model of superimposed components, either Gaussian or Poisson distributed, and are equivalent to maximum likelihood estimation of either mean, variance or intensity parameters. By treating the components as hidden-variables, NMF algorithms can be derived in a typical data augmentation setting. This setting can in particular accommodate regularization constraints on the matrix factors through Bayesian priors. We describe multiplicative, Expectation-Maximization, Markov chain Monte Carlo and Variational Bayes algorithms for the NMF problem. This paper describes in a unified framework both new and known algorithms and aims at providing statistical insights to NMF.

Empirical Bayes Estimates for Large-Scale Prediction Problems.

Abstract: Classical prediction methods, such as Fisher’s linear discriminant function, were designed for small-scale problems in which the number of predictors N is much smaller than the number of observations n. Modern scientific devices often reverse this situation. A microarray analysis, for example, might include n=100 subjects measured on N=10,000 genes, each of which is a potential predictor. This article proposes an empirical Bayes approach to large-scale prediction, where the optimum Bayes prediction rule is estimated employing the data from all of the predictors. Microarray examples are used to illustrate the method. The results demonstrate a close connection with the shrunken centroids algorithm of Tibshirani et al. (2002), a frequentist regularization approach to large-scale prediction, and also with false discovery rate theory.