Showing papers on "Bayesian inference published in 2000"

Bayesian Photometric Redshift Estimation

Abstract: Photometric redshifts are quickly becoming an essential tool of observational cosmology, although their utilization is somewhat hindered by certain shortcomings of the existing methods, e.g., the unreliability of maximum-likelihood techniques or the limited application range of the "training-set" approach. The application of Bayesian inference to the problem of photometric redshift estimation effectively overcomes most of these problems. The use of prior probabilities and Bayesian marginalization facilitates the inclusion of relevant knowledge, such as the expected shape of the redshift distributions and the galaxy type fractions, which can be readily obtained from existing surveys but are often ignored by other methods. If this previous information is lacking or insufficient—for instance, because of the unprecedented depth of the observations—the corresponding prior distributions can be calibrated using even the data sample for which the photometric redshifts are being obtained. An important advantage of Bayesian statistics is that the accuracy of the redshift estimation can be characterized in a way that has no equivalents in other statistical approaches, enabling the selection of galaxy samples with extremely reliable photometric redshifts. In this way, it is possible to determine the properties of individual galaxies more accurately, and simultaneously estimate the statistical properties of a sample in an optimal fashion. Moreover, the Bayesian formalism described here can be easily generalized to deal with a wide range of problems that make use of photometric redshifts. There is excellent agreement between the ≈130 Hubble Deep Field North (HDF-N) spectroscopic redshifts and the predictions of the method, with a rms error of Δz ≈ 0.06(1 + zspec) up to z < 6 and no outliers nor systematic biases. It should be remarked that since these results have not been reached following a training-set procedure, the above value of Δz should be a fair estimate of the expected accuracy for any similar sample. The method is further tested by estimating redshifts in the HDF-N but restricting the color information to the UBVI filters; the results are shown to be significantly more reliable than those obtained with maximum-likelihood techniques.

Monte Carlo Methods in Bayesian Computation

Abstract: This book examines advanced Bayesian computational methods. It presents methods for sampling from posterior distributions and discusses how to compute posterior quantities of interest using Markov chain Monte Carlo (MCMC) samples. This book examines each of these issues in detail and heavily focuses on computing various posterior quantities of interest from a given MCMC sample. Several topics are addressed, including techniques for MCMC sampling, Monte Carlo methods for estimation of posterior quantities, improving simulation accuracy, marginal posterior density estimation, estimation of normalizing constants, constrained parameter problems, highest posterior density interval calculations, computation of posterior modes, and posterior computations for proportional hazards models and Dirichlet process models. The authors also discuss computions involving model comparisons, including both nested and non-nested models, marginal likelihood methods, ratios of normalizing constants, Bayes factors, the Savage-Dickey density ratio, Stochastic Search Variable Selection, Bayesian Model Averaging, the reverse jump algorithm, and model adequacy using predictive and latent residual approaches. The book presents an equal mixture of theory and applications involving real data. The book is intended as a graduate textbook or a reference book for a one semester course at the advanced masters or Ph.D. level. It would also serve as a useful reference book for applied or theoretical researchers as well as practitioners. Ming-Hui Chen is Associate Professor of Mathematical Sciences at Worcester Polytechnic Institute, Qu-Man Shao is Assistant Professor of Mathematics at the University of Oregon. Joseph G. Ibrahim is Associate Professor of Biostatistics at the Harvard School of Public Health and Dana-Farber Cancer Institute.

Computational and inferential difficulties with mixture posterior distributions

Abstract: This article deals with both exploration and interpretation problems related to posterior distributions for mixture models. The specification of mixture posterior distributions means that the presence of k! modes is known immediately. Standard Markov chain Monte Carlo (MCMC) techniques usually have difficulties with well-separated modes such as occur here; the MCMC sampler stays within a neighborhood of a local mode and fails to visit other equally important modes. We show that exploration of these modes can be imposed using tempered transitions. However, if the prior distribution does not distinguish between the different components, then the posterior mixture distribution is symmetric and standard estimators such as posterior means cannot be used. We propose alternatives for Bayesian inference for permutation invariant posteriors, including a clustering device and alternative appropriate loss functions.

Bayesian Inference in Dynamic Econometric Models

Abstract: Chapter 1: Decision Theory and Bayesian Inference Chapter 2: Bayesian Statistics and Linear Regression Chapter 3: Methods of Numerical Integration Chapter 4: Prior Densities for the Regression Model Chapter 5: Dynamic Regression Models Chapter 6: Bayesian Unit Roots Chapter 7: Heteroskedasticity and ARCH Chapter 8: Nonlinear Tome Series Models Chapter 9: Systems of Equations Appendix A: Probability Distributions Appendix B: Generating Random Numbers

Bayesian Dynamic Factor Models and Portfolio Allocation

Abstract: We discuss the development of dynamic factor models for multivariate financial time series, and the incorporation of stochastic volatility components for latent factor processes. Bayesian inference and computation is developed and explored in a study of the dynamic factor structure of daily spot exchange rates for a selection of international currencies. The models are direct generalizations of univariate stochastic volatility models and represent specific varieties of models recently discussed in the growing multivariate stochastic volatility literature. We discuss model fitting based on retrospective data and sequential analysis for forward filtering and short-term forecasting. Analyses are compared with results from the much simpler method of dynamic variance-matrix discounting that, for over a decade, has been a standard approach in applied financial econometrics. We study these models in analysis, forecasting, and sequential portfolio allocation for a selected set of international exchange-rate-retur...

A Differential Approach to Inference in Bayesian Networks

Abstract: We present a new approach for inference in Bayesian networks, which is mainly based on partial differentiation. According to this approach, one compiles a Bayesian network into a multivariate polynomial and then computes the partial derivatives of this polynomial with respect to each variable. We show that once such derivatives are made available, one can compute in constant-time answers to a large class of probabilistic queries, which are central to classical inference, parameter estimation, model validation and sensitivity analysis. We present a number of complexity results relating to the compilation of such polynomials and to the computation of their partial derivatives. We argue that the combined simplicity, comprehensiveness and computational complexity of the presented framework is unique among existing frameworks for inference in Bayesian networks.

Causal Inference without Counterfactuals

Abstract: A popular approach to the framing and answering of causal questions relies on the idea of counterfactuals: Outcomes that would have been observed had the world developed differently; for example, if the patient had received a different treatment. By definition, one can never observe such quantities, nor assess empirically the validity of any modeling assumptions made about them, even though one's conclusions may be sensitive to these assumptions. Here I argue that for making inference about the likely effects of applied causes, counterfactual arguments are unnecessary and potentially misleading. An alternative approach, based on Bayesian decision analysis, is presented. Properties of counterfactuals are relevant to inference about the likely causes of observed effects, but close attention then must be given to the nature and context of the query, as well as to what conclusions can and cannot be supported empirically. In particular, even in the absence of Statistical uncertainty, such inferences m...

Non-linear Bayesian Image Modelling

Abstract: In recent years several techniques have been proposed for modelling the low-dimensional manifolds, or 'subspaces', of natural images. Examples include principal component analysis (as used for instance in 'eigen-faces'), independent component analysis, and auto-encoder neural networks. Such methods suffer from a number of restrictions such as the limitation to linear manifolds or the absence of a probablistic representation. In this paper we exploit recent developments in the fields of variational inference and latent variable models to develop a novel and tractable probabilistic approach to modelling manifolds which can handle complex non-linearities. Our framework comprises a mixture of sub-space components in which both the number of components and the effective dimensionality of the subspaces are determined automatically as part of the Bayesian inference procedure. We illustrate our approach using two classical problems: modelling the manifold of face images and modelling the manifolds of hand-written digits.

Inference for Deterministic Simulation Models: The Bayesian Melding Approach

Abstract: Deterministic simulation models are used in many areas of science, engineering, and policy making. Typically, these are complex models that attempt to capture underlying mechanisms in considerable detail, and they have many user-specified inputs. The inputs are often specified by some form of trial-and-error approach in which plausible values are postulated, the corresponding outputs inspected, and the inputs modified until plausible outputs are obtained. Here we address the issue of more formal inference for such models. A probabilistic approach, called Bayesian synthesis, was shown to suffer from the Borel paradox, according to which the results can depend on the parameterization of the model. We propose a modified approach, called Bayesian melding which takes into full account information and uncertainty about both inputs and outputs to the model, while avoiding the Borel paradox. This is done by recognizing the existence of two priors, one implicit and one explicit, on each input and output; ...

Propagation Algorithms for Variational Bayesian Learning

Abstract: Variational approximations are becoming a widespread tool for Bayesian learning of graphical models. We provide some theoretical results for the variational updates in a very general family of conjugate-exponential graphical models. We show how the belief propagation and the junction tree algorithms can be used in the inference step of variational Bayesian learning. Applying these results to the Bayesian analysis of linear-Gaussian state-space models we obtain a learning procedure that exploits the Kalman smoothing propagation, while integrating over all model parameters. We demonstrate how this can be used to infer the hidden state dimensionality of the state-space model in a variety of synthetic problems and one real high-dimensional data set.

A compound poisson process for relaxing the molecular clock.

Abstract: The molecular clock hypothesis remains an important conceptual and analytical tool in evolutionary biology despite the repeated observation that the clock hypothesis does not perfectly explain observed DNA sequence variation. We introduce a parametric model that relaxes the molecular clock by allowing rates to vary across lineages according to a compound Poisson process. Events of substitution rate change are placed onto a phylogenetic tree according to a Poisson process. When an event of substitution rate change occurs, the current rate of substitution is modified by a gamma-distributed random variable. Parameters of the model can be estimated using Bayesian inference. We use Markov chain Monte Carlo integration to evaluate the posterior probability distribution because the posterior probability involves high dimensional integrals and summations. Specifically, we use the Metropolis-Hastings-Green algorithm with 11 different move types to evaluate the posterior distribution. We demonstrate the method by analyzing a complete mtDNA sequence data set from 23 mammals. The model presented here has several potential advantages over other models that have been proposed to relax the clock because it is parametric and does not assume that rates change only at speciation events. This model should prove useful for estimating divergence times when substitution rates vary across lineages.

Scalable Techniques for Mining Causal Structures

Abstract: Mining for association rules in market basket data has proved a fruitful area of research. Measures such as conditional probability (confidence) and correlation have been used to infer rules of the form “the existence of item A implies the existence of item B.” However, such rules indicate only a statistical relationship between A and B. They do not specify the nature of the relationship: whether the presence of A causes the presence of B, or the converse, or some other attribute or phenomenon causes both to appear together. In applications, knowing such causal relationships is extremely useful for enhancing understanding and effecting change. While distinguishing causality from correlation is a truly difficult problem, recent work in statistics and Bayesian learning provide some avenues of attack. In these fields, the goal has generally been to learn complete causal models, which are essentially impossible to learn in large-scale data mining applications with a large number of variables. In this paper, we consider the problem of determining casual relationships, instead of mere associations, when mining market basket data. We identify some problems with the direct application of Bayesian learning ideas to mining large databases, concerning both the scalability of algorithms and the appropriateness of the statistical techniques, and introduce some initial ideas for dealing with these problems. We present experimental results from applying our algorithms on several large, real-world data sets. The results indicate that the approach proposed here is both computationally feasible and successful in identifying interesting causal structures. An interesting outcome is that it is perhaps easier to infer the lack of causality than to infer causality, information that is useful in preventing erroneous decision making.

BUGS for a Bayesian Analysis of Stochastic Volatility Models

Abstract: This paper reviews the general Bayesian approach to parameter estimation in stochastic volatility models with posterior computations performed by Gibbs sampling. The main purpose is to illustrate the ease with which the Bayesian stochastic volatility model can now be studied routinely via BUGS (Bayesian Inference Using Gibbs Sampling), a recently developed, user-friendly, and freely available software package. It is an ideal software tool for the exploratory phase of model building as any modifications of a model including changes of priors and sampling error distributions are readily realized with only minor changes of the code. BUGS automates the calculation of the full conditional posterior distributions using a model representation by directed acyclic graphs. It contains an expert system for choosing an efficient sampling method for each full conditional. Furthermore, software for convergence diagnostics and statistical summaries is available for the BUGS output. The BUGS implementation of a stochastic volatility model is illustrated using a time series of daily Pound/Dollar exchange rates.

Flexible empirical Bayes estimation for wavelets

Abstract: Summary. Wavelet shrinkage estimation is an increasingly popular method for signal denoising and compression. Although Bayes estimators can provide excellent mean-squared error (MSE) properties, the selection of an effective prior is a difficult task. To address this problem, we propose empirical Bayes (EB) prior selection methods for various error distributions including the normal and the heavier-tailed Student t-distributions. Under such EB prior distributions, we obtain threshold shrinkage estimators based on model selection, and multiple-shrinkage estimators based on model averaging. These EB estimators are seen to be computationally competitive with standard classical thresholding methods, and to be robust to outliers in both the data and wavelet domains. Simulated and real examples are used to illustrate the flexibility and improved MSE performance of these methods in a wide variety of settings.

BUGS for a Bayesian analysis of stochastic volatility models

Abstract: Summary This paper reviews the general Bayesian approach to parameter estimation in stochastic volatility models with posterior computations performed by Gibbs sampling. The main purpose is to illustrate the ease with which the Bayesian stochastic volatility model can now be studied routinely via BUGS (Bayesian inference using Gibbs sampling), a recently developed, user-friendly, and freely available software package. It is an ideal software tool for the exploratory phase of model building as any modifications of a model including changes of priors and sampling error distributions are readily realized with only minor changes of the code. However, due to the single move Gibbs sampler, convergence can be slow. BUGS automates the calculation of the full conditional posterior distributions using a model representation by directed acyclic graphs. It contains an expert system for choosing an effective sampling method for each full conditional. Furthermore, software for convergence diagnostics and statistical summaries is available for the BUGS output. The BUGS implementation of a stochastic volatility model is illustrated using a time series of daily Pound/Dollar exchange rates.

Identifiability and convergence issues for Markov chain Monte Carlo fitting of spatial models.

Abstract: The marked increase in popularity of Bayesian methods in statistical practice over the last decade owes much to the simultaneous development of Markov chain Monte Carlo (MCMC) methods for the evaluation of requisite posterior distributions. However, along with this increase in computing power has come the temptation to fit models larger than the data can readily support, meaning that often the propriety of the posterior distributions for certain parameters depends on the propriety of the associated prior distributions. An important example arises in spatial modelling, wherein separate random effects for capturing unstructured heterogeneity and spatial clustering are of substantive interest, even though only their sum is well identified by the data. Increasing the informative content of the associated prior distributions offers an obvious remedy, but one that hampers parameter interpretability and may also significantly slow the convergence of the MCMC algorithm. In this paper we investigate the relationship among identifiability, Bayesian learning and MCMC convergence rates for a common class of spatial models, in order to provide guidance for prior selection and algorithm tuning. We are able to elucidate the key issues with relatively simple examples, and also illustrate the varying impacts of covariates, outliers and algorithm starting values on the resulting algorithms and posterior distributions.

Bayesian segmentation of protein secondary structure.

Abstract: We present a novel method for predicting the secondary structure of a protein from its amino acid sequence. Most existing methods predict each position in turn based on a local window of residues, sliding this window along the length of the sequence. In contrast, we develop a probabilistic model of protein sequence/structure relationships in terms of structural segments, and formulate secondary structure prediction as a general Bayesian inference problem. A distinctive feature of our approach is the ability to develop explicit probabilistic models for α-helices, β-strands, and other classes of secondary structure, incorporating experimentally and empirically observed aspects of protein structure such as helical capping signals, side chain correlations, and segment length distributions. Our model is Markovian in the segments, permitting efficient exact calculation of the posterior probability distribution over all possible segmentations of the sequence using dynamic programming. The optimal segmentation is...

Estimation of origin-destination matrices from link flows on uncongested networks

Abstract: Given link flow data from an uncongested network over a number of time periods, the problem of estimating the origin–destination (O–D) traffic intensities is considered. A statistical model of the transport system with Poisson distributed O–D flows is developed, in which the variation of route choice proportions is represented. The distribution theory of the model is discussed, parameter identifiability investigated, and the full likelihood function derived. This function proves somewhat too cumbersome for practical use but two feasible estimation procedures are established, both based upon maximization of multivariate normal approximations to the likelihood. Although these methods can operate using link flow data alone, incorporation of prior information into the inferential process is also detailed. The basic statistical model is then extended to encompass measurement error in the link flow data and modified methods of parameter estimation are investigated. The paper finishes with a numerical study of the proposed estimation procedures and discussion of some suggested avenues for future research.

Statistical models of video structure for content analysis and characterization

Abstract: Content structure plays an important role in the understanding of video. In this paper, we argue that knowledge about structure can be used both as a means to improve the performance of content analysis and to extract features that convey semantic information about the content. We introduce statistical models for two important components of this structure, shot duration and activity, and demonstrate the usefulness of these models with two practical applications. First, we develop a Bayesian formulation for the shot segmentation problem that is shown to extend the standard thresholding model in an adaptive and intuitive way, leading to improved segmentation accuracy. Second, by applying the transformation into the shot duration/activity feature space to a database of movie clips, we also illustrate how the Bayesian model captures semantic properties of the content. We suggest ways in which these properties can be used as a basis for intuitive content-based access to movie libraries.

Model uncertainty and health effect studies for particulate matter

Abstract: There are many aspects of model choice that are involved in health effect studies of particulate matter and other pollutants. Some of these choices concern which pollutants and confounding variables should be included in the model, what type of lag structure for the covariates should be used, which interactions need to be considered, and how to model nonlinear trends. Because of the large number of potential variables, model selection is often used to find a parsimonious model. Different model selection strategies may lead to very different models and conclusions for the same set of data. As variable selection may involve numerous test of hypotheses, the resulting significance levels may be called into question, and there is the concern that the positive associations are a result of multiple testing. Bayesian Model Averaging is an alternative that can be used to combine inferences from multiple models and incorporate model uncertainty. This paper presents objective prior distributions for Bayesian Model Averaging in generalized linear models so that Bayesian model selection corresponds to standard methods of model selection, such as the Akaike Information Criterion (AIC) or Bayes Information Criterion (BIC), and inferences within a model are based on standard maximum likelihood estimation. These methods allow non-Bayesians to describe the level of uncertainty due to model selection, and can be used to combine inferences by averaging over a wider class of models using readily available summary statistics from standard model fitting programs. Using Bayesian Model Averaging and objective prior distributions, we re-analyze data from Birmingham, AL and illustrate the role of model uncertainty in inferences about the effect of particulate matter on elderly mortality. Copyright © 2000 John Wiley & Sons, Ltd.

Modeling the Sources of Output Growth in a Panel of Countries

Abstract: This article seeks to improve understanding of cross-country patterns of economic growth. It adopts a stochastic production-frontier model that allows for the decomposition of output change into input, efficiency, and technical change. The production frontier is assumed to depend on effective inputs rather than measured inputs. We develop a model in which effective inputs depend on observed factor use and a correction term that depends on variables such as education. A further extension over related work is our use of a production frontier that varies over regional country groups. Empirical results indicate that both these extensions are very important.

Estimation and Inference Are Missing Data Problems: Unifying Social Science Statistics via Bayesian Simulation

Abstract: Bayesian simulation is increasingly exploited in the social sciences for estimation and inference of model parameters. But an especially useful (if often overlooked) feature of Bayesian simulation is that it can be used to estimate any function of model parameters, including “auxiliary” quantities such as goodness-of-fit statistics, predicted values, and residuals. Bayesian simulation treats these quantities as if they were missing data, sampling from their implied posterior densities. Exploiting this principle also lets researchers estimate models via Bayesian simulation where maximum-likelihood estimation would be intractable. Bayesian simulation thus provides a unified solution for quantitative social science. I elaborate these ideas in a variety of contexts: these include generalized linear models for binary responses using data on bill cosponsorship recently reanalyzed in Political Analysis, item—response models for the measurement of respondent's levels of political information in public opinion surveys, the estimation and analysis of legislators' ideal points from roll-call data, and outlier-resistant regression estimates of incumbency advantage in U.S. Congressional elections

Full Bayesian Inference for GARCH and EGARCH Models

Abstract: A full Bayesian analysis of GARCH and EGARCH models is proposed consisting of parameter estimation, model selection, and volatility prediction. The Bayesian paradigm is implemented via Markov-chain Monte Carlo methodologies. We provide implementation details and illustrations using the General Index of the Athens stock exchange.

Interactive learning and probabilistic retrieval in remote sensing image archives

Abstract: We present a concept of interactive learning and probabilistic retrieval of user-specific cover types in a content-based remote sensing image archive. A cover type is incrementally defined via user-provided positive and negative examples. From these examples, we infer probabilities of the Bayesian network that link the user interests to a pre-extracted content index. Due to the stochastic nature of the cover type definitions, the database system not only retrieves images according to the estimated coverage but also according to the accuracy of that estimation given the current state of learning. For the latter, we introduce the concept of separability. We expand on the steps of Bayesian inference to compute the application-free content index using a family of data models, and on the description of the stochastic link using hyperparameters. In particular, we focus on the interactive nature of our approach, which provides instantaneous feedback to the user in the form of an immediate update of the posterior map, and a very fast, approximate search in the archive. A java-based demonstrator using the presented concept of content-based access to a test archive of Landsat TM, X-SAR, and aerial images are available over the Internet [http://www.vision.ee.ethz.ch/∼rsia/ClickBayes].

The Manhattan World Assumption: Regularities in Scene Statistics which Enable Bayesian Inference

Abstract: Preliminary work by the authors made use of the so-called "Manhattan world" assumption about the scene statistics of city and indoor scenes. This assumption stated that such scenes were built on a cartesian grid which led to regularities in the image edge gradient statistics. In this paper we explore the general applicability of this assumption and show that, surprisingly, it holds in a large variety of less structured environments including rural scenes. This enables us, from a single image, to determine the orientation of the viewer relative to the scene structure and also to detect target objects which are not aligned with the grid. These inferences are performed using a Bayesian model with probability distributions (e.g. on the image gradient statistics) learnt from real data.

Interactive learning and probabilistic retrieval in remote sensing image archives

Abstract: The authors present a concept of interactive learning and probabilistic retrieval of user-specific cover types in a content-based remote sensing image archive. A cover type is incrementally defined via user-provided positive and negative examples. From these examples, the authors infer probabilities of the Bayesian network that link the user interests to a pre-extracted content index. Due to the stochastic nature of the cover type definitions, the database system not only retrieves images according to the estimated coverage but also according to the accuracy of that estimation given the current state of learning. For the latter, they introduce the concept of separability. They expand on the steps of Bayesian inference to compute the application-free content index using a family of data models, and on the description of the stochastic link using hyperparameters. In particular, they focus on the interactive nature of their approach, which provides instantaneous feedback to the user in the form of an immediate update of the posterior map, and a very fast, approximate search in the archive. A java-based demonstrator using the presented concept of content-based access to a test archive of Landsat TM, X-SAR, and aerial images are available over the Internet [http:/www.vision.ee.ethz.ch/-rsia/ClickBayes].

Automated medical decision making utilizing bayesian network knowledge domain modeling

Abstract: The present invention relates to a system and method of medical knowledge domain modeling and automated medical decision-making, such as for online, questionnaire-based medical triage. In the present invention, information such as conditions and characteristics related to a diagnosis or disposition level is modeled in a Bayesian Network. The Bayesian Network may comprise instantiable nodes, fault nodes, intermediary nodes, a utility node and a decision node. Using Bayesian inference, the conditional probability of any pair in the network may be determined in real-time. These conditional probabilities are modified upon the input of evidence, which is typically in the form of answers to a dynamic set of questions designed to identify a diagnosis or disposition level for the patient under evaluation.

Structure Learning in Human Causal Induction

Abstract: We use graphical models to explore the question of how people learn simple causal relationships from data. The two leading psychological theories can both be seen as estimating the parameters of a fixed graph. We argue that a complete account of causal induction should also consider how people learn the underlying causal graph structure, and we propose to model this inductive process as a Bayesian inference. Our argument is supported through the discussion of three data sets.

Bayesian factor analysis for multilevel binary observations

Abstract: Multilevel covariance structure models have become increasingly popular in the psychometric literature in the past few years to account for population heterogeneity and complex study designs. We develop practical simulation based procedures for Bayesian inference of multilevel binary factor analysis models. We illustrate how Markov Chain Monte Carlo procedures such as Gibbs sampling and Metropolis-Hastings methods can be used to perform Bayesian inference, model checking and model comparison without the need for multidimensional numerical integration. We illustrate the proposed estimation methods using three simulation studies and an application involving student's achievement results in different areas of mathematics.