Showing papers on "Bayesian inference published in 1995"

Bayesian Data Analysis

Abstract: FUNDAMENTALS OF BAYESIAN INFERENCE Probability and Inference Single-Parameter Models Introduction to Multiparameter Models Asymptotics and Connections to Non-Bayesian Approaches Hierarchical Models FUNDAMENTALS OF BAYESIAN DATA ANALYSIS Model Checking Evaluating, Comparing, and Expanding Models Modeling Accounting for Data Collection Decision Analysis ADVANCED COMPUTATION Introduction to Bayesian Computation Basics of Markov Chain Simulation Computationally Efficient Markov Chain Simulation Modal and Distributional Approximations REGRESSION MODELS Introduction to Regression Models Hierarchical Linear Models Generalized Linear Models Models for Robust Inference Models for Missing Data NONLINEAR AND NONPARAMETRIC MODELS Parametric Nonlinear Models Basic Function Models Gaussian Process Models Finite Mixture Models Dirichlet Process Models APPENDICES A: Standard Probability Distributions B: Outline of Proofs of Asymptotic Theorems C: Computation in R and Stan Bibliographic Notes and Exercises appear at the end of each chapter.

Bayesian Model Selection in Social Research

Abstract: It is argued that P-values and the tests based upon them give unsatisfactory results, especially in large samples. It is shown that, in regression, when there are many candidate independent variables, standard variable selection procedures can give very misleading results. Also, by selecting a single model, they ignore model uncertainty and so underestimate the uncertainty about quantities of interest. The Bayesian approach to hypothesis testing, model selection, and accounting for model uncertainty is presented. Implementing this is straightforward through the use of the simple and accurate BIC approximation, and it can be done using the output from standard software. Specific results are presented for most of the types of model commonly used in sociology. It is shown that this approach overcomes the difficulties with P-values and standard model selection procedures based on them. It also allows easy comparison of nonnested models, and permits the quantification of the evidence for a null hypothesis of interest, such as a convergence theory or a hypothesis about societal norms.

Bayesian learning for neural networks

Abstract: From the Publisher: Artificial "neural networks" are now widely used as flexible models for regression classification applications, but questions remain regarding what these models mean, and how they can safely be used when training data is limited. Bayesian Learning for Neural Networks shows that Bayesian methods allow complex neural network models to be used without fear of the "overfitting" that can occur with traditional neural network learning methods. Insight into the nature of these complex Bayesian models is provided by a theoretical investigation of the priors over functions that underlie them. Use of these models in practice is made possible using Markov chain Monte Carlo techniques. Both the theoretical and computational aspects of this work are of wider statistical interest, as they contribute to a better understanding of how Bayesian methods can be applied to complex problems. Presupposing only the basic knowledge of probability and statistics, this book should be of interest to many researchers in statistics, engineering, and artificial intelligence. Software for Unix systems that implements the methods described is freely available over the Internet.

Bayesian Density Estimation and Inference Using Mixtures

Abstract: We describe and illustrate Bayesian inference in models for density estimation using mixtures of Dirichlet processes. These models provide natural settings for density estimation and are exemplified by special cases where data are modeled as a sample from mixtures of normal distributions. Efficient simulation methods are used to approximate various prior, posterior, and predictive distributions. This allows for direct inference on a variety of practical issues, including problems of local versus global smoothing, uncertainty about density estimates, assessment of modality, and the inference on the numbers of components. Also, convergence results are established for a general class of normal mixture models.

How to Improve Bayesian Reasoning Without Instruction: Frequency Formats

Abstract: Is the mind, by design, predisposed against performing Bayesian inference? Previous research on base rate neglect suggests that the mind lacks the appropriate cognitive algorithms. However, any claim against the existence of an algorithm, Bayesian or otherwise, is impossible to evaluate unless one specifies the information format in which it is designed to operate. The authors show that Bayesian algorithms are computationally simpler in frequency formats than in the probability formats used in previous research. Frequency formats correspond to the sequential way information is acquired in natural sampling, from animal foraging to neural networks. By analyzing several thousand solutions to Bayesian problems, the authors found that when information was presented in frequency formats, statistically naive participants derived up to 50% of all inferences by Bayesian algorithms. Non-Bayesian algorithms included simple versions of Fisherian and Neyman-Pearsonian inference. Is the mind, by design, predisposed against performing Bayesian inference? The classical probabilists of the Enlightenment, including Condorcet, Poisson, and Laplace, equated probability theory with the common sense of educated people, who were known then as "hommes eclaires." Laplace (1814/ 1951) declared that "the theory of probability is at bottom nothing more than good sense reduced to a calculus which evaluates that which good minds know by a sort of instinct, without being able to explain how with precision" (p. 196). The available mathematical tools, in particular the theorems of Bayes and Bernoulli, were seen as descriptions of actual human judgment (Daston, 1981,1988). However, the years of political upheaval during the French Revolution prompted Laplace, unlike earlier writers such as Condorcet, to issue repeated disclaimers that probability theory, because of the interference of passion and desire, could not account for all relevant factors in human judgment. The Enlightenment view—that the laws of probability are the laws of the mind—moderated as it was through the French Revolution, had a profound influence on 19th- and 20th-century science. This view became the starting point for seminal contributions to mathematics, as when George Boole

Nonlinear Models for Repeated Measurement Data

Abstract: Introduction. Nonlinear regression models for individual data. Hierarchical linear models. Hierarchical nonlinear models. Inference based on individual estimates. Inference based on linearization. Nonperametric and semiparametric inference. Bayesian inference. Pharamcokinetic and pharamacodynamic analysis. Analysis of assay data. Further applications. Open problems and discussion. References. Indices.

Bayesian Computation and Stochastic Systems

Abstract: Markov chain Monte Carlo (MCMC) methods have been used extensively in statistical physics over the last 40 years, in spatial statistics for the past 20 and in Bayesian image analysis over the last decade. In the last five years, MCMC has been introduced into significance testing, general Bayesian inference and maximum likelihood estimation. This paper presents basic methodology of MCMC, emphasizing the Bayesian paradigm, conditional probability and the intimate relationship with Markov random fields in spatial statistics. Hastings algorithms are discussed, including Gibbs, Metropolis and some other variations. Pairwise difference priors are described and are used subsequently in three Bayesian applications, in each of which there is a pronounced spatial or temporal aspect to the modeling. The examples involve logistic regression in the presence of unobserved covariates and ordinal factors; the analysis of agricultural field experiments, with adjustment for fertility gradients; and processing of low-resolution medical images obtained by a gamma camera. Additional methodological issues arise in each of these applications and in the Appendices. The paper lays particular emphasis on the calculation of posterior probabilities and concurs with others in its view that MCMC facilitates a fundamental breakthrough in applied Bayesian modeling.

Model uncertainty, data mining and statistical inference

Abstract: This paper takes a broad, pragmatic view of statistical inference to include all aspects of model formulation. The estimation of model parameters traditionally assumes that a model has a prespecified known form and takes no account of possible uncertainty regarding the model structure. This implicitly assumes the existence of a 'true' model, which many would regard as a fiction. In practice model uncertainty is a fact of life and likely to be more serious than other sources of uncertainty which have received far more attention from statisticians. This is true whether the model is specified on subject-matter grounds or, as is increasingly the case, when a model is formulated, fitted and checked on the same data set in an iterative, interactive way. Modern computing power allows a large number of models to be considered and data-dependent specification searches have become the norm in many areas of statistics. The term data mining may be used in this context when the analyst goes to great lengths to obtain a good fit. This paper reviews the effects of model uncertainty, such as too narrow prediction intervals, and the non-trivial biases in parameter estimates which can follow data-based modelling. Ways of assessing and overcoming the effects of model uncertainty are discussed, including the use of simulation and resampling methods, a Bayesian model averaging approach and collecting additional data wherever possible. Perhaps the main aim of the paper is to ensure that statisticians are aware of the problems and start addressing the issues even if there is no simple, general theoretical fix.

Annealing Markov chain Monte Carlo with applications to ancestral inference

Abstract: Markov chain Monte Carlo (MCMC; the Metropolis-Hastings algorithm) has been used for many statistical problems, including Bayesian inference, likelihood inference, and tests of significance. Though the method generally works well, doubts about convergence often remain. Here we propose MCMC methods distantly related to simulated annealing. Our samplers mix rapidly enough to be usable for problems in which other methods would require eons of computing time. They simulate realizations from a sequence of distributions, allowing the distribution being simulated to vary randomly over time. If the sequence of distributions is well chosen, then the sampler will mix well and produce accurate answers for all the distributions. Even when there is only one distribution of interest, these annealing-like samplers may be the only known way to get a rapidly mixing sampler. These methods are essential for attacking very hard problems, which arise in areas such as statistical genetics. We illustrate the methods wi...

Statistics and the Evaluation of Evidence for Forensic Scientists

Abstract: Preface to the first edition. Preface to the second edition. Uncertainty in forensic science. Variation. The evaluation of evidence. Historical review. Bayesian inference. Sampling. Interpretation. Transfer evidence. Discrete data. Continuous data. Multivariate analysis. Fibres. DNA profiling. Bayesian networks. References. Notation. Cases.

Fractional Bayes factors for model comparison

Abstract: Bayesian comparison of models is achieved simply by calculation of posterior probabilities of the models themselves. However, there are difficulties with this approach when prior information about the parameters of the various models is weak. Partial Bayes factors offer a resoIution of the problem by setting aside part of the data as a training sampIe. The training sampIe is used to obtain an initiaI informative posterior distribution of the parameters in each model. Model comparison is then based on a Bayes factor calculated from the remaining data. Properties of partial Bayes factors are discussed, particularly in the context of weak prior information, and they are found to have advantages over other proposed methods of model comparison. A new variant of the partial Bayes factor, the fractional Bayes factor, is advocated on grounds of consistency, simplicity, robustness and coherence

Adaptive Rejection Metropolis Sampling Within Gibbs Sampling

Abstract: Gibbs sampling is a powerful technique for statistical inference. It involves little more than sampling from full conditional distributions, which can be both complex and computationally expensive to evaluate. Gilks and Wild have shown that in practice full conditionals are often log‐concave, and they proposed a method of adaptive rejection sampling for efficiently sampling from univariate log‐concave distributions. In this paper, to deal with non‐log‐concave full conditional distributions, we generalize adaptive rejection sampling to include a Hastings‐Metropolis algorithm step. One important field of application in which statistical models may lead to non‐log‐concave full conditionals is population pharmacokinetics. Here, the relationship between drug dose and blood or plasma concentration in a group of patients typically is modelled by using nonlinear mixed effects models. Often, the data used for analysis are routinely collected hospital measurements, which tend to be noisy and irregular. Consequently, a robust (t‐distributed) error structure is appropriate to account for outlying observations and/or patients. We propose a robust nonlinear full probability model for population pharmacokinetic data. We demonstrate that our method enables Bayesian inference for this model, through an analysis of antibiotic administration in new‐born babies.

Inference from a Deterministic Population Dynamics Model for Bowhead Whales

Abstract: We consider the problem of inference about a quantity of interest given different sources of information linked by a deterministic population dynamics model. Our approach consists of translating all the available information into a joint premodel distribution on all the model inputs and outputs and then restricting this to the submanifold defined by the model to obtain the joint postmodel distribution. Marginalizing this yields inference, conditional on the model, about quantities of interest, which can be functions of model inputs, model outputs, or both. Samples from the postmodel distribution are obtained by importance sampling and Rubin's SIR algorithm. The framework includes as a special case the situation where the pre-model information about the outputs consists of measurements with error; this reduces to standard Bayesian inference. The results are in the form of a sample from the postmodel distribution and so can be examined using the full range of exploratory data analysis techniques. M...

Bayesian Inference for Stable Distributions

Abstract: Very little work on stable distribution parameter estimation and inference appears in the literature due to the nonexistence of the probability density function. This has led in particular to a dearth of Bayesian work in this area. But Bayesian computation via Markov chain Monte Carlo allows us to sample from the distribution of the parameters of the stable distributions, by exploiting a particular mathematical representation involving the stable density.

Bayesian inference of threshold autoregressive models

Abstract: . The study of non-linear time series has attracted much attention in recent years. Among the models proposed, the threshold autoregressive (TAR) model and bilinear model are perhaps the most popular ones in the literature. However, the TAR model has not been widely used in practice due to the difficulty in identifying the threshold variable and in estimating the associated threshold value. The main focal point of this paper is a Bayesian analysis of the TAR model with two regimes. The desired marginal posterior densities of the threshold value and other parameters are obtained via the Gibbs sampler. This approach avoids sophisticated analytical and numerical multiple integration. It also provides an estimate of the threshold value directly without resorting to a subjective choice from various scatterplots. We illustrate the proposed methodology by using simulation experiments and analysis of a real data set.

On priors providing frequentist validity for Bayesian inference

Abstract: SUMMARY We derive the differential equation that a prior must satisfy if the posterior probability of a one-sided credibility interval for a parametric function and its frequentist probability agree up to O(n-1). This equation turns out to be identical with Stein's equation for a slightly different problem, for which also our method provides a rigorous justification. Our method is different in details from Stein's but similar in spirit to Dawid (1991) and Bickel & Ghosh (1990). Some examples are provided.

Suppressing Random Walks in Markov Chain Monte Carlo Using Ordered Overrelaxation

Abstract: Markov chain Monte Carlo methods such as Gibbs sampling and simple forms of the Metropolis algorithm typically move about the distribution being sampled via a random walk. For the complex, high-dimensional distributions commonly encountered in Bayesian inference and statistical physics, the distance moved in each iteration of these algorithms will usually be small, because it is difficult or impossible to transform the problem to eliminate dependencies between variables. The inefficiency inherent in taking such small steps is greatly exacerbated when the algorithm operates via a random walk, as in such a case moving to a point n steps away will typically take around n^2 iterations. Such random walks can sometimes be suppressed using ``overrelaxed'' variants of Gibbs sampling (a.k.a. the heatbath algorithm), but such methods have hitherto been largely restricted to problems where all the full conditional distributions are Gaussian. I present an overrelaxed Markov chain Monte Carlo algorithm based on order statistics that is more widely applicable. In particular, the algorithm can be applied whenever the full conditional distributions are such that their cumulative distribution functions and inverse cumulative distribution functions can be efficiently computed. The method is demonstrated on an inference problem for a simple hierarchical Bayesian model.

A Bayesian Integration of End-Use Metering and Conditional-Demand Analysis

Abstract: Traditional methods of estimating kilowatt end uses load profiles may face very serious multicollinearity issues. In this article, a Bayesian framework is proposed to combine end uses monitoring information with the aggregate-load/appliance data to allow load researchers to derive more accurate load shapes. Two variants are suggested: The first one uses the raw end-use metered data to construct the prior means and variances. The second method uses actual end-use data to construct the priors of the parameters characterizing the behavior of end uses of specific appliances. From a prediction perspective, the Bayesian methods consistently outperform the predictions generated from conventional conditional-demand formulation.

Bayesian Inference and the Analytic Continuation of Imaginary-Time Quantum Monte Carlo Data

Abstract: We present brief description of how methods of Bayesian inference are used to obtain real frequency information by the analytic continuation of imaginary-time quantum Monte Carlo data. We present the procedure we used, which is due to R. K. Bryan, and summarize several bottleneck issues.

Updating Uncertainty in an Integrated Risk Assessment: Conceptual Framework and Methods

Abstract: Bayesian methods are presented for updating the uncertainty in the predictions of an integrated Environmental Health Risk Assessment (EHRA) model. The methods allow the estimation of posterior uncertainty distributions based on the observation of different model outputs along the chain of the linked assessment framework. Analytical equations are derived for the case of the multiplicative lognormal risk model where the sequential log outputs (log ambient concentration, log applied dose, log delivered dose, and log risk) are each normally distributed. Given observations of a log output made with a normally distributed measurement error, the posterior distributions of the log outputs remain normal, but with modified means and variances, and induced correlations between successive log outputs and log inputs. The analytical equations for forward and backward propagation of the updates are generally applicable to sums of normally distributed variables. The Bayesian Monte-Carlo (BMC) procedure is presented to provide an approximate, but more broadly applicable method for numerically updating uncertainty with concurrent backward and forward propagation. Illustrative examples, presented for the multiplicative lognormal model, demonstrate agreement between the analytical and BMC methods, and show how uncertainty updates can propagate through a linked EHRA. The Bayesian updating methods facilitate the pooling of knowledge encoded in predictive models with that transmitted by research outcomes (e.g., field measurements), and thereby support the practice of iterative risk assessment and value of information appraisals.

Developments in Probabilistic Modelling with Neural Networks — Ensemble Learning

Abstract: Ensemble learning by variational free energy minimization is a framework for statistical inference in which an ensemble of parameter vectors is optimized rather than a single parameter vector. The ensemble approximates the posterior probability distribution of the parameters.

Bayesian Model Discrimination and Bayes Factors for Linear Gaussian State Space Models

Abstract: SUMMARY It is shown how to discriminate between different linear Gaussian state space models for a given time series by means of a Bayesian approach which chooses the model that minimizes the expected loss. A practical implementation of this procedure requires a fully Bayesian analysis for both the state vector and the unknown hyperparameters and is carried out by Markov chain Monte Carlo methods. An application to some non-standard situations such as testing hypotheses on the boundary of the parameter space, discriminating non-nested models and discrimination of more than two models is discussed in detail.

Bayesian sequential Gaussian simulation of lithology with non-linear data

Abstract: A multivariate stochastic simulation application that involves the mapping of a primary variable from a combination for sparse primary data and more densely sampled secondary data The method is applicable when the relationship between the simulated primary variable and one or more secondary variables is non-linear The method employs a Bayesian updating rule to build a local posterior distribution for the primary variable at each simulated location The posterior distribution is the product of a Gaussian kernel function obtained by simple kriging of the primary variable and a secondary probability function obtained directly from a scatter diagram between primary and secondary variables

Bayesian Inference and Portfolio Efficiency

Abstract: A Bayesian approach is used to investigate a sample's information about a portfolio's degree of inefficiency. With standard diffuse priors, posterior distributions for measures of portfolio inefficiency can concentrate well away from values consistent with efficiency, even when the portfolio is exactly efficient in the sample. The data indicate that the NYSE-AMEX market portfolio is rather inefficient in the presence of a riskless asset, although this conclusion is justified only after an analysis using informative priors. Including a riskless asset significantly reduces any sample's ability to produce posterior distributions supporting small degrees of inefficiency.

Bayesian inference for masked system lifetime data

Abstract: Estimating component and system reliabilities frequently requires using data from the system level. Because of cost and time constraints, however, the exact cause of system failure may be unknown. Instead, it may only be ascertained that the cause of system failure is due to a component in a subset of components. This paper develops methods for analysing such masked data from a Bayesian perspective. This work was motivated by a data set on a system unit of a particular type of IBM PS/2 computer. This data set is discussed and our methods are applied to it

Eliciting prior information to enhance the predictive performance of Bayesian graphical models

Abstract: Both knowledge-based systems and statistical models are typically concerned with making predictions about future observables. Here we focus on assessment of predictive performance and provide two techniques for improving the predictive performance of Bayesian graphical models. First, we present Bayesian model averaging, a technique for accounting for model uncertainty. Second, we describe a technique for eliciting a prior distribution for competing models from domain experts. We explore the predictive performance of both techniques in the context of a urological diagnostic problem.

Local sensitivity diagnostics for Bayesian inference

Abstract: We investigate diagnostics for quantifying the effect of small changes to the prior distribution over a k-dimensional parameter space. We show that several previously suggested diagnostics, such as the norm of the Frechet derivative, diverge at rate $n^{k/2}$ if the base prior is an interior point in the class of priors, under the density ratio topology. Diagnostics based on $\phi$-divergences exhibit similar asymptotic behavior. We show that better asymptotic behavior can be obtained by suitably restricting the classes of priors. We also extend the diagnostics to see how various marginals of the prior affect various marginals of the posterior.

Simplicity, scientific inference and econometric modelling

Abstract: Two issues are discussed in this paper. The first is whether a formal definition and justification of simplicity (parsimony) in scientific inference can be found, and whether an optimal level of simplicity is obtainable. A definition of simplicity is possible, as are the optimum conditions for the desired degree of simplicity. The model of inference used here relates Bayesian inference to algorithmic information theory. Simplicity is examined in the light of induction, the Duhem-Quine thesis, and bounded rationality. The second issue relates to the role that simplicity might play in econometric modeling. This is elucidated with some remarks on the 'general to specific' approach to modeling and discussions on the purpose of a model. Copyright 1995 by Royal Economic Society.

A Bayesian Method for Combining Results from Several Binomial Experiments

Abstract: The problem of combining information related to I binomial experiments, each having a distinct probability of success θ i , is considered Instead of using a standard exchangeable prior for θ; = (θ1, …, θ I ), we propose a more flexible distribution that takes into account various degrees of similarity among the θ i 's Using ideas developed by Malec and Sedransk, we consider a partition g of the experiments and take the θ i 's belonging to the same partition subset to be exchangeable and the θ i 's belonging to distinct subsets to be independent Next we perform Bayesian inference on θ; conditional on g Of course, one is typically uncertain about which partition to use, and so a prior distribution is assigned on a set of plausible partitions g The final inference on θ; is obtained by combining the conditional inferences according to the posterior distribution of g The methodology adopted in this article offers a wide flexibility in structuring the dependence among the θ i 's This allows the