Showing papers on "Bayesian inference published in 1991"

Bayesian networks without tears: making Bayesian networks more accessible to the probabilistically unsophisticated

Abstract: I give an introduction to Bayesian networks for AI researchers with a limited grounding in probability theory. Over the last few years, this method of reasoning using probabilities has become popular within the AI probability and uncertainty community. Indeed, it is probably fair to say that Bayesian networks are to a large segment of the AI-uncertainty community what resolution theorem proving is to the AIlogic community. Nevertheless, despite what seems to be their obvious importance, the ideas and techniques have not spread much beyond the research community responsible for them. This is probably because the ideas and techniques are not that easy to understand. I hope to rectify this situation by making Bayesian networks more accessible to the probabilistically unsophisticated.

Efficient Simulation from the Multivariate Normal and Student-t Distributions Subject to Linear Constraints and the Evaluation of Constraint Probabilities

Abstract: John GewekeDepartment of EconomicsUniversity of MinnesotaMinneapolis, MN 55455First draft: April, 1991Phone: (612)625-7563 Fax: (612)624-0209E-mail: geweke@atlas.socsci.umn.eduAbstractThe construction and implementation of a Gibbs sampler for efficient simulation from thetruncated multivariate normal and Student-t distributions is described. It is shown how theaccuracy and convergence of integrals based on the Gibbs sample may be constructed, andhow an estimate of the probability of the constraint set under the unrestricted distributionmay be produced.Keywords: Bayesian inference; Gibbs sampler; Monte Carlo; multiple integration;truncated normalThis paper was prepared for a presentation at the meeting Computing Science and Statistics:the Twenty-Third Symposium on the Interface, Seattle, April 22-24, 1991. Researchassistance from Zhenyu Wang and financial support from National Science FoundationGrant SES-8908365 are gratefully acknowledged. The software for the examples may berequested by electronic mail, and will be returned by that medium.

Is human cognition adaptive

Abstract: Can the output of human cognition be predicted from the assumption that it is an optimal response to the information-processing demands of the environment? A methodology called rational analysis is described for deriving predictions about cognitive phenomena using optimization assumptions. The predictions flow from the statistical structure of the environment and not the assumed structure of the mind. Bayesian inference is used, assuming that people start with a weak prior model of the world which they integrate with experience to develop stronger models of specific aspects of the world. Cognitive performance maximizes the difference between the expected gain and cost of mental effort. (1) Memory performance can be predicted on the assumption that retrieval seeks a maximal trade-off between the probability of finding the relevant memories and the effort required to do so; in (2) categorization performance there is a similar trade-off between accuracy in predicting object features and the cost of hypothesis formation; in (3) casual inference the trade-off is between accuracy in predicting future events and the cost of hypothesis formation; and in (4) problem solving it is between the probability of achieving goals and the cost of both external and mental problem-solving search. The implemention of these rational prescriptions in neurally plausible architecture is also discussed.

A Monte Carlo method for Bayesian inference in frailty models.

Abstract: Many analyses in epidemiological and prognostic studies and in studies of event history data require methods that allow for unobserved covariates or "frailties." Clayton and Cuzick (1985, Journal of the Royal Statistical Society, Series A 148, 82-117) proposed a generalization of the proportional hazards model that implemented such random effects, but the proof of the asymptotic properties of the method remains elusive, and practical experience suggests that the likelihoods may be markedly nonquadratic. This paper sets out a Bayesian representation of the model in the spirit of Kalbfleisch (1978, Journal of the Royal Statistical Society, Series B 40, 214-221) and discusses inference using Monte Carlo methods.

To criticize the critics: An objective bayesian analysis of stochastic trends

Abstract: In two recent articles, Sims (1988) and Sims and Uhlig (1988/1991) question the value of much of the ongoing literature on unit roots and stochastic trends. They characterize the seeds of this literature as ‘sterile ideas’, the application of nonstationary limit theory as ‘wrongheaded and unenlightening’, and the use of classical methods of inference as ‘unreasonable’ and ‘logically unsound’. They advocate in place of classical methods an explicit Bayesian approach to inference that utilizes a flat prior on the autoregressive coefficient. DeJong and Whiteman adopt a related Bayesian approach in a group of papers (1989a,b,c) that seek to re-evaluate the empirical evidence from historical economic time series. Their results appear to be conclusive in turning around the earlier, influential conclusions of Nelson and Plosser (1982) that most aggregate economic time series have stochastic trends. So far these criticisms of unit root econometrics have gone unanswered; the assertions about the impropriety of classical methods and the superiority of flat prior Bayesian methods have been unchallenged; and the empirical re-evaluation of evidence in support of stochastic trends has been left without comment. This paper breaks that silence and offers a new perspective. We challenge the methods, the assertions, and the conclusions of these articles on the Bayesian analysis of unit roots. Our approach is also Bayesian but we employ what are known in the statistical literature as objective ignorance priors in our analysis. These are developed in the paper to accommodate explicitly time series models in which no stationarity assumption is made. Ignorance priors are intended to represent a state of ignorance about the value of a parameter and in many models are very different from flat priors. We demonstrate that in time series models flat priors do not represent ignorance but are actually informative (sic) precisely because they neglect generically available information about how autoregressive coefficients influence observed time series characteristics. Contrary to their apparent intent, flat priors unwittingly bias inferences towards stationary and i.i.d. alternatives where they do represent ignorance, as in the linear regression model. This bias helps to explain the outcome of the simulation experiments in Sims and Uhlig and some of the empirical results of DeJong and Whiteman. Under both flat priors and ignorance priors this paper derives posterior distributions for the parameters in autoregressive models with a deterministic trend and an arbitrary number of lags. Marginal posterior distributions are obtained by using the Laplace approximation for multivariate integrals along the lines suggested by the author (Phillips, 1983) in some earlier work. The bias towards stationary models that arises from the use of flat priors is shown in our simulations to be substantial; and we conclude that it is unacceptably large in models with a fitted deterministic trend, for which the expected posterior probability of a stochastic trend is found to be negligible even though the true data generating mechanism has a unit root. Under ignorance priors, Bayesian inference is shown to accord more closely with the results of classical methods. An interesting outcome of our simulations and our empirical work is the bimodal Bayesian posterior, which demonstrates that Bayesian confidence sets can be disjoint, just like classical confidence intervals that are based on asymptotic theory. The paper concludes with an empirical application of our Bayesian methodology to the Nelson-Plosser series. Seven of the 14 series show evidence of stochastic trends under ignorance priors, whereas under flat priors on the coefficients all but three of the series appear trend stationary. The latter result corresponds closely with the conclusion reached by DeJong and Whiteman (1989b) (based on truncated flat priors). We argue that the DeJong-Whiteman inferences are biased towards trend stationarity through the use of flat priors on the autoregressive coefficients, and that their inferences for some of the series (especially stock prices) are fragile (i.e. not robust) not only to the prior but also to the lag length chosen in the time series specification.

Statistical inference for branching processes

Abstract: Using one of the simplest possible cases, this book illustrates the problems of statistical inference that can arise when the standard assumption of independent observations is relaxed. It contains observations on the generation sizes of a Bienayme-Galton-Watson process. In addition, problems of conditional inference and ancilliarity in stochastic processes are presented in this context, both for estimation and testing problems. Throughout the book, theoretical results are illustrated in special cases and scientific applications of the theory and method are presented. A chapter has been devoted to Bayesian methods of inference.

On Bayesian routes to unit roots

Abstract: textThis paper is a comment on P C B Phillips, `To criticise the critics: an objective Bayesian analysis of stochastic trends' [Phillips, (1991)] Departing from the likelihood of an univariate autoregressive model different routes that lead to a posterior odds analysis of the unit root hypothesis are explored, where the differences in routes are due to the different choices of the prior Improper priors like the uniform and the Jeffreys prior are less suited for Bayesian inference on a sharp null hypothesis as the unit root A proper normal prior on the mean of the process is analysed and empirical results using extended Nelson-Plosser data are presented

Bayesian classification theory

Abstract: The task of inferring a set of classes and class descriptions most likely to explain a given data set can be placed on a firm theoretical foundation using Bayesian statistics. Within this framework and using various mathematical and algorithmic approximations, the AutoClass system searches for the most probable classifications, automatically choosing the number of classes and complexity of class descriptions. A simpler version of AutoClass has been applied to many large real data sets, has discovered new independently-verified phenomena, and has been released as a robust software package. Recent extensions allow attributes to be selectively correlated within particular classes, and allow classes to inherit or share model parameters though a class hierarchy. We summarize the mathematical foundations of AutoClass.

Bayesian analysis in inverse problems

Abstract: Considers some statistical aspects of inverse problems, using Bayesian analysis, particularly estimation and hypothesis-testing questions for parameter-dependent differential equations. The author relates Bayesian maximum likelihood to Tikhonov regularization and applies the expectation-minimization algorithm to the problem of setting regularization levels. Further, he compares Bayesian results with those of a classical statistical approach, through consistency and asymptotic normality. A numerical example illustrates the application of Bayesian techniques. In many cases one is interested in parameters which are infinite dimensional (e.g. functions). Bayesian techniques offer a sound theoretical and computational paradigm, through probability measures on Banach space. He develops a framework for infinite dimensional Bayesian analysis, including convergence of approximations required to perform inference tasks computationally.

Notes on present status and future prospects

Abstract: We note some general features concerning present activity in Maximum Entropy and Bayesian inference, and try to foresee how they may develop in the future We see ahead great promise, but also potential dangers

Medical expert systems based on causal probabilistic networks

Abstract: Causal probabilistic networks (CPNs) offer new methods by which you can build medical expert systems that can handle all types of medical reasoning within a uniform conceptual framework. Based on the experience from a commercially available system and a couple of large prototype systems, it appears that CPNs are now an attractive alternative to other methods. A CPN is an intensional model of a domain. and it is therefore conceptually much closer to qualitative reasoning systems and to simulation systems than to rule-based or logic-based systems. Recent progress in Bayesian inference in networks has yielded computationally efficient methods. The inference method used follows the fundamental axioms of probability theory, and gives a sound framework for causal and diagnostic (deductive and abductive) reasoning under uncertainty. Experience with the prototypes indicates that it may be possible to use decision theory as a rational approach to test planning and therapy planning. The way in which knowledge is acquired and represented in CPNs makes it easy to express ‘deep knowledge’ for example in the form of physiological models, and the facilities for learning make it possible to make a smooth transition from expert opinion to statistics based on empirical data.

Bayesian learning of Gaussian mixture densities for hidden Markov models

Abstract: An investigation into the use of Bayesian learning of the parameters of a multivariate Gaussian mixture density has been carried out. In a continuous density hidden Markov model (CDHMM) framework, Bayesian learning serves as a unified approach for parameter smoothing, speaker adaptation, speaker clustering, and corrective training. The goal of this study is to enhance model robustness in a CDHMM-based speech recognition system so as to improve performance. Our approach is to use Bayesian learning to incorporate prior knowledge into the CDHMM training process in the form of prior densities of the HMM parameters. The theoretical basis for this procedure is presented and preliminary results applying to HMM parameter smoothing, speaker adaptation, and speaker clustering are given.Performance improvements were observed on tests using the DARPA RM task. For speaker adaptation, under a supervised learning mode with 2 minutes of speaker-specific training data, a 31% reduction in word error rate was obtained compared to speaker-independent results. Using Baysesian learning for HMM parameter smoothing and sex-dependent modeling, a 21% error reduction was observed on the FEB91 test.

Bayesian methods in global optimization

Abstract: This paper reviews methods which have been proposed for solving global optimization problems in the framework of the Bayesian paradigm.

Bayesian analyses for a multiple capture-recapture model

Abstract: SUMMARY In multiple capture-recapture surveys, the probability of capture can vary between sampling occasions. The model accounting for this variation is known as At. Bayes, empirical Bayes, and Bayes empirical Bayes solutions are given to the problems of interval estimation, decision making, and point estimation of the population size N. When the number of sampling occasions is small to moderate and the number of recaptured units observed on each sampling occasion is moderate, estimates obtained from empirical Bayes and Bayes empirical Bayes methods compare closely to Bayesian methods using a reference prior distribution for the capture probabilities. However, when the number of sampling occasions is large and the number of recaptured units observed on each sampling occasion is small, inferences obtained using different reference priors can differ considerably.

Bayesian Model Comparison and Backprop Nets

Abstract: The Bayesian model comparison framework is reviewed, and the Bayesian Occam's razor is explained. This framework can be applied to feedforward networks, making possible (1) objective comparisons between solutions using alternative network architectures; (2) objective choice of magnitude and type of weight decay terms; (3) quantified estimates of the error bars on network parameters and on network output. The framework also generates a measure of the effective number of parameters determined by the data. The relationship of Bayesian model comparison to recent work on prediction of generalisation ability (Guyon et al., 1992, Moody, 1992) is discussed.

Gibbs Sampling for Marginal Posterior Expectations

Abstract: In earlier work (Gelfand and Smith, 1990 and Gelfand et al, 1990) a sampling based approach using the Gibbs sampler was offered as a means for developing marginal posterior densities for a wide range of Bayesian problems several of which were previously inaccessible. Our purpose here is two-fold. First we flesh out the implementation of this approach for calculation of arbitrary expectations of interest. Secondly we offer comparison with perhaps the most prominent approach for calculating posterior expectations, analytic approximation involving application of the LaPlace method. Several illustrative examples are discussed as well. Clear advantages for the sampling based approach emerge.

Fisherian Inference in Likelihood and Prequential Frames of Reference

Abstract: SUMMARY In celebration of the centenary of the birth of Sir Ronald Fisher, this paper explores Fisher's conception of statistical inference, with special attention to the importance he placed on choosing an appropriate frame of reference to define the inferential model. In particular, we investigate inferential models which respect the likelihood principle or the prequential principle, and argue that these will typically have an asymptotic sampling theory justification.

An iterative Monte Carlo method for nonconjugate Bayesian analysis

Abstract: The Gibbs sampler has been proposed as a general method for Bayesian calculation in Gelfand and Smith (1990). However, the predominance of experience to date resides in applications assuming conjugacy where implementation is reasonably straightforward. This paper describes a tailored approximate rejection method approach for implementation of the Gibbs sampler when nonconjugate structure is present. Several challenging applications are presented for illustration.

Bayesian Inference with Specified Prior Marginals

Abstract: We show how to find bounds on posterior expectations of arbitrary functions of the parameters when the prior marginals are specified but when the complete joint prior is unspecified. We also give a theorem that is useful for finding posterior bounds in a wide range of Bayesian robustness problems. We apply these techniques to two examples. The first example involves a recent clinical trial for extracorporeal membrane oxygenation (ECMO). Our analysis may be regarded as a follow-up to a detailed Bayesian analysis given by Kass and Greenhouse who concluded that the posterior probability that the treatment is superior to the control is about .95. Their analysis, however, assumed a priori independence of the parameters. We consider other prior distributions with the same marginals as Kass and Greenhouse, but in which the parameters are not independent and conclude that, as long as a priori independence is at least approximately tenable, then ECMO seems superior to the control. The second example is th...

Noninformative priors for inferences in exponential regression models

Abstract: SUMMARY In the exponential regression model, inference concerning the regression parameter is notoriously difficult, even when using the Bayesian noninformative prior approach. The reference prior approach (Bernardo, 1979; Berger & Bernardo, 1989) is considered, and argued to yield very satisfactory inferences. Estimation and credible sets are considered in a specific example.

Bayesian and frequentist predictive inference for the patterns of care studies

Abstract: The Patterns of Care Studies were conducted to determine the quality of care received by cancer patients whose primary treatment modality is radiation therapy In this article, we propose and evaluate models which, if acceptable, permit Bayesian and frequentist model-based predictive inference for the desired finite population parameters Using both hierarchical Bayesian and frequentist mixed linear models, we describe methodology for making the desired inferences, emphasizing the use of transformed random variables Finally, we compare the frequentist, Bayes, and empirical Bayes approaches using data from one of the surveys All three methods produce essentially the same value for the (finite population) mean The standard empirical Bayes and frequentist measures of variability are very much smaller than those derived from the Bayesian approach, the latter reflecting uncertainty about the values of the scale parameters in the model

On Bayes' theorem for fuzzy data

Abstract: There are some ideas concerning a generalization of Bayes' theorem to the situation of fuzzy data. Some of them are given in the references [1], [5], and [7]. But the proposed methods are not generalizations in the sense of the probability content of Bayes' theorem for precise data. In the present paper a generalization of Bayes' theorem to the case of fuzzy data is described which contains Bayes' theorem for precise data as a special case and allows to use the information in fuzzy data in a coherent way. Moreover a generalization of the concept of HPD-regions is explained which makes it possible to model and analyze the situation of fuzzy data. Also a generalization of the concept of predictive distributions is given in order to calculate predictive densities based on fuzzy sample information.

Qualitative probability versus quantitative probability in clinical diagnosis: a study using a computer simulation.

Abstract: The use of Bayes' theorem as a diagnostic tool in clinical medicine normally requires an input of exact probability estimates. However, humans tend to think in categories ("likely," "unlikely," etc.) rather than in terms of exact probability. A computer simulation of the presenting features of a case of pelvic infection has been used to compare the effects of quantitative and qualitative probability estimates on the diagnostic accuracy of Bayes' theorem. For the commoner conditions (prior probability greater than or equal to 0.2) the use of a two- or three-category system is virtually equivalent to the use of exact probability. However, uncommon conditions (prior probability less than or equal to 0.03) are completely ignored by the qualitative system. It is concluded that the use of simple categories of probability is acceptable for a Bayesian diagnostic system provided that the target conditions have a relatively high prior probability.

Coherent Statistical Inference and Bayes Theorem

Abstract: Conditions are given which suffice for the assessment of a coherent inference by means of a Bayesian algorithm, i.e., a suitable extension of the classical Bayes theorem relative to a finite number of alternatives. Under some further hypotheses such inference is shown to be, in addition, coherent in the sense of Heath, Lane and Sudderth. Moreover, a characterization of coherent posteriors is provided, together with some remarks concerning finitely additive conditional probabilities.

Bayesian reasoning in medicine: the contributions of Lee B. Lusted, MD.

Abstract: The role of Bayesian reasoning in medicine is explored from the perspective of the writings of Dr. Lee B. Lusted. Starting with the influential article by Ledley and Lusted published in Science in ...

An accurate method for approximate conditional and Bayesian inference about linear regression models from censored data

Abstract: SUMMARY A method to approximate marginal posterior distribution functions of regression coefficients and scale parameters is presented. It is based on a general asymptotic formula of DiCiccio, Field & Fraser (1990) for approximating marginal tail probabilities. In developing the method, it is shown that the accuracy of the approximation of DiCiccio et al. can be substantially increased in certain cases by using a suitable change of variables, and a general approach to deriving such transformations is considered.

Recursive Bayesian algorithms for blind equalization

Abstract: A novel blind equalization algorithm based on a suboptimum Bayesian symbol sequence estimator is presented. It is shown that a parallel bank of Kalman filters can be used to update a suboptimum Bayesian formula for the sequence possibilities. Two methods are used to reduce the computational complexity of the algorithm. First, it is shown that the Kalman filters can be replaced by simpler least-mean-square (LMS) adaptive filters. Second, the technique of reduced-state sequence estimation is adopted to reduce the number of symbol subsequences considered in the Bayesian updating, and hence the number of parallel filters required. The performance properties of the resulting algorithms are evaluated through bit error simulations, and these are compared to the bounds of optimum maximum-likelihood sequence estimation. It is shown that the Kalman filter and LMS-based algorithms achieve blind start-up and rapid convergence (within 200 iterations) for both binary phase-shift keying (BPSK) and quadrature phase-shift keying (QPSK) modulation formats. >

What Bayesians expect of each other

Abstract: Our goal is to study general properties of one Bayesian's subjective beliefs about the behavior of another Bayesian's subjective beliefs. We consider two Bayesians, A and B, who have different subjective distributions for a parameter θ, and study Bayesian A's expectation of Bayesian B's posterior distribution for θ given some data Y. We show that when θ can take only two values, Bayesian A always expects Bayesian B's posterior distribution to lie between the prior distributions of A and B. Conditions are given under which a similar result holds for an arbitrary real-valued parameter θ. For a vector parameter θ we present useful expressions for the mean vector and covariance matrix of A's expectation of B's posterior distribution. Examples are given illustrating the relevance of the conditions under which the results are derived.