Showing papers on "Bayesian inference published in 1987"

Multiple imputation for nonresponse in surveys

Abstract: Tables and Figures. Glossary. 1. Introduction. 1.1 Overview. 1.2 Examples of Surveys with Nonresponse. 1.3 Properly Handling Nonresponse. 1.4 Single Imputation. 1.5 Multiple Imputation. 1.6 Numerical Example Using Multiple Imputation. 1.7 Guidance for the Reader. 2. Statistical Background. 2.1 Introduction. 2.2 Variables in the Finite Population. 2.3 Probability Distributions and Related Calculations. 2.4 Probability Specifications for Indicator Variables. 2.5 Probability Specifications for (X,Y). 2.6 Bayesian Inference for a Population Quality. 2.7 Interval Estimation. 2.8 Bayesian Procedures for Constructing Interval Estimates, Including Significance Levels and Point Estimates. 2.9 Evaluating the Performance of Procedures. 2.10 Similarity of Bayesian and Randomization--Based Inferences in Many Practical Cases. 3. Underlying Bayesian Theory. 3.1 Introduction and Summary of Repeated--Imputation Inferences. 3.2 Key Results for Analysis When the Multiple Imputations are Repeated Draws from the Posterior Distribution of the Missing Values. 3.3 Inference for Scalar Estimands from a Modest Number of Repeated Completed--Data Means and Variances. 3.4 Significance Levels for Multicomponent Estimands from a Modest Number of Repeated Completed--Data Means and Variance--Covariance Matrices. 3.5 Significance Levels from Repeated Completed--Data Significance Levels. 3.6 Relating the Completed--Data and Completed--Data Posterior Distributions When the Sampling Mechanism is Ignorable. 4. Randomization--Based Evaluations. 4.1 Introduction. 4.2 General Conditions for the Randomization--Validity of Infinite--m Repeated--Imputation Inferences. 4.3Examples of Proper and Improper Imputation Methods in a Simple Case with Ignorable Nonresponse. 4.4 Further Discussion of Proper Imputation Methods. 4.5 The Asymptotic Distibution of (Qm,Um,Bm) for Proper Imputation Methods. 4.6 Evaluations of Finite--m Inferences with Scalar Estimands. 4.7 Evaluation of Significance Levels from the Moment--Based Statistics Dm and Dm with Multicomponent Estimands. 4.8 Evaluation of Significance Levels Based on Repeated Significance Levels. 5. Procedures with Ignorable Nonresponse. 5.1 Introduction. 5.2 Creating Imputed Values under an Explicit Model. 5.3 Some Explicit Imputation Models with Univariate YI and Covariates. 5.4 Monotone Patterns of Missingness in Multivariate YI. 5.5 Missing Social Security Benefits in the Current Population Survey. 5.6 Beyond Monotone Missingness. 6. Procedures with Nonignorable Nonresponse. 6.1 Introduction. 6.2 Nonignorable Nonresponse with Univariate YI and No XI. 6.3 Formal Tasks with Nonignorable Nonresponse. 6.4 Illustrating Mixture Modeling Using Educational Testing Service Data. 6.5 Illustrating Selection Modeling Using CPS Data. 6.6 Extensions to Surveys with Follow--Ups. 6.7 Follow--Up Response in a Survey of Drinking Behavior Among Men of Retirement Age. References. Author Index. Subject Index. Appendix I. Report Written for the Social Security Administration in 1977. Appendix II. Report Written for the Census Bureau in 1983.

Factor Analysis and AIC

Abstract: The information criterion AIC was introduced to extend the method of maximum likelihood to the multimodel situation. It was obtained by relating the successful experience of the order determination of an autoregressive model to the determination of the number of factors in the maximum likelihood factor analysis. The use of the AIC criterion in the factor analysis is particularly interesting when it is viewed as the choice of a Bayesian model. This observation shows that the area of application of AIC can be much wider than the conventional i.i.d. type models on which the original derivation of the criterion was based. The observation of the Bayesian structure of the factor analysis model leads us to the handling of the problem of improper solution by introducing a natural prior distribution of factor loadings.

The calculation of posterior distributions by data augmentation

Abstract: The idea of data augmentation arises naturally in missing value problems, as exemplified by the standard ways of filling in missing cells in balanced two-way tables. Thus data augmentation refers to a scheme of augmenting the observed data so as to make it more easy to analyze. This device is used to great advantage by the EM algorithm (Dempster, Laird, and Rubin 1977) in solving maximum likelihood problems. In situations when the likelihood cannot be approximated closely by the normal likelihood, maximum likelihood estimates and the associated standard errors cannot be relied upon to make valid inferential statements. From the Bayesian point of view, one must now calculate the posterior distribution of parameters of interest. If data augmentation can be used in the calculation of the maximum likelihood estimate, then in the same cases one ought to be able to use it in the computation of the posterior distribution. It is the purpose of this article to explain how this can be done. The basic idea ...

Estimation and Inference by Compact Coding

Abstract: SUMMARY The systematic variation within a set of data, as represented by a usual statistical model, may be used to encode the data in a more compact form than would be possible if they were considered to be purely random. The encoded form has two parts. The first states the inferred estimates of the unknown parameters in the model, the second states the data using an optimal code based on the data probability distribution implied by those parameter estimates. Choosing the model and the estimates that give the most compact coding leads to an interesting general inference procedure. In its strict form it has great generality and several nice properties but is computationally infeasible. An approximate form is developed and its relation to other methods is explored.

Asymptotic theory of statistical inference

Abstract: Probability and Stochastic Processes Limit Theorems for Some Statistics Asymptotic Theory of Estimation Linear Parametric Inference Martingale Approach to inference Inference in Non-Linear Regression Von-Mises Functionals Empirical Characteristic Function and Its Applications Index.

Belief Function Representations of Statistical Evidence

Abstract: In Glenn Shafer's theory of parametric statistical inference, observational evidence and prior evidence are separately represented by belief or commonality functions $Q$ and $R$, which are then combined by Dempster's rule. We characterise, for finite parameter spaces, the functionals $Q$ and $R$ for which statistically independent observations may be combined by Dempster's rule, and those for which Dempster's rule is consistent with Bayes' rule. The functionals are determined up to an arbitrary partition of the parameter space and an arbitrary scale parameter, which might be chosen to reflect aspects of the evidence on which the statistical model is based. Our results suggest that Dempster's rule is not generally suitable for combining evidence from independent observations nor for combining prior beliefs with observational evidence.

Progress with numerical and graphical methods for practical Bayesian statistics

Abstract: One of the main obstacles to the routine implementation of Bayesian methods has been the absence of efficient algorithms for carrying out the computational tasks implicit in the Bayesian approach. In this paper, recent progress towards overcoming this problem is reviewed. In particular, novel numerical integration and interpolation methods, which exploit the opportunities offered by modern interactive computing and graphics facilities, are outlined and illustrated.

Bayesian inference in model-based machine vision

Abstract: We present a thorough integration of hierarchical Bayesian inference with comprehensive physical representation of objects and their relations in a system for reasoning with geometry in machine vision. Bayesian inference provides a framework for accruing probabilities to rank order hypotheses. This is a preliminary version of visual interpretation in SUCCESSOR, an intelligent, model-based vision system integrating multiple sensors.

Convince: A Conversational Inference Consolidation Engine

Abstract: An operational domain-independent decision-aiding system for situation assessment tasks is described. The system elicits the user's perception of a given situation through a stylized English dialogue and focuses the user's attention on the issue of highest relevancy. Elicited problems are structured as networks where nodes represent variables and directed links represent causal relationships. The system uses a Bayesian inference procedure which combines causal and diagnostic reasoning using a bidirectional propagation of evidence in the form of belief parameters. Upon completion of the dialogue the system provides a formal structure representing relevant propositions, their interrelations, and their updated belief distributions.

Bayesian Inference and Forensic Science: Problems and Perspectives

Abstract: The debate about whether or not Bayesian inference provides a model for the process of assessing evidence in a court of law has generated a good amount of literature and some strongly voiced opinions. Let me start by making my own position clear: I am a forensic scientist and I am interested in the modelling of the legal process only to the extent that it enables the role of the scientist's evidence to be defined. The broader arguments centre on concepts such as the odds on guilt, but my professional colleagues jealously guard their detachment from the deliberations of guilt or otherwise and, personally, I am reluctant to handle equations which contain guilt probabilities. I shall concentrate on the problem of evaluating forensic science evidence against two narrower alternatives: C, the event that the suspect was at the crime scene and C, the event that the suspect was not at the crime scene. The great advantage of Bayesian inference is that it enables us to identify and, in principle, to answer the most appropriate range of questions which the scientist should address to be of greatest assistance to the investigator or to the court. Bayes Theorem shows us that, while the investigator or court is concerned with questions of the type: "what is the probability that the suspect was at the crime scene?", the scientist, through the likelihood ratio, should address questions of the type "what is the probability of the evidence given that the suspect was at the crime scene?" and "what is the probability of the evidence given that the suspect was not at the crime scene?". While this might appear almost self-evident to a practising Bayesian, it is only recently that forensic scientists have begun to be converted. Nevertheless there is now a growing realisation that Bayesian methods have something to offer and I am optimistic that over the coming years we can do a lot to fan that small flame. I propose next to describe the essentials of the forensic science transfer problem; then to describe briefly some of the work that has been done so far on developing solutions; then to look at ways in which developments could take place from here, emphasising the major challenges that are to be faced.

Bayesian Methods for Censored Categorical Data

Abstract: Bayesian methods are given for finite-category sampling when some of the observations suffer missing category distinctions. Dickey's (1983) generalization of the Dirichlet family of prior distributions is found to be closed under such censored sampling. The posterior moments and predictive probabilities are proportional to ratios of B. C. Carlson's multiple hypergeometric functions. Closed-form expressions are developed for the case of nested reported sets, when Bayesian estimates can be computed easily from relative frequencies. Effective computational methods are also given in the general case. An example involving surveys of death-penalty attitudes is used throughout to illustrate the theory. A simple special case of categorical missing data is a two-way contingency table with cross-classified count data xij (i = 1, …, r; j = 1, …, c), together with supplementary trials counted only in the margin distinguishing the rows, yi (i = 1, …, r). There could also be further supplementary trials report...

Bayes Linear Estimators for Randomized Response Models

Abstract: Bayes linear estimators provide simple Bayesian methods and require a minimum of prior specification. In this article, Bayes linear estimators are derived for a variety of randomized response models. Randomized response aims to reduce false responses on sensitive questions, at the expense of some loss of information in each observation. In this context, Bayesian methods are attractive because they permit the incorporation of potentially useful prior information. The basic principle of randomized response is that an interviewee answers one of two or more different questions, depending on the outcome of some randomizing device. The interviewer does not know which question has been answered. In this way, it is hoped, the interviewee will feel able to answer sensitive questions honestly, where direct questioning might have produced false responses. Two versions of randomized response are examined: the original formulation of Warner (1965) and the unrelated question method of Simmons (Horvitz, Shah, a...

Is it Necessary to Develop a Fuzzy Bayesian Inference

Abstract: In applications data used for updating a-priori information are often fuzzy. These fuzzy data are usually not described by standard Bayesian inference. Statistical analysis has to take care of this fuzzyness which can be described by fuzzy numbers. Therefore the resulting fuzzyness of a-posteriori distributions has to be modelled and an analogue of predictive distributions under fuzzyness must be developed. Moreover for a fuzzy observation it is not always possible to decide if it is a member of a certain event. This kind of uncertainty states the following question: Is additivity for the measurement of uncertainty in general valid or a generalization of probability, postulating superadditivity, necessary.

Precision, Complexity and Bayesian Model Determination

Abstract: SUMMARY In this paper a new criterion for model determination is proposed. An explicit decision theoretic approach to model selection is employed to derive a Bayes decision rule. The operational characteristics of the criterion are discussed and consistency is shown.

Regularization uses fractal priors

Abstract: Many of the processing tasks arising in early vision involve the solution of ill-posed inverse problems. Two techniques that are often used to solve these inverse problems are regularization and Bayesian modeling. Regularization is used to find a solution that both fits the data and is also sufficiently smooth. Bayesian modeling uses a statistical prior model of the field being estimated to determine an optimal solution. One convenient way of specifying the prior model is to associate an energy function with each possible solution, and to use a Boltzmann distribution to relate the solution energy to its probability. This paper shows that regularization is an example of Bayesian modeling, and that using the regularization energy function for the surface interpolation problem results in a prior model that is fractal (self-affine over a range of scales). We derive an algorithm for generating typical (fractal) estimates from the posterior distribution. We also show how this algorithm can be used to estimate the uncertainty associated with a regularized solution, and how this uncertainty can be used at later stages of processing.

A Bayes Reliability Growth Model for A Development Testing Program

Abstract: The problem of estimating the reliability of a system during development is considered. The development process has several stages at each stage binomial test data are obtained by testing a number of such systems on a success/fail basis. Marginal posterior distributions are derived under the assumption that the development process constrains the reliabilities to be nondecreasing and that the prior distribution for reliability at each stage is uniform. Simulation models are designed to facilitate testing for the validity and computation of the Bayesian model with ordered reliabilities as well as to compare results with other reliability growth models.

A Bayesian Model to Predict Saturation and Logistic Growth

Abstract: This paper formulates a Bayesian model to predict growth and eventual market saturation of a recently introduced (and possibly expensive) consumer-durable product. The mathematical model assumes that each new buyer buys only one item of the product, that the number of new buyers of the product in the next period of time is influenced by the current number of non-buyers and that the probability an individual will buy is the result of a diffusion of news among satisfied buyers. The solution of the prediction problem includes a two-stage Bayesian updating formula which first revises the prior distribution of market saturation based on the most recent number of new buyers and then, conditional on the saturation level, computes the predictive distribution of new buyers in future time-periods.

An Approach to Inference following Model Selection with Applications to Transformation-Based and Adaptive Inference

Abstract: A decision-theory approach is formulated for inference following model selection. Inference here refers to either testing, point estimation, or confidence estimation. The loss function includes components for model selection as well as for inference and allows for flexibility in emphasis on one or the other, if such emphasis is desired. A general prescription for Bayes and generalized Bayes procedures is given. A procedure consists of model selection and inference. The general formulation is applied to transformation-based inference, where model selection is equated to choice of transformation. Hinkley and Runger (1984) did transformation-based inference that has aroused controversy. The formulation here is directed to some of these issues. In this approach we explicitly define the quantity of interest for which an inference is desired. Furthermore, we evaluate procedures properly. An example is given where one is interested in estimating the mean of the model selected and the choice of models is...

Bayesian theory and artificial intelligence: the quarrelsome marriage

Abstract: The problem of knowledge-base updating is addressed from an abstract point of view in the attempt to identify some general desiderata the updating mechanism should satisfy. They are recognized to be basically two: evaluating the local impact of new data on the single items of knowledge already stored, and propagating this effect through the knowledge-base maintaining at the same time its global coherence. It will be shown that Bayesian updating, difficult to implement, satisfies simultaneously these two requirements, and that, on the other hand, Dempster—Shafer updating, easy to implement, does not satisfy the requirement of global coherent propagation. I will point out the existence of a trade-off between coherence and effectiveness in the methods for representing uncertainty currently proposed in AI. Two kinds of learning machines, Boltzmann machines and Harmonium, will be discussed and considered as first attempts to give a non-behavioral characterization of coherence in a cognitive agent, a characterization still consistent with the behavioral (probabilistic) definition.

Inferno: A cautious approach to uncertain inference

Abstract: : Expert systems commonly employ some means of drawing inferences from domain and problem knowledge, where both the knowledge and its implications are less than certain. Methods used include subjective Bayesian reasoning, measures of belief and disbelief, and the Dempster-Shafer theory of evidence. Analysis of systems based on these methods reveals important deficiencies in areas such as the reliability of deductions and the ability to detect inconsistencies in the knowledge from which deductions were made. A new system call INFERNO addresses some of these points. Its approach is probabilistic but makes no assumptions whatsoever about the joint probability distributions of pieces of knowledge, so the correctness of inferences can be guaranteed. INFERNO informs the user of inconsistencies that may be present in the information presented to it, and can make suggestions about changing the information to make it consistent. An example from a Bayesian system is reworked, and the conclusions reached by that system and INFERNO are compared.

Optimal allocation for Bayesian inference about an odds ratio

Abstract: SUMMARY The choice of ratio of sample sizes of two groups of binary observations for comparing two proportions is studied using a Bayesian approach. Prior uncertainty about the unknown proportions thus enters the design calculations.

The classification problem with autoregressive process

Abstract: We consider the problem of assigning a realization into one of several autoregressive soursces that share a common known order and unknown error variance. The approach is to use an informal Bayesian inference based on the marginal posterior distribution of the classification vector. A realization is assigned to that actoregessive process with the largest posterior probability, and an example demomtrates the classification technique behaves in a reasonable way. A generalization is developed.

An approach to the nonstationary process analysis

Abstract: A Bayesian approach to nonstationary process analysis is proposed. Given a set of data, it is divided into several blocks with the same length, and in each block an autoregressive model is fitted to the data. A constraint on the autoregressive coefficients of the successive blocks is considered. This constraint controls the smoothness of the temporal change of spectrum as shown in Section 2. A smoothness parameter, which is called a hyper parameter in this article, is determined with the aid of the minimum ABIC (Akaike Bayesian Information Criterion) procedure. Numerical examples of our procedure are also given.