https://rss.onlinelibrary.wiley.com/doi/pdf/10.1111/j.2517-6161.1977.tb01600.x

Maximum likelihood from incomplete data via the EM algorithm

This paper considers Bayesian counterparts of the classical tests for good- ness of fit and their use in judging the fit of a single Bayesian model to the observed data. We focus on posterior predictive assessment, in a framework that also includes conditioning on auxiliary statistics. The Bayesian formulation facilitates the con- struction and calculation of a meaningful reference distribution not only for any (classical) statistic, but also for any parameter-dependent "statistic" or discrep- ancy. The latter allows us to propose the realized discrepancy assessment of model fitness, which directly measures the true discrepancy between data and the posited model, for any aspect of the model which we want to explore. The computation required for the realized discrepancy assessment is a straightforward byproduct of the posterior simulation used for the original Bayesian analysis. We illustrate with three applied examples. The first example, which serves mainly to motivate the work, illustrates the difficulty of classical tests in assessing the fitness of a Poisson model to a positron emission tomography image that is constrained to be nonnegative. The second and third examples illustrate the details of the posterior predictive approach in two problems: estimation in a model with inequality constraints on the parameters, and estimation in a mixture model. In all three examples, standard test statistics (either a χ 2 or a likelihood ratio) are not pivotal: the difficulty is not just how to compute the reference distribution for the test, but that in the classical framework no such distribution exists, independent of the unknown model parameters.

/pdf/posterior-predictive-assessment-of-model-fitness-via-1wc3dk7ymj.pdf

Posterior predictive assessment of model fitness via realized discrepancies

Bayesian estimation for 2 groups provides complete distributions of credible values for the effect size, group means and their difference, standard deviations and their difference, and the normality of the data. The method handles outliers. The decision rule can accept the null value (unlike traditional t tests) when certainty in the estimate is high (unlike Bayesian model comparison using Bayes factors). The method also yields precise estimates of statistical power for various research goals. The software and programs are free and run on Macintosh, Windows, and Linux platforms.

/pdf/bayesian-estimation-supersedes-the-t-test-57drp31ai6.pdf

Bayesian Estimation Supersedes the t Test

This book provides a compact self-contained introduction to the theory and application of Bayesian statistical methods. The book is accessible to readers havinga basic familiarity with probability, yet allows more advanced readers to quickly grasp the principles underlying Bayesian theory and methods. The examples and computer code allow the reader to understand and implement basic Bayesian data analyses using standard statistical models and to extend the standard models to specialized data analysis situations. The book begins with fundamental notions such as probability, exchangeability and Bayes' rule, and ends with modern topics such as variable selection in regression, generalized linear mixed effects models, and semiparametric copula estimation. Numerous examples from the social, biological and physical sciences show how to implement these methodologies in practice. Monte Carlo summaries of posterior distributions play an important role in Bayesian data analysis. The open-source R statistical computing environment provides sufficient functionality to make Monte Carlo estimation very easy for a large number of statistical models and example R-code is provided throughout the text. Much of the example code can be run as is' in R, and essentially all of it can be run after downloading the relevant datasets from the companion website for this book.

/pdf/a-first-course-in-bayesian-statistical-methods-284th26wuv.pdf

A First Course in Bayesian Statistical Methods

Statistics has struggled for nearly a century over the issue of whether the Bayesian or frequentist paradigm is superior. This debate is far from over and, indeed, should continue, since there are fundamental philosophical and pedagogical issues at stake. At the methodological level, however, the debate has become considerably muted, with the recognition that each approach has a great deal to contribute to statistical practice and each is actually essential for full development of the other approach. In this article, we embark upon a rather idiosyncratic walk through some of these issues.

/pdf/the-interplay-of-bayesian-and-frequentist-analysis-1p7yjujg9j.pdf

The Interplay of Bayesian and Frequentist Analysis

An attack on the problem of goodness of fit is made by combining a Bayesian and sampling argument; the Bayesian part is effected by using the distribution of a future observation, while the sampling argument concerns itself with the distribution of a "chi-squared like" statistic, which measures discrepancies of observed frequencies from those predicted by the distribution of the future observation. Examples are given for the case of sampling from the binomial, Poisson and normal distributions. An interesting application arising from the above approach is a procedure for estimating the degree of a polynomial response function.

The Use of the Concept of a Future Observation in Goodness‐Of‐Fit Problems

A hybrid life test procedure is discussed from the Bayesian viewpoint. A total ofn items is placed on test, failed items are either not replaced or are replaced, and the test is terminated either when a pre-chosen number,K, of items have failed, or when a pre-determined time on test has been reached. Posterior and predictive distributions are obtained under the assumption of an exponential failure distribution, and point and interval estimates are given for the mean life and the life of an untested item. The results are applied to a numerical example.

Bayesian analysis of hybrid life tests with exponential failure times

The performance of three rules for dealing with outliers in small samples of size n from the normal distribution N(μ, σ2) are investigated when the primary objective of sampling is to obtain an accurate estimate of μ. It is assumed that at most one observation in the sample may be biased, arising from either N(μ + aσ, σ2) or N(r, (1 + b)σ2). Performance of each rule is measured in terms of “Protection”, the fractional decrease in the Mean Square Error (MSE) obtained by using the rule when a biased observation actually is present in the sample. Although numerical results have been obtained for n ≤ 10 when σ2 is known, computational difficulties have prevented evaluation of protections when σ2 is unknown except when n = 3.

Investigation of Rules for Dealing With Outliers in Small Samples from the Normal Distribution: I: Estimation of the Mean

: A recent paper by Feldman and Fox (1968) discussed the estimation of the parameter n in the binomial distribution when the other parameter p is known. This paper gives a Bayesian treatment of the problem both when p is known and when p is unknown. The Bayesian approach is a very practical one and is especially easy to apply in small problems because less computing is involved. It also does not require any appeal to asymptotic distributional results. (Author)

Bayesian Estimation of the Binomial Parameter

: There are a number of variable selection procedures in current use in multiple regression analysis. In some of these procedures, a new variable is entered into regression when its contribution to the explanation of the response is larger than that of any other variable not yet entered. The new entrant is then tested by a standard F-test to see if its contribution is 'significant'. Because of the method of selecting the largest contributor, an F-test of this type cannot be theoretically correct but the procedure is used because the exact distributions are not known. This raises questions like 'Is one seriously in error in using the F-tests.' 'If so, can adjustments be made to the F-test to correct the difficulty.' The purpose of this paper is to discuss some new results which throw light on these questions. (Author)

Irwin Guttman

Papers

The Use of the Concept of a Future Observation in Goodness‐Of‐Fit Problems

Bayesian analysis of hybrid life tests with exponential failure times

Investigation of Rules for Dealing With Outliers in Small Samples from the Normal Distribution: I: Estimation of the Mean

Bayesian Estimation of the Binomial Parameter

The distribution of certain regression statistics