https://rss.onlinelibrary.wiley.com/doi/pdf/10.1111/j.2517-6161.1988.tb01721.x

Local computations with probabilities on graphical structures and their application to expert systems

A crucial problem in building a multiple regression model is the selection of predictors to include. The main thrust of this article is to propose and develop a procedure that uses probabilistic considerations for selecting promising subsets. This procedure entails embedding the regression setup in a hierarchical normal mixture model where latent variables are used to identify subset choices. In this framework the promising subsets of predictors can be identified as those with higher posterior probability. The computational burden is then alleviated by using the Gibbs sampler to indirectly sample from this multinomial posterior distribution on the set of possible subset choices. Those subsets with higher probability—the promising ones—can then be identified by their more frequent appearance in the Gibbs sample.

Variable selection via Gibbs sampling

: The classification maximum likelihood approach is sufficiently general to encompass many current clustering algorithms, including those based on the sum of squares criterion and on the criterion of Friedman and Rubin (1967). However, as currently implemented, it does not allow the specification of which features (orientation, size and shape) are to be common to all clusters and which may differ between clusters. Also, it is restricted to Gaussian distributions and it does not allow for noise. We propose ways of overcoming these limitations. A reparameterization of the covariance matrix allows us to specify that some features, but not all, be the same for all clusters. A practical framework for non-Gaussian clustering is outlined, and a means of incorporating noise in the form of a Poisson process is described. An approximate Bayesian method for choosing the number of clusters is given. The performance of the proposed methods is studied by simulation, with encouraging results. The methods are applied to the analysis of a data set arising in the study of diabetes, and the results seem better than those of previous analyses. (RH)

Model-based Gaussian and non-Gaussian clustering

In the field of psychology, the practice ofp value null-hypothesis testing is as widespread as ever. Despite this popularity, or perhaps because of it, most psychologists are not aware of the statistical peculiarities of thep value procedure. In particular,p values are based on data that were never observed, and these hypothetical data are themselves influenced by subjective intentions. Moreover,p values do not quantify statistical evidence. This article reviews thesep value problems and illustrates each problem with concrete examples. The three problems are familiar to statisticians but may be new to psychologists. A practical solution to thesep value problems is to adopt a model selection perspective and use the Bayesian information criterion (BIC) for statistical inference (Raftery, 1995). The BIC provides an approximation to a Bayesian hypothesis test, does not require the specification of priors, and can be easily calculated from SPSS output.

/pdf/a-practical-solution-to-the-pervasive-problems-of-p-values-1qozascdn9.pdf

A practical solution to the pervasive problems of p values.

Exchangeability and related topics

SUMMARY Global and local Bayes factors are defined and their respective roles examined as choice criteria among alternative linear models. The global Bayes factor is seen to function, in appropriate contexts, as a fully automatic Occam's razor and to be closely related to the Schwarz model choice criterion. The local Bayes factor is shown to have a close relationship with the Akaike Information Criterion. THE problem of choosing between alternative models continues to attract a good deal of theoretical attention, much of which has been stimulated by the appearance of the Akaike Information Criterion (Akaike, 1973). The AIC and its variants (see, for example, Bhansali and Downham, 1977) essentially adjust the likelihood ratio test statistic by a constant multiple of the difference in dimensionalities of the two models under consideration. Schwarz (1978) has recently proposed a fundamentally different criterion which replaces the constant multiplier of the AIC by the logarithm of the sample size. Stone (1979) has compared and contrasted these two approaches in terms of certain of their asymptotic properties, and a further contribution in this area is provided by Hannan and Quinn (1979). In this paper, which deals with the important special case of choice between alternative nested linear models, we shall also be concerned with comparing and contrasting these two forms of model choice criteria, but from a rather different, non-asymptotic, perspective. Our starting point is not a consideration of the criteria themselves, but, instead, a discussion of the Bayesian approach to comparing alternative nested linear models on the basis of their posterior probabilities, or, equivalently, on the basis of ratios of posterior to prior odds. Stone has, in effect, argued that the comparison of choice criteria for linear models on the basis of their asymptotic properties is rather arbitrarily dependent on the assumptions made about the embedded sequence of design matrices. Our approach avoids the arbitrary asymptotics and concentrates, instead, on the way in which the dependence of the prior specification on the design matrix influences the forms of model choice criteria which arise from consideration of posterior probabilities. It will be shown that, depending on the nature of the prior specification adopted for model parameters, two fundamentally different forms of odds ratio, or Bayesfactor, arise. The first of these, which we shall call the global Bayes factor, will be discussed in Section 2, and will be shown to lead, essentially, to the Schwarz-type of criterion. The relationship of the global Bayes factor to the so-called Lindley Paradox (Lindley, 1957) will also be examined and, motivated by this, in Section 3 we derive what we shall call the local Bayes factor. This will be shown to lead to a variant of the Akaike criterion, and a comparison will then be made with various Akaike-type procedures, including those of Bhansali and Downham (1977).

Bayes Factors and Choice Criteria for Linear Models

Bayes Factors for Linear and Log-Linear Models with Vague Prior Information

SUMMARY An infinite sequence of real random variables, any finite sample from which has a centred spherically symmetric joint distribution, is shown to have the property that, conditional on two further random variables, M and V, the variables are independent with the same normal distribution N(M, V).

A. F. M. Smith

Papers

Bayes Factors and Choice Criteria for Linear Models

Bayes Factors for Linear and Log-Linear Models with Vague Prior Information

On random sequences with centred spherical symmetry