8. Subset Selection in Regression (Monographs on Statistics and Applied Probability, no. 40). By A. J. Miller. ISBN 0 412 35380 6. Chapman and Hall, London, 1990. 240 pp. £25.00.

Subset Selection in Regression

Robust Bayesian analysis is the study of the sensitivity of Bayesian answers to uncertain inputs. This paper seeks to provide an overview of the subject, one that is accessible to statisticians outside the field. Recent developments in the area are also reviewed, though with very uneven emphasis.

An overview of robust Bayesian analysis

Statistics has struggled for nearly a century over the issue of whether the Bayesian or frequentist paradigm is superior. This debate is far from over and, indeed, should continue, since there are fundamental philosophical and pedagogical issues at stake. At the methodological level, however, the debate has become considerably muted, with the recognition that each approach has a great deal to contribute to statistical practice and each is actually essential for full development of the other approach. In this article, we embark upon a rather idiosyncratic walk through some of these issues.

/pdf/the-interplay-of-bayesian-and-frequentist-analysis-1p7yjujg9j.pdf

The Interplay of Bayesian and Frequentist Analysis

In objective Bayesian model selection, no single criterion has emerged as dominant in defining objective prior distributions. Indeed, many criteria have been separately proposed and utilized to propose differing prior choices. We first formalize the most general and compelling of the various criteria that have been suggested, together with a new criterion. We then illustrate the potential of these criteria in determining objective model selection priors by considering their application to the problem of variable selection in normal linear models. This results in a new model selection objective prior with a number of compelling properties.

Criteria for Bayesian model choice with application to variable selection

/pdf/criteria-for-bayesian-model-choice-with-application-to-180c3qqrvc.pdf

It is shown that the posterior converges in a weak sense in a fairly general set up which includes the exponential, gamma and Weibull with a location parameter, and a reliability change point problem. A representation is obtained for the limiting posterior in case a stronger convergence, e.g. almost sure convergence, holds. This leads to a necessary condition under the above general set up and settles the question of almost sure convergence in the above examples. Under very general conditions it is also shown that the posterior is asymptotically free of the prior. Though primarily developed for non-regular problems, all the theorems apply to regular cases also.

Stability and Convergence of the Posterior in Non-Regular Problems

Nonsubjective Bayes testing—an overview

We consider a family of models that arise in connection with sharp change in hazard rate corresponding to high initial hazard rate dropping to a more stable or slowly changing rate at an unknown change-point θ. Although the Bayes estimates are well behaved and are asymptotically efficient, it is difficult to compute them as the posterior distributions are generally very complicated. We obtain a simple first order asymptotic approximation to the posterior distribution of θ. The accuracy of the approximation is judged through simulation. The approximation performs quite well. Our method is also applied to analyze a real data set.

Approximation of the posterior distribution in a change-point problem

Consider Bayesian variable selection in normal linear regression models based on Zellner’s $$g$$
 -prior. We study theoretical properties of this method when the sample size $$n$$
 grows and consider the cases when the number of regressors, $$p$$
 is fixed and when it grows with $$n$$
 . We first consider the situation where the true model is not in the model space and prove under mild conditions that the method is consistent and “loss efficient” in appropriate sense. We then consider the case when the true model is in the model space and prove that the posterior probability of the true model goes to one as $$n$$
 goes to infinity. “Loss efficiency” is also proved in this situation. We give explicit conditions on the rate of growth of $$g$$
 , possibly depending on that of $$p$$
 as $$n$$
 grows, for our results to hold. This helps in making recommendations for the choice of $$g$$
 .

On consistency and optimality of Bayesian variable selection based on g-prior in normal linear regression models

We study the asymptotic behaviour of the posterior distributions for a one-parameter family of discontinuous densities. It is shown that a suitably centered and normalized posterior converges almost surely to an exponential limit in the total variation norm. Further, asymptotic expansions for the density, distribution function, moments and quantiles of the posterior are also obtained. It is to be noted that, in view of the results of Ghosh et al. (1994, Statistical Decision Theory and Related Topics V, 183-199, Springer, New York) and Ghosal et al. (1995, Ann. Statist., 23, 2145-2152), the nonregular cases considered here are essentially the only ones for which the posterior distributions converge. The results obtained here are also supported by a simulation experiment.

Tapas Samanta

Papers

Stability and Convergence of the Posterior in Non-Regular Problems

Nonsubjective Bayes testing—an overview

Approximation of the posterior distribution in a change-point problem

On consistency and optimality of Bayesian variable selection based on g-prior in normal linear regression models

Asymptotic Expansions of Posterior Distributions in Nonregular Cases