1. Introduction and Overview. 2. Detecting Influential Observations and Outliers. 3. Detecting and Assessing Collinearity. 4. Applications and Remedies. 5. Research Issues and Directions for Extensions. Bibliography. Author Index. Subject Index.

Regression Diagnostics: Identifying Influential Data and Sources of Collinearity

Subjectivism has become the dominant philosophical foundation for Bayesian inference. Yet in practice, most Bayesian analyses are performed with so-called “noninformative” priors, that is, priors constructed by some formal rule. We review the plethora of techniques for constructing such priors and discuss some of the practical and philosophical issues that arise when they are used. We give special emphasis to Jeffreys's rules and discuss the evolution of his viewpoint about the interpretation of priors, away from unique representation of ignorance toward the notion that they should be chosen by convention. We conclude that the problems raised by the research on priors chosen by formal rules are serious and may not be dismissed lightly: When sample sizes are small (relative to the number of parameters being estimated), it is dangerous to put faith in any “default” solution; but when asymptotics take over, Jeffreys's rules and their variants remain reasonable choices. We also provide an annotated b...

The Selection of Prior Distributions by Formal Rules

Often, either from a lack of prior information or simply for convenience, variance components are modeled with improper priors in hierarchical linear mixed models. Although the posterior distributions for these models are rarely available in closed form, the usual conjugate structure of the prior specification allows for painless calculation of the Gibbs conditionals. Thus the Gibbs sampler may be used to explore the posterior distribution without ever having established propriety of the posterior. An example is given showing that the output from a Gibbs chain corresponding to an improper posterior may appear perfectly reasonable. Thus one cannot expect the Gibbs output to provide a “red flag,” informing the user that the posterior is improper. The user must demonstrate propriety before a Markov chain Monte Carlo technique is used. A theorem is given that classifies improper priors according to the propriety of the resulting posteriors. Applications concerning Bayesian analysis of animal breeding...

/pdf/the-effect-of-improper-priors-on-gibbs-sampling-in-1af566i079.pdf

The Effect of Improper Priors on Gibbs Sampling in Hierarchical Linear Mixed Models

Markov chain Monte Carlo algorithms are widely used in the fitting of generalized linear models (GLMs). Such model fitting is somewhat of an art form, requiring suitable trickery and tuning to obtain results in which one can have confidence. A wide range of practical issues arise. The focus here is on parameter identifiability and posterior propriety. In particular, we clarify that nonidentifiability arises for usual GLMs and discuss its implications for simulation-based model fitting. Because often some part of the prior specification is vague, we consider whether the resulting posterior is proper, providing rather general and easily checked results for GLMs. We also show that if a Gibbs sampler is run with an improper posterior, then it may be possible to use the output to obtain meaningful inference for certain model unknowns.

Identifiability, Improper Priors, and Gibbs Sampling for Generalized Linear Models

https://digitalcommons.calpoly.edu/cgi/viewcontent.cgi?article=1129&context=cenv_fac

On the nature of over-dispersion in motor vehicle crash prediction models.

A Bayesian approach is presented for detecting influential observations using general divergence measures on the posterior distributions. A sampling-based approach using a Gibbs or Metropolis-within-Gibbs method is used to compute the posterior divergence measures. Four specific measures are proposed, which convey the effects of a single observation or covariate on the posterior. The technique is applied to a generalized linear model with binary response data, an overdispersed model and a nonlinear model. An asymptotic approximation using Laplace method to obtain the posterior divergence is also briefly discussed.



Cet article presente une approche de Bayes pour depister des observations influentes a l'aide de mesures de divergence generale appliquees aux distributions a posteriori. Une approche fondee sur l'echantillonnage, utilisant la methode Gibbs ou Metropolis avec Gibbs, est utilisee afin de calculer les mesures de divergence a posteriori. Quatre mesures specifiques sont proposees, communiquant les effets d'une seule observation ou facteur a posteriori. La technique est appliquee a un modele lineaire generalise avec donnees de reponse binaires, a un modele surdisperse et a un modele nonlineaire. Une approximation asymptotique utilisant la methode de Laplace dans le but d'obtenir une divergence a posteriori est egalement discutee.

Fengchun Peng

Papers

Bayesian analysis of outlier problems using divergence measures

Overdispersed Generalized Linear Models