Robustness of linear mixed-effects models to violations of distributional assumptions

Preface 1. Long run frequencies 2. The chance set-up 3. Support 4. The long run 5. The law of likelihood 6. Statistical tests 7. Theories of testing 8. Random sampling 9. The fiducial argument 10. Estimation 11. Point estimation 12. Bayes' theory 13. The subjective theory.

Logic of Statistical Inference. By Ian Hacking. Pp. ix, 232. 40s. 1965. (Cambridge University Press)

We propose a framework for general Bayesian inference. We argue that a valid update of a prior belief distribution to a posterior can be made for parameters which are connected to observations through a loss function rather than the traditional likelihood function, which is recovered as a special case. Modern application areas make it increasingly challenging for Bayesians to attempt to model the true data-generating mechanism. For instance, when the object of interest is low dimensional, such as a mean or median, it is cumbersome to have to achieve this via a complete model for the whole data distribution. More importantly, there are settings where the parameter of interest does not directly index a family of density functions and thus the Bayesian approach to learning about such parameters is currently regarded as problematic. Our framework uses loss functions to connect information in the data to functionals of interest. The updating of beliefs then follows from a decision theoretic approach involving cumulative loss functions. Importantly, the procedure coincides with Bayesian updating when a true likelihood is known yet provides coherent subjective inference in much more general settings. Connections to other inference frameworks are highlighted.

/pdf/a-general-framework-for-updating-belief-distributions-kudkpproy9.pdf

A general framework for updating belief distributions.

We empirically show that Bayesian inference can be inconsistent under misspecification in simple linear regression problems, both in a model averaging/selection and in a Bayesian ridge regression setting. We use the standard linear model, which assumes homoskedasticity, whereas the data are heteroskedastic (though, significantly, there are no outliers). As sample size increases, the posterior puts its mass on worse and worse models of ever higher dimension. This is caused by hypercompression, the phenomenon that the posterior puts its mass on distributions that have much larger KL divergence from the ground truth than their average, i.e. the Bayes predictive distribution. To remedy the problem, we equip the likelihood in Bayes' theorem with an exponent called the learning rate, and we propose the SafeBayesian method to learn the learning rate from the data. SafeBayes tends to select small learning rates, and regularizes more, as soon as hypercompression takes place. Its results on our data are quite encouraging.

/pdf/inconsistency-of-bayesian-inference-for-misspecified-linear-1tp3y3ooeo.pdf

Inconsistency of Bayesian inference for misspecified linear models, and a proposal for repairing it

We use the properties of independence estimating equations to adjust the 'independence' loglikelihood function in the presence of clustering. The proposed adjustment relies on the robust sandwich estimator of the parameter covariance matrix, which is easily calculated. The methodology competes favourably with established techniques based on independence estimating equations; we provide some insight as to why this is so. The adjustment is applied to examples relating to the modelling of wind speed in Europe and annual maximum temperatures in the U.K.

Inference for clustered data using the independence loglikelihood

Summary. The strength of statistical evidence is measured by the likelihood ratio. Two key performance properties of this measure are the probability of observing strong misleading evidence and the probability of observing weak evidence. For the likelihood function associated with a parametric statistical model, these probabilities have a simple large sample structure when the model is correct. Here we examine how that structure changes when the model fails. This leads to criteria for determining whether a given likelihood function is robust (continuing to perform satisfactorily when the model fails), and to a simple technique for adjusting both likelihoods and profile likelihoods to make them robust. We prove that the expected information in the robust adjusted likelihood cannot exceed the expected information in the likelihood function from a true model. We note that the robust adjusted likelihood is asymptotically fully efficient when the working model is correct, and we show that in some important examples this efficiency is retained even when the working model fails. In such cases the Bayes posterior probability distribution based on the adjusted likelihood is robust, remaining correct asymptotically even when the model for the observable random variable does not include the true distribution. Finally we note a link to standard frequentist methodology—in large samples the adjusted likelihood functions provide robust likelihood-based confidence intervals.

Tsung Shan Tsou

Papers

Interpreting statistical evidence by using imperfect models: robust adjusted likelihood functions