Summary. We consider the problem of comparing complex hierarchical models in which the number of parameters is not clearly defined. Using an information theoretic argument we derive a measure pD for the effective number of parameters in a model as the difference between the posterior mean of the deviance and the deviance at the posterior means of the parameters of interest. In general pD approximately corresponds to the trace of the product of Fisher's information and the posterior covariance, which in normal models is the trace of the ‘hat’ matrix projecting observations onto fitted values. Its properties in exponential families are explored. The posterior mean deviance is suggested as a Bayesian measure of fit or adequacy, and the contributions of individual observations to the fit and complexity can give rise to a diagnostic plot of deviance residuals against leverages. Adding pD to the posterior mean deviance gives a deviance information criterion for comparing models, which is related to other information criteria and has an approximate decision theoretic justification. The procedure is illustrated in some examples, and comparisons are drawn with alternative Bayesian and classical proposals. Throughout it is emphasized that the quantities required are trivial to compute in a Markov chain Monte Carlo analysis.

/pdf/bayesian-measures-of-model-complexity-and-fit-1ouzmtfu7x.pdf

Bayesian measures of model complexity and fit

Contents Dedication Preface Acknowledgements Notation A general classification notation and diagram Glossary Chapter 1 An introduction to multilevel models 1.1 Hierarchically structured data 1.2 School effectiveness 1.3 Sample survey methods 1.4 Repeated measures data 1.5 Event history and survival models 1.6 Discrete response data 1.7 Multivariate models 1.8 Nonlinear models 1.9 Measurement errors 1.10 Cross classifications and multiple membership structures. 1.11 Factor analysis and structural equation models 1.12 Levels of aggregation and ecological fallacies 1.13 Causality 1.14 The latent normal transformation and missing data 1.15 Other texts 1.16 A caveat Chapter 2 The 2-level model 2.1 Introduction 2.2 The 2-level model 2.3 Parameter estimation 2.4 Maximum likelihood estimation using Iterative Generalised Least Squares (IGLS) 2.5 Marginal models and Generalized Estimating Equations (GEE) 2.6 Residuals 2.7 The adequacy of Ordinary Least Squares estimates. 2.8 A 2-level example using longitudinal educational achievement data 2.9 General model diagnostics 2.10 Higher level explanatory variables and compositional effects 2.11 Transforming to normality 2.12 Hypothesis testing and confidence intervals 2.13 Bayesian estimation using Markov Chain Monte Carlo (MCMC) 2.14 Data augmentation Appendix 2.1 The general structure and maximum likelihood estimation for a multilevel model Appendix 2.2 Multilevel residuals estimation Appendix 2.3 Estimation using profile and extended likelihood Appendix 2.4 The EM algorithm Appendix 2.5 MCMC sampling Chapter 3. Three level models and more complex hierarchical structures. 3.1 Complex variance structures 3.2 A 3-level complex variation model example. 3.3 Parameter Constraints 3.4 Weighting units 3.5 Robust (Sandwich) Estimators and Jacknifing 3.6 The bootstrap 3.7 Aggregate level analyses 3.8 Meta analysis 3.9 Design issues Chapter 4. Multilevel Models for discrete response data 4.1 Generalised linear models 4.2 Proportions as responses 4.3 Examples 4.4 Models for multiple response categories 4.5 Models for counts 4.6 Mixed discrete - continuous response models 4.7 A latent normal model for binary responses 4.8 Partitioning variation in discrete response models Appendix 4.1. Generalised linear model estimation Appendix 4.2 Maximum likelihood estimation for generalised linear models Appendix 4.3 MCMC estimation for generalised linear models Appendix 4.4. Bootstrap estimation for generalised linear models Chapter 5. Models for repeated measures data 5.1 Repeated measures data 5.2 A 2-level repeated measures model 5.3 A polynomial model example for adolescent growth and the prediction of adult height 5.4 Modelling an autocorrelation structure at level 1. 5.5 A growth model with autocorrelated residuals 5.6 Multivariate repeated measures models 5.7 Scaling across time 5.8 Cross-over designs 5.9 Missing data 5.10 Longitudinal discrete response data Chapter 6. Multivariate multilevel data 6.1 Introduction 6.2 The basic 2-level multivariate model 6.3 Rotation Designs 6.4 A rotation design example using Science test scores 6.5 Informative response selection: subject choice in examinations 6.6 Multivariate structures at higher levels and future predictions 6.7 Multivariate responses at several levels 6.8 Principal Components analysis Appendix 6.1 MCMC algorithm for a multivariate normal response model with constraints Chapter 7. Latent normal models for multivariate data 7.1 The normal multilevel multivariate model 7.2 Sampling binary responses 7.3 Sampling ordered categorical responses 7.4 Sampling unordered categorical responses 7.5 Sampling count data 7.6 Sampling continuous non-normal data 7.7 Sampling the level 1 and level 2 covariance matrices 7.8 Model fit 7.9 Partially ordered data 7.10 Hybrid normal/ordered variables 7.11 Discussion Chapter 8. Multilevel factor analysis, structural equation and mixture models 8.1 A 2-stage 2-level factor model 8.2 A general multilevel factor model 8.3 MCMC estimation for the factor model 8.4 Structural equation models 8.5 Discrete response multilevel structural equation models 8.6 More complex hierarchical latent variable models 8.7 Multilevel mixture models Chapter 9. Nonlinear multilevel models 9.1 Introduction 9.2 Nonlinear functions of linear components 9.3 Estimating population means 9.4 Nonlinear functions for variances and covariances 9.5 Examples of nonlinear growth and nonlinear level 1 variance Appendix 9.1 Nonlinear model estimation Chapter 10. Multilevel modelling in sample surveys 10.1 Sample survey structures 10.2 Population structures 10.3 Small area estimation Chapter 11 Multilevel event history and survival models 11.1 Introduction 11.2 Censoring 11.3 Hazard and survival funtions 11.4 Parametric proportional hazard models 11.5 The semiparametric Cox model 11.6 Tied observations 11.7 Repeated events proportional hazard models 11.8 Example using birth interval data 11.9 Log duration models 11.10 Examples with birth interval data and children s activity episodes 11.11 The grouped discrete time hazards model 11.12 Discrete time latent normal event history models Chapter 12. Cross classified data structures 12.1 Random cross classifications 12.2 A basic cross classified model 12.3 Examination results for a cross classification of schools 12.4 Interactions in cross classifications 12.5 Cross classifications with one unit per cell 12.6 Multivariate cross classified models 12.7 A general notation for cross classifications 12.8 MCMC estimation in cross classified models Appendix 12.1 IGLS Estimation for cross classified data. Chapter 13 Multiple membership models 13.1 Multiple membership structures 13.2 Notation and classifications for multiple membership structures 13.3 An example of salmonella infection 13.4 A repeated measures multiple membership model 13.5 Individuals as higher level units 13.5.1 Example of research grant awards 13.6 Spatial models 13.7 Missing identification models Appendix 13.1 MCMC estimation for multiple membership models. Chapter 14 Measurement errors in multilevel models 14.1 A basic measurement error model 14.2 Moment based estimators 14.3 A 2-level example with measurement error at both levels. 14.4 Multivariate responses 14.5 Nonlinear models 14.6 Measurement errors for discrete explanatory variables 14.7 MCMC estimation for measurement error models Appendix 14.1 Measurement error estimation 14.2 MCMC estimation for measurement error models Chapter 15. Smoothing models for multilevel data. 15.1 Introduction 15.2. Smoothing estimators 15.3 Smoothing splines 15.4 Semi parametric smoothing models 15.5 Multilevel smoothing models 15.6 General multilevel semi-parametric smoothing models 15.7 Generalised linear models 15.8 An example Fixed Random 15.9 Conclusions Chapter 16. Missing data, partially observed data and multiple imputation 16.1 Creating a completed data set 16.2 Joint modelling for missing data 16.3 A two level model with responses of different types at both levels. 16.4 Multiple imputation 16.5 A simulation example of multiple imputation for missing data 16.6 Longitudinal data with attrition 16.7 Partially known data values 16.8 Conclusions Chapter 17 Multilevel models with correlated random effects 17.1 Non-independence of level 2 residuals 17.2 MCMC estimation for non-independent level 2 residuals 17.3 Adaptive proposal distributions in MCMC estimation 17.4 MCMC estimation for non-independent level 1 residuals 17.5 Modelling the level 1 variance as a function of explanatory variables with random effects 17.6 Discrete responses with correlated random effects 17.7 Calculating the DIC statistic 17.8 A growth data set 17.9 Conclusions Chapter 18. Software for multilevel modelling References Author index Subject index

Multilevel Statistical Models

Statistical approaches to overdispersion, correlated errors, shrinkage estimation, and smoothing of regression relationships may be encompassed within the framework of the generalized linear mixed model (GLMM). Given an unobserved vector of random effects, observations are assumed to be conditionally independent with means that depend on the linear predictor through a specified link function and conditional variances that are specified by a variance function, known prior weights and a scale factor. The random effects are assumed to be normally distributed with mean zero and dispersion matrix depending on unknown variance components. For problems involving time series, spatial aggregation and smoothing, the dispersion may be specified in terms of a rank deficient inverse covariance matrix. Approximation of the marginal quasi-likelihood using Laplace's method leads eventually to estimating equations based on penalized quasilikelihood or PQL for the mean parameters and pseudo-likelihood for the variances. Im...

https://www.jstor.org/stable/pdfplus/2290687.pdf

Approximate inference in generalized linear mixed models

There has been much recent interest in Bayesian image analysis, including such topics as removal of blur and noise, detection of object boundaries, classification of textures, and reconstruction of two- or three-dimensional scenes from noisy lower-dimensional views. Perhaps the most straightforward task is that of image restoration, though it is often suggested that this is an area of relatively minor practical importance. The present paper argues the contrary, since many problems in the analysis of spatial data can be interpreted as problems of image restoration. Furthermore, the amounts of data involved allow routine use of computer intensive methods, such as the Gibbs sampler, that are not yet practicable for conventional images. Two examples are given, one in archeology, the other in epidemiology. These are preceded by a partial review of pixel-based Bayesian image analysis.

/pdf/bayesian-image-restoration-with-two-applications-in-spatial-5bmqnz4oxw.pdf

Bayesian image restoration, with two applications in spatial statistics

Rejoinder (Bayesian image restoration,with two applications in spatial statistics)

There have been many attempts in recent years to map incidence and mortality from diseases such as cancer. Such maps usually display either relative rates in each district, as measured by a standardized mortality ratio (SMR) or some similar index, or the statistical significance level for a test of the difference between the rates in that district and elsewhere. Neither of these approaches is fully satisfactory and we propose a new approach using empirical Bayes estimation. The resulting estimators represent a weighted compromise between the SMR, the overall mean relative rate, and a local mean of the relative rate in nearby areas. The compromise solution depends on the reliability of each individual SMR and on estimates of the overall amount of dispersion of relative rates over different districts.

https://www.jstor.org/stable/pdfplus/2532003.pdf

Empirical Bayes estimates of age-standardized relative risks for use in disease mapping.

Our first paper reviewed methods for modelling variation in cancer incidence and mortality rates in terms of either period effects or cohort effects in the general multiplicative risk model. There we drew attention to the difficulty of attributing regular trends to either period or cohort influences. In this paper we turn to the more realistic problem in which neither period nor cohort effects alone lead to an adequate description of the data. We describe the age-period-cohort model and show how its ambiguities surrounding regular trends 'intensify'. We recommend methods for presenting the results of analyses based upon this model which minimize the serious risk of misleading implications and critically review previous suggestions. The discussion is illustrated by an analysis of breast cancer mortality in Japan with special reference to the phenomenon of 'Clemmesen's hook'.

Models for temporal variation in cancer rates. II: Age–period–cohort models

A main concern of descriptive epidemiologists is the presentation and interpretation of temporal variations in cancer rates. In its simplest form, this problem is that of the analysis of a set of rates arranged in a two-way table by age group and calendar period. We review the modern approach to the analysis of such data which justifies traditional methods of age standardization in terms of the multiplicative risk model. We discuss the use of this model when the temporal variations are due to purely secular (period) influences and when they are attributable to generational (cohort) influences. Finally we demonstrate the serious difficulties which attend the interpretation of regular trends. The methods described are illustrated by examples for incidence rates of bladder cancer in Birmingham, U.K., mortality from bladder cancer in Italy, and mortality from lung cancer in Belgium.

https://www.bendixcarstensen.com/APC/ClaytonSchifflers.1987.pdf

Models for temporal variation in cancer rates. I: Age-period and age-cohort models.

A new approach to the analysis of bivariate survival data is presented. It involves the development of "a model for bivariate life-tables...with a single association parameter which is unaffected by monotone transformation of the marginal distributions. Methods for testing and estimating this parameter from right-censored sample pairs using only rank-order information are presented. The model is a generalization of the proportional hazards model and includes a random effect representing heterogeneity of frailty or proneness to failure." A discussion on "the analysis of litter-matched and matched-pair failure-time data is [included]. Some uses of the methods in rank regression problems involving only one right-censored dependent variable are described and a test is proposed for proportional hazards against alternative error structures leading to converging hazards. Finally the methods are compared and validated by Monte-Carlo simulations." Comments by several people and a discussion concerning the paper are included (pp. 108-17). (EXCERPT)

Multivariate generalizations of the proportional hazards model

https://rss.onlinelibrary.wiley.com/doi/pdf/10.2307/2981943

David Clayton

Papers

Empirical Bayes estimates of age-standardized relative risks for use in disease mapping.

Models for temporal variation in cancer rates. II: Age–period–cohort models

Models for temporal variation in cancer rates. I: Age-period and age-cohort models.

Multivariate generalizations of the proportional hazards model

Multivariate Generalizations of the Proportional Hazards Model