In psychology and the behavioral sciences generally, the use of the hierarchical linear model (HLM) and its extensions for discrete outcomes are popular methods for modeling clustered data. HLM and its discrete outcome extensions, however, are certainly not the only methods available to model clustered data. Although other methods exist and are widely implemented in other disciplines, it seems that psychologists have yet to consider these methods in substantive studies. This article compares and contrasts HLM with alternative methods including generalized estimating equations and cluster-robust standard errors. These alternative methods do not model random effects and thus make a smaller number of assumptions and are interpreted identically to single-level methods with the benefit that estimates are adjusted to reflect clustering of observations. Situations where these alternative methods may be advantageous are discussed including research questions where random effects are and are not required, when random effects can change the interpretation of regression coefficients, challenges of modeling with random effects with discrete outcomes, and examples of published psychology articles that use HLM that may have benefitted from using alternative methods. Illustrative examples are provided and discussed to demonstrate the advantages of the alternative methods and also when HLM would be the preferred method. (PsycINFO Database Record

On the unnecessary ubiquity of hierarchical linear modeling.

Robustness of linear mixed-effects models to violations of distributional assumptions

Objective: The Prevention of Suicide in Primary Care Elderly: Collaborative Trial (PROSPECT) evaluated the impact of a care management intervention on suicidal ideation and depression in older primary care patients. This is the first report of outcomes over a 2-year period. Method: Study participants were patients 60 years of age or older (N=599) with major or minor depression selected after screening 9,072 randomly identified patients of 20 primary care practices randomly assigned to provide either the PROSPECT intervention or usual care. The intervention consisted of services of 15 trained care managers, who offered algorithm-based recommendations to physicians and helped patients with treatment adherence over 24 months. Results: Compared with patients receiving usual care, those receiving the intervention had a higher likelihood of receiving antidepressants and/or psychotherapy (84.9%–89% versus 49%–62%) and had a 2.2 times greater decline in suicidal ideation over 24 months. Treatment response occurre...

Reducing Suicidal Ideation and Depression in Older Primary Care Patients: 24-Month Outcomes of the PROSPECT Study

Modified Poisson regression, which combines a log Poisson regression model with robust variance estimation, is a useful alternative to log binomial regression for estimating relative risks. Previous studies have shown both analytically and by simulation that modified Poisson regression is appropriate for independent prospective data. This method is often applied to clustered prospective data, despite a lack of evidence to support its use in this setting. The purpose of this article is to evaluate the performance of the modified Poisson regression approach for estimating relative risks from clustered prospective data, by using generalized estimating equations to account for clustering. A simulation study is conducted to compare log binomial regression and modified Poisson regression for analyzing clustered data from intervention and observational studies. Both methods generally perform well in terms of bias, type I error, and coverage. Unlike log binomial regression, modified Poisson regression is not prone to convergence problems. The methods are contrasted by using example data sets from 2 large studies. The results presented in this article support the use of modified Poisson regression as an alternative to log binomial regression for analyzing clustered prospective data when clustering is taken into account by using generalized estimating equations.

/pdf/performance-of-the-modified-poisson-regression-approach-for-lquref2cd5.pdf

Performance of the Modified Poisson Regression Approach for Estimating Relative Risks From Clustered Prospective Data

With the advent of new technologies, animal locations are being collected at ever finer spatio-temporal scales. We review analytical methods for dealing with correlated data in the context of resource selection, including post hoc variance inflation techniques, ‘two-stage’ approaches based on models fit to each individual, generalized estimating equations and hierarchical mixed-effects models. These methods are applicable to a wide range of correlated data problems, but can be difficult to apply and remain especially challenging for use–availability sampling designs because the correlation structure for combinations of used and available points are not likely to follow common parametric forms. We also review emerging approaches to studying habitat selection that use fine-scale temporal data to arrive at biologically based definitions of available habitat, while naturally accounting for autocorrelation by modelling animal movement between telemetry locations. Sophisticated analyses that explicitly model correlation rather than consider it a nuisance, like mixed effects and state-space models, offer potentially novel insights into the process of resource selection, but additional work is needed to make them more generally applicable to large datasets based on the use–availability designs. Until then, variance inflation techniques and two-stage approaches should offer pragmatic and flexible approaches to modelling correlated data.

/pdf/correlation-and-studies-of-habitat-selection-problem-red-1xbyadq9wx.pdf

Correlation and studies of habitat selection: problem, red herring or opportunity?

Generalized linear mixed models (GLMMs) have become a frequently used tool for the analysis of non-Gaussian longitudinal data. Estimation is based on maximum likelihood theory, which assumes that the underlying probability model is correctly specified. Recent research is showing that the results obtained from these models are not always robust against departures from the assumptions on which these models are based. In the present work we have used simulations with a logistic random-intercept model to study the impact of misspecifying the random-effects distribution on the type I and II errors of the tests for the mean structure in GLMMs. We found that the misspecification can either increase or decrease the power of the tests, depending on the shape of the underlying random-effects distribution, and it can considerably inflate the type I error rate. Additionally, we have found a theoretical result which states that whenever a subset of fixed-effects parameters, not included in the random-effects structure equals zero, the corresponding maximum likelihood estimator will consistently estimate zero. This implies that under certain conditions a significant effect could be considered as a reliable result, even if the random-effects distribution is misspecified.

/pdf/type-i-and-type-ii-error-under-random-effects-1vskgtuvvg.pdf

Type I and Type II Error Under Random-Effects Misspecification in Generalized Linear Mixed Models

Estimation in generalized linear mixed models (GLMMs) is often based on maximum likelihood theory, assuming that the underlying probability model is correctly specified. However, the validity of this assumption is sometimes difficult to verify. In this paper we study, through simulations, the impact of misspecifying the random-effects distribution on the estimation and hypothesis testing in GLMMs. It is shown that the maximum likelihood estimators are inconsistent in the presence of misspecification. The bias induced in the mean-structure parameters is generally small, as far as the variability of the underlying random-effects distribution is small as well. However, the estimates of this variability are always severely biased. Given that the variance components are the only tool to study the variability of the true distribution, it is difficult to assess whether problems in the estimation of the mean structure occur. The type I error rate and the power of the commonly used inferential procedures are also severely affected. The situation is aggravated if more than one random effect is included in the model. Further, we propose to deal with possible misspecification by way of sensitivity analysis, considering several random-effects distributions. All the results are illustrated using data from a clinical trial in schizophrenia.

/pdf/the-impact-of-a-misspecified-random-effects-distribution-on-b2hpximy8h.pdf

The impact of a misspecified random-effects distribution on the estimation and the performance of inferential procedures in generalized linear mixed models.

[Litiere, S.; Alonso, A.; Molenberghs, G.] Univ Hasselt, Interuniv Inst Biostat & Stat Bioinformat, Diepenbeek, Belgium. [Molenberghs, G.] Katholieke Univ Leuven, Leuven, Belgium.

The impact of a misspecified random-effects distribution on the estimation and the performance of inferential procedures in generalized linear mixed models (vol 27, pg 3125, 2008)

A family of tests to detect misspecifications in the random-effects structure of generalized linear mixed models

S UMMARY Generalized linear mixed models have become a frequently used tool for the analysis of non-Gaussian longitudinal data. Estimation is often based on maximum likelihood theory, which assumes that the underlying probability model is correctly specified. Recent research shows that the results obtained from these models are not always robust against departures from the assumptions on which they are based. Therefore, diagnostic tools for the detection of model misspecifications are of the utmost importance. In this paper, we propose 2 diagnostic tests that are based on 2 equivalent representations of the model information matrix. We evaluate the power of both tests using theoretical considerations as well as via simulation. In the simulations, the performance of the new tools is evaluated in many settings of practical relevance, focusing on misspecification of the random-effects structure. In all the scenarios, the results were encouraging, however, the tests also exhibited inflated Type I error rates when the sample size was small or moderate. Importantly, a parametric bootstrap version of the tests seems to overcome this problem, although more research in this direction may be needed. Finally, both tests were also applied to analyze a real case study in psychiatry.

/pdf/testing-for-misspecification-in-generalized-linear-mixed-rdqo2ohz61.pdf

Saskia Litière

Papers

Type I and Type II Error Under Random-Effects Misspecification in Generalized Linear Mixed Models

The impact of a misspecified random-effects distribution on the estimation and the performance of inferential procedures in generalized linear mixed models.

The impact of a misspecified random-effects distribution on the estimation and the performance of inferential procedures in generalized linear mixed models (vol 27, pg 3125, 2008)

A family of tests to detect misspecifications in the random-effects structure of generalized linear mixed models

Testing for misspecification in generalized linear mixed models