Cochrane Reviews have recently started including the quantity I 2 to help readers assess the consistency of the results of studies in meta-analyses. What does this new quantity mean, and why is assessment of heterogeneity so important to clinical practice? 

Systematic reviews and meta-analyses can provide convincing and reliable evidence relevant to many aspects of medicine and health care.1 Their value is especially clear when the results of the studies they include show clinically important effects of similar magnitude. However, the conclusions are less clear when the included studies have differing results. In an attempt to establish whether studies are consistent, reports of meta-analyses commonly present a statistical test of heterogeneity. The test seeks to determine whether there are genuine differences underlying the results of the studies (heterogeneity), or whether the variation in findings is compatible with chance alone (homogeneity). However, the test is susceptible to the number of trials included in the meta-analysis. We have developed a new quantity, I 2, which we believe gives a better measure of the consistency between trials in a meta-analysis.

Assessment of the consistency of effects across studies is an essential part of meta-analysis. Unless we know how consistent the results of studies are, we cannot determine the generalisability of the findings of the meta-analysis. Indeed, several hierarchical systems for grading evidence state that the results of studies must be consistent or homogeneous to obtain the highest grading.2–4

Tests for heterogeneity are commonly used to decide on methods for combining studies and for concluding consistency or inconsistency of findings.5 6 But what does the test achieve in practice, and how should the resulting P values be interpreted?

A test for heterogeneity examines the null hypothesis that all studies are evaluating the same effect. The usual test statistic …

/pdf/measuring-inconsistency-in-meta-analyses-4iysmttp08.pdf

Measuring inconsistency in meta-analyses

Species distribution models (SDMs) are numerical tools that combine observations of species occurrence or abundance with environmental estimates. They are used to gain ecological and evolutionary insights and to predict distributions across landscapes, sometimes requiring extrapolation in space and time. SDMs are now widely used across terrestrial, freshwater, and marine realms. Differences in methods between disciplines reflect both differences in species mobility and in “established use.” Model realism and robustness is influenced by selection of relevant predictors and modeling method, consideration of scale, how the interplay between environmental and geographic factors is handled, and the extent of extrapolation. Current linkages between SDM practice and ecological theory are often weak, hindering progress. Remaining challenges include: improvement of methods for modeling presence-only data and for model selection and evaluation; accounting for biotic interactions; and assessing model uncertainty.

/pdf/species-distribution-models-ecological-explanation-and-1szux2yl31.pdf

Species Distribution Models: Ecological Explanation and Prediction Across Space and Time

Internal validation of predictive models: efficiency of some procedures for logistic regression analysis.

Prediction models are developed to aid health-care providers in estimating the probability or risk that a specific disease or condition is present (diagnostic models) or that a specific event will occur in the future (prognostic models), to inform their decision making. However, the overwhelming evidence shows that the quality of reporting of prediction model studies is poor. Only with full and clear reporting of information on all aspects of a prediction model can risk of bias and potential usefulness of prediction models be adequately assessed. The Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD) Initiative developed a set of recommendations for the reporting of studies developing, validating, or updating a prediction model, whether for diagnostic or prognostic purposes. This article describes how the TRIPOD Statement was developed. An extensive list of items based on a review of the literature was created, which was reduced after a Web-based survey and revised during a 3-day meeting in June 2011 with methodologists, health-care professionals, and journal editors. The list was refined during several meetings of the steering group and in e-mail discussions with the wider group of TRIPOD contributors. The resulting TRIPOD Statement is a checklist of 22 items, deemed essential for transparent reporting of a prediction model study. The TRIPOD Statement aims to improve the transparency of the reporting of a prediction model study regardless of the study methods used. The TRIPOD Statement is best used in conjunction with the TRIPOD explanation and elaboration document. To aid the editorial process and readers of prediction model studies, it is recommended that authors include a completed checklist in their submission (also available at www.tripod-statement.org).

https://www.bmj.com/content/bmj/350/bmj.g7594.full.pdf

Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD): The TRIPOD statement

Prediction models are developed to aid health care providers in estimating the probability or risk that a specific disease or condition is present (diagnostic models) or that a specific event will occur in the future (prognostic models), to inform their decision making. However, the overwhelming evidence shows that the quality of reporting of prediction model studies is poor. Only with full and clear reporting of information on all aspects of a prediction model can risk of bias and potential usefulness of prediction models be adequately assessed. The Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD) Initiative developed a set of recommendations for the reporting of studies developing, validating, or updating a prediction model, whether for diagnostic or prognostic purposes. This article describes how the TRIPOD Statement was developed. An extensive list of items based on a review of the literature was created, which was reduced after a Web-based survey and revised during a 3-day meeting in June 2011 with methodologists, health care professionals, and journal editors. The list was refined during several meetings of the steering group and in e-mail discussions with the wider group of TRIPOD contributors. The resulting TRIPOD Statement is a checklist of 22 items, deemed essential for transparent reporting of a prediction model study. The TRIPOD Statement aims to improve the transparency of the reporting of a prediction model study regardless of the study methods used. The TRIPOD Statement is best used in conjunction with the TRIPOD explanation and elaboration document. To aid the editorial process and readers of prediction model studies, it is recommended that authors include a completed checklist in their submission (also available at www.tripod-statement.org).

Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD): The TRIPOD Statement

Stepwise selection in small data sets : A simulation study of bias in logistic regression analysis

The MISCAN simulation program for the evaluation of screening for disease

Logistic regression analysis may well be used to develop a predictive model for a dichotomous medical outcome, such as short-term mortality. When the data set is small compared to the number of covariables studied, shrinkage techniques may improve predictions. We compared the performance of three variants of shrinkage techniques: 1) a linear shrinkage factor, which shrinks all coefficients with the same factor; 2) penalized maximum likelihood (or ridge regression), where a penalty factor is added to the likelihood function such that coefficients are shrunk individually according to the variance of each covariable; 3) the Lasso, which shrinks some coefficients to zero by setting a constraint on the sum of the absolute values of the coefficients of standardized covariables.



Logistic regression models were constructed to predict 30-day mortality after acute myocardial infarction. Small data sets were created from a large randomized controlled trial, half of which provided independent validation data. We found that all three shrinkage techniques improved the calibration of predictions compared to the standard maximum likelihood estimates. This study illustrates that shrinkage is a valuable tool to overcome some of the problems of overfitting in medical data.

/pdf/application-of-shrinkage-techniques-in-logistic-regression-wxbfalpng7.pdf

Application of shrinkage techniques in logistic regression analysis: a case study

Prognostic models can be developed with multiple regression analysis of a data set containing individual patient data. Often this data set is relatively small, while previously published studies present results for larger numbers of patients. We describe a method to combine univariable regression results from the medical literature with univariable and multivariable results from the data set containing individual patient data. This ‘adaptation method’ exploits the generally strong correlation between univariable and multivariable regression coefficients. The method is illustrated with several logistic regression models to predict 30-day mortality in patients with acute myocardial infarction. The regression coefficients showed considerably less variability when estimated with the adaptation method, compared to standard maximum likelihood estimates. Also, model performance, as distinguished in calibration and discrimination, improved clearly when compared to models including shrunk or penalized estimates. We conclude that prognostic models may benefit substantially from explicit incorporation of literature data. Copyright © 2000 John Wiley & Sons, Ltd.

Prognostic models based on literature and individual patient data in logistic regression analysis.

A computer simulation approach is used to test assumptions about sensitivity of mammography and physical examination, and about the duration of preclinical screen-detectable breast cancer. Values between 50% and 80% for the combined sensitivity of the 2 tests give a good explanation of the results of the HIP randomized trial of breast cancer screening. The mean duration of the preclinical stage can vary from 1.6 years for high sensitivity values to 2.7 years for low values. In comparison with previous analyses of the HIP data our estimate for the sensitivity is lower, and the mean duration of the preclinical stage is longer. This is a consequence of the use of a more detailed model in our analysis, allowing for a more complete use of the HIP data in testing model assumptions. Similar analyses of data from recent screening projects in The Netherlands resulted in compatible estimates for the duration of preclinical breast cancer.

J. D. F. Habbema

Papers

Stepwise selection in small data sets : A simulation study of bias in logistic regression analysis

The MISCAN simulation program for the evaluation of screening for disease

Application of shrinkage techniques in logistic regression analysis: a case study

Prognostic models based on literature and individual patient data in logistic regression analysis.

A model-based analysis of the HIP project for breast cancer screening.