The combination of estimates from different experiments.
TL;DR: The problem of making a combined estimate has been discussed previously by Cochran and Yates and Cochran (1937) for agricultural experiments, and by Bliss (1952) for bioassays in different laboratories as discussed by the authors.
Abstract: When we are trying to make the best estimate of some quantity A that is available from the research conducted to date, the problem of combining results from different experiments is encountered. The problem is often troublesome, particularly if the individual estimates were made by different workers using different procedures. This paper discusses one of the simpler aspects of the problem, in which there is sufficient uniformity of experimental methods so that the ith experiment provides an estimate xi of u, and an estimate si of the standard error of xi . The experiments may be, for example, determinations of a physical or astronomical constant by different scientists, or bioassays carried out in different laboratories, or agricultural field experiments laid out in different parts of a region. The quantity xi may be a simple mean of the observations, as in a physical determination, or the difference between the means of two treatments, as in a comparative experiment, or a median lethal dose, or a regression coefficient. The problem of making a combined estimate has been discussed previously by Cochran (1937) and Yates and Cochran (1938) for agricultural experiments, and by Bliss (1952) for bioassays in different laboratories. The last two papers give recommendations for the practical worker. My purposes in treating the subject again are to discuss it in more general terms, to take account of some recent theoretical research, and, I hope, to bring the practical recommendations to the attention of some biologists who are not acquainted with the previous papers. The basic issue with which this paper deals is as follows. The simplest method of combining estimates made in a number of different experiments is to take the arithmetic mean of the estimates. If, however, the experiments vary in size, or appear to be of different precision, the investigator may wonder whether some kind of weighted meani would be more precise. This paper gives recommendations about the kinds of weighted mean that are appropriate, the situations in which they
Citations
More filters
TL;DR: A new quantity is developed, I 2, which the authors believe gives a better measure of the consistency between trials in a meta-analysis, which is susceptible to the number of trials included in the meta- analysis.
Abstract: Cochrane Reviews have recently started including the quantity I 2 to help readers assess the consistency of the results of studies in meta-analyses. What does this new quantity mean, and why is assessment of heterogeneity so important to clinical practice?
Systematic reviews and meta-analyses can provide convincing and reliable evidence relevant to many aspects of medicine and health care.1 Their value is especially clear when the results of the studies they include show clinically important effects of similar magnitude. However, the conclusions are less clear when the included studies have differing results. In an attempt to establish whether studies are consistent, reports of meta-analyses commonly present a statistical test of heterogeneity. The test seeks to determine whether there are genuine differences underlying the results of the studies (heterogeneity), or whether the variation in findings is compatible with chance alone (homogeneity). However, the test is susceptible to the number of trials included in the meta-analysis. We have developed a new quantity, I 2, which we believe gives a better measure of the consistency between trials in a meta-analysis.
Assessment of the consistency of effects across studies is an essential part of meta-analysis. Unless we know how consistent the results of studies are, we cannot determine the generalisability of the findings of the meta-analysis. Indeed, several hierarchical systems for grading evidence state that the results of studies must be consistent or homogeneous to obtain the highest grading.2–4
Tests for heterogeneity are commonly used to decide on methods for combining studies and for concluding consistency or inconsistency of findings.5 6 But what does the test achieve in practice, and how should the resulting P values be interpreted?
A test for heterogeneity examines the null hypothesis that all studies are evaluating the same effect. The usual test statistic …
45,105 citations
TL;DR: It is concluded that H and I2, which can usually be calculated for published meta-analyses, are particularly useful summaries of the impact of heterogeneity, and one or both should be presented in publishedMeta-an analyses in preference to the test for heterogeneity.
Abstract: The extent of heterogeneity in a meta-analysis partly determines the difficulty in drawing overall conclusions. This extent may be measured by estimating a between-study variance, but interpretation is then specific to a particular treatment effect metric. A test for the existence of heterogeneity exists, but depends on the number of studies in the meta-analysis. We develop measures of the impact of heterogeneity on a meta-analysis, from mathematical criteria, that are independent of the number of studies and the treatment effect metric. We derive and propose three suitable statistics: H is the square root of the chi2 heterogeneity statistic divided by its degrees of freedom; R is the ratio of the standard error of the underlying mean from a random effects meta-analysis to the standard error of a fixed effect meta-analytic estimate, and I2 is a transformation of (H) that describes the proportion of total variation in study estimates that is due to heterogeneity. We discuss interpretation, interval estimates and other properties of these measures and examine them in five example data sets showing different amounts of heterogeneity. We conclude that H and I2, which can usually be calculated for published meta-analyses, are particularly useful summaries of the impact of heterogeneity. One or both should be presented in published meta-analyses in preference to the test for heterogeneity.
25,460 citations
TL;DR: METAL provides a computationally efficient tool for meta-analysis of genome-wide association scans, which is a commonly used approach for improving power complex traits gene mapping studies.
Abstract: Summary: METAL provides a computationally efficient tool for meta-analysis of genome-wide association scans, which is a commonly used approach for improving power complex traits gene mapping studies. METAL provides a rich scripting interface and implements efficient memory management to allow analyses of very large data sets and to support a variety of input file formats.
Availability and implementation: METAL, including source code, documentation, examples, and executables, is available at http://www.sph.umich.edu/csg/abecasis/metal/
Contact: ude.hcimu@olacnog
3,994 citations
Cites methods from "The combination of estimates from d..."
...METAL implements Cochran’s Q-test for heterogeneity (Cochran, 1954) and the appropriate statistics can be calculated if requested by the user....
[...]
...METAL implements Cochran’s Q test for heterogeneity (Cochran, 1954) and the appropriate statistics can be calculated if requested by the user....
[...]
TL;DR: In this pooled analysis of individual data from 9 selected cohorts, gait speed was associated with survival in older adults and predicted survival was as accurate as predicted based on age, sex, use of mobility aids, and self-reported function.
Abstract: Context Survival estimates help individualize goals of care for geriatric patients, but life tables fail to account for the great variability in survival. Physical performance measures, such as gait speed, might help account for variability, allowing clinicians to make more individualized estimates. Objective To evaluate the relationship between gait speed and survival. Design, Setting, and Participants Pooled analysis of 9 cohort studies (collected between 1986 and 2000), using individual data from 34 485 community-dwelling older adults aged 65 years or older with baseline gait speed data, followed up for 6 to 21 years. Participants were a mean (SD) age of 73.5 (5.9) years; 59.6%, women; and 79.8%, white; and had a mean (SD) gait speed of 0.92 (0.27) m/s. Main Outcome Measures Survival rates and life expectancy. Results There were 17 528 deaths; the overall 5-year survival rate was 84.8% (confidence interval [CI], 79.6%-88.8%) and 10-year survival rate was 59.7% (95% CI, 46.5%-70.6%). Gait speed was associated with survival in all studies (pooled hazard ratio per 0.1 m/s, 0.88; 95% CI, 0.87-0.90; P Conclusion In this pooled analysis of individual data from 9 selected cohorts, gait speed was associated with survival in older adults.
3,393 citations
Additional excerpts
...6 77 (72-81) 57 (49-64) 31 (24-39) 88 (85-90) 75 (68-80) 61 (50-70) 53 (41-64) 23 (15-31) 6 (3-11) 67 (61-72) 42 (36-48) 18 (9-30)...
[...]
...2 90 (85-93) 83 (76-87) 68 (57-77) 96 (94-98) 91 (87-94) 61 (35-79) 69 (63-74) 53 (46-59) 50 (6-84) 86 (82-89) 73 (70-77) 33 (13-54)...
[...]
...8 79 (74-83) 65 (57-71) 49 (35-61) 91 (89-93) 82 (78-86) 74 (69-78) 57 (52-62) 31 (24-38) 11 (3-28) 74 (71-77) 52 (46-57) 23 (18-28)...
[...]
...4 68 (47-82) 60 (38-76) 25 (15-36) 80 (71-86) 69 (58-78) 47 (40-54) 56 (23-80) 15 (4-33) 8 (3-18) 58 (46-69) 35 (24-47) 11 (5-19)...
[...]
...0 85 (82-88) 75 (69-79) 54 (43-64) 93 (91-95) 89 (86-91) 73 (59-83) 67 (62-71) 43 (36-50) 14 (7-25) 80 (75-83) 62 (56-68) 39 (22-56)...
[...]
TL;DR: The results show the utility of the I(2) index as a complement to the Q test, although it has the same problems of power with a small number of studies.
Abstract: In meta-analysis, the usual way of assessing whether a set of single studies is homogeneous is by means of the Q test. However, the Q test only informs meta-analysts about the presence versus the absence of heterogeneity, but it does not report on the extent of such heterogeneity. Recently, the I(2) index has been proposed to quantify the degree of heterogeneity in a meta-analysis. In this article, the performances of the Q test and the confidence interval around the I(2) index are compared by means of a Monte Carlo simulation. The results show the utility of the I(2) index as a complement to the Q test, although it has the same problems of power with a small number of studies.
2,750 citations
Cites background or methods from "The combination of estimates from d..."
...…2006, Vol. 11, No. 2, 193–206 Copyright 2006 by the American Psychological Association 1082-989X/06/$12.00 DOI: 10.1037/1082-989X.11.2.193 193 The usual way of assessing whether there is true heterogeneity in a meta-analysis has been to use the Q test, a statistical test defined by Cochran (1954)....
[...]
...…test usually applied in meta-analysis for determining whether there is true heterogeneity among the studies’ effects is the Q test, proposed by Cochran (1954) and defined by Hedges and Olkin (1985, p. 123, Equation 25) as: Q wi Ti T 2, (12) where wi is the weighting factor for the ith study…...
[...]
...…Q k 1 c 0 for Q (k 1) for Q (k 1) (10) being c c wi wi2 wi, (11) where wi is the weighting factor for the ith study assuming a fixed-effects model (wi 1/̂i 2), k is the number of studies, and Q is the statistical test for heterogeneity proposed by Cochran (1954) and defined in Equation 12....
[...]
References
More filters
3,586 citations
1,383 citations
Book•
15 Jan 1952TL;DR: In this article, Monterey describes a books design and analysis of experiments, and the pronouncement as without difficulty as perspicacity of this design and analyses of experiments montgomery can be taken as skillfully as picked to act.
Abstract: Yeah, reviewing a books design and analysis of experiments montgomery could mount up your close associates listings. This is just one of the solutions for you to be successful. As understood, achievement does not suggest that you have wonderful points. Comprehending as skillfully as covenant even more than extra will have enough money each success. next-door to, the pronouncement as without difficulty as perspicacity of this design and analysis of experiments montgomery can be taken as skillfully as picked to act. Page Url
1,064 citations
TL;DR: It is pointed out that the ordinary analysis of variance procedure suitable for dealing with the results of a single experiment may require modification, owing to lack of equality in the errors of the different experiments, and owing to non-homogeneity of the components of the interaction of treatments with places and times.
Abstract: When a set of experiments involving the same or similar treatments is carried out at a number of places, or in a number of years, the results usually require comprehensive examination and summary. In general, each set of results must be considered on its merits, and it is not possible to lay down rules of procedure that will be applicable in all cases, but there are certain preliminary steps in the analysis which can be dealt with in general terms. These are discussed in the present paper and illustrated by actual examples. It is pointed out that the ordinary analysis of variance procedure suitable for dealing with the results of a single experiment may require modification, owing to lack of equality in the errors of the different experiments, and owing to non-homogeneity of the components of the interaction of treatments with places and times.
837 citations