Subgroup analysis, covariate adjustment and baseline comparisons in clinical trial reporting: current practice and problems.
TL;DR: Key issues include: the overuse and overinterpretation of subgroup analyses; the underuse of appropriate statistical tests for interaction; inconsistencies in the use of covariate-adjustment; the lack of clear guidelines on covariate selection; the over use of baseline comparisons in some studies; the misuses of significance tests for baseline comparability, and the need for trials to have a predefined statistical analysis plan.
Abstract: Clinical trial investigators often record a great deal of baseline data on each patient at randomization When reporting the trial's findings such baseline data can be used for (i) subgroup analyses which explore whether there is evidence that the treatment difference depends on certain patient characteristics, (ii) covariate-adjusted analyses which aim to refine the analysis of the overall treatment difference by taking account of the fact that some baseline characteristics are related to outcome and may be unbalanced between treatment groups, and (iii) baseline comparisons which compare the baseline characteristics of patients in each treatment group for any possible (unlucky) differences This paper examines how these issues are currently tackled in the medical journals, based on a recent survey of 50 trial reports in four major journals The statistical ramifications are explored, major problems are highlighted and recommendations for future practice are proposed Key issues include: the overuse and overinterpretation of subgroup analyses; the underuse of appropriate statistical tests for interaction; inconsistencies in the use of covariate-adjustment; the lack of clear guidelines on covariate selection; the overuse of baseline comparisons in some studies; the misuses of significance tests for baseline comparability, and the need for trials to have a predefined statistical analysis plan for all these uses of baseline data
Citations
More filters
TL;DR: Exercise training was associated with modest significant reductions for both all-cause mortality or hospitalization and cardiovascular mortality or heart failure hospitalization, and in key secondary clinical end points.
Abstract: Context Guidelines recommend that exercise training be considered for medically stable outpatients with heart failure. Previous studies have not had adequate statistical power to measure the effects of exercise training on clinical outcomes. Objective To test the efficacy and safety of exercise training among patients with heart failure. Design, Setting, and Patients Multicenter, randomized controlled trial of 2331 medically stable outpatients with heart failure and reduced ejection fraction. Participants in Heart Failure: A Controlled Trial Investigating Outcomes of Exercise Training (HF-ACTION) were randomized from April 2003 through February 2007 at 82 centers within the United States, Canada, and France; median follow-up was 30 months. Interventions Usual care plus aerobic exercise training, consisting of 36 supervised sessions followed by home-based training, or usual care alone. Main Outcome Measures Composite primary end point of all-cause mortality or hospitalization and prespecified secondary end points of all-cause mortality, cardiovascular mortality or cardiovascular hospitalization, and cardiovascular mortality or heart failure hospitalization. Results The median age was 59 years, 28% were women, and 37% had New York Heart Association class III or IV symptoms. Heart failure etiology was ischemic in 51%, and median left ventricular ejection fraction was 25%. Exercise adherence decreased from a median of 95 minutes per week during months 4 through 6 of follow-up to 74 minutes per week during months 10 through 12. A total of 759 patients (65%) in the exercise training group died or were hospitalized compared with 796 patients (68%) in the usual care group (hazard ratio [HR], 0.93 [95% confidence interval {CI}, 0.84-1.02]; P = .13). There were nonsignificant reductions in the exercise training group for mortality (189 patients [16%] in the exercise training group vs 198 patients [17%] in the usual care group; HR, 0.96 [95% CI, 0.79-1.17]; P = .70), cardiovascular mortality or cardiovascular hospitalization (632 [55%] in the exercise training group vs 677 [58%] in the usual care group; HR, 0.92 [95% CI, 0.83-1.03]; P = .14), and cardiovascular mortality or heart failure hospitalization (344 [30%] in the exercise training group vs 393 [34%] in the usual care group; HR, 0.87 [95% CI, 0.75-1.00]; P = .06). In prespecified supplementary analyses adjusting for highly prognostic baseline characteristics, the HRs were 0.89 (95% CI, 0.81-0.99; P = .03) for all-cause mortality or hospitalization, 0.91 (95% CI, 0.82-1.01; P = .09) for cardiovascular mortality or cardiovascular hospitalization, and 0.85 (95% CI, 0.74-0.99; P = .03) for cardiovascular mortality or heart failure hospitalization. Other adverse events were similar between the groups. Conclusions In the protocol-specified primary analysis, exercise training resulted in nonsignificant reductions in the primary end point of all-cause mortality or hospitalization and in key secondary clinical end points. After adjustment for highly prognostic predictors of the primary end point, exercise training was associated with modest significant reductions for both all-cause mortality or hospitalization and cardiovascular mortality or heart failure hospitalization. Trial Registration clinicaltrials.gov Identifier: NCT00047437
1,777 citations
TL;DR: The analysis of subgroups is often used as a way to glean additional information from data sets and the strengths and weaknesses of this approach and new Journal policies concerning the reporting of subgroup analyses are discussed.
Abstract: The analysis of subgroups is often used as a way to glean additional information from data sets. The strengths and weaknesses of this approach and new Journal policies concerning the reporting of subgroup analyses are discussed in this article.
1,204 citations
TL;DR: This report describes a set of scientific procedures used to assess the impact of foods and food ingredients on the expression of appetite (psychological and behavioural), and allows the evaluation of the strength of health claims about the effects of foods on appetite.
Abstract: This report describes a set of scientific procedures used to assess the impact of foods and food ingredients on the expression of appetite (psychological and behavioural). An overarching priority has been to enable potential evaluators of health claims about foods to identify justified claims and to exclude claims that are not supported by scientific evidence for the effect cited. This priority follows precisely from the principles set down in the PASSCLAIM report. The report allows the evaluation of the strength of health claims, about the effects of foods on appetite, which can be sustained on the basis of the commonly used scientific designs and experimental procedures. The report includes different designs for assessing effects on satiation as opposed to satiety, detailed coverage of the extent to which a change in hunger can stand alone as a measure of appetite control and an extensive discussion of the statistical procedures appropriate for handling data in this field of research. Because research in this area is continually evolving, new improved methodologies may emerge over time and will need to be incorporated into the framework. One main objective of the report has been to produce guidance on good practice in carrying out appetite research, and not to set down a series of commandments that must be followed.
858 citations
TL;DR: Applied researchers are encouraged to use propensity score matching and IPTW using the propensity score when estimating the relative effect of treatment on time-to-event outcomes, as both approaches allow for the estimation of marginal hazard ratios with minimal bias.
Abstract: Propensity score methods are increasingly being used to reduce or minimize the effects of confounding when estimating the effects of treatments, exposures, or interventions when using observational or non-randomized data. Under the assumption of no unmeasured confounders, previous research has shown that propensity score methods allow for unbiased estimation of linear treatment effects (e.g., differences in means or proportions). However, in biomedical research, time-to-event outcomes occur frequently. There is a paucity of research into the performance of different propensity score methods for estimating the effect of treatment on time-to-event outcomes. Furthermore, propensity score methods allow for the estimation of marginal or population-average treatment effects. We conducted an extensive series of Monte Carlo simulations to examine the performance of propensity score matching (1:1 greedy nearest-neighbor matching within propensity score calipers), stratification on the propensity score, inverse probability of treatment weighting (IPTW) using the propensity score, and covariate adjustment using the propensity score to estimate marginal hazard ratios. We found that both propensity score matching and IPTW using the propensity score allow for the estimation of marginal hazard ratios with minimal bias. Of these two approaches, IPTW using the propensity score resulted in estimates with lower mean squared error when estimating the effect of treatment in the treated. Stratification on the propensity score and covariate adjustment using the propensity score result in biased estimation of both marginal and conditional hazard ratios. Applied researchers are encouraged to use propensity score matching and IPTW using the propensity score when estimating the relative effect of treatment on time-to-event outcomes. Copyright © 2012 John Wiley & Sons, Ltd.
821 citations
Additional excerpts
..., marginal) estimate [27]....
[...]
TL;DR: Among patients with stable atherosclerosis, low‐dose methotrexate did not reduce levels of interleukin‐1β, interleUKin‐6, or C‐reactive protein and did not result in fewer cardiovascular events than placebo and was associated with elevations in liver‐enzyme levels, reductions in leukocyte counts and hematocrit levels, and a higher incidence of non–basal‐cell skin cancers than placebo.
Abstract: Background Inflammation is causally related to atherothrombosis. Treatment with canakinumab, a monoclonal antibody that inhibits inflammation by neutralizing interleukin-1β, resulted in a lower rate of cardiovascular events than placebo in a previous randomized trial. We sought to determine whether an alternative approach to inflammation inhibition with low-dose methotrexate might provide similar benefit. Methods We conducted a randomized, double-blind trial of low-dose methotrexate (at a target dose of 15 to 20 mg weekly) or matching placebo in 4786 patients with previous myocardial infarction or multivessel coronary disease who additionally had either type 2 diabetes or the metabolic syndrome. All participants received 1 mg of folate daily. The primary end point at the onset of the trial was a composite of nonfatal myocardial infarction, nonfatal stroke, or cardiovascular death. Near the conclusion of the trial, but before unblinding, hospitalization for unstable angina that led to urgent revas...
802 citations
References
More filters
TL;DR: The revised CONSORT statement is intended to improve the reporting of an RCT, enabling readers to understand a trial's conduct and to assess the validity of its results.
4,977 citations
TL;DR: The Consort Statement as mentioned in this paper is a group of scientists and editors developed to improve the quality of reporting of randomized, controlled trials (RCTs) by providing guidance to authors about how to improve their reporting of their trials.
Abstract: Overwhelming evidence now indicates that the quality of reporting of randomized, controlled trials (RCTs) is less than optimal. Recent methodologic analyses indicate that inadequate reporting and design are associated with biased estimates of treatment effects. Such systematic error is seriously damaging to RCTs, which boast the elimination of systematic error as their primary hallmark. Systematic error in RCTs reflects poor science, and poor science threatens proper ethical standards. A group of scientists and editors developed the CONSORT (Consolidated Standards of Reporting Trials) statement to improve the quality of reporting of RCTs. The statement consists of a checklist and flow diagram that authors can use for reporting an RCT. Many leading medical journals and major international editorial groups have adopted the CONSORT statement. The CONSORT statement facilitates critical appraisal and interpretation of RCTs by providing guidance to authors about how to improve the reporting of their trials. This explanatory and elaboration document is intended to enhance the use, understanding, and dissemination of the CONSORT statement. The meaning and rationale for each checklist item are presented. For most items, at least one published example of good reporting and, where possible, references to relevant empirical studies are provided. Several examples of flow diagrams are included. The CONSORT statement, this explanatory and elaboration document, and the associated Web site (http://www.consort -statement.org) should be helpful resources to improve reporting of randomized trials.
3,647 citations
TL;DR: The revised CONSORT statement is intended to improve the reporting of an RCT, enabling readers to understand a trial's conduct and to assess the validity of its results.
Abstract: To comprehend the results of a randomized controlled trial (RCT), readers must understand its design, conduct, analysis and interpretation. That goal can only be achieved through complete transparency from authors. Despite several decades of educational efforts, the reporting of RCTs needs improvement. Investigators and editors developed the original CONSORT (Consolidated Standards of Reporting Trials) statement to help authors improve reporting by using a checklist and flow diagram. The revised CONSORT statement presented in this paper incorporates new evidence and addresses some criticisms of the original statement.
2,444 citations
"Subgroup analysis, covariate adjust..." refers methods in this paper
...guidelines such as CONSORT [27, 28], this paper has identi ed some important de ciencies and inconsistencies as regards the uses of baseline data in clinical trial reports....
[...]
TL;DR: The revised CONSORT statement is intended to improve the reporting of an RCT, enabling readers to understand a trial's conduct and to assess the validity of its results.
Abstract: To comprehend the results of a randomised controlled trial (RCT), readers must understand its design, conduct, analysis, and interpretation. That goal can be achieved only through total transparency from authors. Despite several decades of educational efforts, the reporting of RCTs needs improvement. Investigators and editors developed the original CONSORT (Consolidated Standards of Reporting Trials) statement to help authors improve reporting by use of a checklist and flow diagram. The revised CONSORT statement presented here incorporates new evidence and addresses some criticisms of the original statement. The checklist items pertain to the content of the Title, Abstract, Introduction, Methods, Results, and Discussion. The revised checklist includes 22 items selected because empirical evidence indicates that not reporting this information is associated with biased estimates of treatment effect, or because the information is essential to judge the reliability or relevance of the findings. We intended the flow diagram to depict the passage of participants through an RCT. The revised flow diagram depicts information from four stages of a trial (enrollment, intervention allocation, follow- up, and analysis). The diagram explicitly shows the number of participants, for each intervention group, included in the primary data analysis. Inclusion of these numbers allows the reader to judge whether the authors have done an intention- to-treat analysis. In sum, the CONSORT statement is intended to improve the reporting of an RCT, enabling readers to understand a trial's conduct and to assess the validity of its results.
2,011 citations
"Subgroup analysis, covariate adjust..." refers methods in this paper
...CONCLUSIONS While there have been improvements in the standards of reporting for clinical trials, aided by guidelines such as CONSORT [27, 28], this paper has identi ed some important de ciencies and inconsistencies as regards the uses of baseline data in clinical trial reports....
[...]
...guidelines such as CONSORT [27, 28], this paper has identi ed some important de ciencies and inconsistencies as regards the uses of baseline data in clinical trial reports....
[...]
TL;DR: In this paper, the authors examined the long-term effects of home visitation by nurses on women's life course and child abuse and neglect in a semi-urban community in New York.
Abstract: Context. —Home-visitation services have been promoted as a means of improving maternal and child health and functioning. However, long-term effects have not been examined. objective. —To examine the long-term effects of a program of prenatal and early childhood home visitation by nurses on women's life course and child abuse and neglect. Design. —Randomized trial. Setting. —Semirural community in New York. Participants. —Of 400 consecutive pregnant women with no previous live births enrolled, 324 participated in a follow-up study when their children were 15 years old. Intervention. —Families received a mean of 9 home visits during pregnancy and 23 home visits from the child's birth through the second birthday. Data Sources and Measures. —Women's use of welfare and number of subsequent children were based on self-report; their arrests and convictions were based on self-report and archived data from New York State. Verified reports of child abuse and neglect were abstracted from state records. Main Results. —During the 15-year period after the birth of their first child, in contrast to women in the comparison group, women who were visited by nurses during pregnancy and infancy were identified as perpetrators of child abuse and neglect in 0.29 vs 0.54 verified reports ( P P =.02), 65 vs 37 months between the birth of the first and a second child ( P =.001), 60 vs 90 months' receiving Aid to Families With Dependent Children ( P =.005), 0.41 vs 0.73 behavioral impairments due to use of alcohol and other drugs ( P =.03), 0.18 vs 0.58 arrests by self-report ( P P Conclusions. —This program of prenatal and early childhood home visitation by nurses can reduce the number of subsequent pregnancies, the use of welfare, child abuse and neglect, and criminal behavior on the part of low-income, unmarried mothers for up to 15 years after the birth of the first child.
1,291 citations