The Number, Quality, and Coverage of Randomized Controlled Trials in Nephrology
01 Jan 2020-Vol. 4, Iss: 3, pp 11-12
TL;DR: In this paper, the authors evaluated the number, quality, and coverage of randomized controlled trials (RCT) in nephrology and found that the quality of RCT reporting has not improved over the past 30 years with unclear allocation concealment (89%), lack of reported blinding of outcome assessors (92%), and failure to perform?intention-to-treat analysis? particularly frequent.
Abstract: Randomized controlled trials (RCT) are the optimal study design to answer intervention questions. The authors evaluated the number, quality, and coverage of RCT in nephrology. MEDLINE was searched using the relevant medical subject headings for nephrology and 12 major specialties in internal medicine, limited by ?randomized controlled trial? as a publication type. A random selection of 160 RCT in nephrology (40 for each decade) published since 1966 and an additional 270 RCT from ongoing or published Cochrane systematic reviews in various areas of nephrology, dialysis, and transplantation were evaluated for quality of reporting using standard criteria. The number of RCT published in nephrology from 1966 to 2002 (2779) is fewer than all other specialties of internal medicine (range: 5335 in hematology to 27109 in cardiology) with the proportion of all citations which are RCT being the third lowest (1.15%). There has been an increase in both indices from 1966 to 1996, but not at a greater rate than other specialties, and there has been no increase over the past 5 yr. Some areas of nephrology, in particular glomerulonephritis, are clear outliers with very low numbers of RCT to guide clinical decision-making. Overall the quality of RCT reporting in nephrology is low and has not improved over the past 30 yr with unclear allocation concealment (89%), lack of reported blinding of outcome assessors (92%), and failure to perform ?intention-to-treat analysis? (50%) particularly frequent. The challenges of improving the quality and quantity of trials in nephrology are substantial, but they can be overcome by using standard guidelines and checklists for trial reporting, greater attention to the trial methods and not just the results, involving experts in trial design and reporting, multicenter collaboration, and larger and simpler trials.
Citations
More filters
TL;DR: The STOP ACEi trial will strengthen the evidence base and shed light on the potential merits and dangers of ACEi/ARB use in advanced CKD on renal function and cardiovascular outcomes.
Abstract: Chronic kidney disease (CKD) is a worldwide public health problem associated with a high prevalence of cardiovascular disease (CVD) and impaired quality of life. Previous research for preventing loss of glomerular filtration rate (GFR) has focused on reducing blood pressure (BP) and proteinuria. Angiotensin converting enzyme inhibitors (ACEi) and angiotensin II receptor antagonists (ARB) are commonly used in patients with early CKD, but their value in advanced CKD (estimated GFR (eGFR) ≤30 ml/min/1.73 m2) is unknown. There remains a debate about the omission of ACEi/ARB in patients with advanced CKD and their use in association with CVD or heart failure. Does the potential gain in eGFR with ACEi/ARB cessation outweigh the potential adverse cardiovascular outcomes? This paper reviews the current literature that addresses this issue. Several controversies are discussed. Although lowering BP reduces cardiovascular events, evidence suggests that ACEi/ARBs are not superior to other antihypertensive agents. There are no studies assessing the benefits of ACEi/ARB therapy in cardiovascular risk reduction in advanced non-dialysis CKD. The STOP ACEi trial will strengthen the evidence base and shed light on the potential merits and dangers of ACEi/ARB use in advanced CKD on renal function and cardiovascular outcomes.
1,419 citations
TL;DR: A five-stage paradigm is proposed to describe the development of innovative surgical procedures so that introduction and adoption of surgical innovations can derive from evidence-based principles rather than trial and error.
508 citations
TL;DR: A scientific workshop to critically examine available data to determine whether alternative GFR-based end points have sufficiently strong relationships with important clinical outcomes of CKD to be used in clinical trials concluded that a confirmed decline in estimated GFR of 30% over 2 to 3 years may be an acceptable surrogate end point in some circumstances.
411 citations
TL;DR: The first three KDIGO guidelines—treatment of hepatitis C, management of bone and mineral disease, and care of kidney transplant recipients—have been finalized and the next three—acute kidney injury, Management of glomerulonephritis, and management of blood pressure in chronic kidney disease—are under development.
Abstract: Kidney Disease: Improving Global Outcomes (KDIGO) is an independent organization with the mission to improve care and outcomes of patients with kidney disease worldwide through the development and coordination of clinical practice guidelines. KDIGO has established firm links with other organizations that have previously produced clinical practice guidelines in the field of kidney disease. The first three KDIGO guidelines--treatment of hepatitis C, management of bone and mineral disease, and care of kidney transplant recipients--have been finalized and the next three--acute kidney injury, management of glomerulonephritis, and management of blood pressure in chronic kidney disease--are under development. The ultimate goal is to cover most major aspects of care for patients with kidney disease. Corner stones of KDIGO's guideline development process are independent, multidisciplinary, international work groups, close collaboration with professional methodology experts who perform systematic evidence reviews, and open public review of each guideline. Grades of Recommendation Assessment, Development, and Evaluation (GRADE) methodology is applied for grading the quality of evidence and strength of recommendations. International conferences organized by KDIGO support the coordination of guideline development, assess the suitability of guideline topics and help to establish global consensus on definitions and policies.
298 citations
TL;DR: There appears to be sufficient evidence to recommend changes in proteinuria as a surrogate for kidney disease progression in only selected circumstances, and collaboration among many groups, including academia, industry, the FDA, and the National Institutes of Health are recommended to share data from past and future studies.
264 citations
References
More filters
TL;DR: An instrument to assess the quality of reports of randomized clinical trials (RCTs) in pain research is described and its use to determine the effect of rater blinding on the assessments of quality is described.
15,740 citations
TL;DR: Empirical evidence is provided that inadequate methodological approaches in controlled trials, particularly those representing poor allocation concealment, are associated with bias.
Abstract: Objective. —To determine if inadequate approaches to randomized controlled trial design and execution are associated with evidence of bias in estimating treatment effects. Design. —An observational study in which we assessed the methodological quality of 250 controlled trials from 33 meta-analyses and then analyzed, using multiple logistic regression models, the associations between those assessments and estimated treatment effects. Data Sources. —Meta-analyses from the Cochrane Pregnancy and Childbirth Database. Main Outcome Measures. —The associations between estimates of treatment effects and inadequate allocation concealment, exclusions after randomization, and lack of double-blinding. Results. —Compared with trials in which authors reported adequately concealed treatment allocation, trials in which concealment was either inadequate or unclear (did not report or incompletely reported a concealment approach) yielded larger estimates of treatment effects ( P P =.01), with odds ratios being exaggerated by 17%. Conclusions. —This study provides empirical evidence that inadequate methodological approaches in controlled trials, particularly those representing poor allocation concealment, are associated with bias. Readers of trial reports should be wary of these pitfalls, and investigators must improve their design, execution, and reporting of trials. ( JAMA . 1995;273:408-412)
5,765 citations
TL;DR: The revised CONSORT statement is intended to improve the reporting of an RCT, enabling readers to understand a trial's conduct and to assess the validity of its results.
4,977 citations
TL;DR: Study of low methodological quality in which the estimate of quality is incorporated into the meta-analyses can alter the interpretation of the benefit of intervention, whether a scale or component approach is used in the assessment of trial quality.
3,129 citations
Journal Article•
TL;DR: In this article, the authors studied the potential association between reported methodologic quality and intervention effects to assess whether discrepancies between the results of large and small randomized trials in meta-analyses.
Abstract: Discrepancies may occur between the results of large randomized trials and the pooled results of several small trials in meta-analyses (14). Previous studies have suggested that discrepancies may be due to publication bias, that is, the fact that small trials are more likely to be published if they show a statistically significant intervention effect (58). Previous empirical studies of the association between methodologic quality and intervention effects have had inconsistent conclusions (912). In theory, adequate randomization requires adequate generation of the allocation sequence and adequate allocation concealment. The assumption is partly supported by studies from Schulz and colleagues (10) and Moher and associates (11, 12), who found that trials with inadequate allocation concealment exaggerate intervention effects significantly compared with trials reporting adequate allocation concealment. However, Emerson and coworkers (9) found no association between reported allocation concealment and intervention effects. Furthermore, none of the studies (912) found a significant association between generation of allocation sequence and intervention effects, although Schulz and colleagues found a nonsignificant trend (10). Schulz and colleagues (10), who analyzed trials in pregnancy and childbirth, found that trials without double blinding exaggerate intervention effects significantly compared with double-blind trials. However, Emerson and coworkers (9) and Moher and associates (11), who included trials from several therapeutic areas, found no association between double blinding and intervention effects. Methodologic quality can be assessed by using separate components, as in the studies discussed above, or by using one of several quality scales (13). One popular scale, developed by Jadad and colleagues in 1996 (14), has thus far received 233 registered citations (Institute for Scientific Information, Philadelphia, Pennsylvania). The scale includes assessment of the reported generation of allocation sequence, double blinding, and follow-up. Moher and associates (11, 12) found that trials with a low score on this scale exaggerate intervention effects significantly compared with trials that have high quality scores. However, the use of this and other quality scales has been disputed by Jni and coworkers (15), who showed that several quality scales produce inconsistent conclusions. We studied the potential association between reported methodologic quality and intervention effects to assess whether methodologic quality may explain discrepancies between the results of large and small randomized trials in meta-analyses. Methods Identification and Selection of Meta-Analyses and Trials According to suggestions in other studies (24), we arbitrarily defined trials with 1000 or more participants as large. We searched the Cochrane Library, MEDLINE on PubMed (using meta-analysis, review, randomi*ed, and controlled clinical trial as free text search words), and reference lists of relevant articles to identify potentially eligible meta-analyses that included at least one large trial. We identified 23 eligible meta-analyses. Nine were excluded because they included trials that were also included in larger eligible meta-analyses (n = 5), lacked references to the primary trials (n = 3), or excluded low-quality trials (n = 1). Accordingly, 14 meta-analyses (1626) were included. Three of the included Cochrane Reviews included two meta-analyses each (22, 25, 26). The meta-analyses included 248 trials, of which 58 were excluded because they were unpublished (n = 33), were quasi-randomized (n = 15), or were published as abstracts (n = 8). We were unable to translate 2 Spanish-language articles. The remaining 190 trials, published as English-language (n = 188) or German-language (n = 2) full-length articles, were included. Our analyses included 23 large and 167 small randomized trials and a total of 136 164 participants. On the basis of the study by Schulz and colleagues (10), who analyzed 250 controlled trials with 62 091 participants, we estimated that our sample size would be large enough to show significant differences between intervention benefits in high-quality and low-quality trials. Assessment of Methodologic Quality Methodologic quality was defined as the confidence that the trial's design, conduct, analysis, and presentation minimized or avoided biases in the trial's intervention comparisons (12). The reported methodologic quality was assessed in an unmasked manner by using four separate components and a composite quality scale. The four components were generation of the allocation sequence (adequate [computer-generated random numbers or similar] or inadequate [not described]), allocation concealment (adequate [central independent unit, sealed envelopes, or similar] or inadequate [not described or open table of random numbers or similar]), double blinding (adequate [identical placebo tablets or similar] or inadequate [not performed or tablets versus injections or similar]), and follow-up (adequate [number and reasons for dropouts and withdrawals described] or inadequate [number or reasons for dropouts and withdrawals not described]). The five-point quality scale included generation of the allocation sequence (2 points, computer-generated random numbers or similar; 1 point, not described; 0 points, quasi-randomized trial [which we excluded]), double blinding (2 points, identical placebo tablets or similar; 1 point, not described; 0 points, no blinding or inadequate method, such as tablets versus injections or similar), and follow-up (1 point, number and reasons for dropouts and withdrawals described; 0 points, number or reasons for dropouts and withdrawals not described). The quality score was ranked as low ( 2 points) or high ( 3 points), as suggested elsewhere (11). Two reviewers assessed the effect of masking and the interobserver reliability of the quality assessments. The reported methodologic quality of 30 trials was assessed with and without masking the names of the authors and journal, the year of publication, acknowledgments, institutional affiliations, and funding. The unmasked assessments were performed first. The masked assessments were performed 3 months later, ensuring that the assessors did not remember the trials. The difference between masked and unmasked quality assessments was not significant (mean [SE] quality score, 3.70 0.19 vs. 3.63 0.21; P > 0.2). Interobserver reliability was assessed by using 30 randomized trials randomly selected from the Cochrane Hepato-Biliary Group Controlled Trials Register and was found to be high (intraclass correlation coefficient, 0.96 [95% CI, 0.92 to 0.98]) (27). After assessing the quality of 100 trials, we reassessed the methodologic quality of the trials from the Cochrane Hepato-Biliary Group Controlled Trials Register and found that the testretest reliability was high (0.98 [CI, 0.97 to 0.99]) (27). Data Extraction Data were extracted independently by two reviewers. First, the primary binary outcome measure described by the largest number of trials in each meta-analysis was identified. We then extracted the number of events in the intervention and control groups and the number of participants randomly assigned to the intervention and control groups. All disagreements were due to inaccurate data extraction and were resolved through further consulting of the original articles and meta-analyses. Consensus was achieved before analyses were done in all cases. Statistical Analysis Analyses were performed by using SAS for Windows, version 6.12 (SAS Institute, Inc., Cary, North Carolina) or SPSS for Windows, version 10.0 (SPSS, Inc., Chicago, Illinois). Differences between the masked and unmasked quality assessments were estimated by using the KruskalWallis test. The number of participants and the year of publication in trials with adequate versus inadequate generation of allocation sequence, allocation concealment, double blinding, and follow-up and high versus low quality scores were compared by using the Pearson chi-square test. To estimate evidence of publication bias and other biases, we used linear regression to analyze funnel-plot asymmetry (7). The standard normal deviate, defined as the log odds ratio divided by its standard error, was regressed against the precision (the inverse of the standard error). If funnel-plot asymmetry is present, the regression line will not run through the origin, and the intercept will provide a measure of asymmetry. Intervention effects were estimated by using the number of events and participants in the treatment group and the number of events and participants in the control group (Appendix). Accordingly, two observations were needed per trial, one for the intervention group and one for the control group. When necessary, positive outcomes were re-expressed as unwanted end points, for example, mortality instead of survival. Discrepancies between intervention effects in large and small trials were estimated by the ratio of odds ratios (10), which is the summary odds ratio of large trials divided by the summary odds ratio of small trials. In this modeling convention, a ratio of odds ratios less than 1.0 indicates that a group of trials (for example, small trials with inadequate allocation concealment) exaggerates the intervention effect compared with the referent group (for example, large trials). Variance and confidence intervals were increased to adjust for overdispersion (Appendix). The Pearson chi-square test was used to estimate the potential overlap between quality components. Results Characteristics of Included Trials We were able to locate and retrieve all 190 included trials and to reproduce the results reported in the meta-analyses (1626) (Table 1). Of the 190 trials, 81 (43%) reported adequate generation of the allocation sequence, 68 (36%) reported adequate allocation concealment, 103 (54%) reported adequate double blinding, and 1
1,594 citations