Methods of Formal Consensus in Classification/Diagnostic Criteria and Guideline Development
TL;DR: 4 types of formal consensus methods used in the health field and their applications in rheumatology are discussed: the Delphi method, Nominal Group Technique, RAND/UCLA Appropriateness Method, and National Institutes of Health consensus development conference.
About: This article is published in Seminars in Arthritis and Rheumatism.The article was published on 2011-10-01 and is currently open access. It has received 296 citations till now. The article focuses on the topics: Nominal group technique & Delphi method.
Citations
More filters
Dresden University of Technology1, Brigham and Women's Hospital2, University of California, San Francisco3, University of Düsseldorf4, University of Pisa5, Northwestern University6, Medical University of Vienna7, National and Kapodistrian University of Athens8, Medical University of South Carolina9, University of Cambridge10, University of Barcelona11, The Feinstein Institute for Medical Research12, Toronto Western Hospital13, University of California, Los Angeles14, Humboldt University of Berlin15, Copenhagen University Hospital16, University of Michigan17, University of the Basque Country18, University Health Network19, University of Crete20, University of Zagreb21, University of Paris-Sud22, University of Hong Kong23, University of Calgary24, Hospital for Special Surgery25, University of Pécs26, University of Padua27, Medical University of Graz28, National Institutes of Health29, New York University30, Université Paris-Saclay31, University Hospital Complex Of Vigo32, University of Occupational and Environmental Health Japan33, University of Porto34, Leeds Teaching Hospitals NHS Trust35, Cedars-Sinai Medical Center36, Istanbul Bilim University37, McMaster University38
TL;DR: To develop new classification criteria for systemic lupus erythematosus (SLE) jointly supported by the European League Against Rheumatism and the American College of Rheumatology (ACR).
Abstract: Objective To develop new classification criteria for systemic lupus erythematosus (SLE) jointly supported by the European League Against Rheumatism (EULAR) and the American College of Rheumatology (ACR). Methods This international initiative had four phases. 1) Evaluation of antinuclear antibody (ANA) as an entry criterion through systematic review and meta-regression of the literature and criteria generation through an international Delphi exercise, an early patient cohort, and a patient survey. 2) Criteria reduction by Delphi and nominal group technique exercises. 3) Criteria definition and weighting based on criterion performance and on results of a multi-criteria decision analysis. 4) Refinement of weights and threshold scores in a new derivation cohort of 1,001 subjects and validation compared with previous criteria in a new validation cohort of 1,270 subjects. Results The 2019 EULAR/ACR classification criteria for SLE include positive ANA at least once as obligatory entry criterion; followed by additive weighted criteria grouped in 7 clinical (constitutional, hematologic, neuropsychiatric, mucocutaneous, serosal, musculoskeletal, renal) and 3 immunologic (antiphospholipid antibodies, complement proteins, SLE-specific antibodies) domains, and weighted from 2 to 10. Patients accumulating ≥10 points are classified. In the validation cohort, the new criteria had a sensitivity of 96.1% and specificity of 93.4%, compared with 82.8% sensitivity and 93.4% specificity of the ACR 1997 and 96.7% sensitivity and 83.7% specificity of the Systemic Lupus International Collaborating Clinics 2012 criteria. Conclusion These new classification criteria were developed using rigorous methodology with multidisciplinary and international input, and have excellent sensitivity and specificity. Use of ANA entry criterion, hierarchically clustered, and weighted criteria reflects current thinking about SLE and provides an improved foundation for SLE research.
1,018 citations
Cites methods from "Methods of Formal Consensus in Clas..."
...A consensus meeting of 19 international SLE experts (n = 7 nominal group technique [NGT] experts + steering committee + DK [moderator]) using NGT was conducted to reduce the list of criteria (20)....
[...]
Boston Children's Hospital1, Royal Children's Hospital2, Amrita Institute of Medical Sciences and Research Centre3, Stellenbosch University4, James Cook University5, Universidade Federal de Minas Gerais6, All India Institute of Medical Sciences7, Johns Hopkins University8, Sri Jayadeva Institute of Cardiovascular Sciences and Research9, University of Cape Town10
TL;DR: The standardized criteria for RHD aim to permit rapid and consistent identification of individuals with RHD without a clear history of acute rheumatic fever and hence allow enrollment into secondary prophylaxis programs.
Abstract: Over the past 5 years, the advent of echocardiographic screening for rheumatic heart disease (RHD) has revealed a higher RHD burden than previously thought. In light of this global experience, the development of new international echocardiographic guidelines that address the full spectrum of the rheumatic disease process is opportune. Systematic differences in the reporting of and diagnostic approach to RHD exist, reflecting differences in local experience and disease patterns. The World Heart Federation echocardiographic criteria for RHD have, therefore, been developed and are formulated on the basis of the best available evidence. Three categories are defined on the basis of assessment by 2D, continuous-wave, and color-Doppler echocardiography: 'definite RHD', 'borderline RHD', and 'normal'. Four subcategories of 'definite RHD' and three subcategories of 'borderline RHD' exist, to reflect the various disease patterns. The morphological features of RHD and the criteria for pathological mitral and aortic regurgitation are also defined. The criteria are modified for those aged over 20 years on the basis of the available evidence. The standardized criteria aim to permit rapid and consistent identification of individuals with RHD without a clear history of acute rheumatic fever and hence allow enrollment into secondary prophylaxis programs. However, important unanswered questions remain about the importance of subclinical disease (borderline or definite RHD on echocardiography without a clinical pathological murmur), and about the practicalities of implementing screening programs. These standardized criteria will help enable new studies to be designed to evaluate the role of echocardiographic screening in RHD control.
621 citations
Dresden University of Technology1, Brigham and Women's Hospital2, University of California, San Francisco3, University of Düsseldorf4, University of Pisa5, Northwestern University6, Medical University of Vienna7, National and Kapodistrian University of Athens8, University of Cyprus9, Medical University of South Carolina10, University of Cambridge11, University of Barcelona12, University Health Network13, The Feinstein Institute for Medical Research14, Toronto Western Hospital15, University of California, Los Angeles16, Humboldt University of Berlin17, Copenhagen University Hospital18, University of Michigan19, Harvard University20, University of the Basque Country21, University of Crete22, University of Kiel23, University Hospital Centre Zagreb24, University of Paris-Sud25, University of Hong Kong26, University of Calgary27, Hospital for Special Surgery28, University of Pécs29, University of Padua30, Medical University of Graz31, National Institutes of Health32, New York University33, Université Paris-Saclay34, University Hospital Complex Of Vigo35, University of Occupational and Environmental Health Japan36, University of Porto37, Leeds Teaching Hospitals NHS Trust38, University of Leeds39, Cedars-Sinai Medical Center40, Istanbul Bilim University41, McMaster University42, University of Toronto43, University of Paris44
TL;DR: These new classification criteria for systemic lupus erythematosus have excellent sensitivity and specificity, and were developed using rigorous methodology with multidisciplinary and international input.
Abstract: Objective To develop new classification criteria for systemic lupus erythematosus (SLE) jointly supported by the European League Against Rheumatism (EULAR) and the American College of Rheumatology (ACR). Methods This international initiative had four phases. (1) Evaluation of antinuclear antibody (ANA) as an entry criterion through systematic review and meta-regression of the literature and criteria generation through an international Delphi exercise, an early patient cohort and a patient survey. (2) Criteria reduction by Delphi and nominal group technique exercises. (3) Criteria definition and weighting based on criterion performance and on results of a multi-criteria decision analysis. (4) Refinement of weights and threshold scores in a new derivation cohort of 1001 subjects and validation compared with previous criteria in a new validation cohort of 1270 subjects. Results The 2019 EULAR/ACR classification criteria for SLE include positive ANA at least once as obligatory entry criterion; followed by additive weighted criteria grouped in seven clinical (constitutional, haematological, neuropsychiatric, mucocutaneous, serosal, musculoskeletal, renal) and three immunological (antiphospholipid antibodies, complement proteins, SLE-specific antibodies) domains, and weighted from 2 to 10. Patients accumulating ≥10 points are classified. In the validation cohort, the new criteria had a sensitivity of 96.1% and specificity of 93.4%, compared with 82.8% sensitivity and 93.4% specificity of the ACR 1997 and 96.7% sensitivity and 83.7% specificity of the Systemic Lupus International Collaborating Clinics 2012 criteria. Conclusion These new classification criteria were developed using rigorous methodology with multidisciplinary and international input, and have excellent sensitivity and specificity. Use of ANA entry criterion, hierarchically clustered and weighted criteria reflect current thinking about SLE and provide an improved foundation for SLE research.
606 citations
TL;DR: Substantial variation was found concerning the quality of the study conduct and the transparency of reporting of Delphi studies used for the development of best practice guidance in palliative care.
Abstract: BACKGROUND: The Delphi technique is widely used for the development of guidance in palliative care, having impact on decisions with relevance for patient care. AIM: To systematically examine the application of the Delphi technique for the development of best practice guidelines in palliative care. DESIGN: A methodological systematic review was undertaken using the databases PubMed, CINAHL, Web of Science, Academic Search Complete and EMBASE. DATA SOURCES: Original articles (English language) were included when reporting on empirical studies that had used the Delphi technique to develop guidance for good clinical practice in palliative care. Data extraction included a quality appraisal on the rigour in conduct of the studies and the quality of reporting. RESULTS: A total of 30 empirical studies (1997-2015) were considered for full-text analysis. Considerable differences were identified regarding the rigour of the design and the reporting of essential process and outcome parameters. Furthermore, discrepancies regarding the use of terms for describing the method were observed, for example, concerning the understanding of a 'round' or a 'modified Delphi study'. CONCLUSION: Substantial variation was found concerning the quality of the study conduct and the transparency of reporting of Delphi studies used for the development of best practice guidance in palliative care. Since credibility of the resulting recommendations depends on the rigorous use of the Delphi technique, there is a need for consistency and quality both in the conduct and reporting of studies. To allow a critical appraisal of the methodology and the resulting guidance, a reporting standard for Conducting and REporting of DElphi Studies (CREDES) is proposed.
605 citations
Cites background or methods from "Methods of Formal Consensus in Clas..."
...In the following, the main findings will be summarised and related to previous treatises on the Delphi technique, with a focus on (1) the rigour of the design and conduct of the analysed studies, (2) the quality of reporting and (3) the dissemination politics for the resulting guidance....
[...]
...Data extraction included details on (1) inclusion and exclusion criteria of the article, (2) the focus of the study, (3) the rationale for the use of the Delphi technique, (4) the overall study design, (5) the applied methods and the procedure, (6) data analysis and (7) key outcomes of the consensus process....
[...]
...694 Palliative Medicine 31(8) particular setting or work field (n = 23) and (3) relevant clinical and/or academic expertise (n = 20; Table 5)....
[...]
...4,5 It is characterised by four methodological features which enable the involvement of experts with diverse backgrounds irrespective of their geographical location:4,6–8 (1) a group of experts, called ‘panellists’, is questioned about the issue of interest; (2) the process is anonymous in order to avoid social pressure and conformity to a dominant view (bandwagon effect); (3) the procedure is iterative in nature, comprising several rounds of enquiry; and (4) the design of subsequent rounds is informed by a summary of the group response of the previous round....
[...]
TL;DR: In this paper, a review of physical rehabilitation approaches for stroke patients is presented, where the authors explore the effects of approaches that incorporate individual treatment components, categorised as functional task training, musculoskeletal intervention (active), MCI, neurophysiological intervention (passive), cardiopulmonary intervention, assistive device or modality.
Abstract: Background
Various approaches to physical rehabilitation may be used after stroke, and considerable controversy and debate surround the effectiveness of relative approaches. Some physiotherapists base their treatments on a single approach; others use a mixture of components from several different approaches.
Objectives
To determine whether physical rehabilitation approaches are effective in recovery of function and mobility in people with stroke, and to assess if any one physical rehabilitation approach is more effective than any other approach.
For the previous versions of this review, the objective was to explore the effect of 'physiotherapy treatment approaches' based on historical classifications of orthopaedic, neurophysiological or motor learning principles, or on a mixture of these treatment principles. For this update of the review, the objective was to explore the effects of approaches that incorporate individual treatment components, categorised as functional task training, musculoskeletal intervention (active), musculoskeletal intervention (passive), neurophysiological intervention, cardiopulmonary intervention, assistive device or modality.
In addition, we sought to explore the impact of time after stroke, geographical location of the study, dose of the intervention, provider of the intervention and treatment components included within an intervention.
Search methods
We searched the Cochrane Stroke Group Trials Register (last searched December 2012), the Cochrane Central Register of Controlled Trials (CENTRAL) (The Cochrane Library Issue 12, 2012), MEDLINE (1966 to December 2012), EMBASE (1980 to December 2012), AMED (1985 to December 2012) and CINAHL (1982 to December 2012). We searched reference lists and contacted experts and researchers who have an interest in stroke rehabilitation.
Selection criteria
Randomised controlled trials (RCTs) of physical rehabilitation approaches aimed at promoting the recovery of function or mobility in adult participants with a clinical diagnosis of stroke. Outcomes included measures of independence in activities of daily living (ADL), motor function, balance, gait velocity and length of stay. We included trials comparing physical rehabilitation approaches versus no treatment, usual care or attention control and those comparing different physical rehabilitation approaches.
Data collection and analysis
Two review authors independently categorised identified trials according to the selection criteria, documented their methodological quality and extracted the data.
Main results
We included a total of 96 studies (10,401 participants) in this review. More than half of the studies (50/96) were carried out in China. Generally the studies were heterogeneous, and many were poorly reported.
Physical rehabilitation was found to have a beneficial effect, as compared with no treatment, on functional recovery after stroke (27 studies, 3423 participants; standardised mean difference (SMD) 0.78, 95% confidence interval (CI) 0.58 to 0.97, for Independence in ADL scales), and this effect was noted to persist beyond the length of the intervention period (nine studies, 540 participants; SMD 0.58, 95% CI 0.11 to 1.04). Subgroup analysis revealed a significant difference based on dose of intervention (P value < 0.0001, for independence in ADL), indicating that a dose of 30 to 60 minutes per day delivered five to seven days per week is effective. This evidence principally arises from studies carried out in China. Subgroup analyses also suggest significant benefit associated with a shorter time since stroke (P value 0.003, for independence in ADL).
We found physical rehabilitation to be more effective than usual care or attention control in improving motor function (12 studies, 887 participants; SMD 0.37, 95% CI 0.20 to 0.55), balance (five studies, 246 participants; SMD 0.31, 95% CI 0.05 to 0.56) and gait velocity (14 studies, 1126 participants; SMD 0.46, 95% CI 0.32 to 0.60). Subgroup analysis demonstrated a significant difference based on dose of intervention (P value 0.02 for motor function), indicating that a dose of 30 to 60 minutes delivered five to seven days a week provides significant benefit. Subgroup analyses also suggest significant benefit associated with a shorter time since stroke (P value 0.05, for independence in ADL).
No one physical rehabilitation approach was more (or less) effective than any other approach in improving independence in ADL (eight studies, 491 participants; test for subgroup differences: P value 0.71) or motor function (nine studies, 546 participants; test for subgroup differences: P value 0.41). These findings are supported by subgroup analyses carried out for comparisons of intervention versus no treatment or usual care, which identified no significant effects of different treatment components or categories of interventions.
Authors' conclusions
Physical rehabilitation, comprising a selection of components from different approaches, is effective for recovery of function and mobility after stroke. Evidence related to dose of physical therapy is limited by substantial heterogeneity and does not support robust conclusions. No one approach to physical rehabilitation is any more (or less) effective in promoting recovery of function and mobility after stroke. Therefore, evidence indicates that physical rehabilitation should not be limited to compartmentalised, named approaches, but rather should comprise clearly defined, well-described, evidenced-based physical treatments, regardless of historical or philosophical origin.
389 citations
References
More filters
TL;DR: The advantages of the GRADE system are explored, which is increasingly being adopted by organisations worldwide and which is often praised for its high level of consistency.
Abstract: Guidelines are inconsistent in how they rate the quality of evidence and the strength of recommendations. This article explores the advantages of the GRADE system, which is increasingly being adopted by organisations worldwide
13,324 citations
Book•
01 Jan 1975
TL;DR: The present model clarifies some of the conceptual problems associated with cross-impact analysis, and supplies a relatively sound basis for revising probability estimates in the limited case where interactions can be approximated by relative probabilities.
Abstract: Cross-impact analysis is a method for revising estimated probabilities of future events in terms of estimated interactions among those events. This Report presents an elementary cross-impact model where the cross-impacts are formulated as relative probabilities. Conditions are derived for the consistency of the matrix of relative probabilities of n events. An extension also provides a necessary condition for the vector of absolute probabilities to be consistent with the relative probability matrix. An averaging technique is formulated for resolving inconsistencies in the matrix, and a nearest-point computation derived for resolving inconsistencies between the set of absolute probabilities and the matrix. Although elementary, the present model clarifies some of the conceptual problems associated with cross-impact analysis, and supplies a relatively sound basis for revising probability estimates in the limited case where interactions can be approximated by relative probabilities.
5,102 citations
TL;DR: The DELPHI method was devised in order to obtain the most reliable opinion consensus of a group of experts by subjecting them to a series of questionnaires in depth interspersed with controlled opinion feedback as mentioned in this paper.
Abstract: This paper gives an account of an experiment in the use of the so-called DELPHI method, which was devised in order to obtain the most reliable opinion consensus of a group of experts by subjecting them to a series of questionnaires in depth interspersed with controlled opinion feedback.
4,856 citations
TL;DR: Though prevalent in white postmenopausal women, osteoporosis occurs in all populations and at all ages and has significant physical, psychosocial, and financial consequences.
Abstract: OBJECTIVES
To clarify the factors associated with prevention, diagnosis, and treatment of osteoporosis, and to present the most recent information available in these areas.
PARTICIPANTS
From March 27-29, 2000, a nonfederal, nonadvocate, 13-member panel was convened, representing the fields of internal medicine, family and community medicine, endocrinology, epidemiology, orthopedic surgery, gerontology, rheumatology, obstetrics and gynecology, preventive medicine, and cell biology. Thirty-two experts from these fields presented data to the panel and an audience of 699. Primary sponsors were the National Institute of Arthritis and Musculoskeletal and Skin Diseases and the National Institutes of Health Office of Medical Applications of Research.
EVIDENCE
MEDLINE was searched for January 1995 through December 1999, and a bibliography of 2449 references provided to the panel. Experts prepared abstracts for presentations with relevant literature citations. Scientific evidence was given precedence over anecdotal experience.
CONSENSUS PROCESS
The panel, answering predefined questions, developed conclusions based on evidence presented in open forum and the literature. The panel composed a draft statement, which was read and circulated to the experts and the audience for public discussion. The panel resolved conflicts and released a revised statement at the end of the conference. The draft statement was posted on the Web on March 30, 2000, and updated with the panel's final revisions within a few weeks.
CONCLUSIONS
Though prevalent in white postmenopausal women, osteoporosis occurs in all populations and at all ages and has significant physical, psychosocial, and financial consequences. Risks for osteoporosis (reflected by low bone mineral density [BMD]) and for fracture overlap but are not identical. More attention should be paid to skeletal health in persons with conditions associated with secondary osteoporosis. Clinical risk factors have an important but poorly validated role in determining who should have BMD measurement, in assessing fracture risk, and in determining who should be treated. Adequate calcium and vitamin D intake is crucial to develop optimal peak bone mass and to preserve bone mass throughout life. Supplementation with these 2 nutrients may be necessary in persons not achieving recommended dietary intake. Gonadal steroids are important determinants of peak and lifetime bone mass in men, women, and children. Regular exercise, especially resistance and high-impact activities, contributes to development of high peak bone mass and may reduce risk of falls in older persons. Assessment of bone mass, identification of fracture risk, and determination of who should be treated are the optimal goals when evaluating patients for osteoporosis. Fracture prevention is the primary treatment goal for patients with osteoporosis. Several treatments have been shown to reduce the risk of osteoporotic fractures, including those that enhance bone mass and reduce the risk or consequences of falls. Adults with vertebral, rib, hip, or distal forearm fractures should be evaluated for osteoporosis and given appropriate therapy.
4,623 citations
TL;DR: Two consensus methods commonly adopted in medical, nursing, and health services research--the Delphi process and the nominal group technique (also known as the expert panel)--are described, together with the most appropriate situations for using them.
Abstract: Health providers face the problem of trying to make decisions in situations where there is insufficient information and also where there is an overload of (often contradictory) information. Statistical methods such as meta-analysis have been developed to summarise and to resolve inconsistencies in study findings—where information is available in an appropriate form. Consensus methods provide another means of synthesising information, but are liable to use a wider range of information than is common in statistical methods, and where published information is inadequate or non-existent these methods provide a means of harnessing the insights of appropriate experts to enable decisions to be made. Two consensus methods commonly adopted in medical, nursing, and health services research—the Delphi process and the nominal group technique (also known as the expert panel)—are described, together with the most appropriate situations for using them; an outline of the process involved in undertaking a study using each method is supplemented by illustrations of the authors' work. Key methodological issues in using the methods are discussed, along with the distinct contribution of consensus methods as aids to decision making, both in clinical practice and in health service development.
This is the sixth in a series of seven articles describing non-quantitative techniquesand showing their value in health research
Quantitative methods such as meta-analysis have been developed to provide statistical overviews of the results of clinical trials and to resolve inconsistencies in the results of published studies. Consensus methods are another means of dealing with conflicting scientific evidence. They allow a wider range of study types to be considered than is usual in statistical reviews. In addition they allow a greater role for the qualitative assessment of evidence (box 1). These methods, unlike those described in the other papers in this series, are primarily concerned with deriving quantitative estimates through qualitative …
2,961 citations