Bivariate analysis of sensitivity and specificity produces informative summary measures in diagnostic reviews.

Objective: To improve the accuracy and completeness of reporting of studies of diagnostic accuracy, to allow readers to assess the potential for bias in a study, and to evaluate a study9s generalisability. Methods: The Standards for Reporting of Diagnostic Accuracy (STARD) steering committee searched the literature to identify publications on the appropriate conduct and reporting of diagnostic studies and extracted potential items into an extensive list. Researchers, editors, and members of professional organisations shortened this list during a two day consensus meeting, with the goal of developing a checklist and a generic flow diagram for studies of diagnostic accuracy. Results: The search for published guidelines about diagnostic research yielded 33 previously published checklists, from which we extracted a list of 75 potential items. At the consensus meeting, participants shortened the list to a 25 item checklist, by using evidence, whenever available. A prototype of a flow diagram provides information about the method of patient recruitment, the order of test execution, and the numbers of patients undergoing the test under evaluation and the reference standard, or both. Conclusions: Evaluation of research depends on complete and accurate reporting. If medical journals adopt the STARD checklist and flow diagram, the quality of reporting of studies of diagnostic accuracy should improve to the advantage of clinicians, researchers, reviewers, journals, and the public. The Standards for Reporting of Diagnostic Accuracy (STARD) steering group aims to improve the accuracy and completeness of reporting of studies of diagnostic accuracy. The group describes and explains the development of a checklist and flow diagram for authors of reports

https://www.bmj.com/content/bmj/326/7379/41.1.full.pdf

Towards complete and accurate reporting of studies of diagnostic accuracy: the STARD initiative

ContextThe literature contains a large number of potential biases in the evaluation
of diagnostic tests. Strict application of appropriate methodological criteria
would invalidate the clinical application of most study results.ObjectiveTo empirically determine the quantitative effect of study design shortcomings
on estimates of diagnostic accuracy.Design and SettingObservational study of the methodological features of 184 original studies
evaluating 218 diagnostic tests. Meta-analyses on diagnostic tests were identified
through a systematic search of the literature using MEDLINE, EMBASE, and DARE
databases and the Cochrane Library (1996-1997). Associations between study
characteristics and estimates of diagnostic accuracy were evaluated with a
regression model.Main Outcome MeasuresRelative diagnostic odds ratio (RDOR), which compared the diagnostic
odds ratios of studies of a given test that lacked a particular methodological
feature with those without the corresponding shortcomings in design.ResultsFifteen (6.8%) of 218 evaluations met all 8 criteria; 64 (30%) met 6
or more. Studies evaluating tests in a diseased population and a separate
control group overestimated the diagnostic performance compared with studies
that used a clinical population (RDOR, 3.0; 95% confidence interval [CI],
2.0-4.5). Studies in which different reference tests were used for positive
and negative results of the test under study overestimated the diagnostic
performance compared with studies using a single reference test for all patients
(RDOR, 2.2; 95% CI, 1.5-3.3). Diagnostic performance was also overestimated
when the reference test was interpreted with knowledge of the test result
(RDOR, 1.3; 95% CI, 1.0-1.9), when no criteria for the test were described
(RDOR, 1.7; 95% CI, 1.1-2.5), and when no description of the population under
study was provided (RDOR, 1.4; 95% CI, 1.1-1.7).ConclusionThese data provide empirical evidence that diagnostic studies with methodological
shortcomings may overestimate the accuracy of a diagnostic test, particularly
those including nonrepresentative patients or applying different reference
standards.

https://jamanetwork.com/journals/jama/articlepdf/191668/joc90732.pdf

Empirical evidence of design-related bias in studies of diagnostic tests

Summary Background Disease-modifying treatment strategies for Alzheimer's disease have led to an urgent need for biomarkers to identify the disease at a very early stage. Here, we assess the association between CSF biomarkers and incipient Alzheimer's in patients with mild cognitive impairment (MCI). Methods From a series of 180 consecutive patients with MCI, we assessed 137 who underwent successful lumbar puncture at baseline. Patients at risk of developing dementia were followed clinically for 4–6 years. Additionally, 39 healthy individuals, cognitively stable over 3 years, served as controls. We analysed CSF concentrations of β amyloid 1–42 (Aβ42), total tau (T-tau), and phosphorylated tau (P-tau 181 ) using Luminex xMAP technology. Findings During follow-up, 57 (42%) patients with MCI developed Alzheimer's disease, 21 (15%) developed other forms of dementia, and 56 (41%) remained cognitively stable for 5·2 years (range 4·0–6·8). A combination of CSF T-tau and Aβ42 at baseline yielded a sensitivity of 95% and a specificity of 83% for detection of incipient AD in patients with MCI. The relative risk of progression to Alzheimer's disease was substantially increased in patients with MCI who had pathological concentrations of T-tau and Aβ42 at baseline (hazard ratio 17·7, p APOE genotype, and plasma homocysteine. The combination of T-tau and Aβ42/P-tau 181 ratio yielded closely similar results (sensitivity 95%, specificity 87%, hazard ratio 19·8). Interpretation Concentrations of T-tau, P-tau 181 , and Aβ42 in CSF are strongly associated with future development of Alzheimer's disease in patients with MCI.

/pdf/association-between-csf-biomarkers-and-incipient-alzheimer-s-2dcjb7b222.pdf

Association between CSF biomarkers and incipient Alzheimer's disease in patients with mild cognitive impairment: a follow-up study

The quality of reporting of studies of diagnostic accuracy is less than optimal. Complete and accurate reporting is necessary to enable readers to assess the potential for bias in the study and to evaluate the generalizability of the results. A group of scientists and editors has developed the STARD (Standards for Reporting of Diagnostic Accuracy) statement to improve the reporting the quality of reporting of studies of diagnostic accuracy. The statement consists of a checklist of 25 items and flow diagram that authors can use to ensure that all relevant information is present. This explanatory document aims to facilitate the use, understanding, and dissemination of the checklist. The document contains a clarification of the meaning, rationale, and optimal use of each item on the checklist, as well as a short summary of the available evidence on bias and applicability. The STARD statement, checklist, flowchart, and this explanation and elaboration document should be useful resources to improve reporting of diagnostic accuracy studies. Complete and informative reporting can only lead to better decisions in health care.

The STARD Statement for Reporting Studies of Diagnostic Accuracy: Explanation and Elaboration

Objective. —To determine the frequency and temporal changes in application of seven accepted methodological standards for the evaluation of diagnostic tests. Data Sources. —A search of the MEDLINE database yielded 1302 articles about diagnostic test studies, during a 16-year secular interval, 1978 through 1993, in four prominent general medical journals. Study Selection. —In the 112 eligible studies, the test was intended for clinical use, indexes of accuracy (sensitivity and specificity or likelihood ratios) were provided, and more than 10 patients were enrolled. Data Extraction. —Although each study was critically reviewed by one primary observer, a subset was independently evaluated for interrater consistency. Data Synthesis. —The percentage of studies that fulfilled criteria for each of the seven methodological standards are as follows: (1) specify spectrum of evaluated patients, 27%; (2) report test indexes for clinical subgroups, 8%; (3) avoid workup bias, 46%; (4) avoid review bias, 38%; (5) provide numerical precision for test indexes, 11%; (6) report frequency and management of indeterminate results when calculating test indexes, 22%; and (7) specify test reproducibility, 23%. Secular increases were found for six of the seven standards in ranges of use from 14% to 31% during 1978-1981 to 1990-1993. Nevertheless, only one standard, avoidance of workup bias, was fulfilled by more than 50% of studies in the most recent secular interval. Conclusions. —These results indicate that most diagnostic tests are still inadequately appraised. The routine demand for methodological standards could raise the quality of diagnostic test information, and the careful predissemination evaluation of diagnostic tests could eliminate useless tests before they receive widespread application. (JAMA. 1995;274:645-651)

https://jamanetwork.com/HttpHandlers/ArticlePdfHandler.ashx?journal=JAMA&articleId=389496&pdfFileName=jama_274_8_042.pdf

Use of methodological standards in diagnostic test research. Getting better but still not good

The field of rheumatology has shown great interest in comorbidity-related risk among patients with rheumatoid arthritis (RA). A recent paper reported that a person with RA has a similar risk of sustaining a myocardial infarction as a person with diabetes mellitus (DM), and this risk is comparable to that of a healthy person 10 years older.1 Several other papers have also recently reported that people with RA and DM share a similar cardiovascular risk.2 3 These findings are strikingly similar to those of the Charlson Index published 25 years ago.4

The Charlson Comorbidity Index is the most commonly used prognostic measure of illness burden in contemporary clinical research.5 Cited in nearly 7000 studies, it is considered the gold standard to assess comorbid risk in clinical research.6 Over time the Charlson Index has increased in use and relevance, likely related to increased rates of chronic disease, supporting the need to measure and adjust for these conditions in research.

The original objective was to develop a prognostic taxonomy for comorbid conditions that could singly or in combination prognosticate …

/pdf/the-risk-of-comorbidity-17m9zv8lyu.pdf

M C Reid

Papers

Use of methodological standards in diagnostic test research. Getting better but still not good

The Risk of Comorbidity