The accuracy of novel antigen rapid diagnostics for SARS-CoV-2: a living systematic review and meta-analysis.

TLDR: In this article, a systematic review and meta-analysis of commercially available rapid diagnostic tests (Ag-RDTs) is presented, where the clinical accuracy (sensitivity and specificity) of these tests are assessed.

Abstract: Background SARS-CoV-2 antigen rapid diagnostic tests (Ag-RDTs) are increasingly being integrated in testing strategies around the world. Studies of the Ag-RDTs have shown variable performance. In this systematic review and meta-analysis, we assessed the clinical accuracy (sensitivity and specificity) of commercially available Ag-RDTs. Methods We registered the review on PROSPERO (Registration number: CRD42020225140). We systematically searched multiple databases (PubMed, Web of Science Core Collection, medRvix and bioRvix, FINDdx) for publications up until December 11th, 2020. Descriptive analyses of all studies were performed and when more than four studies were available, a random-effects meta-analysis was used to estimate pooled sensitivity and specificity in comparison to reverse transcriptase polymerase chain reaction testing. We assessed heterogeneity by subgroup analyses ((1) performed con-form with manufacturer’s instructions for use (IFU) or not, (2) symptomatic vs. asymptomatic, (3) duration of symptoms less than seven days vs. more than seven days, (4) Ct-value Results From a total of 11,715 articles, we extracted 98 analytical and clinical data sets. 74 clinical accuracy data sets were evaluated that included 31,202 samples. Across all meta-analyzed samples, the pooled Ag-RDT sensitivity was 73.8% (CI 68.6 to 78.5). If analysis was restricted to studies that followed the Ag-RDT manufacturers’ instructions using fresh upper respiratory swab samples, the sensitivity increased to 79.1% (95%CI 75.0 to 82.8). The SD Biosensor Standard Q and Abbott Panbio showed the highest sensitivity with 81.7% and 72.7%, respectively. The best Ag-RDT performance was found with nasopharyngeal sampling (77.3%, CI 72.0 to 81.9) in comparison to other sample types (e.g., anterior nasal or mid turbinate 63.5%, CI 49.5 to 75.5). Testing in the first week from symptom onset resulted in higher sensitivity (87.5%, CI 86.0 to 89.1) compared to testing after one week (64.1%, CI 54.4 to 73.8). The tests performed markedly better on samples with lower Ct-values, i.e., Conclusion As Ag-RDTs detect most cases within the first week of symptom onset and those with high viral load, they can have high utility for screening purposes in the early phase of disease, and thus can be a valuable tool to fight the spread of SARS-CoV-2. Standardization of conduct and reporting of clinical accuracy studies would improve comparability and use of data. Summary In this living systematic review we analyzed 98 data sets for performance of SARS-CoV-2 Ag-RDTs compared to RT-PCR. Best-performing tests achieved a sensitivity of 81.7%. Highest sensitivity was found in patients within seven days of symptom onset when NP swabs were utilized.

Full text

1
The accuracy of novel antigen rapid diagnostics for SARS-CoV-2: a living systematic review
1
and meta-analysis.
2
3
Lukas E. Brümmer
1
*, Stephan Katzenschlager
2
*, Mary Gaeddert
1
, Christian Erdmann
3
, Stephani 4
Schmitz
1
, Marc Bota
4
, Maurizio Grilli
5
, Jan Larmann
2
, Markus A. Weigand
2
, Nira R. Pollock
6
, Aurélien 5
Macé
7
, Sergio Carmona
7
, Stefano Ongarello
7
, Jilian A. Sacks
7
, Claudia M. Denkinger
1, 8
6
7
* These authors contributed equally 8
9
1) Division of Tropical Medicine, Center for Infectious Diseases, Heidelberg University Hospital, Hei-10
delberg, Germany 11
2) Departme nt of Anesthe si ology, Heidel berg University Hospi tal, Heid elb erg, Ger many 12
3) FH Muenster University of Applied Sciences, Muenster, Germany 13
4) Agaplesion Bethesda Hospital, Hamburg, Germany 14
5) Library, University Medical Center Mannheim, Mannheim, Germany 15
6) Department of Laboratory Medicine, Boston Children’s Hospital, Boston, MA USA 16
7) FIND, Geneva, Switzerland 17
8) German Center for Infection Research (DZIF), partner site Heidelberg University Hospital, Heidel-18
berg, Germany 19
20
21
22
Keywords: SARS-CoV-2; Antigen Test; PCR; meta-analys is; 23
Running Title: Meta-Analysis: Rapid antigen tests for SARS-CoV-2 24
25
Corresponding author:
26
Claudia M. Denkinger 27
Division of Tropical Medicine, Center for Infectious Diseases, Heidelberg University Hospital, Heidel-28
berg, Germany 29
Claudia.Denkinger@uni-heidelberg.de
30
+49 6221 56 36637 31
32
. CC-BY-ND 4.0 International licenseIt is made available under a
is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review)
The copyright holder for this preprint this version posted June 19, 2021. ; https://doi.org/10.1101/2021.02.26.21252546doi: medRxiv preprint
NOTE: This preprint reports new research that has not been certified by peer review and should not be used to guide clinical practice.

2
ABSTRACT
33
34
Background:
SARS- Co V-2 antigen rapid dia gnos tic te st s (Ag-RDTs) are increasing ly being inte-35
grated in testing strategies around the world. Studies of the Ag-RDTs have shown variable perfor-36
mance. In this systematic review and meta-analysis, we assessed the clinical accuracy (sensitivity and 37
specificity) of commercially available Ag-RDTs.
38
Methods and Results:
We registere d the r ev iew on P ROSP ER O ( Regi stration number: 39
CRD420202251 40 ). We systema tica lly searched multiple d ata ba se s (Pub Med , Web o f Sc ience Co r e 40
Collection, medRvix and bioRvix, FIND) for publications evaluating the accuracy of Ag-RDTs for SARS-41
CoV-2 up until April 30
th
, 2021. Descriptive analyses of all studies were performed and when more 42
than four studies were available, a random-effects meta-analysis was used to estimate pooled sensi-43
tivity and specificity in comparison to reverse transcriptase polymerase chain reaction testing. We 44
assessed heterogeneity by subgroup analyses, and rated study quality and risk of bias using the 45
QUADAS 2 a s sessment tool. From a total o f 14 ,254 articl e s, we included 13 3 analy tical and c linic a l 46
studies resulting in 214 clinical accuracy data sets with 112,323 samples. Across all meta-analyzed 47
sampl es, the po oled Ag-RDT sensitivity was 71.2% (95% confide nce inte rval [ C I] 68.2 t o 74.0) an d 48
increased to 76.3% (CI 73.1 to 79.2) if analysis was restricted to studies that followed the Ag-RDT 49
manufacturers’ instructions. The LumiraDx showed the highest sensitivity with 88.2% (CI 59.0 to 50
97.5). Of instrumen t- free Ag-RDTs, Standa rd Q nasal perf ormed bes t with 80.2% sensi tivity (CI 70.3 t o 51
87.4). Across all Ag-RDTs sensitivity was markedly better on samples with lower Ct-values, i.e., <20 52
(96.5%, CI 92.6 to 98.4) and <25 (95.8%, CI 92.3 to 97.8), in comparison to those with Ct ≥25 (50.7%, 53
CI 35.6 to 65.8) and ≥30 (20.9%, CI 12.5 to 32.8). Testing in the first week from symptom onset re-54
sulted in substantially higher sensitivity (83.8%, CI 76.3 to 89.2) compared to testing after one week 55
(61.5%, CI 52.2 to 70.0). The best Ag-RDT sensitivity was found with anterior nasal sampling (75.5%, 56
CI 70.4 to 79.9) in comparison to other sample types (e.g., nasopharyngeal 71.6%, CI 68.1 to 74.9) 57
although CIs were overlapping. Concerns of bias were raised across all data sets, and financial sup-58
port from the manu fac turer wa s repor t ed in 24 .1% of d at a sets. Our analy sis was l imited by the in-59
cluded studies’ heterogeneity in design and reporting, making it difficult to draw conclusions from.
60
Conclusion:
In this study we found that Ag-RDTs detect the vast majority of cases within the first 61
week of symptom onset and those with high viral load. Thus, they can have high utility for diagnostic 62
purposes in the early phase of disease, making them a valuable tool to fight the spread of SARS-CoV-63
2. Standardization in conduct and reporting of clinical accuracy studies would improve comparability 64
and use of data. 65
. CC-BY-ND 4.0 International licenseIt is made available under a
is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review)
The copyright holder for this preprint this version posted June 19, 2021. ; https://doi.org/10.1101/2021.02.26.21252546doi: medRxiv preprint

3
AUTHOR SUMMARY
66
67
Why was this study done? 68
-
Antigen rapid diagnostic tests (Ag-RDTs) are considered an important diagnostic tool to fight 69
the spread of SARS-CoV-2 70
-
An increasing number of Ag-RDTs is offered on the market, and a constantly growing body 71
of literature evaluating their performance is available 72
-
To inform decision makers about the best test to choose, an up to date summar y of their 73
performance is needed 74
75
What d id the rese ar chers d o an d fi nd ? 76
-
On a weekly basis, we search multiple data bases for evaluations of Ag-RDTs detecting 77
SARS-CoV-2 and post the results on www.diagnosticsglobalhealth.org
78
-
Based on the search results up until April 30
th
, 2021, we conducted a systematic review and 79
meta-analysis, including a total of 133 clinical and analytical accuracy studies 80
-
Across all meta-analyzed studies, when Ag-RDTs were performed according to manufactur-81
ers’ recommendations, they s howe d a sensitivi ty of 76.3% (CI 73. 1 t o 79.2) , with the 82
LumiraDx (sensitivity 88.2%, CI 59.0 to 97.5) and of the instrument-free Ag-RDT Standard Q 83
(74.9% sensitivity, CI 69.3 to 79.7) performing best. 84
-
Across all Ag-RDTs, sensitivity increased to 95.8% (CI 92.3 to 97.8) when restricting the anal-85
ysis to samples with high vir al loads (i.e., a Ct-value <25) and to 83.8% (CI 76.3 to 89.2) when 86
tests were perfo rmed on patien t s within the fir st week afte r symptom on set 87
88
What do these findings mean? 89
-
Ag-RDTs detect the vast majority of cases within the first week of symptom onset and those 90
with high viral load. Thus, they can have high utilit y for diagnostic purposes in the early 91
phase of disease 92
-
Out of all assessed tests, the Lumira Dx showed the highest accuracy. The Standard Q was 93
the best performing test when only considering those that don’t require an instrument 94
-
A standardization of reporting methods for clinical accuracy studies would enhance future 95
test-comparisons 96
. CC-BY-ND 4.0 International licenseIt is made available under a
is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review)
The copyright holder for this preprint this version posted June 19, 2021. ; https://doi.org/10.1101/2021.02.26.21252546doi: medRxiv preprint

4
ABBREVIATIONS
97
Ag-RDT = antigen rapid diagnostic test 98
AN/MT = anterior nasal or mid-turbinate 99
BAL/TW = bronchoalveolar lavage or throat wash 100
CI = confid enc e interval 101
Ct-value = cycle threshold value 102
FIND = Foundation for Innovative New Diagnostics 103
FP = false positive 104
FN = false negative 105
IFU = instructions for use 106
LRT = lower respiratory tract 107
ML = milliliter 108
N = sample size 109
NP = nasopharyngeal 110
OP = oropharyngeal 111
PFU = plaque forming units 112
RT-PCR = reverse transcriptase polymerase chain reaction 113
TP = true positive 114
TN = true negative 115
VTM/UTM = viral or universal transport medium 116
. CC-BY-ND 4.0 International licenseIt is made available under a
is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review)
The copyright holder for this preprint this version posted June 19, 2021. ; https://doi.org/10.1101/2021.02.26.21252546doi: medRxiv preprint

5
INTRODUCTION
117
As the COVID-19 pandemic continues around the globe, antigen rapid diagnostic tests (Ag-RDTs) 118
for SARS-CoV-2 are seen as an important diagnostic tool to fight the virus’ spread [1,2]. The number 119
of Ag-RDTs on the ma rk et i s i nc rea sing constantly [3 ]. Initi al data from ind epen d e nt e valua tio ns sug-120
gests that the performance of SARS-CoV-2 Ag-RDTs may be lower than what is reported by the manu-121
facturers. In addition, Ag-RDT accuracy seems to vary substantially between tests [4-6]. 122
With the increased availability of Ag-RDTs, an increasing number of independent validations 123
have been published. Such evaluations differ widely in their quality, methods and results, making it 124
difficult to assess the tr ue performance of the respective tests [7]. To inform decision makers on the 125
best choice of individual tests, an aggregated, widely available and frequently updated assessment of 126
the quality, performance and independence of the data i s urgen tly nec es sary . Whi le other sys temat ic 127
reviews hav e been publi shed , they only includ e data up until Nov 2020 [8-11 ], ex clude preprin ts [12], 128
or were industry sponsored [13]. In addition, only one assessed the quality of studies in detail, with 129
data up until Nov, 2020 [7,11]. 130
With ou r systematic review and meta-ana lysis , we aim to close this gap in the literat ure and link 131
to a website (www.diagnosticsglobalhealth.org) that is regularly updated.
132
. CC-BY-ND 4.0 International licenseIt is made available under a
is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review)
The copyright holder for this preprint this version posted June 19, 2021. ; https://doi.org/10.1101/2021.02.26.21252546doi: medRxiv preprint

Frequently asked questions

Q1. What contributions have the authors mentioned in the paper "The accuracy of novel antigen rapid diagnostics for sars-cov-2: a living systematic review" ?

The accuracy of novel antigen rapid diagnostics for SARS-CoV-2: a living systematic review 1 and meta-analysis this paper. 

Q2. What type of sensitivity analysis was planned?

Two types of sensitivity analyses were planned: an estimation of sensitivity and specificity ex-247cluding case-control studies, and estimation of sensitivity and specificity excluding non-peer-248reviewed studies. 

Q3. What is the p-value for the slope coefficient?

The authors also performed the Deeks’ test for funnel-plot asymmetry as recom-241mended to investigate publication bias for diagnostic test accuracy meta-analyses ([18], using the 242‘midas’ command in Stata version 15); a p-value<Z0.10 for the slope coefficient indicates significant 243asymmetry. 

Q4. What was the meta-analysis of the SARS-CoV-2?

If four or more data sets were available with at least 20 positive RT-PCR samples 217per data set for a predefined analysis, a meta-analysis was performed. 

Q5. how many studies were included in the systematic review?

Out of the 133 studies, nine 261reported analytical accuracy [173,191,198,208,227,256,274,275,282] and the remaining 124 report-262ed clinical accuracy. 

Q6. What is the RT-PCR standard for SARS-CoV-2?

Since RT-PCR tests are more widely available and SARS-CoV-2 RNA (as reflected by RT-PCR 171cycle threshold (Ct) value) highly correlates with SARS-CoV-2 antigen quantities, the authors considered it an 172acceptable reference standard for the purposes of this systematic review [16]. 

Q7. What databases were used to search for SARS-CoV-2?

The authors performed a search of the databases PubMed, Web of Science, medRxiv and bioRxiv using 140search terms that were developed with an experienced medical librarian (MauG) using combinations 141of subject headings (when applicable) and text-words for the concepts of the search question. 

Q8. How many authors extracted data from clinical samples?

; https://doi.org/10.1101/2021.02.26.21252546doi: medRxiv preprint8At first, four authors (SK, CE, SS, MB) extracted five randomly selected papers in parallel to align 184data extraction methods. 

Q9. How many points were relaxed for the Ct-value 230groups?

In an effort to use as much of the heterogeneous data as possible, the cut-offs for the Ct-value 230groups were relaxed by 2-3 points within each range. 

Q10. How many studies were excluded from the systematic review?

Of these, 148 were excluded 255because they did not present primary data [13,19-131] or the Ag-RDT was not commercially available 256[16,132-164], leaving 133 studies to be included in the systematic review (Fig 1) [4,165-296].