Comparative performance of SARS CoV-2 lateral flow antigen tests
1
demonstrates their utility for high sensitivity detection of infectious virus in
2
clinical specimens
3
4
5
Suzanne Pickering
1
, Rahul Batra
2
, Luke B. Snell
2
, Blair Merrick
2
, Gaia Nebbia
2
, Sam
6
Douthwaite
2
, Amita Patel
2
, Mark Tan Kia Ik
2
, Bindi Patel
2
, Themoula Charalampous
2
, Adela
7
Alcolea-Medina
2,3
, Maria Jose Lista
1
, Penelope R. Cliff
3
, Emma Cunningham
3
, Jane Mullen
3
,
8
Katie J. Doores
1
, Jonathan D. Edgeworth
1,2
, Michael H. Malim
1
, Stuart J.D. Neil
1*
, Rui Pedro
9
Galão
1*
10
11
1
Department of Infectious Diseases, School of Immunology & Microbial Sciences, King’s
12
College London, London SE1 9RT, UK
13
14
2
Centre for Clinical Infection and Diagnostics Research, Department of Infectious Diseases,
15
Guy’s and St Thomas’ NHS Foundation Trust, London SE1 7EH
16
17
3
Viapath Group LLP, Guy’s and St Thomas’ NHS Foundation Trust, London, UK
18
19
20
*
Joint corresponding authors
21
Rui Pedro Galão rui_pedro.galao@kcl.ac.uk
22
+44 207 848 2610
23
24
Stuart J.D. Neil stuart.neil@kcl.ac.uk
25
+44 207 848 9617
26
27
Abstract
28
29
Background: Rapid antigen lateral flow devices (LFDs) are set to become a cornerstone of
30
SARS-CoV-2 mass community testing. However, their reduced sensitivity compared to PCR
31
has raised questions of how well they identify infectious cases. Understanding their capabilities
32
and limitations is therefore essential for successful implementation. To address this, we
33
evaluated six commercial LFDs on the same collection of clinical samples and assessed their
34
correlation with infectious virus culture and cycle threshold (Ct) values.
35
36
Methods: A head-to-head comparison of specificities and sensitivities was performed on six
37
commercial rapid antigen tests using combined nasal/oropharyngeal swabs, and their limits of
38
detection determined using viral plaque forming units (PFU). Three of the LFDs were selected
39
for a further study, correlating antigen test result with RT-PCR Ct values and positive viral
40
culture in Vero-E6 cells. This included sequential swabs and matched serum samples obtained
41
from four infected individuals with varying disease severities. Detection of antibodies was
42
performed using an IgG/IgM Rapid Test Cassette, and neutralising antibodies by infectious
43
virus assay. Finally, the sensitivities of selected rapid antigen LFTs were assessed in swabs
44
with confirmed B.1.1.7 variant, currently the dominant genotype in the UK.
45
46
Findings: Most of the rapid antigen LFDs showed a high specificity (>98%), and accurately
47
detected 50 PFU/test (equivalent N1 Ct of 23.7 or RNA copy number of 3x10
6
/ml).
48
. CC-BY-NC-ND 4.0 International licenseIt is made available under a
is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review)
The copyright holder for this preprintthis version posted March 2, 2021. ; https://doi.org/10.1101/2021.02.27.21252427doi: medRxiv preprint
NOTE: This preprint reports new research that has not been certified by peer review and should not be used to guide clinical practice.
Sensitivities of the LFDs performed on clinical samples ranged from 65 to 89%. These
49
sensitivities increased in most tests to over 90% for samples with Cts lower than 25. Positive
50
virus culture was achieved for 57 out of 141 samples, with 80% of the positive cultures from
51
swabs with Cts lower than 23. Importantly, sensitivity of the LFDs increased to over 95% when
52
compared with the detection of infectious virus alone, irrespective of Ct. Longitudinal studies
53
of PCR-positive samples showed that most of the tests identified all infectious samples as
54
positive, but differences in test sensitivities can lead to missed cases in the absence of repeated
55
testing. Finally, test performance was not impacted when re-assessed against swabs positive
56
for the dominant UK variant B.1.1.7.
57
58
Interpretation: In this comprehensive comparison of antigen LFD and virus infectivity, we
59
demonstrate a clear relationship between Ct values, quantitative culture of infectious virus and
60
antigen LFD positivity in clinical samples. Our data support regular testing of target groups
61
using LFDs to supplement the current PCR testing capacity, to rapidly identify infected
62
individuals in situations where they would otherwise go undetected.
63
64
Funding: King’s Together Rapid COVID-19, Medical Research Council, Wellcome Trust,
65
Huo Family Foundation.
66
67
. CC-BY-NC-ND 4.0 International licenseIt is made available under a
is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review)
The copyright holder for this preprintthis version posted March 2, 2021. ; https://doi.org/10.1101/2021.02.27.21252427doi: medRxiv preprint
Introduction
68
69
Covid-19 continues to have a profound impact on global health, with many countries resorting
70
to economically and socially damaging restrictions to minimise the spread of SARS-CoV-2
71
and protect healthcare systems from being overwhelmed.
72
73
Pathways out of national lockdowns - and strategies to mitigate the need for them in the future
74
– depend upon the successful implementation of mass vaccination programmes, effective
75
contact tracing systems and mass community testing. In addition to the existing PCR-based
76
testing systems, the latter may take the form of targeted intensive testing in increasing
77
incidence areas, alongside regular routine screening in healthcare, education, workplace and
78
leisure settings. Realistically, the expansion of mass regular testing relies heavily on an element
79
of low-infrastructure- or self-testing, such as that offered by rapid antigen lateral flow devices
80
(LFDs).
1,2
81
82
Thoroughly understanding the advantages and limitations of rapid antigen LFDs is, therefore,
83
a priority and will help to inform decisions about where these tests will have the most utility
84
and, conversely, where they could be contraindicated. There are concerns about their reduced
85
sensitivity in comparison to PCR, and controversies have arisen over the suitability of their
86
implementation.
3–5
Problems with comparing Ct values from RT-qPCR between different
87
protocols, and even between the same protocols at different locations, combined with
88
uncertainty about the range of viral loads that constitute a transmission risk, have been the root
89
of many of the issues.
5,6
Individuals are most infectious around the time of symptom onset,
90
when viral loads in the upper respiratory tract are highest,
7,8
with recent studies confirming an
91
association between viral load and increased transmission of SARS-CoV-2.
9
For asymptomatic
92
individuals, infectivity and viral load dynamics involve a similar, limited, period of infectivity,
93
and asymptomatic and pre-symptomatic contributions to spread in the community remain
94
problematic.
10,11
95
96
Several studies have shown a relationship between Ct value and virus infectivity,
8, 12–14
and
97
manufacturers of rapid antigen LFDs have implied a link between antigen test positivity and
98
infectious potential. Here we present a detailed assessment of the relationship between Ct,
99
quantitative culture of infectious virus and antigen test positivity, alongside an independent
100
and unbiased head-to-head comparison of six widely available commercial antigen tests. Most
101
tests showed good sensitivity (>90%) at Cts of less than 25, with sensitivity increasing to over
102
95% when compared to infectious samples. Longitudinal studies of PCR-positive samples
103
highlight the importance of regular testing. We also re-assessed the tests against the dominant
104
genotype in the UK, B.1.1.7, and found no difference in test performance.
105
106
107
. CC-BY-NC-ND 4.0 International licenseIt is made available under a
is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review)
The copyright holder for this preprintthis version posted March 2, 2021. ; https://doi.org/10.1101/2021.02.27.21252427doi: medRxiv preprint
Materials and Methods
108
109
Study samples and ethics
110
111
Combined nasal/oropharyngeal swabs were submitted to the diagnostic laboratory in 1 ml of
112
viral transport medium (VTM) for routine real time SARS-CoV-2 RT-PCR testing. Surplus
113
VTM was routinely stored at -80
o
C by the diagnostic laboratory for future technology
114
evaluations. VTM from 100 laboratory-confirmed SARS-CoV-2 positive swabs selected to
115
cover a wide range of Cts (14 to 39) and 100 confirmed negative swabs were used for head-to-
116
head comparisons of 6 commercial antigen tests. Similarly, VTM from an additional 141
117
confirmed positive swabs were used for comparative studies on infectivity and antigen testing.
118
All samples were collected between March and October 2020. A further 23 laboratory-
119
confirmed SARS-CoV-2 positive swabs, collected in January 2021, were shown by on-site
120
whole-genome sequencing to be from the B.1.1.7 variant and used for comparative evaluation
121
of the sensitivity of selected tests.
122
123
Matched routinely-collected serum samples stored for up to 48 hours at 4
o
C in the Viapath
124
Blood Sciences laboratory were retrieved after routine diagnostic testing prior to planned
125
discard and stored at -80
o
C for future serological analysis. All swabs, VTM and serum samples
126
were stored in the Directorate of Infection. Samples for research were retrieved by the primary
127
care team and anonymised before sending to the King’s College London laboratories for
128
analysis along with dates of symptom onset and sample collection, and any relevant routine
129
laboratory result obtained from that sample. All studies were performed in accordance with the
130
UK Policy Framework for Health and Social Care Research and with specific Research Ethics
131
Committee approval (REC 20/SC/0310).
132
133
134
Viral growth assays
135
136
For the comparative studies on infectivity and antigen positivity, each swab was subjected to
137
the following procedures: RNA extraction for subsequent RT-PCR and sequencing; titration
138
and viral titre determination by plaque assay; titration and infectivity determination by
139
intracellular anti-N staining; viral propagation for isolation of virus; and rapid antigen testing.
140
Viral growth assays were performed on Vero.E6 cells. For plaque assays, VTM was 10-fold
141
serially diluted and applied to Vero.E6 cells in 12-well plates, in a volume of 500 ul per well,
142
and incubated for 1 hour at 37
o
C. 500 ul of pre-warmed overlay (0.1% agarose in DMEM
143
supplemented with 2% FCS, pen/strep and amphotericin B) was then applied to each well, and
144
cultures were incubated for 72 hours at 37
o
C, before fixing with 4% formaldehyde. A solution
145
of 0.05% crystal violet in ethanol was applied to each well, incubated for 5 minutes at room
146
temperature, before washing with PBS, air drying and counting plaques.
147
148
RT-PCR
149
150
Initial diagnostic laboratory testing was carried using the AusDiagnostics Multiplex Tandem
151
SARS-CoV-2 PCR assays at Viapath Infection Sciences laboratory, St Thomas’ Hospital,
152
London, and positive or negative swabs were selected on basis of this diagnostic test. For
153
additional PCR testing and to ensure uniformity of RT-PCR conditions and Ct determination,
154
RNA was extracted from 100ul of swab using the Qiagen QIAamp Viral RNA Kit following
155
manufacturer’s instructions and eluted in 60ul of water. RT-PCR reactions (total volume 20µl)
156
were performed with 5µl of eluted RNA, 4x TaqMan Fast Virus 1-Step Master mix (Applied
157
. CC-BY-NC-ND 4.0 International licenseIt is made available under a
is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review)
The copyright holder for this preprintthis version posted March 2, 2021. ; https://doi.org/10.1101/2021.02.27.21252427doi: medRxiv preprint
Biosystems) and CDC’s IDT Primer-Probes Sets targeting SARS-CoV-2 N gene regions or
158
human RNAse P, using a QuantStudio 5 (ThermoFisher Scientific). RNA standards were
159
extracted as above from serial dilutions of a NATtrol™ SARS-CoV-2 Stock (ZeptoMetrix),
160
which is constituted of inactivated intact viral particles with known RNA viral load.
161
162
Rapid antigen tests
163
164
Tests were performed according to the manufacturers’ instructions, with the exception that
165
swabs stored in viral transport medium (VTM) were used for evaluations, rather than direct
166
swabs performed immediately prior to test performance. 50 ul of stored swab was mixed with
167
100 ul of buffer supplied with the test kit, and 100 ul of this was applied to the test cassette.
168
Results were scored at the time stipulated by the manufacturer (between 10 and 30 minutes).
169
Results were recorded independently by two readers, compared, and in the event of a discordant
170
score were referred to a third individual. For the purpose of comparison, chromatographic tests
171
were scored according to whether the test band was strongly positive (2), clearly positive (1),
172
weakly positive (0.5) or negative (0).
173
174
Table 1: Rapid antigen LFD names and manufacturers
175
176
Neutralisation assays
177
178
Neutralisation assays were performed on Vero.E6 cells with replication-competent SARS-
179
CoV-2 (England 02/2020) as previously described.
15
Briefly, 20,000 cells were seeded per well
180
of a 96-well plate the day before assay. Heat-inactivated sera were 3-fold serially diluted and
181
incubated with 400 PFU per well of SARS-CoV-2 for 1 hour at 37
o
C. Medium was removed
182
from the cells and replaced with the virus/serum mixtures. After 24 hours at 37
o
C, cells were
183
fixed in 4% formaldehyde before intracellular nucleocapsid staining.
184
185
Intracellular nucleocapsid staining
186
187
Immunostaining for SARS-CoV-2 nucleocapsid detection in Vero.E6 cells was performed in
188
situ in formaldehyde-fixed 96-well plates, to verify viral culture experiments and as a read-out
189
for infectious virus neutralisation assays, as described previously.
15
Briefly, cells were
190
permeabilised with 0.1% triton in PBS for 15 minutes, then blocked in 3% milk for 15 minutes
191
at room temperature. Primary antibody (murinized anti-N 3009) was incubated at a final
192
concentration of 2 ug/mL in 1% milk for 45 minutes at room temperature, before washing twice
193
with PBS and incubating with secondary antibody (goat-anti-mouse IgG HRP-linked, Cell
194
Signaling Technology, 1:2000) in 1% milk for 45 minutes at room temperature. Cells were
195
washed twice with PBS, before addition of substrate. For SARS-CoV-2 plaque verification
196
assays, TrueBlue HRP substrate was used (Seracare Life Sciences Inc.); for neutralisation
197
Rapid Antigen LFD
Distributor/Manufacturer
Reference
SARS-CoV-2 Antigen Test Kit
Innova Med Group
N/A
SARS-CoV-2 Antigen Rapid Test Cassette
(Swab)
Spring Healthcare
SP-SW 106
SARS-CoV-2 Antigen Rapid Test Cassette
(Nasopharyngeal Swab)
SureScreen Diagnostics Ltd
COVID19 AGVCT
SARS-CoV-2 Antigen Rapid Test Device
Emmo Pharma / Encode
N/A
COVID-19 Antigen Rapid Fluorescent
Cassette
SureScreen Diagnostics Ltd
COVID19 AGC
Rapid Diagnostic Test
E25Bio
N/A
. CC-BY-NC-ND 4.0 International licenseIt is made available under a
is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review)
The copyright holder for this preprintthis version posted March 2, 2021. ; https://doi.org/10.1101/2021.02.27.21252427doi: medRxiv preprint