Featured Articles
CSF biomarker variability in the Alzheimer’s Association quality
control program
Niklas Mattsson
a,b,
*
, Ulf Andreasson
a
, Staffan Persson
a
, Maria C. Carrillo
c
, Steven Collins
d
,
Sonia Chalbot
e
, Neal Cutler
f
, Diane Dufour-Rainfray
g
, Anne M. Fagan
h
, Niels H. H. Heegaard
i
,
Ging-Yuek Robin Hsiung
j
, Bradley Hyman
k
, Khalid Iqbal
e
, D. Richard Lachno
l
, Alberto Lle
o
m
,
Piotr Lewczuk
n
, Jos
e L. Molinuevo
o
, Piero Parchi
p
, Axel Regeniter
q
, Robert Rissman
r
,
Hanna Rosenmann
s
, Giuseppe Sancesario
t
, Johannes Schr
€
oder
u
, Leslie M. Shaw
v
,
Charlotte E. Teunissen
w
, John Q. Trojanowski
v
, Hugo Vanderstichele
x
, Manu Vandijck
y
,
Marcel M. Verbeek
z
, Henrik Zetterberg
a,aa
, Kaj Blennow
a
, Stephan A. K
€
aser
bb
; on behalf of the
Alzheimer’s Association QC Program Work Group
a
Clinical Neurochemistry Laboratory, Institute of Neuroscience and Physiology, Department of Psychiatry and Neurochemistry, the Sahlgrenska Academy at the
University of Gothenburg, Sahlgrenska University Hospital, M
€
olndal, Sweden
b
San Francisco VA Medical Center, Center for Imaging of Neurodegenerative Diseases, University of California San Francisco, San Francisco, CA, USA
c
The Alzheimer’s Association, Chicago, IL, USA
d
Department of Pathology, Florey Institute of Neuroscience and Mental Health, The University of Melbourne, Melbourne, Australia
e
New York State Institute for Basic Research, Staten Island, NY, USA
f
Worldwide Clinical Trials, Beverly Hills, CA, USA
g
CHRU de Tours, Laboratoire de m
edecine nucl
eaire; Universit
e Franc¸ois-Rabelais de Tours, PRES Centre-Val de Loire Universit
e; and UMR INSERM U 930
Imagerie et cerveau, Universit
e Franc¸ois-Rabelais de Tours, PRES Centre-Val de Loire Universit
e, Tours, France
h
Washington University, St. Louis, MO, USA
i
Department of Clinical Biochemistry, Immunology & Genetics, Statens Serum Institut, Copenhagen, Denmark
j
University of British Columbia, Vancouver, Canada
k
MassGeneral Institute for Neurodegenerative Disease, Charlestown, MA, USA
l
Eli Lilly and Company, Windlesham, UK
m
Hospital de Sant Pau, Barcelona, Spain
n
Universit
€
atsklinikum Erlangen, Erlangen, Germany
o
Alzheimer’s Disease and Other Cognitive Disorders Unit, Neurology Service, ICN Hospital Clinic i Universitari and Pasqual Maragall Foundation,
Barcelona, Spain
p
Institute of Neurological Sciences and Department of Biomedical and Neuromotor Sciences, University of Bologna, Bologna, Italy
q
Basel University Hospital, Basel, Switzerland
r
School of Medicine, University California, San Diego, La Jolla, CA, USA
s
Hadassah Hebrew University Medical Center, Jerusalem, Israel
t
General Hospital of University of Rome Tor Vergata, Rome, Italy
u
Sektion Gerontopsychiatrie Universit
€
atsklinikum, Heidelberg, Germany
v
University of Pennsylvania, Philadelphia, PA, USA
w
Department of Clinical Chemistry, VU University Medical Center, Amsterdam, Netherlands
x
ADx NeuroSciences, Gent, Belgium
y
Innogenetics nv (part of Fujirebio), Gent, Belgium
z
Departments of Neurology and Laboratory Medicine, Donders Institute for Brain, Cognition and Behaviour, Radboud University Medical Centre,
Nijmegen, Netherlands
aa
Institute of Neurology, University College London, Queen Square, London, UK
bb
University of T€ubingen, T€ubingen, Germany
K. B., H. Z., N. M., and U. A. designed the study. N. M. and U. A. per-
formed statistical analyses. N. M. drafted the manuscript. S. P. was the study
coordinator. All authors participated in interpretation of data, revised the
manuscript for intellectual content, and approved the final version.
*Corresponding author. Tel.: 11-415-494-1139; Fax: 11-415-668-
2864.
E-mail address: niklas.mattsson@neuro.gu.se
1552-5260/$ - see front matter Ó 2013 The Alzheimer’s Association. All rights reserved.
http://dx.doi.org/10.1016/j.jalz.2013.01.010
Alzheimer’s & Dementia 9 (2013) 251–261
Abstract Background: The cerebrospinal fluid (CSF) biomarkers amyloid beta 1–42, total tau, and phosphor-
ylated tau are used increasingly for Alzheimer’s disease (AD) research and patient management.
However, there are large variations in biomarker measurements among and within laboratories.
Methods: Data from the first nine rounds of the Alzheimer’s Association quality control program
was used to define the extent and sources of analytical variability. In each round, three CSF samples
prepared at the Clinical Neurochemistry Laboratory (M
€
olndal, Sweden) were analyzed by single-
analyte enzyme-linked immunosorbent assay (ELISA), a multiplexing xMAP assay, or an immuno-
assay with electrochemoluminescence detection.
Results: A total of 84 laboratories participated. Coefficients of variation (CVs) between laboratories
were around 20% to 30%; within-run CVs, less than 5% to 10%; and longitudinal within-laboratory
CVs, 5% to 19%. Interestingly, longitudinal within-laboratory CV differed between biomarkers at in-
dividual laboratories, suggesting that a component of it was assay dependent. Variability between kit
lots and between laboratories both had a major influence on amyloid beta 1–42 measurements, but for
total tau and phosphorylated tau, between-kit lot effects were much less than between-laboratory ef-
fects. Despite the measurement variability, the between-laboratory consistency in classification of
samples (using prehoc-derived cutoffs for AD) was high (.90% in 15 of 18 samples for ELISA
and in 12 of 18 samples for xMAP).
Conclusions: The overall variability remains too high to allow assignment of universal biomarker
cutoff values for a specific intended use. Each laboratory must ensure longitudinal stability in its mea-
surements and use internally qualified cutoff levels. Further standardization of laboratory procedures
and improvement of kit performance will likely increase the usefulness of CSF AD biomarkers for
researchers and clinicians.
Ó 2013 The Alzheimer’s Association. All rights reserved.
Keywords: Alzheimer’s disease; Cerebrospinal fluid; Biomarkers; External assurance; Quality control; Proficiency testing
1. Introduction
Cerebrospinal fluid (CSF) examination in Alzheimer’s
disease (AD) typically shows reduced levels of amyloid
b 1–42 (Ab42), and increased levels of total tau (T-tau)
and phosphorylated tau (P-tau) [1–3]. The presence of this
CSF pattern has recently been proposed for use in the
research diagnostic criteria for AD [4–7]. Clinical
diagnostic testing of CSF samples is already available
from several hospital laboratories as well as from
commercial laboratories. The measured biomarker levels,
however, differ among studies, which may be the result of
a number of preanalytical, analytical, or assay-related fac-
tors [8–10]. To overcome this situation, several
standardization efforts have been initiated to harmonize
laboratory procedures [11], give guidelines on CSF collec-
tion and handling procedures [12], define reference mea-
surement procedures [13], and construct reference
materials for assay calibration [14].
The Alzheimer’s Association launched an international
quality control (QC) program for CSF biomarkers in 2009
[15]. The program was established to monitor total analyti-
cal variability for Ab and tau proteins in CSF, to provide
a network where sources of variation could be identified,
and to implement actions originating from standardization
efforts. There are no requirements or obligations to become
a participant for the QC program other than using a commer-
cially available assay for Ab or tau. Three complete rounds
of samples, each including two round-specific samples and
one longitudinal sample that remains the same over years,
are prepared at the Clinical Neurochemistry Labora tory in
M
€
olndal, Sweden, and shipped yearly to participating labo-
ratories. Moreover, five experienced laboratories that pro-
cess large numbers of samples routinely serve as reference
laboratories and analyze the samples multiple times. The re-
sults from the first two rounds, involving 40 laboratories,
have been described previously [15].
Herein, we report the development of the program during
2010 to 2012, and describe results through to program round
9. During this time, the number of participating sites dou-
bled, and the large amount of data collected increased our
capability to identify sources of measurement variability, in-
cluding differences between laboratories and between lots of
analytical kits.
2. Methods
2.1. CSF samples and laboratory procedures
As reported previously [15], human CSF pools were pre-
pared in M
€
olndal, Sweden, from a large number of fresh, de-
identified samples obtained during routine clinical workflow
(all samples underwent one freeze/thaw cycle before pool-
ing). No extra amount (spiking) of analyte was added to
the samples. The pools were prepared by experienced and
certified laboratory technicians during continuous mixing
to ensure homogeneity of the pools. The total volumes of
the pools were 75 to 1500 mL. The pools were divided
into 500-mL aliquots in polypropylene screw-cap tubes
(art. no. 72.692, 1.5 mL; Sarstedt AG & Co., N
€
umbrecht,
N. Mattsson et al. / Alzheimer’s & Dementia 9 (2013) 251–261252
Germany; except for samples 2011-6A, 2011-7A, 2012-8B,
and 2012-9B, for which we used art. no. 72.730.007, 0.5 mL;
Sarstedt AG & Co.). The samples were refrozen at 280
C,
followed by distribution to the participating laboratories on
dry ice by courier. All shipments included three samples.
Two (blinded challenge samples) were specific to the round
(designated 2009-1A, 2009-1B, 2010-2A, 2010-2B, and so
forth), and one sample (quality control longitudinal sample
[QC-L]) was from a pool used to evaluate longitudinal sta-
bility (used until round 7 [total shelf life of the sample, 26
months], when it was discontinued because of a supply
shortage and was exchanged for a new longitudinal sample).
The blinded chal lenge samples differed in their AD bio-
marker profiles (Fig. 1).
All laboratories verified that the samples had arrived fro-
zen. The analyses were done by each participant in duplicate
as part of their routine laboratory activities. No extra freeze/
thawing of samples was allowed. The reference laboratories
(located in Amsterdam, M
€
oln dal, Erlangen, Ghent, and Penn-
sylvania) analyzed the samples six times (with one aliquot
per run) using different plates to assess within-laboratory pre-
cision. All results were reported back to M
€
olndal for data
analysis together with a questionnaire that gave an overview
of the applied materials and handling procedures for the spe-
cific run for data analysis on the reported results.
2.2. Participating laboratories and assay systems
The size and exposure of the Alzheimer’s Association QC
program has grown continuously since its start in 2009. The
majority of the participants use INNOTEST enzyme-linked
immunosorbent assays (ELISAs; n 5 61 in round 9) or
bead-based xMAP platforms with the INNO-BIA AlzBio3
(both Innogenetics, Gent, Belgium; www.innogenetics.
com;n5 12 in round 9) to quantify Ab42, T-tau and P-tau
(181P) (or simply P-tau). Meso Scale Discovery (MSD; Gai-
thersburg, MD; www.mesoscale.com) tec hnology was used
by a smaller number of laboratories (n 5 8 in round 9) for
AbN-42, AbN-40, and AbN-38 (Ab triplex). MSD Ab tri-
plex was used with either 4G8 (epitope Ab17–24) or 6E10
(epitope Ab9–12) as detection antibodies. The volume of
provided samples (500 mL) was sufficient to allow for dupli-
cate analyses of the sample with ELISA (T-tau, 2 ! 25 mL;
Ab42, 2 ! 25mL; and P-tau, 2 ! 75 mL), xMAP (2 ! 75
mL), MSD (Ab triplex 2 ! 25 mL), or combinations thereof.
Several laboratories (n 5 9, 13% in round 9) used multiple
Fig. 1. Measurements of blinded quality control test samples. Dots with error bars show mean measured concentrations and standard deviation from all par-
ticipating sites (the samples were made from different pools of cerebrospinal fluid [CSF], so constant concentrations were not expected). Connected lines show
the coefficient of variation (CV; right-hand y-axes). (A–C) INNOTEST Enzyme-linked immunosorbent assay (ELISA). (D–F) INNO-BIA xMAP. (G–I) Meso
Scale Discovery (MSD) amyloid beta (Ab) triplex. The CV for xMAP total tau (T-tau) sample 7B (E) was very high (64%) because of a single extreme outlier
(the CV was 22% after removal of this outlier). P-tau, phosphorylated tau.
N. Mattsson et al. / Alzheimer’s & Dementia 9 (2013) 251–261 253
techniques. Note that samples were analyzed as part of the
laboratories’ routine activities, and a large total numb er of
different production lots of analytical kits were used through-
out the program. The total numbers of different kit lots used
were 44 for ELISA Ab42, 39 for ELISAT-tau, 33 for ELISA
P-tau, 21 for xMAP, and 29 for MSD. However, some lots
were overrepresented in the program (about 50% of measure-
ments for each analyte were done using only seven different
kit lots for ELISA Ab42, seven for ELISA T-tau, five for
ELISA P-tau, five for xMAP, and eight for MSD).
2.3. Estimates of variability
The overall variability of attained results may be de-
scribed by the coefficient of variation (CV; standar d devia-
tion ! 100 divided by the mean) for each sample and
assay. Some of the variables are the responsibility of the ven-
dors of the assays, whereas other variables are considered to
be responsibility of the performing laboratory. The overall
variability is affected by several different factors, including
within-assay run variability (between duplicate samples),
within-laboratory longitudinal variability, between-
laboratory variability, and within- and between-assay kit
lot variability. Variability depends also on a combination
of trueness (bias, systematic deviation from a reference
value) or precision (impr ecision, random deviation from
a value). In this study, we aimed to estimate the size and
source of these different types of variability.
2.4. Statistical analysis
Biomarker results were analyzed statistically and group-
ed by rounds, samples, and analytical techniques. Mean
levels, standard deviations, and CVs were calculated.
Between-group differences were assessed using nonpara-
metric tests (Mann-Whitney U or Kruskal-Wallis tests).
Analysis of variance was performed using restricted
maximum likelihood estimation of covariances (the esti-
mated variance components were between-laboratory and
between-batch lot variability). SPSS version 20 (IBM Cor-
poration, Armonk, NY, USA) and GraphPad Prism 5 (Graph-
Pad Software Inc., La Jolla, CA, USA) were used.
3. Results
3.1. Overall variability
The overall CV was 20% to 30% for most assays and
samples. All mean levels, standard deviations, and CVs for
blinded test samples are presented in Fig. 1. For ELI SAs,
mean CV was 23% (range, 17%–29%) for Ab 42, 18%
(range, 12%–27%) for T-tau, and 19% (range, 12%–28%)
for P-tau. For xMAP, mean CV was 28% (range, 17%–
38%) for Ab42, 20% (range, 13%–28%, after removal of
one significant outlier, see Fig. 1) for T-tau, and 21% (range,
11%–30%) for P-tau. For MSD, mean CV was 24% (range,
13%–36%) for Ab42, 26% (range, 16%–37%) for Ab40, and
27% (range, 10%–60%) for Ab38. These data combined
MSD assays using different Ab detection antibodies (see
Supplemental Fig. 1 for MSD data stratified by antibody).
3.2. Within-run variability
In rounds 4 to 7, the laboratories reported within-run
variability as CV of duplicate measurements for the QC-
L sample. Median within-run CV was less than 4% for
ELISA, 1.9% to 7.4% for xMAP, and 1.5% to 17% for
MSD assays (17% was an outlier for the MSD assays,
for which most within-run CVs were less than 10%; see
Table 1). No trend in the within-run variability over the
study was noted, which could indicate that all laboratories,
independent from their experience level, have a comparable
within-run variability.
3.3. Longitudinal variability
Longitudinal variability was estimated separately at the
five reference laboratories (using several different samples
measured at six different time points) and at all laboratories
(using the QC-L sample at laboratories participating in at
least three rounds; Fig. 2).
Table 1
Within-run variability in cerebrospinal fluid measurements among laboratories
Round
ELISA xMAP MSD 6E10 MSD 4G8
Ab42 T-tau P-tau Ab42 T-tau P-tau Ab42 Ab40 Ab38 Ab42 Ab40 Ab38
4 3.2
(1.2, 6, 29)
3.0
(2.0, 5.2, 32)
2.1
(1.0, 3.7, 31)
2.5
(1.2, 4.8, 12)
5.2
(2.2, 9.1, 12)
3.6
(2.7, 6.1, 12)
5.7
(5.1, 6.8, 5)
3.9
(2.3, 4.1, 5)
4.2
(3.8, 4.4, 5)
13* 5* 17*
5 2.5
(1.7, 4.7, 43)
3.0
(1.4, 6.9, 46)
1.6
(0.5, 2.9, 44)
4.1
(2.6, 6.1, 14)
3.5
(1.3, 6.9, 14)
3.2
(2.3, 5.4, 14)
6.7
(3.8, 13, 5)
1.5
(1.0, 5.4, 5)
5.9
(5.4, 9, 5)
4.6
(2.7, 11, 3)
6.3
(4.7, 9.1, 3)
8.7
(5.1, 13, 3)
6 3.7
(1.7, 6.6, 47)
3.4
(1.9, 7.3, 50)
1.6
(0.9, 4.8, 48)
4.9
(1.4, 6.2, 15)
4.0
(1.7, 8.4, 15)
5.7
(2.9, 8.9, 15)
6.1
(4.5, 7.7, 4)
2.7
(2.0, 3.0, 4)
3
(1.2, 4.7, 4)
5.9
(5.4, 6.5, 2)
5.5
(3.7, 7.2, 2)
7.6
(4.4, 11, 2)
7 2.7
(1.7, 4.2, 52)
2.9
(0.9, 5.8, 52)
2.0
(0.7, 4,2, 53)
7.3
(2.9, 12, 14)
3.7
(2.2, 12, 13)
3.7
(1.7,
6.6, 13)
2.1
(0.7, 3.7, 4)
3.1
(2.0, 4.2, 4)
3.0
(1.8, 3.8, 4)
4.4
(2.8, 6.2, 3)
4.2
(3.1, 10, 3)
1.5
(1.3, 1.8, 2)
Abbreviations: ELISA, enzyme-linked immunosorbent assay; MSD, Meso Scale Discovery; Ab, amyloid beta; T-tau, total tau; P-tau, phosphorylated tau.
NOTE. Within-run variability was calculated using duplicate measurements (two wells) of the quality control longitudinal sample. Data are expressed as
median of the coefficient of variation (25th percentile, 75th percentile, n). The results were similar for the other samples (data not shown).
*Data available from one laboratory only.
N. Mattsson et al. / Alzheimer’s & Dementia 9 (2013) 251–261254
At the reference laboratories, the mean longitudinal
within-laboratory CV was 8% to 13% for ELISA (n 5 4)
and 5% to 17% for xMAP measurements (n 5 3). There
were differences in CVs between the reference laboratories
and also between analytes at the same laboratory. For exam-
ple, the variability for xMAP P-tau was very high at refer-
ence laboratory 3 (Fig. 2B). The cause for this is unknown,
but we verified that it did not depend on single outliers, or
errors in reporting results, and that the CVs for simultaneous
measurements of xMAP Ab42 and T-tau were not elevated,
the latter suggesting that assay-dependent factors rather than
factors related to laboratory procedures were important.
The within-laboratory longitudinal CVs at all participat-
ing laboratories were often higher than the CVs at the refer-
ence laboratories (12%–19%, Fig. 2), with the highest CV
seen for Ab42. The overall variability for the QC-L samples
was approximately 20% to 30% (comparable with the
blinded test sample results described earlier, Fig. 1), with
no significant change over time in mean concentrations
(Fig. 3). This result suppor ts that the QC-L samples were sta-
ble during storage at 280
C for 26 months. However, we
noted that the variability was lower among the reference lab-
oratories than amo ng all laboratories, especially for Ab42
(Fig. 3A, B).
3.4. Between-laboratory vs between-lot variability
It is important to establish how much of the overall vari-
ability is caused by differences between laboratories vs dif-
ferences between manufactured lots. Analysis of variance
was used to estimate the separate contributions of these com-
ponents. For ELISA measurements of Ab42, between-
laboratory and between-kit lot components demonstrated
approximately equal contributions, but for T-tau and P-tau
the between-laboratory component was much larger than
the between-lot component (Fig. 4). For xMAP measure-
ments, both components contributed to the Ab42 variability,
but for T-tau and P-tau the between-lot component was re-
dundant, suggesting that its contribution was very small. Be-
cause of the unbalanced design and limited amo unt of data
per assay lot and laboratory, variance components were esti-
mated with large uncertainties. The results should therefore
be interpreted as rankings of the different factors rather than
exact calculations of their contributions.
3.5. Bias vs imprecision
We next examined bias and imprecision, which are
descriptions of systematic and random deviations from a ref-
erence value, respectively. As expected [15], there was
a large bias in analyte concentrations when evaluated against
the different assay formats (see Fig. 1, but for measurements
correlated between assay formats, see Supplemental Fig. 2).
All subsequent statistical analyses were theref ore performed
by comparing laboratories using identical instrument plat-
forms. Because there are no available standardized reference
methods for CSF AD biomarker measurements, mean
concentrations were used as reference values. For each mea-
surement, the relative difference from the corresponding ref-
erence value (mean of measurements in all laboratories) was
calculated. For each laboratory, the average of those relative
differences was used to calculate the bias, whereas the vari-
ance of the differences was used to calculate the imprecision.
For example, if a laboratory systematically reported higher
than average concentrations, it had a positive bias; if
Fig. 2. (A, B) Within-laboratory longitudinal coefficients of variation
(CVs) were calculated by repeated measurements at reference laboratories
(Refx; using six measurements per sample, varying number of samples
per laboratory) and in the whole program (using the quality control
longitudinal sample at laboratories measuring the sample at least three
times during rounds 1 through 7). Data are means of CV (error bars are
standard deviations) for each biomarker, ordered by laboratory (x-axes).
Enzyme-linked immunosorbent assay (ELISA) reference laboratory 4
(Ref4) only used two lots of analytical kits for each analyte, which limits
the influence of lot-dependent variability. Meso Scale Discovery (MSD)
data are not included in the figure, since only one reference laboratory re-
ported data for MSD within-laboratory longitudinal CV (mean CV at that
laboratory was 11% to 17% (standard deviation, 4%–8%) for all amyloid
beta (Ab) triplex measurements using either 6E10 or 4G8 as the detection
antibody. T-tau, total tau; P-tau, phosphorylated tau.
N. Mattsson et al. / Alzheimer’s & Dementia 9 (2013) 251–261 255
