Radiomics
Citation for published version (APA):
Lambin, P., Leijenaar, R. T. H., Deist, T. M., Peerlings, J., de Jong, E. E. C., van Timmeren, J.,
Sanduleanu, S., Larue, R. T. H. M., Even, A. J. G., Jochems, A., van Wijk, Y., Woodruff, H., van Soest, J.,
Lustberg, T., Roelofs, E., van Elmpt, W., Dekker, A., Mottaghy, F. M., Wildberger, J. E., & Walsh, S.
(2017). Radiomics: the bridge between medical imaging and personalized medicine. Nature Reviews
Clinical Oncology, 14(12), 749-762. https://doi.org/10.1038/nrclinonc.2017.141
Document status and date:
Published: 01/12/2017
DOI:
10.1038/nrclinonc.2017.141
Document Version:
Publisher's PDF, also known as Version of record
Document license:
Taverne
Please check the document version of this publication:
• A submitted manuscript is the version of the article upon submission and before peer-review. There can
be important differences between the submitted version and the official published version of record.
People interested in the research are advised to contact the author for the final version of the publication,
or visit the DOI to the publisher's website.
• The final author version and the galley proof are versions of the publication after peer review.
• The final published version features the final layout of the paper including the volume, issue and page
numbers.
Link to publication
General rights
Copyright and moral rights for the publications made accessible in the public portal are retained by the authors and/or other copyright
owners and it is a condition of accessing publications that users recognise and abide by the legal requirements associated with these
rights.
• Users may download and print one copy of any publication from the public portal for the purpose of private study or research.
• You may not further distribute the material or use it for any profit-making activity or commercial gain
• You may freely distribute the URL identifying the publication in the public portal.
If the publication is distributed under the terms of Article 25fa of the Dutch Copyright Act, indicated by the “Taverne” license above,
please follow below link for the End User Agreement:
www.umlib.nl/taverne-license
Take down policy
If you believe that this document breaches copyright please contact us at:
repository@maastrichtuniversity.nl
providing details and we will investigate your claim.
Download date: 09 Aug. 2022
Imaging is an important technology in medical sci-
ence and is used in clinical practice to aid decision
making
1
. The role of medical imaging, however, is
swiftly evolving from being primarily a diagnos-
tic tool to also include a central role in the context
of personalized precision medicine
2
. In radiomics
3,4
,
digitally encrypted medical images that hold infor-
mation related to tumour pathophysiology are trans-
formed into mineable high-dimensional data
1
. This
information can be harnessed through quantitative
image analyses
5
and leveraged via clinical-decision
support systems (CDSS)
6
to improve medical decision-
making. Radiomics builds upon several decades of
computer-aided diagnosis, prognosis, and thera peutics
research
7,8
. The process used in radiomics involves the
identification of vast arrays of quantitative features
within digital images, storage of such data in federated
databases (that is, a system in which several independ-
ent databases function as a single entity) and the sub-
sequent mining of the data for knowledge extraction
and application
9
. Innumerable quantitative features
can now be extracted using high-throughput comput-
ing from medical images such as CT, MR, and/or PET.
The creation of databases that link immense volumes
of radiomics data (ideally with all other pertinent data)
from millions of patients to form vast, rapid learn-
ing healthcare (RLHC) networks is conceivable, but
presents a considerable data management hurdle
10–13
.
Radiomics is not a panacea for clinical decision-
making. Radiomic features (such as intensity, shape,
texture or wavelet) offer information on cancer pheno-
type as well as the tumour microenvironment that is
distinct and complementary to other pertinent data
sources (including clinically obtained, treatment- related
or genomic data)
14
. Radiomics-derived data, when
combined with other pertinent data and correlated
and/or inferred with outcomes data, can produce
accurate robust evidence-basedCDSS.
The potential of radiomics to improve CDSS is
beyond doubt
15
and the field is evolving rapidly. The
principal challenge is the optimal collection and
integration of diverse multimodal data sources in a
quantitative manner that delivers unambiguous clin-
ical predictions that accurately and robustly enable
outcome prediction as a function of the impending
decisions
16
. Many published prediction models that
account for factors related to both disease and treat-
ment are available, but these models lack standardized
1
The D‑Lab: Decision Support
for Precision Medicine,
GROW – School for Oncology
and Developmental Biology,
Maastricht University Medical
Centre, Universiteitssingel 40,
6229 ER, Maastricht,
The Netherlands.
2
Department of Radiology
and Nuclear Medicine,
GROW – School for Oncology
and Developmental Biology,
Maastricht University
Medical Centre, Doctor
Tanslaan 12, 6229 ET,
Maastricht, The Netherlands.
3
Department of Radiation
Oncology (MAASTRO),
GROW – School for Oncology
and Developmental Biology,
Maastricht University
Medical Centre, Doctor
Tanslaan 12, 6229 ET,
Maastricht, The Netherlands.
4
Department of Nuclear
Medicine, University Hospital
RWTH Aachen,
Pauwelsstraße 30,
52074 Aachen, Germany.
*These authors contributed
equally to this work.
Correspondence to P.L.
philippe.lambin@
maastrichtuniversity.nl
doi:10.1038/nrclinonc.2017.141
Published online 4 Oct 2017
Radiomics: the bridge between medical
imaging and personalized medicine
Philippe Lambin
1
, Ralph T.H.Leijenaar
1
*, Timo M.Deist
1
*, Jurgen Peerlings
1,2
,
Evelyn E.C.de Jong
1
, Janita van Timmeren
1
, Sebastian Sanduleanu
1
,
Ruben T.H.M.Larue
1
, Aniek J.G.Even
1
, Arthur Jochems
1
, Yvonka van Wijk
1
,
Henry Woodruff
1
, Johan van Soest
3
, Tim Lustberg
3
, Erik Roelofs
1,3
, Wouter van Elmpt
3
,
Andre Dekker
3
, Felix M.Mottaghy
2,4
, Joachim E.Wildberger
2
and Sean Walsh
1
Abstract
|
Radiomics, the high-throughput mining of quantitative image features from
standard-of-care medical imaging that enables data to be extracted and applied within
clinical-decision support systems to improve diagnostic, prognostic, and predictive accuracy,
is gaining importance in cancer research. Radiomic analysis exploits sophisticated image analysis
tools and the rapid development and validation of medical imaging data that uses image-based
signatures for precision diagnosis and treatment, providing a powerful tool in modern medicine.
Herein, we describe the process of radiomics, its pitfalls, challenges, opportunities, and its
capacity to improve clinical decision making, emphasizing the utility for patients with cancer.
Currently, the field of radiomics lacks standardized evaluation of both the scientific integrity and
the clinical relevance of the numerous published radiomics investigations resulting from the
rapid growth of this area. Rigorous evaluation criteria and reporting guidelines need to be
established in order for radiomics to mature as a discipline. Herein, we provide guidance for
investigations to meet this urgent need in the field of radiomics.
REVIEWS
NATURE REVIEWS
|
CLINICAL ONCOLOGY VOLUME 14
|
DECEMBER 2017
|
749
©
2
0
1
7
M
a
c
m
i
l
l
a
n
P
u
b
l
i
s
h
e
r
s
L
i
m
i
t
e
d
,
p
a
r
t
o
f
S
p
r
i
n
g
e
r
N
a
t
u
r
e
.
A
l
l
r
i
g
h
t
s
r
e
s
e
r
v
e
d
.
Exploratory
analysis
Medical
imaging
Data
selection
VOI
Prediction target
Imaging protocols
Test feature stability
Report protocols
Add prediction target
Report algorithms
Store data
Add clinical variables
Archetypal features
Robust segmentation
Feature selection
Internal validation
Radiomics
External validation
Report methodology
Modeling
Feature
extraction
RQS 1
RQS 2
RQS 3
RQS Total 36
Image
protocol
quality
+1 or +2
RQS
checkpoint 1
total: 2
+1+1+1
RQS
checkpoint 2
total: 3
Multiple
segmentation
Phantom
study
Imaging at
multiple
time points
Feature
reduction or
adjustment
for multiple
testing
Multivariable
analysis
Biological
correlates
Cut-off
analysis
Discrimination
statistics
Calibration
statistics
Prospective
study
Validation Comparison
to ‘gold
standard’
Cost-
effectiveness
analysis
Potential
clinical
applications
Open
science
and data
+1 +1 +1 +1 or +2 +1 or +2 +7 +2 +2 +1 -3 or +3 -5 to +5 +1 to +4
RQS
checkpoint 3
total: 31
Nature Reviews | Clinical Oncology
evaluation of their performance, reproducibility,
and/or clinical utility
17
. Consequently, these models
might not be appropriate forCDSS.
In this Review, we describe the process of radiomics
along with latest developments in the field. The pitfalls,
challenges, and opportunities presented by radiomics
to improve CDSS for personalized precision oncology
are highlighted, with an emphasis on the methodo-
logical aspects of radiomics prediction model devel-
opment and validation. We explore the advanced and
innovative information technologies that are essential
for the data management of diverse multimodal data
sources. Finally, we offer a vision of the necessary
steps to ensure continued progression and widespread
acceptance of both radiomics andCDSS.
The workflow of radiomics
Radiomics is defined as the quantitative mapping, that
is, extraction, analysis and modelling of many medical
image features in relation to prediction targets, such as
clinical end points and genomic features. A radiomics
study can be structured in five phases: data selection,
medical imaging, feature extraction, exploratory ana-
lysis, and modelling (FIG.1). To assess the quality of
radiomics studies, we propose the radiomics quality
score (RQS).
Data selection
Radiomic analyses begins with the choice of an imaging
protocol, the volume of interest (VOI) and a prediction
target—the event one wishes to predict. Typically, the
entire primary tumour is analysed and linked to available
data on treatment outcomes, such as survival. Radiomic
analyses can be performed on subregions of the tumour
(habitats), metastatic lesions, as well as in normal tis-
sues. Analysis of these regions might yield radiosensitive
phenotypes, which has implications for treatment plan-
ning strategies. Radiomics analysis, however, is not
restricted to radiotherapy and can be applied to any
image generated in the clinical setting (FIG.2).
The importance of using standardized imaging proto-
cols to eliminate unnecessary confounding variability is
recognized
9,18
; however, nonstandardized imaging proto-
cols are commonplace. Therefore, reproducibility and
Key points
• Radiomics is becoming increasingly more important in medical imaging
• The explosion of medical imaging data creates an environment ideal for
machine-learning and data-based science
• Radiomics-based decision-support systems for precision diagnosis and treatment can
be a powerful tool in modern medicine
• Large-scale data sharing is necessary for the validation and full potential that
radiomics represents
• Standardized data collection, evaluation criteria, and reporting guidelines are
required for radiomics to mature as a discipline
Figure 1
|
Flowchart depicting the workflow of radiomics and the application of the RQS. The workflow includes the
necessary steps in a radiomic analysis. The RQS both rewards and penalizes the methodology and analyses of a study,
consequently encouraging the best scientific practice. RSQ, radiomics quality score; VOI, volume of interest.
REVIEWS
750
|
DECEMBER 2017
|
VOLUME 14 www.nature.com/nrclinonc
©
2
0
1
7
M
a
c
m
i
l
l
a
n
P
u
b
l
i
s
h
e
r
s
L
i
m
i
t
e
d
,
p
a
r
t
o
f
S
p
r
i
n
g
e
r
N
a
t
u
r
e
.
A
l
l
r
i
g
h
t
s
r
e
s
e
r
v
e
d
.
©
2
0
1
7
M
a
c
m
i
l
l
a
n
P
u
b
l
i
s
h
e
r
s
L
i
m
i
t
e
d
,
p
a
r
t
o
f
S
p
r
i
n
g
e
r
N
a
t
u
r
e
.
A
l
l
r
i
g
h
t
s
r
e
s
e
r
v
e
d
.
Nature Reviews | Clinical Oncology
Cardiac CT Radiomics MACE Analysis
Phantom studies
An artificial structure that
imitates human tissue
properties is scanned on
multiple machines to
characterize scan output
against a known physical
standard.
comparability of radiomic studies can be achieved only
by extensive disclosure of imaging protocols. We wish
to emphasize this point, and provide examples of how
protocols should be reported in future radiomics studies
(Supplementary informationS1).
Medical imaging
Segmentation. VOIs are segmented manually or (semi-)
automatically
19
. This segmentation determines which vox-
els within an image are analysed, thus, the variability in seg-
mentation can introduce bias in the evaluation of derived
radiomic features
20
. Multiple-segmentation is a method to
limit the extent of this bias. Examples that enable robust
features to be observed
21
include: evaluation by multiple
clinicians, perturb segmentations with noise, combination
of diverse algorithms, or use different stages of the breath-
ing cycle. Key considerations are how the segmentation was
performed, and how sensitive the radiomics analysis is to
different segmentation methods
22
. For example, a semi-
automatic segmentation method can result in different
radiomic features than a manual delineation.
Phantom studies. The determination of inter-scanner and
inter-vendor variability of features is important in radi-
omics
23
. In cases in which radiomic studies rely on data
from multiple scanners, neglecting this variability can
jeopardize the analysis of studies—that is, the proposed
radiomic-based prediction model might not perform ade-
quately on external datasets if new data are acquired on
different scanners. As data from patients scanned on mul-
tiple devices is scarce and subject to uncertainties (such as
organ motion, or different imaging protocols), phantom
studies are a suitable means to gauge these uncertainties
and identify features that rely on the vendor. In essence,
phantom studies provide a risk-mitigation strategy to help
navigate from the current clinical imaging scenario to the
desired optimal imaging scenario.
Imaging at multiple time points. Additional sources
of variability in radiomics features are organ motion or
expansion or shrinkage of the target volume. Radiomics
features that are strongly dependent on these factors can
have limited applicability. To account for these sources of
variability, available test-retest data
24–26
can be exploited
to measure radiomics feature stability. For example, two
datasets of images acquired within a small period of time
from a patientcohort.
Feature extraction
The essence of radiomics is the high-throughput extrac-
tion of quantitative image features to characterize VOIs.
Feature values are dependent upon factors that can
include image pre-processing (for example, filtering, or
intensity discretization) and reconstruction (for exam-
ple, filtered back projection, or iterative reconstruction).
Furthermore, variation exists in feature nomenclature,
mathematical definition, methodology, and software
implementation of the applied feature extraction
algorithms
27–29
. In order to facilitate inter- operability
of radiomic features, differences in nomenclature,
algorithms, software implementations, as well as other
methodological aspects must be elucidated.
Exploratory analysis
Radiomic and non-radiomic features should be com-
bined with the prediction target to create a single
dataset. This approach enables the investigation of
relationships between features. Groups of highly cor-
related radiomics features can be identified via clus-
tering, and these features can be reduced to single
archetypal features per cluster. Radiomic features that
are well-correlated with routine clinical features (such
as tumour stage) do not provide additional information.
Auxiliary feature data collected from multiple segmen-
tations, multiple imaging, and phantom studies, can be
exploited to assess feature robustness. Volatile or robust
features can be identified and subsequently excluded
from model development. For example, a feature that
is robust for the prediction of overall survival for lung
cancer (that is, imaged and segmented in a certain way)
for a given dataset could be volatile for the prediction of
pneumonitis in lung cancer (imaged and segmented in
an alternative way) for a given dataset. Thus, the pro-
cess of feature reduction and/or exclusion should be
described clearly.
Modelling
Radiomic modelling involves three major aspects: fea-
ture selection, modelling methodology, and validation.
Feature selection should be data-driven owing to the
vast in- human range of possible radiomics features; such
analysis should be performed in a robust and transparent
manner. To achieve holistic models, features beyond radi-
omics (such as data from clinical records, data obtained
during treatment or biological and/or genetic) should
also be incorporated. Regarding the choice of modelling
methodology, the identification of optimal machine-
learning methods for radiomic applications is a crucial
step towards stable and clinically relevant CDSS; thus, in
the ideal scenario, multiple machine-learning methods
should be employed
30
and the implementation should be
comprehensively documented. A non-validated model is
Figure 2
|
Radiomics in cardiology. The current gold standard
for quantification of coronary calcifications visible on CT is the
‘Agatston’ method (based upon intensity and volume).
Radiomic features can improve quantification, differentiation
between calcified and non-calcified plaques, and thus the
prediction of Major Adverse Cardiac Events (MACE).
REVIEWS
NATURE REVIEWS
|
CLINICAL ONCOLOGY VOLUME 14
|
DECEMBER 2017
|
751
©
2
0
1
7
M
a
c
m
i
l
l
a
n
P
u
b
l
i
s
h
e
r
s
L
i
m
i
t
e
d
,
p
a
r
t
o
f
S
p
r
i
n
g
e
r
N
a
t
u
r
e
.
A
l
l
r
i
g
h
t
s
r
e
s
e
r
v
e
d
.
©
2
0
1
7
M
a
c
m
i
l
l
a
n
P
u
b
l
i
s
h
e
r
s
L
i
m
i
t
e
d
,
p
a
r
t
o
f
S
p
r
i
n
g
e
r
N
a
t
u
r
e
.
A
l
l
r
i
g
h
t
s
r
e
s
e
r
v
e
d
.
Calibration-in-the-large
Describes whether the
predictions deviate
systematically (intercept),
whereas the calibration slope
should ideally be equal to 1.
The independence
assumption
The definition in terms of
conditional probabilities is that
the probability of B is not
changed by knowing that A has
occurred. Statistically
independent variables are
always uncorrelated, but the
converse is not necessarily
true.
Feature discretization
The process of converting
continuous features to discrete
binned interval features.
Bootstrapping
Measures the accuracy
(defined in terms of bias,
variance, confidence intervals,
prediction error, etc.) to
characterize the sample
distribution by way of repeated
random sampling methods.
of limited value; validation is an indispensable compo-
nent of a complete radiomic analysis. Models must be
internally validated and, ideally, should be externally
validated.
Feature selection. Depending on the number of filters,
feature categories, and other adjustable parameters, the
possible number of radiomic features that can be extracted
from images is virtually unlimited. The inclusion of
all possible features in a model would inevitably result
in overfitting, which jeopardizes model performance in
patients not previously evaluated. To avoid overfitting,
features that lack robustness against sources of variability
should be eliminated, and archetypal features selected via
dimensionality reduction techniques (such as principal
component analysis or clustering). For example, a feature
that is archetypal for the prediction of overall survival
in patients with lung cancer for a given dataset (imaged
and segmented in a certain way) could be redundant for
the prediction of pneumonitis in lung cancer for a given
dataset (imaged and segmented in an alternative way).
Modelling methodology. The modelling methodology
chosen is often a single technique, selected according to
the preference and experience of those conducting the
study. Different techniques are associated with distinct
inherent limitations, which include the independence
assumption for features in logistic regression, the need for
feature discretization in Bayesian networks, or the network
configuration dependency in deep learning. The choice of
modelling technique has been shown to affect prediction
performance in radiomics
30
. Thus, multiple- modelling
methodology implementations are desirable, but not
essential. The key aspect in the selection of a modelling
methodology is that, when reported, the work must be
entirely reproducible. This goal can be achieved, ideally,
by making the software code available (for example,
via github
31
). (See Supplementary information S1 material
for an overview of machine learning techniques).
Validation. Validation techniques are useful tools
to assess model performance, and thus, internal
and/or preferably external validation must be performed.
Researchers must assess whether the model is predic-
tive for the target patient population or just for a par-
ticular subset of samples analysed. Model performance
is typically measured in terms of discrimination and
calibration. Discrimination can be reported in terms of
the receiver operating characteristic (ROC) curve, or the
area under the ROC curve (AUC). The AUC quantifies
the sensitivity and specificity of the model and repre-
sents the probability that a randomly selected patient
matching an outcome is assigned that outcome by the
prediction model with a larger event-probability than
a randomly chosen patient who does not match the
outcome. Calibration refers to the agreement between
observed outcomes and model predictions, typically
based on grouping of predictions. For example, the
predictions are grouped according to high, medium or
low prob ability. If the mean prediction of tumour recur-
rence in the high-probability group is 25%, the observed
frequency of tumour recurrence in this group should ide-
ally be 25 out of 100 patients. Calibration can be reported
using a calibration plot and calibration-in-the-large/slope.
A measure of overall performance is the Brier score,
the mean squared prediction error. All statistical meth-
ods should be reported for training data and validation
data. Valid models should exhibit statistical consistency
between the training and validation sets. Bootstrapping
techniques can be used to estimate confidence intervals
for the abovementioned statistics and should be reported.
An externally validated model has more credibility than
an internally validated model, because data obtained with
the former approach are considered more independent,
which reinforces the validation. A large body of literature
on validation techniques is available
32–35
.
Reporting open-access scientific data
Validation is the first step towards a model being
accepted in both the scientific and clinical communi-
ties. Independent verification of the results is a necessary
additional step. Reproduction means verification of the
results by independent researchers repeating the ana-
lysis using an identical technique and the same dataset
and/or patient cohort, ensuring that the analysis is con-
ducted without error. Replication means independent
verification of the results by independent researchers
repeating the analysis using the same technique and
different (but appropriately selected) datasets and/or
patient cohorts, aiming for a stronger affirmation of the
findings
36–39
. Radiomic studies involve multiple complex
subprocesses (such as data selection, image acquisition,
feature extraction, or modelling), each one affected by a
wide range of decisions, use of nonstandardized terminol-
ogy, establishment of parameters, and software selection.
Reproducibility and replicability in radiomics are impos-
sible if researchers do not disclose these intricacies. The
amount of necessary information far exceeds the limits
of a traditional manuscript. We propose that future pub-
lications including radiomic results should provide the
following as supplementary material: disclosure of imag-
ing protocols, analysed scans, segmentations of VOIs,
detailed accounts of how features were extracted (includ-
ing the formulae), and of the modelling metho dology
used (ideally, the code). This level of meticulous detail is
required in order to facilitate reproduction and replica-
tion. Furthermore, multiple radiomics software packages
are available and are subject to updates or version-control.
We recognize that the publication of data derived from
patients might not be feasible in all circumstances. As a
minimal means of comparison, and to alleviate this lack of
transparency, we propose that researchers publish numer-
ical values of their investigated features computed on the
digital phantom described in the supplementary material
of this manuscript (available online
40
).
To compare different software implementations for
radiomic feature-extraction algorithms, we present an
example, in which CT-obtained data of the primary
tumour region and the corresponding tumour contours
of four patients with lung cancer serve as ‘real-life’ digital
phantoms (FIG.3). Using the preprocessed image data, we
calculated a set of commonly used features to serve as a
REVIEWS
752
|
DECEMBER 2017
|
VOLUME 14 www.nature.com/nrclinonc
©
2
0
1
7
M
a
c
m
i
l
l
a
n
P
u
b
l
i
s
h
e
r
s
L
i
m
i
t
e
d
,
p
a
r
t
o
f
S
p
r
i
n
g
e
r
N
a
t
u
r
e
.
A
l
l
r
i
g
h
t
s
r
e
s
e
r
v
e
d
.
©
2
0
1
7
M
a
c
m
i
l
l
a
n
P
u
b
l
i
s
h
e
r
s
L
i
m
i
t
e
d
,
p
a
r
t
o
f
S
p
r
i
n
g
e
r
N
a
t
u
r
e
.
A
l
l
r
i
g
h
t
s
r
e
s
e
r
v
e
d
.