LETTER TO THE EDITOR
Clinical prediction models
Hendrik-Jan Mijderwijk
1
& Thomas Beez
1
& Daniel Hänggi
1
& Daan Nieboer
2
Received: 27 February 2020 /Accepted: 6 March 2020
#
The Author(s) 2020
Dear Editor:
The number of articles presenting a neurosurgical prediction
model is rapidly increasing [1]. Although the number of pub-
lications reporting a clinical prediction model in Child’s
Nervous System, Journal of Neurosurgery Pediatrics,and
Pediatric Neurosurgery is relatively constant over the years
(Fig. 1a), circa two-thirds of these publications have been
published since 2015 (Fig. 1b). Nowadays, with the rise of
so many prediction models, we should be able to make firm
conclusions which model to use on our patients.
Clinical prediction models aim to predict an outcome of
interest, for example, survival in high-grade glioma (HGG)
patients or intraventricula r hemorrhage in preterm infants, by
combining two or more patient-related variables. The obtained
predictions of these models can then be used for medical and
shared decision-making such as initiating surgical treatment,
and for example for guidance in planning future lifestyle.
The developm ent and evaluation of clinical predi ction
models involve multiple methodological steps. It is well-
known that these steps are often inadequately addressed and/
or inadequately reported in a publication which clearly limits
the usefulness of the presented prediction model. Utilizing
invalid prediction models may jeopardize adequate decision-
making in our daily clinical practice. Therefore, we want to
point out a few crucial aspects of prediction models.
1. Sample size—In clinical prediction model studies, the
number of events in the study population defines the ef-
fective sample size. As a rule of thumb, the minimum
required number of 10 events per prognostic variable
considered has been generally accepted; although recent-
ly,moreadvancedcalculationshavebeenproposedwhich
may yield a different ratio. The number of events is the
smaller of the number of patients having the event or not
having the event. Thus, if in a set of 110 HGG patients, 50
HGG patients died during the study period, considering
up to 5 prognostic variables seems reasonable for the de-
velopment of a prediction model. Considering too many
prognostic variables increases the risk of overfitting [2].
Overfitted models show promising results when evaluated
on the patients on which the model was developed but
show disappointing results when applied to other sets of
patients. This point is essential because many pediatric
neurosurgical studies have relatively low effective sample
sizes.
2. Validation—It is vital to gauge the validity of the predic-
tions provided by a prediction model. As a result of
overfitting, prediction models tend to have a too optimis-
tic predictive performance in terms of discrimination and
calibration [2]. Internal validation aims to quantify this
optimism. These techniques reuse the same set of patients
on which the model was developed. This step is a mini-
mum requirement for publication of a prediction model.
External validation assesses the performance of the pre-
diction model on a different set of patients, for example,
collected at other geographical locations and/or time pe-
riods. External validation is imperative before a clinical
uptake of the proposed model can take place.
3. Reporting—Guidelines for reporting prediction models
do exist. The TRIPOD (transparent reporting of a multi-
variable prediction model for individual prognosis or di-
agnosis) guideline together with its e xplanatory paper
provides crucial information for the design, conduction,
and evaluation of a prediction model [3, 4]. It is highly
recommended to adhere to the items of the TRIPOD
checklist for proper and transparent reporting. Often
graphical presentations of the underlying statistical model
are provided in a publication, but (statistical) details are
lacking. For example, when Cox regression is used for
predicting survival, the regression coefficients and the
* Hendrik-Jan Mijderwijk
Hendrik-Jan.Mijderwijk@med.uni-duesseldorf.de
1
Department of Neurosurgery, Heinrich-Heine University Medical
Center, Moorenstraße 5, 40225 Düsseldorf, Germany
2
Department of Public Health, Erasmus MC, University Medical
Center Rotterdam, ‘s Gravendijkwal 230, 3015
CE Rotterdam, The Netherlands
Child's Nervous System
https://doi.org/10.1007/s00381-020-04577-8
baseline survival at a given timepoint should be reported
to enable external validation.
We encourage the journal and its readers to critically
review studies on prediction models. Published work does
not always provide sufficient details of the model, al-
though this is necessary to judge the quality of the pre-
diction model. More details of the presented methodolog-
ical concerns and others including the selection of candi-
date prognostic variables and evaluating model perfor-
mance measures have been recently published and illus-
trated with clinical examples specifically for neurosur-
geons [1]. Furthermore, 7 key steps for the development
and evaluation of a neurosurgical clinical prediction mod-
el are tabulated for a quick overview. Pediatric neurosur-
geons should ideally be aware of this methodology as it is
highly con s equ ent ia l t o our daily clinica l pr ac ti ce .
Acknowledgments We thank Maarten F.M. Engel, an information spe-
cialist at Erasmus MC, for performing the literature search.
Funding Information Open Access funding provided by Projekt DEAL.
Compliance with ethical standards
Conflict of interest On behalf of all authors, the corresponding author
states that there is no conflict of interest.
2000 2005 2010 2015 2020
0
10
20
30
40
Year of publication
Number of publications
Total Pediatric
Neurosurgical Clinical
Prediction Models
Clinical Prediction Models
published in Child's Nervous
System
Clinical Prediction Models
published in Journal of
Neurosurgery Pediatrics
Clinical Prediction Models
published in Pediatric
Neurosurgery
2000 2005 2010 2015 2020
0
10
20
30
40
Year of publication
Number of publications
Total Pediatric
Neurosurgical Clinical
Prediction Models
Total Pediatric
Neurosurgical Clinical
Prediction Models
Nervous System, Journal
of Neurosurgery
Pediatrics, and Pediatric
Neurosurgery
a
b
Fig. 1 a, b Time trends of the published number of pediatric neurosurgical
clinical prediction models according to the following systematic searches in
PubMed until December 31, 2019. For the black results: (Prognostic-
index* OR Prognostic-rule* OR Prognostic-model* OR prognostic-scor*
OR prediction-index* OR prediction-rule* OR prediction-model* OR
prediction-scor* OR predictive-index* OR predictive-rule* OR
predictive-model* OR predictive-scor*) AND (Neurosurgery [mh] OR
Neurosurgical Procedures [mh] OR Neurosurg* OR Neurological-surg*)
AND (“child”[mh] OR “Infant” [mh] OR “Adolescent”[mh] OR
“Minors”[mh] OR “Pediatrics”[mh] OR “Child Health Services”[mh] OR
“Hospitals, Pediatric”[mh] OR “Intensive Care Units, Pediatric”[Mesh] OR
infan*[tiab] OR newborn*[tiab] OR new born*[tiab] OR baby [tiab] OR
babies [tiab] OR neonat* [tiab] OR perinat*[tia b] OR postnat*[t iab] OR
prematur*[tiab] OR pre-matur*[tiab] OR child [mesh] OR child [tiab] OR
child’s [tiab] OR childhood*[tiab] OR children*[tiab] OR kid [tiab] OR
kids [tiab] OR toddler*[tiab] OR adoles*[tiab ] OR teen*[tiab] OR
boy*[tiab] OR girl*[tiab] OR minors*[tiab] OR underag*[tiab] OR under
age*[tiab] OR under aging [tiab] OR under ageing [tiab] OR juvenil*[tiab]
OR youth*[tiab] OR kindergar*[tiab] OR puber*[tiab] OR pubescen*[tiab]
OR prepubescen*[tiab] OR prepuberty*[tiab] OR pediatric*[tiab] OR
peadiatric*[tiab] OR schoolchil d*[tiab] OR preschool*[tiab ] OR
highschool*[tiab] OR suckling*[tiab] OR PICU [tiab] OR N ICU [tiab]
OR PICUs [tiab] OR NICUs [tiab]). For the colored results: (Prognostic-
index* OR Prognostic-rule* OR Prognostic-model* OR prognostic-scor*
OR prediction-index* OR prediction-rule* OR prediction-model* OR
prediction-scor* OR predictive-index* OR predictive-rule* OR
predictive-model* OR predictive-scor*) AND (Neurosurgery [mh] OR
Neurosurgical Procedures [mh] OR Neurosurg* OR Neurological-surg*)
AND (“Childs Nerv Syst”[Journal] OR “Journal of Neurosurgery
Pediatrics”[Journal] OR “Pediatric Neurosurgery”[Journal])
Childs Nerv Syst
Open Access This article is licensed under a Creative Commons
Attribution 4.0 International License, which permits use, sharing, adap-
tation, distribution and reproduction in any medium or format, as long as
you give appropriate credit to the original author(s) and the source, pro-
vide a link to the Creative Commons licence, and indicate if changes were
made. The images or other third party material in this article are included
in the article's Creative Commons licence, unless indicated otherwise in a
credit line to the material. If material is not included in the article's
Creative Commons licence and your intended use is not permitted by
statutory regulation or exceeds the permitted use, you will need to obtain
permission directly from the copyright holder. To view a copy of this
licence, visit http://creativecommons.org/licenses/by/4.0/.
References
1. Mijderwijk HJ, Steyerberg EW, Steiger HJ, Fischer I, Kamp MA
(2019) Fundamentals of clinical prediction modeling for the
neurosurgeon. Neurosurgery 85:302–311. https://doi.org/10.1093/
neuros/nyz282
2. Steyerberg EW (2019) Clinical prediction models. A practical ap-
proach to development, validation, and updating. Springer, New
York. https://doi.org/10.1007/978-3-030-16399-0
3. Coll ins GS, Reitsma JB, Altman DG, Moons KGM (2015)
Transparent reporting of a multivariable prediction model for indi-
vidual prognosis or diagnosis (TRIPOD): the TRIPOD statement.
BMC Med 13:1–10. https://doi.org/10.1186/s12916-014-0241-z
4. Moons KGM, Altman DG, Reitsma JB, Ioannidis JPA, Macaskill P,
Steyerberg EW, Vickers AJ, Ransohoff DF, Collins GS (2015)
Transparent reporting of a multivariable prediction model for indi-
vidual prognosis or diagnosis (TRIPOD): explanation and elabora-
tion. Ann Intern Med 162:W1–W73. https://doi.org/10.7326/M14-
0698
Publisher’snoteSpringer Nature remains neutral with regard to jurisdic-
tional claims in published maps and institutional affiliations.
Childs Nerv Syst
![Fig. 1 a, b Time trends of the published number of pediatric neurosurgical clinical prediction models according to the following systematic searches in PubMed until December 31, 2019. For the black results: (Prognosticindex* OR Prognostic-rule* OR Prognostic-model* OR prognostic-scor* OR prediction-index* OR prediction-rule* OR prediction-model* OR prediction-scor* OR predictive-index* OR predictive-rule* OR predictive-model* OR predictive-scor*) AND (Neurosurgery [mh] OR Neurosurgical Procedures [mh] OR Neurosurg* OR Neurological-surg*) AND (“child”[mh] OR “Infant”[mh] OR “Adolescent”[mh] OR “Minors”[mh] OR “Pediatrics”[mh] OR “Child Health Services”[mh] OR “Hospitals, Pediatric”[mh] OR “Intensive Care Units, Pediatric”[Mesh] OR infan*[tiab] OR newborn*[tiab] OR new born*[tiab] OR baby [tiab] OR babies [tiab] OR neonat*[tiab] OR perinat*[tiab] OR postnat*[tiab] OR prematur*[tiab] OR pre-matur*[tiab] OR child [mesh] OR child [tiab] OR child’s [tiab] OR childhood*[tiab] OR children*[tiab] OR kid [tiab] OR](/figures/fig-1-a-b-time-trends-of-the-published-number-of-pediatric-2y4t942q.webp)