A Comparison of the Ability of the Physiologic Components of Medical Emergency Team Criteria and the U.K. National Early Warning Score to Discriminate Patients at Risk of a Range of Adverse Clinical Outcomes

TLDR: Comparing the ability of medical emergency team criteria and the National Early Warning Score to discriminate cardiac arrest, unanticipated ICU admission and death within 24 hours of a vital signs measurement and to quantify the associated workloads found some medical emergencyteam systems have a lower specificity and would generate greater workloads.

Abstract: To compare the ability of medical emergency team criteria and the National Early Warning Score to discriminate cardiac arrest, unanticipated ICU admission and death within 24 hours of a vital signs measurement, and to quantify the associated workload. Retrospective cohort study. A large U.K. National Health Service District General Hospital. Adults hospitalized from May 25, 2011, to December 31, 2013. None. We applied the National Early Warning Score and 44 sets of medical emergency team criteria to a database of 2,245,778 vital signs sets (103,998 admissions). The National Early Warning Score's performance was assessed using the area under the receiver-operating characteristic curve and compared with sensitivity/specificity for different medical emergency team criteria. Area under the receiver-operating characteristic curve (95% CI) for the National Early Warning Score for the combined outcome (i.e., death, cardiac arrest, or unanticipated ICU admission) was 0.88 (0.88-0.88). A National Early Warning Score value of 7 had sensitivity/specificity values of 44.5% and 97.4%, respectively. For the 44 sets of medical emergency team criteria studied, sensitivity ranged from 19.6% to 71.2% and specificity from 71.5% to 98.5%. For all outcomes, the position of the National Early Warning Score receiver-operating characteristic curve was above and to the left of all medical emergency team criteria points, indicating better discrimination. Similarly, the positions of all medical emergency team criteria points were above and to the left of the National Early Warning Score efficiency curve, indicating higher workloads (trigger rates). When medical emergency team systems are compared to a National Early Warning Score value of greater than or equal to 7, some medical emergency team systems have a higher sensitivity than National Early Warning Score values of greater than or equal to 7. However, all of these medical emergency team systems have a lower specificity and would generate greater workloads.

Figures

Table 1: Patient demographics, number of vital signs observations, vital sign values and observations followed by an adverse outcome for the study group (after exclusions).

Table 2: The sensitivity and specificity of 44 MET calling criteria, and NEWS values of 3-7, for cardiac arrest, unanticipated ICU admission, death, and any of these outcomes, each within 24 h of a vital signs dataset.

Full text

This is a repository copy of A Comparison of the Ability of the Physiologic Components of
Medical Emergency Team Criteria and the U.K. National Early Warning Score to
Discriminate Patients at Risk of a Range of Adverse Clinical Outcomes.
White Rose Research Online URL for this paper:
https://eprints.whiterose.ac.uk/104932/
Version: Accepted Version
Article:
Smith, Gary B., Prytherch, David R, Jarvis, Stuart William orcid.org/0000-0001-8447-0306
et al. (4 more authors) (2016) A Comparison of the Ability of the Physiologic Components
of Medical Emergency Team Criteria and the U.K. National Early Warning Score to
Discriminate Patients at Risk of a Range of Adverse Clinical Outcomes. Critical Care
Medicine. 2171–2181. ISSN 0090-3493
https://doi.org/10.1097/CCM.0000000000002000
eprints@whiterose.ac.uk
https://eprints.whiterose.ac.uk/
Reuse
Items deposited in White Rose Research Online are protected by copyright, with all rights reserved unless
indicated otherwise. They may be downloaded and/or printed for private study, or other acts as permitted by
national copyright laws. The publisher or other rights holders may allow further reproduction and re-use of
the full text version. This is indicated by the licence information on the White Rose Research Online record
for the item.
Takedown
If you consider content in White Rose Research Online to be in breach of UK law, please notify us by
emailing eprints@whiterose.ac.uk including the URL of the record and the reason for the withdrawal request.

1
A comparison of the ability of the physiological components of Medical Emergency Team criteria and
the UK National Early Warning Score (NEWS) to discriminate patients at risk of a range of adverse
clinical outcomes
Professor Gary B Smith, FRCA, FRCP
1
Professor David R Prytherch, PhD, MIPEM, CSci
2,3
Dr. Stuart Jarvis PhD.
3,4
Mrs. Caroline Kovacs, BSc
3
Dr. Paul Meredith, PhD
2
Dr. Paul E Schmidt, MRCP, B.Med.Sc, MBA
2
Dr. Jim Briggs, BA, DPhil
3
1
Faculty of Health and Social Sciences, University of Bournemouth, Bournemouth, UK
2
Portsmouth Hospitals NHS Trust, Portsmouth, UK
3
Centre for Healthcare Modelling & Informatics, University of Portsmouth, Portsmouth, UK
4
Department of Health Sciences, University of York, York, UK
Work performed at Portsmouth Hospitals NHS Trust & University of Portsmouth
No reprints will be available
Correspondence from:
Professor G B Smith, FRCA, FRCP,
Centre of Postgraduate Medical Research & Education (CoPMRE),
Faculty of Health and Social Sciences,
Bournemouth University, Royal London House,
Christchurch Road, Bournemouth,
Dorset BH1 3LT, United Kingdom
Tel: +44 (0) 1202 962782; Fax: +44 (0) 1202 962218
Email: gbsresearch@virginmedia.com
Funding: Nil
Key words: Hospital rapid response team; Monitoring, Physiologic; Quality improvement; Vital Signs,
Medical Emergency Team.
Word count = 2998. Abstract word count = 248
Number of references = 37; Figures = 3; Tables = 2; Supplementary Digital Content = 10

2
ABSTRACT
Objective: To compare the ability of Medical Emergency Team (MET) criteria and the National Early
Warning Score (NEWS) to discriminate cardiac arrest, unanticipated ICU admission and death within 24 h of
a vital signs measurement, and to quantify the associated workload.
Design: Retrospective cohort study.
Setting: A large UK NHS District General Hospital.
Patients: Adults hospitalized from 25/05/2011 to 31/12/2013.
Interventions: None
Measurements and Main Results: We applied NEWS and 44 sets of MET criteria to a database of 2,245,778
vital signs sets (103,998 admissions). NEWS’ performance was assessed using the area under the receiver-
operating characteristic (ROC) curve (AUROC) and compared with sensitivity/specificity for the different MET
criteria. AUROC (95% CI) for NEWS for the combined outcome (i.e., death, cardiac arrest or unanticipated
ICU admission) was 0.88 (0.88 - 0.88). A NEWS value of 7 had sensitivity/specificity values of 44.5%/97.4%.
For the 44 sets of MET criteria studied, sensitivity ranged from 19.6% to 71.2%, and specificity from 71.5% to
98.5%. For all outcomes, the position of the NEWS ROC curve was above and to the left of all MET criteria
points, indicating better discrimination. Similarly, the positions of all MET criteria points were above and to the
left of the NEWS efficiency curve, indicating higher workloads (trigger rates).
Conclusions: When MET systems are compared to a NEWS value of >7, some MET systems have a higher
sensitivity than NEWS values of >7. However, all of these MET systems have a lower specificity and would
generate greater workloads.

3
CONFLICT OF INTERESTS STATEMENT
VitalPAC is a collaborative development of The Learning Clinic Ltd (TLC) and Portsmouth Hospitals NHS
Trust (PHT). At the time of the research, PHT had a royalty agreement with TLC to pay for the use of PHT
intellectual property within the VitalPAC product. Professor Prytherch, Dr Schmidt, and Dr Meredith are
employed by PHT. Professor Smith was an employee of PHT until 31/03/2011. Professors Smith and
Prytherch, and Dr Schmidt, are unpaid research advisors to TLC and have received reimbursement of travel
expenses from TLC for attending symposia in the UK. Dr Briggs' research has previously received funding
from TLC through a Knowledge Transfer Partnership. Professor Smith acted as expert advisor to the
National Institute for Health and Clinical Excellence during the development of the NICE clinical guideline 50:
'Acutely ill patients in hospital: recognition of and response to acute illness in adults in hospital'. He was also
a member of the National Patient Safety Agency committee that wrote the two reports: 'Recognising and
responding appropriately to early signs of deterioration in hospitalised patients' and ' Safer care for the
acutely ill patient: learning from serious incidents'. He was a member of the Royal College of Physicians
of
London’s National Early Warning Score Development and Implementation Group (NEWSDIG). Professor
Prytherch assisted the Royal College of Physicians of London in the analysis of data validating NEWS. Dr
Jarvis and Mrs Kovacs have no conflicts of interest.

4
INTRODUCTION
Staff failures in recognising and responding to patient deterioration have led hospitals to use early
warning scoring systems (EWSS) (1) or Medical Emergency Team (MET) calling criteria (2) to improve vital
signs monitoring and facilitate the calling of expert help to a patient’s bedside.
EWSS allocate points in a weighted manner, based on the derangement of a patient’s measured
vital signs from arbitrarily agreed “normal” ranges - the sum of these is termed the early warning score
(EWS). The EWS is used to direct subsequent care, e.g. changes to vital signs monitoring frequency;
involvement of more experienced ward staff; or calling a rapid response team (RRT). Many EWSS are in
use, with marked variation in measured physiological variables, assigned weightings and outcome
discrimination (3-8). In 2012, the Royal College of Physicians of London (RCPL) recommended the use of a
standardised EWSS in the National Health Service (NHS) - the National EWS (NEWS) (Supplementary
Digital Content 1) (9)
. To produce NEWS, the RCPL used clinical opinion to make minor adjustments to the
VitalPAC Early Warning Score (ViEWS) (5). The RCPL recommends that NEWS values of >7 should prompt
assessment by an RRT (9). NEWS demonstrates better ability than other published EWSS to discriminate
patients at risk of a range of clinical outcomes (6) and has been validated outside its development site (10-
13).
Some hospitals, especially in the USA and Australia, use MET calling criteria in preference to EWSS.
Most MET criteria are based on extreme values of specific objective physiological parameters (e.g., pulse
rate <40 or >120 beats.min
−1
) (2) (Supplementary Digital Content 2). When one or more objective MET
criteria occurs, or staff are ‘worried’ about a patient, a MET or other RRT is called to provide expert
assistance (14). As with EWSS, a wide range of MET criteria is in use, with varied abilities to discriminate
patients at risk of adverse events (3, 15-17).
Ideally, hospitals should use an RRT triggering system that provides the highest discrimination of
patient outcome and the lowest trigger rate, thereby minimising both the risk of missing serious outcomes
and of excessive staff workload. A recent study comparing the performances of NEWS and one set of MET
criteria suggests that NEWS is a better (and earlier) detector of patient deterioration (13). Therefore, we used
a large database of vital sign measurements to (a) compare the abilities of NEWS and 44 different MET

Frequently asked questions

Q1. What have the authors contributed in "This is a repository copy of a comparison of the ability of the physiologic components of medical emergency team criteria and the u.k. national early warning score to discriminate patients at risk of a range of adverse clinical outcomes. white rose research online url for this paper: https://eprints.whiterose.ac.uk/104932/" ?

This is indicated by the licence information on the White Rose Research Online record for the item.