A Comparison of the Ability of the Physiologic Components of Medical Emergency Team Criteria and the U.K. National Early Warning Score to Discriminate Patients at Risk of a Range of Adverse Clinical Outcomes
Summary (3 min read)
INTRODUCTION
- Staff failures in recognising and responding to patient deterioration have led hospitals to use early warning scoring systems (EWSS) (1) or Medical Emergency Team (MET) calling criteria (2) to improve vital signs monitoring and facilitate the calling of expert help to a patient’s bedside.
- The EWS is used to direct subsequent care, e.g. changes to vital signs monitoring frequency; involvement of more experienced ward staff; or calling a rapid response team (RRT).
- Many EWSS are in use, with marked variation in measured physiological variables, assigned weightings and outcome discrimination (3-8).
- To produce NEWS, the RCPL used clinical opinion to make minor adjustments to the VitalPAC Early Warning Score (5).
Setting and study population
- Portsmouth Hospitals NHS Trust (PHT) is a single site NHS District General Hospital with ~1000 inpatient beds and ~5500 staff.
- It provides all acute services except burns, spinal injury, neurosurgical and cardiothoracic surgery to a local population of ~540,000.
- Data from patients discharged alive from the hospital before midnight on the day of admission were excluded.
- Where oxygen was used, VitalPAC estimated its fractional inspired concentration (FiO2) using the mask type +/- flow rate (or in the case of a Venturi mask, the concentration), which were recorded during each vital signs collection.
Evaluation of NEWS and MET criteria
- The vital signs database was used to evaluate the performance of NEWS and 44 different MET criteria (identified from two previous publications (16, 17) - see Supplementary Digital Content 3).
- As the subjective component of MET criteria - staff concern (14) – is also used to escalate care when using NEWS, the authors made an a priori decision to evaluate only the following physiological components of the MET criteria: 7 high/low pulse rate, high/low breathing rate, high/low systolic blood pressure, high/low temperature, SpO2 and reduced consciousness.
- For the same reason, the authors did not evaluate criteria such as threatened airway or repeated/prolonged seizures.
- The remainder require only an SpO2 value or simply whether supplemental O2 was being administered.
- The authors removed hospital episodes where FiO2 could not be estimated.
Outcomes
- Deaths, cardiac arrests and unanticipated ICU admission data were identified from the hospital’s patient administration system (PAS), cardiac arrest database and ICU admission database, respectively.
- The authors excluded episodes of care where (i) the episode had a first outcome before the first observation set and (ii) the episode did not have an observation set within the last 24 h before the outcome.
Statistical analysis
- All data manipulation was performed using Microsoft® Visual FoxPro 9.0.
- The authors used IBM SPSS Statistics v22 and R v3.02. (22) to calculate the AUROC; R was also used to generate the figures.
- An ROC curve plots sensitivity against 1-specificity, and each point on it represents a sensitivity/specificity pairing corresponding to a particular decision threshold for NEWS.
- For each set of MET criteria, and for each outcome, the authors calculated the sensitivity and specificity.
- To compare the efficiency of NEWS and the MET criteria, the authors superimposed the sensitivity/trigger rate points for the 44 sets of MET criteria on the NEWS efficiency curves.
Additional analyses
- The authors have previously shown that the use of multiple observation sets from a single episode does not bias the ranking of EWSs when assessing the performance of these systems (24).
- This has not previously been done for sets of MET criteria.
- Therefore, the authors repeated the above analyses using 10,000 samples of observation sets, each sample being constructed by selecting one observation set at random from every care episode (i.e., so each observation set in an episode had an equal chance of being selected in each sample).
RESULTS
- For some of these 5809 episodes there were other observation sets where FiO2 could be estimated, so the episode itself remained in the analysis (with fewer observation sets).
- Figures 2a-d and Supplementary Digital Content 5 show the sensitivity and specificity (plotted as 1 - specificity) points for NEWS (i.e., the NEWS ROC curve) and the MET criteria for the outcomes studied.
- The relative positions of the MET criteria and NEWS were essentially unchanged when using the 10,000 random sample sets (see Supplementary Digital Content 9 and 10).
DISCUSSION
- The selection of an RRT triggering system can be based upon several criteria, including the balance between its sensitivity and the workload it generates.
- Depending upon their specific criteria, all sets of MET calling criteria have fixed relationships between sensitivity and workload, and the resulting clinical response can only ever be of an ‘all or none’ nature.
- That EWSs, such as NEWS, are better discriminators of outcomes than MET criteria is perhaps not surprising.
- It considers all completed admissions over 31 months.
- The current study differs markedly from the NEWS development work, using a larger database, a different study period, medical and surgical patients (compared to only acute medicine) and vital signs from the whole patient admission rather than merely from the patient’s stay in the Medical Assessment Unit.
CONCLUSIONS
- When MET systems are compared to a NEWS value of >7 (i.e., the recommended triggers for RRT intervention for each system), some MET systems have a higher sensitivity than NEWS.
- All of these MET systems have a lower specificity and would generate greater workloads.
- NEWS also provides the opportunity to titrate the trigger value against available resources, and permits a graduated, multi-tiered clinical response, whereas the clinical response resulting from a MET call can only ever be of an ‘all or none’ nature.
Did you find this useful? Give us your feedback
Citations
3 citations
3 citations
2 citations
Cites background or result from "A Comparison of the Ability of the ..."
...35 Aims These main findings from the literature indicate a positive effect on reducing SAEs with the use of an RRS....
[...]
...37 Methods Designs and settings Studies I-III were, except study I’s translation and cultural adaptation, primarily focused on SAEs (ICU-admission, mortality and IHCA) (Figure 8)....
[...]
...Serious adverse events in Sweden Corresponding figures for SAEs in Swedish hospital settings are largely unreported....
[...]
...The patients scoring medium-risk on the NEWS seem to be a complicated group to assess and many of them will actually develop into SAEs....
[...]
...The construct validity of the NEWS could be translated into the relation between SAEs and the NEWS with its physiological parameters....
[...]
2 citations
Cites background from "A Comparison of the Ability of the ..."
...High NEWS scores are associated with more severe disease and poorer outcomes [8,9]....
[...]
2 citations
References
19,398 citations
1,423 citations
1,229 citations
749 citations
"A Comparison of the Ability of the ..." refers background in this paper
...NEWS demonstrates better ability than other published EWSS to discriminate patients at risk of a range of clinical outcomes (6) and has been validated outside its development site (10–13)....
[...]
...The study data were collected from 66,712 unique patients admitted to medicine (34,204), surgery (33,808), and other specialties (6,441)....
[...]
495 citations
"A Comparison of the Ability of the ..." refers background or methods in this paper
...A further weakness is that the study was conducted in a single site, where the precursor of NEWS—ViEWS (5)— was developed....
[...]
...We also plotted an efficiency curve (5) for NEWS for each outcome....
[...]
...To produce NEWS, the RCPL used clinical opinion to make minor adjustments to the VitalPAC EWS (ViEWS) (5)....
[...]