An emphasis on overly broad notions of generalisability as it pertains to applications of machine learning in health care can overlook situations in which machine learning might provide clinical utility. We believe that this narrow focus on generalisability should be replaced with wider considerations for the ultimate goal of building machine learning systems that are useful at the bedside.

The myth of generalisability in clinical research and machine learning in health care.

Respiratory rate is a fundamental vital sign that is sensitive to different pathological conditions (e.g., adverse cardiac events, pneumonia, and clinical deterioration) and stressors, including emotional stress, cognitive load, heat, cold, physical effort, and exercise-induced fatigue. The sensitivity of respiratory rate to these conditions is superior compared to that of most of the other vital signs, and the abundance of suitable technological solutions measuring respiratory rate has important implications for healthcare, occupational settings, and sport. However, respiratory rate is still too often not routinely monitored in these fields of use. This review presents a multidisciplinary approach to respiratory monitoring, with the aim to improve the development and efficacy of respiratory monitoring services. We have identified thirteen monitoring goals where the use of the respiratory rate is invaluable, and for each of them we have described suitable sensors and techniques to monitor respiratory rate in specific measurement scenarios. We have also provided a physiological rationale corroborating the importance of respiratory rate monitoring and an original multidisciplinary framework for the development of respiratory monitoring services. This review is expected to advance the field of respiratory monitoring and favor synergies between different disciplines to accomplish this goal.

https://www.mdpi.com/1424-8220/20/21/6396/pdf

The Importance of Respiratory Rate Monitoring: From Healthcare to Sport and Exercise.

Summary Machine learning is increasingly integrated into clinical practice, with applications ranging from pre-clinical data processing, bedside diagnosis assistance, patient stratification, treatment decision making, and early warning as part of primary and secondary prevention. However, a multitude of technological, medical, and ethical considerations are critical in machine-learning utilization, including the necessity for careful validation of machine-learning-based technologies in real-life contexts, unbiased evaluation of benefits and risks, and avoidance of technological over-dependence and associated loss of clinical, ethical, and social-related decision-making capacities. Other challenges include the need for careful benchmarking and external validations, dissemination of end-user knowledge from computational experts to field users, and responsible code and data sharing, enabling transparent assessment of pipelines. In this review, we highlight key promises and achievements in integration of machine-learning platforms into clinical medicine while highlighting limitations, pitfalls, and challenges toward enhanced integration of learning systems into the medical realm.

Machine learning in clinical decision making

Coronavirus disease 2019 (COVID-19) is a respiratory disease with rapid human-to-human transmission caused by the severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2). Due to the exponential growth of infections, identifying patients with the highest mortality risk early is critical to enable effective intervention and prioritisation of care. Here, we present the COVID-19 early warning system (CovEWS), a risk scoring system for assessing COVID-19 related mortality risk that we developed using data amounting to a total of over 2863 years of observation time from a cohort of 66 430 patients seen at over 69 healthcare institutions. On an external cohort of 5005 patients, CovEWS predicts mortality from 78.8% (95% confidence interval [CI]: 76.0, 84.7%) to 69.4% (95% CI: 57.6, 75.2%) specificity at sensitivities greater than 95% between, respectively, 1 and 192 h prior to mortality events. CovEWS could enable earlier intervention, and may therefore help in preventing or mitigating COVID-19 related mortality. Identifying COVID-19 patients with the highest mortality risk early is critical to enable effective intervention and optimal prioritisation of care. Here, the authors present a clinical risk scoring system trained on a large data set of patients from 69 healthcare institutions in multiple countries.

/pdf/real-time-prediction-of-covid-19-related-mortality-using-1qaacrrbs1.pdf

Real-time prediction of COVID-19 related mortality using electronic health records.

OBJECTIVES: A simple evaluation tool for patients with novel coronavirus disease 2019 (COVID-19) could assist the physicians to triage COVID-19 patients effectively and rapidly. This study aimed to evaluate the predictive value of 5 early warning scores based on the admission data of critical COVID-19 patients. METHODS: Overall, medical records of 319 COVID-19 patients were included in the study. Demographic and clinical characteristics on admission were used for calculating the Standardized Early Warning Score (SEWS), National Early Warning Score (NEWS), National Early Warning Score2 (NEWS2), Hamilton Early Warning Score (HEWS), and Modified Early Warning Score (MEWS). Data on the outcomes (survival or death) were collected for each case and extracted for overall and subgroup analysis. Receiver operating characteristic curve analyses were performed. RESULTS: The area under the receiver operating characteristic curve for the SEWS, NEWS, NEWS2, HEWS, and MEWS in predicting mortality were 0.841 (95% CI: 0.765-0.916), 0.809 (95% CI: 0.727-0.891), 0.809 (95% CI: 0.727-0.891), 0.821 (95% CI: 0.748-0.895), and 0.670 (95% CI: 0.573-0.767), respectively. CONCLUSIONS: SEWS, NEWS, NEWS2, and HEWS demonstrated moderate discriminatory power and, therefore, offer potential utility as prognostic tools for screening severely ill COVID-19 patients. However, MEWS is not a good prognostic predictor for COVID-19.

/pdf/predictive-value-of-5-early-warning-scores-for-critical-1olyjx156f.pdf

Predictive Value of 5 Early Warning Scores for Critical COVID-19 Patients.

Objective To provide an overview and critical appraisal of early warning scores for adult hospital patients. Design Systematic review. Data sources Medline, CINAHL, PsycInfo, and Embase until June 2019. Eligibility criteria for study selection Studies describing the development or external validation of an early warning score for adult hospital inpatients. Results 13 171 references were screened and 95 articles were included in the review. 11 studies were development only, 23 were development and external validation, and 61 were external validation only. Most early warning scores were developed for use in the United States (n=13/34, 38%) and the United Kingdom (n=10/34, 29%). Death was the most frequent prediction outcome for development studies (n=10/23, 44%) and validation studies (n=66/84, 79%), with different time horizons (the most frequent was 24 hours). The most common predictors were respiratory rate (n=30/34, 88%), heart rate (n=28/34, 83%), oxygen saturation, temperature, and systolic blood pressure (all n=24/34, 71%). Age (n=13/34, 38%) and sex (n=3/34, 9%) were less frequently included. Key details of the analysis populations were often not reported in development studies (n=12/29, 41%) or validation studies (n=33/84, 39%). Small sample sizes and insufficient numbers of event patients were common in model development and external validation studies. Missing data were often discarded, with just one study using multiple imputation. Only nine of the early warning scores that were developed were presented in sufficient detail to allow individualised risk prediction. Internal validation was carried out in 19 studies, but recommended approaches such as bootstrapping or cross validation were rarely used (n=4/19, 22%). Model performance was frequently assessed using discrimination (development n=18/22, 82%; validation n=69/84, 82%), while calibration was seldom assessed (validation n=13/84, 15%). All included studies were rated at high risk of bias. Conclusions Early warning scores are widely used prediction models that are often mandated in daily clinical practice to identify early clinical deterioration in hospital patients. However, many early warning scores in clinical use were found to have methodological weaknesses. Early warning scores might not perform as well as expected and therefore they could have a detrimental effect on patient care. Future work should focus on following recommended approaches for developing and evaluating early warning scores, and investigating the impact and safety of using these scores in clinical practice. Systematic review registration PROSPERO CRD42017053324.

https://www.bmj.com/content/bmj/369/bmj.m1501.full.pdf

Early warning scores for detecting deterioration in adult hospital patients: systematic review and critical appraisal of methodology

Introduction: A full blood count (FBC) blood test includes 20 components. We systematically reviewed studies that assessed the association of the FBC and diagnosis of colorectal cancer to identify components as risk factors. We reviewed FBC-based prediction models for colorectal cancer risk. Methods: MEDLINE, EMBASE, CINAHL, and Web of Science were searched until 3 September 2019. We meta-analysed the mean difference in FBC components between those with and without a diagnosis and critically appraised the development and validation of FBC-based prediction models. Results: We included 53 eligible articles. Three of four meta-analysed components showed an association with diagnosis. In the remaining 16 with insufficient data for meta-analysis, three were associated with colorectal cancer. Thirteen FBC-based models were developed. Model performance was commonly assessed using the c-statistic (range 0.72–0.91) and calibration plots. Some models appeared to work well for early detection but good performance may be driven by early events. Conclusion: Red blood cells, haemoglobin, mean corpuscular volume, red blood cell distribution width, white blood cell count, and platelets are associated with diagnosis and could be used for referral. Existing FBC-based prediction models might not perform as well as expected and need further critical testing.

https://www.mdpi.com/2072-6694/12/9/2348/pdf

The Full Blood Count Blood Test for Colorectal Cancer Detection: A Systematic Review, Meta-Analysis, and Critical Appraisal.

A Full Blood Count (FBC) is a common blood test including 20 parameters, such as haemoglobin and platelets. FBCs from Electronic Health Record (EHR) databases provide a large sample of anonymised individual patient data and are increasingly used in research. We describe the quality of the FBC data in one EHR. The Test dataset from the Clinical Research Practice Datalink (CPRD) was accessed, which contains results of tests performed in primary care, such as FBC blood tests. Medical codes and entity codes, two coding systems used within CPRD to identify FBC records, were compared, with levels of mismatched coding, and number that could be rectified reported. The reliability of units of measurement are also described and missing data discussed. There were 14 entity codes and 138 medical codes for the FBC in the data. Medical and entity codes consistently corresponded to the same FBC parameter in 95.2% (n = 217,752,448) of parameters. In the 4.8% (n = 10,955,006) mismatches, the most common parameter rectified was mean platelet volume (n = 2,041,360) and 1,191,540 could not be rectified and were removed. Units of measurement were often either missing, partially entered, or did not appear to correspond to the blood value. The final dataset contained 16,537,017 FBC tests. Applying mathematical equations to derive some missing parameters in these FBCs resulted in 15 of 20 parameters available per FBC on average, with 0.3% of FBCs having all 20 parameters. Performing data quality checks can help to understand the extent of any issues in the dataset. We emphasise balancing large sample sizes with reliability of the data.

/pdf/assessing-data-quality-from-the-clinical-practice-research-3dj4b39jyb.pdf

Assessing data quality from the Clinical Practice Research Datalink: a methodological approach applied to the full blood count blood test

A Prospective Study of Diffusion-weighted Magnetic Resonance Imaging as an Early Prognostic Biomarker in Chemoradiotherapy in Squamous Cell Carcinomas of the Anus.

Pradeep S. Virdee

Papers

Early warning scores for detecting deterioration in adult hospital patients: systematic review and critical appraisal of methodology

The Full Blood Count Blood Test for Colorectal Cancer Detection: A Systematic Review, Meta-Analysis, and Critical Appraisal.

Assessing data quality from the Clinical Practice Research Datalink: a methodological approach applied to the full blood count blood test

A Prospective Study of Diffusion-weighted Magnetic Resonance Imaging as an Early Prognostic Biomarker in Chemoradiotherapy in Squamous Cell Carcinomas of the Anus.