Machine Learning is the study of methods for programming computers to learn. Computers are applied to a wide range of tasks, and for most of these it is relatively easy for programmers to design and implement the necessary software. However, there are many tasks for which this is difficult or impossible. These can be divided into four general categories. First, there are problems for which there exist no human experts. For example, in modern automated manufacturing facilities, there is a need to predict machine failures before they occur by analyzing sensor readings. Because the machines are new, there are no human experts who can be interviewed by a programmer to provide the knowledge necessary to build a computer system. A machine learning system can study recorded data and subsequent machine failures and learn prediction rules. Second, there are problems where human experts exist, but where they are unable to explain their expertise. This is the case in many perceptual tasks, such as speech recognition, hand-writing recognition, and natural language understanding. Virtually all humans exhibit expert-level abilities on these tasks, but none of them can describe the detailed steps that they follow as they perform them. Fortunately, humans can provide machines with examples of the inputs and correct outputs for these tasks, so machine learning algorithms can learn to map the inputs to the outputs. Third, there are problems where phenomena are changing rapidly. In finance, for example, people would like to predict the future behavior of the stock market, of consumer purchases, or of exchange rates. These behaviors change frequently, so that even if a programmer could construct a good predictive computer program, it would need to be rewritten frequently. A learning program can relieve the programmer of this burden by constantly modifying and tuning a set of learned prediction rules. Fourth, there are applications that need to be customized for each computer user separately. Consider, for example, a program to filter unwanted electronic mail messages. Different users will need different filters. It is unreasonable to expect each user to program his or her own rules, and it is infeasible to provide every user with a software engineer to keep the rules up-to-date. A machine learning system can learn which mail messages the user rejects and maintain the filtering rules automatically. Machine learning addresses many of the same research questions as the fields of statistics, data mining, and psychology, but with differences of emphasis. Statistics focuses on understanding the phenomena that have generated the data, often with the goal of testing different hypotheses about those phenomena. Data mining seeks to find patterns in the data that are understandable by people. Psychological studies of human learning aspire to understand the mechanisms underlying the various learning behaviors exhibited by people (concept learning, skill acquisition, strategy change, etc.).

Machine learning

Predictive modeling with electronic health record (EHR) data is anticipated to drive personalized medicine and improve healthcare quality. Constructing predictive statistical models typically requires extraction of curated predictor variables from normalized EHR data, a labor-intensive process that discards the vast majority of information in each patient’s record. We propose a representation of patients’ entire raw EHR records based on the Fast Healthcare Interoperability Resources (FHIR) format. We demonstrate that deep learning methods using this representation are capable of accurately predicting multiple medical events from multiple centers without site-specific data harmonization. We validated our approach using de-identified EHR data from two US academic medical centers with 216,221 adult patients hospitalized for at least 24 h. In the sequential format we propose, this volume of EHR data unrolled into a total of 46,864,534,945 data points, including clinical notes. Deep learning models achieved high accuracy for tasks such as predicting: in-hospital mortality (area under the receiver operator curve [AUROC] across sites 0.93–0.94), 30-day unplanned readmission (AUROC 0.75–0.76), prolonged length of stay (AUROC 0.85–0.86), and all of a patient’s final discharge diagnoses (frequency-weighted AUROC 0.90). These models outperformed traditional, clinically-used predictive models in all cases. We believe that this approach can be used to create accurate and scalable predictions for a variety of clinical scenarios. In a case study of a particular prediction, we demonstrate that neural networks can be used to identify relevant information from the patient’s chart.

/pdf/scalable-and-accurate-deep-learning-with-electronic-health-eu4vc3mdb6.pdf

Scalable and accurate deep learning with electronic health records

Predictive modeling with electronic health record (EHR) data is anticipated to drive personalized medicine and improve healthcare quality. Constructing predictive statistical models typically requires extraction of curated predictor variables from normalized EHR data, a labor-intensive process that discards the vast majority of information in each patient's record. We propose a representation of patients' entire, raw EHR records based on the Fast Healthcare Interoperability Resources (FHIR) format. We demonstrate that deep learning methods using this representation are capable of accurately predicting multiple medical events from multiple centers without site-specific data harmonization. We validated our approach using de-identified EHR data from two U.S. academic medical centers with 216,221 adult patients hospitalized for at least 24 hours. In the sequential format we propose, this volume of EHR data unrolled into a total of 46,864,534,945 data points, including clinical notes. Deep learning models achieved high accuracy for tasks such as predicting in-hospital mortality (AUROC across sites 0.93-0.94), 30-day unplanned readmission (AUROC 0.75-0.76), prolonged length of stay (AUROC 0.85-0.86), and all of a patient's final discharge diagnoses (frequency-weighted AUROC 0.90). These models outperformed state-of-the-art traditional predictive models in all cases. We also present a case-study of a neural-network attribution system, which illustrates how clinicians can gain some transparency into the predictions. We believe that this approach can be used to create accurate and scalable predictions for a variety of clinical scenarios, complete with explanations that directly highlight evidence in the patient's chart.

/pdf/scalable-and-accurate-deep-learning-for-electronic-health-55ghjalidu.pdf

Scalable and accurate deep learning for electronic health records

Background There is interest in using convolutional neural networks (CNNs) to analyze medical imaging to provide computer-aided diagnosis (CAD). Recent work has suggested that image classification CNNs may not generalize to new data as well as previously believed. We assessed how well CNNs generalized across three hospital systems for a simulated pneumonia screening task. Methods and findings A cross-sectional design with multiple model training cohorts was used to evaluate model generalizability to external sites using split-sample validation. A total of 158,323 chest radiographs were drawn from three institutions: National Institutes of Health Clinical Center (NIH; 112,120 from 30,805 patients), Mount Sinai Hospital (MSH; 42,396 from 12,904 patients), and Indiana University Network for Patient Care (IU; 3,807 from 3,683 patients). These patient populations had an age mean (SD) of 46.9 years (16.6), 63.2 years (16.5), and 49.6 years (17) with a female percentage of 43.5%, 44.8%, and 57.3%, respectively. We assessed individual models using the area under the receiver operating characteristic curve (AUC) for radiographic findings consistent with pneumonia and compared performance on different test sets with DeLong’s test. The prevalence of pneumonia was high enough at MSH (34.2%) relative to NIH and IU (1.2% and 1.0%) that merely sorting by hospital system achieved an AUC of 0.861 (95% CI 0.855–0.866) on the joint MSH–NIH dataset. Models trained on data from either NIH or MSH had equivalent performance on IU (P values 0.580 and 0.273, respectively) and inferior performance on data from each other relative to an internal test set (i.e., new data from within the hospital system used for training data; P values both <0.001). The highest internal performance was achieved by combining training and test data from MSH and NIH (AUC 0.931, 95% CI 0.927–0.936), but this model demonstrated significantly lower external performance at IU (AUC 0.815, 95% CI 0.745–0.885, P = 0.001). To test the effect of pooling data from sites with disparate pneumonia prevalence, we used stratified subsampling to generate MSH–NIH cohorts that only differed in disease prevalence between training data sites. When both training data sites had the same pneumonia prevalence, the model performed consistently on external IU data (P = 0.88). When a 10-fold difference in pneumonia rate was introduced between sites, internal test performance improved compared to the balanced model (10× MSH risk P < 0.001; 10× NIH P = 0.002), but this outperformance failed to generalize to IU (MSH 10× P < 0.001; NIH 10× P = 0.027). CNNs were able to directly detect hospital system of a radiograph for 99.95% NIH (22,050/22,062) and 99.98% MSH (8,386/8,388) radiographs. The primary limitation of our approach and the available public data is that we cannot fully assess what other factors might be contributing to hospital system–specific biases. Conclusion Pneumonia-screening CNNs achieved better internal than external performance in 3 out of 5 natural comparisons. When models were trained on pooled data from sites with different pneumonia prevalence, they performed better on new pooled data from these sites but not on external data. CNNs robustly identified hospital system and department within a hospital, which can have large differences in disease burden and may confound predictions.

/pdf/variable-generalization-performance-of-a-deep-learning-model-qqzxzrs5ql.pdf

Variable generalization performance of a deep learning model to detect pneumonia in chest radiographs: A cross-sectional study.

A promise of machine learning in health care is the avoidance of biases in diagnosis and treatment; a computer algorithm could objectively synthesize and interpret the data in the medical record. Integration of machine learning with clinical decision support tools, such as computerized alerts or diagnostic support, may offer physicians and others who provide health care targeted and timely information that can improve clinical decisions. Machine learning algorithms, however, may also be subject to biases. The biases include those related to missing data and patients not identified by algorithms, sample size and underestimation, and misclassification and measurement error. There is concern that biases and deficiencies in the data used by machine learning algorithms may contribute to socioeconomic disparities in health care. This Special Communication outlines the potential biases that may be introduced into machine learning–based clinical decision support tools that use electronic health record data and proposes potential solutions to the problems of overreliance on automation, algorithms based on biased data, and algorithms that do not provide information that is clinically meaningful. Existing health care disparities should not be amplified by thoughtless or excessive reliance on machines.

/pdf/potential-biases-in-machine-learning-algorithms-using-4n43rc97ch.pdf

Potential Biases in Machine Learning Algorithms Using Electronic Health Record Data.

Over the past decade, machine learning techniques have made substantial advances in many domains. In health care, global interest in the potential of machine learning has increased; for example, a deep learning algorithm has shown high accuracy in detecting diabetic retinopathy.1 There have been suggestions that machine learning will drive changes in health care within a few years, specifically in medical disciplines that require more accurate prognostic models (eg, oncology) and those based on pattern recognition (eg, radiology and pathology). However, comparative studies on the effectiveness of machine learning–based decision support systems (ML-DSS) in medicine are lacking, especially regarding the effects on health outcomes. Moreover, the introduction of new technologies in health care has not always been straightforward or without unintended and adverse effects.2 In this Viewpoint we consider the potential unintended consequences that may result from the application of ML-DSS in clinical practice.

https://jamanetwork.com/journals/jama/articlepdf/2645762/jama_Cabitza_2017_vp_170094.pdf

Unintended Consequences of Machine Learning in Medicine.

The use of telemonitoring and telemedicine is a relatively new but quickly developing area in medicine. As new digital tools and applications are being created and used to manage medical conditions such as heart failure, many implications require close consideration and further study, including the effectiveness and safety of these telemonitoring tools in diagnosing, treating and managing heart failure compared to traditional face-to-face doctor-patient interaction. When compared to multidisciplinary intervention programs which are frequently hindered by economic, geographic and bureaucratic barriers, non-invasive remote monitoring could be a solution to support and promote the care of patients over time. Therefore it is crucial to identify the most relevant biological parameters to monitor, which heart failure sub-populations may gain real benefits from telehealth interventions and in which specific healthcare subsets these interventions should be implemented in order to maximise value.

Value of Telemonitoring and Telemedicine in Heart Failure Management.

This paper considers the use of machine learning in medicine by focusing on the main problem that it has been aimed at solving or at least minimizing: uncertainty. However, we point out how uncertainty is so ingrained in medicine that it biases also the representation of clinical phenomena, that is the very input of this class of computational models, thus undermining the clinical significance of their output. Recognizing this can motivate researchers to pursue different ways to assess the value of these decision aids, as well as alternative techniques that do not “sweep uncertainty under the rug” within an objectivist fiction (which doctors can come up by trusting).

A Giant with Feet of Clay: On the Validity of the Data that Feed Machine Learning in Medicine

This article focuses on the production side of clinical data work, or data recording work, and in particular, on its multiplicity in terms of data variability. We report the findings from two case ...

https://journals.sagepub.com/doi/pdf/10.1177/1460458218824705

The elephant in the record: On the multiplicity of data recording work.

Coronary stent thrombosis (CST) is a major concern of interventional cardiology. Several risk factors for CST have been identified, but as a whole they do not explain the pathophysiology of CST. This study was designed to investigate whether acute infection–inflammation could facilitate the occurrence of CST. Forty-one patients, aged 66.6 ± 11 years, consecutively admitted to our catheterization laboratory for acute, subacute or late CST, were retrospectively analysed. Transient acute infection–inflammation on admission for CST was diagnosed by predefined criteria. Prevalence of known risk factors for CST was also investigated. Twenty-one patients (51%) met predefined criteria for the occurrence of acute infection–inflammation. On admission, in these patients, levels of systemic humoral and cellular inflammatory markers were significantly higher than those of patients without recent or ongoing acute infection–inflammation (p < 0.05 for all). 62% of patients with acute infection–inflammation had less than two known risk factors for CST whereas only 37% patients without infection–inflammation showed less than two risk factors (p = 0.03) and showed more frequent interruption of antiplatelet treatment (17 vs. 2.4%, p = 0.02), mean longer stent length (20.5 ± 4.8 vs. 16.5 ± 5.1 mm, p = 0.02) and lower left ventricular ejection fraction before CST (42.9 ± 14 vs. 47.3 ± 11%, p = 0.02). In conclusion, acute infection–inflammation could play a role in facilitating the occurrence of CST in a subgroup with low risk profile for known risk factors. Our findings, if confirmed, could suggest new opportunities for prevention and treatment of CST.

Raffaele Rasoini

Papers

Unintended Consequences of Machine Learning in Medicine.

Value of Telemonitoring and Telemedicine in Heart Failure Management.

A Giant with Feet of Clay: On the Validity of the Data that Feed Machine Learning in Medicine

The elephant in the record: On the multiplicity of data recording work.

Acute infection–inflammation and coronary stent thrombosis: an observational study