The assessment of calibration performance of risk prediction models based on regression or more flexible machine learning algorithms receives little attention. Herein, we argue that this needs to change immediately because poorly calibrated algorithms can be misleading and potentially harmful for clinical decision-making. We summarize how to avoid poor calibration at algorithm development and how to assess calibration at algorithm validation, emphasizing balance between model complexity and the available sample size. At external validation, calibration curves require sufficiently large samples. Algorithm updating should be considered for appropriate support of clinical practice. Efforts are required to avoid poor calibration when developing prediction models, to evaluate calibration when validating models, and to update models when indicated. The ultimate aim is to optimize the utility of predictive analytics for shared decision-making and patient counseling.

/pdf/calibration-the-achilles-heel-of-predictive-analytics-gqef6dspha.pdf

Calibration: the Achilles heel of predictive analytics

Advancements in deep learning techniques carry the potential to make significant contributions to healthcare, particularly in fields that utilize medical imaging for diagnosis, prognosis, and treatment decisions. The current state-of-the-art deep learning models for radiology applications consider only pixel-value information without data informing clinical context. Yet in practice, pertinent and accurate non-imaging data based on the clinical history and laboratory data enable physicians to interpret imaging findings in the appropriate clinical context, leading to a higher diagnostic accuracy, informative clinical decision making, and improved patient outcomes. To achieve a similar goal using deep learning, medical imaging pixel-based models must also achieve the capability to process contextual data from electronic health records (EHR) in addition to pixel data. In this paper, we describe different data fusion techniques that can be applied to combine medical imaging with EHR, and systematically review medical data fusion literature published between 2012 and 2020. We conducted a systematic search on PubMed and Scopus for original research articles leveraging deep learning for fusion of multimodality data. In total, we screened 985 studies and extracted data from 17 papers. By means of this systematic review, we present current knowledge, summarize important results and provide implementation guidelines to serve as a reference for researchers interested in the application of multimodal fusion in medical imaging.

/pdf/fusion-of-medical-imaging-and-electronic-health-records-2qnwm494ez.pdf

Fusion of medical imaging and electronic health records using deep learning: a systematic review and implementation guidelines.

Deep learning (DL) has the potential to transform medical diagnostics. However, the diagnostic accuracy of DL is uncertain. Our aim was to evaluate the diagnostic accuracy of DL algorithms to identify pathology in medical imaging. Searches were conducted in Medline and EMBASE up to January 2020. We identified 11,921 studies, of which 503 were included in the systematic review. Eighty-two studies in ophthalmology, 82 in breast disease and 115 in respiratory disease were included for meta-analysis. Two hundred twenty-four studies in other specialities were included for qualitative review. Peer-reviewed studies that reported on the diagnostic accuracy of DL algorithms to identify pathology using medical imaging were included. Primary outcomes were measures of diagnostic accuracy, study design and reporting standards in the literature. Estimates were pooled using random-effects meta-analysis. In ophthalmology, AUC's ranged between 0.933 and 1 for diagnosing diabetic retinopathy, age-related macular degeneration and glaucoma on retinal fundus photographs and optical coherence tomography. In respiratory imaging, AUC's ranged between 0.864 and 0.937 for diagnosing lung nodules or lung cancer on chest X-ray or CT scan. For breast imaging, AUC's ranged between 0.868 and 0.909 for diagnosing breast cancer on mammogram, ultrasound, MRI and digital breast tomosynthesis. Heterogeneity was high between studies and extensive variation in methodology, terminology and outcome measures was noted. This can lead to an overestimation of the diagnostic accuracy of DL algorithms on medical imaging. There is an immediate need for the development of artificial intelligence-specific EQUATOR guidelines, particularly STARD, in order to provide guidance around key issues in this field.

/pdf/diagnostic-accuracy-of-deep-learning-in-medical-imaging-a-4y9gv3abpx.pdf

Diagnostic accuracy of deep learning in medical imaging: a systematic review and meta-analysis.

The European Collaborative on Personalized Early Detection and Prevention of Breast Cancer (ENVISION) brings together several international research consortia working on different aspects of the personalized early detection and prevention of breast cancer. In a consensus conference held in 2019, the members of this network identified research areas requiring development to enable evidence-based personalized interventions that might improve the benefits and reduce the harms of existing breast cancer screening and prevention programmes. The priority areas identified were: 1) breast cancer subtype-specific risk assessment tools applicable to women of all ancestries; 2) intermediate surrogate markers of response to preventive measures; 3) novel non-surgical preventive measures to reduce the incidence of breast cancer of poor prognosis; and 4) hybrid effectiveness–implementation research combined with modelling studies to evaluate the long-term population outcomes of risk-based early detection strategies. The implementation of such programmes would require health-care systems to be open to learning and adapting, the engagement of a diverse range of stakeholders and tailoring to societal norms and values, while also addressing the ethical and legal issues. In this Consensus Statement, we discuss the current state of breast cancer risk prediction, risk-stratified prevention and early detection strategies, and their implementation. Throughout, we highlight priorities for advancing each of these areas.

/pdf/personalized-early-detection-and-prevention-of-breast-cancer-rar9urauvb.pdf

Personalized early detection and prevention of breast cancer: ENVISION consensus statement.

Overview of radiomics in breast cancer diagnosis and prognostication

Background Mammographic density improves the accuracy of breast cancer risk models. However, the use of breast density is limited by subjective assessment, variation across radiologists, and restricted data. A mammography-based deep learning (DL) model may provide more accurate risk prediction. Purpose To develop a mammography-based DL breast cancer risk model that is more accurate than established clinical breast cancer risk models. Materials and Methods This retrospective study included 88 994 consecutive screening mammograms in 39 571 women between January 1, 2009, and December 31, 2012. For each patient, all examinations were assigned to either training, validation, or test sets, resulting in 71 689, 8554, and 8751 examinations, respectively. Cancer outcomes were obtained through linkage to a regional tumor registry. By using risk factor information from patient questionnaires and electronic medical records review, three models were developed to assess breast cancer risk within 5 years: a risk-factor-based logistic regression model (RF-LR) that used traditional risk factors, a DL model (image-only DL) that used mammograms alone, and a hybrid DL model that used both traditional risk factors and mammograms. Comparisons were made to an established breast cancer risk model that included breast density (Tyrer-Cuzick model, version 8 [TC]). Model performance was compared by using areas under the receiver operating characteristic curve (AUCs) with DeLong test (P < .05). Results The test set included 3937 women, aged 56.20 years ± 10.04. Hybrid DL and image-only DL showed AUCs of 0.70 (95% confidence interval [CI]: 0.66, 0.75) and 0.68 (95% CI: 0.64, 0.73), respectively. RF-LR and TC showed AUCs of 0.67 (95% CI: 0.62, 0.72) and 0.62 (95% CI: 0.57, 0.66), respectively. Hybrid DL showed a significantly higher AUC (0.70) than TC (0.62; P < .001) and RF-LR (0.67; P = .01). Conclusion Deep learning models that use full-field mammograms yield substantially improved risk discrimination compared with the Tyrer-Cuzick (version 8) model. © RSNA, 2019 Online supplemental material is available for this article. See also the editorial by Sitek and Wolfe in this issue.

https://pubs.rsna.org/doi/pdf/10.1148/radiol.2019182716

A Deep Learning Mammography-based Model for Improved Breast Cancer Risk Prediction

OBJECTIVE. The purpose of this study is to develop an image-based deep learning (DL) model to predict the 5-year risk of breast cancer on the basis of a single breast MR image from a screening exam...

Tally Portnoi

Papers

A Deep Learning Mammography-based Model for Improved Breast Cancer Risk Prediction

Deep Learning Model to Assess Cancer Risk on the Basis of a Breast MR Image Alone.