Showing papers by "Dorin Comaniciu published in 2020"

Quantification of Tomographic Patterns associated with COVID-19 from Chest CT

Abstract: PURPOSE: To present a method that automatically segments and quantifies abnormal CT patterns commonly present in coronavirus disease 2019 (COVID-19), namely ground glass opacities and consolidations MATERIALS AND METHODS: In this retrospective study, the proposed method takes as input a non-contrasted chest CT and segments the lesions, lungs, and lobes in three dimensions, based on a dataset of 9749 chest CT volumes The method outputs two combined measures of the severity of lung and lobe involvement, quantifying both the extent of COVID-19 abnormalities and presence of high opacities, based on deelearning and deereinforcement learning The first measure of (PO, PHO) is global, while the second of (LSS, LHOS) is lobewise Evaluation of the algorithm is reported on CTs of 200 participants (100 COVID-19 confirmed patients and 100 healthy controls) from institutions from Canada, Europe and the United States collected between 2002-Present (April, 2020) Ground truth is established by manual annotations of lesions, lungs, and lobes Correlation and regression analyses were performed to compare the prediction to the ground truth RESULTS: Pearson correlation coefficient between method prediction and ground truth for COVID-19 cases was calculated as 0 92 for PO (< 001), 0 97 for PHO(< 001), 0 91 for LSS (< 001), 0 90 for LHOS (< 001) 98 of 100 healthy controls had a predicted PO of less than 1%, 2 had between 1-2% Automated processing time to compute the severity scores was 10 seconds per case compared to 30 minutes required for manual annotations CONCLUSION: A new method segments regions of CT abnormalities associated with COVID-19 and computes (PO, PHO), as well as (LSS, LHOS) severity scores

Autonomous Detection and Classification of PI-RADS Lesions in an MRI Screening Population Incorporating Multicenter-Labeled Deep Learning and Biparametric Imaging: Proof of Concept

Abstract: Background: Opportunistic prostate cancer (PCa) screening is a controversial topic. Magnetic resonance imaging (MRI) has proven to detect prostate cancer with a high sensitivity and specificity, leading to the idea to perform an image-guided prostate cancer (PCa) screening; Methods: We evaluated a prospectively enrolled cohort of 49 healthy men participating in a dedicated image-guided PCa screening trial employing a biparametric MRI (bpMRI) protocol consisting of T2-weighted (T2w) and diffusion weighted imaging (DWI) sequences. Datasets were analyzed both by human readers and by a fully automated artificial intelligence (AI) software using deep learning (DL). Agreement between the algorithm and the reports—serving as the ground truth—was compared on a per-case and per-lesion level using metrics of diagnostic accuracy and k statistics; Results: The DL method yielded an 87% sensitivity (33/38) and 50% specificity (5/10) with a k of 0.42. 12/28 (43%) Prostate Imaging Reporting and Data System (PI-RADS) 3, 16/22 (73%) PI-RADS 4, and 5/5 (100%) PI-RADS 5 lesions were detected compared to the ground truth. Targeted biopsy revealed PCa in six participants, all correctly diagnosed by both the human readers and AI. Conclusions: The results of our study show that in our AI-assisted, image-guided prostate cancer screening the software solution was able to identify highly suspicious lesions and has the potential to effectively guide the targeted-biopsy workflow.

Artificial Intelligence in Diagnostic Imaging: Status Quo, Challenges, and Future Opportunities.

Abstract: In this review article, the current and future impact of artificial intelligence (AI) technologies on diagnostic imaging is discussed, with a focus on cardio-thoracic applications. The processing of imaging data is described at 4 levels of increasing complexity and wider implications. At the examination level, AI aims at improving, simplifying, and standardizing image acquisition and processing. Systems for AI-driven automatic patient iso-centering before a computed tomography (CT) scan, patient-specific adaptation of image acquisition parameters, and creation of optimized and standardized visualizations, for example, automatic rib-unfolding, are discussed. At the reading and reporting levels, AI focuses on automatic detection and characterization of features and on automatic measurements in the images. A recently introduced AI system for chest CT imaging is presented that reports specific findings such as nodules, low-attenuation parenchyma, and coronary calcifications, including automatic measurements of, for example, aortic diameters. At the prediction and prescription levels, AI focuses on risk prediction and stratification, as opposed to merely detecting, measuring, and quantifying images. An AI-based approach for individualizing radiation dose in lung stereotactic body radiotherapy is discussed. The digital twin is presented as a concept of individualized computational modeling of human physiology, with AI-based CT-fractional flow reserve modeling as a first example. Finally, at the cohort and population analysis levels, the focus of AI shifts from clinical decision-making to operational decisions.

False Positive Reduction Using Multiscale Contextual Features for Prostate Cancer Detection in Multi-Parametric MRI Scans

Abstract: Prostate cancer (PCa) is the most prevalent and one of the leading causes of cancer death among men. Multi-parametric MRI (mp-MRI) is a prominent diagnostic scan, which could help in avoiding unnecessary biopsies for men screened for PCa. Artificial intelligence (AI) systems could help radiologists to be more accurate and consistent in diagnosing clinically significant cancer from mp-MRI scans. Lack of specificity has been identified recently as one of weak points of such assistance systems. In this paper, we propose a novel false positive reduction network to be added to the overall detection system to further analyze lesion candidates. The new network utilizes multiscale 2D image stacks of these candidates to discriminate between true and false positive detections. We trained and validated our network on a dataset with 2170 cases from seven different institutions and tested it on a separate independent dataset with 243 cases. With the proposed model, we achieved area under curve (AUC) of 0.876 on discriminating between true and false positive detected lesions and improved the AUC from 0.825 to 0.867 on overall identification of clinically significant cases.

Quantifying and Leveraging Predictive Uncertainty for Medical Image Assessment

Abstract: The interpretation of medical images is a challenging task, often complicated by the presence of artifacts, occlusions, limited contrast and more. Most notable is the case of chest radiography, where there is a high inter-rater variability in the detection and classification of abnormalities. This is largely due to inconclusive evidence in the data or subjective definitions of disease appearance. An additional example is the classification of anatomical views based on 2D Ultrasound images. Often, the anatomical context captured in a frame is not sufficient to recognize the underlying anatomy. Current machine learning solutions for these problems are typically limited to providing probabilistic predictions, relying on the capacity of underlying models to adapt to limited information and the high degree of label noise. In practice, however, this leads to overconfident systems with poor generalization on unseen data. To account for this, we propose a system that learns not only the probabilistic estimate for classification, but also an explicit uncertainty measure which captures the confidence of the system in the predicted output. We argue that this approach is essential to account for the inherent ambiguity characteristic of medical images from different radiologic exams including computed radiography, ultrasonography and magnetic resonance imaging. In our experiments we demonstrate that sample rejection based on the predicted uncertainty can significantly improve the ROC-AUC for various tasks, e.g., by 8% to 0.91 with an expected rejection rate of under 25% for the classification of different abnormalities in chest radiographs. In addition, we show that using uncertainty-driven bootstrapping to filter the training data, one can achieve a significant increase in robustness and accuracy.

3D Tomographic Pattern Synthesis for Enhancing the Quantification of COVID-19

Abstract: The Coronavirus Disease (COVID-19) has affected 1.8 million people and resulted in more than 110,000 deaths as of April 12, 2020. Several studies have shown that tomographic patterns seen on chest Computed Tomography (CT), such as ground-glass opacities, consolidations, and crazy paving pattern, are correlated with the disease severity and progression. CT imaging can thus emerge as an important modality for the management of COVID-19 patients. AI-based solutions can be used to support CT based quantitative reporting and make reading efficient and reproducible if quantitative biomarkers, such as the Percentage of Opacity (PO), can be automatically computed. However, COVID-19 has posed unique challenges to the development of AI, specifically concerning the availability of appropriate image data and annotations at scale. In this paper, we propose to use synthetic datasets to augment an existing COVID-19 database to tackle these challenges. We train a Generative Adversarial Network (GAN) to inpaint COVID-19 related tomographic patterns on chest CTs from patients without infectious diseases. Additionally, we leverage location priors derived from manually labeled COVID-19 chest CTs patients to generate appropriate abnormality distributions. Synthetic data are used to improve both lung segmentation and segmentation of COVID-19 patterns by adding 20% of synthetic data to the real COVID-19 training data. We collected 2143 chest CTs, containing 327 COVID-19 positive cases, acquired from 12 sites across 7 countries. By testing on 100 COVID-19 positive and 100 control cases, we show that synthetic data can help improve both lung segmentation (+6.02% lesion inclusion rate) and abnormality segmentation (+2.78% dice coefficient), leading to an overall more accurate PO computation (+2.82% Pearson coefficient).

Deep Attentive Panoptic Model for Prostate Cancer Detection Using Biparametric MRI Scans

Abstract: Multi-parametric MRI (mp-MRI) has recently been established in major guidelines as a first-line diagnostic test for men suspected of having prostate cancer (PCa) primarily to detect and classify clinically significant lesions. However, widespread utilization is still challenged by 1) the difficulty of interpretation specifically for radiologists less experienced in reading mp-MRI scans, and 2) decreased productivity associated with increased time spent per case for reading these complex scans. Deep learning based lesion detection and segmentation methods have been proposed for radiologists to perform their tasks more accurately and efficiently. In this work, we present a novel panoptic lesion detection and segmentation method with both semantic and instance branches as well as an attention module to optimally incorporate both local and global image features. In a free-response receiver operating characteristics (FROC) analysis for lesion sensitivity on an independent dataset with 243 patients, our method has achieved 89% sensitivity and 85% with 0.94 and 0.62 false positives per patient, respectively. Using the proposed method, we have achieved an unprecedented area under ROC curve (AUC) of 0.897 in identifying clinically significant cases.

Machine Learning Automatically Detects COVID-19 using Chest CTs in a Large Multicenter Cohort

Abstract: Objectives: To investigate machine-learning classifiers and interpretable models using chest CT for detection of COVID-19 and differentiation from other pneumonias, ILD and normal CTs. Methods: Our retrospective multi-institutional study obtained 2096 chest CTs from 16 institutions (including 1077 COVID-19 patients). Training/testing cohorts included 927/100 COVID-19, 388/33 ILD, 189/33 other pneumonias, and 559/34 normal (no pathologies) CTs. A metric-based approach for classification of COVID-19 used interpretable features, relying on logistic regression and random forests. A deep learning-based classifier differentiated COVID-19 via 3D features extracted directly from CT attenuation and probability distribution of airspace opacities. Results: Most discriminative features of COVID-19 are percentage of airspace opacity and peripheral and basal predominant opacities, concordant with the typical characterization of COVID-19 in the literature. Unsupervised hierarchical clustering compares feature distribution across COVID-19 and control cohorts. The metrics-based classifier achieved AUC=0.83, sensitivity=0.74, and specificity=0.79 of versus respectively 0.93, 0.90, and 0.83 for the DL-based classifier. Most of ambiguity comes from non-COVID-19 pneumonia with manifestations that overlap with COVID-19, as well as mild COVID-19 cases. Non-COVID-19 classification performance is 91% for ILD, 64% for other pneumonias and 94% for no pathologies, which demonstrates the robustness of our method against different compositions of control groups. Conclusions: Our new method accurately discriminates COVID-19 from other types of pneumonia, ILD, and no pathologies CTs, using quantitative imaging features derived from chest CT, while balancing interpretability of results and classification performance, and therefore may be useful to facilitate diagnosis of COVID-19.

Target detection in latent space

Abstract: A method for processing medical image data comprises: inputting medical image data to a variational autoencoder configured to reduce a dimensionality of the medical image data to a latent space having one or more latent variables with latent variable values, such that the latent variable values corresponding to an image with no tissue of a target tissue type fit within one or more clusters; determining a probability that the latent variable values corresponding to the medical image data fit within the one or more clusters based on the latent variable values; and determining that a tissue of the target tissue type is present in response to a determination that the medical image data have less than a threshold probability of fitting within any of the one or more clusters based on the latent variable values.

Automated Quantification of CT Patterns Associated with COVID-19 from Chest CT

Abstract: Purpose: To present a method that automatically segments and quantifies abnormal CT patterns commonly present in coronavirus disease 2019 (COVID-19), namely ground glass opacities and consolidations. Materials and Methods: In this retrospective study, the proposed method takes as input a non-contrasted chest CT and segments the lesions, lungs, and lobes in three dimensions, based on a dataset of 9749 chest CT volumes. The method outputs two combined measures of the severity of lung and lobe involvement, quantifying both the extent of COVID-19 abnormalities and presence of high opacities, based on deep learning and deep reinforcement learning. The first measure of (PO, PHO) is global, while the second of (LSS, LHOS) is lobewise. Evaluation of the algorithm is reported on CTs of 200 participants (100 COVID-19 confirmed patients and 100 healthy controls) from institutions from Canada, Europe and the United States collected between 2002-Present (April, 2020). Ground truth is established by manual annotations of lesions, lungs, and lobes. Correlation and regression analyses were performed to compare the prediction to the ground truth. Results: Pearson correlation coefficient between method prediction and ground truth for COVID-19 cases was calculated as 0.92 for PO (P < .001), 0.97 for PHO(P < .001), 0.91 for LSS (P < .001), 0.90 for LHOS (P < .001). 98 of 100 healthy controls had a predicted PO of less than 1%, 2 had between 1-2%. Automated processing time to compute the severity scores was 10 seconds per case compared to 30 minutes required for manual annotations. Conclusion: A new method segments regions of CT abnormalities associated with COVID-19 and computes (PO, PHO), as well as (LSS, LHOS) severity scores.

Automated detection and quantification of COVID-19 airspace disease on chest radiographs: A novel approach achieving radiologist-level performance using a CNN trained on digital reconstructed radiographs (DRRs) from CT-based ground-truth

Abstract: Purpose: To leverage volumetric quantification of airspace disease (AD) derived from a superior modality (CT) serving as ground truth, projected onto digitally reconstructed radiographs (DRRs) to: 1) train a convolutional neural network to quantify airspace disease on paired CXRs; and 2) compare the DRR-trained CNN to expert human readers in the CXR evaluation of patients with confirmed COVID-19. Materials and Methods: We retrospectively selected a cohort of 86 COVID-19 patients (with positive RT-PCR), from March-May 2020 at a tertiary hospital in the northeastern USA, who underwent chest CT and CXR within 48 hrs. The ground truth volumetric percentage of COVID-19 related AD (POv) was established by manual AD segmentation on CT. The resulting 3D masks were projected into 2D anterior-posterior digitally reconstructed radiographs (DRR) to compute area-based AD percentage (POa). A convolutional neural network (CNN) was trained with DRR images generated from a larger-scale CT dataset of COVID-19 and non-COVID-19 patients, automatically segmenting lungs, AD and quantifying POa on CXR. CNN POa results were compared to POa quantified on CXR by two expert readers and to the POv ground-truth, by computing correlations and mean absolute errors. Results: Bootstrap mean absolute error (MAE) and correlations between POa and POv were 11.98% [11.05%-12.47%] and 0.77 [0.70-0.82] for average of expert readers, and 9.56%-9.78% [8.83%-10.22%] and 0.78-0.81 [0.73-0.85] for the CNN, respectively. Conclusion: Our CNN trained with DRR using CT-derived airspace quantification achieved expert radiologist level of accuracy in the quantification of airspace disease on CXR, in patients with positive RT-PCR for COVID-19.

Graph Attention Network based Pruning for Reconstructing 3D Liver Vessel Morphology from Contrasted CT Images

Abstract: With the injection of contrast material into blood vessels, multi-phase contrasted CT images can enhance the visibility of vessel networks in the human body. Reconstructing the 3D geometric morphology of liver vessels from the contrasted CT images can enable multiple liver preoperative surgical planning applications. Automatic reconstruction of liver vessel morphology remains a challenging problem due to the morphological complexity of liver vessels and the inconsistent vessel intensities among different multi-phase contrasted CT images. On the other side, high integrity is required for the 3D reconstruction to avoid decision making biases. In this paper, we propose a framework for liver vessel morphology reconstruction using both a fully convolutional neural network and a graph attention network. A fully convolutional neural network is first trained to produce the liver vessel centerline heatmap. An over-reconstructed liver vessel graph model is then traced based on the heatmap using an image processing based algorithm. We use a graph attention network to prune the false-positive branches by predicting the presence probability of each segmented branch in the initial reconstruction using the aggregated CNN features. We evaluated the proposed framework on an in-house dataset consisting of 418 multi-phase abdomen CT images with contrast. The proposed graph network pruning improves the overall reconstruction F1 score by 6.4% over the baseline. It also outperformed the other state-of-the-art curvilinear structure reconstruction algorithms.

Treatment Intensity Stratification in COVID-19 by Fully Automated Analysis of Pulmonary and Cardiovascular Metrics on Initial Chest CT using Deep Learning

Abstract: ObjectivesTo predict ultimate treatment intensity of COVID-19 patients using pulmonary and cardiovascular metrics fully automatically extracted from initial chest CTs.Methods All patients tested positive for SARS-CoV-2 by RT-PCR at our emergency department between March 25 and April 14, 2020 were identified (n=391). For those patients, all initial chest CTs were analyzed (n=85). Multiple pulmonary and cardiovascular metrics were extracted using deep convolutional neural networks. Three clinical treatment intensity groups were defined according to the most intensive treatment of a patient, determined six weeks later: Group 1 (outpatient), group 2 (general ward), and group 3 (intensive care unit; ICU). Univariate analyses were performed to analyze differences between groups. Subsequently, multiple metrics were combined in two binary logistic regression analyses and resulting prediction probabilities used to classify whether a patient needed hospitalization or ICU care. For analysis of discriminatory power, ROC curves were plotted and areas-under-the-curves (AUCs) calculated.ResultsThe mean interval between presentation at the emergency department and the chest CT was 1.4 days. Among others, mean percentage of lung volume affected by opacities (PO) and mean total pericardial volume (TPV) increased statistically significantly with higher treatment intensity [from group 1 to 3, standard deviation in brackets: PO: 0.8%(1.5)–11.6%(13.1)–31.6%(20.1); TPV: 733.4ml(231.7)–866.2ml(211.2)–925.7ml(125.5); both: p<0.001]. AUCs were 0.85 (ICU vs. no ICU) and 0.94 (hospitalization vs. no hospitalization).Conclusions Metrics fully automatically extracted from initial chest CTs increase with treatment intensity of COVID-19 patients. This information can be exploited to prospectively manage allocation of healthcare resources.

Artificial Intelligence for Healthcare

Abstract: We discuss the current and future impact of artificial intelligence (AI) technologies on healthcare. We consider four hierarchical levels of healthcare data generation and processing of increasing complexity and wider implications. At the imaging scanner and instrument level, AI aims at improving, simplifying, and standardizing data acquisition and preparation. We present examples of systems for AI-driven automatic patient iso-centering before a computed tomography scan, deep learning-based image reconstruction, and creation of optimized and standardized visualizations, for example, automatic rib-unfolding. At the reading and reporting levels, AI focuses on the detection and characterization of abnormalities and on automatic measurements in images. We introduce multiple AI systems for the brain, heart, lung, prostate, and musculoskeletal disease. The third level is exemplified by the integrated nature of the clinical data in a patient-specific manner. The AI algorithms at this level focus on risk prediction and stratification, as opposed to merely detecting, measuring, and quantifying images. An AI-based approach for individualizing radiation dose in lung stereotactic body radiotherapy is discussed. The digital twin is presented as a concept of individualized computational modeling of human physiology. Finally, at the cohort and population analysis levels, the focus of AI shifts from clinical decision-making to operational decisions and process optimization.

Extracting and Leveraging Nodule Features with Lung Inpainting for Local Feature Augmentation

Abstract: Chest X-ray (CXR) is the most common examination for fast detection of pulmonary abnormalities. Recently, automated algorithms have been developed to classify multiple diseases and abnormalities in CXR scans. However, because of the limited availability of scans containing nodules and the subtle properties of nodules in CXRs, state-of-the-art methods do not perform well on nodule classification. To create additional data for the training process, standard augmentation techniques are applied. However, the variance introduced by these methods are limited as the images are typically modified globally. In this paper, we propose a method for local feature augmentation by extracting local nodule features using a generative inpainting network. The network is applied to generate realistic, healthy tissue and structures in patches containing nodules. The nodules are entirely removed in the inpainted representation. The extraction of the nodule features is processed by subtraction of the inpainted patch from the nodule patch. With arbitrary displacement of the extracted nodules in the lung area across different CXR scans and further local modifications during training, we significantly increase the nodule classification performance and outperform state-of-the-art augmentation methods.

Class-Aware Adversarial Pulmonary Nodule Synthesis

Abstract: Systems and methods are provided for generating a synthesized medical image patch of a nodule. An initial medical image patch and a class label associated with a nodule to be synthesized are received. The initial medical image patch has a masked portion and an unmasked portion. A synthesized medical image patch is generated using a trained generative adversarial network. The synthesized medical image patch includes the unmasked portion of the initial medical image patch and a synthesized nodule replacing the masked portion of the initial medical image patch. The synthesized nodule is synthesized according to the class label. The synthesized medical image patch is output.

Determining Malignancy of Pulmonary Nodules using Deep Learning

Abstract: Systems and method are described for determining a malignancy of a nodule. A medical image of a nodule of a patient is received. A patch surrounding the nodule is identified in the medical image. A malignancy of the nodule in the patch is predicted using a trained deep image-to-image network.

Extracting and Leveraging Nodule Features with Lung Inpainting for Local Feature Augmentation

Abstract: Chest X-ray (CXR) is the most common examination for fast detection of pulmonary abnormalities. Recently, automated algorithms have been developed to classify multiple diseases and abnormalities in CXR scans. However, because of the limited availability of scans containing nodules and the subtle properties of nodules in CXRs, state-of-the-art methods do not perform well on nodule classification. To create additional data for the training process, standard augmentation techniques are applied. However, the variance introduced by these methods are limited as the images are typically modified globally. In this paper, we propose a method for local feature augmentation by extracting local nodule features using a generative inpainting network. The network is applied to generate realistic, healthy tissue and structures in patches containing nodules. The nodules are entirely removed in the inpainted representation. The extraction of the nodule features is processed by subtraction of the inpainted patch from the nodule patch. With arbitrary displacement of the extracted nodules in the lung area across different CXR scans and further local modifications during training, we significantly increase the nodule classification performance and outperform state-of-the-art augmentation methods.

Learning cardiac anatomy

Abstract: An essential prerequisite for constructing the heart model is fast and robust parsing of the cardiovascular anatomy based on image data. This entails the detection, segmentation and tracking of anatomical structures or pathologies in the human heart and vascular system. Current solutions for these problems are based on machine learning and require large annotated image databases for effective training. In practice, however, these techniques often suffer from inherent limitations related to the efficiency in scanning high-dimensional parametric spaces and the learning of representative features for describing the image content. In this chapter, we present several established techniques for cardiac image parsing and structure tracking. For image parsing, we describe the marginal space learning framework, including the original version of the system that relies on handcrafted steerable features, as well as the modern redesign of the framework based on the latest automatic feature learning technology using deep learning. To address the limitations of these techniques that rely on exhaustive search, we present the concept of intelligent image parsing. Based on deep reinforcement learning and scale-space theory, this approach enables the efficient parsing of high-resolution volumetric data in real-time. Several experiments are included to analyze the performance of these methods on different problems using large datasets. This chapter also briefly describes a modern deep image-to-image neural network architecture for whole heart isolation. For cardiac structure tracking, a comprehensive review is presented of state-of-the-art structure tracking methods based on convolutional neural networks and recurrent neural networks.