Multilevel UNet for pancreas segmentation from non-contrast CT scans through domain adaptation
16 Mar 2020-Vol. 11314, pp 116-121
TL;DR: This work proposes a novel DL framework that can segment the pancreas from non-contrast CT scans through training with the help of IVC-enhanced CT scans and introduces a multilevel 3D UNet architecture to perform pancakes segmentation that significantly outperforms the baseline.
Abstract: A persistent issue in deep learning (DL) is the inability of models to function in a domain in which they were not trained. For example, a model trained to segment an organ in MRI scans often dramatically fails when tested in the domain of computed tomography (CT) scans. Since manual segmentation is extremely timeconsuming, it is often not feasible to acquire an annotated dataset in the target domain. Domain adaptation allows transfer of knowledge about a labelled source domain into a target domain. In this work, we attempt to address the differences in model performance when segmenting from intravenous contrast (IVC) enhanced or from non-contrast (NC) CT scans. Most of the publicly available, large-scale, annotated CT datasets are IVCenhanced. However, physicians frequently use NC scans in clinical practice. This necessitates methods capable of reliably functioning across both domains. We propose a novel DL framework that can segment the pancreas from non-contrast CT scans through training with the help of IVC-enhanced CT scans. Our method first utilizes a CycleGAN to create synthetic NC (s-NC) variants from IVC scans. Subsequently, we introduce a multilevel 3D UNet architecture to perform pancreas segmentation. The proposed method significantly outperforms the baseline. Experimental results show 6.2% percent improvement compared to the baseline model in terms of the Dice coefficient. To our knowledge, this method is the first of its kind in pancreas segmentation from NC CTs.
Citations
More filters
Journal Article•
3,940 citations
TL;DR: In this article, a supervised learning approach for the automatic estimation of the damage/severity level of the hit areas after the wildfire extinction is proposed, leveraging on the combination of a classification algorithm and a regression one.
Abstract: Wildfire damage severity census is a crucial activity for estimating monetary losses and for planning a prompt restoration of the affected areas It consists in assigning, after a wildfire, a numerical damage/severity level, between 0 and 4, to each sub-area of the hit area While burned area identification has been automatized by means of machine learning algorithms, the wildfire damage severity census operation is usually still performed manually and requires a significant effort of domain experts through the analysis of imagery and, sometimes, on-site missions In this paper, we propose a novel supervised learning approach for the automatic estimation of the damage/severity level of the hit areas after the wildfire extinction Specifically, the proposed approach, leveraging on the combination of a classification algorithm and a regression one, predicts the damage/severity level of the sub-areas of the area under analysis by processing a single post-fire satellite acquisition Our approach has been validated in five different European countries and on 21 wildfires It has proved to be robust for the application in several geographical contexts presenting similar geological aspects
30 citations
04 Oct 2020
TL;DR: In this article, a patch-based model using shared latent variables from a Gaussian mixture was proposed to preserve fine structures during medical image translation, which is significant for many clinical applications, such as segmenting small calcified plaques in the aorta and pelvic arteries.
Abstract: Current deep learning based segmentation models generalize poorly to different domains due to the lack of sufficient labelled image data. An important example in radiology is generalizing from contrast enhanced CT to non-contrast CT. In real-world clinical applications, cross-domain image analysis tools are in high demand since medical images from different domains are generally used to achieve precise diagnoses. For example, contrast enhanced CT at different phases are used to enhance certain pathologies or internal organs. Many existing cross-domain image-to-image translation models show impressive results on large organ segmentation by successfully preserving large structures across domains. However, such models lack the ability to preserve fine structures during the translation process, which is significant for many clinical applications, such as segmenting small calcified plaques in the aorta and pelvic arteries. In order to preserve fine structures during medical image translation, we propose a patch-based model using shared latent variables from a Gaussian mixture. We compare our image translation framework to several state-of-the-art methods on cross-domain image translation and show our model does a better job preserving fine structures. The superior performance of our model is verified by performing two tasks with the translated images - detection and segmentation of aortic plaques and pancreas segmentation. We expect the utility of our framework will extend to other problems beyond segmentation due to the improved quality of the generated images and enhanced ability to preserve small structures.
12 citations
TL;DR: The diagnosis of type 2 diabetes mellitus was associated with abdominal CT biomarkers, especially measures of pancreatic CT attenuation and visceral fat.
Abstract: Background CT biomarkers both inside and outside the pancreas can potentially be used to diagnose type 2 diabetes mellitus. Previous studies on this topic have shown significant results but were limited by manual methods and small study samples. Purpose To investigate abdominal CT biomarkers for type 2 diabetes mellitus in a large clinical data set using fully automated deep learning. Materials and Methods For external validation, noncontrast abdominal CT images were retrospectively collected from consecutive patients who underwent routine colorectal cancer screening with CT colonography from 2004 to 2016. The pancreas was segmented using a deep learning method that outputs measurements of interest, including CT attenuation, volume, fat content, and pancreas fractal dimension. Additional biomarkers assessed included visceral fat, atherosclerotic plaque, liver and muscle CT attenuation, and muscle volume. Univariable and multivariable analyses were performed, separating patients into groups based on time between type 2 diabetes diagnosis and CT date and including clinical factors such as sex, age, body mass index (BMI), BMI greater than 30 kg/m2, and height. The best set of predictors for type 2 diabetes were determined using multinomial logistic regression. Results A total of 8992 patients (mean age, 57 years ± 8 [SD]; 5009 women) were evaluated in the test set, of whom 572 had type 2 diabetes mellitus. The deep learning model had a mean Dice similarity coefficient for the pancreas of 0.69 ± 0.17, similar to the interobserver Dice similarity coefficient of 0.69 ± 0.09 (P = .92). The univariable analysis showed that patients with diabetes had, on average, lower pancreatic CT attenuation (mean, 18.74 HU ± 16.54 vs 29.99 HU ± 13.41; P < .0001) and greater visceral fat volume (mean, 235.0 mL ± 108.6 vs 130.9 mL ± 96.3; P < .0001) than those without diabetes. Patients with diabetes also showed a progressive decrease in pancreatic attenuation with greater duration of disease. The final multivariable model showed pairwise areas under the receiver operating characteristic curve (AUCs) of 0.81 and 0.85 between patients without and patients with diabetes who were diagnosed 0-2499 days before and after undergoing CT, respectively. In the multivariable analysis, adding clinical data did not improve upon CT-based AUC performance (AUC = 0.67 for the CT-only model vs 0.68 for the CT and clinical model). The best predictors of type 2 diabetes mellitus included intrapancreatic fat percentage, pancreatic fractal dimension, plaque severity between the L1 and L4 vertebra levels, average liver CT attenuation, and BMI. Conclusion The diagnosis of type 2 diabetes mellitus was associated with abdominal CT biomarkers, especially measures of pancreatic CT attenuation and visceral fat. © RSNA, 2022 Online supplemental material is available for this article.
10 citations
TL;DR: MMFlood as discussed by the authors is a large-scale multimodal remote sensing dataset designed for flood delineation, which contains 1,748 Sentinel-1 acquisitions, comprising 95 flood events distributed across 42 countries.
Abstract: Accurate flood delineation is crucial in many disaster management tasks, such as risk map production and update, impact estimation, claim verification, or planning of countermeasures for disaster risk reduction. Open remote sensing resources such as the data provided by the Copernicus ecosystem enable to carry out this activity, which benefits from frequent revisit times on a global scale. In the last decades, satellite imagery has been successfully applied to flood delineation problems, especially considering Synthetic Aperture Radar (SAR) signals. However, current remote mapping services rely on time-consuming manual or semi-automated approaches, requiring the intervention of domain experts. The implementation of accurate and scalable automated pipelines is hindered by the scarcity of large-scale annotated datasets. To address these issues, we propose MMFlood, a multimodal remote sensing dataset purposely designed for flood delineation. The dataset contains 1,748 Sentinel-1 acquisitions, comprising 95 flood events distributed across 42 countries. Along with satellite imagery, the dataset includes the Digital Elevation Model (DEM), hydrography maps, and flood delineation maps provided by Copernicus EMS, which is considered as ground truth. To provide baseline performances on the MMFlood test set, we conduct a number of experiments of the flood delineation task using state-of-art deep learning models, and we evaluate the performance gains of entropy-based sampling and multi-encoder architectures, which are respectively used to tackle two of the main challenges posed by MMFlood, namely the class unbalance and the multimodal setting. Lastly, we provide a future outlook on how to further improve the performance of the flood delineation task. Dataset and code can be found at https://github.com/edornd/mmflood.
3 citations
References
More filters
27 Jun 2016
TL;DR: In this article, the authors proposed a residual learning framework to ease the training of networks that are substantially deeper than those used previously, which won the 1st place on the ILSVRC 2015 classification task.
Abstract: Deeper neural networks are more difficult to train. We present a residual learning framework to ease the training of networks that are substantially deeper than those used previously. We explicitly reformulate the layers as learning residual functions with reference to the layer inputs, instead of learning unreferenced functions. We provide comprehensive empirical evidence showing that these residual networks are easier to optimize, and can gain accuracy from considerably increased depth. On the ImageNet dataset we evaluate residual nets with a depth of up to 152 layers—8× deeper than VGG nets [40] but still having lower complexity. An ensemble of these residual nets achieves 3.57% error on the ImageNet test set. This result won the 1st place on the ILSVRC 2015 classification task. We also present analysis on CIFAR-10 with 100 and 1000 layers. The depth of representations is of central importance for many visual recognition tasks. Solely due to our extremely deep representations, we obtain a 28% relative improvement on the COCO object detection dataset. Deep residual nets are foundations of our submissions to ILSVRC & COCO 2015 competitions1, where we also won the 1st places on the tasks of ImageNet detection, ImageNet localization, COCO detection, and COCO segmentation.
123,388 citations
17 Oct 2016
TL;DR: In this paper, the authors propose a network for volumetric segmentation that learns from sparsely annotated volumetrized images, which is trained end-to-end from scratch, i.e., no pre-trained network is required.
Abstract: This paper introduces a network for volumetric segmentation that learns from sparsely annotated volumetric images. We outline two attractive use cases of this method: (1) In a semi-automated setup, the user annotates some slices in the volume to be segmented. The network learns from these sparse annotations and provides a dense 3D segmentation. (2) In a fully-automated setup, we assume that a representative, sparsely annotated training set exists. Trained on this data set, the network densely segments new volumetric images. The proposed network extends the previous u-net architecture from Ronneberger et al. by replacing all 2D operations with their 3D counterparts. The implementation performs on-the-fly elastic deformations for efficient data augmentation during training. It is trained end-to-end from scratch, i.e., no pre-trained network is required. We test the performance of the proposed method on a complex, highly variable 3D structure, the Xenopus kidney, and achieve good results for both use cases.
4,629 citations
Journal Article•
3,940 citations
Posted Content•
TL;DR: A novel attention gate (AG) model for medical imaging that automatically learns to focus on target structures of varying shapes and sizes is proposed to eliminate the necessity of using explicit external tissue/organ localisation modules of cascaded convolutional neural networks (CNNs).
Abstract: We propose a novel attention gate (AG) model for medical imaging that automatically learns to focus on target structures of varying shapes and sizes. Models trained with AGs implicitly learn to suppress irrelevant regions in an input image while highlighting salient features useful for a specific task. This enables us to eliminate the necessity of using explicit external tissue/organ localisation modules of cascaded convolutional neural networks (CNNs). AGs can be easily integrated into standard CNN architectures such as the U-Net model with minimal computational overhead while increasing the model sensitivity and prediction accuracy. The proposed Attention U-Net architecture is evaluated on two large CT abdominal datasets for multi-class image segmentation. Experimental results show that AGs consistently improve the prediction performance of U-Net across different datasets and training sizes while preserving computational efficiency. The code for the proposed architecture is publicly available.
2,452 citations
"Multilevel UNet for pancreas segmen..." refers methods in this paper
...840.(8) Following this principle of focusing in on the pancreas ROI, the current state-of-the-art model uses a two-stage approach to first generate a rough segmentation from the entire CT volume using a standard 3D UNet, then use that rough segmentation to bound the pancreas ROI and input just that ROI into a second 3D UNet....
[...]
Posted Content•
TL;DR: A large, curated dataset representative of several highly variable segmentation tasks that was used in a crowd-sourced challenge - the Medical Segmentation Decathlon held during the 2018 Medical Image Computing and Computer Aided Interventions Conference in Granada, Spain.
Abstract: Semantic segmentation of medical images aims to associate a pixel with a label in a medical image without human initialization. The success of semantic segmentation algorithms is contingent on the availability of high-quality imaging data with corresponding labels provided by experts. We sought to create a large collection of annotated medical image datasets of various clinically relevant anatomies available under open source license to facilitate the development of semantic segmentation algorithms. Such a resource would allow: 1) objective assessment of general-purpose segmentation methods through comprehensive benchmarking and 2) open and free access to medical image data for any researcher interested in the problem domain. Through a multi-institutional effort, we generated a large, curated dataset representative of several highly variable segmentation tasks that was used in a crowd-sourced challenge - the Medical Segmentation Decathlon held during the 2018 Medical Image Computing and Computer Aided Interventions Conference in Granada, Spain. Here, we describe these ten labeled image datasets so that these data may be effectively reused by the research community.
588 citations