Segmentation and classification in MRI and US fetal imaging: Recent trends and future prospects.
TL;DR: This review covers state‐of‐the‐art segmentation and classification methodologies for the whole fetus and, more specifically, the fetal brain, lungs, liver, heart and placenta in magnetic resonance imaging and (3D) ultrasound for the first time.
About: This article is published in Medical Image Analysis.The article was published on 2019-01-01. It has received 70 citations till now.
Citations
More filters
TL;DR: CA-Net as mentioned in this paper proposes a joint spatial attention module to make the network focus more on the foreground region and a novel channel attention module is proposed to adaptively recalibrate channel-wise feature responses and highlight the most relevant feature channels.
Abstract: Accurate medical image segmentation is essential for diagnosis and treatment planning of diseases. Convolutional Neural Networks (CNNs) have achieved state-of-the-art performance for automatic medical image segmentation. However, they are still challenged by complicated conditions where the segmentation target has large variations of position, shape and scale, and existing CNNs have a poor explainability that limits their application to clinical decisions. In this work, we make extensive use of multiple attentions in a CNN architecture and propose a comprehensive attention-based CNN (CA-Net) for more accurate and explainable medical image segmentation that is aware of the most important spatial positions, channels and scales at the same time. In particular, we first propose a joint spatial attention module to make the network focus more on the foreground region. Then, a novel channel attention module is proposed to adaptively recalibrate channel-wise feature responses and highlight the most relevant feature channels. Also, we propose a scale attention module implicitly emphasizing the most salient feature maps among multiple scales so that the CNN is adaptive to the size of an object. Extensive experiments on skin lesion segmentation from ISIC 2018 and multi-class segmentation of fetal MRI found that our proposed CA-Net significantly improved the average segmentation Dice score from 87.77% to 92.08% for skin lesion, 84.79% to 87.08% for the placenta and 93.20% to 95.88% for the fetal brain respectively compared with U-Net. It reduced the model size to around 15 times smaller with close or even better accuracy compared with state-of-the-art DeepLabv3+. In addition, it has a much higher explainability than existing networks by visualizing the attention weight maps. Our code is available at https://github.com/HiLab-git/CA-Net .
205 citations
TL;DR: This work makes extensive use of multiple attentions in a CNN architecture and proposes a comprehensive attention-based CNN (CA-Net) for more accurate and explainable medical image segmentation that is aware of the most important spatial positions, channels and scales at the same time.
Abstract: Accurate medical image segmentation is essential for diagnosis and treatment planning of diseases. Convolutional Neural Networks (CNNs) have achieved state-of-the-art performance for automatic medical image segmentation. However, they are still challenged by complicated conditions where the segmentation target has large variations of position, shape and scale, and existing CNNs have a poor explainability that limits their application to clinical decisions. In this work, we make extensive use of multiple attentions in a CNN architecture and propose a comprehensive attention-based CNN (CA-Net) for more accurate and explainable medical image segmentation that is aware of the most important spatial positions, channels and scales at the same time. In particular, we first propose a joint spatial attention module to make the network focus more on the foreground region. Then, a novel channel attention module is proposed to adaptively recalibrate channel-wise feature responses and highlight the most relevant feature channels. Also, we propose a scale attention module implicitly emphasizing the most salient feature maps among multiple scales so that the CNN is adaptive to the size of an object. Extensive experiments on skin lesion segmentation from ISIC 2018 and multi-class segmentation of fetal MRI found that our proposed CA-Net significantly improved the average segmentation Dice score from 87.77% to 92.08% for skin lesion, 84.79% to 87.08% for the placenta and 93.20% to 95.88% for the fetal brain respectively compared with U-Net. It reduced the model size to around 15 times smaller with close or even better accuracy compared with state-of-the-art DeepLabv3+. In addition, it has a much higher explainability than existing networks by visualizing the attention weight maps. Our code is available at this https URL
174 citations
Cites background from "Segmentation and classification in ..."
...and the placenta is important for fetal growth assessment and motion correction [41]....
[...]
TL;DR: Deep features are extracted from the inceptionv3 model, in which score vector is acquired from softmax and supplied to the quantum variational classifier (QVR) for discrimination between glioma, meningiomas, no tumor, and pituitary tumor to prove the proposed model's effectiveness.
Abstract: A brain tumor is an abnormal enlargement of cells if not properly diagnosed. Early detection of a brain tumor is critical for clinical practice and survival rates. Brain tumors arise in a variety of shapes, sizes, and features, with variable treatment options. Manual detection of tumors is difficult, time-consuming, and error-prone. Therefore, a significant requirement for computerized diagnostics systems for accurate brain tumor detection is present. In this research, deep features are extracted from the inceptionv3 model, in which score vector is acquired from softmax and supplied to the quantum variational classifier (QVR) for discrimination between glioma, meningioma, no tumor, and pituitary tumor. The classified tumor images have been passed to the proposed Seg-network where the actual infected region is segmented to analyze the tumor severity level. The outcomes of the reported research have been evaluated on three benchmark datasets such as Kaggle, 2020-BRATS, and local collected images. The model achieved greater than 90% detection scores to prove the proposed model's effectiveness.
22 citations
TL;DR: Wang et al. as discussed by the authors proposed a simple yet effective residual learning diagnosis system (RLDS) for diagnosing fetal CHD to improve diagnostic accuracy, which adopts convolutional neural networks to extract discriminative features of the fetal cardiac anatomical structures.
20 citations
TL;DR: The results suggest that the model used has a high potential to help cardiologists complete the initial screening for fetal congenital heart disease and a strong correlation between the predicted septal defects and ground truth as a mean average precision (mAP).
Abstract: Accurate screening for septal defects is important for supporting radiologists’ interpretative work. Some previous studies have proposed semantic segmentation and object detection approaches to carry out fetal heart detection; unfortunately, the models could not segment different objects of the same class. The semantic segmentation method segregates regions that only contain objects from the same class. In contrast, the fetal heart may contain multiple objects, such as the atria, ventricles, valves, and aorta. Besides, blurry boundaries (shadows) or a lack of consistency in the acquisition ultrasonography can cause wide variations. This study utilizes Mask-RCNN (MRCNN) to handle fetal ultrasonography images and employ it to detect and segment defects in heart walls with multiple objects. To our knowledge, this is the first study involving a medical application for septal defect detection using instance segmentation. The use of MRCNN architecture with ResNet50 as a backbone and a 0.0001 learning rate allows for two times faster training of the model on fetal heart images compared to other object detection methods, such as Faster-RCNN (FRCNN). We demonstrate a strong correlation between the predicted septal defects and ground truth as a mean average precision (mAP). As shown in the results, the proposed MRCNN model achieves good performance in multiclass detection of the heart chamber, with 97.59% for the right atrium, 99.67% for the left atrium, 86.17% for the left ventricle, 98.83% for the right ventricle, and 99.97% for the aorta. We also report competitive results for the defect detection of holes in the atria and ventricles via semantic and instance segmentation. The results show that the mAP for MRCNN is about 99.48% and 82% for FRCNN. We suggest that evaluation and prediction with our proposed model provide reliable detection of septal defects, including defects in the atria, ventricles, or both. These results suggest that the model used has a high potential to help cardiologists complete the initial screening for fetal congenital heart disease.
19 citations
Cites background or methods from "Segmentation and classification in ..."
...Unfortunately, such methods (with threshold-based techniques, for example) yield the best results when the regions of interest in an image exhibit a massive difference in strength from the background of the image, but this results in more similar images with problems, dramatically reducing the efficiency and decreasing the applicability of these methods [6], [27]....
[...]
...It can aid doctors in making more accurate treatment plans [27]....
[...]
...The segmentation process is the key to exploring fetal heart abnormalities, especially defect conditions [27]....
[...]
References
More filters
TL;DR: The aim of this study was to correlate the percentage of calcification defined by the clinician using a new software tool for calculating the extent of placental calcification with traditional ultrasound methods and with pregnancy outcome.
Abstract: Objectives
Placental calcification is associated with an increased risk of perinatal morbidity and mortality. The subjectivity of current ultrasound methods of assessment of placental calcification indicates that a more objective method is required. The aim of this study was to correlate the percentage of calcification defined by the clinician using a new software tool for calculating the extent of placental calcification with traditional ultrasound methods and with pregnancy outcome.
Methods
Ninety placental images were individually assessed. An upper threshold was defined, based on high intensity, to quantify calcification within the placenta. Output metrics were then produced including the overall percentage of calcification with respect to the total number of pixels within the region of interest. The results were correlated with traditional ultrasound methods of assessment of placental calcification and with pregnancy outcome.
Results
The results demonstrate a significant correlation between placental calcification, as defined using the software, and traditional methods of Grannum grading of placental calcification. Whilst correlation with perinatal outcome and cord pH was not significant as a result of small numbers, patients with placental calcification assessed using the computerized software at the upper quartile had higher rates of poor perinatal outcome when compared with those at the lower quartile (8/22 (36%) vs 3/23 (13%); P = 0.069).
Conclusion
These results suggest that this computerized software tool has the potential to become an alternative method of assessing placental calcification. Copyright © 2012 ISUOG. Published by John Wiley & Sons, Ltd.
14 citations
TL;DR: Experimental study based on second trimester cine-loop sequences confirms the suitability of the proposed technique for detection of heart chambers.
Abstract: This letter proposes an automated region mask for the detection of cardiac chambers from ultrasonic fetal heart biometry. The fetal biometry consists of two dimensional ultrasonic cine-loop sequences of apical four chamber view of fetal heart, which are comparatively The clinical motion information of individual frame is extracted by keeping a constant frame rate of 25 frames per second (fps). The region mask is designed based on the superimposition of motion information from a set of consecutive frames that belong to one cardiac cycle followed by connected component labelling. The borders and edges of all four chambers are thus recognized leading to formation of binary region mask. Experimental study based on second trimester cine-loop sequences confirms the suitability of the proposed technique for detection of heart chambers.
13 citations
01 Dec 2015
TL;DR: Experimental results demonstrate that the proposed MFV outperformed traditional methods for placental maturity staging and also achieves an area under the receiver of characteristics (AUC) of 96.77%, sensitivity of 98.04% and specificity of 93.75%, respectively.
Abstract: In this paper, a new method is proposed to automatically stage the placental maturity from B-mode ultrasound (US) images based on multi-layer Fisher vector (MFV) and densely sampled visual features. The proposed method first densely extracts visual features at a regular grid based on dense sampling instead of a few unreliable interest points. These features are clustered using generative Gaussian mixture model (GMM) to have soft clustering ability, and then learned discriminatively by Fisher vector (FV), which incorporates high-order statistics to enhance the staging accuracy. Differing from the previous studies, a multi-layer FV instead of a single layer FV is adopted in our method to exploit the spatial information of the features. Experimental results show that the proposed method achieves an area under the receiver of characteristics (AUC) of 96.77%, sensitivity of 98.04% and specificity of 93.75%, respectively, for staging placental maturity. Moreover, experimental results also demonstrate that the proposed MFV outperformed traditional methods for placental maturity staging.
13 citations
11 Nov 2010
TL;DR: Preliminary results show that the approach reported is able to automatically segment the cerebellum in 3D ultrasounds of fetal brains.
Abstract: Analysis of fetal biometric parameters on ultrasound images is widely performed and it is essential to estimate the gestational age, as well as the fetal growth pattern. The use of three dimensional ultrasound (3D US) is preferred over other tomographic modalities such as CT or MRI, due to its inherent safety and availability. However, the image quality of 3D US is not as good as MRI and therefore there is little work on the automatic segmentation of anatomic structures in 3D US of fetal brains. In this work we present preliminary results of the development of a 3D Point Distribution Model (PDM), for automatic segmentation, of the cerebellum in 3D US of the fetal brain. The model is adjusted to a fetal 3D ultrasound, using a genetic algorithm which optimizes a model fitting function. Preliminary results show that the approach reported is able to automatically segment the cerebellum in 3D ultrasounds of fetal brains.
13 citations
31 Jul 2014
TL;DR: Experimental results show that the hierarchical supervised learning framework for automatically detecting standard plane in consecutive 2D US images significantly outperforms both the full abdomen and the separate anatomy detection methods without geometric constrains.
Abstract: The acquisition of standard planes is crucial for medical ultrasound (US) diagnosis. In this paper, we present a hierarchical supervised learning framework for automatically detecting standard plane in consecutive 2D US images. The technique is demonstrated by developing a system that localizes fetal abdominal standard plane (FASP) from US videos. We first propose a novel radial component-based model (RCM) to describe the geometric constrains of key anatomical structures (KAS). In order to enhance the detection accuracy, we further adopt random forests classifier for detection of KAS within the regions constrained by RCM. Finally, a second-level classifier combines the results of component detectors to identify a US image as a “FASP” or a “nonFASP”. Experimental results show that our method significantly outperforms both the full abdomen and the separate anatomy detection methods without geometric constrains.
12 citations