Segmentation and classification in MRI and US fetal imaging: Recent trends and future prospects.
TL;DR: This review covers state‐of‐the‐art segmentation and classification methodologies for the whole fetus and, more specifically, the fetal brain, lungs, liver, heart and placenta in magnetic resonance imaging and (3D) ultrasound for the first time.
About: This article is published in Medical Image Analysis.The article was published on 2019-01-01. It has received 70 citations till now.
Citations
More filters
TL;DR: CA-Net as mentioned in this paper proposes a joint spatial attention module to make the network focus more on the foreground region and a novel channel attention module is proposed to adaptively recalibrate channel-wise feature responses and highlight the most relevant feature channels.
Abstract: Accurate medical image segmentation is essential for diagnosis and treatment planning of diseases. Convolutional Neural Networks (CNNs) have achieved state-of-the-art performance for automatic medical image segmentation. However, they are still challenged by complicated conditions where the segmentation target has large variations of position, shape and scale, and existing CNNs have a poor explainability that limits their application to clinical decisions. In this work, we make extensive use of multiple attentions in a CNN architecture and propose a comprehensive attention-based CNN (CA-Net) for more accurate and explainable medical image segmentation that is aware of the most important spatial positions, channels and scales at the same time. In particular, we first propose a joint spatial attention module to make the network focus more on the foreground region. Then, a novel channel attention module is proposed to adaptively recalibrate channel-wise feature responses and highlight the most relevant feature channels. Also, we propose a scale attention module implicitly emphasizing the most salient feature maps among multiple scales so that the CNN is adaptive to the size of an object. Extensive experiments on skin lesion segmentation from ISIC 2018 and multi-class segmentation of fetal MRI found that our proposed CA-Net significantly improved the average segmentation Dice score from 87.77% to 92.08% for skin lesion, 84.79% to 87.08% for the placenta and 93.20% to 95.88% for the fetal brain respectively compared with U-Net. It reduced the model size to around 15 times smaller with close or even better accuracy compared with state-of-the-art DeepLabv3+. In addition, it has a much higher explainability than existing networks by visualizing the attention weight maps. Our code is available at https://github.com/HiLab-git/CA-Net .
205 citations
TL;DR: This work makes extensive use of multiple attentions in a CNN architecture and proposes a comprehensive attention-based CNN (CA-Net) for more accurate and explainable medical image segmentation that is aware of the most important spatial positions, channels and scales at the same time.
Abstract: Accurate medical image segmentation is essential for diagnosis and treatment planning of diseases. Convolutional Neural Networks (CNNs) have achieved state-of-the-art performance for automatic medical image segmentation. However, they are still challenged by complicated conditions where the segmentation target has large variations of position, shape and scale, and existing CNNs have a poor explainability that limits their application to clinical decisions. In this work, we make extensive use of multiple attentions in a CNN architecture and propose a comprehensive attention-based CNN (CA-Net) for more accurate and explainable medical image segmentation that is aware of the most important spatial positions, channels and scales at the same time. In particular, we first propose a joint spatial attention module to make the network focus more on the foreground region. Then, a novel channel attention module is proposed to adaptively recalibrate channel-wise feature responses and highlight the most relevant feature channels. Also, we propose a scale attention module implicitly emphasizing the most salient feature maps among multiple scales so that the CNN is adaptive to the size of an object. Extensive experiments on skin lesion segmentation from ISIC 2018 and multi-class segmentation of fetal MRI found that our proposed CA-Net significantly improved the average segmentation Dice score from 87.77% to 92.08% for skin lesion, 84.79% to 87.08% for the placenta and 93.20% to 95.88% for the fetal brain respectively compared with U-Net. It reduced the model size to around 15 times smaller with close or even better accuracy compared with state-of-the-art DeepLabv3+. In addition, it has a much higher explainability than existing networks by visualizing the attention weight maps. Our code is available at this https URL
174 citations
Cites background from "Segmentation and classification in ..."
...and the placenta is important for fetal growth assessment and motion correction [41]....
[...]
TL;DR: Deep features are extracted from the inceptionv3 model, in which score vector is acquired from softmax and supplied to the quantum variational classifier (QVR) for discrimination between glioma, meningiomas, no tumor, and pituitary tumor to prove the proposed model's effectiveness.
Abstract: A brain tumor is an abnormal enlargement of cells if not properly diagnosed. Early detection of a brain tumor is critical for clinical practice and survival rates. Brain tumors arise in a variety of shapes, sizes, and features, with variable treatment options. Manual detection of tumors is difficult, time-consuming, and error-prone. Therefore, a significant requirement for computerized diagnostics systems for accurate brain tumor detection is present. In this research, deep features are extracted from the inceptionv3 model, in which score vector is acquired from softmax and supplied to the quantum variational classifier (QVR) for discrimination between glioma, meningioma, no tumor, and pituitary tumor. The classified tumor images have been passed to the proposed Seg-network where the actual infected region is segmented to analyze the tumor severity level. The outcomes of the reported research have been evaluated on three benchmark datasets such as Kaggle, 2020-BRATS, and local collected images. The model achieved greater than 90% detection scores to prove the proposed model's effectiveness.
22 citations
TL;DR: Wang et al. as discussed by the authors proposed a simple yet effective residual learning diagnosis system (RLDS) for diagnosing fetal CHD to improve diagnostic accuracy, which adopts convolutional neural networks to extract discriminative features of the fetal cardiac anatomical structures.
20 citations
TL;DR: The results suggest that the model used has a high potential to help cardiologists complete the initial screening for fetal congenital heart disease and a strong correlation between the predicted septal defects and ground truth as a mean average precision (mAP).
Abstract: Accurate screening for septal defects is important for supporting radiologists’ interpretative work. Some previous studies have proposed semantic segmentation and object detection approaches to carry out fetal heart detection; unfortunately, the models could not segment different objects of the same class. The semantic segmentation method segregates regions that only contain objects from the same class. In contrast, the fetal heart may contain multiple objects, such as the atria, ventricles, valves, and aorta. Besides, blurry boundaries (shadows) or a lack of consistency in the acquisition ultrasonography can cause wide variations. This study utilizes Mask-RCNN (MRCNN) to handle fetal ultrasonography images and employ it to detect and segment defects in heart walls with multiple objects. To our knowledge, this is the first study involving a medical application for septal defect detection using instance segmentation. The use of MRCNN architecture with ResNet50 as a backbone and a 0.0001 learning rate allows for two times faster training of the model on fetal heart images compared to other object detection methods, such as Faster-RCNN (FRCNN). We demonstrate a strong correlation between the predicted septal defects and ground truth as a mean average precision (mAP). As shown in the results, the proposed MRCNN model achieves good performance in multiclass detection of the heart chamber, with 97.59% for the right atrium, 99.67% for the left atrium, 86.17% for the left ventricle, 98.83% for the right ventricle, and 99.97% for the aorta. We also report competitive results for the defect detection of holes in the atria and ventricles via semantic and instance segmentation. The results show that the mAP for MRCNN is about 99.48% and 82% for FRCNN. We suggest that evaluation and prediction with our proposed model provide reliable detection of septal defects, including defects in the atria, ventricles, or both. These results suggest that the model used has a high potential to help cardiologists complete the initial screening for fetal congenital heart disease.
19 citations
Cites background or methods from "Segmentation and classification in ..."
...Unfortunately, such methods (with threshold-based techniques, for example) yield the best results when the regions of interest in an image exhibit a massive difference in strength from the background of the image, but this results in more similar images with problems, dramatically reducing the efficiency and decreasing the applicability of these methods [6], [27]....
[...]
...It can aid doctors in making more accurate treatment plans [27]....
[...]
...The segmentation process is the key to exploring fetal heart abnormalities, especially defect conditions [27]....
[...]
References
More filters
TL;DR: It is demonstrated that in about a third of stillbirths there is a congenital hypodevelopment of both lung and arcuate nucleus and in these cases the ARCn hypoplasia would exert a negative effect on respiratory movements in utero and therefore on lung development.
Abstract: OBJECTIVE: To investigate lung development and to correlate pulmonary hypoplasia with hypoplasia of the arcuate nucleus in stillbirths STUDY DESIGN: We examined 26 stillbirths which occurred after 25 complete gestational weeks The brainstem and the lung were the particular focus of this study The brainstem was examined according to the protocol routinely followed in our Institute As regards the lung examination, the development stage was evaluated on the basis of the correlation between lung and body weight (LW/BW), and according to microscopic parameters, that is, the presence of cartilaginous bronchi up to the distal level and the radial alveolar count (RAC) The normal reference values for the last 3 months of gestation correspond to >0022 for LW/BW and from 22 to 44 for RAC RESULTS: In 17 cases (65%) pulmonary hypoplasia was observed, characterized by a LW/BW value below 0022 and RAC below 22 In nine cases (35%), microscopic examination of brainstem serial sections showed varying degrees of hypoplasia of the arcuate nucleus (ARCn) In eight cases (31%) the pulmonary hypoplasia was associated with hypoplasia/agenesis of the ARCn CONCLUSIONS: This study demonstrated that in about a third of stillbirths there is a congenital hypodevelopment of both lung and arcuate nucleus In these cases the ARCn hypoplasia would exert a negative effect on respiratory movements in utero and therefore on lung development When the pulmonary hypoplasia is not accompanied by hypodevelopment of this nucleus the explanation could be a failure to block the inhibitory action of the Kolliker–Fuse nucleus
45 citations
Posted Content•
TL;DR: In this article, an attention-gated network is applied to real-time automated scan plane detection for fetal ultrasound screening, which can be easily incorporated into any existing classification architectures, while only requiring a few additional parameters.
Abstract: In this work, we apply an attention-gated network to real-time automated scan plane detection for fetal ultrasound screening. Scan plane detection in fetal ultrasound is a challenging problem due the poor image quality resulting in low interpretability for both clinicians and automated algorithms. To solve this, we propose incorporating self-gated soft-attention mechanisms. A soft-attention mechanism generates a gating signal that is end-to-end trainable, which allows the network to contextualise local information useful for prediction. The proposed attention mechanism is generic and it can be easily incorporated into any existing classification architectures, while only requiring a few additional parameters. We show that, when the base network has a high capacity, the incorporated attention mechanism can provide efficient object localisation while improving the overall performance. When the base network has a low capacity, the method greatly outperforms the baseline approach and significantly reduces false positives. Lastly, the generated attention maps allow us to understand the model's reasoning process, which can also be used for weakly supervised object localisation.
43 citations
18 Apr 2017
TL;DR: This work provides a state-of-art solution for detecting the fetal heart and classifying each individual frame as belonging to one of the standard viewing planes using fully convolutional neural networks (FCNs).
Abstract: Automatic analysis of fetal echocardiography screening images could aid in the identification of congenital heart diseases. The first step towards automatic fetal echocardiography analysis is locating the fetal heart in an image and identifying the viewing (imaging) plane. This is highly challenging since the fetal heart is small with relatively indistinct anatomical structural appearance. This is further compounded by the presence of artefacts in ultrasound images. Herein we provide a state-of-art solution for detecting the fetal heart and classifying each individual frame as belonging to one of the standard viewing planes using fully convolutional neural networks (FCNs). Our FCN model achieves a classification error rate of 23.48% on real-world clinical ultrasound data. We also present comparative performance for analysis of different FCN architectures.
43 citations
TL;DR: The RW algorithm was found to provide results concordant with those for manual segmentation and to outperform VOCAL in aspects of observer reliability, and with appropriate training, the RW method can be used for fast, repeatable 3-D measurement of placental volume.
Abstract: Volumetric segmentation of the placenta using 3-D ultrasound is currently performed clinically to investigate correlation between organ volume and fetal outcome or pathology Previously, interpolative or semi-automatic contour-based methodologies were used to provide volumetric results We describe the validation of an original random walker (RW)-based algorithm against manual segmentation and an existing semi-automated method, virtual organ computer-aided analysis (VOCAL), using initialization time, inter- and intra-observer variability of volumetric measurements and quantification accuracy (with respect to manual segmentation) as metrics of success Both semi-automatic methods require initialization Therefore, the first experiment compared initialization times Initialization was timed by one observer using 20 subjects This revealed significant differences (p < 0001) in time taken to initialize the VOCAL method compared with the RW method In the second experiment, 10 subjects were used to analyze intra-/inter-observer variability between two observers Bland-Altman plots were used to analyze variability combined with intra- and inter-observer variability measured by intra-class correlation coefficients, which were reported for all three methods Intra-class correlation coefficient values for intra-observer variability were higher for the RW method than for VOCAL, and both were similar to manual segmentation Inter-observer variability was 094 (088, 097), 091 (081, 095) and 080 (061, 090) for manual, RW and VOCAL, respectively Finally, a third observer with no prior ultrasound experience was introduced and volumetric differences from manual segmentation were reported Dice similarity coefficients for observers 1, 2 and 3 were respectively 084 ± 012, 094 ± 008 and 084 ± 011, and the mean was 087 ± 013 The RW algorithm was found to provide results concordant with those for manual segmentation and to outperform VOCAL in aspects of observer reliability The training of an additional untrained observer was investigated, and results revealed that with the appropriate initialization protocol, results for observers with varying levels of experience were concordant We found that with appropriate training, the RW method can be used for fast, repeatable 3-D measurement of placental volume
43 citations
TL;DR: To develop a novel application of a tool for semi‐automatic volume segmentation and adapt it for analysis of fetal cardiac cavities and vessels from heart volume datasets.
Abstract: Objective
To develop a novel application of a tool for semi-automatic volume segmentation and adapt it for analysis of fetal cardiac cavities and vessels from heart volume datasets.
Methods
We studied retrospectively virtual cardiac volume cycles obtained with spatiotemporal image correlation (STIC) from six fetuses with postnatally confirmed diagnoses: four with normal hearts between 19 and 29 completed gestational weeks, one with d-transposition of the great arteries and one with hypoplastic left heart syndrome. The volumes were analyzed offline using a commercially available segmentation algorithm designed for ovarian folliculometry. Using this software, individual ‘cavities’ in a static volume are selected and assigned individual colors in cross-sections and in 3D-rendered views, and their dimensions (diameters and volumes) can be calculated.
Results
Individual segments of fetal cardiac cavities could be separated, adjacent segments merged and the resulting electronic casts studied in their spatial context. Volume measurements could also be performed. Exemplary images and interactive videoclips showing the segmented digital casts were generated.
Conclusion
The approach presented here is an important step towards an automated fetal volume echocardiogram. It has the potential both to help in obtaining a correct structural diagnosis, and to generate exemplary visual displays of cardiac anatomy in normal and structurally abnormal cases for consultation and teaching. Copyright © 2008 ISUOG. Published by John Wiley & Sons, Ltd.
43 citations