Segmentation and classification in MRI and US fetal imaging: Recent trends and future prospects.
TL;DR: This review covers state‐of‐the‐art segmentation and classification methodologies for the whole fetus and, more specifically, the fetal brain, lungs, liver, heart and placenta in magnetic resonance imaging and (3D) ultrasound for the first time.
About: This article is published in Medical Image Analysis.The article was published on 2019-01-01. It has received 70 citations till now.
Citations
More filters
TL;DR: CA-Net as mentioned in this paper proposes a joint spatial attention module to make the network focus more on the foreground region and a novel channel attention module is proposed to adaptively recalibrate channel-wise feature responses and highlight the most relevant feature channels.
Abstract: Accurate medical image segmentation is essential for diagnosis and treatment planning of diseases. Convolutional Neural Networks (CNNs) have achieved state-of-the-art performance for automatic medical image segmentation. However, they are still challenged by complicated conditions where the segmentation target has large variations of position, shape and scale, and existing CNNs have a poor explainability that limits their application to clinical decisions. In this work, we make extensive use of multiple attentions in a CNN architecture and propose a comprehensive attention-based CNN (CA-Net) for more accurate and explainable medical image segmentation that is aware of the most important spatial positions, channels and scales at the same time. In particular, we first propose a joint spatial attention module to make the network focus more on the foreground region. Then, a novel channel attention module is proposed to adaptively recalibrate channel-wise feature responses and highlight the most relevant feature channels. Also, we propose a scale attention module implicitly emphasizing the most salient feature maps among multiple scales so that the CNN is adaptive to the size of an object. Extensive experiments on skin lesion segmentation from ISIC 2018 and multi-class segmentation of fetal MRI found that our proposed CA-Net significantly improved the average segmentation Dice score from 87.77% to 92.08% for skin lesion, 84.79% to 87.08% for the placenta and 93.20% to 95.88% for the fetal brain respectively compared with U-Net. It reduced the model size to around 15 times smaller with close or even better accuracy compared with state-of-the-art DeepLabv3+. In addition, it has a much higher explainability than existing networks by visualizing the attention weight maps. Our code is available at https://github.com/HiLab-git/CA-Net .
205 citations
TL;DR: This work makes extensive use of multiple attentions in a CNN architecture and proposes a comprehensive attention-based CNN (CA-Net) for more accurate and explainable medical image segmentation that is aware of the most important spatial positions, channels and scales at the same time.
Abstract: Accurate medical image segmentation is essential for diagnosis and treatment planning of diseases. Convolutional Neural Networks (CNNs) have achieved state-of-the-art performance for automatic medical image segmentation. However, they are still challenged by complicated conditions where the segmentation target has large variations of position, shape and scale, and existing CNNs have a poor explainability that limits their application to clinical decisions. In this work, we make extensive use of multiple attentions in a CNN architecture and propose a comprehensive attention-based CNN (CA-Net) for more accurate and explainable medical image segmentation that is aware of the most important spatial positions, channels and scales at the same time. In particular, we first propose a joint spatial attention module to make the network focus more on the foreground region. Then, a novel channel attention module is proposed to adaptively recalibrate channel-wise feature responses and highlight the most relevant feature channels. Also, we propose a scale attention module implicitly emphasizing the most salient feature maps among multiple scales so that the CNN is adaptive to the size of an object. Extensive experiments on skin lesion segmentation from ISIC 2018 and multi-class segmentation of fetal MRI found that our proposed CA-Net significantly improved the average segmentation Dice score from 87.77% to 92.08% for skin lesion, 84.79% to 87.08% for the placenta and 93.20% to 95.88% for the fetal brain respectively compared with U-Net. It reduced the model size to around 15 times smaller with close or even better accuracy compared with state-of-the-art DeepLabv3+. In addition, it has a much higher explainability than existing networks by visualizing the attention weight maps. Our code is available at this https URL
174 citations
Cites background from "Segmentation and classification in ..."
...and the placenta is important for fetal growth assessment and motion correction [41]....
[...]
TL;DR: Deep features are extracted from the inceptionv3 model, in which score vector is acquired from softmax and supplied to the quantum variational classifier (QVR) for discrimination between glioma, meningiomas, no tumor, and pituitary tumor to prove the proposed model's effectiveness.
Abstract: A brain tumor is an abnormal enlargement of cells if not properly diagnosed. Early detection of a brain tumor is critical for clinical practice and survival rates. Brain tumors arise in a variety of shapes, sizes, and features, with variable treatment options. Manual detection of tumors is difficult, time-consuming, and error-prone. Therefore, a significant requirement for computerized diagnostics systems for accurate brain tumor detection is present. In this research, deep features are extracted from the inceptionv3 model, in which score vector is acquired from softmax and supplied to the quantum variational classifier (QVR) for discrimination between glioma, meningioma, no tumor, and pituitary tumor. The classified tumor images have been passed to the proposed Seg-network where the actual infected region is segmented to analyze the tumor severity level. The outcomes of the reported research have been evaluated on three benchmark datasets such as Kaggle, 2020-BRATS, and local collected images. The model achieved greater than 90% detection scores to prove the proposed model's effectiveness.
22 citations
TL;DR: Wang et al. as discussed by the authors proposed a simple yet effective residual learning diagnosis system (RLDS) for diagnosing fetal CHD to improve diagnostic accuracy, which adopts convolutional neural networks to extract discriminative features of the fetal cardiac anatomical structures.
20 citations
TL;DR: The results suggest that the model used has a high potential to help cardiologists complete the initial screening for fetal congenital heart disease and a strong correlation between the predicted septal defects and ground truth as a mean average precision (mAP).
Abstract: Accurate screening for septal defects is important for supporting radiologists’ interpretative work. Some previous studies have proposed semantic segmentation and object detection approaches to carry out fetal heart detection; unfortunately, the models could not segment different objects of the same class. The semantic segmentation method segregates regions that only contain objects from the same class. In contrast, the fetal heart may contain multiple objects, such as the atria, ventricles, valves, and aorta. Besides, blurry boundaries (shadows) or a lack of consistency in the acquisition ultrasonography can cause wide variations. This study utilizes Mask-RCNN (MRCNN) to handle fetal ultrasonography images and employ it to detect and segment defects in heart walls with multiple objects. To our knowledge, this is the first study involving a medical application for septal defect detection using instance segmentation. The use of MRCNN architecture with ResNet50 as a backbone and a 0.0001 learning rate allows for two times faster training of the model on fetal heart images compared to other object detection methods, such as Faster-RCNN (FRCNN). We demonstrate a strong correlation between the predicted septal defects and ground truth as a mean average precision (mAP). As shown in the results, the proposed MRCNN model achieves good performance in multiclass detection of the heart chamber, with 97.59% for the right atrium, 99.67% for the left atrium, 86.17% for the left ventricle, 98.83% for the right ventricle, and 99.97% for the aorta. We also report competitive results for the defect detection of holes in the atria and ventricles via semantic and instance segmentation. The results show that the mAP for MRCNN is about 99.48% and 82% for FRCNN. We suggest that evaluation and prediction with our proposed model provide reliable detection of septal defects, including defects in the atria, ventricles, or both. These results suggest that the model used has a high potential to help cardiologists complete the initial screening for fetal congenital heart disease.
19 citations
Cites background or methods from "Segmentation and classification in ..."
...Unfortunately, such methods (with threshold-based techniques, for example) yield the best results when the regions of interest in an image exhibit a massive difference in strength from the background of the image, but this results in more similar images with problems, dramatically reducing the efficiency and decreasing the applicability of these methods [6], [27]....
[...]
...It can aid doctors in making more accurate treatment plans [27]....
[...]
...The segmentation process is the key to exploring fetal heart abnormalities, especially defect conditions [27]....
[...]
References
More filters
12 Apr 2007
TL;DR: To the authors' knowledge this is the first time these existing fetal cardiac non shape based segmentation algorithms have been modified for shape awareness in this way.
Abstract: The fetal heart has very thin intra-chamber walls which are often not resolved by ultrasound scanners and may drop out as a result of imaging. In order to measure blood volumes from all chambers in isolation, deformable model approaches were used to segment the chambers and fill in the missing structural information. Three level set algorithms in the fetal cardiac segmentation literature (two without and, one with the use of a shape prior) were applied to real ultrasound data. The shape prior term was extracted from the shape prior level set and incorporated into the amorphous snakes for a fairer comparison. To our knowledge this is the first time these existing fetal cardiac non shape based segmentation algorithms have been modified for shape awareness in this way
19 citations
07 Apr 2013
TL;DR: A segmentation framework is introduced that combines three different types of information: pixel intensity distribution, shape prior on the fetal envelope and a back model varying with fetus age, which shows satisfactory results on 3D obstetric ultrasound images.
Abstract: This paper presents a novel shape-guided variational segmentation method for extracting the fetus envelope on 3D obstetric ultrasound images. Indeed, due to the inherent low quality of these images, classical segmentation methods tend to fail at segmenting these data. To compensate for the lack of contrast and of explicit boundaries, we introduce a segmentation framework that combines three different types of information: pixel intensity distribution, shape prior on the fetal envelope and a back model varying with fetus age. The intensity distributions, different for each tissue, and the shape prior, encoded with Legendre moments, are added as energy terms in the functional to be optimized. The back model is used in a post-processing step. Results on 3D ultrasound data are presented and compared to a set of manual segmentations. Both visual and quantitative comparisons show the satisfactory results obtained by this method on the tested data.
18 citations
22 Sep 2013
TL;DR: Validation on 51 3D fetal neurosonography images shows that the proposed technique is capable of segmenting fetal brain structures and providing promising qualitative and quantitative results.
Abstract: Neurosonography is the most widely used imaging technique for assessing neuro-development of the growing fetus in clinical practice. 3D neurosonography has an advantage of quick acquisition but is yet to demonstrate improvements in clinical workflow. In this paper we propose an automatic technique to segment four important fetal brain structures in 3D ultrasound. The technique is built within a Random Decision Forests framework. Our solution includes novel pre-processing and new features. The pre-processing step makes sure that all volumes are in the same coordinate. The new features constrain the appearance framework by adding a novel distance feature. Validation on 51 3D fetal neurosonography images shows that the proposed technique is capable of segmenting fetal brain structures and providing promising qualitative and quantitative results.
18 citations
TL;DR: Quantitative sonography using texture analysis of the placenta was useful in practice to determine gestational age, and good agreement was found between multiple linear regression results for the three observers.
Abstract: Routine ultrasound screening to predict gestational age is important for risk assessment of pregnancy complications among pregnant women. We explored a quantitative method for sonographic analysis of placental texture, with the objective of reproducible measurement. We studied 151 pregnant women; the gestational ages of their fetuses ranged from 10 to 38 weeks. Three experienced sonographers delineated the placental contour to define the region of interest (ROI). From these sonograms, 72 texture features were derived from the spatial gray-level dependence matrices and gray-level difference matrices. We used these as input variables in a multiple linear regression analysis. A significant positive correlation (P < 0.01) was found between the multiple linear regression results and the corresponding gestation ages by the three assessors (r
A = 0.755, r
B = 0.851, and r
C = 0.832). We also found good agreement between multiple linear regression results for the three observers. Their κ statistic values were 0.685 between assessors A and B, 0.679 between A and C, and 0.804 between B and C. Quantitative sonography using texture analysis of the placenta was useful in practice to determine gestational age.
18 citations
15 Apr 2004
TL;DR: The percentage of correct classification or accuracy was computed for different subsets of features, with different sizes of the region of interest showing that generally a small number of features are enough for achieving the highest accuracy.
Abstract: The characterization of ultrasonic images of the placenta is one of the clinical procedures followed for assessing the progress of pregnancy. In this work, the classification of scans of the placenta according with Grannum grading is attempted. Feature selection was used for determining the relevant textural features that were extracted from the scans. Three different sets of textural features, namely cooccurrence matrices, Laws masks and neighborhood gray-tone difference matrices (NGTDM) were used. A set of 59 images corresponding to the four grades was sampled in subimages of different sizes, the textural features were computed and weighted using the relief-F algorithm. The strategy used for classification was the k-nearest neighbor algorithm using leave-one-out cross-validation. The percentage of correct classification or accuracy was computed for different subsets of features, with different sizes of the region of interest showing that generally a small number of features are enough for achieving the highest accuracy.
18 citations