Segmentation and classification in MRI and US fetal imaging: Recent trends and future prospects.
TL;DR: This review covers state‐of‐the‐art segmentation and classification methodologies for the whole fetus and, more specifically, the fetal brain, lungs, liver, heart and placenta in magnetic resonance imaging and (3D) ultrasound for the first time.
About: This article is published in Medical Image Analysis.The article was published on 2019-01-01. It has received 70 citations till now.
Citations
More filters
TL;DR: CA-Net as mentioned in this paper proposes a joint spatial attention module to make the network focus more on the foreground region and a novel channel attention module is proposed to adaptively recalibrate channel-wise feature responses and highlight the most relevant feature channels.
Abstract: Accurate medical image segmentation is essential for diagnosis and treatment planning of diseases. Convolutional Neural Networks (CNNs) have achieved state-of-the-art performance for automatic medical image segmentation. However, they are still challenged by complicated conditions where the segmentation target has large variations of position, shape and scale, and existing CNNs have a poor explainability that limits their application to clinical decisions. In this work, we make extensive use of multiple attentions in a CNN architecture and propose a comprehensive attention-based CNN (CA-Net) for more accurate and explainable medical image segmentation that is aware of the most important spatial positions, channels and scales at the same time. In particular, we first propose a joint spatial attention module to make the network focus more on the foreground region. Then, a novel channel attention module is proposed to adaptively recalibrate channel-wise feature responses and highlight the most relevant feature channels. Also, we propose a scale attention module implicitly emphasizing the most salient feature maps among multiple scales so that the CNN is adaptive to the size of an object. Extensive experiments on skin lesion segmentation from ISIC 2018 and multi-class segmentation of fetal MRI found that our proposed CA-Net significantly improved the average segmentation Dice score from 87.77% to 92.08% for skin lesion, 84.79% to 87.08% for the placenta and 93.20% to 95.88% for the fetal brain respectively compared with U-Net. It reduced the model size to around 15 times smaller with close or even better accuracy compared with state-of-the-art DeepLabv3+. In addition, it has a much higher explainability than existing networks by visualizing the attention weight maps. Our code is available at https://github.com/HiLab-git/CA-Net .
205 citations
TL;DR: This work makes extensive use of multiple attentions in a CNN architecture and proposes a comprehensive attention-based CNN (CA-Net) for more accurate and explainable medical image segmentation that is aware of the most important spatial positions, channels and scales at the same time.
Abstract: Accurate medical image segmentation is essential for diagnosis and treatment planning of diseases. Convolutional Neural Networks (CNNs) have achieved state-of-the-art performance for automatic medical image segmentation. However, they are still challenged by complicated conditions where the segmentation target has large variations of position, shape and scale, and existing CNNs have a poor explainability that limits their application to clinical decisions. In this work, we make extensive use of multiple attentions in a CNN architecture and propose a comprehensive attention-based CNN (CA-Net) for more accurate and explainable medical image segmentation that is aware of the most important spatial positions, channels and scales at the same time. In particular, we first propose a joint spatial attention module to make the network focus more on the foreground region. Then, a novel channel attention module is proposed to adaptively recalibrate channel-wise feature responses and highlight the most relevant feature channels. Also, we propose a scale attention module implicitly emphasizing the most salient feature maps among multiple scales so that the CNN is adaptive to the size of an object. Extensive experiments on skin lesion segmentation from ISIC 2018 and multi-class segmentation of fetal MRI found that our proposed CA-Net significantly improved the average segmentation Dice score from 87.77% to 92.08% for skin lesion, 84.79% to 87.08% for the placenta and 93.20% to 95.88% for the fetal brain respectively compared with U-Net. It reduced the model size to around 15 times smaller with close or even better accuracy compared with state-of-the-art DeepLabv3+. In addition, it has a much higher explainability than existing networks by visualizing the attention weight maps. Our code is available at this https URL
174 citations
Cites background from "Segmentation and classification in ..."
...and the placenta is important for fetal growth assessment and motion correction [41]....
[...]
TL;DR: Deep features are extracted from the inceptionv3 model, in which score vector is acquired from softmax and supplied to the quantum variational classifier (QVR) for discrimination between glioma, meningiomas, no tumor, and pituitary tumor to prove the proposed model's effectiveness.
Abstract: A brain tumor is an abnormal enlargement of cells if not properly diagnosed. Early detection of a brain tumor is critical for clinical practice and survival rates. Brain tumors arise in a variety of shapes, sizes, and features, with variable treatment options. Manual detection of tumors is difficult, time-consuming, and error-prone. Therefore, a significant requirement for computerized diagnostics systems for accurate brain tumor detection is present. In this research, deep features are extracted from the inceptionv3 model, in which score vector is acquired from softmax and supplied to the quantum variational classifier (QVR) for discrimination between glioma, meningioma, no tumor, and pituitary tumor. The classified tumor images have been passed to the proposed Seg-network where the actual infected region is segmented to analyze the tumor severity level. The outcomes of the reported research have been evaluated on three benchmark datasets such as Kaggle, 2020-BRATS, and local collected images. The model achieved greater than 90% detection scores to prove the proposed model's effectiveness.
22 citations
TL;DR: Wang et al. as discussed by the authors proposed a simple yet effective residual learning diagnosis system (RLDS) for diagnosing fetal CHD to improve diagnostic accuracy, which adopts convolutional neural networks to extract discriminative features of the fetal cardiac anatomical structures.
20 citations
TL;DR: The results suggest that the model used has a high potential to help cardiologists complete the initial screening for fetal congenital heart disease and a strong correlation between the predicted septal defects and ground truth as a mean average precision (mAP).
Abstract: Accurate screening for septal defects is important for supporting radiologists’ interpretative work. Some previous studies have proposed semantic segmentation and object detection approaches to carry out fetal heart detection; unfortunately, the models could not segment different objects of the same class. The semantic segmentation method segregates regions that only contain objects from the same class. In contrast, the fetal heart may contain multiple objects, such as the atria, ventricles, valves, and aorta. Besides, blurry boundaries (shadows) or a lack of consistency in the acquisition ultrasonography can cause wide variations. This study utilizes Mask-RCNN (MRCNN) to handle fetal ultrasonography images and employ it to detect and segment defects in heart walls with multiple objects. To our knowledge, this is the first study involving a medical application for septal defect detection using instance segmentation. The use of MRCNN architecture with ResNet50 as a backbone and a 0.0001 learning rate allows for two times faster training of the model on fetal heart images compared to other object detection methods, such as Faster-RCNN (FRCNN). We demonstrate a strong correlation between the predicted septal defects and ground truth as a mean average precision (mAP). As shown in the results, the proposed MRCNN model achieves good performance in multiclass detection of the heart chamber, with 97.59% for the right atrium, 99.67% for the left atrium, 86.17% for the left ventricle, 98.83% for the right ventricle, and 99.97% for the aorta. We also report competitive results for the defect detection of holes in the atria and ventricles via semantic and instance segmentation. The results show that the mAP for MRCNN is about 99.48% and 82% for FRCNN. We suggest that evaluation and prediction with our proposed model provide reliable detection of septal defects, including defects in the atria, ventricles, or both. These results suggest that the model used has a high potential to help cardiologists complete the initial screening for fetal congenital heart disease.
19 citations
Cites background or methods from "Segmentation and classification in ..."
...Unfortunately, such methods (with threshold-based techniques, for example) yield the best results when the regions of interest in an image exhibit a massive difference in strength from the background of the image, but this results in more similar images with problems, dramatically reducing the efficiency and decreasing the applicability of these methods [6], [27]....
[...]
...It can aid doctors in making more accurate treatment plans [27]....
[...]
...The segmentation process is the key to exploring fetal heart abnormalities, especially defect conditions [27]....
[...]
References
More filters
11 Apr 2018
TL;DR: An automatic method for the detection of standard planes in 3D volumes by utilising a convolutional neural network to learn the relationship between a 2D plane image and the transformation parameters required to move that plane towards the corresponding standard plane is proposed.
Abstract: Standard scan plane detection in 3D fetal brain ultrasound (US) is a crucial step in the assessment of fetal brain development. We propose an automatic method for the detection of standard planes in 3D volumes by utilising a convolutional neural network (CNN) to learn the relationship between a 2D plane image and the transformation parameters required to move that plane towards the corresponding standard plane. In addition, we explore the effect of using two different training loss functions which exploit the geometric information and the image data of the extracted plane respectively. When evaluated on 72 subjects, our method achieves a plane detection error of 3.45mm and 12.4◦.
3 citations
06 Mar 2014
TL;DR: A novel method to measure the volume of a fetus from ultrasonic sound image using image processing technique and makes use of the differences of importance between different edges in the image.
Abstract: The paper presents a novel method to measure the volume of a fetus from ultrasonic sound image using image processing technique. Even though Ultrasonic image are poor in diagnostic quality and highly noise susceptible, they are preferred for the reason that Ultrasound imaging techniques were the safest for diagnosing pregnant women fetus, since they are almost free from hazardous radiation. Edge enhancement is made to improve this poor quality image which is to be used as input image for segmentation simultaneously smoothens the unwanted image edge to enhance only important edges of fetus. Anisotropic diffusion technique are used for image enhancement technique. Iso-intensity and Edge-focusing segmentation techniques were applied to US-images to obtain appropriate and clear edge image. By Iso-intensity contour approach, the pixel values with equal intensities were selected. Another contour selection technique, which had been used, was the Edge-focusing technique, which makes use of the differences of importance between different edges in the image. Determining the contour as a set of the points in the 3D-volume was the foremost and important step in fixing the shape of the fetus. The natural shape of body or head is like a sphere or an ellipsoid, both of these figures have been fitted. After the process of fitting an analytical figure has been completed, the volume of fetus can be calculated. These image processing techniques are applied to the fetal image through image processing tool in Matlab software. Post-processing operations are required after each techniques is used.
3 citations
TL;DR: This work presents a segmentation framework applied to fetal cardiac images based on a Point Distribution Model (PDM) and evaluates the proposed method in the segmentation of the left ventricle of fetal ultrasound data.
Abstract: In this work we present a segmentation framework applied to fetal cardiac images. One of the main problems of the segmentation in ultrasound images is the speckle pattern that makes difficult to model images features such as edges and homogeneous regions. Our approach is based on two main processes. The first one aims at enhancing the ultrasound image using a noise reduction scheme. The Hermite transform is used for this purpose. In the second process a Point Distribution Model (PDM), previously trained, is used for the segmentation of the desired object. The filtering process is then employed before the segmentation stage with the aim of improving the results. The obtained result in the filtering process is used as a way to make more robust the segmentation stage. We evaluate the proposed method in the segmentation of the left ventricle of fetal ultrasound data. Different metrics are used to validate and compare the performance with other methods applied to fetal echocardiographic images.
2 citations
TL;DR: The study findings confirmed the lack of reproducibility associated with Grannum grading of the placenta despite optimal viewing conditions and highlight the need for new methods of assessing placental health in order to improve neonatal outcomes.
Abstract: Current ultrasound assessment of placental calcification relies on Grannum grading. The aim of this study was to assess
if this method is reproducible by measuring inter- and intra-observer variation in grading placental images, under strictly
controlled viewing conditions. Thirty placental images were acquired and digitally saved. Five experienced sonographers
independently graded the images on two separate occasions. In order to eliminate any technological factors which could
affect data reliability and consistency all observers reviewed images at the same time. To optimise viewing conditions
ambient lighting was maintained between 25-40 lux, with monitors calibrated to the GSDF standard to ensure consistent
brightness and contrast. Kappa (κ) analysis of the grades assigned was used to measure inter- and intra-observer
reliability. Intra-observer agreement had a moderate mean κ-value of 0.55, with individual comparisons ranging from
0.30 to 0.86. Two images saved from the same patient, during the same scan, were each graded as I, II and III by the
same observer. A mean κ-value of 0.30 (range from 0.13 to 0.55) indicated fair inter-observer agreement over the two
occasions and only one image was graded consistently the same by all five observers. The study findings confirmed the
lack of reproducibility associated with Grannum grading of the placenta despite optimal viewing conditions and
highlight the need for new methods of assessing placental health in order to improve neonatal outcomes. Alternative
methods for quantifying placental calcification such as a software based technique and 3D ultrasound assessment need to
be explored.
2 citations
19 Nov 2013
TL;DR: In this article, the authors used two multiresolution methods: one based on wavelet decomposition and the other based on the Hermite transform for segmentation of the left ventricle in ultrasound images.
Abstract: In this paper, we propose to use filtering methods and a segmentation algorithm for the analysis of fetal heart in ultrasound images. Since noise speckle makes difficult the analysis of ultrasound images, the filtering process becomes a useful task in these types of applications. The filtering techniques consider in this work assume that the speckle noise is a random variable with a Rayleigh distribution. We use two multiresolution methods: one based on wavelet decomposition and the another based on the Hermite transform. The filtering process is used as way to strengthen the performance of the segmentation tasks. For the wavelet-based approach, a Bayesian estimator at subband level for pixel classification is employed. The Hermite method computes a mask to find those pixels that are corrupted by speckle. On the other hand, we picked out a method based on a deformable model or "snake" to evaluate the influence of the filtering techniques in the segmentation task of left ventricle in fetal echocardiographic images.
1 citations