Segmentation and classification in MRI and US fetal imaging: Recent trends and future prospects.
TL;DR: This review covers state‐of‐the‐art segmentation and classification methodologies for the whole fetus and, more specifically, the fetal brain, lungs, liver, heart and placenta in magnetic resonance imaging and (3D) ultrasound for the first time.
About: This article is published in Medical Image Analysis.The article was published on 2019-01-01. It has received 70 citations till now.
Citations
More filters
TL;DR: CA-Net as mentioned in this paper proposes a joint spatial attention module to make the network focus more on the foreground region and a novel channel attention module is proposed to adaptively recalibrate channel-wise feature responses and highlight the most relevant feature channels.
Abstract: Accurate medical image segmentation is essential for diagnosis and treatment planning of diseases. Convolutional Neural Networks (CNNs) have achieved state-of-the-art performance for automatic medical image segmentation. However, they are still challenged by complicated conditions where the segmentation target has large variations of position, shape and scale, and existing CNNs have a poor explainability that limits their application to clinical decisions. In this work, we make extensive use of multiple attentions in a CNN architecture and propose a comprehensive attention-based CNN (CA-Net) for more accurate and explainable medical image segmentation that is aware of the most important spatial positions, channels and scales at the same time. In particular, we first propose a joint spatial attention module to make the network focus more on the foreground region. Then, a novel channel attention module is proposed to adaptively recalibrate channel-wise feature responses and highlight the most relevant feature channels. Also, we propose a scale attention module implicitly emphasizing the most salient feature maps among multiple scales so that the CNN is adaptive to the size of an object. Extensive experiments on skin lesion segmentation from ISIC 2018 and multi-class segmentation of fetal MRI found that our proposed CA-Net significantly improved the average segmentation Dice score from 87.77% to 92.08% for skin lesion, 84.79% to 87.08% for the placenta and 93.20% to 95.88% for the fetal brain respectively compared with U-Net. It reduced the model size to around 15 times smaller with close or even better accuracy compared with state-of-the-art DeepLabv3+. In addition, it has a much higher explainability than existing networks by visualizing the attention weight maps. Our code is available at https://github.com/HiLab-git/CA-Net .
205 citations
TL;DR: This work makes extensive use of multiple attentions in a CNN architecture and proposes a comprehensive attention-based CNN (CA-Net) for more accurate and explainable medical image segmentation that is aware of the most important spatial positions, channels and scales at the same time.
Abstract: Accurate medical image segmentation is essential for diagnosis and treatment planning of diseases. Convolutional Neural Networks (CNNs) have achieved state-of-the-art performance for automatic medical image segmentation. However, they are still challenged by complicated conditions where the segmentation target has large variations of position, shape and scale, and existing CNNs have a poor explainability that limits their application to clinical decisions. In this work, we make extensive use of multiple attentions in a CNN architecture and propose a comprehensive attention-based CNN (CA-Net) for more accurate and explainable medical image segmentation that is aware of the most important spatial positions, channels and scales at the same time. In particular, we first propose a joint spatial attention module to make the network focus more on the foreground region. Then, a novel channel attention module is proposed to adaptively recalibrate channel-wise feature responses and highlight the most relevant feature channels. Also, we propose a scale attention module implicitly emphasizing the most salient feature maps among multiple scales so that the CNN is adaptive to the size of an object. Extensive experiments on skin lesion segmentation from ISIC 2018 and multi-class segmentation of fetal MRI found that our proposed CA-Net significantly improved the average segmentation Dice score from 87.77% to 92.08% for skin lesion, 84.79% to 87.08% for the placenta and 93.20% to 95.88% for the fetal brain respectively compared with U-Net. It reduced the model size to around 15 times smaller with close or even better accuracy compared with state-of-the-art DeepLabv3+. In addition, it has a much higher explainability than existing networks by visualizing the attention weight maps. Our code is available at this https URL
174 citations
Cites background from "Segmentation and classification in ..."
...and the placenta is important for fetal growth assessment and motion correction [41]....
[...]
TL;DR: Deep features are extracted from the inceptionv3 model, in which score vector is acquired from softmax and supplied to the quantum variational classifier (QVR) for discrimination between glioma, meningiomas, no tumor, and pituitary tumor to prove the proposed model's effectiveness.
Abstract: A brain tumor is an abnormal enlargement of cells if not properly diagnosed. Early detection of a brain tumor is critical for clinical practice and survival rates. Brain tumors arise in a variety of shapes, sizes, and features, with variable treatment options. Manual detection of tumors is difficult, time-consuming, and error-prone. Therefore, a significant requirement for computerized diagnostics systems for accurate brain tumor detection is present. In this research, deep features are extracted from the inceptionv3 model, in which score vector is acquired from softmax and supplied to the quantum variational classifier (QVR) for discrimination between glioma, meningioma, no tumor, and pituitary tumor. The classified tumor images have been passed to the proposed Seg-network where the actual infected region is segmented to analyze the tumor severity level. The outcomes of the reported research have been evaluated on three benchmark datasets such as Kaggle, 2020-BRATS, and local collected images. The model achieved greater than 90% detection scores to prove the proposed model's effectiveness.
22 citations
TL;DR: Wang et al. as discussed by the authors proposed a simple yet effective residual learning diagnosis system (RLDS) for diagnosing fetal CHD to improve diagnostic accuracy, which adopts convolutional neural networks to extract discriminative features of the fetal cardiac anatomical structures.
20 citations
TL;DR: The results suggest that the model used has a high potential to help cardiologists complete the initial screening for fetal congenital heart disease and a strong correlation between the predicted septal defects and ground truth as a mean average precision (mAP).
Abstract: Accurate screening for septal defects is important for supporting radiologists’ interpretative work. Some previous studies have proposed semantic segmentation and object detection approaches to carry out fetal heart detection; unfortunately, the models could not segment different objects of the same class. The semantic segmentation method segregates regions that only contain objects from the same class. In contrast, the fetal heart may contain multiple objects, such as the atria, ventricles, valves, and aorta. Besides, blurry boundaries (shadows) or a lack of consistency in the acquisition ultrasonography can cause wide variations. This study utilizes Mask-RCNN (MRCNN) to handle fetal ultrasonography images and employ it to detect and segment defects in heart walls with multiple objects. To our knowledge, this is the first study involving a medical application for septal defect detection using instance segmentation. The use of MRCNN architecture with ResNet50 as a backbone and a 0.0001 learning rate allows for two times faster training of the model on fetal heart images compared to other object detection methods, such as Faster-RCNN (FRCNN). We demonstrate a strong correlation between the predicted septal defects and ground truth as a mean average precision (mAP). As shown in the results, the proposed MRCNN model achieves good performance in multiclass detection of the heart chamber, with 97.59% for the right atrium, 99.67% for the left atrium, 86.17% for the left ventricle, 98.83% for the right ventricle, and 99.97% for the aorta. We also report competitive results for the defect detection of holes in the atria and ventricles via semantic and instance segmentation. The results show that the mAP for MRCNN is about 99.48% and 82% for FRCNN. We suggest that evaluation and prediction with our proposed model provide reliable detection of septal defects, including defects in the atria, ventricles, or both. These results suggest that the model used has a high potential to help cardiologists complete the initial screening for fetal congenital heart disease.
19 citations
Cites background or methods from "Segmentation and classification in ..."
...Unfortunately, such methods (with threshold-based techniques, for example) yield the best results when the regions of interest in an image exhibit a massive difference in strength from the background of the image, but this results in more similar images with problems, dramatically reducing the efficiency and decreasing the applicability of these methods [6], [27]....
[...]
...It can aid doctors in making more accurate treatment plans [27]....
[...]
...The segmentation process is the key to exploring fetal heart abnormalities, especially defect conditions [27]....
[...]
References
More filters
TL;DR: Preliminary findings suggest a reduced incidence of shunt-dependent hydrocephalus, as well as an improvement in hindbrain herniation, in patients treated with intrauterine MMC repair in the USA.
Abstract: Myelomeningocele is a common dysraphic defect leading to severe impairment throughout the patient's lifetime. Although surgical closure of this anomaly is usually performed in the early postnatal period, an estimated 330 cases of intrauterine repair have been performed in a few specialized centers worldwide. It was hoped prenatal intervention would improve the prognosis of affected patients, and preliminary findings suggest a reduced incidence of shunt-dependent hydrocephalus, as well as an improvement in hindbrain herniation. However, the expectations for improved neurological outcome have not been fulfilled and not all patients benefit from fetal surgery in the same way. Therefore, a multicenter randomized controlled trial was initiated in the USA to compare intrauterine with conventional postnatal care, in order to establish the procedure-related benefits and risks. The primary study endpoints include the need for shunt at 1 year of age, and fetal and infant mortality. No data from the trial will be published before the final analysis has been completed in 2008, and until then, the number of centers offering intrauterine MMC repair in the USA is limited to 3 in order to prevent the uncontrolled proliferation of new centers offering this procedure. In future, refined, risk-reduced surgical techniques and new treatment options for preterm labor and preterm rupture of the membranes are likely to reduce associated maternal and fetal risks and improve outcome, but further research will be needed.
85 citations
17 Oct 2016
TL;DR: A fully automatic segmentation framework of the placenta is proposed from structural T2-weighted scans of the whole uterus, as well as an extension in order to provide an intuitive pre-natal view into this vital organ.
Abstract: Recently, magnetic resonance imaging has revealed to be important for the evaluation of placenta’s health during pregnancy. Quantitative assessment of the placenta requires a segmentation, which proves to be challenging because of the high variability of its position, orientation, shape and appearance. Moreover, image acquisition is corrupted by motion artifacts from both fetal and maternal movements. In this paper we propose a fully automatic segmentation framework of the placenta from structural T2-weighted scans of the whole uterus, as well as an extension in order to provide an intuitive pre-natal view into this vital organ. We adopt a 3D multi-scale convolutional neural network to automatically identify placental candidate pixels. The resulting classification is subsequently refined by a 3D dense conditional random field, so that a high resolution placental volume can be reconstructed from multiple overlapping stacks of slices. Our segmentation framework has been tested on 66 subjects at gestational ages 20–38 weeks achieving a Dice score of \(71.95\pm 19.79\,\%\) for healthy fetuses with a fixed scan sequence and \(66.89\pm 15.35\,\%\) for a cohort mixed with cases of intrauterine fetal growth restriction using varying scan parameters.
82 citations
TL;DR: A new technique to fully automate the segmentation of an organ from 3D ultrasound (3D-US) volumes, using the placenta as the target organ, demonstrated good similarity to the ground-truth and almost identical clinical results for the prediction of SGA.
Abstract: Objectives: We present a new technique to fully automate the segmentation of an organ from 3D ultrasound (3D-US) volumes, using the placenta as the target organ. Image analysis tools to estimate organ volume do exist but are too time consuming and operator-dependant. Fully automating the segmentation process would potentially allow the use of placental volume to screen for increased risk of pregnancy complications. Methods: The placenta was segmented from 2393 first trimester 3D-US volumes using a semi-automated technique. This was quality controlled by three operators to produce the ‘ground-truth’ dataset. A fully convolutional neural network (OxNNet) was trained using this ‘ground-truth’ dataset to automatically segment the placenta. Findings: OxNNet delivered state of the art automatic segmentation (median Dice similarity coefficient of 0.84). The effect of training set size on the performance of OxNNet demonstrated the need for large datasets (n=1200, median DSC (inter-quartile range) 0.81 (0.15)). The clinical utility of placental volume was tested by looking at prediction of small-for-gestational-age (SGA) babies at term. The receiver-operating characteristics curves demonstrated almost identical results (OxNNet 0.65 (95% CI; 0.61-0.69) and ‘ground-truth’ 0.65 (95% CI; 0.61-0.69)). Conclusions: Our results demonstrated good similarity to the ‘ground-truth’ and almost identical clinical results for the prediction of SGA. Our open source software, OxNNet, and trained models are available on request.
77 citations
TL;DR: Reduced placental efficiency and altered placental function with AMA in women, with evidence of placental adaptations in normal pregnancies is identified, highlighting placental dysfunction as a potential mechanism for susceptibility to FGR and stillbirth with AMA.
Abstract: Pregnancies in women of advanced maternal age (AMA) are susceptible to fetal growth restriction (FGR) and stillbirth. We hypothesised that maternal ageing is associated with utero-placental dysfunction, predisposing to adverse fetal outcomes. Women of AMA (≥35 years) and young controls (20-30 years) with uncomplicated pregnancies were studied. Placentas from AMA women exhibited increased syncytial nuclear aggregates and decreased proliferation, and had increased amino acid transporter activity. Chorionic plate and myometrial artery relaxation was increased compared to controls. AMA was associated with lower maternal serum PAPP-A and sFlt and a higher PlGF:sFlt ratio. AMA mice (38-41 weeks) at E17.5 had fewer pups, more late fetal deaths, reduced fetal weight, increased placental weight and reduced fetal:placental weight ratio compared to 8-12 week controls. Maternofetal clearance of 14C-MeAIB and 3H-taurine was reduced and uterine arteries showed increased relaxation. These studies identify reduced placental efficiency and altered placental function with AMA in women, with evidence of placental adaptations in normal pregnancies. The AMA mouse model complements the human studies, demonstrating high rates of adverse fetal outcomes and commonalities in placental phenotype. These findings highlight placental dysfunction as a potential mechanism for susceptibility to FGR and stillbirth with AMA.
77 citations
TL;DR: Molecular and morphological changes in vivo and in vitro were prevented by a single dose of MitoQ bound to nanoparticles, which were shown to localise and prevent oxidative stress in the placenta but not in the fetus.
Abstract: Some neuropsychiatric disease, including schizophrenia, may originate during prenatal development, following periods of gestational hypoxia and placental oxidative stress. Here we investigated if gestational hypoxia promotes damaging secretions from the placenta that affect fetal development and whether a mitochondria-targeted antioxidant MitoQ might prevent this. Gestational hypoxia caused low birth-weight and changes in young adult offspring brain, mimicking those in human neuropsychiatric disease. Exposure of cultured neurons to fetal plasma or to secretions from the placenta or from model trophoblast barriers that had been exposed to altered oxygenation caused similar morphological changes. The secretions and plasma contained altered microRNAs whose targets were linked with changes in gene expression in the fetal brain and with human schizophrenia loci. Molecular and morphological changes in vivo and in vitro were prevented by a single dose of MitoQ bound to nanoparticles, which were shown to localise and prevent oxidative stress in the placenta but not in the fetus. We suggest the possibility of developing preventative treatments that target the placenta and not the fetus to reduce risk of psychiatric disease in later life.
75 citations