Segmentation and classification in MRI and US fetal imaging: Recent trends and future prospects.

Abstract: Accurate medical image segmentation is essential for diagnosis and treatment planning of diseases. Convolutional Neural Networks (CNNs) have achieved state-of-the-art performance for automatic medical image segmentation. However, they are still challenged by complicated conditions where the segmentation target has large variations of position, shape and scale, and existing CNNs have a poor explainability that limits their application to clinical decisions. In this work, we make extensive use of multiple attentions in a CNN architecture and propose a comprehensive attention-based CNN (CA-Net) for more accurate and explainable medical image segmentation that is aware of the most important spatial positions, channels and scales at the same time. In particular, we first propose a joint spatial attention module to make the network focus more on the foreground region. Then, a novel channel attention module is proposed to adaptively recalibrate channel-wise feature responses and highlight the most relevant feature channels. Also, we propose a scale attention module implicitly emphasizing the most salient feature maps among multiple scales so that the CNN is adaptive to the size of an object. Extensive experiments on skin lesion segmentation from ISIC 2018 and multi-class segmentation of fetal MRI found that our proposed CA-Net significantly improved the average segmentation Dice score from 87.77% to 92.08% for skin lesion, 84.79% to 87.08% for the placenta and 93.20% to 95.88% for the fetal brain respectively compared with U-Net. It reduced the model size to around 15 times smaller with close or even better accuracy compared with state-of-the-art DeepLabv3+. In addition, it has a much higher explainability than existing networks by visualizing the attention weight maps. Our code is available at https://github.com/HiLab-git/CA-Net .

Abstract: We propose a method to extract the human placenta at late gestation using multi-view 3D US images. This is the first step towards automatic quantification of placental volume and morphology from US images along the whole pregnancy beyond early stages (where the entire placenta can be captured with a single 3D US image). Our method uses 3D US images from different views acquired with a multi-probe system. A whole placenta segmentation is obtained from these images by using a novel technique based on 3D convolutional neural networks. We demonstrate the performance of our method on 3D US images of the placenta in the last trimester. We achieve a high Dice overlap of up to 0.8 with respect to manual annotations, and the derived placental volumes are comparable to corresponding volumes extracted from MR.

Abstract: Twin-to-twin transfusion syndrome (TTTS) is a serious condition that may occur in pregnancies when two or more fetuses share the same placenta. It is characterized by abnormal vascular connections in the placenta that cause blood to flow unevenly between the babies. If left untreated, perinatal mortality occurs in 90% of cases, whilst neurological injuries are still present in TTTS survivors. Minimally invasive fetoscopic laser surgery is the standard and optimal treatment for this condition, but is technically challenging and can lead to complications. Acquiring and maintaining the required surgical skills need consistent practice, and a steep learning curve. An accurate preoperative planning is thus vital for complex TTTS cases. To this end, we propose the first TTTS fetal surgery planning and simulation platform. The soft tissue of the mother, the uterus, the umbilical cords, the placenta and its vascular tree are segmented and registered automatically from magnetic resonance imaging and 3D ultrasound using computer vision and deep learning techniques. The proposed state-of-the-art technology is integrated into a flexible C++ and MITK-based application to provide a full exploration of the intrauterine environment by simulating the fetoscope camera as well as the laser ablation, determining the correct entry point, training doctors’ movements and trajectory ahead of operation, which allows improving upon current practice. A comprehensive usability study is reported. Experienced surgeons rated highly our TTTS planner and simulator, thus being a potential tool to be implemented in real and complex TTTS surgeries.

Abstract: Recent advances in fetal imaging open the door to enhanced detection of fetal disorders and computer-assisted surgical planning. However, precise segmentation of womb's tissues is challenging due to motion artifacts caused by fetal movements and maternal respiration during acquisition. This work aims to efficiently segment different intrauterine tissues in fetal magnetic resonance imaging (MRI) and 3D ultrasound (US). First, a large set of ninety-four radiomic features are extracted to characterize the mother uterus, placenta, umbilical cord, fetal lungs, and brain. The optimal features for each anatomy are identified using both K-best and Sequential Forward Feature Selection techniques. These features are then fed to a Support Vector Machine with instance balancing to accurately segment the intrauterine anatomies. To the best of our knowledge, this is the first time that "Radiomics" is expanded from classification tasks to segmentation purposes to deal with challenging fetal images. In addition, we evaluate several state-of-the-art deep learning-based segmentation approaches. Validation is extensively performed on a set of 60 axial MRI and 3D US images from pathological and clinical cases. Our results suggest that combining the selected 10 radiomic features per anatomy along with DeepLabV3+ or BiSeNet architectures for MRI, and PSPNet or Tiramisu for 3D US, can lead to the highest fetal / maternal tissue segmentation performance, robustness, informativeness, and heterogeneity. Therefore, this work opens new avenues for advancement of segmentation techniques and, in particular, for improved fetal surgical planning.

Abstract: Purpose of review To review the advance of maternal--fetal surgery, the research of stem cell transplantation and tissue engineering in prenatal management of fetal meningomyelocele (fMMC). Recent findings Advance in the imaging study provides more accurate assessment of fMMC in utero. Prenatal maternal--fetal surgery in fMMC demonstrates favourable postnatal outcome. Minimally invasive fetal surgery minimizes uterine wall disruption. Endoscopic fetal surgery is performed via laparotomy-assisted or entirely percutaneous approach. The postnatal outcome for open and endoscopic fetal surgery shares no difference. Single layer closure during repair of fMMC is preferred to reduce postnatal surgical intervention. All maternal--fetal surgeries impose anesthetic and obstetric risk to pregnant woman. Ruptured of membrane and preterm delivery are common complications. Trans-amniotic stem cell therapy (TRASCET) showed potential tissue regeneration in animal models. Fetal tissue engineering with growth factors and dura substitutes with biosynthetic materials promote spinal cord regeneration. This will overcome the challenge of closure in large fMMC. Planning of the maternal--fetal surgery should adhere to ethical framework to minimize morbidity to both fetus and mother. Summary Combination of endoscopic fetal surgery with TRASCET or tissue engineering will be a new vision to achieve to improve the outcome of prenatal intervention in fMMC.

Abstract: This work is supported by the EPSRC First Grant scheme (grant ref no. EP/N023668/1) and partially funded under the 7th Framework Programme by the European Commission (TBIcare: http: //www.tbicare.eu/ ; CENTER-TBI: https://www.center-tbi.eu/). This work was further supported by a Medical Research Council (UK) Program Grant (Acute brain injury: heterogeneity of mechanisms, therapeutic targets and outcome effects [G9439390 ID 65883]), the UK National Institute of Health Research Biomedical Research Centre at Cambridge and Technology Platform funding provided by the UK Department of Health. KK is supported by the Imperial College London PhD Scholarship Programme. VFJN is supported by a Health Foundation/Academy of Medical Sciences Clinician Scientist Fellowship. DKM is supported by an NIHR Senior Investigator Award. We gratefully acknowledge the support of NVIDIA Corporation with the donation of two Titan X GPUs for our research.

Abstract: The trial was stopped for efficacy of prenatal surgery after the recruitment of 183 of a planned 200 patients. This report is based on results in 158 patients whose children were evaluated at 12 months. The first primary outcome occurred in 68% of the infants in the prenatal-surgery group and in 98% of those in the postnatalsurgery group (relative risk, 0.70; 97.7% confidence interval [CI], 0.58 to 0.84; P<0.001). Actual rates of shunt placement were 40% in the prenatal-surgery group and 82% in the postnatal-surgery group (relative risk, 0.48; 97.7% CI, 0.36 to 0.64; P<0.001). Prenatal surgery also resulted in improvement in the composite score for mental development and motor function at 30 months (P = 0.007) and in improvement in several secondary outcomes, including hindbrain herniation by 12 months and ambulation by 30 months. However, prenatal surgery was associated with an increased risk of preterm delivery and uterine dehiscence at delivery. Conclusions Prenatal surgery for myelomeningocele reduced the need for shunting and improved motor outcomes at 30 months but was associated with maternal and fetal risks. (Funded by the National Institutes of Health; ClinicalTrials.gov number, NCT00060606.)

Abstract: Convolutional neural networks (CNNs) have achieved state-of-the-art performance for automatic medical image segmentation. However, they have not demonstrated sufficiently accurate and robust results for clinical use. In addition, they are limited by the lack of image-specific adaptation and the lack of generalizability to previously unseen object classes (a.k.a. zero-shot learning). To address these problems, we propose a novel deep learning-based interactive segmentation framework by incorporating CNNs into a bounding box and scribble-based segmentation pipeline. We propose image-specific fine tuning to make a CNN model adaptive to a specific test image, which can be either unsupervised (without additional user interactions) or supervised (with additional scribbles). We also propose a weighted loss function considering network and interaction-based uncertainty for the fine tuning. We applied this framework to two applications: 2-D segmentation of multiple organs from fetal magnetic resonance (MR) slices, where only two types of these organs were annotated for training and 3-D segmentation of brain tumor core (excluding edema) and whole brain tumor (including edema) from different MR sequences, where only the tumor core in one MR sequence was annotated for training. Experimental results show that: 1) our model is more robust to segment previously unseen objects than state-of-the-art CNNs; 2) image-specific fine tuning with the proposed weighted loss function significantly improves segmentation accuracy; and 3) our method leads to accurate results with fewer user interactions and less user time than traditional interactive segmentation methods.

Abstract: A practical classification of placental maturity changes has been developed based on a review of multiple ultrasound evaluations of placental texture over a 4 year period. This classification grades placentas from 0 to 3 according to specific ultrasonic findings at the basal and chorionic plates as well as within the substance of the organ itself. The placentas of 129 patients were graded according to this system at the time of ultrasound evaluation. Eighty-six patients had placentas classified as Grade 1 or greater and all of these had lecithin-sphingomyelin (L/S) ratio determinations performed. Mature L/S ratios (2.0) were found in 68% of Grade I (21/31), 88% of Grade II (28/32), and 100% of Grade III placentas (23/23). These results suggest a correlation between maturational changes of the placenta as seen by ultrasound and fetal pulmonic maturity as indicated by L/S ratio.