Automatic Segmentation Of the Placenta and its Peripheral Vasculature in Volumetric Ultrasound for TTTS Fetal Surgery
08 Apr 2019-pp 772-775
TL;DR: A fully-automated framework to achieve an accurate segmentation of the placenta and its peripheral vasculature in 3D US for twin-to-twin transfusion syndrome is proposed for the first time.
Abstract: Twin-to-twin transfusion syndrome is a serious condition that can affect pregnancies when identical twins share the placenta. In these cases, abnormal placental vessel connections (anastomoses) cause an uneven blood distribution between the babies. Ultrasound (US) enormously facilitates the assessment of these cases, but placenta segmentation is still a challenging task due to artifacts and high variability in its position, orientation, shape and appearance. We propose for the first time a fully-automated framework to achieve an accurate segmentation of the placenta and its peripheral vasculature in 3D US. A conditional Generative Adversarial Network is used to automatically identify the placenta. Afterwards, the entire vasculature is extracted using Modified Spatial Kernelized Fuzzy C-Means and Markov Random Fields. The method is tested on singleton and twin pregnancies from 15 to 38 gestational weeks, achieving a mean Dice coefficient of 0.75 ± 0.12 and 0.70 ± 0.14 for the placenta and its vessels.
Citations
More filters
TL;DR: A novel multi-task stacked generative adversarial framework is proposed to jointly learn synthetic fetal US generation, multi-class segmentation of the placenta, its inner acoustic shadows and peripheral vasculature, andplacenta shadowing removal and could be implemented in a TTTS fetal surgery planning software.
Abstract: Twin-to-twin transfusion syndrome (TTTS) is characterized by an unbalanced blood transfer through placental abnormal vascular connections. Prenatal ultrasound (US) is the imaging technique to monitor monochorionic pregnancies and diagnose TTTS. Fetoscopic laser photocoagulation is an elective treatment to coagulate placental communications between both twins. To locate the anomalous connections ahead of surgery, preoperative planning is crucial. In this context, we propose a novel multi-task stacked generative adversarial framework to jointly learn synthetic fetal US generation, multi-class segmentation of the placenta, its inner acoustic shadows and peripheral vasculature, and placenta shadowing removal. Specifically, the designed architecture is able to learn anatomical relationships and global US image characteristics. In addition, we also extract for the first time the umbilical cord insertion on the placenta surface from 3D HD-flow US images. The database consisted of 70 US volumes including singleton, mono- and dichorionic twins at 17-37 gestational weeks. Our experiments show that 71.8% of the synthesized US slices were categorized as realistic by clinicians, and that the multi-class segmentation achieved Dice scores of 0.82 ± 0.13, 0.71 ± 0.09, and 0.72 ± 0.09, for placenta, acoustic shadows, and vasculature, respectively. Moreover, fetal surgeons classified 70.2% of our completed placenta shadows as satisfactory texture reconstructions. The umbilical cord was successfully detected on 85.45% of the volumes. The framework developed could be implemented in a TTTS fetal surgery planning software to improve the intrauterine scene understanding and facilitate the location of the optimum fetoscope entry point.
4 citations
Cites methods from "Automatic Segmentation Of the Place..."
...In our previous work [25], we implemented a cGAN to identify the placenta followed by a modified spatial kernelized...
[...]
TL;DR: In this paper , a multi-task learning approach that combines the classification of placental location (e.g., anterior, posterior) and semantic placenta segmentation in a single convolutional neural network is proposed.
4 citations
TL;DR: A semiautomatic algorithm to detect the placenta, both umbilical cord insertions and the placental vasculature from Doppler ultrasound and provides a near real-time user experience and requires short training without compromising the segmentation accuracy.
Abstract: Twin-to-twin transfusion syndrome (TTTS) is a serious condition that occurs in about 10–15% of monochorionic twin pregnancies. In most instances, the blood flow is unevenly distributed throughout the placenta anastomoses leading to the death of both fetuses if no surgical procedure is performed. Fetoscopic laser coagulation is the optimal therapy to considerably improve co-twin prognosis by clogging the abnormal anastomoses. Notwithstanding progress in recent years, TTTS surgery is highly risky. Computer-assisted planning of the intervention can thus improve the outcome.
In this work, we implement a GPU-accelerated random walker (RW) algorithm to detect the placenta, both umbilical cord insertions and the placental vasculature from Doppler ultrasound (US). Placenta and background seeds are manually initialized in 10–20 slices (out of 245). Vessels are automatically initialized in the same slices by means of Otsu thresholding. The RW finds the boundaries of the placenta and reconstructs the vasculature. We evaluate our semiautomatic method in 5 monochorionic and 24 singleton pregnancies. Although satisfactory performance is achieved on placenta segmentation (Dice ≥ 84.0%), some vascular connections are still neglected due to the presence of US reverberation artifacts (Dice ≥ 56.9%). We also compared inter-user variability and obtained Dice coefficients of ≥ 76.8% and ≥ 97.42% for placenta and vasculature, respectively. After a 3-min manual initialization, our GPU approach speeds the computation 10.6 times compared to the CPU. Our semiautomatic method provides a near real-time user experience and requires short training without compromising the segmentation accuracy. A powerful approach is thus presented to rapidly plan the fetoscope insertion point ahead of TTTS surgery.
1 citations
References
More filters
21 Jul 2017
TL;DR: Conditional adversarial networks are investigated as a general-purpose solution to image-to-image translation problems and it is demonstrated that this approach is effective at synthesizing photos from label maps, reconstructing objects from edge maps, and colorizing images, among other tasks.
Abstract: We investigate conditional adversarial networks as a general-purpose solution to image-to-image translation problems. These networks not only learn the mapping from input image to output image, but also learn a loss function to train this mapping. This makes it possible to apply the same generic approach to problems that traditionally would require very different loss formulations. We demonstrate that this approach is effective at synthesizing photos from label maps, reconstructing objects from edge maps, and colorizing images, among other tasks. Moreover, since the release of the pix2pix software associated with this paper, hundreds of twitter users have posted their own artistic experiments using our system. As a community, we no longer hand-engineer our mapping functions, and this work suggests we can achieve reasonable results without handengineering our loss functions either.
11,958 citations
Additional excerpts
...Hence, we demonstrate the feasibility of applying the cGAN image-to-image translation approach [7] to better fit the placenta appearance (see Figure 2)....
[...]
Posted Content•
TL;DR: Conditional Adversarial Network (CA) as discussed by the authors is a general-purpose solution to image-to-image translation problems, which can be used to synthesize photos from label maps, reconstructing objects from edge maps, and colorizing images, among other tasks.
Abstract: We investigate conditional adversarial networks as a general-purpose solution to image-to-image translation problems. These networks not only learn the mapping from input image to output image, but also learn a loss function to train this mapping. This makes it possible to apply the same generic approach to problems that traditionally would require very different loss formulations. We demonstrate that this approach is effective at synthesizing photos from label maps, reconstructing objects from edge maps, and colorizing images, among other tasks. Indeed, since the release of the pix2pix software associated with this paper, a large number of internet users (many of them artists) have posted their own experiments with our system, further demonstrating its wide applicability and ease of adoption without the need for parameter tweaking. As a community, we no longer hand-engineer our mapping functions, and this work suggests we can achieve reasonable results without hand-engineering our loss functions either.
11,127 citations
TL;DR: Two simple models of random systems made up out of a finite collection of elements are considered: one generalizing the notion of “Gibbs ensemble” abstracted from statistical physics; the other, “Markov fields” derived from the idea of a Markov chain.
Abstract: This paper concerns random systems made up out of a finite collection of elements. We are interested in how a fixed structure of interactions reflects on the assignment of probabilities to overall states. In particular, we consider two simple models of random systems: one generalizing the notion of “Gibbs ensemble” abstracted from statistical physics; the other, “Markov fields” derived from the idea of a Markov chain. We give background for these two types, review proofs that they are in fact identical for systems with nonzero probabilities, and explore the new behavior that arises with constraints. Finally, we discuss unsolved problems and make suggestions for further work.
192 citations
"Automatic Segmentation Of the Place..." refers methods in this paper
...Our implementation utilizes the maximum a posteriori estimates for modeling the MRF....
[...]
...The initial MSKFCM mask is provided as MRF input to guarantee an accurate segmentation of the peripheral blood vessels....
[...]
...As for the placenta vasculature segmentation, our MSKFCM can efficiently deal with the black tubular vessels attached to the placenta surface thanks to the MRF refinement....
[...]
...In order to refine the resulting fuzzy segmentation, a Markov random field (MRF) [9] is employed....
[...]
TL;DR: A new technique to fully automate the segmentation of an organ from 3D ultrasound (3D-US) volumes, using the placenta as the target organ, demonstrated good similarity to the ground-truth and almost identical clinical results for the prediction of SGA.
Abstract: Objectives: We present a new technique to fully automate the segmentation of an organ from 3D ultrasound (3D-US) volumes, using the placenta as the target organ. Image analysis tools to estimate organ volume do exist but are too time consuming and operator-dependant. Fully automating the segmentation process would potentially allow the use of placental volume to screen for increased risk of pregnancy complications. Methods: The placenta was segmented from 2393 first trimester 3D-US volumes using a semi-automated technique. This was quality controlled by three operators to produce the ‘ground-truth’ dataset. A fully convolutional neural network (OxNNet) was trained using this ‘ground-truth’ dataset to automatically segment the placenta. Findings: OxNNet delivered state of the art automatic segmentation (median Dice similarity coefficient of 0.84). The effect of training set size on the performance of OxNNet demonstrated the need for large datasets (n=1200, median DSC (inter-quartile range) 0.81 (0.15)). The clinical utility of placental volume was tested by looking at prediction of small-for-gestational-age (SGA) babies at term. The receiver-operating characteristics curves demonstrated almost identical results (OxNNet 0.65 (95% CI; 0.61-0.69) and ‘ground-truth’ 0.65 (95% CI; 0.61-0.69)). Conclusions: Our results demonstrated good similarity to the ‘ground-truth’ and almost identical clinical results for the prediction of SGA. Our open source software, OxNNet, and trained models are available on request.
77 citations
"Automatic Segmentation Of the Place..." refers background or result in this paper
...Similar Dice coefficients compared to [4] and [5] are obtained by using less training data....
[...]
...In the same line, they also proposed the OxNNet [5] inspired by a U-Net / V-Net architecture....
[...]
10 Sep 2017
TL;DR: This work proposes the first and fully automatic framework for the simultaneous segmentation of multiple objects, including fetus, gestational sac and placenta, in ultrasound volumes, which remains as a rarely-studied but great challenge.
Abstract: 3D ultrasound is rapidly emerging as a viable imaging modality for routine prenatal examinations. However, lacking of efficient tools to decompose the volumetric data greatly limits its widespread. In this paper, we are looking at the problem of volumetric segmentation in ultrasound to promote the volume-based, precise maternal and fetal health monitoring. Our contribution is threefold. First, we propose the first and fully automatic framework for the simultaneous segmentation of multiple objects, including fetus, gestational sac and placenta, in ultrasound volumes, which remains as a rarely-studied but great challenge. Second, based on our customized 3D Fully Convolutional Network, we propose to inject a Recurrent Neural Network (RNN) to flexibly explore 3D semantic knowledge from a novel, sequential perspective, and therefore significantly refine the local segmentation result which is initially corrupted by the ubiquitous boundary uncertainty in ultrasound volumes. Third, considering sequence hierarchy, we introduce a hierarchical deep supervision mechanism to effectively boost the information flow within RNN and further improve the semantic segmentation results. Extensively validated on our in-house large datasets, our approach achieves superior performance and presents to be promising in boosting the interpretation of prenatal ultrasound volumes. Our framework is general and can be easily extended to other volumetric ultrasound segmentation tasks.
66 citations
"Automatic Segmentation Of the Place..." refers methods in this paper
...[6] implemented a 3D fully CNN with dense voxel-wise semantic labeling to simultaneously segment the fetus, the gestational sac and the placenta....
[...]