A shape-constraint adversarial framework with instance-normalized spatio-temporal features for inter-fetal membrane segmentation

TLDR: In this paper, a new deep learning framework for inter-fetal membrane segmentation on in-vivo fetoscopic videos is presented, which enhances existing architectures by encoding a novel (instance-normalized) dense block, invariant to illumination changes, that extracts spatio-temporal features to enforce pixel connectivity in time, and relying on an adversarial training, which constrains macro appearance.

About: This article is published in Medical Image Analysis.The article was published on 2021-02-19 and is currently open access. It has received 9 citations till now.

Figures

Figure 10: Sample frames in which the membrane is not present.Each frame was extracted from a video not used to train the network, as described in Sec. 3.4. The predicted segmentation is highlighted in yellow.

Figure 1: Sample frames from our dataset. The frames are extracted from intra-operative videos acquired in the actual surgical practice for Twin-to-Twin Transfusion Syndrome (TTTS). Each frame refers to a different video. Although video acquisition was performed with the same equipment, the frames present high variability, in terms of: (i) different membrane position, shape, tissue area in the field of view, contrast and texture, (ii) noise and blur, (iii) presence of amniotic fluid particles, (iv) vessels along the membrane equator, (v) different levels of illumination, (vi) presence of laser-guide light.

Table 1: Architecture details for the (top) segmentor and (bottom) critic. The IN Conv3D and BN Conv3D refer to Instance Normalization - leaky ReLu - 3D Convolution and Batch Normalization - leaky ReLu - 3D Convolution, respectively.

Figure 8: Sample results of inter-fetal membrane segmentation for three consecutive frames in a clip. Results are shown for the (second column) 2D adversarial framework and (third column) the proposed framework. The gold standard and segmentation prediction are highlighted in white and yellow, respectively.

Table 4: Results of the sliding window configuration tested in E5, E6 in the ablation study. Segmentation Accuracy (Acc), Dice Similarity Coefficient (DSC ) and Sensitivity (Sens) on the test set are reported in terms of mean ± standard deviation. The best results are highlighted in bold.

Figure 5: Boxplot of performance comparison between E1, E2, E3, E4 in the ablation study and Casella et al. (2020). The comparison is shown in terms of Dice similarity coefficient (DSC ) for each fold. Black asterisks highlight significant differences between the different architectures (Mann–Whitney–Wilcoxon) (∗p < 0.05, ∗ ∗ p < 0.01, ∗ ∗ ∗p < 0.001).