Deep Learning in Medical Ultrasound Analysis: A Review

In this review the application of deep learning for medical diagnosis is addressed. A thorough analysis of various scientific articles in the domain of deep neural networks application in the medical field has been conducted. More than 300 research articles were obtained, and after several selection steps, 46 articles were presented in more detail. The results indicate that convolutional neural networks (CNN) are the most widely represented when it comes to deep learning and medical image analysis. Furthermore, based on the findings of this article, it can be noted that the application of deep learning technology is widespread, but the majority of applications are focused on bioinformatics, medical diagnosis and other similar fields.

Deep Learning and Medical Diagnosis: A Review of Literature

Ultrasound (US) imaging is the most commonly performed cross-sectional diagnostic imaging modality in the practice of medicine. It is low-cost, non-ionizing, portable, and capable of real-time image acquisition and display. US is a rapidly evolving technology with significant challenges and opportunities. Challenges include high inter- and intra-operator variability and limited image quality control. Tremendous opportunities have arisen in the last decade as a result of exponential growth in available computational power coupled with progressive miniaturization of US devices. As US devices become smaller, enhanced computational capability can contribute significantly to decreasing variability through advanced image processing. In this paper, we review leading machine learning (ML) approaches and research directions in US, with an emphasis on recent ML advances. We also present our outlook on future opportunities for ML techniques to further improve clinical workflow and US-based disease diagnosis and characterization.

/pdf/machine-learning-for-medical-ultrasound-status-methods-and-2jqbj31nfj.pdf

Machine learning for medical ultrasound: status, methods, and future opportunities.

Purpose The required training sample size for a particular machine learning (ML) model applied to medical imaging data is often unknown. The purpose of this study was to provide a descriptive review of current sample-size determination methodologies in ML applied to medical imaging and to propose recommendations for future work in the field. Methods We conducted a systematic literature search of articles using Medline and Embase with keywords including “machine learning,” “image,” and “sample size.” The search included articles published between 1946 and 2018. Data regarding the ML task, sample size, and train-test pipeline were collected. Results A total of 167 articles were identified, of which 22 were included for qualitative analysis. There were only 4 studies that discussed sample-size determination methodologies, and 18 that tested the effect of sample size on model performance as part of an exploratory analysis. The observed methods could be categorized as pre hoc model-based approaches, which relied on features of the algorithm, or post hoc curve-fitting approaches requiring empirical testing to model and extrapolate algorithm performance as a function of sample size. Between studies, we observed great variability in performance testing procedures used for curve-fitting, model assessment methods, and reporting of confidence in sample sizes. Conclusions Our study highlights the scarcity of research in training set size determination methodologies applied to ML in medical imaging, emphasizes the need to standardize current reporting practices, and guides future work in development and streamlining of pre hoc and post hoc sample size approaches.

Sample-Size Determination Methodologies for Machine Learning in Medical Imaging Research: A Systematic Review

Fully-automated alignment of 3D fetal brain ultrasound to a canonical reference space using multi-task learning.

Objectives: We present a new technique to fully automate the segmentation of an organ from 3D ultrasound (3D-US) volumes, using the placenta as the target organ. Image analysis tools to estimate organ volume do exist but are too time consuming and operator-dependant. Fully automating the segmentation process would potentially allow the use of placental volume to screen for increased risk of pregnancy complications. Methods: The placenta was segmented from 2393 first trimester 3D-US volumes using a semi-automated technique. This was quality controlled by three operators to produce the ‘ground-truth’ dataset. A fully convolutional neural network (OxNNet) was trained using this ‘ground-truth’ dataset to automatically segment the placenta. Findings: OxNNet delivered state of the art automatic segmentation (median Dice similarity coefficient of 0.84). The effect of training set size on the performance of OxNNet demonstrated the need for large datasets (n=1200, median DSC (inter-quartile range) 0.81 (0.15)). The clinical utility of placental volume was tested by looking at prediction of small-for-gestational-age (SGA) babies at term. The receiver-operating characteristics curves demonstrated almost identical results (OxNNet 0.65 (95% CI; 0.61-0.69) and ‘ground-truth’ 0.65 (95% CI; 0.61-0.69)). Conclusions: Our results demonstrated good similarity to the ‘ground-truth’ and almost identical clinical results for the prediction of SGA. Our open source software, OxNNet, and trained models are available on request.

/pdf/fully-automated-real-time-3d-ultrasound-segmentation-to-1nl8s7aoov.pdf

Fully automated, real-time 3D ultrasound segmentation to estimate first trimester placental volume using deep learning

Placental volume measured with 3D ultrasound in the first trimester has been shown to be correlated to adverse pregnancy outcomes. This could potentially be used as a screening test to predict the “at risk” pregnancy. However, manual segmentation whilst previously shown to be accurate and repeatable is very time consuming and semi-automated methods still require operator input. To generate a screening tool, fully automated placental segmentation is required. In this work, a deep convolutional neural network (cNN), DeepMedic, was trained using the output of the semi-automated Random Walker method as ground truth. 300 3D ultrasound scans of first trimester placentas were used to train, validate and test the cNN. Compared against the semi-automated segmentation, resultant median (1st Quartile, 3rd Quartile) Dice Similarity Coefficient was 0.73 (0.66, 0.76). The median (1st Quartile, 3rd Quartile) Hausdorff distance was 27 mm (18 mm, 36 mm). We present the first attempt at using a deep cNN for segmentation of 3D ultrasound of the placenta. This work shows that feasible results compared to ground truth were obtained that could form the basis of a fully automatic segmentation method.

/pdf/automatic-3d-ultrasound-segmentation-of-the-first-trimester-12221qq1mj.pdf

Automatic 3D ultrasound segmentation of the first trimester placenta using deep learning

Volumetric placental measurement using 3-D ultrasound has proven clinical utility in predicting adverse pregnancy outcomes. However, this metric cannot currently be employed as part of a screening test due to a lack of robust and real-time segmentation tools. We present a multiclass (MC) convolutional neural network (CNN) developed to segment the placenta, amniotic fluid, and fetus. The ground-truth data set consisted of 2093 labeled placental volumes augmented by 300 volumes with placenta, amniotic fluid, and fetus annotated. A two-pathway, hybrid (HB) model using transfer learning, a modified loss function, and exponential average weighting was developed and demonstrated the best performance for placental segmentation (PS), achieving a Dice similarity coefficient (DSC) of 0.84- and 0.38-mm average Hausdorff distances (HDAV). The use of a dual-pathway architecture improved the PS by 0.03 DSC and reduced HDAV by 0.27 mm compared with a naive MC model. The incorporation of exponential weighting produced a further small improvement in DSC by 0.01 and a reduction of HDAV by 0.44 mm. Per volume inference using the FCNN took 7–8 s. This method should enable clinically relevant morphometric measurements (such as volume and total surface area) to be automatically generated for the placenta, amniotic fluid, and fetus. The ready availability of such metrics makes a population-based screening test for adverse pregnancy outcomes possible.

/pdf/fully-automated-3-d-ultrasound-segmentation-of-the-placenta-2p5vm4t2ia.pdf

Fully Automated 3-D Ultrasound Segmentation of the Placenta, Amniotic Fluid, and Fetus for Early Pregnancy Assessment

Prediction of term small for gestational age babies using first trimester placental volume; a comparison of a novel, fully automated technique, oxnnet with a commercially available, semi-automatic tool, vocal (tm)

Image analysis tools to estimate organ volume do exist but are too time consuming and operator-dependant. Fully automating the segmentation process would potentially allow the use of placental volume to screen for increased risk of pregnancy complications. Previously Looney et al. [2017] trained Deepmedic [Kamnitsas et al., 2017] on 300 ultrasound volumes and obtained a Dice similarity coefficient (DSC) of 0.73. Yang et al. [2017] used 104 cases and obtained a DSC of 0.64 for the placenta. In this work a new technique [Looney et al., 2018] to fully automate the segmentation of an organ from 3D ultrasound (3D-US) volumes is presented, using the placenta as the target organ. The architecture of the CNN is a modified version of U-Net for 3D [Ronneberger et al., 2015]. The placenta was segmented from 2393 first trimester 3D-US volumes using a semiautomated technique. This was quality controlled by three operators to produce the ‘ground-truth’ dataset. A fully convolutional neural network (OxNNet) was trained using this ‘ground-truth’ dataset to automatically segment the placenta. OxNNet delivered state of the art automatic segmentation (median Dice similarity coefficient of 0.84). The effect of training set size on the performance of OxNNet demonstrated the need for large datasets (n=1200, median DSC (inter-quartile range) 0.81 (0.15)). By looking at prediction of small-for-gestational-age (SGA) babies at term the clinical utility of placental volume was tested. The receiveroperating characteristics curves demonstrated almost identical results (OxNNet 0.65 (95% CI; 0.61-0.69) and ‘ground-truth’ 0.65 (95% CI; 0.61-0.69)). Our results demonstrated good similarity to the ‘ground-truth’ and almost identical clinical results for the prediction of SGA. The software used to obtain these results is available online [Looney, 2018]. 1st Conference on Medical Imaging with Deep Learning (MIDL 2018), Amsterdam, The Netherlands.

Stavros Natsis

Papers

Fully automated, real-time 3D ultrasound segmentation to estimate first trimester placental volume using deep learning

Automatic 3D ultrasound segmentation of the first trimester placenta using deep learning

Fully Automated 3-D Ultrasound Segmentation of the Placenta, Amniotic Fluid, and Fetus for Early Pregnancy Assessment

Prediction of term small for gestational age babies using first trimester placental volume; a comparison of a novel, fully automated technique, oxnnet with a commercially available, semi-automatic tool, vocal (tm)

Using deep learning to develop a fully automated, real-time 3D-ultrasound segmentation tool to estimate placental volume in the first trimester.