Research in artificial intelligence for radiology and radiotherapy has recently become increasingly reliant on the use of deep learning-based algorithms. While the performance of the models which these algorithms produce can significantly outperform more traditional machine learning methods, they do rely on larger datasets being available for training. To address this issue, data augmentation has become a popular method for increasing the size of a training dataset, particularly in fields where large datasets aren't typically available, which is often the case when working with medical images. Data augmentation aims to generate additional data which is used to train the model and has been shown to improve performance when validated on a separate unseen dataset. This approach has become commonplace so to help understand the types of data augmentation techniques used in state-of-the-art deep learning models, we conducted a systematic review of the literature where data augmentation was utilised on medical images (limited to CT and MRI) to train a deep learning model. Articles were categorised into basic, deformable, deep learning or other data augmentation techniques. As artificial intelligence models trained using augmented data make their way into the clinic, this review aims to give an insight to these techniques and confidence in the validity of the models produced.

A review of medical image data augmentation techniques for deep learning applications.

Machine learning in medical applications: A review of state-of-the-art methods

Artificial intelligence and machine learning for medical imaging: A technology review.

Siam-U-Net: encoder-decoder siamese network for knee cartilage tracking in ultrasound images

A review of deep learning based methods for medical image multi-organ segmentation.

Automatic delineation of organs at risk (OAR) in computed tomography (CT) images is a crucial step for treatment planning in radiation oncology. However, manual delineation of organs is a challenging and time-consuming task subject to inter-observer variabilities. Automatic organ delineation has been relying on non-rigid registrations and atlases. However, lately deep learning appears as a strong competitor with specific architectures dedicated to image segmentation like UNet. In this paper, we first assessed the standard UNet to delineate multiple organs in CT images. Second, we observed the effect of dilated convolutional layers in UNet to better capture the global context from the CT images and effectively learn the anatomy, which results in increased localization of organ delineation. We evaluated the performance of a standard UNet and a dilated UNet (with dilated convolutional layers) on four chest organs (esophagus, left lung, right lung, and spinal cord) from 29 lung image acquisitions and observed that dilated UNet delineates the soft tissues notably esophagus and spinal cord with higher accuracy than the standard UNet. We quantified the segmentation accuracy of both models by computing spatial overlap measures like Dice similarity coefficient, recall & precision, and Hausdorff distance. Compared to the standard UNet, dilated UNet yields the best Dice scores for soft organs whereas for lungs, no significant difference in the Dice score was observed: \(0.84\pm 0.07\) vs \(0.71\pm 0.10\) for esophagus, \(0.99\pm {0.01}\) vs \(0.99\pm {0.01}\) for left lung, \(0.99\pm {0.01}\) vs \(0.99\pm {0.01}\) for right lung and \(0.91\pm {0.05}\) vs \(0.88\pm {0.04}\) for spinal cord.

Multi-organ Segmentation of Chest CT Images in Radiation Oncology: Comparison of Standard and Dilated UNet

Purpose Monte Carlo (MC) algorithms offer accurate modeling of dose calculation by simulating the transport and interactions of many particles through the patient geometry. However, given their random nature, the resulting dose distributions have statistical uncertainty (noise), which prevents making reliable clinical decisions. This issue is partly addressable using a huge number of simulated particles but is computationally expensive as it results in significantly greater computation times. Therefore, there is a trade-off between the computation time and the noise level in MC dose maps. In this work, we address the mitigation of noise inherent to MC dose distributions using dilated U-Net - an encoder-decoder-styled fully convolutional neural network, which allows fast and fully automated denoising of whole-volume dose maps. Methods We use mean squared error (MSE) as loss function to train the model, where training is done in 2D and 2.5D settings by considering a number of adjacent slices. Our model is trained on proton therapy MC dose distributions of different tumor sites (brain, head and neck, liver, lungs, and prostate) acquired from 35 patients. We provide the network with input MC dose distributions simulated using 1 × 10 6 particles while keeping 1 × 10 9 particles as reference. Results After training, our model successfully denoises new MC dose maps. On average (averaged over five patients with different tumor sites), our model recovers D 95 of 55.99 Gy from the noisy MC input of 49.51 Gy, whereas the low noise MC (reference) offers 56.03 Gy. We observed a significant reduction in average RMSE (thresholded >10% max ref) for reference vs denoised (1.25 Gy) than reference vs input (16.96 Gy) leading to an improvement in signal-to-noise ratio (ISNR) by 18.06 dB. Moreover, the inference time of our model for a dose distribution is less than 10 s vs 100 min (MC simulation using 1 × 10 9 particles). Conclusions We propose an end-to-end fully convolutional network that can denoise Monte Carlo dose distributions. The networks provide comparable qualitative and quantitative results as the MC dose distribution simulated with 1 × 10 9 particles, offering a significant reduction in computation time.

Mitigating inherent noise in Monte Carlo dose distributions using dilated U-Net.

For prostate cancer patients, large organ deformations occurring between the sessions of a fractionated radiotherapy treatment lead to uncertainties in the doses delivered to the tumour and the surrounding organs at risk. The segmentation of those structures in cone beam CT (CBCT) volumes acquired before every treatment session is desired to reduce those uncertainties. In this work, we perform a fully automatic bladder segmentation of CBCT volumes with u-net, a 3D fully convolutional neural network (FCN). Since annotations are hard to collect for CBCT volumes, we consider augmenting the training dataset with annotated CT volumes and show that it improves the segmentation performance. Our network is trained and tested on 48 annotated CBCT volumes using a 6-fold cross-validation scheme. The network reaches a mean Dice similarity coefficient (DSC) of 0:801 ± 0:137 with 32 training CBCT volumes. This result improves to 0:848 ± 0:085 when the training set is augmented with 64 CT volumes. The segmentation accuracy increases both with the number of CBCT and CT volumes in the training set. As a comparison, the state-of-the-art deformable image registration (DIR) contour propagation between planning CT and daily CBCT available in RayStation reaches a DSC of 0:744 ± 0:144 on the same dataset, which is below our FCN result.

Using planning CTs to enhance CNN-based bladder segmentation on cone beam CT

Although deep convolutional neural networks (CNNs) have outperformed state-of-the-art in many medical image segmentation tasks, deep network architectures generally fail in exploiting common sense prior to drive the segmentation. In particular, the availability of a segmented (source) image observed in a CT slice that is adjacent to the slice to be segmented (or target image) has not been considered to improve the deep models segmentation accuracy. In this paper, we investigate a CNN architecture that maps a joint input, composed of the target image and the source segmentation, to a target segmentation. We observe that our solution succeeds in taking advantage of the source segmentation when it is sufficiently close to the target segmentation, without being penalized when the source is far from the target.

Umair Javaid

Papers

Artificial intelligence and machine learning for medical imaging: A technology review.

Multi-organ Segmentation of Chest CT Images in Radiation Oncology: Comparison of Standard and Dilated UNet

Mitigating inherent noise in Monte Carlo dose distributions using dilated U-Net.

Using planning CTs to enhance CNN-based bladder segmentation on cone beam CT

Contour Propagation in CT Scans with Convolutional Neural Networks