AI in health and medicine

This paper tries to give a gentle introduction to deep learning in medical image processing, proceeding from theoretical foundations to applications. We first discuss general reasons for the popularity of deep learning, including several major breakthroughs in computer science. Next, we start reviewing the fundamental basics of the perceptron and neural networks, along with some fundamental theory that is often omitted. Doing so allows us to understand the reasons for the rise of deep learning in many application domains. Obviously medical image processing is one of these areas which has been largely affected by this rapid progress, in particular in image detection and recognition, image segmentation, image registration, and computer-aided diagnosis. There are also recent trends in physical simulation, modeling, and reconstruction that have led to astonishing results. Yet, some of these approaches neglect prior knowledge and hence bear the risk of producing implausible results. These apparent weaknesses highlight current limitations of deep ()learning. However, we also briefly discuss promising approaches that might be able to resolve these problems in the future.

A gentle introduction to deep learning in medical image processing

Deep-learning-based tomographic imaging is an important application of artificial intelligence and a new frontier of machine learning. Deep learning has been widely used in computer vision and image analysis, which deal with existing images, improve these images, and produce features from them. Since 2016, deep learning techniques have been actively researched for tomographic imaging, especially in the context of biomedicine, with impressive results and great potential. Tomographic reconstruction produces images of multi-dimensional structures from externally measured ‘encoded’ data in the form of various tomographic transforms (integrals, harmonics, echoes and so on). In this Review, we provide a general background, highlight representative results with an emphasis on medical imaging, and discuss key issues that need to be addressed in this emerging field. In particular, tomographic imaging is an integral part of modern medicine, and will play a key role in personalized, preventive and precision medicine and make it intelligent, inexpensive and indiscriminate. The popularity of deep learning is leading to new areas in biomedical applications. Wang and colleagues summarize in this Review the recent development and future directions of deep neural networks for superior image quality in the tomographic imaging field.

Deep learning for tomographic image reconstruction

In this article, we consider deep learning strategies in ultrasound systems, from the front end to advanced applications. Our goal is to provide the reader with a broad understanding of the possible impact of deep learning methodologies on many aspects of ultrasound imaging. In particular, we discuss methods that lie at the interface of signal acquisition and machine learning, exploiting both data structure (e.g., sparsity in some domain) and data dimensionality (big data) already at the raw radio-frequency channel stage. As some examples, we outline efficient and effective deep learning solutions for adaptive beamforming and adaptive spectral Doppler through artificial agents, learn compressive encodings for the color Doppler, and provide a framework for structured signal recovery by learning fast approximations of iterative minimization problems, with applications to clutter suppression and super-resolution ultrasound. These emerging technologies may have a considerable impact on ultrasound imaging, showing promise across key components in the receive processing chain.

Deep Learning in Ultrasound Imaging

This paper tries to give a gentle introduction to deep learning in medical image processing, proceeding from theoretical foundations to applications. We first discuss general reasons for the popularity of deep learning, including several major breakthroughs in computer science. Next, we start reviewing the fundamental basics of the perceptron and neural networks, along with some fundamental theory that is often omitted. Doing so allows us to understand the reasons for the rise of deep learning in many application domains. Obviously medical image processing is one of these areas which has been largely affected by this rapid progress, in particular in image detection and recognition, image segmentation, image registration, and computer-aided diagnosis. There are also recent trends in physical simulation, modelling, and reconstruction that have led to astonishing results. Yet, some of these approaches neglect prior knowledge and hence bear the risk of producing implausible results. These apparent weaknesses highlight current limitations of deep learning. However, we also briefly discuss promising approaches that might be able to resolve these problems in the future.

A Gentle Introduction to Deep Learning in Medical Image Processing

Phase-contrast magnetic resonance imaging (MRI) provides time-resolved quantification of blood flow dynamics that can aid clinical diagnosis. Long in vivo scan times due to repeated three-dimensional (3D) volume sampling over cardiac phases and breathing cycles necessitate accelerated imaging techniques that leverage data correlations. Standard compressed sensing reconstruction methods require tuning of hyperparameters and are computationally expensive, which diminishes the potential reduction of examination times. We propose an efficient model-based deep neural reconstruction network and evaluate its performance on clinical aortic flow data. The network is shown to reconstruct undersampled 4D flow MRI data in under a minute on standard consumer hardware. Remarkably, the relatively low amounts of tunable parameters allowed the network to be trained on images from 11 reference scans while generalizing well to retrospective and prospective undersampled data for various acceleration factors and anatomies. 4D MRI scans can reconstruct cardiovascular flow, although they typically take many minutes, hindering real-time assessment. Vishnevskiy et al. develop a deep variational network to permit high-fidelity image reconstruction in a matter of seconds, allowing integration of 4D flow MRI into clinical workflows.

Deep variational network for rapid 4D flow MRI reconstruction

For beamforming ultrasound (US) signals, typically a spatially constant speed-of-sound (SoS) is assumed to calculate delays. As SoS in tissue may vary relatively largely, this approximation may cause wavefront aberrations, thus degrading effective imaging resolution. In the literature, corrections have been proposed based on unidirectional SoS estimation or computationally-expensive a posteriori phase rectification. In this paper we demonstrate a direct delay correction approach for US beamforming, by leveraging 2D spatial SoS distribution estimates from plane-wave imaging. We show both in simulations and with ex vivo measurements that resolutions close to the wavelength limit can be achieved using our proposed local SoS-adaptive beamforming, yielding a lateral resolution improvement of 22% to 29% on tissue samples with up to 3% SoS-contrast (45 m/s). We verify that our method accurately images absolute positions of tissue structures down to sub-pixel resolution of a tenth of a wavelength, whereas a global SoS assumption leads to artifactual localizations.

Ultrasound Aberration Correction based on Local Speed-of-Sound Map Estimation

Recent ultrasound imaging modalities based on ultrasound computed tomography indicate a huge potential to detect pathologies is tissue due to altered biomechanical properties. Especially the imaging of speed-of-sound (SoS) distribution in tissue has shown clinical promise and thus gained increasing attention in the field -- with several methods proposed based on transmission mode tomography. SoS imaging using conventional ultrasound (US) systems would be convenient and easy for clinical translation, but this requires using conventional US probes with single-sides tissue access and thus pulse-echo imaging sequences. Recent pulse-echo SoS imaging methods rely on plane wave (PW) insonifications, which is prone to strong aberration effects for non-homogeneous tissue composition. In this paper we propose to use diverging waves (DW) for SoS imaging and thus substantially improve the reconstruction of SoS distributions. We study this proposition by first plane wavefront aberrations compared to DW. We then present the sensitivity of both approaches to major parameterization choices on a set of simulated phantoms. Using the optimum parameter combination for each method for a given transducer model and imaging sequence, we analyze the SoS imaging performance comparatively between the two approaches. Results indicate that using DW instead of PW, the reconstruction accuracy improves substantially, by over 22% in reconstruction error (RMSE) and by 55% in contrast (CNR). We also demonstrate improvements in SoS reconstructions from an actual US acquisition of a breast phantom with tumor- and cyst-representative inclusions, with high and low SoS contrast, respectively.

/pdf/speed-of-sound-imaging-using-diverging-waves-36mibsjuua.pdf

Speed-of-Sound Imaging using Diverging Waves

Clinical presentation of left ventricular non-compaction cardiomyopathy (LVNC) can be heterogeneous from asymptomatic expression to congestive heart failure. Deformation indices assessed by cardiovascular magnetic resonance (CMR) can determine subclinical alterations of myocardial function and have been reported to be more sensitive to functional changes than ejection fraction. The objective of the present study was to investigate the determinants of myocardial deformation indices in patients with LVNC. Twenty patients with LVNC (44.7 ± 14.0 years) and twenty age- and gender-matched controls (49.1 ± 12.4 years) underwent functional CMR imaging using an ECG-triggered steady state-free-precession sequence (SSFP). Deformation indices derived with a feature tracking algorithm were calculated including end-systolic global longitudinal strain (GLS), circumferential strain (GCS), longitudinal and circumferential strain rate (SRll and SRcc). Twist and rotation were determined using an in-house developed post-processing pipeline. Global deformation indices (GLS, GCS, SRll and SRcc) were significantly lower in patients with LVNC compared to healthy controls (all, p < 0.01), especially for midventricular and apical regions. Apical rotation and twist were impaired for LVNC (p = 0.007 and p = 0.012), but basal rotation was preserved. Deformation indices of strain, strain rate and twist correlated well with parameters of the non-compacted myocardium, but not with the total myocardial mass or the thinning of the compacted myocardium, e.g. r = 0.595 between GLS and the non-compacted mass (p < 0.001). In conclusion, CMR deformation indices are reduced in patients with LVNC especially in affected midventricular and apical slices. The impairment of all strain and twist parameters correlates well with the extent of non-compacted myocardium.

/pdf/determinants-of-myocardial-function-characterized-by-cmr-5e7nhmer2z.pdf

Determinants of myocardial function characterized by CMR-derived strain parameters in left ventricular non-compaction cardiomyopathy

Tomographic image reconstruction is relevant for many medical imaging modalities including X-ray, ultrasound (US) computed tomography (CT) and photoacoustics, for which the access to full angular range tomographic projections might be not available in clinical practice due to physical or time constraints. Reconstruction from incomplete data in low signal-to-noise ratio regime is a challenging and ill-posed inverse problem that usually leads to unsatisfactory image quality. While informative image priors may be learned using generic deep neural network architectures, the artefacts caused by an ill-conditioned design matrix often have global spatial support and cannot be efficiently filtered out by means of convolutions. In this paper we propose to learn an inverse mapping in an end-to-end fashion via unrolling optimization iterations of a prototypical reconstruction algorithm. We herein introduce a network architecture that performs filtering jointly in both sinogram and spatial domains. To efficiently train such deep network we propose a novel regularization approach based on deep exponential weighting. Experiments on US and X-ray CT data show that our proposed method is qualitatively and quantitatively superior to conventional non-linear reconstruction methods as well as state-of-the-art deep networks for image reconstruction. Fast inference time of the proposed algorithm allows for sophisticated reconstructions in real-time critical settings, demonstrated with US SoS imaging of an ex vivo bovine phantom.

Valery Vishnevskiy

Papers

Deep variational network for rapid 4D flow MRI reconstruction

Ultrasound Aberration Correction based on Local Speed-of-Sound Map Estimation

Speed-of-Sound Imaging using Diverging Waves

Determinants of myocardial function characterized by CMR-derived strain parameters in left ventricular non-compaction cardiomyopathy

Deep Variational Networks with Exponential Weighting for Learning Computed Tomography