Biomedical photoacoustic tomography, which can provide high-resolution 3D soft tissue images based on optical absorption, has advanced to the stage at which translation from the laboratory to clinical settings is becoming possible. The need for rapid image formation and the practical restrictions on data acquisition that arise from the constraints of a clinical workflow are presenting new image reconstruction challenges. There are many classical approaches to image reconstruction, but ameliorating the effects of incomplete or imperfect data through the incorporation of accurate priors is challenging and leads to slow algorithms. Recently, the application of deep learning (DL), or deep neural networks, to this problem has received a great deal of attention. We review the literature on learned image reconstruction, summarizing the current trends and explain how these approaches fit within, and to some extent have arisen from, a framework that encompasses classical reconstruction methods. In particular, it shows how these techniques can be understood from a Bayesian perspective, providing useful insights. We also provide a concise tutorial demonstration of three prototypical approaches to learned image reconstruction. The code and data sets for these demonstrations are available to researchers. It is anticipated that it is in in vivo applications—where data may be sparse, fast imaging critical, and priors difficult to construct by hand—that DL will have the most impact. With this in mind, we conclude with some indications of possible future research directions.

https://www.spiedigitallibrary.org/journals/journal-of-biomedical-optics/volume-25/issue-11/112903/Deep-learning-in-photoacoustic-tomography--current-approaches-and-future/10.1117/1.JBO.25.11.112903.pdf

Deep learning in photoacoustic tomography: current approaches and future directions

In this work, we investigate the application of deep learning methods for computed tomography in the context of having a low-data regime. As motivation, we review some of the existing approaches and obtain quantitative results after training them with different amounts of data. We find that the learned primal-dual has an outstanding performance in terms of reconstruction quality and data efficiency. However, in general, end-to-end learned methods have two issues: a) lack of classical guarantees in inverse problems and b) lack of generalization when not trained with enough data. To overcome these issues, we bring in the deep image prior approach in combination with classical regularization. The proposed methods improve the state-of-the-art results in the low data-regime.

https://iopscience.iop.org/article/10.1088/1361-6420/aba415/pdf

Computed Tomography Reconstruction Using Deep Image Prior and Learned Reconstruction Methods

The reconstruction of computed tomography (CT) images is an active area of research. Following the rise of deep learning methods, many data-driven models have been proposed in recent years. In this work, we present the results of a data challenge that we organized, bringing together algorithm experts from different institutes to jointly work on quantitative evaluation of several data-driven methods on two large, public datasets during a ten day sprint. We focus on two applications of CT, namely, low-dose CT and sparse-angle CT. This enables us to fairly compare different methods using standardized settings. As a general result, we observe that the deep learning-based methods are able to improve the reconstruction quality metrics in both CT applications while the top performing methods show only minor differences in terms of peak signal-to-noise ratio (PSNR) and structural similarity (SSIM). We further discuss a number of other important criteria that should be taken into account when selecting a method, such as the availability of training data, the knowledge of the physical measurement model and the reconstruction speed.

Quantitative Comparison of Deep Learning-Based Image Reconstruction Methods for Low-Dose and Sparse-Angle CT Applications.

Over recent years, deep learning (DL) has established itself as a powerful tool across a broad spectrum of domains in imaging—e [...]

https://www.mdpi.com/2313-433X/7/4/74/pdf

Deep Learning in Medical Image Analysis.

Over recent years, deep learning methods have become an increasingly popular choice for solving tasks from the field of inverse problems. Many of these new data-driven methods have produced impressive results, although most only give point estimates for the reconstruction. However, especially in the analysis of ill-posed inverse problems, the study of uncertainties is essential. In our work, we apply generative flow-based models based on invertible neural networks to two challenging medical imaging tasks, i.e., low-dose computed tomography and accelerated medical resonance imaging. We test different architectures of invertible neural networks and provide extensive ablation studies. In most applications, a standard Gaussian is used as the base distribution for a flow-based model. Our results show that the choice of a radial distribution can improve the quality of reconstructions.

Conditional Invertible Neural Networks for Medical Imaging.

LoDoPaB-CT Challenge Reconstructions compared in "Quantitative comparison of deep learning-based image reconstruction methods for low-dose and sparse-angle CT applications"

Image reconstruction from computed tomography (CT) measurement is a challenging statistical inverse problem since a high-dimensional conditional distribution needs to be estimated. Based on training data obtained from high-quality reconstructions, we aim to learn a conditional density of images from noisy low-dose CT measurements. To tackle this problem, we propose a hybrid conditional normalizing flow, which integrates the physical model by using the filtered back-projection as conditioner. We evaluate our approach on a low-dose CT benchmark and demonstrate superior performance in terms of structural similarity of our flow-based method compared to other deep learning based approaches.

Conditional Normalizing Flows for Low-Dose Computed Tomography Image Reconstruction

Learning neural networks using only a small amount of data is an important on-going research topic with tremendous potential for applications. In this paper, we introduce a regularizer for the variational modeling of inverse problems in imaging based on normalizing ﬂows. Our regularizer, called patchNR, involves a normalizing ﬂow learned on patches of very few images. The subsequent reconstruction method is completely unsupervised and the same regularizer can be used for different forward operators acting on the same class of images. By investigating the distribution of patches versus those of the whole image class, we prove that our variational model is indeed a MAP approach. Our model can be generalized to conditional patchNRs, if additional supervised information is available. Numerical examples for low-dose CT, limited-angle CT and superresolution of material images demonstrate that our method provides high quality results among unsupervised methods, but requires only few data.

Alexander Denker

Papers

Quantitative Comparison of Deep Learning-Based Image Reconstruction Methods for Low-Dose and Sparse-Angle CT Applications.

LoDoPaB-CT Challenge Reconstructions compared in "Quantitative comparison of deep learning-based image reconstruction methods for low-dose and sparse-angle CT applications"

Conditional Normalizing Flows for Low-Dose Computed Tomography Image Reconstruction

Conditional Invertible Neural Networks for Medical Imaging.

PatchNR: Learning from Small Data by Patch Normalizing Flow Regularization