A survey on deep learning in medical image analysis

When building a unified vision system or gradually adding new capabilities to a system, the usual assumption is that training data for all tasks is always available. However, as the number of tasks grows, storing and retraining on such data becomes infeasible. A new problem arises where we add new capabilities to a Convolutional Neural Network (CNN), but the training data for its existing capabilities are unavailable. We propose our Learning without Forgetting method, which uses only new task data to train the network while preserving the original capabilities. Our method performs favorably compared to commonly used feature extraction and fine-tuning adaption techniques and performs similarly to multitask learning that uses original task data we assume unavailable. A more surprising observation is that Learning without Forgetting may be able to replace fine-tuning with similar old and new task datasets for improved new task performance.

Learning without Forgetting

What has happened in machine learning lately, and what does it mean for the future of medical image analysis? Machine learning has witnessed a tremendous amount of attention over the last few years. The current boom started around 2009 when so-called deep artificial neural networks began outperforming other established models on a number of important benchmarks. Deep neural networks are now the state-of-the-art machine learning models across a variety of areas, from image analysis to natural language processing, and widely deployed in academia and industry. These developments have a huge potential for medical imaging technology, medical data analysis, medical diagnostics and healthcare in general, slowly being realized. We provide a short overview of recent advances and some associated challenges in machine learning applied to medical image processing and image analysis. As this has become a very broad and fast expanding field we will not survey the entire landscape of applications, but put particular focus on deep learning in MRI. 
Our aim is threefold: (i) give a brief introduction to deep learning with pointers to core references; (ii) indicate how deep learning has been applied to the entire MRI processing chain, from acquisition to image retrieval, from segmentation to disease prediction; (iii) provide a starting point for people interested in experimenting and perhaps contributing to the field of machine learning for medical imaging by pointing out good educational resources, state-of-the-art open-source code, and interesting sources of data and problems related medical imaging.

An overview of deep learning in medical imaging focusing on MRI

Rapid advances in hardware-based technologies during the past decades have opened up new possibilities for life scientists to gather multimodal data in various application domains, such as omics , bioimaging , medical imaging , and (brain/body)–machine interfaces . These have generated novel opportunities for development of dedicated data-intensive machine learning techniques. In particular, recent research in deep learning (DL), reinforcement learning (RL), and their combination (deep RL) promise to revolutionize the future of artificial intelligence. The growth in computational power accompanied by faster and increased data storage, and declining computing costs have already allowed scientists in various fields to apply these techniques on data sets that were previously intractable owing to their size and complexity. This paper provides a comprehensive survey on the application of DL, RL, and deep RL techniques in mining biological data. In addition, we compare the performances of DL techniques when applied to different data sets across various application domains. Finally, we outline open issues in this challenging research area and discuss future development perspectives.

/pdf/applications-of-deep-learning-and-reinforcement-learning-to-3l72kwyi7n.pdf

Applications of Deep Learning and Reinforcement Learning to Biological Data

What has happened in machine learning lately, and what does it mean for the future of medical image analysis? Machine learning has witnessed a tremendous amount of attention over the last few years. The current boom started around 2009 when so-called deep artificial neural networks began outperforming other established models on a number of important benchmarks. Deep neural networks are now the state-of-the-art machine learning models across a variety of areas, from image analysis to natural language processing, and widely deployed in academia and industry. These developments have a huge potential for medical imaging technology, medical data analysis, medical diagnostics and healthcare in general, slowly being realized. We provide a short overview of recent advances and some associated challenges in machine learning applied to medical image processing and image analysis. As this has become a very broad and fast expanding field we will not survey the entire landscape of applications, but put particular focus on deep learning in MRI. Our aim is threefold: (i) give a brief introduction to deep learning with pointers to core references; (ii) indicate how deep learning has been applied to the entire MRI processing chain, from acquisition to image retrieval, from segmentation to disease prediction; (iii) provide a starting point for people interested in experimenting and perhaps contributing to the field of deep learning for medical imaging by pointing out good educational resources, state-of-the-art open-source code, and interesting sources of data and problems related medical imaging.

Partial alignment for 3 D point sets is a challenging problem for laser calibration and robot calibration due to the unbalance of data sets, especially when the overlap of data sets is low. Geometric features can promote the accuracy of alignment. However, the corresponding feature extraction methods are time consuming. The purpose of this paper is to find a framework for partial alignment by an adaptive trimmed strategy.,First, the authors propose an adaptive trimmed strategy based on point feature histograms (PFH) coding. Second, they obtain an initial transformation based on this partition, which improves the accuracy of the normal direction weighted trimmed iterative closest point (ICP) method. Third, they conduct a series of GPU parallel implementations for time efficiency.,The initial partition based on PFH feature improves the accuracy of the partial registration significantly. Moreover, the parallel GPU algorithms accelerate the alignment process.,This study is applicable to rigid transformation so far. It could be extended to non-rigid transformation.,In practice, point set alignment for calibration is a technique widely used in the fields of aircraft assembly, industry examination, simultaneous localization and mapping and surgery navigation.,Point set calibration is a building block in the field of intelligent manufacturing.,The contributions are as follows: first, the authors introduce a novel coarse alignment as an initial calibration by PFH descriptor similarity, which can be viewed as a coarse trimmed process by partitioning the data to the almost overlap part and the rest part; second, they reduce the computation time by GPU parallel coding during the acquisition of feature descriptor; finally, they use the weighted trimmed ICP method to refine the transformation.

Parallel calibration based on modified trim strategy

In this paper, we propose a novel inter-group image registration method to register different groups of images (e.g., young and elderly brains) simultaneously. Specifically, we use a hierarchical two-level graph to model the distribution of entire images on the manifold, with intra-graph representing the image distribution in each group and the inter-graph describing the relationship between two groups. Then the procedure of inter-group registration is formulated as a dynamic evolution of graph shrinkage. The advantage of our method is that the topology of entire image distribution is explored to guide the image registration. In this way, each image coordinates with its neighboring images on the manifold to deform towards the population center, by following the deformation pathway simultaneously optimized within the graph. Our proposed method has been also compared with other state-of-the-art inter-group registration methods, where our method achieves better registration results in terms of registration accuracy and robustness.

/pdf/inter-group-image-registration-by-hierarchical-graph-6ncvm8sdi7.pdf

Inter-group image registration by hierarchical graph shrinkage

Low-rank matrix completion aims to recover an unknown matrix from a subset of observed entries. In this paper, we solve the problem via optimization of the matrix manifold. Specially, we apply QR factorization to retraction during optimization. We devise two fast algorithms based on steepest gradient descent and conjugate gradient descent, and demonstrate their superiority over the promising baseline with the ratio of at least 24%.

Low-Rank Matrix Completion via QR-Based Retraction on Manifolds

Image registration aims to align two images through a spatial transformation. It plays a significant role in brain imaging analysis. In this research, we propose a symmetric diffeomorphic image registration model based on multi-label segmentation masks to solve the problems in brain MRI registration. We first introduce the similarity metric of the multi-label masks to the energy function, which improves the alignment of the brain region boundaries and the robustness to the noise. Next, we establish the model on the diffeomorphism group through the relaxation method and the inverse consistent constraint. The algorithm is designed through the local linearization and least-squares method. We then give spatially adaptive parameters to coordinate the descent of the energy function in different regions. The results show that our approach, compared with the mainstream methods, has better accuracy and noise resistance, and the transformations are more smooth and more reasonable.

/pdf/symmetric-diffeomorphic-image-registration-with-multi-label-13a0n7as.pdf

Symmetric Diffeomorphic Image Registration with Multi-Label Segmentation Masks

It is feasible to improve the performance of B-mode ultrasound (BUS) based computer-aided diagnosis (CAD) for liver cancers by transferring knowledge from contrast-enhanced ultrasound (CEUS) images. In this work, we propose a novel feature transformation based support vector machine plus (SVM+) algorithm for this transfer learning task by introducing feature transformation into the SVM+ framework (named FSVM+). Specifically, the transformation matrix in FSVM+ is learned to minimize the radius of the enclosing ball of all samples, while the SVM+ is used to maximize the margin between two classes. Moreover, to capture more transferable information from multiple CEUS phase images, a multi-view FSVM+ (MFSVM+) is further developed, which transfers knowledge from three CEUS images from three phases, i.e., arterial phase, portal venous phase, and delayed phase, to the BUS-based CAD model. MFSVM+ innovatively assigns appropriate weights for each CEUS image by calculating the maximum mean discrepancy between a pair of BUS and CEUS images, which can capture the relationship between source and target domains. The experimental results on a bi-modal ultrasound liver cancer dataset demonstrate that MFSVM+ achieves the best classification accuracy of 88.24±1.28%, sensitivity of 88.32±2.88%, specificity of 88.17±2.91%, suggesting its effectiveness in promoting the diagnostic accuracy of BUS-based CAD.

Shihui Ying

Papers

Parallel calibration based on modified trim strategy

Inter-group image registration by hierarchical graph shrinkage

Low-Rank Matrix Completion via QR-Based Retraction on Manifolds

Symmetric Diffeomorphic Image Registration with Multi-Label Segmentation Masks

Multi-View Feature Transformation Based SVM+ for Computer-Aided Diagnosis of Liver Cancers With Ultrasound Images