A survey on deep learning in medical image analysis

When building a unified vision system or gradually adding new capabilities to a system, the usual assumption is that training data for all tasks is always available. However, as the number of tasks grows, storing and retraining on such data becomes infeasible. A new problem arises where we add new capabilities to a Convolutional Neural Network (CNN), but the training data for its existing capabilities are unavailable. We propose our Learning without Forgetting method, which uses only new task data to train the network while preserving the original capabilities. Our method performs favorably compared to commonly used feature extraction and fine-tuning adaption techniques and performs similarly to multitask learning that uses original task data we assume unavailable. A more surprising observation is that Learning without Forgetting may be able to replace fine-tuning with similar old and new task datasets for improved new task performance.

Learning without Forgetting

What has happened in machine learning lately, and what does it mean for the future of medical image analysis? Machine learning has witnessed a tremendous amount of attention over the last few years. The current boom started around 2009 when so-called deep artificial neural networks began outperforming other established models on a number of important benchmarks. Deep neural networks are now the state-of-the-art machine learning models across a variety of areas, from image analysis to natural language processing, and widely deployed in academia and industry. These developments have a huge potential for medical imaging technology, medical data analysis, medical diagnostics and healthcare in general, slowly being realized. We provide a short overview of recent advances and some associated challenges in machine learning applied to medical image processing and image analysis. As this has become a very broad and fast expanding field we will not survey the entire landscape of applications, but put particular focus on deep learning in MRI. 
Our aim is threefold: (i) give a brief introduction to deep learning with pointers to core references; (ii) indicate how deep learning has been applied to the entire MRI processing chain, from acquisition to image retrieval, from segmentation to disease prediction; (iii) provide a starting point for people interested in experimenting and perhaps contributing to the field of machine learning for medical imaging by pointing out good educational resources, state-of-the-art open-source code, and interesting sources of data and problems related medical imaging.

An overview of deep learning in medical imaging focusing on MRI

Rapid advances in hardware-based technologies during the past decades have opened up new possibilities for life scientists to gather multimodal data in various application domains, such as omics , bioimaging , medical imaging , and (brain/body)–machine interfaces . These have generated novel opportunities for development of dedicated data-intensive machine learning techniques. In particular, recent research in deep learning (DL), reinforcement learning (RL), and their combination (deep RL) promise to revolutionize the future of artificial intelligence. The growth in computational power accompanied by faster and increased data storage, and declining computing costs have already allowed scientists in various fields to apply these techniques on data sets that were previously intractable owing to their size and complexity. This paper provides a comprehensive survey on the application of DL, RL, and deep RL techniques in mining biological data. In addition, we compare the performances of DL techniques when applied to different data sets across various application domains. Finally, we outline open issues in this challenging research area and discuss future development perspectives.

/pdf/applications-of-deep-learning-and-reinforcement-learning-to-3l72kwyi7n.pdf

Applications of Deep Learning and Reinforcement Learning to Biological Data

What has happened in machine learning lately, and what does it mean for the future of medical image analysis? Machine learning has witnessed a tremendous amount of attention over the last few years. The current boom started around 2009 when so-called deep artificial neural networks began outperforming other established models on a number of important benchmarks. Deep neural networks are now the state-of-the-art machine learning models across a variety of areas, from image analysis to natural language processing, and widely deployed in academia and industry. These developments have a huge potential for medical imaging technology, medical data analysis, medical diagnostics and healthcare in general, slowly being realized. We provide a short overview of recent advances and some associated challenges in machine learning applied to medical image processing and image analysis. As this has become a very broad and fast expanding field we will not survey the entire landscape of applications, but put particular focus on deep learning in MRI. Our aim is threefold: (i) give a brief introduction to deep learning with pointers to core references; (ii) indicate how deep learning has been applied to the entire MRI processing chain, from acquisition to image retrieval, from segmentation to disease prediction; (iii) provide a starting point for people interested in experimenting and perhaps contributing to the field of deep learning for medical imaging by pointing out good educational resources, state-of-the-art open-source code, and interesting sources of data and problems related medical imaging.

Person re-identification is one of the key issues in both machine learning and video monitor application. In particular, defining an appropriate distance metric between the person images is very important. Existing metric learning approaches used in person re-identification either learn a single measure, or ignore the positive semi-definite (PSD) of measurement matrix, at the same time, since the number of negative sample pairs largely exceeds the number of positive sample pairs, some metric learning methods are largely influenced by the sample imbalance. Considering the above issues, we propose a new adaptive local metric learning method with positive semi-definite (PSD) constraint. Unlike existing metric learning methods which learn a single distance metric, we use an approximation error bound of a smooth metric matrix function over the data manifold to learn local metrics as linear combinations of basis metrics defined on anchor points over different regions of the instance space. Besides, we develop an efficient two stage algorithm that first learns the anchor points and the linear combinations of each instance, then learns the metric matrices of the anchor points. We employ the fast iterative shrinkage-thresholding algorithm which is a fast first-order optimization algorithm in the learning process of the linear combinations as well as the basis metrics of the anchor points. Our metric learning method has excellent performance. We firstly apply the proposed method on 5 UCI databases, which are widely used in machine learning, to test and evaluate the effectiveness of the proposed method. Then the proposed approach is applied for person re-identification, achieving better performance on three challenging databases (GRID, VIPeR, CUHK01) than the existing methods. The experimental results show that the proposed method can prvide the theoretical and practical support for the person re-identification problem.

Asymmetric Local Metric Learning with PSD Constraint for Person Re-identification

Computer-aided diagnosis (CAD) can help pathologists improve diagnostic accuracy together with consistency and repeatability for cancers. However, the CAD models trained with the histopathological images only from a single center (hospital) generally suffer from the generalization problem due to the straining inconsistencies among different centers. In this work, we propose a pseudo-data based self-supervised federated learning (FL) framework, named SSL-FT-BT, to improve both the diagnostic accuracy and generalization of CAD models. Specifically, the pseudo histopathological images are generated from each center, which contains inherent and specific properties corresponding to the real images in this center, but does not include the privacy information. These pseudo images are then shared in the central server for self-supervised learning (SSL). A multi-task SSL is then designed to fully learn both the center-specific information and common inherent representation according to the data characteristics. Moreover, a novel Barlow Twins based FL (FL-BT) algorithm is proposed to improve the local training for the CAD model in each center by conducting contrastive learning, which benefits the optimization of the global model in the FL procedure. The experimental results on three public histopathological image datasets indicate the effectiveness of the proposed SSL-FL-BT on both diagnostic accuracy and generalization.

Pseudo-Data based Self-Supervised Federated Learning for Classification of Histopathological Images

The accuracy of classification and retrieval significantly depends on the metric used to compute the similarity between samples. For preserving the geometric structure, the symmetric positive definite (SPD) manifold is introduced into the metric learning problem. However, the SPD constraint is too strict to describe the real data distribution. In this paper, we extend the intrinsic metric learning problem to semi-definite case, by which the data distribution is better described for various classification tasks. First, we formulate the metric learning as a minimization problem to the SPD manifold on subspace, which not only considers to balance the information between inner classes and inter classes by an adaptive tradeoff parameter but also improves the robustness by the low-rank subspaces presentation. Thus, it benefits to design a structure-preserving algorithm on subspace by using the geodesic structure of the SPD subspace. To solve this model, we develop an iterative strategy to update the intrinsic metric and the subspace structure, respectively. Finally, we compare our proposed method with ten state-of-the-art methods on four data sets. The numerical results validate that our method can significantly improve the description of the data distribution, and hence, the performance of the image classification task.

/pdf/intrinsic-metric-learning-with-subspace-representation-2tlr6cbpku.pdf

Intrinsic Metric Learning With Subspace Representation

In this paper, we address the unsupervised subspace learning from a geometric viewpoint. First, we formulate the subspace learning as an inverse problem on Grassmannian manifold by considering all subspaces as points on it. Then, to make the model computable, we parameterize the Grassmannian manifold by using an orbit of rotation group action on all standard subspaces, which are spanned by the orthonormal basis. Further, to improve the robustness, we introduce a low-rank regularizer which makes the dimension of subspace as low as possible. Thus, the subspace learning problem is transferred to a minimization problem with variables of rotation and dimension. Then, we adopt the alternately iterative strategy to optimize the variables, where a structure-preserving method, based on the geodesic structure of the rotation group, is designed to update the rotation. Finally, we compare the proposed approach with six state-of-the-art methods on three different kinds of real datasets. The experimental results validate that our proposed method outperforms all compared methods.

/pdf/geometric-understanding-for-unsupervised-subspace-learning-4qx44vy44p.pdf

Geometric Understanding for Unsupervised Subspace Learning.

In this paper, a 2D shape registration algorithm for noisy data is established by combining the Iterative Closest
Point (ICP) method, Expectation Maximization (EM) method, and Lie Group representation. First, the problem
is formulated by a minimization problem with two sets of variables: the point-to-point correspondence, and the
transformation (i.e., rotation, scaling and translation) between two data sets. The conventional way for solving
this model is by iterating alternatively the following two steps: 1) having the transformation fixed, solve the
correspondence, and 2) having the correspondence fixed, solve the transformation. In our approach, to enhance
the robustness, the EM algorithm is introduced to find the correspondence by a probability which covers the
relationship of all points, instead of one-to-one closest correspondence in ICP. Meanwhile, Lie group is used to
parameterize transformation, i.e., in the iteration, the rotation, scaling and translation are all elements within
respective Lie groups, and we use the element of Lie algebra to represent that of Lie group near the identity via
exponential map. This forms a unified framework for registration algorithms. Then, transformation is estimated
by solving a quadratic programming. The experimental result in 2D shape registration demonstrates that,
compared with Lie-ICP, our algorithm is robuster and more accurate.

Shihui Ying

Papers

Asymmetric Local Metric Learning with PSD Constraint for Person Re-identification

Pseudo-Data based Self-Supervised Federated Learning for Classification of Histopathological Images

Intrinsic Metric Learning With Subspace Representation

Geometric Understanding for Unsupervised Subspace Learning.

Lie-EM-ICP algorithm: a novel frame for 2D shape registration