The tremendous success of machine learning algorithms at image recognition tasks in recent years intersects with a time of dramatically increased use of electronic medical records and diagnostic imaging. This review introduces the machine learning algorithms as applied to medical image analysis, focusing on convolutional neural networks, and emphasizing clinical aspects of the field. The advantage of machine learning in an era of medical big data is that significant hierarchal relationships within the data can be discovered algorithmically without laborious hand-crafting of features. We cover key research areas and applications of medical image classification, localization, detection, segmentation, and registration. We conclude by discussing research obstacles, emerging trends, and possible future directions.

/pdf/deep-learning-applications-in-medical-image-analysis-2l6wv26xks.pdf

Deep Learning Applications in Medical Image Analysis

Judgement, as one of the core tenets of medicine, relies upon the integration of multilayered data with nuanced decision making. Cancer offers a unique context for medical decisions given not only its variegated forms with evolution of disease but also the need to take into account the individual condition of patients, their ability to receive treatment, and their responses to treatment. Challenges remain in the accurate detection, characterization, and monitoring of cancers despite improved technologies. Radiographic assessment of disease most commonly relies upon visual evaluations, the interpretations of which may be augmented by advanced computational analyses. In particular, artificial intelligence (AI) promises to make great strides in the qualitative interpretation of cancer imaging by expert clinicians, including volumetric delineation of tumors over time, extrapolation of the tumor genotype and biological course from its radiographic phenotype, prediction of clinical outcome, and assessment of the impact of disease and treatment on adjacent organs. AI may automate processes in the initial interpretation of images and shift the clinical workflow of radiographic detection, management decisions on whether or not to administer an intervention, and subsequent observation to a yet to be envisioned paradigm. Here, the authors review the current state of AI as applied to medical imaging of cancer and describe advances in 4 tumor types (lung, brain, breast, and prostate) to illustrate how common clinical problems are being addressed. Although most studies evaluating AI applications in oncology to date have not been vigorously validated for reproducibility and generalizability, the results do highlight increasingly concerted efforts in pushing AI technology to clinical use and to impact future directions in cancer care.

https://acsjournals.onlinelibrary.wiley.com/doi/epdf/10.3322/caac.21552

Artificial intelligence in cancer imaging: Clinical challenges and applications.

The goals of this review paper on deep learning (DL) in medical imaging and radiation therapy are to (a) summarize what has been achieved to date; (b) identify common and unique challenges, and strategies that researchers have taken to address these challenges; and (c) identify some of the promising avenues for the future both in terms of applications as well as technical innovations. We introduce the general principles of DL and convolutional neural networks, survey five major areas of application of DL in medical imaging and radiation therapy, identify common themes, discuss methods for dataset expansion, and conclude by summarizing lessons learned, remaining challenges, and future directions.

https://aapm.onlinelibrary.wiley.com/doi/epdf/10.1002/mp.13264

Deep learning in medical imaging and radiation therapy.

Medical image classification plays an essential role in clinical treatment and teaching tasks. However, the traditional method has reached its ceiling on performance. Moreover, by using them, much time and effort need to be spent on extracting and selecting classification features. The deep neural network is an emerging machine learning method that has proven its potential for different classification tasks. Notably, the convolutional neural network dominates with the best results on varying image classification tasks. However, medical image datasets are hard to collect because it needs a lot of professional expertise to label them. Therefore, this paper researches how to apply the convolutional neural network (CNN) based algorithm on a chest X-ray dataset to classify pneumonia. Three techniques are evaluated through experiments. These are linear support vector machine classifier with local rotation and orientation free features, transfer learning on two convolutional neural network models: Visual Geometry Group i.e., VGG16 and InceptionV3, and a capsule network training from scratch. Data augmentation is a data preprocessing method applied to all three methods. The results of the experiments show that data augmentation generally is an effective way for all three algorithms to improve performance. Also, Transfer learning is a more useful classification method on a small dataset compared to a support vector machine with oriented fast and rotated binary (ORB) robust independent elementary features and capsule network. In transfer learning, retraining specific features on a new target dataset is essential to improve performance. And, the second important factor is a proper network complexity that matches the scale of the dataset.

/pdf/deep-convolutional-neural-network-based-medical-image-2y1n15pcmh.pdf

Deep convolutional neural network based medical image classification for disease diagnosis

Artificial intelligence (AI) is rapidly moving from an experimental phase to an implementation phase in many fields, including medicine. The combination of improved availability of large datasets, increasing computing power, and advances in learning algorithms has created major performance breakthroughs in the development of AI applications. In the last 5 years, AI techniques known as deep learning have delivered rapidly improving performance in image recognition, caption generation, and speech recognition. Radiology, in particular, is a prime candidate for early adoption of these techniques. It is anticipated that the implementation of AI in radiology over the next decade will significantly improve the quality, value, and depth of radiology's contribution to patient care and population health, and will revolutionize radiologists' workflows. The Canadian Association of Radiologists (CAR) is the national voice of radiology committed to promoting the highest standards in patient-centered imaging, lifelong learning, and research. The CAR has created an AI working group with the mandate to discuss and deliberate on practice, policy, and patient care issues related to the introduction and implementation of AI in imaging. This white paper provides recommendations for the CAR derived from deliberations between members of the AI working group. This white paper on AI in radiology will inform CAR members and policymakers on key terminology, educational needs of members, research and development, partnerships, potential clinical applications, implementation, structure and governance, role of radiologists, and potential impact of AI on radiology in Canada.

Canadian Association of Radiologists White Paper on Artificial Intelligence in Radiology.

/pdf/deep-learning-in-medical-image-analysis-and-multimodal-55sfa0vx66.pdf

Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support

In this work we propose a technique to automatically estimate circular cross-sections of the vessels in CT scans. First, a circular contour is extracted for each slice of the CT by using the Hough transform. Afterward, the locations of the circles are optimized by means of a parametric snake model, and those circles which best fit the contours of the vessels are selected by applying a robust quality criterion. Finally, this collection of circles is used to estimate the local probability density functions of the image intensity inside and outside the vessels. We present a large variety of experiments on CT scans which show the reliability of the proposed method.

Intravascular Imaging and Computer Assisted Stenting, and Large-Scale Annotation of Biomedical Data and Expert Label Synthesis

We describe a detector of robotic instrument parts in imageguided surgery. The detector consists of a huge ensemble of scale-variant and pose-dedicated, rigid appearance templates. The templates, which are equipped with pose-related keypoints and segmentation masks, allow for explicit pose estimation and segmentation of multiple end-effectors as well as fine-grained non-maximum suppression. We train the templates by grouping examples of end-effector articulations, imaged at various viewpoints, in thus arising space of instrument shapes. Proposed shapebased grouping forms tight clusters of pose-specific end-effector appearance. Experimental results show that the proposed method can effectively estimate the end-effector pose and delineate its boundary while being trained with moderately sized data clusters. We then show that matching such huge ensemble of templates takes less than one second on commodity hardware.

Computer Assisted and Robotic Endoscopy and Clinical Image-Based Procedures

Positron Emission Tomography (PET) is widely used for lymphoma detection. It is often combined with the CT scan in order to provide anatomical information for helping lymphoma detection. Two common types of approaches can be distinguished for lymphoma detection and segmentation in PET. The first one is ROI dependent which needs a ROI defined by physicians who firstly detect where lymphomas are. The second one is based on machine learning methods which need a large learning database. However, such a large standard database is quite rare in medical field. Considering these problems, we propose a new approach which combines a multi-atlas segmentation of the CT with CRFs (Conditional Random Fields) segmentation method in PET. It consists of 3 steps. Firstly, an anatomical multi-atlas segmentation is applied on CT to locate and remove the organs having hyper metabolism in PET. Secondly, CRFs detect and segment the lymphoma regions in PET. The conditional probabilities used in CRFs are usually estimated by a learning step. In this work, we propose to estimate them in an unsupervised way. A list of the detected regions in 3D is visualized. The final step is to select real lymphomas by simply clicking on them. Our method is tested on ten patients. The rate of good detection is 100%. The average of Dice index over 10 patients for measuring the lymphoma is 80% compared to manual lymphoma segmentation. Comparing with other methods in terms of Dice index shows the best performance of our method.

Molecular Imaging, Reconstruction and Analysis of Moving Body Organs, and Stroke Imaging and Treatment

Accurate estimation of intracranial volume (ICV) is key in neuro-imaging-based volumetric studies, since estimation errors directly propagate to the ICV-corrected volumes used in subsequent analyses. ICV estimation through registration to a reference atlas has the advantage of not requiring manually delineated data, and can thus be applied to populations for which labeled data might be inexistent or scarce, e.g., preterm born animal models. However, such method is not robust, since the estimation depends on a single registration. Here we present a groupwise, template-free ICV estimation method that overcomes this limitation. The method quickly aligns pairs of images using linear registration at low resolution, and then computes the most likely ICV values using a Bayesian framework. The algorithm is robust against single registration errors, which are corrected by registrations to other subjects. The algorithm was evaluated on a pilot dataset of rabbit brain MRI (N = 7), in which the estimated ICV was highly correlated (ρ = 0.99) with ground truth values derived from manual delineations. Additional regression and discrimination experiments with human hippocampal volume on a subset of ADNI (N = 150) yielded reduced sample sizes and increased classification accuracy, compared with using a reference atlas.

Tal Arbel

Papers

Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support

Intravascular Imaging and Computer Assisted Stenting, and Large-Scale Annotation of Biomedical Data and Expert Label Synthesis

Computer Assisted and Robotic Endoscopy and Clinical Image-Based Procedures

Molecular Imaging, Reconstruction and Analysis of Moving Body Organs, and Stroke Imaging and Treatment

Fetal, Infant and Ophthalmic Medical Image Analysis