We propose a deep convolutional neural network architecture codenamed Inception that achieves the new state of the art for classification and detection in the ImageNet Large-Scale Visual Recognition Challenge 2014 (ILSVRC14). The main hallmark of this architecture is the improved utilization of the computing resources inside the network. By a carefully crafted design, we increased the depth and width of the network while keeping the computational budget constant. To optimize quality, the architectural decisions were based on the Hebbian principle and the intuition of multi-scale processing. One particular incarnation used in our submission for ILSVRC14 is called GoogLeNet, a 22 layers deep network, the quality of which is assessed in the context of classification and detection.

/pdf/going-deeper-with-convolutions-1yobw2o2ds.pdf

Going deeper with convolutions

A survey on deep learning in medical image analysis

Convolutional Neural Networks (CNNs) have been recently employed to solve problems from both the computer vision and medical image analysis fields. Despite their popularity, most approaches are only able to process 2D images while most medical data used in clinical practice consists of 3D volumes. In this work we propose an approach to 3D image segmentation based on a volumetric, fully convolutional, neural network. Our CNN is trained end-to-end on MRI volumes depicting prostate, and learns to predict segmentation for the whole volume at once. We introduce a novel objective function, that we optimise during training, based on Dice coefficient. In this way we can deal with situations where there is a strong imbalance between the number of foreground and background voxels. To cope with the limited number of annotated volumes available for training, we augment the data applying random non-linear transformations and histogram matching. We show in our experimental evaluation that our approach achieves good performances on challenging test data while requiring only a fraction of the processing time needed by other previous methods.

V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation

Remarkable progress has been made in image recognition, primarily due to the availability of large-scale annotated datasets and deep convolutional neural networks (CNNs). CNNs enable learning data-driven, highly representative, hierarchical image features from sufficient training data. However, obtaining datasets as comprehensively annotated as ImageNet in the medical imaging domain remains a challenge. There are currently three major techniques that successfully employ CNNs to medical image classification: training the CNN from scratch, using off-the-shelf pre-trained CNN features, and conducting unsupervised CNN pre-training with supervised fine-tuning. Another effective method is transfer learning, i.e., fine-tuning CNN models pre-trained from natural image dataset to medical image tasks. In this paper, we exploit three important, but previously understudied factors of employing deep convolutional neural networks to computer-aided detection problems. We first explore and evaluate different CNN architectures. The studied models contain 5 thousand to 160 million parameters, and vary in numbers of layers. We then evaluate the influence of dataset scale and spatial image context on performance. Finally, we examine when and why transfer learning from pre-trained ImageNet (via fine-tuning) can be useful. We study two specific computer-aided detection (CADe) problems, namely thoraco-abdominal lymph node (LN) detection and interstitial lung disease (ILD) classification. We achieve the state-of-the-art performance on the mediastinal LN detection, and report the first five-fold cross-validation classification results on predicting axial CT slices with ILD categories. Our extensive empirical evaluation, CNN model analysis and valuable insights can be extended to the design of high performance CAD systems for other medical imaging tasks.

Deep Convolutional Neural Networks for Computer-Aided Detection: CNN Architectures, Dataset Characteristics and Transfer Learning

Humans perceive the three-dimensional structure of the world with apparent ease. However, despite all of the recent advances in computer vision research, the dream of having a computer interpret an image at the same level as a two-year old remains elusive. Why is computer vision such a challenging problem and what is the current state of the art? Computer Vision: Algorithms and Applications explores the variety of techniques commonly used to analyze and interpret images. It also describes challenging real-world applications where vision is being successfully used, both for specialized applications such as medical imaging, and for fun, consumer-level tasks such as image editing and stitching, which students can apply to their own personal photos and videos. More than just a source of recipes, this exceptionally authoritative and comprehensive textbook/reference also takes a scientific approach to basic vision problems, formulating physical models of the imaging process before inverting them to produce descriptions of a scene. These problems are also analyzed using statistical models and solved using rigorous engineering techniques Topics and features: structured to support active curricula and project-oriented courses, with tips in the Introduction for using the book in a variety of customized courses; presents exercises at the end of each chapter with a heavy emphasis on testing algorithms and containing numerous suggestions for small mid-term projects; provides additional material and more detailed mathematical topics in the Appendices, which cover linear algebra, numerical techniques, and Bayesian estimation theory; suggests additional reading at the end of each chapter, including the latest research in each sub-field, in addition to a full Bibliography at the end of the book; supplies supplementary course material for students at the associated website, http://szeliski.org/Book/. Suitable for an upper-level undergraduate or graduate-level course in computer science or engineering, this textbook focuses on basic techniques that work under real-world conditions and encourages students to push their creative boundaries. Its design and exposition also make it eminently suitable as a unique reference to the fundamental techniques and current research literature in computer vision.

/pdf/computer-vision-algorithms-and-applications-25dn6wu83j.pdf

Computer Vision: Algorithms and Applications

Pose estimation, i.e. predicting a 3D rigid transformation with respect to a fixed co-ordinate frame in, SE(3), is an omnipresent problem in medical image analysis. Deep learning methods often parameterise poses with a representation that separates rotation and translation. As commonly available frameworks do not provide means to calculate loss on a manifold, regression is usually performed using the L2-norm independently on the rotation’s and the translation’s parameterisations. This is a metric for linear spaces that does not take into account the Lie group structure of SE(3).

Computing CNN Loss and Gradients for Pose Estimation with Riemannian Geometry

Nowadays, hepatic artery catheterizations are performed under live 2D X-ray fluoroscopy guidance, where the visualization of blood vessels requires the injection of contrast agent. The projection of a 3D static roadmap of the complex branches of the liver artery system onto 2D fluoroscopy images can aid catheter navigation and minimize the use of contrast agent. However, the presence of a significant hepatic motion due to patient's respiration necessitates a real-time motion correction in order to align the projected vessels. The objective of our work is to introduce dynamic roadmaps into clinical workflow for hepatic artery catheterizations and allow for continuous visualization of the vessels in 2D fluoroscopy images without additional contrast injection. To this end, we propose a method for real-time estimation of the apparent displacement of the hepatic arteries in 2D flouroscopy images. Our approach approximates respiratory motion of hepatic arteries from the catheter motion in 2D fluoroscopy images. The proposed method consists of two main steps. First, a filtering is applied to 2D fluoroscopy images in order to enhance the catheter and reduce the noise level. Then, a part of the catheter is tracked in the filtered images using template matching. A dynamic template update strategy makes our method robust to deformations. The accuracy and robustness of the algorithm are demonstrated by experimental studies on 22 simulated and 4 clinical sequences containing 330 and 571 image frames, respectively.

/pdf/real-time-respiratory-motion-tracking-roadmap-correction-for-44yane24y9.pdf

Real-time respiratory motion tracking: roadmap correction for hepatic artery catheterizations

This is an empirical study to investigate the impact of scanner effects when using machine learning on multi-site neuroimaging data. We utilize structural T1-weighted brain MRI obtained from two different studies, Cam-CAN and UK Biobank. For the purpose of our investigation, we construct a dataset consisting of brain scans from 592 age- and sex-matched individuals, 296 subjects from each original study. Our results demonstrate that even after careful pre-processing with state-of-the-art neuroimaging pipelines a classifier can easily distinguish between the origin of the data with very high accuracy. Our analysis on the example application of sex classification suggests that current approaches to harmonize data are unable to remove scanner-specific bias leading to overly optimistic performance estimates and poor generalization. We conclude that multi-site data harmonization remains an open challenge and particular care needs to be taken when using such data with advanced machine learning methods for predictive modelling.

Machine Learning with Multi-Site Imaging Data: An Empirical Study on the Impact of Scanner Effects.

Methods that leverage neighbourhood structures in high-dimensional image spaces have recently attracted attention These approaches extract information from a new image using its “neighbours” in the image space equipped with an application-specific distance Finding the neighbourhood of a given image is challenging due to large dataset sizes and costly distance evaluations Furthermore, automatic neighbourhood search for a new image is currently not possible when the distance is based on ground truth annotations In this article we present a general and efficient solution to these problems “Neighbourhood Approximation Forests” (NAF) is a supervised learning algorithm that approximates the neighbourhood structure resulting from an arbitrary distance As NAF uses only image intensities to infer neighbours it can also be applied to distances based on ground truth annotations We demonstrate NAF in two scenarios: i) choosing neighbours with respect to a deformation-based distance, and ii) age prediction from brain MRI The experiments show NAF’s approximation quality, computational advantages and use in different contexts

/pdf/neighbourhood-approximation-forests-22x9qfhsc9.pdf

Neighbourhood approximation forests.

Accurate and reliable registration of longitudinal spine images is essential for assessment of disease progression and surgical outcome. Implementing a fully automatic and robust registration for clinical use, however, is challenging since standard registration techniques often fail due to poor initial alignment. The main causes of registration failure are the small overlap between scans which focus on different parts of the spine and/or substantial change in shape (e.g. after correction of abnormal curvature) and appearance (e.g. due to surgical implants). To overcome these issues we propose a registration approach which incorporates estimates of vertebrae locations obtained from a learning-based classification method. These location priors are used to initialize the registration and to provide semantic information within the optimization process. Quantitative evaluation on a database of 93 patients with a total of 276 registrations on longitudinal spine CT demonstrate that our registration method significantly reduces the number of failure cases.

/pdf/robust-registration-of-longitudinal-spine-ct-pgnwz05ajg.pdf

Ben Glocker

Papers

Computing CNN Loss and Gradients for Pose Estimation with Riemannian Geometry

Real-time respiratory motion tracking: roadmap correction for hepatic artery catheterizations

Machine Learning with Multi-Site Imaging Data: An Empirical Study on the Impact of Scanner Effects.

Neighbourhood approximation forests.

Robust registration of longitudinal spine CT