In this paper we report the set-up and results of the Multimodal Brain Tumor Image Segmentation Benchmark (BRATS) organized in conjunction with the MICCAI 2012 and 2013 conferences Twenty state-of-the-art tumor segmentation algorithms were applied to a set of 65 multi-contrast MR scans of low- and high-grade glioma patients—manually annotated by up to four raters—and to 65 comparable scans generated using tumor image simulation software Quantitative evaluations revealed considerable disagreement between the human raters in segmenting various tumor sub-regions (Dice scores in the range 74%–85%), illustrating the difficulty of this task We found that different algorithms worked best for different sub-regions (reaching performance comparable to human inter-rater variability), but that no single algorithm ranked in the top for all sub-regions simultaneously Fusing several good algorithms using a hierarchical majority vote yielded segmentations that consistently ranked above all individual algorithms, indicating remaining opportunities for further methodological improvements The BRATS image data and manual annotations continue to be publicly available through an online evaluation system as an ongoing benchmarking resource

/pdf/the-multimodal-brain-tumor-image-segmentation-benchmark-2gfraj98dk.pdf

The Multimodal Brain Tumor Image Segmentation Benchmark (BRATS)

Automatic whole-brain extraction from magnetic resonance images (MRI), also known as skull stripping, is a key component in most neuroimage pipelines. As the first element in the chain, its robustness is critical for the overall performance of the system. Many skull stripping methods have been proposed, but the problem is not considered to be completely solved yet. Many systems in the literature have good performance on certain datasets (mostly the datasets they were trained/tuned on), but fail to produce satisfactory results when the acquisition conditions or study populations are different. In this paper we introduce a robust, learning-based brain extraction system (ROBEX). The method combines a discriminative and a generative model to achieve the final result. The discriminative model is a Random Forest classifier trained to detect the brain boundary; the generative model is a point distribution model that ensures that the result is plausible. When a new image is presented to the system, the generative model is explored to find the contour with highest likelihood according to the discriminative model. Because the target shape is in general not perfectly represented by the generative model, the contour is refined using graph cuts to obtain the final segmentation. Both models were trained using 92 scans from a proprietary dataset but they achieve a high degree of robustness on a variety of other datasets. ROBEX was compared with six other popular, publicly available methods (BET, BSE, FreeSurfer, AFNI, BridgeBurner, and GCUT) on three publicly available datasets (IBSR, LPBA40, and OASIS, 137 scans in total) that include a wide range of acquisition hardware and a highly variable population (different age groups, healthy/diseased). The results show that ROBEX provides significantly improved performance measures for almost every method/dataset combination.

/pdf/robust-brain-extraction-across-datasets-and-comparison-with-2z6hms1c27.pdf

Robust Brain Extraction Across Datasets and Comparison With Publicly Available Methods

Image registration and fusion algorithms exist in almost every software system that creates or uses images in radiotherapy. Most treatment planning systems support some form of image registration and fusion to allow the use of multimodality and time-series image data and even anatomical atlases to assist in target volume and normal tissue delineation. Treatment delivery systems perform registration and fusion between the planning images and the in-room images acquired during the treatment to assist patient positioning. Advanced applications are beginning to support daily dose assessment and enable adaptive radiotherapy using image registration and fusion to propagate contours and accumulate dose between image data taken over the course of therapy to provide up-to-date estimates of anatomical changes and delivered dose. This information aids in the detection of anatomical and functional changes that might elicit changes in the treatment plan or prescription.

As the output of the image registration process is always used as the input of another process for planning or delivery, it is important to understand and communicate the uncertainty associated with the software in general and the result of a specific registration. Unfortunately, there is no standard mathematical formalism to perform this for real-world situations where noise, distortion, and complex anatomical variations can occur. Validation of the software systems performance is also complicated by the lack of documentation available from commercial systems leading to use of these systems in undesirable ‘black-box’ fashion.

In view of this situation and the central role that image registration and fusion play in treatment planning and delivery, the Therapy Physics Committee of the American Association of Physicists in Medicine commissioned Task Group 132 to review current approaches and solutions for image registration (both rigid and deformable) in radiotherapy and to provide recommendations for quality assurance and quality control of these clinical processes.

https://aapm.onlinelibrary.wiley.com/doi/epdf/10.1002/mp.12256

Use of image registration and fusion algorithms and techniques in radiotherapy: Report of the AAPM Radiation Therapy Committee Task Group No. 132.

Integration of visual characterization of emphysema and airway abnormalities with physiologic and quantitative CT assessment permits categorization of chronic obstructive pulmonary disease into distinct structurally and functionally defined subtypes.

/pdf/ct-definable-subtypes-of-chronic-obstructive-pulmonary-42hf1lgqbk.pdf

CT-Definable Subtypes of Chronic Obstructive Pulmonary Disease: A Statement of the Fleischner Society.

The establishment of image correspondence through robust image registration is critical to many clinical tasks such as image fusion, organ atlas creation, and tumor growth monitoring and is a very challenging problem. Since the beginning of the recent deep learning renaissance, the medical imaging research community has developed deep learning-based approaches and achieved the state-of-the-art in many applications, including image registration. The rapid adoption of deep learning for image registration applications over the past few years necessitates a comprehensive summary and outlook, which is the main scope of this survey. This requires placing a focus on the different research areas as well as highlighting challenges that practitioners face. This survey, therefore, outlines the evolution of deep learning-based medical image registration in the context of both research challenges and relevant innovations in the past few years. Further, this survey highlights future research directions to show how this field may be possibly moved forward to the next level.

Deep learning in medical image registration: a survey

This paper describes a framework for establishing a reference airway tree segmentation, which was used to quantitatively evaluate 15 different airway tree extraction algorithms in a standardized manner. Because of the sheer difficulty involved in manually constructing a complete reference standard from scratch, we propose to construct the reference using results from all algorithms that are to be evaluated. We start by subdividing each segmented airway tree into its individual branch segments. Each branch segment is then visually scored by trained observers to determine whether or not it is a correctly segmented part of the airway tree. Finally, the reference airway trees are constructed by taking the union of all correctly extracted branch segments. Fifteen airway tree extraction algorithms from different research groups are evaluated on a diverse set of 20 chest computed tomography (CT) scans of subjects ranging from healthy volunteers to patients with severe pathologies, scanned at different sites, with different CT scanner brands, models, and scanning protocols. Three performance measures covering different aspects of segmentation quality were computed for all participating algorithms. Results from the evaluation showed that no single algorithm could extract more than an average of 74% of the total length of all branches in the reference standard, indicating substantial differences between the algorithms. A fusion scheme that obtained superior results is presented, demonstrating that there is complementary information provided by the different algorithms and there is still room for further improvements in airway segmentation algorithms.

/pdf/extraction-of-airways-from-ct-exact-09-3csyoa7gg4.pdf

Extraction of Airways From CT (EXACT'09)

Sliding motion is a challenge for deformable image registration because it leads to discontinuities in the sought deformation. In this paper, we present a method to handle sliding motion using multiple B-spline transforms. The proposed method decomposes the sought deformation into sliding regions to allow discontinuities at their interfaces, but prevents unrealistic solutions by forcing those interfaces to match. The method was evaluated on 16 lung cancer patients against a single B-spline transform approach and a multi B-spline transforms approach without the sliding constraint at the interface. The target registration error (TRE) was significantly lower with the proposed method (TRE = 1.5 mm) than with the single B-spline approach (TRE = 3.7 mm) and was comparable to the multi B-spline approach without the sliding constraint (TRE = 1.4 mm). The proposed method was also more accurate along region interfaces, with 37% less gaps and overlaps when compared to the multi B-spline transforms without the sliding constraint.

/pdf/registration-of-sliding-objects-using-direction-dependent-b-3vway3oca2.pdf

Registration of sliding objects using direction dependent B-splines decomposition.

In this paper, an improved method for intrathoracic airway tree segmentation is proposed. We present efficiency and robustness enhancements to a segmentation procedure previously proposed by Tschirren et al. IEEE TMI, 24(12), 2005. Their method is reviewed and our approach is described. Our method is demonstrated on experimental CT data. Finally, we give some concluding remarks and discuss future implementations.

Efficient approaches to intrathoracic airway tree segmentations

VV, a 4D slicer

Purpose: To automatically learn directional relative positions (DRP) between mediastinal lymph node stations and anatomical organs. Those spatial relationships are used to semiautomatically segment the stations in thoracic CT images. Methods: Fuzzy maps of DRP were automatically extracted by a learning procedure from a database composed of images with stations and anatomical structures manually segmented by consensus between experts. Spatial relationships common to all patients were retained. The segmentation of a new image used an initial rough delineation of anatomical organs and applied the DRP operators. The algorithm was tested with a leave-one-out approach on a database of 5 patients with 10 lymph stations and 30 anatomical structures each. Results were compared to expert delineations with dice similarity coefficient (DSC) and bidirectional local distance (BLD). Results: The overall mean DSC was 66% and the mean BLD was 1.7 mm. Best matches were obtained from stations S3P or S4R while lower matches were obtained for stations 1R and 1L. On average, more than 30 spatial relationships were automatically extracted for each station. Conclusions: This feasibility study suggests that mediastinal lymph node stations could be satisfactory segmented from thoracic CT using automatically extracted positional relationships with anatomical organs. This approach requires the anatomical structures to be initially roughly delineated. A similar approach could be applied to other sites where spatial relationships exists between anatomical structures. The complete database of the five reference cases is made publicly available.

/pdf/learning-directional-relative-positions-between-mediastinal-3ejgssgrjm.pdf

Romulo Pinho

Papers

Extraction of Airways From CT (EXACT'09)

Registration of sliding objects using direction dependent B-splines decomposition.

Efficient approaches to intrathoracic airway tree segmentations

VV, a 4D slicer

Learning directional relative positions between mediastinal lymph node stations and organs.