What has happened in machine learning lately, and what does it mean for the future of medical image analysis? Machine learning has witnessed a tremendous amount of attention over the last few years. The current boom started around 2009 when so-called deep artificial neural networks began outperforming other established models on a number of important benchmarks. Deep neural networks are now the state-of-the-art machine learning models across a variety of areas, from image analysis to natural language processing, and widely deployed in academia and industry. These developments have a huge potential for medical imaging technology, medical data analysis, medical diagnostics and healthcare in general, slowly being realized. We provide a short overview of recent advances and some associated challenges in machine learning applied to medical image processing and image analysis. As this has become a very broad and fast expanding field we will not survey the entire landscape of applications, but put particular focus on deep learning in MRI. 
Our aim is threefold: (i) give a brief introduction to deep learning with pointers to core references; (ii) indicate how deep learning has been applied to the entire MRI processing chain, from acquisition to image retrieval, from segmentation to disease prediction; (iii) provide a starting point for people interested in experimenting and perhaps contributing to the field of machine learning for medical imaging by pointing out good educational resources, state-of-the-art open-source code, and interesting sources of data and problems related medical imaging.

An overview of deep learning in medical imaging focusing on MRI

What has happened in machine learning lately, and what does it mean for the future of medical image analysis? Machine learning has witnessed a tremendous amount of attention over the last few years. The current boom started around 2009 when so-called deep artificial neural networks began outperforming other established models on a number of important benchmarks. Deep neural networks are now the state-of-the-art machine learning models across a variety of areas, from image analysis to natural language processing, and widely deployed in academia and industry. These developments have a huge potential for medical imaging technology, medical data analysis, medical diagnostics and healthcare in general, slowly being realized. We provide a short overview of recent advances and some associated challenges in machine learning applied to medical image processing and image analysis. As this has become a very broad and fast expanding field we will not survey the entire landscape of applications, but put particular focus on deep learning in MRI. Our aim is threefold: (i) give a brief introduction to deep learning with pointers to core references; (ii) indicate how deep learning has been applied to the entire MRI processing chain, from acquisition to image retrieval, from segmentation to disease prediction; (iii) provide a starting point for people interested in experimenting and perhaps contributing to the field of deep learning for medical imaging by pointing out good educational resources, state-of-the-art open-source code, and interesting sources of data and problems related medical imaging.

Deep learning is a branch of artificial intelligence where networks of simple interconnected units are used to extract patterns from data in order to solve complex problems. Deep-learning algorithms have shown groundbreaking performance in a variety of sophisticated tasks, especially those related to images. They have often matched or exceeded human performance. Since the medical field of radiology mainly relies on extracting useful information from images, it is a very natural application area for deep learning, and research in this area has rapidly grown in recent years. In this article, we discuss the general context of radiology and opportunities for application of deep-learning algorithms. We also introduce basic concepts of deep learning, including convolutional neural networks. Then, we present a survey of the research in deep learning applied to radiology. We organize the studies by the types of specific tasks that they attempt to solve and review a broad range of deep-learning algorithms being utilized. Finally, we briefly discuss opportunities and challenges for incorporating deep learning in the radiology practice of the future. Level of Evidence: 3 Technical Efficacy: Stage 1 J. Magn. Reson. Imaging 2019;49:939-954.

/pdf/deep-learning-in-radiology-an-overview-of-the-concepts-and-a-35hnf1jtsa.pdf

Deep learning in radiology: An overview of the concepts and a survey of the state of the art with focus on MRI.

Computed tomography (CT) is widely used in screening, diagnosis, and image-guided therapy for both clinical and research purposes. Since CT involves ionizing radiation, an overarching thrust of related technical research is development of novel methods enabling ultrahigh quality imaging with fine structural details while reducing the X-ray radiation. In this paper, we present a semi-supervised deep learning approach to accurately recover high-resolution (HR) CT images from low-resolution (LR) counterparts. Specifically, with the generative adversarial network (GAN) as the building block, we enforce the cycle-consistency in terms of the Wasserstein distance to establish a nonlinear end-to-end mapping from noisy LR input images to denoised and deblurred HR outputs. We also include the joint constraints in the loss function to facilitate structural preservation. In this deep imaging process, we incorporate deep convolutional neural network (CNN), residual learning, and network in network techniques for feature extraction and restoration. In contrast to the current trend of increasing network depth and complexity to boost the CT imaging performance, which limit its real-world applications by imposing considerable computational and memory overheads, we apply a parallel $1\times1$ CNN to compress the output of the hidden layer and optimize the number of layers and the number of filters for each convolutional layer. Quantitative and qualitative evaluations demonstrate that our proposed model is accurate, efficient and robust for super-resolution (SR) image restoration from noisy LR input images. In particular, we validate our composite SR networks on three large-scale CT datasets, and obtain promising results as compared to the other state-of-the-art methods.

CT Super-resolution GAN Constrained by the Identical, Residual, and Cycle Learning Ensemble(GAN-CIRCLE)

Deep learning is a branch of artificial intelligence where networks of simple interconnected units are used to extract patterns from data in order to solve complex problems. Deep learning algorithms have shown groundbreaking performance in a variety of sophisticated tasks, especially those related to images. They have often matched or exceeded human performance. Since the medical field of radiology mostly relies on extracting useful information from images, it is a very natural application area for deep learning, and research in this area has rapidly grown in recent years. In this article, we review the clinical reality of radiology and discuss the opportunities for application of deep learning algorithms. We also introduce basic concepts of deep learning including convolutional neural networks. Then, we present a survey of the research in deep learning applied to radiology. We organize the studies by the types of specific tasks that they attempt to solve and review the broad range of utilized deep learning algorithms. Finally, we briefly discuss opportunities and challenges for incorporating deep learning in the radiology practice of the future.

Deep learning in radiology: an overview of the concepts and a survey of the state of the art.

PURPOSE To develop a super-resolution technique using convolutional neural networks for generating thin-slice knee MR images from thicker input slices, and compare this method with alternative through-plane interpolation methods. METHODS We implemented a 3D convolutional neural network entitled DeepResolve to learn residual-based transformations between high-resolution thin-slice images and lower-resolution thick-slice images at the same center locations. DeepResolve was trained using 124 double echo in steady-state (DESS) data sets with 0.7-mm slice thickness and tested on 17 patients. Ground-truth images were compared with DeepResolve, clinically used tricubic interpolation, and Fourier interpolation methods, along with state-of-the-art single-image sparse-coding super-resolution. Comparisons were performed using structural similarity, peak SNR, and RMS error image quality metrics for a multitude of thin-slice downsampling factors. Two musculoskeletal radiologists ranked the 3 data sets and reviewed the diagnostic quality of the DeepResolve, tricubic interpolation, and ground-truth images for sharpness, contrast, artifacts, SNR, and overall diagnostic quality. Mann-Whitney U tests evaluated differences among the quantitative image metrics, reader scores, and rankings. Cohen's Kappa (κ) evaluated interreader reliability. RESULTS DeepResolve had significantly better structural similarity, peak SNR, and RMS error than tricubic interpolation, Fourier interpolation, and sparse-coding super-resolution for all downsampling factors (p < .05, except 4 × and 8 × sparse-coding super-resolution downsampling factors). In the reader study, DeepResolve significantly outperformed (p < .01) tricubic interpolation in all image quality categories and overall image ranking. Both readers had substantial scoring agreement (κ = 0.73). CONCLUSION DeepResolve was capable of resolving high-resolution thin-slice knee MRI from lower-resolution thicker slices, achieving superior quantitative and qualitative diagnostic performance to both conventionally used and state-of-the-art methods.

Super-resolution musculoskeletal MRI using deep learning.

Giant cell tumor (GCT) of bone is a generally benign, but often locally aggressive, neoplasm of bone, with a propensity for recurrence. Sarcomatous transformation is rare and typically occurs with a history of recurrences and radiation treatment. Denosumab, an inhibitor of the RANK ligand involved in bone resorption in GCT, is increasingly used in treatment of recurrent or unresectable giant cell tumor of bone. We report two cases of sarcomatous transformation of GCT to osteosarcoma in patients receiving denosumab. One was a 59-year-old male with a 12-year history of GCT and multiple recurrences taking denosumab for 2.5 years. The second case was in a 56-year-old male with a seven-year history of GCT taking denosumab for six months. Review of the literature shows one case report of malignant transformation of GCT in a patient being treated with denosumab. As the use of denosumab for treatment of GCT will likely increase, larger, controlled studies are needed to ascertain whether denosumab may play a role in malignant transformation of giant cell tumor of bone.

/pdf/two-cases-of-sarcoma-arising-in-giant-cell-tumor-of-bone-478qv4krjj.pdf

Two Cases of Sarcoma Arising in Giant Cell Tumor of Bone Treated with Denosumab

BACKGROUND Super-resolution is an emerging method for enhancing MRI resolution; however, its impact on image quality is still unknown. PURPOSE To evaluate MRI super-resolution using quantitative and qualitative metrics of cartilage morphometry, osteophyte detection, and global image blurring. STUDY TYPE Retrospective. POPULATION In all, 176 MRI studies of subjects at varying stages of osteoarthritis. FIELD STRENGTH/SEQUENCE Original-resolution 3D double-echo steady-state (DESS) and DESS with 3× thicker slices retrospectively enhanced using super-resolution and tricubic interpolation (TCI) at 3T. ASSESSMENT A quantitative comparison of femoral cartilage morphometry was performed for the original-resolution DESS, the super-resolution, and the TCI scans in 17 subjects. A reader study by three musculoskeletal radiologists assessed cartilage image quality, overall image sharpness, and osteophytes incidence in all three sets of scans. A referenceless blurring metric evaluated blurring in all three image dimensions for the three sets of scans. STATISTICAL TESTS Mann-Whitney U-tests compared Dice coefficients (DC) of segmentation accuracy for the DESS, super-resolution, and TCI images, along with the image quality readings and blurring metrics. Sensitivity, specificity, and diagnostic odds ratio (DOR) with 95% confidence intervals compared osteophyte detection for the super-resolution and TCI images, with the original-resolution as a reference. RESULTS DC for the original-resolution (90.2 ± 1.7%) and super-resolution (89.6 ± 2.0%) were significantly higher (P < 0.001) than TCI (86.3 ± 5.6%). Segmentation overlap of super-resolution with the original-resolution (DC = 97.6 ± 0.7%) was significantly higher (P < 0.0001) than TCI overlap (DC = 95.0 ± 1.1%). Cartilage image quality for sharpness and contrast levels, and the through-plane quantitative blur factor for super-resolution images, was significantly (P < 0.001) better than TCI. Super-resolution osteophyte detection sensitivity of 80% (76-82%), specificity of 93% (92-94%), and DOR of 32 (22-46) was significantly higher (P < 0.001) than TCI sensitivity of 73% (69-76%), specificity of 90% (89-91%), and DOR of 17 (13-22). DATA CONCLUSION Super-resolution appears to consistently outperform naive interpolation and may improve image quality without biasing quantitative biomarkers. LEVEL OF EVIDENCE 2 Technical Efficacy: Stage 2 J. Magn. Reson. Imaging 2020;51:768-779.

Utility of deep learning super-resolution in the context of osteoarthritis MRI biomarkers.

Background Clinical knee MRI protocols require upwards of 15 minutes of scan time. Purpose/hypothesis To compare the imaging appearance of knee abnormalities depicted with a 5-minute 3D double-echo in steady-state (DESS) sequence with separate echo images, with that of a routine clinical knee MRI protocol. A secondary goal was to compare the imaging appearance of knee abnormalities depicted with 5-minute DESS paired with a 2-minute coronal proton-density fat-saturated (PDFS) sequence. Study type Prospective. Subjects Thirty-six consecutive patients (19 male) referred for a routine knee MRI. Field strength/sequences DESS and PDFS at 3T. Assessment Five musculoskeletal radiologists evaluated all images for the presence of internal knee derangement using DESS, DESS+PDFS, and the conventional imaging protocol, and their associated diagnostic confidence of the reading. Statistical tests Differences in positive and negative percent agreement (PPA and NPA, respectively) and 95% confidence intervals (CIs) for DESS and DESS+PDFS compared with the conventional protocol were calculated and tested using exact McNemar tests. The percentage of observations where DESS or DESS+PDFS had equivalent confidence ratings to DESS+Conv were tested with exact symmetry tests. Interreader agreement was calculated using Krippendorff's alpha. Results DESS had a PPA of 90% (88-92% CI) and NPA of 99% (99-99% CI). DESS+PDFS had increased PPA of 99% (95-99% CI) and NPA of 100% (99-100% CI) compared with DESS (both P Data conclusion Both 1) 5-minute 3D-DESS with separated echoes and 2) 5-minute 3D-DESS paired with a 2-minute coronal PDFS sequence depicted knee abnormalities similarly to a routine clinical knee MRI protocol, which may be a promising technique for abbreviated knee MRI. Level of evidence 2 Technical Efficacy: Stage 2 J. Magn. Reson. Imaging 2018.

Combined 5-minute double-echo in steady-state with separated echoes and 2-minute proton-density-weighted 2D FSE sequence for comprehensive whole-joint knee MRI assessment.

Jeffrey P. Wood

Papers

Super-resolution musculoskeletal MRI using deep learning.

Two Cases of Sarcoma Arising in Giant Cell Tumor of Bone Treated with Denosumab

Utility of deep learning super-resolution in the context of osteoarthritis MRI biomarkers.

Combined 5-minute double-echo in steady-state with separated echoes and 2-minute proton-density-weighted 2D FSE sequence for comprehensive whole-joint knee MRI assessment.

Distal tibia allograft for glenohumeral instability: does radius of curvature match?