IEEE Transactions on Pattern Analysis and Machine Intelligence

High-dynamic range (HDR) imaging is an essential imaging modality for a wide range of applications in uncontrolled environments, including autonomous driving, robotics, and mobile phone cameras. However, existing HDR techniques in commodity devices struggle with dynamic scenes due to multi-shot acquisition and post-processing time, e.g. mobile phone burst photography, making such approaches unsuitable for real-time applications. In this work, we propose a method for snapshot HDR imaging by learning an optical HDR encoding in a single image which maps saturated highlights into neighboring unsaturated areas using a diffractive optical element (DOE). We propose a novel rank-1 parameterization of the proposed DOE which avoids vast trainable parameters and keeps high frequencies' encoding compared with conventional end-to-end design methods. We further propose a reconstruction network tailored to this rank-1 parametrization for recovery of clipped information from the encoded measurements. The proposed end-to-end framework is validated through simulation and real-world experiments and improves the PSNR by more than 7 dB over state-of-the-art end-to-end designs.

/pdf/learning-rank-1-diffractive-optics-for-single-shot-high-3qh5gmqz9a.pdf

Learning Rank-1 Diffractive Optics for Single-Shot High Dynamic Range Imaging

Magnetic resonance fingerprinting (MRF) is a general framework to quantify multiple MR-sensitive tissue properties with a single acquisition. There have been numerous advances in MRF in the years since its inception. In this work we highlight some of the recent technical developments in MRF, focusing on sequence optimization, modifications for reconstruction and pattern matching, new methods for partial volume analysis, and applications of machine and deep learning. Level of Evidence: 2 Technical Efficacy: Stage 2 J. Magn. Reson. Imaging 2020;51:993-1007.

/pdf/magnetic-resonance-fingerprinting-review-part-2-technique-4aobe41vl3.pdf

Magnetic resonance fingerprinting review part 2: Technique and directions.

Purpose Subjective quality assessment of displayed magnetic resonance (MR) images plays a key role in diagnosis and the resultant treatment. Therefore, this study aims to introduce a new no-reference (NR) image quality assessment (IQA) method for the objective, automatic evaluation of MR images and compare its judgments with those of similar techniques. Methods A novel NR-IQA method was developed. The method uses a sequence of scaled images filtered to enhance high-frequency components and preserve low-frequency parts. Since the human visual system (HVS) is sensitive to local image variations and local features often mimic the attraction of the HVS to high-frequency image regions, they were detected in the filtered images and described. Then, the statistics of obtained descriptors were used to build a quality model via the Support Vector Regression method. Results The method was compared with 21 state-of-the-art techniques for NR-IQA on a new dataset of 70 distorted MR images assessed by 31 experienced radiologists, using typical evaluation criteria for the comparison of NR measures. The introduced method significantly outperforms the compared approaches, in terms of the correlation with human judgments. Conclusions It is demonstrated that the presented NR-IQA method for the assessment of MR images is superior to the state-of-the-art NR techniques. The method would be beneficial for a wide range of image processing applications, assessing their outputs and affecting the directions of their development.

No-reference image quality assessment of magnetic resonance images with high-boost filtering and local features.

An investigation of diseases using magnetic resonance (MR) imaging requires automatic image quality assessment methods able to exclude low-quality scans. Such methods can be also employed for an optimization of parameters of imaging systems or evaluation of image processing algorithms. Therefore, in this paper, a novel blind image quality assessment (BIQA) method for the evaluation of MR images is introduced. It is observed that the result of filtering using non-maximum suppression (NMS) strongly depends on the perceptual quality of an input image. Hence, in the method, the image is first processed by the NMS with various levels of acceptable local intensity difference. Then, the quality is efficiently expressed by the entropy of a sequence of extrema numbers obtained with the thresholded NMS. The proposed BIQA approach is compared with ten state-of-the-art techniques on a dataset containing MR images and subjective scores provided by 31 experienced radiologists. The Pearson, Spearman, Kendall correlation coefficients and root mean square error for the method assessing images in the dataset were 0.6741, 0.3540, 0.2428, and 0.5375, respectively. The extensive experimental evaluation of the BIQA methods reveals that the introduced measure outperforms related techniques by a large margin as it correlates better with human scores.

Magnetic Resonance Image Quality Assessment by Using Non-Maximum Suppression and Entropy Analysis.

This study aimed to develop a fully automatic, no-reference image-quality assessment (IQA) method for MR images.New quality-aware features were obtained by applying multidirectional filters to MR images and examining the feature statistics. A histogram of these features was then fitted to a generalized Gaussian distribution function for which the shape parameters yielded different values depending on the type of distortion in the MR image. Standard feature statistics were established through a training process based on high-quality MR images without distortion. Subsequently, the feature statistics of a test MR image were calculated and compared with the standards. The quality score was calculated as the difference between the shape parameters of the test image and the undistorted standard images.The proposed IQA method showed a >0.99 correlation with the conventional full-reference assessment methods; accordingly, this proposed method yielded the best performance among no-reference IQA methods for images containing six types of synthetic, MR-specific distortions. In addition, for authentically distorted images, the proposed method yielded the highest correlation with subjective assessments by human observers, thus demonstrating its superior performance over other no-reference IQAs.Our proposed IQA was designed to consider MR-specific features and outperformed other no-reference IQAs designed mainly for photographic images. Magn Reson Med 80:914-924, 2018. © 2018 International Society for Magnetic Resonance in Medicine.

Quality evaluation of no-reference MR images using multidirectional filters and image statistics.

We propose a deep learning-based spike sorting method for extracellular recordings. For analysis of extracellular single unit activity, the process of detecting and classifying action potentials called “spike sorting” has become essential. This is achieved through distinguishing the morphological differences of the spikes from each neuron, which arises from the differences of the surrounding environment and characteristics of the neurons. However, cases of high structural similarity and noise make the task difficult. And for manual spike sorting, it requires professional knowledge along with extensive time cost and suffers from human bias. We propose a deep learning-based spike sorting method on extracellular recordings from a single electrode that is efficient, robust to noise, and accurate. In circumstances where labelled data does not exist, we created pseudo-labels through principal component analysis and K-means clustering to be used for multi-layer perceptron training and built high performing spike classification model. When tested, our model outperformed conventional methods by 2.1% on simulation data of various noise levels, by 6.0% on simulation data of various clusters count, and by 1.7% on in-vivo data. As a result, we showed that the deep learning-based classification can classify spikes from extracellular recordings, even showing high classification accuracy on spikes that are difficult even for manual classification.

Deep Learning-Based Template Matching Spike Classification for Extracellular Recordings

Purpose To generate short tau, or short inversion time (TI), inversion recovery (STIR) images from three multi-contrast MR images, without additional scanning, using a deep neural network. Methods For simulation studies, we used multi-contrast simulation images. For in-vivo studies, we acquired knee MR images including 288 slices of T1 -weighted (T1 -w), T2 -weighted (T2 -w), gradient-recalled echo (GRE), and STIR images taken from 12 healthy volunteers. Our MR image synthesis method generates a new contrast MR image from multi-contrast MR images. We used a deep neural network to identify the complex relationships between MR images that show various contrasts for the same tissues. Our contrast-conversion deep neural network (CC-DNN) is an end-to-end architecture that trains the model to create one image from three (T1 -w, T2 -w, and GRE images). We propose a new loss function to take into account intensity differences, misregistration, and local intensity variations. The CC-DNN-generated STIR images were evaluated with four quantitative evaluation metrics, including mean squared error, peak signal-to-noise ratio (PSNR), structural similarity (SSIM), and multi-scale SSIM (MS-SSIM). Furthermore, a subjective evaluation was performed by musculoskeletal radiologists. Results Our method showed improved results in all quantitative evaluations compared with other methods and received the highest scores in subjective evaluations by musculoskeletal radiologists. Conclusion This study suggests the feasibility of our method for generating STIR sequence images without additional scanning that offered a potential alternative to the STIR pulse sequence when additional scanning is limited or STIR artifacts are severe.

Deep-learned short tau inversion recovery imaging using multi-contrast MR images

Deep-learned spike representations and sorting via an ensemble of auto-encoders.

In modern digital photographs, most images have low dynamic range (LDR) formats, which means that the range of light intensities from the darkest to the brightest is much lower than the range that can be perceived by the human eye. Therefore, to visualize images as naturally as possible on devices that display them in high dynamic range (HDR) format, the LDR images need to be converted into HDR images. The aim of this study was to develop an adaptive inverse tone mapping operator (iTMO) that can convert a single LDR image into a realistic HDR image based on artificial neural networks. In contrast to conventional iTMO algorithms, our technique was developed by learning the complicated relationship between various LDR–HDR pair images, which enabled nearly ground-truth HDR images to be generated from various types of LDR images. The novel learning technique is called cumulative histogram-based learning and color difference learning. The superior performance of our technique over conventional methods was assessed through objective evaluations of various types of LDR and HDR images.

/pdf/dynamic-range-expansion-using-cumulative-histogram-learning-4p2hp4elfu.pdf

Hanbyol Jang

Papers

Quality evaluation of no-reference MR images using multidirectional filters and image statistics.

Deep Learning-Based Template Matching Spike Classification for Extracellular Recordings

Deep-learned short tau inversion recovery imaging using multi-contrast MR images

Deep-learned spike representations and sorting via an ensemble of auto-encoders.

Dynamic Range Expansion Using Cumulative Histogram Learning for High Dynamic Range Image Generation