Multimodal medical image fusion, as a powerful tool for the clinical applications, has developed with the advent of various imaging modalities in medical imaging. The main motivation is to capture most relevant information from sources into a single output, which plays an important role in medical diagnosis. In this paper, a novel fusion framework is proposed for multimodal medical images based on non-subsampled contourlet transform (NSCT). The source medical images are first transformed by NSCT followed by combining low- and high-frequency components. Two different fusion rules based on phase congruency and directive contrast are proposed and used to fuse low- and high-frequency coefficients. Finally, the fused image is constructed by the inverse NSCT with all composite coefficients. Experimental results and comparative study show that the proposed fusion framework provides an effective way to enable more accurate analysis of multimodality images. Further, the applicability of the proposed framework is carried out by the three clinical examples of persons affected with Alzheimer, subacute stroke and recurrent tumor.

Directive Contrast Based Multimodal Medical Image Fusion in NSCT Domain

As an effective way to integrate the information contained in multiple medical images with different modalities, medical image fusion has emerged as a powerful technique in various clinical applications such as disease diagnosis and treatment planning. In this paper, a new multimodal medical image fusion method in nonsubsampled shearlet transform (NSST) domain is proposed. In the proposed method, the NSST decomposition is first performed on the source images to obtain their multiscale and multidirection representations. The high-frequency bands are fused by a parameter-adaptive pulse-coupled neural network (PA-PCNN) model, in which all the PCNN parameters can be adaptively estimated by the input band. The low-frequency bands are merged by a novel strategy that simultaneously addresses two crucial issues in medical image fusion, namely, energy preservation and detail extraction. Finally, the fused image is reconstructed by performing inverse NSST on the fused high-frequency and low-frequency bands. The effectiveness of the proposed method is verified by four different categories of medical image fusion problems [computed tomography (CT) and magnetic resonance (MR), MR-T1 and MR-T2, MR and positron emission tomography, and MR and single-photon emission CT] with more than 80 pairs of source images in total. Experimental results demonstrate that the proposed method can obtain more competitive performance in comparison to nine representative medical image fusion methods, leading to state-of-the-art results on both visual quality and objective assessment.

Medical Image Fusion With Parameter-Adaptive Pulse Coupled Neural Network in Nonsubsampled Shearlet Transform Domain

This study proposes a novel unified and unsupervised end-to-end image fusion network, termed as U2Fusion, which is capable of solving different fusion problems, including multi-modal, multi-exposure, and multi-focus cases. Using feature extraction and information measurement, U2Fusion automatically estimates the importance of corresponding source images and comes up with adaptive information preservation degrees. Hence, different fusion tasks are unified in the same framework. Based on the adaptive degrees, a network is trained to preserve the adaptive similarity between the fusion result and source images. Therefore, the stumbling blocks in applying deep learning for image fusion, e.g., the requirement of ground-truth and specifically designed metrics, are greatly mitigated. By avoiding the loss of previous fusion capabilities when training a single model for different tasks sequentially, we obtain a unified model that is applicable to multiple fusion tasks. Moreover, a new aligned infrared and visible image dataset, RoadScene (available at https://github.com/hanna-xu/RoadScene), is released to provide a new option for benchmark evaluation. Qualitative and quantitative experimental results on three typical image fusion tasks validate the effectiveness and universality of U2Fusion. Our code is publicly available at https://github.com/hanna-xu/U2Fusion.

U2Fusion: A Unified Unsupervised Image Fusion Network.

Multi-modality is widely used in medical imaging, because it can provide multi information about a target (tumor, organ or tissue). Segmentation using multimodality consists of fusing multi-information to improve the segmentation. Recently, deep learning-based approaches have presented the state-of-the-art performance in image classification, segmentation, object detection and tracking tasks. Due to their self-learning and generalization ability over large amounts of data, deep learning recently has also gained great interest in multi-modal medical image segmentation. In this paper, we give an overview of deep learning-based approaches for multi-modal medical image segmentation task. Firstly, we introduce the general principle of deep learning and multi-modal medical image segmentation. Secondly, we present different deep learning network architectures, then analyze their fusion strategies and compare their results. The earlier fusion is commonly used, since it's simple and it focuses on the subsequent segmentation network architecture. However, the later fusion gives more attention on fusion strategy to learn the complex relationship between different modalities. In general, compared to the earlier fusion, the later fusion can give more accurate result if the fusion method is effective enough. We also discuss some common problems in medical image segmentation. Finally, we summarize and provide some perspectives on the future research.

A review: Deep learning for medical image segmentation using multi-modality fusion

Pathological brain detection based on AlexNet and transfer learning

In this article, a novel multimodal medical image fusion (MIF) method based on non-subsampled contourlet transform (NSCT) and pulse-coupled neural network (PCNN) is presented. The proposed MIF scheme exploits the advantages of both the NSCT and the PCNN to obtain better fusion results. The source medical images are first decomposed by NSCT. The low-frequency subbands (LFSs) are fused using the ‘max selection’ rule. For fusing the high-frequency subbands (HFSs), a PCNN model is utilized. Modified spatial frequency in NSCT domain is input to motivate the PCNN, and coefficients in NSCT domain with large firing times are selected as coefficients of the fused image. Finally, inverse NSCT (INSCT) is applied to get the fused image. Subjective as well as objective analysis of the results and comparisons with state-of-the-art MIF techniques show the effectiveness of the proposed scheme in fusing multimodal medical images.

NSCT-based multimodal medical image fusion using pulse-coupled neural network and modified spatial frequency

This paper addresses a novel approach to the multimodal medical image fusion (MIF) problem, employing multiscale geometric analysis of the nonsubsampled contourlet transform and fuzzy-adaptive reduced pulse-coupled neural network (RPCNN). The linking strengths of the RPCNNs' neurons are adaptively set by modeling them as the fuzzy membership values, representing their significance in the corresponding source image. Use of the RPCNN with a less complex structure and having less number of parameters leads to computational efficiency-an important requirement of point-of-care health care technologies. The proposed scheme is free from the common shortcomings of the state-of-the-art MIF techniques: contrast reduction, loss of image fine details, and unwanted image degradations, etc. Subjective and objective evaluations show better performance of this new approach compared to the existing techniques.

https://www.isical.ac.in/~malay/Papers/IEEE%20TBME_2013.pdf

A Neuro-Fuzzy Approach for Medical Image Fusion

We propose an automatic and accurate technique for classifying normal and abnormal magnetic resonance (MR) images of human brain. Ripplet transform Type-I (RT), an e-cient multiscale geometric analysis (MGA) tool for digital images, is used to represent the salient features of the brain MR images. The dimensionality of the image representative feature vector is reduced by principal component analysis (PCA). A computationally less expensive support vector machine (SVM), called least square-SVM (LS-SVM) is used to classify the brain MR images. Extensive experiments were carried out to evaluate the performance of the proposed system. Two benchmark MR image datasets and a new larger dataset were used in the experiments, consisting 66, 160 and 255 images, respectively. The generalization capability of the proposed technique is enhanced by 5 £ 5 cross validation procedure. For all the datasets used in the experiments, the proposed system shows high classiflcation accuracies (on an average > 99%). Experimental results and performance comparisons with state-of-the-art techniques, show that the proposed scheme is e-cient in brain MR image classiflcation.

/pdf/brain-mr-image-classification-using-multiscale-geometric-wcsxtkabcu.pdf

Brain Mr Image Classification Using Multiscale Geometric Analysis of Ripplet

http://www.isical.ac.in/~malay/Papers/CMPB-PP-2013.pdf

Effective management of medical information through ROI-lossless fragile image watermarking technique

In this article, a new fragile, blind, high payload capacity, ROI (Region of Interest) preserving Medical image watermarking (MIW) technique in the spatial domain for gray scale medical images is proposed. We present a watermarking scheme that combines lossless data compression and encryption technique in application to medical images. The effectiveness of the proposed scheme, proven through experiments on various medical images through various image quality measure matrices such as PSNR, MSE and MSSIM enables us to argue that, the method will help to maintain Electronic Patient Report(EPR)/DICOM data privacy and medical image integrity.

http://www.isical.ac.in/~malay/Papers/Conf/icpr_10.pdf

Sudeb Das

Papers

NSCT-based multimodal medical image fusion using pulse-coupled neural network and modified spatial frequency

A Neuro-Fuzzy Approach for Medical Image Fusion

Brain Mr Image Classification Using Multiscale Geometric Analysis of Ripplet

Effective management of medical information through ROI-lossless fragile image watermarking technique

Lossless ROI Medical Image Watermarking Technique with Enhanced Security and High Payload Embedding