Medical image fusion is the process of registering and combining multiple images from single or multiple imaging modalities to improve the imaging quality and reduce randomness and redundancy in order to increase the clinical applicability of medical images for diagnosis and assessment of medical problems. Multi-modal medical image fusion algorithms and devices have shown notable achievements in improving clinical accuracy of decisions based on medical images. This review article provides a factual listing of methods and summarizes the broad scientific challenges faced in the field of medical image fusion. We characterize the medical image fusion research based on (1) the widely used image fusion methods, (2) imaging modalities, and (3) imaging of organs that are under study. This review concludes that even though there exists several open ended technological and scientific challenges, the fusion of medical images has proved to be useful for advancing the clinical reliability of using medical imaging for medical diagnostics and analysis, and is a scientific discipline that has the potential to significantly grow in the coming years.

Medical Image Fusion: A survey of the state of the art

Medical image fusion: A survey of the state of the art

In this paper, we proposed a new end-to-end model, termed as dual-discriminator conditional generative adversarial network (DDcGAN), for fusing infrared and visible images of different resolutions. Our method establishes an adversarial game between a generator and two discriminators. The generator aims to generate a real-like fused image based on a specifically designed content loss to fool the two discriminators, while the two discriminators aim to distinguish the structure differences between the fused image and two source images, respectively, in addition to the content loss. Consequently, the fused image is forced to simultaneously keep the thermal radiation in the infrared image and the texture details in the visible image. Moreover, to fuse source images of different resolutions, e.g. , a low-resolution infrared image and a high-resolution visible image, our DDcGAN constrains the downsampled fused image to have similar property with the infrared image. This can avoid causing thermal radiation information blurring or visible texture detail loss, which typically happens in traditional methods. In addition, we also apply our DDcGAN to fusing multi-modality medical images of different resolutions, e.g. , a low-resolution positron emission tomography image and a high-resolution magnetic resonance image. The qualitative and quantitative experiments on publicly available datasets demonstrate the superiority of our DDcGAN over the state-of-the-art, in terms of both visual effect and quantitative metrics. Our code is publicly available at https://github.com/jiayi-ma/DDcGAN .

DDcGAN: A Dual-Discriminator Conditional Generative Adversarial Network for Multi-Resolution Image Fusion

An overview of multi-modal medical image fusion

Detail preserved fusion of visible and infrared images using regional saliency extraction and multi-scale image decomposition

Multimodal medical image fusion based on IHS and PCA

Currently, an increasing number of convolutional neural networks (CNNs) focus specifically on capturing contextual features (con. feat) to improve performance in semantic segmentation tasks. However, high-level con. feat are biased towards encoding features of large objects, disregard spatial details, and have a limited capacity to discriminate between easily confused classes (e.g., trees and grasses). As a result, we incorporate low-level features (low. feat) and class-specific discriminative features (dis. feat) to boost model performance further, with low. feat helping the model in recovering spatial information and dis. feat effectively reducing class confusion during segmentation. To this end, we propose a novel deep multi-feature learning framework for the semantic segmentation of VHR RSIs, dubbed MFNet. The proposed MFNet adopts a multi-feature learning mechanism to learn more complete features, including con. feat, low. feat, and dis. feat. More specifically, aside from a widely used context aggregation module for capturing con. feat, we additionally append two branches for learning low. feat and dis. feat. One focuses on learning low. feat at a shallow layer in the backbone network through local contrast processing, while the other groups con. feat and then optimizes each class individually to generate dis. feat with better inter-class discriminative capability. Extensive quantitative and qualitative evaluations demonstrate that the proposed MFNet outperforms most state-of-the-art models on the ISPRS Vaihingen and Potsdam datasets. In particular, thanks to the mechanism of multi-feature learning, our model achieves an overall accuracy score of 91.91% on the Potsdam test set with VGG16 as a backbone, performing favorably against advanced models with ResNet101.

/pdf/semantic-segmentation-of-very-high-resolution-remote-sensing-1je4ioet.pdf

Semantic Segmentation of Very-High-Resolution Remote Sensing Images via Deep Multi-Feature Learning

This article is concerned with exponential stabilization almost surely for a class of discrete-time switched T–S fuzzy systems (DTSTSFSs), in which each mode switches according to transition probability and evolves with mode-dependent average dwell time. Parallel distributed compensation (PDC) event-triggered dynamic output quantization (ETDOQ) control is considered for the DTSTSFSs, where event-triggered (ET) mechanism without Zeno behavior is designed, which is mode-dependent and does not restrict the dwell time of each mode. By using the mode-dependent Lyapunov function method, two sufficient conditions termed by linear matrix inequalities are provided. Different from most of existing results, no inequality is resorted in dealing with the uncertainties coming from the quantization. It is interesting that the controlled modes are not required to be stable. New algorithm is designed for the control gains of the ETDOQ controller and the parameters of the ET condition, which overcomes the mismatching difficulty of the parameters induced by the PDC. Numerical simulations demonstrate the merits of the theoretical analysis.

Event-Triggered Dynamic Output Quantization Control of Switched T–S Fuzzy Systems With Unstable Modes

It is challenging to deal with the relationship between the triggering interval and the dwell time (DT) when mode-pendent event-triggered control is considered for switched systems, especially complete actuator faults occur in some subsystems (or modes). Therefore, this paper considers the exponential stabilization almost surely (ES a.s.) for a class of T-S fuzzy systems with delay and stochastic switching by designing a novel mode-dependent event-triggered output control. It is assumed that only the measured output can be utilized in designing the controller and complete actuator faults occur in some modes. Periodic sampling is utilized to avoid the Zeno phenomenon. Combining the sampling interval with delay-partitioning method, a fuzzy Lyapunov-Krasovskii functional (LKF) is designed on the basis of the derivative of the membership functions to obtain less conservative results. Especially, the increment coefficient of the multiple LKF at switching instant is smaller than one. The maximum feasible value of the sampling interval is derived and the control gains are designed by solving linear matrix inequalities (LMIs). Our results show that the ES a.s. is ensured though some modes are not controlled. Numerical simulations demonstrate the merits of the theoretical analysis. It is significant to find by numerical simulations that the time delay cannot be divided into too small values though delay-partitioning method is effective in reducing the conservativeness. 

Mode-Dependent Event-Triggered Output Control for Switched T-S Fuzzy Systems With Stochastic Switching

A new strategy was established for the degradation of wastewater-based organic pollutants. Laser-induced plasma (LIP) was used as an alternative UV light source to realise rapid photodegradation of methylene blue (MB), an organic pollutant. A conventional 1064 nm Nd:YAG laser was used for plasma excitation to degrade MB solutions. The results show that the LIP effectively degraded the organic matter, and the degradation efficiency was related to the UV component with wavelength less than 400 nm. The compositions of the plasma excited by different dielectric substrates are different owing to various mechanisms involving moderate heat dissipation and sonoluminescence. However, metallic substrates, especially Fe, can enhance the proportion of UV light and accelerate the degradation efficiency. In the process of methylene blue degradation, solution parameters, such as initial dye concentration, pH, irradiation time and hydrogen peroxide concentration, will affect the degradation efficiency. The conditions of effective degradation of methylene blue (10 mg L−1 MB−1 concentration, 50 mL L−1 H2O2 concentration, pH = 3 and P = 60 mW) were obtained in this study, which can provide reference for practical application.

https://pubs.rsc.org/en/content/articlepdf/2022/ra/d2ra03633a

Changtao He

Papers

Multimodal medical image fusion based on IHS and PCA

Semantic Segmentation of Very-High-Resolution Remote Sensing Images via Deep Multi-Feature Learning

Event-Triggered Dynamic Output Quantization Control of Switched T–S Fuzzy Systems With Unstable Modes

Mode-Dependent Event-Triggered Output Control for Switched T-S Fuzzy Systems With Stochastic Switching

Rapid photodegradation of methylene blue by laser-induced plasma