: This report discusses the use of modern 2-D spectral estimation algorithms for SAR imaging, and makes two principal contributions to the field of adaptive SAR imaging. First, it is a comprehensive comparison of 2-D spectral estimation methods for SAR imaging. It provides a synopsis of the algorithms available, discusses their relative merits for SAR imaging, and illustrates their performance on simulated and collected SAR imagery. The discussion of autoregressive linear predictive techniques (ARLP), including the Tufts Kumaresan variant, is somewhat more general than appears in most of the literature, in that it allows the prediction element to be varied throughout the subaperture. This generality leads to a theoretical link between ARLP and one of Pisarenko's methods. The report also provides a theoretical analysis that predicts the impact of the adaptive sidelobe reduction (ASR) algorithm on target to clutter ratio and provides insight into order and constraint selection. Second, this work develops multi-channel variants of three related algorithms, minimum variance method (MVM), reduced rank MVM (RRMVM), and ASR to estimate both reflectivity intensity and interferometric height from polarimetric displaced-aperture interferometric data. Examples illustrate that MVM and ASR both offer significant advantages over Fourier methods for estimating both scattering intensity and interferometric height, and allow empirical comparison of the accuracies of Fourier, MVM, and- ometric height estimates.

SAR Imaging via Modern 2-D Spectral Estimation Methods. Volume 1. Imaging Methods.

RFN-Nest: An end-to-end residual fusion network for infrared and visible images

A review of multimodal image matching: Methods and applications

Image fusion meets deep learning: A survey and perspective

Visible images contain rich texture information, whereas infrared images have significant contrast. It is advantageous to combine these two kinds of information into a single image so that it not only has good contrast but also contains rich texture details. In general, previous fusion methods cannot achieve this goal well, where the fused results are inclined to either a visible or an infrared image. To address this challenge, a new fusion framework called generative adversarial network with multiclassification constraints (GANMcC) is proposed, which transforms image fusion into a multidistribution simultaneous estimation problem to fuse infrared and visible images in a more reasonable way. We adopt a generative adversarial network with multiclassification to estimate the distributions of visible light and infrared domains at the same time, in which the game of multiclassification discrimination will make the fused result to have these two distributions in a more balanced manner, so as to have significant contrast and rich texture details. In addition, we design a specific content loss to constrain the generator, which introduces the idea of main and auxiliary into the extraction of gradient and intensity information, which will enable the generator to extract more sufficient information from source images in a complementary manner. Extensive experiments demonstrate the advantages of our GANMcC over the state-of-the-art methods in terms of both qualitative effect and quantitative metric. Moreover, our method can achieve good fused results even the visible image is overexposed. Our code is publicly available at https://github.com/jiayi-ma/GANMcC .

GANMcC: A Generative Adversarial Network With Multiclassification Constraints for Infrared and Visible Image Fusion

In this paper, we propose a fast unified image fusion network based on proportional maintenance of gradient and intensity (PMGI), which can end-to-end realize a variety of image fusion tasks, including infrared and visible image fusion, multi-exposure image fusion, medical image fusion, multi-focus image fusion and pan-sharpening. We unify the image fusion problem into the texture and intensity proportional maintenance problem of the source images. On the one hand, the network is divided into gradient path and intensity path for information extraction. We perform feature reuse in the same path to avoid loss of information due to convolution. At the same time, we introduce the pathwise transfer block to exchange information between different paths, which can not only pre-fuse the gradient information and intensity information, but also enhance the information to be processed later. On the other hand, we define a uniform form of loss function based on these two kinds of information, which can adapt to different fusion tasks. Experiments on publicly available datasets demonstrate the superiority of our PMGI over the state-of-the-art in terms of both visual effect and quantitative metric in a variety of fusion tasks. In addition, our method is faster compared with the state-of-the-art.

/pdf/rethinking-the-image-fusion-a-fast-unified-image-fusion-4hg5lvqf7e.pdf

Rethinking the Image Fusion: A Fast Unified Image Fusion Network based on Proportional Maintenance of Gradient and Intensity

MFF-GAN: An unsupervised generative adversarial network with adaptive and gradient joint constraints for multi-focus image fusion

In this article, we propose an infrared and visible image fusion network based on the salient target detection, termed STDFusionNet, which can preserve the thermal targets in infrared images and the texture structures in visible images. First, a salient target mask is dedicated to annotating regions of the infrared image that humans or machines pay more attention to, so as to provide spatial guidance for the integration of different information. Second, we combine this salient target mask to design a specific loss function to guide the extraction and reconstruction of features. Specifically, the feature extraction network can selectively extract salient target features from infrared images and background texture features from visible images, while the feature reconstruction network can effectively fuse these features and reconstruct the desired results. It is worth noting that the salient target mask is only required in the training phase, which enables the proposed STDFusionNet to be an end-to-end model. In other words, our STDFusionNet can fulfill salient target detection and key information fusion in an implicit manner. Extensive qualitative and quantitative experiments demonstrate the superiority of our fusion algorithm over the current state of the arts, where our algorithm is much faster and the fusion results look like high-quality visible images with clear highlighted infrared targets. Moreover, the experimental results on the public datasets reveal that our algorithm can improve the entropy (EN), mutual information (MI), visual information fidelity (VIF), and spatial frequency (SF) metrics with about 1.25%, 22.65%, 4.3%, and 0.89% gains, respectively. Our code is publicly available at https://github.com/jiayi-ma/STDFusionNet .

Hao Zhang

Papers

Rethinking the Image Fusion: A Fast Unified Image Fusion Network based on Proportional Maintenance of Gradient and Intensity

Image fusion meets deep learning: A survey and perspective

GANMcC: A Generative Adversarial Network With Multiclassification Constraints for Infrared and Visible Image Fusion

MFF-GAN: An unsupervised generative adversarial network with adaptive and gradient joint constraints for multi-focus image fusion

STDFusionNet: An Infrared and Visible Image Fusion Network Based on Salient Target Detection