Pixel level fusion techniques for SAR and optical images: A review

The visual similarity of oil slick and other elements, known as look-alike, affects the reliability of synthetic aperture radar (SAR) images for marine oil spill detection. So far, detection and discrimination of oil spill and look-alike are still limited to the use of traditional machine learning algorithms and semantic segmentation deep learning models with limited accuracy. Thus, this study developed a novel deep learning oil spill detection model using computer vision instance segmentation Mask-Region-based Convolutional Neural Network (Mask R-CNN) model. The model training was conducted using transfer learning on the ResNet 101 on COCO as backbone in combination with Feature Pyramid Network (FPN) architecture for feature extraction at 30 epochs with 0.001 learning rate. Testing of the model was conducted using the least training and validation loss value on the withheld testing images. The model’s performance was evaluated using precision, recall, specificity, IoU, F1-measure and overall accuracy values. Ship detection and segmentation had the highest performance with overall accuracy of 98.3%. The model equally showed a higher accuracy for oil spill and look-alike detection and segmentation although oil spill detection outperformed look-alike with overall accuracy values of 96.6% and 91.0% respectively. The study concluded that the deep learning instance segmentation model performs better than conventional machine learning models and deep learning semantic segmentation models in detection and segmentation.

A novel deep learning instance segmentation model for automated marine oil spill detection

Remote sensing of temperate and boreal forest phenology: A review of progress, challenges and opportunities in the intercomparison of in-situ and satellite phenological metrics

With the extremely rapid advances in remote sensing (RS) technology, a great quantity of Earth observation (EO) data featuring considerable and complicated heterogeneity is readily available nowadays, which renders researchers an opportunity to tackle current geoscience applications in a fresh way. With the joint utilization of EO data, much research on multimodal RS data fusion has made tremendous progress in recent years, yet these developed traditional algorithms inevitably meet the performance bottleneck due to the lack of the ability to comprehensively analyse and interpret these strongly heterogeneous data. Hence, this non-negligible limitation further arouses an intense demand for an alternative tool with powerful processing competence. Deep learning (DL), as a cutting-edge technology, has witnessed remarkable breakthroughs in numerous computer vision tasks owing to its impressive ability in data representation and reconstruction. Naturally, it has been successfully applied to the ﬁeld of multimodal RS data fusion, yielding great improvement compared with traditional methods. This survey aims to present a systematic overview in DL-based multimodal RS data fusion. More speciﬁcally, some essential knowledge about this topic is ﬁrst given. Subsequently, a literature survey is conducted to analyse the trends of this ﬁeld. Some prevalent sub-ﬁelds in the multimodal RS data fusion are then reviewed in terms of the to-be-fused data modalities, i.e., spatiospectral, spatiotemporal, light detection and ranging-optical, synthetic aperture radar-optical, and RS-Geospatial Big Data fusion. Furthermore, We collect and summarize some valuable resources for the sake of the development in multimodal RS data fusion. Finally, the remaining challenges and potential future directions are highlighted.

Deep Learning in Multimodal Remote Sensing Data Fusion: A Comprehensive Review

Due to severe speckle noise in synthetic aperture radar (SAR) images and the large nonlinear intensity differences between SAR and optical images, the registration of SAR and optical images is a challenging problem that remains to be solved. In this paper, an improved nonlinear scale-invariant feature transform (SIFT)-framework-based algorithm that combines spatial feature detection with local frequency-domain description for the registration of SAR and optical images is proposed. First, multiscale representations of the SAR and optical images are constructed based on nonlinear diffusion to better preserve edges and obtain consistent edge information. The ratio of exponentially weighted averages (ROEWA) operator and the Sobel operator are utilized in the process of scale space construction to calculate consistent gradient information. Then, a new feature detection strategy based on the Harris–Laplace ROEWA and Harris–Laplace Sobel techniques is proposed to detect stable and repeatable keypoints in the scale space. Finally, a novel descriptor, called the rotation-invariant amplitudes of log-Gabor orientation histograms (RI-ALGH), and a simplified version, ALGH, are proposed. The proposed descriptors are built based on the amplitudes of multiscale and multiorientation log-Gabor responses and utilize an improved spatial structure of the gradient location and orientation histogram (GLOH) descriptor, which is robust to local distortions. The experimental results on both simulated and real images demonstrate that the proposed method can achieve better results than other state-of-the-art methods in terms of registration accuracy.

Universal SAR and optical image registration via a novel SIFT framework based on nonlinear diffusion and a polar spatial-frequency descriptor

Synthetic Aperture Radar (SAR) imagery is available all the time and, in all weathers, due to its ability to penetrate through cloud cover, dust, haze except heavy rainfall. SAR imagery mainly characterizes structural properties of various earth objects and provides a data which is rich in spatial information. Processing of SAR data has been of great interest due to these advantages. However, due to coherent processing of SAR signal, imagery is corrupted by speckle noise which is multiplicative in nature. Reduction of speckle noise from SAR imagery is helpful for analysis and interpretation of SAR images. In this paper, performance of conventional speckle reduction filters, namely Lee, Enhanced Lee, Frost, Enhanced Frost and Gamma MAP are compared with wavelet-based filtering technique for RISAT-1 imagery. Results are compared visually and using quality metrics, Speckle Suppression Index (SSI), Equivalent Number of Looks (ENL) and Speckle Suppression and Mean Preservation Index (SMPI). From visual interpretation of de-speckled image, it is evident that wavelet-based filtering out performs the conventional filters in terms of edge and texture preservation.

Comparison of Different Speckle Noise Reduction Filters for RISAT ‐1 SAR Imagery

Multisensor image fusion has gained tremendous significance due to various satellites operating in different parts of the electromagnetic spectrum. We present a hybrid fusion approach to integrate information from synthetic aperture radar (SAR) and multispectral (MS) imagery to improve land use land cover (LULC) classification. The major concern in SAR and optical fusion is the spectral distortion in the fused image, which is significantly less in pansharpening algorithms. The primary objective of our work is to inject unique spatial information from the SAR image into MS images, deriving enhanced data. The proposed approach is based on the integration of principal component analysis and wavelet decomposition to reduce spectral distortion in the fused image. Fused images are evaluated visually and statistically. Results are compared with conventional fusion approaches. In order to explore the effectiveness of the proposed technique, LULC classification is performed on the fused and original data. The LULC classification results are analytically compared with the standard thematic map to derive classification accuracy. A comparative analysis with other approaches conclusively proves that the proposed hybrid approach is superior to conventional approaches.

Hybrid fusion approach for synthetic aperture radar and multispectral imagery for improvement in land use land cover classification

This paper presents a pixel level wavelet-based approach to fuse synthetic aperture radar (SAR) imagery with multispectral (MS) imagery. Image fusion combines information from two or more images to generate a new image, which is rich in information. Due to complementary nature of SAR and multispectral imagery, fusion of these images is of significant interest in the field of remote sensing. The primary objective of this work is to enhance spatial information in multispectral images by injecting structural information derived from SAR image. Due to negative correlation between SAR and multispectral data, conventional component substitution methods face the problem of spectral distortion in the fused image. Wavelet based fusion approaches overcome this problem due to excellent localization in spatial and frequency domain. Here, different wavelet-based fusion rules are applied for fusion of SAR and multispectral images. Fusion rules applied to fuse approximate sub-bands and detail sub-bands of these images consider spectral dis-similarity between them. Results are evaluated visually, as well as using standard quality metrics and are compared with component substitution fusion techniques namely, principal component analysis and generalized IHS transform. Trade-off between spectral and spatial quality of fused image has been observed while fusing SAR and multispectral images.

Fusion of RISAT-1 SAR Image and Resourcesat-2 Multispectral Images Using Wavelet Transform

The fusion of multispectral and synthetic aperture radar (SAR) images is of vital importance in many remote sensing applications. Spectral distortion and trade-off between the spatial and spectral quality of the fused image are significant issues in SAR-multispectral image fusion. Our study attempts to improve the performance of SAR-multispectral image fusion concerning these two issues. The primary objective of our study is to optimize the performance of hybrid fusion approach based on principal component analysis and discrete wavelet transform (PCA-DWT) using Taguchi orthogonal array. The fused data are evaluated using visual analysis and standard quality metrics. The results are compared with recent hybrid fusion approaches applied to the SAR-multispectral image fusion. The utility of the fused data is evaluated based on the remote sensing application, namely, land use land cover classification. The classification results are compared to a standard thematic map available on the Bhuvan geoportal to check classification accuracy. A comparative analysis with recent hybrid approaches conclusively demonstrates that the proposed optimization in the PCA-DWT based fusion is superior to conventional hybrid methods.

Samadhan C. Kulkarni

Papers

Pixel level fusion techniques for SAR and optical images: A review

Comparison of Different Speckle Noise Reduction Filters for RISAT ‐1 SAR Imagery

Hybrid fusion approach for synthetic aperture radar and multispectral imagery for improvement in land use land cover classification

Fusion of RISAT-1 SAR Image and Resourcesat-2 Multispectral Images Using Wavelet Transform

Application of Taguchi method to improve land use land cover classification using PCA-DWT-based SAR-multispectral image fusion