How to handle spatial correlations in SAR despeckling? Resampling strategies and deep learning approaches
29 Mar 2021-
TL;DR: A standard training strategy for deep learning of speckle correlations is proposed and the increased robustness brought by including a Total Variation term in the loss function is analyzed on Sentinel-1 images.
Abstract: Speckle noise strongly affects Synthetic Aperture Radar (SAR) images, causing strong intensity fluctuations that make them difficult to analyze. Although many speckle reduction algorithms have been proposed, how to effectively deal with the spatial correlations of speckle remains an open question, especially in the most recent deep learning approaches. This paper tries to address this problem. Existing approaches to tackle the speckle correlations are described. Then, a standard training strategy for deep learning is proposed. Two models are trained and the increased robustness brought by including a Total Variation (TV) term in the loss function is analyzed on Sentinel-1 images.
Citations
More filters
TL;DR: A deep learning algorithm with semi-supervision is proposed in this article: SAR2SAR, where Multitemporal time series are leveraged and the neural network learns to restore SAR images by only looking at noisy acquisitions.
Abstract: Speckle reduction is a key step in many remote sensing applications By strongly affecting synthetic aperture radar (SAR) images, it makes them difficult to analyze Due to the difficulty to model the spatial correlation of speckle, a deep learning algorithm with semi-supervision is proposed in this article: SAR2SAR Multitemporal time series are leveraged and the neural network learns to restore SAR images by only looking at noisy acquisitions To this purpose, the recently proposed noise2noise framework [1] has been employed The strategy to adapt it to SAR despeckling is presented, based on a compensation of temporal changes and a loss function adapted to the statistics of speckle A study with synthetic speckle noise is presented to compare the performances of the proposed method with other state-of-the-art filters Then, results on real images are discussed, to show the potential of the proposed algorithm The code is made available to allow testing and reproducible research in this field
57 citations
TL;DR: In this article, a CNN model is applied to remove additive white Gaussian noise from natural images, and a hybrid approach is also analyzed: the CNN is trained on speckle-free SAR images, which are used to evaluate the quality of denoising and discuss the pros and cons of the different strategies.
Abstract: Speckle reduction is a longstanding topic in synthetic aperture radar (SAR) images. Many different schemes have been proposed for the restoration of intensity SAR images. Among the different possible approaches, methods based on convolutional neural networks (CNNs) have recently shown to reach state-of-the-art performance for SAR image restoration. CNN training requires good training data: many pairs of speckle-free/speckle-corrupted images. This is an issue in SAR applications, given the inherent scarcity of speckle-free images. To handle this problem, this paper analyzes different strategies one can adopt, depending on the speckle removal task one wishes to perform and the availability of multitemporal stacks of SAR data. The first strategy applies a CNN model, trained to remove additive white Gaussian noise from natural images, to a recently proposed SAR speckle removal framework: MuLoG (MUlti-channel LOgarithm with Gaussian denoising). No training on SAR images is performed, the network is readily applied to speckle reduction tasks. The second strategy considers a novel approach to construct a reliable dataset of speckle-free SAR images necessary to train a CNN model. Finally, a hybrid approach is also analyzed: the CNN used to remove additive white Gaussian noise is trained on speckle-free SAR images. The proposed methods are compared to other state-of-the-art speckle removal filters, to evaluate the quality of denoising and to discuss the pros and cons of the different strategies. Along with the paper, we make available the weights of the trained network to allow its usage by other researchers.
33 citations
TL;DR: In this paper, a self-supervised strategy called MERLIN (coMplex sElf-supeRvised despeckLINg) is proposed to separate real and imaginary parts of single-look complex SAR images.
Abstract: Speckle fluctuations seriously limit the interpretability of synthetic
aperture radar (SAR) images. Speckle reduction has thus been the subject of
numerous works spanning at least four decades. Techniques based on deep neural
networks have recently achieved a new level of performance in terms of SAR
image restoration quality. Beyond the design of suitable network architectures
or the selection of adequate loss functions, the construction of training sets
is of uttermost importance. So far, most approaches have considered a
supervised training strategy: the networks are trained to produce outputs as
close as possible to speckle-free reference images. Speckle-free images are
generally not available, which requires resorting to natural or optical images
or the selection of stable areas in long time series to circumvent the lack of
ground truth. Self-supervision, on the other hand, avoids the use of
speckle-free images. We introduce a self-supervised strategy based on the
separation of the real and imaginary parts of single-look complex SAR images,
called MERLIN (coMplex sElf-supeRvised despeckLINg), and show that it offers a
straightforward way to train all kinds of deep despeckling networks. Networks
trained with MERLIN take into account the spatial correlations due to the SAR
transfer function specific to a given sensor and imaging mode. By requiring
only a single image, and possibly exploiting large archives, MERLIN opens the
door to hassle-free as well as large-scale training of despeckling networks.
The code of the trained models is made freely available at
https://gitlab.telecom-paris.fr/RING/MERLIN.
15 citations
11 Jul 2021
TL;DR: Deep learning for speckle reduction is a very active research topic and already shows restoration performances that exceed that of the previous generations of methods based on the concepts of patches, sparsity, wavelet transform or total variation minimization as discussed by the authors.
Abstract: The speckle phenomenon remains a major hurdle for the analysis of SAR images. The development of speckle reduction methods closely follows methodological progress in the field of image restoration. The advent of deep neural networks has offered new ways to tackle this longstanding problem. Deep learning for speckle reduction is a very active research topic and already shows restoration performances that exceed that of the previous generations of methods based on the concepts of patches, sparsity, wavelet transform or total variation minimization. The objective of this paper is to give an overview of the most recent works and point the main research directions and current challenges of deep learning for SAR image restoration.
10 citations
11 Jul 2021
TL;DR: In this paper, a multi-temporal average and the image at a given date in the form of a ratio image are combined to remove the speckle in this ratio image.
Abstract: Deep learning approaches show unprecedented results for speckle reduction in SAR amplitude images. The wide availability of multi-temporal stacks of SAR images can improve even further the quality of denoising. In this paper, we propose a flexible yet efficient way to integrate temporal information into a deep neural network for speckle suppression. Archives provide access to long time-series of SAR images, from which multi-temporal averages can be computed with virtually no remaining speckle fluctuations. The proposed method combines this multi-temporal average and the image at a given date in the form of a ratio image and uses a state-of-the-art neural network to remove the speckle in this ratio image. This simple strategy is shown to offer a noticeable improvement compared to filtering the original image without knowledge of the multi-temporal average.
6 citations
References
More filters
TL;DR: In this paper, the distribution of scale sizes in a speckle pattern (i.e., the Wiener spectrum) is investigated from a physical point of view, and it is shown that adding M uncorrelated speckles on an intensity basis can reduce the contrast by 1/√M.
Abstract: A speckle pattern formed in polarized monochromatic light may be regarded as resulting from a classical random walk in the complex plane. The resulting irradiance fluctuations obey negative exponential statistics, with ratio of standard deviation to mean (i.e., contrast) of unity. Reduction of this contrast, or smoothing of the speckle, requires diversity in polarization, space, frequency, or time. Addition of M uncorrelated speckle patterns on an intensity basis can reduce the contrast by 1/√M. However, addition of speckle patterns on a complex amplitude basis provides no reduction of contrast. The distribution of scale sizes in a speckle pattern (i.e., the Wiener spectrum) is investigated from a physical point of view.
2,093 citations
"How to handle spatial correlations ..." refers background in this paper
...[22] developed the fully-developed speckle model where the measured intensity I is related to the underlying reflectivity R and the speckle S by the multiplicative model...
[...]
TL;DR: A model for the radar imaging process is derived and a method for smoothing noisy radar images is presented and it is shown that the filter can be easily implemented in the spatial domain and is computationally efficient.
Abstract: Standard image processing techniques which are used to enhance noncoherent optically produced images are not applicable to radar images due to the coherent nature of the radar imaging process. A model for the radar imaging process is derived in this paper and a method for smoothing noisy radar images is also presented. The imaging model shows that the radar image is corrupted by multiplicative noise. The model leads to the functional form of an optimum (minimum MSE) filter for smoothing radar images. By using locally estimated parameter values the filter is made adaptive so that it provides minimum MSE estimates inside homogeneous areas of an image while preserving the edge structure. It is shown that the filter can be easily implemented in the spatial domain and is computationally efficient. The performance of the adaptive filter is compared (qualitatively and quantitatively) with several standard filters using real and simulated radar images.
1,906 citations
TL;DR: The adaptive noise smoothing filter is a systematic derivation of Lee's algorithm with some extensions that allow different estimators for the local image variance and its easy extension to deal with various types of signal-dependent noise.
Abstract: In this paper, we consider the restoration of images with signal-dependent noise. The filter is noise smoothing and adapts to local changes in image statistics based on a nonstationary mean, nonstationary variance (NMNV) image model. For images degraded by a class of uncorrelated, signal-dependent noise without blur, the adaptive noise smoothing filter becomes a point processor and is similar to Lee's local statistics algorithm [16]. The filter is able to adapt itself to the nonstationary local image statistics in the presence of different types of signal-dependent noise. For multiplicative noise, the adaptive noise smoothing filter is a systematic derivation of Lee's algorithm with some extensions that allow different estimators for the local image variance. The advantage of the derivation is its easy extension to deal with various types of signal-dependent noise. Film-grain and Poisson signal-dependent restoration problems are also considered as examples. All the nonstationary image statistical parameters needed for the filter can be estimated from the noisy image and no a priori information about the original image is required.
1,475 citations
"How to handle spatial correlations ..." refers background in this paper
...[6]) within a fixed window, or (iii) in the spectral domain (by averaging intensity images obtained from sub-aperture synthesis)....
[...]
TL;DR: This paper develops a statistical technique to define a noise model, and then successfully applies a local statistics noise filtering algorithm to a set of actual SEASAT SAR images, resulting in smoothed images that permit observers to resolve fine detail with an enhanced edge effect.
880 citations
"How to handle spatial correlations ..." refers methods in this paper
...Multilooking can be applied either (i) in the temporal domain, when several images are acquired over an area where no significant changes have occurred, (ii) in the spatial domain, by averaging pixels (or linearly combining them [1] [2] [3] [4] [5]...
[...]
TL;DR: A new approach in polarimetric synthetic aperture radar (SAR) speckle filtering is proposed that uses edge-aligned nonsquare windows and applies the local statistics filter and is quite dramatic in boosting classification performance.
Abstract: This paper proposes a new approach in polarimetric synthetic aperture radar (SAR) speckle filtering. The new approach emphasizes preserving polarimetric properties and statistical correlation between channels, not introducing crosstalk, and not degrading the image quality. In the last decade, speckle reduction of polarimetric SAR imagery has been studied using several different approaches. All of these approaches exploited the degree of statistical independence between linear polarization channels. The preservation of polarimetric properties and statistical characteristics such as correlation between channels were not carefully addressed. To avoid crosstalk, each element of the covariance matrix must be filtered independently. This rules out current methods of polarimetric SAR filtering. To preserve the polarimetric signature, each element of the covariance matrix should be filtered in a way similar to multilook processing by averaging the covariance matrix of neighboring pixels. However, this must be done without the deficiency of smearing the edges, which degrades image quality and corrupts polarimetric properties. The proposed polarimetric SAR filter uses edge-aligned nonsquare windows and applies the local statistics filter. The impact of using this polarimetric speckle filtering on terrain classification is quite dramatic in boosting classification performance. Airborne polarimetric radar images are used for illustration.
785 citations