Simultaneously Sparse and Low-Rank Abundance Matrix Estimation for Hyperspectral Image Unmixing
TL;DR: Two novel unmixing algorithms are introduced in an attempt to exploit both spatial correlation and sparse representation of pixels lying in the homogeneous regions of hyperspectral images and are illustrated in experiments conducted both on simulated and real data.
Abstract: In a plethora of applications dealing with inverse problems, e.g., image processing, social networks, compressive sensing, and biological data processing, the signal of interest is known to be structured in several ways at the same time. This premise has recently guided research into the innovative and meaningful idea of imposing multiple constraints on the unknown parameters involved in the problem under study. For instance, when dealing with problems whose unknown parameters form sparse and low-rank matrices, the adoption of suitably combined constraints imposing sparsity and low rankness is expected to yield substantially enhanced estimation results. In this paper, we address the spectral unmixing problem in hyperspectral images. Specifically, two novel unmixing algorithms are introduced in an attempt to exploit both spatial correlation and sparse representation of pixels lying in the homogeneous regions of hyperspectral images. To this end, a novel mixed penalty term is first defined consisting of the sum of the weighted $\ell_{1}$ and the weighted nuclear norm of the abundance matrix corresponding to a small area of the image determined by a sliding square window. This penalty term is then used to regularize a conventional quadratic cost function and impose simultaneous sparsity and low rankness on the abundance matrix. The resulting regularized cost function is minimized by: 1) an incremental proximal sparse and low-rank unmixing algorithm; and 2) an algorithm based on the alternating direction method of multipliers . The effectiveness of the proposed algorithms is illustrated in experiments conducted both on simulated and real data.
Citations
More filters
TL;DR: An overview of nonconvex regularization based sparse and low-rank recovery in various fields in signal processing, statistics, and machine learning, including compressive sensing, sparse regression and variable selection, sparse signals separation, sparse principal component analysis (PCA), large covariance and inverse covariance matrices estimation, matrix completion, and robust PCA is given.
Abstract: In the past decade, sparse and low-rank recovery has drawn much attention in many areas such as signal/image processing, statistics, bioinformatics, and machine learning. To achieve sparsity and/or low-rankness inducing, the $\ell _{1}$ norm and nuclear norm are of the most popular regularization penalties due to their convexity. While the $\ell _{1}$ and nuclear norm are convenient as the related convex optimization problems are usually tractable, it has been shown in many applications that a nonconvex penalty can yield significantly better performance. In recent, nonconvex regularization-based sparse and low-rank recovery is of considerable interest and it in fact is a main driver of the recent progress in nonconvex and nonsmooth optimization. This paper gives an overview of this topic in various fields in signal processing, statistics, and machine learning, including compressive sensing, sparse regression and variable selection, sparse signals separation, sparse principal component analysis (PCA), large covariance and inverse covariance matrices estimation, matrix completion, and robust PCA. We present recent developments of nonconvex regularization based sparse and low-rank recovery in these fields, addressing the issues of penalty selection, applications and the convergence of nonconvex algorithms. Code is available at https://github.com/FWen/ncreg.git .
132 citations
Cites background from "Simultaneously Sparse and Low-Rank ..."
...observation, the works [220]–[223], [231] consider the re-...
[...]
TL;DR: This work proposes a subspace-based unmixing model using low-rank learning strategy, called SULoRA, robustly against spectral variability in inverse problems of hyperspectral un Mixing, and adopts an alternating direction method of multipliers based algorithm to solve the resulting optimization problem.
Abstract: To support high-level analysis of spaceborne imaging spectroscopy (hyperspectral) imagery, spectral unmixing has been gaining significance in recent years. However, from the inevitable spectral variability, caused by illumination and topography change, atmospheric effects and so on make it difficult to accurately estimate abundance maps in spectral unmixing. Classical unmixing methods, e.g., linear mixing model (LMM) and extended LMM, fail to robustly handle this issue, particularly facing complex spectral variability. To this end, we propose a subspace-based unmixing model using low-rank learning strategy, called subspace unmixing with low-rank attribute embedding (SULoRA), robustly against spectral variability in inverse problems of hyperspectral unmixing. Unlike those previous approaches that unmix the spectral signatures directly in original space, SULoRA is a general subspace unmixing framework that jointly estimates subspace projections and abundance maps in order to find a raw subspace that is more suitable for carrying out the unmixing procedure. More importantly, we model such raw subspace with low-rank attribute embedding. By projecting the original data into a low-rank subspace, SULoRA can effectively address various spectral variabilities in spectral unmixing. Furthermore, we adopt an alternating direction method of multipliers based algorithm to solve the resulting optimization problem. Extensive experiments on synthetic and real datasets are performed to demonstrate the superiority and effectiveness of the proposed method in comparison with the previous state-of-the-art methods.
90 citations
Cites methods from "Simultaneously Sparse and Low-Rank ..."
...In this section, we quantitatively and visually evaluate the unmixing performance of the proposed SULoRA on a synthetic dataset presented in [14] and two real datasets over the areas of Urban and MUFFLE Gulfport Campus, in comparison with eight classical and state-of-the-art methods, including FCLSU, PCLSU, SPCLSU, SUnSAL, SSUnSAL (scaled SUnSAL), SLRU (sparse and low-rank unmixing) [38], PLMM and ELMM....
[...]
TL;DR: This paper proposes a joint-sparsity-blocks model for abundance estimation problem and develops a two-level reweighting strategy to enhance the sparsity along the rows within each block for joint-sparse-blocks regression problem.
Abstract: Hyperspectral unmixing has attracted much attention in recent years. Single sparse unmixing assumes that a pixel in a hyperspectral image consists of a relatively small number of spectral signatures from large, ever-growing, and available spectral libraries. Joint-sparsity (or row-sparsity) model typically enforces all pixels in a neighborhood to share the same set of spectral signatures. The two sparse models are widely used in the literature. In this paper, we propose a joint-sparsity-blocks model for abundance estimation problem. Namely, the abundance matrix of size $m\times n$ is partitioned to have one row block and $s$ column blocks and each column block itself is joint-sparse. It generalizes both the single (i.e., $s=n$ ) and the joint (i.e., $s=1$ ) sparsities. Moreover, concatenating the proposed joint-sparsity-blocks structure and low rankness assumption on the abundance coefficients, we develop a new algorithm called joint-sparse-blocks and low-rank unmixing . In particular, for the joint-sparse-blocks regression problem, we develop a two-level reweighting strategy to enhance the sparsity along the rows within each block. Simulated and real-data experiments demonstrate the effectiveness of the proposed algorithm.
79 citations
Cites background or methods from "Simultaneously Sparse and Low-Rank ..."
...straint becomes a powerful unmixing scheme, leading to many state-of-the-art algorithms (see [8], [10], [12], [18]–[21] and reference therein)....
[...]
...The reweighting strategy is widely used for many practical problems (see [12], [29], [61])....
[...]
...Though theoretical convergence analysis is hard to estimate, a series of research works has numerically shown the remarkable performance of the reweighting 1 in [12] and...
[...]
...On the other hand, a low-rank constraint of the abundance matrix has been increasingly adopted for sparse unmixing, providing a new perspective for spatial correlation [12], [21], [26]–[28], and as well as in other applications, such as compressive sensing [29] and tensor completion [30], [31]....
[...]
...[12] simultaneously impose single sparsity and low rankness on the abundance matrix for pixels lying in the homogeneous regions of HSIs....
[...]
TL;DR: A novel fusion framework for HSI classification that combines subpixel, pixel, and superpixel-based complementary information is proposed and can demonstrate the effectiveness of the proposed fusion schemes in improving discrimination capability, when compared with the classification results relied on each individual feature.
Abstract: Supervised classification of hyperspectral images (HSI) is a very challenging task due to the existence of noisy and mixed spectral characteristics. Recently, the widely developed spectral unmixing techniques offer the possibility to extract spectral mixture information at a subpixel level, which can contribute to the categorization of seriously mixed spectral pixels. Besides, it has been demonstrated that the discrimination between different materials will be improved by integrating the geometry and structure information, which can be derived from the variance between neighboring pixels. Furthermore, by incorporating the spatial context, the superpixel-based spectral–spatial similarity information can be used to smooth classification results in homogeneous regions. Therefore, a novel fusion framework for HSI classification that combines subpixel, pixel, and superpixel-based complementary information is proposed in this paper. Here, both feature fusion and decision fusion schemes are introduced. For the feature fusion scheme, the first step is to extract subpixel-level, pixel-level, and superpixel-level features from HSI, respectively. Then, the multiple feature-induced kernels are fused to form one composite kernel, which is incorporated with a support vector machine (SVM) classifier for label assignment. For the decision fusion scheme, class probabilities based on three different features are estimated by the probabilistic SVM classifier first. Then, the class probabilities are adaptively fused to form a probabilistic decision rule for classification. Experimental results tested on different real HSI images can demonstrate the effectiveness of the proposed fusion schemes in improving discrimination capability, when compared with the classification results relied on each individual feature.
68 citations
TL;DR: The experimental results on both simulated and real HSI data sets validated that the proposed method outperformed many state-of-the-art methods in terms of quantitative assessment and visual quality.
Abstract: This letter presents a novel mixed noise (i.e., Gaussian, impulse, stripe noises, or dead lines) reduction method for hyperspectral image (HSI) by utilizing low-rank representation (LRR) on spectral difference image. The proposed method is based on the assumption that all spectra in the spectral difference space of HSI lie in the same low-rank subspace. The LRR on the spectral difference space was exploited by nuclear norm of difference image along the spectral dimension. It showed great potential in removing structured sparse noise (e.g., stripes or dead lines located at the same place of each band) and heavy Gaussian noise. To simultaneously solve the proposed model and reduce computational load, alternating direction method of multipliers was utilized to achieve robust reconstruction. The experimental results on both simulated and real HSI data sets validated that the proposed method outperformed many state-of-the-art methods in terms of quantitative assessment and visual quality.
56 citations
References
More filters
Book•
[...]
01 Jan 1983
34,729 citations
"Simultaneously Sparse and Low-Rank ..." refers background in this paper
..., 2019 DRAFT 11 Concerning the computational complexity of IPSpLRU, the most complex step is the SVD of the abundance matrix Wt, which takes place in each iteration and is of the order of O(KN2+ K3), [32]. When the endmembers’ dictionary is ill-conditioned(which is a very usual situation in hyperspectral unmixing applications due to the high correlation of endmembers signatures), convergence of Wt (fr...
[...]
Book•
23 May 2011TL;DR: It is argued that the alternating direction method of multipliers is well suited to distributed convex optimization, and in particular to large-scale problems arising in statistics, machine learning, and related areas.
Abstract: Many problems of recent interest in statistics and machine learning can be posed in the framework of convex optimization. Due to the explosion in size and complexity of modern datasets, it is increasingly important to be able to solve problems with a very large number of features or training examples. As a result, both the decentralized collection or storage of these datasets as well as accompanying distributed solution methods are either necessary or at least highly desirable. In this review, we argue that the alternating direction method of multipliers is well suited to distributed convex optimization, and in particular to large-scale problems arising in statistics, machine learning, and related areas. The method was developed in the 1970s, with roots in the 1950s, and is equivalent or closely related to many other algorithms, such as dual decomposition, the method of multipliers, Douglas–Rachford splitting, Spingarn's method of partial inverses, Dykstra's alternating projections, Bregman iterative algorithms for l1 problems, proximal methods, and others. After briefly surveying the theory and history of the algorithm, we discuss applications to a wide variety of statistical and machine learning problems of recent interest, including the lasso, sparse logistic regression, basis pursuit, covariance selection, support vector machines, and many others. We also discuss general distributed optimization, extensions to the nonconvex setting, and efficient implementation, including some details on distributed MPI and Hadoop MapReduce implementations.
17,433 citations
"Simultaneously Sparse and Low-Rank ..." refers background or methods in this paper
...holds for the primal and dual residuals, where ζ = √ (3N + L)Kζ [13] (the relative tolerance ζ > 0 takes its value depending on the application and, in our experimental study, has been empirically determined to 10−4), or the maximum number of iterations is reached....
[...]
...To proceed, we utilize the auxiliary matrix variables Ω1, Ω2, Ω3, and Ω4 of proper dimensions (similar to [11] and [24]) and reformulate the original problem (P2) into its equivalent ADMM form [13], i....
[...]
...At the final step of the proposed method, the scaled Lagrange multipliers in Λ are sequentially updated by performing gradient ascent on the dual problem [13], as follows:...
[...]
...The first algorithm comes from the family of incremental proximal algorithms, which was recently presented and analyzed in [19], and makes use of the proximal operators of all the terms appearing in (P2), whereas the second algorithm exploits the splitting strategy of the ADMM philosophy [13]....
[...]
...Minimization of the resulting regularized cost function is performed by an alternating direction method of multipliers (ADMM) [13]....
[...]
TL;DR: A new version of the lasso is proposed, called the adaptive lasso, where adaptive weights are used for penalizing different coefficients in the ℓ1 penalty, and the nonnegative garotte is shown to be consistent for variable selection.
Abstract: The lasso is a popular technique for simultaneous estimation and variable selection. Lasso variable selection has been shown to be consistent under certain conditions. In this work we derive a necessary condition for the lasso variable selection to be consistent. Consequently, there exist certain scenarios where the lasso is inconsistent for variable selection. We then propose a new version of the lasso, called the adaptive lasso, where adaptive weights are used for penalizing different coefficients in the l1 penalty. We show that the adaptive lasso enjoys the oracle properties; namely, it performs as well as if the true underlying model were given in advance. Similar to the lasso, the adaptive lasso is shown to be near-minimax optimal. Furthermore, the adaptive lasso can be solved by the same efficient algorithm for solving the lasso. We also discuss the extension of the adaptive lasso in generalized linear models and show that the oracle properties still hold under mild regularity conditions. As a bypro...
6,765 citations
"Simultaneously Sparse and Low-Rank ..." refers background in this paper
...As is widely known [25], [26], [29], proper selection of these parameters is quite crucial as for the accuracy of the estimations....
[...]
...It is thus empirically verified that the enhanced efficiency of the reweighted 1 and nuclear norms, emphatically advocated in [25]–[27], is retained when using the sum of these two norms....
[...]
TL;DR: A novel method for sparse signal recovery that in many situations outperforms ℓ1 minimization in the sense that substantially fewer measurements are needed for exact recovery.
Abstract: It is now well understood that (1) it is possible to reconstruct sparse signals exactly from what appear to be highly incomplete sets of linear measurements and (2) that this can be done by constrained l1 minimization. In this paper, we study a novel method for sparse signal recovery that in many situations outperforms l1 minimization in the sense that substantially fewer measurements are needed for exact recovery. The algorithm consists of solving a sequence of weighted l1-minimization problems where the weights used for the next iteration are computed from the value of the current solution. We present a series of experiments demonstrating the remarkable performance and broad applicability of this algorithm in the areas of sparse signal recovery, statistical estimation, error correction and image processing. Interestingly, superior gains are also achieved when our method is applied to recover signals with assumed near-sparsity in overcomplete representations—not by reweighting the l1 norm of the coefficient sequence as is common, but by reweighting the l1 norm of the transformed object. An immediate consequence is the possibility of highly efficient data acquisition protocols by improving on a technique known as Compressive Sensing.
4,869 citations
"Simultaneously Sparse and Low-Rank ..." refers background in this paper
...As is widely known [25], [26], [29], proper selection of these parameters is quite crucial as for the accuracy of the estimations....
[...]
...4 Nevertheless, numerous research works advocate the positive impact of these nonconvex weighted norms on the performance of general constrained estimation tasks [26], [27], [29], [35] as well as in hyperspectral unmixing [36], [37]....
[...]
...Additionally, the reweighting norm minimization problem is known to be inherently nonconvex [26], whereas its theoretical convergence analysis for these cases is difficult to be established....
[...]
Book•
[...]
TL;DR: The many different interpretations of proximal operators and algorithms are discussed, their connections to many other topics in optimization and applied mathematics are described, some popular algorithms are surveyed, and a large number of examples of proxiesimal operators that commonly arise in practice are provided.
Abstract: This monograph is about a class of optimization algorithms called proximal algorithms. Much like Newton's method is a standard tool for solving unconstrained smooth optimization problems of modest size, proximal algorithms can be viewed as an analogous tool for nonsmooth, constrained, large-scale, or distributed versions of these problems. They are very generally applicable, but are especially well-suited to problems of substantial recent interest involving large or high-dimensional datasets. Proximal methods sit at a higher level of abstraction than classical algorithms like Newton's method: the base operation is evaluating the proximal operator of a function, which itself involves solving a small convex optimization problem. These subproblems, which generalize the problem of projecting a point onto a convex set, often admit closed-form solutions or can be solved very quickly with standard or simple specialized methods. Here, we discuss the many different interpretations of proximal operators and algorithms, describe their connections to many other topics in optimization and applied mathematics, survey some popular algorithms, and provide a large number of examples of proximal operators that commonly arise in practice.
3,627 citations
"Simultaneously Sparse and Low-Rank ..." refers background in this paper
...Let us first recall that the proximal operator of a function f(·) is defined as [31], [32]...
[...]