Sparse representation has attracted much attention from researchers in fields of signal processing, image processing, computer vision, and pattern recognition. Sparse representation also has a good reputation in both theoretical research and practical applications. Many different algorithms have been proposed for sparse representation. The main purpose of this paper is to provide a comprehensive study and an updated review on sparse representation and to supply guidance for researchers. The taxonomy of sparse representation methods can be studied from various viewpoints. For example, in terms of different norm minimizations used in sparsity constraints, the methods can be roughly categorized into five groups: 1) sparse representation with $l_{0}$ -norm minimization; 2) sparse representation with $l_{p}$ -norm ( $0 ) minimization; 3) sparse representation with $l_{1}$ -norm minimization; 4) sparse representation with $l_{2,1}$ -norm minimization; and 5) sparse representation with $l_{2}$ -norm minimization. In this paper, a comprehensive overview of sparse representation is provided. The available sparse representation algorithms can also be empirically categorized into four groups: 1) greedy strategy approximation; 2) constrained optimization; 3) proximity algorithm-based optimization; and 4) homotopy algorithm-based sparse representation. The rationales of different algorithms in each category are analyzed and a wide range of sparse representation applications are summarized, which could sufficiently reveal the potential nature of the sparse representation theory. In particular, an experimentally comparative study of these sparse representation algorithms was presented.

A Survey of Sparse Representation: Algorithms and Applications

Preface On the Semilocal Convergence of Newton's Method Under Unifying Conditions Inequalities and Fixed Points in Menger Convex Metric Spaces Applications of the Perov's Fixed Point Theorem to Delay Integro-Differential Equations On Vector Equilibrium Problems with Multifunctions Fixed Points in Generalised Metric Spaces and the Stability of a Cubic Functional Equation Continuous Selection and Coincidence Theorems on Product G-Convex Space Sensitivity Analysis of Solution Set for a New Class of Generalised Implicit Quasi-Variational Inclusions Fixed Points in Probabilistic-Quasi-Metric Spaces Common Fixed Point Theorems for Condensed Mappings The Strongly Convergence Theorems of Fixed Points for Local Strictly F-Pseudocontractive On Generalised Non-linear Variational Inequalities A Note on a Paper of Hadic and Pap The Ishiwaka and Mann Iteration Methods On Certain Applications of Leray-Schauder Alternate Remarks on Concepts of Generalised Convex Spaces Generic Existence and Non-Existence of Approximate Fixed Points General System of Relaxed g-?-r-Pseudococoercive Non-linear Variational Inequalities and Projection Methods Three-Step Iteration Methods with Errors for Non-expansive Mappings in Uniformly Convex Banach Spaces On Nonnegative Linear Composite Games of NTU-Game Probabilistic Contractor and Non-linear Operator Equations with Set-Valued Operator in Probabilistic Normed Spaces Index.

Fixed Point Theory and Applications

Nowadays, a large amount of information has to be transmitted or processed. This implies high-power processing, large memory density, and increased energy consumption. In several applications, such as imaging, radar, speech recognition, and data acquisition, the signals involved can be considered sparse or compressive in some domain. The compressive sensing theory could be a proper candidate to deal with these constraints. It can be used to recover sparse or compressive signals with fewer measurements than the traditional methods. Two problems must be addressed by compressive sensing theory: design of the measurement matrix and development of an efficient sparse recovery algorithm. These algorithms are usually classified into three categories: convex relaxation, non-convex optimization techniques, and greedy algorithms. This paper intends to supply a comprehensive study and a state-of-the-art review of these algorithms to researchers who wish to develop and use them. Moreover, a wide range of compressive sensing theory applications is summarized and some open research challenges are presented.

/pdf/a-review-of-sparse-recovery-algorithms-3a8ag4tbpe.pdf

A Review of Sparse Recovery Algorithms

In the past decade, sparse and low-rank recovery has drawn much attention in many areas such as signal/image processing, statistics, bioinformatics, and machine learning. To achieve sparsity and/or low-rankness inducing, the $\ell _{1}$ norm and nuclear norm are of the most popular regularization penalties due to their convexity. While the $\ell _{1}$ and nuclear norm are convenient as the related convex optimization problems are usually tractable, it has been shown in many applications that a nonconvex penalty can yield significantly better performance. In recent, nonconvex regularization-based sparse and low-rank recovery is of considerable interest and it in fact is a main driver of the recent progress in nonconvex and nonsmooth optimization. This paper gives an overview of this topic in various fields in signal processing, statistics, and machine learning, including compressive sensing, sparse regression and variable selection, sparse signals separation, sparse principal component analysis (PCA), large covariance and inverse covariance matrices estimation, matrix completion, and robust PCA. We present recent developments of nonconvex regularization based sparse and low-rank recovery in these fields, addressing the issues of penalty selection, applications and the convergence of nonconvex algorithms. Code is available at https://github.com/FWen/ncreg.git .

A Survey on Nonconvex Regularization-Based Sparse and Low-Rank Recovery in Signal Processing, Statistics, and Machine Learning

Objective: An autoencoder-based framework that simultaneously reconstruct and classify biomedical signals is proposed. Previous work has treated reconstruction and classification as separate problems. This is the first study that proposes a combined framework to address the issue in a holistic fashion. Methods: For telemonitoring purposes, reconstruction techniques of biomedical signals are largely based on compressed sensing (CS); these are “designed” techniques where the reconstruction formulation is based on some “assumption” regarding the signal. In this study, we propose a new paradigm for reconstruction—the reconstruction is “learned,” using an autoencoder; it does not require any assumption regarding the signal as long as there is sufficiently large training data. But since the final goal is to analyze/classify the signal, the system can also learn a linear classification map that is added inside the autoencoder. The ensuing optimization problem is solved using the Split Bregman technique. Results: Experiments were carried out on reconstructing and classifying electrocardiogram (ECG) (arrhythmia classification) and EEG (seizure classification) signals. Conclusion: Our proposed tool is capable of operating in a semi-supervised fashion. We show that our proposed method is better in reconstruction and more than an order magnitude faster than CS based methods; it is capable of real-time operation. Our method also yields better results than recently proposed classification methods. Significance: This is the first study offering an alternative to CS-based reconstruction. It also shows that the representation learning approach can yield better results than traditional methods that use hand-crafted features for signal analysis.

Semi-supervised Stacked Label Consistent Autoencoder for Reconstruction and Analysis of Biomedical Signals

A new algorithm for the reconstruction of sparse signals, which is referred to as the lp-regularized least squares ( lp-RLS) algorithm, is proposed. The new algorithm is based on the minimization of a smoothed lp-norm regularized square error with p <; 1 . It uses a conjugate-gradient (CG) optimization method in a sequential minimization strategy that involves a two-parameter continuation technique. An improved version of the new algorithm is also proposed, which entails a bisection technique that optimizes an inherent regularization parameter. Extensive simulation results show that the new algorithm offers improved signal reconstruction performance and requires reduced computational effort relative to several state-of-the-art competing algorithms. The improved version of the lp-RLS algorithm offers better performance than the basic version, although this is achieved at the cost of increased computational effort.

New Improved Algorithms for Compressive Sensing Based on $\ell_{p}$ Norm

A new algorithm for the reconstruction of electrocardiogram (ECG) signals and a dictionary learning algorithm for the enhancement of its reconstruction performance for a class of signals are proposed. The signal reconstruction algorithm is based on minimizing the $\ell _{p}$ pseudo-norm of the second-order difference, called as the $\ell _{p}^{2d}$ pseudo-norm, of the signal. The optimization involved is carried out using a sequential conjugate-gradient algorithm. The dictionary learning algorithm uses an iterative procedure wherein a signal reconstruction and a dictionary update steps are repeated until a convergence criterion is satisfied. The signal reconstruction step is implemented by using the proposed signal reconstruction algorithm and the dictionary update step is implemented by using the linear least-squares method. Extensive simulation results demonstrate that the proposed algorithm yields improved reconstruction performance for temporally correlated ECG signals relative to the state-of-the-art $\ell _{p}^{1d}$ -regularized least-squares and Bayesian learning based algorithms. Also for a known class of signals, the reconstruction performance of the proposed algorithm can be improved by applying it in conjunction with a dictionary obtained using the proposed dictionary learning algorithm.

Compressive Sensing of Electrocardiogram Signals by Promoting Sparsity on the Second-Order Difference and by Using Dictionary Learning

A new algorithm for signal reconstruction in a compressive sensing framework is presented. The algorithm is based on minimizing a re-weighted approximate l 0 -norm in the null space of the measurement matrix, and the unconstrained optimization involved is performed by using a quasi-Newton algorithm. Simulation results are presented which demonstrate that the proposed algorithm yields improved signal reconstruction performance and requires a reduced amount of computation relative to iteratively re-weighted algorithms based on the l p -norm with p 0 -norm, improved signal reconstruction is achieved although the amount of computation is increased somewhat.

Reconstruction of sparse signals by minimizing a re-weighted approximate ℓ 0 -norm in the null space of the measurement matrix

A new algorithm for signal reconstruction in a compressive sensing framework is presented. The algorithm is based on minimizing an unconstrained regularized l p norm with p &#60; 1 in the null space of the measurement matrix. The unconstrained optimization involved is performed by using a quasi-Newton algorithm in which a new line search based on Banach's fixed-point theorem is used. Simulation results are presented, which demonstrate that the proposed algorithm yields improved reconstruction performance and requires a reduced amount of computation relative to several known algorithms.

/pdf/unconstrained-regularized-l-p-norm-based-algorithm-for-the-2v65pttfe5.pdf

Unconstrained regularized ℓ p -norm based algorithm for the reconstruction of sparse signals

A new algorithm for the reconstruction of signals in compressive sensing framework is proposed. The algorithm is based on a least-squares method which incorporates a regularization to promote sparsity on the gradient of the signal. It uses a sequential basic conjugate-gradient method, and it is especially suited for the reconstruction of signals which exhibit temporal correlation, e.g., electrocardiogram (ECG) signals. Simulation results are presented which demonstrate that the proposed algorithm yields upto 80.28% reduction in mean square error and from 49.95% to 65.64% reduction in the required amount of computation, relative to the state-of-the-art block sparse Bayesian learning bound-optimization algorithm.

Jeevan K. Pant

Papers

New Improved Algorithms for Compressive Sensing Based on $\ell_{p}$ Norm

Compressive Sensing of Electrocardiogram Signals by Promoting Sparsity on the Second-Order Difference and by Using Dictionary Learning

Reconstruction of sparse signals by minimizing a re-weighted approximate ℓ 0 -norm in the null space of the measurement matrix

Unconstrained regularized ℓ p -norm based algorithm for the reconstruction of sparse signals

Reconstruction of ECG signals for compressive sensing by promoting sparsity on the gradient