This article proposes diffusion LMS strategies for distributed estimation over adaptive networks that are able to exploit sparsity in the underlying system model. The approach relies on convex regularization, common in compressive sensing, to enhance the detection of sparsity via a diffusive process over the network. The resulting algorithms endow networks with learning abilities and allow them to learn the sparse structure from the incoming data in real-time, and also to track variations in the sparsity of the model. We provide convergence and mean-square performance analysis of the proposed method and show under what conditions it outperforms the unregularized diffusion version. We also show how to adaptively select the regularization parameter. Simulation results illustrate the advantage of the proposed filters for sparse data recovery.

Sparse Distributed Learning Based on Diffusion Adaptation

Sparse adaptive channel estimation problem is one of the most important topics in broadband wireless communications systems due to its simplicity and robustness. So far many sparsity-aware channel estimation algorithms have been developed based on the well-known minimum mean square error (MMSE) criterion, such as the zero-attracting least mean square (ZALMS),which are robust under Gaussian assumption. In non-Gaussian environments, however, these methods are often no longer robust especially when systems are disturbed by random impulsive noises. To address this problem, we propose in this work a robust sparse adaptive filtering algorithm using correntropy induced metric (CIM) penalized maximum correntropy criterion (MCC) rather than conventional MMSE criterion for robust channel estimation. Specifically, MCC is utilized to mitigate the impulsive noise while CIM is adopted to exploit the channel sparsity efficiently. Both theoretical analysis and computer simulations are provided to corroborate the proposed methods.

Maximum correntropy criterion based sparse adaptive filtering algorithms for robust channel estimation under non-Gaussian environments

We address the problem of in-network distributed estimation for sparse vectors In order to exploit the underlying sparsity of the vector of interest, we incorporate the l1- and l0-norm constraints into the cost function of the standard diffusion least-mean squares (LMS) This technique is equivalent to adding a zero-attracting term in the iteration of the LMS-based algorithm, which accelerates the convergence rates of the zero or near-zero components The rules for selecting the intensity of the zero-attracting term are derived and verified Simulation results show that the performances of the proposed schemes depend on the degree of sparsity Provided that suitable intensities of the zero-attracting term are selected, they can outperform the standard diffusion LMS when the considered vector is sparse In addition, a practical application of the proposed sparse algorithms in spectrum estimation for a narrow-band source is presented

Diffusion Sparse Least-Mean Squares Over Networks

As one of the recently proposed algorithms for sparse system identification, I0 norm constraint Least Mean Square (io-LMS) algorithm modifies the cost function of the traditional method with a penalty of tap-weight sparsity. The performance of I0-LMS is quite attractive compared with its various precursors. However, there has been no detailed study of its performance. This paper presents comprehensive theoretical performance analysis of I0-LMS for white Gaussian input data based on some reasonable assumptions, which are reasonable in a large range of parameter setting. Expressions for steady-state mean square deviation (MSD) are derived and discussed with respect to algorithm parameters and system sparsity. The parameter selection rule is established for achieving the best performance. Approximated with Taylor series, the instantaneous behavior is also derived. In addition, the relationship between I0-LMS and some previous arts and the sufficient conditions for I0-LMS to accelerate convergence are set up. Finally, all of the theoretical results are compared with simulations and are shown to agree well in a wide range of parameters.

Performance Analysis of $l_0$ Norm Constraint Least Mean Square Algorithm

In this paper, we propose a type of sparsity-aware set-membership normalized least mean square (SM-NLMS) algorithm for sparse channel estimation and echo cancelation. The proposed algorithm incorporates an l 1 -norm penalty into the cost function of the conventional SM-NLMS algorithm to exploit the sparsity of the sparse systems, which is denoted as zero-attracting SM-NLMS (ZASM-NLMS) algorithm. Furthermore, an improved ZASM-NLMS algorithm is also derived by using a log-sum function instead of the l 1 -norm penalty in the ZASM-NLMS, which is denoted as reweighted ZASM-NLMS (RZASM-NLMS) algorithm. These zero-attracting SM-NLMS algorithms are equivalent to adding shrinkages in their update equations, which result in fast convergence speed and low estimation error when most of the unknown channel coefficients are zero or close to zero. These proposed algorithms are described and analyzed in detail, while the performances of these algorithms are investigated by using computer simulations. The simulation results obtained from sparse channel estimation and echo cancelation demonstrate that the proposed sparse SM-NLMS algorithms are superior to the previously proposed NLMS, SM-NLMS as well as zero-attracting NLMS (ZA-NLMS) algorithms.

Peng Shi

Papers

Fast communication: Convergence analysis of sparse LMS algorithms with l1-norm penalty based on white input signal

Fast communication: Adaptive sparse Volterra system identification with l 0 -norm penalty