Given a dictionary D = {dk} of vectors dk, we seek to represent a signal S as a linear combination S = ∑k γ(k)dk, with scalar coefficients γ(k). In particular, we aim for the sparsest representation possible. In general, this requires a combinatorial optimization process. Previous work considered the special case where D is an overcomplete system consisting of exactly two orthobases and has shown that, under a condition of mutual incoherence of the two bases, and assuming that S has a sufficiently sparse representation, this representation is unique and can be found by solving a convex optimization problem: specifically, minimizing the l1 norm of the coefficients γ. In this article, we obtain parallel results in a more general setting, where the dictionary D can arise from two or several bases, frames, or even less structured systems. We sketch three applications: separating linear features from planar ones in 3D data, noncooperative multiuser encoding, and identification of over-complete independent component models.

https://www.pnas.org/content/pnas/100/5/2197.full.pdf

Optimally sparse representation in general (nonorthogonal) dictionaries via 1 minimization

In this paper, we analyse two well-known objective image quality metrics, the peak-signal-to-noise ratio (PSNR) as well as the structural similarity index measure (SSIM), and we derive a simple mathematical relationship between them which works for various kinds of image degradations such as Gaussian blur, additive Gaussian white noise, jpeg and jpeg2000 compression. A series of tests realized on images extracted from the Kodak database gives a better understanding of the similarity and difference between the SSIM and the PSNR.

Image Quality Metrics: PSNR vs. SSIM

Overcomplete representations are attracting interest in signal processing theory, particularly due to their potential to generate sparse representations of signals. However, in general, the problem of finding sparse representations must be unstable in the presence of noise. This paper establishes the possibility of stable recovery under a combination of sufficient sparsity and favorable structure of the overcomplete system. Considering an ideal underlying signal that has a sufficiently sparse representation, it is assumed that only a noisy version of it can be observed. Assuming further that the overcomplete system is incoherent, it is shown that the optimally sparse approximation to the noisy data differs from the optimally sparse decomposition of the ideal noiseless signal by at most a constant multiple of the noise level. As this optimal-sparsity method requires heavy (combinatorial) computational effort, approximation algorithms are considered. It is shown that similar stability is also available using the basis and the matching pursuit algorithms. Furthermore, it is shown that these methods result in sparse approximation of the noisy data that contains only terms also appearing in the unique sparsest representation of the ideal noiseless sparse signal.

Stable recovery of sparse overcomplete representations in the presence of noise

Suppose a discrete-time signal S(t), 0/spl les/t<N, is a superposition of atoms taken from a combined time-frequency dictionary made of spike sequences 1/sub {t=/spl tau/}/ and sinusoids exp{2/spl pi/iwt/N}//spl radic/N. Can one recover, from knowledge of S alone, the precise collection of atoms going to make up S? Because every discrete-time signal can be represented as a superposition of spikes alone, or as a superposition of sinusoids alone, there is no unique way of writing S as a sum of spikes and sinusoids in general. We prove that if S is representable as a highly sparse superposition of atoms from this time-frequency dictionary, then there is only one such highly sparse representation of S, and it can be obtained by solving the convex optimization problem of minimizing the l/sup 1/ norm of the coefficients among all decompositions. Here "highly sparse" means that N/sub t/+N/sub w/</spl radic/N/2 where N/sub t/ is the number of time atoms, N/sub w/ is the number of frequency atoms, and N is the length of the discrete-time signal. Underlying this result is a general l/sup 1/ uncertainty principle which says that if two bases are mutually incoherent, no nonzero signal can have a sparse representation in both bases simultaneously. For the above setting, the bases are sinusoids and spikes, and mutual incoherence is measured in terms of the largest inner product between different basis elements. The uncertainty principle holds for a variety of interesting basis pairs, not just sinusoids and spikes. The results have idealized applications to band-limited approximation with gross errors, to error-correcting encryption, and to separation of uncoordinated sources. Related phenomena hold for functions of a real variable, with basis pairs such as sinusoids and wavelets, and for functions of two variables, with basis pairs such as wavelets and ridgelets. In these settings, if a function f is representable by a sufficiently sparse superposition of terms taken from both bases, then there is only one such sparse representation; it may be obtained by minimum l/sup 1/ norm atomic decomposition. The condition "sufficiently sparse" becomes a multiscale condition; for example, that the number of wavelets at level j plus the number of sinusoids in the jth dyadic frequency band are together less than a constant times 2/sup j/2/.

/pdf/uncertainty-principles-and-ideal-atomic-decomposition-16unopg10o.pdf

Uncertainty principles and ideal atomic decomposition

http://www.yaroslavvb.com/papers/bozdogan-akaike.pdf

Akaike's information criterion and recent developments in information complexity

The adaptive nonlinear predistorter is an effective technique to compensate for the nonlinear distortion existing in digital communication and control systems. However, available adaptive nonlinear predistorters using indirect learning are sensitive to measurement noise and do not perform optimally. Other available types are either slow to converge, complicated in structure and computationally expensive, or do not consider the memory effects in nonlinear systems such as a high power amplifier (HPA). In this paper, we first propose several novel adaptive nonlinear predistorters based on direct learning algorithms - the nonlinear filtered-x RLS (NFXRLS) algorithm, the nonlinear adjoint LMS (NALMS) algorithm, and the nonlinear adjoint RLS (NARLS) algorithm. Using these new learning algorithms, we design adaptive nonlinear predistorters for an HPA with memory effects or for an HPA following a linear system. Because of the direct learning algorithm, these novel adaptive predistorters outperform nonlinear predistorters that are based on the indirect learning method in the sense of normalized mean square error (NMSE), bit error rate (BER), and spectral regrowth. Moreover, our developed adaptive nonlinear predistorters are computationally efficient and/or converge rapidly when compared to other adaptive nonlinear predistorters that use direct learning, and furthermore can be easily implemented. We further simplify our proposed algorithms by exploring the robustness of our proposed algorithm as well as by examining the statistical properties of what we call the "instantaneous equivalent linear" (IEL) filter. Simulation results confirm the effectiveness of our proposed algorithms

Novel Adaptive Nonlinear Predistorters Based on the Direct Learning Algorithm

In this paper, we treat nonlinear active noise control (NANC) with a linear secondary path (LSP) and with a nonlinear secondary path (NSP) in a unified structure by introducing a new virtual secondary path filter concept and using a general function expansion nonlinear filter. We discover that using the filtered-error structure results in greatly reducing the computational complexity of NANC. As a result, we extend the available filtered-error-based algorithms to solve NANC/LSP problems and, furthermore, develop our adjoint filtered-error-based algorithms for NANC/NSP. This family of algorithms is computationally efficient and possesses a simple structure. We also find that the computational complexity of NANC/NSP can be reduced even more using block-oriented nonlinear models, such as the Wiener, Hammerstein, or linear-nonlinear-linear (LNL) models for the NSP. Finally, we use the statistical properties of the virtual secondary path and the robustness of our proposed methods to further reduce the computational complexity and simplify the implementation structure of NANC/NSP when the NSP satisfies certain conditions. Computational complexity and simulation results are given to confirm the efficiency and effectiveness of all of our proposed methods

Efficient Adaptive Nonlinear Filters for Nonlinear Active Noise Control

Active noise control (ANC) has been widely applied in industry to reduce environmental noise and equipment vibrations. Most available control algorithms require the identification of the secondary path, which increases the control system complexity, contributes to an increased residual noise power, and can even cause the control system to fail if the identified secondary path is not sufficiently close to the actual path. In this paper, based on the geometric analysis and the strict positive real (SPR) property of the filtered-x LMS algorithm, we introduce a new ANC algorithm suitable for single-tone noises as well as some specific narrowband noises that does not require the identification of the secondary path, though its convergence can be very slow in some special cases. We are able to extend the developed ANC algorithm to the case of active control of broadband noises through our use of a subband implementation of the ANC algorithm. Compared to other available control algorithms that do not require secondary path identification, our developed method is simple to implement, yields good performance, and converges quickly. Simulation results confirm the effectiveness of our proposed algorithm

A New Active Noise Control Algorithm That Requires No Secondary Path Identification Based on the SPR Property

http://cimms.ou.edu/~lakshman/Papers/wdssii_kmeans.pdf

Multiscale storm identification and forecast

We introduce a new measure H/sub p/ that is related to the Heisenberg uncertainty principle. The measure predicts the compactness of discrete-time signal descriptions in the sample-frequency phase plane. We conjecture a lower limit on the compaction in the phase plane and show that discretized Gaussians may not provide the most compact basis.

Victor DeBrunner

Papers

Novel Adaptive Nonlinear Predistorters Based on the Direct Learning Algorithm

Efficient Adaptive Nonlinear Filters for Nonlinear Active Noise Control

A New Active Noise Control Algorithm That Requires No Secondary Path Identification Based on the SPR Property

Multiscale storm identification and forecast

Resolution in time-frequency