Signal Recovery from Random Measurements Via Orthogonal Matching Pursuit: The Gaussian Case
01 Aug 2007-
TL;DR: In this paper, a greedy algorithm called Orthogonal Matching Pursuit (OMP) was proposed to recover a signal with m nonzero entries in dimension 1 given O(m n d) random linear measurements of that signal.
Abstract: This report demonstrates theoretically and empirically that a greedy algorithm called
Orthogonal Matching Pursuit (OMP) can reliably recover a signal with m nonzero entries in dimension
d given O(mln d) random linear measurements of that signal. This is a massive improvement
over previous results, which require O(m2) measurements. The new results for OMP are comparable
with recent results for another approach called Basis Pursuit (BP). In some settings, the
OMP algorithm is faster and easier to implement, so it is an attractive alternative to BP for signal
recovery problems.
Citations
More filters
TL;DR: A so-called “one-stone-three-bird” solution for medical image acquisition and transmission based on compressive sensing theory that can be reduced to 20%, the resulted images have very well confidentiality and supporting additively homomorphic aggregation.
Abstract: Efficient medical image sampling and transferring becomes one of key research areas in computer science and healthcare application industries. In particular, the technique of body area networking and personal area networking are very useful in various image-based medical monitoring systems that cover a wide range of healthcare services, such as early detecting of emergency conditions and remote online instructing of surgeries. However, medical images are highly privacy sensitive and redundant. Thus, proper protection on privacy and secure data aggregation/compression are also highly expected in medical image processing. Based on compressive sensing theory, we conceive a so-called “one-stone-three-bird” solution for medical image acquisition and transmission in this paper. The size of the original medical images can be reduced to 20%, the resulted images have very well confidentiality and supporting additively homomorphic aggregation.
51 citations
TL;DR: This paper proposes novel unsupervised and semi-supervised dimensionality reduction algorithms by exploiting sparse data representations and shows that the proposed approaches outperform state-of-the-art dimensionality Reduction methods.
51 citations
05 Jan 2014
TL;DR: In this article, Krahmer-Mendelson-Rauhut et al. showed that the restricted isometry property is sufficient condition for the efficient reconstruction of a nearly k-sparse vector x e Cd from linear measurements Φx.
Abstract: In this paper, we present novel constructions of matrices with the restricted isometry property (RIP) that support fast matrix-vector multiplication. Our guarantees are the best known, and can also be used to obtain the best known guarantees for fast Johnson Lindenstrauss transforms.In compressed sensing, the restricted isometry property is a sufficient condition for the efficient reconstruction of a nearly k-sparse vector x e Cd from m linear measurements Φx. It is desirable for m to be small, and further it is desirable for Φ to support fast matrix-vector multiplication. Among other applications, fast multiplication improves the runtime of iterative recovery algorithms which repeatedly multiply by Φ or Φ*.The main contribution of this work is a novel randomized construction of RIP matrices Φ e Cmxd, preserving the e2 norms of all k-sparse vectors with distortion 1 + e, where the matrix-vector multiply Φx can be computed in nearly linear time. The number of rows m is on the order of e-2k log dlog2(kl oge d), an improvement on previous analyses by a logarithmic factor. Our construction, together with a connection between RIP matrices and the Johnson-Lindenstrauss lemma in [Krahmer-Ward, SIAM. J. Math. Anal. 2011], also implies fast Johnson-Lindenstrauss embeddings with asymptotically fewer rows than previously known.Our construction is actually a recipe for improving any existing family of RIP matrices. Briefly, we apply an appropriate sparse hash matrix with sign flips to any suitable family of RIP matrices. We show that the embedding properties of the original family are maintained, while at the same time improving the number of rows. The main tool in our analysis is a recent bound for the supremum of certain types of Rademacher chaos processes in [Krahmer-Mendelson-Rauhut, Comm. Pure Appl. Math. to appear].
51 citations
TL;DR: A novel vibro-acoustic modulation method, which uses linear swept sine waves for both low-frequency and high-frequency excitations, which avoids a priori knowledge of the structure and a new entropy, namely the Gnome entropy with acronym gEn, is proposed in this paper.
Abstract: Bolted connections are prone to losing their preloads with the increasing service life, thus inducing engineering accidents and economic losses in industries. Therefore, it is important to detect bolt loosening, while current structural health monitoring methods mainly focus on single-bolt joints, whose applications in industries are limited. Thus, in this paper, a novel vibro-acoustic modulation (VAM) method, is developed to detect looseness of the multi-bolt connection. Compared to traditional VAM, the proposed method uses linear swept sine waves for both low-frequency and high-frequency excitations, which avoids a priori knowledge of the structure. Moreover, the orthogonal matching pursuit method is applied to compress original modulated signals and exclude redundant features. Then, a new entropy, namely the Gnome entropy with acronym gEn, is proposed in this paper. According to simulation analysis, the gEn has better anti-noise capacity and fewer parameters than traditional entropy. Finally, after quantifying the dynamic characteristics of compressed signals to obtain feature sets through the gEn, we feed feature sets into a random forest classifier and achieve looseness detection of the multi-bolt connection. Moreover, the proposed method in this paper has great potential to detect other structural damages and provides guidance for further investigations on the VAM method.
51 citations
TL;DR: A new SAR signal processing technique based on compressed sensing is proposed for autofocused image reconstruction on subsampled raw SAR data and it is demonstrated that, even at high under-sampling ratios, the proposed technique provides reconstruction quality comparable to that obtained by the classical techniques which require full-band data without any under-Sampling.
51 citations
References
More filters
Book•
[...]
TL;DR: It is possible to design n=O(Nlog(m)) nonadaptive measurements allowing reconstruction with accuracy comparable to that attainable with direct knowledge of the N most important coefficients, and a good approximation to those N important coefficients is extracted from the n measurements by solving a linear program-Basis Pursuit in signal processing.
Abstract: Suppose x is an unknown vector in Ropfm (a digital image or signal); we plan to measure n general linear functionals of x and then reconstruct. If x is known to be compressible by transform coding with a known transform, and we reconstruct via the nonlinear procedure defined here, the number of measurements n can be dramatically smaller than the size m. Thus, certain natural classes of images with m pixels need only n=O(m1/4log5/2(m)) nonadaptive nonpixel samples for faithful recovery, as opposed to the usual m pixel samples. More specifically, suppose x has a sparse representation in some orthonormal basis (e.g., wavelet, Fourier) or tight frame (e.g., curvelet, Gabor)-so the coefficients belong to an lscrp ball for 0
18,609 citations
TL;DR: Basis Pursuit (BP) is a principle for decomposing a signal into an "optimal" superposition of dictionary elements, where optimal means having the smallest l1 norm of coefficients among all such decompositions.
Abstract: The time-frequency and time-scale communities have recently developed a large number of overcomplete waveform dictionaries --- stationary wavelets, wavelet packets, cosine packets, chirplets, and warplets, to name a few. Decomposition into overcomplete systems is not unique, and several methods for decomposition have been proposed, including the method of frames (MOF), Matching pursuit (MP), and, for special dictionaries, the best orthogonal basis (BOB).
Basis Pursuit (BP) is a principle for decomposing a signal into an "optimal" superposition of dictionary elements, where optimal means having the smallest l1 norm of coefficients among all such decompositions. We give examples exhibiting several advantages over MOF, MP, and BOB, including better sparsity and superresolution. BP has interesting relations to ideas in areas as diverse as ill-posed problems, in abstract harmonic analysis, total variation denoising, and multiscale edge denoising.
BP in highly overcomplete dictionaries leads to large-scale optimization problems. With signals of length 8192 and a wavelet packet dictionary, one gets an equivalent linear program of size 8192 by 212,992. Such problems can be attacked successfully only because of recent advances in linear programming by interior-point methods. We obtain reasonable success with a primal-dual logarithmic barrier method and conjugate-gradient solver.
9,950 citations
TL;DR: The authors introduce an algorithm, called matching pursuit, that decomposes any signal into a linear expansion of waveforms that are selected from a redundant dictionary of functions, chosen in order to best match the signal structures.
Abstract: The authors introduce an algorithm, called matching pursuit, that decomposes any signal into a linear expansion of waveforms that are selected from a redundant dictionary of functions. These waveforms are chosen in order to best match the signal structures. Matching pursuits are general procedures to compute adaptive signal representations. With a dictionary of Gabor functions a matching pursuit defines an adaptive time-frequency transform. They derive a signal energy distribution in the time-frequency plane, which does not include interference terms, unlike Wigner and Cohen class distributions. A matching pursuit isolates the signal structures that are coherent with respect to a given dictionary. An application to pattern extraction from noisy signals is described. They compare a matching pursuit decomposition with a signal expansion over an optimized wavepacket orthonormal basis, selected with the algorithm of Coifman and Wickerhauser see (IEEE Trans. Informat. Theory, vol. 38, Mar. 1992). >
9,380 citations
Stanford University1, Cleveland Clinic2, University of Toronto3, Centre national de la recherche scientifique4, Université Paris-Saclay5, University of Paris-Sud6, Avaya7, Rutgers University8, RAND Corporation9, IBM10, University of Pennsylvania11, University of Western Australia12, University of Minnesota13
TL;DR: A publicly available algorithm that requires only the same order of magnitude of computational effort as ordinary least squares applied to the full set of covariates is described.
Abstract: The purpose of model selection algorithms such as All Subsets, Forward Selection and Backward Elimination is to choose a linear model on the basis of the same set of data to which the model will be applied. Typically we have available a large collection of possible covariates from which we hope to select a parsimonious set for the efficient prediction of a response variable. Least Angle Regression (LARS), a new model selection algorithm, is a useful and less greedy version of traditional forward selection methods. Three main properties are derived: (1) A simple modification of the LARS algorithm implements the Lasso, an attractive version of ordinary least squares that constrains the sum of the absolute regression coefficients; the LARS modification calculates all possible Lasso estimates for a given problem, using an order of magnitude less computer time than previous methods. (2) A different LARS modification efficiently implements Forward Stagewise linear regression, another promising new model selection method; this connection explains the similar numerical results previously observed for the Lasso and Stagewise, and helps us understand the properties of both methods, which are seen as constrained versions of the simpler LARS algorithm. (3) A simple approximation for the degrees of freedom of a LARS estimate is available, from which we derive a Cp estimate of prediction error; this allows a principled choice among the range of possible LARS estimates. LARS and its variants are computationally efficient: the paper describes a publicly available algorithm that requires only the same order of magnitude of computational effort as ordinary least squares applied to the full set of covariates.
7,828 citations