Signal Recovery from Random Measurements Via Orthogonal Matching Pursuit: The Gaussian Case
01 Aug 2007-
TL;DR: In this paper, a greedy algorithm called Orthogonal Matching Pursuit (OMP) was proposed to recover a signal with m nonzero entries in dimension 1 given O(m n d) random linear measurements of that signal.
Abstract: This report demonstrates theoretically and empirically that a greedy algorithm called
Orthogonal Matching Pursuit (OMP) can reliably recover a signal with m nonzero entries in dimension
d given O(mln d) random linear measurements of that signal. This is a massive improvement
over previous results, which require O(m2) measurements. The new results for OMP are comparable
with recent results for another approach called Basis Pursuit (BP). In some settings, the
OMP algorithm is faster and easier to implement, so it is an attractive alternative to BP for signal
recovery problems.
Citations
More filters
TL;DR: In this paper, a photonic-assisted multi-channel compressive sampling scheme is proposed with one pseudo-random binary sequence (PRBS) source and Wavelength Division Multiplexing-based time delay, and the restricted isometry property of sensing matrix determined by the optimized time delay pattern is analyzed.
Abstract: In this paper, a photonic-assisted multi-channel compressive sampling scheme is proposed with one pseudo-random binary sequence (PRBS) source and Wavelength Division Multiplexing-based time delay Meanwhile, the restricted isometry property of sensing matrix determined by the optimized time delay pattern is analyzed In experiment, a four-channel photonic-assisted system with 5-GHz bandwidth was set up, where four-channel PRBS signals were generated by adding fiber-induced constant time delays to four-wavelength modulated PRBS signal, and a signal composed of twenty tones was recovered faithfully with four analog-to-digital converters (ADCs) with only 120-MHz-bandwidth
39 citations
Book•
07 Jan 2016
TL;DR: This book is designed for students, professionals and researchers in the field of multimedia and related fields with a need to learn the basics of multimedia systems and signal processing with special attention to the compressive sensing area.
Abstract: This book is designed for students, professionals and researchers in the field of multimedia and related fields with a need to learn the basics of multimedia systems and signal processing. Emphasis is given to the analysis and processing of multimedia signals (audio, images, and video). Detailed insight into the most relevant mathematical apparatus and transformations used in multimedia signal processing is given. A unique relationship between different transformations is also included, opening new perspectives for defining novel transforms in specific applications. Special attention is dedicated to the compressive sensing area, which has a great potential to contribute to further improvement of modern multimedia systems. In addition to the theoretical concepts, various standard and more recently accepted algorithms for the reconstruction of different types of signals are considered. Additional information and details are also provided to enable a comprehensive analysis of audio and video compression algorithms. Finally, the book connects these principles to other important elements of multimedia systems, such as the analysis of optical media, digital watermarking, and telemedicine. New to this edition:Introduction of the generalization concept to consolidate the time-frequency signal analysis, wavelet transformation, and Hermite transformation Inclusion of prominent robust transformation theory used in the processing of noisy multimedia data as well as advanced multimedia data filtering approaches, including image filtering techniques for impulse noise environment Extended video compression algorithms Detailed coverage of compressive sensing in multimedia applications
39 citations
TL;DR: This work sharpen the approximation bounds under more relaxed conditions of the sparsest representation of the over complete dictionary and derives analogous results for a stepwise projection algorithm.
Abstract: Sparse over complete representations have attracted much interest recently for their applications to signal processing. In a recent work, Donoho, Elad, and Temlyakov (2006) showed that, assuming sufficient sparsity of the ideal underlying signal and approximate orthogonality of the over complete dictionary, the sparsest representation can be found, at least approximately if not exactly, by either an orthogonal greedy algorithm or by lscr1-norm minimization subject to a noise tolerance constraint. In this paper, we sharpen the approximation bounds under more relaxed conditions. We also derive analogous results for a stepwise projection algorithm.
39 citations
TL;DR: The Chebyshev chaotic sensing matrix (CsCSM) is proved to satisfy RIP with overwhelming probability and can be easily implemented in hardware circuit and will be more beneficial in some applications which require security and privacy, as opposed to random sensing matrices.
Abstract: Compressive sensing is a new sampling theory which allows for signal sampling at a sub-Nyquist rate. In order to ensure exact reconstruction from very few measurements, one should design a stable sensing matrix, which satisfies restricted isometry property (RIP), such that it preserves the significant information of original signal in sensing procedure. In this paper, a novel sensing matrix is proposed based on the Chebyshev chaotic system, and the Chebyshev chaotic sensing matrix (CsCSM) is proved to satisfy RIP with overwhelming probability. Numerical simulations show that the CsCSM is sufficient to guarantee exact recovery, which is similar to random sensing matrices such as Gaussian sensing matrix. However, the CsCSM can be easily implemented in hardware circuit and will be more beneficial in some applications which require security and privacy, as opposed to random sensing matrices.
39 citations
TL;DR: This paper explores the efficacy of several object recognition algorithms at classifying ships and other ocean vessels in commercial panchromatic satellite imagery and discusses how these algorithms are being used in existing systems to detect and classify vessels in satellite imagery.
Abstract: Recognition and classification of vessels in maritime imagery is a challenging problem with applications to security and military scenarios. Aspects of this problem are similar to well-studied problems in object recognition, but it is in many ways more complex than a problem such as face recognition. A vessel's appearance can vary significantly from image to image depending on factors such as lighting condition, viewing geometry, and sea state, and there is often wide variation between ships of the same class. This paper explores the efficacy of several object recognition algorithms at classifying ships and other ocean vessels in commercial panchromatic satellite imagery. The recognition algorithms tested include traditional classification methods as well as more recent methods utilizing sparse matrix representations and dictionary learning. The impacts on classification accuracy of various pre-processing steps on vessel imagery are explored, and we discuss how these algorithms are being used in existing systems to detect and classify vessels in satellite imagery.
39 citations
References
More filters
Book•
[...]
TL;DR: It is possible to design n=O(Nlog(m)) nonadaptive measurements allowing reconstruction with accuracy comparable to that attainable with direct knowledge of the N most important coefficients, and a good approximation to those N important coefficients is extracted from the n measurements by solving a linear program-Basis Pursuit in signal processing.
Abstract: Suppose x is an unknown vector in Ropfm (a digital image or signal); we plan to measure n general linear functionals of x and then reconstruct. If x is known to be compressible by transform coding with a known transform, and we reconstruct via the nonlinear procedure defined here, the number of measurements n can be dramatically smaller than the size m. Thus, certain natural classes of images with m pixels need only n=O(m1/4log5/2(m)) nonadaptive nonpixel samples for faithful recovery, as opposed to the usual m pixel samples. More specifically, suppose x has a sparse representation in some orthonormal basis (e.g., wavelet, Fourier) or tight frame (e.g., curvelet, Gabor)-so the coefficients belong to an lscrp ball for 0
18,609 citations
TL;DR: Basis Pursuit (BP) is a principle for decomposing a signal into an "optimal" superposition of dictionary elements, where optimal means having the smallest l1 norm of coefficients among all such decompositions.
Abstract: The time-frequency and time-scale communities have recently developed a large number of overcomplete waveform dictionaries --- stationary wavelets, wavelet packets, cosine packets, chirplets, and warplets, to name a few. Decomposition into overcomplete systems is not unique, and several methods for decomposition have been proposed, including the method of frames (MOF), Matching pursuit (MP), and, for special dictionaries, the best orthogonal basis (BOB).
Basis Pursuit (BP) is a principle for decomposing a signal into an "optimal" superposition of dictionary elements, where optimal means having the smallest l1 norm of coefficients among all such decompositions. We give examples exhibiting several advantages over MOF, MP, and BOB, including better sparsity and superresolution. BP has interesting relations to ideas in areas as diverse as ill-posed problems, in abstract harmonic analysis, total variation denoising, and multiscale edge denoising.
BP in highly overcomplete dictionaries leads to large-scale optimization problems. With signals of length 8192 and a wavelet packet dictionary, one gets an equivalent linear program of size 8192 by 212,992. Such problems can be attacked successfully only because of recent advances in linear programming by interior-point methods. We obtain reasonable success with a primal-dual logarithmic barrier method and conjugate-gradient solver.
9,950 citations
TL;DR: The authors introduce an algorithm, called matching pursuit, that decomposes any signal into a linear expansion of waveforms that are selected from a redundant dictionary of functions, chosen in order to best match the signal structures.
Abstract: The authors introduce an algorithm, called matching pursuit, that decomposes any signal into a linear expansion of waveforms that are selected from a redundant dictionary of functions. These waveforms are chosen in order to best match the signal structures. Matching pursuits are general procedures to compute adaptive signal representations. With a dictionary of Gabor functions a matching pursuit defines an adaptive time-frequency transform. They derive a signal energy distribution in the time-frequency plane, which does not include interference terms, unlike Wigner and Cohen class distributions. A matching pursuit isolates the signal structures that are coherent with respect to a given dictionary. An application to pattern extraction from noisy signals is described. They compare a matching pursuit decomposition with a signal expansion over an optimized wavepacket orthonormal basis, selected with the algorithm of Coifman and Wickerhauser see (IEEE Trans. Informat. Theory, vol. 38, Mar. 1992). >
9,380 citations
Stanford University1, Cleveland Clinic2, University of Toronto3, Centre national de la recherche scientifique4, Université Paris-Saclay5, University of Paris-Sud6, Avaya7, Rutgers University8, RAND Corporation9, IBM10, University of Pennsylvania11, University of Western Australia12, University of Minnesota13
TL;DR: A publicly available algorithm that requires only the same order of magnitude of computational effort as ordinary least squares applied to the full set of covariates is described.
Abstract: The purpose of model selection algorithms such as All Subsets, Forward Selection and Backward Elimination is to choose a linear model on the basis of the same set of data to which the model will be applied. Typically we have available a large collection of possible covariates from which we hope to select a parsimonious set for the efficient prediction of a response variable. Least Angle Regression (LARS), a new model selection algorithm, is a useful and less greedy version of traditional forward selection methods. Three main properties are derived: (1) A simple modification of the LARS algorithm implements the Lasso, an attractive version of ordinary least squares that constrains the sum of the absolute regression coefficients; the LARS modification calculates all possible Lasso estimates for a given problem, using an order of magnitude less computer time than previous methods. (2) A different LARS modification efficiently implements Forward Stagewise linear regression, another promising new model selection method; this connection explains the similar numerical results previously observed for the Lasso and Stagewise, and helps us understand the properties of both methods, which are seen as constrained versions of the simpler LARS algorithm. (3) A simple approximation for the degrees of freedom of a LARS estimate is available, from which we derive a Cp estimate of prediction error; this allows a principled choice among the range of possible LARS estimates. LARS and its variants are computationally efficient: the paper describes a publicly available algorithm that requires only the same order of magnitude of computational effort as ordinary least squares applied to the full set of covariates.
7,828 citations