Showing papers by "Emmanuel J. Candès published in 2007"

The Dantzig selector: Statistical estimation when p is much larger than n

Abstract: In many important statistical applications, the number of variables or parameters p is much larger than the number of observations n. Suppose then that we have observations y=Xβ+z, where β∈Rp is a parameter vector of interest, X is a data matrix with possibly far fewer rows than columns, n≪p, and the zi’s are i.i.d. N(0, σ^2). Is it possible to estimate β reliably based on the noisy data y?

Sparsity and incoherence in compressive sampling

Abstract: We consider the problem of reconstructing a sparse signal x 0 2 R n from a limited number of linear measurements. Given m randomly selected samples of Ux 0 , where U is an orthonormal matrix, we show that ‘1 minimization recovers x 0 exactly when the number of measurements exceeds m Const ·µ 2 (U) ·S · logn, where S is the number of nonzero components in x 0 , and µ is the largest entry in U properly normalized: µ(U) = p n · maxk,j |Uk,j|. The smaller µ, the fewer samples needed. The result holds for “most” sparse signals x 0 supported on a fixed (but arbitrary) set T. Given T, if the sign of x 0 for each nonzero entry on T and the observed values of Ux 0 are drawn at random, the signal is recovered with overwhelming probability. Moreover, there is a sense in which this is nearly optimal since any method succeeding with the same probability would require just about this many samples.

Enhancing Sparsity by Reweighted L1 Minimization

Abstract: It is now well understood that (1) it is possible to reconstruct sparse signals exactly from what appear to be highly incomplete sets of linear measurements and (2) that this can be done by constrained L1 minimization. In this paper, we study a novel method for sparse signal recovery that in many situations outperforms L1 minimization in the sense that substantially fewer measurements are needed for exact recovery. The algorithm consists of solving a sequence of weighted L1-minimization problems where the weights used for the next iteration are computed from the value of the current solution. We present a series of experiments demonstrating the remarkable performance and broad applicability of this algorithm in the areas of sparse signal recovery, statistical estimation, error correction and image processing. Interestingly, superior gains are also achieved when our method is applied to recover signals with assumed near-sparsity in overcomplete representations--not by reweighting the L1 norm of the coefficient sequence as is common, but by reweighting the L1 norm of the transformed object. An immediate consequence is the possibility of highly efficient data acquisition protocols by improving on a technique known as compressed sensing.

Fast Computation of Fourier Integral Operators

Abstract: We introduce a general purpose algorithm for rapidly computing certain types of oscillatory integrals which frequently arise in problems connected to wave propagation, general hyperbolic equations, and curvilinear tomography. The problem is to numerically evaluate a so-called Fourier integral operator (FIO) of the form $\int e^{2\pi i \Phi(x,\xi)} a(x,\xi) \hat{f}(\xi) \d\xi$ at points given on a Cartesian grid. Here, $\xi$ is a frequency variable, $\hat f(\xi)$ is the Fourier transform of the input $f$, $a(x,\xi)$ is an amplitude, and $\Phi(x,\xi)$ is a phase function, which is typically as large as $|\xi|$; hence the integral is highly oscillatory. Because a FIO is a dense matrix, a naive matrix vector product with an input given on a Cartesian grid of size $N$ by $N$ would require $O(N^4)$ operations. This paper develops a new numerical algorithm which requires $O(N^{2.5} \log N)$ operations and as low as $O(\sqrt{N})$ in storage space (the constants in front of these estimates are small). It operates by localizing the integral over polar wedges with small angular aperture in the frequency plane. On each wedge, the algorithm factorizes the kernel $e^{2 \pi i \Phi(x,\xi)} a(x,\xi)$ into two components: (1) a diffeomorphism which is handled by means of a nonuniform FFT and (2) a residual factor which is handled by numerical separation of the spatial and frequency variables. The key to the complexity and accuracy estimates is the fact that the separation rank of the residual kernel is provably independent of the problem size. Several numerical examples demonstrate the numerical accuracy and low computational complexity of the proposed methodology. We also discuss the potential of our ideas for various applications such as reflection seismology.

Rejoinder: the dantzig selector: statistical estimation when p is much larger than n

Abstract: First of all, we would like to thank all the discussants for their interest and comments, as well as for their thorough investigation. The comments all underlie the importance and timeliness of the topics discussed in our paper, namely, accurate statistical estimation in high dimensions. We would also like to thank the editors for this opportunity to comment briefly on a few issues raised in the discussions. Of special interest is the diversity of perspectives, which include theoretical, practical and computational issues. With this being said, there are two main points in the discussions that are quite recurrent: 1. Is it possible to extend and refine our theoretical results, and how do they compare against the very recent literature? 2. How does the Dantzig Selector (DS) compare with the Lasso? We will address these issues in this rejoinder but before we begin, we would like to restate as simply as possible the main point of our paper and put this work in a broader context so as to avoid confusion about our point of view and motivations.

Sparse signal and image recovery from compressive samples

Abstract: In this paper we present an introduction to compressive sampling (CS), an emerging model-based framework for data acquisition and signal recovery based on the premise that a signal having a sparse representation in one basis can be reconstructed from a small number of measurements collected in a second basis that is incoherent with the first. Interestingly, a random noise-like basis will suffice for the measurement process. We will overview the basic CS theory, discuss efficient methods for signal reconstruction, and highlight applications in medical imaging

Searching for a trail of evidence in a maze

Abstract: Consider a graph with a set of vertices and oriented edges connecting pairs of vertices. Each vertex is associated with a random variable and these are assumed to be independent. In this setting, suppose we wish to solve the following hypothesis testing problem: under the null, the random variables have common distribution N(0,1) while under the alternative, there is an unknown path along which random variables have distribution $N(\mu,1)$, $\mu> 0$, and distribution N(0,1) away from it. For which values of the mean shift $\mu$ can one reliably detect and for which values is this impossible? Consider, for example, the usual regular lattice with vertices of the form \[\{(i,j):0\le i,-i\le j\le i and j has the parity of i\}\] and oriented edges $(i,j)\to (i+1,j+s)$, where $s=\pm1$. We show that for paths of length $m$ starting at the origin, the hypotheses become distinguishable (in a minimax sense) if $\mu_m\gg1/\sqrt{\log m}$, while they are not if $\mu_m\ll1/\log m$. We derive equivalent results in a Bayesian setting where one assumes that all paths are equally likely; there, the asymptotic threshold is $\mu_m\approx m^{-1/4}$. We obtain corresponding results for trees (where the threshold is of order 1 and independent of the size of the tree), for distributions other than the Gaussian and for other graphs. The concept of the predictability profile, first introduced by Benjamini, Pemantle and Peres, plays a crucial role in our analysis.

Error correction and convex programming

Abstract: This article discusses a recently proposed error correction method involving convex optimization [1]. From an encoded and corrupted real-valued message, a receiver would like to determine the original message. A few entries of the encoded message are corrupted arbitrarily (which we call gross errors) and all the entries of the encoded message are corrupted slightly. We show that it is possible to recover the message with nearly the same accuracy as in the setting where no gross errors occur. (© 2008 WILEY-VCH Verlag GmbH & Co. KGaA, Weinheim)