Showing papers in "Journal of Fourier Analysis and Applications in 2008"

Enhancing Sparsity by Reweighted ℓ 1 Minimization

Abstract: It is now well understood that (1) it is possible to reconstruct sparse signals exactly from what appear to be highly incomplete sets of linear measurements and (2) that this can be done by constrained l1 minimization. In this paper, we study a novel method for sparse signal recovery that in many situations outperforms l1 minimization in the sense that substantially fewer measurements are needed for exact recovery. The algorithm consists of solving a sequence of weighted l1-minimization problems where the weights used for the next iteration are computed from the value of the current solution. We present a series of experiments demonstrating the remarkable performance and broad applicability of this algorithm in the areas of sparse signal recovery, statistical estimation, error correction and image processing. Interestingly, superior gains are also achieved when our method is applied to recover signals with assumed near-sparsity in overcomplete representations—not by reweighting the l1 norm of the coefficient sequence as is common, but by reweighting the l1 norm of the transformed object. An immediate consequence is the possibility of highly efficient data acquisition protocols by improving on a technique known as Compressive Sensing.

Iterative thresholding for sparse approximations

Abstract: Sparse signal expansions represent or approximate a signal using a small number of elements from a large collection of elementary waveforms. Finding the optimal sparse expansion is known to be NP hard in general and non-optimal strategies such as Matching Pursuit, Orthogonal Matching Pursuit, Basis Pursuit and Basis Pursuit De-noising are often called upon. These methods show good performance in practical situations, however, they do not operate on the l0 penalised cost functions that are often at the heart of the problem. In this paper we study two iterative algorithms that are minimising the cost functions of interest. Furthermore, each iteration of these strategies has computational complexity similar to a Matching Pursuit iteration, making the methods applicable to many real world problems. However, the optimisation problem is non-convex and the strategies are only guaranteed to find local solutions, so good initialisation becomes paramount. We here study two approaches. The first approach uses the proposed algorithms to refine the solutions found with other methods, replacing the typically used conjugate gradient solver. The second strategy adapts the algorithms and we show on one example that this adaptation can be used to achieve results that lie between those obtained with Matching Pursuit and those found with Orthogonal Matching Pursuit, while retaining the computational complexity of the Matching Pursuit algorithm.

Accelerated Projected Gradient Method for Linear Inverse Problems with Sparsity Constraints

Abstract: Regularization of ill-posed linear inverse problems via l1 penalization has been proposed for cases where the solution is known to be (almost) sparse. One way to obtain the minimizer of such an l1 penalized functional is via an iterative soft-thresholding algorithm. We propose an alternative implementation to l1-constraints, using a gradient method, with projection on l1-balls. The corresponding algorithm uses again iterative soft-thresholding, now with a variable thresholding parameter. We also propose accelerated versions of this iterative method, using ingredients of the (linear) steepest descent method. We prove convergence in norm for one of these projected gradient methods, without and with acceleration.

Linear Convergence of Iterative Soft-Thresholding

Abstract: In this article a unified approach to iterative soft-thresholding algorithms for the solution of linear operator equations in infinite dimensional Hilbert spaces is presented. We formulate the algorithm in the framework of generalized gradient methods and present a new convergence analysis. As main result we show that the algorithm converges with linear rate as soon as the underlying operator satisfies the so-called finite basis injectivity property or the minimizer possesses a so-called strict sparsity pattern. Moreover it is shown that the constants can be calculated explicitly in special cases (i.e. for compact operators). Furthermore, the techniques also can be used to establish linear convergence for related methods such as the iterative thresholding algorithm for joint sparsity and the accelerated gradient projection method.

FFTs on the Rotation Group

Abstract: We discuss an implementation of an efficient algorithm for the numerical computation of Fourier transforms of bandlimited functions defined on the rotation group SO(3). The implementation is freely available on the web. The algorithm described herein uses O(B 4) operations to compute the Fourier coefficients of a function whose Fourier expansion uses only (the O(B 3)) spherical harmonics of degree at most B. This compares very favorably with the direct O(B 6) algorithm derived from a basic quadrature rule on O(B 3) sample points. The efficient Fourier transform also makes possible the efficient calculation of convolution over SO(3) which has been used as the analytic engine for some new approaches to searching 3D databases (Funkhouser et al., ACM Trans. Graph. 83–105, [2003]; Kazhdan et al., Eurographics Symposium in Geometry Processing, pp. 167–175, [2003]). Our implementation is based on the “Separation of Variables” technique (see, e.g., Maslen and Rockmore, Proceedings of the DIMACS Workshop on Groups and Computation, pp. 183–237, [1997]). In conjunction with techniques developed for the efficient computation of orthogonal polynomial expansions (Driscoll et al., SIAM J. Comput. 26(4):1066–1099, [1997]), our fast SO(3) algorithm can be improved to give an algorithm of complexity O(B 3log 2 B), but at a cost in numerical reliability. Numerical and empirical results are presented establishing the empirical stability of the basic algorithm. Examples of applications are presented as well.

Atoms of All Channels, Unite! Average Case Analysis of Multi-Channel Sparse Recovery Using Greedy Algorithms

Abstract: This paper provides new results on computing simultaneous sparse approximations of multichannel signals over redundant dictionaries using two greedy algorithms. The first one, p-thresholding, selects the S atoms that have the largest p-correlation while the second one, p-simultaneous matching pursuit (p-SOMP), is a generalisation of an algorithm studied by Tropp in (Signal Process. 86:572–588, 2006). We first provide exact recovery conditions as well as worst case analyses of all algorithms. The results, expressed using the standard cumulative coherence, are very reminiscent of the single channel case and, in particular, impose stringent restrictions on the dictionary. We unlock the situation by performing an average case analysis of both algorithms. First, we set up a general probabilistic signal model in which the coefficients of the atoms are drawn at random from the standard Gaussian distribution. Second, we show that under this model, and with mild conditions on the coherence, the probability that p-thresholding and p-SOMP fail to recover the correct components is overwhelmingly small and gets smaller as the number of channels increases. Furthermore, we analyse the influence of selecting the set of correct atoms at random. We show that, if the dictionary satisfies a uniform uncertainty principle (Candes and Tao, IEEE Trans. Inf. Theory, 52(12):5406–5425, 2006), the probability that simultaneous OMP fails to recover any sufficiently sparse set of atoms gets increasingly smaller as the number of channels increases.

Representation of Fourier Integral Operators Using Shearlets

Abstract: Traditional methods of time-frequency and multiscale analysis, such as wavelets and Gabor frames, have been successfully employed for representing most classes of pseudodifferential operators. However, these methods are not equally effective in dealing with Fourier Integral Operators in general. In this article, we show that the shearlets, recently introduced by the authors and their collaborators, provide very efficient representations for a large class of Fourier Integral Operators. The shearlets are an affine-like system of well-localized waveforms at various scales, locations and orientations, which are particularly efficient in representing anisotropic functions. Using this approach, we prove that the matrix representation of a Fourier Integral Operator with respect to a Parseval frame of shearlets is sparse and well-organized. This fact recovers a similar result recently obtained by Candes and Demanet using curvelets, which illustrates the benefits of directional multiscale representations (such as curvelets and shearlets) in the study of those functions and operators where traditional multiscale methods are unable to provide the appropriate geometric analysis in the phase space.

On the Linear Independence of Spikes and Sines

Abstract: The purpose of this work is to survey what is known about the linear independence of spikes and sines. The paper provides new results for the case where the locations of the spikes and the frequencies of the sines are chosen at random. This problem is equivalent to studying the spectral norm of a random submatrix drawn from the discrete Fourier transform matrix. The proof depends on an extrapolation argument of Bourgain and Tzafriri.

Decomposition of Spaces of Distributions Induced by Hermite Expansions

Abstract: Decomposition systems with rapidly decaying elements (needlets) based on Hermite functions are introduced and explored. It is proved that the Triebel-Lizorkin and Besov spaces on ℝ d induced by Hermite expansions can be characterized in terms of the needlet coefficients. It is also shown that the Hermite-Triebel-Lizorkin and Besov spaces are, in general, different from the respective classical spaces.

Optimal Non-Linear Models for Sparsity and Sampling

Abstract: Given a set of vectors (the data) in a Hilbert space ℋ, we prove the existence of an optimal collection of subspaces minimizing the sum of the square of the distances between each vector and its closest subspace in the collection. This collection of subspaces gives the best sparse representation for the given data, in a sense defined in the paper, and provides an optimal model for sampling in union of subspaces. The results are proved in a general setting and then applied to the case of low dimensional subspaces of ℝN and to infinite dimensional shift-invariant spaces in L2(ℝd). We also present an iterative search algorithm for finding the solution subspaces. These results are tightly connected to the new emergent theories of compressed sensing and dictionary design, signal models for signals with finite rate of innovation, and the subspace segmentation problem.

Reproducing Kernel Hilbert Spaces Associated with Analytic Translation-Invariant Mercer Kernels

Abstract: In this article we study reproducing kernel Hilbert spaces (RKHS) associated with translation-invariant Mercer kernels. Applying a special derivative reproducing property, we show that when the kernel is real analytic, every function from the RKHS is real analytic. This is used to investigate subspaces of the RKHS generated by a set of fundamental functions. The analyticity of functions from the RKHS enables us to derive some estimates for the covering numbers which form an essential part for the analysis of some algorithms in learning theory.

Gaussian Stationary Processes: Adaptive Wavelet Decompositions, Discrete Approximations, and Their Convergence

Abstract: We establish particular wavelet-based decompositions of Gaussian stationary processes in continuous time. These decompositions have a multiscale structure, independent Gaussian random variables in high-frequency terms, and the random coefficients of low-frequency terms approximating the Gaussian stationary process itself. They can also be viewed as extensions of the earlier wavelet-based decompositions of Zhang and Walter (IEEE Trans. Signal Process. 42(7):1737–1745, [1994]) for stationary processes, and Meyer et al. (J. Fourier Anal. Appl. 5(5):465–494, [1999]) for fractional Brownian motion. Several examples of Gaussian random processes are considered such as the processes with rational spectral densities. An application to simulation is presented where an associated Fast Wavelet Transform-like algorithm plays a key role.

Boundedness Properties of Pseudo-Differential and Calderón-Zygmund Operators on Modulation Spaces

Abstract: In this article, we study the boundedness of pseudo-differential operators with symbols in Sρ,δm on the modulation spaces Mp,q. We discuss the order m for the boundedness Op(Sρ,δm)⊂ℒ(Mp,q) to be true. We also prove the existence of a Calderon-Zygmund operator which is not bounded on the modulation space Mp,q with q≠2. This unboundedness is still true even if we assume a generalized T(1) condition. These results are induced by the unboundedness of pseudo-differential operators on Mp,q whose symbols are of the class S1,δ0 with 0<δ<1.

The Role of Frame Force in Quantum Detection

Abstract: A general method is given to solve tight frame optimization problems, borrowing notions from classical mechanics. In this article, we focus on a quantum detection problem, where the goal is to construct a tight frame that minimizes an error term, which in quantum physics has the interpretation of the probability of a detection error. The method converts the frame problem into a set of ordinary differential equations using concepts from classical mechanics and orthogonal group techniques. The minimum energy solutions of the differential equations are proven to correspond to the tight frames that minimize the error term. Because of this perspective, several numerical methods become available to compute the tight frames. Beyond the applications of quantum detection in quantum mechanics, solutions to this frame optimization problem can be viewed as a generalization of classical matched filtering solutions. As such, the methods we develop are a generalization of fundamental detection techniques in radar.

Beurling Dimension of Gabor Pseudoframes for Affine Subspaces

Abstract: Pseudoframes for subspaces have been recently introduced by Li and Ogawa as a tool to analyze lower dimensional data with arbitrary flexibility of both the analyzing and the dual sequence.

On some deterministic dictionaries supporting sparsity

Abstract: We describe a new construction of an incoherent dictionary, referred to as the oscillator dictionary, which is based on considerations in the representation theory of finite groups. The oscillator dictionary consists of approximately p5 unit vectors in a Hilbert space of dimension p, whose pairwise inner products have magnitude of at most \(4/\sqrt{p}\) . An explicit algorithm to construct a large portion of the oscillator dictionary is presented.

Harmonic Analysis on a Galois Field and Its Subfields

Abstract: Complex functions χ(m) where m belongs to a Galois field GF(p l ), are considered. Fourier transforms, displacements in the GF(p l )×GF(p l ) phase space and symplectic transforms of these functions are studied. It is shown that the formalism inherits many features from the theory of Galois fields. For example, Frobenius transformations and Galois groups are introduced in the present context. The relationship between harmonic analysis on GF(p l ) and harmonic analysis on its subfields, is studied.

On Schrödinger Propagator for the Special Hermite Operator

Abstract: We establish a Strichartz type estimate for the Schrodinger propagator e itℒ for the special Hermite operator ℒ on ℂ n . Our method relies on a regularization technique. We show that no admissibility condition is required on (q,p) when 1≤q≤2.

Weak Convergence of Greedy Algorithms in Banach Spaces

Abstract: We study the convergence of certain greedy algorithms in Banach spaces. We introduce the WN property for Banach spaces and prove that the algorithms converge in the weak topology for general dictionaries in uniformly smooth Banach spaces with the WN property. We show that reflexive spaces with the uniform Opial property have the WN property. We show that our results do not extend to algorithms which employ a ‘dictionary dual’ greedy step.

Sampling Theorem and Discrete Fourier Transform on the Riemann Sphere

Abstract: Using coherent-state techniques, we prove a sampling theorem for Majorana’s (holomorphic) functions on the Riemann sphere and we provide an exact reconstruction formula as a convolution product of N samples and a given reconstruction kernel (a sinc-type function). We also discuss the effect of over- and under-sampling. Sample points are roots of unity, a fact which allows explicit inversion formulas for resolution and overlapping kernel operators through the theory of Circulant Matrices and Rectangular Fourier Matrices. The case of band-limited functions on the Riemann sphere, with spins up to J, is also considered. The connection with the standard Euler angle picture, in terms of spherical harmonics, is established through a discrete Bargmann transform.

On Approximation Methods Generated by Bochner-Riesz Kernels

Abstract: Means and families of operators generated by Bochner-Riesz kernels are studied. Some sharp results on their convergence are achieved. The equivalence of the approximation errors of these methods to smoothness quantities related to the Laplacian is proved.

Three Novel Edge Detection Methods for Incomplete and Noisy Spectral Data

Abstract: We propose three novel methods for recovering edges in piecewise smooth functions from their possibly incomplete and noisy spectral information. The proposed methods utilize three different approaches: #1. The randomly-based sparse Inverse Fast Fourier Transform (sIFT); #2. The Total Variation-based (TV) compressed sensing; and #3. The modified zero crossing. The different approaches share a common feature: edges are identified through separation of scales. To this end, we advocate here the use of concentration kernels (Tadmor, Acta Numer. 16:305–378, 2007), to convert the global spectral data into an approximate jump function which is localized in the immediate neighborhoods of the edges. Building on these concentration kernels, we show that the sIFT method, the TV-based compressed sensing and the zero crossing yield effective edge detectors, where finitely many jump discontinuities are accurately recovered. One- and two-dimensional numerical results are presented.

Semyanistyi’s Integrals and Radon Transforms on Matrix Spaces

Abstract: We introduce a new analytic family of intertwining operators which include the Radon transform over matrix planes and its inverse. These operators generalize integral transformations introduced by Semyanistyi (Dokl. Akad. Nauk SSSR 134:536–539, [1960]) in his research related to the hyperplane Radon transform in ℝ n . We obtain an extended version of Fuglede’s formula, connecting generalized Semyanistyi’s integrals, Radon transforms and Riesz potentials on the space of real rectangular matrices. This result combined with the matrix analog of the Hilbert transform leads to variety of new explicit inversion formulas for the Radon transform of functions of matrix argument.

Fourier Series Approximation of Linear Fractional Stable Motion

Abstract: An approximation of the linear fractional stable motion by a Fourier sum is presented. In the continuous sample path case precise error bounds are derived. This approximation method is used to develop a simulation method of the sample path of linear fractional stable motions.

Wavelet Frames, Sobolev Embeddings and Negative Spectrum of Schrödinger Operators on Manifolds with Bounded Geometry

Abstract: We consider weighted function spaces of Sobolev-Besov type and Schrodinger type operators on noncompact Riemannian manifolds with bounded geometry. First we give characterization of the spaces in terms of wavelet frames. Then we describe the necessary and sufficient conditions for the compactness of Sobolev embeddings between the spaces. An asymptotic behavior of the corresponding entropy numbers is calculated. At the end we use the asymptotic behavior to estimate the number of negative eigenvalues of the Schrodinger type operators.

Nondeterminism of Linear Operators and Lower Entropy Estimates

Abstract: Let u be a (bounded, linear) operator from a Hilbert space ℋ into the Banach space C(T), the space of continuous functions on the compact metric space T. We introduce and investigate numbers τn(u), n≥1, measuring the degree of determinism of the operator u. The slower τn(u) decreases, the less determined are functions in the range of u by their values on a certain set of points. It is shown that n−1/2τn(u)≤2en(u), where en(u) are the (dyadic) entropy numbers of u. Furthermore, we transform the notion of strong local nondeterminism from the language of stochastic processes into that of linear operators. This property, together with a lower entropy estimate for the compact space T, leads to a lower estimate for τn(u), hence also for en(u). These results are used to prove sharp lower entropy estimates for some integral operators, among them, Riemann–Liouville operators with values in C(T) for some fractal set T. Some multi-dimensional extensions are treated as well.

Affine Synthesis onto L p when 0<p≤1

Abstract: The affine synthesis operator \(Sc=\sum_{j>0}\,\sum_{k\in { {\mathbb{Z}}^{d}}}c_{j,k}\psi_{j,k}\) is shown to map the coefficient space lp(ℤ+×ℤd) surjectively onto Lp(ℝd), for p∈(0,1] Here ψj,k(x)=|det aj|1/pψ(ajx−k) for dilation matrices aj that expand, and the synthesizer ψ∈Lp(ℝd) need satisfy only mild restrictions, for example, ψ∈L1(ℝd) with nonzero integral or else with periodization that is real-valued, nontrivial and bounded below

A Decay Result for Certain Windows Generating Orthogonal Gabor Bases

Abstract: We consider tight Gabor frames (h,a=1,b=1) at critical density with h of the form Z −1(Zg/|Zg|). Here Z is the standard Zak transform and g is an even, real, well-behaved window such that Zg has exactly one zero, at \((\frac{1}{2},\frac{1}{2})\) , in [0,1)2. We show that h and its Fourier transform have maximal decay as allowed by the Balian-Low theorem. Our result illustrates a theorem of Benedetto, Czaja, Gadzinski, and Powell, case p=q=2, on sharpness of the Balian-Low theorem.

Convolution Estimates and Model Surfaces of Low Codimension

Abstract: For k≥d/2 we give examples of measures on k-surfaces in ℝ d . These measures satisfy convolution estimates which are nearly optimal.

Proof of the Hyperplane Zeros Conjecture of Lagarias and Wang

Abstract: We prove that a real analytic subset of a torus group that is contained in its image under an expanding endomorphism is a finite union of translates of closed subgroups. This confirms the hyperplane zeros conjecture of Lagarias and Wang for real analytic varieties. Our proof uses real analytic geometry, topological dynamics, and Fourier analysis.