The time-frequency and time-scale communities have recently developed an enormous number of over-complete signal dictionaries, wavelets, wavelet packets, cosine packets, Wilson bases, chirplets, warped bases, and hyperbolic cross bases being a few examples. Basis pursuit is a technique for decomposing a signal into an "optimal" superposition of dictionary elements. The optimization criterion is the l/sup 1/ norm of coefficients. The method has several advantages over matching pursuit and best ortho basis, including super-resolution and stability. >

Basis pursuit

Many advances in the development of Krylov subspace methods for the iterative solution of linear systems during the last decade and a half are reviewed. These new developments include different versions of restarted, augmented, deflated, flexible, nested, and inexact methods. Also reviewed are methods specifically tailored to systems with special properties such as special forms of symmetry and those depending on one or more parameters. Copyright © 2006 John Wiley & Sons, Ltd.

/pdf/recent-computational-developments-in-krylov-subspace-methods-3tkevs7v96.pdf

Recent computational developments in krylov subspace methods for linear systems

This paper describes the “fractional Fourier transform,” which admits computation by an algorithm that has complexity proportional to the fast Fourier transform algorithm. Whereas the discrete Fourier transform (DFT) is based on integral roots of unity $e^{{{ - 2\pi i} / n}} $, the fractional Fourier transform is based on fractional roots of unity $e^{ - 2\pi i\alpha } $ where $\alpha $ is arbitrary. The fractional Fourier transform and the corresponding fast algorithm are useful for such applications as computing DFTs of sequences with prime lengths, computing DFTs of sparse sequences, analyzing sequences with noninteger periodicities, performing high-resolution trigonometric interpolation, detecting lines in noisy images, and detecting signals with linearly drifting frequencies. In many cases, the resulting algorithms are faster by arbitrarily large factors than conventional techniques.

The fractional Fourier transform and applications

In traditional physicochemical modeling, one derives evolution equations at the (macroscopic, coarse) scale of interest; these are used to perform a variety of tasks (simulation, bifurcation analysis, optimization) using an arsenal of analytical and numerical techniques. For many complex systems, however, although one observes evolution at a macroscopic scale of interest, accurate models are only given at a more detailed (fine-scale, microscopic) level of description (e.g., lattice Boltzmann, kinetic Monte Carlo, molecular dynamics). Here, we review a framework for computer-aided multiscale analysis, which enables macroscopic computational tasks (over extended spatiotemporal scales) using only appropriately initialized microscopic simulation on short time and length scales. The methodology bypasses the derivation of macroscopic evolution equations when these equations conceptually exist but are not available in closed form—hence the term equation-free. We selectively discuss basic algorithms and underlyin...

Equation-Free Multiscale Computation: Algorithms and Applications

The generalized minimum residual method (GMRES) is well known for solving large nonsymmetric systems of linear equations. It generally uses restarting, which slows the convergence. However, some information can be retained at the time of the restart and used in the next cycle. We present algorithms that use implicit restarting in order to retain this information. Approximate eigenvectors determined from the previous subspace are included in the new subspace. This deflates the smallest eigenvalues and thus improves the convergence. The subspace that contains the approximate eigenvectors is itself a Krylov subspace, but not with the usual starting vector. The implicitly restarted FOM algorithm includes standard Ritz vectors in the subspace. The eigenvalue portion of its calculations is equivalent to Sorensen's IRA algorithm. The implicitly restarted GMRES algorithm uses harmonic Ritz vectors. This algorithm also gives a new approach to computing interior eigenvalues.

Implicitly Restarted GMRES and Arnoldi Methods for Nonsymmetric Systems of Equations

This paper explores the use of adaptive polynomial preconditioning for Hermitian positive definite linear systems, $Ax = b$. Such preconditioners are easy to employ and well suited to vector and/or parallel machines. After examining the role of polynomial preconditioning in conjugate gradient methods, the least squares and Chebyshev preconditioning polynomials are discussed. Eigenvalue distributions for which each is well suited are then determined. An adaptive procedure for dynamically computing the best Chebyshev polynomial preconditioner is also described. Finally, the effectiveness of adaptive polynomial preconditioning is demonstrated in a variety of numerical experiments on a Cray X-MP/48 and Alliant FX/8. The results suggest that relatively low degree (2–16) polynomials are usually best.

A comparison of adaptive Chebyshev and least squares polynomial preconditioning for Hermitian positive definite linear systems

Preconditioning strategies for elliptic partial differential operators and corresponding difference operators are compared by way of a model problem. The technique considered here makes use of a selfadjoint positive definite operator B as the preconditioner for the nonselfadjoint convection -diffusion operator A. Preconditioning by A’s leading term plus a positive zeroth-order term $\sigma I$ and optimizing over $\sigma $ is proposed. The analysis of a model problem indicates that when A has large first-order terms, both the spectral and norm condition of $B^{ - 1} A$ are minimized by $\sigma \gg 0$.

Optimal equivalent preconditioners

The original Cooley-Tukey FFT was published in 1965 and presented for sequences with length N equal to a power of two. However, in the same paper they noted that their algorithm could be generalized to composite N in which the length of the sequence was a product of small primes. In 1967, Bergland presented an algorithm for composite N and variants of his mixed radix FFT are currently in wide use. In 1968, Bluestein presented an FFT for arbitrary N including large primes. However, for composite N, Bluestein's FFT was not competitive with Bergland's FFT. Since it is usually possible to select a composite N, Bluestein's FFT did not receive much attention. Nevertheless because of its minimal communication requirements, the Bluestein FFT may be the algorithm of choice on multiprocessors, particularly those with the hypercube architecture. In contrast to the mixed radix FFT, the communication pattern of the Bluestein FFT maps quite well onto the hypercube. With P = 2^d processors, an ordered Bluestein FFT requires 2d communication cycles with packet length N/2P which is comparable to the requirements of a power of two FFT. For fine-grain computations, the Bluestein FFT requires 20log"2N computational cycles. Although this is double that required for a mixed radix FFT, the Bluestein FFT may nevertheless be preferred because of its lower communication costs. For most values of N it is also shown to be superior to another alternative, namely parallel multiplication.

Paper: Bluestein's FFT for arbitrary N on the hypercube

In this paper we explore two sets of polynomials, the orthogonal polynomials and the residual polynomials, associated with a preconditioned conjugate gradient iteration for the solution of the linear system Ax = b. In the context of preconditioning by the matrix C, we show that the roots of the orthogonal polynomials, also known as generalized Ritz values, are the eigenvalues of an orthogonal section of the matrix C A while the roots of the residual polynomials, also known as pseudo-Ritz values (or roots of kernel polynomials), are the reciprocals of the eigenvalues of an orthogonal section of the matrix (C A)−1. When C A is selfadjoint positive definite, this distinction is minimal, but for the indefinite or nonselfadjoint case this distinction becomes important. We use these two sets of roots to form possibly nonconvex regions in the complex plane that describe the spectrum of C A.

On the roots of the orthogonal polynomials and residual polynomials associated with a conjugate gradient method

We use Fourier analysis to examine multigrid convergence for a variety of discretizations of the two-dimensional convection-diffusion equation with grid-aligned flow velocity. Emphasis is placed on the problem with a small diffusion coefficient. We consider three distinct discretizations of the problem: centered differencing, standard upwind differencing, and a generalization to two dimensions of the El-Mistikawy and Werle discretization (EMW). The analysis covers the effectiveness of block-Jacobi relaxation for the problem and stability (as the diffusion coefficient $\epsilon \to 0$) of the discretizations. In stark contrast to the problem with general (non-grid-aligned) flow, we obtain V -cycle rates independent of meshwidth h for central differencing, as well as for the other discretizations. A striking similarity is found between tandard upwinding and the second-order accurate generalization of EMW. The latter is also found to be superior to central differencing in terms of stability. Superior multigr...

James S. Otto

Papers

A comparison of adaptive Chebyshev and least squares polynomial preconditioning for Hermitian positive definite linear systems

Optimal equivalent preconditioners

Paper: Bluestein's FFT for arbitrary N on the hypercube

On the roots of the orthogonal polynomials and residual polynomials associated with a conjugate gradient method

Multigrid convergence for discretizations of singular perturbation problems with grid-aligned flow