The Geometry of Algorithms with Orthogonality Constraints
01 Apr 1999-SIAM Journal on Matrix Analysis and Applications (Society for Industrial and Applied Mathematics)-Vol. 20, Iss: 2, pp 303-353
TL;DR: The theory proposed here provides a taxonomy for numerical linear algebra algorithms that provide a top level mathematical view of previously unrelated algorithms and developers of new algorithms and perturbation theories will benefit from the theory.
Abstract: In this paper we develop new Newton and conjugate gradient algorithms on the Grassmann and Stiefel manifolds. These manifolds represent the constraints that arise in such areas as the symmetric eigenvalue problem, nonlinear eigenvalue problems, electronic structures computations, and signal processing. In addition to the new algorithms, we show how the geometrical framework gives penetrating new insights allowing us to create, understand, and compare algorithms. The theory proposed here provides a taxonomy for numerical linear algebra algorithms that provide a top level mathematical view of previously unrelated algorithms. It is our hope that developers of new algorithms and perturbation theories will benefit from the theory, methods, and examples in this paper.
Citations
More filters
TL;DR: A fully-vectorial, three-dimensional algorithm to compute the definite-frequency eigenstates of Maxwell's equations in arbitrary periodic dielectric structures, including systems with anisotropy or magnetic materials, using preconditioned block-iterative eigensolvers in a planewave basis is described.
Abstract: We describe a fully-vectorial, three-dimensional algorithm to compute the definite-frequency eigenstates of Maxwell's equations in arbitrary periodic dielectric structures, including systems with anisotropy (birefringence) or magnetic materials, using preconditioned block-iterative eigensolvers in a planewave basis. Favorable scaling with the system size and the number of computed bands is exhibited. We propose a new effective dielectric tensor for anisotropic structures, and demonstrate that O Delta x;2 convergence can be achieved even in systems with sharp material discontinuities. We show how it is possible to solve for interior eigenvalues, such as localized defect modes, without computing the many underlying eigenstates. Preconditioned conjugate-gradient Rayleigh-quotient minimization is compared with the Davidson method for eigensolution, and a number of iteration variants and preconditioners are characterized. Our implementation is freely available on the Web.
2,861 citations
Book•
23 Dec 2007TL;DR: Optimization Algorithms on Matrix Manifolds offers techniques with broad applications in linear algebra, signal processing, data mining, computer vision, and statistical analysis and will be of interest to applied mathematicians, engineers, and computer scientists.
Abstract: Many problems in the sciences and engineering can be rephrased as optimization problems on matrix search spaces endowed with a so-called manifold structure. This book shows how to exploit the special structure of such problems to develop efficient numerical algorithms. It places careful emphasis on both the numerical formulation of the algorithm and its differential geometric abstraction--illustrating how good algorithms draw equally from the insights of differential geometry, optimization, and numerical analysis. Two more theoretical chapters provide readers with the background in differential geometry necessary to algorithmic development. In the other chapters, several well-known optimization methods such as steepest descent and conjugate gradients are generalized to abstract manifolds. The book provides a generic development of each of these methods, building upon the material of the geometric chapters. It then guides readers through the calculations that turn these geometrically formulated methods into concrete numerical algorithms. The state-of-the-art algorithms given as examples are competitive with the best existing algorithms for a selection of eigenspace problems in numerical linear algebra. Optimization Algorithms on Matrix Manifolds offers techniques with broad applications in linear algebra, signal processing, data mining, computer vision, and statistical analysis. It can serve as a graduate-level textbook and will be of interest to applied mathematicians, engineers, and computer scientists.
2,586 citations
Book•
01 Jan 20002,534 citations
TL;DR: Specific developments discussed include fast methods for density functional theory calculations, linear scaling evaluation of energies, NMR chemical shifts and electric properties, fast auxiliary basis function methods for correlated energies and gradients, equation-of-motion coupled cluster methods for ground and excited states, geminal wavefunctions, embedding methods and techniques for exploring potential energy surfaces.
Abstract: Advances in theory and algorithms for electronic structure calculations must be incorporated into program packages to enable them to become routinely used by the broader chemical community. This work reviews advances made over the past five years or so that constitute the major improvements contained in a new release of the Q-Chem quantum chemistry package, together with illustrative timings and applications. Specific developments discussed include fast methods for density functional theory calculations, linear scaling evaluation of energies, NMR chemical shifts and electric properties, fast auxiliary basis function methods for correlated energies and gradients, equation-of-motion coupled cluster methods for ground and excited states, geminal wavefunctions, embedding methods and techniques for exploring potential energy surfaces.
2,527 citations
TL;DR: Applications of image registration include combining images of the same subject from different modalities, aligning temporal sequences of images to compensate for motion of the subject between scans, image guidance during interventions and aligning images from multiple subjects in cohort studies.
Abstract: Radiological images are increasingly being used in healthcare and medical research. There is, consequently, widespread interest in accurately relating information in the different images for diagnosis, treatment and basic science. This article reviews registration techniques used to solve this problem, and describes the wide variety of applications to which these techniques are applied. Applications of image registration include combining images of the same subject from different modalities, aligning temporal sequences of images to compensate for motion of the subject between scans, image guidance during interventions and aligning images from multiple subjects in cohort studies. Current registration algorithms can, in many cases, automatically register images that are related by a rigid body transformation (i.e. where tissue deformation can be ignored). There has also been substantial progress in non-rigid registration algorithms that can compensate for tissue deformation, or align images from different subjects. Nevertheless many registration problems remain unsolved, and this is likely to continue to be an active field of research in the future.
2,166 citations
Cites methods from "The Geometry of Algorithms with Ort..."
...The ratio image uniformity (RIU) algorithm was originally introduced by Woods et al (1992) for the registration of serial PET studies, but has more recently been widely used for serial MR registration (Woodset al 1998), and is available in the AIR registration package from UCLA....
[...]
References
More filters
TL;DR: In this paper, the Hartree and Hartree-Fock equations are applied to a uniform electron gas, where the exchange and correlation portions of the chemical potential of the gas are used as additional effective potentials.
Abstract: From a theory of Hohenberg and Kohn, approximation methods for treating an inhomogeneous system of interacting electrons are developed. These methods are exact for systems of slowly varying or high density. For the ground state, they lead to self-consistent equations analogous to the Hartree and Hartree-Fock equations, respectively. In these equations the exchange and correlation portions of the chemical potential of a uniform electron gas appear as additional effective potentials. (The exchange portion of our effective potential differs from that due to Slater by a factor of $\frac{2}{3}$.) Electronic systems at finite temperatures and in magnetic fields are also treated by similar methods. An appendix deals with a further correction for systems with short-wavelength density oscillations.
47,477 citations
Additional excerpts
...40 Edelman, Arias, and Smith The fundamental theorems which make the ab initio approach tractable come from the density functional theory of Hohenberg and Kohn [50] and Kohn and Sham [55]....
[...]
...The fundamental theorems which make the ab initio approach tractable come from the density functional theory of Hohenberg and Kohn [50] and Kohn and Sham [55]....
[...]
...The Kohn–Sham minimization then becomes E0 = min XTX=IN E(X)(4.9) ≡ min XTX=IN tr(XTHX) + f ρ(X) , where each column ofX is a different electronic orbital sampled on the spatial grid, ρ is the vector ρi(X) ≡ n |Xin|2, H is an M -by-M matrix (single-particle Hamiltonian), and f is a function which we leave unspecified in this discussion....
[...]
TL;DR: In this article, the ground state of an interacting electron gas in an external potential was investigated and it was proved that there exists a universal functional of the density, called F[n(mathrm{r})], independent of the potential of the electron gas.
Abstract: This paper deals with the ground state of an interacting electron gas in an external potential $v(\mathrm{r})$. It is proved that there exists a universal functional of the density, $F[n(\mathrm{r})]$, independent of $v(\mathrm{r})$, such that the expression $E\ensuremath{\equiv}\ensuremath{\int}v(\mathrm{r})n(\mathrm{r})d\mathrm{r}+F[n(\mathrm{r})]$ has as its minimum value the correct ground-state energy associated with $v(\mathrm{r})$. The functional $F[n(\mathrm{r})]$ is then discussed for two situations: (1) $n(\mathrm{r})={n}_{0}+\stackrel{\ifmmode \tilde{}\else \~{}\fi{}}{n}(\mathrm{r})$, $\frac{\stackrel{\ifmmode \tilde{}\else \~{}\fi{}}{n}}{{n}_{0}}\ensuremath{\ll}1$, and (2) $n(\mathrm{r})=\ensuremath{\phi}(\frac{\mathrm{r}}{{r}_{0}})$ with $\ensuremath{\phi}$ arbitrary and ${r}_{0}\ensuremath{\rightarrow}\ensuremath{\infty}$. In both cases $F$ can be expressed entirely in terms of the correlation energy and linear and higher order electronic polarizabilities of a uniform electron gas. This approach also sheds some light on generalized Thomas-Fermi methods and their limitations. Some new extensions of these methods are presented.
38,160 citations
17 Jun 1964
28,969 citations
"The Geometry of Algorithms with Ort..." refers methods in this paper
...The fundamental theorems which make the ab initio approach tractable come from the density functional theory of Hohenberg and Kohn [50] and Kohn and Sham [55]....
[...]
...The Kohn–Sham minimization then becomes E0 = min XTX=IN E(X)(4.9) ≡ min XTX=IN tr(XTHX) + f ρ(X) , where each column ofX is a different electronic orbital sampled on the spatial grid, ρ is the vector ρi(X) ≡ n |Xin|2, H is an M -by-M matrix (single-particle Hamiltonian), and f is a function which we leave unspecified in this discussion....
[...]
TL;DR: In this article, a description of the multiple signal classification (MUSIC) algorithm, which provides asymptotically unbiased estimates of 1) number of incident wavefronts present; 2) directions of arrival (DOA) (or emitter locations); 3) strengths and cross correlations among the incident waveforms; 4) noise/interference strength.
Abstract: Processing the signals received on an array of sensors for the location of the emitter is of great enough interest to have been treated under many special case assumptions. The general problem considers sensors with arbitrary locations and arbitrary directional characteristics (gain/phase/polarization) in a noise/interference environment of arbitrary covariance matrix. This report is concerned first with the multiple emitter aspect of this problem and second with the generality of solution. A description is given of the multiple signal classification (MUSIC) algorithm, which provides asymptotically unbiased estimates of 1) number of incident wavefronts present; 2) directions of arrival (DOA) (or emitter locations); 3) strengths and cross correlations among the incident waveforms; 4) noise/interference strength. Examples and comparisons with methods based on maximum likelihood (ML) and maximum entropy (ME), as well as conventional beamforming are included. An example of its use as a multiple frequency estimator operating on time series is included.
12,446 citations