Showing papers on "Conjugate gradient method published in 1996"
TL;DR: A detailed description and comparison of algorithms for performing ab-initio quantum-mechanical calculations using pseudopotentials and a plane-wave basis set is presented in this article. But this is not a comparison of our algorithm with the one presented in this paper.
47,666 citations
TL;DR: In this paper, a moving-particle semi-implicit (MPS) method for simulating fragmentation of incompressible fluids is presented, where the motion of each particle is calculated through interactions with neighboring particles covered with the kernel function.
Abstract: A moving-particle semi-implicit (MPS) method for simulating fragmentation of incompressible fluids is presented. The motion of each particle is calculated through interactions with neighboring particles covered with the kernel function. Deterministic particle interaction models representing gradient, Laplacian, and free surfaces are proposed. Fluid density is implicitly required to be constant as the incompressibility condition, while the other terms are explicitly calculated. The Poisson equation of pressure is solved by the incomplete Cholesky conjugate gradient method. Collapse of a water column is calculated using MPS. The effect of parameters in the models is investigated in test calculations. Good agreement with an experiment is obtained even if fragmentation and coalescence of the fluid take place.
1,653 citations
01 Oct 1996
TL;DR: In this paper, the authors proposed an explicit discretization of the elastic wave mechanism, which allows the elastic-viscous-plastic model to capture the ice response to variations in the imposed stress more accurately.
Abstract: The standard model for sea ice dynamics treats the ice pack as a viscous-plastic material that flows plastically under typical stress conditions but behaves as a linear viscous fluid where strain rates are small and the ice becomes nearly rigid. Because of large viscosities in these regions, implicit numerical methods are necessary for timesteps larger than a few seconds. Current solution methods for these equations use iterative relaxation methods, which are time consuming, scale poorly with mesh resolution, and are not well adapted to parallel computation. To remedy this, we have developed and tested two separate methods. First, by demonstrating that the viscous-plastic rheology can be represented by a symmetric, negative definite matrix operator, we have implemented the faster and better behaved preconditioned conjugate gradient method. Second, realizing that only the response of the ice on time scales associated with wind forcing need be accurately resolved, we have modified the model to reduce to the viscous-plastic model at these time scales; at shorter time scales the adjustment process takes place by a numerically efficient elastic wave mechanism. This modification leads to a fully explicit numerical scheme which further improves the computational efficiency and is an advantage for implementations on parallel machines. Furthermore, we observe that the standard viscous-plastic model has poor dynamic response to forcing on a daily time scale, given the standard time step (1 day) used by the ice modeling community. In contrast, the explicit discretization of the elastic wave mechanism allows the elastic-viscous-plastic model to capture the ice response to variations in the imposed stress more accurately. Thus, the elastic-viscous-plastic model provides more accurate results for shorter time scales associated with physical forcing, reproduces viscous-plastic model behavior on longer time scales, and is computationally more efficient. 49 refs., 13 figs., 6 tabs.
912 citations
TL;DR: Some of the latest developments in using preconditioned conjugate gradient methods for solving Toeplitz systems are surveyed, finding that the complexity of solving a large class of $n-by-n$ ToePlitz systems is reduced to $O(n \log n)$ operations.
Abstract: In this expository paper, we survey some of the latest developments in using preconditioned conjugate gradient methods for solving Toeplitz systems. One of the main results is that the complexity of solving a large class of $n$-by-$n$ Toeplitz systems is reduced to $O(n \log n)$ operations as compared to $O(n \log ^2 n)$ operations required by fast direct Toeplitz solvers. Different preconditioners proposed for Toeplitz systems are reviewed. Applications to Toeplitz-related systems arising from partial differential equations, queueing networks, signal and image processing, integral equations, and time series analysis are given.
780 citations
TL;DR: It is proved that in exact arithmetic the preconditioner is well defined if $A$ is an H-matrix and the resulting factorized sparse approximate inverse is used as an explicit preconditionser for conjugate gradient calculations.
Abstract: A method for computing a sparse incomplete factorization of the inverse of a symmetric positive definite matrix $A$ is developed, and the resulting factorized sparse approximate inverse is used as an explicit preconditioner for conjugate gradient calculations. It is proved that in exact arithmetic the preconditioner is well defined if $A$ is an H-matrix. The results of numerical experiments are presented.
402 citations
TL;DR: The numerical simulation of groundwater flow through heterogeneous porous media is discussed, with a focus on the performance of a parallel multigrid preconditioner for accelerating convergence of conjugate gradients, which is used to compute the pressure head.
Abstract: The numerical simulation of groundwater flow through heterogeneous porous media is discussed. The focus is on the performance of a parallel multigrid preconditioner for accelerating convergence of conjugate gradients, which is used to compute the pressure head. The numerical investigation considers the effects of boundary conditions, coarse grid solver strategy, increasing the grid resolution, enlarging the domain, and varying the geostatistical parameters used to define the subsurface realization. Scalability is also examined. The results were obtained using the PARFLOW groundwater flow simulator on the CRAY T3D massively parallel computer.
395 citations
TL;DR: A well-known space-adaptive regularization method for image restoration is extended, which effectively utilizes, among others, the piecewise smoothness of both the image and the PSF to solve the scale problem inherent to the cost function.
Abstract: The primary difficulty with blind image restoration, or joint blur identification and image restoration, is insufficient information. This calls for proper incorporation of a priori knowledge about the image and the point-spread function (PSF). A well-known space-adaptive regularization method for image restoration is extended to address this problem. This new method effectively utilizes, among others, the piecewise smoothness of both the image and the PSF. It attempts to minimize a cost function consisting of a restoration error measure and two regularization terms (one for the image and the other for the blur) subject to other hard constraints. A scale problem inherent to the cost function is identified, which, if not properly treated, may hinder the minimization/blind restoration process. Alternating minimization is proposed to solve this problem so that algorithmic efficiency as well as simplicity is significantly increased. Two implementations of alternating minimization based on steepest descent and conjugate gradient methods are presented. Good performance is observed with numerically and photographically blurred images, even though no stringent assumptions about the structure of the underlying blur operator is made.
391 citations
TL;DR: In this article, two efficient solution procedures are presented that dramatically reduce the amount of computations involved in numerically solving these problems and demonstrate that the linear algebraic systems from the underlying SSFEM formulation can be solved with considerably less effort in memory and computation time than their size suggests.
308 citations
TL;DR: This paper presents a new method for weighted least squares phase unwrapping that is a multigrid technique that solves the equations on smaller, coarser grids by means of Gauss-Seidel relaxation schemes and transfers the intermediate results to the finer grids.
Abstract: Weighted least squares phase unwrapping is a robust approach to phase unwrapping that unwraps around (rather than through) regions of corrupted phase. Currently, the only practical method for solving the weighted least squares equations is a preconditioned conjugate gradient (PCG) technique. In this paper the authors present a new method for weighted least squares phase unwrapping. Their method is a multigrid technique that solves the equations on smaller, coarser grids by means of Gauss-Seidel relaxation schemes and transfers the intermediate results to the finer grids. A key idea of their approach is to maintain the partial derivatives of the given phase data in separate arrays and to correct these derivatives at the boundaries of the coarser grids. This correction maintains the boundary conditions necessary for convergence to the correct solution. Another key idea of their approach is to transfer the weighting values to the coarser grids in a carefully defined manner. They also present methods for defining the initial phase weights in an automated fashion. The resulting multigrid algorithm converges in only one or two multigrid cycles and is generally 15-25 times faster than the PCG technique.
242 citations
TL;DR: An adaptive control strategy for the number of iterations on successive refinement levels for possibly highly non-uniform grids is worked out on the basis of a posteriori estimates and numerical tests confirm the efficiency and robustness of the cascadic multigrid method.
Abstract: The paper deals with certain adaptive multilevel methods at the confluence of nested multigrid methods and iterative methods based on the cascade principle of [10]. From the multigrid point of view, no correction cycles are needed; from the cascade principle view, a basic iteration method without any preconditioner is used at successive refinement levels. For a prescribed error tolerance on the final level, more iterations must be spent on coarser grids in order to allow for less iterations on finer grids. A first candidate of such a cascadic multigrid method was the recently suggested cascadic conjugate gradient method of [9], in short CCG method, whichused the CG method as basic iteration method on each level. In [18] it has been proven, that the CCG method is accurate with optimal complexity for elliptic problems in 2D and quasi-uniform triangulations. The present paper simplifies that theory and extends it to more general basic iteration methods like the traditional multigrid smoothers. Moreover, an adaptive control strategy for the number of iterations on successive refinement levels for possibly highly non-uniform grids is worked out on the basis of a posteriori estimates. Numerical tests confirm the efficiency and robustness of the cascadic multigrid method.
218 citations
TL;DR: In this paper, a staggered-grid, finite-difference approximation for electromagnetic induction in the Earth has been derived using the biconjugate gradient method, preconditioned with a modified, partial Cholesky decomposition of A. This method takes advantage of the sparsity of A, and converges much more quickly than methods used previously to solve the 3D induction problem.
Abstract: The preceding paper derives a staggered-grid, finite-difference approximation applicable to electromagnetic induction in the Earth. The staggered-grid, finite-difference approximation results in a linear system of equations Ax = b, where A is symmetric but not Hermitian. This is solved using the biconjugate gradient method, preconditioned with a modified, partial Cholesky decomposition of A. This method takes advantage of the sparsity of A, and converges much more quickly than methods used previously to solve the 3-D induction problem. When simulating a conductivity model at a number of frequencies, the rate of convergence slows as frequency approaches 0. The convergence rate at low frequencies can be improved by an order of magnitude, by alternating the incomplete Cholesky preconditioned biconjugate gradient method with a procedure designed to make the approximate solutions conserve current.
TL;DR: A reliable and efficient computational algorithm for restoring blurred and noisy images that can be used in an adaptive/interactive manner in situations when knowledge of the noise variance is either unavailable or unreliable is proposed.
Abstract: A reliable and efficient computational algorithm for restoring blurred and noisy images is proposed. The restoration process is based on the minimal total variation principle introduced by Rudin et al. For discrete images, the proposed algorithm minimizes a piecewise linear l/sub 1/ function (a measure of total variation) subject to a single 2-norm inequality constraint (a measure of data fit). The algorithm starts by finding a feasible point for the inequality constraint using a (partial) conjugate gradient method. This corresponds to a deblurring process. Noise and other artifacts are removed by a subsequent total variation minimization process. The use of the linear l/sub 1/ objective function for the total variation measurement leads to a simpler computational algorithm. Both the steepest descent and an affine scaling Newton method are considered to solve this constrained piecewise linear l/sub 1/ minimization problem. The resulting algorithm, when viewed as an image restoration and enhancement process, has the feature that it can be used in an adaptive/interactive manner in situations when knowledge of the noise variance is either unavailable or unreliable. Numerical examples are presented to demonstrate the effectiveness of the proposed iterative image restoration and enhancement process.
TL;DR: A formula for the computation of the gradient of the energy with respect to the self-consistent potential is proposed, in order to obtain an efficient preconditioning operator for order-N algorithms.
Abstract: The determination of the total energy within density-functional theory can be formulated as a minimization problem,in a space of trial self-consistent potentials. In order to apply a conjugate-gradient algorithm to this problem, a formula for the computation of the gradient of the energy with respect to the self-consistent potential is proposed. The second derivative of the energy with respect to potential changes is also analyzed, in order to obtain an efficient preconditioning operator. The wave functions do not appear explicitly in this approach, so that order-N algorithms could take advantage of it. The results of preliminary tests are reported.
TL;DR: An overview of various finite element techniques based on the magnetic vector potential for the solution of three-dimensional magnetostatic problems is presented in this paper, where the vector potential is interpolated with the aid of edge finite elements and no gauge is enforced.
Abstract: An overview of various finite element techniques based on the magnetic vector potential for the solution of three-dimensional magnetostatic problems is presented. If nodal finite elements are used for the approximation of the vector potential, a lack of gauging results in an ill-conditioned system. The implicit enforcement of the Coulomb gauge dramatically improves the numerical stability, but the normal component of the vector potential must be allowed to be discontinuous on iron/air interfaces. If the vector potential is is interpolated with the aid of edge finite elements and no gauge is enforced, a singular system results. It can be solved efficiently by conjugate gradient methods, provided care is taken to ensure that the current density is divergence free. Finally, if a tree-cotree gauging of the vector potential is introduced, the numerical stability depends on how the tree is selected with no obvious optimal choice available.
TL;DR: In this paper, the weighted integral method and the Monte Carlo simulation are used together with innovative solution strategies based on the Preconditioned Conjugate Gradient method (PCG) to produce robust and efficient solutions for the stochastic finite element analysis of space frames.
TL;DR: In this article, the Dirac operator with Wilson fermions in four-dimensional SU(2) gauge fields was studied and the conjugate gradient method was combined with exact diagonalization in the subspace spanned by numerically computed eigenvectors.
TL;DR: An aerodynamic shape optimization procedure based on discrete sensitivity analysis is extended to treat three-dimensional geometries and results in a significant factor of 50 decrease in computational time and a factor of eight reduction in memory over the most efficient design strategies in current use.
Abstract: An aerodynamic shape optimization procedure based on discrete sensitivity analysis is extended to treat three-dimensional geometries. The function of sensitivity analysis is to directly couple computational fluid dynamics (CFD) with numerical optimization techniques, which facilitates the construction of efficient direct-design methods. The development of a practical three-dimensional design procedures entails many challenges, such as: (1) the demand for significant efficiency improvements over current design methods; (2) a general and flexible three-dimensional surface representation; and (3) the efficient solution of very large systems of linear algebraic equations. It is demonstrated that each of these challenges is overcome by: (1) employing fully implicit (Newton) methods for the CFD analyses; (2) adopting a Bezier-Bernstein polynomial parameterization of two- and three-dimensional surfaces; and (3) using preconditioned conjugate gradient-like linear system solvers. Whereas each of these extensions independently yields an improvement in computational efficiency, the combined effect of implementing all the extensions simultaneously results in a significant factor of 50 decrease in computational time and a factor of eight reduction in memory over the most efficient design strategies in current use. The new aerodynamic shape optimization procedure is demonstrated in the design of both two- and three-dimensional inviscid aerodynamic problems including a two-dimensional supersonic internal/external nozzle, two-dimensional transonic airfoils (resulting in supercritical shapes), three-dimensional transport wings, and three-dimensional supersonic delta wings. Each design application results in realistic and useful optimized shapes.
TL;DR: In this paper, the authors investigated the global convergence properties of the FR method for unconstrained optimization and proved that a kind of inexact line search condition can ensure the convergence of the method.
Abstract: This paper investigates the global convergence properties of the Fletcher-Reeves (FR) method for unconstrained optimization. In a simple way, we prove that a kind of inexact line search condition can ensure the convergence of the FR method. Several examples are constructed to show that, if the search conditions are relaxed, the FR method may produce an ascent search direction, which implies that our result cannot be improved.
TL;DR: In this article, a self-adjoint convolution operator is constructed for deconvolution from a given noise-contaminated observation, and a symmetric indefinite conjugate gradient technique is used to avoid a breakdown of the iteration.
Abstract: We consider an ill-posed deconvolution problem from astronomical imaging with a given noise-contaminated observation, and an approximately known convolution kernel. The limitations of the mathematical model and the shape of the kernel function motivate and legitimate a further approximation of the convolution operator by one that is self-adjoint. This simplifies the reconstruction problem substantially because the efficient conjugate gradient method can now be used for an iterative computation of a (regularized) approximation of the true unblurred image. Since the constructed self-adjoint operator fails to be positive definite, a symmetric indefinite conjugate gradient technique, called MR-II is used to avoid a breakdown of the iteration. We illustrate how the L-curve method can be used to stop the iterations, and suggest a preconditioner for further reducing the computations.
TL;DR: In this article, a solenoidal superconducting magnetic energy storage with active and passive shielding has been optimized by means of different optimization procedures based on the global search algorithm, evolution strategies, simulated annealing and conjugate gradient method, all coupled to integral or finite element codes.
Abstract: A proposal for benchmark problems to test electromagnetic optimization methods, relevant to multiobjective optimization of a solenoidal superconducting magnetic energy storage with active and passive shielding is presented. The system has been optimized by means of different optimization procedures based on the global search algorithm, evolution strategies, simulated annealing and the conjugate gradient method, all coupled to integral or finite element codes. A comparison of results is performed and the features of the problem as a test of optimization procedures are discussed.
TL;DR: Solving deconvolution problems for atmospherically blurred images by the preconditionsed conjugate gradient algorithm, where a new approximate inverse preconditioner is used to increase the rate of convergence.
Abstract: Removing a linear shift-invariant blur from a signal or image can be accomplished by inverse or Wiener filtering, or by an iterative least-squares deblurring procedure. Because of the ill-posed characteristics of the deconvolution problem, in the presence of noise, filtering methods often yield poor results. On the other hand, iterative methods often suffer from slow convergence at high spatial frequencies. This paper concerns solving deconvolution problems for atmospherically blurred images by the preconditioned conjugate gradient algorithm, where a new approximate inverse preconditioner is used to increase the rate of convergence. Theoretical results are established to show that fast convergence can be expected, and test results are reported for a ground-based astronomical imaging problem.
TL;DR: A series-expansion approach and an operator framework are used to derive a new, fast, and accurate Fourier algorithm for iterative tomographic reconstruction that is applicable for parallel-ray projections collected at a finite number of arbitrary view angles and radially sampled at a rate high enough that aliasing errors are small.
Abstract: We use a series-expansion approach and an operator framework to derive a new, fast, and accurate Fourier algorithm for iterative tomographic reconstruction. This algorithm is applicable for parallel-ray projections collected at a finite number of arbitrary view angles and radially sampled at a rate high enough that aliasing errors are small. The conjugate gradient (CG) algorithm is used to minimize a regularized, spectrally weighted least-squares criterion, and we prove that the main step in each iteration is equivalent to a 2-D discrete convolution, which can be cheaply and exactly implemented via the fast Fourier transform (FFT). The proposed algorithm requires O(N/sup 2/logN) floating-point operations per iteration to reconstruct an N/spl times/N image from P view angles, as compared to O(N/sup 2/P) floating-point operations per iteration for iterative convolution-backprojection algorithms or general algebraic algorithms that are based on a matrix formulation of the tomography problem. Numerical examples using simulated data demonstrate the effectiveness of the algorithm for sparse- and limited-angle tomography under realistic sampling scenarios. Although the proposed algorithm cannot explicitly account for noise with nonstationary statistics, additional simulations demonstrate that for low to moderate levels of nonstationary noise, the quality of reconstruction is almost unaffected by assuming that the noise is stationary.
TL;DR: A cascadic conjugate-gradient method (shortly called the CCG-algorithm) which was proposed by P. Deuflhard and can be considered as a simpler version of a multigrid (multilevel) method for discrete self-adjoint positive-definite problems on a sequence of grids.
Abstract: The paper deals with a cascadic conjugate-gradient method (shortly called the CCG-algorithm) which was proposed by P. Deuflhard and can be considered as a simpler version of a multigrid (multilevel) method. We define it recurrently for discrete self-adjoint positive-definite problems on a sequence of grids. On the coarsest grid, the linear discrete algebraic system is solved directly. On the finer grids, the system is iteratively solved by the conjugate-gradient method where the starting guess is an interpolation of the approximated solution on the previous grid. Any preconditioning or restriction to coarser grids is not implemented. Nevertheless, the CCG-algorithm has the same optimal property compared to multigrid methods; namely, the algorithm converges with a rate which is independent of the number of unknowns and the number of grids. As an example, this property is proved for elliptic second order Dirichlet problems in two-dimensional, convex, polygonal bounded domains. For ensuring convergence, the number of iterations on each grid level has to increase from finer to coarser grids. The optimal dependence of these numbers is established with respect to the mesh size and the number of unknowns. The theory has been presented in an abstract setting which allows the application to both finite element and finite difference methods.
TL;DR: In this paper, an advanced A-V method employing edge-based finite elements for the vector potential A and nodal shape functions for the scalar potential V is proposed, which is particularly well suited for efficient iterative solvers such as the preconditioned conjugate gradient method.
Abstract: An advanced A-V method employing edge-based finite elements for the vector potential A and nodal shape functions for the scalar potential V is proposed. Both gauged and ungauged formulations are considered. The novel scheme is particularly well suited for efficient iterative solvers such as the preconditioned conjugate gradient method, since it leads to significantly faster numerical convergence rates than pure edge element schemes. In contrast to nodal finite element implementations, spurious solutions do not occur and the inherent singularities of the electromagnetic fields in the vicinity of perfectly conducting edges and corners are handled correctly. Several numerical examples are presented to verify the suggested approach.
Book•
01 Aug 1996
TL;DR: This text on linear and non-linear conjugate gradient-related methods includes such topics as: conjugated gradients and related KMP algorithms; conjugates methods and nonlinear optimization; and conjugATE gradient-like methods for Eigen-like problems.
Abstract: This text on linear and non-linear conjugate gradient-related methods includes such topics as: conjugate gradients and related KMP algorithms; conjugate gradient methods and nonlinear optimization; and conjugate gradient-like methods for Eigen-like problems.
TL;DR: In this article, an iterative-reconstruction algorithm based on a conjugate-gradient method was proposed to reconstruct the complex permittivity profile of lossy dielectric objects from measured far-lied data.
Abstract: The reconstruct of the complex permittivity profile of lossy dielectric objects from measured far-lied data is considered, with application to perfectly conducting (PEC) objects. From an integral representation of the electric field (EFIE), and applying a moment-method solution, an iterative-reconstruction algorithm based on a conjugate-gradient method, is derived. In order to start the iterative procedure with an initial guess, a back-propagation scheme is used. For testing the algorithm on real, measured data, the reconstruction of two PEC (cylinder and strip) objects is presented.
TL;DR: This paper uses the feedforward network as a parametric representation of the object surface and reformulate the shape from shading problem as the minimization of an error function over the network weights using the stochastic gradient and conjugate gradient methods.
Abstract: The multilayer feedforward network has often been used for learning a nonlinear mapping based on a set of examples of the input-output data. In this paper, we present a novel use of the network, in which the example data are not explicitly given. We consider the problem of shape from shading in computer vision, where the input (image coordinates) and the output (surface depth) satisfy only a known differential equation. We use the feedforward network as a parametric representation of the object surface and reformulate the shape from shading problem as the minimization of an error function over the network weights. The stochastic gradient and conjugate gradient methods are used for the minimization. Boundary conditions for either surface depth or surface normal (or both) can be imposed by adjusting the same network at different levels. It is further shown that the light source direction can be estimated, based on an initial guess, by integrating the source estimation with the surface estimation. Extensions of the method to a wider class of problems are discussed. The efficiency of the method is verified by examples of both synthetic and real images.
TL;DR: In this article, a 3D inversion of the linear system of equations is inverted using a preconditioned conjugate gradient approach to reduce memory requirements and avoid large matrix multiplications.
Abstract: A 3-D inversion approach is outlined that determines a distribution of susceptibility that produces a given magnetic anomaly. The subsurface model consists of a 3-D array of rectangular blocks, each with a constant susceptibility. The inversion incorporates a model norm that allows smoothing and depth-weighting of the solution. Since the number of parameters can be many thousands, even for small problems, the linear system of equations is inverted using a preconditioned conjugate gradient approach. This reduces memory requirements and avoids large matrix multiplications. The method is used to determine the 3-D susceptibility distribution responsible for the Temagami magnetic anomaly in southern Ontario, Canada.
TL;DR: It is shown that the appropriately trained neural network can be used for the nonconventional solution of these standard signal processing tasks with satisfactory accuracy.