Showing papers on "Conjugate gradient method published in 2001"

Toward the Optimal Preconditioned Eigensolver: Locally Optimal Block Preconditioned Conjugate Gradient Method

Abstract: We describe new algorithms of the locally optimal block preconditioned conjugate gradient (LOBPCG) method for symmetric eigenvalue problems, based on a local optimization of a three-term recurrence, and suggest several other new methods. To be able to compare numerically different methods in the class, with different preconditioners, we propose a common system of model tests, using random preconditioners and initial guesses. As the "ideal" control algorithm, we advocate the standard preconditioned conjugate gradient method for finding an eigenvector as an element of the null-space of the corresponding homogeneous system of linear equations under the assumption that the eigenvalue is known. We recommend that every new preconditioned eigensolver be compared with this "ideal" algorithm on our model test problems in terms of the speed of convergence, costs of every iteration, and memory requirements. We provide such comparison for our LOBPCG method. Numerical results establish that our algorithm is practically as efficient as the ``ideal'' algorithm when the same preconditioner is used in both methods. We also show numerically that the LOBPCG method provides approximations to first eigenpairs of about the same quality as those by the much more expensive global optimization method on the same generalized block Krylov subspace. We propose a new version of block Davidson's method as a generalization of the LOBPCG method. Finally, direct numerical comparisons with the Jacobi--Davidson method show that our method is more robust and converges almost two times faster.

A computationally efficient superresolution image reconstruction algorithm

Abstract: Superresolution reconstruction produces a high-resolution image from a set of low-resolution images. Previous iterative methods for superresolution had not adequately addressed the computational and numerical issues for this ill-conditioned and typically underdetermined large scale problem. We propose efficient block circulant preconditioners for solving the Tikhonov-regularized superresolution problem by the conjugate gradient method. We also extend to underdetermined systems the derivation of the generalized cross-validation method for automatic calculation of regularization parameters. The effectiveness of our preconditioners and regularization techniques is demonstrated with superresolution results for a simulated sequence and a forward looking infrared (FLIR) camera image sequence.

New Conjugacy Conditions and Related Nonlinear Conjugate Gradient Methods

Abstract: Conjugate gradient methods are a class of important methods for unconstrained optimization, especially when the dimension is large. This paper proposes a new conjugacy condition, which considers an inexact line search scheme but reduces to the old one if the line search is exact. Based on the new conjugacy condition, two nonlinear conjugate gradient methods are constructed. Convergence analysis for the two methods is provided. Our numerical results show that one of the methods is very efficient for the given test problems.

A Spectral Conjugate Gradient Method for Unconstrained Optimization

Abstract: A family of scaled conjugate gradient algorithms for large-scale unconstrained minimization is defined. The Perry, the Polak—Ribiere and the Fletcher—Reeves formulae are compared using a spectral scaling derived from Raydan's spectral gradient optimization method. The best combination of formula, scaling and initial choice of step-length is compared against well known algorithms using a classical set of problems. An additional comparison involving an ill-conditioned estimation problem in Optics is presented.

Regularization methods for near-field acoustical holography

Abstract: The reconstruction of the pressure and normal surface velocity provided by near-field acoustical holography (NAH) from pressure measurements made near a vibrating structure is a linear, ill-posed inverse problem due to the existence of strongly decaying, evanescentlike waves. Regularization provides a technique of overcoming the ill-posedness and generates a solution to the linear problem in an automated way. We present four robust methods for regularization; the standard Tikhonov procedure along with a novel improved version, Landweber iteration, and the conjugate gradient approach. Each of these approaches can be applied to all forms of interior or exterior NAH problems; planar, cylindrical, spherical, and conformal. We also study two parameter selection procedures, the Morozov discrepancy principle and the generalized cross validation, which are crucial to any regularization theory. In particular, we concentrate here on planar and cylindrical holography. These forms of NAH which rely on the discrete Fourier transform are important due to their popularity and to their tremendous computational speed. In order to use regularization theory for the separable geometry problems we reformulate the equations of planar, cylindrical, and spherical NAH into an eigenvalue problem. The resulting eigenvalues and eigenvectors couple easily to regularization theory, which can be incorporated into the NAH software with little sacrifice in computational speed. The resulting complete automation of the NAH algorithm for both separable and nonseparable geometries overcomes the last significant hurdle for NAH.

On the Solution of Equality Constrained Quadratic Programming Problems Arising in Optimization

Abstract: We consider the application of the conjugate gradient method to the solution of large equality constrained quadratic programs arising in nonlinear optimization. Our approach is based implicitly on a reduced linear system and generates iterates in the null space of the constraints. Instead of computing a basis for this null space, we choose to work directly with the matrix of constraint gradients, computing projections into the null space by either a normal equations or an augmented system approach. Unfortunately, in practice such projections can result in significant rounding errors. We propose iterative refinement techniques, as well as an adaptive reformulation of the quadratic problem, that can greatly reduce these errors without incurring high computational overheads. Numerical results illustrating the efficacy of the proposed approaches are presented.

An Efficient Hybrid Conjugate Gradient Method for Unconstrained Optimization

Abstract: Recently, we propose a nonlinear conjugate gradient method, which produces a descent search direction at every iteration and converges globally provided that the line search satisfies the weak Wolfe conditions. In this paper, we will study methods related to the new nonlinear conjugate gradient method. Specifically, if the size of the scalar β k with respect to the one in the new method belongs to some interval, then the corresponding methods are proved to be globally convergent; otherwise, we are able to construct a convex quadratic example showing that the methods need not converge. Numerical experiments are made for two combinations of the new method and the Hestenes–Stiefel conjugate gradient method. The initial results show that, one of the hybrid methods is especially efficient for the given test problems.

Efficient large-scale power grid analysis based on preconditioned krylov-subspace iterative methods

Abstract: In this paper, we propose preconditioned Krylov-subspace iterative methods to perform efficient DC and transient simulations for large-scale linear circuits with an emphasis on power delivery circuits. We also prove that a circuit with inductors can be simplified from MNA to NA format, and the matrix becomes an s.p.d. matrix. This property makes it suitable for the conjugate gradient with incomplete Cholesky decomposition as the preconditioner, which is faster than other direct and iterative methods. Extensive experimental results on large-scale industrial power grid circuits show that our method is over 200 times faster for DC analysis and around 10 times faster for transient simulation compared to SPICE3. Furthermore, our algorithm reduces over 75% of memory usage than SPICE3 while the accuracy is not compromised.

A deterministic least-squares approach to space-time adaptive processing (STAP)

Abstract: A direct data domain (D/sup 3/) least-squares space-time adaptive processing (STAP) approach is presented for adaptively enhancing signals in a nonhomogeneous environment. The nonhomogeneous environment may consist of nonstationary clutter and could include blinking jammers. The D/sup 3/ approach is applied to data collected by an antenna array utilizing space and in time (Doppler) diversity. Conventional STAP generally utilizes statistical methodologies based on estimating a covariance matrix of the interference using data from secondary range cells. As the results are derived from ensemble averages, one filter (optimum in a probabilistic sense) is obtained for the operational environment, assumed to be wide sense stationary. However for highly transient and inhomogeneous environments the conventional statistical methodology is difficult to apply. Hence, the D/sup 3/ method is presented as it analyzes the data in space and time over each range cell separately. The D/sup 3/ method is deterministic in approach. From an operational standpoint, an optimum method could be a combination of these two diverse methodologies. This paper represents several new D/sup 3/ approaches. One is based on the computation of a generalized eigenvalue for the signal strength and the others are based on the solution of a set of block Hankel matrix equations. Since the matrix of the system of equations to be solved has a block Hankel structure, the conjugate gradient method and the fast Fourier transform (FFT) can be utilized for efficient solution of the adaptive problem. Illustrative examples presented in this paper use measured data from the multichannel airborne radar measurements (MCARM) database to detect a Sabreliner in the presence of urban, land, and sea clutter. An added advantage for the D/sup 3/ method in solving real-life problems is that simultaneously many realizations can be obtained for the same solution for the signal of interest (SOI). The degree of variability amongst the different results can provide a confidence level of the processed results. The D/sup 3/ method may also be used for mobile communications.

Inverse scattering of two-dimensional dielectric objects buried in a lossy earth using the distorted Born iterative method

Abstract: An efficient inverse-scattering algorithm is developed to reconstruct both the permittivity and conductivity profiles of two-dimensional (2D) dielectric objects buried in a lossy earth using the distorted Born iterative method (DBIM). In this algorithm, the measurement data are collected on (or over) the air-earth interface for multiple transmitter and receiver locations at single frequency. The nonlinearity due to the multiple scattering of pixels to pixels, and pixels to the air-earth interface has been taken into account in the iterative minimization scheme. At each iteration, a conjugate gradient (CG) method is chosen to solve the linearized problem, which takes the calling number of the forward solver to a minimum. To reduce the CPU time, the forward solver for buried dielectric objects is implemented by the CG method and fast Fourier transform (FFT). Numerous numerical examples are given to show the convergence, stability, and error tolerance of the algorithm.

On the Construction of Deflation-Based Preconditioners

Abstract: In this article we introduce new bounds on the effective condition number of deflated and preconditioned-deflated symmetric positive definite linear systems. For the case of a subdomain deflation such as that of Nicolaides [SIAM J. Numer. Anal., 24 (1987), pp. 355--365], these theorems can provide direction in choosing a proper decomposition into subdomains. If grid refinement is performed, keeping the subdomain grid resolution fixed, the condition number is insensitive to the grid size. Subdomain deflation is very easy to implement and has been parallelized on a distributed memory system with only a small amount of additional communication. Numerical experiments for a steady-state convection-diffusion problem are included.

Preconditioned all-at-once methods for large, sparse parameter estimation problems

Abstract: The problem of recovering a parameter function based on measurements of solutions of a system of partial differential equations in several space variables leads to a number of computational challenges. Upon discretization of a regularized formulation a large, sparse constrained optimization problem is obtained. Typically in the literature, the constraints are eliminated and the resulting unconstrained formulation is solved by some variant of Newton's method, usually the Gauss–Newton method. A preconditioned conjugate gradient algorithm is applied at each iteration for the resulting reduced Hessian system. Alternatively, in this paper we apply a preconditioned Krylov method directly to the KKT system arising from a Newton-type method for the constrained formulation (an 'all-at-once' approach). A variant of symmetric QMR is employed, and an effective preconditioner is obtained by solving the reduced Hessian system approximately. Since the reduced Hessian system presents significant expense already in forming a matrix–vector product, the savings in doing so only approximately are substantial. The resulting preconditioner may be viewed as an incomplete block-LU decomposition, and we obtain conditions guaranteeing bounds for the condition number of the preconditioned matrix. Numerical experiments are performed for the dc-resistivity and the magnetostatic problems in 3D, comparing the two approaches for solving the linear system at each Gauss–Newton iteration. A substantial efficiency gain is demonstrated. The relative efficiency of our proposed method is even higher in the context of inexact Newton-type methods, where the linear system at each iteration is solved less accurately.

Constrained texture mapping for polygonal meshes

Abstract: Recently, time and effort have been devoted to automatic texture mapping. It is possible to study the parameterization function and to describe the texture mapping process in terms of a functional optimization problem. Several methods of this type have been proposed to minimize deformations. However, these existing methods suffer from several limitations. For instance, it is difficult to put details of the texture in correspondence with features of the model, since most of the existing methods can only constrain iso-parametric curves.We introduce in this paper a new optimization-based method for parameterizing polygonal meshes with minimum deformations, while enabling the user to interactively define and edit a set of constraints. Each user-defined constraint consists of a relation linking a 3D point picked on the surface and a 2D point of the texture. Moreover, the non-deformation criterion introduced here can act as an extrapolator, thus making it unnecessary to constrain the border of the surface, in contrast with classic methods. To minimize the criterion, a conjugate gradient algorithm is combined with a compressed representation of sparse matrices, making it possible to achieve a fast convergence.

Use of the preconditioned conjugate gradient algorithm as a generic solver for mixed-model equations in animal breeding applications.

Abstract: Utility of the preconditioned conjugate gradient algorithm with a diagonal preconditioner for solving mixed-model equations in animal breeding applications was evaluated with 16 test problems. The problems included single- and multiple-trait analyses, with data on beef, dairy, and swine ranging from small examples to national data sets. Multiple-trait models considered low and high genetic correlations. Convergence was based on relative differences between left- and right-hand sides. The ordering of equations was fixed effects followed by random effects, with no special ordering within random effects. The preconditioned conjugate gradient program implemented with double precision converged for all models. However, when implemented in single precision, the preconditioned conjugate gradient algorithm did not converge for seven large models. The preconditioned conjugate gradient and successive overrelaxation algorithms were subsequently compared for 13 of the test problems. The preconditioned conjugate gradient algorithm was easy to implement with the iteration on data for general models. However, successive overrelaxation requires specific programming for each set of models. On average, the preconditioned conjugate gradient algorithm converged in three times fewer rounds of iteration than successive overrelaxation. With straightforward implementations, programs using the preconditioned conjugate gradient algorithm may be two or more times faster than those using successive overrelaxation. However, programs using the preconditioned conjugate gradient algorithm would use more memory than would comparable implementations using successive overrelaxation. Extensive optimization of either algorithm can influence rankings. The preconditioned conjugate gradient implemented with iteration on data, a diagonal preconditioner, and in double precision may be the algorithm of choice for solving mixed-model equations when sufficient memory is available and ease of implementation is essential.

Construction of Nearly Orthogonal Nedelec Bases for Rapid Convergence with Multilevel Preconditioned Solvers

Abstract: This paper presents a systematic approach to constructing high-order tangential vector basis functions for the multilevel finite element solution of electromagnetic wave problems. The new bases allow easy computation of a preconditioner to eliminate or at least weaken the indefiniteness of the system matrix and thus reduce the condition number of the system matrix. When these bases are used in multilevel solutions, where the multilevels correspond to the order of the basis functions, the resulting p-multilevel-ILU preconditioned conjugate gradient method (MPCG) provides an optimal rate of convergence. We first derive an admissible set of vectors of order p, and decompose this set into two subspaces---rotational and irrotational (gradient). We then reduce the number of vectors by making them orthogonal to all previously constructed lower-order bases. The remaining vectors are made mutually orthogonal in both the vector space and in the range space of the curl operator. The resulting vector basis functions provide maximum orthogonality while maintaining tangential continuity of the field. The zeroth-order space is further decomposed using a scalar-vector formulation to eliminate convergence problems at extremely low frequencies. Numerical experiments show that number of iterations needed for the solution by MPCG is basically constant, regardless of the order of the basis or of the matrix size. Computational speed is improved by several orders of magnitude due to the fast matrix solution of MPCG and to the high accuracy of the higher-order bases.

Finite element simulation of fully non‐linear interaction between vertical cylinders and steep waves. Part 1: methodology and numerical procedure

Abstract: A methodology for computing three-dimensional interaction between waves and fixed bodies is developed based on a fully non-linear potential flow theory. The associated boundary value problem is solved using a finite element method (FEM). A recovery technique has been implemented to improve the FEM solution. The velocity is calculated by a numerical differentiation technique. The corresponding algebraic equations are solved by the conjugate gradient method with a symmetric successive overrelaxation (SSOR) preconditioner. The radiation condition at a truncated boundary is imposed based on the combination of a damping zone and the Sommerfeld condition. This paper (Part 1) focuses on the technical procedure, while Part 2 [Finite element simulation of fully non-linear interaction between vertical cylinders and steep waves. Part 2. Numerical results and validation. International Journal for Numerical Methods in Fluids 2001] gives detailed numerical results, including validation, for the cases of steep waves interacting with one or two vertical cylinders. Copyright © 2001 John Wiley & Sons, Ltd.

Global Convergence of Conjugate Gradient Methods without Line Search.

Abstract: Global convergence results are derived for well-known conjugate gradient methods in which the line search step is replaced by a step whose length is determined by a formula. The results include the following cases: (1) The Fletcher–Reeves method, the Hestenes–Stiefel method, and the Dai–Yuan method applied to a strongly convex LC 1 objective function; (2) The Polak–Ribiere method and the Conjugate Descent method applied to a general, not necessarily convex, LC 1 objective function.

Superlinear Convergence of Conjugate Gradients

Abstract: We give a theoretical explanation for superlinear convergence behavior observed while solving large symmetric systems of equations using the conjugate gradient method or other Krylov subspace methods. We present a new bound on the relative error after $n$ iterations. This bound is valid in an asymptotic sense when the size $N$ of the system grows together with the number of iterations. The bound depends on the asymptotic eigenvalue distribution and on the ratio $n/N$. Under appropriate conditions we show that the bound is asymptotically sharp. Our findings are related to some recent results concerning asymptotics of discrete orthogonal polynomials. An important tool in our investigations is a constrained energy problem in logarithmic potential theory. The new asymptotic bounds for the rate of convergence are illustrated by discussing Toeplitz systems as well as a model problem stemming from the discretization of the Poisson equation.

On the numerical solution of a three-dimensional inverse medium scattering problem

Abstract: We examine the scattering of time-harmonic acoustic waves in inhomogeneous media The problem is to recover a spatially varying refractive index in a three-dimensional medium from far-field measurements of scattered waves corresponding to incoming waves from all directions This problem is exponentially ill-posed and of a large scale since a solution of the direct problem corresponds to solving a partial differential equation in 3 for each incident wave We construct a preconditioner for the conjugate gradient method applied to the normal equation to solve the regularized linearized operator equation in each Newton step This reduces the number of operator evaluations dramatically compared to standard regularized Newton methods Our method can also be applied effectively to other exponentially ill-posed problems, for example, in impedance tomography, heat conduction and obstacle scattering To solve the direct problems, we use an improved fast solver for the Lippmann–Schwinger equation suggested by Vainikko

Finite element three-dimensional direct current resistivity modelling: accuracy and efficiency considerations

Abstract: SUMMARY The finite element method is a powerful tool for 3-D DC resistivity modelling and inversion. The solution accuracy and computational efficiency are critical factors in using the method in 3-D resistivity imaging. This paper investigates the solution accuracy and the computational efficiency of two common element-type schemes: trilinear interpolation within a regular 8-node solid parallelepiped, and linear interpolations within six tetrahedral bricks within the same 8-node solid block. Four iterative solvers based on the pre-conditioned conjugate gradient method (SCG, TRIDCG, SORCG and ICCG), and one elimination solver called the banded Choleski factorization are employed for the solutions. The comparisons of the element schemes and solvers were made by means of numerical experiments using three synthetic models. The results show that the tetrahedron element scheme is far superior to the parallelepiped element scheme, both in accuracy and computational efficiency. The tetrahedron element scheme may save 43 per cent storage for an iterative solver, and achieve an accuracy of the maximum relative error of <1 per cent with an appropriate element size. The two iterative solvers, SORCG and ICCG, are suitable options for 3-D resistivity computations on a PC, and both perform comparably in terms of convergence speed in the two element schemes. ICCG achieves the best convergence rate, but nearly doubles the total storage size of the computation. Simple programming codes for the two iterative solvers are presented. We also show that a fine grid, which doubles the density of a coarse grid, will require at least 2 7 =128 times as much computing time when using the banded Choleski factorization. Such an increase, especially for 3-D resistivity inversion, should be compared with SORCG and ICCG solvers in order to find the computationally most efficient method when dealing with a large number of electrodes.

Reconstruction Algorithms for Permittivity and Conductivity Imaging

Abstract: Linear reconstruction algorithms are reviewed using assumed covariance matrices for the conductivity and data and the formulation of Tikhonov regularization using the singular value decomposition (SVD) with covariance norms. It is shown how iterative reconstruction algorithms, such as Landweber and conjugate gradient, can be used for regularization and analysed in terms of the SVD, and implemented directly for a one−step Newton’s method. Where there are known inequality constraints, such as upper and lower bounds, these can be incorporated in iterative methods and have a stabilizing effect on reconstructions.

An improved partition of unity finite element model for diffraction problems

Abstract: The partition of unity finite element method (PUFEM) is explored and improved to deal with practical diffraction problems efficiently. The use of plane waves as an external function space allows an efficient implementation of an approximate exterior non-reflective boundary condition, improving the original proposed by Higdon for general diffraction problems. A ‘virtually’ analytical integration procedure is introduced for multi-dimensional high-frequency problems which exhibits a dramatic decrease in the number of operations for a given error compared with standard integration methods. Suitable conjugate gradient type solvers for the whole range of wavenumbers are used, including such cases in which PUFEM can produce nearly singular matrices caused by ‘round-off’ limits. Copyright © 2001 John Wiley & Sons, Ltd.

Efficient solution of symmetric eigenvalue problems using multigridpreconditioners in the locally optimal block conjugate gradient method

Abstract: We present a short survey of multigrid--based solvers for symmetric eigenvalue problems. We concentrate our attention on ``of the shelf'''' and ``black box'''' methods, which should allow solving eigenvalue problems with minimal, or no, effort on the part of the developer, taking advantage of already existing algorithms and software. We consider a class of such methods, where the multigrid only appears as a black-box tool of constructing the preconditioner of the stiffness matrix, and the base iterative algorithm is one of well-known of-the-shelf preconditioned gradient methods such as the locally optimal block preconditioned conjugate gradient method. We review some known theoretical results for preconditioned gradient methods that guarantee the optimal, with respect to the grid size, convergence speed. Finally, we present results of numerical tests, which demonstrate practical effectiveness of our approach for the locally optimal block conjugate gradient method preconditioned by the standard V-cycle multigrid applied to the stiffness matrix.

Fully compact higher-order computation of steady-state natural convection in a square cavity.

Abstract: The flow in a thermally driven square cavity with adiabatic top and bottom walls and differentially heated vertical walls for a wide range of Rayleigh numbers (10(3)< or =Ra< or =10(7)) has been computed with a fourth-order accurate higher-order compact scheme, which was used earlier only for the stream-function vorticity (psi-omega) form of the two-dimensional steady-state Navier-Stokes equations. The boundary conditions used are also compact and of identical accuracy. In particular, a compact fourth-order accurate Neumann boundary condition has been developed for temperature at the adiabatic walls. The treatment of the derivative source term is also compact and has been done in such a way as to give fourth-order accuracy and easy assimilation with the solution procedure. As the discretization for the psi-omega formulation, boundary conditions, and source term treatment are all fourth-order accurate, highly accurate solutions are obtained on relatively coarser grids. Unlike other compact solution procedure in literature for this physical configuration, the present method is fully compact and fully higher-order accurate. Also, use of conjugate gradient and hybrid biconjugate gradient stabilized algorithms to solve the symmetric and nonsymmetric algebraic systems at every outer iteration, ensures good convergence behavior of the method even at higher Rayleigh numbers.

An Iterative Method with Variable Relaxation Parameters for Saddle-Point Problems

Abstract: In this paper, we propose an inexact Uzawa method with variable relaxation parameters for iteratively solving linear saddle-point problems. The method involves two variable relaxation parameters, which can be updated easily in each iteration, similar to the evaluation of the two iteration parameters in the conjugate gradient method. This new algorithm has an advantage over most existing Uzawa-type algorithms: it is always convergent without any a priori estimates on the spectrum of the preconditioned Schur complement matrix, which may not be easy to achieve in applications. The rate of the convergence of the inexact Uzawa method is analyzed. Numerical results of the algorithm applied for the Stokes problem and a purely linear system of algebraic equations are presented.

GENSMAC3D: a numerical method for solving unsteady three‐dimensional free surface flows

Abstract: A numerical method for solving three-dimensional free surface flows is presented. The technique is an extension of the GENSMAC code for calculating free surface flows in two dimensions. As in GENSMAC, the full Navier-Stokes equations are solved by a finite difference method; the fluid surface is represented by a piecewise linear surface composed of quadrilaterals and triangles containing marker particles on their vertices; the stress conditions on the free surface are accurately imposed; the conjugate gradient method is employed for solving the discrete Poisson equation arising from a velocity update; and an automatic time step routine is used for calculating the time step at every cycle. A program implementing these features has been interfaced with a solid modelling routine defining the flow domain. A user-friendly input data file is employed to allow almost any arbitrary three-dimensional shape to be described. The visualization of the results is performed using computer graphic structures such as phong shade, flat and parallel surfaces. Results demonstrating the applicability of this new technique for solving complex free surface flows, such as cavity filling and jet buckling, are presented.

A Preconditioned Conjugate Gradient Approach to Linear Equality Constrained Minimization

Abstract: We propose a new framework for the application of preconditioned conjugate gradients in the solution of large-scale linear equality constrained minimization problems. This framework allows for the exploitation of structure and sparsity in the context of solving the reduced Newton system (despite the fact that the reduced system may be dense). Numerical experiments performed on a variety of test problems from the Netlib LP collection indicate computational promise.