Showing papers on "Iterative method published in 1993"
TL;DR: A variant of the GMRES algorithm is presented that allows changes in the preconditioning at every step, resulting in a result of the flexibility of the new variant that any iterative method can be used as a preconditionser.
Abstract: A variant of the GMRES algorithm is presented that allows changes in the preconditioning at every step. There are many possible applications of the new algorithm, some of which are briefly discussed. In particular, a result of the flexibility of the new variant is that any iterative method can be used as a preconditioner. For example, the standard GMRES algorithm itself can be used as a preconditioner, as can CGNR (or CGNE), the conjugate gradient method applied to the normal equations. However, the more appealing utilization of the method is in conjunction with relaxation techniques, possibly multilevel techniques. The possibility of changing preconditioners may be exploited to develop efficient iterative methods and to enhance robustness. A few numerical experiments are reported to illustrate this fact.
1,348 citations
TL;DR: Two partition methods that speed up iterative search methods applied to vehicle routing problems including a large number of vehicles, based on the arborescence built from the shortest paths from any city to the depot are presented.
Abstract: This paper presents two partition methods that speed up iterative search methods applied to vehicle routing problems including a large number of vehicles. Indeed, using a simple implementation of taboo search as an iterative search method, every best-known solution to classical problems was found. The first partition method (based on a partition into polar regions) is appropriate for Euclidean problems whose cities are regularly distributed around a central depot. The second partition method is suitable for any problem and is based on the arborescence built from the shortest paths from any city to the depot. Finally, solutions that are believed to be optimum are given for problems generated on a grid. © 1993 by John Wiley & Sons, Inc.
626 citations
TL;DR: In this paper, the fractional step method for solving the incompressible Navier-Stokes equations in primitive variables is analyzed as a block LU decomposition, which allows arbitrarily high temporal order of accuracy.
566 citations
TL;DR: It is shown that Bayesian segmentation using Gauss-Seidel iteration produces useful estimates at much lower signal-to-noise ratios than required for continuously valued reconstruction.
Abstract: A method for Bayesian reconstruction which relies on updates of single pixel values, rather than the entire image, at each iteration is presented. The technique is similar to Gauss-Seidel (GS) iteration for the solution of differential equations on finite grids. The computational cost per iteration of the GS approach is found to be approximately equal to that of gradient methods. For continuously valued images, GS is found to have significantly better convergence at modes representing high spatial frequencies. In addition, GS is well suited to segmentation when the image is constrained to be discretely valued. It is shown that Bayesian segmentation using GS iteration produces useful estimates at much lower signal-to-noise ratios than required for continuously valued reconstruction. The convergence properties of gradient ascent and GS for reconstruction from integral projections are analyzed, and simulations of both maximum-likelihood and maximum a posteriori cases are included. >
543 citations
TL;DR: Algebraic reconstruction techniques (ART) are iterative procedures for recovering objects from their projections as discussed by the authors, which is claimed to produce high-quality reconstructions with excellent computational efficiency This is demonstrated by an example based on a particular (but realistic) medical imaging task, showing that ART can match the performance of the standard expectation-maximization approach for maximizing likelihood, but at an order of magnitude less computational cost.
Abstract: Algebraic reconstruction techniques (ART) are iterative procedures for recovering objects from their projections It is claimed that by a careful adjustment of the order in which the collected data are accessed during the reconstruction procedure and of the so-called relaxation parameters that are to be chosen in an algebraic reconstruction technique, ART can produce high-quality reconstructions with excellent computational efficiency This is demonstrated by an example based on a particular (but realistic) medical imaging task, showing that ART can match the performance of the standard expectation-maximization approach for maximizing likelihood (from the point of view of that particular medical task), but at an order of magnitude less computational cost >
509 citations
TL;DR: In this paper, a new algorithm using the primal-dual interior point method with the predictor-corrector for solving nonlinear optimal power flow (OPF) problems is presented.
Abstract: A new algorithm using the primal-dual interior point method with the predictor-corrector for solving nonlinear optimal power flow (OPF) problems is presented. The formulation and the solution technique are new. Both equalities and inequalities in the OPF are considered and simultaneously solved in a nonlinear manner based on the Karush-Kuhn-Tucker (KKT) conditions. The major computational effort of the algorithm is solving a symmetrical system of equations, whose sparsity structure is fixed. Therefore only one optimal ordering and one symbolic factorization are involved. Numerical results of several test systems ranging in size from 9 to 2423 buses are presented and comparisons are made with the pure primal-dual interior point algorithm. The results show that the predictor-corrector primal-dual interior point algorithm for OPF is computationally more attractive than the pure primal-dual interior point algorithm in terms of speed and iteration count. >
422 citations
TL;DR: In this article, a model for data acquired with the use of a charge-coupled-device camera is given and then used for developing a new iterative method for restoring intensities of objects observed with such a camera.
Abstract: A model for data acquired with the use of a charge-coupled-device camera is given and is then used for developing a new iterative method for restoring intensities of objects observed with such a camera. The model includes the effects of point spread, photoconversion noise, readout noise, nonuniform flat-field response, nonuniform spectral response, and extraneous charge carriers resulting from bias, dark current, and both internal and external background radiation. An iterative algorithm is identified that produces a sequence of estimates converging toward a constrained maximum-likelihood estimate of the intensity distribution of an imaged object. An example is given for restoring images from data acquired with the use of the Hubble Space Telescope.
367 citations
TL;DR: An algorithm is presented for iterative learning of the control input for a linear discrete-time multivariable system and its synthesis and analysis are based on two-dimensional system theory.
Abstract: An algorithm is presented for iterative learning of the control input for a linear discrete-time multivariable system. Necessary and sufficient conditions are stated for convergence of the proposed algorithm. The algorithm synthesis and analysis are based on two-dimensional (2-D) system theory. A numerical example is given. >
363 citations
01 Jul 1993
TL;DR: A spectral approach to multiway ratio-cut partitioning is developed which provides a generalization of the ratio- cut cost metric to k-way partitioning and a lower bound on this cost metric.
Abstract: Recent research on partitioning has focussed on the ratio-cut cost metric which maintains a balance between the sizes of the edges cut and the sizes of the partitions without fixing the size of the partitions a priori. Iterative approaches and spectral approaches to two-way ratio-cut partitioning have yielded higher quality partitioning results. In this paper we develop a spectral approach to multiway ratio-cut partitioning which provides a generalization of the ratio-cut cost metric to k-way partitioning and a lower bound on this cost metric. Our approach involves finding the k smallest eigenvalue/eigenvector pairs of the Laplacian of the graph. The eigenvectors provide an embedding of the graph's n vertices into a k-dimensional subspace. We devise a time and space efficient clustering heuristic to coerce the points in the embedding into k partitions. Advancement over the current work is evidenced by the results of experiments on the standard benchmarks.
361 citations
TL;DR: A class of iterative methods for solving the blind deconvolution problem, i.e. for recovering the input of an unknown possibly nonminimum-phase linear system by observation of its output, is presented and shows that in many cases of practical interest the performance of the proposed methods is far superior to linear prediction methods even for minimum phase systems.
Abstract: A class of iterative methods for solving the blind deconvolution problem, i.e. for recovering the input of an unknown possibly nonminimum-phase linear system by observation of its output, is presented. These methods are universal do not require prior knowledge of the input distribution, are computationally efficient and statistically stable, and converge to the desired solution regardless of initialization at a very fast rate. The effects of finite length of the data, finite length of the equalizer, and additive noise in the system on the attainable performance (intersymbol interference) are analyzed. It is shown that in many cases of practical interest the performance of the proposed methods is far superior to linear prediction methods even for minimum phase systems. Recursive and sequential algorithms are also developed, which allow real-time implementation and adaptive equalization of time-varying systems. >
343 citations
01 Nov 1993
TL;DR: It is shown empirically that some knowledge can greatly improve the convergence speed of a repair-based system, but that too much knowledge can overwhelm a system and result in degraded performance.
Abstract: The GERRY scheduling and rescheduling system being applied to coordinate Space Shuttle ground processing is described. The system uses constraint-based iterative repair, a technique that starts with a complete but possibly flawed schedule and iteratively improves it by using constraint knowledge within repair heuristics. The tradeoff between the informedness and the computational cost of several repair heuristics is explored. It is shown empirically that some knowledge can greatly improve the convergence speed of a repair-based system, but that too much knowledge can overwhelm a system and result in degraded performance. >
TL;DR: In this paper, a look-ahead version of the Lanczos algorithm is proposed to solve large sparse non-Hermitian linear systems with non-asymptotic eigenvalues.
Abstract: The nonsymmetric Lanczos method can be used to compute eigenvalues of large sparse non-Hermitian matrices or to solve large sparse non-Hermitian linear systems. However, the original Lanczos algorithm is susceptible to possible breakdowns and potential instabilities. An implementation is presented of a look-ahead version of the Lanczos algorithm that, except for the very special situation of an incurable breakdown, overcomes these problems by skipping over those steps in which a breakdown or near-breakdown would occur in the standard process. The proposed algorithm can handle look-ahead steps of any length and requires the same number of matrix-vector products and inner products as the standard Lanczos process without look-ahead.
Book•
01 Mar 1993TL;DR: The world of scientific computing linear algebra parallel and vector computing polynomial approximation continuous problems solved discretely direct solution of linear equations parallel direct methods relaxation-type iterative methods conjugate gradient-type methods.
Abstract: The world of scientific computing linear algebra parallel and vector computing polynomial approximation continuous problems solved discretely direct solution of linear equations parallel direct methods relaxation-type iterative methods conjugate gradient-type methods.
TL;DR: In this article, an inversion procedure that uses conjugate gradient relaxation methods was developed to invert magnetotelluric data for 3D earth models, which can be applied to all inverse problems.
Abstract: SUMMARY We have developed an inversion procedure that uses conjugate gradient relaxation methods. Although one can generalize the method to all inverse problems, we demonstrate its use to invert magnetotelluric data for 3-D earth models. This procedure allows us to bypass the actual computation of the sensitivity matrix A or the inversion of the ATA term. In fact, with the relaxation approach, one only needs to compute the effect of the sensitivity matrix or its transpose multiplying an arbitrary vector. We show that each of these requires one forward problem with a distributed set of sources either in the volume (for A multiplying a vector) or on the surface (for AT multiplying a vector). This significantly reduces the computational requirements needed to do a 3-D inversion. For this paper, we have simplified the boundary conditions by assuming the model is repeated in the horizontal directions, but this is not a necessary constraint of the method. The algorithm reduces data errors to the 2 per cent level for noise-free synthetic 3-D magnetotelluric data.
TL;DR: An approach to obtaining high-resolution image reconstruction from low-resolution, blurred, and noisy multiple-input frames is presented and a recursive-least-squares approach with iterative regularization is developed in the discrete Fourier transform (DFT) domain.
Abstract: An approach to obtaining high-resolution image reconstruction from low-resolution, blurred, and noisy multiple-input frames is presented. A recursive-least-squares approach with iterative regularization is developed in the discrete Fourier transform (DFT) domain. When the input frames are processed recursively, the reconstruction does not converge in general due to the measurement noise and ill-conditioned nature of the deblurring. Through the iterative update of the regularization function and the proper choice of the regularization parameter, good high-resolution reconstructions of low-resolution, blurred, and noisy input frames are obtained. The proposed algorithm minimizes the computational requirements and provides a parallel computation structure since the reconstruction is done independently for each DFT element. Computer simulations demonstrate the performance of the algorithm. >
TL;DR: The Recursive Projection Method (RPM) as mentioned in this paper stabilizes fixed-point iterative procedures by computing a projection onto the unstable subspace, which can be used to accelerate iterative procedure when slow convergence is due to slowly decaying modes.
Abstract: Fixed-point iterative procedures for solving nonlinear parameter dependent problems can converge for some interval of parameter values and diverge as the parameter changes. The Recursive Projection Method (RPM), which stabilizes such procedures by computing a projection onto the unstable subspace is presented. On this subspace a Newton or special Newton iteration is performed, and the fixed-point iteration is used on the complement. As continuation in the parameter proceeds, the projection is efficiently updated, possibly increasing or decreasing the dimension of the unstable subspace. The method is extremely effective when the dimension of the unstable subspace is small compared to the dimension of the system. Convergence proofs are given and pseudo-arclength continuation on the unstable subspace is introduced to allow continuation past folds. Examples are presented for an important application of the RPM in which a “black-box” time integration scheme is stabilized, enabling it to compute unstable steady states. The RPM can also be used to accelerate iterative procedures when slow convergence is due to a few slowly decaying modes.
TL;DR: A conjugate gradient-like method is proposed which is applicable to symmetric indefinite problems, the effects of stabilisation on the algebraic structure of the discrete Stokes operator are described and estimates of the eigenvalue spectrum of this operator are derived on which the convergence rate of the iteration depends.
Abstract: Mixed finite element approximation of the classical Stokes problem describing slow viscous incompressible flow gives rise to symmetric indefinite systems for the discrete velocity and pressure variables. Iterative solution of such indefinite systems is feasible and is an attractive approach for large problems. The use of stabilisation methods for convenient (but unstable) mixed elements introduces stabilisation parameters. We show how these can be chosen to obtain rapid iterative convergence. We propose a conjugate gradient-like method (the method of preconditioned conjugate residuals) which is applicable to symmetric indefinite problems, describe the effects of stabilisation on the algebraic structure of the discrete Stokes operator and derive estimates of the eigenvalue spectrum of this operator on which the convergence rate of the iteration depends. Here we discuss the simple case of diagonal preconditioning. Our results apply to both locally and globally stabilised mixed elements as well as to elements which are inherently stable. We demonstrate that convergence rates comparable to that achieved using the diagonally scaled conjugate gradient method applied to the discrete Laplacian are approachable for the Stokes problem.
TL;DR: A modified technique for calculating a direction in weight-space which decreases the error for each class is presented and the rate of learning for two-class classification problems is accelerated by an order of magnitude.
Abstract: The backpropagation algorithm converges very slowly for two-class problems in which most of the exemplars belong to one dominant class. An analysis shows that this occurs because the computed net error gradient vector is dominated by the bigger class so much that the net error for the exemplars in the smaller class increases significantly in the initial iteration. The subsequent rate of convergence of the net error is very low. A modified technique for calculating a direction in weight-space which decreases the error for each class is presented. Using this algorithm, the rate of learning for two-class classification problems is accelerated by an order of magnitude. >
01 Jan 1993
TL;DR: A general strategy for solving the motion planning problem for real analytic, controllable systems without drift by computing a control that provides an exact solution of the original problem if the given system is nilpotent.
Abstract: We propose a general strategy for solving the motion planning problem for real analytic, controllable systems without drift. The procedure starts by computing a control that steers the given initial point to the desired target point for an extended system, in which a number of Lie brackets of the system vector fields are added to the right-hand side. The main point then is to use formal calculations based on the product expansion relative to a P. Hall basis, to produce another control that achieves the desired result on the formal level. It then turns out that this control provides an exact solution of the original problem if the given system is nilpotent. When the system is not nilpotent, one can still produce an iterative algorithm that converges very fast to a solution. Using the theory of feedback nilpotentization, one can find classes of non-nilpotent systems for which the algorithm, in cascade with a precompensator, produces an exact solution in a finite number of steps. We also include results of simulations which illustrate the effectiveness of the procedure.
TL;DR: The detailed description of the NE/SQP method and the associated convergence theory are presented, and the numerical results of an extensive computational study are reported which are aimed at demonstrating the practical efficiency of the method for solving a wide variety of realistic nonlinear complementarity problems.
Abstract: In this paper, we present a new iterative method for solving the nonlinear complementarity problem. This method, which we call NE/SQP (for Nonsmooth Equations/Successive Quadratic Programming), is a damped Gauss--Newton algorithm applied to solve a certain nonsmooth-equation formulation of the complementarity problem; it is intended to overcome a major deficiency of several previous methods of this type. Unlike these earlier algorithms whose convergence critically depends on a solvability assumption on the subproblems, the NE/SQP method involves solving a sequence of nonnegatively constrained convex quadratic programs of the least-squares type; the latter programs are always solvable and their solution can be obtained by a host of efficient quadratic programming subroutines. Hence, the new method is a robust procedure which, not only is very easy to describe and simple to implement, but also has the potential advantage of being capable of solving problems of very large size. Besides the desirable feature of robustness and ease of implementation, the NE/SQP method retains two fundamental attractions of a typical member in the Gauss--Newton family of algorithms; namely, it is globally and locally quadratically convergent. Besides presenting the detailed description of the NE/SQP method and the associated convergence theory, we also report the numerical results of an extensive computational study which is aimed at demonstrating the practical efficiency of the method for solving a wide variety of realistic nonlinear complementarity problems.
TL;DR: Two algorithms are introduced that show exceptional promise in finding molecular conformations using distance geometry on nuclear magnetic resonance data and an iterative algorithm between possible conformations obtained from the first algorithm and permissible data points near the configuration.
Abstract: Two algorithms are introduced that show exceptional promise in finding molecular conformations using distance geometry on nuclear magnetic resonance data. The fist algorithm is a gradient version of the majorization algorithm from multidimensional scaling. The main contribution is a large decrease in CPU time. The second algorithm is an iterative algorithm between possible conformations obtained from the fist al- gorithm and permissible data points near the configuration. These ideas are similar to alternating least squares or alternating projections on convex sets. The iterations significantly improve the conformation from the first algorithm when applied to the small peptide
TL;DR: This paper proposes to modify the Newton method for variational inequality problems by using a certain differentiable merit function to determine a suitable step length and shows that the method is globally convergent and the rate of convergence is quadratic.
Abstract: Variational inequality problems have been used to formulate and study equilibrium problems, which arise in many fields including economics, operations research and regional sciences. For solving variational inequality problems, various iterative methods such as projection methods and the nonlinear Jacobi method have been developed. These methods are convergent to a solution under certain conditions, but their rates of convergence are typically linear. In this paper we propose to modify the Newton method for variational inequality problems by using a certain differentiable merit function to determine a suitable step length. The purpose of introducing this merit function is to provide some measure of the discrepancy between the solution and the current iterate. It is then shown that, under the strong monotonicity assumption, the method is globally convergent and, under some additional assumptions, the rate of convergence is quadratic. Limited computational experience indicates the high efficiency of the proposed method.
TL;DR: It is proved that as soon as eigen values of the original operator are sufficiently well approximated by Ritz values, GMRES from then on converges at least as fast as for a related system in which these eigenvalues (and their eigenvector components) are missing.
TL;DR: A shift-invariant multiresolution representation of signals or images using dilations and translations of the autocorrelation functions of compactly supported wavelets, which reduces the reconstruction problem to that of solving a system of linear algebraic equations.
Abstract: Proposes a shift-invariant multiresolution representation of signals or images using dilations and translations of the autocorrelation functions of compactly supported wavelets. Although these functions do not form an orthonormal basis, their properties make them useful for signal and image analysis. Unlike wavelet-based orthonormal representations, the present representation has (1) symmetric analyzing functions, (2) shift-invariance, (3) associated iterative interpolation schemes, and (4) a simple algorithm for finding the locations of the multiscale edges as zero-crossings. The authors also develop a noniterative method for reconstructing signals from their zero-crossings (and slopes at these zero-crossings) in their representation. This method reduces the reconstruction problem to that of solving a system of linear algebraic equations. >
TL;DR: In this paper, a method for reconstructing the complex index of refraction of a bounded inhomogeneous object from measured scattered field data is presented, where the index and the unknown fields within the object are simultaneously reconstructed in an iterative algorithm.
Abstract: A method for reconstructing the complex index of refraction of a bounded inhomogeneous object from measured scattered field data is presented. The index and the unknown fields within the object are simultaneously reconstructed in an iterative algorithm. The method is a refinement of earlier work which incorporates a more effective way to update the unknowns at each stage of the iteration. Considerable efficiency in the algorithm is achieved. Some numerical examples are given indicating the limits on the contrasts which can be reconstructed. These limits show that the range of contrasts that may be reconstructed is extended over that achievable with the earlier work.
01 Dec 1993
TL;DR: Various methods are presented to solve the determination of the direct kinematics of fully parallel manipulators: two kinds of iterative schemes, a reduced iterative scheme, and a polynomial method.
Abstract: The determination of the direct kinematics of fully parallel manipulators is in general a difficult problem but has to be solved for any practical use. Various methods are presented to solve this problem: two kinds of iterative schemes, a reduced iterative scheme, and a polynomial method. The computation time of these methods are compared and their various advantages are shown. >
TL;DR: Three different preconditioners, namely, the incomplete LU factorization (ILU), block diagonal factorization, and the symmetric successive over-relaxation (SSOR) are investigated, which have been optimized to have good vectorization properties.
TL;DR: Shows how polynomial acceleration techniques which have been developed for the solution of large linear systems can be employed to improve and accelerate the frame algorithm.
Abstract: Shows how polynomial acceleration techniques which have been developed for the solution of large linear systems can be employed to improve and accelerate the frame algorithm. These methods permit a reduction in the number of necessary iterations by an order of magnitude when the frame algorithm is slow. The author gives several examples from the theory of irregular sampling, from wavelet theory and from Gabor theory where these methods are probably mandatory for efficient reconstruction. >
TL;DR: A comprehensive study of the general l/sub 1/-optimal multiblock problem and a new linear programming algorithm for computing suboptimal controllers are presented and further insight is gained into the structure of the optimal solution.
Abstract: A comprehensive study of the general l/sub 1/-optimal multiblock problem and a new linear programming algorithm for computing suboptimal controllers are presented. By formulating the interpolation conditions in a concise and natural way, the general theory is developed in simpler terms and with a minimum number of assumptions. In addition, further insight is gained into the structure of the optimal solution, and different classes of multiblock problems are distinguished. This leads to a conceptually attractive, iterative method for finding approximate solutions. >
TL;DR: In this article, a four-dimensional variational data assimilation is applied to a regional forecast model as part of the development of a new data-assimilation system at the National Meteorological Center (NMC).
Abstract: Four-dimensional variational data assimilation is applied to a regional forecast model as part of the development of a new data assimilation system at the National Meteorological Center (NMC). The assimilation employs an operational version of the NMC's new regional forecast model defined in eta vertical coordinates, and data used are operationally produced optimal interpolation (OI) analyses (using the first guess from the NMC's global spectral model), available every 3 h. Humidity and parameterized processes are not included in the adjoint model integration. The calculation of gradients by the adjoint model is approximate since the forecast model is used in its full-physics operational form. All experiments are over a 12-h assimilation period with subsequent 48-h forecast. Three different types of assimilation experiments are performed: adjustment of initial conditions only (standard “adjoint” approach), adjustment of a correction to the model equations only (variational continuous assimilation...