Showing papers on "Convergence (routing) published in 1991"
TL;DR: The convergence of a wide class of approximation schemes to the viscosity solution of fully nonlinear second-order elliptic or parabolic, possibly degenerate, partial differential equations is studied in this paper.
Abstract: The convergence of a wide class of approximation schemes to the viscosity solution of fully nonlinear second-order elliptic or parabolic, possibly degenerate, partial differential equations is studied. It is proved that any monotone, stable, and consistent scheme converges (to the correct solution), provided that there exists a comparison principle for the limiting equation. Several examples are given where the result applies. >
1,063 citations
Journal Article•
TL;DR: Results suggest how the sizing equation may be viewed as a coarse delineation of a boundary between what a physicist might call two distinct phases of GA behavior, and how these results may one day lead to rigorous proofs of convergence for recombinative G As operating on problems of bounded description.
Abstract: This paper considers the effect of stochasticity on the quality of convergence of genetic algorithms (GAs). In many problems, the variance of building-block fitness or so-called collateral noise is the major source of variance, and a population-sizing equation is derived to ensure that average signal-to-collateral-noise ratios are favorable to the discrimination of the best building blocks required to solve a problem of bounded deception. The sizing relation is modified to permit the inclusion of other sources of stochasticity, such as the noise of selection, the noise of genetic operators, and the explicit noise or nondeterminism of the objective function. In a test suite of five functions, the sizing relation proves to be a conservative predictor of average correct convergence, as long as all major sources of noise are considered in the sizing calculation. These results suggest how the sizing equation may be viewed as a coarse delineation of a boundary between what a physicist might call two distinct phases of GA behavior. At low population sizes the GA makes many errors of decision, and the quality of convergence is largely left to the vagaries of chance or the serial fixup of flawed results through mutation or other serial injection of diversity. At large population sizes, GAs can reliably discriminate between good and bad building blocks, and parallel processing and recombination of building blocks lead to quick solution of even difficult deceptive problems. Additionally, the paper outlines a number of extensions to this work, including the development of more refined models of the relation between generational average error and ultimate convergence quality, the development of online methods for sizing populations via the estimation of populations via the estimation of population-sizing parameters, and the investigation of populationsizing in the context of niching and other schemes designed for use in problems with high cardinality solution sets. The paper also discusses how these results may one day lead to rigorous proofs of convergence for recombinative G As operating on problems of bounded description.
697 citations
TL;DR: This approach is oriented toward applications in three phase distribution system operational analysis rather than planning analysis, and the solution method is the optimally ordered triangular factorization Y/sub BUS/ method (implicit Z/ sub BUS/ Gauss method) which has very good convergence characteristics on distribution problems.
Abstract: This approach is oriented toward applications in three phase distribution system operational analysis rather than planning analysis. The solution method is the optimally ordered triangular factorization Y/sub BUS/ method (implicit Z/sub BUS/ Gauss method) which not only takes advantage of the sparsity of system equations but also has very good convergence characteristics on distribution problems. Detailed component models are needed for all system components in the simulation. Utilizing the phase frame representation for all network elements, a program called Generalized Distribution Analysis Systems, with a number of features and capabilities not found in existing packages, has been developed for large-scale distribution system simulations. The system being analyzed can be balanced or unbalanced and can be a radial, network, or mixed-type distribution system. Furthermore, because the individual phase representation is employed for both system and component models, the system can comprise single, double, and three-phase systems simultaneously. >
492 citations
TL;DR: Three related techniques for performing accurate electronic-structure and total-energy calculations for systems with extended, so-called semicore, states are described and results are found to yield results in good agreement with each other and consistent with the available experimental data when well-converged calculations are performed.
Abstract: Three related techniques for performing accurate electronic-structure and total-energy calculations for systems with extended, so-called semicore, states are described. Total-energy calculations performed using the three methods are reported for fcc lanthanum. The convergence properties and relative accuracies of the three techniques are discussed. They are found to yield results in good agreement with each other and consistent with the available experimental data when well-converged calculations are performed. Highly accurate calculations are reported for the bcc phase, using one of these techniques, and a prediction of the bcc-fcc energy difference is reported.
381 citations
TL;DR: A new technique for proving rate of convergence estimates of multi- grid algorithms for asymmetric positive definite problems for symmetricpositive definite problems will be given in this paper.
Abstract: A new technique for proving rate of convergence estimates of multi- grid algorithms for symmetric positive definite problems will be given in this paper. The standard multigrid theory requires a "regularity and approxima- tion" assumption. In contrast, the new theory requires only an easily verified approximation assumption. This leads to convergence results for multigrid re- finement applications, problems with irregular coefficients, and problems whose coefficients have large jumps. In addition, the new theory shows why it suffices to smooth only in the regions where new nodes are being added in multigrid refinement applications.
250 citations
TL;DR: Conditions under which these approximations can be proved to converge globally to the true Hessian matrix are given, in the case where the Symmetric Rank One update formula is used.
Abstract: Quasi-Newton algorithms for unconstrained nonlinear minimization generate a sequence of matrices that can be considered as approximations of the objective function second derivatives This paper gives conditions under which these approximations can be proved to converge globally to the true Hessian matrix, in the case where the Symmetric Rank One update formula is used The rate of convergence is also examined and proven to be improving with the rate of convergence of the underlying iterates The theory is confirmed by some numerical experiments that also show the convergence of the Hessian approximations to be substantially slower for other known quasi-Newton formulae
219 citations
TL;DR: In this paper, the authors demonstrate the numerical feasibility of 4-D variational assimilation using a multilevel primitive-equation model and demonstrate the efficiency of the variational approach in extracting the information contained in the dynamics of the model.
Abstract: The aim of the paper is to demonstrate the numerical feasibility of 4-D variational assimilation using a multilevel primitive-equation model. The experiments consist in minimizing the distance between the model solution and the observations. The gradient of the cost function thus defined is computed by integrating the adjoint of the model.
Here, assimilations are performed using model-generated observations. In a preliminary set of experiments, assimilations were performed assuming that observations consisting of a full-model-state vector are available only at the end of the assimilation period. The numerical convergence of the method is proved and the results are meteorologically realistic. The use of the Machenhauer nonlinear normal mode initialization scheme and its adjoint turned out to have hastened the convergence and to have controlled to some extent the amount of gravity waves appearing in the solution. We identify a loss of conditioning of the minimization problem with an increase in the length of the assimilation period. The presence of horizontal diffusion in the model has the effect of degrading the convergence.
The second set of experiments evaluates the impact of observations distributed over the whole assimilation period. Through different senarios of sets of observations, we demonstrate the efficiency of the 4-D variational approach in extracting the information contained in the dynamics of the model, together with the information contained in the observations. In particular, observing only the small scales of the flow leads to a good reconstruction of both small scales and large scales. Observations of the mass-field evolution lead to a good reconstruction of the vorticity field in mid latitudes but less so in the tropics.
Increased resolution in the model in the experiments was found to have a negative impact on the speed of convergence of the minimization algorithm.
207 citations
TL;DR: In this article, an analysis of rational iterations for the matrix sign function is presented based on Pade approximations of a certain hypergeometric function and it is shown that l...
Abstract: In this paper an analysis of rational iterations for the matrix sign function is presented. This analysis is based on Pade approximations of a certain hypergeometric function and it is shown that l...
157 citations
TL;DR: It is rigorously established that the sequence of weight estimates can be approximated by a certain ordinary differential equation, in the sense of weak convergence of random processes as epsilon tends to zero.
Abstract: The behavior of neural network learning algorithms with a small, constant learning rate, epsilon , in stationary, random input environments is investigated. It is rigorously established that the sequence of weight estimates can be approximated by a certain ordinary differential equation, in the sense of weak convergence of random processes as epsilon tends to zero. As applications, backpropagation in feedforward architectures and some feature extraction algorithms are studied in more detail. >
154 citations
154 citations
11 Dec 1991
TL;DR: The problem of the synthesis of a feedback control assuring that the system state is ultimately bounded within a given compact set containing the origin with an assigned speed of convergence is investigated and it is shown that such a function may be derived by numerically efficient algorithms involving polyhedral sets.
Abstract: Linear discrete-time systems affected by both parameter and input uncertainties are considered. The problem of the synthesis of a feedback control assuring that the system state is ultimately bounded within a given compact set containing the origin with an assigned speed of convergence is investigated. It is shown that the problem has a solution if and only if there exists a certain Lyapunov function which does not belong to a pre-assigned class of functions (i.e. the quadratic ones) but it is determined by the target set in which ultimate boundedness is desired. One of the advantages of this approach is that one can handle systems with control constraints. No matching assumptions are made. For systems with linearly constrained uncertainties, it is shown that such a function may be derived by numerically efficient algorithms involving polyhedral sets. An extension of the technique to continuous-time systems is presented. >
TL;DR: In this paper, an algorithm using the Jackson polynomials is proposed that achieves an exponential convergence rate for exponentially stable systems, and it is shown that this, and similar identification algorithms, can be successfully combined with model reduction procedure to produce low-order models.
Abstract: We consider system identification in H∞ in the framework proposed by Helmicki, Jacobson and Nett. An algorithm using the Jackson polynomials is proposed that achieves an exponential convergence rate for exponentially stable systems. It is shown that this, and similar identification algorithms, can be successfully combined with a model reduction procedure to produce low-order models. Connections with the Nevanlinna-Pick interpolation problem are explored, and an algorithm is given in which the identified model interpolates the given noisy data. Some numerical results are provided for illustration. Finally, the case of unbounded random noise is discussed and it is shown that one can still obtain convergence with probability 1 under natural assumptions.
11 Dec 1991
TL;DR: In this article, a constructive design procedure for uniform approximation of smooth functions to a chosen degree of accuracy using networks of Gaussian radial basis functions was provided, which provided the basis for stable adaptive neuro-control algorithms for a class of nonlinear plants.
Abstract: Previous work has provided the theoretical foundations of a constructive design procedure for uniform approximation of smooth functions to a chosen degree of accuracy using networks of Gaussian radial basis functions. This construction and the guaranteed uniform bounds were shown to provide the basis for stable adaptive neurocontrol algorithms for a class of nonlinear plants. The authors detail and extend these ideas in three directions. First, some practical details of the construction are provided, explicitly illustrating the relation between the free parameters in the network design and the degree of approximation error on a particular set. Next, the original adaptive control algorithm is modified to permit incorporation of additional prior knowledge of the system dynamics, allowing the neurocontroller to operate in parallel with conventional fixed or adaptive controllers. Finally, it is shown how the Gaussian network construction may also be utilized in recursive identification algorithms with similar guarantees of stability and convergence. >
TL;DR: Trust region algorithms are an important class of methods that can be used to solve unconstrained optimization problems where the gradient values are approximated rather than computed exactly, provided they obey a simple relative error condition.
Abstract: Trust region algorithms are an important class of methods that can be used to solve unconstrained optimization problems. Strong global convergence results are demonstrated for a class of methods where the gradient values are approximated rather than computed exactly, provided they obey a simple relative error condition. No requirement is made that gradients be recomputed to successively greater accuracy after unsuccessful iterations.
01 Feb 1991
TL;DR: The results show that control based on some approximation of theJacobian is possible for a neural network, and shows that the rate of convergence of the neural net does not seem to depend crucially on the values of the Jacobian.
Abstract: The paper investigates the possibility of using a simple approximation for evaluating the error which must be back-propagated to allow a neural net to learn to control a plant in an adaptive way. The algorithm is based on an approximation of the Jacobian of the plant. The method is applied to five simulations. The first two simulations allow a comparison between the proposed algorithm and the standard back-propagation, for which the error to be back-propagated is precisely known. The results for the two methods show equivalent performances, and equivalent convergence time, for the test problems. This shows that the rate of convergence of the neural net does not seem to depend crucially on the values of the Jacobian. The last three simulations investigate the possibility of online adaptive learning. The results show that control based on some approximation of theJacobian is possible for a neural network.
TL;DR: The theory of convergence of a generic GR algorithm for the matrix eigenvalue problem that includes the QR,LR,SR, and other algorithms as special cases is developed and it is shown that with a certain obvious shifting strategy the GR algorithm typically has a quadratic asymptotic convergence rate.
DOI•
01 Jan 1991
TL;DR: Chipot et al. as mentioned in this paper studied the convergence of stationary solutions to stationary solutions for semilinear parabolic equations with nonlinear boundary conditions, where the boundary conditions are nonlinear.
Abstract: Unspecified Posted at the Zurich Open Repository and Archive, University of Zurich ZORA URL: http://doi.org/10.5167/uzh-22758 Originally published at: Chipot, M; Fila, M; Quittner, P (1991). Stationary solutions, blow up and convergence to stationary solutions for semilinear parabolic equations with nonlinear boundary conditions. Acta Mathematica Universitatis Comenianae. New Series, 60(1):35-103. Acta Math. Univ. Comenianae Vol. LX, 1(1991), pp. 35–103 35 STATIONARY SOLUTIONS, BLOW UP AND CONVERGENCE TO STATIONARY SOLUTIONS FOR SEMILINEAR PARABOLIC EQUATIONS WITH NONLINEAR BOUNDARY CONDITIONS M. CHIPOT, M. FILA AND P. QUITTNER
TL;DR: Two examples of molecular geometry which are known to have slow SHAKE convergence are used to show that simple changes to the procedure can result in a five-fold improvement in convergence speed.
TL;DR: In this paper, a generalized linear dynamic model or singular model is used to solve the linear dynamic material balance problem, which is very useful in real-time processing and reduces the computational problem such as singularities and round-off errors.
Abstract: A generalized linear dynamic model or singular model, for which the standard state space representation and the Kalman filtering cannot be applied, is used to develop a new algorithm to solve the linear dynamic material balance problem. This algorithm is based on the method developed in the steady-state case and leads to a recursive scheme, which is very useful in real-time processing. It reduces the computational problem such as singularities and round-off errors that may occur in complex systems. Convergence conditions are given and verified for the dynamic material balance case.
TL;DR: The "maximum" constraint is used in place of the original constraint set to convert a multi-constrained optimization problem to a non-smooth but singly constrained problem; the surrogate constraint concept and themaximum entropy principle are employed to derive a smooth function.
Abstract: This paper presents a new method, called the "aggregate function" method, for solvingnonlinear programming problems. At first, we use the "maximum" constraint in place of theoriginal constraint set to convert a multi-constrained optimization problem to a non-smoothbut singly constrained problem; we then employ the surrogate constraint concept and themaximum entropy principle to derive a smooth function, by which the non-smooth maximumconstraint is approximated and the original problem is converted to a smooth and singly con-strained problem; furthermore, we develop a multiplier penalty algorithm. The presentalgorithm has merits of stable and fast convergence and ease of computer implementation,and is particularly suitable to solving a nonlinear programming problem with a large num-ber of constraints.
TL;DR: The rate of convergence of a partially asynchronous implementation of the gradient projection algorithm of Goldstein and Levitin and Polyak for the problem of minimizing a differentiable function over a closed convex set is analyzed.
Abstract: Recently, Bertsekas and Tsitsiklis proposed a partially asynchronous implementation of the gradient projection algorithm of Goldstein and Levitin and Polyak for the problem of minimizing a differentiable function over a closed convex set. In this paper, the rate of convergence of this algorithm is analyzed. It is shown that if the standard assumptions hold (that is, the solution set is nonempty and the gradient of the function is Lipschitz continuous) and (i) the isocost surfaces of the objective function, restricted to the solution set, are properly separated and (ii) a certain multifunction associated with the problem is locally upper Lipschitzian, then this algorithm attains a linear rate of convergence.
TL;DR: In this article, the minimax grid matching problem in dimensions greater than two was solved and the Glivenko-Cantelli convergence of empirical measures was shown to be correct.
Abstract: In this article we solve the minimax grid matching problem in dimensions greater than two. As a by-product, we settle a long-open problem involving the Glivenko-Cantelli convergence of empirical measures.
TL;DR: This paper extends the convergence properties of the Polyak's subgradient algorithm with a fixed target value to a more general case with variable target values and provides a target value updating scheme which finds an optimal solution without prior knowledge of the optimal objective value.
Abstract: Polyak's subgradient algorithm for nondifferentiable optimization problems requires prior knowledge of the optimal value of the objective function to find an optimal solution. In this paper we extend the convergence properties of the Polyak's subgradient algorithm with a fixed target value to a more general case with variable target values. Then a target value updating scheme is provided which finds an optimal solution without prior knowledge of the optimal objective value. The convergence proof of the scheme is provided and computational results of the scheme are reported.
TL;DR: By means of weak convergence methods, it is shown that the multistage algorithms via averaging have asymptotically optimal convergence speed and are efficient procedures.
Abstract: Stochastic approximation algorithms are considered and Polyak's averaging approach (cf [1]) is revisited Under much weaker conditions, convergence and rate of convergence results are developed In lieu of uncorrelated noise, φ-mixing type of random disturbances are treated By means of weak convergence methods, it is shown that the multistage algorithms via averaging have asymptotically optimal convergence speed and are efficient procedures
TL;DR: Two efficient and robust finite-volume multigrid schemes for solving the Navier-Stokes equations are investigated and exhibit very good convergence rates for a broad range of artificial dissipation coefficients.
Abstract: Two efficient and robust finite-volume multigrid schemes for solving the Navier-Stokes equations are investigated These schemes employ either a cell centred or a cell vertex discretisation technique An explicit Runge-Kutta algorithm is used to advance the solution in time Acceleration techniques are applied to obtain faster steady-state convergence Accuracy and convergence of the schemes are examined Computational results for transonic airfoil flows are essentially the same, even for a coarse mesh Both schemes exhibit very good convergence rates for a broad range of artificial dissipation coefficients
TL;DR: The scenario aggregation algorithm is specialized for stochastic networks and determines a solution that does not depend on hindsight and accounts for the uncertain environment depicted by a number of appropriately weighted scenarios, thus preserving the network structure.
Abstract: The scenario aggregation algorithm is specialized for stochastic networks. The algorithm determines a solution that does not depend on hindsight and accounts for the uncertain environment depicted by a number of appropriately weighted scenarios. The solution procedure decomposes the stochastic program to its constituent scenario subproblems, thus preserving the network structure. Computational results are reported demonstrating the algorithm's convergence behavior. Acceleration schemes are discussed along with termination criteria. The algorithm's potential for execution on parallel multiprocessors is discussed.
TL;DR: Algorithms for solving the problem of minimizing the maximum of a finite number of functions and it is shown that a quadratic rate of convergence is obtained.
Abstract: Algorithms for solving the problem of minimizing the maximum of a finite number of functions are proposed and analyzed. Quadratic approximations to the functions are employed in the determination of a search direction. Global convergence is proven and it is shown that a quadratic rate of convergence is obtained.
TL;DR: A global convergence theory for a class of trust-region algorithms for solving the equality constrained optimization problem is presented and is used to establish global convergence of the 1984 Celis–Dennis–Tapia algorithm with a different scheme for updating the penalty parameter.
Abstract: A global convergence theory for a class of trust-region algorithms for solving the equality constrained optimization problem is presented. This theory is sufficiently general that it holds for any algorithm that generates steps giving at least a fraction of Cauchy decrease in the quadratic model of the constraints, and that uses the augmented Lagrangian as a merit function. This theory is used to establish global convergence of the 1984 Celis–Dennis–Tapia algorithm with a different scheme for updating the penalty parameter. The behavior of the penalty parameter is also discussed.
TL;DR: In this paper, a method for solving the Liouville-von Neumann equation is presented, where the action of operators is calculated locally in coordinate and/or momentum representation, and the Fast Fourier Transform (FFT) is used to pass back and forth between coordinate and momentum representations, this transformation preserving all exact commutation relations.
TL;DR: In this article, the authors considered two-dimensional semimartingale RBM's with rectangular state space, which serve as approximate models of finite queues in tandem and proposed an algorithm for numerical solution of the adjoint relationship.
Abstract: Multidimensional reflected Brownian motions, also called regulated Brownian motions or simply RBM's, arise as approximate models of queueing networks Thus the stationary distributions of these diffusion processes are of interest for steady-state analysis of the corresponding queueing systems This paper considers two-dimensional semimartingale RBM's with rectangular state space, which include the RBM's that serve as approximate models of finite queues in tandem The stationary distribution of such an RBM is uniquely characterized by a certain basic adjoint relationship, and an algorithm is proposed for numerical solution of that relationship We cannot offer a general proof of convergence, but the algorithm has been coded and applied to special cases where the stationary distribution can be determined by other means; the computed solutions agree closely with previously known results and convergence is reasonably fast Our current computer code is specific to two-dimensional rectangles, but the basic logic of the algorithm applies equally well to any semimartingale RBM with bounded polyhedral state space, regardless of dimension To demonstrate the role of the algorithm in practical performance analysis, we use it to derive numerical performance estimates for a particular example of finite queues in tandem; our numerical estimates of both the throughput loss rate and the average queue lengths are found to agree with simulated values to within about five percent