Extension of newton and quasi-newton methods to systems of pc^1 equations
01 Jan 1986-Journal of The Operations Research Society of Japan (The Operations Research Society of Japan)-Vol. 29, Iss: 4, pp 352-375
TL;DR: In this article, the authors extended Newton and quasi-Newton methods to systems of PC 1 equations and established the quadratic convergence property of the extended Newton method and the Q-superlinear convergence property for the extended quasiNewton method.
Abstract: This paper extends Newton and quasi-Newton methods to systems of PC 1 equations and establishes the quadratic convergence property of the extended Newton method and the Q-superlinear convergence property of the extended quasi-Newton method.
Citations
More filters
TL;DR: It is shown that the gradient function of the augmented Lagrangian forC2-nonlinear programming is semismooth, and the extended Newton's method can be used in the augmentedlagrangian method for solving nonlinear programs.
Abstract: Newton's method for solving a nonlinear equation of several variables is extended to a nonsmooth case by using the generalized Jacobian instead of the derivative. This extension includes the B-derivative version of Newton's method as a special case. Convergence theorems are proved under the condition of semismoothness. It is shown that the gradient function of the augmented Lagrangian forC2-nonlinear programming is semismooth. Thus, the extended Newton's method can be used in the augmented Lagrangian method for solving nonlinear programs.
1,464 citations
TL;DR: The PATH solver as mentioned in this paper is an implementation of a stabilized Newton method for the solution of the Mixed Complementarity Problem, which employs a path generation procedure which is used to construct a piecewise-linear path from the current point to the Newton point; a step length acceptance criterion and a non-monotone path search are then used to choose the next iterate.
Abstract: The PATH solver is an implementation of a stabilized Newton method for the solution of the Mixed Complementarity Problem. The stabilization scheme employs a path-generation procedure which is used to construct a piecewise-linear path from the current point to the Newton point; a step length acceptance criterion and a non-monotone pathsearch are then used to choose the next iterate. The algorithm is shown to be globally convergent under assumptions which generalize those required to obtain similar results in the smooth case. Several impleέentation issues are discussed, and extensive computational results obtained from problems commonly found in the literature are given
767 citations
TL;DR: The Kuhn-Tucker conditions of an optimization problem with inequality constraints are transformed equivalently into a special nonlinear system of equations T 0(z) = 0 as mentioned in this paper, and Newton's method for solving this system combines two valuable properties: the local Q-quadratic convergence without assuming the strict complementary slackness condition and the regularity of the Jacobian of T 0 at a point z under reasonable conditions.
Abstract: The Kuhn–Tucker conditions of an optimization problem with inequality constraints are transformed equivalently into a special nonlinear system of equations T 0(z) = 0. It is shown that Newton's method for solving this system combines two valuable properties: The local Q-quadratic convergence without assuming the strict complementary slackness condition and the regularity of the Jacobian of T 0 at a point z under reasonable conditions, so that Newton’s method can be used also far from a Kuhn–Tucker point
718 citations
TL;DR: IfF is monotone in a neighbourhood ofx, it is proved that 0 εδθ(x) is necessary and sufficient forx to be a solution of CP(F) and the generalized Newton method is shown to be locally well defined and superlinearly convergent with the order of 1+p.
Abstract: The paper deals with complementarity problems CP(F), where the underlying functionF is assumed to be locally Lipschitzian. Based on a special equivalent reformulation of CP(F) as a system of equationsź(x)=0 or as the problem of minimizing the merit functionź=1/2źźź22, we extend results which hold for sufficiently smooth functionsF to the nonsmooth case.
In particular, ifF is monotone in a neighbourhood ofx, it is proved that 0 źźź(x) is necessary and sufficient forx to be a solution of CP(F). Moreover, for monotone functionsF, a simple derivative-free algorithm that reducesź is shown to possess global convergence properties. Finally, the local behaviour of a generalized Newton method is analyzed. To this end, the result by Mifflin that the composition of semismooth functions is again semismooth is extended top-order semismooth functions. Under a suitable regularity condition and ifF isp-order semismooth the generalized Newton method is shown to be locally well defined and superlinearly convergent with the order of 1+p.
296 citations
TL;DR: The quadratic convergence of the proposed Newton method for the nearest correlation matrix problem is proved, which confirms the fast convergence and the high efficiency of the method.
Abstract: The nearest correlation matrix problem is to find a correlation matrix which is closest to a given symmetric matrix in the Frobenius norm. The well-studied dual approach is to reformulate this problem as an unconstrained continuously differentiable convex optimization problem. Gradient methods and quasi-Newton methods such as BFGS have been used directly to obtain globally convergent methods. Since the objective function in the dual approach is not twice continuously differentiable, these methods converge at best linearly. In this paper, we investigate a Newton-type method for the nearest correlation matrix problem. Based on recent developments on strongly semismooth matrix valued functions, we prove the quadratic convergence of the proposed Newton method. Numerical experiments confirm the fast convergence and the high efficiency of the method.
288 citations
Cites methods from "Extension of newton and quasi-newto..."
...In extending Kojima and Shindo’s condition for superlinear (quadratic) convergence of Newton’s method for piecewise smooth equations [30], Kummer [32] proposed a general condition for guaranteeing the superlinear convergence of (9)....
[...]
References
More filters
Book•
01 Jun 1970TL;DR: In this article, the authors present a list of basic reference books for convergence of Minimization Methods in linear algebra and linear algebra with a focus on convergence under partial ordering.
Abstract: Preface to the Classics Edition Preface Acknowledgments Glossary of Symbols Introduction Part I. Background Material. 1. Sample Problems 2. Linear Algebra 3. Analysis Part II. Nonconstructive Existence Theorems. 4. Gradient Mappings and Minimization 5. Contractions and the Continuation Property 6. The Degree of a Mapping Part III. Iterative Methods. 7. General Iterative Methods 8. Minimization Methods Part IV. Local Convergence. 9. Rates of Convergence-General 10. One-Step Stationary Methods 11. Multistep Methods and Additional One-Step Methods Part V. Semilocal and Global Convergence. 12. Contractions and Nonlinear Majorants 13. Convergence under Partial Ordering 14. Convergence of Minimization Methods An Annotated List of Basic Reference Books Bibliography Author Index Subject Index.
7,669 citations
TL;DR: In this article, the authors discuss certain modifications to Newton's method designed to reduce the number of function evaluations required during the iterative solution process of an iterative problem solving problem, such that the most efficient process will be that which requires the smallest number of functions evaluations.
Abstract: solution. The functions that require zeroing are real functions of real variables and it will be assumed that they are continuous and differentiable with respect to these variables. In many practical examples they are extremely complicated anld hence laborious to compute, an-d this fact has two important immediate consequences. The first is that it is impracticable to compute any derivative that may be required by the evaluation of the algebraic expression of this derivative. If derivatives are needed they must be obtained by differencing. The second is that during any iterative solution process the bulk of the computing time will be spent in evaluating the functions. Thus, the most efficient process will tenid to be that which requires the smallest number of function evaluations. This paper discusses certain modificatioins to Newton's method designed to reduce the number of function evaluations required. Results of various numerical experiments are given and conditions under which the modified versions are superior to the original are tentatively suggested.
2,481 citations
TL;DR: In this paper, an attempt to motivate and justify quasi-Newton methods as useful modifications of Newton''s method for general and gradient nonlinear systems of equations is made, and references are given to ample numerical justification; here we give an overview of many of the important theoretical results.
Abstract: This paper is an attempt to motivate and justify quasi-Newton methods as useful modifications of Newton''s method for general and gradient nonlinear systems of equations. References are given to ample numerical justification; here we give an overview of many of the important theoretical results and each is accompanied by sufficient discussion to make the results and hence the methods plausible. Key Words and Phrases: unconstrained minimization, nonlinear simultaneous equations, update methods, quasi-Newton methods.
1,435 citations
TL;DR: A regularity condition is introduced for generalized equations and it is shown to be in a certain sense the weakest possible condition under which the stated properties will hold.
Abstract: This paper considers generalized equations, which are convenient tools for formulating problems in complementarity and in mathematical programming, as well as variational inequalities. We introduce a regularity condition for such problems and, with its help, prove existence, uniqueness and Lipschitz continuity of solutions to generalized equations with parametric data. Applications to nonlinear programming and to other areas are discussed, and for important classes of such applications the regularity condition given here is shown to be in a certain sense the weakest possible condition under which the stated properties will hold.
975 citations
"Extension of newton and quasi-newto..." refers methods in this paper
...There have been developed some extensions of the Newton and the quasi-Newton methods; Josephy [8] [9] for strongly regular generalized equations (Robinson [19]), Pang and Chan [18] for variational inequalities including complementarity problems, etc .....
[...]
TL;DR: In this paper, simple constructive proofs are given of solutions to the matric matric system Mz − ω = q; z ≧ 0; ω ≧ 1; zT = 0, for various kinds of data M, q, which embrace quadratic programming and the problem of finding equilibrium points of bimatrix games.
Abstract: Some simple constructive proofs are given of solutions to the matric system Mz − ω = q; z ≧ 0; ω ≧ 0; and zT ω = 0, for various kinds of data M, q, which embrace the quadratic programming problem and the problem of finding equilibrium points of bimatrix games. The general scheme is, assuming non-degeneracy, to generate an adjacent extreme point path leading to a solution. The scheme does not require that some functional be reduced.
966 citations
"Extension of newton and quasi-newto..." refers methods in this paper
...We note that Murty [15] has given an example of a linear complementarity problem which requires well-known Lemke's method (Lemke [12]) to consume an exponential order of arithmetic operations....
[...]