Quasi-Newton methods accelerate the steepest-descent technique for function minimization by using computational history to generate a sequence of approximations to the inverse of the Hessian matrix. This paper presents a class of approximating matrices as a function of a scalar parameter. The problem of optimal conditioning of these matrices under an appropriate norm as a function of the scalar parameter is investigated. A set of computational results verifies the superiority of the new methods arising from conditioning considerations to known methods.

/pdf/conditioning-of-quasi-newton-methods-for-function-4uqbwq4vmj.pdf

Conditioning of Quasi-Newton Methods for Function Minimization

We develop a one parameter family of variable metric updates by considering a fundamental decomposition of the Hessian that underlies Variable Metric Algorithms. The relationship with other Variable Metric Updates is discussed. Considerations based on the condition of the Hessian inverse approximation indicate particular choices of the parameter and these are discussed in the second half of this paper.

A family of variable metric updates

Variable Metric Methods are "Newton--Raphson-like" algorithms for unconstrained minimization in which the inverse Hessian is replaced by an approximation, inferred from previous gradients and updated at each iteration. During the past decade various approaches have been used to derive general classes of such algorithms having the common properties of being Conjugate Directions methods and having "quadratic termination". Observed differences in actual performance of such methods motivated recent attempts to identify variable metric algorithms having additional properties that may be significant in practical situations (e.g. nonquadratic functions, inaccurate linesearch, etc.). The SSVM algorithms, introduced by this first author, are such methods that among their other properties, they automatically compensate for poor scaling of the objective function. This paper presents some new theoretical results identifying a subclass of SSVM algorithms that have the additional property of minimizing a sharp bound on the condition number of the inverse Hessian approximation at each iteration. Reducing this condition number is important for decreasing the roundoff error. The theoretical properties of this subclass are explored and two of its special cases are tested numerically in comparison with other SSVM algorithms.

Optimal conditioning of self-scaling variable Metric algorithms

The structure sensitivity of enantioselective hydrogenations on chirally modified metals was investigated using Pt nanoparticles of different shapes. All three samples had an average particle size of 10 nm, but the fraction of dominantly cubic, cubooctahedral, and octahedral particles varied with decreasing {100} and increasing {111} faces in the same order. In the absence of chiral modifier the hydrogenation of ethyl pyruvate was independent of the shape of the Pt nanoparticles; variation of the specific reaction rates did not exceed the experimental error on all self-prepared catalysts and on a commercial Pt/Al2O3 used as reference. Addition of cinchonidine or quinine induced a significant rate enhancement by a factor of 4−15, and the rate was always higher with quinine. Also, 72−92% ees were achieved, and the reaction was shape selective: both the rate and the ee increased with increasing Pt{111}/Pt{100} ratio. A similar correlation in the hydrogenation of ketopantolactone confirmed that decarbonylatio...

Shape-selective enantioselective hydrogenation on Pt nanoparticles.

where g(k) = Vf(x(k)), the gradient of f a t x Ck), H (k) is a matr ix which is designed to approximate the inverse Hessian matr ix of f a t x (k), and a (k) is an appropr ia te ly chosen scalar. The sequence of matr ices H (k) is chosen to satisfy the quas i -Newton equation H ( k + l ) y (k) = a (k), where a (~) = x (k+~) x (~) and y(k) = g(k+~) _ g(k). I n general, H (k+~) is generated by H (k+~) = H (k) ~D (k), where D (k) is chosen to satisfy the equat ion D(k)y(k) = a(~) H(k)y(k) . The choice of a Ck) can be accomplished either by a linear-search or a step-length method. Perhaps the mos t complete current reference to the general theory is Powell [-8]. Two different methods for determining a (k) are incorporated and are outlined below. The Broyden-Fle tcher -Shanno (BFS) matr ix upda te (developed

Algorithm 500: Minimization of Unconstrained Multivariate Functions [E4]

Quasi-Newton methods accelerate gradient methods for minimizing a function by approximating the inverse Hessian matrix of the function. Several papers in recent literature have dealt with the generation of classes of approximating matrices as a function of a scalar parameter. This paper derives necessary and sufficient conditions on the range of one such parameter to guarantee stability of the method. It further shows that the parameter effects only the length, not the direction, of the search vector at each step, and uses this result to derive several computational algorithms. The algorithms are evaluated on a series of test problems.

/pdf/optimal-conditioning-of-quasi-newton-methods-2jlbeij2jq.pdf

P. C. Kettler

Papers

Optimal conditioning of quasi-Newton methods