This paper studies projected Barzilai-Borwein (PBB) methods for large-scale box-constrained quadratic programming. Recent work on this method has modified the PBB method by incorporating the Grippo-Lampariello-Lucidi (GLL) nonmonotone line search, so as to enable global convergence to be proved. We show by many numerical experiments that the performance of the PBB method deteriorates if the GLL line search is used. We have therefore considered the question of whether the unmodified method is globally convergent, which we show not to be the case, by exhibiting a counter example in which the method cycles. A new projected gradient method (PABB) is then considered that alternately uses the two Barzilai-Borwein steplengths. We also give an example in which this method may cycle, although its practical performance is seen to be superior to the PBB method. With the aim of both ensuring global convergence and preserving the good numerical performance of the unmodified methods, we examine other recent work on nonmonotone line searches, and propose a new adaptive variant with some attractive features. Further numerical experiments show that the PABB method with the adaptive line search is the best BB-like method in the positive definite case, and it compares reasonably well against the GPCG algorithm of More and Toraldo. In the indefinite case, the PBB method with the adaptive line search is shown on some examples to find local minima with better solution values, and hence may be preferred for this reason.

/pdf/projected-barzilai-borwein-methods-for-large-scale-box-3maqkglmvr.pdf

Projected Barzilai-Borwein methods for large-scale box-constrained quadratic programming

The Barzilai-Borwein (BB) gradient method, and some other new gradient methods have shown themselves to be competitive with conjugate gradient methods for solving large dimension nonlinear unconstrained optimization problems. Little is known about the asymptotic behaviour, even when applied to n−dimensional quadratic functions, except in the case that n=2. We show in the quadratic case how it is possible to compute this asymptotic behaviour, and observe that as n increases there is a transition from superlinear to linear convergence at some value of n≥4, depending on the method. By neglecting certain terms in the recurrence relations we define simplified versions of the methods, which are able to predict this transition. The simplified methods also predict that for larger values of n, the eigencomponents of the gradient vectors converge in modulus to a common value, which is a similar to a property observed to hold in the real methods. Some unusual and interesting recurrence relations are analysed in the course of the study.

On the asymptotic behaviour of some new gradient methods

Finite Algorithms in Optimization and Data Analysis.

Finite Algorithms in Optimization and Data Analysis

Due to its simplicity and efficiency, the Barzilai and Borwein (BB) gradi- ent method has received various attentions in different fields. This paper presents a new analysis of the BB method for two-dimensional strictly convex quadratic func- tions. The analysis begins with the assumption that the gradient norms at the first two iterations are fixed. We show that there is a superlinear convergence step in at most three consecutive steps. Meanwhile, we provide a better convergence relation for the BB method. The influence of the starting point and the condition number to the convergence rate is comprehensively addressed.

/pdf/a-new-analysis-on-the-barzilai-borwein-gradient-method-3ggwiqhlo5.pdf

A New Analysis on the Barzilai-Borwein Gradient Method

http://atarazanas.sci.uma.es/docs/articulos/1651208x.pdf

The sagitta method for solving linear programs

The dual Phase-1 algorithm using the most-obtuse-angle row pivot rule is very efficient for providing a dual feasible basis, in either the classical or the basis-deficiency-allowing context. In this paper, we establish a basis-deficiency-allowing Phase-I algorithm using the so-called most-obtuse-angle column pivot rule to produce a primal (deficient or full) basis. Our computational experiments with the smallest test problems from the standard NETLIB set show that a dense projected-gradient implementation largely outperforms that of the variation of the primal simplex method from the commercial code MATLAB LINPROG v1.17, and that a sparse projected-gradient implementation of a normalized revised version of the proposed algorithm runs 34% faster than the sparse implementation of the primal simplex method included in the commercial code TOMLAB LPSOLVE v3.0.

http://www.matap.uma.es/investigacion/tr/ma05_01.pdf

A basis-deficiency-allowing primal phase-I algorithm using the most-obtuse-angle column rule

http://www.matap.uma.es/investigacion/tr/ma03_01.pdf

Phase I cycling under the most-obtuse-angle pivot rule ?

In a recent primal-dual simplex-type algorithm (K. Paparrizos, N. Samaras and G. Stephanides, A new efficient primal dual simplex algorithm, Computers & Operations Research 30 (2003), pp. 1383-1399), its authors show how to take advantage of the knowledge of a primal feasible point and they work with a square basis during the whole process. In this paper we address what could be thought of as its deficient-basis dual counterpart by showing how to take advantage of the knowledge of a dual feasible point in a deficient-basis simplex-type environment. Three small examples are given to illustrate how the specific pivoting rules designed for the proposed algorithm deal with non-unique dual solutions, unbounded dual objectives and a classical exponential example by Goldfarb, thus avoiding some caveats of the dual simplex method. Practical experiments with a new collection of difficult problems for the dual simplex method are reported to justify iteration decrease, and we sketch some details of a sparse projected-gradient implementation in terms of Davis and Hager's CHOLMOD sparse Cholesky factorization (which is row updatable/downdatable) to solve the underlying least-squares subproblems, namely linear least-squares problems and projections onto linear manifolds.

http://www.matap.uma.es/investigacion/tr/ma07_01r.pdf

A deficient-basis dual counterpart of Paparrizos, Samaras and Stephanides' primal-dual simplex-type algorithm

We describe how to update and downdate an upper trapezoidal sparse orthogonal factorization, namely the sparse QR factorization of A T k where A k is a 'tall and thin' full column rank matrix formed with a subset of the columns of a fixed matrix A. In order to do this, we have adapted Saunders' techniques of the early 1970s for square matrices, to rectangular matrices (with fewer columns than rows) by using the static data structure of George and Heath of the early 1980s but allowing row downdating on it. An implicitly determined column permutation allows us to dispense with computing a new ordering after each update/downdate; it fits well into the LINPACK downdating algorithm and ensures that the updated trapezoidal factor will remain sparse. We give all the necessary formulae even if the orthogonal factor is not available, and we comment on our implementation using the sparse toolbox of MATLAB 5.

/pdf/updating-and-downdating-an-upper-trapezoidal-sparse-18knwud27t.pdf

Angel Santos-Palomo

Papers

The sagitta method for solving linear programs

A basis-deficiency-allowing primal phase-I algorithm using the most-obtuse-angle column rule

Phase I cycling under the most-obtuse-angle pivot rule ?

A deficient-basis dual counterpart of Paparrizos, Samaras and Stephanides' primal-dual simplex-type algorithm

Updating and downdating an upper trapezoidal sparse orthogonal factorization