Showing papers in "Mathematical Programming in 1989"

On the limited memory BFGS method for large scale optimization

Abstract: We study the numerical performance of a limited memory quasi-Newton method for large scale optimization, which we call the L-BFGS method. We compare its performance with that of the method developed by Buckley and LeNir (1985), which combines cycles of BFGS steps and conjugate direction steps. Our numerical tests indicate that the L-BFGS method is faster than the method of Buckley and LeNir, and is better able to use additional storage to accelerate convergence. We show that the L-BFGS method can be greatly accelerated by means of a simple scaling. We then compare the L-BFGS method with the partitioned quasi-Newton method of Griewank and Toint (1982a). The results show that, for some problems, the partitioned quasi-Newton method is clearly superior to the L-BFGS method. However we find that for other problems the L-BFGS method is very competitive due to its low iteration cost. We also study the convergence properties of the L-BFGS method, and prove global convergence on uniformly convex problems.

Some numerical experiments with variable-storage quasi-Newton algorithms

Abstract: This paper describes some numerical experiments with variable-storage quasi-Newton methods for the optimization of some large-scale models (coming from fluid mechanics and molecular biology). In addition to assessing these kinds of methods in real-life situations, we compare an algorithm of A. Buckley with a proposal by J. Nocedal. The latter seems generally superior, provided that careful attention is given to some nontrivial implementation aspects, which concern the general question of properly initializing a quasi-Newton matrix. In this context, we find it appropriate to use a diagonal matrix, generated by an update of the identity matrix, so as to fit the Rayleigh ellipsoid of the local Hessian in the direction of the change in the gradient. Also, a variational derivation of some rank one and rank two updates in Hilbert spaces is given.

Interior path following primal-dual algorithms. Part I: Linear programming

Abstract: We describe a primal-dual interior point algorithm for linear programming problems which requires a total of $$O\left( {\sqrt n L} \right)$$ number of iterations, whereL is the input size. Each iteration updates a penalty parameter and finds the Newton direction associated with the Karush-Kuhn-Tucker system of equations which characterizes a solution of the logarithmic barrier function problem. The algorithm is based on the path following idea.

An implementation of Karmarkar's algorithm for linear programming

Abstract: This paper describes the implementation of power series dual affine scaling variants of Karmarkar's algorithm for linear programming. Based on a continuous version of Karmarkar's algorithm, two variants resulting from first and second order approximations of the continuous trajectory are implemented and tested. Linear programs are expressed in an inequality form, which allows for the inexact computation of the algorithm's direction of improvement, resulting in a significant computational advantage. Implementation issues particular to this family of algorithms, such as treatment of dense columns, are discussed. The code is tested on several standard linear programming problems and compares favorably with the simplex codeMinos 4.0.

The boolean quadric polytope: Some characteristics, facets and relatives

Abstract: We study unconstrained quadratic zero---one programming problems havingn variables from a polyhedral point of view by considering the Boolean quadric polytope QPn inn(n+1)/2 dimensions that results from the linearization of the quadratic form. We show that QPn has a diameter of one, descriptively identify three families of facets of QPn and show that QPn is symmetric in the sense that all facets of QPn can be obtained from those that contain the origin by way of a mapping. The naive linear programming relaxation QPnLP of QPn is shown to possess the Trubin-property and its extreme points are shown to be {0,1/2,1}-valued. Furthermore, O(n3) facet-defining inequalities of QPn suffice to cut off all fractional vertices of QPnLP, whereas the family of facets described by us has at least O(3n) members. The problem is also studied for sparse quadratic forms (i.e. when several or many coefficients are zero) and conditions are given for the previous results to carry over to this case. Polynomially solvable problem instances are discussed and it is shown that the known polynomiality result for the maximization of nonnegative quadratic forms can be obtained by simply rounding the solution to the linear programming relaxation. In the case that the graph induced by the nonzero coefficients of the quadratic form is series-parallel, a complete linear description of the associated Boolean quadric polytope is given. The relationship of the Boolean quadric polytope associated to sparse quadratic forms with the vertex-packing polytope is discussed as well.

A polynomial-time algorithm for a class of linear complementary problems

Abstract: Given ann × n matrixM and ann-dimensional vectorq, the problem of findingn-dimensional vectorsx andy satisfyingy = Mx + q, x ≥ 0,y ≥ 0,xiyi = 0 (i = 1, 2,⋯,n) is known as a linear complementarity problem. Under the assumption thatM is positive semidefinite, this paper presents an algorithm that solves the problem in O(n3L) arithmetic operations by tracing the path of centers,{(x, y) ∈ S: xiyi =μ (i = 1, 2,⋯,n) for some μ > 0} of the feasible regionS = {(x, y) ≥ 0:y = Mx + q}, whereL denotes the size of the input data of the problem.

A cutting plane algorithm for a clustering problem

Abstract: In this paper we consider a clustering problem that arises in qualitative data analysis. This problem can be transformed to a combinatorial optimization problem, the clique partitioning problem. We have studied the latter problem from a polyhedral point of view and determined large classes of facets of the associated polytope. These theoretical results are utilized in this paper. We describe a cutting plane algorithm that is based on the simplex method and uses exact and heuristic separation routines for some of the classes of facets mentioned before. We discuss some details of the implementation of our code and present our computational results. We mention applications from, e.g., zoology, economics, and the political sciences.

Interior path following primal-dual algorithms. Part II: Convex quadratic programming

Abstract: We describe a primal-dual interior point algorithm for convex quadratic programming problems which requires a total of\(O\left( {\sqrt n L} \right)\) number of iterations, whereL is the input size. Each iteration updates a penalty parameter and finds an approximate Newton direction associated with the Karush-Kuhn-Tucker system of equations which characterizes a solution of the logarithmic barrier function problem. The algorithm is based on the path following idea. The total number of arithmetic operations is shown to be of the order of O(n3L).

An extension of Karmarkar projective algorithm for convex quadratic programming

Abstract: We present an extension of Karmarkar's linear programming algorithm for solving a more general group of optimization problems: convex quadratic programs. This extension is based on the iterated application of the objective augmentation and the projective transformation, followed by optimization over an inscribing ellipsoid centered at the current solution. It creates a sequence of interior feasible points that converge to the optimal feasible solution in O(Ln) iterations; each iteration can be computed in O(Ln3) arithmetic operations, wheren is the number of variables andL is the number of bits in the input. In this paper, we emphasize its convergence property, practical efficiency, and relation to the ellipsoid method.

Experiments in quadratic 0-1 programming

Abstract: We present computational experience with a cutting plane algorithm for 0–1 quadratic programming without constraints. Our approach is based on a reduction of this problem to a max-cut problem in a graph and on a partial linear description of the cut polytope.

A robust sequential quadratic programming method

Abstract: The sequential quadratic programming method developed by Wilson, Han and Powell may fail if the quadratic programming subproblems become infeasible, or if the associated sequence of search directions is unbounded. This paper considers techniques which circumvent these difficulties by modifying the structure of the constraint region in the quadratic programming subproblems. Furthermore, questions concerning the occurrence of an unbounded sequence of multipliers and problem feasibility are also addressed.

A practical anti-cycling procedure for linearly constrained optimization

Abstract: A procedure is described for preventing cycling in active-set methods for linearly constrained optimization, including the simplex method. The key ideas are a limited acceptance of infeasibilities in all variables, and maintenance of a “working” feasibility tolerance that increases over a long sequence of iterations. The additional work per iteration is nominal, and “stalling” cannot occur with exact arithmetic. The method appears to be reliable, based on computational results for the first 53 linear programming problems in theNetlib set.

A tolerant algorithm for linearly constrained optimization calculations

Abstract: Two extreme techniques when choosing a search direction in a linearly constrained optimization calculation are to take account of all the constraints or to use an active set method that satisfies selected constraints as equations, the remaining constraints being ignored. We prefer an intermediate method that treats all inequality constraints with “small” residuals as inequalities with zero right hand sides and that disregards the other inequality conditions. Thus the step along the search direction is not restricted by any constraints with small residuals, which can help efficiency greatly, particularly when some constraints are nearly degenerate. We study the implementation, convergence properties and performance of an algorithm that employs this idea. The implementation considerations include the choice and automatic adjustment of the tolerance that defines the “small” residuals, the calculation of the search directions, and the updating of second derivative approximations. The main convergence theorem imposes no conditions on the constraints except for boundedness of the feasible region. The numerical results indicate that a Fortran implementation of our algorithm is much more reliable than the software that was tested by Hock and Schittkowski (1981). Therefore the algorithm seems to be very suitable for general use, and it is particularly appropriate for semi-infinite programming calculations that have many linear constraints that come from discretizations of continua.

Facets and algorithms for capacitated lot sizing

Abstract: The dynamic economic lot sizing model, which lies at the core of numerous production planning applications, is one of the most highly studied models in all of operations research. And yet, capacitated multi-item versions of this problem remain computationally elusive. We study the polyhedral structure of an integer programming formulation of a single-item capacitated version of this problem, and use these results to develop solution methods for multi-item applications. In particular, we introduce a set of valid inequalities for the problem and show that they define facets of the underlying integer programming polyhedron. Computational results on several single and multiple product examples show that these inequalities can be used quite effectively to develop an efficient cutting plane/branch and bound procedure. Moreover, our results show that in many instances adding certain of these inequalities a priori to the problem formulation, and avoiding the generation of cutting planes, can be equally effective.

Nonlinear programming and nonsmooth optimization by successive linear programming

Abstract: Methods are considered for solving nonlinear programming problems using an exactl1 penalty function. LP-like subproblems incorporating a trust region constraint are solved successively both to estimate the active set and to provide a foundation for proving global convergence. In one particular method, second order information is represented by approximating the reduced Hessian matrix, and Coleman-Conn steps are taken. A criterion for accepting these steps is given which enables the superlinear convergence properties of the Coleman-Conn method to be retained whilst preserving global convergence and avoiding the Maratos effect. The methods generalize to solve a wide range of composite nonsmooth optimization problems and the theory is presented in this general setting. A range of numerical experiments on small test problems is described.

Open questions concerning Weiszfeld's algorithm for the Fermat-Weber location problem

Abstract: The Fermat—Weber location problem is to find a point in ℝ n that minimizes the sum of the weighted Euclidean distances fromm given points in ℝ n . A popular iterative solution method for this problem was first introduced by Weiszfeld in 1937. In 1973 Kuhn claimed that if them given points are not collinear then for all but a denumerable number of starting points the sequence of iterates generated by Weiszfeld's scheme converges to the unique optimal solution. We demonstrate that Kuhn's convergence theorem is not always correct. We then conjecture that if this algorithm is initiated at the affine subspace spanned by them given points, the convergence is ensured for all but a denumerable number of starting points.

Valid inequalities and facets of the capacitated plant location problem

Abstract: Recently, several successful applications of strong cutting plane methods to combinatorial optimization problems have renewed interest in cutting plane methods, and polyhedral characterizations, of integer programming problems. In this paper, we investigate the polyhedral structure of the capacitated plant location problem. Our purpose is to identify facets and valid inequalities for a wide range of capacitated fixed charge problems that contain this prototype problem as a substructure.

On the set covering polytope: I. all the facets with coefficients in {0, 1, 2}

Abstract: While the set packing polytope, through its connection with vertex packing, has lent itself to fruitful investigations, little is known about the set covering polytope. We characterize the class of valid inequalities for the set covering polytope with coefficients equal to 0, 1 or 2, and give necessary and sufficient conditions for such an inequality to be minimal and to be facet defining. We show that all inequalities in the above class are contained in the elementary closure of the constraint set, and that 2 is the largest value ofk such that all valid inequalities for the set covering polytope with integer coefficients no greater thank are contained in the elementary closure. We point out a connection between minimal inequalities in the class investigated and certain circulant submatrices of the coefficient matrix. Finally, we discuss conditions for an inequality to cut off a fractional solution to the linear programming relaxation of the set covering problem and to improve the lower bound given by a feasible solution to the dual of the linear programming relaxation.

On the facial structure of the set covering polytope dimensional linear programming

Abstract: Given a bipartite graphG = (V, U, E), a cover ofG is a subset\(D \subseteq V\) with the property that each nodeu ∈ U is adjacent to at least one nodev ∈D. If a positive weightcv is associated with each nodev ∈V, the covering problem (CP) is to find a cover ofG having minimum total weight.

Algorithms for proportional matrices in reals and integers

Abstract: Let R be the set of nonnegative matrices whose row and column sums fall between specific limits and whose entries sum to some fixed h > 0. Closely related axiomatic approaches have been developed to ascribe meanings to the statements: the real matrix fe R and the integer matrix a ~ R are "proportional to" a given matrix p ~> 0. These approaches are described, conditions under which proportional solutions exist are characterized, and algorithms are given for finding proportional solutions in each case.

Pure adaptive search in Monte Carlo optimization

Abstract: Pure adaptive search constructs a sequence of points uniformly distributed within a corresponding sequence of nested regions of the feasible space. At any stage, the next point in the sequence is chosen uniformly distributed over the region of feasible space containing all points that are equal or superior in value to the previous points in the sequence. We show that for convex programs the number of iterations required to achieve a given accuracy of solution increases at most linearly in the dimension of the problem. This compares to exponential growth in iterations required for pure random search.

A direct active set algorithm for large sparse quadratic programs with simple bounds

Abstract: We show how a direct active set method for solving definite and indefinite quadratic programs with simple bounds can be efficiently implemented for large sparse problems. All of the necessary factorizations can be carried out in a static data structure that is set up before the numeric computation begins. The space required for these factorizations is no larger than that required for a single sparse Cholesky factorization of a matrix with the same sparsity structure as the Hessian of the quadratic. We propose several improvements to this basic algorithm: a new way to find a search direction in the indefinite case that allows us to free more than one variable at a time and a new heuristic method for finding a starting point. These ideas are motivated by the two-norm trust region problem. Additionally, we also show how projection techniques can be used to add several constraints to the active set at each iteration. Our experimental results show that an algorithm with these improvements runs much faster than the basic algorithm for positive definite problems and finds local minima with lower function values for indefinite problems.

Conical projection algorithms for linear programming

Abstract: The Linear Programming Problem is manipulated to be stated as a Non-Linear Programming Problem in which Karmarkar's logarithmic potential function is minimized in the positive cone generated by the original feasible set. The resulting problem is then solved by a master algorithm that iteratively rescales the problem and calls an internal unconstrained non-linear programming algorithm. Several different procedures for the internal algorithm are proposed, giving priority either to the reduction of the potential function or of the actual cost. We show that Karmarkar's algorithm is equivalent to this method in the special case in which the internal algorithm is reduced to a single steepest descent iteration. All variants of the new algorithm have the same complexity as Karmarkar's method, but the amount of computation is reduced by the fact that only one projection matrix must be calculated for each call of the internal algorithm.

On the 0, 1 facets of the set covering polytope

Abstract: In this paper, we consider inequalities of the formΣ αjxj ≥ β, whereαj equals 0 or 1, andβ is a positive integer. We give necessary and sufficient conditions for such inequalities to define facets of the set covering polytope associated with a 0, 1 constraint matrixA. These conditions are in terms of critical edges and critical cutsets defined in the bipartite incidence graph ofA, and are in the spirit of the work of Balas and Zemel on the set packing problem where similar notions were defined in the intersection graph ofA. Furthermore, we give a polynomial characterization of a class of 0, 1 facets defined from chorded cycles of the bipartite incidence graph. This characterization also yields all the 0, 1 liftings of odd-hole inequalities for the simple plant location polytope.

A new continuation method for complementarity problems with uniform P -functions

Abstract: The complementarity problem with a nonlinear continuous mappingf from the nonnegative orthantR + n ofR n intoR n can be written as the system of equationsF(x, y) = 0 and(x, y) ∈ R + 2n , whereF denotes the mapping from the nonnegative orthantR + 2n ofR 2n intoR + n × Rn defined byF(x, y) = (x 1y1,⋯,xnyn, f1(x) − y1,⋯, fn(x) − yn) for every(x, y) ∈ R + 2n Under the assumption thatf is a uniformP-function, this paper establishes that the mappingF is a homeomorphism ofR + 2n ontoR + n × Rn This result provides a theoretical basis for a new continuation method of tracing the solution curve of the one parameter family of systems of equationsF(x, y) = tF(x 0, y0) and(x, y) ∈ R + 2n from an arbitrary initial point(x 0, y0) ∈ R + 2n witht = 1 until the parametert attains 0 This approach is an extension of the one used in the polynomially bounded algorithm recently given by Kojima, Mizuno and Yoshise for solving linear complementarity problems with positive semi-definite matrices

On the set covering polytope: II. lifting the facets with coefficients in {0, 1, 2}

Abstract: In an earlier paper (Mathematical Programming 43 (1989) 57–69) we characterized the class of facets of the set covering polytope defined by inequalities with coefficients equal to 0, 1 or 2. In this paper we connect that characterization to the theory of facet lifting. In particular, we introduce a family of lower dimensional polytopes and associated inequalities having only three nonzero coefficients, whose lifting yields all the valid inequalities in the above class, with the lifting coefficients given by closed form expressions.

Sublinear upper bounds for stochastic programs with recourse

Abstract: Separable sublinear functions are used to provide upper bounds on the recourse function of a stochastic program. The resulting problem's objective involves the inf-convolution of convex functions. A dual of this problem is formulated to obtain an implementable procedure to calculate the bound. Function evaluations for the resulting convex program only require a small number of single integrations in contrast with previous upper bounds that require a number of function evaluations that grows exponentially in the number of random variables. The sublinear bound can often be used when other suggested upper bounds are intractable. Computational results indicate that the sublinear approximation provides good, efficient bounds on the stochastic program objective value.

On the supermodular knapsack problem

Abstract: In this paper we introduce binary knapsack problems where the objective function is nonlinear, and investigate their Lagrangean and continuous relaxations. Some of our results generalize previously known theorems concerning linear and quadratic knapsack problems. We investigate in particular the case in which the objective function is supermodular. Under this hypothesis, although the problem remains NP-hard, we show that its Lagrangean dual and its continuous relaxation can be solved in polynomial time. We also comment on the complexity of recognizing supermodular functions. The particular case in which the knapsack constraint is of the cardinality type is also addressed and some properties of its optimal value as a function of the right hand side are derived.

Recognition problems for special classes of polynomials in 0-1 variables

Abstract: This paper investigates the complexity of various recognition problems for pseudo-Boolean functions (i.e., real-valued functions defined on the unit hypercubeBn = {0, 1}n), when such functions are represented as multilinear polynomials in their variables. Determining whether a pseudo-Boolean function (a) is monotonic, or (b) is supermodular, or (c) is threshold, or (d) has a unique local maximum in each face ofBn, or (e) has a unique local maximum inBn, is shown to be NP-hard. A polynomial-time recognition algorithm is presented for unimodular functions, previously introduced by Hansen and Simeone as a class of functions whose maximization overBn is reducible to a network minimum cut problem.