Showing papers in "Optimization Methods & Software in 2001"
TL;DR: Computational results show that clustered concave minimization yields test set improvement as high as 20.4% over a linear support vector machine trained on a correspondingly small but randomly chosen subset that is labeled by an expert.
Abstract: A concave minimization approach is proposed for classifying unlabeled data based on the following ideas: (i) A small representative percentage (5% to 10%) of the unlabeled data is chosen by a clustering algorithm and given to an expert or oracle to label, (ii) A linear support vector machine is trained using the small labeled sample while simultaneously assigning the remaining bulk of the unlabeled dataset to one of two classes so as to maximize the margin (distance) between the two bounding planes that determine the separating plane midway between them. This latter problem is formulated as a concave minimization problem on a polyhedral set for which a stationary point is quickly obtained by solving a few (5 to 7) linear programs. Such stationary points turn out to be very effective as evidenced by our computational results which show that clustered concave minimization yields: (a) Test set improvement as high as 20.4% over a linear support vector machine trained on a correspondingly small but randomly ch...
224 citations
TL;DR: A SOCP relaxation method is proposed, which strengthens the lift-and-project LP (linear programming) relaxation method by adding convex quadratic valid inequalities for the positive semidefinite cone involved in the SDP relaxation.
Abstract: A disadvantage of the SDP (semidefinite programming) relaxation method for quadratic and/or combinatorial optimization problems lies in its expensive computational cost. This paper proposes a SOCP (second-order-cone programming) relaxation method, which strengthens the lift-and-project LP (linear programming) relaxation method by adding convex quadratic valid inequalities for the positive semidefinite cone involved in the SDP relaxation. Numerical experiments show that our SOCP relaxation is a reasonable compromise between the effectiveness of the SDP relaxation and the low computational cost of the lift-and-project LP relaxation.
134 citations
TL;DR: A branch-and-bound algorithm for the quadratic assignment problem (QAP) that uses a convex quadratics programming (QP) relaxation to obtain a bound at each node to obtain state-of-the-art computational results on large benchmark QAPs.
Abstract: We describe a branch-and-bound algorithm for the quadratic assignment problem (QAP) that uses a convex quadratic programming (QP) relaxation to obtain a bound at each node. The QP subproblems are approximately solved using the Frank-Wolfe algorithm, which in this case requires the solution of a linear assignment problem on each iteration. Our branching strategy makes extensive use of dual information associated with the QP subproblems. We obtain state-of-the-art computational results on large benchmark QAPs
78 citations
TL;DR: A projected gradient algorithm for solving the semidefinite programming (SDP) relaxation of the maximum cut (maxcut) problem is presented and combined with a randomized method this gives a very efficient approximation algorithm for the maxcut problem.
Abstract: In this paper, we present a projected gradient algorithm for solving the semidefinite programming (SDP) relaxation of the maximum cut (maxcut) problem. Coupled with a randomized method, this gives a very efficient approximation algorithm for the maxcut problem. We report computational results comparing our method with two earlier successful methods on problems with dimension up to 7,000.
78 citations
TL;DR: In this article, the Lagrange functions of interpolation to function values have been used to control the adjustment of a trust region radius in a way that gives excellent convergence properties in an algorithm for unconstrained minimization calculations.
Abstract: Quadratic models are of fundamental importance to the efficiency of many optimization algorithms when second derivatives of the objective function influence the required values of the variables. They may be constructed by interpolation to function values for suitable choices of the interpolation points. We consider the Lagrange functions of this technique, because they have some highly useful properties. In particular, they show whether a change to an interpolation point preserves nonsingularity of the interpolation equations, and they provide a bound on the error of the quadratic model. Further, they can be updated efficiently when an interpolation point is moved. These features are explained. Then it is shown that the error bound can control the adjustment of a trust region radius in a way that gives excellent convergence properties in an algorithm for unconstrained minimization calculations. Finally, a convenient procedure for generating the initial interpolation points is described.
46 citations
TL;DR: In this article, the authors present a constructive scheme based on the parallelotopic estimates of trajectory tubes for solving the feedback terminal target control problems for linear dynamic systems under uncertainty and without that.
Abstract: The constructive schemes based on the parallelotopic estimates of trajectory tubes are presented for solving the feedback terminal target control problems for linear dynamic systems under uncertainty and without that. The families of external and internal parallelotopic estimates for solvability tubes are introduced. Each particular approximate tube satisfies the generalized semigroup property and may be calculated independently of the others. This further allows to parallelize computations. The introduced families ensure exact representations of the tube sections (through intersections and unions) for the case without uncertainty. Locally volume-optimal external and internal estimates are found. The introduced internal estimates are the “parallelotopic solvability tubes”. This fact allows to synthesize a control strategy in an analytical form on the base of a solution of some specific mathematical programming problem. The results of numerical modelling are presented.
30 citations
TL;DR: In this paper, an alternative form of the Farkas Lemma is proposed and it is shown that many well-known theorems of the alternative are simply special cases of this alternative form and can be proved from it.
Abstract: In this paper we prove an alternative form of the Farkas Lemma and show that many well-known theorems of the alternative are simply special cases of this alternative form and can be proved from it by simple substitution.
29 citations
TL;DR: The Gauss-Newton solution as mentioned in this paper is a defining system of equations that is essentially different from the AHO system, which is the over-determined system obtained from the minimization of a nonlinear equation.
Abstract: Most of the directions used in practical interior-point methods for semideinite programming try to follow the approach used in linear programming, i.e.f they are defined using the optimally conditions which are modified with a symmetrization of the perturbed complementarity conditions to allow for application of Newton's method. It is now understood that all the Moeteiro-Zhang family, which include, among others, the popular AHO, NT, HKM, Gu, and Toh directions, can be expressed as a scaling of the problem data and of the iterate followed by the solution of the AHO system of equations, followed by the inverse scaling. All these directions therefore share a defining system of equations. The focus of this work is to propose a defining system of equations that is essentially different from the AHO system: the over-determined system obtained from the minimization of a nonlinear equation. The resulting solution is called the Gauss-Newton
25 citations
TL;DR: In this article, an optimal control problem of a linear elliptic equation with a functional containing a supremum norm term is considered, where the control acts on the boundary and the control problem is transformed into a linear (resp. linear quadratic) programming problem.
Abstract: We consider an optimal control problem of a linear elliptic equation with a functional containing a supremum norm term. The control acts on the boundary. Necessary first order optimality conditions are derived for problems with pointwise control and state constraints. For this purpose the original problem is substituted by an equivalent problem with a differentiable functional. In a second part we discuss a numerical approach to such problems. The control problem is transformed into a linear (resp. linear quadratic) programming problem. In a particular situation we can compare the numerical results with the analytic solutions.
19 citations
TL;DR: A branch and cut algorithm is developed for solving convex 0-1 Mixed Integer Nonlinear Programming (MINLP) problems and the addition of Gomory cuts can reduce the number of nodes in the enumeration tree.
Abstract: A branch and cut algorithm is developed for solving convex 0-1 Mixed Integer Nonlinear Programming (MINLP) problems. The algorithm integrates Branch and Bound, Outer Approximation and Gomory Cutting Planes. Only the initial Mixed Integer Linear Programming (MILP) master problem is considered. At integer solutions Nonlinear Programming (NLP) problems are solved, using a primal-dual interior point algorithm. The objective and constraints are linearized at the optimum solution of these NLP problems and the linearizations are added to all the unsolved nodes of the enumeration tree. Also, Gomory cutting planes, which are valid throughout the tree, are generated at selected nodes. These cuts help the algorithm to locate integer solutions quickly and consequently improve the linear approximation of the objective and constraints held at the unsolved nodes of the tree. Numerical results show that the addition of Gomory cuts can reduce the number of nodes in the enumeration tree.
15 citations
TL;DR: In this article, a regularized logarithmic barrier method for solving convex semi-infinite programming problems is considered, where a multi-step proximal regularization is coupled with an adaptive discretization strategy in the framework of an interior point approach.
Abstract: A regularized logarithmic Barrier method for solving (ill-posed) convex semi-infinite programming problems is considered. In this method a multi-step proximal regularization is coupled with an adaptive discretization strategy in the framework of an interior point approach. Termination of the proximal iterations at each discretization level is controlled by means of estimates, characterizing the efficiency of these iterations. A special deleting rule permits to use only a part of the constraints of the discretized problems. Convergence of the method and its stability with respect to data perturbations in the cone of convex C 1-functions are studied as well as some numerical experiments are presented.
TL;DR: In this paper, the exactness of auxiliary functions is studied and sufficient conditions for their exactness are given, without assumption that the Lagrange function has a saddle point, and some examples of exact auxiliary functions are given.
Abstract: This paper is a continuation of [13]. For each constrained optimization problem we consider certain unconstrained problems, which are constructed by means of auxiliary (Lagrange-type) functions. We study only exact auxiliary functions, it means that the set of their global minimizers coincides with the solution set of the primal constrained optimization problem. Sufficient conditions for the exactness of an auxiliary function are given. These conditions are obtained without assumption that the Lagrange function has a saddle point. Some examples of exact auxiliary functions are given.
TL;DR: In this paper some theorems which characterize the global optimal solutions of nonlinear programming problems are proved and two algorithms are derived using these results.
Abstract: In this paper some theorems which characterize the global optimal solutions of nonlinear programming problems are proved. Two algorithms are derived using these results. The first is a path following algorithm to approximate the Karush Kuhn Tucker points of linearly constrained optimization problems and the second is an algorithm to solve linearly constrained global optimization problems. The convergence of these algorithms is proved under suitable assumptions. Numerical results obtained on several test problems are shown.
TL;DR: This work proposes and analyzes heuristics which dynamically choose the parameters in charge of the decision of leaving or not the current face of the feasible set in the solution of large-scale bound-constrained quadratic problems.
Abstract: In this work we focus our attention on the quadratic subproblem of trust-region algorithms for large-scale bound-constrained minimization. An approach that combines a mild active set strategy with gradient projection techniques is employed in the solution of large-scale bound-constrained quadratic problems. To fill in some gaps that have appeared in previous work, we propose and analyze heuristics which dynamically choose the parameters in charge of the decision of leaving or not the current face of the feasible set. The numerical analysis is based on problem from CUTE collection and randomly generated convex problems with controlled conditioning and degeneracy. The practical consequences of an appropriate decision of such parameters have shown to be crucial, particularly when dual degenerate problems are solved.
TL;DR: It is shown that indefinitely preconditioned symmetric Krylov-subspace methods are very efficient for solving linearized KKT systems arising in equality constrained optimization.
Abstract: We show that indefinitely preconditioned symmetric Krylov-subspace methods are very efficient for solving linearized KKT systems arising in equality constrained optimization. We give a numerical comparison of various Krylov subspace methods in three different forms (original system, null-space transformation, range-space transformation). Furthermore, we give a survey of our previous results concerning indefinite preconditioners and merit functions and prove new propositions.
TL;DR: A survey of auxiliary functions in constrained global optimization can be found in this article, where auxiliary functions can be constructed by means of two-step convolution of constraints and the objective function and some conditions providing the validity of the zero duality gap property.
Abstract: The paper contains some new results and a survey of some known results related to auxiliary (Lagrange-type) functions in constrained optimization. We show that auxiliary functions can be constructed by means of two-step convolution of constraints and the objective function and present some conditions providing the validity of the zero duality gap property. We show that auxiliary functions are closely related to the so-called separation functions in the image space of the constrained problem under consideration. The second part of the paper (see Evtushenko et al., General Lagrange-type functions in constrained global optimization. Part 11: Exact Auxillary functions. Optimization Methods and Software) contains results related to exact auxiliary functions.
TL;DR: Analysis of the affine scaling method reveals several interesting features that characterize its asymptotic behaviour, including the deteriorating effects of small residuals and the usefulness of the proposed safeguards.
Abstract: This paper considers affine scaling methods for solving linear l1 problems of the form The paper explains the motivation behind the method and describes its basic steps. Analysis of the method reveals several interesting features that characterize its asymptotic behaviour. Special attention is given to propagation of rounding errors. It is shown that the formal affine scaling algorithm has an inherent drawback: At the kth iterationk= 1,2,..., the algorithm solves a linear least squares problem and uses its residual vector, rk, to compute a search direction uk. However, as kincreases tends to zero, while the rounding errors in ukgrow faster than where e denotes the machine precision in our computations. That is, the smaller the larger the error. Thus, although in theory in practice ukcontains an error vector that belongs to Range(A). This error component is the major reason for accumulation of rounding errors. Cancelling this component avoids most of the damage. The paper explains how to overcome this diff...
TL;DR: In this article, the ABS algorithm for solving linear Diophantine equations was used to solve linear integer programming problems with m≤n inequalities, where n is the number of variables in the program.
Abstract: Using the recently developed ABS algorithm for solving linear Diophantine equations we give a representation of the solutions of a system of m linear integer inequalities in n variables, m≤n, with full rank coefficient matrix. We apply this result to solve linear integer programming problems with m≤n inequalities
TL;DR: In this paper, the authors proposed an approach for computing a suitable curvilinear trajectory, based on the knowledge of the third order derivatives of the objective function, which was made possible by Automatic Differentiation techniques.
Abstract: Solving in an efficient and robust way an unconstrained optimization problem may prove quite hard in certain difficult situations. Typical examples are highly nonlinear problems, ill-conditioned and badly scaled problems. Particularly in these situations, it may be useful to compute a curvilinear trajectory and follow it by curvilinear searches with the aim to reach the solution in few long steps. In this paper, we proposed an approach for computing a suitable curvilinear trajectory, based on the knowledge of the third order derivatives of the objective function. The numerical implementation of this approach was made possible by Automatic Differentiation techniques. Some preliminary numerical results are very encouraging, especially in the case of very ill-conditioned and badly scaled problems.
TL;DR: Results of testing codes for KKT linear systems based upon the ABS algorithms versus classical codes (from packages ABSPACK and LAPACK) are presented, where codes from ABSPack perform significantly better in terms of speed and accuracy.
Abstract: In this paper we present results of testing codes for KKT linear systems based upon the ABS algorithms versus classical codes (from packages ABSPACK and LAPACK). Classes of problems are found where codes from ABSPACK perform significantly better in terms of speed and accuracy.
TL;DR: This paper confirms Dixon's observation showing how much further improvement can be obtained if symbolic differentiation is replaced by automatic differentiation in a new Newton-Preconditioned Conjugate Gradient like algorithm.
Abstract: A new Newton-Preconditioned Conjugate Gradient (PCG) like algorithm is derived in [1,2], where the usual symbolic differentiation is used. Dixon in [3] pointed out that a further improvement would be achieved if symbolic differentiation is replaced by automatic differentiation. This paper confirms his observation showing how much further improvement can be obtained
TL;DR: The indirect reformulation of Pantoja's algorithm set out in this paper is a suitable basis for verified optimization using interval methods.
Abstract: In 1983 Pantoja described a computationally efficient stagewise construction of the Newton direction for the discrete time optimal control problem. Automatic Differentiation can be used to implement Pantoja's algorithm and calculate the Newton direction, without truncation error, and without extensive manual re-writing of targetfunction code to form derivatives. Pantoja's algorithm is direct, in that the independent variables are the control vectors at each timestep. In this paper we formulate an indirect analogue of Pantoja's algorithm, in which the only independent variables are the components of a costate vector corresponding to the initial timestep. This reformulated algorithm gives exactly the Newton step for the initial costate with respect to a terminal transversality condition: at each timestep we solve implicit equations for the current controlsand successor costates. A remarkable feature of the indirect algorithm is that it is straiehtforward to comensate for the effect of non-zero residuals in ...
TL;DR: An inexact Newton method is proposed as an extension of the algorithm given by Goldfarb and Wang, which is proved to be more efficient from theoretical point of view than Newton's method with Cholesky factorization.
Abstract: In this paper, we consider the unary optimization problem: min , where U(.) is a function of a single argument and . As an extension of the algorithm given by Goldfarb and Wang, an inexact Newton method is proposed. The new algorithm is as good as Newton's method in the sense that it enjoys local quadratic convergence, but in contrast to Goldfarb and Wang's algorithm, it has the advantage that a theoretical estimation of its computational cost per iteration is lower under resonable assumptions. This algorithm is the first algorithm for unary optimization which is proved to be more efficient from theoretical point of view than Newton's method with Cholesky factorization.
TL;DR: Numerical results show that the proposed nonmonotone Broyden method for unconstrained optimization is competitive with its monotone counterpart and numerical experiments show that nonMonotone strategies can sometimes improve the performance of the Broyden Method.
Abstract: In this paper, a nonmonotone Broyden method for unconstrained optimization is proposed and its global convergence is analyzed, Numerical results show that the proposed nonmonotone method is competitive with its monotone counterpart. In particular, our numerical experiments show that nonmonotone strategies can sometimes improve the performance of the Broyden method.
TL;DR: In this paper, a new method of calculating search directions in an interior point method for general nonlinear programming is proposed, based on a comparison of the working of a typical interior point approach with that of a sequential quadratic programming algorithm based upon exterior-point penalty functions.
Abstract: We suggest a new method of calculating search directions in an interior point method for general nonlinear programming. The idea behind this proposal arises from a comparison of the working of a typical interior point approach with that of a sequential quadratic programming algorithm based upon exterior-point penalty functions.
TL;DR: In this article, the image reconstruction in optical tomography is treated as a non-linear optimisation problem and uses the Truncated Newton Conjugate Gradient method with Trast region.
Abstract: In Optical Tomography, an efficient image reconstruction algorithm is needed in order to deduce the tissue optical properties from information obtained from measurements on the boundary. The image reconstruction in this work is treated as a non-linear optimisation problem and uses the Truncated Newton Conjugate Gradient method with Trast region. Reverse differentiation is used in order to speed up the reconstruction. The algorithm is attractive because of the speed increase gained from the use of reverse differentiation, and the storage benefit from the use of the Truncated Newton method. Reverse differentiation is applied to the inverse problem in Optical Tomography in the frequency domain. The problem of using complex variables in reverse differentiation is discussed, and the solution for complex linear systems of equations obtained. Error functions are outlined for phase shift, modulation and amplitude. The speed of the reconstruction using reverse differentiation compared to forward differentiation is...
TL;DR: In this article, the reduction in residual norm at each iteration of CG and GMRES is related to the first column of the inverse of an upper Hessenberg matrix that is obtained from the original coefficient matrix by way of an orthogonal transformation.
Abstract: In this paper we show that the reduction in residual norm at each iteration of CG and GMRES is related to the first column of the inverse of an upper Hessenberg matrix that is obtained from the original coefficient matrix by way of an orthogonal transformation. The orthogonal transformation itself is uniquely defined by the coefficient matrix of the equations and the initial vector of residuals. We then apply this analysis to MINRES and show that, under certain circumstances, this algorithm can exhibit an unusual (and very slow) type of convergence that we refer to as oscillatory convergence.
IBM1
TL;DR: This paper reports the computational experiences with a new O(nlogn) time algorithm for fast evaluation of potential and force field in large Lennard – Jones clusters based on a fair-split tree which guarantees O(NLogn)worst-case time complexity without any restriction on particle distributions.
Abstract: We consider the following force field computation problem: given a duster of n particles in 3-dimensional space, compute the force exerted on each particle by the other particles. Depending on different applications, the pairwise interaction could be either gravitational or Lennard Jones. Since there are n(n−1)/2pairs, direct method requires θ(n 2) time. In this paper, we report our computational experiences with a new O(nlogn) time algorithm for fast evaluation of potential and force field in large Lennard – Jones clusters. This algorithm is based on a fair-split tree which guarantees O(nlogn)worst-case time complexity without any restriction on particle distributions. For randomly generated particle systems, our implementation outperforms the direct method when the number of panicles is 500 or larger This is the lowest cross over point ever reported in the literature.