Showing papers in "Computational Optimization and Applications in 2013"

Variable projection for nonlinear least squares problems

Abstract: The variable projection algorithm of Golub and Pereyra (SIAM J. Numer. Anal. 10:413---432, 1973) has proven to be quite valuable in the solution of nonlinear least squares problems in which a substantial number of the parameters are linear. Its advantages are efficiency and, more importantly, a better likelihood of finding a global minimizer rather than a local one. The purpose of our work is to provide a more robust implementation of this algorithm, include constraints on the parameters, more clearly identify key ingredients so that improvements can be made, compute the Jacobian matrix more accurately, and make future implementations in other languages easy.

A computational study and survey of methods for the single-row facility layout problem

Abstract: The single-row facility layout problem (SRFLP) is an NP-hard combinatorial optimization problem that is concerned with the arrangement of n departments of given lengths on a line so as to minimize the weighted sum of the distances between department pairs. (SRFLP) is the one-dimensional version of the facility layout problem that seeks to arrange rectangular departments so as to minimize the overall interaction cost. This paper compares the different modelling approaches for (SRFLP) and applies a recent SDP approach for general quadratic ordering problems from Hungerlander and Rendl to (SRFLP). In particular, we report optimal solutions for several (SRFLP) instances from the literature with up to 42 departments that remained unsolved so far. Secondly we significantly reduce the best known gaps and running times for large instances with up to 110 departments.

Relaxation methods for mixed-integer optimal control of partial differential equations

Abstract: We consider integer-restricted optimal control of systems governed by abstract semilinear evolution equations. This includes the problem of optimal control design for certain distributed parameter systems endowed with multiple actuators, where the task is to minimize costs associated with the dynamics of the system by choosing, for each instant in time, one of the actuators together with ordinary controls. We consider relaxation techniques that are already used successfully for mixed-integer optimal control of ordinary differential equations. Our analysis yields sufficient conditions such that the optimal value and the optimal state of the relaxed problem can be approximated with arbitrary precision by a control satisfying the integer restrictions. The results are obtained by semigroup theory methods. The approach is constructive and gives rise to a numerical method. We supplement the analysis with numerical experiments.

Bregman operator splitting with variable stepsize for total variation image reconstruction

Abstract: This paper develops a Bregman operator splitting algorithm with variable stepsize (BOSVS) for solving problems of the form $\min\{\phi(Bu) +1/2\|Au-f\|_{2}^{2}\}$ , where ? may be nonsmooth. The original Bregman Operator Splitting (BOS) algorithm employed a fixed stepsize, while BOSVS uses a line search to achieve better efficiency. These schemes are applicable to total variation (TV)-based image reconstruction. The stepsize rule starts with a Barzilai-Borwein (BB) step, and increases the nominal step until a termination condition is satisfied. The stepsize rule is related to the scheme used in SpaRSA (Sparse Reconstruction by Separable Approximation). Global convergence of the proposed BOSVS algorithm to a solution of the optimization problem is established. BOSVS is compared with other operator splitting schemes using partially parallel magnetic resonance image reconstruction problems. The experimental results indicate that the proposed BOSVS algorithm is more efficient than the BOS algorithm and another split Bregman Barzilai-Borwein algorithm known as SBB.

Stochastic Nash equilibrium problems: sample average approximation and applications

Abstract: This paper presents a Nash equilibrium model where the underlying objective functions involve uncertainty and nonsmoothness. The well-known sample average approximation method is applied to solve the problem and the first order equilibrium conditions are characterized in terms of Clarke generalized gradients. Under some moderate conditions, it is shown that with probability one, a statistical estimator (a Nash equilibrium or a Nash-C-stationary point) obtained from sample average approximate equilibrium problem converges to its true counterpart. Moreover, under some calmness conditions of the Clarke generalized derivatives, it is shown that with probability approaching one exponentially fast by increasing sample size, the Nash-C-stationary point converges to a weak Nash-C-stationary point of the true problem. Finally, the model is applied to stochastic Nash equilibrium problem in the wholesale electricity market.

The robust network loading problem with dynamic routing

Abstract: In this paper the Robust Network Loading problem with splittable flows and dynamic routing under polyhedral uncertainty for the demands is considered. Polyhedral results for the capacity formulation of the problem are given. The first exact approach for solving the problem is presented. A branch-and-cut algorithm based on the proposed capacity formulation is developed. Computational results using the hose polyhedron to model the demand uncertainty are discussed.

Inexact projected gradient method for vector optimization

Abstract: In this work, we propose an inexact projected gradient-like method for solving smooth constrained vector optimization problems. In the unconstrained case, we retrieve the steepest descent method introduced by Grana Drummond and Svaiter. In the constrained setting, the method we present extends the exact one proposed by Grana Drummond and Iusem, since it admits relative errors on the search directions. At each iteration, a decrease of the objective value is obtained by means of an Armijo-like rule. The convergence results of this new method extend those obtained by Fukuda and Grana Drummond for the exact version. For partial orders induced by both pointed and nonpointed cones, under some reasonable hypotheses, global convergence to weakly efficient points of all sequences generated by the inexact projected gradient method is established for convex (respect to the ordering cone) objective functions. In the convergence analysis we also establish a connection between the so-called weighting method and the one we propose.

A penalty function-based differential evolution algorithm for constrained global optimization

Abstract: We propose a differential evolution-based algorithm for constrained global optimization. Although differential evolution has been used as the underlying global solver, central to our approach is the penalty function that we introduce. The adaptive nature of the penalty function makes the results of the algorithm mostly insensitive to low values of the penalty parameter. We have also demonstrated both empirically and theoretically that the high value of the penalty parameter is detrimental to convergence, specially for functions with multiple local minimizers. Hence, the penalty function can dispense with the penalty parameter. We have extensively tested our penalty function-based DE algorithm on a set of 24 benchmark test problems. Results obtained are compared with those of some recent algorithms.

On the optimal control of the Schlögl-model

Abstract: Optimal control problems for a class of 1D semilinear parabolic equations with cubic nonlinearity are considered. This class is also known as the Schlogl model. Main emphasis is laid on the control of traveling wave fronts that appear as typical solutions to the state equation. The well-posedness of the optimal control problem and the regularity of its solution are proved. First-order necessary optimality conditions are established by standard adjoint calculus. The state equation is solved by the implicit Euler method in time and a finite element technique with respect to the spatial variable. Moreover, model reduction by Proper Orthogonal Decomposition is applied and compared with the numerical solution of the full problem. To solve the optimal control problems numerically, the performance of different versions of the nonlinear conjugate gradient method is studied. Various numerical examples demonstrate the capacities and limits of optimal control methods.

A derivative-free approximate gradient sampling algorithm for finite minimax problems

Abstract: In this paper we present a derivative-free optimization algorithm for finite minimax problems. The algorithm calculates an approximate gradient for each of the active functions of the finite max function and uses these to generate an approximate subdifferential. The negative projection of 0 onto this set is used as a descent direction in an Armijo-like line search. We also present a robust version of the algorithm, which uses the `almost active' functions of the finite max function in the calculation of the approximate subdifferential. Convergence results are presented for both algorithms, showing that either f(x k )??? or every cluster point is a Clarke stationary point. Theoretical and numerical results are presented for three specific approximate gradients: the simplex gradient, the centered simplex gradient and the Gupal estimate of the gradient of the Steklov averaged function. A performance comparison is made between the regular and robust algorithms, the three approximate gradients, and a regular and robust stopping condition.

Convergence and perturbation resilience of dynamic string-averaging projection methods

Abstract: We consider the convex feasibility problem (CFP) in Hilbert space and concentrate on the study of string-averaging projection (SAP) methods for the CFP, analyzing their convergence and their perturbation resilience. In the past, SAP methods were formulated with a single predetermined set of strings and a single predetermined set of weights. Here we extend the scope of the family of SAP methods to allow iteration-index-dependent variable strings and weights and term such methods dynamic string-averaging projection (DSAP) methods. The bounded perturbation resilience of DSAP methods is relevant and important for their possible use in the framework of the recently developed superiorization heuristic methodology for constrained minimization problems.

On convex quadratic programs with linear complementarity constraints

Abstract: The paper shows that the global resolution of a general convex quadratic program with complementarity constraints (QPCC), possibly infeasible or unbounded, can be accomplished in finite time. The method constructs a minmax mixed integer formulation by introducing finitely many binary variables, one for each complementarity constraint. Based on the primal-dual relationship of a pair of convex quadratic programs and on a logical Benders scheme, an extreme ray/point generation procedure is developed, which relies on valid satisfiability constraints for the integer program. To improve this scheme, we propose a two-stage approach wherein the first stage solves the mixed integer quadratic program with pre-set upper bounds on the complementarity variables, and the second stage solves the program outside this bounded region by the Benders scheme. We report computational results with our method. We also investigate the addition of a penalty term y T Dw to the objective function, where y and w are the complementary variables and D is a nonnegative diagonal matrix. The matrix D can be chosen effectively by solving a semidefinite program, ensuring that the objective function remains convex. The addition of the penalty term can often reduce the overall runtime by at least 50 %. We report preliminary computational testing on a QP relaxation method which can be used to obtain better lower bounds from infeasible points; this method could be incorporated into a branching scheme. By combining the penalty method and the QP relaxation method, more than 90 % of the gap can be closed for some QPCC problems.

Algorithms for detecting optimal hereditary structures in graphs, with application to clique relaxations

Abstract: Given a simple undirected graph, the problem of finding a maximum subset of vertices satisfying a nontrivial, interesting property Â? that is hereditary on induced subgraphs, is known to be NP-hard. Many well-known graph properties meet the above conditions, making the problem widely applicable. This paper proposes a general purpose exact algorithmic framework to solve this problem and investigates key algorithm design and implementation issues that are helpful in tailoring the general framework for specific graph properties. The performance of the algorithms so derived for the maximum s-plex and the maximum s-defective clique problems, which arise in network-based data mining applications, is assessed through a computational study.

A customized proximal point algorithm for convex minimization with linear constraints

Abstract: This paper demonstrates a customized application of the classical proximal point algorithm (PPA) to the convex minimization problem with linear constraints. We show that if the proximal parameter in metric form is chosen appropriately, the application of PPA could be effective to exploit the simplicity of the objective function. The resulting subproblems could be easier than those of the augmented Lagrangian method (ALM), a benchmark method for the model under our consideration. The efficiency of the customized application of PPA is demonstrated by some image processing problems.

Primal and dual alternating direction algorithms for l 1 - l 1 -norm minimization problems in compressive sensing

Abstract: In this paper, we propose, analyze and test primal and dual versions of the alternating direction algorithm for the sparse signal reconstruction from its major noise contained observation data. The algorithm minimizes a convex non-smooth function consisting of the sum of l 1-norm regularization term and l 1-norm data fidelity term. We minimize the corresponding augmented Lagrangian function alternatively from either primal or dual forms. Both of the resulting subproblems admit explicit solutions either by using a one-dimensional shrinkage or by an efficient Euclidean projection. The algorithm is easily implementable and it requires only two matrix-vector multiplications per-iteration. The global convergence of the proposed algorithm is established under some technical conditions. The extensions to the non-negative signal recovery problem and the weighted regularization minimization problem are also discussed and tested. Numerical results illustrate that the proposed algorithm performs better than the state-of-the-art algorithm YALL1.

Self-adaptive differential evolution incorporating a heuristic mixing of operators

Abstract: A considerable number of differential evolution variants have been proposed in the last few decades. However, no variant was able to consistently perform over a wide range of test problems. In this paper, propose two novel differential evolution based algorithms are proposed for solving constrained optimization problems. Both algorithms utilize the strengths of multiple mutation and crossover operators. The appropriate mix of the mutation and crossover operators, for any given problem, is determined through an adaptive learning process. In addition, to further accelerate the convergence of the algorithm, a local search technique is applied to a few selected individuals in each generation. The resulting algorithms are named as Self-Adaptive Differential Evolution Incorporating a Heuristic Mixing of Operators. The algorithms have been tested by solving 60 constrained optimization test instances. The results showed that the proposed algorithms have a competitive, if not better, performance in comparison to the-state-of-the-art algorithms.

Combining Lagrangian decomposition and excessive gap smoothing technique for solving large-scale separable convex optimization problems

Abstract: A new algorithm for solving large-scale convex optimization problems with a separable objective function is proposed. The basic idea is to combine three techniques: Lagrangian dual decomposition, excessive gap and smoothing. The main advantage of this algorithm is that it automatically and simultaneously updates the smoothness parameters which significantly improves its performance. The convergence of the algorithm is proved under weak conditions imposed on the original problem. The rate of convergence is $O(\frac {1}{k})$ , where k is the iteration counter. In the second part of the paper, the proposed algorithm is coupled with a dual scheme to construct a switching variant in a dual decomposition framework. We discuss implementation issues and make a theoretical comparison. Numerical examples confirm the theoretical results.

A subgradient method for multiobjective optimization

Abstract: A method for solving quasiconvex nondifferentiable unconstrained multiobjective optimization problems is proposed in this paper. This method extends to the multiobjective case of the classical subgradient method for real-valued minimization. Assuming the basically componentwise quasiconvexity of the objective components, full convergence (to Pareto optimal points) of all the sequences produced by the method is established.

On optimizing the sum of the Rayleigh quotient and the generalized Rayleigh quotient on the unit sphere

Abstract: Given symmetric matrices B,D?? n×n and a symmetric positive definite matrix W?? n×n , maximizing the sum of the Rayleigh quotient x ? D x and the generalized Rayleigh quotient $\frac{\mathbf{x}^{\top}B \mathbf{x}}{\vphantom{\mathrm{I}^{\mathrm{I}}}\mathbf{x}^{\top}W\mathbf{x}}$ on the unit sphere not only is of mathematical interest in its own right, but also finds applications in practice. In this paper, we first present a real world application arising from the sparse Fisher discriminant analysis. To tackle this problem, our first effort is to characterize the local and global maxima by investigating the optimality conditions. Our results reveal that finding the global solution is closely related with a special extreme nonlinear eigenvalue problem, and in the special case D=μW (μ>0), the set of the global solutions is essentially an eigenspace corresponding to the largest eigenvalue of a specially-defined matrix. The characterization of the global solution not only sheds some lights on the maximization problem, but motives a starting point strategy to obtain the global maximizer for any monotonically convergent iteration. Our second part then realizes the Riemannian trust-region method of Absil, Baker and Gallivan (Found. Comput. Math. 7:303---330, 2007) into a practical algorithm to solve this problem, which enjoys the nice convergence properties: global convergence and local superlinear convergence. Preliminary numerical tests are carried out and empirical evaluation of its performance is reported.

Parallel distributed-memory simplex for large-scale stochastic LP problems

Abstract: We present a parallelization of the revised simplex method for large extensive forms of two-stage stochastic linear programming (LP) problems. These problems have been considered too large to solve with the simplex method; instead, decomposition approaches based on Benders decomposition or, more recently, interior-point methods are generally used. However, these approaches do not provide optimal basic solutions, which allow for efficient hot-starts (e.g., in a branch-and-bound context) and can provide important sensitivity information. Our approach exploits the dual block-angular structure of these problems inside the linear algebra of the revised simplex method in a manner suitable for high-performance distributed-memory clusters or supercomputers. While this paper focuses on stochastic LPs, the work is applicable to all problems with a dual block-angular structure. Our implementation is competitive in serial with highly efficient sparsity-exploiting simplex codes and achieves significant relative speed-ups when run in parallel. Additionally, very large problems with hundreds of millions of variables have been successfully solved to optimality. This is the largest-scale parallel sparsity-exploiting revised simplex implementation that has been developed to date and the first truly distributed solver. It is built on novel analysis of the linear algebra for dual block-angular LP problems when solved by using the revised simplex method and a novel parallel scheme for applying product-form updates.

A class of quasi-variational inequalities for adaptive image denoising and decomposition

Abstract: We introduce a class of adaptive non-smooth convex variational problems for image denoising in terms of a common data fitting term and a support functional as regularizer. Adaptivity is modeled by a set-valued mapping with closed, compact and convex values, that defines and steers the regularizer depending on the variational solution. This extension gives rise to a class of quasi-variational inequalities. We provide sufficient conditions for the existence of fixed points as solutions, and an algorithm based on solving a sequence of variational problems. Denoising experiments with spatial and spatio-temporal image data and an adaptive total variation regularizer illustrate our approach.

A smoothing-regularization approach to mathematical programs with vanishing constraints

Abstract: We consider a numerical approach for the solution of a difficult class of optimization problems called mathematical programs with vanishing constraints. The basic idea is to reformulate the characteristic constraints of the program via a nonsmooth function and to eventually smooth it and regularize the feasible set with the aid of a certain smoothing- and regularization parameter t>0 such that the resulting problem is more tractable and coincides with the original program for t=0. We investigate the convergence behavior of a sequence of stationary points of the smooth and regularized problems by letting t tend to zero. Numerical results illustrating the performance of the approach are given. In particular, a large-scale example from topology optimization of mechanical structures with local stress constraints is investigated.

A novel differential evolution algorithm for binary optimization

Abstract: Differential evolution (DE) is one of the most powerful stochastic search methods which was introduced originally for continuous optimization. In this sense, it is of low efficiency in dealing with discrete problems. In this paper we try to cover this deficiency through introducing a new version of DE algorithm, particularly designed for binary optimization. It is well-known that in its original form, DE maintains a differential mutation, a crossover and a selection operator for optimizing non-linear continuous functions. Therefore, developing the new binary version of DE algorithm, calls for introducing operators having the major characteristics of the original ones and being respondent to the structure of binary optimization problems. Using a measure of dissimilarity between binary vectors, we propose a differential mutation operator that works in continuous space while its consequence is used in the construction of the complete solution in binary space. This approach essentially enables us to utilize the structural knowledge of the problem through heuristic procedures, during the construction of the new solution. To verify effectiveness of our approach, we choose the uncapacitated facility location problem (UFLP)--one of the most frequently encountered binary optimization problems--and solve benchmark suites collected from OR-Library. Extensive computational experiments are carried out to find out the behavior of our algorithm under various setting of the control parameters and also to measure how well it competes with other state of the art binary optimization algorithms. Beside UFLP, we also investigate the suitably of our approach for optimizing numerical functions. We select a number of well-known functions on which we compare the performance of our approach with different binary optimization algorithms. Results testify that our approach is very efficient and can be regarded as a promising method for solving wide class of binary optimization problems.

A new method for solving Pareto eigenvalue complementarity problems

Abstract: In this paper, we introduce a new method, called the Lattice Projection Method (LPM), for solving eigenvalue complementarity problems. The original problem is reformulated to find the roots of a nonsmooth function. A semismooth Newton type method is then applied to approximate the eigenvalues and eigenvectors of the complementarity problems. The LPM is compared to SNMmin and SNMFB, two methods widely discussed in the literature for solving nonlinear complementarity problems, by using the performance profiles as a comparing tool (Dolan, More in Math. Program. 91:201---213, 2002). The performance measures, used to analyze the three solvers on a set of matrices mostly taken from the Matrix Market (Boisvert et al. in The quality of numerical software: assessment and enhancement, pp. 125---137, 1997), are computing time, number of iterations, number of failures and maximum number of solutions found by each solver. The numerical experiments highlight the efficiency of the LPM and show that it is a promising method for solving eigenvalue complementarity problems. Finally, Pareto bi-eigenvalue complementarity problems were solved numerically as an application to confirm the efficiency of our method.

Algorithmic improvements on dynamic programming for the bi-objective {0,1} knapsack problem

Abstract: This paper presents several methodological and algorithmic improvements over a state-of-the-art dynamic programming algorithm for solving the bi-objective {0,1} knapsack problem. The variants proposed make use of new definitions of lower and upper bounds, which allow a large number of states to be discarded. The computation of these bounds are based on the application of dichotomic search, definition of new bound sets, and bi-objective simplex algorithms to solve the relaxed problem. Although these new techniques are not of a common application for dynamic programming, we show that the best variants tested in this work can lead to an average improvement of 10 to 30 % in CPU-time and significant less memory usage than the original approach in a wide benchmark set of instances, even for the most difficult ones in the literature.

Lower and upper bounds for the spanning tree with minimum branch vertices

Abstract: We study a variant of the spanning tree problem where we require that, for a given connected graph, the spanning tree to be found has the minimum number of branch vertices (that is vertices of the tree whose degree is greater than two). We provide four different formulations of the problem and compare different relaxations of them, namely Lagrangian relaxation, continuous relaxation, mixed integer-continuous relaxation. We approach the solution of the Lagrangian dual both by means of a standard subgradient method and an ad-hoc finite ascent algorithm based on updating one multiplier at the time. We provide numerical result comparison of all the considered relaxations on a wide set of benchmark instances. A useful follow-up of tackling the Lagrangian dual is the possibility of getting a feasible solution for the original problem with no extra costs. We evaluate the quality of the resulting upper bound by comparison either with the optimal solution, whenever available, or with the feasible solution provided by some existing heuristic algorithms.

An ADM-based splitting method for separable convex programming

Abstract: We consider the convex minimization problem with linear constraints and a block-separable objective function which is represented as the sum of three functions without coupled variables. To solve this model, it is empirically effective to extend straightforwardly the alternating direction method of multipliers (ADM for short). But, the convergence of this straightforward extension of ADM is still not proved theoretically. Based on ADM's straightforward extension, this paper presents a new splitting method for the model under consideration, which is empirically competitive to the straightforward extension of ADM and meanwhile the global convergence can be proved under standard assumptions. At each iteration, the new method corrects the output of the straightforward extension of ADM by some slight correction computation to generate a new iterate. Thus, the implementation of the new method is almost as easy as that of ADM's straightforward extension. We show the numerical efficiency of the new method by some applications in the areas of image processing and statistics.

On sample size control in sample average approximations for solving smooth stochastic programs

Abstract: We consider smooth stochastic programs and develop a discrete-time optimal-control problem for adaptively selecting sample sizes in a class of algorithms based on variable sample average approximations (VSAA). The control problem aims to minimize the expected computational cost to obtain a near-optimal solution of a stochastic program and is solved approximately using dynamic programming. The optimal-control problem depends on unknown parameters such as rate of convergence, computational cost per iteration, and sampling error. Hence, we implement the approach within a receding-horizon framework where parameters are estimated and the optimal-control problem is solved repeatedly during the calculations of a VSAA algorithm. The resulting sample-size selection policy consistently produces near-optimal solutions in short computing times as compared to other plausible policies in several numerical examples.

Global convergence of modified augmented Lagrangian methods for nonlinear semidefinite programming

Abstract: We investigate in this paper global convergence properties of the augmented Lagrangian method for nonlinear semidefinite programming (NLSDP). Four modified augmented Lagrangian methods for solving NLSDP based on different algorithmic strategies are proposed. Possibly infeasible limit points of the proposed methods are characterized. It is proved that feasible limit points that satisfy the Mangasarian-Fromovitz constraint qualification are KKT points of NLSDP without requiring the boundedness condition of the multipliers. Preliminary numerical results are reported to compare the performance of the modified augmented Lagrangian methods.

Preconditioning Newton---Krylov methods in nonconvex large scale optimization

Abstract: We consider an iterative preconditioning technique for non-convex large scale optimization. First, we refer to the solution of large scale indefinite linear systems by using a Krylov subspace method, and describe the iterative construction of a preconditioner which does not involve matrices products or matrices storage. The set of directions generated by the Krylov subspace method is used, as by product, to provide an approximate inverse preconditioner. Then, we experience our preconditioner within Truncated Newton schemes for large scale unconstrained optimization, where we generalize the truncation rule by Nash---Sofer (Oper. Res. Lett. 9:219---221, 1990) to the indefinite case, too. We use a Krylov subspace method to both approximately solve the Newton equation and to construct the preconditioner to be used at the current outer iteration. An extensive numerical experience shows that the proposed preconditioning strategy, compared with the unpreconditioned strategy and PREQN (Morales and Nocedal in SIAM J. Optim. 10:1079---1096, 2000), may lead to a reduction of the overall inner iterations. Finally, we show that our proposal has some similarities with the Limited Memory Preconditioners (Gratton et al. in SIAM J. Optim. 21:912---935, 2011).