Showing papers in "Mathematical Programming in 2006"

On the implementation of an interior-point filter line-search algorithm for large-scale nonlinear programming

Abstract: We present a primal-dual interior-point algorithm with a filter line-search method for nonlinear programming. Local and global convergence properties of this method were analyzed in previous work. Here we provide a comprehensive description of the algorithm, including the feasibility restoration phase for the filter method, second-order corrections, and inertia correction of the KKT matrix. Heuristics are also considered that allow faster performance. This method has been implemented in the IPOPT code, which we demonstrate in a detailed numerical study based on 954 problems from the CUTEr test set. An evaluation is made of several line-search options, and a comparison is provided with two state-of-the-art interior-point codes for nonlinear programming.

An interior algorithm for nonlinear optimization that combines line search and trust region steps

Abstract: An interior-point method for nonlinear programming is presented It enjoys the flexibility of switching between a line search method that computes steps by factoring the primal-dual equations and a trust region method that uses a conjugate gradient iteration Steps computed by direct factorization are always tried first, but if they are deemed ineffective, a trust region iteration that guarantees progress toward stationarity is invoked To demonstrate its effectiveness, the algorithm is implemented in the Knitro [6,28] software package and is extensively tested on a wide selection of test problems

Cubic regularization of Newton method and its global performance

Abstract: In this paper, we provide theoretical analysis for a cubic regularization of Newton method as applied to unconstrained minimization problem. For this scheme, we prove general local convergence results. However, the main contribution of the paper is related to global worst-case complexity bounds for different problem classes including some nonconvex cases. It is shown that the search direction can be computed by standard linear algebra technique.

Robust Branch-and-Cut-and-Price for the Capacitated Vehicle Routing Problem

Abstract: The best exact algorithms for the Capacitated Vehicle Routing Problem (CVRP) have been based on either branch-and-cut or Lagrangean relaxation/column generation. This paper presents an algorithm that combines both approaches: it works over the intersection of two polytopes, one associated with a traditional Lagrangean relaxation over q-routes, the other defined by bound, degree and capacity constraints. This is equivalent to a linear program with exponentially many variables and constraints that can lead to lower bounds that are superior to those given by previous methods. The resulting branch-and-cut-and-price algorithm can solve to optimality all instances from the literature with up to 135 vertices. This more than doubles the size of the instances that can be consistently solved.

Matrix Sum-of-Squares Relaxations for Robust Semi-Definite Programs

Abstract: We consider robust semi-definite programs which depend polynomially or rationally on some uncertain parameter that is only known to be contained in a set with a polynomial matrix inequality description. On the basis of matrix sum-of-squares decompositions, we suggest a systematic procedure to construct a family of linear matrix inequality relaxations for computing upper bounds on the optimal value of the corresponding robust counterpart. With a novel matrix-version of Putinar's sum-of-squares representation for positive polynomials on compact semi-algebraic sets, we prove asymptotic exactness of the relaxation family under a suitable constraint qualification. If the uncertainty region is a compact polytope, we provide a new duality proof for the validity of Putinar's constraint qualification with an a priori degree bound on the polynomial certificates. Finally, we point out the consequences of our results for constructing relaxations based on the so-called full-block S-procedure, which allows to apply recently developed tests in order to computationally verify the exactness of possibly small-sized relaxations.

Ambiguous chance constrained problems and robust optimization

Abstract: In this paper we study ambiguous chance constrained problems where the distributions of the random parameters in the problem are themselves uncertain. We focus primarily on the special case where the uncertainty set ** of the distributions is of the form ** where ρp denotes the Prohorov metric. The ambiguous chance constrained problem is approximated by a robust sampled problem where each constraint is a robust constraint centered at a sample drawn according to the central measure **. The main contribution of this paper is to show that the robust sampled problem is a good approximation for the ambiguous chance constrained problem with a high probability. This result is established using the Strassen-Dudley Representation Theorem that states that when the distributions of two random variables are close in the Prohorov metric one can construct a coupling of the random variables such that the samples are close with a high probability. We also show that the robust sampled problem can be solved efficiently both in theory and in practice.

Robust game theory

Abstract: We present a distribution-free model of incomplete-information games, both with and without private information, in which the players use a robust optimization approach to contend with payoff uncertainty. Our ``robust game'' model relaxes the assumptions of Harsanyi's Bayesian game model, and provides an alternative distribution-free equilibrium concept, which we call ``robust-optimization equilibrium,'' to that of the ex post equilibrium. We prove that the robust-optimization equilibria of an incomplete-information game subsume the ex post equilibria of the game and are, unlike the latter, guaranteed to exist when the game is finite and has bounded payoff uncertainty set. For arbitrary robust finite games with bounded polyhedral payoff uncertainty sets, we show that we can compute a robust-optimization equilibrium by methods analogous to those for identifying a Nash equilibrium of a finite game with complete information. In addition, we present computational results.

Tractable Approximations to Robust Conic Optimization Problems

Abstract: In earlier proposals, the robust counterpart of conic optimization problems exhibits a lateral increase in complexity, i.e., robust linear programming problems (LPs) become second order cone problems (SOCPs), robust SOCPs become semidefinite programming problems (SDPs), and robust SDPs become NP-hard. We propose a relaxed robust counterpart for general conic optimization problems that (a) preserves the computational tractability of the nominal problem; specifically the robust conic optimization problem retains its original structure, i.e., robust LPs remain LPs, robust SOCPs remain SOCPs and robust SDPs remain SDPs, and (b) allows us to provide a guarantee on the probability that the robust solution is feasible when the uncertain coefficients obey independent and identically distributed normal distributions.

Perspective cuts for a class of convex 0–1 mixed integer programs

Abstract: We show that the convex envelope of the objective function of Mixed-Integer Programming problems with a specific structure is the perspective function of the continuous part of the objective function. Using a characterization of the subdifferential of the perspective function, we derive “perspective cuts”, a family of valid inequalities for the problem. Perspective cuts can be shown to belong to the general family of disjunctive cuts, but they do not require the solution of a potentially costly nonlinear programming problem to be separated. Using perspective cuts substantially improves the performance of Branch & Cut approaches for at least two models that, either “naturally” or after a proper reformulation, have the required structure: the Unit Commitment problem in electrical power production and the Mean-Variance problem in portfolio optimization.

Computational complexity of stochastic programming problems

Abstract: Stochastic programming is the subfield of mathematical programming that considers optimization in the presence of uncertainty. During the last four decades a vast quantity of literature on the subject has appeared. Developments in the theory of computational complexity allow us to establish the theoretical complexity of a variety of stochastic programming problems studied in this literature. Under the assumption that the stochastic parameters are independently distributed, we show that two-stage stochastic programming problems are ?P-hard. Under the same assumption we show that certain multi-stage stochastic programming problems are PSPACE-hard. The problems we consider are non-standard in that distributions of stochastic parameters in later stages depend on decisions made in earlier stages.

Mixed integer models for the stationary case of gas network optimization

Abstract: A gas network basically consists of a set of compressors and valves that are connected by pipes. The problem of gas network optimization deals with the question of how to optimize the flow of the gas and to use the compressors cost-efficiently such that all demands of the gas network are satisfied. This problem leads to a complex mixed integer nonlinear optimization problem. We describe techniques for a piece-wise linear approximation of the nonlinearities in this model resulting in a large mixed integer linear program. We study sub-polyhedra linking these piece-wise linear approximations and show that the number of vertices is computationally tractable yielding exact separation algorithms. Suitable branching strategies complementing the separation algorithms are also presented. Our computational results demonstrate the success of this approach.

A Class of stochastic programs with decision dependent uncertainty

Abstract: We address a class of problems where decisions have to be optimized over a time horizon given that the future is uncertain and that the optimization decisions influence the time of information discovery for a subset of the uncertain parameters. The standard approach to formulate stochastic programs is based on the assumption that the stochastic process is independent of the optimization decisions, which is not true for the class of problems under consideration. We present a hybrid mixed-integer disjunctive programming formulation for the stochastic program corresponding to this class of problems and hence extend the stochastic programming framework. A set of theoretical properties that lead to reduction in the size of the model is identified. A Lagrangean duality based branch and bound algorithm is also presented.

Extending Scope of Robust Optimization: Comprehensive Robust Counterparts of Uncertain Problems

Abstract: In this paper, we propose a new methodology for handling optimization problems with uncertain data. With the usual Robust Optimization paradigm, one looks for the decisions ensuring a required performance for all realizations of the data from a given bounded uncertainty set, whereas with the proposed approach, we require also a controlled deterioration in performance when the data is outside the uncertainty set.The extension of Robust Optimization methodology developed in this paper opens up new possibilities to solve efficiently multi-stage finite-horizon uncertain optimization problems, in particular, to analyze and to synthesize linear controllers for discrete time dynamical systems.

An Algorithmic Framework for the Exact Solution of the Prize-Collecting Steiner Tree Problem

Abstract: The Prize-Collecting Steiner Tree Problem (PCST) on a graph with edge costs and vertex profits asks for a subtree minimizing the sum of the total cost of all edges in the subtree plus the total profit of all vertices not contained in the subtree. PCST appears frequently in the design of utility networks where profit generating customers and the network connecting them have to be chosen in the most profitable way. Our main contribution is the formulation and implementation of a branch-and-cut algorithm based on a directed graph model where we combine several state-of-the-art methods previously used for the Steiner tree problem. Our method outperforms the previously published results on the standard benchmark set of problems. We can solve all benchmark instances from the literature to optimality, including some of them for which the optimum was not known. Compared to a recent algorithm by Lucena and Resende, our new method is faster by more than two orders of magnitude. We also introduce a new class of more challenging instances and present computational results for them. Finally, for a set of large-scale real-world instances arising in the design of fiber optic networks, we also obtain optimal solution values.

Convexity and decomposition of mean-risk stochastic programs

Abstract: Traditional stochastic programming is risk neutral in the sense that it is concerned with the optimization of an expectation criterion. A common approach to addressing risk in decision making problems is to consider a weighted mean-risk objective, where some dispersion statistic is used as a measure of risk. We investigate the computational suitability of various mean-risk objective functions in addressing risk in stochastic programming models. We prove that the classical mean-variance criterion leads to computational intractability even in the simplest stochastic programs. On the other hand, a number of alternative mean-risk functions are shown to be computationally tractable using slight variants of existing stochastic programming decomposition algorithms. We propose decomposition-based parametric cutting plane algorithms to generate mean-risk efficient frontiers for two particular classes of mean-risk objectives.

Conditional Value-at-Risk in Stochastic Programs with Mixed-Integer Recourse

Abstract: In classical two-stage stochastic programming the expected value of the total costs is minimized. Recently, mean-risk models - studied in mathematical finance for several decades - have attracted attention in stochastic programming. We consider Conditional Value-at-Risk as risk measure in the framework of two-stage stochastic integer programming. The paper addresses structure, stability, and algorithms for this class of models. In particular, we study continuity properties of the objective function, both with respect to the first-stage decisions and the integrating probability measure. Further, we present an explicit mixed-integer linear programming formulation of the problem when the probability distribution is discrete and finite. Finally, a solution algorithm based on Lagrangean relaxation of nonanticipativity is proposed.

Decomposition with branch-and-cut approaches for two-stage stochastic mixed-integer programming

Abstract: Decomposition has proved to be one of the more effective tools for the solution of large-scale problems, especially those arising in stochastic programming. A decomposition method with wide applicability is Benders' decomposition, which has been applied to both stochastic programming as well as integer programming problems. However, this method of decomposition relies on convexity of the value function of linear programming subproblems. This paper is devoted to a class of problems in which the second-stage subproblem(s) may impose integer restrictions on some variables. The value function of such integer subproblem(s) is not convex, and new approaches must be designed. In this paper, we discuss alternative decomposition methods in which the second-stage integer subproblems are solved using branch-and-cut methods. One of the main advantages of our decomposition scheme is that Stochastic Mixed-Integer Programming (SMIP) problems can be solved by dividing a large problem into smaller MIP subproblems that can be solved in parallel. This paper lays the foundation for such decomposition methods for two-stage stochastic mixed-integer programs.

New algorithms for singly linearly constrained quadratic programs subject to lower and upper bounds

Abstract: There are many applications related to singly linearly constrained quadratic programs subjected to upper and lower bounds. In this paper, a new algorithm based on secant approximation is provided for the case in which the Hessian matrix is diagonal and positive definite. To deal with the general case where the Hessian is not diagonal, a new efficient projected gradient algorithm is proposed. The basic features of the projected gradient algorithm are: 1) a new formula is used for the stepsize; 2) a recently-established adaptive non-monotone line search is incorporated; and 3) the optimal stepsize is determined by quadratic interpolation if the non-monotone line search criterion fails to be satisfied. Numerical experiments on large-scale random test problems and some medium-scale quadratic programs arising in the training of Support Vector Machines demonstrate the usefulness of these algorithms.

Minimizing Polynomials via Sum of Squares over the Gradient Ideal

Abstract: A method is proposed for finding the global minimum of a multivariate polynomial via sum of squares (SOS) relaxation over its gradient variety. That variety consists of all points where the gradient is zero and it need not be finite. A polynomial which is nonnegative on its gradient variety is shown to be SOS modulo its gradient ideal, provided the gradient ideal is radical or the polynomial is strictly positive on the real gradient variety. This opens up the possibility of solving previously intractable polynomial optimization problems. The related problem of constrained minimization is also considered, and numerical examples are discussed. Experiments show that our method using the gradient variety outperforms prior SOS methods.

A Robust Optimization Approach to Dynamic Pricing and Inventory Control with no Backorders

Abstract: In this paper, we present a robust optimization formulation for dealing with demand uncertainty in a dynamic pricing and inventory control problem for a make-to-stock manufacturing system. We consider a multi-product capacitated, dynamic setting. We introduce a demand-based fluid model where the demand is a linear function of the price, the inventory cost is linear, the production cost is an increasing strictly convex function of the production rate and all coefficients are time-dependent. A key part of the model is that no backorders are allowed. We show that the robust formulation is of the same order of complexity as the nominal problem and demonstrate how to adapt the nominal (deterministic) solution algorithm to the robust problem.

Assessing solution quality in stochastic programs

Abstract: Determining whether a solution is of high quality (optimal or near optimal) is fundamental in optimization theory and algorithms. In this paper, we develop Monte Carlo sampling-based procedures for assessing solution quality in stochastic programs. Quality is defined via the optimality gap and our procedures' output is a confidence interval on this gap. We review a multiple-replications procedure that requires solution of, say, 30 optimization problems and then, we present a result that justifies a computationally simplified single-replication procedure that only requires solving one optimization problem. Even though the single replication procedure is computationally significantly less demanding, the resulting confidence interval might have low coverage probability for small sample sizes for some problems. We provide variants of this procedure that require two replications instead of one and that perform better empirically. We present computational results for a newsvendor problem and for two-stage stochastic linear programs from the literature. We also discuss when the procedures perform well and when they fail, and we propose using ɛ-optimal solutions to strengthen the performance of our procedures.

An approximation technique for robust nonlinear optimization

Abstract: Nonlinear equality and inequality constrained optimization problems with uncertain parameters can be addressed by a robust worst-case formulation that is, however, difficult to treat computationally. In this paper we propose and investigate an approximate robust formulation that employs a linearization of the uncertainty set. In case of any norm bounded parameter uncertainty, this formulation leads to penalty terms employing the respective dual norm of first order derivatives of the constraints. The main advance of the paper is to present two sparsity preserving ways for efficient computation of these derivatives in the case of large scale problems, one similar to the forward mode, the other similar to the reverse mode of automatic differentiation. We show how to generalize the techniques to optimal control problems, and discuss how even infinite dimensional uncertainties can be treated efficiently. Finally, we present optimization results for an example from process engineering, a batch distillation.

A new ILP-based refinement heuristic for Vehicle Routing Problems

Abstract: In this paper we address the Distance-Constrained Capacitated Vehicle Routing Problem (DCVRP), where k minimum-cost routes through a central depot have to be constructed so as to cover all customers while satisfying, for each route, both a capacity and a total-distance-travelled limit.

Hedging Uncertainty: Approximation Algorithms for Stochastic Optimization Problems

Abstract: We study two-stage, finite-scenario stochastic versions of several combinatorial optimization problems, and provide nearly tight approximation algorithms for them. Our problems range from the graph-theoretic (shortest path, vertex cover, facility location) to set-theoretic (set cover, bin packing), and contain representatives with different approximation ratios.The approximation ratio of the stochastic variant of a typical problem is found to be of the same order of magnitude as its deterministic counterpart. Furthermore, we show that common techniques for designing approximation algorithms such as LP rounding, the primal-dual method, and the greedy algorithm, can be adapted to obtain these results.

Computation of Error Bounds for P-matrix Linear Complementarity Problems

Abstract: We give new error bounds for the linear complementarity problem where the involved matrix is a P-matrix. Computation of rigorous error bounds can be turned into a P-matrix linear interval system. Moreover, for the involved matrix being an H-matrix with positive diagonals, an error bound can be found by solving a linear system of equations, which is sharper than the Mathias-Pang error bound. Preliminary numerical results show that the proposed error bound is efficient for verifying accuracy of approximate solutions.

Portfolio construction based on stochastic dominance and target return distributions

Abstract: Mean-risk models have been widely used in portfolio optimization. However, such models may produce portfolios that are dominated with respect to second order stochastic dominance and therefore not optimal for rational and risk-averse investors. This paper considers the problem of constructing a portfolio which is non-dominated with respect to second order stochastic dominance and whose return distribution has specified desirable properties. The problem is multi-objective and is transformed into a single objective problem by using the reference point method, in which target levels, known as aspiration points, are specified for the objective functions. A model is proposed in which the aspiration points relate to ordered outcomes for the portfolio return. This concept is extended by additionally specifying reservation points, which act pre-emptively in the optimization model. The theoretical properties of the models are studied. The performance of the models on real data drawn from the Hang Seng index is also investigated.

Optimization-based simulation of nonsmooth rigid multibody dynamics

Abstract: We present a time-stepping method to simulate rigid multibody dynamics with inelastic collision, contact, and friction. The method progresses with fixed time step without backtracking for collision and solves at every step a strictly convex quadratic program. We prove that a solution sequence of the method converges to the solution of a measure differential inclusion. We present numerical results for a few examples, and we illustrate the difference between the results from our scheme and previous, linear-complementarity-based time-stepping schemes.

Projection results for vehicle routing

Abstract: A variety of integer programming formulations have been proposed for Vehicle Routing Problems (VRPs), including the so-called two- and three-index formulations, the set partitioning formulation, and various formulations based on extra variables representing the flow of one or more commodities. Until now, there has not been a systematic study of how these formulations relate to each other. An exception is a paper of Luis Gouveia, which shows that a one-commodity flow formulation of Gavish and Graves yields, by projection, certain `multistar' inequalities in the two-index space. We give a survey of formulations for the capacitated VRP, and then present various results of a similar flavour to those of Gouveia. In particular, we show that: – the three-index formulation, augmented by certain families of valid inequalities, gives the same lower bound as the two-index formulation, augmented by certain simpler families of valid inequalities, – the two-commodity flow formulation of Baldacci et al. gives the same lower bound and the same multistar inequalities as the one-commodity Gavish and Graves formulation, – a certain non-standard multi-commodity flow formulation, with one commodity per customer, implies by projection certain `hypotour-like' inequalities in the two-index space, – the set partitioning formulation implies by projection both multistar and hypotour-like inequalities in the two-index space. We also briefly look at some other variants of the VRP, such as the VRP with time windows, and derive multistar-like inequalities for them. We also present polynomial-time separation algorithms for some of the new inequalities.

Formulations and Valid Inequalities for the Heterogeneous Vehicle Routing Problem

Abstract: We consider the vehicle routing problem where one can choose among vehicles with different costs and capacities to serve the trips. We develop six different formulations: the first four based on Miller-Tucker-Zemlin constraints and the last two based on flows. We compare the linear programming bounds of these formulations. We derive valid inequalities and lift some of the constraints to improve the lower bounds. We generalize and strengthen subtour elimination and generalized large multistar inequalities.

A 2 + ɛ approximation algorithm for the k -MST problem

Abstract: For any ɛ > 0 we give a (2 + ɛ)-approximation algorithm for the problem of finding a minimum tree spanning any k vertices in a graph (k-MST), improving a 3-approximation algorithm by Garg [10]. As in [10] the algorithm extends to a (2 + ɛ)-approximation algorithm for the minimum tour that visits any k vertices, provided the edge costs satisfy the triangle inequality.