scispace - formally typeset
Search or ask a question

Showing papers in "Mathematical Programming in 2014"


Journal ArticleDOI
TL;DR: A self-contained convergence analysis framework is derived and it is established that each bounded sequence generated by PALM globally converges to a critical point.
Abstract: We introduce a proximal alternating linearized minimization (PALM) algorithm for solving a broad class of nonconvex and nonsmooth minimization problems. Building on the powerful Kurdyka---?ojasiewicz property, we derive a self-contained convergence analysis framework and establish that each bounded sequence generated by PALM globally converges to a critical point. Our approach allows to analyze various classes of nonconvex-nonsmooth problems and related nonconvex proximal forward---backward algorithms with semi-algebraic problem's data, the later property being shared by many functions arising in a wide variety of fundamental applications. A by-product of our framework also shows that our results are new even in the convex setting. As an illustration of the results, we derive a new and simple globally convergent algorithm for solving the sparse nonnegative matrix factorization problem.

1,563 citations


Journal ArticleDOI
TL;DR: In this paper, a randomized block-coordinate descent method for minimizing the sum of a smooth and a simple nonsmooth block-separable convex function was developed, and it was shown that the algorithm converges linearly.
Abstract: In this paper we develop a randomized block-coordinate descent method for minimizing the sum of a smooth and a simple nonsmooth block-separable convex function and prove that it obtains an $$\varepsilon $$ -accurate solution with probability at least $$1-\rho $$ in at most $$O((n/\varepsilon ) \log (1/\rho ))$$ iterations, where $$n$$ is the number of blocks. This extends recent results of Nesterov (SIAM J Optim 22(2): 341–362, 2012), which cover the smooth case, to composite minimization, while at the same time improving the complexity by the factor of 4 and removing $$\varepsilon $$ from the logarithmic term. More importantly, in contrast with the aforementioned work in which the author achieves the results by applying the method to a regularized version of the objective function with an unknown scaling factor, we show that this is not necessary, thus achieving first true iteration complexity bounds. For strongly convex functions the method converges linearly. In the smooth case we also allow for arbitrary probability vectors and non-Euclidean norms. Finally, we demonstrate numerically that the algorithm is able to solve huge-scale $$\ell _1$$ -regularized least squares problems with a billion variables.

790 citations


Journal ArticleDOI
TL;DR: It is demonstrated that the superiority of fast gradient methods over the classical ones is no longer absolute when an inexact oracle is used, and it is proved that, contrary to simple gradient schemes,fast gradient methods must necessarily suffer from error accumulation.
Abstract: We introduce the notion of inexact first-order oracle and analyze the behavior of several first-order methods of smooth convex optimization used with such an oracle. This notion of inexact oracle naturally appears in the context of smoothing techniques, Moreau---Yosida regularization, Augmented Lagrangians and many other situations. We derive complexity estimates for primal, dual and fast gradient methods, and study in particular their dependence on the accuracy of the oracle and the desired accuracy of the objective function. We observe that the superiority of fast gradient methods over the classical ones is no longer absolute when an inexact oracle is used. We prove that, contrary to simple gradient schemes, fast gradient methods must necessarily suffer from error accumulation. Finally, we show that the notion of inexact oracle allows the application of first-order methods of smooth convex optimization to solve non-smooth or weakly smooth convex problems.

531 citations


Journal ArticleDOI
TL;DR: This paper studies the relationship between optimality conditions in nonlinear programming theory and finite convergence of Lasserre’s hierarchy and results imply that, under archimedeanness, the hierarchy has finite convergence generically.
Abstract: Lasserre's hierarchy is a sequence of semidefinite relaxations for solving polynomial optimization problems globally. This paper studies the relationship between optimality conditions in nonlinear programming theory and finite convergence of Lasserre's hierarchy. Our main results are: (i) Lasserre's hierarchy has finite convergence when the constraint qualification, strict complementarity and second order sufficiency conditions hold at every global minimizer, under the standard archimedean condition; the proof uses a result of Marshall on boundary hessian conditions. (ii) These optimality conditions are all satisfied at every local minimizer if a finite set of polynomials, which are in the coefficients of input polynomials, do not vanish at the input data (i.e., they hold in a Zariski open set). This implies that, under archimedeanness, Lasserre's hierarchy has finite convergence generically.

225 citations


Journal ArticleDOI
TL;DR: In this paper, the worst-case performance of first-order black-box optimization methods is analyzed for smooth unconstrained convex minimization over the Euclidean space.
Abstract: We introduce a novel approach for analyzing the worst-case performance of first-order black-box optimization methods. We focus on smooth unconstrained convex minimization over the Euclidean space. Our approach relies on the observation that by definition, the worst-case behavior of a black-box optimization method is by itself an optimization problem, which we call the performance estimation problem (PEP). We formulate and analyze the PEP for two classes of first-order algorithms. We first apply this approach on the classical gradient method and derive a new and tight analytical bound on its performance. We then consider a broader class of first-order black-box methods, which among others, include the so-called heavy-ball method and the fast gradient schemes. We show that for this broader class, it is possible to derive new bounds on the performance of these methods by solving an adequately relaxed convex semidefinite PEP. Finally, we show an efficient procedure for finding optimal step sizes which results in a first-order black-box method that achieves best worst-case performance.

180 citations


Journal ArticleDOI
TL;DR: In this paper, the authors present a new approach for exactly solving chance-constrained mathematical programs having discrete distributions with finite support and random polyhedral constraints, using both decomposition and integer programming techniques to combine the results of these subproblems to yield strong valid inequalities.
Abstract: We present a new approach for exactly solving chance-constrained mathematical programs having discrete distributions with finite support and random polyhedral constraints. Such problems have been notoriously difficult to solve due to nonconvexity of the feasible region, and most available methods are only able to find provably good solutions in certain very special cases. Our approach uses both decomposition, to enable processing subproblems corresponding to one possible outcome at a time, and integer programming techniques, to combine the results of these subproblems to yield strong valid inequalities. Computational results on a chance-constrained formulation of a resource planning problem inspired by a call center staffing application indicate the approach works significantly better than both an existing mixed-integer programming formulation and a simple decomposition approach that does not use strong valid inequalities. We also demonstrate how the approach can be used to efficiently solve for a sequence of risk levels, as would be done when solving for the efficient frontier of risk and cost.

160 citations


Journal ArticleDOI
TL;DR: New methods for the Lipschitz continuous IRL1 minimization problems are developed and it is shown that any accumulation point of the sequence generated by these methods is a first-order stationary point, provided that the approximation parameter ϵ is below a computable threshold value.
Abstract: In this paper we study general $$l_p$$ l p regularized unconstrained minimization problems. In particular, we derive lower bounds for nonzero entries of the first- and second-order stationary points and hence also of local minimizers of the $$l_p$$ l p minimization problems. We extend some existing iterative reweighted $$l_1$$ l 1 ( $$\mathrm{IRL}_1$$ IRL 1 ) and $$l_2$$ l 2 ( $$\mathrm{IRL}_2$$ IRL 2 ) minimization methods to solve these problems and propose new variants for them in which each subproblem has a closed-form solution. Also, we provide a unified convergence analysis for these methods. In addition, we propose a novel Lipschitz continuous $${\epsilon }$$ ∈ -approximation to $$\Vert x\Vert ^p_p$$ ? x ? p p . Using this result, we develop new $$\mathrm{IRL}_1$$ IRL 1 methods for the $$l_p$$ l p minimization problems and show that any accumulation point of the sequence generated by these methods is a first-order stationary point, provided that the approximation parameter $${\epsilon }$$ ∈ is below a computable threshold value. This is a remarkable result since all existing iterative reweighted minimization methods require that $${\epsilon }$$ ∈ be dynamically updated and approach zero. Our computational results demonstrate that the new $$\mathrm{IRL}_1$$ IRL 1 method and the new variants generally outperform the existing $$\mathrm{IRL}_1$$ IRL 1 methods (Chen and Zhou in 2012; Foucart and Lai in Appl Comput Harmon Anal 26:395---407, 2009).

126 citations


Journal ArticleDOI
TL;DR: Theoretical results show that the minimizers of the L_q-L_p minimization problem have various attractive features due to the concavity and non-Lipschitzian property of the regularization function.
Abstract: We consider the unconstrained $$L_q$$ - $$L_p$$ minimization: find a minimizer of $$\Vert Ax-b\Vert ^q_q+\lambda \Vert x\Vert ^p_p$$ for given $$A \in R^{m\times n}$$ , $$b\in R^m$$ and parameters $$\lambda >0$$ , $$p\in [0, 1)$$ and $$q\ge 1$$ . This problem has been studied extensively in many areas. Especially, for the case when $$q=2$$ , this problem is known as the $$L_2-L_p$$ minimization problem and has found its applications in variable selection problems and sparse least squares fitting for high dimensional data. Theoretical results show that the minimizers of the $$L_q$$ - $$L_p$$ problem have various attractive features due to the concavity and non-Lipschitzian property of the regularization function $$\Vert \cdot \Vert ^p_p$$ . In this paper, we show that the $$L_q$$ - $$L_p$$ minimization problem is strongly NP-hard for any $$p\in [0,1)$$ and $$q\ge 1$$ , including its smoothed version. On the other hand, we show that, by choosing parameters $$(p,\lambda )$$ carefully, a minimizer, global or local, will have certain desired sparsity. We believe that these results provide new theoretical insights to the studies and applications of the concave regularized optimization problems.

120 citations


Journal ArticleDOI
TL;DR: A globally convergent algorithm based on a potential reduction approach is used to solve a general quasi-variational inequality by using its Karush–Kuhn–Tucker conditions, vastly broadening the class of problems that can be solved with theoretical guarantees.
Abstract: We propose to solve a general quasi-variational inequality by using its Karush–Kuhn–Tucker conditions. To this end we use a globally convergent algorithm based on a potential reduction approach. We establish global convergence results for many interesting instances of quasi-variational inequalities, vastly broadening the class of problems that can be solved with theoretical guarantees. Our numerical testings are very promising and show the practical viability of the approach.

107 citations


Journal ArticleDOI
TL;DR: A very efficient implementation of subgradient iterations, which total cost depends logarithmically in the dimension, is suggested, based on a recursive update of the results of matrix/vector products and the values of symmetric functions.
Abstract: We consider a new class of huge-scale problems, the problems with sparse subgradients. The most important functions of this type are piece-wise linear. For optimization problems with uniform sparsity of corresponding linear operators, we suggest a very efficient implementation of subgradient iterations, which total cost depends logarithmically in the dimension. This technique is based on a recursive update of the results of matrix/vector products and the values of symmetric functions. It works well, for example, for matrices with few nonzero diagonals and for max-type functions. We show that the updating technique can be efficiently coupled with the simplest subgradient methods, the unconstrained minimization method by B.Polyak, and the constrained minimization scheme by N.Shor. Similar results can be obtained for a new nonsmooth random variant of a coordinate descent scheme. We present also the promising results of preliminary computational experiments.

106 citations


Journal ArticleDOI
TL;DR: A synthetic convergence theory is state, in which the main arguments are highlighted and which assumption is used to establish each intermediate result is specified, which is comprehensive and generalizes in various ways a number of algorithms proposed in the literature.
Abstract: The last few years have seen the advent of a new generation of bundle methods, capable to handle inexact oracles, polluted by "noise". Proving convergence of a bundle method is never simple and coping with inexact oracles substantially increases the technicalities. Besides, several variants exist to deal with noise, each one needing an ad hoc proof to show convergence. We state a synthetic convergence theory, in which we highlight the main arguments and specify which assumption is used to establish each intermediate result. The framework is comprehensive and generalizes in various ways a number of algorithms proposed in the literature. Based on the ingredients of our synthetic theory, we consider various bundle methods adapted to oracles for which high accuracy is possible, yet it is preferable not to make exact calculations often, because they are too time consuming.

Journal ArticleDOI
TL;DR: An iterative hard thresholding (IHT) method and its variant for solving regularized box constrained convex programming and it is shown that the sequence generated by these methods converges to a local minimizer.
Abstract: In this paper we consider $$l_0$$ l 0 regularized convex cone programming problems. In particular, we first propose an iterative hard thresholding (IHT) method and its variant for solving $$l_0$$ l 0 regularized box constrained convex programming. We show that the sequence generated by these methods converges to a local minimizer. Also, we establish the iteration complexity of the IHT method for finding an $${{\epsilon }}$$ ∈ -local-optimal solution. We then propose a method for solving $$l_0$$ l 0 regularized convex cone programming by applying the IHT method to its quadratic penalty relaxation and establish its iteration complexity for finding an $${{\epsilon }}$$ ∈ -approximate local minimizer. Finally, we propose a variant of this method in which the associated penalty parameter is dynamically updated, and show that every accumulation point is a local izer of the problem.

Journal ArticleDOI
TL;DR: The new algorithm improves on known methods and, when particularized to KKT systems derived from optimality conditions for constrained optimization or variational inequalities, it has theoretical advantages even over methods specifically designed to solve such systems.
Abstract: We define a new Newton-type method for the solution of constrained systems of equations and analyze in detail its properties. Under suitable conditions, that do not include differentiability or local uniqueness of solutions, the method converges locally quadratically to a solution of the system, thus filling an important gap in the existing theory. The new algorithm improves on known methods and, when particularized to KKT systems derived from optimality conditions for constrained optimization or variational inequalities, it has theoretical advantages even over methods specifically designed to solve such systems.

Journal ArticleDOI
TL;DR: The proposed decomposition algorithms akin to the $$L$$-shaped or Benders’ methods are developed by utilizing Gomory cuts to obtain iteratively tighter approximations of the second-stage integer programs.
Abstract: We consider a class of two-stage stochastic integer programs with binary variables in the rst stage and general integer variables in the second stage We develop decomposition algorithms akin to the L-shaped or Benders’ methods by utilizing Gomory cuts to obtain iteratively tighter approximations of the second-stage integer programs We show that the proposed methodology is exible in that it allows several modes of implementation, all of which lead to nitely convergent algorithms We illustrate our algorithms using examples from the literature We report computational results using the stochastic server location problem instances which suggest that our decomposition-based approach scales better with increases in the number of scenarios than a state-of-the art solver which was used to solve the deterministic equivalent formulation

Journal ArticleDOI
TL;DR: It is shown that when the quadratic forms are simultaneously diagonalizable (SD), it is possible to derive an equivalent convex problem, which is a conic quadratics (CQ) one, and as such is significantly more tractable than a semidefinite problem.
Abstract: The problem of minimizing a quadratic objective function subject to one or two quadratic constraints is known to have a hidden convexity property, even when the quadratic forms are indefinite. The equivalent convex problem is a semidefinite one, and the equivalence is based on the celebrated S-lemma. In this paper, we show that when the quadratic forms are simultaneously diagonalizable (SD), it is possible to derive an equivalent convex problem, which is a conic quadratic (CQ) one, and as such is significantly more tractable than a semidefinite problem. The SD condition holds for free for many problems arising in applications, in particular, when deriving robust counterparts of quadratic, or conic quadratic, constraints affected by implementation error. The proof of the hidden CQ property is constructive and does not rely on the S-lemma. This fact may be significant in discovering hidden convexity in some nonquadratic problems.

Journal ArticleDOI
TL;DR: This paper analyzes and discusses the well-posedness of two new variants of the so-called sweeping process introduced by Moreau in the early 70s and shows how elegant modern convex analysis was influenced by moreau’s seminal work.
Abstract: In this paper, we analyze and discuss the well-posedness of two new variants of the so-called sweeping process, introduced by Moreau in the early 70s (Moreau in Sem Anal Convexe Montpellier, 1971) with motivation in plasticity theory. The first new variant is concerned with the perturbation of the normal cone to the moving convex subset $$C(t)$$ C ( t ) , supposed to have a bounded variation, by a Lipschitz mapping. Under some assumptions on the data, we show that the perturbed differential measure inclusion has one and only one right continuous solution with bounded variation. The second variant, for which a large analysis is made, concerns a first order sweeping process with velocity in the moving set $$C(t)$$ C ( t ) . This class of problems subsumes as a particular case, the evolution variational inequalities [widely used in applied mathematics and unilateral mechanics (Duvaut and Lions in Inequalities in mechanics and physics. Springer, Berlin, 1976]. Assuming that the moving subset $$C(t)$$ C ( t ) has a continuous variation for every $$t\in [0,T]$$ t ? [ 0 , T ] with $$C(0)$$ C ( 0 ) bounded, we show that the problem has at least a Lipschitz continuous solution. The well-posedness of this class of sweeping process is obtained under the coercivity assumption of the involved operator. We also discuss some applications of the sweeping process to the study of vector hysteresis operators in the elastoplastic model (Kreja?i in Eur J Appl Math 2:281---292, 1991), to the planning procedure in mathematical economy (Henry in J Math Anal Appl 41:179---186, 1973 and Cornet in J. Math. Anal. Appl. 96:130---147, 1983), and to nonregular electrical circuits containing nonsmooth electronic devices like diodes (Acary et al. Nonsmooth modeling and simulation for switched circuits. Lecture notes in electrical engineering. Springer, New York 2011). The theoretical results are supported by some numerical simulations to prove the efficiency of the algorithm used in the existence proof. Our methodology is based only on tools from convex analysis. Like other papers in this collection, we show in this presentation how elegant modern convex analysis was influenced by Moreau's seminal work.

Journal ArticleDOI
TL;DR: It is concluded that, under mild assumptions, solving a robust LSP or SOCP under matrix-norm uncertainty or polyhedral uncertainty is equivalent to solving a semi-definite linear programming problem and so, their solutions can be validated in polynomial time.
Abstract: The trust-region problem, which minimizes a nonconvex quadratic function over a ball, is a key subproblem in trust-region methods for solving nonlinear optimization problems. It enjoys many attractive properties such as an exact semi-definite linear programming relaxation (SDP-relaxation) and strong duality. Unfortunately, such properties do not, in general, hold for an extended trust-region problem having extra linear constraints. This paper shows that two useful and powerful features of the classical trust-region problem continue to hold for an extended trust-region problem with linear inequality constraints under a new dimension condition. First, we establish that the class of extended trust-region problems has an exact SDP-relaxation, which holds without the Slater constraint qualification. This is achieved by proving that a system of quadratic and affine functions involved in the model satisfies a range-convexity whenever the dimension condition is fulfilled. Second, we show that the dimension condition together with the Slater condition ensures that a set of combined first and second-order Lagrange multiplier conditions is necessary and sufficient for global optimality of the extended trust-region problem and consequently for strong duality. Through simple examples we also provide an insightful account of our development from SDP-relaxation to strong duality. Finally, we show that the dimension condition is easily satisfied for the extended trust-region model that arises from the reformulation of a robust least squares problem (LSP) as well as a robust second order cone programming model problem (SOCP) as an equivalent semi-definite linear programming problem. This leads us to conclude that, under mild assumptions, solving a robust LSP or SOCP under matrix-norm uncertainty or polyhedral uncertainty is equivalent to solving a semi-definite linear programming problem and so, their solutions can be validated in polynomial time.

Journal ArticleDOI
TL;DR: This paper derives explicit formulas for the proximal and limiting normal cone of the graph of the normal cone to the positive semidefinite cone and gives constraint qualifications under which a local solution of SDCMPCC is a S-, M- and C-stationary point.
Abstract: In this paper we consider a mathematical program with semidefinite cone complementarity constraints (SDCMPCC). Such a problem is a matrix analogue of the mathematical program with (vector) complementarity constraints (MPCC) and includes MPCC as a special case. We first derive explicit formulas for the proximal and limiting normal cone of the graph of the normal cone to the positive semidefinite cone. Using these formulas and classical nonsmooth first order necessary optimality conditions we derive explicit expressions for the strong-, Mordukhovich- and Clarke- (S-, M- and C-)stationary conditions. Moreover we give constraint qualifications under which a local solution of SDCMPCC is a S-, M- and C-stationary point. Moreover we show that applying these results to MPCC produces new and weaker necessary optimality conditions.

Journal ArticleDOI
TL;DR: It is proved that the regularized smoothing method converges to the least norm solution of the differential variational inequality, which is a solution ofThe dynamic NEPSC as the regularization parameter.
Abstract: The dynamic Nash equilibrium problem with shared constraints (NEPSC) involves a dynamic decision process with multiple players, where not only the players' cost functionals but also their admissible control sets depend on the rivals' decision variables through shared constraints. For a class of the dynamic NEPSC, we propose a differential variational inequality formulation. Using this formulation, we show the existence of solutions of the dynamic NEPSC, and develop a regularized smoothing method to find a solution of it. We prove that the regularized smoothing method converges to the least norm solution of the differential variational inequality, which is a solution of the dynamic NEPSC as the regularization parameter $$\lambda $$ ? and smoothing parameter $$\mu $$ μ go to zero with the order $$\mu =o(\lambda )$$ μ = o ( ? ) . Numerical examples are given to illustrate the existence and convergence results.

Journal ArticleDOI
TL;DR: It is shown that the correct partition of the objects and features can be recovered from the optimal solution of a semidefinite program in the case that the given data consists of several disjoint sets of objects exhibiting similar features.
Abstract: Identifying clusters of similar objects in data plays a significant role in a wide range of applications As a model problem for clustering, we consider the densest $$k$$ k -disjoint-clique problem, whose goal is to identify the collection of $$k$$ k disjoint cliques of a given weighted complete graph maximizing the sum of the densities of the complete subgraphs induced by these cliques In this paper, we establish conditions ensuring exact recovery of the densest $$k$$ k cliques of a given graph from the optimal solution of a particular semidefinite program In particular, the semidefinite relaxation is exact for input graphs corresponding to data consisting of $$k$$ k large, distinct clusters and a smaller number of outliers This approach also yields a semidefinite relaxation with similar recovery guarantees for the biclustering problem Given a set of objects and a set of features exhibited by these objects, biclustering seeks to simultaneously group the objects and features according to their expression levels This problem may be posed as that of partitioning the nodes of a weighted bipartite complete graph such that the sum of the densities of the resulting bipartite complete subgraphs is maximized As in our analysis of the densest $$k$$ k -disjoint-clique problem, we show that the correct partition of the objects and features can be recovered from the optimal solution of a semidefinite program in the case that the given data consists of several disjoint sets of objects exhibiting similar features Empirical evidence from numerical experiments supporting these theoretical guarantees is also provided

Journal ArticleDOI
TL;DR: This paper defines a class of linear conic programming involving the epigraphs of five commonly used matrix norms and the well studied symmetric cone and calls for more insightful research on MCP so that it can serve as a basic tool to solve more challenging convex matrix optimization problems in years to come.
Abstract: In this paper, we define a class of linear conic programming (which we call matrix cone programming or MCP) involving the epigraphs of five commonly used matrix norms and the well studied symmetric cone. MCP has recently been found to have many important applications, for example, in nuclear norm relaxations of affine rank minimization problems. In order to make the defined MCP tractable and meaningful, we must first understand the structure of these epigraphs. So far, only the epigraph of the Frobenius matrix norm, which can be regarded as a second order cone, has been well studied. Here, we take an initial step to study several important properties, including its closed form solution, calm Bouligand-differentiability and strong semismoothness, of the metric projection operator over the epigraph of the $$l_1,\,l_\infty $$, spectral or operator, and nuclear matrix norm, respectively. These properties make it possible to apply augmented Lagrangian methods, which have recently received a great deal of interests due to their high efficiency in solving large scale semidefinite programming, to this class of MCP problems. The work done in this paper is far from comprehensive. Rather it is intended as a starting point to call for more insightful research on MCP so that it can serve as a basic tool to solve more challenging convex matrix optimization problems in years to come.

Journal ArticleDOI
TL;DR: A smoothing projected gradient algorithm is designed for a general optimization problem with a nonsmooth inequality constraint and a convex set constraint and preliminary numerical experiments show that the algorithm is efficient for solving the simple bilevel program.
Abstract: In this paper, we consider a simple bilevel program where the lower level program is a nonconvex minimization problem with a convex set constraint and the upper level program has a convex set constraint. By using the value function of the lower level program, we reformulate the bilevel program as a single level optimization problem with a nonsmooth inequality constraint and a convex set constraint. To deal with such a nonsmooth and nonconvex optimization problem, we design a smoothing projected gradient algorithm for a general optimization problem with a nonsmooth inequality constraint and a convex set constraint. We show that, if the sequence of penalty parameters is bounded then any accumulation point is a stationary point of the nonsmooth optimization problem and, if the generated sequence is convergent and the extended Mangasarian-Fromovitz constraint qualification holds at the limit then the limit point is a stationary point of the nonsmooth optimization problem. We apply the smoothing projected gradient algorithm to the bilevel program if a calmness condition holds and to an approximate bilevel program otherwise. Preliminary numerical experiments show that the algorithm is efficient for solving the simple bilevel program.

Journal ArticleDOI
TL;DR: The complexity of finding first-order critical points for the general smooth constrained optimization problem is shown to be no worse that O(\epsilon ^{-2}) in terms of function and constraints evaluations, and the presence of possibly nonlinear/nonconvex inequality/equality constraints is irrelevant for this bound to apply.
Abstract: The complexity of finding $$\epsilon $$-approximate first-order critical points for the general smooth constrained optimization problem is shown to be no worse that $$O(\epsilon ^{-2})$$ in terms of function and constraints evaluations. This result is obtained by analyzing the worst-case behaviour of a first-order short-step homotopy algorithm consisting of a feasibility phase followed by an optimization phase, and requires minimal assumptions on the objective function. Since a bound of the same order is known to be valid for the unconstrained case, this leads to the conclusion that the presence of possibly nonlinear/nonconvex inequality/equality constraints is irrelevant for this bound to apply.

Journal ArticleDOI
TL;DR: A simple mechanism to transform multi-criteria approximation schemes into pure approximation schemes for problems whose feasible solutions define an independence system is presented and can be applied to the above bipartite matching algorithm, hence obtaining a pure PTAS.
Abstract: A natural way to deal with multiple, partially conflicting objectives is turning all the objectives but one into budget constraints. Many classical optimization problems, such as maximum spanning tree and forest, shortest path, maximum weight (perfect) matching, maximum weight independent set (basis) in a matroid or in the intersection of two matroids, become NP-hard even with one budget constraint. Still, for most of these problems efficient deterministic and randomized approximation schemes are known. Not much is known however about the case of two or more budgets: filling this gap, at least partially, is the main goal of this paper. In more detail, we obtain the following main results: Using iterative rounding for the first time in multi-objective optimization, we obtain multi-criteria PTASs (which slightly violate the budget constraints) for spanning tree, matroid basis, and bipartite matching with $$k=O(1)$$ k = O ( 1 ) budget constraints. We present a simple mechanism to transform multi-criteria approximation schemes into pure approximation schemes for problems whose feasible solutions define an independence system. This gives improved algorithms for several problems. In particular, this mechanism can be applied to the above bipartite matching algorithm, hence obtaining a pure PTAS. We show that points in low-dimensional faces of any matroid polytope are almost integral, an interesting result on its own. This gives a deterministic approximation scheme for $$k$$ k -budgeted matroid independent set. We present a deterministic approximation scheme for $$k$$ k -budgeted matching (in general graphs), where $$k=O(1)$$ k = O ( 1 ) . Interestingly, to show that our procedure works, we rely on a non-constructive result by Stromquist and Woodall, which is based on the Ham Sandwich Theorem.

Journal ArticleDOI
TL;DR: An improved algorithm for finding exact solutions to Max-Cut and the related binary quadratic programming problem, both classic problems of combinatorial optimization is presented, and extensive experiments show that the algorithm dominates the best existing method.
Abstract: We present an improved algorithm for finding exact solutions to Max-Cut and the related binary quadratic programming problem, both classic problems of combinatorial optimization. The algorithm uses a branch-(and-cut-)and-bound paradigm, using standard valid inequalities and nonstandard semidefinite bounds. More specifically, we add a quadratic regularization term to the strengthened semidefinite relaxation in order to use a quasi-Newton method to compute the bounds. The ratio of the tightness of the bounds to the time required to compute them can be controlled by two real parameters; we show how adjusting these parameters and the set of strengthening inequalities gives us a very efficient bounding procedure. Embedding our bounding procedure in a generic branch-and-bound platform, we get a competitive algorithm: extensive experiments show that our algorithm dominates the best existing method.

Journal ArticleDOI
TL;DR: A version of the bundle scheme for convex nondifferentiable optimization suitable for the case of a sum-function where some of the components are “easy”, that is, they are Lagrangian functions of explicitly known compact convex programs.
Abstract: We propose a version of the bundle scheme for convex nondifferentiable optimization suitable for the case of a sum-function where some of the components are "easy", that is, they are Lagrangian functions of explicitly known compact convex programs. This corresponds to a stabilized partial Dantzig---Wolfe decomposition, where suitably modified representations of the "easy" convex subproblems are inserted in the master problem as an alternative to iteratively inner-approximating them by extreme points, thus providing the algorithm with exact information about a part of the dual objective function. The resulting master problems are potentially larger and less well-structured than the standard ones, ruling out the available specialized techniques and requiring the use of general-purpose solvers for their solution; this strongly favors piecewise-linear stabilizing terms, as opposed to the more usual quadratic ones, which in turn may have an adverse effect on the convergence speed of the algorithm, so that the overall performance may depend on appropriate tuning of all these aspects. Yet, very good computational results are obtained in at least one relevant application: the computation of tight lower bounds for Fixed-Charge Multicommodity Min-Cost Flow problems.

Journal ArticleDOI
TL;DR: The Gram dimension of a graph G is the smallest integer such that any partial real symmetric matrix, whose entries are specified on the diagonal and at the off-diagonal positions corresponding to edges of G, can be completed to a positive semidefinite matrix of rank at most k.
Abstract: The Gram dimension $$\mathrm{gd}(G)$$ of a graph $$G$$ is the smallest integer $$k\ge 1$$ such that any partial real symmetric matrix, whose entries are specified on the diagonal and at the off-diagonal positions corresponding to edges of $$G$$ , can be completed to a positive semidefinite matrix of rank at most $$k$$ (assuming a positive semidefinite completion exists). For any fixed $$k$$ the class of graphs satisfying $$\mathrm{gd}(G) \le k$$ is minor closed, hence it can be characterized by a finite list of forbidden minors. We show that the only minimal forbidden minor is $$K_{k+1}$$ for $$k\le 3$$ and that there are two minimal forbidden minors: $$K_5$$ and $$K_{2,2,2}$$ for $$k=4$$ . We also show some close connections to Euclidean realizations of graphs and to the graph parameter $$ u ^=(G)$$ of van der Holst (Combinatorica 23(4):633---651, 2003). In particular, our characterization of the graphs with $$\mathrm{gd}(G)\le 4$$ implies the forbidden minor characterization of the 3-realizable graphs of Belk (Discret Comput Geom 37:139---162, 2007) and Belk and Connelly (Discret Comput Geom 37:125---137, 2007) and of the graphs with $$ u ^=(G) \le 4$$ of van der Holst (Combinatorica 23(4):633---651, 2003).

Journal ArticleDOI
TL;DR: A generalization of Koenker–Basset error is derived which lays a foundation for superquantile regression as a higher-order extension of quantile regression.
Abstract: Random variables can be described by their cumulative distribution functions, a class of nondecreasing functions on the real line. Those functions can in turn be identified, after the possible vertical gaps in their graphs are filled in, with maximal monotone relations. Such relations are known to be the subdifferentials of convex functions. Analysis of these connections yields new insights. The generalized inversion operation between distribution functions and quantile functions corresponds to graphical inversion of monotone relations. In subdifferential terms, it corresponds to passing to conjugate convex functions under the Legendre---Fenchel transform. Among other things, this shows that convergence in distribution for sequences of random variables is equivalent to graphical convergence of the monotone relations and epigraphical convergence of the associated convex functions. Measures of risk that employ quantiles (VaR) and superquantiles (CVaR), either individually or in mixtures, are illuminated in this way. Formulas for their calculation are seen from a perspective that reveals how they were discovered. The approach leads further to developments in which the superquantiles for a given distribution are interpreted as the quantiles for an overlying "superdistribution." In this way a generalization of Koenker---Basset error is derived which lays a foundation for superquantile regression as a higher-order extension of quantile regression.

Journal ArticleDOI
TL;DR: The $$k$$k-disjoint-clique problem is NP-hard, but it is shown that a convex relaxation can solve it in polynomial time for input instances constructed in a certain way.
Abstract: We consider the k -disjoint-clique problem. The input is an undirected graph G in which the nodes represent data items, and edges indicate a similarity between the corresponding items. The problem is to find within the graph k disjoint cliques that cover the maximum number of nodes of G. This problem may be understood as a general way to pose the classical ‘clustering’ problem. In clustering, one is given data items and a distance function, and one wishes to partition the data into disjoint clusters of data items, such that the items in each cluster are close to each other. Our formulation additionally allows ‘noise’ nodes to be present in the input data that are not part of any of the cliques. The k -disjoint-clique problem is NP-hard, but we show that a convex relaxation can solve it in polynomial time for input instances constructed in a certain way. The input instances for which our algorithm finds the optimal solution consist of k disjoint large cliques (called ‘planted cliques’) that are then obscured by noise edges inserted either at random or by an adversary, as well as additional nodes not belonging to any of the k planted cliques.

Journal ArticleDOI
TL;DR: The equivalent Lagrangian problem is derived and it is shown that it is a convex stochastic programming problem and the finite support case can be solved by using an equivalent second order cone programming reformulation.
Abstract: We present three different robust frameworks using probabilistic ambiguity descriptions of the data in least squares problems. These probability ambiguity descriptions are given by: (1) confidence region over the first two moments; (2) bounds on the probability measure with moments constraints; (3) the Kantorovich probability distance from a given measure. For the first case, we give an equivalent formulation and show that the optimization problem can be solved using a semidefinite optimization reformulation or polynomial time algorithms. For the second case, we derive the equivalent Lagrangian problem and show that it is a convex stochastic programming problem. We further analyze three special subcases: (i) finite support; (ii) measure bounds by a reference probability measure; (iii) measure bounds by two reference probability measures with known density functions. We show that case (i) has an equivalent semidefinite programming reformulation and the sample average approximations of case (ii) and (iii) have equivalent semidefinite programming reformulations. For ambiguity description (3), we show that the finite support case can be solved by using an equivalent second order cone programming reformulation.