Showing papers on "Quadratically constrained quadratic program published in 2010"
TL;DR: Conditions under which strong duality holds and efficient algorithms for the optimal beamforming problem are given and rank reduction procedures to achieve a lower rank solution are proposed.
Abstract: Consider a downlink communication system where multiantenna base stations transmit independent data streams to decentralized single-antenna users over a common frequency band. The goal of the base stations is to jointly adjust the beamforming vectors to minimize the transmission powers while ensuring the signal-to-interference-noise ratio requirement of each user within the system. At the same time, it may be necessary to keep the interference generated on other coexisting systems under a certain tolerable level. In addition, one may want to include general individual shaping constraints on the beamforming vectors. This beamforming problem is a separable homogeneous quadratically constrained quadratic program, and it is difficult to solve in general. In this paper, we give conditions under which strong duality holds and propose efficient algorithms for the optimal beamforming problem. First, we study rank-constrained solutions of general separable semidefinite programs (SDPs) and propose rank reduction procedures to achieve a lower rank solution. Then we show that the SDP relaxation of three classes of optimal beamforming problem always has a rank-one solution, which can be obtained by invoking the rank reduction procedures.
559 citations
TL;DR: The geometry of that manifold is discussed and a second-order optimization method with guaranteed quadratic convergence is derived, which provides some conditions on the rank of the factorization to ensure equivalence with the original problem.
Abstract: We propose an algorithm for solving optimization problems defined on a subset of the cone of symmetric positive semidefinite matrices. This algorithm relies on the factorization $X=YY^T$, where the number of columns of $Y$ fixes an upper bound on the rank of the positive semidefinite matrix $X$. It is thus very effective for solving problems that have a low-rank solution. The factorization $X=YY^T$ leads to a reformulation of the original problem as an optimization on a particular quotient manifold. The present paper discusses the geometry of that manifold and derives a second-order optimization method with guaranteed quadratic convergence. It furthermore provides some conditions on the rank of the factorization to ensure equivalence with the original problem. In contrast to existing methods, the proposed algorithm converges monotonically to the sought solution. Its numerical efficiency is evaluated on two applications: the maximal cut of a graph and the problem of sparse principal component analysis.
280 citations
01 Jan 2010
TL;DR: The purpose of this survey is to introduce the field to interested readers in the optimization community who wish to get an understanding of the basic concepts and recent developments inCopositive programming, including modeling issues and applications, the connection to semidefinite programming and sum-of-squares approaches, as well as algorithmic solution approaches for copositive programs.
Abstract: Copositive programming is a relatively young field in mathematical optimization. It can be seen as a generalization of semidefinite programming, since it means optimizing over the cone of so called copositive matrices. Like semidefinite programming, it has proved particularly useful in combinatorial and quadratic optimization. The purpose of this survey is to introduce the field to interested readers in the optimization community who wish to get an understanding of the basic concepts and recent developments in copositive programming, including modeling issues and applications, the connection to semidefinite programming and sum-of-squares approaches, as well as algorithmic solution approaches for copositive programs.
255 citations
Book•
08 Dec 2010
TL;DR: Optimal Quadratic Programming Algorithms presents recently developed algorithms for solving large QP problems that are, in a sense optimal, i.e., they can solve important classes of problems at a cost proportional to the number of unknowns.
Abstract: Solving optimization problems in complex systems often requires the implementation of advanced mathematical techniques. Quadratic programming (QP) is one technique that allows for the optimization of a quadratic function in several variables in the presence of linear constraints. QP problems arise in fields as diverse as electrical engineering, agricultural planning, and optics. Given its broad applicability, a comprehensive understanding of quadratic programming is a valuable resource in nearly every scientific field. Optimal Quadratic Programming Algorithms presents recently developed algorithms for solving large QP problems. The presentation focuses on algorithms which are, in a sense optimal, i.e., they can solve important classes of problems at a cost proportional to the number of unknowns. For each algorithm presented, the book details its classical predecessor, describes its drawbacks, introduces modifications that improve its performance, and demonstrates these improvements through numerical experiments. This self-contained monograph can serve as an introductory text on quadratic programming for graduate students and researchers. Additionally, since the solution of many nonlinear problems can be reduced to the solution of a sequence of QP problems, it can also be used as a convenient introduction to nonlinear programming. The reader is required to have a basic knowledge of calculus in several variables and linear algebra.
182 citations
TL;DR: Two novel approaches for robust principal component analysis based on semidefinite programming are proposed, the first of which seeks directions of large spread in the data while damping the effect of outliers, and the second produces a low-leverage decomposition of the data that attempts to form a high-rank model for the data by separating out corrupted observations.
Abstract: The performance of principal component analysis (PCA) suffers badly in the presence of outliers This paper proposes two novel approaches for robust PCA based on semidefinite programming The first method, maximum mean absolute deviation rounding (MDR), seeks directions of large spread in the data while damping the effect of outliers The second method produces a low-leverage decomposition (LLD) of the data that attempts to form a low-rank model for the data by separating out corrupted observations This paper also presents efficient computational methods for solving these SDPs Numerical experiments confirm the value of these new techniques
142 citations
Book•
12 Nov 2010TL;DR: The semidefinite programming problem is the problem of optimizing a linear function of matrix variables, subject to finitely many linear inequalities and the positive semidefiiniteness condition on some of the matrix variables.
Abstract: Since the early 1960s, polyhedral methods have played a central role in both the theory and practice of combinatorial optimization. Since the early 1990s, a new technique, semidefinite programming, has been increasingly applied to some combinatorial optimization problems. The semidefinite programming problem is the problem of optimizing a linear function of matrix variables, subject to finitely many linear inequalities and the positive semidefiniteness condition on some of the matrix variables. On certain problems, such as maximum cut, maximum satisfiability, maximum stable set and geometric representations of graphs, semidefinite programming techniques yield important new results. This monograph provides the necessary background to work with semidefinite optimization techniques, usually by drawing parallels to the development of polyhedral techniques and with a special focus on combinatorial optimization, graph theory and lift-and-project methods. It allows the reader to rigorously develop the necessary knowledge, tools and skills to work in the area that is at the intersection of combinatorial optimization and semidefinite optimization. A solid background in mathematics at the undergraduate level and some exposure to linear optimization are required. Some familiarity with computational complexity theory and the analysis of algorithms would be helpful. Readers with these prerequisites will appreciate the important open problems and exciting new directions as well as new connections to other areas in mathematical sciences that the book provides. Titles in this series are co-published with The Fields Institute for Research in Mathematical Sciences (Toronto, Ontario, Canada).|Since the early 1960s, polyhedral methods have played a central role in both the theory and practice of combinatorial optimization. Since the early 1990s, a new technique, semidefinite programming, has been increasingly applied to some combinatorial optimization problems. The semidefinite programming problem is the problem of optimizing a linear function of matrix variables, subject to finitely many linear inequalities and the positive semidefiniteness condition on some of the matrix variables. On certain problems, such as maximum cut, maximum satisfiability, maximum stable set and geometric representations of graphs, semidefinite programming techniques yield important new results. This monograph provides the necessary background to work with semidefinite optimization techniques, usually by drawing parallels to the development of polyhedral techniques and with a special focus on combinatorial optimization, graph theory and lift-and-project methods. It allows the reader to rigorously develop the necessary knowledge, tools and skills to work in the area that is at the intersection of combinatorial optimization and semidefinite optimization. A solid background in mathematics at the undergraduate level and some exposure to linear optimization are required. Some familiarity with computational complexity theory and the analysis of algorithms would be helpful. Readers with these prerequisites will appreciate the important open problems and exciting new directions as well as new connections to other areas in mathematical sciences that the book provides. Titles in this series are co-published with The Fields Institute for Research in Mathematical Sciences (Toronto, Ontario, Canada).
80 citations
TL;DR: A practically efficient decomposition technique, which approximately solves the DNPs while simultaneously producing lower bounds on the original NQP, and the resulting algorithm for globally solving BoxQPs is the most efficient to date.
Abstract: It has recently been shown (Burer, Math Program 120:479–495, 2009) that a large class of NP-hard nonconvex quadratic programs (NQPs) can be modeled as so-called completely positive programs, i.e., the minimization of a linear function over the convex cone of completely positive matrices subject to linear constraints. Such convex programs are NP-hard in general. A basic tractable relaxation is gotten by approximating the completely positive matrices with doubly nonnegative matrices, i.e., matrices which are both nonnegative and positive semidefinite, resulting in a doubly nonnegative program (DNP). Optimizing a DNP, while polynomial, is expensive in practice for interior-point methods. In this paper, we propose a practically efficient decomposition technique, which approximately solves the DNPs while simultaneously producing lower bounds on the original NQP. We illustrate the effectiveness of our approach for solving the basic relaxation of box-constrained NQPs (BoxQPs) and the quadratic assignment problem. For one quadratic assignment instance, a best-known lower bound is obtained. We also incorporate the lower bounds within a branch-and-bound scheme for solving BoxQPs and the quadratic multiple knapsack problem. In particular, to the best of our knowledge, the resulting algorithm for globally solving BoxQPs is the most efficient to date.
79 citations
TL;DR: It is shown that each $\alpha$-factor approximate solution of the relaxed quadratic SDP can be used to generate in randomized polynomial time an $O(\alpha)$-Factor approximate solution for the original quartic optimization problem, where the constant in $O(cdot)$ depends only on problem dimension.
Abstract: We present a general semidefinite relaxation scheme for general $n$-variate quartic polynomial optimization under homogeneous quadratic constraints. Unlike the existing sum-of-squares approach which relaxes the quartic optimization problems to a sequence of (typically large) linear semidefinite programs (SDP), our relaxation scheme leads to a (possibly nonconvex) quadratic optimization problem with linear constraints over the semidefinite matrix cone in $\mathbb{R}^{n\times n}$. It is shown that each $\alpha$-factor approximate solution of the relaxed quadratic SDP can be used to generate in randomized polynomial time an $O(\alpha)$-factor approximate solution for the original quartic optimization problem, where the constant in $O(\cdot)$ depends only on problem dimension. In the case where only one positive definite quadratic constraint is present in the quartic optimization problem, we present a randomized polynomial time approximation algorithm which can provide a guaranteed relative approximation ratio of $(1-O(n^{-2}))$.
74 citations
TL;DR: This paper proposes a method to design stabilizing state feedback control laws for nonlinear quadratic systems subject to input saturation using an LMI-based optimization problem for computing the state feedback gains maximizing the estimate of the stability region of the closed-loop system.
69 citations
TL;DR: This work proves global convergence and provides numerical evidence to show the effectiveness of this modified alternating direction method for solving convex quadratically constrained quadratic semidefinite optimization problems.
64 citations
TL;DR: This correspondence considers the problem of robust waveform design in the presence of colored Gaussian disturbance under a similarity and an energy constraint, and proposes an algorithm with a polynomial computational complexity to generate a good sub-optimal solution for the QCQP.
Abstract: This correspondence considers the problem of robust waveform design in the presence of colored Gaussian disturbance under a similarity and an energy constraint. We resort to a max-min approach, where the worst case detection performance (over the possible Doppler shifts) is optimized with respect to the radar waveform under the previously mentioned constraints. The resulting optimization problem is a non-convex Quadratically Constrained Quadratic Program (QCQP) with an infinite number of constraints, which is NP-hard in general and typically difficult to solve. Hence, we propose an algorithm with a polynomial computational complexity to generate a good sub-optimal solution for the aforementioned QCQP. The analysis, conducted in comparison with some known radar waveforms, shows that the sub-optimal solutions by the algorithm lead to high-quality radar signals.
TL;DR: A new polyhedral approach to the quadratic linear ordering problem that is based on a linearization of the quadRatic objective function, which shows that this approach clearly outperforms other methods for bipartite crossing minimization.
Abstract: The quadratic linear ordering problem naturally generalizes various optimization problems such as bipartite crossing minimization or the betweenness problem, which includes linear arrangement. These problems have important applications, e.g., in automatic graph drawing and computational biology. We present a new polyhedral approach to the quadratic linear ordering problem that is based on a linearization of the quadratic objective function.
Our main result is a reformulation of the 3-dicycle inequalities using quadratic terms. After linearization, the resulting constraints are shown to be face-inducing for the polytope corresponding to the unconstrained quadratic problem. We use this result both within a branch-and-cut algorithm and within a branch-and-bound algorithm based on semidefinite programming. Experimental results for bipartite crossing minimization show that this approach clearly outperforms other methods.
TL;DR: This paper studies inverse optimization for linearly constrained convex separable programming problems that have wide applications in industrial and managerial areas and finds the parameter values that have the smallest adjustments.
TL;DR: In this paper, the authors considered the problem of stochastic finite-and infinite-horizon linear quadratic control under power constraints and showed that the optimal control law can be computed off-line as in the classical LQG Gaussian control theory using dynamic programming.
Abstract: We consider the problem of stochastic finite- and infinite-horizon linear quadratic control under power constraints. The calculations of the optimal control law can be done off-line as in the classical linear quadratic Gaussian control theory using dynamic programming, which turns out to be a special case of the new theory developed in this technical note. A numerical example is solved using the new methods.
TL;DR: This paper considers the downlink beamforming optimization problem that minimizes the total transmission power subject to global shaping constraints and individual shaping constraints, in addition to the constraints of quality of service measured by signal-to-interference-plus-noise ratio (SINR).
Abstract: This paper considers the downlink beamforming optimization problem that minimizes the total transmission power subject to global shaping constraints and individual shaping constraints, in addition to the constraints of quality of service (QoS) measured by signal-to-interference-plus-noise ratio (SINR). This beamforming problem is a separable homogeneous quadratically constrained quadratic program (QCQP), which is difficult to solve in general. Herein we propose efficient algorithms for the problem consisting of two main steps: 1) solving the semidefinite programming (SDP) relaxed problem, and 2) formulating a linear program (LP) and solving the LP (with closed-form solution) to find a rank-one optimal solution of the SDP relaxation. Accordingly, the corresponding optimal beamforming problem (OBP) is proven to be “hidden” convex, namely, strong duality holds true under certain mild conditions. In contrast to the existing algorithms based on either the rank reduction steps (the purification process) or the Perron-Frobenius theorem, the proposed algorithms are based on the linear program strong duality theorem.
TL;DR: This paper considers constraint propagation methods for continuous constraint satisfaction problems consisting of linear and quadratic constraints, and the basic new techniques consist in eliminating bilinear entries from a quadRatic constraint, and solving the resulting separable quadratics constraints by means of a sequence of univariate quadrotic problems.
Abstract: This paper considers constraint propagation methods for continuous constraint satisfaction problems consisting of linear and quadratic constraints. All methods can be applied after suitable preprocessing to arbitrary algebraic constraints. The basic new techniques consist in eliminating bilinear entries from a quadratic constraint, and solving the resulting separable quadratic constraints by means of a sequence of univariate quadratic problems. Care is taken to ensure that all methods correctly account for rounding errors in the computations. Various tests and examples illustrate the advantage of the presented method.
01 Jan 2010
TL;DR: This monograph traces the evolution of the SQP method through some important special cases of nonlinear programming, up to the most general form of problem.
Abstract: Sequential (or Successive) Quadratic Programming (SQP) is a technique for the solution of Nonlinear Programming (NLP)problems. It is, as we shall see, an idealized concept, permitting and indeed necessitating many variations and modifications before becoming available as part of a reliable andefficient production computer code. In this monograph we trace the evolution of the SQP method through some important special cases of nonlinear programming, up to the most general form of problem. To fully understandthese developments it is important to have a thorough grasp of the underlying theoretical concepts, particularly in regard to optimality conditions. In this monograph we include a simple yet rigorous presentation of optimality conditions, which yet covers most cases of interest.
TL;DR: The so-called symmetric mappings that can be used to derive strong cuts for the proposed relaxation model of QAPs are introduced and it is shown that the bounds based on the new models are comparable to some strong bounds in the literature.
Abstract: Quadratic assignment problems (QAPs) are known to be among the hardest discrete optimization problems. Recent study shows that even obtaining a strong lower bound for QAPs is a computational challenge. In this paper, we first discuss how to construct new simple convex relaxations of QAPs based on various matrix splitting schemes. Then we introduce the so-called symmetric mappings that can be used to derive strong cuts for the proposed relaxation model. We show that the bounds based on the new models are comparable to some strong bounds in the literature. Promising experimental results based on the new relaxations are reported.
TL;DR: A convex relaxation method that globally solves for the camera position and orientation given a set of image pixel measurements associated with a scene of reference points of known three-dimensional positions is presented.
TL;DR: It is proved strictly an important theoretical result that, for an arbitrary initial point, the trajectory of the proposed network does converge to the equilibrium, which corresponds to the optimal solution of a convex quadratic bilevel programming problem.
TL;DR: This analysis shows that SDR can provide in polynomial time a provably near-optimal solution, achieving a constant factor approximation of the optimal objective value in probability, and this constant factor remains bounded with increasing problem size.
Abstract: We consider semidefinite programming relaxation (SDR) of a binary quadratic minimization problem. This NP-hard problem arises naturally in the maximum-likelihood detection of discrete signals for digital communications. We analyze the average performance of the SDR algorithm for a class of randomly generated binary quadratic minimization problems. Although the SDR worst-case approximation ratio is unbounded for this NP-hard problem, our analysis shows that SDR can provide in polynomial time a provably near-optimal solution, achieving a constant factor approximation of the optimal objective value in probability. Moreover, this constant factor remains bounded with increasing problem size. Our proof is based on an asymptotic analysis of Karush-Kuhn-Tucker optimality conditions using random matrix theory.
TL;DR: A perturbed version of the classical Josephy–Newton method for solving generalized equations is proposed and analyzed, which achieves superlinear convergence under the second-order sufficient optimality condition and the strict Mangasarian–Fromovitz constraint qualification.
Abstract: We propose and analyze a perturbed version of the classical Josephy–Newton method for solving generalized equations. This perturbed framework is convenient to treat in a unified way standard sequential quadratic programming, its stabilized version, sequential quadratically constrained quadratic programming, and linearly constrained Lagrangian methods. For the linearly constrained Lagrangian methods, in particular, we obtain superlinear convergence under the second-order sufficient optimality condition and the strict Mangasarian–Fromovitz constraint qualification, while previous results in the literature assume (in addition to second-order sufficiency) the stronger linear independence constraint qualification as well as the strict complementarity condition. For the sequential quadratically constrained quadratic programming methods, we prove primal-dual superlinear/quadratic convergence under the same assumptions as above, which also gives a new result.
TL;DR: This method extends to this setting the filter SQP (sequential quadratic programming) algorithm, recently introduced for solving nonlinear programming problems, obtaining the respective global convergence results.
Abstract: This paper proposes a filter method for solving nonlinear semidefinite programming problems. Our method extends to this setting the filter SQP (sequential quadratic programming) algorithm, recently introduced for solving nonlinear programming problems, obtaining the respective global convergence results. Mathematical subject classification: 90C30, 90C55.
01 Sep 2010
TL;DR: This work proposes a consistent framework and accompanying theoretical justification for regularizing linear systems used to derive search directions, and can be interpreted as a simultaneous proximal-point regularization of the primal and dual problems.
Abstract: Interior-point methods in augmented form for linear and convex quadratic programming require the solution of a sequence of symmetric indefinite linear systems which are used to derive search directions. Safeguards are typically required in order to handle free variables or rank-deficient Jacobians. We propose a consistent framework and accompanying theoretical justification for regularizing these linear systems. Our approach can be interpreted as a simultaneous proximal-point regularization of the primal and dual problems. The regularization is termedexact to emphasize that, although the problems are regularized, the algorithm recovers a solution of the original problem, for appropriate values of the regularization parameters.
09 Jun 2010
TL;DR: A branch-and-bound algorithm for minimizing a convex quadratic objective function over integer variables subject to convex constraints by exploiting the integrality of the variables using suitably-defined lattice-free ellipsoids.
Abstract: We present a branch-and-bound algorithm for minimizing a convex quadratic objective function over integer variables subject to convex constraints. In a given node of the enumeration tree, corresponding to the fixing of a subset of the variables, a lower bound is given by the continuous minimum of the restricted objective function. We improve this bound by exploiting the integrality of the variables using suitably-defined lattice-free ellipsoids. Experiments show that our approach is very fast on both unconstrained problems and problems with box constraints. The main reason is that all expensive calculations can be done in a preprocessing phase, while a single node in the enumeration tree can be processed in linear time in the problem dimension.
TL;DR: The goal is the design of general separation routines that can replace detailed polyhedral studies of the resulting polytope and that can be used as a black box in the solution of constrained quadratic 0–1 problems.
Abstract: In many practical applications, the task is to optimize a non-linear objective function over the vertices of a well-studied polytope as, e.g., the matching polytope or the travelling salesman polytope (TSP). Prominent examples are the quadratic assignment problem and the quadratic knapsack problem; further applications occur in various areas such as production planning or automatic graph drawing. In order to apply branch-and-cut methods for the exact solution of such problems, the objective function has to be linearized. However, the standard linearization usually leads to very weak relaxations. On the other hand, problem-specific polyhedral studies are often time-consuming. Our goal is the design of general separation routines that can replace detailed polyhedral studies of the resulting polytope and that can be used as a black box. As unconstrained binary quadratic optimization is equivalent to the maximum-cut problem, knowledge about cut polytopes can be used in our setting. Other separation routines are inspired by the local cuts that have been developed by Applegate, Bixby, Chvatal and Cook for faster solution of large-scale traveling salesman instances. Finally, we apply quadratic reformulations of the linear constraints as proposed by Helmberg, Rendl and Weismantel for the quadratic knapsack problem. By extensive experiments, we show that a suitable combination of these methods leads to a drastic speedup in the solution of constrained quadratic 0–1 problems. We also discuss possible generalizations of these methods to arbitrary non-linear objective functions.
TL;DR: It is proved that solutions of the quadratically constrained least-squares estimator with a semidefinite constraint matrix are both unbiased and consistent in the sense that the averaged problem and limiting problem produce, respectively, unbiased and true (with probability 1) estimators.
Abstract: In this article, we investigate the consistency of parameter estimates obtained from least-squares identification with a quadratic parameter constraint. For generality, we consider infinite impulse-response systems with coloured input and output noise. In the case of finite data, we show that there always exists a possibly indefinite quadratic constraint depending on the noise realisation that results in a constrained optimisation problem that yields the true parameters of the system when a persistency condition is satisfied. When the noise covariance matrix is known to within a scalar multiple, we prove that solutions of the quadratically constrained least-squares (QCLs) estimator with a semidefinite constraint matrix are both unbiased and consistent in the sense that the averaged problem and limiting problem produce, respectively, unbiased and true (with probability 1) estimators. In addition, we provide numerical results that illustrate these properties of the QCLS estimator.
TL;DR: This work proves an exact cone programming reformulation for establishing rigorous yet affordable bounds and finding improving directions in multi-StQPs.
Abstract: A Standard Quadratic Optimization Problem (StQP) consists of maximizing a (possibly indefinite) quadratic form over the standard simplex. Likewise, in a multi-StQP we have to maximize a (possibly indefinite) quadratic form over the Cartesian product of several standard simplices (of possibly different dimensions). Among many other applications, multi-StQPs occur in Machine Learning Problems. Several converging monotone interior point methods are established, which differ from the usual ones used in cone programming. Further, we prove an exact cone programming reformulation for establishing rigorous yet affordable bounds and finding improving directions.
TL;DR: A class of volumetric barrier decomposition algorithms for SSDPs are derived and polynomial complexity of certain members of the class is proved to prove.
Abstract: Ariyawansa and Zhu have recently proposed a new class of optimization problems termed stochastic semidefinite programs (SSDPs). SSDPs may be viewed as an extension of two-stage stochastic (linear) programs with recourse (SLPs). Zhao has derived a decomposition algorithm for SLPs based on a logarithmic barrier and proved its polynomial complexity. Mehrotra and Özevin have extended the work of Zhao to the case of SSDPs to derive a polynomial logarithmic barrier decomposition algorithm for SSDPs. An alternative to the logarithmic barrier is the volumetric barrier of Vaidya. There is no work based on the volumetric barrier analogous to that of Zhao for SLPs or to the work of Mehrotra and Özevin for SSDPs. The purpose of this paper is to derive a class of volumetric barrier decomposition algorithms for SSDPs, and to prove polynomial complexity of certain members of the class.
TL;DR: A sequential quadratically constrained quadratic programming (SQCQP) norm-relaxed algorithm of strongly sub-feasible directions for the solution of inequality constrained optimization problems and some preliminary numerical results show that the proposed algorithm is also encouraging.