scispace - formally typeset
Search or ask a question

Showing papers on "Convex optimization published in 2012"


Journal ArticleDOI
TL;DR: In this paper, a convex programming problem is used to find the matrix with the minimum nuclear norm that is consistent with the observed entries in a low-rank matrix, which is then used to recover all the missing entries from most sufficiently large subsets.
Abstract: Suppose that one observes an incomplete subset of entries selected from a low-rank matrix. When is it possible to complete the matrix and recover the entries that have not been seen? We demonstrate that in very general settings, one can perfectly recover all of the missing entries from most sufficiently large subsets by solving a convex programming problem that finds the matrix with the minimum nuclear norm agreeing with the observed entries. The techniques used in this analysis draw upon parallels in the field of compressed sensing, demonstrating that objects other than signals and images can be perfectly reconstructed from very limited information.

2,327 citations


Journal ArticleDOI
TL;DR: Surprisingly enough, for certain classes of objective functions, the proposed methods for solving huge-scale optimization problems are better than the standard worst-case bounds for deterministic algorithms.
Abstract: In this paper we propose new methods for solving huge-scale optimization problems. For problems of this size, even the simplest full-dimensional vector operations are very expensive. Hence, we propose to apply an optimization technique based on random partial update of decision variables. For these methods, we prove the global estimates for the rate of convergence. Surprisingly, for certain classes of objective functions, our results are better than the standard worst-case bounds for deterministic algorithms. We present constrained and unconstrained versions of the method and its accelerated variant. Our numerical test confirms a high efficiency of this technique on problems of very big size.

1,454 citations


Journal ArticleDOI
TL;DR: This paper provides a general framework to convert notions of simplicity into convex penalty functions, resulting in convex optimization solutions to linear, underdetermined inverse problems.
Abstract: In applications throughout science and engineering one is often faced with the challenge of solving an ill-posed inverse problem, where the number of available measurements is smaller than the dimension of the model to be estimated. However in many practical situations of interest, models are constrained structurally so that they only have a few degrees of freedom relative to their ambient dimension. This paper provides a general framework to convert notions of simplicity into convex penalty functions, resulting in convex optimization solutions to linear, underdetermined inverse problems. The class of simple models considered includes those formed as the sum of a few atoms from some (possibly infinite) elementary atomic set; examples include well-studied cases from many technical fields such as sparse vectors (signal processing, statistics) and low-rank matrices (control, statistics), as well as several others including sums of a few permutation matrices (ranked elections, multiobject tracking), low-rank tensors (computer vision, neuroscience), orthogonal matrices (machine learning), and atomic measures (system identification). The convex programming formulation is based on minimizing the norm induced by the convex hull of the atomic set; this norm is referred to as the atomic norm. The facial structure of the atomic norm ball carries a number of favorable properties that are useful for recovering simple models, and an analysis of the underlying convex geometry provides sharp estimates of the number of generic measurements required for exact and robust recovery of models from partial information. These estimates are based on computing the Gaussian widths of tangent cones to the atomic norm ball. When the atomic set has algebraic structure the resulting optimization problems can be solved or approximated via semidefinite programming. The quality of these approximations affects the number of measurements required for recovery, and this tradeoff is characterized via some examples. Thus this work extends the catalog of simple models (beyond sparse vectors and low-rank matrices) that can be recovered from limited linear information via tractable convex programming.

1,431 citations


Journal ArticleDOI
TL;DR: In this article, a necessary and sufficient condition is provided to guarantee the existence of no duality gap for the optimal power flow problem, which is the dual of an equivalent form of the OPF problem.
Abstract: The optimal power flow (OPF) problem is nonconvex and generally hard to solve. In this paper, we propose a semidefinite programming (SDP) optimization, which is the dual of an equivalent form of the OPF problem. A global optimum solution to the OPF problem can be retrieved from a solution of this convex dual problem whenever the duality gap is zero. A necessary and sufficient condition is provided in this paper to guarantee the existence of no duality gap for the OPF problem. This condition is satisfied by the standard IEEE benchmark systems with 14, 30, 57, 118, and 300 buses as well as several randomly generated systems. Since this condition is hard to study, a sufficient zero-duality-gap condition is also derived. This sufficient condition holds for IEEE systems after small resistance (10-5 per unit) is added to every transformer that originally assumes zero resistance. We investigate this sufficient condition and justify that it holds widely in practice. The main underlying reason for the successful convexification of the OPF problem can be traced back to the modeling of transformers and transmission lines as well as the non-negativity of physical quantities such as resistance and inductance.

1,225 citations


Journal ArticleDOI
TL;DR: This work develops and analyze distributed algorithms based on dual subgradient averaging and provides sharp bounds on their convergence rates as a function of the network size and topology, and shows that the number of iterations required by the algorithm scales inversely in the spectral gap of thenetwork.
Abstract: The goal of decentralized optimization over a network is to optimize a global objective formed by a sum of local (possibly nonsmooth) convex functions using only local computation and communication. It arises in various application domains, including distributed tracking and localization, multi-agent coordination, estimation in sensor networks, and large-scale machine learning. We develop and analyze distributed algorithms based on dual subgradient averaging, and we provide sharp bounds on their convergence rates as a function of the network size and topology. Our analysis allows us to clearly separate the convergence of the optimization algorithm itself and the effects of communication dependent on the network structure. We show that the number of iterations required by our algorithm scales inversely in the spectral gap of the network, and confirm this prediction's sharpness both by theoretical lower bounds and simulations for various networks. Our approach includes the cases of deterministic optimization and communication, as well as problems with stochastic optimization and/or communication.

1,224 citations


Journal ArticleDOI
TL;DR: This paper reduces this extremely challenging optimization problem to a sequence of convex programs that minimize the sum of l1-norm and nuclear norm of the two component matrices, which can be efficiently solved by scalable convex optimization techniques.
Abstract: This paper studies the problem of simultaneously aligning a batch of linearly correlated images despite gross corruption (such as occlusion). Our method seeks an optimal set of image domain transformations such that the matrix of transformed images can be decomposed as the sum of a sparse matrix of errors and a low-rank matrix of recovered aligned images. We reduce this extremely challenging optimization problem to a sequence of convex programs that minimize the sum of l1-norm and nuclear norm of the two component matrices, which can be efficiently solved by scalable convex optimization techniques. We verify the efficacy of the proposed robust alignment algorithm with extensive experiments on both controlled and uncontrolled real data, demonstrating higher accuracy and efficiency than existing methods over a wide range of realistic misalignments and corruptions.

846 citations


Journal ArticleDOI
TL;DR: This paper describes how CVXGEN is implemented, and gives some results on the speed and reliability of the automatically generated solvers.
Abstract: CVXGEN is a software tool that takes a high level description of a convex optimization problem family, and automatically generates custom C code that compiles into a reliable, high speed solver for the problem family. The current implementation targets problem families that can be transformed, using disciplined convex programming techniques, to convex quadratic programs of modest size. CVXGEN generates simple, flat, library-free code suitable for embedding in real-time applications. The generated code is almost branch free, and so has highly predictable run-time behavior. The combination of regularization (both static and dynamic) and iterative refinement in the search direction computation yields reliable performance, even with poor quality data. In this paper we describe how CVXGEN is implemented, and give some results on the speed and reliability of the automatically generated solvers.

836 citations


Journal ArticleDOI
TL;DR: Two distributed primal-dual subgradient algorithms can be implemented over networks with dynamically changing topologies but satisfying a standard connectivity property, and allow the agents to asymptotically agree on optimal solutions and optimal values of the optimization problem under the Slater's condition.
Abstract: We consider a general multi-agent convex optimization problem where the agents are to collectively minimize a global objective function subject to a global inequality constraint, a global equality constraint, and a global constraint set. The objective function is defined by a sum of local objective functions, while the global constraint set is produced by the intersection of local constraint sets. In particular, we study two cases: one where the equality constraint is absent, and the other where the local constraint sets are identical. We devise two distributed primal-dual subgradient algorithms based on the characterization of the primal-dual optimal solutions as the saddle points of the Lagrangian and penalty functions. These algorithms can be implemented over networks with dynamically changing topologies but satisfying a standard connectivity property, and allow the agents to asymptotically agree on optimal solutions and optimal values of the optimization problem under the Slater's condition.

728 citations


Posted Content
TL;DR: This paper proposes to accelerate the computation of the l2, 1-norm regularized regression model by reformulating it as two equivalent smooth convex optimization problems which are then solved via the Nesterov's method---an optimal first-order black-box method for smooth conveX optimization.
Abstract: The problem of joint feature selection across a group of related tasks has applications in many areas including biomedical informatics and computer vision. We consider the l2,1-norm regularized regression model for joint feature selection from multiple tasks, which can be derived in the probabilistic framework by assuming a suitable prior from the exponential family. One appealing feature of the l2,1-norm regularization is that it encourages multiple predictors to share similar sparsity patterns. However, the resulting optimization problem is challenging to solve due to the non-smoothness of the l2,1-norm regularization. In this paper, we propose to accelerate the computation by reformulating it as two equivalent smooth convex optimization problems which are then solved via the Nesterov's method-an optimal first-order black-box method for smooth convex optimization. A key building block in solving the reformulations is the Euclidean projection. We show that the Euclidean projection for the first reformulation can be analytically computed, while the Euclidean projection for the second one can be computed in linear time. Empirical evaluations on several data sets verify the efficiency of the proposed algorithms.

630 citations


Journal ArticleDOI
TL;DR: To ensure scalability with respect to the number of nodes, robustness to isolated communication outages, and data privacy and integrity, the proposed SDP is solved in a distributed fashion by resorting to the alternating direction method of multipliers and guarantees faster convergence compared to competing alternatives.
Abstract: Optimal power flow (OPF) is considered for microgrids, with the objective of minimizing either the power distribution losses, or, the cost of power drawn from the substation and supplied by distributed generation (DG) units, while effecting voltage regulation. The microgrid is unbalanced, due to unequal loads in each phase and non-equilateral conductor spacings on the distribution lines. Similar to OPF formulations for balanced systems, the considered OPF problem is nonconvex. Nevertheless, a semidefinite programming (SDP) relaxation technique is advocated to obtain a convex problem solvable in polynomial-time complexity. Enticingly, numerical tests demonstrate the ability of the proposed method to attain the globally optimal solution of the original nonconvex OPF. To ensure scalability with respect to the number of nodes, robustness to isolated communication outages, and data privacy and integrity, the proposed SDP is solved in a distributed fashion by resorting to the alternating direction method of multipliers. The resulting algorithm entails iterative message-passing among groups of consumers and guarantees faster convergence compared to competing alternatives

602 citations


Journal Article
TL;DR: This work presents the distributed mini-batch algorithm, a method of converting many serial gradient-based online prediction algorithms into distributed algorithms that is asymptotically optimal for smooth convex loss functions and stochastic inputs and proves a regret bound for this method.
Abstract: Online prediction methods are typically presented as serial algorithms running on a single processor. However, in the age of web-scale prediction problems, it is increasingly common to encounter situations where a single processor cannot keep up with the high rate at which inputs arrive. In this work, we present the distributed mini-batch algorithm, a method of converting many serial gradient-based online prediction algorithms into distributed algorithms. We prove a regret bound for this method that is asymptotically optimal for smooth convex loss functions and stochastic inputs. Moreover, our analysis explicitly takes into account communication latencies between nodes in the distributed environment. We show how our method can be used to solve the closely-related distributed stochastic optimization problem, achieving an asymptotically linear speed-up over multiple processors. Finally, we demonstrate the merits of our approach on a web-scale online prediction problem.

Journal ArticleDOI
TL;DR: The accelerated stochastic approximation (AC-SA) algorithm based on Nesterov’s optimal method for smooth CP is introduced, and it is shown that the AC-SA algorithm can achieve the aforementioned lower bound on the rate of convergence for SCO.
Abstract: This paper considers an important class of convex programming (CP) problems, namely, the stochastic composite optimization (SCO), whose objective function is given by the summation of general nonsmooth and smooth stochastic components. Since SCO covers non-smooth, smooth and stochastic CP as certain special cases, a valid lower bound on the rate of convergence for solving these problems is known from the classic complexity theory of convex programming. Note however that the optimization algorithms that can achieve this lower bound had never been developed. In this paper, we show that the simple mirror-descent stochastic approximation method exhibits the best-known rate of convergence for solving these problems. Our major contribution is to introduce the accelerated stochastic approximation (AC-SA) algorithm based on Nesterov’s optimal method for smooth CP (Nesterov in Doklady AN SSSR 269:543–547, 1983; Nesterov in Math Program 103:127–152, 2005), and show that the AC-SA algorithm can achieve the aforementioned lower bound on the rate of convergence for SCO. To the best of our knowledge, it is also the first universally optimal algorithm in the literature for solving non-smooth, smooth and stochastic CP problems. We illustrate the significant advantages of the AC-SA algorithm over existing methods in the context of solving a special but broad class of stochastic programming problems.

Journal ArticleDOI
TL;DR: Simulation result shows that the energy scheduling of SAs and other appliances can be determined simultaneously using the proposed CP formulation, and its major advantage is that the overall DR optimization problem remains to be convex and therefore the solution can be found efficiently.
Abstract: Demand response (DR) is very important in the future smart grid, aiming to encourage consumers to reduce their demand during peak load hours. However, if binary decision variables are needed to specify start-up time of a particular appliance, the resulting mixed integer combinatorial problem is in general difficult to solve. In this paper, we study a versatile convex programming (CP) DR optimization framework for the automatic load management of various household appliances in a smart home. In particular, an L1 regularization technique is proposed to deal with schedule-based appliances (SAs), for which their on/off statuses are governed by binary decision variables. By relaxing these variables from integer to continuous values, the problem is reformulated as a new CP problem with an additional L1 regularization term in the objective. This allows us to transform the original mixed integer problem into a standard CP problem. Its major advantage is that the overall DR optimization problem remains to be convex and therefore the solution can be found efficiently. Moreover, a wide variety of appliances with different characteristics can be flexibly incorporated. Simulation result shows that the energy scheduling of SAs and other appliances can be determined simultaneously using the proposed CP formulation.

Journal ArticleDOI
TL;DR: In this paper, a mixed-integer conic programming formulation for the minimum loss distribution network reconfiguration problem is proposed, which employs a convex representation of the network model which is based on the conic quadratic format of the power flow equations.
Abstract: This paper proposes a mixed-integer conic programming formulation for the minimum loss distribution network reconfiguration problem. This formulation has two features: first, it employs a convex representation of the network model which is based on the conic quadratic format of the power flow equations and second, it optimizes the exact value of the network losses. The use of a convex model in terms of the continuous variables is particularly important because it ensures that an optimal solution obtained by a branch-and-cut algorithm for mixed-integer conic programming is global. In addition, good quality solutions with a relaxed optimality gap can be very efficiently obtained. A polyhedral approximation which is amenable to solution via more widely available mixed-integer linear programming software is also presented. Numerical results on practical test networks including distributed generation show that mixed-integer convex optimization is an effective tool for network reconfiguration.

Proceedings ArticleDOI
16 Jun 2012
TL;DR: The proposed framework and theoretical foundations are illustrated with examples in video summarization and image classification using representatives and can be extended to detect and reject outliers in datasets, and to efficiently deal with new observations and large datasets.
Abstract: We consider the problem of finding a few representatives for a dataset, i.e., a subset of data points that efficiently describes the entire dataset. We assume that each data point can be expressed as a linear combination of the representatives and formulate the problem of finding the representatives as a sparse multiple measurement vector problem. In our formulation, both the dictionary and the measurements are given by the data matrix, and the unknown sparse codes select the representatives via convex optimization. In general, we do not assume that the data are low-rank or distributed around cluster centers. When the data do come from a collection of low-rank models, we show that our method automatically selects a few representatives from each low-rank model. We also analyze the geometry of the representatives and discuss their relationship to the vertices of the convex hull of the data. We show that our framework can be extended to detect and reject outliers in datasets, and to efficiently deal with new observations and large datasets. The proposed framework and theoretical foundations are illustrated with examples in video summarization and image classification using representatives.

Journal ArticleDOI
TL;DR: A primal-dual splitting algorithm for solving monotone inclusions involving a mixture of sums, linear compositions, and parallel sums of set-valued and Lipschitzian operators was proposed in this paper.
Abstract: We propose a primal-dual splitting algorithm for solving monotone inclusions involving a mixture of sums, linear compositions, and parallel sums of set-valued and Lipschitzian operators. An important feature of the algorithm is that the Lipschitzian operators present in the formulation can be processed individually via explicit steps, while the set-valued operators are processed individually via their resolvents. In addition, the algorithm is highly parallel in that most of its steps can be executed simultaneously. This work brings together and notably extends various types of structured monotone inclusion problems and their solution methods. The application to convex minimization problems is given special attention.

Journal ArticleDOI
TL;DR: Sufficient conditions for the obtained filtering error system are proposed by applying an input-output approach and a two-term approximation method, which is employed to approximate the time-varying delay.
Abstract: In this paper, the problem of l2- l∞ filtering for a class of discrete-time Takagi-Sugeno (T-S) fuzzy time-varying delay systems is studied. Our attention is focused on the design of full- and reduced-order filters that guarantee the filtering error system to be asymptotically stable with a prescribed H∞ performance. Sufficient conditions for the obtained filtering error system are proposed by applying an input-output approach and a two-term approximation method, which is employed to approximate the time-varying delay. The corresponding full- and reduced-order filter design is cast into a convex optimization problem, which can be efficiently solved by standard numerical algorithms. Finally, simulation examples are provided to illustrate the effectiveness of the proposed approaches.

Journal ArticleDOI
TL;DR: This paper proposes to linearize the ALM and the ADM for some nuclear norm involved minimization problems such that closed-form solutions of these linearized subproblems can be easily derived.
Abstract: The nuclear norm is widely used to induce low-rank solutions for many optimization problems with matrix variables. Recently, it has been shown that the augmented Lagrangian method (ALM) and the alternating direction method (ADM) are very efficient for many convex programming problems arising from various applications, provided that the resulting subproblems are sufficiently simple to have closed-form solutions. In this paper, we are interested in the application of the ALM and the ADM for some nuclear norm involved minimization problems. When the resulting subproblems do not have closed-form solutions, we propose to linearize these subproblems such that closed-form solutions of these linearized subproblems can be easily derived. Global convergence of these linearized ALM and ADM are established under standard assumptions. Finally, we verify the effectiveness and efficiency of these new methods by some numerical experiments.

Proceedings ArticleDOI
16 Jun 2012
TL;DR: A novel paradigm to deal with depth reconstruction from 4D light fields in a variational framework is presented, taking into account the special structure of light field data, and reformulate the problem of stereo matching to a constrained labeling problem on epipolar plane images.
Abstract: We present a novel paradigm to deal with depth reconstruction from 4D light fields in a variational framework. Taking into account the special structure of light field data, we reformulate the problem of stereo matching to a constrained labeling problem on epipolar plane images, which can be thought of as vertical and horizontal 2D cuts through the field. This alternative formulation allows to estimate accurate depth values even for specular surfaces, while simultaneously taking into account global visibility constraints in order to obtain consistent depth maps for all views. The resulting optimization problems are solved with state-of-the-art convex relaxation techniques. We test our algorithm on a number of synthetic and real-world examples captured with a light field gantry and a plenoptic camera, and compare to ground truth where available. All data sets as well as source code are provided online for additional evaluation.

Journal ArticleDOI
TL;DR: This paper investigates the AC-SA algorithms for solving strongly convex stochastic composite optimization problems in more detail by establishing the large-deviation results associated with the convergence rates and introducing an efficient validation procedure to check the accuracy of the generated solutions.
Abstract: In this paper we present a generic algorithmic framework, namely, the accelerated stochastic approximation (AC-SA) algorithm, for solving strongly convex stochastic composite optimization (SCO) problems. While the classical stochastic approximation algorithms are asymptotically optimal for solving differentiable and strongly convex problems, the AC-SA algorithm, when employed with proper stepsize policies, can achieve optimal or nearly optimal rates of convergence for solving different classes of SCO problems during a given number of iterations. Moreover, we investigate these AC-SA algorithms in more detail, such as by establishing the large-deviation results associated with the convergence rates and introducing an efficient validation procedure to check the accuracy of the generated solutions.

Journal ArticleDOI
TL;DR: Numerical results demonstrate that the developed algorithms are able to locate the global optimal solutions by only a few iterations and they are superior to the previously proposed methods in both performance and computation complexity.
Abstract: Power allocations in an interference-limited wireless network for global maximization of the weighted sum throughput or global optimization of the minimum weighted rate among network links are not only important but also very hard optimization problems due to their nonconvexity nature. Recently developed methods are either unable to locate the global optimal solutions or prohibitively complex for practical applications. This paper exploits the d.c. (difference of two convex functions/sets) structure of either the objective function or constraints of these global optimization problems to develop efficient iterative algorithms with very low complexity. Numerical results demonstrate that the developed algorithms are able to locate the global optimal solutions by only a few iterations and they are superior to the previously-proposed methods in both performance and computation complexity.

Journal ArticleDOI
TL;DR: A mathematical framework called fractional programming is presented that provides insight into this class of optimization problems, as well as algorithms for computing the solution, that show that a broad class of EE maximization problems can be solved efficiently, provided the rate is a concave function of the transmit power.
Abstract: The dramatic increase of network infrastructure comes at the cost of rapidly increasing energy consumption, which makes optimization of energy efficiency (EE) an important topic. Since EE is often modeled as the ratio of rate to power, we present a mathematical framework called fractional programming that provides insight into this class of optimization problems, as well as algorithms for computing the solution. The main idea is that the objective function is transformed to a weighted sum of rate and power. A generic problem formulation for systems dissipating transmit-independent circuit power in addition to transmit-dependent power is presented. We show that a broad class of EE maximization problems can be solved efficiently, provided the rate is a concave function of the transmit power. We elaborate examples of various system models including time-varying parallel channels. Rate functions with an arbitrary discrete modulation scheme are also treated. The examples considered lead to water-filling solutions, but these are different from the dual problems of power minimization under rate constraints and rate maximization under power constraints, respectively, because the constraints need not be active. We also demonstrate that if the solution to a rate maximization problem is known, it can be utilized to reduce the EE problem into a one-dimensional convex problem.

Journal ArticleDOI
TL;DR: In this paper, the authors show that the straightforward extension of ADM is valid for the general case of $m\ge 3$ if it is combined with a Gaussian back substitution procedure and prove its convergence via the analytic framework of contractive-type methods.
Abstract: We consider the linearly constrained separable convex minimization problem whose objective function is separable into m individual convex functions with nonoverlapping variables. A Douglas–Rachford alternating direction method of multipliers (ADM) has been well studied in the literature for the special case of $m=2$. But the convergence of extending ADM to the general case of $m\ge 3$ is still open. In this paper, we show that the straightforward extension of ADM is valid for the general case of $m\ge 3$ if it is combined with a Gaussian back substitution procedure. The resulting ADM with Gaussian back substitution is a novel approach towards the extension of ADM from $m=2$ to $m\ge 3$, and its algorithmic framework is new in the literature. For the ADM with Gaussian back substitution, we prove its convergence via the analytic framework of contractive-type methods, and we show its numerical efficiency by some application problems.

Journal Article
TL;DR: A novel metric learning approach called DML-eig is introduced which is shown to be equivalent to a well-known eigen value optimization problem called minimizing the maximal eigenvalue of a symmetric matrix.
Abstract: The main theme of this paper is to develop a novel eigenvalue optimization framework for learning a Mahalanobis metric. Within this context, we introduce a novel metric learning approach called DML-eig which is shown to be equivalent to a well-known eigenvalue optimization problem called minimizing the maximal eigenvalue of a symmetric matrix (Overton, 1988; Lewis and Overton, 1996). Moreover, we formulate LMNN (Weinberger et al., 2005), one of the state-of-the-art metric learning methods, as a similar eigenvalue optimization problem. This novel framework not only provides new insights into metric learning but also opens new avenues to the design of efficient metric learning algorithms. Indeed, first-order algorithms are developed for DML-eig and LMNN which only need the computation of the largest eigenvector of a matrix per iteration. Their convergence characteristics are rigorously established. Various experiments on benchmark data sets show the competitive performance of our new approaches. In addition, we report an encouraging result on a difficult and challenging face verification data set called Labeled Faces in the Wild (LFW).

01 Jan 2012
TL;DR: In this article, the authors implemented a robust face recognition system via sparse representation and convex optimization, which treated each test sample as sparse linear combination of training samples, and got the sparse solution via L1-minimization.
Abstract: In this project, we implement a robust face recognition system via sparse representation and convex optimization We treat each test sample as sparse linear combination of training samples, and get the sparse solution via L1-minimization We also explore the group sparseness (L2-norm) as well as normal L1-norm regularizationWe discuss the role of feature extraction and classification robustness to occlusion or pixel corruption of face recognition system The experiments demonstrate the choice of features is no longer critical once the sparseness is properly harnessed We also verify that the proposed algorithm outperforms other methods

Journal ArticleDOI
TL;DR: In this article, the authors consider situations where they are not only interested in sparsity, but where some structural prior knowledge is available as well, and show that the $\ell_1$-norm can then be extended to structured norms built on either disjoint or overlapping groups of variables.
Abstract: Sparse estimation methods are aimed at using or obtaining parsimonious representations of data or models. While naturally cast as a combinatorial optimization problem, variable or feature selection admits a convex relaxation through the regularization by the $\ell_1$-norm. In this paper, we consider situations where we are not only interested in sparsity, but where some structural prior knowledge is available as well. We show that the $\ell_1$-norm can then be extended to structured norms built on either disjoint or overlapping groups of variables, leading to a flexible framework that can deal with various structures. We present applications to unsupervised learning, for structured sparse principal component analysis and hierarchical dictionary learning, and to supervised learning in the context of non-linear variable selection.

Journal ArticleDOI
TL;DR: The primal-dual optimization algorithm developed in Chambolle and Pock (CP) is applied to various convex optimization problems of interest in computed tomography (CT) image reconstruction and its potential for prototyping is demonstrated by explicitly deriving CP algorithm instances for many optimization problems relevant to CT.
Abstract: The primal–dual optimization algorithm developed in Chambolle and Pock (CP) (2011 J. Math. Imag. Vis. 40 1–26) is applied to various convex optimization problems of interest in computed tomography (CT) image reconstruction. This algorithm allows for rapid prototyping of optimization problems for the purpose of designing iterative image reconstruction algorithms for CT. The primal–dual algorithm is briefly summarized in this paper, and its potential for prototyping is demonstrated by explicitly deriving CP algorithm instances for many optimization problems relevant to CT. An example application modeling breast CT with low-intensity x-ray illumination is presented.

Proceedings ArticleDOI
01 Dec 2012
TL;DR: This work describes and proves convergence of a new algorithm called Push-Sum Distributed Dual Averaging which combines a recent optimization algorithm with a push-sum consensus protocol.
Abstract: Recently there has been a significant amount of research on developing consensus based algorithms for distributed optimization motivated by applications that vary from large scale machine learning to wireless sensor networks. This work describes and proves convergence of a new algorithm called Push-Sum Distributed Dual Averaging which combines a recent optimization algorithm [1] with a push-sum consensus protocol [2]. As we discuss, the use of push-sum has significant advantages. Restricting to doubly stochastic consensus protocols is not required and convergence to the true average consensus is guaranteed without knowing the stationary distribution of the update matrix in advance. Furthermore, the communication semantics of just summing the incoming information make this algorithm truly asynchronous and allow a clean analysis when varying intercommunication intervals and communication delays are modelled. We include experiments in simulation and on a small cluster to complement the theoretical analysis.

Proceedings ArticleDOI
24 Dec 2012
TL;DR: An algorithm that generates collision-free trajectories in three dimensions for multiple vehicles within seconds using sequential convex programming that approximates non-convex constraints by using convex ones is presented.
Abstract: This paper presents an algorithm that generates collision-free trajectories in three dimensions for multiple vehicles within seconds. The problem is cast as a non-convex optimization problem, which is iteratively solved using sequential convex programming that approximates non-convex constraints by using convex ones. The method generates trajectories that account for simple dynamics constraints and is thus independent of the vehicle's type. An extensive a posteriori vehicle-specific feasibility check is included in the algorithm. The algorithm is applied to a quadrocopter fleet. Experimental results are shown.

Journal ArticleDOI
TL;DR: This paper proposes an efficient approximation method for solving the nonconvex centralized problem, using semidefinite relaxation (SDR), an approximation technique based on convex optimization, and analytically shows the convergence of the proposed distributed robust MCBF algorithm to the optimal centralized solution.
Abstract: Multicell coordinated beamforming (MCBF), where multiple base stations (BSs) collaborate with each other in the beamforming design for mitigating the intercell interference (ICI), has been a subject drawing great attention recently. Most MCBF designs assume perfect channel state information (CSI) of mobile stations (MSs); however CSI errors are inevitable at the BSs in practice. Assuming elliptically bounded CSI errors, this paper studies the robust MCBF design problem that minimizes the weighted sum power of BSs subject to worst-case signal-to-interference-plus-noise ratio (SINR) constraints on the MSs. Our goal is to devise a distributed optimization method to obtain the worst-case robust beamforming solutions in a decentralized fashion with only local CSI used at each BS and limited backhaul information exchange between BSs. However, the considered problem is difficult to handle even in the centralized form. We first propose an efficient approximation method for solving the nonconvex centralized problem, using semidefinite relaxation (SDR), an approximation technique based on convex optimization. Then a distributed robust MCBF algorithm is further proposed, using a distributed convex optimization technique known as alternating direction method of multipliers (ADMM). We analytically show the convergence of the proposed distributed robust MCBF algorithm to the optimal centralized solution. We also extend the worst-case robust beamforming design as well as its decentralized implementation method to a fully coordinated scenario. Simulation results are presented to examine the effectiveness of the proposed SDR method and the distributed robust MCBF algorithm.