Convex optimization

About: Convex optimization is a research topic. Over the lifetime, 24906 publications have been published within this topic receiving 908795 citations. The topic is also known as: convex optimisation.

Optimal Stochastic Approximation Algorithms for Strongly Convex Stochastic Composite Optimization I: A Generic Algorithmic Framework

Abstract: In this paper we present a generic algorithmic framework, namely, the accelerated stochastic approximation (AC-SA) algorithm, for solving strongly convex stochastic composite optimization (SCO) problems. While the classical stochastic approximation algorithms are asymptotically optimal for solving differentiable and strongly convex problems, the AC-SA algorithm, when employed with proper stepsize policies, can achieve optimal or nearly optimal rates of convergence for solving different classes of SCO problems during a given number of iterations. Moreover, we investigate these AC-SA algorithms in more detail, such as by establishing the large-deviation results associated with the convergence rates and introducing an efficient validation procedure to check the accuracy of the generated solutions.

Robust Low-Rank Tensor Recovery: Models and Algorithms

Abstract: Robust tensor recovery plays an instrumental role in robustifying tensor decompositions for multilinear data analysis against outliers, gross corruptions, and missing values and has a diverse array of applications. In this paper, we study the problem of robust low-rank tensor recovery in a convex optimization framework, drawing upon recent advances in robust principal component analysis and tensor completion. We propose tailored optimization algorithms with global convergence guarantees for solving both the constrained and the Lagrangian formulations of the problem. These algorithms are based on the highly efficient alternating direction augmented Lagrangian and accelerated proximal gradient methods. We also propose a nonconvex model that can often improve the recovery results from the convex models. We investigate the empirical recoverability properties of the convex and nonconvex formulations and compare the computational performance of the algorithms on simulated data. We demonstrate through a number o...

LMI Techniques for Optimization Over Polynomials in Control: A Survey

Abstract: Numerous tasks in control systems involve optimization problems over polynomials, and unfortunately these problems are in general nonconvex. In order to cope with this difficulty, linear matrix inequality (LMI) techniques have been introduced because they allow one to obtain bounds to the sought solution by solving convex optimization problems and because the conservatism of these bounds can be decreased in general by suitably increasing the size of the problems. This survey aims to provide the reader with a significant overview of the LMI techniques that are used in control systems for tackling optimization problems over polynomials, describing approaches such as decomposition in sum of squares, Positivstellensatz, theory of moments, Polya's theorem, and matrix dilation. Moreover, it aims to provide a collection of the essential problems in control systems where these LMI techniques are used, such as stability and performance investigations in nonlinear systems, uncertain systems, time-delay systems, and genetic regulatory networks. It is expected that this survey may be a concise useful reference for all readers.

Accelerated proximal gradient methods for nonconvex programming

Abstract: Nonconvex and nonsmooth problems have recently received considerable attention in signal/image processing, statistics and machine learning. However, solving the nonconvex and nonsmooth optimization problems remains a big challenge. Accelerated proximal gradient (APG) is an excellent method for convex programming. However, it is still unknown whether the usual APG can ensure the convergence to a critical point in nonconvex programming. In this paper, we extend APG for general nonconvex and nonsmooth programs by introducing a monitor that satisfies the sufficient descent property. Accordingly, we propose a monotone APG and a nonmonotone APG. The latter waives the requirement on monotonic reduction of the objective function and needs less computation in each iteration. To the best of our knowledge, we are the first to provide APG-type algorithms for general nonconvex and nonsmooth problems ensuring that every accumulation point is a critical point, and the convergence rates remain O(1/k2) when the problems are convex, in which k is the number of iterations. Numerical results testify to the advantage of our algorithms in speed.

On relaxed LMI-based designs for fuzzy regulators and fuzzy observers

Abstract: Relaxed conditions for stability of nonlinear, continuous and discrete-time systems given by fuzzy models are presented. A theoretical analysis shows that the proposed methods provide better or at least the same results of the methods presented in the literature. Numerical results exemplify this fact. These results are also used for fuzzy regulators and observers designs. The nonlinear systems are represented by fuzzy models proposed by Takagi and Sugeno (1985). The stability analysis and the design of controllers are described by linear matrix inequalities, that can be solved efficiently using convex programming techniques. The specification of the decay rate, constrains on control input and output are also discussed.