The time-frequency and time-scale communities have recently developed a large number of overcomplete waveform dictionaries --- stationary wavelets, wavelet packets, cosine packets, chirplets, and warplets, to name a few. Decomposition into overcomplete systems is not unique, and several methods for decomposition have been proposed, including the method of frames (MOF), Matching pursuit (MP), and, for special dictionaries, the best orthogonal basis (BOB).
Basis Pursuit (BP) is a principle for decomposing a signal into an "optimal" superposition of dictionary elements, where optimal means having the smallest l1 norm of coefficients among all such decompositions. We give examples exhibiting several advantages over MOF, MP, and BOB, including better sparsity and superresolution. BP has interesting relations to ideas in areas as diverse as ill-posed problems, in abstract harmonic analysis, total variation denoising, and multiscale edge denoising.
BP in highly overcomplete dictionaries leads to large-scale optimization problems. With signals of length 8192 and a wavelet packet dictionary, one gets an equivalent linear program of size 8192 by 212,992. Such problems can be attacked successfully only because of recent advances in linear programming by interior-point methods. We obtain reasonable success with a primal-dual logarithmic barrier method and conjugate-gradient solver.

Atomic Decomposition by Basis Pursuit

SeDuMi is an add-on for MATLAB, which lets you solve optimization problems with linear, quadratic and semidefiniteness constraints. It is possible to have complex valued data and variables in SeDuMi. Moreover, large scale optimization problems are solved efficiently, by exploiting sparsity. This paper describes how to work with this toolbox.

Using SeDuMi 1.02, a MATLAB toolbox for optimization over symmetric cones

/pdf/convex-analysisnoer-san-nojin-zhan-nituite-5dl2rbmoyr.pdf

Convex Analysisの二,三の進展について

The time-frequency and time-scale communities have recently developed a large number of overcomplete waveform dictionaries---stationary wavelets, wavelet packets, cosine packets, chirplets, and warplets, to name a few. Decomposition into overcomplete systems is not unique, and several methods for decomposition have been proposed, including the method of frames (MOF), matching pursuit (MP), and, for special dictionaries, the best orthogonal basis (BOB).
Basis pursuit (BP) is a principle for decomposing a signal into an "optimal"' superposition of dictionary elements, where optimal means having the smallest l1 norm of coefficients among all such decompositions. We give examples exhibiting several advantages over MOF, MP, and BOB, including better sparsity and superresolution. BP has interesting relations to ideas in areas as diverse as ill-posed problems, abstract harmonic analysis, total variation denoising, and multiscale edge denoising.
BP in highly overcomplete dictionaries leads to large-scale optimization problems. With signals of length 8192 and a wavelet packet dictionary, one gets an equivalent linear program of size 8192 by 212,992. Such problems can be attacked successfully only because of recent advances in linear and quadratic programming by interior-point methods. We obtain reasonable success with a primal-dual logarithmic barrier method and conjugate-gradient solver.

/pdf/atomic-decomposition-by-basis-pursuit-2z0e8wwyak.pdf

In semidefinite programming, one minimizes a linear function subject to the constraint that an affine combination of symmetric matrices is positive semidefinite. Such a constraint is nonlinear and nonsmooth, but convex, so semidefinite programs are convex optimization problems. Semidefinite programming unifies several standard problems (e.g., linear and quadratic programming) and finds many applications in engineering and combinatorial optimization. Although semidefinite programs are much more general than linear programs, they are not much harder to solve. Most interior-point methods for linear programming have been generalized to semidefinite programs. As in linear programming, these methods have polynomial worst-case complexity and perform very well in practice. This paper gives a survey of the theory and applications of semidefinite programs and an introduction to primaldual interior-point methods for their solution.

Semidefinite programming

Based on the work of the Nesterov and Todd on self-scaled cones an implementation of a primal-dual interior-point method for solving large-scale sparse conic quadratic optimization problems is presented. The main features of the implementation are it is based on a homogeneous and self-dual model, it handles rotated quadratic cones directly, it employs a Mehrotra type predictor-corrector extension and sparse linear algebra to improve the computational efficiency. Finally, the implementation exploits fixed variables which naturally occurs in many conic quadratic optimization problems. This is a novel feature for our implementation. Computational results are also presented to document that the implementation can solve very large problems robustly and efficiently.

/pdf/on-implementing-a-primal-dual-interior-point-method-for-338pml1pb6.pdf

On implementing a primal-dual interior-point method for conic quadratic optimization

Partial table of contents: INTRODUCTION: THEORY AND COMPLEXITY. Duality Theory for Linear Optimization. A Polynomial Algorithm for the Skew-Symmetric Model. Solving the Canonical Problem. THE LOGARITHMIC BARRIER APPROACH. The Dual Logarithmic Barrier Method. Initialization. THE TARGET-FOLLOWING APPROACH. The Primal-Dual Newton Method. Application to the Method of Centers. MISCELLANEOUS TOPICS. Karmarkar's Projective Method. More Properties of the Central Path. Partial Updating. High-Order Methods. Parametric and Sensitivity Analysis. Implementing Interior Point Methods. Appendices. Bibliography. Indexes.

Theory and algorithms for linear optimization : an interior point approach

In this survey we review the many faces of the S-lemma, a result about the correctness of the S-procedure. The basic idea of this widely used method came from control theory but it has important consequences in quadratic and semidefinite optimization, convex geometry, and linear algebra as well. These were all active research areas, but as there was little interaction between researchers in these different areas, their results remained mainly isolated. Here we give a unified analysis of the theory by providing three different proofs for the S-lemma and revealing hidden connections with various areas of mathematics. We prove some new duality results and present applications from control theory, error estimation, and computational geometry.

/pdf/a-survey-of-the-s-lemma-b72eycsitt.pdf

A Survey of the S-Lemma

Interior-point methods (IPMs) are among the most efficient methods for solving linear, and also wide classes of other convex optimization problems. Since the path-breaking work of Karmarkar [48], much research was invested in IPMs. Many algorithmic variants were developed for Linear Optimization (LO). The new approach forced to reconsider all aspects of optimization problems. Not only the research on algorithms and complexity issues, but implementation strategies, duality theory and research on sensitivity analysis got also a new impulse. After more than a decade of turbulent research, the IPM community reached a good understanding of the basics of IPMs. Several books were published that summarize and explore different aspects of IPMs. The seminal work of Nesterov and Nemirovski [63] provides the most general framework for polynomial IPMs for convex optimization. Den Hertog [42] gives a thorough survey of primal and dual path-following IPMs for linear and structured convex optimization problems. Jansen [45] discusses primal-dual target following algorithms for linear optimization and complementarity problems.Wright [93] also concentrates on primal-dual IPMs, with special attention on infeasible IPMs, numerical issues and local, asymptotic convergence properties. The volume [80] contains 13 survey papers that cover almost all aspects of IPMs, their extensions and some applications. The book of Ye [96] is a rich source of polynomial IPMs not only for LO, but for convex optimization problems as well. It extends the IPM theory to derive bounds and approximations for classes of nonconvex optimization problems as well. Finally, Roos, Terlaky and Vial [72] present a thorough treatment of the IPM based theory - duality, complexity, sensitivity analysis - and wide classes of IPMs for LO.

/pdf/interior-point-methods-for-nonlinear-optimization-14bn5kt1er.pdf

Interior Point Methods for Nonlinear Optimization

An apparatus and method for continuously detecting oxygen in a carrier gas stream. The apparatus employs a sensor film comprising a fluorescent material emitting from 3000 - 8000 A, dissolved in a carrier or solvent, the film being supported on a suitable substrate. Exciting radiation of about 2000 - 6500 A is applied to the film causing the fluorescent material to emit.

Tamás Terlaky

Papers

On implementing a primal-dual interior-point method for conic quadratic optimization

Theory and algorithms for linear optimization : an interior point approach

A Survey of the S-Lemma

Interior Point Methods for Nonlinear Optimization

Self-Regularity: A New Paradigm for Primal-Dual Interior-Point Algorithms