scispace - formally typeset
Search or ask a question
Journal ArticleDOI

Some comments of Wolfe's `away step'

01 May 1986-Mathematical Programming (Springer-Verlag)-Vol. 35, Iss: 1, pp 110-119
TL;DR: It is given a detailed proof, under slightly weaker conditions on the objective function, that a modified Frank-Wolfe algorithm based on Wolfe's ‘away step’ strategy can achieve geometric convergence, provided a strict complementarity assumption holds.
Abstract: We give a detailed proof, under slightly weaker conditions on the objective function, that a modified Frank-Wolfe algorithm based on Wolfe's ‘away step’ strategy can achieve geometric convergence, provided a strict complementarity assumption holds.
Citations
More filters
Proceedings Article
16 Jun 2013
TL;DR: A new general framework for convex optimization over matrix factorizations, where every Frank-Wolfe iteration will consist of a low-rank update, is presented, and the broad application areas of this approach are discussed.
Abstract: We provide stronger and more general primal-dual convergence results for Frank-Wolfe-type algorithms (a.k.a. conditional gradient) for constrained convex optimization, enabled by a simple framework of duality gap certificates. Our analysis also holds if the linear subproblems are only solved approximately (as well as if the gradients are inexact), and is proven to be worst-case optimal in the sparsity of the obtained solutions. On the application side, this allows us to unify a large variety of existing sparse greedy methods, in particular for optimization over convex hulls of an atomic set, even if those sets can only be approximated, including sparse (or structured sparse) vectors or matrices, low-rank matrices, permutation matrices, or max-norm bounded matrices. We present a new general framework for convex optimization over matrix factorizations, where every Frank-Wolfe iteration will consist of a low-rank update, and discuss the broad application areas of this approach.

1,246 citations


Cites background or methods from "Some comments of Wolfe's `away step..."

  • ...Using away-steps, a faster linear convergence can be obtained for some special problem class (GuéLat & Marcotte, 1986)....

    [...]

  • ...Another important variant is the use of away-steps, as explained in (GuéLat & Marcotte, 1986), which we can unfortunately not discuss in detail here due to the lack of space....

    [...]

  • ...Here we will focus on one large class of structured norms, proposed by (Obozinski et al., 2011), which due to the atomic structure is particularly suitable to be used with the Frank-Wolfe algorithm....

    [...]

Journal ArticleDOI
01 Mar 1970

1,097 citations

Posted Content
TL;DR: Submodular functions are relevant to machine learning for at least two reasons: (1) some problems may be expressed directly as the optimization of submodular function and (2) the lovasz extension of sub-modular Functions provides a useful set of regularization functions for supervised and unsupervised learning as discussed by the authors.
Abstract: Submodular functions are relevant to machine learning for at least two reasons: (1) some problems may be expressed directly as the optimization of submodular functions and (2) the lovasz extension of submodular functions provides a useful set of regularization functions for supervised and unsupervised learning. In this monograph, we present the theory of submodular functions from a convex analysis perspective, presenting tight links between certain polyhedra, combinatorial optimization and convex optimization problems. In particular, we show how submodular function minimization is equivalent to solving a wide variety of convex optimization problems. This allows the derivation of new efficient algorithms for approximate and exact submodular function minimization with theoretical guarantees and good practical performance. By listing many examples of submodular functions, we review various applications to machine learning, such as clustering, experimental design, sensor placement, graphical model structure learning or subset selection, as well as a family of structured sparsity-inducing norms that can be derived and used from submodular functions.

336 citations

Book
21 Nov 2013
TL;DR: In Learning with Submodular Functions: A Convex Optimization Perspective, the theory of submodular functions is presented in a self-contained way from a convex analysis perspective, presenting tight links between certain polyhedra, combinatorial optimization and convex optimization problems.
Abstract: Submodular functions are relevant to machine learning for at least two reasons: (1) some problems may be expressed directly as the optimization of submodular functions and (2) the Lovsz extension of submodular functions provides a useful set of regularization functions for supervised and unsupervised learning. In Learning with Submodular Functions: A Convex Optimization Perspective, the theory of submodular functions is presented in a self-contained way from a convex analysis perspective, presenting tight links between certain polyhedra, combinatorial optimization and convex optimization problems. In particular, it describes how submodular function minimization is equivalent to solving a wide variety of convex optimization problems. This allows the derivation of new efficient algorithms for approximate and exact submodular function minimization with theoretical guarantees and good practical performance. By listing many examples of submodular functions, it reviews various applications to machine learning, such as clustering, experimental design, sensor placement, graphical model structure learning or subset selection, as well as a family of structured sparsity-inducing norms that can be derived and used from submodular functions. Learning with Submodular Functions: A Convex Optimization Perspective is an ideal reference for researchers, scientists, or engineers with an interest in applying submodular functions to machine learning problems.

325 citations

References
More filters
Journal ArticleDOI

3,154 citations


"Some comments of Wolfe's `away step..." refers methods in this paper

  • ...Throughout the paper we will denote by R the set of extreme points of S. In [ 2 ] Frank and Wolfe proposed the following algorithm for solving P:...

    [...]

Journal ArticleDOI

2,140 citations


"Some comments of Wolfe's `away step..." refers methods in this paper

  • ...Global convergence of the algorithm can be proved by showing that the algorithm map is closed (see Zangwill [6] or Luenberger [ 3 ]) or by bounding from below the decrease of the objective at each iteration (see next section)....

    [...]

Journal ArticleDOI
01 Mar 1970

1,097 citations

Journal ArticleDOI

1,051 citations