Graph Implementations for Nonsmooth Convex Programs
Summary (2 min read)
Introduction
- In contrast, there are many solvers available for smooth convex programs, as well as for certain standard forms such as semidefinite programs (SDPs), that are efficient in both theory and practice; e.g., [13, 16, 17, 18].
- The versatility of the approach, of course, depends upon the variety of transformations known by the modeler.
- A graph implementation encapsulates a method for transforming instances of a specific function in a constraint or objective into a form compatible with the underlying solver's standard form.
- The conditions imposed by the DCP ruleset ensure that these transformations always preserve equivalence and convexity.
2 A B r i e f I n t r o d u c t i o n t o cvx
- The indentat ion is purely for stylistic reasons and is optional.
- The code is relatively self-explanatory, but a few notes are in order: 9 9 Variables must be declared before their first use.
- (6) This function implicitly constrains P to be symmetric and positive definite.
- This is, of course, a considerably abbreviated introduction to cvx, intended only to give the reader an idea of the basic syntax and structure of cvx models.
3 Disciplined Convex Programming
- Disciplined convex programming was first named and described by Grant, Boyd, and Ye in [9] and Grant in [10].
- Such modelers do not simply construct arbitrary nonlinear programs and attempt to verify convexity after the fact; rather, they begin with a mental library of functions and sets with known geometries, and combine them in ways which convex analysis guarantees will preserve convexity.
- It consists of two key components: 9 an atom library--a collection of functions or sets with known properties of curvature (convexity and concavity) and monotonicity; and 9 the DCP ruleset--a finite enumeration of ways in which atoms may be combined in objectives and constraints while preserving convexity.
- The rules are drawn from basic principles of convex analysis, and are easy to learn, once you have had an exposure to convex analysis and convex optimization.
- The authors will provide examples of such violations and their resolution later in this section.
3.1 Preliminaries
- The rules of disciplined convex programming depend primarily upon the curvature of numeric expressions.
- Complex constant and affine expressions are considered as well, but of course convex and concave expressions are real by definition.
- Three categories of monotonicity are considered: nondecreasing, nonincreasing, and nonmonotonic.
- Secondly, for functions with multiple arguments, curvature is considered jointly, while montonicity is considered separately for each argument.
- With terminology defined, the authors now proceed to the ruleset itself.
3 .4 Compositions
- The four compos i t ion rules are: 9 (In each case, the authors assume that the argument is of compatible size.).
- In fact, nearly every rule in the DCP ruleset can be derived from these composition rules.
- As the authors stated in the introduction to this section, the DCP rules are sufficient but not necessary conditions for the convexity (or concavity) of an expression, constraint, or objective.
- Obviously, both norm and the longer, non-compliant version are equivalent numerically, but the latter form enables cvx to complete the verification and conversion process.
- Of course, because the library is finite, there will inevitably be instances where a simple substitution is not possible.
4 Graph Implementations
- Any modeling framework for optimization must provide a computational description of the functions its supports to the underlying solver.
- If f_abs is called with a numeric value of x, then the cvx specification it contains will construct a linear program with a single variable and two inequalities.
- What cvx does in this circumstance is to incorporate the specification into the surrounding model itself, in a manner not unlike the expansion of a inline function in C++.
- Thus a graph implementation can solve the nondifferentiability problem.
- Key omissions include logarithms, exponentials, and entropy; such functions simply cannot be exactly represented in an SQLP solver.
5 Final W o r d s
- The authors believe that disciplined convex programming closes a significant gap between the theory and practice of convex optimization.
- A modeling framework that supports disciplined convex programming provides a truly attractive alternative in most of these cases.
Did you find this useful? Give us your feedback
Citations
1,215 citations
Cites methods from "Graph Implementations for Nonsmooth..."
...The conversion is done using graph implementations of convex functions (Grant and Boyd, 2008)....
[...]
1,157 citations
1,067 citations
Cites methods from "Graph Implementations for Nonsmooth..."
...12), using the software package cvx [66, 67] to compute Âr....
[...]
958 citations
Cites background from "Graph Implementations for Nonsmooth..."
...The corresponding algorithmic routines are based on disciplined convex programming [13]....
[...]
References
[...]
33,341 citations
7,676 citations
"Graph Implementations for Nonsmooth..." refers background or methods in this paper
...Disciplined convex programming was first named and described by Grant, Boyd, and Ye in [GBY06] and Grant in [ Gra04 ]....
[...]
...We call such a definition a graph implementation, a term first coined in [ Gra04 ] and inspired by the properties of epigraphs and hypographs of convex and concave functions, respectively....
[...]
7,655 citations
"Graph Implementations for Nonsmooth..." refers background in this paper
...For more information about these problems, consult [VB96, LVBL98], or the documentation on the solvers themselves [ Stu99 , TTT99]....
[...]
...In contrast, there are many solvers available for smooth convex programs, as well as for certain standard forms such as semidefinite programs (SDPs), that are efficient in both theory and practice;e.g., [Van99, MOS07, TTT99, Stu99 ]....
[...]
5,679 citations
Related Papers (5)
Frequently Asked Questions (9)
Q2. What contributions have the authors mentioned in the paper "Graph implementations for nonsmooth convex programs" ?
The transformation approach dates back to the very first days of linear programing and is usually taught as a collection of tricks that a modeler can use to reformulate problems by hand this paper.
Q3. What is the definition of convex programming?
It consists of two key components:9 an atom library--a collection of functions or sets with known properties of curvature (convexity and concavity) and monotonicity; and 9 the DCP ruleset--a finite enumeration of ways in which atoms may be combined in objectives and constraints while preserving convexity.
Q4. What are the key omissions in the graph?
Key omissions include logarithms, exponentials, and entropy; such functions simply cannot be exactly represented in an SQLP solver.
Q5. What is the function de t_ inv?
The function de t_ inv represents faet_inv('), including the implicit constraint that its argument be symmetric and positive definite.
Q6. What is the exact transformation of the Huber penalty?
Note that the precise transformation depends on how square and abs are themselves implemented; multilevel transformations like this are quite typical.
Q7. What is the definition of a valid const ra in t?
A valid const ra in t is - a set membersh ip relat ion (E) in which the lef t-hand side (LHS) is affineand the r igh t -hand side (RHS) is a convex set.- an equali ty (=) with an affine LHS and an affine RHS.
Q8. what is the fundamental principle of convex analysis?
4.1 T h e BasicsRecall the definition of the epigraph of a function f : R n --~ (R U +oc):epi f _a { (x, y) e R n x a The authorf (x) _ y }. (11)A fundamental principle of convex analysis states that f is a convex function if and only if epi f is a convex set.
Q9. what is the cvx a t o m funct ?
So if x is a real variable, thensquare( x )is accepted by cvx; and, thanks to the above rule, so issquare( A 9 x + b )if A and b are cons tan t matr ices of compat ib le size.