Practical Methods of Optimization

Book•

Practical Methods of Optimization

01 Jan 2009-

TL;DR: The aim of this book is to provide a Discussion of Constrained Optimization and its Applications to Linear Programming and Other Optimization Problems.

read less

Abstract: Preface Table of Notation Part 1: Unconstrained Optimization Introduction Structure of Methods Newton-like Methods Conjugate Direction Methods Restricted Step Methods Sums of Squares and Nonlinear Equations Part 2: Constrained Optimization Introduction Linear Programming The Theory of Constrained Optimization Quadratic Programming General Linearly Constrained Optimization Nonlinear Programming Other Optimization Problems Non-Smooth Optimization References Subject Index.

...read moreread less

Citations

PDF

Open Access

More filters

Journal Article•DOI•

QUANTUM ESPRESSO: a modular and open-source software project for quantum simulations of materials

[...]

Paolo Giannozzi¹, Stefano Baroni², Stefano Baroni³, Nicola Bonini⁴, Matteo Calandra⁵, Roberto Car⁶, Carlo Cavazzoni, Davide Ceresoli⁴, Guido L. Chiarotti, Matteo Cococcioni⁷, Ismaila Dabo⁸, Andrea Dal Corso², Andrea Dal Corso³, Stefano de Gironcoli³, Stefano de Gironcoli², Stefano Fabris², Stefano Fabris³, Guido Fratesi⁹, Ralph Gebauer², Ralph Gebauer¹⁰, Uwe Gerstmann¹¹, Christos Gougoussis⁵, Anton Kokalj², Michele Lazzeri⁵, Layla Martin-Samos², Nicola Marzari⁴, Francesco Mauri⁵, Riccardo Mazzarello¹², Stefano Paolini³, Alfredo Pasquarello¹³, Lorenzo Paulatto², Lorenzo Paulatto³, Carlo Sbraccia², Sandro Scandolo¹⁰, Sandro Scandolo², Gabriele Sclauzero³, Gabriele Sclauzero², Ari P. Seitsonen⁵, Alexander Smogunov¹⁰, Paolo Umari², Renata M. Wentzcovitch - Show less +37 more•Institutions (13)

University of Udine¹, National Research Council², International School for Advanced Studies³, Massachusetts Institute of Technology⁴, University of Paris⁵, Princeton University⁶, University of Minnesota⁷, ParisTech⁸, University of Milan⁹, International Centre for Theoretical Physics¹⁰, University of Paderborn¹¹, ETH Zurich¹², École Polytechnique Fédérale de Lausanne¹³

30 Sep 2009-Journal of Physics: Condensed Matter

TL;DR: QUANTUM ESPRESSO as discussed by the authors is an integrated suite of computer codes for electronic-structure calculations and materials modeling, based on density functional theory, plane waves, and pseudopotentials (norm-conserving, ultrasoft, and projector-augmented wave).

...read moreread less

Abstract: QUANTUM ESPRESSO is an integrated suite of computer codes for electronic-structure calculations and materials modeling, based on density-functional theory, plane waves, and pseudopotentials (norm-conserving, ultrasoft, and projector-augmented wave). The acronym ESPRESSO stands for opEn Source Package for Research in Electronic Structure, Simulation, and Optimization. It is freely available to researchers around the world under the terms of the GNU General Public License. QUANTUM ESPRESSO builds upon newly-restructured electronic-structure codes that have been developed and tested by some of the original authors of novel electronic-structure algorithms and applied in the last twenty years by some of the leading materials modeling groups worldwide. Innovation and efficiency are still its main focus, with special attention paid to massively parallel architectures, and a great effort being devoted to user friendliness. QUANTUM ESPRESSO is evolving towards a distribution of independent and interoperable codes in the spirit of an open-source project, where researchers active in the field of electronic-structure calculations are encouraged to participate in the project by contributing their own codes or by implementing their own ideas into existing codes.

...read moreread less

19,985 citations

Journal Article•DOI•

A Tutorial on Support Vector Machines for Pattern Recognition

[...]

Christopher John Burges¹•Institutions (1)

Alcatel-Lucent¹

01 Jun 1998-Data Mining and Knowledge Discovery

TL;DR: There are several arguments which support the observed high accuracy of SVMs, which are reviewed and numerous examples and proofs of most of the key theorems are given.

...read moreread less

Abstract: The tutorial starts with an overview of the concepts of VC dimension and structural risk minimization. We then describe linear Support Vector Machines (SVMs) for separable and non-separable data, working through a non-trivial example in detail. We describe a mechanical analogy, and discuss when SVM solutions are unique and when they are global. We describe how support vector training can be practically implemented, and discuss in detail the kernel mapping technique which is used to construct SVM solutions which are nonlinear in the data. We show how Support Vector machines can have very large (even infinite) VC dimension by computing the VC dimension for homogeneous polynomial and Gaussian radial basis function kernels. While very high VC dimension would normally bode ill for generalization performance, and while at present there exists no theory which shows that good generalization performance is guaranteed for SVMs, there are several arguments which support the observed high accuracy of SVMs, which we review. Results of some experiments which were inspired by these arguments are also presented. We give numerous examples and proofs of most of the key theorems. There is new material, and I hope that the reader will find that even old material is cast in a fresh light.

...read moreread less

15,696 citations

Cites background from "Practical Methods of Optimization"

...This particular dual formulation of the problem is called the Wolfe dual (Fletcher, 1987)....
[...]
...This is a property of any convex programming problem (Fletcher, 1987)....
[...]
...For the primal problem above, the KKT conditions may be stated (Fletcher, 1987): ∂ ∂wν LP = wν − ∑ i αiyixiν = 0 ν = 1, · · · , d (17) ∂ ∂b LP = − ∑ i αiyi = 0 (18) yi(xi · w+ b) − 1 ≥ 0 i = 1, · · · , l (19) αi ≥ 0 ∀i (20) αi(yi(w · xi + b) − 1) = 0 ∀i (21) The KKT conditions are satisfied at the…...
[...]
...For more on nonlinear programming techniques see (Fletcher, 1987; Mangasarian, 1969; McCormick, 1983)....
[...]
...…with any kind of constraints, provided that the intersection of the set of feasible directions with the set of descent directions coincides with the intersection of the set of feasible directions for linearized constraints with the set of descent directions (see Fletcher, 1987; McCormick, 1983))....
[...]

Journal Article•DOI•

A tutorial on support vector regression

[...]

Alexander J. Smola¹, Bernhard Schölkopf²•Institutions (2)

Australian National University¹, Max Planck Society²

01 Aug 2004-Statistics and Computing

TL;DR: This tutorial gives an overview of the basic ideas underlying Support Vector (SV) machines for function estimation, and includes a summary of currently used algorithms for training SV machines, covering both the quadratic programming part and advanced methods for dealing with large datasets.

...read moreread less

Abstract: In this tutorial we give an overview of the basic ideas underlying Support Vector (SV) machines for function estimation. Furthermore, we include a summary of currently used algorithms for training SV machines, covering both the quadratic (or convex) programming part and advanced methods for dealing with large datasets. Finally, we mention some modifications and extensions that have been applied to the standard SV algorithm, and discuss the aspect of regularization from a SV perspective.

...read moreread less

10,696 citations

Cites background from "Practical Methods of Optimization"

...This requirement is made, as we want to ensure the existence and uniqueness (for strict convexity) of a minimum of optimization problems [Fletcher, 1989]....
[...]
...[Fletcher, 1989]....
[...]

Journal Article•DOI•

Least Squares Support Vector Machine Classifiers

[...]

Johan A. K. Suykens¹, Joos Vandewalle¹•Institutions (1)

Katholieke Universiteit Leuven¹

01 Jun 1999-Neural Processing Letters

TL;DR: A least squares version for support vector machine (SVM) classifiers that follows from solving a set of linear equations, instead of quadratic programming for classical SVM's.

...read moreread less

Abstract: In this letter we discuss a least squares version for support vector machine (SVM) classifiers. Due to equality type constraints in the formulation, the solution follows from solving a set of linear equations, instead of quadratic programming for classical SVM‘s. The approach is illustrated on a two-spiral benchmark classification problem.

...read moreread less

8,811 citations

Cites background from "Practical Methods of Optimization"

...Because the matrixassociated with suykens.tex; 24/11/1999; 16:59; p.3 this quadratic programming problem is not indefinite, the soluti n to (11) will be global (Fletcher, 1987)....
[...]
...…(16) One defines the Lagrangian L3(w;b;e;α) = J3(w;b;e) N∑ k=1αkfyk[wTϕ(xk)+b] 1+ekg (17) suykens.tex; 24/11/1999; 16:59; p.4 whereαk are Lagrange multipliers (which can be either positive or negative now due to the equality constraints as follows from the Kuhn-Tucker conditions (Fletcher, 1987))....
[...]
...which is equivalent to ykTwT 0.xk/CbU1 ;k D 1;:::;N; ( 3 )...
[...]
...Furthermore, one can show that hyperplanes ( 3 ) satisfying the constraintkwk2 a have a VC-dimensionh which is bounded by...
[...]
...In order to have the possibility to violate ( 3 ), in case a separating hyperplane in this higher dimensional space does not exist, variables xk are introduced such that ykTwT 0.xk/CbU1 xk ;k D 1;:::;N;...
[...]

Journal Article•DOI•

On the implementation of an interior-point filter line-search algorithm for large-scale nonlinear programming

[...]

Andreas Wächter¹, Lorenz T. Biegler²•Institutions (2)

IBM¹, Carnegie Mellon University²

01 May 2006-Mathematical Programming

TL;DR: A comprehensive description of the primal-dual interior-point algorithm with a filter line-search method for nonlinear programming is provided, including the feasibility restoration phase for the filter method, second-order corrections, and inertia correction of the KKT matrix.

...read moreread less

Abstract: We present a primal-dual interior-point algorithm with a filter line-search method for nonlinear programming. Local and global convergence properties of this method were analyzed in previous work. Here we provide a comprehensive description of the algorithm, including the feasibility restoration phase for the filter method, second-order corrections, and inertia correction of the KKT matrix. Heuristics are also considered that allow faster performance. This method has been implemented in the IPOPT code, which we demonstrate in a detailed numerical study based on 954 problems from the CUTEr test set. An evaluation is made of several line-search options, and a comparison is provided with two state-of-the-art interior-point codes for nonlinear programming.

...read moreread less

7,966 citations

Cites background from "Practical Methods of Optimization"

..., [7, 12]) to improve the proposed step if a trial point has been rejected....
[...]
...Note that if the regularization parameter ζ > 0 is chosen sufficiently small, the optimization problem (30) is the exact penalty formulation [12] of the problem “find the feasible point that is closest (in a weighted norm) to the reference point x̄R ,”...
[...]

Collapse

Practical Methods of Optimization

Citations

Cites background from "Practical Methods of Optimization"

Cites background from "Practical Methods of Optimization"

Cites background from "Practical Methods of Optimization"

Cites background from "Practical Methods of Optimization"

Related Papers (5)