Journal ArticleDOI
Convergence Properties of the Nelder--Mead Simplex Method in Low Dimensions
Reads0
Chats0
TLDR
This paper presents convergence properties of the Nelder--Mead algorithm applied to strictly convex functions in dimensions 1 and 2, and proves convergence to a minimizer for dimension 1, and various limited convergence results for dimension 2.Abstract:
The Nelder--Mead simplex algorithm, first published in 1965, is an enormously popular direct search method for multidimensional unconstrained minimization. Despite its widespread use, essentially no theoretical results have been proved explicitly for the Nelder--Mead algorithm. This paper presents convergence properties of the Nelder--Mead algorithm applied to strictly convex functions in dimensions 1 and 2. We prove convergence to a minimizer for dimension 1, and various limited convergence results for dimension 2. A counterexample of McKinnon gives a family of strictly convex functions in two dimensions and a set of initial conditions for which the Nelder--Mead algorithm converges to a nonminimizer. It is not yet known whether the Nelder--Mead method can be proved to converge to a minimizer for a more specialized class of convex functions in two dimensions.read more
Citations
More filters
Journal ArticleDOI
Optimization by Direct Search: New Perspectives on Some Classical and Modern Methods ∗
TL;DR: This review begins by briefly summarizing the history of direct search methods and considering the special properties of problems for which they are well suited, then turns to a broad class of methods for which the underlying principles allow general-ization to handle bound constraints and linear constraints.
Journal ArticleDOI
Robust Optimization - A Comprehensive Survey
TL;DR: The main focus will be on the different approaches to perform robust optimization in practice including the methods of mathematical programming, deterministic nonlinear optimization, and direct search methods such as stochastic approximation and evolutionary computation.
Journal ArticleDOI
Derivative-free optimization: a review of algorithms and comparison of software implementations
TL;DR: It is found that the ability of all these solvers to obtain good solutions diminishes with increasing problem size, and TomLAB/MULTIMIN, TOMLAB/GLCCLUSTER, MCS and TOMLab/LGO are better, on average, than other derivative-free solvers in terms of solution quality within 2,500 function evaluations.
Journal ArticleDOI
Random Gradient-Free Minimization of Convex Functions
Yurii Nesterov,Vladimir Spokoiny +1 more
TL;DR: New complexity bounds for methods of convex optimization based only on computation of the function value are proved, which appears that such methods usually need at most n times more iterations than the standard gradient methods, where n is the dimension of the space of variables.
Optimization Toolbox User's Guide
TL;DR: The software described in this document is furnished under a license agreement and the rights of the Government regarding its use, reproduction and disclosure are as set forth in Clause 52.227-19(c)(2) of the FAR.
References
More filters
Journal ArticleDOI
A simplex method for function minimization
John A. Nelder,R. Mead +1 more
TL;DR: A method is described for the minimization of a function of n variables, which depends on the comparison of function values at the (n 41) vertices of a general simplex, followed by the replacement of the vertex with the highest value by another point.
Numerical recipes in C
TL;DR: The Diskette v 2.06, 3.5''[1.44M] for IBM PC, PS/2 and compatibles [DOS] Reference Record created on 2004-09-07, modified on 2016-08-08.
Journal ArticleDOI
On the Convergence of Pattern Search Algorithms
TL;DR: The characterization of pattern search methods is exploited to establish a global convergence theory that does not enforce a notion of sufficient decrease, and is possible because the iterates of a pattern search method lie on a scaled, translated integer lattice.
Book
Dynamical systems and numerical analysis
TL;DR: In this paper, the authors unify the study of dynamical systems and numerical solution of differential equations by formulating them as dynamical system and examining the convergence and stability properties of the methods.
Journal ArticleDOI
Convergence of the Nelder--Mead Simplex Method to a Nonstationary Point
TL;DR: This paper analyzes the behavior of the Nelder--Mead simplex method for a family of examples which cause the method to converge to a nonstationary point and shows that this behavior cannot occur for functions with more than three continuous derivatives.
Related Papers (5)
Differential Evolution – A Simple and Efficient Heuristic for Global Optimization over Continuous Spaces
Rainer Storn,Kenneth Price +1 more