We present a new algorithm for finding the global minimum of a multivariate function subject to simple bounds. The algorithm is a modification of the standard Lipschitzian approach that eliminates the need to specify a Lipschitz constant. This is done by carrying out simultaneous searches using all possible constants from zero to infinity. On nine standard test functions, the new algorithm converges in fewer function evaluations than most competing methods.

The motivation for the new algorithm stems from a different way of looking at the Lipschitz constant. In particular, the Lipschitz constant is viewed as a weighting parameter that indicates how much emphasis to place on global versus local search. In standard Lipschitzian methods, this constant is usually large because it must equal or exceed the maximum rate of change of the objective function. As a result, these methods place a high emphasis on global search and exhibit slow convergence. In contrast, the new algorithm carries out simultaneous searches using all possible constants, and therefore operates at both the global and local level. Once the global part of the algorithm finds the basin of convergence of the optimum, the local part of the algorithm quickly and automatically exploits it. This accounts for the fast convergence of the new algorithm on the test functions.

/pdf/lipschitzian-optimization-without-the-lipschitz-constant-1lc0hthf81.pdf

Lipschitzian optimization without the Lipschitz constant

This book presents a carefully selected group of methods for unconstrained and bound constrained optimization problems and analyzes them in depth both theoretically and algorithmically. It focuses on clarity in algorithmic description and analysis rather than generality, and while it provides pointers to the literature for the most general theoretical results and robust software, the author thinks it is more important that readers have a complete understanding of special cases that convey essential ideas. A companion to Kelley's book, Iterative Methods for Linear and Nonlinear Equations (SIAM, 1995), this book contains many exercises and examples and can be used as a text, a tutorial for self-study, or a reference. Iterative Methods for Optimization does more than cover traditional gradient-based optimization: it is the first book to treat sampling methods, including the Hooke& Jeeves, implicit filtering, MDS, and Nelder& Mead schemes in a unified way.

/pdf/iterative-methods-for-optimization-4099pll8og.pdf

Iterative Methods for Optimization

Aggregation of information is of primary importance in the construction of knowledge based systems in various domains, ranging from medicine, economics, and engineering to decision-making processes, artificial intelligence, robotics, and machine learning. This book gives a broad introduction into the topic of aggregation functions, and provides a concise account of the properties and the main classes of such functions, including classical means, medians, ordered weighted averaging functions, Choquet and Sugeno integrals, triangular norms, conorms and copulas, uninorms, nullnorms, and symmetric sums. It also presents some state-of-the-art techniques, many graphical illustrations and new interpolatory aggregation functions. A particular attention is paid to identification and construction of aggregation functions from application specific requirements and empirical data. This book provides scientists, IT specialists and system architects with a self-contained easy-to-use guide, as well as examples of computer code and a software package. It will facilitate construction of decision support, expert, recommender, control and many other intelligent systems.

Aggregation Functions: A Guide for Practitioners

This paper addresses the solution of bound-constrained optimization problems using algorithms that require only the availability of objective function values but no derivative information. We refer to these algorithms as derivative-free algorithms. Fueled by a growing number of applications in science and engineering, the development of derivative-free optimization algorithms has long been studied, and it has found renewed interest in recent time. Along with many derivative-free algorithms, many software implementations have also appeared. The paper presents a review of derivative-free algorithms, followed by a systematic comparison of 22 related implementations using a test set of 502 problems. The test bed includes convex and nonconvex problems, smooth as well as nonsmooth problems. The algorithms were tested under the same conditions and ranked under several criteria, including their ability to find near-global solutions for nonconvex problems, improve a given starting point, and refine a near-optimal solution. A total of 112,448 problem instances were solved. We find that the ability of all these solvers to obtain good solutions diminishes with increasing problem size. For the problems used in this study, TOMLAB/MULTIMIN, TOMLAB/GLCCLUSTER, MCS and TOMLAB/LGO are better, on average, than other derivative-free solvers in terms of solution quality within 2,500 function evaluations. These global solvers outperform local solvers even for convex problems. Finally, TOMLAB/OQNLP, NEWUOA, and TOMLAB/MULTIMIN show superior performance in terms of refining a near-optimal solution.

/pdf/derivative-free-optimization-a-review-of-algorithms-and-2qmveq3j89.pdf

Derivative-free optimization: a review of algorithms and comparison of software implementations

A protein molecule is a covalent chain of amino acid residues. Although it is topologically linear, in physiological conditions it folds into a unique (though flexible) three-dimensional structure. This structure, which has been determined by x-ray crystallography and nuclear magnetic resonance for many proteins (Bernstein et al., 1977; Abola et al., 1987), is referred to as the native structure. As demonstrated by the experiments of Anfinsen and co-workers (Anfinsen et al., 1961; Anfinsen, 1973), at least some protein molecules, when denatured (unfolded) by disrupting conditions in their environment (such as acidity or high temperature) can spontaneously refold to their native structures when proper physiological conditions are restored. Thus, all of the information necessary to determine the native structure can be contained in the amino acid sequence.

Computational Complexity, Protein Structure Prediction, and the Levinthal Paradox

In this paper a sequential search method for finding the global maximum of an objective function is proposed. The method is applicable to an objective function of a single variable defined on a closed interval and such that some bound on its rate of change is available. The method is shown to be minimax. Computational aspects of the method are also discussed.

Bruno O. Shubert

Papers

A Sequential Method Seeking the Global Maximum of a Function