Numerical Optimization presents a comprehensive and up-to-date description of the most effective methods in continuous optimization. It responds to the growing interest in optimization in engineering, science, and business by focusing on the methods that are best suited to practical problems. For this new edition the book has been thoroughly updated throughout. There are new chapters on nonlinear interior methods and derivative-free methods for optimization, both of which are used widely in practice and the focus of much current research. Because of the emphasis on practical methods, as well as the extensive illustrations and exercises, the book is accessible to a wide audience. It can be used as a graduate text in engineering, operations research, mathematics, computer science, and business. It also serves as a handbook for researchers and practitioners in the field. The authors have strived to produce a text that is pleasant to read, informative, and rigorous - one that reveals both the beautiful nature of the discipline and its practical side.

Numerical Optimization

Since its popularization in the late 1970s, Sequential Quadratic Programming (SQP) has arguably become the most successful method for solving nonlinearly constrained optimization problems. As with most optimization methods, SQP is not a single algorithm, but rather a conceptual method from which numerous specific algorithms have evolved. Backed by a solid theoretical and computational foundation, both commercial and public-domain SQP algorithms have been developed and used to solve a remarkably large set of important practical problems. Recently large-scale versions have been devised and tested with promising results.

Sequential Quadratic Programming

This book is about the theoretical foundations of optimization algorithms, and also provides practical insights on how such methods should be implemented and applied. The goal of this text is not focused on in-depth details of state-of-the-art methods, but rather to provide a good foundation of the tried and trusted methods and how they can be used to solve real-world problems efficiently. The target audience for the text seems to be graduate students and engineers in industry as a good reference or self-study. However, it does not lend itself to easy adaption in a classroom setting due to the fact that only some chapters have problem sets and most chapters do not have a summary section. The book has four main parts each written by the one of the different authors. Two of the parts deal with standard unconstrained and constrained optimization techniques. The other two section deal with more complicated issues including solving problems that exhibit nonsmooth behavior and interior point methods. Solving optimization problems that have nonsmooth objectives or constraints is a formidable challenge because sensitivity information cannot be calculated using traditional approaches; the book provides a variety of ways to approach such problems. Interior point methods have been gaining popularity in the engineering optimization community for their efficiency and ability to maintain feasibility throughout the optimization process. There have been many textbooks written on the subject of numerical optimization. This book has the same flavor as many of these past books. A book with a very similar title by Nocedal and Wright [1] covered much of the same material, but it did not address nonsmooth problems. A classical book in the field by Flecther [2] considers nonsmooth problems but not interior point methods. Also, if one is more interested in comparing the performance of the various methods applied to engineering design problems, the book by Vanderplaats [3] may be more suitable. The appeal of this book is in the authors’ attention to detail and their advice on practical implementation issues. For many of the different optimization algorithms detailed flow charts are given. They also provide adequate examples to help the reader understand the methods better and explore possible pitfalls. There are, however, a few downsides of this book. The main one is that through translation of the book to English, it is evident that some of the finer points were lost and in some cases the text is hard to understand. The book could also have benefited from more figures; there are only 26. Finally, each of the dif

Numerical Optimization: Theoretical and Practical Aspects

Deriving priorities from fuzzy pairwise comparison judgements

We present pyOpt, an object-oriented framework for formulating and solving nonlinear constrained optimization problems in an efficient, reusable and portable manner. The framework uses object-oriented concepts, such as class inheritance and operator overloading, to maintain a distinct separation between the problem formulation and the optimization approach used to solve the problem. This creates a common interface in a flexible environment where both practitioners and developers alike can solve their optimization problems or develop and benchmark their own optimization algorithms. The framework is developed in the Python programming language, which allows for easy integration of optimization software programmed in Fortran, C, C+?+, and other languages. A variety of optimization algorithms are integrated in pyOpt and are accessible through the common interface. We solve a number of problems of increasing complexity to demonstrate how a given problem is formulated using this framework, and how the framework can be used to benchmark the various optimization algorithms.

pyOpt: a Python-based object-oriented framework for nonlinear constrained optimization

When solving inequality constrained optimization problems via Sequential Quadratic Programming (SQP), it is potentially advantageous to generate iterates that all satisfy the constraints: all quadratic programs encountered are then feasible and there is no need for a surrogate merit function. (Feasibility of the successive iterates is in fact required in many contexts such as in real-time applications or when the objective function is not defined outside the feasible set.) It has recently been shown that this is, indeed, possible, by means of a suitable perturbation of the original SQP iteration, without losing superlinear convergence. In this context, the well-known Maratos effect is compounded by the possible infeasibility of the full step of one even close to a solution. These difficulties have been accommodated by making use of a suitable modification of a “bending” technique proposed by Mayne and Polak, requiring evaluation of the constraints function at an auxiliary point at each iteration.In Part I...

Avoiding the Maratos effect by means of a nonmonotone line search II. Inequality constrained problems—feasible iterates

An error is pointed out in the local convergence proof in the quoted paper [J. L. Zhou and A. L. Tits, SIAM J. Optim., 6 (1996), pp. 461--487]. A correct proof is given.

https://epubs.siam.org/doi/pdf/10.1137/S1052623496313659

An SQP Algorithm for Finely Discretized Continuous Minimax Problems and Other Minimax Problems with Many Objective Functions

An algorithm for linear programming (LP) and convex quadratic programming (CQP) is proposed, based on an interior point iteration introduced more than ten years ago by J. Herskovits for the solution of nonlinear programming problems. Herskovits’ iteration can be simplified significantly in the LP/CQP case, and quadratic convergence from any initial point can be achieved. Interestingly the linear system solved at each iteration is identical to that of the primal-dual affine scaling scheme recently considered by Monteiro et al. independently of Herskovits’ work. The proposed algorithm, however, uses an iteratively selected step length, different for each component of the dual variable.

/pdf/a-simple-quadratically-convergent-interior-point-algorithm-3gcn2gz72y.pdf

Jian L. Zhou

Papers

Avoiding the Maratos effect by means of a nonmonotone line search II. Inequality constrained problems—feasible iterates

An SQP Algorithm for Finely Discretized Continuous Minimax Problems and Other Minimax Problems with Many Objective Functions

A Simple, Quadratically Convergent Interior Point Algorithm for Linear Programming and Convex Quadratic Programming