Numerical Optimization presents a comprehensive and up-to-date description of the most effective methods in continuous optimization. It responds to the growing interest in optimization in engineering, science, and business by focusing on the methods that are best suited to practical problems. For this new edition the book has been thoroughly updated throughout. There are new chapters on nonlinear interior methods and derivative-free methods for optimization, both of which are used widely in practice and the focus of much current research. Because of the emphasis on practical methods, as well as the extensive illustrations and exercises, the book is accessible to a wide audience. It can be used as a graduate text in engineering, operations research, mathematics, computer science, and business. It also serves as a handbook for researchers and practitioners in the field. The authors have strived to produce a text that is pleasant to read, informative, and rigorous - one that reveals both the beautiful nature of the discipline and its practical side.

Numerical Optimization

In the first part of this thesis, we introduce a specific class of Linear Matrix Inequalities (LMI) whose optimal solution can be characterized exactly. This family corresponds to the case where the associated linear operator maps the cone of positive semidefinite matrices onto itself. In this case, the optimal value equals the spectral radius of the operator. It is shown that some rank minimization problems, as well as generalizations of the structured singular value ($mu$) LMIs, have exactly this property.

In the same spirit of exploiting structure to achieve computational efficiency, an algorithm for the numerical solution of a special class of frequency-dependent LMIs is presented. These optimization problems arise from robustness analysis questions, via the Kalman-Yakubovich-Popov lemma. The procedure is an outer approximation method based on the algorithms used in the computation of hinf norms for linear, time invariant systems. The result is especially useful for systems with large state dimension.

The other main contribution in this thesis is the formulation of a convex optimization framework for semialgebraic problems, i.e., those that can be expressed by polynomial equalities and inequalities. The key element is the interaction of concepts in real algebraic geometry (Positivstellensatz) and semidefinite programming.

To this end, an LMI formulation for the sums of squares decomposition for multivariable polynomials is presented. Based on this, it is shown how to construct sufficient Positivstellensatz-based convex tests to prove that certain sets are empty. Among other applications, this leads to a nonlinear extension of many LMI based results in uncertain linear system analysis.

Within the same framework, we develop stronger criteria for matrix copositivity, and generalizations of the well-known standard semidefinite relaxations for quadratic programming.

Some applications to new and previously studied problems are presented. A few examples are Lyapunov function computation, robust bifurcation analysis, structured singular values, etc. It is shown that the proposed methods allow for improved solutions for very diverse questions in continuous and combinatorial optimization.

Structured semidefinite programs and semialgebraic geometry methods in robustness and optimization

A hierarchy of convex relaxations for semialgebraic problems is introduced. For questions reducible to a finite number of polynomial equalities and inequalities, it is shown how to construct a complete family of polynomially sized semidefinite programming conditions that prove infeasibility. The main tools employed are a semidefinite programming formulation of the sum of squares decomposition for multivariate polynomials, and some results from real algebraic geometry. The techniques provide a constructive approach for finding bounded degree solutions to the Positivstellensatz, and are illustrated with examples from diverse application fields.

/pdf/semidefinite-programming-relaxations-for-semialgebraic-54kejoyrbk.pdf

Semidefinite programming relaxations for semialgebraic problems

We consider linear programs with uncertain parameters, lying in some prescribed uncertainty set, where part of the variables must be determined before the realization of the uncertain parameters (‘‘non-adjustable variables’’), while the other part are variables that can be chosen after the realization (‘‘adjustable variables’’). We extend the Robust Optimization methodology ([1, 3-6, 9, 13, 14]) to this situation by introducing the Adjustable Robust Counterpart (ARC) associated with an LP of the above structure. Often the ARC is significantly less conservative than the usual Robust Counterpart (RC), however, in most cases the ARC is computationally intractable (NP-hard). This difficulty is addressed by restricting the adjustable variables to be affine functions of the uncertain data. The ensuing Affinely Adjustable Robust Counterpart (AARC) problem is then shown to be, in certain important cases, equivalent to a tractable optimization problem (typically an LP or a Semidefinite problem), and in other cases, having a tight approximation which is tractable. The AARC approach is illustrated by applying it to a multi-stage inventory management problem.

https://www2.isye.gatech.edu/~nemirovs/MP_Elana_2004.pdf

Adjustable robust solutions of uncertain linear programs

Preface. Contributing Authors.- 1. The Traveling Salesman Problem: Applications, Formulations and Variations.- 2. Polyhedral Theory and Branch-and-Cut Algorithms for the Symmetric TSP.- 3. Polyhedral Theory for the Asymmetric Traveling Salesman Problem.- 4. Exact Methods for the Asymmetric Traveling Salesman Problem.- 5. Approximation Algorithms for Geometric TSP.- 6. Exponential Neighborhoods and Domination Analysis for the TSP.- 7. Probabilistic Analysis of the TSP.- 8. Local Search and Metaheuristics.- 9. Experimental Analysis of Heuristics for the STSP.- 10. Experimental Analysis of Heuristics for the ATSP.- 11. Polynomially Solvable Cases of the TSP.- 12. The Maximum TSP.- 13. The Generalized Traveling Salesman and Orienteering Problems.- 14. The Prize Collecting Traveling Salesman Problem and Its Applications.- 15. The Bottleneck TSP.- 16. TSP Software.- Appendix A: Sets, Graphs and Permutations. Appendix B: Computational Complexity. References. List of Figures. List of Tables. Index.

/pdf/the-traveling-salesman-problem-and-its-variations-4ya507d5u2.pdf

The traveling salesman problem and its variations

In continuous variable, smooth, nonconvex nonlinear programming, we analyze the complexity of checking whether(a)a given feasible solution is not a local minimum, and(b)the objective function is not bounded below on the set of feasible solutions.

We construct a special class of indefinite quadratic programs, with simple constraints and integer data, and show that checking (a) or (b) on this class is NP-complete. As a corollary, we show that checking whether a given integer square matrix is not copositive, is NP-complete.

/pdf/some-np-complete-problems-in-quadratic-and-nonlinear-3skjnc9tqf.pdf

Some NP-complete problems in quadratic and nonlinear programming

Fuzzy models for single-period inventory problem

http://www.optimization-online.org/DB_FILE/2009/12/2491.pdf

The minimum spanning tree problem with conflict constraints and its variations

We show that the 2-Opt and 3-Opt heuristics for the traveling salesman problem (TSP) on the complete graph Kn produce a solution no worse than the average cost of a tour in Kn in a polynomial number of iterations. As a consequence, we get that the domination numbers of the 2- Opt , 3- Opt , Carlier—Villon, Shortest Path Ejection Chain, and Lin—Kernighan heuristics are all at least (n-2)! / 2 . The domination number of the Christofides heuristic is shown to be no more than $\lceil{n}/{2}\rceil !$ , and for the Double Tree heuristic and a variation of the Christofides heuristic the domination numbers are shown to be one (even if the edge costs satisfy the triangle inequality). Further, unless P = NP, no polynomial time approximation algorithm exists for the TSP on the complete digraph $\vec{K}_n$ with domination number at least (n-1)!-k for any constant k or with domination number at least (n-1)! - (( k /(k+1))(n+r))!-1 for any non-negative constants r and k such that (n+r) $\equiv$ 0 mod (k+1). The complexities of finding the median value of costs of all the tours in $\vec{K}_n$ and of similar problems are also studied.

Santosh N. Kabadi

Papers

Some NP-complete problems in quadratic and nonlinear programming

Fuzzy models for single-period inventory problem

The minimum spanning tree problem with conflict constraints and its variations

TSP Heuristics: Domination Analysis and Complexity

A linear time algorithm for the Koopmans–Beckmann QAP linearization and related problems