Numerical Optimization presents a comprehensive and up-to-date description of the most effective methods in continuous optimization. It responds to the growing interest in optimization in engineering, science, and business by focusing on the methods that are best suited to practical problems. For this new edition the book has been thoroughly updated throughout. There are new chapters on nonlinear interior methods and derivative-free methods for optimization, both of which are used widely in practice and the focus of much current research. Because of the emphasis on practical methods, as well as the extensive illustrations and exercises, the book is accessible to a wide audience. It can be used as a graduate text in engineering, operations research, mathematics, computer science, and business. It also serves as a handbook for researchers and practitioners in the field. The authors have strived to produce a text that is pleasant to read, informative, and rigorous - one that reveals both the beautiful nature of the discipline and its practical side.

Numerical Optimization

Many problems in signal processing and statistical inference involve finding sparse solutions to under-determined, or ill-conditioned, linear systems of equations. A standard approach consists in minimizing an objective function which includes a quadratic (squared ) error term combined with a sparseness-inducing regularization term. Basis pursuit, the least absolute shrinkage and selection operator (LASSO), wavelet-based deconvolution, and compressed sensing are a few well-known examples of this approach. This paper proposes gradient projection (GP) algorithms for the bound-constrained quadratic programming (BCQP) formulation of these problems. We test variants of this approach that select the line search parameters in different ways, including techniques based on the Barzilai-Borwein method. Computational experiments show that these GP approaches perform well in a wide range of applications, often being significantly faster (in terms of computation time) than competing methods. Although the performance of GP methods tends to degrade as the regularization term is de-emphasized, we show how they can be embedded in a continuation scheme to recover their efficient practical performance.

/pdf/gradient-projection-for-sparse-reconstruction-application-to-jxqvrma0ll.pdf

Gradient Projection for Sparse Reconstruction: Application to Compressed Sensing and Other Inverse Problems

Introduction to stochastic programming

JuMP is an open-source modeling language that allows users to express a wide range of optimization problems (linear, mixed-integer, quadratic, conic-quadratic, semidefinite, and nonlinear) in a high-level, algebraic syntax. JuMP takes advantage of advanced features of the Julia programming language to offer unique functionality while achieving performance on par with commercial modeling tools for standard tasks. In this work we will provide benchmarks, present the novel aspects of the implementation, and discuss how JuMP can be extended to new problem classes and composed with state-of-the-art tools for visualization and interactivity.

JuMP: A Modeling Language for Mathematical Optimization

The existence of locally optimal solutions to the AC optimal power flow problem (OPF) has been a question of interest for decades This paper presents examples of local optima on a variety of test networks including modified versions of common networks We show that local optima can occur because the feasible region is disconnected and/or because of nonlinearities in the constraints Standard local optimization techniques are shown to converge to these local optima The voltage bounds of all the examples in this paper are between ±5% and ±10% off-nominal The examples with local optima are available in an online archive (http://wwwmathsedacuk/optenergy/LocalOpt/) and can be used to test local or global optimization techniques for OPF Finally we use our test examples to illustrate the behavior of a recent semi-definite programming approach that aims to find the global solution of OPF

/pdf/local-solutions-of-the-optimal-power-flow-problem-mpqu5fm0vi.pdf

Local Solutions of the Optimal Power Flow Problem

In this paper, a flexible optimization-based framework for intentional islanding is presented. The decision is made of which transmission lines to switch in order to split the network while minimizing disruption, the amount of load shed, or grouping coherent generators. The approach uses a piecewise linear model of AC power flow, which allows the voltage and reactive power to be considered directly when designing the islands. Demonstrations on standard test networks show that solution of the problem provides islands that are balanced in real and reactive power, satisfy AC power flow laws, and have a healthy voltage profile.

Optimization-Based Islanding of Power Networks Using Piecewise Linear AC Power Flow

OOPS is an object-oriented parallel solver using the primal–dual interior point methods. Its main component is an object-oriented linear algebra library designed to exploit nested block structure that is often present in truly large-scale optimization problems such as those appearing in Stochastic Programming. This is achieved by treating the building blocks of the structured matrices as objects, that can use their inherent linear algebra implementations to efficiently exploit their structure both in a serial and parallel environment. Virtually any nested block-structure can be exploited by representing the matrices defining the problem as a tree build from these objects. OOPS can be run on a wide variety of architectures and has been used to solve a financial planning problem with over 109 decision variables. We give details of supported structures and their implementations. Further we give details of how parallelisation is managed in the object-oriented framework.

/pdf/exploiting-structure-in-parallel-implementation-of-interior-1uuhk5bmti.pdf

Exploiting structure in parallel implementation of interior point methods for optimization

MILP formulation for controlled islanding of power networks

Many practical large-scale optimization problems are not only sparse, but also display some form of block-structure such as primal or dual block angular structure. Often these structures are nested: each block of the coarse top level structure is block-structured itself. Problems with these characteristics appear frequently in stochastic programming but also in other areas such as telecommunication network modelling.

/pdf/parallel-interior-point-solver-for-structured-quadratic-30zk8wll4h.pdf

Andreas Grothey

Papers

Local Solutions of the Optimal Power Flow Problem

Optimization-Based Islanding of Power Networks Using Piecewise Linear AC Power Flow

Exploiting structure in parallel implementation of interior point methods for optimization

MILP formulation for controlled islanding of power networks

Parallel interior-point solver for structured quadratic programs: Application to financial planning problems