Probability Distributions.- Linear Models for Regression.- Linear Models for Classification.- Neural Networks.- Kernel Methods.- Sparse Kernel Machines.- Graphical Models.- Mixture Models and EM.- Approximate Inference.- Sampling Methods.- Continuous Latent Variables.- Sequential Data.- Combining Models.

Pattern Recognition and Machine Learning

Many optimal decision problems in scientific, engineering, and public sector applications involve both discrete decisions and nonlinear system dynamics that affect the quality of the final design or plan. These decision problems lead to mixed-integer nonlinear programming (MINLP) problems that combine the combinatorial difficulty of optimizing over discrete variable sets with the challenges of handling nonlinear functions. We review models and applications of MINLP, and survey the state of the art in methods for solving this challenging class of problems.Most solution methods for MINLP apply some form of tree search. We distinguish two broad classes of methods: single-tree and multitree methods. We discuss these two classes of methods first in the case where the underlying problem functions are convex. Classical single-tree methods include nonlinear branch-and-bound and branch-and-cut methods, while classical multitree methods include outer approximation and Benders decomposition. The most efficient class of methods for convex MINLP are hybrid methods that combine the strengths of both classes of classical techniques.Non-convex MINLPs pose additional challenges, because they contain non-convex functions in the objective function or the constraints; hence even when the integer variables are relaxed to be continuous, the feasible region is generally non-convex, resulting in many local minima. We discuss a range of approaches for tackling this challenging class of problems, including piecewise linear approximations, generic strategies for obtaining convex relaxations for non-convex functions, spatial branch-and-bound methods, and a small sample of techniques that exploit particular types of non-convex structures to obtain improved convex relaxations.We finish our survey with a brief discussion of three important aspects of MINLP. First, we review heuristic techniques that can obtain good feasible solution in situations where the search-tree has grown too large or we require real-time solutions. Second, we describe an emerging area of mixed-integer optimal control that adds systems of ordinary differential equations to MINLP. Third, we survey the state of the art in software for MINLP.

/pdf/mixed-integer-nonlinear-optimization-3yf57zxyv3.pdf

Mixed-integer nonlinear optimization

This manuscript introduces ANTIGONE, Algorithms for coNTinuous/Integer Global Optimization of Nonlinear Equations, a general mixed-integer nonlinear global optimization framework. ANTIGONE is the evolution of the Global Mixed-Integer Quadratic Optimizer, GloMIQO, to general nonconvex terms. The purpose of this paper is to show how the extensible structure of ANTIGONE realizes our previously-proposed mixed-integer quadratically-constrained quadratic program and mixed-integer signomial optimization computational frameworks. To demonstrate the capacity of ANTIGONE, this paper presents computational results on a test suite of $$2{,}571$$ 2 , 571 problems from standard libraries and the open literature; we compare ANTIGONE to other state-of-the-art global optimization solvers.

ANTIGONE: Algorithms for coNTinuous / Integer Global Optimization of Nonlinear Equations

> > . . .. .. ... .... ..... ..... ...... ........ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ... . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . † †

/pdf/complete-search-in-continuous-global-optimization-and-5ecbfk4fdi.pdf

Complete search in continuous global optimization and constraint satisfaction

This paper presents an overview of the research progress in deterministic global optimization during the last decade (1998---2008). It covers the areas of twice continuously differentiable nonlinear optimization, mixed-integer nonlinear optimization, optimization with differential-algebraic models, semi-infinite programming, optimization with grey box/nonfactorable models, and bilevel nonlinear optimization.

/pdf/a-review-of-recent-advances-in-global-optimization-169pg8n90t.pdf

A review of recent advances in global optimization

Global optimization in the 21st century: Advances and challenges

We present a new class of convex underestimators for arbitrarily nonconvex and twice continuously differentiable functions. The underestimators are derived by augmenting the original nonconvex function by a nonlinear relaxation function. The relaxation function is a separable convex function, that involves the sum of univariate parametric exponential functions. An efficient procedure that finds the appropriate values for those parameters is developed. This procedure uses interval arithmetic extensively in order to verify whether the new underestimator is convex. For arbitrarily nonconvex functions it is shown that these convex underestimators are tighter than those generated by the ?BB method. Computational studies complemented with geometrical interpretations demonstrate the potential benefits of the proposed improved convex underestimators.

A New Class of Improved Convex Underestimators for Twice Continuously Differentiable Constrained NLPs

In this paper, we develop a modeling and optimization procedure for minimizing the operating costs of a combined cooling, heating, and power (CCHP) plant at the University of California, Irvine, which uses co-generation and Thermal Energy Storage (TES) capabilities. Co-generation allows the production of thermal energy along with electricity, by recovering heat from the generators in a power plant. TES provides the ability to ‘reshape’ the cooling demands during the course of a day, in refrigeration and air-conditioning plants. Therefore, both cogeneration and TES provide a potential to improve the efficiency and economy of energy conversion. The proposed modeling and optimization approach aims to design a supervisory control strategy to effectively utilize this potential, and involves analysis over multiple physical domains which the CCHP system spans, such as thermal, mechanical, chemical and electrical. Advantages of the proposed methodology are demonstrated using simulation case studies.

/pdf/modeling-and-optimization-of-a-combined-cooling-heating-and-1fi7v4furl.pdf

Modeling and optimization of a combined cooling, heating and power plant system

In Akrotirianakis and Floudas (2004) we presented the theoretical foundations of a new class of convex underestimators for C2 nonconvex functions. In this paper, we present computational experience with those underestimators incorporated within a Branch-and-Bound algorithm for box-conatrained problems. The algorithm can be used to solve global optimization problems that involve C2 functions. We discuss several ways of incorporating the convex underestimators within a Branch-and-Bound framework. The resulting Branch-and-Bound algorithm is then used to solve a number of difficult box-constrained global optimization problems. A hybrid algorithm is also introduced, which incorporates a stochastic algorithm, the Random-Linkage method, for the solution of the nonconvex underestimating subproblems, arising within a Branch-and-Bound framework. The resulting algorithm also solves efficiently the same set of test problems.

Computational Experience with a New Class of Convex Underestimators: Box-constrained NLP Problems

In Akrotirianakis and Floudas (2004) we presented the theoretical foundations of a new class of convex underestimators for C 2 nonconvex functions. In this paper, we present computational experience with those underestimators incorporated within a Branch-and-Bound algorithm for box-conatrained problems. The algorithm can be used to solve global optimization problems that involve C 2 functions. We discuss several ways of incorporating the convex underestimators within a Branch-and-Bound framework. The resulting Branch-and-Bound algorithm is then used to solve a number of difficult box-constrained global optimization problems. A hybrid algorithm is also introduced, which incorporates a stochastic algorithm, the Random-Linkage method, for the solution of the nonconvex underestimating subproblems, arising within a Branch-and-Bound framework. The resulting algorithm also solves efficiently the same set of test problems.

Ioannis Akrotirianakis

Papers

Global optimization in the 21st century: Advances and challenges

A New Class of Improved Convex Underestimators for Twice Continuously Differentiable Constrained NLPs

Modeling and optimization of a combined cooling, heating and power plant system

Computational Experience with a New Class of Convex Underestimators: Box-constrained NLP Problems

Computational Experience with a New Class of Convex Underestimators: Box-constrained