This paper is devoted to bilevel optimization, a branch of mathematical programming of both practical and theoretical interest. Starting with a simple example, we proceed towards a general formulation. We then present fields of application, focus on solution approaches, and make the connection with MPECs (Mathematical Programs with Equilibrium Constraints).

/pdf/an-overview-of-bilevel-optimization-4x7xo28qzy.pdf

An overview of bilevel optimization

This paper gives an overview of the majorization-minimization (MM) algorithmic framework, which can provide guidance in deriving problem-driven algorithms with low computational cost. A general introduction of MM is presented, including a description of the basic principle and its convergence results. The extensions, acceleration schemes, and connection to other algorithmic frameworks are also covered. To bridge the gap between theory and practice, upperbounds for a large number of basic functions, derived based on the Taylor expansion, convexity, and special inequalities, are provided as ingredients for constructing surrogate functions. With the pre-requisites established, the way of applying MM to solving specific problems is elaborated by a wide range of applications in signal processing, communications, and machine learning.

Majorization-Minimization Algorithms in Signal Processing, Communications, and Machine Learning

Finding the optimal solution to a complex optimization problem is of great importance in many fields, ranging from protein structure prediction to the design of microprocessor circuitry. Some recent progress in finding the global minima of potential energy functions is described, focusing on applications of the simple "basin-hopping" approach to atomic and molecular clusters and more complicated hypersurface deformation techniques for crystals and biomolecules. These methods have produced promising results and should enable larger and more complex systems to be treated in the future.

/pdf/global-optimization-of-clusters-crystals-and-biomolecules-2ehy01ydei.pdf

Global optimization of clusters, crystals, and biomolecules

We present a new and fast method, called the cross-entropy method, for finding the optimal solution of combinatorial and continuous nonconvex optimization problems with convex bounded domains. To find the optimal solution we solve a sequence of simple auxiliary smooth optimization problems based on Kullback-Leibler cross-entropy, importance sampling, Markov chain and Boltzmann distribution. We use importance sampling as an important ingredient for adaptive adjustment of the temperature in the Boltzmann distribution and use Kullback-Leibler cross-entropy to find the optimal solution. In fact, we use the mode of a unimodal importance sampling distribution, like the mode of beta distribution, as an estimate of the optimal solution for continuous optimization and Markov chains approach for combinatorial optimization. In the later case we show almost surely convergence of our algorithm to the optimal solution. Supporting numerical results for both continuous and combinatorial optimization problems are given as well. Our empirical studies suggest that the cross-entropy method has polynomial in the size of the problem running time complexity.

The Cross-Entropy Method for Combinatorial and Continuous Optimization

Network Optimization: Continuous and Discrete Models

Introduction to global optimization

Mathematical programming problems dealing with functions, each of which can be represented as a difference of two convex functions, are called DC programming problems. The purpose of this overview is to discuss main theoretical results, some applications, and solution methods for this interesting and important class of programming problems. Some modifications and new results on the optimality conditions and development of algorithms are also presented.

DC programming: overview

For the problem of minimizing a concave function over a polytope a class of convergent algorithms is proposed, which is based upon a combination of the branch and bound technique with the cutting method developed earlier by H. Tuy.

Convergent Algorithms for Minimizing a Concave Function

On finding new vertices and redundant constraints in cutting plane algorithms for global optimization

We consider a convex multiplicative programming problem of the form % MathType!MTEF!2!1!+-% feaafiart1ev1aaatCvAUfeBSjuyZL2yd9gzLbvyNv2CaerbuLwBLn% hiov2DGi1BTfMBaeXatLxBI9gBaerbd9wDYLwzYbItLDharqqtubsr% 4rNCHbGeaGqiVu0Je9sqqrpepC0xbbL8F4rqqrFfpeea0xe9Lq-Jc9% vqaqpepm0xbba9pwe9Q8fs0-yqaqpepae9qq-f0-yqaqVeLsFr0-vr% 0-vr0db8meaabaqaciGacaGaaeqabaWaaeaaeaaakeaacaGG7bGaam% OzamaaBaaaleaacaaIXaaabeaakiaacIcacaWG4bGaaiykaiabgwSi% xlaadAgadaWgaaWcbaGaaGOmaaqabaGccaGGOaGaamiEaiaacMcaca% GG6aGaamiEaiabgIGiolaadIfacaGG9baaaa!4A08!\[\{ f_1 (x) \cdot f_2 (x):x \in X\} \] where X is a compact convex set of ℝ
 n
 and f
1, f
2 are convex functions which have nonnegative values over X. Using two additional variables we transform this problem into a problem with a special structure in which the objective function depends only on two of the (n+2) variables. Following a decomposition concept in global optimization we then reduce this problem to a master problem of minimizing a quasi-concave function over a convex set in ℝ2
2. This master problem can be solved by an outer approximation method which requires performing a sequence of simplex tableau pivoting operations. The proposed algorithm is finite when the functions f
i, (i=1, 2) are affine-linear and X is a polytope and it is convergent for the general convex case.

Nguyen Van Thoai

Papers

Introduction to global optimization

DC programming: overview

Convergent Algorithms for Minimizing a Concave Function

On finding new vertices and redundant constraints in cutting plane algorithms for global optimization

A global optimization approach for solving the convex multiplicative programming problem