Showing papers on "Nonlinear programming published in 2005"

SNOPT: An SQP Algorithm for Large-Scale Constrained Optimization

Abstract: Sequential quadratic programming (SQP) methods have proved highly effective for solving constrained optimization problems with smooth nonlinear functions in the objective and constraints. Here we consider problems with general inequality constraints (linear and nonlinear). We assume that first derivatives are available and that the constraint gradients are sparse. Second derivatives are assumed to be unavailable or too expensive to calculate. We discuss an SQP algorithm that uses a smooth augmented Lagrangian merit function and makes explicit provision for infeasibility in the original problem and the QP subproblems. The Hessian of the Lagrangian is approximated using a limited-memory quasi-Newton method. SNOPT is a particular implementation that uses a reduced-Hessian semidefinite QP solver (SQOPT) for the QP subproblems. It is designed for problems with many thousands of constraints and variables but is best suited for problems with a moderate number of degrees of freedom (say, up to 2000). Numerical results are given for most of the CUTEr and COPS test collections (about 1020 examples of all sizes up to 40000 constraints and variables, and up to 20000 degrees of freedom).

A polyhedral branch-and-cut approach to global optimization

Abstract: A variety of nonlinear, including semidefinite, relaxations have been developed in recent years for nonconvex optimization problems. Their potential can be realized only if they can be solved with sufficient speed and reliability. Unfortunately, state-of-the-art nonlinear programming codes are significantly slower and numerically unstable compared to linear programming software.In this paper, we facilitate the reliable use of nonlinear convex relaxations in global optimization via a polyhedral branch-and-cut approach. Our algorithm exploits convexity, either identified automatically or supplied through a suitable modeling language construct, in order to generate polyhedral cutting planes and relaxations for multivariate nonconvex problems. We prove that, if the convexity of a univariate or multivariate function is apparent by decomposing it into convex subexpressions, our relaxation constructor automatically exploits this convexity in a manner that is much superior to developing polyhedral outer approximators for the original function. The convexity of functional expressions that are composed to form nonconvex expressions is also automatically exploited.Root-node relaxations are computed for 87 problems from globallib and minlplib, and detailed computational results are presented for globally solving 26 of these problems with BARON 7.2, which implements the proposed techniques. The use of cutting planes for these problems reduces root-node relaxation gaps by up to 100% and expedites the solution process, often by several orders of magnitude.

Uncertain convex programs: randomized solutions and confidence levels

Abstract: Many engineering problems can be cast as optimization problems subject to convex constraints that are parameterized by an uncertainty or ‘instance’ parameter. Two main approaches are generally available to tackle constrained optimization problems in presence of uncertainty: robust optimization and chance-constrained optimization. Robust optimization is a deterministic paradigm where one seeks a solution which simultaneously satisfies all possible constraint instances. In chance-constrained optimization a probability distribution is instead assumed on the uncertain parameters, and the constraints are enforced up to a pre-specified level of probability. Unfortunately however, both approaches lead to computationally intractable problem formulations. In this paper, we consider an alternative ‘randomized’ or ‘scenario’ approach for dealing with uncertainty in optimization, based on constraint sampling. In particular, we study the constrained optimization problem resulting by taking into account only a finite set of N constraints, chosen at random among the possible constraint instances of the uncertain problem. We show that the resulting randomized solution fails to satisfy only a small portion of the original constraints, provided that a sufficient number of samples is drawn. Our key result is to provide an efficient and explicit bound on the measure (probability or volume) of the original constraints that are possibly violated by the randomized solution. This volume rapidly decreases to zero as N is increased.

Linear array geometry synthesis with minimum sidelobe level and null control using particle swarm optimization

Abstract: This paper describes the synthesis method of linear array geometry with minimum sidelobe level and null control using the particle swarm optimization (PSO) algorithm. The PSO algorithm is a newly discovered, high-performance evolutionary algorithm capable of solving general N-dimensional, linear and nonlinear optimization problems. Compared to other evolutionary methods such as genetic algorithms and simulated annealing, the PSO algorithm is much easier to understand and implement and requires the least of mathematical preprocessing. The array geometry synthesis is first formulated as an optimization problem with the goal of sidelobe level (SLL) suppression and/or null placement in certain directions, and then solved by the PSO algorithm for the optimum element locations. Three design examples are presented that illustrate the use of the PSO algorithm, and the optimization goal in each example is easily achieved. The results of the PSO algorithm are validated by comparing with results obtained using the quadratic programming method (QPM).

A simple multimembered evolution strategy to solve constrained optimization problems

Abstract: This work presents a simple multimembered evolution strategy to solve global nonlinear optimization problems. The approach does not require the use of a penalty function. Instead, it uses a simple diversity mechanism based on allowing infeasible solutions to remain in the population. This technique helps the algorithm to find the global optimum despite reaching reasonably fast the feasible region of the search space. A simple feasibility-based comparison mechanism is used to guide the process toward the feasible region of the search space. Also, the initial stepsize of the evolution strategy is reduced in order to perform a finer search and a combined (discrete/intermediate) panmictic recombination technique improves its exploitation capabilities. The approach was tested with a well-known benchmark. The results obtained are very competitive when comparing the proposed approach against other state-of-the art techniques and its computational cost (measured by the number of fitness function evaluations) is lower than the cost required by the other techniques compared.

A Real-Time Iteration Scheme for Nonlinear Optimization in Optimal Feedback Control

Abstract: An efficient Newton-type scheme for the approximate on-line solution of optimization problems as they occur in optimal feedback control is presented. The scheme allows a fast reaction to disturbances by delivering approximations of the exact optimal feedback control which are iteratively refined during the runtime of the controlled process. The contractivity of this real-time iteration scheme is proven, and a bound on the loss of optimality---compared with the theoretical optimal solution---is given. The robustness and excellent real-time performance of the method is demonstrated in a numerical experiment, the control of an unstable system, namely, an airborne kite that shall fly loops.

Geometric Programming for Communication Systems

Abstract: Geometric Programming (GP) is a class of nonlinear optimization with many useful theoretical and computational properties. Over the last few years, GP has been used to solve a variety of problems in the analysis and design of communication systems in several 'layers' in the communication network architecture, including information theory problems, signal processing algorithms, basic queuing system optimization, many network resource allocation problems such as power control and congestion control, and cross-layer design. We also start to understand why, in addition to how, GP can be applied to a surprisingly wide range of problems in communication systems. These applications have in turn spurred new research activities on GP, especially generalizations of GP formulations and development of distributed algorithms to solve GP in a network. This text provides both an in-depth tutorial on the theory, algorithms, and modeling methods of GP, and a comprehensive survey on the applications of GP to the study of communication systems.

Bilevel programming: A survey

Abstract: This paper provides an introductory survey of a class of optimization problems known as bilevel programming. We motivate this class through a simple application, and then proceed with the general formulation of bilevel programs. We consider various cases (linear, linear-quadratic, nonlinear), describe their main properties and give an overview of solution approaches.

Local Minima and Convergence in Low-Rank Semidefinite Programming

Abstract: The low-rank semidefinite programming problem LRSDPr is a restriction of the semidefinite programming problem SDP in which a bound r is imposed on the rank of X, and it is well known that LRSDPr is equivalent to SDP if r is not too small. In this paper, we classify the local minima of LRSDPr and prove the optimal convergence of a slight variant of the successful, yet experimental, algorithm of Burer and Monteiro [5], which handles LRSDPr via the nonconvex change of variables X=RRT. In addition, for particular problem classes, we describe a practical technique for obtaining lower bounds on the optimal solution value during the execution of the algorithm. Computational results are presented on a set of combinatorial optimization relaxations, including some of the largest quadratic assignment SDPs solved to date.

On energy provisioning and relay node placement for wireless sensor networks

Abstract: Wireless sensor networks that operate on batteries have limited network lifetime. There have been extensive recent research efforts on how to design protocols and algorithms to prolong network lifetime. However, due to energy constraint, even under the most efficient protocols and algorithms, the network lifetime may still be unable to meet the mission's requirements. In this paper, we consider the energy provisioning (EP) problem for a two-tiered wireless sensor network. In addition to provisioning additional energy on the existing nodes, we also consider deploying relay nodes (RNs) into the network to mitigate network geometric deficiencies and prolong network lifetime. We formulate the joint problem of EP and RN placement (EP-RNP) into a mixed-integer nonlinear programming (MINLP) problem. Since an MINLP problem is NP-hard in general, and even state-of-the-art software and techniques are unable to offer satisfactory solutions, we develop a heuristic algorithm, called Smart Pairing and INtelligent Disc Search (SPINDS), to address this problem. We show a number of novel algorithmic design techniques in the design of SPINDS that effectively transform a complex MINLP problem into a linear programming (LP) problem without losing critical points in its search space. Through numerical results, we show that SPINDS offers a very attractive solution and some important insights to the EP-RNP problem.

Line Search Filter Methods for Nonlinear Programming: Motivation and Global Convergence

Abstract: Line search methods are proposed for nonlinear programming using Fletcher and Leyffer's filter method [Math. Program., 91 (2002), pp. 239--269], which replaces the traditional merit function. Their global convergence properties are analyzed. The presented framework is applied to active set sequential quadratic programming (SQP) and barrier interior point algorithms. Under mild assumptions it is shown that every limit point of the sequence of iterates generated by the algorithm is feasible, and that there exists at least one limit point that is a stationary point for the problem under consideration. A new alternative filter approach employing the Lagrangian function instead of the objective function with identical global convergence properties is briefly discussed.

Damped Newton algorithms for matrix factorization with missing data

Abstract: The problem of low-rank matrix factorization in the presence of missing data has seen significant attention in recent computer vision research. The approach that dominates the literature is EM-like alternation of closed-form solutions for the two factors of the matrix. An obvious alternative is nonlinear optimization of both factors simultaneously, a strategy which has seen little published research. This paper provides a comprehensive comparison of the two strategies by evaluating previously published factorization algorithms as well as some second order methods not previously presented for this problem. We conclude that, although alternation approaches can be very quick, their propensity to glacial convergence in narrow valleys of the cost function means that average-case performance is worse than second-order strategies. Further, we demonstrate the importance of two main observations: one, that schemes based on closed-form solutions alone are not suitable and that non-linear optimization strategies are faster, more accurate and provide more flexible frameworks for continued progress; and two, that basic objective functions are not adequate and that regularization priors must be incorporated, a process that is easier with nonlinear methods.

Solution algorithm for the bi-level discrete network design problem

Abstract: The discrete network design problem deals with the selection of link additions to an existing road network, with given demand from each origin to each destination. The objective is to make an optimal investment decision in order to minimize the total travel cost in the network, while accounting for the route choice behaviors of network users. Because of the computational difficulties experienced with the solution algorithm of nonlinear bi-level mixed integer programming with a large number of 0–1 variables, the discrete network design problem has been recognized as one of the most difficult yet challenging problems in transport. In this paper, at first a traditional bi-level programming model for the discrete network design problem is introduced, and then a new solution algorithm is proposed by using the support function concept to express the relationship between improvement flows and the new additional links in the existing urban network. Finally, the applications of the new algorithm are illustrated with two numerical examples. Numerical results indicate that the proposed algorithm would be efficient in practice.

Experimental Testing of Advanced Scatter Search Designs for Global Optimization of Multimodal Functions

Abstract: Scatter search is an evolutionary method that, unlike genetic algorithms, operates on a small set of solutions and makes only limited use of randomization as a proxy for diversification when searching for a globally optimal solution. The scatter search framework is flexible, allowing the development of alternative implementations with varying degrees of sophistication. In this paper, we test the merit of several scatter search designs in the context of global optimization of multimodal functions. We compare these designs among themselves and choose one to compare against a well-known genetic algorithm that has been specifically developed for this class of problems. The testing is performed on a set of benchmark multimodal functions with known global minima.

Line Search Filter Methods for Nonlinear Programming: Local Convergence

Abstract: A line search method is proposed for nonlinear programming using Fletcher and Leyffer's filter method, which replaces the traditional merit function. A simple modification of the method proposed in a companion paper [SIAM J. Optim., 16 (2005), pp. 1--31] introducing second order correction steps is presented. It is shown that the proposed method does not suffer from the Maratos effect, so that fast local convergence to second order sufficient local solutions is achieved.

A recurrent neural network for solving nonlinear convex programs subject to linear constraints

Abstract: In this paper, we propose a recurrent neural network for solving nonlinear convex programming problems with linear constraints. The proposed neural network has a simpler structure and a lower complexity for implementation than the existing neural networks for solving such problems. It is shown here that the proposed neural network is stable in the sense of Lyapunov and globally convergent to an optimal solution within a finite time under the condition that the objective function is strictly convex. Compared with the existing convergence results, the present results do not require Lipschitz continuity condition on the objective function. Finally, examples are provided to show the applicability of the proposed neural network.

An evolutionary algorithm for solving nonlinear bilevel programming based on a new constraint-handling scheme

Abstract: In this paper, a special nonlinear bilevel programming problem (nonlinear BLPP) is transformed into an equivalent single objective nonlinear programming problem. To solve the equivalent problem effectively, we first construct a specific optimization problem with two objectives. By solving the specific problem, we can decrease the leader's objective value, identify the quality of any feasible solution from infeasible solutions and the quality of two feasible solutions for the equivalent single objective optimization problem, force the infeasible solutions moving toward the feasible region, and improve the feasible solutions gradually. We then propose a new constraint-handling scheme and a specific-design crossover operator. The new constraint-handling scheme can make the individuals satisfy all linear constraints exactly and the nonlinear constraints approximately. The crossover operator can generate high quality potential offspring. Based on the constraint-handling scheme and the crossover operator, we propose a new evolutionary algorithm and prove its global convergence. A distinguishing feature of the algorithm is that it can be used to handle nonlinear BLPPs with nondifferentiable leader's objective functions. Finally, simulations on 31 benchmark problems, 12 of which have nondifferentiable leader's objective functions, are made and the results demonstrate the effectiveness of the proposed algorithm.

Computational approach based on a particle swarm optimizer for microwave imaging of two-dimensional dielectric scatterers

Abstract: A computational approach based on an innovative stochastic algorithm, namely, the particle swarm optimizer (PSO), is proposed for the solution of the inverse-scattering problem arising in microwave-imaging applications. The original inverse-scattering problem is reformulated in a global nonlinear optimization one by defining a suitable cost function, which is minimized through a customized PSO. In such a framework, this paper is aimed at assessing the effectiveness of the proposed approach in locating, shaping, and reconstructing the dielectric parameters of unknown two-dimensional scatterers. Such an analysis is carried out by comparing the performance of the PSO-based approach with other state-of-the-art methods (deterministic, as well as stochastic) in terms of retrieval accuracy, as well as from a computational point-of-view. Moreover, an integrated strategy (based on the combination of the PSO and the iterative multiscaling method) is proposed and analyzed to fully exploit complementary advantages of nonlinear optimization techniques and multiresolution approaches. Selected numerical experiments concerning dielectric scatterers different in shape, dimension, and dielectric profile, are performed starting from synthetic, as well as experimental inverse-scattering data.

Parametric study for an ant algorithm applied to water distribution system optimization

Abstract: Much research has been carried out on the optimization of water distribution systems (WDSs). Within the last decade, the focus has shifted from the use of traditional optimization methods, such as linear and nonlinear programming, to the use of heuristics derived from nature (HDNs), namely, genetic algorithms, simulated annealing and more recently, ant colony optimization (ACO), an optimization algorithm based on the foraging behavior of ants. HDNs have been seen to perform better than more traditional optimization methods and amongst the HDNs applied to WDS optimization, a recent study found ACO to outperform other HDNs for two well-known case studies. One of the major problems that exists with the use of HDNs, particularly ACO, is that their searching behavior and, hence, performance, is governed by a set of user-selected parameters. Consequently, a large calibration phase is required for successful application to new problems. The aim of this paper is to provide a deeper understanding of ACO parameters and to develop parametric guidelines for the application of ACO to WDS optimization. For the adopted ACO algorithm, called AS/sub i-best/ (as it uses an iteration-best pheromone updating scheme), seven parameters are used: two decision policy control parameters /spl alpha/ and /spl beta/, initial pheromone value /spl tau//sub 0/, pheromone persistence factor /spl rho/, number of ants m, pheromone addition factor Q, and the penalty factor (PEN). Deterministic and semi-deterministic expressions for Q and PEN are developed. For the remaining parameters, a parametric study is performed, from which guidelines for appropriate parameter settings are developed. Based on the use of these heuristics, the performance of AS/sub i-best/ was assessed for two case studies from the literature (the New York Tunnels Problem, and the Hanoi Problem) and an additional larger case study (the Doubled New York Tunnels Problem). The results show that AS/sub i-best/ achieves the best performance presented in the literature, in terms of efficiency and solution quality, for the New York Tunnels Problem. Although AS/sub i-best/ does not perform as well as other algorithms from the literature for the Hanoi Problem (a notably difficult problem), it successfully finds the known least cost solution for the larger Doubled New York Tunnels Problem.

Generalized analytical technique for the synthesis of unequally spaced arrays with linear, planar, cylindrical or spherical geometry

Abstract: An effective method for optimizing the performance of a fixed current distribution, uniformly spaced antenna array has been to adjust its element positions to provide performance improvement. In comparison with the default uniform structure, this approach yields performance improvements such as smaller sidelobe levels or beamwidth values. Additionally, it provides practical advantages such as reductions in size, weight and number of antenna elements. The objective of this paper is to describe a unified mathematical approach to nonlinear optimization of multidimensional array geometries. The approach utilizes a class of limiting properties of sinusoidal, Bessel or Legendre functions that are dictated by the array geometry addressed. The efficacy of the method is demonstrated by its generalized application to synthesis of rectangular, cylindrical and spherical arrays. The unified mathematical approach presented below is a synthesis technique founded on the mathematical transformation of the desired field pattern, followed by the application of limiting forms of the transformation, and resulting in the development of a closed form expression for the element positions. The method offers the following advantages over previous techniques such as direct nonlinear optimization or genetic algorithms. First, it is not an iterative, searching algorithm, and provides element spacing values directly in a single run of the algorithm, thereby saving valuable CPU time and memory storage. Second, It permits the array designer to place practical constraints on the array geometry, (e.g., the minimum/maximum spacing between adjacent elements)

Automated design of total water systems

Abstract: An automated method for the design of total water systems is developed in this paper. This approach considers simultaneously the optimal distribution of water to satisfy process demands and optimal treatment of effluent streams. Treatment can be for discharge to the environment or for regeneration of wastewater. The cases of regeneration reuse and regeneration recycle can be distinguished in the approach. It combines engineering insights with mathematical programming tools based on a superstructure model that results in a mixed-integer nonlinear programming problem. The approach features a fast and robust solution strategy. Complex tradeoffs involving operating, as well as capital, costs and other practical constraints have been included. In particular, piping and sewer costs, which are a major element in the capital cost of such systems, can be included. Network complexity is controlled by specifying the minimum permissible flow rates in the network, the maximum number of streams allowed at mixing juncti...

An application of swarm optimization to nonlinear programming

Abstract: Particle swarm optimization (PSO) is an optimization technique based on population, which has similarities to other evolutionary algorithms. It is initialized with a population of random solutions and searches for optima by updating generations. Particle swarm optimization has become the hotspot of evolutionary computation because of its excellent performance and simple implementation. After introducing the basic principle of the PSO, a particle swarm optimization algorithm embedded with constraint fitness priority-based ranking method is proposed in this paper to solve nonlinear programming problem. By designing the fitness function and constraints-handling method, the proposed PSO can evolve with a dynamic neighborhood and varied inertia weighted value to find the global optimum. The results from this preliminary investigation are quite promising and show that this algorithm is reliable and applicable to almost all of the problems in multiple-dimensional, nonlinear and complex constrained programming. It is proved to be efficient and robust by testing some example and benchmarks of the constrained nonlinear programming problems.

Thermal via planning for 3-D ICs

Abstract: Heat dissipation is one of the most serious challenges in 3-D IC designs. One effective way of reducing circuit temperature is to introduce thermal through-the-silicon (TTS) vias. In this paper, we extended the TTS-via planning in a multilevel routing framework by Cong and Zhang (2005), but use a much enhanced TTS-via planning algorithm. We formulate the TTS-via minimization problem with temperature constraints as a constrained nonlinear programming problem (NLP) based on the thermal resistive model and develop an efficient heuristic algorithm, named m-ADVP, which solves a sequence of simplified via planning subproblems in alternating direction in a multilevel framework. The vertical via distribution is formulated as a convex programming problem, and the horizontal via planning is based on two efficient techniques: path counting and heat propagation. Experimental results show that the m-ADVP algorithm is more than 200/spl times/ faster than the direct solution to the NPL formulation for via planning with very similar solution quality (within 1% of TS-vias count). However, compared to a recent work of multilevel TS-via planning algorithm based on temperature profiling (Cong and Zhang, 2005), our algorithm can reduce the total TS-via number by over 68% for the same required temperature with similar runtime.

Cross-layer rate control for end-to-end proportional fairness in wireless networks with random access

Abstract: In this paper, we address the rate control problem in a multi-hop random access wireless network, with the objective of achieving proportional fairness amongst the end-to-end sessions. The problem is considered in the framework of nonlinear optimization. Compared to its counterpart in a wired network where link capacities are assumed to be fixed, rate control in a multi-hop random access network is much more complex and requires joint optimization at both the transport layer and the link layer. This is due to the fact that the attainable throughput on each link in the network is `elastic' and is typically a non-convex and non-separable function of the transmission attempt rates. Two cross-layer algorithms, a dual based algorithm and a primal based algorithm, are proposed in this paper to solve the rate control problem in a multi-hop random access network. Both algorithms can be implemented in a distributed manner, and work at the link layer to adjust link attempt probabilities and at the transport layer to adjust session rates. We prove rigorously that the two proposed algorithms converge to the globally optimal solutions. Simulation results are provided to support our conclusions.