Showing papers on "Discrete optimization published in 2010"

A Novel Set-Based Particle Swarm Optimization Method for Discrete Optimization Problems

Abstract: Particle swarm optimization (PSO) is predominately used to find solutions for continuous optimization problems. As the operators of PSO are originally designed in an n-dimensional continuous space, the advancement of using PSO to find solutions in a discrete space is at a slow pace. In this paper, a novel set-based PSO (S-PSO) method for the solutions of some combinatorial optimization problems (COPs) in discrete space is presented. The proposed S-PSO features the following characteristics. First, it is based on using a set-based representation scheme that enables S-PSO to characterize the discrete search space of COPs. Second, the candidate solution and velocity are defined as a crisp set, and a set with possibilities, respectively. All arithmetic operators in the velocity and position updating rules used in the original PSO are replaced by the operators and procedures defined on crisp sets, and sets with possibilities in S-PSO. The S-PSO method can thus follow a similar structure to the original PSO for searching in a discrete space. Based on the proposed S-PSO method, most of the existing PSO variants, such as the global version PSO, the local version PSO with different topologies, and the comprehensive learning PSO (CLPSO), can be extended to their corresponding discrete versions. These discrete PSO versions based on S-PSO are tested on two famous COPs: the traveling salesman problem and the multidimensional knapsack problem. Experimental results show that the discrete version of the CLPSO algorithm based on S-PSO is promising.

Process Optimization: A Statistical Approach

Abstract: Preliminaries.- An Overview of Empirical Process Optimization.- Elements of Response Surface Methods.- Optimization Of First Order Models.- Experimental Designs For First Order Models.- Analysis and Optimization of Second Order Models.- Experimental Designs for Second Order Models.- Statistical Inference in Process Optimization.- Statistical Inference in First Order RSM Optimization.- Statistical Inference in Second Order RSM Optimization.- Bias Vs. Variance.- Robust Parameter Design and Robust Optimization.- Robust Parameter Design.- Robust Optimization.- Bayesian Approaches in Process Optimization.- to Bayesian Inference.- Bayesian Methods for Process Optimization.- to Optimization of Simulation and Computer Models.- Simulation Optimization.- Kriging and Computer Experiments.- Appendices.- Basics of Linear Regression.- Analysis of Variance.- Matrix Algebra and Optimization Results.- Some Probability Results Used in Bayesian Inference.

Structural optimization based on CAD-CAE integration and metamodeling techniques

Abstract: Traditional structural optimization, which identifies the best combination of geometrical parameters to improve the product's performance and to save the material, is often carried out manually. This paper presents a framework that performs the integration between commercial CAD-CAE software by using common scripting, programming languages and Application Programming Interface. The loop of design-analysis-redesign in optimization process was done automatically and seamlessly without interaction with designer. Along with CAD-CAE computer-aided tools, metamodeling techniques including response surface methodology and radial basis function were applied to structural optimization according to the number of design variables. This approach reduces the time for solving computation-intensive design optimization problems and the designers are free from monotonous repetitive tasks. Three case studies were carried out in order to verify the feasibility and general-purpose characteristics of the proposed method for the structural optimization process of mechanical components. The results show that the proposed method facilitates the structural optimization process and reduces the computing cost compared to other approaches.

Adaptive Harmony Search Method for Structural Optimization

Abstract: This paper presents an adaptive harmony search algorithm for solving structural optimization problems. The harmony memory considering rate and pitch adjusting rate are conceived as the two main parameters of the technique for generating new solution vectors. In the standard implementation of the technique appropriate constant values are assigned to these parameters following a sensitivity analysis for each problem considered. The success of the optimization process is directly related on a chosen parameter value set. The adaptive harmony search algorithm proposed here incorporates a new approach for adjusting these parameters automatically during the search for the most efficient optimization process. The efficiency of the proposed algorithm is numerically investigated using two large-scale steel frameworks that are designed for minimum weight according to the provisions of ASD-AISC specification. The solutions obtained are compared with those of the standard algorithm as well as of the other metaheuristic search techniques. It is shown that the proposed algorithm improves performance of the technique and it renders unnecessary the initial selection of the harmony search parameters.

An algorithm for nonlinear optimization problems with binary variables

Abstract: One of the challenging optimization problems is determining the minimizer of a nonlinear programming problem that has binary variables. A vexing difficulty is the rate the work to solve such problems increases as the number of discrete variables increases. Any such problem with bounded discrete variables, especially binary variables, may be transformed to that of finding a global optimum of a problem in continuous variables. However, the transformed problems usually have astronomically large numbers of local minimizers, making them harder to solve than typical global optimization problems. Despite this apparent disadvantage, we show that the approach is not futile if we use smoothing techniques. The method we advocate first convexifies the problem and then solves a sequence of subproblems, whose solutions form a trajectory that leads to the solution. To illustrate how well the algorithm performs we show the computational results of applying it to problems taken from the literature and new test problems with known optimal solutions.

From Transformation-Based Dimensionality Reduction to Feature Selection

Abstract: Many learning applications are characterized by high dimensions. Usually not all of these dimensions are relevant and some are redundant. There are two main approaches to reduce dimensionality: feature selection and feature transformation. When one wishes to keep the original meaning of the features, feature selection is desired. Feature selection and transformation are typically presented separately. In this paper, we introduce a general approach for converting transformation-based methods to feature selection methods through l1/l∞ regularization. Instead of solving feature selection as a discrete optimization, we relax and formulate the problem as a continuous optimization problem. An additional advantage of our formulation is that our optimization criterion optimizes for feature relevance and redundancy removal automatically. Here, we illustrate how our approach can be utilized to convert linear discriminant analysis (LDA) and the dimensionality reduction version of the Hilbert-Schmidt Independence Criterion (HSIC) to two new feature selection algorithms. Experiments show that our new feature selection methods out-perform related state-of-the-art feature selection approaches.

Optimization : theory and practice

Abstract: 1. Introduction: Examples of Optimization Problems, Historical Overview.- 2. Optimality Conditions: Convex Sets, Inequalities, Local First- and Second-Order Optimality Conditions, Duality.- 3. Unconstrained Optimization Problems: Elementary Search and Localization Methods, Descent Methods with Line Search, Trust Region Methods, Conjugate Gradient Methods, Quasi-Newton Methods.- 4. Linearly Constrained Optimization Problems: Linear and Quadratic Optimization, Projection Methods.- 5. Nonlinearly Constrained Optimization Methods: Penalty Methods, SQP Methods.- 6. Interior-Point Methods for Linear Optimization: The Central Path, Newton's Method for the Primal-Dual System, Path-Following Algorithms, Predictor-Corrector Methods.- 7. Semidefinite Optimization: Selected Special Cases, The S-Procedure, The Function log det, Path-Following Methods, How to Solve SDO Problems?, Icing on the Cake: Pattern Separation via Ellipsoids.- 8. Global Optimization: Branch and Bound Methods, Cutting Plane Methods.- Appendices: A Second Look at the Constraint Qualifications, The Fritz John Condition, Optimization Software Tools for Teaching and Learning.- Bibliography.- Index of Symbols.- Subject Index.

Social Emotional Optimization Algorithm for Nonlinear Constrained Optimization Problems

Abstract: Nonlinear programming problem is one important branch in operational research, and has been successfully applied to various real-life problems. In this paper, a new approach called Social emotional optimization algorithm (SEOA) is used to solve this problem which is a new swarm intelligent technique by simulating the human behavior guided by emotion. Simulation results show that the social emotional optimization algorithm proposed in this paper is effective and efficiency for the nonlinear constrained programming problems.

Time-bounded sequential parameter optimization

Abstract: The optimization of algorithm performance by automatically identifying good parameter settings is an important problem that has recently attracted much attention in the discrete optimization community. One promising approach constructs predictive performance models and uses them to focus attention on promising regions of a design space. Such methods have become quite sophisticated and have achieved significant successes on other problems, particularly in experimental design applications. However, they have typically been designed to achieve good performance only under a budget expressed as a number of function evaluations (e.g., target algorithm runs). In this work, we show how to extend the Sequential Parameter Optimization framework [SPO; see 5] to operate effectively under time bounds. Our methods take into account both the varying amount of time required for different algorithm runs and the complexity of model building and evaluation; they are particularly useful for minimizing target algorithm runtime. Specifically, we avoid the up-front cost of an initial design, introduce a time-bounded intensification mechanism, and show how to reduce the overhead incurred by constructing and using models. Overall, we show that our method represents a new state of the art in model-based optimization of algorithms with continuous parameters on single problem instances.

SamACO: Variable Sampling Ant Colony Optimization Algorithm for Continuous Optimization

Abstract: An ant colony optimization (ACO) algorithm offers algorithmic techniques for optimization by simulating the foraging behavior of a group of ants to perform incremental solution constructions and to realize a pheromone laying-and-following mechanism. Although ACO is first designed for solving discrete (combinatorial) optimization problems, the ACO procedure is also applicable to continuous optimization. This paper presents a new way of extending ACO to solving continuous optimization problems by focusing on continuous variable sampling as a key to transforming ACO from discrete optimization to continuous optimization. The proposed SamACO algorithm consists of three major steps, i.e., the generation of candidate variable values for selection, the ants' solution construction, and the pheromone update process. The distinct characteristics of SamACO are the cooperation of a novel sampling method for discretizing the continuous search space and an efficient incremental solution construction method based on the sampled values. The performance of SamACO is tested using continuous numerical functions with unimodal and multimodal features. Compared with some state-of-the-art algorithms, including traditional ant-based algorithms and representative computational intelligence algorithms for continuous optimization, the performance of SamACO is seen competitive and promising.

A simple and efficient constrained particle swarm optimization and its application to engineering design problems

Abstract: Engineering optimization problems usually contain various constraints and mixed integer-discrete-continuous type of design variables. This article proposes an efficient particle swarm optimization (PSO) algorithm for such problems. First, the constrained optimization problem is transformed into an unconstrained problem without introducing any problem-dependent or user-defined parameters such as penalty factors or Lagrange multipliers, though such parameters are usually required in general optimization algorithms. Then, the above PSO method is extended to handle integer, discrete, and continuous design variables in a simple manner, yet with a high degree of precision. The proposed PSO scheme is fairly simple and thus it is easy to implement. In order to demonstrate the effectiveness of our method, several mechanical design optimization problems are solved, and the numerical results are compared with those reported in the literature.

Domain decomposition and model reduction for the numerical solution of PDE constrained optimization problems with localized optimization variables

Abstract: We introduce a technique for the dimension reduction of a class of PDE constrained optimization problems governed by linear time dependent advection diffusion equations for which the optimization variables are related to spatially localized quantities. Our approach uses domain decomposition applied to the optimality system to isolate the subsystem that explicitly depends on the optimization variables from the remaining linear optimality subsystem. We apply balanced truncation model reduction to the linear optimality subsystem. The resulting coupled reduced optimality system can be interpreted as the optimality system of a reduced optimization problem. We derive estimates for the error between the solution of the original optimization problem and the solution of the reduced problem. The approach is demonstrated numerically on an optimal control problem and on a shape optimization problem.

Practical mixed-integer optimization for geometry processing

Abstract: Solving mixed-integer problems, i.e., optimization problems where some of the unknowns are continuous while others are discrete, is NP-hard. Unfortunately, real-world problems like e.g., quadrangular remeshing usually have a large number of unknowns such that exact methods become unfeasible. In this article we present a greedy strategy to rapidly approximate the solution of large quadratic mixed-integer problems within a practically sufficient accuracy. The algorithm, which is freely available as an open source library implemented in C++, determines the values of the discrete variables by successively solving relaxed problems. Additionally the specification of arbitrary linear equality constraints which typically arise as side conditions of the optimization problem is possible. The performance of the base algorithm is strongly improved by two novel extensions which are (1) simultaneously estimating sets of discrete variables which do not interfere and (2) a fill-in reducing reordering of the constraints. Exemplarily the solver is applied to the problem of quadrilateral surface remeshing, enabling a great flexibility by supporting different types of user guidance within a real-time modeling framework for input surfaces of moderate complexity.

A new multi-objective programming scheme for topology optimization of compliant mechanisms

Abstract: This paper presents an alternative method in implementing multi-objective optimization of compliant mechanisms in the field of continuum-type topology optimization. The method is designated as “SIMP-PP” and it achieves multi-objective topology optimization by merging what is already a mature topology optimization method—solid isotropic material with penalization (SIMP) with a variation of the robust multi-objective optimization method—physical programming (PP). By taking advantages of both sides, the combination causes minimal variation in computation algorithm and numerical scheme, yet yields improvements in the multi-objective handling capability of topology optimization. The SIMP-PP multi-objective scheme is introduced into the systematic design of compliant mechanisms. The final optimization problem is formulated mathematically using the aggregate objective function which is derived from the original individual design objectives with PP, subjected to the specified constraints. A sequential convex programming method, the method of moving asymptotes (MMA) is then utilized to process the optimization evolvement based on the design sensitivity analysis. The main findings in this study include distinct advantages of the SIMP-PP method in various aspects such as computation efficiency, adaptability in convex and non-convex multi-criteria environment, and flexibility in problem formulation. Observations are made regarding its performance and the effect of multi-objective optimization on the final topologies. In general, the proposed SIMP-PP method is an appealing multi-objective topology optimization scheme suitable for “real world” problems, and it bridges the gap between standard topological design and multi-criteria optimization. The feasibility of the proposed topology optimization method is exhibited by benchmark examples.

Covering trees and lower-bounds on quadratic assignment

Abstract: Many computer vision problems involving feature correspondence among images can be formulated as an assignment problem with a quadratic cost function. Such problems are computationally infeasible in general but recent advances in discrete optimization such as tree-reweighted belief propagation (TRW) often provide high-quality solutions. In this paper, we improve upon these algorithms in two ways. First, we introduce covering trees, a variant of TRW which provide the same bounds on the MAP energy as TRW with far fewer variational parameters. Optimization of these parameters can be carried out efficiently using either fixed–point iterations (as in TRW) or sub-gradient based techniques. Second, we introduce a new technique that utilizes bipartite matching applied to the min-marginals produced with covering trees in order to compute a tighter lower-bound for the quadratic assignment problem. We apply this machinery to the problem of finding correspondences with pairwise energy functions, and demonstrate the resulting hybrid method outperforms TRW alone and a recent related subproblem decomposition algorithm on benchmark image correspondence problems.

Optimization strategies for additional actuators of kinematically redundant parallel kinematic machines

Abstract: In this paper five different optimization strategies for kinematically redundant mechanisms, i.e. mechanisms having additional actuator(s) in at least one kinematic chain, are presented. They are based on two main approaches, a discrete optimization and a classical continuous optimization. Exemplarily, a planar, kinematically redundant 3RRR-based mechanism is introduced. The position of its redundant actuator, i.e. the robot geometry, is optimized according to an optimization criterion that is denoted as the gain of the maximal homogenized pose error. Several analysis examples demonstrate the effectiveness of kinematic redundancy with respect to the introduced optimization procedures. It is shown that in comparison to discrete approaches, classical continuousbased optimization strategies do not necessarily lead to more appropriate results in terms of performance improvement.

Continuous limits of discrete perimeters

Abstract: We consider a class of discrete convex functionals which satisfy a (generalized) coarea formula. These functionals, based on submodular interactions, arise in discrete optimization and are known as a large class of problems which can be solved in polynomial time. In particular, some of them can be solved very efficiently by maximal flow algorithms and are quite popular in the image processing community. We study the limit in the continuum of these functionals, show that they always converge to some "crystalline" perimeter/total variation, and provide an almost explicit formula for the limiting functional.

Non-rigid Registration using Discrete MRFs: Application to Thoracic CT Images

Abstract: . We recently introduced labeling of discrete Markov random fields (MRFs) as an attractive approach for non-rigid image registration. Our MRF framework makes use of recent advances in discrete optimization, is efficient in terms of computation time, and provides great flexibility. Any similarity measure can be encoded right away, since no differentiation is needed. In this work, we investigate the performance of our framework in a challenging scenario: the registration of thoracic CT images. In order to assess the potential of the discrete MRF setting, we employ the simplest registration objective function based on intensity differences. The registration is fully-automatic, constant parameters are used throughout the experiments, we omit the use of the available segmentations, and (except for linear pre-alignment) no pre-processing of the data is performed. Despite the simplicity of our experimental setup, we are able to obtain accurate registration results for most of the data in a very efficient manner. Our registration software is freely available.

On an Optimization Problem for a Class of Impulsive Hybrid Systems

Abstract: This contribution addresses the problem of optimal control for a class of hybrid systems, where discrete transitions are accompanied by instantaneous changes in the continuous state variables, and where these changes can be considered as control variables. Based on a variational approach, necessary conditions of optimality are first established. The problem is then cast as a parametric optimization problem for which gradient information is derived. Finally, we discuss assumptions that guarantee convergence of a conceptual algorithm to a stationary solution. A brief discussion on the main implementation issues is also included.

Discrete Particle Swarm Optimization for the Orienteering Problem

Abstract: In this paper a novel discrete Particle Swarm Optimization (PSO) algorithm is proposed to solve the Orienteering Problem (OP). Discrete evolution is achieved by re-defining all operators and operands used in PSO. To obtain better results, Strengthened-PSO which improves both exploration and exploitation during the search process is employed for experimental evaluation. Our proposed algorithm either achieves or improves the best known solutions compared to previous heuristics for the OP.

The differential Ant-Stigmergy Algorithm for large-scale global optimization

Abstract: Ant-colony optimization (ACO) is a popular swarm intelligence metaheuristic scheme that can be applied to almost any optimization problem. In this paper, we address a performance evaluation of an ACO-based algorithm for solving large-scale global optimization problems with continuous variables, labeled Differential Ant-Stigmergy Algorithm (DASA). The DASA transforms a real-parameter optimization problem into a graph-search problem. The parameters' differences assigned to the graph vertices are used to navigate through the search space. The performance of the DASA is evaluated on the set of benchmark problems provided for CEC'2010 Special Session and Competition on Large-Scale Global Optimization.

Reconstructing dna copy number by penalized estimation and imputation.

Abstract: Recent advances in genomics have underscored the surprising ubiquity of DNA copy number variation (CNV). Fortunately, modern genotyping platforms also detect CNVs with fairly high reliability. Hidden Markov models and algorithms have played a dominant role in the interpretation of CNV data. Here we explore CNV reconstruction via estimation with a fused-lasso penalty as suggested by Tibshirani and Wang [Biostatistics 9 (2008) 18–29]. We mount a fresh attack on this difficult optimization problem by the following: (a) changing the penalty terms slightly by substituting a smooth approximation to the absolute value function, (b) designing and implementing a new MM (majorization-minimization) algorithm, and (c) applying a fast version of Newton's method to jointly update all model parameters. Together these changes enable us to minimize the fused-lasso criterion in a highly effective way. We also reframe the reconstruction problem in terms of imputation via discrete optimization. This approach is easier and more accurate than parameter estimation because it relies on the fact that only a handful of possible copy number states exist at each SNP. The dynamic programming framework has the added bonus of exploiting information that the current fused-lasso approach ignores. The accuracy of our imputations is comparable to that of hidden Markov models at a substantially lower computational cost.

Multiobjective Optimization of Temporal Processes

Abstract: This paper presents a dynamic predictive-optimization framework of a nonlinear temporal process. Data-mining (DM) and evolutionary strategy algorithms are integrated in the framework for solving the optimization model. DM algorithms learn dynamic equations from the process data. An evolutionary strategy algorithm is then applied to solve the optimization problem guided by the knowledge extracted by the DM algorithm. The concept presented in this paper is illustrated with the data from a power plant, where the goal is to maximize the boiler efficiency and minimize the limestone consumption. This multiobjective optimization problem can be either transformed into a single-objective optimization problem through preference aggregation approaches or into a Pareto-optimal optimization problem. The computational results have shown the effectiveness of the proposed optimization framework.