Showing papers on "Local search (optimization) published in 2004"

Stochastic Local Search: Foundations & Applications

Abstract: Prologue Part I Foundations 1 Introduction 2 SLS Methods 3 Generalised Local Search Machines 4 Empirical Analysis of SLS Algorithms 5 Search Space Structure and SLS Performance Part II Applications 6 SAT and Constraint Satisfaction 7 MAX-SAT and MAX-CSP 8 Travelling Salesman Problems 9 Scheduling Problems 10 Other Combinatorial Problems Epilogue Glossary

Hybrid genetic algorithms for feature selection

Abstract: This paper proposes a novel hybrid genetic algorithm for feature selection. Local search operations are devised and embedded in hybrid GAs to fine-tune the search. The operations are parameterized in terms of their fine-tuning power, and their effectiveness and timing requirements are analyzed and compared. The hybridization technique produces two desirable effects: a significant improvement in the final performance and the acquisition of subset-size control. The hybrid GAs showed better convergence properties compared to the classical GAs. A method of performing rigorous timing analysis was developed, in order to compare the timing requirement of the conventional and the proposed algorithms. Experiments performed with various standard data sets revealed that the proposed hybrid GA is superior to both a simple GA and sequential search algorithms.

Meta-Lamarckian learning in memetic algorithms

Abstract: Over the last decade, memetic algorithms (MAs) have relied on the use of a variety of different methods as the local improvement procedure. Some recent studies on the choice of local search method employed have shown that this choice significantly affects the efficiency of problem searches. Given the restricted theoretical knowledge available in this area and the limited progress made on mitigating the effects of incorrect local search method choice, we present strategies for MA control that decide, at runtime, which local method is chosen to locally improve the next chromosome. The use of multiple local methods during a MA search in the spirit of Lamarckian learning is here termed Meta-Lamarckian learning. Two adaptive strategies for Meta-Lamarckian learning are proposed in this paper. Experimental studies with Meta-Lamarckian learning strategies on continuous parametric benchmark problems are also presented. Further, the best strategy proposed is applied to a real-world aerodynamic wing design problem and encouraging results are obtained. It is shown that the proposed approaches aid designers working on complex engineering problems by reducing the probability of employing inappropriate local search methods in a MA, while at the same time, yielding robust and improved design search performance.

Exact Bayesian Structure Discovery in Bayesian Networks

Abstract: Learning a Bayesian network structure from data is a well-motivated but computationally hard task. We present an algorithm that computes the exact posterior probability of a subnetwork, e.g., a directed edge; a modified version of the algorithm finds one of the most probable network structures. This algorithm runs in time O(n 2n + nk+1C(m)), where n is the number of network variables, k is a constant maximum in-degree, and C(m) is the cost of computing a single local marginal conditional likelihood for m data instances. This is the first algorithm with less than super-exponential complexity with respect to n. Exact computation allows us to tackle complex cases where existing Monte Carlo methods and local search procedures potentially fail. We show that also in domains with a large number of variables, exact computation is feasible, given suitable a priori restrictions on the structures; combining exact and inexact methods is also possible. We demonstrate the applicability of the presented algorithm on four synthetic data sets with 17, 22, 37, and 100 variables.

An Evolutionary Approach

Abstract: In the previous three chapters we discussed various classic problem-solving methods, including dynamic programming, branch and bound, and local search algorithms, as well as some modern heuristic methods like simulated annealing and tabu search. Some of these techniques were seen to be deterministic. Essentially you “turn the crank” and out pops the answer. For these methods, given a search space and an evaluation function, some would always return the same solution (e.g., dynamic programming), while others could generate different solutions based on the initial configuration or starting point (e.g., a greedy algorithm or the hill-climbing technique). Still other methods were probabilistic, incorporating random variation into the search for optimal solutions. These methods (e.g., simulated annealing) could return different final solutions even when given the same initial configuration. No two trials with these algorithms could be expected to take exactly the same course. Each trial is much like a person’s fingerprint: although there are broad similarities across fingerprints, no two are exactly alike.

A Two-Stage Hybrid Local Search for the Vehicle Routing Problem with Time Windows

Abstract: The vehicle routing problem with time windows is a hard combinatorial optimization problem that has received considerable attention in the last decades. This paper proposes a two-stage hybrid algorithm for this transportation problem. The algorithm first minimizes the number of vehicles, using simulated annealing. It then minimizes travel cost by using a large neighborhood search that may relocate a large number of customers. Experimental results demonstrate the effectiveness of the algorithm, which has improved 10 (17%) of the 56 best published solutions to the Solomon benchmarks, while matching or improving the best solutions in 46 problems (82%). More important perhaps, the algorithm is shown to be very robust. With a fixed configuration of its parameters, it returns either the best published solutions (or improvements thereof) or solutions very close in quality on all Solomon benchmarks. Very preliminary results on the extended Solomon benchmarks are also given.

Real-coded memetic algorithms with crossover hill-climbing

Abstract: This paper presents a real-coded memetic algorithm that applies a crossover hill-climbing to solutions produced by the genetic operators. On the one hand, the memetic algorithm provides global search (reliability) by means of the promotion of high levels of population diversity. On the other, the crossover hill-climbing exploits the self-adaptive capacity of real-parameter crossover operators with the aim of producing an effective local tuning on the solutions (accuracy). An important aspect of the memetic algorithm proposed is that it adaptively assigns different local search probabilities to individuals. It was observed that the algorithm adjusts the global/local search balance according to the particularities of each problem instance. Experimental results show that, for a wide range of problems, the method we propose here consistently outperforms other real-coded memetic algorithms which appeared in the literature.

Globalized Nelder–Mead method for engineering optimization

Abstract: One of the fundamental difficulties in engineering design is the multiplicity of local solutions. This has triggered great efforts to develop global search algorithms. Globality, however, often has a prohibitively high numerical cost for real problems. A fixed cost local search, which sequentially becomes global is developed. Globalization is achieved by probabilistic restart. A spatial probability of starting a local search is built based on past searches. An improved Nelder-Mead algorithm makes the local optimizer. It accounts for variable bounds. It is additionally made more robust by reinitializing degenerated simplexes. The resulting method, called Globalized Bounded Nelder-Mead (GBNM) algorithm, is particularly adapted to tackle multimodal, discontinuous optimization problems, for which it is uncertain that a global optimization can be afforded. Different strategies for restarting the local search are discussed. Numerical experiments are given on analytical test functions and composite laminate design problems. The GBNM method compares favorably to an evolutionary algorithm, both in terms of numerical cost and accuracy.

A knowledge based genetic algorithm for path planning of a mobile robot

Abstract: In this paper, a knowledge based genetic algorithm (GA) for path planning of a mobile robot is proposed, which uses problem-specific genetic algorithms for robot path planning instead of the standard GAs. The proposed knowledge based genetic algorithm incorporates the domain knowledge into its specialized operators, where some also combine a local search technique. The proposed genetic algorithm also features a unique and simple path representation and a simple but effective evaluation method. The knowledge based genetic algorithm is capable of finding an optimal or near-optimal robot path in both complex static and dynamic environments. The effectiveness and efficiency of the proposed genetic algorithm is demonstrated by simulation studies. The irreplaceable role of the specialized genetic operators in the proposed GA for solving robot path planning problem is demonstrated by a comparison study.

A time-predefined local search approach to exam timetabling problems

Abstract: In recent years the processing speed of computers has increased dramatically. This in turn has allowed search algorithms to execute more iterations in a given amount of real-time. Does this necessarily always lead to an improvement in the quality of final solutions? This paper is devoted to the investigation of that question. We present two variants of local search where the search time can be set as an input parameter. These two approaches are: a time-predefined variant of simulated annealing and an adaptation of the “great deluge” method. We present a comprehensive series of experiments which show that these approaches significantly outperform the previous best results (in terms of solution quality) on a range of benchmark exam timetabling problems. Of course, there is a price to pay for such better results: increased execution time. We discuss the impact of this trade-off between quality and execution time. In particular we discuss issues involving the proper estimation of the algorithm's execution tim...

Particle swarm optimization algorithm for single machine total weighted tardiness problem

Abstract: In This work we present a particle swarm optimization algorithm to solve the single machine total weighted tardiness problem. A heuristic rule, the smallest position value (SPV) rule, is developed to enable the continuous particle swarm optimization algorithm to be applied to all classes of sequencing problems, which are NP-hard in the literature. A simple but very efficient local search method is embedded in the particle swarm optimization algorithm. The computational results show that the particle swarm algorithm is able to find the optimal and best-known solutions on all instances of widely used benchmarks from the OR library.

Ant colony optimization and local search for bin packing and cutting stock problems

Abstract: The Bin Packing Problem and the Cutting Stock Problem are two related classes of NP-hard combinatorial optimization problems. Exact solution methods can only be used for very small instances, so for real-world problems, we have to rely on heuristic methods. In recent years, researchers have started to apply evolutionary approaches to these problems, including Genetic Algorithms and Evolutionary Programming. In the work presented here, we used an ant colony optimization (ACO) approach to solve both Bin Packing and Cutting Stock Problems. We present a pure ACO approach, as well as an ACO approach augmented with a simple but very effective local search algorithm. It is shown that the pure ACO approach can compete with existing evolutionary methods, whereas the hybrid approach can outperform the best-known hybrid evolutionary solution methods for certain problem classes. The hybrid ACO approach is also shown to require different parameter values from the pure ACO approach and to give a more robust performance across different problems with a single set of parameter values. The local search algorithm is also run with random restarts and shown to perform significantly worse than when combined with ACO.

Pareto Local Optimum Sets in the Biobjective Traveling Salesman Problem: An Experimental Study.

Abstract: In this article, we study Pareto local optimum sets for the biobjective Traveling Salesman Problem applying straightforward extensions of local search algorithms for the single objective case. The performance of the local search algorithms is illustrated by experimental results obtained for well known benchmark instances and comparisons to methods from literature. In fact, a 3-opt local search is able to compete with the best performing metaheuristics in terms of solution quality. Finally, we also present an empirical study of the features of the solutions found by 3-opt on a set of randomly generated instances. The results indicate the existence of several clusters of near-optimal solutions that are separated by only a few edges.

Pareto Local Optimum Sets in the Biobjective Traveling Salesman Problem: An Experimental Study

Abstract: In this article, we study Pareto local optimum sets for the biobjective Traveling Salesman Problem applying straightforward extensions of local search algorithms for the single objective case. The performance of the local search algorithms is illustrated by experimental results obtained for well known benchmark instances and comparisons to methods from literature. In fact, a 3-opt local search is able to compete with the best performing metaheuristics in terms of solution quality. Finally, we also present an empirical study of the features of the solutions found by 3-opt on a set of randomly generated instances. The results indicate the existence of several clusters of near-optimal solutions that are separated by only a few edges.

UBCSAT: an implementation and experimentation environment for SLS algorithms for SAT and MAX-SAT

Abstract: In this paper we introduce UBCSAT, a new implementation and experimentation environment for Stochastic Local Search (SLS) algorithms for SAT and MAX-SAT. Based on a novel triggered procedure architecture, UBCSAT provides implementations of numerous well-known and widely used SLS algorithms for SAT and MAX-SAT, including GSAT, WalkSAT, and SAPS; these implementations generally match or exceed the efficiency of the respective original reference implementations. Through numerous reporting and statistical features, including the measurement of run-time distributions, UBCSAT facilitates the advanced empirical analysis of these algorithms. New algorithm variants, SLS algorithms, and reporting features can be added to UBCSAT in a straightforward and efficient way. UBCSAT is implemented in C and runs on numerous platforms and operating systems; it is publicly and freely available at www.satlib.org/ubcsat.

Choosing search heuristics by non-stationary reinforcement learning

Abstract: Search decisions are often made using heuristic methods because real-world applications can rarely be tackled without any heuristics. In many cases, multiple heuristics can potentially be chosen, and it is not clear a priori which would perform best. In this article, we propose a procedure that learns, during the search process, how to select promising heuristics. The learning is based on weight adaptation and can even switch between different heuristics during search. Different variants of the approach are evaluated within a constraint-programming environment.

Advanced fitness landscape analysis and the performance of memetic algorithms

Abstract: Memetic algorithms (MAs) have demonstrated very effective in combinatorial optimization. This paper offers explanations as to why this is so by investigating the performance of MAs in terms of efficiency and effectiveness. A special class of MAs is used to discuss efficiency and effectiveness for local search and evolutionary metasearch. It is shown that the efficiency of MAs can be increased drastically with the use of domain knowledge. However, effectiveness highly depends on the structure of the problem. As is well-known, identifying this structure is made easier with the notion of fitness landscapes: the local properties of the fitness landscape strongly influence the effectiveness of the local search while the global properties strongly influence the effectiveness of the evolutionary meta-search. This paper also introduces new techniques for analyzing the fitness landscapes of combinatorial problems; these techniques focus on the investigation of random walks in the fitness landscape starting at locally optimal solutions as well as on the escape from the basins of attractions of current local optima. It is shown for NK-landscapes and landscapes of the unconstrained binary quadratic programming problem (BQP) that a random walk to another local optimum can be used to explain the efficiency of recombination in comparison to mutation. Moreover, the paper shows that other aspects like the size of the basins of attractions of local optima are important for the efficiency of MAs and a local search escape analysis is proposed. These simple analysis techniques have several advantages over previously proposed statistical measures and provide valuable insight into the behaviour of MAs on different kinds of landscapes.

An Ejection Chain Approach for the Generalized Assignment Problem

Abstract: We propose a tabu search algorithm for the generalized assignment problem, which is one of the representative combinatorial optimization problems known to be NP-hard. The algorithm features an ejection chain approach, which is embedded in a neighborhood construction to create more complex and powerful moves. We also incorporate an adaptive mechanism for adjusting search parameters, to maintain a balance between visits to feasible and infeasible regions. Computational results on benchmark instances of small sizes show that the method obtains solutions that are optimal or that deviate by at most 0.16% from the best known solutions. Comparisons with other approaches from the literature show that, for instances of larger sizes, our method obtains the best solutions among all heuristics tested.

1001 optimal PDB structure alignments: integer programming methods for finding the maximum contact map overlap.

Abstract: Protein structure comparison is a fundamental problem for structural genomics, with applications to drug design, fold prediction, protein clustering, and evolutionary studies. Despite its importance, there are very few rigorous methods and widely accepted similarity measures known for this problem. In this paper we describe the last few years of developments on the study of an emerging measure, the contact map overlap (CMO), for protein structure comparison. A contact map is a list of pairs of residues which lie in three-dimensional proximity in the protein's native fold. Although this measure is in principle computationally hard to optimize, we show how it can in fact be computed with great accuracy for related proteins by integer linear programming techniques. These methods have the advantage of providing certificates of near-optimality by means of upper bounds to the optimal alignment value. We also illustrate effective heuristics, such as local search and genetic algorithms. We were able to obtain for...