An analogy with the way ant colonies function has suggested the definition of a new computational paradigm, which we call ant system (AS). We propose it as a viable new approach to stochastic combinatorial optimization. The main characteristics of this model are positive feedback, distributed computation, and the use of a constructive greedy heuristic. Positive feedback accounts for rapid discovery of good solutions, distributed computation avoids premature convergence, and the greedy heuristic helps find acceptable solutions in the early stages of the search process. We apply the proposed methodology to the classical traveling salesman problem (TSP), and report simulation results. We also discuss parameter selection and the early setups of the model, and compare it with tabu search and simulated annealing using TSP. To demonstrate the robustness of the approach, we show how the ant system (AS) can be applied to other optimization problems like the asymmetric traveling salesman, the quadratic assignment and the job-shop scheduling. Finally we discuss the salient characteristics-global data structure revision, distributed communication and probabilistic transitions of the AS.

/pdf/ant-system-optimization-by-a-colony-of-cooperating-agents-gw00wix2tf.pdf

Ant system: optimization by a colony of cooperating agents

A framework is developed to explore the connection between effective optimization algorithms and the problems they are solving. A number of "no free lunch" (NFL) theorems are presented which establish that for any algorithm, any elevated performance over one class of problems is offset by performance over another class. These theorems result in a geometric interpretation of what it means for an algorithm to be well suited to an optimization problem. Applications of the NFL theorems to information-theoretic aspects of optimization and benchmark measures of performance are also presented. Other issues addressed include time-varying optimization problems and a priori "head-to-head" minimax distinctions between optimization algorithms, distinctions that result despite the NFL theorems' enforcing of a type of uniformity over all algorithms.

/pdf/no-free-lunch-theorems-for-optimization-aoo4zkggf4.pdf

No free lunch theorems for optimization

This paper introduces the ant colony system (ACS), a distributed algorithm that is applied to the traveling salesman problem (TSP). In the ACS, a set of cooperating agents called ants cooperate to find good solutions to TSPs. Ants cooperate using an indirect form of communication mediated by a pheromone they deposit on the edges of the TSP graph while building solutions. We study the ACS by running experiments to understand its operation. The results show that the ACS outperforms other nature-inspired algorithms such as simulated annealing and evolutionary computation, and we conclude comparing ACS-3-opt, a version of the ACS augmented with a local search procedure, to some of the best performing algorithms for symmetric and asymmetric TSPs.

http://www-igm.univ-mlv.fr/~lombardy/ens/JavaTTT0708/fourmis.pdf

Ant colony system: a cooperative learning approach to the traveling salesman problem

This is the second half of a two part series devoted to the tabu search metastrategy for optimization problems. Part I introduced the fundamental ideas of tabu search as an approach for guiding other heuristics to overcome the limitations of local optimality, both in a deterministic and a probabilistic framework. Part I also reported successful applications from a wide range of settings, in which tabu search frequently made it possible to obtain higher quality solutions than previously obtained with competing strategies, generally with less computational effort. Part II, in this issue, examines refinements and more advanced aspects of tabu search. Following a brief review of notation, Part II introduces new dynamic strategies for managing tabu lists, allowing fuller exploitation of underlying evaluation functions. In turn, the elements of staged search and structured move sets are characterized, which bear on the issue of finiteness. Three ways of applying tabu search to the solution of integer programmin...

https://leeds-faculty.colorado.edu/glover/ts - part ii-orsa-aw.pdf

Tabu Search—Part II

Highly-interconnected networks of nonlinear analog neurons are shown to be extremely effective in computing. The networks can rapidly provide a collectively-computed solution (a digital output) to a problem on the basis of analog input information. The problems to be solved must be formulated in terms of desired optima, often subject to constraints. The general principles involved in constructing networks to solve specific problems are discussed. Results of computer simulations of a network designed to solve a difficult but well-defined optimization problem-the Traveling-Salesman Problem-are presented and used to illustrate the computational power of the networks. Good solutions to this problem are collectively computed within an elapsed time of only a few neural time constants. The effectiveness of the computation involves both the nonlinear analog response of the neurons and the large connectivity among them. Dedicated networks of biological or microelectronic neurons could provide the computational capabilities described for a wide class of problems having combinatorial complexity. The power and speed naturally displayed by such collective networks may contribute to the effectiveness of biological information processing.

Neural computation of decisions in optimization problems

The theory of deterministic sequencing and scheduling has expanded rapidly during the past years. In this paper we survey the state of the art with respect to optimization and approximation algorithms and interpret these in terms of computational complexity theory. Special cases considered are single machine scheduling, identical, uniform and unrelated parallel machine scheduling, and open shop, flow shop and job shop scheduling. We indicate some problems for future research and include a selective bibliography.

/pdf/optimization-and-approximation-in-deterministic-sequencing-327pfewdb6.pdf

Optimization and Approximation in Deterministic Sequencing and Scheduling: a Survey

Combinatorial optimization: networks and matroids

The essential features of the branch-and-bound approach to constrained optimization are described, and several specific applications are reviewed. These include integer linear programming Land-Doig and Balas methods, nonlinear programming minimization of nonconvex objective functions, the traveling-salesman problem Eastman and Little, et al. methods, and the quadratic assignment problem Gilmore and Lawler methods. Computational considerations, including trade-offs between length of computation and storage requirements, are discussed and a comparison with dynamic programming is made. Various applications outside the domain of mathematical programming are also mentioned.

https://www.jstor.org/stable/pdfplus/168733.pdf

Branch-and-Bound Methods: A Survey

When a two-dimensional (2D) traveling salesman problem (TSP) is presented on a computer screen, human subjects can produce near-optimal tours in linear time. In this study we tested human performance on a real and virtual floor, as well as in a threedimensional (3D) virtual space. Human performance on the real floor is as good as that on a computer screen. Performance on a virtual floor is very similar, while that in a 3D space is slightly but systematically worse. We modeled these results by a graph pyramid algorithm. The same algorithm can account for the results with 2D and 3D problems, which suggests that deterioration of performance in the 3D space can be attributed to geometrical relations between hierarchical clustering in a 3D space and coarse-to-fine production of a tour.

The traveling salesman problem

History (A. Hoffman and P. Wolfe). Motivation and Modeling (R. Garfinkel). Computational Complexity (D. Johnson and C. Papadimitriou). Well-Solved Special Cases (P. Gilmore, et al.). Performance Guarantees for Heuristics (D. Johnson and C. Papadimitriou). Probabilistic Analysis of Heuristics (R. Karp and J. Steele). Empirical Analysis of Heuristics (B. Golden and W. Stewart). Polyhedral Theory (M. Grotschel and M. Padberg). Polyhedral Algorithms (M. Padberg and M. Grotschel). Branch and Bound Methods (E. Balas and P. Toth). Hamiltonian Cycles (V. Chvatal). Vehicle Routing (N. Christofides). Bibliography.

Eugene L. Lawler

Papers

Optimization and Approximation in Deterministic Sequencing and Scheduling: a Survey

Combinatorial optimization: networks and matroids

Branch-and-Bound Methods: A Survey

The traveling salesman problem

The Traveling Salesman Problem: A Guided Tour of Combinatorial Optimization