Three main streams of evolutionary algorithms (EAs), probabilistic optimization algorithms based on the model of natural evolution, are compared in this article: evolution strategies (ESs), evolutionary programming (EP), and genetic algorithms (GAs). The comparison is performed with respect to certain characteristic components of EAs: the representation scheme of object variables, mutation, recombination, and the selection operator. Furthermore, each algorithm is formulated in a high-level notation as an instance of the general, unifying basic algorithm, and the fundamental theoretical results on the algorithms are presented. Finally, after presenting experimental results for three test functions representing a unimodal and a multimodal case as well as a step function with discontinuities, similarities and differences of the algorithms are elaborated, and some hints to open research questions are sketched.

/pdf/an-overview-of-evolutionary-algorithms-for-parameter-2biia1csgr.pdf

An overview of evolutionary algorithms for parameter optimization

Evolutionary computation has started to receive significant attention during the last decade, although the origins can be traced back to the late 1950's. This article surveys the history as well as the current state of this rapidly growing field. We describe the purpose, the general structure, and the working principles of different approaches, including genetic algorithms (GA) (with links to genetic programming (GP) and classifier systems (CS)), evolution strategies (ES), and evolutionary programming (EP) by analysis and comparison of their most important constituents (i.e. representations, variation operators, reproduction, and selection mechanism). Finally, we give a brief overview on the manifold of application domains, although this necessarily must remain incomplete.

/pdf/evolutionary-computation-comments-on-the-history-and-current-2wv5xjjqp7.pdf

Evolutionary computation: comments on the history and current state

Genetic algorithms (GAs) are biologically motivated adaptive systems which have been used, with varying degrees of success, for function optimization. In this study, an abstraction of the basic genetic algorithm, the Equilibrium Genetic Algorithm (EGA), and the GA in turn, are reconsidered within the framework of competitive learning. This new perspective reveals a number of different possibilities for performance improvements. This paper explores population-based incremental learning (PBIL), a method of combining the mechanisms of a generational genetic algorithm with simple competitive learning. The combination of these two methods reveals a tool which is far simpler than a GA, and which out-performs a GA on large set of optimization problems in terms of both speed and accuracy. This paper presents an empirical analysis of where the proposed technique will outperform genetic algorithms, and describes a class of problems in which a genetic algorithm may be able to perform better. Extensions to this algorithm are discussed and analyzed. PBIL and extensions are compared with a standard GA on twelve problems, including standard numerical optimization functions, traditional GA test suite problems, and NP-Complete problems.

/pdf/population-based-incremental-learning-a-method-for-29e7jv0x8h.pdf

Population-Based Incremental Learning: A Method for Integrating Genetic Search Based Function Optimization and Competitive Learning

Different researchers hold different views of what the term meta-learning exactly means. The first part of this paper provides our own perspective view in which the goal is to build self-adaptive learners (i.e. learning algorithms that improve their bias dynamically through experience by accumulating meta-knowledge). The second part provides a survey of meta-learning as reported by the machine-learning literature. We find that, despite different views and research lines, a question remains constant: how can we exploit knowledge about learning (i.e. meta-knowledge) to improve the performance of learning algorithms? Clearly the answer to this question is key to the advancement of the field and continues being the subject of intensive research.

/pdf/a-perspective-view-and-survey-of-meta-learning-3z92vjfsfw.pdf

A perspective view and survey of meta-learning

The paper proposes an algorithm for data mining called Ant-Miner (ant-colony-based data miner). The goal of Ant-Miner is to extract classification rules from data. The algorithm is inspired by both research on the behavior of real ant colonies and some data mining concepts as well as principles. We compare the performance of Ant-Miner with CN2, a well-known data mining algorithm for classification, in six public domain data sets. The results provide evidence that: 1) Ant-Miner is competitive with CN2 with respect to predictive accuracy, and 2) the rule lists discovered by Ant-Miner are considerably simpler (smaller) than those discovered by CN2.

/pdf/data-mining-with-an-ant-colony-optimization-algorithm-1027iub9p4.pdf

Data mining with an ant colony optimization algorithm

In this article, we explore the use of genetic algorithms (GAs) as a key element in the design and implementation of robust concept learning systems. We describe and evaluate a GA-based system called GABIL that continually learns and refines concept classification rules from its interaction with the environment. The use of GAs is motivated by recent studies showing the effects of various forms of bias built into different concept learning systems, resulting in systems that perform well on certain concept classes (generally, those well matched to the biases) and poorly on others. By incorporating a GA as the underlying adaptive search mechanism, we are able to construct a concept learning system that has a simple, unified architecture with several important features. First, the system is surprisingly robust even with minimal bias. Second, the system can be easily extended to incorporate traditional forms of bias found in other concept learning systems. Finally, the architecture of the system encourages explicit representation of such biases and, as a result, provides for an important additional feature: the ability to dynamically adjust system bias. The viability of this approach is illustrated by comparing the performance of GABIL with that of four other more traditional concept learners (AQ14, C4.5, ID5R, and IACL) on a variety of target concepts. We conclude with some observations about the merits of this approach and about possible extensions.

/pdf/using-genetic-algorithms-for-concept-learning-4qctv3r2dr.pdf

Using Genetic Algorithms for Concept Learning

We introduce a novel framework called "artificial physics", which provides distributed control of large collections of agents. The agents react to artificial forces that are motivated by natural physical laws. This framework provides an effective mechanism for achieving self-assembly, fault-tolerance, and self-repair. Examples are shown for various regular geometric configurations of agents. A further example demonstrates that self-assembly via distributed control can also perform distributed computation.

/pdf/using-artificial-physics-to-control-agents-17ma8wg0wd.pdf

Using artificial physics to control agents

In this introduction, we define the term bias as it is used in machine learning systems We motivate the importance of automated methods for evaluating and selecting biases using a framework of bias selection as search in bias and meta-bias spaces Recent research in the field of machine learning bias is summarized

https://link.springer.com/content/pdf/10.1023/A:1022630017346.pdf

Evaluation and Selection of Biases in Machine Learning

Our theoretical understanding of the properties of genetic algorithms (GAs) being used for function optimization (GAFOs) is not as strong as we would like. Traditional schema analysis provides some first order insights, but doesn't capture the non-linear dynamics of the GA search process very well. Markov chain theory has been used primarily for steady state analysis of GAs. In this paper we explore the use of transient Markov chain analysis to model and understand the behavior of finite population GAFOs observed while in transition to steady states. This approach appears to provide new insights into the circumstances under which GAFOs will (will not) perform well. Some preliminary results are presented and an initial evaluation of the merits of this approach is provided.

/pdf/using-markov-chains-to-analyze-gafos-2z04ze43xc.pdf

Using Markov Chains to Analyze GAFOs

The “Conservation Law for Generalization Performance” [Schaffer, 1994] states that for any learning algorithm and bias, “generalization is a zero-sum enterprise.” In this paper we study the law and show that while the law is true, the manner in which the Conservation Law adds up generalization performance over all target concepts, without regard to the probability with which each concept occurs, is relevant only in a uniformly random universe. We then introduce a more meaningful measure of generalization, expected generalization performance. Unlike the Conservation Law's measure of generalization performance (which is, in essence, defined to be zero), expected generalization performance is conserved only when certain symmetric properties hold in our universe. There is no reason to believe, a priori, that such symmetries exist; learning algorithms may well exhibit non-zero (expected) generalization performance.

Diana F. Gordon

Papers

Using Genetic Algorithms for Concept Learning

Using artificial physics to control agents

Evaluation and Selection of Biases in Machine Learning

Using Markov Chains to Analyze GAFOs

For Every Generalization Action, Is There Really an Equal and Opposite Reaction? Analysis of the Conservation Law for Generalization Performance