scispace - formally typeset
Search or ask a question

Showing papers in "IEEE Transactions on Evolutionary Computation in 2005"


Journal ArticleDOI
TL;DR: This paper attempts to provide a comprehensive overview of the related work within a unified framework on addressing different uncertainties in evolutionary computation, which has been scattered in a variety of research areas.
Abstract: Evolutionary algorithms often have to solve optimization problems in the presence of a wide range of uncertainties. Generally, uncertainties in evolutionary computation can be divided into the following four categories. First, the fitness function is noisy. Second, the design variables and/or the environmental parameters may change after optimization, and the quality of the obtained optimal solution should be robust against environmental changes or deviations from the optimal point. Third, the fitness function is approximated, which means that the fitness function suffers from approximation errors. Fourth, the optimum of the problem to be solved changes over time and, thus, the optimizer should be able to track the optimum continuously. In all these cases, additional measures must be taken so that evolutionary algorithms are still able to work satisfactorily. This paper attempts to provide a comprehensive overview of the related work within a unified framework, which has been scattered in a variety of research areas. Existing approaches to addressing different uncertainties are presented and discussed, and the relationship between the different categories of uncertainties are investigated. Finally, topics for future research are suggested.

1,528 citations


Journal ArticleDOI
TL;DR: This paper reviews some works on the application of MAs to well-known combinatorial optimization problems, and places them in a framework defined by a general syntactic model, which provides them with a classification scheme based on a computable index D, which facilitates algorithmic comparisons and suggests areas for future research.
Abstract: The combination of evolutionary algorithms with local search was named "memetic algorithms" (MAs) (Moscato, 1989). These methods are inspired by models of natural systems that combine the evolutionary adaptation of a population with individual learning within the lifetimes of its members. Additionally, MAs are inspired by Richard Dawkin's concept of a meme, which represents a unit of cultural evolution that can exhibit local refinement (Dawkins, 1976). In the case of MA's, "memes" refer to the strategies (e.g., local refinement, perturbation, or constructive methods, etc.) that are employed to improve individuals. In this paper, we review some works on the application of MAs to well-known combinatorial optimization problems, and place them in a framework defined by a general syntactic model. This model provides us with a classification scheme based on a computable index D, which facilitates algorithmic comparisons and suggests areas for future research. Also, by having an abstract model for this class of metaheuristics, it is possible to explore their design space and better understand their behavior from a theoretical standpoint. We illustrate the theoretical and practical relevance of this model and taxonomy for MAs in the context of a discussion of important design issues that must be addressed to produce effective and efficient MAs.

719 citations


Journal ArticleDOI
TL;DR: The proposed approach to solve global nonlinear optimization problems uses a simple diversity mechanism based on allowing infeasible solutions to remain in the population to find the global optimum despite reaching reasonably fast the feasible region of the search space.
Abstract: This work presents a simple multimembered evolution strategy to solve global nonlinear optimization problems. The approach does not require the use of a penalty function. Instead, it uses a simple diversity mechanism based on allowing infeasible solutions to remain in the population. This technique helps the algorithm to find the global optimum despite reaching reasonably fast the feasible region of the search space. A simple feasibility-based comparison mechanism is used to guide the process toward the feasible region of the search space. Also, the initial stepsize of the evolution strategy is reduced in order to perform a finer search and a combined (discrete/intermediate) panmictic recombination technique improves its exploitation capabilities. The approach was tested with a well-known benchmark. The results obtained are very competitive when comparing the proposed approach against other state-of-the art techniques and its computational cost (measured by the number of fitness function evaluations) is lower than the cost required by the other techniques compared.

585 citations


Journal ArticleDOI
TL;DR: This work presents a new algorithm based on the foraging behavior of E. coli bacteria in the authors' intestine to estimate the harmonic components present in power system voltage/current waveforms, presenting the hybrid method.
Abstract: Harmonic estimation for a signal distorted with additive noise has been an area of interest for researchers in many disciplines of science and engineering. This work presents a new algorithm based on the foraging behavior of E. coli bacteria in our intestine to estimate the harmonic components present in power system voltage/current waveforms. The basic foraging strategy is made adaptive, through a Takagi-Sugeno fuzzy scheme, depending on the operating condition to make the convergence faster. Besides, the harmonic estimation is linear in amplitude and nonlinear in phase. As the proposed algorithm does not rely on Newton-like gradient descent methods, this is used for phase estimation whereas the linear least square scheme estimates the amplitude, thereby presenting the hybrid method. The improvement in %error, as well as the processing time compared with the conventional discrete Fourier transform and genetic algorithm method is demonstrated in this paper. Besides, the performance is quite acceptable even in the presence of decaying dc component as well as to change in amplitude and phase angle of harmonic components.

436 citations


Journal ArticleDOI
TL;DR: The real-time Neuroevolution of Augmenting Topologies (rtNEAT) method for evolving increasingly complex artificial neural networks in real time, as a game is being played, makes possible an entirely new genre of video games in which the player trains a team of agents through a series of customized exercises.
Abstract: In most modern video games, character behavior is scripted; no matter how many times the player exploits a weakness, that weakness is never repaired. Yet, if game characters could learn through interacting with the player, behavior could improve as the game is played, keeping it interesting. This paper introduces the real-time Neuroevolution of Augmenting Topologies (rtNEAT) method for evolving increasingly complex artificial neural networks in real time, as a game is being played. The rtNEAT method allows agents to change and improve during the game. In fact, rtNEAT makes possible an entirely new genre of video games in which the player trains a team of agents through a series of customized exercises. To demonstrate this concept, the Neuroevolving Robotic Operatives (NERO) game was built based on rtNEAT. In NERO, the player trains a team of virtual robots for combat against other players' teams. This paper describes results from this novel application of machine learning, and demonstrates that rtNEAT makes possible video games like NERO where agents evolve and adapt in real time. In the future, rtNEAT may allow new kinds of educational and training applications through interactive and adapting games.

417 citations


Journal ArticleDOI
TL;DR: This paper studies static and dynamic decentralized versions of the search model known as cellular genetic algorithm (cGA), in which individuals are located in a specific topology and interact only with their neighbors and concludes that dynamic cGAs have the most desirable behavior among all the evaluated ones in terms of efficiency and accuracy.
Abstract: This paper studies static and dynamic decentralized versions of the search model known as cellular genetic algorithm (cGA), in which individuals are located in a specific topology and interact only with their neighbors. Making changes in the shape of such topology or in the neighborhood may give birth to a high number of algorithmic variants. We perform these changes in a methodological way by tuning the concept of ratio. Since the relationship (ratio) between the topology and the neighborhood shape defines the search selection pressure, we propose to analyze in depth the influence of this ratio on the exploration/exploitation tradeoff. As we will see, it is difficult to decide which ratio is best suited for a given problem. Therefore, we introduce a preprogrammed change of this ratio during the evolution as a possible additional improvement that removes the need of specifying a single ratio. A later refinement will lead us to the first adaptive dynamic kind of cellular models to our knowledge. We conclude that these dynamic cGAs have the most desirable behavior among all the evaluated ones in terms of efficiency and accuracy; we validate our results on a set of seven different problems of considerable complexity in order to better sustain our conclusions.

408 citations


Journal ArticleDOI
TL;DR: This paper elaborate on how the constrained optimization problem requires a balance of exploration and exploitation under different problem scenarios and come to the conclusion that a nondominated ranking between the individuals will help the algorithm explore further, while the elitist scheme will facilitate in exploitation.
Abstract: In this paper, we propose a generic, two-phase framework for solving constrained optimization problems using genetic algorithms. In the first phase of the algorithm, the objective function is completely disregarded and the constrained optimization problem is treated as a constraint satisfaction problem. The genetic search is directed toward minimizing the constraint violation of the solutions and eventually finding a feasible solution. A linear rank-based approach is used to assign fitness values to the individuals. The solution with the least constraint violation is archived as the elite solution in the population. In the second phase, the simultaneous optimization of the objective function and the satisfaction of the constraints are treated as a biobjective optimization problem. We elaborate on how the constrained optimization problem requires a balance of exploration and exploitation under different problem scenarios and come to the conclusion that a nondominated ranking between the individuals will help the algorithm explore further, while the elitist scheme will facilitate in exploitation. We analyze the proposed algorithm under different problem scenarios using Test Case Generator-2 and demonstrate the proposed algorithm's capability to perform well independent of various problem characteristics. In addition, the proposed algorithm performs competitively with the state-of-the-art constraint optimization algorithms on 11 test cases which were widely studied benchmark functions in literature.

309 citations


Journal ArticleDOI
TL;DR: This paper proposes a general framework for designing neural network ensembles by means of cooperative coevolution, and applies the proposed model to ten real-world classification problems of a very different nature from the UCI machine learning repository and proben1 benchmark set.
Abstract: This paper presents a cooperative coevolutive approach for designing neural network ensembles. Cooperative coevolution is a recent paradigm in evolutionary computation that allows the effective modeling of cooperative environments. Although theoretically, a single neural network with a sufficient number of neurons in the hidden layer would suffice to solve any problem, in practice many real-world problems are too hard to construct the appropriate network that solve them. In such problems, neural network ensembles are a successful alternative. Nevertheless, the design of neural network ensembles is a complex task. In this paper, we propose a general framework for designing neural network ensembles by means of cooperative coevolution. The proposed model has two main objectives: first, the improvement of the combination of the trained individual networks; second, the cooperative evolution of such networks, encouraging collaboration among them, instead of a separate training of each network. In order to favor the cooperation of the networks, each network is evaluated throughout the evolutionary process using a multiobjective method. For each network, different objectives are defined, considering not only its performance in the given problem, but also its cooperation with the rest of the networks. In addition, a population of ensembles is evolved, improving the combination of networks and obtaining subsets of networks to form ensembles that perform better than the combination of all the evolved networks. The proposed model is applied to ten real-world classification problems of a very different nature from the UCI machine learning repository and proben1 benchmark set. In all of them the performance of the model is better than the performance of standard ensembles in terms of generalization error. Moreover, the size of the obtained ensembles is also smaller.

239 citations


Journal ArticleDOI
TL;DR: An evolutionary algorithm with guided mutation (EA/G) for the maximum clique problem is proposed in this paper and experimental results show that EA/G outperforms the heuristic genetic algorithm of Marchiori and a MIMIC algorithm on DIMACS benchmark graphs.
Abstract: Estimation of distribution algorithms sample new solutions (offspring) from a probability model which characterizes the distribution of promising solutions in the search space at each generation. The location information of solutions found so far (i.e., the actual positions of these solutions in the search space) is not directly used for generating offspring in most existing estimation of distribution algorithms. This paper introduces a new operator, called guided mutation. Guided mutation generates offspring through combination of global statistical information and the location information of solutions found so far. An evolutionary algorithm with guided mutation (EA/G) for the maximum clique problem is proposed in this paper. Besides guided mutation, EA/G adopts a strategy for searching different search areas in different search phases. Marchiori's heuristic is applied to each new solution to produce a maximal clique in EA/G. Experimental results show that EA/G outperforms the heuristic genetic algorithm of Marchiori (the best evolutionary algorithm reported so far) and a MIMIC algorithm on DIMACS benchmark graphs.

225 citations


Journal ArticleDOI
TL;DR: This paper presents a general framework covering most optimization scenarios and shows that in self-play there are free lunches: in coevolution some algorithms have better performance than other algorithms, averaged across all possible problems.
Abstract: Recent work on the foundational underpinnings of black-box optimization has begun to uncover a rich mathematical structure. In particular, it is now known that an inner product between the optimization algorithm and the distribution of optimization problems likely to be encountered fixes the distribution over likely performances in running that algorithm. One ramification of this is the "No Free Lunch" (NFL) theorems, which state that any two algorithms are equivalent when their performance is averaged across all possible problems. This highlights the need for exploiting problem-specific knowledge to achieve better than random performance. In this paper, we present a general framework covering most optimization scenarios. In addition to the optimization scenarios addressed in the NFL results, this framework covers multiarmed bandit problems and evolution of multiple coevolving players. As a particular instance of the latter, it covers "self-play" problems. In these problems, the set of players work together to produce a champion, who then engages one or more antagonists in a subsequent multiplayer game. In contrast to the traditional optimization case where the NFL results hold, we show that in self-play there are free lunches: in coevolution some algorithms have better performance than other algorithms, averaged across all possible problems. However, in the typical coevolutionary scenarios encountered in biology, where there is no champion, the NFL theorems still hold.

224 citations


Journal ArticleDOI
TL;DR: The experimental results show that the evolutionary sequence design by NACST/Seq outperforms in its reliability the existing sequence design techniques such as conventional EAs, simulated annealing, and specialized heuristic methods.
Abstract: DNA computing relies on biochemical reactions of DNA molecules and may result in incorrect or undesirable computations. Therefore, much work has focused on designing the DNA sequences to make the molecular computation more reliable. Sequence design involves with a number of heterogeneous and conflicting design criteria and traditional optimization methods may face difficulties. In this paper, we formulate the DNA sequence design as a multiobjective optimization problem and solve it using a constrained multiobjective evolutionary algorithm (EA). The method is implemented into the DNA sequence design system, NACST/Seq, with a suite of sequence-analysis tools to help choose the best solutions among many alternatives. The performance of NACST/Seq is compared with other sequence design methods, and analyzed on a traveling salesman problem solved by bio-lab experiments. Our experimental results show that the evolutionary sequence design by NACST/Seq outperforms in its reliability the existing sequence design techniques such as conventional EAs, simulated annealing, and specialized heuristic methods.

Journal ArticleDOI
TL;DR: Although AS/sub i-best/ does not perform as well as other algorithms from the literature for the Hanoi Problem, it successfully finds the known least cost solution for the larger Doubled New York Tunnels Problem.
Abstract: Much research has been carried out on the optimization of water distribution systems (WDSs). Within the last decade, the focus has shifted from the use of traditional optimization methods, such as linear and nonlinear programming, to the use of heuristics derived from nature (HDNs), namely, genetic algorithms, simulated annealing and more recently, ant colony optimization (ACO), an optimization algorithm based on the foraging behavior of ants. HDNs have been seen to perform better than more traditional optimization methods and amongst the HDNs applied to WDS optimization, a recent study found ACO to outperform other HDNs for two well-known case studies. One of the major problems that exists with the use of HDNs, particularly ACO, is that their searching behavior and, hence, performance, is governed by a set of user-selected parameters. Consequently, a large calibration phase is required for successful application to new problems. The aim of this paper is to provide a deeper understanding of ACO parameters and to develop parametric guidelines for the application of ACO to WDS optimization. For the adopted ACO algorithm, called AS/sub i-best/ (as it uses an iteration-best pheromone updating scheme), seven parameters are used: two decision policy control parameters /spl alpha/ and /spl beta/, initial pheromone value /spl tau//sub 0/, pheromone persistence factor /spl rho/, number of ants m, pheromone addition factor Q, and the penalty factor (PEN). Deterministic and semi-deterministic expressions for Q and PEN are developed. For the remaining parameters, a parametric study is performed, from which guidelines for appropriate parameter settings are developed. Based on the use of these heuristics, the performance of AS/sub i-best/ was assessed for two case studies from the literature (the New York Tunnels Problem, and the Hanoi Problem) and an additional larger case study (the Doubled New York Tunnels Problem). The results show that AS/sub i-best/ achieves the best performance presented in the literature, in terms of efficiency and solution quality, for the New York Tunnels Problem. Although AS/sub i-best/ does not perform as well as other algorithms from the literature for the Hanoi Problem (a notably difficult problem), it successfully finds the known least cost solution for the larger Doubled New York Tunnels Problem.

Journal ArticleDOI
TL;DR: The hierarchical RSS-DSS algorithm is introduced for dynamically filtering large datasets based on the concepts of training pattern age and difficulty, while utilizing a data structure to facilitate the efficient use of memory hierarchies.
Abstract: The hierarchical RSS-DSS algorithm is introduced for dynamically filtering large datasets based on the concepts of training pattern age and difficulty, while utilizing a data structure to facilitate the efficient use of memory hierarchies. Such a scheme provides the basis for training genetic programming (GP) on a data set of half a million patterns in 15 min. The method is generic, thus, not specific to a particular GP structure, computing platform, or application context. The method is demonstrated on the real-world KDD-99 intrusion detection data set, resulting in solutions competitive with those identified in the original KDD-99 competition, while only using a fraction of the original features. Parameters of the RSS-DSS algorithm are demonstrated to be effective over a wide range of values. An analysis of different cost functions indicates that hierarchical fitness functions provide the most effective solutions.

Journal ArticleDOI
TL;DR: Improvements are introduced including mutations to the nonlinear simplex method to search around the boundary of the feasible region and to control the convergence speed of the method, the /spl alpha/ constrained method is applied and the improved /splalpha/ constrainedsimplex method is proposed for constrained optimization problems.
Abstract: Constrained optimization problems are very important and frequently appear in the real world. The /spl alpha/ constrained method is a new transformation method for constrained optimization. In this method, a satisfaction level for the constraints is introduced, which indicates how well a search point satisfies the constraints. The /spl alpha/ level comparison, which compares search points based on their level of satisfaction of the constraints, is also introduced. The /spl alpha/ constrained method can convert an algorithm for unconstrained problems into an algorithm for constrained problems by replacing ordinary comparisons with the /spl alpha/ level comparisons. In this paper, we introduce some improvements including mutations to the nonlinear simplex method to search around the boundary of the feasible region and to control the convergence speed of the method, we apply the /spl alpha/ constrained method and we propose the improved /spl alpha/ constrained simplex method for constrained optimization problems. The effectiveness of the /spl alpha/ constrained simplex method is shown by comparing its performance with that of the stochastic ranking method on various constrained problems.

Journal ArticleDOI
TL;DR: In this paper, a coevolutionary algorithm for inferring the topology and parameters of a wide range of hidden nonlinear systems with a minimum of experimentation on the target system is presented.
Abstract: We present a coevolutionary algorithm for inferring the topology and parameters of a wide range of hidden nonlinear systems with a minimum of experimentation on the target system. The algorithm synthesizes an explicit model directly from the observed data produced by intelligently generated tests. The algorithm is composed of two coevolving populations. One population evolves candidate models that estimate the structure of the hidden system. The second population evolves informative tests that either extract new information from the hidden system or elicit desirable behavior from it. The fitness of candidate models is their ability to explain behavior of the target system observed in response to all tests carried out so far; the fitness of candidate tests is their ability to make the models disagree in their predictions. We demonstrate the generality of this estimation-exploration algorithm by applying it to four different problems-grammar induction, gene network inference, evolutionary robotics, and robot damage recovery-and discuss how it overcomes several of the pathologies commonly found in other coevolutionary algorithms. We show that the algorithm is able to successfully infer and/or manipulate highly nonlinear hidden systems using very few tests, and that the benefit of this approach increases as the hidden systems possess more degrees of freedom, or become more biased or unobservable. The algorithm provides a systematic method for posing synthesis or analysis tasks to a coevolutionary system.

Journal ArticleDOI
TL;DR: This paper investigates the domain of competence of XCS by means of a methodology that characterizes the complexity of a classification problem by a set of geometrical descriptors, and focuses on XCS with hyperrectangle codification, which has been predominantly used for real-attributed domains.
Abstract: The XCS classifier system has recently shown a high degree of competence on a variety of data mining problems, but to what kind of problems XCS is well and poorly suited is seldom understood, especially for real-world classification problems. The major inconvenience has been attributed to the difficulty of determining the intrinsic characteristics of real-world classification problems. This paper investigates the domain of competence of XCS by means of a methodology that characterizes the complexity of a classification problem by a set of geometrical descriptors. In a study of 392 classification problems along with their complexity characterization, we are able to identify difficult and easy domains for XCS. We focus on XCS with hyperrectangle codification, which has been predominantly used for real-attributed domains. The results show high correlations between XCS's performance and measures of length of class boundaries, compactness of classes, and nonlinearities of decision boundaries. We also compare the relative performance of XCS with other traditional classifier schemes. Besides confirming the high degree of competence of XCS in these problems, we are able to relate the behavior of the different classifier schemes to the geometrical complexity of the problem. Moreover, the results highlight certain regions of the complexity measurement space where a classifier scheme excels, establishing a first step toward determining the best classifier scheme for a given classification problem.

Journal ArticleDOI
TL;DR: Three different coevolutionary PSO techniques used to evolve playing strategies for the nonzero sum problem of the iterated prisoner's dilemma are presented, with results indicating that NNs cooperate well, but may develop weak strategies that can cause catastrophic collapses.
Abstract: This paper presents and investigates the application of coevolutionary training techniques based on particle swarm optimization (PSO) to evolve playing strategies for the nonzero sum problem of the iterated prisoner's dilemma (IPD). Three different coevolutionary PSO techniques are used, differing in the way that IPD strategies are presented: A neural network (NN) approach in which the NN is used to predict the next action, a binary PSO approach in which the particle represents a complete playing strategy, and finally, a novel approach that exploits the symmetrical structure of man-made strategies. The last technique uses a PSO algorithm as a function approximator to evolve a function that characterizes the dynamics of the IPD. These different PSO approaches are compared experimentally with one another, and with popular man-made strategies. The performance of these approaches is evaluated in both clean and noisy environments. Results indicate that NNs cooperate well, but may develop weak strategies that can cause catastrophic collapses. The binary PSO technique does not have the same deficiency, instead resulting in an overall state of equilibrium in which some strategies are allowed to exploit the population, but never dominate. The symmetry approach is not as successful as the binary PSO approach in maintaining cooperation in both noisy and noiseless environments-exhibiting selfish behavior against the benchmark strategies and depriving them of receiving almost any payoff. Overall, the PSO techniques are successful at generating a variety of strategies for use in the IPD, duplicating and improving on existing evolutionary IPD population observations.

Journal ArticleDOI
TL;DR: Theoretical results are in agreement with experimental values, showing that the selection intensity can be controlled by using different update methods, and it is seen that the usual logistic approximation breaks down for low-dimensional lattices and should be replaced by a polynomial approximation.
Abstract: In this paper, we present quantitative models for the selection pressure of cellular evolutionary algorithms on regular one- and two-dimensional (2-D) lattices. We derive models based on probabilistic difference equations for synchronous and several asynchronous cell update policies. The models are validated using two customary selection methods: binary tournament and linear ranking. Theoretical results are in agreement with experimental values, showing that the selection intensity can be controlled by using different update methods. It is also seen that the usual logistic approximation breaks down for low-dimensional lattices and should be replaced by a polynomial approximation. The dependence of the models on the neighborhood radius is studied for both topologies. We also derive results for 2-D lattices with variable grid axes ratio.

Journal ArticleDOI
TL;DR: This work introduces the concept of competition-balanced system (CBS), which is a property of the combination of an ACO algorithm with a problem instance that may suffer from a bias that leads to second-order deception, and shows that the choice of an appropriate pheromone model is crucial for the success of the ACO algorithms, and it can help avoid second- order deception.
Abstract: One of the problems encountered when applying ant colony optimization (ACO) to combinatorial optimization problems is that the search process is sometimes biased by algorithm features such as the pheromone model and the solution construction process. Sometimes this bias is harmful and results in a decrease in algorithm performance over time, which is called second-order deception. In this work, we study the reasons for the occurrence of second-order deception. In this context, we introduce the concept of competition-balanced system (CBS), which is a property of the combination of an ACO algorithm with a problem instance. We show by means of an example that combinations of ACO algorithms with problem instances that are not CBSs may suffer from a bias that leads to second-order deception. Finally, we show that the choice of an appropriate pheromone model is crucial for the success of the ACO algorithm, and it can help avoid second-order deception.

Journal ArticleDOI
TL;DR: This paper studies the IPD game with both noise and multiple levels of cooperation (intermediate choices) in a coevolutionary environment, where players can learn and adapt their strategies through an evolutionary algorithm.
Abstract: Real-world dilemmas rarely involve just two choices and perfect interactions without mistakes. In the iterated prisoner's dilemma (IPD) game, intermediate choices or mistakes (noise) have been introduced to extend its realism. This paper studies the IPD game with both noise and multiple levels of cooperation (intermediate choices) in a coevolutionary environment, where players can learn and adapt their strategies through an evolutionary algorithm. The impact of noise on the evolution of cooperation is first examined. It is shown that the coevolutionary models presented in this paper are robust against low noise (when mistakes occur with low probability). That is, low levels of noise have little impact on the evolution of cooperation. On the other hand, high noise (when mistakes occur with high probability) creates misunderstandings and discourages cooperation. However, the evolution of cooperation in the IPD with more choices in a coevolutionary learning setting also depends on behavioral diversity. This paper further investigates the issue of behavioral diversity in the coevolution of strategies for the IPD with more choices and noise. The evolution of cooperation is more difficult to achieve if a coevolutionary model with low behavioral diversity is used for IPD games with higher levels of noise. The coevolutionary model with high behavioral diversity in the population is more resistant to noise. It is shown that strategy representations can have a significant impact on the evolutionary outcomes because of different behavioral diversities that they generate. The results further show the importance of behavioral diversity in coevolutionary learning.

Journal ArticleDOI
TL;DR: An agent based computational economics approach for studying the effect of alternative structures and mechanisms on behavior in electricity markets and the potential benefit of an evolutionary economics approach to market modeling is demonstrated.
Abstract: The deregulation of electricity markets has continued apace around the globe. The best structure for deregulated markets is a subject of much debate, and the consequences of poor structural choices can be dramatic. Understanding the effect of structure on behavior is essential, but the traditional economics approaches of field studies and experimental studies are particularly hard to conduct in relation to electricity markets. This paper describes an agent based computational economics approach for studying the effect of alternative structures and mechanisms on behavior in electricity markets. Autonomous adaptive agents, using hierarchical learning classifier systems, learn through competition in a simulated model of the UK market in electricity generation. The complex agent structure was developed through a sequence of experimentation to test whether it was capable of meeting the following requirements: first, that the agents are able to learn optimal strategies when competing against nonadaptive agents; second, that the agents are able to learn strategies observable in the real world when competing against other adaptive agents; and third, that cooperation without explicit communication can evolve in certain market situations. The potential benefit of an evolutionary economics approach to market modeling is demonstrated by examining the effects of alternative payment mechanisms on the behavior of agents.

Journal ArticleDOI
TL;DR: The extension of XCS to gradient-based update methods results in a classifier system that is more robust and more parameter independent, solving large and difficult maze problems reliably.
Abstract: The accuracy-based XCS classifier system has been shown to solve typical data mining problems in a machine-learning competitive way. However, successful applications in multistep problems, modeled by a Markov decision process, were restricted to very small problems. Until now, the temporal difference learning technique in XCS was based on deterministic updates. However, since a prediction is actually generated by a set of rules in XCS and Learning Classifier Systems in general, gradient-based update methods are applicable. The extension of XCS to gradient-based update methods results in a classifier system that is more robust and more parameter independent, solving large and difficult maze problems reliably. Additionally, the extension to gradient methods highlights the relation of XCS to other function approximation methods in reinforcement learning.

Journal ArticleDOI
TL;DR: Through computer simulations, it is demonstrated that small neighborhood structures facilitate the evolution of cooperative behavior under random pairing in game-playing.
Abstract: We discuss the evolution of strategies in a spatial iterated prisoner's dilemma (IPD) game in which each player is located in a cell of a two-dimensional grid-world. Following the concept of structured demes, two neighborhood structures are used. One is for the interaction among players through the IPD game. A player in each cell plays against its neighbors defined by this neighborhood structure. The other is for mating strategies by genetic operations. A new strategy for a player is generated by genetic operations from a pair of parent strings, which are selected from its neighboring cells defined by the second neighborhood structure. After examining the effect of the two neighborhood structures on the evolution of cooperative behavior with standard pairing in game-playing, we introduce a random pairing scheme in which each player plays against a different randomly chosen neighbor at every round (i.e., every iteration) of the game. Through computer simulations, we demonstrate that small neighborhood structures facilitate the evolution of cooperative behavior under random pairing in game-playing.

Journal ArticleDOI
TL;DR: A method for automated synthesis of analog circuits using evolutionary search and a set of circuit design rules based on topological reuse and the design of the evaluation function-which evaluates each generated circuit using SPICE simulations-has been automated to a great extent.
Abstract: We present a method for automated synthesis of analog circuits using evolutionary search and a set of circuit design rules based on topological reuse. The system requires only moderate expert knowledge on part of the user. It allows circuit size, circuit topology, and device values to evolve. The circuit representation scheme employs a topological reuse-based approach-it uses commonly used subcircuits for analog design as inputs and utilizes these to create the final circuit. The connectivity between these blocks is governed by a well-defined set of rules and the scheme is capable of representing most standard analog circuit topologies. The system operation consists of two phases-in the first phase, the circuit size and topology are evolved. A limited amount of device sizing also occurs in this phase. The second phase consists entirely of device value optimization. The design of the evaluation function-which evaluates each generated circuit using SPICE simulations-has also been automated to a great extent. The evaluation function is generated automatically depending on a behavioral description of the circuit. We present several experimental results obtained using this scheme, including two types of comparators, two types of oscillators, and an XOR logic gate. The generated circuits closely resemble hand designed circuits. The computational needs of the system are modest.

Journal ArticleDOI
TL;DR: It is shown how case injection can be used to learn to play better from a human's or system's game-playing experience and the approach to acquiring experience from human players showcases an elegant solution to the knowledge acquisition bottleneck in this domain.
Abstract: We use case-injected genetic algorithms (CIGARs) to learn to competently play computer strategy games. CIGARs periodically inject individuals that were successful in past games into the population of the GA working on the current game, biasing search toward known successful strategies. Computer strategy games are fundamentally resource allocation games characterized by complex long-term dynamics and by imperfect knowledge of the game state. CIGAR plays by extracting and solving the game's underlying resource allocation problems. We show how case injection can be used to learn to play better from a human's or system's game-playing experience and our approach to acquiring experience from human players showcases an elegant solution to the knowledge acquisition bottleneck in this domain. Results show that with an appropriate representation, case injection effectively biases the GA toward producing plans that contain important strategic elements from previously successful strategies.

Journal ArticleDOI
TL;DR: Two learning methods for acquiring position evaluation for small Go boards are studied and compared and it was found that the temporal-difference method learned faster, and in most cases also achieved a higher level of play than coevolution, providing that the gradient descent step size was chosen suitably.
Abstract: Two learning methods for acquiring position evaluation for small Go boards are studied and compared. In each case the function to be learned is a position-weighted piece counter and only the learning method differs. The methods studied are temporal difference learning (TDL) using the self-play gradient-descent method and coevolutionary learning, using an evolution strategy. The two approaches are compared with the hope of gaining a greater insight into the problem of searching for "optimal" zero-sum game strategies. Using tuned standard setups for each algorithm, it was found that the temporal-difference method learned faster, and in most cases also achieved a higher level of play than coevolution, providing that the gradient descent step size was chosen suitably. The performance of the coevolution method was found to be sensitive to the design of the evolutionary algorithm in several respects. Given the right configuration, however, coevolution achieved a higher level of play than TDL. Self-play results in optimal play against a copy of itself. A self-play player will prefer moves from which it is unlikely to lose even when it occasionally makes random exploratory moves. An evolutionary player forced to perform exploratory moves in the same way can achieve superior strategies to those acquired through self-play alone. The reason for this is that the evolutionary player is exposed to more varied game-play, because it plays against a diverse population of players.

Journal ArticleDOI
TL;DR: The results show that many selection methods are inappropriate for finding polymorphic Nash solutions to variable-sum games.
Abstract: We use evolutionary game theory (EGT) to investigate the dynamics and equilibria of selection methods in coevolutionary algorithms. The canonical selection method used in EGT is equivalent to the standard "fitness-proportional" selection method used in evolutionary algorithms. All attractors of the EGT dynamic are Nash equilibria; we focus on simple symmetric variable-sum games that have polymorphic Nash-equilibrium attractors. Against the dynamics of proportional selection, we contrast the behaviors of truncation selection, (/spl mu/,/spl lambda/),(/spl mu/+/spl lambda/), linear ranking, Boltzmann, and tournament selection. Except for Boltzmann selection, each of the methods we test unconditionally fail to achieve polymorphic Nash equilibrium. Instead, we find point attractors that lack game-theoretic justification, cyclic dynamics, or chaos. Boltzmann selection converges onto polymorphic Nash equilibrium only when selection pressure is sufficiently low; otherwise, we obtain attracting limit-cycles or chaos. Coevolutionary algorithms are often used to search for solutions (e.g., Nash equilibria) of games of strategy; our results show that many selection methods are inappropriate for finding polymorphic Nash solutions to variable-sum games. Another application of coevolution is to model other systems; our results emphasize the degree to which the model's behavior is sensitive to implementation details regarding selection-details that we might not otherwise believe to be critical.

Journal ArticleDOI
TL;DR: A one-parameter genetic algorithm (GA) is used to solve the minimum labeling spanning tree problem and clearly outperforms MVCA in computational tests.
Abstract: Given a connected, undirected graph G whose edges are labeled (or colored), the minimum labeling spanning tree (MLST) problem seeks a spanning tree on G with the minimum number of distinct labels (or colors). In recent work, the MLST problem has been shown to be NP-hard and an effective heuristic [maximum vertex covering algorithm (MVCA)] has been proposed and analyzed. We use a one-parameter genetic algorithm (GA) to solve the problem. In computational tests, the GA clearly outperforms MVCA.

Journal ArticleDOI
TL;DR: Results show that neural networks can be evolved as evaluation functions, despite the general difficulties associated with this approach, and a simple spatial preprocessing layer in the neural network that captured spatial information.
Abstract: A study was conducted to find out how game-playing strategies for Othello (also known as reversi) can be learned without expert knowledge. The approach used the coevolution of a fixed-architecture neural-network-based evaluation function combined with a standard minimax search algorithm. Comparisons between evolving neural networks and computer players that used deterministic strategies allowed evolution to be observed in real-time. Neural networks evolved to outperform the computer players playing at higher ply-depths, despite being handicapped by playing black and using minimax at ply-depth of two. In addition, the playing ability of the population progressed from novice, to intermediate, and then to master's level. Individual neural networks discovered various game-playing strategies, starting with positional and later mobility. These results show that neural networks can be evolved as evaluation functions, despite the general difficulties associated with this approach. Success in this case was due to a simple spatial preprocessing layer in the neural network that captured spatial information, self-adaptation of every weight and bias of the neural network, and a selection method that allowed a diverse population of neural networks to be carried forward from one generation to the next.

Journal ArticleDOI
TL;DR: An integrated technique of genetic programming (GP) and reinforcement learning (RL) is proposed to enable a real robot to adapt its actions to a real environment and makes it possible for real robots to learn effective actions.
Abstract: We propose an integrated technique of genetic programming (GP) and reinforcement learning (RL) to enable a real robot to adapt its actions to a real environment. Our technique does not require a precise simulator because learning is achieved through the real robot. In addition, our technique makes it possible for real robots to learn effective actions. Based on this proposed technique, we acquire common programs, using GP, which are applicable to various types of robots. Through this acquired program, we execute RL in a real robot. With our method, the robot can adapt to its own operational characteristics and learn effective actions. In this paper, we show experimental results from two different robots: a four-legged robot "AIBO" and a humanoid robot "HOAP-1." We present results showing that both effectively solved the box-moving task; the end result demonstrates that our proposed technique performs better than the traditional Q-learning method.