Showing papers by "Santa Fe Institute published in 1996"

An Introduction to Genetic Algorithms

Abstract: From the Publisher: "This is the best general book on Genetic Algorithms written to date. It covers background, history, and motivation; it selects important, informative examples of applications and discusses the use of Genetic Algorithms in scientific models; and it gives a good account of the status of the theory of Genetic Algorithms. Best of all the book presents its material in clear, straightforward, felicitous prose, accessible to anyone with a college-level scientific background. If you want a broad, solid understanding of Genetic Algorithms -- where they came from, what's being done with them, and where they are going -- this is the book. -- John H. Holland, Professor, Computer Science and Engineering, and Professor of Psychology, The University of Michigan; External Professor, the Santa Fe Institute. Genetic algorithms have been used in science and engineering as adaptive algorithms for solving practical problems and as computational models of natural evolutionary systems. This brief, accessible introduction describes some of the most interesting research in the field and also enables readers to implement and experiment with genetic algorithms on their own. It focuses in depth on a small set of important and interesting topics -- particularly in machine learning, scientific modeling, and artificial life -- and reviews a broad span of research, including the work of Mitchell and her colleagues. The descriptions of applications and modeling projects stretch beyond the strict boundaries of computer science to include dynamical systems theory, game theory, molecular biology, ecology, evolutionary biology, and population genetics, underscoring the exciting "general purpose" nature of genetic algorithms as search methods that can be employed across disciplines. An Introduction to Genetic Algorithms is accessible to students and researchers in any scientific discipline. It includes many thought and computer exercises that build on and reinforce the reader's understanding of the text. The first chapter introduces genetic algorithms and their terminology and describes two provocative applications in detail. The second and third chapters look at the use of genetic algorithms in machine learning (computer programs, data analysis and prediction, neural networks) and in scientific models (interactions among learning, evolution, and culture; sexual selection; ecosystems; evolutionary activity). Several approaches to the theory of genetic algorithms are discussed in depth in the fourth chapter. The fifth chapter takes up implementation, and the last chapter poses some currently unanswered questions and surveys prospects for the future of evolutionary computation.

Growing Artificial Societies: Social Science from the Bottom Up

Abstract: How do social structures and group behaviors arise from the interaction of individuals? Growing Artificial Societies approaches this question with cutting-edge computer simulation techniques. Fundamental collective behaviors such as group formation, cultural transmission, combat, and trade are seen to "emerge" from the interaction of individual agents following a few simple rules. In their program, named Sugarscape, Epstein and Axtell begin the development of a "bottom up" social science that is capturing the attention of researchers and commentators alike. The study is part of the 2050 Project, a joint venture of the Santa Fe Institute, the World Resources Institute, and the Brookings Institution. The project is an international effort to identify conditions for a sustainable global system in the next century and to design policies to help achieve such a system.

The lack of a priori distinctions between learning algorithms

Abstract: This is the first of two papers that use off-training set (OTS) error to investigate the assumption-free relationship between learning algorithms. This first paper discusses the senses in which there are no a priori distinctions between learning algorithms. (The second paper discusses the senses in which there are such distinctions.) In this first paper it is shown, loosely speaking, that for any two algorithms A and B, there are “as many” targets (or priors over targets) for which A has lower expected OTS error than B as vice versa, for loss functions like zero-one loss. In particular, this is true if A is cross-validation and B is “anti-cross-validation” (choose the learning algorithm with largest cross-validation error). This paper ends with a discussion of the implications of these results for computational learning theory. It is shown that one cannot say: if empirical misclassification rate is low, the Vapnik-Chervonenkis dimension of your generalizer is small, and the training set is large, then with high probability your OTS error is small. Other implications for “membership queries” algorithms and “punting” algorithms are also discussed.

Growing Artificial Societies: Social Science from the Bottom Up

Abstract: Part I Introduction: "Artificial Society" models life and death on the sugarscape sex, culture and conflict - the emergence of history sugar and spice - trade comes to the sugarscape disease agents a society is born artificial societies versus traditional models artificial societies versus a life toward generative social science - can you grow it?. Part II Life and death on the sugarscape: in the beginning - there was sugar the agents artificial society on the sugarscape wealth and its distribution in the agent population social networks of neighbours migration summary. Part III Sex, culture and conflict - the emergence of history: sexual reproduction cultural processes combat the proto-history. Part IV Sugar and spice - trade comes to the sugarscape: spice - a second commodity trade rules markets of bilateral traders emergent economic networks social computation, emergent computation summary and conclusions. Part V Disease processes: models of disease transmission and immune response immune system response disease transmission digital diseases on the sugarscape disease transmission networks. Part VI Conclusions: summary some extensions of the current model other artificial societies formal analysis of artificial societies generative social science looking ahead. Appendices: software engineering aspects of artificial societies summary of rule notation state-dependence on the welfare function.

The Swarm Simulation System: A Toolkit for Building Multi-Agent Simulations

Abstract: Swarm is a multi-agent software platform for the simulation of complex adaptive systems. In the Swarm system the basic unit of si8mulation is the swarm, a collection of agents executing a schedule of actions. Swarm supports hierarchical modeling approaches whereby agents can be composed of swarms of other agents in nested structures. Swarm provides object oriented libraries of reusable components for building models and analyzing, displaying, and controlling experiments on those models. Swarm is currently available as a beta version in full, free source code form. It requires the GNU C Compiler, Unix, and X Windows. More information about Swarm can be obtained from our web pages, [ Swarm ].

The coevolution of automata in the repeated Prisoner's Dilemma

Abstract: A model of learning and adaptation is used to analyze the coevolution of strategies in the repeated Prisoner's Dilemma game under both perfect and imperfect reporting. Metaplayers submit finite automata strategies and update their choices through an explicit evolutionary process modeled by a genetic algorithm. Using this framework, adaptive strategic choice and the emergence of cooperation are studied through ‘computational experiments’. The results of the analyses indicate that information conditions lead to significant differences among the evolving strategies. Furthermore, they suggest that the general methodology may have much wider applicability to the analysis of adaptation in economic and social systems.

Quantitative Study of the Fixed Threshold Model for the Regulation of Division of Labour in Insect Societies

Abstract: A simple model of regulation of division of labour in insect societies is introduced and studied. Individuals are assumed to respond to task-related stimuli with response thresholds. When the intensity of a particular stimulus exceeds an individual9s response threshold, the individual engages in task performance with high probability, and successful task performance reduces the intensity of the stimulus. If individuals belonging to different (physical or behavioural) castes have different response thresholds, and if thresholds are assumed to remain fixed over the timescales of experiments, this model can account for some observations on ant species of Pheidole (Wilson 1984).

Information measures, effective complexity, and total information

Abstract: This article defines the concept of an information measure and shows how common information measures such as entropy, Shannon information, and algorithmic information content can be combined to solve problems of characterization, inference, and learning for complex systems. Particularly useful quantities are the effective complexity, which is roughly the length of a compact description of the identified regularities of an entity, and total information, which is effective complexity plus an entropy term that measures the information required to describe the random aspects of the entity. Mathematical definitions are given for both quantities and some applications are discussed. In particular, it is pointed out that if one compares different sets of identified regularities of an entity, the ‘best’ set minimizes the total information, and then, subject to that constraint, minimizes the effective complexity; the resulting effective complexity is then in many respects independent of the observer. © 1996 John Wiley & Sons, Inc.

Perspective metazoan complexity and evolution: is there a trend?

Abstract: The notion that complexity increases in evolution is widely accepted, but the best-known evidence is highly impressionistic. Here I propose a scheme for understanding complexity that provides a conceptual basis for objective measurement. The scheme also shows complexity to be a broad term covering four independent types. For each type, I describe some of the measures that have been devised and review the evidence for trends in the maximum and mean. In metazoans as a whole, there is good evidence only for an early-Phanerozoic trend, and only in one type of complexity. For each of the other types, some trends have been documented, but only in a small number of metazoan subgroups.

Landscapes and their correlation functions

Abstract: Fitness landscapes are an important concept in molecular evolution. Many important examples of landscapes in physics and combinatorial optimization, which are widely used as model landscapes in simulations of molecular evolution and adaptation, are “elementary”, i.e., they are (up to an additive constant) eigenfunctions of a graph Laplacian. It is shown that elementary landscapes are characterized by their correlation functions. The correlation functions are in turn uniquely determined by the geometry of the underlying configuration space and the nearest neighbor correlation of the elementary landscape. Two types of correlation functions are investigated here: the correlation of a time series sampled along a random walk on the landscape and the correlation function with respect to a partition of the set of all vertex pairs.

Nonlinearity in normal human EEG: cycles, temporal asymmetry, nonstationarity and randomness, not chaos

Abstract: Two-hour vigilance and sleep electroencephalogram (EEG) recordings from five healthy volunteers were analyzed using a method for identifying nonlinearity and chaos which combines the redundancy–linear redundancy approach with the surrogate data technique. A nonlinear component in the EEG was detected, however, inconsistent with the hypothesis of low-dimensional chaos. A possibility that a temporally asymmetric process may underlie or influence the EEG dynamics was indicated. A process that merges nonstationary nonlinear deterministic oscillations with randomness is proposed for an explanation of observed properties of the analyzed EEG signals. Taking these results into consideration, the use of dimensional and related chaos-based algorithms in quantitative EEG analysis is critically discussed.

Adaptive Individuals In Evolving Populations: Models And Algorithms

Abstract: * Introduction R.K. Belew and M. Mitchell Biology * Overview * Adaptive Computation in Ecology and Evolution: A Guide to Future Research J. Roughgarden, A. Bergman, S. Shafir, and C. Taylor Reprinted Classics * The Classics in Their Context, and in Ours J. Schull * Of the Influence of the Environment on the Activities and Habits of Animals, and the Influence of the Activities and Habits of These Living Bodies in Modifying Their Organization and Structure J.B. Lamarck * A New Factor in Evolution J.M. Baldwin * On Modification and Variation C. Lloyd Morgan * Canalization of Development and the Inheritance of Acquired Characters C.H. Waddington * The Baldwin Effect G.G. Simpson * The Role of Somatic Change in Evolution G. Bateson New Work * A Model of Individual Adaptive Behavior in a Fluctuating Environment L. A. Zhivotovsky, A. Bergman, and M. W. Feldman * The Baldwin Effect in the Immune System: Learning by Somatic Hypermutation R. Hightower, S. Forrest, and A. S. Perelson * The Effect of Memory Length on Individual Fitness in a Lizard S. Shafir and J. Roughgarden * Latent Energy Environments F. Menczer and R. K. Belew Psychology * Overview * The Causes and Effects of Evolutionary Simulation in the Behavioral Sciences P.M. Todd Reprinted Classics * Excerpts from Principles of Biology H. Spencer * Excerpts from Principles of Psychology H. Spencer * William James and the Broader Implications of a Multilevel Selectionism J. Schull * Excerpts from The Phylogeny and Ontogeny of Behavior B.F. Skinner * Excerpts from Adaptation and Intelligence: Organic Selection and Phenocopy J. Piaget * Selective Costs and Benefits of in the Evolution of Learning T. D. Johnston New Work * Sexual Selection and the Evolution of Learning P. M. Todd * Discontinuity in Evolution: How Different Levels of Organization Imply Preadaptation O. Miglino, S. Nolfi, and D. Parisi * The Influence of Learning on Evolution D. Parisi and S. Nolfi Computer Science * Overview * Computation and the Natural Sciences R. K. Belew, M. Mitchell, and D. H. Ackley Reprinted Classics * How Learning Can Guide Evolution G. E. Hinton and S. J. Nowlan * Natural Selection: When Learning Guides Evolution J. Maynard Smith New Work * Simulations Combining Evolution and Learning M. L. Littman * Optimization with Genetic Algorithm Hybrids that Use Local Searches W. E. Hart and R. K. Belew

Analysis of RNA sequence structure maps by exhaustive enumeration II. Structures of neutral networks and shape space covering

Abstract: The relations between RNA sequences and secondary structures are investigated by exhaustive folding of allGC andAU sequences with chain lengths up to 30. The technique oftries is used for economic data storage and fast retrieval of information. The computed structural data are evaluated through exhaustive enumeration and used as an exact reference for testing analytical results derived from mathematical models and sampling based on statistical methods. Several new concepts of RNA sequence to secondary structure mappings are investigated, among them the structure ofneutral networks (being sets of RNA sequences folding into the same structure),percolation of sequence space by neutral networks, and the principle ofshape space covering. The data of exhaustive enumeration are compared to the analytical results of arandom graph model that reveals the generic properties of sequence to structure mappings based on some base pairing logic. The differences between the numerical and the analytical results are interpreted in terms of specific biophysical properties of RNA molecules.

Rational behavior and economic theory

Abstract: Nowadays, it seems almost universally presumed that the fundamental characteristic of homo oeconomicus is his rationality. We analyze the role played by the concept of rationality in economic theory, and demonstrate that it is necessarily constrained to be an essentially contentless notion. We show that the main body of economic theory is firmly grounded, and that some contrasting approaches to rationality, although leading to heated debates and vivid confusion, have no fundamental significance for economics. With a refereshed view on the essence of economics, we argue that the principles of economic theory form an essential methodological guide for the emergent line of research based on the use of so-called ‘evolutive’ models.

The Barrier of Objects: From Dynamical Systems to Bounded Organizations

Abstract: ion Ax ∪ {x : τ ′} e : τ A λx.e : τ ′ → τ Let A e : σ Ax ∪ {x : σ} e′ : τ A (let x = e in e′) : τ The meaning of the Taut-rule is simply that a free variable has the type assigned to it in the boundary condition. If there is no assignment the expression x is not typable, and is barred from the universe. The meaning of rule App is also clear. It is useful, however, to know how the rule is implemented, since it introduces an important concept. Suppose that we have an object e whose type has been established to be σ, and that we want to apply it to an object e′ of type τ ′. For this to be possible e must have a type of the form τ ′ → τ where τ stands for a generic unknown type of (e)e′ that needs to be determined. Hence, for the interaction (e)e′ to be possible e’s established type σ and the required type τ ′ → τ must be made equal. This may be possible, since σ and τ ′ may contain type variables which can be made more specific in order to satisfy the equality. This means we must look for some type substitution T of the free variables in σ and in τ ′ → τ such that Tσ = T (τ ′ → τ). T is called a unifier, and the procedure for finding T is called unification. It boils down to solving a set of equations. For details about how this procedure is carried out the reader is referred to any standard textbook on type theory. The point is that a

Analysis of RNA sequence structure maps by exhaustive enumeration I. Neutral networks

Abstract: Global relations between RNA sequences and secondary structures are understood as mappings from sequence space into shape space. These mappings are investigated by exhaustive folding of allGC andAU sequences with chain lengths up to 30. The computed structural data are evaluated through exhaustive enumeration and used as an exact reference for testing analytical results derived from mathematical models and sampling based on statistical methods. Several new concepts of RNA sequence to secondary structure mappings are investigated, among them that ofneutral networks (being sets of sequences folding into the same structure). Exhaustive enumeration allows to test several previously suggested relations: the number of (minimum free energy) secondary structures as a function of the chain length as well as the frequency distribution of structures at constant chain length (commonly resulting in generalized forms ofZipf's law).

The existence of a priori distinctions between learning algorithms

Abstract: This is the second of two papers that use off-training set (OTS) error to investigate the assumption-free relationship between learning algorithms. The first paper discusses a particular set of ways to compare learning algorithms, according to which there are no distinctions between learning algorithms. This second paper concentrates on different ways of comparing learning algorithms from those used in the first paper. In particular this second paper discusses the associated a priori distinctions that do exist between learning algorithms. In this second paper it is shown, loosely speaking, that for loss functions other than zero-one (e.g., quadratic loss), there are a priori distinctions between algorithms. However, even for such loss functions, it is shown here that any algorithm is equivalent on average to its “randomized” version, and in this still has no first principles justification in terms of average error. Nonetheless, as this paper discusses, it may be that (for example) cross-validation has better head-to-head minimax properties than “anti-cross-validation” (choose the learning algorithm with the largest cross-validation error). This may be true even for zero-one loss, a loss function for which the notion of “randomization” would not be relevant. This paper also analyzes averages over hypotheses rather than targets. Such analyses hold for all possible priors over targets. Accordingly they prove, as a particular example, that cross-validation cannot be justified as a Bayesian procedure. In fact, for a very natural restriction of the class of learning algorithms, one should use anti-cross-validation rather than cross-validation (!).

A measure of landscapes

Abstract: This paper introduces a statistical fitness landscape analysis, based on Weinberger's random walk method and on a time series analysis known as the Box-Jenkins approach, to measure and express the correlation structure of fitness landscapes. The analysis has some additions to and advantages over previous methods for measuring this structure. The analysis is demonstrated on fitness landscapes constructed with Kauffman's NK-model, using two operators (point mutation and a form of crossover) and a combination of the two. Furthermore, the predictive value of the method is shown.

Self-Organized Criticality, Evolution and the Fossil Extinction Record

Abstract: Statistical analysis indicates that the fossil extinction record is compatible with a distribution of extinction events whose frequency is related to their size by a power law with exponent т ≈ 2. This result is in agreement with predictions based on self-organized critical models of extinction, and might well be taken as evidence of critical behaviour in terrestrial evolution. We argue however that there is a much simpler explanation for the appearance of a power law in terms of extinctions caused by stresses (either biotic or abiotic) to which species are subjected by their environment. We give an explicit model of this process and discuss its properties and implications for the interpretation of the fossil record.

Life Without Death is P-complete

Abstract: It is shown that if a cellular automaton (CA) in two or more dimensions supports growing "ladders" which can turn or block each other, then it can express arbitrary boolean circuits. Thus the problem of predicting the CA for a finite amount of time becomes P-complete, the question of whether a finite configuration grows to infinity is P-hard, and the long-term behavior of initial conditions with a periodic background is undecidable. This class includes the "Life Without Death" rule, in which cells turn on if exactly three of their neighbors are on, and never turn o!.

Individual adaptation in a path-based simulation of the freeway network of Northrhine-Westfalia

Abstract: Traffic simulations are made more realistic by giving individual drivers intentions, i.e., an idea of where they want to go. One possible implementation of this idea is to give each driver an exact pre-computed path, that is, a sequence of roads this driver wants to follow. This paper shows, in a realistic road network, how repeated simulations can be used so that drivers can explore different paths, and how macroscopic quantities such as locations of jams or network throughput change as a result of this.

Canonical approximation of fitness landscapes

Abstract: We present a method for approximating a fitness landscape as a superposition of “elementary” landscapes. Given a correlation function of the landscape in question we show that the relative amplitudes of contributions with p-ary interactions can be computed. We show an application to RNA free energy landscapes. © 1996 John Wiley & Sons, Inc.