Showing papers presented at "Simulation of Adaptive Behavior in 1991"

The dynamics of collective sorting robot-like ants and ant-like robots

Abstract: A distributed sorting algorithm, inspired by how ant colonies sort their brood is presented for use by robot teams The robots move randomly, do not communicate have no hierarchical organisation, have no global representation can only perceive objects just in front of them, but can distinguish between objects of two or more types with a certain degree of error The probability that they pick up or put down an object is modulated as a function of how many of the same objects they have met in the recent past This generates a positive feed-back that is sufficient to coordinate the robots' activity, resulting in their sorting the objects into common clusters While less efficient than a hierarchically controlled sorting, this decentralised organisation offers the advantages of simplicity, flexibility and robustness

A possibility for implementing curiosity and boredom in model-building neural controllers

Abstract: This paper introduces a framework for`curious neural controllers' which employ an adaptive world model for goal directed on-line learning. First an on-line reinforcement learning algorithm for autonomousànimats' is described. The algorithm is based on two fully recurrent`self-supervised' continually running networks which learn in parallel. One of the networks learns to represent a complete model of the environmental dynamics and is called thèmodel network'. It provides completècredit assignment paths' into the past for the second network which controls the animats physical actions in a possibly reactive environment. The an-imats goal is to maximize cumulative reinforcement and minimize cumulativèpain'. The algorithm has properties which allow to implement something like the desire to improve the model network's knowledge about the world. This is related to curiosity. It is described how the particular algorithm (as well as similar model-building algorithms) may be augmented by dynamic curiosity and boredom in a natural manner. This may be done by introducing (delayed) reinforcement for actions that increase the model network's knowledge about the world. This in turn requires the model network to model its own ignorance, thus showing a rudimentary form of self-introspective behavior.

The animat path to AI

Abstract: A research methodology is proposed for understanding intelligence through simulation of artificial animals (“animats”) in progressively more challenging environments while retaining characteristics of holism, pragmatism, perception, categorization, and adaptation that are often underrepresented in standard AI approaches to intelligence. It is suggested that basic elements of the methodology should include a theory/taxonomy of environments by which they can be ordered in difficulty—one is offered—and a theory of animat efficiency It is also suggested that the methodology offers a new approach to the problem of perception.

Task differentiation in Polistes wasp colonies: a model for self-organizing groups of robots

Abstract: In this paper we described a simplified model for a case of functional self-organization. It deals with the emergence of a particular form of task assignment and parallel hierarchical organization within a social group which depend basically on the interactions occuning between individuals and with their immediate local suroundings. The task organization within the colony appeared to be a distributed function which does not require the presence of an individualized cenral organizer. We discussed how such elementary processes could potentially be applied in the coordination and self-organization of groups of interacting robots with simple local computational properties to perform a wide range of tasks.

Navigating with a rat brain: a neurobiologically-inspired model for robot spatial representation

Abstract: F. FaureC. FaisstnauerG. HesinaVienna University of Technology, Austriafrancois, faisst, gerd @cg.tuwien.ac.atA. AubelEcole Polytechnique F´´ ed´erale de Lausanne, Switzerlandaubel@lig.di.epfl.chM. EscherMIRALab, Geneve, SwitzerlandMarc.Escher@cui.unige.chF. LabrosseUniversity of Bath, United KingdomF.Labrosse@maths.bath.ac.ukJ.-C. NebelUniversity of Glasgow, United Kingdomjc@dcs.gla.ac.ukJ.-D. GascueliMAGIS, Grenoble, FranceJean-Dominique.Gascuel@imag.frAbstract

Simulation of adaptive behavior in animats: review and prospect

Abstract: Following a general presentation of the numerous means whereby animats — i.e. simulated animals or autonomous robots — are enabled to display adaptive behaviors, various works making use of such means are discussed. This review cites 176 references and is organized into three pans dealing respectively with preprogrammed adaptive behaviors, with learned adaptive behaviors, and with the evolution of these behaviors. A closing section addresses directions in which it would be desirable to see future research oriented, so as to provide something other than proofs of principle or ad hoc solutions to specific problems, however interesting such proofs or solutions may be in their own right.

Challenges for complete creature architectures

Abstract: In recent years there has been a move within the artificial intelligence and robotics communities towards building complete autonomous creatures that operate in the physical world. Certain approaches have proven quite successful, and have caused a re-analysis within the field of artificial intelligence of what components are necessary in the intellectual architecture of such creatures. However nothing built thus far yet comes close the dreams that many people hold dearly. Furthermore there has been quite some criticism of the new approaches for lacking adequate theoretical justification. In this paper we outline some of the more obvious challenges that remain for these new approaches, and suggest new ways of thinking about the tasks ahead, in order to decompose the field into a number of manageable sub-areas that can be used to shape further research.

Evolution and co-evolution of computer programs to control independently-acting agents

Abstract: This paper describes the recently developed "genetic programming" paradigm which genetically breeds populations of computer programs to solve problems. In genetic programming, the individuals in the population are hierarchical computer programs of various sizes and shapes. This paper also extends the genetic programming paradigm to a "co-evolution" algorithm which operates simultaneously on two populations of independently-acting hierarchical computer programs of various sizes and shapes.

Self-improving reactive agents: case studies of reinforcement learning frameworks

Abstract: The purpose of this work is to investigate and evaluate different reinforcement learning frameworks using connectionist networks. I study four frameworks, which are adopted from the ideas developed by Rich Sutton and his colleagues. The four frameworks are based on two learning procedures: the Temporal Difference methods for solving the credit assignment problem, and the backpropagation algorithm for developing appropriate internal representations. Two of them also involve learning a world model and using it to speed learning. To evaluate their performance, I design a dynamic environment and implement different learning agents, using the different frameworks, to survive in it. The environment is nontrivial and nondeterministic. Surprisingly, all of the agents can learn to survive fairly well in a reasonable time frame. This paper describes the learning agents and their performance, and summarizes the learning algorithms and the lessons I learned from this study. This research was supported by NASA under Contract NAGW-1175. The views and conclusions contained in this document are those of the author and should not be interpreted as representing the official policies, either expressed or implied, of NASA.

Learning hill-climbing functions as a strategy for generating behaviors in a mobile robot

Abstract: International Conference on Simulation of Adaptive Behavior Cambridge, MA: MIT Press/Bradford Books, 1991. Learning Hill-Climbing Functions as a Strategy for Generating Behaviors in a Mobile Robot David Pierce Department of Computer Sciences University of Texas at Austin Austin, TX 78712 (dmpierce@cs.utexas.edu) Benjamin Kuipers Department of Computer Sciences University of Texas at Austin Austin, TX 78712 (kuipers@cs.utexas.edu) Abstract We consider the problem of a robot with uninterpreted sensors and e ectors which must learn, in an unknown environment, behaviors (i.e., sequences of actions) which can be taken to achieve a given goal. This general problem involves a learning agent interacting with a reactive environment: the agent produces actions that a ect the environment and in turn receives sensory feedback from the environment. The agent must learn, through experimentation, behaviors that consistently achieve the goal. The di culty lies in the fact that the robot does not know a priori what its sensors mean, nor what e ects its motor apparatus has on the world. We propose a method by which the robot may analyze its sensory information in order to derive (when possible) a function de ned in terms of the sensory data which is maximized at the goal and which is suitable for hillclimbing. Given this function, the robot solves its problem by learning a behavior that maximizes the function thereby resulting in motion to the goal. 1 The credit assignment problem The learning problem addressed in this paper is illustrated in Figure 1. The learning agent, which we are calling a \critter," receives sensory input (vector s) from the world and acts on the world via motor outputs (represented by a, the action vector). In addition, the critter has access to a reward signal, r, by which it knows when it has achieved its goal. (In the experiments discussed later, the reward signal is incorporated into the sense vector for simplicity.) The critter's task is to learn a behavior which reliably achieves the goal. This behavior is a sequence of actions (most likely dependent on the concomitant sequence of sense vectors) which takes the r s a CRITTER

Autonomous agents, AI and chaos theory

Abstract: Agent theory in AI and related disciplines deals with the structure and behaviour of autonomous, intelligent systems, capable of adaptive action to pursue their interests. In this paper it is proposed that a natural reinterpretation of agent-theoretic intentional concepts like knowing, wanting, liking, etc., can be found in process dynamics. This reinterpretation of agent theory serves two purposes. On the one hand we gain a well established mathematical theory which can be used as the formal mathematical interpretation (semantics) of the abstract agent theory. On the other hand, since process dynamics is a theory that can also be applied to physical systems of various kinds, we gain an implementation route for the construction of artificial agents as bundles of processes in machines. The paper is intended as a basis for dialogue with workers in dynamics, AI, ethology and cognitive science.