Showing papers presented at "Simulation of Adaptive Behavior in 1998"

Learning to perceive the world as articulated: an approach for hierarchical learning in sensory-motor systems

Abstract: This paper describes how agents can learn an internal model of the world structurally by focusing on the problem of behavior-based articulation. We develop an on-line learning scheme-the so-called mixture of recurrent neural net (RNN) experts-in which a set of RNN modules become self-organized as experts on multiple levels, in order to account for the different categories of sensory-motor flow which the robot experiences. Autonomous switching of activated modules in the lower level actually represents the articulation of the sensory-motor flow. In the meantime, a set of RNNs in the higher level competes to learn the sequences of module switching in the lower level, by which articulation at a further, more abstract level can be achieved. The proposed scheme was examined through simulation experiments involving the navigation learning problem. Our dynamical system analysis clarified the mechanism of the articulation. The possible correspondence between the articulation mechanism and the attention switching mechanism in thalamo-cortical loops is also discussed.

Learning hierarchical control structures for multiple tasks and changing environments

Abstract: While the need for hierarchies within control systems is apparent, it is also clear to many researchers that such hierarchies should be learned. Learning both the structure and the component behaviors is a di cult task. The bene t of learning the hierarchical structures of behaviors is that the decomposition of the control structure into smaller transportable chunks allows previously learned knowledge to be applied to new but related tasks. Presented in this paper are improvements to Nested Q-learning (NQL) that allow more realistic learning of control hierarchies in reinforcement environments. Also presented is a simulation of a simple robot performing a series of related tasks that is used to compare both hierarchical and non-hierarchal learning techniques.

Efficient model-based exploration

Abstract: Model-Based Reinforcement Learning (MBRL) can greatly profit from using world models for estimating the consequences of selecting particular actions: an animat can construct such a model from its experiences and use it for computing rewarding behavior. We study the problem of collecting useful experiences through exploration in stochastic environments. Towards this end we use MBRL to maximize exploration rewards (in addition to environmental rewards) for visits of states that promise information gain. We also combine MBRL and the Interval Estimation algorithm (Kaelbling, 1993). Experimental results demonstrate the advantages of our approaches.

Animat navigation using a cognitive graph

Abstract: This article describes a computational model of the hippocampus that makes it possible for a simulated rat to navigate in a continuous environment containing obstacles. This model views the hippocampus as a “cognitive graph”, that is, a hetero-associative network that learns temporal sequences of visited places and stores a topological representation of the environment. Calling upon place cells, head direction cells, and “goal cells”, it suggests a biologically plausible way of exploiting such a spatial representation for navigation that does not require complicated graph-search algorithms. Moreover, it permits “latent learning” during exploration, that is, the building of a spatial representation without the need of any reinforcement. When the rat occasionally discovers some rewarding place it may wish to rejoin subsequently, it simply records within its cognitive graph, through a series of goal and sub-goal cells, the direction in which to move from any given start place. Accordingly, the model implements a simple “place-recognition-triggered response” navigation strategy. Two implementations of place cell management are studied in parallel. The first one associates place cells with place fields that are given a priori and that are uniformly distributed in the environment. The second one dynamically recruits place cells as exploration proceeds and adjusts the density of such cells to the local complexity of the environment. Both implementations lead to identical results. The article ends with a few predictions about results to be expected in experiments involving simultaneous recordings of multiple cells in the rat hippocampus.

Collective sorting and segregation in robots with minimal sensing

Abstract: Many ants sort their brood so that brood items at similar stages of development are grouped together, and separated from items at different stages of development. Brood items are moved individually. The only model proposed to date assumes that ants engaged in brood sorting can sense both the type of element they are carrying, and the local spatial density of that type of element; the model was demonstrated in simulation using two types of objects. This paper describes a system of simple homogeneous autonomous mobile robots which are able to segregate or sort two types of physically identical objects differing only in colour, yet can sense only the colour of the object they are carrying, and have no capacity for spatial orientation or memory. This shows that this sorting problem can be solved by agents simpler than was previously supposed possible.

Competitive co-evolutionary robotics: from theory to practice

Abstract: It is argued that competitive co-evolution is a viable methodology for developing truly autonomous and intelligent machines capable of setting their own goals in order to face new and continuously changing challenges. The paper starts giving an introduction to the dynamics of competitive co-evolutionary systems and reviews their relevance from a computational perspective. The method is then applied to two mobile robots, a predator and a prey, which quickly and autonomously develop efficient chase and evasion strategies. The results are then explained and put in a long-term framework resorting to a visualization of the Red Queen effect on the fitness landscape. Finally, comparative data on different selection criteria are used to indicate that co-evolution does not optimize "intuitive" objective criteria.

Computational models of classical conditioning: a comparative study

Abstract: We describe computer simulation of a number of associative models of classical conditioning in an attempt to assess the strengths and weaknesses of each model. The behavior of the Sutton-Barto model, the TD model, the Klopf model, the Balkenius model and the Schmajuk-DiCarlo model are investigated in a number of simple learning situations. All models are shown to have problems explaining some of the available data from animal experiments. The ISI curves for trace and delay conditioning for all the models are presented together with simulations of acquisition and extinction, reacquisition, blocking, conditioned inhibition, secondary conditioning and facilitation by an intermittent stimulus. We also present cases where some of the models show an unexpected behavior.

From lampreys to salamanders: evolving neural controllers for swimming and walking

Abstract: FromlampreystosalamandersevolvingneuralcontrollersforswimmingandwalkingAukeJanIjsp eertJohnHallamDavidWillshawDeptof Articial Intelligence U ofEdinburghForrest Hill EdinburghEHQ UKfaukei johngdaiedacukCentreforCognitive ScienceUof EdinburghBuccleuchPlace EdinburghEHLWUKdavidcnsedacukAbstractThis pap er presents how neural swimming controllersfor a lamprey can b e adapted for controlling b oth theswimmingandthewalkingofasalamanderlikeanimatUsingaGeneticAlgorithmGAweextendaconnectionistmo delofthebiologicalCentralPattern Generator CPG controlling the swimming of alamprey Ekeb erg to control the lo comotion ofa D mechanical simulation of a salamanderWerstsummarizeexp erimentsontheevolutionofalternativeswimmingcontrollersforthelampreyIjsp eertet alTheaimofthatworkastostudy whether there exists other neural congurationsthanthatfoundin thelampreywhichcouldcontrolswimmingwiththesamee ciencyandtodevelopametho dfordevelopingneurallo comotioncontrollers using a GA We then presenthow that metho dnamelyastagedevolutionoftheneuralconguration of a connectionist mo del can b e used to extendswimming controllerstocontrol b oth swimming andwalking Controllers which similarly to CPGs of animals canpro duce complex oscillationswhen receivingsimple excitatory signals are thus develop ed In particular we generate a controller which can switch fromswimming to walking and pro duce dierent sp eeds ofmotion dep ending on its excitationIntro ductionAlthoughthereexistsavarietyofdierentlo comotormovements among vertebrates their control systems appearrelativelysimilaracrossdierentsp ecies Grillneret al Lo comotion is controlled by networks of interneurons lo cated in the spinal cord which pro duce thecomplex patterns of oscillations necessary for motion Asdemonstrated in the s byOrlovsky s classical exp eriment on the cat as rep orted in Grillner thesenetworks called Central Pattern Generators needonlysimple excitatory signals from the brain to pro duce theco ordinated oscillations necessary for motionOur research fo cusses on biologically inspired neurallo comotion controllers for autonomous agents and howtodevelop them using a GeneticAlgorithm GA In particular we are interested in which kinds of neural conguration can control the anguiliform swimming observedin lampreys and the swimming and walking of salamanders Both lampreys and salamanders pro ject themselvesin water by undulation of the b o dy without use of nsor limbs The undulation is a traveling wae propagatingfromheadtotailWhenthesalamander switchesswimmingtowalking itstrunkceases topropagate anundulation and instead p erforms an S shap ed standingwave with theno des atthe level ofthe girdles Frolichand Biewener The b ending of the trunk helps thesalamander to increase thereach of its limbs which areattached laterally to the trunkThe work presented here follows exp eriments in whichwe used a GAtoevolve swimming controllers for a simulatedlampreywithalternativeneuralcongurationstotheobservedbiological conguration Ijsp eertet al The work was inspired by a biological connectionistmo delofthelamprey sCPGEkeb erg Wedevelop ed articial controllers using an incremental approach whereincreasingly complexcontrol mechanismsare develop ed using elements of theprevious evolutionarystageHereasthemetho dprovedtobeausefuldesigntechniqueweapplyitagaintostudywhethercontrollersforswimmingcanbeextendedtotrolb oth swimming and walking of a salamanderlike animatWalking controllers based on oscillatory networks of theswimming controllers are generated for the control of adimensional mechanical simulation of the salamanderWeare interested in the development of control mechanismswhichsimilarlytotheCPGsofanimalscanpro ducecomplexoscillationswhenreceivingsimpleinputsignals Ap otentialapplication ofthese controllerscould b e the control of lo comotion of an amphibian rob ot Another motivation is to gain some insights on howCPGs function in real animals Note that the evolutionfrom swimming to walking controllers and its applicationto rob otics has b een studied b efore Lewis but at

Experiments on human-robot communication with Robota, an imitative learning and communicating doll robot

Abstract: Note: Also published electronically in full, as part of the Centre for Policy Modelling technical report series: No. CPM-98-38. Reference LSA3-CONF-1998-001 URL: http://www.isab.org/confs/sab98.php Record created on 2005-11-16, modified on 2017-05-10

Robot sheepdog project achieves automatic flock control

Abstract: This paper describes a mobile robot that can enter a circular arena, gather a ock of ducks and manouvre them safely to a speci ed goal position. A minimal simulation model of the ducks' ocking behaviour was developed and used as a tool to guide the design of a general ock-control algorithm. The algorithm was rst tested in simulation then tranferred unchanged to a physical robot which succeeds in gathering a real ock of ducks. This is the rst example of a robot system that exploits and controls an animal's behaviour to achieve a useful task. Robots successfully manipulate many objects in factories and laboratories. Research continues on manipulating objects with complex or variable shapes and dynamics eg. food products (Juste et al., 1997). Recent work in mobile robotics has focused on `adaptive behaviour' in animals, in order to extend the abilities of robots (Maes, 1990) (Hallam and Hayes, 1994), and to better understand the processes occurring in real creatures (Webb, 1994) (McFarland and Bosser, 1993). The Robot Sheepdog Project examines the robotic manipulation of animals by exploiting their adaptive behaviour. In contrast, previous work combining robots and animals (Trevelyan's robot sheep-shearer (Trevelyan, 1992), Silsoe Research Institute's milking robot (Frost et al., 1993)) has deliberately minimised animal behaviour by physical restraint. We have demonstrated a mobile robot that can enter a circular arena, gather a ock of ducks and manouvre them safely to a speci ed goal position. This is the rst example of a robot system that exploits and controls an animal's behaviour to achieve a useful task. The sheepdog's gather-and-fetch task was chosen because of its familiarity and the strong interaction between the dog, shepherd and ock animals. Using ducks instead of sheep allows us to experiment on a conveniently small scale, in a controlled indoor environment. Duck ocking behaviour is recognised by shepherds as similar to sheep; ducks are often used to train sheepdogs because of their relatively slow movement. Figure 1 Sheepdog with duck ock in Lancashire, 1996. Flocking is considered an adaptive behaviour, as it a ords various advantages in hazard-avoidance, mating and foraging. Models of ocking behaviour exist in the literature and are generally derived from Hamilton's observation that ocking may be produced by the mass action of individual animals, each seeking the proximity of its nearest neighbours (Hamilton, 1971). It was later suggested that this behaviour can be well modelled by an attractive `force' acting between the animals, with the magnitude of the attraction varying with the inverse square of the animals' mutual distance (Partridge, 1982) (Warburton and Lazarus, 1991). It is argued that this relationship represents a linear response to sensory information which itself varies with the inverse square of distance. Similar models have produced realistic computer animations of bird ocks (Reynolds, 1987). These ideas are familiar in robotics, where such po-

Modeling ant navigation with an autonomous agent

Abstract: The ability to navigate in a complex environment is crucial for both animals and robots. Many animals use a combination of strategies to return to significant locations in their environment. For example, the desert ant Gata glyphis is able to explore its desert habitat for hundreds of meters while foraging and return back to its nest precisely and on a straight line. The three main strategies that Gata glyphis is using to accomplish this task are path integration, visual piloting and systematic search (Wehner et al., 1996). In this study, we use the autonomous agents approach (Pfeifer, 1996) to gain additional insights into the navigation behavior of Gata glyphis. Inspired by the insect's navigation system we have constructed mechanisms for path integration and visual piloting that were successfully employed on the mobile robot Sahabot 2. The results of the navigation experiments indicate that a combination of these two mechanisms is capable to guide the agent precisely back to the target position.

Evolving action selection and selective attention without actions, attention, or selection

Abstract: A minimal animat architecture, consisting only of a set of autonomous, direct, and continuously active sensorimotor links, is shown to support a full range of `action selection' phenomena. A genetic algorithm is used to engineer the activation functions supported by these links. No `actions' are `selected' in this model, and the use of artificial evolution means that there is no artificial separation of the problems of `link design' from `link fusion'. Implications are drawn for how the concepts of `action selection' and `selective attention' may relate to the idea of coherence between sensorimotor processes.

Landmark-based navigation for a mobile robot

Abstract: In this paper we present a landmark based navigation mechanism for a mobile robot. The system uses a self-organising mechanism to map the environment as the robot is led around that environment by an operator. Detected landmarks, and their relative position towards each other, are recorded in a map that can subsequently be used to plan and execute paths from the robot's current location to a `goal' location speciied by the user. The main motivation for the research described in this paper is to develop a mobile robot navigation system that is robust (through the use of perceptual landmarks), and allows the robot to plan arbitrary paths within its known environment. The system presented here achieves these objectives.

Emergence of functional modularity in robots

Abstract: The origin and structural and functional significance of modular design in organisms represent an important issue debated in many different disciplines. To be eventually successful in clarifying the evolutionary mechanisms underpinning the emergence of modular design in complex organisms, one should be able to cover all different levels of biological hierarchy. Specifically, one should be able to investigate modularity at the behavioral level the level on which natural selection operates and understand how this level is related to the genetic level – the level at which natural selection works through mutation and recombination. We describe a simulation of the evolution of a population of robots that must execute a complex behavioral task to reproduce. During evolution modular neural networks, which control the robots’ behavior, emerge as a result of genetic duplications. Simulation results show that the stepwise addition of structural units, in this case genetic and neural 'modules', can lead to a matching between specific behaviors and their structural representation, i.e., to functional

Animal-animat coevolution: using the animal population as fitness function

Abstract: We show an artificial world where animals (humans) and animats (software agents) interact in a coevolutionary arms race. The two species each use adaptation schemes of their own. Learning through interaction with humans has been out of reach for evolutionary learning techniques because too many iterations are necessary. Our work demonstrates that the Internet is a new environment where this may be possible through an appropriate setup that creates mutualism, a relationship where human and animat species benefit from their interactions with each other.

Risk sensitive and ambiguity reducing dominance interactions in a virtual laboratory

Abstract: In this paper the effects of dominance interactions on social-spatial structure (i.e. centrality of dominants) and social interaction patterns are studied in a simple virtual world: artificial entities inhabiting it are just grouping and performing dominance interactions in which the effects of winning and losing are self-reinforcing. I compare three strategies of attack. The first is based on the contention that aggression stops once individuals recognize the status of others (‘the Ambiguity Reducing strategy’). The second represents another popular view among ethologists, that attack depends on risks involved. The third version is a kind of control system. Here, entities attack obligatory whomever they encounter. In all three systems a dominance hierarchy develops and an increase in intensity of aggression leads to clearer differentiation of ranks, which has a cascade of effects on frequency of aggression, stability of ranks and spatial structure. Furthermore, a spatial structure with dominants in the center emerges in the control and risk sensitive strategy. In the risk sensitive system, bidirectionality of attack is decreased at higher intensities of aggression. Since the latter is observed in certain primate species too, other patterns that emerged for the risk sensitive strategy may be used as research hypotheses for these species.

Perpetuating evolutionary emergence

Abstract: Perpetuating evolutionary emergence is the key to artificially evolving increasingly complex systems. In order to generate complex entities with adaptive behaviors beyond our manual design capability, longterm incremental evolution with continuing emergence is called for. Purely artificial selection models, such as traditional genetic algorithms, are argued to be fundamentally inadequate for this calling and existing natural selection systems are evaluated. Thus some requirements for perpetuating evolutionary emergence are revealed. A new environment containing simple virtual autonomous organisms has been created to satisfy these requirements. Resulting evolutionary emergent behaviors are reported alongside of their neural correlates. In one example, the collective behavior of one species clearly provides a selective force which is overcome by another species, demonstrating the perpetuation of evolutionary emergence via naturally arising coevolution. 1. Evolutionary emergence Emergence is related to qualitatively novel structures and behaviors which are not reducible to those hierarchically below them. It poses an attractive methodology for tackling Descartes’ Dictum: “how can a designer build a device which outperforms the designer’s specifications?” (Cariani, 1991, page 776). Most importantly, it is necessary for the generation of complex entities with behaviors beyond our manual design capability. Cariani identified the three current tracts of thought on emergence, calling them “computational”, “thermodynamic” and “relative to a model” (Cariani, 1991). Computational emergence is related to the manifestation of new global forms, such as flocking behavior and chaos, from local interactions. Thermodynamic emergence is concerned with issues such as the origins of life, where order emerges from noise. The emergence relative to a model concept deals with situations where observers need to change their model in order to keep up with a system’s behavior. This is close to Steels’ concept of emergence, which refers to ongoing processes which produce results invoking vocabulary not previously involved in the description of the system’s inner components – “new descriptive categories” (Steels, 1994, section 4.1). Evolutionary emergence falls into the ‘emergence relative to a model’ category. Consider a virtual world of organisms that can move, reproduce and kill according to rules sensitive to the presence of other organisms, evolving under natural selection. Should flocking manifest itself in this system, we could classify it as emergent in two senses: firstly in the ‘computational’ sense from the interaction of local rules, flocking being a collective behavior, and secondly in the ‘relative to a model’ sense from the evolution, the behavior being novel to the system. While the first is also relevant to our goal, in that complex adaptive systems will involve such emergence, the second is the key to understanding evolutionary emergence. Harvey’s Species Adaptation Genetic Algorithm (SAGA) theory (Harvey, 1992) provides a framework for incremental evolution, necessary for evolutionary emergence. In this paradigm a population, with possibly just a few tens of members, evolves for many thousands of generations, with gradual changes in genotype information content. Increases in complexity must therefore result from evolution itself. This is in contrast to the common use of the Genetic Programming (GP) paradigm, where a population of millions may be evolved for less than a hundred generations (Harvey, 1997, section 5). In the GP case, recombination effectively mixes the random initial population, exhausting variation in few generations. Because (genetic codings of) computer programs result in rugged fitness landscapes, there can be little further evolution of this converged population. Here we see one of the requirements of SAGA: a smooth fitness landscape. Having specified what is meant by evolutionary emergence, we will now explore the two types of selection which might be used to bring evolutionary emergence about. Packard referred to these as “extrinsic adaptation, where evolution is governed by a specified fitness function, and intrinsic adaptation, where evolution occurs “automatically” as a result of the dynamics of a system caused by the evolution of many interacting subsystems” (Packard, 1989, abstract). We will refer to them as artificial and natural selection respectively, because the first involves the imposition of an artifice crafted for some cause external to a system beneath it while the second relies solely on the innate dynamics of a system. 2. Artificial selection Within the artificial evolution field, variants of the optimization paradigm have proven fruitful. Even where the concepts of SAGA theory are dominant, practice still holds to the use of fitness functions. But as the complexity of behaviors attempted increases, flaws in the artificial selection approach are appearing. Zaera, Cliff and Bruten’s failed attempts at evolving schooling behavior in artificial ‘fish’ (Zaera et al., 1996) provide an account of the difficulties faced. An extract from the abstract of their paper still yields an excellent summary of the state of artificial selection work within the field: “The problem appears to be due to the difficulty of formulating an evaluation function which captures what schooling is. We argue that formulating an effective fitness evaluation function for use in evolving controllers can be at least as difficult as hand-crafting an effective controller design. Although our paper concentrates on schooling, we believe that this is likely to be a general issue, and is a serious problem which can be expected to be experienced over a variety of problem domains.” Zaera et al. considered possible reasons for their failure. The argument which most convinced them was that real schooling arises through complex interactions, and that their simulations lacked sufficient complexity (Zaera et al., 1996, section 5). They cited two promising works: Reynolds’ evolution of coordinated group motion in ‘prey’ animats pursued by a hard-wired ‘predator’ (Reynolds, 1992), and Rucker’s ‘ecosystem’ model (Rucker, 1993) in which Boid-like animat controllers (or rather their parameters) were evolved. Both of these are moves towards more intrinsic, automatic evolution. The use of coevolutionary models is fast becoming a dominant approach in the adaptive behavior field. This is essentially a response to the problems encountered when trying to use artificial selection to evolve complex behaviors. However, artificial selection has kept its hold so far – most systems still use fitness functions. The reasoning given for imposing coevolution is often that it helps in overcoming problems arising from the use of static fitness landscapes. From the discussion so far, one might assume our argument to be that evolutionary emergence is not possible in a system using artificial selection. This is not quite so, although we do argue that artificial selection is neither sufficient nor necessary. In the context of evolutionary emergence, any artificial selection used constitutes just one of the parts of a system. Artificial selection can only select for that which it is specified to. Therefore anything that emerges during evolution must be due to another aspect of selection, which must in turn be due to the innate dynamics of the system – natural selection. 3. Natural selection As noted in section 1. , genetic codings of computer programs result in rugged fitness landscapes, making them unsuitable for incremental evolution. However, most natural selection work has been program code evolution, following the initial success of ‘Tierra’ (Ray, 1991). 3.1 Natural selection of program code Tierra is a system of self-replicating machine code programs, initialized as a single manually designed selfreplicating program. To make evolution possible, random bit-flipping was imposed on the memory. A degree of artificial selection was imposed by the system deleting the oldest programs in order to free memory, with an added bias against programs that generated error conditions. Tierra was implemented as a virtual computer, allowing Ray to design a machine language with some properties suiting it to evolution. One aspect of this language was that it contained no numeric constants (such as 13). Thus direct memory addressing was not possible. Instead, the manually designed program used consecutive NOP (No-OPeration) instructions which acted as templates that could be found by certain machine code instructions. This ‘addressing by templates’ is how the program determined the points at which to begin and end copying. Another aspect of the system was that computational errors were introduced at random. Such errors could lead to genetic changes by affecting replication. When Tierra was run, various classes of programs evolved. ‘Parasites’ had shed almost half of their code; they replicated by executing the copy loop from neighboring organisms, which could easily be found by template matching instructions as before. Because the parasites depended on their ‘hosts’, they could not displace them and the host and parasite populations entered into Lotka-Volterra population cycles. Ray reported that coevolution occurred as the hosts became immune to the parasites, which overcame these defenses, and so on. ‘Hyper-parasite’ hosts emerged containing instructions that caused a parasite to copy the host rather than the parasite; this could lead to the rapid elimination of parasites. Ray also reported cooperation (symbiosis) in replication followed by ‘cheaters’ (social parasites) which took advantage of the cooperators. The above are examples of ecological adaptations. Another class of adaptations found was “optimizations”. For example, non-parasitic replicators almost a quarter the length of the initial replicator were found, as were programs with ‘unrolled’ copy loops which copi

Coevolving communicative behavior in a linear pursuer-evadergame

Abstract: The pursuer-evader (PE) game is recognized as an important domain in which to study the coevolution of robust adaptive behavior and protean behavior (Miller and Cli , 1994). Nevertheless, the potential of the game is largely unrealized due to methodological hurdles in coevolutionary simulation raised by PE; versions of the game that have optimal solutions (Isaacs, 1965) are closed-ended, while other formulations are opaque with respect to their solution space, for the lack of a rigorous metric of agent behavior. This inability to characterize behavior, in turn, obfuscates coevolutionary dynamics. We present a new formulation of PE that a ords a rigorous measure of agent behavior and system dynamics. The game is moved from the two-dimensional plane to the one-dimensional bitstring; at each time step, the evader generates a bit that the pursuer must simultaneously predict. Because behavior is expressed as a time series, we can employ information theory to provide quantitative analysis of agent activity. Further, this version of PE opens vistas onto the communicative component of pursuit and evasion behavior, providing an open-ended serial communications channel and an open world (via coevolution). Results show that subtle changes to our game determine whether it is open-ended, and profoundly a ect the viability of arms-race dynamics.

MMC- a recurrent neural network which can be used as manipulable body model

Abstract: A recurrent network is proposed which can be used as a manipulable body model to solve different kinematic tasks as the inverse kinematic problem, the direct kinematic problem or any mixed problem. The model may be used for planning a movement, or ,,thinking" , by being uncoupled from the motor output, or it may be used for direct motor control. The network is based on a new type of neuronal network called MMC net which is similar to but shows some essential differences to the Hopfield tlpe network. These are (l) no symmetrical rveights are necessary in the MMC net. (2) Furthermore, no clipping functions are necessary which allows for real valued outputs. (3) No limited number of discrete attractors. but an infinite number of attractors ll'hich form a continuum are possible in the MMC network.

Planification versus sensory-motor conditioning: what are the issues?

Abstract: This paper deals with the problem of linking a planiication level to a sensory-motor level. We discuss the interest of conditioning when the environment is not predictable enough for motivated planiication to work properly. We show how our probabilistic conditioning rule can be used to solve such a problem. We then present a neural implementation of the planiication which consists in linking situation recognition and diiusion the activity on this \cognitive map". We emphasize the diiculty to ground this map to the real world and we propose an architecture which tries to connect planiication level with sensory-motor level. We discuss the necessity to take into account the dynamic in the internal representation used by the planiication. Abstract This paper deals with the problem of linking a plani-cation level to a sensory-motor level. We discuss the interest of conditioning when the environment is not predictable enough for motivated planiication to work properly. We then present a neural implementation of the planiication and we propose an architecture integrating both sensory-motor and planiication level. We particularly discuss the interaction between the two levels.

Robots and representations

Abstract: The design of complex interactive robots inherently yields a form of representation — an interactive form. Interactive representation is, arguably, the foundational form of representation from which all others are derived. It constitutes the emergence of representational truth value for the system itself, a criterion not addressed in current literature. There is a form of representation that arises naturally in the design of complex interactive systems — robots. This form arguably constitutes an emergence of the fundamental form of representation, out of which increasingly complex forms are constructed and derived. Furthermore, this form of representation naturally satisfies an essential metaepistemological criterion for original representation: system detectable truth value. No alternative approach to representation in the current literature addresses this criterion. Recognizing and exploiting the emergence of this form of representation in robotics and dynamic systems is a rich frontier for exploration. In standard artificial intelligence and cognitive science models, inputs are received and processed, and, perhaps, outputs emitted. The critical consideration that arises in robot design is the possibility of a closure of this sequence of process such that robot outputs influence subsequent inputs via the environment, and, therefore, influence subsequent internal states and processes in the robot. That is, the critical consideration is the closure of input, processing, and output to include full interaction, not just action. This simple closure introduces several important possibilities. In particular, possible internal states that might be consequent on some action or course of action can be functionally indicated in the robot. Because such possible consequent states in the robot will depend, in part, on the environment, those states, or some one of those states, may or may not actually occur — the environment may or may not yield the appropriate input(s) in response to the output(s) to induce those indicated states in the robot. If none of those indicated states are entered by the robot, then the indications are false, and are falsified for the robot. The error in such indications is detectable by and for the system itself. In effect, to indicate such internal states as consequent on particular interactions on the part of the robot is to implicitly predicate of that environment whatever properties are sufficient to support those indications. It is to anticipate that the environment will in fact respond as indicated, if the interaction is engaged in. Some environments will possess a sufficiency of those properties, and will yield one of the indicated states, while other environments will not possess such properties, and will not yield any of the indicated states. For those environments that do not yield an indicated state, to set up such an indication is to set up an implicit predication, an anticipation, that is false, and potentially falsifiable by the system (Bickhard, 1993, in press; Bickhard & Terveen, 1995). The possibility of error, and especially of system detectable error, is a fundamental meta-epistemological criterion for representation. Whatever representation is, it must be capable of some sort of truth value. Conversely, something is representation for a particular system only if it is capable of some sort of truth value for that system. This is critical because many states and conditions and phenomena re representational — can have truth value — but only for some user or designer or obser ver outside of the system itself, not for the system itself (Bickhard, 1993; Bickhard & Terveen, 1995). Moderately complex robots, then, naturally involve a form of representation that is representational for the robot, not just for an observer or analyst or designer or user of the robot. This claim generates five questions: 1) How can notions such as ‘indication’ in the above discussion be made good in a functional manner in a robot, without committing a logical circularity by presupposing the very representationality that is allegedly being modeled? 2) Why would it be useful for a robot to have such representations of interactive potentialities? 3) How could such a notion of representation possibly be adequate to “normal” representational and cognitive phenomena such as representation of objects; representation of abstractions, such as numbers; language; perception; rationality; and so on? I will only outline an answer to the first of these questions, referring others to other sources. 4) On what basis would a robot set up such indications? And 5) How does this model of representation relate to contemporary research in artificial intelligence, cognitive science, connectionism, and robotics? My responses to this question too will, obviously, be abbreviated. The Functional Story First, I need to address the question of how interactive representation could be implemented without presupposing representation. All that is needed are some architectural principles adequate to the model that are themselves strictly functional — not representational. This is, in fact, rather simple. The indicated internal outcome states for an interaction function like final states in an automaton recognizer, but for an automaton that emits outputs to an interactive environment (Bickhard, 1980a). The indication of such states can be implemented with pointers — a pointer, say, to some location that will contain a “1” in the state being indicated. This is certainly not the only architecture that will implement the notions required, but it does suffice. To indicate the interaction itself, upon which the indications of final states are based, requires only a pointer to the subsystem — perhaps the subroutine or interactive recognizer — that would engage in those interactions. So, a pointer to a subsystem together with a pointer or pointers to final states associated with that subsystem suffices for the implicit predication of interactive representation, but none of these pointers themselves are or require representation. Insofar as there is representation here, it is genuinely emergent in the architectural organization. The Usefulness of Interactive Representations Choice. Why would it be useful for a robot to have such indications? For two reasons: First, if there are multiple interactions possible in a particular environment, the indicated internal outcomes of those interactions can be used in selecting which interaction to actually engage in (Bickhard, 1997b). A frog seeing a fly might set up indications of the possibility of tongue-flicking-and-eating, while a frog seeing a shadow of a hawk might set up indications of the possibility of jumping in the water. A frog seeing both needs some way to decide, and internal outcome indications provide a basis for such decision — e.g., select the interaction with the indicated outcomes that have the highest priority relative to current set-points. (Note that if the relevant outcomes are presumed to be represented , rather than indicated — as must be the case if the outcomes are considered to be external outcomes in the environment — then there is a circularity involved in using such notions to model representation.) Error . The second reason why such indications might be useful is that they create the possibility of error, and — most importantly — the possibility of the detection of error by the system. Detection of error, in turn, can be useful for guiding heuristics and strategies of interaction, and for evoking and guiding learning processes. Any general form of learning, in fact, requires such system detection of error (Bickhard & Terveen, 1995). In slogan form: Only anticipations can be falsified; therefore only anticipations can be learned. On the Adequacy of Interactive Representation Interactive, or robotic, representati on might seem adequate for the kinds of interactive properties that interactive indications will implicitly predicate of the environment, but there are many other things to be represented that do not prima facie look like interactive properties. To make good on claims of the adequacy of interactive representation as a general form of representation would require a programmatic treatment of many or most of these representational phenomena. There isn’t space to even begin that explication here (see, for example, Bickhard, 1980a, 1980b, 1992, 1993, in press, forthcoming; Bickhard & Campbell, 1992; Bickhard & Richie, 1983; Bickhard & Terveen, 1995; Campbell & Bickhard, 1986, 1992), but I will outline an approach to the interactive representation of physical objects in order to indicate that this is at least a plausible programme. Complexities of Interactive Indications. Before addressing objects per se, I need to outline some forms of complexity that can be involved in interactive indications. The first is that there may be multiple interactive possibilities indicated at a given time. The second is that interactive indications can be conditionalized on each other: interaction A with possible outcome Q might be indicated, and, if A is engaged in and Q is in fact obtained, then interaction B with possible outcome R becomes possible. There are other kinds of complications possible, but branchings and conditionalized iterations of interactive indications will suffice for briefly addressing the problem of object representation. Webs. Branchings and conditional iterations yield the possibility of interactive indications forming potentially complex webs or nets of indications. In effect, the whole of such a web is indicated as currently possible, but actually reaching some parts of the web will be contingent on perhaps many intermediate interactions and outcomes. Objects. Some sub-networks in such a complex web may have two critical properties: 1) A subnet may be closed in the sense that, if any part of it is reachable — possible — then all parts of it are. That is, all points (p

Some armchair worries about wheeled behavior

Abstract: Autonomous agents research aims to understand adaptive behavior by building models that exhibit such behavior. In this paper, I describe some theoretical worries about the tendency to simplify the sensory-motor control of these behaving models as much as possible. Wheeled robots provide a good example of this tendency. The worries derive from the idea that a complex sensory-motor system is a necessary requirement for reliable functional behavior in a natural environment. The evidence on which I base this conviction is three-fold: (a) when animals, but not wheeled robots, move across a surface this results from variable spatio-temporal patterns across a complex neuro-musculo-skeletal system, (b) a theoretical analysis according to which the stability of distal behavior arises as a result of the variability of proximal behavior, and (c) the possibility to interpret the behavior of animals, but not that of wheeled robots, as a process of self-organization. The use of wheeled robots therefore seems to side step a core problem for understanding adaptive behavior.