Showing papers presented at "Simulation of Adaptive Behavior in 2002"

Inferring Hand Motion from Multi-Cell Recordings in Motor Cortex using a Kalman Filter

Abstract: This paper develops a control-theoretic approach to the problem of decoding neural activity in motor cortex. Our goal is to infer the position and velocity of a subject's hand from the neural spiking activity of 25 cells simultaneously recorded in primary motor cortex. We propose to model the encoding and decoding of the neural data using a Kalman lter. Towards that end we specify a measurement model that assumes the ring rate of a cell within 50ms is a stochastic linear function of position, velocity, and acceleration of the hand. This model is learned from training data along with a system model that encodes how the hand moves. Experimental results show that the reconstructed trajectories are superior to those obtained by linear ltering. Additionally, the Kalman lter provides insight into the neural encoding of hand motion. For example, analysis of the measurement model suggests that, while the neural ring is closely related to the position and velocity of the hand, the acceleration is redundant. Furthermore, the Kalman lter framework is exploited to recover the optimal lag time between hand movement and neural ring.

Minimalist coherent swarming of wireless networked autonomous mobile robots

Abstract: This paper presents the results of simulations investigating a decentralised control algorithm able to maintain the coherence of a swarm of radio-connected robots. In this work the radius of communication is considerably less than the global diameter of the swarm. The study explores coherent movement towards a beacon while avoiding obstacles and maintaining global shape. All behaviours are emergent as the study constrains itself to using only a restricted range omnidirectional radio, a beacon sensor and avoiding sensors. In spite of such restrictions we achieve the proposed aims. Moreover, the fact that the algorithm relies only on local information makes it highly scalable with an increase in the number of robots. The proposed approach also shows graceful degradation in the presence of noise.

Levels of dynamics and adaptive behavior in evolutionary neural controllers

Abstract: Two classes of dynamical recurrent neural networks, Continuous Time Recurrent Neural Networks (CTRNNs) (Yamauchi and Beer, 1994) and Plastic Neural Networks (PNNs) (Floreano and Urzelai, 2000) are compared on two behavioral tasks aimed at exploring their capabilities to display reinforcement-learning like behaviors and adaptation to unpredictable environmental changes. The networks report similar performances on both tasks, but PNNs display significantly better performance when sensory-motor re-adaptation is required after the evolutionary process. These results are discussed in the context of behavioral, biological, and computational definitions of learning.

Active vision and feature selection in evolutionary behavioral systems

Abstract: We describe an evolutionary approach to active vision systems for dynamic feature selection. After summarizing recent work on evolution of a simulated active retina for complex shape discrimination, we describe in detail experiments that extend this approach to an all-terrain mobile robot equipped with a mobile camera. We show that evolved robots are capable of selecting simple visual features and actively maintaining them on the same retinal position, which largely simplifies the "recognition" task, in order to generate efficient navigation trajectories with an extremely simple neural control system. Analysis of evolved solutions indicates that robots develop a simple and yet very efficient version of edge detection and visual looming to detect obstacles and move away from them. Two evolved sensory-motor strategies are described, one where the mobile camera is actively used throughout the entire navigation and one where it is used only at the beginning to point towards relevant environmental features. The relationship between these two strategies are discussed in the context of the underlying visuomotor mechanisms and of the evolutionary conditions.

Active perception: a sensorimotor account of object categorization

Abstract: We describe the results of a set of experiments in which we evolved the control system of artificial agents that are asked to categorize objects with different shapes on the basis of tactile information. Agents are provided with a 3-segments arm with 6 degrees of freedom and extremely coarse touch sensors. As we will see, despite such a limited sensory systems, evolved individuals are perfectly able to solve the problem. The analysis of the obtained results shows that evolved individuals always develop a well defined behavioral strategy that allows them to easily and robustly discriminate different objects despite the limitation of their sensory apparatus. Moreover, we discuss the general advantage of the evolutionary method for the synthesis of effective artificial agents.

Global localization and topological map-learning for robot navigation

Abstract: This paper describes a navigation system implemented on a real mobile robot Using simple sonar and visual sensors, it makes possible the autonomous construction of a dense topological map representing the environment At any time during the mapping process, this system is able to globally localize the robot, ie to estimate the robot's position even if the robot is passively moved from one place to another within the mapped area This is achieved using algorithms inspired by Hidden Markov Models adapted to the on-line building of the map Advantages and drawbacks of the system are discussed, along with its potential implications for the understanding of biological navigation systems

A method for isolating morphological effects on evolved behaviour

Abstract: As the field of embodied cognitive science begins to mature, it is imperative to develop methods for identifying and quantifying the constraints and opportunities an agent's body places on its possible behaviours In this paper we present results from a set of experiments conducted on 10 different legged agents, in which we evolve neural controllers for locomotion The genetic algorithm and neural network architecture were kept constant across the agent set, but the agents had different sizes, masses and body plans It was found that increased mass has a negative effect on the evolution of locomotion, but that this does not hold for all of the agents tested Also, the number of legs has an effect on evolved behaviours, with hexapedal agents being the easiest for which to evolve locomotion, and wormlike agents being the most difficult Moreover, it was found that repeating the experiments with a larger neural network increased the evolutionary potential of some of the agents, but not for all of them The results suggest that by employing this methodology we can test hypotheses about the behavioural effect of specific morphological features, which has to date eluded precise quantitative analysis

Visual orientation and motion control of MAKRO: adaptation to the sewer environment

Abstract: Adaptation has become an important aspect of robot design. The work here describes the perception and motion control of MAKRO - an autonomous robot for sewer inspection - from the point of view of MAKRO's adaptation to specific features of the sewer environment. Two features are crucial for MAKRO's adaptation. First, narrow sewer pipes connected into a unified system via junctions, compose a graph-like structure with rather constraint surface geometry. Second, a sewer interior is absolutely dark. The visual sensing of MAKRO is not only well adapted to these specific conditions, in fact it benefits from them. Visual orientation by a hybrid vision system gives rise to a rather simple vision model, which is capable of supporting real time orientation in the sewer. This instantiates an important principle of embodied cognition, which states that adaptation of an agent to an environment allows the use of simple principles of "cheap vision" for navigation purposes. Moreover, a fast visual processing enables MAKRO to react rapidly to events in its surroundings. This in turn, changes our approach to movement control: MAKRO does not act in the "plan - move" fashion; instead, it explores the environment, updates its heading and finds the right direction for the next move in real time. This leads to a second principle of the current work: if visual orientation of an agent operates in real time, all that is required for its successful navigation is to continuously update the right direction of motion. Navigation of MAKRO gives a powerful demonstration of how adaptation to an ecological niche and the exploitation of environmental constraints can lead to extraordinarily robust performance in a mobile robot.

An activation based behaviour control architecture for walking machines

Abstract: This paper introduces a behaviour network architecture for controlling walking machines. The behavior coordination problem is solved by distributing the activation of the behaviours according to the sensoric information as well as the specified task of the robot. This approach heavily emphasises the loop-back of the behaviour activities and the satisfaction of their goals. The results of initial experiments with this architecture on the four-legged walking machine BISAM are also presented.

Phonemic coding might result from sensory-motor coupling dynamics

Abstract: Human sound systems are invariably phone-mically coded Furthermore, phoneme inventories follow very particular tendancies To explain these phenomena, there existed so far three kinds of approaches : "Chomskyan"/cognitive innatism, morpho-perceptual innatism and the more recent approach of "language as a complex cultural system which adapts under the pressure of efficient communication" The two first approaches are clearly not satisfying, while the third, even if much more convincing, makes a lot of speculative assumptions and did not really bring answers to the question of phonemic coding We propose here a new hypothesis based on a low-level model of sensory-motor interactions We show that certain very simple and non language-specific neural devices allow a population of agents to build signalling systems without any functional pressure Moreover, these systems are phonemically coded Using a realistic vowel articulatory synthesizer, we show that the inventories of vowels have striking similarities with human vowel systems

Environment-specific novelty detection

Abstract: Novelty detection, recognising features that differ from those that are normally seen, is a potentially useful ability for a mobile robot. Once a robot can detect those features that are novel the amount of learning that has to be done can be reduced (as only new things need to be learnt), the attention of the robot can be focused onto the new features, and the robot can be used as an inspection agent.However, features that are novel in one place could be completely normal elsewhere - for example, tables and chairs are usually seen in offices, but very rarely seen in corridors. This paper suggests a method by which a set of novelty filters can be trained for different environments and the correct filter autonomously selected for the environment that the robot is currently travelling in. The method can also extend itself, so that further environments that are seen by the robot can be added without any retraining.

Asynchronous learning by emotions and cognition

Abstract: The existence of emotion and cognition as two interacting systems, both with important roles in decision-making, has been advocated by neurophysiological research (LeDoux, 1998; Damasio, 1994). Following this idea, this paper proposes the ALEC agent architecture which has both emotion and cognition learning capabilities to adapt to real-world environments. These two learning mechanisms embody very different properties which can be related with those of natural emotion and cognitive systems.Experimental results show that both systems contribute positively for the learning performance of the agent.

Evolving integrated controllers for autonomous learning robots using dynamic neural networks

Abstract: In 1994, Yamauchi and Beer (1994) attempted to evolve a dynamic neural network as a control system for a simulated agent capable of performing learning behaviour. They tried to evolve an integrated network, i.e. not modularized; this attempt failed. They ended up having to use independent evolution of separate controller modules, arbitrarily partitioned by the researcher. Moreover, they "provided" the agents with hard-wired reinforcement signals.The model we describe in this paper demonstrates that it is possible to evolve an integrated dynamic neural network that successfully controls the behaviour of a khepera robot engaged in a simple learning task. We show that dynamic neural networks, based on leaky-integrator neuron, shaped by evolution, appear to be able to integrate reactive and learned behaviour with an integrated control system which also benefits from its own evolved reinforcement signal.

Comparing a brain-inspired robot action selection mechanism with winner-takes-all

Abstract: We present a new robotic implementation of a brain-inspired model of action selection described by Gurney et al. (Gurney et al., 2001a, Gurney et al., 2001b) based on neural circuits located in the basal ganglia and thalamus of the vertebrate brain. Compared on an earlier robot implementation (Montes-Gonzalez et al., 2000), the new model demonstrates the capacity of the selection system to produce efficient 'energy' consumption/conversion in a 'feeding/resting' task whilst maintaining essential state variables within a 'zone of viability'. Generating appropriate action selection in this new setting entailed using biologically plausible Sigma-Pi units that can exploit correlated and anti-correlated dependencies between input signals when computing the 'salience' (urgency) of competing actions. A comparison between this brain-inspired selection mechanism and classical 'winner-takes-all' showed that the former can provide better behavioral persistence leading to more efficient energy intake.

On the use of sensors in self-reconfigurable robots

Abstract: In this paper we investigate the use of sensors in self-reconfigurable robots. We review several physically realized self-reconfigurable robots and conclude that little attention has been paid to the use of sensors. This is unfortunate since sensors can provide essential feedback that can be used to guide self-reconfiguration and control. In the systems that do use sensor feedback, the feedback is used locally on each module. However we identify a need in some situations to use sensor feedback globally. We therefore propose an approach where raw sensor values are abstracted and propagated to all modules. The sensor values are abstracted differently depending on the position of the producing sensor on the robot. We combine this approach with role based control, a control method for self-reconfigurable robots that we have developed earlier. We demonstrate that by combing these two approaches it is possible to make a self-reconfigurable robot consisting of six modules walk and avoid obstacles. However the reaction time of the robot is slow and therefore we discus possible ways of reducing the reaction time.

Using IIDs to estimate sound source direction

Abstract: Poster. Sounds recorded using a binaural head are analysed to find the azimuthal direction of a sound source. Two techniques for interaural intensity difference (IID) estimation are compared. In both, signals were filtered into a number of wideband logarithmically spaced frequency bands. In method 1, an estimate of the IID in each frequency band was made every 20ms, and in method 2, estimates were made only when a cluster of onsets had been found. Onsets were detected using a biologically plausible spike-based technique. IID vectors were converted to directions using estimates of the impulse response of the binaural head. The onset-based technique provides better results, particularly in reverberant environments.

The road sign problem revisited: handling delayed response tasks with neural robot controllers

Abstract: The 'road sign problem' is a class of delayed response tasks in which an agent's correct turning direction at a T-junction is dependent on a stimulus it has encountered earlier. Neural robot controllers of four different architectures have been evaluated in experiments with six different variations of the problem. The highest reliability was achieved by Extended Sequential Cascaded Networks, a higher-order recurrent neural network architecture.

An evolutionary approach to quantify internal states needed for the woods problem

Abstract: The Woods Problem is a difficult problem for purely reactive systems to handle. The difficulties are related to the perceptual aliasing problem, and the use of internal memory has been suggested to solve the problem. In this paper a novel approach in evolutionary computation is introduced to quantify the amount of memory required for a given task. The approach has been applied to Woods Problems such as wood101, woods102, Sutton's gridworld and woods14.Finite state machine controllers are used, as these permit easy measurement of the amount of memory in the controller. A concurrent evolutionary search for-the minimal but optimal control structure in memory-based systems, using an evolutionary Pareto-optimal search mechanism, determines the best behavior fitness for each level of controller memory. This memory analysis demonstrates the effect of internal memory in evolved controllers for Woods Problems and is also used to investigate the relationship between the number of sensors available to an agent and the amount of memory necessary for effective behavior.

Constructing complex minds through multiple authors

Abstract: The World-Wide-Mind (WWM) was introduced in [Humphrys, 2001]. For a short introduction see [Humphrys, 2001a]. Briefly, this is a scheme for putting animat "minds" online (as WWM "servers") so that large complex minds may be constructed from many remote components. The aim is to address the scaling up of animat research, or how to construct minds more complex than could be written by one author (or one research group).The first part of this paper describes how a number of existing animat architectures could be implemented as WWM servers. Any unified mind can easily map to a single WWM server. So most of the discussion here is on action selection (or behavior or goal selection), where each module could be a different WWM server (written by a different author).The second part of this paper describes the first implementation of WWM servers and clients, and explains in particular how to write a WWM server. Most animats researchers are programmers but not network programmers. Almost all protocols for remote services (CORBA, SOAP, etc.) assume the programmer is a networks specialist. This paper rejects these solutions, and shows how any animats researcher can put their animat "mind" or "world" online as a server by simply converting it into a command-line program that reads standard input and writes to standard output.

Agent-based modelling and the environmental complexity thesis

Abstract: A variation of Godfrey-Smith's 'environmental complexity thesis' is described which draws together two broad themes; the relation of functional properties of behaviour to environmental structure, and the distinction between behavioural and mechanistic levels of description The specific idea defended here is that behavioural and/or mechanistic complexity can be understood in terms of mediating well-adapted responses to environmental variability Particular attention is paid to the value of agent-based modelling within this framework

Evolution of a circuit of spiking neurons for phototaxis in a Braitenberg vehicle

Abstract: Animal nervous systems have evolved to use spiking neurons but the 'artificial nervous systems' of animats typically are designed, not evolved, and use networks of formal, artificial neurons. We describe the evolution of circuits of spiking neurons for a robot, motivated by the desire to study links between neurophysiology and behaviour in artificial and (ultimately) natural animals. Spiking neurons have computational capabilities additional to those of artificial neurons based on activation functions. In particular, they should be better suited to processing temporal sequences. Thus, we describe early work aimed at evolution of neural circuitry which, when implanted in a Braitenberg type 2b vehicle, promotes phototaxis behaviour in the form of movement towards flashing lights of a particular frequency. The longer-term aim is to evolve natural taxis behaviours such as that observed in the cricket.

Learning to autonomously select landmarks for navigation and communication

Abstract: Selecting landmarks for use by a navigating mobile robot is important for map-building systems However, it can also provide a way by which robots can communicate route information, so that one robot can tell another how to find a goal location A route through an environment can be described by the landmarks encountered along the path, and a robot following the same path must identify the perceptions corresponding to the actual landmarks in the description in order to localise itself This paper presents an algorithm to automatically select landmarks, choosing as landmarks places that do not fit into a model of typical perceptions acquired by the robot Four methods of aligning the landmarks between different runs on the same route are also presented The different alignment methods are evaluated according to both how well they produce matching landmarks and how suitable such alignment methods would be for use in a route communication system

Competitive foraging, decision making, and the ecological rationality of the matching law

Abstract: The matching law describes how individual foragers often allocate their choices, occasionally suboptimally, in experimental situations. The 'ideal free distribution' predicts how groups of foraging agents should distribute themselves, optimally, over patchy environments. This paper explores the possibility that a single behavioural heuristic can account for both phenomena, allowing the potential suboptimality of matching to be understood in terms of adaptation to a group context. Two simple heuristics are compared, e-sampling and ω-sampling: the latter is successful in both cases, but contrary to prior claims in the literature the former is successful in neither. These results emphasise the importance of multiple environmental value estimates in effective decision making.

Synthetic social relationships in animated virtual characters

Abstract: We describe a multi-agent system based on the social behavior of the gray wolf (Canis lupus). This system, shown as an interactive installation at SIGGRAPH 2001, allows several participants to direct semi-autonomous wolf pups in a virtual pack. The heart of the system is a simple, biologically-inspired mechanism by which synthetic entities form social relationships with each other. This mechanism enables the virtual wolves to form relationships with each other that are both biologically plausible and engaging to participants in the installation. Systems like the one described in this paper could be of use in a variety of domains, for example, as platforms for simulation, as educational aids, and as entertainment media.

Cortico-hippocampal maps and navigation strategies in robots and rodents

Abstract: A biologically inspired integrated model of different hippocampal subsystems makes a distinction between place cells (PC) within entorhinal cortex (diffuse) or dentate gyrus (segregated), and transition cells (TC) in CA3-CA1 that encode transitions between events. These two types of codes support two kinds of hippocampocortical cognitive maps: -A context-independent map in subiculum and EC encodes essentially the spatial layout of the environment thanks to a local dominance of ideothetic movement-related information over allothetic (visual) information; -A task-and-temporal-context dependent map based on the TCs in CA3-CA1 allows encoding, in higher order structures, maps as graphs resulting from combination of learned sequences of events. The dominantly spatial and the temporal-task-dependent maps are permanently stored in parietal cortex and prefrontal cortex respectively. On the basis of these two maps two distinct goal-oriented navigation strategies were designed in experimental robotic paradigms: -one based on a (population) vector code of the location-actions pairs to learn and implement to reach the goal; another based on linking TCs together as conditioning chains that will be implemented under the top-down guidance of drives and motivations.

Behaviour control using a functional and emotional model

Abstract: Animals are capable of pursuing long term goals while coping with the ever-changing events in their scenario In the quest of animats towards autonomy, we propose to use a functional model biased by emotions to drive their behaviours In addition to that, we suggest that emotions can be used as source of reinforcement to build functional description of objects, on the grounds of which to decide what behaviour to take next

Using a net to catch a mate: evolving CTRNNs for the dowry problem

Abstract: Choosing one option from a sequence of possibilities seen one at a time is a common problem facing agents whenever resources, such as mates or habitats, are distributed in time or space. Optimal algorithms have been developed for solving a form of this sequential search task known as the Dowry Problem (finding the highest dowry in a sequence of 100 values); here we explore whether continuous time recurrent neural networks (CTRNNs) can be evolved to perform adaptively in Dowry Problem scenarios, as an example of minimally cognitive behavior [Beer, 1996]. We show that even 4-neuron CTRNNs can successfully solve this sequential search problem, and we offer some initial analysis of how they can achieve this feat.

Learning in multi-robot scenarios through physically embedded genetic algorithms

Abstract: Motivation. For certain applications of autonomous mobile robots — surveillance, cleaning or exploration come immediately to mind — it is attractive to employ several robots simultaneously. Tasks such as the ones mentioned above are easily divisible between independent robots, and using several robots simultaneously promises a speedup of task execution, as well as more reliable and robust performance. For any robot operating in the real world, the question of how control is to be achieved is of prime importance. While fixed behavioural strategies, defined by the user, can indeed be used to control robots, they tend to be brittle in practice, due to the noisy and partly unpredictable nature of the real world. Therefore, instead of using fixed and pre-defined control procedures, learning is an attractive alternative. To determine a suitable control strategy for a mobile robot operating in noisy and possibly dynamic environments through learning requires a search through a very large state space. By parallelising this process through the use of several robots and collaborative learning, this learning process can be accelerated. A physically embedded GA (PEGA). In this paper, we present experiments conducted with two communicating mobile robots. Each robot’s control policy was encoded through a genetic string. By communicating genetic strings and fitnesses to one another at regular intervals, robots modified their individual control policy, using a genetic algorithm (GA). Contrary to common GA approaches, we did not use a simulate-and-transfer method, but implemented the GA directly on the robots. We were able to show that the following competences can all be acquired using the PEGA approach:

Genetic programming for robot vision

Abstract: Genetic Programming was used to create the vision subsystem of a reactive obstacle avoidance system for an autonomous mobile robot. The representation of algorithms was specifically chosen to capture the spirit of existing, hand written vision algorithms. Traditional computer vision operators such as Sobel gradient magnitude, median filters and the Moravec interest operator were combined arbitrarily. Images from an office hallway were used as training data. The evolved programs took a black and white camera image as input and estimated the location of the lowest non-ground pixel in a given column. The computed estimates were then given to a handwritten obstacle avoidance algorithm and used to control the robot in real time. Evolved programs successfully navigated in unstructured hallways, performing on par with hand-crafted systems.

Sharing a charging station without explicit communication in collective robotics

Abstract: This research focuses on the design of teams of mobile robots that are able to operate in real life situations and non-controlled environments. In order to perform any service, robots have firstly to display two properties: autonomy and self-sufficiency. This paper explores briefly a simple mechanism to enable a team of robots to behave in an autonomous and self-sufficient manner for as long as possible while taking into account the needs of partners.