Showing papers presented at "Simulation of Adaptive Behavior in 2006"

Evolving spatiotemporal coordination in a modular robotic system

Abstract: In this paper we present a novel information-theoretic measure of spatiotemporal coordination in a modular robotic system, and use it as a fitness function in evolving the system This approach exemplifies a new methodology formalizing co-evolution in multi-agent adaptive systems: information-driven evolutionary design The methodology attempts to link together different aspects of information transfer involved in adaptive systems, and suggests to approximate direct task-specific fitness functions with intrinsic selection pressures In particular, the information-theoretic measure of coordination employed in this work estimates the generalized correlation entropy K2 and the generalized excess entropy E2 computed over a multivariate time series of actuators' states The simulated modular robotic system evolved according to the new measure exhibits regular locomotion and performs well in challenging terrains.

Simbad: an autonomous robot simulation package for education and research

Abstract: Simbad is an open source Java 3d robot simulator for scientific and educational purposes It is mainly dedicated to researchers and programmers who want a simple basis for studying Situated Artificial Intelligence, Machine Learning, and more generally AI algorithms, in the context of Autonomous Robotics and Autonomous Agents It is is kept voluntarily readable and simple for fast implementation in the field of Research and/or Education Moreover, Simbad embeds two stand-alone additional packages : a Neural Network library (feed-forward NN, recurrent NN, etc) and an Artificial Evolution Framework for Genetic Algorithm, Evolutionary Strategies and Genetic Programming These packages are targeted towards Evolutionary Robotics The Simbad Package is available from http://simbadsourceforgenet/ under the conditions of the GPL (GNU General Public Licence)

Communication in a swarm of miniature robots: the e-Puck as an educational tool for swarm robotics

Abstract: Swarm intelligence, and swarm robotics in particular, are reaching a point where leveraging the potential of communication within an artificial systempromises to uncover newand varied directions for interesting research without compromising the key properties of swarmintelligent systems such as self-organization, scalability, and robustness. However, the physical constraints of using radios in a robotic swarm are hardly obvious, and the intuitive models often used for describing such systems do not always capture them with adequate accuracy. In order to demonstrate this effectively in the classroom, certain tools can be used, including simulation and real robots. Most instructors currently focus on simulation, as it requires significantly less investment of time, money, and maintenance--but to really understand the differences between simulation and reality, it is also necessary to work with the real platforms from time to time. To our knowledge, our coursemay be the only one in the world where individual students are consistently afforded the opportunity to work with a networked multi-robot system on a tabletop. The e-Puck, a low-cost small-scale mobile robotic platform designed for educational use, allows us bringing real robotic hardware into the classroom in numbers sufficient to demonstrate and teach swarm-robotic concepts.We present here a custom module for local radio communication as a stackable extension board for the e-Puck, enabling information exchange between robots and also with any other IEEE 802.15.4-compatible devices. Transmission power can be modified in software to yield effective communication ranges as small as fifteen centimeters. This intentionally small range allows us to demonstrate interesting collective behavior based on local information and control in a limited amount of physical space, where ordinary radios would typically result in a completely connected network. Here we show the use of this module facilitating a collective decision among a group of 10 robots.

Visual control of flight speed and height in the honeybee

Abstract: The properties of visually guided flight speed and height control were investigated by training honeybees (Apis mellifera L.) to fly through a tunnel in which the visual cues in the lateral and ventral visual fields could be varied by changing the patterns on the walls and floor of the tunnel The results show that honeybees regulate their flight speed by keeping the velocity of the image of the environment in their eye constant The results also show that honeybees use visual information from the ground to control their height above the ground The findings of this study reveal that the mechanisms of flight speed and height control in the honeybee are perfectly adapted for extracting information from a complex visual environment using simple sensors and computations Consequently, the techniques of visual guidance that are reported here suggest insect-inspired strategies for the control of aircraft flight.

An adaptive robot motivational system

Abstract: We present a robot motivational system design framework The framework represents the underlying (possibly conflicting) goals of the robot as a set of drives, while ensuring comparable drive levels and providing a mechanism for drive priority adaptation during the robot's lifetime The resulting drive reward signals are compatible with existing reinforcement learning methods for balancing multiple reward functions We illustrate the framework with an experiment that demonstrates some of its benefits.

Believability testing and bayesian imitation in interactive computer games

Abstract: In imitation learning, agents are trained to carry out certain actions by examining a demonstration of the task at hand Though common in robotics, little work has been done in translating these concepts to computer games Given that present-day games generally use antiquated AI techniques which can often lead to stilted, mechanical and conspicuously artificial behaviour, it seems likely that approaches based on the imitation of human players may produce agents which convey a more humanlike impression than their traditional counterparts At the same time, there exists no formal method of quantifying what constitutes a ‘humanlike' impression; an equivalent of the Turing test is needed, with the requirement that an agent's appearance and behaviour be capable of deceiving an observer into misidentifying it as human The aims of this paper are thus threefold; we describe an approach to the imitation of strategic behaviour and motion, propose a formal method of quantifying the degree to which different agents are perceived as ‘humanlike', and present the results of a series of experiments using these two systems.

Incremental evolution of target-following neuro-controllers for flapping-wing animats

Abstract: Using an incremental multi-objective evolutionary algorithm and the ModNet encoding, we generated working neuro-controllers for target-following behavior in a simulated flapping-wing animat To this end, we evolved tail controllers that were combined with two closed-loop wing-beat controllers previously generated, and able to secure straight flight at constant altitude and speed The corresponding results demonstrate that a wing-beat strategy that consists in continuously adapting the twist of the external wing panel leads to better manoeuvring capabilities than another strategy that adapts the beating amplitude Such differences suggest that further improvements in flying control should better rely on some sort of automatic incremental evolution procedure than on any hand-designed decomposition of the problem.

From motor babbling to purposive actions: emerging self-exploration in a dynamical systems approach to early robot development

Abstract: Self-organization and the phenomenon of emergence play an essential role in living systems and form a challenge to artificial life systems This is not only because systems become more lifelike, but also since self-organization may help in reducing the design efforts in creating complex behavior systems The present paper studies self-exploration based on a general approach to the self-organization of behavior, which has been developed and tested in various examples in recent years This is a step towards autonomous early robot development We consider agents under the close sensorimotor coupling paradigm with a certain cognitive ability realized by an internal forward model Starting from tabula rasa initial conditions we overcome the bootstrapping problem and show emerging self-exploration Apart from that, we analyze the effect of limited actions, which lead to deprivation of the world model We show that our paradigm explicitly avoids this by producing purposive actions in a natural way Examples are given using a simulated simple wheeled robot and a spherical robot driven by shifting internal masses.

Origins of communication in evolving robots

Abstract: In this paper we describe how a population of simulated robots evolved for the ability to solve a collective navigation problem develop individual and social/communication skills In particular, we analyze the evolutionary origins of motor and signaling behaviors Obtained results indicate that signals and the meaning of the signals produced by evolved robots are grounded not only on the robots sensory-motor system but also on robots' behavioral capabilities previously acquired Moreover, the analysis of the co-evolution of robots individual and communicative abilities indicate how innovation in the former might create the adaptive basis for further innovations in the latter and vice versa.

Adaptive four legged locomotion control based on nonlinear dynamical systems

Abstract: Dynamical systems have been increasingly studied in the last decade for designing locomotion controllers They offer several advantages over previous solutions like synchronization, smooth transitions under parameter variation, and robustness In this paper, we present an adaptive locomotion controller for four-legged robots The controller is composed of a set of coupled nonlinear dynamical systems Using our controller the robot is capable of adapting its locomotion to the physical properties of the robot, in particular its resonant frequency Our approach aims at developing an on-line learning system that attempts to minimize the energy necessary for the gait We have implemented the model both in a simulated physical environment (Webots) and on a Sony Aibo robot We present a series of experiments which demonstrate how the controller can tune its frequency to the resonant frequency of the robot, and modify it when the weight of the robot is changed.

Visual learning of affordance based cues

Abstract: This work is about the relevance of Gibson's concept of affordances [1] for visual perception in interactive and autonomous robotic systems In extension to existing functional views on visual feature representations, we identify the importance of learning in perceptual cueing for the anticipation of opportunities for interaction of robotic agents We investigate how the originally defined representational concept for the perception of affordances – in terms of using either optical flow or heuristically determined 3D features of perceptual entities – should be generalized to using arbitrary visual feature representations In this context we demonstrate the learning of causal relationships between visual cues and predictable interactions, using both 3D and 2D information In addition, we emphasize a new framework for cueing and recognition of affordance-like visual entities that could play an important role in future robot control architectures We argue that affordance-like perception should enable systems to react to environment stimuli both more efficient and autonomous, and provide a potential to plan on the basis of responses to more complex perceptual configurations We verify the concept with a concrete implementation applying state-of-the-art visual descriptors and regions of interest that were extracted from a simulated robot scenario and prove that these features were successfully selected for their relevance in predicting opportunities of robot interaction.

Experimental study on task teaching to real rats through interaction with a robotic rat

Abstract: “Learning” has been well studied in several research areas such as psychology, brain science, computer science and robotics In these studies, many experiments using animals have been performed On the other hand, several researchers have been studying adaptive interactions and task learning through interactions between humans and robots We then focus on adaptive interactions between animals and robots The purpose of our research is to develop a framework of adaptive interactions between animals and robots through interaction experiments between rats and a robotic rat We propose a novel behavior generation algorithm for the robot to enable it to autonomously teach a simple behavior task to rats as an example of adaptive interaction This algorithm was implemented in the robot and the experimental setup, and then verified through the experiment.

Use your illusion: sensorimotor self-simulation allows complex agents to plan with incomplete self-knowledge

Abstract: We present a practical application of sensorimotor self- simulation for a mobile robot Using its self-simulation, the robot can reason about its ability to perform tasks, despite having no model of many of its internal processes and thus no way to create an a priori configuration space in which to search We suggest that this in-the-head rehearsal of tasks is particularly useful when the tasks carry a high risk of robot “death”, as it provides a source of negative feedback in perfect safety This approach is a useful complement to existing work using forward models for anticipatory behaviour A minimal system is shown to be effective in simulation and real-world experiments The virtues and limitations of the approach are discussed and future work suggested

A schema based model of the praying mantis

Abstract: We present a schema-based agent architecture which is inspired by an ethological model of the praying mantis It includes an inner state, perceptual and motor schemas, several routines, a fovea and a motor We describe the design and implementation of the architecture and we use it for comparing two models: the former uses reactive, stimulus-response schemas; the latter involves also forward models, which are used by the schemas for generating predictions Our results show an advantage in using anticipatory components inside the schemas

Incremental evolution of robot controllers for a highly integrated task

Abstract: In this paper we apply incremental evolution for automatic synthesis of neural network controllers for a group of physically connected mobile robots called s-bots The robots should be able to safely and cooperatively perform phototaxis in an arena containing holes We experiment with two approaches to incremental evolution, namely behavioral decomposition and environmental complexity increase Our results are compared with results obtained in a previous study where several non-incremental evolutionary algorithms were tested and in which the evolved controllers were shown to transfer successfully to real robots Surprisingly, none of the incremental evolutionary strategies performs any better than the non-incremental approach We discuss the main reasons for this and why it can be difficult to apply incremental evolution successfully in highly integrated tasks.

The emergence of communication by evolving dynamical systems

Abstract: In the context of minimally cognitive behavior, we used multi-robotic systems to investigate the emergence of communication and cooperation during the evolution of recurrent neural networks The networks are systematically analyzed to identify their relevant dynamical properties Evolution efficiently adapts these properties through small structural changes within the networks when specific environmental conditions are altered, such as the number of interacting robots The findings signify the importance of reducing the predefined knowledge about resulting behaviors, dynamical properties of control, and the topology of neural networks in order to utilize the strength of the Evolutionary Robotics approach to Artificial Life.

Integration of an autonomous artificial agent in an insect society: experimental validation

Abstract: In mixed societies of robots and cockroaches, several insect-like-robot (Insbot) and animals interact in order to perform collective decision-making Many gregarious species are able to collectively select a resting site without any leadership The key process is based on the modulation of the probability of leaving the shelter according to the total population under this shelter and its light intensity It is important that cockroaches perceive the robot as a “congener” This recognition is mainly based on a chemical blend The aim of this study is to validate experimentally (1) the behavioral patterns expressed by the cockroaches in presence of shelters and of an Insbot, and (2) the important role played by the chemical blend on collective decision-makings.

Incremental skill acquisition for self-motivated learning animats

Abstract: A central role in the development process of children is played by self-exploratory activities Through a playful interaction with the surrounding environment, they test their own capabilities, explore novel situations, and understand how their actions affect the world During this kind of exploration, interesting situations may be discovered By learning to reach these situations, a child incrementally develops more and more complex skills Inspired by studies from psychology, neuroscience, and machine learning, we designed SMILe (Self-Motivated Incremental Learning), a learning framework that allows artificial agents to autonomously identify and learn a set of abilities useful to face several different tasks, through an iterated three phase process: by means of a random exploration of the environment (babbling phase), the agent identifies interesting situations and generates an intrinsic motivation (motivating phase) aimed at learning how to get into these situations (skill acquisition phase) This process incrementally increases the skills of the agent, so that new interesting configurations can be experienced We present results on two gridworld environments to show how SMILe makes it possible to learn skills that enable the agent to perform well and robustly in many different tasks.

Combining self-organizing maps with mixtures of experts: application to an actor-critic model of reinforcement learning in the basal ganglia

Abstract: In a reward-seeking task performed in a continuous environment, our previous work compared several Actor-Critic (AC) architectures implementing dopamine-like reinforcement learning mechanisms in the rat's basal ganglia The task complexity imposes the coordination of several AC submodules, each module being an expert trained in a particular subset of the task We showed that the classical method where the choice of the expert to train at a given time depends on each expert's performance suffered from strong limitations We rather proposed to cluster the continuous state space by an ad hoc method that lacked autonomy and generalization abilities In the present work we have combined the mixture of experts with self-organizing maps in order to cluster autonomously the experts' responsibility space On the one hand, we find that classical Kohonen maps give very variable results: some task decompositions provide very good and stable reinforcement learning performances, whereas some others are unadapted to the task Moreover, they require the number of experts to be set a priori On the other hand, algorithms like Growing Neural Gas or Growing When Required have the property to choose autonomously and incrementally the number of experts to train They lead to good performances, even if they are still weaker than our hand-tuned task decomposition and than the best Kohonen maps that we got We finally discuss on propositions about what information to add to these algorithms, such as knowledge of current behavior, in order to make the task decomposition appropriate to the reinforcement learning process

Spiking neural controllers for pushing objects around

Abstract: We evolve spiking neural networks that implement a seek-push-release drive for a simple simulated agent interacting with objects The evolved agents display minimally-cognitive behavior, by switching as a function of context between the three sub-behaviors and by being able to discriminate relative object size The neural controllers have either static synapses or synapses featuring spike-timing-dependent plasticity (STDP) Both types of networks are able to solve the task with similar efficacy, but networks with plastic synapses evolved faster In the evolved networks, plasticity plays a minor role during the interaction with the environment and is used mostly to tune synapses when networks start to function.

A model of sensorimotor coordination in the rat whisker system

Abstract: The rat has a sophisticated tactile sensory system centred around the facial whiskers During normal behaviour, rats sweep their longer whiskers (macrovibrissae) through the environment to obtain large-scale information, whilst gathering small-scale information with the sensory apparatus around their snout The macrovibrissae are actively and differentially controlled Using high-speed video recording, we have observed that temporal and spatial parameters of whisking pattern generation are modulated to match environmental features such as the position and orientation of nearby surfaces Whisking is also closely co-ordinated with head and body movements, allowing the animal to locate and orient to interesting stimuli detected through whisker contact In this paper, we present a hybrid (spiking-neuron/arithmetic) model of the neural systems underlying these observed adaptive sensorimotor behaviours, and demonstrate its performance in a simulated robot with rat-like morphology We also report progress towards embedding these control systems in a physical robot with biomimetic whiskers.

Transition cells for navigation and planning in an unknown environment

Abstract: We present a navigation and planning system using vision for extracting non predefined landmarks, a dead-reckoning system generating the integrated movement and a topological map Localisation and planning remain possible even if the map is partially unknown An omnidirectional camera gives a panoramic images from which unpredefined landmarks are extracted The set of landmarks and their azimuths relative to a fixed orientation defines a particular location without any need of an external environment map Transitions between two locations recognized at time t and t-1 are explicitly coded, and define spatio-temporal transitions These transitions are the sensory-motor unit chosen to support planning During exploration, a topological map (our cognitive map) is learned on-line from these transitions without any cartesian coordinates nor occupancy grids The edges of this map may be modified in order to take into account dynamical changes of the environment The transitions are linked with the integrated movement used for moving from one place to the others When planning is required, the activities of transitions coding for the required goal in the cognitive map are enough to bias predicted transitions and to obtain the required movement.

A model of reaching that integrates reinforcement learning and population encoding of postures

Abstract: When monkeys tackle novel complex behavioral tasks by trial-and-error they select actions from repertoires of sensorimotor primitives that allow them to search solutions in a space which is coarser than the space of fine movements Neuroscientific findings suggested that upper-limb sensorimotor primitives might be encoded, in terms of the final goal-postures they pursue, in premotor cortex A previous work by the authors reproduced these results in a model based on the idea that cortical pathways learn sensorimotor primitives while basal ganglia learn to assemble and trigger them to pursue complex reward-based goals This paper extends that model in several directions: a) it uses a Kohonen network to create a neural map with population encoding of postural primitives; b) it proposes an actor-critic reinforcement learning algorithm capable of learning to select those primitives in a biologically plausible fashion (i.e., through a dynamic competition between postures); c) it proposes a procedure to pre-train the actor to select promising primitives when tackling novel reinforcement learning tasks Some tests (obtained with a task used for studying monkeys engaged in learning reaching-action sequences) show that the model is computationally sound and capable of learning to select sensorimotor primitives from the postures' continuous space on the basis of their population encoding.

Robot learning in a social robot

Abstract: In this paper, research work on Arisco is described Arisco is a social robot built around a robotic head with gesture ability, visual and auditive perception and learning It is intended for interacting with people The general architecture is first described in the paper Then, the learning capacity of Arisco is addressed It learns and performs associations between different stimulus responses through several dynamic neural networks, guided by motivational drives Main contribution of this paper is the integration in a real robot of conditioning learning models based on a neural competitive network A number of experiments are discussed, covering stimulus competition, habituation and first and second order conditioning.

POTBUG: a mind's eye approach to providing BUG-Like guarantees for adaptive obstacle navigation using dynamic potential fields

Abstract: The problem we address is adaptive obstacle navigation for autonomous robotic agents in an unknown or dynamically changing environment with a 2-D travel surface without the use of a global map Two well known but hitherto apparently antithetical approaches to the problem, potential fields and BUG algorithms, are synthesised here The best of both approaches is attempted by combining a Mind's Eye with dynamic potential fields and BUG-like travel modes The resulting approach, using only sensed goal directions and obstacle distances relative to the robot, is compatible with a wide variety of robots and provides robust BUG-like guarantees for successful navigation of obstacles Simulation experiments are reported for both near-sighted (POTBUG) and far-sighted (POTSMOOTH) robots The results are shown to support the theoretical design's intentions that the guarantees persist in the face of significant sensor perturbation and that they may also be attained with smoother paths than existing BUG paths.

Bubbleworld.Evo: artificial evolution of behavioral decisions in a simulated predator-prey ecosystem

Abstract: This article presents a multi-agent simulation of an abstract ecosystem which is inhabited by two species: a predator species and a prey species Both species show the typical behaviors found in such an ecological relationship that are: hunting behavior and escaping behavior In the simulation, the actors make behavioral decisions according to “genetically fixed” weighting parameters These parameters determine which prey item is selected by the predator and which predators are avoided the most by prey Thus these parameters shape the decisions performed by both species We incorporated artificial evolution by allowing successful animals to pass their features to their offspring, a process that includes mutation and recombination of these “genes” The simulation shows that different kinds of optimal behavioral choices emerge out of artificial evolution, when the simulation is run with different physiological and morphological parameters of the actors.

Some adaptive advantages of the ability to make predictions

Abstract: We describe some simple simulations showing two possible adaptive advantages of the ability to predict the consequences of one's actions: predicted inputs can replace missing inputs and predicted success vs failure can help deciding whether to actually executing a planned action or not The neural networks controlling the organisms' behaviour include distinct modules whose connection weights are all genetically inherited and evolved using a genetic algorithm except those of the predictive module which are learned during life.

Hierarchical cooperative coevolution facilitates the redesign of agent-based systems

Abstract: The current work addresses the problem of redesigning brain-inspired artificial cognitive systems in order to gradually enrich them with advanced cognitive skills In the proposed approach, properly formulated neural agents are employed to represent brain areas A cooperative coevolutionary method, with the inherent ability to co-adapt substructures, supports the design of agents Interestingly enough, the same method provides a consistent mechanism to reconfigure (if necessary) the structure of agents, facilitating follow-up modelling efforts In the present work we demonstrate partial redesign of a brain-inspired cognitive system, in order to furnish it with learning abilities The implemented model is successfully embedded in a simulated robotic platform which supports environmental interaction, exhibiting the ability of the improved cognitive system to adopt, in real-time, two different operating strategies.

Spatially constrained networks and the evolution of modular control systems

Abstract: This paper investigates the relationship between spatially embedded neural network models and modularity It is hypothesised that spatial constraints lead to a greater chance of evolving modular structures Firstly, this is tested in a minimally modular task/controller scenario Spatial networks were shown to possess the ability to generate modular controllers which were not found in standard, non-spatial forms of network connectivity We then apply this insight to examine the effect of varying degrees of spatial constraint on the modularity of a controller operating in a more complex, situated and embodied simulated environment We conclude that a bias towards modularity is perhaps not always a desirable property for a control system paradigm to possess.

Evolutionary active vision toward three dimensional landmark-navigation

Abstract: Active vision may be useful to perform landmark-based navigation where landmark relationship requires active scanning of the environment In this article we explore this hypothesis by evolving the neural system controlling vision and behavior of a mobile robot equipped with a pan/tilt camera so that it can discriminate visual patterns and arrive at the goal zone The experimental setup employed in this article requires the robot to actively move its gaze direction and integrate information over time in order to accomplish the task We show that the evolved robot can detect separate features in a sequential manner and discriminate the spatial relationships An intriguing hypothesis on landmark-based navigation in insects derives from the present results.