Showing papers presented at "Simulation of Adaptive Behavior in 2010"

Indirectly encoding neural plasticity as a pattern of local rules

Abstract: Biological brains can adapt and learn from past experience. In neuroevolution, i.e. evolving artificial neural networks (ANNs), one way that agents controlled by ANNs can evolve the ability to adapt is by encoding local learning rules. However, a significant problem with most such approaches is that local learning rules for every connection in the network must be discovered separately. This paper aims to show that learning rules can be effectively indirectly encoded by extending the Hypercube-based NeuroEvolution of Augmenting Topologies (HyperNEAT) method. Adaptive HyperNEAT is introduced to allow not only patterns of weights across the connectivity of an ANN to be generated by a function of its geometry, but also patterns of arbitrary learning rules. Several such adaptive models with different levels of generality are explored and compared. The long-term promise of the new approach is to evolve large-scale adaptive ANNs, which is a major goal for neuroevolution.

Information dynamics of evolved agents

Abstract: Information-theoretic techniques have received much recent attention as tools for the analysis of embodied agents. However, while techniques for quantifying static information structure are well-established, the application of information theory to the analysis of temporal behavior is still in its infancy. Here we formulate a novel information-theoretic approach for analyzing the dynamics of information flow in embodied systems. To demonstrate our approach, we apply it to analyze a previously evolved model of relational categorization. The results of this analysis demonstrate the unique strengths of our approach for exploring the detailed structure of information dynamics, and point towards a natural synergy between temporally-extended information theory and dynamical systems theory.

Taming the beast: guided self-organization of behavior in autonomous robots

Abstract: Self-organizing processes are crucial for the development of living beings. Practical applications in robots may benefit from the self-organization of behavior, e.g. for the increased fault tolerance and enhanced flexibility provided that external goals can also be achieved. We present several methods for the guidance of self-organizing control by externally prescribed criteria. We show that the degree of self-organized explorativity of the robot can be regulated and that problem-specific error functions, hints, or abstract symbolic descriptions of a goal can be reconciled with the continuous robot dynamics.

SCRATCHbot: active tactile sensing in a whiskered mobile robot

Abstract: The rodent vibrissal (whisker) system is one of the most widely investigated model sensory systems in neuroscience owing to its discrete organisation from the sensory apparatus (the whisker shaft) all the way to the sensory cortex, its ease of manipulation, and its presence in common laboratory animals. Neurobiology shows us that the brain nuclei and circuits that process vibrissal touch signals, and that control the positioning and movement of the whiskers, form a neural architecture that is a good model of how the mammalian brain, in general, coordinates sensing with action. In this paper we describe SCRATCHbot, a biomimetic robot based on the rat whisker system, and show how this robot is providing insight into the operation of neural systems underlying vibrissal control, and is helping us to understand the active sensing strategies that animals employ in order to boost the quality and quantity of information provided by their sensory organs.

A model of symmetry breaking in collective decision-making

Abstract: Symmetry breaking is commonly found in self-organized collective decision making. It serves an important functional role, specifically in biological and bio-inspired systems. The analysis of symmetry breaking is thus an important key to understanding self-organized decision making. However, in many systems of practical importance available analytic methods cannot be applied due to the complexity of the scenario and consequentially the model. This applies specifically to self-organization in bio-inspired engineering. We propose a new modeling approach which allows us to formally analyze important properties of such processes. The core idea of our approach is to infer a compact model based on stochastic processes for a one-dimensional symmetry parameter. This enables us to analyze the fundamental properties of even complex collective decision making processes via Fokker-Planck theory. We are able to quantitatively address the effectiveness of symmetry breaking, the stability, the time taken to reach a consensus, and other parameters. This is demonstrated with two examples from swarm robotics.

Simulating human table tennis with a biomimetic robot setup

Abstract: Playing table tennis is a difficult motor task which requires fast movements, accurate control and adaptation to task parameters. Although human beings see and move slower than most robot systems they outperform all table tennis robots significantly. In this paper we study human table tennis and present a robot system that mimics human striking behavior. Therefore we model the human movements involved in hitting a table tennis ball using discrete movement stages and the virtual hitting point hypothesis. The resulting model is implemented on an anthropomorphic robot arm with 7 degrees of freedom using robotics methods. We verify the functionality of the model both in a physical realistic simulation of an anthropomorphic robot arm and on a real Barrett WAM™.

Attentional modulation of mutually dependent behaviors

Abstract: In this paper, we investigate simple attentional mechanisms suitable for sensing rate regulation and action coordination in the presence of mutually dependent behaviors. We present our architecture along with a case study where a real robotic system is to manage and harmonize conflicting tasks. This research focuses on attentional mechanisms for regulating the frequencies of sensor readings and action activations in a behavior-based robotic system. Such mechanisms are to direct sensors toward the most salient sources of information and filter the available sensory data to prevent unnecessary information processing.

Adaptive Locomotive Behaviors of a Biped Robot: Patterns Generation and Classification

Abstract: Neurobiological studies showed the important role of Cen- teral Pattern Generators for spinal cord in the control and sensory feed- back of animals' locomotion. In this paper, this role is taken into account in modeling bipedal locomotion of a robot. Indeed, as a rhythm gener- ator, a non-classical model of a neuron that can generate oscillatory as well as diverse motor patterns is presented. This allows di®erent motion patterns on the joints to be generated easily. Complex tasks, like walk- ing, running, and obstacle avoidance require more than just oscillatory movements. Our model provides the ability to switch between intrinsic behaviors, to enable the robot to react against environmental changes quickly. To achieve complex tasks while handling external perturbations, a new space for joints' patterns is introduced. Patterns are generated by our learning mechanism based on success and failure with the concept of vigilance. This allows the robot to be prudent at the beginning and adventurous at the end of the learning process, inducing a more e±cient exploration for new patterns. Motion patterns of the joint are classi¯ed into classes according to a metric, which re°ects the kinetic energy of the limb. Due to the classi¯cation metric, high-level control for action learning is introduced. For instance, an adaptive behavior of the rhythm generator neurons in the hip and the knee joints against external per- turbation are shown to demonstrate the e®ectiveness of the proposed learning approach.

Distributed online learning of central pattern generators in modular robots

Abstract: In this paper we study distributed online learning of locomotion gaits for modular robots. The learning is based on a stochastic approximation method, SPSA, which optimizes the parameters of coupled oscillators used to generate periodic actuation patterns. The strategy is implemented in a distributed fashion, based on a globally shared reward signal, but otherwise utilizing local communication only. In a physics-based simulation of modular Roombots robots we experiment with online learning of gaits and study the effects of: module failures, different robot morphologies, and rough terrains. The experiments demonstrate fast online learning, typically 5-30 min. for convergence to high performing gaits (≅ 30 cm/sec), despite high numbers of open parameters (45-54). We conclude that the proposed approach is efficient, effective and a promising candidate for online learning on many other robotic platforms.

Fractal gene regulatory networks for robust locomotion control of modular robots

Abstract: Designing controllers for modular robots is difficult due to the distributed and dynamic nature of the robots. In this paper fractal gene regulatory networks are evolved to control modular robots in a distributed way. Experiments with different morphologies of modular robot are performed and the results show good performance compared to previous results achieved using learning methods. Furthermore, some experiments are performed to investigate evolvability of the achieved solutions in the case of module failure and it is shown that the system is capable of come up with new effective solutions.

Simulation of how neuromodulation influences cooperative behavior

Abstract: Neuromodulators can have a strong effect on how organisms cooperate and compete for resources To better understand the effect of neuromodulation on cooperative behavior, a computational model of the dopaminergic and serotonergic systems was constructed and tested in games of conflict and cooperation This neural model was based on the assumptions that dopaminergic activity increases as expected reward increases, and serotonergic activity increases as the expected cost of an action increases The neural model guided the behavior of an agent that played a series of Hawk-Dove games against an opponent The agent adapted its behavior appropriately to changes in environmental conditions and to changes in its opponent's strategy The neural agent tended to engage in Hawk-like behavior in low-risk situations and Dove-like behavior in high-risk situations When the simulated dopaminergic activity was greater than the serotonergic activity, the agent tended to escalate a fight These results suggest how the neuromodulatory systems shape decision-making and adaptive behavior in competitive and cooperative situations

How virtual machinery can bridge the explanatory gap, in natural and artificial systems

Abstract: We can now show in principle how evolution could have produced the "mysterious" aspects of consciousness if, like engineers in the last six or seven decades, it had to solve increasingly complex problems of representation and control by producing systems with increasingly abstract, but effective, mechanisms, including self-observation capabilities, implemented in non-physical virtual machines which, in turn, are implemented in lower level physical mechanisms. For this, evolution would have had to produce far more complex virtual machines than human engineers have so far managed, but the key idea might be the same. However it is not yet clear whether the biological virtual machines could have been implemented in the kind of discrete technology used in computers as we know them.

Tactile discrimination using template classifiers: towards a model of feature extraction in mammalian vibrissal systems

Abstract: Rats and other whiskered mammals are capable of making sophisticated sensory discriminations using tactile signals from their facial whiskers (vibrissae). As part of a programme of work to develop biomimetic technologies for vibrissal sensing, including whiskered robots, we are devising algorithms for the fast extraction of object parameters from whisker deflection data. Previous work has demonstrated that radial distance to contact can be estimated from forces measured at the base of the whisker shaft. We show that in the case of a moving object contacting a whisker, the measured force can be ambiguous in distinguishing a nearby object moving slowly from a more distant object moving rapidly. This ambiguity can be resolved by simultaneously extracting object position and speed from the whisker deflection time series - that is by attending to the dynamics of the whisker's interaction with the object. We compare a simple classifier with an adaptive EM (Expectation Maximisation) classifier. Both systems are effective at simultaneously extracting the two parameters, the EM-classifier showing similar performance to a handpicked template classifier. We propose that adaptive classification algorithms can provide insights into the types of computations performed in the rat vibrissal system when the animal is faced with a discrimination task.

Autonomous development of social referencing skills

Abstract: In this work, we are interested in understanding how emotional interactions with a social partner can bootstrap increasingly complex behaviors such as social referencing. Our idea is that social referencing as well as facial expression recognition can emerge from a simple sensori-motor system involving emotional stimuli. Without knowing that the other is an agent, the robot is able to learn some complex tasks if the human partner has some "empathy" or at least "resonate" with the robot head (low level emotional resonance). Hence we advocate the idea that social referencing can be bootstrapped from a simple sensorimotor system not dedicated to social interactions.

A study of adaptive locomotive behaviors of a biped robot: patterns generation and classification

Abstract: Neurobiological studies showed the important role of Centeral Pattern Generators for spinal cord in the control and sensory feed-back of animals' locomotion. In this paper, this role is taken into account in modeling bipedal locomotion of a robot. Indeed, as a rhythm generator, a non-classical model of a neuron that can generate oscillatory as well as diverse motor patterns is presented. This allows different motion patterns on the joints to be generated easily. Complex tasks, like walking, running, and obstacle avoidance require more than just oscillatory movements. Our model provides the ability to switch between intrinsic behaviors, to enable the robot to react against environmental changes quickly. To achieve complex tasks while handling external perturbations, a new space for joints' patterns is introduced. Patterns are generated by our learning mechanism based on success and failure with the concept of vigilance. This allows the robot to be prudent at the beginning and adventurous at the end of the learning process, inducing a more efficient exploration for new patterns. Motion patterns of the joint are classified into classes according to a metric, which reflects the kinetic energy of the limb. Due to the classification metric, high-level control for action learning is introduced. For instance, an adaptive behavior of the rhythm generator neurons in the hip and the knee joints against external perturbation are shown to demonstrate the effectiveness of the proposed learning approach.

BeeIP: bee-inspired protocol for routing in mobile ad-hoc networks

Abstract: We introduce a new bee-inspired routing protocol for mobile ad hoc networks. Emphasis is given to the ability of bees to evaluate paths by considering several quality factors. In order to achieve similar behaviour in the networking environment, BeeIP is using cross-layering. Fetching parameters from the lower PHY and MAC layers to the core of the protocol, offers the artificial bees the ability to make predictions about the link's future performance. Our approach is compared with two well-known routing protocols in the area, the destination sequenced distance-vector protocol (DSDV), and the adaptive on-demand distance vector protocol (AODV). The outcome shows that BeeIP achieves higher data delivery rates and less control overhead than DSDV, and slightly better results compared to AODV, initializing less route discovery processes.

Cooperative stigmergic navigation in a heterogeneous robotic swarm

Abstract: We study self-organized cooperation in a heterogeneous robotic swarm consisting of two sub-swarms. The robots of each sub-swarm play distinct roles based on their different characteristics. We investigate how the swarm as a whole can solve complex tasks through a self-organized process based on local interactions between the sub-swarms. We focus on an indoor navigation task, in which we use a swarm of wheeled robots, called foot-bots, and a swarm of flying robots that can attach to the ceiling, called eye-bots. Foot-bots have to move back and forth between a source and a target location. Eye-bots are deployed in stationary positions against the ceiling, with the goal of guiding foot-bots. We study how the combined system can find efficient paths through a cluttered environment in a distributed way. The key component of our approach is a process of mutual adaptation, in which foot-bots execute instructions given by eye-bots, and eye-bots observe the behavior of foot-bots to adapt the instructions they give. The system is based on pheromone mediated navigation of ant colonies, as eye-bots function as stigmergic markers for foot-bots. Through simulation, we show that the system finds feasible paths in cluttered environments, converges onto the shortest of two paths, and spreads over different paths in case of congestion.

Internal models in the cerebellum: a coupling scheme for online and offline learning in procedural tasks

Abstract: The cerebellum plays a major role in motor control. It is thought to mediate the acquisition of forward and inverse internal models of the body-environment interaction [1]. In this study, the main processing components of the cerebellar microcomplex are modelled as a network of spiking neural populations. The model cerebellar circuit is shown to be suitable for learning both forward and inverse models. A new coupling scheme is put forth to optimise on-line adaptation and support offline learning. The proposed model is validated on two procedural tasks and the simulation results are consistent with data from human experiments on adaptive motor control and sleep-dependent consolidation [2, 3]. This work corroborates the hypothesis that both forward and inverse internal models can be learnt and stored by the same cerebellar circuit, and that their coupling favours online and offline learning of procedural memories.

Learning inverse kinematics for pose-constraint bi-manual movements

Abstract: We present a neural network approach to learn inverse kinematics of the humanoid robot ASIMO, where we focus on bimanual tool use. The learning copes with both the highly redundant inverse kinematics of ASIMO and the additional arbitrary constraint imposed by the tool that couples both hands. We show that this complex kinematics can be learned from few ground-truth examples using an efficient recurrent reservoir framework, which has been introduced previously for kinematics learning and movement generation. We analyze and quantify the network's generalization for a given tool by means of reproducing the constraint in untrained target motions.

eMOSAIC model for humanoid robot control

Abstract: In this study, we propose a novel extension of the MOSAIC architecture to control real humanoid robots. The MOSAIC architecture was originally proposed by neuroscientists to clarify the human ability of adaptive control. The modular architecture of the MOSAIC model can be useful for solving nonlinear and nonstationary control problems. Both humans and humanoid robots have nonlinear body dynamics and many degrees of freedom. In addition, they can carry objects, and this makes the dynamics nonstationary. Therefore, the MOSAIC architecture can be considered a promising candidate as a motor-control model of humans and a control framework for humanoid robots. However, the application of the MOSAIC model has been limited to simple simulated dynamics. Since each module of the MOSAIC has a forward model, we can adopt this model to construct a state estimator. By using the state estimators, the extended MOSAIC model can deal with large observation noise and partially observable systems. Thanks to these advantages, the proposed control framework can be applied to real systems such as humanoid robots.

Learning new motion primitives in the mirror neuron system: a self-organising computational model

Abstract: Computational models of the mirror (neuron) system are attractive in robotics as they may inspire novel approaches to implement e.g. action understanding. Here, we present a simple self-organising map which forms the first part of larger ongoing work in building such a model. We show that minor modifications to the standard implementation of such a map allows it to continuously learn new motor concepts. We find that this learning is facilitated by an initial motor babbling phase, which is in line with an embodied view of cognition. Interestingly, we also find that the map is capable of reproducing neurophysiological data on goal-encoding mirror neurons. Overall, our model thus fulfils the crucial requirement of being able to learn new information throughout its lifetime. Further, although conceptually simple, its behaviour has interesting parallels to both cognitive and neuroscientific evidence.

Multi-objective evolutionary algorithms to investigate neurocomputational issues: the case study of basal ganglia models

Abstract: The basal ganglia (BG) are a set of subcortical nuclei involved in action selection processes. We explore here the automatic parameterization of two models of the basal ganglia (the GPR and the CBG) using multi-objective evolutionary algorithms. We define two objective functions characterizing the supposed winner-takes-all functionality of the BG and obtain a set of solutions lying on the Pareto front for each model. We show that the CBG architecture leads to solutions dominating the GPR ones, this highlights the usefulness of the CBG additional connections with regards to the GPR. We then identify the most satisfying solutions on the fronts in terms of both functionality and plausibility. We finally define critical and indifferent parameters by analyzing their variations and values on the fronts, helping us to understand the dynamics governing the selection process in the BG models.

Why and how hippocampal transition cells can be used in reinforcement learning

Abstract: In this paper we present a model of reinforcement learning (RL) which can be used to solve goal-oriented navigation tasks. Our model supposes that transitions between places are learned in the hippocampus (CA pyramidal cells) and associated with information coming from path-integration. The RL neural network acts as a bias on these transitions to perform action selection. RL originates in the basal ganglia and matches observations of reward-based activity in dopaminergic neurons. Experiments were conducted in a simulated environment. We show that our model using transitions and inspired by Q-learning performs more efficiently than traditional actor-critic models of the basal ganglia based on temporal difference (TD) learning and using static states.

Multimodal predictive control in crickets

Abstract: A combination of behavioural testing and robotic modelling was used to investigate the interaction between sound localisation (phonotaxis) and optomotor following in crickets. Three hypotheses describing simple interactions -- summation, gain modulation and chaining -- were eliminated, leaving efference copy as the most likely mechanism. A speculative but plausible model for predicting re-afference was implemented and evaluated on a robot.

How to pick the right one: investigating tradeoffs among female mate choice strategies in treefrogs

Abstract: Two main theories of female mate choice, that females either pick the best from the n closest males (best-of-n) or the closest with some minimum quality (min-threshold), make different behavioral predictions in some cases, yet both are supported by biological data. We present a computational agent-based model that is well-suited for investigating the differences between the two strategies for the biological model organism Hyla versicolor ("gray treefrog"). We show, based on results from systematic simulation studies, that min-threshold overall is the dominant strategy, even though best-of-n has some areas in parameter space where it dominates min-threshold.

The dependence of braking strategies on optical variables in an evolved model of visually-guided braking

Abstract: This paper presents results from two sets of experiments which investigate how strategies used by embodied dynamical agents in a simple braking task are affected by the perceptual information that the agents receive. Agents are evolved in a simple 2D environment containing one stationary object. The task of the agents is to stop as close as possible to the object without hitting it. The results of these experiments demonstrate that most of the evolved agents use an impulsive braking strategy, in which deceleration is not controlled continuously. Potential causes of this impulsive braking strategy and possible future directions are discussed.

Predicting affordances from gist

Abstract: This paper presents an incremental learning mechanism to create associations between the affordances provided by the environment and its gist. The proposed model aims at helping the agent on the prioritisation of its perceptual resources, and consequently on visual attention. The focus on affordances, rather than on objects, enables a self-supervised learning mechanism without assuming the existence of symbolic object representations, thus facilitating its integration on a developmental framework. The focus on affordances also contributes to our understanding on the role of sensorimotor coordination on the organisation of adaptive behaviour. Promising results are obtained with a physical experiment on a natural environment, where a camera was handled as if it was being carried by an actual robot performing obstacle avoidance, trail following and wandering behaviours.

Adaptation of coupled sensorimotor mappings: an investigation towards developmental learning of humanoids

Abstract: The engineering of humanoid or similar robot systems requires frameworks and architectures that support the integration of a variety of sensorimotor modalities. Within our computational framework for visually guided reaching we ask how coupled sensorimotor mappings of different modalities can be learned autonomously from scratch. Based on a learning process that allows continuous adaptation of a single sensorimotor mapping, we introduce three strategies (parallel, sequential, and synchronous) for the learning of coupled mappings. These strategies are systematically tested in a simplified simulation. The experiments indicate that stages of development can emerge from synchronous adaptation of sensorimotor mappings of different characteristics. Thus, observed stages in development are not necessarily the result of explicitly defined and triggered learning tasks.

Simulating the morphological feasibility of adaptive beamforming in bats

Abstract: It has been suggested that it is advantageous for bats to adapt their emission beam pattern depending on the situation. Hartley [9] has proposed that bats could steer the direction in which they emit most energy by controlling the phase relationship between the sound emerging from both nostrils. In this paper, we evaluate based on simulations, whether such an adaptive mechanism would be viable in FM bats given their specialized facial morphology. We find that these bats could indeed relocate the center of their emission beam pattern using a phased array mechanism. Furthermore, we list two ways in which this would help bats localizing target objects.

Path integration working memory for multi tasks dead reckoning and visual navigation

Abstract: Biologically inspired models for navigation use mechanisms like path integration or sensori-motor learning. This paper describes the use of a proprioceptive working memory to give path integration the potential to store several goals. Then we coupled the path integration working memory to place cell sensori-motor learning to test the potential autonomy this gives to the robot. This navigation architecture intends to combine the benefits of both strategies in order to overcome their drawbacks. The robot uses a low level motivational system based on a simulated physiology. Experimental evaluation is done with a robot in a real environment performing a multi goal navigation task.