scispace - formally typeset
Search or ask a question

Showing papers presented at "Simulation of Adaptive Behavior in 2014"


Book ChapterDOI
22 Jul 2014
TL;DR: The Max-Pooling Convolutional Neural Network (MPCNN) compressor is evolved online, maximizing the distances between normalized feature vectors computed from the images collected by the recurrent neural network (RNN) controllers during their evaluation in the environment.
Abstract: Dealing with high-dimensional input spaces, like visual input, is a challenging task for reinforcement learning (RL). Neuroevolution (NE), used for continuous RL problems, has to either reduce the problem dimensionality by (1) compressing the representation of the neural network controllers or (2) employing a pre-processor (compressor) that transforms the high-dimensional raw inputs into low-dimensional features. In this paper we extend the approach in [16]. The Max-Pooling Convolutional Neural Network (MPCNN) compressor is evolved online, maximizing the distances between normalized feature vectors computed from the images collected by the recurrent neural network (RNN) controllers during their evaluation in the environment. These two interleaved evolutionary searches are used to find MPCNN compressors and RNN controllers that drive a race car in the TORCS racing simulator using only visual input.

26 citations


Book ChapterDOI
22 Jul 2014
TL;DR: The proposed algorithm involves the probabilistic signal processing modelling techniques for analysis of different types of collective behaviors based on interactions among people and classification models to estimate emotions as positive or negative.
Abstract: Detecting emotions of a crowd to control the situation is an area of emerging interest. The purpose of this paper is to present a novel idea to detect the emotions of the crowd. Emotions are defined as evolving quantities arising from the reaction to contextual situations in a set of dynamic pattern of events. These events depend on internal and external interaction states in an already mapped space. The emotions of multiple people constituting a crowd in any surveillance environment are estimated by their social and collective behaviors using sensor signals e.g., a camera, which captures and tracks their motion. The feature space is constructed based on local features to model the contextual situations and the different interactions corresponding to different emergent behaviors are modeled using bio-inspired dynamic model. The changes in emotions correspond to behavioral changes which are produced to regulate behaviors under different encountered situations. Proposed algorithm involves the probabilistic signal processing modelling techniques for analysis of different types of collective behaviors based on interactions among people and classification models to estimate emotions as positive or negative. The evaluations are performed on simulated data show the proposed algorithm effectively recognizes the emotions of the crowd under specific situations.

24 citations


Book ChapterDOI
22 Jul 2014
TL;DR: Simulation results show that this bio-inspired approach generates self-organizing emergent locomotion allowing the robot to adaptively form regular patterns, to stably walk while pushing an object with its front legs or performing multiple stepping of the front legs, to deal with morphological change, and to synchronize its movement with another robot during a collaborative task.
Abstract: Walking animals show versatile locomotion. They can also adapt their movement according to the changes of their morphology and the environmental conditions. These emergent properties are realized by biomechanics, distributed central pattern generators (CPGs), local sensory feedback, and their interactions during body and leg movements through the environment. Based on this concept, we present here an artificial bio-inspired walking system. Its intralimb coordination is formed by multiple decoupled CPGs while its interlimb coordination is attained by the interactions between body dynamics and the environment through local sensory feedback of each leg. Simulation results show that this bio-inspired approach generates self-organizing emergent locomotion allowing the robot to adaptively form regular patterns, to stably walk while pushing an object with its front legs or performing multiple stepping of the front legs, to deal with morphological change, and to synchronize its movement with another robot during a collaborative task.

18 citations


Book ChapterDOI
22 Jul 2014
TL;DR: This paper focuses on devising and analyzing a task allocation strategy that allows swarm robotics systems to execute tasks characterized by soft deadlines and to minimize the costs associated with missing the task deadlines.
Abstract: Developing swarm robotics systems for real-time applications is a challenging mission. Task deadlines are among the kind of constraints which characterize a large set of real applications. This paper focuses on devising and analyzing a task allocation strategy that allows swarm robotics systems to execute tasks characterized by soft deadlines and to minimize the costs associated with missing the task deadlines.

13 citations


Book ChapterDOI
22 Jul 2014
TL;DR: This work studies an intrinsic motivation system for behavioral self-exploration based on the maximization of the predictive information using the Stumpy robot, which is the first evaluation of the algorithm on a real robot.
Abstract: One of the long-term goals of artificial life research is to create autonomous, self-motivated, and intelligent animats. We study an intrinsic motivation system for behavioral self-exploration based on the maximization of the predictive information using the Stumpy robot, which is the first evaluation of the algorithm on a real robot. The control is organized in a closed-loop fashion with a reactive controller that is subject to fast synaptic dynamics. Even though the available sensors of the robot produce very noisy and peaky signals, the self-exploration algorithm was successful and various emerging behaviors were observed.

11 citations


Book ChapterDOI
22 Jul 2014
TL;DR: An agent-based model is presented to show how sensorsimotor attunement can be understood as a dynamic and non-representational process in which a particular sensorimotor coordination is enacted as a response to a given environmental context, without requiring deliberative action selection.
Abstract: The sensorimotor approach argues that in order to perceive one needs to first “master” the relevant sensorimotor contingencies, and then exercise the acquired practical know-how to become “attuned” to the actual and potential contingencies a particular situation entails. But the approach provides no further detail about how this mastery is achieved or what precisely it means to become attuned to a situation. We here present an agent-based model to show how sensorimotor attunement can be understood as a dynamic and non-representational process in which a particular sensorimotor coordination is enacted as a response to a given environmental context, without requiring deliberative action selection.

11 citations


Book ChapterDOI
22 Jul 2014
TL;DR: A robust multi modal compass for a vision based navigation system that mimics several aspects of the head direction cells found in the postsubiculum of the rat to maintain the temporal coherency of its behavior.
Abstract: In this paper, we study a robust multi modal compass for a vision based navigation system. The model mimics several aspects of the head direction cells found in the postsubiculum of the rat. Idiothetic information is recalibrated according to the learning of visual stimuli associated to robust landmarks. The model is based on dynamic neural fields allowing building attractors associated to the compass direction. The novelty of the model relies in the way the decision of the sensor fusion is re-injected in the visual compass allowing a robust decision-making. Robotics experiments show the capability of the model to merge different sources of information when their predictions are coherent. When the information become incoherent because the inputs propose quite different directions, the system is able to bifurcate on one coherent solution in order to maintain the temporal coherency of its behavior.

9 citations


Book ChapterDOI
22 Jul 2014
TL;DR: The present paper elaborates on time-cognition coupling suggesting that the equipment of artificial agents with human-like time perception and time processing capacities is a prerequisite for bringing robotic cognition close to human intelligence.
Abstract: Contemporary research endeavors aim at equipping autono-mous robots with human-like cognitive skills, in an attempt to promote robotic intelligence and make artificial agents more natural and more human-friendly However, despite the crucial role that sense of time has in our daily activities, the capacity of artificial agents to experience the flow of time remains largely unexplored The inability of existing systems to perceive time acts as an obstacle in implementing conscious artificial agents that put their experiences on the past-present-future timeline and develop durable symbiotic relationships with humans The present paper elaborates on time-cognition coupling suggesting that the equipment of artificial agents with human-like time perception and time processing capacities is a prerequisite for bringing robotic cognition close to human intelligence

8 citations


Book ChapterDOI
22 Jul 2014
TL;DR: The results show that a cerebellum based architecture can efficiently learn to reduce errors through anticipation and suggest that a sensory-to-sensory prediction could be less expensive in terms of energy cost and more robust when events violate the acquired prediction.
Abstract: Postural adjustments are acquired compensatory and anticipatory motor responses maintaining balance and equilibrium against self-induced or external perturbations. It has been proposed that the cerebellum could be involved in issuing such predictive motor actions. However, it remains unclear what strategy is adopted by the brain in order to make such prediction and how anticipatory and compensatory components are integrated into a single response. Within this study we are interested in the computational mechanisms underlying the acquisition of anticipatory responses in a postural task. We compare two alternative architectures representing two different hypotheses: anticipation either as sensory-to-motor association or as sensory-to-sensory association. We propose to use a cerebellar model to control the acquisition of an adaptive motor response in a simulated robotic setup. We devise a scenario where a cart-pole robot is trained to predict a perturbation and issue an anticipatory action to minimize the disturbance on its state of equilibrium. Our results show that a cerebellum based architecture can efficiently learn to reduce errors through anticipation. We also suggest that a sensory-to-sensory prediction could be less expensive in terms of energy cost and more robust when events violate the acquired prediction.

7 citations


Book ChapterDOI
22 Jul 2014
TL;DR: It is found that tracking with a fixed template fails very quickly in the course of a learning flight, but that continuously updating the template allowed us to reliably estimate nest direction in reconstructed image sequences.
Abstract: Ants, bees and wasps are central place foragers. They leave their nests to forage and routinely return to their home-base. Most are guided by memories of the visual panorama and the visual appearance of the local nest environment when pinpointing their nest. These memories are acquired during highly structured learning walks or flights that are performed when leaving the nest for the first time or whenever the insects had difficulties finding the nest during their previous return. Ground-nesting bees and wasps perform such learning flights daily when they depart for the first time. During these flights, the insects turn back to face the nest entrance and subsequently back away from the nest while flying along ever increasing arcs that are centred on the nest. Flying along these arcs, the insects counter-turn in such a way that the nest entrance is always seen in the frontal visual field at slightly lateral positions. Here we asked how the insects may achieve keeping track of the nest entrance location given that it is a small, inconspicuous hole in the ground, surrounded by complex natural structures that undergo unpredictable perspective transformations as the insect pivots around the area and gains distance from it. We reconstructed the natural visual scene experienced by wasps and bees during their learning flights and applied a number of template-based tracking methods to these image sequences. We find that tracking with a fixed template fails very quickly in the course of a learning flight, but that continuously updating the template allowed us to reliably estimate nest direction in reconstructed image sequences. This is true even for later sections of learning flights when the insects are so far away from the nest that they cannot resolve the nest entrance as a visual feature. We discuss why visual goal-anchoring is likely to be important during the acquisition of visual-spatial memories and describe experiments to test whether insects indeed update nest-related templates during their learning flights.

7 citations


Book ChapterDOI
22 Jul 2014
TL;DR: An agent-based model which is inspired by bacterial conjugation of DNA plasmids is presented, demonstrating that in a model based on free interactions among autonomous agents, optimal results emerge by incrementing heterogeneity levels and decentralizing communication structures, leading to a global adaptation of the system.
Abstract: Bacteria have demonstrated an amazing capacity to overcome environmental changes by collective adaptation through genetic exchanges. Using a distributed communication system and sharing individual strategies, bacteria propagate mutations as innovations that allow them to survive in different environments. In this paper we present an agent-based model which is inspired by bacterial conjugation of DNA plasmids. In our approach, agents with bounded rationality interact in a common environment guided by local rules, leading to Complex Adaptive Systems that are named ’artificial societies’. We have demonstrated that in a model based on free interactions among autonomous agents, optimal results emerge by incrementing heterogeneity levels and decentralizing communication structures, leading to a global adaptation of the system. This organic approach to model peer-to-peer dynamics in Complex Adaptive Systems is what we have named ‘bacterial-based algorithms’ because agents exchange strategic information in the same way that bacteria use conjugation and share genome.

Book ChapterDOI
22 Jul 2014
TL;DR: A population coding approach is applied to detect the orientation of the web vibration source and the result in the vibration experiments shows a distribution of theweb string tension effectively transfers the vibration of a spider web into another place.
Abstract: Most of the spiders can hunt a prey and avoid the threat of predators by sensing web vibration. Spiders have eight legs, and it has the sense organs to detect vibrations. These vibration sensing organs can be observed at the slit sensilla on each leg. A distribution of the web string tension effectivelytransfer the vibration of a spider web into another place. In order to investigate the characteristics of the spider web, we test various sensors in the artificial web. We apply a population coding approach to detect the orientation of the web vibration source. We demonstrate the result in the vibration experiments.

Book ChapterDOI
22 Jul 2014
TL;DR: The Voxbot is a cubic (voxel) shaped robot actuated by expansion and contraction of its 12 edges designed for running evolutionary experiments, built as cheaply as possible and very easy to build and replicate.
Abstract: The Voxbot is a cubic (voxel) shaped robot actuated by expansion and contraction of its 12 edges designed for running evolutionary experiments, built as cheaply as possible Each edge was made of a single 10ml medical syringe for pneumatic control These were connected to an array of 12 servos situated on an external housing and controlled with an Arduino microcontroller from a laptop With twenty motor primitive commands and the slow response of its pneumatics this robot allows real time controllers to be evolved in situ rather than just in simulation With simple combinations and sequencing of motor primitives the Voxbot can be made to walk, rotate and crab crawl The device is available in kit form and is very easy to build and replicate Other morphologies can be built easily

Book ChapterDOI
22 Jul 2014
TL;DR: An adaptive landmark-based navigation system based on sequential reinforcement learning is developed that allows the robots to successfully learn to navigate to distal goals in complex environments.
Abstract: The goal-directed navigational ability of animals is an essential prerequisite for them to survive. They can learn to navigate to a distal goal in a complex environment. During this long-distance navigation, they exploit environmental features, like landmarks, to guide them towards their goal. Inspired by this, we develop an adaptive landmark-based navigation system based on sequential reinforcement learning. In addition, correlation-based learning is also integrated into the system to improve learning performance. The proposed system has been applied to simulated simple wheeled and more complex hexapod robots. As a result, it allows the robots to successfully learn to navigate to distal goals in complex environments.

Book ChapterDOI
22 Jul 2014
TL;DR: The analysis of the self-organizing process of the parametric biases revealed an infant-like developmental change in action learning: the RNNPB first adapted to the goal and then to the means, causing this phased development.
Abstract: Developmental studies have suggested that infants’ action is goal-directed. When imitating an action, younger infants tend to reproduce the goal while ignoring the means (i.e., the movement to achieve the goal) whereas older infants can imitate both. We suggest that the developmental dynamics of a Recurrent Neural Network with Parametric Bias (RNNPB) may explain the mechanism of infant development. Our RNNPB model was trained to reproduce six types of actions (2 different goals x 3 different means), during which parametric biases were self-organized to represent the difference with respect to both the goal and means. Our analysis of the self-organizing process of the parametric biases revealed an infant-like developmental change in action learning: the RNNPB first adapted to the goal and then to the means. The different saliency of these two features caused this phased development. We discuss the analogy of our result to infant action development.

Book ChapterDOI
22 Jul 2014
TL;DR: A bio-inspired and developmental neural model is proposed that allows a robot, after learning its own dynamics during a babbling phase, to gain imitative and shape recognition abilities leading to early attempts for physical and social interactions.
Abstract: In this paper, we propose a bio-inspired and developmental neural model that allows a robot, after learning its own dynamics during a babbling phase, to gain imitative and shape recognition abilities leading to early attempts for physical and social interactions. We use a motor controller based on oscillators. During the babbling step, the robot learns to associate its motor primitives (oscillators) to the visual optical flow induced by its own arm. It also statically learn to recognize its arm by selecting moving local view (feature points) in the visual field. In real indoor experiments we demonstrate that, using the same model, early physical (reaching objects) and social (immediate imitation) interactions can emerge through visual ambiguities induced by the external visual stimuli.

Book ChapterDOI
22 Jul 2014
TL;DR: A motivational system for an agent undergoing reinforcement learning (RL), which enables it to balance multiple drives, each of which is satiated by different types of stimuli, using Minor Component Analysis to model the agent’s internal drive state.
Abstract: We present a motivational system for an agent undergoing reinforcement learning (RL), which enables it to balance multiple drives, each of which is satiated by different types of stimuli. Inspired by drive reduction theory, it uses Minor Component Analysis (MCA) to model the agent’s internal drive state, and modulates incoming stimuli on the basis of how strongly the stimulus satiates the currently active drive. The agent’s dynamic policy continually changes through least-squares temporal difference updates. It automatically seeks stimuli that first satiate the most active internal drives, then the next most active drives, etc. We prove that our algorithm is stable under certain conditions. Experimental results illustrate its behavior.

Book ChapterDOI
22 Jul 2014
TL;DR: A novel framework which allows modular robots to adapt physically as well as internally to achieve high-level tasks (e.g. to learn the behavior) and allows to achieve complex task easily without need to optimize complex behaviors of the robot.
Abstract: In future space missions, versatile, robust, autonomous and adaptive robotic systems will be required to perform complex tasks. This can be realized using modular robots with the ability to reconfigure to various structures, which allows them to adapt to the environment as well as to a given task. As it is not possible to program beforehand the robots to cope with every possible situation, they will have to adapt autonomously. In this paper, we introduce a novel framework which allows modular robots to adapt physically (i.e., to change the structure) as well as internally (i.e. to learn the behavior) to achieve high-level tasks (e.g. ’climb-up the cliff’). The framework utilizes evolutionary methods for structure adaptation as well as to find a suitable behavior. The main idea of the framework is the utilization of simple motion skills combined by a motion planner to achieve the high-level task. This allows to achieve complex task easily without need to optimize complex behaviors of the robot.

Book ChapterDOI
22 Jul 2014
TL;DR: This paper examines the estimation of robot egomotion from visual input by unsupervised online learning using a sparse optical flow field constructed from discrete motion detectors and shows how online linear Principal Component Analysis can be applied to enable a robot to continuously adapt to a changing environment.
Abstract: Adaptive behaviour of animats largely depends on the processing of their sensory information. In this paper, we examine the estimation of robot egomotion from visual input by unsupervised online learning. The input is a sparse optical flow field constructed from discrete motion detectors. The global flow field properties depend on the robot motion, the spatial distribution of motion detectors with respect to the robot body and the visual environment. We show how online linear Principal Component Analysis can be applied to this problem to enable a robot to continuously adapt to a changing environment.

Book ChapterDOI
22 Jul 2014
TL;DR: A control approach for teleoperated robot groups in which the shape of the formation can be reconfigured by a supervisor in which two haptic devices are used to implement the formation control.
Abstract: In this paper we propose a control approach for teleoperated robot groups in which the shape of the formation can be reconfigured by a supervisor. To implement the formation control, the supervisor uses two haptic devices: the first haptic device is used to control the leader of the formation; the second is used to modify the formation. The haptic (force) feedbacks reflect the presence of obstacles: the first reflects the proximity of the obstacles from the formation leader; the second reflects the nearness of the obstacles from the center of the formation. An obstacle avoidance algorithm was also proposed for the members of the robot group. A simulation environment was developed to analyze the behavior of the proposed robot formation control approach. The simulated formation can be controlled through the Internet using real haptic devices.

Book ChapterDOI
22 Jul 2014
TL;DR: This work investigates reaction times for classification of visual stimuli composed of combinations of shapes, to distinguish between parallel and serial processing of stimuli.
Abstract: We investigate reaction times for classification of visual stimuli composed of combinations of shapes, to distinguish between parallel and serial processing of stimuli. Reaction times in a visual XOR task are slower than in AND/OR tasks in which pairs of shapes are categorised. This behaviour is explained by the time needed to perceive shapes in the various tasks, using a parallel drift diffusion model. The parallel model explains reaction times in an extension of the XOR task, up to 7 shapes. Subsequently, the behaviour is explained by a combined model that assumes perceptual chunking, processing shapes within chunks in parallel, and chunks themselves in serial. The pure parallel model also explains reaction times for ALL and EXISTS tasks. An extension to the perceptual chunking model adds time taken to apply a logical rule. We are able to improve the fit to the data by including this extra parameter, but using model selection the extra parameter is not supported. We further simulate the behaviour exhibited using an echo state network, successfully recreating the behaviour seen in humans.

Book ChapterDOI
22 Jul 2014
TL;DR: The concept of Deferred Restructuring of Experience in Autonomous Machines (DREAM) is postulated and applied in the context of the Multilevel Darwinist Brain architecture to ensure that the robot becomes more efficient and adaptive in its subsequent interactions with the world.
Abstract: This paper is concerned with the exploration of the benefits that can be derived within a cognitive architecture for robots through the application of nature inspired sleep related cognitive restructuring processes To this end, the concept of Deferred Restructuring of Experience in Autonomous Machines (DREAM) is postulated and applied in the context of the Multilevel Darwinist Brain architecture This concept implies a series of consolidation, enhancement and internal imaging based exploration processes that can be applied over the experience, in terms of models and behavioral structures, a robot has acquired in its interaction with the world during its lifetime The result is a re-representation of all of this experience so that the robot becomes more efficient and adaptive in its subsequent interactions with the world A couple of simple proof of concept experiments demonstrate the capabilities of the approach

Book ChapterDOI
22 Jul 2014
TL;DR: The novel planning method that uses an inverse kinematics solver to build task-relevant roadmaps by searching the configuration space via the Natural Evolution Strategies (NES) algorithm is scaled-up to a fully-parallelized implementation where additional constraints coordinate the interaction between independent NES searches running on separate threads.
Abstract: Planning movements for humanoid robots is still a major challenge due to the very high degrees-of-freedom involved. Most humanoid control frameworks incorporate dynamical constraints related to a task that require detailed knowledge of the robot’s dynamics, making them impractical as efficient planning. In previous work, we introduced a novel planning method that uses an inverse kinematics solver called Natural Gradient Inverse Kinematics (NGIK) to build task-relevant roadmaps (graphs in task space representing robot configurations that satisfy task constraints) by searching the configuration space via the Natural Evolution Strategies (NES) algorithm. The approach places minimal requirements on the constraints, allowing for complex planning in the task space. However, building a roadmap via NGIK is too slow for dynamic environments. In this paper, the approach is scaled-up to a fully-parallelized implementation where additional constraints coordinate the interaction between independent NES searches running on separate threads. Parallelization yields a 12× speedup that moves this promising planning method a major step closer to working in dynamic environments.

Book ChapterDOI
22 Jul 2014
TL;DR: This work pursues an approach in which decision making is considered as a multiobjective problem and approximately solved using a hierarchical reinforcement learning architecture, and executes the selected strategy while interacting with a continuous, partially observable environment.
Abstract: Goal-driven agents are generally expected to be capable of pursuing simultaneously a variety of goals. As these goals may compete in certain circumstances, the agent must be able to constantly trade them off and shift their priorities in a rational way. One aspect of rationality is to evaluate its needs and make decisions accordingly. We endow the agent with a set of needs, or drives, that change over time as a function of external stimuli and internal consumption, and the decision making process hast to generate actions that maintain balance between these needs. The proposed framework pursues an approach in which decision making is considered as a multiobjective problem and approximately solved using a hierarchical reinforcement learning architecture. At a higher-level, a Q-learning learns to select the best learning strategy that improves the well-being of the agent. At a lower-level, an actor-critic design executes the selected strategy while interacting with a continuous, partially observable environment. We provide simulation results to demonstrate the efficiency of the approach.

Book ChapterDOI
22 Jul 2014
TL;DR: This paper presents an automatic method that exploits the system’s capabilities in order to find a linear chain of soft cells that self-folds into a target 2-D shape.
Abstract: Programmable self-assembly of chained modules holds potential for the automatic shape formation of morphologically adapted robots. However, current systems are limited to modules of uniform rigidity, which restricts the range of obtainable morphologies and thus the functionalities of the system. To address these challenges, we previously introduced “soft cells” as modules that can obtain different mechanical softness pre-setting. We showed that such a system can obtain a higher diversity of morphologies compared to state-of-the-art systems and we illustrated the system’s potential by demonstrating the self-assembly of complex morphologies. In this paper, we extend our previous work and present an automatic method that exploits our system’s capabilities in order to find a linear chain of soft cells that self-folds into a target 2-D shape.

Book ChapterDOI
22 Jul 2014
TL;DR: This paper aims to shed light on the benefits of the cognitive processes in the generation of emergent structures that allow the cognitive robots to succeed the objects’ aggregation task.
Abstract: This paper aims to shed light on the benefits of the cognitive processes in the generation of emergent structures that allow the cognitive robots to succeed the objects’ aggregation task. In the multi-robot system, every robot uses local rules and an on-line building and learning of its own cognitive map. This fusion alters the positive impact of the individual behavior in the improvement of the overall system performance. A series of simulations and experiments allowed us to present and discuss the system.

Book ChapterDOI
22 Jul 2014
TL;DR: This work provides robots with general monitoring strategies based on attentional mechanisms, for filtering data and actively focusing only on relevant information in response to human-robot teamwork.
Abstract: Human-robot teamwork requires agents to pay attention to both surrounding environment and teammates. Bandwidth and computational limitations prevent an agent to continuously execute this monitoring activity. Inspired by the behavior of human beings, paying frequent attention to timers while approaching deadlines, we provide robots with general monitoring strategies based on attentional mechanisms, for filtering data and actively focusing only on relevant information. We consider a convoy task (led by a human or a robot) as a benchmark to evaluate and compare human and robot monitoring behaviors.

Book ChapterDOI
22 Jul 2014
TL;DR: A recent framework for integrating reinforcement learning and dynamic neural fields is extended, by using the principle of shaping, in order to reduce the search space of the learning agent.
Abstract: We present here a simulated model of a mobile Kuka Youbot which makes use of Dynamic Field Theory for its underlying perceptual and motor control systems, while learning behavioral sequences through Reinforcement Learning. Although dynamic neural fields have previously been used for robust control in robotics, high-level behavior has generally been pre-programmed by hand. In the present work we extend a recent framework for integrating reinforcement learning and dynamic neural fields, by using the principle of shaping, in order to reduce the search space of the learning agent.

Book ChapterDOI
22 Jul 2014
TL;DR: RANA, in its present stage of development, is shown to be able to handle the problem of modelling calling frogs, and several fruitful extensions are proposed and motivated.
Abstract: A new agent-based modelling tool has been developed to allow the modelling of populations of individuals whose interactions are characterised by tightly timed dynamics. The tool was developed to model male frog calling dynamics, to facilitate research into what local rules may be employed by individuals to generate their observed population behaviour. A number of existing agent-modelling frameworks are considered, but none have the ability to handle large numbers of time-dependent event-generating agents; hence the construction of a new tool, RANA. The calling behaviour of the Puerto Rican Tree Frog, E. coqui, is implemented as a case study for the presentation and discussion of the tool, and results from this model are presented. RANA, in its present stage of development, is shown to be able to handle the problem of modelling calling frogs, and several fruitful extensions are proposed and motivated.

Book ChapterDOI
22 Jul 2014
TL;DR: BeeIP is compared with the state-of-the-art DSR, AODV and its multipath version AOMDV using four benchmark performance metrics for both TCP and UDP traffic and is shown second best in terms of control overhead for both transport layer protocols.
Abstract: This paper discusses BeeIP, a reactive multipath routing protocol inspired by honeybees, and examines its performance for both connection-oriented and connectionless traffic within mobile ad hoc networks using a new modification to the algorithm for artificial swarming. Artificial agents follow concepts borrowed from the communication and foraging activities of real honeybees to detect new routing paths and maintain successful and robust data traffic. Paths are evaluated by constantly monitoring their quality based on a list of well-defined low-level parameters. The protocol is compared with the state-of-the-art DSR, AODV and its multipath version AOMDV using four benchmark performance metrics for both TCP and UDP traffic. The results suggest that BeeIP is able to achieve high packet delivery ratio, end-to-end delay and average receiving throughput, while it is shown second best in terms of control overhead for both transport layer protocols.