In this paper we explore the potential of mobile robots with simulation-based internal models for safety in highly dynamic environments. We propose a robot with a simulation of itself, other dynamic actors and its environment, inside itself. Operating in real-time this simulation-based internal model is able to look ahead and predict the consequences of both the robot's own actions and those of the other dynamic actors in its vicinity. Hence the robot continuously modifies its own actions in order to actively maintain its own safety while also achieving its goal. Inspired by the problem of how mobile robots could move quickly and safely through crowds of moving humans, we present experimental results which compare the performance of our internal simulation-based controller with a purely reactive approach as a proof-of-concept study for the practical use of simulation-based internal models.

/pdf/simulation-based-internal-models-for-safer-robots-49paohklza.pdf

Simulation-based internal models for safer robots

Reinforcement learning provides a general framework for learning robotic skills while minimizing engineering effort. However, most reinforcement learning algorithms assume that a well-designed reward function is provided, and learn a single behavior for that single reward function. Such reward functions can be difficult to design in practice. Can we instead develop efficient reinforcement learning methods that acquire diverse skills without any reward function, and then repurpose these skills for downstream tasks? In this paper, we demonstrate that a recently proposed unsupervised skill discovery algorithm can be extended into an efficient off-policy method, making it suitable for performing unsupervised reinforcement learning in the real world. Firstly, we show that our proposed algorithm provides substantial improvement in learning efficiency, making reward-free real-world training feasible. Secondly, we move beyond the simulation environments and evaluate the algorithm on real physical hardware. On quadrupeds, we observe that locomotion skills with diverse gaits and different orientations emerge without any rewards or demonstrations. We also demonstrate that the learned skills can be composed using model predictive control for goal-oriented navigation, without any additional training.

Emergent Real-World Robotic Skills via Unsupervised Off-Policy Reinforcement Learning

Value systems for developmental cognitive robotics

Recently, two theoretical models for morphological computation have been proposed [13,14]. Based on a rigorous mathematical framework and simulations it has been demonstrated that compliant, complex physical bodies can be effectively employed as computational resources. Even more recent work showed that these models are not only of theoretical nature, but are also applicable to a number of different soft robotic platforms. Motivated by these encouraging results we discuss a number of remarkable implications when real physical bodies are employed as computational resources. Morphological Computation – The Physical Body as a Computational Resource

https://research-information.bris.ac.uk/files/53583202/Chapter_20_Hauser.pdf

Morphological Computation: The Body as a Computational Resource

Active inference is an ambitious theory which treats perception and inference on one side, and action selection on the other side under the heading of a single principle. It also provides biologically plausible explanations for many cognitive phenomena, including consciousness. Action selection in active inference is driven by an objective function which evaluates possible future actions with respect to current, inferred beliefs about the world. Such independence from extrinsic rewards comes with a high level of robustness across e.g. different environments or agent morphologies. In the literature, more functions with these favourable properties have been summarised under the notion of intrinsic motivations. In contrast to active inference however, these formal models of motivation usually come without a commitment to particular inference and action selection mechanisms. Up to now it was not clear whether the inference and action selection machinery of active inference could also be used by alternatives to the originally included intrinsic motivation. Here we reconstruct the active inference approach, locate the original formulation within, and show how alternative intrinsic motivations can be used while keeping many of the original features intact. The perception-action loop serves as unifying formal concept to relate different components within active inference. Active inference research may profit from comparisons of the dynamics induced by alternative intrinsic motivations. Research on intrinsic motivations may profit from an additional way to implement intrinsically motivated agents that may also share the biological plausibility of active inference.

/pdf/expanding-the-active-inference-landscape-more-intrinsic-rxegqxsnal.pdf

Expanding the Active Inference Landscape: More Intrinsic Motivations in the Perception-Action Loop.

One of the long-term goals of artificial life research is to create autonomous, self-motivated, and intelligent animats. We study an intrinsic motivation system for behavioral self-exploration based on the maximization of the predictive information using the Stumpy robot, which is the first evaluation of the algorithm on a real robot. The control is organized in a closed-loop fashion with a reactive controller that is subject to fast synaptic dynamics. Even though the available sensors of the robot produce very noisy and peaky signals, the self-exploration algorithm was successful and various emerging behaviors were observed.

/pdf/self-exploration-of-the-stumpy-robot-with-predictive-5fsm3emq65.pdf

Luisa Jahn

Papers

Self-exploration of the Stumpy Robot with Predictive Information Maximization