The PASCAL Visual Object Classes Challenge

Ageing societies and the associated pressure on the care systems are major drivers for new developments in socially assistive robotics. To understand better the real-world potential of robot-based assistance, we undertook a 10-week case study in a care home involving groups of residents, caregivers and managers as stakeholders. We identified both, enablers and barriers to the potential implementation of robot systems. The study employed the robot platform Pepper, which was deployed with a view to understanding better multi-domain interventions with a robot supporting physical activation, cognitive training and social facilitation. We employed the robot in a group setting in a care facility over the course of 10 weeks and 20 sessions, observing how stakeholders, including residents and caregivers, appropriated, adapted to, and perceived the robot. We also conducted interviews with 11 residents and caregivers. Our results indicate that the residents were positively engaged in the training sessions that were moderated by the robot. The study revealed that such humanoid robots can work in a care home but that there is a moderating person needed, that is in control of the robot.

Exploring Human-Robot Interaction with the Elderly: Results from a Ten-Week Case Study in a Care Home

Artificial intelligence (AI) and machine learning (ML) tools play a significant role in the recent evolution of smart systems. AI solutions are pushing towards a significant shift in many fields such as healthcare, autonomous airplanes and vehicles, security, marketing customer profiling and other diverse areas. One of the main challenges hindering the AI potential is the demand for high-performance computation resources. Recently, hardware accelerators are developed in order to provide the needed computational power for the AI and ML tools. In the literature, hardware accelerators are built using FPGAs, GPUs and ASICs to accelerate computationally intensive tasks. These accelerators provide high-performance hardware while preserving the required accuracy. In this work, we present a systematic literature review that focuses on exploring the available hardware accelerators for the AI and ML tools. More than 169 different research papers published between the years 2009 and 2019 are studied and analysed.

A systematic literature review on hardware implementation of artificial intelligence algorithms

In this letter, we propose a map-less visual navigation system for biped humanoid robots, which extracts information from color images to derive motion commands using deep reinforcement learning (DRL). The map-less visual navigation policy is trained using the Deep Deterministic Policy Gradients (DDPG) algorithm, which corresponds to an actor-critic DRL algorithm. The algorithm is implemented using two separate networks, one for the actor and one for the critic, but with similar structures. In addition to convolutional and fully connected layers, Long Short-Term Memory (LSTM) layers are included to address the limited observability present in the problem. As a proof of concept, we consider the case of robotic soccer using humanoid NAO V5 robots, which have reduced computational capabilities, and low-cost Red - Green - Blue (RGB) cameras as main sensors. The use of DRL allowed to obtain a complex and high performant policy from scratch, without any prior knowledge of the domain, or the dynamics involved. The visual navigation policy is trained in a robotic simulator and then successfully transferred to a physical robot, where it is able to run in 20 ms, allowing its use in real-time applications.

Visual Navigation for Biped Humanoid Robots Using Deep Reinforcement Learning

Wildlife conservation organizations task rangers to deter and capture wildlife poachers. Since rangers are responsible for patrolling vast areas, adversary behavior modeling can help more effectively direct future patrols. In this innovative application track paper, we present an adversary behavior modeling system, INTERCEPT (INTERpretable Classification Ensemble to Protect Threatened species), and provide the most extensive evaluation in the AI literature of one of the largest poaching datasets from Queen Elizabeth National Park (QENP) in Uganda, comparing INTERCEPT with its competitors; we also present results from a month-long test of INTERCEPT in the field. We present three major contributions. First, we present a paradigm shift in modeling and forecasting wildlife poacher behavior. Some of the latest work in the AI literature (and in Conservation) has relied on models similar to the Quantal Response model from Behavioral Game Theory for poacher behavior prediction. In contrast, INTERCEPT presents a behavior model based on an ensemble of decision trees (i) that more effectively predicts poacher attacks and (ii) that is more effectively interpretable and verifiable. We augment this model to account for spatial correlations and construct an ensemble of the best models, significantly improving performance. Second, we conduct an extensive evaluation on the QENP dataset, comparing 41 models in prediction performance over two years. Third, we present the results of deploying INTERCEPT for a one-month field test in QENP - a first for adversary behavior modeling applications in this domain. This field test has led to finding a poached elephant and more than a dozen snares (including a roll of elephant snares) before they were deployed, potentially saving the lives of multiple animals - including elephants.

Cloudy with a Chance of Poaching: Adversary Behavior Modeling and Forecasting with Real-World Poaching Data

Decentralized reinforcement learning of robot behaviors

Hierarchical task decomposition strategies allow robots and agents in general to address complex decision-making tasks. Layered learning is a hierarchical machine learning paradigm where a complex behavior is learned from a series of incrementally trained sub-tasks. This paper describes how layered learning can be applied to design individual behaviors in the context of soccer robotics. Three different layered learning strategies are implemented and analyzed using a ball-dribbling behavior as a case study. Performance indices for evaluating dribbling speed and ball-control are defined and measured. Experimental results validate the usefulness of the implemented layered learning strategies showing a trade-off between performance and learning speed.

/pdf/a-study-of-layered-learning-strategies-applied-to-individual-3l38fimo3m.pdf

A Study of Layered Learning Strategies Applied to Individual Behaviors in Robot Soccer

The main goal of this paper is to analyze the general problem of using Convolutional Neural Networks (CNNs) in robots with limited computational capabilities, and to propose general design guidelines for their use. In addition, two different CNN based NAO robot detectors that are able to run in real-time while playing soccer are proposed. One of the detectors is based on the XNOR-Net and the other on the SqueezeNet. Each detector is able to process a robot object-proposal in ~1ms, with an average number of 1.5 proposals per frame obtained by the upper camera of the NAO. The obtained detection rate is ~97%.

Using Convolutional Neural Networks in Robots with Limited Computational Resources: Detecting NAO Robots while Playing Soccer

In the context of the humanoid robotics soccer, ball dribbling is a complex and challenging behavior that requires a proper interaction of the robot with the ball and the floor. We propose a methodology for modeling this behavior by splitting it in two sub problems: alignment and ball pushing. Alignment is achieved using a fuzzy controller in conjunction with an automatic foot selector. Ball-pushing is achieved using a reinforcement-learning based controller, which learns how to keep the robot near the ball, while controlling its speed when approaching and pushing the ball. Four different models for the reinforcement learning of the ball-pushing behavior are proposed and compared. The entire dribbling engine is tested using a 3D simulator and real NAO robots. Performance indices for evaluating the dribbling speed and ball-control are defined and measured. The obtained results validate the usefulness of the proposed methodology, showing asymptotic convergence in around fifty training episodes, and similar performance between simulated and real robots.

/pdf/ball-dribbling-for-humanoid-biped-robots-a-reinforcement-4ymhvmjnkb.pdf

Ball Dribbling for Humanoid Biped Robots: A Reinforcement Learning and Fuzzy Control Approach

Deep Reinforcement Learning (DRL) has become a powerful strategy to solve complex decision making problems based on Deep Neural Networks (DNNs). However, it is highly data demanding, so unfeasible in physical systems for most applications. In this work, we approach an alternative Interactive Machine Learning (IML) strategy for training DNN policies based on human corrective feedback, with a method called Deep COACH (D-COACH). This approach not only takes advantage of the knowledge and insights of human teachers as well as the power of DNNs, but also has no need of a reward function (which sometimes implies the need of external perception for computing rewards). We combine Deep Learning with the COrrective Advice Communicated by Humans (COACH) framework, in which non-expert humans shape policies by correcting the agent’s actions during execution. The D-COACH framework has the potential to solve complex problems without much data or time required. Experimental results validated the efficiency of the framework in three different problems (two simulated, one with a real robot), with state spaces of low and high dimensions, showing the capacity to successfully learn policies for continuous action spaces like in the Car Racing and Cart-Pole problems faster than with DRL.

/pdf/interactive-learning-with-corrective-feedback-for-policies-1908qp5mp7.pdf

Javier Ruiz-del-Solar

Papers

Decentralized reinforcement learning of robot behaviors

A Study of Layered Learning Strategies Applied to Individual Behaviors in Robot Soccer

Using Convolutional Neural Networks in Robots with Limited Computational Resources: Detecting NAO Robots while Playing Soccer

Ball Dribbling for Humanoid Biped Robots: A Reinforcement Learning and Fuzzy Control Approach

Interactive Learning with Corrective Feedback for Policies Based on Deep Neural Networks