Multi-task learning is commonly used in autonomous driving for solving various visual perception tasks. It offers significant benefits in terms of both performance and computational complexity. Current work on multi-task learning networks focus on processing a single input image and there is no known implementation of multi-task learning handling a sequence of images. In this work, we propose a multi-stream multi-task network to take advantage of using feature representations from preceding frames in a video sequence for joint learning of segmentation, depth, and motion. The weights of the current and previous encoder are shared so that features computed in the previous frame can be leveraged without additional computation. In addition, we propose to use the geometric mean of task losses as a better alternative to the weighted average of task losses. The proposed loss function facilitates better handling of the difference in convergence rates of different tasks. Experimental results on KITTI, Cityscapes and SYNTHIA datasets demonstrate that the proposed strategies outperform various existing multi-task learning solutions.

/pdf/multinet-multi-stream-feature-aggregation-and-geometric-loss-1oittk5eo9.pdf

MultiNet++: Multi-Stream Feature Aggregation and Geometric Loss Strategy for Multi-Task Learning

Control policies for home automation robots can be learned from human demonstrations, and hierarchical control has the potential to reduce the required number of demonstrations. When learning multiple policies for related tasks, demonstrations can be reused between the tasks to further reduce the number of demonstrations needed to learn each new policy. We present HIL-MT, a framework for Multi-Task Hierarchical Imitation Learning, involving a human teacher, a networked Toyota HSR robot, and a cloud-based server that stores demonstrations and trains models. In our experiments, HIL-MT learns a policy for clearing a table of dishes from 11.2 demonstrations on average. Learning to set the table requires 19 new demonstrations when training separately, but only 11.6 new demonstrations when also reusing demonstrations of clearing the table. HIL-MT learns policies for building 3- and 4-level pyramids of glass cups from 8.2 and 5 demonstrations, respectively, but reusing the 3-level demonstrations for learning a 4-level policy only requires 2.7 new demonstrations. These results suggest that learning hierarchical policies for structured domestic tasks can reuse existing demonstrations of related tasks to reduce the need for new demonstrations.

/pdf/multi-task-hierarchical-imitation-learning-for-home-12lbvlrwfg.pdf

Multi-Task Hierarchical Imitation Learning for Home Automation

Driven by recent technological advancements within the field of artificial intelligence (AI), deep learning (DL) has been emerged as a promising representation learning technique across different machine learning (ML) classes, especially within the reinforcement learning (RL) arena. This new direction has given rise to the evolution of a new technological domain named deep reinforcement learning (DRL) that combines the high representational learning capabilities of DL with existing RL methods. Performance optimization achieved by RL-based intelligent agents designed with model-free-based approaches was majorly limited to systems with RL algorithms focused on learning a single task. The aforementioned approach was found to be quite data inefficient, whenever DRL agents needed to interact with more complex, data-rich environments. This is primarily due to the limited applicability of DRL algorithms to many scenarios across related tasks from the same distribution. One of the possible approaches to mitigate this issue is by adopting the method of multi-task learning. The objective of this research paper is to present a hybrid multi-task learning-oriented approach for the optimization of DRL agents operating within different but semantically similar environments with related tasks. The proposed framework will be built with multiple, individual actor-critic models functioning within independent environments and transferring knowledge among themselves through a global network to optimize performance. The empirical results obtained by the hybrid multi-task learning model on OpenAI Gym based Atari 2600 video gaming environment demonstrates that the proposed model enhances the performance of the DRL agent relatively in the range of 15% to 20% margin.

/pdf/a-hybrid-multi-task-learning-approach-for-optimizing-deep-h2y2sso6es.pdf

A Hybrid Multi-Task Learning Approach for Optimizing Deep Reinforcement Learning Agents

MultiNet++: Multi-Stream Feature Aggregation and Geometric Loss Strategy for Multi-Task Learning.

This paper discusses the concept and parameter design of a robust stair-climbing compliant modular robot, capable of tackling stairs with overhangs. Geometry modifications of the periphery of the wheels of our robot helped in tackling overhangs. Along with establishing a concept design, the robust design parameters are set to minimize performance variations. The Grey-based Taguchi method is applied to provide an optimal setting for the design parameters of the robot. The robot prototype is shown to have successfully scaled stairs of varying dimensions, with overhang, thus corroborating the analysis performed.

Design of a robust stair-climbing compliant modular robot to tackle overhang on stairs

Stair Climbing is a key functionality desired for robots deployed in Urban Search and Rescue (USAR) scenarios. A novel compliant modular robot was proposed earlier to climb steep and big obstacles. This work extends the functionality of this robot to ascend and descend stairs of dimensions that are also typical of an urban setting. Stair Climbing is realized by equipping the robot's link joints with optimally designed passive spring pairs that resist clockwise and counter clockwise moments generated by the ground during the climbing motion. This 3-module robot is only propelled by wheel actuators. Desirable stair climbing configurations are estimated a-priori and used to obtain the optimal stiffness for springs. Extensive numerical simulation results over different stair configurations are shown. The numerical simulations are corroborated by experimentation using the prototype and its performance is tabulated for different types of surfaces.

Stair Climbing using a compliant modular robot

Most reinforcement learning algorithms are inefficient for learning multiple tasks in complex robotic systems, where different tasks share a set of actions. In such environments a compound policy may be learnt with shared neural network parameters, which performs multiple tasks concurrently. However such compound policy may get biased towards a task or the gradients from different tasks negate each other, making the learning unstable and sometimes less data efficient. In this paper, we propose a new approach for simultaneous training of multiple tasks sharing a set of common actions in continuous action spaces, which we call as DiGrad (Differential Policy Gradient). The proposed framework is based on differential policy gradients and can accommodate multi-task learning in a single actor-critic network. We also propose a simple heuristic in the differential policy gradient update to further improve the learning. The proposed architecture was tested on 8 link planar manipulator and 27 degrees of freedom(DoF) Humanoid for learning multi-goal reachability tasks for 3 and 2 end effectors respectively. We show that our approach supports efficient multi-task learning in complex robotic systems, outperforming related methods in continuous action spaces.

DiGrad: Multi-Task Reinforcement Learning with Shared Actions.

The purpose of this paper is to present the model of a Humanoid robot inspired by Poppy, modified for heavier load capacity and the balancing of humanoid in multiple work environments. The design has been modified in order to use MX-64 servos with more torque capacity than MX-28 servos which were used in original design. We have also redesigned the ankle joint and feet to make it a more accurate human like model. We are using an integrated approach using zero moment point (ZMP) with force sensing resistor (FSR) and center of mass (CoM) to find balance margins of the robot. Balancing and forward bending experiments on robot are conducted by providing some basic motion to the robot and ensuring that the robot is balanced and moving within safe margins using this approach.

Design and development of a humanoid with articulated torso

This work focuses on enhancing step descending ability of the modular robot proposed in [16]. The proposed robot consists of three modules connected with each other through passive joints. It is propelled using an active pair of wheels per module. Since there are no actuators at the joints, the joints are not susceptible to losing operability while traversing on rugged terrain. However with the absence of actuators, we face the issue of the robot toppling over when an abnormally large obstacle is encountered. This shortcoming is overcome with the use of compliant joints. The compliant joints are designed by employing springs of optimal stiffness, which is calculated through an optimization formulation aided with the constraints presented by the static analysis of the robot. The novelty lies in the systematic design of compliant joint for step descent. The robot is successful in climbing and descending obstacles of dimension 17 cm. Simulations of the mathematically modelled robot are carried out. The results from the same are validated on a working prototype and presented.

An improved compliant joint design of a modular robot for descending big obstacles

With the advent of artificial intelligence and machine learning, humanoid robots are made to learn a variety of skills which humans possess. One of fundamental skills which humans use in day-to-day activities is performing tasks with coordination between both the hands. In case of humanoids, learning such skills require optimal motion planning which includes avoiding collisions with the surroundings. In this paper, we propose a framework to learn coordinated tasks in cluttered environments based on DiGrad - A multi-task reinforcement learning algorithm for continuous action-spaces. Further, we propose an algorithm to smooth the joint space trajectories obtained by the proposed framework in order to reduce the noise instilled during training. The proposed framework was tested on a 27 degrees of freedom (DoF) humanoid with articulated torso for performing coordinated object-reaching task with both the hands in four different environments with varying levels of difficulty. It is observed that the humanoid is able to plan collision free trajectory in real-time. Simulation results also reveal the usefulness of the articulated torso for performing tasks which require coordination between both the arms.

S. Phani Teja

Papers

Stair Climbing using a compliant modular robot

DiGrad: Multi-Task Reinforcement Learning with Shared Actions.

Design and development of a humanoid with articulated torso

An improved compliant joint design of a modular robot for descending big obstacles

Learning Coordinated Tasks using Reinforcement Learning in Humanoids.