Learning and then recognizing a route, whether travelled during the day or at night, in clear or inclement weather, and in summer or winter is a challenging task for state of the art algorithms in computer vision and robotics. In this paper, we present a new approach to visual navigation under changing conditions dubbed SeqSLAM. Instead of calculating the single location most likely given a current image, our approach calculates the best candidate matching location within every local navigation sequence. Localization is then achieved by recognizing coherent sequences of these “local best matches”. This approach removes the need for global matching performance by the vision front-end - instead it must only pick the best match within any short sequence of images. The approach is applicable over environment changes that render traditional feature-based techniques ineffective. Using two car-mounted camera datasets we demonstrate the effectiveness of the algorithm and compare it to one of the most successful feature-based SLAM algorithms, FAB-MAP. The perceptual change in the datasets is extreme; repeated traverses through environments during the day and then in the middle of the night, at times separated by months or years and in opposite seasons, and in clear weather and extremely heavy rain. While the feature-based method fails, the sequence-based algorithm is able to match trajectory segments at 100% precision with recall rates of up to 60%.

SeqSLAM : visual route-based navigation for sunny summer days and stormy winter nights

The wide interest of research and industry in the human–robot interaction HRI related topics is proportional to the increased productivity and flexibility of the production lines, as it combines human and robot capabilities. This paper presents a review of recent research and progress on HRI, related to task planning/coordination and programming with emphasis on the manufacturing/production environment. Human–robot task allocation and scheduling, metrics for HRI, as well as the social aspects are reviewed. The role of digital human modelling systems for human–robot task planning related issues is also discussed. The process of learning by demonstration as well as the instructive systems is reviewed, focussing mainly on programming through visual guidance and imitation, voice commands and haptic interaction. The aspect of physical HRI as well as the safety related issues are also discussed. Additionally, a survey on multimodal communication frameworks is presented. Challenges encountered and directions for future research are discussed.

Human–robot interaction review and challenges on task planning and programming

This is a study of a Human-Robot Collaboration HRC framework for the execution of collaborative tasks in hybrid assembly cells. Robots and humans coexist in the same cell and share tasks according to their capabilities. An intelligent decision-making method that allows human-robot task allocation is proposed and is integrated within a Robot Operating System ROS framework. The proposed method enables the allocation of sequential tasks assigned to a robot and a human in separate workspaces. The focus is rather given to the human–robot coexistence for the execution of sequential tasks, in order for the automation level in manual or even hybrid assembly lines to be increased. Body gestures are the means of a human’s interaction with a robot for commanding and guiding reasons. The proposed framework is implemented into a case coming from the manual assembly lines of an automotive industry. A preliminary design of a hybrid assembly cell is presented, focusing on the assembly of a hydraulic pump by robots and humans.

https://tandfonline.com/doi/pdf/10.1080/0951192X.2016.1187297

On a human-robot collaboration in an assembly cell

A wide range of human–robot collaborative applications in diverse domains, such as manufacturing, health care, the entertainment industry, and social interactions, require an autonomous robot to fo...

Person-following by autonomous robots: A categorical overview:

The very first step in many facial analysis systems is face detection. Though face detection has been studied for many years, there is not still a benchmark public database to be widely accepted among researchers for which both color and depth information are obtained by the same sensor. Most of the available 3d databases have already automatically or manually detected the face images and they are therefore mostly used for face recognition not detection. This paper purposes an RGB-D database containing 1581 images (and their depth counterparts) taken from 31 persons in 17 different poses and facial expressions using a Kinect device. The faces in the images are not extracted neither in the RGB images nor in the depth hereof, therefore they can be used for both detection and recognition. The proposed database has been used in a face detection algorithm which is based on the depth information of the images. The challenges and merits of the database have been highlighted through experimental results.

/pdf/an-rgb-d-database-using-microsoft-s-kinect-for-windows-for-22hm9e3wp3.pdf

An RGB-D Database Using Microsoft's Kinect for Windows for Face Detection

To achieve an improved human-robot interaction it is necessary to allow the human participant to interact with the robot in a natural way. In this work, a gesture recognition algorithm, based on dynamic time warping, was implemented with a use-case scenario of natural interaction with a mobile robot. Inputs are gesture trajectories obtained using a Microsoft Kinect sensor. Trajectories are stored in the person's frame of reference. Furthermore, the recognition is position-invariant, meaning that only one learned sample is needed to recognize the same gesture performed at another position in the gestural space. In experiments, a set of gestures for a robot waiter was used to train the gesture recognition algorithm. The experimental results show that the proposed modifications of the standard gesture recognition algorithm improve the robustness of the recognition.

Position-invariant, real-time gesture recognition based on dynamic time warping

This paper presents a generalized method for the design of a gesture vocabulary (GV) for intuitive and natural two-way human-robot dialog. Two GV design methodologies are proposed; one for a robot GV (RGV) and a second for a human GV (HGV). The design is based on motion gestures exerted from a cohort of subjects in response to a set of tasks needed to execute several robot waiter (RW)-customer dialogs. Using a RW setting as a case study, preliminary experimental results indicate the unique nature of the HGV obtained.

Dynamic gesture vocabulary design for intuitive human-robot dialog

For a human-robot interaction to take place, a robot needs to perceive humans. The space where a robot can perceive humans is restrained by the limitations of robot's sensors. These restrictions can be circumvented by the use of external sensors, like in intelligent environments; otherwise humans have to ensure that they can be perceived. With the robotic platform presented here, the roles are reversed and the robot autonomously ensures that the human is within the area perceived by the robot. This is achieved by a combination of hardware and algorithms capable of autonomously tracking the person, estimating their position and following them, while recognizing their gestures and navigating through environment.

Spatially unconstrained, gesture-based human-robot interaction

There is still a way to go to achieve intuitive interaction between humanoid robots and humans. This paper identifies two relatively simple but important prerequisites: saliency detection and attention manipulation by the robot. By creating a saliency based attentional model and combining it with a robot ego-sphere, the robot can engage in an interaction with a human, starting an interaction game including objects as a first step towards a joint attention.

Robot ego-sphere: An approach for saliency detection and attention manipulation in humanoid robots for intuitive interaction

Hand-eye coordination is an important motor skill acquired in infancy which precedes pointing behavior. Pointing facilitates social interactions by directing attention of engaged participants. It is thus essential for the natural flow of human-robot interaction. Here, we attempt to explain how pointing emerges from sensorimotor learning of hand-eye coordination in a humanoid robot. During a body babbling phase with a random walk strategy, a robot learned mappings of joints for different arm postures. Arm joint configurations were used to train biologically inspired models consisting of SOMs. We show that such a model implemented on a robotic platform accounts for pointing behavior while humans present objects out of reach of the robot’s hand.

/pdf/learning-hand-eye-coordination-for-a-humanoid-robot-using-26az2yam28.pdf

Sasa Bodiroza

Papers

Position-invariant, real-time gesture recognition based on dynamic time warping

Dynamic gesture vocabulary design for intuitive human-robot dialog

Spatially unconstrained, gesture-based human-robot interaction

Robot ego-sphere: An approach for saliency detection and attention manipulation in humanoid robots for intuitive interaction

Learning hand-eye coordination for a humanoid robot using SOMs