We present a novel interface for human-robot interaction that enables a human to intuitively and unambiguously select a 3D location in the world and communicate it to a mobile robot. The human points at a location of interest and illuminates it (``clicks it'') with an unaltered, off-the-shelf, green laser pointer. The robot detects the resulting laser spot with an omnidirectional, catadioptric camera with a narrow-band green filter. After detection, the robot moves its stereo pan/tilt camera to look at this location and estimates the location's 3D position with respect to the robot's frame of reference.Unlike previous approaches, this interface for gesture-based pointing requires no instrumentation of the environment, makes use of a non-instrumented everyday pointing device, has low spatial error out to 3 meters, is fully mobile, and is robust enough for use in real-world applications.We demonstrate that this human-robot interface enables a person to designate a wide variety of everyday objects placed throughout a room. In 99.4% of these tests, the robot successfully looked at the designated object and estimated its 3D position with low average error. We also show that this interface can support object acquisition by a mobile manipulator. For this application, the user selects an object to be picked up from the floor by ``clicking'' on it with the laser pointer interface. In 90% of these trials, the robot successfully moved to the designated object and picked it up off of the floor.

/pdf/a-point-and-click-interface-for-the-real-world-laser-4liyecvwkm.pdf

A point-and-click interface for the real world: laser designation of objects for mobile manipulation

A laser pointer can be a powerful tool for robot control. However, in the past, their use in the field of robotics has been limited to simple target designation, without exploring their potential as versatile input devices. This paper proposes to create a laser pointer-based user interface for giving various instructions to a robot by applying stroke gesture recognition to the laser's trajectory. Through this interface, the user can draw stroke gestures using a laser pointer to specify target objects and commands for the robot to execute accordingly. This system, which includes lasso and dwelling gestures for object selection, stroke gestures for robot operation, and push-button commands for movement cancellation, has been refined from its prototype form through several user-study evaluations. Our results suggest that laser pointers can be effective not only for target designation but also for specifying command and target location for a robot to perform.

/pdf/designing-laser-gesture-interface-for-robot-control-16lfz1s94s.pdf

Designing Laser Gesture Interface for Robot Control

A crew organization and four-step operational protocol is recommended based on a cumulative descriptive field study of teleoperated rotary-wing micro air vehicles (MAV) used for structural inspection during the response and recovery phases of Hurricanes Katrina and Wilma. The use of MAVs for real civilian missions in real operating environments provides a unique opportunity to consider human-robot interaction. The analysis of the human-robot interaction during 8 days, 14 missions, and 38 flights finds that a three person crew is currently needed to perform distinct roles: Pilot, Mission Specialist, and Flight Director. The general operations procedure is driven by the need for safety of bystanders, other aircraft, the tactical team, and the MAV itself, which leads to missions being executed as a series of short, line-of-sight flights rather than a single flight. Safety concerns may limit the utility of autonomy in reducing the crew size or enabling beyond line-of-sight-operations but autonomy could lead to an increase in flights per mission and reduced Pilot training demands. This paper is expected to contribute to set a foundation for future research in HRI and MAV autonomy and to help establish regulations and acquisition guidelines for civilian operations. Additional research in autonomy, interfaces, attention, and out-of-the-loop (OOTL) control is warranted.

Crew roles and operational protocols for rotary-wing micro-uavs in close urban environments

In this paper, we describe a prototype interface that facilitates the control of a mobile robot team by a single operator, using a sketch interface on a Tablet PC. The user draws a sketch map of the scene and includes the robots in approximate starting positions. Both path and target position commands are supported as well as editing capabilities. Sensor feedback from the robots is included in the display such that the sketch interface acts as a two-way communication device between the user and the robots. The paper also includes results of a usability study, in which users were asked to perform a series of tasks.

Using a hand-drawn sketch to control a team of robots

When a user is in close proximity to a robot, physical contact becomes a potentially valuable channel for communication. People often use direct physical contact to guide a person to a desired location (e.g., leading a child by the hand) or to adjust a person's posture for a task (e.g., a dance instructor working with a dancer). Within this paper, we present an implementation and evaluation of a direct physical interface for a human-scale anthropomorphic robot. We define a direct physical interface (DPI) to be an interface that enables a user to influence a robot's behavior by making contact with its body. Human-human interaction inspired our interface design, which enables a user to lead our robot by the hand and position its arms. We evaluated this interface in the context of assisting nurses with patient lifting, which we expect to be a high-impact application area. Our evaluation consisted of a controlled laboratory experiment with 18 nurses from the Atlanta area of Georgia, USA. We found that our DPI significantly outperformed a comparable wireless gamepad interface in both objective and subjective measures, including number of collisions, time to complete the tasks, workload (Raw Task Load Index), and overall preference. In contrast, we found no significant difference between the two interfaces with respect to the users' perceptions of personal safety.

Lead me by the hand: evaluation of a direct physical interface for nursing assistant robots

The armed forces have a considerable amount of experience in using robots for bomb removal and mine clearing. Emerging technology also enables the targeting of other applications. To evaluate if real deployment of new technology is justified, tactical advantages gained have to be compared to drawbacks imposed. Evaluation calls for realistic tests which in turn require methods dictating how to deploy the new features. The present study has had two objectives: first, to gain a comprehensive view of a potential user of man-portable robots; second, to embed a robot system with users for assessment of present technology in real deployment. In this project we investigated an army company specialized in urban operations performing their tasks with the support of the iRobot Packbot Scout. The robot was integrated and deployed as an ordinary piece of equipment which required modifying and retraining a number of standard behaviors. The reported results were acquired through a long-term test ranging over a period of six months. This paper focuses on the characteristics of the users and their current ways of operation; how the robot was implemented and deployed. Additionally, this paper describes benefits and drawbacks from the users' perspective. A number of limitations in current robot technology are also identified. The findings show that the military relies on precise and thoroughly trained actions that can be executed with a minimum of ambiguity. To make use of robots, new behavioral schemes, which call for tactical optimization over several years, are needed. The most common application during the trials was reconnaissance inside buildings with uncertain enemy presence when time was not critical. Deploying the robot took more time than completing the task by traditional means, but in return kept the soldiers out of harm's way and enabled them to decrease weapon deployment. The range of the radio link, limited video feedback, and the bulky operator control unit were the features constraining the system's overall performance the most. On the other hand, did properties of the system, such as ruggedness, size, weight, terrain ability, and endurance, prove to match the application. The users were of the opinion that robots such as the Packbot Scout would be a valuable standard feature in urban intervention. © 2007 Wiley Periodicals, Inc.

Long-term study of a portable field robot in urban terrain

Operating robots in an outdoor setting poses interesting problems in terms of interaction. To interact with the robot there is a need for a flexible computer interface. In this paper a PDA-based (personal digital assistant, i.e. a handheld computer) approach to robot interaction is presented. The system is designed to allow non-expert users to utilise the robot for operation in an urban exploration setup. The basic design is outlined and a first set of experiments are reported.

PDA interface for a field robot

Robots have a potential to be a significant aid in high risk, unstructured and stressing situations such as experienced by police, fire brigade, rescue workers and military. In this project we have explored the abilities of today's robot technology in the mentioned fields. This was done by, studying the user, identifying scenarios where a robot could be used and implementing a robot system for these cases. We have concluded that highly portable field robots are emerging to be an available technology but that the human-robot interaction is currently a major limiting factor of today's systems. Further we have found that operational protocols, stating how to use the robots, have to be designed in order to make robots an effective tool in harsh and unstructured field environments.

https://www.cc.gatech.edu/~hic/hic-papers/ROMAN-CL5.pdf

The use of robots in harsh and unstructured field applications

This research has examined robot operators' abilities to gain situational awareness while performing tele-operation with video feedback. The research included a user study in which 20 test persons explored and drew a map of a corridor and several rooms, which they had not visited before. Half of the participants did the exploration and mapping using a teleoperated robot (IRobot PackBot) with video feedback but without being able to see or enter the exploration area themselves. The other half fulfilled the task manually by walking through the premises. The two groups were evaluated regarding time consumption and the rendered maps were evaluated concerning error rate and dimensional and logical accuracy. Dimensional accuracy describes the test person's ability to estimate and reproduce dimensions in the map. Logical accuracy refers to missed, added, misinterpreted, reversed and inconsistent objects or shapes in the depiction. The evaluation showed that fulfilling the task with the robot on average took 96% longer time and rendered 44% more errors than doing it without the robot. Robot users overestimated dimensions with an average of 16% while non-robot users made an average overestimation of 1%. Further, the robot users had a 69% larger standard deviation in their dimensional estimations and on average made 23% more logical errors during the test

/pdf/evaluation-of-mapping-with-a-tele-operated-robot-with-video-1wsia3czzo.pdf

Evaluation of Mapping with a Tele-operated Robot with Video Feedback

Robots have been successfully deployed within bomb squads all over the world for decades. Recent technical improvements are increasing the prospects to achieve the same benefits also for other high risk professions. As the number of applications increase issues of collaboration and coordination come into question. Can several groups deploy the same type of robot? Can they deploy the same methods? Can resources be shared? What characterizes the different applications? What are the similarities and differences between different groups? This paper reports on a study of four areas in which robots are already, or are about to be deployed: Military Operations in Urban Terrain (MOUT), Military and Police Explosive Ordnance Disposal (EOD), Military Chemical Biological Radiological Nuclear contamination control (CBRN), and Fire Fighting (FF). The aim of the study has been to achieve a general overview across the four areas to survey and compare their similarities and differences. It has also been investigated to what extent it is possible for the them to deploy the same type of robot. It was found that the groups share many requirements, but, that they also have a few individual hard constrains. A comparison across the groups showed the demands of man-portability, ability to access narrow premises, and ability to handle objects of different weight to be decisive; two or three different sizes of robots will be needed to satisfy the need of the four areas.

Carl Lundberg

Papers

Long-term study of a portable field robot in urban terrain

PDA interface for a field robot

The use of robots in harsh and unstructured field applications

Evaluation of Mapping with a Tele-operated Robot with Video Feedback

Evaluation of robot deployment in live missions with the military, police, and fire brigade