It is noted that a head-mounted display (HMD) system would be a useful teleoperator system display if it increased operator performance of the desired task but could degrade performance because of display update rate limitations and communication delays. 3-D tracking and pick-and-place tasks are simulated to characterize performance levels for a range of update rates and delays. How performance changes as the display is altered to maintain the update rate is also examined. The results of the update experiments indicate that performance levels degrade only slightly until the update rate is below 1 or 2 Hz. Attempts to maintain update rate by eliminating display features such as stereo and occlusion cues show that the presence of either cue alone is sufficient to maintain performance levels in the simple, short-term tasks considered. Delays also degrade performance in both manipulation tasks considered. These HMD performance problems are not lessened by the presentation of a predictive feedback display. Instead, the delay in the head movement tracking loop must be eliminated. >

Some of what one needs to know about using head-mounted displays to improve teleoperator performance

The authors evaluate the performance and try to identify the strategies of human operators (HOs) teleoperating a robot along 2-D-trajectories in simulations of place-like tasks in an obstacle encumbered environment. The experiments utilize computer graphic simulations of a remote robot whose end-effector displacements are dynamically controlled in the X-Y plane by the HO's hand displacements. The performance is described in the various displayed conditions in terms of movement duration, spatial and temporal errors, and energy. The data show that performance depends on the nature of the cues and instructions provided to the operators. Comparative analysis of the various conditions suggests that visual feedback position control is used when continuous static or dynamic information about the trajectory is provided, while feedforward control, corrected by sampled visual information, is adopted when a self-generated movement can be planned and executed. The data also show that for a given set of execution cues, equivalent performance is achieved in both visual frames of reference. >

Hand movement strategies in telecontrolled motion along 2-D trajectories

Proposes a new type of visual interface system for teleoperated systems. Remote operations, particularly those in space, require robotic devices that are capable of performing a variety of complex mechanical manipulations with a high degree of precision and reliability. These devices must also be able to act and react accordingly to unforeseen circumstances that arise as a result of the unpredictable nature of space. Since fully autonomous motion for robots in space environments is not yet realizable, these systems must be controlled by direct dialogue through an ergonometrically designed user interface. Effective display, that includes accurate depth perception, of the remote scene is essential and vitally important for these interfaces. Although stereoscopic vision systems have been in existence for several years, most produce non-linearly mapped (distorted) images. The system proposed in the paper produces linearly-mapped images, thus facilitating the execution of teleoperated tasks. >

A linearly-mapping stereoscopic visual interface for teleoperation

Legged and wheeled locomotion are two standard methods used by robots to perform navigation. Combining them to create a hybrid legged-wheeled locomotion results in increased speed, agility, and reconfigurability for the robot, allowing it to traverse a multitude of environments. The CENTAURO robot has these advantages, but they are accompanied by a higher-dimensional search space for formulating autonomous economical motion plans, especially in cluttered environments. In this article, we first review our previously presented legged-wheeled footprint reconfiguring global planner. We describe the two incremental prototypes, where the primary goal of the algorithms is to reduce the search space of possible footprints such that plans that expand the robot over the low-lying wide obstacles or narrow into passages can be computed with speed and efficiency. The planner also considers the cost of avoiding obstacles versus negotiating them by expanding over them. The second part of this article presents our new work on local obstacle pushing, which further increases the number of tight scenarios the planner can solve. The goal of the new local push-planner is to place any movable obstacle of unknown mass and inertial properties, obstructing the previously planned trajectory from our global planner, to a location devoid of obstruction. This is done while minimising the distance traveled by the robot, the distance the object is pushed, and its rotation caused by the push. Together, the local and global planners form a major part of the agile reconfigurable navigation suite for the legged-wheeled hybrid CENTAURO robot.

https://ieeexplore.ieee.org/ielx7/6287639/6514899/09665773.pdf

Reconfigurable and Agile Legged-Wheeled Robot Navigation in Cluttered Environments With Movable Obstacles

Visual Factors Affecting Human Operator Performance with a Helmet-Mounted Display

With major emphasis on simulation, a university laboratory telerobotics facility permits problems to be approached by groups of graduate students. Helmet-mounded displays provide realism; the slaving of the display to the human operator's viewpoint gives a sense of 'telepresence' that may be useful for prolonged tasks. Using top-down 3-D model control of distant images allows distant images to be reduced to a few parameters to update the model used for display to the human operator in a preview model to circumvent, in part, the communication delay. Also, the model can be used as a format for supervisory control and permit short-term local autonomous operations. Image processing algorithms can be made simpler and faster without trying to construct sensible images from the bottom. Control studies of telerobots lead to preferential manual control modes and, in this university environment, to basic paradigms for human motion and thence, perhaps, to redesign of robotic control, trajectory path planning, and rehabilitation prosthetics. Speculation as to future industrial drives for this telerobotic field suggests efficient roles for government agencies such as NASA. >

Telerobotics: problems and research needs

While mobile navigation has been focused on obstacle avoidance, Navigation Among Movable Obstacles (NAMO) via interaction with the environment, is a problem that is still open and challenging. This paper, presents a novel system integration to handle NAMO using visual feedback. In order to explore the capabilities of our introduced system, we explore the solution of the problem via graph-based path planning in a photorealistic simulator (NVIDIA Isaac Sim), in order to identify if the simulation-to-reality (sim2real) problem in robot navigation can be resolved. We consider the case where a wheeled robot navigates in a warehouse, in which movable boxes are common obstacles. We enable online real-time object localization and obstacle movability detection, to either avoid objects or, if it is not possible, to clear them out from the robot planned path by using pushing actions. We firstly test the integrated system in photorealistic environments, and we then validate the method on a real-world mobile wheeled robot (UCL MPPL) and its on-board sensory and computing system.

Navigation Among Movable Obstacles with Object Localization using Photorealistic Simulation

With the majority of mobile robot path planning methods being focused on obstacle avoidance, this paper, studies the problem of Navigation Among Movable Obstacles (NAMO) in an unknown environment, with static (i.e., that cannot be moved by a robot) and movable (i.e., that can be moved by a robot) objects. In particular, we focus on a specific instance of the NAMO problem in which the obstacles have to be moved to predefined storage zones. To tackle this problem, we propose an online planning algorithm that allows the robot to reach the desired goal position while detecting movable objects with the objective to push them towards storage zones to shorten the planned path. Moreover, we tackle the challenging problem where an obstacle might block the movability of another one, and thus, a combined displacement plan needs to be applied. To demonstrate the new algorithm’s correctness and efficiency, we report experimental results on various challenging path planning scenarios. The presented method has significantly better time performance than the baseline, while also introducing multiple novel functionalities for the NAMO problem.

K. Ellis

Papers

Telerobotics: problems and research needs

Navigation Among Movable Obstacles with Object Localization using Photorealistic Simulation

Navigation Among Movable Obstacles via Multi-Object Pushing Into Storage Zones