Objective: This article provides an overview of tactile displays. Its goal is to assist human factors practitioners in deciding when and how to employ the sense of touch for the purpose of information representation. The article also identifies important research needs in this area. Background: First attempts to utilize the sense of touch as a medium for communication date back to the late 1950s. For the next 35 years progress in this area was relatively slow, but recent years have seen a surge in the interest and development of tactile displays and the integration of tactile signals in multimodal interfaces. A thorough understanding of the properties of this sensory channel and its interaction with other modalities is needed to ensure the effective and robust use of tactile displays. Methods: First, an overview of vibrotactile perception is provided. Next, the design of tactile displays is discussed with respect to available technologies. The potential benefit of including tactile cues in multimodal inte...

Tactile displays: guidance for their design and application.

Presenting waypoint navigation on a visual display is not suited for all situations. The present experiments investigate if it is feasible to present the navigation information on a tactile display. Important design issue of the display is how direction and distance information must be coded. Important usability issues are the resolution of the display and its usefulness in vibrating environments. In a pilot study with 12 pedestrians, different distance-coding schemes were compared. The schemes translated distance to vibration rhythm while the direction was translated into vibration location. The display consisted of eight tactors around the user's waist. The results show that mapping waypoint direction on the location of vibration is an effective coding scheme that requires no training, but that coding for distance does not improve performance compared to a control condition with no distance information. In Experiment 2, the usefulness of the tactile display was shown in two case studies with a helicopter and a fast boat.

Waypoint navigation with a vibrotactile waist belt

Longitude: The True Story of a Lone Genius Who Solved the Greatest Scientific Problem of His Time

A three-dimensional position and orientation tracking system uses accelerometers to measure acceleration of a moveable object (e.g., a head-mounted display unit or a data glove). A tracking processor generates both position and orientation information on the object relative to a simulation environment as a function of the acceleration data. In one embodiment, a simplified radar-based tracking system is disposed relative to the object and periodically provides additional tracking data on the object to the tracking processor. The tracking processor uses the additional data to correct the position and orientation information using a feedback filter process. The position and orientation information signals generated can be used, for example, in a simulation or virtual reality application. Position and orientation information is received by a simulation processor relative to the object. The simulation processor modifies a simulation environment as a function of the position and orientation information received. Modified simulation environment information (e.g., video and/or audio information) is then presented to a user.

Method and apparatus for determining position and orientation of a moveable object using accelerometers

In a meta-analysis of 43 studies, we examined the effects of multimodal feedback on user performance, comparing visual-auditory and visual-tactile feedback to visual feedback alone. Results indicate that adding an additional modality to visual feedback improves performance overall. Both visual-auditory feedback and visual-tactile feedback provided advantages in reducing reaction times and improving performance scores, but were not effective in reducing error rates. Effects are moderated by task type, workload, and number of tasks. Visual-auditory feedback is most effective when a single task is being performed (g = .87), and under normal workload conditions (g = .71). Visual-tactile feedback is more effective when multiple tasks are begin performed (g = .77) and workload conditions are high (g = .84). Both types of multimodal feedback are effective for target acquisition tasks; but vary in effectiveness for other task types. Implications for practice and research are discussed.

Comparing the effects of visual-auditory and visual-tactile feedback on user performance: a meta-analysis

: The Joint Strike Fighter (JSF) - Tactile Situation Awareness System (TSAS) flight demonstration project originated at the NASA Johnson Space Center and the Naval Aerospace Medical Research Laboratory with funding provided from the JSF Program Office. The JSF-TSAS project was conceived as a short-duration technology maturation and flight demonstration program. The JSF Flight Systems Integrated Product Team (IPT) provided overall project management and funding throughout the demonstration effort. The project integrated a tactile display, F-22 cooling vest, and GPS/INS technologies into a single synergistic system in a UH-60 helicopter. A 10-event test operation was conducted to demonstrate the utility of this advanced human-machine interface. Four demonstration pilots were selected to fly the flight events. Summary results showed that TSAS technologies have the potential to increase pilot situational awareness and reduce pilot workload when using the tactile display, especially during simulated shipboard operations in Instrument Meteorological Conditions (IMC). Prototype hardware development showed that tactile displays could be integrated into existing flight gear. The test pilots successfully performed all maneuvers with degraded outside visual cues, relying on tactile cues for the necessary information. Using TSAS, pilots demonstrated improved control of aircraft during complex flight conditions.

/pdf/tactile-situation-awareness-system-flight-demonstration-15zrvfhauc.pdf

Tactile Situation Awareness System Flight Demonstration

The dynamics of spatial orientation perception were examined in a series of experiments in which a total of 43 subjects were passively exposed to various combinations of linear and angular acceleration during centrifuge runs. Perceptual effects during deceleration were much stronger than effects during acceleration. The dynamics of spatial orientation perception differed substantially from changes in the vestibulo-ocular reflex (VOR). VOR was fairly well predicted by a current model, but our experiments revealed perceived change in attitude (roll, pitch, yaw tilt position in space) and perceived angular velocity in space that was not reflected by parallel changes in the plane or magnitude of the VOR. This series of experiments establishes several facts concerning spatial orientation perception beyond the predictive domain of any current model. New concepts are needed and several are suggested to deal with changing reactions to complex combinations of linear and angular accelerations.

http://www.jvr-web.org/Download/Volume_02/Number_4/v02_n4_a2.pdf

The dynamics of spatial orientation during complex and changing linear and angular acceleration.

Summary United States (US) military Special Forces teams currently use 2D visual displays for navigation information in the air, in water, and on the ground These current displays demand the user’s visual attention, which can compromise mission effectiveness, and using visual displays in low light visibility environments can cause fatigue, degrade performance, and compromise a clandestine situation If navigation equipment that is dependent on visual displays were integrated with a tactile display, the need to use vision for navigation could be minimized The operator could be more effective if his eyes were used to survey the surroundings rather than continuously monitor a visual display The Tactile Situation Awareness System for Special Forces (TSAS-SF) was developed to investigate the potential of tactile displays for Special Forces operations The TSAS-SF will upgrade present 2D visual navigation displays and will provide non-visual, non-audible navigation information to Special Forces personnel by interfacing navigation information with a tactile display This new capability will provide 2D direction cues to the skin, which will free the user’s visual senses for higher priority tasks (eg contact identification and classification) Preliminary testing in a High Altitude, High Opening (HAHO) parachute environment and a ground environment, and earlier testing in an underwater environment (McTrusty, Walters, 1997, Rupert, McTrusty, Peak, 1999), have demonstrated that navigation can be performed faster with tactile cues than visual cues, and superior navigational accuracy can be achieved with less mental fatigue on the operator These results suggest that a tactile display that provides ‘eyes free’ and ‘hands free’ air and ground navigation information may provide the opportunity to devote more time to other instruments and tasks when operating in high workload conditions These effects can increase mission effectiveness The preliminary results from the air and ground navigation tests justify continued testing and evaluation to extend the capabilities of the tactile display, for use as an operational device for navigation in sea, air and land environments

/pdf/enhanced-situation-awareness-in-sea-air-and-land-1bo5l9bxpr.pdf

Enhanced Situation Awareness in Sea, Air and Land Environments

When investigators fail to find a materiel cause for a mishap, pilot error may be implicated. Controlled flight into terrain (CFIT), aircraft upset (AU), and loss of control (LOC) are important mishap contributors. Spatial disorientation (SD) is a key causal factor of CFIT and its presence can precipitate or exacerbate LOC. SD typically is inferred based on unusual control inputs by the pilot, insufficient outside visual cues, the presence of factors which would distract the pilot from the primary flight instruments, and the qualitative similarity of the flight circumstances to those known to induce orientation illusions. We recommend that SD analysis should be more quantitative and comprehensive, involving the matching of information from on-board recorders (e.g., acceleration, pilot control inputs) to mathematical models of human orientation functioning that are designed to fit what is known about human perception. We recently combined several established scientific models into one Perception Toolbox which allows comparison of the predictions from each model. We also devised our own model, called the Orientation Modeling System (OMS), which is intended to improve the explanation of mishaps and laboratory perceptual experiments. The OMS and the Perception Toolbox are described briefly in this report. The modeling tool has been used to describe several illusions in laboratory, simulator and flight settings. Two applied examples are highlighted which entailed the successful modeling of a recent SD mishap and some published experimental results relevant to the disorienting effects of head movement during centrifuge-based simulator training. Lessons learned from the model will enhance future aeromedical capabilities by helping investigators determine if SD is a factor in a given mishap, develop educational videos of mishaps to improve mishap evaluation and pilot training, evaluate new instrumentation solutions to prevent AU, LOC, and CFIT, and improve simulation profiles in advanced centrifuge-based simulators.

The Role of Perceptual Modeling in the Understanding of Spatial Disorientation During Flight and Ground-based Simulator Training

Spatial disorientation and the subsequent loss of situation awareness account for a significant percentage of fatal mishaps in aviation. In our normal earth-bound environment, spatial orientation is continuously maintained by correct information from three independent, redundant, and concordant sensory systems: vision, the inner ear or vestibular system, and the skin, joint, and muscle receptors or somatosensory system. However, in the aviation environment, the vestibular and somatosensory sensations currently do not provide accurate orientation information. The only reliable source of orientation information is vision. For this reason, spatial disorientation mishaps occur when information from the visual system is compromised (e.g., temporary distraction, increased workload, transitions between visual meteorological conditions and instrument meteorological conditions, reduced visibility, or boredom), and the pilot subsequently becomes disorientated. The Tactile Situation Awareness System (TSAS) is an advanced flight instrument that uses the sensory channel of touch to provide situation awareness information to pilots. The TSAS system accepts data from various aircraft sensors and presents this information via tactile stimulators or “tactors” integrated into flight garments. TSAS has the capability of presenting a variety of flight parameter information, including, attitude, altitude, velocity, navigation, acceleration, threat location, and/or target location. A series of flight demonstrations were conducted to demonstrate that a pilot could receive situation awareness information using a tactile instrument during flying operations. The first flight demonstration project was conducted in a United States Navy T-34 aircraft. The objectives of the T-34 TSAS flight demonstration program were to demonstrate: • That a significant amount of orientation information can be intuitively provided continuously by the under-utilized sense of touch. • That a pilot using TSAS can effectively maintain control of an aircraft in normal and certain acrobatic flight conditions with no visual cues. The first flight of the TSAS-modified T-34 was 11 October 1995, and seven flight test events were completed by 19 October 1995. During the flight tests, the test pilot in the rear seat was shrouded to block any outside visual cues and all visual flight instruments in the rear cockpit were removed. The test pilot flew the following maneuvers: straight and level for 5 minutes; bank angle capture; pitch angle capture; climbing and descending turns; unusual attitude recovery; loops; aileron rolls; and ground controlled approaches. Results showed that roll and pitch information could be provided by tactile cues via an array of tactors incorporated into a torso harness. The test pilot performed all maneuvers without visual cues, relying solely on tactile cues for attitude information. The T-34 TSAS flight demonstration showed that a pilot relying on tactile information could maintain control of an aircraft. Following the success of the T-34 flight test program, the Joint Strike Fighter TSAS flight demonstration project integrated an array of tactors, a cooling vest, and Global Positioning/Inertial Navigation System technologies into a single system for evaluation in an UH-60 Helicopter. A 10-event test operation was conducted to demonstrate the utility of this advanced human-machine interface for performing hover operations. Objectives of the Joint Strike Fighter TSAS flight demonstration program were to demonstrate: 1 Pronounced Tee Sas.

/pdf/tactile-instrument-for-aviation-2gb2ccxcaf.pdf

Braden J. McGrath

Papers

Tactile Situation Awareness System Flight Demonstration

The dynamics of spatial orientation during complex and changing linear and angular acceleration.

Enhanced Situation Awareness in Sea, Air and Land Environments

The Role of Perceptual Modeling in the Understanding of Spatial Disorientation During Flight and Ground-based Simulator Training

Tactile Instrument for Aviation