Methodologies for the empirical measurement of situation awareness are reviewed, including a discussion of the advantages and disadvantages of each method and the potential limitations of the measures from a theoretical and practical viewpoint. Two studies are presented that investigate questions of validity and intrusiveness regarding a query-based technique. This technique requires that a simulation of the operational tasks be momentarily interrupted in order to query operators on their situation awareness. The results of the two studies indicate that the query technique is not intrusive on normal subject behavior during the trial and does not suffer from limitations of human memory, which provides an indication of empirical validity. The results of other validity studies regarding the technique are discussed along with recommendations for its use in measuring situation awareness in varied settings.

Measurement of Situation Awareness in Dynamic Systems

The enhancement of operator situation awareness (SA) has become a major designgoal for those developing operator interfaces, automation concepts and training programs ina wide variety of fields, including aircraft, air traffic control, power plants, and advancedmanufacturing systems. This dramatic growth in interest in SA, beginning in the mid-1980’s and accelerating through the 1990’s, was spurred on by many factors, chief amongthem the challenges of a new class of technology. One can easily see that situation awareness has always been needed in order forpeople to perform tasks effectively. Prehistoric man undoubtedly needed to be aware ofmany cues in his environment in order to successfully hunt and keep from being hunted.For many years, having good situation awareness was largely a matter of training andexperience — learning the important cues to watch for and what they meant. With the advent of the machine age, our emphasis shifted to creating a new class oftools to help people perform tasks, largely those physical in nature. The computer age andnow the information age have followed rapidly on the heals of basic mechanization. Thetools provided are no longer simple; they are amazingly complex, focused on not justphysical tasks, but elaborate perceptual and cognitive tasks as well. The pilot of today’saircraft, the air traffic controller, the power plant operator, the anesthesiologist: all mustperceive and comprehend a dazzling array of data which is often changing very rapidly. Ihave taken to calling this challenge the information gap (Figure 1).Today’s systems are capable of producing a huge amount of data, both on the statusof their own components, and on the status of the external environment. Due toachievements in various types of datalink and internet technologies, systems can alsoprovide data on almost anything anywhere in the world. The problem with today’s systemsis not a lack of information, but finding what is needed when it is needed.Unfortunately, in the face of this torrent of data, many operators may be even lessinformed than ever before. This is because there is a huge gap between the tons of databeing produced and disseminated and people’s ability to find the bits that are needed andprocess them together with the other bits to arrive at the actual information that is requiredfor their decisions. This information must be integrated and interpreted correctly as well; afrequently tricky task. This problem is real and ongoing, whether the job is in the cockpitor behind a desk. It is becoming widely recognized that more data does not equal moreinformation. Issues of automation and “intelligent systems” have frequently onlyexacerbated the problem, rather than aided it (Endsley & Kiris, 1995; Sarter & Woods,1995).The criteria for what we are seeking from system designs have correspondinglychanged. In addition to designing systems that provide the operator with the neededinformation and capabilities, we must also insure that it is provided in a way that is useablecognitively as well as physically. We want to know how well the system design supportsthe operator’s ability to get the needed information under dynamic operational constraints.(i.e. How well does it bridge the information gap?) This design objective and measure ofmerit has been termed situation awareness.

/pdf/theoretical-underpinnings-of-situation-awareness-a-critical-2zswqslgyi.pdf

Theoretical Underpinnings of Situation Awareness, A Critical Review

At first glance it seems absurd that busy people doing important jobs should want their computers to interrupt them. Interruptions are disruptive and people need to concentrate to make good decisions. However, successful job performance also frequently depends on people's abilities to (a) constantly monitor their dynamically changing information environments, (b) collaborate and communicate with other people in the system, and (c) supervise background autonomous services. These critical abilities can require people to simultaneously query a large set of information sources, continuously monitor for important events, and respond to and communicate with other human operators. Automated monitoring and alerting systems minimize the need to constantly monitor, but they induce alerts that may interrupt other activities. Such interrupting technologies are already widespread and include concurrent multitasking; mixed-initiative interaction; support for delegation and supervisory control of automation, including intelligent agents; and other distributed, background services and technologies that increase human-human communication.

People do not perform sustained, simultaneous, multichannel sampling well; however, they have great capacity to manage concurrent activities when given specific kinds of interface support. Literature from many domains shows deleterious consequences of human performance in interrupt-laden situations when interfaces do not support this aspect of the task environment. This article identifies why human interruption is an important human-computer interaction problem, and why it will continue to grow in ubiquity and importance. We provide examples of this problem in real-world systems, and we review theoretical tools for understanding human interruption. Based on interdisciplinary scientific results, we suggest potential approaches to user-interface design to help people effectively manage interruptions.

https://www.interruptions.net/literature/McFarlane-HCI02_1.pdf

The scope and importance of human interruption in human-computer interaction design

The Situation Awareness Global Assessment Technique (SAGAT) is a global tool developed to assess SA across all of its elements based on a comprehensive assessment of operator SA requirements. SAGAT queries allow for detailed information about subject SA to be collected on an element by element basis that can be evaluated against reality, thus providing an objective assessment of operator SA. In Mica Endsley and Mark Rodgers, people found that the percentage of aircraft reported by experienced controllers via SAGAT significantly decreased with increases in the number of aircraft in an air traffic control simulation. Based on the goal-directed task analysis, a list of the SA requirements for a domain can be constructed either as a whole or for particular operator goals and subgoals. The SAGAT technique has thus far been shown to have a high degree of validity, sensitivity, and reliability for measuring SA.

Direct Measurement of Situation Awareness: Validity and Use of SAGAT

As important as navigation is to human performance in virtual worlds, it is an often overlooked problem in the design process. This article reports an experiment intended to show that real‐world wayfinding and environmental design principles are effective in designing virtual worlds that support skilled wayfinding behavior. The study measures participant performance on a complex searching task in a number of virtual worlds with differing environmental cues. The worlds are augmented with either a radial grid, a map, or both a grid and a map. The control condition provided no additional navigational cues. The results showed that navigational performance was superior under both map treatments as compared to the control and grid conditions. The grid was, however, shown to provide superior directional information as compared to the other conditions. The control condition provided the worst performance, with participants often becoming disoriented and experiencing extreme difficulty completing the tasks.

Navigating large virtual spaces

An image processing apparatus for aiding analysis of images through the introduction of additonal physically displaced image elements in complementary color which are generated by digital processing of original image elements. The added elements and original image are viewed stereoscopically on an electronic reconstructed image display.

Anaglyphic stereoscopic image apparatus and method

The effectiveness of ego-centered (moving map) and earth-centered (moving plane) displays was studied with subjects monitoring an animated aircraft situational awareness display. Other independent variables were subject experience (aircrew vs non-aircrew) and path complexity (straight vs turning). Periodically, the display blanked and probe questions were asked concerning the relationship of the aircraft to the simulated world. Questions included judgements about angles, distances, time and terrain. Simple paths elicited a 28 percent lower error rate than did complex paths. Moving map displays had a 32 percent lower error rate than moving plane displays. No other significant effects were observed. Subjective ratings by subjects after the experiment revealed unanimous preference for the moving plane display and that the moving plane condition was believed to be easier! This contradiction indicates subjective data is limited in determining display effectiveness.

Situational Awareness in Map Displays

The mapping provides a capability for quantifying an external scene in a manner equivalent to the allocation at perceptual bandwidth in the human retina. The device samples the scene, either photographically or by video; records a digital version of the scene, processes the digital version so that the effects of the distribution rods in the retina provides the weighting for allocating signal processing bandwidth differentially across the scene in accordance with human physiological data, and produces a recording of the processed digital image. The processing uses a selected acuity factor as a parameter, a neighborhood is assigned to each pixel, with a radius r equal to (1.0+(acuity factor×R))/2.0, where R is the distance from the center of the image. The digital values of all pixels in the neighborhood are summed and the sum is divided by the number of pixels in the neighborhood to obtain an average value. This value is then assigned to the pixel in forming a processed image frame.

Retinal information mapping system

Digital apparatus for generating moving grates of stimulus modulated bar patterns is disclosed. The bar patterns are modulated according to sinusoidal or other mathematical relations and can be alternately displayed as color and/or intensity variations on the display screen. Multiple display images, operator control of grate movement, and the use of image processing apparatus are also incorporated.

Dynamic bar pattern method

This research and development effort was directed to the design and proof-of-concept demonstration of a dynamic tactical situation display (TSD) applicable to an advanced conceptual bomber crew system. The TSD provides a primary source of mission pacing and situational awareness information in the Strategic Avionics Battle-Management Evaluation and Research (SABER) exploratory development simulator at the Human Engineering Division of the Armstrong Aerospace Medical Research Laboratory. Four levels of situational awareness information are supported by SABER: (1) conventional paper products, (2) digitized (softcopy) versions of these hardcopy materials, (3) dynamic graphic representation of horizontal situation, and (4) horizontal situation display with digital terrain elevation and cultural feature underlay. The TSD described in this paper is being applied in the SABER simulation facility to support research directed to the optimization of the bomber crew system in the context of future mission requirements.

Gilbert G. Kuperman

Papers

Anaglyphic stereoscopic image apparatus and method

Situational Awareness in Map Displays

Retinal information mapping system

Dynamic bar pattern method

The Design of a Tactical Situation Display