Showing papers on "Situation awareness published in 2007"

Human Performance Issues and User Interface Design for Teleoperated Robots

Abstract: In the future, it will become more common for humans to team up with robotic systems to perform tasks that humans cannot realistically accomplish alone. Even for autonomous and semiautonomous systems, teleoperation will be an important default mode. However, teleoperation can be a challenging task because the operator is remotely located. As a result, the operator's situation awareness of the remote environment can be compromised and the mission effectiveness can suffer. This paper presents a detailed examination of more than 150 papers covering human performance issues and suggested mitigation solutions. The paper summarizes the performance decrements caused by video images bandwidth, time lags, frame rates, lack of proprioception, frame of reference, two-dimensional views, attention switches, and motion effects. Suggested solutions and their limitations include stereoscopic displays, synthetic overlay, multimodal interfaces, and various predicative and decision support systems.

Designing for flexible interaction between humans and automation: delegation interfaces for supervisory control.

Abstract: OBJECTIVE: To develop a method enabling human-like, flexible supervisory control via delegation to automation. BACKGROUND: Real-time supervisory relationships with automation are rarely as flexible as human task delegation to other humans. Flexibility in human-adaptable automation can provide important benefits, including improved situation awareness, more accurate automation usage, more balanced mental workload, increased user acceptance, and improved overall performance. METHOD: We review problems with static and adaptive (as opposed to "adaptable") automation; contrast these approaches with human-human task delegation, which can mitigate many of the problems; and revise the concept of a "level of automation" as a pattern of task-based roles and authorizations. We argue that delegation requires a shared hierarchical task model between supervisor and subordinates, used to delegate tasks at various levels, and offer instruction on performing them. A prototype implementation called Playbook is described. RESULTS: On the basis of these analyses, we propose methods for supporting human-machine delegation interactions that parallel human-human delegation in important respects. We develop an architecture for machine-based delegation systems based on the metaphor of a sports team's "playbook." Finally, we describe a prototype implementation of this architecture, with an accompanying user interface and usage scenario, for mission planning for uninhabited air vehicles. CONCLUSION: Delegation offers a viable method for flexible, multilevel human-automation interaction to enhance system performance while maintaining user workload at a manageable level. APPLICATION: Most applications of adaptive automation (aviation, air traffic control, robotics, process control, etc.) are potential avenues for the adaptable, delegation approach we advocate. We present an extended example for uninhabited air vehicle mission planning. Language: en

Modeling and Predicting Mental Workload in En Route Air Traffic Control: Critical Review and Broader Implications

Abstract: Objective: We perform a critical review of research on mental workload in en route air traffic control (ATC). We present a model of operator strategic behavior and workload management through which workload can be predicted within ATC and other complex work systems. Background: Air traffic volume is increasing worldwide. If air traffic management organizations are to meet future demand safely, better models of controller workload are needed. Method: We present the theoretical model and then review investigations of how effectively traffic factors, airspace factors, and operational constraints predict controller workload. Results: Although task demand has a strong relationship with workload, evidence suggests that the relationship depends on the capacity of the controllers to select priorities, manage their cognitive resources, and regulate their own performance. We review research on strategies employed by controllers to minimize the control activity and information-processing requirements of control tasks. Conclusion: Controller workload will not be effectively modeled until controllers' strategies for regulating the cognitive impact of task demand have been modeled. Application: Actual and potential applications of our conclusions include a reorientation of workload modeling in complex work systems to capture the dynamic and adaptive nature of the operator's work. Models based around workload regulation may be more useful in helping management organizations adapt to future control regimens in complex work systems.

Ecological Interfaces for Improving Mobile Robot Teleoperation

Abstract: Navigation is an essential element of many remote robot operations including search and rescue, reconnaissance, and space exploration. Previous reports on using remote mobile robots suggest that navigation is difficult due to poor situation awareness. It has been recommended by experts in human-robot interaction that interfaces between humans and robots provide more spatial information and better situational context in order to improve an operator's situation awareness. This paper presents an ecological interface paradigm that combines video, map, and robot-pose information into a 3-D mixed-reality display. The ecological paradigm is validated in planar worlds by comparing it against the standard interface paradigm in a series of simulated and real-world user studies. Based on the experiment results, observations in the literature, and working hypotheses, we present a series of principles for presenting information to an operator of a remote robot.

Developing operator capacity estimates for supervisory control of autonomous vehicles.

Abstract: OBJECTIVE: This study examined operators' capacity to successfully reallocate highly autonomous in-flight missiles to time-sensitive targets while performing secondary tasks of varying complexity. BACKGROUND: Regardless of the level of autonomy for unmanned systems, humans will be necessarily involved in the mission planning, higher level operation, and contingency interventions, otherwise known as human supervisory control. As a result, more research is needed that addresses the impact of dynamic decision support systems that support rapid planning and replanning in time-pressured scenarios, particularly on operator workload. METHOD: A dual screen simulation that allows a single operator the ability to monitor and control 8, 12, or 16 missiles through high level replanning was tested on 42 U.S. Navy personnel. RESULTS: The most significant finding was that when attempting to control 16 missiles, participants' performance on three separate objective performance metrics and their situation awareness were significantly degraded. CONCLUSION: These results mirror studies of air traffic control that demonstrate a similar decline in performance for controllers managing 17 aircraft as compared with those managing only 10 to 11 aircraft. Moreover, the results suggest that a 70% utilization (percentage busy time) score is a valid threshold for predicting significant performance decay and could be a generalizable metric that can aid in manning predictions. APPLICATION: This research is relevant to human supervisory control of networked military and commercial unmanned vehicles in the air, on the ground, and on and under the water. Language: en

Concepts, Models, and Tools for Information Fusion

Abstract: An international panel of leading experts gives you a penetrating and cohesive look at current and future information fusion technology in this unique volume Examining decision-making models and the command and control process, you develop a solid foundation that lets you explore the latest in situation awareness The book examines human factors and their effect on analysis to help you judge users' trust in decision support systems You find practical mathematical techniques and visualization methods, as well as tested heuristics, that show you how systems can achieve higher levels of fusion The book details the Visual Fusion and Adaptive Sensor Fusion systems developed by the US Department of Defense, offering insight into implementation issues and performance requirements Moreover this leading-edge reference discusses systems interoperability among coalition forces as they prepare to confront future conflicts that transcend national boundaries

Attention-Situation Awareness (A-SA) Model of Pilot Error

Abstract: We describe three applications of an attention-situation awareness model to predicting pilot performance in high fidelity flight simulations. The model incorporates the SEEV model of selective attention, in which eye movements are driven by salience, expectancy and value, and are inhibited by the effort required to move the eyes over longer distance and of concurrent tasks. In application 1, we predict the taxi-way errors made in a simulation in which commercial airline pilots navigate the airport surface at Chicago O'Hare airport. In application 2, we predict the scanning behavior of three commercial pilots flying a simulated approach to Santa Barbara Airport, using a synthetic vision system display, and responding to occasional unexpected events. In application 3, we predict scanning behavior and performance of 8 general aviation pilots flying a simulated landing approach to a terrain challenged airport. Various forms of the computational model are evaluated to predict scanning performance, off-normal event detection and, in application 3, multi-task performance of flight path control and traffic detection. The model is shown to be quite valid, accounting for around 85% of the variance in scanning across different areas of interest in the pilots' visual field. The model fit is not improved by including the effort-inhibition parameter. The degree to which pilots conform to the expected value (optimal) version of the model is shown to positively predict the indices of multi-task performance. The role of redundancy is also addressed.

Adaptive Automation for Human-Robot Teaming in Future Command and Control Systems

Abstract: : Advanced command and control (C2) systems such as the U.S. Army's Future Combat Systems (FCS) will increasingly use more flexible, reconfigurable components, including numerous robotic (unmanned) air and ground vehicles. Human operators will be involved in supervisory control of unmanned vehicles (UVs) with the need for occasional manual intervention. This paper discusses the design of automation support in C2 systems with multiple UVs. Following a model of effective human-automation interaction design, the authors propose that operators can best be supported by high-level automation of information acquisition and analysis functions. Automation of decision-making functions, on the other hand, should be set at a moderate level, unless 100 percent reliability can be assured. The use of adaptive automation support technologies also is discussed. They present a framework for adaptive and adaptable processes as methods that can enhance human-system performance while avoiding some of the common pitfalls of "static" automation such as over-reliance, skill degradation, and reduced situational awareness. Adaptive automation invocation processes are based on critical mission events, operator modeling, and real-time operator performance and physiological assessment, or hybrid combinations of these methods. They describe the results of human-in-the-loop experiments involving human operator supervision of multiple UVs under multi-task conditions in simulations of reconnaissance missions. The results support the use of adaptive automation to enhance human-system performance in supervision of multiple UVs, balance operator workload, and enhance situational awareness. Implications for the design and fielding of adaptive automation architectures for C2 systems involving UVs are discussed.

Chapter 1 Air Transportation: Irregular Operations and Control

Abstract: Publisher Summary This chapter discusses problems related to the management of air traffic and airline operations for minimizing the impact and cost of disruptions. Flight and crew schedules and passenger itineraries have become increasingly fragile because of the growing complexity of the air transportation system and the tight coupling of its various elements. The resulting direct and indirect economic costs are very large, certainly amounting to several billion dollars annually. The airline industry has a vital stake in research aimed at mitigating the effects of severe weather and other disruptive events and at expediting recovery from irregular operations. A significant body of recent and ongoing work has led to major progress toward these objectives. Two breakthrough developments have been the primary drivers behind this progress. First, collaborative decision making has made it possible to apply the principles of information sharing and distributed decision making to air traffic flow management (ATFM) by expanding the databases available to airline and Federal Aviation Administration (FAA) traffic flow managers, by creating common situational awareness, and by introducing shared real-time tools and procedures. Second, there is growing recognition in the airline industry of the fact that planning for schedule robustness and reliability may be just as important as planning for minimizing costs in the complex, highly stochastic, and dynamic environment of air transportation.

Holistic Sensing and Active Displays for Intelligent Driver Support Systems

Abstract: A multidisciplinary research effort at UCSD focuses on the design, development, and evaluation of novel computational frameworks for vehicle-based safety systems. The dynamic active display presents visual alerts to the driver based on the surrounding environment, vehicle dynamics, and driver's state as well as a driver-intent analysis and situational awareness system.

Shared Situational Awareness in Emergency Management Mitigation and Response

Abstract: The US is replacing its historical federalist concept of emergency management where primary responsibility resides with state and local governments and their emergency management and first responder resources for coordinating emergency response and recovery, supported by the resources federal government (coordinated by FEMA) with a homeland security national response system where response to events is controlled by DHS using a military command and control model. This model assumes that those controlling and coordinating the response and recovery would attain and maintain an accurate, shared common operating picture and situational awareness. The objective of this paper is to discuss why the transfer of this concept from its safety and combat origins to the complex, heterogeneous emergency management structure of the United States would would be exceedingly difficult, and that short term strategies based on the assumption that shared situational awareness would be easily achieved are doomed to failure

Situation awareness and driving performance in a simulated navigation task.

Abstract: The objective of this study was to identify task and vehicle factors that may affect driver situation awareness (SA) and its relationship to performance, particularly in strategic (navigation) tasks. An experiment was conducted to assess the effects of in-vehicle navigation aids and reliability on driver SA and performance in a simulated navigation task. A total of 20 participants drove a virtual car and navigated a large virtual suburb. They were required to follow traffic signs and navigation directions from either a human aid via a mobile phone or an automated aid presented on a laptop. The navigation aids operated under three different levels of information reliability (100%, 80% and 60%). A control condition was used in which each aid presented a telemarketing survey and participants navigated using a map. Results revealed perfect navigation information generally improved driver SA and performance compared to unreliable navigation information and the control condition (task-irrelevant information). In-vehicle automation appears to mediate the relationship of driver SA to performance in terms of operational and strategic (navigation) behaviours. The findings of this work support consideration of driver SA in the design of future vehicle automation for navigation tasks.

From images to traffic behavior - A UAV tracking and monitoring application

Abstract: An implemented system for achieving high level situation awareness about traffic situations in an urban area is described. It takes as input sequences of color and thermal images which are used to construct and maintain qualitative object structures and to recognize the traffic behavior of the tracked vehicles in real time. The system is tested both in simulation and on data collected during test flights. To facilitate the signal to symbol transformation and the easy integration of the streams of data from the sensors with the GIS and the chronicle recognition system, DyKnow, a stream-based knowledge processing middleware, is used. It handles the processing of streams, including the temporal aspects of merging and synchronizing streams, and provides suitable abstractions to allow high level reasoning and narrow the sense reasoning gap.

Predicting Operator Capacity for Supervisory Control of Multiple UAVs

Abstract: With reduced radar signatures, increased endurance and the removal of humans from immediate threat, uninhabited (also known as unmanned) aerial vehicles (UAVs) have become indispensable assets to militarized forces. UAVs require human guidance to varying degrees and often through several operators. However, with current military focus on streamlining operations, increasing automation, and reducing manning, there has been an increasing effort to design systems such that the current many-toone ratio of operators to vehicles can be inverted. An increasing body of literature has examined the effectiveness of a single operator controlling multiple uninhabited aerial vehicles. While there have been numerous experimental studies that have examined contextually how many UAVs a single operator could control, there is a distinct gap in developing predictive models for operator capacity. In this chapter, we will discuss previous experimental research for multiple UAV control, as well as previous attempts to develop predictive models for operator capacity based on temporal measures. We extend this previous research by explicitly considering a cost-performance model that relates operator performance to mission costs and complexity. We conclude with a meta-analysis of the temporal methods outlined and provide recommendation for future applications.

Enhanced Situational Awareness: Application of DDDAS Concepts to Emergency and Disaster Management

Abstract: We describe a prototype emergency and disaster information system designed and implemented using DDDAS concepts. The system is designed to use real-time cell phone calling data from a geographical region, including calling activity --- who calls whom, call duration, services in use, and cell phone location information --- to provide enhanced situational awareness for managers in emergency operations centers (EOCs) during disaster events. Powered-on cell phones maintain contact with one or more within-range cell towers so as to receive incoming calls. Thus, location data about all phones in an area are available, either directly from GPS equipped phones, or by cell tower, cell sector, distance from tower and triangulation methods. This permits the cell phones of a geographical region to serve as an ad hoc mobile sensor net, measuring the movement and calling patterns of the population. A prototype system, WIPER, serves as a test bed to research open DDDAS design issues, including dynamic validation of simulations, algorithms to interpret high volume data streams, ensembles of simulations, runtime execution, middleware services, and experimentation frameworks [1].

Application of Wireless Sensor Network to Military Information Integration

Abstract: The purpose of this investigative study is to develop a model for a system of systems to improve situation awareness and targeting through a real-time aerially deployed wireless sensor network. The main hypothesis of this work is that scalable, affordable, real-time wireless network can contribute to the common operating picture. This network can be used to overcome targeting discrepancies and counter asymmetric threats in the new warfare paradigm that pertains to opaque environments. Surveys identified requirements for a system of systems to meet requirements for situation assessment. This paper describes 1. a model to support the hypothesis that this network can to improve response to asymmetric threats, and 2. a system architecture based on commercial-off-the-shelf technology for military operations in this new combat paradigm.

INformation fusion engine for real-time decision-making (INFERD): A perceptual system for cyber attack tracking

Abstract: Information fusion engine for real-time decision- making (INFERD) is a perceptual information fusion engine designed and developed for the purpose of cyber attack tracking and network situational awareness. While the original application was cyber orientated, the engine itself is designed to generalize and has been ported to other application environments such as maritime domain awareness and medical syndromic surveillance. Comparisons and contrasts are drawn to the traditional Kalman ground target tracking science, motivating high level architectural modules and presenting the cyber environment complexities and assumptions. Performance results are presented showing success in both detection accuracy and temporal expedience, an important design goal.

LASSOing HRI: analyzing situation awareness in map-centric and video-centric interfaces

Abstract: Good situation awareness (SA) is especially necessary when robots and their operators are not collocated, such as in urban search and rescue (USAR). This paper compares how SA is attained in two systems: one that has an emphasis on video and another that has an emphasis on a three-dimensional map. We performed a within-subjects study with eight USAR domain experts. To analyze the utterances made by the participants, we developed a SA analysis technique, called LASSO, which includes five awareness categories: location, activities, surroundings, status, and overall mission. Using our analysis technique, we show that a map-centric interface is more effective in providing good location and status awareness while a video-centric interface is more effective in providing good surroundings and activities awareness.

Assessing the scalability of a multiple robot interface

Abstract: As multiple robot systems become more common, it is necessary to develop scalable human-robot interfaces that permit the inclusion of additional robots without reducing the overall system performance. Workload and situational awareness play key roles in determining the ratio of m operators to n robots. A scalable interface, where m is much smaller than n, will have to manage the operator's workload and promote a high level of situation awareness. This work focused on the development of a scalable interface for a single human-multiple robot system. This interface introduces a relational “halo” display that augments a camera view to promote situational awareness and the management of multiple robots by providing information regarding the robots' relative locations with respect to a selected robot. An evaluation was conducted to determine the scalability of the interface focusing on the effects of increasing the number of robots on workload, situation awareness, and robot usage. Twenty participants completed two bomb defusing tasks: one employing six robots, the other nine. The results indicated that increasing the number of robots increased overall workload and the operator's situation awareness.

Trajectory clustering for coastal surveillance

Abstract: Achieving superior situation awareness is a key task for military, as well as civilian, decision makers. Today, automatic systems provide us with an excellent opportunity for assisting the human decision maker in achieving this awareness. Due to the potential of information overload one important aspect is to understand where to focus attention. Anomaly detection is concerned with finding deviations from normalcy and it is an increasingly important topic when providing decision support, since it can give hints towards where more analysis is needed. In this paper we explore trajectory clustering as a means for representing normal behavior in a coastal surveillance scenario. Trajectory clustering however suffers from some drawbacks in this type of setting and we therefore propose a new approach, spline-based clustering, with a potential for solving the task of representing the normal course of events.

Visual Analytics on Mobile Devices for Emergency Response

Abstract: Using mobile devices for visualization provides a ubiquitous environment for accessing information and effective decision making. These visualizations are critical in satisfying the knowledge needs of operators in areas as diverse as education, business, law enforcement, protective services, medical services, scientific discovery, and homeland security. In this paper, we present an efficient and interactive mobile visual analytic system for increased situational awareness and decision making in emergency response and training situations. Our system provides visual analytics with locational scene data within a simple interface tailored to mobile device capabilities. In particular, we focus on processing and displaying sensor network data for first responders. To verify our system, we have used simulated data of The Station nightclub fire evacuation.

Situation Awareness and Driving: A Cognitive Model

Abstract: One of the major preconditions of safe driving is that drivers correctly perceive and interpret the relevant objects and elements of the current traffic situation and that they consider these elements in planning and controlling their behaviour. Such elements may be other drivers, the condition of the street or traffic signs. For each of these elements drivers do not just have to perceive them but they must understand them according to their relevance to their goals. In addition, drivers must also make assumptions about the future actions or states of these elements. For example, perceiving a car coming from the right when entering a crossroads is far from being enough in order to react accordingly. The driver must interpret this car according to its relevance to his own goal, that is, safely passing the crossroads. He has to take into account whether he or the car from the right has to give way. But even this is not enough to select the appropriate action. If the other car has to give way, the driver will try to assess from the speed of the car whether the other car will indeed stop. A concept that has recently become rather popular in aviation psychology and that aims at describing and integrating these different cognitive processes is called situation awareness.

Decision Support in Space Situational Awareness

Abstract: n To maintain the space catalog, the sensors of Air Force Space Command routinely track over 10,000 orbiting space objects. Because of the limited number of sensors, however, we cannot maintain persistent surveillance on these objects. This article describes algorithms and systems developed by Lincoln Laboratory to provide commercial and military analysts with better space situational awareness and decision support as they address problems in the space arena. The first problem is collision avoidance in the increasingly crowded geosynchronous earth orbit (GEO) belt, where there is continuous potential for on-orbit collisions between active satellites and debris, dead satellites, or other active payloads. This case is known as cooperative monitoring, since the owners of the satellites of concern share their operating data. Another problem is noncooperative GEO satellite monitoring, in which space analysts have no information about the satellite station keeping and maneuver plans. In this case, space surveillance data provide the only method to determine orbital status. This article summarizes GEO satellite orbits and their control, and describes a cooperative monitoring system for assisting satellite operators in maintaining safe spacing to nearby objects. We also address the noncooperative GEO monitoring problem by using Bayesian networks to combine signature and metric information from space surveillance sensors, which allows us to detect satellite status changes and produce automated alerts.

Situation Management: Basic Concepts and Approaches

Abstract: This paper scopes the issues of Situation Management in dynamic systems, defines the basic concepts of Situation Management, and identifies several key enabling technologies. Particular focus of the paper is given to situation modeling. The paper reviews major aspects of situation modeling and discusses associated technologies, including Situation Calculus, Situation Semantics, Situation Control, Situation Awareness and others. In more detail we discuss an approach to situation management based on multi-agent systems, event correlation and case-based reasoning.

System and method for integrated facility and fireground management

Abstract: The present invention provides an intelligent, integrated facility and fireground management system which is efficient, assures first responder, pedestrian and appliance safety, as well as precise performance in extreme emergency situations, regulatory compliance, easy and flexible integration with building systems and add-on components, as well as advanced internal component monitoring and event logging. The present invention additionally provides systems and method for real-time first responder situational awareness and real-time fireground situational awareness.

Interdependency modeling and emergency response

Abstract: In large-scale disaster events, infrastructure owners are faced with many challenges in deciding the allocation ofresources for preparation and response actions. This decision process involves building situation awareness, evaluating course of action, and effecting response. This paper describes a modeling and simulation system called CIMS© that presents a visual environment for assessing the causal effects of events and actions in complex environments. Specifically, CIMS© provides a framework for evaluating cascading effects associated with infrastructure interdependencies, thus providing greater situational awareness to infrastructure owners and decision-makers. This paper first presents the area of interdependency analysis and then presents CIMS© as a network framework for simulating the interactions between multiple infrastructures. Also introduced is the integration of infrastructure simulation with a decision support systems.

Customizable situational awareness dashboard and alerts, and associated systems and methods

Abstract: A customizable situational awareness system is described that allows a user to customize the type of information that the user receives through a user interface, e.g., a dashboard, displayed by the system. In particular embodiments, the system can allow the user to configure alerts that are sent to the user's mobile or other device based on situational awareness events. In further embodiments, the customizable situational awareness system receives user configuration data related to flight events, maintenance events, and alerts. When the system receives situational awareness data, the system can retrieve the user's configuration data and display the situational awareness data based on the user's configuration data. Thus, aspects of the customizable situational awareness system allow the user to access information that is more relevant to the user and to filter out less relevant information.

FOCUS: A Model of Sensemaking

Abstract: : Sensemaking is a relatively new concept that has largely been associated with Weick (1995) and his work in organizational behavior. Sensemaking refers to the set of processes involved in trying to improve one's understanding of a situation, often in response to surprise. The primary purpose of the current project was to unpack and develop the concept of sensemaking, principally by developing and testing a cognitive model of the processes involved. The resulting Data/Frame model posits a highly interactive relationship between data inputs and mental representations or "frames" for interpreting data. The Data/Frame model also proposes six key sensemaking activities for handling frames in light of (anomalous) data: Elaborating, Questioning, Comparing, Preserving, Reframing, and Seeking. A secondary aim was to provide recommendations for training and other applications of the model that would be of direct benefit to the warfighter. To that end, several specific links to applied issues in domains such as information operations, intelligence analysis and combat systems design for UAV control have been developed and pursued. At this juncture, the concept of sensemaking and the Data/Frame model appear to be supported by the data, and also quite useful for military applications.