NASA-TLX is a multi-dimensional scale designed to obtain workload estimates from one or more operators while they are performing a task or immediately afterwards. The years of research that precede...

/pdf/nasa-task-load-index-nasa-tlx-20-years-later-4zos9jgvch.pdf

Nasa-Task Load Index (NASA-TLX); 20 Years Later:

This article presents and tests a group-level model of safety climate to supplement the available organization-level model. Climate perceptions in this case are related to supervisory safety practices rather than to company policies and procedures. The study included 53 work groups in a single manufacturing company. Safety climate perceptions, measured with a newly developed scale, revealed both within-group homogeneity and between-groups variation. Predictive validity was measured with a new outcome measure, microaccidents, that refers to behavior-dependent on-the-job minor injuries requiring medical attention. Climate perceptions significantly predicted microaccident records during the 5-month recording period that followed climate measurement, when the effects of group- and individual-level risk factors were controlled. The study establishes an empirical link between safety climate perceptions and objective injury data.

A group-level model of safety climate: testing the effect of group climate on microaccidents in manufacturing jobs.

The out-of-the-loop performance problem, a major potential consequence of automation, leaves operators of automated systems handicapped in their ability to take over manual operations in the event of automation failure. This is attributed to a possible loss of skills and of situation awareness (SA) arising from vigilance and complacency problems, a shift from active to passive information processing, and change in feedback provided to the operator. We studied the automation of a navigation task using an expert system and demonstrated that low SA corresponded with out-of-the-loop performance decrements in decision time following a failure of the expert system. Level of operator control in interacting with automation is a major factor in moderating this loss of SA. Results indicated that the shift from active to passive processing was most likely responsible for decreased SA under automated conditions.

The Out-of-the-Loop Performance Problem and Level of Control in Automation:

Various levels of automation (LOA) designating the degree of human operator and computer control were explored within the context of a dynamic control task as a means of improving overall human/machine performance. Automated systems have traditionally been explored as binary function allocations; either the human or the machine is assigned to a given task. More recently, intermediary levels of automation have been discussed as a means of maintaining operator involvement in system performance, leading to improvements in situation awareness and reductions in out-of-the-loop performance problems. A LOA taxonomy applicable to a wide range of psychomotor and cognitive tasks is presented here. The taxonomy comprises various schemes of generic control system function allocations. The functions allocated to a human operator and/or computer included monitoring displays, generating processing options, selecting an 'optimal' option and implementing that option. The impact of the LOA taxonomy was assessed within a dynamic and complex cognitive control task by measuring its effect on human/system performance, situation awareness and workload. Thirty subjects performed simulation trials involving various levels of automation. Several automation failures occurred and out-of-the-loop performance decrements were assessed. Results suggest that, in terms of performance, human operators benefit most from automation of the implementation portion of the task, but only under normal operating conditions; in contrast, removal of the operator from task implementation is detrimental to performance recovery if the automated system fails. Joint human/system option generation significantly degraded performance in comparison to human or automated option generation alone. Lower operator workload and higher situation awareness were observed under automation of the decision making portion of the task (i.e. selection of options), although human/system performance was only slightly improved. The implications of these findings for the design of automated systems are discussed.

/pdf/level-of-automation-effects-on-performance-situation-1se1noyefn.pdf

Level of automation effects on performance, situation awareness and workload in a dynamic control task

The MedTeams project is a translational research effort to apply crew resource management (CRM) behavioral principles developed in aviation to emergency medical care. Hospital emergency departments share many of the same characteristics with workplaces where CRM is effective, such as time-stress, dispersed and complex information, multiple players, and high-stakes outcomes. Preliminary observations in emergency departments (ED) established that the same CRM behaviors employed by highly effective aviation teams could be useful in the ED (Weiner, Kanki, and Helmreich 1993; Simon, Morey, and Locke 1997). A retrospective review of ED closed claims revealed that failure to engage in one or more of these teamwork behaviors was associated with an adverse event and indemnity payments. In 43 percent of the cases reviewed, teamwork behaviors would have prevented or mitigated the adverse event had they been applied (Risser, Rice et al. 1999). Similar analyses attribute about 80 percent of anesthesia mishaps to human error and 70 percent of commercial aviation accidents to crew errors (Gardner-Bonneau 1993; Taggart 1994).

Crew training has led to reductions in aviation mishaps beyond those produced by improvements in equipment and technology. The aviation community began introducing CRM training two decades ago and it is now required for all military and commercial U.S. aviation crews and air carriers operating internationally (Helmreich and Foushee 1993; Helmreich 1997). The basic principle of CRM is that crew communication and coordination behaviors are identifiable, teachable, and applicable to high-stakes environments. An additional principle is that those behaviors, although seen spontaneously, are not practiced reliably, regularly, or well unless specific training and reinforcement has established them. Specific CRM behaviors have been identified through experimentation and observation of high-reliability teams in demanding, time-stressed environments such as combat aviation and naval command and control centers (Leedom and Simon 1995; McIntyre and Salas 1995). Crew resource management training has been shown to be effective in these environments and is being extended into other domains (Helmreich and Foushee 1993; Salas et al. 1999).

Finally, an essential principle of CRM is that a team needs to be formally established for teamwork behaviors to be effective. In contrast to the notion of a team as any loosely coordinated group of caregivers and support staff, the formal teamwork structure of this study stipulates that a team be made up of between 3 to 10 members. A team is composed of physicians, nurses, and technicians who are organized for a shift. The number of designated teams for a shift depends on factors such as staffing levels and patient volume. From these larger teams, ad hoc teams are formed to respond to emergent events such as resuscitations. In this model, teamwork is sustained by a shared set of teamwork skills rather than permanent assignments that carry over from day to day.

Teamwork theory development has focused on input variables affecting team functioning such as task, work environment, and team member characteristics (Salas et al. 1992), what and how to train (Salas and Cannon-Bowers 2001), and outcome constructs of teamwork effectiveness (McIntyre, Salas 1995). Explanatory mechanisms of team processes exist in the form of constructs such as situational awareness and shared mental models, but relating team processes to work productivity outcomes has been limited by measurement difficulties with these constructs. Because this study examined the applicability of CRM to health care, a goal of this study was to generate a set of testable teamwork process–outcome propositions for future healthcare research. With respect to the training intervention, we sought to gain insight into the effectiveness of the training materials and methods, and features of the curriculum that needed revision. In addition, we sought to determine if the training intervention changed staff attitudes and behaviors and had an impact on patient care. These complementary perspectives are referred to as formative and summative evaluation in the education research literature (Bloom, Hastings, and Madaus 1971).

The first objective of this study was to adapt an aviation-oriented teamwork curriculum to the particular circumstances of EDs by developing and then implementing a training curriculum (Emergency Team Coordination Course [ETCC]) organized around five team dimensions (maintain team structure and climate, apply problem-solving strategies, communicate with the team, execute plans and manage workload, and improve team skills) (Risser, Rice et al. 1999; Risser, Simon et al. 1999). The second objective was to evaluate the effectiveness of the intervention with measures developed to address three outcome constructs: Team Behaviors, Attitudes and Opinions, and ED Performance.

/pdf/error-reduction-and-performance-improvement-in-the-emergency-4319uq441c.pdf

Error Reduction and Performance Improvement in the Emergency Department through Formal Teamwork Training: Evaluation Results of the MedTeams Project

The results of a multi-year research program to identify the factors associated with variations in subjective workload within and between different types of tasks are reviewed. Subjective evaluations of 10 workload-related factors were obtained from 16 different experiments. The experimental tasks included simple cognitive and manual control tasks, complex laboratory and supervisory control tasks, and aircraft simulation. Task-, behavior-, and subject-related correlates of subjective workload experiences varied as a function of difficulty manipulations within experiments, different sources of workload between experiments, and individual differences in workload definition. A multi-dimensional rating scale is proposed in which information about the magnitude and sources of six workload-related factors are combined to derive a sensitive and reliable estimate of workload.

Lowell E. Staveland

Papers

Development of NASA-TLX (Task Load Index): Results of Empirical and Theoretical Research