Showing papers in "Journal of Cognitive Engineering and Decision Making in 2013"
TL;DR: In three experiments, evidence is provided to show that superior prediction performance was supported by a situation model-type mechanism as proposed by long-term working memory (LTWM) theory rather than simpler heuristics, such as TTF or RPD.
Abstract: In popular models of expertise and decision making in complex environments, such as the recognition-primed decision (RPD) model and take-the-first (TTF) heuristic, expert and skilled decision makers are described as generating few response options and typically choose the best option first. To explain these behaviors, proponents of TTF have suggested that a negative relationship exists between the number of options generated and decision quality. In the current article, we use a prediction and situational option generation paradigm to assess perceptual-cognitive skill in the complex domain of soccer to determine whether these claims explain how decision makers make predictions about others in the environment. In three experiments we provide evidence to show that superior prediction performance was supported by a situation model-type mechanism as proposed by long-term working memory (LTWM) theory rather than simpler heuristics, such as TTF or RPD. The similarity between LTWM mechanisms and relevant macroco...
77 citations
TL;DR: The present research examined the utility of a Paired-Concept Association Task (P-CAT) as a basis for discriminating expert and novice cue activation in the context of offender profiling and revealed a new method for objectively validating expert cue use.
Abstract: Cues have been identified as important precursors to successful diagnoses among expert practitioners However, current approaches to the identification of expert cues typically rely on subjective methods, making the validity of cues difficult to establish The present research examined the utility of a Paired-Concept Association Task (P-CAT) as a basis for discriminating expert and novice cue activation in the context of offender profiling Three studies are reported: 1A employed a cognitive interview for the acquisition of cue-based concepts used by experts and novices; 1B presented pairs of concepts as part of the P-CAT, which recorded response latency; and, 1C employed a survey to further gauge participants’ perceptions of the concepts The results revealed differences between experts and novices in the cue-based associations activated, and in the response latencies associated with the P-CAT, across expertise The P-CAT accurately discriminated expert from novice cue activation and consequently offers a new method for objectively validating expert cue use
58 citations
TL;DR: This article examined the effect of cognitive task analysis (CTA) on the overall effectiveness of CTA across practitioners and settings in relation to other means for identifying and representing instructional content, concluding that the information elicited through CTA provides a strong basis for highly effective instruction.
Abstract: Cognitive task analysis (CTA) is enjoying growing popularity in both research and practice as a foundational element of instructional design. However, there exists relatively little research exploring its value as a foundation for training through controlled studies. Furthermore, highly individualized approaches to conducting CTA do not permit broadly generalizable conclusions to be drawn from the findings of individual studies. Thus, examining the magnitude of observed effects across studies from various domains and CTA practitioners is essential for assessing replicable effects. This study reports the findings from a metaanalysis that examines the overall effectiveness of CTA across practitioners and settings in relation to other means for identifying and representing instructional content. Overall, the effect of CTA-based instruction is large (Hedges’s g = 0.871). However, effect sizes vary substantially by both CTA method used and training context. Though limited by a relatively small number of studies, the notable effect size indicates that the information elicited through CTA provides a strong basis for highly effective instruction.
54 citations
TL;DR: In this paper, the strengths and weaknesses of using simulation-based training environments for research are discussed and analyzed. But the authors focus on the strengths of the simulation based training environments and not the weaknesses of the training environments.
Abstract: This article provides an integration and synthesis of the strengths and weaknesses of utilizing simulation-based training environments for research. It provides information for researchers interest...
54 citations
TL;DR: In this article, the authors explore cognitive work analysis (CWA), a method typically used for deriving design requirements to support single operators, and rework this methodology into a version that is intended to show requirements required to support successful team collaboration.
Abstract: Today’s complex sociotechnical systems involve more than just single operators and often include rich team interactions. In this article, the authors explore cognitive work analysis (CWA), a method typically used for deriving design requirements to support single operators, and rework this methodology into a version that is intended to show requirements required to support successful team collaboration. Team-suitable approaches are presented for the first two steps of CWA, work domain analysis and control task analysis. These approaches are collected from past attempts to model teams with CWA and two new teamwork visualizations, collaboration tables and decision wheels. The applicability of the extended models is demonstrated with a health care example.
50 citations
TL;DR: In this paper, the theoretical underpinnings and preliminary experimental support for option awareness (OA) were described, which is the perception and comprehension of the relative desirability of available optio...
Abstract: This article describes the theoretical underpinnings and preliminary experimental support for option awareness (OA): the perception and comprehension of the relative desirability of available optio...
48 citations
TL;DR: In this paper, a factor analytic approach to WM was adopted based on performance on three different WM tasks and these factor scores were then related to SA which was measured over two forms of scenarios in a complex dynamic decision-making task.
Abstract: Situation awareness (SA) brings together theories in cognition to examine what an operator perceives, understands and predicts about their environment. Previous characterization of working memory (WM) influence in levels of awareness however is sparse and has several shortcomings, including how both WM and SA have been measured. In the current experiment, a factor analytic approach to WM was adopted based on performance on three different WM tasks. These factor scores were then related to SA which was measured over two forms of scenarios in a complex dynamic decision-making task. In one scenario, Level 1 SA was assessed, and the other assessed Level 3 processes implicitly. Findings from 99 participants indicate WM was unrelated to Level 1, but was related to Level 3 SA with the relationship strengthening with increasing task experience. These results refine and contribute to ongoing theory in the context of SA and dynamic task performance, and provide future directives to individual differences research i...
44 citations
TL;DR: In this paper, an exploratory field study that uses applied cognitive task analysis (ACTA) to examine financial day traders' preferences was conducted, showing that the fourfold pattern of preferences as studied in the wild is risk seeking for medium- and high-probability gains, risk averse for small probability gains (small probability gains), risk adverse for medium probability gains, and risk adverse to small probability losses (medium probability gains).
Abstract: The objective of this article is to improve our understanding of preferences in experienced-based choice. Positioned within the framework of naturalistic decision making, this article responds to the recent call to complement the examination of experience-based choice with studies of cognition in the “wild.” We document an exploratory field study that uses applied cognitive task analysis (ACTA) to examine financial day traders’ preferences. Providing real-world examples, our study illustrates how day traders construct their understanding of gains relative to losses and emphasizes the relevance of prospect theory for understanding the asymmetry of human choice. The fourfold pattern of preferences as studied in the wild is risk seeking for medium- and high-probability gains, risk averse for small-probability gains, risk averse for small-probability losses, and risk averse for medium- and high-probability losses. Our results differ from the fourfold pattern of preferences exhibited by experience-based choice when studied in the laboratory. The implications of this work for prospect theory and the distinction between “experience through learning” and “experience through professional training” are discussed alongside the merits of the ACTA technique for professional expert domain-based knowledge elicitation.
42 citations
TL;DR: The results from this research demonstrate the need to consider the multidimensionality of the operator’s cognitive resources when implementing automation into a system and system designers should consider the type of automation necessary to support the specific cognitive resources burdened by the task.
Abstract: The implementation of automation relies on the assumption that automation will reduce the operator’s cognitive demand and improve performance. However, accepted models demonstrate the multidimensionality of cognitive resources, suggesting that automation must support an appropriate resource dimension to have an appreciable effect. To evaluate this theory, the present study examined the impact of various types of automation on an unmanned ground vehicle (UGV) operator’s performance, workload, and stress. The use of a visually demanding task allowed for comparison between an auditory alert (supporting the heavily burdened visual dimension) and a driving aid (supporting action execution, a relatively unburdened cognitive dimension). Static and adaptive (fluctuating based on task demand) levels were implemented for each automation type. Those receiving auditory alerts exhibited better performance and reduced Worry, but also increased Temporal Demand and Effort relative to those receiving driving automation. Adaptive automation reduced workload for those receiving the auditory alerts, and increased workload for those receiving the driving automation. The results from this research demonstrate the need to consider the multidimensionality of the operator’s cognitive resources when implementing automation into a system. System designers should consider the type of automation necessary to support the specific cognitive resources burdened by the task.
27 citations
TL;DR: This work investigated the contribution of providing either information analysis automation strategy information, task environment information, or both, on human judgment performance in a domain where noisy sensor data are used by both the human and the information analysis automate to make judgments.
Abstract: Displaying both the strategy that information analysis automation employs to makes its judgments and variability in the task environment may improve human judgment performance, especially in cases where this variability impacts the judgment performance of the information analysis automation. This work investigated the contribution of providing either information analysis automation strategy information, task environment information, or both, on human judgment performance in a domain where noisy sensor data are used by both the human and the information analysis automation to make judgments. In a simplified air traffic conflict prediction experiment, 32 participants made probability of horizontal conflict judgments under different display content conditions. After being exposed to the information analysis automation, judgment achievement significantly improved for all participants as compared to judgments without any of the automation's information. Participants provided with additional display content pertaining to cue variability in the task environment had significantly higher aided judgment achievement compared to those provided with only the automation's judgment of a probability of conflict. When designing information analysis automation for environments where the automation's judgment achievement is impacted by noisy environmental data, it may be beneficial to show additional task environment information to the human judge in order to improve judgment performance.
26 citations
TL;DR: An integrative approach to understanding flight crew activity is described, which combines contemporary innovations in cognitive science theory with a new suite of methods for measuring, analyzing, and visualizing the activities of commercial airline flight crews in interaction with the complex automated systems found on the modern flight deck.
Abstract: In this paper, we describe an integrative approach to understanding flight crew activity. Our approach combines contemporary innovations in cognitive science theory with a new suite of methods for measuring, analyzing, and visualizing the activities of commercial airline flight crews in interaction with the complex automated systems found on the modern flight deck. Our unit of analysis is the multiparty, multimodal activity system. We installed a variety of recording devices in high-fidelity flight simulators to produce rich, multistream time-series data sets. The complexity of such data sets and the need for manual coding of high-level events make large-scale analysis prohibitively expensive. We break through this analysis bottleneck by using our newly developed integrated software system called ChronoViz, which supports visualization and analysis of multiple sources of time-coded data, including multiple sources of high-definition video, simulation data, transcript data, paper notes, and eye gaze data. Four examples of flight crew activity serve to illustrate the methods, the theory, and the kinds of findings that are now possible in the study of flight crew interaction with flight deck automation.
TL;DR: This study examined the benefits of presenting video feed from UAVs and unmanned ground vehicles in a combined interface, relative to presenting aerial feed alone, and generated consistent benefits with regard to the identification tasks, perceived mental demand, and reduction of false reports.
Abstract: Operational tactics in urban areas are often aided by information from unmanned aerial vehicles (UAVs). A major challenge for dismounted soldiers, particularly in urban environments, is to understand the conflict area in general and particularly from the UAV feed. The UAV feed is usually used to enhance soldiers’ situation awareness abilities but less for identifying specific elements. A possible way to further enhance soldiers’ abilities is to provide them with multiple sources of information (e.g., aerial and ground views). This study examined the benefits of presenting video feed from UAVs and unmanned ground vehicles (UGVs) in a combined interface, relative to presenting aerial feed alone. Thirty former infantry soldiers with no experience in operating unmanned vehicles participated. Objective performance, subjective evaluations, and eye-tracking patterns were examined in two scenarios. In Scenario 1, performance scores in both identification and orientation tasks were superior in the combined configu...
TL;DR: In this paper, the authors examined whether measuring team members' autonomic nervous system activity could predict subsequent performance on a dynamic process control task and found that team performance can be predicted from team autonomic activity, which supports the argument that a team's physiological state could serve as an indicator of cognitive readiness.
Abstract: Teams that operate in complex and dynamic environments must maintain a certain level of cognitive readiness among team members to ensure high levels of performance in response to potentially uncertain and time sensitive situations. In the current study, the authors sought to identify a physiological measure that could help predict team performance during a complex and dynamic task. Specifically, they examined whether measuring team members' autonomic nervous system activity could predict subsequent performance on a dynamic process control task. Thirty-four teams of two (35 males, 33 females) completed a processing plant simulation during four varying levels of individual and team difficulty. Sympathetic and parasympathetic nervous system activity was measured throughout the task with an electrocardiogram and an impedance cardiogram and was combined to create a measure of team autonomic activity. Regression analyses showed that team autonomic activity accounted for 10% of the variance in team performance scores. In conclusion, the current study showed that team performance can be predicted from team autonomic activity, which supports the argument that a team's physiological state could serve as an indicator of cognitive readiness.
TL;DR: The all-engine-out landing of Air Transat Flight 236 in the Azores Islands is described and certain aspects of that accident are used to motivate a conceptual framework for the organization and display of information in complex human-interactive systems.
Abstract: We describe the all-engine-out landing of Air Transat Flight 236 in the Azores Islands (August 24, 2001) and use certain aspects of that accident to motivate a conceptual framework for the organiza...
TL;DR: In this article, the authors provide guidelines on how to conduct research for the enhancement of training for cognitive readiness (CR), and suggest a basic module for training-of-CR research, which includes a pretest measuring the baseline of the participants' knowledge, skills, and attitudes; person-related variables as control variables; predictors and moderators of CR training success; and two posttests for the assessment of training performance directly after training and following a retention interval for both temporal and adaptive transfer.
Abstract: In the present article, we aim to provide guidelines on how to conduct research for the enhancement of training for cognitive readiness (CR). After distinguishing between trainable CR skills and stable person-related variables, CR components that are trainable need to be defined in terms of knowledge, skills, and attitudes, and person-related variables need to be selected as prerequisites for training success, as they define the composite construct of CR. Training success in turn needs to include indicators of successful performance in unpredictable environments after training of CR that operationalizes temporal and adaptive transfer. We suggest a basic module for training-of-CR research, which includes a pretest measuring the baseline of the participants' knowledge, skills, and attitudes; person-related variables as control variables; predictors and moderators of CR training success; and two posttests for the assessment of training performance directly after training and following a retention interval for both temporal and adaptive transfer. For further research, we suggest extending the basic module with research on spacing of practice, performance under stress, and individual differences in cognition and personality. This research will enhance the understanding of the interplay between person- and training-related variables that best predicts the prerequisites for establishing and sustaining competent performance in complex and stressful environments.
TL;DR: Human-automation interaction (HAI) takes place in virtually every high-technology domain under a variety of operational conditions as mentioned in this paper, where operators make HAI decisions such as which mode to use.
Abstract: Human–automation interaction (HAI) takes place in virtually every high-technology domain under a variety of operational conditions. Because operators make HAI decisions such as which mode to use, a...
TL;DR: An empirical investigation and a system dynamics model of human decision priming are presented, which revealed that the basic features of decisionPriming in humans could be simulated with this architecture and could provide useful information for the design of multi-channel displays.
Abstract: The authors present an empirical investigation and a system dynamics model of human decision priming. Decision priming occurs when initial information creates the expectation that a given decision is appropriate, which speeds up or slows down decision making. A conjunction benefits-and-costs paradigm was used to collect the empirical data, whereas system dynamics techniques were used to create a computational model of decision priming. Decision priming occurred with simulated naturalistic stimuli (i.e., models of military tanks in a desert scene presented in perspective view), the results of which were modeled in a parallel-channels coactive architecture. Simulation revealed that the basic features of decision priming in humans could be simulated with this architecture. Decision priming likely occurs in naturalistic settings. Predictions derived from the model could provide useful information for the design of multimodal or multichannel displays.
TL;DR: A user-friendly system that yields operational benefit results from data-driven prototype evaluations and benefits analyses that iteratively feed back into the prototype design and development is presented in this paper.
Abstract: A user-friendly system that yields operational benefit results from data-driven prototype evaluations and benefits analyses that iteratively feed back into the prototype design and development. In ...
TL;DR: It is demonstrated that theVisualizations provide insights complementary to quantitative measures regarding what process steps participants likely used to identify errors, with the visualizations requiring less work to produce.
Abstract: The objective of this study is to demonstrate the effectiveness of visualizations for exploring one error-prone health care process: nurses verifying patients' identities during the medication administration process. We employed three types of process visualizations (i.e., Markov chains, sparklines, and timeline belt visualizations) to explore process execution data from an experiment wherein nurse participants (N = 20) administered medications to three patients in a simulated clinical setting. One patient had an embedded error, with the medication being incorrect for the patient. The visualizations allowed us to view aggregate and individual-level process execution data, providing insights into the orders in which participants completed process steps. Although we used eye tracking videos, the system developed in this study can automatically generate visualizations using large process execution data sets produced from an array of sources, including observations, sensors, and health IT audit trails. In this article, we demonstrate that the visualizations provide insights complementary to quantitative measures regarding what process steps participants likely used to identify errors, with the visualizations requiring less work to produce. Therefore, the visualizations may be an effective means for efficiently comparing sets of process execution data (e.g., different individuals completing a process, pre- and post-technology implementation, pre- and post-quality improvement intervention).
TL;DR: In this article, eye and head-based psychophysiological cues were used to determine whether any hazardous items (guns or knives) were present during long-duration tasks and individual vigilance levels were determined by scoring the number of correct detections during eight 5-min periods.
Abstract: The objective of this study was to determine whether eye and head-based psychophysiological cues were indicative of vigilance levels during long-duration tasks. For this study, 48 participants reviewed 600 visual search images to determine whether any hazardous items (guns or knives) were present. Individual vigilance levels were determined by scoring the number of correct detections during eight 5-min periods (total study length = 40 min). With an eye-tracking machine, four concurrent eye and head activity measures (blink rates, saccades, pupil diameter, and head position) were used to model changes in vigilance level throughout a simulated baggage screening task. At the end of the study, changes in blink rates and saccade rates proved to be significant predictors of an individual's ability to detect the presence of hazardous items among other nonsignificant baggage items. Language: en
TL;DR: The latest advances in cognitive readiness research documented in this three-part special issue into the design, development, deployment, selection, and training of human-machine systems will enable the operational community to enhance human performance and effectiveness.
Abstract: The capabilities enabled by cognitive readiness underlie successful human performance in any complex environment. Cognitive readiness can be viewed as encompassing individuals’ inherent traits, their training and experience, and their current dynamic state. Because of this, collaboration across the physiological and cognitive research communities is critical. Incorporating the latest advances in cognitive readiness research that are documented in this three-part special issue into the design, development, deployment, selection, and training of human-machine systems will enable the operational community to enhance human performance and effectiveness.