Multioperator tasks often require complex cognitive processing at the team level. Many team cognitive processes, such as situation assessment and coordination, are thought to rely on team knowledge. Team knowledge is multifaceted and comprises relatively generic knowledge in the form of team mental models and more specific team situation models. In this methodological review paper, we review recent efforts to measure team knowledge in the context of mapping specific methods onto features of targeted team knowledge. Team knowledge features include type, homogeneity versus heterogeneity, and rate of knowledge change. Measurement features include knowledge elicitation method, team metric, and aggregation method. When available, we highlight analytical conclusions or empirical data that support a connection between team knowledge and measurement method. In addition, we present empirical results concerning the relation between team knowledge and performance for each measurement method and identify research and methodological needs. Addressing issues surrounding the measurement of team knowledge is a prerequisite to understanding team cognition and its relation to team performance and to designing training programs or devices to facilitate team cognition.

https://cerici.org/media&pubs/documents/docs/measteamknowfinal.pdf

Measuring Team Knowledge

Cognitive work analysis is gaining momentum as an approach for the analysis, design and evaluation of complex sociotechnical systems. This paper focuses on control task analysis (ConTA), the second phase of cognitive work analysis. The paper consolidates existing approaches to ConTA and extends the basic concepts, in particular, by asserting that activity in some work systems is better characterized by both work situations and work functions and by introducing a new formative representation for ConTA called the contextual activity template. In addition, the paper proposes a set of methodological guidelines for performing ConTA and presents a case study of a ConTA for a first-of-a-kind military system called Airborne Early Warning and Control. As well as illustrating the conceptual extensions and methodological guidelines for ConTA by example, this case study highlights some of the difficulties of conducting ConTA for first-of-a-kind, complex systems during the early stages of development.

Analysing activity in complex systems with cognitive work analysis: concepts, guidelines and case study for control task analysis

The purpose of this article is to present the best practices of cross-cultural training (CCT) in terms of what organizations are doing and what they should be doing. In presenting the best practice...

http://disciplinas.stoa.usp.br/pluginfile.php/3500/mod_resource/content/1/2005%20A%20Review%20of%20Cross-Cultural%20Training-%20Best%20Practices%2C%20Guidelines%2C%20and%20Research%20Needs.pdf

A review of cross-cultural training: Best practices, guidelines, and research needs.

: This report contributes to the development of a coherent theoretical and methodological approach for work domain analysis (WDA), the first phase of cognitive work analysis The report: (1) addresses a number of conceptual issues relating to WDA, including differences in the approaches of Rasmussen, Pejtersen, and Goodstein (1994) and Vicente (1999); (2) proposes a methodology for performing WDA; and (3) illustrates the theoretical concepts and methodology for WDA with a work domain of a home - a `system' that will be highly familiar to everyone This research will help to: make WDA more accessible to researchers and practitioners who were not involved in the development of WDA or who cannot be apprenticed to experts in WDA; reduce the amount of time and effort it takes to perform WDA even for experts in WDA; and facilitate the application of WDA to large-scale industry projects In addition, by making the methodology for WDA more explicit, this research will allow the methodology, or at least parts of the methodology, to be tested empirically

/pdf/work-domain-analysis-theoretical-concepts-and-methodology-43d3zhx39j.pdf

Work Domain Analysis: Theoretical Concepts and Methodology

An important part of the application of sociotechnical systems theory (STS) is the development of methods, tools and techniques to assess human factors and ergonomics workplace requirements. We focus in this paper on describing and evaluating current STS methods for workplace safety, as well as outlining a set of six case studies covering the application of these methods to a range of safety contexts. We also describe an evaluation of the methods in terms of ratings of their ability to address a set of theoretical and practical questions (e.g. the degree to which methods capture static/dynamic aspects of tasks and interactions between system levels). The outcomes from the evaluation highlight a set of gaps relating to the coverage and applicability of current methods for STS and safety (e.g. coverage of external influences on system functioning; method usability). The final sections of the paper describe a set of future challenges, as well as some practical suggestions for tackling these.

Practitioner Summary: We provide an up-to-date review of STS methods, a set of case studies illustrating their use and an evaluation of their strengths and weaknesses. The paper concludes with a ‘roadmap’ for future work.

https://www.tandfonline.com/doi/pdf/10.1080/00140139.2015.1015622

Defining the methodological challenges and opportunities for an effective science of sociotechnical systems and safety

The Applied Cognitive Work Analysis methodology has been specifically tailored to be the Cognitive Task Design portion of a high quality, affordable systems engineering process. The methodology is presented in its entirety from its knowledge elicitation beginnings to handoff to the software development team. Each step in the methodology is presented in ‘how to’ format, including design review and quality checkpoints that bring Cognitive Task Design to the same level of maturity as the best software engineering implementation practices. A very small-

Applied Cognitive Work Analysis: A Pragmatic Methodology for Designing Revolutionary Cognitive Affordances

Training Team Problem Solving Skills: An Event-Based Approach

Case Studies: Applied Cognitive Work Analysis in the Design of Innovative Decision Support

In order for cognitive systems engineering (CSE) to deliver the benefits of recent theoretical advances in actual systems being developed, the insights of CSE must be transformed into pragmatic engineering practices. The CSE engineering practices described in this article (using the applied cognitive systems engineering methodology as an exemplar) are typical of just such an engineering adaptation of rev- olutionary science and represent engineering practices that are dedicated to building effective systems. CSE research insights can have a significant impact on their corre- sponding systems engineering (SE) practices by expanding SE's concept of a system from just the technology components to a joint cognitive system (Hollnagel & Woods, 2005) and expanding the associated SE practices appropriately. This article uses the Department of Defense system life cycle and the SE V-model (Forsberg & Mooz, 1992) as SE process exemplars to illustrate how CSE engineering practices can be integrated into the SE process. Specifically, we propose four key integration points where CSE can contribute significantly to the SE process: concept refinement, software development, testing, and postsystem development (i.e., operations support, training, and mainte- nance). Our approach shows that the practice of CSE does not compete with SE but, instead, augments current SE practices to ensure that the technology components are engineered with the users' cognitive needs in mind.

Integrating Cognitive Systems Engineering Throughout the Systems Engineering Process

Cognitive Systems Engineering (CSE) techniques are widely used for the description and analysis of the sources of cognitive complexity and explicating the basis of expertise within a work domain. However, the results of the CSE techniques often focus on work analysis and are only weakly coupled to the design of decision support systems that are built based on those analyses. In fact, some within the CSE community have suggested that such a design epiphany occurs as if by magic. If CSE is to be treated as an engineering discipline, it cannot rely on magic to create systems. The approach described in this paper assumes that an explicit relationship between system design and supported cognitive work is fundamental to the design's effectiveness. The goal is a pragmatic, effective engineering process that explicitly designs systems according to relationships between cognitive work requirements and decision support concepts.

James W. Gualtieri

Papers

Applied Cognitive Work Analysis: A Pragmatic Methodology for Designing Revolutionary Cognitive Affordances

Training Team Problem Solving Skills: An Event-Based Approach

Case Studies: Applied Cognitive Work Analysis in the Design of Innovative Decision Support

Integrating Cognitive Systems Engineering Throughout the Systems Engineering Process

Cognitive System Engineering - Based Design: Alchemy or Engineering