Although training evaluation is recognized as an important component of the instructional design model, there are no theoretically based models of training evaluation. This article attempts to move toward such a model by developing a classification scheme for evaluating learning outcomes. Learning constructs are derived from a variety of research domains, such as cognitive, social, and instructional psychology and human factors. Drawing from this research, we propose cognitive, skill-based, and affective learning outcomes (relevant to training) and recommend potential evaluation measures. The learning outcomes and associated evaluation measures are organized into a classification scheme. Requirements for providing construct-oriented evidence of validity for the scheme are also discussed

Application of cognitive, skill-based, and affective theories of learning outcomes to new methods of training evaluation.

Cognitive and usability engineering methods for the evaluation of clinical information systems

How do people perceive routine events, such as making a bed, as these events unfold in time? Research on knowledge structures suggests that people conceive of events as goal-directed partonomic hierarchies. Here, participants segmented videos of events into coarse and fine units on separate viewings; some described the activity of each unit as well. Both segmentation and descriptions support the hierarchical bias hypothesis in event perception: Observers spontaneously encoded the events in terms of partonomic hierarchies. Hierarchical organization was strengthened by simultaneous description and, to a weaker extent, by familiarity. Describing from memory rather than perception yielded fewer units but did not alter the qualitative nature of the descriptions. Although the descriptions were telegraphic and without communicative intent, their hierarchical structure was evident to naive readers. The data suggest that cognitive schemata mediate between perceptual and functional information about events and indicate that these knowledge structures may be organized around object/action units.

Perceiving, remembering, and communicating structure in events

▪ Abstract Cognitive skill acquisition is acquiring the ability to solve problems in intellectual tasks, where success is determined more by subjects' knowledge than by their physical prowess. This review considers research conducted in the past ten years on cognitive skill acquisition. It covers the initial stages of acquiring a single principle or rule, the initial stages of acquiring a collection of interacting pieces of knowledge, and the final stages of acquiring a skill, wherein practice causes increases in speed and accuracy.

Cognitive skill acquisition

This review pursues a twofold goal, the first is to preserve and enhance the chronicles of recent educational data mining (EDM) advances development; the second is to organize, analyze, and discuss the content of the review based on the outcomes produced by a data mining (DM) approach. Thus, as result of the selection and analysis of 240 EDM works, an EDM work profile was compiled to describe 222 EDM approaches and 18 tools. A profile of the EDM works was organized as a raw data base, which was transformed into an ad-hoc data base suitable to be mined. As result of the execution of statistical and clustering processes, a set of educational functionalities was found, a realistic pattern of EDM approaches was discovered, and two patterns of value-instances to depict EDM approaches based on descriptive and predictive models were identified. One key finding is: most of the EDM approaches are ground on a basic set composed by three kinds of educational systems, disciplines, tasks, methods, and algorithms each. The review concludes with a snapshot of the surveyed EDM works, and provides an analysis of the EDM strengths, weakness, opportunities, and threats, whose factors represent, in a sense, future work to be fulfilled.

https://romisatriawahono.net/lecture/rm/survey/machine%20learning/Ayala%20-%20Educational%20Data%20Mining%20-%202014.pdf

Review: Educational data mining: A survey and a data mining-based analysis of recent works

Contents: Maj. K.R. Rose, Foreword: What the Army Expects from Intelligent Training Systems. J. Psotka, L.D. Massey, S.A. Mutter, Introduction. Section I:Knowledge Acquisition. W.B. Johnson, Developing Expert System Knowledge Bases in Technical Training Environments. B. Means, S.P. Gott, Cognitive Task Analysis as a Basis for Tutor Development: Articulating Abstract Knowledge Representations. Y.J. Tenney, L.C. Kurland, The Development of Troubleshooting Expertise in Radar Mechanics. D. Kieras, What Mental Model Should be Taught: Choosing Instructional Content for Complex Engineered Systems. Section II:Intelligent Instructional Design. L.C. Kurland, Y.J. Tenney, Issues in Developing an Intelligent Tutor for a Real-World Domain: Training Radar Mechanics. P.L. Pirolli, J.G. Greeno, The Problem Space of Instructional Design. D.M. Russell, T.P. Moran, D.S. Jordan, The Instructional-Design Environment. S.A. MacMillan, D. Emme, M. Berkowitz, Instructional Planners: Lessons Learned. D.C. Wilkins, W.J. Clancey, B.G. Buchanan, Using and Evaluating Differential Modeling in Intelligent Tutoring and Apprentice Learning Systems. Section III:Knowledge Representation. F. Ritter, W. Feurzeig, Teaching Real-Time Tactical Thinking. T. Govindaraj, Intelligent Computer Aids for Fault Diagnosis Training of Expert Operators of Large Dynamic Systems. D.M. Russell, IDE: The Interpreter. J.R. Frederiksen, B.Y. White, A. Collins, G. Eggan, Intelligent Tutoring Systems for Electronic Troubleshooting. L.D. Massey, J. de Bruin, B. Roberts, A Training System for Radar Maintenance. Section IV:Intelligent Tutoring Architectures. J.G. Bonar, R. Cunningham, Bridge: Tutoring the Programming Process. W. Feurzeig, F. Ritter, Understanding Reflective Problem Solving. D. Frye, D.C. Littman, E. Soloway, The Next Wave of Problems in ITS: Confronting the "User Issues" of Interface Design and System Evaluation. D.M. Towne, A. Munro, The Intelligent Maintenance Training System.

L. Daniel Massey

Papers

Intelligent tutoring systems : lessons learned