scispace - formally typeset
Search or ask a question
Topic

Task analysis

About: Task analysis is a research topic. Over the lifetime, 10432 publications have been published within this topic receiving 283481 citations.


Papers
More filters
01 Jan 2003
TL;DR: A suite of Cognitive Tutor Authoring Tools (CTAT) intended to make tutor development both easier and faster for experienced modelers and possible for potential modelers who are not experts in cognitive psychology or artificial intelligence programming.
Abstract: We are developing a suite of Cognitive Tutor Authoring Tools (CTAT) intended to make tutor development both easier and faster for experienced modelers and possible for potential modelers who are not experts in cognitive psychology or artificial intelligence programming. Our goal is to demonstrate a reduction in development time by a factor of three. We employ Human-Computer Interaction (HCI) methods and Cognitive Science principles to design development tools that are both useful and useable. Our preliminary analytic and empirical analyses compare use of CTAT with use of our current develop environment and indicate a potential reduction in development time by a factor of about two. Toward a Rapid Development Environment for Cognitive Tutors Kenneth R. KOEDINGER Vincent A.W.M.M. ALEVEN Human-Computer Interaction Institute Carnegie Mellon University Neil HEFFERNAN Computer Science Department Worcester Polytechnic University Abstract. We are developing a suite of Cognitive Tutor Authoring Tools (CTAT) intended to make tutor development both easier and faster for experienced modelers and possible for potential modelers who are not experts in cognitive psychology or artificial intelligence programming. Our goal is to demonstrate a reduction in development time by a factor of three. We employ Human-Computer Interaction (HCI) methods and Cognitive Science principles to design development tools that are both useful and useable. Our preliminary analytic and empirical analyses compare use of CTAT with use of our current develop environment and indicate a potential reduction in development time by a factor of about two. We are developing a suite of Cognitive Tutor Authoring Tools (CTAT) intended to make tutor development both easier and faster for experienced modelers and possible for potential modelers who are not experts in cognitive psychology or artificial intelligence programming. Our goal is to demonstrate a reduction in development time by a factor of three. We employ Human-Computer Interaction (HCI) methods and Cognitive Science principles to design development tools that are both useful and useable. Our preliminary analytic and empirical analyses compare use of CTAT with use of our current develop environment and indicate a potential reduction in development time by a factor of about two. Cognitive Tutors have been demonstrated to yield dramatic improvements in student learning. For example, evaluations of the Algebra Cognitive Tutor have demonstrated that students in tutor classes outperform students in comparison classes [4]. This tutor is being marketed and in use in over 1000 schools across the US (see www.carnegielearning.com). Despite the great potential of Cognitive Tutors to improve student learning in other areas, development of such systems is currently costly and has been limited to just a few research teams. Such teams currently require PhD level expertise in cognitive task analysis and advanced AI programming to create the cognitive models that drive Cognitive Tutors. We have begun to create a development environment that addresses these difficulties. Our goal is to make tutor development both easier and faster for current developers and possible for researchers, trainers, and educators who are not experts in cognitive psychology or AI. We are designing, implementing, and evaluating Cognitive Tutor Authoring Tools (CTAT) that will support all phases of design and development. Creating an effective development environment is as much about getting the HCI details right as it is about innovation in algorithms. We use both empirical HCI methods, like Think Aloud user studies [3], as well as analytical methods, like Keystroke Level Modeling [2], to guide interface design. We employed such methods in an earlier project [5] involving the redesign of part of RIDES [6], an authoring environment for simulation-based intelligent tutoring systems, and were able to reduce programmer time by a factor of 2.6. 1. The Cognitive Tutor Authoring Tools Our rapid development environment, illustrated in Figure 1, consists of the following tools: An Intelligent GUI Builder, whose windows are shown in the top-left of Figure 1, can be used to create a graphical user interface (GUI) to be used in the tutor. The modeler can use the interface to demonstrate how to carry out the task to be modeled. A Behavior Recorder (top right), which records solution paths through a given problem scenario, as the modeler demonstrates these paths in the GUI. Figure 1: The prototype Cognitive Tutor Authoring Tools A WME Editor and a Production Rule Editor, dedicated editors used to implement the production rules that model the demonstrated paths (bottom left and bottom middle). A debugging tool called the Cognitive Model Visualizer, which has two windows shown in the bottom right (“Conflict Tree” and “Rule Instantiation”). 2. Preliminary Empirical and Analytic Evaluations to Guide Design In order to get an initial impression of the savings afforded by CTAT and of the ways in which the tools might be improved, we conducted preliminary analytical and empirical evaluations. Our preliminary analysis of CTAT used a method called the Keystroke Level Model (KLM) [2]. KLM is a way of estimating the time required for expert performance on routine tasks in a computer interface. The analyst creates a detailed specification of the task at the level of keystrokes and mouse clicks and uses it to estimate the time the task will take. Time estimates derived from KLM correlate well with expert performance times [2]. Using this method, we compared our existing modeling tools, an environment called TDK [1], which has been used for over a decade to create many large-scale Cognitive Tutors to both 1) the initial CTAT environment created in four months and 2) a preliminary redesign of CTAT that was not implemented at the time of the initial evaluation. We created simplified KLM models for three common modeling tasks, namely (1) creating the initial configuration in working memory for a problem scenario, (2) writing a production rule of medium complexity, and (3) debugging why a rule that was expected to fire did not. As shown in Table 1, the KLM analysis predicts that the current CTAT will reduce the time needed to create an initial working memory configuration by a factor of 2.3. It predicts lesser savings for the other tasks. However, the results in the future CTAT column indicate significant future savings in debugging, where programmers spend much of their time. We also conducted a preliminary empirical analysis comparing the amount of time it takes to complete a modeling task with the existing TDK and the current preliminary version of CTAT. The task was to implement, test, and debug a working memory Dr r ag & dr r op i i nt t er r f f ac c e des s i i gn & i i mpl l ement t at t i i on Us s er r gui i ded gener r al l i i z z at t i i on f f or r model l des s i i gn & i i mpl l ement t at t i i on Aut t omat t ed t t es s t t i i ng Debuggi i ng v v i i s s ual l i i z z at t i i ons s C C ogni i t t i i v v e t t as s k k anal l y y s s i i s s by y demons s t t r r at t i i on Table 1: Results of a preliminary evaluation of CTAT using Keystroke Level Models: estimates of the time (in seconds) spent at the keystroke level for three commonly-occurring modeling tasks TDK Current CTAT Future CTAT Initialization 209.3 90.6 (90.6) Writing rule 203.1 207.3 130.1 Debugging 39.3 30.8 12.5 representation of a problem scenario and a single production rule. One of the authors completed this task in 50 minutes using TDK. He then did the same task using CTAT, this time taking only 15 minutes. Finally, to address the possibility of confounding effects due to learning, the same author re-did the task using the TDK tools. This time he needed 30 minutes. Thus the time savings due to CTAT were considerable. As shown in Table 2, the majority of savings occurred at the debugging stage. We anticipate that there will be further savings as we continue to develop the tools and scale up to larger evaluation studies. Table 2: Minutes spent on various sub-tasks for each trial in a preliminary empirical evaluation of CTAT. 1. TDK 2. CTAT 3. Redo TDK Initialization 10 5 8 Writing Rule 12 5 5 Testing & Debugging 28 5 17 Total 50 15 30

79 citations

Journal ArticleDOI
TL;DR: In this article, a model of individual performance was investigated based on the specificity and difficulty of goals assigned and by the frequency of performance intervals allocated for task completion, and the form of these relationships was considered to be shaped by the prescriptiveness of the task process involved.

79 citations

Journal ArticleDOI
TL;DR: A simple search task is sufficient to detect a performance advantage associated with higher levels of autistic traits and has the advantages of a solid footing in visual theory and being readily repeatable for the purpose of assessing performance variability and change with interventions.

79 citations

Journal ArticleDOI
Junya Fukuta1
TL;DR: This article examined learners' attention shifts during repeated task engagement by methodological triangulation and found that learners focused more on the syntactic encoding process and less on conceptualizing process at the second task enactment when they engaged in the same task twice.
Abstract: Task repetition facilitates learners’ performance, at least temporarily: Since learners are already familiar with the content of the task at the initial enactment, they are capable of focusing their attention on linguistic form during the following enactment. However, the analysis in previous studies treated various aspects of ‘form’ as one package. This study examined learners’ attention shifts during repeated task engagement by methodological triangulation. The experiment required 28 Japanese learners of English to perform narrative tasks twice. Learners’ performance was analysed, and the protocol data elicited by stimulated recall were coded along their attention orientations. The result partially supported the form-focused effects of task repetition: learners focused more on the syntactic encoding process and less on the conceptualizing process at the second task enactment when they engaged in the same task twice. Possible theoretical and pedagogical implications are discussed.

79 citations


Network Information
Related Topics (5)
Feature extraction
111.8K papers, 2.1M citations
78% related
Robustness (computer science)
94.7K papers, 1.6M citations
78% related
User interface
85.4K papers, 1.7M citations
78% related
The Internet
213.2K papers, 3.8M citations
77% related
Deep learning
79.8K papers, 2.1M citations
77% related
Performance
Metrics
No. of papers in the topic in previous years
YearPapers
202328
202264
2021665
2020819
2019737
2018834