Showing papers on "Intelligent tutoring system published in 2013"

AutoTutor and affective autotutor: Learning by talking with cognitively and emotionally intelligent computers that talk back

Abstract: We present AutoTutor and Affective AutoTutor as examples of innovative 21st century interactive intelligent systems that promote learning and engagement. AutoTutor is an intelligent tutoring system that helps students compose explanations of difficult concepts in Newtonian physics and enhances computer literacy and critical thinking by interacting with them in natural language with adaptive dialog moves similar to those of human tutors. AutoTutor constructs a cognitive model of students' knowledge levels by analyzing the text of their typed or spoken responses to its questions. The model is used to dynamically tailor the interaction toward individual students' zones of proximal development. Affective AutoTutor takes the individualized instruction and human-like interactivity to a new level by automatically detecting and responding to students' emotional states in addition to their cognitive states. Over 20 controlled experiments comparing AutoTutor with ecological and experimental controls such reading a textbook have consistently yielded learning improvements of approximately one letter grade after brief 30--60-minute interactions. Furthermore, Affective AutoTutor shows even more dramatic improvements in learning than the original AutoTutor system, particularly for struggling students with low domain knowledge. In addition to providing a detailed description of the implementation and evaluation of AutoTutor and Affective AutoTutor, we also discuss new and exciting technologies motivated by AutoTutor such as AutoTutor-Lite, Operation ARIES, GuruTutor, DeepTutor, MetaTutor, and AutoMentor. We conclude this article with our vision for future work on interactive and engaging intelligent tutoring systems.

Motivation and performance in a game-based intelligent tutoring system.

Abstract: One strength of educational games stems from their potential to increase students’ motivation and engagement during educational tasks. However, game features may also detract from principle learning goals and interfere with students’ ability to master the target material. To assess the potential impact of game-based learning environments, in this study we examined motivation and learning for 84 high-school students across eight 1-hr sessions comparing 2 versions of a reading strategy tutoring system, an intelligent tutoring system (iSTART) and its game-based version (iSTART–ME). The results demonstrate equivalent target task performance (i.e., learning) across environments at pretest, posttest, and retention, but significantly higher levels of enjoyment and motivation for the game-based system. Analyses of performance across sessions reveal an initial decrease in performance followed by improvement within the game-based training condition. These results suggest possible constraints and benefits of game-based training, including time-scale effects. The findings from this study offer a potential explanation for some of the mixed findings within the literature and support the integration of game-based features within intelligent tutoring environments that require long-term interactions for students to develop skill mastery.

Using adaptive learning technologies to personalize instruction to student interests: The impact of relevant contexts on performance and learning outcomes.

Abstract: Adaptive learning technologies are emerging in educational settings as a means to customize instruction to learners’ background, experiences, and prior knowledge. Here, a technology-based personalization intervention within an intelligent tutoring system (ITS) for secondary mathematics was used to adapt instruction to students’ personal interests. We conducted a learning experiment where 145 ninth-grade Algebra I students were randomly assigned to 2 conditions in the Cognitive Tutor Algebra ITS. For 1 instructional unit, half of the students received normal algebra story problems, and half received matched problems personalized to their out-of-school interests in areas such as sports, music, and movies. Results showed that students in the personalization condition solved problems faster and more accurately within the modified unit. The impact of personalization was most pronounced for 1 skill in particular—writing symbolic equations from story scenarios—and for 1 group of students in particular—students who were struggling to learn within the tutoring environment. Once the treatment had been removed, students who had received personalization continued to write symbolic equations for normal story problems with increasingly complex structures more accurately and with greater efficiency. Thus, we provide evidence that interest-based interventions can promote robust learning outcomes—such as transfer and accelerated future learning—in secondary mathematics. These interest-based connections may allow for abstract ideas to become perceptually grounded in students’ experiences such that they become easier to grasp. Adaptive learning technologies that utilize interest may be a powerful way to support learners in gaining fluency with abstract representational systems.

New Potentials for Data-Driven Intelligent Tutoring System Development and Optimization

Abstract: Increasing widespread use of educational technologies is producing vast amounts of data. Such data can be used to help advance our understanding of student learning and enable more intelligent, interactive, engaging, and effective education. In this article, we discuss the status and prospects of this new and powerful opportunity for data-driven development and optimization of educational technologies, focusing on intelligent tutoring systems We provide examples of use of a variety of techniques to develop or optimize the select, evaluate, suggest, and update functions of intelligent tutors, including probabilistic grammar learning, rule induction, Markov decision process, classification, and integrations of symbolic search and statistical inference.

From Log Files to Assessment Metrics: Measuring Students' Science Inquiry Skills Using Educational Data Mining

Abstract: We present a method for assessing science inquiry performance, specifically for the inquiry skill of designing and conducting experiments, using educational data mining on students' log data from online microworlds in the Inq-ITS system (Inquiry Intelligent Tutoring System; www.inq-its.org). In our approach, we use a 2-step process: First we use text replay tagging, a type of rapid protocol analysis in which categories are developed and, in turn, used to hand-score students' log data. In the second step, educational data mining is conducted using a combination of the text replay data and machine-distilled features of student interactions in order to produce an automated means of assessing the inquiry skill in question; this is referred to as a detector. Once this detector is appropriately validated, it can be applied to students' log files for auto-assessment and, in the future, to drive scaffolding in real time. Furthermore, we present evidence that this detector developed in 1 scientific domain, phase c...

Writing pal: Feasibility of an intelligent writing strategy tutor in the high school classroom

Abstract: The Writing Pal (W-Pal) is a novel intelligent tutoring system (ITS) that offers writing strategy instruction, game-based practice, essay writing practice, and formative feedback to developing writers. Compared to more tractable and constrained learning domains for ITS, writing is an ill-defined domain because the features of effective writing are difficult to quantify and individual writers can employ diverse strategies to achieve similar goals. The development of an ITS in an ill-defined domain presents particular challenges regarding comprehensive instruction, modularized content, extended practice, and formative feedback. In this article, we describe how the development of W-Pal has uniquely addressed these concerns and present the results of a study assessing the feasibility of this system in high school English classrooms. This study included 2 teachers and their 141 10th grade English class students who utilized W-Pal over a 6-month period during the academic year. Log-file analyses showed that students used all aspects of W-Pal, but activity and engagement was uneven throughout the year and decreased over time. Essay scores improved over time and surveys indicated that students perceived the lessons, games, and feedback as beneficial. However, specific aspects of the learning environment were critiqued as annoying, challenging, or lacking specificity. Overall, the results suggest that the system was generally well-received by the students but also offer insights for the development of ITSs in ill-defined domains.

Natural language processing in an intelligent writing strategy tutoring system

Abstract: The Writing Pal is an intelligent tutoring system that provides writing strategy training. A large part of its artificial intelligence resides in the natural language processing algorithms to assess essay quality and guide feedback to students. Because writing is often highly nuanced and subjective, the development of these algorithms must consider a broad array of linguistic, rhetorical, and contextual features. This study assesses the potential for computational indices to predict human ratings of essay quality. Past studies have demonstrated that linguistic indices related to lexical diversity, word frequency, and syntactic complexity are significant predictors of human judgments of essay quality but that indices of cohesion are not. The present study extends prior work by including a larger data sample and an expanded set of indices to assess new lexical, syntactic, cohesion, rhetorical, and reading ease indices. Three models were assessed. The model reported by McNamara, Crossley, and McCarthy (Written Communication 27:57-86, 2010) including three indices of lexical diversity, word frequency, and syntactic complexity accounted for only 6% of the variance in the larger data set. A regression model including the full set of indices examined in prior studies of writing predicted 38% of the variance in human scores of essay quality with 91% adjacent accuracy (i.e., within 1 point). A regression model that also included new indices related to rhetoric and cohesion predicted 44% of the variance with 94% adjacent accuracy. The new indices increased accuracy but, more importantly, afford the means to provide more meaningful feedback in the context of a writing tutoring system.

Natural Language Processing in an Intelligent Writing Strategy Tutoring System.

Abstract: The Writing Pal is an intelligent tutoring system that provides writing strategy training. A large part of its artificial intelligence resides in the natural language processing algorithms to assess essay quality and guide feedback to students. Because writing is often highly nuanced and subjective, the development of these algorithms must consider a broad array of linguistic, rhetorical, and contextual features. This study assesses the potential for computational indices to predict human ratings of essay quality. Past studies have demonstrated that linguistic indices related to lexical diversity, word frequency, and syntactic complexity are significant predictors of human judgments of essay quality but that indices of cohesion are not. The present study extends prior work by including a larger data sample and an expanded set of indices to assess new lexical, syntactic, cohesion, rhetorical, and reading ease indices. Three models were assessed. The model reported by McNamara, Crossley, and McCarthy (Written Communication 27:57-86, 2010) including three indices of lexical diversity, word frequency, and syntactic complexity accounted for only 6% of the variance in the larger data set. A regression model including the full set of indices examined in prior studies of writing predicted 38% of the variance in human scores of essay quality with 91% adjacent accuracy (i.e., within 1 point). A regression model that also included new indices related to rhetoric and cohesion predicted 44% of the variance with 94% adjacent accuracy. The new indices increased accuracy but, more importantly, afford the means to provide more meaningful feedback in the context of a writing tutoring system.

Recent Advances in Conversational Intelligent Tutoring Systems

Abstract: We report recent advances in intelligent tutoring systems with conversational dialogue. We highlight progress in terms of macro and microadaptivity. Macroadaptivity refers to a system’s capability to select appropriate instructional tasks for the learner to work on. Microadaptivity refers to a system’s capability to adapt its scaffolding while the learner is working on a particular task. The advances in macro and microadaptivity that are presented here were made possible by the use of learning progressions, deeper dialogue and natural language processing techniques, and by the use of affect-enabled components. Learning progressions and deeper dialogue and natural language processing techniques are key features of DeepTutor, the first intelligent tutoring system based on learning progressions. These improvements extend the bandwidth of possibilities for tailoring instruction to each individual student which is needed for maximizing engagement and ultimately learning.

Predicting College Enrollment from Student Interaction with an Intelligent Tutoring System in Middle School.

Abstract: Research shows that middle school is an important juncture for a student where he or she starts to be conscious about academic achievement and thinks about college attendance. It is already known that access to financial resources, family background, career aspirations and academic ability are indicative of a student’s choice to attend college; though these variables are interesting, they do not necessarily give sufficient actionable information to instructors or guidance counselors to intervene for individual students. However, increasing numbers of students are using educational software at this phase of their education, and detectors of specific aspects of student learning and engagement have been developed for these types of learning environments. If these types of models can be used to predict college attendance, it may provide more actionable information than the previous generation of predictive models. In this paper, we predict college attendance from these types of detectors, in the context of 3,747 students using the ASSISTment system in New England, producing detection that is both successful and potentially more actionable than previous approaches; we can distinguish between a student who will attend college and a student who will not attend college 68.6% of the time.

The effect of positive feedback in a constraint-based intelligent tutoring system

Abstract: Tutoring technologies for supporting learning from errors via negative feedback are highly developed and have proven their worth in empirical evaluations However, observations of empirical tutoring dialogs highlight the importance of positive feedback in the practice of expert tutoring We hypothesize that positive feedback works by reducing student uncertainty about tentative but correct problem solving steps Positive feedback should communicate three pieces of explanatory information: (a) those features of the situation that made the action the correct one, both in general terms and with reference to the specifics of the problem state; (b) the description of the action at a conceptual level and (c) the important aspect of the change in the problem state brought about by the action We describe how a positive feedback capability was implemented in a mature, constraint-based tutoring system, SQL-Tutor, which teaches by helping students learn from their errors Empirical evaluation shows that students who were interacting with the augmented version of SQL-Tutor learned at twice the speed as the students who interacted with the standard, error feedback only, version We compare our approach with some alternative techniques to provide positive feedback in intelligent tutoring systems

Clustering and Profiling Students According to their Interactions with an Intelligent Tutoring System Fostering Self-Regulated Learning

Abstract: In this paper, we present the results obtained using a clustering algorithm (Expectation-Maximization) on data collected from 106 college students learning about the circulatory system with MetaTutor, an agent-based Intelligent Tutoring System (ITS) designed to foster self-regulated learning (SRL). The three extracted clusters were validated and analyzed using multivariate statistics (MANOVAs) in order to characterize three distinct profiles of students, displaying statistically significant differences over all 12 variables used for the clusters formation (including performance, use of note-taking and number of sub-goals attempted). We show through additional analyses that variations also exist between the clusters regarding prompts they received by the system to perform SRL processes. We conclude with a discussion of implications for designing a more adaptive ITS based on an identification of learners’ profiles.

Supporting Students’ Self-Regulated Learning with an Open Learner Model in a Linear Equation Tutor

Abstract: Self-assessment and study choice are two important metacognitive processes involved in Self-Regulated Learning. Yet not much empirical work has been conducted in ITSs to investigate how we can best support these two processes and improve students’ learning outcomes. The present work redesigned an Open Learner Model (OLM) with three features aimed at supporting self-assessment (self-assessment prompts, delaying the update of the skill bars and progress information on the problem type level). We also added a problem selection feature. A 2x2 experiment with 62 7th graders using variations of an ITS for linear equation solving found that students who had access to the OLM performed significantly better on the post-test. To the best of our knowledge, the study is the first experimental study that shows an OLM enhances students’ learning outcomes with an ITS. It also helps establish that self-assessment has key influence on student learning of problem solving tasks.

Design and development of an innovative individualized adaptive and intelligent e-learning system for teaching-learning of probability unit: Details of UZWEBMAT

Abstract: In this study, an innovative adaptive and intelligent web based e-learning system, UZWEBMAT (Turkish abbreviation of Adaptive and INtelligent WEB based MAThematics teaching-learning system) was designed, developed and implemented. This e-learning system was intended for learning and teaching secondary school level permutation-combination-binomial expansion and probability subjects. Content which was prepared according to Turkish curriculum for secondary school mathematics course was transformed into learning objects in three different ways in accordance with VAK (Visual-Auditory-Kinesthetic) learning styles. Primary/secondary/tertiary learning styles of learners registering the system are determined and each learner receives the content appropriate for his/her dominant learning style. Also, they can be directed to contents of other styles according to their performances thanks to an expert system. Learning objects constituting the content were prepared according to constructivist approach. An active role for the learner was the purpose. Tips and intelligent solution supports within the learning objects were presented with expert system support to the learners. With this structure, UZWEBMAT bears the characteristics of intelligent tutoring system as well as an adaptive e-learning environment. All the movements of learners studying with UZWEBMAT are recorded and the necessary information is reported to both learners and teachers in a visualized way.

Rule-based expert systems to support step-by-step guidance in algebraic problem solving: The case of the tutor PAT2Math

Abstract: In order for an Intelligent Tutoring System (ITS) to correct students’ exercises, it must know how to solve the same type of problems that students do and the related knowledge components. It can, thereby, compare the desirable solution with the student’s answer. This task can be accomplished by an expert system. However, it has some drawbacks, such as an exponential complexity time, which impairs the desirable real-time response. In this paper we describe the expert system (ES) module of an Algebra ITS, called PAT2Math. The ES is responsible for correcting student steps and modeling student knowledge components during equations problem solving. Another important function of this module is to demonstrate to students how to solve a problem. In this paper, we focus mainly on the implementation of this module as a rule-based expert system. We also describe how we reduced the complexity of this module from O(nd) to O(d), where n is the number of rules in the knowledge base, by implementing some meta-rules that aim at inferring the operations students applied in order to produce a step. We evaluated our approach through a user study with forty-three seventh grade students. The students who interacted with our tool showed statistically higher scores on equation solving tests, after solving algebra exercises with PAT2Math during an approximately two-hour session, than students who solved the same exercises using only paper and pencil.

Towards an Understanding of Affect and Knowledge from Student Interaction with an Intelligent Tutoring System

Abstract: Csikszentmihalyi’s Flow theory states that a balance between challenge and skill leads to high engagement, overwhelming challenge leads to anxiety or frustration, and insufficient challenge leads to boredom. In this paper, we test this theory within the context of student interaction with an intelligent tutoring system. Automated detectors of student affect and knowledge were developed, validated, and applied to a large data set. The results did not match Flow theory: boredom was more common for poorly-known material, and frustration was common both for very difficult material and very easy material. These results suggest that design for optimal engagement within online learning may require further study of the factors leading students to become bored on difficult material, and frustrated on very well-known material.

My Science Tutor: A Conversational Multimedia Virtual Tutor.

Abstract: My Science Tutor (MyST) is an intelligent tutoring system designed to improve science learning by elementary school students through conversational dialogs with a virtual science tutor in an interactive multimedia environment. Marni, a lifelike 3-D character, engages individual students in spoken dialogs following classroom investigations using the kit-based Full Option Science System program. MyST attempts to elicit self-expression from students; process their spoken explanations to assess understanding; and scaffold learning by asking open-ended questions accompanied by illustrations, animations, or interactive simulations related to the science concepts being learned. MyST uses automatic speech recognition, natural language processing, and dialog-modeling technologies to interpret student responses and manage the dialog. Sixteen 20-min tutorials were developed for each of 4 areas of science taught in 3rd, 4th, and 5th grades. During summative evaluation of the program, students received one-on-one tutoring via MyST or an expert human tutor following classroom instruction on the science topic, representing over 4.5 hr of tutoring across the 16 sessions. A quasi-experimental design was used to compare average learning gain for 3 groups: human tutoring, virtual tutoring, and no tutoring. Learning gain was measured using standardized assessments given to students in each condition before and after each science module. Results showed that students in both the human and virtual tutoring groups had significant learning gains relative to students in the control classrooms and that there were no significant differences in learning gains between students in the human and MyST human tutoring conditions. Both teachers and students gave high-positive survey ratings to MyST.

High-fidelity implementation of web-based intelligent tutoring system improves fourth and fifth graders content area reading comprehension

Abstract: Technologies and their effectiveness are impacted by how well they are implemented. A large scale randomized controlled trial was conducted to study the efficacy of a web-based intelligent tutoring system to deliver the structure strategy to improve content area reading comprehension. We present our theory of change focusing on the theoretical framework: structure strategy, delivery approach of web-based intelligent tutoring systems, and contextual conditions for successful adoption of the tool with fidelity. Results from the optimal implementation schools show statistically significantly better performance by ITSS classrooms compared to their control counterparts with moderate to large effect sizes. Conditions for implementing technology-based interventions with fidelity in schools are discussed.

A fuzzy linguistic algorithm for adaptive test in Intelligent Tutoring System based on competences

Abstract: The Computerized Adaptive Tests (CAT) are common tools for the diagnosis process in Intelligent Tutor System based on Competency education (ITS-C). The item selection process to form a CAT plays a key role because it must ensure the selection of the item that best contributes to student assessment at any time. The item selection mechanisms proposed in the literature present some limitations that decrease the efficiency of CAT and its adaptation to the student profile. This paper introduces a new item selection algorithm, based on a multi-criteria decision model that integrates experts' knowledge modeled by fuzzy linguistic information that overcomes previous limitations and enhances the accuracy of diagnosis and the adaptation of CAT to student's competence level. Finally, such an algorithm is deployed in a mobile tool for an ITS-C.

Fundamentals of the design and the operation of an intelligent tutoring system for the learning of the arithmetical and algebraic way of solving word problems

Abstract: Designers of interactive learning environments with a focus on word problem solving usually have to compromise between the amount of resolution paths that a user is allowed to follow and the quality of the feedback provided. We have built an intelligent tutoring system (ITS) that is able to both track the user's actions and provide adequate supervision during the resolution. This is done without imposing any restriction on the resolution paths that are allowed. Instead, the system attempts to enforce metacognitive learning by requiring an appropriate definition of quantities before they are used. The program (a) supports both the arithmetical and algebraic way of solving problems; (b) allows the recurrence to one or more equations when solving a problem in an algebraic way; (c) determines the validity of the expressions when they are introduced; and (d) admits the incorporation of new problems without the need of being reprogrammed. In this paper, we explain the design foundations, which are mainly based on (a) a reflection of the steps that a student should follow to solve a problem in an algebraic way, and (b) the use of a domain specific notation to represent both the problem structure and the current state of the resolution process. In particular, hypergraphs are introduced as an adequate way to support tracking in both the arithmetical and algebraic case. Moreover, we offer an extensive experimental evaluation which highlights the potential of the ITS as a learning tool.

Learning intercultural communication skills with virtual humans: Feedback and fidelity.

Abstract: In the context of practicing intercultural communication skills, we investigated the role of fidelity in a game-based, virtual learning environment as well as the role of feedback delivered by an intelligent tutoring system. In 2 experiments, we compared variations on the game interface, use of the tutoring system, and the form of the feedback. Our findings suggest that for learning basic intercultural communicative skills, a 3-dimensional (3-D) interface with animation and sound produced equivalent learning to a more static 2-D interface. However, learners took significantly longer to analyze and respond to the actions of animated virtual humans, suggesting a deeper engagement. We found large gains in learning across conditions. There was no differential effect with the tutor engaged, but it was found to have a positive impact on learner success in a transfer task. This difference was most pronounced when the feedback was delivered in a more general form versus a concrete style.

Domain-specific knowledge representation and inference engine for an intelligent tutoring system

Abstract: One of the most challenging steps in learning algebra is the translation of word problems into symbolic notation. This paper describes an Intelligent Tutoring System (ITS) that focuses on this stage of the problem solving process. On the one hand, a domain specific inference engine and a knowledge representation mechanism are proposed. These are based on a description language based on hypergraphs, and the idea of using conceptual schemes to represent the student's knowledge. As a result, the system is able to simultaneously: (a) represent all potential algebraic solutions to a given word problem; (b) keep track of the student's actions; (c) univocally determine the current state of the resolution process; (d) build a student model; and (e) provide adaptive automatic remediation. On the other hand, the Graphical User Interface (GUI) has been designed to force the student to follow the sequence of steps described in the Cartesian method. The ITS has been evaluated in an educational environment. The results show significant gains for the experimental group and hence support the use of the ITS in practice.

Predicting Robust Learning With the Visual Form of the Moment-by-Moment Learning Curve

Abstract: We present a new method for analyzing a student's learning over time for a specific skill: analysis of the graph of the student's moment-by-moment learning over time. Moment-by-moment learning is calculated using a data-mined model that assesses the probability that a student learned a skill or concept at a specific time during learning (Baker, Goldstein, & Heffernan, 2010, 2011). Two coders labeled data from students who used an intelligent tutoring system for college genetics. They coded in terms of 7 forms that the moment-by-moment learning curve can take. These labels are correlated to test data on the robustness of students’ learning. We find that different visual forms are correlated with very different learning outcomes. This work suggests that analysis of moment-by-moment learning curves may be able to shed light on the implications of students’ different patterns of learning over time.

Using Automatic Scoring Models to Detect Changes in Student Writing in an Intelligent Tutoring System

Abstract: This study compares automated scoring increases and linguistic changes for student writers in two groups: a group that used an intelligent tutoring system embedded with an automated writing evaluation component (Writing Pal) and a group that used only the automated writing evaluation component. The primary goal is to examine automated scoring differences in both groups from pretest to posttest essays to investigate score gains and linguistic development. The study finds that both groups show significant increases in automated writing scores and significant development in lexical, syntactic, cohesion, and rhetorical features. However, the Writing-Pal group shows greater raw frequency gains (i.e., negative v. positive gains).

The development and analysis of tutorial dialogues in AutoTutor Lite.

Abstract: The goal of intelligent tutoring systems (ITS) that interact in natural language is to emulate the benefits that a well-trained human tutor provides to students, by interpreting student answers and appropriately responding in order to encourage elaboration. BRCA Gist is an ITS developed using AutoTutor Lite, a Web-based version of AutoTutor. Fuzzy-trace theory theoretically motivated the development of BRCA Gist, which engages people in tutorial dialogues to teach them about genetic breast cancer risk. We describe an empirical method to create tutorial dialogues and fine-tune the calibration of BRCA Gist’s semantic processing engine without a team of computer scientists. We created five interactive dialogues centered on pedagogic questions such as “What should someone do if she receives a positive result for genetic risk of breast cancer?” This method involved an iterative refinement process of repeated testing with different texts and successively making adjustments to the tutor’s expectations and settings in order to improve performance. The goal of this method was to enable BRCA Gist to interpret and respond to answers in a manner that best facilitated learning. We developed a method to analyze the efficacy of the tutor’s dialogues. We found that BRCA Gist’s assessment of participants’ answers was highly correlated with the quality of the answers found by trained human judges using a reliable rubric. The dialogue quality between users and BRCA Gist predicted performance on a breast cancer risk knowledge test completed after exposure to the tutor. The appropriateness of BRCA Gist’s feedback also predicted the quality of answers and breast cancer risk knowledge test scores.

Towards Automatically Detecting Whether Student Learning is Shallow

Abstract: Recent research has extended student modeling to infer not just whether a student knows a skill or set of skills, but also whether the student has achieved robust learning—learning that enables the student to transfer their knowledge and prepares them for future learning (PFL). However, a student may fail to have robust learning in two fashions: they may have no learning, or they may have shallow learning (learning that applies only to the current skill, and does not support transfer or PFL). Within this paper, we present automated detectors which identify shallow learners, who are likely to need different intervention than students who have not yet learned at all. These detectors are developed using K* machine learned models, with data from college students learning introductory genetics from an intelligent tutoring system.

Using Automated Indices of Cohesion to Evaluate an Intelligent Tutoring System and an Automated Writing Evaluation System

Abstract: We present an evaluation of the Writing Pal (W-Pal) intelligent tutoring system (ITS) and the W-Pal automated writing evaluation (AWE) system through the use ofcomputational indices related to text cohesion. Sixty-four students participated in this study. Each student was assigned to either the W-Pal ITS condition or the W-Pal AWE condition. The W-Pal ITS includes strategy instruction, game-based practice, and essay-based practice with automated feedback. In the ITS condition, students received strategy training and wrote and revised one essay in each of the 8 training sessions. In the AWE condition, students only interacted with the essay writing and feedback tools. These students wrote and revised two essays in each of the 8 sessions. Indices of local and global cohesion reported by the computational tools Coh-Metrix and the Writing Assessment Tool (WAT) were used to investigate pretest and posttest writing gains. For both the ITS and the AWE systems, training led to the increased use of global cohesion features in essay writing. This study demonstrates that automated indices of text cohesion can be used to evaluate the effects of ITSs and AWE systems and further demonstrates how text cohesion develops as a result of instruction, writing, and automated feedback.

Fine-Grained Assessment of Motivation over Long Periods of Learning with an Intelligent Tutoring System: Methodology, Advantages, and Preliminary Results

Abstract: Models of self-regulated learning (SRL) describe the complex and dynamic interplay of learners’ cognitions, motivations, and behaviors when engaged in a learning activity Recently, researchers have begun to use fine-grained behavioral data such as think aloud protocols and log-file data from educational software to test hypotheses regarding the cognitive and metacognitive processes underlying SRL Motivational states, however, have been more difficult to trace through these methods and have primarily been studied via pre- and posttest questionnaires This is problematic because motivation can change during an activity or unit and without fine-grained assessment, dynamic relations between motivation, cognitive, and metacognitive processes cannot be studied In this chapter we describe a method for collecting fine-grained assessments of motivational variables and examine their association with cognitive and metacognitive behaviors for students learning mathematics with intelligent tutoring systems Students completed questionnaires embedded in the tutoring software before and after a math course and at multiple time points during the course We describe the utility of this method for assessing motivation and use these assessments to test hypotheses of self-regulated learning and motivation Learners’ reports of their motivation varied across domain and unit-level assessments and were differently predictive of learning behaviors

Online Homework Managers and Intelligent Tutoring Systems: A Study of Their Impact on Student Learning in the Introductory Financial Accounting Classroom

Abstract: The online homework manager (OHM) and the intelligent tutoring system (ITS) are two supplemental teaching tools available for accounting educators' use in the introductory financial accounting course. While research related to these systems is limited, prior studies find a tenuous performance advantage related to their use. To advance the literature in this area, this paper evaluates the performance benefit related to an OHM and an ITS, each employed independently as an additional study aid during the first course unit in one of two sections of the introductory financial accounting course. A third section used paper-and-pencil only and served as a control group. Results of tests on several performance measures did not identify a learning advantage associated with either the OHM or the ITS. Nor was a learning advantage identified when this study's results were compared to exam results from 14 previous semesters. Implications for accounting educators and future research directions are discussed.

Using student interactions to foster rule-diagram mapping during problem solving in an intelligent tutoring system

Abstract: In many domains, problem solving involves the application of general domain principles to specific problem representations. In 3 classroom studies with an intelligent tutoring system, we examined the impact of (learner-generated) interactions and (tutor-provided) visual cues designed to facilitate rule–diagram mapping (where students connect domain knowledge to problem diagrams), with the goal of promoting students’ understanding of domain principles. Understanding was not supported when students failed to form a visual representation of rule–diagram mappings (Experiment 1); student interactions with diagrams promoted understanding of domain principles, but providing visual representations of rule–diagram mappings negated the benefits of interaction (Experiment 2). However, scaffolding student generation of rule–diagram mappings via diagram highlighting supported better understanding of domain rules that manifested at delayed testing, even when students already interacted with problem diagrams (Experiment 3). This work extends the literature on learning technologies, generative processing, and desirable difficulties by demonstrating the potential of visually based interaction techniques implemented during problem solving to have long-term impact on the type of knowledge that students develop during intelligent tutoring.