This article is a review of experiments comparing the effectiveness of human tutoring, computer tutoring, and no tutoring. “No tutoring” refers to instruction that teaches the same content without tutoring. The computer tutoring systems were divided by their granularity of the user interface interaction into answer-based, step-based, and substep-based tutoring systems. Most intelligent tutoring systems have step-based or substep-based granularities of interaction, whereas most other tutoring systems (often called CAI, CBT, or CAL systems) have answer-based user interfaces. It is widely believed as the granularity of tutoring decreases, the effectiveness increases. In particular, when compared to No tutoring, the effect sizes of answer-based tutoring systems, intelligent tutoring systems, and adult human tutors are believed to be d = 0.3, 1.0, and 2.0 respectively. This review did not confirm these beliefs. Instead, it found that the effect size of human tutoring was much lower: d = 0.79. Moreover, the eff...

The Relative Effectiveness of Human Tutoring, Intelligent Tutoring Systems, and Other Tutoring Systems

Computers have transformed every facet of our culture, most dramatically communication, transportation, finance, science, and the economy. Yet their impact has not been generally felt in education due to lack of hardware, teacher training, and sophisticated software. Another reason is that current instructional software is neither truly responsive to student needs nor flexible enough to emulate teaching. The more instructional software can reason about its own teaching process, know what it is teaching, and which method to use for teaching, the greater is its impact on education. 

Building Intelligent Interactive Tutors discusses educational systems that assess a student's knowledge and are adaptive to a student's learning needs. Dr. Woolf taps into 20 years of research on intelligent tutors to bring designers and developers a broad range of issues and methods that produce the best intelligent learning environments possible, whether for classroom or life-long learning. The book describes multidisciplinary approaches to using computers for teaching, reports on research, development, and real-world experiences, and discusses intelligent tutors, web-based learning systems, adaptive learning systems, intelligent agents and intelligent multimedia.

*Combines both theory and practice to offer most in-depth and up-to-date treatment of intelligent tutoring systems available
*Presents powerful drivers of virtual teaching systems, including cognitive science, artificial intelligence, and the Internet
*Features algorithmic material that enables programmers and researchers to design building components and intelligent systems

Building Intelligent Interactive Tutors: Student-centered Strategies for Revolutionizing E-learning

Introduction                                  434Concepts                                     434Functions: Supporting System Use                435Taking Over Parts of Routine Tasks                435Adapting the Interface                          436Helping With System Use                        437Mediating Interaction with the Real World          438Controlling a Dialogue                          439Functions: Supporting InformationAcquisition                                    440Helping Users to Find Information                440Support for browsing                        441Support for query-based search or filtering       442Spontaneous Provision of Information             442Recommending Products                        442Tailoring Information Presentation                443Supporting Collaboration                       444Supporting Learning                           445Usability Challenges                            446Threats to Predictability and Comprehensibility      447Threats to Controllability                        447Obtrusiveness                                 448Threats to Privacy                              448Breadth of Experience                          448Dealing With Trade-offs                         448Obtaining Information About Users               449Explicit Self-Reports and -Assessments              449Self-reports about objective personalcharacteristics                              449Self-assessments of interests and knowledge     449Self-reports on specific evaluations             450Responses to test items                      450Nonexplicit Input                              450Naturally Occurring Actions                      450Previously Stored Information                    450Low-Level Indices of Psychological States           451Signals Concerning the Current Surroundings       451Special Considerations ConcerningEmpirical Methods                             451Use of Data Collected With a Nonadaptive System     451Early Studies of Usage Scenarios andUser Requirements                             452Wizard of Oz Studies                           452Comparisons With the Work of Human Designers     453Experimental Comparisons of Adaptiveand Nonadaptive Systems                        453Taking Into Account Individual Differences          454Checking Usability Under Realistic Conditions       454The Future of User-Adaptive Systems              454Growing Need for User-Adaptivity                 454Diversity of users and contexts of use           454Number and complexity of interactive systems    454Scope of information to be dealt with           454Increasing Feasibility of Successful Adaptation       455Ways of acquiring information about users       455Advances in techniques for learning,inference, and decision                      455Attention to empirical methods                455Acknowledgments                              455References                                    455This chapter covers a broad range of interactive systems They
all have one idea in common: It can be worthwhile for a system
to learn something about each individual user and adapt its behavior to them in some nontrivial way

/pdf/adaptive-interfaces-and-agents-3siw53txj4.pdf

Adaptive interfaces and agents

This review describes a meta-analysis of findings from 50 controlled evaluations of intelligent computer tutoring systems The median effect of intelligent tutoring in the 50 evaluations was to raise test scores 066 standard deviations over conventional levels, or from the 50th to the 75th percentile However, the amount of improvement found in an evaluation depended to a great extent on whether improvement was measured on locally developed or standardized tests, suggesting that alignment of test and instructional objectives is a critical determinant of evaluation results The review also describes findings from two groups of evaluations that did not meet all of the selection requirements for the meta-analysis: six evaluations with nonconventional control groups and four with flawed implementations of intelligent tutoring systems Intelligent tutoring effects in these evaluations were small, suggesting that evaluation results are also affected by the nature of control treatments and the adequacy of progr

Effectiveness of Intelligent Tutoring Systems A Meta-Analytic Review

As computing becomes a mainstream discipline embedded in the school curriculum and acts as an enabler for an increasing range of academic disciplines in higher education, the literature on introductory programming is growing. Although there have been several reviews that focus on specific aspects of introductory programming, there has been no broad overview of the literature exploring recent trends across the breadth of introductory programming. This paper is the report of an ITiCSE working group that conducted a systematic review in order to gain an overview of the introductory programming literature. Partitioning the literature into papers addressing the student, teaching, the curriculum, and assessment, we explore trends, highlight advances in knowledge over the past 15 years, and indicate possible directions for future research.

/pdf/introductory-programming-a-systematic-literature-review-3yllvt7k18.pdf

Introductory programming: a systematic literature review

E-learning is becoming more and more popular with the widespread use of computers and the Internet in educational institutions. Current e-learning courses are nearly always developed using course management systems (CMS), such as WebCT or Blackboard. Although CMS tools provide support for some administrative tasks and enable instructors to provide online instructional material, they offer no deep support for learning: students have access to on-line material, simple multi-choice quizzes and chat tools, but there is no ability to track student’s progress and adapt the learning material and instructional session to the individual student. In this paper we present our experiences with three Web-based intelligent tutoring systems in the area of databases. SQL-Tutor teaches the SQL query language, NORMIT is a data normalization tutor, and KERMIT teaches conceptual database modelling using the Entity-Relationship data model. All three tutors in DB-suite have been used and evaluated in the context of genuine teaching activities. We present the most important features of these systems, as well as evaluation results. The DB-suite tutors have proved to be very effective in supporting deep learning, and are well liked by students.

/pdf/db-suite-experiences-with-three-intelligent-web-based-3n7h8s638j.pdf

DB-Suite: Experiences with Three Intelligent, Web-Based Database Tutors.

There is a wealth of literature dealing with the difficulties of novice programmers with basic programming constructs such as variables, assignment and conditionals. In this paper we extend the study to two other core CS1 topics: loops and vectors (represented as single dimensional arrays). By the end of their first semester of instruction, students are expected to have acquired both the ability to reproduce given syntactic structure and basic design skills that allow them to write small pieces of code that extend, modify or combine in new ways the basic programming constructs. This work presents an evaluation framework that uses the SOLO taxonomy to assess programming questions' complexity. Our framework extends SOLO by using the term "building block" as an adaptable parameter that explicitly defines the student's ability to increasingly write more complex pieces of code. The granularity of a "building block" is determined by the amount of programming practice students have carried out up to that point. The analysis of final exam answers using this framework allows us to quantify the progress made by one cohort of novice programmers in the mastery of basic design skills and to study correlations between mastery of these skills and overall course performance. Furthermore, we identify common errors that illustrate the challenges students face when trying to combine programming constructs in non-trivial ways.

A Study of Code Design Skills in Novice Programmers using the SOLO taxonomy

Safeguarding academic integrity is an issue of concern to all computing academics due to high and rising levels of plagiarism and other cheating in computing courses. There have been many studies of the cheating and plagiarism practices of computing students and the factors that can influence these practices, and a variety of strategies for reducing cheating have been proposed. This national study of first-year computing programs provides insights into what strategies computing academics use to discourage or prevent their students from cheating. Having interviewed 30 academics from 25 universities we found 21 different types of strategy, which we classified into five themes: education; discouraging cheating; making cheating difficult; and empowerment. We also found that academics often employ strategies across all of these themes.

Strategies for Maintaining Academic Integrity in First-Year Computing Courses

ITSs for ill-defined domains have attracted a lot of attention recently, which is well-deserved, as such ITSs are hard to develop. The first step towards such ITSs is reaching a wide agreement about the terminology used in the area. In this paper, we discuss the two important dimensions of ill-definedness: the domain and the instructional task. By the domain we assume declarative domain knowledge, or the domain theory, while the instructional task is the task the student is learning, in terms of problem-solving skills. It is possible to have a well-defined domain and still have ill-defined instructional tasks in the same domain. We look deeper at the features of ill-defined tasks, which all contribute to their ill/well defined nature. The paper discusses model-tracing and constraint-based modeling, in terms of their suitability for ill-defined tasks and domains. We show that constraint-based modeling can be used in both well-and illdefined domains, and illustrate our conclusion using several instructional tasks.

/pdf/revisiting-ill-definedness-and-the-consequences-for-itss-1h0qhbwxwv.pdf

Revisiting Ill-Definedness and the Consequences for ITSs

Self-explanation has been used successfully in teaching Mathematics and Physics to facilitate deep learning. We are interested in investigating whether self-explanation can be used in an open-ended, ill-structured domain. For this purpose, we enhanced KERMIT, an intelligent tutoring system that teaches conceptual database design. The resulting system, KERMIT-SE, supports self-explanation by engaging students in tutorial dialogues when their solutions are erroneous. The results of an evaluation study indicate that self-explanation leads to improved performance in both conceptual and procedural knowledge.

Amali Weerasinghe

Papers

DB-Suite: Experiences with Three Intelligent, Web-Based Database Tutors.

A Study of Code Design Skills in Novice Programmers using the SOLO taxonomy

Strategies for Maintaining Academic Integrity in First-Year Computing Courses

Revisiting Ill-Definedness and the Consequences for ITSs

Facilitating deep learning through self-explanation in an open-ended domain