Computers have transformed every facet of our culture, most dramatically communication, transportation, finance, science, and the economy. Yet their impact has not been generally felt in education due to lack of hardware, teacher training, and sophisticated software. Another reason is that current instructional software is neither truly responsive to student needs nor flexible enough to emulate teaching. The more instructional software can reason about its own teaching process, know what it is teaching, and which method to use for teaching, the greater is its impact on education. 

Building Intelligent Interactive Tutors discusses educational systems that assess a student's knowledge and are adaptive to a student's learning needs. Dr. Woolf taps into 20 years of research on intelligent tutors to bring designers and developers a broad range of issues and methods that produce the best intelligent learning environments possible, whether for classroom or life-long learning. The book describes multidisciplinary approaches to using computers for teaching, reports on research, development, and real-world experiences, and discusses intelligent tutors, web-based learning systems, adaptive learning systems, intelligent agents and intelligent multimedia.

*Combines both theory and practice to offer most in-depth and up-to-date treatment of intelligent tutoring systems available
*Presents powerful drivers of virtual teaching systems, including cognitive science, artificial intelligence, and the Internet
*Features algorithmic material that enables programmers and researchers to design building components and intelligent systems

Building Intelligent Interactive Tutors: Student-centered Strategies for Revolutionizing E-learning

The ASSISTments project is an ecosystem of a few hundred teachers, a platform, and researchers working together. Development professionals help train teachers and get teachers to participate in studies. The platform and these teachers help researchers (sometimes explicitly and sometimes implicitly) simply by using content the teacher selects. The platform, hosted by Worcester Polytechnic Institute, allows teachers to write individual ASSISTments (composed of questions with answers and associated hints, solutions, web-based videos, etc.) or to use pre-built ASSISTments, bundle them together in a problem set, and assign these to students. The system gives immediate feedback to students while they are working and provides student-level data to teachers on any assignment. The word “ASSISTments” blends tutoring “assistance” with “assessment” reporting to teachers and students. While originally focused on mathematics, the platform now has content from many other subjects (e.g., science, English, Statistics, etc.). Due to the large library of mathematics content, however, it is mostly used by math teachers. Over 50,000 students used ASSISTments last school year (2013–4) and this number has been doubling each year for the last 8 years. The platform allows any user, mostly researchers, to create randomized controlled trials in the content, which has helped us use the tool in over 18 published and an equal number of unpublished studies. The data collected by the system has also been used in a few dozen peer-reviewed data mining publications. This paper will not seek to review these publications, but instead we will share why ASSISTments has been successful and what lessons were learned along the way. The first lesson learned was to build a platform for learning sciences, not a product that focused on a math topic. That is, ASSISTments is a tool, not a curriculum. A second lesson learned is expressed by the mantra “Put the teacher in charge, not the computer.” This second lesson is about building a flexible system that allows teachers to use the tool in concert with the classroom routine. Once teachers are using the tool they are more likely to want to participate in research studies. These lessons were born from the design decisions about what the platform supports and does not support. In conclusion, goals for the future will be presented.

/pdf/the-assistments-ecosystem-building-a-platform-that-brings-wvmoktxzpt.pdf

The ASSISTments Ecosystem: Building a Platform that Brings Scientists and Teachers Together for Minimally Invasive Research on Human Learning and Teaching

This review describes a meta-analysis of findings from 50 controlled evaluations of intelligent computer tutoring systems The median effect of intelligent tutoring in the 50 evaluations was to raise test scores 066 standard deviations over conventional levels, or from the 50th to the 75th percentile However, the amount of improvement found in an evaluation depended to a great extent on whether improvement was measured on locally developed or standardized tests, suggesting that alignment of test and instructional objectives is a critical determinant of evaluation results The review also describes findings from two groups of evaluations that did not meet all of the selection requirements for the meta-analysis: six evaluations with nonconventional control groups and four with flawed implementations of intelligent tutoring systems Intelligent tutoring effects in these evaluations were small, suggesting that evaluation results are also affected by the nature of control treatments and the adequacy of progr

Effectiveness of Intelligent Tutoring Systems A Meta-Analytic Review

Modern information technologies increase the range of feedback strategies that can be implemented in computer based learning environments; however, the design and implementation of feedback strategies are very complex tasks that are often based more on intuition than on psychologically sound design principles. The purpose of this chapter is to present theoretically and empirically based guidelines for the design and evaluation of feedback strategies. To this end, this chapter describes an interactive, two-feedback-loop model that explains core factors and effects of feedback in interactive instruction (Narciss, 2006). Based on these theoretical considerations, a multidimensional view of designing and evaluating multiples feedback strategies under multiple individual and situational conditions is presented. This multidimensional view integrates recommendations of prior research on elab-KEYWORDS Cybernetics: System theory concerned with the issues of regulation, order, and stability confronting us in the treatment of complex systems and processes. Feedback: Output of a system that is fed back to the controller of the system as an input signal to regulate the system with regard to a reference value (cybernetic definition); post-response information that is provided to learners to inform them of their actual state of learning or performance (instructional context). Informative tutoring feedback: Multiple-try feedback strategies providing elaborated feedback components that guide the learner toward successful task completion without offering immediately the correct response. Interactive learning task: Tasks providing multiple response steps or tries and instructional components such as feedback, guiding questions, prompts, simulation facilities, and so on.

Feedback Strategies for Interactive Learning Tasks

Secondary teachers across the United States are being asked to use formative assessment data (Black and Wiliam 1998a,b; Roediger and Karpicke 2006) to inform their classroom instruction. At the same time, critics of US government's No Child Left Behind legislation are calling the bill "No Child Left Untested". Among other things, critics point out that every hour spent assessing students is an hour lost from instruction. But, does it have to be? What if we better integrated assessment into classroom instruction and allowed students to learn during the test? We developed an approach that provides immediate tutoring on practice assessment items that students cannot solve on their own. Our hypothesis is that we can achieve more accurate assessment by not only using data on whether students get test items right or wrong, but by also using data on the effort required for students to solve a test item with instructional assistance. We have integrated assistance and assessment in the ASSISTment system. The system helps teachers make better use of their time by offering instruction to students while providing a more detailed evaluation of student abilities to the teachers, which is impossible under current approaches. Our approach for assessing student math proficiency is to use data that our system collects through its interactions with students to estimate their performance on an end-of-year high stakes state test. Our results show that we can do a reliably better job predicting student end-of-year exam scores by leveraging the interaction data, and the model based on only the interaction information makes better predictions than the traditional assessment model that uses only information about correctness on the test items.

/pdf/addressing-the-assessment-challenge-with-an-online-system-3kwx8ap58f.pdf

Addressing the assessment challenge with an online system that tutors as it assesses

Intelligent tutoring systems are quite difficult and time inten- sive to develop. In this paper, we describe a method and set of software tools that ease the process of cognitive task analysis and tutor development by allowing the author to demonstrate, instead of programming, the behav- ior of an intelligent tutor. We focus on the subset of our tools that allow authors to create "Pseudo Tutors" that exhibit the behavior of intelligent tu- tors without requiring AI programming. Authors build user interfaces by di- rect manipulation and then use a Behavior Recorder tool to demonstrate al- ternative correct and incorrect actions. The resulting behavior graph is an- notated with instructional messages and knowledge labels. We present some preliminary evidence of the effectiveness of this approach, both in terms of reduced development time and learning outcome. Pseudo Tutors have now been built for economics, analytic logic, mathematics, and language learn- ing. Our data supports an estimate of about 25:1 ratio of development time to instruction time for Pseudo Tutors, which compares favorably to the 200:1 estimate for Intelligent Tutors, though we acknowledge and discuss limitations of such estimates.

/pdf/opening-the-door-to-non-programmers-authoring-intelligent-10mjwlcsn4.pdf

Opening the Door to Non-Programmers: Authoring Intelligent Tutor Behavior by Demonstration

This paper introduces sustainable design applications for small businesses through the Life Cycle Assessment and supply chain publishing platform Sourcemap.org. This web-based tool was developed through a year-long participatory design process with five small businesses in Scotland and in New England. Sourcemap was used as a diagnostic tool for carbon accounting, design and supply chain management. It offers a number of ways to market sustainable practices through embedded and printed visualizations. Our experiences confirm the potential of web sustainability tools and social media to expand the discourse and to negotiate the diverse goals inherent in social and environmental sustainability.

/pdf/small-business-applications-of-sourcemap-a-web-tool-for-2kpq4zh085.pdf

Small business applications of sourcemap: a web tool for sustainable design and supply chain transparency

An authoring tool supporting the creation, editing and use of a “Mobile Media Documentary” (MMD) is discussed. An MMD is an interactive tour of physical and virtual locations that is accompanied by multimedia content mapped to spatial data for a known path. The multimedia content includes character-driven audio narrative supplemented by graphics, text, video and interactive content.

System and method for using known path data in delivering enhanced multimedia content to mobile devices

We introduce a technique for exploring multi-layered images by scraping arbitrary areas to determine meaningful relationships. Our system, called Wetpaint, uses perceptual depth cues to help users intuitively navigate between corresponding layers of an image, allowing a rapid assessment of changes and relationships between different views of the same area. Inspired by art diagnostic techniques, this tactile method could have distinct advantages in the general domain as shown by our user study. We propose that the physical metaphor of scraping facilitates the process of determining correlations between layers of an image because it compresses the process of planning, comparison and annotation into a single gesture. We discuss applications for geography, design, and medicine.

/pdf/wetpaint-scraping-through-multi-layered-images-3glub93z13.pdf

Wetpaint: scraping through multi-layered images

Intelligent tutoring systems are quite difficult and time intensive to develop. In this paper, we describe a method and set of software tools that ease the process of cognitive task analysis and tutor development by allowing the author to demonstrate, instead of programming, the behavior of an intelligent tutor. We focus on the subset of our tools that allow authors to create Pseudo Tutors that exhibit the behavior of intelligent tutors without requiring AI programming. Authors build user interfaces by direct manipulation and then use a Behavior Recorder tool to demonstrate alternative correct and incorrect actions. The resulting behavior graph is annotated with instructional messages and knowledge labels. We present some preliminary evidence of the effectiveness of this approach, both in terms of reduced development time and learning outcome. Pseudo Tutors have now been built for economics, analytic logic, mathematics, and language learning. Our data supports an estimate of about 25:1 ratio of development time to instruction time for Pseudo Tutors, which compares favorably to the 200:1 estimate for Intelligent Tutors, though we acknowledge and discuss limitations of such estimates.

Matthew Hockenberry

Papers

Opening the Door to Non-Programmers: Authoring Intelligent Tutor Behavior by Demonstration

Small business applications of sourcemap: a web tool for sustainable design and supply chain transparency

System and method for using known path data in delivering enhanced multimedia content to mobile devices

Wetpaint: scraping through multi-layered images

Opening the door to non-programmers: Authoring Intelligent tutor behavior by demonstration