Showing papers by "Peter Brusilovsky published in 2005"

Individualized exercises for self-assessment of programming knowledge: An evaluation of QuizPACK

Abstract: Individualized exercises are a promising feature in promoting modern e-learning. The focus of this article is on the QuizPACK system, which is able to generate parameterized exercises for the C language and automatically evaluate the correctness of student answers. We introduce QuizPACK and present the results of its comprehensive classroom evaluation during four consecutive semesters. Our studies demonstrate that when QuizPACK is used for out-of-class self-assessment, it is an exceptional learning tool. The students' work with QuizPACK significantly improved their knowledge of semantics and positively affected higher-level knowledge and skills. The students themselves praised the system highly as a learning tool. We also demonstrated that the use of the system in self-assessment mode can be significantly increased by basing later classroom paper-and-pencil quizzes on QuizPACK questions, motivating students to practice them more.

User modeling in a distributed e-learning architecture

Abstract: This paper is focused on user modeling and adaptation in distributed E-Learning systems. We describe here CUMULATE, a generic student modeling server developed for a distributed E-Learning architecture, KnowledgeTree. We also introduce a specific, topic-based knowledge modeling approach which has been implemented as an inference agent in CUMULATE and used in QuizGuide, an adaptive system that helps students select the most relevant self-assessment quizzes. We also discuss our attempts to evaluate this multi-level student modeling.

Ontology-based Framework for User Model Interoperability in Distributed Learning Environments

Abstract: This paper presents an update of our work on developing a distributed architecture for adaptive E-Learning. We introduce ADAPT – an extension of KnowledgeTree architecture that was presented at one of the past E-Learn conferences. To accommodate a larger variety of adaptive components, ADAPT employs a higher-level mechanism for ontology-based interoperability of self-contained adaptive Web-based systems within our architecture. We provide an overview of ADAPT, argue for the importance of ontology-based interoperability, and describe our implementation of this functionality in ADAPT.

Engaging students to work with self-assessment questions: a study of two approaches

Abstract: We explored two approaches for encouraging introductory programming students to use the web-based, self-assessment system, QuizPACK. An "organizational" approach applied specially constructed classroom quizzes, while the "technical" approach introduced adaptive guidance. Our studies demonstrated that each of these caused a dramatic increase in system use. This approach could be useful in many other contexts, when an educationally beneficial system is underused by students.

Social navigation support through annotation-based group modeling

Abstract: Closed corpus AH systems demonstrate what is possible to achieve with adaptive hypermedia technologies. However, they are impractical for dealing with the large volume of open corpus resources. Our Knowledge Sea project explores social navigation support, an approach for providing open corpus personalized guidance that is based on past learners' interaction with the system. The most recent stage of our project focuses on using annotations for social navigation support. We present here Knowledge Sea II, which implements annotation-based social navigation support, and report the results of several classroom studies, which have evaluated this technology.

Comprehensive personalized information access in an educational digital library

Abstract: This paper explores two ways to help students locate most relevant resources in educational digital libraries. One method gives a more comprehensive access to educational resources, through multiple pathways of information access, including browsing and information visualization. The second method is to access personalized information through social navigation support. This paper presents the details of the Knowledge Sea II system for comprehensive personalized access to educational resources and also presents the results of a classroom study. The study delivered a convincing argument for the importance of providing multiple information presentations modes, showing that only about 10% of all resource accesses were made through the traditional search interface. We have also collected some solid evidence in favor of the social navigation support.

Towards user modeling meta-ontology

Abstract: The paper proposes meta-ontology of the user modeling field. Ontology is meant to structure the state-of-the-art in the field and serve as a central reference point and as a tool to index systems, papers and learning media. Such ontology is beneficial for both the user modeling research community and the students as it creates a shared conceptualization of the known approaches to building user models and their implementations.

NavEx: Providing Navigation Support for Adaptive Browsing of Annotated Code Examples

Abstract: This paper presents NavEx, an adaptive environment for accessing interactive programming examples. NavEx implements a specific kind of adaptive navigation support known as adaptive annotation. The classroom study of NavEx confirmed that adaptive navigation support can visibly increase student motivation to work with non-mandatory educational content. NavEx boosted the overall amount of work and the average length of a session. In addition, various features of NavEx were highly regarded by the students.

Preface to the special issue on automated assessment of programming assignments

Abstract: Programming computers is becoming an increasingly popular activity, not only for computer science students but across a large number of other disciplines. But funding and other resources for teaching programming have not, for the most part, kept pace with the continuous growth in the number of students studying programming. Academic institutions face the challenge of providing their students with better quality teaching while minimizing the amount of additional work for staff. While traditional teaching methods can be enhanced by audio/visual means and advances in online learning, in the area of assessment the problems continue to persist. Computer-based assessment (CBA), which over the years has become an increasingly important teaching tool, can help educators solve these problems. For the past twenty years, educators have reported on the practical and pedagogic benefits of using automated assessment tools to assess student coursework in programming. The purpose of this issue of JERIC is to explain cutting-edge research on the automated assessment of student programs, and other kinds of programming assignments, to a wider audience. It is devoted to both fully automated assessment and to partial student program assessment by machine. We do not provide an overview of the field in this introductory statement. A recent survey by Ala-Mutka [Ala-Mutka 2005], the review of Douce et al. in this issue [Douce et al. 2006], and several other articles in this issue provide a comprehensive overview and offer various ways to classify the multitude of known work. However, to introduce the articles in this issue, the editors found it useful to distinguish three categories of systems: (1) to assess program-tracing skills; (2) to assess program-writing skills; and (3) assess intelligent programming tutors. The first group of systems attempts to assess students’ knowledge of programming language semantics by presenting students with a program and asking them to trace it. The answer to this type of problem results from its execution: What was printed? What was the final state of the variables in the data structures? The ability to automatically evaluate student answers stems from the system’s ability to execute the program or the algorithm with the same data and compare that result with the one entered by the student. The QuizPACK system reported by Brusilovsky and Sosnovsky [2006] in this issue _________________________________________________________________________________________

World-Wide Intelligent Textbooks.

Abstract: New WWW technologies allow for integrating distance education power of WWW with interactivity and intelligence. Integrating on-line presentation of learning materials with the interactivity of problem solving environments and the intelligence of intelligent tutoring systems results in a new quality of learning materials that we call I3-textbooks. In this paper, we describe the development of ELM-ART, an I3-textbook for learning programming that can be accessed via Internet and that is based on the on-site learning environment ELM-PE.

One Practical Algorithm of Creating Teaching Ontologies

Abstract: The paper presents one practical approach aimed at developing teaching ontologies. The underlying research framework is pursuing a methodology that will scaffold the process of knowledge structuring and ontology design. Moreover, special stress should be placed on visual design as a powerful learning mind tool. For more comprehensible understanding the process of developing a practical ontology from the domain of introductory C programming is described.

Interactive Authoring Support for Adaptive Educational Systems

Abstract: A well-known challenge of adaptive educational systems is the need to develop intelligent content, which is very time and expertise consuming. In traditional approaches a teacher is kept at a distance from intelligent authoring. This paper advocates the involvement of teachers in creating intelligent content. We are presenting an approach to the development of intelligent content as well as an authoring tool for teachers that support our approach. This approach has two main stages: elicitation of concepts from content elements and the identification of a prerequisite/outcome structure for the course. The resulting sequence of adaptive activities reflects the author's view of the course's organization. The developed tool facilitates concept elicitation in two ways: it provides an author with an automatic indexing component and also allows her/him to edit the index using the domain ontology as an authoring map.

Development of XML-based tools to support user interaction with algorithm visualization

Abstract: As a report of a working group at ITiCSE 2005, this paper represents a vision of the use of XML specifications and tools in algorithm visualization, particularly with regard to supporting user interaction. A detailed description is given of how an interesting event to be visualized is decomposed, combined with interactive questions, narratives, control flow code and metadata, and finally rendered into graphical primitive and transformation specifications. The heart of the paper is our discussion of XML specifications for content generation (the object being visualized), interactive questions, and graphical primitives and transformations, with briefer discussions of narratives and metadata. Examples are provided for each in an appendix, with fuller details to be published on an associated website that we hope will become a source of future standards in this area. In conclusion, the approach of the working group is discussed, and important remaining challenges are identified.

Investigating Users' Needs and Behavior for Social Search

Abstract: Traditional Web search systems have limitations due to its unrealistic assumption on users' query formulation and lack of context-sensitivity. To overcome these limitations, we designed and implemented a social search sys- tem which is based on a social adaptive navigation system Knowledge Sea and exploits the past usage history of users. By conducting a survey and transaction log analysis, we could observe users' strong attitudes to the need for the social search capability. We could also observe their active use of the new feature and change of behavior while they were using the search system. At the initial stage of our experiment, users did not show big difference in their usage of the system compared to the conventional search services but as time passes and the usage history accumulates, meaningful changes in their behavior toward the use of so- cial navigation support features of the system were discovered.

High-level translation of adaptive hypermedia applications

Abstract: In the early years of the adaptive hypermedia research a large number of special-purpose adaptive hypermedia systems (AHS) have been developed, to illustrate research ideas, or to serve a single application. Many of these systems are now obsolete. In this paper we propose to bring new life to these applications by means of translation to a general purpose adaptive hypermedia architecture. We illustrate that this approach can work by showing a high-level translation from InterBook [2] to AHA! [5]. Such a translation consists of three parts: the structure of concepts and concept relationships needs to be translated, the adaptive behavior for these concept relationships must be defined, and the layout and presentation of the source application must be "simulated". Our high-level translation covers all three parts.

Authoring and delivery of adaptive electronic textbooks made easy

Abstract: The vast majority of textbooks (even when offered on-line) are still traditional book-like static documents with a fixed structure and content. Authoring a textbook in a "simple" environment like Microsoft Word is much easier than using special authoring environments for adaptive electronic textbooks. In this paper we show how to translate a Word-based textbook into an adaptive website that provides adaptive navigation support (through link annotation) as well as adaptive presentation (through conditionally included explanations). The generated adaptive textbook is used through the powerful and highly customizable AHA! adaptive delivery platform. The essence (and novelty) of the presented approach is that the translation from Word (using an extended set of InterBook annotations) to AHA! is performed at the conceptual level. The further translation into (low level rules defining the) adaptive behavior is completely independent from the conceptual structure and content of the textbook (and can also be customized).

Adaptive Information Visualization for Personalized Access to Educational Digital Libraries

Abstract: Personalization is one of the emerging ways to increase the power of modern Digital Libraries. The Knowledge Sea II system presented in this paper explores social navigation support, an approach for providing personalized guidance within the open corpus of educational resources. Following the concepts of social navigation we have attempted to organize a personalized navigation support that is based on past learners’ interaction with the system. The study indicates that Knowledge Sea II became the students' primary tool for accessing the open corpus documents used in a programming course. The social navigation support implemented in this system was considered useful by students participating in the study of Knowledge Sea II. At the same time, some user comments indicated the need to provide more powerful navigational support, such as the ability to rank the usefulness of a page.

Collaborative Paper Exchange

Abstract: In many graduate and undergraduate courses, instructors attempt to go “beyond the textbook” by assigning research papers for the students to read and briefly summarize, in order to bring the students a more complete and state-of-the-art understanding of their field. Typically these summaries are not used after being graded. We claim that these summaries are an unexplored treasure of information that can provide an important connection between the core course content found in a textbook and more recent trends that can only be distilled from research literature. To provide all students access to this accumulated treasure of information, we have developed Collaborative Paper Exchange (CoPE) – a system that allows the student to submit their assigned summaries and browse those submitted by others. This paper presents the system and summarizes the results of the first classroom study of this system.

Towards user modeling meta-ontology

Abstract: The paper proposes meta-ontology of the user modeling field. Ontology is meant to structure the state-of-the-art in the field and serve as a central reference point and as a tool to index systems, papers and learning media. Such ontology is beneficial for both the user modeling research community and the students as it creates a shared conceptualization of the known approaches to building user models and their implementations.

User modeling in a distributed E-Learning architecture

Abstract: This paper is focused on user modeling and adaptation in distributed E-Learning systems. We describe here CUMULATE, a generic student modeling server developed for a distributed E-Learning architecture, KnowledgeTree. We also introduce a specific, topic-based knowledge modeling approach which has been implemented as an inference agent in CUMULATE and used in QuizGuide, an adaptive system that helps students select the most relevant self-assessment quizzes. We also discuss our attempts to evaluate this multi-level student modeling.