Showing papers by "Peter Brusilovsky published in 2009"

Collaborative filtering for social tagging systems: an experiment with CiteULike

Abstract: Collaborative tagging systems pose new challenges to the developers of recommender systems. As observed by recent research, traditional implementations of classic recommender approaches, such as collaborative filtering, are not working well in this new context. To address these challenges, a number of research groups worldwide work on adapting these approaches to the specific nature of collaborative tagging systems. In joining this stream of research, we have developed and compared three variants of user-based collaborative filtering algorithms to provide recommendations of articles on CiteULike. The first approach, Classic Collaborative filtering (CCF) uses Pearson correlation to calculate similarity between users and a classic adjusted ratings formula to rank the recommendations. The second approach, Neighbor-weighted Collaborative Filtering, takes into account the number of raters in the ranking formula of the recommendations. The third approach explores an innovative way to form the user neighborhood based on a modified version of the Okapi BM25 model over users' tags. Our results suggest that both alterations of CCF are beneficial. Incorporating the number of raters into the algorithms leads to an improvement of precision, while tag-based BM25 can be considered as an alternative to Pearson correlation to calculate the similarity between users and their neighbors.

Addictive links: the motivational value of adaptive link annotation

Abstract: Adaptive link annotation is a popular adaptive navigation support technology. Empirical studies of adaptive annotation in the educational context have demonstrated that it can help students to acquire knowledge faster, improve learning outcomes, reduce navigational overhead, and encourage non-sequential navigation. In this paper, we present our exploration of a lesser known effect of adaptive annotation, its ability to significantly increase students' motivation to work with non-mandatory educational content. We explored this effect and confirmed its significance in the context of two different adaptive hypermedia systems. The paper presents and discusses the results of our work.

Reinforcing Recommendation Using Implicit Negative Feedback

Abstract: Recommender systems have explored a range of implicit feedback approaches to capture users' current interests and preferences without intervention of users' work. However, current research focuses mostly on implicit positive feedback. Implicit negative feedback is still a challenge because users mainly target information they want. There have been few studies assessing the value of negative implicit feedback. In this paper, we explore a specific approach to employ implicit negative feedback and assess whether it can be used to improve recommendation quality.

Does trust influence information similarity

Abstract: In collaborative filtering recommender systems, users cannot get involved in the choice of their peer group. It leaves users defenseless against various spamming or "shilling" attacks. Other social Web-based systems, however, allow users to self-select trustworthy peers and build a network of trust. We argue that users self-defined networks of trust could be valuable to increase the quality of recommendation in CF systems. To prove the feasibility of this idea we examined how similar are interests of users connected by a self-defined relationship in a social Web system, CiteuLike. Interest similarity was measured by similarity of items and meta-data they share. Our study shows that users connected by a network of trust exhibit significantly higher similarity on items and meta-data than non-connected users. This similarity is highest for directly connected users and decreases with the increase of distance between users.

Adaptive Navigation Support for Parameterized Questions in Object-Oriented Programming

Abstract: This paper explores the impact of adaptive navigation support on student work with parameterized questions in the domain of object-oriented programming. In the past, we developed QuizJET system, which is able to generate and assess parameterized Java programming questions. More recently, we developed JavaGuide system, which enhances QuizJET questions with adaptive navigation support. This system introduces QuizJET and JavaGuide and reports the results of classroom studies, which explored the impact of these systems and assessed an added value of adaptive navigation support. The results of the studies indicate that adaptive navigation support encourages students use parameterized questions more extensively. Students are also 2.5 times more likely to answer parameterized questions correctly with adaptive navigation support than without such support. In addition, we found that adaptive navigation support especially benefit weaker students helping to close the gap between strong and weak students.

Personal Services: Debating the Wisdom of Personalisation

Abstract: Personalisation has been seen as the answer to the vast quantity of information now available digitally. Information is tailored to meet each user's apparent needs, whether making use of Web-based learning materials, purchasing from online sites, accessing entertainment products or any other potentially-personalisable facility. Yet before rushing headlong into personalising everything, including online learning systems, we need to consider the threats posed by the capture of so much data about individuals, the implications of personalised information delivery, and whether the benefits of personalisation justify the costs. In this paper, we look at both the issues and benefits of personalisation, and consider how to steer our way to a safe and valuable use of personalisation.

Semantic Integration of Adaptive Educational Systems

Abstract: With the growth of adaptive educational systems available to students, integration of these systems is evolving from an interesting research problem into an important practical task. One of the challenges that needs to be addressed is the development of mechanisms for student model integration. The architectural principles and representation technologies employed by adaptive educational systems define the applicability of a particular integration approach. This chapter reviews the existing mechanisms and details one of them: the evidence integration.

Social Navigation Support for Information Seeking: If You Build It, Will They Come?

Abstract: Navigating through the ever-changing information space is becoming increasingly difficult. Social navigation support is a technique for guiding users to interesting and relevant information by leveraging the browsing behavior of past users. Effect of social navigation support on users' information seeking behavior has been studied mostly from conceptual basis or under natural experiments. In the current work, we have designed and conducted a controlled experiment to investigate the effect of social navigation support through a multifaceted method. This paper reports on the design of the study and the result of log data, subjective evaluation, and eye movement data analysis.

Ensemble: creating a national digital library for computing education

Abstract: Ensemble is a new NSF NSDL Pathways project working to establish a national, distributed digital library for computing education. Our project is building a distributed portal providing access to a broad range of existing educational resources for computing while preserving the collections and their associated curation processes. We want to encourage contribution, use, reuse, review and evaluation of educational materials at multiple levels of granularity and we seek to support the full range of computing education communities including computer science, computer engineering, software engineering, information science, information systems and information technology as well as other areas often called "computing + X" or "X informatics".

Problem Solving Examples as First Class Objects in Educational Digital Libraries: Three Obstacles to Overcome

Abstract: The paper analyzes three major problems encountered by our team as we endeavored to turn problem solving examples in the domain of programming into highly reusable educational activities, which could be included as learning objects in various educational digital libraries. It also suggests three specific approaches to resolving these problems, and reports on the evaluation of the suggested approaches. Our successful experience, presented in this paper, demonstrates how to make program examples selfsufficient, guide individual students to the most appropriate examples, and increase the volume of annotated examples.

CiteAware: Visual Group Awareness for a Reference Sharing System

Abstract: Publication sharing portals, such as CiteULike and BibSonomy are very popular among research community. Users create interest groups, participate in existi ng groups, share papers and tag contents. With increasing numb er of groups, members, contributions and tags, it is very difficult to keep track of all the group activities and extra ct useful knowledge out of user contributions. In this paper, we present our ongoing work on CiteAware ‐ a visual group awareness system for CiteULike. Our system collects most recent activity timestamps of a user's public groups or a group's participating members , transforming them into an easily perceivable holistic visualization. A preliminary user study results are discussed here.

Extending parameterized problem-tracing questions for Java with personalized guidance

Abstract: Problem-tracing questions are popular among teachers of various programming languages. In an assessment mode these questions allows to evaluate student knowledge of language semantics. In a self-assessment mode, they provide an excellent learning tool. A 2004 ITiCSE working group report [4] stressed the importance of this type of questions to build foundation of higher-level knowledge. Yet the use of problem-tracing questions is still limited due to a large authoring overhead. To resolve this bottleneck, we explored the idea of parameterized question generation [2]. We developed QuizPACK [1], a system which can generate parameterized problem-tracing questions for C programming language. We also developed QuizGuide [1], a personalized guidance system for QuizPACK, which models student knowledge and guides students individually to most appropriate questions to try. The results of our studies demonstrated that QuizPACK strongly benefits student knowledge and that QuizGuide personalized guidance technology increased student ability to answer questions correctly and encouraged them to use the system more extensively (which, in turn, positively impacted their knowledge) [1]. However, parameterized questions in area of C programming were not as diverse from the complexity point of view as parameterized questions explored in other areas such as physics [2]. As a result, it was left unclear whether personalized guidance technology can successfully guide students to a broader range of questions from relatively simple to very difficult.The work reported in this poster expands our work on parameterized questions to a more sophisticated domain of object-oriented Java programming, which allowed us to introduce questions of much broader. Capitalizing on our experience with QuizPACK, we developed QuizJET (Java Evaluation Toolkit), which supports authoring, delivery, and evaluation of parameterized questions for Java [3]. We also implemented JavaGuide system (Figure 1), which provides personalized guidance for QuizJET questions. We assessed the impact of adaptive navigation support to student work with questions of different complexity as well as the impact of this technology on weaker and stronger students. The results of two classroom studies indicate that personalized guidance encouraged students to use parameterized questions more extensively and also helped them to access right questions at the right time. Students were 2.5 times more likely to answer a quiz correctly with personalized guidance than without such it. In addition, we found that personalized guidance especially benefited weak students to achieve scores comparable with the scores of strong students on each complexity level of questions.

Presentation Session 2: Enhancing Electronic Books with Spatial Annotation and Social Navigation Support

Abstract: Recent projects such as Carnegie Mellon Million Books started the process of rapid digitization of traditional books. With other industrial and academic efforts following this lead the main research challenge for both researchers and practitioner is moving from “how we can scan a million books” to “what we can do” with the newly available “million books”. This paper presents a specific approach, which can extend enriching the experience and productivity of users of digital books by giving them a combination of annotation and social navigation support. This approach was implemented evaluated in several classroom studies. 1. Motivation Recent projects such as Carnegie Mellon Million Books (Reddy & StClair, 2001) started the process of rapid digitization of traditional books. With other industrial and academic efforts following this lead the main research challenge for both researchers and practitioner is moving from “how we can scan a million books” to “what we can do” with the newly available “million books” (Crane, 2006). What is the value of having all these books in digital form? Is it just 24/7 anytime, anywhere access? Along with other researchers focusing on enriching the experience and productivity of users of digital books, we argue that the availability of many electronic books in digital formal can offer unique opportunities to their users. Our own work on enhancing the experience of electronic book users focuses on supporting the users of electronic books with shared annotation and social navigation functionalities. In our earlier work on Knowledge Sea II (Farzan & Brusilovsky, 2005) and AnnotatED (Farzan & Brusilovsky, 2008) projects we explored the use of these technologies with HTML-based electronic textbooks. The new version of our annotation technology presented in this paper was designed to support the users of fully spatial electronic books – similar to those which are being scanned by the Million Books project. We developed an approach which combines source annotation and social navigation in the context of authentically spatial electronic books. The feasibility of this approach was explored in several classroom studies where a collection of textbook on the subject played the role of a small digital library. This paper provides a brief review of Web annotation tool, demonstrate its application for social navigation, and discuss the results of its evaluation. More technical details can be found in (Kim, Farzan & Brusilovsky, 2008). Figure 1: Viewing a selected annotation 2. Spatial Annotation for Scanned Textbooks Our spatial annotation interface allows the user to create a new annotation by clicking and dragging the mouse to create a rectangle of the desired size and, if desired, adding the first comment in this location. The interface also offers students the ability to make their annotations public or private as regards visibility of the annotation, and to choose type of note, such as praise or general, and authorship. We added these options to motivate students to share their feedback with their classmates. We anticipate two major ways to use annotations: for user own needs (active reading) and to communicate with other users. The interface provides three visibility control buttons all, me, and off, located at the top-right of each page (Figure 1). The “off” button hides all annotations, the “me” button shows only user’s own annotation, and “all” shows also all public annotation made by other users on this page. As shown in Figure 1, when the mouse cursor hovers over an annotated area, it highlights the corresponding annotation to show students which annotation is being currently selected and displays a small balloon containing other information, such as note text and how many notes belong to it. 3. Annotation-based Social Navigation Support The goal of annotation-based social navigation support (SNS) is to guide users to most interesting and popular pages and fragments. Fragment guidance is provided on a page level by changing border and background style of annotations (Figure 2). A filled background means that the annotation or a comment associated with the annotation was created by the current user, while no background shows that the annotations were all created by others. In the case of personal annotations, light orange represents general type annotations while light green represents praise type annotations Border style represents annotations by other users (not the current user). The border is orange if all the comments associated with the notes are general and green if at least one comment is praise. The thickness of the border represents the number of comments associated with the annotation. The border gets thicker as the number of comments associated with it grows. Figure 2: Visualization Properties of Annotations Page guidance is provided by adaptive link annotation. A good example of its implementation is provided by Knowledge Sea II (Farzan & Brusilovsky, 2005) system, which we used in our courses to integrate several textbooks on the subject. The navigation is facilitated through trafficand annotation-based SNS. Traffic-based SNS guides the students to resources which attracted a higher number of visits. Annotation-based SNS guides students to resources which attracted personal or group annotations. Access to resources in KnowledgeSea II is done through three levels: map, cell, and page. SNS is provided at all levels. Figure 3 shows the three levels of KnowledgeSea II. The map is an 8 by 8 table in which every cell includes links to online tutorials. A set of keywords represents the content of the material linked to the cell. The adjacent cells present similar materials. Traffic-based SNS is presented through the background color of the cell. A higher number of visits is represented by a darker color. Cells including any resource with students’ annotations are augmented with a yellow sticky note. Cells including positive personal annotations are augmented with a red sticky note. Overall viewpoints of the annotations are visualized with a thermometer. The temperature grows warmer as the resources inside the cell attract more positive annotations. A cell shows a list of resources, which are augmented with visual cues representing trafficand annotation-based SNS. A human icon inside a colored square represents traffic-based SNS. Annotation-based SNS is represented by an annotation icon in a colored square. The icon represents the type of personal annotation, with a sticky note for general annotation and thumbs up for positive annotation. The color saturation of the icon represents the number of personal annotations. Darker colors represent a higher number of annotations. The background color represents the type of group annotation and the density of the color represents the number of annotations. 4. Evaluation The system was evaluated in the context of interactive system design course which about 30 students enrolled. The student created several hundred annotations during the course. Most of them were created for bookmarking a region of interest while the others were used to leave simple notes, as reminders or to share opinions. Comparative log analysis of this course and a previous version of the course which has not use spatial annotations demonstrated that the availability of spatial annotations significantly increased the number of comments made by the users as well as their reading activity. As expected, social annotations demonstrated their ability to guide users to annotated pages. We also observed that the majority of annotations appeared on important pages and important fragments within the page. The users regarded new functionality very positively all aspects of new interface. At the same time, some students mentioned that the icons and style of annotation are not familiar to them and it might cause cognitive loads. It seems like that more intuitive way to represent the icon and annotation is required for the students who have little experience with social navigation functionality Figure 3: Navigation support functionality in Knowledge Sea II 5. Conclusions The paper presented our attempt to extend the value of scanned electronic books with spatial annotation and social navigation support. With this approach, users create regions on the electronic textbooks as a bookmark, leave notes and exchange ideas with the community of readers. Annotations are used to enhance social navigation support and to guide readers to most interesting pages. Our results show that our approach is useful at least in one context of using electronic books – in the classroom. We expect, however, that this approach can improve the reader’s experience in many other contexts. 6. Acknowledgements This work is supported by National Science Foundation under Grant IIS-0447083 and the Korea Research Foundation Grant funded by the Korean Government (MOEHRD). KRF-2007-357D00231.