Advances in artificial intelligence, sensors and big data management have far-reaching societal impacts. As these systems augment our everyday lives, it becomes increasing-ly important for people to understand them and remain in control. We investigate how HCI researchers can help to develop accountable systems by performing a literature analysis of 289 core papers on explanations and explaina-ble systems, as well as 12,412 citing papers. Using topic modeling, co-occurrence and network analysis, we mapped the research space from diverse domains, such as algorith-mic accountability, interpretable machine learning, context-awareness, cognitive psychology, and software learnability. We reveal fading and burgeoning trends in explainable systems, and identify domains that are closely connected or mostly isolated. The time is ripe for the HCI community to ensure that the powerful new autonomous systems have intelligible interfaces built-in. From our results, we propose several implications and directions for future research to-wards this goal.

https://dl.acm.org/doi/pdf/10.1145/3173574.3174156

Trends and Trajectories for Explainable, Accountable and Intelligible Systems: An HCI Research Agenda

The goal of text ranking is to generate an ordered list of texts retrieved from a corpus in response to a query. Although the most common formulation of text ranking is search, instances of the task can also be found in many natural language processing applications. This survey provides an overview of text ranking with neural network architectures known as transformers, of which BERT is the best-known example. The combination of transformers and self-supervised pretraining has been responsible for a paradigm shift in natural language processing (NLP), information retrieval (IR), and beyond. In this survey, we provide a synthesis of existing work as a single point of entry for practitioners who wish to gain a better understanding of how to apply transformers to text ranking problems and researchers who wish to pursue work in this area. We cover a wide range of modern techniques, grouped into two high-level categories: transformer models that perform reranking in multi-stage architectures and dense retrieval techniques that perform ranking directly. There are two themes that pervade our survey: techniques for handling long documents, beyond typical sentence-by-sentence processing in NLP, and techniques for addressing the tradeoff between effectiveness (i.e., result quality) and efficiency (e.g., query latency, model and index size). Although transformer architectures and pretraining techniques are recent innovations, many aspects of how they are applied to text ranking are relatively well understood and represent mature techniques. However, there remain many open research questions, and thus in addition to laying out the foundations of pretrained transformers for text ranking, this survey also attempts to prognosticate where the field is heading.

Pretrained Transformers for Text Ranking: BERT and Beyond

This chapter presents an analysis of recommender systems in Technology-Enhanced Learning along their 15 years existence (2000–2014). All recommender systems considered for the review aim to support educational stakeholders by personalising the learning process. In this meta-review 82 recommender systems from 35 different countries have been investigated and categorised according to a given classification framework. The reviewed systems have been classified into seven clusters according to their characteristics and analysed for their contribution to the evolution of the RecSysTEL research field. Current challenges have been identified to lead the work of the forthcoming years.

Panorama of Recommender Systems to Support Learning

Community question-answering (CQA) systems, such as Yahooe Answers or Stack Overflow, belong to a prominent group of successful and popular Web 2.0 applications, which are used every day by millions of users to find an answer on complex, subjective, or context-dependent questions. In order to obtain answers effectively, CQA systems should optimally harness collective intelligence of the whole online community, which will be impossible without appropriate collaboration support provided by information technologies. Therefore, CQA became an interesting and promising subject of research in computer science and now we can gather the results of 10 years of research. Nevertheless, in spite of the increasing number of publications emerging each year, so far the research on CQA systems has missed a comprehensive state-of-the-art survey. We attempt to fill this gap by a review of 265 articles published between 2005 and 2014, which were selected from major conferences and journals. According to this evaluation, at first we propose a framework that defines descriptive attributes of CQA approaches. Second, we introduce a classification of all approaches with respect to problems they are aimed to solve. The classification is consequently employed in a review of a significant number of representative approaches, which are described by means of attributes from the descriptive framework. As a part of the survey, we also depict the current trends as well as highlight the areas that require further attention from the research community.

A Comprehensive Survey and Classification of Approaches for Community Question Answering

This study investigated the differences in learners' performance and perceived engagement between three intervention groups in a Science class, using two types of pedagogical intervention: traditional instruction with paper-based quizzes and gamified instruction with gamified e-quizzes as formative assessments. With respect to the gamified instruction, three types of gamification applications were employed: Socrative, Quizizz, and iSpring Learn LMS. The effects of the instructional intervention (n = 94), as well as evaluative feedback, were obtained with the aid of formative quizzes, post-questionnaire surveys, and personal interviews. The results showed that the employment of innovative gamified e-quiz applications (i.e., Socrative, Quizizz, and iSpring Learn LMS) and paper-based quizzes were effective in evaluating students' learning performance, particularly as formative assessment after completing each topic. Finding ways to apply games or game concepts in the classroom can be a promising and innovative tool for educators to engage their students in creative learning skills and attractive competition.

The role of gamified e-quizzes on student learning and engagement: An interactive gamification solution for a formative assessment system

In the current time of globalization, collaboration among people in virtual environments is becoming an important precondition of success. This trend is reflected also in the educational domain where students collaborate in various short-term groups created repetitively but changing in each round (e.g. in MOOCs). Students in these kind of dynamic groups quite often encounter various difficulties, which are obvious mainly when students’ characteristics do not complement each other. In spite of various group formation methods aimed to solve the group compatibility problem, most of the existing approaches do not consider dynamic groups. We describe (i) a proposal of a novel group formation method based on Group Technology approach, which considers feedback on students’ collaboration to improve group formations; (ii) an application of the method as a part of a collaborative platform PopCorm, which provides students in the created groups with a set of real-time collaboration tools; (iii) a long-term experiment in which the groups created by our method achieved significantly better results in comparison with the reference approaches. Our results indicate that considering feedback from students’ collaboration can improve the group formation process as the groups created by our method achieved higher collaboration quality with next iterations.

Dynamic Group Formation as an Approach to Collaborative Learning Support

Enormous amounts of knowledge sharing occur every day in community question answering (CQA) sites, some of which (for example, Stack Overflow or Ask Ubuntu) have become popular with software developers and users. In spite of these systems' overall success, problems are emerging in some of them--increased failure and churn rates. To investigate this trend, researchers conducted a case study of Stack Overflow. First, they evaluated the community perception that the emerging problems are heavily related to the growing amount of low-quality content created by undesired groups of users (help vampires, noobs, and reputation collectors). Reproducible data analyses of content and community evolution supported these findings. Suggestions to deal with the emerging problems include providing users with responder-oriented adaptive support that involves a whole community in QA. Such approaches represent an eminent attitude change regarding QA support, with the aim to preserve CQA ecosystems' long-term sustainability.

Why is Stack Overflow Failing? Preserving Sustainability in Community Question Answering

Web 2.0 has had a tremendous impact on education. It facilitates access and availability of learning content in variety of new formats, content creation, learning tailored to students’ individual preferences, and collaboration. The range of Web 2.0 tools and features is constantly evolving, with focus on users and ways that enable users to socialize, share and work together on (user-generated) content. In this chapter we present ALEF—Adaptive Learning Framework that responds to the challenges posed on educational systems in Web 2.0 era. Besides its base functionality—to deliver educational content—ALEF particularly focuses on making the learning process more efficient by delivering tailored learning experience via personalized recommendation, and enabling learners to collaborate and actively participate in learning via interactive educational components. Our existing and successfully utilized solution serves as the medium for presenting key concepts that enable realizing Web 2.0 principles in education, namely lightweight models, and three components of framework infrastructure important for constant evolution and inclusion of students directly into the educational process—annotation framework, feedback infrastructure and widgets. These make possible to devise and implement various mechanisms for recommendation and collaboration—we also present selected methods for personalized recommendation and collaboration together with their evaluation in ALEF.

/pdf/alef-from-application-to-platform-for-adaptive-collaborative-1nlxmh17t0.pdf

ALEF: From Application to Platform for Adaptive Collaborative Learning

The negative effects of misinformation filter bubbles in adaptive systems have been known to researchers for some time. Several studies investigated, most prominently on YouTube, how fast a user can get into a misinformation filter bubble simply by selecting “wrong choices” from the items offered. Yet, no studies so far have investigated what it takes to “burst the bubble”, i.e., revert the bubble enclosure. We present a study in which pre-programmed agents (acting as YouTube users) delve into misinformation filter bubbles by watching misinformation promoting content (for various topics). Then, by watching misinformation debunking content, the agents try to burst the bubbles and reach more balanced recommendation mixes. We recorded the search results and recommendations, which the agents encountered, and analyzed them for the presence of misinformation. Our key finding is that bursting of a filter bubble is possible, albeit it manifests differently from topic to topic. Moreover, we observe that filter bubbles do not truly appear in some situations. We also draw a direct comparison with a previous study. Sadly, we did not find much improvements in misinformation occurrences, despite recent pledges by YouTube.

Ivan Srba

Papers

A Comprehensive Survey and Classification of Approaches for Community Question Answering

Dynamic Group Formation as an Approach to Collaborative Learning Support

Why is Stack Overflow Failing? Preserving Sustainability in Community Question Answering

ALEF: From Application to Platform for Adaptive Collaborative Learning

An Audit of Misinformation Filter Bubbles on YouTube: Bubble Bursting and Recent Behavior Changes