In recent years, two communities have grown around a joint interest on how big data can be exploited to benefit education and the science of learning: Educational Data Mining and Learning Analytics. This article discusses the relationship between these two communities, and the key methods and approaches of educational data mining. The article discusses how these methods emerged in the early days of research in this area, which methods have seen particular interest in the EDM and learning analytics communities, and how this has changed as the field matures and has moved to making significant contributions to both educational research and practice.

/pdf/educational-data-mining-and-learning-analytics-2kepggrl5r.pdf

Educational Data Mining and Learning Analytics

Research on recommender systems typically focuses on the accuracy of prediction algorithms. Because accuracy only partially constitutes the user experience of a recommender system, this paper proposes a framework that takes a user-centric approach to recommender system evaluation. The framework links objective system aspects to objective user behavior through a series of perceptual and evaluative constructs (called subjective system aspects and experience, respectively). Furthermore, it incorporates the influence of personal and situational characteristics on the user experience. This paper reviews how current literature maps to the framework and identifies several gaps in existing work. Consequently, the framework is validated with four field trials and two controlled experiments and analyzed using Structural Equation Modeling. The results of these studies show that subjective system aspects and experience variables are invaluable in explaining why and how the user experience of recommender systems comes about. In all studies we observe that perceptions of recommendation quality and/or variety are important mediators in predicting the effects of objective system aspects on the three components of user experience: process (e.g. perceived effort, difficulty), system (e.g. perceived system effectiveness) and outcome (e.g. choice satisfaction). Furthermore, we find that these subjective aspects have strong and sometimes interesting behavioral correlates (e.g. reduced browsing indicates higher system effectiveness). They also show several tradeoffs between system aspects and personal and situational characteristics (e.g. the amount of preference feedback users provide is a tradeoff between perceived system usefulness and privacy concerns). These results, as well as the validated framework itself, provide a platform for future research on the user-centric evaluation of recommender systems.

/pdf/explaining-the-user-experience-of-recommender-systems-2akchbybuf.pdf

Explaining the user experience of recommender systems

The authors are grateful for the deliberations of our technical working group (TWG) of academic experts in educational data mining and learning analytics. These experts provided constructive guidance and comments for this issue brief. The TWG comprised Ryan S. In data mining and data analytics, tools and techniques once confined to research laboratories are being adopted by forward-looking industries to generate business intelligence for improving decision making. Higher education institutions are beginning to use analytics for improving the services they provide and for increasing student grades and retention. The U.S. Department of Education's National Education Technology Plan, as one part of its model for 21st-century learning powered by technology, envisions ways of using data from online learning systems to improve instruction. With analytics and data mining experiments in education starting to proliferate, sorting out fact from fiction and identifying research possibilities and practical applications are not easy. This issue brief is intended to help policymakers and administrators understand how analytics and data mining have been—and can be—applied for educational improvement. At present, educational data mining tends to focus on developing new tools for discovering patterns in data. These patterns are generally about the microconcepts involved in learning: one-digit multiplication, subtraction with carries, and so on. Learning analytics—at least as it is currently contrasted with data mining—focuses on applying tools and techniques at larger scales, such as in courses and at schools and postsecondary institutions. But both disciplines work with patterns and prediction: If we can discern the pattern in the data and make sense of what is happening, we can predict what should come next and take the appropriate action. Educational data mining and learning analytics are used to research and build models in several areas that can influence online learning systems. One area is user modeling, which encompasses what a learner knows, what a learner's behavior and motivation are, what the user experience is like, and how satisfied users are with online learning. At the simplest level, analytics can detect when a student in an online course is going astray and nudge him or her on to a course correction. At the most complex, they hold promise of detecting boredom from patterns of key clicks and redirecting the student's attention. Because these data are gathered in real time, there is a real possibility of continuous improvement via multiple feedback loops that operate at different time scales—immediate to the student …

/pdf/enhancing-teaching-and-learning-through-educational-data-695k7hhb05.pdf

Enhancing Teaching and Learning Through Educational Data Mining and Learning Analytics: An Issue Brief

Web-Based Education for All: A Tool for Development Adaptive Courseware.

This review pursues a twofold goal, the first is to preserve and enhance the chronicles of recent educational data mining (EDM) advances development; the second is to organize, analyze, and discuss the content of the review based on the outcomes produced by a data mining (DM) approach. Thus, as result of the selection and analysis of 240 EDM works, an EDM work profile was compiled to describe 222 EDM approaches and 18 tools. A profile of the EDM works was organized as a raw data base, which was transformed into an ad-hoc data base suitable to be mined. As result of the execution of statistical and clustering processes, a set of educational functionalities was found, a realistic pattern of EDM approaches was discovered, and two patterns of value-instances to depict EDM approaches based on descriptive and predictive models were identified. One key finding is: most of the EDM approaches are ground on a basic set composed by three kinds of educational systems, disciplines, tasks, methods, and algorithms each. The review concludes with a snapshot of the surveyed EDM works, and provides an analysis of the EDM strengths, weakness, opportunities, and threats, whose factors represent, in a sense, future work to be fulfilled.

https://romisatriawahono.net/lecture/rm/survey/machine%20learning/Ayala%20-%20Educational%20Data%20Mining%20-%202014.pdf

Review: Educational data mining: A survey and a data mining-based analysis of recent works

This paper examines the sufficiency of existing eLearning standards for facilitating and supporting the introduction of adaptive techniques in computer-based learning systems. To that end, the main representational and operational requirements of adaptive learning environments are examined and contrasted against current eLearning standards. The motivation behind this preliminary analysis is attainment of: interoperability between adaptive learning systems; reuse of adaptive learning materials; and, the facilitation of adaptively supported, distributed learning activities.

/pdf/adaptive-learning-environments-and-e-learning-standards-2sp7pbl99a.pdf

Adaptive Learning Environments and e-Learning Standards.

The evaluation of interactive adaptive systems has long been acknowledged to be a complicated and demanding endeavour. Some promising approaches in the recent past have attempted tackling the problem of evaluating adaptivity by "decomposing" and evaluating it in a "piece-wise" manner. Separating the evaluation of different aspects can help to identify problems in the adaptation process. This paper presents a framework that can be used to guide the "layered" evaluation of adaptive systems, and a set of formative methods that have been tailored or specially developed for the evaluation of adaptivity. The proposed framework unifies previous approaches in the literature and has already been used, in various guises, in recent research work. The presented methods are related to the layers in the framework and the stages in the development lifecycle of interactive systems. The paper also discusses practical issues surrounding the employment of the above, and provides a brief overview of complementary and alternative approaches in the literature.

/pdf/layered-evaluation-of-interactive-adaptive-systems-framework-9qzhck97td.pdf

Layered evaluation of interactive adaptive systems: framework and formative methods

Adaptive systems are usually interactive systems As such they stand to benefit considerably from a development lifecycle that ensures user involvement from the early design stages, and embraces evaluation, in both formative and summative forms Evaluating an adaptive system involves a number of specific problems and pitfalls that need to be addressed by the selection of specific methods, techniques and criteria This tutorial aims to introduce participants to the peculiarities that arise when evaluating adaptive interactive systems A layered evaluation framework is used to separate the evaluation process into a number of different aspects which can be applied at different stages throughout the development life-cycle

/pdf/evaluation-of-adaptive-systems-4s5j4vx2sg.pdf

Evaluation of Adaptive Systems

Monitoring and interpreting sequential learner activities has the potential to improve adaptivity and personalization within educational environments. We present an approach based on the modeling of learners' problem solving activity sequences, and on the use of the models in targeted, and ultimately automated clustering, resulting in the discovery of new, semantically meaningful information about the learners. The approach is applicable at different levels: to detect pre-defined, well-established problem solving styles, to identify problem solving styles by analyzing learner behaviour along known learning dimensions, and to semi-automatically discover learning dimensions and concrete problem solving patterns. This article describes the approach itself, demonstrates the feasibility of applying it on real-world data, and discusses aspects of the approach that can be adjusted for different learning contexts. Finally, we address the incorporation of the proposed approach in the adaptation cycle, from data acquisition to adaptive system interventions in the interaction process.

/pdf/activity-sequence-modelling-and-dynamic-clustering-for-4kg946xwmx.pdf

Activity sequence modelling and dynamic clustering for personalized e-learning

Adaptive User Interfaces (AUIs) are a continuously growing area of research, with numerous experimental and commercial systems reported in the literature, in diverse application areas. Although there exist today a number of established approaches and frameworks for the design and implementation of AUIs, their evaluation is yet to be addressed in a comprehensive way. Arguably, the main drawback of existing evaluation methods and techniques is that they fail to provide adequate feedback into the AUI design process, or to generate empirical findings that can be reused across application boundaries. This paper proposes a new, modular approach to the evaluation of AUIs, which is specifically intended to cater for the aforementioned problems. The adaptive Nautilus Web browser, on which the approach will be applied is also briefly presented.

Alexandros Paramythis

Papers

Adaptive Learning Environments and e-Learning Standards.

Layered evaluation of interactive adaptive systems: framework and formative methods

Evaluation of Adaptive Systems

Activity sequence modelling and dynamic clustering for personalized e-learning

A modular approach to the evaluation of Adaptive User Interfaces