Showing papers by "Dirk T. Tempelaar published in 2021"

The contribution of dispositional learning analytics to precision education

Abstract: Precision education requires two equally important conditions: accurate predictions of academic performance based on early observations of the learning process and the availability of relevant educational intervention options. The field of learning analytics (LA) has made important contributions to the realisation of the first condition, especially in the context of blended learning and online learning. Prediction models that use data from institutional information systems and logs of learning management systems have gained a good reputation in predicting underperformance and dropout risk. However, less progress is made in resolving the second condition: applying LA generated feedback to design educational interventions. In our contribution, we make a plea for applying dispositional learning analytics (DLA) to make LA precise and actionable. DLA combines learning data, as in LA, with learners’ disposition data measured through self-report surveys. The advantage of DLA is twofold: first, it improves the accuracy of prediction, specifically early in the module, when limited LMS trace data are available. Second, the main benefit of DLA is in the design of effective interventions: interventions that focus on addressing individual learning dispositions that are less developed but important for being successful in the module. We provide an empirical analysis of DLA in an introductory mathematics module, demonstrating the important role that a broad range of learning dispositions can play in realising precision education.

Dispositional Learning Analytics for Supporting Individualized Learning Feedback

Abstract: An important goal of learning analytics (LA) is to improve learning by providing students with meaningful feedback. Feedback is often generated by prediction models of student success using data about students and their learning processes based on digital traces of learning activities. However, early in the learning process, when feedback is most fruitful, trace-data-based prediction models often have limited information about the initial ability of students, making it difficult to produce accurate prediction and personalized feedback to individual students. Furthermore, feedback generated from trace data without appropriate consideration of learners’ dispositions might hamper effective interventions. By providing an example of the role of learning dispositions in an LA application directed at predictive modeling in an introductory mathematics & statistics module, we make a plea for applying dispositional learning analytics (DLA) to make LA precise and actionable. DLA combines learning data with learners’ disposition data measured through for example self-report surveys. The advantage of DLA is twofold: first, to improve the accuracy of early predictions; and second, to link LA predictions with meaningful learning interventions that focus on addressing less developed learning dispositions. Dispositions in our DLA example include students’ mindsets, operationalized as entity and incremental theories of intelligence, and corresponding effort beliefs. These dispositions were inputs for a cluster analysis generating different learning profiles. These profiles were compared for other dispositions and module performance. The finding of profile differences suggests that the inclusion of disposition data and mindset data, in particular, adds predictive power to LA applications.

Personality traits and academic performance: Correcting self-assessed traits with vignettes.

Abstract: In this study, we investigate whether Conscientiousness, Emotional Stability and Risk Preference relate to student performance in higher education. We employ anchoring vignettes to correct for heterogeneous scale use in these non-cognitive skills. Our data are gathered among first-year students at a Dutch university. The results show that Conscientiousness is positively related to student performance, but the estimates are strongly biased upward if we use the uncorrected variables. We do not find significant relationships for Emotional Stability but find that the point estimates are larger when using the uncorrected variables. Measured Risk Preference is negatively related to student performance, yet this is fully explained by heterogeneous scale use. These results indicate the importance of using more objective measurements of personality traits.

Enabling Precision Education by Learning Analytics Applying Trace, Survey and Assessment Data

Abstract: Accurate and timely measurement of learning engagement is crucial for the application of precision education. At the same time, it is still a central research theme, both in the learning analytics community as in the broader area of educational research. 'Engagement is one of the hottest research topics in the field of educational psychology' is for a good reason the opening sentence of a recent special issue. In our contribution, we propose a holistic approach to the measurement of engagement by integrating data of behavioral type through traces of learning processes captured from log files into affective, behavioral, and cognitive measures of engagement collected with surveys and cognitive measures from assessments for and as learning. We apply this holistic approach in an empirical analysis of dispositional learning analytics. Starting from four different engagement profiles created by two-step clustering, we find that these profiles primarily differ in their timing of engagement with learning. Next, we develop regression-based prediction models that make clear that trace, survey, and assessment data have complementary roles in signaling students at risk for failure and are all three crucial constituents of prediction equations that differ in the timing of learning feedback.

Types of boredom and other learning activity emotions: A person-centred investigation of inter-individual data

Abstract: Whether boredom is a unitary construct or if multiple types of boredom exist is a long-standing debate. Recent research has established the existence of boredom types based on frequency observations of boredom by experience sampling. This work tries to expand our understanding of boredom and replicate these previous findings by applying intensity observations of cross-sectional type for four discrete learning activity emotions: boredom, anxiety, hopelessness, and enjoyment. Latent class analysis based on activity emotion scores from 9863 first-year students of a business and economics program results in seven profiles. Five of these profiles allow a linear ordering from low to high control and value scores (the direct antecedents of emotions), low to high positive, and high to low negative emotions. Two profiles differ from this pattern: one ‘high boredom’ profile and one ‘low boredom’ profile. We next compare antecedent relationships of activity emotions at three different levels: inter-individual, inter-class or between classes, and intra-class or within classes. Some of these relationships are invariant for the choice of level of analysis, such as hopelessness. Other relationships, such as boredom, are highly variant: within-class relationships differ from inter-individual relationships. Indeed, our results confirm that boredom is not a unitary construct. The types of boredom found and their implications for educational practice are discussed and shared in this article.