Personalizing Computer Science Education by Leveraging Multimodal Learning Analytics

TL;DR: A framework that harness sources of programming learning analytics on three computer programming courses a Higher Education Institution and built predictive models using student characteristics, prior academic history, logged interactions between students and online resources, and students’ progress in programming laboratory work to improve personalized learning.

Abstract: This Research Full Paper implements a framework that harness sources of programming learning analytics on three computer programming courses a Higher Education Institution. The platform, called PredictCS, automatically detects lower-performing or “at-risk” students in programming courses and automatically and adaptively sends them feedback. This system has been progressively adopted at the classroom level to improve personalized learning. A visual analytics dashboard is developed and accessible to Faculty. This contains information about the models deployed and insights extracted from student’s data. By leveraging historical student data we built predictive models using student characteristics, prior academic history, logged interactions between students and online resources, and students’ progress in programming laboratory work. Predictions were generated every week during the semester’s classes. In addition, during the second half of the semester, students who opted-in received pseudo real-time personalised feedback. Notifications were personalised based on students’ predicted performance on the course and included a programming suggestion from a top-student in the class if any programs submitted had failed to meet the specified criteria. As a result, this helped students who corrected their programs to learn more and reduced the gap between lower and higher-performing students.

Summary

Introduction

• In the last two decades, Synthetic Aperture Radar (SAR) polarimetry has shown promise through airborne research campaigns, which lead to the planning of space-borne missions that offer fully polarimetric modes (i.e., ALOS-PALSAR, TerraSAR-X and RADARSAT-2).
• SAR data classification is known to be challenging due to speckle, that results in large variation of the backscatter across neighboring pixels within the same distributed target (e.g., a field of wheat).
• Over the last two decades, research in computer vision has sought methodologies that can utilize these ideas and in the last few years a promising technique for grouping applications, spectral graph partitioning, has emerged [3], [4].
• Utilizing multiple cues that contribute to perceptual grouping process (i.e., similarity in brightness, color or texture, proximity, and contour continuity) results in segmentations that are perceptually plausible (i.e., consistent with what humans perceive).
• Detailed analysis of different aspects of the proposed scheme and validation for multiple data sets is underway.

II. SPECTRAL GRAPH PARTITIONING

• Both clustering and image segmentation can be formulated as a graph partitioning problem, by representing a set of points in an arbitrary feature space using an undirected graph G = {V,E}, where V and E represent the nodes and the edges (i.e., connections) respectively.
• Each node on the graph corresponds to a data point in feature space and the edge between two nodes, u and υ, is associated with a weight, ω(u, υ), that indicates the similarity of that pair.
• Note that the expression in (3) is the Rayleigh quotient, and if the condition on y is relaxed so that it can take on real values, the solution can be obtained by solving the generalized eigenvalue system, (D − W )y = λD y (4) where D −W is known as the graph Laplacian.
• The eigenvector that corresponds to the second smallest eigenvalue is the real valued solution for (3).

A. An algorithm for k-way partitioning: Spectral Clustering

• Based on the approach described in the previous section, the spectral clustering algorithm given by Ng et al. [4] provides k-way partitioning.
• A slightly different notation is used, where the matrix W is now called the affinity matrix, A, and I −L is replaced with L, the normalized affinity matrix.
• Therefore the eigenvectors that correspond to the largest eigenvalues are used instead of the smallest ones.
• 12 ) 5) Cluster rows of Y using the K-means algorithm Since pairwise similarities are used to determine the groups, it becomes possible to recover complicated manifold structures in the feature space, which can not be achieved by central grouping techniques (e.g., k-means or EM) that require each group member to be close to a prototype (i.e., the cluster center).
• This procedure automates the parameter selection and provides good results by adaptively choosing a scaling parameter σi for each point si.

B. Computational Complexity and Fast Approximate Solutions

• The spectral graph partitioning framework involves solving the eigenvalue problem for the normalized affinity matrix, of size N ×N , where N is the number of points (or pixels for image segmentation problems).
• The computational cost of this dense solution quickly becomes prohibitive for images of useful size.
• Therefore, an iterative method (e.g., Lanczos) can be used to obtain the solution with the time complexity of O(N2); 2) Sparse representation:.
• In the context of solving the eigenvalue problem for the normalized affinity matrix, this turns out to be very useful.
• The details of this method with application to spectral clustering can be found in [6].

III. PROPOSED SCHEME FOR CLASSIFICATION OF POLARIMETRIC SAR DATA

• The proposed scheme for classification of POLSAR data is based on the spectral clustering algorithm: Perform multi-looking on single look complex (SLC) data (i.e., for Convair-580 data set, use 10 azimuth looks).
• Apply a polarimetric SAR speckle filter, as suggested in Lee et al. [7] (i.e., use window size of 7 × 7) Apply the spectral clustering algorithm with the following modifications:.
• 4) Perform Steps 2 to 5 in the spectral clustering algorithm as in Section II-A (i.e., form matrix D and L, calculate the first k eigenvectors of L, form matrix X , normalize its rows and cluster using the K-means algorithm).

IV. RESULTS AND DISCUSSION

• The results presented in this section are obtained using a subset of the Westham Island scene shown in Figure 1.
• Therefore the authors have chosen the region of interest (ROI) shown in Figure 1(b), where most of the fields have ground truth information given in Figure 1(d).
• Figure 2 shows a set of results obtained using the proposed scheme and the Wishart classifier.
• This figure demonstrates how these normalized eigenvectors can be used to obtain “SC Result” in Figure 3.

V. CONCLUSION

• A new technique based on spectral graph partitioning is proposed for polarimetric SAR data classification and the spectral clustering algorithm is modified to account for the properties of such data.
• Edge-aligned patch-based similarity measured by the χ2 distance between histograms and spatial proximity of pixels are used to form the affinity matrix.
• It is shown that this approach not only outperforms the Wishart classifier, but also allows further improvement by offering flexibility in using additional cues (e.g., continuity, texture, optical data) and different affinity functions.

Figures

TABLE IV STUDENT OPT-INS AND OPT-OUTS TO INTERVENTIONS

Fig. 6. PF2 F1 Classifiers Performance

TABLE V COURSE INFORMATION, PASS RATES, AT-RISK PREDICTION RATES, PREDICTION METRIC RESULTS AND CORRELATIONS PER COURSE

TABLE VI INTERVENTIONS

Fig. 11. Personalized notification sent to a student in SH1

TABLE VII LEARNING IMPROVEMENT

Fig. 1. Enrolment numbers on the courses studied

TABLE I COURSE PASS RATES ON GROUNDTRUTH DATA

TABLE IX SURVEY RESPONSES FROM STUDENTS ABOUT THE PROJECT

TABLE VIII IMPROVEMENT BETWEEN HIGHER AND LOWER-PERFORMING STUDENTS OVER THE TWO ACADEMIC YEARS

TABLE II LEARNING ALGORITHM SELECTED FOR EACH COURSE

Fig. 4. SH1 F1 Classifiers Performance

Fig. 5. SH1 F1 Fail class Classifiers Performance

Fig. 3. Performance of the bag of classifiers using the F1-metric for course CS2 looking at the fail class

Fig. 2. CS2 F1 Classifiers Performance

Fig. 8. CS2 ROC AUC Features Analysis

Fig. 10. CS2 Top Features Week 2

Fig. 7. PF2 F1 Classifiers Performance for the likely-to-fail class

TABLE III CORRELATION BETWEEN THE FEATURE VALUES AND THE TARGET PERFORMANCE GRADES TO BE PREDICTED

Fig. 9. Fail class Feature Analysis using the F1-metric for course SH1 looking at the fail class

Full text

Personalizing Computer Science Education by
Leveraging Multimodal Learning Analytics
David Azcona
Insight Centre for Data Analytics
Dublin City University
Dublin, Ireland
David.Azcona@insight-centre.org
I-Han Hsiao
School of Computing, Informatics
& Decision Systems Engineering
Arizona State University
Tempe, Arizona, USA
Sharon.Hsiao@asu.edu
Alan F. Smeaton
Insight Centre for Data Analytics
Dublin City University
Dublin, Ireland
Alan.Smeaton@insight-centre.org
Abstract—This Research Full Paper implements a framework
that harness sources of programming learning analytics on three
computer programming courses a Higher Education Institution.
The platform, called PredictCS, automatically detects lower-
performing or “at-risk” students in programming courses and
automatically and adaptively sends them feedback. This system
has been progressively adopted at the classroom level to improve
personalized learning. A visual analytics dashboard is developed
and accessible to Faculty. This contains information about the
models deployed and insights extracted from student’s data. By
leveraging historical student data we built predictive models
using student characteristics, prior academic history, logged
interactions between students and online resources, and students’
progress in programming laboratory work. Predictions were
generated every week during the semester’s classes. In addition,
during the second half of the semester, students who opted-in
received pseudo real-time personalised feedback. Notifications
were personalised based on students’ predicted performance on
the course and included a programming suggestion from a top-
student in the class if any programs submitted had failed to
meet the specified criteria. As a result, this helped students who
corrected their programs to learn more and reduced the gap
between lower and higher-performing students.
Index Terms—Computer Science Education, Learning Ana-
lytics, Predictive Modelling, Peer Learning, Machine Learning,
Educational Data Mining
I. INTRODUCTION
PredictCS is a Predictive Analytics platform for Computer
Science courses that notifies students based on their perfor-
mance using past student data and recommends most suitable
resources for students to consult [1]. The first implementation
of this framework was on an introductory computer program-
ming course in 2017 [2]. This implementation trained a model
of student performance using one year of groundtruth with
data features including engagement and programming effort.
Pseudo real-time predictions were run on a new cohort of stu-
dents while “at-risk” students were targeted during laboratory
sessions.
The framework has since been updated by the implemen-
tation of a new multimodal predictive analytics system that
aggregates sources of student digital footprints from blended
classroom settings [3]. Advanced data mining techniques are
adopted to engineer models to provide realtime prediction
and dynamic feedback. Preliminary results on a programming
course show the potential of this solution. In this work, the
system is implemented in three more computer programming
courses. The models add further features, namely static student
information along with more engagement features. In short, the
system uses machine learning techniques and predicts student
performance in three programming courses at first and second-
year undergraduate level.
A retrospective analysis was carried out to verify the vi-
ability of the models, pseudo real-time predictions were run
weekly during the teaching period on new cohorts of students,
and automatic feedback was sent to students who opted in
during the second half of the semester. This feedback was built
using our performance predictions and code suggestions from
top-ranked students in the class. We propose the following
research questions:
RQ1: How accurate are the proposed predictive models with
generic static and dynamic student data features identifying
students in need in programming courses across a variety of
first and second-year programming courses?
RQ2: What are the effects of timely automatic adaptive
support and peer-programming feedback on students perfor-
mances across these courses and what are the differences
among them?
RQ3: What are the students and teachers perspectives and
experiences on these courses?
II. RELATED WORK
In CSEd research, based on the programming events’
granularity (type and frequency), different models have been
developed for modelling student programming learning be-
haviour. Researchers have leveraged key strokes, program
edits, compilations, executions and submissions [4]. In our
university’s automated grading system for the teaching of
computer programming we collect programming submissions
and web logs. We have a fine-grained footprint about each
submission but we are limited by the frequency of the students
submitting their solutions and we miss the programming
actions in-between.
Feedback is an important research avenue for the teaching
of programming and an effective way to motivate novice
programmers. Recently, researchers have been working on

augmenting the IDE or programming environment by crowd-
sourcing code solutions. Students are suggested error cor-
rections or solutions that peers have applied before. Java
has been the programming language targeted the most with
solutions such as BlueFix [5] or HelpMeOut [6]. This latter
social recommender system was also applied to Arduino. In
addition, Crowd::Debug [7] was presented as a similar solution
for Ruby, a test-driven development language. In terms of
notifying students how they are progressing throughout a
semester, Purdue University’s Course Signals [8] sends a
personalised mail and posts a traffic signal as an indicator of
their performance while Dublin City University’s PredictED
[9] project notifies students how they are doing and where they
are within their own class. Both systems yielded impressive
improvement in first-year retention rates. Our programming
grading system also provides real-time feedback on each
submission by running a suite of test cases but provides no
code suggestions or personalised help for errors.
Researches have focused on generating models which ac-
curately predict student performance and identifying which
factors carry more predictive power. However, after identifying
those “at-risk” students, we should intervene and help those
students. Targeting these weak students during laboratory
sessions can aid some students [2], but lecturers usually do
not have the time or resources to support many students in
large classes or to spend as much time identifying what the
student knows and does not know. Automatic interventions
for programming classes are having great success in other
institutions and environments and we are eager to develop our
own strategies using our platforms and resources.
III. DATA COLLECTION
Dublin City University’s academic year is divided in two
semesters with a week of inter-semester break in between.
Semesters are comprised of a 12-week teaching or classes
period, 2-week study period and 2-week exam period. Lab-
oratory sessions and computer-based examinations are carried
out during the teaching period. Our previous study [2] was
done on Computer Programming I, CS1, that introduces first-
year students to computer programming and the fundamentals
of computational problem solving during their first semester.
We work with the following courses that are taught in the
second semester (Fall):
• Computer Programming II, CS2: This course intro-
duces first-year students to more advanced programming
concepts, particularly object-oriented programming, pro-
gramming libraries, data structures and file handling.
Students are expected to engage extensively in hands-on
programming with the Python programming language. A
previous version of CS2 was taught using Java before it
was redesigned. The current version with Python has been
taught for two academic years, 2015/2016 and 2016/2017.
The course is a continuation of Computer Programming
I, an introductory programming course.
• Managing Enterprise Computer Systems, SH1: This
course equips first-year students with the basic skills nec-
essary to administer modern enterprise operating systems
and shows students how to manage Unix and Unix-like
systems. Specifically, they study the Unix shell and work
shell scripting programming exercises using tools like
test, find or grep and concepts like loops, pipes or file
handling. This course has been taught for the past seven
academic years since 2010/2011. Students work with the
Bash Unix shell and the command language.
• Programming Fundamentals III, PF3: This course
teaches second-year students fundamental data structures
and algorithms in computational problem solving. The
material includes linked lists, stacks, queues or binary-
search trees; and other techniques, like recursion. This
is a new course that has been taught for the first time
this academic year 2016/2017 and the language chosen
is Python. PF3 is a continuing course of Programming
Fundamentals II, PF2, that was taught to second-year
EC students for the first time on the first semester. Even,
Programming Fundamentals I, PF1, was taught to first-
year students for the first time this year but second-
year students could not take it last year as it didn’t
exist. In PF2, students learn to design simple algorithms
using structured data types like lists and dictionaries,
and write and debug computer programs requiring these
data structures in Python. This course is also taught in
Python. PF3 uses the Python language. It emerged along
with PF1 and PF2 for a need for enterprise computing
students to have deeper computer programming skills in
the workplace.
In all courses, students are assessed by taking two laboratory
computer-based programming exams, a mid-semester and an
end-of-semester assessment, during the teaching period. In
PF3, instead of an end-of-semester lab exam, students demo a
project. Each laboratory exam or demo contributes equally to
their continuous assessment mark; 15% in CS2, 25% in SH1
and 20% in PF3. Students are not required to submit their
laboratory work for SH1 or PF2. In contrast, laboratory work
count towards their final grade of the course for CS2 and PF3,
both 10% of the overall grade for the course. The CS1 Lecturer
developed a custom Virtual Learning Environment (VLE)
for the teaching of computer programming. This automated
grading platform is currently used in a variety of programming
courses across CS; including CS1, CS2, SH1, PF2 and PF3.
Students can browse course material, submit and verify their
laboratory work. This platform has been used for the past
two academic years and that is the data we are using for our
analysis.
In terms of numbers, 134 students registered for CS2 during
2015/2016 and 140 students in 2016/2017. For SH1, 70 and
81 students enrolled respectively for the past two academic
years. For PF3, 60 students registered in 2016/2017 as that
has been the only version of the course. Enrolment numbers
from previous years are shown in Figure 1. CS1 and PF2 are
taught in the first semester of the academic year. CS2, SH1
and PF3, the main subject of this study, are taught during the

2010 2011 2012 2013 2014 2015 2016 2017
25
50
75
100
125
150
CS1
CS2
SH1
PF3
Fig. 1. Enrolment numbers on the courses studied
second.
IV. PREDICTIVE MODELLING
Predictive Analytics models were developed to automati-
cally classify students having issues with programming mate-
rial. For each of the courses introduced a distinctive predictive
model was built, one per course. These models use the
student’s digital footprints to predict their performance in
computer-based laboratory programming assessments.
A. Student’s Digital Footprint
For each student, we build a digital footprint by leveraging
the data modalities available and modelling student interaction,
engagement and effort in programming courses. The data
sources are the following:
• Student Characteristics
• Prior Academic history
• Programming submissions
• Behavioural logs
Features are handcrafted using those data sources. In our
previous work [10], we analysed 950 first-year Computer
Science (CS) entrants across a seven year period and showed
the significant relationship between their entry points or math
skills and how they perform in their first year and in program-
ming courses at our university.
B. Training predictive models
A classification model was built for each course to distin-
guish “at-risk” students. CS2 and SH1’s models were trained
with historical student data from 2015/2016. In contrast, we
did not have any student training data for PF3 as 2016/2017
was the first academic year that course had being taught.
As a workaround, we trained a model using Programming
Fundamentals II (PF2)’s 2016/2017 student data which was
taught during the first semester. The target was to predict
whether each student would pass or fail their next laboratory
exam. These programming courses are quite dynamic and
programming laboratory exercises vary considerably from year
to year. However, concepts and knowledge being taught should
remain the same.
TABLE I
COURSE PASS RATES ON GROUNDTRUTH DATA
Course Year Semester # Students 1-Exam 2-Exam
CS2 2015/16 2 149 44.30% 46.98%
SH1 2015/16 2 73 32.88% 69.33%
PF2 2016/17 1 60 51.67% 36.67%
After deriving the features, a model for each course was
trained to predict a student’s likelihood of passing or fail-
ing the next computer-based laboratory exam. We developed
classifiers for each week of the semester. In 2015/2016, there
was a mid-semester exam and an end-of-semester exam for
the courses we are building our groundtruth with: CS2, SH1,
PF2. To clarify, classifiers from week 1 to 6 were trained to
predict the mid-semester’s laboratory exam’s outcome (pass or
fail for each student) and from 7 to 12 the end-of-semester’s
laboratory exam’s outcome.
The Empirical Error Minimization (EMR) approach has been
employed to determine the learning algorithm with the fewest
empirical errors from a bag of classifiers C [11, 2]. On these
models, instead of taking the learning algorithm with the
lowest empirical risk or the highest metric (namely accuracy
or F1-score), we looked at these metrics per class (pass
or fail). Generally, the results on the next laboratory exam,
our target variable, are quite imbalanced as in some courses
there might many more students that pass rather than fail an
exam. The resulting accuracy of a learning algorithm could
be misinterpreted if we weight the predictions based on the
numbers per class. Our goal is to identify weak students as
we prefer to classify students “on the edge” as likely to fail
rather than not flagging them at all and miss the opportunity to
intervene and help. See the training data pass rates in Table I.
C. Retrospective Analysis
Following a customized EMR approach, we selected a
classifier for each course that minimized the empirical risk
on average for the 12 weeks looking at the fail class and had
a good balance between both classes. The learning algorithms
and their corresponding metric values on our training data
are shown in Table II. For instance, in CS2, we selected a
K-Neighbors classifier which gave us a high F1-metric on
average and the highest on weeks 5 and 6. Those weeks are
key to identify who is struggling before their first assessment.
Figure 2 shows the performance of the bag of classifiers for
each week on average of the Cross-Validation folds. Figure 3
only shows the likely-to-fail class. Following this approach,
in CS2, we selected a K-Neighbors classifier which was the
second-highest on average for the F1-metric looking at both
classes. The highest was a Random Forest which was giving
the values 75.39% and 62.25% for the F1-score metric for
the fail and pass classes respectively. However, two classifiers
on week 5 and 6 were getting slightly better results by the
K-Neighbors classifier and those weeks are key to identify
who is struggling. We decided to choose the K-Neighbors then
so we could better identify them before their first laboratory

Fig. 2. CS2 F1 Classifiers Performance
1 2 3 4 5 6 7 8 9 10 11 12
Week
0.0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1.0
F1
Logistic Regression
SVC linear kernel
SVC rbf kernel
Random Forest
Decision Tree
K Neighbors
Fig. 3. Performance of the bag of classifiers using the F1-metric for course
CS2 looking at the fail class
examination on week 6. In SH1, Random Forest looked like
the most promising classifier with the highest fail class F1-
score, 74.26% but a very low value for the pass class: 32.20%.
The pass class values were particularly very low the first few
weeks of the semester as it was classifying most of the students
as failing the next laboratory exam which does not help us
that much. We went for an SVM with linear kernel with the
values shown in Table II. See Figures 4 and 5. We can see
how the SVM with Gaussian kernel fails to learn and predicts
all students to fail. It gets good accuracy some weeks of the
semester but we are aiming for a good balance for both classes
with an emphasis on the failing class. Lastly, for PF2, we
picked up the Decision Tree classifier as it gave us the highest
F1-score for the fail class and one of the highest for the pass
class. Recall, PF3’s model is based on PF2’s training data. See
Figures 6 and 7.
In addition we ran a statistical significance for the classifiers
selected with respect to the others in the bag of classifiers. The
predictions were only statistically significant compared to to
the SVM with Gaussian kernel which failed to learn properly.
This indicates the predictions are very similar as we only have
groundtruth data for one academic year so far and we do not
have enough information to determine the selected classifiers’
predictions were statistically independent.
Fig. 4. SH1 F1 Classifiers Performance
1 2 3 4 5 6 7 8 9 10 11 12
Week
0.0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1.0
F1
Logistic Regression
SVC linear kernel
SVC rbf kernel
Random Forest
Decision Tree
K Neighbors
Fig. 5. SH1 F1 Fail class Classifiers Performance
Fig. 6. PF2 F1 Classifiers Performance
TABLE II
LEARNING ALGORITHM SELECTED FOR EACH COURSE
Course Algorithm Class F1-score Precision Recall
CS2 K-Neighbors
Fail 74.50% 71.41% 81.03%
Pass 59.81% 68.80% 58.74%
SH1 Linear SVM
Fail 67.77% 71.46% 69.54%
Pass 41.61% 45.53% 42.82%
PF2 Decision Tree
Fail 66.73% 68.58% 72.14%
Pass 55.00% 58.62% 59.64%

1 2 3 4 5 6 7 8 9 10 11 12
Week
0.0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1.0
F1
Logistic Regression
SVC linear kernel
SVC rbf kernel
Random Forest
Decision Tree
K Neighbors
Fig. 7. PF2 F1 Classifiers Performance for the likely-to-fail class
TABLE III
CORRELATION BETWEEN THE FEATURE VALUES AND THE TARGET
PERFORMANCE GRADES TO BE PREDICTED
Course Feature description Pearson Spearman
CS2 CS1 Exam-2 2015/2016 35%
*
40%
*
CS2 Programming Week 6 73%
*
73%
*
SH1 Years on this course -31%
*
-26%
SH1 Week(end) rate Week 10 36%
*
35%
*
SH1 Programming Week 12 51%
*
51%
*
PF2 Hours Spent Week 8 48%
*
45%
*
*
p − value < 0.01
We measured the predictive power of our features by
calculating the correlation between the students’ grades and
our target, the next laboratory exam results, using the linear
(Pearson) and non-linear (Spearman) correlation coefficients.
Table III shows examples of previous academic performance,
static characteristics, interaction and programming features.
This analysis confirms the power of our features and the
programming weekly and cumulative progress features in-
creasingly gain importance throughout the semester as students
put more effort into the courses.
In a similar manner, we built a trees classifier for each
course and per week that fits a number of randomized decision
trees. By building this type of forest, we were able to compute
the importance for each feature. Based on this forest of impor-
tances, we selected the top 10 features every week to avoid
over-fitting. A similar comparative analysis to the classifier
analysis was run to verify this approach for feature selection
improved our metric values, see Figure 8 and Figure 9 for CS2
and SH1 respectively [12]. In addition, Figure 10 shows the
top features for a particular week for CS2.
This retrospective analysis shows us we can successfully
gather student data about their learning progress in those
programming courses leveraging students’ digital footprints.
V. STUDENT FEEDBACK
Students that decided to opt-in received weekly customized
notifications via email. After their first laboratory exam in
week 6, a feature was enabled in the grading platform platform
Fig. 8. CS2 ROC AUC Features Analysis
1 2 3 4 5 6 7 8 9 10 11 12
Week
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1.0
F1
In-course features + Demographics + Leaving Cert
In course features
(Web + lab)
Web events features
Laboratory Work features
Top features
Fig. 9. Fail class Feature Analysis using the F1-metric for course SH1 looking
at the fail class
for the teaching of computing programming where students
could freely opt-in or out from these notifications and read
about the project. After it was enabled, students were not able
to submit any programs before they either opted-in or out.
Table IV shows the number of people who replied to the opt-
in option. Most students opted-in on each course and they
received weekly notifications from that moment onwards.
The customized notifications were personalized by leverag-
ing our weekly predictions. Based on the associated probability
Fig. 10. CS2 Top Features Week 2

Frequently asked questions

Q1. What are the contributions mentioned in the paper "Personalizing computer science education by leveraging multimodal learning analytics" ?

This Research Full Paper implements a framework that harness sources of programming learning analytics on three computer programming courses a Higher Education Institution. Notifications were personalised based on students ’ predicted performance on the course and included a programming suggestion from a topstudent in the class if any programs submitted had failed to meet the specified criteria. 

Q2. How do the authors build a digital footprint for students?

For each student, the authors build a digital footprint by leveraging the data modalities available and modelling student interaction, engagement and effort in programming courses. 

Q3. What is the purpose of the course?

This course introduces first-year students to more advanced programming concepts, particularly object-oriented programming, programming libraries, data structures and file handling. 

Q4. How did the authors measure the predictive power of their features?

The authors measured the predictive power of their features by calculating the correlation between the students’ grades and their target, the next laboratory exam results, using the linear (Pearson) and non-linear (Spearman) correlation coefficients. 

Q5. What is the main purpose of PF3?

In PF2, students learn to design simple algorithms using structured data types like lists and dictionaries, and write and debug computer programs requiring these data structures in Python. 

Q6. What is the way to grade programming submissions?

Their programming grading system also provides real-time feedback on each submission by running a suite of test cases but provides no code suggestions or personalised help for errors. 

Q7. What was the approach used for the programming recommendations?

The approach the authors chose for the programming recommendations was to pick the closest text program from top-ranked students in the class that year. 

Q8. What was the classifier for the fail class?

In SH1, Random Forest looked like the most promising classifier with the highest fail class F1score, 74.26% but a very low value for the pass class: 32.20%. 

Q9. What are the advantages of using automatic interventions for programming classes?

Automatic interventions for programming classes are having great success in other institutions and environments and the authors are eager to develop their own strategies using their platforms and resources.