Intelligent tutoring systems have become increasingly common in assisting students but are often aimed at isolated subject-domain tasks without creating a scaffolding system from lower- to higher-level cognitive skills, with low-level skills often neglected. We designed and developed an intelligent tutoring system, CompPrehension, which aims to improve the comprehension level of Bloom’s taxonomy. The system features plug-in-based architecture, easily adding new subject domains and learning strategies. It uses formal models and software reasoners to solve the problems and judge the answers, and generates explanatory feedback about the broken domain rules and follow-up questions to stimulate the students’ thinking. We developed two subject domain models: an Expressions domain for teaching the expression order of evaluation, and a Control Flow Statements domain for code-tracing tasks. The chief novelty of our research is that the developed models are capable of automatic problem classification, determining the knowledge required to solve them and so the pedagogical conditions to use the problem without human participation. More than 100 undergraduate first-year Computer Science students took part in evaluating the system. The results in both subject domains show medium but statistically significant learning gains after using the system for a few days; students with worse previous knowledge gained more. In the Control Flow Statements domain, the number of completed questions correlates positively with the post-test grades and learning gains. The students’ survey showed a slightly positive perception of the system.

Improving Comprehension: Intelligent Tutoring System Explaining the Domain Rules When Students Break Them

https://robots.iopscience.iop.org/article/10.1088/1757-899X/483/1/012091/pdf

Method of lemmatizer selections in multiplexing lemmatization

The article discusses the problem of visualization of complex multiparameter systems, defined by datasets on their structure, functional structure, and activity in the form of complex graphs and transition of traditional representation of the data acquired by graph mining to a compact image built by pictographic methods. In these situations, we propose using the Unified Graphic Visualization of Activity (UGVA) method for data concentration and structuring. The UGVA method allows coding in an anthropomorphic image of elements of graphs with data on structural and functional features of systems and overlaying these images with the data on the system’s activity using coloring and artifacts. The image can be composed in different ways: it can include the zone of integral evaluation parameters, segmented data axes of five types, and four types of symmetry. We describe the method of creating UGVA images, which consists of 13 stages: the parametric model is represented as a structural image that is converted to a basic image that is then detailed into the particular image by defining geometric parameters of the primitives and to the individualized image with the data about a particular object. We show how the individualized image can be overlaid with the operative data as color coding and artifacts and describe the principles of interpreting UGVA images. This allows solving tasks of evaluation, comparison, and monitoring of complex multiparameter systems by showing the decision-maker an anthropomorphic image instead of the graph. We describe a case study of using the UGVA method for visualization of data about an educational process: curricula and graduate students, including the data mined from the university’s learning management system at the Siberian Federal University for students majoring in “informatics and computing”. The case study demonstrates all stages of image synthesis and examples of their interpretation for situation assessment, monitoring, and comparison of students and curricula. It allowed for finding problematic moments in learning for individual students and their entire group by analyzing the development of their competence profiles and formulating recommendations for further learning. The effectiveness of the resulting images is compared to the other approaches: elastic maps and Chernoff faces. We discuss using graph mining to generate learning problems in order to lessen the workload of gathering raw data for the UGVA method and provide general recommendations for using the UGVA method based on our experience of supporting decision making.

/pdf/evaluation-comparison-and-monitoring-of-multiparameter-2jbw4d2o.pdf

Evaluation, Comparison and Monitoring of Multiparameter Systems by Unified Graphic Visualization of Activity Method on the Example of Learning Process

The use of formal knowledge representation models in intelligent tutoring systems often requires logical reasoning on these models by predefined rules. This process can be time and memory consuming, so finding effective software reasoners for different applications is an important research field. This problem is relevant for cognitive and constraint-based intelligent tutoring systems. We performed a comparative study of various software reasoners (Pellet, Apache Jena inference subsystem, Apache Jena SPARQL query processor, SWI-Prolog with semweb package, Closed World Machine, and Answer Set Programming solvers Clingo and DLV) for solving tasks specific to intelligent tutoring systems using three formal models with different properties and corresponding rule sets created for intelligent tutoring systems in introductory programming education domain. We compared features of rule-definition formalisms for different approaches and measured run and wall time, average CPU load, and peak RAM usage based on the count of inferred RDF triples. The experiments show that Apache Jena infers the solution quicker than other reasoners on the majority of tasks but consumes a significant amount of memory, while Clingo performs significantly better for combinatorial problems.

Inference Engines Performance in Reasoning Tasks for Intelligent Tutoring Systems.

The paper describes the problem of creating a single mapping of the learning situation during the processing of digital footprint and decision making in intelligent tutoring systems. We propose creating a Cognitive Map of Knowledge Diagnosis as a way of summarising data in the digital footprint. The elements of this cognitive map and the details of their visualisation are described. The results of the use of cognitive maps are demonstrated on the example of personalising the content of an online learning course and providing detailed feedback.

Creating and Visualising Cognitive Maps of Knowledge Diagnosis During the Processing of Learning Digital Footprint.

Intelligent tutoring systems become increasingly common in assisting human learners, but they are often aimed at isolated domain tasks without creating a scaffolding system from lower- to higher-level cognitive skills. We designed and implemented an intelligent tutoring system CompPrehension aimed at the comprehension level of Bloom’s taxonomy that often gets neglected in favour of the higher levels. The system features plugin-based architecture, easing adding new domains and learning strategies; using formal models and software reasoners to solve the problems and judge the answers; and generating explanatory feedback and follow-up questions to stimulate the learners’ thinking. The architecture and workflow are shown. We demonstrate the process of interacting with the system in the Control Flow Statements domain. The advantages and limits of the developed system are discussed.

CompPrehension - Model-Based Intelligent Tutoring System on Comprehension Level

The article describes the process of developing an automated tool to determine the evaluation order in the expression on C programming language and describe the faults made by a student in this task. The main part of this tool is based on the SWRL rule set, which can be divided into several different tasks: calculation of the evaluation order of expression based on constructing abstract syntax tree of expression, determine contradictions with the partial student’s evaluation order, and describing fault reasons.

/pdf/ontology-based-determining-of-evaluation-order-of-c-1or0l0sjcr.pdf

Ontology-based determining of Evaluation Order of C expressions and the Fault Reason for Incorrect Answers.

Inflectors – the programs that generate different forms of the given lemma – are important part of natural language processing, especially natural language generation. They, along with lemmatizers – the programs that determine the lemma of the word – also can be used in teaching grammar. However, writing an efficient inflector is not an easy task and many modern applications have deficiencies. The article compares several modern open source inflectors using samelemma detection method, which defines if inflector could be used to identify, that two different words belong to the same lemma.

/pdf/a-study-of-efficiency-of-modern-inflection-and-lemmatization-3gg4smkuh1.pdf

A Study of Efficiency of Modern Inflection and Lemmatization Software

When developing automated assessments and intelligent tutoring systems, a lot of routine effort goes into developing the bank of learning problems. Problem generation is the way to automate this process. In this paper, we present a method of generating expression-related problems for teaching introductory programming courses. The problems are generated from open-source software code which allows keeping learning problems similar to the production code the students should learn to analyze and write. Generated problems are automatically classified by their difficulties and the knowledge they need to solve, represented as sets of possible errors. This allows seamless integration with adaptive learning algorithms. The evaluation showed that the generated problems are indistinguishable from human-authored problems and suitable for use in the educational process.

Nikita Penskoy

Papers

Improving Comprehension: Intelligent Tutoring System Explaining the Domain Rules When Students Break Them

CompPrehension - Model-Based Intelligent Tutoring System on Comprehension Level

Ontology-based determining of Evaluation Order of C expressions and the Fault Reason for Incorrect Answers.

A Study of Efficiency of Modern Inflection and Lemmatization Software

Generating Expression Evaluation Learning Problems from Existing Program Code