LANA-I: An Arabic Conversational Intelligent Tutoring System for Children with ASD
Abstract: Children with Autism Spectrum Disorder (ASD) share certain difficulties but being autistic will affect them in different ways in terms of their level of intellectual ability. Children with high functioning autism or Asperger syndrome are very intelligent academically but they still have difficulties in social and communication skills. Many of these children are taught within mainstream schools but there is a shortage of specialised teachers to deal with their specific needs. One solution is to use a virtual tutor to supplement the education of children with ASD in mainstream schools. This paper describes research to develop a novel Arabic Conversational Intelligent Tutoring System, called LANA-I, for children with ASD that adapts to the Visual, Auditory and Kinaesthetic learning styles model (VAK) to enhance learning. This paper also proposes an evaluation methodology and describes an experimental evaluation of LANA-I. The evaluation was conducted with neurotypical children and indicated promising results with a statistically significant difference between user’s scores with and without adapting to learning style. Moreover, the results show that LANA-I is effective as an Arabic Conversational Agent (CA) with the majority of conversations leading to the goal of completing the tutorial and the majority of the correct responses (89%).
Summary (4 min read)
- The number of children being diagnosed with autism spectrum disorder (ASD) is increasing .
- In addition, traditional education using human tutors is a challenge for students with autism, who have difficulties in communication and social © Springer Nature Switzerland AG 2019 K. Arai et al. (Eds.): CompCom 2019, AISC 997, pp. 498–516, 2019.
- The main contributions in this paper are: A novel architecture for an Arabic CITS using VAK model for an appropriate education scenario. .
- This paper is organized as follows: Sect. 2 describes the architecture and methodology of implementing LANA-I.
- Section 4 provides the results and discussion.
2 LANA-I Architecture and Implementation
- LANA-I was developed based on two phases.
- The proposed framework for the Arabic CA consists of five components as shown in Fig. 1: 1. Graphical User Interface: Responsible for the communication between the user and CA (in this case the CITS tutor) through a web interface with panels to display supporting material.
- Responsible for controlling and directing the conversation through contexts, also known as 3. Conversation Agent Manager.
- The ITS manager, which is the main component that interacts with the user through the GUI, and personalises the tutorial based on the user’s learning style, also known as 2. Tutor Model.
- The proposed LANA-I CITS architecture, shown in Fig. 3 contains three main components that are described in the following sections: email@example.com.
2.1 The Knowledge Base
- The knowledge base consists of four sub-components: (1) the Tutorial Knowledge base, (2) Arabic general contexts (e.g. weather, and greetings), (3) user’s profile, and (4) the log file.
- A short interview was conducted to design the knowledge base with three primary school teachers in Saudi Arabia, who teach Science.
- The LANA-I knowledge base consists of two contexts: the domain, which is the science tutorials, and a general context, illustrated in Fig.
- General contexts deal with general conversation that is not related to the domain, such as weather, greeting, rude words, and user leaving words (any words or sentences means that the user will leave the conversation).
- Once the tutorial was designed and approved by the primary school teachers then the tutorial questions were mapped to the VAK learning style model.
2.2 Arabic Scripting Language
- The three different approaches to develop an Arabic CA and a number of challenges faced by Arabic language were discussed in survey paper .
- It was concluded that there is a lack of Arabic NLP resources leading to limited capabilities within Arabic CAs.
- InfoChat was designed using English scripting language and based on the pattern matching (PM) technique, where the domain is organised into a number of contexts and each context contains rules, each rule in the domain contains a number of patterns and a response that forms the CA output to the user.
- Based on the scripting methodology reported by Latham  the procedures to create the scripts within the Knowledge base are: 1. The methodology for scripting each context is: Create a context table, which has a record with a unique name to represent that context . .
- For each rule, create patterns that match user utterances.
2.3 Scripting Arabic CA for LANA-I
- In LANA-I, the tutorial topics were represented as the contexts and the agent’s questions for such topic were represented as the rules, while the pattern represent the user’s utterances, which belong to such a rule.
- The scripting language in LANA-I includes the following features: Provide supporting material to the user depending on the user’s learning style (Visual: images and videos – Auditory: Sounds – Kinaesthetic: Instructions and objects).
- All images, videos and audios provide the right answer.
- When the user is visual learner, the rule is fired with the video or image material.
- Figure 5 shows the models that are used with the Kinaesthetic learning style.
2.4 The Arabic Conversational Agent
- The second component of LANA-I (Fig. 3) is the Arabic CA.
- The controller then checks if the conversation is within the tutorial scenario or not by communicate with the CA.
- The Conversational Agent Engine contains a combination of methods of string similarity and PM approaches to determine the similarity between two sets of strings within CA’s, while traditional CA’s used only a PM approach that involves a strength calculation through different aspects of the user utterance and the scripted pattern such as activation level and number of words, etc.
- In PM technique, the user utterance will be matched to the stored patterns; these patterns contain wildcard firstname.lastname@example.org characters to represent any number of words of characters.
- The similarity between two pieces of text is determined by representing each piece of text in the form of word vector.
2.5 The Workflow of LANA-I CA Engine
- In the beginning, the PM Wildcard will be used to match the user utterance with the patterns stored in the database.
- If the match is not found, the STS Cosine similarity will be applied.
- Assume the pattern stored in the LANA-I database was (S1), while the user utterance was (S2), as shown in Table 4: The utterance is not recognised by the PM approach because of the word order or minor differences from the pattern.
- Therefore, the system applies the Cosine similarity, which is illustrated in the following steps: email@example.com Create Matrix where the columns are the unique words from S1 and S2, and the rows are the words sequence of S1 and S2.
- When the user’s utterance is recognized by the similarity measure, the corresponding response will be generated and delivered to the user.
2.6 LANA-I ITS
- The third component in the LANA-I architecture (Fig. 3) contains: The Graphics User Interface (GUI), and the ITS manager.
- This character appears in all the system interfaces to make the conversation more natural and engaging for the users.
- The LANA character was designed by the author and then evaluated by primary school teachers in Saudi Arabia in order to make the tutorial more engaging.
- There were three questions focused on Smith’s visual, auditory, and kinaesthetic styles (VAK).
- For each question, the pupils had to respond ‘yes’ or ‘no’ to each question.
2.7 LANA-I Workflow
- This section describes the LANA-I workflow from perspectives of teacher and the pupil in order to understand how each activity communicates with others.
- Initially LANA-I starts from the learning style questionnaire, which is taken by the teacher.
- After completing this stage, the system shows the pre-test interface, and asks the user to complete the test.
- The next interface after submitting the test is the CA tutorial, The ITS manager is responsible in this stage for personalising the tutoring session according to the user’s learning style by providing the CA components, through the controller, the suitable materials from the Knowledge base component.
- The ITS manager also saves the user’s registration information and the pre-test score in the log file/student’s profile.
3 Experimental Methodology
- The LANA-I prototype was tested through two main experiments to evaluate the system.
- The objective metrics were measured through the log file/temporal memory and the pre-test and post-test score.
- The subjective metrics were measured using the user feedback questionnaire.
- The main hypothesis of the experiments is: HA0: LANA-I using VAK model cannot be adapted the student learning style.
- The second experiment was conducted to test the Hypothesis B (LANA-I is an effective Arabic CA).
- The total size of the sample was 24 neurotypical participants within the age group (10– 12) years and their first language was Arabic.
- The participants recruited for the evaluation were residents of the Greater Manchester area within the UK and none of them had previous experience of using LANA-I.
- All participants’ parents received an information sheet about the project and its aims, and a consent form to obtain their permission before conducting the experiment.
- The participants were divided into two groups.
- The first group is a control group (n = 12), who used the LANA-I without adapting to the learning style VAK model as basis comparison.
3.2 Experiment 1: LANA-I Tutoring with and Without VAK Learning Style
- Subjective and objective metrics were used to answer the two questions related to Hypothesis A. Each group of participants was asked first to register into the system and complete the pre-test questions.
- They started the tutorial without the VAK questionnaire, whereas the experimental Group, who used the LANA-I with adapting to VAK learning style model, were asked to fill the VAK learning style questionnaire in order for LANA-I to be adapted to the learning style.
- After adapting the learning style, the tutorial provided the most suitable material during the session such as videos, images or firstname.lastname@example.org instructions and physical resources.
- When the session ended, both groups did the post-test questions in order to measure their learning gain.
3.3 Experiment 2: LANA-I CA System Robustness
- The data for this experiment was gathered from the LANA-I log file and the user feedback questionnaire whilst participants were completing experiment 1.
- The subjective and objective metrics were used to answer questions related to Hypothesis B.
- The data gathered from the log file allows assessment of the performance of LANA-I CA and the algorithms deployed in the architecture.
- This data will measure success using objective metrics.
- The data from user feedback questionnaire will be analysed in order to measure success using subjective metrics.
4 Results and Discussion
- The learning gain was measured using a pre-test and post-test approach [17–19].
- Average test score improvements were calculated and compared using the following formula: Relative learning gain ¼ PostTest PreTestð Þ= TotalScore PreTestð Þ ð4Þ Table 6 illustrates the results of the MannWhitney U test conducted in order to measure the relative learning gain between Control Group and Experimental Group.
- This result indicates that HA1 can be accepted.
- Further tests have been carried out to find out whether there was a significant difference in the scores for participant’s satisfaction with adapting the tutorial to the VAK learning style.
- The results illustrated that (57.18%) of all the utterances input by the users were actually different from the scripted patterns and in this case the system used the Short Text Similarity algorithm (Cosine algorithm).
- This paper outlined a novel Arabic CITS called LANA-I, which used the VAK learning style model to enhance the learning of children.
- The authors findings provide evidence for theses novel features: 1. LANA-I can be adapted to the VAK learning style for the tutorial.
- The first evaluation highlighted areas of weakness within LANA-I architecture.
- New methodologies will be researched and developed to overcome the spelling variations in the Arabic language, which affect the performance of the similarity algorithm.
- These weakness and further enhancements will be addressed by further research and development, to make the system ready for use with autistic pupils.
Did you find this useful? Give us your feedback
Cites background from "LANA-I: An Arabic Conversational In..."
...This paper is a part of our investigation within LANA CITS project [22, 23]....
...LANA CITS   is a novel Arabic CITS, which delivers topics related to the science subject by engaging with the user in Arabic language....
Related Papers (5)
Frequently Asked Questions (2)
Q1. What contributions have the authors mentioned in the paper "Aljameel, s, o’shea, j, crockett, k orcid logoorcid: https://orcid.org/0000-0003-1941-6201, latham, a and kaleem, m (2019) lana-i: an arabic conversational intelligent tutoring system for children" ?
This paper describes research to develop a novel Arabic Conversational Intelligent Tutoring System, called LANA-I, for children with ASD that adapts to the Visual, Auditory and Kinaesthetic learning styles model ( VAK ) to enhance learning. This paper also proposes an evaluation methodology and describes an experimental evaluation of LANA-I. The evaluation was conducted with neurotypical children and indicated promising results with a statistically significant difference between user ’ s scores with and without adapting to learning style.
Q2. What future works have the authors mentioned in the paper "Aljameel, s, o’shea, j, crockett, k orcid logoorcid: https://orcid.org/0000-0003-1941-6201, latham, a and kaleem, m (2019) lana-i: an arabic conversational intelligent tutoring system for children" ?
Further research is required to make components and algorithms within LANAI more robust and to achieve the main objective, which is developing an Arabic CITS for people with Autism. These weakness and further enhancements will be addressed by further research and development, to make the system ready for use with autistic pupils. Additional research is required as follows: • Further improvement to the knowledge base and the CA engine will be made based on the results of the first pilot study. New methodologies will be researched and developed to overcome the spelling variations in the Arabic language, which affect the performance of the similarity algorithm.