Automatic multiple choice question (MCQ) generation from a text is a popular research area. MCQs are widely accepted for large-scale assessment in various domains and applications. However, manual generation of MCQs is expensive and time-consuming. Therefore, researchers have been attracted toward automatic MCQ generation since the late 90's. Since then, many systems have been developed for MCQ generation. We perform a systematic review of those systems. This paper presents our findings on the review. We outline a generic workflow for an automatic MCQ generation system. The workflow consists of six phases. For each of these phases, we find and discuss the list of techniques adopted in the literature. We also study the evaluation techniques for assessing the quality of the system generated MCQs. Finally, we identify the areas where the current research focus should be directed toward enriching the literature.

Automatic Multiple Choice Question Generation From Text: A Survey

Machine learning are having a tremendous impact on the teaching industry. Teaching industry is adopting new technologies to predict the future of education system. It is Machine learning which predict the future nature of education environment by adapting new advanced intelligent technologies. This work explores the application of Machine Learning in teaching and learning for further improvement in the learning environment in higher education. We explore the application of machine learning in customized teaching and learning environment and explore further directions for research. Customized teaching and learning consider student background, individual student aptitude, learning speed and response of each student. This customized teaching and learning approach provide feedback to teacher after real-time processing of the data. This way a teacher can easily recognize student attention and take corrective measures. This will improve student participation and hence the overall results. Individual student concepts and goals can easily be track with the help of Machine learning by taking real-time feedback. Based on that feedback, curriculum, topics and methodology can be improved further. In simple terms, machine learning makes the process automatic for decision making process and analyzed the individual student data. Overall, the assessment process is made more streamlined, accurate and unbiased with the help of machine learning. In the near future, machine learning will be more efficient and produce even better results.

Applications of Machine Learning in Improving Learning Environment

Assessment plays an important role in learning and Multiple Choice Questions (MCQs) are quite popular in large-scale evaluations. Technology-enabled learning necessitates a smart assessment. Therefore, automatic MCQ generation became increasingly popular in the last two decades. Despite a large amount of research effort, system generated MCQs are not useful in real educational applications. This is because of the inability to produce diverse and human-alike distractors. Distractors are the wrong choices given along with the correct answer (key) to befuddle the examinee. In several domains, the MCQs deal with names or named entities. However, existing literature is not adequate in generating quality named entity distractors. In this paper, we present a method for automatic generation of named entity distractors. The technique uses a combination of statistical and semantic similarities. To compute the statistical similarity, a set of class-specific attributes are defined are their values are extracted from the web. Semantic similarity is computed using a predicate-argument extraction based method. The proposed technique is tested in cricket domain because of the availability of a large number of web resources and MCQs for dataset preparation. An evaluation strategy is proposed along with a set of metrics. A set of human evaluators performed the evaluation and they found that the average closeness value of the distractors as 2.3. This value indicates that 2.3 out of 3 system-generated distractors are as good as human-generated distractors. Two good distractors make an MCQ usable in real assessment. So, the proposed technique is capable of generating high-quality distractors.

A hybrid approach for automatic generation of named entity distractors for multiple choice questions

The key objective of the teaching-learning process (TLP) is to impart the knowledge to the learner. In the digital world, the computer-based system emphasis teaching through online mode known as e-learning. The expertise level of the learner in learned subjects can be measured through e-assessment in which multiple choice questions (MCQ) is considered to be an effective one. The assessment questions play the vital role which decides the ability level of a learner. In manual preparation, covering all the topics is difficult and time consumable. Hence, this article proposes a system which automatically generates two different types of question helps to identify the skill level of a learner. First, the MCQ questions with the distractor set are created using named entity recognizer (NER). Further, based on blooms taxonomy the Subjective questions are generated using natural language processing (NLP). The objective of the proposed system is to generate the questions dynamically which helps to reduce the occupation of memory concept.

Developing the Assessment Questions Automatically to Determine the Cognitive Level of the E-Learner Using NLP Techniques

Multiple-choice question (MCQ) plays a significant role in educational assessment. Automatic MCQ generation has been an active research area for years, and many systems have been developed for MCQ generation. Still, we could not find any system that generates accurate MCQs from school-level textbook contents that are useful in real examinations. This observation motivated us to develop a system that generates MCQs to assess the student's recall of factual information. Also, the available systems are often domain, subject, or application-specific in nature. Although the MCQ generation task demands a specific setup, we expect a level of generalization can be achieved. In this development, we also focus on this issue. We propose a pipeline for automatic generation of MCQs from textbooks of middle-school level subjects, and the pipeline is partially subject-independent. The proposed pipeline comprises four core modules: preprocessing, sentence selection, key selection, and distractor generation. Several techniques have been employed to implement individual modules. These include sentence simplification, syntactic and semantic processing of the sentences, entity recognition, semantic relationship extraction among entities, WordNet, neural word embedding, neural sentence embedding, and computation of intersentence similarity. The system is evaluated using the National Council of Educational Research and Training (NCERT), India, textbooks for three subjects. The quality of system-generated questions is assessed by human experts using various system-level and individual module-level metrics. The experimental results demonstrate that the proposed system is capable of generating quality questions that could be useful in a real examination.

Generation of Multiple-Choice Questions From Textbook Contents of School-Level Subjects

This research introduces an automatic multiple choice question generation system to evaluate the understanding of the semantic role labels and named entities in a text. The system provided selects the informative sentence and the keyword to be asked based on the semantic labels and named entities that exist in the sentence, the distractors are chosen based on a similarity measure between sentences in the data set. The system is tested using a set of sentences extracted from the TREC 2007 dataset for question answering. From the experimental results, it can be induced that the semantic role labeling and named entity recognition approaches could be used as   a good keyword selection mechanism. The second conclusion is that the string similarity measures proved to be a very good approach that can used in generating the distractors for an automatic multiple choice question. Also, combining the similarity measures of different algorithms would lead to generate a good distractors.

/pdf/automatic-multiple-choice-question-generation-system-for-23elfpyuch.pdf

Automatic Multiple Choice Question Generation System for Semantic Attributes Using String Similarity Measures

This research introduces a semantic based Automatic Question Generation System using both Semantic Role Labeling and Named Entity Recognition techniques to convert the input sentence into a semantic pattern. A training phase applied to build a classifier using an Artificial Immune System that will be able to classify the patterns according to the question type. The question types considered here are set of WH-questions like who, when, where, why, and how. Then a pattern matching phase is applied to select the best matching question pattern for the test sentence. The proposed system is tested against a set of sentences obtained from different sources like Wikipedia articles, TREC 2007dataset for question answering, and English book of grade II prep. The proposed model shows promising results in determining the question type with classification accuracy increases 95%, and also in generating (matching) the new question patterns with 87%.

/pdf/semantic-based-automatic-question-generation-using-1p6u5s7gjk.pdf

Ibrahim Eldesoky Fattoh

Papers

Automatic Multiple Choice Question Generation System for Semantic Attributes Using String Similarity Measures

Semantic Based Automatic Question Generation using Artificial Immune System