On-line construction of suffix trees

Question answering is a subfield of information retrieval. It is a task of answering a question posted in a natural language. A question answering system (QAS) may be considered a good alternative to search engines that return a set of related documents. The QAS system is composed of three main modules; question analysis, passage retrieval, and answer extraction. Over the years, numerous QASs have been presented for use in different languages. However, the the development of Arabic QASs has been slowed by linguistic challenges and the lack of resources and tools available to researchers. In this survey, we start with the challenges due to the language and how these challenges make the development of new Arabic QAS more difficult. Next, we do a detailed review of several Arabic QASs. This is followed by an in-depth analysis of the techniques and approaches in the three modules of a QAS. We present an overview of important and recent tools that were developed to help the researchers in this field. We also cover the available Arabic and multilingual datasets, and a look at the different measures used to assess QASs. Finally, the survey delves into the future direction of Arabic QAS systems based on the current state-of-the-art techniques developed for question answering in other languages.

Arabic question answering system: a survey

A Conversational Agent (CA), or dialogue system, is a computer system that has the ability to respond to humans automatically using natural language. CAs offer instant responses and can concurrently assist a potentially unlimited number of users. The modeling of CAs in Arabic has so far received less attention when compared with other languages due to the complexity of the Arabic language, the existence of several dialects, and a lack of data resources. The literature contends that modeling a CA in Arabic mostly done using pattern-matching and information retrieval, employing classification approaches with a closed-domain data source. There is extremely limited research so far on modeling an open-domain CA in the Arabic dialect. This research has utilized a deep-learning architecture, known as the Seq2Seq neural network, to build a CA in the Arabic Gulf dialect. We formulated the CA problem as a machine translation problem and, therefore, built our corpus from tweets, in the post-reply format, to train and evaluate the model. We investigated the effects of pretrained embeddings on the performance of the CA. For our evaluation, a Bilingual Evaluation Understudy (BLEU) score and human evaluators were used. The performance of the model was found to be comparable with existing deep learning models that have been trained on much larger corpora and in other languages. Our results present a promising first step towards building an open-domain CA in the Gulf Arabic dialect.

/pdf/a-seq2seq-neural-network-based-conversational-agent-for-gulf-2ronudzky0.pdf

A seq2seq Neural Network based Conversational Agent for Gulf Arabic Dialect

Question-answering (QA) systems aim to provide answers for given questions. The answers can be extracted or generated from either unstructured or structured text. Therefore, QA is considered an important field that can be used to evaluate machine text understanding. Arabic is a challenging language for many reasons; although it is spoken by more than 330 million native speakers, research on this language is limited. A few QA systems created for Arabic text are available. They were created to experiment on small datasets, some of which are unavailable. The research on QA systems can be expanded into different components of QA systems, such as question analysis, information retrieval, and answer extraction. The objective of this research is to analyze the QA systems created for Arabic text by reviewing, categorizing, and analyzing the gaps by providing advice to those who would like to work in this field. Six benchmark datasets are available for testing and evaluating Arabic QA systems, and 26 selected Arabic QA systems are analyzed and discussed in this research.

/pdf/arabic-question-answering-systems-gap-analysis-ooxi1vrvmx.pdf

Arabic Question Answering Systems: Gap Analysis

The interchangeably connected Web technologies and the advancements that accompany the semantic web content’s leaps, have raised many challenges in the results’ retrieval process especially for the Arabic Language. This research targets an important, yet insufficiently precedent, area in using Linked Open Data (LOD) for Automatic Question Answering systems in the Arabic Language. The significance of work presented, comes from its ability to overcome many challenges in querying Arabic content. Some of these challenges are: (a) bridging the gap between natural language and linked data by mapping users’ queries to a standard semantic web query language such as SPARQL, (b) facilitating multilingual access to semantic data, and (c) maintaining the quality of data. Another challenging aspect was the lack of related work and publicly available resources for Arabic Question Answering Systems over Linked Data, despite the vastly growing Arabic corpus on the web. This paper presents a novel approach that targets Automatic Arabic Questions’ Answering Systems whilst bypassing many featured challenges in the field. A hybrid approach that evaluates the effectiveness of using LOD to automatically answer Arabic questions is developed. The approach is developed to map users’ questions in Modern Standard Arabic, to a standard query language for LOD (i.e. SPARQL) through: (i) extracting entities from questions and linking them over the web using Named-Entity Recognition and Disambiguation (NER/NED), and (ii) extracting properties among extracted named entities using a dependency parsing approach integrated with Wikidata ontology. To evaluate our proposed system, an Arabic questions dataset was created including: (a) Question body in Arabic language, (b) Question type, (c) SPARQL Query formulation, and (d) Question answer. Evaluation results are promising with a Precision of 84%, a Recall of 81.3%, and an F-Measure of 82.8%.

/pdf/leveraging-linked-open-data-to-automatically-answer-arabic-3o55edguk4.pdf

Leveraging Linked Open Data to Automatically Answer Arabic Questions

Building Arabic Question Answering systems is a challenging problem compared to their English counterparts due to several limitations inherent in the Arabic language and the scarceness of available Arabic training datasets. In our proposed Arabic Question Answering system, we combine several previously successful algorithms and add a novel approach to the answer extraction process that has not been used by any Arabic Question Answering system before. We use the state-of-the-art MADAMIRA Arabic morphological analyser for preprocessing questions and retrieved passages. We also enhance and extend the question classification and use the Explicit Semantic Approach (ESA) in the passage retrieval process to rank passages that most probably contain the correct answer. We also introduce a new answer extraction pattern, which matches the patterns formed according to the question type with the sentences in the retrieved passages in order to provide the correct answer. A performance evaluation study shows that our system gives promising results compared to other existing Arabic Question Answering systems, especially with the newly introduced answer extraction

AlQuAnS - An Arabic Language Question Answering System.

The challenges of the Arabic language and the lack of resources have made it difficult to provide Arabic Question Answering (QA) systems with high accuracy. These challenges motivated us to propose AlQuAnS-an Arabic Language Question Answering System that gives promising accuracy results. This paper proposes a modified version of AlQuAnS with a higher accuracy. The proposed system enhances the accuracy of the question classification, semantic interpreter and answer extraction modules. The provided performance evaluation study shows that our modified system outperforms other existing Arabic QA systems, especially with the newly introduced answer extraction module.

Ghada Khairy

Papers

AlQuAnS - An Arabic Language Question Answering System.

A Modified Version of AlQuAnS: An Arabic Language Question Answering System