Abstract Natural language processing (NLP) tools have sparked a great deal of interest due to rapid improvements in information and communications technologies. As a result, many different NLP tools are being produced. However, there are many challenges for developing efficient and effective NLP tools that accurately process natural languages. One such tool is part of speech (POS) tagging, which tags a particular sentence or words in a paragraph by looking at the context of the sentence/words inside the paragraph. Despite enormous efforts by researchers, POS tagging still faces challenges in improving accuracy while reducing false-positive rates and in tagging unknown words. Furthermore, the presence of ambiguity when tagging terms with different contextual meanings inside a sentence cannot be overlooked. Recently, Deep learning (DL) and Machine learning (ML)-based POS taggers are being implemented as potential solutions to efficiently identify words in a given sentence across a paragraph. This article first clarifies the concept of part of speech POS tagging. It then provides the broad categorization based on the famous ML and DL techniques employed in designing and implementing part of speech taggers. A comprehensive review of the latest POS tagging articles is provided by discussing the weakness and strengths of the proposed approaches. Then, recent trends and advancements of DL and ML-based part-of-speech-taggers are presented in terms of the proposed approaches deployed and their performance evaluation metrics. Using the limitations of the proposed approaches, we emphasized various research gaps and presented future recommendations for the research in advancing DL and ML-based POS tagging. 

/pdf/part-of-speech-tagging-a-systematic-review-of-deep-learning-3i0scxua.pdf

Part of speech tagging: a systematic review of deep learning and machine learning approaches

The vast amount of unstructured data spread on a daily basis rises the need for developing effective information retrieval and extraction methods. Named Entity Recognition is a challenging classification task for structuring data into pre-defined labels, and is even more complicated when being applied on the Arabic language due to its special traits and complex nature. This article presents a novel Deep Learning approach for Standard Arabic Named Entity Recognition that proved its out-performance when being compared to previous works. The main aim of building a new model is to provide better fine-grained results for use in the Natural Language Processing fields. In our proposed methodology we utilized transfer learning with deep neural networks to build a Pooled-GRU model combined with the Multilingual Universal Sentence Encoder. Our proposed model scored about 17% enhancement when being compared to previous work.

Transfer Learning for Arabic Named Entity Recognition With Deep Neural Networks

Real-time prediction of domain names that are generated using the Domain Generation Algorithms (DGAs) is a challenging cyber security task. Scope to collect the vast amount of data for training favored data-driven techniques and deep learning architectures have the potential to address this challenge. This paper proposes a deep learning framework using long short-term memory (LSTM) architecture for prediction of the domain names that are generated using the DGAs. Binary classification had benign and DGA domain names and multiclass classification was performed using 20 different DGAs. For the binary classification, LSTM model gave accuracy of 98.7% and 71.3% on two different test data sets and for the multi-class classification, it gave accuracy of 68.3% and 67.0% respectively. Two diversified data sets were used to analyze the robustness of the LSTM architecture.

Deep Learning Framework for Domain Generation Algorithms Prediction Using Long Short-term Memory

/pdf/part-of-speech-tagging-a-systematic-review-of-deep-learning-177mxppz.pdf

Cybercriminals use domain generation algorithms (DGAs) to prevent their servers from being potentially blacklisted or shut down. Existing reverse engineering techniques for DGA detection is labor intensive, extremely time-consuming, prone to human errors, and have significant limitations. Hence, an automated real-time technique with a high detection rate is warranted in such applications. In this article, we present a novel technique to detect randomly generated domain names and domain name system (DNS) homograph attacks without the need for any reverse engineering or using nonexistent domain (NXDomain) inspection using deep learning. We provide an extensive evaluation of our model over four large, real-world, publicly available datasets. We further investigate the robustness of our model against three different adversarial attacks: DeepDGA, CharBot, and MaskDGA. Our evaluation demonstrates that our method is effectively able to identify DNS homograph attacks and DGAs and also is resilient to common evading cyberattacks. Promising results show that our approach provides a more effective detection rate with an accuracy of 0.99. Additionally, the performance of our model is compared against the most popular deep learning architectures. Our findings highlight the essential need for more robust detection models to counter adversarial learning. 

Adversarial Defense: DGA-Based Botnets and DNS Homographs Detection Through Integrated Deep Learning

Part of Speech (POS) tagging is the most fundamental task in various natural language processing(NLP) applications such as speech recognition, information extraction and retrieval and so on. POS tagging involves annotation of appropriate tag for each token in the corpus based on its context and the syntax of the language. In computational linguistics, optimal POS tagger is of paramount importance since tagging errors can critically affect the performance of the complex NLP systems. Developing an efficient POS tagger for morphologically rich languages like Nepali is a challenging task. In this paper, a deep learning based POS tagger for Nepali text is proposed which is built using Recurrent Neural Network (RNN), Long Short-Term Memory Networks (LSTM), Gated Recurrent Unit (GRU) and their bidirectional variants. Performance metrics such as accuracy, precision, recall and F1-score were chosen for the model evaluation. It is observed from the results that our model shows significant improvement and outperforms the state-of-art POS taggers with more than 99% accuracy.

A Deep Learning Approach for Part-of-Speech Tagging in Nepali Language

In recent days, malicious authors use domain generation algorithms so that they can easily evade blacklisting and heuristics mechanism. DGAs is used by a larger number of malware families to generate many pseudo-random domain names to connect to C2 server. In this paper, the deep neural network is employed along with 3-gram representation to transform the domain names into a numeric representation. Deep neural networks have a certain level of complexity since it uses sophisticated mathematical modeling to process data. The network parameters and network 3-gram representation is used to transform the domain names into a numeric representation. The network parameters and network structures for DNN are selected by following the hyperparameter selection method. All experiments are run until one hundred times with learning rate inside the range [0.01–0.5]. The experiments of DNN are run on DGA corpus given by DMD-2018 shared task organizer.

Detecting DGA Using Deep Neural Networks (DNNs)

Named Entity Recognition (NER) involves finding and categorizing minute text components into pre- defined categories such as name of person, location etc. NER is a type of information extraction task which has a crucial role in improving the performance of various NLP applications. For a morphologically abundant Semitic language like Arabic, the NER task is highly challenging due to its unique morphological characteristics and peculiarities. This paper introduces a deep learning based approach for Arabic NER which make use of well-known deep neural network (DNN) architectures like Recurrent neural network (RNN), Long short term memory (LSTM), Gated recurrent unit (GRU), stacked and bidirectional versions of these three architectures. ANERcorp dataset is used for the evaluation of the Arabic NER model and Accuracy is chosen as the performance metric. On model evaluation, it is observed that bidirectional variants of DNNs provide better accuracy measures compared to their unidirectional variants.

K. K. Shahina

Papers

A Deep Learning Approach for Part-of-Speech Tagging in Nepali Language

Detecting DGA Using Deep Neural Networks (DNNs)

A Sequential Labelling Approach for the Named Entity Recognition in Arabic Language Using Deep Learning Algorithms

Detecting DGA using deep neural networks (DNNs)