Abstract Natural language processing (NLP) tools have sparked a great deal of interest due to rapid improvements in information and communications technologies. As a result, many different NLP tools are being produced. However, there are many challenges for developing efficient and effective NLP tools that accurately process natural languages. One such tool is part of speech (POS) tagging, which tags a particular sentence or words in a paragraph by looking at the context of the sentence/words inside the paragraph. Despite enormous efforts by researchers, POS tagging still faces challenges in improving accuracy while reducing false-positive rates and in tagging unknown words. Furthermore, the presence of ambiguity when tagging terms with different contextual meanings inside a sentence cannot be overlooked. Recently, Deep learning (DL) and Machine learning (ML)-based POS taggers are being implemented as potential solutions to efficiently identify words in a given sentence across a paragraph. This article first clarifies the concept of part of speech POS tagging. It then provides the broad categorization based on the famous ML and DL techniques employed in designing and implementing part of speech taggers. A comprehensive review of the latest POS tagging articles is provided by discussing the weakness and strengths of the proposed approaches. Then, recent trends and advancements of DL and ML-based part-of-speech-taggers are presented in terms of the proposed approaches deployed and their performance evaluation metrics. Using the limitations of the proposed approaches, we emphasized various research gaps and presented future recommendations for the research in advancing DL and ML-based POS tagging. 

/pdf/part-of-speech-tagging-a-systematic-review-of-deep-learning-3i0scxua.pdf

Part of speech tagging: a systematic review of deep learning and machine learning approaches

/pdf/part-of-speech-tagging-a-systematic-review-of-deep-learning-177mxppz.pdf

In linguistics, morphology is a study regarding word, word formation, its analysis, and generation. A morphological analyzer is a tool to understand grammatical characteristics and constituent’s part-of-speech information. A morphological analyzer is a useful tool in many NLP implementations such as syntactic parser, spell checker, information retrieval, and machine translation. Here, 328 Sanskrit words are tested through four morphological analyzers namely—Samsaadhanii, morphological analyzers by JNU and TDIL, both of which are available online and locally developed and installed Sanguj morphological analyzer. There is a negligible divergence in the reflected results.

Peer Analysis of “Sanguj” with Other Sanskrit Morphological Analyzers

Rule based approach for compound segmentation and paraphrase generation in Sanskrit

Language is the most fundamental and historically normal means of communication today. Grammar plays a critical role in the excellence of a language. As individuals have already been educated throughout our existence with an accumulation of knowledge that is accrued, mastered over time with guidelines and a restriction of significance that allows us to comprehend and interact one another. But also to translate such awareness into a computer, to be capable of interpreting and classifying contextual evidence into a proper syntactical form, thereby validating that the information was in the correct form, is incredibly necessary at the current time since it is a sophisticated activity. The paper addresses the issue and asserts the advancement of such grammar verifying mechanism for the Dravidian language Kannada. Among the first account would be that the intricacy of the language poses a problem and preferring to have a rule based stance is an easier route and makes it possible to identify detected flaws competently. It takes a linguistic specialist to compile hundreds of parallel standards that are difficult to preserve. Here, a model is advocated that employs a deep learning method to train the LSTM (Long Short Term Memory) neural network trained over a massive data set to fulfill the necessary categorisation, using a context-based retention of the data attained through Word2Vec along with the TensorFlow and Keras packages. The proposed system is able to perform Grammatical Error Detection (GED) effectively.

Kannada Grammar Checker Using LSTM Neural Network

In this paper, we present a deep learning based approach to assign POS tags to words in a piece of text given to it as input. We propose an unsupervised approach owing to the lack of a large Sanskrit annotated corpora and use the untagged Sanskrit Corpus prepared by JNU for our purpose. The only tagged corpora for Sanskrit is created by JNU which has 115,000 words which are not sufficient to apply supervised deep learning approaches. For the tag assignment purpose and determining model accuracy, we utilize this tagged corpus. We explore various methods through which each Sanskrit word can be represented as a point multi-dimensional vector space whose position accurately captures its meaning and semantic information associated with it. We also explore other data sources to improve performance and robustness of the vector representations. We use these rich vector representations and explore autoencoder based approaches for dimensionality reduction to compress these into encodings which are suitable for clustering in the vector space. We experiment with different dimensions of these compressed representations and present one which was found to offer the best clustering performance. For modelling the sequence in order to preserve the semantic information we feed these embeddings to a bidirectional LSTM autoencoder. We assign a POS tag to each of the obtained clusters and produce our result by testing the model on the tagged corpus.

Vrashabh Prasad Jain

Papers

Deep Learning Based Unsupervised POS Tagging for Sanskrit