Though naive Bayes text classifiers are widely used because of its simplicity, the techniques for improving performances of these classifiers have been rarely studied. In this paper, we propose and evaluate some general and effective techniques for improving performance of the naive Bayes text classifier. We suggest document model based parameter estimation and document length normalization to alleviate the problems in the traditional multinomial approach for text classification. In addition, Mutual-Information-weighted naive Bayes text classifier is proposed to increase the effect of highly informative words. Our techniques are evaluated on the Reuters21578 and 20 Newsgroups collections, and significant improvements are obtained over the existing multinomial naive Bayes approach.

Effective methods for improving naive Bayes text classifiers

In recent years, sentiment analysis is becoming more and more important as the number of digital text resources increases in parallel with the development of information technology. Feature selection is a crucial sub-stage for the sentiment analysis as it can improve the overall predictive performance of a classifier while reducing the dimensionality of a problem. In this study, we propose a novel wrapper feature selection algorithm based on Iterated Greedy (IG) metaheuristic for sentiment classification. We also develop a selection procedure that is based on pre-calculated filter scores for the greedy construction part of the IG algorithm. A comprehensive experimental study is conducted on commonly-used sentiment analysis datasets to assess the performance of the proposed method. The computational results show that the proposed algorithm achieves 96.45% and 90.74% accuracy rates on average by using Multinomial Naive Bayes classifier for 9 public sentiment and 4 Amazon product reviews datasets, respectively. The results also reveal that our algorithm outperforms state-of-the-art results for the 9 public sentiment datasets. Moreover, the proposed algorithm produces highly competitive results with state-of-the-art feature selection algorithms for 4 Amazon datasets.

A novel wrapper feature selection algorithm based on iterated greedy metaheuristic for sentiment classification

Automatic text categorization is the operation of sorting out the text documents into pre-defined text categories using some machine learning algorithms. Normally, it defines the most important approaches to organizing and making the use of a large volume of information exists in unstructured form. Nowadays, text categorization is becoming an extensively researched field of text mining and processing of languages. Word sense, semantic relationships among terms, text documents and categories are quite essential in order of enhancing the performances of categorization. Various surveys on text categorization have already been available which involve techniques of various text representation schemes to such extent but do not include several approaches that have been explored in text categorization over the standard techniques. Here, an exhaustive analysis of different text categorization approaches over the conventional approaches has been undertaken. This survey paper explores a wide variety of algorithms used for categorizing text documents and tries to assemble the existing works into three basic fields: conventional methods, fuzzy logic-based methods, deep learning-based methods. Further, conventional methods have been categorized into three fields: text categorization using handcrafted features, text categorization using nature-inspired algorithms and text categorization using graph-based methods. Furthermore, this survey provides a clear idea about the available libraries used for different algorithms, availability of datasets, categorization technologies explored in various non-Indian and Indian languages as well.

Text categorization: past and present

In this study, we used SVM for text classification. There is stemming or Iemmatization word normalization with the addition of Chi-squarefeature selection on the classification that we made. There are also pre-processing data being performed, namely stopwords removal and tokenize. We used BBC dataset containing 2,225 documents and 5 categories. There are 21,813. features resulting from the use of stemming and 31,007 features resulting from the use of lemmatization. Each feature represents the number of words that come out in the document. We used confusion matrix to evaluate the results of text clasification. SVM text classification performance using stemming enhanced by Chi-squared (method 1) get better results than using lemmatization enhanced by Chi-squared (method 2). The best performance was obtained using 80% feature reduction where method 1 received a precision value of 95%, a recall value of 95%, and an accuracy value of 95.05%. Method 2 only received a precision value of 93%, a recall value of 93%, and an accuracy value of 93.24% using the same amount of feature reduction.

Influence of Word Normalization and Chi-Squared Feature Selection on Support Vector Machine (SVM) Text Classification

FCTFS works in both autonomous and user guided mode.The defined taxonomy helps in arriving at optimal number of good quality clusters.Feature elimination due to irrelevance and redundancy is clearly isolated.It is faster than traditional search based methods.Yields superior results compared to some state of the art methods over 24 data sets. Feature subset selection is basically an optimization problem for choosing the most important features from various alternatives in order to facilitate classification or mining problems. Though lots of algorithms have been developed so far, none is considered to be the best for all situations and researchers are still trying to come up with better solutions. In this work, a flexible and user-guided feature subset selection algorithm, named as FCTFS (Feature Cluster Taxonomy based Feature Selection) has been proposed for selecting suitable feature subset from a large feature set. The proposed algorithm falls under the genre of clustering based feature selection techniques in which features are initially clustered according to their intrinsic characteristics following the filter approach. In the second step the most suitable feature is selected from each cluster to form the final subset following a wrapper approach. The two stage hybrid process lowers the computational cost of subset selection, especially for large feature data sets. One of the main novelty of the proposed approach lies in the process of determining optimal number of feature clusters. Unlike currently available methods, which mostly employ a trial and error approach, the proposed method characterises and quantifies the feature clusters according to the quality of the features inside the clusters and defines a taxonomy of the feature clusters. The selection of individual features from a feature cluster can be done judiciously considering both the relevancy and redundancy according to users intention and requirement. The algorithm has been verified by simulation experiments with different bench mark data set containing features ranging from 10 to more than 800 and compared with other currently used feature selection algorithms. The simulation results prove the superiority of our proposal in terms of model performance, flexibility of use in practical problems and extendibility to large feature sets. Though the current proposal is verified in the domain of unsupervised classification, it can be easily used in case of supervised classification.

A feature cluster taxonomy based feature selection technique

With the proliferation of unstructured data, text classification or text categorization has found many applications in topic classification, sentiment analysis, authorship identification, spam detection, and so on. There are many classification algorithms available. Naive Bayes remains one of the oldest and most popular classifiers. On one hand, implementation of naive Bayes is simple and, on the other hand, this also requires fewer amounts of training data. From the literature review, it is found that naive Bayes performs poorly compared to other classifiers in text classification. As a result, this makes the naive Bayes classifier unusable in spite of the simplicity and intuitiveness of the model. In this paper, we propose a two-step feature selection method based on firstly a univariate feature selection and then feature clustering, where we use the univariate feature selection method to reduce the search space and then apply clustering to select relatively independent feature sets. We demonstrate the effectiveness of our method by a thorough evaluation and comparison over 13 datasets. The performance improvement thus achieved makes naive Bayes comparable or superior to other classifiers. The proposed algorithm is shown to outperform other traditional methods like greedy search based wrapper or CFS.

https://downloads.hindawi.com/archive/2014/717092.pdf

Javed Aktar

Papers

A Novel Feature Selection Technique for Text Classification Using Naïve Bayes