Using VADER sentiment and SVM for predicting customer response sentiment
TL;DR: Results indicate a predictable pattern in e-mail conversation that enables predicting the sentiment of a not-yet-seen e-mails that can be used e.g. to prepare particular actions for customers that are likely to have a negative response.
Abstract: Customer support is important to corporate operations, which involves dealing with disgruntled customer and content customers that can have different requirements. As such, it is important to quickly extract the sentiment of support errands. In this study we investigate sentiment analysis in customer support for a large Swedish Telecom corporation. The data set consists of 168,010 e-mails divided into 69,900 conversation threads without any sentiment information available. Therefore, VADER sentiment is used together with a Swedish sentiment lexicon in order to provide initial labeling of the e-mails. The e-mail content and sentiment labels are then used to train two Support Vector Machine models in extracting/classifying the sentiment of e-mails. Further, the ability to predict sentiment of not-yet-seen e-mail responses is investigated. Experimental results show that the LinearSVM model was able to extract sentiment with a mean F1-score of 0.834 and mean AUC of 0.896. Moreover, the LinearSVM algorithm was also able to predict the sentiment of an e-mail one step ahead in the thread (based on the text in the an already sent e-mail) with a mean F1-score of 0.688 and the mean AUC of 0.805. The results indicate a predictable pattern in e-mail conversation that enables predicting the sentiment of a not-yet-seen e-mail. This can be used e.g. to prepare particular actions for customers that are likely to have a negative response. It can also provide feedback on possible sentiment reactions to customer support e-mails.
Citations
More filters
TL;DR: Sentiment analysis is the process of gathering and analyzing people's opinions, thoughts, and impressions regarding various topics, products, subjects, and services as mentioned in this paper , which can be beneficial to corporations, governments and individuals for collecting information and making decisions based on opinion.
Abstract: The rapid growth of Internet-based applications, such as social media platforms and blogs, has resulted in comments and reviews concerning day-to-day activities. Sentiment analysis is the process of gathering and analyzing people’s opinions, thoughts, and impressions regarding various topics, products, subjects, and services. People’s opinions can be beneficial to corporations, governments, and individuals for collecting information and making decisions based on opinion. However, the sentiment analysis and evaluation procedure face numerous challenges. These challenges create impediments to accurately interpreting sentiments and determining the appropriate sentiment polarity. Sentiment analysis identifies and extracts subjective information from the text using natural language processing and text mining. This article discusses a complete overview of the method for completing this task as well as the applications of sentiment analysis. Then, it evaluates, compares, and investigates the approaches used to gain a comprehensive understanding of their advantages and disadvantages. Finally, the challenges of sentiment analysis are examined in order to define future directions.
138 citations
TL;DR: Sentiment analysis is the process of gathering and analyzing people's opinions, thoughts, and impressions regarding various topics, products, subjects, and services as mentioned in this paper , which can be beneficial to corporations, governments and individuals for collecting information and making decisions based on opinion.
Abstract: The rapid growth of Internet-based applications, such as social media platforms and blogs, has resulted in comments and reviews concerning day-to-day activities. Sentiment analysis is the process of gathering and analyzing people’s opinions, thoughts, and impressions regarding various topics, products, subjects, and services. People’s opinions can be beneficial to corporations, governments, and individuals for collecting information and making decisions based on opinion. However, the sentiment analysis and evaluation procedure face numerous challenges. These challenges create impediments to accurately interpreting sentiments and determining the appropriate sentiment polarity. Sentiment analysis identifies and extracts subjective information from the text using natural language processing and text mining. This article discusses a complete overview of the method for completing this task as well as the applications of sentiment analysis. Then, it evaluates, compares, and investigates the approaches used to gain a comprehensive understanding of their advantages and disadvantages. Finally, the challenges of sentiment analysis are examined in order to define future directions.
71 citations
TL;DR: In this article , a state-of-the-art systematic review of academic papers and a Machine Learning-based analysis of grey literature on the social implications of Industry 4.0 are presented.
33 citations
TL;DR: The hotel industry is the one which has confronted the unprecedented effect of the coronavirus disease 2019 (COVID-19) pandemic to significant social and economic risks as mentioned in this paper.
Abstract: Hotel industry is the one which has confronted the unprecedented effect of the coronavirus disease 2019 (COVID-19) pandemic to significant social and economic risks. The COVID-19 pandemic has chall...
32 citations
References
More filters
Journal Article•
TL;DR: A set of simple, yet safe and robust non-parametric tests for statistical comparisons of classifiers is recommended: the Wilcoxon signed ranks test for comparison of two classifiers and the Friedman test with the corresponding post-hoc tests for comparisons of more classifiers over multiple data sets.
Abstract: While methods for comparing two learning algorithms on a single data set have been scrutinized for quite some time already, the issue of statistical tests for comparisons of more algorithms on multiple data sets, which is even more essential to typical machine learning studies, has been all but ignored. This article reviews the current practice and then theoretically and empirically examines several suitable tests. Based on that, we recommend a set of simple, yet safe and robust non-parametric tests for statistical comparisons of classifiers: the Wilcoxon signed ranks test for comparison of two classifiers and the Friedman test with the corresponding post-hoc tests for comparison of more classifiers over multiple data sets. Results of the latter can also be neatly presented with the newly introduced CD (critical difference) diagrams.
10,306 citations
Journal Article•
TL;DR: LIBLINEAR is an open source library for large-scale linear classification that supports logistic regression and linear support vector machines and provides easy-to-use command-line tools and library calls for users and developers.
Abstract: LIBLINEAR is an open source library for large-scale linear classification. It supports logistic regression and linear support vector machines. We provide easy-to-use command-line tools and library calls for users and developers. Comprehensive documents are available for both beginners and advanced users. Experiments demonstrate that LIBLINEAR is very efficient on large sparse data sets.
7,848 citations
01 Jan 2002
TL;DR: In this paper, the problem of classifying documents not by topic, but by overall sentiment, e.g., determining whether a review is positive or negative, was considered and three machine learning methods (Naive Bayes, maximum entropy classiflcation, and support vector machines) were employed.
Abstract: We consider the problem of classifying documents not by topic, but by overall sentiment, e.g., determining whether a review is positive or negative. Using movie reviews as data, we flnd that standard machine learning techniques deflnitively outperform human-produced baselines. However, the three machine learning methods we employed (Naive Bayes, maximum entropy classiflcation, and support vector machines) do not perform as well on sentiment classiflcation as on traditional topic-based categorization. We conclude by examining factors that make the sentiment classiflcation problem more challenging.
6,980 citations
TL;DR: This survey paper tackles a comprehensive overview of the last update in this field of sentiment analysis with sophisticated categorizations of a large number of recent articles and the illustration of the recent trend of research in the sentiment analysis and its related areas.
2,152 citations
TL;DR: Analysis and empirical evidence suggest that the evaluation results on some versions of Reuters were significantly affected by the inclusion of a large portion of unlabelled documents, mading those results difficult to interpret and leading to considerable confusions in the literature.
Abstract: This paper focuses on a comparative evaluation of a wide-range of text categorization methods, including previously published results on the Reuters corpus and new results of additional experiments. A controlled study using three classifiers, kNN, LLSF and WORD, was conducted to examine the impact of configuration variations in five versions of Reuters on the observed performance of classifiers. Analysis and empirical evidence suggest that the evaluation results on some versions of Reuters were significantly affected by the inclusion of a large portion of unlabelled documents, mading those results difficult to interpret and leading to considerable confusions in the literature. Using the results evaluated on the other versions of Reuters which exclude the unlabelled documents, the performance of twelve methods are compared directly or indirectly. For indirect compararions, kNN, LLSF and WORD were used as baselines, since they were evaluated on all versions of Reuters that exclude the unlabelled documents. As a global observation, kNN, LLSF and a neural network method had the best performances except for a Naive Bayes approach, the other learning algorithms also performed relatively well.
2,130 citations