International Journal of Computer and Applications

Sentiment analysis on Twitter: A text mining approach to the Syrian refugee crisis

https://hal-audencia.archives-ouvertes.fr/hal-01996486/document

Big data analytics and organizational culture as complements to swift trust and collaborative performance in the humanitarian supply chain

The era of big data and analytics is opening up new possibilities for disaster management (DM). Due to its ability to visualize, analyze and predict disasters, big data is changing the humanitarian operations and crisis management dramatically. Yet, the relevant literature is diverse and fragmented, which calls for its review in order to ascertain its development. A number of publications have dealt with the subject of big data and its applications for minimizing disasters. Based on a systematic literature review, this study examines big data in DM to present main contributions, gaps, challenges and future research agenda. The study presents the findings in terms of yearly distribution, main journals, and most cited papers. The findings also show a classification of publications, an analysis of the trends and the impact of published research in the DM context. Overall the study contributes to a better understanding of the importance of big data in disaster management.

/pdf/big-data-and-disaster-management-a-systematic-review-and-3gsgopeqcz.pdf

Big data and disaster management: a systematic review and agenda for future research

Sentiment analysis of Twitter data during critical events through Bayesian networks classifiers

Feature selection remains one of the most important steps for usability of a model for both supervised and unsupervised classification. For a dataset, with n features, the number of possible feature subsets is 2n. Even for a moderate size of n, there is a combinatorial explosion in the search space. Feature selection is a NP-hard problem; hence finding the optimal solution is not feasible. Typically various kinds of intelligent and metaheuristic search techniques can be employed for this purpose. Hill climbing is arguably the simplest of such techniques. It has many variants based on (a) trade-off between greediness and randomness, (b) direction of the search, and (c) size of the neighborhood. Consequently it might not be trivial for the practitioner to choose a suitable method for the task in hand. In this paper, we have attempted to address this issue in the context of feature selection. The descriptions of the methods are followed by an extensive empirical study over 20 publicly available datasets. Finally a comparison has been done with genetic algorithm, which shows the effectiveness of hill climbing methods in the context of feature selection.

Filter-Based Feature Selection Methods Using Hill Climbing Approach

In this paper the use of an evolutionary computational algorithm such Differential Evolution have been studied for their use in feature selection problems. Feature selection is rapidly increasing in importance due to the exponential increase in the size of datasets. The dimensionality problem arising due to these large datasets can be dealt with by using feature selection, which provides a multitude of advantages when it comes to datasets. The advantages have been discussed in the paper. The use of Differential Evolution in this regard is a novel approach and provides us with comparable results with existing algorithms like GA. This paper also deals with the approach that involves changing the mutation scheme to allow for conversion to binary before selecting the features.

Feature selection using differential evolution with binary mutation scheme

Clustering is one of the main tasks in exploratory data analysis and descriptive statistics where the main objective is partitioning observations in groups. Clustering has a broad range of application in varied domains like climate, business, information retrieval, biology, psychology, to name a few. A variety of methods and algorithms have been developed for clustering tasks in the last few decades. We observe that most of these algorithms define a cluster in terms of value of the attributes, density, distance etc. However these definitions fail to attach a clear meaning/semantics to the generated clusters. We argue that clusters having understandable and distinct semantics defined in terms of quartiles/halves are more appealing to business analysts than the clusters defined by data boundaries or prototypes. On the samepremise, we propose our new algorithm named as quartile clustering technique. Through a series of experiments we establish efficacy of this algorithm. We demonstrate that the quartile clustering technique adds clear meaning to each of the clusters compared to K-means. We use DB Index to measure goodness of the clusters and show our method is comparable to EM (Expectation Maximization), PAM (Partition around Medoid) and K Means. We have explored its capability in detecting outlier and the benefit of added semantics. We discuss some of the limitations in its present form and also provide a rough direction in addressing the issue of merging the generated clusters.

Quartile Clustering: A quartile based technique for Generating Meaningful Clusters

Topic models allude to statistical algorithms for finding out an extensive text body’s latent semantic structures. Standing here in today’s world, the measure of the textual data and information we come across in our day-to-day lives is basically beyond our handling limit. Topic models can provide a way out for us to understand and manage the vast accumulations of unstructured textual data and information. Initially emerged as a text-mining instrument, topic models have found applications in various other fields. This paper makes a thorough comparative study of LSA with that of commonly used TF-IDF approach for text classification and proves that LSA yields better accuracy in classifying texts. The novelty of the paper lies in the fact that we are using a much sparser representation than usual TF-IDF and also, LSA can get from the topic if there are any synonym words. This paper proposes a method, using the concept of entropy, which further increases the accuracy of text classification.

Topic Modeling for Text Classification

Feature selection is one of the most important preprocessing steps in Machine Learning. This can be broadly divided into search based methods and ranking based methods. The ranking based methods are very popular because they need much lesser computational power. There can be many different ways to rank the features. One of the ways to measure effectiveness of a feature is by evaluating its ability to separate the classes involved. These interclass Separability based measures can be directly used as a feature ranking tool for binary classification problems. Bhattacharya Distance which is the most popular among them has been used majorly in a recursive setup to select good quality feature subsets. Jeffries-Matusita (JM) distance improves Bhattacharya distance by normalizing it between 0 and 2. In this paper, we have ranked the features based on JM distance. The results are comparable with mutual information, Relief and Chi Squared based measures as per experiments conducted over 24 public datasets but in much lesser time. JM distance also provide some intuition about the dataset prior to any feature selection or machine learning algorithm. A comparison has been done on classification accuracy and JM scores of these datasets, which can provide a good intuition on how good a dataset is for classification and point out the need of or lack of further feature collection.

Saptarsi Goswami

Papers

Filter-Based Feature Selection Methods Using Hill Climbing Approach

Feature selection using differential evolution with binary mutation scheme

Quartile Clustering: A quartile based technique for Generating Meaningful Clusters

Topic Modeling for Text Classification

Jeffries-Matusita distance as a tool for feature selection