This paper presents a survey on hate speech detection. Given the steadily growing body of social media content, the amount of online hate speech is also increasing. Due to the massive scale of the web, methods that automatically detect hate speech are required. Our survey describes key areas that have been explored to automatically recognize these types of utterances using natural language processing. We also discuss limits of those approaches.

/pdf/a-survey-on-hate-speech-detection-using-natural-language-1hnsqiizvn.pdf

A Survey on Hate Speech Detection using Natural Language Processing

A key challenge for automatic hate-speech detection on social media is the separation of hate speech from other instances of offensive language. Lexical detection methods tend to have low precision because they classify all messages containing particular terms as hate speech and previous work using supervised learning has failed to distinguish between the two categories. We used a crowd-sourced hate speech lexicon to collect tweets containing hate speech keywords. We use crowd-sourcing to label a sample of these tweets into three categories: those containing hate speech, only offensive language, and those with neither. We train a multi-class classifier to distinguish between these different categories. Close analysis of the predictions and the errors shows when we can reliably separate hate speech from other offensive language and when this differentiation is more difficult. We find that racist and homophobic tweets are more likely to be classified as hate speech but that sexist tweets are generally classified as offensive. Tweets without explicit hate keywords are also more difficult to classify.

Automated Hate Speech Detection and the Problem of Offensive Language

The scientific study of hate speech, from a computer science point of view, is recent. This survey organizes and describes the current state of the field, providing a structured overview of previous approaches, including core algorithms, methods, and main features used. This work also discusses the complexity of the concept of hate speech, defined in many platforms and contexts, and provides a unifying definition. This area has an unquestionable potential for societal impact, particularly in online communities and digital media platforms. The development and systematization of shared resources, such as guidelines, annotated datasets in multiple languages, and algorithms, is a crucial step in advancing the automatic detection of hate speech.

/pdf/a-survey-on-automatic-detection-of-hate-speech-in-text-430g1cqe3i.pdf

A Survey on Automatic Detection of Hate Speech in Text

In recent years, there has been an exponential growth in the number of complex documents and texts that require a deeper understanding of machine learning methods to be able to accurately classify texts in many applications. Many machine learning approaches have achieved surpassing results in natural language processing. The success of these learning algorithms relies on their capacity to understand complex models and non-linear relationships within data. However, finding suitable structures, architectures, and techniques for text classification is a challenge for researchers. In this paper, a brief overview of text classification algorithms is discussed. This overview covers different text feature extractions, dimensionality reduction methods, existing algorithms and techniques, and evaluations methods. Finally, the limitations of each technique and their application in real-world problems are discussed.

Text Classification Algorithms: A Survey

In recent years, there has been an exponential growth in the number of complex documents and texts that require a deeper understanding of machine learning methods to be able to accurately classify texts in many applications. Many machine learning approaches have achieved surpassing results in natural language processing. The success of these learning algorithms relies on their capacity to understand complex models and non-linear relationships within data. However, finding suitable structures, architectures, and techniques for text classification is a challenge for researchers. In this paper, a brief overview of text classification algorithms is discussed. This overview covers different text feature extractions, dimensionality reduction methods, existing algorithms and techniques, and evaluations methods. Finally, the limitations of each technique and their application in the real-world problem are discussed.

We explore the idea of creating a classifier that can be used to detect presence of hate speech in web discourses such as web forums and blogs. In this work, hate speech problem is abstracted into three main thematic areas of race, nationality and religion. The goal of our research is to create a model classifier that uses sentiment analysis techniques and in particular subjectivity detection to not only detect that a given sentence is subjective but also to identify and rate the polarity of sentiment expressions. We begin by whittling down the document size by removing objective sentences. Then, using subjectivity and semantic features related to hate speech, we create a lexicon that is employed to build a classifier for hate speech detection. Experiments with a hate corpus show significant practical application for a real-world web discourse.

/pdf/a-lexicon-based-approach-for-hate-speech-detection-2jmpeeqcnf.pdf

A Lexicon-based Approach for Hate Speech Detection

A new supervised term weighting scheme called TF-IGM is proposed.It adopts a new statistical model to measure a term's class distinguishing power.It makes full use of the fine-grained term distribution across different classes.It is adaptive to different text datasets by providing options or parameters.It outperforms TF-IDF and state-of-the-art supervised term weighting schemes. Massive textual data management and mining usually rely on automatic text classification technology. Term weighting is a basic problem in text classification and directly affects the classification accuracy. Since the traditional TF-IDF (term frequency & inverse document frequency) is not fully effective for text classification, various alternatives have been proposed by researchers. In this paper we make comparative studies on different term weighting schemes and propose a new term weighting scheme, TF-IGM (term frequency & inverse gravity moment), as well as its variants. TF-IGM incorporates a new statistical model to precisely measure the class distinguishing power of a term. Particularly, it makes full use of the fine-grained term distribution across different classes of text. The effectiveness of TF-IGM is validated by extensive experiments of text classification using SVM (support vector machine) and kNN (k nearest neighbors) classifiers on three commonly used corpora. The experimental results show that TF-IGM outperforms the famous TF-IDF and the state-of-the-art supervised term weighting schemes. In addition, some new findings different from previous studies are obtained and analyzed in depth in the paper.

Turning from TF-IDF to TF-IGM for term weighting in text classification

A failure of adaptive inference—misinterpreting available sensory information for appropriate perception and action—is at the heart of clinical manifestations of schizophrenia, implicating key subcortical structures in the brain including the hippocampus. We used high-resolution, three-dimensional (3D) fractal geometry analysis to study subtle and potentially biologically relevant structural alterations (in the geometry of protrusions, gyri and indentations, sulci) in subcortical gray matter (GM) in patients with schizophrenia relative to healthy individuals. In particular, we focus on utilizing Fractal Dimension (FD), a compact shape descriptor that can be computed using inputs with irregular (i.e., not necessarily smooth) surfaces in order to quantify complexity (of geometrical properties and configurations of structures across spatial scales) of subcortical GM in this disorder. Probabilistic (entropy-based) information FD was computed based on the box-counting approach for each of the seven subcortical structures, bilaterally, as well as the brainstem from high-resolution magnetic resonance (MR) images in chronic patients with schizophrenia (n = 19) and age-matched healthy controls (n = 19) (age ranges: patients, 22.7–54.3 and healthy controls, 24.9–51.6 years old). We found a significant reduction of FD in the left hippocampus (median: 2.1460, range: 2.07–2.18 vs. median: 2.1730, range: 2.15–2.23, p<0.001; Cohen’s effect size, U3 = 0.8158 (95% Confidence Intervals, CIs: 0.6316, 1.0)), the right hippocampus (median: 2.1430, range: 2.05–2.19 vs. median: 2.1760, range: 2.12–2.21, p = 0.004; U3 = 0.8421 (CIs: 0.5263, 1)), as well as left thalamus (median: 2.4230, range: 2.40–2.44, p = 0.005; U3 = 0.7895 (CIs: 0.5789, 0.9473)) in schizophrenia patients, relative to healthy individuals. Our findings provide in-vivo quantitative evidence for reduced surface complexity of hippocampus, with reduced FD indicating a less complex, less regular GM surface detected in schizophrenia.

/pdf/fractal-dimension-analysis-of-subcortical-gray-matter-53v966j5fn.pdf

Fractal Dimension Analysis of Subcortical Gray Matter Structures in Schizophrenia.

Recognition of vital nodes in complex networks retains great importance in the improvement of network’s robustness and vulnerability. Consistent research proposed various approaches like local-structure-based methods, e.g., degree centrality, pagerank, etc., and global-structure-based methods, e.g., betweenness, closeness centrality, etc., to evaluate the concerned nodes. Though their performance is amazingly well, these methods have undergone some intrinsic limitations. For instance, local-structure-based methods lose some sort of global information and global-structure-based methods are too complicated to measure the important nodes, particularly in networks where sizes become large. To tackle these challenges, we propose a Local-and-Global-Centrality (LGC) measuring algorithm to identify the vital nodes through handling local as well as global topological aspects of a network simultaneously. In order to assess the performance of the proposed algorithm with respect to the state-of-the-art methodologies, we performed experiments through LCG, Betweenness (BNC), Closeness (CNC), Gravity (GIC), Page-Rank (PRC), Eigenvector (EVC), Global and Local Structure (GLS), Global Structure Model (GSM), and Profit-leader (PLC) methods on differently sized real-world networks. Our experiments disclose that LGC outperformed many of the compared techniques.

Identifying vital nodes from local and global perspectives in complex networks

Necessary and sufficient conditions for an attribute set to be a new intent.The procedure of searching for canonical generators is improved.We prove that the number of children of any concept has an upper bound.The procedure of updating the upper neighbors of a new concept is improved.An efficient incremental algorithm for constructing concept lattices is proposed. Incremental algorithms for constructing concept lattices can update a concept lattice according to new objects added to the formal context. In this paper, we propose an efficient incremental algorithm for concept lattice construction. The algorithm, called FastAddIntent, results as a modification of AddIntent in which we improve two fundamental procedures including searching for canonical generators and fixing the covering relation. We describe the algorithm completely, prove correctness of our improvements, discuss time complexity issues, and present an experimental evaluation of its performance and comparison with AddIntent. Theoretical and empirical analyses show the advantages of our algorithm when applied to large or (and) dense formal contexts.

Jun Long

Papers

A Lexicon-based Approach for Hate Speech Detection

Turning from TF-IDF to TF-IGM for term weighting in text classification

Fractal Dimension Analysis of Subcortical Gray Matter Structures in Schizophrenia.

Identifying vital nodes from local and global perspectives in complex networks

A fast incremental algorithm for constructing concept lattices