We survey 146 papers analyzing "bias" in NLP systems, finding that their motivations are often vague, inconsistent, and lacking in normative reasoning, despite the fact that analyzing "bias" is an inherently normative process. We further find that these papers' proposed quantitative techniques for measuring or mitigating "bias" are poorly matched to their motivations and do not engage with the relevant literature outside of NLP. Based on these findings, we describe the beginnings of a path forward by proposing three recommendations that should guide work analyzing "bias" in NLP systems. These recommendations rest on a greater recognition of the relationships between language and social hierarchies, encouraging researchers and practitioners to articulate their conceptualizations of "bias"---i.e., what kinds of system behaviors are harmful, in what ways, to whom, and why, as well as the normative reasoning underlying these statements---and to center work around the lived experiences of members of communities affected by NLP systems, while interrogating and reimagining the power relations between technologists and such communities.

Language (Technology) is Power: A Critical Survey of "Bias" in NLP

Accessibility of digital libraries and other web-based repositories has caused the illusion of accessibility of the full texts of scientific papers. However, in the majority of cases such an access (at least free access) is limited only to abstracts having no more then 50-100 words. Traditional keyword-based approach for clustering this type of documents gives unstable and imprecise results. We show that they can be easy improved with more adequate keyword selection and document similarity evaluation. We suggest simple procedures for this. We evaluate our approach on the data from two international conferences. One of our conclusions is the suggestion for the digital libraries and other repositories to provide document images of full texts of the papers along with their abstracts for open access via Internet.

Clustering abstracts instead of full texts

The vector space model (VSM) is a textual representation method that is widely used in documents classification. However, it remains to be a space-challenging problem. One attempt to alleviate the space problem is by using dimensionality reduction techniques, however, such techniques have deficiencies such as losing some important information. In this paper, we propose a novel text classification method that neither uses VSM nor dimensionality reduction techniques. The proposed method is a space efficient method that utilizes the first order Markov model for hierarchical Arabic text classification. For each category and sub-category, a Markov chain model is prepared based on the neighboring characters sequences. The prepared models are then used for scoring documents for classification purposes. For evaluation, we used a hierarchical Arabic text data collection that contains 11,191 documents that belong to eight topics distributed into 3-levels. The experimental results show that the Markov chains based method significantly outperforms the baseline system that employs the latent semantic indexing (LSI) method. That is, the proposed method enhances the F1-measure by 3.47%. The novelty of this work lies on the idea of decomposing words into sequences of characters, which found to be a promising approach in terms of space and accuracy. Based on our best knowledge, this is the first attempt to conduct research for hierarchical Arabic text classification with such relatively large data collection.

Beyond vector space model for hierarchical Arabic text classification: A Markov chain approach

Short documents are typically represented by very sparse vectors, in the space of terms. In this case, traditional techniques for calculating text similarity results in measures which are very close to zero, since documents even the very similar ones have a very few or mostly no terms in common. In order to alleviate this limitation, the representation of short-text segments should be enriched by incorporating information about correlation between terms. In other words, if two short segments do not have any common words, but terms from the first segment appear frequently with terms from the second segment in other documents, this means that these segments are semantically related, and their similarity measure should be high. Towards achieving this goal, we employ a method for enhancing document clustering using statistical semantics. However, the problem of high computation time arises when calculating correlation between all terms. In this work, we propose the selection of a few terms, and using these terms with the Nystr\"om method to approximate the term-term correlation matrix. The selection of the terms for the Nystr\"om method is performed by randomly sampling terms with probabilities proportional to the lengths of their vectors in the document space. This allows more important terms to have more influence on the approximation of the term-term correlation matrix and accordingly achieves better accuracy.

Short-Text Clustering using Statistical Semantics

Recent research has revealed undesirable biases in NLP data and models. However, these efforts focus of social disparities in West, and are not directly portable to other geo-cultural contexts. In this paper, we focus on NLP fairness in the context of India. We start with a brief account of the prominent axes of social disparities in India. We build resources for fairness evaluation in the Indian context and use them to demonstrate prediction biases along some of the axes. We then delve deeper into social stereotypes for Region and Religion, demonstrating its prevalence in corpora and models. Finally, we outline a holistic research agenda to re-contextualize NLP fairness research for the Indian context, accounting for Indian societal context, bridging technological gaps in NLP capabilities and resources, and adapting to Indian cultural values. While we focus on India, this framework can be generalized to other geo-cultural contexts.

Re-contextualizing Fairness in NLP: The Case of India

Word-embedding is a major machine learning technique for computational applications of languages. For a given corpus, the process of word-embedding is to embed each word onto multi-dimensional space such that semantic similarities between similar words are retained. While learning the similarity as encapsulated in the training corpus, the embedding process inadvertently captures many other inherent features present in the corpus. One such thing is the bias arising out of stereotyping present in almost all the corpus no matter how extensively used and trusted they are. We study this aspect of word-embedding in the context of Hindi language. We show that many gender-neutral words in Hindi are mapped to vectors which are inclined towards one gender or the other in multi-dimensional space. We propose a new algorithm of debiasing and demonstrate its efficacy in the context of Hindi language. Further, we build a SVM-based classifier that determines whether a gender-neutral word is classified as neutral or otherwise. We corroborate our claim with experimental results on large number of individual words. This work is first ever result on debiasing in Hindi Language and our new debiasing algorithm can be applicable in the context of any language.

Debiasing Gender biased Hindi Words with Word-embedding

We propose a technique of determining the number of clusters of a corpus of short-text documents A spectral algorithm suitable for short-texts is used to generate an ensemble A Markov chain induced by the co-association matrix is studied to observe nearly uncoupling phenomenon over iterations A large spectral gap and number of eigenvectors close to 1 indicate the number of clusters We demonstrate by experimenting on several datasets

Spectral approach to find number of clusters of short-text documents

Short-texts mining has become an important area of research in IR and data mining. Ncut-term weighting is recently proposed for clustering of short-texts using non-negative matrix factorization. Non-negative factorization can be employed for such term weighting when the similarity measure is the inner product of term-document matrix. We propose a new weighting scheme and devise a new clustering algorithm using Hadamard product of similarity matrices. We demonstrate that our technique yields much better clustering in comparison to ncut weighting scheme. We use three measures for evaluating clustering qualities, namely purity, normalized mutual information and adjusted Rand index. We use standard benchmark datasets and also compare the performance of our algorithm with well-known document clustering technique of Ng-Jordan-Weiss. Experimental results suggest that the weighting process by Hadamard product gives better clustering of document of short-texts.

Clustering Short-Text Using Non-negative Matrix Factorization of Hadamard Product of Similarities

Collaborative filtering (CF), the most prevalent technique in the area of recommender systems (RSs), provides suggestions to users based on the tastes of their similar users. However, the new user and sparsity problems, degrade its efficiency of recommendations. Trust can enhance the recommendation quality by mimicking social dictum “friend of a friend will be a friend”. However distrust, the another face of coin is yet to be explored along with trust in the area of RSs. Our work in this paper is an attempt toward introducing trust-distrust enhanced recommendations based on the novel similarity measure that combines user ratings and trust values for generating more quality recommendations. Our approach also exploits distrust links among users and analyses their propagation effects. Further, distrust values are also used for filtering more distrust-worthy neighbours from the neighbourhood set. Our experimental results show that our proposed approaches outperform the traditional CF and existing trust enhanced approaches in terms of various performance measures.

Trust Distrust Enhanced Recommendations Using an Effective Similarity Measure

Text mining works widely in the field of research techniques, which allow an individual to store text and its important terms in form of electronic document (.doc, .txt). Obliviously, one cannot remember such huge amount of text; moreover, the manual approach is more time-taking, unreliable, and accessible to that person only. Text mining techniques optimize this approach by extracting and storing this data. Computational comparison, file read, file write are more efficiently done. With the help of Pico-Nym Cloud (PNC), we generated more semantically similar, related, and significant patterns. The give, generate, and get sequence modeling is adopted. Over the other available Web applications, we present our application with improved stemming, relation, and average case consideration. This approach does not limit the displayed number of words as all the generated sets can be traversed with the GUI, with opted size of patterns. This PNC is highly applicable in bioinformatics, related information retrieval from document, sentimental analysis using social Web sites (Twitter and Facebook), query expansion (Google) and many more.

Mukesh K. Jadon

Papers

Debiasing Gender biased Hindi Words with Word-embedding

Spectral approach to find number of clusters of short-text documents

Clustering Short-Text Using Non-negative Matrix Factorization of Hadamard Product of Similarities

Trust Distrust Enhanced Recommendations Using an Effective Similarity Measure

Pico-Nym Cloud (PNC): A Method to Devise and Peruse Semantically Related Biological Patterns