We survey 146 papers analyzing "bias" in NLP systems, finding that their motivations are often vague, inconsistent, and lacking in normative reasoning, despite the fact that analyzing "bias" is an inherently normative process. We further find that these papers' proposed quantitative techniques for measuring or mitigating "bias" are poorly matched to their motivations and do not engage with the relevant literature outside of NLP. Based on these findings, we describe the beginnings of a path forward by proposing three recommendations that should guide work analyzing "bias" in NLP systems. These recommendations rest on a greater recognition of the relationships between language and social hierarchies, encouraging researchers and practitioners to articulate their conceptualizations of "bias"---i.e., what kinds of system behaviors are harmful, in what ways, to whom, and why, as well as the normative reasoning underlying these statements---and to center work around the lived experiences of members of communities affected by NLP systems, while interrogating and reimagining the power relations between technologists and such communities.

Language (Technology) is Power: A Critical Survey of "Bias" in NLP

"Meaningful Differences in the Everyday Experience of Young American Children" is the story of the landmark research study that uncovered the widely cited &#34word gap" between children from low-income homes and their more economically advantaged peers. This groundbreaking research has spurred hundreds of studies and programs, including the White Housea s "Bridging the Word Gap" campaign and "Too Small to Fail," a joint initiative of the Bill, Hillary, and Chelsea Clinton foundation. Betty Hart and Todd Risley wanted to know why, despite best efforts in preschool programs to equalize opportunity, children from low-income homes remain well behind their more economically advantaged peers years later in school. Each month, they recorded one full hour of every word spoken at home between parent and child in 42 families, categorized as professional, working class, or welfare families. Two and a half years of coding and analyzing every utterance in 1,318 transcripts followed. By age 3, the recorded spoken vocabularies of the children from the professional families were larger than those of the parents in the welfare families. Between professional and welfare parents, there was a difference of almost 300 words spoken per hour. Extrapolating this verbal interaction to four years, a child in a professional family would accumulate experience with almost 45 million words, while an average child in a welfare family would hear just 13 million coining the phrase "the 30 million word gap." The implications of this painstaking study are staggering: Hart and Risley's follow-up studies at age 9 show that the large differences in children's language experience were tightly linked to large differences in child outcomes. As the authors note in their preface to the 2002 printing of "Meaningful Differences," "the most important aspect to evaluate in child care settings for very young children is the amount of talk actually going on, moment by moment, between children and their caregivers." By giving children positive interactions and experiences with adults who take the time to teach vocabulary, oral language concepts, and emergent literacy concepts, children should have a better chance to succeed at school and in the workplace. Learn more about how parent and children's language interactions affect learning to talk in Hart & Risley's companion book The Social World of Children Learning to Talk."

/pdf/meaningful-differences-in-the-everyday-experience-of-young-2buviennyx.pdf

Meaningful Differences in the Everyday Experience of Young American Children

We survey 146 papers analyzing "bias" in NLP systems, finding that their motivations are often vague, inconsistent, and lacking in normative reasoning, despite the fact that analyzing "bias" is an inherently normative process We further find that these papers' proposed quantitative techniques for measuring or mitigating "bias" are poorly matched to their motivations and do not engage with the relevant literature outside of NLP Based on these findings, we describe the beginnings of a path forward by proposing three recommendations that should guide work analyzing "bias" in NLP systems These recommendations rest on a greater recognition of the relationships between language and social hierarchies, encouraging researchers and practitioners to articulate their conceptualizations of "bias"---ie, what kinds of system behaviors are harmful, in what ways, to whom, and why, as well as the normative reasoning underlying these statements---and to center work around the lived experiences of members of communities affected by NLP systems, while interrogating and reimagining the power relations between technologists and such communities

/pdf/language-technology-is-power-a-critical-survey-of-bias-in-40xzysfmb0.pdf

Person reidentification is a key technique to match different persons observed in nonoverlapping camera views. Many researchers treat it as a special object-retrieval problem, where ranking optimization plays an important role. Existing ranking optimization methods mainly utilize the similarity relationship between the probe and gallery images to optimize the original ranking list, but seldom consider the important dissimilarity relationship. In this paper, we propose to use both similarity and dissimilarity cues in a ranking optimization framework for person reidentification. Its core idea is that the true match should not only be similar to those strongly similar galleries of the probe, but also be dissimilar to those strongly dissimilar galleries of the probe. Furthermore, motivated by the philosophy of multiview verification, a ranking aggregation algorithm is proposed to enhance the detection of similarity and dissimilarity based on the following assumption: the true match should be similar to the probe in different baseline methods. In other words, if a gallery blue image is strongly similar to the probe in one method, while simultaneously strongly dissimilar to the probe in another method, it will probably be a wrong match of the probe. Extensive experiments conducted on public benchmark datasets and comparisons with different baseline methods have shown the great superiority of the proposed ranking optimization method.

Person Reidentification via Ranking Aggregation of Similarity Pulling and Dissimilarity Pushing

An increasing number of natural language processing papers address the effect of bias on predictions, introducing mitigation techniques at different parts of the standard NLP pipeline (data and models). However, these works have been conducted individually, without a unifying framework to organize efforts within the field. This situation leads to repetitive approaches, and focuses overly on bias symptoms/effects, rather than on their origins, which could limit the development of effective countermeasures. In this paper, we propose a unifying predictive bias framework for NLP. We summarize the NLP literature and suggest general mathematical definitions of predictive bias. We differentiate two consequences of bias: outcome disparities and error disparities, as well as four potential origins of biases: label bias, selection bias, model overamplification, and semantic bias. Our framework serves as an overview of predictive bias in NLP, integrating existing work into a single structure, and providing a conceptual baseline for improved frameworks.

/pdf/predictive-biases-in-natural-language-processing-models-a-2a6dfu0hs7.pdf

Predictive Biases in Natural Language Processing Models: A Conceptual Framework and Overview

Word embedding models have gained a lot of traction in the Natural Language Processing community, however, they suffer from unintended demographic biases. Most approaches to evaluate these biases rely on vector space based metrics like the Word Embedding Association Test (WEAT). While these approaches offer great geometric insights into unintended biases in the embedding vector space, they fail to offer an interpretable meaning for how the embeddings could cause discrimination in downstream NLP applications. In this work, we present a transparent framework and metric for evaluating discrimination across protected groups with respect to their word embedding bias. Our metric (Relative Negative Sentiment Bias, RNSB) measures fairness in word embeddings via the relative negative sentiment associated with demographic identity terms from various protected groups. We show that our framework and metric enable useful analysis into the bias in word embeddings.

A Transparent Framework for Evaluating Unintended Demographic Bias in Word Embeddings.

This paper is concerned with automatic speech recognition (ASR) for accented speech. Given a small amount of speech from a new speaker, is it better to apply speaker adaptation to the baseline, or to use accent identification (AID) to identify the speaker’s accent and select an accent-dependent acoustic model? Three accent-based model selection methods are investigated: using the ‘true’ accent model, and unsupervised model selection using i-Vector and phonotactic-based AID. All three methods outperform the unadapted baseline. Most significantly, AID-based model selection using 43s of speech performs better than unsupervised speaker adaptation, even if the latter uses five times more adaptation data. Combining unsupervised AIDbased model selection and speaker adaptation gives an average relative reduction in ASR error rate of up to 47%. Index Terms: speech recognition, acoustic model selection, accent identification, i-Vector, phonotactics

/pdf/unsupervised-model-selection-for-recognition-of-regional-27ue2o3rfg.pdf

Unsupervised Model Selection for Recognition of Regional Accented Speech

The para-linguistic information in a speech signal includes clues to the geographical and social background of the speaker. This paper is concerned with recognition of the 14 regional accents of British English. For Accent Identification (AID), acoustic methods exploit differences between the distributions of sounds, while phonotactic approaches exploit the sequences in which these sounds occur. We demonstrate these methods are good complements for each other and use their confusion matrices for further analysis. Our relatively simple i-vector and phonotactic fused system with recognition accuracy of 84.87% outperforms the i-vector fused results reported in literature, by 4.7%. Further analysis on distribution of British English accents has been carried out by analyzing the low dimensional representation of i-vector AID feature space.

Identification of British English regional accents using fusion of i-vector and multi-accent phonotactic systems.

In this paper, we investigate different approaches for Dialect Identification (DID) in Arabic broadcast speech Dialects differ in their inventory of phonological segments This paper proposes a new phonotactic based feature representation approach which enables discrimination among different occurrences of the same phone n-grams with different phone duration and probability statistics To achieve further gain in accuracy we used multi-lingual phone recognizers, trained separately on Arabic, English, Czech, Hungarian and Russian languages We use Support Vector Machines (SVMs), and Convolutional Neural Networks (CNN s) as backend classifiers throughout the study The final system fusion results in 247% and 190% relative error rate reduction compared to that of a conventional phonotactic DID, and i-vectors with bottleneck features

Exploiting Convolutional Neural Networks for Phonotactic Based Dialect Identification

The use of word embedding models in sentiment analysis has gained a lot of traction in the Natural Language Processing (NLP) community. However, many inherently neutral word vectors describing demographic identity have unintended implicit correlations with negative or positive sentiment, resulting in unfair downstream machine learning algorithms. We leverage adversarial learning to decorrelate demographic identity term word vectors with positive or negative sentiment, and re-embed them into the word embeddings. We show that our method effectively minimizes unfair positive/negative sentiment polarity while retaining the semantic accuracy of the word embeddings. Furthermore, we show that our method effectively reduces unfairness in downstream sentiment regression and can be extended to reduce unfairness in toxicity classification tasks.

https://dl.acm.org/doi/pdf/10.1145/3351095.3372837

Maryam Najafian

Papers

A Transparent Framework for Evaluating Unintended Demographic Bias in Word Embeddings.

Unsupervised Model Selection for Recognition of Regional Accented Speech

Identification of British English regional accents using fusion of i-vector and multi-accent phonotactic systems.

Exploiting Convolutional Neural Networks for Phonotactic Based Dialect Identification

Reducing sentiment polarity for demographic attributes in word embeddings using adversarial learning