The paper “Gender Shades: Intersectional Accuracy Disparities in Commercial Gender Classification” by Joy Buolamwini and Timnit Gebru, that will be presented at the Conference on Fairness, Accountability, and Transparency (FAT*) in February 2018, evaluates three commercial API-based classifiers of gender from facial images, including IBM Watson Visual Recognition. The study finds these services to have recognition capabilities that are not balanced over genders and skin tones [1]. In particular, the authors show that the highest error involves images of dark-skinned women, while the most accurate result is for light-skinned men.

/pdf/gender-shades-intersectional-accuracy-disparities-in-4qgeu0c1i3.pdf

Gender Shades: Intersectional Accuracy Disparities in Commercial Gender Classification

As Natural Language Processing (NLP) and Machine Learning (ML) tools rise in popularity, it becomes increasingly vital to recognize the role they play in shaping societal biases and stereotypes. Although NLP models have shown success in modeling various applications, they propagate and may even amplify gender bias found in text corpora. While the study of bias in artificial intelligence is not new, methods to mitigate gender bias in NLP are relatively nascent. In this paper, we review contemporary studies on recognizing and mitigating gender bias in NLP. We discuss gender bias based on four forms of representation bias and analyze methods recognizing gender bias. Furthermore, we discuss the advantages and drawbacks of existing gender debiasing methods. Finally, we discuss future studies for recognizing and mitigating gender bias in NLP.

Mitigating Gender Bias in Natural Language Processing: Literature Review

In this paper, we study counterfactual fairness in text classification, which asks the question: How would the prediction change if the sensitive attribute referenced in the example were different? Toxicity classifiers demonstrate a counterfactual fairness issue by predicting that "Some people are gay" is toxic while "Some people are straight" is nontoxic. We offer a metric, counterfactual token fairness (CTF), for measuring this particular form of fairness in text classifiers, and describe its relationship with group fairness. Further, we offer three approaches, blindness, counterfactual augmentation, and counterfactual logit pairing (CLP), for optimizing counterfactual token fairness during training, bridging the robustness and fairness literature. Empirically, we find that blindness and CLP address counterfactual token fairness. The methods do not harm classifier performance, and have varying tradeoffs with group fairness. These approaches, both for measurement and optimization, provide a new path forward for addressing fairness concerns in text classification.

https://dl.acm.org/doi/pdf/10.1145/3306618.3317950

Counterfactual Fairness in Text Classification through Robustness

We investigate a new commonsense inference task: given an event described in a short free-form text (“X drinks coffee in the morning”), a system reasons about the likely intents (“X wants to stay awake”) and reactions (“X feels alert”) of the event’s participants. To support this study, we construct a new crowdsourced corpus of 25,000 event phrases covering a diverse range of everyday events and situations. We report baseline performance on this task, demonstrating that neural encoder-decoder models can successfully compose embedding representations of previously unseen events and reason about the likely intents and reactions of the event participants. In addition, we demonstrate how commonsense inference on people’s intents and reactions can help unveil the implicit gender inequality prevalent in modern movie scripts.

/pdf/event2mind-commonsense-inference-on-events-intents-and-aiqhbnfycw.pdf

Event2Mind: Commonsense Inference on Events, Intents, and Reactions

Individual fairness is an intuitive definition of algorithmic fairness that addresses some of the drawbacks of group fairness. Despite its benefits, it depends on a task specific fair metric that encodes our intuition of what is fair and unfair for the ML task at hand, and the lack of a widely accepted fair metric for many ML tasks is the main barrier to broader adoption of individual fairness. In this paper, we present two simple ways to learn fair metrics from a variety of data types. We show empirically that fair training with the learned metrics leads to improved fairness on three machine learning tasks susceptible to gender and racial biases. We also provide theoretical guarantees on the statistical performance of both approaches.

/pdf/two-simple-ways-to-learn-individual-fairness-metric-from-3c30s5gwc0.pdf

Two Simple Ways to Learn Individual Fairness Metric from Data

Machine Learning has seen tremendous growth recently, which has led to larger adoption of ML systems for educational assessments, credit risk, healthcare, employment, criminal justice, to name a few. The trustworthiness of ML and NLP systems is a crucial aspect and requires a guarantee that the decisions they make are fair and robust. Aligned with this, we propose a framework GYC, to generate a set of counterfactual text samples, which are crucial for testing these ML systems. Our main contributions include a) We introduce GYC, a framework to generate counterfactual samples such that the generation is plausible, diverse, goal-oriented, and effective, b) We generate counterfactual samples, that can direct the generation towards a corresponding condition such as named-entity tag, semantic role label, or sentiment. Our experimental results on various domains show that GYC generates counterfactual text samples exhibiting the above four properties. GYC generates counterfactuals that can act as test cases to evaluate a model and any text debiasing algorithm.

Generate Your Counterfactuals: Towards Controlled Counterfactual Generation for Text

The presence of gender stereotypes in many aspects of society is a well-known phenomenon. In this paper, we focus on studying such stereotypes and bias in Hindi movie industry (Bollywood) and propose an algorithm to remove these stereotypes from text. We analyze movie plots and posters for all movies released since 1970. The gender bias is detected by semantic modeling of plots at sentence and intra-sentence level. Different features like occupation, introductions, associated actions and descriptions are captured to show the pervasiveness of gender bias and stereotype in movies. Using the derived semantic graph, we compute centrality of each character and observe similar bias there. We also show that such bias is not applicable for movie posters where females get equal importance even though their character has little or no impact on the movie plot. The silver lining is that our system was able to identify 30 movies over last 3 years where such stereotypes were broken. The next step, is to generate debiased stories. The proposed debiasing algorithm extracts gender biased graphs from unstructured piece of text in stories from movies and de-bias these graphs to generate plausible unbiased stories.

/pdf/analyze-detect-and-remove-gender-stereotyping-from-bollywood-2j2p1g35av.pdf

Analyze, Detect and Remove Gender Stereotyping from Bollywood Movies

Machine Learning has seen tremendous growth recently, which has led to a larger adaptation of ML systems for educational assessments, credit risk, healthcare, employment, criminal justice, to name a few. The trustworthiness of ML and NLP systems is a crucial aspect and requires a guarantee that the decisions they make are fair and robust. Aligned with this, we propose a novel framework GYC, to generate a set of exhaustive counterfactual text, which are crucial for testing these ML systems. Our main contributions include a) We introduce GYC, a framework to generate counterfactual samples such that the generation is plausible, diverse, goal-oriented, and effective, b) We generate counterfactual samples, that can direct the generation towards a corresponding \texttt{condition} such as named-entity tag, semantic role label, or sentiment. Our experimental results on various domains show that GYC generates counterfactual text samples exhibiting the above four properties. GYC generates counterfactuals that can act as test cases to evaluate a model and any text debiasing algorithm.

Generate Your Counterfactuals: Towards Controlled Counterfactual Generation for Text.

Regulations are rules and directives published by authorities to safeguard consumer interest in an industry. Compliance with such regulations is getting increasingly hard due both to the complexity of these documents, which require experts to read, understand, and interpret them manually, and to the sheer volume of regulatory change. Many CFOs rate this as their top challenge. The authors’ Cogpliance platform uses a cognitive approach to achieve regulatory compliance. Here, they describe key compliance-related tasks and demonstrate how Cogpliance helps compliance officers to handle those tasks effectively.

Cognitive Compliance for Financial Regulations

Governmental authorities publish rules and directives that govern the operations of an industry. These documents, called regulations, are meant to safeguard the interests of consumers. With increasing number, size and complexity of such documents, companies face an uphill task to comply with them. We present a cognitive system, called Cogpliance, for exploring and understanding regulatory documents with the goal of assisting compliance officers in attaining regulatory compliance. Cogpliance automatically reads natural language regulatory documents, extracts key concepts and presents an interactive information exploration user interface for answering compliance officers queries.

Nishtha Madaan

Papers

Generate Your Counterfactuals: Towards Controlled Counterfactual Generation for Text

Analyze, Detect and Remove Gender Stereotyping from Bollywood Movies

Generate Your Counterfactuals: Towards Controlled Counterfactual Generation for Text.

Cognitive Compliance for Financial Regulations

Visual Exploration of Unstructured Regulatory Documents